binman: Simple templating feature and mkimage conversion
Subsystems (2)
cover binman: Simple templating feature and mkimage conversion
This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.
A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.
The templating implementation works by appending the template nodes to the target node. It is probably better to insert the template nodes before any subnodes in the target, so that the ordering of nodes in the template is preserved. But that involves rewriting the Fdt classs, since it can currently only add a subnode after the existing ones. This is left for later.
Marek Vasut (1):
binman: Convert mkimage to Entry_section
Simon Glass (11):
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Provide a way to specific the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
arch/arm/dts/stm32mp15-u-boot.dtsi | 1 + tools/binman/binman.rst | 87 ++++++++++++++++++ tools/binman/control.py | 9 ++ tools/binman/elf_test.py | 5 ++ tools/binman/entries.rst | 6 ++ tools/binman/entry.py | 10 +-- tools/binman/etype/blob_phase.py | 5 ++ tools/binman/etype/fit.py | 9 ++ tools/binman/etype/mkimage.py | 79 ++++++++++------- tools/binman/etype/section.py | 22 ++--- tools/binman/etype/u_boot_spl_bss_pad.py | 2 +- tools/binman/etype/u_boot_tpl_bss_pad.py | 2 +- tools/binman/etype/u_boot_vpl_bss_pad.py | 2 +- tools/binman/ftest.py | 103 +++++++++++++++++++++- tools/binman/test/282_symbols_disable.dts | 25 ++++++ tools/binman/test/283_mkimage_special.dts | 24 +++++ tools/binman/test/284_fit_fdt_list.dts | 58 ++++++++++++ tools/binman/test/285_spl_expand.dts | 13 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++ tools/binman/test/Makefile | 5 +- tools/binman/test/bss_data.c | 3 +- tools/binman/test/bss_data_zero.c | 16 ++++ tools/binman/test/bss_data_zero.lds | 15 ++++ tools/binman/test/embed_data.lds | 1 + tools/dtoc/fdt.py | 38 ++++++++ tools/dtoc/test/dtoc_test_simple.dts | 13 ++- tools/dtoc/test_fdt.py | 61 +++++++++++++ 27 files changed, 599 insertions(+), 57 deletions(-) create mode 100644 tools/binman/test/282_symbols_disable.dts create mode 100644 tools/binman/test/283_mkimage_special.dts create mode 100644 tools/binman/test/284_fit_fdt_list.dts create mode 100644 tools/binman/test/285_spl_expand.dts create mode 100644 tools/binman/test/286_entry_template.dts create mode 100644 tools/binman/test/bss_data_zero.c create mode 100644 tools/binman/test/bss_data_zero.lds
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 1/12 binman: Init align_default in entry_Section
This should be set up in the init function, to avoid a warning about a property not set up there. Fix it.
---
tools/binman/etype/section.py | 1 + 1 file changed, 1 insertion(+)
tools/binman/etype/section.py +1 -0
@@ -168,6 +168,7 @@ class Entry_section(Entry): self._end_4gb = False self._ignore_missing = False self._filename = None + self.align_default = 0 def IsSpecialSubnode(self, node): """Check if a node is a special one used by the section itself
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 2/12 binman: Use GetEntries() to obtain section contents
In reply to Simon GlassSome section types don't have a simple _entries list. Use the GetEntries() method in GetEntryContents() and other places to handle this.
This makes the behaviour more consistent.
---
tools/binman/etype/section.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tools/binman/etype/section.py +9 -9
@@ -720,7 +720,7 @@ class Entry_section(Entry): next_todo.append(entry) return entry - todo = self._entries.values() + todo = self.GetEntries().values() for passnum in range(3): threads = state.GetThreads() next_todo = []
@@ -893,7 +893,7 @@ class Entry_section(Entry): allow_missing: True if allowed, False if not allowed """ self.allow_missing = allow_missing - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.SetAllowMissing(allow_missing) def SetAllowFakeBlob(self, allow_fake):
@@ -903,7 +903,7 @@ class Entry_section(Entry): allow_fake: True if allowed, False if not allowed """ super().SetAllowFakeBlob(allow_fake) - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.SetAllowFakeBlob(allow_fake) def CheckMissing(self, missing_list):
@@ -915,7 +915,7 @@ class Entry_section(Entry): Args: missing_list: List of Entry objects to be added to """ - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckMissing(missing_list) def CheckFakedBlobs(self, faked_blobs_list):
@@ -926,7 +926,7 @@ class Entry_section(Entry): Args: faked_blobs_list: List of Entry objects to be added to """ - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckFakedBlobs(faked_blobs_list) def CheckOptional(self, optional_list):
@@ -937,7 +937,7 @@ class Entry_section(Entry): Args: optional_list (list): List of Entry objects to be added to """ - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckOptional(optional_list) def check_missing_bintools(self, missing_list):
@@ -949,7 +949,7 @@ class Entry_section(Entry): missing_list: List of Bintool objects to be added to """ super().check_missing_bintools(missing_list) - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.check_missing_bintools(missing_list) def _CollectEntries(self, entries, entries_by_name, add_entry):
@@ -999,12 +999,12 @@ class Entry_section(Entry): entry.Raise(f'Missing required properties/entry args: {missing}') def CheckAltFormats(self, alt_formats): - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckAltFormats(alt_formats) def AddBintools(self, btools): super().AddBintools(btools) - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.AddBintools(btools) def read_elf_segments(self):
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 3/12 binman: Read _multiple_data_files in the correct place
In reply to Simon GlassMove this to the ReadEntries() function where it belongs.
---
tools/binman/etype/mkimage.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/binman/etype/mkimage.py +2 -1
@@ -121,7 +121,6 @@ class Entry_mkimage(Entry): """ def __init__(self, section, etype, node): super().__init__(section, etype, node) - self._multiple_data_files = fdt_util.GetBool(self._node, 'multiple-data-files') self._mkimage_entries = OrderedDict() self._imagename = None self._filename = fdt_util.GetString(self._node, 'filename')
@@ -129,6 +128,8 @@ class Entry_mkimage(Entry): def ReadNode(self): super().ReadNode() + self._multiple_data_files = fdt_util.GetBool(self._node, + 'multiple-data-files') self._args = fdt_util.GetArgs(self._node, 'args') self._data_to_imagename = fdt_util.GetBool(self._node, 'data-to-imagename')
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 4/12 binman: Allow disabling symbol writing
In reply to Simon GlassSome boards don't use symbol writing but do access the symbols in SPL. Provide an option to work around this.
---
tools/binman/binman.rst | 7 ++++++ tools/binman/entry.py | 4 +++- tools/binman/etype/blob_phase.py | 5 ++++ tools/binman/ftest.py | 28 +++++++++++++++++++---- tools/binman/test/282_symbols_disable.dts | 25 ++++++++++++++++++++ 5 files changed, 64 insertions(+), 5 deletions(-) create mode 100644 tools/binman/test/282_symbols_disable.dts
tools/binman/binman.rst +7 -0
@@ -831,6 +831,13 @@ write-symbols: binman. This is automatic for certain entry types, e.g. `u-boot-spl`. See binman_syms_ for more information. +no-write-symbols: + Disables symbol writing for this entry. This can be used in entry types + where symbol writing is automatic. For example, if `u-boot-spl` refers to + the `u_boot_any_image_pos` symbol but U-Boot is not available in the image + containing SPL, this can be used to disable the writing. Quite likely this + indicates a bug in your setup. + elf-filename: Sets the file name of a blob's associated ELF file. For example, if the blob is `zephyr.bin` then the ELF file may be `zephyr.elf`. This allows
tools/binman/entry.py +3 -1
@@ -158,6 +158,7 @@ class Entry(object): self.offset_from_elf = None self.preserve = False self.build_done = False + self.no_write_symbols = False @staticmethod def FindEntryClass(etype, expanded):
@@ -321,6 +322,7 @@ class Entry(object): 'offset-from-elf') self.preserve = fdt_util.GetBool(self._node, 'preserve') + self.no_write_symbols = fdt_util.GetBool(self._node, 'no-write-symbols') def GetDefaultFilename(self): return None
@@ -695,7 +697,7 @@ class Entry(object): Args: section: Section containing the entry """ - if self.auto_write_symbols: + if self.auto_write_symbols and not self.no_write_symbols: # Check if we are writing symbols into an ELF file is_elf = self.GetDefaultFilename() == self.elf_fname elf.LookupAndWriteSymbols(self.elf_fname, self, section.GetImage(),
tools/binman/etype/blob_phase.py +5 -0
@@ -52,3 +52,8 @@ class Entry_blob_phase(Entry_section): # Read entries again, now that we have some self.ReadEntries() + + # Propagate the no-write-symbols property + if self.no_write_symbols: + for entry in self._entries.values(): + entry.no_write_symbols = True
tools/binman/ftest.py +24 -4
@@ -1452,7 +1452,7 @@ class TestFunctional(unittest.TestCase): self.assertEqual(U_BOOT_SPL_NODTB_DATA, data[:len(U_BOOT_SPL_NODTB_DATA)]) def checkSymbols(self, dts, base_data, u_boot_offset, entry_args=None, - use_expanded=False): + use_expanded=False, no_write_symbols=False): """Check the image contains the expected symbol values Args:
@@ -1481,9 +1481,14 @@ class TestFunctional(unittest.TestCase): sym_values = struct.pack('<LLQLL', elf.BINMAN_SYM_MAGIC_VALUE, 0x00, u_boot_offset + len(U_BOOT_DATA), 0x10 + u_boot_offset, 0x04) - expected = (sym_values + base_data[24:] + - tools.get_bytes(0xff, 1) + U_BOOT_DATA + sym_values + - base_data[24:]) + if no_write_symbols: + expected = (base_data + + tools.get_bytes(0xff, 0x38 - len(base_data)) + + U_BOOT_DATA + base_data) + else: + expected = (sym_values + base_data[24:] + + tools.get_bytes(0xff, 1) + U_BOOT_DATA + sym_values + + base_data[24:]) self.assertEqual(expected, data) def testSymbols(self):
@@ -6676,6 +6681,21 @@ fdt fdtmap Extract the devicetree blob from the fdtmap ['fit']) self.assertIn("Node '/fit': Missing tool: 'mkimage'", str(e.exception)) + def testSymbolNoWrite(self): + """Test disabling of symbol writing""" + self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_DATA, 0x1c, + no_write_symbols=True) + + def testSymbolNoWriteExpanded(self): + """Test disabling of symbol writing in expanded entries""" + entry_args = { + 'spl-dtb': '1', + } + self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_NODTB_DATA + + U_BOOT_SPL_DTB_DATA, 0x38, + entry_args=entry_args, use_expanded=True, + no_write_symbols=True) + if __name__ == "__main__": unittest.main()
tools/binman/test/282_symbols_disable.dts new file +25 -0
@@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + pad-byte = <0xff>; + u-boot-spl { + no-write-symbols; + }; + + u-boot { + offset = <0x38>; + no-expanded; + }; + + u-boot-spl2 { + type = "u-boot-spl"; + no-write-symbols; + }; + }; +};
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 5/12 stm32mp15: Avoid writing symbols in SPL
In reply to Simon GlassThese boards use SPL in a mkimage entry and apparently access the symbol containing the image position of U-Boot, but put U-Boot in another image. This means that binman is unable to fill in the symbol correctly in the SPL binary.
This doesn't matter at present since mkimage doesn't support symbol writing. But with the upcoming conversion to a section, it will. So add a property to disable symbol writing.
---
arch/arm/dts/stm32mp15-u-boot.dtsi | 1 + 1 file changed, 1 insertion(+)
arch/arm/dts/stm32mp15-u-boot.dtsi +1 -0
@@ -226,6 +226,7 @@ mkimage { args = "-T stm32image -a 0x2ffc2500 -e 0x2ffc2500"; u-boot-spl { + no-write-symbols; }; }; };
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 6/12 binman: Convert mkimage to Entry_section
In reply to Simon GlassFrom: Marek Vasut <[email protected]>
This is needed to handle mkimage with inner section located itself in a section.
Use BuildSectionData() instead of ObtainContents(), add tests and a few other minor fixes:
---
tools/binman/entry.py | 6 +- tools/binman/etype/mkimage.py | 76 ++++++++++++++--------- tools/binman/ftest.py | 45 +++++++++++++- tools/binman/test/283_mkimage_special.dts | 24 +++++++ 4 files changed, 117 insertions(+), 34 deletions(-) create mode 100644 tools/binman/test/283_mkimage_special.dts
tools/binman/entry.py +2 -4
@@ -1311,10 +1311,8 @@ features to produce new behaviours. """ data = b'' for entry in entries: - # First get the input data and put it in a file. If not available, - # try later. - if not entry.ObtainContents(fake_size=fake_size): - return None, None, None + # First get the input data and put it in a file + entry.ObtainContents(fake_size=fake_size) data += entry.GetData() uniq = self.GetUniqueName() fname = tools.get_output_filename(f'{prefix}.{uniq}')
tools/binman/etype/mkimage.py +47 -29
164 lines not on this page · raw message
tools/binman/ftest.py +44 -1
74 lines not on this page · raw message
tools/binman/test/283_mkimage_special.dts new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + mkimage { + args = "-T script"; + + u-boot { + }; + + hash { + }; + + imagename { + type = "u-boot"; + }; + }; + }; +};
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
Re: [PATCH 06/12] binman: Convert mkimage to Entry_section
Hi Simon,
Quoted 171 lines · Simon Glass
From: Marek Vasut <[email protected]>
This is needed to handle mkimage with inner section located itself in a section.
Signed-off-by Marek Vasut <[email protected]>Use
BuildSectionData()instead ofObtainContents(), add tests and a few other minor fixes:Signed-off-by Simon Glass <[email protected]>---
tools/binman/entry.py | 6 +- tools/binman/etype/mkimage.py | 76 ++++++++++++++--------- tools/binman/ftest.py | 45 +++++++++++++- tools/binman/test/283_mkimage_special.dts | 24 +++++++ 4 files changed, 117 insertions(+), 34 deletions(-) create mode 100644 tools/binman/test/283_mkimage_special.dtstools/binman/entry.py +2
-4@@ -1311,10 +1311,8 @@ features to produce new behaviours. """ data = b'' for entry in entries: - # First get the input data and put it in a file. If not available, - # try later. - if not entry.ObtainContents(fake_size=fake_size): - return None, None, None + # First get the input data and put it in a file + entry.ObtainContents(fake_size=fake_size) data += entry.GetData() uniq = self.GetUniqueName() fname = tools.get_output_filename(f'{prefix}.{uniq}')tools/binman/etype/mkimage.py +40
-25@@ -8,10 +8,11 @@ from collections import OrderedDict from binman.entry import Entry +from binman.etype.section import Entry_section from dtoc import fdt_util from u_boot_pylib import tools -class Entry_mkimage(Entry): +class Entry_mkimage(Entry_section): """Binary produced by mkimage Properties / Entry arguments:@@ -121,10 +122,8 @@ class Entry_mkimage(Entry): """ def __init__(self, section, etype, node): super().__init__(section, etype, node) - self._mkimage_entries = OrderedDict() self._imagename = None - self._filename = fdt_util.GetString(self._node, 'filename') - self.align_default = None + self._multiple_data_files = False def ReadNode(self): super().ReadNode()@@ -135,41 +134,60 @@ class Entry_mkimage(Entry): 'data-to-imagename') if self._data_to_imagename and self._node.FindNode('imagename'): self.Raise('Cannot use both imagename node and data-to-imagename') - self.ReadEntries() def ReadEntries(self): """Read the subnodes to find out what should go in this image""" for node in self._node.subnodes: - entry = Entry.Create(self, node) + if self.IsSpecialSubnode(node): + continue + entry = Entry.Create(self, node, + expanded=self.GetImage().use_expanded, + missing_etype=self.GetImage().missing_etype) entry.ReadNode() + entry.SetPrefix(self._name_prefix) if entry.name == 'imagename': self._imagename = entry else: - self._mkimage_entries[entry.name] = entry + self._entries[entry.name] = entry - def ObtainContents(self): + def BuildSectionData(self, required): + """Build mkimage entry contents + + Runs mkimage to build the entry contents + + Args: + required (bool): True if the data must be present, False if it is OK + to return None + + Returns: + bytes: Contents of the section + """ # Use a non-zero size for any fake files to keep mkimage happy # Note that testMkimageImagename() relies on this 'mkimage' parameter fake_size = 1024 if self._multiple_data_files: fnames = [] uniq = self.GetUniqueName() - for entry in self._mkimage_entries.values(): - if not entry.ObtainContents(fake_size=fake_size): - return False - if entry._pathname: - fnames.append(entry._pathname) + for entry in self._entries.values(): + entry.ObtainContents(fake_size=fake_size) + + # If this is a section, put the contents in a temporary file. + # Otherwise, assume it is a blob and use the pathname + if isinstance(entry, Entry_section): + ename = f'mkimage-in-{uniq}-{entry.name}' + fname = tools.get_output_filename(ename) + tools.write_file(fname, entry.data) + elif entry._pathname: + fname = entry._pathname + fnames.append(fname) input_fname = ":".join(fnames) + data = b'' else: data, input_fname, uniq = self.collect_contents_to_file( - self._mkimage_entries.values(), 'mkimage', fake_size) - if data is None: - return False + self._entries.values(), 'mkimage', fake_size) if self._imagename: image_data, imagename_fname, _ = self.collect_contents_to_file( [self._imagename], 'mkimage-n', 1024) - if image_data is None: - return False outfile = self._filename if self._filename else 'mkimage-out.%s' % uniq output_fname = tools.get_output_filename(outfile)@@ -177,8 +195,7 @@ class Entry_mkimage(Entry): self.CheckMissing(missing_list) self.missing = bool(missing_list) if self.missing: - self.SetContents(b'') - return self.allow_missing + return b'' args = ['-d', input_fname] if self._data_to_imagename:@@ -187,17 +204,15 @@ class Entry_mkimage(Entry): args += ['-n', imagename_fname] args += self._args + [output_fname] if self.mkimage.run_cmd(*args) is not None: - self.SetContents(tools.read_file(output_fname)) + return tools.read_file(output_fname) else: # Bintool is missing; just use the input data as the output self.record_missing_bintool(self.mkimage) - self.SetContents(data) - - return True + return data def GetEntries(self): # Make a copy so we don't change the original - entries = OrderedDict(self._mkimage_entries) + entries = OrderedDict(self._entries) if self._imagename: entries['imagename'] = self._imagename return entries
tools/binman/etype/mkimage.py class Entry_mkimage(Entry): · line 227
Quoted 5 lines
tools/binman/etype/mkimage.py +1
-1@@ -209,7 +224,7 @@ class Entry_mkimage(Entry): allow_missing: True if allowed, False if not allowed """ self.allow_missing = allow_missing - for entry in self._mkimage_entries.values(): + for entry in self._entries.values():
With the changes to use GetEntries() in Entry_section for this and following Check and Set functions, I suspect these can be removed and use the base implementation from Entry_section.
GetEntries() returns self._entries + self._imagename
Regards,
Jonas
Quoted 150 lines
tools/binman/etype/mkimage.py +6
-3entry.SetAllowMissing(allow_missing) if self._imagename: self._imagename.SetAllowMissing(allow_missing)@@ -220,7 +235,7 @@ class Entry_mkimage(Entry): Args: allow_fake: True if allowed, False if not allowed """ - for entry in self._mkimage_entries.values(): + for entry in self._entries.values(): entry.SetAllowFakeBlob(allow_fake) if self._imagename: self._imagename.SetAllowFakeBlob(allow_fake)@@ -234,7 +249,7 @@ class Entry_mkimage(Entry): Args: missing_list: List of Entry objects to be added to """ - for entry in self._mkimage_entries.values(): + for entry in self._entries.values(): entry.CheckMissing(missing_list) if self._imagename: self._imagename.CheckMissing(missing_list)@@ -247,7 +262,7 @@ class Entry_mkimage(Entry): Args: faked_blobs_list: List of Entry objects to be added to """ - for entry in self._mkimage_entries.values(): + for entry in self._entries.values(): entry.CheckFakedBlobs(faked_blobs_list) if self._imagename: self._imagename.CheckFakedBlobs(faked_blobs_list)@@ -255,3 +270,6 @@ class Entry_mkimage(Entry): def AddBintools(self, btools): super().AddBintools(btools) self.mkimage = self.AddBintool(btools, 'mkimage') + + def CheckEntries(self): + passtools/binman/ftest.py +44
-1@@ -3737,6 +3737,7 @@ class TestFunctional(unittest.TestCase): def testMkimage(self): """Test using mkimage to build an image""" + self._SetupSplElf() data = self._DoReadFile('156_mkimage.dts') # Just check that the data appears in the file somewhere@@ -3744,6 +3745,7 @@ class TestFunctional(unittest.TestCase): def testMkimageMissing(self): """Test that binman still produces an image if mkimage is missing""" + self._SetupSplElf() with test_util.capture_sys_output() as (_, stderr): self._DoTestFile('156_mkimage.dts', force_missing_bintools='mkimage')@@ -5952,6 +5954,7 @@ fdt fdtmap Extract the devicetree blob from the fdtmap def testMkimageCollection(self): """Test using a collection referring to an entry in a mkimage entry""" + self._SetupSplElf() data = self._DoReadFile('247_mkimage_coll.dts') expect = U_BOOT_SPL_DATA + U_BOOT_DATA self.assertEqual(expect, data[:len(expect)])@@ -6051,6 +6054,39 @@ fdt fdtmap Extract the devicetree blob from the fdtmap expect += U_BOOT_SPL_DATA self.assertEqual(expect, data[-len(expect):]) + def testMkimageMultipleExpanded(self): + """Test passing multiple files to mkimage in a mkimage entry""" + self._SetupSplElf() + self._SetupTplElf() + entry_args = { + 'spl-bss-pad': 'y', + 'spl-dtb': 'y', + } + data = self._DoReadFileDtb('252_mkimage_mult_data.dts', + use_expanded=True, entry_args=entry_args)[0] + pad_len = 10 + tpl_expect = U_BOOT_TPL_DATA + spl_expect = U_BOOT_SPL_NODTB_DATA + tools.get_bytes(0, pad_len) + spl_expect += U_BOOT_SPL_DTB_DATA + + content = data[0x40:] + lens = struct.unpack('>III', content[:12]) + + # Size of files are packed in their 4B big-endian format + # Size info is always followed by a 4B zero value. + self.assertEqual(len(tpl_expect), lens[0]) + self.assertEqual(len(spl_expect), lens[1]) + self.assertEqual(0, lens[2]) + + rest = content[12:] + self.assertEqual(tpl_expect, rest[:len(tpl_expect)]) + + rest = rest[len(tpl_expect):] + align_pad = len(tpl_expect) % 4 + self.assertEqual(tools.get_bytes(0, align_pad), rest[:align_pad]) + rest = rest[align_pad:] + self.assertEqual(spl_expect, rest) + def testMkimageMultipleNoContent(self): """Test passing multiple data files to mkimage with one data file having no content""" with self.assertRaises(ValueError) as exc:@@ -6696,6 +6732,13 @@ fdt fdtmap Extract the devicetree blob from the fdtmap entry_args=entry_args, use_expanded=True, no_write_symbols=True) + def testMkimageSpecial(self): + """Test mkimage ignores special hash-1 node""" + data = self._DoReadFile('283_mkimage_special.dts') + + # Just check that the data appears in the file somewhere + self.assertIn(U_BOOT_DATA, data) + -if __name__ == "__main__": +if __name__ == "_s_main__": unittest.main()tools/binman/test/283_mkimage_special.dts new file +24
-0@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + mkimage { + args = "-T script"; + + u-boot { + }; + + hash { + }; + + imagename { + type = "u-boot"; + }; + }; + }; +};
Re: [PATCH 06/12] binman: Convert mkimage to Entry_section
Hi Jonas,
Quoted 184 lines · Jonas Karlman
Hi Simon,
Quoted 171 lines · Simon Glass
From: Marek Vasut <[email protected]>
This is needed to handle mkimage with inner section located itself in a section.
Signed-off-by Marek Vasut <[email protected]>Use
BuildSectionData()instead ofObtainContents(), add tests and a few other minor fixes:Signed-off-by Simon Glass <[email protected]>---
tools/binman/entry.py | 6 +- tools/binman/etype/mkimage.py | 76 ++++++++++++++--------- tools/binman/ftest.py | 45 +++++++++++++- tools/binman/test/283_mkimage_special.dts | 24 +++++++ 4 files changed, 117 insertions(+), 34 deletions(-) create mode 100644 tools/binman/test/283_mkimage_special.dtstools/binman/entry.py +2
-4@@ -1311,10 +1311,8 @@ features to produce new behaviours. """ data = b'' for entry in entries: - # First get the input data and put it in a file. If not available, - # try later. - if not entry.ObtainContents(fake_size=fake_size): - return None, None, None + # First get the input data and put it in a file + entry.ObtainContents(fake_size=fake_size) data += entry.GetData() uniq = self.GetUniqueName() fname = tools.get_output_filename(f'{prefix}.{uniq}')tools/binman/etype/mkimage.py +40
-25@@ -8,10 +8,11 @@ from collections import OrderedDict from binman.entry import Entry +from binman.etype.section import Entry_section from dtoc import fdt_util from u_boot_pylib import tools -class Entry_mkimage(Entry): +class Entry_mkimage(Entry_section): """Binary produced by mkimage Properties / Entry arguments:@@ -121,10 +122,8 @@ class Entry_mkimage(Entry): """ def __init__(self, section, etype, node): super().__init__(section, etype, node) - self._mkimage_entries = OrderedDict() self._imagename = None - self._filename = fdt_util.GetString(self._node, 'filename') - self.align_default = None + self._multiple_data_files = False def ReadNode(self): super().ReadNode()@@ -135,41 +134,60 @@ class Entry_mkimage(Entry): 'data-to-imagename') if self._data_to_imagename and self._node.FindNode('imagename'): self.Raise('Cannot use both imagename node and data-to-imagename') - self.ReadEntries() def ReadEntries(self): """Read the subnodes to find out what should go in this image""" for node in self._node.subnodes: - entry = Entry.Create(self, node) + if self.IsSpecialSubnode(node): + continue + entry = Entry.Create(self, node, + expanded=self.GetImage().use_expanded, + missing_etype=self.GetImage().missing_etype) entry.ReadNode() + entry.SetPrefix(self._name_prefix) if entry.name == 'imagename': self._imagename = entry else: - self._mkimage_entries[entry.name] = entry + self._entries[entry.name] = entry - def ObtainContents(self): + def BuildSectionData(self, required): + """Build mkimage entry contents + + Runs mkimage to build the entry contents + + Args: + required (bool): True if the data must be present, False if it is OK + to return None + + Returns: + bytes: Contents of the section + """ # Use a non-zero size for any fake files to keep mkimage happy # Note that testMkimageImagename() relies on this 'mkimage' parameter fake_size = 1024 if self._multiple_data_files: fnames = [] uniq = self.GetUniqueName() - for entry in self._mkimage_entries.values(): - if not entry.ObtainContents(fake_size=fake_size): - return False - if entry._pathname: - fnames.append(entry._pathname) + for entry in self._entries.values(): + entry.ObtainContents(fake_size=fake_size) + + # If this is a section, put the contents in a temporary file. + # Otherwise, assume it is a blob and use the pathname + if isinstance(entry, Entry_section): + ename = f'mkimage-in-{uniq}-{entry.name}' + fname = tools.get_output_filename(ename) + tools.write_file(fname, entry.data) + elif entry._pathname: + fname = entry._pathname + fnames.append(fname) input_fname = ":".join(fnames) + data = b'' else: data, input_fname, uniq = self.collect_contents_to_file( - self._mkimage_entries.values(), 'mkimage', fake_size) - if data is None: - return False + self._entries.values(), 'mkimage', fake_size) if self._imagename: image_data, imagename_fname, _ = self.collect_contents_to_file( [self._imagename], 'mkimage-n', 1024) - if image_data is None: - return False outfile = self._filename if self._filename else 'mkimage-out.%s' % uniq output_fname = tools.get_output_filename(outfile)@@ -177,8 +195,7 @@ class Entry_mkimage(Entry): self.CheckMissing(missing_list) self.missing = bool(missing_list) if self.missing: - self.SetContents(b'') - return self.allow_missing + return b'' args = ['-d', input_fname] if self._data_to_imagename:@@ -187,17 +204,15 @@ class Entry_mkimage(Entry): args += ['-n', imagename_fname] args += self._args + [output_fname] if self.mkimage.run_cmd(*args) is not None: - self.SetContents(tools.read_file(output_fname)) + return tools.read_file(output_fname) else: # Bintool is missing; just use the input data as the output self.record_missing_bintool(self.mkimage) - self.SetContents(data) - - return True + return data def GetEntries(self): # Make a copy so we don't change the original - entries = OrderedDict(self._mkimage_entries) + entries = OrderedDict(self._entries) if self._imagename: entries['imagename'] = self._imagename return entriestools/binman/etype/mkimage.py class Entry_mkimage(Entry): · line 227
Quoted 5 lines
tools/binman/etype/mkimage.py +1
-1@@ -209,7 +224,7 @@ class Entry_mkimage(Entry): allow_missing: True if allowed, False if not allowed """ self.allow_missing = allow_missing - for entry in self._mkimage_entries.values(): + for entry in self._entries.values():With the changes to use
GetEntries()inEntry_sectionfor this and following Check and Set functions, I suspect these can be removed and use the base implementation fromEntry_section.
Quoted 1 line
GetEntries()returns self._entries + self._imagename
Yes, good idea, will do in v2. Tht special code has annoyed me.
Regards,
Simon
PATCH 7/12 binman: Provide a way to specific the fdt-list directly
In reply to Simon GlassSometimes multiple boards are built with binman and it is useful to specify a different FDT list for each. At present this is not possible without providing multiple values of the of-list entryarg (which is not supported in the U-Boot build system).
Allow a fit,fdt-list-val string-list property to be used instead.
---
tools/binman/entries.rst | 6 +++ tools/binman/etype/fit.py | 9 ++++ tools/binman/ftest.py | 14 ++++++- tools/binman/test/284_fit_fdt_list.dts | 58 ++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/284_fit_fdt_list.dts
tools/binman/entries.rst +6 -0
@@ -615,6 +615,12 @@ The top-level 'fit' node supports the following special properties: `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed to binman. + fit,fdt-list-val + As an alternative to fit,fdt-list the list of device tree files + can be provided in this property as a string list, e.g.:: + + fit,fdt-list-val = "dtb1", "dtb2"; + Substitutions ~~~~~~~~~~~~~
tools/binman/etype/fit.py +9 -0
@@ -81,6 +81,12 @@ class Entry_fit(Entry_section): `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed to binman. + fit,fdt-list-val + As an alternative to fit,fdt-list the list of device tree files + can be provided in this property as a string list, e.g.:: + + fit,fdt-list-val = "dtb1", "dtb2"; + Substitutions ~~~~~~~~~~~~~
@@ -361,6 +367,9 @@ class Entry_fit(Entry_section): [EntryArg(self._fit_list_prop.value, str)]) if fdts is not None: self._fdts = fdts.split() + else: + self._fdts = fdt_util.GetStringList(self._node, 'fit,fdt-list-val') + self._fit_default_dt = self.GetEntryArgsOrProps([EntryArg('default-dt', str)])[0]
tools/binman/ftest.py +13 -1
@@ -6739,6 +6739,18 @@ fdt fdtmap Extract the devicetree blob from the fdtmap # Just check that the data appears in the file somewhere self.assertIn(U_BOOT_DATA, data) + def testFitFdtList(self): + """Test an image with an FIT with the fit,fdt-list-val option""" + entry_args = { + 'default-dt': 'test-fdt2', + } + data = self._DoReadFileDtb( + '284_fit_fdt_list.dts', + entry_args=entry_args, + extra_indirs=[os.path.join(self._indir, TEST_FDT_SUBDIR)])[0] + self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):]) + fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)] + -if __name__ == "_s_main__": +if __name__ == "__main__": unittest.main()
tools/binman/test/284_fit_fdt_list.dts new file +58 -0
58 lines not on this page · raw message
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
Re: [PATCH 07/12] binman: Provide a way to specific the fdt-list directly
Quoted 139 lines · Simon Glass
Sometimes multiple boards are built with binman and it is useful to specify a different FDT list for each. At present this is not possible without providing multiple values of the of-list entryarg (which is not supported in the U-Boot build system).
Allow a fit,fdt-list-val string-list property to be used instead.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/entries.rst | 6 +++ tools/binman/etype/fit.py | 9 ++++ tools/binman/ftest.py | 14 ++++++- tools/binman/test/284_fit_fdt_list.dts | 58 ++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/284_fit_fdt_list.dtstools/binman/entries.rst +6
-0@@ -615,6 +615,12 @@ The top-level 'fit' node supports the following special properties: `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed to binman. + fit,fdt-list-val + As an alternative to fit,fdt-list the list of device tree files + can be provided in this property as a string list, e.g.:: + + fit,fdt-list-val = "dtb1", "dtb2"; + Substitutions ~~~~~~~~~~~~~tools/binman/etype/fit.py +9
-0@@ -81,6 +81,12 @@ class Entry_fit(Entry_section): `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed to binman. + fit,fdt-list-val + As an alternative to fit,fdt-list the list of device tree files + can be provided in this property as a string list, e.g.:: + + fit,fdt-list-val = "dtb1", "dtb2"; + Substitutions ~~~~~~~~~~~~~@@ -361,6 +367,9 @@ class Entry_fit(Entry_section): [EntryArg(self._fit_list_prop.value, str)]) if fdts is not None: self._fdts = fdts.split() + else: + self._fdts = fdt_util.GetStringList(self._node, 'fit,fdt-list-val') + self._fit_default_dt = self.GetEntryArgsOrProps([EntryArg('default-dt', str)])[0]tools/binman/ftest.py +13
-1@@ -6739,6 +6739,18 @@ fdt fdtmap Extract the devicetree blob from the fdtmap # Just check that the data appears in the file somewhere self.assertIn(U_BOOT_DATA, data) + def testFitFdtList(self): + """Test an image with an FIT with the fit,fdt-list-val option""" + entry_args = { + 'default-dt': 'test-fdt2', + } + data = self._DoReadFileDtb( + '284_fit_fdt_list.dts', + entry_args=entry_args, + extra_indirs=[os.path.join(self._indir, TEST_FDT_SUBDIR)])[0] + self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):]) + fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)] + -if __name__ == "_s_main__": +if __name__ == "__main__": unittest.main()tools/binman/test/284_fit_fdt_list.dts new file +48
-0@@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot { + }; + fit { + description = "test-desc"; + #address-cells = <1>; + fit,fdt-list-val = "test-fdt1", "test-fdt2"; + + images { + kernel { + description = "Vanilla Linux kernel"; + type = "kernel"; + arch = "ppc"; + os = "linux"; + compression = "gzip"; + load = <00000000>; + entry = <00000000>; + hash-1 { + algo = "crc32"; + }; + hash-2 { + algo = "sha1"; + }; + u-boot { + }; + }; + @fdt-SEQ { + description = "fdt-NAME.dtb"; + type = "flat_dt"; + compression = "none"; + hash { + algo = "sha256"; + }; + }; + }; + + configurations { + default = "@config-DEFAULT-SEQ"; + @config-SEQ { + description = "conf-NAME.dtb";
tools/binman/test/284_fit_fdt_list.dts line 58
Quoted 10 lines
tools/binman/test/284_fit_fdt_list.dts new file +10
-0+ firmware = "uboot"; + loadables = "atf"; + fdt = "fdt-SEQ"; + }; + }; + }; + u-boot-nodtb { + }; + }; +};
In contrast, this already works nicely and obsoletes our need to patch the binman command line for the IOT2050.
Jan
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
PATCH 8/12 binman: Drop __bss_size variable in bss_data.c
In reply to Simon GlassThis is not needed since the linker script sets it up. Drop the variable to avoid confusion.
Fix the prototype for main() while we are here.
---
tools/binman/test/bss_data.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
tools/binman/test/bss_data.c +1 -2
@@ -7,9 +7,8 @@*/ int bss_data[10 ]; -int __bss_size =sizeof (bss_data); -int main () +int main (void ) { bss_data[2 ] =2 ;
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 9/12 binman: Correct handling of zero bss size
In reply to Simon GlassFix the check for the __bss_size symbol, since it may be 0. Unfortunately there was no test coverage for this.
---
tools/binman/elf_test.py | 5 +++++ tools/binman/etype/u_boot_spl_bss_pad.py | 2 +- tools/binman/etype/u_boot_tpl_bss_pad.py | 2 +- tools/binman/etype/u_boot_vpl_bss_pad.py | 2 +- tools/binman/ftest.py | 12 ++++++++++++ tools/binman/test/285_spl_expand.dts | 13 +++++++++++++ tools/binman/test/Makefile | 5 ++++- tools/binman/test/bss_data_zero.c | 16 ++++++++++++++++ tools/binman/test/bss_data_zero.lds | 15 +++++++++++++++ tools/binman/test/embed_data.lds | 1 + 10 files changed, 69 insertions(+), 4 deletions(-) create mode 100644 tools/binman/test/285_spl_expand.dts create mode 100644 tools/binman/test/bss_data_zero.c create mode 100644 tools/binman/test/bss_data_zero.lds
tools/binman/elf_test.py +5 -0
@@ -369,6 +369,11 @@ class TestElf(unittest.TestCase): elf.GetSymbolOffset(fname, 'embed') self.assertIn('__image_copy_start', str(e.exception)) + def test_get_symbol_address(self): + fname = self.ElfTestFile('embed_data') + addr = elf.GetSymbolAddress(fname, 'region_size') + self.assertEqual(0, addr) + if __name__ == '__main__': unittest.main()
tools/binman/etype/u_boot_spl_bss_pad.py +1 -1
@@ -38,7 +38,7 @@ class Entry_u_boot_spl_bss_pad(Entry_blob): def ObtainContents(self): fname = tools.get_input_filename('spl/u-boot-spl') bss_size = elf.GetSymbolAddress(fname, '__bss_size') - if not bss_size: + if bss_size is None: self.Raise('Expected __bss_size symbol in spl/u-boot-spl') self.SetContents(tools.get_bytes(0, bss_size)) return True
tools/binman/etype/u_boot_tpl_bss_pad.py +1 -1
@@ -38,7 +38,7 @@ class Entry_u_boot_tpl_bss_pad(Entry_blob): def ObtainContents(self): fname = tools.get_input_filename('tpl/u-boot-tpl') bss_size = elf.GetSymbolAddress(fname, '__bss_size') - if not bss_size: + if bss_size is None: self.Raise('Expected __bss_size symbol in tpl/u-boot-tpl') self.SetContents(tools.get_bytes(0, bss_size)) return True
tools/binman/etype/u_boot_vpl_bss_pad.py +1 -1
@@ -38,7 +38,7 @@ class Entry_u_boot_vpl_bss_pad(Entry_blob): def ObtainContents(self): fname = tools.get_input_filename('vpl/u-boot-vpl') bss_size = elf.GetSymbolAddress(fname, '__bss_size') - if not bss_size: + if bss_size is None: self.Raise('Expected __bss_size symbol in vpl/u-boot-vpl') self.SetContents(tools.get_bytes(0, bss_size)) return True
tools/binman/ftest.py +12 -0
@@ -6751,6 +6751,18 @@ fdt fdtmap Extract the devicetree blob from the fdtmap self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):]) fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)] + def testSplEmptyBss(self): + """Test an expanded SPL with a zero-size BSS""" + # ELF file with a '__bss_size' symbol + self._SetupSplElf(src_fname='bss_data_zero') + + entry_args = { + 'spl-bss-pad': 'y', + 'spl-dtb': 'y', + } + data = self._DoReadFileDtb('285_spl_expand.dts', + use_expanded=True, entry_args=entry_args)[0] + if __name__ == "__main__": unittest.main()
tools/binman/test/285_spl_expand.dts new file +13 -0
@@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-spl { + }; + }; +};
tools/binman/test/Makefile +4 -1
@@ -32,7 +32,7 @@ LDS_BINMAN_EMBED := -T $(SRC)u_boot_binman_embed.lds LDS_EFL_SECTIONS := -T $(SRC)elf_sections.lds LDS_BLOB := -T $(SRC)blob_syms.lds -TARGETS = u_boot_ucode_ptr u_boot_no_ucode_ptr bss_data \ +TARGETS = u_boot_ucode_ptr u_boot_no_ucode_ptr bss_data bss_data_zero \ u_boot_binman_syms u_boot_binman_syms.bin u_boot_binman_syms_bad \ u_boot_binman_syms_size u_boot_binman_syms_x86 embed_data \ u_boot_binman_embed u_boot_binman_embed_sm elf_sections blob_syms.bin
@@ -48,6 +48,9 @@ u_boot_ucode_ptr: u_boot_ucode_ptr.c bss_data: CFLAGS += $(SRC)bss_data.lds bss_data: bss_data.c +bss_data_zero: CFLAGS += $(SRC)bss_data_zero.lds +bss_data_zero: bss_data_zero.c + embed_data: CFLAGS += $(SRC)embed_data.lds embed_data: embed_data.c
tools/binman/test/bss_data_zero.c new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* +* Copyright (c) 2016 Google, Inc +* +* Simple program to create a bss_data region so the symbol can be read +* by binutils. This is used by binman tests. +*/ + +int bss_data[10 ]; + +int main (void ) +{ + bss_data[2 ] =2 ; + +return 0 ; +}
tools/binman/test/bss_data_zero.lds new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright (c) 2016 Google, Inc + */ + +OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386") +OUTPUT_ARCH(i386) +ENTRY(_start) + +SECTIONS +{ + . = 0xfffffdf0; + _start = .; + __bss_size = 0; +}
tools/binman/test/embed_data.lds +1 -0
@@ -17,6 +17,7 @@ SECTIONS embed_start = .; *(.embed*) embed_end = .; + region_size = 0; . = ALIGN(32); *(.data*) }
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 10/12 dtoc: Support copying the contents of a node into another
In reply to Simon GlassThis permits implementation of a simple templating system, where a node can be reused as a base for others.
For now this adds new subnodes after any existing ones.
---
tools/dtoc/fdt.py | 22 ++++++++++++++ tools/dtoc/test/dtoc_test_simple.dts | 3 ++ tools/dtoc/test_fdt.py | 43 ++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+)
tools/dtoc/fdt.py +22 -0
@@ -13,6 +13,7 @@ from dtoc import fdt_util import libfdt from libfdt import QUIET_NOTFOUND from u_boot_pylib import tools +from u_boot_pylib import tout # This deals with a device tree, presenting it as an assortment of Node and # Prop objects, representing nodes and properties, respectively. This file
@@ -635,6 +636,27 @@ class Node: prop.Sync(auto_resize) return added + def copy_node(self, src): + """Copy a node and all its subnodes into this node + + Args: + src (Node): Node to copy + + This works recursively. + + The new node is put after all other nodes. If the node already + exists, just its properties are copied. Properties which exist in the + destination node already are not copied. + """ + dst = self.FindNode(src.name) + if not dst: + dst = self.AddSubnode(src.name) + for name, src_prop in src.props.items(): + if name not in dst.props: + dst.props[name] = Prop(dst, None, name, src_prop.bytes) + for node in src.subnodes: + dst.copy_node(node) + class Fdt: """Provides simple access to a flat device tree blob using libfdts.
tools/dtoc/test/dtoc_test_simple.dts +3 -0
@@ -45,6 +45,9 @@ stringarray = "one"; longbytearray = [09 0a 0b 0c 0d 0e 0f 10]; maybe-empty-int = <1>; + + first-node { + }; }; i2c@0 {
tools/dtoc/test_fdt.py +43 -0
49 lines not on this page · raw message
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 11/12 dtoc: Allow inserting a list of nodes into another
In reply to Simon GlassProvide a way to specify a phandle list of nodes which are to be inserted into an existing node.
---
tools/dtoc/fdt.py | 16 ++++++++++++++++ tools/dtoc/test/dtoc_test_simple.dts | 10 ++++++++-- tools/dtoc/test_fdt.py | 18 ++++++++++++++++++ 3 files changed, 42 insertions(+), 2 deletions(-)
tools/dtoc/fdt.py +16 -0
@@ -657,6 +657,22 @@ class Node: for node in src.subnodes: dst.copy_node(node) + def copy_subnodes_from_phandles(self, phandle_list): + """Copy subnodes of a list of nodes into another node + + Args: + phandle_list (list of int): List of phandles of nodes to copy + + For each node in the phandle list, its subnodes and their properties are + copied recursively. Note that it does not copy the node itself, nor its + properties. + """ + for phandle in phandle_list: + parent = self.GetFdt().LookupPhandle(phandle) + tout.debug(f'adding template {parent.path} to node {self.path}') + for node in parent.subnodes: + self.copy_node(node) + class Fdt: """Provides simple access to a flat device tree blob using libfdts.
tools/dtoc/test/dtoc_test_simple.dts +8 -2
@@ -50,7 +50,7 @@ }; }; - i2c@0 { + i2c: i2c@0 { compatible = "sandbox,i2c"; bootph-all; #address-cells = <1>;
@@ -63,10 +63,16 @@ }; }; - orig-node { + orig: orig-node { orig = <1 23 4>; args = "-n first", "second", "-p", "123,456", "-x"; args2 = "a space", "there"; args3 = "-n first second -p 123,456 -x"; + + copy-list = <&i2c &orig>; + + subnode { + a-prop; + }; }; };
tools/dtoc/test_fdt.py +18 -0
@@ -349,6 +349,24 @@ class TestNode(unittest.TestCase): self.dtb.Sync(auto_resize=True) + def test_copy_subnodes_from_phandles(self): + """Test copy_node() function""" + pmic = self.dtb.GetNode('/spl-test3/i2c@0/pmic@9') + self.assertIsNone(pmic) + + orig = self.dtb.GetNode('/orig-node') + node_list = fdt_util.GetPhandleList(orig, 'copy-list') + + dst = self.dtb.GetNode('/spl-test3') + dst.copy_subnodes_from_phandles(node_list) + + pmic = self.dtb.GetNode('/spl-test3/pmic@9') + self.assertTrue(pmic) + + subn = self.dtb.GetNode('/spl-test3/subnode') + self.assertTrue(subn) + self.assertEqual({'a-prop'}, subn.props.keys()) + class TestProp(unittest.TestCase): """Test operation of the Prop class"""
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 12/12 binman: Support simple templates
In reply to Simon GlassCollections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dts
tools/binman/binman.rst +80 -0
@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.
@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================
tools/binman/control.py +9 -0
@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout
@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree
@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:
tools/binman/etype/section.py +2 -1
@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""
tools/binman/ftest.py +8 -0
@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()
tools/binman/test/286_entry_template.dts new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
Re: [PATCH 12/12] binman: Support simple templates
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;
tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};
I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template {
anchor: some-node {
value-1 = <1>;
};
};user-a {
insert-template = <&common>;
};user-b {
insert-template = <&common>;
};&anchor-a {
value-2 = <2>;
};&anchor-b {
value-2 = <3>;
};2. We also have the need to customize a value in a generator sequence that is part of the template. See IOT2050_FLASH_EXTRA_LOADABLES in @config-SEQ in [1].
Jan
[1]
https://github.com/siemens/u-boot/commit/393ce2b78cee9214edda7f7e04f6ca2ce144195e
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };
Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };
Rather than using a phandle, can you not use the node name in the template? For example:
user-a {
insert-template = <&common>;
some-node {
value2 = <2>;}
user-b {
insert-template = <&common>;
some-node {
value2 = <3>;}
Quoted 5 lines
2. We also have the need to customize a value in a generator sequence that is part of the template. See
IOT2050_FLASH_EXTRA_LOADABLESin @config-SEQ in [1].
Again, can you not just override it in the user, as above?
Regards,
Simon
Quoted 10 lines
Jan
[1]
https://github.com/siemens/u-boot/commit/393ce2b78cee9214edda7f7e04f6ca2ce144195e--
Siemens AG, Technology
Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;
Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ {
binman: binman {
multiple-images;
my_template: template {
blob-ext@0 {
filename = "my-blob.bin";
offset = <0>;
};
blob-ext@100 {
filename = "/dev/null";
offset = <0x100>;
};
};
my-image {
filename = "my-image.bin";
insert-template = <&my_template>;
blob-ext@100 {
filename = "my-blob2.bin";
};
};
};
};First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
binman: Node '/binman/my-image/blob-ext@0': Offset 0x0 (0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at 0x107 (263)
Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Jan
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Regards,
Simon
[1] https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Re: [PATCH 12/12] binman: Support simple templates
Quoted 333 lines · Simon Glass
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Quoted 6 lines
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Hmm, already the test pattern from [1] still gives me
binman: Node '/binman/image/blob-ext@0': Offset 0x0 (0) overlaps with previous entry '/binman/image/blob-ext@8' ending at 0xf (15)
I've applied this series and then [1] on top.
Jan
Quoted 4 lines
Regards,
Simon[1] https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Hi Jan,
Quoted 342 lines · Jan Kiszka
Quoted 333 lines · Simon Glass
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Quoted 6 lines
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Quoted 6 lines
Hmm, already the test pattern from [1] still gives me
binman: Node '/binman/image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/image/blob-ext@8' ending at0xf(15)I've applied this series and then [1] on top.
Can you please list the commits so I can check it? I have:
88a31cccc20 (HEAD -> mkim2, dm/mkim-working) binman: Support templating with multiple images71f1ef46d0c (dm-public/mkim-working) binman: Support simple templates 8841829b4c8 dtoc: Allow inserting a list of nodes into another fd2fb91c616 dtoc: Support copying the contents of a node into another 288f9e7c3b6 binman: Correct handling of zero bss size 4725a43d0ef binman: Drop __bss_size variable in bss_data.c c4bfe4917c9 binman: Provide a way to specific the fdt-list directly 6e571e4d260 binman: Convert mkimage to Entry_section 5fd2892283f stm32mp15: Avoid writing symbols in SPL 24c60031f48 binman: Allow disabling symbol writing 5c6a660333c binman: Read _multiple_data_files in the correct place 24a273113bf binman: Use GetEntries() to obtain section contents 1e0659321d8 binman: Init align_default in entry_Section cc3699d1d04 binman: Correct coverage gap in control 67d8b46e6ef (us/WIP/28Jun2023-next) Merge tag 'u-boot-amlogic-next-20230628' of
https://source.denx.de/u-boot/custodians/u-boot-amlogic into next
Regards,
Simon
Quoted 12 lines
Jan
Quoted 4 lines
Regards,
Simon[1] https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
--
Siemens AG, Technology
Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Quoted 364 lines · Simon Glass
Hi Jan,
Quoted 342 lines · Jan Kiszka
Quoted 333 lines · Simon Glass
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Quoted 6 lines
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Quoted 6 lines
Hmm, already the test pattern from [1] still gives me
binman: Node '/binman/image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/image/blob-ext@8' ending at0xf(15)I've applied this series and then [1] on top.
Can you please list the commits so I can check it? I have:
88a31cccc20(HEAD -> mkim2, dm/mkim-working) binman: Support templating with multiple images71f1ef46d0c(dm-public/mkim-working) binman: Support simple templates8841829b4c8dtoc: Allow inserting a list of nodes into anotherfd2fb91c616dtoc: Support copying the contents of a node into another288f9e7c3b6binman: Correct handling of zero bss size4725a43d0efbinman: Drop__bss_sizevariable inbss_data.cc4bfe4917c9binman: Provide a way to specific the fdt-list directly6e571e4d260binman: Convert mkimage toEntry_section5fd2892283fstm32mp15: Avoid writing symbols in SPL
Quoted 9 lines
24c60031f48binman: Allow disabling symbol writing5c6a660333cbinman: Read_multiple_data_filesin the correct place24a273113bfbinman: UseGetEntries()to obtain section contents1e0659321d8binman: Initalign_defaultinentry_Sectioncc3699d1d04binman: Correct coverage gap in control67d8b46e6ef(us/WIP/28Jun2023-next) Merge tag 'u-boot-amlogic-next-20230628' of
https://source.denx.de/u-boot/custodians/u-boot-amlogic into next
I was using the dtoc patches from this series, not the updated but apparently not yet sent ones from your branch. Those seem to make a difference. Continuing to test.
Jan
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Hi Jan,
Quoted 376 lines · Jan Kiszka
Quoted 364 lines · Simon Glass
Hi Jan,
Quoted 342 lines · Jan Kiszka
Quoted 333 lines · Simon Glass
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Quoted 6 lines
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Quoted 6 lines
Hmm, already the test pattern from [1] still gives me
binman: Node '/binman/image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/image/blob-ext@8' ending at0xf(15)I've applied this series and then [1] on top.
Can you please list the commits so I can check it? I have:
88a31cccc20(HEAD -> mkim2, dm/mkim-working) binman: Support templating with multiple images71f1ef46d0c(dm-public/mkim-working) binman: Support simple templates8841829b4c8dtoc: Allow inserting a list of nodes into anotherfd2fb91c616dtoc: Support copying the contents of a node into another288f9e7c3b6binman: Correct handling of zero bss size4725a43d0efbinman: Drop__bss_sizevariable inbss_data.cc4bfe4917c9binman: Provide a way to specific the fdt-list directly6e571e4d260binman: Convert mkimage toEntry_section5fd2892283fstm32mp15: Avoid writing symbols in SPLQuoted 9 lines
24c60031f48binman: Allow disabling symbol writing5c6a660333cbinman: Read_multiple_data_filesin the correct place24a273113bfbinman: UseGetEntries()to obtain section contents1e0659321d8binman: Initalign_defaultinentry_Sectioncc3699d1d04binman: Correct coverage gap in control67d8b46e6ef(us/WIP/28Jun2023-next) Merge tag 'u-boot-amlogic-next-20230628' of
https://source.denx.de/u-boot/custodians/u-boot-amlogic into next
Quoted 3 lines
I was using the dtoc patches from this series, not the updated but apparently not yet sent ones from your branch. Those seem to make a difference. Continuing to test.
I thought I did send out v2. There are some test fails with large numbers of CPUs which I need to fix in v3, but anyway, please use the branch I pushed for testing, and sorry for any discrepancy.
Regards,
Simon
Re: [PATCH 12/12] binman: Support simple templates
Quoted 379 lines · Simon Glass
Hi Jan,
Quoted 376 lines · Jan Kiszka
Quoted 373 lines · Simon Glass
Hi Jan,
Quoted 342 lines · Jan Kiszka
Quoted 333 lines · Simon Glass
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Quoted 6 lines
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Quoted 6 lines
Hmm, already the test pattern from [1] still gives me
binman: Node '/binman/image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/image/blob-ext@8' ending at0xf(15)I've applied this series and then [1] on top.
Can you please list the commits so I can check it? I have:
88a31cccc20(HEAD -> mkim2, dm/mkim-working) binman: Support templating with multiple images71f1ef46d0c(dm-public/mkim-working) binman: Support simple templates8841829b4c8dtoc: Allow inserting a list of nodes into anotherfd2fb91c616dtoc: Support copying the contents of a node into another288f9e7c3b6binman: Correct handling of zero bss size4725a43d0efbinman: Drop__bss_sizevariable inbss_data.cc4bfe4917c9binman: Provide a way to specific the fdt-list directly6e571e4d260binman: Convert mkimage toEntry_section5fd2892283fstm32mp15: Avoid writing symbols in SPL24c60031f48binman: Allow disabling symbol writing5c6a660333cbinman: Read_multiple_data_filesin the correct place24a273113bfbinman: UseGetEntries()to obtain section contents1e0659321d8binman: Initalign_defaultinentry_Sectioncc3699d1d04binman: Correct coverage gap in control67d8b46e6ef(us/WIP/28Jun2023-next) Merge tag 'u-boot-amlogic-next-20230628' of
https://source.denx.de/u-boot/custodians/u-boot-amlogic into next
Quoted 8 lines
Quoted 3 lines
I was using the dtoc patches from this series, not the updated but apparently not yet sent ones from your branch. Those seem to make a difference. Continuing to test.
I thought I did send out v2. There are some test fails with large numbers of CPUs which I need to fix in v3, but anyway, please use the branch I pushed for testing, and sorry for any discrepancy.
Doing this now. And here is the next issue - maybe the last:
/dts-v1/;
/ {
binman: binman {
multiple-images;
my_template: template {
fit@0 {
images {
kernel-1 {
};
kernel-2 {
};
};
};
};
image {
filename = "my-image.bin";
insert-template = <&my_template>;
fit@0 {
images {
kernel-3 {
};
};
};
};
};
};Gives
binman: pylibfdt error -4: FDT_ERR_BADOFFSET
when trying to build it. That subnode kernel-3 can't be added.
Jan
Signature · 2 lines
Siemens AG, Technology Competence Center Embedded Linux
Re: [PATCH 12/12] binman: Support simple templates
Hi Jan,
Quoted 423 lines · Jan Kiszka
Quoted 379 lines · Simon Glass
Hi Jan,
Quoted 376 lines · Jan Kiszka
Quoted 373 lines · Simon Glass
Hi Jan,
Quoted 342 lines · Jan Kiszka
Quoted 333 lines · Simon Glass
Hi Jan,
Quoted 322 lines · Jan Kiszka
Quoted 277 lines · Simon Glass
Hi Jan,
Quoted 259 lines · Jan Kiszka
Hi Simon,
Quoted 223 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
tools/binman/binman.rst | 80 ++++++++++++++++++++++++ tools/binman/control.py | 9 +++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_entry_template.dts | 42 +++++++++++++ 5 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_entry_template.dtstools/binman/binman.rst +80
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,79 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + header { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + header { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +At present there is an unpleasant limitation on this feature: it works by +appending the template nodes after any existing subnodes to the target node. +This means that if the target node includes any subnodes, these appear in order +before the template node. In the above example, 'header' becomes the first +subnode of each image, followed by `fit` and `text`. If this is not what is +desired, there is no way to adjust it. + +Note: The above limitation will likely be removed in future, so that the +template subnodes appear before the target subnodes. + + Updating an ELF file ====================tools/binman/control.py +9
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,12 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +527,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6763,6 +6763,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testEntryTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_entry_template.dts') + first = U_BOOT_DTB_DATA + U_BOOT_DATA + VGA_DATA + second = U_BOOT_DTB_DATA + b'#' + VGA_DATA + U_BOOT_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_entry_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;tools/binman/test/286_entry_template.dts line 42
Quoted 10 lines
tools/binman/test/286_entry_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};I tried to apply this on the IOT2050 use case, but it failed in two cases:
1. We need a template where nodes can be manipulated afterwards, thus we need anchors into those nodes. Naturally, those anchors need to get unique names so that something like this is possible:
common: template { anchor: some-node { value-1 = <1>; }; };user-a { insert-template = <&common>; };user-b { insert-template = <&common>; };Quoted 7 lines
&anchor-a { value-2 = <2>; };&anchor-b { value-2 = <3>; };Rather than using a phandle, can you not use the node name in the template? For example:
user-a { insert-template = <&common>; some-node { value2 = <2>;Quoted 8 lines
}
user-b { insert-template = <&common>; some-node { value2 = <3>;}
I don't think this is working already. This is what I tried:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { blob-ext@0 { filename = "my-blob.bin"; offset = <0>; }; blob-ext@100 { filename = "/dev/null"; offset = <0x100>; }; }; my-image { filename = "my-image.bin"; insert-template = <&my_template>; blob-ext@100 { filename = "my-blob2.bin"; }; }; }; };First, I had to apply that null filename workaround, and the overridden filename is taken for the final image (if I leave out blob-ext@0), but this is at least ugly.
However, the file above as-is does not build:
Quoted 7 lines
binman: Node '/binman/my-image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/my-image/blob-ext@100' ending at0x107(263)Probably for the same reason, leaving fit,fdt-list-val out in a template also generates an error during build. Putting an empty string there and overriding it later does not work.
Quoted 6 lines
I think it is just that it doesn't support templating of multiple-images nodes property. I sent a patch for this [1]
There may be other cases too, but if you have a failure, please see if you can adjust that test (287) to fail for you.
Quoted 6 lines
Hmm, already the test pattern from [1] still gives me
binman: Node '/binman/image/blob-ext@0': Offset
0x0(0) overlaps with previous entry '/binman/image/blob-ext@8' ending at0xf(15)I've applied this series and then [1] on top.
Can you please list the commits so I can check it? I have:
88a31cccc20(HEAD -> mkim2, dm/mkim-working) binman: Support templating with multiple images71f1ef46d0c(dm-public/mkim-working) binman: Support simple templates8841829b4c8dtoc: Allow inserting a list of nodes into anotherfd2fb91c616dtoc: Support copying the contents of a node into another288f9e7c3b6binman: Correct handling of zero bss size4725a43d0efbinman: Drop__bss_sizevariable inbss_data.cc4bfe4917c9binman: Provide a way to specific the fdt-list directly6e571e4d260binman: Convert mkimage toEntry_section5fd2892283fstm32mp15: Avoid writing symbols in SPL24c60031f48binman: Allow disabling symbol writing5c6a660333cbinman: Read_multiple_data_filesin the correct place24a273113bfbinman: UseGetEntries()to obtain section contents1e0659321d8binman: Initalign_defaultinentry_Sectioncc3699d1d04binman: Correct coverage gap in control67d8b46e6ef(us/WIP/28Jun2023-next) Merge tag 'u-boot-amlogic-next-20230628' of
https://source.denx.de/u-boot/custodians/u-boot-amlogic into nextQuoted 8 lines
Quoted 3 lines
I was using the dtoc patches from this series, not the updated but apparently not yet sent ones from your branch. Those seem to make a difference. Continuing to test.
I thought I did send out v2. There are some test fails with large numbers of CPUs which I need to fix in v3, but anyway, please use the branch I pushed for testing, and sorry for any discrepancy.
Doing this now. And here is the next issue - maybe the last:
/dts-v1/;
/ { binman: binman { multiple-images; my_template: template { fit@0 { images { kernel-1 { }; kernel-2 { }; }; }; }; image { filename = "my-image.bin"; insert-template = <&my_template>; fit@0 { images { kernel-3 { }; }; }; }; }; };
Quoted 5 lines
Gives
binman: pylibfdt error -4:
FDT_ERR_BADOFFSETwhen trying to build it. That subnode kernel-3 can't be added.
OK I can repeat that. I think this is a problem with multi-level (in hierarchy) inserting of nodes. I'll need to fiddle with it so will try at weekend.
Regards,
Simon