binman: Simple templating feature and mkimage conversion
Subsystems (2)
v5 cover binman: Simple templating feature and mkimage conversion
This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.
A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.
In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.
This series is available at u-boot-dm/mkim-working
Changes in v5:
- Drop method implementations for which the entry_Section one is enough
Changes in v4:
- Avoid copying phandle nodes
- Support copying over properties from each template node
- Make sure phandles are not copied
- Copy over properties from the top-level template node
- Add new patch to reduce state.SetInt and bintool cmd to debug level
Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image
Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos
Marek Vasut (1):
binman: Convert mkimage to Entry_section
Simon Glass (19):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image
binman: Reduce state.SetInt and bintool cmd to debug level
arch/arm/dts/stm32mp15-u-boot.dtsi | 1 + tools/binman/binman.rst | 94 +++++++++ tools/binman/bintool.py | 2 +- tools/binman/control.py | 34 +++- tools/binman/elf.py | 13 +- tools/binman/elf_test.py | 13 +- tools/binman/entries.rst | 6 + tools/binman/entry.py | 11 +- tools/binman/etype/blob_phase.py | 5 + tools/binman/etype/fit.py | 9 + tools/binman/etype/mkimage.py | 136 ++++++------- tools/binman/etype/section.py | 54 +++-- tools/binman/etype/u_boot_spl_bss_pad.py | 2 +- tools/binman/etype/u_boot_tpl_bss_pad.py | 2 +- tools/binman/etype/u_boot_vpl_bss_pad.py | 2 +- tools/binman/ftest.py | 218 ++++++++++++++++++++- tools/binman/state.py | 4 +- tools/binman/test/282_symbols_disable.dts | 25 +++ tools/binman/test/283_mkimage_special.dts | 24 +++ tools/binman/test/284_fit_fdt_list.dts | 58 ++++++ tools/binman/test/285_spl_expand.dts | 13 ++ tools/binman/test/286_template.dts | 42 ++++ tools/binman/test/287_template_multi.dts | 27 +++ tools/binman/test/288_template_fit.dts | 37 ++++ tools/binman/test/289_template_section.dts | 52 +++++ tools/binman/test/290_mkimage_sym.dts | 27 +++ tools/binman/test/Makefile | 5 +- tools/binman/test/bss_data.c | 3 +- tools/binman/test/bss_data_zero.c | 16 ++ tools/binman/test/bss_data_zero.lds | 15 ++ tools/binman/test/embed_data.lds | 1 + tools/dtoc/fdt.py | 141 ++++++++++++- tools/dtoc/test/dtoc_test_copy.dts | 88 +++++++++ tools/dtoc/test_fdt.py | 113 +++++++++++ 34 files changed, 1184 insertions(+), 109 deletions(-) create mode 100644 tools/binman/test/282_symbols_disable.dts create mode 100644 tools/binman/test/283_mkimage_special.dts create mode 100644 tools/binman/test/284_fit_fdt_list.dts create mode 100644 tools/binman/test/285_spl_expand.dts create mode 100644 tools/binman/test/286_template.dts create mode 100644 tools/binman/test/287_template_multi.dts create mode 100644 tools/binman/test/288_template_fit.dts create mode 100644 tools/binman/test/289_template_section.dts create mode 100644 tools/binman/test/290_mkimage_sym.dts create mode 100644 tools/binman/test/bss_data_zero.c create mode 100644 tools/binman/test/bss_data_zero.lds create mode 100644 tools/dtoc/test/dtoc_test_copy.dts
Signature · 1 line
2.41.0.455.g037347b96a-goog
PATCH v5 1/20 binman: Correct coverage gap in control
Add a pragma to deal with the code-coverage gap which drops binman down to 90% coverage.
---
(no changes since v1)
tools/binman/control.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/binman/control.py +1 -1
@@ -9,7 +9,7 @@ from collections import OrderedDict import glob try: import importlib.resources -except ImportError: +except ImportError: # pragma: no cover # for Python 3.6 import importlib_resources import os
Signature · 1 line
2.41.0.455.g037347b96a-goog
PATCH v5 2/20 binman: Init align_default in entry_Section
In reply to Simon GlassThis should be set up in the init function, to avoid a warning about a property not set up there. Fix it.
---
(no changes since v1)
tools/binman/etype/section.py | 1 + 1 file changed, 1 insertion(+)
tools/binman/etype/section.py +1 -0
@@ -168,6 +168,7 @@ class Entry_section(Entry): self._end_4gb = False self._ignore_missing = False self._filename = None + self.align_default = 0 def IsSpecialSubnode(self, node): """Check if a node is a special one used by the section itself
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 02/20] binman: Init align_default in entry_Section
This should be set up in the init function, to avoid a warning about a property not set up there. Fix it.
---
(no changes since v1)
tools/binman/etype/section.py | 1 + 1 file changed, 1 insertion(+)
Applied to u-boot-dm, thanks!
PATCH v5 3/20 binman: Use GetEntries() to obtain section contents
In reply to Simon GlassSome section types don't have a simple _entries list. Use the GetEntries() method in GetEntryContents() and other places to handle this.
This makes the behaviour more consistent.
---
(no changes since v1)
tools/binman/etype/section.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tools/binman/etype/section.py +9 -9
@@ -720,7 +720,7 @@ class Entry_section(Entry): next_todo.append(entry) return entry - todo = self._entries.values() + todo = self.GetEntries().values() for passnum in range(3): threads = state.GetThreads() next_todo = []
@@ -893,7 +893,7 @@ class Entry_section(Entry): allow_missing: True if allowed, False if not allowed """ self.allow_missing = allow_missing - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.SetAllowMissing(allow_missing) def SetAllowFakeBlob(self, allow_fake):
@@ -903,7 +903,7 @@ class Entry_section(Entry): allow_fake: True if allowed, False if not allowed """ super().SetAllowFakeBlob(allow_fake) - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.SetAllowFakeBlob(allow_fake) def CheckMissing(self, missing_list):
@@ -915,7 +915,7 @@ class Entry_section(Entry): Args: missing_list: List of Entry objects to be added to """ - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckMissing(missing_list) def CheckFakedBlobs(self, faked_blobs_list):
@@ -926,7 +926,7 @@ class Entry_section(Entry): Args: faked_blobs_list: List of Entry objects to be added to """ - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckFakedBlobs(faked_blobs_list) def CheckOptional(self, optional_list):
@@ -937,7 +937,7 @@ class Entry_section(Entry): Args: optional_list (list): List of Entry objects to be added to """ - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckOptional(optional_list) def check_missing_bintools(self, missing_list):
@@ -949,7 +949,7 @@ class Entry_section(Entry): missing_list: List of Bintool objects to be added to """ super().check_missing_bintools(missing_list) - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.check_missing_bintools(missing_list) def _CollectEntries(self, entries, entries_by_name, add_entry):
@@ -999,12 +999,12 @@ class Entry_section(Entry): entry.Raise(f'Missing required properties/entry args: {missing}') def CheckAltFormats(self, alt_formats): - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.CheckAltFormats(alt_formats) def AddBintools(self, btools): super().AddBintools(btools) - for entry in self._entries.values(): + for entry in self.GetEntries().values(): entry.AddBintools(btools) def read_elf_segments(self):
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 03/20] binman: Use GetEntries() to obtain section contents
Some section types don't have a simple _entries list. Use the GetEntries() method in GetEntryContents() and other places to handle this.
This makes the behaviour more consistent.
---
(no changes since v1)
tools/binman/etype/section.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
Applied to u-boot-dm, thanks!
PATCH v5 4/20 binman: Read _multiple_data_files in the correct place
In reply to Simon GlassMove this to the ReadEntries() function where it belongs.
---
(no changes since v1)
tools/binman/etype/mkimage.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/binman/etype/mkimage.py +2 -1
@@ -121,7 +121,6 @@ class Entry_mkimage(Entry): """ def __init__(self, section, etype, node): super().__init__(section, etype, node) - self._multiple_data_files = fdt_util.GetBool(self._node, 'multiple-data-files') self._mkimage_entries = OrderedDict() self._imagename = None self._filename = fdt_util.GetString(self._node, 'filename')
@@ -129,6 +128,8 @@ class Entry_mkimage(Entry): def ReadNode(self): super().ReadNode() + self._multiple_data_files = fdt_util.GetBool(self._node, + 'multiple-data-files') self._args = fdt_util.GetArgs(self._node, 'args') self._data_to_imagename = fdt_util.GetBool(self._node, 'data-to-imagename')
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 04/20] binman: Read _multiple_data_files in the correct place
Move this to the ReadEntries() function where it belongs.
---
(no changes since v1)
tools/binman/etype/mkimage.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Applied to u-boot-dm, thanks!
PATCH v5 5/20 binman: Allow disabling symbol writing
In reply to Simon GlassSome boards don't use symbol writing but do access the symbols in SPL. Provide an option to work around this.
---
(no changes since v1)
tools/binman/binman.rst | 7 ++++++ tools/binman/entry.py | 4 +++- tools/binman/etype/blob_phase.py | 5 ++++ tools/binman/ftest.py | 28 +++++++++++++++++++---- tools/binman/test/282_symbols_disable.dts | 25 ++++++++++++++++++++ 5 files changed, 64 insertions(+), 5 deletions(-) create mode 100644 tools/binman/test/282_symbols_disable.dts
tools/binman/binman.rst +7 -0
@@ -831,6 +831,13 @@ write-symbols: binman. This is automatic for certain entry types, e.g. `u-boot-spl`. See binman_syms_ for more information. +no-write-symbols: + Disables symbol writing for this entry. This can be used in entry types + where symbol writing is automatic. For example, if `u-boot-spl` refers to + the `u_boot_any_image_pos` symbol but U-Boot is not available in the image + containing SPL, this can be used to disable the writing. Quite likely this + indicates a bug in your setup. + elf-filename: Sets the file name of a blob's associated ELF file. For example, if the blob is `zephyr.bin` then the ELF file may be `zephyr.elf`. This allows
tools/binman/entry.py +3 -1
@@ -158,6 +158,7 @@ class Entry(object): self.offset_from_elf = None self.preserve = False self.build_done = False + self.no_write_symbols = False @staticmethod def FindEntryClass(etype, expanded):
@@ -321,6 +322,7 @@ class Entry(object): 'offset-from-elf') self.preserve = fdt_util.GetBool(self._node, 'preserve') + self.no_write_symbols = fdt_util.GetBool(self._node, 'no-write-symbols') def GetDefaultFilename(self): return None
@@ -695,7 +697,7 @@ class Entry(object): Args: section: Section containing the entry """ - if self.auto_write_symbols: + if self.auto_write_symbols and not self.no_write_symbols: # Check if we are writing symbols into an ELF file is_elf = self.GetDefaultFilename() == self.elf_fname elf.LookupAndWriteSymbols(self.elf_fname, self, section.GetImage(),
tools/binman/etype/blob_phase.py +5 -0
@@ -52,3 +52,8 @@ class Entry_blob_phase(Entry_section): # Read entries again, now that we have some self.ReadEntries() + + # Propagate the no-write-symbols property + if self.no_write_symbols: + for entry in self._entries.values(): + entry.no_write_symbols = True
tools/binman/ftest.py +24 -4
@@ -1452,7 +1452,7 @@ class TestFunctional(unittest.TestCase): self.assertEqual(U_BOOT_SPL_NODTB_DATA, data[:len(U_BOOT_SPL_NODTB_DATA)]) def checkSymbols(self, dts, base_data, u_boot_offset, entry_args=None, - use_expanded=False): + use_expanded=False, no_write_symbols=False): """Check the image contains the expected symbol values Args:
@@ -1481,9 +1481,14 @@ class TestFunctional(unittest.TestCase): sym_values = struct.pack('<LLQLL', elf.BINMAN_SYM_MAGIC_VALUE, 0x00, u_boot_offset + len(U_BOOT_DATA), 0x10 + u_boot_offset, 0x04) - expected = (sym_values + base_data[24:] + - tools.get_bytes(0xff, 1) + U_BOOT_DATA + sym_values + - base_data[24:]) + if no_write_symbols: + expected = (base_data + + tools.get_bytes(0xff, 0x38 - len(base_data)) + + U_BOOT_DATA + base_data) + else: + expected = (sym_values + base_data[24:] + + tools.get_bytes(0xff, 1) + U_BOOT_DATA + sym_values + + base_data[24:]) self.assertEqual(expected, data) def testSymbols(self):
@@ -6676,6 +6681,21 @@ fdt fdtmap Extract the devicetree blob from the fdtmap ['fit']) self.assertIn("Node '/fit': Missing tool: 'mkimage'", str(e.exception)) + def testSymbolNoWrite(self): + """Test disabling of symbol writing""" + self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_DATA, 0x1c, + no_write_symbols=True) + + def testSymbolNoWriteExpanded(self): + """Test disabling of symbol writing in expanded entries""" + entry_args = { + 'spl-dtb': '1', + } + self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_NODTB_DATA + + U_BOOT_SPL_DTB_DATA, 0x38, + entry_args=entry_args, use_expanded=True, + no_write_symbols=True) + if __name__ == "__main__": unittest.main()
tools/binman/test/282_symbols_disable.dts new file +25 -0
@@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + pad-byte = <0xff>; + u-boot-spl { + no-write-symbols; + }; + + u-boot { + offset = <0x38>; + no-expanded; + }; + + u-boot-spl2 { + type = "u-boot-spl"; + no-write-symbols; + }; + }; +};
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 05/20] binman: Allow disabling symbol writing
Some boards don't use symbol writing but do access the symbols in SPL. Provide an option to work around this.
---
(no changes since v1)
tools/binman/binman.rst | 7 ++++++ tools/binman/entry.py | 4 +++- tools/binman/etype/blob_phase.py | 5 ++++ tools/binman/ftest.py | 28 +++++++++++++++++++---- tools/binman/test/282_symbols_disable.dts | 25 ++++++++++++++++++++ 5 files changed, 64 insertions(+), 5 deletions(-) create mode 100644 tools/binman/test/282_symbols_disable.dts
Applied to u-boot-dm, thanks!
PATCH v5 6/20 stm32mp15: Avoid writing symbols in SPL
In reply to Simon GlassThese boards use SPL in a mkimage entry and apparently access the symbol containing the image position of U-Boot, but put U-Boot in another image. This means that binman is unable to fill in the symbol correctly in the SPL binary.
This doesn't matter at present since mkimage doesn't support symbol writing. But with the upcoming conversion to a section, it will. So add a property to disable symbol writing.
---
(no changes since v1)
arch/arm/dts/stm32mp15-u-boot.dtsi | 1 + 1 file changed, 1 insertion(+)
arch/arm/dts/stm32mp15-u-boot.dtsi +1 -0
@@ -226,6 +226,7 @@ mkimage { args = "-T stm32image -a 0x2ffc2500 -e 0x2ffc2500"; u-boot-spl { + no-write-symbols; }; }; };
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 06/20] stm32mp15: Avoid writing symbols in SPL
These boards use SPL in a mkimage entry and apparently access the symbol containing the image position of U-Boot, but put U-Boot in another image. This means that binman is unable to fill in the symbol correctly in the SPL binary.
This doesn't matter at present since mkimage doesn't support symbol writing. But with the upcoming conversion to a section, it will. So add a property to disable symbol writing.
---
(no changes since v1)
arch/arm/dts/stm32mp15-u-boot.dtsi | 1 + 1 file changed, 1 insertion(+)
Applied to u-boot-dm, thanks!
PATCH v5 7/20 binman: Update elf to return number of written symbols
In reply to Simon GlassUpdate the LookupAndWriteSymbols() function to return the number of symbols written. Also add some logging for when debugging is not enabled.
---
(no changes since v3)
Changes in v3:
- Add new patch for elf to return number of written symbols
tools/binman/elf.py | 13 +++++++++++-- tools/binman/elf_test.py | 8 +++++--- 2 files changed, 16 insertions(+), 5 deletions(-)
tools/binman/elf.py +11 -2
@@ -248,6 +248,9 @@ def LookupAndWriteSymbols(elf_fname, entry, section, is_elf=False, entry: Entry to process section: Section which can be used to lookup symbol values base_sym: Base symbol marking the start of the image + + Returns: + int: Number of symbols written """ if not base_sym: base_sym = '__image_copy_start'
@@ -269,12 +272,13 @@ def LookupAndWriteSymbols(elf_fname, entry, section, is_elf=False, if not syms: tout.debug('LookupAndWriteSymbols: no syms') - return + return 0 base = syms.get(base_sym) if not base and not is_elf: tout.debug('LookupAndWriteSymbols: no base') - return + return 0 base_addr = 0 if is_elf else base.address + count = 0 for name, sym in syms.items(): if name.startswith('_binman'): msg = ("Section '%s': Symbol '%s'\n in entry '%s'" %
@@ -307,6 +311,11 @@ def LookupAndWriteSymbols(elf_fname, entry, section, is_elf=False, (msg, name, offset, value, len(value_bytes))) entry.data = (entry.data[:offset] + value_bytes + entry.data[offset + sym.size:]) + count += 1 + if count: + tout.detail( + f"Section '{section.GetPath()}': entry '{entry.GetPath()}' : {count} symbols") + return count def GetSymbolValue(sym, data, msg): """Get the value of a symbol
tools/binman/elf_test.py +5 -3
@@ -141,7 +141,8 @@ class TestElf(unittest.TestCase): entry = FakeEntry(10) section = FakeSection() elf_fname = self.ElfTestFile('u_boot_binman_syms_bad') - elf.LookupAndWriteSymbols(elf_fname, entry, section) + count = elf.LookupAndWriteSymbols(elf_fname, entry, section) + self.assertEqual(0, count) def testBadSymbolSize(self): """Test that an attempt to use an 8-bit symbol are detected
@@ -162,7 +163,7 @@ class TestElf(unittest.TestCase): def testNoValue(self): """Test the case where we have no value for the symbol - This should produce -1 values for all thress symbols, taking up the + This should produce -1 values for all three symbols, taking up the first 16 bytes of the image. """ if not elf.ELF_TOOLS:
@@ -170,7 +171,8 @@ class TestElf(unittest.TestCase): entry = FakeEntry(28) section = FakeSection(sym_value=None) elf_fname = self.ElfTestFile('u_boot_binman_syms') - elf.LookupAndWriteSymbols(elf_fname, entry, section) + count = elf.LookupAndWriteSymbols(elf_fname, entry, section) + self.assertEqual(5, count) expected = (struct.pack('<L', elf.BINMAN_SYM_MAGIC_VALUE) + tools.get_bytes(255, 20) + tools.get_bytes(ord('a'), 4))
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 07/20] binman: Update elf to return number of written symbols
Update the LookupAndWriteSymbols() function to return the number of symbols written. Also add some logging for when debugging is not enabled.
---
(no changes since v3)
Changes in v3:
- Add new patch for elf to return number of written symbols
tools/binman/elf.py | 13 +++++++++++-- tools/binman/elf_test.py | 8 +++++--- 2 files changed, 16 insertions(+), 5 deletions(-)
Applied to u-boot-dm, thanks!
PATCH v5 8/20 binman: Add more detail on how ObtainContents() works
In reply to Simon GlassThis area of binman can be a bit confusing. Add some more comments to help.
---
(no changes since v3)
Changes in v3:
- Add new patch with more detail on how ObtainContents() works
tools/binman/entry.py | 3 +++ tools/binman/etype/section.py | 32 +++++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-)
tools/binman/entry.py +3 -0
@@ -474,6 +474,9 @@ class Entry(object): def ObtainContents(self, skip_entry=None, fake_size=0): """Figure out the contents of an entry. + For missing blobs (where allow-missing is enabled), the contents are set + to b'' and self.missing is set to True. + Args: skip_entry (Entry): Entry to skip when obtaining section contents fake_size (int): Size of fake file to create if needed
tools/binman/etype/section.py +31 -1
@@ -316,12 +316,15 @@ class Entry_section(Entry): This should be overridden by subclasses which want to build their own data structure for the section. + Missing entries will have be given empty (or fake) data, so are + processed normally here. + Args: required: True if the data must be present, False if it is OK to return None Returns: - Contents of the section (bytes) + Contents of the section (bytes), None if not available """ section_data = bytearray()
@@ -711,6 +714,33 @@ class Entry_section(Entry): def GetEntryContents(self, skip_entry=None): """Call ObtainContents() for each entry in the section + The overall goal of this function is to read in any available data in + this entry and any subentries. This includes reading in blobs, setting + up objects which have predefined contents, etc. + + Since entry types which contain entries call ObtainContents() on all + those entries too, the result is that ObtainContents() is called + recursively for the whole tree below this one. + + Entries with subentries are generally not *themselves& processed here, + i.e. their ObtainContents() implementation simply obtains contents of + their subentries, skipping their own contents. For example, the + implementation here (for entry_Section) does not attempt to pack the + entries into a final result. That is handled later. + + Generally, calling this results in SetContents() being called for each + entry, so that the 'data' and 'contents_size; properties are set, and + subsequent calls to GetData() will return value data. + + Where 'allow_missing' is set, this can result in the 'missing' property + being set to True if there is no data. This is handled by setting the + data to b''. This function will still return success. Future calls to + GetData() for this entry will return b'', or in the case where the data + is faked, GetData() will return that fake data. + + Args: + skip_entry: (single) Entry to skip, or None to process all entries + Note that this may set entry.absent to True if the entry is not actually needed """
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 08/20] binman: Add more detail on how ObtainContents() works
This area of binman can be a bit confusing. Add some more comments to help.
---
(no changes since v3)
Changes in v3:
- Add new patch with more detail on how ObtainContents() works
tools/binman/entry.py | 3 +++ tools/binman/etype/section.py | 32 +++++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-)
Applied to u-boot-dm, thanks!
PATCH v5 9/20 binman: Convert mkimage to Entry_section
In reply to Simon GlassFrom: Marek Vasut <[email protected]>
This is needed to handle mkimage with inner section located itself in a section.
Use BuildSectionData() instead of ObtainContents(), add tests and a few other minor fixes:
---
Changes in v5:
- Drop method implementations for which the entry_Section one is enough
Changes in v3:
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
Changes in v2:
- Drop now-unnecessary methods in mkimage etype
tools/binman/entry.py | 6 +- tools/binman/etype/mkimage.py | 113 ++++++++-------------- tools/binman/ftest.py | 67 ++++++++++++- tools/binman/test/283_mkimage_special.dts | 24 +++++ 4 files changed, 130 insertions(+), 80 deletions(-) create mode 100644 tools/binman/test/283_mkimage_special.dts
tools/binman/entry.py +2 -4
@@ -1314,10 +1314,8 @@ features to produce new behaviours. """ data = b'' for entry in entries: - # First get the input data and put it in a file. If not available, - # try later. - if not entry.ObtainContents(fake_size=fake_size): - return None, None, None + # First get the input data and put it in a file + entry.ObtainContents(fake_size=fake_size) data += entry.GetData() uniq = self.GetUniqueName() fname = tools.get_output_filename(f'{prefix}.{uniq}')
tools/binman/etype/mkimage.py +38 -75
178 lines not on this page · raw message
tools/binman/ftest.py +66 -1
213 lines not on this page · raw message
tools/binman/test/283_mkimage_special.dts new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + mkimage { + args = "-T script"; + + u-boot { + }; + + hash { + }; + + imagename { + type = "u-boot"; + }; + }; + }; +};
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 09/20] binman: Convert mkimage to Entry_section
From: Marek Vasut <[email protected]>
This is needed to handle mkimage with inner section located itself in a section.
Use BuildSectionData() instead of ObtainContents(), add tests and a few other minor fixes:
---
Changes in v5:
- Drop method implementations for which the entry_Section one is enough
Changes in v3:
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
Changes in v2:
- Drop now-unnecessary methods in mkimage etype
tools/binman/entry.py | 6 +- tools/binman/etype/mkimage.py | 113 ++++++++-------------- tools/binman/ftest.py | 67 ++++++++++++- tools/binman/test/283_mkimage_special.dts | 24 +++++ 4 files changed, 130 insertions(+), 80 deletions(-) create mode 100644 tools/binman/test/283_mkimage_special.dts
Applied to u-boot-dm, thanks!
PATCH v5 10/20 binman: Provide a way to specify the fdt-list directly
In reply to Simon GlassSometimes multiple boards are built with binman and it is useful to specify a different FDT list for each. At present this is not possible without providing multiple values of the of-list entryarg (which is not supported in the U-Boot build system).
Allow a fit,fdt-list-val string-list property to be used instead.
---
(no changes since v3)
Changes in v3:
- Fix 'specific' typo
tools/binman/entries.rst | 6 +++ tools/binman/etype/fit.py | 9 ++++ tools/binman/ftest.py | 12 ++++++ tools/binman/test/284_fit_fdt_list.dts | 58 ++++++++++++++++++++++++++ 4 files changed, 85 insertions(+) create mode 100644 tools/binman/test/284_fit_fdt_list.dts
tools/binman/entries.rst +6 -0
@@ -615,6 +615,12 @@ The top-level 'fit' node supports the following special properties: `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed to binman. + fit,fdt-list-val + As an alternative to fit,fdt-list the list of device tree files + can be provided in this property as a string list, e.g.:: + + fit,fdt-list-val = "dtb1", "dtb2"; + Substitutions ~~~~~~~~~~~~~
tools/binman/etype/fit.py +9 -0
@@ -81,6 +81,12 @@ class Entry_fit(Entry_section): `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed to binman. + fit,fdt-list-val + As an alternative to fit,fdt-list the list of device tree files + can be provided in this property as a string list, e.g.:: + + fit,fdt-list-val = "dtb1", "dtb2"; + Substitutions ~~~~~~~~~~~~~
@@ -361,6 +367,9 @@ class Entry_fit(Entry_section): [EntryArg(self._fit_list_prop.value, str)]) if fdts is not None: self._fdts = fdts.split() + else: + self._fdts = fdt_util.GetStringList(self._node, 'fit,fdt-list-val') + self._fit_default_dt = self.GetEntryArgsOrProps([EntryArg('default-dt', str)])[0]
tools/binman/ftest.py +12 -0
@@ -6761,6 +6761,18 @@ fdt fdtmap Extract the devicetree blob from the fdtmap # Just check that the data appears in the file somewhere self.assertIn(U_BOOT_DATA, data) + def testFitFdtList(self): + """Test an image with an FIT with the fit,fdt-list-val option""" + entry_args = { + 'default-dt': 'test-fdt2', + } + data = self._DoReadFileDtb( + '284_fit_fdt_list.dts', + entry_args=entry_args, + extra_indirs=[os.path.join(self._indir, TEST_FDT_SUBDIR)])[0] + self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):]) + fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)] + if __name__ == "__main__": unittest.main()
tools/binman/test/284_fit_fdt_list.dts new file +58 -0
58 lines not on this page · raw message
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 10/20] binman: Provide a way to specify the fdt-list directly
Sometimes multiple boards are built with binman and it is useful to specify a different FDT list for each. At present this is not possible without providing multiple values of the of-list entryarg (which is not supported in the U-Boot build system).
Allow a fit,fdt-list-val string-list property to be used instead.
---
(no changes since v3)
Changes in v3:
- Fix 'specific' typo
tools/binman/entries.rst | 6 +++ tools/binman/etype/fit.py | 9 ++++ tools/binman/ftest.py | 12 ++++++ tools/binman/test/284_fit_fdt_list.dts | 58 ++++++++++++++++++++++++++ 4 files changed, 85 insertions(+) create mode 100644 tools/binman/test/284_fit_fdt_list.dts
Applied to u-boot-dm, thanks!
PATCH v5 11/20 binman: Drop __bss_size variable in bss_data.c
In reply to Simon GlassThis is not needed since the linker script sets it up. Drop the variable to avoid confusion.
Fix the prototype for main() while we are here.
---
(no changes since v1)
tools/binman/test/bss_data.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
tools/binman/test/bss_data.c +1 -2
@@ -7,9 +7,8 @@*/ int bss_data[10 ]; -int __bss_size =sizeof (bss_data); -int main () +int main (void ) { bss_data[2 ] =2 ;
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 11/20] binman: Drop __bss_size variable in bss_data.c
This is not needed since the linker script sets it up. Drop the variable to avoid confusion.
Fix the prototype for main() while we are here.
---
(no changes since v1)
tools/binman/test/bss_data.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
Applied to u-boot-dm, thanks!
PATCH v5 12/20 binman: Correct handling of zero bss size
In reply to Simon GlassFix the check for the __bss_size symbol, since it may be 0. Unfortunately there was no test coverage for this.
---
(no changes since v1)
tools/binman/elf_test.py | 5 +++++ tools/binman/etype/u_boot_spl_bss_pad.py | 2 +- tools/binman/etype/u_boot_tpl_bss_pad.py | 2 +- tools/binman/etype/u_boot_vpl_bss_pad.py | 2 +- tools/binman/ftest.py | 12 ++++++++++++ tools/binman/test/285_spl_expand.dts | 13 +++++++++++++ tools/binman/test/Makefile | 5 ++++- tools/binman/test/bss_data_zero.c | 16 ++++++++++++++++ tools/binman/test/bss_data_zero.lds | 15 +++++++++++++++ tools/binman/test/embed_data.lds | 1 + 10 files changed, 69 insertions(+), 4 deletions(-) create mode 100644 tools/binman/test/285_spl_expand.dts create mode 100644 tools/binman/test/bss_data_zero.c create mode 100644 tools/binman/test/bss_data_zero.lds
tools/binman/elf_test.py +5 -0
@@ -371,6 +371,11 @@ class TestElf(unittest.TestCase): elf.GetSymbolOffset(fname, 'embed') self.assertIn('__image_copy_start', str(e.exception)) + def test_get_symbol_address(self): + fname = self.ElfTestFile('embed_data') + addr = elf.GetSymbolAddress(fname, 'region_size') + self.assertEqual(0, addr) + if __name__ == '__main__': unittest.main()
tools/binman/etype/u_boot_spl_bss_pad.py +1 -1
@@ -38,7 +38,7 @@ class Entry_u_boot_spl_bss_pad(Entry_blob): def ObtainContents(self): fname = tools.get_input_filename('spl/u-boot-spl') bss_size = elf.GetSymbolAddress(fname, '__bss_size') - if not bss_size: + if bss_size is None: self.Raise('Expected __bss_size symbol in spl/u-boot-spl') self.SetContents(tools.get_bytes(0, bss_size)) return True
tools/binman/etype/u_boot_tpl_bss_pad.py +1 -1
@@ -38,7 +38,7 @@ class Entry_u_boot_tpl_bss_pad(Entry_blob): def ObtainContents(self): fname = tools.get_input_filename('tpl/u-boot-tpl') bss_size = elf.GetSymbolAddress(fname, '__bss_size') - if not bss_size: + if bss_size is None: self.Raise('Expected __bss_size symbol in tpl/u-boot-tpl') self.SetContents(tools.get_bytes(0, bss_size)) return True
tools/binman/etype/u_boot_vpl_bss_pad.py +1 -1
@@ -38,7 +38,7 @@ class Entry_u_boot_vpl_bss_pad(Entry_blob): def ObtainContents(self): fname = tools.get_input_filename('vpl/u-boot-vpl') bss_size = elf.GetSymbolAddress(fname, '__bss_size') - if not bss_size: + if bss_size is None: self.Raise('Expected __bss_size symbol in vpl/u-boot-vpl') self.SetContents(tools.get_bytes(0, bss_size)) return True
tools/binman/ftest.py +12 -0
@@ -6773,6 +6773,18 @@ fdt fdtmap Extract the devicetree blob from the fdtmap self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):]) fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)] + def testSplEmptyBss(self): + """Test an expanded SPL with a zero-size BSS""" + # ELF file with a '__bss_size' symbol + self._SetupSplElf(src_fname='bss_data_zero') + + entry_args = { + 'spl-bss-pad': 'y', + 'spl-dtb': 'y', + } + data = self._DoReadFileDtb('285_spl_expand.dts', + use_expanded=True, entry_args=entry_args)[0] + if __name__ == "__main__": unittest.main()
tools/binman/test/285_spl_expand.dts new file +13 -0
@@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-spl { + }; + }; +};
tools/binman/test/Makefile +4 -1
@@ -32,7 +32,7 @@ LDS_BINMAN_EMBED := -T $(SRC)u_boot_binman_embed.lds LDS_EFL_SECTIONS := -T $(SRC)elf_sections.lds LDS_BLOB := -T $(SRC)blob_syms.lds -TARGETS = u_boot_ucode_ptr u_boot_no_ucode_ptr bss_data \ +TARGETS = u_boot_ucode_ptr u_boot_no_ucode_ptr bss_data bss_data_zero \ u_boot_binman_syms u_boot_binman_syms.bin u_boot_binman_syms_bad \ u_boot_binman_syms_size u_boot_binman_syms_x86 embed_data \ u_boot_binman_embed u_boot_binman_embed_sm elf_sections blob_syms.bin
@@ -48,6 +48,9 @@ u_boot_ucode_ptr: u_boot_ucode_ptr.c bss_data: CFLAGS += $(SRC)bss_data.lds bss_data: bss_data.c +bss_data_zero: CFLAGS += $(SRC)bss_data_zero.lds +bss_data_zero: bss_data_zero.c + embed_data: CFLAGS += $(SRC)embed_data.lds embed_data: embed_data.c
tools/binman/test/bss_data_zero.c new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* +* Copyright (c) 2016 Google, Inc +* +* Simple program to create a bss_data region so the symbol can be read +* by binutils. This is used by binman tests. +*/ + +int bss_data[10 ]; + +int main (void ) +{ + bss_data[2 ] =2 ; + +return 0 ; +}
tools/binman/test/bss_data_zero.lds new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Copyright (c) 2016 Google, Inc + */ + +OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386") +OUTPUT_ARCH(i386) +ENTRY(_start) + +SECTIONS +{ + . = 0xfffffdf0; + _start = .; + __bss_size = 0; +}
tools/binman/test/embed_data.lds +1 -0
@@ -17,6 +17,7 @@ SECTIONS embed_start = .; *(.embed*) embed_end = .; + region_size = 0; . = ALIGN(32); *(.data*) }
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 12/20] binman: Correct handling of zero bss size
Fix the check for the __bss_size symbol, since it may be 0. Unfortunately there was no test coverage for this.
---
(no changes since v1)
tools/binman/elf_test.py | 5 +++++ tools/binman/etype/u_boot_spl_bss_pad.py | 2 +- tools/binman/etype/u_boot_tpl_bss_pad.py | 2 +- tools/binman/etype/u_boot_vpl_bss_pad.py | 2 +- tools/binman/ftest.py | 12 ++++++++++++ tools/binman/test/285_spl_expand.dts | 13 +++++++++++++ tools/binman/test/Makefile | 5 ++++- tools/binman/test/bss_data_zero.c | 16 ++++++++++++++++ tools/binman/test/bss_data_zero.lds | 15 +++++++++++++++ tools/binman/test/embed_data.lds | 1 + 10 files changed, 69 insertions(+), 4 deletions(-) create mode 100644 tools/binman/test/285_spl_expand.dts create mode 100644 tools/binman/test/bss_data_zero.c create mode 100644 tools/binman/test/bss_data_zero.lds
Applied to u-boot-dm, thanks!
PATCH v5 13/20 dtoc: Support copying the contents of a node into another
In reply to Simon GlassThis permits implementation of a simple templating system, where a node can be reused as a base for others.
For now this adds new subnodes after any existing ones.
---
(no changes since v4)
Changes in v4:
- Avoid copying phandle nodes
Changes in v3:
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
tools/dtoc/fdt.py | 119 ++++++++++++++++++++++++++++- tools/dtoc/test/dtoc_test_copy.dts | 76 ++++++++++++++++++ tools/dtoc/test_fdt.py | 74 ++++++++++++++++++ 3 files changed, 266 insertions(+), 3 deletions(-) create mode 100644 tools/dtoc/test/dtoc_test_copy.dts
tools/dtoc/fdt.py +116 -3
@@ -13,6 +13,7 @@ from dtoc import fdt_util import libfdt from libfdt import QUIET_NOTFOUND from u_boot_pylib import tools +from u_boot_pylib import tout # This deals with a device tree, presenting it as an assortment of Node and # Prop objects, representing nodes and properties, respectively. This file
@@ -264,6 +265,13 @@ class Prop: fdt_obj.setprop(node.Offset(), self.name, self.bytes) self.dirty = False + def purge(self): + """Set a property offset to None + + The property remains in the tree structure and will be recreated when + the FDT is synced + """ + self._offset = None class Node: """A device tree node
@@ -534,8 +542,8 @@ class Node: """ return self.AddData(prop_name, struct.pack('>I', val)) - def AddSubnode(self, name): - """Add a new subnode to the node + def Subnode(self, name): + """Create new subnode for the node Args: name: name of node to add
@@ -544,10 +552,72 @@ class Node: New subnode that was created """ path = self.path + '/' + name - subnode = Node(self._fdt, self, None, name, path) + return Node(self._fdt, self, None, name, path) + + def AddSubnode(self, name): + """Add a new subnode to the node, after all other subnodes + + Args: + name: name of node to add + + Returns: + New subnode that was created + """ + subnode = self.Subnode(name) self.subnodes.append(subnode) return subnode + def insert_subnode(self, name): + """Add a new subnode to the node, before all other subnodes + + This deletes other subnodes and sets their offset to None, so that they + will be recreated after this one. + + Args: + name: name of node to add + + Returns: + New subnode that was created + """ + # Deleting a node invalidates the offsets of all following nodes, so + # process in reverse order so that the offset of each node remains valid + # until deletion. + for subnode in reversed(self.subnodes): + subnode.purge(True) + subnode = self.Subnode(name) + self.subnodes.insert(0, subnode) + return subnode + + def purge(self, delete_it=False): + """Purge this node, setting offset to None and deleting from FDT""" + if self._offset is not None: + if delete_it: + CheckErr(self._fdt._fdt_obj.del_node(self.Offset()), + "Node '%s': delete" % self.path) + self._offset = None + self._fdt.Invalidate() + + for prop in self.props.values(): + prop.purge() + + for subnode in self.subnodes: + subnode.purge(False) + + def move_to_first(self): + """Move the current node to first in its parent's node list""" + parent = self.parent + if parent.subnodes and parent.subnodes[0] == self: + return + for subnode in reversed(parent.subnodes): + subnode.purge(True) + + new_subnodes = [self] + for subnode in parent.subnodes: + #subnode.purge(False) + if subnode != self: + new_subnodes.append(subnode) + parent.subnodes = new_subnodes + def Delete(self): """Delete a node
@@ -635,6 +705,49 @@ class Node: prop.Sync(auto_resize) return added + def merge_props(self, src): + """Copy missing properties (except 'phandle') from another node + + Args: + src (Node): Node containing properties to copy + + Adds properties which are present in src but not in this node. Any + 'phandle' property is not copied since this might result in two nodes + with the same phandle, thus making phandle references ambiguous. + """ + for name, src_prop in src.props.items(): + if name != 'phandle' and name not in self.props: + self.props[name] = Prop(self, None, name, src_prop.bytes) + + def copy_node(self, src): + """Copy a node and all its subnodes into this node + + Args: + src (Node): Node to copy + + Returns: + Node: Resulting destination node + + This works recursively. + + The new node is put before all other nodes. If the node already + exists, just its subnodes and properties are copied, placing them before + any existing subnodes. Properties which exist in the destination node + already are not copied. + """ + dst = self.FindNode(src.name) + if dst: + dst.move_to_first() + else: + dst = self.insert_subnode(src.name) + dst.merge_props(src) + + # Process in reverse order so that they appear correctly in the result, + # since copy_node() puts the node first in the list + for node in reversed(src.subnodes): + dst.copy_node(node) + return dst + class Fdt: """Provides simple access to a flat device tree blob using libfdts.
tools/dtoc/test/dtoc_test_copy.dts new file +76 -0
76 lines not on this page · raw message
tools/dtoc/test_fdt.py +74 -0
80 lines not on this page · raw message
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 13/20] dtoc: Support copying the contents of a node into another
This permits implementation of a simple templating system, where a node can be reused as a base for others.
For now this adds new subnodes after any existing ones.
---
(no changes since v4)
Changes in v4:
- Avoid copying phandle nodes
Changes in v3:
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
tools/dtoc/fdt.py | 119 ++++++++++++++++++++++++++++- tools/dtoc/test/dtoc_test_copy.dts | 76 ++++++++++++++++++ tools/dtoc/test_fdt.py | 74 ++++++++++++++++++ 3 files changed, 266 insertions(+), 3 deletions(-) create mode 100644 tools/dtoc/test/dtoc_test_copy.dts
Applied to u-boot-dm, thanks!
PATCH v5 14/20 dtoc: Allow inserting a list of nodes into another
In reply to Simon GlassProvide a way to specify a phandle list of nodes which are to be inserted into an existing node.
---
(no changes since v4)
Changes in v4:
- Support copying over properties from each template node
- Make sure phandles are not copied
Changes in v3:
- Adjust to use the new example file
tools/dtoc/fdt.py | 22 +++++++++++++++++ tools/dtoc/test/dtoc_test_copy.dts | 14 ++++++++++- tools/dtoc/test_fdt.py | 39 ++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+), 1 deletion(-)
tools/dtoc/fdt.py +22 -0
@@ -748,6 +748,28 @@ class Node: dst.copy_node(node) return dst + def copy_subnodes_from_phandles(self, phandle_list): + """Copy subnodes of a list of nodes into another node + + Args: + phandle_list (list of int): List of phandles of nodes to copy + + For each node in the phandle list, its subnodes and their properties are + copied recursively. Note that it does not copy the node itself, nor its + properties. + """ + # Process in reverse order, since new nodes are inserted at the start of + # the destination's node list. We want them to appear in order of the + # phandle list + for phandle in phandle_list.__reversed__(): + parent = self.GetFdt().LookupPhandle(phandle) + tout.debug(f'adding template {parent.path} to node {self.path}') + for node in parent.subnodes.__reversed__(): + dst = self.copy_node(node) + + tout.debug(f'merge props from {parent.path} to {dst.path}') + self.merge_props(parent) + class Fdt: """Provides simple access to a flat device tree blob using libfdts.
tools/dtoc/test/dtoc_test_copy.dts +13 -1
@@ -11,6 +11,7 @@ #address-cells = <1>; #size-cells = <1>; reference = <&over>; /* nake sure that the 'over' phandle exists */ + copy-list = <&another &base>; dest { bootph-all;
@@ -46,7 +47,7 @@ }; }; - base { + base: base { compatible = "sandbox,i2c"; bootph-all; #address-cells = <1>;
@@ -73,4 +74,15 @@ }; }; }; + + another: another { + new-prop = "hello"; + earlier { + wibble = <2>; + }; + + later { + fibble = <3>; + }; + }; };
tools/dtoc/test_fdt.py +39 -0
@@ -380,6 +380,45 @@ class TestNode(unittest.TestCase): dst = new_dtb.GetNode('/dest') do_copy_checks(new_dtb, dst, expect_none=False) + def test_copy_subnodes_from_phandles(self): + """Test copy_node() function""" + dtb = fdt.FdtScan(find_dtb_file('dtoc_test_copy.dts')) + + orig = dtb.GetNode('/') + node_list = fdt_util.GetPhandleList(orig, 'copy-list') + + dst = dtb.GetNode('/dest') + dst.copy_subnodes_from_phandles(node_list) + + pmic = dtb.GetNode('/dest/over') + self.assertTrue(pmic) + + subn = dtb.GetNode('/dest/first@0') + self.assertTrue(subn) + self.assertEqual({'a-prop', 'b-prop', 'reg'}, subn.props.keys()) + + self.assertEqual( + ['/dest/earlier', '/dest/later', '/dest/over', '/dest/first@0', + '/dest/second', '/dest/existing', '/dest/base'], + [n.path for n in dst.subnodes]) + + # Make sure that the phandle for 'over' is not copied + over = dst.FindNode('over') + print('keys', over.props.keys()) + self.assertNotIn('phandle', over.props.keys()) + + # Check the merged properties, first the base ones in '/dest' + expect = {'bootph-all', 'compatible', 'stringarray', 'longbytearray', + 'maybe-empty-int'} + + # Properties from 'base' + expect.update({'#address-cells', '#size-cells'}) + + # Properties from 'another' + expect.add('new-prop') + + self.assertEqual(expect, set(dst.props.keys())) + class TestProp(unittest.TestCase): """Test operation of the Prop class"""
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 14/20] dtoc: Allow inserting a list of nodes into another
Provide a way to specify a phandle list of nodes which are to be inserted into an existing node.
---
(no changes since v4)
Changes in v4:
- Support copying over properties from each template node
- Make sure phandles are not copied
Changes in v3:
- Adjust to use the new example file
tools/dtoc/fdt.py | 22 +++++++++++++++++ tools/dtoc/test/dtoc_test_copy.dts | 14 ++++++++++- tools/dtoc/test_fdt.py | 39 ++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+), 1 deletion(-)
Applied to u-boot-dm, thanks!
PATCH v5 15/20 binman: Support simple templates
In reply to Simon GlassCollections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
---
(no changes since v4)
Changes in v4:
- Copy over properties from the top-level template node
Changes in v2:
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos
tools/binman/binman.rst | 87 ++++++++++++++++++++++++++++++ tools/binman/control.py | 26 +++++++++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_template.dts | 42 +++++++++++++++ 5 files changed, 165 insertions(+), 1 deletion(-) create mode 100644 tools/binman/test/286_template.dts
tools/binman/binman.rst +87 -0
@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.
@@ -1172,6 +1179,86 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + some-property; + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + footer { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + footer { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +Template nodes appear first in each node that they are inserted into and +ordering of template nodes is preserved. Other nodes come afterwards. If a +template node also appears in the target node, then the template node sets the +order. Thus the template can be used to set the ordering, even if the target +node provides all the properties. In the above example, `fit` and `text` appear +first in the `spi-image` and `mmc-image` images, followed by `footer`. + +Where there are multiple template nodes, they are inserted in that order. so +the first template node appears first, then the second. + +Properties in the template node are inserted into the destination node if they +do not exist there. In the example above, `some-property` is added to each of +`spi-image` and `mmc-image`. + +Note that template nodes are not removed from the binman description at present. + + Updating an ELF file ====================
tools/binman/control.py +26 -0
@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout
@@ -478,6 +479,29 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + """Handle any templates in the binman description + + Args: + parent: Binman node to process (typically /binman) + + Search though each target node looking for those with an 'insert-template' + property. Use that as a list of references to template nodes to use to + adjust the target node. + + Processing involves copying each subnode of the template node into the + target node. + + For now this is not done recursively, so templates must be at the top level + of the binman image. + + See 'Templates' in the Binman documnentation for details. + """ + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree
@@ -520,6 +544,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:
tools/binman/etype/section.py +2 -1
@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""
tools/binman/ftest.py +8 -0
@@ -6785,6 +6785,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_template.dts') + first = U_BOOT_DATA + VGA_DATA + U_BOOT_DTB_DATA + second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()
tools/binman/test/286_template.dts new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 15/20] binman: Support simple templates
Hi Simon
Quoted 256 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
(no changes since v4)
Changes in v4:
- Copy over properties from the top-level template nodeChanges in v2:
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typostools/binman/binman.rst | 87 ++++++++++++++++++++++++++++++ tools/binman/control.py | 26 +++++++++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_template.dts | 42 +++++++++++++++ 5 files changed, 165 insertions(+), 1 deletion(-)create mode 100644
tools/binman/test/286_template.dtstools/binman/binman.rst +87
-0@@ -727,6 +727,13 @@ optional: Note that missing, optional blobs do not produce a non-zero exit code from binman, although it does show a warning about the missing external blob. +insert-template: + This is not strictly speaking an entry property, since it is processed early + in Binman before the entries are read. It is a list of phandles of nodes to + include in the current (target) node. For each node, its subnodes and their + properties are brought into the target node. See Templates_ below for + more information. + The attributes supported for images and sections are described below. Several are similar to those for entries.@@ -1172,6 +1179,86 @@ If you are having trouble figuring out what is going on, you can use arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi" +Templates +========= + +Sometimes multiple images need to be created which have all have a common +part. For example, a board may generate SPI and eMMC images which both include +a FIT. Since the FIT includes many entries, it is tedious to repeat them twice +in the image description. + +Templates provide a simple way to handle this:: + + binman { + multiple-images; + common_part: template-1 { + some-property; + fit { + ... lots of entries in here + }; + + text { + text = "base image"; + }; + }; + + spi-image { + filename = "image-spi.bin"; + insert-template = <&fit>; + + /* things specific to SPI follow */ + footer { + ]; + + text { + text = "SPI image"; + }; + }; + + mmc-image { + filename = "image-mmc.bin"; + insert-template = <&fit>; + + /* things specific to MMC follow */ + footer { + ]; + + text { + text = "MMC image"; + }; + }; + }; + +The template node name must start with 'template', so it is not considered to be +an image itself. + +The mechanism is very simple. For each phandle in the 'insert-templates' +property, the source node is looked up. Then the subnodes of that source node +are copied into the target node, i.e. the one containing the `insert-template` +property. + +If the target node has a node with the same name as a template, its properties +override corresponding properties in the template. This allows the template to +be uses as a base, with the node providing updates to the properties as needed. +The overriding happens recursively. + +Template nodes appear first in each node that they are inserted into and +ordering of template nodes is preserved. Other nodes come afterwards. If a +template node also appears in the target node, then the template node sets the +order. Thus the template can be used to set the ordering, even if the target +node provides all the properties. In the above example, `fit` and `text` appear +first in the `spi-image` and `mmc-image` images, followed by `footer`. + +Where there are multiple template nodes, they are inserted in that order. so +the first template node appears first, then the second. + +Properties in the template node are inserted into the destination node if they +do not exist there. In the example above, `some-property` is added to each of +`spi-image` and `mmc-image`. + +Note that template nodes are not removed from the binman description at present. + + Updating an ELF file ====================tools/binman/control.py +26
-0@@ -22,6 +22,7 @@ from binman import bintool from binman import cbfs_util from binman import elf from binman import entry +from dtoc import fdt_util from u_boot_pylib import command from u_boot_pylib import tools from u_boot_pylib import tout@@ -478,6 +479,29 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths, AfterReplace(image, allow_resize=True, write_map=write_map) +def _ProcessTemplates(parent): + """Handle any templates in the binman description + + Args: + parent: Binman node to process (typically /binman) + + Search though each target node looking for those with an 'insert-template' + property. Use that as a list of references to template nodes to use to + adjust the target node. + + Processing involves copying each subnode of the template node into the + target node. + + For now this is not done recursively, so templates must be at the top level + of the binman image. + + See 'Templates' in the Binman documnentation for details. + """ + for node in parent.subnodes: + tmpl = fdt_util.GetPhandleList(node, 'insert-template') + if tmpl: + node.copy_subnodes_from_phandles(tmpl) + def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree@@ -520,6 +544,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): raise ValueError("Device tree '%s' does not have a 'binman' " "node" % dtb_fname) + _ProcessTemplates(node) + images = _ReadImageDesc(node, use_expanded) if select_images:tools/binman/etype/section.py +2
-1@@ -179,7 +179,8 @@ class Entry_section(Entry): Returns: bool: True if the node is a special one, else False """ - return node.name.startswith('hash') or node.name.startswith('signature') + start_list = ('hash', 'signature', 'template') + return any(node.name.startswith(name) for name in start_list) def ReadNode(self): """Read properties from the section node"""tools/binman/ftest.py +8
-0@@ -6785,6 +6785,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = self._DoReadFileDtb('285_spl_expand.dts', use_expanded=True, entry_args=entry_args)[0] + def testTemplate(self): + """Test using a template""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('286_template.dts') + first = U_BOOT_DATA + VGA_DATA + U_BOOT_DTB_DATA + second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + if __name__ == "__main__": unittest.main()tools/binman/test/286_template.dts new file +32
-0@@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-img { + }; + + common_part: template { + u-boot { + }; + + intel-vga { + filename = "vga.bin"; + }; + }; + + first { + type = "section"; + insert-template = <&common_part>; + + u-boot-dtb { + }; + }; + + second { + type = "section"; + insert-template = <&common_part>;
tools/binman/test/286_template.dts line 42
Quoted 10 lines
tools/binman/test/286_template.dts new file +10
-0+ + u-boot-dtb { + }; + + intel-vga { + filename = "vga2.bin"; + }; + }; + }; +};
While trying to use the templating feature in the K3 boot-loader generation, I seem to face an issue getting a node via phandle. My dts looks something like this:
ti_spl_template: template-1 {
fit {
images {
atf {
ti-secure {
content = <&atf>;
keyfile = "key.pem";
};
atf: atf-bl31 {
};
};
};
};
};ti-spl {
insert-template = <&ti_spl_template>;
fit {
images {
fdt-0 {
ti-secure {
content = <&foo_dtb>;
keyfile = "key.pem";
};
foo_dtb: blob-ext {
filename = "foo.dtb";
};
};
};
};
};The signing in the template node throws the error:
binman: Node '/binman/ti-spl/fit/images/atf/ti-secure': Cannot find entry for node 'atf-bl31'
I understand the phandle is not copied over. And I can work around this by moving the signing contents over to the target node instead of the template. But again, trying for least code duplication here; so any way around?
Signature · 2 lines
Thanking You Neha Malcom Francis
Re: [PATCH v5 15/20] binman: Support simple templates
Hi Neha,
Quoted 33 lines · Neha Malcom Francis
Hi Simon
Quoted 29 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
(no changes since v4)
Changes in v4:
- Copy over properties from the top-level template nodeChanges in v2:
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typostools/binman/binman.rst | 87 ++++++++++++++++++++++++++++++ tools/binman/control.py | 26 +++++++++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_template.dts | 42 +++++++++++++++ 5 files changed, 165 insertions(+), 1 deletion(-)create mode 100644
tools/binman/test/286_template.dts
[..]
Quoted 36 lines
While trying to use the templating feature in the K3 boot-loader generation, I seem to face an issue getting a node via phandle. My dts looks something like this:
ti_spl_template: template-1 { fit { images { atf { ti-secure { content = <&atf>; keyfile = "key.pem"; }; atf: atf-bl31 { }; }; }; }; };ti-spl { insert-template = <&ti_spl_template>; fit { images { fdt-0 { ti-secure { content = <&foo_dtb>; keyfile = "key.pem"; }; foo_dtb: blob-ext { filename = "foo.dtb"; }; }; }; }; };
Quoted 7 lines
The signing in the template node throws the error:
binman: Node '/binman/ti-spl/fit/images/atf/ti-secure': Cannot find entry for node 'atf-bl31'I understand the phandle is not copied over. And I can work around this by moving the signing contents over to the target node instead of the template. But again, trying for least code duplication here; so any way around?
I can repeat this, so will take a look, hopefully tomorrow.
Regards,
Simon
Re: [PATCH v5 15/20] binman: Support simple templates
Hi Neha,
Quoted 80 lines · Simon Glass
Hi Neha,
Quoted 33 lines · Neha Malcom Francis
Hi Simon
Quoted 29 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
(no changes since v4)
Changes in v4:
- Copy over properties from the top-level template nodeChanges in v2:
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typostools/binman/binman.rst | 87 ++++++++++++++++++++++++++++++ tools/binman/control.py | 26 +++++++++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_template.dts | 42 +++++++++++++++ 5 files changed, 165 insertions(+), 1 deletion(-)create mode 100644
tools/binman/test/286_template.dts[..]
Quoted 40 lines
While trying to use the templating feature in the K3 boot-loader generation, I seem to face an issue getting a node via phandle. My dts looks something like this:
ti_spl_template: template-1 { fit { images { atf { ti-secure { content = <&atf>; keyfile = "key.pem"; }; atf: atf-bl31 { }; }; }; }; };ti-spl { insert-template = <&ti_spl_template>; fit { images { fdt-0 { ti-secure { content = <&foo_dtb>; keyfile = "key.pem"; }; foo_dtb: blob-ext { filename = "foo.dtb"; }; }; }; }; };The signing in the template node throws the error:
binman: Node '/binman/ti-spl/fit/images/atf/ti-secure': Cannot find entry for node 'atf-bl31'
Quoted 5 lines
Quoted 3 lines
I understand the phandle is not copied over. And I can work around this by moving the signing contents over to the target node instead of the template. But again, trying for least code duplication here; so any way around?
I can repeat this, so will take a look, hopefully tomorrow.
I have an initial series to deal with this, available at u-boot-dm/bina-working
I will send the patches.
Regards,
Simon
Re: [PATCH v5 15/20] binman: Support simple templates
Hi Neha,
Quoted 80 lines · Simon Glass
Hi Neha,
Quoted 33 lines · Neha Malcom Francis
Hi Simon
Quoted 29 lines · Simon Glass
Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.
It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.
Implement this mechanism.
Signed-off-by Simon Glass <[email protected]>---
(no changes since v4)
Changes in v4:
- Copy over properties from the top-level template nodeChanges in v2:
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typostools/binman/binman.rst | 87 ++++++++++++++++++++++++++++++ tools/binman/control.py | 26 +++++++++ tools/binman/etype/section.py | 3 +- tools/binman/ftest.py | 8 +++ tools/binman/test/286_template.dts | 42 +++++++++++++++ 5 files changed, 165 insertions(+), 1 deletion(-)create mode 100644
tools/binman/test/286_template.dts[..]
Quoted 40 lines
While trying to use the templating feature in the K3 boot-loader generation, I seem to face an issue getting a node via phandle. My dts looks something like this:
ti_spl_template: template-1 { fit { images { atf { ti-secure { content = <&atf>; keyfile = "key.pem"; }; atf: atf-bl31 { }; }; }; }; };ti-spl { insert-template = <&ti_spl_template>; fit { images { fdt-0 { ti-secure { content = <&foo_dtb>; keyfile = "key.pem"; }; foo_dtb: blob-ext { filename = "foo.dtb"; }; }; }; }; };The signing in the template node throws the error:
binman: Node '/binman/ti-spl/fit/images/atf/ti-secure': Cannot find entry for node 'atf-bl31'
Quoted 5 lines
Quoted 3 lines
I understand the phandle is not copied over. And I can work around this by moving the signing contents over to the target node instead of the template. But again, trying for least code duplication here; so any way around?
I can repeat this, so will take a look, hopefully tomorrow.
I have an initial series to deal with this, available at u-boot-dm/bina-working
I will send the patches.
Regards,
Simon
Applied to u-boot-dm, thanks!
PATCH v5 16/20 binman: Support templating with multiple images
In reply to Simon GlassAllow a template to appear in the top level description when using multiple images.
---
(no changes since v3)
Changes in v3:
- Drop duplicate dts-v1 header
tools/binman/control.py | 5 +++-- tools/binman/ftest.py | 12 +++++++++++ tools/binman/test/287_template_multi.dts | 27 ++++++++++++++++++++++++ 3 files changed, 42 insertions(+), 2 deletions(-) create mode 100644 tools/binman/test/287_template_multi.dts
tools/binman/control.py +3 -2
@@ -57,8 +57,9 @@ def _ReadImageDesc(binman_node, use_expanded): images = OrderedDict() if 'multiple-images' in binman_node.props: for node in binman_node.subnodes: - images[node.name] = Image(node.name, node, - use_expanded=use_expanded) + if 'template' not in node.name: + images[node.name] = Image(node.name, node, + use_expanded=use_expanded) else: images['image'] = Image('image', binman_node, use_expanded=use_expanded) return images
tools/binman/ftest.py +12 -0
@@ -6793,6 +6793,18 @@ fdt fdtmap Extract the devicetree blob from the fdtmap second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA self.assertEqual(U_BOOT_IMG_DATA + first + second, data) + def testTemplateBlobMulti(self): + """Test using a template with 'multiple-images' enabled""" + TestFunctional._MakeInputFile('my-blob.bin', b'blob') + TestFunctional._MakeInputFile('my-blob2.bin', b'other') + retcode = self._DoTestFile('287_template_multi.dts') + + self.assertEqual(0, retcode) + image = control.images['image'] + image_fname = tools.get_output_filename('my-image.bin') + data = tools.read_file(image_fname) + self.assertEqual(b'blob@@@@other', data) + if __name__ == "__main__": unittest.main()
tools/binman/test/287_template_multi.dts new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; +/ { + binman: binman { + multiple-images; + + my_template: template { + blob-ext@0 { + filename = "my-blob.bin"; + offset = <0>; + }; + blob-ext@8 { + offset = <8>; + }; + }; + + image { + pad-byte = <0x40>; + filename = "my-image.bin"; + insert-template = <&my_template>; + blob-ext@8 { + filename = "my-blob2.bin"; + }; + }; + }; +};
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 16/20] binman: Support templating with multiple images
Allow a template to appear in the top level description when using multiple images.
---
(no changes since v3)
Changes in v3:
- Drop duplicate dts-v1 header
tools/binman/control.py | 5 +++-- tools/binman/ftest.py | 12 +++++++++++ tools/binman/test/287_template_multi.dts | 27 ++++++++++++++++++++++++ 3 files changed, 42 insertions(+), 2 deletions(-) create mode 100644 tools/binman/test/287_template_multi.dts
Applied to u-boot-dm, thanks!
PATCH v5 17/20 binman: Add a test for templating in a FIT
In reply to Simon GlassAdd this as a separate test case.
---
(no changes since v3)
Changes in v3:
- Add new test case for templating in a FIT
tools/binman/ftest.py | 7 +++++ tools/binman/test/288_template_fit.dts | 37 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) create mode 100644 tools/binman/test/288_template_fit.dts
tools/binman/ftest.py +7 -0
@@ -6805,6 +6805,13 @@ fdt fdtmap Extract the devicetree blob from the fdtmap data = tools.read_file(image_fname) self.assertEqual(b'blob@@@@other', data) + def testTemplateFit(self): + """Test using a template in a FIT""" + fit_data = self._DoReadFile('288_template_fit.dts') + fname = os.path.join(self._indir, 'fit_data.fit') + tools.write_file(fname, fit_data) + out = tools.run('dumpimage', '-l', fname) + if __name__ == "__main__": unittest.main()
tools/binman/test/288_template_fit.dts new file +37 -0
@@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + binman: binman { + multiple-images; + + my_template: template { + fit@0 { + images { + kernel-1 { + }; + kernel-2 { + }; + }; + }; + }; + + image { + filename = "image.bin"; + insert-template = <&my_template>; + + fit@0 { + description = "desc"; + configurations { + }; + images { + kernel-3 { + }; + kernel-4 { + }; + }; + }; + }; + }; +};
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 17/20] binman: Add a test for templating in a FIT
Add this as a separate test case.
---
(no changes since v3)
Changes in v3:
- Add new test case for templating in a FIT
tools/binman/ftest.py | 7 +++++ tools/binman/test/288_template_fit.dts | 37 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) create mode 100644 tools/binman/test/288_template_fit.dts
Applied to u-boot-dm, thanks!
PATCH v5 18/20 binman: Support templates at any level
In reply to Simon GlassAllow templates to be used inside a section, not just in the top-level /binman node.
---
(no changes since v1)
tools/binman/control.py | 5 ++- tools/binman/ftest.py | 8 ++++ tools/binman/test/289_template_section.dts | 52 ++++++++++++++++++++++ 3 files changed, 63 insertions(+), 2 deletions(-) create mode 100644 tools/binman/test/289_template_section.dts
tools/binman/control.py +3 -2
@@ -493,8 +493,8 @@ def _ProcessTemplates(parent): Processing involves copying each subnode of the template node into the target node. - For now this is not done recursively, so templates must be at the top level - of the binman image. + This is done recursively, so templates can be at any level of the binman + image, e.g. inside a section. See 'Templates' in the Binman documnentation for details. """
@@ -502,6 +502,7 @@ def _ProcessTemplates(parent): tmpl = fdt_util.GetPhandleList(node, 'insert-template') if tmpl: node.copy_subnodes_from_phandles(tmpl) + _ProcessTemplates(node) def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded): """Prepare the images to be processed and select the device tree
tools/binman/ftest.py +8 -0
@@ -6812,6 +6812,14 @@ fdt fdtmap Extract the devicetree blob from the fdtmap tools.write_file(fname, fit_data) out = tools.run('dumpimage', '-l', fname) + def testTemplateSection(self): + """Test using a template in a section (not at top level)""" + TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA) + data = self._DoReadFile('289_template_section.dts') + first = U_BOOT_DATA + VGA_DATA + U_BOOT_DTB_DATA + second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA + self.assertEqual(U_BOOT_IMG_DATA + first + second + first, data) + if __name__ == "__main__": unittest.main()
tools/binman/test/289_template_section.dts new file +52 -0
52 lines not on this page · raw message
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 18/20] binman: Support templates at any level
Allow templates to be used inside a section, not just in the top-level /binman node.
---
(no changes since v1)
tools/binman/control.py | 5 ++- tools/binman/ftest.py | 8 ++++ tools/binman/test/289_template_section.dts | 52 ++++++++++++++++++++++ 3 files changed, 63 insertions(+), 2 deletions(-) create mode 100644 tools/binman/test/289_template_section.dts
Applied to u-boot-dm, thanks!
PATCH v5 19/20 binman: Support writing symbols inside a mkimage image
In reply to Simon GlassAdd support for writing symbols and determining the assumed position of binaries inside a mkimage image. This is useful as an example for other entry types which might want to do the same thing.
---
(no changes since v3)
Changes in v3:
- Add new patch to support writing symbols inside a mkimage image
tools/binman/entry.py | 2 - tools/binman/etype/mkimage.py | 36 +++++++++++++++ tools/binman/ftest.py | 64 +++++++++++++++++++++++++++ tools/binman/test/290_mkimage_sym.dts | 27 +++++++++++ 4 files changed, 127 insertions(+), 2 deletions(-) create mode 100644 tools/binman/test/290_mkimage_sym.dts
tools/binman/entry.py +0 -2
@@ -1314,8 +1314,6 @@ features to produce new behaviours. """ data = b'' for entry in entries: - # First get the input data and put it in a file - entry.ObtainContents(fake_size=fake_size) data += entry.GetData() uniq = self.GetUniqueName() fname = tools.get_output_filename(f'{prefix}.{uniq}')
tools/binman/etype/mkimage.py +36 -0
@@ -218,3 +218,39 @@ class Entry_mkimage(Entry_section): def CheckEntries(self): pass + + def ProcessContents(self): + # The blob may have changed due to WriteSymbols() + ok = super().ProcessContents() + data = self.BuildSectionData(True) + ok2 = self.ProcessContentsUpdate(data) + return ok and ok2 + + def SetImagePos(self, image_pos): + """Set the position in the image + + This sets each subentry's offsets, sizes and positions-in-image + according to where they ended up in the packed mkimage file. + + NOTE: This assumes a legacy mkimage and assumes that the images are + written to the output in order. SoC-specific mkimage handling may not + conform to this, in which case these values may be wrong. + + Args: + image_pos (int): Position of this entry in the image + """ + # The mkimage header consists of 0x40 bytes, following by a table of + # offsets for each file + upto = 0x40 + + # Skip the 0-terminated list of offsets (assume a single image) + upto += 4 + 4 + for entry in self.GetEntries().values(): + entry.SetOffsetSize(upto, None) + + # Give up if any entries lack a size + if entry.size is None: + return + upto += entry.size + + super().SetImagePos(image_pos)
tools/binman/ftest.py +64 -0
70 lines not on this page · raw message
tools/binman/test/290_mkimage_sym.dts new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0+ + +/dts-v1/; + +/ { + #address-cells = <1>; + #size-cells = <1>; + + binman { + u-boot-dtb { + }; + + mkimage { + args = "-n test -T script"; + + u-boot-spl { + }; + + u-boot-spl2 { + type = "u-boot-spl"; + }; + }; + + u-boot { + }; + }; +};
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 19/20] binman: Support writing symbols inside a mkimage image
Add support for writing symbols and determining the assumed position of binaries inside a mkimage image. This is useful as an example for other entry types which might want to do the same thing.
---
(no changes since v3)
Changes in v3:
- Add new patch to support writing symbols inside a mkimage image
tools/binman/entry.py | 2 - tools/binman/etype/mkimage.py | 36 +++++++++++++++ tools/binman/ftest.py | 64 +++++++++++++++++++++++++++ tools/binman/test/290_mkimage_sym.dts | 27 +++++++++++ 4 files changed, 127 insertions(+), 2 deletions(-) create mode 100644 tools/binman/test/290_mkimage_sym.dts
Applied to u-boot-dm, thanks!
PATCH v5 20/20 binman: Reduce state.SetInt and bintool cmd to debug level
In reply to Simon GlassThese are not very important message. Change them to use the 'debug' level instead of 'detail'.
---
(no changes since v4)
Changes in v4:
- Add new patch to reduce state.SetInt and bintool cmd to debug level
tools/binman/bintool.py | 2 +- tools/binman/state.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
tools/binman/bintool.py +1 -1
@@ -288,7 +288,7 @@ class Bintool: name = os.path.expanduser(self.name) # Expand paths containing ~ all_args = (name,) + args env = tools.get_env_with_path() - tout.detail(f"bintool: {' '.join(all_args)}") + tout.debug(f"bintool: {' '.join(all_args)}") result = command.run_pipe( [all_args], capture=True, capture_stderr=True, env=env, raise_on_error=False, binary=binary)
tools/binman/state.py +2 -2
@@ -385,8 +385,8 @@ def SetInt(node, prop, value, for_repack=False): for_repack: True is this property is only needed for repacking """ for n in GetUpdateNodes(node, for_repack): - tout.detail("File %s: Update node '%s' prop '%s' to %#x" % - (n.GetFdt().name, n.path, prop, value)) + tout.debug("File %s: Update node '%s' prop '%s' to %#x" % + (n.GetFdt().name, n.path, prop, value)) n.SetInt(prop, value) def CheckAddHashProp(node):
Signature · 1 line
2.41.0.455.g037347b96a-goog
Re: [PATCH v5 20/20] binman: Reduce state.SetInt and bintool cmd to debug level
These are not very important message. Change them to use the 'debug' level instead of 'detail'.
---
(no changes since v4)
Changes in v4:
- Add new patch to reduce state.SetInt and bintool cmd to debug level
tools/binman/bintool.py | 2 +- tools/binman/state.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
Applied to u-boot-dm, thanks!