Skip to contentSkip to conversation

binman: Simple templating feature and mkimage conversion

PATCH Superseded Simon Glass 3 participants 27 messages Jul 10, 2023 – Jul 11, 2023

Subsystems (2)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover binman: Simple templating feature and mkimage conversion

This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.

A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.

In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.

Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image

Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

Marek Vasut (1):
binman: Convert mkimage to Entry_section

Simon Glass (18):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image

 arch/arm/dts/stm32mp15-u-boot.dtsi         |   1 +
 tools/binman/binman.rst                    |  89 +++++++++
 tools/binman/control.py                    |  34 +++-
 tools/binman/elf.py                        |  13 +-
 tools/binman/elf_test.py                   |  13 +-
 tools/binman/entries.rst                   |   6 +
 tools/binman/entry.py                      |  11 +-
 tools/binman/etype/blob_phase.py           |   5 +
 tools/binman/etype/fit.py                  |   9 +
 tools/binman/etype/mkimage.py              | 110 ++++++++---
 tools/binman/etype/section.py              |  54 +++--
 tools/binman/etype/u_boot_spl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py   |   2 +-
 tools/binman/ftest.py                      | 218 ++++++++++++++++++++-
 tools/binman/test/282_symbols_disable.dts  |  25 +++
 tools/binman/test/283_mkimage_special.dts  |  24 +++
 tools/binman/test/284_fit_fdt_list.dts     |  58 ++++++
 tools/binman/test/285_spl_expand.dts       |  13 ++
 tools/binman/test/286_template.dts         |  42 ++++
 tools/binman/test/287_template_multi.dts   |  27 +++
 tools/binman/test/288_template_fit.dts     |  37 ++++
 tools/binman/test/289_template_section.dts |  52 +++++
 tools/binman/test/290_mkimage_sym.dts      |  27 +++
 tools/binman/test/Makefile                 |   5 +-
 tools/binman/test/bss_data.c               |   3 +-
 tools/binman/test/bss_data_zero.c          |  16 ++
 tools/binman/test/bss_data_zero.lds        |  15 ++
 tools/binman/test/embed_data.lds           |   1 +
 tools/dtoc/fdt.py                          | 122 +++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts         |  86 ++++++++
 tools/dtoc/test_fdt.py                     |  93 +++++++++
 32 files changed, 1147 insertions(+), 68 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
 create mode 100644 tools/binman/test/283_mkimage_special.dts
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
 create mode 100644 tools/binman/test/285_spl_expand.dts
 create mode 100644 tools/binman/test/286_template.dts
 create mode 100644 tools/binman/test/287_template_multi.dts
 create mode 100644 tools/binman/test/288_template_fit.dts
 create mode 100644 tools/binman/test/289_template_section.dts
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#1 lore raw

PATCH v3 1/19 binman: Correct coverage gap in control

Add a pragma to deal with the code-coverage gap which drops binman down to 90% coverage.

Fixes de65b122a25 (tools: Fall back to importlib_resources on Python 3.6)

---

(no changes since v1)

 tools/binman/control.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/binman/control.py +1 -1
index 68597c4e7792..7e2dd3541b96 100644
@@ -9,7 +9,7 @@ from collections import OrderedDict
 import glob
 try:
     import importlib.resources
-except ImportError:
+except ImportError:  # pragma: no cover
     # for Python 3.6
     import importlib_resources
 import os
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#2 lore raw

PATCH v3 2/19 binman: Init align_default in entry_Section

In reply to Simon Glass

This should be set up in the init function, to avoid a warning about a property not set up there. Fix it.

---

(no changes since v1)

 tools/binman/etype/section.py | 1 +
 1 file changed, 1 insertion(+)
tools/binman/etype/section.py +1 -0
index c36edd13508b..77250a7525c6 100644
@@ -168,6 +168,7 @@ class Entry_section(Entry):
         self._end_4gb = False
         self._ignore_missing = False
         self._filename = None
+        self.align_default = 0
 
     def IsSpecialSubnode(self, node):
         """Check if a node is a special one used by the section itself
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#3 lore raw

PATCH v3 3/19 binman: Use GetEntries() to obtain section contents

In reply to Simon Glass

Some section types don't have a simple _entries list. Use the GetEntries() method in GetEntryContents() and other places to handle this.

This makes the behaviour more consistent.

---

(no changes since v1)

 tools/binman/etype/section.py | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
tools/binman/etype/section.py +9 -9
index 77250a7525c6..d56cc11d1023 100644
@@ -720,7 +720,7 @@ class Entry_section(Entry):
                     next_todo.append(entry)
             return entry
 
-        todo = self._entries.values()
+        todo = self.GetEntries().values()
         for passnum in range(3):
             threads = state.GetThreads()
             next_todo = []
@@ -893,7 +893,7 @@ class Entry_section(Entry):
             allow_missing: True if allowed, False if not allowed
         """
         self.allow_missing = allow_missing
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.SetAllowMissing(allow_missing)
 
     def SetAllowFakeBlob(self, allow_fake):
@@ -903,7 +903,7 @@ class Entry_section(Entry):
             allow_fake: True if allowed, False if not allowed
         """
         super().SetAllowFakeBlob(allow_fake)
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.SetAllowFakeBlob(allow_fake)
 
     def CheckMissing(self, missing_list):
@@ -915,7 +915,7 @@ class Entry_section(Entry):
         Args:
             missing_list: List of Entry objects to be added to
         """
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.CheckMissing(missing_list)
 
     def CheckFakedBlobs(self, faked_blobs_list):
@@ -926,7 +926,7 @@ class Entry_section(Entry):
         Args:
             faked_blobs_list: List of Entry objects to be added to
         """
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.CheckFakedBlobs(faked_blobs_list)
 
     def CheckOptional(self, optional_list):
@@ -937,7 +937,7 @@ class Entry_section(Entry):
         Args:
             optional_list (list): List of Entry objects to be added to
         """
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.CheckOptional(optional_list)
 
     def check_missing_bintools(self, missing_list):
@@ -949,7 +949,7 @@ class Entry_section(Entry):
             missing_list: List of Bintool objects to be added to
         """
         super().check_missing_bintools(missing_list)
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.check_missing_bintools(missing_list)
 
     def _CollectEntries(self, entries, entries_by_name, add_entry):
@@ -999,12 +999,12 @@ class Entry_section(Entry):
             entry.Raise(f'Missing required properties/entry args: {missing}')
 
     def CheckAltFormats(self, alt_formats):
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.CheckAltFormats(alt_formats)
 
     def AddBintools(self, btools):
         super().AddBintools(btools)
-        for entry in self._entries.values():
+        for entry in self.GetEntries().values():
             entry.AddBintools(btools)
 
     def read_elf_segments(self):
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#4 lore raw

PATCH v3 4/19 binman: Read _multiple_data_files in the correct place

In reply to Simon Glass

Move this to the ReadEntries() function where it belongs.

---

(no changes since v1)

 tools/binman/etype/mkimage.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/binman/etype/mkimage.py +2 -1
index e028c4407081..cb3e10672ad7 100644
@@ -121,7 +121,6 @@ class Entry_mkimage(Entry):
     """
     def __init__(self, section, etype, node):
         super().__init__(section, etype, node)
-        self._multiple_data_files = fdt_util.GetBool(self._node, 'multiple-data-files')
         self._mkimage_entries = OrderedDict()
         self._imagename = None
         self._filename = fdt_util.GetString(self._node, 'filename')
@@ -129,6 +128,8 @@ class Entry_mkimage(Entry):
 
     def ReadNode(self):
         super().ReadNode()
+        self._multiple_data_files = fdt_util.GetBool(self._node,
+                                                     'multiple-data-files')
         self._args = fdt_util.GetArgs(self._node, 'args')
         self._data_to_imagename = fdt_util.GetBool(self._node,
                                                    'data-to-imagename')
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#5 lore raw

PATCH v3 5/19 binman: Allow disabling symbol writing

In reply to Simon Glass

Some boards don't use symbol writing but do access the symbols in SPL. Provide an option to work around this.

---

(no changes since v1)

 tools/binman/binman.rst                   |  7 ++++++
 tools/binman/entry.py                     |  4 +++-
 tools/binman/etype/blob_phase.py          |  5 ++++
 tools/binman/ftest.py                     | 28 +++++++++++++++++++----
 tools/binman/test/282_symbols_disable.dts | 25 ++++++++++++++++++++
 5 files changed, 64 insertions(+), 5 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
tools/binman/binman.rst +7 -0
index 23cbb99b4b0b..a4b31fe5397b 100644
@@ -831,6 +831,13 @@ write-symbols:
     binman. This is automatic for certain entry types, e.g. `u-boot-spl`. See
     binman_syms_ for more information.
 
+no-write-symbols:
+    Disables symbol writing for this entry. This can be used in entry types
+    where symbol writing is automatic. For example, if `u-boot-spl` refers to
+    the `u_boot_any_image_pos` symbol but U-Boot is not available in the image
+    containing SPL, this can be used to disable the writing. Quite likely this
+    indicates a bug in your setup.
+
 elf-filename:
     Sets the file name of a blob's associated ELF file. For example, if the
     blob is `zephyr.bin` then the ELF file may be `zephyr.elf`. This allows
tools/binman/entry.py +3 -1
index 39456906a477..328b5bc568a9 100644
@@ -158,6 +158,7 @@ class Entry(object):
         self.offset_from_elf = None
         self.preserve = False
         self.build_done = False
+        self.no_write_symbols = False
 
     @staticmethod
     def FindEntryClass(etype, expanded):
@@ -321,6 +322,7 @@ class Entry(object):
                                                              'offset-from-elf')
 
         self.preserve = fdt_util.GetBool(self._node, 'preserve')
+        self.no_write_symbols = fdt_util.GetBool(self._node, 'no-write-symbols')
 
     def GetDefaultFilename(self):
         return None
@@ -695,7 +697,7 @@ class Entry(object):
         Args:
           section: Section containing the entry
         """
-        if self.auto_write_symbols:
+        if self.auto_write_symbols and not self.no_write_symbols:
             # Check if we are writing symbols into an ELF file
             is_elf = self.GetDefaultFilename() == self.elf_fname
             elf.LookupAndWriteSymbols(self.elf_fname, self, section.GetImage(),
tools/binman/etype/blob_phase.py +5 -0
index b937158756fd..951d9934050e 100644
@@ -52,3 +52,8 @@ class Entry_blob_phase(Entry_section):
 
         # Read entries again, now that we have some
         self.ReadEntries()
+
+        # Propagate the no-write-symbols property
+        if self.no_write_symbols:
+            for entry in self._entries.values():
+                entry.no_write_symbols = True
tools/binman/ftest.py +24 -4
index 43b4f850a691..dabb3f689fdb 100644
@@ -1452,7 +1452,7 @@ class TestFunctional(unittest.TestCase):
         self.assertEqual(U_BOOT_SPL_NODTB_DATA, data[:len(U_BOOT_SPL_NODTB_DATA)])
 
     def checkSymbols(self, dts, base_data, u_boot_offset, entry_args=None,
-                     use_expanded=False):
+                     use_expanded=False, no_write_symbols=False):
         """Check the image contains the expected symbol values
 
         Args:
@@ -1481,9 +1481,14 @@ class TestFunctional(unittest.TestCase):
         sym_values = struct.pack('<LLQLL', elf.BINMAN_SYM_MAGIC_VALUE,
                                  0x00, u_boot_offset + len(U_BOOT_DATA),
                                  0x10 + u_boot_offset, 0x04)
-        expected = (sym_values + base_data[24:] +
-                    tools.get_bytes(0xff, 1) + U_BOOT_DATA + sym_values +
-                    base_data[24:])
+        if no_write_symbols:
+            expected = (base_data +
+                        tools.get_bytes(0xff, 0x38 - len(base_data)) +
+                        U_BOOT_DATA + base_data)
+        else:
+            expected = (sym_values + base_data[24:] +
+                        tools.get_bytes(0xff, 1) + U_BOOT_DATA + sym_values +
+                        base_data[24:])
         self.assertEqual(expected, data)
 
     def testSymbols(self):
@@ -6676,6 +6681,21 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
                                 ['fit'])
         self.assertIn("Node '/fit': Missing tool: 'mkimage'", str(e.exception))
 
+    def testSymbolNoWrite(self):
+        """Test disabling of symbol writing"""
+        self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_DATA, 0x1c,
+                          no_write_symbols=True)
+
+    def testSymbolNoWriteExpanded(self):
+        """Test disabling of symbol writing in expanded entries"""
+        entry_args = {
+            'spl-dtb': '1',
+        }
+        self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_NODTB_DATA +
+                          U_BOOT_SPL_DTB_DATA, 0x38,
+                          entry_args=entry_args, use_expanded=True,
+                          no_write_symbols=True)
+
 
 if __name__ == "__main__":
     unittest.main()
tools/binman/test/282_symbols_disable.dts new file +25 -0
new file mode 100644
index 000000000000..6efa9335041b
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	#size-cells = <1>;
+
+	binman {
+		pad-byte = <0xff>;
+		u-boot-spl {
+			no-write-symbols;
+		};
+
+		u-boot {
+			offset = <0x38>;
+			no-expanded;
+		};
+
+		u-boot-spl2 {
+			type = "u-boot-spl";
+			no-write-symbols;
+		};
+	};
+};
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#6 lore raw

PATCH v3 6/19 stm32mp15: Avoid writing symbols in SPL

In reply to Simon Glass

These boards use SPL in a mkimage entry and apparently access the symbol containing the image position of U-Boot, but put U-Boot in another image. This means that binman is unable to fill in the symbol correctly in the SPL binary.

This doesn't matter at present since mkimage doesn't support symbol writing. But with the upcoming conversion to a section, it will. So add a property to disable symbol writing.

---

(no changes since v1)

 arch/arm/dts/stm32mp15-u-boot.dtsi | 1 +
 1 file changed, 1 insertion(+)
arch/arm/dts/stm32mp15-u-boot.dtsi +1 -0
index d872c6fc5679..573dd4d3ed56 100644
@@ -226,6 +226,7 @@
 		mkimage {
 			args = "-T stm32image -a 0x2ffc2500 -e 0x2ffc2500";
 			u-boot-spl {
+				no-write-symbols;
 			};
 		};
 	};
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#7 lore raw

PATCH v3 7/19 binman: Update elf to return number of written symbols

In reply to Simon Glass

Update the LookupAndWriteSymbols() function to return the number of symbols written. Also add some logging for when debugging is not enabled.

---

Changes in v3:
- Add new patch for elf to return number of written symbols

 tools/binman/elf.py      | 13 +++++++++++--
 tools/binman/elf_test.py |  8 +++++---
 2 files changed, 16 insertions(+), 5 deletions(-)
tools/binman/elf.py +11 -2
index 5816284c32a2..4219001feac3 100644
@@ -248,6 +248,9 @@ def LookupAndWriteSymbols(elf_fname, entry, section, is_elf=False,
         entry: Entry to process
         section: Section which can be used to lookup symbol values
         base_sym: Base symbol marking the start of the image
+
+    Returns:
+        int: Number of symbols written
     """
     if not base_sym:
         base_sym = '__image_copy_start'
@@ -269,12 +272,13 @@ def LookupAndWriteSymbols(elf_fname, entry, section, is_elf=False,
 
     if not syms:
         tout.debug('LookupAndWriteSymbols: no syms')
-        return
+        return 0
     base = syms.get(base_sym)
     if not base and not is_elf:
         tout.debug('LookupAndWriteSymbols: no base')
-        return
+        return 0
     base_addr = 0 if is_elf else base.address
+    count = 0
     for name, sym in syms.items():
         if name.startswith('_binman'):
             msg = ("Section '%s': Symbol '%s'\n   in entry '%s'" %
@@ -307,6 +311,11 @@ def LookupAndWriteSymbols(elf_fname, entry, section, is_elf=False,
                        (msg, name, offset, value, len(value_bytes)))
             entry.data = (entry.data[:offset] + value_bytes +
                         entry.data[offset + sym.size:])
+            count += 1
+    if count:
+        tout.detail(
+            f"Section '{section.GetPath()}': entry '{entry.GetPath()}' : {count} symbols")
+    return count
 
 def GetSymbolValue(sym, data, msg):
     """Get the value of a symbol
tools/binman/elf_test.py +5 -3
index c98083961b53..2fb3f6f28ff1 100644
@@ -141,7 +141,8 @@ class TestElf(unittest.TestCase):
         entry = FakeEntry(10)
         section = FakeSection()
         elf_fname = self.ElfTestFile('u_boot_binman_syms_bad')
-        elf.LookupAndWriteSymbols(elf_fname, entry, section)
+        count = elf.LookupAndWriteSymbols(elf_fname, entry, section)
+        self.assertEqual(0, count)
 
     def testBadSymbolSize(self):
         """Test that an attempt to use an 8-bit symbol are detected
@@ -162,7 +163,7 @@ class TestElf(unittest.TestCase):
     def testNoValue(self):
         """Test the case where we have no value for the symbol
 
-        This should produce -1 values for all thress symbols, taking up the
+        This should produce -1 values for all three symbols, taking up the
         first 16 bytes of the image.
         """
         if not elf.ELF_TOOLS:
@@ -170,7 +171,8 @@ class TestElf(unittest.TestCase):
         entry = FakeEntry(28)
         section = FakeSection(sym_value=None)
         elf_fname = self.ElfTestFile('u_boot_binman_syms')
-        elf.LookupAndWriteSymbols(elf_fname, entry, section)
+        count = elf.LookupAndWriteSymbols(elf_fname, entry, section)
+        self.assertEqual(5, count)
         expected = (struct.pack('<L', elf.BINMAN_SYM_MAGIC_VALUE) +
                     tools.get_bytes(255, 20) +
                     tools.get_bytes(ord('a'), 4))
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#8 lore raw

PATCH v3 8/19 binman: Add more detail on how ObtainContents() works

In reply to Simon Glass

This area of binman can be a bit confusing. Add some more comments to help.

---

Changes in v3:
- Add new patch with more detail on how ObtainContents() works

 tools/binman/entry.py         |  3 +++
 tools/binman/etype/section.py | 32 +++++++++++++++++++++++++++++++-
 2 files changed, 34 insertions(+), 1 deletion(-)
tools/binman/entry.py +3 -0
index 328b5bc568a9..f20f32213a9b 100644
@@ -474,6 +474,9 @@ class Entry(object):
     def ObtainContents(self, skip_entry=None, fake_size=0):
         """Figure out the contents of an entry.
 
+        For missing blobs (where allow-missing is enabled), the contents are set
+        to b'' and self.missing is set to True.
+
         Args:
             skip_entry (Entry): Entry to skip when obtaining section contents
             fake_size (int): Size of fake file to create if needed
tools/binman/etype/section.py +31 -1
index d56cc11d1023..d9b9e428024a 100644
@@ -316,12 +316,15 @@ class Entry_section(Entry):
         This should be overridden by subclasses which want to build their own
         data structure for the section.
 
+        Missing entries will have be given empty (or fake) data, so are
+        processed normally here.
+
         Args:
             required: True if the data must be present, False if it is OK to
                 return None
 
         Returns:
-            Contents of the section (bytes)
+            Contents of the section (bytes), None if not available
         """
         section_data = bytearray()
 
@@ -711,6 +714,33 @@ class Entry_section(Entry):
     def GetEntryContents(self, skip_entry=None):
         """Call ObtainContents() for each entry in the section
 
+        The overall goal of this function is to read in any available data in
+        this entry and any subentries. This includes reading in blobs, setting
+        up objects which have predefined contents, etc.
+
+        Since entry types which contain entries call ObtainContents() on all
+        those entries too, the result is that ObtainContents() is called
+        recursively for the whole tree below this one.
+
+        Entries with subentries are generally not *themselves& processed here,
+        i.e. their ObtainContents() implementation simply obtains contents of
+        their subentries, skipping their own contents. For example, the
+        implementation here (for entry_Section) does not attempt to pack the
+        entries into a final result. That is handled later.
+
+        Generally, calling this results in SetContents() being called for each
+        entry, so that the 'data' and 'contents_size; properties are set, and
+        subsequent calls to GetData() will return value data.
+
+        Where 'allow_missing' is set, this can result in the 'missing' property
+        being set to True if there is no data. This is handled by setting the
+        data to b''. This function will still return success. Future calls to
+        GetData() for this entry will return b'', or in the case where the data
+        is faked, GetData() will return that fake data.
+
+        Args:
+            skip_entry: (single) Entry to skip, or None to process all entries
+
         Note that this may set entry.absent to True if the entry is not
         actually needed
         """
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#9 lore raw

PATCH v3 9/19 binman: Convert mkimage to Entry_section

In reply to Simon Glass

From: Marek Vasut <[email protected]>

This is needed to handle mkimage with inner section located itself in a section.

Use BuildSectionData() instead of ObtainContents(), add tests and a few other minor fixes:

---

Changes in v3:
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed

Changes in v2:
- Drop now-unnecessary methods in mkimage etype

 tools/binman/entry.py                     |  6 +-
 tools/binman/etype/mkimage.py             | 71 ++++++++++++++---------
 tools/binman/ftest.py                     | 69 +++++++++++++++++++++-
 tools/binman/test/283_mkimage_special.dts | 24 ++++++++
 4 files changed, 135 insertions(+), 35 deletions(-)
 create mode 100644 tools/binman/test/283_mkimage_special.dts
tools/binman/entry.py +2 -4
index f20f32213a9b..0d4cb94f7002 100644
@@ -1314,10 +1314,8 @@ features to produce new behaviours.
         """
         data = b''
         for entry in entries:
-            # First get the input data and put it in a file. If not available,
-            # try later.
-            if not entry.ObtainContents(fake_size=fake_size):
-                return None, None, None
+            # First get the input data and put it in a file
+            entry.ObtainContents(fake_size=fake_size)
             data += entry.GetData()
         uniq = self.GetUniqueName()
         fname = tools.get_output_filename(f'{prefix}.{uniq}')
tools/binman/etype/mkimage.py +42 -29

159 lines not on this page · raw message

tools/binman/ftest.py +67 -2

214 lines not on this page · raw message

tools/binman/test/283_mkimage_special.dts new file +24 -0
new file mode 100644
index 000000000000..c234093e6ece
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	#size-cells = <1>;
+
+	binman {
+		mkimage {
+			args = "-T script";
+
+			u-boot {
+			};
+
+			hash {
+			};
+
+			imagename {
+				type = "u-boot";
+			};
+		};
+	};
+};
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#10 lore raw

Re: [PATCH v3 09/19] binman: Convert mkimage to Entry_section

Hi Simon,

Quoted 9 lines · Simon Glass

From: Marek Vasut <[email protected]>

This is needed to handle mkimage with inner section located itself in a section.

Use BuildSectionData() instead of ObtainContents(), add tests and a few other minor fixes:

Quoted 8 lines

---

Changes in v3:
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed

Changes in v2:
- Drop now-unnecessary methods in mkimage etype

Looks like this change never made it into the patch?

The following functions are still in this file and can now be removed:
- SetAllowMissing
- SetAllowFakeBlob
- CheckMissing
- CheckFakedBlobs

Quoted 429 lines
 tools/binman/entry.py                     |  6 +-
 tools/binman/etype/mkimage.py             | 71 ++++++++++++++---------
 tools/binman/ftest.py                     | 69 +++++++++++++++++++++-
 tools/binman/test/283_mkimage_special.dts | 24 ++++++++
 4 files changed, 135 insertions(+), 35 deletions(-)
 create mode 100644 tools/binman/test/283_mkimage_special.dts
tools/binman/entry.py +2 -4
index f20f32213a9b..0d4cb94f7002 100644
@@ -1314,10 +1314,8 @@ features to produce new behaviours.
         """
         data = b''
         for entry in entries:
-            # First get the input data and put it in a file. If not available,
-            # try later.
-            if not entry.ObtainContents(fake_size=fake_size):
-                return None, None, None
+            # First get the input data and put it in a file
+            entry.ObtainContents(fake_size=fake_size)
             data += entry.GetData()
         uniq = self.GetUniqueName()
         fname = tools.get_output_filename(f'{prefix}.{uniq}')
tools/binman/etype/mkimage.py +42 -29
index cb3e10672ad7..493761da59f5 100644
@@ -8,10 +8,11 @@
 from collections import OrderedDict
 
 from binman.entry import Entry
+from binman.etype.section import Entry_section
 from dtoc import fdt_util
 from u_boot_pylib import tools
 
-class Entry_mkimage(Entry):
+class Entry_mkimage(Entry_section):
     """Binary produced by mkimage
 
     Properties / Entry arguments:
@@ -121,10 +122,8 @@ class Entry_mkimage(Entry):
     """
     def __init__(self, section, etype, node):
         super().__init__(section, etype, node)
-        self._mkimage_entries = OrderedDict()
         self._imagename = None
-        self._filename = fdt_util.GetString(self._node, 'filename')
-        self.align_default = None
+        self._multiple_data_files = False
 
     def ReadNode(self):
         super().ReadNode()
@@ -135,41 +134,55 @@ class Entry_mkimage(Entry):
                                                    'data-to-imagename')
         if self._data_to_imagename and self._node.FindNode('imagename'):
             self.Raise('Cannot use both imagename node and data-to-imagename')
-        self.ReadEntries()
 
     def ReadEntries(self):
         """Read the subnodes to find out what should go in this image"""
         for node in self._node.subnodes:
-            entry = Entry.Create(self, node)
+            if self.IsSpecialSubnode(node):
+                continue
+            entry = Entry.Create(self, node,
+                                 expanded=self.GetImage().use_expanded,
+                                 missing_etype=self.GetImage().missing_etype)
             entry.ReadNode()
+            entry.SetPrefix(self._name_prefix)
             if entry.name == 'imagename':
                 self._imagename = entry
             else:
-                self._mkimage_entries[entry.name] = entry
+                self._entries[entry.name] = entry
 
-    def ObtainContents(self):
+    def BuildSectionData(self, required):
+        """Build mkimage entry contents
+
+        Runs mkimage to build the entry contents
+
+        Args:
+            required (bool): True if the data must be present, False if it is OK
+                to return None
+
+        Returns:
+            bytes: Contents of the section
+        """
         # Use a non-zero size for any fake files to keep mkimage happy
         # Note that testMkimageImagename() relies on this 'mkimage' parameter
         fake_size = 1024
         if self._multiple_data_files:
             fnames = []
             uniq = self.GetUniqueName()
-            for entry in self._mkimage_entries.values():
-                if not entry.ObtainContents(fake_size=fake_size):
-                    return False
-                if entry._pathname:
-                    fnames.append(entry._pathname)
+            for entry in self._entries.values():
+                # Put the contents in a temporary file
+                ename = f'mkimage-in-{uniq}-{entry.name}'
+                fname = tools.get_output_filename(ename)
+                data = entry.GetData(required)
+                tools.write_file(fname, data)
+                fnames.append(fname)
             input_fname = ":".join(fnames)
+            data = b''
         else:
             data, input_fname, uniq = self.collect_contents_to_file(
-                self._mkimage_entries.values(), 'mkimage', fake_size)
-            if data is None:
-                return False
+                self._entries.values(), 'mkimage', fake_size)
         if self._imagename:
             image_data, imagename_fname, _ = self.collect_contents_to_file(
                 [self._imagename], 'mkimage-n', 1024)
-            if image_data is None:
-                return False
         outfile = self._filename if self._filename else 'mkimage-out.%s' % uniq
         output_fname = tools.get_output_filename(outfile)
 
@@ -177,8 +190,7 @@ class Entry_mkimage(Entry):
         self.CheckMissing(missing_list)
         self.missing = bool(missing_list)
         if self.missing:
-            self.SetContents(b'')
-            return self.allow_missing
+            return b''
 
         args = ['-d', input_fname]
         if self._data_to_imagename:
@@ -187,17 +199,15 @@ class Entry_mkimage(Entry):
             args += ['-n', imagename_fname]
         args += self._args + [output_fname]
         if self.mkimage.run_cmd(*args) is not None:
-            self.SetContents(tools.read_file(output_fname))
+            return tools.read_file(output_fname)
         else:
             # Bintool is missing; just use the input data as the output
             self.record_missing_bintool(self.mkimage)
-            self.SetContents(data)
-
-        return True
+            return data
 
     def GetEntries(self):
         # Make a copy so we don't change the original
-        entries = OrderedDict(self._mkimage_entries)
+        entries = OrderedDict(self._entries)
         if self._imagename:
             entries['imagename'] = self._imagename
         return entries
@@ -209,7 +219,7 @@ class Entry_mkimage(Entry):
             allow_missing: True if allowed, False if not allowed
         """
         self.allow_missing = allow_missing
-        for entry in self._mkimage_entries.values():
+        for entry in self._entries.values():
             entry.SetAllowMissing(allow_missing)
         if self._imagename:
             self._imagename.SetAllowMissing(allow_missing)
@@ -220,7 +230,7 @@ class Entry_mkimage(Entry):
         Args:
             allow_fake: True if allowed, False if not allowed
         """
-        for entry in self._mkimage_entries.values():
+        for entry in self._entries.values():
             entry.SetAllowFakeBlob(allow_fake)
         if self._imagename:
             self._imagename.SetAllowFakeBlob(allow_fake)
@@ -234,7 +244,7 @@ class Entry_mkimage(Entry):
         Args:
             missing_list: List of Entry objects to be added to
         """
-        for entry in self._mkimage_entries.values():
+        for entry in self._entries.values():
             entry.CheckMissing(missing_list)
         if self._imagename:
             self._imagename.CheckMissing(missing_list)
@@ -247,7 +257,7 @@ class Entry_mkimage(Entry):
         Args:
             faked_blobs_list: List of Entry objects to be added to
         """
-        for entry in self._mkimage_entries.values():
+        for entry in self._entries.values():
             entry.CheckFakedBlobs(faked_blobs_list)
         if self._imagename:
             self._imagename.CheckFakedBlobs(faked_blobs_list)
@@ -255,3 +265,6 @@ class Entry_mkimage(Entry):
     def AddBintools(self, btools):
         super().AddBintools(btools)
         self.mkimage = self.AddBintool(btools, 'mkimage')
+
+    def CheckEntries(self):
+        pass
tools/binman/ftest.py +59 -1
index dabb3f689fdb..60e69443c400 100644
@@ -1103,6 +1103,7 @@ class TestFunctional(unittest.TestCase):
 
     def testPackZeroOffset(self):
         """Test that an entry at offset 0 is not given a new offset"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as e:
             self._DoTestFile('025_pack_zero_size.dts')
         self.assertIn("Node '/binman/u-boot-spl': Offset 0x0 (0) overlaps "
@@ -1116,6 +1117,7 @@ class TestFunctional(unittest.TestCase):
 
     def testPackX86RomNoSize(self):
         """Test that the end-at-4gb property requires a size property"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as e:
             self._DoTestFile('027_pack_4gb_no_size.dts')
         self.assertIn("Image '/binman': Section size must be provided when "
@@ -1124,6 +1126,7 @@ class TestFunctional(unittest.TestCase):
     def test4gbAndSkipAtStartTogether(self):
         """Test that the end-at-4gb and skip-at-size property can't be used
         together"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as e:
             self._DoTestFile('098_4gb_and_skip_at_start_together.dts')
         self.assertIn("Image '/binman': Provide either 'end-at-4gb' or "
@@ -1131,6 +1134,7 @@ class TestFunctional(unittest.TestCase):
 
     def testPackX86RomOutside(self):
         """Test that the end-at-4gb property checks for offset boundaries"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as e:
             self._DoTestFile('028_pack_4gb_outside.dts')
         self.assertIn("Node '/binman/u-boot': Offset 0x0 (0) size 0x4 (4) "
@@ -1423,6 +1427,7 @@ class TestFunctional(unittest.TestCase):
 
     def testPackUbootSplMicrocode(self):
         """Test that x86 microcode can be handled correctly in SPL"""
+        self._SetupSplElf()
         self._PackUbootSplMicrocode('049_x86_ucode_spl.dts')
 
     def testPackUbootSplMicrocodeReorder(self):
@@ -1442,6 +1447,7 @@ class TestFunctional(unittest.TestCase):
 
     def testSplDtb(self):
         """Test that an image with spl/u-boot-spl.dtb can be created"""
+        self._SetupSplElf()
         data = self._DoReadFile('051_u_boot_spl_dtb.dts')
         self.assertEqual(U_BOOT_SPL_DTB_DATA, data[:len(U_BOOT_SPL_DTB_DATA)])
 
@@ -1962,6 +1968,8 @@ class TestFunctional(unittest.TestCase):
 
     def testUpdateFdtAll(self):
         """Test that all device trees are updated with offset/size info"""
+        self._SetupSplElf()
+        self._SetupTplElf()
         data = self._DoReadFileRealDtb('082_fdt_update_all.dts')
 
         base_expected = {
@@ -3284,6 +3292,8 @@ class TestFunctional(unittest.TestCase):
 
     def testUpdateFdtAllRepack(self):
         """Test that all device trees are updated with offset/size info"""
+        self._SetupSplElf()
+        self._SetupTplElf()
         data = self._DoReadFileRealDtb('134_fdt_update_all_repack.dts')
         SECTION_SIZE = 0x300
         DTB_SIZE = 602
@@ -3737,6 +3747,7 @@ class TestFunctional(unittest.TestCase):
 
     def testMkimage(self):
         """Test using mkimage to build an image"""
+        self._SetupSplElf()
         data = self._DoReadFile('156_mkimage.dts')
 
         # Just check that the data appears in the file somewhere
@@ -3744,6 +3755,7 @@ class TestFunctional(unittest.TestCase):
 
     def testMkimageMissing(self):
         """Test that binman still produces an image if mkimage is missing"""
+        self._SetupSplElf()
         with test_util.capture_sys_output() as (_, stderr):
             self._DoTestFile('156_mkimage.dts',
                              force_missing_bintools='mkimage')
@@ -3856,6 +3868,7 @@ class TestFunctional(unittest.TestCase):
 
     def testSimpleFit(self):
         """Test an image with a FIT inside"""
+        self._SetupSplElf()
         data = self._DoReadFile('161_fit.dts')
         self.assertEqual(U_BOOT_DATA, data[:len(U_BOOT_DATA)])
         self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):])
@@ -5375,6 +5388,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testFitSubentryHashSubnode(self):
         """Test an image with a FIT inside"""
+        self._SetupSplElf()
         data, _, _, out_dtb_name = self._DoReadFileDtb(
             '221_fit_subentry_hash.dts', use_real_dtb=True, update_dtb=True)
 
@@ -5893,6 +5907,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageImagename(self):
         """Test using mkimage with -n holding the data too"""
+        self._SetupSplElf()
         data = self._DoReadFile('242_mkimage_name.dts')
 
         # Check that the data appears in the file somewhere
@@ -5910,6 +5925,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageImage(self):
         """Test using mkimage with -n holding the data too"""
+        self._SetupSplElf()
         data = self._DoReadFile('243_mkimage_image.dts')
 
         # Check that the data appears in the file somewhere
@@ -5930,6 +5946,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageImageNoContent(self):
         """Test using mkimage with -n and no data"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as exc:
             self._DoReadFile('244_mkimage_image_no_content.dts')
         self.assertIn('Could not complete processing of contents',
@@ -5937,6 +5954,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageImageBad(self):
         """Test using mkimage with imagename node and data-to-imagename"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as exc:
             self._DoReadFile('245_mkimage_image_bad.dts')
         self.assertIn('Cannot use both imagename node and data-to-imagename',
@@ -5952,6 +5970,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageCollection(self):
         """Test using a collection referring to an entry in a mkimage entry"""
+        self._SetupSplElf()
         data = self._DoReadFile('247_mkimage_coll.dts')
         expect = U_BOOT_SPL_DATA + U_BOOT_DATA
         self.assertEqual(expect, data[:len(expect)])
@@ -6037,6 +6056,8 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageMultipleDataFiles(self):
         """Test passing multiple files to mkimage in a mkimage entry"""
+        self._SetupSplElf()
+        self._SetupTplElf()
         data = self._DoReadFile('252_mkimage_mult_data.dts')
         # Size of files are packed in their 4B big-endian format
         expect = struct.pack('>I', len(U_BOOT_TPL_DATA))
@@ -6051,8 +6072,42 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         expect += U_BOOT_SPL_DATA
         self.assertEqual(expect, data[-len(expect):])
 
+    def testMkimageMultipleExpanded(self):
+        """Test passing multiple files to mkimage in a mkimage entry"""
+        self._SetupSplElf()
+        self._SetupTplElf()
+        entry_args = {
+            'spl-bss-pad': 'y',
+            'spl-dtb': 'y',
+        }
+        data = self._DoReadFileDtb('252_mkimage_mult_data.dts',
+                                   use_expanded=True, entry_args=entry_args)[0]
+        pad_len = 10
+        tpl_expect = U_BOOT_TPL_DATA
+        spl_expect = U_BOOT_SPL_NODTB_DATA + tools.get_bytes(0, pad_len)
+        spl_expect += U_BOOT_SPL_DTB_DATA
+
+        content = data[0x40:]
+        lens = struct.unpack('>III', content[:12])
+
+        # Size of files are packed in their 4B big-endian format
+        # Size info is always followed by a 4B zero value.
+        self.assertEqual(len(tpl_expect), lens[0])
+        self.assertEqual(len(spl_expect), lens[1])
+        self.assertEqual(0, lens[2])
+
+        rest = content[12:]
+        self.assertEqual(tpl_expect, rest[:len(tpl_expect)])
+
+        rest = rest[len(tpl_expect):]
+        align_pad = len(tpl_expect) % 4
+        self.assertEqual(tools.get_bytes(0, align_pad), rest[:align_pad])
+        rest = rest[align_pad:]
+        self.assertEqual(spl_expect, rest)
+
     def testMkimageMultipleNoContent(self):
         """Test passing multiple data files to mkimage with one data file having no content"""
+        self._SetupSplElf()
         with self.assertRaises(ValueError) as exc:
             self._DoReadFile('253_mkimage_mult_no_content.dts')
         self.assertIn('Could not complete processing of contents',
@@ -6060,6 +6115,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testMkimageFilename(self):
         """Test using mkimage to build a binary with a filename"""
+        self._SetupSplElf()
         retcode = self._DoTestFile('254_mkimage_filename.dts')
         self.assertEqual(0, retcode)
         fname = tools.get_output_filename('mkimage-test.bin')
@@ -6534,6 +6590,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testReplaceFitSibling(self):
         """Test an image with a FIT inside where we replace its sibling"""
+        self._SetupSplElf()
         fname = TestFunctional._MakeInputFile('once', b'available once')
         self._DoReadFileRealDtb('277_replace_fit_sibling.dts')
         os.remove(fname)
@@ -6608,7 +6665,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
                 Private key
                 DTB
         """
-
+        self._SetupSplElf()
         data = self._DoReadFileRealDtb(dts)
         updated_fname = tools.get_output_filename('image-updated.bin')
         tools.write_file(updated_fname, data)
@@ -6683,6 +6740,7 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
 
     def testSymbolNoWrite(self):
         """Test disabling of symbol writing"""
+        self._SetupSplElf()
         self.checkSymbols('282_symbols_disable.dts', U_BOOT_SPL_DATA, 0x1c,
                           no_write_symbols=True)
 
@@ -6696,6 +6754,13 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
                           entry_args=entry_args, use_expanded=True,
                           no_write_symbols=True)
 

tools/binman/ftest.py fdt fdtmap Extract the devicetree blob from the fdtmap · line 6765

Quoted 10 lines
tools/binman/ftest.py +8 -1
+    def testMkimageSpecial(self):
+        """Test mkimage ignores special hash-1 node"""
+        data = self._DoReadFile('283_mkimage_special.dts')
+
+        # Just check that the data appears in the file somewhere
+        self.assertIn(U_BOOT_DATA, data)
+
 
-if __name__ == "__main__":
+if __name__ == "_s_main__":

This looks like an unintended change in this patch?

Regards,
Jonas

Quoted 31 lines
tools/binman/ftest.py +0 -0
     unittest.main()
tools/binman/test/283_mkimage_special.dts new file +24 -0
new file mode 100644
index 000000000000..c234093e6ece
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	#size-cells = <1>;
+
+	binman {
+		mkimage {
+			args = "-T script";
+
+			u-boot {
+			};
+
+			hash {
+			};
+
+			imagename {
+				type = "u-boot";
+			};
+		};
+	};
+};
#11 lore raw

Re: [PATCH v3 09/19] binman: Convert mkimage to Entry_section

Hi Jonas,

Quoted 22 lines · Jonas Karlman

Hi Simon,

Quoted 9 lines · Simon Glass

From: Marek Vasut <[email protected]>

This is needed to handle mkimage with inner section located itself in a section.

Use BuildSectionData() instead of ObtainContents(), add tests and a few other minor fixes:

Quoted 8 lines

---

Changes in v3:
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed

Changes in v2:
- Drop now-unnecessary methods in mkimage etype

Quoted 8 lines

Looks like this change never made it into the patch?

The following functions are still in this file and can now be removed:
- SetAllowMissing
- SetAllowFakeBlob
- CheckMissing
- CheckFakedBlobs

Yes, I partially reverted to an earlier version by mistake. Will fix.

Regards,
Simon

#12 lore raw

PATCH v3 10/19 binman: Provide a way to specify the fdt-list directly

In reply to Simon Glass

Sometimes multiple boards are built with binman and it is useful to specify a different FDT list for each. At present this is not possible without providing multiple values of the of-list entryarg (which is not supported in the U-Boot build system).

Allow a fit,fdt-list-val string-list property to be used instead.

---

Changes in v3:
- Fix 'specific' typo

 tools/binman/entries.rst               |  6 +++
 tools/binman/etype/fit.py              |  9 ++++
 tools/binman/ftest.py                  | 14 ++++++-
 tools/binman/test/284_fit_fdt_list.dts | 58 ++++++++++++++++++++++++++
 4 files changed, 86 insertions(+), 1 deletion(-)
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
tools/binman/entries.rst +6 -0
index b71af801fdad..b55f424620a3 100644
@@ -615,6 +615,12 @@ The top-level 'fit' node supports the following special properties:
         `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed
         to binman.
 
+    fit,fdt-list-val
+        As an alternative to fit,fdt-list the list of device tree files
+        can be provided in this property as a string list, e.g.::
+
+            fit,fdt-list-val = "dtb1", "dtb2";
+
 Substitutions
 ~~~~~~~~~~~~~
 
tools/binman/etype/fit.py +9 -0
index c395706ece5f..ef4d0667578d 100644
@@ -81,6 +81,12 @@ class Entry_fit(Entry_section):
             `of-list` meaning that `-a of-list="dtb1 dtb2..."` should be passed
             to binman.
 
+        fit,fdt-list-val
+            As an alternative to fit,fdt-list the list of device tree files
+            can be provided in this property as a string list, e.g.::
+
+                fit,fdt-list-val = "dtb1", "dtb2";
+
     Substitutions
     ~~~~~~~~~~~~~
 
@@ -361,6 +367,9 @@ class Entry_fit(Entry_section):
                 [EntryArg(self._fit_list_prop.value, str)])
             if fdts is not None:
                 self._fdts = fdts.split()
+        else:
+            self._fdts = fdt_util.GetStringList(self._node, 'fit,fdt-list-val')
+
         self._fit_default_dt = self.GetEntryArgsOrProps([EntryArg('default-dt',
                                                                   str)])[0]
 
tools/binman/ftest.py +13 -1
index 60e69443c400..ed1940ed9f71 100644
@@ -6761,6 +6761,18 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         # Just check that the data appears in the file somewhere
         self.assertIn(U_BOOT_DATA, data)
 
+    def testFitFdtList(self):
+        """Test an image with an FIT with the fit,fdt-list-val option"""
+        entry_args = {
+            'default-dt': 'test-fdt2',
+        }
+        data = self._DoReadFileDtb(
+            '284_fit_fdt_list.dts',
+            entry_args=entry_args,
+            extra_indirs=[os.path.join(self._indir, TEST_FDT_SUBDIR)])[0]
+        self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):])
+        fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)]
+
 
-if __name__ == "_s_main__":
+if __name__ == "__main__":
     unittest.main()
tools/binman/test/284_fit_fdt_list.dts new file +58 -0

58 lines not on this page · raw message

Signature · 1 line
2.41.0.390.g38632f3daf-goog
#13 lore raw

PATCH v3 11/19 binman: Drop __bss_size variable in bss_data.c

In reply to Simon Glass

This is not needed since the linker script sets it up. Drop the variable to avoid confusion.

Fix the prototype for main() while we are here.

---

(no changes since v1)

 tools/binman/test/bss_data.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
tools/binman/test/bss_data.c +1 -2
index 4f9b64cef9ef..7047a3bb014d 100644
@@ -7,9 +7,8 @@
  */
 
 int bss_data[10];
-int __bss_size = sizeof(bss_data);
 
-int main()
+int main(void)
 {
 	bss_data[2] = 2;
 
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#14 lore raw

PATCH v3 12/19 binman: Correct handling of zero bss size

In reply to Simon Glass

Fix the check for the __bss_size symbol, since it may be 0. Unfortunately there was no test coverage for this.

---

(no changes since v1)

 tools/binman/elf_test.py                 |  5 +++++
 tools/binman/etype/u_boot_spl_bss_pad.py |  2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py |  2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py |  2 +-
 tools/binman/ftest.py                    | 12 ++++++++++++
 tools/binman/test/285_spl_expand.dts     | 13 +++++++++++++
 tools/binman/test/Makefile               |  5 ++++-
 tools/binman/test/bss_data_zero.c        | 16 ++++++++++++++++
 tools/binman/test/bss_data_zero.lds      | 15 +++++++++++++++
 tools/binman/test/embed_data.lds         |  1 +
 10 files changed, 69 insertions(+), 4 deletions(-)
 create mode 100644 tools/binman/test/285_spl_expand.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
tools/binman/elf_test.py +5 -0
index 2fb3f6f28ff1..cc95b424b338 100644
@@ -371,6 +371,11 @@ class TestElf(unittest.TestCase):
             elf.GetSymbolOffset(fname, 'embed')
         self.assertIn('__image_copy_start', str(e.exception))
 
+    def test_get_symbol_address(self):
+        fname = self.ElfTestFile('embed_data')
+        addr = elf.GetSymbolAddress(fname, 'region_size')
+        self.assertEqual(0, addr)
+
 
 if __name__ == '__main__':
     unittest.main()
tools/binman/etype/u_boot_spl_bss_pad.py +1 -1
index 1ffeb3911fd8..4af4045d3702 100644
@@ -38,7 +38,7 @@ class Entry_u_boot_spl_bss_pad(Entry_blob):
     def ObtainContents(self):
         fname = tools.get_input_filename('spl/u-boot-spl')
         bss_size = elf.GetSymbolAddress(fname, '__bss_size')
-        if not bss_size:
+        if bss_size is None:
             self.Raise('Expected __bss_size symbol in spl/u-boot-spl')
         self.SetContents(tools.get_bytes(0, bss_size))
         return True
tools/binman/etype/u_boot_tpl_bss_pad.py +1 -1
index 29c6a9541296..46d2cd58f7e2 100644
@@ -38,7 +38,7 @@ class Entry_u_boot_tpl_bss_pad(Entry_blob):
     def ObtainContents(self):
         fname = tools.get_input_filename('tpl/u-boot-tpl')
         bss_size = elf.GetSymbolAddress(fname, '__bss_size')
-        if not bss_size:
+        if bss_size is None:
             self.Raise('Expected __bss_size symbol in tpl/u-boot-tpl')
         self.SetContents(tools.get_bytes(0, bss_size))
         return True
tools/binman/etype/u_boot_vpl_bss_pad.py +1 -1
index bba38ccf9e93..12b286a71987 100644
@@ -38,7 +38,7 @@ class Entry_u_boot_vpl_bss_pad(Entry_blob):
     def ObtainContents(self):
         fname = tools.get_input_filename('vpl/u-boot-vpl')
         bss_size = elf.GetSymbolAddress(fname, '__bss_size')
-        if not bss_size:
+        if bss_size is None:
             self.Raise('Expected __bss_size symbol in vpl/u-boot-vpl')
         self.SetContents(tools.get_bytes(0, bss_size))
         return True
tools/binman/ftest.py +12 -0
index ed1940ed9f71..dc9a95d341e5 100644
@@ -6773,6 +6773,18 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         self.assertEqual(U_BOOT_NODTB_DATA, data[-len(U_BOOT_NODTB_DATA):])
         fit_data = data[len(U_BOOT_DATA):-len(U_BOOT_NODTB_DATA)]
 
+    def testSplEmptyBss(self):
+        """Test an expanded SPL with a zero-size BSS"""
+        # ELF file with a '__bss_size' symbol
+        self._SetupSplElf(src_fname='bss_data_zero')
+
+        entry_args = {
+            'spl-bss-pad': 'y',
+            'spl-dtb': 'y',
+        }
+        data = self._DoReadFileDtb('285_spl_expand.dts',
+                                   use_expanded=True, entry_args=entry_args)[0]
+
 
 if __name__ == "__main__":
     unittest.main()
tools/binman/test/285_spl_expand.dts new file +13 -0
new file mode 100644
index 000000000000..9c88ccb287b1
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	#size-cells = <1>;
+
+	binman {
+		u-boot-spl {
+		};
+	};
+};
tools/binman/test/Makefile +4 -1
index cd66a3038be2..4d152eee9c09 100644
@@ -32,7 +32,7 @@ LDS_BINMAN_EMBED := -T $(SRC)u_boot_binman_embed.lds
 LDS_EFL_SECTIONS := -T $(SRC)elf_sections.lds
 LDS_BLOB := -T $(SRC)blob_syms.lds
 
-TARGETS = u_boot_ucode_ptr u_boot_no_ucode_ptr bss_data \
+TARGETS = u_boot_ucode_ptr u_boot_no_ucode_ptr bss_data bss_data_zero \
 	u_boot_binman_syms u_boot_binman_syms.bin u_boot_binman_syms_bad \
 	u_boot_binman_syms_size u_boot_binman_syms_x86 embed_data \
 	u_boot_binman_embed u_boot_binman_embed_sm elf_sections blob_syms.bin
@@ -48,6 +48,9 @@ u_boot_ucode_ptr: u_boot_ucode_ptr.c
 bss_data: CFLAGS += $(SRC)bss_data.lds
 bss_data: bss_data.c
 
+bss_data_zero: CFLAGS += $(SRC)bss_data_zero.lds
+bss_data_zero: bss_data_zero.c
+
 embed_data: CFLAGS += $(SRC)embed_data.lds
 embed_data: embed_data.c
 
tools/binman/test/bss_data_zero.c new file +16 -0
new file mode 100644
index 000000000000..7047a3bb014d
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (c) 2016 Google, Inc
+ *
+ * Simple program to create a bss_data region so the symbol can be read
+ * by binutils. This is used by binman tests.
+ */
+
+int bss_data[10];
+
+int main(void)
+{
+	bss_data[2] = 2;
+
+	return 0;
+}
tools/binman/test/bss_data_zero.lds new file +15 -0
new file mode 100644
index 000000000000..8fa0210a8f46
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (c) 2016 Google, Inc
+ */
+
+OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386")
+OUTPUT_ARCH(i386)
+ENTRY(_start)
+
+SECTIONS
+{
+	. = 0xfffffdf0;
+	_start = .;
+	__bss_size = 0;
+}
tools/binman/test/embed_data.lds +1 -0
index 908bf66c294b..d416cb211107 100644
@@ -17,6 +17,7 @@ SECTIONS
 		embed_start = .;
 		*(.embed*)
 		embed_end = .;
+		region_size = 0;
 		. = ALIGN(32);
 		*(.data*)
 	}
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#15 lore raw

PATCH v3 13/19 dtoc: Support copying the contents of a node into another

In reply to Simon Glass

This permits implementation of a simple templating system, where a node can be reused as a base for others.

For now this adds new subnodes after any existing ones.

---

Changes in v3:
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments

 tools/dtoc/fdt.py                  | 103 ++++++++++++++++++++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts |  75 +++++++++++++++++++++
 tools/dtoc/test_fdt.py             |  70 ++++++++++++++++++++
 3 files changed, 245 insertions(+), 3 deletions(-)
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts
tools/dtoc/fdt.py +100 -3
index a8e05349a720..dcd78b9e5fbf 100644
@@ -13,6 +13,7 @@ from dtoc import fdt_util
 import libfdt
 from libfdt import QUIET_NOTFOUND
 from u_boot_pylib import tools
+from u_boot_pylib import tout
 
 # This deals with a device tree, presenting it as an assortment of Node and
 # Prop objects, representing nodes and properties, respectively. This file
@@ -264,6 +265,13 @@ class Prop:
                 fdt_obj.setprop(node.Offset(), self.name, self.bytes)
             self.dirty = False
 
+    def purge(self):
+        """Set a property offset to None
+
+        The property remains in the tree structure and will be recreated when
+        the FDT is synced
+        """
+        self._offset = None
 
 class Node:
     """A device tree node
@@ -534,8 +542,8 @@ class Node:
         """
         return self.AddData(prop_name, struct.pack('>I', val))
 
-    def AddSubnode(self, name):
-        """Add a new subnode to the node
+    def Subnode(self, name):
+        """Create new subnode for the node
 
         Args:
             name: name of node to add
@@ -544,10 +552,72 @@ class Node:
             New subnode that was created
         """
         path = self.path + '/' + name
-        subnode = Node(self._fdt, self, None, name, path)
+        return Node(self._fdt, self, None, name, path)
+
+    def AddSubnode(self, name):
+        """Add a new subnode to the node, after all other subnodes
+
+        Args:
+            name: name of node to add
+
+        Returns:
+            New subnode that was created
+        """
+        subnode = self.Subnode(name)
         self.subnodes.append(subnode)
         return subnode
 
+    def insert_subnode(self, name):
+        """Add a new subnode to the node, before all other subnodes
+
+        This deletes other subnodes and sets their offset to None, so that they
+        will be recreated after this one.
+
+        Args:
+            name: name of node to add
+
+        Returns:
+            New subnode that was created
+        """
+        # Deleting a node invalidates the offsets of all following nodes, so
+        # process in reverse order so that the offset of each node remains valid
+        # until deletion.
+        for subnode in reversed(self.subnodes):
+            subnode.purge(True)
+        subnode = self.Subnode(name)
+        self.subnodes.insert(0, subnode)
+        return subnode
+
+    def purge(self, delete_it=False):
+        """Purge this node, setting offset to None and deleting from FDT"""
+        if self._offset is not None:
+            if delete_it:
+                CheckErr(self._fdt._fdt_obj.del_node(self.Offset()),
+                     "Node '%s': delete" % self.path)
+            self._offset = None
+            self._fdt.Invalidate()
+
+        for prop in self.props.values():
+            prop.purge()
+
+        for subnode in self.subnodes:
+            subnode.purge(False)
+
+    def move_to_first(self):
+        """Move the current node to first in its parent's node list"""
+        parent = self.parent
+        if parent.subnodes and parent.subnodes[0] == self:
+            return
+        for subnode in reversed(parent.subnodes):
+            subnode.purge(True)
+
+        new_subnodes = [self]
+        for subnode in parent.subnodes:
+            #subnode.purge(False)
+            if subnode != self:
+                new_subnodes.append(subnode)
+        parent.subnodes = new_subnodes
+
     def Delete(self):
         """Delete a node
 
@@ -635,6 +705,33 @@ class Node:
             prop.Sync(auto_resize)
         return added
 
+    def copy_node(self, src):
+        """Copy a node and all its subnodes into this node
+
+        Args:
+            src (Node): Node to copy
+
+        This works recursively.
+
+        The new node is put before all other nodes. If the node already
+        exists, just its subnodes and properties are copied, placing them before
+        any existing subnodes. Properties which exist in the destination node
+        already are not copied.
+        """
+        dst = self.FindNode(src.name)
+        if dst:
+            dst.move_to_first()
+        else:
+            dst = self.insert_subnode(src.name)
+        for name, src_prop in src.props.items():
+            if name not in dst.props:
+                dst.props[name] = Prop(dst, None, name, src_prop.bytes)
+
+        # Process in reverse order so that they appear correctly in the result,
+        # since copy_node() puts the node first in the list
+        for node in reversed(src.subnodes):
+            dst.copy_node(node)
+
 
 class Fdt:
     """Provides simple access to a flat device tree blob using libfdts.
tools/dtoc/test/dtoc_test_copy.dts new file +75 -0

75 lines not on this page · raw message

tools/dtoc/test_fdt.py +70 -0

76 lines not on this page · raw message

Signature · 1 line
2.41.0.390.g38632f3daf-goog
#16 lore raw

PATCH v3 14/19 dtoc: Allow inserting a list of nodes into another

In reply to Simon Glass

Provide a way to specify a phandle list of nodes which are to be inserted into an existing node.

---

Changes in v3:
- Adjust to use the new example file

 tools/dtoc/fdt.py                  | 19 +++++++++++++++++++
 tools/dtoc/test/dtoc_test_copy.dts | 13 ++++++++++++-
 tools/dtoc/test_fdt.py             | 25 ++++++++++++++++++++++++-
 3 files changed, 55 insertions(+), 2 deletions(-)
tools/dtoc/fdt.py +19 -0
index dcd78b9e5fbf..4375d3923fbf 100644
@@ -732,6 +732,25 @@ class Node:
         for node in reversed(src.subnodes):
             dst.copy_node(node)
 
+    def copy_subnodes_from_phandles(self, phandle_list):
+        """Copy subnodes of a list of nodes into another node
+
+        Args:
+            phandle_list (list of int): List of phandles of nodes to copy
+
+        For each node in the phandle list, its subnodes and their properties are
+        copied recursively. Note that it does not copy the node itself, nor its
+        properties.
+        """
+        # Process in reverse order, since new nodes are inserted at the start of
+        # the destination's node list. We want them to appear in order of the
+        # phandle list
+        for phandle in phandle_list.__reversed__():
+            parent = self.GetFdt().LookupPhandle(phandle)
+            tout.debug(f'adding template {parent.path} to node {self.path}')
+            for node in parent.subnodes.__reversed__():
+                self.copy_node(node)
+
 
 class Fdt:
     """Provides simple access to a flat device tree blob using libfdts.
tools/dtoc/test/dtoc_test_copy.dts +12 -1
index bf2b50e92d71..1939256349fe 100644
@@ -10,6 +10,7 @@
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
+	copy-list = <&another &base>;
 
 	dest {
 		bootph-all;
@@ -45,7 +46,7 @@
 		};
 	};
 
-	base {
+	base: base {
 		compatible = "sandbox,i2c";
 		bootph-all;
 		#address-cells = <1>;
@@ -72,4 +73,14 @@
 			};
 		};
 	};
+
+	another: another {
+		earlier {
+			wibble = <2>;
+		};
+
+		later {
+			fibble = <3>;
+		};
+	};
 };
tools/dtoc/test_fdt.py +24 -1
index b3593cbee8b2..0ee4956591f9 100755
@@ -316,7 +316,8 @@ class TestNode(unittest.TestCase):
             chk = dtb.GetNode('/dest/base')
             self.assertTrue(chk)
             self.assertEqual(
-                {'compatible', 'bootph-all', '#address-cells', '#size-cells'},
+                {'compatible', 'bootph-all', '#address-cells', '#size-cells',
+                 'phandle'},
                 chk.props.keys())
 
             # Check the first property
@@ -376,6 +377,28 @@ class TestNode(unittest.TestCase):
         dst = new_dtb.GetNode('/dest')
         do_copy_checks(new_dtb, dst, expect_none=False)
 
+    def test_copy_subnodes_from_phandles(self):
+        """Test copy_node() function"""
+        dtb = fdt.FdtScan(find_dtb_file('dtoc_test_copy.dts'))
+
+        orig = dtb.GetNode('/')
+        node_list = fdt_util.GetPhandleList(orig, 'copy-list')
+
+        dst = dtb.GetNode('/dest')
+        dst.copy_subnodes_from_phandles(node_list)
+
+        pmic = dtb.GetNode('/dest/over')
+        self.assertTrue(pmic)
+
+        subn = dtb.GetNode('/dest/first@0')
+        self.assertTrue(subn)
+        self.assertEqual({'a-prop', 'b-prop', 'reg'}, subn.props.keys())
+
+        self.assertEqual(
+            ['/dest/earlier', '/dest/later', '/dest/over', '/dest/first@0',
+             '/dest/second', '/dest/existing', '/dest/base'],
+            [n.path for n in dst.subnodes])
+
 
 class TestProp(unittest.TestCase):
     """Test operation of the Prop class"""
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#17 lore raw

PATCH v3 15/19 binman: Support simple templates

In reply to Simon Glass

Collections can used to collect the contents of other entries into a single entry, but they result in a single entry, with the original entries 'left behind' in their old place.

It is useful to be able to specific a set of entries ones and have it used in multiple images, or parts of an image.

Implement this mechanism.

---

(no changes since v2)

Changes in v2:
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

 tools/binman/binman.rst            | 82 ++++++++++++++++++++++++++++++
 tools/binman/control.py            | 26 ++++++++++
 tools/binman/etype/section.py      |  3 +-
 tools/binman/ftest.py              |  8 +++
 tools/binman/test/286_template.dts | 42 +++++++++++++++
 5 files changed, 160 insertions(+), 1 deletion(-)
 create mode 100644 tools/binman/test/286_template.dts
tools/binman/binman.rst +82 -0
index a4b31fe5397b..73d38e6b64bb 100644
@@ -727,6 +727,13 @@ optional:
     Note that missing, optional blobs do not produce a non-zero exit code from
     binman, although it does show a warning about the missing external blob.
 
+insert-template:
+    This is not strictly speaking an entry property, since it is processed early
+    in Binman before the entries are read. It is a list of phandles of nodes to
+    include in the current (target) node. For each node, its subnodes and their
+    properties are brought into the target node. See Templates_ below for
+    more information.
+
 The attributes supported for images and sections are described below. Several
 are similar to those for entries.
 
@@ -1172,6 +1179,81 @@ If you are having trouble figuring out what is going on, you can use
      arch/arm/dts/u-boot.dtsi ... found: "arch/arm/dts/juno-r2-u-boot.dtsi"
 
 
+Templates
+=========
+
+Sometimes multiple images need to be created which have all have a common
+part. For example, a board may generate SPI and eMMC images which both include
+a FIT. Since the FIT includes many entries, it is tedious to repeat them twice
+in the image description.
+
+Templates provide a simple way to handle this::
+
+    binman {
+        multiple-images;
+        common_part: template-1 {
+            fit {
+                ... lots of entries in here
+            };
+
+            text {
+                text = "base image";
+            };
+        };
+
+        spi-image {
+            filename = "image-spi.bin";
+            insert-template = <&fit>;
+
+            /* things specific to SPI follow */
+            footer {
+            ];
+
+            text {
+                text = "SPI image";
+            };
+        };
+
+        mmc-image {
+            filename = "image-mmc.bin";
+            insert-template = <&fit>;
+
+            /* things specific to MMC follow */
+            footer {
+            ];
+
+            text {
+                text = "MMC image";
+            };
+        };
+    };
+
+The template node name must start with 'template', so it is not considered to be
+an image itself.
+
+The mechanism is very simple. For each phandle in the 'insert-templates'
+property, the source node is looked up. Then the subnodes of that source node
+are copied into the target node, i.e. the one containing the `insert-template`
+property.
+
+If the target node has a node with the same name as a template, its properties
+override corresponding properties in the template. This allows the template to
+be uses as a base, with the node providing updates to the properties as needed.
+The overriding happens recursively.
+
+Template nodes appear first in each node that they are inserted into and
+ordering of template nodes is preserved. Other nodes come afterwards. If a
+template node also appears in the target node, then the template node sets the
+order. Thus the template can be used to set the ordering, even if the target
+node provides all the properties. In the above example, `fit` and `text` appear
+first in the `spi-image` and `mmc-image` images, followed by `footer`.
+
+Where there are multiple template nodes, they are inserted in that order. so
+the first template node appears first, then the second.
+
+Note that template nodes are not removed from the binman description at present.
+
+
 Updating an ELF file
 ====================
 
tools/binman/control.py +26 -0
index 7e2dd3541b96..e9c4a65a75a1 100644
@@ -22,6 +22,7 @@ from binman import bintool
 from binman import cbfs_util
 from binman import elf
 from binman import entry
+from dtoc import fdt_util
 from u_boot_pylib import command
 from u_boot_pylib import tools
 from u_boot_pylib import tout
@@ -478,6 +479,29 @@ def SignEntries(image_fname, input_fname, privatekey_fname, algo, entry_paths,
 
     AfterReplace(image, allow_resize=True, write_map=write_map)
 
+def _ProcessTemplates(parent):
+    """Handle any templates in the binman description
+
+    Args:
+        parent: Binman node to process (typically /binman)
+
+    Search though each target node looking for those with an 'insert-template'
+    property. Use that as a list of references to template nodes to use to
+    adjust the target node.
+
+    Processing involves copying each subnode of the template node into the
+    target node.
+
+    For now this is not done recursively, so templates must be at the top level
+    of the binman image.
+
+    See 'Templates' in the Binman documnentation for details.
+    """
+    for node in parent.subnodes:
+        tmpl = fdt_util.GetPhandleList(node, 'insert-template')
+        if tmpl:
+            node.copy_subnodes_from_phandles(tmpl)
+
 def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded):
     """Prepare the images to be processed and select the device tree
 
@@ -520,6 +544,8 @@ def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded):
         raise ValueError("Device tree '%s' does not have a 'binman' "
                             "node" % dtb_fname)
 
+    _ProcessTemplates(node)
+
     images = _ReadImageDesc(node, use_expanded)
 
     if select_images:
tools/binman/etype/section.py +2 -1
index d9b9e428024a..7c4d312c16c0 100644
@@ -179,7 +179,8 @@ class Entry_section(Entry):
         Returns:
             bool: True if the node is a special one, else False
         """
-        return node.name.startswith('hash') or node.name.startswith('signature')
+        start_list = ('hash', 'signature', 'template')
+        return any(node.name.startswith(name) for name in start_list)
 
     def ReadNode(self):
         """Read properties from the section node"""
tools/binman/ftest.py +8 -0
index dc9a95d341e5..fc5d8a839e6b 100644
@@ -6785,6 +6785,14 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         data = self._DoReadFileDtb('285_spl_expand.dts',
                                    use_expanded=True, entry_args=entry_args)[0]
 
+    def testTemplate(self):
+        """Test using a template"""
+        TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA)
+        data = self._DoReadFile('286_template.dts')
+        first = U_BOOT_DATA + VGA_DATA + U_BOOT_DTB_DATA
+        second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA
+        self.assertEqual(U_BOOT_IMG_DATA + first + second, data)
+
 
 if __name__ == "__main__":
     unittest.main()
tools/binman/test/286_template.dts new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.41.0.390.g38632f3daf-goog
#18 lore raw

PATCH v3 16/19 binman: Support templating with multiple images

In reply to Simon Glass

Allow a template to appear in the top level description when using multiple images.

---

Changes in v3:
- Drop duplicate dts-v1 header

 tools/binman/control.py                  |  5 +++--
 tools/binman/ftest.py                    | 12 +++++++++++
 tools/binman/test/287_template_multi.dts | 27 ++++++++++++++++++++++++
 3 files changed, 42 insertions(+), 2 deletions(-)
 create mode 100644 tools/binman/test/287_template_multi.dts
tools/binman/control.py +3 -2
index e9c4a65a75a1..f92c152285ce 100644
@@ -57,8 +57,9 @@ def _ReadImageDesc(binman_node, use_expanded):
     images = OrderedDict()
     if 'multiple-images' in binman_node.props:
         for node in binman_node.subnodes:
-            images[node.name] = Image(node.name, node,
-                                      use_expanded=use_expanded)
+            if 'template' not in node.name:
+                images[node.name] = Image(node.name, node,
+                                          use_expanded=use_expanded)
     else:
         images['image'] = Image('image', binman_node, use_expanded=use_expanded)
     return images
tools/binman/ftest.py +12 -0
index fc5d8a839e6b..dd6075b871d4 100644
@@ -6793,6 +6793,18 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA
         self.assertEqual(U_BOOT_IMG_DATA + first + second, data)
 
+    def testTemplateBlobMulti(self):
+        """Test using a template with 'multiple-images' enabled"""
+        TestFunctional._MakeInputFile('my-blob.bin', b'blob')
+        TestFunctional._MakeInputFile('my-blob2.bin', b'other')
+        retcode = self._DoTestFile('287_template_multi.dts')
+
+        self.assertEqual(0, retcode)
+        image = control.images['image']
+        image_fname = tools.get_output_filename('my-image.bin')
+        data = tools.read_file(image_fname)
+        self.assertEqual(b'blob@@@@other', data)
+
 
 if __name__ == "__main__":
     unittest.main()
tools/binman/test/287_template_multi.dts new file +27 -0
new file mode 100644
index 000000000000..122bfccd5652
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+/ {
+	binman: binman {
+		multiple-images;
+
+		my_template: template {
+			blob-ext@0 {
+				filename = "my-blob.bin";
+				offset = <0>;
+			};
+			blob-ext@8 {
+				offset = <8>;
+			};
+		};
+
+		image {
+			pad-byte = <0x40>;
+			filename = "my-image.bin";
+			insert-template = <&my_template>;
+			blob-ext@8 {
+				filename = "my-blob2.bin";
+			};
+		};
+	};
+};
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#19 lore raw

PATCH v3 17/19 binman: Add a test for templating in a FIT

In reply to Simon Glass

Add this as a separate test case.

---

Changes in v3:
- Add new test case for templating in a FIT

 tools/binman/ftest.py                  |  7 +++++
 tools/binman/test/288_template_fit.dts | 37 ++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
 create mode 100644 tools/binman/test/288_template_fit.dts
tools/binman/ftest.py +7 -0
index dd6075b871d4..dfca6316624c 100644
@@ -6805,6 +6805,13 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         data = tools.read_file(image_fname)
         self.assertEqual(b'blob@@@@other', data)
 
+    def testTemplateFit(self):
+        """Test using a template in a FIT"""
+        fit_data = self._DoReadFile('288_template_fit.dts')
+        fname = os.path.join(self._indir, 'fit_data.fit')
+        tools.write_file(fname, fit_data)
+        out = tools.run('dumpimage', '-l', fname)
+
 
 if __name__ == "__main__":
     unittest.main()
tools/binman/test/288_template_fit.dts new file +37 -0
new file mode 100644
index 000000000000..d84dca4ea41e
@@ -0,0 +1,37 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+
+/ {
+	binman: binman {
+		multiple-images;
+
+		my_template: template {
+			fit@0 {
+				images {
+					kernel-1 {
+					};
+					kernel-2 {
+					};
+				};
+			};
+		};
+
+		image {
+			filename = "image.bin";
+			insert-template = <&my_template>;
+
+			fit@0 {
+				description = "desc";
+				configurations {
+				};
+				images {
+					kernel-3 {
+					};
+					kernel-4 {
+					};
+				};
+			};
+		};
+	};
+};
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#20 lore raw

PATCH v3 18/19 binman: Support templates at any level

In reply to Simon Glass

Allow templates to be used inside a section, not just in the top-level /binman node.

---

(no changes since v1)

 tools/binman/control.py                    |  5 ++-
 tools/binman/ftest.py                      |  8 ++++
 tools/binman/test/289_template_section.dts | 52 ++++++++++++++++++++++
 3 files changed, 63 insertions(+), 2 deletions(-)
 create mode 100644 tools/binman/test/289_template_section.dts
tools/binman/control.py +3 -2
index f92c152285ce..25e66814837d 100644
@@ -493,8 +493,8 @@ def _ProcessTemplates(parent):
     Processing involves copying each subnode of the template node into the
     target node.
 
-    For now this is not done recursively, so templates must be at the top level
-    of the binman image.
+    This is done recursively, so templates can be at any level of the binman
+    image, e.g. inside a section.
 
     See 'Templates' in the Binman documnentation for details.
     """
@@ -502,6 +502,7 @@ def _ProcessTemplates(parent):
         tmpl = fdt_util.GetPhandleList(node, 'insert-template')
         if tmpl:
             node.copy_subnodes_from_phandles(tmpl)
+        _ProcessTemplates(node)
 
 def PrepareImagesAndDtbs(dtb_fname, select_images, update_fdt, use_expanded):
     """Prepare the images to be processed and select the device tree
tools/binman/ftest.py +8 -0
index dfca6316624c..e96223cbd892 100644
@@ -6812,6 +6812,14 @@ fdt         fdtmap                Extract the devicetree blob from the fdtmap
         tools.write_file(fname, fit_data)
         out = tools.run('dumpimage', '-l', fname)
 
+    def testTemplateSection(self):
+        """Test using a template in a section (not at top level)"""
+        TestFunctional._MakeInputFile('vga2.bin', b'#' + VGA_DATA)
+        data = self._DoReadFile('289_template_section.dts')
+        first = U_BOOT_DATA + VGA_DATA + U_BOOT_DTB_DATA
+        second = U_BOOT_DATA + b'#' + VGA_DATA + U_BOOT_DTB_DATA
+        self.assertEqual(U_BOOT_IMG_DATA + first + second + first, data)
+
 
 if __name__ == "__main__":
     unittest.main()
tools/binman/test/289_template_section.dts new file +52 -0

52 lines not on this page · raw message

Signature · 1 line
2.41.0.390.g38632f3daf-goog
#21 lore raw

PATCH v3 19/19 binman: Support writing symbols inside a mkimage image

In reply to Simon Glass

Add support for writing symbols and determining the assumed position of binaries inside a mkimage image. This is useful as an example for other entry types which might want to do the same thing.

---

Changes in v3:
- Add new patch to support writing symbols inside a mkimage image

 tools/binman/entry.py                 |  2 -
 tools/binman/etype/mkimage.py         | 36 +++++++++++++++
 tools/binman/ftest.py                 | 64 +++++++++++++++++++++++++++
 tools/binman/test/290_mkimage_sym.dts | 27 +++++++++++
 4 files changed, 127 insertions(+), 2 deletions(-)
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
tools/binman/entry.py +0 -2
index 0d4cb94f7002..42e0b7b91450 100644
@@ -1314,8 +1314,6 @@ features to produce new behaviours.
         """
         data = b''
         for entry in entries:
-            # First get the input data and put it in a file
-            entry.ObtainContents(fake_size=fake_size)
             data += entry.GetData()
         uniq = self.GetUniqueName()
         fname = tools.get_output_filename(f'{prefix}.{uniq}')
tools/binman/etype/mkimage.py +36 -0
index 493761da59f5..9d60c85722b3 100644
@@ -268,3 +268,39 @@ class Entry_mkimage(Entry_section):
 
     def CheckEntries(self):
         pass
+
+    def ProcessContents(self):
+        # The blob may have changed due to WriteSymbols()
+        ok = super().ProcessContents()
+        data = self.BuildSectionData(True)
+        ok2 = self.ProcessContentsUpdate(data)
+        return ok and ok2
+
+    def SetImagePos(self, image_pos):
+        """Set the position in the image
+
+        This sets each subentry's offsets, sizes and positions-in-image
+        according to where they ended up in the packed mkimage file.
+
+        NOTE: This assumes a legacy mkimage and assumes that the images are
+        written to the output in order. SoC-specific mkimage handling may not
+        conform to this, in which case these values may be wrong.
+
+        Args:
+            image_pos (int): Position of this entry in the image
+        """
+        # The mkimage header consists of 0x40 bytes, following by a table of
+        # offsets for each file
+        upto = 0x40
+
+        # Skip the 0-terminated list of offsets (assume a single image)
+        upto += 4 + 4
+        for entry in self.GetEntries().values():
+            entry.SetOffsetSize(upto, None)
+
+            # Give up if any entries lack a size
+            if entry.size is None:
+                return
+            upto += entry.size
+
+        super().SetImagePos(image_pos)
tools/binman/ftest.py +64 -0

70 lines not on this page · raw message

tools/binman/test/290_mkimage_sym.dts new file +27 -0
new file mode 100644
index 000000000000..2dfd286ad44f
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0+
+
+/dts-v1/;
+
+/ {
+	#address-cells = <1>;
+	#size-cells = <1>;
+
+	binman {
+		u-boot-dtb {
+		};
+
+		mkimage {
+			args = "-n test -T script";
+
+			u-boot-spl {
+			};
+
+			u-boot-spl2 {
+				type = "u-boot-spl";
+			};
+		};
+
+		u-boot {
+		};
+	};
+};
Signature · 1 line
2.41.0.390.g38632f3daf-goog
#22 lore raw
In reply to Simon Glass
Quoted 90 lines · Simon Glass

This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.

A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.

In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.

Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image

Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

Marek Vasut (1):
binman: Convert mkimage to Entry_section

Simon Glass (18):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image

 arch/arm/dts/stm32mp15-u-boot.dtsi         |   1 +
 tools/binman/binman.rst                    |  89 +++++++++
 tools/binman/control.py                    |  34 +++-
 tools/binman/elf.py                        |  13 +-
 tools/binman/elf_test.py                   |  13 +-
 tools/binman/entries.rst                   |   6 +
 tools/binman/entry.py                      |  11 +-
 tools/binman/etype/blob_phase.py           |   5 +
 tools/binman/etype/fit.py                  |   9 +
 tools/binman/etype/mkimage.py              | 110 ++++++++---
 tools/binman/etype/section.py              |  54 +++--
 tools/binman/etype/u_boot_spl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py   |   2 +-
 tools/binman/ftest.py                      | 218 ++++++++++++++++++++-
 tools/binman/test/282_symbols_disable.dts  |  25 +++
 tools/binman/test/283_mkimage_special.dts  |  24 +++
 tools/binman/test/284_fit_fdt_list.dts     |  58 ++++++
 tools/binman/test/285_spl_expand.dts       |  13 ++
 tools/binman/test/286_template.dts         |  42 ++++
 tools/binman/test/287_template_multi.dts   |  27 +++
 tools/binman/test/288_template_fit.dts     |  37 ++++
 tools/binman/test/289_template_section.dts |  52 +++++
 tools/binman/test/290_mkimage_sym.dts      |  27 +++
 tools/binman/test/Makefile                 |   5 +-
 tools/binman/test/bss_data.c               |   3 +-
 tools/binman/test/bss_data_zero.c          |  16 ++
 tools/binman/test/bss_data_zero.lds        |  15 ++
 tools/binman/test/embed_data.lds           |   1 +
 tools/dtoc/fdt.py                          | 122 +++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts         |  86 ++++++++
 tools/dtoc/test_fdt.py                     |  93 +++++++++
 32 files changed, 1147 insertions(+), 68 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
 create mode 100644 tools/binman/test/283_mkimage_special.dts
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
 create mode 100644 tools/binman/test/285_spl_expand.dts
Quoted 9 lines
 create mode 100644 tools/binman/test/286_template.dts
 create mode 100644 tools/binman/test/287_template_multi.dts
 create mode 100644 tools/binman/test/288_template_fit.dts
 create mode 100644 tools/binman/test/289_template_section.dts
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts

Works much better!

What does not work yet:

/dts-v1/;

/ {
	binman: binman {
		multiple-images;

		my_template: template {
			size = <0x100>;
			pad-byte = <0xff>;

			blob@0 {
				filename = "my-blob.bin";
			};
		};

		image {
			filename = "my-image.bin";
			insert-template = <&my_template>;
		};
	};
};

The properties size and pad-byte get lost in this case.

Jan

Signature · 2 lines
Siemens AG, Technology
Competence Center Embedded Linux
#23 lore raw

Hi Jan,

Quoted 128 lines · Jan Kiszka
Quoted 90 lines · Simon Glass

This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.

A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.

In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.

Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image

Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

Marek Vasut (1):
binman: Convert mkimage to Entry_section

Simon Glass (18):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image

 arch/arm/dts/stm32mp15-u-boot.dtsi         |   1 +
 tools/binman/binman.rst                    |  89 +++++++++
 tools/binman/control.py                    |  34 +++-
 tools/binman/elf.py                        |  13 +-
 tools/binman/elf_test.py                   |  13 +-
 tools/binman/entries.rst                   |   6 +
 tools/binman/entry.py                      |  11 +-
 tools/binman/etype/blob_phase.py           |   5 +
 tools/binman/etype/fit.py                  |   9 +
 tools/binman/etype/mkimage.py              | 110 ++++++++---
 tools/binman/etype/section.py              |  54 +++--
 tools/binman/etype/u_boot_spl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py   |   2 +-
 tools/binman/ftest.py                      | 218 ++++++++++++++++++++-
 tools/binman/test/282_symbols_disable.dts  |  25 +++
 tools/binman/test/283_mkimage_special.dts  |  24 +++
 tools/binman/test/284_fit_fdt_list.dts     |  58 ++++++
 tools/binman/test/285_spl_expand.dts       |  13 ++
 tools/binman/test/286_template.dts         |  42 ++++
 tools/binman/test/287_template_multi.dts   |  27 +++
 tools/binman/test/288_template_fit.dts     |  37 ++++
 tools/binman/test/289_template_section.dts |  52 +++++
 tools/binman/test/290_mkimage_sym.dts      |  27 +++
 tools/binman/test/Makefile                 |   5 +-
 tools/binman/test/bss_data.c               |   3 +-
 tools/binman/test/bss_data_zero.c          |  16 ++
 tools/binman/test/bss_data_zero.lds        |  15 ++
 tools/binman/test/embed_data.lds           |   1 +
 tools/dtoc/fdt.py                          | 122 +++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts         |  86 ++++++++
 tools/dtoc/test_fdt.py                     |  93 +++++++++
 32 files changed, 1147 insertions(+), 68 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
 create mode 100644 tools/binman/test/283_mkimage_special.dts
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
 create mode 100644 tools/binman/test/285_spl_expand.dts
Quoted 9 lines
 create mode 100644 tools/binman/test/286_template.dts
 create mode 100644 tools/binman/test/287_template_multi.dts
 create mode 100644 tools/binman/test/288_template_fit.dts
 create mode 100644 tools/binman/test/289_template_section.dts
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts

Works much better!

What does not work yet:

/dts-v1/;

/ {
        binman: binman {
                multiple-images;

                my_template: template {
                        size = <0x100>;
                        pad-byte = <0xff>;

                        blob@0 {
                                filename = "my-blob.bin";
                        };
                };

                image {
                        filename = "my-image.bin";
                        insert-template = <&my_template>;
                };
        };
};
Quoted 1 line

The properties size and pad-byte get lost in this case.

Yes it does not copy the properties, only the subnodes. It could certainly be implemented...is that needed?

Regards,
Simon

#24 lore raw
Quoted 131 lines · Simon Glass

Hi Jan,

Quoted 128 lines · Jan Kiszka
Quoted 90 lines · Simon Glass

This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.

A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.

In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.

Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image

Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

Marek Vasut (1):
binman: Convert mkimage to Entry_section

Simon Glass (18):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image

 arch/arm/dts/stm32mp15-u-boot.dtsi         |   1 +
 tools/binman/binman.rst                    |  89 +++++++++
 tools/binman/control.py                    |  34 +++-
 tools/binman/elf.py                        |  13 +-
 tools/binman/elf_test.py                   |  13 +-
 tools/binman/entries.rst                   |   6 +
 tools/binman/entry.py                      |  11 +-
 tools/binman/etype/blob_phase.py           |   5 +
 tools/binman/etype/fit.py                  |   9 +
 tools/binman/etype/mkimage.py              | 110 ++++++++---
 tools/binman/etype/section.py              |  54 +++--
 tools/binman/etype/u_boot_spl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py   |   2 +-
 tools/binman/ftest.py                      | 218 ++++++++++++++++++++-
 tools/binman/test/282_symbols_disable.dts  |  25 +++
 tools/binman/test/283_mkimage_special.dts  |  24 +++
 tools/binman/test/284_fit_fdt_list.dts     |  58 ++++++
 tools/binman/test/285_spl_expand.dts       |  13 ++
 tools/binman/test/286_template.dts         |  42 ++++
 tools/binman/test/287_template_multi.dts   |  27 +++
 tools/binman/test/288_template_fit.dts     |  37 ++++
 tools/binman/test/289_template_section.dts |  52 +++++
 tools/binman/test/290_mkimage_sym.dts      |  27 +++
 tools/binman/test/Makefile                 |   5 +-
 tools/binman/test/bss_data.c               |   3 +-
 tools/binman/test/bss_data_zero.c          |  16 ++
 tools/binman/test/bss_data_zero.lds        |  15 ++
 tools/binman/test/embed_data.lds           |   1 +
 tools/dtoc/fdt.py                          | 122 +++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts         |  86 ++++++++
 tools/dtoc/test_fdt.py                     |  93 +++++++++
 32 files changed, 1147 insertions(+), 68 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
 create mode 100644 tools/binman/test/283_mkimage_special.dts
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
 create mode 100644 tools/binman/test/285_spl_expand.dts
Quoted 9 lines
 create mode 100644 tools/binman/test/286_template.dts
 create mode 100644 tools/binman/test/287_template_multi.dts
 create mode 100644 tools/binman/test/288_template_fit.dts
 create mode 100644 tools/binman/test/289_template_section.dts
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts

Works much better!

What does not work yet:

/dts-v1/;

/ {
        binman: binman {
                multiple-images;

                my_template: template {
                        size = <0x100>;
                        pad-byte = <0xff>;

                        blob@0 {
                                filename = "my-blob.bin";
                        };
                };

                image {
                        filename = "my-image.bin";
                        insert-template = <&my_template>;
                };
        };
};
Quoted 4 lines
Quoted 1 line

The properties size and pad-byte get lost in this case.

Yes it does not copy the properties, only the subnodes. It could certainly be implemented...is that needed?

I can duplicate the entries into the images, works but not nice. Or is it possible to pull them as defaults onto the binman level?

Jan

Signature · 2 lines
Siemens AG, Technology
Competence Center Embedded Linux
#25 lore raw

Hi Jan,

Quoted 133 lines · Jan Kiszka
Quoted 131 lines · Simon Glass

Hi Jan,

Quoted 128 lines · Jan Kiszka
Quoted 90 lines · Simon Glass

This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.

A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.

In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.

Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image

Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

Marek Vasut (1):
binman: Convert mkimage to Entry_section

Simon Glass (18):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image

 arch/arm/dts/stm32mp15-u-boot.dtsi         |   1 +
 tools/binman/binman.rst                    |  89 +++++++++
 tools/binman/control.py                    |  34 +++-
 tools/binman/elf.py                        |  13 +-
 tools/binman/elf_test.py                   |  13 +-
 tools/binman/entries.rst                   |   6 +
 tools/binman/entry.py                      |  11 +-
 tools/binman/etype/blob_phase.py           |   5 +
 tools/binman/etype/fit.py                  |   9 +
 tools/binman/etype/mkimage.py              | 110 ++++++++---
 tools/binman/etype/section.py              |  54 +++--
 tools/binman/etype/u_boot_spl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py   |   2 +-
 tools/binman/ftest.py                      | 218 ++++++++++++++++++++-
 tools/binman/test/282_symbols_disable.dts  |  25 +++
 tools/binman/test/283_mkimage_special.dts  |  24 +++
 tools/binman/test/284_fit_fdt_list.dts     |  58 ++++++
 tools/binman/test/285_spl_expand.dts       |  13 ++
 tools/binman/test/286_template.dts         |  42 ++++
 tools/binman/test/287_template_multi.dts   |  27 +++
 tools/binman/test/288_template_fit.dts     |  37 ++++
 tools/binman/test/289_template_section.dts |  52 +++++
 tools/binman/test/290_mkimage_sym.dts      |  27 +++
 tools/binman/test/Makefile                 |   5 +-
 tools/binman/test/bss_data.c               |   3 +-
 tools/binman/test/bss_data_zero.c          |  16 ++
 tools/binman/test/bss_data_zero.lds        |  15 ++
 tools/binman/test/embed_data.lds           |   1 +
 tools/dtoc/fdt.py                          | 122 +++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts         |  86 ++++++++
 tools/dtoc/test_fdt.py                     |  93 +++++++++
 32 files changed, 1147 insertions(+), 68 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
 create mode 100644 tools/binman/test/283_mkimage_special.dts
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
 create mode 100644 tools/binman/test/285_spl_expand.dts
Quoted 9 lines
 create mode 100644 tools/binman/test/286_template.dts
 create mode 100644 tools/binman/test/287_template_multi.dts
 create mode 100644 tools/binman/test/288_template_fit.dts
 create mode 100644 tools/binman/test/289_template_section.dts
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts

Works much better!

What does not work yet:

/dts-v1/;

/ {
        binman: binman {
                multiple-images;

                my_template: template {
                        size = <0x100>;
                        pad-byte = <0xff>;

                        blob@0 {
                                filename = "my-blob.bin";
                        };
                };

                image {
                        filename = "my-image.bin";
                        insert-template = <&my_template>;
                };
        };
};
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

The properties size and pad-byte get lost in this case.

Yes it does not copy the properties, only the subnodes. It could certainly be implemented...is that needed?

I can duplicate the entries into the images, works but not nice. Or is it possible to pull them as defaults onto the binman level?

Well it shouldn't be hard to implement this. I'll take a look in the morning.

Regards,
Simon

#26 lore raw

Hi Jan,

Quoted 139 lines · Simon Glass

Hi Jan,

Quoted 135 lines · Jan Kiszka
Quoted 133 lines · Simon Glass

Hi Jan,

Quoted 129 lines · Jan Kiszka
Quoted 90 lines · Simon Glass

This series converts the mkimage entry type to be a section, i.e. based on the entry_Section class. This makes it more consistent in its behaviour, e.g. allowing symbol writing and expanded entries.

A simple templating feature is also introduced, to reduce duplication when a set of entries must be used in multiple images.

In this version the nodes from the template are placed before any other nodes, meaning that the template sets the node order. This seems more consistent than other mechanisms.

Changes in v3:
- Add new patch for elf to return number of written symbols
- Add new patch with more detail on how ObtainContents() works
- Fix up some tests which now need SPL and TPL
- Avoid calling ObtainContents() when not needed
- Fix 'specific' typo
- Add a new devicetree file especially for node copying
- Correct logic for merging nodes in order
- Tidy up some comments
- Adjust to use the new example file
- Drop duplicate dts-v1 header
- Add new test case for templating in a FIT
- Add new patch to support writing symbols inside a mkimage image

Changes in v2:
- Drop now-unnecessary methods in mkimage etype
- Correct ordering of template nodes
- Fix 'preseverd' and 'inserter' typos

Marek Vasut (1):
binman: Convert mkimage to Entry_section

Simon Glass (18):
binman: Correct coverage gap in control
binman: Init align_default in entry_Section
binman: Use GetEntries() to obtain section contents
binman: Read _multiple_data_files in the correct place
binman: Allow disabling symbol writing
stm32mp15: Avoid writing symbols in SPL
binman: Update elf to return number of written symbols
binman: Add more detail on how ObtainContents() works
binman: Provide a way to specify the fdt-list directly
binman: Drop __bss_size variable in bss_data.c
binman: Correct handling of zero bss size
dtoc: Support copying the contents of a node into another
dtoc: Allow inserting a list of nodes into another
binman: Support simple templates
binman: Support templating with multiple images
binman: Add a test for templating in a FIT
binman: Support templates at any level
binman: Support writing symbols inside a mkimage image

 arch/arm/dts/stm32mp15-u-boot.dtsi         |   1 +
 tools/binman/binman.rst                    |  89 +++++++++
 tools/binman/control.py                    |  34 +++-
 tools/binman/elf.py                        |  13 +-
 tools/binman/elf_test.py                   |  13 +-
 tools/binman/entries.rst                   |   6 +
 tools/binman/entry.py                      |  11 +-
 tools/binman/etype/blob_phase.py           |   5 +
 tools/binman/etype/fit.py                  |   9 +
 tools/binman/etype/mkimage.py              | 110 ++++++++---
 tools/binman/etype/section.py              |  54 +++--
 tools/binman/etype/u_boot_spl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_tpl_bss_pad.py   |   2 +-
 tools/binman/etype/u_boot_vpl_bss_pad.py   |   2 +-
 tools/binman/ftest.py                      | 218 ++++++++++++++++++++-
 tools/binman/test/282_symbols_disable.dts  |  25 +++
 tools/binman/test/283_mkimage_special.dts  |  24 +++
 tools/binman/test/284_fit_fdt_list.dts     |  58 ++++++
 tools/binman/test/285_spl_expand.dts       |  13 ++
 tools/binman/test/286_template.dts         |  42 ++++
 tools/binman/test/287_template_multi.dts   |  27 +++
 tools/binman/test/288_template_fit.dts     |  37 ++++
 tools/binman/test/289_template_section.dts |  52 +++++
 tools/binman/test/290_mkimage_sym.dts      |  27 +++
 tools/binman/test/Makefile                 |   5 +-
 tools/binman/test/bss_data.c               |   3 +-
 tools/binman/test/bss_data_zero.c          |  16 ++
 tools/binman/test/bss_data_zero.lds        |  15 ++
 tools/binman/test/embed_data.lds           |   1 +
 tools/dtoc/fdt.py                          | 122 +++++++++++-
 tools/dtoc/test/dtoc_test_copy.dts         |  86 ++++++++
 tools/dtoc/test_fdt.py                     |  93 +++++++++
 32 files changed, 1147 insertions(+), 68 deletions(-)
 create mode 100644 tools/binman/test/282_symbols_disable.dts
 create mode 100644 tools/binman/test/283_mkimage_special.dts
 create mode 100644 tools/binman/test/284_fit_fdt_list.dts
 create mode 100644 tools/binman/test/285_spl_expand.dts
Quoted 9 lines
 create mode 100644 tools/binman/test/286_template.dts
 create mode 100644 tools/binman/test/287_template_multi.dts
 create mode 100644 tools/binman/test/288_template_fit.dts
 create mode 100644 tools/binman/test/289_template_section.dts
 create mode 100644 tools/binman/test/290_mkimage_sym.dts
 create mode 100644 tools/binman/test/bss_data_zero.c
 create mode 100644 tools/binman/test/bss_data_zero.lds
 create mode 100644 tools/dtoc/test/dtoc_test_copy.dts

Works much better!

What does not work yet:

/dts-v1/;

/ {
        binman: binman {
                multiple-images;

                my_template: template {
                        size = <0x100>;
                        pad-byte = <0xff>;

                        blob@0 {
                                filename = "my-blob.bin";
                        };
                };

                image {
                        filename = "my-image.bin";
                        insert-template = <&my_template>;
                };
        };
};

The properties size and pad-byte get lost in this case.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Yes it does not copy the properties, only the subnodes. It could certainly be implemented...is that needed?

I can duplicate the entries into the images, works but not nice. Or is it possible to pull them as defaults onto the binman level?

Well it shouldn't be hard to implement this. I'll take a look in the morning.

I sent v4 which includes support for that, so far as I understand the use case.

It is at u-boot-dm/mkim-working as well.

Regards,
Simon