Skip to content

binman: Add support for bintools and missing tools

Simon Glass u-boot 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v2 Latest38 patches 44 repliesIn review

Changes in v2 from the cover letter

- Substantial rewrite, introducing the concept of bintools

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/38Makefile: Fake external blobs by default with binmanMakefile+1 -18 replies
2/38binman: Tweak elf tests for a toolchain changetools/binman/elf_test.py+4 -41 reply
3/38mkimage: Show the external-offset errortools/fit_image.c+3 -21 reply
4/38binman: Expand the external FIT test a littletools/binman/ftest.py tools/binman/test/162_fit_external.dts+18 -11 reply
5/38patman: Allow running a tool and returning the full resulttools/patman/tools.py+46 -101 reply
6/38buildman: Move the download function to toolstools/buildman/toolchain.py tools/patman/tools.py+46 -451 reply
7/38patman: Tidy up the download function a littletools/buildman/toolchain.py tools/patman/tools.py+11 -71 reply
8/38patman: Add a function to find a tool on the pathtools/patman/tools.py+23 -01 reply
9/38binman: Write fake blobs to the output directorytools/binman/control.py tools/binman/entry.py tools/binman/etype/blob.py +1 more+26 -110 replies
10/38binman: Drop the image name from the fake-blob messagetools/binman/control.py+3 -41 reply
11/38binman: Allow faked blobs in blob-ext-listtools/binman/entry.py tools/binman/etype/blob_ext_list.py tools/binman/etype/section.py +2 more+25 -11 reply
12/38binman: Correct path for fip_utiltools/binman/fip_util_test.py+1 -11 reply
13/38binman: Add installation instructionstools/binman/binman.rst+25 -21 reply
14/38binman: Add support for bintoolstools/binman/bintool.py+421 -01 reply
15/38binman: Plumb in support for bintoolstools/binman/cmdline.py tools/binman/control.py tools/binman/entry.py +3 more+64 -11 reply
16/38binman: Add tests for bintooltools/binman/bintool_test.py tools/binman/btool/_testing.py+389 -01 reply
17/38binman: Add a bintool implementation for cbfstooltools/binman/btool/cbfstool.py+219 -01 reply
18/38binman: Add a bintool implementation for fiptooltools/binman/btool/fiptool.py+123 -01 reply
19/38binman: Add a bintool implementation for futilitytools/binman/btool/futility.py+178 -01 reply
20/38binman: Add a bintool implementation for ifwitooltools/binman/btool/ifwitool.py+166 -01 reply
21/38binman: Add a bintool implementation for mkimagetools/binman/btool/mkimage.py+80 -01 reply
22/38binman: Enable bintool tests including cmdline processingtools/binman/ftest.py tools/binman/main.py+36 -31 reply
23/38binman: Convert to using the CBFS bintooltools/binman/cbfs_util.py tools/binman/cbfs_util_test.py+19 -561 reply
24/38binman: Convert to using the FIP bintooltools/binman/fip_util.py tools/binman/fip_util_test.py+8 -411 reply
25/38binman: Convert to using the futility bintooltools/binman/etype/gbb.py tools/binman/etype/vblock.py+41 -281 reply
26/38binman: Convert to using the ifwitool bintooltools/binman/etype/intel_ifwi.py tools/binman/ftest.py tools/patman/tools.py+21 -391 reply
27/38binman: Convert to using the mkimage bintooltools/binman/etype/fit.py tools/binman/etype/mkimage.py+27 -61 reply
28/38binman: Move compression into binmantools/binman/cbfs_util.py tools/binman/comp_util.py tools/binman/entry.py +3 more+102 -891 reply
29/38binman: Tidy up pylint warnings in comp_utiltools/binman/cbfs_util.py tools/binman/comp_util.py tools/binman/entry.py +2 more+19 -201 reply
30/38binman: Add a bintool implementation for lz4tools/binman/btool/lz4.py+140 -01 reply
31/38binman: Convert to using the lz4 bintooltools/binman/cbfs_util_test.py tools/binman/comp_util.py tools/binman/ftest.py+10 -161 reply
32/38binman: Add a bintool implementation for lzma_alonetools/binman/btool/lzma_alone.py+126 -01 reply
33/38binman: Convert to using the lzma_alone bintooltools/binman/comp_util.py+5 -81 reply
34/38binman: Plumb in support for missing bintoolstools/binman/cmdline.py tools/binman/control.py tools/binman/entry.py +2 more+103 -21 reply
35/38binman: Complete test coverage of comp_utiltools/binman/comp_util.py tools/binman/ftest.py+11 -141 reply
36/38binman: Add a command to generate bintool docstools/binman/bintool.py tools/binman/cmdline.py tools/binman/control.py +2 more+80 -11 reply
37/38binman: Add documentation for bintoolsdoc/develop/package/bintools.rst tools/binman/binman.rst tools/binman/bintools.rst+187 -01 reply
38/38RFC: Move Odroid-C2 to use binman to produce the imagearch/arm/dts/meson-sm1-odroid-c4-u-boot.dtsi arch/arm/mach-meson/Kconfig doc/board/amlogic/odroid-c4.rst +10 more+868 -920 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 00/38] binman: Add support for bintools and missing tools

At present binman uses binary tools (like cbfstool, futiltiy, lz4) in an ad-hoc manner. Various parts of binman use tools.Run() to run tools as needed. If a tool is missing, an error is produced and binman stops.

However this is not ideal. CI systems want to be able to complete the build, even if tools are missing. Ideally binman would deal with missing binary tools the same way it deals with missing binary blobs: make a note of it and move on.

This series introduces this feature to binman.

Bintool is the name binman gives to a binary tool which it uses to create and manipulate binaries that binman cannot handle itself.

Binman provides various features to manage bintools:

- Determining whether the tool is currently installed
- Downloading or building the tool
- Determining the version of the tool that is installed
- Deciding which tools are needed to build an image

As with external blobs, bintools (which are like 'external' tools) can be missing. When building an image requires a bintool and it is not installed, binman detects this and reports the problem, but continues to build an image.

Of course the image will not work, but binman reports which bintools are needed and also provide a way to fetch them.

The final patch shows how this works in practice with a chosen board. The Odroid-C2 is quite a complicated image with many steps. It is an ideal example for how Binman can be used.

The series is available at u-boot-dm/bin-working

Changes in v2:
- Substantial rewrite, introducing the concept of bintools

Simon Glass (38):
Makefile: Fake external blobs by default with binman
binman: Tweak elf tests for a toolchain change
mkimage: Show the external-offset error
binman: Expand the external FIT test a little
patman: Allow running a tool and returning the full result
buildman: Move the download function to tools
patman: Tidy up the download function a little
patman: Add a function to find a tool on the path
binman: Write fake blobs to the output directory
binman: Drop the image name from the fake-blob message
binman: Allow faked blobs in blob-ext-list
binman: Correct path for fip_util
binman: Add installation instructions
binman: Add support for bintools
binman: Plumb in support for bintools
binman: Add tests for bintool
binman: Add a bintool implementation for cbfstool
binman: Add a bintool implementation for fiptool
binman: Add a bintool implementation for futility
binman: Add a bintool implementation for ifwitool
binman: Add a bintool implementation for mkimage
binman: Enable bintool tests including cmdline processing
binman: Convert to using the CBFS bintool
binman: Convert to using the FIP bintool
binman: Convert to using the futility bintool
binman: Convert to using the ifwitool bintool
binman: Convert to using the mkimage bintool
binman: Move compression into binman
binman: Tidy up pylint warnings in comp_util
binman: Add a bintool implementation for lz4
binman: Convert to using the lz4 bintool
binman: Add a bintool implementation for lzma_alone
binman: Convert to using the lzma_alone bintool
binman: Plumb in support for missing bintools
binman: Complete test coverage of comp_util
binman: Add a command to generate bintool docs
binman: Add documentation for bintools
RFC: Move Odroid-C2 to use binman to produce the image

 Makefile                                     |   2 +-
 arch/arm/dts/meson-sm1-odroid-c4-u-boot.dtsi | 107 +++++
 arch/arm/mach-meson/Kconfig                  |   1 +
 doc/board/amlogic/odroid-c4.rst              | 127 ++---
 doc/develop/package/bintools.rst             |   1 +
 tools/binman/binman.rst                      |  98 +++-
 tools/binman/bintool.py                      | 466 +++++++++++++++++++
 tools/binman/bintool_test.py                 | 353 ++++++++++++++
 tools/binman/bintools.rst                    | 139 ++++++
 tools/binman/btool/_aml_common.py            |  47 ++
 tools/binman/btool/_testing.py               |  36 ++
 tools/binman/btool/aml_encrypt_g12a.py       |  82 ++++
 tools/binman/btool/aml_encrypt_g12b.py       |  83 ++++
 tools/binman/btool/cbfstool.py               | 219 +++++++++
 tools/binman/btool/fiptool.py                | 123 +++++
 tools/binman/btool/futility.py               | 178 +++++++
 tools/binman/btool/ifwitool.py               | 166 +++++++
 tools/binman/btool/lz4.py                    | 140 ++++++
 tools/binman/btool/lzma_alone.py             | 126 +++++
 tools/binman/btool/mkimage.py                |  80 ++++
 tools/binman/cbfs_util.py                    |  33 +-
 tools/binman/cbfs_util_test.py               |  59 +--
 tools/binman/cmdline.py                      |  12 +
 tools/binman/comp_util.py                    |  76 +++
 tools/binman/control.py                      |  50 +-
 tools/binman/elf_test.py                     |   8 +-
 tools/binman/entries.rst                     | 138 ++++++
 tools/binman/entry.py                        |  67 ++-
 tools/binman/etype/aml_encrypt.py            | 262 +++++++++++
 tools/binman/etype/blob.py                   |   8 +-
 tools/binman/etype/blob_ext_list.py          |   1 +
 tools/binman/etype/fit.py                    |  20 +-
 tools/binman/etype/gbb.py                    |  37 +-
 tools/binman/etype/intel_ifwi.py             |  25 +-
 tools/binman/etype/mkimage.py                |  13 +-
 tools/binman/etype/section.py                |  19 +-
 tools/binman/etype/vblock.py                 |  32 +-
 tools/binman/fip_util.py                     |  26 --
 tools/binman/fip_util_test.py                |  25 +-
 tools/binman/ftest.py                        | 173 ++++++-
 tools/binman/image.py                        |  14 +
 tools/binman/main.py                         |  11 +-
 tools/binman/missing-blob-help               |   6 +
 tools/binman/test/162_fit_external.dts       |   2 +-
 tools/binman/test/213_aml_encrypt.dts        |  51 ++
 tools/binman/test/214_list_no_dtb.dts        |  23 +
 tools/binman/test/218_blob_ext_list_fake.dts |  14 +
 tools/buildman/toolchain.py                  |  46 +-
 tools/fit_image.c                            |   5 +-
 tools/patman/tools.py                        | 238 +++++-----
 50 files changed, 3619 insertions(+), 449 deletions(-)
 create mode 120000 doc/develop/package/bintools.rst
 create mode 100644 tools/binman/bintool.py
 create mode 100644 tools/binman/bintool_test.py
 create mode 100644 tools/binman/bintools.rst
 create mode 100644 tools/binman/btool/_aml_common.py
 create mode 100644 tools/binman/btool/_testing.py
 create mode 100644 tools/binman/btool/aml_encrypt_g12a.py
 create mode 100644 tools/binman/btool/aml_encrypt_g12b.py
 create mode 100644 tools/binman/btool/cbfstool.py
 create mode 100644 tools/binman/btool/fiptool.py
 create mode 100644 tools/binman/btool/futility.py
 create mode 100644 tools/binman/btool/ifwitool.py
 create mode 100644 tools/binman/btool/lz4.py
 create mode 100644 tools/binman/btool/lzma_alone.py
 create mode 100644 tools/binman/btool/mkimage.py
 create mode 100644 tools/binman/comp_util.py
 create mode 100644 tools/binman/etype/aml_encrypt.py
 create mode 100644 tools/binman/test/213_aml_encrypt.dts
 create mode 100644 tools/binman/test/214_list_no_dtb.dts
 create mode 100644 tools/binman/test/218_blob_ext_list_fake.dts
Signature · 1 line
2.34.1.575.g55b058a8bb-goog