binman: Show missing blob message when building U-Boot
v2 cover binman: Show missing blob message when building U-Boot
binman currently support showing a helpful missing blob message, however only when the badly named --allow-missing flag is used.
This changes so that binman is invoked with the --allow-missing flag and the helpful message can be shown by default when building U-Boot.
Using the following:
make rockpro64-rk3399_defconfig
make CROSS_COMPILE="aarch64-linux-gnu-"
Before this series a build fails with:
binman: Filename 'atf-bl31' not found in input path (...)
After this series a build fails with:
Image 'simple-bin' is missing external blobs and is non-functional: atf-bl31
/binman/simple-bin/fit/images/@atf-SEQ/atf-bl31 (atf-bl31):
See the documentation for your board. You may need to build ARM Trusted Firmware and build with BL31=/path/to/bl31.bin
Image 'simple-bin' is missing optional external blobs but is still functional: tee-os
/binman/simple-bin/fit/images/@tee-SEQ/tee-os (tee-os):
See the documentation for your board. You may need to build Open Portable Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.bin
Some images are invalid
Builds will continue to fail when there is missing blobs, and the use of BINMAN_ALLOW_MISSING=1 now only enables the --ignore-missing flag.
Changes in v2:
- Drop merged patches.
- Add patches to improve help text.
- Add tests.
- Collect r-b tags.
In v1 there was discussion regarding the name of binman --ignore-missing flag, see [1]. The flag is kept as-is in v2 and a rename or change of default behavior is something that someone can do in a follow up series.
The series is based on top of Simon's "binman: Simple templating feature and mkimage conversion" v5 series. This series can also be found at [2].
[1] https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
[2] https://github.com/Kwiboo/u-boot-rockchip/commits/binman-missing-v2
Jonas Karlman (7):
binman: Update tee-os missing blob help text
binman: Update missing optional external blob warning text
binman: Report missing external blobs using error level
binman: Override CheckOptional in fit entry
binman: Fix blank line usage for invalid images warning text
binman: Show filename in missing blob help message
Makefile: Show binman missing blob message
Makefile | 2 +- tools/binman/control.py | 34 +++++++++++++++--------- tools/binman/etype/fit.py | 7 +++++ tools/binman/ftest.py | 11 +++++++- tools/binman/missing-blob-help | 2 +- tools/binman/test/264_tee_os_opt_fit.dts | 1 + 6 files changed, 41 insertions(+), 16 deletions(-)
Signature · 1 line
2.41.0
PATCH v2 1/7 binman: Update tee-os missing blob help text
Make it a little bit more clear that it is U-Boot that should be built with TEE=/path/to/tee.bin and not OP-TEE itself.
---
v2:
- New patch
tools/binman/missing-blob-help | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/binman/missing-blob-help +1 -1
@@ -43,7 +43,7 @@ for the external TPL binary is https://github.com/rockchip-linux/rkbin. tee-os: See the documentation for your board. You may need to build Open Portable -Trusted Execution Environment (OP-TEE) with TEE=/path/to/tee.bin +Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.bin opensbi: See the documentation for your board. The OpenSBI git repo is at
Signature · 1 line
2.41.0
Re: [PATCH v2 1/7] binman: Update tee-os missing blob help text
Quoted 11 lines · Jonas Karlman
Make it a little bit more clear that it is U-Boot that should be built with TEE=/path/to/tee.bin and not OP-TEE itself.
Signed-off-by Jonas Karlman <[email protected]>---
v2:
- New patchtools/binman/missing-blob-help | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Simon Glass
Reviewed-byRe: [PATCH v2 1/7] binman: Update tee-os missing blob help text
Quoted 11 lines · Jonas Karlman
Make it a little bit more clear that it is U-Boot that should be built with TEE=/path/to/tee.bin and not OP-TEE itself.
Signed-off-by Jonas Karlman <[email protected]>---
v2:
- New patchtools/binman/missing-blob-help | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Applied to u-boot-dm, thanks!
PATCH v2 2/7 binman: Update missing optional external blob warning text
In reply to Jonas KarlmanMake it more clear that the missing external blob is optional in the printed warning message.
---
v2:
- New patch
tools/binman/control.py | 2 +- tools/binman/ftest.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/binman/control.py +1 -1
@@ -674,7 +674,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, image.CheckOptional(optional_list) if optional_list: tout.warning( - "Image '%s' is missing external blobs but is still functional: %s" % + "Image '%s' is missing optional external blobs but is still functional: %s" % (image.name, ' '.join([e.name for e in optional_list]))) _ShowHelpForMissingBlobs(optional_list)
tools/binman/ftest.py +1 -1
@@ -6330,7 +6330,7 @@ fdt fdtmap Extract the devicetree blob from the fdtmap err = stderr.getvalue() self.assertRegex( err, - "Image '.*' is missing external blobs but is still functional: missing") + "Image '.*' is missing optional external blobs but is still functional: missing") def testSectionInner(self): """Test an inner section with a size"""
Signature · 1 line
2.41.0
Re: [PATCH v2 2/7] binman: Update missing optional external blob warning text
Quoted 40 lines · Jonas Karlman
Make it more clear that the missing external blob is optional in the printed warning message.
Signed-off-by Jonas Karlman <[email protected]>---
v2:
- New patchtools/binman/control.py | 2 +- tools/binman/ftest.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)tools/binman/control.py +1
-1@@ -674,7 +674,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, image.CheckOptional(optional_list) if optional_list: tout.warning( - "Image '%s' is missing external blobs but is still functional: %s" % + "Image '%s' is missing optional external blobs but is still functional: %s" % (image.name, ' '.join([e.name for e in optional_list]))) _ShowHelpForMissingBlobs(optional_list)tools/binman/ftest.py +1
-2@@ -6330,7 +6330,7 @@ fdt fdtmap Extract the devicetree blob from the fdtmap err = stderr.getvalue() self.assertRegex( err, - "Image '.*' is missing external blobs but is still functional: missing") + "Image '.*' is missing optional external blobs but is still functional: missing") def testSectionInner(self): """Test an inner section with a size""" --
Quoted 2 lines
2.41.0
Re: [PATCH v2 2/7] binman: Update missing optional external blob warning text
Quoted 4 lines · Jonas Karlman
Make it more clear that the missing external blob is optional in the printed warning message.
Quoted 9 lines
Signed-off-by Jonas Karlman <[email protected]>---
v2:
- New patchtools/binman/control.py | 2 +- tools/binman/ftest.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
Applied to u-boot-dm, thanks!
PATCH v2 3/7 binman: Report missing external blobs using error level
In reply to Jonas KarlmanPrint missing external blobs using error level and missing optional external blobs using warning level. Also change to only print the header line in color, red for missing and yellow for optional.
---
v2:
- New patch
tools/binman/control.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tools/binman/control.py +9 -9
@@ -112,12 +112,12 @@ def _ReadMissingBlobHelp(): _FinishTag(tag, msg, result) return result -def _ShowBlobHelp(path, text): - tout.warning('\n%s:' % path) +def _ShowBlobHelp(level, path, text): + tout.do_output(level, '\n%s:' % path) for line in text.splitlines(): - tout.warning(' %s' % line) + tout.do_output(level, ' %s' % line) -def _ShowHelpForMissingBlobs(missing_list): +def _ShowHelpForMissingBlobs(level, missing_list): """Show help for each missing blob to help the user take action Args:
@@ -134,7 +134,7 @@ def _ShowHelpForMissingBlobs(missing_list): # Show the first match help message for tag in tags: if tag in missing_blob_help: - _ShowBlobHelp(entry._node.path, missing_blob_help[tag]) + _ShowBlobHelp(level, entry._node.path, missing_blob_help[tag]) break def GetEntryModules(include_testing=True):
@@ -658,9 +658,9 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, missing_list = [] image.CheckMissing(missing_list) if missing_list: - tout.warning("Image '%s' is missing external blobs and is non-functional: %s" % - (image.name, ' '.join([e.name for e in missing_list]))) - _ShowHelpForMissingBlobs(missing_list) + tout.error("Image '%s' is missing external blobs and is non-functional: %s" % + (image.name, ' '.join([e.name for e in missing_list]))) + _ShowHelpForMissingBlobs(tout.ERROR, missing_list) faked_list = [] image.CheckFakedBlobs(faked_list)
@@ -676,7 +676,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, tout.warning( "Image '%s' is missing optional external blobs but is still functional: %s" % (image.name, ' '.join([e.name for e in optional_list]))) - _ShowHelpForMissingBlobs(optional_list) + _ShowHelpForMissingBlobs(tout.WARNING, optional_list) missing_bintool_list = [] image.check_missing_bintools(missing_bintool_list)
Signature · 1 line
2.41.0
Re: [PATCH v2 3/7] binman: Report missing external blobs using error level
Quoted 5 lines · Jonas Karlman
Print missing external blobs using error level and missing optional external blobs using warning level. Also change to only print the header line in color, red for missing and yellow for optional.
Quoted 7 lines
Signed-off-by Jonas Karlman <[email protected]>---
v2:
- New patchtools/binman/control.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
Simon Glass
Reviewed-byRe: [PATCH v2 3/7] binman: Report missing external blobs using error level
Quoted 5 lines · Jonas Karlman
Print missing external blobs using error level and missing optional external blobs using warning level. Also change to only print the header line in color, red for missing and yellow for optional.
Quoted 7 lines
Signed-off-by Jonas Karlman <[email protected]>---
v2:
- New patchtools/binman/control.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
Applied to u-boot-dm, thanks!
PATCH v2 4/7 binman: Override CheckOptional in fit entry
In reply to Jonas KarlmanMissing optional blobs was not reported for generated entries, e.g. tee-os on rockchip targets. Implement a CheckOptional to fix this.
After this the following can be shown:
Image 'simple-bin' is missing optional external blobs but is still functional: tee-os
/binman/simple-bin/fit/images/@tee-SEQ/tee-os (tee-os):
See the documentation for your board. You may need to build Open Portable Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.bin
---
v2:
- Add test
- Collect r-b tag
tools/binman/etype/fit.py | 7 +++++++ tools/binman/ftest.py | 7 +++++++ tools/binman/test/264_tee_os_opt_fit.dts | 1 + 3 files changed, 15 insertions(+)
tools/binman/etype/fit.py +7 -0
@@ -842,6 +842,13 @@ class Entry_fit(Entry_section): for entry in self._priv_entries.values(): entry.CheckMissing(missing_list) + def CheckOptional(self, optional_list): + # We must use our private entry list for this since generator nodes + # which are removed from self._entries will otherwise not show up as + # optional + for entry in self._priv_entries.values(): + entry.CheckOptional(optional_list) + def CheckEntries(self): pass
tools/binman/ftest.py +7 -0
@@ -6298,6 +6298,13 @@ fdt fdtmap Extract the devicetree blob from the fdtmap fdt_util.fdt32_to_cpu(node.props['entry'].value)) self.assertEqual(U_BOOT_DATA, node.props['data'].bytes) + with test_util.capture_sys_output() as (stdout, stderr): + self.checkFitTee('264_tee_os_opt_fit.dts', '') + err = stderr.getvalue() + self.assertRegex( + err, + "Image '.*' is missing optional external blobs but is still functional: tee-os") + def testFitTeeOsOptionalFitBad(self): """Test an image with a FIT with an optional OP-TEE binary""" with self.assertRaises(ValueError) as exc:
tools/binman/test/264_tee_os_opt_fit.dts +1 -0
@@ -25,6 +25,7 @@ fit,data; tee-os { + optional; }; }; };
Signature · 1 line
2.41.0
Re: [PATCH v2 4/7] binman: Override CheckOptional in fit entry
Missing optional blobs was not reported for generated entries, e.g. tee-os on rockchip targets. Implement a CheckOptional to fix this.
After this the following can be shown:
Image 'simple-bin' is missing optional external blobs but is still
functional: tee-os
/binman/simple-bin/fit/images/@tee-SEQ/tee-os (tee-os):
See the documentation for your board. You may need to build Open Portable Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.bin
---
v2:
- Add test
- Collect r-b tag
tools/binman/etype/fit.py | 7 +++++++ tools/binman/ftest.py | 7 +++++++ tools/binman/test/264_tee_os_opt_fit.dts | 1 + 3 files changed, 15 insertions(+)
Applied to u-boot-dm, thanks!
PATCH v2 5/7 binman: Fix blank line usage for invalid images warning text
In reply to Jonas KarlmanThere is no blank line between last missing blob help message and the header line for optional blob help messages.
Image 'simple-bin' is missing external blobs and is non-functional: atf-bl31
/binman/simple-bin/fit/images/@atf-SEQ/atf-bl31:
See the documentation for your board. You may need to build ARM Trusted Firmware and build with BL31=/path/to/bl31.bin
Image 'simple-bin' is missing external blobs but is still functional: tee-os
/binman/simple-bin/fit/images/@tee-SEQ/tee-os:
See the documentation for your board. You may need to build Open Portable Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.bin
Some images are invalid
With this a blank line is inserted to make the text more readable.
---
v2:
- Collect r-b tag
tools/binman/control.py | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
tools/binman/control.py +7 -6
@@ -113,9 +113,10 @@ def _ReadMissingBlobHelp(): return result def _ShowBlobHelp(level, path, text): - tout.do_output(level, '\n%s:' % path) + tout.do_output(level, '%s:' % path) for line in text.splitlines(): tout.do_output(level, ' %s' % line) + tout.do_output(level, '') def _ShowHelpForMissingBlobs(level, missing_list): """Show help for each missing blob to help the user take action
@@ -658,7 +659,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, missing_list = [] image.CheckMissing(missing_list) if missing_list: - tout.error("Image '%s' is missing external blobs and is non-functional: %s" % + tout.error("Image '%s' is missing external blobs and is non-functional: %s\n" % (image.name, ' '.join([e.name for e in missing_list]))) _ShowHelpForMissingBlobs(tout.ERROR, missing_list)
@@ -666,7 +667,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, image.CheckFakedBlobs(faked_list) if faked_list: tout.warning( - "Image '%s' has faked external blobs and is non-functional: %s" % + "Image '%s' has faked external blobs and is non-functional: %s\n" % (image.name, ' '.join([os.path.basename(e.GetDefaultFilename()) for e in faked_list])))
@@ -674,7 +675,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, image.CheckOptional(optional_list) if optional_list: tout.warning( - "Image '%s' is missing optional external blobs but is still functional: %s" % + "Image '%s' is missing optional external blobs but is still functional: %s\n" % (image.name, ' '.join([e.name for e in optional_list]))) _ShowHelpForMissingBlobs(tout.WARNING, optional_list)
@@ -682,7 +683,7 @@ def ProcessImage(image, update_fdt, write_map, get_contents=True, image.check_missing_bintools(missing_bintool_list) if missing_bintool_list: tout.warning( - "Image '%s' has missing bintools and is non-functional: %s" % + "Image '%s' has missing bintools and is non-functional: %s\n" % (image.name, ' '.join([os.path.basename(bintool.name) for bintool in missing_bintool_list]))) return any([missing_list, faked_list, missing_bintool_list])
@@ -827,7 +828,7 @@ def Binman(args): # This can only be True if -M is provided, since otherwise binman # would have raised an error already if invalid: - msg = '\nSome images are invalid' + msg = 'Some images are invalid' if args.ignore_missing: tout.warning(msg) else:
Signature · 1 line
2.41.0
Re: [PATCH v2 5/7] binman: Fix blank line usage for invalid images warning text
Hi Jonas,
Quoted 17 lines · Jonas Karlman
There is no blank line between last missing blob help message and the header line for optional blob help messages.
Image 'simple-bin' is missing external blobs and is non-functional: atf-bl31
/binman/simple-bin/fit/images/@atf-SEQ/atf-bl31:
See the documentation for your board. You may need to build ARM Trusted Firmware and build with BL31=/path/to/bl31.bin
Image 'simple-bin' is missing external blobs but is still functional: tee-os/binman/simple-bin/fit/images/@tee-SEQ/tee-os:
See the documentation for your board. You may need to build Open Portable Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.binSome images are invalid
Quoted 7 lines
With this a blank line is inserted to make the text more readable.
Signed-off-by Jonas Karlman <[email protected]>Reviewed-by Simon Glass <[email protected]>---
v2:
- Collect r-b tag
IMO you don't need to do this...just collect them and say 'no changes'.
'patman status' is your friend.
Regards,
Simon
Re: [PATCH v2 5/7] binman: Fix blank line usage for invalid images warning text
Hi Jonas,
Quoted 17 lines · Jonas Karlman
There is no blank line between last missing blob help message and the header line for optional blob help messages.
Image 'simple-bin' is missing external blobs and is non-functional: atf-bl31
/binman/simple-bin/fit/images/@atf-SEQ/atf-bl31:
See the documentation for your board. You may need to build ARM Trusted Firmware and build with BL31=/path/to/bl31.bin
Image 'simple-bin' is missing external blobs but is still functional: tee-os/binman/simple-bin/fit/images/@tee-SEQ/tee-os:
See the documentation for your board. You may need to build Open Portable Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.binSome images are invalid
Quoted 7 lines
With this a blank line is inserted to make the text more readable.
Signed-off-by Jonas Karlman <[email protected]>Reviewed-by Simon Glass <[email protected]>---
v2:
- Collect r-b tag
IMO you don't need to do this...just collect them and say 'no changes'.
'patman status' is your friend.
Regards,
Simon
Applied to u-boot-dm, thanks!
PATCH v2 6/7 binman: Show filename in missing blob help message
In reply to Jonas KarlmanShow the filename next to the node path in missing blob help messages, also show a generic missing blob message when there was no help message for the help tag.
---
v2:
- Add test
- Collect r-b tag
tools/binman/control.py | 13 ++++++++++--- tools/binman/ftest.py | 2 ++ 2 files changed, 12 insertions(+), 3 deletions(-)
tools/binman/control.py +10 -3
@@ -112,8 +112,8 @@ def _ReadMissingBlobHelp(): _FinishTag(tag, msg, result) return result -def _ShowBlobHelp(level, path, text): - tout.do_output(level, '%s:' % path) +def _ShowBlobHelp(level, path, text, fname): + tout.do_output(level, '%s (%s):' % (path, fname)) for line in text.splitlines(): tout.do_output(level, ' %s' % line) tout.do_output(level, '')
@@ -133,10 +133,17 @@ def _ShowHelpForMissingBlobs(level, missing_list): tags = entry.GetHelpTags() # Show the first match help message + shown_help = False for tag in tags: if tag in missing_blob_help: - _ShowBlobHelp(level, entry._node.path, missing_blob_help[tag]) + _ShowBlobHelp(level, entry._node.path, missing_blob_help[tag], + entry.GetDefaultFilename()) + shown_help = True break + # Or a generic help message + if not shown_help: + _ShowBlobHelp(level, entry._node.path, "Missing blob", + entry.GetDefaultFilename()) def GetEntryModules(include_testing=True): """Get a set of entry class implementations
tools/binman/ftest.py +2 -0
@@ -3781,6 +3781,7 @@ class TestFunctional(unittest.TestCase): allow_missing=True) self.assertEqual(103, ret) err = stderr.getvalue() + self.assertIn('(missing-file)', err) self.assertRegex(err, "Image 'image'.*missing.*: blob-ext") self.assertIn('Some images are invalid', err)
@@ -3791,6 +3792,7 @@ class TestFunctional(unittest.TestCase): allow_missing=True, ignore_missing=True) self.assertEqual(0, ret) err = stderr.getvalue() + self.assertIn('(missing-file)', err) self.assertRegex(err, "Image 'image'.*missing.*: blob-ext") self.assertIn('Some images are invalid', err)
Signature · 1 line
2.41.0
Re: [PATCH v2 6/7] binman: Show filename in missing blob help message
Show the filename next to the node path in missing blob help messages, also show a generic missing blob message when there was no help message for the help tag.
---
v2:
- Add test
- Collect r-b tag
tools/binman/control.py | 13 ++++++++++--- tools/binman/ftest.py | 2 ++ 2 files changed, 12 insertions(+), 3 deletions(-)
Applied to u-boot-dm, thanks!
PATCH v2 7/7 Makefile: Show binman missing blob message
In reply to Jonas KarlmanWhen binman is invoked during a build of U-Boot and an external blob is missing, the user is usually presented with a generic file not found in input path message.
Invoke binman with --allow-missing so that binman can show relevant missing blob help messages. Build continue to fail with missing blobs unless BINMAN_ALLOW_MISSING=1 is used, same as before.
This changes the following error message during a normal build:
binman: Filename 'atf-bl31' not found in input path (...)
to the following:
Image 'itb' is missing external blobs and is non-functional: atf-blob
/binman/itb/fit/images/atf/atf-blob (bl31.bin):
See the documentation for your board. You may need to build ARM Trusted Firmware and build with BL31=/path/to/bl31.bin
---
v2:
- Collect r-b tag
Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
@@ -1334,7 +1334,7 @@ cmd_binman = $(srctree)/tools/binman/binman $(if $(BINMAN_DEBUG),-D) \ --toolpath $(objtree)/tools \ $(if $(BINMAN_VERBOSE),-v$(BINMAN_VERBOSE)) \ build -u -d u-boot.dtb -O . -m \ - $(if $(BINMAN_ALLOW_MISSING),--allow-missing --ignore-missing) \ + --allow-missing $(if $(BINMAN_ALLOW_MISSING),--ignore-missing) \ -I . -I $(srctree) -I $(srctree)/board/$(BOARDDIR) \ -I arch/$(ARCH)/dts -a of-list=$(CONFIG_OF_LIST) \ $(foreach f,$(BINMAN_INDIRS),-I $(f)) \
Signature · 1 line
2.41.0
Re: [PATCH v2 7/7] Makefile: Show binman missing blob message
When binman is invoked during a build of U-Boot and an external blob is missing, the user is usually presented with a generic file not found in input path message.
Invoke binman with --allow-missing so that binman can show relevant missing blob help messages. Build continue to fail with missing blobs unless BINMAN_ALLOW_MISSING=1 is used, same as before.
This changes the following error message during a normal build:
binman: Filename 'atf-bl31' not found in input path (...)
to the following:
Image 'itb' is missing external blobs and is non-functional: atf-blob
/binman/itb/fit/images/atf/atf-blob (bl31.bin):
See the documentation for your board. You may need to build ARM Trusted Firmware and build with BL31=/path/to/bl31.bin
---
v2:
- Collect r-b tag
Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Applied to u-boot-dm, thanks!