Skip to contentSkip to conversation

produce working binaries by default

RFC PATCH New Petr Štetiar 4 participants 16 messages Mar 18, 2020 – Mar 31, 2020

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

cover produce working binaries by default

Currently its not possible to distinguish between normal builds and builds performed by the build bots/CI, thus leading to a workarounds like for example in 4c78028737c3 ("mksunxi_fit_atf.sh: Allow for this to complete when bl31.bin is missing"), where producing unusable binaries is prefered in favor of a green automatic builds.

So lets try to fix this properly, add BUILDBOT config options which could be set on the build bots/CI and the codebase can use this new config option to workaround the issues in more clear manner.

Petr ?tetiar (9):
Kconfig: add config options for automatic builds
Makefile: export config options for automatic builds mksunxi_fit_atf.sh: produce working binaries by default make_fit_atf.py: produce working binaries by default fit_spl_optee.sh: produce working binaries by default mkimage_fit_atf.sh: produce working binaries by default fit_spl_atf.sh: produce working binaries by default k3_fit_atf.sh: produce working binaries by default mkimage_fit_opensbi.sh: produce working binaries by default

 Kconfig                                       | 12 +++++++++++
 Makefile                                      |  5 +++++
 arch/arm/mach-rockchip/fit_spl_optee.sh       | 10 +++++++---
 arch/arm/mach-rockchip/make_fit_atf.py        |  9 ++++++---
 arch/arm/mach-zynqmp/mkimage_fit_atf.sh       | 14 ++++++++-----
 arch/riscv/lib/mkimage_fit_opensbi.sh         |  8 ++++++--
 board/sunxi/mksunxi_fit_atf.sh                | 10 +++++++---
 .../puma_rk3399/fit_spl_atf.sh                | 20 +++++++++++++------
 tools/k3_fit_atf.sh                           | 16 +++++++++++----
 9 files changed, 78 insertions(+), 26 deletions(-)
#1 lore raw

RFC PATCH 1/9 Kconfig: add config options for automatic builds

Currently its not possible to distinguish between normal builds and builds performed by the build bots/CI, thus leading to a workarounds like for example in 4c78028737c3 ("mksunxi_fit_atf.sh: Allow for this to complete when bl31.bin is missing"), where producing unusable binaries is prefered in favor of a green automatic builds.

So lets try to fix this properly, add BUILDBOT config options which could be set on the build bots/CI and the codebase can use this new config option to workaround the issues in more clear manner.

---
 Kconfig | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Kconfig +12 -0
index 66148ce47790..24960cf7abbf 100644
@@ -20,6 +20,18 @@ config BROKEN
 	  This option cannot be enabled. It is used as dependency
 	  for broken and incomplete features.
 
+config BUILDBOT
+	bool "Set build defaults for automatic builds"
+	help
+	  This option allows setting of usable defaults for automatic builds.
+
+config BUILDBOT_BROKEN_BINARIES
+	bool "Allow building of broken binaries"
+	depends on BUILDBOT
+	help
+	  Resulting images wont be used for runtime testing, thus completition
+	  of build is preferred.
+
 config DEPRECATED
 	bool
 	help
#2 lore raw

RFC PATCH 2/9 Makefile: export config options for automatic builds

In reply to Petr Štetiar

Make config options related to build bots accessible within the scripts.

---
 Makefile | 5 +++++
 1 file changed, 5 insertions(+)
Makefile +5 -0
index fa687f13a588..44776b8efcc4 100644
@@ -426,6 +426,11 @@ KBUILD_AFLAGS	+= $(call cc-option,-fno-PIE)
 UBOOTRELEASE = $(shell cat include/config/uboot.release 2> /dev/null)
 UBOOTVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
 
+BUILDBOT = $(CONFIG_BUILDBOT)
+BUILDBOT_BROKEN_BINARIES = $(CONFIG_BUILDBOT_BROKEN_BINARIES)
+
+export BUILDBOT BUILDBOT_BROKEN_BINARIES
+
 export VERSION PATCHLEVEL SUBLEVEL UBOOTRELEASE UBOOTVERSION
 export ARCH CPU BOARD VENDOR SOC CPUDIR BOARDDIR
 export CONFIG_SHELL HOSTCC HOSTCFLAGS HOSTLDFLAGS CROSS_COMPILE AS LD CC
#3 lore raw

RFC PATCH 3/9 mksunxi_fit_atf.sh: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if bl31.bin file is missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 board/sunxi/mksunxi_fit_atf.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
board/sunxi/mksunxi_fit_atf.sh +7 -3
index 88ad71974706..708b4248549d 100755
@@ -8,9 +8,13 @@
 [ -z "$BL31" ] && BL31="bl31.bin"
 
 if [ ! -f $BL31 ]; then
-	echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
-	echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2
-	BL31=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		BL31=/dev/null
+	else
+		echo "ERROR: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
+		echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2
+		exit 1
+	fi
 fi
 
 if grep -q "^CONFIG_MACH_SUN50I_H6=y" .config; then
#4 lore raw

RFC PATCH 3/9 mksunxi_fit_atf.sh: produce working binaries by default

Hi,

Quoted 16 lines · Petr ?tetiar

At this moment unusable binaries are produced if bl31.bin file is missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 board/sunxi/mksunxi_fit_atf.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
board/sunxi/mksunxi_fit_atf.sh +0 -0
index 88ad71974706..708b4248549d 100755

board/sunxi/mksunxi_fit_atf.sh line 14

Quoted 10 lines
board/sunxi/mksunxi_fit_atf.sh +4 -3
@@ -8,9 +8,13 @@
 [ -z "$BL31" ] && BL31="bl31.bin"
 
 if [ ! -f $BL31 ]; then
-	echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
-	echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2
-	BL31=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		BL31=/dev/null
+	else
+		echo "ERROR: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2

This error message is not quite correct anymore because no binary is built now.

board/sunxi/mksunxi_fit_atf.sh line 15

Quoted 1 line
board/sunxi/mksunxi_fit_atf.sh +1 -0
+		echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2

Not only relevant to this patch, but to the following ones as well: Maybe it makes sense to mention CONFIG_BUILDBOT_BROKEN_BINARIES in the error message so people who are not aware of this change and want to continue building broken binaries don't have to search the list archive to find out what to do? For example:

+ echo "ERROR: BL31 file $BL31 NOT found." >&2
+ echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2
+ echo "" >&2
+ echo "If you want to build without a BL31 file (creating a NON-FUNCTIONAL binary), please" >&2
+ echo "enable CONFIG_BUILDBOT_BROKEN_BINARIES." >&2

Quoted 5 lines
board/sunxi/mksunxi_fit_atf.sh +2 -0
+		exit 1
+	fi
 fi
 
 if grep -q "^CONFIG_MACH_SUN50I_H6=y" .config; then
Signature · 1 line
Harald
#5 lore raw

RFC PATCH 4/9 make_fit_atf.py: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if bl31.elf file is missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 arch/arm/mach-rockchip/make_fit_atf.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
arch/arm/mach-rockchip/make_fit_atf.py +6 -3
index d15c32b30329..4c55a87b51f2 100755
@@ -196,17 +196,20 @@ def unpack_elf(filename):
 def main():
     uboot_elf = "./u-boot"
     fit_its = sys.stdout
+    broken_binaries = os.getenv("BUILDBOT_BROKEN_BINARIES") == "y"
     if "BL31" in os.environ:
         bl31_elf=os.getenv("BL31");
     elif os.path.isfile("./bl31.elf"):
         bl31_elf = "./bl31.elf"
-    else:
+    elif broken_binaries:
         os.system("echo 'int main(){}' > bl31.c")
         os.system("${CROSS_COMPILE}gcc -c bl31.c -o bl31.elf")
         bl31_elf = "./bl31.elf"
+    else:
         logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
-        logging.warning(' BL31 file bl31.elf NOT found, resulting binary is non-functional')
-        logging.warning(' Please read Building section in doc/README.rockchip')
+        logging.error(' BL31 file bl31.elf NOT found, resulting binary would be non-functional')
+        logging.error(' Please read Building section in doc/README.rockchip')
+        sys.exit(1)
 
     if "TEE" in os.environ:
         tee_elf = os.getenv("TEE")
#6 lore raw

RFC PATCH 5/9 fit_spl_optee.sh: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if tee.bin file is missing in order to allow passing of various CI tests. This intention of broken images has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 arch/arm/mach-rockchip/fit_spl_optee.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
arch/arm/mach-rockchip/fit_spl_optee.sh +7 -3
index 4118472d9f22..bdf61c4ba16d 100755
@@ -11,9 +11,13 @@
 [ -z "$TEE" ] && TEE="tee.bin"
 
 if [ ! -f $TEE ]; then
-	echo "WARNING: TEE file $TEE NOT found, U-Boot.itb is non-functional" >&2
-	echo "Please export path for TEE or copy tee.bin to U-Boot folder" >&2
-	TEE=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		TEE=/dev/null
+	else
+		echo "ERROR: TEE file $TEE NOT found, U-Boot.itb is non-functional" >&2
+		echo "Please export path for TEE or copy tee.bin to U-Boot folder" >&2
+		exit 1
+	fi
 fi
 
 dtname=$1
#7 lore raw

RFC PATCH 6/9 mkimage_fit_atf.sh: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if bl31.bin file is missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 arch/arm/mach-zynqmp/mkimage_fit_atf.sh | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
arch/arm/mach-zynqmp/mkimage_fit_atf.sh +9 -5
index 1e770ba111d3..5effe05abdee 100755
@@ -29,11 +29,15 @@ else
 fi
 
 if [ ! -f $BL31 ]; then
-	echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
-	BL31=/dev/null
-	# But U-Boot proper could be loaded in EL3 by specifying
-	# firmware = "uboot";
-	# instead of "atf" in config node
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		BL31=/dev/null
+		# But U-Boot proper could be loaded in EL3 by specifying
+		# firmware = "uboot";
+		# instead of "atf" in config node
+	else
+		echo "ERROR: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
+		exit 1
+	fi
 fi
 
 cat << __HEADER_EOF
#8 lore raw

RFC PATCH 6/9 mkimage_fit_atf.sh: produce working binaries by default

Hi Petr,

Quoted 27 lines · Petr ?tetiar

At this moment unusable binaries are produced if bl31.bin file is missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 arch/arm/mach-zynqmp/mkimage_fit_atf.sh | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
arch/arm/mach-zynqmp/mkimage_fit_atf.sh +2 -5
index 1e770ba111d3..5effe05abdee 100755
@@ -29,11 +29,15 @@ else
 fi
 
 if [ ! -f $BL31 ]; then
-	echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
-	BL31=/dev/null
-	# But U-Boot proper could be loaded in EL3 by specifying
-	# firmware = "uboot";
-	# instead of "atf" in config node
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		BL31=/dev/null

arch/arm/mach-zynqmp/mkimage_fit_atf.sh else · line 43

Quoted 10 lines
arch/arm/mach-zynqmp/mkimage_fit_atf.sh +7 -0
+		# But U-Boot proper could be loaded in EL3 by specifying
+		# firmware = "uboot";
+		# instead of "atf" in config node
+	else
+		echo "ERROR: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
+		exit 1
+	fi
 fi
 
 cat << __HEADER_EOF

I think instead of fixing it on several places we should merge things together and fix this issue there.

Take a look at thread where we discussed it with Tom.
https://lists.denx.de/pipermail/u-boot/2019-December/393556.html

Thanks,
Michal

#9 lore raw

RFC PATCH 6/9 mkimage_fit_atf.sh: produce working binaries by default

Michal Simek <[email protected]> [2020-03-20 11:20:17]:

Quoted 2 lines

I think instead of fixing it on several places we should merge things together and fix this issue there.

What do you mean exactly? Checking for the deps one layer up, like this for example in sunxi?

  diff --git a/Makefile b/Makefile
  index 44776b8efcc4..1f4bff4374cf 100644
  --- a/Makefile
  +++ b/Makefile
  @@ -1276,6 +1276,19 @@ ifndef CONFIG_SYS_UBOOT_START
   CONFIG_SYS_UBOOT_START := $(CONFIG_SYS_TEXT_BASE)
   endif

+define check_its_dep
+ @if ! test -f "$(1)"; then \
+ if test "$(CONFIG_BUILDBOT_BROKEN_BINARIES)" = "y"; then \
+ touch "$(1)"; \
+ else \
+ echo "ERROR: $(2) file $(1) NOT found. If you want to build without " >&2; \
+ echo "a $(2) file (creating a NON-FUNCTIONAL binary), then enable" >&2; \
+ echo "config option CONFIG_BUILDBOT_BROKEN_BINARIES." >&2; \
+ false; \
+ fi \
+ fi
+endef
+
# Boards with more complex image requirements can provide an .its source file

   # or a generator script
   ifneq ($(CONFIG_SPL_FIT_SOURCE),"")
  @@ -1289,6 +1302,13 @@ endif
   ifeq ($(CONFIG_SPL_FIT_GENERATOR),"arch/arm/mach-rockchip/make_fit_atf.py")
   U_BOOT_ITS_DEPS += u-boot
   endif
  +ifeq ($(CONFIG_SPL_FIT_GENERATOR),"board/sunxi/mksunxi_fit_atf.sh")
  +BL31 := $(CURDIR)/bl31.bin
  +export BL31
  +$(BL31):
  +	$(call check_its_dep,$@,BL31)
  +U_BOOT_ITS_DEPS += $(BL31)
  +endif
   $(U_BOOT_ITS): $(U_BOOT_ITS_DEPS) FORCE
	  $(srctree)/$(CONFIG_SPL_FIT_GENERATOR) \
	  $(patsubst %,arch/$(ARCH)/dts/%.dtb,$(subst ",,$(CONFIG_OF_LIST))) > $@
  diff --git a/board/sunxi/mksunxi_fit_atf.sh b/board/sunxi/mksunxi_fit_atf.sh
  index 88ad71974706..1b0e5ee7a77a 100755
  --- a/board/sunxi/mksunxi_fit_atf.sh
  +++ b/board/sunxi/mksunxi_fit_atf.sh
  @@ -5,14 +5,6 @@
   #
   # usage: $0 <dt_name> [<dt_name> [<dt_name] ...]

-[ -z "$BL31" ] && BL31="bl31.bin"
-
-if [ ! -f $BL31 ]; then
- echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
- echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2
- BL31=/dev/null
-fi
-
if grep -q "^CONFIG_MACH_SUN50I_H6=y" .config; then

	  BL31_ADDR=0x104000

else

Quoted 2 lines

Take a look at thread where we discussed it with Tom.
https://lists.denx.de/pipermail/u-boot/2019-December/393556.html

Ok, but that N'th variation was merged anyway. So whats the plan? It's not clear from that discussion or I don't get it.

-- ynezz

#10 lore raw

RFC PATCH 6/9 mkimage_fit_atf.sh: produce working binaries by default

Quoted 30 lines · Petr ?tetiar

Michal Simek <[email protected]> [2020-03-20 11:20:17]:

Quoted 2 lines

I think instead of fixing it on several places we should merge things together and fix this issue there.

What do you mean exactly? Checking for the deps one layer up, like this for example in sunxi?

  diff --git a/Makefile b/Makefile
  index 44776b8efcc4..1f4bff4374cf 100644
  --- a/Makefile
  +++ b/Makefile
  @@ -1276,6 +1276,19 @@ ifndef CONFIG_SYS_UBOOT_START
   CONFIG_SYS_UBOOT_START := $(CONFIG_SYS_TEXT_BASE)
   endif
   
  +define check_its_dep
  +	@if ! test -f "$(1)"; then \
  +		if test "$(CONFIG_BUILDBOT_BROKEN_BINARIES)" = "y"; then \
  +			touch "$(1)"; \
  +		else \
  +			echo "ERROR: $(2) file $(1) NOT found. If you want to build without " >&2; \
  +			echo "a $(2) file (creating a NON-FUNCTIONAL binary), then enable" >&2; \
  +			echo "config option CONFIG_BUILDBOT_BROKEN_BINARIES." >&2; \
  +			false; \
  +		fi \
  +	fi
  +endef
  +
   # Boards with more complex image requirements can provide an .its source file
Quoted 8 lines
   # or a generator script
   ifneq ($(CONFIG_SPL_FIT_SOURCE),"")
  @@ -1289,6 +1302,13 @@ endif
   ifeq ($(CONFIG_SPL_FIT_GENERATOR),"arch/arm/mach-rockchip/make_fit_atf.py")
   U_BOOT_ITS_DEPS += u-boot
   endif
  +ifeq ($(CONFIG_SPL_FIT_GENERATOR),"board/sunxi/mksunxi_fit_atf.sh")
  +BL31 := $(CURDIR)/bl31.bin

here should be at least ?=

Quoted 25 lines

+export BL31
+$(BL31):
+ $(call check_its_dep,$@,BL31)
+U_BOOT_ITS_DEPS += $(BL31)
+endif
$(U_BOOT_ITS): $(U_BOOT_ITS_DEPS) FORCE

	  $(srctree)/$(CONFIG_SPL_FIT_GENERATOR) \
	  $(patsubst %,arch/$(ARCH)/dts/%.dtb,$(subst ",,$(CONFIG_OF_LIST))) > $@

diff --git a/board/sunxi/mksunxi_fit_atf.sh b/board/sunxi/mksunxi_fit_atf.sh index 88ad71974706..1b0e5ee7a77a 100755
--- a/board/sunxi/mksunxi_fit_atf.sh
+++ b/board/sunxi/mksunxi_fit_atf.sh
@@ -5,14 +5,6 @@
#
# usage: $0 <dt_name> [<dt_name> [<dt_name] ...]

-[ -z "$BL31" ] && BL31="bl31.bin"
-
-if [ ! -f $BL31 ]; then
- echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
- echo "Please read the section on ARM Trusted Firmware (ATF) in board/sunxi/README.sunxi64" >&2
- BL31=/dev/null
-fi
-
if grep -q "^CONFIG_MACH_SUN50I_H6=y" .config; then

Quoted 8 lines
	  BL31_ADDR=0x104000

else

Quoted 2 lines

Take a look at thread where we discussed it with Tom.
https://lists.denx.de/pipermail/u-boot/2019-December/393556.html

Ok, but that N'th variation was merged anyway. So whats the plan? It's not clear from that discussion or I don't get it.

My script was merged mostly because of fedora support and I expect it was the last one. Plan is to merge all of these to one shell script because most of the content in that script is very similar.

The issue which I see is that patch above is changing current behavior which means when this is applied all travis/gitlab/azure jobs will fail because none has setup this config option.

And truth is that even regular user doesn't know if error out is not check that final binary won't work because nothing fails.

Anyway I have sent series which enables to run U-Boot to run EL3 that's why at least zynqmp should be ok to build and run u-boot.itb. But better to run it with ATF.

Thanks,
Michal

#11 lore raw

RFC PATCH 6/9 mkimage_fit_atf.sh: produce working binaries by default

Michal Simek <[email protected]> [2020-03-23 14:58:55]:

Hi,

Quoted 2 lines

Plan is to merge all of these to one shell script because most of the content in that script is very similar.

to me it all just seems as values and templates hidden inside shell script called generator. Each of this generators uses/references 1-N variables, 1-N out-of-tree dependencies and 1-N DT sections.

So instead of cooking custom shell templating engine I took different approach[1] and simply removed the shell script generator in favor of Python based templating engine, allowing for ITS dependency tracking and values directly in Make.

Lets see what is preferred.

Quoted 3 lines

The issue which I see is that patch above is changing current behavior which means when this is applied all travis/gitlab/azure jobs will fail because none has setup this config option.

If that is desired, then in order to keep the current behavior, I see following options:

1. Disable building of boards having out-of-tree dependencies on build bot/CI
2. Enable BUILDBOT_BROKEN_BINARIES=y by default
3. Add detection of various build bot/CI platforms and enable BUILDBOT_BROKEN_BINARIES=y by default, which then could cause further issues when one would actually run test the binaries from build bot/CI
4. Include the out-of-tree dependencies (unlikely)

What is your suggestion?

1. https://patchwork.ozlabs.org/project/uboot/list/?series=166092

Cheers,

Petr

#12 lore raw

RFC PATCH 6/9 mkimage_fit_atf.sh: produce working binaries by default

Quoted 11 lines · Petr ?tetiar

Michal Simek <[email protected]> [2020-03-23 14:58:55]:

Hi,

Quoted 2 lines

Plan is to merge all of these to one shell script because most of the content in that script is very similar.

to me it all just seems as values and templates hidden inside shell script called generator. Each of this generators uses/references 1-N variables, 1-N out-of-tree dependencies and 1-N DT sections.

Quoted 6 lines

So instead of cooking custom shell templating engine I took different approach[1] and simply removed the shell script generator in favor of Python based templating engine, allowing for ITS dependency tracking and values directly in Make.

Lets see what is preferred.

An example of one is never enough to see if something is generic enough for everyone. Even if you can't functional-test other platforms an initial conversion will help and allow others to test. And it'll help show if the idea is or is not cleaner.

Quoted 7 lines
Quoted 3 lines

The issue which I see is that patch above is changing current behavior which means when this is applied all travis/gitlab/azure jobs will fail because none has setup this config option.

If that is desired, then in order to keep the current behavior, I see following options:

Quoted 6 lines

1. Disable building of boards having out-of-tree dependencies on build bot/CI
2. Enable BUILDBOT_BROKEN_BINARIES=y by default
3. Add detection of various build bot/CI platforms and enable BUILDBOT_BROKEN_BINARIES=y by default, which then could cause further issues when one would actually run test the binaries from build bot/CI
4. Include the out-of-tree dependencies (unlikely)

Cycling back here, sorry for the delay. First, you need to make CI still work. And we can't remove boards that require something else from CI as that means basically dropping all of arm64.

Quoted 1 line

What is your suggestion?

There's two sides to this. First, the problem of "all of these similar but not quite the same tools" needs a generic solution. A template based tool is probably the right direction here, yes.

But second, what would it take to modify the CI jobs to pass in an empty file or whatever so they link but not function. So that a real user would have to provide a file too. And whichever of Azure, GitLab or Travis is easier for you to test / adapt for this, I can update the others once it looks clean. Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200331/ddbb6a7e/attachment.sig>
#13 lore raw

RFC PATCH 7/9 fit_spl_atf.sh: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if either bl31.bin or rk3399m0.bin files are missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 .../puma_rk3399/fit_spl_atf.sh                | 20 +++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
board/theobroma-systems/puma_rk3399/fit_spl_atf.sh +14 -6
index 420e7daf4ceb..b53cd5f3de30 100755
@@ -14,17 +14,25 @@
 [ -z "$BL31" ] && BL31="bl31.bin"
 
 if [ ! -f $BL31 ]; then
-	echo "WARNING: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
-	echo "Please read Building section in doc/README.rockchip" >&2
-	BL31=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		BL31=/dev/null
+	else
+		echo "ERROR: BL31 file $BL31 NOT found, resulting binary is non-functional" >&2
+		echo "Please read Building section in doc/README.rockchip" >&2
+		exit 1
+	fi
 fi
 
 [ -z "$PMUM0" ] && PMUM0="rk3399m0.bin"
 
 if [ ! -f $PMUM0 ]; then
-	echo "WARNING: PMUM0 file $PMUM0 NOT found, resulting binary is non-functional" >&2
-	echo "Please read Building section in doc/README.rockchip" >&2
-	PMUM0=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		PMUM0=/dev/null
+	else
+		echo "ERROR: PMUM0 file $PMUM0 NOT found, resulting binary is non-functional" >&2
+		echo "Please read Building section in doc/README.rockchip" >&2
+		exit 1
+	fi
 fi
 
 cat << __HEADER_EOF
#14 lore raw

RFC PATCH 8/9 k3_fit_atf.sh: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if either bl31.bin or bl32.bin files are missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 tools/k3_fit_atf.sh | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
tools/k3_fit_atf.sh +12 -4
index 4e9f69c08789..8e342b46c97b 100755
@@ -10,15 +10,23 @@
 [ -z "$ATF" ] && ATF="bl31.bin"
 
 if [ ! -f $ATF ]; then
-	echo "WARNING ATF file $ATF NOT found, resulting binary is non-functional" >&2
-	ATF=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		ATF=/dev/null
+	else
+		echo "ERROR: ATF file $ATF NOT found, resulting binary is non-functional" >&2
+		exit 1
+	fi
 fi
 
 [ -z "$TEE" ] && TEE="bl32.bin"
 
 if [ ! -f $TEE ]; then
-	echo "WARNING OPTEE file $TEE NOT found, resulting might be non-functional" >&2
-	TEE=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		TEE=/dev/null
+	else
+		echo "ERROR: OPTEE file $TEE NOT found, resulting might be non-functional" >&2
+		exit 1
+	fi
 fi
 
 if [ ! -z "$IS_HS" ]; then
#15 lore raw

RFC PATCH 9/9 mkimage_fit_opensbi.sh: produce working binaries by default

In reply to Petr Štetiar

At this moment unusable binaries are produced if bl31.bin file is missing in order to allow passing of various CI tests. This intention of broken binaries has to be now explicitly confirmed via new BUILDBOT_BROKEN_BINARIES config option, so usable binaries are produced by default from now on.

---
 arch/riscv/lib/mkimage_fit_opensbi.sh | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/riscv/lib/mkimage_fit_opensbi.sh +6 -2
index d6f95e5bfd2c..bf0968e683d2 100755
@@ -17,8 +17,12 @@ if [ -z "$OPENSBI_LOAD_ADDR" ]; then
 fi
 
 if [ ! -f $OPENSBI ]; then
-	echo "WARNING: OpenSBI binary \"$OPENSBI\" not found, resulting binary is not functional." >&2
-	OPENSBI=/dev/null
+	if [ "$BUILDBOT_BROKEN_BINARIES" = "y" ]; then
+		OPENSBI=/dev/null
+	else
+		echo "ERROR: OpenSBI binary \"$OPENSBI\" not found, resulting binary would be non-functional." >&2
+		exit 1
+	fi
 fi
 
 cat << __HEADER_EOF