boot/uboot: host-dtc is not used as expected

Hi,

U-Boot's build system (at least in 2018.01) does not use $(HOST_DIR)/bin/dtc to compile custom dts files. Instead it tries to use it's internal dtc who is not compiled by default and result in a build failure:

DTC arch/arm/dts/foo.dtb
/bin/sh: ./scripts/dtc/dtc: No such file or directory make[4]: *** [scripts/Makefile.lib:329: arch/arm/dts/foo.dtb] Error 127

To reproduce this error, use a uboot config that contains the following:

CONFIG_MULTI_DTB_FIT=y
CONFIG_OF_LIST="foo"

Unfortunately, even by applying the patch in this series to really use host-dtc, it then fails with bunch of errors like the following:

FATAL ERROR: Unrecognized check name "simple_bus_reg" make[4]: *** [scripts/Makefile.lib:329: arch/arm/dts/foo.dtb] Error 1

So, I though to just bump dtc to v1.4.7 who seems to supports these parameters but then again, there is other errors with uboot & even with linux:

In file included from output/host/include/libfdt.h:54:0, from scripts/dtc/libfdt/fdt.c:54: output/host/include/libfdt_env.h:82:24:
error: redefinition of ?fdt16_to_cpu?
static inline uint16_t fdt16_to_cpu(fdt16_t x)

u-boot on his side is experiencing the same problem + a dependency that it try to build and that doesn't exist in the tree:

make[3]: *** No rule to make target 'arch/arm/dts/unset.dts', needed by 'arch/arm/dts/unset.dtb'. Stop.
make[3]: *** Waiting for unfinished jobs....

I never found from where that "unset.dts" is coming from...

In the meantime, Geoff submitted the same patch to bump to the v1.4.7 who leed to the same problem I was faced.

So... I'm a bit lost at that point and that's why I'm writing this RFC.

Thanks in advance for your feedbacks.

Regards,
David.

David De Grave (Essensium/Mind) (1):
boot/uboot: Really use host-dtc

 boot/uboot/uboot.mk | 1 +
 1 file changed, 1 insertion(+)

2.13.6

U-Boot's build system (at least in 2018.01) does not use $(HOST_DIR)/bin/dtc to compile dts files. Instead it tries to use it's internal dtc who is not compiled by default and result in a build failure:

DTC arch/arm/dts/foo.dtb
/bin/sh: ./scripts/dtc/dtc: No such file or directory make[4]: *** [scripts/Makefile.lib:329: arch/arm/dts/foo.dtb] Error 127

To reproduce this error, use a uboot config that contains the following:

CONFIG_MULTI_DTB_FIT=y
CONFIG_OF_LIST="foo"

---
 boot/uboot/uboot.mk | 1 +
 1 file changed, 1 insertion(+)

index bf227f2893..e877de221e 100644
@@ -141,6 +141,7 @@ UBOOT_MAKE_OPTS += BL31=$(BINARIES_DIR)/bl31.bin
 endif
 
 ifeq ($(BR2_TARGET_UBOOT_NEEDS_DTC),y)
+UBOOT_MAKE_OPTS += DTC=$(HOST_DIR)/bin/dtc
 UBOOT_DEPENDENCIES += host-dtc
 endif
 

2.13.6

Hello,

I just tested building dra7xx_evm on U-Boot 2018.07, and it builds fine, scripts/dtc/dtc is used and has been built:

mkdir -p arch/arm/dts/ ; (cat arch/arm/dts/dra72-evm.dts; echo '#include "omap5-u-boot.dtsi"') > arch/arm/dts/.dra72-evm.dtb.pre.tmp; /home/thomas/projets/buildroot/output/host/bin/arm-linux-gnueabihf-gcc -E -Wp,-MD,arch/arm/dts/.dra72-evm.dtb.d.pre.tmp -nostdinc -I./arch/arm/dts -I./arch/arm/dts/include -Iinclude -I./include -I./arch/arm/include -include ./include/linux/kconfig.h -D__ASSEMBLY__ -undef -D__DTS__ -x assembler-with-cpp -o arch/arm/dts/.dra72-evm.dtb.dts.tmp arch/arm/dts/.dra72-evm.dtb.pre.tmp ; ./scripts/dtc/dtc -O dtb -o arch/arm/dts/dra72-evm.dtb -b 0 -i arch/arm/dts/ -Wno-unit_address_vs_reg -Wno-simple_bus_reg -Wno-unit_address_format -Wno-pci_bridge -Wno-pci_device_bus_num -Wno-pci_device_reg -d arch/arm/dts/.dra72-evm.dtb.d.dtc.tmp arch/arm/dts/.dra72-evm.dtb.dts.tmp ; cat arch/arm/dts/.dra72-evm.dtb.d.pre.tmp arch/arm/dts/.dra72-evm.dtb.d.dtc.tmp > arch/arm/dts/.dra72-evm.dtb.d
mkdir -p arch/arm/dts/ ; (cat arch/arm/dts/dra7-evm.dts; echo '#include "dra7-evm-u-boot.dtsi"') > arch/arm/dts/.dra7-evm.dtb.pre.tmp; /home/thomas/projets/buildroot/output/host/bin/arm-linux-gnueabihf-gcc -E -Wp,-MD,arch/arm/dts/.dra7-evm.dtb.d.pre.tmp -nostdinc -I./arch/arm/dts -I./arch/arm/dts/include -Iinclude -I./include -I./arch/arm/include -include ./include/linux/kconfig.h -D__ASSEMBLY__ -undef -D__DTS__ -x assembler-with-cpp -o arch/arm/dts/.dra7-evm.dtb.dts.tmp arch/arm/dts/.dra7-evm.dtb.pre.tmp ; ./scripts/dtc/dtc -O dtb -o arch/arm/dts/dra7-evm.dtb -b 0 -i arch/arm/dts/ -Wno-unit_address_vs_reg -Wno-simple_bus_reg -Wno-unit_address_format -Wno-pci_bridge -Wno-pci_device_bus_num -Wno-pci_device_reg -d arch/arm/dts/.dra7-evm.dtb.d.dtc.tmp arch/arm/dts/.dra7-evm.dtb.dts.tmp ; cat arch/arm/dts/.dra7-evm.dtb.d.pre.tmp arch/arm/dts/.dra7-evm.dtb.d.dtc.tmp > arch/arm/dts/.dra7-evm.dtb.d mkdir -p arch/arm/dts/ ; (cat arch/arm/dts/dra72-evm-revc.dts; echo '#include "dra72-evm-revc-u-boot.dtsi"') > arch/arm/dts/.dra72-evm-revc.dtb.pre.tmp; /home/thomas/projets/buildroot/output/host/bin/arm-linux-gnueabihf-gcc -E -Wp,-MD,arch/arm/dts/.dra72-evm-revc.dtb.d.pre.tmp -nostdinc -I./arch/arm/dts -I./arch/arm/dts/include -Iinclude -I./include -I./arch/arm/include -include ./include/linux/kconfig.h -D__ASSEMBLY__ -undef -D__DTS__ -x assembler-with-cpp -o arch/arm/dts/.dra72-evm-revc.dtb.dts.tmp arch/arm/dts/.dra72-evm-revc.dtb.pre.tmp ; ./scripts/dtc/dtc -O dtb -o arch/arm/dts/dra72-evm-revc.dtb -b 0 -i arch/arm/dts/ -Wno-unit_address_vs_reg -Wno-simple_bus_reg -Wno-unit_address_format -Wno-pci_bridge -Wno-pci_device_bus_num -Wno-pci_device_reg -d arch/arm/dts/.dra72-evm-revc.dtb.d.dtc.tmp arch/arm/dts/.dra72-evm-revc.dtb.dts.tmp ; cat arch/arm/dts/.dra72-evm-revc.dtb.d.pre.tmp arch/arm/dts/.dra72-evm-revc.dtb.d.dtc.tmp > arch/arm/dts/.dra72-evm-revc.dtb.d mkdir -p arch/arm/dts/ ; (cat arch/arm/dts/dra71-evm.dts; echo '#include "dra71-evm-u-boot.dtsi"') > arch/arm/dts/.dra71-evm.dtb.pre.tmp; /home/thomas/projets/buildroot/output/host/bin/arm-linux-gnueabihf-gcc -E -Wp,-MD,arch/arm/dts/.dra71-evm.dtb.d.pre.tmp -nostdinc -I./arch/arm/dts -I./arch/arm/dts/include -Iinclude -I./include -I./arch/arm/include -include ./include/linux/kconfig.h -D__ASSEMBLY__ -undef -D__DTS__ -x assembler-with-cpp -o arch/arm/dts/.dra71-evm.dtb.dts.tmp arch/arm/dts/.dra71-evm.dtb.pre.tmp ; ./scripts/dtc/dtc -O dtb -o arch/arm/dts/dra71-evm.dtb -b 0 -i arch/arm/dts/ -Wno-unit_address_vs_reg -Wno-simple_bus_reg -Wno-unit_address_format -Wno-pci_bridge -Wno-pci_device_bus_num -Wno-pci_device_reg -d arch/arm/dts/.dra71-evm.dtb.d.dtc.tmp arch/arm/dts/.dra71-evm.dtb.dts.tmp ; cat arch/arm/dts/.dra71-evm.dtb.d.pre.tmp arch/arm/dts/.dra71-evm.dtb.d.dtc.tmp > arch/arm/dts/.dra71-evm.dtb.d mkdir -p arch/arm/dts/ ; (cat arch/arm/dts/dra76-evm.dts; echo '#include "dra76-evm-u-boot.dtsi"') > arch/arm/dts/.dra76-evm.dtb.pre.tmp; /home/thomas/projets/buildroot/output/host/bin/arm-linux-gnueabihf-gcc -E -Wp,-MD,arch/arm/dts/.dra76-evm.dtb.d.pre.tmp -nostdinc -I./arch/arm/dts -I./arch/arm/dts/include -Iinclude -I./include -I./arch/arm/include -include ./include/linux/kconfig.h -D__ASSEMBLY__ -undef -D__DTS__ -x assembler-with-cpp -o arch/arm/dts/.dra76-evm.dtb.dts.tmp arch/arm/dts/.dra76-evm.dtb.pre.tmp ; ./scripts/dtc/dtc -O dtb -o arch/arm/dts/dra76-evm.dtb -b 0 -i arch/arm/dts/ -Wno-unit_address_vs_reg -Wno-simple_bus_reg -Wno-unit_address_format -Wno-pci_bridge -Wno-pci_device_bus_num -Wno-pci_device_reg -d arch/arm/dts/.dra76-evm.dtb.d.dtc.tmp arch/arm/dts/.dra76-evm.dtb.dts.tmp ; cat arch/arm/dts/.dra76-evm.dtb.d.pre.tmp arch/arm/dts/.dra76-evm.dtb.d.dtc.tmp > arch/arm/dts/.dra76-evm.dtb.d

$ ls -l output/build/uboot-2018.07/scripts/dtc/dtc
-rwxr-xr-x 1 thomas thomas 132704 Jul 28 21:37 output/build/uboot-2018.07/scripts/dtc/dtc

Could you provide more details about which configuration is causing problem ?

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Thomas,

Thanks for your reply.
I'm working on a beagle bone black wireless board with a custom config based on "beaglebone_defconfig".

Indeed, I was not able to reproduce it from the buildroot directory also... It seems the problem comes only when I build outside of it.
To avoid changing something in the buildroot tree and make it easier to update it later, I made a wrapper script that change to buildroot dir and use all the configs/packages/... a directory level below it.

A short example:

root-project

|- buildroot/
|  |- arch/
|  |- configs/
|  |- ...
|- output/
|- beaglebone_defconfig
|- uboot.fragment

....

So, to reproduce that problem, I did the following:

mkdir test && cd test
git clone git://git.buildroot.net/buildroot.git buildroot cd buildroot && git checkout 2018.05-1 && cd ..

make -C buildroot O=$PWD/output defconfig
BR2_DEFCONFIG=../beaglebone_defconfig
make -C buildroot O=$PWD/output menuconfig

changed:
BR2_TARGET_UBOOT_BOARD_DEFCONFIG="am335x_boneblack" BR2_TARGET_UBOOT_CONFIG_FRAGMENT_FILES="../uboot.fragment"

uboot.fragment:
CONFIG_MULTI_DTB_FIT=y
CONFIG_OF_LIST="am335x-boneblack"

make -C buildroot O=$PWD/output uboot
make: Entering directory '/home/ddg/test/buildroot' /usr/bin/make -j1 O=/home/ddg/test/output HOSTCC="/usr/lib64/ccache/gcc" HOSTCXX="/usr/lib64/ccache/g++" silentoldconfig
GEN /home/ddg/test/output/Makefile

[...]

[...]

[...]
CC spl/lib/libfdt/fdt_overlay.o
CC spl/lib/libfdt/fdt_ro.o
OBJCOPY u-boot-nodtb.bin
CC spl/lib/linux_compat.o
make[3]: *** No rule to make target 'arch/arm/dts/unset.dts', needed by 'arch/arm/dts/unset.dtb'. Stop.
make[3]: *** Waiting for unfinished jobs....
DTC arch/arm/dts/am335x-boneblack.dtb
CC spl/drivers/core/fdtaddr.o
/bin/sh: ./scripts/dtc/dtc: No such file or directory
make[4]: *** [scripts/Makefile.lib:329: arch/arm/dts/am335x-boneblack.dtb] Error 127
make[3]: *** [dts/Makefile:62: arch-dtbs] Error 2
make[2]: *** [Makefile:876: dts/dt.dtb] Error 2
make[2]: *** Waiting for unfinished jobs....

  CC      spl/drivers/gpio/gpio-uclass.o
  CC      spl/lib/libfdt/fdt_rw.o
  CC      spl/drivers/gpio/omap_gpio.o
  CC      spl/drivers/core/lists.o

make[1]: *** [package/pkg-generic.mk:226:
/home/ddg/test/output/build/uboot-2018.01/.stamp_built] Error 2
make: *** [Makefile:79: _all] Error 2
make: Leaving directory '/home/ddg/test/buildroot'

ls output/build/uboot-2018.01/scripts/dtc/dtc
ls: cannot access 'output/build/uboot-2018.01/scripts/dtc/dtc': No such file or directory

If I remove the uboot.fragment, all is building well up to the end. Moreover, why do he use ./scripts/dtc/dtc ? It should use $(HOST_DIR)/bin/dtc at the moment host-dtc is build, no ?

Regards,
David

*David De Grave*
Senior Embedded Software Developer
Gsm : +32(0)496.364.960 | Tel : +32-16-28.65.00 | Fax : +32-16-28a.65.01
Essensium-Mind <https://www.mind.be/> - Gaston Geenslaan 9, B-3001 Leuven,
Belgium
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180731/b7611036/attachment.html>

Hello,

Thanks for the detailed instructions, I'll try to reproduce.

I think the optional dependency on host-dtc depends from a time where U-Boot did not have its own copy of DTC. I'm not sure it makes a lot of sense now.

If U-Boot tries to use ./scripts/dtc/dtc, then it should have built it. If it didn't build it but tries to use it nonetheless, it sounds like a bug in U-Boot.

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[snip]

[snip]

... but we still want to support those older U-Boot versions, so we can't
remove it.

Ideally, we should find out between which two versions it is needed, and
mention that in the help text.

Oh yes, now I see it, it's indeed a bug in U-Boot. CONFIG_OF_LIST should either
depend on CONFIG_OF_CONTROL or select it, otherwise dtc is not built.

Regards,
Arnout

Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

Hello,

Absolutely.

But the case found by David is not a case where U-Boot needs a DTC available on the host: it tries to use its own internal DTC, but forgets to build it. That's a bug in U-Boot.

Right, so it's indeed a bug in U-Boot. That being said, the change proposed by David is not too bad: it makes sense to use the host DTC if it was built, rather than the one built by U-Boot. It's just that the initial problem explanation wasn't really complete enough, so I was a bit confused.

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Thomas, Arnout,

In the other hand, using the host-dtc is working but lead to other problems:

- Version 1.4.4 don't handle most of the parameters issued by uboot like the following:

FATAL ERROR: Unrecognized check name "simple_bus_reg"

=> this can be fixed by setting DTC_FLAGS to the ones who works (only 1 if I remember well).

- If we upgrade DTC to version 1.4.7 (who is the case now I guess), then it still can't find a "unset.dtb" target that I can't figure out from where it comes:

LDS spl/u-boot-spl.lds
OBJCOPY u-boot-nodtb.bin
make[3]: *** No rule to make target 'arch/arm/dts/unset.dts', needed by 'arch/arm/dts/unset.dtb'. Stop.
make[3]: *** Waiting for unfinished jobs....
OBJCOPY u-boot.srec

  SYM     u-boot.sym
  DTC     arch/arm/dts/am335x-boneblack.dtb
  LD      spl/drivers/core/built-in.o

FATAL ERROR: Unrecognized check name "simple_bus_reg"
make[4]: *** [scripts/Makefile.lib:329: arch/arm/dts/am335x-boneblack.dtb] Error 1
make[3]: *** [dts/Makefile:62: arch-dtbs] Error 2
make[2]: *** [Makefile:876: dts/dt.dtb] Error 2
make[2]: *** Waiting for unfinished jobs....

Also I checked in the mainstream of u-boot and saw some patches for dtc... I see a dtc bump to 1.4.6-9.
So, we can maybe reuse some patches from there to fix this ? Or try to fix the bug that Arnout discovered ourself ?

Regards,
David.

*David De Grave*
Senior Embedded Software Developer
Gsm : +32(0)496.364.960 | Tel : +32-16-28.65.00 | Fax : +32-16-28a.65.01
Essensium-Mind <https://www.mind.be/> - Gaston Geenslaan 9, B-3001 Leuven,
Belgium
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180801/17a751d6/attachment.html>

Hello,

So, using the U-Boot provided DTC is better when available. So indeed, I think you should just fix U-Boot so that when it
uses ./scripts/dtc/dtc, it builds it first :-)

dts/Makefile contains:

DEVICE_TREE ?= $(CONFIG_DEFAULT_DEVICE_TREE:"%"=%) ifeq ($(DEVICE_TREE),)
DEVICE_TREE := unset
endif

This is where your unset.dts comes from.

Best regards,

Thomas

Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Thomas,

Indeed it sounds a better approach and submitted a new patch. Not sure however if
it's well done. At least everything is running fine now on my side.

Indeed. Thx! :-)

Regards,
David.

*David De Grave*
Senior Embedded Software Developer
Gsm : +32(0)496.364.960 | Tel : +32-16-28.65.00 | Fax : +32-16-28a.65.01
Essensium-Mind <https://www.mind.be/> - Gaston Geenslaan 9, B-3001 Leuven,
Belgium
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180801/3c434e2a/attachment.html>