of: build dbts with symbols when CONFIG_OF_OVERLAY is set

PATCH In review Andre Heider 2 participants 5 messages Dec 14, 2017 – Dec 20, 2017

Subsystems (2)

Kbuild Device tree

lore

Share & tools

lore

Andre Heider

author

2017-12-14 15:12 UTC #0 lore raw

PATCH 1/1 of: build dbts with symbols when CONFIG_OF_OVERLAY is set

The overlay feature requires the base dtb to be built with symbols, so lets build the dtbs with symbols when overlay support was explicitly enabled.

With CONFIG_OF_ALL_DTBS on ARCH=arm the 989 dtb files grow about ~38% on average.

Totals in bytes with the 3 biggest ones:

Before:

	90471	arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb
	90521	arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb
	92639	arch/arm/boot/dts/dra7-evm.dtb
	25731296	total

After:

	133203	arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb
	133237	arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb
	134545	arch/arm/boot/dts/dra7-evm.dtb
	35464440	total

Signed-off-by Andre Heider <[email protected]>

---

Hi,

while playing around with overlays I noticed that I needed to rebuilt my distro's device trees because they didn't come with symbols.

Is that for a reason, maybe the not so minor increase in size?

Thanks,
Andre

 drivers/of/unittest-data/Makefile | 7 -------
 scripts/Makefile.lib              | 5 +++++
 2 files changed, 5 insertions(+), 7 deletions(-)

drivers/of/unittest-data/Makefile +0 -7

index 32389acfa616..b65061013512 100644
@@ -15,13 +15,6 @@ targets += overlay.dtb overlay.dtb.S
 targets += overlay_bad_phandle.dtb overlay_bad_phandle.dtb.S
 targets += overlay_bad_symbol.dtb overlay_bad_symbol.dtb.S
 targets += overlay_base.dtb overlay_base.dtb.S
-
-# enable creation of __symbols__ node
-DTC_FLAGS_overlay := -@
-DTC_FLAGS_overlay_bad_phandle := -@
-DTC_FLAGS_overlay_bad_symbol := -@
-DTC_FLAGS_overlay_base := -@
-
 endif
 
 .PRECIOUS: \

scripts/Makefile.lib +5 -0

index 1ca4dcd2d500..c7ba4aa8a07a 100644
@@ -278,6 +278,11 @@ DTC_FLAGS += -Wnode_name_chars_strict \
 	-Wproperty_name_chars_strict
 endif
 
+ifeq ($(CONFIG_OF_OVERLAY),y)
+# enable creation of __symbols__ node
+DTC_FLAGS += -@
+endif
+
 DTC_FLAGS += $(DTC_FLAGS_$(basetarget))
 
 # Generate an assembly file to wrap the output of the device tree compiler

Signature · 1 line

2.15.1

Frank Rowand

2017-12-15 21:06 UTC #1 lore raw

Quoted 22 lines · Andre Heider

The overlay feature requires the base dtb to be built with symbols, so lets build the dtbs with symbols when overlay support was explicitly enabled.
With CONFIG_OF_ALL_DTBS on ARCH=arm the 989 dtb files grow about ~38% on average.
Totals in bytes with the 3 biggest ones:
Before:
	90471	arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb
	90521	arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb
	92639	arch/arm/boot/dts/dra7-evm.dtb
	25731296	total
After:
	133203	arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb
	133237	arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb
	134545	arch/arm/boot/dts/dra7-evm.dtb
	35464440	total
Signed-off-by Andre Heider <[email protected]>

Quoted 8 lines

---
Hi,
while playing around with overlays I noticed that I needed to rebuilt my distro's device trees because they didn't come with symbols.
Is that for a reason, maybe the not so minor increase in size?

Yes, size is the issue.

Quoted 16 lines

Thanks,
Andre

 drivers/of/unittest-data/Makefile | 7 -------
 scripts/Makefile.lib              | 5 +++++
 2 files changed, 5 insertions(+), 7 deletions(-)

drivers/of/unittest-data/Makefile +0 -0

index 32389acfa616..b65061013512 100644
@@ -15,13 +15,6 @@ targets += overlay.dtb overlay.dtb.S
 targets += overlay_bad_phandle.dtb overlay_bad_phandle.dtb.S
 targets += overlay_bad_symbol.dtb overlay_bad_symbol.dtb.S
 targets += overlay_base.dtb overlay_base.dtb.S

drivers/of/unittest-data/Makefile targets += overlay.dtb overlay.dtb.S · line 20

Quoted 10 lines

drivers/of/unittest-data/Makefile +0 -7

-
-# enable creation of __symbols__ node
-DTC_FLAGS_overlay := -@
-DTC_FLAGS_overlay_bad_phandle := -@
-DTC_FLAGS_overlay_bad_symbol := -@
-DTC_FLAGS_overlay_base := -@
-
 endif
 
 .PRECIOUS: \

No. The unittests require these to be set unconditionally.

Quoted 7 lines

scripts/Makefile.lib +0 -0

index 1ca4dcd2d500..c7ba4aa8a07a 100644
@@ -278,6 +278,11 @@ DTC_FLAGS += -Wnode_name_chars_strict \
 	-Wproperty_name_chars_strict

scripts/Makefile.lib DTC_FLAGS += -Wnode_name_chars_strict \ · line 288

Quoted 10 lines

scripts/Makefile.lib +5 -0

 endif
 
+ifeq ($(CONFIG_OF_OVERLAY),y)
+# enable creation of __symbols__ node
+DTC_FLAGS += -@
+endif
+
 DTC_FLAGS += $(DTC_FLAGS_$(basetarget))
 
 # Generate an assembly file to wrap the output of the device tree compiler

Not needed. Instead set DTC_FLAGS in the make command. For example:

DTC_FLAGS=-@ make qcom-apq8074-dragonboard.dtb

There are a few architecture Makefiles that need to be fixed to not unconditionally set DTC_FLAGS.

-Frank

Andre Heider

author

2017-12-16 17:25 UTC #2 lore raw

Hi Frank,

Quoted 85 lines · Frank Rowand

Quoted 22 lines · Andre Heider
The overlay feature requires the base dtb to be built with symbols, so lets build the dtbs with symbols when overlay support was explicitly enabled.
With CONFIG_OF_ALL_DTBS on ARCH=arm the 989 dtb files grow about ~38% on average.
Totals in bytes with the 3 biggest ones:
Before:
	90471	arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb
	90521	arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb
	92639	arch/arm/boot/dts/dra7-evm.dtb
	25731296	total
After:
	133203	arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb
	133237	arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb
	134545	arch/arm/boot/dts/dra7-evm.dtb
	35464440	total
Signed-off-by Andre Heider <[email protected]>
Quoted 8 lines
---
Hi,
while playing around with overlays I noticed that I needed to rebuilt my distro's device trees because they didn't come with symbols.
Is that for a reason, maybe the not so minor increase in size?
Yes, size is the issue.
Quoted 16 lines
Thanks,
Andre
  drivers/of/unittest-data/Makefile | 7 -------
  scripts/Makefile.lib              | 5 +++++
  2 files changed, 5 insertions(+), 7 deletions(-)
drivers/of/unittest-data/Makefile +0 -0
index 32389acfa616..b65061013512 100644
@@ -15,13 +15,6 @@ targets += overlay.dtb overlay.dtb.S
  targets += overlay_bad_phandle.dtb overlay_bad_phandle.dtb.S
  targets += overlay_bad_symbol.dtb overlay_bad_symbol.dtb.S
  targets += overlay_base.dtb overlay_base.dtb.S
drivers/of/unittest-data/Makefile targets += overlay.dtb overlay.dtb.S · line 20
Quoted 10 lines
drivers/of/unittest-data/Makefile +0 -7
-
-# enable creation of __symbols__ node
-DTC_FLAGS_overlay := -@
-DTC_FLAGS_overlay_bad_phandle := -@
-DTC_FLAGS_overlay_bad_symbol := -@
-DTC_FLAGS_overlay_base := -@
-
  endif
 
  .PRECIOUS: \
No. The unittests require these to be set unconditionally.
Quoted 7 lines
scripts/Makefile.lib +0 -0
index 1ca4dcd2d500..c7ba4aa8a07a 100644
@@ -278,6 +278,11 @@ DTC_FLAGS += -Wnode_name_chars_strict \
  	-Wproperty_name_chars_strict
scripts/Makefile.lib DTC_FLAGS += -Wnode_name_chars_strict \ · line 288
Quoted 10 lines
scripts/Makefile.lib +5 -0
  endif
 
+ifeq ($(CONFIG_OF_OVERLAY),y)
+# enable creation of __symbols__ node
+DTC_FLAGS += -@
+endif
+
  DTC_FLAGS += $(DTC_FLAGS_$(basetarget))
 
  # Generate an assembly file to wrap the output of the device tree compiler
Not needed. Instead set DTC_FLAGS in the make command. For example:

Quoted 4 lines

DTC_FLAGS=-@ make qcom-apq8074-dragonboard.dtb
There are a few architecture Makefiles that need to be fixed to not unconditionally set DTC_FLAGS.

Rebuilding the dts files manually with symbols isn't the problem. The idea was to enable it with a switch which distros simply can flip.

Passing DTC_FLAGS to make would work too I guess, but on multi platform builds that's still the same issue if there're boards which can't handle that because of the size increase.

Would a solution which enables symbols per family work? E.g. CONFIG_MACH_SUN7I chooses to use -@ because it can handle it.

Thanks,
Andre

Frank Rowand

2017-12-18 22:10 UTC #3 lore raw

Quoted 97 lines · Andre Heider

Hi Frank,
Quoted 82 lines · Frank Rowand
Quoted 22 lines · Andre Heider
The overlay feature requires the base dtb to be built with symbols, so lets build the dtbs with symbols when overlay support was explicitly enabled.
With CONFIG_OF_ALL_DTBS on ARCH=arm the 989 dtb files grow about ~38% on average.
Totals in bytes with the 3 biggest ones:
Before:
????90471??? arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb ????90521??? arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb ????92639??? arch/arm/boot/dts/dra7-evm.dtb ????25731296??? total
After:
????133203??? arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb ????133237??? arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb ????134545??? arch/arm/boot/dts/dra7-evm.dtb ????35464440??? total
Signed-off-by Andre Heider <[email protected]>
Quoted 8 lines
---
Hi,
while playing around with overlays I noticed that I needed to rebuilt my distro's device trees because they didn't come with symbols.
Is that for a reason, maybe the not so minor increase in size?
Yes, size is the issue.
Quoted 23 lines
Thanks,
Andre
? drivers/of/unittest-data/Makefile | 7 ------- ? scripts/Makefile.lib????????????? | 5 +++++ ? 2 files changed, 5 insertions(+), 7 deletions(-)
drivers/of/unittest-data/Makefile +0 -0
index 32389acfa616..b65061013512 100644
@@ -15,13 +15,6 @@ targets += overlay.dtb overlay.dtb.S
? targets += overlay_bad_phandle.dtb overlay_bad_phandle.dtb.S ? targets += overlay_bad_symbol.dtb overlay_bad_symbol.dtb.S ? targets += overlay_base.dtb overlay_base.dtb.S
drivers/of/unittest-data/Makefile +0 -7
-
-# enable creation of __symbols__ node
-DTC_FLAGS_overlay := -@
-DTC_FLAGS_overlay_bad_phandle := -@
-DTC_FLAGS_overlay_bad_symbol := -@
-DTC_FLAGS_overlay_base := -@
-
Quoted 2 lines
? endif
? ? .PRECIOUS: \
No.? The unittests require these to be set unconditionally.
Quoted 12 lines
scripts/Makefile.lib +0 -0
index 1ca4dcd2d500..c7ba4aa8a07a 100644
@@ -278,6 +278,11 @@ DTC_FLAGS += -Wnode_name_chars_strict \
????? -Wproperty_name_chars_strict
? endif
? +ifeq ($(CONFIG_OF_OVERLAY),y)
scripts/Makefile.lib +4 -0
+# enable creation of __symbols__ node
+DTC_FLAGS += -@
+endif
+
Quoted 3 lines
? DTC_FLAGS += $(DTC_FLAGS_$(basetarget))
? ? # Generate an assembly file to wrap the output of the device tree compiler
Not needed.? Instead set DTC_FLAGS in the make command.? For example:
Quoted 4 lines
??? DTC_FLAGS=-@ make qcom-apq8074-dragonboard.dtb
There are a few architecture Makefiles that need to be fixed to not unconditionally set DTC_FLAGS.
Rebuilding the dts files manually with symbols isn't the problem. The idea was to enable it with a switch which distros simply can flip.
Passing DTC_FLAGS to make would work too I guess, but on multi platform builds that's still the same issue if there're boards which can't handle that because of the size increase.

Quoted 5 lines

Would a solution which enables symbols per family work? E.g. CONFIG_MACH_SUN7I chooses to use -@ because it can handle it.
Thanks,
Andre

Making it easy for distros to increase the size of .dtb files is not the correct answer. We are trying to decrease the footprint of devicetree, not increase it.

-Frank

Andre Heider

author

2017-12-20 08:08 UTC #4 lore raw

Quoted 101 lines · Frank Rowand

Quoted 100 lines · Andre Heider
Hi Frank,
Quoted 82 lines · Frank Rowand
Quoted 22 lines · Andre Heider
The overlay feature requires the base dtb to be built with symbols, so lets build the dtbs with symbols when overlay support was explicitly enabled.
With CONFIG_OF_ALL_DTBS on ARCH=arm the 989 dtb files grow about ~38% on average.
Totals in bytes with the 3 biggest ones:
Before:
????90471??? arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb ????90521??? arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb ????92639??? arch/arm/boot/dts/dra7-evm.dtb ????25731296??? total
After:
????133203??? arch/arm/boot/dts/am57xx-beagle-x15-revc.dtb ????133237??? arch/arm/boot/dts/am57xx-beagle-x15-revb1.dtb ????134545??? arch/arm/boot/dts/dra7-evm.dtb ????35464440??? total
Signed-off-by Andre Heider <[email protected]>
Quoted 8 lines
---
Hi,
while playing around with overlays I noticed that I needed to rebuilt my distro's device trees because they didn't come with symbols.
Is that for a reason, maybe the not so minor increase in size?
Yes, size is the issue.
Quoted 15 lines
Thanks,
Andre
 ? drivers/of/unittest-data/Makefile | 7 -------
 ? scripts/Makefile.lib????????????? | 5 +++++
? 2 files changed, 5 insertions(+), 7 deletions(-)
drivers/of/unittest-data/Makefile +0 -0
index 32389acfa616..b65061013512 100644
@@ -15,13 +15,6 @@ targets += overlay.dtb overlay.dtb.S
 ? targets += overlay_bad_phandle.dtb overlay_bad_phandle.dtb.S
 ? targets += overlay_bad_symbol.dtb overlay_bad_symbol.dtb.S
drivers/of/unittest-data/Makefile targets += overlay.dtb overlay.dtb.S · line 19
Quoted 10 lines
drivers/of/unittest-data/Makefile +0 -7
 ? targets += overlay_base.dtb overlay_base.dtb.S
-
-# enable creation of __symbols__ node
-DTC_FLAGS_overlay := -@
-DTC_FLAGS_overlay_bad_phandle := -@
-DTC_FLAGS_overlay_bad_symbol := -@
-DTC_FLAGS_overlay_base := -@
-
 ? endif
 ? ? .PRECIOUS: \
No.? The unittests require these to be set unconditionally.
Quoted 5 lines
scripts/Makefile.lib +0 -0
index 1ca4dcd2d500..c7ba4aa8a07a 100644
scripts/Makefile.lib DTC_FLAGS += -Wnode_name_chars_strict \ · line 287
Quoted 10 lines
scripts/Makefile.lib +4 -0
@@ -278,6 +278,11 @@ DTC_FLAGS += -Wnode_name_chars_strict \
 ????? -Wproperty_name_chars_strict
 ? endif
 ? +ifeq ($(CONFIG_OF_OVERLAY),y)
+# enable creation of __symbols__ node
+DTC_FLAGS += -@
+endif
+
 ? DTC_FLAGS += $(DTC_FLAGS_$(basetarget))
 ? ? # Generate an assembly file to wrap the output of the device tree compiler
 
Not needed.? Instead set DTC_FLAGS in the make command.? For example:
Quoted 4 lines
??? DTC_FLAGS=-@ make qcom-apq8074-dragonboard.dtb
There are a few architecture Makefiles that need to be fixed to not unconditionally set DTC_FLAGS.
Rebuilding the dts files manually with symbols isn't the problem. The idea was to enable it with a switch which distros simply can flip.
Passing DTC_FLAGS to make would work too I guess, but on multi platform builds that's still the same issue if there're boards which can't handle that because of the size increase.
Would a solution which enables symbols per family work? E.g. CONFIG_MACH_SUN7I chooses to use -@ because it can handle it.

Quoted 6 lines

Quoted 2 lines
Thanks,
Andre
Making it easy for distros to increase the size of .dtb files is not the correct answer. We are trying to decrease the footprint of devicetree, not increase it.

I'm not sure I follow. If it's not acceptable to increase the size, why's there overlay support in the first place?

There're distros/downstreams that support overlays today, like armbian, beaglebone and rpi. All carrying their own patches to enable symbols [1]
[2] [3] and I'm sure others will follow. Why can't there be some sort of switch for those to build their dtbs with symbols?

Regards,
Andre

[1]
https://github.com/armbian/build/blob/master/patch/kernel/sunxi-next/add-overlay-compilation-support.patch#L98
[2]
https://github.com/beagleboard/linux/commit/ed6b9450c2a2ec21149f14ff24770b69888abda6
[3]
https://github.com/raspberrypi/linux/blob/rpi-4.15.y/arch/arm/boot/dts/Makefile#L1124