Skip to contentSkip to conversation

arm: spl: Fix DTB offset after binman symbol table

PATCH In review Fabio Estevam 3 participants 10 messages Jan 23 – Jan 26 Arm (32-bit)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm: spl: Fix DTB offset after binman symbol table

From: Fabio Estevam <[email protected]>

When CONFIG_SPL_SEPARATE_BSS is disabled, SPL expects the embedded device tree to start exactly at __bss_end. This address is fixed by the linker script and SPL probes this location directly for FDT_MAGIC at runtime.

Commit 8b0ebe054bb3 ("arm: Update linker scripts to ensure appended device tree is aligned") modified the linker layout such that the .binman_sym_table section may end at a non-word-aligned offset. Since no alignment is enforced at the end of this section, the DTB is written a few bytes past __bss_end in u-boot-spl.bin, even though the __bss_end symbol itself remains correctly aligned and unchanged.

As a result, SPL reads zeroes at __bss_end instead of FDT_MAGIC, leading to a "Missing DTB" failure, while the DTB is actually present at __bss_end + 4.

Align the end of the .binman_sym_table section to 8 bytes so that the DTB is placed exactly at __bss_end as expected.

This fixes SPL boot regressions observed on Rockchip RV1103 and RV1126.

Fixes 8b0ebe054bb3 ("arm: Update linker scripts to ensure appended device tree is aligned")
---
 arch/arm/cpu/u-boot-spl.lds | 1 +
 1 file changed, 1 insertion(+)
arch/arm/cpu/u-boot-spl.lds +1 -0
index d780a5060774..fc2b7fcb969f 100644
@@ -40,6 +40,7 @@ SECTIONS
 	.binman_sym_table : {
 		__binman_sym_start = .;
 		KEEP(*(SORT(.binman_sym*)));
+		. = ALIGN(8);
 		__binman_sym_end = .;
 	}
 
Signature · 1 line
2.34.1
#1 lore raw
Quoted 33 lines · Fabio Estevam

From: Fabio Estevam <[email protected]>

When CONFIG_SPL_SEPARATE_BSS is disabled, SPL expects the embedded device tree to start exactly at __bss_end. This address is fixed by the linker script and SPL probes this location directly for FDT_MAGIC at runtime.

Commit 8b0ebe054bb3 ("arm: Update linker scripts to ensure appended device tree is aligned") modified the linker layout such that the .binman_sym_table section may end at a non-word-aligned offset. Since no alignment is enforced at the end of this section, the DTB is written a few bytes past __bss_end in u-boot-spl.bin, even though the __bss_end symbol itself remains correctly aligned and unchanged.

As a result, SPL reads zeroes at __bss_end instead of FDT_MAGIC, leading to a "Missing DTB" failure, while the DTB is actually present at __bss_end + 4.

Align the end of the .binman_sym_table section to 8 bytes so that the DTB is placed exactly at __bss_end as expected.

This fixes SPL boot regressions observed on Rockchip RV1103 and RV1126.

Fixes 8b0ebe054bb3 ("arm: Update linker scripts to ensure appended device tree is aligned")
---
 arch/arm/cpu/u-boot-spl.lds | 1 +
 1 file changed, 1 insertion(+)
arch/arm/cpu/u-boot-spl.lds +0 -0
index d780a5060774..fc2b7fcb969f 100644

arch/arm/cpu/u-boot-spl.lds SECTIONS · line 46

Quoted 7 lines
arch/arm/cpu/u-boot-spl.lds +1 -0
@@ -40,6 +40,7 @@ SECTIONS
 	.binman_sym_table : {
 		__binman_sym_start = .;
 		KEEP(*(SORT(.binman_sym*)));
+		. = ALIGN(8);
 		__binman_sym_end = .;
 	}
 

I've been talking with Ilias for the last few hours. This still isn't right, it just happens to work in some cases, but not others. What's going on is that while we always have u-boot-nodtb.bin 8-byte aligned because while the BSS itself may end 8 byte aligned, we actually care about the size of the BSS being 8 byte aligned. Can you please try:

arch/arm/cpu/u-boot-spl.lds +1 -0
index d780a5060774..dae8a1ec665b 100644
@@ -58,6 +58,7 @@ SECTIONS
 	_end = .;
 
 	.bss __rel_dyn_start (OVERLAY) : {
+		. = ALIGN(8);
 		__bss_start = .;
 		*(.bss*)
 		 . = ALIGN(8);

And see if it works at run time? hexdump'ing looks good. If so, I'll do something more broad to catch the SPL_SEPARATE_BSS=n case, and add the ASSERTS that are missing.

Signature · 1 line
Tom
author Tested-by
#2 lore raw

Hi Tom,

Quoted 19 lines · Tom Rini

I've been talking with Ilias for the last few hours. This still isn't right, it just happens to work in some cases, but not others. What's going on is that while we always have u-boot-nodtb.bin 8-byte aligned because while the BSS itself may end 8 byte aligned, we actually care about the size of the BSS being 8 byte aligned. Can you please try:

arch/arm/cpu/u-boot-spl.lds +1 -0
index d780a5060774..dae8a1ec665b 100644
@@ -58,6 +58,7 @@ SECTIONS
        _end = .;
 
        .bss __rel_dyn_start (OVERLAY) : {
+               . = ALIGN(8);
                __bss_start = .;
                *(.bss*)
                 . = ALIGN(8);
Quoted 3 lines

And see if it works at run time? hexdump'ing looks good. If so, I'll do something more broad to catch the SPL_SEPARATE_BSS=n case, and add the ASSERTS that are missing.

This fixes the boot on RV1103, thanks!

When you send a formal patch, feel free to add:

Thanks

#3 lore raw

Hi Tom, Fabio

Quoted 18 lines · Fabio Estevam

Hi Tom,

Quoted 13 lines · Tom Rini

I've been talking with Ilias for the last few hours. This still isn't right, it just happens to work in some cases, but not others. What's going on is that while we always have u-boot-nodtb.bin 8-byte aligned because while the BSS itself may end 8 byte aligned, we actually care about the size of the BSS being 8 byte aligned. Can you please try:

arch/arm/cpu/u-boot-spl.lds +0 -0
index d780a5060774..dae8a1ec665b 100644
@@ -58,6 +58,7 @@ SECTIONS
        _end = .;
 
Quoted 6 lines

arch/arm/cpu/u-boot-spl.lds SECTIONS · line 64

Quoted 6 lines
arch/arm/cpu/u-boot-spl.lds +1 -0
        .bss __rel_dyn_start (OVERLAY) : {
+               . = ALIGN(8);
                __bss_start = .;
                *(.bss*)
                 . = ALIGN(8);

I think the reason this is happening is that we *force* thje rel.dyn alignment now, and as a result we create the section. That section might be 0 or 4 bytes depending on the layout and that'w hy adding ALIGN() commands at various places fiorx the problem.
For 'fun',can you force a -dirty compilation of your broken tree? The update of version_string[], which lives in .rodata will update the alignment of your bss_end and the binary will start working...

## readelf before 8b0ebe054bb3 (no rel.dyn present) Section Headers:

  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .text             PROGBITS        00000000 001000 0086e0 00  AX  0   0 32
  [ 2] .rodata           PROGBITS        000086e0 0096e0 001d6e 00   A  0   0  4
  [ 3] .data             PROGBITS        0000a450 00b450 000538 00  WA  0   0  4
  [ 4] __u_boot_list     PROGBITS        0000a988 00b988 000798 00  WA  0   0  4
  [ 5] .binman_sym_table PROGBITS        0000b120 00c120 000014 00  WA  0   0  4
  [ 6] .bss              PROGBITS        0000b134 00c134 000054 00   W  0   0  4

[...]

## readelf after 8b0ebe054bb3

Section Headers:

  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .text             PROGBITS        00000000 001000 008848 00  AX  0   0 32
  [ 2] .rodata           PROGBITS        00008848 009848 001dfd 00   A  0   0  4
  [ 3] .data             PROGBITS        0000a648 00b648 000530 00  WA  0   0  4
  [ 4] __u_boot_list     PROGBITS        0000ab78 00bb78 000798 00  WA  0   0  4
  [ 5] .binman_sym_table PROGBITS        0000b310 00c310 000014 00  WA  0   0  4
  [ 6] .rel.dyn          REL             0000b324 00c324 000004 08   A  0   0  4
  [ 7] .bss              PROGBITS        0000b324 00c328 000054 00   W  0   0  4

[...]

Fabio can you please try this

arch/arm/cpu/u-boot-spl.lds +1 -1
index d780a5060774..af9f3a3c39c0 100644
@@ -51,8 +51,8 @@ SECTIONS
                __rel_dyn_start = .;
                *(.rel*)
                __rel_dyn_end = .;
-               . = ALIGN(8);
        }
+       . = ALIGN(8);
 
        _image_binary_end = .;
        _end = .;
Signature · 2 lines
Cheers
/Ilias
Quoted 12 lines
Quoted 3 lines

And see if it works at run time? hexdump'ing looks good. If so, I'll do something more broad to catch the SPL_SEPARATE_BSS=n case, and add the ASSERTS that are missing.

This fixes the boot on RV1103, thanks!

When you send a formal patch, feel free to add:

Thanks

#4 lore raw

Hi Ilias,

Quoted 6 lines · Ilias Apalodimas

Fabio can you please try this

arch/arm/cpu/u-boot-spl.lds +0 -0
index d780a5060774..af9f3a3c39c0 100644

arch/arm/cpu/u-boot-spl.lds SECTIONS · line 58

Quoted 9 lines
arch/arm/cpu/u-boot-spl.lds +1 -1
@@ -51,8 +51,8 @@ SECTIONS
                __rel_dyn_start = .;
                *(.rel*)
                __rel_dyn_end = .;
-               . = ALIGN(8);
        }
+       . = ALIGN(8);
 
        _image_binary_end = .;
        _end = .;

Yes, this works. This is a partial revert of 8b0ebe054bb3.

Thanks

#5 lore raw
Quoted 19 lines · Fabio Estevam

Hi Ilias,

Quoted 13 lines · Ilias Apalodimas

Fabio can you please try this

arch/arm/cpu/u-boot-spl.lds +1 -1
index d780a5060774..af9f3a3c39c0 100644
@@ -51,8 +51,8 @@ SECTIONS
                __rel_dyn_start = .;
                *(.rel*)
                __rel_dyn_end = .;
-               . = ALIGN(8);
        }
+       . = ALIGN(8);
 
Quoted 4 lines

arch/arm/cpu/u-boot-spl.lds SECTIONS · line 58

Quoted 2 lines
arch/arm/cpu/u-boot-spl.lds +0 -0
        _image_binary_end = .;
        _end = .;

Yes, this works. This is a partial revert of 8b0ebe054bb3.

Yea, i don't want to apply this, just verify what I am seeing. So, Tom the bss size doesn't have to be 8b aligned (look below). It's worth noting that we do have some implicit assumptions in our asm that rel{a}.dyn must be 4b/8b aligned in armv7/v8 respectively, but that's not what's happening here.

We want _image_binary_end to still be 8b aligned etc. Since ALIGN() has been moved into the dyn section, we now create a section for .dyn in SPL that was previously empty. This new section also gets copied by objcopy. That section will be 0 or 4 bytes depending on the alignment of the preceding code. That's what blows up on Fabios set up.

We can probably apply the patch you suggested, but we need comments in the linker script + an update to the commit message

Cheers
/Ilias

Quoted 2 lines

Thanks

#6 lore raw
Quoted 31 lines · Ilias Apalodimas
Quoted 19 lines · Fabio Estevam

Hi Ilias,

Quoted 13 lines · Ilias Apalodimas

Fabio can you please try this

arch/arm/cpu/u-boot-spl.lds +1 -1
index d780a5060774..af9f3a3c39c0 100644
@@ -51,8 +51,8 @@ SECTIONS
                __rel_dyn_start = .;
                *(.rel*)
                __rel_dyn_end = .;
-               . = ALIGN(8);
        }
+       . = ALIGN(8);
 
Quoted 4 lines

arch/arm/cpu/u-boot-spl.lds SECTIONS · line 58

Quoted 2 lines
arch/arm/cpu/u-boot-spl.lds +0 -0
        _image_binary_end = .;
        _end = .;

Yes, this works. This is a partial revert of 8b0ebe054bb3.

Yea, i don't want to apply this, just verify what I am seeing. So, Tom the bss size doesn't have to be 8b aligned (look below). It's worth noting that we do have some implicit assumptions in our asm that rel{a}.dyn must be 4b/8b aligned in armv7/v8 respectively, but that's not what's happening here.

Quoted 8 lines

We want _image_binary_end to still be 8b aligned etc. Since ALIGN() has been moved into the dyn section, we now create a section for .dyn in SPL that was previously empty. This new section also gets copied by objcopy. That section will be 0 or 4 bytes depending on the alignment of the preceding code. That's what blows up on Fabios set up.

We can probably apply the patch you suggested, but we need comments in the linker script + an update to the commit message

What should the commit message say? I'm loathe to add big comments at least to the linker scripts because while we hit this on armv7 it's another case of needing to modify most of the linker scripts. But a "/* Ensure alignmnet for device tree appending */" is fine, esp since git blame will point out the real details in the commit message. And since you mentioned on IRC maybe being able to turn Fabio's hexdump in to a build time check, we really need that too.

Signature · 1 line
Tom
#7 lore raw
In reply to Fabio Estevam
Quoted 29 lines · Fabio Estevam

Hi Tom,

Quoted 19 lines · Tom Rini

I've been talking with Ilias for the last few hours. This still isn't right, it just happens to work in some cases, but not others. What's going on is that while we always have u-boot-nodtb.bin 8-byte aligned because while the BSS itself may end 8 byte aligned, we actually care about the size of the BSS being 8 byte aligned. Can you please try:

arch/arm/cpu/u-boot-spl.lds +1 -0
index d780a5060774..dae8a1ec665b 100644
@@ -58,6 +58,7 @@ SECTIONS
        _end = .;
 
        .bss __rel_dyn_start (OVERLAY) : {
+               . = ALIGN(8);
                __bss_start = .;
                *(.bss*)
                 . = ALIGN(8);
Quoted 3 lines

And see if it works at run time? hexdump'ing looks good. If so, I'll do something more broad to catch the SPL_SEPARATE_BSS=n case, and add the ASSERTS that are missing.

This fixes the boot on RV1103, thanks!

Quoted 4 lines

When you send a formal patch, feel free to add:

After talking more with Ilias, we're going to handle this in two-steps. Step one gets more platforms booting today. It's *slightly* different than what I posted here, however it's binary identical (I just checked and got the same checksum as this patch produces, build-testing sonoff-ihost-rv1126). So I'll add your tags to the commit.

Signature · 1 line
Tom
#8 lore raw

Hi Tom,

Quoted 5 lines · Tom Rini

After talking more with Ilias, we're going to handle this in two-steps. Step one gets more platforms booting today. It's *slightly* different than what I posted here, however it's binary identical (I just checked and got the same checksum as this patch produces, build-testing sonoff-ihost-rv1126). So I'll add your tags to the commit.

Sounds good.

When you send the formal patch, I'll test it again on RV1103 to be on the safe side.

Thanks

#9 lore raw
Quoted 10 lines · Fabio Estevam

Hi Tom,

Quoted 5 lines · Tom Rini

After talking more with Ilias, we're going to handle this in two-steps. Step one gets more platforms booting today. It's *slightly* different than what I posted here, however it's binary identical (I just checked and got the same checksum as this patch produces, build-testing sonoff-ihost-rv1126). So I'll add your tags to the commit.

Quoted 4 lines

Sounds good.

When you send the formal patch, I'll test it again on RV1103 to be on the safe side.

That's good. I'm back to thinking that what we've been doing and having you test so far is still a work-around because it's really that when we overlay the BSS, we used to always have a size 0 .rel.dyn, and now sometimes it's 0 and sometimes it's 4 (as Ilias noted), and the changes I've proposed, in limited testing, mean it's been size 0, but I don't know that's a guaranty.

Signature · 1 line
Tom