armv8 relocation questions

I'm hoping someone can help answer these questions about armv8 relocation.

The CONFIG_SYS_TEXT_BASE seems to be be usually setup to a decent amount of alignment. For the purposes of this discussion, let's say it would normally be 0x88000000 and all is well. The relocation address moves to near the end of memory, to say, 0xfffa8000. So far so good.

Now let's say I want to shift the image a bit so that I can add a small 32-byte header required by a previous bootloader. So I set CONFIG_SYS_TEXT_BASE to 0x88000020, and the relocated address is still 0xfffa8000 and the relocated vectors should be at 0xfffa9000. The image crashes so after some debugging, I find that the code appears to be relocated fine, but some sections have symbols that are not relocated properly. The vectors try to relocate to 0xfffa8fe0 and rodata.str1.1 printf format strings are also 0x20 off. There are likely other offset sections with issues as well.

The relocation offset is 0x77fa7fe0 due to the calculations in arch/arm/lib/board.c. Simplifying, they look like this:

        addr = CONFIG_SYS_SDRAM_BASE + gd->ram_size;

        /* round down to next 4 kB limit */
        addr &= ~(4096 - 1);
        debug("Top of RAM usable for U-Boot at: %08lx\n", addr);

        /*
         * reserve memory for U-Boot code, data & bss
         * round down to next 4 kB limit
         */
        addr -= gd->mon_len;
        addr &= ~(4096 - 1);
        
        addr += 0x20; // hack to adjust relocaddr to aligned address...

<snip>

        gd->relocaddr = addr;
        gd->start_addr_sp = addr_sp;
        gd->reloc_off = addr - _TEXT_BASE;
        debug("relocation Offset is: %08lx\n", gd->reloc_off);

Since _TEXT_BASE is 0x88000020 and addr is 0xfffa8000, the reloc_off is a number like 0x77fa7fe0.

Now if I add 0x20 to 'addr' above just before the <snip>, relocaddr becomes 0x77fa8000 and the relocation works perfectly and no more crashes happen.

So my question - is the CONFIG_SYS_TEXT_BASE alignment requirement related to to any assumptions in the linker itself about image base alignment, specifically referring to creation of the rela.dyn sections and their use for image relocation?

A related question is if CONFIG_SYS_TEXT_BASE needs to be at a specific alignment. The maximum alignment in the armv8 code base is ".align 11" which I believe means 0x800 or 2048.

Note that an armv7 target appears to relocate properly with smaller offsets such as 0x20.

Thanks.

hi Darwin,
It's a little late.

Quoted 58 lines

I'm hoping someone can help answer these questions about armv8 relocation.

The CONFIG_SYS_TEXT_BASE seems to be be usually setup to a decent amount of alignment. For the purposes of this discussion, let's say it would normally be 0x88000000 and all is well. The relocation address moves to near the end of memory, to say, 0xfffa8000. So far so good.

Now let's say I want to shift the image a bit so that I can add a small 32-byte header required by a previous bootloader. So I set CONFIG_SYS_TEXT_BASE to 0x88000020, and the relocated address is still 0xfffa8000 and the relocated vectors should be at 0xfffa9000. The image crashes so after some debugging, I find that the code appears to be relocated fine, but some sections have symbols that are not relocated properly. The vectors try to relocate to 0xfffa8fe0 and rodata.str1.1 printf format strings are also 0x20 off. There are likely other offset sections with issues as well.

The relocation offset is 0x77fa7fe0 due to the calculations in arch/arm/lib/board.c. Simplifying, they look like this:

addr = CONFIG_SYS_SDRAM_BASE + gd->ram_size;

        /* round down to next 4 kB limit */
        addr &= ~(4096 - 1);
        debug("Top of RAM usable for U-Boot at: %08lx\n", addr);

        /*
         * reserve memory for U-Boot code, data & bss
         * round down to next 4 kB limit
         */
        addr -= gd->mon_len;
        addr &= ~(4096 - 1);

        addr += 0x20; // hack to adjust relocaddr to aligned address...

<snip>

gd->relocaddr = addr;
        gd->start_addr_sp = addr_sp;
        gd->reloc_off = addr - _TEXT_BASE;
        debug("relocation Offset is: %08lx\n", gd->reloc_off);

Since _TEXT_BASE is 0x88000020 and addr is 0xfffa8000, the reloc_off is a number like 0x77fa7fe0.

Now if I add 0x20 to 'addr' above just before the <snip>, relocaddr becomes 0x77fa8000 and the relocation works perfectly and no more crashes happen.

So my question - is the CONFIG_SYS_TEXT_BASE alignment requirement related to to any assumptions in the linker itself about image base alignment, specifically referring to creation of the rela.dyn sections and their use for image relocation?

Quoted 10 lines

A related question is if CONFIG_SYS_TEXT_BASE needs to be at a specific alignment. The maximum alignment in the armv8 code base is ".align 11" which I believe means 0x800 or 2048.

Note that an armv7 target appears to relocate properly with smaller offsets such as 0x20.

Thanks.

I traced the problem you described and found it is caused by
'adrp' instruction. 'adrp' instruction produces 4kb aligned address of a label. So, if CONFIG_SYS_TEXT_BASE is not 4kb aligned the address produced by 'adrp' and following 'add' instruction will be incorrect.

      For example, if CONFIG_SYS_TEXT_BASE = 0x20 then address of 
'_start' is 0x20 and address of '_end_ofs' is 0x30, where u-boot 
access variable '_end_ofs' gcc generate code as following:
     adrp  x0,  ...
     add   x0, x0, 0x30
     We noticed that 0x30 is added to 'x0' to produce the address 

of '_end_ofs'. So when CONFIG_SYS_TEXT_BASE=0x20 and relocated destination address is not 0x****020 register x0 contain incorrect address of '_end_ofs'.

David.

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020

Any section in the image that requires a particular alignment must have that alignment respected after relocation. You cannot relocate to an arbitrary address if it breaks the maximum image alignment requirement after relocation.

But if for some reason, the hardware ever required a 0x2000 (.align 13) alignment, then the generic code's 0x1000 (.align 12) relocation alignment would not work because the alignment after relocation would not respect the .align 13 directive. We just haven't run into this issue

yet and may never do so, but it is important to understand the limitations of relocation relative to image alignment requirements. The current hardcoded 4096 (0x1000) image relocation alignment just happens to work and looks nice, that's all, but not by consideration of image alignment.

And if any text base alignment is less than the image's maximum alignment requirement, the load will fail, and then we likely scratch our heads and set the CONFIG_SYS_TEXT_BASE alignment higher until it works.

But since most people just use higher alignments naturally, this issues remain mostly hidden I think.

Dear Darwin,

Quoted 3 lines · In message <[email protected]> you wrote:

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020

You completely fail to respond to my repeated statement that a CONFIG_SYS_TEXT_BASE like this is bogus.

Quoted 9 lines · Wolfgang Denk

Dear Darwin,

Quoted 3 lines · In message <[email protected]> you wrote:

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020

You completely fail to respond to my repeated statement that a CONFIG_SYS_TEXT_BASE like this is bogus.

This. What the heck is going on? CONFIG_SYS_TEXT_BASE is where the text section ends up, and the rest of the binary that follows, until we relocate. If we're being loaded by something else, it needs to be placing us at the right spot. If there's some header on top of the image to be considered by the loader, adjust where THAT loads us. If we cannot, then you must change CONFIG_SYS_TEXT_BASE around. Pad things up a bit for proper alignment. It seems where somewhere around "Doctor, it hurts when I hit myself with a hammer!" and can't stop hammering our poor finger.

Hello Tom,

Quoted 11 lines · Tom Rini

Quoted 9 lines · Wolfgang Denk

Dear Darwin,

Quoted 3 lines · In message <[email protected]> you wrote:

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020

You completely fail to respond to my repeated statement that a CONFIG_SYS_TEXT_BASE like this is bogus.

Quoted 9 lines

This. What the heck is going on? CONFIG_SYS_TEXT_BASE is where the text section ends up, and the rest of the binary that follows, until we relocate. If we're being loaded by something else, it needs to be placing us at the right spot. If there's some header on top of the image to be considered by the loader, adjust where THAT loads us. If we cannot, then you must change CONFIG_SYS_TEXT_BASE around. Pad things up a bit for proper alignment. It seems where somewhere around "Doctor, it hurts when I hit myself with a hammer!" and can't stop hammering our poor finger.

Not really I guess, I read it as "Doctor, it hurts when I hit myself with a hammer!" But morphine prevents it to hurt, so everybody should use morphine.

But a bit more serious, as far as understood it u-boot is loaded to the CONFIG_SYS_TEXT_BASE being set, but not aligned to the requirements of adrp (which assumes 4k). Did someone find a decent description of adrp and how it should be relocated? Is adrp always required or an optimization?

We can not make gcc-aarch64 do not use adrp instruction when constructing address of label.

So, I think the 4kb alignment would be a requirement or restriction. Gcc did not declare it explicitly
due to in normal world memory are allocated with page aligned. If u-boot for aarch64 want to be compiled at address not 4kb aligned the relocated address
should also be shifted with the same offset.

Dear fenghua,

Quoted 11 lines · In message <[email protected]> you wrote:

We can not make gcc-aarch64 do not use adrp instruction when constructing address of label.

So, I think the 4kb alignment would be a requirement or restriction. Gcc did not declare it explicitly
due to in normal world memory are allocated with page aligned. If u-boot for aarch64 want to be compiled at address not 4kb aligned the relocated address
should also be shifted with the same offset.

Sorry, I don't understand anything here. At which exact place is there any such 4 k alignment restriction? When we relocate U-Boot, we just process a list of addresses. Even if the start of the image is aligned to a 4 k boundary, there are a zillion of other addresses that are not, and these can be relocated just fine.

Quoted 8 lines

Quoted 7 lines

So, I think the 4kb alignment would be a requirement or restriction. Gcc did not declare it explicitly
due to in normal world memory are allocated with page aligned. If u-boot for aarch64 want to be compiled at address not 4kb aligned the relocated address
should also be shifted with the same offset.

Quoted 6 lines

Sorry, I don't understand anything here. At which exact place is there any such 4 k alignment restriction? When we relocate U-Boot, we just process a list of addresses. Even if the start of the image is aligned to a 4 k boundary, there are a zillion of other addresses that are not, and these can be relocated just fine.

The following document [1] mentions:

"ADRP Xd, label

And apparently gcc choose to use it as such. Since the instructions in

question are relative to the most significant bits of the pc it does not need fixups, so it is not included in the "list of addresses" you mention. The compiler does create the 4k requirement though by using the instruction the way it does.

Quoted 15 lines · In message <1400416665.2394.20.camel@yellow> you wrote:

Quoted 14 lines

Quoted 7 lines

So, I think the 4kb alignment would be a requirement or restriction. Gcc did not declare it explicitly
due to in normal world memory are allocated with page aligned. If u-boot for aarch64 want to be compiled at address not 4kb aligned the relocated address
should also be shifted with the same offset.

Sorry, I don't understand anything here. At which exact place is there any such 4 k alignment restriction? When we relocate U-Boot, we just process a list of addresses. Even if the start of the image is aligned to a 4 k boundary, there are a zillion of other addresses that are not, and these can be relocated just fine.

Quoted 3 lines

The following document [1] mentions:

"ADRP Xd, label

...

Quoted 1 line

And apparently gcc choose to use it as such. Since the instructions in

Where exactly does it chose to do so?

I cannot understand why that
should be a problem for _start, but not for any of the other symbols we're relocating?

Quoted 4 lines

question are relative to the most significant bits of the pc it does not need fixups, so it is not included in the "list of addresses" you mention. The compiler does create the 4k requirement though by using the instruction the way it does.

Sorry, I cannot follow. where exactly does this happen, and why there, and not anywhere else?

Quoted 6 lines

Quoted 2 lines

"ADRP Xd, label

...

Quoted 1 line

And apparently gcc choose to use it as such. Since the instructions in

Where exactly does it chose to do so?

In the code emitter which decide to use the following instructions to look up a label, as mentioned by David [1]. Since it only uses _part of_ the pc, it introduces the need to have the relocation offset be a multiple of 4k.

     adrp  x0,  ...
     add   x0, x0, 0x30

Quoted 3 lines

I cannot understand why that
should be a problem for _start, but not for any of the other symbols we're relocating?

It is not a problem of _start. Any label / symbol which is looked up with only the adrp + compile time offset will be incorrect if the relocation offset is not n * 4k. It just that moving _start around makes the relocation offset not obey this requirement.

Quoted 2 lines

Sorry, I cannot follow. where exactly does this happen, and why there, and not anywhere else?

In any code emitted which uses above pattern. And above code works fine if text is 4k natural aligned before and after relocation. It also works fine when the same offset is used before and after relocation.

Dear Jeroen,

Quoted 7 lines · In message <1400485047.1998.17.camel@yellow> you wrote:

Quoted 6 lines

Quoted 2 lines

"ADRP Xd, label

...

Quoted 1 line

And apparently gcc choose to use it as such. Since the instructions in

Where exactly does it chose to do so?

Quoted 7 lines

In the code emitter which decide to use the following instructions to look up a label, as mentioned by David [1]. Since it only uses _part of_ the pc, it introduces the need to have the relocation offset be a multiple of 4k.

     adrp  x0,  ...
     add   x0, x0, 0x30

No. I mean: to which exact lines of U-Boot C source code will be translated into that?

I mean, from what has been discussed here that would be the relocation loop, right? And there this should not happen, as we deal with random addresses. So I wonder if this is either a coding error in the armv8 related assembler code, or a compiler bug, or something else?

Quoted 4 lines

Quoted 3 lines

I cannot understand why that
should be a problem for _start, but not for any of the other symbols we're relocating?

Quoted 4 lines

It is not a problem of _start. Any label / symbol which is looked up with only the adrp + compile time offset will be incorrect if the relocation offset is not n * 4k. It just that moving _start around makes the relocation offset not obey this requirement.

Well, in this case relocation can never work correctly, so we're back at my question above: bad assembly code of the armv8 port, or a compiler bug?

The question is - where does such code get emitted? Is it generated from C code, or manually created assembly?

mm, forget about U-Boot for a second and please consider the following, complicated, program:

#include <stdio.h>

int main(int c, char *argv[])

{
	puts("Hello World");
	return 0;
}

compiled with aarch64-linux-gnu-gcc -g -Wall hello.c results in:

int main(int c, char *argv[])
{
400590: a9be7bfd stp x29, x30, [sp,#-32]!
400594: 910003fd mov x29, sp
400598: b90013a0 str w0, [x29,#16]
40059c: f9000fa1 str x1, [x29,#24]

	puts("Hello World");
  4005a0:	90000000 	adrp	x0, 400000 <_init-0x3b8>
  4005a4:	91194000 	add	x0, x0, #0x650
  4005a8:	97ffff9e 	bl	400420 <puts@plt>
	return 0;

4005ac: 52800000 mov w0, #0x0 // #0
}
4005b0: a8c27bfd ldp x29, x30, [sp],#32
4005b4: d65f03c0 ret

Notice the adrp and hardcoded offset to load the location of the argument. The 90000000 decodes to adrp x0, 0, the value of the program counter with its lower 12bits cleared -> which is the 400000 when not being relocated. Then the harcoded 650 offset is added to point to:

Contents of section .rodata:
400648 01000200 00000000 48656c6c 6f20576f ........Hello Wo
400658 726c6400 00000000 rld.....

Quoted 3 lines

The question is - where does such code get emitted? Is it generated from C code, or manually created assembly?

It is generated, there are no literal adrp instructions in U-boot. As far as I understand it, it is a compiler feature..

Dear Jeroen Hofstee,

Quoted 3 lines · In message <1400523005.1985.33.camel@yellow> you wrote:

mm, forget about U-Boot for a second and please consider the following, complicated, program:

I think we should NOT forget U-Boot, which comes with it's own, specifically tuned set of compiler options and linker scripts.

Quoted 1 line

compiled with aarch64-linux-gnu-gcc -g -Wall hello.c results in:

I have no idea how the tool chain was configured here, and what the linker script might look like.

Maybe. This is with some (unknown here) specific set of compiler options and linker statements. This does not mean that the same or silimar code will be generated for U-Boot,

or - if it really is - that
this has to be like that. Eventually the tool chain needs some tweaking flags to tell it that we intend to relocate the image later?

Quoted 5 lines

Quoted 3 lines

The question is - where does such code get emitted? Is it generated from C code, or manually created assembly?

It is generated, there are no literal adrp instructions in U-boot. As far as I understand it, it is a compiler feature..

That depends on specific compiler options / linker settings, eventually?

Quoted 3 lines

I have no idea how the tool chain was configured here, and what the linker script might look like.

for what it is worth:
aarch64-linux-gnu-gcc (Ubuntu/Linaro 4.8.1-10ubuntu7) 4.8.1 with its default linker script.

Quoted 3 lines

Maybe. This is with some (unknown here) specific set of compiler options and linker statements. This does not mean that the same or silimar code will be generated for U-Boot,

Could be, but the vexpress_aemv8a shows the same pattern. It has around 1688 of adrp instructions.

I wondered about that. I could not find such a flag, nor did David think it exists.. [1]

Quoted 2 lines

That depends on specific compiler options / linker settings, eventually?

Perhaps eventually. But for now it seems that also U-boot is forced to keep the relocation offset a multiple of 4k.

Dear Jeroen,

Quoted 1 line · In message <1400532143.1985.57.camel@yellow> you wrote:

Quoted 6 lines

Quoted 2 lines

I have no idea how the tool chain was configured here, and what the linker script might look like.

for what it is worth:
aarch64-linux-gnu-gcc (Ubuntu/Linaro 4.8.1-10ubuntu7) 4.8.1 with its default linker script.

OK, now I know where it's coming from; did you look into this?

Quoted 6 lines

Quoted 3 lines

Maybe. This is with some (unknown here) specific set of compiler options and linker statements. This does not mean that the same or silimar code will be generated for U-Boot,

Could be, but the vexpress_aemv8a shows the same pattern. It has around 1688 of adrp instructions.

Probably with the same / similar compiler options?

Quoted 2 lines

I wondered about that. I could not find such a flag, nor did David think it exists.. [1]

You mean this statement:

We can not make gcc-aarch64 do not use adrp instruction when constructing address of label.

?

Quoted 3 lines

Perhaps eventually. But for now it seems that also U-boot is forced to keep the relocation offset a multiple of 4k.

For aarch64, you mean? Because the tool chain and/or our current understanding how it generates code requires it, right?

Dear Darwin,

Quoted 3 lines · In message <[email protected]> you wrote:

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020

You completely fail to respond to my repeated statement that a CONFIG_SYS_TEXT_BASE like this is bogus.

I guess i give up on this.

Quoted 5 lines

Any section in the image that requires a particular alignment must have that alignment respected after relocation. You cannot relocate to an arbitrary address if it breaks the maximum image alignment requirement after relocation.

Who was it who asked why we had such unreasonable strict alignment requirements for the relocation address?

Quoted 5 lines

But if for some reason, the hardware ever required a 0x2000 (.align 13) alignment, then the generic code's 0x1000 (.align 12) relocation alignment would not work because the alignment after relocation would not respect the .align 13 directive. We just haven't run into this issue

Is this a theoretical or a practical question? Where exactly do you have such a use case?

Quoted 6 lines

yet and may never do so, but it is important to understand the limitations of relocation relative to image alignment requirements. The current hardcoded 4096 (0x1000) image relocation alignment just happens to work and looks nice, that's all, but not by consideration of image alignment.

Ummm... experience from 15 years of PPCBoot / U-Boot history don't count here, I guess?

Quoted 3 lines

And if any text base alignment is less than the image's maximum alignment requirement, the load will fail, and then we likely scratch our heads and set the CONFIG_SYS_TEXT_BASE alignment higher until it works.

CONFIG_SYS_TEXT_BASE should always be reasonable aligned. There is no good reason to add arbitrary small offsets like you do. I've explained to you a feww times before that you should include your header into the text segment, and the problem would be just gone.

Quoted 3 lines

But since most people just use higher alignments naturally, this issues remain mostly hidden I think.

Not hidden, they don't exist. Non-random sig this time.

Best regards,

Wolfgang Denk

Quoted 7 lines · Wolfgang Denk

Dear Darwin,

Quoted 3 lines · In message <[email protected]> you wrote:

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020

Quoted 2 lines

You completely fail to respond to my repeated statement that a CONFIG_SYS_TEXT_BASE like this is bogus.

I want to set it this way to avoid carrying a downstream patch in start.S if the header is added by a post processing step. I understand that I can put a custom header inside start.S but wanted a different solution that keeps the code with less patches.

Quoted 47 lines

hi Darwin,
It's a little late.

Quoted 35 lines

I'm hoping someone can help answer these questions about armv8 relocation.

The CONFIG_SYS_TEXT_BASE seems to be be usually setup to a decent amount of alignment. For the purposes of this discussion, let's say it would normally be 0x88000000 and all is well. The relocation address moves to near the end of memory, to say, 0xfffa8000. So far so good.

Now let's say I want to shift the image a bit so that I can add a small 32-byte header required by a previous bootloader. So I set CONFIG_SYS_TEXT_BASE to 0x88000020, and the relocated address is still 0xfffa8000 and the relocated vectors should be at 0xfffa9000. The image crashes so after some debugging, I find that the code appears to be relocated fine, but some sections have symbols that are not relocated properly. The vectors try to relocate to 0xfffa8fe0 and rodata.str1.1 printf format strings are also 0x20 off. There are likely other offset sections with issues as well.

The relocation offset is 0x77fa7fe0 due to the calculations in arch/arm/lib/board.c. Simplifying, they look like this:

        addr = CONFIG_SYS_SDRAM_BASE + gd->ram_size;

        /* round down to next 4 kB limit */
        addr &= ~(4096 - 1);
        debug("Top of RAM usable for U-Boot at: %08lx\n", addr);

        /*
         * reserve memory for U-Boot code, data & bss
         * round down to next 4 kB limit
         */
        addr -= gd->mon_len;
        addr &= ~(4096 - 1);
        
        addr += 0x20; // hack to adjust relocaddr to aligned address...

<snip>

        gd->relocaddr = addr;
        gd->start_addr_sp = addr_sp;
        gd->reloc_off = addr - _TEXT_BASE;
        debug("relocation Offset is: %08lx\n", gd->reloc_off);

Since _TEXT_BASE is 0x88000020 and addr is 0xfffa8000, the reloc_off is a number like 0x77fa7fe0.

Now if I add 0x20 to 'addr' above just before the <snip>, relocaddr becomes 0x77fa8000 and the relocation works perfectly and no more crashes happen.

Quoted 9 lines

So my question - is the CONFIG_SYS_TEXT_BASE alignment requirement related to to any assumptions in the linker itself about image base alignment, specifically referring to creation of the rela.dyn sections and their use for image relocation?

A related question is if CONFIG_SYS_TEXT_BASE needs to be at a specific alignment. The maximum alignment in the armv8 code base is ".align 11" which I believe means 0x800 or 2048.

Note that an armv7 target appears to relocate properly with smaller offsets such as 0x20.

Thanks.

Quoted 6 lines

       I traced the problem you described and found it is caused by 'adrp' instruction. 'adrp' instruction produces 4kb aligned address of a label. So, if CONFIG_SYS_TEXT_BASE is not 4kb aligned the address produced by 'adrp' and following 'add' instruction will be incorrect.
      For example, if CONFIG_SYS_TEXT_BASE = 0x20 then address of '_start' is 0x20 and address of '_end_ofs' is 0x30, where u-boot access variable '_end_ofs' gcc generate code as following:
     adrp  x0,  ...
     add   x0, x0, 0x30
     
     We noticed that 0x30 is added to 'x0' to produce the address of '_end_ofs'. So when CONFIG_SYS_TEXT_BASE=0x20 and relocated destination address is not 0x****020 register x0 contain incorrect address of '_end_ofs'.

Thank you David. I agree that the adrp will cause problems if the string sections and label used are not relocated to correct boundaries. The adrp used before the relocation works because the symbols are on aligned boundaries.

I think I have a way to explain this problem better now.

1. Working generic code:
CONFIG_SYS_TEXT_BASE = 0x88000000
unrelocated vectors = 0x888001000
relocation address = 0xfffa8000 = 0xfffa8000 - 0x88000000
relocation offset = 77fa8000
relocated vectors = 0xfffa9000 (0x800 alignment, but pushed to fffa9000 because of code after 0xfffa8800)

Now in this case, the .align directive for the vectors section wants the vectors sitting at 0x800 alignment, which they are. When the symbols are relocated, the vectors are now at 0xfffa9000 which is aligned properly.

2. Failing offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020
unrelocated vectors = 0x888001000 (respecting .align 11 directive) relocation address = 0xfffa8000
relocation offset = 77fa7fe0 = 0xfffa8000 - 0x88000020
relocated vectors = 0xfffa8fe0 (BAD ALIGNMENT)
Now the relocated rodata.str1.1 string tables are not aligned, which I believe breaks the adrp instruction. The strings are offset by 0x20 and the printf fails.

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020
unrelocated vectors = 0x888001000 (respecting .align 11 directive) relocation address = 0xfffa8020
relocation offset = 77fa8000 = 0xfffa8020 - 0x88000020
relocated vectors = 0xfffa9000 (GOOD ALIGNMENT, respects .align 11 after relocation)

So in this fixed offset case, the adrp label is aligned to a 4K page size.

General comments:

Any section in the image that requires a particular alignment must have that alignment respected after relocation. You cannot relocate to an arbitrary address if it breaks the maximum image alignment requirement after relocation.

If the vectors want an 0x800 alignment, and the relocation puts them on a 0x1000 boundary, then that will work fine. If the string section wants a 0x1000 alignment and are relocated to a 0x1000 boundary, that also works.

But if for some reason, the hardware ever required a 0x2000 (.align 13) alignment, then the generic code's 0x1000 (.align 12) relocation alignment would not work because the alignment after relocation would not respect the .align 13 directive. We just haven't run into this issue yet and may never do so, but it is important to understand the limitations of relocation relative to image alignment requirements. The current hardcoded 4096 (0x1000) image relocation alignment just happens to work and looks nice, that's all, but not by consideration of image alignment.

And if any text base alignment is less than the image's maximum alignment requirement, the load will fail, and then we likely scratch our heads and set the CONFIG_SYS_TEXT_BASE alignment higher until it works.

Quoted 5 lines

And if any text base alignment is less than the image's maximum alignment requirement, the load will fail, and then we likely scratch our heads and set the CONFIG_SYS_TEXT_BASE alignment higher until it works.

Whatever CONFIG_SYS_TEXT_BASE is the alignment will be ok if the relocated address satisfy the align requirement. So CONFIG_SYS_TEXT_BASE could be any value(compiler maybe require the text base aligned with 0x20), just make relocated address has the same offset.

Dear David,

Quoted 10 lines · In message <[email protected]> you wrote:

Quoted 5 lines

And if any text base alignment is less than the image's maximum alignment requirement, the load will fail, and then we likely scratch our heads and set the CONFIG_SYS_TEXT_BASE alignment higher until it works.

Whatever CONFIG_SYS_TEXT_BASE is the alignment will be ok if the relocated address satisfy the align requirement. So CONFIG_SYS_TEXT_BASE could be any value(compiler maybe require the text base aligned with 0x20), just make relocated address has the same offset.

There is a point which has not been answered yet, wnd which I really would like to understand. maybe you could be so kind and try to explain it to me?

I wonder if the use of the "ardp" instruction by the compiler can be influenced or maybe even prevent for specific code parts, for example by specific compiler options? For example, there is a number of PIC/PIE-related compiler options - would these have any influence here?

I wonder how "adrp" would work in any code that is supposed to be position-independent?

I am will glad to share my understanding.

Quoted 2 lines

I wonder how "adrp" would work in any code that is supposed to be position-independent?

In my understanding, with adrp the program is still relocatable except that the text base should be 4KB aligned or the relocated address has the same offset with text base. Darwin want CONFIG_SYS_TEXT_BASE to be not 4KB aligned, so the relocated address should be offsetted in the same manner.

Position-independent code is usually page aligned, so gcc-aarch64 implicitly apply the restriction of text base.

Actually, u-boot for aarch64 apply another align requirement that is interrupt vector. The interrupt vector should be 2KB aligned. So, even without adrp CONFIG_SYS_TEXT_BASE should be 2KB aligned yet or relocated with the same offset, otherwise bug exist.

hi Darwin,
It's a little late.

Quoted 35 lines

I'm hoping someone can help answer these questions about armv8 relocation.

The CONFIG_SYS_TEXT_BASE seems to be be usually setup to a decent amount of alignment. For the purposes of this discussion, let's say it would normally be 0x88000000 and all is well. The relocation address moves to near the end of memory, to say, 0xfffa8000. So far so good.

Now let's say I want to shift the image a bit so that I can add a small 32-byte header required by a previous bootloader. So I set CONFIG_SYS_TEXT_BASE to 0x88000020, and the relocated address is still 0xfffa8000 and the relocated vectors should be at 0xfffa9000. The image crashes so after some debugging, I find that the code appears to be relocated fine, but some sections have symbols that are not relocated properly. The vectors try to relocate to 0xfffa8fe0 and rodata.str1.1 printf format strings are also 0x20 off. There are likely other offset sections with issues as well.

The relocation offset is 0x77fa7fe0 due to the calculations in arch/arm/lib/board.c. Simplifying, they look like this:

       addr = CONFIG_SYS_SDRAM_BASE + gd->ram_size;

       /* round down to next 4 kB limit */
       addr &= ~(4096 - 1);
       debug("Top of RAM usable for U-Boot at: %08lx\n", addr);

       /*
        * reserve memory for U-Boot code, data & bss
        * round down to next 4 kB limit
        */
       addr -= gd->mon_len;
       addr &= ~(4096 - 1);

       addr += 0x20; // hack to adjust relocaddr to aligned address...

<snip>

       gd->relocaddr = addr;
       gd->start_addr_sp = addr_sp;
       gd->reloc_off = addr - _TEXT_BASE;
       debug("relocation Offset is: %08lx\n", gd->reloc_off);

Since _TEXT_BASE is 0x88000020 and addr is 0xfffa8000, the reloc_off is a number like 0x77fa7fe0.

Now if I add 0x20 to 'addr' above just before the <snip>, relocaddr becomes 0x77fa8000 and the relocation works perfectly and no more crashes happen.

Quoted 9 lines

So my question - is the CONFIG_SYS_TEXT_BASE alignment requirement related to to any assumptions in the linker itself about image base alignment, specifically referring to creation of the rela.dyn sections and their use for image relocation?

A related question is if CONFIG_SYS_TEXT_BASE needs to be at a specific alignment. The maximum alignment in the armv8 code base is ".align 11" which I believe means 0x800 or 2048.

Note that an armv7 target appears to relocate properly with smaller offsets such as 0x20.

Thanks.

      I traced the problem you described and found it is caused by 'adrp' instruction. 'adrp' instruction produces 4kb aligned address of a label. So, if CONFIG_SYS_TEXT_BASE is not 4kb aligned the address produced by 'adrp' and following 'add' instruction will be incorrect.
     For example, if CONFIG_SYS_TEXT_BASE = 0x20 then address of '_start' is 0x20 and address of '_end_ofs' is 0x30, where u-boot access variable '_end_ofs' gcc generate code as following:
    adrp  x0,  ...
    add   x0, x0, 0x30

    We noticed that 0x30 is added to 'x0' to produce the address of '_end_ofs'. So when CONFIG_SYS_TEXT_BASE=0x20 and relocated destination address is not 0x****020 register x0 contain incorrect address of '_end_ofs'.

David.