ARM: relocate: fix hang when CONFIG_ARMV7_SECURE_BASE

When added above configuration, iram fix up plus relocate offset may locate in invalidate space. Write back fix up value will cause data abort.

Add address check, skip psci code.

Signed-off-by Frank Li <[email protected]>

---
arch/arm/lib/relocate.S | 4 ++++

1 file changed, 4 insertions(+)

arch/arm/lib/relocate.S +0 -0

index 475d503..6795a1b 100644

arch/arm/lib/relocate.S +0 -0

 	/* relative fix: increase location by offset */
 	add	r0, r0, r4
 	ldr	r1, [r0]

Signature · 6 lines

2.5.2

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Quoted 18 lines · Frank Li

When added above configuration, iram fix up plus relocate offset may locate in invalidate space. Write back fix up value will cause data abort.

Add address check, skip psci code.

Signed-off-by Frank Li <[email protected]>

---
arch/arm/lib/relocate.S | 4 ++++

1 file changed, 4 insertions(+)

arch/arm/lib/relocate.S +0 -0

index 475d503..6795a1b 100644
@@ -99,6 +99,10 @@ fixloop:
 	cmp	r1, #23			/* relative fixup? */
 	bne	fixnext
 

arch/arm/lib/relocate.S fixloop: · line 105

Quoted 4 lines

arch/arm/lib/relocate.S +4 -0

+	ldr	r1, =__image_copy_start
+	cmp	r0, r1
+	blo	fixnext
+

Hi Tom, Albert,

This is a bug fix, please consider to apply this patch.

The secure code such as PSCI is not relocated, so there is no need to fix the code which generate relocate entry in rel.dyn section. We should only need take code from __image_copy_start to __image_copy_end into consideration.

Regards,
Peng.

Hello Peng,

Quoted 26 lines · Peng Fan

Quoted 15 lines · Frank Li

When added above configuration, iram fix up plus relocate offset may locate in invalidate space. Write back fix up value will cause data abort.

Add address check, skip psci code.

Signed-off-by Frank Li <[email protected]>

---
arch/arm/lib/relocate.S | 4 ++++

1 file changed, 4 insertions(+)

arch/arm/lib/relocate.S +0 -0

index 475d503..6795a1b 100644

arch/arm/lib/relocate.S fixloop: · line 105

Quoted 7 lines

arch/arm/lib/relocate.S +4 -0

@@ -99,6 +99,10 @@ fixloop:
 	cmp	r1, #23			/* relative fixup? */
 	bne	fixnext
 
+	ldr	r1, =__image_copy_start
+	cmp	r0, r1
+	blo	fixnext
+

Hi Tom, Albert,

Quoted 1 line

This is a bug fix, please consider to apply this patch.

Sorry for not spotting your patch earlier.

Took me some time to understand the commit summary. Let me see if I'm getting this right by paraphrasing it:

If CONFIG_ARMV7_SECURE_BASE is defined and if there is a fixup to the location at __image_copy_start, then U-Boot will try to fix that location and since it is write-protected, it will result in a data abort.

If I got this right, then this raises the following questions:

1) if this location cannot be written into for relocation fixup, then how could it be written into for the copying which precedes relocation?

2) if there is a fixup to the location, it means that either this location will not work properly without a fix, or the compiler emitted an erroneous relocation record. In either case, just ignoring the fixup seems like papering over the issue.

Besides, this patch will prevent image base fixups on all targets, even those for which CONFIG_ARMV7_SECURE_BASE is not defined.

So, for now, I would NAK it.

Now, assuming the answers to the two questions above do make a valid point that the fix above should indeed be implemented, then:

Quoted 3 lines

The secure code such as PSCI is not relocated, so there is no need to fix the code which generate relocate entry in rel.dyn section. We should only need take code from __image_copy_start to __image_copy_end into consideration.

If some part of an image needs copying but not relocating,then the

right solution is not to hard-code some arbitrary location as 'not relocatable' in the relocation routine; the right solution is to put in place a generic mechanism to allow the linker script to define which part of the image is relocatable. This can be done as follows:

- in the linker script, border the non-relocatable part of the image with two symbols, say 'relocatable_image_start' and '..._end', and ensure that text (code) which should *not* be relocated is mapped either before '..._start' or '..._end'. Not-to-be-relocated code would be recognized by its text section name, e.g. '.nonreloc.text*'.

- in the relocation routine, only fix up those locations which lie between the 'relocatable_image_start' and '..._end' symbols.

- in the relevant makefile(s), make the non-relocatable object files use a test section name of the for defined in the first step (e.g. '.nonreloc.text*').

Again, this is suggested *only* if it is shown that the data abort cannot be avoided some other way.

Quoted 2 lines

Regards,
Peng.

Amicalement,

Signature · 1 line

Albert.

Hi Albert,

Quoted 30 lines · Albert ARIBAUD

Hello Peng,

Quoted 26 lines · Peng Fan

Quoted 15 lines · Frank Li

When added above configuration, iram fix up plus relocate offset may locate in invalidate space. Write back fix up value will cause data abort.

Add address check, skip psci code.

Signed-off-by Frank Li <[email protected]>

---
arch/arm/lib/relocate.S | 4 ++++

1 file changed, 4 insertions(+)

arch/arm/lib/relocate.S +0 -0

index 475d503..6795a1b 100644

arch/arm/lib/relocate.S fixloop: · line 105

Quoted 7 lines

arch/arm/lib/relocate.S +4 -0

@@ -99,6 +99,10 @@ fixloop:
 	cmp	r1, #23			/* relative fixup? */
 	bne	fixnext
 
+	ldr	r1, =__image_copy_start
+	cmp	r0, r1
+	blo	fixnext
+

Hi Tom, Albert,

Quoted 3 lines

Quoted 1 line

This is a bug fix, please consider to apply this patch.

Sorry for not spotting your patch earlier.

Not from me -:)

Quoted 8 lines

Took me some time to understand the commit summary. Let me see if I'm getting this right by paraphrasing it:

If CONFIG_ARMV7_SECURE_BASE is defined and if there is a fixup to the location at __image_copy_start, then U-Boot will try to fix that location and since it is write-protected, it will result in a data abort.

The commit msg needs to be refined.

Quoted 11 lines

If I got this right, then this raises the following questions:

1) if this location cannot be written into for relocation fixup, then how could it be written into for the copying which precedes relocation?

2) if there is a fixup to the location, it means that either this location will not work properly without a fix, or the compiler emitted an erroneous relocation record. In either case, just ignoring the fixup seems like papering over the issue.

Quoted 2 lines

Besides, this patch will prevent image base fixups on all targets, even those for which CONFIG_ARMV7_SECURE_BASE is not defined.

From my understanding, U-Boot will be relocated to high address of the DRAM space, exactly code from __image_copy_start to __image_copy_end.

Following is part of the dump of .rel.dyn section of u-boot elf. SECURE BASE is 0x180000, uboot entry is 0x87800000.

Relocation section '.rel.dyn' at offset 0x577d4 contains 4032 entries:
Offset Info Type Sym.Value Sym. Name

 0018410c  00000017 R_ARM_RELATIVE   
 00184154  00000017 R_ARM_RELATIVE   
 0018415c  00000017 R_ARM_RELATIVE   
 00184164  00000017 R_ARM_RELATIVE   
 0018416c  00000017 R_ARM_RELATIVE   
 00184304  00000017 R_ARM_RELATIVE   
 00184368  00000017 R_ARM_RELATIVE   
 87800020  00000017 R_ARM_RELATIVE   
 87800024  00000017 R_ARM_RELATIVE   
 87800028  00000017 R_ARM_RELATIVE   
 8780002c  00000017 R_ARM_RELATIVE

We can see that there are several relocate entries for secure code which is not copied to high address of DRAM.

However SECURE code will not be copied to high address and should not be copied. The code locates from __image_copy_start to __image_copy_end are copied to high address, after copied, we need to fix variable and function reference, means we need to fix this according to the relocation entry. The relocation entry for SECURE code is not needed and should not to be fixed, alought they are in the rel.dyn section.

Quoted 5 lines

So, for now, I would NAK it.

Now, assuming the answers to the two questions above do make a valid point that the fix above should indeed be implemented, then:

Quoted 5 lines

Quoted 3 lines

The secure code such as PSCI is not relocated, so there is no need to fix the code which generate relocate entry in rel.dyn section. We should only need take code from __image_copy_start to __image_copy_end into consideration.

If some part of an image needs copying but not relocating,then the

part of an image not needs copying and not needs relocating.

Quoted 10 lines

right solution is not to hard-code some arbitrary location as 'not relocatable' in the relocation routine; the right solution is to put in place a generic mechanism to allow the linker script to define which part of the image is relocatable. This can be done as follows:

- in the linker script, border the non-relocatable part of the image with two symbols, say 'relocatable_image_start' and '..._end', and ensure that text (code) which should *not* be relocated is mapped either before '..._start' or '..._end'. Not-to-be-relocated code would be recognized by its text section name, e.g. '.nonreloc.text*'.

We have another fix is to add the following linker in u-boot.lds for arm

Then relocation entry for secure code will not be touched.

But from my point, when doing relocation fix like the following c code:

if ((entry < __image_copy_end)  && (entry > __image_copy_start)) {
     /* fixup according relocation entry */
}

Regards,
Peng.

Hello Peng,

Quoted 60 lines · Peng Fan

Hi Albert,

Quoted 30 lines · Albert ARIBAUD

Hello Peng,

Quoted 26 lines · Peng Fan

Quoted 15 lines · Frank Li

When added above configuration, iram fix up plus relocate offset may locate in invalidate space. Write back fix up value will cause data abort.

Add address check, skip psci code.

Signed-off-by Frank Li <[email protected]>

---
arch/arm/lib/relocate.S | 4 ++++

1 file changed, 4 insertions(+)

arch/arm/lib/relocate.S +0 -0

index 475d503..6795a1b 100644

arch/arm/lib/relocate.S fixloop: · line 105

Quoted 7 lines

arch/arm/lib/relocate.S +4 -0

@@ -99,6 +99,10 @@ fixloop:
 	cmp	r1, #23			/* relative fixup? */
 	bne	fixnext
 
+	ldr	r1, =__image_copy_start
+	cmp	r0, r1
+	blo	fixnext
+

Hi Tom, Albert,

Quoted 3 lines

Quoted 1 line

This is a bug fix, please consider to apply this patch.

Sorry for not spotting your patch earlier.

Not from me -:)

Quoted 8 lines

Took me some time to understand the commit summary. Let me see if I'm getting this right by paraphrasing it:

If CONFIG_ARMV7_SECURE_BASE is defined and if there is a fixup to the location at __image_copy_start, then U-Boot will try to fix that location and since it is write-protected, it will result in a data abort.

The commit msg needs to be refined.

Quoted 11 lines

If I got this right, then this raises the following questions:

1) if this location cannot be written into for relocation fixup, then how could it be written into for the copying which precedes relocation?

2) if there is a fixup to the location, it means that either this location will not work properly without a fix, or the compiler emitted an erroneous relocation record. In either case, just ignoring the fixup seems like papering over the issue.

Quoted 5 lines

Quoted 2 lines

Besides, this patch will prevent image base fixups on all targets, even those for which CONFIG_ARMV7_SECURE_BASE is not defined.

From my understanding, U-Boot will be relocated to high address of the DRAM space, exactly code from __image_copy_start to __image_copy_end.

Correct [minus the pedantic nitpick that __image_copy_start is included in the copy while __image_copy_end is excluded from it].

Quoted 20 lines

Following is part of the dump of .rel.dyn section of u-boot elf. SECURE BASE is 0x180000, uboot entry is 0x87800000.

Relocation section '.rel.dyn' at offset 0x577d4 contains 4032 entries:
Offset Info Type Sym.Value Sym. Name

 0018410c  00000017 R_ARM_RELATIVE   
 00184154  00000017 R_ARM_RELATIVE   
 0018415c  00000017 R_ARM_RELATIVE   
 00184164  00000017 R_ARM_RELATIVE   
 0018416c  00000017 R_ARM_RELATIVE   
 00184304  00000017 R_ARM_RELATIVE   
 00184368  00000017 R_ARM_RELATIVE   
 87800020  00000017 R_ARM_RELATIVE   
 87800024  00000017 R_ARM_RELATIVE   
 87800028  00000017 R_ARM_RELATIVE   
 8780002c  00000017 R_ARM_RELATIVE

We can see that there are several relocate entries for secure code which is not copied to high address of DRAM.

Quoted 6 lines

However SECURE code will not be copied to high address and should not be copied. The code locates from __image_copy_start to __image_copy_end are copied to high address, after copied, we need to fix variable and function reference, means we need to fix this according to the relocation entry. The relocation entry for SECURE code is not needed and should not to be fixed, alought they are in the rel.dyn section.

Thanks, this makes the situation much clearer -- though not entirely.

Relocation is done by:

a) compiling and linking the image as relocatable code;

a) linking the image for loading and execution at the link-time base address (here 0x87800000); the image won't work at any other address without relocation;

b) copying the code from wherever it was loaded (which may or may not be the link-time base address) to its run-time base address (as high in DDR as possible); this makes the code unfit for execution until it is relocated [pedantry: ... unless the run-time base address happens to be the link-time base address] ;

c) fixing the image based on the relocation records, the link-time base address, and the run-time base address; this makes the code fit for execution again.

So obviously, phase c should only operate on code which was linked between __image_copy_start [included] and __image_copy_end [excluded].

This could be an argument for checking relocation targets against those limits... But then, only code that *requires* relocation should have any relocation record -- IOW, there is no reason for the PSCI code to have such records if it is not going to be relocated like the rest of the image.

This, in turn, leads to new questions:

1. How is this PSCI code put in place? Is it bundled with the image, with a specificy copy routine which puts it in place then locks the

memory range against writing? Or is it loaded in place by some other agent than U-Boot, and how does this agent know what to put where?

2. Does this code and the relocatable image refer to symbols of one another,

or do they interact through an IPC independent of their
respective location (in which case, one wonders why they are built into a single ELF file)?

More generally... which target do you see this problem on? Reproducing the build myself would save both you and me some time, I guess. :)

Quoted 10 lines

Quoted 10 lines

So, for now, I would NAK it.

Now, assuming the answers to the two questions above do make a valid point that the fix above should indeed be implemented, then:

Quoted 3 lines

The secure code such as PSCI is not relocated, so there is no need to fix the code which generate relocate entry in rel.dyn section. We should only need take code from __image_copy_start to __image_copy_end into consideration.

Quoted 3 lines

Quoted 1 line

If some part of an image needs copying but not relocating,then the

part of an image not needs copying and not needs relocating.

This seems to imply the PSCI code is not even put in place by U-Boot. So how does it end up in place?

Quoted 12 lines

Quoted 11 lines

right solution is not to hard-code some arbitrary location as 'not relocatable' in the relocation routine; the right solution is to put in place a generic mechanism to allow the linker script to define which part of the image is relocatable. This can be done as follows:

- in the linker script, border the non-relocatable part of the image with two symbols, say 'relocatable_image_start' and '..._end', and ensure that text (code) which should *not* be relocated is mapped either before '..._start' or '..._end'. Not-to-be-relocated code would be recognized by its text section name, e.g. '.nonreloc.text*'.

Review

Quoted 5 lines

We have another fix is to add the following linker in u-boot.lds for arm

(quoted diff) +4 -0

+	.rel.secure :
+	{
+		*(.rel._secure*)
+	}

Do you need these relocation records? If not, then just append the '*(.rel._secure*) input section spec (with a suitable comment) to the already existing DISCARD output section. By the way, doesn't this linker script change alone fix the data abort?

Still, if there *are* relocation records emitted, that's because the toolchain considered them necessary -- IOW, it was (wrongly) told the PSCI code must be relocatable [or the code really is relocatable and we just haven't not hit a bug about this yet].

So the real root cause fix would be to add suitable compiler and linker flags to the PSCI object files so that they are generated as really non- relocatable.

(still, I am wondering how this code ends up in place without U-Boot at least copying it from the loaded image into its target location; I'll certainly get a better understanding of it once I know which target we're talking about.)

Quoted 7 lines

Then relocation entry for secure code will not be touched.

But from my point, when doing relocation fix like the following c code:

if ((entry < __image_copy_end)  && (entry > __image_copy_start)) {
     /* fixup according relocation entry */
}

Testing every single relocation record target against __image_copy_start and __image_copy_end would be rather costly in the tight loop of the relocation routine, which should run as fast as possible.

There is at least one fix to your problem which removes the out-of- bounds relocation records, and quite probably one better fix which prevents generating them in the first place.

Therefore, instead of the currently proposed patch which increases the size (very slightly) and boot time (statistically in O(n) proportion) of the image, I prefer one of the two alternative solutions which decrease the image size (by removing useless relocation records) and leave the boot time unchanged (since the fix is at build time only).

Quoted 3 lines

Regards,
Peng.

Amicalement,

Signature · 1 line

Albert.

Quoted 4 lines

This, in turn, leads to new questions:

1. How is this PSCI code put in place? Is it bundled with the image, with a specificy copy routine which puts it in place then locks the

Yeah.

Quoted 2 lines

memory range against writing? Or is it loaded in place by some other agent than U-Boot, and how does this agent know what to put where?

In arch/arm/cpu/armv7/virt-v7.c, relocate_secure_section() does this job.

CONFIG_ARMV7_SECURE_BASE is the location that secure code will be copied to. To ARMV7, PSCI code is bundled with the uboot image.

CONFIG_ARMV7_SECURE_BASE is the location that secure code will be copied to. To ARMV7, PSCI code is bundled with the uboot image.

Quoted 3 lines

2. Does this code and the relocatable image refer to symbols of one another,

Yeah. There is asm function reference. You can see arch/arm/cpu/armv7/psci.S

Quoted 3 lines

or do they interact through an IPC independent of their
respective location (in which case, one wonders why they are built into a single ELF file)?

This comes a question, why PSCI framework intergated into U-Boot? I have no idea.

Quoted 2 lines

More generally... which target do you see this problem on? Reproducing the build myself would save both you and me some time, I guess. :)

Build target: mx7dsabresd

Needs to apply the three patches:
https://patchwork.ozlabs.org/patch/527040/
https://patchwork.ozlabs.org/patch/527038/
https://patchwork.ozlabs.org/patch/527039/

There is a small difference from my previous log, since my previous log use 0x180000 as the secure address, but the patches use 0x900000 as the secure address. But sure there will be relocation entry there.

Quoted 4 lines

Do you need these relocation records? If not, then just append the '*(.rel._secure*) input section spec (with a suitable comment) to the already existing DISCARD output section. By the way, doesn't this linker script change alone fix the data abort?

Yeah. Since the relocation entry for secure section will be stored in rel.secure section. And this section will not be touched in relocate.S

Quoted 4 lines

Still, if there *are* relocation records emitted, that's because the toolchain considered them necessary -- IOW, it was (wrongly) told the PSCI code must be relocatable [or the code really is relocatable and we just haven't not hit a bug about this yet].

The PSCI code is bundled with the u-boot image. But it's running address is not same with u-boot. In my example, u-boot is compiled with starting address 0x87800000, but to PSCI, it's running address is 0x180000. relocate_secure_section is just copy the psci code from uboot to 0x180000.

compliation address is same with running address, to PSCI part.

Quoted 8 lines

Testing every single relocation record target against __image_copy_start and __image_copy_end would be rather costly in the tight loop of the relocation routine, which should run as fast as possible.

There is at least one fix to your problem which removes the out-of- bounds relocation records, and quite probably one better fix which prevents generating them in the first place.

Quoted 5 lines

Therefore, instead of the currently proposed patch which increases the size (very slightly) and boot time (statistically in O(n) proportion) of the image, I prefer one of the two alternative solutions which decrease the image size (by removing useless relocation records) and leave the boot time unchanged (since the fix is at build time only).

Agree.

Regards,
Peng.

Hello Peng,

(cutting a bit through the previous mails quoting)

Quoted 4 lines

Quoted 4 lines

This, in turn, leads to new questions:

1. How is this PSCI code put in place? Is it bundled with the image, with a specificy copy routine which puts it in place then locks the

Quoted 8 lines

Yeah.

Quoted 2 lines

memory range against writing? Or is it loaded in place by some other agent than U-Boot, and how does this agent know what to put where?

In arch/arm/cpu/armv7/virt-v7.c, relocate_secure_section() does this job.

CONFIG_ARMV7_SECURE_BASE is the location that secure code will be copied to. To ARMV7, PSCI code is bundled with the uboot image.

Hmm, the 'relocate' part of the function name is a somewhat non-optimal choice, since the routine just copies data without modifying it.

Looking at relocate_secure_section(), I see that it it is called long after the U-Boot code was relocated -- actually, it is called during bootm.

This means that for any one of the other boards around that seem to use PSCI, either "their" PSCI code does not have any relocations (which I doubt), or it has but they don't cause any data abort when done by the relocate_code() routine.

So what's the difference between those and your board? My bet is that their CONFIG_ARMV7_SECURE_BASE is not write-protected at the time the relocate_code() routine executes (whereas on your board it is), and will be unlocked in some way by the time relocate_secure_section() executes.

I'm not saying that the correct solution would be to enable write access to CONFIG_ARMV7_SECURE_BASE before running relocate_code(), because doing that would be obviously wrong: relocate_code() would try to fix code which is not there yet.

I'm just saying that's what actually happens, and if I am right, then it confirms that the correct fix is to not let relocations to the secure stay in the U-Boot ELF, or better yet, to not let them happen to boot [pun half-intended].

Quoted 8 lines

CONFIG_ARMV7_SECURE_BASE is the location that secure code will be copied to. To ARMV7, PSCI code is bundled with the uboot image.

Quoted 3 lines

2. Does this code and the relocatable image refer to symbols of one another,

Yeah. There is asm function reference. You can see arch/arm/cpu/armv7/psci.S

Looking at this file, I suspect that actually, PSCI is called through a software interrupt / exception vector, not through a function name, and the only symbolic relationship is via __secure_start and __secure_end -- and those must (and will) be correctly relocated along with the U-Boot image.

Quoted 5 lines

Quoted 3 lines

or do they interact through an IPC independent of their
respective location (in which case, one wonders why they are built into a single ELF file)?

This comes a question, why PSCI framework intergated into U-Boot? I have no idea.

There could have been alternative designs, indeed. Here is the design now, so let's handle it.

Quoted 5 lines

Quoted 2 lines

More generally... which target do you see this problem on? Reproducing the build myself would save both you and me some time, I guess. :)

Build target: mx7dsabresd

Quoted 8 lines

Needs to apply the three patches:
https://patchwork.ozlabs.org/patch/527040/
https://patchwork.ozlabs.org/patch/527038/
https://patchwork.ozlabs.org/patch/527039/

There is a small difference from my previous log, since my previous log use 0x180000 as the secure address, but the patches use 0x900000 as the secure address. But sure there will be relocation entry there.

Thanks. I'll try and play with compiler / linker options, but from my own experience, this can be tricky. Meanwhile, I suggest you for for the not-too-quick-and-not-too-dirty linker script solution.

Quoted 7 lines

Quoted 4 lines

Do you need these relocation records? If not, then just append the '*(.rel._secure*) input section spec (with a suitable comment) to the already existing DISCARD output section. By the way, doesn't this linker script change alone fix the data abort?

Yeah. Since the relocation entry for secure section will be stored in rel.secure section. And this section will not be touched in relocate.S

Ok, so that's our first clean, linker-script-based, solution confirmed.

Quoted 10 lines

Quoted 4 lines

Still, if there *are* relocation records emitted, that's because the toolchain considered them necessary -- IOW, it was (wrongly) told the PSCI code must be relocatable [or the code really is relocatable and we just haven't not hit a bug about this yet].

The PSCI code is bundled with the u-boot image. But it's running address is not same with u-boot. In my example, u-boot is compiled with starting address 0x87800000, but to PSCI, it's running address is 0x180000. relocate_secure_section is just copy the psci code from uboot to 0x180000.

Quoted 1 line

compliation address is same with running address, to PSCI part.

(I understand that. My question was not "why is PSCI built for its run-time address?" but "Since PSCI is built for its run-time address, and since there is little dependency on how come it is built as relocatable?")

Quoted 9 lines

Quoted 9 lines

Testing every single relocation record target against __image_copy_start and __image_copy_end would be rather costly in the tight loop of the relocation routine, which should run as fast as possible.

There is at least one fix to your problem which removes the out-of- bounds relocation records, and quite probably one better fix which prevents generating them in the first place.

Quoted 7 lines

Quoted 5 lines

Therefore, instead of the currently proposed patch which increases the size (very slightly) and boot time (statistically in O(n) proportion) of the image, I prefer one of the two alternative solutions which decrease the image size (by removing useless relocation records) and leave the boot time unchanged (since the fix is at build time only).

Agree.

OK. So, for now, I suggest that you:

- resubmit v2 of this series with patch 1/3 reworked into a linker script change rather than a relocate routine change (and collect the secure relocation records input sections into the DISCARD output section rather than into a purposeless non-discarded section);

- make sure that you CC: the maintainers for the other boards which use PSCI and explicitly ask them to test the change on their boards and give their "Tested-by". It seems like the list of candidates for testing includes jetson-tk1, ls1021aqds, ls1021atwr and all sun[678]i boards, but I am by no means a PSCI specialist, so anyone feel free to correct me about this list.

Quoted 2 lines

Regards,
Peng.

Amicalement,

Signature · 1 line

Albert.

-----Original Message-----
From: Peng Fan [mailto:b51431 at freescale.com]
Sent: Monday, October 19, 2015 7:47 AM
To: Albert ARIBAUD <[email protected]>
Cc: Li Frank-B20596 <[email protected]>; lznuaa at gmail.com; u- boot at lists.denx.de; sbabic at denx.de; Estevam Fabio-R49496 <[email protected]>; otavio at ossystems.com.br; trini at konsulko.com; hdegoede at redhat.com; Fan Peng-B51431 <[email protected]>
Subject: Re: [U-Boot] [PATCH 1/3] ARM: relocate: fix hang when CONFIG_ARMV7_SECURE_BASE

Hi Albert,

Quoted 18 lines · Albert ARIBAUD

Hello Peng,

(cutting a bit through the previous mails quoting)

Quoted 5 lines

Quoted 5 lines

This, in turn, leads to new questions:

1. How is this PSCI code put in place? Is it bundled with the image, with a specificy copy routine which puts it in place then locks
the

Quoted 8 lines

Yeah.

Quoted 2 lines

memory range against writing? Or is it loaded in place by some other agent than U-Boot, and how does this agent know what to put where?

In arch/arm/cpu/armv7/virt-v7.c, relocate_secure_section() does this job.

CONFIG_ARMV7_SECURE_BASE is the location that secure code will be copied to. To ARMV7, PSCI code is bundled with the uboot image.

Quoted 6 lines

Hmm, the 'relocate' part of the function name is a somewhat non-optimal choice, since the routine just copies data without modifying it.

Looking at relocate_secure_section(), I see that it it is called long after the U-Boot code was relocated -- actually, it is called during bootm.

oh. You remind me, thanks. There maybe something wrong with my previous analysis.

I am not the one who finds the issue, just my understanding. The data abort may be triggered by line 104 or line 106. 96 fixloop:

97         ldmia   r2!, {r0-r1}            /* (r0,r1) <- (SRC location,fixup) */
98         and     r1, r1, #0xff
99         cmp     r1, #23                 /* relative fixup? */
100         bne     fixnext
101
102         /* relative fix: increase location by offset */
103         add     r0, r0, r4
104         ldr     r1, [r0]
105         add     r1, r1, r4
106         str     r1, [r0]

At line 104 or 106, the value of r0 may be an address that can not be accessed which trigger data abort.

Frank, am I right?

I'll set up one board to test this.

Quoted 5 lines

This means that for any one of the other boards around that seem to use PSCI, either "their" PSCI code does not have any relocations (which I doubt), or it has but they don't cause any data abort when done by the relocate_code() routine.

There maybe something wrong from me. See above.

Quoted 11 lines

So what's the difference between those and your board? My bet is that their CONFIG_ARMV7_SECURE_BASE is not write-protected at the time the relocate_code() routine executes (whereas on your board it is), and will be unlocked in some way by the time relocate_secure_section() executes.

I'm not saying that the correct solution would be to enable write access to CONFIG_ARMV7_SECURE_BASE before running relocate_code(), because doing that would be obviously wrong: relocate_code() would try to fix code which is not there yet.

Yeah. This is true, relocate_code may fix code which is not there.

Quoted 5 lines

I'm just saying that's what actually happens, and if I am right, then it confirms that the correct fix is to not let relocations to the secure stay in the U-Boot ELF, or better yet, to not let them happen to boot [pun half-intended].

They do not happen to boot. My bad. See above.

Quoted 7 lines

Quoted 7 lines

CONFIG_ARMV7_SECURE_BASE is the location that secure code will be copied to. To ARMV7, PSCI code is bundled with the uboot image.

Quoted 3 lines

2. Does this code and the relocatable image refer to symbols of one another,

Quoted 14 lines

Quoted 2 lines

Yeah. There is asm function reference. You can see arch/arm/cpu/armv7/psci.S

Looking at this file, I suspect that actually, PSCI is called through a software interrupt / exception vector, not through a function name, and the only symbolic relationship is via __secure_start and __secure_end -- and those must (and will) be correctly relocated along with the U-Boot image.

Quoted 5 lines

Quoted 3 lines

or do they interact through an IPC independent of their
respective location (in which case, one wonders why they are built into a single ELF file)?

This comes a question, why PSCI framework intergated into U-Boot? I have

no idea.

Quoted 29 lines

There could have been alternative designs, indeed. Here is the design now, so let's handle it.

Quoted 11 lines

Quoted 3 lines

More generally... which target do you see this problem on? Reproducing the build myself would save both you and me some time, I guess. :)

Build target: mx7dsabresd

Needs to apply the three patches:
https://patchwork.ozlabs.org/patch/527040/
https://patchwork.ozlabs.org/patch/527038/
https://patchwork.ozlabs.org/patch/527039/

Quoted 4 lines

There is a small difference from my previous log, since my previous log use
0x180000 as the secure address, but the patches use 0x900000 as the secure address. But sure there will be relocation entry there.

Thanks. I'll try and play with compiler / linker options, but from my own experience, this can be tricky. Meanwhile, I suggest you for for the not-too-quick-and-not-too-dirty linker script solution.

Quoted 5 lines

Quoted 4 lines

Do you need these relocation records? If not, then just append the '*(.rel._secure*) input section spec (with a suitable comment) to the already existing DISCARD output section. By the way, doesn't this linker script change alone fix the data abort?

Quoted 14 lines

Quoted 3 lines

Yeah. Since the relocation entry for secure section will be stored in rel.secure section. And this section will not be touched in relocate.S

Ok, so that's our first clean, linker-script-based, solution confirmed.

Quoted 8 lines

Quoted 4 lines

Still, if there *are* relocation records emitted, that's because the toolchain considered them necessary -- IOW, it was (wrongly) told the PSCI code must be relocatable [or the code really is relocatable and we just haven't not hit a bug about this yet].

The PSCI code is bundled with the u-boot image. But it's running address is not same with u-boot. In my example, u-boot is compiled with starting address 0x87800000, but to PSCI, it's running address is

0x180000.

Quoted 8 lines

Quoted 3 lines

relocate_secure_section is just copy the psci code from uboot to 0x180000.

compliation address is same with running address, to PSCI part.

(I understand that. My question was not "why is PSCI built for its run-time address?" but "Since PSCI is built for its run-time address, and since there is little dependency on how come it is built as relocatable?")

uboot will not effect after switch to kernel, and its image will be overriden by kernel. But PSCI will always effect during kernel lifecyle.
Maybe built PSCI part non-relocatable is an alternative choice.

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Testing every single relocation record target against __image_copy_start and __image_copy_end would be rather costly in the tight loop of the relocation routine, which should run as fast as possible.

There is at least one fix to your problem which removes the out-of- bounds relocation records, and quite probably one better fix which prevents generating them in the first place.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Therefore, instead of the currently proposed patch which increases the size (very slightly) and boot time (statistically in O(n) proportion) of the image, I prefer one of the two alternative solutions which decrease the image size (by removing useless relocation records) and leave the boot time unchanged (since the fix is at

build time only).

Quoted 17 lines

Quoted 2 lines

Agree.

OK. So, for now, I suggest that you:

- resubmit v2 of this series with patch 1/3 reworked into a linker script change rather than a relocate routine change (and collect the secure relocation records input sections into the DISCARD output section rather than into a purposeless non-discarded section);

- make sure that you CC: the maintainers for the other boards which use PSCI and explicitly ask them to test the change on their boards and give their "Tested-by". It seems like the list of candidates for testing includes jetson-tk1, ls1021aqds, ls1021atwr and all sun[678]i boards, but I am by no means a PSCI specialist, so anyone feel free to correct me about this list.

Quoted 6 lines

Quoted 2 lines

Regards,
Peng.

Amicalement,
--
Albert.

--