Skip to contentSkip to conversation

FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

U-Boot Atish Patra 4 participants 6 messages Sep 8, 2019 – Sep 10, 2019

lore
Share & tools
lore
#0 lore raw

Hi All,

I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.

OpenSBI copies the FDT to following addresses for respective platforms

Qemu: 0x82200000
Unleashed: 0x88000000

As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to gd->fdt_blob and then gets relocated to a different address(gd-

Quoted 1 line

new_fdt) in function reloc_fdt.

As a result, FDT is present at two locations.

1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2. gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.

However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to 0x88000000 which is wrong as nobody copies fdt to above address in Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.

As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will work only for Unleashed but not for Qemu.

I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.

1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.

2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use fdt only from fdt_addr_r not fdtcontroladdr.

@david: What was the reason behind changing it for Qemu ?

3. Fix gd->new_fdt computation. This may affect every board which is not a very good idea either.

4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.

Signature · 2 lines
Regards,
Atish
#1 lore raw

Hi,

I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.

I suggest that for all platforms U-Boot explicitly load FDT from somewhere so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command

I also suspect that in-future for certain platforms FDT passed to U-Boot and FDT passed to Linux might be little different due to U-Boot specific changes in DTS.

Thoughts ??

Regards,
Anup

Quoted 64 lines

-----Original Message-----
From: Atish Patra
Sent: Sunday, September 8, 2019 5:40 PM
To: david.abdurachmanov at sifive.com; Alistair Francis <[email protected]>; Anup Patel <[email protected]>
Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

Hi All,

I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.

OpenSBI copies the FDT to following addresses for respective platforms

Qemu: 0x82200000
Unleashed: 0x88000000

As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to gd->fdt_blob and then gets relocated to a different address(gd-

Quoted 1 line

new_fdt) in function reloc_fdt.

As a result, FDT is present at two locations.

1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2. gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.

However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to 0x88000000 which is wrong as nobody copies fdt to above address in Qemu. Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.

As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will work only for Unleashed but not for Qemu.

I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.

1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.

2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use fdt only from fdt_addr_r not fdtcontroladdr.

@david: What was the reason behind changing it for Qemu ?

3. Fix gd->new_fdt computation. This may affect every board which is not a very good idea either.

4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.

--
Regards,
Atish

#2 lore raw
Quoted 13 lines · Anup Patel

Hi,

I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.

I suggest that for all platforms U-Boot explicitly load FDT from somewhere so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command

Quoted 5 lines

I also suspect that in-future for certain platforms FDT passed to U-Boot and FDT passed to Linux might be little different due to U-Boot specific changes in DTS.

Thoughts ??

Do not forget PXE and EXTLINUX boot options. These options must always be able to override DTB from previous stages. See below what PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus fdt_addr needs to be set if DTB is coming from somewhere in the firmware. This is why we had CONFIG_PREBOOT to set it.

[..]
* Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is an fdt_addr specified, pass it along to
* bootm, and adjust argc appropriately.
*
* Scenario 3: fdt blob is not available.
[..]

david

Quoted 72 lines

Regards,
Anup

Quoted 61 lines

-----Original Message-----
From: Atish Patra
Sent: Sunday, September 8, 2019 5:40 PM
To: david.abdurachmanov at sifive.com; Alistair Francis <[email protected]>; Anup Patel <[email protected]>
Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

Hi All,

I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.

OpenSBI copies the FDT to following addresses for respective platforms

Qemu:                 0x82200000
Unleashed:    0x88000000

As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to gd->fdt_blob and then gets relocated to a different address(gd-

Quoted 1 line

new_fdt) in function reloc_fdt.

As a result, FDT is present at two locations.

1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2. gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.

However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to 0x88000000 which is wrong as nobody copies fdt to above address in Qemu. Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.

As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will work only for Unleashed but not for Qemu.

I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.

1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.

2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use fdt only from fdt_addr_r not fdtcontroladdr.

@david: What was the reason behind changing it for Qemu ?

3. Fix gd->new_fdt computation. This may affect every board which is not a very good idea either.

4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.

Quoted 3 lines

--
Regards,
Atish

_______________________________________________
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#3 lore raw
Quoted 33 lines · David Abdurachmanov
Quoted 17 lines · Anup Patel

Hi,

I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.

I suggest that for all platforms U-Boot explicitly load FDT from somewhere
so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly
copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command

Quoted 7 lines

I also suspect that in-future for certain platforms FDT passed to U-Boot and
FDT passed to Linux might be little different due to U-Boot specific changes
in DTS.

Thoughts ??

Do not forget PXE and EXTLINUX boot options. These options must always be able to override DTB from previous stages. See below what PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus fdt_addr needs to be set if DTB is coming from somewhere in the firmware.
This is why we had CONFIG_PREBOOT to set it.

Quoted 8 lines

[..]
* Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is an fdt_addr specified, pass it along to
* bootm, and adjust argc appropriately.
*

How about if we do this in PREBOOT ?

1. copy fdt from fdtcontroladdr to fdt_addr_r
2. setenv fdt_addr $fdt_addr_r

In this way, U-Boot will not have any direct dependancies on OpenSBI. As long as U-Boot is configured with a fdt_addr_r, it should work. It will be still valid for loading fdt from tftp server to fdt_addr_r as well.

Quoted 95 lines

* Scenario 3: fdt blob is not available.
[..]

david

Quoted 90 lines

Regards,
Anup

Quoted 80 lines

-----Original Message-----
From: Atish Patra
Sent: Sunday, September 8, 2019 5:40 PM
To: david.abdurachmanov at sifive.com; Alistair Francis <[email protected]>; Anup Patel <[email protected]>
Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V

Hi All,

I noticed following issues around U-Boot fdt location in Unleashed and Qemu
virt machine.

OpenSBI copies the FDT to following addresses for respective platforms

Qemu:                 0x82200000
Unleashed:    0x88000000

As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to
gd->fdt_blob and then gets relocated to a different address(gd-

Quoted 1 line

new_fdt) in function reloc_fdt.

As a result, FDT is present at two locations.

1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this
address.
2. gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.

However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to
0x88000000 which is wrong as nobody copies fdt to above address in Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.

As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will
work only for Unleashed but not for Qemu.

I am not sure what should be the ideal solution to avoid these kind of fdt
placement issues in future. Here are the few possible ones.

1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for
Qemu as well. This won't work as Qemu copies initrd to that address. I guess
best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for
Qemu accordingly.

2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use
fdt only from fdt_addr_r not fdtcontroladdr.

@david: What was the reason behind changing it for Qemu ?

3. Fix gd->new_fdt computation. This may affect every board which is not a
very good idea either.

4. Mandate loading fdt only from tftp or sdcard which is the safest option and
will avoid these kind of complications. However, I think a default booting
method without tftp server should at least work. Let me know if that is not a
sane request. In that case, we should update documentation to clearly say
that only tftpboot or sdcard loading method works.

Quoted 3 lines

--
Regards,
Atish

_______________________________________________
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Signature · 2 lines
Regards,
Atish
#4 lore raw
Quoted 45 lines · Atish Patra
Quoted 33 lines · David Abdurachmanov
Quoted 17 lines · Anup Patel

Hi,

I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.

I suggest that for all platforms U-Boot explicitly load FDT from somewhere
so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly
copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command

Quoted 7 lines

I also suspect that in-future for certain platforms FDT passed to U-Boot and
FDT passed to Linux might be little different due to U-Boot specific changes
in DTS.

Thoughts ??

Do not forget PXE and EXTLINUX boot options. These options must always be able to override DTB from previous stages. See below what PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus fdt_addr needs to be set if DTB is coming from somewhere in the firmware.
This is why we had CONFIG_PREBOOT to set it.

Quoted 8 lines

[..]
* Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is an fdt_addr specified, pass it along to
* bootm, and adjust argc appropriately.
*

How about if we do this in PREBOOT ?

Quoted 7 lines

1. copy fdt from fdtcontroladdr to fdt_addr_r
2. setenv fdt_addr $fdt_addr_r

In this way, U-Boot will not have any direct dependancies on OpenSBI. As long as U-Boot is configured with a fdt_addr_r, it should work. It will be still valid for loading fdt from tftp server to fdt_addr_r as well.

Since, arg, it looks like we're already seeing some of the original mistakes from ARM land (fdt address in a bad place and getting stomped by ...), lets take a look at how this is being handled today there and mirror it? We should for example pick the fdt-in-memory location such that we have enough room between it and the maximum kernel + bss, round up and place it there. Then round up again for an overly large fdt to say that's where ramdisk should go. Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190910/5bb22a12/attachment.sig>
#5 lore raw
In reply to Atish Patra
Quoted 40 lines · Atish Patra

Hi All,

I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.

OpenSBI copies the FDT to following addresses for respective platforms

Qemu:           0x82200000
Unleashed:      0x88000000

As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to gd->fdt_blob and then gets relocated to a different address(gd-

Quoted 1 line

new_fdt) in function reloc_fdt.

As a result, FDT is present at two locations.

1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2. gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.

However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to 0x88000000 which is wrong as nobody copies fdt to above address in Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.

As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will work only for Unleashed but not for Qemu.

I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.

1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.

Quoted 4 lines

2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use fdt only from fdt_addr_r not fdtcontroladdr.

@david: What was the reason behind changing it for Qemu ?

Not enough space for the kernel. Thus it was moved from 16M to 64M (which was a common thing for some ARM boards, e.g. 96Boards).

I that point I didn't know about QEMU limitation and initrd placement (IIUC only if you use -initrd).

Quoted 10 lines

3. Fix gd->new_fdt computation. This may affect every board which is not a very good idea either.

4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.

I don't think we should require this. FDT should be part of the firmware (FSBL, U-Boot SPI, OpenSBI, U-Boot, etc.). There should be a way to override the default one from the firmware (e.g. via tfptboot, PXE or EXTLINUX configs), but that's optional as majority people will not do that.

david

Quoted 9 lines

--
Regards,
Atish
_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
https://lists.denx.de/listinfo/u-boot