FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-V
Hi All,
I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.
OpenSBI copies the FDT to following addresses for respective platforms
Qemu: 0x82200000
Unleashed: 0x88000000
As CONFIG_PRIOR_STAGE is set for both platforms, fdt is first copied to gd->fdt_blob and then gets relocated to a different address(gd-
Quoted 1 line
new_fdt) in functionreloc_fdt.
As a result, FDT is present at two locations.
1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2. gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.
However, commit ac12c6190927 (riscv: set CONFIG_SYS_BOOTM_LEN to SZ_64M) in U-boot changed Qemu config so that fdt_addr_r points to 0x88000000 which is wrong as nobody copies fdt to above address in Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.
As a result, fdtcontroladdr won't work on any platform and fdt_addr_r will work only for Unleashed but not for Qemu.
I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.
1. Change the FDT_JUMP_ADDR in OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.
2. Change fdt_addr_r to 0x82200000 in Qemu. Update documentation to use fdt only from fdt_addr_r not fdtcontroladdr.
@david: What was the reason behind changing it for Qemu ?
3. Fix gd->new_fdt computation. This may affect every board which is not a very good idea either.
4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.
Signature · 2 lines
Regards, Atish
Hi,
I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.
I suggest that for all platforms U-Boot explicitly load FDT from somewhere so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command
I also suspect that in-future for certain platforms FDT passed to U-Boot and FDT passed to Linux might be little different due to U-Boot specific changes in DTS.
Thoughts ??
Regards,
Anup
Quoted 64 lines
-----Original Message-----
From: Atish Patra
Sent: Sunday, September 8, 2019 5:40 PM
To: david.abdurachmanov at sifive.com; Alistair Francis <[email protected]>; Anup Patel <[email protected]>
Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-VHi All,
I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.
OpenSBI copies the FDT to following addresses for respective platforms
Qemu:
0x82200000
Unleashed:0x88000000As
CONFIG_PRIOR_STAGEis set for both platforms, fdt is first copied togd->fdt_bloband then gets relocated to a different address(gd-Quoted 1 line
new_fdt) in functionreloc_fdt.As a result, FDT is present at two locations.
1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2.gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.However, commit
ac12c6190927(riscv: setCONFIG_SYS_BOOTM_LENtoSZ_64M) in U-boot changed Qemu config so thatfdt_addr_rpoints to0x88000000which is wrong as nobody copies fdt to above address in Qemu. Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.As a result, fdtcontroladdr won't work on any platform and
fdt_addr_rwill work only for Unleashed but not for Qemu.I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.
1. Change the
FDT_JUMP_ADDRin OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.2. Change
fdt_addr_rto0x82200000in Qemu. Update documentation to use fdt only fromfdt_addr_rnot fdtcontroladdr.@david: What was the reason behind changing it for Qemu ?
3. Fix
gd->new_fdtcomputation. This may affect every board which is not a very good idea either.4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.
--
Regards,
Atish
Quoted 13 lines · Anup Patel
Hi,
I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.
I suggest that for all platforms U-Boot explicitly load FDT from somewhere so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy command
Quoted 5 lines
I also suspect that in-future for certain platforms FDT passed to U-Boot and FDT passed to Linux might be little different due to U-Boot specific changes in DTS.
Thoughts ??
Do not forget PXE and EXTLINUX boot options. These options must always be able to override DTB from previous stages. See below what PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus fdt_addr needs to be set if DTB is coming from somewhere in the firmware. This is why we had CONFIG_PREBOOT to set it.
[..]
* Scenario 1: If fdt_addr_r specified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Pass fdt_addr_r to bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is an fdt_addr specified, pass it along to
* bootm, and adjust argc appropriately.
*
* Scenario 3: fdt blob is not available.
[..]
david
Quoted 72 lines
Regards,
AnupQuoted 61 lines
-----Original Message-----
From: Atish Patra
Sent: Sunday, September 8, 2019 5:40 PM
To: david.abdurachmanov at sifive.com; Alistair Francis <[email protected]>; Anup Patel <[email protected]>
Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-VHi All,
I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.
OpenSBI copies the FDT to following addresses for respective platforms
Qemu: 0x82200000 Unleashed: 0x88000000As
CONFIG_PRIOR_STAGEis set for both platforms, fdt is first copied togd->fdt_bloband then gets relocated to a different address(gd-Quoted 1 line
new_fdt) in functionreloc_fdt.As a result, FDT is present at two locations.
1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2.gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.However, commit
ac12c6190927(riscv: setCONFIG_SYS_BOOTM_LENtoSZ_64M) in U-boot changed Qemu config so thatfdt_addr_rpoints to0x88000000which is wrong as nobody copies fdt to above address in Qemu. Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.As a result, fdtcontroladdr won't work on any platform and
fdt_addr_rwill work only for Unleashed but not for Qemu.I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.
1. Change the
FDT_JUMP_ADDRin OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.2. Change
fdt_addr_rto0x82200000in Qemu. Update documentation to use fdt only fromfdt_addr_rnot fdtcontroladdr.@david: What was the reason behind changing it for Qemu ?
3. Fix
gd->new_fdtcomputation. This may affect every board which is not a very good idea either.4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.
Quoted 3 lines
--
Regards,
Atish_______________________________________________
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
Quoted 33 lines · David Abdurachmanov
Quoted 17 lines · Anup Patel
Hi,
I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.
I suggest that for all platforms U-Boot explicitly load FDT from somewhere
so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly
copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy commandQuoted 7 lines
I also suspect that in-future for certain platforms FDT passed to U-Boot and
FDT passed to Linux might be little different due to U-Boot specific changes
in DTS.Thoughts ??
Do not forget PXE and EXTLINUX boot options. These options must always be able to override DTB from previous stages. See below what PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus
fdt_addrneeds to be set if DTB is coming from somewhere in the firmware.
This is why we hadCONFIG_PREBOOTto set it.
Quoted 8 lines
[..]
* Scenario 1: Iffdt_addr_rspecified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Passfdt_addr_rto bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is anfdt_addrspecified, pass it along to
* bootm, and adjust argc appropriately.
*
How about if we do this in PREBOOT ?
1. copy fdt from fdtcontroladdr to fdt_addr_r
2. setenv fdt_addr $fdt_addr_r
In this way, U-Boot will not have any direct dependancies on OpenSBI. As long as U-Boot is configured with a fdt_addr_r, it should work. It will be still valid for loading fdt from tftp server to fdt_addr_r as well.
Quoted 95 lines
* Scenario 3: fdt blob is not available.
[..]david
Quoted 90 lines
Regards,
AnupQuoted 80 lines
-----Original Message-----
From: Atish Patra
Sent: Sunday, September 8, 2019 5:40 PM
To: david.abdurachmanov at sifive.com; Alistair Francis <[email protected]>; Anup Patel <[email protected]>
Cc: u-boot at lists.denx.de; opensbi at lists.infradead.org
Subject: FDT placement in OpenSBI + U-Boot + Linux kernel issue for RISC-VHi All,
I noticed following issues around U-Boot fdt location in Unleashed and Qemu
virt machine.OpenSBI copies the FDT to following addresses for respective platforms
Qemu: 0x82200000 Unleashed: 0x88000000As
CONFIG_PRIOR_STAGEis set for both platforms, fdt is first copied togd->fdt_bloband then gets relocated to a different address(gd-Quoted 1 line
new_fdt) in functionreloc_fdt.As a result, FDT is present at two locations.
1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this
address.
2.gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.However, commit
ac12c6190927(riscv: setCONFIG_SYS_BOOTM_LENtoSZ_64M) in U-boot changed Qemu config so thatfdt_addr_rpoints to0x88000000which is wrong as nobody copies fdt to above address in Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.As a result, fdtcontroladdr won't work on any platform and
fdt_addr_rwill
work only for Unleashed but not for Qemu.I am not sure what should be the ideal solution to avoid these kind of fdt
placement issues in future. Here are the few possible ones.1. Change the
FDT_JUMP_ADDRin OpenSBI to 0x88000000(RAM+128MB) for
Qemu as well. This won't work as Qemu copies initrd to that address. I guess
best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for
Qemu accordingly.2. Change
fdt_addr_rto0x82200000in Qemu. Update documentation to use
fdt only fromfdt_addr_rnot fdtcontroladdr.@david: What was the reason behind changing it for Qemu ?
3. Fix
gd->new_fdtcomputation. This may affect every board which is not a
very good idea either.4. Mandate loading fdt only from tftp or sdcard which is the safest option and
will avoid these kind of complications. However, I think a default booting
method without tftp server should at least work. Let me know if that is not a
sane request. In that case, we should update documentation to clearly say
that only tftpboot or sdcard loading method works.Quoted 3 lines
--
Regards,
Atish_______________________________________________
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
Signature · 2 lines
Regards, Atish
Quoted 45 lines · Atish Patra
Quoted 33 lines · David Abdurachmanov
Quoted 17 lines · Anup Patel
Hi,
I think keeping FDT placement in-sync between U-Boot and OpenSBI across platforms is going to be painful.
I suggest that for all platforms U-Boot explicitly load FDT from somewhere
so:
1. U-Boot ${fdt_addr_r} default value will be recommended location of FDT
2. U-Boot ${fdtcontroladdr} will always point to copy of FDT passed by OpenSBI
3. To forward FDT passed by OpenSBI to Linux, U-Boot users can always explicitly
copy FDT from ${fdtcontroladdr} to ${fdt_addr_r} using U-Boot copy commandQuoted 7 lines
I also suspect that in-future for certain platforms FDT passed to U-Boot and
FDT passed to Linux might be little different due to U-Boot specific changes
in DTS.Thoughts ??
Do not forget PXE and EXTLINUX boot options. These options must always be able to override DTB from previous stages. See below what PXE/EXT use. For Fedora/RISCV we end up in scenario #2 and thus
fdt_addrneeds to be set if DTB is coming from somewhere in the firmware.
This is why we hadCONFIG_PREBOOTto set it.Quoted 8 lines
[..]
* Scenario 1: Iffdt_addr_rspecified and "fdt" label is defined in
* pxe file, retrieve fdt blob from server. Passfdt_addr_rto bootm,
* and adjust argc appropriately.
*
* Scenario 2: If there is anfdt_addrspecified, pass it along to
* bootm, and adjust argc appropriately.
*How about if we do this in PREBOOT ?
Quoted 7 lines
1. copy fdt from fdtcontroladdr to
fdt_addr_r
2. setenvfdt_addr$fdt_addr_rIn this way, U-Boot will not have any direct dependancies on OpenSBI. As long as U-Boot is configured with a
fdt_addr_r, it should work. It will be still valid for loading fdt from tftp server tofdt_addr_ras well.
Since, arg, it looks like we're already seeing some of the original mistakes from ARM land (fdt address in a bad place and getting stomped by ...), lets take a look at how this is being handled today there and mirror it? We should for example pick the fdt-in-memory location such that we have enough room between it and the maximum kernel + bss, round up and place it there. Then round up again for an overly large fdt to say that's where ramdisk should go. Thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190910/5bb22a12/attachment.sig>
Quoted 40 lines · Atish Patra
Hi All,
I noticed following issues around U-Boot fdt location in Unleashed and Qemu virt machine.
OpenSBI copies the FDT to following addresses for respective platforms
Qemu: 0x82200000 Unleashed: 0x88000000As
CONFIG_PRIOR_STAGEis set for both platforms, fdt is first copied togd->fdt_bloband then gets relocated to a different address(gd-Quoted 1 line
new_fdt) in functionreloc_fdt.As a result, FDT is present at two locations.
1. 0x88000000(Unleashed)/0x82200000(Qemu) : OpenSBI copied FDT to this address.
2.gd->new_fdt: U-Boot relocated the fdt to this address. fdtcontroladdr will also point to this address.However, commit
ac12c6190927(riscv: setCONFIG_SYS_BOOTM_LENtoSZ_64M) in U-boot changed Qemu config so thatfdt_addr_rpoints to0x88000000which is wrong as nobody copies fdt to above address in Qemu.
Also, 5.3-rc+ kernels overwrite the address pointed by fdtcontroladdr.As a result, fdtcontroladdr won't work on any platform and
fdt_addr_rwill work only for Unleashed but not for Qemu.I am not sure what should be the ideal solution to avoid these kind of fdt placement issues in future. Here are the few possible ones.
1. Change the
FDT_JUMP_ADDRin OpenSBI to 0x88000000(RAM+128MB) for Qemu as well. This won't work as Qemu copies initrd to that address. I guess best next option is to copy to 0x84000000(RAM+64MB) and U-boot config for Qemu accordingly.
Quoted 4 lines
2. Change
fdt_addr_rto0x82200000in Qemu. Update documentation to use fdt only fromfdt_addr_rnot fdtcontroladdr.@david: What was the reason behind changing it for Qemu ?
Not enough space for the kernel. Thus it was moved from 16M to 64M (which was a common thing for some ARM boards, e.g. 96Boards).
I that point I didn't know about QEMU limitation and initrd placement (IIUC only if you use -initrd).
Quoted 10 lines
3. Fix
gd->new_fdtcomputation. This may affect every board which is not a very good idea either.4. Mandate loading fdt only from tftp or sdcard which is the safest option and will avoid these kind of complications. However, I think a default booting method without tftp server should at least work. Let me know if that is not a sane request. In that case, we should update documentation to clearly say that only tftpboot or sdcard loading method works.
I don't think we should require this. FDT should be part of the firmware (FSBL, U-Boot SPI, OpenSBI, U-Boot, etc.). There should be a way to override the default one from the firmware (e.g. via tfptboot, PXE or EXTLINUX configs), but that's optional as majority people will not do that.
david
Quoted 9 lines
--
Regards,
Atish
_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
https://lists.denx.de/listinfo/u-boot