ARM: Start using fdt_high for relocation
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting fdt_high (in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131206/26377e53/attachment.pgp>
Hi Tom,
Quoted 9 lines · Tom Rini
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Amicalement,
Signature · 1 line
Albert.
Quoted 14 lines · Albert ARIBAUD
Hi Tom,
Quoted 9 lines · Tom Rini
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Quoted 3 lines
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Sure. Lets take am335x_evm builds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131206/8437870f/attachment.pgp>
Hi Tom,
Quoted 19 lines · Tom Rini
Quoted 14 lines · Albert ARIBAUD
Hi Tom,
Quoted 9 lines · Tom Rini
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?Quoted 3 lines
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Quoted 10 lines
Sure. Lets take
am335x_evmbuilds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
Amicalement,
Signature · 1 line
Albert.
Quoted 34 lines · Albert ARIBAUD
Hi Tom,
Quoted 19 lines · Tom Rini
Quoted 14 lines · Albert ARIBAUD
Hi Tom,
Quoted 9 lines · Tom Rini
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?Quoted 3 lines
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Quoted 10 lines
Sure. Lets take
am335x_evmbuilds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.
Quoted 7 lines
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
fdt_high is only an upper bound on what we may relocate to (setting aside the magic value of 0xffffffff which means no relocation). I just confirmed this too:
U-Boot# bdi
...boot_params = 0x80000100
DRAM bank = 0x00000000 -> start = 0x80000000 -> size = 0x40000000
...
U-Boot# setenv fdt_high 0xe0000000
...
U-Boot# bootz $loadaddr - $fdtaddr
Kernel image @ 0x80200000 [ 0x000000 - 0x3e5068 ] ## Flattened Device Tree blob at 80f80000
Booting using the fdt blob at 0x80f80000 Loading Device Tree to bf62a000, end bf630d9a ... OK
Of course, 0xbf62a000 is a bad address in that it won't be seen by the kernel, but it was in the higher-than-we-have-memory-at limit I set.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131206/947a51e0/attachment.pgp>
Quoted 61 lines · Tom Rini
Quoted 34 lines · Albert ARIBAUD
Hi Tom,
Quoted 19 lines · Tom Rini
Quoted 14 lines · Albert ARIBAUD
Hi Tom,
Quoted 9 lines · Tom Rini
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?Quoted 3 lines
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Quoted 10 lines
Sure. Lets take
am335x_evmbuilds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.Quoted 7 lines
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
fdt_highis only an upper bound on what we may relocate to (setting aside the magic value of0xffffffffwhich means no relocation). I just confirmed this too:
U-Boot# bdi
...boot_params=0x80000100DRAM bank = 0x00000000 -> start = 0x80000000 -> size = 0x40000000...
U-Boot# setenvfdt_high0xe0000000
...
U-Boot# bootz $loadaddr - $fdtaddr
Kernel image @0x80200000[0x000000-0x3e5068] ## Flattened Device Tree blob at80f80000
Booting using the fdt blob at0x80f80000Loading Device Tree tobf62a000, endbf630d9a... OK
Quoted 2 lines
Of course,
0xbf62a000is a bad address in that it won't be seen by the kernel, but it was in the higher-than-we-have-memory-at limit I set.
I haven't really followed this thread since I just noticed it tangentially, but on Tegra...
Since commit 7f1b767aea94 "ARM: tegra: define CONFIG_SYS_BOOTMAPSZ" all Tegra boards define BOOTMAPSZ to influence where the DTB gets relocated. I wonder what's the benefit of using BOOTMAPSZ vs. defining a default value for $fdt_high?
And couldn't you just move $fdt_addr_r high enough in RAM so even if the DTB wasn't relocated it'd never overlap BSS? Tegra's $kernel_addr_r and $fdt_addr_r are likely set up that way already, although I didn't check.
Quoted 65 lines · Stephen Warren
Quoted 61 lines · Tom Rini
Quoted 34 lines · Albert ARIBAUD
Hi Tom,
Quoted 19 lines · Tom Rini
Quoted 14 lines · Albert ARIBAUD
Hi Tom,
Quoted 9 lines · Tom Rini
Hey all,
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?Quoted 3 lines
Not sure I'm getting the issue clear, and I would like to avoid (me and others) having to switch back and forth between threads. Can you sketch the failure scenario in a couple of lines?
Quoted 10 lines
Sure. Lets take
am335x_evmbuilds (so Beaglebone Black/White, etc). If you start enabling all of the tracing options in the kernel (function tracing, graphs, etc), you get an uncompressed kernel and BSS that will use up the first ~16MiB of DDR. We default to placing the DT at about 15MiB into memory. So the kernel runs, clears BSS and eats the DT. System now hangs, and depending on debug options set you may or may not see anything at all from the kernel. U-Boot couldn't detect this failure because we don't know how big the kernel BSS is, only how big the zImage is (and where it is) and how big the fdt is and where it is. No overlaps, go ahead and run.Quoted 7 lines
Thanks.
The only issue I have with the RFC is that the +512 MiB value will only work with targets which have more than 512 MiB DDR, right? But since you're suggesting this should be set in configs, you are only suggesting +512 MiB, and any target could actually specify a lower value as long as it's greater than or eqal to 16 MiB. Correct?
fdt_highis only an upper bound on what we may relocate to (setting aside the magic value of0xffffffffwhich means no relocation). I just confirmed this too:
U-Boot# bdi
...boot_params=0x80000100DRAM bank = 0x00000000 -> start = 0x80000000 -> size = 0x40000000...
U-Boot# setenvfdt_high0xe0000000
...
U-Boot# bootz $loadaddr - $fdtaddr
Kernel image @0x80200000[0x000000-0x3e5068] ## Flattened Device Tree blob at80f80000
Booting using the fdt blob at0x80f80000Loading Device Tree tobf62a000, endbf630d9a... OKQuoted 2 lines
Of course,
0xbf62a000is a bad address in that it won't be seen by the kernel, but it was in the higher-than-we-have-memory-at limit I set.
Quoted 7 lines
I haven't really followed this thread since I just noticed it tangentially, but on Tegra...
Since commit
7f1b767aea94"ARM: tegra: defineCONFIG_SYS_BOOTMAPSZ" all Tegra boards define BOOTMAPSZ to influence where the DTB gets relocated. I wonder what's the benefit of using BOOTMAPSZ vs. defining a default value for $fdt_high?
I believe they both get the same thing done but SYS_BOOTMAPSZ also influences relocation of ramdisks (and in these cases we disable ramdisk relocation with initrd_high=0xffffffff).
Quoted 4 lines
And couldn't you just move $fdt_addr_r high enough in RAM so even if the DTB wasn't relocated it'd never overlap BSS? Tegra's $kernel_addr_r and $fdt_addr_r are likely set up that way already, although I didn't check.
There's two problems here. The first problem is that we have between 256MiB and 1GiB of DDR on the platform, but we could just design for the smallest case. The second problem is, what's big enough? You've got 32MiB (tegra30) which I would hope is enough (and I suggested as much in Dennis' thread) for kernel + BSS, but how big is a multi platform kernel with a few big features going to get? Or do we say that's an unreasonable out of box use case?
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131206/501509bc/attachment.pgp>
On 12/06/2013 02:04 PM, Tom Rini wrote:
...
Quoted 7 lines
There's two problems here. The first problem is that we have between 256MiB and 1GiB of DDR on the platform, but we could just design for the smallest case. The second problem is, what's big enough? You've got 32MiB (tegra30) which I would hope is enough (and I suggested as much in Dennis' thread) for kernel + BSS, but how big is a multi platform kernel with a few big features going to get? Or do we say that's an unreasonable out of box use case?
Is there any limit on .data/.bss size like there is .text (due to the limited range of relative jump encoding), or would data accesses just fall back to a relative load of the absolute address, and hence be unbounded.
FWIW, I see the following sizes currently:
tegra_defconfig
.text 005862c4
.data 0005eb68
.bss 00055bf0
multi_v7_defconfig
.text 004a5560
.data 00092600
.bss 00046014
(I think multi_v7_defconfig doesn't yet have that many drivers enabled, and when it does presumably they'd be modules)
... so BSS isn't terribly large at the moment.
Dear Tom,
Quoted 8 lines · In message <20131206154850.GW420@bill-the-cat> you wrote:
I've been thinking. We've had a thread on i.MX platforms about fdt being overwritten and needing to be moved to another address. And I've also had an internal problem about fdt being overwritten. So, how about as a rule of thumb we start setting
fdt_high(in configs) to memory-start + 512MiB, as that's the lowmem limit we should always have available. This will fix the problem of BSS overwriting the DT, which is the problem we won't catch in normal bootm/bootz usage. Thoughts?
Maybe I'm missing something - but what about systems with less than 1 GB RAM? Actually only a small percentage of devices we see here have 512 MB or more...
Best regards,
Wolfgang Denk
Signature · 5 lines
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de In the bathtub of history the truth is harder to hold than the soap, and much more difficult to find ... - Terry Pratchett, _Sourcery_