flattened device tree: Passing a dtb (blob) to Linux.
RFC PATCH 1/1 flattened device tree: Passing a dtb (blob) to Linux.
A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a reserved_map entry for the blob itself. This means that if passed to Linux, Linux will not get lmb_reserve() the blob and it could be over.
The following patch will explicitly reserve the "blob" as it was given to us and stops prom_init.c from creating a reserved mapping for the blob.
NOTE: that the dtc must also not generate the blob reservation entry. Should we try to detect this redundant entry? Should we bump the DT version number?
--
arch/powerpc/kernel/prom.c +0 -0
@@ -1132,6 +1132,11 @@ static void __init early_reserve_mem(voi
reserve_map = (u64 *)(((unsigned long)initial_boot_params) + initial_boot_params->off_mem_rsvmap);
+
+ /* before we do anything, lets reserve the dt blob */
+ lmb_reserve(__pa((unsigned long)initial_boot_params),
+ initial_boot_params->totalsize);
+
#ifdef CONFIG_PPC32 /* * Handle the case where we might be booting from an old kexec
arch/powerpc/kernel/prom_init.c +0 -0
@@ -1909,11 +1909,7 @@ static void __init flatten_device_tree(v
/* Version 16 is not backward compatible */ hdr->last_comp_version = 0x10;
- /* Reserve the whole thing and copy the reserve map in, we
- * also bump mem_reserve_cnt to cause further reservations to
- * fail since it's too late.
- */
- reserve_mem(RELOC(dt_header_start), hdr->totalsize);
+ /* Copy the reserve map in */
memcpy(rsvmap, RELOC(mem_reserve_map), sizeof(mem_reserve_map)); #ifdef DEBUG_PROM @@ -1926,6 +1922,9 @@ static void __init flatten_device_tree(v RELOC(mem_reserve_map)[i].size); }
#endif
+ /* Bump mem_reserve_cnt to cause further reservations to fail
+ * since it's too late.
+ */
RELOC(mem_reserve_cnt) = MEM_RESERVE_MAP_SIZE;
prom_printf("Device tree strings 0x%x -> 0x%x\n",Quoted 3 lines · Jimi Xenidis
NOTE: that the dtc must also not generate the blob reservation entry.
This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.
Quoted 2 lines
Should we try to detect this redundant entry? Should we bump
the DT version number?
-JX
Hey Jimi,
Quoted 5 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.
The user space kexec tools create a reserve map entry for the blob as well.
Mikey
Quoted 10 lines · Michael Neuling
Hey Jimi,
Quoted 5 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.
The user space kexec tools create a reserve map entry for the blob as well.
good point, but I thought the kexec tools used dtc, no? looking passed my own world I see:
- bootx: creates a reservation entry
- iSeries: not reserving the blob at all
-JX
Quoted 7 lines · Jimi Xenidis
Quoted 5 lines · Michael Neuling
Hey Jimi,
Quoted 3 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.
The user space kexec tools create a reserve map entry for the blob as well.
good point, but I thought the kexec tools used dtc, no?
No, but they should.
Quoted 2 lines
looking passed my own world I see:
- iSeries: not reserving the blob at all
That sounds right. I think having the kernel do it is definitely the right option.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Actually, is this even an issue? can the LMB handle repeated reservations? -JX
Quoted 35 lines · Michael Ellerman
Quoted 7 lines · Jimi Xenidis
Quoted 5 lines · Michael Neuling
Hey Jimi,
Quoted 3 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.
The user space kexec tools create a reserve map entry for the blob as well.
good point, but I thought the kexec tools used dtc, no?
No, but they should.
Quoted 2 lines
looking passed my own world I see:
- iSeries: not reserving the blob at allThat sounds right. I think having the kernel do it is definitely the right option.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Quoted 2 lines · Jimi Xenidis
Actually, is this even an issue? can the LMB handle repeated reservations?
It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.
cheers
Quoted 39 lines · Michael Ellerman
Quoted 37 lines · Jimi Xenidis
Quoted 7 lines · Michael Neuling
Quoted 6 lines
Hey Jimi,
Quoted 3 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.
The user space kexec tools create a reserve map entry for the blob as
well.good point, but I thought the kexec tools used dtc, no?
No, but they should.
Quoted 2 lines
looking passed my own world I see:
- iSeries: not reserving the blob at allThat sounds right. I think having the kernel do it is definitely the right option.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
Quoted 17 lines · Michael Ellerman
Quoted 2 lines · Jimi Xenidis
Actually, is this even an issue? can the LMB handle repeated reservations?
It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
looking passed my own world I see:
- iSeries: not reserving the blob at allThat sounds right. I think having the kernel do it is definitely the right option.
OK, I'm back to reading the list and beginning to catch up some here...
Let me see if I understand the consensus and direction:
1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,
2) Kernel should reserve the blob space early so as not to step on itself later,
3a) Kernel LMB handling should be modified to warn
for overlapping LMB reservations,
Except that Ben says:
3b) We should make lmb_reserve() of redudant/overlapping
entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map.
I think we should interpret "harmless" to be "warn" and not cause an error at this point in time.
I do not think we should have the blob generate its own reservation because it is possible that some post-processing (like U-Boot) can modify and extend it. Only after that can the blob's true size be determined. (Sure, it could update on the fly too... but double blah).
In all of this, I'm on deck for step 1) above.
jdl
Quoted 44 lines · Jon Loeliger
Quoted 17 lines · Michael Ellerman
Quoted 2 lines · Jimi Xenidis
Actually, is this even an issue? can the LMB handle repeated reservations?
It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
looking passed my own world I see:
- iSeries: not reserving the blob at allThat sounds right. I think having the kernel do it is definitely the right option.
OK, I'm back to reading the list and beginning to catch up some here...
Let me see if I understand the consensus and direction:
1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,
2) Kernel should reserve the blob space early so as not to step on itself later,
3a) Kernel LMB handling should be modified to warn
for overlapping LMB reservations,Except that Ben says:
3b) We should make
lmb_reserve()of redudant/overlapping
entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map.I think we should interpret "harmless" to be "warn" and not cause an error at this point in time.
Quoted 7 lines
I do not think we should have the blob generate its own reservation because it is possible that some post-processing (like U-Boot) can modify and extend it. Only after that can the blob's true size be determined. (Sure, it could update on the fly too... but double blah).
In all of this, I'm on deck for step 1) above.
Nice summary :)
I'm up for 3a, we should make redundant/overlapping reserves "harmless", by which I mean "not an error", but there should definitely be a warning in the dmesg - as it will _usually_ indicate a bug.
cheers
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
ok, to accept (1) means to accept (2).
AFAICT LMB is already handling overlaps.
So the first rough patch I sent should be sufficient.
Quoted 73 lines · Michael Ellerman
Quoted 44 lines · Jon Loeliger
Quoted 17 lines · Michael Ellerman
Quoted 2 lines · Jimi Xenidis
Actually, is this even an issue? can the LMB handle repeated reservations?
It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
NOTE: that the dtc must also not generate the blob reservation entry.
looking passed my own world I see:
- iSeries: not reserving the blob at allThat sounds right. I think having the kernel do it is definitely the right option.
OK, I'm back to reading the list and beginning to catch up some here...
Let me see if I understand the consensus and direction:
1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,
2) Kernel should reserve the blob space early so as not to step on itself later,
3a) Kernel LMB handling should be modified to warn
for overlapping LMB reservations,Except that Ben says:
3b) We should make
lmb_reserve()of redudant/overlapping
entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map.I think we should interpret "harmless" to be "warn" and not cause an error at this point in time.
Quoted 7 lines
I do not think we should have the blob generate its own reservation because it is possible that some post-processing (like U-Boot) can modify and extend it. Only after that can the blob's true size be determined. (Sure, it could update on the fly too... but double blah).
In all of this, I'm on deck for step 1) above.
Nice summary :)
I'm up for 3a, we should make redundant/overlapping reserves "harmless", by which I mean "not an error", but there should definitely be a warning in the dmesg - as it will _usually_ indicate a bug.cheers
Signature · 12 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person _______________________________________________ Linuxppc-dev mailing list [email protected] https://ozlabs.org/mailman/listinfo/linuxppc-dev
Quoted 9 lines · Michael Ellerman
Quoted 9 lines
OK, I'm back to reading the list and beginning to catch up some here...
Let me see if I understand the consensus and direction:
1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,
In all of this, I'm on deck for step 1) above.
I have updated the DTC to have several bug fixes:
- asm_emit_cell() little endian bug fix
- use two .long and not .quad directives
- Eliminate emitting memreserve for blob in ASM output
These updates are available here:
www.jdl.com/git_repos
and
git://ozlabs.org/srv/projects/dtc/dtc.git
Please verify and let me know of any problems!
Quoted 1 line
Nice summary :)
Thanks.
Quoted 3 lines
I'm up for 3a, we should make redundant/overlapping reserves "harmless", by which I mean "not an error", but there should definitely be a warning in the dmesg - as it will _usually_ indicate a bug.
Please feel free to patch these into existence as needed! :-)
Thanks,
jdl
Quoted 18 lines · Jon Loeliger
Quoted 9 lines · Michael Ellerman
Quoted 9 lines
OK, I'm back to reading the list and beginning to catch up some here...
Let me see if I understand the consensus and direction:
1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,
In all of this, I'm on deck for step 1) above.
I have updated the DTC to have several bug fixes:
-
asm_emit_cell()little endian bug fix
- use two .long and not .quad directives
- Eliminate emitting memreserve for blob in ASM output
Quoted 7 lines
These updates are available here:
www.jdl.com/git_repos
and
git://ozlabs.org/srv/projects/dtc/dtc.gitPlease verify and let me know of any problems!
Hey John,
I did a quick test and the new dtc seems to work fine for my dts file.
Mark
Quoted 11 lines · Jimi Xenidis
A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a
reserved_mapentry for the blob itself. This means that if passed to Linux, Linux will not getlmb_reserve()the blob and it could be over.
The following patch will explicitly reserve the "blob" as it was given to us and stopsprom_init.cfrom creating a reserved mapping for the blob.NOTE: that the dtc must also not generate the blob reservation entry. Should we try to detect this redundant entry? Should we bump the DT version number?
We should make lmb_reserve() of redudant/overlapping entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map
Ben.
Quoted 3 lines · Benjamin Herrenschmidt
We should make
lmb_reserve()of redudant/overlapping entries become harmless I think.
Hmm.. I think it would be worthy of a warning, no?
Quoted 2 lines
We need to be backward compatible with earlier blobs that do contain themselves in the reserve map
Do you think it is possible that the blob may have a single reservation that includes the blob but is larger? if not then we could simply do.. --
arch/powerpc/kernel/prom.c +0 -0
@@ -1129,9 +1129,17 @@ static void __init early_reserve_mem(voi
{
u64 base, size;
u64 *reserve_map;
+ unsigned long self_base;
+ unsigned long self_size;
reserve_map = (u64 *)(((unsigned long)initial_boot_params) +
initial_boot_params->off_mem_rsvmap);+
+ /* before we do anything, lets reserve the dt blob */
+ self_base = __pa((unsigned long)initial_boot_params);
+ self_size = initial_boot_params->totalsize;
+ lmb_reserve(self_base, self_size);
+
#ifdef CONFIG_PPC32
/*
* Handle the case where we might be booting from an old kexec
@@ -1146,6 +1154,9 @@ static void __init early_reserve_mem(voi
size_32 = *(reserve_map_32++);
if (size_32 == 0)
break;
+ /* skip if the reservation is for the blob */
+ if (base_32 == self_base && size_32 == self_size)
+ continue;
DBG("reserving: %x -> %x\n", base_32, size_32);
lmb_reserve(base_32, size_32);
}
@@ -1157,6 +1168,9 @@ static void __init early_reserve_mem(voi
size = *(reserve_map++);
if (size == 0)
break;
+ /* skip if the reservation is for the blob */
+ if (base == self_base && size == self_size)
+ continue;
DBG("reserving: %llx -> %llx\n", base, size);
lmb_reserve(base, size);
}Quoted 6 lines · Jimi Xenidis
Quoted 3 lines · Benjamin Herrenschmidt
We should make
lmb_reserve()of redudant/overlapping entries become harmless I think.Hmm.. I think it would be worthy of a warning, no?
Definitely. We had weird bugs in kexec because regions were overlapping. I just haven't got 'round to writing the patch .. maybe I should :)
Quoted 6 lines
Quoted 2 lines
We need to be backward compatible with earlier blobs that do contain themselves in the reserve map
Do you think it is possible that the blob may have a single reservation that includes the blob but is larger? if not then we could simply do..
Well it's definitely possible, but I don't know if anyone actually does it. But I guess we should try to be nice and not warn for old blobs.
This patch looks like a decent approach, although I'd want to add a check so that if the blob is reserved _three_ times we do warn, it's getting a bit silly isn't it :)
cheers
Quoted 45 lines
--
diff -reb0990a251a9arch/powerpc/kernel/prom.carch/powerpc/kernel/prom.c +14
-0@@ -1129,9 +1129,17 @@ static void __init early_reserve_mem(voi {u64 base, size;u64 *reserve_map; +unsigned long self_base; +unsigned long self_size; reserve_map = (u64 *)(((unsigned long )initial_boot_params) + initial_boot_params->off_mem_rsvmap); + +/* before we do anything, lets reserve the dt blob */ + self_base =__pa ((unsigned long )initial_boot_params); + self_size = initial_boot_params->totalsize; +lmb_reserve (self_base, self_size); +#ifdef CONFIG_PPC32 /* * Handle the case where we might be booting from an old kexec @@ -1146,6 +1154,9 @@ static void __init early_reserve_mem(voi size_32 = *(reserve_map_32++);if (size_32 ==0 )break ; +/* skip if the reservation is for the blob */ +if (base_32 == self_base && size_32 == self_size) +continue ;DBG ("reserving: %x -> %x\n" , base_32, size_32);lmb_reserve (base_32, size_32); }@@ -1157,6 +1168,9 @@ static void __init early_reserve_mem(voi size = *(reserve_map++);if (size ==0 )break ; +/* skip if the reservation is for the blob */ +if (base == self_base && size == self_size) +continue ;DBG ("reserving: %llx -> %llx\n" , base, size);lmb_reserve (base, size); }_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev
Signature · 8 lines
Michael Ellerman IBM OzLabs wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person
PATCH 1/1 powerpc: Auto reserve of device tree blob
In reply to Jimi XenidisFrom: Jimi Xenidis <[email protected]>
A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a reserved_map entry for the blob itself. This means that if passed to Linux, Linux will not get lmb_reserve() the blob and it could be over. The following patch will explicitly reserve the "blob" as it was given to us and stops prom_init.c from creating a reserved mapping for the blob.
NOTE: that the dtc/kexec should not generate the blob reservation entry. Although if they do, LMB reserver handles overlaps.
---
After some discussion last time, it seemed this was the right was to do this. I'll post the kexec cleanups if/when this gets upstream.
Mikey
arch/powerpc/kernel/prom.c | 5 +++++ arch/powerpc/kernel/prom_init.c | 9 ++++----- 2 files changed, 9 insertions(+), 5 deletions(-)
arch/powerpc/kernel/prom.c +5 -0
@@ -1240,6 +1240,11 @@ static void __init early_reserve_mem(voi reserve_map = (u64 *)(((unsigned long )initial_boot_params) + initial_boot_params->off_mem_rsvmap); + +/* before we do anything, lets reserve the dt blob */ +lmb_reserve (__pa ((unsigned long )initial_boot_params), + initial_boot_params->totalsize); +#ifdef CONFIG_PPC32 /* * Handle the case where we might be booting from an old kexec
arch/powerpc/kernel/prom_init.c +4 -5
@@ -2031,11 +2031,7 @@ static void __init flatten_device_tree(v/* Version 16 is not backward compatible */ hdr->last_comp_version =0x10 ; -/* Reserve the whole thing and copy the reserve map in, we -* also bump mem_reserve_cnt to cause further reservations to -* fail since it's too late. -*/ -reserve_mem (RELOC (dt_header_start), hdr->totalsize); +/* Copy the reserve map in */ memcpy (rsvmap,RELOC (mem_reserve_map),sizeof (mem_reserve_map));#ifdef DEBUG_PROM
@@ -2048,6 +2044,9 @@ static void __init flatten_device_tree(v RELOC(mem_reserve_map)[i].size); }#endif +/* Bump mem_reserve_cnt to cause further reservations to fail +* since it's too late. +*/ RELOC (mem_reserve_cnt) =MEM_RESERVE_MAP_SIZE ; prom_printf("Device tree strings 0x%x -> 0x%x\n",
Re: [PATCH] powerpc: Auto reserve of device tree blob
Quoted 11 lines · Michael Neuling
From: Jimi Xenidis <[email protected]>
A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a
reserved_mapentry for the blob itself. This means that if passed to Linux, Linux will not getlmb_reserve()the blob and it could be over. The following patch will explicitly reserve the "blob" as it was given to us and stopsprom_init.cfrom creating a reserved mapping for the blob.NOTE: that the dtc/kexec should not generate the blob reservation entry. Although if they do, LMB reserver handles overlaps.
The current DTC does not issue the memory reservation request. As discussed earlier, it is now relying on the kernel to do a reservation for the itself. While I've not seen it applied yet, I am assuming that Jimi's patch from 14 April is what is needed here and will be applied. (Maybe it just needs a "Signed-off" to make it happen. Dunno.)
Thanks,
jdl
Re: [PATCH] powerpc: Auto reserve of device tree blob
Quoted 3 lines · Jon Loeliger
While I've not seen
it applied yet, I am assuming that Jimi's patch from 14 April is what is needed here and will be applied.
And that patch appears to be in Paul's tree now!
jdl