Skip to contentSkip to conversation

flattened device tree: Passing a dtb (blob) to Linux.

RFC PATCH In review Jimi Xenidis 7 participants 18 messages Apr 13, 2006 – May 23, 2006 PowerPC

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 flattened device tree: Passing a dtb (blob) to Linux.

A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a reserved_map entry for the blob itself. This means that if passed to Linux, Linux will not get lmb_reserve() the blob and it could be over.
The following patch will explicitly reserve the "blob" as it was given to us and stops prom_init.c from creating a reserved mapping for the blob.

NOTE: that the dtc must also not generate the blob reservation entry. Should we try to detect this redundant entry? Should we bump the DT version number?

Signed-off-by <[email protected]>

--

arch/powerpc/kernel/prom.c +0 -0
diff -r eb0990a251a9 arch/powerpc/kernel/prom.c
@@ -1132,6 +1132,11 @@ static void __init early_reserve_mem(voi
	reserve_map = (u64 *)(((unsigned long)initial_boot_params) +
					initial_boot_params->off_mem_rsvmap);

+
+ /* before we do anything, lets reserve the dt blob */
+ lmb_reserve(__pa((unsigned long)initial_boot_params),
+ initial_boot_params->totalsize);
+

#ifdef CONFIG_PPC32
	/*
	 * Handle the case where we might be booting from an old kexec
arch/powerpc/kernel/prom_init.c +0 -0
diff -r eb0990a251a9 arch/powerpc/kernel/prom_init.c
@@ -1909,11 +1909,7 @@ static void __init flatten_device_tree(v
	/* Version 16 is not backward compatible */
	hdr->last_comp_version = 0x10;

- /* Reserve the whole thing and copy the reserve map in, we
- * also bump mem_reserve_cnt to cause further reservations to
- * fail since it's too late.
- */
- reserve_mem(RELOC(dt_header_start), hdr->totalsize);
+ /* Copy the reserve map in */

	memcpy(rsvmap, RELOC(mem_reserve_map), sizeof(mem_reserve_map));
#ifdef DEBUG_PROM
@@ -1926,6 +1922,9 @@ static void __init flatten_device_tree(v
				    RELOC(mem_reserve_map)[i].size);
	}

#endif
+ /* Bump mem_reserve_cnt to cause further reservations to fail
+ * since it's too late.
+ */

	RELOC(mem_reserve_cnt) = MEM_RESERVE_MAP_SIZE;
	prom_printf("Device tree strings 0x%x -> 0x%x\n",
#1 lore raw
Quoted 3 lines · Jimi Xenidis

NOTE: that the dtc must also not generate the blob reservation entry.

This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.

Quoted 2 lines

Should we try to detect this redundant entry? Should we bump
the DT version number?

-JX

#2 lore raw

Hey Jimi,

Quoted 5 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.

The user space kexec tools create a reserve map entry for the blob as well.

Mikey

#3 lore raw
Quoted 10 lines · Michael Neuling

Hey Jimi,

Quoted 5 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.

The user space kexec tools create a reserve map entry for the blob as well.

good point, but I thought the kexec tools used dtc, no? looking passed my own world I see:
- bootx: creates a reservation entry
- iSeries: not reserving the blob at all
-JX

#4 lore raw
Quoted 7 lines · Jimi Xenidis
Quoted 5 lines · Michael Neuling

Hey Jimi,

Quoted 3 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.

The user space kexec tools create a reserve map entry for the blob as well.

good point, but I thought the kexec tools used dtc, no?

No, but they should.

Quoted 2 lines

looking passed my own world I see:
- iSeries: not reserving the blob at all

That sounds right. I think having the kernel do it is definitely the right option.

cheers

Signature · 8 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
#5 lore raw

Actually, is this even an issue? can the LMB handle repeated reservations? -JX

Quoted 35 lines · Michael Ellerman
Quoted 7 lines · Jimi Xenidis
Quoted 5 lines · Michael Neuling

Hey Jimi,

Quoted 3 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.

The user space kexec tools create a reserve map entry for the blob as well.

good point, but I thought the kexec tools used dtc, no?

No, but they should.

Quoted 2 lines

looking passed my own world I see:
- iSeries: not reserving the blob at all

That sounds right. I think having the kernel do it is definitely the right option.

cheers

Signature · 8 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
#6 lore raw
Quoted 2 lines · Jimi Xenidis

Actually, is this even an issue? can the LMB handle repeated reservations?

It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.

cheers

Quoted 39 lines · Michael Ellerman
Quoted 37 lines · Jimi Xenidis
Quoted 7 lines · Michael Neuling
Quoted 6 lines

Hey Jimi,

Quoted 3 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

This reservation is only generated for "asm" dtc targets so it is only that case that need to be changed.

The user space kexec tools create a reserve map entry for the blob as
well.

good point, but I thought the kexec tools used dtc, no?

No, but they should.

Quoted 2 lines

looking passed my own world I see:
- iSeries: not reserving the blob at all

That sounds right. I think having the kernel do it is definitely the right option.

cheers

Signature · 8 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
Signature · 8 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
#7 lore raw
Quoted 17 lines · Michael Ellerman
Quoted 2 lines · Jimi Xenidis

Actually, is this even an issue? can the LMB handle repeated reservations?

It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

looking passed my own world I see:
- iSeries: not reserving the blob at all

That sounds right. I think having the kernel do it is definitely the right option.

OK, I'm back to reading the list and beginning to catch up some here...

Let me see if I understand the consensus and direction:

1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,

2) Kernel should reserve the blob space early so as not to step on itself later,

3a) Kernel LMB handling should be modified to warn
for overlapping LMB reservations,

Except that Ben says:

3b) We should make lmb_reserve() of redudant/overlapping
entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map.

I think we should interpret "harmless" to be "warn" and not cause an error at this point in time.

I do not think we should have the blob generate its own reservation because it is possible that some post-processing (like U-Boot) can modify and extend it. Only after that can the blob's true size be determined. (Sure, it could update on the fly too... but double blah).

In all of this, I'm on deck for step 1) above.

jdl

#8 lore raw
Quoted 44 lines · Jon Loeliger
Quoted 17 lines · Michael Ellerman
Quoted 2 lines · Jimi Xenidis

Actually, is this even an issue? can the LMB handle repeated reservations?

It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

looking passed my own world I see:
- iSeries: not reserving the blob at all

That sounds right. I think having the kernel do it is definitely the right option.

OK, I'm back to reading the list and beginning to catch up some here...

Let me see if I understand the consensus and direction:

1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,

2) Kernel should reserve the blob space early so as not to step on itself later,

3a) Kernel LMB handling should be modified to warn
for overlapping LMB reservations,

Except that Ben says:

3b) We should make lmb_reserve() of redudant/overlapping
entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map.

I think we should interpret "harmless" to be "warn" and not cause an error at this point in time.

Quoted 7 lines

I do not think we should have the blob generate its own reservation because it is possible that some post-processing (like U-Boot) can modify and extend it. Only after that can the blob's true size be determined. (Sure, it could update on the fly too... but double blah).

In all of this, I'm on deck for step 1) above.

Nice summary :)
I'm up for 3a, we should make redundant/overlapping reserves "harmless", by which I mean "not an error", but there should definitely be a warning in the dmesg - as it will _usually_ indicate a bug.

cheers

Signature · 8 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
#9 lore raw

ok, to accept (1) means to accept (2).

AFAICT LMB is already handling overlaps.
So the first rough patch I sent should be sufficient.

Quoted 73 lines · Michael Ellerman
Quoted 44 lines · Jon Loeliger
Quoted 17 lines · Michael Ellerman
Quoted 2 lines · Jimi Xenidis

Actually, is this even an issue? can the LMB handle repeated reservations?

It can, but we were thinking about adding code to check and warn if reservations overlap, because it usually indicates a bug. Although that's probably ok in this case, as long as dtc gets fixed eventually. Another option would be to not warn for identical reservations.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

NOTE: that the dtc must also not generate the blob reservation entry.

looking passed my own world I see:
- iSeries: not reserving the blob at all

That sounds right. I think having the kernel do it is definitely the right option.

OK, I'm back to reading the list and beginning to catch up some here...

Let me see if I understand the consensus and direction:

1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,

2) Kernel should reserve the blob space early so as not to step on itself later,

3a) Kernel LMB handling should be modified to warn
for overlapping LMB reservations,

Except that Ben says:

3b) We should make lmb_reserve() of redudant/overlapping
entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map.

I think we should interpret "harmless" to be "warn" and not cause an error at this point in time.

Quoted 7 lines

I do not think we should have the blob generate its own reservation because it is possible that some post-processing (like U-Boot) can modify and extend it. Only after that can the blob's true size be determined. (Sure, it could update on the fly too... but double blah).

In all of this, I'm on deck for step 1) above.

Nice summary :)
I'm up for 3a, we should make redundant/overlapping reserves "harmless", by which I mean "not an error", but there should definitely be a warning in the dmesg - as it will _usually_ indicate a bug.

cheers

Signature · 12 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev
#10 lore raw
In reply to Michael Ellerman
Quoted 9 lines · Michael Ellerman
Quoted 9 lines

OK, I'm back to reading the list and beginning to catch up some here...

Let me see if I understand the consensus and direction:

1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,

In all of this, I'm on deck for step 1) above.

I have updated the DTC to have several bug fixes:

- asm_emit_cell() little endian bug fix
- use two .long and not .quad directives
- Eliminate emitting memreserve for blob in ASM output

These updates are available here:

www.jdl.com/git_repos
and
git://ozlabs.org/srv/projects/dtc/dtc.git

Please verify and let me know of any problems!

Quoted 1 line

Nice summary :)

Thanks.

Quoted 3 lines

I'm up for 3a, we should make redundant/overlapping reserves "harmless", by which I mean "not an error", but there should definitely be a warning in the dmesg - as it will _usually_ indicate a bug.

Please feel free to patch these into existence as needed! :-)

Thanks,
jdl

#11 lore raw
Quoted 18 lines · Jon Loeliger
Quoted 9 lines · Michael Ellerman
Quoted 9 lines

OK, I'm back to reading the list and beginning to catch up some here...

Let me see if I understand the consensus and direction:

1) DTC should NOT reserve its own blob space in the memory map, as it does for generated ASM code now,

In all of this, I'm on deck for step 1) above.

I have updated the DTC to have several bug fixes:

- asm_emit_cell() little endian bug fix
- use two .long and not .quad directives
- Eliminate emitting memreserve for blob in ASM output

Quoted 7 lines

These updates are available here:

www.jdl.com/git_repos
and
git://ozlabs.org/srv/projects/dtc/dtc.git

Please verify and let me know of any problems!

Hey John,

I did a quick test and the new dtc seems to work fine for my dts file.

Mark

#12 lore raw
In reply to Jimi Xenidis
Quoted 11 lines · Jimi Xenidis

A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a reserved_map entry for the blob itself. This means that if passed to Linux, Linux will not get lmb_reserve() the blob and it could be over.
The following patch will explicitly reserve the "blob" as it was given to us and stops prom_init.c from creating a reserved mapping for the blob.

NOTE: that the dtc must also not generate the blob reservation entry. Should we try to detect this redundant entry? Should we bump the DT version number?

We should make lmb_reserve() of redudant/overlapping entries become harmless I think. We need to be backward compatible with earlier blobs that do contain themselves in the reserve map

Ben.

#13 lore raw
Quoted 3 lines · Benjamin Herrenschmidt

We should make lmb_reserve() of redudant/overlapping entries become harmless I think.

Hmm.. I think it would be worthy of a warning, no?

Quoted 2 lines

We need to be backward compatible with earlier blobs that do contain themselves in the reserve map

Do you think it is possible that the blob may have a single reservation that includes the blob but is larger? if not then we could simply do.. --

arch/powerpc/kernel/prom.c +0 -0
diff -r eb0990a251a9 arch/powerpc/kernel/prom.c
@@ -1129,9 +1129,17 @@ static void __init early_reserve_mem(voi
{
	u64 base, size;
	u64 *reserve_map;
+	unsigned long self_base;
+	unsigned long self_size;
	reserve_map = (u64 *)(((unsigned long)initial_boot_params) +
					initial_boot_params->off_mem_rsvmap);

+
+ /* before we do anything, lets reserve the dt blob */
+ self_base = __pa((unsigned long)initial_boot_params);
+ self_size = initial_boot_params->totalsize;
+ lmb_reserve(self_base, self_size);
+

#ifdef CONFIG_PPC32
	/*
	 * Handle the case where we might be booting from an old kexec
@@ -1146,6 +1154,9 @@ static void __init early_reserve_mem(voi
			size_32 = *(reserve_map_32++);
			if (size_32 == 0)
				break;
+			/* skip if the reservation is for the blob */
+			if (base_32 == self_base && size_32 == self_size)
+				continue;
			DBG("reserving: %x -> %x\n", base_32, size_32);
			lmb_reserve(base_32, size_32);
		}
@@ -1157,6 +1168,9 @@ static void __init early_reserve_mem(voi
		size = *(reserve_map++);
		if (size == 0)
			break;
+		/* skip if the reservation is for the blob */
+		if (base == self_base && size == self_size)
+			continue;
		DBG("reserving: %llx -> %llx\n", base, size);
		lmb_reserve(base, size);
	}
#14 lore raw
Quoted 6 lines · Jimi Xenidis
Quoted 3 lines · Benjamin Herrenschmidt

We should make lmb_reserve() of redudant/overlapping entries become harmless I think.

Hmm.. I think it would be worthy of a warning, no?

Definitely. We had weird bugs in kexec because regions were overlapping. I just haven't got 'round to writing the patch .. maybe I should :)

Quoted 6 lines
Quoted 2 lines

We need to be backward compatible with earlier blobs that do contain themselves in the reserve map

Do you think it is possible that the blob may have a single reservation that includes the blob but is larger? if not then we could simply do..

Well it's definitely possible, but I don't know if anyone actually does it. But I guess we should try to be nice and not warn for old blobs.

This patch looks like a decent approach, although I'd want to add a check so that if the blob is reserved _three_ times we do warn, it's getting a bit silly isn't it :)

cheers

Quoted 45 lines

--
diff -r eb0990a251a9 arch/powerpc/kernel/prom.c

arch/powerpc/kernel/prom.c +14 -0
@@ -1129,9 +1129,17 @@ static void __init early_reserve_mem(voi
 {
 	u64 base, size;
 	u64 *reserve_map;
+	unsigned long self_base;
+	unsigned long self_size;
 	reserve_map = (u64 *)(((unsigned long)initial_boot_params) +
 					initial_boot_params->off_mem_rsvmap);
+
+	/* before we do anything, lets reserve the dt blob */
+	self_base = __pa((unsigned long)initial_boot_params);
+	self_size = initial_boot_params->totalsize;
+	lmb_reserve(self_base, self_size);
+
 #ifdef CONFIG_PPC32
 	/*
 	 * Handle the case where we might be booting from an old kexec
@@ -1146,6 +1154,9 @@ static void __init early_reserve_mem(voi
 			size_32 = *(reserve_map_32++);
 			if (size_32 == 0)
 				break;
+			/* skip if the reservation is for the blob */
+			if (base_32 == self_base && size_32 == self_size)
+				continue;
 			DBG("reserving: %x -> %x\n", base_32, size_32);
 			lmb_reserve(base_32, size_32);
 		}
@@ -1157,6 +1168,9 @@ static void __init early_reserve_mem(voi
 		size = *(reserve_map++);
 		if (size == 0)
 			break;
+		/* skip if the reservation is for the blob */
+		if (base == self_base && size == self_size)
+			continue;
 		DBG("reserving: %llx -> %llx\n", base, size);
 		lmb_reserve(base, size);
 	}

_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev

Signature · 8 lines
Michael Ellerman
IBM OzLabs

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person
#15 lore raw

PATCH 1/1 powerpc: Auto reserve of device tree blob

In reply to Jimi Xenidis

From: Jimi Xenidis <[email protected]>

A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a reserved_map entry for the blob itself. This means that if passed to Linux, Linux will not get lmb_reserve() the blob and it could be over. The following patch will explicitly reserve the "blob" as it was given to us and stops prom_init.c from creating a reserved mapping for the blob.

NOTE: that the dtc/kexec should not generate the blob reservation entry. Although if they do, LMB reserver handles overlaps.

---
After some discussion last time, it seemed this was the right was to do this. I'll post the kexec cleanups if/when this gets upstream.

Mikey

 arch/powerpc/kernel/prom.c      |    5 +++++
 arch/powerpc/kernel/prom_init.c |    9 ++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)
arch/powerpc/kernel/prom.c +5 -0
Index: linux-2.6-powerpc/arch/powerpc/kernel/prom.c
===================================================================
@@ -1240,6 +1240,11 @@ static void __init early_reserve_mem(voi
 
 	reserve_map = (u64 *)(((unsigned long)initial_boot_params) +
 					initial_boot_params->off_mem_rsvmap);
+
+	/* before we do anything, lets reserve the dt blob */
+	lmb_reserve(__pa((unsigned long)initial_boot_params),
+		    initial_boot_params->totalsize);
+
 #ifdef CONFIG_PPC32
 	/* 
 	 * Handle the case where we might be booting from an old kexec
arch/powerpc/kernel/prom_init.c +4 -5
Index: linux-2.6-powerpc/arch/powerpc/kernel/prom_init.c
===================================================================
@@ -2031,11 +2031,7 @@ static void __init flatten_device_tree(v
 	/* Version 16 is not backward compatible */
 	hdr->last_comp_version = 0x10;
 
-	/* Reserve the whole thing and copy the reserve map in, we
-	 * also bump mem_reserve_cnt to cause further reservations to
-	 * fail since it's too late.
-	 */
-	reserve_mem(RELOC(dt_header_start), hdr->totalsize);
+	/* Copy the reserve map in */
 	memcpy(rsvmap, RELOC(mem_reserve_map), sizeof(mem_reserve_map));
 
 #ifdef DEBUG_PROM
@@ -2048,6 +2044,9 @@ static void __init flatten_device_tree(v
 				    RELOC(mem_reserve_map)[i].size);
 	}
 #endif
+	/* Bump mem_reserve_cnt to cause further reservations to fail
+	 * since it's too late.
+	 */
 	RELOC(mem_reserve_cnt) = MEM_RESERVE_MAP_SIZE;
 
 	prom_printf("Device tree strings 0x%x -> 0x%x\n",
#16 lore raw

Re: [PATCH] powerpc: Auto reserve of device tree blob

Quoted 11 lines · Michael Neuling

From: Jimi Xenidis <[email protected]>

A devtree compiler (dtc) generated devtree blob is "relocatable" and so does not contain a reserved_map entry for the blob itself. This means that if passed to Linux, Linux will not get lmb_reserve() the blob and it could be over. The following patch will explicitly reserve the "blob" as it was given to us and stops prom_init.c from creating a reserved mapping for the blob.

NOTE: that the dtc/kexec should not generate the blob reservation entry. Although if they do, LMB reserver handles overlaps.

The current DTC does not issue the memory reservation request. As discussed earlier, it is now relying on the kernel to do a reservation for the itself. While I've not seen it applied yet, I am assuming that Jimi's patch from 14 April is what is needed here and will be applied. (Maybe it just needs a "Signed-off" to make it happen. Dunno.)

Thanks,
jdl

#17 lore raw

Re: [PATCH] powerpc: Auto reserve of device tree blob

Quoted 3 lines · Jon Loeliger

While I've not seen
it applied yet, I am assuming that Jimi's patch from 14 April is what is needed here and will be applied.

And that patch appears to be in Paul's tree now!

jdl