Skip to contentSkip to conversation

alignment bugs in prom_init

PATCH New Olaf Hering 5 participants 16 messages Mar 3, 2006 – Mar 4, 2006 PowerPC

lore
Share & tools
lore
#0 lore raw

Some G5 and pSeries models dont boot with recent kernels. The reason is likely the casting of pointers of stack variables to u32. One example is the prom_getprop() call in prom_init_stdout().

sp is 0x0023e784, val is at offset 120, which makes 0x0023e7fc. This address is casted to u32, which changes it to 0x0023e7f8. The firmware writes to the wrong addres and things go downhill very quick.

c00000000040baa8: 3b 21 00 78 addi r25,r1,120
..
c00000000040baf4: 57 28 00 38 rlwinm r8,r25,0,0,28
..
c00000000040bb10: 4b ff d3 3d bl c000000000408e4c <.call_prom>

If I remove the casts and pass the pointer as is, everything starts to work as expected? Why is all this (u32)(unsigned long) casting in arch/powerpc/kernel/prom_init.c required?

Does -Os vs -O2 make a difference here?

#1 lore raw
Quoted 2 lines · Olaf Hering

sp is 0x0023e784, val is at offset 120, which makes 0x0023e7fc. This address is casted to u32, which changes it to 0x0023e7f8.

Since when does casting to u32 cut off the low bits?

Andreas.

Signature · 4 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#2 lore raw
Quoted 6 lines · Andreas Schwab
Quoted 2 lines · Olaf Hering

sp is 0x0023e784, val is at offset 120, which makes 0x0023e7fc. This address is casted to u32, which changes it to 0x0023e7f8.

Since when does casting to u32 cut off the low bits?

Way before gcc4.0 release at least.

#3 lore raw
In reply to Olaf Hering
Quoted 14 lines · Olaf Hering

Some G5 and pSeries models dont boot with recent kernels. The reason is likely the casting of pointers of stack variables to u32. One example is the prom_getprop() call in prom_init_stdout().

sp is 0x0023e784, val is at offset 120, which makes 0x0023e7fc. This address is casted to u32, which changes it to 0x0023e7f8. The firmware writes to the wrong addres and things go downhill very quick.

c00000000040baa8: 3b 21 00 78 addi r25,r1,120
..
c00000000040baf4: 57 28 00 38 rlwinm r8,r25,0,0,28
..
c00000000040bb10: 4b ff d3 3d bl c000000000408e4c <.call_prom>

Quoted 5 lines

If I remove the casts and pass the pointer as is, everything starts to work as expected? Why is all this (u32)(unsigned long) casting in arch/powerpc/kernel/prom_init.c required?

Does -Os vs -O2 make a difference here?

Hi Olaf, me again...

<http://gcc.gnu.org/onlinedocs/gcc-4.1.0/gcc/RS_002f6000-and-PowerPC-Options.html>

Looks like you should use -malign-natural to make it do what you want it to do:

-malign-natural
-malign-power
On AIX, 32-bit Darwin, and 64-bit PowerPC GNU/Linux, the option
-malign-natural overrides the ABI-defined alignment of larger types, such as floating-point doubles, on their natural size-based boundary. The option -malign-power instructs GCC to follow the ABI-specified alignment rules. GCC defaults to the standard alignment defined in the ABI.

On 64-bit Darwin, natural alignment is the default, and
-malign-power is not supported.

gvb

#5 lore raw
In reply to Olaf Hering
Quoted 18 lines · Olaf Hering

Some G5 and pSeries models dont boot with recent kernels. The reason is likely the casting of pointers of stack variables to u32. One example is the prom_getprop() call in prom_init_stdout().

sp is 0x0023e784, val is at offset 120, which makes 0x0023e7fc. This address is casted to u32, which changes it to 0x0023e7f8. The firmware writes to the wrong addres and things go downhill very quick.

c00000000040baa8: 3b 21 00 78 addi r25,r1,120
..
c00000000040baf4: 57 28 00 38 rlwinm r8,r25,0,0,28
..
c00000000040bb10: 4b ff d3 3d bl c000000000408e4c <.call_prom>

If I remove the casts and pass the pointer as is, everything starts to work as expected? Why is all this (u32)(unsigned long) casting in arch/powerpc/kernel/prom_init.c required?

Quoted 5 lines

Does -Os vs -O2 make a difference here?
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev

Hi Olaf,

The casting is 8-byte aligning the address because it is a 64 bit variable and it is frowned on (and on some processors, fatal) to have an 8-byte variable misaligned (not on a 8 byte boundary).

In your example above, the variable is named "sp"... the sp is suppose to be on a 8 byte boundary per the EABI (quoted below... note that the ABI requires it to be on a 16 byte boundary):
----
The Stack Frame
Unlike the SVR4 ABI, the stack pointer (GPR1) shall maintain 8-byte alignment, from initialization through all routine calls and dynamic stack space allocation.
----
In the instance above, the proper solution (but I don't know if it is a realistic solution :-/) is to properly align the stack pointer on a 8 byte boundary. I also don't know if there are other, non sp variable, problems. It sounds like the prom isn't 64 bit clean. What are our options to make it 64 bit clean?

Disclaimer: Yeah, I know most PPCs handle misaligned longs, but that doesn't make it _right_ and it definitely doesn't make it efficient.

gvb

#6 lore raw

PATCH 1/1 Workaround gcc bug #26549 which causes pointers to be truncated.

In reply to Olaf Hering

Workaround gcc bug #26549 which causes pointers to be truncated.

The address of variable val in prom_init_stdout is passed to prom_getprop. prom_getprop casts the pointer to u32 and passes it to call_prom in the hope that OpenFirmware stores something there.
But the pointer is truncated in the lower bits and the expected value is stored somewhere else.
This gcc bug does not exist in SLES9 gcc33 toolchain, but it is present in gcc4.0+, likely also in gcc3.4. Compiling a testcase without any -O produces also correct code.

In my testing I had a stackpointer of 0x0023e6b4. val was at offset 120, wich has address 0x0023e72c. But the value passed to OF was 0x0023e728.

c00000000040b710: 3b 01 00 78 addi r24,r1,120
...
c00000000040b754: 57 08 00 38 rlwinm r8,r24,0,0,28
...
c00000000040b784: 80 01 00 78 lwz r0,120(r1)
...
c00000000040b798: 90 1b 00 0c stw r0,12(r27)
...

simple testcase:

int f(unsigned);
void g(void)

{
	unsigned a;
	unsigned int b = (unsigned long)(void*)(&a);
	f(b);
}

asm should look like this:

   c:   38 61 00 70     addi    r3,r1,112
  10:   78 63 00 20     clrldi  r3,r3,32
  14:   48 00 00 01     bl      14 <.g+0x14>

But instead it looks like:

   c:   38 61 00 70     addi    r3,r1,112
  10:   54 63 00 36     rlwinm  r3,r3,0,0,27
  14:   48 00 00 01     bl      14 <.g+0x14>

So just uninline prom_getprop to workaround this gcc bug.

c000000000409034 <.prom_getprop>:

c000000000409054:       7c bc 2b 78     mr      r28,r5
c000000000409074:       7b 88 00 20     clrldi  r8,r28,32

...
c00000000040b29c <.prom_init>:
...
c00000000040b3d0: 3b 41 00 74 addi r26,r1,116
...

c00000000040b3f8:       e8 82 a4 a8     ld      r4,-23384(r2)
c00000000040b3fc:       7f 45 d3 78     mr      r5,r26
c00000000040b404:       80 7d 00 04     lwz     r3,4(r29)
c00000000040b408:       38 c0 00 04     li      r6,4
c00000000040b410:       4b ff dc 25     bl      c000000000409034 <.prom_getprop>

...
c00000000040b424: 80 01 00 74 lwz r0,116(r1)

 arch/powerpc/kernel/prom_init.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/kernel/prom_init.c +2 -1
Index: linux-2.6.16-rc5-olh/arch/powerpc/kernel/prom_init.c
===================================================================
@@ -422,7 +422,8 @@ static int __init prom_next_node(phandle
 	}
 }
 
-static int inline prom_getprop(phandle node, const char *pname,
+/* do not mark as inline to work around gcc bug #26549 */
+static int __init prom_getprop(phandle node, const char *pname,
 			       void *value, size_t valuelen)
 {
 	return call_prom("getprop", 4, 1, node, ADDR(pname),
#4 lore raw
In reply to Jerry Van Baren
Quoted 1 line · Jerry Van Baren

Looks like you should use -malign-natural to make it do what you want it

Same story.

c00000000040b924:       3b 01 00 78     addi    r24,r1,120
c00000000040b968:       57 08 00 38     rlwinm  r8,r24,0,0,28
c00000000040b984:       4b ff d6 01     bl      c000000000408f84 <.call_prom>
c00000000040b998:       80 01 00 78     lwz     r0,120(r1)

,28 is even worse.

#7 lore raw

PATCH 1/1 force stackpointer alignment in 64bit kernel

In reply to Olaf Hering

Fix gcc "bug" #26549 which causes pointers to be truncated.

The address of variable val in prom_init_stdout is passed to prom_getprop. prom_getprop casts the pointer to u32 and passes it to call_prom in the hope that OpenFirmware stores something there.
But the pointer is truncated in the lower bits and the expected value is stored somewhere else.

In my testing I had a stackpointer of 0x0023e6b4. val was at offset 120, wich has address 0x0023e72c. But the value passed to OF was 0x0023e728.

c00000000040b710: 3b 01 00 78 addi r24,r1,120
...
c00000000040b754: 57 08 00 38 rlwinm r8,r24,0,0,28
...
c00000000040b784: 80 01 00 78 lwz r0,120(r1)
...
c00000000040b798: 90 1b 00 0c stw r0,12(r27)
...

The stackpointer came from 32bit code, which appearently has different alignment rules than 64bit code. The chain was yaboot -> zImage -> vmlinux Force the stackpointer to be 16 byte aligned.

 arch/powerpc/kernel/head_64.S |    2 ++
 1 files changed, 2 insertions(+)
arch/powerpc/kernel/head_64.S +2 -0
Index: linux-2.6.16-rc5-olh/arch/powerpc/kernel/head_64.S
===================================================================
@@ -1547,6 +1547,8 @@ _STATIC(__boot_from_prom)
 	mr	r27,r7
 
 	/* Make sure we are running in 64 bits mode */
+	addi	r1,r1,16
+	rlwinm	r1,r1,0,0,28
 	bl	.enable_64b_mode
 
 	/* put a relocation offset into r3 */
#8 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

arch/powerpc/kernel/head_64.S _STATIC(__boot_from_prom) · line 1550

Quoted 1 line · Olaf Hering
arch/powerpc/kernel/head_64.S +1 -0
+	addi	r1,r1,16

Guess that should be -16 ...

#9 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

In reply to Olaf Hering
Quoted 3 lines

The stackpointer came from 32bit code, which appearently has different alignment rules than 64bit code. The chain was yaboot -> zImage -> vmlinux Force the stackpointer to be 16 byte aligned.

The stack pointer is required to be 16-byte aligned when the client program is started, on 32-bit as well.

arch/powerpc/kernel/head_64.S _STATIC(__boot_from_prom) · line 1552

Quoted 4 lines
arch/powerpc/kernel/head_64.S +2 -0
 	/* Make sure we are running in 64 bits mode */
+	addi	r1,r1,16
+	rlwinm	r1,r1,0,0,28
 	bl	.enable_64b_mode

Not addi +16, not -16, just no addi at all...

Segher

#10 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

Quoted 6 lines

arch/powerpc/kernel/head_64.S _STATIC(__boot_from_prom) · line 1552

Quoted 4 lines
arch/powerpc/kernel/head_64.S +2 -0
 	/* Make sure we are running in 64 bits mode */
+	addi	r1,r1,16
+	rlwinm	r1,r1,0,0,28
 	bl	.enable_64b_mode

Not addi +16, not -16, just no addi at all...

Not rlwinm either, GPR1 can potentially be at >= 4GB (of course we'd probably hit other problems then, but hey). So please use rldicr or similar.

Segher

#11 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

In reply to Segher Boessenkool
Quoted 7 lines · Segher Boessenkool
Quoted 4 lines

The stackpointer came from 32bit code, which appearently has different alignment rules than 64bit code. The chain was yaboot -> zImage -> vmlinux
Force the stackpointer to be 16 byte aligned.

The stack pointer is required to be 16-byte aligned when the client program is started, on 32-bit as well.

So the magic word is yaboot.

#12 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

Quoted 1 line · Olaf Hering

So the magic word is yaboot.

sp was 0x0023e6b4 when entering zImage.

yaboot_start has 64 bytes
yaboot_main has 304 bytes

This would mean the stackpointer for yaboot was 0x0023e824. Will check on Monday.

#13 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

In reply to Segher Boessenkool
Quoted 7 lines · Segher Boessenkool
Quoted 4 lines

The stackpointer came from 32bit code, which appearently has different alignment rules than 64bit code. The chain was yaboot -> zImage -> vmlinux
Force the stackpointer to be 16 byte aligned.

The stack pointer is required to be 16-byte aligned when the client program is started, on 32-bit as well.

See http://ozlabs.org/pipermail/linuxppc-dev/2006-March/021342.html

Either yaboot and/or zImage need to force a stack alignment, or we apply the 2 liner.

#14 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

Quoted 7 lines · Olaf Hering
Quoted 5 lines · Segher Boessenkool
Quoted 4 lines

The stackpointer came from 32bit code, which appearently has different alignment rules than 64bit code. The chain was yaboot -> zImage -> vmlinux
Force the stackpointer to be 16 byte aligned.

Quoted 7 lines
Quoted 2 lines

The stack pointer is required to be 16-byte aligned when the client program is started, on 32-bit as well.

See http://ozlabs.org/pipermail/linuxppc-dev/2006-March/021342.html

Either yaboot and/or zImage need to force a stack alignment, or we apply the 2 liner.

Happy to look at/receive patches for yaboot. I'll look on Monday.

Paul

#15 lore raw

Re: [PATCH] force stackpointer alignment in 64bit kernel

In reply to Olaf Hering
Quoted 7 lines
Quoted 2 lines

The stack pointer is required to be 16-byte aligned when the client program is started, on 32-bit as well.

See http://ozlabs.org/pipermail/linuxppc-dev/2006-March/021342.html

Either yaboot and/or zImage need to force a stack alignment, or we apply the 2 liner.

Yaboot should do it. Of course, it won't hurt if the kernel will do it itself, too...

Segher