Skip to contentSkip to conversation

powerpc: enforce a non-spe kernel build even on broken compilers

PATCH In review Sebastian Siewior 4 participants 14 messages Sep 1, 2008 – Oct 24, 2008 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 powerpc: enforce a non-spe kernel build even on broken compilers

From: Thiemo Seufer <[email protected]>

those two are requried on my fresh gcc 4.3.1

---
Not sure if this is intendent or a gcc bug but with -mno-spe the spe opcodes were not used floating point anymore but for 64bit save/restore for instance.

 arch/powerpc/Makefile |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
arch/powerpc/Makefile +3 -0
index b7d4c4c..3727e4f 100644
@@ -108,7 +108,10 @@ endif
 KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
 
 # No SPE instruction when building kernel
+# (We use all available options to help semi-broken compilers)
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)
+KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)
 
 # Enable unit-at-a-time mode when possible. It shrinks the
 # kernel considerably.
Signature · 1 line
1.5.6.5
#1 lore raw
Quoted 8 lines · Sebastian Siewior

those two are requried on my fresh gcc 4.3.1

---
Not sure if this is intendent or a gcc bug but with -mno-spe the spe opcodes were not used floating point anymore but for 64bit save/restore for instance.

I wouldn't say this is due to a broken compiler. As I understand it, -mabi=no-spe and -mspe=no serve two different purposes. One is for disabling the SPE instructions and the other controls the ABI (which would make those 64-bit save/restores I'm guessing). I don't know why you'd ever want to use the SPE ABI without -mspe=yes, but gcc does provide that flexibility.

-mno-spe: Deprecated way to say "no SPE instructions" -mspe=no: New way to do -mno-spe -mabi=no-spe: Disable SPE ABI

Some compilers may enable "-mabi=spe" and/or "-mspe=yes" by default, so explicitly disabling both is necessary. I recently built a SPE toolchain which enabled both by default, so I ran into the "SPE used in kernel" problem when the kernel only passed "-mno-spe".

- Nate Case <[email protected]>

#2 lore raw
Quoted 3 lines · Nate Case

-mno-spe: Deprecated way to say "no SPE instructions" -mspe=no: New way to do -mno-spe

Sorry, I got this backwards (as Kumar pointed out in his other reply). -mspe=no is actually deprecated.

- Nate Case <[email protected]>

#3 lore raw
In reply to Sebastian Siewior
Quoted 10 lines · Sebastian Siewior

From: Thiemo Seufer <[email protected]>

those two are requried on my fresh gcc 4.3.1

---
Not sure if this is intendent or a gcc bug but with -mno-spe the spe opcodes were not used floating point anymore but for 64bit save/restore for instance.

what code is getting generated for you that is causing issue?

Quoted 8 lines

arch/powerpc/Makefile | 3 +++

1 files changed, 3 insertions(+), 0 deletions(-)
arch/powerpc/Makefile +0 -0
index b7d4c4c..3727e4f 100644

arch/powerpc/Makefile endif · line 113

Quoted 6 lines
arch/powerpc/Makefile +2 -0
@@ -108,7 +108,10 @@ endif
 KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
 
 # No SPE instruction when building kernel
+# (We use all available options to help semi-broken compilers)
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)

Why does -mno-spe work?

From my gcc-4.3 info pages:

`-mspe=YES/NO'
This option has been deprecated. Use -mspe' and -mno-spe' instead.

arch/powerpc/Makefile endif · line 114

Quoted 1 line
arch/powerpc/Makefile +1 -0
+KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)

is the -mabi=no-spe really needed?

- k

#4 lore raw
Quoted 11 lines · Kumar Gala

Why does -mno-spe work?

From my gcc-4.3 info pages:

`-mspe=YES/NO'
This option has been deprecated. Use -mspe' and -mno-spe' instead.

Quoted 1 line

+KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)

is the -mabi=no-spe really needed?

My guess is that the -mabi=no-spe was the real key of what made it work for him. I went through the same thing with my toolchain.

You do need -mabi=no-spe if your toolchain defaults to -mabi=spe like mine does. I know that the more generic toolchains out there (CodeSourcery, ELDK) default to -mabi=no-spe, so in that case it would not be necessary.

I don't know what generated instructions are actually to blame, but I do know that if you compile certain programs with "-mno-spe -mabi=spe" vs. "-mno-spe -mabi=no-spe", the results will differ. In the case of the kernel, you'll get a bunch of "SPE used in kernel" messages with the "-mno-spe -mabi=spe" combination.

- Nate Case <[email protected]>

#5 lore raw
In reply to Kumar Gala
Quoted 5 lines · * Kumar Gala | 2008-10-14 16:23:05 [-0500]:
Quoted 3 lines

Not sure if this is intendent or a gcc bug but with -mno-spe the spe opcodes were not used floating point anymore but for 64bit save/restore for instance.

what code is getting generated for you that is causing issue?

Without the patch do_syslog() looks like this:

|c0025840 <do_syslog>:
|c0025840:       94 21 ff 50     stwu    r1,-176(r1)
|c0025844:       7c 08 02 a6     mflr    r0
|c0025848:       13 21 73 21     evstdd  r25,112(r1)
|c002584c:       13 41 7b 21     evstdd  r26,120(r1)
|c0025850:       7c 9a 23 78     mr      r26,r4
|c0025854:       13 61 83 21     evstdd  r27,128(r1)
|c0025858:       7c bb 2b 78     mr      r27,r5
|c002585c:       13 e1 a3 21     evstdd  r31,160(r1)
|c0025860:       7c 7f 1b 78     mr      r31,r3
|c0025864:       90 01 00 b4     stw     r0,180(r1)
|c0025868:       12 21 33 21     evstdd  r17,48(r1)
|c002586c:       12 41 3b 21     evstdd  r18,56(r1)
|c0025870:       12 61 43 21     evstdd  r19,64(r1)
|c0025874:       12 81 4b 21     evstdd  r20,72(r1)
|c0025878:       12 a1 53 21     evstdd  r21,80(r1)
|c002587c:       12 c1 5b 21     evstdd  r22,88(r1)
|c0025880:       12 e1 63 21     evstdd  r23,96(r1)
|c0025884:       13 01 6b 21     evstdd  r24,104(r1)
|c0025888:       13 81 8b 21     evstdd  r28,136(r1)
|c002588c:       13 a1 93 21     evstdd  r29,144(r1)
|c0025890:       13 c1 9b 21     evstdd  r30,152(r1)
|c0025894:       48 0f 00 b5     bl      c0115948 <cap_syslog>
|c0025898:       7c 79 1b 79     mr.     r25,r3
|c002589c:       40 82 04 54     bne-    c0025cf0 <do_syslog+0x4b0>
|c00258a0:       2b 9f 00 0a     cmplwi  cr7,r31,10
|c00258a4:       41 9d 04 40     bgt-    cr7,c0025ce4 <do_syslog+0x4a4>
|c00258a8:       3d 20 c0 27     lis     r9,-16345
|c00258ac:       57 e0 10 3a     rlwinm  r0,r31,2,0,29
|c00258b0:       39 29 a6 e4     addi    r9,r9,-22812
|c00258b4:       7c 09 00 2e     lwzx    r0,r9,r0
|c00258b8:       7c 00 4a 14     add     r0,r0,r9
|c00258bc:       7c 09 03 a6     mtctr   r0
|c00258c0:       4e 80 04 20     bctr
|c00258c4:       3b e0 00 00     li      r31,0

....

do_syslog() is not the only one.

Quoted 9 lines
Quoted 9 lines
1 files changed, 3 insertions(+), 0 deletions(-)
arch/powerpc/Makefile +0 -0
index b7d4c4c..3727e4f 100644
@@ -108,7 +108,10 @@ endif
 KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
 
Quoted 6 lines

arch/powerpc/Makefile endif · line 113

Quoted 4 lines
arch/powerpc/Makefile +2 -0
 # No SPE instruction when building kernel
+# (We use all available options to help semi-broken compilers)
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)

Why does -mno-spe work?

Good question. Without this option and only with -mabi=no-spe the do_syslog() looks like the following:

|c0025270 <do_syslog>:
|c0025270:       94 21 ff 90     stwu    r1,-112(r1)
|c0025274:       7c 08 02 a6     mflr    r0
|c0025278:       be 21 00 34     stmw    r17,52(r1)
|c002527c:       7c 9a 23 78     mr      r26,r4
|c0025280:       90 01 00 74     stw     r0,116(r1)
|c0025284:       7c bb 2b 78     mr      r27,r5
|c0025288:       7c 7f 1b 78     mr      r31,r3
|c002528c:       48 0e ce c5     bl      c0112150 <cap_syslog>
|c0025290:       7c 79 1b 79     mr.     r25,r3
|c0025294:       40 82 04 54     bne-    c00256e8 <do_syslog+0x478>
|c0025298:       2b 9f 00 0a     cmplwi  cr7,r31,10
|c002529c:       41 9d 04 40     bgt-    cr7,c00256dc <do_syslog+0x46c>
|c00252a0:       3d 20 c0 26     lis     r9,-16346
|c00252a4:       57 e0 10 3a     rlwinm  r0,r31,2,0,29
|c00252a8:       39 29 36 e4     addi    r9,r9,14052
|c00252ac:       7c 09 00 2e     lwzx    r0,r9,r0
|c00252b0:       7c 00 4a 14     add     r0,r0,r9
|c00252b4:       7c 09 03 a6     mtctr   r0
|c00252b8:       4e 80 04 20     bctr
|c00252bc:       3b e0 00 00     li      r31,0
|c00252c0:       48 00 01 c4     b       c0025484 <do_syslog+0x214>
|c00252c4:       2f 9a 00 00     cmpwi   cr7,r26,0
|c00252c8:       41 9e 04 14     beq-    cr7,c00256dc <do_syslog+0x46c>
|c00252cc:       2f 9b 00 00     cmpwi   cr7,r27,0
|c00252d0:       41 9c 04 0c     blt-    cr7,c00256dc <do_syslog+0x46c>
|c00252d4:       41 9e 04 14     beq-    cr7,c00256e8 <do_syslog+0x478>
|c00252d8:       80 02 03 3c     lwz     r0,828(r2)
|c00252dc:       7f 9a 00 40     cmplw   cr7,r26,r0
|c00252e0:       41 9d 04 04     bgt-    cr7,c00256e4 <do_syslog+0x474>
|c00252e4:       7c 1a 00 50     subf    r0,r26,r0
|c00252e8:       39 3b ff ff     addi    r9,r27,-1
|c00252ec:       7f 89 00 40     cmplw   cr7,r9,r0
|c00252f0:       41 9d 03 f4     bgt-    cr7,c00256e4 <do_syslog+0x474>
|c00252f4:       3d 20 c0 39     lis     r9,-16327
|c00252f8:       39 09 8e f8     addi    r8,r9,-28936
|c00252fc:       81 29 8e f8     lwz     r9,-28936(r9)
|c0025300:       80 08 00 04     lwz     r0,4(r8)
|c0025304:       7f 80 48 00     cmpw    cr7,r0,r9
|c0025308:       40 be 00 a4     bne+    cr7,c00253ac <do_syslog+0x13c>
|c002530c:       3d 20 c0 04     lis     r9,-16380
|c0025310:       39 61 00 14     addi    r11,r1,20
|c0025314:       39 29 ae 78     addi    r9,r9,-20872
|c0025318:       10 00 02 16     evxor   r0,r0,r0
|c002531c:       10 01 0b 21     evstdd  r0,8(r1)
|c0025320:       7d 1f 43 78     mr      r31,r8

boom.

Quoted 7 lines

From my gcc-4.3 info pages:

`-mspe=YES/NO'
This option has been deprecated. Use -mspe' and -mno-spe' instead.

so it could be gcc bug then and I should open a bugzilla. I didn't know that it is deprecated.

Quoted 3 lines

arch/powerpc/Makefile endif · line 114

Quoted 1 line
arch/powerpc/Makefile +1 -0
+KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)

is the -mabi=no-spe really needed?

It seems to work the other way around (without -mabi=no-spe but with -mabi=no-spe) alteast I did not find anything in do_syslog() or while browsing through the dissasm. I do a boot check tomorrow.

Quoted 2 lines

- k

Sebastian

#6 lore raw
Quoted 4 lines · * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
Quoted 1 line

is the -mabi=no-spe really needed?

It seems to work the other way around (without -mabi=no-spe but with -mabi=no-spe) alteast I did not find anything in do_syslog() or while browsing through the dissasm. I do a boot check tomorrow.

Okay. I recompiled with -mspe=no and was able to boot without trouble. It didn't work with -mabi=no-spe.

Quoted 2 lines
Quoted 2 lines

- k

Sebastian

#7 lore raw
Quoted 9 lines · Sebastian Andrzej Siewior
Quoted 4 lines · * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
Quoted 1 line

is the -mabi=no-spe really needed?

It seems to work the other way around (without -mabi=no-spe but with -mabi=no-spe) alteast I did not find anything in do_syslog() or while browsing through the dissasm. I do a boot check tomorrow.

Okay. I recompiled with -mspe=no and was able to boot without trouble. It didn't work with -mabi=no-spe.

I'm confused.. what did you change from your patch?

- k

#8 lore raw
Quoted 6 lines · * Kumar Gala | 2008-10-15 08:25:54 [-0500]:
Quoted 3 lines · Sebastian Andrzej Siewior
Quoted 1 line · * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
Quoted 1 line

is the -mabi=no-spe really needed?

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

It seems to work the other way around (without -mabi=no-spe but with -mabi=no-spe) alteast I did not find anything in do_syslog() or while browsing through the dissasm. I do a boot check tomorrow.

Okay. I recompiled with -mspe=no and was able to boot without trouble. It didn't work with -mabi=no-spe.

I'm confused.. what did you change from your patch?

This is the patch it right now:

arch/powerpc/Makefile +1 -0
@@ -103,6 +103,7 @@ KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
 
 # No SPE instruction when building kernel
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)
 
 # Enable unit-at-a-time mode when possible. It shrinks the
 # kernel considerably.
Signature · 2 lines
With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.
Quoted 1 line

- k

Sebastian

#9 lore raw
Quoted 33 lines · Sebastian Andrzej Siewior
Quoted 11 lines · * Kumar Gala | 2008-10-15 08:25:54 [-0500]:
Quoted 8 lines · Sebastian Andrzej Siewior
Quoted 5 lines · * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
Quoted 1 line

is the -mabi=no-spe really needed?

It seems to work the other way around (without -mabi=no-spe but with -mabi=no-spe) alteast I did not find anything in do_syslog() or while
browsing through the dissasm. I do a boot check tomorrow.

Quoted 4 lines
Quoted 2 lines

Okay. I recompiled with -mspe=no and was able to boot without trouble. It didn't work with -mabi=no-spe.

I'm confused.. what did you change from your patch?

This is the patch it right now:

arch/powerpc/Makefile +1 -0
@@ -103,6 +103,7 @@ KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
 
 # No SPE instruction when building kernel
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)
 
 # Enable unit-at-a-time mode when possible. It shrinks the
 # kernel considerably.
Signature · 2 lines
With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.

ok. can post a clean patch w/just this change.

- k

#10 lore raw
In reply to Sebastian Andrzej Siewior
Quoted 2 lines · Sebastian Andrzej Siewior

With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.

Please don't accept this patch yet. My past testing showed that "-mabi=no-spe" was required for my toolchain. I'll go back and double check though.

- Nate Case <[email protected]>

#11 lore raw
Quoted 7 lines · * Nate Case | 2008-10-17 09:02:11 [-0500]:
Quoted 2 lines · Sebastian Andrzej Siewior

With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.

Please don't accept this patch yet. My past testing showed that "-mabi=no-spe" was required for my toolchain. I'll go back and double check though.

Okay. Here my compiler details:

| powerpc-linux-gnuspe-gcc -v 
| Using built-in specs.
| Target: powerpc-linux-gnuspe
| Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.1-9'
| --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
| --enable-languages=c,c++,obj-c++ --prefix=/usr --enable-shared
| --with-system-zlib --libexecdir=/usr/lib --without-included-gettext
| --enable-threads=posix --enable-nls
| --with-gxx-include-dir=/usr/powerpc-linux-gnuspe/include/c++/4.3.1
| --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug
| --disable-multilib --disable-multilib --with-cpu=8540
| --enable-e500_double --with-long-double-128 --enable-checking=release
| --program-prefix=powerpc-linux-gnuspe-
| --includedir=/usr/powerpc-linux-gnuspe/include --build=i486-linux-gnu
| --host=i486-linux-gnu --target=powerpc-linux-gnuspe
| Thread model: posix
| gcc version 4.3.1 (Debian 4.3.1-9) 

ths told me that we need all three options in this order because of how they interact with gcc's internal flags.

Quoted 1 line

- Nate Case <[email protected]>

Sebastian

#12 lore raw
In reply to Nate Case
Quoted 6 lines · Nate Case
Quoted 2 lines

With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.

Please don't accept this patch yet. My past testing showed that "-mabi=no-spe" was required for my toolchain. I'll go back and double check though.

OK, I went back and re-tested.

Kernel: 2.6.27
CPU: MPC8572
Toolchain:
Cross-compiler built using crosstool-ng
gcc 4.3.1, default target CFLAGS include '-mabi=spe -mspe' binutils 2.18.90 snapshot (built with --enable-spe=yes)

Kbuild flags                    Result
------------                    ------
-mno-spe (*)                    FAILED
-mno-spe -mabi=no-spe           FAILED
-mno-spe -mspe=no               OK
-mspe=no                        OK

(*) 2.6.27 default

In the failure case, the kernel would repeatedly dump out "SPE used in kernel (task=xxxxxxxx, pc=xxxxxxxx)".

I think I was fooled before because I added _both_ "-mspe=no" and "-mabi=no-spe" to my KBUILD_CFLAGS and saw the problem go away. Since I trusted the documentation that -mspe=no and -mno-spe were the same, I assumed that -mabi=no-spe was the key.

So, I've changed my mind. I now agree with Sebastian that "-mabi=no-spe" is not required. "-mno-spe -mspe=no" is probably the safe way to go.

Signature · 1 line
Nate Case <[email protected]>
#13 lore raw
Quoted 29 lines · Nate Case
Quoted 6 lines · Nate Case
Quoted 2 lines

With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.

Please don't accept this patch yet. My past testing showed that "-mabi=no-spe" was required for my toolchain. I'll go back and double check though.

OK, I went back and re-tested.

Kernel: 2.6.27
CPU: MPC8572
Toolchain:
Cross-compiler built using crosstool-ng
gcc 4.3.1, default target CFLAGS include '-mabi=spe -mspe' binutils 2.18.90 snapshot (built with --enable-spe=yes)

Kbuild flags Result
------------ ------

(quoted diff) +0 -4
-mno-spe (*)                    FAILED
-mno-spe -mabi=no-spe           FAILED
-mno-spe -mspe=no               OK
-mspe=no                        OK

(*) 2.6.27 default

In the failure case, the kernel would repeatedly dump out "SPE used in kernel (task=xxxxxxxx, pc=xxxxxxxx)".

Quoted 11 lines

I think I was fooled before because I added _both_ "-mspe=no" and "-mabi=no-spe" to my KBUILD_CFLAGS and saw the problem go away. Since I trusted the documentation that -mspe=no and -mno-spe were the same, I assumed that -mabi=no-spe was the key.

So, I've changed my mind. I now agree with Sebastian that "-mabi=no-spe" is not required. "-mno-spe -mspe=no" is probably the safe way to go.

Signature · 1 line
Nate Case <[email protected]>

thanks for testing this all out.

I'll submit a patch to remove the -mabi=.*spe.* foo.

- k