Skip to contentSkip to conversation

mpc8xx and ld.so problem

DISCUSSION Marcelo Tosatti 9 participants 24 messages Jul 1, 2005 – Jul 19, 2005 4 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

Marcelo Tosatti

author
#3 lore raw

Hi Anton,

(moving to ppc-embedded since it might be of interesting for other=20 8xx users)

Quoted 11 lines · On Thu, Jun 30, 2005 at 04:13:30PM +0200, Anton W=F6llert wrote:

Hello Marcelo
=20
I suggest you should find out why binaries hang and where.

Quoted 5 lines

=20
You can see where processes are sleeping with:
=20
ps -axeo "command nwchan"
=20

=20
thank you for that tip. but i found out (what i should have had to do=20 before), that the application doesn't hang in kernel-mode. so wchan doe=

sn't=20

Quoted 1 line

say anything. but with gdb i saw the problem, the application hangs in =

the=20

Quoted 1 line

function memset at the instruction dcbz, this should be a instruction, =

that=20

Quoted 1 line

loops until it something is zero or so ( sorry, that i didn't looked up=

it=20

Quoted 1 line

yet, i will do that ). and because of the bug of these dcbx instruction=

s on=20

Quoted 1 line

8xx i think, that this is the cause. here my gdb-session, i hope you ma=

y=20

Quoted 4 lines

find it helpful or give me an advise how to fix that :
=20
awoeller@zwiebel :~/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/busybox-1.00$po=

werpc-linux-gdb

Quoted 3 lines

GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and yo=

u are

Quoted 4 lines

welcome to change it and/or distribute copies of it under certain=20 conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for detai=

ls.

Quoted 1 line

This GDB was configured as "--host=3Di686-linux --target=3Dpowerpc-linu=

x".

Quoted 4 lines

(gdb) set solib-absolute-prefix /tmp/fakelibc=20
(gdb) file busybox
Reading symbols from=20 /home/awoeller/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/bus=

ybox-

Quoted 4 lines

1.00/busybox...done.
(gdb) b main
Breakpoint 1 at 0x1000398c: file=20 /home/awoeller/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/bus=

ybox-

Quoted 6 lines

1.00/applets/busybox.c, line 75.
(gdb) target remote tqm850l:123
Remote debugging using tqm850l:123
0x300103f4 in ?? ()
(gdb) cont
Continuing.

Quoted 11 lines
# here i interrupt, because it hangs
Program received signal SIGINT, Interrupt.
0x30013e58 in ?? ()
(gdb) return
Make selected stack frame return now? (y or n) y
#0 0x3000e374 in ?? ()
(gdb) cont
Continuing.
=20
Breakpoint 1, main (argc=3D1, argv=3D0x7ffffeb4) at=20
/home/awoeller/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/bus=

ybox-

Quoted 186 lines

1.00/applets/busybox.c:75
75 bb_applet_name =3D argv[0];
=20
(gdb) disas 0x30013e58
Dump of assembler code for function memset:
0x30013ba4 <memset+0>: cmplwi cr1,r5,4 0x30013ba8 <memset+4>: andi. r7,r3,3
0x30013bac <memset+8>: mr r6,r3
0x30013bb0 <memset+12>: ble- cr1,0x30013d40 <memset+412> 0x30013bb4 <memset+16>: cmplwi cr5,r5,31 0x30013bb8 <memset+20>: rlwimi r4,r4,8,16,23 0x30013bbc <memset+24>: beq+ 0x30013be0 <memset+60> 0x30013bc0 <memset+28>: mtcrf 1,r3
0x30013bc4 <memset+32>: subfic r7,r7,4 0x30013bc8 <memset+36>: add r6,r6,r7
0x30013bcc <memset+40>: subf r5,r7,r5
0x30013bd0 <memset+44>: bns+ cr7,0x30013bdc <memset+56> 0x30013bd4 <memset+48>: stb r4,0(r3)
0x30013bd8 <memset+52>: beq- cr7,0x30013be0 <memset+60> 0x30013bdc <memset+56>: sth r4,-2(r6)
0x30013be0 <memset+60>: mtcrf 1,r5
0x30013be4 <memset+64>: rlwimi r4,r4,16,0,15 0x30013be8 <memset+68>: ble- cr5,0x30013d80 <memset+476> 0x30013bec <memset+72>: andi. r7,r6,28 0x30013bf0 <memset+76>: subfic r7,r7,32 0x30013bf4 <memset+80>: beq- 0x30013c34 <memset+144> 0x30013bf8 <memset+84>: mtcrf 1,r7
0x30013bfc <memset+88>: add r6,r6,r7
0x30013c00 <memset+92>: subf r5,r7,r5
0x30013c04 <memset+96>: cmplwi cr1,r7,16 0x30013c08 <memset+100>: mr r8,r6
0x30013c0c <memset+104>: bge- cr7,0x30013c18 <memset+116> 0x30013c10 <memset+108>: stw r4,-4(r8) 0x30013c14 <memset+112>: stwu r4,-8(r8) 0x30013c18 <memset+116>: blt- cr1,0x30013c2c <memset+136> 0x30013c1c <memset+120>: stw r4,-4(r8) 0x30013c20 <memset+124>: stw r4,-8(r8) 0x30013c24 <memset+128>: stw r4,-12(r8) 0x30013c28 <memset+132>: stwu r4,-16(r8) 0x30013c2c <memset+136>: ble- cr7,0x30013c34 <memset+144> 0x30013c30 <memset+140>: stw r4,-4(r8) 0x30013c34 <memset+144>: cmplwi cr1,r4,0 0x30013c38 <memset+148>: rlwinm. r7,r5,0,0,26 0x30013c3c <memset+152>: mtcrf 1,r5
0x30013c40 <memset+156>: beq- cr1,0x30013de0 <memset+572> 0x30013c44 <memset+160>: rlwinm r0,r7,27,5,31 0x30013c48 <memset+164>: mtctr r0
0x30013c4c <memset+168>: beq- 0x30013d80 <memset+476> 0x30013c50 <memset+172>: clrlwi. r5,r5,27 0x30013c54 <memset+176>: add r6,r6,r7
0x30013c58 <memset+180>: li r8,-64
0x30013c5c <memset+184>: bdz- 0x30013c90 <memset+236> 0x30013c60 <memset+188>: dcbtst r8,r6
0x30013c64 <memset+192>: stw r4,-4(r6) 0x30013c68 <memset+196>: stw r4,-8(r6) 0x30013c6c <memset+200>: stw r4,-12(r6) 0x30013c70 <memset+204>: stw r4,-16(r6) 0x30013c74 <memset+208>: nop
0x30013c78 <memset+212>: stw r4,-20(r6) 0x30013c7c <memset+216>: stw r4,-24(r6) 0x30013c80 <memset+220>: nop
0x30013c84 <memset+224>: stw r4,-28(r6) 0x30013c88 <memset+228>: stwu r4,-32(r6) 0x30013c8c <memset+232>: bdnz+ 0x30013c60 <memset+188> 0x30013c90 <memset+236>: stw r4,-4(r6) 0x30013c94 <memset+240>: stw r4,-8(r6) 0x30013c98 <memset+244>: stw r4,-12(r6) 0x30013c9c <memset+248>: stw r4,-16(r6) 0x30013ca0 <memset+252>: stw r4,-20(r6) 0x30013ca4 <memset+256>: cmplwi cr1,r5,16 0x30013ca8 <memset+260>: stw r4,-24(r6) 0x30013cac <memset+264>: stw r4,-28(r6) 0x30013cb0 <memset+268>: stwu r4,-32(r6) 0x30013cb4 <memset+272>: beqlr=20
0x30013cb8 <memset+276>: add r6,r6,r7
0x30013cbc <memset+280>: b 0x30013d84 <memset+480> 0x30013cc0 <memset+284>: nop
0x30013cc4 <memset+288>: clrlwi r5,r5,27 0x30013cc8 <memset+292>: mtcrf 2,r7
0x30013ccc <memset+296>: rlwinm. r0,r7,25,7,31 ---Type <return> to continue, or q <return> to quit---=20 0x30013cd0 <memset+300>: mtctr r0
0x30013cd4 <memset+304>: li r7,32
0x30013cd8 <memset+308>: li r8,-64
0x30013cdc <memset+312>: cmplwi cr1,r5,16 0x30013ce0 <memset+316>: bne- cr6,0x30013cec <memset+328> 0x30013ce4 <memset+320>: dcbz r0,r6
0x30013ce8 <memset+324>: addi r6,r6,32 0x30013cec <memset+328>: li r9,-32
0x30013cf0 <memset+332>: ble- cr6,0x30013d00 <memset+348> 0x30013cf4 <memset+336>: dcbz r0,r6
0x30013cf8 <memset+340>: dcbz r7,r6
0x30013cfc <memset+344>: addi r6,r6,64 0x30013d00 <memset+348>: cmplwi cr5,r5,0 0x30013d04 <memset+352>: beq- 0x30013d80 <memset+476> 0x30013d08 <memset+356>: dcbz r0,r6
0x30013d0c <memset+360>: dcbz r7,r6
0x30013d10 <memset+364>: addi r6,r6,128 0x30013d14 <memset+368>: dcbz r8,r6
0x30013d18 <memset+372>: dcbz r9,r6
0x30013d1c <memset+376>: bdnz+ 0x30013d08 <memset+356> 0x30013d20 <memset+380>: beqlr cr5
0x30013d24 <memset+384>: b 0x30013d84 <memset+480> 0x30013d28 <memset+388>: nop
0x30013d2c <memset+392>: nop
0x30013d30 <memset+396>: nop
0x30013d34 <memset+400>: nop
0x30013d38 <memset+404>: nop
0x30013d3c <memset+408>: nop
0x30013d40 <memset+412>: cmplwi cr5,r5,1 0x30013d44 <memset+416>: cmplwi cr1,r5,3 0x30013d48 <memset+420>: bltlr cr5
0x30013d4c <memset+424>: stb r4,0(r6)
0x30013d50 <memset+428>: beqlr cr5
0x30013d54 <memset+432>: nop
0x30013d58 <memset+436>: stb r4,1(r6)
0x30013d5c <memset+440>: bltlr cr1
0x30013d60 <memset+444>: stb r4,2(r6)
0x30013d64 <memset+448>: beqlr cr1
0x30013d68 <memset+452>: nop
0x30013d6c <memset+456>: stb r4,3(r6)
0x30013d70 <memset+460>: blr
0x30013d74 <memset+464>: nop
0x30013d78 <memset+468>: nop
0x30013d7c <memset+472>: nop
0x30013d80 <memset+476>: cmplwi cr1,r5,16 0x30013d84 <memset+480>: add r6,r6,r5
0x30013d88 <memset+484>: bso- cr7,0x30013da8 <memset+516> 0x30013d8c <memset+488>: beq- cr7,0x30013db0 <memset+524> 0x30013d90 <memset+492>: bgt- cr7,0x30013db8 <memset+532> 0x30013d94 <memset+496>: bge- cr1,0x30013dc0 <memset+540> 0x30013d98 <memset+500>: bgelr cr7
0x30013d9c <memset+504>: stw r4,-4(r6) 0x30013da0 <memset+508>: stw r4,-8(r6) 0x30013da4 <memset+512>: blr
0x30013da8 <memset+516>: stbu r4,-1(r6) 0x30013dac <memset+520>: bne- cr7,0x30013d90 <memset+492> 0x30013db0 <memset+524>: sthu r4,-2(r6) 0x30013db4 <memset+528>: ble- cr7,0x30013d94 <memset+496> 0x30013db8 <memset+532>: stwu r4,-4(r6) 0x30013dbc <memset+536>: blt- cr1,0x30013dd0 <memset+556> 0x30013dc0 <memset+540>: stw r4,-4(r6) 0x30013dc4 <memset+544>: stw r4,-8(r6) 0x30013dc8 <memset+548>: stw r4,-12(r6) 0x30013dcc <memset+552>: stwu r4,-16(r6) 0x30013dd0 <memset+556>: bgelr cr7
0x30013dd4 <memset+560>: stw r4,-4(r6) 0x30013dd8 <memset+564>: stw r4,-8(r6) 0x30013ddc <memset+568>: blr
0x30013de0 <memset+572>: mflr r0
0x30013de4 <memset+576>: beq+ 0x30013d80 <memset+476> 0x30013de8 <memset+580>: bl 0x30029000 <_dl_auxv+180> 0x30013dec <memset+584>: mflr r9
0x30013df0 <memset+588>: lwz r9,1832(r9) 0x30013df4 <memset+592>: lwz r8,0(r9)
0x30013df8 <memset+596>: mtlr r0
0x30013dfc <memset+600>: cmplwi cr1,r8,0
---Type <return> to continue, or q <return> to quit--- 0x30013e00 <memset+604>: beq+ cr1,0x30013c44 <memset+160> 0x30013e04 <memset+608>: cmplwi cr1,r8,32 0x30013e08 <memset+612>: beq+ cr1,0x30013cc4 <memset+288> 0x30013e0c <memset+616>: dcbtst r0,r6
0x30013e10 <memset+620>: addi r9,r8,-1 0x30013e14 <memset+624>: cmplwi cr1,r5,32 0x30013e18 <memset+628>: and. r0,r9,r6 0x30013e1c <memset+632>: blt- cr1,0x30013e68 <memset+708> 0x30013e20 <memset+636>: beq- 0x30013e50 <memset+684> 0x30013e24 <memset+640>: addi r6,r6,32 0x30013e28 <memset+644>: addi r5,r5,-32 0x30013e2c <memset+648>: stw r4,-32(r6) 0x30013e30 <memset+652>: stw r4,-28(r6) 0x30013e34 <memset+656>: stw r4,-24(r6) 0x30013e38 <memset+660>: stw r4,-20(r6) 0x30013e3c <memset+664>: stw r4,-16(r6) 0x30013e40 <memset+668>: stw r4,-12(r6) 0x30013e44 <memset+672>: stw r4,-8(r6) 0x30013e48 <memset+676>: stw r4,-4(r6) 0x30013e4c <memset+680>: b 0x30013e14 <memset+624> 0x30013e50 <memset+684>: cmplw cr1,r5,r8 0x30013e54 <memset+688>: blt- cr1,0x30013e68 <memset+708> =20
=20
0x30013e58 <memset+692>: dcbz r0,r6 #<--- the problem =20
=20
0x30013e5c <memset+696>: subf r5,r8,r5

Quoted 8 lines

0x30013e60 <memset+700>: add r6,r6,r8 0x30013e64 <memset+704>: b 0x30013e50 <memset+684> 0x30013e68 <memset+708>: rlwinm. r7,r5,0,0,26 0x30013e6c <memset+712>: b 0x30013c44 <memset+160> 0x30013e70 <memset+716>: nop
0x30013e74 <memset+720>: nop
0x30013e78 <memset+724>: nop
0x30013e7c <memset+728>: nop

Can you please examine in more detail what is going on here?=20

What are the contents of the r0 and r6 registers, where exactly is the task stopped.

I _guess_ it might be some form of "dcbz" misbehaviour, in such case I=20 imagine that either the task would loop forever or execute an invalid exception/instruction.

#4 lore raw
Quoted 6 lines · Marcelo Tosatti

Hi Anton,

(moving to ppc-embedded since it might be of interesting for other 8xx users)

Apply this patch to glibc, and recompile:

rm -f glibc/sysdeps/powerpc/powerpc32/memset.S

The PPC32 dbcz semantics don't seem to work properly on 8xx in all cases. Removing the '.S' file makes glibc fall back on the .c implementation.

Signature · 2 lines
Jason McMullan <[email protected]>
"Sure, send me the latest Knoppix DVD as an attachment..."
author
#5 lore raw
Quoted 8 lines · Jason McMullan
Quoted 6 lines · Marcelo Tosatti

Hi Anton,

(moving to ppc-embedded since it might be of interesting for other 8xx users)

Quoted 8 lines

Apply this patch to glibc, and recompile:

rm -f glibc/sysdeps/powerpc/powerpc32/memset.S

The PPC32 dbcz semantics don't seem to work properly on 8xx in all cases. Removing the '.S' file makes glibc fall back on the .c implementation.

Hi Jason,

That was a quick response - thanks.

Two questions:

- Do you happen to know about details of dcbz's (mis)behaviour on 8xx?

I ask that mainly because I worry about in-kernel dcbz users.

- Shouldnt upstream glibc have that fixed for 8xx by now?

#6 lore raw
Quoted 7 lines · Marcelo Tosatti

That was a quick response - thanks.

Two questions:

- Do you happen to know about details of dcbz's (mis)behaviour on 8xx?

I ask that mainly because I worry about in-kernel dcbz users.

IIRC, it isn't used in any 8xx code paths.

Quoted 1 line

- Shouldnt upstream glibc have that fixed for 8xx by now?

Ha. Funny. The glibc powerpc maintainer doesn't want any embedded
fixes in the mainline. Last I checked, that was for 'the tools vendors' to fix.

"We won't work around processor bugs" is their philosophy.

I went through a similar (unsuccessful) battle with the
amcc 440ep's "blrl" errata and gcc/glibc.

It would be nice if the politics there have changed
(maybe they just didn't like me personally), but I don't have much hope.

Signature · 2 lines
Jason McMullan <[email protected]>
"Sure, send me the latest Knoppix DVD as an attachment..."
author
#7 lore raw
Quoted 10 lines · Jason McMullan
Quoted 7 lines · Marcelo Tosatti

That was a quick response - thanks.

Two questions:

- Do you happen to know about details of dcbz's (mis)behaviour on 8xx?

I ask that mainly because I worry about in-kernel dcbz users.

IIRC, it isn't used in any 8xx code paths.

OK!

Quoted 4 lines
Quoted 1 line

- Shouldnt upstream glibc have that fixed for 8xx by now?

Ha. Funny. The glibc powerpc maintainer doesn't want any embedded
fixes in the mainline.

Who is that?

Quoted 1 line

Last I checked, that was for 'the tools vendors' to fix.

Silly - so embedded developers are supposed rely on "embedded vendors" and not on the mainstream software distributions?

Quoted 8 lines

"We won't work around processor bugs" is their philosophy.

I went through a similar (unsuccessful) battle with the
amcc 440ep's "blrl" errata and gcc/glibc.

It would be nice if the politics there have changed
(maybe they just didn't like me personally), but I don't have much hope.

If enough people complain they will, hopefully, listen.

#8 lore raw
In reply to Jason McMullan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jason McMullan writes:

[...deleted...]

Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any Jason> embedded fixes in the mainline. Last I checked, that was for Jason> 'the tools vendors' to fix.

Jason> "We won't work around processor bugs" is their philosophy.

[...deleted...]

I investigated the problem a bit when I had trouble with a self-compiled glibc a year or so ago. IIRC, I found bug in the memset code, not in the chip. The code was just wrong for cache line sizes not equal to 32. So memset.S is good for 60x series (PQII included) but for 8xx it fails. We use dcbX instructions in some kernel drivers and since we never had any problems with those drivers I'm a bit surprised to hear that all 8xx chips have got that bug.

Signature · 4 lines
========================================================================
 Yuli Barcohen       | Phone +972-9-765-1788 |  Software Project Leader
 [email protected] | Fax   +972-9-765-7494 | Arabella Software, Israel
========================================================================
#9 lore raw
Quoted 12 lines · Yuli Barcohen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jason McMullan writes:

[...deleted...]

Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any Jason> embedded fixes in the mainline. Last I checked, that was for Jason> 'the tools vendors' to fix.

Jason> "We won't work around processor bugs" is their philosophy.

[...deleted...]

Quoted 7 lines

I investigated the problem a bit when I had trouble with a self-compiled glibc a year or so ago. IIRC, I found bug in the memset code, not in the chip. The code was just wrong for cache line sizes not equal to 32. So memset.S is good for 60x series (PQII included) but for 8xx it fails. We use dcbX instructions in some kernel drivers and since we never had any problems with those drivers I'm a bit surprised to hear that all 8xx chips have got that bug.

It's also OK on a multiple of 32, iirc, but not smaller. And using the information the kernel does export would be too slow. Or at least no one figured out a good way to do it, userspace side.

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
#10 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Tom Rini writes:

Yuli> [...deleted...]

Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any Jason> embedded fixes in the mainline. Last I checked, that was for Jason> 'the tools vendors' to fix.

Jason> "We won't work around processor bugs" is their philosophy.

Yuli> [...deleted...]

Yuli> I investigated the problem a bit when I had trouble with a Yuli> self-compiled glibc a year or so ago. IIRC, I found bug in the Yuli> memset code, not in the chip. The code was just wrong for Yuli> cache line sizes not equal to 32. So memset.S is good for 60x Yuli> series (PQII included) but for 8xx it fails. We use dcbX Yuli> instructions in some kernel drivers and since we never had any Yuli> problems with those drivers I'm a bit surprised to hear that Yuli> all 8xx chips have got that bug.

Tom> It's also OK on a multiple of 32, iirc, but not smaller. And Tom> using the information the kernel does export would be too slow. Tom> Or at least no one figured out a good way to do it, userspace Tom> side.

IMHO the cache line size from cputable is not used in the kernel but only passed to the ELF interpreter. I did not want to re-build glibc so I changed the entry for mpc8xx to pass 0 instead of 16. This disabled the assembler implementation of memset and since then all our mpc8xx-based boards work properly. In any case, since it looks like a coding bug and not CPU errata, maybe the glibc maintainer would be willing to fix it?

Signature · 4 lines
========================================================================
 Yuli Barcohen       | Phone +972-9-765-1788 |  Software Project Leader
 [email protected] | Fax   +972-9-765-7494 | Arabella Software, Israel
========================================================================
author
#11 lore raw
In reply to Yuli Barcohen

Hi Yuli,

Quoted 8 lines · Yuli Barcohen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jason McMullan writes:

[...deleted...]

Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any Jason> embedded fixes in the mainline. Last I checked, that was for Jason> 'the tools vendors' to fix.

Quoted 8 lines

Jason> "We won't work around processor bugs" is their philosophy.

[...deleted...]

I investigated the problem a bit when I had trouble with a self-compiled glibc a year or so ago. IIRC, I found bug in the memset code, not in the chip. The code was just wrong for cache line sizes not equal to 32. So memset.S is good for 60x series (PQII included) but for 8xx it fails.

I suppose you didnt actually use dcbz for userspace memset on 8xx?

Quoted 3 lines

We use dcbX instructions in some kernel drivers and since we never had any problems with those drivers I'm a bit surprised to hear that all 8xx chips have got that bug.

The problem is that the DAR register is correctly unset (it comes as NULL IIRC) on pagefaults for the dcbz instruction. The dcbz instructions you issue are probably always works on kernel addresses whose pagetables are present?

Joakim has developed a workaround for the problem... although I promised him several times to test it I never managed to get dcbz to work on the kernel copying functions. :(

He has posted it here in the past... here it is attached anyway.

===== head_8xx.S 1.21 vs edited =====

head_8xx.S renamed from arch/ppc/kernel/head_8xx.S +225 -5
@@ -32,6 +32,8 @@
 #include <asm/ppc_asm.h>
 #include <asm/offsets.h>
 
+#define CONFIG_8xx_DCBxFIXED
+
 /* Macro to make the code more readable. */
 #ifdef CONFIG_8xx_CPU6
 #define DO_8xx_CPU6(val, reg)	\

@@ -41,6 +43,20 @@
 #else
 #define DO_8xx_CPU6(val, reg)
 #endif
+
+#ifdef CONFIG_8xx_DCBxFIXED
+/* These macros are used to tag DAR with a known value so that the
+ * DataTLBError can recognize a buggy dcbx instruction and workaround
+ * the problem.
+ */
+#define TAG_VAL 0x00f0	/*  -1 may also be used */
+#define TAG_DAR_R10 	\
+	li	r10, TAG_VAL;\
+	mtspr	SPRN_DAR, r10;
+#else
+#define TAG_DAR_R10
+#endif
+
 	.text
 	.globl	_stext
 _stext:

@@ -174,6 +190,7 @@
 	xfer(n, hdlr)
 
 #define EXC_XFER_TEMPLATE(n, hdlr, trap, copyee, tfer, ret)	\
+	TAG_DAR_R10;						\
 	li	r10,trap;					\
 	stw	r10,TRAP(r11);					\
 	li	r10,MSR_KERNEL;					\

@@ -214,6 +231,7 @@
 	mfspr r5,SPRN_DSISR
 	stw r5,_DSISR(r11)
 	addi r3,r1,STACK_FRAME_OVERHEAD
+	TAG_DAR_R10
 	EXC_XFER_STD(0x200, MachineCheckException)
 
 /* Data access exception.

@@ -227,6 +245,7 @@
 	stw	r10,_DSISR(r11)
 	mr	r5,r10
 	mfspr	r4,SPRN_DAR
+	TAG_DAR_R10
 	EXC_XFER_EE_LITE(0x300, handle_page_fault)
 
 /* Instruction access exception.

@@ -252,6 +271,7 @@
 	mfspr	r5,SPRN_DSISR
 	stw	r5,_DSISR(r11)
 	addi	r3,r1,STACK_FRAME_OVERHEAD
+	TAG_DAR_R10
 	EXC_XFER_EE(0x600, AlignmentException)
 
 /* Program check exception */

@@ -414,7 +434,13 @@
 	rlwimi	r10, r11, 0, 24, 28	/* Set 24-27, clear 28 */
 	DO_8xx_CPU6(0x3d80, r3)
 	mtspr	SPRN_MD_RPN, r10	/* Update TLB entry */
-
+#ifdef CONFIG_8xx_DCBxFIXED
+ #if TAG_VAL == 0x00f0 /* Save 1 instr. by reusing the val loaded in r11 above */
+ 	mtspr	SPRN_DAR, r11
+ #else
+ 	TAG_DAR_R10
+ #endif
+#endif
 	mfspr	r10, SPRN_M_TW	/* Restore registers */
 	lwz	r11, 0(r0)
 	mtcr	r11

@@ -450,11 +476,20 @@
 	mfcr	r10
 	stw	r10, 0(r0)
 	stw	r11, 4(r0)
+ 	mfspr	r10, SPRN_DAR
+#ifdef  CONFIG_8xx_DCBxFIXED
+	/* If DAR contains TAG_VAL implies a buggy dcbx instruction
+	 * that did not set DAR.
+	 */
+	cmpwi	cr0, r10, TAG_VAL
+	beq-	100f	/* Branch if TAG_VAL to dcbx workaround procedure */
+101:	/* return from dcbx instruction bug workaround, r10 holds value of DAR */	
+#endif
 
 	/* First, make sure this was a store operation.
 	*/
-	mfspr	r10, SPRN_DSISR
-	andis.	r11, r10, 0x0200	/* If set, indicates store op */
+	mfspr	r11, SPRN_DSISR
+	andis.	r11, r11, 0x0200	/* If set, indicates store op */
 	beq	2f
 
 	/* The EA of a data TLB miss is automatically stored in the MD_EPN

@@ -473,7 +508,7 @@
 	 * are initialized in mapin_ram().  This will avoid the problem,
 	 * assuming we only use the dcbi instruction on kernel addresses.
 	 */
-	mfspr	r10, SPRN_DAR
+	/* DAR is in r10 already */
 	rlwinm	r11, r10, 0, 0, 19
 	ori	r11, r11, MD_EVALID
 	mfspr	r10, SPRN_M_CASID

@@ -523,7 +558,13 @@
 	rlwimi	r10, r11, 0, 24, 28	/* Set 24-27, clear 28 */
 	DO_8xx_CPU6(0x3d80, r3)
 	mtspr	SPRN_MD_RPN, r10	/* Update TLB entry */
-
+#ifdef CONFIG_8xx_DCBxFIXED
+ #if TAG_VAL == 0x00f0 /* Save 1 instr. by reusing the val loaded in r11 above */
+	mtspr	SPRN_DAR, r11
+ #else
+	TAG_DAR_R10
+ #endif
+#endif
 	mfspr	r10, SPRN_M_TW	/* Restore registers */
 	lwz	r11, 0(r0)
 	mtcr	r11

@@ -561,6 +602,185 @@
 
 	. = 0x2000
 
+#ifdef CONFIG_8xx_DCBxFIXED
+/* This is the workaround procedure to calculate the data EA for buggy dcbx,dcbi instructions
+ * by decoding the registers used by the dcbx instruction and adding them.
+ * DAR is set to the calculated address and r10 also holds the EA on exit.
+ */
+//#define INSTR_CHECK /* define to verify if it is a dcbx instr. Should not be needed. */
+//#define NO_SELF_MODIFYING_CODE /* define if you don't want to use self modifying code */
+//#define DEBUG_DCBX_INSTRUCTIONS /* for debugging only. Needs INSTR_CHECK defined as well. */
+//#define KERNEL_SPACE_ONLY /* define if user space do NOT contain dcbx instructions. */
+
+#ifndef KERNEL_SPACE_ONLY
+	nop	/* A few nops to make the modified_instr: space below cache line aligned */
+	nop
+139:	/* fetch instruction from userspace memory */
+	DO_8xx_CPU6(0x3780, r3)
+	mtspr	SPRN_MD_EPN, r10
+	mfspr	r11, SPRN_M_TWB	/* Get level 1 table entry address */
+	lwz	r11, 0(r11)	/* Get the level 1 entry */
+	DO_8xx_CPU6(0x3b80, r3)
+	mtspr	SPRN_MD_TWC, r11	/* Load pte table base address */
+	mfspr	r11, SPRN_MD_TWC	/* ....and get the pte address */
+	lwz	r11, 0(r11)	/* Get the pte */
+	/* concat physical page address(r11) and page offset(r10) */
+	rlwimi	r11, r10, 0, 20, 31
+	b	140f
+#endif
+100:	/* Entry point for dcbx workaround. */
+	/* fetch instruction from memory. */
+	mfspr	r10,SPRN_SRR0
+#ifndef KERNEL_SPACE_ONLY
+	andis.	r11, r10, 0x8000
+	tophys  (r11, r10)
+	beq-	139b		/* Branch if user space address */
+#else
+	tophys  (r11, r10)
+#endif
+140:	lwz	r11,0(r11)
+#ifdef INSTR_CHECK
+/* Check if it really is a dcbx instruction. This is not needed as far as I can tell */
+/* dcbt and dcbtst does not generate DTLB Misses/Errors, no need to include them here */
+	rlwinm	r10, r11, 0, 21, 30
+	cmpwi	cr0, r10, 2028	/* Is dcbz? */
+	beq+	142f
+	cmpwi	cr0, r10, 940	/* Is dcbi? */
+	beq+	142f
+	cmpwi	cr0, r10, 108	/* Is dcbst? */
+	beq+	142f
+	cmpwi	cr0, r10, 172	/* Is dcbf? */
+	beq+	142f
+	cmpwi	cr0, r10, 1964	/* Is icbi? */
+	beq+	142f
+#ifdef DEBUG_DCBX_INSTRUCTIONS
+141:	b 141b /* Stop here if no dcbx instruction */
+#endif
+	mfspr	r10, SPRN_DAR	/* r10 must hold DAR at exit */
+	b 101b			/* None of the above, go back to normal TLB processing */
+142:	/* continue, it was a dcbx instruction. */
+#endif
+#ifdef CONFIG_8xx_CPU6
+	lwz	r3, 8(r0)		/* restore r3 from memory */
+#endif
+#ifndef NO_SELF_MODIFYING_CODE
+	andis.	r10,r11,0x1f	/* test if reg RA is r0 */
+	li	r10,modified_instr@l
+	dcbtst	r0,r10		/* touch for store */
+	rlwinm	r11,r11,0,0,20	/* Zero lower 10 bits */
+	oris	r11,r11,640	/* Transform instr. to a "add r10,RA,RB" */
+	ori	r11,r11,532
+	stw	r11,0(r10)	/* store add/and instruction */
+	dcbf	0,r10		/* flush new instr. to memory. */
+	icbi	0,r10		/* invalidate instr. cache line */
+	lwz	r11, 4(r0)	/* restore r11 from memory */
+	mfspr	r10, SPRN_M_TW	/* restore r10 from M_TW */
+	isync			/* Wait until new instr is loaded from memory */
+modified_instr:
+	.space	4		/* this is where the add/and instr. is stored */
+#ifdef DEBUG_DCBX_INSTRUCTIONS
+	/* fill with some garbage */ 
+	li	r11,0xffff
+	stw	r11,0(r11)
+#endif
+	bne+	143f
+	subf	r10,r0,r10		/* r10=r10-r0, only if reg RA is r0 */
+143:	mtdar	r10			/* store faulting EA in DAR */
+	b	101b			/* Go back to normal TLB handling */
+#else
+	mfctr	r10
+	mtdar	r10			/* save ctr reg in DAR */
+	rlwinm	r10, r11, 24, 24, 28	/* offset into jump table for reg RB */
+	addi	r10, r10, 150f@l	/* add start of table */
+	mtctr	r10			/* load ctr with jump address */
+	xor	r10, r10, r10		/* sum starts at zero */
+	bctr				/* jump into table */
+150:
+	add	r10, r10, r0
+	b	151f
+	add	r10, r10, r1
+	b	151f
+	add	r10, r10, r2
+	b	151f
+	add	r10, r10, r3
+	b	151f
+	add	r10, r10, r4
+	b	151f
+	add	r10, r10, r5
+	b	151f
+	add	r10, r10, r6
+	b	151f
+	add	r10, r10, r7
+	b	151f
+	add	r10, r10, r8
+	b	151f
+	add	r10, r10, r9
+	b	151f
+	mtctr	r11	/* reg 10 needs special handling */
+	b	154f
+	mtctr	r11	/* reg 11 needs special handling */
+	b	153f
+	add	r10, r10, r12
+	b	151f
+	add	r10, r10, r13
+	b	151f
+	add	r10, r10, r14
+	b	151f
+	add	r10, r10, r15
+	b	151f
+	add	r10, r10, r16
+	b	151f
+	add	r10, r10, r17
+	b	151f
+	add	r10, r10, r18
+	b	151f
+	add	r10, r10, r19
+	b	151f
+	add	r10, r10, r20
+	b	151f
+	add	r10, r10, r21
+	b	151f
+	add	r10, r10, r22
+	b	151f
+	add	r10, r10, r23
+	b	151f
+	add	r10, r10, r24
+	b	151f
+	add	r10, r10, r25
+	b	151f
+	add	r10, r10, r25
+	b	151f
+	add	r10, r10, r27
+	b	151f
+	add	r10, r10, r28
+	b	151f
+	add	r10, r10, r29
+	b	151f
+	add	r10, r10, r30
+	b	151f
+	add	r10, r10, r31
+151:
+	rlwinm. r11,r11,19,24,28	/* offset into jump table for reg RA */
+	beq	152f			/* if reg RA is zero, don't add it */ 
+	addi	r11, r11, 150b@l	/* add start of table */
+	mtctr	r11			/* load ctr with jump address */
+	rlwinm	r11,r11,0,16,10		/* make sure we don't execute this more than once */
+	bctr				/* jump into table */
+152:
+	mfdar	r11
+	mtctr	r11			/* restore ctr reg from DAR */
+	mtdar	r10			/* save fault EA to DAR */
+	b	101b			/* Go back to normal TLB handling */
+
+	/* special handling for r10,r11 since these are modified already */
+153:	lwz	r11, 4(r0)	/* load r11 from memory */
+	b	155f
+154:	mfspr	r11, SPRN_M_TW	/* load r10 from M_TW */
+155:	add	r10, r10, r11	/* add it */
+	mfctr	r11		/* restore r11 */
+	b	151b
+#endif
+#endif
 	.globl	giveup_fpu
 giveup_fpu:
 	blr
#12 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Marcelo Tosatti writes:

Yuli> [...deleted...]

Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any Jason> embedded fixes in the mainline. Last I checked, that was for Jason> 'the tools vendors' to fix.

Jason> "We won't work around processor bugs" is their philosophy.

Yuli> [...deleted...]

Yuli> I investigated the problem a bit when I had trouble with a Yuli> self-compiled glibc a year or so ago. IIRC, I found bug in the Yuli> memset code, not in the chip. The code was just wrong for Yuli> cache line sizes not equal to 32. So memset.S is good for 60x Yuli> series (PQII included) but for 8xx it fails.

Marcelo> I suppose you didnt actually use dcbz for userspace memset Marcelo> on 8xx?

Standard glibc did. After the fix, it doesn't do it any more on our systems.

Yuli> We use dcbX instructions in some kernel drivers and since we Yuli> never had any problems with those drivers I'm a bit surprised Yuli> to hear that all 8xx chips have got that bug.

Marcelo> The problem is that the DAR register is correctly unset (it Marcelo> comes as NULL IIRC) on pagefaults for the dcbz Marcelo> instruction. The dcbz instructions you issue are probably Marcelo> always works on kernel addresses whose pagetables are Marcelo> present?

It's not dcbz, it's dcbi/dcbf. And yes, they work on kernel addresses. I never investigated if the page tables are present or not because there were no problems.

Marcelo> Joakim has developed a workaround for the Marcelo> problem... although I promised him several times to test it Marcelo> I never managed to get dcbz to work on the kernel copying Marcelo> functions. :(

[...patch deleted...]

Well, if I manage to find time, I'll try it. No timetables though. I'm not sure if using dcbz in user-space memset is such a great optimisation. It well can be an example of over-engineering.

Signature · 4 lines
========================================================================
 Yuli Barcohen       | Phone +972-9-765-1788 |  Software Project Leader
 [email protected] | Fax   +972-9-765-7494 | Arabella Software, Israel
========================================================================
#13 lore raw

Hi,

It appeard that on my mpc862 target I had also this problem. I am using ELDK3.1.1 which includes a 2.4.25 kernel.
The symptoms however were:
Large application (20MB c++, mostly shared-libs) crashes (sigseg) at
startup BEFORE main() is called.
Attempts to debug the application also fail: the debugger crashes
also (sigseg).
I didn't change the C-lib, I took the 2.4.30/arch/ppc/kernel/head_8xx.S file and copied this over the 2.4.25 head_8xx.S file.
In the 2.4.30 version there is a piece of code mentioning the dcbX instruction problems.

Has anyone an idea why only this large application failed? busybox_1.0 and more applications work fine.
system is a: mcp862/32Mb SDRAM started from u-boot 0.4.1, kernel enhanced with atm/utopia driver.

This driver causes the CPM sometimes to hang when performing a memset, can this be caused by the same problem?
(CPM stops responding, console buffers are not flused anymore and kernel stops waiting for buffers)

Greetings,
Theo Gjaltema

Yuli Barcohen schreef:

Quoted 53 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Marcelo Tosatti writes:

Yuli> [...deleted...]

Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any Jason> embedded fixes in the mainline. Last I checked, that was for Jason> 'the tools vendors' to fix.

Jason> "We won't work around processor bugs" is their philosophy.

Yuli> [...deleted...]

Yuli> I investigated the problem a bit when I had trouble with a Yuli> self-compiled glibc a year or so ago. IIRC, I found bug in the Yuli> memset code, not in the chip. The code was just wrong for Yuli> cache line sizes not equal to 32. So memset.S is good for 60x Yuli> series (PQII included) but for 8xx it fails.

Marcelo> I suppose you didnt actually use dcbz for userspace memset Marcelo> on 8xx?

Standard glibc did. After the fix, it doesn't do it any more on our systems.

Yuli> We use dcbX instructions in some kernel drivers and since we Yuli> never had any problems with those drivers I'm a bit surprised Yuli> to hear that all 8xx chips have got that bug.

Marcelo> The problem is that the DAR register is correctly unset (it Marcelo> comes as NULL IIRC) on pagefaults for the dcbz Marcelo> instruction. The dcbz instructions you issue are probably Marcelo> always works on kernel addresses whose pagetables are Marcelo> present?

It's not dcbz, it's dcbi/dcbf. And yes, they work on kernel addresses. I never investigated if the page tables are present or not because there were no problems.

Marcelo> Joakim has developed a workaround for the Marcelo> problem... although I promised him several times to test it Marcelo> I never managed to get dcbz to work on the kernel copying Marcelo> functions. :(

[...patch deleted...]

Well, if I manage to find time, I'll try it. No timetables though. I'm not sure if using dcbz in user-space memset is such a great optimisation. It well can be an example of over-engineering.

#14 lore raw
Quoted 3 lines · In message <[email protected]> you wrote:

It appeard that on my mpc862 target I had also this problem. I am using ELDK3.1.1 which includes a 2.4.25 kernel.

No, most definitely not.

Quoted 4 lines

Large application (20MB c++, mostly shared-libs) crashes (sigseg) at
startup BEFORE main() is called.
Attempts to debug the application also fail: the debugger crashes
also (sigseg).

I bet that this is a FAQ. Your SDRAM initialization is broken.

Quoted 4 lines

I didn't change the C-lib, I took the 2.4.30/arch/ppc/kernel/head_8xx.S file and copied this over the 2.4.25 head_8xx.S file.
In the 2.4.30 version there is a piece of code mentioning the dcbX instruction problems.

And did this fix the problem completely? I would be a bit surprised.

Quoted 2 lines

This driver causes the CPM sometimes to hang when performing a memset, can this be caused by the same problem?

Yes. When your SDRAM init is broken, strange things will happen.

Best regards,

Wolfgang Denk

Signature · 3 lines
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected]
Work 8 hours, sleep 8 hours; but not the same 8 hours.
#15 lore raw

Wolfgang Denk schreef:

Quoted 12 lines
Quoted 4 lines · In message <[email protected]> you wrote:

It appeard that on my mpc862 target I had also this problem. I am using ELDK3.1.1 which includes a 2.4.25 kernel.

No, most definitely not.

I don't understand this, on /pub/Linux/distributions/eldk/3.1.1/ppc-linux-x86/distribution/common there are the sources of a 2.4.25 kernel.

Quoted 10 lines
Quoted 6 lines

Large application (20MB c++, mostly shared-libs) crashes (sigseg) at
startup BEFORE main() is called.
Attempts to debug the application also fail: the debugger crashes
also (sigseg).

I bet that this is a FAQ. Your SDRAM initialization is broken.

This was one of my guesses also. Since this is beyond my knowledge, I had the parametes initialisation parameters compared to an other platform which did not had the problems (rtos based), the same parameters were used. I did not investigate it any further.

Quoted 13 lines
Quoted 8 lines

I didn't change the C-lib, I took the 2.4.30/arch/ppc/kernel/head_8xx.S file and copied this over the 2.4.25 head_8xx.S file.
In the 2.4.30 version there is a piece of code mentioning the dcbX instruction problems.

And did this fix the problem completely? I would be a bit surprised.

No problem since.

Quoted 9 lines
Quoted 4 lines

This driver causes the CPM sometimes to hang when performing a memset, can this be caused by the same problem?

Yes. When your SDRAM init is broken, strange things will happen.

I'll try the kernel with the original atm/utopia to see what happens.

Quoted 6 lines

Best regards,

Wolfgang Denk

Thanks,
Theo Gjaltema.

#16 lore raw
Quoted 9 lines · In message <[email protected]> you wrote:
Quoted 6 lines
Quoted 3 lines

It appeard that on my mpc862 target I had also this problem. I am using ELDK3.1.1 which includes a 2.4.25 kernel.

No, most definitely not.

I don't understand this, on /pub/Linux/distributions/eldk/3.1.1/ppc-linux-x86/distribution/common there are the sources of a 2.4.25 kernel.

Yes, sure. But most definitely you did not have *this* problem. You have a *different* one.

Quoted 6 lines
Quoted 2 lines

I bet that this is a FAQ. Your SDRAM initialization is broken.

This was one of my guesses also. Since this is beyond my knowledge, I had the parametes initialisation parameters compared to an other platform which did not had the problems (rtos based), the same parameters were used. I did not investigate it any further.

This *is* a FAQ. Please go and read the documentation. See http://www.denx.de/twiki/bin/view/DULG/UBootCrashAfterRelocation

Best regards,

Wolfgang Denk

Signature · 6 lines
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected]
"Obviously, a major malfunction has occurred."
              -- Steve Nesbitt, voice of Mission Control, January 28,
                 1986, as the shuttle Challenger exploded within view
                 of the grandstands.
#17 lore raw
In reply to Theo Gjaltema
Quoted 5 lines · Theo Gjaltema

Has anyone an idea why only this large application failed? busybox_1.0 and more applications work fine.
system is a: mcp862/32Mb SDRAM started from u-boot 0.4.1, kernel enhanced with atm/utopia driver.

Quoted 8 lines

This driver causes the CPM sometimes to hang when performing a memset, can this be caused by the same problem?
(CPM stops responding, console buffers are not flused anymore and kernel stops waiting for buffers)

Greetings,
Theo Gjaltema

the problem ( so i think ) is, that larger applications will need more space, which in turn is satisfied by malloc (or mmap with fd = -1). these allocated pages are not real initialized after malloc ( they will be initialized completely, if a write on it is done, which will cause the page fault handler, that does the hardware mmu-stuff, so that a write can be done). so they will cause a page fault on a first access. no problem so long, but when the memset.S routine in libc tries to set all to zero, it uses the dcbz instructions. this instruction then causes a page fault or better a TLB miss (if i'm right). after the exception is raised, the address where the write to was done is looked up in the DAR (data address register), which could be bogus on the dcb* instruction (on 8xx, where is the errata for that?). to sum up, the chance, that a memset( .., '\0', ..) with dcbz is done on a page (that has just registered structures in the kernel, and not cached in the tlb or even marked as writeable) is bigger :)

that's my opinion. if someone has more knowledge about that, or if i'm wrong - please! correct me.

anton

#18 lore raw

ptrace on linux 2.6.12 causes oops

In reply to Marcelo Tosatti

Hello

when i try to run strace or gdbserver on a program, the following comes:

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: 0300 Not

tainted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK = c0ea8430[761] 'gdbserver' THREAD: c0f34000
Last syscall: 26
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF 00F58000 00000001
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 01000800 00000001
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 C0839300 C01E0000
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 C02ACB00 C02ACB00
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30

Call trace:
[c000b154] update_mmu_cache+0x7c/0xa4
[c005ae98] do_wp_page+0x460/0x5ec
[c005c8a0] handle_mm_fault+0x7cc/0x91c
[c005ccec] get_user_pages+0x2fc/0x65c
[c0027104] access_process_vm+0x9c/0x1d4
[c00076e0] sys_ptrace+0x240/0x4a4
[c0002bd0] ret_from_syscall+0x0/0x44

mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by mm/memory.c/1306

and strace or gdbserver of course says segmentation fault. with gdbserver, this happens every time. with strace, the first time it works nearly all time, but when i strace a second time program again, it segfaults. i think the access_process_vm is accessed trough PEEKDATA and PEEKTEXT in sys_ptrace. so here some more debug :) :

DEBUG: peekdata @ 1006d4ec
DEBUG: peektext @ 1006d4f0
DEBUG: peekdata @ 1006d4f0
DEBUG: peektext @ 1006d4f4
DEBUG: peekdata @ 1006d4f4
DEBUG: peektext @ 1006d4f8
DEBUG: peekdata @ 1006d4f8
DEBUG: peektext @ 1006d4fc
DEBUG: peekdata @ 1006d4fc
DEBUG: peektext @ 1006d500
DEBUG: peekdata @ 1006d500
DEBUG: peektext @ 1006d504
DEBUG: peekdata @ 1006d504
DEBUG: peektext @ 1006d508
DEBUG: peekdata @ 1006d508
DEBUG: peektext @ 1006d50c
DEBUG: peekdata @ 1006d50c
DEBUG: peektext @ 1006d510
DEBUG: peekdata @ 1006d510
DEBUG: peektext @ 1006d514
DEBUG: peekdata @ 1006d514
DEBUG: peektext @ 1006d518
DEBUG: peekdata @ 1006d518
DEBUG: peektext @ 1006d51c
DEBUG: peekdata @ 1006d51c
DEBUG: peektext @ 1006d520
DEBUG: peekdata @ 1006d520
DEBUG: peektext @ 1006d524
DEBUG: peekdata @ 1006d524
DEBUG: peektext @ 1006d528
DEBUG: peekdata @ 1006d528
DEBUG: peektext @ 1006d52c
DEBUG: peekdata @ 1006d52c
DEBUG: peektext @ 00000000
DEBUG: peekdata @ 00000000
DEBUG: peektext @ 3000ac18
DEBUG: peekdata @ 3000ac18
DEBUG: peektext @ 3000ac18
DEBUG: peekdata @ 3000ac18
DEBUG: flush_dcache_icache_page

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: 0300 Not

tainted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK = c0ea8430[761] 'gdbserver' THREAD: c0f34000
Last syscall: 26
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF 00F58000 00000001
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 01000800 00000001
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 C0839300 C01E0000
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 C02ACB00 C02ACB00
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30

Call trace:
[c000b154] update_mmu_cache+0x7c/0xa4
[c005ae98] do_wp_page+0x460/0x5ec
[c005c8a0] handle_mm_fault+0x7cc/0x91c
[c005ccec] get_user_pages+0x2fc/0x65c
[c0027104] access_process_vm+0x9c/0x1d4
[c00076e0] sys_ptrace+0x240/0x4a4
[c0002bd0] ret_from_syscall+0x0/0x44

mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by mm/memory.c/1306

#19 lore raw

Re: ptrace on linux 2.6.12 causes oops

What system is this on?

- kumar

Quoted 113 lines · On Jul 14, 2005, at 3:23 AM, Anton W=F6llert wrote:

Hello

when i try to run strace or gdbserver on a program, the following =20 comes:

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: =20

0300 Not tainted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK =3D c0ea8430[761] 'gdbserver' THREAD: c0f34000 Last syscall: 26
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF =20 00F58000 00000001
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 =20 01000800 00000001
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 =20 C0839300 C01E0000
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 =20 C02ACB00 C02ACB00
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30

Call trace:
 [c000b154] update_mmu_cache+0x7c/0xa4
 [c005ae98] do_wp_page+0x460/0x5ec
 [c005c8a0] handle_mm_fault+0x7cc/0x91c
 [c005ccec] get_user_pages+0x2fc/0x65c
 [c0027104] access_process_vm+0x9c/0x1d4
 [c00076e0] sys_ptrace+0x240/0x4a4
 [c0002bd0] ret_from_syscall+0x0/0x44

mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked =20 by mm/memory.c/1306

and strace or gdbserver of course says segmentation fault. with =20 gdbserver, this happens every time. with strace, the first time it =20 works nearly all time, but when i strace a second time program =20 again, it segfaults. i think the access_process_vm is accessed =20 trough PEEKDATA and PEEKTEXT in sys_ptrace. so here some more =20 debug :) :

DEBUG: peekdata @ 1006d4ec
DEBUG: peektext @ 1006d4f0
DEBUG: peekdata @ 1006d4f0
DEBUG: peektext @ 1006d4f4
DEBUG: peekdata @ 1006d4f4
DEBUG: peektext @ 1006d4f8
DEBUG: peekdata @ 1006d4f8
DEBUG: peektext @ 1006d4fc
DEBUG: peekdata @ 1006d4fc
DEBUG: peektext @ 1006d500
DEBUG: peekdata @ 1006d500
DEBUG: peektext @ 1006d504
DEBUG: peekdata @ 1006d504
DEBUG: peektext @ 1006d508
DEBUG: peekdata @ 1006d508
DEBUG: peektext @ 1006d50c
DEBUG: peekdata @ 1006d50c
DEBUG: peektext @ 1006d510
DEBUG: peekdata @ 1006d510
DEBUG: peektext @ 1006d514
DEBUG: peekdata @ 1006d514
DEBUG: peektext @ 1006d518
DEBUG: peekdata @ 1006d518
DEBUG: peektext @ 1006d51c
DEBUG: peekdata @ 1006d51c
DEBUG: peektext @ 1006d520
DEBUG: peekdata @ 1006d520
DEBUG: peektext @ 1006d524
DEBUG: peekdata @ 1006d524
DEBUG: peektext @ 1006d528
DEBUG: peekdata @ 1006d528
DEBUG: peektext @ 1006d52c
DEBUG: peekdata @ 1006d52c
DEBUG: peektext @ 00000000
DEBUG: peekdata @ 00000000
DEBUG: peektext @ 3000ac18
DEBUG: peekdata @ 3000ac18
DEBUG: peektext @ 3000ac18
DEBUG: peekdata @ 3000ac18
DEBUG: flush_dcache_icache_page

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: =20

0300 Not tainted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK =3D c0ea8430[761] 'gdbserver' THREAD: c0f34000 Last syscall: 26
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF =20 00F58000 00000001
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 =20 01000800 00000001
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 =20 C0839300 C01E0000
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 =20 C02ACB00 C02ACB00
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30

Call trace:
 [c000b154] update_mmu_cache+0x7c/0xa4
 [c005ae98] do_wp_page+0x460/0x5ec
 [c005c8a0] handle_mm_fault+0x7cc/0x91c
 [c005ccec] get_user_pages+0x2fc/0x65c
 [c0027104] access_process_vm+0x9c/0x1d4
 [c00076e0] sys_ptrace+0x240/0x4a4
 [c0002bd0] ret_from_syscall+0x0/0x44

mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked =20 by mm/memory.c/1306

<ATT11632593.txt>

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw

ptrace on linux 2.6.12 causes oops

it's on a tqm850l with mpc850. i've done the following:

in update_mmu_cache:

if (!PageReserved(page)
&& !test_bit(PG_arch_1, &page->flags)) {
// if (vma->vm_mm == current->active_mm){
// _tlbie(address);
// __flush_dcache_icache((void *) address);
// } else
__flush_dcache_icache(page _address(page));
// flush_dcache_icache_page(page);

set_bit(PG_arch_1, &page->flags);
}

like it is in ppc64. now it works. the flush_dcache_icache_page calls __flush_dcache_icache_phys, which temporary turns off the mmu for data-addressing. a bit strange i think. but unfortunately i have too less knowledge about kernel-internal ppc-stuff :(

anton

#23 lore raw

Re: ptrace on linux 2.6.12 causes oops

In reply to Anton Wöllert
Quoted 12 lines

when i try to run strace or gdbserver on a program, the following comes:

NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54 LR [c000b060] flush_dcache_icache_page+0x20/0x30

Call trace:
[c000b154] update_mmu_cache+0x7c/0xa4
[c005ae98] do_wp_page+0x460/0x5ec
[c005c8a0] handle_mm_fault+0x7cc/0x91c
[c005ccec] get_user_pages+0x2fc/0x65c
[c0027104] access_process_vm+0x9c/0x1d4
[c00076e0] sys_ptrace+0x240/0x4a4
[c0002bd0] ret_from_syscall+0x0/0x44
Quoted 2 lines

mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by mm/memory.c/1306

please try to reproduce with 2.6.13-rc2. seems much like the problem Marcelo fixed recently (dcbst misbehaviour with unpopulated TLB entries)

Signature · 1 line
Aristeu
author
#24 lore raw

Re: ptrace on linux 2.6.12 causes oops

Quoted 14 lines · On Thu, Jul 14, 2005 at 05:27:15PM -0300, [email protected] wrote:
Quoted 12 lines

when i try to run strace or gdbserver on a program, the following comes:

NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54 LR [c000b060] flush_dcache_icache_page+0x20/0x30

Call trace:
[c000b154] update_mmu_cache+0x7c/0xa4
[c005ae98] do_wp_page+0x460/0x5ec
[c005c8a0] handle_mm_fault+0x7cc/0x91c
[c005ccec] get_user_pages+0x2fc/0x65c
[c0027104] access_process_vm+0x9c/0x1d4
[c00076e0] sys_ptrace+0x240/0x4a4
[c0002bd0] ret_from_syscall+0x0/0x44
Quoted 2 lines

mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by mm/memory.c/1306

Quoted 2 lines

please try to reproduce with 2.6.13-rc2. seems much like the problem Marcelo fixed recently (dcbst misbehaviour with unpopulated TLB entries)

Yep, just that now its the ptraceing process which is faulting in the page, instead of the (ptraced) process itself.

So Anton, can you move the _tlbie() call up to

                    && !test_bit(PG_arch_1, &page->flags)) {
								<---------- HERE
                        if (vma->vm_mm == current->active_mm)
                                __flush_dcache_icache((void *) address);
                        else
                                flush_dcache_icache_page(page);
                        set_bit(PG_arch_1, &page->flags);

So that it covers both cases instead of just (vma->vm_mm == current->active_mm) ?

Its safe to do it because the address space ID is ignored by tlbie accordingly to the manual page:

The ASID value in the entry is ignored for the purpose of matching an invalidate address, thus multiple entries can be invalidated if they have the same effective address and different ASID values.

#25 lore raw

Re: ptrace on linux 2.6.12 causes oops

Quoted 18 lines

Yep, just that now its the ptraceing process which is faulting in the page,
instead of the (ptraced) process itself.

So Anton, can you move the _tlbie() call up to

&& !test_bit(PG_arch_1, &page->flags)) {
<---------- HERE
if (vma->vm_mm == current->active_mm)
__flush_dcache_icache((void *) address);
else
flush_dcache_icache_page(page);
set_bit(PG_arch_1, &page->flags);

So that it covers both cases instead of just (vma->vm_mm == current->active_mm) ?

Quoted 7 lines

Its safe to do it because the address space ID is ignored by tlbie accordingly
to the manual page:

The ASID value in the entry is ignored for the purpose of matching an invalidate address, thus multiple entries can be invalidated if they have the same effective address and different ASID values.

Well, unfortunately, that doesn't work :(. If i'm right, the __flush_dcache_icache((void *) address) should avoid that the cache says faulting address again.
The flush_dcache_icache_page(page) should flush the cache, where stands, page not mapped. but the flush_dcache_icache_page(page) oopses on my system. but instead of this call, the call __flush_dcache_icache(page_address(page)) works. for me, that also makes more sence. and also, the flush_dcache_icache_page(page) calls the flush_dcache_icache_phys, which turns off the data virtual address mapping. i found that a bit strange. any comments?

author
#26 lore raw

Re: ptrace on linux 2.6.12 causes oops

Hi Anton,

Quoted 2 lines · On Fri, Jul 15, 2005 at 11:42:27AM +0200, Anton W=F6llert wrote:
Quoted 2 lines

=20
Yep, just that now its the ptraceing process which is faulting in the=

=20

Quoted 14 lines
Quoted 14 lines

page,
instead of the (ptraced) process itself.
=20
So Anton, can you move the _tlbie() call up to
=20
&& !test_bit(PG_arch_1, &page->flags)) {
<---------- HERE
if (vma->vm_mm =3D=3D current->active_mm) __flush_dcache_icache((void *) address);
else
flush_dcache_icache_page(page);
set_bit(PG_arch_1, &page->flags);
=20
So that it covers both cases instead of just (vma->vm_mm =3D=3D=20

Quoted 8 lines
Quoted 8 lines

current->active_mm) ?
=20
Its safe to do it because the address space ID is ignored by tlbie=20 accordingly
to the manual page:
=20
The ASID value in the entry is ignored for the purpose of matching an invalidate address, thus multiple entries can be invalida=

ted

Quoted 5 lines
Quoted 1 line

if they have the same effective address and different ASID values.

=20
=20
=20
Well, unfortunately, that doesn't work :(.=20

Oh doh, on the case where the process faulting in the page is not the own= er of the
vma (which is the case with ptrace here), the flushing is done via the ph= ysical address.

So its expected that the _tlbie() on the process virtual address will not=
change=20
a thing, since flush_dcache_icache_page() is working on the physical addr= ess.

Quoted 2 lines

If i'm right, the=20
__flush_dcache_icache((void *) address) should avoid that the cache say=

s=20

Quoted 2 lines

faulting address again.
The flush_dcache_icache_page(page) should flush the cache, where stands=

,=20

Quoted 1 line

page not mapped. but the flush_dcache_icache_page(page) oopses on my sy=

stem.=20

Quoted 1 line

but instead of this call, the call __flush_dcache_icache(page_address(p=

age))=20

Quoted 1 line

works. for me, that also makes more sence.=20

Well, you end up flushing the page through its kernel virtual address map= ping (returned
by page_address()).=20

It will be necessary for the kernel virtual map to be faulted in the TLB,=
which is=20
certainly slower than doing the direct physical address flush (requires u= pdating=20
the MSR twice (+isync) to turn the MMU on/off).

But other than that I see no problem with your suggestion really, as to w= orkaround the=20
oops.

Still, we should try to understand what is going on...

Quoted 1 line

and also, the flush_dcache_icache_page(page) calls the flush_dcache_ica=

che_phys, which=20

Quoted 1 line

turns off the data virtual address mapping. i found that a bit strange.=

any=20

Quoted 1 line

comments?=20

It seems that the instruction at "__flush_dcache_icache_phys+0x38" is icb= i - can you send=20
the disassembly of __flush_dcache_icache_phys? =20

We should figure out what is causing the oops.=20

What is causing DAR to be set to "00000010"? ie why the hell is it trying=
to access=20
the 00000010 address.

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: 0300 Not tai=

nted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK =3D c0ea8430[761] 'gdbserver' THREAD: c0f34000
Last syscall: 26
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF 00F58000 00000001
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 01000800 00000001
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 C0839300 C01E0000
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 C02ACB00 C02ACB00
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30

#27 lore raw
In reply to Marcelo Tosatti
Quoted 5 lines · Marcelo Tosatti

Hi Anton,

(moving to ppc-embedded since it might be of interesting for other 8xx users)

Quoted 7 lines

On Thu, Jun 30, 2005 at 04:13:30PM +0200, Anton Wöllert wrote:

Can you please examine in more detail what is going on here?

What are the contents of the r0 and r6 registers, where exactly is the task stopped.

with backtrace in gdb i found out, that memset() (which is exactly the routine at sysdeps/powerpc/powerpc32/memset.S) called by _dl_allocate_tls_storage (sysdeps/generic/dl-tls.c). btw. memset is called 3 times before and there it works fine.
well, _dl_allocate_tls_storage invokes __libc_memalign (elf/dl-minimal.c). __libc_memalign should return an aligned area of memory for the tls-stuff (which later then will be memset'd with zero).
__libc_memalign now allocates an aligned memory block, but first looks if there is unused space in the previous page of its data-segment. if there is some, it will use that. if more than this is needed, it will call mmap with MAP_ANON|MAP_PRIVATE and PROT_READ|PROT_WRITE. it then looks, if the returned address lies at the end of the old page, if not, the old one will not be used (because then it wouldn't return an continuegous area of memory). will gdb, i saw, that mmap returns a address right after the previous page and so the block continuegous. this block is now returned and given to memset.
memset now zeros out a part of the returned block. as long, as dcbz zeros out the memory, that lies in the old page (before 0x30001000) it works, but when it steps over that address (the page boundary), stepi in gdb just hangs and doesn't return anything ( except i press ctrl-c ).
i think, dcbz should cause a page fault, because the new page was allocated with PROT_READ|PROT_WRITE. i expect, that there is no handling for dcbz's in the page fault handler (which should be do_page_fault in arch/ppc/kernel/fault.c if i'm right).
other possibilities, like false cache_line_size etc should explain that,
because they are all set right (with gdb i inspected nearly
everything, and memset should work as expected, and it does so in other scenarios).

sorry for not delivering the registers and so on, but actually i have no access to the board for one week :). i will post them later, if really needed, but throug my gdb-sessions, they seemed all to be right.

Quoted 4 lines

I _guess_ it might be some form of "dcbz" misbehaviour, in such case I imagine that either the task would loop forever or execute an invalid exception/instruction.

so what should be the real solution, deleting memset.S because the use of dcbz is not allowed or eventually fix the page-fault handler, because he didn't recognize dcbz's (if i'm right).

thanks for your help, anton