2.6.11-rc3-pa3 smp "Segmentation fault" but not up

parisc-linux Joel Soete 2 participants 4 messages Jan 15, 2005 – Jan 25, 2005 2 messages not in the archive

lore

Share & tools

lore

Joel Soete

author

2005-01-15 14:02 UTC #0 lore raw

Hi all,

To stress a bit kernel I launch my usual loop test on the n4k (2way, 4Gb ram):

# while true ; do readprofile -r ; make $OPTB clean ; make $OPTB oldconfig; make -j2 $OPTB vmlinux 2>&1 | tee -a /var/logs/k-loop; readprofile >> /var/logs/prof-loop.doc ; done

With previous kernel it makes panic kernel hazardiously from 1 to more 200 occurences. With this 2.6.11-rc3-pa3 after about 49 runs some "Segmentation fault" occures:
make[3]: *** [drivers/pci/probe.o] Segmentation fault
make[3]: *** [drivers/block/ioctl.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/fpudispatch.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/driver.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/driver.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/frnd.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/driver.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/denormal.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/frnd.o] Segmentation fault
make[3]: *** [drivers/md/raid0.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/driver.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/driver.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/frnd.o] Segmentation fault
make[2]: *** [arch/parisc/math-emu/frnd.o] Segmentation fault

for some child's shell (I attached dmesg because very long).

I so stop this loop to reboot with same kernel but up build and relaunch the loop. After about 12h it already runs 80 times without any segmantation fault.

Any idea?

Thanks,
Joel

PS: this last was build with gcc-3.3-hppa64 and binutils-hppa64 while previous was build with hppa64-gcc-3.0 and same binutils

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Matthew Wilcox

2005-01-15 14:41 UTC #1 lore raw

Quoted 5 lines · Joel Soete

With previous kernel it makes panic kernel hazardiously from 1 to more 200 occurences. With this 2.6.11-rc3-pa3 after about 49 runs some "Segmentation fault" occures:
make[3]: *** [drivers/pci/probe.o] Segmentation fault

Quoted 7 lines

for some child's shell (I attached dmesg because very long).
I so stop this loop to reboot with same kernel but up build and relaunch the loop.
After about 12h it already runs 80 times without any segmantation fault.
Any idea?

I suspect the new clear_user_page code. Could you try backing that out and rerunning your test?

Signature · 10 lines

"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

Joel Soete

author

2005-01-16 00:32 UTC #4 lore raw

Hello Grant,

Sorry for delay but the n stand in the office and join it from home i need to use vpn connection which deasable my local lan (and also my pop3 access to my isp :-( and obviously irc)

Quoted 9 lines · Grant Grundler

Quoted 2 lines · Joel Soete
this patch:
<http://cvs.parisc-linux.org/linux-2.6/arch/parisc/kernel/pacache.S?r1=1.9&r2=1.10> ?
I don't think willy is refering to any particular issue.

Quoted 7 lines

Quoted 4 lines
Well in doubt I revert all changes since:
<http://cvs.parisc-linux.org/*checkout*/linux-2.6/arch/parisc/kernel/pacache.S?rev=1.9.20.2>
Btw what do you suspect that Grant (and I) could have missed (just divid by 2 the number of loop thanks to dword insn)?
No - I mangled that. The code you submitted was correct in this regard.

Agree (but what could we imagine on a system which only have one atomic insn iirc?)

Quoted 7 lines

I fixed the off-by-2x bug in the next commit:
http://cvs.parisc-linux.org/linux-2.6/arch/parisc/kernel/pacache.S?r1=1.10&r2=1.11
My suspicion is the prefetching might be causing problems since it's going past the end of the page. The kernel tmp alias pages might be causing problems since we don't flush past the end of those. (Or something like that).

sorry I didn't have yet understand all details of prefetching

Quoted 2 lines

Can you remove only the prefetching in that algorithm?

Yes, I did and already run one occurence of the loop without pb, .. so we have to be patient to see if first mentioned pb (failures after some loop occurence: about 50 last time) :-)

Quoted 9 lines

Quoted 2 lines
But already strange results:
...
Quoted 1 line
and after those first 11 failed it seems to continue??
Do you mean after restarting the make the build completed?

No, just let continue the loop (that doesn't make sense to me but in fact I already encounter this case with 2.6.10-paXX but don't have time to report it).

Is nicol also encounter the same pb (just in case of small diff in hw config)?

Thanks,
Joel
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Joel Soete

author

2005-01-25 13:56 UTC #5 lore raw

Same pb on b2k 64bit + 2.6.11-rc2-pa0 [Was: [parisc-linux] 2.6.11-rc3-pa3 smp "Segmentation fault" but not up]

Hello Grant,

Quoted 1 line

[...]

Quoted 1 line

Quoted 1 line
My suspicion is the prefetching might be causing problems since it's

going

Quoted 1 line

Quoted 1 line
past the end of the page. The kernel tmp alias pages might be causing=

problems

Quoted 8 lines

Quoted 3 lines
since we don't flush past the end of those. (Or something like that).=
sorry I didn't have yet understand all details of prefetching
Quoted 2 lines
Can you remove only the prefetching in that algorithm?
Yes, I did and already run one occurence of the loop without pb, .. so

Quoted 3 lines

have to be patient to see if first mentioned pb (failures after some loop occurence: about 50 last time) :-)

mmm, I was ongoing to forget why the n was looping on a kernel's build wh= en
the b2k panic due to the same pb with lastest 2.6.11-rc2-pa0 (+workaround= ):
patst005 login: Backtrace:

 [<000000001017cd00>] do_wp_page+0x250/0x950
 [<000000001017ee14>] handle_mm_fault+0x1b4/0x1e0
 [<0000000010110ec4>] do_page_fault+0x2bc/0x360
 [<000000001011299c>] handle_interruption+0x34c/0x748
 [<0000000010107094>] intr_check_sig+0x0/0xc

Kernel Fault: Code=3D15 regs=3D000000001d68c540 (Addr=3D0000000020000000)=

YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00001000000001001110011100001111 Not tainted
r00-03 0000000000000000 0000000000000000 000000001017cd00 0000000010a0df= c8
r04-07 00000000105e2020 000000001d874000 0000000010983960 000000000007e2= b0
r08-11 000000001d71cad0 000000001e7e9c00 000000000ffffb25 000000001d4043= f0
r12-15 0000000010983960 000000001d8dc000 0000000000000001 000000000006a4= 04
r16-19 000000001d432740 000000000006a404 000000000006b404 00000000000000= 00
r20-23 0000000000000000 0000000000000000 0000000000000000 00000000000000= 00
r24-27 00000000000000d2 0000000020000000 000000001d875000 00000000105e20= 20
r28-31 0000000010983960 000000001d68c510 000000001d68c540 00000000000000= 20
sr0-3 0000000000429800 0000000000000000 0000000000000000 00000000004298=
00
sr4-7 0000000000000000 0000000000000000 0000000000000000 00000000000000=
00

IASQ: 0000000000000000 0000000000000000 IAOQ: 00000000101012c4 0000000010= 1012c8
IIR: 0f2010d3 ISR: 0000000000000000 IOR: 0000000020000000
CPU: 0 CR30: 000000001d68c000 CR31: 00000000105a0000
ORIG_R28: 00000000103e3e68
IAOQ[0]: copy_user_page_asm+0xa4/0xb0
IAOQ[1]: copy_user_page_asm+0xa8/0xb0
RP(r2): do_wp_page+0x250/0x950

Kernel panic - not syncing: Kernel Fault

I so applied the same work around:
--- arch/parisc/kernel/pacache.S.Orig 2005-01-25 13:44:26.000000000 +01=
00
+++ arch/parisc/kernel/pacache.S 2005-01-25 13:44:55.000000000 +01=
00
@@ -306,13 +306,18 @@

        ldd             0(%r25), %r19           /* bundle 1 */
        ldi             32, %r1                 /* PAGE_SIZE/128 =3D=3D 3=

2 */
+
+#if 0 /* Comment out prefetching awaiting better fix */
ldw 64(%r25), %r0 /* prefetch 1 cacheline a=
head
*/

        ldw             128(%r25), %r0          /* prefetch 2 */

+#endif

1: ldd 8(%r25), %r20
+#if 0 /* Comment out prefetching awaiting better fix */

        ldw             192(%r25), %r0          /* prefetch 3 */
        ldw             256(%r25), %r0          /* prefetch 4 */

nop /* pad to preserve alignment */
+#endif

        ldd             16(%r25), %r21          /* bundle 2 */
        ldd             24(%r25), %r22

=3D=3D=3D=3D=3D=3D=3D=3D=3D><=3D=3D=3D=3D=3D=3D=3D=3D=3D

which help too

(I ran the build kernel loop on the smp n4k during a week: 612 runs amoun= g
111 failed because of various page_fault :-)

Thanks,
Joel

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux