Skip to contentSkip to conversation

gsyprf11 and 2.6.13-rc3-pa1

parisc-linux John David Anglin 4 participants 10 messages Aug 13, 2005 – Aug 16, 2005 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 8 lines · John David Anglin
Quoted 6 lines

ldw instruction at 101068ac has executed. The sequence of insns starting at 101068ac is the beginning of the epilogue for handle_interruption (register restores and return). Thus, handle_interruption is going to return to 101068ac. I don't know but the insn at 101068ac might be generating TLB miss faults.

Quoted 10 lines

Yes, I think we said almost the same thing. You are describing the case where we return from the fault and I'm describing the case where 101068ac causes the initial fault while handling a fault. Sounds like %sp was corrupted since references to %sp shouldn't ever fault...unless we went off the end of the stack?

I'm getting the impression this discussion should happen on the parisc-linux mailing list.

Quoted 1 line

I will try to set up a testcase.

With 2.6.8.1-pa11, cc1plus generates a SIGSEGV here:

0x00418d98 <ggc_set_mark+0>: stw rp,-14(,sp)

The backtrace is:

(gdb) bt
#0 ggc_set_mark (p=0x404c12a0) at ../../gcc/gcc/ggc-page.c:1254 #1 0x00109ec0 in gt_ggc_mx_lang_tree_node (x_p=0x404c12a0) at gt-cp-tree.h:69 ... [lots more frames]

(gdb) disass gt_ggc_mx_lang_tree_node
Dump of assembler code for function gt_ggc_mx_lang_tree_node:
0x00109e84 <gt_ggc_mx_lang_tree_node+0>: stw rp,-14(,sp)
0x00109e88 <gt_ggc_mx_lang_tree_node+4>: stw,ma r7,80(,sp)

(gdb) p/x $sp
$2 = 0xc0700040

So, the frame allocation by gt_ggc_mx_lang_tree_node appears to span a page boundary and the store into the frame marker by ggc_set_mark causes a data TLB miss and the SIGSEGV. This causes the 2.6.13 kernel to crash in what's apparently an infinite loop involving handle_interruption.

So, I think the bug is that 2.6.13 (32-bit) crashes when we have a data TLB miss for an address outside the allowed bounds for the stack. Probably, any simple program that recursively allocates stack frames until the allowed stack size is exceeded will demonstrate the problem.

The C++ program that causes the system crash when compiled with cc1plus from the GCC head is shown below. Note that there is a change to GCC's inlining in 4.1, and 4.0 doesn't cause the SIGSEGV. My stack limit is the default 8192 kbytes. The c3k kernel is default except I had to disable CONFIG_PDC_STABLE and CONFIG_INFINIBAND.

Dave

Signature · 18 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

// { dg-options "-O2 -ftemplate-depth-20000" }

template <int I>
inline void g() { g<I-1>(); return; }

template <>
inline void g<0>() { int i; return; }

void h() {
  g<250>();
}
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#2 lore raw
Quoted 25 lines · John David Anglin

With 2.6.8.1-pa11, cc1plus generates a SIGSEGV here:

0x00418d98 <ggc_set_mark+0>: stw rp,-14(,sp)

The backtrace is:

(gdb) bt
#0 ggc_set_mark (p=0x404c12a0) at ../../gcc/gcc/ggc-page.c:1254 #1 0x00109ec0 in gt_ggc_mx_lang_tree_node (x_p=0x404c12a0) at gt-cp-tree.h:69 ... [lots more frames]

(gdb) disass gt_ggc_mx_lang_tree_node
Dump of assembler code for function gt_ggc_mx_lang_tree_node:
0x00109e84 <gt_ggc_mx_lang_tree_node+0>: stw rp,-14(,sp)
0x00109e88 <gt_ggc_mx_lang_tree_node+4>: stw,ma r7,80(,sp)

(gdb) p/x $sp
$2 = 0xc0700040

So, the frame allocation by gt_ggc_mx_lang_tree_node appears to span a page boundary and the store into the frame marker by ggc_set_mark causes a data TLB miss and the SIGSEGV. This causes the 2.6.13 kernel to crash in what's apparently an infinite loop involving handle_interruption.

Quoted 4 lines

So, I think the bug is that 2.6.13 (32-bit) crashes when we have a data TLB miss for an address outside the allowed bounds for the stack. Probably, any simple program that recursively allocates stack frames until the allowed stack size is exceeded will demonstrate the problem.

While I believe you are probably right, I could not reproduce this on my c3k here at home:

svenc3k:/# ls -l foo.*
-rw-r--r--  1 root root 180 Aug 13 04:40 foo.cc
-rw-r--r--  1 root root 748 Aug 13 04:42 foo.o
svenc3k:/# g++-4.0 -O2 -ftemplate-depth-20000 -c foo.cc
svenc3k:/# ls -l foo.*
-rw-r--r--  1 root root 180 Aug 13 04:40 foo.cc
-rw-r--r--  1 root root 748 Aug 13 04:44 foo.o
svenc3k:/# ulimit -a
core file size        (blocks, -c) 0
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited
open files                    (-n) 1024
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) 8192
cpu time             (seconds, -t) unlimited
max user processes            (-u) unlimited
virtual memory        (kbytes, -v) unlimited
svenc3k:/#
svenc3k:/# gcc-4.0 -v
Using built-in specs.
Target: hppa-linux-gnu
Configured with: ../src/configure -v --enable-languages=c,c++,java,f95,objc,ada,treelang --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --enable-nls --without-included-gettext --enable-threads=posix --program-suffix=-4.0 --enable-__cxa_atexit --enable-libstdcxx-allocator=mt --enable-clocale=gnu --enable-libstdcxx-debug --enable-java-gc=boehm --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-4.0-1.4.2.0/jre --enable-mpfr --disable-werror --enable-checking=release hppa-linux-gnu
Thread model: posix
gcc version 4.0.2 20050806 (prerelease) (Debian 4.0.1-4)
Quoted 3 lines

The C++ program that causes the system crash when compiled with cc1plus from the GCC head is shown below. Note that there is a change to GCC's inlining in 4.1, and 4.0 doesn't cause the SIGSEGV.

I expect this is the difference...I need to use the same g++ version. Does someone want to hack a small test program that consumes stack space until it bombs out becuase of the ulimit?

Quoted 3 lines

My stack limit is
the default 8192 kbytes. The c3k kernel is default except I had to disable CONFIG_PDC_STABLE and CONFIG_INFINIBAND.

I'm not certain I have a perfectly clean -rc3-pa1 kernel either. I expect I have some PCI changes and the same .config changes you have listed above.

thanks,
grant
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

#3 lore raw
Quoted 2 lines

While I believe you are probably right, I could not reproduce this on my c3k here at home:

One additional detail is the kernel was built with gcc version 4.0.1 (Debian 4.0.1-2).

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#4 lore raw
In reply to Grant Grundler
Quoted 6 lines
svenc3k:/# gcc-4.0 -v
Using built-in specs.
Target: hppa-linux-gnu
Configured with: ../src/configure -v --enable-languages=c,c++,java,f95,objc,ada,treelang --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --enable-nls --without-included-gettext --enable-threads=posix --program-suffix=-4.0 --enable-__cxa_atexit --enable-libstdcxx-allocator=mt --enable-clocale=gnu --enable-libstdcxx-debug --enable-java-gc=boehm --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-4.0-1.4.2.0/jre --enable-mpfr --disable-werror --enable-checking=release hppa-linux-gnu
Thread model: posix
gcc version 4.0.2 20050806 (prerelease) (Debian 4.0.1-4)

I think that i said in the report that it's a change to the inlining in gcc 4.1.0 that triggers this.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#5 lore raw
In reply to Grant Grundler
Quoted 3 lines · Grant Grundler

I expect this is the difference...I need to use the same g++ version. Does someone want to hack a small test program that consumes stack space until it bombs out becuase of the ulimit?

Here's a test program that does this:

#include <stdio.h>

char *stack_base_estimate;

void
recurse(void)

{
	char a[256];

	memset(a, '\0', sizeof(*a));
	printf("Estimating stack consumed 0x%lx\n",
	       (unsigned long)(a - stack_base_estimate));
	recurse();
}

int
main(int argc, char *argv[])

{
	char b;

	stack_base_estimate = &b;
	printf("Estimating stack base at 0x%p\n", stack_base_estimate);
	recurse();
	return 0;
}

But the results are as expected (on 2.6.13-rc6-pa1):

jejb@raven> ulimit -a

core file size        (blocks, -c) 0
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited

open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
jejb@raven> a.out
Estimating stack base at 0x0xc0114548
Estimating stack consumed 0x80
[...]
Estimating stack consumed 0x7fefc0
Estimating stack consumed 0x7ff100
Segmentation fault

James

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

#6 lore raw
Quoted 7 lines · Grant Grundler
Quoted 3 lines

I expect this is the difference...I need to use the same g++ version. Does someone want to hack a small test program that consumes stack space until it bombs out becuase of the ulimit?

Here's a test program that does this:
But the results are as expected (on 2.6.13-rc6-pa1):

It doesn't trigger the fault on my c3k with 2.6.13-rc3-pa1 either.

However,

cc1plus -O2 -ftemplate-depth-20000 inline4.C

consistently crashes my c3k. There must be something more than just running out of stack space. Strangely, the system doesn't crash when cc1plus is run under gdb.

dave@hiauly6:~/gnu/gcc-4.0/objdir/gcc$ ./cc1plus --version
GNU C++ version 4.1.0 20050813 (experimental) (hppa-linux)
        compiled by GNU C version 4.1.0 20050813 (experimental).
GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096

Another TOC gave the following:

r2 0x1010c260 search_exception_tables+0x34
IIA 0x1020c76c search_extable+0x10

I've seen this once before.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#7 lore raw
Quoted 5 lines

However,

cc1plus -O2 -ftemplate-depth-20000 inline4.C

consistently crashes my c3k.

I tried this with builds of 2.6.13-rc6-pa1 with debian gcc-3.3, gcc-3.4 and gcc-4.0. They all crash in the same manner.

I will send cc1plus by separate mail to anyone interested in trying to duplicate this.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#8 lore raw
In reply to James Bottomley

Can somebody with a 32-bit machine try this slightly modified test case? In the presence of a signal handler for SIGSEGV, the program will loop segfaulting because there's not enough stack space left for the signal handler. On an a500, this program just loops, but perhaps on a 32-bit machine it crashes the machine if run long enough. Seems to suggest a problem in our kernel entry/exit paths?

randolph

================================================================

#include <stdio.h>
#include <signal.h>
#include <stdlib.h>

char *stack_base_estimate;

static void sighandler(int sig)

{
        printf("Got signal %d\n", sig);
        exit(0);
}

void
recurse(void)

{
        char a[256];

        memset(a, '\0', sizeof(*a));
        printf("Estimating stack consumed 0x%lx\n",
               (unsigned long)(a - stack_base_estimate));
        recurse();
}

int
main(int argc, char *argv[])

{
        char b;

        signal(SIGSEGV, sighandler);

        stack_base_estimate = &b;
        printf("Estimating stack base at 0x%p\n", stack_base_estimate);
        recurse();
        return 0;
}

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

#9 lore raw
Quoted 4 lines

static void sighandler(int sig)

{
        printf("Got signal %d\n", sig);
        exit(0);

I think you need to use _exit in a signal handler although this may not be an issue when there is only one thread.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#10 lore raw
In reply to a message not in the archive
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Odd that it uses "rp" for the insn at 101068ac and "%r2" for the b,l insn at 101068a4.

I'll look at it. It's a small inconsistency in hppa-dis.c, affecting objdump, etc.

ok. minor problem though.

This is now fixed in binutils head along with a couple of other special cases where we printed '%' before register names.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux