2.6.14-rc3-rt10 crashes on boot
Subsystems (2)
Now that rt kernels are compiling again on x86_64 I'm getting a recurrence of a boot problem I had with 2.6.13-rt4 on a dual Opteron. If I enable latency tracing it crashes during boot, but if I use exactly the same .config except with latency tracing disabled it boots and runs fine. Every version of rt I've managed to compile since then has crashed in the same place with latency tracing enabled.
Below are excerpts from .config and from boot messages via serial console.
________________________
CONFIG_SMP=yCONFIG_PREEMPT_RT=yCONFIG_PREEMPT=yCONFIG_PREEMPT_SOFTIRQS=yCONFIG_PREEMPT_HARDIRQS=yCONFIG_PREEMPT_BKL=yCONFIG_PREEMPT_RCU=yCONFIG_RWSEM_GENERIC_SPINLOCK=y
#
# Kernel hacking
#
# CONFIG_PRINTK_TIME is not set
# CONFIG_PRINTK_IGNORE_LOGLEVEL is not setCONFIG_DEBUG_KERNEL=yCONFIG_MAGIC_SYSRQ=yCONFIG_LOG_BUF_SHIFT=15CONFIG_DETECT_SOFTLOCKUP=yCONFIG_SCHEDSTATS=y
# CONFIG_DEBUG_SLAB is not setCONFIG_DEBUG_PREEMPT=yCONFIG_DEBUG_IRQ_FLAGS=yCONFIG_WAKEUP_TIMING=yCONFIG_WAKEUP_LATENCY_HIST=yCONFIG_PREEMPT_TRACE=y
# CONFIG_CRITICAL_PREEMPT_TIMING is not set
# CONFIG_CRITICAL_IRQSOFF_TIMING is not setCONFIG_LATENCY_TIMING=yCONFIG_LATENCY_HIST=yCONFIG_LATENCY_TRACE=yCONFIG_MCOUNT=yCONFIG_RT_DEADLOCK_DETECT=y
# CONFIG_DEBUG_RT_LOCKING_MODE is not set
# CONFIG_DEBUG_KOBJECT is not setCONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_FS is not setCONFIG_FRAME_POINTER=y
# CONFIG_INIT_DEBUG is not set
# CONFIG_IOMMU_DEBUG is not set
# CONFIG_KPROBES is not set
______________________________
<snip>
Bootdata ok (command line is root=/dev/hda7 ro console=tty0 console=ttyS0,38400 )
Linux version 2.6.14-rc3-rt10-mindriv-debug-amd64-k8-smp (jqr@mj0lnir) (gcc version 4.0.2 20050917 (prerelease) (Debian 4.0.1-8)) #1 SMP PREEMPT Thu Oct 6 22:43:49 UTC 2005 </snip>
...
<snip>
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hdd: _NEC DVD_RW ND-3520A, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
hda: max request size: 128KiB
hda: 80293248 sectors (41110 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(133)
hda: cache flushes supported
hda: hda1 hda2 < hda5 hda6 hda7 >
umount: devfs: not mounted
mount: unknown filesystem type 'devfs'
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
umount: devfs: not mounted
INIT: version 2.86 booting
hotplug[877]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffff8bee68 error 15 hotplug[878]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffffb1a408 error 15 hotplug[879]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffff878408 error 15 hotplug[880]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffffad36d8 error 15 init[1]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffffc00b10 error 15 init[1]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffffc003b8 error 15 rcS[882]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffff967428 error 15 init[1]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffffc003b8 error 15 init[1]: segfault at ffffffff8010f588 rip ffffffff8010f588 rsp 00007fffffc003b8 error 15 </snip>
After this point the same error message is repeated in an infinite loop and a hard reboot is required.
If I get time in the next few days I'll try and find the earliest version of rt that does this (haven't tried anything earlier than 2.6.13-rt4).
John
Quoted 2 lines · John Rigg
Below are excerpts from .config and from boot messages via serial console.
Hmm, I wonder if you're getting a stack overflow?
Have you tried this with turning on CONFIG_DEBUG_STACKOVERFLOW?
-- Steve
Quoted 8 lines · Steven Rostedt
Quoted 2 lines · John Rigg
Below are excerpts from .config and from boot messages via serial console.
Hmm, I wonder if you're getting a stack overflow?
Have you tried this with turning on
CONFIG_DEBUG_STACKOVERFLOW?
i got overflows in initramfs's gunzip with certain debug options. I have improved the stack footprint of the worst offenders in -rt11 (see the standalone patch below) - John, does it boot any better?
Ingo
-------
this patch reduces the ~2500+ worst-case stack footprint of zlib to ~500 bytes, by making the largest arrays static and by introducing a spinlock to protect access to them.
arch/arm/boot/compressed/misc.c | 1 arch/arm26/boot/compressed/misc.c | 1 arch/i386/boot/compressed/misc.c | 1 arch/x86_64/boot/compressed/misc.c | 1 lib/inflate.c | 44 +++++++++++++++++++++++++++--------- lib/zlib_inflate/inftrees.c | 45 ++++++++++++++++++++++++++++--------- 6 files changed, 72 insertions(+), 21 deletions(-)
arch/arm/boot/compressed/misc.c +1 -0
@@ -199,6 +199,7 @@ static ulg free_mem_ptr_end;#define HEAP_SIZE 0x2000 +#define ZLIB_INFLATE_NO_INFLATE_LOCK #include "../../../../lib/inflate.c" #ifndef STANDALONE_DEBUG
arch/arm26/boot/compressed/misc.c +1 -0
@@ -184,6 +184,7 @@ static ulg free_mem_ptr_end;#define HEAP_SIZE 0x2000 +#define ZLIB_INFLATE_NO_INFLATE_LOCK #include "../../../../lib/inflate.c" #ifndef STANDALONE_DEBUG
arch/i386/boot/compressed/misc.c +1 -0
@@ -125,6 +125,7 @@ static int lines, cols;static void * xquad_portio =NULL ;#endif +#define ZLIB_INFLATE_NO_INFLATE_LOCK #include "../../../../lib/inflate.c" static void *malloc (int size)
arch/x86_64/boot/compressed/misc.c +1 -0
@@ -114,6 +114,7 @@ static char *vidmem = (char *)0xb8000;static int vidport;static int lines, cols; +#define ZLIB_INFLATE_NO_INFLATE_LOCK #include "../../../../lib/inflate.c" static void *malloc (int size)
lib/inflate.c +33 -11
85 lines not on this page · raw message
lib/zlib_inflate/inftrees.c +35 -10
110 lines not on this page · raw message