2.4.0 oops bdflush
We have a development SMP machine which runs a myriad of server applications for our development purposes -- Apache, Oracle, several others. Under 2.4.0 the machine locks up, seemingly at random. Usually it simply stops responding without fanfare -- you can, oddly enough, switch consoles with Alt+F?, but typing gets no response and all network services have stopped responding. However, on the most recent failure I was lucky enough to find that it had managed to spit out a kernel oops message before biting it, which I have (hopefully) decoded (properly):
root@fs1:/usr/src/linux.2.4.0# ksymoops -v /usr/src/linux.2.4.0/vmlinux -m \
/usr/src/linux.2.4.0/System.map -o /lib/modules/2.4.0/
ksymoops 2.3.7 on i686 2.2.18. Options used
-v /usr/src/linux.2.4.0/vmlinux (specified)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.0/ (specified)
-m /usr/src/linux.2.4.0/System.map (specified)Error (regular_file): read_ksyms stat /proc/ksyms failed
ksymoops: No such file or directory
No modules in ksyms, skipping objects
No ksyms, skipping lsmod
Reading Oops report from the terminal
invalid operand: 0000
CPU: 0
EIP: 0010:[<c012c37e>]
EFLAGS: 00010296
eax: 0000001c ebx: c1068518 ecx: 00000000 edx: 00000026
esi: c10684fc edi: 0000021c ebp: 00000001 esp: c14f9fa4
ds: 0018 es: 0018 ss: 0018
Process bdflush (pid: 5, stackpage=c14f9000)
Stack: c01f9865 c01f9a24 00000299 c14f8000 c01fb96e 00000000 0008e000 00000000
00000004 00000040 00000000 00000110 c01369c2 00000007 00000000 00010f00
cff93f84 cff93fd0 0008e000 c0107507 cff93fbc cff93fbc cff93fbc
Call Trace: [<c01369c2>] [<c0107507>]
Code: 0f 0b 83 c4 0c 90 8b 46 14 85 c0 75 19 68 99 02 00 00 68 24
invalid operand: 0000
CPU: 0
EIP: 0010:[<c012c37e>]Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010296
eax: 0000001c ebx: c1068518 ecx: 00000000 edx: 00000026
esi: c10684fc edi: 0000021c ebp: 00000001 esp: c14f9fa4
ds: 0018 es: 0018 ss: 0018
Process bdflush (pid: 5, stackpage=c14f9000)
Stack: c01f9865 c01f9a24 00000299 c14f8000 c01fb96e 00000000 0008e000 00000000
00000004 00000040 00000000 00000110 c01369c2 00000007 00000000 00010f00
cff93f84 cff93fd0 0008e000 c0107507 cff93fbc cff93fbc cff93fbc
Call Trace: [<c01369c2>] [<c0107507>]
Code: 0f 0b 83 c4 0c 90 8b 46 14 85 c0 75 19 68 99 02 00 00 68 24Quoted 1 line
Quoted 1 line
EIP;
c012c37e<page_launder+716/868> <=====
Trace; c01369c2 <bdflush+96/dc>
Trace; c0107507 <kernel_thread+23/30>
Code; c012c37e <page_launder+716/868>
00000000 <_EIP>:
Code; c012c37e <page_launder+716/868> <=====
0: 0f 0b ud2a <=====
Code; c012c380 <page_launder+718/868>
2: 83 c4 0c addl $0xc,%esp
Code; c012c383 <page_launder+71b/868>
5: 90 nop
Code; c012c384 <page_launder+71c/868>
6: 8b 46 14 movl 0x14(%esi),%eax
Code; c012c387 <page_launder+71f/868>
9: 85 c0 testl %eax,%eax
Code; c012c389 <page_launder+721/868>
b: 75 19 jne 26 <_EIP+0x26> c012c3a4 <page_launder+73c/868>
Code; c012c38b <page_launder+723/868>
d: 68 99 02 00 00 pushl $0x299
Code; c012c390 <page_launder+728/868>
12: 68 24 00 00 00 pushl $0x24
This machine has been running flawlessly on 2.2.18 for weeks now, which seems to preclude a hardware issue. And since I've been personally running 2.4.0 on my uniprocessor machine since day one without incident, I suspect some bizarre interaction in SMP-land. But I'm hardly a kernel programmer....
Unforunately I can't find exact specs on the machine; it's a Dell Precision 420, most likely built with the hardware du jour about six months ago. The config options used are below:
CONFIG_X86=yCONFIG_ISA=yCONFIG_UID16=yCONFIG_M686FXSR=yCONFIG_X86_WP_WORKS_OK=yCONFIG_X86_INVLPG=yCONFIG_X86_CMPXCHG=yCONFIG_X86_BSWAP=yCONFIG_X86_POPAD_OK=yCONFIG_X86_L1_CACHE_SHIFT=5CONFIG_X86_TSC=yCONFIG_X86_GOOD_APIC=yCONFIG_X86_PGE=yCONFIG_X86_USE_PPRO_CHECKSUM=yCONFIG_X86_FXSR=yCONFIG_X86_XMM=yCONFIG_X86_MSR=yCONFIG_X86_CPUID=yCONFIG_NOHIGHMEM=yCONFIG_MTRR=yCONFIG_SMP=yCONFIG_HAVE_DEC_LOCK=yCONFIG_NET=yCONFIG_X86_IO_APIC=yCONFIG_X86_LOCAL_APIC=yCONFIG_PCI=yCONFIG_PCI_GOANY=yCONFIG_PCI_BIOS=yCONFIG_PCI_DIRECT=yCONFIG_PCI_NAMES=yCONFIG_SYSVIPC=yCONFIG_BSD_PROCESS_ACCT=yCONFIG_SYSCTL=yCONFIG_KCORE_ELF=yCONFIG_BINFMT_AOUT=yCONFIG_BINFMT_ELF=yCONFIG_BINFMT_MISC=yCONFIG_BLK_DEV_FD=yCONFIG_PACKET=yCONFIG_PACKET_MMAP=yCONFIG_NETLINK=yCONFIG_FILTER=yCONFIG_UNIX=yCONFIG_INET=yCONFIG_SYN_COOKIES=yCONFIG_IDE=yCONFIG_BLK_DEV_IDE=yCONFIG_BLK_DEV_IDEDISK=yCONFIG_BLK_DEV_IDECD=yCONFIG_BLK_DEV_IDETAPE=yCONFIG_BLK_DEV_IDEPCI=yCONFIG_IDEPCI_SHARE_IRQ=yCONFIG_BLK_DEV_IDEDMA_PCI=yCONFIG_IDEDMA_PCI_AUTO=yCONFIG_BLK_DEV_IDEDMA=yCONFIG_BLK_DEV_PIIX=yCONFIG_PIIX_TUNING=yCONFIG_IDEDMA_AUTO=yCONFIG_BLK_DEV_IDE_MODES=yCONFIG_NETDEVICES=yCONFIG_NET_ETHERNET=yCONFIG_NET_VENDOR_3COM=yCONFIG_VORTEX=yCONFIG_VT=yCONFIG_VT_CONSOLE=yCONFIG_UNIX98_PTYS=yCONFIG_UNIX98_PTY_COUNT=256CONFIG_MOUSE=yCONFIG_PSMOUSE=yCONFIG_RTC=yCONFIG_QUOTA=yCONFIG_FAT_FS=yCONFIG_MSDOS_FS=yCONFIG_VFAT_FS=yCONFIG_ISO9660_FS=yCONFIG_JOLIET=yCONFIG_MINIX_FS=yCONFIG_PROC_FS=yCONFIG_DEVPTS_FS=yCONFIG_EXT2_FS=yCONFIG_MSDOS_PARTITION=yCONFIG_NLS=yCONFIG_NLS_DEFAULT="iso8859-1"CONFIG_NLS_CODEPAGE_437=yCONFIG_NLS_ISO8859_1=yCONFIG_VGA_CONSOLE=y
I can find out more if necessary. We've gone back to 2.2.18 for now, so I'm sure /proc can divulge useful information, if I knew where to look. -
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/