growfiles and other processes in "ps aux" STAT D
Linux Test Project "growfiles" makes a lot of I/O processes hang in with "ps aux" STAT of D. (sometimes requires reboot)
This phenomenon occurs on late 2.4.9-ac1[0-8] kernels.
2.4.10 kernel on another box (laptop with ext3 filesystem, 192M ram) finished with LTP Killed with "errno=12 Cannot allocate memory".
LTP has completed on laptop with 2.4.9-ac17. laptop has only been tested 5 times in total with kernels >= 2.4.9-ac7.
The rest of information below is from LTP on a thunderbird:
Worst Case Symptoms (happened on at least 2 runs of 24 recent tests)
growfiles "hangs" - top and "ps aux" show STAT of D
Many new processes hang in D STAT, vi, ed, logout, su, sync.
syslogd in D STAT.
Other running processes, BitchX, links, vmstat, iostat continue working normally. Some commands work fine: ls, ps
<ctrl \> and kill -9 won't kill processes in D STAT
vmstat shows cpu 100% idle
iostat shows 0 tps and 0 blks/s
load average is around 4 for 1, 5, 15 minutes.
sysreq <Sync reBoot Umount> doesn't work, but sysreq (showMem showTasks) does <ctrl alt del> doesn't reboot.
clear with reset button if necessary.
Last command executed before test had to be terminated. The mmap001 tests did not require reboot.
2.4.9-ac10
mmap001
2.4.9-ac15+aging patch
mmap001
mmap001 -m 10000
These required reboot:
2.4.9-ac17 4 completed, 1 incomplete on:
growfiles -D 0 -b -i 0 -L 60 -u -B 1000b -e 1 -r 512-64000:1024 -R 1-384000 -T 4 gf-inf-$$
2.4.9-ac18 1 incomplete run.
growfiles -D 0 -b -i 0 -L 60 -u -B 1000b -e 1 -r 128-32768:128 -R 512-64000 -T 4 gfsmallio-$$
I started tracking the kernel version/test results recently, because until Sep 23, the tests weren't producing alarming results. (24 total tests since Aug 8 on this machine). The earlier LTP tests were released in April and June of 2001, and I didn't save those results, as there were no surprises.
Distro: Linux From Scratch 3.0
reiserfs on all filesystems.
Hardware:
AMD 1333mhz Thunderbird
IWill KK266 motherboard
512 Mb RAM
40 Gb 7200 rpm 2mb cache IDE disk
00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03) 00:01.0 PCI bridge: VIA Technologies, Inc.: Unknown device 8305 00:07.0 ISA bridge: VIA Technologies, Inc. VT82C686 [Apollo Super] (rev 40) 00:07.1 IDE interface: VIA Technologies, Inc. VT82C586 IDE [Apollo] (rev 06) 00:07.4 Bridge: VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] (rev 40) 00:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139 (rev 10) 00:0f.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10) 01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 04)
hdparm -I /dev/hda
/dev/hda:
Model=CI530L04VARE700- , FwRev=REO44AA1, SerialNo= S 0XXSCL0734
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=40 BuffType=DualPortCache, BuffSize=1916kB, MaxMultSect=16, MultSect=16 CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=80418240 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-5 T13 1321D revision 1 : ATA-2 ATA-3 ATA-4 ATA-5
I removed bogus CONFIG_TOSHIBA and CONFIG*PCMCIA* settings during ac17 (when I started preparing this report. Also removed framebuffer, drm, and power management support.
grep ^CONFIG /usr/src/linux-2.4.9-ac18/.configCONFIG_X86=yCONFIG_ISA=yCONFIG_UID16=yCONFIG_EXPERIMENTAL=yCONFIG_MODULES=yCONFIG_KMOD=yCONFIG_M586=yCONFIG_X86_WP_WORKS_OK=yCONFIG_X86_INVLPG=yCONFIG_X86_CMPXCHG=yCONFIG_X86_XADD=yCONFIG_X86_BSWAP=yCONFIG_X86_POPAD_OK=yCONFIG_RWSEM_XCHGADD_ALGORITHM=yCONFIG_X86_L1_CACHE_SHIFT=5CONFIG_X86_USE_STRING_486=yCONFIG_X86_ALIGNMENT_16=yCONFIG_NOHIGHMEM=yCONFIG_MTRR=yCONFIG_NET=yCONFIG_PCI=yCONFIG_PCI_GOANY=yCONFIG_PCI_BIOS=yCONFIG_PCI_DIRECT=yCONFIG_PCI_NAMES=yCONFIG_SYSVIPC=yCONFIG_SYSCTL=yCONFIG_KCORE_ELF=yCONFIG_BINFMT_ELF=yCONFIG_PNP=yCONFIG_BLK_DEV_FD=yCONFIG_BLK_DEV_LOOP=mCONFIG_BLK_DEV_RAM=yCONFIG_BLK_DEV_RAM_SIZE=4096CONFIG_BLK_DEV_INITRD=yCONFIG_PACKET=yCONFIG_PACKET_MMAP=yCONFIG_NETLINK=yCONFIG_RTNETLINK=yCONFIG_NETLINK_DEV=yCONFIG_NETFILTER=yCONFIG_UNIX=yCONFIG_INET=yCONFIG_IP_MULTICAST=yCONFIG_IP_NF_CONNTRACK=mCONFIG_IP_NF_FTP=mCONFIG_IP_NF_QUEUE=mCONFIG_IP_NF_IPTABLES=mCONFIG_IP_NF_MATCH_LIMIT=mCONFIG_IP_NF_MATCH_MULTIPORT=mCONFIG_IP_NF_MATCH_STATE=mCONFIG_IP_NF_FILTER=mCONFIG_IP_NF_NAT=mCONFIG_IP_NF_NAT_NEEDED=yCONFIG_IP_NF_TARGET_MASQUERADE=mCONFIG_IP_NF_NAT_FTP=mCONFIG_IDE=yCONFIG_BLK_DEV_IDE=yCONFIG_BLK_DEV_IDEDISK=yCONFIG_IDEDISK_MULTI_MODE=yCONFIG_BLK_DEV_IDECD=mCONFIG_BLK_DEV_IDEPCI=yCONFIG_BLK_DEV_IDEDMA_PCI=yCONFIG_BLK_DEV_ADMA=yCONFIG_IDEDMA_PCI_AUTO=yCONFIG_BLK_DEV_IDEDMA=yCONFIG_BLK_DEV_VIA82CXXX=yCONFIG_IDEDMA_AUTO=yCONFIG_BLK_DEV_IDE_MODES=yCONFIG_NETDEVICES=yCONFIG_TUN=mCONFIG_ETHERTAP=mCONFIG_NET_ETHERNET=yCONFIG_NET_PCI=yCONFIG_8139TOO=yCONFIG_PPP=yCONFIG_PPP_ASYNC=mCONFIG_PPP_DEFLATE=mCONFIG_PPP_BSDCOMP=mCONFIG_VT=yCONFIG_VT_CONSOLE=yCONFIG_SERIAL=yCONFIG_UNIX98_PTYS=yCONFIG_UNIX98_PTY_COUNT=128CONFIG_MOUSE=yCONFIG_PSMOUSE=yCONFIG_RTC=mCONFIG_REISERFS_FS=yCONFIG_FAT_FS=mCONFIG_MSDOS_FS=mCONFIG_VFAT_FS=mCONFIG_ISO9660_FS=mCONFIG_PROC_FS=yCONFIG_DEVPTS_FS=yCONFIG_EXT2_FS=yCONFIG_MSDOS_PARTITION=yCONFIG_NLS=yCONFIG_NLS_DEFAULT="iso8859-1"CONFIG_NLS_CODEPAGE_437=mCONFIG_VGA_CONSOLE=yCONFIG_VIDEO_SELECT=yCONFIG_SOUND=yCONFIG_SOUND_CMPCI=mCONFIG_SOUND_CMPCI_CM8738=yCONFIG_SOUND_CMPCI_SPEAKERS=2CONFIG_DEBUG_KERNEL=yCONFIG_MAGIC_SYSRQ=y
There have been a few kernel Oops, but the most recent ones haven't been saveable because vi, ed, and cat >oops.txt<<!! go into D STAT. (The 2 oops in messages that I do have are for a kernel that isn't on my system anymore (2.4.9-ac15+aging)).
Here they are the stack traces from /var/log/messages:
journal-1577: handle trans id 371103 != current trans id 371104 invalid operand: 0000 CPU: 0 EIP: 0010:[reiserfs_panic+36/48] EFLAGS: 00010286 eax: 00000043 ebx: c01fe940 ecx: da104000 edx: daea25a0 esi: c18d4240 edi: 00000000 ebp: dfe57400 esp: da105b94 ds: 0018 es: 0018 ss: 0018 Process growfiles (pid: 445, stackpage=da105000) Stack: c01fc17a c0277720 c01fe940 da105bb4 da105e20 c0168292 dfe57400 c01fe940 0005a99f 0005a9a0 dd157120 da105c50 da105c50 da105e7c 00000286 00000003 e08182f0 c0159741 da105e20 dfe57400 c18d4240 dfe57400 c18d4240 00000001 Call Trace: [journal_mark_dirty+82/760] [fix_nodes+105/1064] [do_journal_end+2838/2848] [reiserfs_paste_into_item+106/224] [reiserfs_get_block+1539/2088]
[__block_prepare_write+167/644] [block_prepare_write+34/60] [reiserfs_get_block+0/2088] [reiserfs_prepare_write+92/100] [reiserfs_get_block+0/2088] [generic_file_write+922/1480] [sys_write+143/196] [system_call+51/64]
Code: 0f 0b 68 20 77 27 c0 e8 a0 63 fb ff 57 56 53 8b 7c 24 10 bb
<7>cmpci: dma timed out??
journal-1409: journal_mark_dirty returning because j_wcount was 0
journal-1577: handle trans id 420964 != current trans id 420965 invalid operand: 0000 CPU: 0 EIP: 0010:[reiserfs_panic+36/48] EFLAGS: 00010286 eax: 00000043 ebx: c01fe940 ecx: db77c000 edx: dc127640 esi: c18d4240 edi: 00000000 ebp: dfe57400 esp: db77db94 0018 Process growfiles (pid: 439, stackpage=db77d000) Stack: c01fc17a c0277720 c01fe940 db77dbb4 db77de20 c0168292 dfe57400 c01fe940 00066c64 00066c65 df3bff80 db77dc50 db77dc50 db77de7c 00000286 00000003 e0818894 c0159741 db77de20 dfe57400 c18d4240 dfe57400 c18d4240 00000001 Call Trace: [journal_mark_dirty+82/760] [fix_nodes+105/1064] [do_journal_end+2838/2848] [reiserfs_paste_into_item+106/224] [reiserfs_get_block+1539/2088]
[__block_prepare_write+167/644] [block_prepare_write+34/60] [reiserfs_get_block+0/2088] [reiserfs_prepare_write+92/100] [reiserfs_get_block+0/2088] [generic_file_write+922/1480] [sys_write+143/196] [system_call+51/64]
Code: 0f 0b 68 20 77 27 c0 e8 a0 63 fb ff 57 56 53 8b 7c 24 10 bb
Signature · 3 lines
Randy Hron = Linux - because it's all about freedom = Linux News at http://lwn.net/
Where can I get a copy of "growfiles"??
--
M. Edward (Ed) Borasky, Chief Scientist, Borasky Research
http://www.borasky-research.net http://www.aracnet.com/~znmeb
mailto:[email protected] mailto:[email protected]
Q: How do you tell when a pineapple is ready to eat?
A: It picks up its knife and fork.
Quoted 299 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of [email protected]
Sent: Sunday, September 30, 2001 11:36 AM
To: [email protected]
Subject: [LTP] growfiles and other processes in "ps aux" STAT DLinux Test Project "growfiles" makes a lot of I/O processes hang in with "ps aux" STAT of D. (sometimes requires reboot)
This phenomenon occurs on late 2.4.9-ac1[0-8] kernels.
2.4.10 kernel on another box (laptop with ext3 filesystem, 192M ram) finished
with LTP Killed with "errno=12 Cannot allocate memory".
LTP has completed on laptop with 2.4.9-ac17. laptop has only been tested 5 times in total with kernels >= 2.4.9-ac7.The rest of information below is from LTP on a thunderbird:
Worst Case Symptoms (happened on at least 2 runs of 24 recent tests)
growfiles "hangs" - top and "ps aux" show STAT of D Many new processes hang in D STAT, vi, ed, logout, su, sync. syslogd in D STAT.
Other running processes, BitchX, links, vmstat, iostat continue working normally.
Some commands work fine: ls, ps
<ctrl \> and kill -9 won't kill processes in D STAT vmstat shows cpu 100% idle
iostat shows 0 tps and 0 blks/s
load average is around 4 for 1, 5, 15 minutes.
sysreq <Sync reBoot Umount> doesn't work, but sysreq (showMem showTasks) does
<ctrl alt del> doesn't reboot.
clear with reset button if necessary.Last command executed before test had to be terminated. The mmap001 tests did not require reboot.
2.4.9-ac10
mmap0012.4.9-ac15+aging patch
mmap001
mmap001 -m 10000These required reboot:
2.4.9-ac17 4 completed, 1 incomplete on:
growfiles -D 0 -b -i 0 -L 60 -u -B 1000b -e 1 -r 512-64000:1024 -R 1-384000 -T 4 gf-inf-$$2.4.9-ac18 1 incomplete run.
growfiles -D 0 -b -i 0 -L 60 -u -B 1000b -e 1 -r 128-32768:128 -R 512-64000 -T 4 gfsmallio-$$I started tracking the kernel version/test results recently, because until Sep 23,
the tests weren't producing alarming results. (24 total tests since Aug 8 on this machine).
The earlier LTP tests were released in April and June of 2001, and I didn't save those results,
as there were no surprises.Distro: Linux From Scratch 3.0
reiserfs on all filesystems.Hardware:
AMD 1333mhz Thunderbird
IWill KK266 motherboard
512 Mb RAM
40 Gb 7200 rpm 2mb cache IDE disk
00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03) 00:01.0 PCI bridge: VIA Technologies, Inc.: Unknown device 8305 00:07.0 ISA bridge: VIA Technologies, Inc. VT82C686 [Apollo Super] (rev 40)
00:07.1 IDE interface: VIA Technologies, Inc. VT82C586 IDE [Apollo] (rev 06)
00:07.4 Bridge: VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] (rev 40)
00:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139 (rev 10)
00:0f.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10)
01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 04)hdparm -I
/dev/hda
/dev/hda:Model=CI530L04VARE700- , FwRev=REO44AA1,
SerialNo= S 0XXSCL0734
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=40 BuffType=DualPortCache, BuffSize=1916kB, MaxMultSect=16, MultSect=16 CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=80418240 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-5 T13 1321D revision 1 : ATA-2 ATA-3
ATA-4 ATA-5I removed bogus
CONFIG_TOSHIBAand CONFIG*PCMCIA* settings during ac17 (when
I started preparing this report. Also removed framebuffer, drm, and power
management support.grep ^CONFIG
/usr/src/linux-2.4.9-ac18/.configCONFIG_X86=yCONFIG_ISA=yCONFIG_UID16=yCONFIG_EXPERIMENTAL=yCONFIG_MODULES=yCONFIG_KMOD=yCONFIG_M586=yCONFIG_X86_WP_WORKS_OK=yCONFIG_X86_INVLPG=yCONFIG_X86_CMPXCHG=yCONFIG_X86_XADD=yCONFIG_X86_BSWAP=yCONFIG_X86_POPAD_OK=yCONFIG_RWSEM_XCHGADD_ALGORITHM=yCONFIG_X86_L1_CACHE_SHIFT=5CONFIG_X86_USE_STRING_486=yCONFIG_X86_ALIGNMENT_16=yCONFIG_NOHIGHMEM=yCONFIG_MTRR=yCONFIG_NET=yCONFIG_PCI=yCONFIG_PCI_GOANY=yCONFIG_PCI_BIOS=yCONFIG_PCI_DIRECT=yCONFIG_PCI_NAMES=yCONFIG_SYSVIPC=yCONFIG_SYSCTL=yCONFIG_KCORE_ELF=yCONFIG_BINFMT_ELF=yCONFIG_PNP=yCONFIG_BLK_DEV_FD=yCONFIG_BLK_DEV_LOOP=mCONFIG_BLK_DEV_RAM=yCONFIG_BLK_DEV_RAM_SIZE=4096CONFIG_BLK_DEV_INITRD=yCONFIG_PACKET=yCONFIG_PACKET_MMAP=yCONFIG_NETLINK=yCONFIG_RTNETLINK=yCONFIG_NETLINK_DEV=yCONFIG_NETFILTER=yCONFIG_UNIX=yCONFIG_INET=yCONFIG_IP_MULTICAST=yCONFIG_IP_NF_CONNTRACK=mCONFIG_IP_NF_FTP=mCONFIG_IP_NF_QUEUE=mCONFIG_IP_NF_IPTABLES=mCONFIG_IP_NF_MATCH_LIMIT=mCONFIG_IP_NF_MATCH_MULTIPORT=mCONFIG_IP_NF_MATCH_STATE=mCONFIG_IP_NF_FILTER=mCONFIG_IP_NF_NAT=mCONFIG_IP_NF_NAT_NEEDED=yCONFIG_IP_NF_TARGET_MASQUERADE=mCONFIG_IP_NF_NAT_FTP=mCONFIG_IDE=yCONFIG_BLK_DEV_IDE=yCONFIG_BLK_DEV_IDEDISK=yCONFIG_IDEDISK_MULTI_MODE=yCONFIG_BLK_DEV_IDECD=mCONFIG_BLK_DEV_IDEPCI=yCONFIG_BLK_DEV_IDEDMA_PCI=yCONFIG_BLK_DEV_ADMA=yCONFIG_IDEDMA_PCI_AUTO=yCONFIG_BLK_DEV_IDEDMA=yCONFIG_BLK_DEV_VIA82CXXX=yCONFIG_IDEDMA_AUTO=yCONFIG_BLK_DEV_IDE_MODES=yCONFIG_NETDEVICES=yCONFIG_TUN=mCONFIG_ETHERTAP=mCONFIG_NET_ETHERNET=yCONFIG_NET_PCI=yCONFIG_8139TOO=yCONFIG_PPP=yCONFIG_PPP_ASYNC=mCONFIG_PPP_DEFLATE=mCONFIG_PPP_BSDCOMP=mCONFIG_VT=yCONFIG_VT_CONSOLE=yCONFIG_SERIAL=yCONFIG_UNIX98_PTYS=yCONFIG_UNIX98_PTY_COUNT=128CONFIG_MOUSE=yCONFIG_PSMOUSE=yCONFIG_RTC=mCONFIG_REISERFS_FS=yCONFIG_FAT_FS=mCONFIG_MSDOS_FS=mCONFIG_VFAT_FS=mCONFIG_ISO9660_FS=mCONFIG_PROC_FS=yCONFIG_DEVPTS_FS=yCONFIG_EXT2_FS=yCONFIG_MSDOS_PARTITION=yCONFIG_NLS=yCONFIG_NLS_DEFAULT="iso8859-1"CONFIG_NLS_CODEPAGE_437=mCONFIG_VGA_CONSOLE=yCONFIG_VIDEO_SELECT=yCONFIG_SOUND=yCONFIG_SOUND_CMPCI=mCONFIG_SOUND_CMPCI_CM8738=yCONFIG_SOUND_CMPCI_SPEAKERS=2CONFIG_DEBUG_KERNEL=yCONFIG_MAGIC_SYSRQ=yThere have been a few kernel Oops, but the most recent ones haven't been saveable because vi, ed, and cat >oops.txt<<!! go into D STAT. (The 2 oops in messages that I do have are for a kernel that isn't on my system anymore (2.4.9-ac15+aging)).
Here they are the stack traces from
/var/log/messages:journal-1577: handle trans id 371103 != current trans id 371104 invalid operand: 0000 CPU: 0 EIP: 0010:[reiserfs_panic+36/48] EFLAGS: 00010286 eax: 00000043 ebx: c01fe940 ecx: da104000 edx: daea25a0 esi: c18d4240 edi: 00000000 ebp: dfe57400 esp: da105b94 ds: 0018 es: 0018 ss: 0018 Process growfiles (pid: 445, stackpage=da105000) Stack: c01fc17a c0277720 c01fe940 da105bb4 da105e20 c0168292 dfe57400 c01fe940 0005a99f 0005a9a0 dd157120 da105c50 da105c50 da105e7c 00000286 00000003 e08182f0 c0159741 da105e20 dfe57400 c18d4240 dfe57400 c18d4240 00000001 Call Trace: [journal_mark_dirty+82/760] [fix_nodes+105/1064][do_journal_end+2838/2848] [reiserfs_paste_into_item+106/224] [reiserfs_get_block+1539/2088] [__block_prepare_write+167/644] [block_prepare_write+34/60] [reiserfs_get_block+0/2088] [reiserfs_prepare_write+92/100] [reiserfs_get_block+0/2088] [generic_file_write+922/1480] [sys_write+143/196] [system_call+51/64]
Code: 0f 0b 68 20 77 27 c0 e8 a0 63 fb ff 57 56 53 8b 7c 24 10 bb<7>cmpci: dma timed out??
journal-1409:journal_mark_dirtyreturning becausej_wcountwas 0journal-1577: handle trans id 420964 != current trans id 420965 invalid operand: 0000 CPU: 0 EIP: 0010:[reiserfs_panic+36/48] EFLAGS: 00010286 eax: 00000043 ebx: c01fe940 ecx: db77c000 edx: dc127640 esi: c18d4240 edi: 00000000 ebp: dfe57400 esp: db77db94 0018 Process growfiles (pid: 439, stackpage=db77d000) Stack: c01fc17a c0277720 c01fe940 db77dbb4 db77de20 c0168292 dfe57400 c01fe940 00066c64 00066c65 df3bff80 db77dc50 db77dc50 db77de7c 00000286 00000003 e0818894 c0159741 db77de20 dfe57400 c18d4240 dfe57400 c18d4240 00000001 Call Trace: [journal_mark_dirty+82/760] [fix_nodes+105/1064][do_journal_end+2838/2848] [reiserfs_paste_into_item+106/224] [reiserfs_get_block+1539/2088] [__block_prepare_write+167/644] [block_prepare_write+34/60] [reiserfs_get_block+0/2088] [reiserfs_prepare_write+92/100] [reiserfs_get_block+0/2088] [generic_file_write+922/1480] [sys_write+143/196] [system_call+51/64]
Code: 0f 0b 68 20 77 27 c0 e8 a0 63 fb ff 57 56 53 8b 7c 24 10 bb--
Randy Hron
= Linux - because it's all about freedom =
Linux News at http://lwn.net/-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Just out of curiosity, have you looked at how much memory is in "cached"?
--
M. Edward (Ed) Borasky, Chief Scientist, Borasky Research
http://www.borasky-research.net http://www.aracnet.com/~znmeb
mailto:[email protected] mailto:[email protected]
Q: How do you tell when a pineapple is ready to eat?
A: It picks up its knife and fork.
Quoted 299 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of [email protected]
Sent: Sunday, September 30, 2001 11:36 AM
To: [email protected]
Subject: [LTP] growfiles and other processes in "ps aux" STAT DLinux Test Project "growfiles" makes a lot of I/O processes hang in with "ps aux" STAT of D. (sometimes requires reboot)
This phenomenon occurs on late 2.4.9-ac1[0-8] kernels.
2.4.10 kernel on another box (laptop with ext3 filesystem, 192M ram) finished
with LTP Killed with "errno=12 Cannot allocate memory".
LTP has completed on laptop with 2.4.9-ac17. laptop has only been tested 5 times in total with kernels >= 2.4.9-ac7.The rest of information below is from LTP on a thunderbird:
Worst Case Symptoms (happened on at least 2 runs of 24 recent tests)
growfiles "hangs" - top and "ps aux" show STAT of D Many new processes hang in D STAT, vi, ed, logout, su, sync. syslogd in D STAT.
Other running processes, BitchX, links, vmstat, iostat continue working normally.
Some commands work fine: ls, ps
<ctrl \> and kill -9 won't kill processes in D STAT vmstat shows cpu 100% idle
iostat shows 0 tps and 0 blks/s
load average is around 4 for 1, 5, 15 minutes.
sysreq <Sync reBoot Umount> doesn't work, but sysreq (showMem showTasks) does
<ctrl alt del> doesn't reboot.
clear with reset button if necessary.Last command executed before test had to be terminated. The mmap001 tests did not require reboot.
2.4.9-ac10
mmap0012.4.9-ac15+aging patch
mmap001
mmap001 -m 10000These required reboot:
2.4.9-ac17 4 completed, 1 incomplete on:
growfiles -D 0 -b -i 0 -L 60 -u -B 1000b -e 1 -r 512-64000:1024 -R 1-384000 -T 4 gf-inf-$$2.4.9-ac18 1 incomplete run.
growfiles -D 0 -b -i 0 -L 60 -u -B 1000b -e 1 -r 128-32768:128 -R 512-64000 -T 4 gfsmallio-$$I started tracking the kernel version/test results recently, because until Sep 23,
the tests weren't producing alarming results. (24 total tests since Aug 8 on this machine).
The earlier LTP tests were released in April and June of 2001, and I didn't save those results,
as there were no surprises.Distro: Linux From Scratch 3.0
reiserfs on all filesystems.Hardware:
AMD 1333mhz Thunderbird
IWill KK266 motherboard
512 Mb RAM
40 Gb 7200 rpm 2mb cache IDE disk
00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03) 00:01.0 PCI bridge: VIA Technologies, Inc.: Unknown device 8305 00:07.0 ISA bridge: VIA Technologies, Inc. VT82C686 [Apollo Super] (rev 40)
00:07.1 IDE interface: VIA Technologies, Inc. VT82C586 IDE [Apollo] (rev 06)
00:07.4 Bridge: VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] (rev 40)
00:0d.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139 (rev 10)
00:0f.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10)
01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 04)hdparm -I
/dev/hda
/dev/hda:Model=CI530L04VARE700- , FwRev=REO44AA1,
SerialNo= S 0XXSCL0734
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=40 BuffType=DualPortCache, BuffSize=1916kB, MaxMultSect=16, MultSect=16 CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=80418240 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-5 T13 1321D revision 1 : ATA-2 ATA-3
ATA-4 ATA-5I removed bogus
CONFIG_TOSHIBAand CONFIG*PCMCIA* settings during ac17 (when
I started preparing this report. Also removed framebuffer, drm, and power
management support.grep ^CONFIG
/usr/src/linux-2.4.9-ac18/.configCONFIG_X86=yCONFIG_ISA=yCONFIG_UID16=yCONFIG_EXPERIMENTAL=yCONFIG_MODULES=yCONFIG_KMOD=yCONFIG_M586=yCONFIG_X86_WP_WORKS_OK=yCONFIG_X86_INVLPG=yCONFIG_X86_CMPXCHG=yCONFIG_X86_XADD=yCONFIG_X86_BSWAP=yCONFIG_X86_POPAD_OK=yCONFIG_RWSEM_XCHGADD_ALGORITHM=yCONFIG_X86_L1_CACHE_SHIFT=5CONFIG_X86_USE_STRING_486=yCONFIG_X86_ALIGNMENT_16=yCONFIG_NOHIGHMEM=yCONFIG_MTRR=yCONFIG_NET=yCONFIG_PCI=yCONFIG_PCI_GOANY=yCONFIG_PCI_BIOS=yCONFIG_PCI_DIRECT=yCONFIG_PCI_NAMES=yCONFIG_SYSVIPC=yCONFIG_SYSCTL=yCONFIG_KCORE_ELF=yCONFIG_BINFMT_ELF=yCONFIG_PNP=yCONFIG_BLK_DEV_FD=yCONFIG_BLK_DEV_LOOP=mCONFIG_BLK_DEV_RAM=yCONFIG_BLK_DEV_RAM_SIZE=4096CONFIG_BLK_DEV_INITRD=yCONFIG_PACKET=yCONFIG_PACKET_MMAP=yCONFIG_NETLINK=yCONFIG_RTNETLINK=yCONFIG_NETLINK_DEV=yCONFIG_NETFILTER=yCONFIG_UNIX=yCONFIG_INET=yCONFIG_IP_MULTICAST=yCONFIG_IP_NF_CONNTRACK=mCONFIG_IP_NF_FTP=mCONFIG_IP_NF_QUEUE=mCONFIG_IP_NF_IPTABLES=mCONFIG_IP_NF_MATCH_LIMIT=mCONFIG_IP_NF_MATCH_MULTIPORT=mCONFIG_IP_NF_MATCH_STATE=mCONFIG_IP_NF_FILTER=mCONFIG_IP_NF_NAT=mCONFIG_IP_NF_NAT_NEEDED=yCONFIG_IP_NF_TARGET_MASQUERADE=mCONFIG_IP_NF_NAT_FTP=mCONFIG_IDE=yCONFIG_BLK_DEV_IDE=yCONFIG_BLK_DEV_IDEDISK=yCONFIG_IDEDISK_MULTI_MODE=yCONFIG_BLK_DEV_IDECD=mCONFIG_BLK_DEV_IDEPCI=yCONFIG_BLK_DEV_IDEDMA_PCI=yCONFIG_BLK_DEV_ADMA=yCONFIG_IDEDMA_PCI_AUTO=yCONFIG_BLK_DEV_IDEDMA=yCONFIG_BLK_DEV_VIA82CXXX=yCONFIG_IDEDMA_AUTO=yCONFIG_BLK_DEV_IDE_MODES=yCONFIG_NETDEVICES=yCONFIG_TUN=mCONFIG_ETHERTAP=mCONFIG_NET_ETHERNET=yCONFIG_NET_PCI=yCONFIG_8139TOO=yCONFIG_PPP=yCONFIG_PPP_ASYNC=mCONFIG_PPP_DEFLATE=mCONFIG_PPP_BSDCOMP=mCONFIG_VT=yCONFIG_VT_CONSOLE=yCONFIG_SERIAL=yCONFIG_UNIX98_PTYS=yCONFIG_UNIX98_PTY_COUNT=128CONFIG_MOUSE=yCONFIG_PSMOUSE=yCONFIG_RTC=mCONFIG_REISERFS_FS=yCONFIG_FAT_FS=mCONFIG_MSDOS_FS=mCONFIG_VFAT_FS=mCONFIG_ISO9660_FS=mCONFIG_PROC_FS=yCONFIG_DEVPTS_FS=yCONFIG_EXT2_FS=yCONFIG_MSDOS_PARTITION=yCONFIG_NLS=yCONFIG_NLS_DEFAULT="iso8859-1"CONFIG_NLS_CODEPAGE_437=mCONFIG_VGA_CONSOLE=yCONFIG_VIDEO_SELECT=yCONFIG_SOUND=yCONFIG_SOUND_CMPCI=mCONFIG_SOUND_CMPCI_CM8738=yCONFIG_SOUND_CMPCI_SPEAKERS=2CONFIG_DEBUG_KERNEL=yCONFIG_MAGIC_SYSRQ=yThere have been a few kernel Oops, but the most recent ones haven't been saveable because vi, ed, and cat >oops.txt<<!! go into D STAT. (The 2 oops in messages that I do have are for a kernel that isn't on my system anymore (2.4.9-ac15+aging)).
Here they are the stack traces from
/var/log/messages:journal-1577: handle trans id 371103 != current trans id 371104 invalid operand: 0000 CPU: 0 EIP: 0010:[reiserfs_panic+36/48] EFLAGS: 00010286 eax: 00000043 ebx: c01fe940 ecx: da104000 edx: daea25a0 esi: c18d4240 edi: 00000000 ebp: dfe57400 esp: da105b94 ds: 0018 es: 0018 ss: 0018 Process growfiles (pid: 445, stackpage=da105000) Stack: c01fc17a c0277720 c01fe940 da105bb4 da105e20 c0168292 dfe57400 c01fe940 0005a99f 0005a9a0 dd157120 da105c50 da105c50 da105e7c 00000286 00000003 e08182f0 c0159741 da105e20 dfe57400 c18d4240 dfe57400 c18d4240 00000001 Call Trace: [journal_mark_dirty+82/760] [fix_nodes+105/1064][do_journal_end+2838/2848] [reiserfs_paste_into_item+106/224] [reiserfs_get_block+1539/2088] [__block_prepare_write+167/644] [block_prepare_write+34/60] [reiserfs_get_block+0/2088] [reiserfs_prepare_write+92/100] [reiserfs_get_block+0/2088] [generic_file_write+922/1480] [sys_write+143/196] [system_call+51/64]
Code: 0f 0b 68 20 77 27 c0 e8 a0 63 fb ff 57 56 53 8b 7c 24 10 bb<7>cmpci: dma timed out??
journal-1409:journal_mark_dirtyreturning becausej_wcountwas 0journal-1577: handle trans id 420964 != current trans id 420965 invalid operand: 0000 CPU: 0 EIP: 0010:[reiserfs_panic+36/48] EFLAGS: 00010286 eax: 00000043 ebx: c01fe940 ecx: db77c000 edx: dc127640 esi: c18d4240 edi: 00000000 ebp: dfe57400 esp: db77db94 0018 Process growfiles (pid: 439, stackpage=db77d000) Stack: c01fc17a c0277720 c01fe940 db77dbb4 db77de20 c0168292 dfe57400 c01fe940 00066c64 00066c65 df3bff80 db77dc50 db77dc50 db77de7c 00000286 00000003 e0818894 c0159741 db77de20 dfe57400 c18d4240 dfe57400 c18d4240 00000001 Call Trace: [journal_mark_dirty+82/760] [fix_nodes+105/1064][do_journal_end+2838/2848] [reiserfs_paste_into_item+106/224] [reiserfs_get_block+1539/2088] [__block_prepare_write+167/644] [block_prepare_write+34/60] [reiserfs_get_block+0/2088] [reiserfs_prepare_write+92/100] [reiserfs_get_block+0/2088] [generic_file_write+922/1480] [sys_write+143/196] [system_call+51/64]
Code: 0f 0b 68 20 77 27 c0 e8 a0 63 fb ff 57 56 53 8b 7c 24 10 bb--
Randy Hron
= Linux - because it's all about freedom =
Linux News at http://lwn.net/-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 1 line · M. Edward Borasky
Where can I get a copy of "growfiles"??
growfiles is part of the Linux Test Project (see my sig).
Signature · 3 lines
Nate Straz [email protected] sgi, inc http://www.sgi.com/ Linux Test Project http://ltp.sf.net/