JDIRTY JWAIT errors in 2.4.19

i'm getting the following error logged every 11 seconds or so:

Dec 14 01:00:49 phylum kernel: vs-3050: wait_buffer_until_released: nobody

releases buffer (dev 16:01, size 4096, blocknr 2916352, count 3, list 0, state 0x10019, page c1172108, (UPTODATE, CLEAN, UNLOCKED)). Still waiting (-1320000000) JDIRTY !JWAIT

Also, some processes are blocking, include ps (so I can't get a complete process list), and shutdown.

This is on an old PII that's been up with no problems for about a month.
Is this a known problem? Is there any way to force a clean shutdown in such circumstances? Is this associated with reiserfs which all partitions are running? A googling turned up one or two references to this problem, but never with a kernel as recent as 2.4.19, and associated with, though not necessarily blamed on reiserfs.

This machine is still up, so if more info would be useful, it can probably be provided. Most processes are still working fine.

-Tupshin

Hello!

Can you please execute SysRq-T, decode it with ksymoops and send us the result?

Yes, this is reiserfs error message.

Thank you.

Bye,
Oleg

Sigh...unfortunately that machine didn't have magic SysRq compiled in. I'm going to build an idential kernel with Sysrq and reboot. Maybe I'll see the error again in a month or so ;-(

-Tupshin

Sigh...unfortunately that machine didn't have magic SysRq compiled in. I'm going to build an identical kernel with Sysrq and reboot. Maybe I'll see the error again in a month or so ;-(

-Tupshin

I have succeeded in reproducing this problem at will (and deterministically) with a SysRq enabled kernel.

I can reproduce the problem consistently by doing a cp -av of a directory(haven't tried other cp permutations, guessing it wouldn't make a difference) containing three mp3 files from one location on a partition to another location on the same reiserfs partition. During the third file, cp hangs(continues to use 80%+ of the CPU), and is unkillable. A few minutes later, it starts generating the same JDIRTY error messages that I reported before.

The attached file is a ksymoops processed version of SysRq-T after this problem has started to occur. There is one cp taking place which is the one that triggered the problem, and there is also a sync command that is blocking as well which I tried to do after the cp hung.

I have not tried to do any kind of reiserfs check or repair on the filesystem, and I can trigger this problem at any point if you would like me to test further.

-Tupshin

ksymoops 2.4.6 on i686 2.4.19-sysrq. Options used
-V (default)
-k /proc/ksyms (specified)
-l /proc/modules (specified)
-o /lib/modules/2.4.19-sysrq/ (specified) -m /boot/System.map-2.4.19-sysrq (specified)

Dec 15 12:08:30 phylum kernel: init          S C12F5F28  4500     1      0  1135               (NOTLB)
Using defaults from ksymoops -t elf32-i386 -a i386
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: keventd       S C12D8000  6152     2      1             3       (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [context_thread+253/424] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: ksoftirqd_CPU S C12D4000  6388     3      1             4     2 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [ksoftirqd+116/180] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: kswapd        S C12D2000  6108     4      1             5     3 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [kswapd+127/196] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: bdflush       S 00000282  6560     5      1             6     4 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [interruptible_sleep_on+61/80] [bdflush+166/168] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: kupdated      D 00000282  5664     6      1             9     5 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sleep_on+61/80] [check_journal_end+294/556] [do_journal_end+178/2584] [flush_old_commits+267/304] [flush_old_commits+281/304]
Dec 15 12:08:30 phylum kernel: mdrecoveryd   S C138C000  6580     9      1             8     6 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [md_thread+201/316] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: khubd         S C139DFDC  6000     8      1            10     9 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [usb_hub_thread+127/168] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: kreiserfsd    S CA33DFB0  6064    10      1            24     8 (L-TLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [interruptible_sleep_on_timeout+66/92] [reiserfs_journal_commit_thread+153/208] [kernel_thread+40/56]
Dec 15 12:08:30 phylum kernel: devfsd        S C9DB4000  5964    24      1           206    10 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [devfsd_read+238/868] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: portmap       S 7FFFFFFF     0   206      1           328    24 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_poll+117/204] [do_poll+168/204] [sys_poll+473/748] [sys_socketcall+383/408]
Dec 15 12:08:30 phylum kernel: syslogd       S 7FFFFFFF  4752   328      1           331   206 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: klogd         R C455C000  4752   331      1           339   328 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [do_syslog+208/776] [kmsg_read+18/24] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: named         S C42A9FAC  5408   339      1   340     361   331 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_rt_sigsuspend+238/264] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: named         S CA16FF28  1248   340    339   343               (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_poll+168/204] [sys_poll+473/748] [do_softirq+75/160]
Dec 15 12:08:30 phylum kernel: named         S C4645FAC     0   341    340           342       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_rt_sigsuspend+238/264] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: named         S CA1F9F88     0   342    340           343   341 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: named         S 7FFFFFFF     0   343    340                 342 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: slapd         S C3E97FAC  4752   361      1   364     366   339 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_rt_sigsuspend+238/264] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: slapd         S C3E09F28  6292   364    361   365               (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_poll+168/204] [sys_poll+473/748] [schedule+714/756]
Dec 15 12:08:30 phylum kernel: slapd         S 7FFFFFFF  2660   365    364                     (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: spamd         S 7FFFFFFF  4752   366      1           517   361 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: proxytest     S C2A07F88  5172   517      1           525   366 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: authdaemond.p S C29C1F28  1024   525      1   531     538   517 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: authdaemond.p S C299BF28  6332   526    525           527       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: authdaemond.p S C2993F28  6332   527    525           528   526 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: authdaemond.p S C298BF28  6332   528    525           529   527 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: authdaemond.p S C2983F28  6332   529    525           531   528 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: authdaemond.p S C2979F28  6392   531    525                 529 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: couriertcpd   S 7FFFFFFF     4   538      1           542   525 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: courierlogger S C29CA000     0   542      1           552   538 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [pipe_wait+121/164] [pipe_read+191/544] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: couriertcpd   S 7FFFFFFF     0   552      1           555   542 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: courierlogger S C456A000     0   555      1           559   552 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [pipe_wait+121/164] [pipe_read+191/544] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: cupsd         S C2863F28  4752   559      1           783   555 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: dhcpd         S 7FFFFFFF  2656   783      1           793   559 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: inetd         S 7FFFFFFF   104   793      1           811   783 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: safe_mysqld   S 00000000  2656   811      1   843     870   793 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_wait4+860/908] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: mysqld        S 7FFFFFFF     0   843    811   844               (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: mysqld        S C1967F28  6292   844    843   846               (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_poll+168/204] [sys_poll+473/748] [sys_wait4+896/908]
Dec 15 12:08:30 phylum kernel: mysqld        S C1963FAC  4752   845    844           846       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_rt_sigsuspend+238/264] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: mysqld        S C18F3FAC  6600   846    844                 845 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_rt_sigsuspend+238/264] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: netsaint      S C1871F88     0   870      1           916   811 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: pptpd         S 7FFFFFFF     0   916      1           919   870 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: privoxy       S 7FFFFFFF     0   919      1           924   916 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: qmail-send    S C168DF28  4600   924      1   934     929   919 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: tcpserver     S 7FFFFFFF  4600   928      1           936   929 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: splogger      S C1656000   160   929      1           928   924 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [pipe_wait+121/164] [pipe_read+191/544] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: splogger      S C164A000     0   931    924           932       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [pipe_wait+121/164] [pipe_read+191/544] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: qmail-lspawn  S 7FFFFFFF  2532   932    924           933   931 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: qmail-rspawn  S 7FFFFFFF     0   933    924           934   932 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: qmail-clean   S C1688000     0   934    924                 933 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [pipe_wait+121/164] [pipe_read+191/544] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: rebootmgr     S 7FFFFFFF  4600   936      1           949   928 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [free_page_and_swap_cache+53/60] [schedule_timeout+23/148] [do_select+30/468] [do_select+412/468] [sys_select+800/1100]
Dec 15 12:08:30 phylum kernel: nmbd          S C0DEFF28  4584   949      1           951   936 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: smbd          S 7FFFFFFF  5076   951      1           954   949 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: slpd          S 7FFFFFFF  6068   954      1           957   951 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: smartd        S C09C3F88  5680   957      1           966   954 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: sshd          S 7FFFFFFF  2656   966      1  1327     991   957 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: rpc.statd     S 7FFFFFFF  2656   991      1           995   966 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: ntpd          S 7FFFFFFF  4692   995      1  1005    1001   991 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: rpc.nfsd      S 7FFFFFFF     0  1001      1          1007   995 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_poll+117/204] [do_poll+168/204] [sys_poll+473/748] [sys_sigreturn+182/224]
Dec 15 12:08:30 phylum kernel: ntpd          S CBD67FAC     0  1005    995                     (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_rt_sigsuspend+238/264] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: rpc.mountd    S 7FFFFFFF     0  1007      1          1026  1001 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_poll+117/204] [do_poll+168/204] [sys_poll+473/748] [sys_sigreturn+182/224]
Dec 15 12:08:30 phylum kernel: squid         S 00000000     0  1026      1  1031    1052  1007 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_wait4+860/908] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: squid         S CBC55F28     0  1031   1026  1043               (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_poll+168/204] [sys_poll+473/748] [filp_close+89/104]
Dec 15 12:08:30 phylum kernel: squid_redirec S 7FFFFFFF  2656  1036   1031          1039       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [tcp_data_wait+212/304] [tcp_recvmsg+1060/1956] [lru_cache_del+8/24] [inet_recvmsg+58/84]
Dec 15 12:08:30 phylum kernel: squid_redirec S 7FFFFFFF  2656  1039   1031          1040  1036 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [tcp_data_wait+212/304] [tcp_recvmsg+1060/1956] [lru_cache_del+8/24] [inet_recvmsg+58/84]
Dec 15 12:08:30 phylum kernel: squid_redirec S 7FFFFFFF  2656  1040   1031          1041  1039 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [tcp_data_wait+212/304] [tcp_recvmsg+1060/1956] [lru_cache_del+8/24] [inet_recvmsg+58/84]
Dec 15 12:08:30 phylum kernel: squid_redirec S 7FFFFFFF     0  1041   1031          1042  1040 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [tcp_data_wait+212/304] [tcp_recvmsg+1060/1956] [lru_cache_del+8/24] [inet_recvmsg+58/84]
Dec 15 12:08:30 phylum kernel: squid_redirec S 7FFFFFFF  2656  1042   1031          1043  1041 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [tcp_data_wait+212/304] [tcp_recvmsg+1060/1956] [lru_cache_del+8/24] [inet_recvmsg+58/84]
Dec 15 12:08:30 phylum kernel: unlinkd       S CB808000     0  1043   1031                1042 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [pipe_wait+121/164] [pipe_read+191/544] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: proftpd       S CB559F28     0  1052      1          1062  1026 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: usb_perms     S CB42BF88     0  1062      1          1066  1052 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: atd           S CB0E7F88   152  1066      1          1069  1062 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: cron          S CB02DF88  2656  1069      1          1097  1066 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: apache        S C8739F28     0  1097      1  1119    1102  1069 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [generic_file_write+1217/1624] [sys_select+800/1100]
Dec 15 12:08:30 phylum kernel: apache-ssl    S C8417F28     8  1102      1  1141    1128  1097 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [do_select+412/468] [generic_file_write+1217/1624] [sys_select+800/1100]
Dec 15 12:08:30 phylum kernel: apache        S 7FFFFFFF     0  1103   1097          1105       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache        S 7FFFFFFF     0  1105   1097          1106  1103 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache        S 7FFFFFFF  2656  1106   1097          1107  1105 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache        S 7FFFFFFF     0  1107   1097          1119  1106 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache        S 7FFFFFFF     0  1119   1097                1107 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: bash          S 00000000  4600  1128      1  1175    1129  1102 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_wait4+860/908] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: bash          S 00000000     0  1129      1  1201    1130  1128 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_wait4+860/908] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: bash          S 00000000     0  1130      1  1205    1131  1129 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [sys_wait4+860/908] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: getty         S 7FFFFFFF     0  1131      1          1132  1130 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [read_chan+834/1644] [tty_read+174/216] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: getty         S 7FFFFFFF     0  1132      1          1135  1131 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [read_chan+834/1644] [tty_read+174/216] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: getty         S 7FFFFFFF  2656  1135      1                1132 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [read_chan+834/1644] [tty_read+174/216] [sys_read+152/244] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: gcache        S 7FFFFFFF     0  1136   1102          1137       (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_packet+206/320] [skb_recv_datagram+175/204] [unix_accept+73/192] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache-ssl    S 7FFFFFFF    44  1137   1102          1138  1136 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache-ssl    S 7FFFFFFF  2656  1138   1102          1139  1137 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache-ssl    S 7FFFFFFF  2656  1139   1102          1140  1138 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache-ssl    S 7FFFFFFF  2656  1140   1102          1141  1139 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: apache-ssl    S 7FFFFFFF  2656  1141   1102                1140 (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [wait_for_connect+205/372] [tcp_accept+137/428] [inet_accept+42/308] [sys_accept+97/252]
Dec 15 12:08:30 phylum kernel: cp            R current      0  1175   1128                     (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule+100/756] [wait_buffer_until_released+174/216] [get_empty_nodes+292/432] [fix_nodes+347/1068] [is_tree_node+71/80]
Dec 15 12:08:30 phylum kernel: sync          D CA0CBC44  1632  1201   1129                     (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [__down+81/152] [__down_failed+11/20] [.text.lock.super+123/237] [fsync_dev+34/52] [sys_sync+10/16]
Dec 15 12:08:30 phylum kernel: tail          S C72A1F88     0  1205   1130                     (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+115/148] [process_timeout+0/72] [sys_nanosleep+268/388] [system_call+51/56]
Dec 15 12:08:30 phylum kernel: sshd          S 7FFFFFFF   196  1327    966  1328               (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [unix_stream_data_wait+168/224] [unix_stream_recvmsg+389/928] [sock_recvmsg+55/168] [sock_read+135/144]
Dec 15 12:08:30 phylum kernel: sshd          S 7FFFFFFF  2532  1328   1327                     (NOTLB)
Dec 15 12:08:30 phylum kernel: Call Trace:    [schedule_timeout+23/148] [do_select+412/468] [sys_select+800/1100] [system_call+51/56]

Warning (Oops_read): Code line not seen, dumping what data is available

Proc; init

Proc; keventd

Proc; ksoftirqd_CPU

Proc; kswapd

Proc; bdflush

Proc; kupdated

Proc; mdrecoveryd

Proc; khubd

Proc; kreiserfsd

Proc; devfsd

Proc; portmap

Proc; syslogd

Proc; klogd

Proc; named

Proc; named

Proc; named

Proc; named

Proc; named

Proc; slapd

Proc; slapd

Proc; slapd

Proc; spamd

Proc; proxytest

Proc; authdaemond.p

Proc; authdaemond.p

Proc; authdaemond.p

Proc; authdaemond.p

Proc; authdaemond.p

Proc; authdaemond.p

Proc; couriertcpd

Proc; courierlogger

Proc; couriertcpd

Proc; courierlogger

Proc; cupsd

Proc; dhcpd

Proc; inetd

Proc; safe_mysqld

Proc; mysqld

Proc; mysqld

Proc; mysqld

Proc; mysqld

Proc; netsaint

Proc; pptpd

Proc; privoxy

Proc; qmail-send

Proc; tcpserver

Proc; splogger

Proc; splogger

Proc; qmail-lspawn

Proc; qmail-rspawn

Proc; qmail-clean

Proc; rebootmgr

Proc; nmbd

Proc; smbd

Proc; slpd

Proc; smartd

Proc; sshd

Proc; rpc.statd

Proc; ntpd

Proc; rpc.nfsd

Proc; ntpd

Proc; rpc.mountd

Proc; squid

Proc; squid

Proc; squid_redirec

Proc; squid_redirec

Proc; squid_redirec

Proc; squid_redirec

Proc; squid_redirec

Proc; unlinkd

Proc; proftpd

Proc; usb_perms

Proc; atd

Proc; cron

Proc; apache

Proc; apache-ssl

Proc; apache

Proc; apache

Proc; apache

Proc; apache

Proc; apache

Proc; bash

Proc; bash

Proc; bash

Proc; getty

Proc; getty

Proc; getty

Proc; gcache

Proc; apache-ssl

Proc; apache-ssl

Proc; apache-ssl

Proc; apache-ssl

Proc; apache-ssl

Proc; cp

Proc; sync

Proc; tail

Proc; sshd

Proc; sshd

1 warning issued. Results may not be reliable.

Hello!

Ok.

Can you please produce a metadump data for us?
use debugreiserfs -p /dev/yourdevice | gzip -9c >metadata.gz
then tell us where we can get that file.
Also please use recent reiserfsprogs for that operation
(e.g. version 3.6.4 available from us: ftp://ftp.namesys.com/pub/reiserprogs )

Let's see if we can reproduce it locally after you give us the metadata.

Thank you.

Bye,
Oleg