Skip to contentSkip to conversation

nfs4 client bug

PATCH New Bret Towe 6 participants 20 messages Sep 4, 2005 – Sep 12, 2005 Documentation

lore
Share & tools
lore

Bret Towe

author
#0 lore raw

i encountered the following error while using nfs4
ive hit this error i think twice now not sure what causes it yet tho this time the only io related items going on was emerge sync running in the background (which shouldnt of touched nfs at all) and xmms playing some music

another problem i had was iowait was showing near 100% but nothing over nfs was working
but no errors or anything was showing in dmesg if i encounter this again is there a way to
find out where it locked up so i can give a report on what the problem is?

attached is my config the box is an athlon64
if any further information is needed let me know

Unable to handle kernel paging request at 0000000000100108 RIP:
<ffffffff80189538>{generic_drop_inode+56}
PGD 27bec067 PUD 27be9067 PMD 0
Oops: 0002 [1]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 149, comm: kswapd0 Tainted: P M 2.6.13

RIP: 0010:[<ffffffff80189538>] <ffffffff80189538>{generic_drop_inode+56}
RSP: 0018:ffff81002f9d9b78  EFLAGS: 00010246
RAX: 0000000000100100 RBX: ffff810022d4d950 RCX: 0000000000200200
RDX: ffff810022d4d960 RSI: ffff81002ea21000 RDI: ffff810022d4d950
RBP: ffff810022d4d950 R08: 00000000fffffffa R09: ffff810022d4da68
R10: 0000000000000001 R11: ffffffff80189500 R12: 0000000000000000
R13: ffff810022d4d7d0 R14: ffff810022d4d860 R15: ffff81002c3cec00
FS:  00002aaaabc64f80(0000) GS:ffffffff805bc800(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000100108 CR3: 0000000027468000 CR4: 00000000000006e0
Process kswapd0 (pid: 149, threadinfo ffff81002f9d8000, task ffff81002f9d2760)
Stack: ffff81002a65fa00 ffffffff801e2925 000000012f9d9bf8 ffff81002f9d9c28
       ffffffffffffffff ffff81002f9d9c28 ffff81002f9d9c10 ffff810022d4d938
       0000000000000001 0000000000000000
Call Trace:<ffffffff801e2925>{__nfs_revalidate_inode+261}
<ffffffff80151dcf>{find_get_pages_tag+31}
       <ffffffff8015af2a>{pagevec_lookup_tag+26}
<ffffffff801517fe>{wait_on_page_writeback_range+206}
       <ffffffff801fc0ca>{nfs_do_return_delegation+42}
<ffffffff801fc1f5>{nfs_inode_return_delegation+197}
       <ffffffff801e3910>{nfs4_clear_inode+32}
<ffffffff8018839e>{clear_inode+158}
       <ffffffff80188fee>{dispose_list+94}
<ffffffff80189222>{shrink_icache_memory+434}
       <ffffffff8015b77b>{shrink_slab+219} <ffffffff8015cb99>{balance_pgdat+617}
       <ffffffff8015c932>{balance_pgdat+2} <ffffffff8015ce17>{kswapd+295}
       <ffffffff80144730>{autoremove_wake_function+0}
<ffffffff80144730>{autoremove_wake_function+0}
       <ffffffff8010f3e6>{child_rip+8} <ffffffff8015ccf0>{kswapd+0}
       <ffffffff8010f3de>{child_rip+0}
Code: 48 89 48 08 48 89 01 48 8b 05 8a 4e 30 00 48 89 50 08 48 89
RIP <ffffffff80189538>{generic_drop_inode+56} RSP <ffff81002f9d9b78>
CR2: 0000000000100108
 ----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at "fs/inode.c":1142
invalid operand: 0000 [2]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 9539, comm: rpciod/0 Tainted: P M 2.6.13

RIP: 0010:[<ffffffff8018853e>] <ffffffff8018853e>{iput+30}
RSP: 0018:ffff81002a181e08  EFLAGS: 00010246
RAX: ffffffff80494a00 RBX: ffff810023da8d40 RCX: ffff81002311f490
RDX: ffff810022919680 RSI: ffff81002614da40 RDI: ffff810023da8d40
RBP: ffff8100286af780 R08: ffff8100230f64e0 R09: 0000000000000000
R10: 0000000000000001 R11: ffffffff801f8c90 R12: ffff81002311f480
R13: ffff81002614da00 R14: ffff81002c3cec00 R15: ffffffff805c3fb0
FS:  00002aaaaade6b00(0000) GS:ffffffff805bc800(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00002aaaaade0680 CR3: 0000000022d19000 CR4: 00000000000006e0
Process rpciod/0 (pid: 9539, threadinfo ffff81002a180000, task ffff81002a4a2330)
Stack: ffff81002311f480 ffffffff801fb0fd 00000000fffffff3 ffff81002fb0f1c0
       ffff8100286af780 ffffffff801efa51 ffff81002fb0f1c0 ffff81002d9be310
       ffff81002fb0f2a8 ffff81002d9be320
Call Trace:<ffffffff801fb0fd>{nfs4_put_open_state+109}
<ffffffff801efa51>{nfs4_close_done+225}
       <ffffffff803940a5>{__rpc_execute+165}
<ffffffff8014046a>{worker_thread+442}
       <ffffffff8012e060>{default_wake_function+0}
<ffffffff8012e060>{default_wake_function+0}
       <ffffffff801402b0>{worker_thread+0} <ffffffff8014424d>{kthread+205}
       <ffffffff8010f3e6>{child_rip+8}
<ffffffff80144290>{keventd_create_kthread+0}
       <ffffffff80144180>{kthread+0} <ffffffff8010f3de>{child_rip+0}
Code: 0f 0b a3 e0 61 3c 80 ff ff ff ff c2 76 04 66 66 66 90 48 85

RIP <ffffffff8018853e>{iput+30} RSP <ffff81002a181e08>

author
#1 lore raw
Quoted 100 lines · Bret Towe

i encountered the following error while using nfs4
ive hit this error i think twice now not sure what causes it yet tho this time the only io related items going on was emerge sync running in the background (which shouldnt of touched nfs at all) and xmms playing some music

another problem i had was iowait was showing near 100% but nothing over nfs was working
but no errors or anything was showing in dmesg if i encounter this again is there a way to
find out where it locked up so i can give a report on what the problem is?

attached is my config the box is an athlon64
if any further information is needed let me know

Unable to handle kernel paging request at 0000000000100108 RIP:
<ffffffff80189538>{generic_drop_inode+56}
PGD 27bec067 PUD 27be9067 PMD 0
Oops: 0002 [1]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 149, comm: kswapd0 Tainted: P M 2.6.13

RIP: 0010:[<ffffffff80189538>] <ffffffff80189538>{generic_drop_inode+56}
RSP: 0018:ffff81002f9d9b78  EFLAGS: 00010246
RAX: 0000000000100100 RBX: ffff810022d4d950 RCX: 0000000000200200
RDX: ffff810022d4d960 RSI: ffff81002ea21000 RDI: ffff810022d4d950
RBP: ffff810022d4d950 R08: 00000000fffffffa R09: ffff810022d4da68
R10: 0000000000000001 R11: ffffffff80189500 R12: 0000000000000000
R13: ffff810022d4d7d0 R14: ffff810022d4d860 R15: ffff81002c3cec00
FS:  00002aaaabc64f80(0000) GS:ffffffff805bc800(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000100108 CR3: 0000000027468000 CR4: 00000000000006e0
Process kswapd0 (pid: 149, threadinfo ffff81002f9d8000, task ffff81002f9d2760)
Stack: ffff81002a65fa00 ffffffff801e2925 000000012f9d9bf8 ffff81002f9d9c28
       ffffffffffffffff ffff81002f9d9c28 ffff81002f9d9c10 ffff810022d4d938
       0000000000000001 0000000000000000
Call Trace:<ffffffff801e2925>{__nfs_revalidate_inode+261}
<ffffffff80151dcf>{find_get_pages_tag+31}
       <ffffffff8015af2a>{pagevec_lookup_tag+26}
<ffffffff801517fe>{wait_on_page_writeback_range+206}
       <ffffffff801fc0ca>{nfs_do_return_delegation+42}
<ffffffff801fc1f5>{nfs_inode_return_delegation+197}
       <ffffffff801e3910>{nfs4_clear_inode+32}
<ffffffff8018839e>{clear_inode+158}
       <ffffffff80188fee>{dispose_list+94}
<ffffffff80189222>{shrink_icache_memory+434}
       <ffffffff8015b77b>{shrink_slab+219} <ffffffff8015cb99>{balance_pgdat+617}
       <ffffffff8015c932>{balance_pgdat+2} <ffffffff8015ce17>{kswapd+295}
       <ffffffff80144730>{autoremove_wake_function+0}
<ffffffff80144730>{autoremove_wake_function+0}
       <ffffffff8010f3e6>{child_rip+8} <ffffffff8015ccf0>{kswapd+0}
       <ffffffff8010f3de>{child_rip+0}
Code: 48 89 48 08 48 89 01 48 8b 05 8a 4e 30 00 48 89 50 08 48 89
RIP <ffffffff80189538>{generic_drop_inode+56} RSP <ffff81002f9d9b78>
CR2: 0000000000100108
 ----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at "fs/inode.c":1142
invalid operand: 0000 [2]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 9539, comm: rpciod/0 Tainted: P M 2.6.13

RIP: 0010:[<ffffffff8018853e>] <ffffffff8018853e>{iput+30}
RSP: 0018:ffff81002a181e08  EFLAGS: 00010246
RAX: ffffffff80494a00 RBX: ffff810023da8d40 RCX: ffff81002311f490
RDX: ffff810022919680 RSI: ffff81002614da40 RDI: ffff810023da8d40
RBP: ffff8100286af780 R08: ffff8100230f64e0 R09: 0000000000000000
R10: 0000000000000001 R11: ffffffff801f8c90 R12: ffff81002311f480
R13: ffff81002614da00 R14: ffff81002c3cec00 R15: ffffffff805c3fb0
FS:  00002aaaaade6b00(0000) GS:ffffffff805bc800(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00002aaaaade0680 CR3: 0000000022d19000 CR4: 00000000000006e0
Process rpciod/0 (pid: 9539, threadinfo ffff81002a180000, task ffff81002a4a2330)
Stack: ffff81002311f480 ffffffff801fb0fd 00000000fffffff3 ffff81002fb0f1c0
       ffff8100286af780 ffffffff801efa51 ffff81002fb0f1c0 ffff81002d9be310
       ffff81002fb0f2a8 ffff81002d9be320
Call Trace:<ffffffff801fb0fd>{nfs4_put_open_state+109}
<ffffffff801efa51>{nfs4_close_done+225}
       <ffffffff803940a5>{__rpc_execute+165}
<ffffffff8014046a>{worker_thread+442}
       <ffffffff8012e060>{default_wake_function+0}
<ffffffff8012e060>{default_wake_function+0}
       <ffffffff801402b0>{worker_thread+0} <ffffffff8014424d>{kthread+205}
       <ffffffff8010f3e6>{child_rip+8}
<ffffffff80144290>{keventd_create_kthread+0}
       <ffffffff80144180>{kthread+0} <ffffffff8010f3de>{child_rip+0}
Quoted 5 lines
Code: 0f 0b a3 e0 61 3c 80 ff ff ff ff c2 76 04 66 66 66 90 48 85

RIP <ffffffff8018853e>{iput+30} RSP <ffff81002a181e08>

after moving some files on the server to a new location then trying to add the files
to xmms playlist i found the following in dmesg after xmms froze wonder how many more items i can find...

nfs_update_inode: inode number mismatch
expected (0:11/0x27c27), got (0:11/0xe4c7d3) ----------- [cut here ] --------- [please bite here ] ---------

Kernel BUG at "fs/inode.c":1142
invalid operand: 0000 [1]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 11028, comm: gvim Tainted: P M 2.6.13

RIP: 0010:[<ffffffff8018853e>] <ffffffff8018853e>{iput+30}
RSP: 0018:ffff81000e999c98  EFLAGS: 00010246
RAX: ffffffff80494a00 RBX: ffff810012235d80 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff810012235d80 RDI: ffff810012235d80
RBP: ffff810012235d80 R08: ffff81002ccb3d40 R09: ffff81002ccb3cf8
R10: ffff81002ccb3d40 R11: ffffffff80154a40 R12: ffff81000e999d28
R13: ffff81000e999d38 R14: ffff81002cb08cc0 R15: 00007ffffff53b30
FS:  00002aaaafc47b80(0000) GS:ffffffff805bc800(0000) knlGS:00000000563ca620
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000511338 CR3: 0000000011d53000 CR4: 00000000000006e0
Process gvim (pid: 11028, threadinfo ffff81000e998000, task ffff81001a732a60)
Stack: ffff81002ccb3cf8 ffffffff801867c6 ffff81000e999d38 ffff81002ccb3cf8
       ffff81000e999e68 ffffffff8017d2ee 68706f7200000007 00000000fffffff5
       ffff81002fd24001 0000000000000000
Call Trace:<ffffffff801867c6>{dput+390} <ffffffff8017d2ee>{do_lookup+414}
       <ffffffff8017d801>{__link_path_walk+849}
<ffffffff8017e2aa>{link_path_walk+186}
       <ffffffff80229f7a>{strncpy_from_user+74}
<ffffffff8017e521>{path_lookup+385}
       <ffffffff8017e7ee>{__user_walk+62} <ffffffff80178d89>{vfs_stat+41}
       <ffffffff801791df>{sys_newstat+31} <ffffffff8010e8b6>{system_call+126}
Code: 0f 0b a3 e0 61 3c 80 ff ff ff ff c2 76 04 66 66 66 90 48 85
RIP <ffffffff8018853e>{iput+30} RSP <ffff81000e999c98>
 <0>general protection fault: 0000 [2]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 11074, comm: klauncher Tainted: P M 2.6.13

RIP: 0010:[<ffffffff801de262>] <ffffffff801de262>{nfs_lookup_revalidate+386}
RSP: 0018:ffff81001a341b18  EFLAGS: 00010282
RAX: ed05fe0eed52fe10 RBX: ffff81002195c638 RCX: 425f676209090945
RDX: 0000000000000001 RSI: ffff81001ba065a0 RDI: ffff81001ba065a0
RBP: ffff81001ba065a0 R08: 0000000000000000 R09: 000000000000000b
R10: ffff81002a4e58d8 R11: 0000000000000246 R12: ffff81001bd2d2d8
R13: ffff810012235270 R14: ffff81001a341e68 R15: 00007ffffff4b400
FS:  00002aaaaea99a00(0000) GS:ffffffff805bc800(0000) knlGS:00000000563ca620
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00002aaaab1c4520 CR3: 0000000018294000 CR4: 00000000000006e0
Process klauncher (pid: 11074, threadinfo ffff81001a340000, task
ffff81001ab2f440)
Stack: ffff81001a341cd8 00008001000002c7 30ba174331000000 0000000000000000
       0000000000000000 0000000000000000 ffff810013cf38d8 ffff81002e2a0a00
       0000000000000000 ffff810028b56700
Call Trace:<ffffffff801e2f85>{nfs_revalidate_inode+37}
<ffffffff801de34c>{nfs_lookup_revalidate+620}
       <ffffffff80394c90>{rpcauth_lookup_credcache+320}
<ffffffff80394c4b>{rpcauth_lookup_credcache+251}
       <ffffffff801de95d>{nfs_open_revalidate+269}
<ffffffff8017d2cb>{do_lookup+379}
       <ffffffff8017dd6d>{__link_path_walk+2237}
<ffffffff8017e2aa>{link_path_walk+186}
       <ffffffff80229f7a>{strncpy_from_user+74}
<ffffffff8017e521>{path_lookup+385}
       <ffffffff8017e7ee>{__user_walk+62} <ffffffff80178df6>{vfs_lstat+38}
       <ffffffff8017922f>{sys_newlstat+31} <ffffffff8010e8b6>{system_call+126}
Code: 48 8b b8 50 02 00 00 74 17 41 8b 46 20 a8 40 75 19 a8 14 75
RIP <ffffffff801de262>{nfs_lookup_revalidate+386} RSP <ffff81001a341b18>
 <0>general protection fault: 0000 [3]
CPU 0
Modules linked in: fglrx agpgart snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 11067, comm: konsole Tainted: P M 2.6.13

RIP: 0010:[<ffffffff801de262>] <ffffffff801de262>{nfs_lookup_revalidate+386}
RSP: 0018:ffff81000f4f1b18  EFLAGS: 00010282
RAX: ed05fe0eed52fe10 RBX: ffff81002195c638 RCX: 425f676209090945
RDX: 0000000000000001 RSI: ffff81001ba065a0 RDI: ffff81001ba065a0
RBP: ffff81001ba065a0 R08: 0000000000000000 R09: 000000000000000b
R10: ffff81002a4e58d8 R11: 0000000000000246 R12: ffff81001bd2d2d8
R13: ffff810012235270 R14: ffff81000f4f1e68 R15: 00007fffffc71c60
FS:  00002aaaaef4c0a0(0000) GS:ffffffff805bc800(0000) knlGS:00000000563ca620
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fffffc70c48 CR3: 000000001cab2000 CR4: 00000000000006e0
Process konsole (pid: 11067, threadinfo ffff81000f4f0000, task ffff810018467900)
Stack: ffff81000f4f1cd8 0000000000000001 ffff81002e347240 ffff810018467900
       ffff810018467938 0000000000000073 0000023459aab717 ffffffff803a4ee0
       ffff81000f4f1bd8 0000000000000082
Call Trace:<ffffffff803a4ee0>{thread_return+0}
<ffffffff801e2f85>{nfs_revalidate_inode+37}
       <ffffffff801de34c>{nfs_lookup_revalidate+620}
<ffffffff803a580e>{schedule_timeout+30}
       <ffffffff80394c4b>{rpcauth_lookup_credcache+251}
<ffffffff801de95d>{nfs_open_revalidate+269}
       <ffffffff8017d2cb>{do_lookup+379}
<ffffffff8017dd6d>{__link_path_walk+2237}
       <ffffffff8017e2aa>{link_path_walk+186}
<ffffffff80229f7a>{strncpy_from_user+74}
       <ffffffff8017e521>{path_lookup+385} <ffffffff8017e7ee>{__user_walk+62}
       <ffffffff80178df6>{vfs_lstat+38} <ffffffff8017922f>{sys_newlstat+31}
       <ffffffff8010e8b6>{system_call+126}
Code: 48 8b b8 50 02 00 00 74 17 41 8b 46 20 a8 40 75 19 a8 14 75

RIP <ffffffff801de262>{nfs_lookup_revalidate+386} RSP <ffff81000f4f1b18>

#2 lore raw

Bret Towe <[email protected]> :
[...]

Quoted 4 lines

after moving some files on the server to a new location then trying to add the files
to xmms playlist i found the following in dmesg after xmms froze wonder how many more items i can find...

The system includes some binary only stuff. Please contact your vendor or provide the traces for a configuration wherein the relevant module was not loaded after boot. It may make sense to get in touch with [email protected] then.

--
Ueimor

author
#3 lore raw
Quoted 11 lines · Francois Romieu

Bret Towe <[email protected]> :
[...]

Quoted 4 lines

after moving some files on the server to a new location then trying to add the files
to xmms playlist i found the following in dmesg after xmms froze wonder how many more items i can find...

The system includes some binary only stuff. Please contact your vendor or provide the traces for a configuration wherein the relevant module was not loaded after boot. It may make sense to get in touch with [email protected] then.

the 'binary only stuff' is ati-drivers kernel module and it crashs with or without it
ill provide a 'untainted' trace as soon as i can repeat the bug again

Quoted 3 lines

--
Ueimor

author
#4 lore raw
Quoted 8 lines · Bret Towe
Quoted 7 lines · Francois Romieu

Bret Towe <[email protected]> :
[...]

Quoted 4 lines

after moving some files on the server to a new location then trying to add the files
to xmms playlist i found the following in dmesg after xmms froze wonder how many more items i can find...

Quoted 8 lines
Quoted 4 lines

The system includes some binary only stuff. Please contact your vendor or provide the traces for a configuration wherein the relevant module was not loaded after boot. It may make sense to get in touch with [email protected] then.

the 'binary only stuff' is ati-drivers kernel module and it crashs with or without it
ill provide a 'untainted' trace as soon as i can repeat the bug again

ok without ati-drivers kernel module loaded the computer basicly just hard locks when
some bug hits dunno if its the same item

to repeat it tho one needs laptop-mode enabled have xmms playing music (flac in my case)
which resides on nfs then just put the computer under some local load for a little bit
till which im guessing it needs to clear some memory or somethin and it hits this hard lock
or the errors i mailed previously when ati-drivers is loaded

#5 lore raw
Quoted 14 lines · Bret Towe
Quoted 13 lines · Bret Towe
Quoted 11 lines · Francois Romieu

Bret Towe <[email protected]> :
[...]

Quoted 4 lines

after moving some files on the server to a new location then trying to add the files
to xmms playlist i found the following in dmesg after xmms froze wonder how many more items i can find...

The system includes some binary only stuff. Please contact your vendor or provide the traces for a configuration wherein the relevant module was not loaded after boot. It may make sense to get in touch with [email protected] then.

Quoted 7 lines
Quoted 3 lines

the 'binary only stuff' is ati-drivers kernel module and it crashs with or without it
ill provide a 'untainted' trace as soon as i can repeat the bug again

ok without ati-drivers kernel module loaded the computer basicly just hard locks when
some bug hits dunno if its the same item

Do you get anything from alt-sysrq-T?

Quoted 7 lines

to repeat it tho one needs laptop-mode enabled have xmms playing music (flac in my case)
which resides on nfs then just put the computer under some local load for a little bit
till which im guessing it needs to clear some memory or somethin and it hits this hard lock
or the errors i mailed previously when ati-drivers is loaded

What kernel version is this?

--b.

author
#6 lore raw
Quoted 19 lines · J. Bruce Fields
Quoted 18 lines · Bret Towe
Quoted 16 lines · Bret Towe
Quoted 11 lines · Francois Romieu

Bret Towe <[email protected]> :
[...]

Quoted 4 lines

after moving some files on the server to a new location then trying to add the files
to xmms playlist i found the following in dmesg after xmms froze wonder how many more items i can find...

The system includes some binary only stuff. Please contact your vendor or provide the traces for a configuration wherein the relevant module was not loaded after boot. It may make sense to get in touch with [email protected] then.

the 'binary only stuff' is ati-drivers kernel module and it crashs with or without it
ill provide a 'untainted' trace as soon as i can repeat the bug again

Quoted 5 lines
Quoted 3 lines

ok without ati-drivers kernel module loaded the computer basicly just hard locks when
some bug hits dunno if its the same item

Do you get anything from alt-sysrq-T?

no i havent used that im usally in x when its freezing
x wont even switch to console would it still give me anything then?

Quoted 9 lines
Quoted 7 lines

to repeat it tho one needs laptop-mode enabled have xmms playing music (flac in my case)
which resides on nfs then just put the computer under some local load for a little bit
till which im guessing it needs to clear some memory or somethin and it hits this hard lock
or the errors i mailed previously when ati-drivers is loaded

What kernel version is this?

2.6.13

Quoted 2 lines

--b.

#7 lore raw
Quoted 5 lines · Bret Towe
Quoted 1 line · J. Bruce Fields

Do you get anything from alt-sysrq-T?

no i havent used that im usally in x when its freezing
x wont even switch to console would it still give me anything then?

Well, you can try something like:
alt-sysrq-T
wait a couple seconds, then
alt-sysrq-S
alt-sysrq-U
alt-sysrq-B
with maybe a second between each to give stuff a chance to get to disk.

Then if you're lucky you may find the stack dumps in your log after you reboot.

--b.

author
#8 lore raw
Quoted 15 lines · J. Bruce Fields
Quoted 5 lines · Bret Towe
Quoted 1 line · J. Bruce Fields

Do you get anything from alt-sysrq-T?

no i havent used that im usally in x when its freezing
x wont even switch to console would it still give me anything then?

Well, you can try something like:
alt-sysrq-T
wait a couple seconds, then
alt-sysrq-S
alt-sysrq-U
alt-sysrq-B
with maybe a second between each to give stuff a chance to get to disk.

Quoted 2 lines

Then if you're lucky you may find the stack dumps in your log after you reboot.

tried it and so far no luck ill keep trying a few more times and see if i can get it
to spit somethin out to disk but i dont think ill be that lucky as that would prob make life to easy wouldnt it?

#9 lore raw
Quoted 16 lines · Bret Towe
Quoted 15 lines · J. Bruce Fields
Quoted 5 lines · Bret Towe
Quoted 1 line · J. Bruce Fields

Do you get anything from alt-sysrq-T?

no i havent used that im usally in x when its freezing
x wont even switch to console would it still give me anything then?

Well, you can try something like:
alt-sysrq-T
wait a couple seconds, then
alt-sysrq-S
alt-sysrq-U
alt-sysrq-B
with maybe a second between each to give stuff a chance to get to disk.

Quoted 7 lines
Quoted 2 lines

Then if you're lucky you may find the stack dumps in your log after you reboot.

tried it and so far no luck ill keep trying a few more times and see if i can get it
to spit somethin out to disk but i dont think ill be that lucky as that would prob make life to easy wouldnt it?

How about

serial console over a cross-over cable to a second box. netconsole will let you put the console on a different box over the network. console on line printer will let you have a permanent record of the console output on paper.

See
Documentation/serial-console.txt Documentation/networking/netconsole.txt
the help entry for "config LP_CONSOLE" (in drivers/char/Kconfig)

Would any of those perhaps help you in capturing anything ?

Signature · 3 lines
Jesper Juhl <[email protected]>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html
author
#10 lore raw
Quoted 32 lines · Jesper Juhl
Quoted 16 lines · Bret Towe
Quoted 15 lines · J. Bruce Fields
Quoted 5 lines · Bret Towe
Quoted 1 line · J. Bruce Fields

Do you get anything from alt-sysrq-T?

no i havent used that im usally in x when its freezing
x wont even switch to console would it still give me anything then?

Well, you can try something like:
alt-sysrq-T
wait a couple seconds, then
alt-sysrq-S
alt-sysrq-U
alt-sysrq-B
with maybe a second between each to give stuff a chance to get to disk.

Quoted 7 lines
Quoted 2 lines

Then if you're lucky you may find the stack dumps in your log after you reboot.

tried it and so far no luck ill keep trying a few more times and see if i can get it
to spit somethin out to disk but i dont think ill be that lucky as that would prob make life to easy wouldnt it?

How about

serial console over a cross-over cable to a second box. netconsole will let you put the console on a different box over the network. console on line printer will let you have a permanent record of the console output on paper.

Quoted 6 lines

See
Documentation/serial-console.txt Documentation/networking/netconsole.txt
the help entry for "config LP_CONSOLE" (in drivers/char/Kconfig)

Would any of those perhaps help you in capturing anything ?

netconsole++

got the following and ill be watching for anything that looks different as i continue to push my luck

NMI Watchdog detected LOCKUP on CPU0CPU 0

Modules linked in: netconsole snd_seq_midi snd_emu10k1_synth

snd_emux_synth snd_seq_virmidi snd_seq_midi_emul snd_pcm_oss snd_mixer_oss snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd_page_alloc snd_util_mem snd_hwdep snd w83627hf i2c_sensor i2c_isa i2c_core usb_storage r8169 ehci_hcd uhci_hcd dm_mirror dm_snapshot dm_mod
Pid: 14169, comm: xmms Tainted: G M 2.6.13

RIP: 0010:[<ffffffff80158d3b>] <ffffffff80158d3b>{cache_alloc_refill+347}
RSP: 0018:ffff81001306fa48  EFLAGS: 00000017
RAX: ffff81002f995c90 RBX: ffff81002f994800 RCX: ffff81000a8a9000
RDX: ffff81002f995c90 RSI: 000000000000000e RDI: ffff810005cfc028
RBP: ffff81002f995c80 R08: ffff81002f994810 R09: ffff81002f995ca0
R10: ffff81002f995cb0 R11: ffffffff80154a40 R12: ffff81002f995c90
R13: ffff81002f98d5c0 R14: 00000000000000d0 R15: ffffffff801e1550
FS:  0000000041802960(0063) GS:ffffffff805bc800(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00002aaaab351000 CR3: 00000000297c8000 CR4: 00000000000006e0
Process xmms (pid: 14169, threadinfo ffff81001306e000, task ffff81001beae0b0)
Stack: ffff81002fb0f1c0 0000000000000000 ffff81001306faa8 00000000000000d0
       ffff81002f995c80 ffff81002d450400 ffff81001306fb18 ffff810001bb2078
       ffffffff801e1550 ffffffff80158a7d
Call Trace:<ffffffff801e1550>{nfs_find_actor+0}
<ffffffff80158a7d>{kmem_cache_alloc+77}
       <ffffffff801e4305>{nfs_alloc_inode+21} <ffffffff80187fc2>{alloc_inode+18}
       <ffffffff80188e48>{iget5_locked+200}
<ffffffff801e15c0>{nfs_init_locked+0}
       <ffffffff801e191e>{nfs_fhget+110} <ffffffff801de5f5>{nfs_lookup+309}
       <ffffffff801eede7>{_nfs4_proc_access+215}
<ffffffff8017d235>{do_lookup+229}
       <ffffffff8017d801>{__link_path_walk+849}
<ffffffff8017e2aa>{link_path_walk+186}
       <ffffffff8016f668>{get_unused_fd+88} <ffffffff8017e521>{path_lookup+385}
       <ffffffff8017ed6f>{open_namei+175} <ffffffff8012d6e4>{deactivate_task+20}
       <ffffffff8016f5e7>{filp_open+39} <ffffffff8016f668>{get_unused_fd+88}
       <ffffffff8016f7b4>{sys_open+84} <ffffffff8010e8b6>{system_call+126}
Code: 4c 89 61 08 49 89 0c 24 85 f6 0f 8f 3c ff ff ff 8b 3b 89 f8

console shuts up ...
<0>Kernel panic - not syncing: Aiee, killing interrupt handler! <6>SysRq : Show State

sibling
task PC pid father child younger older
init S 000000010001d013 0 1 0 2 (NOTLB)
ffff810001c61d88 0000000000000082 0000000000000000 ffffffff8048c4b8

       ffff81001beaee10 ffff810001c5f440 ffff81001beaee10 ffff810001c5f658
       ffff810001c5f440 0000001001c61e68
Call Trace:<ffffffff803a5744>{schedule_timeout+148}
<ffffffff80139a30>{process_timeout+0}
       <ffffffff8017c052>{pipe_poll+66} <ffffffff80182d77>{do_select+967}
       <ffffffff801828c0>{__pollwait+0} <ffffffff801830bc>{sys_select+748}
       <ffffffff8010e8b6>{system_call+126}
ksoftirqd/0   S 0000000000000000     0     2      1             3       (L-TLB)
ffff810001c65f08 0000000000000046 ffff810001c65f18 ffff81002c8102b0
       ffff810026a85110 ffff810001c5ed90 ffff810026a85110 ffff810001c5efa8
       ffff810001c5f440 ffff810001c65f08
Call Trace:<ffffffff80135e60>{ksoftirqd+0} <ffffffff80135ea5>{ksoftirqd+69}
       <ffffffff80135e60>{ksoftirqd+0} <ffffffff8014424d>{kthread+205}
       <ffffffff8010f3e6>{child_rip+8} <ffffffff80144180>{kthread+0}
       <ffffffff8010f3de>{child_rip+0}
events/0      R  running task       0     3      1             4     2 (L-TLB)
khelper       S ffff810001c3b300     0     4      1             5     3 (L-TLB)
ffff810001ef9e78 0000000000000046 0000000000000000 ffffffff80140040
       ffff81002fbd0e90 ffff810001c5e030 ffff81002fbd0e90 ffff810001c5e248
       ffff810001c3b330 ffff810001c3b320
Call Trace:<ffffffff80140040>{__call_usermodehelper+0}
<ffffffff801403c6>{worker_thread+278}
       <ffffffff8012e060>{default_wake_function+0}
<ffffffff8012e060>{default_wake_function+0}
       <ffffffff801402b0>{worker_thread+0} <ffffffff8014424d>{kthread+205}
       <ffffffff8010f3e6>{child_rip+8} <ffffffff80144180>{kthread+0}
       <ffffffff8010f3de>{child_rip+0}
kthread       S ffff810001ec3300     0     5      1     7     147     4 (L-TLB)
ffff810001f1de78 0000000000000046 0000000000000000 ffff81002f9d5d28
       ffff81002f9d34c0 ffff810001f1b480 ffff81002f9d34c0 ffff810001f1b698
       ffff810001ec3330 ffff810001ec3320
Call Trace:<ffffffff801403c6>{worker_thread+278}
<ffffffff8012e060>{default_wake_function+0}
       <ffffffff8012e060>{default_wake_function+0}
<ffffffff801402b0>{worker_thread+0}
       <ffffffff8014424d>{kthread+205} <ffffffff8010f3e6>{child_rip+8}
       <ffffffff80144180>{kthread+0} <ffffffff8010f3de>{child_rip+0}

kacpid S ffff810001ec30c0 0 7 5 90 (L-TLB)
ffff81002f911e78 0000000000000046 403b0520522700f8 604000005101c501

       ffff810001c5f440 ffff810001f1add0 ffff810001c5f440 ffff810001f1afe8
       c408018023918081 0000000000010000
#12 lore raw
Quoted 5 lines · J. Bruce Fields
Quoted 1 line · Bret Towe

Pid: 14169, comm: xmms Tainted: G M 2.6.13

Hm, can someone explain what that means? A proprietary module was loaded then unloaded, maybe?

'M' means Machine Check, which sets the Tainted flag. So the processor thought that there was some kind of problem.

(/we needs to update Documentation/oops-tracing.txt)

Quoted 6 lines

You may also want to retest with

http://www.citi.umich.edu/projects/nfsv4/linux/kernel-patches/2.6.13-1/linux-2.6.13-001-NFS_ALL_MODIFIED.dif

applied, to make sure there isn't a patch in Trond's series that already fixes the bug.

Signature · 1 line
~Randy
#13 lore raw
Quoted 10 lines · Randy.Dunlap
Quoted 5 lines · J. Bruce Fields
Quoted 1 line · Bret Towe

Pid: 14169, comm: xmms Tainted: G M 2.6.13

Hm, can someone explain what that means? A proprietary module was loaded then unloaded, maybe?

'M' means Machine Check, which sets the Tainted flag. So the processor thought that there was some kind of problem.

Does this NMI watchdog event ("NMI Watchdog detected LOCKUP on CPU0CPU 0") set that flag?

Quoted 1 line

(/we needs to update Documentation/oops-tracing.txt)

Oops, thanks, I overlooked that!

--b.

#14 lore raw
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Randy.Dunlap
Quoted 3 lines · J. Bruce Fields
Quoted 1 line · Bret Towe

Pid: 14169, comm: xmms Tainted: G M 2.6.13

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Hm, can someone explain what that means? A proprietary module was loaded then unloaded, maybe?

'M' means Machine Check, which sets the Tainted flag. So the processor thought that there was some kind of problem.

Does this NMI watchdog event ("NMI Watchdog detected LOCKUP on CPU0CPU 0") set that flag?

Not that I can see.

There should be a logged MCE (machine check exception) in there somewhere AFAICT.

Quoted 5 lines
Quoted 1 line

(/we needs to update Documentation/oops-tracing.txt)

Oops, thanks, I overlooked that!

--b.

Signature · 1 line
~Randy
#15 lore raw
In reply to J. Bruce Fields
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Randy.Dunlap
Quoted 3 lines · J. Bruce Fields
Quoted 1 line · Bret Towe

Pid: 14169, comm: xmms Tainted: G M 2.6.13

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Hm, can someone explain what that means? A proprietary module was loaded then unloaded, maybe?

'M' means Machine Check, which sets the Tainted flag. So the processor thought that there was some kind of problem.

Does this NMI watchdog event ("NMI Watchdog detected LOCKUP on CPU0CPU 0") set that flag?

Not directly - but if the MCE wedged a processor, that could cause the NMI to fire complaining about a lockup. You should have a MCE logged someplace.

#16 lore raw

PATCH 1/1 Doc: update oops-tracing.txt (Tainted flags)

In reply to J. Bruce Fields

From: Randy Dunlap <[email protected]>

Update Documentation/oops-tracing.txt:
- add descriptions of 3 more "Tainted" flags;
- fix some typos;

---

 Documentation/oops-tracing.txt |   25 +++++++++++++++++--------
 1 files changed, 17 insertions(+), 8 deletions(-)
Documentation/oops-tracing.txt renamed from Documentation/oops-tracing.txt~doc_taint_update +17 -8
diff -Naurp linux-2613-work/Documentation/oops-tracing.txt~doc_taint_update linux-2613-work/Documentation/oops-tracing.txt
@@ -205,8 +205,8 @@ Phone: 701-234-7556
 Tainted kernels:
 
 Some oops reports contain the string 'Tainted: ' after the program
-counter, this indicates that the kernel has been tainted by some
-mechanism.  The string is followed by a series of position sensitive
+counter. This indicates that the kernel has been tainted by some
+mechanism.  The string is followed by a series of position-sensitive
 characters, each representing a particular tainted value.
 
   1: 'G' if all modules loaded have a GPL or compatible license, 'P' if
@@ -214,16 +214,25 @@ characters, each representing a particul
      MODULE_LICENSE or with a MODULE_LICENSE that is not recognised by
      insmod as GPL compatible are assumed to be proprietary.
 
-  2: 'F' if any module was force loaded by insmod -f, ' ' if all
+  2: 'F' if any module was force loaded by "insmod -f", ' ' if all
      modules were loaded normally.
 
   3: 'S' if the oops occurred on an SMP kernel running on hardware that
-      hasn't been certified as safe to run multiprocessor.
-	  Currently this occurs only on various Athlons that are not
-	  SMP capable.
+     hasn't been certified as safe to run multiprocessor.
+     Currently this occurs only on various Athlons that are not
+     SMP capable.
+
+  4: 'R' if a module was force unloaded by "rmmod -f", ' ' if all
+     modules were unloaded normally.
+
+  5: 'M' if any processor has reported a Machine Check Exception,
+     ' ' if no Machine Check Exceptions have occurred.
+
+  6: 'B' if a page-release function has found a bad page reference or
+     some unexpected page flags.
 
 The primary reason for the 'Tainted: ' string is to tell kernel
 debuggers if this is a clean kernel or if anything unusual has
-occurred.  Tainting is permanent, even if an offending module is
-unloading the tainted value remains to indicate that the kernel is not
+occurred.  Tainting is permanent: even if an offending module is
+unloaded, the tainted value remains to indicate that the kernel is not
 trustworthy.

---

author
#17 lore raw
In reply to J. Bruce Fields
Quoted 6 lines · J. Bruce Fields
Quoted 1 line · Bret Towe

Pid: 14169, comm: xmms Tainted: G M 2.6.13

Hm, can someone explain what that means? A proprietary module was loaded then unloaded, maybe?

Quoted 9 lines

You may also want to retest with

http://www.citi.umich.edu/projects/nfsv4/linux/kernel-patches/2.6.13-1/linux-2.6.13-001-NFS_ALL_MODIFIED.dif

applied, to make sure there isn't a patch in Trond's series that already fixes the bug.

--b.

ive been running this since i got the url and so far i havent hit it ive also been a bit busy so i havent been able to make sure its good this weekend i should be able to test it and make sure its solved

author
#18 lore raw
Quoted 14 lines · Bret Towe
Quoted 13 lines · J. Bruce Fields
Quoted 1 line · Bret Towe

Pid: 14169, comm: xmms Tainted: G M 2.6.13

Hm, can someone explain what that means? A proprietary module was loaded then unloaded, maybe?

You may also want to retest with

http://www.citi.umich.edu/projects/nfsv4/linux/kernel-patches/2.6.13-1/linux-2.6.13-001-NFS_ALL_MODIFIED.dif

applied, to make sure there isn't a patch in Trond's series that already fixes the bug.

Quoted 7 lines
Quoted 2 lines

--b.

ive been running this since i got the url and so far i havent hit it ive also been a bit busy so i havent been able to make sure its good this weekend i should be able to test it and make sure its solved

ran it pretty hard over the weekend and i had no crashs at all so i think its safe to say this patch fixes the issues i was seeing

#19 lore raw
Quoted 2 lines · Bret Towe

ran it pretty hard over the weekend and i had no crashs at all so i think its safe to say this patch fixes the issues i was seeing

Good, thanks for the testing.--b.