Call trace with 2.6.27.39
Hi
I have tested kernel 2.6.27.39 with fsstress, LVM, DD and snapshots and after a few hours I got call trace.I had 2 volumes and 10 snapshots for each volume.I created and deleted these snapshots from time to time and wrote to origin volumes with use DD from /dev/zero.Fsstress was run for 1 volume.
Filesystem "dm-41": xfs_log_force: error 5 returned.
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 IP: [<ffffffff802e99c3>] bdevname+0x3/0x20 PGD 0 Oops: 0000 [1] SMP CPU 0 Modules linked in: iscsi_scst scst_vdisk scst drbd twofish twofish_common serpent blowfish cast5 sha256_generic crypto_null iscsi_tcp libiscsi scsi_transport_iscsi bonding button thermal processor e1000e sg ftdi_sio usbserial nls_iso8859_1 nls_cp437 arcmsr uhci_hcd ohci_hcd ehci_hcd vfat fat aufs [last unloaded: ohci_hcd] Pid: 404, comm: xfslogd/0 Not tainted 2.6.27.39 #65 RIP: 0010:[<ffffffff802e99c3>] [<ffffffff802e99c3>] bdevname+0x3/0x20 RSP: 0000:ffff88021d957de8 EFLAGS: 00010286 RAX: ffff88021d957e00 RBX: ffff88007fbf6cc0 RCX: 0000000001770520 RDX: ffff88007fbf6cc0 RSI: ffff88021d957e00 RDI: 0000000000000000 RBP: ffff880207fe8800 R08: ffff88021d956000 R09: 0000000000000000 R10: 0000000000000000 R11: ffffffff803d5020 R12: 0000000001770520 R13: ffffffff8078cf02 R14: ffff88021d957e00 R15: 0000000000000005 FS: 0000000000000000(0000) GS:ffffffff80952080(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000000201000 CR4: 00000000000006a0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process xfslogd/0 (pid: 404, threadinfo ffff88021d956000, task ffff88021fafd890) Stack: ffffffff803e7526 ffff880028046970 ffffffff80817020 ffff88021d957e38 00000000ffff8b92 ffff88021fafdac0 ffff88021fafd890 000000001d957eb8 ffff880207fe8800 ffff88007fbf6cc0 ffff88021d937bc8 ffff88021d937bc8 Call Trace: [<ffffffff803e7526>] ? xfs_ioerror_alert+0x56/0xc0 [<ffffffff803d5060>] ? xlog_recover_iodone+0x40/0x90 [<ffffffff803eaa60>] ? xfs_buf_iodone_work+0x0/0x80 [<ffffffff8024692a>] ? run_workqueue+0x5a/0x110 [<ffffffff80246a7c>] ? worker_thread+0x9c/0xf0 [<ffffffff8024a7b0>] ? autoremove_wake_function+0x0/0x30 [<ffffffff8024a7b0>] ? autoremove_wake_function+0x0/0x30 [<ffffffff802469e0>] ? worker_thread+0x0/0xf0 [<ffffffff80249fcc>] ? kthread+0x6c/0xa0 [<ffffffff8020d1c9>] ? child_rip+0xa/0x11 [<ffffffff80249f60>] ? kthread+0x0/0xa0 [<ffffffff8020d1bf>] ? child_rip+0x0/0x11 Code: 13 00 48 89 d8 5b c3 48 89 df 48 c7 c2 ea b5 7a 80 be 20 00 00 00 31 c0 e8 cb e7 13 00 48 89 d8 5b c3 66 66 90 66 66 90 48 89 f0 <8b> 37 48 8b bf 98 00 00 00 48 89 c2 81 e6 ff ff 0f 00 2b 77 04 RIP [<ffffffff802e99c3>] bdevname+0x3/0x20 RSP <ffff88021d957de8> CR2: 0000000000000000 ---[ end trace e0541894570554ab ]---
Filesystem "dm-41": xfs_log_force: error 5 returned. Filesystem "dm-41": xfs_log_force: error 5 returned.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 6 lines · On Tue, Dec 29, 2009 at 07:18:32PM +0100, aluno3 wrote:
Hi
I have tested kernel 2.6.27.39 with fsstress, LVM, DD and snapshots and after a few hours I got call trace.I had 2 volumes and 10 snapshots for each volume.I created and deleted these snapshots from time to time and wrote to origin volumes with use DD from
/dev/zero.Fsstresswas run for 1 volume.Filesystem "dm-41":
xfs_log_force: error 5 returned.
What were the errors in the log before this? i.e. why did xfs_log_force() get an EIO error? It sounds like there's something already gone wrong before this point.
Quoted 2 lines
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 IP: [<ffffffff802e99c3>] bdevname+0x3/0x20
....
Quoted 8 lines
Process xfslogd/0 (pid: 404, threadinfo
ffff88021d956000, taskffff88021fafd890)Stack: ffffffff803e7526 ffff880028046970 ffffffff80817020 ffff88021d957e38 00000000ffff8b92 ffff88021fafdac0 ffff88021fafd890 000000001d957eb8 ffff880207fe8800 ffff88007fbf6cc0 ffff88021d937bc8 ffff88021d937bc8 Call Trace: [<ffffffff803e7526>] ? xfs_ioerror_alert+0x56/0xc0 [<ffffffff803d5060>] ? xlog_recover_iodone+0x40/0x90 [<ffffffff803eaa60>] ? xfs_buf_iodone_work+0x0/0x80
This has died during log recovery (on a different filesytem?), which implies that a buffer has not been set up properly with a buftarg or mount stucture. I can't see how that would happen from the code because each place that explicitly sets xlog_recover_iodone() as the completion also sets the bp->b_mount correctly. Still, there's been an IO error of some kind to get to xfs_ioerror_alert(), so it comes back to whether there were errors reported before this failure?
Also, were you mounting a snapshot when this occurred? If so, you shouldn't be doing log recovery on a snapshot because that writes to the snapshot - you should mount them with the "ro,norecovery" options to avoid this.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs