Having parent transid verify failed

DISCUSSION Konstantinos Skarlatos 4 participants 13 messages May 5, 2011 – Jun 2, 2011 1 message not in the archive Btrfs file system

lore

Share & tools

lore

Konstantinos Skarlatos

author

2011-05-05 11:19 UTC #0 lore raw

Hello, I have a 5.5TB Btrfs filesystem on top of a md-raid 5 device. Now if i run some file operations like find, i get these messages. kernel is 2.6.38.5-1 on arch linux

May  5 14:15:12 mail kernel: [13559.089713] parent transid verify failed on 3062073683968 wanted 5181 found 5188
May  5 14:15:12 mail kernel: [13559.089834] parent transid verify failed on 3062073683968 wanted 5181 found 5188
May  5 14:15:14 mail kernel: [13560.752074] btrfs-transacti D ffff88007211ac78     0  5339      2 0x00000000
May  5 14:15:14 mail kernel: [13560.752078]  ffff880023167d30 0000000000000046 ffff880000000000 ffff8800195b6000
May  5 14:15:14 mail kernel: [13560.752082]  ffff880023167c10 000002c8f27b4000 ffff880023167fd8 ffff88007211a9a0
May  5 14:15:14 mail kernel: [13560.752085]  ffff880023167fd8 ffff880023167fd8 ffff88007211ac80 ffff880023167fd8
May  5 14:15:14 mail kernel: [13560.752087] Call Trace:
May  5 14:15:14 mail kernel: [13560.752101]  [<ffffffffa0850d02>] ? run_clustered_refs+0x132/0x830 [btrfs]
May  5 14:15:14 mail kernel: [13560.752105]  [<ffffffff813aff3d>] schedule_timeout+0x2fd/0x380
May  5 14:15:14 mail kernel: [13560.752108]  [<ffffffff813b0cf9>] ? mutex_unlock+0x9/0x10
May  5 14:15:14 mail kernel: [13560.752115]  [<ffffffffa087e9f4>] ? btrfs_run_ordered_operations+0x1f4/0x210 [btrfs]
May  5 14:15:14 mail kernel: [13560.752122]  [<ffffffffa0860fa3>] btrfs_commit_transaction+0x263/0x750 [btrfs]
May  5 14:15:14 mail kernel: [13560.752126]  [<ffffffff81079ff0>] ? autoremove_wake_function+0x0/0x40
May  5 14:15:14 mail kernel: [13560.752131]  [<ffffffffa085a9bd>] transaction_kthread+0x26d/0x290 [btrfs]
May  5 14:15:14 mail kernel: [13560.752137]  [<ffffffffa085a750>] ? transaction_kthread+0x0/0x290 [btrfs]
May  5 14:15:14 mail kernel: [13560.752139]  [<ffffffff81079717>] kthread+0x87/0x90
May  5 14:15:14 mail kernel: [13560.752142]  [<ffffffff8100bc24>] kernel_thread_helper+0x4/0x10
May  5 14:15:14 mail kernel: [13560.752145]  [<ffffffff81079690>] ? kthread+0x0/0x90
May  5 14:15:14 mail kernel: [13560.752147]  [<ffffffff8100bc20>] ? kernel_thread_helper+0x0/0x10
May  5 14:15:17 mail kernel: [13564.092081] verify_parent_transid: 40736 callbacks suppressed
May  5 14:15:17 mail kernel: [13564.092084] parent transid verify failed on 3062073683968 wanted 5181 found 5188

--snip--

May  5 14:17:13 mail kernel: [13679.169772] parent transid verify failed on 3062073683968 wanted 5181 found 5188
--snip--
May  5 14:17:14 mail kernel: [13680.751996] btrfs-transacti D ffff88007211ac78     0  5339      2 0x00000000
May  5 14:17:14 mail kernel: [13680.752000]  ffff880023167d30 0000000000000046 ffff880000000000 ffff8800195b6000
May  5 14:17:14 mail kernel: [13680.752004]  ffff880023167c10 000002c8f27b4000 ffff880023167fd8 ffff88007211a9a0
May  5 14:17:14 mail kernel: [13680.752006]  ffff880023167fd8 ffff880023167fd8 ffff88007211ac80 ffff880023167fd8
May  5 14:17:14 mail kernel: [13680.752009] Call Trace:
May  5 14:17:14 mail kernel: [13680.752024]  [<ffffffffa0850d02>] ? run_clustered_refs+0x132/0x830 [btrfs]
May  5 14:17:14 mail kernel: [13680.752030]  [<ffffffff813aff3d>] schedule_timeout+0x2fd/0x380
May  5 14:17:14 mail kernel: [13680.752032]  [<ffffffff813b0cf9>] ? mutex_unlock+0x9/0x10
May  5 14:17:14 mail kernel: [13680.752040]  [<ffffffffa087e9f4>] ? btrfs_run_ordered_operations+0x1f4/0x210 [btrfs]
May  5 14:17:14 mail kernel: [13680.752046]  [<ffffffffa0860fa3>] btrfs_commit_transaction+0x263/0x750 [btrfs]
May  5 14:17:14 mail kernel: [13680.752051]  [<ffffffff81079ff0>] ? autoremove_wake_function+0x0/0x40
May  5 14:17:14 mail kernel: [13680.752057]  [<ffffffffa085a9bd>] transaction_kthread+0x26d/0x290 [btrfs]
May  5 14:17:14 mail kernel: [13680.752062]  [<ffffffffa085a750>] ? transaction_kthread+0x0/0x290 [btrfs]
May  5 14:17:14 mail kernel: [13680.752065]  [<ffffffff81079717>] kthread+0x87/0x90
May  5 14:17:14 mail kernel: [13680.752068]  [<ffffffff8100bc24>] kernel_thread_helper+0x4/0x10
May  5 14:17:14 mail kernel: [13680.752070]  [<ffffffff81079690>] ? kthread+0x0/0x90
May  5 14:17:14 mail kernel: [13680.752072]  [<ffffffff8100bc20>] ? kernel_thread_helper+0x0/0x10
May  5 14:17:14 mail kernel: [13680.752079] dd              D ffff8800714c4838     0  5792   5740 0x00000004
May  5 14:17:14 mail kernel: [13680.752082]  ffff88006a205b38 0000000000000082 ffff88006a205af8 0000000000000246
May  5 14:17:14 mail kernel: [13680.752085]  ffffea00017f57e8 ffff88006a205fd8 ffff88006a205fd8 ffff8800714c4560
May  5 14:17:14 mail kernel: [13680.752088]  ffff88006a205fd8 ffff88006a205fd8 ffff8800714c4840 ffff88006a205fd8
May  5 14:17:14 mail kernel: [13680.752090] Call Trace:
May  5 14:17:14 mail kernel: [13680.752095]  [<ffffffff810ff145>] ? zone_statistics+0x75/0x90
May  5 14:17:14 mail kernel: [13680.752098]  [<ffffffff810ea8b7>] ? get_page_from_freelist+0x3c7/0x820
May  5 14:17:14 mail kernel: [13680.752101]  [<ffffffff810e3588>] ? find_get_page+0x68/0xb0
May  5 14:17:14 mail kernel: [13680.752108]  [<ffffffffa08603f9>] wait_current_trans.isra.19+0x79/0xd0 [btrfs]
May  5 14:17:14 mail kernel: [13680.752110]  [<ffffffff81079ff0>] ? autoremove_wake_function+0x0/0x40
May  5 14:17:14 mail kernel: [13680.752117]  [<ffffffffa08618cf>] start_transaction+0xdf/0x270 [btrfs]
May  5 14:17:14 mail kernel: [13680.752123]  [<ffffffffa0861d9e>] btrfs_start_transaction+0xe/0x10 [btrfs]
May  5 14:17:14 mail kernel: [13680.752130]  [<ffffffffa086aa6b>] btrfs_setattr+0x10b/0x2b0 [btrfs]
May  5 14:17:14 mail kernel: [13680.752133]  [<ffffffff81159a91>] notify_change+0x181/0x350
May  5 14:17:14 mail kernel: [13680.752136]  [<ffffffff8113dd46>] do_truncate+0x56/0x90
May  5 14:17:14 mail kernel: [13680.752138]  [<ffffffff8114dde0>] finish_open+0x140/0x1b0
May  5 14:17:14 mail kernel: [13680.752140]  [<ffffffff8114dece>] do_last+0x7e/0x3f0
May  5 14:17:14 mail kernel: [13680.752142]  [<ffffffff8114e580>] do_filp_open+0x340/0x760
May  5 14:17:14 mail kernel: [13680.752145]  [<ffffffff81108cc1>] ? handle_mm_fault+0x1a1/0x360
May  5 14:17:14 mail kernel: [13680.752147]  [<ffffffff8113ee14>] do_sys_open+0x64/0x110
May  5 14:17:14 mail kernel: [13680.752149]  [<ffffffff8113f48e>] ? sys_lseek+0x6e/0x80
May  5 14:17:14 mail kernel: [13680.752151]  [<ffffffff8113eedb>] sys_open+0x1b/0x20
May  5 14:17:14 mail kernel: [13680.752153]  [<ffffffff8100ae12>] system_call_fastpath+0x16/0x1b
May  5 14:17:18 mail kernel: [13684.172013] verify_parent_transid: 40752 callbacks suppressed
May  5 14:17:18 mail kernel: [13684.172016] parent transid verify failed on 3062073683968 wanted 5181 found 5188

Chris Mason

2011-05-05 11:42 UTC #1 lore raw

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 07:19:52 -0400:

Quoted 3 lines

Hello, I have a 5.5TB Btrfs filesystem on top of a md-raid 5 device. Now if i run some file operations like find, i get these messages. kernel is 2.6.38.5-1 on arch linux

Are all of the messages for this one block?

parent transid verify failed on 3062073683968 wanted 5181 found 5188

-chris

Konstantinos Skarlatos

author

2011-05-05 11:45 UTC #2 lore raw

Quoted 1 line · Chris Mason

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 07:19:52=

-0400:

Quoted 1 line

Quoted 1 line
Hello, I have a 5.5TB Btrfs filesystem on top of a md-raid 5 device.=

Now

Quoted 6 lines

Quoted 2 lines
if i run some file operations like find, i get these messages. kernel is 2.6.38.5-1 on arch linux
Are all of the messages for this one block?
parent transid verify failed on 3062073683968 wanted 5181 found 5188

yes, only this block

Quoted 2 lines

-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Chris Mason

2011-05-05 14:23 UTC #3 lore raw

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 07:45:08 -= 0400:

Quoted 3 lines

=20
Quoted 1 line · Chris Mason
Excerpts from Konstantinos Skarlatos's message of 2011-05-05 07:19:=

52 -0400:

Quoted 1 line

Quoted 1 line
Quoted 1 line
Hello, I have a 5.5TB Btrfs filesystem on top of a md-raid 5 devic=

e. Now

Quoted 6 lines

Quoted 6 lines
Quoted 2 lines
if i run some file operations like find, i get these messages. kernel is 2.6.38.5-1 on arch linux
Are all of the messages for this one block?
parent transid verify failed on 3062073683968 wanted 5181 found 518=

Quoted 1 line

yes, only this block

Ok, what were the call traces in there? Was there an oops or a hung task? It looks like part of the messages are missing.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Chris Mason

2011-05-05 15:06 UTC #5 lore raw

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 10:27:30 -0400:

Quoted 2 lines

attached you can find the whole dmesg log. I can trigger the error again if more logs are needed

Yes, I'll send you a patch to get rid of the printk for the transid failed message. That way we can get a clean view of the other errors.

Will you be able to compile/test it?

-chris

Konstantinos Skarlatos

author

2011-05-05 16:07 UTC #6 lore raw

Quoted 1 line · Chris Mason

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 10:27:30=

-0400:

Quoted 1 line

Quoted 1 line
attached you can find the whole dmesg log. I can trigger the error a=

gain

Quoted 4 lines

Quoted 1 line
if more logs are needed
Yes, I'll send you a patch to get rid of the printk for the transid failed message. That way we can get a clean view of the other errors=

=2E

Quoted 2 lines

Will you be able to compile/test it?

Yes, i think i will be able to make it, but because i have only done=20 this once and in a quite hackish way, i may need some help in order to=20 do it right.

Quoted 2 lines

-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Konstantinos Skarlatos

author

2011-05-05 20:27 UTC #7 lore raw

In reply to Chris Mason

I think i made some progress. When i tried to remove the directory that= =20
i suspect contains the problematic file, i got this on the console

rm -rf serverloft/

2011 May  5 23:32:53 mail [  200.580195] Oops: 0000 [#1] PREEMPT SMP
2011 May  5 23:32:53 mail [  200.580220] last sysfs file:=20
/sys/module/vt/parameters/default_utf8
2011 May  5 23:32:53 mail [  200.581145] Stack:
2011 May  5 23:32:53 mail [  200.581276] Call Trace:
2011 May  5 23:32:53 mail [  200.581732] Code: cc 00 00 48 8d 91 28 e0=20

ff ff 48 89 e5 48 81 ec 90 00 00 00 48 89 5d d8 4c 89 65 e0 48 89 f3 4c= =20
89 6d e8 4c 89 75 f0 4c 89 7d f8 <48> 8b 76 30 83 42 1c 01 48 b8 00 00=20 00 00 00 16 00 00 48 01 f0
2011 May 5 23:32:53 mail [ 200.583376] CR2: 0000000000000030

here is the part of dmesg that does not contain the thousands of=20 parent transid verify failed messages

May  5 23:32:51 mail kernel: [  198.371084] parent transid verify faile=
d=20
on 3062073683968 wanted 5181 found 5188
May  5 23:32:51 mail kernel: [  198.371204] parent transid verify faile=
d=20
on 3062073683968 wanted 5181 found 5188
May  5 23:32:53 mail kernel: [  200.572774] Modules linked in: ipv6=20

btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov=20 async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_storag= e=20
uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq=20 snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_intel= =20
snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss=20 snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wmi=20 i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg=20 r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4=20 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata scs= i_mod

May  5 23:32:53 mail kernel: [  200.572808] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.572810] Call Trace:
May  5 23:32:53 mail kernel: [  200.572817]  [<ffffffff813a932b>] ?=20
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.572820]  [<ffffffff813af827>] ?=20
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.572823]  [<ffffffff811e5827>] ?=20
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.572827]  [<ffffffffa07ac164>] ?=20
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.572830]  [<ffffffffa07ac223>] ?=20
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.572833]  [<ffffffff81012d79>] ?=20
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.572837]  [<ffffffff8108418c>] ?=20
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.572840]  [<ffffffff810e36c0>] ?=20
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.572842]  [<ffffffff813af96e>] ?=20
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.572844]  [<ffffffff810e36fb>] ?=20
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.572846]  [<ffffffff813b0077>] ?=20
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.572848]  [<ffffffff810e38c0>] ?=20
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.572851]  [<ffffffff8107a030>] ?=20
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.572861]  [<ffffffffa08348d2>] ?=20
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.572867]  [<ffffffffa0809e00>] ?=20
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572873]  [<ffffffffa080ac7e>] ?=20
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572880]  [<ffffffffa080c0bc>] ?=20
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.572884]  [<ffffffffa07f272b>] ?=20
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.572890]  [<ffffffffa08417d1>] ?=20
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.572895]  [<ffffffffa07f5ca0>] ?=20
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.572900]  [<ffffffffa07fb3a6>] ?=20
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.572904]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.572908]  [<ffffffffa07fd452>] ?=20
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572914]  [<ffffffffa0800f59>] ?=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.572920]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.572925]  [<ffffffffa08014c0>] ?=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.572927]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.572933]  [<ffffffffa0810db8>] ?=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.572936]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.572941]  [<ffffffffa080a9bd>] ?=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.572947]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.572949]  [<ffffffff81079717>] ?=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.572952]  [<ffffffff8100bc24>] ?=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.572954]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.572956]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.572971] Modules linked in: ipv6=20

May  5 23:32:53 mail kernel: [  200.572996] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.572997] Call Trace:
May  5 23:32:53 mail kernel: [  200.572999]  [<ffffffff813a932b>] ?=20
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.573001]  [<ffffffff813af827>] ?=20
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.573003]  [<ffffffff811e5827>] ?=20
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.573006]  [<ffffffffa07ac164>] ?=20
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.573009]  [<ffffffffa07ac223>] ?=20
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.573011]  [<ffffffff81012d79>] ?=20
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.573013]  [<ffffffff8108418c>] ?=20
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.573016]  [<ffffffff810e36c0>] ?=20
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.573017]  [<ffffffff813af96e>] ?=20
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.573020]  [<ffffffff810e36fb>] ?=20
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.573021]  [<ffffffff813b0077>] ?=20
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.573024]  [<ffffffff810e38c0>] ?=20
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.573026]  [<ffffffff8107a030>] ?=20
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573032]  [<ffffffffa08348d2>] ?=20
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.573038]  [<ffffffffa0809e00>] ?=20
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573044]  [<ffffffffa080ac7e>] ?=20
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573050]  [<ffffffffa080c0bc>] ?=20
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573054]  [<ffffffffa07f272b>] ?=20
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.573060]  [<ffffffffa08417d1>] ?=20
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573065]  [<ffffffffa07f5ca0>] ?=20
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.573070]  [<ffffffffa07fb3a6>] ?=20
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.573072]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.573077]  [<ffffffffa07fd452>] ?=20
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573083]  [<ffffffffa0800f59>] ?=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.573088]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.573094]  [<ffffffffa08014c0>] ?=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.573096]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.573102]  [<ffffffffa0810db8>] ?=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.573104]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573110]  [<ffffffffa080a9bd>] ?=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573116]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573118]  [<ffffffff81079717>] ?=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.573120]  [<ffffffff8100bc24>] ?=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.573122]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.573124]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.573138] Modules linked in: ipv6=20

May  5 23:32:53 mail kernel: [  200.573162] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.573163] Call Trace:
May  5 23:32:53 mail kernel: [  200.573165]  [<ffffffff813a932b>] ?=20
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.573167]  [<ffffffff813af827>] ?=20
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.573169]  [<ffffffff811e5827>] ?=20
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.573172]  [<ffffffffa07ac164>] ?=20
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.573175]  [<ffffffffa07ac223>] ?=20
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.573177]  [<ffffffff81012d79>] ?=20
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.573179]  [<ffffffff8108418c>] ?=20
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.573182]  [<ffffffff810e36c0>] ?=20
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.573183]  [<ffffffff813af96e>] ?=20
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.573186]  [<ffffffff810e36fb>] ?=20
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.573187]  [<ffffffff813b0077>] ?=20
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.573190]  [<ffffffff810e38c0>] ?=20
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.573192]  [<ffffffff8107a030>] ?=20
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573198]  [<ffffffffa08348d2>] ?=20
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.573204]  [<ffffffffa0809e00>] ?=20
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573209]  [<ffffffffa080ac7e>] ?=20
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573215]  [<ffffffffa080c0bc>] ?=20
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573220]  [<ffffffffa07f272b>] ?=20
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.573226]  [<ffffffffa08417d1>] ?=20
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573231]  [<ffffffffa07f5ca0>] ?=20
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.573236]  [<ffffffffa07fb3a6>] ?=20
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.573238]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.573243]  [<ffffffffa07fd452>] ?=20
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573248]  [<ffffffffa0800f59>] ?=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.573254]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.573259]  [<ffffffffa08014c0>] ?=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.573261]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.573267]  [<ffffffffa0810db8>] ?=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.573270]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573275]  [<ffffffffa080a9bd>] ?=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573281]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573283]  [<ffffffff81079717>] ?=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.573285]  [<ffffffff8100bc24>] ?=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.573287]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.573289]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.580169] PGD 68be4067 PUD 68b42067 P=
MD 0
May  5 23:32:53 mail kernel: [  200.580250] CPU 2
May  5 23:32:53 mail kernel: [  200.580260] Modules linked in: ipv6=20

May  5 23:32:53 mail kernel: [  200.580642]
May  5 23:32:53 mail kernel: [  200.580652] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1 Gigabyte Technology Co., Ltd=
=2E=20
GA-MA785G-UD3H/GA-MA785G-UD3H
May  5 23:32:53 mail kernel: [  200.580713] RIP:=20
0010:[<ffffffffa0805192>]  [<ffffffffa0805192>]=20
btrfs_print_leaf+0x32/0x880 [btrfs]
May  5 23:32:53 mail kernel: [  200.580760] RSP: 0018:ffff88006dc97b00=20
EFLAGS: 00010286
May  5 23:32:53 mail kernel: [  200.580785] RAX: 00000000fffffffb RBX:=20
0000000000000000 RCX: ffff88006dc97fd8
May  5 23:32:53 mail kernel: [  200.580817] RDX: ffff88006dc96000 RSI:=20
0000000000000000 RDI: ffff880071cc6800
May  5 23:32:53 mail kernel: [  200.580849] RBP: ffff88006dc97b90 R08:=20
0000000000000000 R09: 0000000000000000
May  5 23:32:53 mail kernel: [  200.580881] R10: 0000000000000001 R11:=20
00000000fffffffb R12: 0000000000000007
May  5 23:32:53 mail kernel: [  200.580913] R13: 0000028b982f4000 R14:=20
00000000fffffffb R15: 0000000000001000
May  5 23:32:53 mail kernel: [  200.580946] FS:  00007f618e3d3720(0000)=
=20
GS:ffff880077b00000(0000) knlGS:0000000000000000
May  5 23:32:53 mail kernel: [  200.580982] CS:  0010 DS: 0000 ES: 0000=
=20
CR0: 000000008005003b
May  5 23:32:53 mail kernel: [  200.581008] CR2: 0000000000000030 CR3:=20
0000000068b62000 CR4: 00000000000006e0
May  5 23:32:53 mail kernel: [  200.581040] DR0: 0000000000000000 DR1:=20
0000000000000000 DR2: 0000000000000000
May  5 23:32:53 mail kernel: [  200.581073] DR3: 0000000000000000 DR6:=20
00000000ffff0ff0 DR7: 0000000000000400
May  5 23:32:53 mail kernel: [  200.581105] Process btrfs-transacti=20
(pid: 1037, threadinfo ffff88006dc96000, task ffff880071e51bc0)
May  5 23:32:53 mail kernel: [  200.581155]  0000000000002000=20
0000000000001000 ffff88006dc97c38 0000028b982f4000
May  5 23:32:53 mail kernel: [  200.581196]  0000000000000000=20
ffffffff8112b8d3 ffff880071cc6800 0000000000000090
May  5 23:32:53 mail kernel: [  200.581236]  0000000000000090=20
0000805070ac4120 a80000028b982f40 0000000000001000
May  5 23:32:53 mail kernel: [  200.581292]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.581324]  [<ffffffffa07fd9b7>]=20
__btrfs_free_extent+0x627/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.581359]  [<ffffffffa0800f59>]=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.581395]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.581431]  [<ffffffffa08014c0>]=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.581464]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.581493]  [<ffffffffa0810db8>]=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.581528]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.581562]  [<ffffffffa080a9bd>]=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.581597]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.581628]  [<ffffffff81079717>]=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.581652]  [<ffffffff8100bc24>]=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.581681]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.581704]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.583376]  RSP <ffff88006dc97b00>
May  5 23:32:53 mail kernel: [  200.587924] ---[ end trace=20
fed5a0f4104a02ee ]---
May  5 23:32:56 mail kernel: [  203.373416] verify_parent_transid: 4103=
0=20
callbacks suppressed
May  5 23:32:56 mail kernel: [  203.374625] parent transid verify faile=
d=20
on 3062073683968 wanted 5181 found 5188
May  5 23:32:56 mail kernel: [  203.376004] parent transid verify faile=

d=20
on 3062073683968 wanted 5181 found 5188

Quoted 1 line · Chris Mason

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 10:27:30=

-0400:

Quoted 1 line

Quoted 1 line
attached you can find the whole dmesg log. I can trigger the error a=

gain

Quoted 4 lines

Quoted 1 line
if more logs are needed
Yes, I'll send you a patch to get rid of the printk for the transid failed message. That way we can get a clean view of the other errors=

=2E

Quoted 4 lines

Will you be able to compile/test it?
-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Chris Mason

2011-05-05 20:32 UTC #8 lore raw

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 16:27:54 -= 0400:

Quoted 1 line

I think i made some progress. When i tried to remove the directory th=

at=20

Quoted 3 lines

i suspect contains the problematic file, i got this on the console =20
rm -rf serverloft/

Ok, our one bad block is in the extent allocation tree. This is going to be the very hardest thing to fix.

Until I finish off the code to rebuild parts of the extent allocation tree, I think your best bet is to copy the files off.

The big question is, what happened to make this error? Can you describ= e
your setup in more detail?

-chris

Quoted 7 lines

=20

2011 May  5 23:32:53 mail [  200.580195] Oops: 0000 [#1] PREEMPT SMP
2011 May  5 23:32:53 mail [  200.580220] last sysfs file:=20
/sys/module/vt/parameters/default_utf8
2011 May  5 23:32:53 mail [  200.581145] Stack:
2011 May  5 23:32:53 mail [  200.581276] Call Trace:
2011 May  5 23:32:53 mail [  200.581732] Code: cc 00 00 48 8d 91 28 e=

0=20

Quoted 1 line

ff ff 48 89 e5 48 81 ec 90 00 00 00 48 89 5d d8 4c 89 65 e0 48 89 f3 =

4c=20

Quoted 1 line

89 6d e8 4c 89 75 f0 4c 89 7d f8 <48> 8b 76 30 83 42 1c 01 48 b8 00 0=

0=20

Quoted 9 lines

00 00 00 16 00 00 48 01 f0
2011 May 5 23:32:53 mail [ 200.583376] CR2: 0000000000000030 =20
=20
here is the part of dmesg that does not contain the thousands of=20 parent transid verify failed messages
=20
=20
May  5 23:32:51 mail kernel: [  198.371084] parent transid verify fai=

led=20

Quoted 2 lines

on 3062073683968 wanted 5181 found 5188
May  5 23:32:51 mail kernel: [  198.371204] parent transid verify fai=

led=20

Quoted 4 lines

on 3062073683968 wanted 5181 found 5188
May  5 23:32:53 mail kernel: [  200.572774] Modules linked in: ipv6=20
btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov=20 async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_stor=

age=20

Quoted 2 lines

uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq=20 snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_int=

el=20

Quoted 2 lines

snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss=20 snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wmi=

=20

Quoted 3 lines

i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg=20 r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4=20 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata s=

csi_mod

Quoted 74 lines

May  5 23:32:53 mail kernel: [  200.572808] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.572810] Call Trace:
May  5 23:32:53 mail kernel: [  200.572817]  [<ffffffff813a932b>] ?=20
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.572820]  [<ffffffff813af827>] ?=20
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.572823]  [<ffffffff811e5827>] ?=20
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.572827]  [<ffffffffa07ac164>] ?=20
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.572830]  [<ffffffffa07ac223>] ?=20
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.572833]  [<ffffffff81012d79>] ?=20
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.572837]  [<ffffffff8108418c>] ?=20
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.572840]  [<ffffffff810e36c0>] ?=20
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.572842]  [<ffffffff813af96e>] ?=20
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.572844]  [<ffffffff810e36fb>] ?=20
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.572846]  [<ffffffff813b0077>] ?=20
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.572848]  [<ffffffff810e38c0>] ?=20
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.572851]  [<ffffffff8107a030>] ?=20
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.572861]  [<ffffffffa08348d2>] ?=20
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.572867]  [<ffffffffa0809e00>] ?=20
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572873]  [<ffffffffa080ac7e>] ?=20
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572880]  [<ffffffffa080c0bc>] ?=20
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.572884]  [<ffffffffa07f272b>] ?=20
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.572890]  [<ffffffffa08417d1>] ?=20
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.572895]  [<ffffffffa07f5ca0>] ?=20
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.572900]  [<ffffffffa07fb3a6>] ?=20
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.572904]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.572908]  [<ffffffffa07fd452>] ?=20
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572914]  [<ffffffffa0800f59>] ?=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.572920]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.572925]  [<ffffffffa08014c0>] ?=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.572927]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.572933]  [<ffffffffa0810db8>] ?=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.572936]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.572941]  [<ffffffffa080a9bd>] ?=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.572947]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.572949]  [<ffffffff81079717>] ?=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.572952]  [<ffffffff8100bc24>] ?=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.572954]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.572956]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.572971] Modules linked in: ipv6=20

Quoted 2 lines

btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov=20 async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_stor=

age=20

Quoted 2 lines

uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq=20 snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_int=

el=20

Quoted 2 lines

snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss=20 snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wmi=

=20

Quoted 3 lines

i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg=20 r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4=20 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata s=

csi_mod

Quoted 74 lines

May  5 23:32:53 mail kernel: [  200.572996] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.572997] Call Trace:
May  5 23:32:53 mail kernel: [  200.572999]  [<ffffffff813a932b>] ?=20
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.573001]  [<ffffffff813af827>] ?=20
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.573003]  [<ffffffff811e5827>] ?=20
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.573006]  [<ffffffffa07ac164>] ?=20
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.573009]  [<ffffffffa07ac223>] ?=20
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.573011]  [<ffffffff81012d79>] ?=20
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.573013]  [<ffffffff8108418c>] ?=20
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.573016]  [<ffffffff810e36c0>] ?=20
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.573017]  [<ffffffff813af96e>] ?=20
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.573020]  [<ffffffff810e36fb>] ?=20
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.573021]  [<ffffffff813b0077>] ?=20
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.573024]  [<ffffffff810e38c0>] ?=20
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.573026]  [<ffffffff8107a030>] ?=20
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573032]  [<ffffffffa08348d2>] ?=20
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.573038]  [<ffffffffa0809e00>] ?=20
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573044]  [<ffffffffa080ac7e>] ?=20
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573050]  [<ffffffffa080c0bc>] ?=20
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573054]  [<ffffffffa07f272b>] ?=20
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.573060]  [<ffffffffa08417d1>] ?=20
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573065]  [<ffffffffa07f5ca0>] ?=20
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.573070]  [<ffffffffa07fb3a6>] ?=20
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.573072]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.573077]  [<ffffffffa07fd452>] ?=20
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573083]  [<ffffffffa0800f59>] ?=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.573088]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.573094]  [<ffffffffa08014c0>] ?=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.573096]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.573102]  [<ffffffffa0810db8>] ?=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.573104]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573110]  [<ffffffffa080a9bd>] ?=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573116]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573118]  [<ffffffff81079717>] ?=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.573120]  [<ffffffff8100bc24>] ?=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.573122]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.573124]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.573138] Modules linked in: ipv6=20

Quoted 2 lines

btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov=20 async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_stor=

age=20

Quoted 2 lines

uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq=20 snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_int=

el=20

Quoted 2 lines

snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss=20 snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wmi=

=20

Quoted 3 lines

i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg=20 r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4=20 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata s=

csi_mod

Quoted 66 lines

May  5 23:32:53 mail kernel: [  200.573162] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.573163] Call Trace:
May  5 23:32:53 mail kernel: [  200.573165]  [<ffffffff813a932b>] ?=20
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.573167]  [<ffffffff813af827>] ?=20
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.573169]  [<ffffffff811e5827>] ?=20
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.573172]  [<ffffffffa07ac164>] ?=20
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.573175]  [<ffffffffa07ac223>] ?=20
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.573177]  [<ffffffff81012d79>] ?=20
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.573179]  [<ffffffff8108418c>] ?=20
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.573182]  [<ffffffff810e36c0>] ?=20
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.573183]  [<ffffffff813af96e>] ?=20
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.573186]  [<ffffffff810e36fb>] ?=20
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.573187]  [<ffffffff813b0077>] ?=20
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.573190]  [<ffffffff810e38c0>] ?=20
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.573192]  [<ffffffff8107a030>] ?=20
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573198]  [<ffffffffa08348d2>] ?=20
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.573204]  [<ffffffffa0809e00>] ?=20
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573209]  [<ffffffffa080ac7e>] ?=20
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573215]  [<ffffffffa080c0bc>] ?=20
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573220]  [<ffffffffa07f272b>] ?=20
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.573226]  [<ffffffffa08417d1>] ?=20
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573231]  [<ffffffffa07f5ca0>] ?=20
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.573236]  [<ffffffffa07fb3a6>] ?=20
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.573238]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.573243]  [<ffffffffa07fd452>] ?=20
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573248]  [<ffffffffa0800f59>] ?=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.573254]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.573259]  [<ffffffffa08014c0>] ?=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.573261]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.573267]  [<ffffffffa0810db8>] ?=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.573270]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573275]  [<ffffffffa080a9bd>] ?=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573281]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573283]  [<ffffffff81079717>] ?=20

Quoted 8 lines

kthread+0x87/0x90

May  5 23:32:53 mail kernel: [  200.573285]  [<ffffffff8100bc24>] ?=20
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.573287]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.573289]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.580169] PGD 68be4067 PUD 68b42067=

PMD 0

Quoted 4 lines

May  5 23:32:53 mail kernel: [  200.580250] CPU 2
May  5 23:32:53 mail kernel: [  200.580260] Modules linked in: ipv6=20
btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov=20 async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_stor=

age=20

Quoted 2 lines

uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq=20 snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_int=

el=20

Quoted 2 lines

snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss=20 snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wmi=

=20

Quoted 3 lines

i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg=20 r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4=20 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata s=

csi_mod

Quoted 3 lines

May  5 23:32:53 mail kernel: [  200.580642]
May  5 23:32:53 mail kernel: [  200.580652] Pid: 1037, comm:=20
btrfs-transacti Not tainted 2.6.38-ARCH #1 Gigabyte Technology Co., L=

td.=20

Quoted 5 lines

GA-MA785G-UD3H/GA-MA785G-UD3H

May  5 23:32:53 mail kernel: [  200.580713] RIP:=20
0010:[<ffffffffa0805192>]  [<ffffffffa0805192>]=20
btrfs_print_leaf+0x32/0x880 [btrfs]
May  5 23:32:53 mail kernel: [  200.580760] RSP: 0018:ffff88006dc97b0=

0=20

Quoted 2 lines

EFLAGS: 00010286

May  5 23:32:53 mail kernel: [  200.580785] RAX: 00000000fffffffb RBX=

:=20

Quoted 2 lines

0000000000000000 RCX: ffff88006dc97fd8
May  5 23:32:53 mail kernel: [  200.580817] RDX: ffff88006dc96000 RSI=

:=20

Quoted 2 lines

0000000000000000 RDI: ffff880071cc6800
May  5 23:32:53 mail kernel: [  200.580849] RBP: ffff88006dc97b90 R08=

:=20

Quoted 2 lines

0000000000000000 R09: 0000000000000000
May  5 23:32:53 mail kernel: [  200.580881] R10: 0000000000000001 R11=

:=20

Quoted 2 lines

00000000fffffffb R12: 0000000000000007
May  5 23:32:53 mail kernel: [  200.580913] R13: 0000028b982f4000 R14=

:=20

Quoted 2 lines

00000000fffffffb R15: 0000000000001000
May  5 23:32:53 mail kernel: [  200.580946] FS:  00007f618e3d3720(000=

0)=20

Quoted 2 lines

GS:ffff880077b00000(0000) knlGS:0000000000000000
May  5 23:32:53 mail kernel: [  200.580982] CS:  0010 DS: 0000 ES: 00=

00=20

Quoted 2 lines

CR0: 000000008005003b

May  5 23:32:53 mail kernel: [  200.581008] CR2: 0000000000000030 CR3=

:=20

Quoted 2 lines

0000000068b62000 CR4: 00000000000006e0
May  5 23:32:53 mail kernel: [  200.581040] DR0: 0000000000000000 DR1=

:=20

Quoted 2 lines

0000000000000000 DR2: 0000000000000000
May  5 23:32:53 mail kernel: [  200.581073] DR3: 0000000000000000 DR6=

:=20

Quoted 33 lines

00000000ffff0ff0 DR7: 0000000000000400

May  5 23:32:53 mail kernel: [  200.581105] Process btrfs-transacti=20
(pid: 1037, threadinfo ffff88006dc96000, task ffff880071e51bc0)
May  5 23:32:53 mail kernel: [  200.581155]  0000000000002000=20
0000000000001000 ffff88006dc97c38 0000028b982f4000
May  5 23:32:53 mail kernel: [  200.581196]  0000000000000000=20
ffffffff8112b8d3 ffff880071cc6800 0000000000000090
May  5 23:32:53 mail kernel: [  200.581236]  0000000000000090=20
0000805070ac4120 a80000028b982f40 0000000000001000
May  5 23:32:53 mail kernel: [  200.581292]  [<ffffffff8112b8d3>] ?=20
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.581324]  [<ffffffffa07fd9b7>]=20
__btrfs_free_extent+0x627/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.581359]  [<ffffffffa0800f59>]=20
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.581395]  [<ffffffffa084d900>] ?=20
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.581431]  [<ffffffffa08014c0>]=20
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.581464]  [<ffffffff813b0cf9>] ?=20
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.581493]  [<ffffffffa0810db8>]=20
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.581528]  [<ffffffff81079ff0>] ?=20
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.581562]  [<ffffffffa080a9bd>]=20
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.581597]  [<ffffffffa080a750>] ?=20
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.581628]  [<ffffffff81079717>]=20
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.581652]  [<ffffffff8100bc24>]=20

kernel_thread_helper+0x4/0x10

Quoted 8 lines

May  5 23:32:53 mail kernel: [  200.581681]  [<ffffffff81079690>] ?=20
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.581704]  [<ffffffff8100bc20>] ?=20
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.583376]  RSP <ffff88006dc97b00>
May  5 23:32:53 mail kernel: [  200.587924] ---[ end trace=20
fed5a0f4104a02ee ]---
May  5 23:32:56 mail kernel: [  203.373416] verify_parent_transid: 41=

030=20

Quoted 2 lines

callbacks suppressed

May  5 23:32:56 mail kernel: [  203.374625] parent transid verify fai=

led=20

Quoted 2 lines

on 3062073683968 wanted 5181 found 5188
May  5 23:32:56 mail kernel: [  203.376004] parent transid verify fai=

led=20

Quoted 4 lines

on 3062073683968 wanted 5181 found 5188
=20
Quoted 1 line · Chris Mason
Excerpts from Konstantinos Skarlatos's message of 2011-05-05 10:27:=

30 -0400:

Quoted 1 line

Quoted 1 line
Quoted 1 line
attached you can find the whole dmesg log. I can trigger the error=

again

Quoted 4 lines

Quoted 4 lines
Quoted 1 line
if more logs are needed
Yes, I'll send you a patch to get rid of the printk for the transid failed message. That way we can get a clean view of the other erro=

rs.

Quoted 4 lines

Quoted 4 lines
Will you be able to compile/test it?
-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Konstantinos Skarlatos

author

2011-05-05 21:04 UTC #9 lore raw

Quoted 1 line · Chris Mason

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 16:27:54=

-0400:

Quoted 1 line

Quoted 1 line
I think i made some progress. When i tried to remove the directory t=

hat

Quoted 5 lines

Quoted 3 lines
i suspect contains the problematic file, i got this on the console
rm -rf serverloft/
Ok, our one bad block is in the extent allocation tree. This is goin=

Quoted 6 lines

to be the very hardest thing to fix.
Until I finish off the code to rebuild parts of the extent allocation tree, I think your best bet is to copy the files off.
The big question is, what happened to make this error? Can you descr=

ibe

Quoted 1 line

your setup in more detail?

I created this btrfs filesystem on an arch linux system (amd64, quad=20 core) with kernel 2.3.38.1. it is on top of a md raid 5.

[root@linuxserver ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md0 : active raid5 sde1[3] sdc1[1] sda1[0] sdf1[4]

      5860535808 blocks super 1.2 level 5, 512k chunk, algorithm 2=20

[4/4] [UUUU]

the raid was grown from 3 devices to 4, and then btrfs was grown to max= =20
size. mount options were clear_cache,compress-force.

I was investigating a performance issue that i had, because over the=20 network i could only write to the filesystem at about 32mb/sec.

when writing btrfs-delalloc- cpu usage was at 100%.

While investigating i disabled compression, enabled space_cache and=20 tried zlib compression, and various combinations, while copying large=20 files back and forth using samba.

BTW I tried to change some mount options using mount -o remount but=20 although the new options were printed on dmesg i think that they were=20 not enabled.

I got the first error when i was copying some files and at the same tim= e=20
created a directory over samba. After a while i upgraded to 2.6.38.5 bu= t=20
nothing seems to have changed.

I really dont think there is a hardware error here, but to be safe I am= =20
now running a check on the raid

Quoted 10 lines

-chris

Quoted 7 lines

2011 May  5 23:32:53 mail [  200.580195] Oops: 0000 [#1] PREEMPT SMP
2011 May  5 23:32:53 mail [  200.580220] last sysfs file:
/sys/module/vt/parameters/default_utf8
2011 May  5 23:32:53 mail [  200.581145] Stack:
2011 May  5 23:32:53 mail [  200.581276] Call Trace:
2011 May  5 23:32:53 mail [  200.581732] Code: cc 00 00 48 8d 91 28 =

Quoted 1 line

Quoted 1 line
ff ff 48 89 e5 48 81 ec 90 00 00 00 48 89 5d d8 4c 89 65 e0 48 89 f3=

Quoted 1 line

Quoted 1 line
89 6d e8 4c 89 75 f0 4c 89 7d f8<48> 8b 76 30 83 42 1c 01 48 b8 00 =

Quoted 9 lines

Quoted 9 lines
00 00 00 16 00 00 48 01 f0
2011 May 5 23:32:53 mail [ 200.583376] CR2: 0000000000000030
here is the part of dmesg that does not contain the thousands of parent transid verify failed messages
May  5 23:32:51 mail kernel: [  198.371084] parent transid verify fa=

iled

Quoted 2 lines

Quoted 2 lines
on 3062073683968 wanted 5181 found 5188
May  5 23:32:51 mail kernel: [  198.371204] parent transid verify fa=

iled

Quoted 4 lines

Quoted 4 lines
on 3062073683968 wanted 5181 found 5188
May  5 23:32:53 mail kernel: [  200.572774] Modules linked in: ipv6
btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_sto=

rage

Quoted 2 lines

Quoted 2 lines
uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_in=

tel

Quoted 2 lines

Quoted 2 lines
snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wm=

Quoted 3 lines

Quoted 3 lines
i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata =

scsi_mod

Quoted 68 lines

May  5 23:32:53 mail kernel: [  200.572808] Pid: 1037, comm:
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.572810] Call Trace:
May  5 23:32:53 mail kernel: [  200.572817]  [<ffffffff813a932b>] ?
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.572820]  [<ffffffff813af827>] ?
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.572823]  [<ffffffff811e5827>] ?
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.572827]  [<ffffffffa07ac164>] ?
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.572830]  [<ffffffffa07ac223>] ?
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.572833]  [<ffffffff81012d79>] ?
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.572837]  [<ffffffff8108418c>] ?
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.572840]  [<ffffffff810e36c0>] ?
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.572842]  [<ffffffff813af96e>] ?
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.572844]  [<ffffffff810e36fb>] ?
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.572846]  [<ffffffff813b0077>] ?
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.572848]  [<ffffffff810e38c0>] ?
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.572851]  [<ffffffff8107a030>] ?
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.572861]  [<ffffffffa08348d2>] ?
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.572867]  [<ffffffffa0809e00>] ?
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572873]  [<ffffffffa080ac7e>] ?
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572880]  [<ffffffffa080c0bc>] ?
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.572884]  [<ffffffffa07f272b>] ?
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.572890]  [<ffffffffa08417d1>] ?
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.572895]  [<ffffffffa07f5ca0>] ?
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.572900]  [<ffffffffa07fb3a6>] ?
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.572904]  [<ffffffff8112b8d3>] ?
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.572908]  [<ffffffffa07fd452>] ?
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.572914]  [<ffffffffa0800f59>] ?
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.572920]  [<ffffffffa084d900>] ?
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.572925]  [<ffffffffa08014c0>] ?
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.572927]  [<ffffffff813b0cf9>] ?
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.572933]  [<ffffffffa0810db8>] ?
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.572936]  [<ffffffff81079ff0>] ?
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.572941]  [<ffffffffa080a9bd>] ?
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.572947]  [<ffffffffa080a750>] ?
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.572949]  [<ffffffff81079717>] ?
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.572952]  [<ffffffff8100bc24>] ?

Quoted 8 lines

Quoted 8 lines
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.572954]  [<ffffffff81079690>] ?
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.572956]  [<ffffffff8100bc20>] ?
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.572971] Modules linked in: ipv6
btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_sto=

rage

Quoted 2 lines

Quoted 2 lines
uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_in=

tel

Quoted 2 lines

Quoted 2 lines
snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wm=

Quoted 3 lines

Quoted 3 lines
i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata =

scsi_mod

Quoted 68 lines

May  5 23:32:53 mail kernel: [  200.572996] Pid: 1037, comm:
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.572997] Call Trace:
May  5 23:32:53 mail kernel: [  200.572999]  [<ffffffff813a932b>] ?
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.573001]  [<ffffffff813af827>] ?
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.573003]  [<ffffffff811e5827>] ?
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.573006]  [<ffffffffa07ac164>] ?
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.573009]  [<ffffffffa07ac223>] ?
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.573011]  [<ffffffff81012d79>] ?
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.573013]  [<ffffffff8108418c>] ?
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.573016]  [<ffffffff810e36c0>] ?
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.573017]  [<ffffffff813af96e>] ?
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.573020]  [<ffffffff810e36fb>] ?
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.573021]  [<ffffffff813b0077>] ?
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.573024]  [<ffffffff810e38c0>] ?
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.573026]  [<ffffffff8107a030>] ?
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573032]  [<ffffffffa08348d2>] ?
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.573038]  [<ffffffffa0809e00>] ?
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573044]  [<ffffffffa080ac7e>] ?
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573050]  [<ffffffffa080c0bc>] ?
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573054]  [<ffffffffa07f272b>] ?
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.573060]  [<ffffffffa08417d1>] ?
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573065]  [<ffffffffa07f5ca0>] ?
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.573070]  [<ffffffffa07fb3a6>] ?
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.573072]  [<ffffffff8112b8d3>] ?
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.573077]  [<ffffffffa07fd452>] ?
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573083]  [<ffffffffa0800f59>] ?
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.573088]  [<ffffffffa084d900>] ?
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.573094]  [<ffffffffa08014c0>] ?
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.573096]  [<ffffffff813b0cf9>] ?
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.573102]  [<ffffffffa0810db8>] ?
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.573104]  [<ffffffff81079ff0>] ?
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573110]  [<ffffffffa080a9bd>] ?
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573116]  [<ffffffffa080a750>] ?
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573118]  [<ffffffff81079717>] ?
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.573120]  [<ffffffff8100bc24>] ?

Quoted 8 lines

Quoted 8 lines
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.573122]  [<ffffffff81079690>] ?
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.573124]  [<ffffffff8100bc20>] ?
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.573138] Modules linked in: ipv6
btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_sto=

rage

Quoted 2 lines

Quoted 2 lines
uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_in=

tel

Quoted 2 lines

Quoted 2 lines
snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wm=

Quoted 3 lines

Quoted 3 lines
i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata =

scsi_mod

Quoted 66 lines

May  5 23:32:53 mail kernel: [  200.573162] Pid: 1037, comm:
btrfs-transacti Not tainted 2.6.38-ARCH #1
May  5 23:32:53 mail kernel: [  200.573163] Call Trace:
May  5 23:32:53 mail kernel: [  200.573165]  [<ffffffff813a932b>] ?
__schedule_bug+0x59/0x5d
May  5 23:32:53 mail kernel: [  200.573167]  [<ffffffff813af827>] ?
schedule+0x9f7/0xad0
May  5 23:32:53 mail kernel: [  200.573169]  [<ffffffff811e5827>] ?
generic_unplug_device+0x37/0x40
May  5 23:32:53 mail kernel: [  200.573172]  [<ffffffffa07ac164>] ?
md_raid5_unplug_device+0x64/0x110 [raid456]
May  5 23:32:53 mail kernel: [  200.573175]  [<ffffffffa07ac223>] ?
raid5_unplug_queue+0x13/0x20 [raid456]
May  5 23:32:53 mail kernel: [  200.573177]  [<ffffffff81012d79>] ?
read_tsc+0x9/0x20
May  5 23:32:53 mail kernel: [  200.573179]  [<ffffffff8108418c>] ?
ktime_get_ts+0xac/0xe0
May  5 23:32:53 mail kernel: [  200.573182]  [<ffffffff810e36c0>] ?
sync_page+0x0/0x50
May  5 23:32:53 mail kernel: [  200.573183]  [<ffffffff813af96e>] ?
io_schedule+0x6e/0xb0
May  5 23:32:53 mail kernel: [  200.573186]  [<ffffffff810e36fb>] ?
sync_page+0x3b/0x50
May  5 23:32:53 mail kernel: [  200.573187]  [<ffffffff813b0077>] ?
__wait_on_bit+0x57/0x80
May  5 23:32:53 mail kernel: [  200.573190]  [<ffffffff810e38c0>] ?
wait_on_page_bit+0x70/0x80
May  5 23:32:53 mail kernel: [  200.573192]  [<ffffffff8107a030>] ?
wake_bit_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573198]  [<ffffffffa08348d2>] ?
read_extent_buffer_pages+0x412/0x480 [btrfs]
May  5 23:32:53 mail kernel: [  200.573204]  [<ffffffffa0809e00>] ?
btree_get_extent+0x0/0x1b0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573209]  [<ffffffffa080ac7e>] ?
btree_read_extent_buffer_pages.isra.60+0x5e/0xb0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573215]  [<ffffffffa080c0bc>] ?
read_tree_block+0x3c/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573220]  [<ffffffffa07f272b>] ?
read_block_for_search.isra.34+0x1fb/0x410 [btrfs]
May  5 23:32:53 mail kernel: [  200.573226]  [<ffffffffa08417d1>] ?
btrfs_tree_unlock+0x51/0x60 [btrfs]
May  5 23:32:53 mail kernel: [  200.573231]  [<ffffffffa07f5ca0>] ?
btrfs_search_slot+0x430/0xa30 [btrfs]
May  5 23:32:53 mail kernel: [  200.573236]  [<ffffffffa07fb3a6>] ?
lookup_inline_extent_backref+0x96/0x460 [btrfs]
May  5 23:32:53 mail kernel: [  200.573238]  [<ffffffff8112b8d3>] ?
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.573243]  [<ffffffffa07fd452>] ?
__btrfs_free_extent+0xc2/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.573248]  [<ffffffffa0800f59>] ?
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.573254]  [<ffffffffa084d900>] ?
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.573259]  [<ffffffffa08014c0>] ?
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.573261]  [<ffffffff813b0cf9>] ?
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.573267]  [<ffffffffa0810db8>] ?
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.573270]  [<ffffffff81079ff0>] ?
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.573275]  [<ffffffffa080a9bd>] ?
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573281]  [<ffffffffa080a750>] ?
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.573283]  [<ffffffff81079717>] ?

Quoted 8 lines

kthread+0x87/0x90

May  5 23:32:53 mail kernel: [  200.573285]  [<ffffffff8100bc24>] ?
kernel_thread_helper+0x4/0x10
May  5 23:32:53 mail kernel: [  200.573287]  [<ffffffff81079690>] ?
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.573289]  [<ffffffff8100bc20>] ?
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.580169] PGD 68be4067 PUD 68b4206=

7 PMD 0

Quoted 4 lines

Quoted 4 lines
May  5 23:32:53 mail kernel: [  200.580250] CPU 2
May  5 23:32:53 mail kernel: [  200.580260] Modules linked in: ipv6
btrfs zlib_deflate crc32c libcrc32c ext2 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx md_mod usb_sto=

rage

Quoted 2 lines

Quoted 2 lines
uas snd_seq_dummy snd_seq_oss radeon snd_seq_midi_event ttm snd_seq snd_hda_codec_hdmi snd_seq_device drm_kms_helper ohci_hcd snd_hda_in=

tel

Quoted 2 lines

Quoted 2 lines
snd_hda_codec snd_pcm_oss snd_hwdep drm i2c_algo_bit snd_mixer_oss snd_pcm i2c_piix4 snd_timer snd soundcore snd_page_alloc ehci_hcd wm=

Quoted 3 lines

Quoted 3 lines
i2c_core usbcore evdev processor button k10temp serio_raw pcspkr sg r8169 edac_core shpchp pci_hotplug edac_mce_amd mii sp5100_tco ext4 mbcache jbd2 crc16 sd_mod pata_acpi ahci libahci pata_atiixp libata =

scsi_mod

Quoted 3 lines

Quoted 3 lines
May  5 23:32:53 mail kernel: [  200.580642]
May  5 23:32:53 mail kernel: [  200.580652] Pid: 1037, comm:
btrfs-transacti Not tainted 2.6.38-ARCH #1 Gigabyte Technology Co., =

Ltd.

Quoted 5 lines

GA-MA785G-UD3H/GA-MA785G-UD3H

May  5 23:32:53 mail kernel: [  200.580713] RIP:
0010:[<ffffffffa0805192>]  [<ffffffffa0805192>]
btrfs_print_leaf+0x32/0x880 [btrfs]
May  5 23:32:53 mail kernel: [  200.580760] RSP: 0018:ffff88006dc97b=

Quoted 2 lines

Quoted 2 lines
EFLAGS: 00010286
May  5 23:32:53 mail kernel: [  200.580785] RAX: 00000000fffffffb RB=

Quoted 2 lines

Quoted 2 lines
0000000000000000 RCX: ffff88006dc97fd8
May  5 23:32:53 mail kernel: [  200.580817] RDX: ffff88006dc96000 RS=

Quoted 2 lines

Quoted 2 lines
0000000000000000 RDI: ffff880071cc6800
May  5 23:32:53 mail kernel: [  200.580849] RBP: ffff88006dc97b90 R0=

Quoted 2 lines

Quoted 2 lines
0000000000000000 R09: 0000000000000000
May  5 23:32:53 mail kernel: [  200.580881] R10: 0000000000000001 R1=

Quoted 2 lines

Quoted 2 lines
00000000fffffffb R12: 0000000000000007
May  5 23:32:53 mail kernel: [  200.580913] R13: 0000028b982f4000 R1=

Quoted 2 lines

Quoted 2 lines
00000000fffffffb R15: 0000000000001000
May  5 23:32:53 mail kernel: [  200.580946] FS:  00007f618e3d3720(00=

00)

Quoted 2 lines

Quoted 2 lines
GS:ffff880077b00000(0000) knlGS:0000000000000000
May  5 23:32:53 mail kernel: [  200.580982] CS:  0010 DS: 0000 ES: 0=

000

Quoted 2 lines

Quoted 2 lines
CR0: 000000008005003b
May  5 23:32:53 mail kernel: [  200.581008] CR2: 0000000000000030 CR=

Quoted 2 lines

Quoted 2 lines
0000000068b62000 CR4: 00000000000006e0
May  5 23:32:53 mail kernel: [  200.581040] DR0: 0000000000000000 DR=

Quoted 2 lines

Quoted 2 lines
0000000000000000 DR2: 0000000000000000
May  5 23:32:53 mail kernel: [  200.581073] DR3: 0000000000000000 DR=

Quoted 33 lines

00000000ffff0ff0 DR7: 0000000000000400

May  5 23:32:53 mail kernel: [  200.581105] Process btrfs-transacti
(pid: 1037, threadinfo ffff88006dc96000, task ffff880071e51bc0)
May  5 23:32:53 mail kernel: [  200.581155]  0000000000002000
0000000000001000 ffff88006dc97c38 0000028b982f4000
May  5 23:32:53 mail kernel: [  200.581196]  0000000000000000
ffffffff8112b8d3 ffff880071cc6800 0000000000000090
May  5 23:32:53 mail kernel: [  200.581236]  0000000000000090
0000805070ac4120 a80000028b982f40 0000000000001000
May  5 23:32:53 mail kernel: [  200.581292]  [<ffffffff8112b8d3>] ?
kmem_cache_alloc+0x133/0x150
May  5 23:32:53 mail kernel: [  200.581324]  [<ffffffffa07fd9b7>]
__btrfs_free_extent+0x627/0x6d0 [btrfs]
May  5 23:32:53 mail kernel: [  200.581359]  [<ffffffffa0800f59>]
run_clustered_refs+0x389/0x830 [btrfs]
May  5 23:32:53 mail kernel: [  200.581395]  [<ffffffffa084d900>] ?
btrfs_find_ref_cluster+0x10/0x190 [btrfs]
May  5 23:32:53 mail kernel: [  200.581431]  [<ffffffffa08014c0>]
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
May  5 23:32:53 mail kernel: [  200.581464]  [<ffffffff813b0cf9>] ?
mutex_unlock+0x9/0x10
May  5 23:32:53 mail kernel: [  200.581493]  [<ffffffffa0810db8>]
btrfs_commit_transaction+0x78/0x750 [btrfs]
May  5 23:32:53 mail kernel: [  200.581528]  [<ffffffff81079ff0>] ?
autoremove_wake_function+0x0/0x40
May  5 23:32:53 mail kernel: [  200.581562]  [<ffffffffa080a9bd>]
transaction_kthread+0x26d/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.581597]  [<ffffffffa080a750>] ?
transaction_kthread+0x0/0x290 [btrfs]
May  5 23:32:53 mail kernel: [  200.581628]  [<ffffffff81079717>]
kthread+0x87/0x90
May  5 23:32:53 mail kernel: [  200.581652]  [<ffffffff8100bc24>]

kernel_thread_helper+0x4/0x10

Quoted 8 lines

May  5 23:32:53 mail kernel: [  200.581681]  [<ffffffff81079690>] ?
kthread+0x0/0x90
May  5 23:32:53 mail kernel: [  200.581704]  [<ffffffff8100bc20>] ?
kernel_thread_helper+0x0/0x10
May  5 23:32:53 mail kernel: [  200.583376]  RSP<ffff88006dc97b00>
May  5 23:32:53 mail kernel: [  200.587924] ---[ end trace
fed5a0f4104a02ee ]---
May  5 23:32:56 mail kernel: [  203.373416] verify_parent_transid: 4=

1030

Quoted 2 lines

Quoted 2 lines
callbacks suppressed
May  5 23:32:56 mail kernel: [  203.374625] parent transid verify fa=

iled

Quoted 2 lines

Quoted 2 lines
on 3062073683968 wanted 5181 found 5188
May  5 23:32:56 mail kernel: [  203.376004] parent transid verify fa=

iled

Quoted 4 lines

Quoted 4 lines
on 3062073683968 wanted 5181 found 5188
Quoted 1 line · Chris Mason
Excerpts from Konstantinos Skarlatos's message of 2011-05-05 10:27:=

30 -0400:

Quoted 1 line

Quoted 1 line
Quoted 1 line
Quoted 1 line
attached you can find the whole dmesg log. I can trigger the error=

again

Quoted 4 lines

Quoted 4 lines
Quoted 4 lines
Quoted 1 line
if more logs are needed
Yes, I'll send you a patch to get rid of the printk for the transid failed message. That way we can get a clean view of the other erro=

rs.

Quoted 4 lines

Quoted 4 lines
Quoted 4 lines
Will you be able to compile/test it?
-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Chris Mason

2011-05-05 23:50 UTC #10 lore raw

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 17:04:00 -= 0400:

Quoted 2 lines · Chris Mason

Quoted 1 line
Excerpts from Konstantinos Skarlatos's message of 2011-05-05 16:27:=

54 -0400:

Quoted 1 line

Quoted 1 line
Quoted 1 line
I think i made some progress. When i tried to remove the directory=

that

Quoted 5 lines

Quoted 5 lines
Quoted 3 lines
i suspect contains the problematic file, i got this on the console
rm -rf serverloft/
Ok, our one bad block is in the extent allocation tree. This is go=

ing

Quoted 3 lines

Quoted 3 lines
to be the very hardest thing to fix.
Until I finish off the code to rebuild parts of the extent allocati=

Quoted 3 lines

Quoted 3 lines
tree, I think your best bet is to copy the files off.
The big question is, what happened to make this error? Can you des=

cribe

Quoted 8 lines

Quoted 1 line
your setup in more detail?
=20
I created this btrfs filesystem on an arch linux system (amd64, quad=20 core) with kernel 2.3.38.1. it is on top of a md raid 5.
=20
[root@linuxserver ~]# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde1[3] sdc1[1] sda1[0] sdf1[4]

Quoted 4 lines

       5860535808 blocks super 1.2 level 5, 512k chunk, algorithm 2=20
[4/4] [UUUU]
=20
the raid was grown from 3 devices to 4, and then btrfs was grown to m=

ax=20

Quoted 9 lines

size. mount options were clear_cache,compress-force.
=20
I was investigating a performance issue that i had, because over the=20 network i could only write to the filesystem at about 32mb/sec. =20
when writing btrfs-delalloc- cpu usage was at 100%.
=20
While investigating i disabled compression, enabled space_cache and=20 tried zlib compression, and various combinations, while copying large=

=20

Quoted 4 lines

files back and forth using samba.
=20
BTW I tried to change some mount options using mount -o remount but=20 although the new options were printed on dmesg i think that they were=

=20

Quoted 3 lines

not enabled.
=20
I got the first error when i was copying some files and at the same t=

ime=20

Quoted 1 line

created a directory over samba. After a while i upgraded to 2.6.38.5 =

but=20

Quoted 3 lines

nothing seems to have changed.
=20
I really dont think there is a hardware error here, but to be safe I =

am=20

Quoted 1 line

now running a check on the raid

This error basically means we didn't write the block. It could be because the write went to the wrong spot, or the hardware stack messed it up, or because of a btrfs bug. But, 2.6.38 is relatively recent. I= t
doesn't look like memory corruption because the transids are fairly close.

When you grew the raid device, did you grow a partition as well? We've had trouble in the past with block dev flushing code kicking in as devices are resized.

Samba isn't doing anything exotic, and 2.6.38 has my recent fixes for rare metadata corruption bugs in btrfs.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Peter Stuge

2011-05-06 00:01 UTC #11 lore raw

Quoted 2 lines · Chris Mason

We've had trouble in the past with block dev flushing code kicking in as devices are resized.

Might this be the problem with my root node? I wish my problem was in only one directory. :)

//Peter

Konstantinos Skarlatos

author

2011-05-06 05:58 UTC #12 lore raw

In reply to Chris Mason

Quoted 1 line · Chris Mason

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 17:04:00=

-0400:

Quoted 2 lines

Quoted 2 lines · Chris Mason
Quoted 1 line
Excerpts from Konstantinos Skarlatos's message of 2011-05-05 16:27:=

54 -0400:

Quoted 1 line

Quoted 1 line
Quoted 1 line
Quoted 1 line
I think i made some progress. When i tried to remove the directory=

that

Quoted 5 lines

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
i suspect contains the problematic file, i got this on the console
rm -rf serverloft/
Ok, our one bad block is in the extent allocation tree. This is go=

ing

Quoted 3 lines

Quoted 3 lines
Quoted 3 lines
to be the very hardest thing to fix.
Until I finish off the code to rebuild parts of the extent allocati=

Quoted 3 lines

Quoted 3 lines
Quoted 3 lines
tree, I think your best bet is to copy the files off.
The big question is, what happened to make this error? Can you des=

cribe

Quoted 5 lines

Quoted 5 lines
Quoted 1 line
your setup in more detail?
I created this btrfs filesystem on an arch linux system (amd64, quad core) with kernel 2.3.38.1. it is on top of a md raid 5.

Quoted 7 lines

Quoted 7 lines
[root@linuxserver ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md0 : active raid5 sde1[3] sdc1[1] sda1[0] sdf1[4]
        5860535808 blocks super 1.2 level 5, 512k chunk, algorithm 2
[4/4] [UUUU]
the raid was grown from 3 devices to 4, and then btrfs was grown to =

max

Quoted 9 lines

Quoted 9 lines
size. mount options were clear_cache,compress-force.
I was investigating a performance issue that i had, because over the network i could only write to the filesystem at about 32mb/sec.
when writing btrfs-delalloc- cpu usage was at 100%.
While investigating i disabled compression, enabled space_cache and tried zlib compression, and various combinations, while copying larg=

Quoted 4 lines

Quoted 4 lines
files back and forth using samba.
BTW I tried to change some mount options using mount -o remount but although the new options were printed on dmesg i think that they wer=

Quoted 3 lines

Quoted 3 lines
not enabled.
I got the first error when i was copying some files and at the same =

time

Quoted 1 line

Quoted 1 line
created a directory over samba. After a while i upgraded to 2.6.38.5=

but

Quoted 3 lines

Quoted 3 lines
nothing seems to have changed.
I really dont think there is a hardware error here, but to be safe I=

Quoted 4 lines

Quoted 1 line
now running a check on the raid
This error basically means we didn't write the block. It could be because the write went to the wrong spot, or the hardware stack messe=

Quoted 1 line

it up, or because of a btrfs bug. But, 2.6.38 is relatively recent. =

Quoted 4 lines

doesn't look like memory corruption because the transids are fairly close.
When you grew the raid device, did you grow a partition as well? We'=

Quoted 2 lines

had trouble in the past with block dev flushing code kicking in as devices are resized.

no, I did not grow any partitions, I just added one disk to the Raid 5=20 md0 device, and then grew the btrfs filesystem to max size(no partition= s=20
on md0).

I can remember that as a test (to see if shrink works) i shrank the fs=20 by 1 gb and then grew it again to max size.

Quoted 5 lines

Samba isn't doing anything exotic, and 2.6.38 has my recent fixes for rare metadata corruption bugs in btrfs.
-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Johannes Hirte

2011-06-02 16:36 UTC #13 lore raw

In reply to Chris Mason

Quoted 9 lines · On Thursday 05 May 2011 22:32:42 Chris Mason wrote:

Excerpts from Konstantinos Skarlatos's message of 2011-05-05 16:27:54 -0400:
Quoted 4 lines
I think i made some progress. When i tried to remove the directory that i suspect contains the problematic file, i got this on the console
rm -rf serverloft/
Ok, our one bad block is in the extent allocation tree. This is going to be the very hardest thing to fix.

Quoted 7 lines

Until I finish off the code to rebuild parts of the extent allocation tree, I think your best bet is to copy the files off.
The big question is, what happened to make this error? Can you describe your setup in more detail?
-chris

It seems that I run into the same problem:

parent transid verify failed on 32940560384 wanted 210334 found 210342

BUG: scheduling while atomic: chrome/17058/0x00000002
Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device

snd_pcm_oss snd_mixer_oss fuse dm_crypt dm_mod usbhid snd_intel8x0 snd_ac97_codec sr_mod cdrom ac97_bus snd_pcm sg snd_timer snd e1000 fschmd uhci_hcd snd_page_alloc i2c_i801 [last unloaded: microcode]
Pid: 17058, comm: chrome Tainted: G W 2.6.39 #29

Call Trace:
 [<c13cf70c>] ? schedule+0x78/0x6ef
 [<c11acabb>] ? generic_make_request+0x1d5/0x22f
 [<c11acbad>] ? submit_bio+0x98/0x9f
 [<c118026a>] ? btrfs_map_bio+0x1ab/0x1b5
 [<c13cfdc2>] ? io_schedule+0x3f/0x50
 [<c105723d>] ? sleep_on_page+0x5/0x8
 [<c13d0292>] ? __wait_on_bit+0x31/0x58
 [<c1057238>] ? __lock_page+0x52/0x52
 [<c1057388>] ? wait_on_page_bit+0x5a/0x62
 [<c1037f92>] ? autoremove_wake_function+0x29/0x29
 [<c117ab39>] ? read_extent_buffer_pages+0x33a/0x3b5
 [<c115891f>] ? btree_read_extent_buffer_pages.clone.51+0x44/0x9e
 [<c11578b0>] ? verify_parent_transid+0x147/0x147
 [<c11593aa>] ? read_tree_block+0x2d/0x3e
 [<c1144f90>] ? read_block_for_search.clone.36+0xc3/0x35d
 [<c11863bf>] ? btrfs_tree_unlock+0x19/0x3a
 [<c11420bb>] ? unlock_up+0x88/0x9f
 [<c1146f7e>] ? btrfs_search_slot+0x39d/0x4fe
 [<c1149fa1>] ? lookup_inline_extent_backref+0x116/0x49b
 [<c11773b0>] ? set_extent_dirty+0x19/0x1d
 [<c114cbd0>] ? __btrfs_free_extent+0xe2/0x6c6
 [<c114fa28>] ? run_clustered_refs+0x6ad/0x720
 [<c1191330>] ? btrfs_find_ref_cluster+0x53/0x11f
 [<c114fb53>] ? btrfs_run_delayed_refs+0xb8/0x18d
 [<c115d395>] ? __btrfs_end_transaction+0x5a/0x17f
 [<c115d4dc>] ? btrfs_end_transaction+0x9/0xb
 [<c1165e19>] ? btrfs_evict_inode+0x190/0x1a7
 [<c1092c45>] ? evict+0x56/0xeb
 [<c108baa8>] ? do_unlinkat+0xc3/0x103
 [<c13d1c90>] ? sysenter_do_call+0x12/0x26
 [<c13d0000>] ? console_conditional_schedule+0x8/0xf
parent transid verify failed on 32940560384 wanted 210334 found 210342
BUG: scheduling while atomic: chrome/17058/0x00000002
Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device

Call Trace:
 [<c13cf70c>] ? schedule+0x78/0x6ef
 [<c11acabb>] ? generic_make_request+0x1d5/0x22f
 [<c11acbad>] ? submit_bio+0x98/0x9f
 [<c118026a>] ? btrfs_map_bio+0x1ab/0x1b5
 [<c13cfdc2>] ? io_schedule+0x3f/0x50
 [<c105723d>] ? sleep_on_page+0x5/0x8
 [<c13d0292>] ? __wait_on_bit+0x31/0x58
 [<c1057238>] ? __lock_page+0x52/0x52
 [<c1057388>] ? wait_on_page_bit+0x5a/0x62
 [<c1037f92>] ? autoremove_wake_function+0x29/0x29
 [<c117ab39>] ? read_extent_buffer_pages+0x33a/0x3b5
 [<c116bd50>] ? lookup_extent_mapping+0x5a/0x148
 [<c115891f>] ? btree_read_extent_buffer_pages.clone.51+0x44/0x9e
 [<c11578b0>] ? verify_parent_transid+0x147/0x147
 [<c11593aa>] ? read_tree_block+0x2d/0x3e
 [<c1144f90>] ? read_block_for_search.clone.36+0xc3/0x35d
 [<c11863bf>] ? btrfs_tree_unlock+0x19/0x3a
 [<c11420bb>] ? unlock_up+0x88/0x9f
 [<c1146f7e>] ? btrfs_search_slot+0x39d/0x4fe
 [<c1149fa1>] ? lookup_inline_extent_backref+0x116/0x49b
 [<c11773b0>] ? set_extent_dirty+0x19/0x1d
 [<c114cbd0>] ? __btrfs_free_extent+0xe2/0x6c6
 [<c114fa28>] ? run_clustered_refs+0x6ad/0x720
 [<c1191330>] ? btrfs_find_ref_cluster+0x53/0x11f
 [<c114fb53>] ? btrfs_run_delayed_refs+0xb8/0x18d
 [<c115d395>] ? __btrfs_end_transaction+0x5a/0x17f
 [<c115d4dc>] ? btrfs_end_transaction+0x9/0xb
 [<c1165e19>] ? btrfs_evict_inode+0x190/0x1a7
 [<c1092c45>] ? evict+0x56/0xeb
 [<c108baa8>] ? do_unlinkat+0xc3/0x103
 [<c13d1c90>] ? sysenter_do_call+0x12/0x26
 [<c13d0000>] ? console_conditional_schedule+0x8/0xf
parent transid verify failed on 32940560384 wanted 210334 found 210342
BUG: scheduling while atomic: chrome/17058/0x00000002
Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device

Call Trace:
 [<c13cf70c>] ? schedule+0x78/0x6ef
 [<c11acabb>] ? generic_make_request+0x1d5/0x22f
 [<c11acbad>] ? submit_bio+0x98/0x9f
 [<c118026a>] ? btrfs_map_bio+0x1ab/0x1b5
 [<c13cfdc2>] ? io_schedule+0x3f/0x50
 [<c105723d>] ? sleep_on_page+0x5/0x8
 [<c13d0292>] ? __wait_on_bit+0x31/0x58
 [<c1057238>] ? __lock_page+0x52/0x52
 [<c1057388>] ? wait_on_page_bit+0x5a/0x62
 [<c1037f92>] ? autoremove_wake_function+0x29/0x29
 [<c117ab39>] ? read_extent_buffer_pages+0x33a/0x3b5
 [<c116bd50>] ? lookup_extent_mapping+0x5a/0x148
 [<c115891f>] ? btree_read_extent_buffer_pages.clone.51+0x44/0x9e
 [<c11578b0>] ? verify_parent_transid+0x147/0x147
 [<c11593aa>] ? read_tree_block+0x2d/0x3e
 [<c1144f90>] ? read_block_for_search.clone.36+0xc3/0x35d
 [<c11863bf>] ? btrfs_tree_unlock+0x19/0x3a
 [<c11420bb>] ? unlock_up+0x88/0x9f
 [<c1146f7e>] ? btrfs_search_slot+0x39d/0x4fe
 [<c1149fa1>] ? lookup_inline_extent_backref+0x116/0x49b
 [<c11773b0>] ? set_extent_dirty+0x19/0x1d
 [<c114cbd0>] ? __btrfs_free_extent+0xe2/0x6c6
 [<c114fa28>] ? run_clustered_refs+0x6ad/0x720
 [<c1191330>] ? btrfs_find_ref_cluster+0x53/0x11f
 [<c114fb53>] ? btrfs_run_delayed_refs+0xb8/0x18d
 [<c115d395>] ? __btrfs_end_transaction+0x5a/0x17f
 [<c115d4dc>] ? btrfs_end_transaction+0x9/0xb
 [<c1165e19>] ? btrfs_evict_inode+0x190/0x1a7
 [<c1092c45>] ? evict+0x56/0xeb
 [<c108baa8>] ? do_unlinkat+0xc3/0x103
 [<c13d1c90>] ? sysenter_do_call+0x12/0x26
 [<c13d0000>] ? console_conditional_schedule+0x8/0xf
parent transid verify failed on 32940560384 wanted 210334 found 210342
parent transid verify failed on 32940560384 wanted 210334 found 210342
BUG: unable to handle kernel NULL pointer dereference at 0000001c
IP: [<c1153acd>] btrfs_print_leaf+0xd/0x76e
*pdpt = 00000000220ee001 *pde = 0000000000000000 
Oops: 0000 [#1] PREEMPT SMP 
last sysfs file: /sys/devices/virtual/vtconsole/vtcon1/uevent
Modules linked in: snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device

Pid: 17058, comm: chrome Tainted: G W 2.6.39 #29 FUJITSU SIEMENS
SCENIC P / SCENICO P/D1561
EIP: 0060:[<c1153acd>] EFLAGS: 00010296 CPU: 1
EIP is at btrfs_print_leaf+0xd/0x76e
EAX: f6414800 EBX: f6414800 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: f602b1c0 EBP: fffffffb ESP: d63ddd44
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process chrome (pid: 17058, ti=d63dc000 task=e2375b00 task.ti=d63dc000)

Stack:
 00000000 00000000 00000000 f602b1c0 00000000 c114cbd0 d63ddde4 d5d86000
 00000022 00001000 00000000 00000000 00000000 0000014e 00000000 0000014e
 00000000 f602b1c0 fffffffb c114ce06 00000000 00000000 00000026 f6414800
Call Trace:
 [<c114cbd0>] ? __btrfs_free_extent+0xe2/0x6c6
 [<c114ce06>] ? __btrfs_free_extent+0x318/0x6c6
 [<c114fa28>] ? run_clustered_refs+0x6ad/0x720
 [<c1191330>] ? btrfs_find_ref_cluster+0x53/0x11f
 [<c114fb53>] ? btrfs_run_delayed_refs+0xb8/0x18d
 [<c115d395>] ? __btrfs_end_transaction+0x5a/0x17f
 [<c115d4dc>] ? btrfs_end_transaction+0x9/0xb
 [<c1165e19>] ? btrfs_evict_inode+0x190/0x1a7
 [<c1092c45>] ? evict+0x56/0xeb
 [<c108baa8>] ? do_unlinkat+0xc3/0x103
 [<c13d1c90>] ? sysenter_do_call+0x12/0x26
 [<c13d0000>] ? console_conditional_schedule+0x8/0xf
Code: 01 00 8b 0c 24 51 56 53 55 57 52 50 68 bd b0 4b c1 e8 87 b9 27 00 83 c4

2c 5b 5e 5f 5d c3 55 57 56 53 83 ec 3c 89 c3 89 54 24 04 <8b> 42 1c e8 68 8e ec ff 8b 50 60 89 54 24 24 e8 f4 8c ec ff 8b
EIP: [<c1153acd>] btrfs_print_leaf+0xd/0x76e SS:ESP 0068:d63ddd44
CR2: 000000000000001c

---[ end trace 8a0695cc90b08578 ]---

The setup is a single disk installation with one filesystem and three subvolumes (default one, one for / and one for /home). When it first time happened, the kernel was 2.6.39-rc6 and only the home-subvolume was affected. Now I get a panic when booting the system.
If you need more infos, please tell me.

regards,
Johannes