Warning and BUG using btrfs-vol -b
I was playing with btrfs with 2 files of 3.5 GB (using loop), I completely zeroed one of the files. As expected, I had checksum failures, and I run btrfs-vol -b just to see what happened, and I got this (using -rc3):
[25765.340492] btrfs csum failed ino 260 off 122880 csum 2566472073 private 3262812575 [25765.340530] btrfs csum failed ino 260 off 118784 csum 2566472073 private 1515197691 [25765.340560] btrfs csum failed ino 260 off 114688 csum 2566472073 private 1467250717 [25765.340589] btrfs csum failed ino 260 off 110592 csum 2566472073 private 551456760 [25765.340618] btrfs csum failed ino 260 off 106496 csum 2566472073 private 2751401677 [25765.340647] btrfs csum failed ino 260 off 102400 csum 2566472073 private 2648834377 [25765.340675] btrfs csum failed ino 260 off 98304 csum 2566472073 private 467965630 [25765.340703] btrfs csum failed ino 260 off 94208 csum 2566472073 private 3164409631 [25765.340732] btrfs csum failed ino 260 off 90112 csum 2566472073 private 2042500030 [25765.340760] btrfs csum failed ino 260 off 86016 csum 2566472073 private 830495785 [25809.292651] btrfs: relocating block group 3541041152 flags 9 [25809.416493] __ratelimit: 158 callbacks suppressed [25809.416498] btrfs csum failed ino 257 off 61440 csum 2566472073 private 1989812792 [25809.416537] btrfs csum failed ino 257 off 57344 csum 2566472073 private 3105762401 [25809.416603] btrfs csum failed ino 257 off 53248 csum 2566472073 private 2215503813 [25809.416719] btrfs csum failed ino 257 off 323584 csum 2566472073 private 883932373 [25809.416731] btrfs csum failed ino 257 off 49152 csum 2566472073 private 1747807236 [25809.416752] btrfs csum failed ino 257 off 319488 csum 2566472073 private 2519068844 [25809.416774] btrfs csum failed ino 257 off 45056 csum 2566472073 private 3487590687 [25809.416783] btrfs csum failed ino 257 off 315392 csum 2566472073 private 2599709027 [25809.416813] btrfs csum failed ino 257 off 311296 csum 2566472073 private 157132003 [25809.416823] btrfs csum failed ino 257 off 40960 csum 2566472073 private 599615997 [25809.657271] ------------[ cut here ]------------ [25809.657294] WARNING: at fs/btrfs/relocation.c:3588 btrfs_relocate_block_group+0x27f/0x390 [btrfs]() [25809.657298] Hardware name: System Product Name [25809.657300] Modules linked in: btrfs loop crc32c libcrc32c ppp_deflate zlib_deflate bsd_comp ppp_async crc_ccitt ppp_generic slhc kvm_intel kvm i915 snd_ca0106 snd_rawmidi drm_kms_helper cfbcopyarea snd_ac97_codec ac97_bus video snd_pcm_oss snd_mixer_oss snd_pcm backlight snd_timer cfbimgblt snd processor button cfbfillrect intel_agp wacom soundcore snd_page_alloc usbhid fan fuse ehci_hcd uhci_hcd thermal thermal_sys usbcore atl1 mii [last unloaded: btrfs] [25809.657350] Pid: 15354, comm: btrfs-vol Not tainted 2.6.32-rc3 #1 [25809.657353] Call Trace: [25809.657361] [<ffffffff8104f9bb>] warn_slowpath_common+0x7b/0xc0 [25809.657366] [<ffffffff8104fa14>] warn_slowpath_null+0x14/0x20 [25809.657377] [<ffffffffa0396b7f>] btrfs_relocate_block_group+0x27f/0x390 [btrfs] [25809.657390] [<ffffffffa037cfab>] btrfs_relocate_chunk+0x8b/0x5e0 [btrfs] [25809.657404] [<ffffffffa0372a0b>] ? map_extent_buffer+0xdb/0xe0 [btrfs] [25809.657417] [<ffffffffa0371b13>] ? unmap_extent_buffer+0x13/0x40 [btrfs] [25809.657431] [<ffffffffa0366c54>] ? btrfs_item_offset+0xe4/0xf0 [btrfs] [25809.657444] [<ffffffffa037db3f>] btrfs_balance+0x1ef/0x270 [btrfs] [25809.657449] [<ffffffff810cc0ef>] ? find_get_page+0x7f/0xf0 [25809.657461] [<ffffffffa0382e10>] btrfs_ioctl+0x510/0xa20 [btrfs] [25809.657465] [<ffffffff810cc3e7>] ? unlock_page+0x27/0x30 [25809.657470] [<ffffffff810e8409>] ? __do_fault+0x449/0x540 [25809.657476] [<ffffffff8111bf62>] vfs_ioctl+0x22/0xa0 [25809.657481] [<ffffffff813cf4e9>] ? sub_preempt_count+0x59/0x60 [25809.657486] [<ffffffff8111c108>] do_vfs_ioctl+0x88/0x570 [25809.657491] [<ffffffff8120ecc9>] ? __up_read+0x99/0xc0 [25809.657497] [<ffffffff8107323e>] ? up_read+0xe/0x10 [25809.657501] [<ffffffff8111c63f>] sys_ioctl+0x4f/0x80 [25809.657507] [<ffffffff8100b5db>] system_call_fastpath+0x16/0x1b [25809.657511] ---[ end trace b3050419e02a5436 ]--- [25809.662186] ------------[ cut here ]------------ [25809.662190] kernel BUG at fs/btrfs/volumes.c:1746! [25809.662193] invalid opcode: 0000 [#1] PREEMPT SMP [25809.662198] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.2/host3/target3:0:0/3:0:0:0/model [25809.662201] CPU 0 [25809.662203] Modules linked in: btrfs loop crc32c libcrc32c ppp_deflate zlib_deflate bsd_comp ppp_async crc_ccitt ppp_generic slhc kvm_intel kvm i915 snd_ca0106 snd_rawmidi drm_kms_helper cfbcopyarea snd_ac97_codec ac97_bus video snd_pcm_oss snd_mixer_oss snd_pcm backlight snd_timer cfbimgblt snd processor button cfbfillrect intel_agp wacom soundcore snd_page_alloc usbhid fan fuse ehci_hcd uhci_hcd thermal thermal_sys usbcore atl1 mii [last unloaded: btrfs] [25809.662245] Pid: 15354, comm: btrfs-vol Tainted: G W 2.6.32-rc3 #1 System Product Name [25809.662248] RIP: 0010:[<ffffffffa037d4e4>] [<ffffffffa037d4e4>] btrfs_relocate_chunk+0x5c4/0x5e0 [btrfs] [25809.662261] RSP: 0018:ffff880073afdc18 EFLAGS: 00010282 [25809.662263] RAX: 00000000fffffffb RBX: ffff880077210000 RCX: ffffffffa0396ae7 [25809.662266] RDX: 0000000000000007 RSI: ffffffff8110670f RDI: ffffffff811044d5 [25809.662269] RBP: ffff880073afdd08 R08: 0000000000000000 R09: ffff880073afd938 [25809.662271] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880077153000 [25809.662274] R13: 0000000000000000 R14: 00000000d3100000 R15: ffff880073afdd48 [25809.662277] FS: 00007f3e04ec5730(0000) GS:ffff880001800000(0000) knlGS:0000000000000000 [25809.662281] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [25809.662283] CR2: 00007f79213f8000 CR3: 00000000bc95f000 CR4: 00000000000026e0 [25809.662286] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [25809.662289] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [25809.662292] Process btrfs-vol (pid: 15354, threadinfo ffff880073afc000, task ffff880001a84500) [25809.662294] Stack: [25809.662296] ffff880073afdc88 ffffffffa0372a0b ffff880073afdcc0 0000000000000001 [25809.662300] <0> 0000000000000100 0000000000000000 ffff8800b65e1800 ffff880073afdd48 [25809.662306] <0> ffff880077153000 00000000d3100000 00000000000000fb ffff88007b6ee990 [25809.662312] Call Trace: [25809.662324] [<ffffffffa0372a0b>] ? map_extent_buffer+0xdb/0xe0 [btrfs] [25809.662335] [<ffffffffa0371b13>] ? unmap_extent_buffer+0x13/0x40 [btrfs] [25809.662347] [<ffffffffa0366c54>] ? btrfs_item_offset+0xe4/0xf0 [btrfs] [25809.662358] [<ffffffffa037db3f>] btrfs_balance+0x1ef/0x270 [btrfs] [25809.662362] [<ffffffff810cc0ef>] ? find_get_page+0x7f/0xf0 [25809.662373] [<ffffffffa0382e10>] btrfs_ioctl+0x510/0xa20 [btrfs] [25809.662376] [<ffffffff810cc3e7>] ? unlock_page+0x27/0x30 [25809.662380] [<ffffffff810e8409>] ? __do_fault+0x449/0x540 [25809.662385] [<ffffffff8111bf62>] vfs_ioctl+0x22/0xa0 [25809.662388] [<ffffffff813cf4e9>] ? sub_preempt_count+0x59/0x60 [25809.662392] [<ffffffff8111c108>] do_vfs_ioctl+0x88/0x570 [25809.662396] [<ffffffff8120ecc9>] ? __up_read+0x99/0xc0 [25809.662400] [<ffffffff8107323e>] ? up_read+0xe/0x10 [25809.662404] [<ffffffff8111c63f>] sys_ioctl+0x4f/0x80 [25809.662409] [<ffffffff8100b5db>] system_call_fastpath+0x16/0x1b [25809.662411] Code: ff 31 c0 e9 c0 fa ff ff 0f 1f 80 00 00 00 00 0f 0b eb fe 0f 1f 40 00 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe <0f> 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f [25809.662463] RIP [<ffffffffa037d4e4>] btrfs_relocate_chunk+0x5c4/0x5e0 [btrfs] [25809.662475] RSP <ffff880073afdc18> [25809.662478] ---[ end trace b3050419e02a5437 ]---
Quoted 3 lines · Diego Calleja
I was playing with btrfs with 2 files of 3.5 GB (using loop), I completely zeroed one of the files. As expected, I had checksum failures, and I run btrfs-vol -b just to see what happened, and I got this (using -rc3):
Thanks, I'll try to reproduce. Which raid level did you use for data? If not raid1, could you try with raid1? ;)
-chris
On Wednesday 07 October 2009 05:17:54 Chris Mason escribi=F3:
Quoted 1 line
Thanks, I'll try to reproduce. Which raid level did you use for data=
?
Quoted 1 line
If not raid1, could you try with raid1? ;)
I'm not sure, since the utils won't tell. I mkfs'ed and mounted one of = the 3.5GB
files with no special options, and copied some files on it. I mkfs'ed t= he second
file, put it in a loop device, and added it to the pool with btrfs-vol = -a. Then
I run btrfs-vol -b and I copied more files and rebalanced several times= =2E
Then I zeroed one of the files used as disks, and tried to read the fil= es in
the volume (unsucessfully). Then I rebalanced, and I hit the oops. It s= eems to
be very reproducible.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines · Diego Calleja
On Wednesday 07 October 2009 05:17:54 Chris Mason escribi=F3:
=20Quoted 1 line
Thanks, I'll try to reproduce. Which raid level did you use for da=
ta?
Quoted 3 lines
Quoted 1 line
If not raid1, could you try with raid1? ;)
=20
I'm not sure, since the utils won't tell. I mkfs'ed and mounted one o=
f the 3.5GB
Quoted 1 line
files with no special options, and copied some files on it. I mkfs'ed=
the second
Quoted 1 line
file, put it in a loop device, and added it to the pool with btrfs-vo=
l -a. Then
Quoted 1 line
I run btrfs-vol -b and I copied more files and rebalanced several tim=
es.
Quoted 2 lines
=20
Then I zeroed one of the files used as disks, and tried to read the f=
iles in
Quoted 1 line
the volume (unsucessfully). Then I rebalanced, and I hit the oops. It=
seems to
Quoted 1 line
be very reproducible.
Ok, in this case you ended up with raid0 on the data. If you:
mkfs.btrfs -d raid1 /dev/loop0 /dev/loop1 you'll get data raid1.
-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wednesday 07 October 2009 16:33:09 Chris Mason escribi=F3:
Quoted 3 lines
Ok, in this case you ended up with raid0 on the data. If you:
=20
mkfs.btrfs -d raid1/dev/loop0/dev/loop1you'll get data raid1.
I got a BUG...but it may be a different bug. Unlike the last time, I didn't even need to zero one of the "disks", the second I rebalanced it oopsed:
[ 705.060358] btrfs: relocating block group 4194304 flags 4 =
=20
[ 1006.106133] btrfs: relocating block group 5704777728 flags 18 =
=20
[ 1006.336521] btrfs: found 1 extents =
=20
[ 1006.469967] btrfs: relocating block group 5384896512 flags 20 =
=20
[ 1006.471043] btrfs allocation failed flags 20, wanted 4096 =
=20
[ 1006.471048] space_info has 317378560 free, is full =
=20
[ 1006.471052] space_info total=3D319881216, pinned=3D16384, delalloc=3D=
0, may_use=3D0, used=3D2355200, root=3D2875392, super=3D131072, reserve=
d=3D0 =
=20
[ 1006.471058] block group 5384896512 has 319881216 bytes, 2355200 used=
16384 pinned 0 reserved =20
[ 1006.471063] entry offset 5384896512, bytes 483328, bitmap yes =
=20
[ 1006.471065] block group has cluster?: yes =
=20
[ 1006.471067] 1 blocks of free space at or bigger than bytes is =
=20
[ 1006.471092] ------------[ cut here ]------------ =
=20
[ 1006.471094] kernel BUG at fs/btrfs/transaction.c:1001! =
=20
[ 1006.471098] invalid opcode: 0000 [#1] PREEMPT SMP =
=20
[ 1006.471103] last sysfs file: /sys/devices/virtual/block/loop0/range =
=20
[ 1006.471106] CPU 0 =
=20
[ 1006.471109] Modules linked in: ppp_deflate bsd_comp ppp_async crc_cc=itt btrfs zlib_deflate crc32c libcrc32c loop ppp_generic slhc kvm_intel=
kvm i915 snd_ca0106 snd_rawmidi drm_kms_helper snd_ac97_codec ac97_bus= cfbcopyarea snd_pcm_oss video snd_mixer_oss backlight snd_pcm snd_time=
r cfbimgblt processor wacom intel_agp cfbfillrect button snd soundcore = snd_page_alloc usbhid fan fuse ehci_hcd uhci_hcd thermal thermal_sys at=
l1 mii usbcore =
=20
[ 1006.471159] Pid: 8207, comm: btrfs-vol Not tainted 2.6.32-rc3 #1 Sys=
tem Product Name =20
[ 1006.471162] RIP: 0010:[<ffffffffa0257665>] [<ffffffffa0257665>] btr=
fs_commit_transaction+0x6c5/0x6f0 [btrfs] =20
[ 1006.471183] RSP: 0018:ffff8800bc285a78 EFLAGS: 00010286 =
=20
[ 1006.471186] RAX: 00000000ffffffe4 RBX: ffff8800bdef5800 RCX: 0000000=
000000016 =20
[ 1006.471189] RDX: 0000000000000000 RSI: ffff8800bc2859f8 RDI: ffff880=
0a86d8060 =20
[ 1006.471192] RBP: ffff8800bc285b18 R08: 0000000000000000 R09: 0000000=
000000000 =20
[ 1006.471195] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880=
0a181f5c0 =20
[ 1006.471199] R13: ffff8800a1eb1ae0 R14: ffff8800bc285ab8 R15: ffff880=
0a1eb1b50
[ 1006.471202] FS: 00007f7c5427f730(0000) GS:ffff880001800000(0000) kn=
lGS:0000000000000000
[ 1006.471206] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 1006.471209] CR2: 0000000002812500 CR3: 00000000a2f88000 CR4: 0000000=
0000026f0
[ 1006.471212] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000=
000000000
[ 1006.471215] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000=
000000400
[ 1006.471219] Process btrfs-vol (pid: 8207, threadinfo ffff8800bc28400=
0, task ffff88000b2bae00)
[ 1006.471221] Stack:
[ 1006.471223] ffff8800bc285ad8 0000000000000206 00000001bf15ad00 0000=
000001000010
[ 1006.471229] <0> ffff8800a181f5c0 0000000000000004 ffff8800a1eb1b80 f=
fff8800a1eb1ae0
[ 1006.471235] <0> 0000000000000000 ffff88000b2bae00 ffffffff8106eb30 f=
fff8800bc285ad0
[ 1006.471242] Call Trace:
[ 1006.471250] [<ffffffff8106eb30>] ? autoremove_wake_function+0x0/0x4=
0
[ 1006.471264] [<ffffffffa0256d0e>] ? start_transaction+0x11e/0x170 [b=
trfs]
[ 1006.471285] [<ffffffffa029630c>] relocate_block_group+0xec/0x4b0 [b=
trfs]
[ 1006.471291] [<ffffffff813cf4e9>] ? sub_preempt_count+0x59/0x60
[ 1006.471301] [<ffffffffa025596b>] ? btrfs_clean_old_snapshots+0x8b/0=
x1c0 [btrfs]
[ 1006.471314] [<ffffffffa02714fb>] ? btrfs_wait_ordered_extents+0x1bb=
/0x1d0 [btrfs]
[ 1006.471325] [<ffffffffa0296a5a>] btrfs_relocate_block_group+0x15a/0=
x390 [btrfs]
[ 1006.471336] [<ffffffffa027cfab>] btrfs_relocate_chunk+0x8b/0x5e0 [b=
trfs]
[ 1006.471348] [<ffffffffa0272a0b>] ? map_extent_buffer+0xdb/0xe0 [btr=
fs]
[ 1006.471359] [<ffffffffa0271b13>] ? unmap_extent_buffer+0x13/0x40 [b=
trfs]
[ 1006.471372] [<ffffffffa0266c54>] ? btrfs_item_offset+0xe4/0xf0 [btr=
fs]
[ 1006.471383] [<ffffffffa027db3f>] btrfs_balance+0x1ef/0x270 [btrfs]
[ 1006.471387] [<ffffffff810cc0ef>] ? find_get_page+0x7f/0xf0
[ 1006.471398] [<ffffffffa0282e10>] btrfs_ioctl+0x510/0xa20 [btrfs]
[ 1006.471401] [<ffffffff810cc3e7>] ? unlock_page+0x27/0x30
[ 1006.471406] [<ffffffff810e8409>] ? __do_fault+0x449/0x540
[ 1006.471412] [<ffffffff8111bf62>] vfs_ioctl+0x22/0xa0
[ 1006.471415] [<ffffffff813cf4e9>] ? sub_preempt_count+0x59/0x60
[ 1006.471419] [<ffffffff8111c108>] do_vfs_ioctl+0x88/0x570
[ 1006.471424] [<ffffffff8120ecc9>] ? __up_read+0x99/0xc0
[ 1006.471428] [<ffffffff8107323e>] ? up_read+0xe/0x10
[ 1006.471432] [<ffffffff8111c63f>] sys_ioctl+0x4f/0x80
[ 1006.471438] [<ffffffff8100b5db>] system_call_fastpath+0x16/0x1b
[ 1006.471440] Code: 00 00 00 00 e9 c4 fe ff ff be ab 03 00 00 48 c7 c7=
1a 84 29 a0 e8 ac 83 df e0 e9 e8 fa ff ff 0f 0b eb fe 0f 0b eb fe 0f 0=
b eb fe <0f> 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe be d7 03 00 0=
0
[ 1006.471492] RIP [<ffffffffa0257665>] btrfs_commit_transaction+0x6c5=
/0x6f0 [btrfs]
[ 1006.471504] RSP <ffff8800bc285a78>
[ 1006.471507] ---[ end trace 14afea9f89710f14 ]-----
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Diego Calleja
On Wednesday 07 October 2009 16:33:09 Chris Mason escribi=F3:
Quoted 3 lines
Ok, in this case you ended up with raid0 on the data. If you:
=20
mkfs.btrfs -d raid1/dev/loop0/dev/loop1you'll get data raid1.=20
I got a BUG...but it may be a different bug. Unlike the last time, I didn't even need to zero one of the "disks", the second I rebalanced it oopsed:
I'm afraid this is good old enospc. Balancing still needs some work to be completely safe.
-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wednesday 07 October 2009 21:45:29 Chris Mason escribi=F3:
Quoted 1 line
I'm afraid this is good old enospc. Balancing still needs some work =
to
Quoted 1 line
be completely safe.
I've tried using less data and raid1, but I can't reproduce it. --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
By the way, i think it'd be useful if debug-tree would tell which policy the fs is applying to each chunk. Something like this:
item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 8379826176) itemoff 3495 itemsize 112
chunk length 319881216 owner 2 type 17 (data on RAID1) num_stripes 2
stripe 0 devid 2 offset 2265382912
stripe 1 devid 1 offset 989003776print-tree.c +26 -4
@@ -79,15 +79,37 @@return 0 ; } +static void print_chunk_type (u64 chunk_flags) +{ +printf (" type %llu (" , chunk_flags); + +if (chunk_flags &BTRFS_BLOCK_GROUP_DATA ) +printf ("data" ); +if (chunk_flags &BTRFS_BLOCK_GROUP_METADATA ) +printf ("metadata" ); +if (chunk_flags &BTRFS_BLOCK_GROUP_SYSTEM ) +printf ("system??" ); +if (chunk_flags &BTRFS_BLOCK_GROUP_DUP ) +printf ("dup??" ); +if (chunk_flags &BTRFS_BLOCK_GROUP_RAID0 ) +printf (" on RAID0" ); +if (chunk_flags &BTRFS_BLOCK_GROUP_RAID1 ) +printf (" on RAID1" ); +if (chunk_flags &BTRFS_BLOCK_GROUP_RAID10 ) +printf (" on RAID10" ); +printf (") " ); +} +static void print_chunk (struct extent_buffer *eb,struct btrfs_chunk *chunk) {int num_stripes =btrfs_chunk_num_stripes (eb, chunk);int i; -printf ("\t\tchunk length %llu owner %llu type %llu num_stripes %d\n" , +printf ("\t\tchunk length %llu owner %llu" , (unsigned long long )btrfs_chunk_length (eb, chunk), - (unsigned long long )btrfs_chunk_owner (eb, chunk), - (unsigned long long )btrfs_chunk_type (eb, chunk), - num_stripes); + (unsigned long long )btrfs_chunk_owner (eb, chunk)); +print_chunk_type ((unsigned long long )btrfs_chunk_type (eb, chunk)); +printf ("num_stripes %d\n" , num_stripes); +for (i =0 ; i < num_stripes ; i++) { printf("\t\t\tstripe %d devid %llu offset %llu\n", i, (unsigned long long)btrfs_stripe_devid_nr(eb, chunk, i),