kernel BUG at fs/btrfs/relocation.c:1062
Got this while running balance. Some other operations were running in parallel (rsync etc.).
[152727.584641] ------------[ cut here ]------------ [152727.584723] kernel BUG at fs/btrfs/relocation.c:1062! [152727.584802] invalid opcode: 0000 [#1] SMP [152727.584943] Modules linked in: veth ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop video i2c_i801 i2c_core button acpi_cpufreq pcspkr ehci_pci ehci_hcd lpc_ich mfd_core ext4 crc16 jbd2 mbcache raid1 sg sd_mod r8169 mii ahci libahci libata scsi_mod [152727.586767] CPU: 3 PID: 1540 Comm: btrfs Tainted: G W 3.13.0-rc4 #1 [152727.586888] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [152727.587011] task: ffff8805bedf9710 ti: ffff8805a9d9a000 task.ti: ffff8805a9d9a000 [152727.587132] RIP: 0010:[<ffffffffa02ab31f>] [<ffffffffa02ab31f>] build_backref_tree+0xb03/0xe7a [btrfs] [152727.587300] RSP: 0018:ffff8805a9d9b898 EFLAGS: 00010246 [152727.587379] RAX: ffff8805a9d9b900 RBX: ffff8805a9d9b920 RCX: ffff88055a72fa10 [152727.587499] RDX: ffff88055a72ff50 RSI: ffff88055a72ff80 RDI: ffff8807e8878600 [152727.587619] RBP: ffff8805a9d9b998 R08: ffff8807e8878b80 R09: 0000000000001000 [152727.587739] R10: 0000000000001000 R11: 0000160000000000 R12: 0000000000000000 [152727.587859] R13: ffff88055a72ff90 R14: ffff8805a9d9b930 R15: ffff88028e648000 [152727.587980] FS: 00007fca66eca840(0000) GS:ffff88081fac0000(0000) knlGS:0000000000000000 [152727.588101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [152727.588180] CR2: 00007fac0ba80000 CR3: 00000007006d9000 CR4: 00000000001407e0 [152727.588300] Stack: [152727.588374] ffff8800611992c0 0000000000000000 ffff8806c99dd500 ffff8807e8878600 [152727.588623] ffff88055a72fde0 ffff8806c99dd240 ffff88028e648124 ffff88028e648124 [152727.588871] ffff88028e648120 ffff88028e648020 ffff8807ece02c60 ffff8807ece02a20 [152727.589120] Call Trace: [152727.589205] [<ffffffffa02ac13a>] relocate_tree_blocks+0x1c0/0x544 [btrfs] [152727.589292] [<ffffffffa024f2a9>] ? btrfs_release_path+0x6b/0x8a [btrfs] [152727.589382] [<ffffffffa02ad399>] relocate_block_group+0x23f/0x4c7 [btrfs] [152727.589470] [<ffffffffa02ad76f>] btrfs_relocate_block_group+0x14e/0x28d [btrfs] [152727.589598] [<ffffffffa028c8d2>] btrfs_relocate_chunk.isra.65+0x58/0x60e [btrfs] [152727.589727] [<ffffffffa029a1ff>] ? btrfs_set_lock_blocking_rw+0x89/0xb2 [btrfs] [152727.589853] [<ffffffffa024f1b7>] ? btrfs_set_path_blocking+0x23/0x54 [btrfs] [152727.589979] [<ffffffffa0253a1b>] ? btrfs_search_slot+0x72f/0x789 [btrfs] [152727.590068] [<ffffffffa0288be1>] ? free_extent_buffer+0x6f/0x7c [btrfs] [152727.590157] [<ffffffffa028f709>] btrfs_balance+0x9fe/0xbe0 [btrfs] [152727.590245] [<ffffffffa02954c9>] btrfs_ioctl_balance+0x220/0x29f [btrfs] [152727.590334] [<ffffffffa0298a66>] btrfs_ioctl+0xfce/0x2128 [btrfs] [152727.590417] [<ffffffff810c98c1>] ? handle_mm_fault+0x24f/0x965 [152727.590498] [<ffffffff810ccc11>] ? __vm_enough_memory+0x26/0x13d [152727.590581] [<ffffffff8110219e>] do_vfs_ioctl+0x3f7/0x441 [152727.590661] [<ffffffff8110223a>] SyS_ioctl+0x52/0x80 [152727.590741] [<ffffffff8138f222>] system_call_fastpath+0x16/0x1b [152727.590821] Code: 60 71 fd 49 89 40 50 49 89 40 58 49 8b 40 58 4d 89 68 58 49 83 c0 50 4d 89 45 00 49 89 45 08 4c 89 28 e9 aa 00 00 00 a8 10 75 02 <0f> 0b 83 e0 01 41 39 c4 74 02 0f 0b 45 85 e4 75 32 49 8b 70 18 [152727.593073] RIP [<ffffffffa02ab31f>] build_backref_tree+0xb03/0xe7a [btrfs] [152727.593194] RSP <ffff8805a9d9b898> [152727.593290] ---[ end trace b67170e8ece9f591 ]---
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Hello Tomasz,
This seems a known bug that has been fixed by josef, what is your kernel version?
Thanks,
Wang
Quoted 47 lines · Tomasz Chmielewski
Got this while running balance. Some other operations were running in parallel (rsync etc.).
[152727.584641] ------------[ cut here ]------------ [152727.584723] kernel BUG at fs/btrfs/relocation.c:1062! [152727.584802] invalid opcode: 0000 [#1] SMP [152727.584943] Modules linked in: veth ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop video i2c_i801 i2c_core button acpi_cpufreq pcspkr ehci_pci ehci_hcd lpc_ich mfd_core ext4 crc16 jbd2 mbcache raid1 sg sd_mod r8169 mii ahci libahci libata scsi_mod [152727.586767] CPU: 3 PID: 1540 Comm: btrfs Tainted: G W 3.13.0-rc4 #1 [152727.586888] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [152727.587011] task: ffff8805bedf9710 ti: ffff8805a9d9a000 task.ti: ffff8805a9d9a000 [152727.587132] RIP: 0010:[<ffffffffa02ab31f>] [<ffffffffa02ab31f>] build_backref_tree+0xb03/0xe7a [btrfs] [152727.587300] RSP: 0018:ffff8805a9d9b898 EFLAGS: 00010246 [152727.587379] RAX: ffff8805a9d9b900 RBX: ffff8805a9d9b920 RCX: ffff88055a72fa10 [152727.587499] RDX: ffff88055a72ff50 RSI: ffff88055a72ff80 RDI: ffff8807e8878600 [152727.587619] RBP: ffff8805a9d9b998 R08: ffff8807e8878b80 R09: 0000000000001000 [152727.587739] R10: 0000000000001000 R11: 0000160000000000 R12: 0000000000000000 [152727.587859] R13: ffff88055a72ff90 R14: ffff8805a9d9b930 R15: ffff88028e648000 [152727.587980] FS: 00007fca66eca840(0000) GS:ffff88081fac0000(0000) knlGS:0000000000000000 [152727.588101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [152727.588180] CR2: 00007fac0ba80000 CR3: 00000007006d9000 CR4: 00000000001407e0 [152727.588300] Stack: [152727.588374] ffff8800611992c0 0000000000000000 ffff8806c99dd500 ffff8807e8878600 [152727.588623] ffff88055a72fde0 ffff8806c99dd240 ffff88028e648124 ffff88028e648124 [152727.588871] ffff88028e648120 ffff88028e648020 ffff8807ece02c60 ffff8807ece02a20 [152727.589120] Call Trace: [152727.589205] [<ffffffffa02ac13a>] relocate_tree_blocks+0x1c0/0x544 [btrfs] [152727.589292] [<ffffffffa024f2a9>] ? btrfs_release_path+0x6b/0x8a [btrfs] [152727.589382] [<ffffffffa02ad399>] relocate_block_group+0x23f/0x4c7 [btrfs] [152727.589470] [<ffffffffa02ad76f>] btrfs_relocate_block_group+0x14e/0x28d [btrfs] [152727.589598] [<ffffffffa028c8d2>] btrfs_relocate_chunk.isra.65+0x58/0x60e [btrfs] [152727.589727] [<ffffffffa029a1ff>] ? btrfs_set_lock_blocking_rw+0x89/0xb2 [btrfs] [152727.589853] [<ffffffffa024f1b7>] ? btrfs_set_path_blocking+0x23/0x54 [btrfs] [152727.589979] [<ffffffffa0253a1b>] ? btrfs_search_slot+0x72f/0x789 [btrfs] [152727.590068] [<ffffffffa0288be1>] ? free_extent_buffer+0x6f/0x7c [btrfs] [152727.590157] [<ffffffffa028f709>] btrfs_balance+0x9fe/0xbe0 [btrfs] [152727.590245] [<ffffffffa02954c9>] btrfs_ioctl_balance+0x220/0x29f [btrfs] [152727.590334] [<ffffffffa0298a66>] btrfs_ioctl+0xfce/0x2128 [btrfs] [152727.590417] [<ffffffff810c98c1>] ? handle_mm_fault+0x24f/0x965 [152727.590498] [<ffffffff810ccc11>] ? __vm_enough_memory+0x26/0x13d [152727.590581] [<ffffffff8110219e>] do_vfs_ioctl+0x3f7/0x441 [152727.590661] [<ffffffff8110223a>] SyS_ioctl+0x52/0x80 [152727.590741] [<ffffffff8138f222>] system_call_fastpath+0x16/0x1b [152727.590821] Code: 60 71 fd 49 89 40 50 49 89 40 58 49 8b 40 58 4d 89 68 58 49 83 c0 50 4d 89 45 00 49 89 45 08 4c 89 28 e9 aa 00 00 00 a8 10 75 02 <0f> 0b 83 e0 01 41 39 c4 74 02 0f 0b 45 85 e4 75 32 49 8b 70 18 [152727.593073] RIP [<ffffffffa02ab31f>] build_backref_tree+0xb03/0xe7a [btrfs] [152727.593194] RSP <ffff8805a9d9b898> [152727.593290] ---[ end trace b67170e8ece9f591 ]---
It was 3.13.0-rc4.
Quoted 77 lines · On Thu, 19 Dec 2013 20:14:17 +0800 Wang Shilong <[email protected]>…
Hello Tomasz,
This seems a known bug that has been fixed by josef, what is your kernel version?
Thanks,
WangQuoted 67 lines · Tomasz Chmielewski
Got this while running balance. Some other operations were running in parallel (rsync etc.).
[152727.584641] ------------[ cut here ]------------ [152727.584723] kernel BUG at fs/btrfs/relocation.c:1062! [152727.584802] invalid opcode: 0000 [#1] SMP [152727.584943] Modules linked in: veth ipt_MASQUERADE iptable_nat
nf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop videoi2c_i801i2c_corebuttonacpi_cpufreqpcspkrehci_pciehci_hcdlpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modr8169 mii ahci libahci libatascsi_mod[152727.586767] CPU: 3 PID: 1540 Comm: btrfs Tainted: G W 3.13.0-rc4 #1 [152727.586888] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [152727.587011] task: ffff8805bedf9710 ti: ffff8805a9d9a000task.ti:
ffff8805a9d9a000[152727.587132] RIP:
0010:[<ffffffffa02ab31f>] [<ffffffffa02ab31f>]build_backref_tree+0xb03/0xe7a[btrfs] [152727.587300] RSP:
0018:ffff8805a9d9b898 EFLAGS: 00010246 [152727.587379] RAX:ffff8805a9d9b900RBX:ffff8805a9d9b920RCX:ffff88055a72fa10[152727.587499] RDX: ffff88055a72ff50 RSI: ffff88055a72ff80 RDI: ffff8807e8878600 [152727.587619] RBP: ffff8805a9d9b998 R08: ffff8807e8878b80 R09: 0000000000001000 [152727.587739] R10: 0000000000001000 R11: 0000160000000000 R12: 0000000000000000 [152727.587859] R13: ffff88055a72ff90 R14: ffff8805a9d9b930 R15: ffff88028e648000 [152727.587980] FS: 00007fca66eca840(0000) GS:ffff88081fac0000(0000) knlGS:0000000000000000 [152727.588101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [152727.588180] CR2: 00007fac0ba80000 CR3: 00000007006d9000 CR4: 00000000001407e0 [152727.588300] Stack: [152727.588374] ffff8800611992c0 0000000000000000 ffff8806c99dd500 ffff8807e8878600 [152727.588623] ffff88055a72fde0 ffff8806c99dd240 ffff88028e648124 ffff88028e648124 [152727.588871] ffff88028e648120 ffff88028e648020 ffff8807ece02c60 ffff8807ece02a20 [152727.589120] Call Trace: [152727.589205] [<ffffffffa02ac13a>] relocate_tree_blocks+0x1c0/0x544 [btrfs] [152727.589292] [<ffffffffa024f2a9>] ? btrfs_release_path+0x6b/0x8a [btrfs] [152727.589382] [<ffffffffa02ad399>] relocate_block_group+0x23f/0x4c7 [btrfs] [152727.589470] [<ffffffffa02ad76f>] btrfs_relocate_block_group+0x14e/0x28d [btrfs] [152727.589598] [<ffffffffa028c8d2>] btrfs_relocate_chunk.isra.65+0x58/0x60e [btrfs] [152727.589727] [<ffffffffa029a1ff>] ? btrfs_set_lock_blocking_rw+0x89/0xb2 [btrfs] [152727.589853] [<ffffffffa024f1b7>] ? btrfs_set_path_blocking+0x23/0x54 [btrfs] [152727.589979] [<ffffffffa0253a1b>] ? btrfs_search_slot+0x72f/0x789 [btrfs] [152727.590068] [<ffffffffa0288be1>] ? free_extent_buffer+0x6f/0x7c [btrfs] [152727.590157] [<ffffffffa028f709>] btrfs_balance+0x9fe/0xbe0 [btrfs] [152727.590245] [<ffffffffa02954c9>] btrfs_ioctl_balance+0x220/0x29f [btrfs] [152727.590334] [<ffffffffa0298a66>] btrfs_ioctl+0xfce/0x2128 [btrfs] [152727.590417] [<ffffffff810c98c1>] ? handle_mm_fault+0x24f/0x965 [152727.590498] [<ffffffff810ccc11>] ? __vm_enough_memory+0x26/0x13d [152727.590581] [<ffffffff8110219e>] do_vfs_ioctl+0x3f7/0x441 [152727.590661] [<ffffffff8110223a>] SyS_ioctl+0x52/0x80 [152727.590741] [<ffffffff8138f222>] system_call_fastpath+0x16/0x1b [152727.590821] Code: 60 71 fd 49 8940 50 49 89 40 58 49 8b 40 58 4d 89 68 58 49 83 c0 50 4d 89 45 00 49 89 45 08 4c 89 28 e9 aa 00 00 00 a8 10 75 02 <0f> 0b 83 e0 01 41 39 c4 74 02 0f 0b 45 85 e4 75 32 49 8b 70 18 [152727.593073] RIP [<
ffffffffa02ab31f>]build_backref_tree+0xb03/0xe7a[btrfs][152727.593194] RSP <ffff8805a9d9b898> [152727.593290] ---[ endtrace
b67170e8ece9f591]---
Quoted 1 line · Tomasz Chmielewski
It was 3.13.0-rc4.
I take a look at line 1062, this should be a new bug!!!
Thanks,
Wang
Quoted 86 lines
Quoted 76 lines · On Thu, 19 Dec 2013 20:14:17 +0800 Wang Shilong <[email protected]>…
Hello Tomasz,
This seems a known bug that has been fixed by josef, what is your kernel version?
Thanks,
WangQuoted 67 lines · Tomasz Chmielewski
Got this while running balance. Some other operations were running in parallel (rsync etc.).
[152727.584641] ------------[ cut here ]------------ [152727.584723] kernel BUG at fs/btrfs/relocation.c:1062! [152727.584802] invalid opcode: 0000 [#1] SMP [152727.584943] Modules linked in: veth ipt_MASQUERADE iptable_nat
nf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop videoi2c_i801i2c_corebuttonacpi_cpufreqpcspkrehci_pciehci_hcdlpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modr8169 mii ahci libahci libatascsi_mod[152727.586767] CPU: 3 PID: 1540 Comm: btrfs Tainted: G W 3.13.0-rc4 #1 [152727.586888] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [152727.587011] task: ffff8805bedf9710 ti: ffff8805a9d9a000task.ti:
ffff8805a9d9a000[152727.587132] RIP:
0010:[<ffffffffa02ab31f>] [<ffffffffa02ab31f>]build_backref_tree+0xb03/0xe7a[btrfs] [152727.587300] RSP:
0018:ffff8805a9d9b898 EFLAGS: 00010246 [152727.587379] RAX:ffff8805a9d9b900RBX:ffff8805a9d9b920RCX:ffff88055a72fa10[152727.587499] RDX: ffff88055a72ff50 RSI: ffff88055a72ff80 RDI: ffff8807e8878600 [152727.587619] RBP: ffff8805a9d9b998 R08: ffff8807e8878b80 R09: 0000000000001000 [152727.587739] R10: 0000000000001000 R11: 0000160000000000 R12: 0000000000000000 [152727.587859] R13: ffff88055a72ff90 R14: ffff8805a9d9b930 R15: ffff88028e648000 [152727.587980] FS: 00007fca66eca840(0000) GS:ffff88081fac0000(0000) knlGS:0000000000000000 [152727.588101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [152727.588180] CR2: 00007fac0ba80000 CR3: 00000007006d9000 CR4: 00000000001407e0 [152727.588300] Stack: [152727.588374] ffff8800611992c0 0000000000000000 ffff8806c99dd500 ffff8807e8878600 [152727.588623] ffff88055a72fde0 ffff8806c99dd240 ffff88028e648124 ffff88028e648124 [152727.588871] ffff88028e648120 ffff88028e648020 ffff8807ece02c60 ffff8807ece02a20 [152727.589120] Call Trace: [152727.589205] [<ffffffffa02ac13a>] relocate_tree_blocks+0x1c0/0x544 [btrfs] [152727.589292] [<ffffffffa024f2a9>] ? btrfs_release_path+0x6b/0x8a [btrfs] [152727.589382] [<ffffffffa02ad399>] relocate_block_group+0x23f/0x4c7 [btrfs] [152727.589470] [<ffffffffa02ad76f>] btrfs_relocate_block_group+0x14e/0x28d [btrfs] [152727.589598] [<ffffffffa028c8d2>] btrfs_relocate_chunk.isra.65+0x58/0x60e [btrfs] [152727.589727] [<ffffffffa029a1ff>] ? btrfs_set_lock_blocking_rw+0x89/0xb2 [btrfs] [152727.589853] [<ffffffffa024f1b7>] ? btrfs_set_path_blocking+0x23/0x54 [btrfs] [152727.589979] [<ffffffffa0253a1b>] ? btrfs_search_slot+0x72f/0x789 [btrfs] [152727.590068] [<ffffffffa0288be1>] ? free_extent_buffer+0x6f/0x7c [btrfs] [152727.590157] [<ffffffffa028f709>] btrfs_balance+0x9fe/0xbe0 [btrfs] [152727.590245] [<ffffffffa02954c9>] btrfs_ioctl_balance+0x220/0x29f [btrfs] [152727.590334] [<ffffffffa0298a66>] btrfs_ioctl+0xfce/0x2128 [btrfs] [152727.590417] [<ffffffff810c98c1>] ? handle_mm_fault+0x24f/0x965 [152727.590498] [<ffffffff810ccc11>] ? __vm_enough_memory+0x26/0x13d [152727.590581] [<ffffffff8110219e>] do_vfs_ioctl+0x3f7/0x441 [152727.590661] [<ffffffff8110223a>] SyS_ioctl+0x52/0x80 [152727.590741] [<ffffffff8138f222>] system_call_fastpath+0x16/0x1b [152727.590821] Code: 60 71 fd 49 8940 50 49 89 40 58 49 8b 40 58 4d 89 68 58 49 83 c0 50 4d 89 45 00 49 89 45 08 4c 89 28 e9 aa 00 00 00 a8 10 75 02 <0f> 0b 83 e0 01 41 39 c4 74 02 0f 0b 45 85 e4 75 32 49 8b 70 18 [152727.593073] RIP [<
ffffffffa02ab31f>]build_backref_tree+0xb03/0xe7a[btrfs][152727.593194] RSP <ffff8805a9d9b898> [152727.593290] ---[ endtrace
b67170e8ece9f591]-----
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
If it matters, I had to hard reboot after that bug; the balance continued after the system booted again and I got this a while later (filesystem was remounted read only):
[ 1781.321219] btrfs: found 232188 extents [ 1781.994796] btrfs: relocating block group 3443586498560 flags 20 [ 2603.422490] btrfs: found 203955 extents [ 2606.188826] btrfs: relocating block group 3051670732800 flags 20 [ 2806.720510] BTRFS debug (device sdb5): run_one_delayed_ref returned -28 [ 2806.720513] ------------[ cut here ]------------ [ 2806.720530] WARNING: CPU: 1 PID: 2359 at fs/btrfs/super.c:254 __btrfs_abort_transaction+0x4d/0xff [btrfs]() [ 2806.720544] btrfs: Transaction aborted (error -28) [ 2806.720544] Modules linked in: veth ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop pcspkr lpc_ich mfd_core i2c_i801 i2c_core button acpi_cpufreq ehci_pci video ehci_hcd ext4 crc16 jbd2 mbcache raid1 sg sd_mod ahci libahci libata scsi_mod r8169 mii [ 2806.720626] CPU: 1 PID: 2359 Comm: btrfs-transacti Not tainted 3.13.0-rc4 #1 [ 2806.720636] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 2806.720650] 0000000000000009 ffff8807ed73dc48 ffffffff8138a37d 0000000000000006 [ 2806.720705] ffff8807ed73dc98 ffff8807ed73dc88 ffffffff810370a9 ffff8807ed73dd80 [ 2806.720797] ffffffffa0227524 00000000ffffffe4 ffff8807f3389000 ffff8807efad4f00 [ 2806.720889] Call Trace: [ 2806.720935] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 2806.720986] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 2806.721005] BTRFS error (device sdb5) in __btrfs_free_extent:5783: errno=-28 No space left [ 2806.721006] BTRFS info (device sdb5): forced readonly [ 2806.721007] BTRFS debug (device sdb5): run_one_delayed_ref returned -28 [ 2806.721008] BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-28 No space left [ 2806.721276] [<ffffffffa0227524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 2806.721372] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 2806.721426] [<ffffffffa0227524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 2806.721482] [<ffffffffa023c6ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 2806.721538] [<ffffffffa0249aef>] btrfs_commit_transaction+0x70/0x7df [btrfs] [ 2806.721593] [<ffffffffa0248345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 2806.721646] [<ffffffffa0248256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 2806.721697] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 2806.721744] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 2806.721794] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 2806.721844] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 2806.721893] ---[ end trace 5d2cc0a807b9d02a ]--- [ 2806.721942] BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-28 No space left
Signature · 5 lines
Tomasz Chmielewski http://wpkg.org On Thu, 19 Dec 2013 20:49:04 +0800
Quoted 96 lines · Wang Shilong
Quoted 1 line · Tomasz Chmielewski
It was 3.13.0-rc4.
I take a look at line 1062, this should be a new bug!!!
Thanks,
WangQuoted 89 lines
Quoted 79 lines · On Thu, 19 Dec 2013 20:14:17 +0800 Wang Shilong <[email protected]>…
Hello Tomasz,
This seems a known bug that has been fixed by josef, what is your kernel version?
Thanks,
WangQuoted 70 lines · Tomasz Chmielewski
Got this while running balance. Some other operations were running in parallel (rsync etc.).
[152727.584641] ------------[ cut here ]------------ [152727.584723] kernel BUG at fs/btrfs/relocation.c:1062! [152727.584802] invalid opcode: 0000 [#1] SMP [152727.584943] Modules linked in: veth ipt_MASQUERADE iptable_nat
nf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop videoi2c_i801i2c_corebuttonacpi_cpufreqpcspkrehci_pciehci_hcdlpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modr8169 mii ahci libahci libatascsi_mod[152727.586767] CPU: 3 PID: 1540 Comm: btrfs Tainted:
G W 3.13.0-rc4 #1 [152727.586888] Hardware name: System
manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013[152727.587011] task: ffff8805bedf9710 ti: ffff8805a9d9a000task.ti:
ffff8805a9d9a000[152727.587132] RIP:
0010:[<ffffffffa02ab31f>] [<ffffffffa02ab31f>]build_backref_tree+0xb03/0xe7a[btrfs] [152727.587300] RSP:
0018:ffff8805a9d9b898 EFLAGS: 00010246 [152727.587379] RAX:ffff8805a9d9b900RBX:ffff8805a9d9b920RCX:ffff88055a72fa10[152727.587499] RDX: ffff88055a72ff50 RSI: ffff88055a72ff80 RDI: ffff8807e8878600 [152727.587619] RBP: ffff8805a9d9b998 R08: ffff8807e8878b80 R09: 0000000000001000 [152727.587739] R10: 0000000000001000 R11: 0000160000000000 R12: 0000000000000000 [152727.587859] R13: ffff88055a72ff90 R14: ffff8805a9d9b930 R15: ffff88028e648000 [152727.587980] FS: 00007fca66eca840(0000) GS:ffff88081fac0000(0000) knlGS:0000000000000000 [152727.588101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [152727.588180] CR2: 00007fac0ba80000 CR3: 00000007006d9000 CR4: 00000000001407e0 [152727.588300] Stack: [152727.588374] ffff8800611992c00000000000000000
ffff8806c99dd500ffff8807e8878600[152727.588623]ffff88055a72fde0ffff8806c99dd240ffff88028e648124ffff88028e648124[152727.588871]ffff88028e648120ffff88028e648020ffff8807ece02c60ffff8807ece02a20[152727.589120] Call Trace: [152727.589205] [<ffffffffa02ac13a>] relocate_tree_blocks+0x1c0/0x544 [btrfs] [152727.589292] [<ffffffffa024f2a9>] ? btrfs_release_path+0x6b/0x8a [btrfs] [152727.589382] [<ffffffffa02ad399>] relocate_block_group+0x23f/0x4c7 [btrfs] [152727.589470] [<ffffffffa02ad76f>] btrfs_relocate_block_group+0x14e/0x28d [btrfs] [152727.589598] [<ffffffffa028c8d2>] btrfs_relocate_chunk.isra.65+0x58/0x60e [btrfs] [152727.589727] [<ffffffffa029a1ff>] ? btrfs_set_lock_blocking_rw+0x89/0xb2 [btrfs] [152727.589853] [<ffffffffa024f1b7>] ? btrfs_set_path_blocking+0x23/0x54 [btrfs] [152727.589979] [<ffffffffa0253a1b>] ? btrfs_search_slot+0x72f/0x789 [btrfs] [152727.590068] [<ffffffffa0288be1>] ? free_extent_buffer+0x6f/0x7c [btrfs] [152727.590157] [<ffffffffa028f709>] btrfs_balance+0x9fe/0xbe0 [btrfs] [152727.590245] [<ffffffffa02954c9>] btrfs_ioctl_balance+0x220/0x29f [btrfs] [152727.590334] [<ffffffffa0298a66>] btrfs_ioctl+0xfce/0x2128 [btrfs] [152727.590417] [<ffffffff810c98c1>] ? handle_mm_fault+0x24f/0x965 [152727.590498] [<ffffffff810ccc11>] ? __vm_enough_memory+0x26/0x13d [152727.590581] [<ffffffff8110219e>] do_vfs_ioctl+0x3f7/0x441 [152727.590661] [<ffffffff8110223a>] SyS_ioctl+0x52/0x80 [152727.590741] [<ffffffff8138f222>]
system_call_fastpath+0x16/0x1b[152727.590821] Code: 60 71 fd 49 89 40 50 49 89 40 58 49 8b 40 58 4d 89 68 58 49 83 c0 50 4d 89 45 00 49 89 45 08 4c 89 28 e9 aa 00 00 00 a8 10 75 02 <0f> 0b 83 e0 01 41 39 c4 74 02 0f 0b 45 85 e4 75 32 49 8b 70 18[152727.593073] RIP [<ffffffffa02ab31f>]
build_backref_tree+0xb03/0xe7a[btrfs] [152727.593194] RSP <ffff8805a9d9b898> [152727.593290] ---[ end traceb67170e8ece9f591]-----
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines · On Thu, 19 Dec 2013 22:07:37 +0900 Tomasz Chmielewski <[email protected]> wrote:
If it matters, I had to hard reboot after that bug; the balance continued after the system booted again and I got this a while later (filesystem was remounted read only):
Actually, looks like the fs is quite hosed now :(
If I want to do an operation like removing a snapshot (btrfs sub del ...), the command returns, there is IO for a few minutes, but then, the kernel complains and fs is remounted readonly:
[ 5111.773900] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 5111.773902] ------------[ cut here ]------------ [ 5111.773957] WARNING: CPU: 0 PID: 15042 at fs/btrfs/super.c:254 __btrfs_abort_transaction+0x4d/0xff [btrfs]() [ 5111.774047] btrfs: Transaction aborted (error -17) [ 5111.774048] Modules linked in: ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop pcspkr button video ehci_pci ehci_hcd acpi_cpufreq i2c_i801 i2c_core lpc_ich mfd_core ext4 crc16 jbd2 mbcache raid1 sg sd_mod ahci libahci libata scsi_mod r8169 mii [ 5111.774355] CPU: 0 PID: 15042 Comm: btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 5111.774443] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 5111.774533] 0000000000000009 ffff8800374ddc48 ffffffff8138a37d 0000000000000006 [ 5111.774622] ffff8800374ddc98 ffff8800374ddc88 ffffffff810370a9 ffff8800374ddd80 [ 5111.774711] ffffffffa020d524 00000000ffffffef ffff8807ead7d800 ffff8807ff0cc8c0 [ 5111.774800] Call Trace: [ 5111.774846] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 5111.774894] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 5111.774944] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775032] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 5111.775081] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775135] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 5111.775189] [<ffffffffa022fdec>] btrfs_commit_transaction+0x36d/0x7df [btrfs] [ 5111.775281] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 5111.775333] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 5111.775382] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 5111.775428] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775477] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 5111.775524] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775572] ---[ end trace b552aca9a0cff3cb ]--- [ 5111.775618] BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 5111.775707] BTRFS info (device sdb5): forced readonly [ 5111.775754] BTRFS warning (device sdb5): Skipping commit of aborted transaction. [ 5111.775841] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already exists
If I run balance (after unmounting and mounting the filesystem), it ends in a similar way:
[ 5927.338989] btrfs: relocating block group 4647284637696 flags 17 [ 6184.333629] btrfs: found 983 extents [ 6269.512577] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 6269.512579] ------------[ cut here ]------------ [ 6269.512636] WARNING: CPU: 7 PID: 17836 at fs/btrfs/super.c:254 __btrfs_abort_transaction+0x4d/0xff [btrfs]() [ 6269.512727] btrfs: Transaction aborted (error -17) [ 6269.512727] Modules linked in: ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop pcspkr button video ehci_pci ehci_hcd acpi_cpufreq i2c_i801 i2c_core lpc_ich mfd_core ext4 crc16 jbd2 mbcache raid1 sg sd_mod ahci libahci libata scsi_mod r8169 mii [ 6269.513036] CPU: 7 PID: 17836 Comm: btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 6269.513124] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 6269.513214] 0000000000000009 ffff8807c5b07c48 ffffffff8138a37d 0000000000000006 [ 6269.513304] ffff8807c5b07c98 ffff8807c5b07c88 ffffffff810370a9 ffff8807c5b07d80 [ 6269.513393] ffffffffa020d524 00000000ffffffef ffff8807ead7e800 ffff8800371eedc0 [ 6269.513483] Call Trace: [ 6269.513529] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 6269.513577] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 6269.513627] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513716] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 6269.513766] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513820] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 6269.513874] [<ffffffffa022faef>] btrfs_commit_transaction+0x70/0x7df [btrfs] [ 6269.513927] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 6269.513980] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 6269.514029] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 6269.514076] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514125] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 6269.514172] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514220] ---[ end trace b552aca9a0cff3cc ]--- [ 6269.514266] BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 6269.514356] BTRFS info (device sdb5): forced readonly [ 6269.514430] BTRFS warning (device sdb5): Skipping commit of aborted transaction. [ 6269.514559] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already exists [ 6269.527034] btrfs: found 983 extents
Is there any fix I can try?
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
Hello Tomasz,
There should be some bugs flying on, unitil now, you can try to remount:
something like:
# mount /dev/sda /mnt -o skip_balance
It will skip unfinshed balance, hopley it can help you before we give a bug fix.
Thanks,
Wang
2013/12/20 Tomasz Chmielewski <[email protected]>:
Quoted 94 lines
Quoted 3 lines · On Thu, 19 Dec 2013 22:07:37 +0900 Tomasz Chmielewski <[email protected]> wrote:
If it matters, I had to hard reboot after that bug; the balance continued after the system booted again and I got this a while later (filesystem was remounted read only):
Actually, looks like the fs is quite hosed now :(
If I want to do an operation like removing a snapshot (btrfs sub del ...), the command returns, there is IO for a few minutes, but then, the kernel complains and fs is remounted readonly:
[ 5111.773900] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 5111.773902] ------------[ cut here ]------------ [ 5111.773957] WARNING: CPU: 0 PID: 15042 at fs/btrfs/super.c:254 __btrfs_abort_transaction+0x4d/0xff [btrfs]() [ 5111.774047] btrfs: Transaction aborted (error -17) [ 5111.774048] Modules linked in: ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop pcspkr button video ehci_pci ehci_hcd acpi_cpufreq i2c_i801 i2c_core lpc_ich mfd_core ext4 crc16 jbd2 mbcache raid1 sg sd_mod ahci libahci libata scsi_mod r8169 mii [ 5111.774355] CPU: 0 PID: 15042 Comm: btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 5111.774443] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 5111.774533] 0000000000000009 ffff8800374ddc48 ffffffff8138a37d 0000000000000006 [ 5111.774622] ffff8800374ddc98 ffff8800374ddc88 ffffffff810370a9 ffff8800374ddd80 [ 5111.774711] ffffffffa020d524 00000000ffffffef ffff8807ead7d800 ffff8807ff0cc8c0 [ 5111.774800] Call Trace: [ 5111.774846] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 5111.774894] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 5111.774944] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775032] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 5111.775081] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775135] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 5111.775189] [<ffffffffa022fdec>] btrfs_commit_transaction+0x36d/0x7df [btrfs] [ 5111.775281] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 5111.775333] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 5111.775382] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 5111.775428] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775477] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 5111.775524] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775572] ---[ end trace b552aca9a0cff3cb ]--- [ 5111.775618] BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 5111.775707] BTRFS info (device sdb5): forced readonly [ 5111.775754] BTRFS warning (device sdb5): Skipping commit of aborted transaction. [ 5111.775841] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already existsIf I run balance (after unmounting and mounting the filesystem), it ends in a similar way:
[ 5927.338989] btrfs: relocating block group 4647284637696 flags 17 [ 6184.333629] btrfs: found 983 extents [ 6269.512577] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 6269.512579] ------------[ cut here ]------------ [ 6269.512636] WARNING: CPU: 7 PID: 17836 at fs/btrfs/super.c:254 __btrfs_abort_transaction+0x4d/0xff [btrfs]() [ 6269.512727] btrfs: Transaction aborted (error -17) [ 6269.512727] Modules linked in: ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables cpufreq_ondemand cpufreq_conservative cpufreq_powersave cpufreq_stats bridge stp llc ipv6 btrfs xor raid6_pq zlib_deflate loop pcspkr button video ehci_pci ehci_hcd acpi_cpufreq i2c_i801 i2c_core lpc_ich mfd_core ext4 crc16 jbd2 mbcache raid1 sg sd_mod ahci libahci libata scsi_mod r8169 mii [ 6269.513036] CPU: 7 PID: 17836 Comm: btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 6269.513124] Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 6269.513214] 0000000000000009 ffff8807c5b07c48 ffffffff8138a37d 0000000000000006 [ 6269.513304] ffff8807c5b07c98 ffff8807c5b07c88 ffffffff810370a9 ffff8807c5b07d80 [ 6269.513393] ffffffffa020d524 00000000ffffffef ffff8807ead7e800 ffff8800371eedc0 [ 6269.513483] Call Trace: [ 6269.513529] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 6269.513577] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 6269.513627] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513716] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 6269.513766] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513820] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 6269.513874] [<ffffffffa022faef>] btrfs_commit_transaction+0x70/0x7df [btrfs] [ 6269.513927] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 6269.513980] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 6269.514029] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 6269.514076] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514125] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 6269.514172] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514220] ---[ end trace b552aca9a0cff3cc ]--- [ 6269.514266] BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 6269.514356] BTRFS info (device sdb5): forced readonly [ 6269.514430] BTRFS warning (device sdb5): Skipping commit of aborted transaction. [ 6269.514559] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already exists [ 6269.527034] btrfs: found 983 extentsIs there any fix I can try?
--
Tomasz Chmielewski
http://wpkg.org--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
-o skip_balance - didn't know this.
Actually, I was able to "skip" the balance, sort of, with this:
mount /mnt/btrfs ; btrfs fi balance cancel /mnt/btrfs
However, the fs is extremely unstable (will remount read only quite fast, i.e. if I start removing snapshot).
I'm trying to run btrfsck without the --repair option to see if it shows anything interesting.
Signature · 5 lines
Tomasz Chmielewski http://wpkg.org On Fri, 20 Dec 2013 23:52:08 +0800
Quoted 133 lines · Shilong Wang
Hello Tomasz,
There should be some bugs flying on, unitil now, you can try to
remount: something like:# mount /dev/sda /mnt -o skip_balanceIt will skip unfinshed balance, hopley it can help you before we give a bug fix.
Thanks,
Wang
2013/12/20 Tomasz Chmielewski <[email protected]>:Quoted 120 lines
Quoted 3 lines · On Thu, 19 Dec 2013 22:07:37 +0900 Tomasz Chmielewski <[email protected]> wrote:
If it matters, I had to hard reboot after that bug; the balance continued after the system booted again and I got this a while later (filesystem was remounted read only):
Actually, looks like the fs is quite hosed now :(
If I want to do an operation like removing a snapshot (btrfs sub del ...), the command returns, there is IO for a few minutes, but then, the kernel complains and fs is remounted readonly:
[ 5111.773900] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 5111.773902] ------------[ cut here ]------------ [ 5111.773957] WARNING: CPU: 0 PID: 15042 at fs/btrfs/super.c:254
__btrfs_abort_transaction+0x4d/0xff[btrfs]() [ 5111.774047] btrfs:
Transaction aborted (error -17) [ 5111.774048] Modules linked in:ipt_MASQUERADEiptable_natnf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop pcspkr button videoehci_pciehci_hcdacpi_cpufreqi2c_i801i2c_corelpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modahci libahci libatascsi_modr8169 mii [ 5111.774355] CPU: 0 PID: 15042 Comm:
btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 5111.774443]Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 5111.774533] 0000000000000009 ffff8800374ddc48 ffffffff8138a37d 0000000000000006 [ 5111.774622] ffff8800374ddc98 ffff8800374ddc88 ffffffff810370a9 ffff8800374ddd80 [ 5111.774711] ffffffffa020d524 00000000ffffffef ffff8807ead7d800 ffff8807ff0cc8c0 [ 5111.774800] Call Trace: [ 5111.774846] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 5111.774894] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 5111.774944] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775032] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 5111.775081] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775135] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 5111.775189] [<ffffffffa022fdec>] btrfs_commit_transaction+0x36d/0x7df [btrfs] [ 5111.775281] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 5111.775333] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 5111.775382] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 5111.775428] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775477] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 5111.775524] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775572] ---[ end trace b552aca9a0cff3cb ]--- [ 5111.775618]BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 5111.775707] BTRFS info (device sdb5):
forced readonly [ 5111.775754] BTRFS warning (device sdb5):
Skipping commit of aborted transaction. [ 5111.775841] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already existsIf I run balance (after unmounting and mounting the filesystem), it ends in a similar way:
[ 5927.338989] btrfs: relocating block group 4647284637696 flags 17 [ 6184.333629] btrfs: found 983 extents [ 6269.512577] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 6269.512579] ------------[ cut here ]------------ [ 6269.512636] WARNING: CPU: 7 PID: 17836 at fs/btrfs/super.c:254
__btrfs_abort_transaction+0x4d/0xff[btrfs]() [ 6269.512727] btrfs:
Transaction aborted (error -17) [ 6269.512727] Modules linked in:ipt_MASQUERADEiptable_natnf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop pcspkr button videoehci_pciehci_hcdacpi_cpufreqi2c_i801i2c_corelpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modahci libahci libatascsi_modr8169 mii [ 6269.513036] CPU: 7 PID: 17836 Comm:
btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 6269.513124]Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 6269.513214] 0000000000000009 ffff8807c5b07c48 ffffffff8138a37d 0000000000000006 [ 6269.513304] ffff8807c5b07c98 ffff8807c5b07c88 ffffffff810370a9 ffff8807c5b07d80 [ 6269.513393] ffffffffa020d524 00000000ffffffef ffff8807ead7e800 ffff8800371eedc0 [ 6269.513483] Call Trace: [ 6269.513529] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 6269.513577] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 6269.513627] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513716] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 6269.513766] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513820] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 6269.513874] [<ffffffffa022faef>] btrfs_commit_transaction+0x70/0x7df [btrfs] [ 6269.513927] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 6269.513980] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 6269.514029] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 6269.514076] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514125] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 6269.514172] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514220] ---[ end trace b552aca9a0cff3cc ]--- [ 6269.514266]BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 6269.514356] BTRFS info (device sdb5):
forced readonly [ 6269.514430] BTRFS warning (device sdb5):
Skipping commit of aborted transaction. [ 6269.514559] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already exists [ 6269.527034] btrfs: found 983 extentsIs there any fix I can try?
--
Tomasz Chmielewski
http://wpkg.org--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
2013/12/20 Tomasz Chmielewski <[email protected]>:
Quoted 7 lines
-o
skip_balance- didn't know this.Actually, I was able to "skip" the balance, sort of, with this:
mount
/mnt/btrfs; btrfs fi balance cancel/mnt/btrfsFrom your previous email, i suspend your filesystem is nearly fully.
"mount -o skip_balance" will avoid balance continuing while use btrfs balance cancel still
can not avoid balance totally, so i recommend you use skip_balance option when remounting.
Quoted 9 lines
However, the fs is extremely unstable (will remount read only quite fast, i.e. if I start removing snapshot).
I'm trying to run btrfsck without the --repair option to see if it shows anything interesting.
I'd like you not to use btrfsck --repair, it is a little dangerous anyway. If you still want to try, just use btrfsck is ok to check if your filesystem is consistent.
Or you can remount with option degraded, recovery(maybe help you to make filesystem stable
at least).
Thanks,
Wang
Quoted 143 lines
--
Tomasz Chmielewski
http://wpkg.orgQuoted 133 lines · On Fri, 20 Dec 2013 23:52:08 +0800 Shilong Wang <[email protected]>…
Hello Tomasz,
There should be some bugs flying on, unitil now, you can try to
remount: something like:# mount /dev/sda /mnt -o skip_balanceIt will skip unfinshed balance, hopley it can help you before we give a bug fix.
Thanks,
Wang
2013/12/20 Tomasz Chmielewski <[email protected]>:Quoted 120 lines
Quoted 3 lines · On Thu, 19 Dec 2013 22:07:37 +0900 Tomasz Chmielewski <[email protected]> wrote:
If it matters, I had to hard reboot after that bug; the balance continued after the system booted again and I got this a while later (filesystem was remounted read only):
Actually, looks like the fs is quite hosed now :(
If I want to do an operation like removing a snapshot (btrfs sub del ...), the command returns, there is IO for a few minutes, but then, the kernel complains and fs is remounted readonly:
[ 5111.773900] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 5111.773902] ------------[ cut here ]------------ [ 5111.773957] WARNING: CPU: 0 PID: 15042 at fs/btrfs/super.c:254
__btrfs_abort_transaction+0x4d/0xff[btrfs]() [ 5111.774047] btrfs:
Transaction aborted (error -17) [ 5111.774048] Modules linked in:ipt_MASQUERADEiptable_natnf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop pcspkr button videoehci_pciehci_hcdacpi_cpufreqi2c_i801i2c_corelpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modahci libahci libatascsi_modr8169 mii [ 5111.774355] CPU: 0 PID: 15042 Comm:
btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 5111.774443]Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 5111.774533] 0000000000000009 ffff8800374ddc48 ffffffff8138a37d 0000000000000006 [ 5111.774622] ffff8800374ddc98 ffff8800374ddc88 ffffffff810370a9 ffff8800374ddd80 [ 5111.774711] ffffffffa020d524 00000000ffffffef ffff8807ead7d800 ffff8807ff0cc8c0 [ 5111.774800] Call Trace: [ 5111.774846] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 5111.774894] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 5111.774944] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775032] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 5111.775081] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 5111.775135] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 5111.775189] [<ffffffffa022fdec>] btrfs_commit_transaction+0x36d/0x7df [btrfs] [ 5111.775281] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 5111.775333] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 5111.775382] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 5111.775428] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775477] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 5111.775524] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 5111.775572] ---[ end trace b552aca9a0cff3cb ]--- [ 5111.775618]BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 5111.775707] BTRFS info (device sdb5):
forced readonly [ 5111.775754] BTRFS warning (device sdb5):
Skipping commit of aborted transaction. [ 5111.775841] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already existsIf I run balance (after unmounting and mounting the filesystem), it ends in a similar way:
[ 5927.338989] btrfs: relocating block group 4647284637696 flags 17 [ 6184.333629] btrfs: found 983 extents [ 6269.512577] BTRFS debug (device sdb5): run_one_delayed_ref returned -17 [ 6269.512579] ------------[ cut here ]------------ [ 6269.512636] WARNING: CPU: 7 PID: 17836 at fs/btrfs/super.c:254
__btrfs_abort_transaction+0x4d/0xff[btrfs]() [ 6269.512727] btrfs:
Transaction aborted (error -17) [ 6269.512727] Modules linked in:ipt_MASQUERADEiptable_natnf_conntrack_ipv4nf_defrag_ipv4nf_nat_ipv4nf_natnf_conntrackip_tablesx_tablescpufreq_ondemandcpufreq_conservativecpufreq_powersavecpufreq_statsbridge stp llc ipv6 btrfs xorraid6_pqzlib_deflateloop pcspkr button videoehci_pciehci_hcdacpi_cpufreqi2c_i801i2c_corelpc_ichmfd_coreext4 crc16 jbd2 mbcache raid1 sgsd_modahci libahci libatascsi_modr8169 mii [ 6269.513036] CPU: 7 PID: 17836 Comm:
btrfs-transacti Tainted: G W 3.13.0-rc4 #1 [ 6269.513124]Hardware name: System manufacturer System Product Name/P8H77-M PRO, BIOS 1101 02/04/2013 [ 6269.513214] 0000000000000009 ffff8807c5b07c48 ffffffff8138a37d 0000000000000006 [ 6269.513304] ffff8807c5b07c98 ffff8807c5b07c88 ffffffff810370a9 ffff8807c5b07d80 [ 6269.513393] ffffffffa020d524 00000000ffffffef ffff8807ead7e800 ffff8800371eedc0 [ 6269.513483] Call Trace: [ 6269.513529] [<ffffffff8138a37d>] dump_stack+0x46/0x58 [ 6269.513577] [<ffffffff810370a9>] warn_slowpath_common+0x77/0x91 [ 6269.513627] [<ffffffffa020d524>] ? __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513716] [<ffffffff81037157>] warn_slowpath_fmt+0x41/0x43 [ 6269.513766] [<ffffffffa020d524>] __btrfs_abort_transaction+0x4d/0xff [btrfs] [ 6269.513820] [<ffffffffa02226ed>] btrfs_run_delayed_refs+0x253/0x46f [btrfs] [ 6269.513874] [<ffffffffa022faef>] btrfs_commit_transaction+0x70/0x7df [btrfs] [ 6269.513927] [<ffffffffa022e345>] transaction_kthread+0xef/0x1c2 [btrfs] [ 6269.513980] [<ffffffffa022e256>] ? open_ctree+0x1ac7/0x1ac7 [btrfs] [ 6269.514029] [<ffffffff8104ee9a>] kthread+0xcd/0xd5 [ 6269.514076] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514125] [<ffffffff8138f17c>] ret_from_fork+0x7c/0xb0 [ 6269.514172] [<ffffffff8104edcd>] ? kthread_freezable_should_stop+0x43/0x43 [ 6269.514220] ---[ end trace b552aca9a0cff3cc ]--- [ 6269.514266]BTRFS error (device sdb5) in btrfs_run_delayed_refs:2730: errno=-17 Object already exists [ 6269.514356] BTRFS info (device sdb5):
forced readonly [ 6269.514430] BTRFS warning (device sdb5):
Skipping commit of aborted transaction. [ 6269.514559] BTRFS error (device sdb5) in cleanup_transaction:1553: errno=-17 Object already exists [ 6269.527034] btrfs: found 983 extentsIs there any fix I can try?
--
Tomasz Chmielewski
http://wpkg.org--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 5 lines · On Sat, 21 Dec 2013 00:07:19 +0800 Shilong Wang <[email protected]>…
2013/12/20 Tomasz Chmielewski <[email protected]>:
Quoted 4 lines
-o
skip_balance- didn't know this.Actually, I was able to "skip" the balance, sort of, with this:
Quoted 7 lines
Quoted 1 line
mount
/mnt/btrfs; btrfs fi balance cancel/mnt/btrfsFrom your previous email, i suspend your filesystem is nearly fully. "mount -o
skip_balance" will avoid balance continuing while use btrfs balance cancel still
can not avoid balance totally, so i recommend you useskip_balanceoption when remounting.
My other thread a few days ago ("no space left, metadata usage almost full?"), if that's what you're referring to, was about a different filesystem.
This one should have enough free space left:
# btrfs fi
show /mnt/lxc1 Label: lxc1 uuid: 8d08ad6d-4543-4fe5-8b1b-640dc1423d41
Total devices 2 FS bytes used 2.02TiB
devid 1 size 2.62TiB used 2.02TiB path /dev/sda5
devid 2 size 2.62TiB used 2.02TiB path /dev/sdb5Btrfs v3.12
# btrfs fi df /mnt/lxc1 Data, RAID1: total=1.97TiB, used=1.97TiB System, RAID1: total=32.00MiB, used=300.00KiB Metadata, RAID1: total=50.00GiB, used=49.21GiB
I've tried using skip_balance - it mounted, but breaks soon after I try to remove any snapshot.
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org
2013/12/21 Tomasz Chmielewski <[email protected]>:
Quoted 30 lines
Quoted 5 lines · On Sat, 21 Dec 2013 00:07:19 +0800 Shilong Wang <[email protected]>…
2013/12/20 Tomasz Chmielewski <[email protected]>:
Quoted 4 lines
-o
skip_balance- didn't know this.Actually, I was able to "skip" the balance, sort of, with this:
Quoted 7 lines
Quoted 1 line
mount
/mnt/btrfs; btrfs fi balance cancel/mnt/btrfsFrom your previous email, i suspend your filesystem is nearly fully. "mount -o
skip_balance" will avoid balance continuing while use btrfs balance cancel still
can not avoid balance totally, so i recommend you useskip_balanceoption when remounting.My other thread a few days ago ("no space left, metadata usage almost full?"), if that's what you're referring to, was about a different filesystem.
This one should have enough free space left:
# btrfs fi show /mnt/lxc1 Label: lxc1 uuid: 8d08ad6d-4543-4fe5-8b1b-640dc1423d41 Total devices 2 FS bytes used 2.02TiB devid 1 size 2.62TiB used 2.02TiB path /dev/sda5 devid 2 size 2.62TiB used 2.02TiB path /dev/sdb5Btrfs v3.12
Quoted 9 lines
# btrfs fi df /mnt/lxc1 Data, RAID1: total=1.97TiB, used=1.97TiB System, RAID1: total=32.00MiB, used=300.00KiB Metadata, RAID1: total=50.00GiB, used=49.21GiBI've tried using
skip_balance- it mounted, but breaks soon after I try to remove any snapshot.
ok, It seems that your filesystem is not in consistency anymore. To confirm your filesystem status, you can try to run btrfsck to check if there is something wrong.
To get a stable fs now, you can try to mount btrfs with option skip_balance and recovery, recovery options try to get a previous good tree root which might help you
get a stable fs.
Quoted 4 lines
--
Tomasz Chmielewski
http://wpkg.org
Quoted 8 lines · On Sat, 21 Dec 2013 01:11:00 +0800 Shilong Wang <[email protected]>…
ok, It seems that your filesystem is not in consistency anymore. To confirm your filesystem status, you can try to run btrfsck to check if there is something wrong.
To get a stable fs now, you can try to mount btrfs with option
skip_balanceand recovery, recovery options try to get a previous good tree root which might help you
get a stable fs.
Unfortunately even after using these mount options it breaks as before, as soon as I remove snapshots, or even copy some data in with rsync.
Oh well, at least I have read only access...
Signature · 2 lines
Tomasz Chmielewski http://wpkg.org