Skip to contentSkip to conversation

soft lockup when using btrfs dev del

DISCUSSION Charles Wright 2 participants 3 messages Nov 8, 2015 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hello,
I'm running Linux 4.3 and also btrfs-progs-v4.3

I'm replacing one drive with another in my system. Both drives are healthy as far as I know.

I can start btrfs dev del and it will run for a while and then I get a cpu soft lockup.

I've restarted the process a few times and each time it runs a while, getting a bit further and then I get the cpu soft lockup again. (at which point the I/O has stopped and I typically have to hard power the system)

I do this to monitor the progress.

root@x1:~# while sleep 60; do /apps/src/btrfs-progs-v4.3/btrfs fi show /mnt/btrfs-seagate2tbp3-root; done
Label: none  uuid: eebd1291-7ccd-49da-b3c3-93942527ead4
    Total devices 2 FS bytes used 1.48TiB
    devid    1 size 0.00B used 420.04GiB path 
/dev/mapper/seagate2tbp3_crypt
    devid    2 size 1.82TiB used 1.08TiB path /dev/mapper/sde1_crypt

btrfs-progs v4.3
Label: none uuid: eebd1291-7ccd-49da-b3c3-93942527ead4
Total devices 2 FS bytes used 1.48TiB
devid 1 size 0.00B used 418.04GiB path
/dev/mapper/seagate2tbp3_crypt
devid 2 size 1.82TiB used 1.08TiB path /dev/mapper/sde1_crypt

Any suggestions on how to prevent this from happening?

Thanks.

Nov  8 01:02:30 x1 kernel: [22954.038620] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [btrfs:3086]
Nov  8 01:02:30 x1 kernel: [22954.038621] Modules linked in: pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables usblp snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic nls_utf8 nls_cp437 vfat intel_rapl fat iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm snd_hda_intel snd_hda_codec snd_hda_core snd_pcsp snd_hwdep serio_raw snd_pcm snd_timer snd soundcore i2c_i801 efivars i915 evdev i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm mei_me shpchp mei battery wmi video button acpi_pad processor fuse efivarfs autofs4 algif_skcipher af_alg hid_generic usbhid dm_crypt dm_mod sg sr_mod cdrom sd_mod crct10dif_pclmul crc32_pclmul crc32c_intel jitterentropy_rng sha256_ssse3 sha256_generic hmac drbg ansi_cprng aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ahci libahci xhci_pci libata xhci_hcd scsi_mod usbcore usb_common fan thermal i2c_hid i2c_core hid
Nov  8 01:02:30 x1 kernel: [22954.038684] CPU: 1 PID: 3086 Comm: btrfs Tainted: G        W  O    4.3.0 #1
Nov  8 01:02:30 x1 kernel: [22954.038685] Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./H170M-DS3H-CF, BIOS F2 10/14/2015
Nov  8 01:02:30 x1 kernel: [22954.038686] task: ffff8803f30e91c0 ti: ffff8803f2e28000 task.ti: ffff8803f2e28000
Nov  8 01:02:30 x1 kernel: [22954.038687] RIP: 0010:[<ffffffff815e5060>]  [<ffffffff815e5060>] _raw_spin_lock+0x10/0x20
Nov  8 01:02:30 x1 kernel: [22954.038702] RSP: 0018:ffff8803f2e2baf8  EFLAGS: 00000246
Nov  8 01:02:30 x1 kernel: [22954.038703] RAX: 0000000000000000 RBX: ffff8803f2e2bb38 RCX: 0000000000000000
Nov  8 01:02:30 x1 kernel: [22954.038704] RDX: 0000000000000001 RSI: ffff880471c9cf00 RDI: ffff88028d94dd70
Nov  8 01:02:30 x1 kernel: [22954.038704] RBP: ffff880131e94800 R08: ffff88046108a000 R09: 00000000ffffffe2
Nov  8 01:02:30 x1 kernel: [22954.038705] R10: 000000000000002a R11: 000000000000000b R12: ffff88028d94d800
Nov  8 01:02:30 x1 kernel: [22954.038706] R13: ffff8803f2e2bb38 R14: ffff8801e6a48800 R15: ffff88028d94dd78
Nov  8 01:02:30 x1 kernel: [22954.038706] FS: 00007f9ad24948c0(0000) GS:ffff880471c80000(0000) knlGS:0000000000000000
Nov  8 01:02:30 x1 kernel: [22954.038707] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov  8 01:02:30 x1 kernel: [22954.038708] CR2: 00000000014e2d38 CR3: 00000003d8989000 CR4: 00000000003406e0
Nov  8 01:02:30 x1 kernel: [22954.038709] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov  8 01:02:30 x1 kernel: [22954.038709] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Nov  8 01:02:30 x1 kernel: [22954.038710] Stack:
Nov  8 01:02:30 x1 kernel: [22954.038710]  ffffffff812f41e7 ffff8803f2e2bb38 ffff880131e94800 ffff88028d94d800
Nov  8 01:02:30 x1 kernel: [22954.038711]  ffffffff812f42dd ffff880131e94bc0 ffffffff812f8b00 000000005698f3c0
Nov  8 01:02:30 x1 kernel: [22954.038713]  ffff880131e94bc0 ffff880131e94bc0 ffff88028d94d8e8 ffff8802a77e6f50
Nov  8 01:02:30 x1 kernel: [22954.038714] Call Trace:
Nov  8 01:02:30 x1 kernel: [22954.038716] [<ffffffff812f41e7>] ? __del_reloc_root+0x27/0x100
Nov  8 01:02:30 x1 kernel: [22954.038718] [<ffffffff812f42dd>] ? free_reloc_roots+0x1d/0x30
Nov  8 01:02:30 x1 kernel: [22954.038719] [<ffffffff812f8b00>] ? merge_reloc_roots+0x170/0x230
Nov  8 01:02:30 x1 kernel: [22954.038720] [<ffffffff812f8ffc>] ? relocate_block_group+0x43c/0x670
Nov  8 01:02:30 x1 kernel: [22954.038721] [<ffffffff812f93d0>] ? btrfs_relocate_block_group+0x1a0/0x280
Nov  8 01:02:30 x1 kernel: [22954.038723] [<ffffffff812d1691>] ? btrfs_relocate_chunk.isra.36+0x41/0xc0
Nov  8 01:02:30 x1 kernel: [22954.038724] [<ffffffff812d203b>] ? btrfs_shrink_device+0x19b/0x520
Nov  8 01:02:30 x1 kernel: [22954.038725] [<ffffffff812d6058>] ? btrfs_rm_device+0x2a8/0x820
Nov  8 01:02:30 x1 kernel: [22954.038727] [<ffffffff812e1869>] ? btrfs_ioctl+0x19d9/0x27b0
Nov  8 01:02:30 x1 kernel: [22954.038729] [<ffffffff811b605a>] ? filename_lookup+0xda/0x150
Nov  8 01:02:30 x1 kernel: [22954.038731] [<ffffffff810ee1ae>] ? from_kgid_munged+0xe/0x20
Nov  8 01:02:30 x1 kernel: [22954.038733] [<ffffffff811ac053>] ? cp_new_stat+0x143/0x160
Nov  8 01:02:30 x1 kernel: [22954.038734] [<ffffffff811b9943>] ? do_vfs_ioctl+0x2c3/0x490
Nov  8 01:02:30 x1 kernel: [22954.038735] [<ffffffff811ac099>] ? SYSC_newstat+0x29/0x40
Nov  8 01:02:30 x1 kernel: [22954.038737] [<ffffffff811b9b4b>] ? SyS_ioctl+0x3b/0x70
Nov  8 01:02:30 x1 kernel: [22954.038738] [<ffffffff815e53b6>] ? entry_SYSCALL_64_fastpath+0x16/0x75
Nov  8 01:02:30 x1 kernel: [22954.038739] Code: 8b 07 85 c0 74 03 31 c0 c3 ba 01 00 00 00 f0 0f b1 17 85 c0 75 f0 b0 01 c3 66 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 f0 0f b1 17 <85> c0 75 01 c3 89 c6 e9 a4 34 ac ff 0f 1f 40 00 0f 1f 44 00 00
Signature · 4 lines
Copyright © Charles Wright 2015 All Rights Reserved.
No part of this message may be reproduced, forwarded
or published without Charles Wright's express written consent.
Thanks for your cooperation.
#1 lore raw

Charles Wright posted on Sun, 08 Nov 2015 11:59:41 -0500 as excerpted:

Since you claim full copyright and don't allow quotes without written consent... the normal behavior of quoting and replying in context can't be done, as the implied consent when posting to a list isn't enough to meet the higher legal barrier you claim.

So, I can't quote the problem, and thus, can't reply in context...

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#2 lore raw

Hi Duncan,
I give my written permission for people on this list to quote the
problem.
That's my standard email signature that I should have probably removed in this case and I wasn't intending it to prevent a reply.

Thanks.