Skip to contentSkip to conversation

Error during balancing '/': Input/output error

DISCUSSION Michael Ruiz 2 participants 9 messages Dec 27, 2019 – Dec 28, 2019 1 message not in the archive Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi,

I recently did a btrfs-convert from ext4 to btrfs and it went well, so I deleted the ext2 'home-backup' or whatever subvolume it created and I proceeded to balance the disk. However, I keep getting this error when I try to run this command (see bottom of message for debug info.) This might be a simple question, but I am worried that my root drive is corrupt and if I restart it might not boot so I haven't tried rebooting and mounting the disk from a live usb. Any suggestions pointing me in the right direction are most appreciated.

Thanks,
Michael

``` BEGIN DEBUG INFO ```
[michael@linux /]$ sudo btrfs balance /
ERROR: error during balancing '/': Input/output error
There may be more info in syslog - try dmesg | tail

[michael@linux /]$ dmesg | tail
[ 1235.433416] BTRFS info (device dm-1): relocating block group 265709158400 
flags metadata
[ 1375.701515] BTRFS info (device dm-1): found 65034 extents
[ 1381.270271] BTRFS info (device dm-1): relocating block group 264635416576 
flags metadata
[ 1483.628290] BTRFS info (device dm-1): found 64995 extents
[ 1485.160174] BTRFS info (device dm-1): relocating block group 253562454016 
flags data
[ 1485.352336] BTRFS warning (device dm-1): csum failed root -9 ino 265 off 
1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1
[ 1485.353008] BTRFS warning (device dm-1): csum failed root -9 ino 265 off 
1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1
[ 1485.375271] BTRFS info (device dm-1): balance: ended with status: -5
[ 1485.544838] audit: type=1106 audit(1577448716.970:152): pid=4178 uid=0 
auid=1000 ses=1 msg='op=PAM:session_close 
grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" 
hostname=? addr=? terminal=/dev/pts/2 res=success'
[ 1485.545220] audit: type=1104 audit(1577448716.970:153): pid=4178 uid=0 
auid=1000 ses=1 msg='op=PAM:setcred grantors=pam_unix,pam_permit,pam_env 
acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/2 
res=success'
``` END DEBUG INFO ```
#1 lore raw
Quoted 10 lines · Michael Ruiz

Hi,

I recently did a btrfs-convert from ext4 to btrfs and it went well, so I deleted the ext2 'home-backup' or whatever subvolume it created and I proceeded to balance the disk. However, I keep getting this error when I try to run this command (see bottom of message for debug info.) This might be a simple question, but I am worried that my root drive is corrupt and if I restart it might not boot so I haven't tried rebooting and mounting the disk from a live usb. Any suggestions pointing me in the right direction are most appreciated.

There are quite some reports about this problem. So far no real corrupted confirmed.

You can confirm it by scrub.

I'm afraid there is some bug in data balance code which leads to this. We will dig further to pin down the cause.

BTW, does that problem always show up at block group 253562454016? Or it randomly shows up at different block groups?

Thanks,
Qu

Quoted 34 lines

Thanks,
Michael

``` BEGIN DEBUG INFO ```
[michael@linux /]$ sudo btrfs balance /
ERROR: error during balancing '/': Input/output error
There may be more info in syslog - try dmesg | tail

[michael@linux /]$ dmesg | tail
[ 1235.433416] BTRFS info (device dm-1): relocating block group 265709158400 
flags metadata
[ 1375.701515] BTRFS info (device dm-1): found 65034 extents
[ 1381.270271] BTRFS info (device dm-1): relocating block group 264635416576 
flags metadata
[ 1483.628290] BTRFS info (device dm-1): found 64995 extents
[ 1485.160174] BTRFS info (device dm-1): relocating block group 253562454016 
flags data
[ 1485.352336] BTRFS warning (device dm-1): csum failed root -9 ino 265 off 
1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1
[ 1485.353008] BTRFS warning (device dm-1): csum failed root -9 ino 265 off 
1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1
[ 1485.375271] BTRFS info (device dm-1): balance: ended with status: -5
[ 1485.544838] audit: type=1106 audit(1577448716.970:152): pid=4178 uid=0 
auid=1000 ses=1 msg='op=PAM:session_close 
grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" 
hostname=? addr=? terminal=/dev/pts/2 res=success'
[ 1485.545220] audit: type=1104 audit(1577448716.970:153): pid=4178 uid=0 
auid=1000 ses=1 msg='op=PAM:setcred grantors=pam_unix,pam_permit,pam_env 
acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/2 
res=success'
``` END DEBUG INFO ```
#9 lore raw
In reply to Michael Ruiz
Quoted 5 lines

There are quite some reports about this problem. So far no real corrupted confirmed.

You can confirm it by scrub.

Quoted 8 lines

I'm afraid there is some bug in data balance code which leads to this. We will dig further to pin down the cause.

BTW, does that problem always show up at block group 253562454016? Or it randomly shows up at different block groups?

Thanks,
Qu

Hi Qu,
It appears to be the same data blocks giving me this issue. I have tried it while mounting from a bootable usb and in my arch linux install both with similar results. I should also mention I have a btrfs root partition within a luks lvm partition becuase btrfs provides no encryption. I'm not sure this makes a difference. Would you suggest running somekind of disk check or by running scrub? Please let me know what I should try to resolve this. 谢谢

``` BEGIN DEBUG INFO ```
ERROR: error during balancing '/': Input/output error
[michael@linux /]$ dmesg | tail
[  462.460626] audit: type=1104 audit(1577525576.905:124): pid=3415 uid=0 
auid=1000 ses=1 msg='op=PAM:setcred grantors=pam_unix,pam_permit,pam_env 
acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/2 
res=success'
[  465.880807] BTRFS info (device dm-1): found 25 extents
[  470.501319] BTRFS info (device dm-1): found 25 extents
[  471.713073] BTRFS info (device dm-1): relocating block group 253562454016 
flags data
[  472.251356] BTRFS warning (device dm-1): csum failed root -9 ino 262 off 
1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1
[  472.254390] BTRFS warning (device dm-1): csum failed root -9 ino 262 off 
1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1
[  472.716215] BTRFS info (device dm-1): balance: ended with status: -5
[  473.122721] audit: type=1106 audit(1577525587.565:125): pid=2775 uid=0 
auid=1000 ses=1 msg='op=PAM:session_close 
grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" 
hostname=? addr=? terminal=/dev/pts/3 res=success'
[  473.122840] audit: type=1104 audit(1577525587.565:126): pid=2775 uid=0 
auid=1000 ses=1 msg='op=PAM:setcred grantors=pam_unix,pam_permit acct="root" 
exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/3 res=success'
[  629.160619] audit: type=1131 audit(1577525743.603:127): pid=1 uid=0 
auid=4294967295 ses=4294967295 msg='unit=packagekit comm="systemd" exe="/usr/
lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
``` END DEBUG INFO ```
#3 lore raw
In reply to a message not in the archive
Quoted 11 lines · Michael Ruiz
Quoted 11 lines

There are quite some reports about this problem. So far no real corrupted confirmed.

You can confirm it by scrub.

I'm afraid there is some bug in data balance code which leads to this. We will dig further to pin down the cause.

BTW, does that problem always show up at block group 253562454016? Or it randomly shows up at different block groups?

Quoted 7 lines
Quoted 2 lines

Thanks,
Qu

Hi Qu,
It appears to be the same data blocks giving me this issue. I have tried it while mounting from a bootable usb and in my arch linux install both with similar results.

So it looks like you will hit the same -EIO even just relocating that specific block group using vrange filter.

Quoted 3 lines

I should also mention I have a btrfs root partition within a luks lvm partition becuase btrfs provides no encryption. I'm not sure this makes a difference.

I don't believe it would cause any difference.

Quoted 2 lines

Would you suggest running somekind of disk check or by running scrub? Please let me know what I should try to resolve this.

Running a scrub is always preferred.
An btrfs check run would be even better.

If all above reports no error, we need more info to dig. (See below).

Quoted 10 lines

谢谢

``` BEGIN DEBUG INFO ```
ERROR: error during balancing '/': Input/output error

[  465.880807] BTRFS info (device dm-1): found 25 extents
[  470.501319] BTRFS info (device dm-1): found 25 extents
[  471.713073] BTRFS info (device dm-1): relocating block group 253562454016 
flags data
[  472.251356] BTRFS warning (device dm-1): csum failed root -9 ino 262 off 

1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1

Another question, is the reported csum (both have and expected) always the same?

Quoted 2 lines
[  472.254390] BTRFS warning (device dm-1): csum failed root -9 ino 262 off 

1048576 csum 0x9a7afaa8 expected csum 0x63e15594 mirror 1

This line means two things:
- We have extra inodes in data reloc tree
Normally data reloc only has one inode, 256.
It's possible we have some extra orphan inodes.
Would you like to do the following dump?

  # btrfs ins dump-tree -t data_reloc /dev/dm-1

Although the result is not something representing the content when we hit the bug, it may still provide some clue.

- The original data is from logical 253563502592 Would you like to check which file(s) owns that logical? It can be done by the following command:

  # btrfs ins logical-resolve 253563502592 <mnt>

Unlike previous command, it needs to be executed on a mounted btrfs other than the block device.

And the info about that inode is also helpful, like its attr (nodatacow?).

Thanks,
Qu

Quoted 2 lines

``` END DEBUG INFO ```

#4 lore raw

Hi Qu,
Thanks for the quick reply. This is what I could find.

[michael@linux /]$ sudo btrfs ins dump-tree -t data_reloc /dev/dm-1 btrfs-progs v5.4
data reloc tree key (DATA_RELOC_TREE ROOT_ITEM 2)
leaf 286941609984 items 2 free space 16061 generation 1186381 owner DATA_RELOC_TREE
leaf 286941609984 flags 0x1(WRITTEN) backref revision 1 fs uuid <redacted>
chunk uuid <redacted>

        item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
                generation 2 transid 0 size 0 nbytes 16384
                block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0
                sequence 0 flags 0x0(none)
                atime 1577186710.0 (2019-12-24 03:25:10)
                ctime 1577186710.0 (2019-12-24 03:25:10)
                mtime 1577186710.0 (2019-12-24 03:25:10)
                otime 1577186710.0 (2019-12-24 03:25:10)
        item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12
                index 0 namelen 2 name: ..

total bytes 255307284480
bytes used 94295339008
uuid <redacted>

[michael@archlinux /]$ sudo btrfs ins logical-resolve 253563502592 / //home/michael/.mozilla/firefox/default/storage/default/https++ +www.pinterest.com/cache/morgue/16/{b696bf53-d26a-48eb-9688- ab3c5fd49010}.final

``` BTRFS SCRUB ```
sudo btrfs scrub status /
UUID:             <redacted>
Scrub started:    Sat Dec 28 03:17:49 2019
Status:           finished
Duration:         0:05:09
Total to scrub:   87.82GiB
Rate:             291.04MiB/s
Error summary:    csum=16
  Corrected:      0
  Uncorrectable:  16
  Unverified:     0
```
#5 lore raw
Quoted 32 lines · Michael Ruiz

Hi Qu,
Thanks for the quick reply. This is what I could find.

[michael@linux /]$ sudo btrfs ins dump-tree -t data_reloc /dev/dm-1 btrfs-progs v5.4
data reloc tree key (DATA_RELOC_TREE ROOT_ITEM 2)
leaf 286941609984 items 2 free space 16061 generation 1186381 owner DATA_RELOC_TREE
leaf 286941609984 flags 0x1(WRITTEN) backref revision 1 fs uuid <redacted>
chunk uuid <redacted>

        item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
                generation 2 transid 0 size 0 nbytes 16384
                block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0
                sequence 0 flags 0x0(none)
                atime 1577186710.0 (2019-12-24 03:25:10)
                ctime 1577186710.0 (2019-12-24 03:25:10)
                mtime 1577186710.0 (2019-12-24 03:25:10)
                otime 1577186710.0 (2019-12-24 03:25:10)
        item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12
                index 0 namelen 2 name: ..

total bytes 255307284480
bytes used 94295339008
uuid <redacted>

[michael@archlinux /]$ sudo btrfs ins logical-resolve 253563502592 / //home/michael/.mozilla/firefox/default/storage/default/https++ +www.pinterest.com/cache/morgue/16/{b696bf53-d26a-48eb-9688- ab3c5fd49010}.final

Quoted 13 lines
``` BTRFS SCRUB ```
sudo btrfs scrub status /
UUID:             <redacted>
Scrub started:    Sat Dec 28 03:17:49 2019
Status:           finished
Duration:         0:05:09
Total to scrub:   87.82GiB
Rate:             291.04MiB/s
Error summary:    csum=16
  Corrected:      0
  Uncorrectable:  16
  Unverified:     0
```

Dmesg please, as that shows which file(s) is involved.

Thanks,
Qu

#6 lore raw
Quoted 5 lines · PST you

Dmesg please, as that shows which file(s) is involved.

Thanks,
Qu

Sure thing, it appears they are all from firefox cache.. Perhaps they are failing checksum because they were modified since I converted the partition earlier this week? I'm just guessing. Would you reccomend deleting these files manually or running btrfs check?
Thanks

``` BEGIN DMESG OUTPUT ```
[ 5638.996798] BTRFS warning (device dm-1): checksum error at logical 
253563510784 on dev /dev/mapper/volgroup0-lv_root, physical 85319491584, root 
5, inode 6477931, offset 8192, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.996805] BTRFS warning (device dm-1): checksum error at logical 
253563502592 on dev /dev/mapper/volgroup0-lv_root, physical 85319483392, root 
5, inode 6477931, offset 0, length 4096, links 1 (path: home/michael/.mozilla/
firefox/default/storage/default/https+++www.pinterest.com/cache/morgue/16/
{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.996808] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 17, gen 0
[ 5638.996812] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563510784 on dev /dev/mapper/volgroup0-lv_root
[ 5638.996815] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 18, gen 0
[ 5638.996818] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563502592 on dev /dev/mapper/volgroup0-lv_root
[ 5638.997338] BTRFS warning (device dm-1): checksum error at logical 
253563506688 on dev /dev/mapper/volgroup0-lv_root, physical 85319487488, root 
5, inode 6477931, offset 4096, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.997344] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 19, gen 0
[ 5638.997348] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563506688 on dev /dev/mapper/volgroup0-lv_root
[ 5638.997526] BTRFS warning (device dm-1): checksum error at logical 
253563514880 on dev /dev/mapper/volgroup0-lv_root, physical 85319495680, root 
5, inode 6477931, offset 12288, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.997531] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 20, gen 0
[ 5638.997533] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563514880 on dev /dev/mapper/volgroup0-lv_root
[ 5638.998086] BTRFS warning (device dm-1): checksum error at logical 
253563518976 on dev /dev/mapper/volgroup0-lv_root, physical 85319499776, root 
5, inode 6477931, offset 16384, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.998097] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 21, gen 0
[ 5638.998101] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563518976 on dev /dev/mapper/volgroup0-lv_root
[ 5638.998542] BTRFS warning (device dm-1): checksum error at logical 
253563523072 on dev /dev/mapper/volgroup0-lv_root, physical 85319503872, root 
5, inode 6477931, offset 20480, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.998551] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 22, gen 0
[ 5638.998554] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563523072 on dev /dev/mapper/volgroup0-lv_root
[ 5638.998975] BTRFS warning (device dm-1): checksum error at logical 
253563527168 on dev /dev/mapper/volgroup0-lv_root, physical 85319507968, root 
5, inode 6477931, offset 24576, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.998981] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 23, gen 0
[ 5638.998984] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563527168 on dev /dev/mapper/volgroup0-lv_root
[ 5638.999356] BTRFS warning (device dm-1): checksum error at logical 
253563531264 on dev /dev/mapper/volgroup0-lv_root, physical 85319512064, root 
5, inode 6477931, offset 28672, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.999362] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 24, gen 0
[ 5638.999366] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563531264 on dev /dev/mapper/volgroup0-lv_root
[ 5638.999592] BTRFS warning (device dm-1): checksum error at logical 
253563535360 on dev /dev/mapper/volgroup0-lv_root, physical 85319516160, root 
5, inode 6477931, offset 32768, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.999596] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 25, gen 0
[ 5638.999598] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563535360 on dev /dev/mapper/volgroup0-lv_root
[ 5638.999861] BTRFS warning (device dm-1): checksum error at logical 
253563539456 on dev /dev/mapper/volgroup0-lv_root, physical 85319520256, root 
5, inode 6477931, offset 36864, length 4096, links 1 (path: home/
michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/
cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)
[ 5638.999866] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 26, gen 0
[ 5638.999869] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563539456 on dev /dev/mapper/volgroup0-lv_root
[ 5692.239308] BTRFS info (device dm-1): scrub: finished on devid 1 with 
status: 0
``` END DMESG OUTPUT ```
#7 lore raw
Quoted 13 lines · Michael Ruiz
Quoted 5 lines · PST you

Dmesg please, as that shows which file(s) is involved.

Thanks,
Qu

Sure thing, it appears they are all from firefox cache.. Perhaps they are failing checksum because they were modified since I converted the partition earlier this week? I'm just guessing. Would you reccomend deleting these files manually or running btrfs check?
Thanks

Quoted 6 lines

``` BEGIN DMESG OUTPUT ```

[ 5638.996798] BTRFS warning (device dm-1): checksum error at logical 

253563510784 on dev /dev/mapper/volgroup0-lv_root, physical 85319491584, root 5, inode 6477931, offset 8192, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

Doesn't this filename look familiar with previous logical-resolve output? :)

[michael@archlinux /]$ sudo btrfs ins logical-resolve 253563502592 / //home/michael/.mozilla/firefox/default/storage/default/https++ +www.pinterest.com/cache/morgue/16/{b696bf53-d26a-48eb-9688- ab3c5fd49010}.final

The URL, the UUID, they all matches!

So yep, the file itself doesn't match its csum in the first place.

It looks like balance doesn't go regular file csum checking, but copy all data and all csum without verifying them, then check the csum of data reloc tree instead.

Thus this causes this hard-to-locate corrupted files. We should make it more user-friendly I guess.

And obviously, just remove the offending files should allow you continue.

Thanks,
Qu

Quoted 89 lines
[ 5638.996805] BTRFS warning (device dm-1): checksum error at logical 

253563502592 on dev /dev/mapper/volgroup0-lv_root, physical 85319483392, root 5, inode 6477931, offset 0, length 4096, links 1 (path: home/michael/.mozilla/ firefox/default/storage/default/https+++www.pinterest.com/cache/morgue/16/ {b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.996808] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 17, gen 0
[ 5638.996812] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563510784 on dev /dev/mapper/volgroup0-lv_root
[ 5638.996815] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 18, gen 0
[ 5638.996818] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563502592 on dev /dev/mapper/volgroup0-lv_root
[ 5638.997338] BTRFS warning (device dm-1): checksum error at logical 

253563506688 on dev /dev/mapper/volgroup0-lv_root, physical 85319487488, root 5, inode 6477931, offset 4096, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.997344] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 19, gen 0
[ 5638.997348] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563506688 on dev /dev/mapper/volgroup0-lv_root
[ 5638.997526] BTRFS warning (device dm-1): checksum error at logical 

253563514880 on dev /dev/mapper/volgroup0-lv_root, physical 85319495680, root 5, inode 6477931, offset 12288, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.997531] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 20, gen 0
[ 5638.997533] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563514880 on dev /dev/mapper/volgroup0-lv_root
[ 5638.998086] BTRFS warning (device dm-1): checksum error at logical 

253563518976 on dev /dev/mapper/volgroup0-lv_root, physical 85319499776, root 5, inode 6477931, offset 16384, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.998097] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 21, gen 0
[ 5638.998101] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563518976 on dev /dev/mapper/volgroup0-lv_root
[ 5638.998542] BTRFS warning (device dm-1): checksum error at logical 

253563523072 on dev /dev/mapper/volgroup0-lv_root, physical 85319503872, root 5, inode 6477931, offset 20480, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.998551] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 22, gen 0
[ 5638.998554] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563523072 on dev /dev/mapper/volgroup0-lv_root
[ 5638.998975] BTRFS warning (device dm-1): checksum error at logical 

253563527168 on dev /dev/mapper/volgroup0-lv_root, physical 85319507968, root 5, inode 6477931, offset 24576, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.998981] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 23, gen 0
[ 5638.998984] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563527168 on dev /dev/mapper/volgroup0-lv_root
[ 5638.999356] BTRFS warning (device dm-1): checksum error at logical 

253563531264 on dev /dev/mapper/volgroup0-lv_root, physical 85319512064, root 5, inode 6477931, offset 28672, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.999362] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 24, gen 0
[ 5638.999366] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563531264 on dev /dev/mapper/volgroup0-lv_root
[ 5638.999592] BTRFS warning (device dm-1): checksum error at logical 

253563535360 on dev /dev/mapper/volgroup0-lv_root, physical 85319516160, root 5, inode 6477931, offset 32768, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.999596] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 25, gen 0
[ 5638.999598] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563535360 on dev /dev/mapper/volgroup0-lv_root
[ 5638.999861] BTRFS warning (device dm-1): checksum error at logical 

253563539456 on dev /dev/mapper/volgroup0-lv_root, physical 85319520256, root 5, inode 6477931, offset 36864, length 4096, links 1 (path: home/ michael/.mozilla/firefox/default/storage/default/https+++www.pinterest.com/ cache/morgue/16/{b696bf53-d26a-48eb-9688-ab3c5fd49010}.final)

[ 5638.999866] BTRFS error (device dm-1): bdev /dev/mapper/volgroup0-lv_root 
errs: wr 0, rd 0, flush 0, corrupt 26, gen 0
[ 5638.999869] BTRFS error (device dm-1): unable to fixup (regular) error at 
logical 253563539456 on dev /dev/mapper/volgroup0-lv_root
[ 5692.239308] BTRFS info (device dm-1): scrub: finished on devid 1 with 

status: 0
``` END DMESG OUTPUT ```

#8 lore raw
Quoted 15 lines · PST you

Doesn't this filename look familiar with previous logical-resolve output? :)

[michael@archlinux /]$ sudo btrfs ins logical-resolve 253563502592 / //home/michael/.mozilla/firefox/default/storage/default/https++ +www.pinterest.com/cache/morgue/16/{b696bf53-d26a-48eb-9688- ab3c5fd49010}.final

The URL, the UUID, they all matches!

So yep, the file itself doesn't match its csum in the first place.

It looks like balance doesn't go regular file csum checking, but copy all data and all csum without verifying them, then check the csum of data reloc tree instead.

Quoted 8 lines

Thus this causes this hard-to-locate corrupted files. We should make it more user-friendly I guess.

And obviously, just remove the offending files should allow you continue.

Thanks,
Qu

Wow, you are right. I was worried my file system was corrupted and I'm not sure the exact cause leading up to this error. If there is any more debug info I can give you so that you can get a better idea of why this is happening, please let me know.

Currently I am running balance, but I predict it will finally be able to complete sucessfully. However, if there are any issues I will send a reply here. (If I don't figure it out myself using what you just taught me) Thanks a lot for your help. I am new to using mailing lists such as these, so hopefully I did everything by the rules here.
Thanks again Qu!