Skip to contentSkip to conversation

5.5 kernel and btrfs-progs v5.6 create and cannot fix 'root 204162 inode 14058737 errors 1000, some csum missing'

DISCUSSION Marc MERLIN 2 participants 9 messages May 24, 2020 – May 26, 2020 Btrfs file system

lore
Share & tools
lore
#0 lore raw

My data is fine, it's double backed up and the filesystem is still mountable without issues. But I had an error that broke btrfs send, and after fixing it with repair, I'm stuck with thses 'csum missing'

Any idea if I can fix them without deleting the filesystem?

Current state:

saruman:~# btrfs check --repair /dev/mapper/cr
enabling repair mode
Starting repair.
Opening filesystem to check...
Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
root 356 inode 14058737 errors 1000, some csum missing
root 197362 inode 14058737 errors 1000, some csum missing
root 199551 inode 14058737 errors 1000, some csum missing
(snip)
root 204167 inode 14058737 errors 1000, some csum missing
root 204168 inode 14058737 errors 1000, some csum missing
ERROR: errors found in fs roots
found 112392159232 bytes used, error(s) found
total csum bytes: 105763716
total tree bytes: 4036018176
total fs tree bytes: 3666313216
total extent tree bytes: 214728704
btree space waste bytes: 875560453
file data blocks allocated: 207903920128
 referenced 206918037504

I can run multiple times, 'some csum missing' does not get fixed.

Lowmem does not fix it either:

saruman:~# btrfs check --mode=lowmem --repair /dev/mapper/cr
Starting repair.
WARNING: low-memory mode repair support is only partial
Opening filesystem to check...
Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
ERROR: root 356 EXTENT_DATA[14058737 0] csum missing, have: 0, expected: 8192
ERROR: root 356 EXTENT_DATA[14058737 0] compressed extent must have csum, but only 0 bytes have, expect 8192
Add a hole [0, 4096] in inode [33037178]
Add a hole [0, 4096] in inode [33037180]
Add a hole [0, 4096] in inode [33037181]
Add a hole [0, 4096] in inode [33037188]
Add a hole [0, 4096] in inode [33037189]
Add a hole [0, 4096] in inode [33037191]
ERROR: root 197362 EXTENT_DATA[14058737 0] csum missing, have: 0, expected: 8192
ERROR: root 197362 EXTENT_DATA[14058737 0] compressed extent must have csum, but only 0 bytes have, expect 8192
Add a hole [0, 4096] in inode [33037178]
Add a hole [0, 4096] in inode [33037180]
(...)
Add a hole [0, 4096] in inode [33037189]
Add a hole [0, 4096] in inode [33037191]
ERROR: errors found in fs roots
found 112392294400 bytes used, error(s) found
total csum bytes: 105763716
total tree bytes: 4035985408
total fs tree bytes: 3666313216
total extent tree bytes: 214728704
btree space waste bytes: 875550920
file data blocks allocated: 207903920128
 referenced 206918037504

This originally started with:
BTRFS error (device dm-0): did not find backref in send_root. inode=14058737, offset=0, disk_byte=2694234112 found extent=2694234112

which I was able to fix with the repair below, but now I'm stuck with the 'some csum missing'. Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
[2/7] checking extents
data backref 2694234112 root 356 owner 14058737 offset 0 num_refs 0 not found in extent tree incorrect local backref count on 2694234112 root 356 owner 14058737 offset 0 found 1 wanted 0 back 0x55e7383a3a00 incorrect local backref count on 2694234112 root 2147484004 owner 14058737 offset 0 found 0 wanted 1 back 0x55e733d1c2b0 backref disk bytenr does not match extent record, bytenr=2694234112, ref bytenr=0
backpointer mismatch on [2694234112 8192]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space cache
[4/7] checking fs roots
root 356 inode 33037179 errors 100, file extent discount
Found file extent holes:

        start: 0, len: 8192
root 203332 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 203669 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204006 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204010 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204024 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204037 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204049 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204167 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204168 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192

ERROR: errors found in fs roots
found 111202623488 bytes used, error(s) found
total csum bytes: 105763728
total tree bytes: 2846048256
total fs tree bytes: 2485485568
total extent tree bytes: 205619200
btree space waste bytes: 668593131
file data blocks allocated: 149272375296
referenced 140124020736

Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08
#1 lore raw

I forgot to add:

I'll keep my system in the broken state for maybe a day or two at most, but given that this is my root filesystem and that I'm booted off a backup media, I can't stay in that state too long.

This may not be nothing new and if it's already known that the way out is to wipe and restore backups, let me know.

Thnaks,
Marc

Quoted 137 lines · Marc MERLIN

My data is fine, it's double backed up and the filesystem is still mountable without issues. But I had an error that broke btrfs send, and after fixing it with repair, I'm stuck with thses 'csum missing'

Any idea if I can fix them without deleting the filesystem?

Current state:

saruman:~# btrfs check --repair /dev/mapper/cr
enabling repair mode
Starting repair.
Opening filesystem to check...
Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
root 356 inode 14058737 errors 1000, some csum missing
root 197362 inode 14058737 errors 1000, some csum missing
root 199551 inode 14058737 errors 1000, some csum missing
(snip)
root 204167 inode 14058737 errors 1000, some csum missing
root 204168 inode 14058737 errors 1000, some csum missing
ERROR: errors found in fs roots
found 112392159232 bytes used, error(s) found
total csum bytes: 105763716
total tree bytes: 4036018176
total fs tree bytes: 3666313216
total extent tree bytes: 214728704
btree space waste bytes: 875560453
file data blocks allocated: 207903920128
 referenced 206918037504

I can run multiple times, 'some csum missing' does not get fixed.

Lowmem does not fix it either:

saruman:~# btrfs check --mode=lowmem --repair /dev/mapper/cr
Starting repair.
WARNING: low-memory mode repair support is only partial
Opening filesystem to check...
Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
Fixed 0 roots.
[2/7] checking extents
No device size related problem found
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
ERROR: root 356 EXTENT_DATA[14058737 0] csum missing, have: 0, expected: 8192
ERROR: root 356 EXTENT_DATA[14058737 0] compressed extent must have csum, but only 0 bytes have, expect 8192
Add a hole [0, 4096] in inode [33037178]
Add a hole [0, 4096] in inode [33037180]
Add a hole [0, 4096] in inode [33037181]
Add a hole [0, 4096] in inode [33037188]
Add a hole [0, 4096] in inode [33037189]
Add a hole [0, 4096] in inode [33037191]
ERROR: root 197362 EXTENT_DATA[14058737 0] csum missing, have: 0, expected: 8192
ERROR: root 197362 EXTENT_DATA[14058737 0] compressed extent must have csum, but only 0 bytes have, expect 8192
Add a hole [0, 4096] in inode [33037178]
Add a hole [0, 4096] in inode [33037180]
(...)
Add a hole [0, 4096] in inode [33037189]
Add a hole [0, 4096] in inode [33037191]
ERROR: errors found in fs roots
found 112392294400 bytes used, error(s) found
total csum bytes: 105763716
total tree bytes: 4035985408
total fs tree bytes: 3666313216
total extent tree bytes: 214728704
btree space waste bytes: 875550920
file data blocks allocated: 207903920128
 referenced 206918037504

This originally started with:
BTRFS error (device dm-0): did not find backref in send_root. inode=14058737, offset=0, disk_byte=2694234112 found extent=2694234112

which I was able to fix with the repair below, but now I'm stuck with the 'some csum missing'. Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
[2/7] checking extents
data backref 2694234112 root 356 owner 14058737 offset 0 num_refs 0 not found in extent tree incorrect local backref count on 2694234112 root 356 owner 14058737 offset 0 found 1 wanted 0 back 0x55e7383a3a00 incorrect local backref count on 2694234112 root 2147484004 owner 14058737 offset 0 found 0 wanted 1 back 0x55e733d1c2b0 backref disk bytenr does not match extent record, bytenr=2694234112, ref bytenr=0
backpointer mismatch on [2694234112 8192]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space cache
[4/7] checking fs roots
root 356 inode 33037179 errors 100, file extent discount
Found file extent holes:

        start: 0, len: 8192
root 203332 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 203669 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204006 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204010 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204024 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204037 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204049 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204167 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192
root 204168 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192

ERROR: errors found in fs roots
found 111202623488 bytes used, error(s) found
total csum bytes: 105763728
total tree bytes: 2846048256
total fs tree bytes: 2485485568
total extent tree bytes: 205619200
btree space waste bytes: 668593131
file data blocks allocated: 149272375296
referenced 140124020736

Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08
Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/  
#2 lore raw
In reply to Marc MERLIN
Quoted 3 lines · Marc MERLIN

My data is fine, it's double backed up and the filesystem is still mountable without issues. But I had an error that broke btrfs send, and after fixing it with repair, I'm stuck with thses 'csum missing'

I'm not following the sequence of events. The send|receive failed? Did you try deleting the failed received snapshot?

Quoted 15 lines

which I was able to fix with the repair below, but now I'm stuck with the 'some csum missing'. Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
[2/7] checking extents
data backref 2694234112 root 356 owner 14058737 offset 0 num_refs 0 not found in extent tree incorrect local backref count on 2694234112 root 356 owner 14058737 offset 0 found 1 wanted 0 back 0x55e7383a3a00 incorrect local backref count on 2694234112 root 2147484004 owner 14058737 offset 0 found 0 wanted 1 back 0x55e733d1c2b0 backref disk bytenr does not match extent record, bytenr=2694234112, ref bytenr=0
backpointer mismatch on [2694234112 8192]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space cache
[4/7] checking fs roots
root 356 inode 33037179 errors 100, file extent discount
Found file extent holes:

Quoted 4 lines
        start: 0, len: 8192
root 203332 inode 33037179 errors 100, file extent discount
Found file extent holes:
        start: 0, len: 8192

Is no-holes enabled on either file system?

Signature · 1 line
Chris Murphy
#3 lore raw
Quoted 7 lines · Chris Murphy
Quoted 3 lines · Marc MERLIN

My data is fine, it's double backed up and the filesystem is still mountable without issues. But I had an error that broke btrfs send, and after fixing it with repair, I'm stuck with thses 'csum missing'

I'm not following the sequence of events. The send|receive failed? Did you try deleting the failed received snapshot?

btrfs send failed because there was an error on the source filesystem.
Then I found this in the logs
BTRFS error (device dm-0): did not find backref in send_root. inode=14058737, offset=0, disk_byte=2694234112 found extent=2694234112

then I ran the btrfs check repair, with and without lowmem. They fixed some things, but leave me

Quoted 1 line

Is no-holes enabled on either file system?

Not intentionally. How do I check?

Either way, things happened, not exactly sure what, but I'm just trying to find out if check --repair can/should do a better job, and if I can give anything useful before wiping the filesystem and starting over.

As of right now, I have:

aruman:~# btrfs check /dev/mapper/cr
Opening filesystem to check...
Checking filesystem on /dev/mapper/cr
UUID: 4cb82363-e833-444e-b23e-1597a14a8aab
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
there is no free space entry for 24565465088-24565481472
there is no free space entry for 24565465088-24717033472
cache appears valid but isn't 23643291648
there is no free space entry for 45118177280-45118193664
there is no free space entry for 45118177280-46191869952
cache appears valid but isn't 45118128128
wanted offset 1659663106048, found 1659663089664
wanted offset 1659663106048, found 1659663089664
cache appears valid but isn't 1659663024128
[4/7] checking fs roots
root 356 inode 14058737 errors 1000, some csum missing
root 197362 inode 14058737 errors 1000, some csum missing
root 199551 inode 14058737 errors 1000, some csum missing
root 201742 inode 14058737 errors 1000, some csum missing
root 202370 inode 14058737 errors 1000, some csum missing
root 202683 inode 14058737 errors 1000, some csum missing
root 202999 inode 14058737 errors 1000, some csum missing
root 203332 inode 14058737 errors 1000, some csum missing
root 203669 inode 14058737 errors 1000, some csum missing
root 204006 inode 14058737 errors 1000, some csum missing
root 204010 inode 14058737 errors 1000, some csum missing
root 204024 inode 14058737 errors 1000, some csum missing
root 204037 inode 14058737 errors 1000, some csum missing
root 204049 inode 14058737 errors 1000, some csum missing
root 204063 inode 14058737 errors 1000, some csum missing
root 204077 inode 14058737 errors 1000, some csum missing
root 204091 inode 14058737 errors 1000, some csum missing
root 204105 inode 14058737 errors 1000, some csum missing
root 204119 inode 14058737 errors 1000, some csum missing
root 204133 inode 14058737 errors 1000, some csum missing
root 204146 inode 14058737 errors 1000, some csum missing
root 204147 inode 14058737 errors 1000, some csum missing
root 204159 inode 14058737 errors 1000, some csum missing
root 204160 inode 14058737 errors 1000, some csum missing
root 204161 inode 14058737 errors 1000, some csum missing
root 204162 inode 14058737 errors 1000, some csum missing
root 204163 inode 14058737 errors 1000, some csum missing
root 204164 inode 14058737 errors 1000, some csum missing
root 204165 inode 14058737 errors 1000, some csum missing
root 204166 inode 14058737 errors 1000, some csum missing
root 204167 inode 14058737 errors 1000, some csum missing
root 204168 inode 14058737 errors 1000, some csum missing
ERROR: errors found in fs roots
found 112391372800 bytes used, error(s) found
total csum bytes: 105763716
total tree bytes: 4036018176
total fs tree bytes: 3666313216
total extent tree bytes: 214728704
btree space waste bytes: 875571363
file data blocks allocated: 207903133696
 referenced 20

Thanks,
Marc

Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/  
#4 lore raw
Quoted 10 lines · Marc MERLIN
Quoted 7 lines · Chris Murphy
Quoted 3 lines · Marc MERLIN

My data is fine, it's double backed up and the filesystem is still mountable without issues. But I had an error that broke btrfs send, and after fixing it with repair, I'm stuck with thses 'csum missing'

I'm not following the sequence of events. The send|receive failed? Did you try deleting the failed received snapshot?

Quoted 6 lines

btrfs send failed because there was an error on the source filesystem.
Then I found this in the logs
BTRFS error (device dm-0): did not find backref in send_root. inode=14058737, offset=0, disk_byte=2694234112 found extent=2694234112

then I ran the btrfs check repair, with and without lowmem. They fixed some things, but leave me

OK I didn't understand that the problem is with only the sending file system, not the receive file system. And also it sounds like the send did not cause the problem, but it's somehow a pre-existing problem that --repair isn't completely fixing up, or maybe is making different (or worse).

So I guess the real question is what happened to this file system before the send, that got it into this weird state.

Quoted 4 lines
Quoted 2 lines

Is no-holes enabled on either file system?

Not intentionally. How do I check?

btrfs insp dump-s

It's not yet the default and can't be inadvertently enabled so chances are it's the original holes implementation.

Signature · 1 line
Chris Murphy
#5 lore raw
Quoted 5 lines · Chris Murphy

OK I didn't understand that the problem is with only the sending file system, not the receive file system. And also it sounds like the send did not cause the problem, but it's somehow a pre-existing problem that --repair isn't completely fixing up, or maybe is making different (or worse).

Correct on all points.

Quoted 2 lines

So I guess the real question is what happened to this file system before the send, that got it into this weird state.

That too, but honestly there are a lot of variables, and it feels like a bit of wild goose chase.

Basically it looked like the issues with the FS are pretty minor (I was able to cp -av the entire data without any file error), but that btrfs check --repair is unable to make it right, which will likely force me to wipe and restore.
I know chedk is WIP, and that's why I'm providing feedback :)

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Is no-holes enabled on either file system?

Not intentionally. How do I check?

btrfs insp dump-s

It's not yet the default and can't be inadvertently enabled so chances are it's the original holes implementation.

The filesystem was definitely created a while ago (2-3 years?)

I have been recently been playing with bees, but I'm reasonably sure I didn't run in on that filesystem it lists support for "HOLE extents and btrfs no-holes feature"

saruman:~# btrfs insp dump-s /dev/mapper/cr
superblock: bytenr=65536, device=/dev/mapper/cr
---------------------------------------------------------
csum_type               0 (crc32c)
csum_size               4
csum                    0x34a9cfed [match]
bytenr                  65536
flags                   0x1
                        ( WRITTEN )
magic                   _BHRfS_M [match]
fsid                    4cb82363-e833-444e-b23e-1597a14a8aab
metadata_uuid           4cb82363-e833-444e-b23e-1597a14a8aab
label                   btrfs_boot
generation              3801933
root                    1686707896320
sys_array_size          97
chunk_root_generation   3801931
root_level              1
chunk_root              1679158165504
chunk_root_level        1
log_root                0
log_root_transid        0
log_root_level          0
total_bytes             161057079296
bytes_used              112391540736
sectorsize              4096
nodesize                16384
leafsize (deprecated)   16384
stripesize              4096
root_dir                6
num_devices             1
compat_flags            0x0
compat_ro_flags         0x0
incompat_flags          0x169
                        ( MIXED_BACKREF |
                          COMPRESS_LZO |
                          BIG_METADATA |
                          EXTENDED_IREF |
                          SKINNY_METADATA )
cache_generation        3801932
uuid_tree_generation    3801932
dev_item.uuid           924bcb26-9c4c-4106-b3ce-1bb66e66de1f
dev_item.fsid           4cb82363-e833-444e-b23e-1597a14a8aab [match]
dev_item.type           0
dev_item.total_bytes    161057079296
dev_item.bytes_used     136423931904
dev_item.io_align       4096
dev_item.io_width       4096
dev_item.sector_size    4096
dev_item.devid          1
dev_item.dev_group      0
dev_item.seek_speed     0
dev_item.bandwidth      0
dev_item.generation     0
Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/  
#6 lore raw
Quoted 8 lines · Marc MERLIN
Quoted 5 lines · Chris Murphy

OK I didn't understand that the problem is with only the sending file system, not the receive file system. And also it sounds like the send did not cause the problem, but it's somehow a pre-existing problem that --repair isn't completely fixing up, or maybe is making different (or worse).

Quoted 7 lines

Correct on all points.

Quoted 2 lines

So I guess the real question is what happened to this file system before the send, that got it into this weird state.

That too, but honestly there are a lot of variables, and it feels like a bit of wild goose chase.

Maybe. The story arc on Btrfs is that check --repair only fixes the things it knows how to fix. It's gotten better but still has the scary warning, and lately has a 10 second delay to really make sure the user meant to use it. And regardless of mode, it's slow and just can't scale. Neither does "wipe and restore from backup". So the problems of inconsistency need to be understood to avoid the problem in the first place.

Quoted 6 lines

Basically it looked like the issues with the FS are pretty minor (I was able to cp -av the entire data without any file error), but that btrfs check --repair is unable to make it right, which will likely force me to wipe and restore.
I know chedk is WIP, and that's why I'm providing feedback :)

What about finding inode 14058737 and deleting it? In all of the listed subvolumes? And then unmount and check again?

Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Is no-holes enabled on either file system?

Not intentionally. How do I check?

btrfs insp dump-s

Quoted 7 lines
Quoted 2 lines

It's not yet the default and can't be inadvertently enabled so chances are it's the original holes implementation.

The filesystem was definitely created a while ago (2-3 years?)

I have been recently been playing with bees, but I'm reasonably sure I didn't run in on that filesystem it lists support for "HOLE extents and btrfs no-holes feature"

It's default except for LZO.

Signature · 1 line
Chris Murphy
#7 lore raw
Quoted 28 lines · Chris Murphy
Quoted 8 lines · Marc MERLIN
Quoted 5 lines · Chris Murphy

OK I didn't understand that the problem is with only the sending file system, not the receive file system. And also it sounds like the send did not cause the problem, but it's somehow a pre-existing problem that --repair isn't completely fixing up, or maybe is making different (or worse).

Quoted 7 lines

Correct on all points.

Quoted 2 lines

So I guess the real question is what happened to this file system before the send, that got it into this weird state.

That too, but honestly there are a lot of variables, and it feels like a bit of wild goose chase.

Maybe. The story arc on Btrfs is that check --repair only fixes the things it knows how to fix. It's gotten better but still has the scary warning, and lately has a 10 second delay to really make sure the user meant to use it. And regardless of mode, it's slow and just can't scale. Neither does "wipe and restore from backup". So the problems of inconsistency need to be understood to avoid the problem in the first place.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

Basically it looked like the issues with the FS are pretty minor (I was able to cp -av the entire data without any file error), but that btrfs check --repair is unable to make it right, which will likely force me to wipe and restore.
I know chedk is WIP, and that's why I'm providing feedback :)

What about finding inode 14058737 and deleting it? In all of the listed subvolumes? And then unmount and check again?

Before deleting it, can you check if chattr +C is set? Or what kind of file is this? Because there's an old loophole in older kernels where chattr +C could be set, but if defragmented while mounted with a compression option, it would get compressed. And btrfs check complains about it, but it's not actually a problem. Newer kernels don't do this but I'm not certain about the send error's relationship with this.

You could try just making a 'cp --reflink=never' copy of the file. And delete the original (and all of its copies in all subvolumes). Now 'btrfs check'.

Signature · 1 line
Chris Murphy
#8 lore raw
Quoted 2 lines · Chris Murphy

Maybe. The story arc on Btrfs is that check --repair only fixes the things it knows how to fix. It's gotten better but still has the scary

that's always been true indeed :)

Quoted 5 lines

warning, and lately has a 10 second delay to really make sure the user meant to use it. And regardless of mode, it's slow and just can't scale. Neither does "wipe and restore from backup". So the problems of inconsistency need to be understood to avoid the problem in the first place.

that's ideal, but honestly it's a multi prong approach. I'm stil hoping that check --repair can improve.

Quoted 2 lines

What about finding inode 14058737 and deleting it? In all of the listed subvolumes? And then unmount and check again?

it's in all the subvolumes, including the btrfs send ones. So sure, I can delete my btrfs send snapshots and have to start over, but really btrfs check should fix this better than rm, even if it makes the file not what it was. I can always replace it with a good copy

Quoted 2 lines · Chris Murphy

Before deleting it, can you check if chattr +C is set? Or what kind of file is this? Because there's an old loophole in older kernels where

saruman:~# lsattr /mnt/mnt/root/usr/share/locale/en_GB/LC_MESSAGES/gstreamer-1.0.mo
-------------------- /mnt/mnt/root/usr/share/locale/en_GB/LC_MESSAGES/gstreamer-1.0.mo
Quoted 3 lines

You could try just making a 'cp --reflink=never' copy of the file. And delete the original (and all of its copies in all subvolumes). Now 'btrfs check'.

I can restore the file from backup without issues, it's more that with btrfs send, I'm not allowed to delete the inode until it cycles out, and I can't cycle it out because btrfs send dies when it sees the inconsistent state.

I mean, yes, I can do this and delete all the snapshots, I was just hoping to do slighty better as this is close to wiping the filesystem and starting over (actually it's mostly the same to be honest)

Marc

Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/