btrfs-progs: introduce a skeleton version of "btrfs rescue fix-data-csums"
Subsystems (3)
cover btrfs-progs: introduce a skeleton version of "btrfs rescue fix-data-csums"
We have a long history of data csum mismatch, caused by direct IO and buffered being modified during writeback.
Although the problem is worked around in v6.15 (and being backported), for the affected fs there is no good way to fix them, other than complex manually find out which files are affected and delete them.
This series introduce the initial implementation of "btrfs rescue fix-data-csums", which is designed to fix such problem by either:
- Delete the affected files
- Update the csum items using the data from specified copy
But for the first step, just detect such csum mismatch (pretty much the same as "btrfs check --check-data-csum"), and print out the affected files.
No real repair is done yet, only dry run is supported (the default and only mode yet).
Qu Wenruo (3):
btrfs-progs: introduce "btrfs rescue fix-data-csums"
btrfs-progs: fix a bug in btrfs_find_item()
btrfs-progs: fix-data-csums: show affected files
Documentation/btrfs-rescue.rst | 19 ++ Makefile | 2 +- cmds/rescue-fix-data-csums.c | 379 +++++++++++++++++++++++++++++++++ cmds/rescue.c | 44 ++++ cmds/rescue.h | 6 + kernel-shared/ctree.c | 17 +- 6 files changed, 463 insertions(+), 4 deletions(-) create mode 100644 cmds/rescue-fix-data-csums.c
--
2.49.0
PATCH 1/3 btrfs-progs: introduce "btrfs rescue fix-data-csums"
It's a long known problem that direct IO can lead to data checksum mismatches if the user space is also modifying its buffer during writeback.
Although it's fixed in the recent v6.15 release (and backported to older kernels), we still need a user friendly way to fix those problems.
This patch introduce the dryrun version of "btrfs rescue fix-data-csums", which reports the logical bytenr and corrupted mirrors.
It is planned to implement the following repair methods:
- Update the data checksum item
Using specified mirror or the most agreed checksum using on-disk data, to update the checksum item in checksum tree.
- Deleting all involved files
--- Documentation/btrfs-rescue.rst | 19 ++ Makefile | 2 +- cmds/rescue-fix-data-csums.c | 326 +++++++++++++++++++++++++++++++++ cmds/rescue.c | 44 +++++ cmds/rescue.h | 6 + 5 files changed, 396 insertions(+), 1 deletion(-) create mode 100644 cmds/rescue-fix-data-csums.c
Documentation/btrfs-rescue.rst +19 -0
@@ -50,6 +50,25 @@ fix-device-size <device> WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs] +fix-data-csums [-d] <device> + fix data checksum mismatch + + There is a long existing problem that if a user space program is doing + direct IO and modifies the buffer before the write back finished, it + can lead to data checksum mismatches. + + This problem is known but not fixed until upstream release v6.15 + (backported to older kernels). So it's possible to hit false data + checksum mismatch. + + In that case this program can be utilized to repair such problem. + + ``Options`` + + -d + dryrun, will report errors but not repair, the default behavior. + + .. _man-rescue-clear-ino-cache: clear-ino-cache <device>
Makefile +1 -1
@@ -256,7 +256,7 @@ cmds_objects = cmds/subvolume.o cmds/subvolume-list.o \ cmds/inspect.o cmds/balance.o cmds/send.o cmds/receive.o \ cmds/quota.o cmds/qgroup.o cmds/replace.o check/main.o \ cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o \ - cmds/rescue-super-recover.o \ + cmds/rescue-super-recover.o cmds/rescue-fix-data-csums.o \ cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o \ cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o \ cmds/reflink.o \
cmds/rescue-fix-data-csums.c new file +326 -0
326 lines not on this page · raw message
cmds/rescue.c +44 -0
62 lines not on this page · raw message
cmds/rescue.h +6 -0
@@ -20,7 +20,13 @@#ifndef __BTRFS_RESCUE_H__#define __BTRFS_RESCUE_H__ +enum btrfs_fix_data_csums_mode { +BTRFS_FIX_DATA_CSUMS_DRY_RUN , +BTRFS_FIX_DATA_CSUMS_LAST , +}; +int btrfs_recover_superblocks (const char *path,int yes);int btrfs_recover_chunk_tree (const char *path,int yes); +int btrfs_recover_fix_data_csums (const char *path,enum btrfs_fix_data_csums_mode mode);#endif
Signature · 1 line
2.49.0
PATCH 2/3 btrfs-progs: fix a bug in btrfs_find_item()
In reply to Qu Wenruo[BUG]
There is a seldomly utilized function, btrfs_find_item(), which has no document and is not behaving correctly.
Inside backref.c, iterate_inode_refs() and btrfs_ref_to_path() both utilize this function, to find the parent inode.
However btrfs_find_item() will never return 0 if @ioff is passed as 0 for such usage, result early failure for all kinds of inode iteration functions.
[CAUSE]
Both functions pass 0 as the @ioff parameter initially, e.g:
We have the following fs tree root:
item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
generation 3 transid 9 size 6 nbytes 16384 block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0 sequence 1 flags 0x0(none) item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12 index 0 namelen 2 name: .. item 2 key (256 DIR_ITEM 2507850652) itemoff 16078 itemsize 33 location key (257 INODE_ITEM 0) type FILE transid 9 data_len 0 name_len 3 name: foo item 3 key (256 DIR_INDEX 2) itemoff 16045 itemsize 33 location key (257 INODE_ITEM 0) type FILE transid 9 data_len 0 name_len 3 name: foo item 4 key (257 INODE_ITEM 0) itemoff 15885 itemsize 160 generation 9 transid 9 size 16384 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 4 flags 0x0(none) item 5 key (257 INODE_REF 256) itemoff 15872 itemsize 13 index 2 namelen 3 name: foo item 6 key (257 EXTENT_DATA 0) itemoff 15819 itemsize 53 generation 9 type 1 (regular) extent data disk byte 13631488 nr 16384 extent data offset 0 nr 16384 ram 16384 extent compression 0 (none)
Then we call paths_from_inode() with:
- @inum = 257
- ipath = {.fs_root = 5}
Then we got the following sequence:
iterate_irefs(257, fs_root, inode_to_path)
|- iterate_inode_refs()
|- inode_ref_info()
|- btrfs_find_item(257, 0, fs_root)
| Returned 1, with @found_key updated to
| (257, INODE_REF, 256). This makes iterate_irefs() exit immediately, but obviously that btrfs_find_item() call is to find any INODE_REF, not to find the exact match.
[FIX]
If btrfs_find_item() found an item matching the objectid and type, then it should return 0 other than 1.
Fix it and keep the behavior the same across btrfs-progs and the kernel.
Since we're here, also add some comments explaining the function.
--- kernel-shared/ctree.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
kernel-shared/ctree.c +14 -3
@@ -1246,6 +1246,17 @@ static void reada_for_search(struct btrfs_fs_info *fs_info, } } +/* +* Find the first key in @fs_root that matches all the following conditions: +* +* - key.obojectid == @iobjectid +* - key.type == @key_type +* - key.offset >= ioff +* +* Return 0 if such key can be found, and @found_key is updated. +* Return >0 if no such key can be found. +* Return <0 for critical errors. +*/ int btrfs_find_item (struct btrfs_root *fs_root,struct btrfs_path *found_path,u64 iobjectid,u64 ioff,u8 key_type,struct btrfs_key *found_key)
@@ -1280,10 +1291,10 @@ int btrfs_find_item(struct btrfs_root *fs_root, struct btrfs_path *found_path,btrfs_item_key_to_cpu (eb, found_key, path->slots[0 ]);if (found_key->type != key.type || - found_key->objectid != key.objectid) { + found_key->objectid != key.objectid) ret =1 ; - goto out; - } +else + ret =0 ; out:if (path != found_path)
Signature · 1 line
2.49.0
PATCH 3/3 btrfs-progs: fix-data-csums: show affected files
In reply to Qu WenruoPreviously "btrfs rescue fix-data-csums" only show affected logical bytenr, which is not helpful to determine which files are affected.
Enhance the output by also outputting the affected subvolumes (in numeric form), and the file paths inside that subvolume.
The example looks like this:
logical=13631488 corrtuped mirrors=1 affected files:
(subvolume 5)/foo
(subvolume 5)/dir/bar
logical=13635584 corrtuped mirrors=1 affected files:
(subvolume 5)/foo
(subvolume 5)/dir/bar
Although the end result is still not perfect, it's still much easier to find out which files are affected.
--- cmds/rescue-fix-data-csums.c | 59 ++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 3 deletions(-)
cmds/rescue-fix-data-csums.c +56 -3
@@ -18,6 +18,7 @@#include "kernel-shared/disk-io.h" #include "kernel-shared/ctree.h" #include "kernel-shared/volumes.h" +#include "kernel-shared/backref.h" #include "common/messages.h" #include "common/open-utils.h" #include "cmds/rescue.h"
@@ -201,6 +202,49 @@ static int iterate_one_csum_item(struct btrfs_fs_info *fs_info, struct btrfs_patreturn ret; } +static int print_filenames (u64 ino,u64 offset,u64 rootid,void *ctx) +{ +struct btrfs_fs_info *fs_info = ctx; +struct btrfs_root *root; +struct btrfs_key key; +struct inode_fs_paths *ipath; +struct btrfs_path path = {0 }; +int ret; + + key.objectid = rootid; + key.type =BTRFS_ROOT_ITEM_KEY ; + key.offset = (u64)-1 ; + + root =btrfs_read_fs_root (fs_info, &key); +if (IS_ERR (root)) { + ret =PTR_ERR (root); + errno = -ret; +error ("failed to get subvolume %llu: %m" , rootid); +return ret; + } + ipath =init_ipath (128 *BTRFS_PATH_NAME_MAX , root, &path); +if (IS_ERR (ipath)) { + ret =PTR_ERR (ipath); + errno = -ret; +error ("failed to initialize ipath: %m" ); +return ret; + } + ret =paths_from_inode (ino, ipath); +if (ret <0 ) { + errno = -ret; +error ("failed to resolve root %llu ino %llu to paths: %m" , rootid, ino); + goto out; + } +for (int i =0 ; i < ipath->fspath->elem_cnt; i++) +printf (" (subvolume %llu)/%s\n" , rootid, (char *)ipath->fspath->val[i]); +if (ipath->fspath->elem_missed) +printf (" (subvolume %llu) %d files not printed\n" , rootid, + ipath->fspath->elem_missed); +out: +free_ipath (ipath); +return ret; +} +static int iterate_csum_root (struct btrfs_fs_info *fs_info,struct btrfs_root *csum_root) {struct btrfs_path path = {0 };
@@ -240,9 +284,10 @@ next:return ret; } -static void report_corrupted_blocks (void ) +static void report_corrupted_blocks (struct btrfs_fs_info *fs_info) {struct corrupted_block *entry; +struct btrfs_path path = {0 };if (list_empty (&corrupted_blocks)) {printf ("No data checksum mismatch found\n" );
@@ -251,6 +296,7 @@ static void report_corrupted_blocks(void)list_for_each_entry (entry, &corrupted_blocks, list) {bool has_printed = false; +int ret;printf ("logical=%llu corrtuped mirrors=" , entry->logical);/* Poor man's bitmap print. */
@@ -262,7 +308,14 @@ static void report_corrupted_blocks(void) has_printed=true; } } -printf ("\n" ); +printf (" affected files:\n" ); + ret =iterate_inodes_from_logical (entry->logical, fs_info, &path, + print_filenames, fs_info); +if (ret <0 ) { + errno = -ret; +error ("failed to iterate involved files: %m" ); +break ; + } } }
@@ -318,7 +371,7 @@ int btrfs_recover_fix_data_csums(const char *path, enum btrfs_fix_data_csums_mod errno = -ret;error ("failed to iterate csum tree: %m" ); } -report_corrupted_blocks (); +report_corrupted_blocks (fs_info); out_close:free_corrupted_blocks ();close_ctree_fs_info (fs_info);
Signature · 1 line
2.49.0