Skip to contentSkip to conversation

btrfs-progs: introduce a skeleton version of "btrfs rescue fix-data-csums"

PATCH Superseded Qu Wenruo 1 participant 4 messages May 11, 2025

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: introduce a skeleton version of "btrfs rescue fix-data-csums"

We have a long history of data csum mismatch, caused by direct IO and buffered being modified during writeback.

Although the problem is worked around in v6.15 (and being backported), for the affected fs there is no good way to fix them, other than complex manually find out which files are affected and delete them.

This series introduce the initial implementation of "btrfs rescue fix-data-csums", which is designed to fix such problem by either:

- Delete the affected files
- Update the csum items using the data from specified copy

But for the first step, just detect such csum mismatch (pretty much the same as "btrfs check --check-data-csum"), and print out the affected files.

No real repair is done yet, only dry run is supported (the default and only mode yet).

Qu Wenruo (3):
btrfs-progs: introduce "btrfs rescue fix-data-csums"
btrfs-progs: fix a bug in btrfs_find_item()
btrfs-progs: fix-data-csums: show affected files

 Documentation/btrfs-rescue.rst |  19 ++
 Makefile                       |   2 +-
 cmds/rescue-fix-data-csums.c   | 379 +++++++++++++++++++++++++++++++++
 cmds/rescue.c                  |  44 ++++
 cmds/rescue.h                  |   6 +
 kernel-shared/ctree.c          |  17 +-
 6 files changed, 463 insertions(+), 4 deletions(-)
 create mode 100644 cmds/rescue-fix-data-csums.c

--
2.49.0

author
#1 lore raw

PATCH 1/3 btrfs-progs: introduce "btrfs rescue fix-data-csums"

It's a long known problem that direct IO can lead to data checksum mismatches if the user space is also modifying its buffer during writeback.

Although it's fixed in the recent v6.15 release (and backported to older kernels), we still need a user friendly way to fix those problems.

This patch introduce the dryrun version of "btrfs rescue fix-data-csums", which reports the logical bytenr and corrupted mirrors.

It is planned to implement the following repair methods:

- Update the data checksum item
Using specified mirror or the most agreed checksum using on-disk data, to update the checksum item in checksum tree.

- Deleting all involved files

---
 Documentation/btrfs-rescue.rst |  19 ++
 Makefile                       |   2 +-
 cmds/rescue-fix-data-csums.c   | 326 +++++++++++++++++++++++++++++++++
 cmds/rescue.c                  |  44 +++++
 cmds/rescue.h                  |   6 +
 5 files changed, 396 insertions(+), 1 deletion(-)
 create mode 100644 cmds/rescue-fix-data-csums.c
Documentation/btrfs-rescue.rst +19 -0
index f52e6c26359a..fe2596e5d558 100644
@@ -50,6 +50,25 @@ fix-device-size <device>
 
                 WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]
 
+fix-data-csums [-d] <device>
+	fix data checksum mismatch
+
+	There is a long existing problem that if a user space program is doing
+	direct IO and modifies the buffer before the write back finished, it
+	can lead to data checksum mismatches.
+
+	This problem is known but not fixed until upstream release v6.15
+	(backported to older kernels). So it's possible to hit false data
+	checksum mismatch.
+
+	In that case this program can be utilized to repair such problem.
+
+	``Options``
+
+	-d
+		dryrun, will report errors but not repair, the default behavior.
+
+
 .. _man-rescue-clear-ino-cache:
 
 clear-ino-cache <device>
Makefile +1 -1
index 7e36aa425736..75fa2249b078 100644
@@ -256,7 +256,7 @@ cmds_objects = cmds/subvolume.o cmds/subvolume-list.o \
 	       cmds/inspect.o cmds/balance.o cmds/send.o cmds/receive.o \
 	       cmds/quota.o cmds/qgroup.o cmds/replace.o check/main.o \
 	       cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o \
-	       cmds/rescue-super-recover.o \
+	       cmds/rescue-super-recover.o cmds/rescue-fix-data-csums.o \
 	       cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o \
 	       cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o \
 	       cmds/reflink.o \
cmds/rescue-fix-data-csums.c new file +326 -0

326 lines not on this page · raw message

cmds/rescue.c +44 -0

62 lines not on this page · raw message

cmds/rescue.h +6 -0
index 5a9e46b7aae0..246f15c38564 100644
@@ -20,7 +20,13 @@
 #ifndef __BTRFS_RESCUE_H__
 #define __BTRFS_RESCUE_H__
 
+enum btrfs_fix_data_csums_mode {
+	BTRFS_FIX_DATA_CSUMS_DRY_RUN,
+	BTRFS_FIX_DATA_CSUMS_LAST,
+};
+
 int btrfs_recover_superblocks(const char *path, int yes);
 int btrfs_recover_chunk_tree(const char *path, int yes);
+int btrfs_recover_fix_data_csums(const char *path, enum btrfs_fix_data_csums_mode mode);
 
 #endif
Signature · 1 line
2.49.0
author
#2 lore raw

PATCH 2/3 btrfs-progs: fix a bug in btrfs_find_item()

In reply to Qu Wenruo

[BUG]
There is a seldomly utilized function, btrfs_find_item(), which has no document and is not behaving correctly.

Inside backref.c, iterate_inode_refs() and btrfs_ref_to_path() both utilize this function, to find the parent inode.

However btrfs_find_item() will never return 0 if @ioff is passed as 0 for such usage, result early failure for all kinds of inode iteration functions.

[CAUSE]
Both functions pass 0 as the @ioff parameter initially, e.g:

We have the following fs tree root:

item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160

		generation 3 transid 9 size 6 nbytes 16384
		block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0
		sequence 1 flags 0x0(none)
	item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12
		index 0 namelen 2 name: ..
	item 2 key (256 DIR_ITEM 2507850652) itemoff 16078 itemsize 33
		location key (257 INODE_ITEM 0) type FILE
		transid 9 data_len 0 name_len 3
		name: foo
	item 3 key (256 DIR_INDEX 2) itemoff 16045 itemsize 33
		location key (257 INODE_ITEM 0) type FILE
		transid 9 data_len 0 name_len 3
		name: foo
	item 4 key (257 INODE_ITEM 0) itemoff 15885 itemsize 160
		generation 9 transid 9 size 16384 nbytes 16384
		block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
		sequence 4 flags 0x0(none)
	item 5 key (257 INODE_REF 256) itemoff 15872 itemsize 13
		index 2 namelen 3 name: foo
	item 6 key (257 EXTENT_DATA 0) itemoff 15819 itemsize 53
		generation 9 type 1 (regular)
		extent data disk byte 13631488 nr 16384
		extent data offset 0 nr 16384 ram 16384
		extent compression 0 (none)

Then we call paths_from_inode() with:
- @inum = 257
- ipath = {.fs_root = 5}

Then we got the following sequence:

iterate_irefs(257, fs_root, inode_to_path)
|- iterate_inode_refs()

     |- inode_ref_info()
        |- btrfs_find_item(257, 0, fs_root)
	|  Returned 1, with @found_key updated to
	|  (257, INODE_REF, 256).

This makes iterate_irefs() exit immediately, but obviously that btrfs_find_item() call is to find any INODE_REF, not to find the exact match.

[FIX]
If btrfs_find_item() found an item matching the objectid and type, then it should return 0 other than 1.

Fix it and keep the behavior the same across btrfs-progs and the kernel.

Since we're here, also add some comments explaining the function.

---
 kernel-shared/ctree.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
kernel-shared/ctree.c +14 -3
index 3184c916175e..f90de606e7b1 100644
@@ -1246,6 +1246,17 @@ static void reada_for_search(struct btrfs_fs_info *fs_info,
 	}
 }
 
+/*
+ * Find the first key in @fs_root that matches all the following conditions:
+ *
+ * - key.obojectid == @iobjectid
+ * - key.type == @key_type
+ * - key.offset >= ioff
+ *
+ * Return 0 if such key can be found, and @found_key is updated.
+ * Return >0 if no such key can be found.
+ * Return <0 for critical errors.
+ */
 int btrfs_find_item(struct btrfs_root *fs_root, struct btrfs_path *found_path,
 		u64 iobjectid, u64 ioff, u8 key_type,
 		struct btrfs_key *found_key)
@@ -1280,10 +1291,10 @@ int btrfs_find_item(struct btrfs_root *fs_root, struct btrfs_path *found_path,
 
 	btrfs_item_key_to_cpu(eb, found_key, path->slots[0]);
 	if (found_key->type != key.type ||
-			found_key->objectid != key.objectid) {
+	    found_key->objectid != key.objectid)
 		ret = 1;
-		goto out;
-	}
+	else
+		ret = 0;
 
 out:
 	if (path != found_path)
Signature · 1 line
2.49.0
author
#3 lore raw

PATCH 3/3 btrfs-progs: fix-data-csums: show affected files

In reply to Qu Wenruo

Previously "btrfs rescue fix-data-csums" only show affected logical bytenr, which is not helpful to determine which files are affected.

Enhance the output by also outputting the affected subvolumes (in numeric form), and the file paths inside that subvolume.

The example looks like this:

logical=13631488 corrtuped mirrors=1 affected files:
(subvolume 5)/foo
(subvolume 5)/dir/bar
logical=13635584 corrtuped mirrors=1 affected files:
(subvolume 5)/foo
(subvolume 5)/dir/bar

Although the end result is still not perfect, it's still much easier to find out which files are affected.

---
 cmds/rescue-fix-data-csums.c | 59 ++++++++++++++++++++++++++++++++++--
 1 file changed, 56 insertions(+), 3 deletions(-)
cmds/rescue-fix-data-csums.c +56 -3
index 66e34b279f13..bcba2741f2df 100644
@@ -18,6 +18,7 @@
 #include "kernel-shared/disk-io.h"
 #include "kernel-shared/ctree.h"
 #include "kernel-shared/volumes.h"
+#include "kernel-shared/backref.h"
 #include "common/messages.h"
 #include "common/open-utils.h"
 #include "cmds/rescue.h"
@@ -201,6 +202,49 @@ static int iterate_one_csum_item(struct btrfs_fs_info *fs_info, struct btrfs_pat
 	return ret;
 }
 
+static int print_filenames(u64 ino, u64 offset, u64 rootid, void *ctx)
+{
+	struct btrfs_fs_info *fs_info = ctx;
+	struct btrfs_root *root;
+	struct btrfs_key key;
+	struct inode_fs_paths *ipath;
+	struct btrfs_path path = { 0 };
+	int ret;
+
+	key.objectid = rootid;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = (u64)-1;
+
+	root = btrfs_read_fs_root(fs_info, &key);
+	if (IS_ERR(root)) {
+		ret = PTR_ERR(root);
+		errno = -ret;
+		error("failed to get subvolume %llu: %m", rootid);
+		return ret;
+	}
+	ipath = init_ipath(128 * BTRFS_PATH_NAME_MAX, root, &path);
+	if (IS_ERR(ipath)) {
+		ret = PTR_ERR(ipath);
+		errno = -ret;
+		error("failed to initialize ipath: %m");
+		return ret;
+	}
+	ret = paths_from_inode(ino, ipath);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to resolve root %llu ino %llu to paths: %m", rootid, ino);
+		goto out;
+	}
+	for (int i = 0; i < ipath->fspath->elem_cnt; i++)
+		printf("  (subvolume %llu)/%s\n", rootid, (char *)ipath->fspath->val[i]);
+	if (ipath->fspath->elem_missed)
+		printf("  (subvolume %llu) %d files not printed\n", rootid,
+		       ipath->fspath->elem_missed);
+out:
+	free_ipath(ipath);
+	return ret;
+}
+
 static int iterate_csum_root(struct btrfs_fs_info *fs_info, struct btrfs_root *csum_root)
 {
 	struct btrfs_path path = { 0 };
@@ -240,9 +284,10 @@ next:
 	return ret;
 }
 
-static void report_corrupted_blocks(void)
+static void report_corrupted_blocks(struct btrfs_fs_info *fs_info)
 {
 	struct corrupted_block *entry;
+	struct btrfs_path path = { 0 };
 
 	if (list_empty(&corrupted_blocks)) {
 		printf("No data checksum mismatch found\n");
@@ -251,6 +296,7 @@ static void report_corrupted_blocks(void)
 
 	list_for_each_entry(entry, &corrupted_blocks, list) {
 		bool has_printed = false;
+		int ret;
 
 		printf("logical=%llu corrtuped mirrors=", entry->logical);
 		/* Poor man's bitmap print. */
@@ -262,7 +308,14 @@ static void report_corrupted_blocks(void)
 				has_printed=true;
 			}
 		}
-		printf("\n");
+		printf(" affected files:\n");
+		ret = iterate_inodes_from_logical(entry->logical, fs_info, &path,
+						  print_filenames, fs_info);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to iterate involved files: %m");
+			break;
+		}
 	}
 }
 
@@ -318,7 +371,7 @@ int btrfs_recover_fix_data_csums(const char *path, enum btrfs_fix_data_csums_mod
 		errno = -ret;
 		error("failed to iterate csum tree: %m");
 	}
-	report_corrupted_blocks();
+	report_corrupted_blocks(fs_info);
 out_close:
 	free_corrupted_blocks();
 	close_ctree_fs_info(fs_info);
Signature · 1 line
2.49.0