Skip to contentSkip to conversation

btrfs: check: Check and repair invalid free space cahce inode mode

PATCH Superseded Qu Wenruo 5 participants 21 messages Mar 25, 2019 – Apr 1, 2019 Btrfs file system

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs: check: Check and repair invalid free space cahce inode mode

This patchset can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/free_space_cache_imode_fix
Which is based on v4.20.2 tag.

There is a minor problem reported in mail list where invalid inode mode
(0) can sneak into free space cache inodes:

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

Kernel free space cache code doesn't realy care about the inode mode, but tree-checker inode item checker cares and will report this as big problem, rejecting the image.

I didn't see any reason to make inode item checker any less strict, as there is already fuzzed image can break kernel using invalid inode mode.

So here I just enhance btrfs-progs before strict inode item checker lands.

Next time when users report kernel rejecting previously valid fs, they should have a way to fix it in btrfs-progs.

Qu Wenruo (7):
btrfs-progs: check/lowmem: Add inode mode check
btrfs-progs: check/original: Add inode mode check
btrfs-progs: check/lowmem: Repair invalid inode mode in root tree
btrfs-progs: check/original: Repair invalid inode mode in root tree
btrfs: check/lowmem: Check and repair free space cache inode mode
btrfs: check/original: Check and repair free space cache inode item
btrfs: tests/fsck: Add test image for free space cache mode repair

 check/main.c                                  |  36 +++++++
 check/mode-common.c                           |  48 +++++++++
 check/mode-common.h                           |  27 +++++
 check/mode-lowmem.c                           |  98 ++++++++++++++++++
 check/mode-lowmem.h                           |   3 +
 check/mode-original.h                         |   1 +
 .../test.raw.xz                               | Bin 0 -> 159128 bytes
 7 files changed, 213 insertions(+)
 create mode 100644 tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH 1/7 btrfs-progs: check/lowmem: Add inode mode check

There is one report about invalid free space cache inode mode. Normally free space cache inode should have mode 100600 (regular file, no uid/gid/sticky bit, rw------ bit).

But in that report, we have free space cache inode mode as 0.

So at least btrfs check should report invalid inode mode.

This patch will at least make btrfs check lowmem mode to detect this problem.

Please note that, this check only applies to inodes in fs/subvol trees. It doesn't apply to free space cache inodes.

---
 check/mode-common.h | 24 ++++++++++++++++++++++++
 check/mode-lowmem.c |  5 +++++
 check/mode-lowmem.h |  1 +
 3 files changed, 30 insertions(+)
check/mode-common.h +24 -0
index 1fb3cda7d89d..4c88365abbcc 100644
@@ -126,4 +126,28 @@ int delete_corrupted_dir_item(struct btrfs_trans_handle *trans,
 			      struct btrfs_key *di_key, char *namebuf,
 			      u32 namelen);
 
+/*
+ * Check if the inode mode @imode is valid
+ *
+ * This check focuses on S_FTMT bits and unused bits.
+ * Sticky/setuid/setgid and regular owner/group/other bits won't cause
+ * any problem.
+ */
+static inline bool is_valid_imode(u32 imode)
+{
+	if (imode & ~(S_IFMT | 07777))
+		return false;
+
+	/*
+	 * S_IFMT is not bitmap, nor pure numbering sequence. Need per valid
+	 * number check.
+	 */
+	imode &= S_IFMT;
+	if (imode != S_IFDIR && imode != S_IFCHR && imode != S_IFBLK &&
+	    imode != S_IFREG && imode != S_IFIFO && imode != S_IFLNK &&
+	    imode != S_IFSOCK)
+		return false;
+	return true;
+}
+
 #endif
check/mode-lowmem.c +5 -0
index fc6228a05a1b..1553a4a5d2c1 100644
@@ -2452,6 +2452,11 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)
 	nlink = btrfs_inode_nlink(node, ii);
 	nodatasum = btrfs_inode_flags(node, ii) & BTRFS_INODE_NODATASUM;
 
+	if (!is_valid_imode(mode)) {
+		error("invalid imode mode bits: 0%o", mode);
+		err |= INODE_MODE_ERROR;
+	}
+
 	if (S_ISLNK(mode) &&
 	    flags & (BTRFS_INODE_IMMUTABLE | BTRFS_INODE_APPEND)) {
 		err |= INODE_FLAGS_ERROR;
check/mode-lowmem.h +1 -0
index 46b9b191ce24..e0ab30b770d5 100644
@@ -46,6 +46,7 @@
 #define FATAL_ERROR             (1<<22) /* Fatal bit for errno */
 #define INODE_FLAGS_ERROR	(1<<23) /* Invalid inode flags */
 #define DIR_ITEM_HASH_MISMATCH	(1<<24) /* Dir item hash mismatch */
+#define INODE_MODE_ERROR	(1<<25) /* Bad inode mode */
 
 /*
  * Error bit for low memory mode check.
Signature · 1 line
2.21.0
Reviewed-by on v1 1/7
#2 lore raw

Re: [PATCH 1/7] btrfs-progs: check/lowmem: Add inode mode check

Quoted 8 lines · Qu Wenruo

There is one report about invalid free space cache inode mode. Normally free space cache inode should have mode 100600 (regular file, no uid/gid/sticky bit, rw------ bit).

But in that report, we have free space cache inode mode as 0.

So at least btrfs check should report invalid inode mode.

Quoted 8 lines

This patch will at least make btrfs check lowmem mode to detect this problem.

Please note that, this check only applies to inodes in fs/subvol trees. It doesn't apply to free space cache inodes.

Quoted 68 lines
---
 check/mode-common.h | 24 ++++++++++++++++++++++++
 check/mode-lowmem.c |  5 +++++
 check/mode-lowmem.h |  1 +
 3 files changed, 30 insertions(+)
check/mode-common.h +24 -0
index 1fb3cda7d89d..4c88365abbcc 100644
@@ -126,4 +126,28 @@ int delete_corrupted_dir_item(struct btrfs_trans_handle *trans,
 			      struct btrfs_key *di_key, char *namebuf,
 			      u32 namelen);
 
+/*
+ * Check if the inode mode @imode is valid
+ *
+ * This check focuses on S_FTMT bits and unused bits.
+ * Sticky/setuid/setgid and regular owner/group/other bits won't cause
+ * any problem.
+ */
+static inline bool is_valid_imode(u32 imode)
+{
+	if (imode & ~(S_IFMT | 07777))
+		return false;
+
+	/*
+	 * S_IFMT is not bitmap, nor pure numbering sequence. Need per valid
+	 * number check.
+	 */
+	imode &= S_IFMT;
+	if (imode != S_IFDIR && imode != S_IFCHR && imode != S_IFBLK &&
+	    imode != S_IFREG && imode != S_IFIFO && imode != S_IFLNK &&
+	    imode != S_IFSOCK)
+		return false;
+	return true;
+}
+
 #endif
check/mode-lowmem.c +5 -0
index fc6228a05a1b..1553a4a5d2c1 100644
@@ -2452,6 +2452,11 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)
 	nlink = btrfs_inode_nlink(node, ii);
 	nodatasum = btrfs_inode_flags(node, ii) & BTRFS_INODE_NODATASUM;
 
+	if (!is_valid_imode(mode)) {
+		error("invalid imode mode bits: 0%o", mode);
+		err |= INODE_MODE_ERROR;
+	}
+
 	if (S_ISLNK(mode) &&
 	    flags & (BTRFS_INODE_IMMUTABLE | BTRFS_INODE_APPEND)) {
 		err |= INODE_FLAGS_ERROR;
check/mode-lowmem.h +1 -0
index 46b9b191ce24..e0ab30b770d5 100644
@@ -46,6 +46,7 @@
 #define FATAL_ERROR             (1<<22) /* Fatal bit for errno */
 #define INODE_FLAGS_ERROR	(1<<23) /* Invalid inode flags */
 #define DIR_ITEM_HASH_MISMATCH	(1<<24) /* Dir item hash mismatch */
+#define INODE_MODE_ERROR	(1<<25) /* Bad inode mode */
 
 /*
  * Error bit for low memory mode check.
author
#3 lore raw

PATCH 2/7 btrfs-progs: check/original: Add inode mode check

In reply to Qu Wenruo

Just like lowmem mode, check inode mode, specially for S_IFMT bits and beyond.

Please note that, this check only applies to inodes in fs/subvol trees. It doesn't apply to free space cache inodes.

---
 check/main.c          | 5 +++++
 check/mode-original.h | 1 +
 2 files changed, 6 insertions(+)
check/main.c +5 -0
index 7547209c5604..553c93caa2c9 100644
@@ -616,6 +616,9 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec)
 		fprintf(stderr, ", odd inode flags");
 	if (errors & I_ERR_INLINE_RAM_BYTES_WRONG)
 		fprintf(stderr, ", invalid inline ram bytes");
+	if (errors & I_ERR_INVALID_IMODE)
+		fprintf(stderr, ", invalid inode mode bit 0%o",
+			rec->imode & ~07777);
 	fprintf(stderr, "\n");
 	/* Print the orphan extents if needed */
 	if (errors & I_ERR_FILE_EXTENT_ORPHAN)
@@ -811,6 +814,8 @@ static void maybe_free_inode_rec(struct cache_tree *inode_cache,
 	if (!rec->checked || rec->merging)
 		return;
 
+	if (!is_valid_imode(rec->imode))
+		rec->errors |= I_ERR_INVALID_IMODE;
 	if (S_ISDIR(rec->imode)) {
 		if (rec->found_size != rec->isize)
 			rec->errors |= I_ERR_DIR_ISIZE_WRONG;
check/mode-original.h +1 -0
index 25ca274118a7..e40a12930a6f 100644
@@ -189,6 +189,7 @@ struct file_extent_hole {
 #define I_ERR_ODD_INODE_FLAGS		(1 << 16)
 #define I_ERR_INLINE_RAM_BYTES_WRONG	(1 << 17)
 #define I_ERR_MISMATCH_DIR_HASH		(1 << 18)
+#define I_ERR_INVALID_IMODE		(1 << 19)
 
 struct inode_record {
 	struct list_head backrefs;
Signature · 1 line
2.21.0
#4 lore raw

Re: [PATCH 2/7] btrfs-progs: check/original: Add inode mode check

Quoted 28 lines · Qu Wenruo

Just like lowmem mode, check inode mode, specially for S_IFMT bits and beyond.

Please note that, this check only applies to inodes in fs/subvol trees. It doesn't apply to free space cache inodes.

---
 check/main.c          | 5 +++++
 check/mode-original.h | 1 +
 2 files changed, 6 insertions(+)
check/main.c +3 -0
index 7547209c5604..553c93caa2c9 100644
@@ -616,6 +616,9 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec)
 		fprintf(stderr, ", odd inode flags");
 	if (errors & I_ERR_INLINE_RAM_BYTES_WRONG)
 		fprintf(stderr, ", invalid inline ram bytes");
+	if (errors & I_ERR_INVALID_IMODE)
+		fprintf(stderr, ", invalid inode mode bit 0%o",
+			rec->imode & ~07777);
 	fprintf(stderr, "\n");
 	/* Print the orphan extents if needed */
 	if (errors & I_ERR_FILE_EXTENT_ORPHAN)

check/main.c static void maybe_free_inode_rec(struct cache_tree *inode_cache, · line 818

Quoted 5 lines
check/main.c +2 -0
@@ -811,6 +814,8 @@ static void maybe_free_inode_rec(struct cache_tree *inode_cache,
 	if (!rec->checked || rec->merging)
 		return;
 
+	if (!is_valid_imode(rec->imode))
+		rec->errors |= I_ERR_INVALID_IMODE;

should this check actually be moved before the call to imode_to_type. Because if the mode is busted this means we could potentially get REF_ERR_FILETYPE_UNMATCH error? I.e this check should really fail before anything else ?

Quoted 16 lines
check/main.c +0 -0
 	if (S_ISDIR(rec->imode)) {
 		if (rec->found_size != rec->isize)
 			rec->errors |= I_ERR_DIR_ISIZE_WRONG;
check/mode-original.h +1 -0
index 25ca274118a7..e40a12930a6f 100644
@@ -189,6 +189,7 @@ struct file_extent_hole {
 #define I_ERR_ODD_INODE_FLAGS		(1 << 16)
 #define I_ERR_INLINE_RAM_BYTES_WRONG	(1 << 17)
 #define I_ERR_MISMATCH_DIR_HASH		(1 << 18)
+#define I_ERR_INVALID_IMODE		(1 << 19)
 
 struct inode_record {
 	struct list_head backrefs;
#5 lore raw

Re: [PATCH 2/7] btrfs-progs: check/original: Add inode mode check

Quoted 34 lines · Nikolay Borisov
Quoted 31 lines · Qu Wenruo

Just like lowmem mode, check inode mode, specially for S_IFMT bits and beyond.

Please note that, this check only applies to inodes in fs/subvol trees. It doesn't apply to free space cache inodes.

---
 check/main.c          | 5 +++++
 check/mode-original.h | 1 +
 2 files changed, 6 insertions(+)
check/main.c +3 -0
index 7547209c5604..553c93caa2c9 100644
@@ -616,6 +616,9 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec)
 		fprintf(stderr, ", odd inode flags");
 	if (errors & I_ERR_INLINE_RAM_BYTES_WRONG)
 		fprintf(stderr, ", invalid inline ram bytes");
+	if (errors & I_ERR_INVALID_IMODE)
+		fprintf(stderr, ", invalid inode mode bit 0%o",
+			rec->imode & ~07777);
 	fprintf(stderr, "\n");
 	/* Print the orphan extents if needed */
 	if (errors & I_ERR_FILE_EXTENT_ORPHAN)
@@ -811,6 +814,8 @@ static void maybe_free_inode_rec(struct cache_tree *inode_cache,
 	if (!rec->checked || rec->merging)
 		return;
 
Quoted 7 lines

check/main.c static void maybe_free_inode_rec(struct cache_tree *inode_cache, · line 818

Quoted 2 lines
check/main.c +2 -0
+	if (!is_valid_imode(rec->imode))
+		rec->errors |= I_ERR_INVALID_IMODE;

should this check actually be moved before the call to imode_to_type. Because if the mode is busted this means we could potentially get REF_ERR_FILETYPE_UNMATCH error? I.e this check should really fail before anything else ?

Yes, that should be the case.

I'll update the patch to do this check before using imode.

Thanks,
Qu

Quoted 17 lines
Quoted 17 lines
check/main.c +0 -0
 	if (S_ISDIR(rec->imode)) {
 		if (rec->found_size != rec->isize)
 			rec->errors |= I_ERR_DIR_ISIZE_WRONG;
check/mode-original.h +1 -0
index 25ca274118a7..e40a12930a6f 100644
@@ -189,6 +189,7 @@ struct file_extent_hole {
 #define I_ERR_ODD_INODE_FLAGS		(1 << 16)
 #define I_ERR_INLINE_RAM_BYTES_WRONG	(1 << 17)
 #define I_ERR_MISMATCH_DIR_HASH		(1 << 18)
+#define I_ERR_INVALID_IMODE		(1 << 19)
 
 struct inode_record {
 	struct list_head backrefs;
author
#6 lore raw

PATCH 3/7 btrfs-progs: check/lowmem: Repair invalid inode mode in root tree

In reply to Qu Wenruo

In root tree, we only have 2 types of inodes:
- ROOT_TREE_DIR inode
Its mode is fixed to 40755
- free space cache inodes
Its mode is fixed to 100600

This patch will add the ability to repair such inodes to lowmem mode. For fs/subvolume tree error, at least we haven't see such corruption yet, so we don't need to rush to fix corruption in fs trees yet.

The repair function, repair_imode() can be reused by later original mode patch, so it's placed in check/mode-common.c.

---
 check/mode-common.c | 48 +++++++++++++++++++++++++++++++++++++++++++
 check/mode-common.h |  2 ++
 check/mode-lowmem.c | 50 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 99 insertions(+), 1 deletion(-)
check/mode-common.c +48 -0
index fed102b0ce7a..1c150a3d5a7c 100644
@@ -795,3 +795,51 @@ out:
 	btrfs_release_path(&path);
 	return ret;
 }
+
+/*
+ * Reset inode mode with invalid mode to its known correct value.
+ *
+ * Only support free space cache inode and root tree dir inode of root tree.
+ * Caller should ensure such condition.
+ *
+ * Return 0 if repair is done, @path will point to the correct inode item.
+ * Return <0 for errors.
+ */
+int repair_imode(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+		 struct btrfs_path *path, u64 ino)
+{
+	struct btrfs_inode_item *iitem;
+	struct extent_buffer *leaf;
+	struct btrfs_key key;
+	u32 correct_mode;
+	int slot;
+	int ret;
+
+	ASSERT(root->root_key.objectid == BTRFS_ROOT_TREE_OBJECTID);
+	ASSERT(ino == BTRFS_ROOT_TREE_DIR_OBJECTID || is_fstree(ino));
+
+	if (ino == BTRFS_ROOT_TREE_DIR_OBJECTID)
+		correct_mode = 040755;	/* Directory, 0755 */
+	else
+		correct_mode = 0100600;	/* Regular file, 0600 */
+
+	key.objectid = ino;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+
+	ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+	if (ret > 0)
+		ret = -ENOENT;
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to search tree %llu: %m",
+		      root->root_key.objectid);
+		return ret;
+	}
+	leaf = path->nodes[0];
+	slot = path->slots[0];
+	iitem = btrfs_item_ptr(leaf, slot, struct btrfs_inode_item);
+	btrfs_set_inode_mode(leaf, iitem, correct_mode);
+	btrfs_mark_buffer_dirty(leaf);
+	return ret;
+}
check/mode-common.h +2 -0
index 4c88365abbcc..8895747adf1d 100644
@@ -125,6 +125,8 @@ int delete_corrupted_dir_item(struct btrfs_trans_handle *trans,
 			      struct btrfs_root *root,
 			      struct btrfs_key *di_key, char *namebuf,
 			      u32 namelen);
+int repair_imode(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+		 struct btrfs_path *path, u64 ino);
 
 /*
  * Check if the inode mode @imode is valid
check/mode-lowmem.c +49 -1

62 lines not on this page · raw message

Signature · 1 line
2.21.0
#7 lore raw

Re: [PATCH 3/7] btrfs-progs: check/lowmem: Repair invalid inode mode in root tree

Quoted 31 lines · Qu Wenruo

In root tree, we only have 2 types of inodes:
- ROOT_TREE_DIR inode
Its mode is fixed to 40755
- free space cache inodes
Its mode is fixed to 100600

This patch will add the ability to repair such inodes to lowmem mode. For fs/subvolume tree error, at least we haven't see such corruption yet, so we don't need to rush to fix corruption in fs trees yet.

The repair function, repair_imode() can be reused by later original mode patch, so it's placed in check/mode-common.c.

---
 check/mode-common.c | 48 +++++++++++++++++++++++++++++++++++++++++++
 check/mode-common.h |  2 ++
 check/mode-lowmem.c | 50 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 99 insertions(+), 1 deletion(-)
check/mode-common.c +3 -0
index fed102b0ce7a..1c150a3d5a7c 100644
@@ -795,3 +795,51 @@ out:
 	btrfs_release_path(&path);
 	return ret;
 }
+
+/*
+ * Reset inode mode with invalid mode to its known correct value.

check/mode-common.c out: · line 810

Quoted 10 lines
check/mode-common.c +10 -0
+ *
+ * Only support free space cache inode and root tree dir inode of root tree.
+ * Caller should ensure such condition.
+ *
+ * Return 0 if repair is done, @path will point to the correct inode item.
+ * Return <0 for errors.
+ */
+int repair_imode(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+		 struct btrfs_path *path, u64 ino)
+{

How about renaming this function to reset_imode/set_imode:

reset_imode and make it take a u32 parameter for the imode.

Quoted 77 lines
check/mode-common.c +35 -0
+	struct btrfs_inode_item *iitem;
+	struct extent_buffer *leaf;
+	struct btrfs_key key;
+	u32 correct_mode;
+	int slot;
+	int ret;
+
+	ASSERT(root->root_key.objectid == BTRFS_ROOT_TREE_OBJECTID);
+	ASSERT(ino == BTRFS_ROOT_TREE_DIR_OBJECTID || is_fstree(ino));
+
+	if (ino == BTRFS_ROOT_TREE_DIR_OBJECTID)
+		correct_mode = 040755;	/* Directory, 0755 */
+	else
+		correct_mode = 0100600;	/* Regular file, 0600 */
+
+	key.objectid = ino;
+	key.type = BTRFS_INODE_ITEM_KEY;
+	key.offset = 0;
+
+	ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+	if (ret > 0)
+		ret = -ENOENT;
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to search tree %llu: %m",
+		      root->root_key.objectid);
+		return ret;
+	}
+	leaf = path->nodes[0];
+	slot = path->slots[0];
+	iitem = btrfs_item_ptr(leaf, slot, struct btrfs_inode_item);
+	btrfs_set_inode_mode(leaf, iitem, correct_mode);
+	btrfs_mark_buffer_dirty(leaf);
+	return ret;
+}
check/mode-common.h +2 -0
index 4c88365abbcc..8895747adf1d 100644
@@ -125,6 +125,8 @@ int delete_corrupted_dir_item(struct btrfs_trans_handle *trans,
 			      struct btrfs_root *root,
 			      struct btrfs_key *di_key, char *namebuf,
 			      u32 namelen);
+int repair_imode(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+		 struct btrfs_path *path, u64 ino);
 
 /*
  * Check if the inode mode @imode is valid
check/mode-lowmem.c +20 -0
index 1553a4a5d2c1..f37c3b42e6b6 100644
@@ -2396,6 +2396,48 @@ static bool has_orphan_item(struct btrfs_root *root, u64 ino)
 	return false;
 }
 
+static int repair_imode_lowmem(struct btrfs_root *root,
+			       struct btrfs_path *path)
+{
 
+	struct btrfs_trans_handle *trans;
+	struct btrfs_key key;
+	int ret;
+
+	if (root->root_key.objectid != BTRFS_ROOT_TREE_OBJECTID) {
+		error(
+		"repair inode mode outside of root tree is not supported yet");
+		return -ENOTTY;
+	}
+	btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+	ASSERT(key.type == BTRFS_INODE_ITEM_KEY);
+	if (key.objectid != BTRFS_ROOT_TREE_DIR_OBJECTID &&
+	    !is_fstree(key.objectid)) {
+		error("unsupported ino %llu", key.objectid);
+		return -ENOTTY;
+	}
+

check/mode-lowmem.c static bool has_orphan_item(struct btrfs_root *root, u64 ino) · line 2429

Quoted 10 lines
check/mode-lowmem.c +10 -0
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error("failed to start transaction: %m");
+		return ret;
+	}
+	btrfs_release_path(path);
+
+	ret = repair_imode(trans, root, path, key.objectid);

And calling it here with the correct parameter based on whether ino is BTRFS_ROOT_TREE_DIR_OBJECTID. Also move the asserts from reset_imode to this function.

Otherwise you are separating the checks/asserts across 2 function and this is somewhat cumbersome to follow.

Quoted 31 lines
check/mode-lowmem.c +12 -0
+	if (ret < 0)
+		goto abort;
+	ret = btrfs_commit_transaction(trans, root);
+	if (!ret)
+		printf("reset mode for inode %llu root %llu\n",
+			key.objectid, root->root_key.objectid);
+	return ret;
+abort:
+	btrfs_abort_transaction(trans, ret);
+ 	return ret;
+}
+
 /*
  * Check INODE_ITEM and related ITEMs (the same inode number)

* 1. check link count

check/mode-lowmem.c +7 -1
@@ -2454,7 +2496,13 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)
 
 	if (!is_valid_imode(mode)) {
 		error("invalid imode mode bits: 0%o", mode);
-		err |= INODE_MODE_ERROR;
+		if (repair) {
+			ret = repair_imode_lowmem(root, path);
+			if (ret < 0)
+				err |= INODE_MODE_ERROR;
+		} else {
+			err |= INODE_MODE_ERROR;
+		}
 	}
 
 	if (S_ISLNK(mode) &&
#8 lore raw

Re: [PATCH 3/7] btrfs-progs: check/lowmem: Repair invalid inode mode in root tree

Quoted 48 lines · Nikolay Borisov
Quoted 31 lines · Qu Wenruo

In root tree, we only have 2 types of inodes:
- ROOT_TREE_DIR inode
Its mode is fixed to 40755
- free space cache inodes
Its mode is fixed to 100600

This patch will add the ability to repair such inodes to lowmem mode. For fs/subvolume tree error, at least we haven't see such corruption yet, so we don't need to rush to fix corruption in fs trees yet.

The repair function, repair_imode() can be reused by later original mode patch, so it's placed in check/mode-common.c.

---
 check/mode-common.c | 48 +++++++++++++++++++++++++++++++++++++++++++
 check/mode-common.h |  2 ++
 check/mode-lowmem.c | 50 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 99 insertions(+), 1 deletion(-)
check/mode-common.c +3 -0
index fed102b0ce7a..1c150a3d5a7c 100644
@@ -795,3 +795,51 @@ out:
 	btrfs_release_path(&path);
 	return ret;
 }
+
+/*
+ * Reset inode mode with invalid mode to its known correct value.

check/mode-common.c out: · line 810

Quoted 10 lines
check/mode-common.c +10 -0
+ *
+ * Only support free space cache inode and root tree dir inode of root tree.
+ * Caller should ensure such condition.
+ *
+ * Return 0 if repair is done, @path will point to the correct inode item.
+ * Return <0 for errors.
+ */
+int repair_imode(struct btrfs_trans_handle *trans, struct btrfs_root *root,
+		 struct btrfs_path *path, u64 ino)
+{

How about renaming this function to reset_imode/set_imode:

reset_imode and make it take a u32 parameter for the imode.

[snip]

Quoted 14 lines
Quoted 14 lines
check/mode-common.c +14 -0
+	btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+	ASSERT(key.type == BTRFS_INODE_ITEM_KEY);
+	if (key.objectid != BTRFS_ROOT_TREE_DIR_OBJECTID &&
+	    !is_fstree(key.objectid)) {
+		error("unsupported ino %llu", key.objectid);
+		return -ENOTTY;
+	}
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error("failed to start transaction: %m");
+		return ret;
Quoted 8 lines

check/mode-common.c

Quoted 4 lines
check/mode-common.c +4 -0
+	}
+	btrfs_release_path(path);
+
+	ret = repair_imode(trans, root, path, key.objectid);

And calling it here with the correct parameter based on whether ino is BTRFS_ROOT_TREE_DIR_OBJECTID. Also move the asserts from reset_imode to this function.

Makes sense.

I'll go that direction.

Thanks,
Qu

Quoted 4 lines

Otherwise you are separating the checks/asserts across 2 function and this is somewhat cumbersome to follow.

author
#9 lore raw

PATCH 4/7 btrfs-progs: check/original: Repair invalid inode mode in root tree

In reply to Qu Wenruo

This patch will reuse the mode independent repair_imode() function, to repair invalid inode mode.

---
 check/main.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
check/main.c +20 -0
index 553c93caa2c9..5db0716d23a1 100644
@@ -2698,6 +2698,24 @@ static int repair_mismatch_dir_hash(struct btrfs_trans_handle *trans,
 	return ret;
 }
 
+static int repair_imode_original(struct btrfs_trans_handle *trans,
+				 struct btrfs_root *root,
+				 struct btrfs_path *path,
+				 struct inode_record *rec)
+{
+	int ret;
+
+	if (root->root_key.objectid != BTRFS_ROOT_TREE_OBJECTID)
+		return 0;
+	if (rec->ino != BTRFS_ROOT_TREE_DIR_OBJECTID || !is_fstree(rec->ino))
+		return 0;
+	ret = repair_imode(trans, root, path, rec->ino);
+	if (ret < 0)
+		return ret;
+	rec->errors &= ~I_ERR_INVALID_IMODE;
+	return ret;
+}
+
 static int try_repair_inode(struct btrfs_root *root, struct inode_record *rec)
 {
 	struct btrfs_trans_handle *trans;
@@ -2745,6 +2763,8 @@ static int try_repair_inode(struct btrfs_root *root, struct inode_record *rec)
 		ret = repair_inode_nbytes(trans, root, &path, rec);
 	if (!ret && rec->errors & I_ERR_INLINE_RAM_BYTES_WRONG)
 		ret = repair_inline_ram_bytes(trans, root, &path, rec);
+	if (!ret && rec->errors & I_ERR_INVALID_IMODE)
+		ret = repair_imode_original(trans, root, &path, rec);
 	btrfs_commit_transaction(trans, root);
 	btrfs_release_path(&path);
 	return ret;
Signature · 1 line
2.21.0
author
#10 lore raw

PATCH 5/7 btrfs: check/lowmem: Check and repair free space cache inode mode

In reply to Qu Wenruo

Unlike inodes in fs roots, we don't really check the inode items in root tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF.

This makes invalid inode items sneak into root tree. For example:

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
				   ^ Should be 100600
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

There is a report of such problem in the mail list.

This patch will check and repair inode items of free space cache inodes in lowmem mode.

Since free space cache inodes doesn't have INODE_REF but still has 1 link, we can't use check_inode_item() directly.
Instead we only check the inode mode, as that's the important part.

The check and repair function: check_repair_free_space_inode() is also exported for original mode.

---
 check/mode-common.h |  1 +
 check/mode-lowmem.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 check/mode-lowmem.h |  2 ++
 3 files changed, 48 insertions(+)
check/mode-common.h +1 -0
index 8895747adf1d..855f83617854 100644
@@ -24,6 +24,7 @@
 #include <sys/stat.h>
 #include "ctree.h"
 
+#define FREE_SPACE_CACHE_INODE_MODE	(0100600)
 /*
  * Use for tree walk to walk through trees whose leaves/nodes can be shared
  * between different trees. (Namely subvolume/fs trees)
check/mode-lowmem.c +45 -0

57 lines not on this page · raw message

check/mode-lowmem.h +2 -0
index e0ab30b770d5..d2983fd12eb4 100644
@@ -67,5 +67,7 @@
 
 int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info);
 int check_chunks_and_extents_lowmem(struct btrfs_fs_info *fs_info);
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path);
 
 #endif
Signature · 1 line
2.21.0
#11 lore raw

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

Quoted 52 lines · Qu Wenruo

Unlike inodes in fs roots, we don't really check the inode items in root tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF.

This makes invalid inode items sneak into root tree. For example:

         item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                 generation 30 transid 30 size 65536 nbytes 1507328
                 block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
				   ^ Should be 100600
                 sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                 atime 0.0 (1970-01-01 08:00:00)
                 ctime 1553491158.189771625 (2019-03-25 13:19:18)
                 mtime 0.0 (1970-01-01 08:00:00)
                 otime 0.0 (1970-01-01 08:00:00)

There is a report of such problem in the mail list.

This patch will check and repair inode items of free space cache inodes in lowmem mode.

Since free space cache inodes doesn't have INODE_REF but still has 1 link, we can't use check_inode_item() directly.
Instead we only check the inode mode, as that's the important part.

The check and repair function: check_repair_free_space_inode() is also exported for original mode.

---
  check/mode-common.h |  1 +
  check/mode-lowmem.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
  check/mode-lowmem.h |  2 ++
  3 files changed, 48 insertions(+)
check/mode-common.h +1 -0
index 8895747adf1d..855f83617854 100644
@@ -24,6 +24,7 @@
  #include <sys/stat.h>
  #include "ctree.h"
 
+#define FREE_SPACE_CACHE_INODE_MODE	(0100600)
  /*
   * Use for tree walk to walk through trees whose leaves/nodes can be shared
   * between different trees. (Namely subvolume/fs trees)
check/mode-lowmem.c +0 -0
index f37c3b42e6b6..cdb9b66a63a3 100644
@@ -5091,6 +5091,40 @@ out:
  	return err;

check/mode-lowmem.c out: · line 5101

Quoted 10 lines
check/mode-lowmem.c +8 -0
  }
 
+/*
+ * For free space inodes, we can't call check_inode_item() as free space
+ * cache inode doesn't have INODE_REF.
+ * We just check its inode mode.
+ */
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path)
+{

In personal taste, I prefer check_free_space_inode() to follow existed code.

Quoted 30 lines
check/mode-lowmem.c +26 -0
+	struct btrfs_inode_item *iitem;
+	struct btrfs_key key;
+	u32 mode;
+	int ret = 0;
+
+	btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+	ASSERT(key.type == BTRFS_INODE_ITEM_KEY && is_fstree(key.objectid));
+	iitem = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			       struct btrfs_inode_item);
+	mode = btrfs_inode_mode(path->nodes[0], iitem);
+	if (mode != FREE_SPACE_CACHE_INODE_MODE) {
+		error(
+	"free space cache inode %llu has invalid mode: has 0%o expect 0%o",
+			key.objectid, mode, FREE_SPACE_CACHE_INODE_MODE);
+		ret = -EUCLEAN;
+		if (repair) {
+			ret = repair_imode_lowmem(fs_info->tree_root,
+						  path);
+			if (ret < 0)
+				return ret;
+			return ret;
+		}
+	}
+	return ret;
+}
+
  /*
   * Check all fs/file tree in low_memory mode.
   *

check/mode-lowmem.c int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info) · line 5170

Quoted 7 lines
check/mode-lowmem.c +4 -0
@@ -5130,6 +5164,17 @@ int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info)
  		btrfs_item_key_to_cpu(node, &key, slot);
  		if (key.objectid > BTRFS_LAST_FREE_OBJECTID)
  			goto out;
+		if (key.type == BTRFS_INODE_ITEM_KEY &&
+		    is_fstree(key.objectid)) {
+			ret = check_repair_free_space_inode(fs_info, &path);
+			/* Check if we still have a valid path to continue */

'path.nodes[0]' is quite interesting :).
What about the situation btrfs_search_slot() succeed but failed to commit a transaction?

---
Su

Quoted 23 lines
check/mode-lowmem.c +7 -0
+			if (ret < 0 && path.nodes[0]) {
+				err |= ret;
+				goto next;
+			}
+			if (ret < 0 && !path.nodes[0])
+				goto out;
+		}
  		if (key.type == BTRFS_ROOT_ITEM_KEY &&
  		    fs_root_objectid(key.objectid)) {
  			if (key.objectid == BTRFS_TREE_RELOC_OBJECTID) {
check/mode-lowmem.h +2 -0
index e0ab30b770d5..d2983fd12eb4 100644
@@ -67,5 +67,7 @@
 
  int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info);
  int check_chunks_and_extents_lowmem(struct btrfs_fs_info *fs_info);
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path);
 
  #endif
#12 lore raw

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

Quoted 107 lines · Su Yue
Quoted 55 lines · Qu Wenruo

Unlike inodes in fs roots, we don't really check the inode items in root tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF.

This makes invalid inode items sneak into root tree.
For example:
         item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160 generation 30 transid 30 size 65536 nbytes 1507328 block group 0 mode 0 links 1 uid 0 gid 0 rdev 0 ^ Should be 100600
                 sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                 atime 0.0 (1970-01-01 08:00:00)
                 ctime 1553491158.189771625 (2019-03-25 13:19:18) mtime 0.0 (1970-01-01 08:00:00)
                 otime 0.0 (1970-01-01 08:00:00)

There is a report of such problem in the mail list.

This patch will check and repair inode items of free space cache inodes in
lowmem mode.

Since free space cache inodes doesn't have INODE_REF but still has 1 link, we can't use check_inode_item() directly.
Instead we only check the inode mode, as that's the important part.

The check and repair function: check_repair_free_space_inode() is also exported for original mode.

---

  check/mode-common.h |  1 +
  check/mode-lowmem.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
  check/mode-lowmem.h |  2 ++
  3 files changed, 48 insertions(+)
check/mode-common.h +1 -0
index 8895747adf1d..855f83617854 100644
@@ -24,6 +24,7 @@
   #include <sys/stat.h>
   #include "ctree.h"
 
+#define FREE_SPACE_CACHE_INODE_MODE    (0100600)
   /*
    * Use for tree walk to walk through trees whose leaves/nodes can be

shared
   * between different trees. (Namely subvolume/fs trees)

check/mode-lowmem.c +0 -0
index f37c3b42e6b6..cdb9b66a63a3 100644
@@ -5091,6 +5091,40 @@ out:
       return err;

check/mode-lowmem.c out: · line 5101

Quoted 10 lines
check/mode-lowmem.c +8 -0
   }
 
+/*
+ * For free space inodes, we can't call check_inode_item() as free space
+ * cache inode doesn't have INODE_REF.
+ * We just check its inode mode.
+ */
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+                  struct btrfs_path *path)
+{

In personal taste, I prefer check_free_space_inode() to follow existed code.

Quoted 34 lines
check/mode-lowmem.c +26 -0
+    struct btrfs_inode_item *iitem;
+    struct btrfs_key key;
+    u32 mode;
+    int ret = 0;
+
+    btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+    ASSERT(key.type == BTRFS_INODE_ITEM_KEY && is_fstree(key.objectid));
+    iitem = btrfs_item_ptr(path->nodes[0], path->slots[0],
+                   struct btrfs_inode_item);
+    mode = btrfs_inode_mode(path->nodes[0], iitem);
+    if (mode != FREE_SPACE_CACHE_INODE_MODE) {
+        error(
+    "free space cache inode %llu has invalid mode: has 0%o expect 0%o",
+            key.objectid, mode, FREE_SPACE_CACHE_INODE_MODE);
+        ret = -EUCLEAN;
+        if (repair) {
+            ret = repair_imode_lowmem(fs_info->tree_root,
+                          path);
+            if (ret < 0)
+                return ret;
+            return ret;
+        }
+    }
+    return ret;
+}
+
   /*
    * Check all fs/file tree in low_memory mode.
    *
@@ -5130,6 +5164,17 @@ int check_fs_roots_lowmem(struct btrfs_fs_info
 *fs_info)
           btrfs_item_key_to_cpu(node, &key, slot);
           if (key.objectid > BTRFS_LAST_FREE_OBJECTID)
               goto out;
Quoted 8 lines

check/mode-lowmem.c int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info) · line 5170

Quoted 4 lines
check/mode-lowmem.c +4 -0
+        if (key.type == BTRFS_INODE_ITEM_KEY &&
+            is_fstree(key.objectid)) {
+            ret = check_repair_free_space_inode(fs_info, &path);
+            /* Check if we still have a valid path to continue */

'path.nodes[0]' is quite interesting :).
What about the situation btrfs_search_slot() succeed but failed to commit a transaction?

As long as path.nodes[0] exists, we still have a valid path of root tree, check can continue going.

For that valid path but failed commit case, I believe the most possible problem would be a corrupted extent tree. And in that case, we can still continue checking fs tree.

Thanks,
Qu

Quoted 28 lines

---
Su

Quoted 23 lines
check/mode-lowmem.c +7 -0
+            if (ret < 0 && path.nodes[0]) {
+                err |= ret;
+                goto next;
+            }
+            if (ret < 0 && !path.nodes[0])
+                goto out;
+        }

          if (key.type == BTRFS_ROOT_ITEM_KEY && fs_root_objectid(key.objectid)) {
              if (key.objectid == BTRFS_TREE_RELOC_OBJECTID) {

check/mode-lowmem.h +2 -0
index e0ab30b770d5..d2983fd12eb4 100644
@@ -67,5 +67,7 @@
 
   int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info);
   int check_chunks_and_extents_lowmem(struct btrfs_fs_info *fs_info);
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+                  struct btrfs_path *path);
 
   #endif
#13 lore raw

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

In reply to Qu Wenruo
Quoted 77 lines · Qu Wenruo

Unlike inodes in fs roots, we don't really check the inode items in root tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF.

This makes invalid inode items sneak into root tree. For example:

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
				   ^ Should be 100600
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

There is a report of such problem in the mail list.

This patch will check and repair inode items of free space cache inodes in lowmem mode.

Since free space cache inodes doesn't have INODE_REF but still has 1 link, we can't use check_inode_item() directly.
Instead we only check the inode mode, as that's the important part.

The check and repair function: check_repair_free_space_inode() is also exported for original mode.

---
 check/mode-common.h |  1 +
 check/mode-lowmem.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 check/mode-lowmem.h |  2 ++
 3 files changed, 48 insertions(+)
check/mode-common.h +1 -0
index 8895747adf1d..855f83617854 100644
@@ -24,6 +24,7 @@
 #include <sys/stat.h>
 #include "ctree.h"
 
+#define FREE_SPACE_CACHE_INODE_MODE	(0100600)
 /*
  * Use for tree walk to walk through trees whose leaves/nodes can be shared
  * between different trees. (Namely subvolume/fs trees)
check/mode-lowmem.c +23 -0
index f37c3b42e6b6..cdb9b66a63a3 100644
@@ -5091,6 +5091,40 @@ out:
 	return err;
 }
 
+/*
+ * For free space inodes, we can't call check_inode_item() as free space
+ * cache inode doesn't have INODE_REF.
+ * We just check its inode mode.
+ */
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path)
+{
+	struct btrfs_inode_item *iitem;
+	struct btrfs_key key;
+	u32 mode;
+	int ret = 0;
+
+	btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+	ASSERT(key.type == BTRFS_INODE_ITEM_KEY && is_fstree(key.objectid));
+	iitem = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			       struct btrfs_inode_item);
+	mode = btrfs_inode_mode(path->nodes[0], iitem);
+	if (mode != FREE_SPACE_CACHE_INODE_MODE) {
+		error(
+	"free space cache inode %llu has invalid mode: has 0%o expect 0%o",
+			key.objectid, mode, FREE_SPACE_CACHE_INODE_MODE);
+		ret = -EUCLEAN;

check/mode-lowmem.c out: · line 5126

Quoted 10 lines
check/mode-lowmem.c +10 -0
+		if (repair) {
+			ret = repair_imode_lowmem(fs_info->tree_root,
+						  path);
+			if (ret < 0)
+				return ret;
+			return ret;
+		}
+	}
+	return ret;
+}

You put this function in mode-lowmem, yet it's also used in the next patch to implement check/repair for original mode. Since you are oging to re-use function across the 2 modes it makes no sense to separate them in their names. Perhaps put them in mode-common.c

Quoted 35 lines
check/mode-lowmem.c +12 -0
+
 /*
  * Check all fs/file tree in low_memory mode.
  *
@@ -5130,6 +5164,17 @@ int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info)
 		btrfs_item_key_to_cpu(node, &key, slot);
 		if (key.objectid > BTRFS_LAST_FREE_OBJECTID)
 			goto out;
+		if (key.type == BTRFS_INODE_ITEM_KEY &&
+		    is_fstree(key.objectid)) {
+			ret = check_repair_free_space_inode(fs_info, &path);
+			/* Check if we still have a valid path to continue */
+			if (ret < 0 && path.nodes[0]) {
+				err |= ret;
+				goto next;
+			}
+			if (ret < 0 && !path.nodes[0])
+				goto out;
+		}
 		if (key.type == BTRFS_ROOT_ITEM_KEY &&
 		    fs_root_objectid(key.objectid)) {
 			if (key.objectid == BTRFS_TREE_RELOC_OBJECTID) {
check/mode-lowmem.h +2 -0
index e0ab30b770d5..d2983fd12eb4 100644
@@ -67,5 +67,7 @@
 
 int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info);
 int check_chunks_and_extents_lowmem(struct btrfs_fs_info *fs_info);
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path);
 
 #endif
#14 lore raw

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

[snip]

Quoted 30 lines
Quoted 30 lines
check/mode-lowmem.c +30 -0
+/*
+ * For free space inodes, we can't call check_inode_item() as free space
+ * cache inode doesn't have INODE_REF.
+ * We just check its inode mode.
+ */
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path)
+{
+	struct btrfs_inode_item *iitem;
+	struct btrfs_key key;
+	u32 mode;
+	int ret = 0;
+
+	btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+	ASSERT(key.type == BTRFS_INODE_ITEM_KEY && is_fstree(key.objectid));
+	iitem = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			       struct btrfs_inode_item);
+	mode = btrfs_inode_mode(path->nodes[0], iitem);
+	if (mode != FREE_SPACE_CACHE_INODE_MODE) {
+		error(
+	"free space cache inode %llu has invalid mode: has 0%o expect 0%o",
+			key.objectid, mode, FREE_SPACE_CACHE_INODE_MODE);
+		ret = -EUCLEAN;
+		if (repair) {
+			ret = repair_imode_lowmem(fs_info->tree_root,
+						  path);
+			if (ret < 0)
+				return ret;
+			return ret;
+		}
Quoted 9 lines

check/mode-lowmem.c out: · line 5126

Quoted 3 lines
check/mode-lowmem.c +3 -0
+	}
+	return ret;
+}

You put this function in mode-lowmem, yet it's also used in the next patch to implement check/repair for original mode. Since you are oging to re-use function across the 2 modes it makes no sense to separate them in their names. Perhaps put them in mode-common.c

The limitation is due to repair_imode_lowmem().

Thanks,
Qu

Quoted 35 lines
Quoted 35 lines
check/mode-lowmem.c +12 -0
+
 /*
  * Check all fs/file tree in low_memory mode.
  *
@@ -5130,6 +5164,17 @@ int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info)
 		btrfs_item_key_to_cpu(node, &key, slot);
 		if (key.objectid > BTRFS_LAST_FREE_OBJECTID)
 			goto out;
+		if (key.type == BTRFS_INODE_ITEM_KEY &&
+		    is_fstree(key.objectid)) {
+			ret = check_repair_free_space_inode(fs_info, &path);
+			/* Check if we still have a valid path to continue */
+			if (ret < 0 && path.nodes[0]) {
+				err |= ret;
+				goto next;
+			}
+			if (ret < 0 && !path.nodes[0])
+				goto out;
+		}
 		if (key.type == BTRFS_ROOT_ITEM_KEY &&
 		    fs_root_objectid(key.objectid)) {
 			if (key.objectid == BTRFS_TREE_RELOC_OBJECTID) {
check/mode-lowmem.h +2 -0
index e0ab30b770d5..d2983fd12eb4 100644
@@ -67,5 +67,7 @@
 
 int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info);
 int check_chunks_and_extents_lowmem(struct btrfs_fs_info *fs_info);
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path);
 
 #endif
#15 lore raw

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

Quoted 41 lines · Qu Wenruo

[snip]

Quoted 30 lines
Quoted 30 lines
check/mode-lowmem.c +30 -0
+/*
+ * For free space inodes, we can't call check_inode_item() as free space
+ * cache inode doesn't have INODE_REF.
+ * We just check its inode mode.
+ */
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path)
+{
+	struct btrfs_inode_item *iitem;
+	struct btrfs_key key;
+	u32 mode;
+	int ret = 0;
+
+	btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
+	ASSERT(key.type == BTRFS_INODE_ITEM_KEY && is_fstree(key.objectid));
+	iitem = btrfs_item_ptr(path->nodes[0], path->slots[0],
+			       struct btrfs_inode_item);
+	mode = btrfs_inode_mode(path->nodes[0], iitem);
+	if (mode != FREE_SPACE_CACHE_INODE_MODE) {
+		error(
+	"free space cache inode %llu has invalid mode: has 0%o expect 0%o",
+			key.objectid, mode, FREE_SPACE_CACHE_INODE_MODE);
+		ret = -EUCLEAN;
+		if (repair) {
+			ret = repair_imode_lowmem(fs_info->tree_root,
+						  path);
+			if (ret < 0)
+				return ret;
+			return ret;
+		}
Quoted 9 lines

check/mode-lowmem.c out: · line 5126

Quoted 3 lines
check/mode-lowmem.c +3 -0
+	}
+	return ret;
+}

You put this function in mode-lowmem, yet it's also used in the next patch to implement check/repair for original mode. Since you are oging to re-use function across the 2 modes it makes no sense to separate them in their names. Perhaps put them in mode-common.c

Quoted 1 line

The limitation is due to repair_imode_lowmem().

move it to the common file as well ?

Quoted 39 lines

Thanks,
Qu

Quoted 35 lines
Quoted 35 lines
check/mode-lowmem.c +12 -0
+
 /*
  * Check all fs/file tree in low_memory mode.
  *
@@ -5130,6 +5164,17 @@ int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info)
 		btrfs_item_key_to_cpu(node, &key, slot);
 		if (key.objectid > BTRFS_LAST_FREE_OBJECTID)
 			goto out;
+		if (key.type == BTRFS_INODE_ITEM_KEY &&
+		    is_fstree(key.objectid)) {
+			ret = check_repair_free_space_inode(fs_info, &path);
+			/* Check if we still have a valid path to continue */
+			if (ret < 0 && path.nodes[0]) {
+				err |= ret;
+				goto next;
+			}
+			if (ret < 0 && !path.nodes[0])
+				goto out;
+		}
 		if (key.type == BTRFS_ROOT_ITEM_KEY &&
 		    fs_root_objectid(key.objectid)) {
 			if (key.objectid == BTRFS_TREE_RELOC_OBJECTID) {
check/mode-lowmem.h +2 -0
index e0ab30b770d5..d2983fd12eb4 100644
@@ -67,5 +67,7 @@
 
 int check_fs_roots_lowmem(struct btrfs_fs_info *fs_info);
 int check_chunks_and_extents_lowmem(struct btrfs_fs_info *fs_info);
+int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
+				  struct btrfs_path *path);
 
 #endif
author
#16 lore raw

PATCH 6/7 btrfs: check/original: Check and repair free space cache inode item

In reply to Qu Wenruo

Just like lowmem mode, also check and repair free space cache inode item.

And since we don't really have a good timing/function to check free space chace inodes, we use the same lowmem mode check_repair_free_space_inode() when iterating root tree.

---
 check/main.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
check/main.c +11 -0
index 5db0716d23a1..b920165e3d0d 100644
@@ -3620,6 +3620,17 @@ again:
 			   key.type == BTRFS_ROOT_BACKREF_KEY) {
 			process_root_ref(leaf, path.slots[0], &key,
 					 root_cache);
+		} else if (key.type == BTRFS_INODE_ITEM_KEY &&
+			   is_fstree(key.objectid)) {
+			ret = check_repair_free_space_inode(fs_info, &path);
+			if (ret < 0 && !path.nodes[0]) {
+				err = 1;
+				goto out;
+			}
+			if (ret < 0 && path.nodes[0]) {
+				err = 1;
+				goto next;
+			}
 		}
 next:
 		path.slots[0]++;
Signature · 1 line
2.21.0
author
#17 lore raw

PATCH 7/7 btrfs: tests/fsck: Add test image for free space cache mode repair

In reply to Qu Wenruo

The image has one free space cache inode with invalid mode (0).

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

Both lowmem and original mode should be able to detect and fix it.

The extracted test image is pretty big (1G extracted), as kernel won't cache small chunks.
Even with SSD, such test may still take some seconds just extracting the image.

---
 .../test.raw.xz                                | Bin 0 -> 159128 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz
tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..c2603c024098992bece235d27187eefa696b3fe0
GIT binary patch
Signature · 1 line
2.21.0
#18 lore raw

Re: [PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

Quoted 10 lines · Qu Wenruo

The image has one free space cache inode with invalid mode (0).

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)
Quoted 6 lines

Both lowmem and original mode should be able to detect and fix it.

The extracted test image is pretty big (1G extracted), as kernel won't cache small chunks.
Even with SSD, such test may still take some seconds just extracting the image.

Should this image contain also a non-freespace but root inode (for the root dir case) so it has full coverage of the code you introduced?

Quoted 122 lines
---
 .../test.raw.xz                                | Bin 0 -> 159128 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz
tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..c2603c024098992bece235d27187eefa696b3fe0
GIT binary patch
#19 lore raw

Re: [PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

Quoted 15 lines · Nikolay Borisov
Quoted 12 lines · Qu Wenruo

The image has one free space cache inode with invalid mode (0).

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

Both lowmem and original mode should be able to detect and fix it.

Quoted 8 lines
Quoted 4 lines

The extracted test image is pretty big (1G extracted), as kernel won't cache small chunks.
Even with SSD, such test may still take some seconds just extracting the image.

Should this image contain also a non-freespace but root inode (for the root dir case) so it has full coverage of the code you introduced?

OK, another image will be added for ROOT_TREE_DIR.

Thanks,
Qu

Quoted 123 lines
Quoted 123 lines
---
 .../test.raw.xz                                | Bin 0 -> 159128 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz
tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..c2603c024098992bece235d27187eefa696b3fe0
GIT binary patch
#20 lore raw

Re: [PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

In reply to Nikolay Borisov
Quoted 15 lines · Nikolay Borisov
Quoted 12 lines · Qu Wenruo

The image has one free space cache inode with invalid mode (0).

        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

Both lowmem and original mode should be able to detect and fix it.

Quoted 8 lines
Quoted 4 lines

The extracted test image is pretty big (1G extracted), as kernel won't cache small chunks.
Even with SSD, such test may still take some seconds just extracting the image.

Should this image contain also a non-freespace but root inode (for the root dir case) so it has full coverage of the code you introduced?

In fact the only possible repair is for free space cache. ROOT_TREE_DIR repair is just a common base, and there is no entry for it.

Thanks,
Qu

Quoted 123 lines
Quoted 123 lines
---
 .../test.raw.xz                                | Bin 0 -> 159128 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz
tests/fsck-tests/038-bad-free-space-cache-inode-mode/test.raw.xz new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..c2603c024098992bece235d27187eefa696b3fe0
GIT binary patch