Skip to contentSkip to conversation

btrfs-progs: fsck: detect obviously invalid metadata backref level

PATCH In review Qu Wenruo 3 participants 8 messages Jan 17, 2022 – Feb 1, 2022 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: fsck: detect obviously invalid metadata backref level

There is a report that kernel tree-checker rejects a tree block of extent tree, as it contains an obvious corrupted level (which is an obvious bit flip).

But btrfs check, at least original mode, doesn't detect it at all. While with my crafted image, lowmem mode would just crash due to the large level value overflowing path->nodes[level].

Lowmem is enhanced to reject such level, and the existing code will verify the level and report errors.

Original mode is more tricky, as it doesn't have level check at all. I don't have a good idea to implement full level check at original mode, so here I just introduced a basic check for tree level and reject it.

Finally introduce a test case for this.

Qu Wenruo (3):
btrfs-progs: check/lowmem: fix crash when METADATA_ITEM has invalid
level
btrfs: check/original: reject bad metadata backref with invalid level
btrfs-progs: tests/fsck: add test image with invalid metadata backref
level

 check/main.c                                  |  19 ++++++++++++++++++
 check/mode-lowmem.c                           |  12 ++++++++++-
 .../053-bad-metadata-level/default.img.xz     | Bin 0 -> 2084 bytes
 .../fsck-tests/053-bad-metadata-level/test.sh |  19 ++++++++++++++++++
 4 files changed, 49 insertions(+), 1 deletion(-)
 create mode 100644 tests/fsck-tests/053-bad-metadata-level/default.img.xz
 create mode 100755 tests/fsck-tests/053-bad-metadata-level/test.sh
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH 1/3 btrfs-progs: check/lowmem: fix crash when METADATA_ITEM has invalid level

[BUG]
When running lowmem mode with METADATA_ITEM which has invalid level, it will crash with the following backtrace:

(gdb) bt
#0 0x0000555555616b0b in btrfs_header_bytenr (eb=0x4)

     at ./kernel-shared/ctree.h:2134
 #1  0x0000555555620c78 in check_tree_block_backref (root_id=5,
     bytenr=30457856, level=256) at check/mode-lowmem.c:3818
 #2  0x0000555555621f6c in check_extent_item (path=0x7fffffffd9c0)
     at check/mode-lowmem.c:4334
 #3  0x00005555556235a5 in check_leaf_items (root=0x555555688e10,
     path=0x7fffffffd9c0, nrefs=0x7fffffffda30, account_bytes=1)
     at check/mode-lowmem.c:4835
 #4  0x0000555555623c6d in walk_down_tree (root=0x555555688e10,
     path=0x7fffffffd9c0, level=0x7fffffffd984, nrefs=0x7fffffffda30,
     check_all=1) at check/mode-lowmem.c:4967
 #5  0x000055555562494f in check_btrfs_root (root=0x555555688e10, check_all=1)
     at check/mode-lowmem.c:5266
 #6  0x00005555556254ee in check_chunks_and_extents_lowmem ()
     at check/mode-lowmem.c:5556
 #7  0x00005555555f0b82 in do_check_chunks_and_extents () at check/main.c:9114
 #8  0x00005555555f50ea in cmd_check (cmd=0x55555567c640 <cmd_struct_check>,
     argc=3, argv=0x7fffffffdec0) at check/main.c:10892
 #9  0x000055555556b2b1 in cmd_execute (argv=0x7fffffffdec0, argc=3,
     cmd=0x55555567c640 <cmd_struct_check>) at cmds/commands.h:125

[CAUSE]
For function check_extent_item() it will go through inline extent items and then check their backrefs.

But for METADATA_ITEM, it doesn't really validate key.offset, which is u64 and can contain value way larger than BTRFS_MAX_LEVEL (mostly caused by bit flip).

In that case, if we have a larger value like 256 in key.offset, then later check_tree_block_backref() will use 256 as level, and overflow path->nodes[level] and crash.

[FIX]
Just verify the level, no matter if it's from btrfs_tree_block_level() (which is just u8), or it's from key.offset (which is u64).

To do the check properly and detect higher bits corruption, also change the type of @level from u8 to u64.

Now lowmem mode can detect the problem properly:

...
[2/7] checking extents
ERROR: tree block 30457856 has bad backref level, has 256 expect [0, 7]
ERROR: extent[30457856 16384] level mismatch, wanted: 0, have: 256
ERROR: errors found in extent allocation tree or chunk allocation [3/7] checking free space tree
...

---
 check/mode-lowmem.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
check/mode-lowmem.c +11 -1
index cc6773cd4d0c..99f519631a50 100644
@@ -4242,7 +4242,7 @@ static int check_extent_item(struct btrfs_path *path)
 	u64 owner_offset;
 	u64 super_gen;
 	int metadata = 0;
-	int level;
+	u64 level;	/* To handle corrupted values in skinny backref */
 	struct btrfs_key key;
 	int ret;
 	int err = 0;
@@ -4303,6 +4303,16 @@ static int check_extent_item(struct btrfs_path *path)
 		/* New METADATA_ITEM */
 		level = key.offset;
 	}
+
+	if (metadata && level >= BTRFS_MAX_LEVEL) {
+		error(
+	"tree block %llu has bad backref level, has %llu expect [0, %u]",
+		      key.objectid, level, BTRFS_MAX_LEVEL - 1);
+		err |= BACKREF_MISMATCH;
+		/* This is a critical error, exit right now */
+		goto out;
+	}
+
 	ptr_offset = ptr - (unsigned long)ei;
 
 next:
Signature · 1 line
2.34.1
author
#6 lore raw

PATCH 3/3 btrfs-progs: tests/fsck: add test image with invalid metadata backref level

In reply to Qu Wenruo

The image has a key in extent tree, (30457856 METADATA_ITEM 256), which has invalid level (256 > BTRFS_MAX_LEVEL).

Make sure check can at least detect such problem.

---
 .../053-bad-metadata-level/default.img.xz     | Bin 0 -> 2084 bytes
 .../fsck-tests/053-bad-metadata-level/test.sh |  19 ++++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 tests/fsck-tests/053-bad-metadata-level/default.img.xz
 create mode 100755 tests/fsck-tests/053-bad-metadata-level/test.sh
tests/fsck-tests/053-bad-metadata-level/default.img.xz new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..d7debee7b7c2bb457dbe734da2429859245770ae
GIT binary patch
tests/fsck-tests/053-bad-metadata-level/test.sh new file +19 -0
new file mode 100755
index 000000000000..0ffd7bdf34a1
@@ -0,0 +1,19 @@
+#!/bin/bash
+#
+# Verify that check can detect invalid metadata backref level.
+#
+# There is a report that btrfs-check original mode doesn't report invalid
+# metadata backref level, and lowmem mode would just crash.
+#
+# Make sure btrfs check can at least detect such error.
+
+source "$TEST_TOP/common"
+
+check_prereq btrfs
+
+check_image() {
+	run_mustfail "invalid metadata backref level not detected" \
+		"$TOP/btrfs" check "$1"
+}
+
+check_all_images
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH 2/3 btrfs: check/original: reject bad metadata backref with invalid level

In reply to Qu Wenruo

[BUG]
There is a bug report that kernel tree-checker rejected an invalid metadata item:

corrupt leaf: block=934474399744 slot=68 extent bytenr=425173254144 len=16384 invalid tree level, have 33554432 expect [0, 7]

But original mode btrfs-check reports nothing wrong. (BTW, lowmem mode will just crash, and fixed in previous patch).

[CAUSE]
For original mode it doesn't really check tree level, thus didn't find the problem.

[FIX]
I don't have a good idea to completely make original mode to verify the level in backref and in the tree block (while lowmem does that).

But at least we can detect obviouly corrupted level just like kernel.

Now original mode will detect such problem:

...
[2/7] checking extents
ERROR: tree block 30457856 has bad backref level, has 256 expect [0, 7] ref mismatch on [30457856 16384] extent item 0, found 1 tree backref 30457856 root 5 not found in extent tree backpointer mismatch on [30457856 16384]
ERROR: errors found in extent allocation tree or chunk allocation [3/7] checking free space tree
...

---
 check/main.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
check/main.c +19 -0
index 540130b8e223..2dea2acf5104 100644
@@ -5447,6 +5447,25 @@ static int process_extent_item(struct btrfs_root *root,
 	if (metadata)
 		btrfs_check_subpage_eb_alignment(gfs_info, key.objectid, num_bytes);
 
+	ptr = (unsigned long)(ei + 1);
+	if (metadata) {
+		u64 level;
+
+		if (key.type == BTRFS_EXTENT_ITEM_KEY) {
+			struct btrfs_tree_block_info *info;
+
+			info = (struct btrfs_tree_block_info *)ptr;
+			level = btrfs_tree_block_level(eb, info);
+		} else {
+			level = key.offset;
+		}
+		if (level >= BTRFS_MAX_LEVEL) {
+			error(
+	"tree block %llu has bad backref level, has %llu expect [0, %u]",
+			      key.objectid, level, BTRFS_MAX_LEVEL - 1);
+			return -EIO;
+		}
+	}
 	memset(&tmpl, 0, sizeof(tmpl));
 	tmpl.start = key.objectid;
 	tmpl.nr = num_bytes;
Signature · 1 line
2.34.1
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] btrfs-progs: check/lowmem: fix crash when METADATA_ITEM has invalid level

In reply to Qu Wenruo
Quoted 67 lines · Qu Wenruo

[BUG]
When running lowmem mode with METADATA_ITEM which has invalid level, it will crash with the following backtrace:

(gdb) bt
#0 0x0000555555616b0b in btrfs_header_bytenr (eb=0x4)

     at ./kernel-shared/ctree.h:2134
 #1  0x0000555555620c78 in check_tree_block_backref (root_id=5,
     bytenr=30457856, level=256) at check/mode-lowmem.c:3818
 #2  0x0000555555621f6c in check_extent_item 
 (path=0x7fffffffd9c0)
     at check/mode-lowmem.c:4334
 #3  0x00005555556235a5 in check_leaf_items 
 (root=0x555555688e10,
     path=0x7fffffffd9c0, nrefs=0x7fffffffda30, account_bytes=1)
     at check/mode-lowmem.c:4835
 #4  0x0000555555623c6d in walk_down_tree (root=0x555555688e10,
     path=0x7fffffffd9c0, level=0x7fffffffd984, 
     nrefs=0x7fffffffda30,
     check_all=1) at check/mode-lowmem.c:4967
 #5  0x000055555562494f in check_btrfs_root 
 (root=0x555555688e10, check_all=1)
     at check/mode-lowmem.c:5266
 #6  0x00005555556254ee in check_chunks_and_extents_lowmem ()
     at check/mode-lowmem.c:5556
 #7  0x00005555555f0b82 in do_check_chunks_and_extents () at 
 check/main.c:9114
 #8  0x00005555555f50ea in cmd_check (cmd=0x55555567c640 
 <cmd_struct_check>,
     argc=3, argv=0x7fffffffdec0) at check/main.c:10892
 #9  0x000055555556b2b1 in cmd_execute (argv=0x7fffffffdec0, 
 argc=3,
     cmd=0x55555567c640 <cmd_struct_check>) at 
     cmds/commands.h:125

[CAUSE]
For function check_extent_item() it will go through inline extent items and then check their backrefs.

But for METADATA_ITEM, it doesn't really validate key.offset, which is u64 and can contain value way larger than BTRFS_MAX_LEVEL (mostly caused by bit flip).

In that case, if we have a larger value like 256 in key.offset, then later check_tree_block_backref() will use 256 as level, and overflow path->nodes[level] and crash.

[FIX]
Just verify the level, no matter if it's from btrfs_tree_block_level() (which is just u8), or it's from key.offset (which is u64).

To do the check properly and detect higher bits corruption, also change the type of @level from u8 to u64.

Now lowmem mode can detect the problem properly:

...
[2/7] checking extents
ERROR: tree block 30457856 has bad backref level, has 256 expect [0, 7]
ERROR: extent[30457856 16384] level mismatch, wanted: 0, have:
256
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree
...

Quoted 2 lines

--
Su

Quoted 38 lines
---
 check/mode-lowmem.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
check/mode-lowmem.c +1 -1
index cc6773cd4d0c..99f519631a50 100644
@@ -4242,7 +4242,7 @@ static int check_extent_item(struct 
 btrfs_path *path)
 	u64 owner_offset;
 	u64 super_gen;
 	int metadata = 0;
-	int level;
+	u64 level;	/* To handle corrupted values in skinny 

backref */

check/mode-lowmem.c +10 -0
 	struct btrfs_key key;
 	int ret;
 	int err = 0;
@@ -4303,6 +4303,16 @@ static int check_extent_item(struct 
 btrfs_path *path)
 		/* New METADATA_ITEM */
 		level = key.offset;
 	}
+
+	if (metadata && level >= BTRFS_MAX_LEVEL) {
+		error(
+	"tree block %llu has bad backref level, has %llu expect 
 [0, %u]",
+		      key.objectid, level, BTRFS_MAX_LEVEL - 1);
+		err |= BACKREF_MISMATCH;
+		/* This is a critical error, exit right now */
+		goto out;
+	}
+
 	ptr_offset = ptr - (unsigned long)ei;

next:

Reviewed-by on v1 2/3
#4 lore raw

Re: [PATCH 2/3] btrfs: check/original: reject bad metadata backref with invalid level

In reply to Qu Wenruo
Quoted 70 lines · Qu Wenruo

[BUG]
There is a bug report that kernel tree-checker rejected an invalid metadata item:

corrupt leaf: block=934474399744 slot=68 extent bytenr=425173254144 len=16384 invalid tree level, have 33554432 expect [0, 7]

But original mode btrfs-check reports nothing wrong. (BTW, lowmem mode will just crash, and fixed in previous patch).

[CAUSE]
For original mode it doesn't really check tree level, thus didn't find the problem.

[FIX]
I don't have a good idea to completely make original mode to verify the level in backref and in the tree block (while lowmem does that).

But at least we can detect obviouly corrupted level just like kernel.

Now original mode will detect such problem:

...
[2/7] checking extents
ERROR: tree block 30457856 has bad backref level, has 256 expect [0, 7]
ref mismatch on [30457856 16384] extent item 0, found 1 tree backref 30457856 root 5 not found in extent tree backpointer mismatch on [30457856 16384]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree
...

---
 check/main.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
check/main.c +15 -0
index 540130b8e223..2dea2acf5104 100644
@@ -5447,6 +5447,25 @@ static int process_extent_item(struct 
 btrfs_root *root,
 	if (metadata)
 		btrfs_check_subpage_eb_alignment(gfs_info, 
 key.objectid, num_bytes);
 
+	ptr = (unsigned long)(ei + 1);
+	if (metadata) {
+		u64 level;
+
+		if (key.type == BTRFS_EXTENT_ITEM_KEY) {
+			struct btrfs_tree_block_info *info;
+
+			info = (struct btrfs_tree_block_info 
 *)ptr;
+			level = btrfs_tree_block_level(eb, info);
+		} else {
+			level = key.offset;
+		}
+		if (level >= BTRFS_MAX_LEVEL) {
+			error(
+	"tree block %llu has bad backref level, has %llu expect 

[0, %u]",

check/main.c

Quoted 3 lines
check/main.c +2 -1
+			      key.objectid, level, BTRFS_MAX_LEVEL 
- 1);
+			return -EIO;

-EUCLEAN is better?

--
Su

Quoted 5 lines
check/main.c +2 -0
+		}
+	}
 	memset(&tmpl, 0, sizeof(tmpl));
 	tmpl.start = key.objectid;
 	tmpl.nr = num_bytes;
#5 lore raw

Re: [PATCH 2/3] btrfs: check/original: reject bad metadata backref with invalid level

Quoted 74 lines · Su Yue
Quoted 71 lines · Qu Wenruo

[BUG]
There is a bug report that kernel tree-checker rejected an invalid
metadata item:

corrupt leaf: block=934474399744 slot=68 extent bytenr=425173254144 len=16384 invalid tree level, have 33554432 expect [0, 7]

But original mode btrfs-check reports nothing wrong. (BTW, lowmem mode will just crash, and fixed in previous patch).

[CAUSE]
For original mode it doesn't really check tree level, thus didn't find
the problem.

[FIX]
I don't have a good idea to completely make original mode to verify the
level in backref and in the tree block (while lowmem does that).

But at least we can detect obviouly corrupted level just like kernel.

Now original mode will detect such problem:

...
[2/7] checking extents
ERROR: tree block 30457856 has bad backref level, has 256 expect [0, 7]
ref mismatch on [30457856 16384] extent item 0, found 1 tree backref 30457856 root 5 not found in extent tree backpointer mismatch on [30457856 16384]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree
...

Link:
https://lore.kernel.org/linux-btrfs/[email protected]/

---
 check/main.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
check/main.c +13 -0
index 540130b8e223..2dea2acf5104 100644
@@ -5447,6 +5447,25 @@ static int process_extent_item(struct 
 btrfs_root *root,
 	if (metadata)
 		btrfs_check_subpage_eb_alignment(gfs_info, 
 key.objectid, num_bytes);
 
+	ptr = (unsigned long)(ei + 1);
+	if (metadata) {
+		u64 level;
+
+		if (key.type == BTRFS_EXTENT_ITEM_KEY) {
+			struct btrfs_tree_block_info *info;
+
+			info = (struct btrfs_tree_block_info 
 *)ptr;
+			level = btrfs_tree_block_level(eb, info);
+		} else {
+			level = key.offset;
+		}
+		if (level >= BTRFS_MAX_LEVEL) {
Quoted 8 lines

check/main.c

Quoted 7 lines
check/main.c +2 -0
+			error(
+	"tree block %llu has bad backref level, has %llu expect 

[0, %u]",

check/main.c +2 -1
+			      key.objectid, level, BTRFS_MAX_LEVEL 
- 1);
+			return -EIO;

-EUCLEAN is better?

Yes, updated in patch, thanks.

#7 lore raw
In reply to Qu Wenruo
Quoted 15 lines · Qu Wenruo

There is a report that kernel tree-checker rejects a tree block of extent tree, as it contains an obvious corrupted level (which is an obvious bit flip).

But btrfs check, at least original mode, doesn't detect it at all. While with my crafted image, lowmem mode would just crash due to the large level value overflowing path->nodes[level].

Lowmem is enhanced to reject such level, and the existing code will verify the level and report errors.

Original mode is more tricky, as it doesn't have level check at all. I don't have a good idea to implement full level check at original mode, so here I just introduced a basic check for tree level and reject it.

Quoted 8 lines

Finally introduce a test case for this.

Qu Wenruo (3):
btrfs-progs: check/lowmem: fix crash when METADATA_ITEM has invalid
level
btrfs: check/original: reject bad metadata backref with invalid level
btrfs-progs: tests/fsck: add test image with invalid metadata backref
level

Added to devel, thanks.