Skip to contentSkip to conversation

btrfs-progs: mkfs: extent-tree-v2 related fixes

PATCH In review Qu Wenruo 3 participants 6 messages Oct 7, 2022 – Oct 10, 2022 Btrfs file system

lore
Share & tools
lore
author
#3 lore raw

PATCH 2/2 btrfs-progs: mkfs: fix a stack over-flow when features string are too long

In reply to Qu Wenruo

[BUG]
Even with chunk_objectid bug fixed, mkfs.btrfs can still caused stack overflow when enabling extent-tree-v2 feature (need experimental features enabled):

  # ./mkfs.btrfs  -f -O extent-tree-v2 ~/test.img
  btrfs-progs v5.19.1
  See http://btrfs.wiki.kernel.org for more information.

ERROR: superblock magic doesn't match
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)

  Label:              (null)
  UUID:               205c61e7-f58e-4e8f-9dc2-38724f5c554b
  Node size:          16384
  Sector size:        4096
  Filesystem size:    512.00MiB
  Block group profiles:
    Data:             single            8.00MiB
    Metadata:         DUP              32.00MiB
    System:           DUP               8.00MiB
  SSD detected:       no
  Zoned device:       no

=================================================================
[... Skip full ASAN output ...]
==65655==ABORTING

[CAUSE]
For experimental build, we have unified feature output, but the old buffer size is only 64 bytes, which is too small to cover the new full feature string:

extref, skinny-metadata, no-holes, free-space-tree, block-group-tree, extent-tree-v2

Above feature string is already 84 bytes, over the 64 on-stack memory size.

This can also be proved by the ASAN output:

==65655==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffc4e03b1d0 at pc 0x7ff0fc05fafe bp 0x7ffc4e03ac60 sp 0x7ffc4e03a408 WRITE of size 17 at 0x7ffc4e03b1d0 thread T0

      #0 0x7ff0fc05fafd in __interceptor_strcat /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:377
      #1 0x55cdb7b06ca5 in parse_features_to_string common/fsfeatures.c:316
      #2 0x55cdb7b06ce1 in btrfs_parse_fs_features_to_string common/fsfeatures.c:324
      #3 0x55cdb7a37226 in main mkfs/main.c:1783
      #4 0x7ff0fbe3c28f  (/usr/lib/libc.so.6+0x2328f)
      #5 0x7ff0fbe3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
      #6 0x55cdb7a2cb34 in _start ../sysdeps/x86_64/start.S:115

[FIX]
Introduce a new macro, BTRFS_FEATURE_STRING_BUF_SIZE, along with a new sanity check helper, btrfs_assert_feature_buf_size().

The problem is I can not find a build time method to verify BTRFS_FEATURE_STRING_BUF_SIZE is large enough to contain all feature names, thus have to go the runtime function to do the BUG_ON() to verify the macro size.

Now the minimal buffer size for experimental build is 138 bytes, just bump it to 160 for future expansion.

And if further features go beyond that number, mkfs.btrfs/btrfs-convert will immediately crash at that BUG_ON(), so we can definitely detect it.

---
 common/fsfeatures.c | 26 ++++++++++++++++++++++++++
 common/fsfeatures.h |  7 +++++++
 convert/main.c      |  3 ++-
 mkfs/main.c         |  3 ++-
 4 files changed, 37 insertions(+), 2 deletions(-)
common/fsfeatures.c +26 -0
index dc4b346c040a..e4334e3ea6c0 100644
@@ -251,6 +251,32 @@ static const struct btrfs_feature runtime_features[] = {
 	}
 };
 
+/*
+ * This is a sanity check to make sure BTRFS_FEATURE_STRING_BUF_SIZE is large
+ * enough to contain all strings.
+ *
+ * All callers using btrfs_parse_*_features_to_string() should call this first.
+ */
+void btrfs_assert_feature_buf_size(void)
+{
+	int total_size = 0;
+	int i;
+
+	/*
+	 * This is a little over-calculated, as we include ", list-all".
+	 * But 10 extra bytes should not be a big deal.
+	 */
+	for (i = 0; i < ARRAY_SIZE(mkfs_features); i++)
+		/* The extra 2 bytes are for the ", " prefix. */
+		total_size += strlen(mkfs_features[i].name) + 2;
+	BUG_ON(BTRFS_FEATURE_STRING_BUF_SIZE < total_size);
+
+	total_size = 0;
+	for (i = 0; i < ARRAY_SIZE(runtime_features); i++)
+		total_size += strlen(runtime_features[i].name) + 2;
+	BUG_ON(BTRFS_FEATURE_STRING_BUF_SIZE < total_size);
+}
+
 static size_t get_feature_array_size(enum feature_source source)
 {
 	if (source == FS_FEATURES)
common/fsfeatures.h +7 -0
index 3b5a915c6012..c4ab704862cd 100644
@@ -37,6 +37,12 @@ struct btrfs_mkfs_features {
 #define BTRFS_FEATURE_RUNTIME_QUOTA		(1ULL << 0)
 #define BTRFS_FEATURE_RUNTIME_LIST_ALL		(1ULL << 1)
 
+/*
+ * Such buffer size should be able to contain all feature string, with extra
+ * ", " for each feature.
+ */
+#define BTRFS_FEATURE_STRING_BUF_SIZE		(160)
+
 static const struct btrfs_mkfs_features btrfs_mkfs_default_features = {
 	.compat_ro_flags = BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE |
 			   BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID,
@@ -86,5 +92,6 @@ int btrfs_check_sectorsize(u32 sectorsize);
 int btrfs_check_features(const struct btrfs_mkfs_features *features,
 			 const struct btrfs_mkfs_features *allowed);
 int btrfs_tree_search2_ioctl_supported(int fd);
+void btrfs_assert_feature_buf_size(void);
 
 #endif
convert/main.c +2 -1
index 6bcb0f4876d0..c7be19f4e9bd 100644
@@ -1147,7 +1147,7 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
 	struct btrfs_key key;
 	char subvol_name[SOURCE_FS_NAME_LEN + 8];
 	struct task_ctx ctx;
-	char features_buf[64];
+	char features_buf[BTRFS_FEATURE_STRING_BUF_SIZE];
 	char fsid_str[BTRFS_UUID_UNPARSED_SIZE];
 	struct btrfs_mkfs_config mkfs_cfg;
 	bool btrfs_sb_committed = false;
@@ -1835,6 +1835,7 @@ int BOX_MAIN(convert)(int argc, char *argv[])
 	char fsid[BTRFS_UUID_UNPARSED_SIZE] = {0};
 
 	crc32c_optimization_init();
+	btrfs_assert_feature_buf_size();
 	printf("btrfs-convert from %s\n\n", PACKAGE_STRING);
 
 	while(1) {
mkfs/main.c +2 -1
index e5c1aa669828..c4a4e1986f9b 100644
@@ -1028,6 +1028,7 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 
 	crc32c_optimization_init();
 	btrfs_config_init();
+	btrfs_assert_feature_buf_size();
 
 	while(1) {
 		int c;
@@ -1750,7 +1751,7 @@ raid_groups:
 		}
 	}
 	if (bconf.verbose) {
-		char features_buf[64];
+		char features_buf[BTRFS_FEATURE_STRING_BUF_SIZE];
 
 		update_chunk_allocation(fs_info, &allocation);
 		printf("Label:              %s\n", label);
Signature · 1 line
2.37.3

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: mkfs: extent-tree-v2 related fixes

Although recently we still have some uncertainty around the on-disk format for extent-tree-v2, related to how to determine the number of global roots, most of the on-disk format is fixed.

And even with the uncertain part involved, mkfs.btrfs should not crash for extent-tree-v2 feature (hidden behind the experimental builds).

There are two bugs involved:

- A crash caused by incorrectly set chunk_objectid for block group item As extent-tree-v2 feature reuse that member to indicate which extent tree a block group belongs to.

But the regular fs uses a fixed 256 for that chunk_objectid, and no extent-tree-v2 btrfs would have that many global roots.

This leads to btrfs_extent_root() to return NULL, and cause later segfault.

Fix it by properly setting chunk_objectid.
This is a regression caused by 1430b41427b5 ("btrfs-progs: separate block group tree from extent tree v2").

- A stack-over-flow caused by too long feature string
With extent-tree-v2 enabled, we have at least 84 bytes long feature string (unified features, including compat_ro features likle fst).

This is beyond the hard-coded 64 bytes limit.

Fix it by introducing a new macro to indicate a minimal safe buf size, and a sanity check to make sure that macro is really large enough.

Qu Wenruo (2):
btrfs-progs: mkfs: fix a crash when enabling extent-tree-v2
btrfs-progs: mkfs: fix a stack over-flow when features string are too
long

 common/fsfeatures.c | 26 ++++++++++++++++++++++++++
 common/fsfeatures.h |  7 +++++++
 convert/main.c      |  3 ++-
 mkfs/common.c       | 14 ++++++++++++--
 mkfs/main.c         |  3 ++-
 5 files changed, 49 insertions(+), 4 deletions(-)
Signature · 1 line
2.37.3
author
#1 lore raw

PATCH 1/2 btrfs-progs: mkfs: fix a crash when enabling extent-tree-v2

[BUG]
When enabling extent-tree-v2 feature at mkfs time (need to enable experimental features), mkfs.btrfs will crash:

  # ./mkfs.btrfs  -f -O extent-tree-v2 ~/test.img
  btrfs-progs v5.19.1
  See http://btrfs.wiki.kernel.org for more information.

ERROR: superblock magic doesn't match
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)

Segmentation fault (core dumped)

[CAUSE]
The block group tree looks like this after make_btrfs() call:

(gdb) call btrfs_print_tree(root->fs_info->block_group_root->node, 0) leaf 1163264 items 1 free space 16234 generation 1 owner BLOCK_GROUP_TREE leaf 1163264 flags 0x0() backref revision 1
checksum stored f137c1ac
checksum calced f137c1ac
fs uuid 450d4b15-4954-4574-9801-8c6d248aaec6
chunk uuid 4c4cc54d-f240-4aa4-b88b-bd487db43444

	item 0 key (1048576 BLOCK_GROUP_ITEM 4194304) itemoff 16259 itemsize 24
		block group used 131072 chunk_objectid 256 flags SYSTEM|single
						       ^^^

This looks completely sane, but notice that chunk_objectid 256. That 256 value is the expected one for regular non-extent-tree-v2 btrfs, but for extent-tree-v2, chunk_objectid is reused as the global id of extent tree where the block group belongs to.

With the old 256 value as chunk_objectid, btrfs will not find an extent tree root for the block group, and return NULL for btrfs_extent_root() call, and trigger segfault.

This is a regression caused by commit 1430b41427b5 ("btrfs-progs:
separate block group tree from extent tree v2"), which doesn't take extent-tree-v2 on-disk format into consideration.

[FIX]
For the initial btrfs created by make_btrfs(), all block group items will be in extent-tree global id 0, thus we can reset chunk_objectid to 0, if and only if extent-tree-v2 is enabled.

---
 mkfs/common.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
mkfs/common.c +12 -2
index 3a517a503e61..d77688ba584d 100644
@@ -227,12 +227,22 @@ static int create_block_group_tree(int fd, struct btrfs_mkfs_config *cfg,
 				   u64 bg_offset, u64 bg_size, u64 bg_used)
 {
 	int ret;
+	u64 chunk_objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
+
+	/*
+	 * For extent-tree-v2, chunk_objectid of block group item is reused
+	 * to indicate which extent-tree the block group is in.
+	 *
+	 * Thus for the initial image, we should set the chunk_objectid to 0,
+	 * as all initial bgs are in the extent tree with global id 0.
+	 */
+	if (cfg->features.incompat_flags & BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2)
+		chunk_objectid = 0;
 
 	memset(buf->data + sizeof(struct btrfs_header), 0,
 		cfg->nodesize - sizeof(struct btrfs_header));
 	write_block_group_item(buf, 0, bg_offset, bg_size, bg_used,
-			       BTRFS_FIRST_CHUNK_TREE_OBJECTID,
-			       cfg->leaf_data_size -
+			       chunk_objectid, cfg->leaf_data_size -
 			       sizeof(struct btrfs_block_group_item));
 	btrfs_set_header_bytenr(buf, cfg->blocks[MKFS_BLOCK_GROUP_TREE]);
 	btrfs_set_header_owner(buf, BTRFS_BLOCK_GROUP_TREE_OBJECTID);
Signature · 1 line
2.37.3
Reviewed-by Tested-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] btrfs-progs: mkfs: fix a crash when enabling extent-tree-v2

Quoted 44 lines · Qu Wenruo

[BUG]
When enabling extent-tree-v2 feature at mkfs time (need to enable experimental features), mkfs.btrfs will crash:

   # ./mkfs.btrfs  -f -O extent-tree-v2 ~/test.img
   btrfs-progs v5.19.1
   See http://btrfs.wiki.kernel.org for more information.

ERROR: superblock magic doesn't match
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)

Segmentation fault (core dumped)

[CAUSE]
The block group tree looks like this after make_btrfs() call:

   (gdb) call btrfs_print_tree(root->fs_info->block_group_root->node, 0)
   leaf 1163264 items 1 free space 16234 generation 1 owner BLOCK_GROUP_TREE
   leaf 1163264 flags 0x0() backref revision 1
   checksum stored f137c1ac
   checksum calced f137c1ac
   fs uuid 450d4b15-4954-4574-9801-8c6d248aaec6
   chunk uuid 4c4cc54d-f240-4aa4-b88b-bd487db43444
	item 0 key (1048576 BLOCK_GROUP_ITEM 4194304) itemoff 16259 itemsize 24
		block group used 131072 chunk_objectid 256 flags SYSTEM|single
						       ^^^

This looks completely sane, but notice that chunk_objectid 256. That 256 value is the expected one for regular non-extent-tree-v2 btrfs, but for extent-tree-v2, chunk_objectid is reused as the global id of extent tree where the block group belongs to.

With the old 256 value as chunk_objectid, btrfs will not find an extent tree root for the block group, and return NULL for btrfs_extent_root() call, and trigger segfault.

This is a regression caused by commit 1430b41427b5 ("btrfs-progs:
separate block group tree from extent tree v2"), which doesn't take extent-tree-v2 on-disk format into consideration.

Quoted 6 lines

[FIX]
For the initial btrfs created by make_btrfs(), all block group items will be in extent-tree global id 0, thus we can reset chunk_objectid to 0, if and only if extent-tree-v2 is enabled.

Reviewed-by Tested-by on v1 2/2
#4 lore raw

Re: [PATCH 2/2] btrfs-progs: mkfs: fix a stack over-flow when features string are too long

In reply to Qu Wenruo
Quoted 62 lines · Qu Wenruo

[BUG]
Even with chunk_objectid bug fixed, mkfs.btrfs can still caused stack overflow when enabling extent-tree-v2 feature (need experimental features enabled):

   # ./mkfs.btrfs  -f -O extent-tree-v2 ~/test.img
   btrfs-progs v5.19.1
   See http://btrfs.wiki.kernel.org for more information.

ERROR: superblock magic doesn't match
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)

   Label:              (null)
   UUID:               205c61e7-f58e-4e8f-9dc2-38724f5c554b
   Node size:          16384
   Sector size:        4096
   Filesystem size:    512.00MiB
   Block group profiles:
     Data:             single            8.00MiB
     Metadata:         DUP              32.00MiB
     System:           DUP               8.00MiB
   SSD detected:       no
   Zoned device:       no

=================================================================
[... Skip full ASAN output ...]
==65655==ABORTING

[CAUSE]
For experimental build, we have unified feature output, but the old buffer size is only 64 bytes, which is too small to cover the new full feature string:

extref, skinny-metadata, no-holes, free-space-tree, block-group-tree, extent-tree-v2

Above feature string is already 84 bytes, over the 64 on-stack memory size.

This can also be proved by the ASAN output:

   ==65655==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffc4e03b1d0 at pc 0x7ff0fc05fafe bp 0x7ffc4e03ac60 sp 0x7ffc4e03a408
   WRITE of size 17 at 0x7ffc4e03b1d0 thread T0
       #0 0x7ff0fc05fafd in __interceptor_strcat /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:377
       #1 0x55cdb7b06ca5 in parse_features_to_string common/fsfeatures.c:316
       #2 0x55cdb7b06ce1 in btrfs_parse_fs_features_to_string common/fsfeatures.c:324
       #3 0x55cdb7a37226 in main mkfs/main.c:1783
       #4 0x7ff0fbe3c28f  (/usr/lib/libc.so.6+0x2328f)
       #5 0x7ff0fbe3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
       #6 0x55cdb7a2cb34 in _start ../sysdeps/x86_64/start.S:115

[FIX]
Introduce a new macro, BTRFS_FEATURE_STRING_BUF_SIZE, along with a new sanity check helper, btrfs_assert_feature_buf_size().

The problem is I can not find a build time method to verify BTRFS_FEATURE_STRING_BUF_SIZE is large enough to contain all feature names, thus have to go the runtime function to do the BUG_ON() to verify the macro size.

Quoted 7 lines

Now the minimal buffer size for experimental build is 138 bytes, just bump it to 160 for future expansion.

And if further features go beyond that number, mkfs.btrfs/btrfs-convert will immediately crash at that BUG_ON(), so we can definitely detect it.

#5 lore raw
In reply to Qu Wenruo
Quoted 29 lines · Qu Wenruo

Although recently we still have some uncertainty around the on-disk format for extent-tree-v2, related to how to determine the number of global roots, most of the on-disk format is fixed.

And even with the uncertain part involved, mkfs.btrfs should not crash for extent-tree-v2 feature (hidden behind the experimental builds).

There are two bugs involved:

- A crash caused by incorrectly set chunk_objectid for block group item As extent-tree-v2 feature reuse that member to indicate which extent tree a block group belongs to.

But the regular fs uses a fixed 256 for that chunk_objectid, and no extent-tree-v2 btrfs would have that many global roots.

This leads to btrfs_extent_root() to return NULL, and cause later segfault.

Fix it by properly setting chunk_objectid.
This is a regression caused by 1430b41427b5 ("btrfs-progs: separate block group tree from extent tree v2").

- A stack-over-flow caused by too long feature string
With extent-tree-v2 enabled, we have at least 84 bytes long feature string (unified features, including compat_ro features likle fst).

This is beyond the hard-coded 64 bytes limit.

Quoted 7 lines

Fix it by introducing a new macro to indicate a minimal safe buf size, and a sanity check to make sure that macro is really large enough.

Qu Wenruo (2):
btrfs-progs: mkfs: fix a crash when enabling extent-tree-v2
btrfs-progs: mkfs: fix a stack over-flow when features string are too
long

Added to devel, thanks.