Skip to contentSkip to conversation

btrfs-progs: almost full support for RAID56J profiles

PATCH In review Qu Wenruo 3 participants 8 messages May 15, 2022 – May 17, 2022 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: almost full support for RAID56J profiles

This is the progs companion for the new RAID56J profiles.

Unlike kernel part, progs doesn't really need to implement the full journal, thus the following basic features should be enough:

- Mkfs support
- Check support (both original and lowmem mode)
- Print tree support

The final patch is a fix for a leakage of path which is exposed during kernel development.

Qu Wenruo (5):
btrfs-progs: introduce the basic support for RAID56J feature
btrfs-progs: mkfs: add support for RAID56J creation
btrfs-progs: check: take per device reservation into consideration
btrfs-progs: print-tree: add support for per_dev_reserved of chunk
item
btrfs-progs: check/lowmem: fix path leakage when dev extents are
invalid

 check/common.h              |   7 ++-
 check/main.c                |  16 ++++--
 check/mode-lowmem.c         |  17 ++++--
 cmds/filesystem-usage.c     |   6 ++-
 cmds/rescue-chunk-recover.c |  13 +++--
 common/fsfeatures.c         |   9 ++++
 common/utils.c              |   6 ++-
 kernel-shared/ctree.h       |  42 +++++++++++++--
 kernel-shared/extent-tree.c |  18 +++++--
 kernel-shared/print-tree.c  |   5 +-
 kernel-shared/volumes.c     | 105 +++++++++++++++++++++++++++++++-----
 kernel-shared/volumes.h     |   2 +
 mkfs/main.c                 |   3 ++
 13 files changed, 205 insertions(+), 44 deletions(-)
Signature · 1 line
2.36.1
author
#2 lore raw

PATCH 2/5 btrfs-progs: mkfs: add support for RAID56J creation

The major part is already done in the RAID56J feature introduce commit, for mkfs the only special part is about setting the BTRFS_FEATURE_INCOMPAT_RAID56_JOURNAL flag and the extra incompat flags.

Unlike kernel, btrfs-progs doesn't automatically set the flag based on the chunk type, but has to do it at mkfs time.

---
 common/fsfeatures.c | 9 +++++++++
 mkfs/main.c         | 3 +++
 2 files changed, 12 insertions(+)
common/fsfeatures.c +9 -0
index 23a92c21a2cc..86637606e6af 100644
@@ -142,6 +142,15 @@ static const struct btrfs_feature mkfs_features[] = {
 		VERSION_NULL(default),
 		.desc		= "new extent tree format"
 	},
+	{
+		.name		= "raid56-journal",
+		.flag		= BTRFS_FEATURE_INCOMPAT_RAID56_JOURNAL,
+		.sysfs_name	= "raid56_journal",
+		VERSION_TO_STRING2(compat, 6,10),
+		VERSION_NULL(safe),
+		VERSION_NULL(default),
+		.desc		= "write-ahead journal for RAID56"
+	},
 #endif
 	/* Keep this one last */
 	{
mkfs/main.c +3 -0
index 4e0a46a77aa5..1187440c8db2 100644
@@ -1286,6 +1286,9 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 			"\t to be used besides testing or evaluation.\n");
 	}
 
+	if ((data_profile | metadata_profile) & BTRFS_BLOCK_GROUP_JOURNAL_MASK)
+		features |= BTRFS_FEATURE_INCOMPAT_RAID56_JOURNAL;
+
 	if ((data_profile | metadata_profile) &
 	    (BTRFS_BLOCK_GROUP_RAID1C3 | BTRFS_BLOCK_GROUP_RAID1C4)) {
 		features |= BTRFS_FEATURE_INCOMPAT_RAID1C34;
Signature · 1 line
2.36.1
author
#1 lore raw

PATCH 1/5 btrfs-progs: introduce the basic support for RAID56J feature

In reply to Qu Wenruo

This patch will cross-port the RAID56J feature from the WIP kernel patch to btrfs-progs, allowing us to create a fs with RAID56J.

The RAID56J feature itself is pretty much the same as regular RAID56, with extra btrfs_chunk::per_dev_reserved bytes reserved for each stripe.

The reserved space will be used for write-ahead journal to address the write-hole problem.

Thankfully for btrfs-progs, there isn't much need to fully implement the journal yet.

This patch will just allow chunk allocation/deletion to take the extra reservation into consideration.

And the new feature will only be enabled with experimental feature.

---
 check/common.h              |   6 ++-
 cmds/filesystem-usage.c     |   6 ++-
 cmds/rescue-chunk-recover.c |  13 +++--
 common/utils.c              |   6 ++-
 kernel-shared/ctree.h       |  42 +++++++++++++--
 kernel-shared/extent-tree.c |  18 +++++--
 kernel-shared/volumes.c     | 105 +++++++++++++++++++++++++++++++-----
 kernel-shared/volumes.h     |   2 +
 8 files changed, 162 insertions(+), 36 deletions(-)
check/common.h +4 -2
index ba4e291e8d0d..f6e6eece37aa 100644
@@ -133,9 +133,11 @@ static inline int check_num_stripes(u64 type, int num_stripes)
 {
 	if (num_stripes == 0)
 		return -1;
-	if (type & BTRFS_BLOCK_GROUP_RAID5 && num_stripes <= 1)
+	if (type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID5J) &&
+	    num_stripes <= 1)
 		return -1;
-	if (type & BTRFS_BLOCK_GROUP_RAID6 && num_stripes <= 2)
+	if (type & (BTRFS_BLOCK_GROUP_RAID6 | BTRFS_BLOCK_GROUP_RAID6J) &&
+	    num_stripes <= 2)
 		return -1;
 	return 0;
 }
cmds/filesystem-usage.c +4 -2
index 01729e1886ac..4bdb07eeba86 100644
@@ -356,11 +356,13 @@ static void get_raid56_space_info(struct btrfs_ioctl_space_args *sargs,
 		double l_data_ratio, l_metadata_ratio, l_system_ratio, rt;
 
 		parities_count = btrfs_bg_type_to_nparity(info_ptr->type);
-		if (info_ptr->type & BTRFS_BLOCK_GROUP_RAID5) {
+		if ((BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID5J) &
+		    info_ptr->type) {
 			l_data_ratio = l_data_ratio_r5;
 			l_metadata_ratio = l_metadata_ratio_r5;
 			l_system_ratio = l_system_ratio_r5;
-		} else if (info_ptr->type & BTRFS_BLOCK_GROUP_RAID6) {
+		} else if ((BTRFS_BLOCK_GROUP_RAID6 | BTRFS_BLOCK_GROUP_RAID6J) &
+		    info_ptr->type) {
 			l_data_ratio = l_data_ratio_r6;
 			l_metadata_ratio = l_metadata_ratio_r6;
 			l_system_ratio = l_system_ratio_r6;
cmds/rescue-chunk-recover.c +6 -7
index ec5c206f85e7..67a7bd595b5d 100644
@@ -2093,8 +2093,8 @@ next_csum:
 
 	if (list_empty(&candidates)) {
 		num_unordered = count_devext_records(&unordered);
-		if (chunk->type_flags & BTRFS_BLOCK_GROUP_RAID6
-					&& num_unordered == 2) {
+		if ((BTRFS_BLOCK_GROUP_RAID6 | BTRFS_BLOCK_GROUP_RAID6J) &
+		    chunk->type && num_unordered == 2) {
 			btrfs_release_path(&path);
 			ret = fill_chunk_up(chunk, &unordered, rc);
 			return ret;
@@ -2139,12 +2139,11 @@ out:
 		if (ret)
 			goto fail_out;
 	} else {
-		if ((num_unordered == 2 && chunk->type_flags
-			& BTRFS_BLOCK_GROUP_RAID5)
-		 || (num_unordered == 3 && chunk->type_flags
-			& BTRFS_BLOCK_GROUP_RAID6)) {
+		if ((num_unordered == 2 && chunk->type_flags &
+		    (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID5J))
+		 || (num_unordered == 3 && chunk->type_flags &
+		    (BTRFS_BLOCK_GROUP_RAID6 | BTRFS_BLOCK_GROUP_RAID6J)))
 			ret = fill_chunk_up(chunk, &unordered, rc);
-		}
 	}
 fail_out:
 	ret = !!ret || (list_empty(&unordered) ? 0 : 1);
common/utils.c +4 -2
index 1ed5571f7c1c..e609cca50cde 100644
@@ -602,10 +602,12 @@ int test_num_disk_vs_raid(u64 metadata_profile, u64 data_profile,
 		return 1;
 	}
 
-	if (dev_cnt == 3 && profile & BTRFS_BLOCK_GROUP_RAID6) {
+	if (dev_cnt == 3 && profile &
+	    (BTRFS_BLOCK_GROUP_RAID6 | BTRFS_BLOCK_GROUP_RAID6J)) {
 		warning("RAID6 is not recommended on filesystem with 3 devices only");
 	}
-	if (dev_cnt == 2 && profile & BTRFS_BLOCK_GROUP_RAID5) {
+	if (dev_cnt == 2 && profile &
+	    (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID5J)) {
 		warning("RAID5 is not recommended on filesystem with 2 devices only");
 	}
 	warning_on(!mixed && (data_profile & BTRFS_BLOCK_GROUP_DUP) && ssd,
kernel-shared/ctree.h +37 -5

101 lines not on this page · raw message

kernel-shared/extent-tree.c +14 -4
index 697a8a1e4dec..92655fe32fb0 100644
@@ -3004,7 +3004,9 @@ static int free_chunk_dev_extent_items(struct btrfs_trans_handle *trans,
 	struct btrfs_root *root= fs_info->chunk_root;
 	struct btrfs_path *path;
 	struct btrfs_key key;
+	bool is_journal;
 	u16 num_stripes;
+	u32 per_dev_reserved = 0;
 	int i;
 	int ret;
 
@@ -3025,19 +3027,24 @@ static int free_chunk_dev_extent_items(struct btrfs_trans_handle *trans,
 	}
 	chunk = btrfs_item_ptr(path->nodes[0], path->slots[0],
 			       struct btrfs_chunk);
+	is_journal = btrfs_bg_type_is_journal(btrfs_chunk_type(path->nodes[0],
+					      chunk));
+	if (is_journal)
+		per_dev_reserved = btrfs_chunk_per_dev_reserved(path->nodes[0],
+								chunk);
 	num_stripes = btrfs_chunk_num_stripes(path->nodes[0], chunk);
 	for (i = 0; i < num_stripes; i++) {
 		u64 devid = btrfs_stripe_devid_nr(path->nodes[0], chunk, i);
 		u64 offset = btrfs_stripe_offset_nr(path->nodes[0], chunk, i);
 		u64 length = btrfs_stripe_length(fs_info, path->nodes[0], chunk);
 
+		ASSERT(offset > per_dev_reserved);
 		ret = btrfs_reset_chunk_zones(fs_info, devid, offset, length);
 		if (ret < 0)
 			goto out;
 
-		ret = free_dev_extent_item(trans, fs_info,
-			btrfs_stripe_devid_nr(path->nodes[0], chunk, i),
-			btrfs_stripe_offset_nr(path->nodes[0], chunk, i));
+		ret = free_dev_extent_item(trans, fs_info, devid,
+					   offset - per_dev_reserved);
 		if (ret < 0)
 			goto out;
 	}
@@ -3146,6 +3153,8 @@ static u64 get_dev_extent_len(struct map_lookup *map)
 		break;
 	case BTRFS_BLOCK_GROUP_RAID5:
 	case BTRFS_BLOCK_GROUP_RAID6:
+	case BTRFS_BLOCK_GROUP_RAID5J:
+	case BTRFS_BLOCK_GROUP_RAID6J:
 		div = map->num_stripes - btrfs_bg_type_to_nparity(map->type);
 		break;
 	case BTRFS_BLOCK_GROUP_RAID10:
@@ -3198,7 +3207,8 @@ static int free_block_group_cache(struct btrfs_trans_handle *trans,
 		struct btrfs_device *device;
 
 		device = map->stripes[i].dev;
-		device->bytes_used -= get_dev_extent_len(map);
+		device->bytes_used -= get_dev_extent_len(map) +
+				      map->per_dev_reserved;
 		ret = btrfs_update_device(trans, device);
 		if (ret < 0)
 			goto out;
kernel-shared/volumes.c +91 -14

246 lines not on this page · raw message

kernel-shared/volumes.h +2 -0
index 6e9103a933b7..e1bf0bbe2978 100644
@@ -120,6 +120,7 @@ struct map_lookup {
 	int sector_size;
 	int num_stripes;
 	int sub_stripes;
+	u32 per_dev_reserved;
 	struct btrfs_bio_stripe stripes[];
 };
 
@@ -315,5 +316,6 @@ int btrfs_bg_type_to_nparity(u64 flags);
 int btrfs_bg_type_to_sub_stripes(u64 flags);
 u64 btrfs_bg_flags_for_device_num(int number);
 bool btrfs_bg_type_is_stripey(u64 flags);
+bool btrfs_bg_type_is_journal(u64 flags);
 
 #endif
Signature · 1 line
2.36.1
author
#3 lore raw

PATCH 3/5 btrfs-progs: check: take per device reservation into consideration

In reply to Qu Wenruo

This patch will make both orginal and lowmem mode to take per device reservation into consideration for dev extent and chunk verification.

---
 check/common.h      |  1 +
 check/main.c        | 16 +++++++++++++---
 check/mode-lowmem.c | 15 +++++++++++----
 3 files changed, 25 insertions(+), 7 deletions(-)
check/common.h +1 -0
index f6e6eece37aa..051ffe65cb94 100644
@@ -75,6 +75,7 @@ struct chunk_record {
 	u16 sub_stripes;
 	u32 io_align;
 	u32 io_width;
+	u32 per_dev_reserved;
 	u32 sector_size;
 	struct stripe stripes[0];
 };
check/main.c +13 -3
index bcb016964e7a..d8834b2386d6 100644
@@ -5202,6 +5202,7 @@ struct chunk_record *btrfs_new_chunk_record(struct extent_buffer *leaf,
 	struct btrfs_chunk *ptr;
 	struct chunk_record *rec;
 	int num_stripes, i;
+	bool is_journal;
 
 	ptr = btrfs_item_ptr(leaf, slot, struct btrfs_chunk);
 	num_stripes = btrfs_chunk_num_stripes(leaf, ptr);
@@ -5225,12 +5226,19 @@ struct chunk_record *btrfs_new_chunk_record(struct extent_buffer *leaf,
 	rec->type = key->type;
 	rec->offset = key->offset;
 
+	is_journal = btrfs_bg_type_is_journal(btrfs_chunk_type(leaf, ptr));
 	rec->length = rec->cache.size;
 	rec->owner = btrfs_chunk_owner(leaf, ptr);
 	rec->stripe_len = btrfs_chunk_stripe_len(leaf, ptr);
 	rec->type_flags = btrfs_chunk_type(leaf, ptr);
 	rec->io_width = btrfs_chunk_io_width(leaf, ptr);
-	rec->io_align = btrfs_chunk_io_align(leaf, ptr);
+	if (is_journal) {
+		rec->io_align = rec->io_width;
+		rec->per_dev_reserved = btrfs_chunk_per_dev_reserved(leaf, ptr);
+	} else {
+		rec->io_align = btrfs_chunk_io_align(leaf, ptr);
+		rec->per_dev_reserved = 0;
+	}
 	rec->sector_size = btrfs_chunk_sector_size(leaf, ptr);
 	rec->num_stripes = num_stripes;
 	rec->sub_stripes = btrfs_chunk_sub_stripes(leaf, ptr);
@@ -8445,10 +8453,12 @@ static int check_chunk_refs(struct chunk_record *chunk_rec,
 		return ret;
 
 	length = calc_stripe_length(chunk_rec->type_flags, chunk_rec->length,
-				    chunk_rec->num_stripes);
+				    chunk_rec->num_stripes) +
+		 chunk_rec->per_dev_reserved;
 	for (i = 0; i < chunk_rec->num_stripes; ++i) {
 		devid = chunk_rec->stripes[i].devid;
-		offset = chunk_rec->stripes[i].offset;
+		offset = chunk_rec->stripes[i].offset - chunk_rec->per_dev_reserved;
+
 		dev_extent_item = lookup_cache_extent2(&dev_extent_cache->tree,
 						       devid, offset, length);
 		if (dev_extent_item) {
check/mode-lowmem.c +11 -4
index 68c1adfd04a4..63f9343ba2ff 100644
@@ -4435,6 +4435,7 @@ static int check_dev_extent_item(struct extent_buffer *eb, int slot)
 	struct extent_buffer *l;
 	int num_stripes;
 	u64 length;
+	u32 per_dev_reserved = 0;
 	int i;
 	int found_chunk = 0;
 	int ret;
@@ -4458,8 +4459,10 @@ static int check_dev_extent_item(struct extent_buffer *eb, int slot)
 				      chunk_key.offset);
 	if (ret < 0)
 		goto out;
+	if (btrfs_bg_type_is_journal(btrfs_chunk_type(l, chunk)))
+		per_dev_reserved = btrfs_chunk_per_dev_reserved(l, chunk);
 
-	if (btrfs_stripe_length(gfs_info, l, chunk) != length)
+	if (btrfs_stripe_length(gfs_info, l, chunk) != length - per_dev_reserved)
 		goto out;
 
 	num_stripes = btrfs_chunk_num_stripes(l, chunk);
@@ -4468,7 +4471,7 @@ static int check_dev_extent_item(struct extent_buffer *eb, int slot)
 		u64 offset = btrfs_stripe_offset_nr(l, chunk, i);
 
 		if (devid == devext_key.objectid &&
-		    offset == devext_key.offset) {
+		    offset == devext_key.offset + per_dev_reserved) {
 			found_chunk = 1;
 			break;
 		}
@@ -4648,6 +4651,7 @@ static int check_chunk_item(struct extent_buffer *eb, int slot)
 	struct btrfs_chunk *chunk;
 	struct extent_buffer *leaf;
 	struct btrfs_dev_extent *ptr;
+	u32 per_dev_reserved = 0;
 	u64 length;
 	u64 chunk_end;
 	u64 stripe_len;
@@ -4672,6 +4676,8 @@ static int check_chunk_item(struct extent_buffer *eb, int slot)
 		goto out;
 	}
 	type = btrfs_chunk_type(eb, chunk);
+	if (btrfs_bg_type_is_journal(type))
+		per_dev_reserved = btrfs_chunk_per_dev_reserved(eb, chunk);
 
 	btrfs_init_path(&path);
 	ret = find_block_group_item(&path, chunk_key.offset, length, type);
@@ -4679,13 +4685,14 @@ static int check_chunk_item(struct extent_buffer *eb, int slot)
 		err |= REFERENCER_MISSING;
 
 	num_stripes = btrfs_chunk_num_stripes(eb, chunk);
-	stripe_len = btrfs_stripe_length(gfs_info, eb, chunk);
+	stripe_len = btrfs_stripe_length(gfs_info, eb, chunk) + per_dev_reserved;
 	for (i = 0; i < num_stripes; i++) {
 		btrfs_release_path(&path);
 		btrfs_init_path(&path);
 		devext_key.objectid = btrfs_stripe_devid_nr(eb, chunk, i);
 		devext_key.type = BTRFS_DEV_EXTENT_KEY;
-		devext_key.offset = btrfs_stripe_offset_nr(eb, chunk, i);
+		devext_key.offset = btrfs_stripe_offset_nr(eb, chunk, i) -
+				    per_dev_reserved;
 
 		ret = btrfs_search_slot(NULL, dev_root, &devext_key, &path,
 					0, 0);
Signature · 1 line
2.36.1
author
#4 lore raw

PATCH 4/5 btrfs-progs: print-tree: add support for per_dev_reserved of chunk item

In reply to Qu Wenruo

Just change the prompt string for "io_align" to "per_dev_reserved" based on the chunk type.

---
 kernel-shared/print-tree.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
kernel-shared/print-tree.c +4 -1
index 9c12dfcb4ca5..d9f896ced827 100644
@@ -223,6 +223,7 @@ void print_chunk_item(struct extent_buffer *eb, struct btrfs_chunk *chunk)
 	int i;
 	u32 chunk_item_size;
 	char chunk_flags_str[BG_FLAG_STRING_LEN] = {};
+	bool is_journal;
 
 	/* The chunk must contain at least one stripe */
 	if (num_stripes < 1) {
@@ -237,13 +238,15 @@ void print_chunk_item(struct extent_buffer *eb, struct btrfs_chunk *chunk)
 		return;
 	}
 
+	is_journal = btrfs_bg_type_is_journal(btrfs_chunk_type(eb, chunk));
 	bg_flags_to_str(btrfs_chunk_type(eb, chunk), chunk_flags_str);
 	printf("\t\tlength %llu owner %llu stripe_len %llu type %s\n",
 	       (unsigned long long)btrfs_chunk_length(eb, chunk),
 	       (unsigned long long)btrfs_chunk_owner(eb, chunk),
 	       (unsigned long long)btrfs_chunk_stripe_len(eb, chunk),
 		chunk_flags_str);
-	printf("\t\tio_align %u io_width %u sector_size %u\n",
+	printf("\t\t%s %u io_width %u sector_size %u\n",
+			is_journal ? "per_dev_reserved" : "io_align",
 			btrfs_chunk_io_align(eb, chunk),
 			btrfs_chunk_io_width(eb, chunk),
 			btrfs_chunk_sector_size(eb, chunk));
Signature · 1 line
2.36.1
author
#5 lore raw

PATCH 5/5 btrfs-progs: check/lowmem: fix path leakage when dev extents are invalid

In reply to Qu Wenruo

[BUG]
When testing my new RAID56J code, there is a bug causing dev extents overlapping.

Although both modes can detect the problem, lowmem has leaked some extent buffers:

  $ btrfs check --mode=lowmem /dev/test/scratch1
  Opening filesystem to check...
  Checking filesystem on /dev/test/scratch1
  UUID: 65775ce9-bb9d-4f61-a210-beea52eef090
  [1/7] checking root items
  [2/7] checking extents
  ERROR: dev extent devid 1 offset 1095761920 len 1073741824 overlap with previous dev extent end 1096810496
  ERROR: dev extent devid 2 offset 1351614464 len 1073741824 overlap with previous dev extent end 1352663040
  ERROR: dev extent devid 3 offset 1351614464 len 1073741824 overlap with previous dev extent end 1352663040
  ERROR: errors found in extent allocation tree or chunk allocation
  [3/7] checking free space tree
  [4/7] checking fs roots
  [5/7] checking only csums items (without verifying data)
  [6/7] checking root refs done with fs roots in lowmem mode, skipping
  [7/7] checking quota groups skipped (not enabled on this FS)
  found 3221372928 bytes used, error(s) found
  total csum bytes: 0
  total tree bytes: 147456
  total fs tree bytes: 32768
  total extent tree bytes: 16384
  btree space waste bytes: 136231
  file data blocks allocated: 3221225472
   referenced 3221225472
  extent buffer leak: start 30752768 len 16384
  extent buffer leak: start 30752768 len 16384
  extent buffer leak: start 30752768 len 16384

[CAUSE]
In the function check_dev_item(), we iterate through all the dev extents, but when we found overlapping extents, we exit without releasing the path, causing extent buffer leakage.

[FIX]
Just release the path before we exit the function.

---
 check/mode-lowmem.c | 2 ++
 1 file changed, 2 insertions(+)
check/mode-lowmem.c +2 -0
index 63f9343ba2ff..7dd61e0af0b6 100644
@@ -4561,6 +4561,7 @@ static int check_dev_item(struct extent_buffer *eb, int slot,
 "dev extent devid %llu offset %llu len %llu overlap with previous dev extent end %llu",
 			      devid, physical_offset, physical_len,
 			      prev_dev_ext_end);
+			btrfs_release_path(&path);
 			return ACCOUNTING_MISMATCH;
 		}
 		if (physical_offset + physical_len > total_bytes) {
@@ -4568,6 +4569,7 @@ static int check_dev_item(struct extent_buffer *eb, int slot,
 "dev extent devid %llu offset %llu len %llu is beyond device boundary %llu",
 			      devid, physical_offset, physical_len,
 			      total_bytes);
+			btrfs_release_path(&path);
 			return ACCOUNTING_MISMATCH;
 		}
 		prev_devid = devid;
Signature · 1 line
2.36.1
Reviewed-by on v1 5/5
#6 lore raw

Re: [PATCH 5/5] btrfs-progs: check/lowmem: fix path leakage when dev extents are invalid

Quoted 39 lines · Qu Wenruo

[BUG]
When testing my new RAID56J code, there is a bug causing dev extents overlapping.

Although both modes can detect the problem, lowmem has leaked some extent buffers:

   $ btrfs check --mode=lowmem /dev/test/scratch1
   Opening filesystem to check...
   Checking filesystem on /dev/test/scratch1
   UUID: 65775ce9-bb9d-4f61-a210-beea52eef090
   [1/7] checking root items
   [2/7] checking extents
   ERROR: dev extent devid 1 offset 1095761920 len 1073741824 overlap with previous dev extent end 1096810496
   ERROR: dev extent devid 2 offset 1351614464 len 1073741824 overlap with previous dev extent end 1352663040
   ERROR: dev extent devid 3 offset 1351614464 len 1073741824 overlap with previous dev extent end 1352663040
   ERROR: errors found in extent allocation tree or chunk allocation
   [3/7] checking free space tree
   [4/7] checking fs roots
   [5/7] checking only csums items (without verifying data)
   [6/7] checking root refs done with fs roots in lowmem mode, skipping
   [7/7] checking quota groups skipped (not enabled on this FS)
   found 3221372928 bytes used, error(s) found
   total csum bytes: 0
   total tree bytes: 147456
   total fs tree bytes: 32768
   total extent tree bytes: 16384
   btree space waste bytes: 136231
   file data blocks allocated: 3221225472
    referenced 3221225472
   extent buffer leak: start 30752768 len 16384
   extent buffer leak: start 30752768 len 16384
   extent buffer leak: start 30752768 len 16384

[CAUSE]
In the function check_dev_item(), we iterate through all the dev extents, but when we found overlapping extents, we exit without releasing the path, causing extent buffer leakage.

Quoted 4 lines

[FIX]
Just release the path before we exit the function.

This can go completely independently from the raid56j code.

#7 lore raw

Re: [PATCH 5/5] btrfs-progs: check/lowmem: fix path leakage when dev extents are invalid

Quoted 45 lines · Nikolay Borisov
Quoted 42 lines · Qu Wenruo

[BUG]
When testing my new RAID56J code, there is a bug causing dev extents overlapping.

Although both modes can detect the problem, lowmem has leaked some extent buffers:

   $ btrfs check --mode=lowmem /dev/test/scratch1
   Opening filesystem to check...
   Checking filesystem on /dev/test/scratch1
   UUID: 65775ce9-bb9d-4f61-a210-beea52eef090
   [1/7] checking root items
   [2/7] checking extents
   ERROR: dev extent devid 1 offset 1095761920 len 1073741824 overlap with previous dev extent end 1096810496
   ERROR: dev extent devid 2 offset 1351614464 len 1073741824 overlap with previous dev extent end 1352663040
   ERROR: dev extent devid 3 offset 1351614464 len 1073741824 overlap with previous dev extent end 1352663040
   ERROR: errors found in extent allocation tree or chunk allocation
   [3/7] checking free space tree
   [4/7] checking fs roots
   [5/7] checking only csums items (without verifying data)
   [6/7] checking root refs done with fs roots in lowmem mode, skipping
   [7/7] checking quota groups skipped (not enabled on this FS)
   found 3221372928 bytes used, error(s) found
   total csum bytes: 0
   total tree bytes: 147456
   total fs tree bytes: 32768
   total extent tree bytes: 16384
   btree space waste bytes: 136231
   file data blocks allocated: 3221225472
    referenced 3221225472
   extent buffer leak: start 30752768 len 16384
   extent buffer leak: start 30752768 len 16384
   extent buffer leak: start 30752768 len 16384

[CAUSE]
In the function check_dev_item(), we iterate through all the dev extents, but when we found overlapping extents, we exit without releasing the path, causing extent buffer leakage.

[FIX]
Just release the path before we exit the function.

Quoted 6 lines
Quoted 1 line

This can go completely independently from the raid56j code.

Right, thanks, patch added to devel.