btrfs: separate BLOCK_GROUP_TREE feature from extent-tree-v2
Subsystems (2)
cover btrfs: separate BLOCK_GROUP_TREE feature from extent-tree-v2
This is the kernel part to revive block-group-tree feature.
Thanfully unlike btrfs-progs, the changes to kernel is much smaller, and we can re-use most of the infrastructures from the extent-tree-v2 preparation patches.
But there are still some changes needed:
- Enhance unsupporter compat RO flags handling
Extent tree is only needed for read-write opeartions, and for unsupported compat RO flags, we should not do any write into the fs.
So this patch will make the kernel to skip block group items search if there is any unsupport RO compat flags.
And really make the incoming block-group-tree feature compat RO.
Unfortunately, we need that patch to be backported, or older kernels will still reject RO mounts of fses with block-group-tree feature.
- Don't store block group root into super block
There is no special reason for block group root to be stored in super block.
We should review those preparation patches with more scrutiny.
For the proper time reduction introduced by this patchset, the old data should still be correct, as the on-disk format is not changed.
https://lwn.net/Articles/801990/
Qu Wenruo (3):
btrfs: enhance unsupported compat RO flags handling
btrfs: don't save block group root into super block
btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2
fs/btrfs/block-group.c | 11 ++++++++++- fs/btrfs/block-rsv.c | 1 + fs/btrfs/ctree.h | 30 ++++------------------------ fs/btrfs/disk-io.c | 40 +++++++++++++++++++------------------- fs/btrfs/disk-io.h | 2 +- fs/btrfs/super.c | 9 +++++++++ fs/btrfs/sysfs.c | 2 ++ fs/btrfs/transaction.c | 8 -------- include/uapi/linux/btrfs.h | 6 ++++++ 9 files changed, 53 insertions(+), 56 deletions(-)
Signature · 1 line
2.37.0
PATCH 1/3 btrfs: enhance unsupported compat RO flags handling
Currently there are two corner cases not handling compat RO flags correctly:
- Remount
We can still mount the fs RO with compat RO flags, then remount it RW. We should not allow any write into a fs with unsupported RO flags.
- Still try to search block group items
In fact, behavior/on-disk format change to extent tree should not need a full incompat flag.
And since we can ensure fs with unsupported RO flags never got any writes (with above case fixed), then we can even skip block group items search at mount time.
This patch will enhance the unsupported RO compat flags by:
- Reject RW remount if there is unsupported RO compat flags
- Go dummy block group items directly for unsupported RO compat flags In fact, only changes to chunk/subvolume/root/csum trees should go incompat flags.
The latter part should allow future change to extent tree to be compat RO flags.
Thus this patch also needs to be backported to all stable trees.
--- fs/btrfs/block-group.c | 11 ++++++++++- fs/btrfs/super.c | 9 +++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
fs/btrfs/block-group.c +10 -1
@@ -2200,7 +2200,16 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)int need_clear =0 ;u64 cache_gen; -if (!root) +/* +* Either no extent root (with ibadroots rescue option) or we have +* unsupporter RO options. The fs can never be mounted RW, so no +* need to waste time search block group items. +* +* This also allows new extent tree related changes to be RO compat, +* no need for a full incompat flag. +*/ +if (!root || (btrfs_super_compat_ro_flags (info->super_copy) & + ~BTRFS_FEATURE_COMPAT_RO_SUPP ))return fill_dummy_bgs (info); key.objectid =0 ;
fs/btrfs/super.c +9 -0
@@ -2110,6 +2110,15 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) ret = -EINVAL ; goto restore; } +if (btrfs_super_compat_ro_flags (fs_info->super_copy) & + ~BTRFS_FEATURE_COMPAT_RO_SUPP ) { +btrfs_err (fs_info, +"can not remount read-write due to unsupported optional flags 0x%llx" , +btrfs_super_compat_ro_flags (fs_info->super_copy) & + ~BTRFS_FEATURE_COMPAT_RO_SUPP ); + ret = -EINVAL ; + goto restore; + }if (fs_info->fs_devices->rw_devices ==0 ) { ret = -EACCES ; goto restore;
Signature · 1 line
2.37.0
PATCH 2/3 btrfs: don't save block group root into super block
In reply to Qu WenruoThe extent tree v2 (thankfully not yet fully materialized) needs a new root for storing all block group items.
My initial proposal years ago just added a new tree rootid, and load it from tree root, just like what we did for quota/free space tree/uuid/extent roots.
But the extent tree v2 patches introduced a completely new (and to me, wasteful) way to store block group tree root into super block.
Currently there are only 3 trees stored in super blocks, and they all have their valid reasons:
- Chunk root
Needed for bootstrap.
- Tree root
Really the entrance of all trees.
- Log root
This is special as log root has to be updated out of existing transaction mechanism.
There is not even any reason to put block group root into super blocks, the block group tree is updated at the same timing as old extent tree, no need for extra bootstrap/out-of-transaction update.
So just move block group root from super block into tree root.
--- fs/btrfs/block-rsv.c | 1 + fs/btrfs/ctree.h | 27 ++------------------------- fs/btrfs/disk-io.c | 40 ++++++++++++++++++++-------------------- fs/btrfs/transaction.c | 8 -------- 4 files changed, 23 insertions(+), 53 deletions(-)
fs/btrfs/block-rsv.c +1 -0
@@ -424,6 +424,7 @@ void btrfs_init_root_block_rsv(struct btrfs_root *root)case BTRFS_CSUM_TREE_OBJECTID :case BTRFS_EXTENT_TREE_OBJECTID :case BTRFS_FREE_SPACE_TREE_OBJECTID : +case BTRFS_BLOCK_GROUP_TREE_OBJECTID : root->block_rsv = &fs_info->delayed_refs_rsv;break ;case BTRFS_ROOT_TREE_OBJECTID :
fs/btrfs/ctree.h +2 -25
@@ -288,14 +288,9 @@ struct btrfs_super_block {/* the UUID written into btree blocks */ u8 metadata_uuid[BTRFS_FSID_SIZE ]; -/* Extent tree v2 */ -__le64 block_group_root; -__le64 block_group_root_generation; -u8 block_group_root_level; -/* future expansion */ -u8 reserved8[7 ]; -__le64 reserved[25 ]; +u8 reserved8[8 ]; +__le64 reserved[27 ];u8 sys_chunk_array[BTRFS_SYSTEM_CHUNK_ARRAY_SIZE ];struct btrfs_root_backup super_roots[BTRFS_NUM_BACKUP_ROOTS ];
@@ -2389,17 +2384,6 @@ BTRFS_SETGET_STACK_FUNCS(backup_bytes_used, struct btrfs_root_backup, BTRFS_SETGET_STACK_FUNCS(backup_num_devices, struct btrfs_root_backup, num_devices,64 ); -/* -* For extent tree v2 we overload the extent root with the block group root, as -* we will have multiple extent roots. -*/ -BTRFS_SETGET_STACK_FUNCS(backup_block_group_root, struct btrfs_root_backup, - extent_root,64 ); -BTRFS_SETGET_STACK_FUNCS(backup_block_group_root_gen, struct btrfs_root_backup, - extent_root_gen,64 ); -BTRFS_SETGET_STACK_FUNCS (backup_block_group_root_level, - struct btrfs_root_backup, extent_root_level, 8); -/* struct btrfs_balance_item */ BTRFS_SETGET_FUNCS(balance_flags, struct btrfs_balance_item, flags, 64);
@@ -2532,13 +2516,6 @@ BTRFS_SETGET_STACK_FUNCS(super_cache_generation, struct btrfs_super_block, BTRFS_SETGET_STACK_FUNCS(super_magic, struct btrfs_super_block, magic, 64); BTRFS_SETGET_STACK_FUNCS(super_uuid_tree_generation, struct btrfs_super_block, uuid_tree_generation,64 ); -BTRFS_SETGET_STACK_FUNCS(super_block_group_root, struct btrfs_super_block, - block_group_root,64 ); -BTRFS_SETGET_STACK_FUNCS (super_block_group_root_generation, - struct btrfs_super_block, - block_group_root_generation,64 ); -BTRFS_SETGET_STACK_FUNCS(super_block_group_root_level, struct btrfs_super_block, - block_group_root_level,8 );int btrfs_super_csum_size (const struct btrfs_super_block *s);const char *btrfs_super_csum_name (u16 csum_type);
fs/btrfs/disk-io.c +20 -20
@@ -1606,6 +1606,9 @@ static struct btrfs_root *btrfs_get_global_root(struct btrfs_fs_info *fs_info,if (objectid ==BTRFS_UUID_TREE_OBJECTID )return btrfs_grab_root (fs_info->uuid_root) ? fs_info->uuid_root :ERR_PTR (-ENOENT ); +if (objectid ==BTRFS_BLOCK_GROUP_TREE_OBJECTID ) +return btrfs_grab_root (fs_info->block_group_root) ? + fs_info->block_group_root :ERR_PTR (-ENOENT );if (objectid ==BTRFS_FREE_SPACE_TREE_OBJECTID ) {struct btrfs_root *root =btrfs_global_root (fs_info, &key);
@@ -2056,14 +2059,7 @@ static void backup_super_roots(struct btrfs_fs_info *info)btrfs_set_backup_chunk_root_level (root_backup,btrfs_header_level (info->chunk_root->node)); -if (btrfs_fs_incompat (info,EXTENT_TREE_V2 )) { -btrfs_set_backup_block_group_root (root_backup, - info->block_group_root->node->start); -btrfs_set_backup_block_group_root_gen (root_backup, -btrfs_header_generation (info->block_group_root->node)); -btrfs_set_backup_block_group_root_level (root_backup, -btrfs_header_level (info->block_group_root->node)); - }else { +if (!btrfs_fs_incompat (info,EXTENT_TREE_V2 )) {struct btrfs_root *extent_root =btrfs_extent_root (info,0 );struct btrfs_root *csum_root =btrfs_csum_root (info,0 );
@@ -2596,10 +2592,24 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info)if (ret)return ret; - location.objectid =BTRFS_DEV_TREE_OBJECTID ; location.type =BTRFS_ROOT_ITEM_KEY ; location.offset =0 ; +if (btrfs_fs_incompat (fs_info,EXTENT_TREE_V2 )) { + location.objectid =BTRFS_BLOCK_GROUP_TREE_OBJECTID ; + root =btrfs_read_tree_root (tree_root, &location); +if (IS_ERR (root)) { +if (!btrfs_test_opt (fs_info,IGNOREBADROOTS )) { + ret =PTR_ERR (root); + goto out; + } + }else { +set_bit (BTRFS_ROOT_TRACK_DIRTY , &root->state); + fs_info->block_group_root = root; + } + } + + location.objectid =BTRFS_DEV_TREE_OBJECTID ; root =btrfs_read_tree_root (tree_root, &location);if (IS_ERR (root)) {if (!btrfs_test_opt (fs_info,IGNOREBADROOTS )) {
@@ -2927,17 +2937,7 @@ static int load_important_roots(struct btrfs_fs_info *fs_info)btrfs_warn (fs_info,"couldn't read tree root" );return ret; } - -if (!btrfs_fs_incompat (fs_info,EXTENT_TREE_V2 )) -return 0 ; - - bytenr =btrfs_super_block_group_root (sb); - gen =btrfs_super_block_group_root_generation (sb); - level =btrfs_super_block_group_root_level (sb); - ret =load_super_root (fs_info->block_group_root, bytenr, gen, level); -if (ret) -btrfs_warn (fs_info,"couldn't read block group root" ); -return ret; +return 0 ; }static int __cold init_tree_roots (struct btrfs_fs_info *fs_info)
fs/btrfs/transaction.c +0 -8
@@ -1898,14 +1898,6 @@ static void update_super_roots(struct btrfs_fs_info *fs_info) super->cache_generation =0 ;if (test_bit (BTRFS_FS_UPDATE_UUID_TREE_GEN , &fs_info->flags)) super->uuid_tree_generation = root_item->generation; - -if (btrfs_fs_incompat (fs_info,EXTENT_TREE_V2 )) { - root_item = &fs_info->block_group_root->root_item; - - super->block_group_root = root_item->bytenr; - super->block_group_root_generation = root_item->generation; - super->block_group_root_level = root_item->level; - } }int btrfs_transaction_in_commit (struct btrfs_fs_info *info)
Signature · 1 line
2.37.0
PATCH 3/3 btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2
In reply to Qu WenruoThe problem of long mount time caused by block group item search is already known for over 7 years, and the solution of block group tree is proposed at least for 5 years.
There is really no need to bound this feature into extent tree v2, just introduce compat RO flag, BLOCK_GROUP_TREE, to correctly solve the problem.
All the code handling block group root is already in the upstream kernel, thus this patch really only needs to introduce the new compat RO flag.
--- fs/btrfs/ctree.h | 3 ++- fs/btrfs/disk-io.h | 2 +- fs/btrfs/sysfs.c | 2 ++ include/uapi/linux/btrfs.h | 6 ++++++ 4 files changed, 11 insertions(+), 2 deletions(-)
fs/btrfs/ctree.h +2 -1
@@ -310,7 +310,8 @@ static_assert(sizeof(struct btrfs_super_block) == BTRFS_SUPER_INFO_SIZE);#define BTRFS_FEATURE_COMPAT_RO_SUPP \ (BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE | \ BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID | \ - BTRFS_FEATURE_COMPAT_RO_VERITY) + BTRFS_FEATURE_COMPAT_RO_VERITY | \ + BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE)#define BTRFS_FEATURE_COMPAT_RO_SAFE_SET 0ULL#define BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR 0ULL
fs/btrfs/disk-io.h +1 -1
@@ -103,7 +103,7 @@ static inline struct btrfs_root *btrfs_grab_root(struct btrfs_root *root)static inline struct btrfs_root *btrfs_block_group_root (struct btrfs_fs_info *fs_info) { -if (btrfs_fs_incompat (fs_info,EXTENT_TREE_V2 )) +if (btrfs_fs_compat_ro (fs_info,BLOCK_GROUP_TREE ))return fs_info->block_group_root;return btrfs_extent_root (fs_info,0 ); }
fs/btrfs/sysfs.c +2 -0
@@ -286,6 +286,7 @@ BTRFS_FEAT_ATTR_INCOMPAT(skinny_metadata, SKINNY_METADATA);BTRFS_FEAT_ATTR_INCOMPAT (no_holes,NO_HOLES );BTRFS_FEAT_ATTR_INCOMPAT (metadata_uuid,METADATA_UUID );BTRFS_FEAT_ATTR_COMPAT_RO (free_space_tree,FREE_SPACE_TREE ); +BTRFS_FEAT_ATTR_COMPAT_RO (block_group_tree,BLOCK_GROUP_TREE );BTRFS_FEAT_ATTR_INCOMPAT (raid1c34,RAID1C34 );#ifdef CONFIG_BLK_DEV_ZONED BTRFS_FEAT_ATTR_INCOMPAT (zoned,ZONED );
@@ -316,6 +317,7 @@ static struct attribute *btrfs_supported_feature_attrs[] = {BTRFS_FEAT_ATTR_PTR (no_holes),BTRFS_FEAT_ATTR_PTR (metadata_uuid),BTRFS_FEAT_ATTR_PTR (free_space_tree), +BTRFS_FEAT_ATTR_PTR (block_group_tree),BTRFS_FEAT_ATTR_PTR (raid1c34), #ifdef CONFIG_BLK_DEV_ZONED BTRFS_FEAT_ATTR_PTR(zoned),
include/uapi/linux/btrfs.h +6 -0
@@ -290,6 +290,12 @@ struct btrfs_ioctl_fs_info_args {#define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID (1ULL << 1)#define BTRFS_FEATURE_COMPAT_RO_VERITY (1ULL << 2) +/* +* Put all block group items into a dedicate block group tree, greatly +* reduce mount time for large fs. +*/ +#define BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE (1ULL << 5) +#define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF (1ULL << 0)#define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL (1ULL << 1)#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 2)
Signature · 1 line
2.37.0