btrfs-progs: zoned: support zone capacity and
v2 cover btrfs-progs: zoned: support zone capacity and
Running mkfs.btrfs on a null_blk device with the following setup fails as below.
- zone size: 64MB
- zone capacity: 64MB
- number of conventional zones: 6
- storage size: 2048MB
+ /home/naota/src/btrfs-progs/mkfs.btrfs -d single -m dup -f /dev/nullb0 btrfs-progs v6.10
See https://btrfs.readthedocs.io for more information.
zoned: /dev/nullb0: host-managed device detected, setting zoned feature Resetting device zones /dev/nullb0 (32 zones) ...
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)
bad tree block 268435456, bytenr mismatch, want=268435456, have=0 kernel-shared/disk-io.c:485: write_tree_block: BUG_ON 1 triggered, value 1 /home/naota/src/btrfs-progs/mkfs.btrfs(+0x290ca) [0x55603cf7e0ca] /home/naota/src/btrfs-progs/mkfs.btrfs(write_tree_block+0xa7) [0x55603cf80417] /home/naota/src/btrfs-progs/mkfs.btrfs(__commit_transaction+0xe8) [0x55603cf9b7d8] /home/naota/src/btrfs-progs/mkfs.btrfs(btrfs_commit_transaction+0x176) [0x55603cf9ba66] /home/naota/src/btrfs-progs/mkfs.btrfs(main+0x2831) [0x55603cf67291] /usr/lib64/libc.so.6(+0x271ee) [0x7f5ab706f1ee] /usr/lib64/libc.so.6(__libc_start_main+0x89) [0x7f5ab706f2a9] /home/naota/src/btrfs-progs/mkfs.btrfs(_start+0x25) [0x55603cf6a135]
/home/naota/tmp/test-mkfs.sh: line 13: 821886 Aborted (core dumped)
The crash happens because btrfs-progs failed to set proper allocation pointer when a DUP block group is created over a conventional zone and a sequential write required zone. In that case, the write pointer is recovered from the last allocated extent in the block group. That functionality is not well implemented in btrfs-progs side.
Implementing that functionality is relatively trivial because we can copy the code from the kernel side. However, the code is quite out of sync between the kernel side and user space side. So, this series first refactors btrfs_load_block_group_zone_info() to make it easy to integrate the code from the kernel side.
The main part is the last patch, which fixes allocation pointer calculation for all the profiles.
While at it, this series also adds support for zone capacity and zone activeness. But, zone activeness support is currently limited. It does not attempt to check the zone active limit on the extent allocation, because mkfs.btrfs should work without hitting the limit.
- v2
- Temporarily fails some profiles while adding supports in the patch series.
- v1: https://lore.kernel.org/linux-btrfs/[email protected]/
Naohiro Aota (12):
btrfs-progs: introduce min_not_zero()
btrfs-progs: zoned: introduce a zone_info struct in
btrfs_load_block_group_zone_info
btrfs-progs: zoned: support zone capacity
btrfs-progs: zoned: load zone activeness
btrfs-progs: zoned: activate block group on loading
btrfs-progs: factor out btrfs_load_zone_info()
btrfs-progs: zoned: factor out SINGLE zone info loading
btrfs-progs: zoned: implement DUP zone info loading
btrfs-progs: zoned: implement RAID1 zone info loading
btrfs-progs: zoned: implement RAID0 zone info loading
btrfs-progs: implement RAID10 zone info loading
btrfs-progs: zoned: fix alloc_offset calculation for partly
conventional block groups
include/kerncompat.h | 10 + kernel-shared/ctree.h | 3 + kernel-shared/extent-tree.c | 2 +- kernel-shared/zoned.c | 458 +++++++++++++++++++++++++++++++----- kernel-shared/zoned.h | 3 + 5 files changed, 414 insertions(+), 62 deletions(-)
--
2.48.1
PATCH v2 1/12 btrfs-progs: introduce min_not_zero()
Introduce min_not_zero() macro from the kernel.
--- include/kerncompat.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
include/kerncompat.h +10 -0
@@ -127,6 +127,16 @@ }#endif +/** +* min_not_zero - return the minimum that is _not_ zero, unless both are zero +* @x: value1 +* @y: value2 +*/ +#define min_not_zero (x, y) ({ \ + typeof(x) __x = (x); \ + typeof(y) __y = (y); \ + __x == 0 ? __y : ((__y == 0) ? __x : min(__x, __y)); }) +static inline void print_trace (void ) {#ifndef BTRFS_DISABLE_BACKTRACE
Signature · 1 line
2.48.1
PATCH v2 2/12 btrfs-progs: zoned: introduce a zone_info struct in btrfs_load_block_group_zone_info
In reply to Naohiro AotaThis is an userland side update to follow kernel-side commit 15c12fcc50a1 ("btrfs: zoned: introduce a zone_info struct in btrfs_load_block_group_zone_info"). This will make the code unification easier.
This commit introduces zone_info structure to hold per-zone information in btrfs_load_block_group_zone_info.
--- kernel-shared/zoned.c | 46 ++++++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 22 deletions(-)
kernel-shared/zoned.c +24 -22
@@ -828,6 +828,11 @@ bool zoned_profile_supported(u64 map_type, bool rst)return false; } +struct zone_info { +u64 physical; +u64 alloc_offset; +}; +int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache) {
@@ -837,10 +842,9 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,struct map_lookup *map;u64 logical = cache->start;u64 length = cache->length; -u64 physical =0 ; +struct zone_info *zone_info =NULL ;int ret =0 ;int i; -u64 *alloc_offsets =NULL ;u64 last_alloc =0 ;u32 num_conventional =0 ;
@@ -867,30 +871,29 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, } map = container_of(ce, struct map_lookup, ce); - alloc_offsets =calloc (map->num_stripes,sizeof (*alloc_offsets)); -if (!alloc_offsets) { -error_msg (ERROR_MSG_MEMORY ,"zone offsets" ); + zone_info =calloc (map->num_stripes,sizeof (*zone_info)); +if (!zone_info) { +error_msg (ERROR_MSG_MEMORY ,"zone info" );return -ENOMEM ; }for (i =0 ; i < map->num_stripes; i++) { +struct zone_info *info = &zone_info[i];bool is_sequential;struct blk_zone zone; device = map->stripes[i].dev; - physical = map->stripes[i].physical; + info->physical = map->stripes[i].physical;if (device->fd ==-1 ) { - alloc_offsets[i] =WP_MISSING_DEV ; + info->alloc_offset =WP_MISSING_DEV ;continue ; } - is_sequential =btrfs_dev_is_sequential (device, physical); -if (!is_sequential) - num_conventional++; - + is_sequential =btrfs_dev_is_sequential (device, info->physical);if (!is_sequential) { - alloc_offsets[i] =WP_CONVENTIONAL ; + num_conventional++; + info->alloc_offset =WP_CONVENTIONAL ;continue ; }
@@ -898,28 +901,27 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,* The group is mapped to a sequential zone. Get the zone write * pointer to determine the allocation offset within the zone. */ -WARN_ON (!IS_ALIGNED (physical, fs_info->zone_size)); - zone = device->zone_info->zones[physical / fs_info->zone_size]; +WARN_ON (!IS_ALIGNED (info->physical, fs_info->zone_size)); + zone = device->zone_info->zones[info->physical / fs_info->zone_size];switch (zone.cond) {case BLK_ZONE_COND_OFFLINE :case BLK_ZONE_COND_READONLY :error ("zoned: offline/readonly zone %llu on device %s (devid %llu)" , - physical / fs_info->zone_size, device->name, + info->physical / fs_info->zone_size, device->name, device->devid); - alloc_offsets[i] =WP_MISSING_DEV ; + info->alloc_offset =WP_MISSING_DEV ;break ;case BLK_ZONE_COND_EMPTY : - alloc_offsets[i] =0 ; + info->alloc_offset =0 ;break ;case BLK_ZONE_COND_FULL : - alloc_offsets[i] = fs_info->zone_size; + info->alloc_offset = fs_info->zone_size;break ;default :/* Partially used zone */ - alloc_offsets[i] = - ((zone.wp - zone.start) <<SECTOR_SHIFT ); + info->alloc_offset = ((zone.wp - zone.start) <<SECTOR_SHIFT );break ; } }
@@ -943,7 +945,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret = -EINVAL ; goto out; } - cache->alloc_offset = alloc_offsets[0 ]; + cache->alloc_offset = zone_info[0 ].alloc_offset; out:/* An extent is allocated after the write pointer */
@@ -957,7 +959,7 @@ out:if (!ret) cache->write_offset = cache->alloc_offset; -kfree (alloc_offsets); +kfree (zone_info);return ret; }
Signature · 1 line
2.48.1
PATCH v2 3/12 btrfs-progs: zoned: support zone capacity
In reply to Naohiro AotaThe userland tools did not load and use the zone capacity. Support it properly.
--- kernel-shared/ctree.h | 1 + kernel-shared/extent-tree.c | 2 +- kernel-shared/zoned.c | 9 ++++++++- 3 files changed, 10 insertions(+), 2 deletions(-)
kernel-shared/ctree.h +1 -0
@@ -285,6 +285,7 @@ struct btrfs_block_group {*/ u64 alloc_offset;u64 write_offset; +u64 zone_capacity;u64 global_root_id; };
kernel-shared/extent-tree.c +1 -1
@@ -300,7 +300,7 @@ again: goto new_group;if (btrfs_is_zoned (root->fs_info)) { -if (cache->length - cache->alloc_offset < num) +if (cache->zone_capacity - cache->alloc_offset < num) goto new_group; *start_ret = cache->start + cache->alloc_offset; cache->alloc_offset += num;
kernel-shared/zoned.c +8 -1
@@ -776,7 +776,7 @@ static int calculate_alloc_pointer(struct btrfs_fs_info *fs_info, length = fs_info->nodesize;if (!(found_key.objectid >= cache->start && - found_key.objectid + length <= cache->start + cache->length)) { + found_key.objectid + length <= cache->start + cache->zone_capacity)) { ret = -EUCLEAN ; goto out; }
@@ -830,6 +830,7 @@ bool zoned_profile_supported(u64 map_type, bool rst)struct zone_info {u64 physical; +u64 capacity;u64 alloc_offset; };
@@ -894,6 +895,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,if (!is_sequential) { num_conventional++; info->alloc_offset =WP_CONVENTIONAL ; + info->capacity = device->zone_info->zone_size;continue ; }
@@ -904,6 +906,8 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,WARN_ON (!IS_ALIGNED (info->physical, fs_info->zone_size)); zone = device->zone_info->zones[info->physical / fs_info->zone_size]; + info->capacity = (zone.capacity <<SECTOR_SHIFT ); +switch (zone.cond) {case BLK_ZONE_COND_OFFLINE :case BLK_ZONE_COND_READONLY :
@@ -927,6 +931,8 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, }if (num_conventional >0 ) { +/* Zone capacity is always zone size in emulation */ + cache->zone_capacity = cache->length; ret =calculate_alloc_pointer (fs_info, cache, &last_alloc);if (ret || map->num_stripes == num_conventional) {if (!ret)
@@ -946,6 +952,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, goto out; } cache->alloc_offset = zone_info[0 ].alloc_offset; + cache->zone_capacity = zone_info[0 ].capacity; out:/* An extent is allocated after the write pointer */
Signature · 1 line
2.48.1
PATCH v2 4/12 btrfs-progs: zoned: load zone activeness
In reply to Naohiro AotaProperly load the zone activeness on the userland tool. Also, check if a device has enough active zone limit to run btrfs.
--- kernel-shared/ctree.h | 1 + kernel-shared/zoned.c | 77 +++++++++++++++++++++++++++++++++++++++---- kernel-shared/zoned.h | 3 ++ 3 files changed, 75 insertions(+), 6 deletions(-)
kernel-shared/ctree.h +1 -0
@@ -368,6 +368,7 @@ struct btrfs_fs_info { unsigned int allow_transid_mismatch:1; unsigned int skip_leaf_item_checks:1; unsigned int rebuilding_extent_tree:1; + unsigned int active_zone_tracking:1;int transaction_aborted;
kernel-shared/zoned.c +71 -6
139 lines not on this page · raw message
kernel-shared/zoned.h +3 -0
@@ -72,7 +72,10 @@ struct btrfs_zoned_device_info {enum btrfs_zoned_model model;u64 zone_size;u32 nr_zones; +unsigned int max_active_zones;struct blk_zone *zones; +atomic_t active_zones_left; +unsigned long *active_zones;bool emulated; };
Signature · 1 line
2.48.1
PATCH v2 5/12 btrfs-progs: zoned: activate block group on loading
In reply to Naohiro AotaIntroduce "zone_is_active" member to struct btrfs_block_group and activate it on loading a block group.
Note that activeness check for the extent allocation is currently not implemented. The activeness checking requires to activate a non-active block group on the extent allocation, which also require finishing a zone in the case of hitting the active zone limit. Since mkfs should not hit the limit, implementing the zone finishing code would not be necessary at the moment.
--- kernel-shared/ctree.h | 1 + kernel-shared/zoned.c | 15 +++++++++++++++ 2 files changed, 16 insertions(+)
kernel-shared/ctree.h +1 -0
@@ -286,6 +286,7 @@ struct btrfs_block_group {u64 alloc_offset;u64 write_offset;u64 zone_capacity; +bool zone_is_active;u64 global_root_id; };
kernel-shared/zoned.c +15 -0
@@ -901,6 +901,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,u64 logical = cache->start;u64 length = cache->length;struct zone_info *zone_info =NULL ; +unsigned long *active =NULL ;int ret =0 ;int i;u64 last_alloc =0 ;
@@ -935,6 +936,13 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,return -ENOMEM ; } + active =bitmap_zalloc (map->num_stripes); +if (!active) { +free (zone_info); +error_msg (ERROR_MSG_MEMORY ,"active bitmap" ); +return -ENOMEM ; + } +for (i =0 ; i < map->num_stripes; i++) {struct zone_info *info = &zone_info[i];bool is_sequential;
@@ -948,6 +956,10 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,continue ; } +/* Consider a zone as active if we can allow any number of active zones. */ +if (!device->zone_info->max_active_zones) +set_bit (i, active); + is_sequential =btrfs_dev_is_sequential (device, info->physical);if (!is_sequential) { num_conventional++;
@@ -983,6 +995,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,default :/* Partially used zone */ info->alloc_offset = ((zone.wp - zone.start) <<SECTOR_SHIFT ); +set_bit (i, active);break ; } }
@@ -1008,8 +1021,10 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret = -EINVAL ; goto out; } +/* SINGLE profile case. */ cache->alloc_offset = zone_info[0 ].alloc_offset; cache->zone_capacity = zone_info[0 ].capacity; + cache->zone_is_active =test_bit (0 , active); out:/* An extent is allocated after the write pointer */
Signature · 1 line
2.48.1
PATCH v2 6/12 btrfs-progs: factor out btrfs_load_zone_info()
In reply to Naohiro AotaNow that, we have zone capacity and (basic) zone activeness support. It's time to factor out btrfs_load_zone_info() as same as the kernel side.
--- kernel-shared/zoned.c | 124 ++++++++++++++++++++++++------------------ 1 file changed, 71 insertions(+), 53 deletions(-)
kernel-shared/zoned.c +71 -53
@@ -891,10 +891,76 @@ struct zone_info {u64 alloc_offset; }; +static int btrfs_load_zone_info (struct btrfs_fs_info *fs_info,int zone_idx, +struct zone_info *info,unsigned long *active, +struct map_lookup *map) +{ +struct btrfs_device *device; +struct blk_zone zone; + + info->physical = map->stripes[zone_idx].physical; + + device = map->stripes[zone_idx].dev; + +if (device->fd ==-1 ) { + info->alloc_offset =WP_MISSING_DEV ; +return 0 ; + } + +/* Consider a zone as active if we can allow any number of active zones. */ +if (!device->zone_info->max_active_zones) +set_bit (zone_idx, active); + +if (!btrfs_dev_is_sequential (device, info->physical)) { + info->alloc_offset =WP_CONVENTIONAL ; + info->capacity = device->zone_info->zone_size; +return 0 ; + } + +/* +* The group is mapped to a sequential zone. Get the zone write +* pointer to determine the allocation offset within the zone. +*/ +WARN_ON (!IS_ALIGNED (info->physical, fs_info->zone_size)); + zone = device->zone_info->zones[info->physical / fs_info->zone_size]; + +if (zone.type ==BLK_ZONE_TYPE_CONVENTIONAL ) { +error ("zoned: unexpected conventional zone %llu on device %s (devid %llu)" , + zone.start <<SECTOR_SHIFT , device->name, + device->devid); +return -EIO ; + } + + info->capacity = (zone.capacity <<SECTOR_SHIFT ); + +switch (zone.cond) { +case BLK_ZONE_COND_OFFLINE : +case BLK_ZONE_COND_READONLY : +error ( +"zoned: offline/readonly zone %llu on device %s (devid %llu)" , + info->physical / fs_info->zone_size, device->name, + device->devid); + info->alloc_offset =WP_MISSING_DEV ; +break ; +case BLK_ZONE_COND_EMPTY : + info->alloc_offset =0 ; +break ; +case BLK_ZONE_COND_FULL : + info->alloc_offset = fs_info->zone_size; +break ; +default : +/* Partially used zone */ + info->alloc_offset = ((zone.wp - zone.start) <<SECTOR_SHIFT ); +set_bit (zone_idx, active); +break ; + } + +return 0 ; +} +int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache) { -struct btrfs_device *device;struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree;struct cache_extent *ce;struct map_lookup *map;
@@ -944,60 +1010,12 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, }for (i =0 ; i < map->num_stripes; i++) { -struct zone_info *info = &zone_info[i]; -bool is_sequential; -struct blk_zone zone; - - device = map->stripes[i].dev; - info->physical = map->stripes[i].physical; - -if (device->fd ==-1 ) { - info->alloc_offset =WP_MISSING_DEV ; -continue ; - } - -/* Consider a zone as active if we can allow any number of active zones. */ -if (!device->zone_info->max_active_zones) -set_bit (i, active); + ret =btrfs_load_zone_info (fs_info, i, &zone_info[i], active, map); +if (ret) + goto out; - is_sequential =btrfs_dev_is_sequential (device, info->physical); -if (!is_sequential) { +if (zone_info[i].alloc_offset ==WP_CONVENTIONAL ) num_conventional++; - info->alloc_offset =WP_CONVENTIONAL ; - info->capacity = device->zone_info->zone_size; -continue ; - } - -/* -* The group is mapped to a sequential zone. Get the zone write -* pointer to determine the allocation offset within the zone. -*/ -WARN_ON (!IS_ALIGNED (info->physical, fs_info->zone_size)); - zone = device->zone_info->zones[info->physical / fs_info->zone_size]; - - info->capacity = (zone.capacity <<SECTOR_SHIFT ); - -switch (zone.cond) { -case BLK_ZONE_COND_OFFLINE : -case BLK_ZONE_COND_READONLY : -error ( -"zoned: offline/readonly zone %llu on device %s (devid %llu)" , - info->physical / fs_info->zone_size, device->name, - device->devid); - info->alloc_offset =WP_MISSING_DEV ; -break ; -case BLK_ZONE_COND_EMPTY : - info->alloc_offset =0 ; -break ; -case BLK_ZONE_COND_FULL : - info->alloc_offset = fs_info->zone_size; -break ; -default : -/* Partially used zone */ - info->alloc_offset = ((zone.wp - zone.start) <<SECTOR_SHIFT ); -set_bit (i, active); -break ; - } }if (num_conventional >0 ) {
Signature · 1 line
2.48.1
PATCH v2 7/12 btrfs-progs: zoned: factor out SINGLE zone info loading
In reply to Naohiro AotaCurrently, the userland tool only considers the SINGLE profile, which make it fail when a DUP block group is created over one conventional zone and one sequential required zone.
Before adding the other profiles support, let's factor out per-profile code (actually, SINGLE only) into functions just like as the kernel side.
--- kernel-shared/zoned.c | 47 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 4 deletions(-)
kernel-shared/zoned.c +43 -4
@@ -958,6 +958,26 @@ static int btrfs_load_zone_info(struct btrfs_fs_info *fs_info, int zone_idx,return 0 ; } +static int btrfs_load_block_group_single (struct btrfs_fs_info *fs_info, +struct btrfs_block_group *bg, +struct zone_info *info, +unsigned long *active) +{ +if (info->alloc_offset ==WP_MISSING_DEV ) { +btrfs_err (fs_info, +"zoned: cannot recover write pointer for zone %llu" , + info->physical); +return -EIO ; + } + + bg->alloc_offset = info->alloc_offset; + bg->zone_capacity = info->capacity; +if (test_bit (0 , active)) + bg->zone_is_active =1 ; +return 0 ; +} + +int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache) {
@@ -972,6 +992,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,int i;u64 last_alloc =0 ;u32 num_conventional =0 ; +u64 profile;if (!btrfs_is_zoned (fs_info))return 0 ;
@@ -1039,10 +1060,28 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret = -EINVAL ; goto out; } -/* SINGLE profile case. */ - cache->alloc_offset = zone_info[0 ].alloc_offset; - cache->zone_capacity = zone_info[0 ].capacity; - cache->zone_is_active =test_bit (0 , active); + + profile = map->type &BTRFS_BLOCK_GROUP_PROFILE_MASK ; +switch (profile) { +case 0 :/* single */ + ret =btrfs_load_block_group_single (fs_info, cache, &zone_info[0 ], active); +break ; +case BTRFS_BLOCK_GROUP_DUP : +case BTRFS_BLOCK_GROUP_RAID1 : +case BTRFS_BLOCK_GROUP_RAID1C3 : +case BTRFS_BLOCK_GROUP_RAID1C4 : +case BTRFS_BLOCK_GROUP_RAID0 : +case BTRFS_BLOCK_GROUP_RAID10 : +/* Temporarily fails these case, until following commits. */ + fallthrough; +case BTRFS_BLOCK_GROUP_RAID5 : +case BTRFS_BLOCK_GROUP_RAID6 : +default : +error ("zoned: profile %s not yet supported" , +btrfs_bg_type_to_raid_name (map->type)); + ret = -EINVAL ; + goto out; + } out:/* An extent is allocated after the write pointer */
Signature · 1 line
2.48.1
PATCH v2 8/12 btrfs-progs: zoned: implement DUP zone info loading
In reply to Naohiro AotaDUP support is added like the kernel side.
--- kernel-shared/zoned.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+)
kernel-shared/zoned.c +42 -0
@@ -977,6 +977,46 @@ static int btrfs_load_block_group_single(struct btrfs_fs_info *fs_info,return 0 ; } +static int btrfs_load_block_group_dup (struct btrfs_fs_info *fs_info, +struct btrfs_block_group *bg, +struct map_lookup *map, +struct zone_info *zone_info, +unsigned long *active) +{ +if ((map->type &BTRFS_BLOCK_GROUP_DATA ) && !fs_info->stripe_root) { +btrfs_err (fs_info,"zoned: data DUP profile needs raid-stripe-tree" ); +return -EINVAL ; + } + + bg->zone_capacity =min_not_zero (zone_info[0 ].capacity, zone_info[1 ].capacity); + +if (zone_info[0 ].alloc_offset ==WP_MISSING_DEV ) { +btrfs_err (fs_info, +"zoned: cannot recover write pointer for zone %llu" , + zone_info[0 ].physical); +return -EIO ; + } +if (zone_info[1 ].alloc_offset ==WP_MISSING_DEV ) { +btrfs_err (fs_info, +"zoned: cannot recover write pointer for zone %llu" , + zone_info[1 ].physical); +return -EIO ; + } +if (zone_info[0 ].alloc_offset != zone_info[1 ].alloc_offset) { +btrfs_err (fs_info, +"zoned: write pointer offset mismatch of zones in DUP profile" ); +return -EIO ; + } + +if (test_bit (0 , active) !=test_bit (1 , active)) { +return -EIO ; + }else if (test_bit (0 , active)) { + bg->zone_is_active =1 ; + } + + bg->alloc_offset = zone_info[0 ].alloc_offset; +return 0 ; +}int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache)
@@ -1067,6 +1107,8 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret =btrfs_load_block_group_single (fs_info, cache, &zone_info[0 ], active);break ;case BTRFS_BLOCK_GROUP_DUP : + ret =btrfs_load_block_group_dup (fs_info, cache, map, zone_info, active); +break ;case BTRFS_BLOCK_GROUP_RAID1 :case BTRFS_BLOCK_GROUP_RAID1C3 :case BTRFS_BLOCK_GROUP_RAID1C4 :
Signature · 1 line
2.48.1
PATCH v2 9/12 btrfs-progs: zoned: implement RAID1 zone info loading
In reply to Naohiro AotaImplement it just like the kernel side.
--- kernel-shared/zoned.c | 46 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
kernel-shared/zoned.c +46 -0
@@ -1018,6 +1018,50 @@ static int btrfs_load_block_group_dup(struct btrfs_fs_info *fs_info,return 0 ; } +static int btrfs_load_block_group_raid1 (struct btrfs_fs_info *fs_info, +struct btrfs_block_group *bg, +struct map_lookup *map, +struct zone_info *zone_info, +unsigned long *active) +{ +int i; + +if ((map->type &BTRFS_BLOCK_GROUP_DATA ) && !fs_info->stripe_root) { +btrfs_err (fs_info,"zoned: data %s needs raid-stripe-tree" , +btrfs_bg_type_to_raid_name (map->type)); +return -EINVAL ; + } + +/* In case a device is missing we have a cap of 0, so don't use it. */ + bg->zone_capacity =min_not_zero (zone_info[0 ].capacity, zone_info[1 ].capacity); + +for (i =0 ; i < map->num_stripes; i++) { +if (zone_info[i].alloc_offset ==WP_MISSING_DEV || + zone_info[i].alloc_offset ==WP_CONVENTIONAL ) +continue ; + +if (zone_info[0 ].alloc_offset != zone_info[i].alloc_offset) { +btrfs_err (fs_info, +"zoned: write pointer offset mismatch of zones in %s profile" , +btrfs_bg_type_to_raid_name (map->type)); +return -EIO ; + } +if (test_bit (0 , active) !=test_bit (i, active)) { +return -EIO ; + }else { +if (test_bit (0 , active)) + bg->zone_is_active =1 ; + } + } + +if (zone_info[0 ].alloc_offset !=WP_MISSING_DEV ) + bg->alloc_offset = zone_info[0 ].alloc_offset; +else + bg->alloc_offset = zone_info[i -1 ].alloc_offset; + +return 0 ; +} +int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache) {
@@ -1112,6 +1156,8 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info,case BTRFS_BLOCK_GROUP_RAID1 :case BTRFS_BLOCK_GROUP_RAID1C3 :case BTRFS_BLOCK_GROUP_RAID1C4 : + ret =btrfs_load_block_group_raid1 (fs_info, cache, map, zone_info, active); +break ;case BTRFS_BLOCK_GROUP_RAID0 :case BTRFS_BLOCK_GROUP_RAID10 :/* Temporarily fails these case, until following commits. */
Signature · 1 line
2.48.1
PATCH v2 10/12 btrfs-progs: zoned: implement RAID0 zone info loading
In reply to Naohiro AotaImplement it just like the kernel side.
--- kernel-shared/zoned.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
kernel-shared/zoned.c +32 -0
@@ -1062,6 +1062,36 @@ static int btrfs_load_block_group_raid1(struct btrfs_fs_info *fs_info,return 0 ; } +static int btrfs_load_block_group_raid0 (struct btrfs_fs_info *fs_info, +struct btrfs_block_group *bg, +struct map_lookup *map, +struct zone_info *zone_info, +unsigned long *active) +{ +if ((map->type &BTRFS_BLOCK_GROUP_DATA ) && !fs_info->stripe_root) { +btrfs_err (fs_info,"zoned: data %s needs raid-stripe-tree" , +btrfs_bg_type_to_raid_name (map->type)); +return -EINVAL ; + } + +for (int i =0 ; i < map->num_stripes; i++) { +if (zone_info[i].alloc_offset ==WP_MISSING_DEV || + zone_info[i].alloc_offset ==WP_CONVENTIONAL ) +continue ; + +if (test_bit (0 , active) !=test_bit (i, active)) { +return -EIO ; + }else { +if (test_bit (0 , active)) + bg->zone_is_active =1 ; + } + bg->zone_capacity += zone_info[i].capacity; + bg->alloc_offset += zone_info[i].alloc_offset; + } + +return 0 ; +} +int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache) {
@@ -1159,6 +1189,8 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret =btrfs_load_block_group_raid1 (fs_info, cache, map, zone_info, active);break ;case BTRFS_BLOCK_GROUP_RAID0 : + ret =btrfs_load_block_group_raid0 (fs_info, cache, map, zone_info, active); +break ;case BTRFS_BLOCK_GROUP_RAID10 :/* Temporarily fails these case, until following commits. */ fallthrough;
Signature · 1 line
2.48.1
PATCH v2 11/12 btrfs-progs: implement RAID10 zone info loading
In reply to Naohiro AotaJust same as the kernel side.
--- kernel-shared/zoned.c | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-)
kernel-shared/zoned.c +35 -2
@@ -1092,6 +1092,39 @@ static int btrfs_load_block_group_raid0(struct btrfs_fs_info *fs_info,return 0 ; } +static int btrfs_load_block_group_raid10 (struct btrfs_fs_info *fs_info, +struct btrfs_block_group *bg, +struct map_lookup *map, +struct zone_info *zone_info, +unsigned long *active) +{ +if ((map->type &BTRFS_BLOCK_GROUP_DATA ) && !fs_info->stripe_root) { +btrfs_err (fs_info,"zoned: data %s needs raid-stripe-tree" , +btrfs_bg_type_to_raid_name (map->type)); +return -EINVAL ; + } + +for (int i =0 ; i < map->num_stripes; i++) { +if (zone_info[i].alloc_offset ==WP_MISSING_DEV || + zone_info[i].alloc_offset ==WP_CONVENTIONAL ) +continue ; + +if (test_bit (0 , active) !=test_bit (i, active)) { +return -EIO ; + }else { +if (test_bit (0 , active)) + bg->zone_is_active =1 ; + } + +if ((i % map->sub_stripes) ==0 ) { + bg->zone_capacity += zone_info[i].capacity; + bg->alloc_offset += zone_info[i].alloc_offset; + } + } + +return 0 ; +} +int btrfs_load_block_group_zone_info (struct btrfs_fs_info *fs_info,struct btrfs_block_group *cache) {
@@ -1192,8 +1225,8 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret =btrfs_load_block_group_raid0 (fs_info, cache, map, zone_info, active);break ;case BTRFS_BLOCK_GROUP_RAID10 : -/* Temporarily fails these case, until following commits. */ - fallthrough; + ret =btrfs_load_block_group_raid10 (fs_info, cache, map, zone_info, active); +break ;case BTRFS_BLOCK_GROUP_RAID5 :case BTRFS_BLOCK_GROUP_RAID6 :default :
Signature · 1 line
2.48.1
PATCH v2 12/12 btrfs-progs: zoned: fix alloc_offset calculation for partly conventional block groups
In reply to Naohiro AotaWhen one of two zones composing a DUP block group is a conventional zone, we have the zone_info[i]->alloc_offset = WP_CONVENTIONAL. That will, of course, not match the write pointer of the other zone, and fails that block group.
This commit solves that issue by properly recovering the emulated write pointer from the last allocated extent. The offset for the SINGLE, DUP, and RAID1 are
straight-forward: it is same as the end of last allocated extent. The RAID0 and RAID10 are a bit tricky that we need to do the math of striping.
--- kernel-shared/zoned.c | 65 +++++++++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 14 deletions(-)
kernel-shared/zoned.c +51 -14
@@ -981,7 +981,7 @@ static int btrfs_load_block_group_dup(struct btrfs_fs_info *fs_info, struct btrfs_block_group *bg, struct map_lookup *map, struct zone_info *zone_info, - unsigned long *active) + unsigned long *active, u64 last_alloc) { if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) { btrfs_err(fs_info, "zoned: data DUP profile needs raid-stripe-tree");
@@ -1002,6 +1002,12 @@ static int btrfs_load_block_group_dup(struct btrfs_fs_info *fs_info, zone_info[1].physical);return -EIO ; } + +if (zone_info[0 ].alloc_offset ==WP_CONVENTIONAL ) + zone_info[0 ].alloc_offset = last_alloc; +if (zone_info[1 ].alloc_offset ==WP_CONVENTIONAL ) + zone_info[1 ].alloc_offset = last_alloc; +if (zone_info[0 ].alloc_offset != zone_info[1 ].alloc_offset) {btrfs_err (fs_info,"zoned: write pointer offset mismatch of zones in DUP profile" );
@@ -1022,7 +1028,7 @@ static int btrfs_load_block_group_raid1(struct btrfs_fs_info *fs_info,struct btrfs_block_group *bg, struct map_lookup *map, struct zone_info *zone_info, - unsigned long *active) + unsigned long *active, u64 last_alloc) { int i;
@@ -1036,9 +1042,10 @@ static int btrfs_load_block_group_raid1(struct btrfs_fs_info *fs_info, bg->zone_capacity =min_not_zero (zone_info[0 ].capacity, zone_info[1 ].capacity);for (i =0 ; i < map->num_stripes; i++) { -if (zone_info[i].alloc_offset ==WP_MISSING_DEV || - zone_info[i].alloc_offset ==WP_CONVENTIONAL ) +if (zone_info[i].alloc_offset ==WP_MISSING_DEV )continue ; +if (zone_info[i].alloc_offset ==WP_CONVENTIONAL ) + zone_info[i].alloc_offset = last_alloc;if (zone_info[0 ].alloc_offset != zone_info[i].alloc_offset) { btrfs_err(fs_info,
@@ -1066,7 +1073,7 @@ static int btrfs_load_block_group_raid0(struct btrfs_fs_info *fs_info, struct btrfs_block_group *bg, struct map_lookup *map, struct zone_info *zone_info, - unsigned long *active) + unsigned long *active, u64 last_alloc) { if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) { btrfs_err(fs_info, "zoned: data %s needs raid-stripe-tree",
@@ -1075,9 +1082,24 @@ static int btrfs_load_block_group_raid0(struct btrfs_fs_info *fs_info, }for (int i =0 ; i < map->num_stripes; i++) { -if (zone_info[i].alloc_offset ==WP_MISSING_DEV || - zone_info[i].alloc_offset ==WP_CONVENTIONAL ) +if (zone_info[i].alloc_offset ==WP_MISSING_DEV )continue ; +if (zone_info[i].alloc_offset ==WP_CONVENTIONAL ) { +u64 stripe_nr, full_stripe_nr; +u64 stripe_offset; +int stripe_index; + + stripe_nr = last_alloc / map->stripe_len; + stripe_offset = stripe_nr * map->stripe_len; + full_stripe_nr = stripe_nr / map->num_stripes; + stripe_index = stripe_nr % map->num_stripes; + + zone_info[i].alloc_offset = full_stripe_nr * map->stripe_len; +if (stripe_index > i) + zone_info[i].alloc_offset += map->stripe_len; +else if (stripe_index == i) + zone_info[i].alloc_offset += (last_alloc - stripe_offset); + }if (test_bit (0 , active) !=test_bit (i, active)) {return -EIO ;
@@ -1096,7 +1118,7 @@ static int btrfs_load_block_group_raid10(struct btrfs_fs_info *fs_info, struct btrfs_block_group *bg, struct map_lookup *map, struct zone_info *zone_info, - unsigned long *active) + unsigned long *active, u64 last_alloc) { if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) { btrfs_err(fs_info, "zoned: data %s needs raid-stripe-tree",
@@ -1105,9 +1127,24 @@ static int btrfs_load_block_group_raid10(struct btrfs_fs_info *fs_info, }for (int i =0 ; i < map->num_stripes; i++) { -if (zone_info[i].alloc_offset ==WP_MISSING_DEV || - zone_info[i].alloc_offset ==WP_CONVENTIONAL ) +if (zone_info[i].alloc_offset ==WP_MISSING_DEV )continue ; +if (zone_info[i].alloc_offset ==WP_CONVENTIONAL ) { +u64 stripe_nr, full_stripe_nr; +u64 stripe_offset; +int stripe_index; + + stripe_nr = last_alloc / map->stripe_len; + stripe_offset = stripe_nr * map->stripe_len; + full_stripe_nr = stripe_nr / (map->num_stripes / map->sub_stripes); + stripe_index = stripe_nr % (map->num_stripes / map->sub_stripes); + + zone_info[i].alloc_offset = full_stripe_nr * map->stripe_len; +if (stripe_index > (i / map->sub_stripes)) + zone_info[i].alloc_offset += map->stripe_len; +else if (stripe_index == (i / map->sub_stripes)) + zone_info[i].alloc_offset += (last_alloc - stripe_offset); + }if (test_bit (0 , active) !=test_bit (i, active)) {return -EIO ;
@@ -1214,18 +1251,18 @@ int btrfs_load_block_group_zone_info(struct btrfs_fs_info *fs_info, ret =btrfs_load_block_group_single (fs_info, cache, &zone_info[0 ], active);break ;case BTRFS_BLOCK_GROUP_DUP : - ret =btrfs_load_block_group_dup (fs_info, cache, map, zone_info, active); + ret =btrfs_load_block_group_dup (fs_info, cache, map, zone_info, active, last_alloc);break ;case BTRFS_BLOCK_GROUP_RAID1 :case BTRFS_BLOCK_GROUP_RAID1C3 :case BTRFS_BLOCK_GROUP_RAID1C4 : - ret =btrfs_load_block_group_raid1 (fs_info, cache, map, zone_info, active); + ret =btrfs_load_block_group_raid1 (fs_info, cache, map, zone_info, active, last_alloc);break ;case BTRFS_BLOCK_GROUP_RAID0 : - ret =btrfs_load_block_group_raid0 (fs_info, cache, map, zone_info, active); + ret =btrfs_load_block_group_raid0 (fs_info, cache, map, zone_info, active, last_alloc);break ;case BTRFS_BLOCK_GROUP_RAID10 : - ret =btrfs_load_block_group_raid10 (fs_info, cache, map, zone_info, active); + ret =btrfs_load_block_group_raid10 (fs_info, cache, map, zone_info, active, last_alloc);break ;case BTRFS_BLOCK_GROUP_RAID5 :case BTRFS_BLOCK_GROUP_RAID6 :
Signature · 1 line
2.48.1
Re: [PATCH v2 12/12] btrfs-progs: zoned: fix alloc_offset calculation for partly conventional block groups
Quoted 8 lines · Naohiro Aota
When one of two zones composing a DUP block group is a conventional zone, we have the zone_info[i]->alloc_offset =
WP_CONVENTIONAL. That will, of course, not match the write pointer of the other zone, and fails that block group.This commit solves that issue by properly recovering the emulated write pointer from the last allocated extent. The offset for the SINGLE, DUP, and RAID1 are
straight-forward: it is same as the end of last allocated extent. The RAID0 and RAID10 are a bit tricky that we need to do the math of striping.
I wonder if we need this patch on the kernel side as well. Can happen there too.
Re: [PATCH v2 12/12] btrfs-progs: zoned: fix alloc_offset calculation for partly conventional block groups
Quoted 5 lines · Johannes Thumshirn
Quoted 4 lines · Naohiro Aota
When one of two zones composing a DUP block group is a conventional zone, we have the zone_info[i]->alloc_offset =
WP_CONVENTIONAL. That will, of course, not match the write pointer of the other zone, and fails that block group.
Quoted 7 lines
Quoted 4 lines
This commit solves that issue by properly recovering the emulated write pointer from the last allocated extent. The offset for the SINGLE, DUP, and RAID1 are
straight-forward: it is same as the end of last allocated extent. The RAID0 and RAID10 are a bit tricky that we need to do the math of striping.I wonder if we need this patch on the kernel side as well. Can happen there too.
Yes. I'm going to apply this one to kernel side too.
Looks good to me,
Quoted 57 lines · Naohiro Aota
Running mkfs.btrfs on a
null_blkdevice with the following setup fails as below.- zone size: 64MB
- zone capacity: 64MB
- number of conventional zones: 6
- storage size: 2048MB+
/home/naota/src/btrfs-progs/mkfs.btrfs-d single -m dup -f/dev/nullb0btrfs-progs v6.10
See https://btrfs.readthedocs.io for more information.zoned:
/dev/nullb0: host-managed device detected, setting zoned feature Resetting device zones/dev/nullb0(32 zones) ...
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)bad tree block 268435456, bytenr mismatch, want=268435456, have=0
kernel-shared/disk-io.c:485:write_tree_block:BUG_ON1triggered, value 1 /home/naota/src/btrfs-progs/mkfs.btrfs(+0x290ca) [0x55603cf7e0ca] /home/naota/src/btrfs-progs/mkfs.btrfs(write_tree_block+0xa7) [0x55603cf80417] /home/naota/src/btrfs-progs/mkfs.btrfs(__commit_transaction+0xe8) [0x55603cf9b7d8] /home/naota/src/btrfs-progs/mkfs.btrfs(btrfs_commit_transaction+0x176) [0x55603cf9ba66] /home/naota/src/btrfs-progs/mkfs.btrfs(main+0x2831) [0x55603cf67291] /usr/lib64/libc.so.6(+0x271ee) [0x7f5ab706f1ee] /usr/lib64/libc.so.6(__libc_start_main+0x89) [0x7f5ab706f2a9] /home/naota/src/btrfs-progs/mkfs.btrfs(_start+0x25) [0x55603cf6a135]
/home/naota/tmp/test-mkfs.sh: line 13: 821886 Aborted (core dumped)The crash happens because btrfs-progs failed to set proper allocation pointer when a DUP block group is created over a conventional zone and a sequential write required zone. In that case, the write pointer is recovered from the last allocated extent in the block group. That functionality is not well implemented in btrfs-progs side.
Implementing that functionality is relatively trivial because we can copy the code from the kernel side. However, the code is quite out of sync between the kernel side and user space side. So, this series first refactors
btrfs_load_block_group_zone_info()to make it easy to integrate the code from the kernel side.The main part is the last patch, which fixes allocation pointer calculation for all the profiles.
While at it, this series also adds support for zone capacity and zone activeness. But, zone activeness support is currently limited. It does not attempt to check the zone active limit on the extent allocation, because mkfs.btrfs should work without hitting the limit.
- v2
- Temporarily fails some profiles while adding supports in the patch series.
- v1: https://lore.kernel.org/linux-btrfs/[email protected]/
Quoted 15 lines
Naohiro Aota (12):
btrfs-progs: introducemin_not_zero()
btrfs-progs: zoned: introduce azone_infostruct in
btrfs_load_block_group_zone_info
btrfs-progs: zoned: support zone capacity
btrfs-progs: zoned: load zone activeness
btrfs-progs: zoned: activate block group on loading
btrfs-progs: factor outbtrfs_load_zone_info()
btrfs-progs: zoned: factor out SINGLE zone info loading
btrfs-progs: zoned: implement DUP zone info loading
btrfs-progs: zoned: implement RAID1 zone info loading
btrfs-progs: zoned: implement RAID0 zone info loading
btrfs-progs: implement RAID10 zone info loading
btrfs-progs: zoned: fixalloc_offsetcalculation for partly
conventional block groups
Added to devel, thanks.