btrfs: add a btrfs_chunk_map structure and preparatory cleanups
Subsystems (2)
cover btrfs: add a btrfs_chunk_map structure and preparatory cleanups
From: Filipe Manana <[email protected]>
The following are some cleanups and introducing a dedicated data structure for representing chunk maps, in order to make code simpler and use less memory - this is achieved in patch 7/8. This patchset is also preparatory work for some upcoming changes to extent maps.
Filipe Manana (8):
btrfs: fix off-by-one when checking chunk map includes logical address
btrfs: make error messages more clear when getting a chunk map
btrfs: mark sanity checks when getting chunk map as unlikely
btrfs: split assert into two different asserts when removing block group
btrfs: unexport extent_map_block_end()
btrfs: use btrfs_next_item() at scrub.c:find_first_extent_item()
btrfs: use a dedicated data structure for chunk maps
btrfs: remove stripe size local variable from insert_dev_extents()
fs/btrfs/block-group.c | 167 ++++----- fs/btrfs/block-group.h | 6 +- fs/btrfs/dev-replace.c | 28 +- fs/btrfs/disk-io.c | 7 +- fs/btrfs/extent_map.c | 53 +-- fs/btrfs/extent_map.h | 11 - fs/btrfs/fs.h | 3 +- fs/btrfs/inode.c | 25 +- fs/btrfs/raid56.h | 2 +- fs/btrfs/scrub.c | 52 ++- fs/btrfs/tests/btrfs-tests.c | 3 +- fs/btrfs/tests/btrfs-tests.h | 1 + fs/btrfs/tests/extent-map-tests.c | 40 +-- fs/btrfs/volumes.c | 545 ++++++++++++++++++------------ fs/btrfs/volumes.h | 45 ++- fs/btrfs/zoned.c | 24 +- include/trace/events/btrfs.h | 11 +- 17 files changed, 516 insertions(+), 507 deletions(-)
Signature · 1 line
2.40.1
PATCH 1/8 btrfs: fix off-by-one when checking chunk map includes logical address
From: Filipe Manana <[email protected]>
At btrfs_get_chunk_map() we get the extent map for the chunk that contains the given logical address stored in the 'logical' argument. Then we do sanity checks to verify the extent map contains the logical address. One of these checks verifies if the extent map covers a range with an end offset behind the target logical address - however this check has an off-by-one error since it will consider an extent map whose start offset plus its length matches the target logical address as inclusive, while the fact is that the last byte it covers is behind the target logical address (by 1).
So fix this condition by using '<=' rather than '<' when comparing the extent map's "start + length" against the target logical address.
--- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/volumes.c +1 -1
@@ -3008,7 +3008,7 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,return ERR_PTR (-EINVAL ); } -if (em->start > logical || em->start + em->len < logical) { +if (em->start > logical || em->start + em->len <= logical) {btrfs_crit (fs_info,"found a bad mapping, wanted %llu-%llu, found %llu-%llu" , logical, length, em->start, em->start + em->len);
Signature · 1 line
2.40.1
PATCH 2/8 btrfs: make error messages more clear when getting a chunk map
In reply to [email protected]From: Filipe Manana <[email protected]>
When getting a chunk map, at btrfs_get_chunk_map(), we do some sanity checks to verify we found a chunk map and that map found covers the logical address the caller passed in. However the messages aren't very clear in the sense that don't mention the issue is with a chunk map and one of them prints the 'length' argument as if it were the end offset of the requested range (while the in the string format we use %llu-%llu which suggests a range, and the second %llu-%llu is actually a range for the chunk map). So improve these two details in the error messages.
--- fs/btrfs/volumes.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
fs/btrfs/volumes.c +4 -3
@@ -3003,15 +3003,16 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,read_unlock (&em_tree->lock);if (!em) { -btrfs_crit (fs_info,"unable to find logical %llu length %llu" , +btrfs_crit (fs_info, +"unable to find chunk map for logical %llu length %llu" , logical, length);return ERR_PTR (-EINVAL ); }if (em->start > logical || em->start + em->len <= logical) {btrfs_crit (fs_info, -"found a bad mapping, wanted %llu-%llu, found %llu-%llu" , - logical, length, em->start, em->start + em->len); +"found a bad chunk map, wanted %llu-%llu, found %llu-%llu" , + logical, logical + length, em->start, em->start + em->len);free_extent_map (em);return ERR_PTR (-EINVAL ); }
Signature · 1 line
2.40.1
PATCH 3/8 btrfs: mark sanity checks when getting chunk map as unlikely
In reply to [email protected]From: Filipe Manana <[email protected]>
When getting a chunk map, at btrfs_get_chunk_map(), we do some sanity checks to verify that we found an extent map and that it includes the requested logical address. These are never expected to fail, so mark them as unlikely to make it more clear as well as to allow a compiler to generate more efficient code.
--- fs/btrfs/volumes.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/volumes.c +2 -2
@@ -3002,14 +3002,14 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info, em =lookup_extent_mapping (em_tree, logical, length);read_unlock (&em_tree->lock); -if (!em) { +if (unlikely (!em)) {btrfs_crit (fs_info,"unable to find chunk map for logical %llu length %llu" , logical, length);return ERR_PTR (-EINVAL ); } -if (em->start > logical || em->start + em->len <= logical) { +if (unlikely (em->start > logical || em->start + em->len <= logical)) {btrfs_crit (fs_info,"found a bad chunk map, wanted %llu-%llu, found %llu-%llu" , logical, logical + length, em->start, em->start + em->len);
Signature · 1 line
2.40.1
PATCH 4/8 btrfs: split assert into two different asserts when removing block group
In reply to [email protected]From: Filipe Manana <[email protected]>
When starting a transaction to remove a block group we have one ASSERT that checks we found an extent map and that the extent map's start offset matches the desired chunk offset. In case one of the conditions fails, we get a stack trace that point to the respective line of code, however we can't tell which condition failed: either there's no extent map or we got one with an unexpected start offset. To make such an issue easier to debug and analyse, split the assertion into two, one for each condition. This was actually triggered during development of another upcoming change.
--- fs/btrfs/block-group.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/btrfs/block-group.c +2 -1
@@ -1303,7 +1303,8 @@ struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(read_lock (&em_tree->lock); em =lookup_extent_mapping (em_tree, chunk_offset,1 );read_unlock (&em_tree->lock); -ASSERT (em && em->start == chunk_offset); +ASSERT (em !=NULL ); +ASSERT (em->start == chunk_offset);/* * We need to reserve 3 + N units from the metadata space info in order
Signature · 1 line
2.40.1
PATCH 5/8 btrfs: unexport extent_map_block_end()
In reply to [email protected]From: Filipe Manana <[email protected]>
The helper extent_map_block_end() is currently not used anywhere outside extent_map.c, so move into from extent_map.h into extent_map.c. While at it, also make the extent map pointer argument as const.
--- fs/btrfs/extent_map.c | 7 +++++++ fs/btrfs/extent_map.h | 7 ------- 2 files changed, 7 insertions(+), 7 deletions(-)
fs/btrfs/extent_map.c +7 -0
@@ -182,6 +182,13 @@ static struct rb_node *__tree_search(struct rb_root *root, u64 offset,return NULL ; } +static inline u64 extent_map_block_end (const struct extent_map *em) +{ +if (em->block_start + em->block_len < em->block_start) +return (u64)-1 ; +return em->block_start + em->block_len; +} +/* Check to see if two extent_map structs are adjacent and safe to merge. */ static int mergable_maps (struct extent_map *prev,struct extent_map *next) {
fs/btrfs/extent_map.h +0 -7
@@ -77,13 +77,6 @@ static inline u64 extent_map_end(struct extent_map *em)return em->start + em->len; } -static inline u64 extent_map_block_end (struct extent_map *em) -{ -if (em->block_start + em->block_len < em->block_start) -return (u64)-1 ; -return em->block_start + em->block_len; -} -void extent_map_tree_init (struct extent_map_tree *tree);struct extent_map *lookup_extent_mapping (struct extent_map_tree *tree,u64 start,u64 len);
Signature · 1 line
2.40.1
PATCH 6/8 btrfs: use btrfs_next_item() at scrub.c:find_first_extent_item()
In reply to [email protected]From: Filipe Manana <[email protected]>
There's no reason to open code what btrfs_next_item() does when searching for extent items at scrub.c:scrub.c:find_first_extent_item(), so remove the logic to find the next item and use btrfs_next_item() instead, making the code shorter and less nested code blocks. While at it also fix the comment to the plural "items" instead of "item" and end it with proper punctuation.
--- fs/btrfs/scrub.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-)
fs/btrfs/scrub.c +5 -8
@@ -1409,14 +1409,11 @@ static int find_first_extent_item(struct btrfs_root *extent_root,if (ret >0 )break ; next: - path->slots[0 ]++; -if (path->slots[0 ] >=btrfs_header_nritems (path->nodes[0 ])) { - ret =btrfs_next_leaf (extent_root, path); -if (ret) { -/* Either no more item or fatal error */ -btrfs_release_path (path); -return ret; - } + ret =btrfs_next_item (extent_root, path); +if (ret) { +/* Either no more items or a fatal error. */ +btrfs_release_path (path); +return ret; } }btrfs_release_path (path);
Signature · 1 line
2.40.1
PATCH 7/8 btrfs: use a dedicated data structure for chunk maps
In reply to [email protected]From: Filipe Manana <[email protected]>
Currently we abuse the extent_map structure for two purposes:
1) To actually represent extents for inodes;
2) To represent chunk mappings.
This is odd and has several disadvantages:
1) To create a chunk map, we need to do two memory allocations: one for an extent_map structure and another one for a map_lookup structure, so more potential for an allocation failure and more complicated code to manage and link two structures;
2) For a chunk map we actually only use 3 fields (24 bytes) of the respective extent map structure: the 'start' field to have the logical start address of the chunk, the 'len' field to have the chunk's size, and the 'orig_block_len' field to contain the chunk's stripe size.
Besides wasting a memory, it's also odd and not intuitive at all to have the stripe size in a field named 'orig_block_len'.
We are also using 'block_len' of the extent_map structure to contain the chunk size, so we have 2 fields for the same value, 'len' and 'block_len', which is pointless;
3) When an extent map is associated to a chunk mapping, we set the bit EXTENT_FLAG_FS_MAPPING on its flags and then make its member named 'map_lookup' point to the associated map_lookup structure. This means that for an extent map associated to an inode extent, we are not using this 'map_lookup' pointer, so wasting 8 bytes (on a 64 bits platform);
4) Extent maps associated to a chunk mapping are never merged or split so it's pointless to use the existing extent map infrastructure.
So add a dedicated data structure named 'btrfs_chunk_map' to represent chunk mappings, this is basically the existing map_lookup structure with some extra fields:
1) 'start' to contain the chunk logical address;
2) 'chunk_len' to contain the chunk's length;
3) 'stripe_size' for the stripe size;
4) 'rb_node' for insertion into a rb tree;
5) 'refs' for reference counting.
This way we do a single memory allocation for chunk mappings and we don't waste memory for them with unused/unnecessary fields from an extent_map.
We also save 8 bytes from the extent_map structure by removing the 'map_lookup' pointer, so the size of struct extent_map is reduced from 144 bytes down to 136 bytes, and we can now have 30 extents map per 4K page instead of 28.
--- fs/btrfs/block-group.c | 165 ++++----- fs/btrfs/block-group.h | 6 +- fs/btrfs/dev-replace.c | 28 +- fs/btrfs/disk-io.c | 7 +- fs/btrfs/extent_map.c | 46 --- fs/btrfs/extent_map.h | 4 - fs/btrfs/fs.h | 3 +- fs/btrfs/inode.c | 25 +- fs/btrfs/raid56.h | 2 +- fs/btrfs/scrub.c | 39 +-- fs/btrfs/tests/btrfs-tests.c | 3 +- fs/btrfs/tests/btrfs-tests.h | 1 + fs/btrfs/tests/extent-map-tests.c | 40 +-- fs/btrfs/volumes.c | 540 ++++++++++++++++++------------ fs/btrfs/volumes.h | 45 ++- fs/btrfs/zoned.c | 24 +- include/trace/events/btrfs.h | 11 +- 17 files changed, 501 insertions(+), 488 deletions(-)
fs/btrfs/block-group.c +68 -97
@@ -168,7 +168,7 @@ void btrfs_put_block_group(struct btrfs_block_group *cache) cache);kfree (cache->free_space_ctl); -kfree (cache->physical_map); +btrfs_free_chunk_map (cache->physical_map);kfree (cache); } }
@@ -1047,7 +1047,7 @@ static int remove_block_group_item(struct btrfs_trans_handle *trans, }int btrfs_remove_block_group (struct btrfs_trans_handle *trans, -u64 group_start,struct extent_map *em) +struct btrfs_chunk_map *map) {struct btrfs_fs_info *fs_info = trans->fs_info;struct btrfs_path *path;
@@ -1059,10 +1059,10 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,int index;int factor;struct btrfs_caching_control *caching_ctl =NULL ; -bool remove_em; +bool remove_map;bool remove_rsv = false; - block_group =btrfs_lookup_block_group (fs_info, group_start); + block_group =btrfs_lookup_block_group (fs_info, map->start);BUG_ON (!block_group);BUG_ON (!block_group->ro);
@@ -1252,7 +1252,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,* entries because we already removed them all when we called * btrfs_remove_free_space_cache(). * -* And we must not remove the extent map from the fs_info->mapping_tree +* And we must not remove the chunk map from the fs_info->mapping_tree * to prevent the same logical address range and physical device space * ranges from being reused for a new block group. This is needed to * avoid races with trimming and scrub.
@@ -1268,19 +1268,11 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,* in place until the extents have been discarded completely when * the transaction commit has completed. */ - remove_em = (atomic_read (&block_group->frozen) ==0 ); + remove_map = (atomic_read (&block_group->frozen) ==0 );spin_unlock (&block_group->lock); -if (remove_em) { -struct extent_map_tree *em_tree; - - em_tree = &fs_info->mapping_tree; -write_lock (&em_tree->lock); -remove_extent_mapping (em_tree, em); -write_unlock (&em_tree->lock); -/* once for the tree */ -free_extent_map (em); - } +if (remove_map) +btrfs_remove_chunk_map (fs_info, map); out:/* Once for the lookup reference */
@@ -1295,16 +1287,12 @@ struct btrfs_trans_handle *btrfs_start_trans_remove_block_group( struct btrfs_fs_info *fs_info, const u64 chunk_offset) { struct btrfs_root *root = btrfs_block_group_root(fs_info); -struct extent_map_tree *em_tree = &fs_info->mapping_tree; -struct extent_map *em; -struct map_lookup *map; +struct btrfs_chunk_map *map;unsigned int num_items; -read_lock (&em_tree->lock); - em =lookup_extent_mapping (em_tree, chunk_offset,1 ); -read_unlock (&em_tree->lock); -ASSERT (em !=NULL ); -ASSERT (em->start == chunk_offset); + map =btrfs_find_chunk_map (fs_info, chunk_offset,1 ); +ASSERT (map !=NULL ); +ASSERT (map->start == chunk_offset);/* * We need to reserve 3 + N units from the metadata space info in order
@@ -1325,9 +1313,8 @@ struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(* more device items and remove one chunk item), but this is done at * btrfs_remove_chunk() through a call to check_system_chunk(). */ - map = em->map_lookup; num_items =3 + map->num_stripes; -free_extent_map (em); +btrfs_free_chunk_map (map);return btrfs_start_transaction_fallback_global_rsv (root, num_items); }
@@ -1928,8 +1915,7 @@ void btrfs_mark_bg_to_reclaim(struct btrfs_block_group *bg)static int read_bg_from_eb (struct btrfs_fs_info *fs_info,struct btrfs_key *key,struct btrfs_path *path) { -struct extent_map_tree *em_tree; -struct extent_map *em; +struct btrfs_chunk_map *map;struct btrfs_block_group_item bg;struct extent_buffer *leaf;int slot;
@@ -1939,23 +1925,20 @@ static int read_bg_from_eb(struct btrfs_fs_info *fs_info, struct btrfs_key *key, slot = path->slots[0 ]; leaf = path->nodes[0 ]; - em_tree = &fs_info->mapping_tree; -read_lock (&em_tree->lock); - em =lookup_extent_mapping (em_tree, key->objectid, key->offset); -read_unlock (&em_tree->lock); -if (!em) { + map =btrfs_find_chunk_map (fs_info, key->objectid, key->offset); +if (!map) {btrfs_err (fs_info,"logical %llu len %llu found bg but no related chunk" , key->objectid, key->offset);return -ENOENT ; } -if (em->start != key->objectid || em->len != key->offset) { +if (map->start != key->objectid || map->chunk_len != key->offset) {btrfs_err (fs_info,"block group %llu len %llu mismatch with chunk %llu len %llu" , - key->objectid, key->offset, em->start, em->len); + key->objectid, key->offset, map->start, map->chunk_len); ret = -EUCLEAN ; - goto out_free_em; + goto out_free_map; } read_extent_buffer(leaf, &bg, btrfs_item_ptr_offset(leaf, slot),
@@ -1963,16 +1946,16 @@ static int read_bg_from_eb(struct btrfs_fs_info *fs_info, struct btrfs_key *key, flags =btrfs_stack_block_group_flags (&bg) &BTRFS_BLOCK_GROUP_TYPE_MASK ; -if (flags != (em->map_lookup->type &BTRFS_BLOCK_GROUP_TYPE_MASK )) { +if (flags != (map->type &BTRFS_BLOCK_GROUP_TYPE_MASK )) {btrfs_err (fs_info,"block group %llu len %llu type flags 0x%llx mismatch with chunk type flags 0x%llx" , key->objectid, key->offset, flags, - (BTRFS_BLOCK_GROUP_TYPE_MASK & em->map_lookup->type)); + (BTRFS_BLOCK_GROUP_TYPE_MASK & map->type)); ret = -EUCLEAN ; } -out_free_em: -free_extent_map (em); +out_free_map: +btrfs_free_chunk_map (map);return ret; }
@@ -2025,8 +2008,7 @@ static void set_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)int btrfs_rmap_block (struct btrfs_fs_info *fs_info,u64 chunk_start,u64 physical,u64 **logical,int *naddrs,int *stripe_len) { -struct extent_map *em; -struct map_lookup *map; +struct btrfs_chunk_map *map;u64 *buf;u64 bytenr;u64 data_stripe_length;
@@ -2034,14 +2016,13 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,int i, nr =0 ;int ret =0 ; - em =btrfs_get_chunk_map (fs_info, chunk_start,1 ); -if (IS_ERR (em)) + map =btrfs_get_chunk_map (fs_info, chunk_start,1 ); +if (IS_ERR (map))return -EIO ; - map = em->map_lookup; - data_stripe_length = em->orig_block_len; + data_stripe_length = map->stripe_size; io_stripe_size =BTRFS_STRIPE_LEN ; - chunk_start = em->start; + chunk_start = map->start;/* For RAID5/6 adjust to a full IO stripe length */ if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK )
@@ -2095,7 +2076,7 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start, *naddrs = nr; *stripe_len = io_stripe_size; out: -free_extent_map (em); +btrfs_free_chunk_map (map);return ret; }
@@ -2200,49 +2181,47 @@ static struct btrfs_block_group *btrfs_create_block_group_cache(*/ static int check_chunk_block_group_mappings (struct btrfs_fs_info *fs_info) { -struct extent_map_tree *map_tree = &fs_info->mapping_tree; -struct extent_map *em; -struct btrfs_block_group *bg;u64 start =0 ;int ret =0 ;while (1 ) { -read_lock (&map_tree->lock); +struct btrfs_chunk_map *map; +struct btrfs_block_group *bg; +/* -* lookup_extent_mapping will return the first extent map -* intersecting the range, so setting @len to 1 is enough to +* btrfs_find_chunk_map() will return the first chunk map +* intersecting the range, so setting @length to 1 is enough to * get the first chunk. */ - em =lookup_extent_mapping (map_tree, start,1 ); -read_unlock (&map_tree->lock); -if (!em) + map =btrfs_find_chunk_map (fs_info, start,1 ); +if (!map)break ; - bg =btrfs_lookup_block_group (fs_info, em->start); + bg =btrfs_lookup_block_group (fs_info, map->start);if (!bg) {btrfs_err (fs_info,"chunk start=%llu len=%llu doesn't have corresponding block group" , - em->start, em->len); + map->start, map->chunk_len); ret = -EUCLEAN ; -free_extent_map (em); +btrfs_free_chunk_map (map);break ; } -if (bg->start != em->start || bg->length != em->len || +if (bg->start != map->start || bg->length != map->chunk_len || (bg->flags &BTRFS_BLOCK_GROUP_TYPE_MASK ) != - (em->map_lookup->type &BTRFS_BLOCK_GROUP_TYPE_MASK )) { + (map->type &BTRFS_BLOCK_GROUP_TYPE_MASK )) {btrfs_err (fs_info,"chunk start=%llu len=%llu flags=0x%llx doesn't match block group start=%llu len=%llu flags=0x%llx" , - em->start, em->len, - em->map_lookup->type &BTRFS_BLOCK_GROUP_TYPE_MASK , + map->start, map->chunk_len, + map->type &BTRFS_BLOCK_GROUP_TYPE_MASK , bg->start, bg->length, bg->flags &BTRFS_BLOCK_GROUP_TYPE_MASK ); ret = -EUCLEAN ; -free_extent_map (em); +btrfs_free_chunk_map (map);btrfs_put_block_group (bg);break ; } - start = em->start + em->len; -free_extent_map (em); + start = map->start + map->chunk_len; +btrfs_free_chunk_map (map);btrfs_put_block_group (bg); }return ret;
@@ -2370,28 +2349,25 @@ static int read_one_block_group(struct btrfs_fs_info *info,static int fill_dummy_bgs (struct btrfs_fs_info *fs_info) { -struct extent_map_tree *em_tree = &fs_info->mapping_tree;struct rb_node *node;int ret =0 ; -for (node =rb_first_cached (&em_tree->map); node; node =rb_next (node)) { -struct extent_map *em; -struct map_lookup *map; +for (node =rb_first_cached (&fs_info->mapping_tree); node; node =rb_next (node)) { +struct btrfs_chunk_map *map;struct btrfs_block_group *bg; - em = rb_entry(node, struct extent_map, rb_node); - map = em->map_lookup; - bg =btrfs_create_block_group_cache (fs_info, em->start); + map = rb_entry(node, struct btrfs_chunk_map, rb_node); + bg =btrfs_create_block_group_cache (fs_info, map->start);if (!bg) { ret = -ENOMEM ;break ; }/* Fill dummy cache as FULL */ - bg->length = em->len; + bg->length = map->chunk_len; bg->flags = map->type; bg->cached =BTRFS_CACHE_FINISHED ; - bg->used = em->len; + bg->used = map->chunk_len; bg->flags = map->type; ret =btrfs_add_block_group_cache (fs_info, bg);/*
@@ -2619,19 +2595,17 @@ static int insert_dev_extents(struct btrfs_trans_handle *trans, {struct btrfs_fs_info *fs_info = trans->fs_info;struct btrfs_device *device; -struct extent_map *em; -struct map_lookup *map; +struct btrfs_chunk_map *map;u64 dev_offset;u64 stripe_size;int i;int ret =0 ; - em =btrfs_get_chunk_map (fs_info, chunk_offset, chunk_size); -if (IS_ERR (em)) -return PTR_ERR (em); + map =btrfs_get_chunk_map (fs_info, chunk_offset, chunk_size); +if (IS_ERR (map)) +return PTR_ERR (map); - map = em->map_lookup; - stripe_size = em->orig_block_len; + stripe_size = map->stripe_size;/* * Take the device list mutex to prevent races with the final phase of
@@ -2654,7 +2628,7 @@ static int insert_dev_extents(struct btrfs_trans_handle *trans, }mutex_unlock (&fs_info->fs_devices->device_list_mutex); -free_extent_map (em); +btrfs_free_chunk_map (map);return ret; }
@@ -4407,8 +4381,6 @@ void btrfs_freeze_block_group(struct btrfs_block_group *cache)void btrfs_unfreeze_block_group (struct btrfs_block_group *block_group) {struct btrfs_fs_info *fs_info = block_group->fs_info; -struct extent_map_tree *em_tree; -struct extent_map *em;bool cleanup;spin_lock (&block_group->lock);
@@ -4417,17 +4389,16 @@ void btrfs_unfreeze_block_group(struct btrfs_block_group *block_group)spin_unlock (&block_group->lock);if (cleanup) { - em_tree = &fs_info->mapping_tree; -write_lock (&em_tree->lock); - em =lookup_extent_mapping (em_tree, block_group->start, -1 ); -BUG_ON (!em);/* logic error, can't happen */ -remove_extent_mapping (em_tree, em); -write_unlock (&em_tree->lock); - -/* once for us and once for the tree */ -free_extent_map (em); -free_extent_map (em); +struct btrfs_chunk_map *map; + + map =btrfs_find_chunk_map (fs_info, block_group->start,1 ); +/* Logic error, can't happen. */ +ASSERT (map); + +btrfs_remove_chunk_map (fs_info, map); + +/* Once for our lookup reference. */ +btrfs_free_chunk_map (map);/* * We may have left one free space entry and other possible
fs/btrfs/block-group.h +4 -2
@@ -5,6 +5,8 @@#include "free-space-cache.h" +struct btrfs_chunk_map ; +enum btrfs_disk_cache_state {BTRFS_DC_WRITTEN ,BTRFS_DC_ERROR ,
@@ -243,7 +245,7 @@ struct btrfs_block_group {u64 zone_unusable;u64 zone_capacity;u64 meta_write_pointer; -struct map_lookup *physical_map; +struct btrfs_chunk_map *physical_map;struct list_head active_bg_list;struct work_struct zone_finish_work;struct extent_buffer *last_eb;
@@ -297,7 +299,7 @@ struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(struct btrfs_fs_info *fs_info, const u64 chunk_offset);int btrfs_remove_block_group (struct btrfs_trans_handle *trans, -u64 group_start,struct extent_map *em); +struct btrfs_chunk_map *map);void btrfs_delete_unused_bgs (struct btrfs_fs_info *fs_info);void btrfs_mark_bg_unused (struct btrfs_block_group *bg);void btrfs_reclaim_bgs_work (struct work_struct *work);
fs/btrfs/dev-replace.c +12 -16
@@ -548,8 +548,7 @@ bool btrfs_finish_block_group_to_copy(struct btrfs_device *srcdev, u64 physical) {struct btrfs_fs_info *fs_info = cache->fs_info; -struct extent_map *em; -struct map_lookup *map; +struct btrfs_chunk_map *map;u64 chunk_offset = cache->start;int num_extents, cur_extent;int i;
@@ -565,9 +564,8 @@ bool btrfs_finish_block_group_to_copy(struct btrfs_device *srcdev, }spin_unlock (&cache->lock); - em =btrfs_get_chunk_map (fs_info, chunk_offset,1 ); -ASSERT (!IS_ERR (em)); - map = em->map_lookup; + map =btrfs_get_chunk_map (fs_info, chunk_offset,1 ); +ASSERT (!IS_ERR (map)); num_extents =0 ; cur_extent =0 ;
@@ -581,7 +579,7 @@ bool btrfs_finish_block_group_to_copy(struct btrfs_device *srcdev, cur_extent = i; } -free_extent_map (em); +btrfs_free_chunk_map (map);if (num_extents >1 && cur_extent < num_extents -1 ) {/*
@@ -810,25 +808,23 @@ static void btrfs_dev_replace_update_device_in_mapping_tree(struct btrfs_device *srcdev, struct btrfs_device *tgtdev) { - struct extent_map_tree *em_tree = &fs_info->mapping_tree; -struct extent_map *em; -struct map_lookup *map; u64 start = 0;int i; -write_lock (&em_tree->lock); +write_lock (&fs_info->mapping_tree_lock);do { - em =lookup_extent_mapping (em_tree, start, (u64)-1 ); -if (!em) +struct btrfs_chunk_map *map; + + map =btrfs_find_chunk_map_nolock (fs_info, start,U64_MAX ); +if (!map)break ; - map = em->map_lookup;for (i =0 ; i < map->num_stripes; i++)if (srcdev == map->stripes[i].dev) map->stripes[i].dev = tgtdev; - start = em->start + em->len; -free_extent_map (em); + start = map->start + map->chunk_len; +btrfs_free_chunk_map (map); }while (start); -write_unlock (&em_tree->lock); +write_unlock (&fs_info->mapping_tree_lock); } static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info,
fs/btrfs/disk-io.c +4 -3
@@ -2720,7 +2720,8 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)INIT_LIST_HEAD (&fs_info->allocated_ebs);spin_lock_init (&fs_info->eb_leak_lock);#endif -extent_map_tree_init (&fs_info->mapping_tree); + fs_info->mapping_tree =RB_ROOT_CACHED ; +rwlock_init (&fs_info->mapping_tree_lock);btrfs_init_block_rsv (&fs_info->global_block_rsv,BTRFS_BLOCK_RSV_GLOBAL );btrfs_init_block_rsv (&fs_info->trans_block_rsv,BTRFS_BLOCK_RSV_TRANS );
@@ -3603,7 +3604,7 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_devicebtrfs_stop_all_workers (fs_info);btrfs_free_block_groups (fs_info); fail_alloc: -btrfs_mapping_tree_free (&fs_info->mapping_tree); +btrfs_mapping_tree_free (fs_info);iput (fs_info->btree_inode); fail:
@@ -4386,7 +4387,7 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info)iput (fs_info->btree_inode); -btrfs_mapping_tree_free (&fs_info->mapping_tree); +btrfs_mapping_tree_free (fs_info);btrfs_close_devices (fs_info->fs_devices); }
fs/btrfs/extent_map.c +0 -46
78 lines not on this page · raw message
fs/btrfs/extent_map.h +0 -4
@@ -23,8 +23,6 @@ enum {EXTENT_FLAG_LOGGING ,/* Filling in a preallocated extent */ EXTENT_FLAG_FILLING , -/* filesystem extent mapping type */ -EXTENT_FLAG_FS_MAPPING ,/* This em is merged from two or more physically adjacent ems */ EXTENT_FLAG_MERGED, };
@@ -50,8 +48,6 @@ struct extent_map {*/ u64 generation;unsigned long flags; -/* Used for chunk mappings, flag EXTENT_FLAG_FS_MAPPING must be set */ -struct map_lookup *map_lookup;refcount_t refs;unsigned int compress_type;struct list_head list;
fs/btrfs/fs.h +2 -1
@@ -398,7 +398,8 @@ struct btrfs_fs_info {struct extent_io_tree excluded_extents;/* logical->physical extent mapping */ -struct extent_map_tree mapping_tree; +struct rb_root_cached mapping_tree; +rwlock_t mapping_tree_lock;/* * Block reservation for extent, checksum, root tree and delayed dir
fs/btrfs/inode.c +14 -11
@@ -10560,6 +10560,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,struct extent_io_tree *io_tree = &BTRFS_I (inode)->io_tree;struct extent_state *cached_state =NULL ;struct extent_map *em =NULL ; +struct btrfs_chunk_map *map =NULL ;struct btrfs_device *device =NULL ;struct btrfs_swap_info bsi = { .lowest_ppage = (sector_t)-1ULL ,
@@ -10699,13 +10700,13 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file, goto out; } - em =btrfs_get_chunk_map (fs_info, logical_block_start, len); -if (IS_ERR (em)) { - ret =PTR_ERR (em); + map =btrfs_get_chunk_map (fs_info, logical_block_start, len); +if (IS_ERR (map)) { + ret =PTR_ERR (map); goto out; } -if (em->map_lookup->type &BTRFS_BLOCK_GROUP_PROFILE_MASK ) { +if (map->type &BTRFS_BLOCK_GROUP_PROFILE_MASK ) {btrfs_warn (fs_info,"swapfile must have single data profile" ); ret = -EINVAL ;
@@ -10713,23 +10714,23 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file, }if (device ==NULL ) { - device = em->map_lookup->stripes[0 ].dev; + device = map->stripes[0 ].dev; ret =btrfs_add_swapfile_pin (inode, device, false);if (ret ==1 ) ret =0 ;else if (ret) goto out; - }else if (device != em->map_lookup->stripes[0 ].dev) { + }else if (device != map->stripes[0 ].dev) {btrfs_warn (fs_info,"swapfile must be on one device" ); ret = -EINVAL ; goto out; } - physical_block_start = (em->map_lookup->stripes[0 ].physical + - (logical_block_start - em->start)); - len =min (len, em->len - (logical_block_start - em->start)); -free_extent_map (em); - em =NULL ; + physical_block_start = (map->stripes[0 ].physical + + (logical_block_start - map->start)); + len =min (len, map->chunk_len - (logical_block_start - map->start)); +btrfs_free_chunk_map (map); + map =NULL ; bg =btrfs_lookup_block_group (fs_info, logical_block_start);if (!bg) {
@@ -10782,6 +10783,8 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file, out:if (!IS_ERR_OR_NULL (em))free_extent_map (em); +if (!IS_ERR_OR_NULL (map)) +btrfs_free_chunk_map (map);unlock_extent (io_tree,0 , isize -1 , &cached_state);
fs/btrfs/raid56.h +1 -1
@@ -164,7 +164,7 @@ struct raid56_bio_trace_info {u8 stripe_nr; }; -static inline int nr_data_stripes (const struct map_lookup *map) +static inline int nr_data_stripes (const struct btrfs_chunk_map *map) {return map->num_stripes -btrfs_nr_parity_stripes (map->type); }
fs/btrfs/scrub.c +16 -23
@@ -1280,7 +1280,7 @@ static void scrub_throttle_dev_io(struct scrub_ctx *sctx, struct btrfs_device *d* return 0 if it is a data stripe, 1 means parity stripe. */ static int get_raid56_logic_offset (u64 physical,int num, -struct map_lookup *map,u64 *offset, +struct btrfs_chunk_map *map,u64 *offset,u64 *stripe_start) {int i;
@@ -1893,7 +1893,7 @@ static int queue_scrub_stripe(struct scrub_ctx *sctx, struct btrfs_block_group *static int scrub_raid56_parity_stripe (struct scrub_ctx *sctx,struct btrfs_device *scrub_dev,struct btrfs_block_group *bg, -struct map_lookup *map, +struct btrfs_chunk_map *map,u64 full_stripe_start) {DECLARE_COMPLETION_ONSTACK (io_done);
@@ -2062,7 +2062,7 @@ static int scrub_raid56_parity_stripe(struct scrub_ctx *sctx,*/ static int scrub_simple_mirror (struct scrub_ctx *sctx,struct btrfs_block_group *bg, -struct map_lookup *map, +struct btrfs_chunk_map *map,u64 logical_start,u64 logical_length,struct btrfs_device *device,u64 physical,int mirror_num)
@@ -2121,7 +2121,7 @@ static int scrub_simple_mirror(struct scrub_ctx *sctx, }/* Calculate the full stripe length for simple stripe based profiles */ -static u64 simple_stripe_full_stripe_len (const struct map_lookup *map) +static u64 simple_stripe_full_stripe_len (const struct btrfs_chunk_map *map) {ASSERT (map->type & (BTRFS_BLOCK_GROUP_RAID0 |BTRFS_BLOCK_GROUP_RAID10 ));
@@ -2130,7 +2130,7 @@ static u64 simple_stripe_full_stripe_len(const struct map_lookup *map) }/* Get the logical bytenr for the stripe */ -static u64 simple_stripe_get_logical (struct map_lookup *map, +static u64 simple_stripe_get_logical (struct btrfs_chunk_map *map,struct btrfs_block_group *bg,int stripe_index) {
@@ -2147,7 +2147,7 @@ static u64 simple_stripe_get_logical(struct map_lookup *map, }/* Get the mirror number for the stripe */ -static int simple_stripe_mirror_num (struct map_lookup *map,int stripe_index) +static int simple_stripe_mirror_num (struct btrfs_chunk_map *map,int stripe_index) {ASSERT (map->type & (BTRFS_BLOCK_GROUP_RAID0 |BTRFS_BLOCK_GROUP_RAID10 ));
@@ -2159,7 +2159,7 @@ static int simple_stripe_mirror_num(struct map_lookup *map, int stripe_index)static int scrub_simple_stripe (struct scrub_ctx *sctx,struct btrfs_block_group *bg, -struct map_lookup *map, +struct btrfs_chunk_map *map,struct btrfs_device *device,int stripe_index) {
@@ -2192,18 +2192,17 @@ static int scrub_simple_stripe(struct scrub_ctx *sctx, static noinline_for_stack int scrub_stripe(struct scrub_ctx *sctx,struct btrfs_block_group *bg, -struct extent_map *em, +struct btrfs_chunk_map *map,struct btrfs_device *scrub_dev,int stripe_index) {struct btrfs_fs_info *fs_info = sctx->fs_info; -struct map_lookup *map = em->map_lookup;const u64 profile = map->type &BTRFS_BLOCK_GROUP_PROFILE_MASK ;const u64 chunk_logical = bg->start;int ret;int ret2;u64 physical = map->stripes[stripe_index].physical; -const u64 dev_stripe_len =btrfs_calc_stripe_length (em); +const u64 dev_stripe_len =btrfs_calc_stripe_length (map);const u64 physical_end = physical + dev_stripe_len;u64 logical;u64 logic_end;
@@ -2366,17 +2365,12 @@ static noinline_for_stack int scrub_chunk(struct scrub_ctx *sctx, u64 dev_extent_len) {struct btrfs_fs_info *fs_info = sctx->fs_info; -struct extent_map_tree *map_tree = &fs_info->mapping_tree; -struct map_lookup *map; -struct extent_map *em; +struct btrfs_chunk_map *map;int i;int ret =0 ; -read_lock (&map_tree->lock); - em =lookup_extent_mapping (map_tree, bg->start, bg->length); -read_unlock (&map_tree->lock); - -if (!em) { + map =btrfs_find_chunk_map (fs_info, bg->start, bg->length); +if (!map) {/* * Might have been an unused block group deleted by the cleaner * kthread or relocation.
@@ -2388,22 +2382,21 @@ static noinline_for_stack int scrub_chunk(struct scrub_ctx *sctx,return ret; } -if (em->start != bg->start) +if (map->start != bg->start) goto out; -if (em->len < dev_extent_len) +if (map->chunk_len < dev_extent_len) goto out; - map = em->map_lookup;for (i =0 ; i < map->num_stripes; ++i) {if (map->stripes[i].dev->bdev == scrub_dev->bdev && map->stripes[i].physical == dev_offset) { - ret =scrub_stripe (sctx, bg, em, scrub_dev, i); + ret =scrub_stripe (sctx, bg, map, scrub_dev, i);if (ret) goto out; } } out: -free_extent_map (em); +btrfs_free_chunk_map (map);return ret; }
fs/btrfs/tests/btrfs-tests.c +2 -1
@@ -28,6 +28,7 @@ const char *test_error[] = { [TEST_ALLOC_INODE] = "cannot allocate inode", [TEST_ALLOC_BLOCK_GROUP] = "cannot allocate block group", [TEST_ALLOC_EXTENT_MAP] = "cannot allocate extent map", + [TEST_ALLOC_CHUNK_MAP] = "cannot allocate chunk map", };static const struct super_operations btrfs_test_super_ops = {
@@ -185,7 +186,7 @@ void btrfs_free_dummy_fs_info(struct btrfs_fs_info *fs_info) }spin_unlock (&fs_info->buffer_lock); -btrfs_mapping_tree_free (&fs_info->mapping_tree); +btrfs_mapping_tree_free (fs_info);list_for_each_entry_safe (dev, tmp, &fs_info->fs_devices->devices, dev_list) {btrfs_free_dummy_device (dev);
fs/btrfs/tests/btrfs-tests.h +1 -0
@@ -23,6 +23,7 @@ enum {TEST_ALLOC_INODE ,TEST_ALLOC_BLOCK_GROUP ,TEST_ALLOC_EXTENT_MAP , + TEST_ALLOC_CHUNK_MAP, };extern const char *test_error[];
fs/btrfs/tests/extent-map-tests.c +10 -30
@@ -859,33 +859,21 @@ struct rmap_test_vector {static int test_rmap_block (struct btrfs_fs_info *fs_info,struct rmap_test_vector *test) { -struct extent_map *em; -struct map_lookup *map =NULL ; +struct btrfs_chunk_map *map;u64 *logical =NULL ;int i, out_ndaddrs, out_stripe_len;int ret; - em =alloc_extent_map (); -if (!em) { -test_std_err (TEST_ALLOC_EXTENT_MAP ); -return -ENOMEM ; - } - - map =kmalloc (map_lookup_size (test->num_stripes),GFP_KERNEL ); + map =btrfs_alloc_chunk_map (test->num_stripes,GFP_KERNEL );if (!map) { -kfree (em); -test_std_err (TEST_ALLOC_EXTENT_MAP ); +test_std_err (TEST_ALLOC_CHUNK_MAP );return -ENOMEM ; } -set_bit (EXTENT_FLAG_FS_MAPPING , &em->flags);/* Start at 4GiB logical address */ - em->start =SZ_4G ; - em->len = test->data_stripe_size * test->num_data_stripes; - em->block_len = em->len; - em->orig_block_len = test->data_stripe_size; - em->map_lookup = map; - + map->start =SZ_4G ; + map->chunk_len = test->data_stripe_size * test->num_data_stripes; + map->stripe_size = test->data_stripe_size; map->num_stripes = test->num_stripes; map->type = test->raid_type;
@@ -901,15 +889,13 @@ static int test_rmap_block(struct btrfs_fs_info *fs_info, map->stripes[i].physical = test->data_stripe_phys_start[i]; } -write_lock (&fs_info->mapping_tree.lock); - ret =add_extent_mapping (&fs_info->mapping_tree, em,0 ); -write_unlock (&fs_info->mapping_tree.lock); + ret =btrfs_add_chunk_map (fs_info, map);if (ret) { -test_err ("error adding block group mapping to mapping tree" ); +test_err ("error adding chunk map to mapping tree" ); goto out_free; } - ret =btrfs_rmap_block (fs_info, em->start,btrfs_sb_offset (1 ), + ret =btrfs_rmap_block (fs_info, map->start,btrfs_sb_offset (1 ), &logical, &out_ndaddrs, &out_stripe_len);if (ret || (out_ndaddrs ==0 && test->expected_mapped_addr)) { test_err("didn't rmap anything but expected %d",
@@ -938,14 +924,8 @@ static int test_rmap_block(struct btrfs_fs_info *fs_info, ret =0 ; out: -write_lock (&fs_info->mapping_tree.lock); -remove_extent_mapping (&fs_info->mapping_tree, em); -write_unlock (&fs_info->mapping_tree.lock); -/* For us */ -free_extent_map (em); +btrfs_remove_chunk_map (fs_info, map); out_free: -/* For the tree */ -free_extent_map (em);kfree (logical);return ret; }
fs/btrfs/volumes.c +318 -222
960 lines not on this page · raw message
fs/btrfs/volumes.h +36 -9
81 lines not on this page · raw message
fs/btrfs/zoned.c +8 -16
@@ -1553,8 +1553,7 @@ int btrfs_load_block_group_zone_info(struct btrfs_block_group *cache, bool new) {struct btrfs_fs_info *fs_info = cache->fs_info;struct extent_map_tree *em_tree = &fs_info->mapping_tree; -struct extent_map *em; -struct map_lookup *map; +struct btrfs_chunk_map *map;u64 logical = cache->start;u64 length = cache->length;struct zone_info *zone_info =NULL ;
@@ -1575,17 +1574,11 @@ int btrfs_load_block_group_zone_info(struct btrfs_block_group *cache, bool new)return -EIO ; } -/* Get the chunk mapping */ -read_lock (&em_tree->lock); - em =lookup_extent_mapping (em_tree, logical, length); -read_unlock (&em_tree->lock); - -if (!em) + map =btrfs_find_chunk_map (fs_info, logical, length); +if (!map)return -EINVAL ; - map = em->map_lookup; - - cache->physical_map =kmemdup (map,map_lookup_size (map->num_stripes),GFP_NOFS ); + cache->physical_map =btrfs_clone_chunk_map (map,GFP_NOFS );if (!cache->physical_map) { ret = -ENOMEM ; goto out;
@@ -1687,12 +1680,11 @@ int btrfs_load_block_group_zone_info(struct btrfs_block_group *cache, bool new)spin_unlock (&fs_info->zone_active_bgs_lock); } }else { -kfree (cache->physical_map); +btrfs_free_chunk_map (cache->physical_map); cache->physical_map =NULL ; }bitmap_free (active);kfree (zone_info); -free_extent_map (em);return ret; }
@@ -2082,7 +2074,7 @@ int btrfs_sync_zone_write_pointer(struct btrfs_device *tgt_dev, u64 logical,bool btrfs_zone_activate (struct btrfs_block_group *block_group) {struct btrfs_fs_info *fs_info = block_group->fs_info; -struct map_lookup *map; +struct btrfs_chunk_map *map;struct btrfs_device *device;u64 physical;const bool is_data = (block_group->flags &BTRFS_BLOCK_GROUP_DATA );
@@ -2194,7 +2186,7 @@ static void wait_eb_writebacks(struct btrfs_block_group *block_group)static int do_zone_finish (struct btrfs_block_group *block_group,bool fully_written) {struct btrfs_fs_info *fs_info = block_group->fs_info; -struct map_lookup *map; +struct btrfs_chunk_map *map;const bool is_metadata = (block_group->flags & (BTRFS_BLOCK_GROUP_METADATA |BTRFS_BLOCK_GROUP_SYSTEM ));int ret =0 ;
@@ -2643,7 +2635,7 @@ void btrfs_check_active_zone_reservation(struct btrfs_fs_info *fs_info)/* Release reservation for currently active block groups. */ spin_lock (&fs_info->zone_active_bgs_lock);list_for_each_entry (block_group, &fs_info->zone_active_bgs, active_bg_list) { -struct map_lookup *map = block_group->physical_map; +struct btrfs_chunk_map *map = block_group->physical_map;if (!(block_group->flags & (BTRFS_BLOCK_GROUP_METADATA |BTRFS_BLOCK_GROUP_SYSTEM )))
include/trace/events/btrfs.h +5 -6
@@ -21,7 +21,7 @@ struct btrfs_delayed_data_ref;struct btrfs_delayed_ref_head ;struct btrfs_block_group ;struct btrfs_free_cluster ; -struct map_lookup ; +struct btrfs_chunk_map ;struct extent_buffer ;struct btrfs_work ;struct btrfs_workqueue ;
@@ -277,8 +277,7 @@ DEFINE_EVENT(btrfs__inode, btrfs_inode_evict, { (1 << EXTENT_FLAG_COMPRESSED), "COMPRESSED" },\ { (1 << EXTENT_FLAG_PREALLOC), "PREALLOC" },\ { (1 << EXTENT_FLAG_LOGGING), "LOGGING" },\ - { (1 << EXTENT_FLAG_FILLING), "FILLING" },\ - { (1 << EXTENT_FLAG_FS_MAPPING), "FS_MAPPING" }) + { (1 << EXTENT_FLAG_FILLING), "FILLING" }) TRACE_EVENT_CONDITION(btrfs_get_extent,
@@ -1061,7 +1060,7 @@ DEFINE_EVENT(btrfs_delayed_ref_head, run_delayed_ref_head, DECLARE_EVENT_CLASS(btrfs__chunk, TP_PROTO(const struct btrfs_fs_info *fs_info, - const struct map_lookup *map, u64 offset, u64 size), + const struct btrfs_chunk_map *map, u64 offset, u64 size), TP_ARGS(fs_info, map, offset, size),
@@ -1095,7 +1094,7 @@ DECLARE_EVENT_CLASS(btrfs__chunk,DEFINE_EVENT (btrfs__chunk, btrfs_chunk_alloc, TP_PROTO(const struct btrfs_fs_info *fs_info, - const struct map_lookup *map, u64 offset, u64 size), + const struct btrfs_chunk_map *map, u64 offset, u64 size),TP_ARGS (fs_info, map, offset, size) );
@@ -1103,7 +1102,7 @@ DEFINE_EVENT(btrfs__chunk, btrfs_chunk_alloc,DEFINE_EVENT (btrfs__chunk, btrfs_chunk_free, TP_PROTO(const struct btrfs_fs_info *fs_info, - const struct map_lookup *map, u64 offset, u64 size), + const struct btrfs_chunk_map *map, u64 offset, u64 size),TP_ARGS (fs_info, map, offset, size) );
Signature · 1 line
2.40.1
Re: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
Quoted 63 lines · On Tue, Nov 21, 2023 at 01:38:38PM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
Currently we abuse the
extent_mapstructure for two purposes:1) To actually represent extents for inodes;
2) To represent chunk mappings.This is odd and has several disadvantages:
1) To create a chunk map, we need to do two memory allocations: one for an
extent_mapstructure and another one for amap_lookupstructure, so more potential for an allocation failure and more complicated code to manage and link two structures;2) For a chunk map we actually only use 3 fields (24 bytes) of the respective extent map structure: the 'start' field to have the logical start address of the chunk, the 'len' field to have the chunk's size, and the '
orig_block_len' field to contain the chunk's stripe size.Besides wasting a memory, it's also odd and not intuitive at all to have the stripe size in a field named '
orig_block_len'.We are also using '
block_len' of theextent_mapstructure to contain the chunk size, so we have 2 fields for the same value, 'len' and 'block_len', which is pointless;3) When an extent map is associated to a chunk mapping, we set the bit
EXTENT_FLAG_FS_MAPPINGon its flags and then make its member named 'map_lookup' point to the associatedmap_lookupstructure. This means that for an extent map associated to an inode extent, we are not using this 'map_lookup' pointer, so wasting 8 bytes (on a 64 bits platform);4) Extent maps associated to a chunk mapping are never merged or split so it's pointless to use the existing extent map infrastructure.
So add a dedicated data structure named '
btrfs_chunk_map' to represent chunk mappings, this is basically the existingmap_lookupstructure with some extra fields:1) 'start' to contain the chunk logical address;
2) 'chunk_len' to contain the chunk's length;
3) 'stripe_size' for the stripe size;
4) 'rb_node' for insertion into a rb tree;
5) 'refs' for reference counting.This way we do a single memory allocation for chunk mappings and we don't waste memory for them with unused/unnecessary fields from an
extent_map.We also save 8 bytes from the
extent_mapstructure by removing the 'map_lookup' pointer, so the size of structextent_mapis reduced from 144 bytes down to 136 bytes, and we can now have 30 extents map per 4K page instead of 28.Signed-off-by Filipe Manana <[email protected]>--- fs/btrfs/block-group.c | 165 ++++----- fs/btrfs/block-group.h | 6 +- fs/btrfs/dev-replace.c | 28 +- fs/btrfs/disk-io.c | 7 +- fs/btrfs/extent_map.c | 46 --- fs/btrfs/extent_map.h | 4 - fs/btrfs/fs.h | 3 +- fs/btrfs/inode.c | 25 +-
Quoted 8 lines
fs/btrfs/raid56.h | 2 +- fs/btrfs/scrub.c | 39 +-- fs/btrfs/tests/btrfs-tests.c | 3 +- fs/btrfs/tests/btrfs-tests.h | 1 + fs/btrfs/tests/extent-map-tests.c | 40 +-- fs/btrfs/volumes.c | 540 ++++++++++++++++++------------ fs/btrfs/volumes.h | 45 ++- fs/btrfs/zoned.c | 24 +-
I see a lot of errors when compiling zoned.c, there are still map_lookup structures. Do you have the zoned mode config option enabled?
CC [M] fs/btrfs/zoned.ofs/btrfs/zoned.c:1293:40: warning: ‘struct map_lookup’ declared inside parameter list will not be visible outside of this definition or declaration
1293 | struct map_lookup *map)
| ^~~~~~~~~~fs/btrfs/zoned.c: In function ‘btrfs_load_zone_info’:fs/btrfs/zoned.c:1296:42: error: invalid use of undefined type ‘struct map_lookup’
1296 | struct btrfs_device *device = map->stripes[zone_idx].dev;
| ^~fs/btrfs/zoned.c:1302:29: error: invalid use of undefined type ‘struct map_lookup’
1302 | info->physical = map->stripes[zone_idx].physical;
| ^~fs/btrfs/zoned.c: At top level:fs/btrfs/zoned.c:1396:46: warning: ‘struct map_lookup’ declared inside parameter list will not be visible outside of this definition or declaration
1396 | struct map_lookup *map,
| ^~~~~~~~~~fs/btrfs/zoned.c: In function ‘btrfs_load_block_group_dup’:fs/btrfs/zoned.c:1402:17: error: invalid use of undefined type ‘struct map_lookup’
1402 | if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
| ^~
...
Re: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
Quoted 74 lines · David Sterba
Quoted 63 lines · On Tue, Nov 21, 2023 at 01:38:38PM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
Currently we abuse the
extent_mapstructure for two purposes:1) To actually represent extents for inodes;
2) To represent chunk mappings.This is odd and has several disadvantages:
1) To create a chunk map, we need to do two memory allocations: one for an
extent_mapstructure and another one for amap_lookupstructure, so more potential for an allocation failure and more complicated code to manage and link two structures;2) For a chunk map we actually only use 3 fields (24 bytes) of the respective extent map structure: the 'start' field to have the logical start address of the chunk, the 'len' field to have the chunk's size, and the '
orig_block_len' field to contain the chunk's stripe size.Besides wasting a memory, it's also odd and not intuitive at all to have the stripe size in a field named '
orig_block_len'.We are also using '
block_len' of theextent_mapstructure to contain the chunk size, so we have 2 fields for the same value, 'len' and 'block_len', which is pointless;3) When an extent map is associated to a chunk mapping, we set the bit
EXTENT_FLAG_FS_MAPPINGon its flags and then make its member named 'map_lookup' point to the associatedmap_lookupstructure. This means that for an extent map associated to an inode extent, we are not using this 'map_lookup' pointer, so wasting 8 bytes (on a 64 bits platform);4) Extent maps associated to a chunk mapping are never merged or split so it's pointless to use the existing extent map infrastructure.
So add a dedicated data structure named '
btrfs_chunk_map' to represent chunk mappings, this is basically the existingmap_lookupstructure with some extra fields:1) 'start' to contain the chunk logical address;
2) 'chunk_len' to contain the chunk's length;
3) 'stripe_size' for the stripe size;
4) 'rb_node' for insertion into a rb tree;
5) 'refs' for reference counting.This way we do a single memory allocation for chunk mappings and we don't waste memory for them with unused/unnecessary fields from an
extent_map.We also save 8 bytes from the
extent_mapstructure by removing the 'map_lookup' pointer, so the size of structextent_mapis reduced from 144 bytes down to 136 bytes, and we can now have 30 extents map per 4K page instead of 28.Signed-off-by Filipe Manana <[email protected]>--- fs/btrfs/block-group.c | 165 ++++----- fs/btrfs/block-group.h | 6 +- fs/btrfs/dev-replace.c | 28 +- fs/btrfs/disk-io.c | 7 +- fs/btrfs/extent_map.c | 46 --- fs/btrfs/extent_map.h | 4 - fs/btrfs/fs.h | 3 +- fs/btrfs/inode.c | 25 +-Quoted 8 lines
fs/btrfs/raid56.h | 2 +- fs/btrfs/scrub.c | 39 +-- fs/btrfs/tests/btrfs-tests.c | 3 +- fs/btrfs/tests/btrfs-tests.h | 1 + fs/btrfs/tests/extent-map-tests.c | 40 +-- fs/btrfs/volumes.c | 540 ++++++++++++++++++------------ fs/btrfs/volumes.h | 45 ++- fs/btrfs/zoned.c | 24 +-
Quoted 2 lines
I see a lot of errors when compiling
zoned.c, there are stillmap_lookupstructures. Do you have the zoned mode config option enabled?
It misses this hunk: https://pastebin.com/raw/mr1KMZ8N
Do you want me to send a v2, or do you prefer to fix it up?
Thanks.
Quoted 21 lines
CC [M]
fs/btrfs/zoned.ofs/btrfs/zoned.c:1293:40: warning: ‘struct map_lookup’ declared inside parameter list will not be visible outside of this definition or declaration
1293 | structmap_lookup*map)
| ^~~~~~~~~~fs/btrfs/zoned.c: In function ‘btrfs_load_zone_info’:fs/btrfs/zoned.c:1296:42: error: invalid use of undefined type ‘struct map_lookup’
1296 | structbtrfs_device*device = map->stripes[zone_idx].dev;
| ^~fs/btrfs/zoned.c:1302:29: error: invalid use of undefined type ‘struct map_lookup’
1302 |info->physical= map->stripes[zone_idx].physical;
| ^~fs/btrfs/zoned.c: At top level:fs/btrfs/zoned.c:1396:46: warning: ‘struct map_lookup’ declared inside parameter list will not be visible outside of this definition or declaration
1396 | structmap_lookup*map,
| ^~~~~~~~~~fs/btrfs/zoned.c: In function ‘btrfs_load_block_group_dup’:fs/btrfs/zoned.c:1402:17: error: invalid use of undefined type ‘struct map_lookup’
1402 | if ((map->type&BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
| ^~
...
Re: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
Quoted 17 lines · Filipe Manana
Quoted 16 lines · David Sterba
Quoted 14 lines · On Tue, Nov 21, 2023 at 01:38:38PM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
fs/btrfs/disk-io.c | 7 +- fs/btrfs/extent_map.c | 46 --- fs/btrfs/extent_map.h | 4 - fs/btrfs/fs.h | 3 +- fs/btrfs/inode.c | 25 +- fs/btrfs/raid56.h | 2 +- fs/btrfs/scrub.c | 39 +-- fs/btrfs/tests/btrfs-tests.c | 3 +- fs/btrfs/tests/btrfs-tests.h | 1 + fs/btrfs/tests/extent-map-tests.c | 40 +-- fs/btrfs/volumes.c | 540 ++++++++++++++++++------------ fs/btrfs/volumes.h | 45 ++- fs/btrfs/zoned.c | 24 +-
Quoted 6 lines
Quoted 2 lines
I see a lot of errors when compiling
zoned.c, there are stillmap_lookupstructures. Do you have the zoned mode config option enabled?It misses this hunk: https://pastebin.com/raw/mr1KMZ8N
Do you want me to send a v2, or do you prefer to fix it up?
Simple enough to apply, no need to resend, thanks.
Re: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
In reply to [email protected]Quoted 9 lines · On Tue, Nov 21, 2023 at 01:38:38PM +0000, [email protected] wrote:
extern const struct
btrfs_raid_attrbtrfs_raid_array[BTRFS_NR_RAID_TYPES];fs/btrfs/volumes.h +6
-1-struct map_lookup { +struct btrfs_chunk_map { +struct rb_node rb_node; +refcount_t refs; +u64 start; +u64 chunk_len; +u64 stripe_size;
fs/btrfs/volumes.h struct btrfs_raid_attr { · line 549
Quoted 10 lines
fs/btrfs/volumes.h +2
-1u64 type;int io_align;int io_width;int num_stripes;int sub_stripes; -int verified_stripes;/* For mount time dev extent verification */ +/* For mount time dev extent verification. */ +int verified_stripes;struct btrfs_io_stripe stripes[]; };
This results in two 4 byte holes:
struct btrfs_chunk_map {
struct rb_node rb_node __attribute__((__aligned__(8))); /* 0 24 */
refcount_t refs; /* 24 4 */
/* XXX 4 bytes hole, try to pack */
u64 start; /* 32 8 */
u64 chunk_len; /* 40 8 */
u64 stripe_size; /* 48 8 */
u64 type; /* 56 8 */
/* --- cacheline 1 boundary (64 bytes) --- */
int io_align; /* 64 4 */
int io_width; /* 68 4 */
int num_stripes; /* 72 4 */
int sub_stripes; /* 76 4 */
int verified_stripes; /* 80 4 */
/* XXX 4 bytes hole, try to pack */
struct btrfs_io_stripe stripes[]; /* 88 0 */
/* size: 88, cachelines: 2, members: 12 */ /* sum members: 80, holes: 2, sum holes: 8 */ /* forced alignments: 1 */
/* last cacheline: 24 bytes */
} __attribute__((__aligned__(8)));
I could move verify_stripes after refs or move refs to start of the second cacheline between type and io_align. I suspect some cache bouncing could happen with refcount updates and tree traversal but it's a speculation and I don't think the effects would be measurable.
Re: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
Quoted 51 lines · David Sterba
Quoted 9 lines · On Tue, Nov 21, 2023 at 01:38:38PM +0000, [email protected] wrote:
extern const struct
btrfs_raid_attrbtrfs_raid_array[BTRFS_NR_RAID_TYPES];fs/btrfs/volumes.h +6
-1-struct map_lookup { +struct btrfs_chunk_map { +struct rb_node rb_node; +refcount_t refs; +u64 start; +u64 chunk_len; +u64 stripe_size;fs/btrfs/volumes.h struct btrfs_raid_attr { · line 549
Quoted 10 lines
fs/btrfs/volumes.h +2
-1u64 type;int io_align;int io_width;int num_stripes;int sub_stripes; -int verified_stripes;/* For mount time dev extent verification */ +/* For mount time dev extent verification. */ +int verified_stripes;struct btrfs_io_stripe stripes[]; };This results in two 4 byte holes:
struct
btrfs_chunk_map{
structrb_noderb_node__attribute__((__aligned__(8))); /* 0 24 */
refcount_trefs; /* 24 4 *//* XXX 4 bytes hole, try to pack */
u64 start; /* 32 8 */
u64chunk_len; /* 40 8 */
u64stripe_size; /* 48 8 */
u64 type; /* 56 8 */
/* --- cacheline 1 boundary (64 bytes) --- */
intio_align; /* 64 4 */
intio_width; /* 68 4 */
intnum_stripes; /* 72 4 */
intsub_stripes; /* 76 4 */
intverified_stripes; /* 80 4 *//* XXX 4 bytes hole, try to pack */
struct
btrfs_io_stripestripes[]; /* 88 0 *//* size: 88, cachelines: 2, members: 12 */ /* sum members: 80, holes: 2, sum holes: 8 */ /* forced alignments: 1 */
/* last cacheline: 24 bytes */
} __attribute__((__aligned__(8)));
Quoted 4 lines
I could move
verify_stripesafter refs or move refs to start of the second cacheline between type andio_align. I suspect some cache bouncing could happen with refcount updates and tree traversal but it's a speculation and I don't think the effects would be measurable.
I would prefer to have verified_stripes in the first cache line, right below refs for example, so that
everything needed for map lookups and insertions is in the same cache line.
Do you want to squash such change in this patch? Or should I send it separately?
Thanks.
Re: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
Quoted 30 lines · Filipe Manana
Quoted 29 lines · David Sterba
On Tue, Nov 21, 2023 at 01:38:38PM +0000, [email protected] wrote:
struct
btrfs_chunk_map{
structrb_noderb_node__attribute__((__aligned__(8))); /* 0 24 */
refcount_trefs; /* 24 4 *//* XXX 4 bytes hole, try to pack */
u64 start; /* 32 8 */
u64chunk_len; /* 40 8 */
u64stripe_size; /* 48 8 */
u64 type; /* 56 8 */
/* --- cacheline 1 boundary (64 bytes) --- */
intio_align; /* 64 4 */
intio_width; /* 68 4 */
intnum_stripes; /* 72 4 */
intsub_stripes; /* 76 4 */
intverified_stripes; /* 80 4 *//* XXX 4 bytes hole, try to pack */
struct
btrfs_io_stripestripes[]; /* 88 0 *//* size: 88, cachelines: 2, members: 12 */ /* sum members: 80, holes: 2, sum holes: 8 */ /* forced alignments: 1 */
/* last cacheline: 24 bytes */
} __attribute__((__aligned__(8)));
Quoted 8 lines
Quoted 4 lines
I could move
verify_stripesafter refs or move refs to start of the second cacheline between type andio_align. I suspect some cache bouncing could happen with refcount updates and tree traversal but it's a speculation and I don't think the effects would be measurable.I would prefer to have
verified_stripesin the first cache line, right below refs for example, so that
everything needed for map lookups and insertions is in the same cache line.
Right, that makes sense.
Quoted 1 line
Do you want to squash such change in this patch? Or should I send it separately?
I'll do the change, thanks.
kernel test robot
build botRe: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
In reply to [email protected]kernel test robot noticed the following build warnings:
Hi,
kernel test robot noticed the following build warnings:
[auto build test WARNING on kdave/for-next]
[also build test WARNING on linus/master v6.7-rc2 next-20231121] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/fdmanana-kernel-org/btrfs-fix-off-by-one-when-checking-chunk-map-includes-logical-address/20231121-214139 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next patch link: https://lore.kernel.org/r/777320fd09dfc68a89180723bf5d7368dab06299.1700573314.git.fdmanana%40suse.com patch subject: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps config: i386-randconfig-061-20231122 (https://download.01.org/0day-ci/archive/20231122/[email protected]/config) compiler: clang version 16.0.4 (https://github.com/llvm/llvm-project.git ae42196bc493ffe877a7e3dff8be32035dea4d07) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231122/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> fs/btrfs/zoned.c:1293:12: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map)
^
fs/btrfs/zoned.c:1296:35: error: incomplete definition of type 'struct map_lookup'
struct btrfs_device *device = map->stripes[zone_idx].dev;
~~~^
fs/btrfs/zoned.c:1293:12: note: forward declaration of 'struct map_lookup'
struct map_lookup *map)
^
fs/btrfs/zoned.c:1302:22: error: incomplete definition of type 'struct map_lookup'
info->physical = map->stripes[zone_idx].physical;
~~~^
fs/btrfs/zoned.c:1293:12: note: forward declaration of 'struct map_lookup'
struct map_lookup *map)
^
fs/btrfs/zoned.c:1396:18: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1402:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1396:18: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1438:13: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1445:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1447:36: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1451:21: error: incomplete definition of type 'struct map_lookup'
for (i = 0; i < map->num_stripes; i++) {
~~~^
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1460:37: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1486:13: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1492:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1486:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1494:36: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1486:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1498:25: error: incomplete definition of type 'struct map_lookup'
for (int i = 0; i < map->num_stripes; i++) {
~~~^
fs/btrfs/zoned.c:1486:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1518:14: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1524:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1518:14: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1526:36: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));vim +1293 fs/btrfs/zoned.c
15c12fcc50a1b12 Christoph Hellwig 2023-06-05 1290
09a46725cc84165 Christoph Hellwig 2023-06-05 1291 static int btrfs_load_zone_info(struct btrfs_fs_info *fs_info, int zone_idx,
09a46725cc84165 Christoph Hellwig 2023-06-05 1292 struct zone_info *info, unsigned long *active,
09a46725cc84165 Christoph Hellwig 2023-06-05 @1293 struct map_lookup *map)
09a46725cc84165 Christoph Hellwig 2023-06-05 1294 {
09a46725cc84165 Christoph Hellwig 2023-06-05 1295 struct btrfs_dev_replace *dev_replace = &fs_info->dev_replace;
09a46725cc84165 Christoph Hellwig 2023-06-05 1296 struct btrfs_device *device = map->stripes[zone_idx].dev;
09a46725cc84165 Christoph Hellwig 2023-06-05 1297 int dev_replace_is_ongoing = 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1298 unsigned int nofs_flag;
09a46725cc84165 Christoph Hellwig 2023-06-05 1299 struct blk_zone zone;
09a46725cc84165 Christoph Hellwig 2023-06-05 1300 int ret;
09a46725cc84165 Christoph Hellwig 2023-06-05 1301
09a46725cc84165 Christoph Hellwig 2023-06-05 1302 info->physical = map->stripes[zone_idx].physical;
09a46725cc84165 Christoph Hellwig 2023-06-05 1303
09a46725cc84165 Christoph Hellwig 2023-06-05 1304 if (!device->bdev) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1305 info->alloc_offset = WP_MISSING_DEV;
09a46725cc84165 Christoph Hellwig 2023-06-05 1306 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1307 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1308
09a46725cc84165 Christoph Hellwig 2023-06-05 1309 /* Consider a zone as active if we can allow any number of active zones. */
09a46725cc84165 Christoph Hellwig 2023-06-05 1310 if (!device->zone_info->max_active_zones)
09a46725cc84165 Christoph Hellwig 2023-06-05 1311 __set_bit(zone_idx, active);
09a46725cc84165 Christoph Hellwig 2023-06-05 1312
09a46725cc84165 Christoph Hellwig 2023-06-05 1313 if (!btrfs_dev_is_sequential(device, info->physical)) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1314 info->alloc_offset = WP_CONVENTIONAL;
09a46725cc84165 Christoph Hellwig 2023-06-05 1315 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1316 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1317
09a46725cc84165 Christoph Hellwig 2023-06-05 1318 /* This zone will be used for allocation, so mark this zone non-empty. */
09a46725cc84165 Christoph Hellwig 2023-06-05 1319 btrfs_dev_clear_zone_empty(device, info->physical);
09a46725cc84165 Christoph Hellwig 2023-06-05 1320
09a46725cc84165 Christoph Hellwig 2023-06-05 1321 down_read(&dev_replace->rwsem);
09a46725cc84165 Christoph Hellwig 2023-06-05 1322 dev_replace_is_ongoing = btrfs_dev_replace_is_ongoing(dev_replace);
09a46725cc84165 Christoph Hellwig 2023-06-05 1323 if (dev_replace_is_ongoing && dev_replace->tgtdev != NULL)
09a46725cc84165 Christoph Hellwig 2023-06-05 1324 btrfs_dev_clear_zone_empty(dev_replace->tgtdev, info->physical);
09a46725cc84165 Christoph Hellwig 2023-06-05 1325 up_read(&dev_replace->rwsem);
09a46725cc84165 Christoph Hellwig 2023-06-05 1326
09a46725cc84165 Christoph Hellwig 2023-06-05 1327 /*
09a46725cc84165 Christoph Hellwig 2023-06-05 1328 * The group is mapped to a sequential zone. Get the zone write pointer
09a46725cc84165 Christoph Hellwig 2023-06-05 1329 * to determine the allocation offset within the zone.
09a46725cc84165 Christoph Hellwig 2023-06-05 1330 */
09a46725cc84165 Christoph Hellwig 2023-06-05 1331 WARN_ON(!IS_ALIGNED(info->physical, fs_info->zone_size));
09a46725cc84165 Christoph Hellwig 2023-06-05 1332 nofs_flag = memalloc_nofs_save();
09a46725cc84165 Christoph Hellwig 2023-06-05 1333 ret = btrfs_get_dev_zone(device, info->physical, &zone);
09a46725cc84165 Christoph Hellwig 2023-06-05 1334 memalloc_nofs_restore(nofs_flag);
09a46725cc84165 Christoph Hellwig 2023-06-05 1335 if (ret) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1336 if (ret != -EIO && ret != -EOPNOTSUPP)
09a46725cc84165 Christoph Hellwig 2023-06-05 1337 return ret;
09a46725cc84165 Christoph Hellwig 2023-06-05 1338 info->alloc_offset = WP_MISSING_DEV;
09a46725cc84165 Christoph Hellwig 2023-06-05 1339 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1340 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1341
09a46725cc84165 Christoph Hellwig 2023-06-05 1342 if (zone.type == BLK_ZONE_TYPE_CONVENTIONAL) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1343 btrfs_err_in_rcu(fs_info,
09a46725cc84165 Christoph Hellwig 2023-06-05 1344 "zoned: unexpected conventional zone %llu on device %s (devid %llu)",
09a46725cc84165 Christoph Hellwig 2023-06-05 1345 zone.start << SECTOR_SHIFT, rcu_str_deref(device->name),
09a46725cc84165 Christoph Hellwig 2023-06-05 1346 device->devid);
09a46725cc84165 Christoph Hellwig 2023-06-05 1347 return -EIO;
09a46725cc84165 Christoph Hellwig 2023-06-05 1348 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1349
09a46725cc84165 Christoph Hellwig 2023-06-05 1350 info->capacity = (zone.capacity << SECTOR_SHIFT);
09a46725cc84165 Christoph Hellwig 2023-06-05 1351
09a46725cc84165 Christoph Hellwig 2023-06-05 1352 switch (zone.cond) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1353 case BLK_ZONE_COND_OFFLINE:
09a46725cc84165 Christoph Hellwig 2023-06-05 1354 case BLK_ZONE_COND_READONLY:
09a46725cc84165 Christoph Hellwig 2023-06-05 1355 btrfs_err(fs_info,
09a46725cc84165 Christoph Hellwig 2023-06-05 1356 "zoned: offline/readonly zone %llu on device %s (devid %llu)",
09a46725cc84165 Christoph Hellwig 2023-06-05 1357 (info->physical >> device->zone_info->zone_size_shift),
09a46725cc84165 Christoph Hellwig 2023-06-05 1358 rcu_str_deref(device->name), device->devid);
09a46725cc84165 Christoph Hellwig 2023-06-05 1359 info->alloc_offset = WP_MISSING_DEV;
09a46725cc84165 Christoph Hellwig 2023-06-05 1360 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1361 case BLK_ZONE_COND_EMPTY:
09a46725cc84165 Christoph Hellwig 2023-06-05 1362 info->alloc_offset = 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1363 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1364 case BLK_ZONE_COND_FULL:
09a46725cc84165 Christoph Hellwig 2023-06-05 1365 info->alloc_offset = info->capacity;
09a46725cc84165 Christoph Hellwig 2023-06-05 1366 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1367 default:
09a46725cc84165 Christoph Hellwig 2023-06-05 1368 /* Partially used zone. */
09a46725cc84165 Christoph Hellwig 2023-06-05 1369 info->alloc_offset = ((zone.wp - zone.start) << SECTOR_SHIFT);
09a46725cc84165 Christoph Hellwig 2023-06-05 1370 __set_bit(zone_idx, active);
09a46725cc84165 Christoph Hellwig 2023-06-05 1371 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1372 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1373
09a46725cc84165 Christoph Hellwig 2023-06-05 1374 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1375 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1376 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
In reply to [email protected]kernel test robot noticed the following build errors:
Hi,
kernel test robot noticed the following build errors:
[auto build test ERROR on kdave/for-next]
[also build test ERROR on linus/master v6.7-rc2 next-20231121] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/fdmanana-kernel-org/btrfs-fix-off-by-one-when-checking-chunk-map-includes-logical-address/20231121-214139 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next patch link: https://lore.kernel.org/r/777320fd09dfc68a89180723bf5d7368dab06299.1700573314.git.fdmanana%40suse.com patch subject: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps config: i386-randconfig-061-20231122 (https://download.01.org/0day-ci/archive/20231122/[email protected]/config) compiler: clang version 16.0.4 (https://github.com/llvm/llvm-project.git ae42196bc493ffe877a7e3dff8be32035dea4d07) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231122/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
fs/btrfs/zoned.c:1293:12: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map)
^
>> fs/btrfs/zoned.c:1296:35: error: incomplete definition of type 'struct map_lookup'
struct btrfs_device *device = map->stripes[zone_idx].dev;
~~~^
fs/btrfs/zoned.c:1293:12: note: forward declaration of 'struct map_lookup'
struct map_lookup *map)
^
fs/btrfs/zoned.c:1302:22: error: incomplete definition of type 'struct map_lookup'
info->physical = map->stripes[zone_idx].physical;
~~~^
fs/btrfs/zoned.c:1293:12: note: forward declaration of 'struct map_lookup'
struct map_lookup *map)
^
fs/btrfs/zoned.c:1396:18: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1402:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1396:18: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1438:13: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1445:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1447:36: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1451:21: error: incomplete definition of type 'struct map_lookup'
for (i = 0; i < map->num_stripes; i++) {
~~~^
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1460:37: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1438:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1486:13: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1492:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1486:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1494:36: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1486:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1498:25: error: incomplete definition of type 'struct map_lookup'
for (int i = 0; i < map->num_stripes; i++) {
~~~^
fs/btrfs/zoned.c:1486:13: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1518:14: warning: declaration of 'struct map_lookup' will not be visible outside of this function [-Wvisibility]
struct map_lookup *map,
^
fs/btrfs/zoned.c:1524:10: error: incomplete definition of type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
~~~^
fs/btrfs/zoned.c:1518:14: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1526:36: error: incomplete definition of type 'struct map_lookup'
btrfs_bg_type_to_raid_name(map->type));
~~~^
fs/btrfs/messages.h:46:40: note: expanded from macro 'btrfs_err'
btrfs_printk(fs_info, KERN_ERR fmt, ##args)
^~~~
fs/btrfs/messages.h:27:32: note: expanded from macro 'btrfs_printk'
_btrfs_printk(fs_info, fmt, ##args)
^~~~
fs/btrfs/zoned.c:1518:14: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1530:25: error: incomplete definition of type 'struct map_lookup'
for (int i = 0; i < map->num_stripes; i++) {
~~~^
fs/btrfs/zoned.c:1518:14: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
fs/btrfs/zoned.c:1543:15: error: incomplete definition of type 'struct map_lookup'
if ((i % map->sub_stripes) == 0) {
~~~^
fs/btrfs/zoned.c:1518:14: note: forward declaration of 'struct map_lookup'
struct map_lookup *map,
^
>> fs/btrfs/zoned.c:1555:26: error: incompatible pointer types initializing 'struct extent_map_tree *' with an expression of type 'struct rb_root_cached *' [-Werror,-Wincompatible-pointer-types]
struct extent_map_tree *em_tree = &fs_info->mapping_tree;
^ ~~~~~~~~~~~~~~~~~~~~~~
>> fs/btrfs/zoned.c:1600:65: error: incompatible pointer types passing 'struct btrfs_chunk_map *' to parameter of type 'struct map_lookup *' [-Werror,-Wincompatible-pointer-types]
ret = btrfs_load_zone_info(fs_info, i, &zone_info[i], active, map);
^~~
fs/btrfs/zoned.c:1293:24: note: passing argument to parameter 'map' here
struct map_lookup *map)
^
fs/btrfs/zoned.c:1634:43: error: incompatible pointer types passing 'struct btrfs_chunk_map *' to parameter of type 'struct map_lookup *' [-Werror,-Wincompatible-pointer-types]
ret = btrfs_load_block_group_dup(cache, map, zone_info, active);
^~~
fs/btrfs/zoned.c:1396:30: note: passing argument to parameter 'map' here
struct map_lookup *map,
^
fs/btrfs/zoned.c:1639:45: error: incompatible pointer types passing 'struct btrfs_chunk_map *' to parameter of type 'struct map_lookup *' [-Werror,-Wincompatible-pointer-types]
ret = btrfs_load_block_group_raid1(cache, map, zone_info, active);
^~~
fs/btrfs/zoned.c:1438:25: note: passing argument to parameter 'map' here
struct map_lookup *map,
^
fs/btrfs/zoned.c:1642:45: error: incompatible pointer types passing 'struct btrfs_chunk_map *' to parameter of type 'struct map_lookup *' [-Werror,-Wincompatible-pointer-types]
ret = btrfs_load_block_group_raid0(cache, map, zone_info, active);
^~~ fs/btrfs/zoned.c:1486:25: note: passing argument to parameter 'map' here
struct map_lookup *map,
^
fatal error: too many errors emitted, stopping now [-ferror-limit=] 5 warnings and 20 errors generated.
vim +1296 fs/btrfs/zoned.c
15c12fcc50a1b12 Christoph Hellwig 2023-06-05 1290
09a46725cc84165 Christoph Hellwig 2023-06-05 1291 static int btrfs_load_zone_info(struct btrfs_fs_info *fs_info, int zone_idx,
09a46725cc84165 Christoph Hellwig 2023-06-05 1292 struct zone_info *info, unsigned long *active,
09a46725cc84165 Christoph Hellwig 2023-06-05 @1293 struct map_lookup *map)
09a46725cc84165 Christoph Hellwig 2023-06-05 1294 {
09a46725cc84165 Christoph Hellwig 2023-06-05 1295 struct btrfs_dev_replace *dev_replace = &fs_info->dev_replace;
09a46725cc84165 Christoph Hellwig 2023-06-05 @1296 struct btrfs_device *device = map->stripes[zone_idx].dev;
09a46725cc84165 Christoph Hellwig 2023-06-05 1297 int dev_replace_is_ongoing = 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1298 unsigned int nofs_flag;
09a46725cc84165 Christoph Hellwig 2023-06-05 1299 struct blk_zone zone;
09a46725cc84165 Christoph Hellwig 2023-06-05 1300 int ret;
09a46725cc84165 Christoph Hellwig 2023-06-05 1301
09a46725cc84165 Christoph Hellwig 2023-06-05 1302 info->physical = map->stripes[zone_idx].physical;
09a46725cc84165 Christoph Hellwig 2023-06-05 1303
09a46725cc84165 Christoph Hellwig 2023-06-05 1304 if (!device->bdev) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1305 info->alloc_offset = WP_MISSING_DEV;
09a46725cc84165 Christoph Hellwig 2023-06-05 1306 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1307 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1308
09a46725cc84165 Christoph Hellwig 2023-06-05 1309 /* Consider a zone as active if we can allow any number of active zones. */
09a46725cc84165 Christoph Hellwig 2023-06-05 1310 if (!device->zone_info->max_active_zones)
09a46725cc84165 Christoph Hellwig 2023-06-05 1311 __set_bit(zone_idx, active);
09a46725cc84165 Christoph Hellwig 2023-06-05 1312
09a46725cc84165 Christoph Hellwig 2023-06-05 1313 if (!btrfs_dev_is_sequential(device, info->physical)) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1314 info->alloc_offset = WP_CONVENTIONAL;
09a46725cc84165 Christoph Hellwig 2023-06-05 1315 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1316 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1317
09a46725cc84165 Christoph Hellwig 2023-06-05 1318 /* This zone will be used for allocation, so mark this zone non-empty. */
09a46725cc84165 Christoph Hellwig 2023-06-05 1319 btrfs_dev_clear_zone_empty(device, info->physical);
09a46725cc84165 Christoph Hellwig 2023-06-05 1320
09a46725cc84165 Christoph Hellwig 2023-06-05 1321 down_read(&dev_replace->rwsem);
09a46725cc84165 Christoph Hellwig 2023-06-05 1322 dev_replace_is_ongoing = btrfs_dev_replace_is_ongoing(dev_replace);
09a46725cc84165 Christoph Hellwig 2023-06-05 1323 if (dev_replace_is_ongoing && dev_replace->tgtdev != NULL)
09a46725cc84165 Christoph Hellwig 2023-06-05 1324 btrfs_dev_clear_zone_empty(dev_replace->tgtdev, info->physical);
09a46725cc84165 Christoph Hellwig 2023-06-05 1325 up_read(&dev_replace->rwsem);
09a46725cc84165 Christoph Hellwig 2023-06-05 1326
09a46725cc84165 Christoph Hellwig 2023-06-05 1327 /*
09a46725cc84165 Christoph Hellwig 2023-06-05 1328 * The group is mapped to a sequential zone. Get the zone write pointer
09a46725cc84165 Christoph Hellwig 2023-06-05 1329 * to determine the allocation offset within the zone.
09a46725cc84165 Christoph Hellwig 2023-06-05 1330 */
09a46725cc84165 Christoph Hellwig 2023-06-05 1331 WARN_ON(!IS_ALIGNED(info->physical, fs_info->zone_size));
09a46725cc84165 Christoph Hellwig 2023-06-05 1332 nofs_flag = memalloc_nofs_save();
09a46725cc84165 Christoph Hellwig 2023-06-05 1333 ret = btrfs_get_dev_zone(device, info->physical, &zone);
09a46725cc84165 Christoph Hellwig 2023-06-05 1334 memalloc_nofs_restore(nofs_flag);
09a46725cc84165 Christoph Hellwig 2023-06-05 1335 if (ret) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1336 if (ret != -EIO && ret != -EOPNOTSUPP)
09a46725cc84165 Christoph Hellwig 2023-06-05 1337 return ret;
09a46725cc84165 Christoph Hellwig 2023-06-05 1338 info->alloc_offset = WP_MISSING_DEV;
09a46725cc84165 Christoph Hellwig 2023-06-05 1339 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1340 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1341
09a46725cc84165 Christoph Hellwig 2023-06-05 1342 if (zone.type == BLK_ZONE_TYPE_CONVENTIONAL) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1343 btrfs_err_in_rcu(fs_info,
09a46725cc84165 Christoph Hellwig 2023-06-05 1344 "zoned: unexpected conventional zone %llu on device %s (devid %llu)",
09a46725cc84165 Christoph Hellwig 2023-06-05 1345 zone.start << SECTOR_SHIFT, rcu_str_deref(device->name),
09a46725cc84165 Christoph Hellwig 2023-06-05 1346 device->devid);
09a46725cc84165 Christoph Hellwig 2023-06-05 1347 return -EIO;
09a46725cc84165 Christoph Hellwig 2023-06-05 1348 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1349
09a46725cc84165 Christoph Hellwig 2023-06-05 1350 info->capacity = (zone.capacity << SECTOR_SHIFT);
09a46725cc84165 Christoph Hellwig 2023-06-05 1351
09a46725cc84165 Christoph Hellwig 2023-06-05 1352 switch (zone.cond) {
09a46725cc84165 Christoph Hellwig 2023-06-05 1353 case BLK_ZONE_COND_OFFLINE:
09a46725cc84165 Christoph Hellwig 2023-06-05 1354 case BLK_ZONE_COND_READONLY:
09a46725cc84165 Christoph Hellwig 2023-06-05 1355 btrfs_err(fs_info,
09a46725cc84165 Christoph Hellwig 2023-06-05 1356 "zoned: offline/readonly zone %llu on device %s (devid %llu)",
09a46725cc84165 Christoph Hellwig 2023-06-05 1357 (info->physical >> device->zone_info->zone_size_shift),
09a46725cc84165 Christoph Hellwig 2023-06-05 1358 rcu_str_deref(device->name), device->devid);
09a46725cc84165 Christoph Hellwig 2023-06-05 1359 info->alloc_offset = WP_MISSING_DEV;
09a46725cc84165 Christoph Hellwig 2023-06-05 1360 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1361 case BLK_ZONE_COND_EMPTY:
09a46725cc84165 Christoph Hellwig 2023-06-05 1362 info->alloc_offset = 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1363 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1364 case BLK_ZONE_COND_FULL:
09a46725cc84165 Christoph Hellwig 2023-06-05 1365 info->alloc_offset = info->capacity;
09a46725cc84165 Christoph Hellwig 2023-06-05 1366 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1367 default:
09a46725cc84165 Christoph Hellwig 2023-06-05 1368 /* Partially used zone. */
09a46725cc84165 Christoph Hellwig 2023-06-05 1369 info->alloc_offset = ((zone.wp - zone.start) << SECTOR_SHIFT);
09a46725cc84165 Christoph Hellwig 2023-06-05 1370 __set_bit(zone_idx, active);
09a46725cc84165 Christoph Hellwig 2023-06-05 1371 break;
09a46725cc84165 Christoph Hellwig 2023-06-05 1372 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1373
09a46725cc84165 Christoph Hellwig 2023-06-05 1374 return 0;
09a46725cc84165 Christoph Hellwig 2023-06-05 1375 }
09a46725cc84165 Christoph Hellwig 2023-06-05 1376 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps
In reply to [email protected]kernel test robot noticed the following build errors:
Hi,
kernel test robot noticed the following build errors:
[auto build test ERROR on kdave/for-next]
[also build test ERROR on linus/master v6.7-rc2 next-20231121] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/fdmanana-kernel-org/btrfs-fix-off-by-one-when-checking-chunk-map-includes-logical-address/20231121-214139 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next patch link: https://lore.kernel.org/r/777320fd09dfc68a89180723bf5d7368dab06299.1700573314.git.fdmanana%40suse.com patch subject: [PATCH 7/8] btrfs: use a dedicated data structure for chunk maps config: x86_64-randconfig-121-20231122 (https://download.01.org/0day-ci/archive/20231122/[email protected]/config) compiler: gcc-7 (Ubuntu 7.5.0-6ubuntu2) 7.5.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231122/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All error/warnings (new ones prefixed by >>):
>> fs/btrfs/zoned.c:1293:12: warning: 'struct map_lookup' declared inside parameter list will not be visible outside of this definition or declaration
struct map_lookup *map)
^~~~~~~~~~
fs/btrfs/zoned.c: In function 'btrfs_load_zone_info':
>> fs/btrfs/zoned.c:1296:35: error: dereferencing pointer to incomplete type 'struct map_lookup'
struct btrfs_device *device = map->stripes[zone_idx].dev;
^~
fs/btrfs/zoned.c: At top level:
fs/btrfs/zoned.c:1396:18: warning: 'struct map_lookup' declared inside parameter list will not be visible outside of this definition or declaration
struct map_lookup *map,
^~~~~~~~~~
fs/btrfs/zoned.c: In function 'btrfs_load_block_group_dup':
fs/btrfs/zoned.c:1402:10: error: dereferencing pointer to incomplete type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
^~
fs/btrfs/zoned.c: At top level:
fs/btrfs/zoned.c:1438:13: warning: 'struct map_lookup' declared inside parameter list will not be visible outside of this definition or declaration
struct map_lookup *map,
^~~~~~~~~~
fs/btrfs/zoned.c: In function 'btrfs_load_block_group_raid1':
fs/btrfs/zoned.c:1445:10: error: dereferencing pointer to incomplete type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
^~
fs/btrfs/zoned.c: At top level:
fs/btrfs/zoned.c:1486:13: warning: 'struct map_lookup' declared inside parameter list will not be visible outside of this definition or declaration
struct map_lookup *map,
^~~~~~~~~~
fs/btrfs/zoned.c: In function 'btrfs_load_block_group_raid0':
fs/btrfs/zoned.c:1492:10: error: dereferencing pointer to incomplete type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
^~
fs/btrfs/zoned.c: At top level:
fs/btrfs/zoned.c:1518:14: warning: 'struct map_lookup' declared inside parameter list will not be visible outside of this definition or declaration
struct map_lookup *map,
^~~~~~~~~~
fs/btrfs/zoned.c: In function 'btrfs_load_block_group_raid10':
fs/btrfs/zoned.c:1524:10: error: dereferencing pointer to incomplete type 'struct map_lookup'
if ((map->type & BTRFS_BLOCK_GROUP_DATA) && !fs_info->stripe_root) {
^~
fs/btrfs/zoned.c: In function 'btrfs_load_block_group_zone_info':
>> fs/btrfs/zoned.c:1555:36: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
struct extent_map_tree *em_tree = &fs_info->mapping_tree;
^
>> fs/btrfs/zoned.c:1600:65: error: passing argument 5 of 'btrfs_load_zone_info' from incompatible pointer type [-Werror=incompatible-pointer-types]
ret = btrfs_load_zone_info(fs_info, i, &zone_info[i], active, map);
^~~
fs/btrfs/zoned.c:1291:12: note: expected 'struct map_lookup *' but argument is of type 'struct btrfs_chunk_map *'
static int btrfs_load_zone_info(struct btrfs_fs_info *fs_info, int zone_idx,
^~~~~~~~~~~~~~~~~~~~
>> fs/btrfs/zoned.c:1634:43: error: passing argument 2 of 'btrfs_load_block_group_dup' from incompatible pointer type [-Werror=incompatible-pointer-types]
ret = btrfs_load_block_group_dup(cache, map, zone_info, active);
^~~
fs/btrfs/zoned.c:1395:12: note: expected 'struct map_lookup *' but argument is of type 'struct btrfs_chunk_map *'
static int btrfs_load_block_group_dup(struct btrfs_block_group *bg,
^~~~~~~~~~~~~~~~~~~~~~~~~~
>> fs/btrfs/zoned.c:1639:45: error: passing argument 2 of 'btrfs_load_block_group_raid1' from incompatible pointer type [-Werror=incompatible-pointer-types]
ret = btrfs_load_block_group_raid1(cache, map, zone_info, active);
^~~
fs/btrfs/zoned.c:1437:12: note: expected 'struct map_lookup *' but argument is of type 'struct btrfs_chunk_map *'
static int btrfs_load_block_group_raid1(struct btrfs_block_group *bg,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> fs/btrfs/zoned.c:1642:45: error: passing argument 2 of 'btrfs_load_block_group_raid0' from incompatible pointer type [-Werror=incompatible-pointer-types]
ret = btrfs_load_block_group_raid0(cache, map, zone_info, active);
^~~
fs/btrfs/zoned.c:1485:12: note: expected 'struct map_lookup *' but argument is of type 'struct btrfs_chunk_map *'
static int btrfs_load_block_group_raid0(struct btrfs_block_group *bg,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> fs/btrfs/zoned.c:1645:46: error: passing argument 2 of 'btrfs_load_block_group_raid10' from incompatible pointer type [-Werror=incompatible-pointer-types]
ret = btrfs_load_block_group_raid10(cache, map, zone_info, active);
^~~
fs/btrfs/zoned.c:1517:12: note: expected 'struct map_lookup *' but argument is of type 'struct btrfs_chunk_map *'
static int btrfs_load_block_group_raid10(struct btrfs_block_group *bg,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> fs/btrfs/zoned.c:1555:26: warning: unused variable 'em_tree' [-Wunused-variable]
struct extent_map_tree *em_tree = &fs_info->mapping_tree;
^~~~~~~cc1: some warnings being treated as errors
vim +1296 fs/btrfs/zoned.c
15c12fcc50a1b1 Christoph Hellwig 2023-06-05 1290
09a46725cc8416 Christoph Hellwig 2023-06-05 1291 static int btrfs_load_zone_info(struct btrfs_fs_info *fs_info, int zone_idx,
09a46725cc8416 Christoph Hellwig 2023-06-05 1292 struct zone_info *info, unsigned long *active,
09a46725cc8416 Christoph Hellwig 2023-06-05 @1293 struct map_lookup *map)
09a46725cc8416 Christoph Hellwig 2023-06-05 1294 {
09a46725cc8416 Christoph Hellwig 2023-06-05 1295 struct btrfs_dev_replace *dev_replace = &fs_info->dev_replace;
09a46725cc8416 Christoph Hellwig 2023-06-05 @1296 struct btrfs_device *device = map->stripes[zone_idx].dev;
09a46725cc8416 Christoph Hellwig 2023-06-05 1297 int dev_replace_is_ongoing = 0;
09a46725cc8416 Christoph Hellwig 2023-06-05 1298 unsigned int nofs_flag;
09a46725cc8416 Christoph Hellwig 2023-06-05 1299 struct blk_zone zone;
09a46725cc8416 Christoph Hellwig 2023-06-05 1300 int ret;
09a46725cc8416 Christoph Hellwig 2023-06-05 1301
09a46725cc8416 Christoph Hellwig 2023-06-05 1302 info->physical = map->stripes[zone_idx].physical;
09a46725cc8416 Christoph Hellwig 2023-06-05 1303
09a46725cc8416 Christoph Hellwig 2023-06-05 1304 if (!device->bdev) {
09a46725cc8416 Christoph Hellwig 2023-06-05 1305 info->alloc_offset = WP_MISSING_DEV;
09a46725cc8416 Christoph Hellwig 2023-06-05 1306 return 0;
09a46725cc8416 Christoph Hellwig 2023-06-05 1307 }
09a46725cc8416 Christoph Hellwig 2023-06-05 1308
09a46725cc8416 Christoph Hellwig 2023-06-05 1309 /* Consider a zone as active if we can allow any number of active zones. */
09a46725cc8416 Christoph Hellwig 2023-06-05 1310 if (!device->zone_info->max_active_zones)
09a46725cc8416 Christoph Hellwig 2023-06-05 1311 __set_bit(zone_idx, active);
09a46725cc8416 Christoph Hellwig 2023-06-05 1312
09a46725cc8416 Christoph Hellwig 2023-06-05 1313 if (!btrfs_dev_is_sequential(device, info->physical)) {
09a46725cc8416 Christoph Hellwig 2023-06-05 1314 info->alloc_offset = WP_CONVENTIONAL;
09a46725cc8416 Christoph Hellwig 2023-06-05 1315 return 0;
09a46725cc8416 Christoph Hellwig 2023-06-05 1316 }
09a46725cc8416 Christoph Hellwig 2023-06-05 1317
09a46725cc8416 Christoph Hellwig 2023-06-05 1318 /* This zone will be used for allocation, so mark this zone non-empty. */
09a46725cc8416 Christoph Hellwig 2023-06-05 1319 btrfs_dev_clear_zone_empty(device, info->physical);
09a46725cc8416 Christoph Hellwig 2023-06-05 1320
09a46725cc8416 Christoph Hellwig 2023-06-05 1321 down_read(&dev_replace->rwsem);
09a46725cc8416 Christoph Hellwig 2023-06-05 1322 dev_replace_is_ongoing = btrfs_dev_replace_is_ongoing(dev_replace);
09a46725cc8416 Christoph Hellwig 2023-06-05 1323 if (dev_replace_is_ongoing && dev_replace->tgtdev != NULL)
09a46725cc8416 Christoph Hellwig 2023-06-05 1324 btrfs_dev_clear_zone_empty(dev_replace->tgtdev, info->physical);
09a46725cc8416 Christoph Hellwig 2023-06-05 1325 up_read(&dev_replace->rwsem);
09a46725cc8416 Christoph Hellwig 2023-06-05 1326
09a46725cc8416 Christoph Hellwig 2023-06-05 1327 /*
09a46725cc8416 Christoph Hellwig 2023-06-05 1328 * The group is mapped to a sequential zone. Get the zone write pointer
09a46725cc8416 Christoph Hellwig 2023-06-05 1329 * to determine the allocation offset within the zone.
09a46725cc8416 Christoph Hellwig 2023-06-05 1330 */
09a46725cc8416 Christoph Hellwig 2023-06-05 1331 WARN_ON(!IS_ALIGNED(info->physical, fs_info->zone_size));
09a46725cc8416 Christoph Hellwig 2023-06-05 1332 nofs_flag = memalloc_nofs_save();
09a46725cc8416 Christoph Hellwig 2023-06-05 1333 ret = btrfs_get_dev_zone(device, info->physical, &zone);
09a46725cc8416 Christoph Hellwig 2023-06-05 1334 memalloc_nofs_restore(nofs_flag);
09a46725cc8416 Christoph Hellwig 2023-06-05 1335 if (ret) {
09a46725cc8416 Christoph Hellwig 2023-06-05 1336 if (ret != -EIO && ret != -EOPNOTSUPP)
09a46725cc8416 Christoph Hellwig 2023-06-05 1337 return ret;
09a46725cc8416 Christoph Hellwig 2023-06-05 1338 info->alloc_offset = WP_MISSING_DEV;
09a46725cc8416 Christoph Hellwig 2023-06-05 1339 return 0;
09a46725cc8416 Christoph Hellwig 2023-06-05 1340 }
09a46725cc8416 Christoph Hellwig 2023-06-05 1341
09a46725cc8416 Christoph Hellwig 2023-06-05 1342 if (zone.type == BLK_ZONE_TYPE_CONVENTIONAL) {
09a46725cc8416 Christoph Hellwig 2023-06-05 1343 btrfs_err_in_rcu(fs_info,
09a46725cc8416 Christoph Hellwig 2023-06-05 1344 "zoned: unexpected conventional zone %llu on device %s (devid %llu)",
09a46725cc8416 Christoph Hellwig 2023-06-05 1345 zone.start << SECTOR_SHIFT, rcu_str_deref(device->name),
09a46725cc8416 Christoph Hellwig 2023-06-05 1346 device->devid);
09a46725cc8416 Christoph Hellwig 2023-06-05 1347 return -EIO;
09a46725cc8416 Christoph Hellwig 2023-06-05 1348 }
09a46725cc8416 Christoph Hellwig 2023-06-05 1349
09a46725cc8416 Christoph Hellwig 2023-06-05 1350 info->capacity = (zone.capacity << SECTOR_SHIFT);
09a46725cc8416 Christoph Hellwig 2023-06-05 1351
09a46725cc8416 Christoph Hellwig 2023-06-05 1352 switch (zone.cond) {
09a46725cc8416 Christoph Hellwig 2023-06-05 1353 case BLK_ZONE_COND_OFFLINE:
09a46725cc8416 Christoph Hellwig 2023-06-05 1354 case BLK_ZONE_COND_READONLY:
09a46725cc8416 Christoph Hellwig 2023-06-05 1355 btrfs_err(fs_info,
09a46725cc8416 Christoph Hellwig 2023-06-05 1356 "zoned: offline/readonly zone %llu on device %s (devid %llu)",
09a46725cc8416 Christoph Hellwig 2023-06-05 1357 (info->physical >> device->zone_info->zone_size_shift),
09a46725cc8416 Christoph Hellwig 2023-06-05 1358 rcu_str_deref(device->name), device->devid);
09a46725cc8416 Christoph Hellwig 2023-06-05 1359 info->alloc_offset = WP_MISSING_DEV;
09a46725cc8416 Christoph Hellwig 2023-06-05 1360 break;
09a46725cc8416 Christoph Hellwig 2023-06-05 1361 case BLK_ZONE_COND_EMPTY:
09a46725cc8416 Christoph Hellwig 2023-06-05 1362 info->alloc_offset = 0;
09a46725cc8416 Christoph Hellwig 2023-06-05 1363 break;
09a46725cc8416 Christoph Hellwig 2023-06-05 1364 case BLK_ZONE_COND_FULL:
09a46725cc8416 Christoph Hellwig 2023-06-05 1365 info->alloc_offset = info->capacity;
09a46725cc8416 Christoph Hellwig 2023-06-05 1366 break;
09a46725cc8416 Christoph Hellwig 2023-06-05 1367 default:
09a46725cc8416 Christoph Hellwig 2023-06-05 1368 /* Partially used zone. */
09a46725cc8416 Christoph Hellwig 2023-06-05 1369 info->alloc_offset = ((zone.wp - zone.start) << SECTOR_SHIFT);
09a46725cc8416 Christoph Hellwig 2023-06-05 1370 __set_bit(zone_idx, active);
09a46725cc8416 Christoph Hellwig 2023-06-05 1371 break;
09a46725cc8416 Christoph Hellwig 2023-06-05 1372 }
09a46725cc8416 Christoph Hellwig 2023-06-05 1373
09a46725cc8416 Christoph Hellwig 2023-06-05 1374 return 0;
09a46725cc8416 Christoph Hellwig 2023-06-05 1375 }
09a46725cc8416 Christoph Hellwig 2023-06-05 1376 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH 8/8 btrfs: remove stripe size local variable from insert_dev_extents()
In reply to [email protected]From: Filipe Manana <[email protected]>
It's not needed to have a local variable to store the stripe size at insert_dev_extents(), we can just take from the chunk map as it's only used once and typing 'map->stripe_size' is not much more verbose than simply typing 'stripe_size'. So remove the local variable.
This was added before the recent addition of a dedicated structure for chunk mappings because the stripe size was encoded in the 'orig_block_len' field of an extent_map structure, so the use of the local variable made things more readable.
--- fs/btrfs/block-group.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
fs/btrfs/block-group.c +1 -4
@@ -2597,7 +2597,6 @@ static int insert_dev_extents(struct btrfs_trans_handle *trans,struct btrfs_device *device;struct btrfs_chunk_map *map;u64 dev_offset; -u64 stripe_size;int i;int ret =0 ;
@@ -2605,8 +2604,6 @@ static int insert_dev_extents(struct btrfs_trans_handle *trans,if (IS_ERR (map))return PTR_ERR (map); - stripe_size = map->stripe_size; -/* * Take the device list mutex to prevent races with the final phase of * a device replace operation that replaces the device object associated
@@ -2622,7 +2619,7 @@ static int insert_dev_extents(struct btrfs_trans_handle *trans, dev_offset = map->stripes[i].physical; ret =insert_dev_extent (trans, device, chunk_offset, dev_offset, - stripe_size); + map->stripe_size);if (ret)break ; }
Signature · 1 line
2.40.1
Quoted 7 lines · On Tue, Nov 21, 2023 at 01:38:31PM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
The following are some cleanups and introducing a dedicated data structure for representing chunk maps, in order to make code simpler and use less memory - this is achieved in patch 7/8. This patchset is also preparatory work for some upcoming changes to extent maps.
Quoted 10 lines
Filipe Manana (8):
btrfs: fix off-by-one when checking chunk map includes logical address
btrfs: make error messages more clear when getting a chunk map
btrfs: mark sanity checks when getting chunk map as unlikely
btrfs: split assert into two different asserts when removing block group
btrfs: unexportextent_map_block_end()
btrfs: usebtrfs_next_item()at scrub.c:find_first_extent_item()
btrfs: use a dedicated data structure for chunk maps
btrfs: remove stripe size local variable frominsert_dev_extents()
Thanks,
Josef
Quoted 7 lines · On Tue, Nov 21, 2023 at 01:38:31PM +0000, [email protected] wrote:
From: Filipe Manana <[email protected]>
The following are some cleanups and introducing a dedicated data structure for representing chunk maps, in order to make code simpler and use less memory - this is achieved in patch 7/8. This patchset is also preparatory work for some upcoming changes to extent maps.
Quoted 9 lines
Filipe Manana (8):
btrfs: fix off-by-one when checking chunk map includes logical address
btrfs: make error messages more clear when getting a chunk map
btrfs: mark sanity checks when getting chunk map as unlikely
btrfs: split assert into two different asserts when removing block group
btrfs: unexportextent_map_block_end()
btrfs: usebtrfs_next_item()at scrub.c:find_first_extent_item()
btrfs: use a dedicated data structure for chunk maps
btrfs: remove stripe size local variable frominsert_dev_extents()
I have applied this to misc-next, withot the reordering in 7/8. This can be done later eventually.