btrfs: find_free_extent cleanups
Subsystems (2)
v2 cover btrfs: find_free_extent cleanups
The first patch removes redundant RAID loop logic that became obsolete after previous changes ensured allocations only occur from block groups with matching RAID level.
The second patch adds comprehensive tracing to find_free_extent() to improve debugging and performance analysis capabilities.
Here is a bpftrace script I put together to analyze the allocation behavior, along with output.
Testing:
- ran xfstests btrfs/auto
- verified trace events output correctly
- new fstest that tests RAID conversions under stress
Change log:
v1 -> v2:
PATCH 1
- re-add full_search
PATCH 2
- standardize naming of skip reasons
- remove preapare_allocation_failure reason as it's not a skip
- add more error_or_values for skip reasons
PATCH 3
- new fstests for raid conversions under stress
Leo Martins (2):
btrfs: remove ffe RAID loop
btrfs: add tracing for find_free_extent skip conditions
fs/btrfs/extent-tree.c | 70 ++++++++++++++++++------------------ fs/btrfs/extent-tree.h | 17 +++++++++ include/trace/events/btrfs.h | 66 ++++++++++++++++++++++++++++++++++ 3 files changed, 119 insertions(+), 34 deletions(-)
Signature · 1 line
2.47.3
PATCH v2 1/3 btrfs: remove ffe RAID loop
This patch removes the RAID loop from find_free_extent since it is impossible to allocate from a block group with a different RAID profile.
Historically, we've been able to fulfill allocation requests from mismatched RAID block groups assuming they provided the required duplcation. For example, a request for RAID0 could be fulfilled by a RAID1 block group.
2a28468e525f ("btrfs: extent-tree: Make sure we only allocate extents from block groups with the same type") changed this behavior to skip block groups with different flags
than the request. This makes the duplication compatiblity check
redundant since we're going to keep searching regardless.
--- fs/btrfs/extent-tree.c | 32 +------------------------------- 1 file changed, 1 insertion(+), 31 deletions(-)
fs/btrfs/extent-tree.c +1 -31
@@ -4171,13 +4171,8 @@ static int find_free_extent_update_loop(struct btrfs_fs_info *fs_info,if (ffe_ctl->loop >=LOOP_CACHING_WAIT && ffe_ctl->have_caching_bg)return 1 ; - ffe_ctl->index++; -if (ffe_ctl->index <BTRFS_NR_RAID_TYPES ) -return 1 ; -/* See the comments for btrfs_loop_type for an explanation of the phases. */ if (ffe_ctl->loop <LOOP_NO_EMPTY_SIZE ) { - ffe_ctl->index =0 ;/* * We want to skip the LOOP_CACHING_WAIT step if we don't have * any uncached bgs and we've already done a full search
@@ -4477,9 +4472,7 @@ static noinline int find_free_extent(struct btrfs_root *root, search:trace_btrfs_find_free_extent_search_loop (root, ffe_ctl); ffe_ctl->have_caching_bg = false; -if (ffe_ctl->index ==btrfs_bg_flags_to_raid_index (ffe_ctl->flags) || - ffe_ctl->index ==0 ) - full_search = true; + full_search = true;down_read (&space_info->groups_sem);list_for_each_entry (block_group, &space_info->block_groups[ffe_ctl->index], list) {
@@ -4498,30 +4491,7 @@ static noinline int find_free_extent(struct btrfs_root *root,btrfs_grab_block_group (block_group, ffe_ctl->delalloc); ffe_ctl->search_start = block_group->start; -/* -* this can happen if we end up cycling through all the -* raid types, but we want to make sure we only allocate -* for the proper type. -*/ if (!block_group_bits (block_group, ffe_ctl->flags)) { -u64 extra =BTRFS_BLOCK_GROUP_DUP | -BTRFS_BLOCK_GROUP_RAID1_MASK | -BTRFS_BLOCK_GROUP_RAID56_MASK | -BTRFS_BLOCK_GROUP_RAID10 ; - -/* -* if they asked for extra copies and this block group -* doesn't provide them, bail. This does allow us to -* fill raid0 from raid1. -*/ -if ((ffe_ctl->flags & extra) && !(block_group->flags & extra)) - goto loop; - -/* -* This block group has different flags than we want. -* It's possible that we have MIXED_GROUP flag but no -* block group is mixed. Just skip such block group. -*/ btrfs_release_block_group (block_group, ffe_ctl->delalloc);continue ; }
Signature · 1 line
2.47.3
Re: [PATCH v2 1/3] btrfs: remove ffe RAID loop
Quoted 9 lines · Leo Martins
This patch removes the RAID loop from
find_free_extentsince it is impossible to allocate from a block group with a different RAID profile.Historically, we've been able to fulfill allocation requests from mismatched RAID block groups assuming they provided the required duplcation. For example, a request for RAID0 could be fulfilled by a RAID1 block group.
Quoted 6 lines
2a28468e525f("btrfs: extent-tree: Make sure we only allocate extents from block groups with the same type") changed this behavior to skip block groups with different flags
than the request. This makes the duplication compatiblity check
redundant since we're going to keep searching regardless.Signed-off-by Leo Martins <[email protected]>
Quoted 67 lines
--- fs/btrfs/extent-tree.c | 32 +------------------------------- 1 file changed, 1 insertion(+), 31 deletions(-)fs/btrfs/extent-tree.c +1
-31@@ -4171,13 +4171,8 @@ static int find_free_extent_update_loop(struct btrfs_fs_info *fs_info,if (ffe_ctl->loop >=LOOP_CACHING_WAIT && ffe_ctl->have_caching_bg)return 1 ; - ffe_ctl->index++; -if (ffe_ctl->index <BTRFS_NR_RAID_TYPES ) -return 1 ; -/* See the comments for btrfs_loop_type for an explanation of the phases. */ if (ffe_ctl->loop <LOOP_NO_EMPTY_SIZE ) { - ffe_ctl->index =0 ;/* * We want to skip the LOOP_CACHING_WAIT step if we don't have * any uncached bgs and we've already done a full search @@ -4477,9 +4472,7 @@ static noinline int find_free_extent(struct btrfs_root *root, search:trace_btrfs_find_free_extent_search_loop (root, ffe_ctl); ffe_ctl->have_caching_bg = false; -if (ffe_ctl->index ==btrfs_bg_flags_to_raid_index (ffe_ctl->flags) || - ffe_ctl->index ==0 ) - full_search = true; + full_search = true;down_read (&space_info->groups_sem);list_for_each_entry (block_group, &space_info->block_groups[ffe_ctl->index], list) {@@ -4498,30 +4491,7 @@ static noinline int find_free_extent(struct btrfs_root *root,btrfs_grab_block_group (block_group, ffe_ctl->delalloc); ffe_ctl->search_start = block_group->start; -/* -* this can happen if we end up cycling through all the -* raid types, but we want to make sure we only allocate -* for the proper type. -*/ if (!block_group_bits (block_group, ffe_ctl->flags)) { -u64 extra =BTRFS_BLOCK_GROUP_DUP | -BTRFS_BLOCK_GROUP_RAID1_MASK | -BTRFS_BLOCK_GROUP_RAID56_MASK | -BTRFS_BLOCK_GROUP_RAID10 ; - -/* -* if they asked for extra copies and this block group -* doesn't provide them, bail. This does allow us to -* fill raid0 from raid1. -*/ -if ((ffe_ctl->flags & extra) && !(block_group->flags & extra)) - goto loop; - -/* -* This block group has different flags than we want. -* It's possible that we have MIXED_GROUP flag but no -* block group is mixed. Just skip such block group. -*/ btrfs_release_block_group (block_group, ffe_ctl->delalloc);continue ; }Signature · 1 line
2.47.3
PATCH v2 2/3 btrfs: add tracing for find_free_extent skip conditions
In reply to Leo MartinsAdd detailed tracing to the find_free_extent() function to improve observability of extent allocation behavior. This patch introduces:
- A new trace event btrfs_find_free_extent_skip_block_group() that captures
allocation context and skip reasons
- Comprehensive set of FFE_SKIP_BG_* constants defining specific reasons why block groups are skipped during allocation
- Trace points at all major skip conditions in the allocator loop
The trace event includes key allocation parameters (root, size, flags, loop iteration) and block group details (start, length, flags) along with the specific skip reason and associated error codes.
These trace points will help diagnose allocation performance issues, understand allocation patterns, and debug extent allocator behavior.
--- fs/btrfs/extent-tree.c | 38 +++++++++++++++++++-- fs/btrfs/extent-tree.h | 17 ++++++++++ include/trace/events/btrfs.h | 66 ++++++++++++++++++++++++++++++++++++ 3 files changed, 118 insertions(+), 3 deletions(-)
fs/btrfs/extent-tree.c +35 -3
@@ -4455,6 +4455,8 @@ static noinline int find_free_extent(struct btrfs_root *root,* target because our list pointers are not * valid */ +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_REMOVING ,0 );btrfs_put_block_group (block_group);up_read (&space_info->groups_sem); }else {
@@ -4466,6 +4468,15 @@ static noinline int find_free_extent(struct btrfs_root *root, goto have_block_group; } }else if (block_group) { +if (!block_group_bits (block_group, ffe_ctl->flags)) +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_WRONG_FLAGS , block_group->flags & ~ffe_ctl->flags); +else if (block_group->space_info != space_info) +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_WRONG_SPACE_INFO ,0 ); +else if (block_group->cached ==BTRFS_CACHE_NO ) +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_NOT_CACHED ,0 );btrfs_put_block_group (block_group); } }
@@ -4481,6 +4492,8 @@ static noinline int find_free_extent(struct btrfs_root *root, ffe_ctl->hinted = false;/* If the block group is read-only, we can skip it entirely. */ if (unlikely (block_group->ro)) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_READ_ONLY ,0 );if (ffe_ctl->for_treelog)btrfs_clear_treelog_bg (block_group);if (ffe_ctl->for_data_reloc)
@@ -4492,6 +4505,8 @@ static noinline int find_free_extent(struct btrfs_root *root, ffe_ctl->search_start = block_group->start;if (!block_group_bits (block_group, ffe_ctl->flags)) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_WRONG_FLAGS , block_group->flags & ~ffe_ctl->flags);btrfs_release_block_group (block_group, ffe_ctl->delalloc);continue ; }
@@ -4511,6 +4526,8 @@ static noinline int find_free_extent(struct btrfs_root *root,* error that caused problems, not ENOSPC. */ if (ret <0 ) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_CACHE_ERROR , ret);if (!cache_block_group_error) cache_block_group_error = ret; ret =0 ;
@@ -4520,18 +4537,26 @@ static noinline int find_free_extent(struct btrfs_root *root, }if (unlikely (block_group->cached ==BTRFS_CACHE_ERROR )) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_CACHE_ERROR , -EIO );if (!cache_block_group_error) cache_block_group_error = -EIO ; goto loop; } -if (!find_free_extent_check_size_class (ffe_ctl, block_group)) +if (!find_free_extent_check_size_class (ffe_ctl, block_group)) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_SIZE_CLASS_MISMATCH , block_group->size_class); goto loop; + } bg_ret =NULL ; ret =do_allocation (block_group, ffe_ctl, &bg_ret); -if (ret >0 ) +if (ret >0 ) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_DO_ALLOCATION_FAILED , ret); goto loop; + }if (bg_ret && bg_ret != block_group) {btrfs_release_block_group (block_group, ffe_ctl->delalloc);
@@ -4545,22 +4570,29 @@ static noinline int find_free_extent(struct btrfs_root *root,/* move on to the next group */ if (ffe_ctl->search_start + ffe_ctl->num_bytes > block_group->start + block_group->length) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_SEARCH_BOUNDS , block_group->start + block_group->length);btrfs_add_free_space_unused (block_group, ffe_ctl->found_offset, ffe_ctl->num_bytes); goto loop; } -if (ffe_ctl->found_offset < ffe_ctl->search_start) +if (ffe_ctl->found_offset < ffe_ctl->search_start) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_SEARCH_BOUNDS , ffe_ctl->found_offset);btrfs_add_free_space_unused (block_group, ffe_ctl->found_offset, ffe_ctl->search_start - ffe_ctl->found_offset); + } ret =btrfs_add_reserved_bytes (block_group, ffe_ctl->ram_bytes, ffe_ctl->num_bytes, ffe_ctl->delalloc, ffe_ctl->loop >=LOOP_WRONG_SIZE_CLASS );if (ret == -EAGAIN ) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_ADD_RESERVED_FAILED , -EAGAIN );btrfs_add_free_space_unused (block_group, ffe_ctl->found_offset, ffe_ctl->num_bytes);
fs/btrfs/extent-tree.h +17 -0
@@ -23,6 +23,23 @@ enum btrfs_extent_allocation_policy { BTRFS_EXTENT_ALLOC_ZONED, }; +/* +* Enum for find_free_extent skip reasons used in trace events. +* Each enum corresponds to a specific unhappy path in the allocator. +*/ +enum { +FFE_SKIP_BG_REMOVING , +FFE_SKIP_BG_READ_ONLY , +FFE_SKIP_BG_WRONG_SPACE_INFO , +FFE_SKIP_BG_WRONG_FLAGS , +FFE_SKIP_BG_NOT_CACHED , +FFE_SKIP_BG_CACHE_ERROR , +FFE_SKIP_BG_SIZE_CLASS_MISMATCH , +FFE_SKIP_BG_DO_ALLOCATION_FAILED , +FFE_SKIP_BG_SEARCH_BOUNDS , +FFE_SKIP_BG_ADD_RESERVED_FAILED , +}; +struct find_free_extent_ctl {/* Basic allocation info */ u64 ram_bytes;
include/trace/events/btrfs.h +66 -0
84 lines not on this page · raw message
Signature · 1 line
2.47.3
Re: [PATCH v2 2/3] btrfs: add tracing for find_free_extent skip conditions
Quoted 9 lines · Leo Martins
Add detailed tracing to the
find_free_extent()function to improve observability of extent allocation behavior. This patch introduces:Quoted 1 line
- A new trace event
btrfs_find_free_extent_skip_block_group()thatcaptures
allocation context and skip reasons
- Comprehensive set of FFE_SKIP_BG_* constants defining specific reasons why block groups are skipped during allocation
- Trace points at all major skip conditions in the allocator loop
Quoted 8 lines
The trace event includes key allocation parameters (root, size, flags, loop iteration) and block group details (start, length, flags) along with the specific skip reason and associated error codes.
These trace points will help diagnose allocation performance issues, understand allocation patterns, and debug extent allocator behavior.
One nit inline, but this looks great, thanks.
Quoted 23 lines
Signed-off-by Leo Martins <[email protected]>--- fs/btrfs/extent-tree.c | 38 +++++++++++++++++++-- fs/btrfs/extent-tree.h | 17 ++++++++++ include/trace/events/btrfs.h | 66 ++++++++++++++++++++++++++++++++++++ 3 files changed, 118 insertions(+), 3 deletions(-)fs/btrfs/extent-tree.c +2
-0@@ -4455,6 +4455,8 @@ static noinline int find_free_extent(struct btrfs_root *root,* target because our list pointers are not * valid */ +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_REMOVING ,0 );btrfs_put_block_group (block_group);up_read (&space_info->groups_sem); }else {@@ -4466,6 +4468,15 @@ static noinline int find_free_extent(struct btrfs_root *root, goto have_block_group; }
fs/btrfs/extent-tree.c static noinline int find_free_extent(struct btrfs_root *root, · line 4479
Quoted 10 lines
fs/btrfs/extent-tree.c +9
-0}else if (block_group) { +if (!block_group_bits (block_group, ffe_ctl->flags)) +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_WRONG_FLAGS , block_group->flags & ~ffe_ctl->flags); +else if (block_group->space_info != space_info) +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_WRONG_SPACE_INFO ,0 ); +else if (block_group->cached ==BTRFS_CACHE_NO ) +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_NOT_CACHED ,0 );
nit: I think this would look nicer with the code/data selected by the ifs and then a single trace call.
Quoted 212 lines
fs/btrfs/extent-tree.c +24
-3btrfs_put_block_group (block_group); } }@@ -4481,6 +4492,8 @@ static noinline int find_free_extent(struct btrfs_root *root, ffe_ctl->hinted = false;/* If the block group is read-only, we can skip it entirely. */ if (unlikely (block_group->ro)) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_READ_ONLY ,0 );if (ffe_ctl->for_treelog)btrfs_clear_treelog_bg (block_group);if (ffe_ctl->for_data_reloc)@@ -4492,6 +4505,8 @@ static noinline int find_free_extent(struct btrfs_root *root, ffe_ctl->search_start = block_group->start;if (!block_group_bits (block_group, ffe_ctl->flags)) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_WRONG_FLAGS , block_group->flags & ~ffe_ctl->flags);btrfs_release_block_group (block_group, ffe_ctl->delalloc);continue ; }@@ -4511,6 +4526,8 @@ static noinline int find_free_extent(struct btrfs_root *root,* error that caused problems, not ENOSPC. */ if (ret <0 ) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_CACHE_ERROR , ret);if (!cache_block_group_error) cache_block_group_error = ret; ret =0 ;@@ -4520,18 +4537,26 @@ static noinline int find_free_extent(struct btrfs_root *root, }if (unlikely (block_group->cached ==BTRFS_CACHE_ERROR )) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_CACHE_ERROR , -EIO );if (!cache_block_group_error) cache_block_group_error = -EIO ; goto loop; } -if (!find_free_extent_check_size_class (ffe_ctl, block_group)) +if (!find_free_extent_check_size_class (ffe_ctl, block_group)) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_SIZE_CLASS_MISMATCH , block_group->size_class); goto loop; + } bg_ret =NULL ; ret =do_allocation (block_group, ffe_ctl, &bg_ret); -if (ret >0 ) +if (ret >0 ) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_DO_ALLOCATION_FAILED , ret); goto loop; + }if (bg_ret && bg_ret != block_group) {btrfs_release_block_group (block_group, ffe_ctl->delalloc);@@ -4545,22 +4570,29 @@ static noinline int find_free_extent(struct btrfs_root *root,/* move on to the next group */ if (ffe_ctl->search_start + ffe_ctl->num_bytes > block_group->start + block_group->length) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_SEARCH_BOUNDS , block_group->start + block_group->length);btrfs_add_free_space_unused (block_group, ffe_ctl->found_offset, ffe_ctl->num_bytes); goto loop; } -if (ffe_ctl->found_offset < ffe_ctl->search_start) +if (ffe_ctl->found_offset < ffe_ctl->search_start) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_SEARCH_BOUNDS , ffe_ctl->found_offset);btrfs_add_free_space_unused (block_group, ffe_ctl->found_offset, ffe_ctl->search_start - ffe_ctl->found_offset); + } ret =btrfs_add_reserved_bytes (block_group, ffe_ctl->ram_bytes, ffe_ctl->num_bytes, ffe_ctl->delalloc, ffe_ctl->loop >=LOOP_WRONG_SIZE_CLASS );if (ret == -EAGAIN ) { +trace_btrfs_find_free_extent_skip_block_group (root, ffe_ctl, block_group, +FFE_SKIP_BG_ADD_RESERVED_FAILED , -EAGAIN );btrfs_add_free_space_unused (block_group, ffe_ctl->found_offset, ffe_ctl->num_bytes);fs/btrfs/extent-tree.h +17
-0@@ -23,6 +23,23 @@ enum btrfs_extent_allocation_policy { BTRFS_EXTENT_ALLOC_ZONED, }; +/* +* Enum for find_free_extent skip reasons used in trace events. +* Each enum corresponds to a specific unhappy path in the allocator. +*/ +enum { +FFE_SKIP_BG_REMOVING , +FFE_SKIP_BG_READ_ONLY , +FFE_SKIP_BG_WRONG_SPACE_INFO , +FFE_SKIP_BG_WRONG_FLAGS , +FFE_SKIP_BG_NOT_CACHED , +FFE_SKIP_BG_CACHE_ERROR , +FFE_SKIP_BG_SIZE_CLASS_MISMATCH , +FFE_SKIP_BG_DO_ALLOCATION_FAILED , +FFE_SKIP_BG_SEARCH_BOUNDS , +FFE_SKIP_BG_ADD_RESERVED_FAILED , +}; +struct find_free_extent_ctl {/* Basic allocation info */ u64 ram_bytes;include/trace/events/btrfs.h +66
-0@@ -103,6 +103,19 @@ struct find_free_extent_ctl; EM( COMMIT_TRANS, "COMMIT_TRANS") \ EMe(RESET_ZONES, "RESET_ZONES") +#define FIND_FREE_EXTENT_SKIP_REASONS \ + EM( FFE_SKIP_BG_REMOVING, "BG_REMOVING") \ + EM( FFE_SKIP_BG_READ_ONLY, "BG_READ_ONLY") \ + EM( FFE_SKIP_BG_WRONG_SPACE_INFO, "BG_WRONG_SPACE_INFO") \ + EM( FFE_SKIP_BG_WRONG_FLAGS, "BG_WRONG_FLAGS") \ + EM( FFE_SKIP_BG_NOT_CACHED, "BG_NOT_CACHED") \ + EM( FFE_SKIP_BG_CACHE_ERROR, "BG_CACHE_ERROR") \ + EM( FFE_SKIP_BG_SIZE_CLASS_MISMATCH, "BG_SIZE_CLASS_MISMATCH") \ + EM( FFE_SKIP_BG_DO_ALLOCATION_FAILED, "BG_DO_ALLOCATION_FAILED") \ + EM( FFE_SKIP_BG_SEARCH_BOUNDS, "BG_SEARCH_BOUNDS") \ + EMe(FFE_SKIP_BG_ADD_RESERVED_FAILED, "BG_ADD_RESERVED_FAILED") + +/* * First define the enums in the above macros to be exported to userspace via * TRACE_DEFINE_ENUM(). @@ -118,6 +131,7 @@ FI_TYPESQGROUP_RSV_TYPES IO_TREE_OWNERFLUSH_STATES +FIND_FREE_EXTENT_SKIP_REASONS /* * Now redefine the EM and EMe macros to map the enums to the strings that will @@ -1388,6 +1402,58 @@ DEFINE_EVENT(btrfs__reserve_extent, btrfs_reserve_extent_cluster,TP_ARGS (block_group, ffe_ctl) ); +TRACE_EVENT (btrfs_find_free_extent_skip_block_group, + + TP_PROTO(const struct btrfs_root *root, + const struct find_free_extent_ctl *ffe_ctl, + const struct btrfs_block_group *block_group, + int reason, + s64 error_or_value), + +TP_ARGS (root, ffe_ctl, block_group, reason, error_or_value), + +TP_STRUCT__entry_btrfs ( + __field( u64, root_objectid ) + __field( u64, num_bytes ) + __field( u64, empty_size ) + __field( u64, flags ) + __field( int, loop ) + __field( bool, hinted ) + __field( int, size_class ) + __field( u64, bg_start ) + __field( u64, bg_length ) + __field( u64, bg_flags ) + __field( int, reason ) +__field ( s64, error_or_value ) + ), + +TP_fast_assign_btrfs (root->fs_info, + __entry->root_objectid = btrfs_root_id(root); + __entry->num_bytes = ffe_ctl->num_bytes; + __entry->empty_size = ffe_ctl->empty_size; + __entry->flags = ffe_ctl->flags; + __entry->loop = ffe_ctl->loop; + __entry->hinted = ffe_ctl->hinted; + __entry->size_class = ffe_ctl->size_class; + __entry->bg_start = block_group ? block_group->start : 0; + __entry->bg_length = block_group ? block_group->length : 0; + __entry->bg_flags = block_group ? block_group->flags : 0; + __entry->reason = reason; + __entry->error_or_value = error_or_value; + ), + +TP_printk_btrfs ( +"root=%llu(%s) len=%llu empty_size=%llu flags=%llu(%s) loop=%d hinted=%d size_class=%d bg=[%llu+%llu] bg_flags=%llu(%s) reason=%s error_or_value=%lld" , +show_root_type (__entry->root_objectid), + __entry->num_bytes, __entry->empty_size, __entry->flags, +__print_flags ((unsigned long )__entry->flags,"|" ,BTRFS_GROUP_FLAGS ), + __entry->loop, __entry->hinted, __entry->size_class, + __entry->bg_start, __entry->bg_length, __entry->bg_flags, +__print_flags ((unsigned long )__entry->bg_flags,"|" ,BTRFS_GROUP_FLAGS ), +__print_symbolic (__entry->reason,FIND_FREE_EXTENT_SKIP_REASONS ), + __entry->error_or_value) +); + TRACE_EVENT(btrfs_find_cluster, TP_PROTO(const struct btrfs_block_group *block_group, u64 start,Signature · 1 line
2.47.3
PATCH v2 3/3 fstests: btrfs: test RAID conversions under stress
In reply to Leo MartinsAdd test to test btrfs conversion while being stressed. This is important since btrfs no longer allows allocating from different RAID block_groups during conversions meaning there may be added enospc pressure.
--- tests/btrfs/337 | 95 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 2 + 2 files changed, 97 insertions(+) create mode 100755 tests/btrfs/337 create mode 100644 tests/btrfs/337.out
tests/btrfs/337 new file +95 -0
@@ -0,0 +1,95 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. All Rights Reserved. +# +# FS QA Test btrfs/337 +# +# Test RAID profile conversion with concurrent allocations. +# This combines profile conversion (like btrfs/195) with concurrent +# fsstress allocations (like btrfs/060-064). + +. ./common/preamble +_begin_fstest auto volume balance scrub raid + +_cleanup() +{ + cd / + rm -f $tmp.* + _kill_fsstress +} + +. ./common/filter +# we check scratch dev after each loop +_require_scratch_nocheck +_require_scratch_dev_pool 4 +# Zoned btrfs only supports SINGLE profile +_require_non_zoned_device "${SCRATCH_DEV}" + +# Load up the available configs +_btrfs_get_profile_configs +declare -a TEST_VECTORS=( +# $nr_dev_min:$data:$metadata:$data_convert:$metadata_convert +"4:single:raid1" +"4:single:raid0" +"4:single:raid10" +"4:single:dup" +"4:single:raid5" +"4:single:raid6" +"2:raid1:single" +) + +run_testcase() { + IFS=':' read -ra args <<< $1 + num_disks=${args[0]} + src_type=${args[1]} + dst_type=${args[2]} + + if [[ ! "${_btrfs_profile_configs[@]}" =~ "$dst_type" ]]; then + echo "=== Skipping test: $1 ===" >> $seqres.full + return + fi + + _scratch_dev_pool_get $num_disks + + echo "=== Running test: $1 (converting $src_type -> $dst_type) ===" >> $seqres.full + + _scratch_pool_mkfs -d$src_type -m$src_type >> $seqres.full 2>&1 + _scratch_mount + + echo "Creating initial data..." >> $seqres.full + _run_fsstress -d $SCRATCH_MNT -w -n 10000 >> $seqres.full 2>&1 + + args=`_scale_fsstress_args -p 20 -n 1000 -d $SCRATCH_MNT/stressdir` + echo "Starting fsstress: $args" >> $seqres.full + _run_fsstress_bg $args + + echo "Starting conversion: $src_type -> $dst_type" >> $seqres.full + _run_btrfs_balance_start -f -dconvert=$dst_type $SCRATCH_MNT >> $seqres.full + [ $? -eq 0 ] || echo "$1: Failed convert" + + echo "Waiting for fsstress to complete..." >> $seqres.full + _wait_for_fsstress + + # Verify the conversion was successful + echo "Checking filesystem profile after conversion..." >> $seqres.full + $BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full + + # Scrub to verify data integrity + echo "Scrubbing filesystem..." >> $seqres.full + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "$1: Scrub found errors" + fi + + _scratch_unmount + _check_scratch_fs + _scratch_dev_pool_put +} + +echo "Silence is golden" +for i in "${TEST_VECTORS[@]}"; do + run_testcase $i +done + +status=0 +exit
tests/btrfs/337.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 337 +Silence is golden
Signature · 1 line
2.47.3
Re: [PATCH v2 3/3] fstests: btrfs: test RAID conversions under stress
Quoted 6 lines · Leo Martins
Add test to test btrfs conversion while being stressed. This is important since btrfs no longer allows allocating from different RAID
block_groupsduring conversions meaning there may be added enospc pressure.Signed-off-by Leo Martins <[email protected]>
Please do not mix patches for different projects into the same patchset.
A lot of us are using b4 to merge (kernel/progs) patches, which will merge the whole series, including the one intended to fstests.
Thanks,
Qu
Quoted 116 lines
--- tests/btrfs/337 | 95 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 2 + 2 files changed, 97 insertions(+)create mode 100755 tests/btrfs/337
create mode 100644 tests/btrfs/337.outtests/btrfs/337 new file +95
-0@@ -0,0 +1,95 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. All Rights Reserved. +# +# FS QA Test btrfs/337 +# +# Test RAID profile conversion with concurrent allocations. +# This combines profile conversion (like btrfs/195) with concurrent +# fsstress allocations (like btrfs/060-064). + +. ./common/preamble +_begin_fstest auto volume balance scrub raid + +_cleanup() +{ + cd / + rm -f $tmp.* + _kill_fsstress +} + +. ./common/filter +# we check scratch dev after each loop +_require_scratch_nocheck +_require_scratch_dev_pool 4 +# Zoned btrfs only supports SINGLE profile +_require_non_zoned_device "${SCRATCH_DEV}" + +# Load up the available configs +_btrfs_get_profile_configs +declare -a TEST_VECTORS=( +# $nr_dev_min:$data:$metadata:$data_convert:$metadata_convert +"4:single:raid1" +"4:single:raid0" +"4:single:raid10" +"4:single:dup" +"4:single:raid5" +"4:single:raid6" +"2:raid1:single" +) + +run_testcase() { + IFS=':' read -ra args <<< $1 + num_disks=${args[0]} + src_type=${args[1]} + dst_type=${args[2]} + + if [[ ! "${_btrfs_profile_configs[@]}" =~ "$dst_type" ]]; then + echo "=== Skipping test: $1 ===" >> $seqres.full + return + fi + + _scratch_dev_pool_get $num_disks + + echo "=== Running test: $1 (converting $src_type -> $dst_type) ===" >> $seqres.full + + _scratch_pool_mkfs -d$src_type -m$src_type >> $seqres.full 2>&1 + _scratch_mount + + echo "Creating initial data..." >> $seqres.full + _run_fsstress -d $SCRATCH_MNT -w -n 10000 >> $seqres.full 2>&1 + + args=`_scale_fsstress_args -p 20 -n 1000 -d $SCRATCH_MNT/stressdir` + echo "Starting fsstress: $args" >> $seqres.full + _run_fsstress_bg $args + + echo "Starting conversion: $src_type -> $dst_type" >> $seqres.full + _run_btrfs_balance_start -f -dconvert=$dst_type $SCRATCH_MNT >> $seqres.full + [ $? -eq 0 ] || echo "$1: Failed convert" + + echo "Waiting for fsstress to complete..." >> $seqres.full + _wait_for_fsstress + + # Verify the conversion was successful + echo "Checking filesystem profile after conversion..." >> $seqres.full + $BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full + + # Scrub to verify data integrity + echo "Scrubbing filesystem..." >> $seqres.full + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "$1: Scrub found errors" + fi + + _scratch_unmount + _check_scratch_fs + _scratch_dev_pool_put +} + +echo "Silence is golden" +for i in "${TEST_VECTORS[@]}"; do + run_testcase $i +done + +status=0 +exittests/btrfs/337.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 337 +Silence is golden
Re: [PATCH v2 3/3] fstests: btrfs: test RAID conversions under stress
Quoted 10 lines · On Sat, 4 Oct 2025 11:24:27 +0930 Qu Wenruo <[email protected]> wrote:
Quoted 6 lines · Leo Martins
Add test to test btrfs conversion while being stressed. This is important since btrfs no longer allows allocating from different RAID
block_groupsduring conversions meaning there may be added enospc pressure.Signed-off-by Leo Martins <[email protected]>
Quoted 7 lines
Please do not mix patches for different projects into the same patchset.
A lot of us are using b4 to merge (kernel/progs) patches, which will merge the whole series, including the one intended to fstests.
Thanks,
Qu
Got it, sorry about that. Should I resend the patches?
Re: [PATCH v2 3/3] fstests: btrfs: test RAID conversions under stress
Quoted 14 lines · Leo Martins
Quoted 12 lines · On Sat, 4 Oct 2025 11:24:27 +0930 Qu Wenruo <[email protected]> wrote:
Quoted 6 lines · Leo Martins
Add test to test btrfs conversion while being stressed. This is important since btrfs no longer allows allocating from different RAID
block_groupsduring conversions meaning there may be added enospc pressure.Signed-off-by Leo Martins <[email protected]>Please do not mix patches for different projects into the same patchset.
Quoted 7 lines
Quoted 5 lines
A lot of us are using b4 to merge (kernel/progs) patches, which will merge the whole series, including the one intended to fstests.
Thanks,
QuGot it, sorry about that. Should I resend the patches?
Please don't unless there are changes. We can always apply the patches one by one, but more patch versions in the mailinglist need more tracking and sometimes discussions are torn.
Re: [PATCH v2 3/3] fstests: btrfs: test RAID conversions under stress
In reply to Leo MartinsQuoted 5 lines · Leo Martins
Add test to test btrfs conversion while being stressed. This is important since btrfs no longer allows allocating from different RAID
block_groupsduring conversions meaning there may be added enospc pressure.
Aside from the patch intermingling stuff, this test looks good to me, thanks for adding it.
Quoted 120 lines
Signed-off-by Leo Martins <[email protected]>--- tests/btrfs/337 | 95 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/337.out | 2 + 2 files changed, 97 insertions(+) create mode 100755 tests/btrfs/337 create mode 100644 tests/btrfs/337.outtests/btrfs/337 new file +95
-0@@ -0,0 +1,95 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. All Rights Reserved. +# +# FS QA Test btrfs/337 +# +# Test RAID profile conversion with concurrent allocations. +# This combines profile conversion (like btrfs/195) with concurrent +# fsstress allocations (like btrfs/060-064). + +. ./common/preamble +_begin_fstest auto volume balance scrub raid + +_cleanup() +{ + cd / + rm -f $tmp.* + _kill_fsstress +} + +. ./common/filter +# we check scratch dev after each loop +_require_scratch_nocheck +_require_scratch_dev_pool 4 +# Zoned btrfs only supports SINGLE profile +_require_non_zoned_device "${SCRATCH_DEV}" + +# Load up the available configs +_btrfs_get_profile_configs +declare -a TEST_VECTORS=( +# $nr_dev_min:$data:$metadata:$data_convert:$metadata_convert +"4:single:raid1" +"4:single:raid0" +"4:single:raid10" +"4:single:dup" +"4:single:raid5" +"4:single:raid6" +"2:raid1:single" +) + +run_testcase() { + IFS=':' read -ra args <<< $1 + num_disks=${args[0]} + src_type=${args[1]} + dst_type=${args[2]} + + if [[ ! "${_btrfs_profile_configs[@]}" =~ "$dst_type" ]]; then + echo "=== Skipping test: $1 ===" >> $seqres.full + return + fi + + _scratch_dev_pool_get $num_disks + + echo "=== Running test: $1 (converting $src_type -> $dst_type) ===" >> $seqres.full + + _scratch_pool_mkfs -d$src_type -m$src_type >> $seqres.full 2>&1 + _scratch_mount + + echo "Creating initial data..." >> $seqres.full + _run_fsstress -d $SCRATCH_MNT -w -n 10000 >> $seqres.full 2>&1 + + args=`_scale_fsstress_args -p 20 -n 1000 -d $SCRATCH_MNT/stressdir` + echo "Starting fsstress: $args" >> $seqres.full + _run_fsstress_bg $args + + echo "Starting conversion: $src_type -> $dst_type" >> $seqres.full + _run_btrfs_balance_start -f -dconvert=$dst_type $SCRATCH_MNT >> $seqres.full + [ $? -eq 0 ] || echo "$1: Failed convert" + + echo "Waiting for fsstress to complete..." >> $seqres.full + _wait_for_fsstress + + # Verify the conversion was successful + echo "Checking filesystem profile after conversion..." >> $seqres.full + $BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full + + # Scrub to verify data integrity + echo "Scrubbing filesystem..." >> $seqres.full + $BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >> $seqres.full 2>&1 + if [ $? -ne 0 ]; then + echo "$1: Scrub found errors" + fi + + _scratch_unmount + _check_scratch_fs + _scratch_dev_pool_put +} + +echo "Silence is golden" +for i in "${TEST_VECTORS[@]}"; do + run_testcase $i +done + +status=0 +exittests/btrfs/337.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 337 +Silence is goldenSignature · 1 line
2.47.3