My current patch queue
Subsystems (2)
v4 cover My current patch queue
v3->v4:
- added stacktraces to all the changelogs
- added the various reviewed-by's.
- fixed the loop in inode_rsv_refill to not use goto again;
v2->v3:
- reworked the truncate/evict throttling, we were still occasionally hitting enospc aborts in production in these paths because we were too aggressive with space usage.
- reworked the delayed iput stuff to be a little less racey and less deadlocky.
- Addressed the comments from Dave and Omar.
- A lot of production testing.
v1->v2:
- addressed all of the issues brought up.
- added more comments.
- split up some patches.
original message:
This is the current queue of things that I've been working on. The main thing these patches are doing is separating out the delayed refs reservations from the global reserve into their own block rsv. We have been consistently hitting issues in production where we abort a transaction because we run out of the global reserve either while running delayed refs or while updating dirty block groups. This is because the math around global reserves is made up bullshit magic that has been tweaked more and more throughout the years. The result is something that is inconsistent across the board and sometimes wrong. So instead we need a way to know exactly how much space we need to keep around in order to satisfy our outstanding delayed refs and our dirty block groups.
Since we don't know how many delayed refs we need at the start of any modification we simply use the nr_items passed into btrfs_start_transaction() as a guess for what we may need. This has the side effect of putting more pressure on the ENOSPC system, but it's pressure we can deal with more intelligently because we always know how much space we have outstanding, instead of guessing with weird global reserve math.
This works similar to every other reservation we have, we reserve the worst case up front, and then at transaction end time we free up any space we didn't actually use for delayed refs.
My performance tests show that we are bit faster now since we can do more intelligent flushing and don't have to fall back on simply committing the transaction in hopes that we have enough space for everything we need to do.
That leads me to the 2nd part of this pull, there's a bunch of fixes around ENOSPC. Because we are a bit faster now there were a bunch of things uncovered in testing, but they seem to be all resolved now.
The final chunk of fixes are around transaction aborts. There were a lot of accounting bugs I was running into while running generic/435, so I fixed a bunch of those up so now it runs cleanly.
I have been running these patches through xfstests on multiple machines for a while, they are pretty solid and ready for wider testing and review. Thanks,
Josef
PATCH v4 1/42 btrfs: add btrfs_delete_ref_head helper
From: Josef Bacik <[email protected]>
We do this dance in cleanup_ref_head and check_ref_cleanup, unify it into a helper and cleanup the calling functions.
--- fs/btrfs/delayed-ref.c | 14 ++++++++++++++ fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/extent-tree.c | 22 +++------------------- 3 files changed, 19 insertions(+), 20 deletions(-)
fs/btrfs/delayed-ref.c +14 -0
@@ -393,6 +393,20 @@ btrfs_select_ref_head(struct btrfs_trans_handle *trans)return head; } +void btrfs_delete_ref_head (struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head) +{ +lockdep_assert_held (&delayed_refs->lock); +lockdep_assert_held (&head->lock); + +rb_erase (&head->href_node, &delayed_refs->href_root); +RB_CLEAR_NODE (&head->href_node); +atomic_dec (&delayed_refs->num_entries); + delayed_refs->num_heads--; +if (head->processing ==0 ) + delayed_refs->num_heads_ready--; +} +/* * Helper to insert the ref_node to the tail or merge with tail. *
fs/btrfs/delayed-ref.h +2 -1
@@ -261,7 +261,8 @@ static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) {mutex_unlock (&head->mutex); } - +void btrfs_delete_ref_head (struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head);struct btrfs_delayed_ref_head *btrfs_select_ref_head (struct btrfs_trans_handle *trans);
fs/btrfs/extent-tree.c +3 -19
@@ -2492,12 +2492,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans,spin_unlock (&delayed_refs->lock);return 1 ; } - delayed_refs->num_heads--; -rb_erase (&head->href_node, &delayed_refs->href_root); -RB_CLEAR_NODE (&head->href_node); +btrfs_delete_ref_head (delayed_refs, head);spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock); -atomic_dec (&delayed_refs->num_entries);trace_run_delayed_ref_head (fs_info, head,0 );
@@ -6984,22 +6981,9 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (!mutex_trylock (&head->mutex)) goto out; -/* -* at this point we have a head with no other entries. Go -* ahead and process it. -*/ -rb_erase (&head->href_node, &delayed_refs->href_root); -RB_CLEAR_NODE (&head->href_node); -atomic_dec (&delayed_refs->num_entries); - -/* -* we don't take a ref on the node because we're removing it from the -* tree, so we just steal the ref the tree was holding. -*/ - delayed_refs->num_heads--; -if (head->processing ==0 ) - delayed_refs->num_heads_ready--; +btrfs_delete_ref_head (delayed_refs, head); head->processing =0 ; +spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock);
Signature · 1 line
2.14.3
PATCH v4 2/42 btrfs: add cleanup_ref_head_accounting helper
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We were missing some quota cleanups in check_ref_cleanup, so break the ref head accounting cleanup into a helper and call that from both check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that we don't screw up accounting in the future for other things that we add.
--- fs/btrfs/extent-tree.c | 67 +++++++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 28 deletions(-)
fs/btrfs/extent-tree.c +39 -28
@@ -2461,6 +2461,41 @@ static int cleanup_extent_op(struct btrfs_trans_handle *trans,return ret ? ret :1 ; } +static void cleanup_ref_head_accounting (struct btrfs_trans_handle *trans, +struct btrfs_delayed_ref_head *head) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + +if (head->total_ref_mod <0 ) { +struct btrfs_space_info *space_info; +u64 flags; + +if (head->is_data) + flags =BTRFS_BLOCK_GROUP_DATA ; +else if (head->is_system) + flags =BTRFS_BLOCK_GROUP_SYSTEM ; +else + flags =BTRFS_BLOCK_GROUP_METADATA ; + space_info =__find_space_info (fs_info, flags); +ASSERT (space_info); +percpu_counter_add_batch (&space_info->total_bytes_pinned, + -head->num_bytes, +BTRFS_TOTAL_BYTES_PINNED_BATCH ); + +if (head->is_data) { +spin_lock (&delayed_refs->lock); + delayed_refs->pending_csums -= head->num_bytes; +spin_unlock (&delayed_refs->lock); + } + } + +/* Also free its reserved qgroup space */ +btrfs_qgroup_free_delayed_ref (fs_info, head->qgroup_ref_root, + head->qgroup_reserved); +} +static int cleanup_ref_head (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *head) {
@@ -2496,31 +2531,6 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans,spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock); -trace_run_delayed_ref_head (fs_info, head,0 ); - -if (head->total_ref_mod <0 ) { -struct btrfs_space_info *space_info; -u64 flags; - -if (head->is_data) - flags =BTRFS_BLOCK_GROUP_DATA ; -else if (head->is_system) - flags =BTRFS_BLOCK_GROUP_SYSTEM ; -else - flags =BTRFS_BLOCK_GROUP_METADATA ; - space_info =__find_space_info (fs_info, flags); -ASSERT (space_info); -percpu_counter_add_batch (&space_info->total_bytes_pinned, - -head->num_bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ); - -if (head->is_data) { -spin_lock (&delayed_refs->lock); - delayed_refs->pending_csums -= head->num_bytes; -spin_unlock (&delayed_refs->lock); - } - } -if (head->must_insert_reserved) {btrfs_pin_extent (fs_info, head->bytenr, head->num_bytes,1 );
@@ -2530,9 +2540,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } -/* Also free its reserved qgroup space */ -btrfs_qgroup_free_delayed_ref (fs_info, head->qgroup_ref_root, - head->qgroup_reserved); +cleanup_ref_head_accounting (trans, head); + +trace_run_delayed_ref_head (fs_info, head,0 );btrfs_delayed_ref_unlock (head);btrfs_put_delayed_ref_head (head);return 0 ;
@@ -6991,6 +7001,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (head->must_insert_reserved) ret =1 ; +cleanup_ref_head_accounting (trans, head);mutex_unlock (&head->mutex);btrfs_put_delayed_ref_head (head);return ret;
Signature · 1 line
2.14.3
PATCH v4 3/42 btrfs: cleanup extent_op handling
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
The cleanup_extent_op function actually would run the extent_op if it needed running, which made the name sort of a misnomer. Change it to run_and_cleanup_extent_op, and move the actual cleanup work to cleanup_extent_op so it can be used by check_ref_cleanup() in order to unify the extent op handling.
--- fs/btrfs/extent-tree.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-)
fs/btrfs/extent-tree.c +23 -13
@@ -2442,19 +2442,33 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_refbtrfs_delayed_ref_unlock (head); } -static int cleanup_extent_op (struct btrfs_trans_handle *trans, -struct btrfs_delayed_ref_head *head) +static struct btrfs_delayed_extent_op * +cleanup_extent_op (struct btrfs_trans_handle *trans, +struct btrfs_delayed_ref_head *head) {struct btrfs_delayed_extent_op *extent_op = head->extent_op; -int ret;if (!extent_op) -return 0 ; - head->extent_op =NULL ; +return NULL ; +if (head->must_insert_reserved) { + head->extent_op =NULL ;btrfs_free_delayed_extent_op (extent_op); -return 0 ; +return NULL ; } +return extent_op; +} + +static int run_and_cleanup_extent_op (struct btrfs_trans_handle *trans, +struct btrfs_delayed_ref_head *head) +{ +struct btrfs_delayed_extent_op *extent_op = +cleanup_extent_op (trans, head); +int ret; + +if (!extent_op) +return 0 ; + head->extent_op =NULL ;spin_unlock (&head->lock); ret =run_delayed_extent_op (trans, head, extent_op);btrfs_free_delayed_extent_op (extent_op);
@@ -2506,7 +2520,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, delayed_refs = &trans->transaction->delayed_refs; - ret =cleanup_extent_op (trans, head); + ret =run_and_cleanup_extent_op (trans, head);if (ret <0 ) {unselect_delayed_ref_head (delayed_refs, head);btrfs_debug (fs_info,"run_delayed_extent_op returned %d" , ret);
@@ -6977,12 +6991,8 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (!RB_EMPTY_ROOT (&head->ref_tree)) goto out; -if (head->extent_op) { -if (!head->must_insert_reserved) - goto out; -btrfs_free_delayed_extent_op (head->extent_op); - head->extent_op =NULL ; - } +if (cleanup_extent_op (trans, head) !=NULL ) + goto out;/* * waiting for the lock here would deadlock. If someone else has it
Signature · 1 line
2.14.3
PATCH v4 4/42 btrfs: only track ref_heads in delayed_ref_updates
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We use this number to figure out how many delayed refs to run, but __btrfs_run_delayed_refs really only checks every time we need a new delayed ref head, so we always run at least one ref head completely no matter what the number of items on it. Fix the accounting to only be adjusted when we add/remove a ref head.
--- fs/btrfs/delayed-ref.c | 3 --- 1 file changed, 3 deletions(-)
fs/btrfs/delayed-ref.c +0 -3
@@ -234,8 +234,6 @@ static inline void drop_delayed_ref(struct btrfs_trans_handle *trans, ref->in_tree =0 ;btrfs_put_delayed_ref (ref);atomic_dec (&delayed_refs->num_entries); -if (trans->delayed_ref_updates) - trans->delayed_ref_updates--; } static bool merge_ref(struct btrfs_trans_handle *trans,
@@ -460,7 +458,6 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,if (ref->action ==BTRFS_ADD_DELAYED_REF )list_add_tail (&ref->add_list, &href->ref_add_list);atomic_inc (&root->num_entries); - trans->delayed_ref_updates++;spin_unlock (&href->lock);return ret; }
Signature · 1 line
2.14.3
PATCH v4 5/42 btrfs: only count ref heads run in __btrfs_run_delayed_refs
In reply to Josef BacikWe pick the number of ref's to run based on the number of ref heads, and only make the decision to stop once we've processed entire ref heads, so only count the ref heads we've run and bail once we've hit the number of ref heads we wanted to process.
--- fs/btrfs/extent-tree.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
fs/btrfs/extent-tree.c +1 -4
@@ -2592,6 +2592,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,spin_unlock (&delayed_refs->lock);break ; } + count++;/* grab the lock that says we are going to process * all the refs for this head */
@@ -2605,7 +2606,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,*/ if (ret == -EAGAIN ) { locked_ref =NULL ; - count++;continue ; } }
@@ -2633,7 +2633,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,unselect_delayed_ref_head (delayed_refs, locked_ref); locked_ref =NULL ;cond_resched (); - count++;continue ; }
@@ -2651,7 +2650,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,return ret; } locked_ref =NULL ; - count++;continue ; }
@@ -2702,7 +2700,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, }btrfs_put_delayed_ref (ref); - count++;cond_resched (); }
Signature · 1 line
2.14.3
PATCH v4 6/42 btrfs: introduce delayed_refs_rsv
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong.
So instead give them their own block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism.
For now the delayed_refs_rsv will hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via the delayed_refs so that will stay there.
--- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-)
fs/btrfs/ctree.h +19 -8
@@ -452,8 +452,9 @@ struct btrfs_space_info {#define BTRFS_BLOCK_RSV_TRANS 3#define BTRFS_BLOCK_RSV_CHUNK 4#define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8struct btrfs_block_rsv {u64 size;
@@ -794,6 +795,8 @@ struct btrfs_fs_info {struct btrfs_block_rsv chunk_block_rsv;/* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; +/* block reservation for delayed refs */ +struct btrfs_block_rsv delayed_refs_rsv;struct btrfs_block_rsv empty_block_rsv;
@@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info,int btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans,struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);void btrfs_dec_block_group_reservations (struct btrfs_fs_info *fs_info,const u64 start);void btrfs_wait_block_group_reservations (struct btrfs_block_group_cache *bg);
@@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum {enum btrfs_flush_state {FLUSH_DELAYED_ITEMS_NR =1 ,FLUSH_DELAYED_ITEMS =2 , -FLUSH_DELALLOC =3 , -FLUSH_DELALLOC_WAIT =4 , -ALLOC_CHUNK =5 , -COMMIT_TRANS =6 , +FLUSH_DELAYED_REFS_NR =3 , +FLUSH_DELAYED_REFS =4 , +FLUSH_DELALLOC =5 , +FLUSH_DELALLOC_WAIT =6 , +ALLOC_CHUNK =7 , +COMMIT_TRANS =8 , };int btrfs_alloc_data_chunk_ondemand (struct btrfs_inode *inode,u64 bytes);
@@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,void btrfs_block_rsv_release (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *block_rsv,u64 num_bytes); +void btrfs_delayed_refs_rsv_release (struct btrfs_fs_info *fs_info,int nr); +void btrfs_update_delayed_refs_rsv (struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *src, +u64 num_bytes);int btrfs_inc_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_dec_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_put_block_group_cache (struct btrfs_fs_info *info);
fs/btrfs/delayed-ref.c +23 -5
@@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,* existing and update must have the same bytenr */ static noinline void -update_existing_head_ref (struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *existing,struct btrfs_delayed_ref_head *update,int *old_ref_mod_ret) { +struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; +struct btrfs_fs_info *fs_info = trans->fs_info;int old_ref_mod;BUG_ON (existing->is_data != update->is_data);
@@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs,* versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { -if (existing->total_ref_mod >=0 && old_ref_mod <0 ) +u64 csum_items = +btrfs_csum_bytes_to_leaves (fs_info, + existing->num_bytes); + +if (existing->total_ref_mod >=0 && old_ref_mod <0 ) { delayed_refs->pending_csums -= existing->num_bytes; -if (existing->total_ref_mod <0 && old_ref_mod >=0 ) +btrfs_delayed_refs_rsv_release (fs_info, csum_items); + } +if (existing->total_ref_mod <0 && old_ref_mod >=0 ) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } }spin_unlock (&existing->lock); }
@@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); -update_existing_head_ref (delayed_refs, existing, head_ref, +update_existing_head_ref (trans, existing, head_ref, old_ref_mod);/* * we've updated the existing ref, free the newly
@@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, }else {if (old_ref_mod) *old_ref_mod =0 ; -if (head_ref->is_data && head_ref->ref_mod <0 ) +if (head_ref->is_data && head_ref->ref_mod <0 ) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += +btrfs_csum_bytes_to_leaves (trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++;atomic_inc (&delayed_refs->num_entries);
@@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans); trace_add_delayed_tree_ref(fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ?
@@ -866,6 +882,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans); trace_add_delayed_data_ref(trans->fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ?
@@ -903,6 +920,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, NULL, NULL, NULL);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans);return 0 ; }
fs/btrfs/disk-io.c +4 -0
@@ -2692,6 +2692,9 @@ int open_ctree(struct super_block *sb,btrfs_init_block_rsv (&fs_info->empty_block_rsv,BTRFS_BLOCK_RSV_EMPTY );btrfs_init_block_rsv (&fs_info->delayed_block_rsv,BTRFS_BLOCK_RSV_DELOPS ); +btrfs_init_block_rsv (&fs_info->delayed_refs_rsv, +BTRFS_BLOCK_RSV_DELREFS ); +atomic_set (&fs_info->async_delalloc_pages,0 );atomic_set (&fs_info->defrag_running,0 );atomic_set (&fs_info->qgroup_op_seq,0 );
@@ -4419,6 +4422,7 @@ void btrfs_cleanup_dirty_bgs(struct btrfs_transaction *cur_trans,spin_unlock (&cur_trans->dirty_bgs_lock);btrfs_put_block_group (cache); +btrfs_delayed_refs_rsv_release (fs_info,1 );spin_lock (&cur_trans->dirty_bgs_lock); }spin_unlock (&cur_trans->dirty_bgs_lock);
fs/btrfs/extent-tree.c +228 -51
467 lines not on this page · raw message
fs/btrfs/inode.c +1 -1
@@ -5382,7 +5382,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,* Try to steal from the global reserve if there is space for * it. */ -if (!btrfs_check_space_for_delayed_refs (trans, fs_info) && +if (!btrfs_check_space_for_delayed_refs (fs_info) && !btrfs_block_rsv_migrate (global_rsv, rsv, min_size,0 ))return trans;
fs/btrfs/transaction.c +35 -42
140 lines not on this page · raw message
include/trace/events/btrfs.h +2 -0
@@ -1048,6 +1048,8 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_ITEMS, "FLUSH_DELAYED_ITEMS"}, \ { FLUSH_DELALLOC, "FLUSH_DELALLOC"}, \ { FLUSH_DELALLOC_WAIT, "FLUSH_DELALLOC_WAIT"}, \ + { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ + { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ { COMMIT_TRANS, "COMMIT_TRANS"})
Signature · 1 line
2.14.3
PATCH v4 7/42 btrfs: check if free bgs for commit
In reply to Josef Bacikmay_commit_transaction will skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space.
--- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-)
fs/btrfs/extent-tree.c +19 -14
@@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ; -/* See if there is enough pinned space to make this reservation */ -if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) + trans =btrfs_join_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +return -ENOSPC ; + +/* +* See if there is enough pinned space to make this reservation, or if +* we have bg's that are going to be freed, allowing us to possibly do a +* chunk allocation the next loop through. +*/ +if (test_bit (BTRFS_TRANS_HAVE_FREE_BGS , &trans->transaction->flags) || +__percpu_counter_compare (&space_info->total_bytes_pinned, bytes, +BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) goto commit;/*
@@ -4841,7 +4849,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,* this reservation. */ if (space_info != delayed_rsv->space_info) -return -ENOSPC ; + goto enospc;spin_lock (&delayed_rsv->lock); reclaim_bytes += delayed_rsv->reserved;
@@ -4855,17 +4863,14 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, bytes -= reclaim_bytes;if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) <0 ) { -return -ENOSPC ; - } - + bytes, +BTRFS_TOTAL_BYTES_PINNED_BATCH ) <0 ) + goto enospc; commit: - trans =btrfs_join_transaction (fs_info->extent_root); -if (IS_ERR (trans)) -return -ENOSPC ; -return btrfs_commit_transaction (trans); +enospc: +btrfs_end_transaction (trans); +return -ENOSPC ; }/*
Signature · 1 line
2.14.3
PATCH v4 8/42 btrfs: dump block_rsv whe dumping space info
In reply to Josef BacikFor enospc_debug having the block rsvs is super helpful to see if we've done something wrong.
--- fs/btrfs/extent-tree.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
fs/btrfs/extent-tree.c +15 -0
@@ -7918,6 +7918,15 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,return ret; } +#define DUMP_BLOCK_RSV (fs_info, rsv_name) \ +do { \ + struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \ + spin_lock(&__rsv->lock); \ + btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \ + __rsv->size, __rsv->reserved); \ + spin_unlock(&__rsv->lock); \ +} while (0) +static void dump_space_info (struct btrfs_fs_info *fs_info,struct btrfs_space_info *info,u64 bytes,int dump_block_groups)
@@ -7937,6 +7946,12 @@ static void dump_space_info(struct btrfs_fs_info *fs_info, info->bytes_readonly);spin_unlock (&info->lock); +DUMP_BLOCK_RSV (fs_info, global_block_rsv); +DUMP_BLOCK_RSV (fs_info, trans_block_rsv); +DUMP_BLOCK_RSV (fs_info, chunk_block_rsv); +DUMP_BLOCK_RSV (fs_info, delayed_block_rsv); +DUMP_BLOCK_RSV (fs_info, delayed_refs_rsv); +if (!dump_block_groups)return ;
Signature · 1 line
2.14.3
PATCH v4 9/42 btrfs: release metadata before running delayed refs
In reply to Josef BacikWe want to release the unused reservation we have since it refills the delayed refs reserve, which will make everything go smoother when running the delayed refs if we're short on our reservation.
--- fs/btrfs/transaction.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/btrfs/transaction.c +3 -3
@@ -1922,6 +1922,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)return ret; } +btrfs_trans_release_metadata (trans); + trans->block_rsv =NULL ; +/* make a pass through all the delayed refs we have so far * any runnings procs may add more while we are here */
@@ -1931,9 +1934,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)return ret; } -btrfs_trans_release_metadata (trans); - trans->block_rsv =NULL ; - cur_trans = trans->transaction;/*
Signature · 1 line
2.14.3
PATCH v4 10/42 btrfs: protect space cache inode alloc with nofs
In reply to Josef BacikIf we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail.
--- fs/btrfs/free-space-cache.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/btrfs/free-space-cache.c +8 -0
@@ -10,6 +10,7 @@#include <linux/math64.h> #include <linux/ratelimit.h> #include <linux/error-injection.h> +#include <linux/sched/mm.h> #include "ctree.h" #include "free-space-cache.h" #include "transaction.h"
@@ -47,6 +48,7 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root,struct btrfs_free_space_header *header;struct extent_buffer *leaf;struct inode *inode =NULL ; +unsigned nofs_flag;int ret; key.objectid =BTRFS_FREE_SPACE_OBJECTID ;
@@ -68,7 +70,13 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root,btrfs_disk_key_to_cpu (&location, &disk_key);btrfs_release_path (path); +/* +* We are often under a trans handle at this point, so we need to make +* sure NOFS is set to keep us from deadlocking. +*/ + nofs_flag =memalloc_nofs_save (); inode =btrfs_iget (fs_info->sb, &location, root,NULL ); +memalloc_nofs_restore (nofs_flag);if (IS_ERR (inode))return inode;
Signature · 1 line
2.14.3
PATCH v4 11/42 btrfs: fix truncate throttling
In reply to Josef BacikWe have a bunch of magic to make sure we're throttling delayed refs when truncating a file. Now that we have a delayed refs rsv and a mechanism for refilling that reserve simply use that instead of all of this magic.
--- fs/btrfs/inode.c | 79 ++++++++++++-------------------------------------------- 1 file changed, 17 insertions(+), 62 deletions(-)
fs/btrfs/inode.c +17 -62
@@ -4493,31 +4493,6 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry)return err; } -static int truncate_space_check (struct btrfs_trans_handle *trans, -struct btrfs_root *root, -u64 bytes_deleted) -{ -struct btrfs_fs_info *fs_info = root->fs_info; -int ret; - -/* -* This is only used to apply pressure to the enospc system, we don't -* intend to use this reservation at all. -*/ - bytes_deleted =btrfs_csum_bytes_to_leaves (fs_info, bytes_deleted); - bytes_deleted *= fs_info->nodesize; - ret =btrfs_block_rsv_add (root, &fs_info->trans_block_rsv, - bytes_deleted,BTRFS_RESERVE_NO_FLUSH ); -if (!ret) { -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, - bytes_deleted,1 ); - trans->bytes_reserved += bytes_deleted; - } -return ret; - -} -/* * Return this if we need to call truncate_block for the last bit of the * truncate.
@@ -4562,7 +4537,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,u64 bytes_deleted =0 ;bool be_nice = false;bool should_throttle = false; -bool should_end = false;BUG_ON (new_size >0 && min_type !=BTRFS_EXTENT_DATA_KEY );
@@ -4775,15 +4749,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,btrfs_abort_transaction (trans, ret);break ; } -if (btrfs_should_throttle_delayed_refs (trans, fs_info)) -btrfs_async_run_delayed_refs (fs_info, - trans->delayed_ref_updates *2 , - trans->transid,0 );if (be_nice) { -if (truncate_space_check (trans, root, - extent_num_bytes)) { - should_end = true; - }if (btrfs_should_throttle_delayed_refs (trans, fs_info)) should_throttle = true;
@@ -4795,7 +4761,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,if (path->slots[0 ] ==0 || path->slots[0 ] != pending_del_slot || - should_throttle || should_end) { + should_throttle) {if (pending_del_nr) { ret = btrfs_del_items(trans, root, path, pending_del_slot,
@@ -4807,23 +4773,24 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, pending_del_nr =0 ; }btrfs_release_path (path); -if (should_throttle) { -unsigned long updates = trans->delayed_ref_updates; -if (updates) { - trans->delayed_ref_updates =0 ; - ret =btrfs_run_delayed_refs (trans, - updates *2 ); -if (ret) -break ; - } - } +/* -* if we failed to refill our space rsv, bail out -* and let the transaction restart +* We can generate a lot of delayed refs, so we need to +* throttle every once and a while and make sure we're +* adding enough space to keep up with the work we are +* generating. Since we hold a transaction here we +* can't flush, and we don't want to FLUSH_LIMIT because +* we could have generated too many delayed refs to +* actually allocate, so just bail if we're short and +* let the normal reservation dance happen higher up. */ -if (should_end) { - ret = -EAGAIN ; -break ; +if (should_throttle) { + ret =btrfs_throttle_delayed_refs (fs_info, +BTRFS_RESERVE_NO_FLUSH ); +if (ret) { + ret = -EAGAIN ; +break ; + } } goto search_again; }else {
@@ -4849,18 +4816,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, }btrfs_free_path (path); - -if (be_nice && bytes_deleted >SZ_32M && (ret >=0 || ret == -EAGAIN )) { -unsigned long updates = trans->delayed_ref_updates; -int err; - -if (updates) { - trans->delayed_ref_updates =0 ; - err =btrfs_run_delayed_refs (trans, updates *2 ); -if (err) - ret = err; - } - }return ret; }
Signature · 1 line
2.14.3
PATCH v4 12/42 btrfs: don't use global rsv for chunk allocation
In reply to Josef BacikWe've done this forever because of the voodoo around knowing how much space we have. However we have better ways of doing this now, and on normal file systems we'll easily have a global reserve of 512MiB, and since metadata chunks are usually 1GiB that means we'll allocate metadata chunks more readily. Instead use the actual used amount when determining if we need to allocate a chunk or not.
--- fs/btrfs/extent-tree.c | 9 --------- 1 file changed, 9 deletions(-)
fs/btrfs/extent-tree.c +0 -9
@@ -4374,21 +4374,12 @@ static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global)static int should_alloc_chunk (struct btrfs_fs_info *fs_info,struct btrfs_space_info *sinfo,int force) { -struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;u64 bytes_used =btrfs_space_info_used (sinfo, false);u64 thresh;if (force ==CHUNK_ALLOC_FORCE )return 1 ; -/* -* We need to take into account the global rsv because for all intents -* and purposes it's used space. Don't worry about locking the -* global_rsv, it doesn't change except when the transaction commits. -*/ -if (sinfo->flags &BTRFS_BLOCK_GROUP_METADATA ) - bytes_used +=calc_global_rsv_need_space (global_rsv); -/* * in limited mode, we want to have some free space up to * about 1% of the FS size.
Signature · 1 line
2.14.3
PATCH v4 13/42 btrfs: add ALLOC_CHUNK_FORCE to the flushing code
In reply to Josef BacikWith my change to no longer take into account the global reserve for metadata allocation chunks we have this side-effect for mixed block group fs'es where we are no longer allocating enough chunks for the data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE step to the flushing state machine. This will only get used if we've already made a full loop through the flushing machinery and tried committing the transaction. If we have then we can try and force a chunk allocation since we likely need it to make progress. This resolves the issues I was seeing with the mixed bg tests in xfstests with my previous patch.
--- fs/btrfs/ctree.h | 3 ++- fs/btrfs/extent-tree.c | 18 +++++++++++++++++- include/trace/events/btrfs.h | 1 + 3 files changed, 20 insertions(+), 2 deletions(-)
fs/btrfs/ctree.h +2 -1
@@ -2730,7 +2730,8 @@ enum btrfs_flush_state {FLUSH_DELALLOC =5 ,FLUSH_DELALLOC_WAIT =6 ,ALLOC_CHUNK =7 , - COMMIT_TRANS = 8, +ALLOC_CHUNK_FORCE =8 , + COMMIT_TRANS = 9, };int btrfs_alloc_data_chunk_ondemand (struct btrfs_inode *inode,u64 bytes);
fs/btrfs/extent-tree.c +17 -1
@@ -4914,6 +4914,7 @@ static void flush_space(struct btrfs_fs_info *fs_info,btrfs_end_transaction (trans);break ;case ALLOC_CHUNK : +case ALLOC_CHUNK_FORCE : trans =btrfs_join_transaction (root);if (IS_ERR (trans)) { ret =PTR_ERR (trans);
@@ -4921,7 +4922,9 @@ static void flush_space(struct btrfs_fs_info *fs_info, } ret =do_chunk_alloc (trans,btrfs_metadata_alloc_profile (fs_info), -CHUNK_ALLOC_NO_FORCE ); + (state ==ALLOC_CHUNK ) ? +CHUNK_ALLOC_NO_FORCE : +CHUNK_ALLOC_FORCE );btrfs_end_transaction (trans);if (ret >0 || ret == -ENOSPC ) ret =0 ;
@@ -5057,6 +5060,19 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) commit_cycles--; } +/* +* We don't want to force a chunk allocation until we've tried +* pretty hard to reclaim space. Think of the case where we +* free'd up a bunch of space and so have a lot of pinned space +* to reclaim. We would rather use that than possibly create a +* underutilized metadata chunk. So if this is our first run +* through the flushing state machine skip ALLOC_CHUNK_FORCE and +* commit the transaction. If nothing has changed the next go +* around then we can force a chunk allocation. +*/ +if (flush_state ==ALLOC_CHUNK_FORCE && !commit_cycles) + flush_state++; +if (flush_state >COMMIT_TRANS ) { commit_cycles++;if (commit_cycles >2 ) {
include/trace/events/btrfs.h +1 -0
@@ -1051,6 +1051,7 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ + { ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) TRACE_EVENT(btrfs_flush_space,
Signature · 1 line
2.14.3
PATCH v4 14/42 btrfs: reset max_extent_size properly
In reply to Josef BacikIf we use up our block group before allocating a new one we'll easily get a max_extent_size that's set really really low, which will result in a lot of fragmentation. We need to make sure we're resetting the max_extent_size when we add a new chunk or add new space.
--- fs/btrfs/extent-tree.c | 2 ++ 1 file changed, 2 insertions(+)
fs/btrfs/extent-tree.c +2 -0
@@ -4573,6 +4573,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags, goto out; }else { ret =1 ; + space_info->max_extent_size =0 ; } space_info->force_alloc =CHUNK_ALLOC_NO_FORCE ;
@@ -6671,6 +6672,7 @@ static int btrfs_free_reserved_bytes(struct btrfs_block_group_cache *cache, space_info->bytes_readonly += num_bytes; cache->reserved -= num_bytes; space_info->bytes_reserved -= num_bytes; + space_info->max_extent_size =0 ;if (delalloc) cache->delalloc_bytes -= num_bytes;
Signature · 1 line
2.14.3
Re: [PATCH 14/42] btrfs: reset max_extent_size properly
Quoted 7 lines · Josef Bacik
If we use up our block group before allocating a new one we'll easily get a
max_extent_sizethat's set really really low, which will result in a lot of fragmentation. We need to make sure we're resetting themax_extent_sizewhen we add a new chunk or add new space.Signed-off-by Josef Bacik <[email protected]>
Quoted 27 lines
--- fs/btrfs/extent-tree.c | 2 ++ 1 file changed, 2 insertions(+)fs/btrfs/extent-tree.c +2
-1@@ -4573,6 +4573,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags, goto out; }else { ret =1 ; + space_info->max_extent_size =0 ; } space_info->force_alloc =CHUNK_ALLOC_NO_FORCE ;@@ -6671,6 +6672,7 @@ static int btrfs_free_reserved_bytes(struct btrfs_block_group_cache *cache, space_info->bytes_readonly += num_bytes; cache->reserved -= num_bytes; space_info->bytes_reserved -= num_bytes; + space_info->max_extent_size =0 ;if (delalloc) cache->delalloc_bytes -= num_bytes; --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
PATCH v4 15/42 btrfs: don't enospc all tickets on flush failure
In reply to Josef BacikWith the introduction of the per-inode block_rsv it became possible to have really really large reservation requests made because of data fragmentation. Since the ticket stuff assumed that we'd always have relatively small reservation requests it just killed all tickets if we were unable to satisfy the current request. However this is generally not the case anymore. So fix this logic to instead see if we had a ticket that we were able to give some reservation to, and if we were continue the flushing loop again. Likewise we make the tickets use the space_info_add_old_bytes() method of returning what reservation they did receive in hopes that it could satisfy reservations down the line.
--- fs/btrfs/extent-tree.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
fs/btrfs/extent-tree.c +25 -20
@@ -4779,6 +4779,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, }struct reserve_ticket { +u64 orig_bytes;u64 bytes;int error;struct list_head list;
@@ -5000,7 +5001,7 @@ static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info, !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)); } -static void wake_all_tickets (struct list_head *head) +static bool wake_all_tickets (struct list_head *head) {struct reserve_ticket *ticket;
@@ -5009,7 +5010,10 @@ static void wake_all_tickets(struct list_head *head)list_del_init (&ticket->list); ticket->error = -ENOSPC ;wake_up (&ticket->wait); +if (ticket->bytes != ticket->orig_bytes) +return true; } +return false; }/*
@@ -5077,8 +5081,12 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work)if (flush_state >COMMIT_TRANS ) { commit_cycles++;if (commit_cycles >2 ) { -wake_all_tickets (&space_info->tickets); - space_info->flush =0 ; +if (wake_all_tickets (&space_info->tickets)) { + flush_state =FLUSH_DELAYED_ITEMS_NR ; + commit_cycles--; + }else { + space_info->flush =0 ; + } }else { flush_state =FLUSH_DELAYED_ITEMS_NR ; }
@@ -5130,10 +5138,11 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,static int wait_reserve_ticket (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info, -struct reserve_ticket *ticket,u64 orig_bytes) +struct reserve_ticket *ticket) {DEFINE_WAIT (wait); +u64 reclaim_bytes =0 ;int ret =0 ;spin_lock (&space_info->lock);
@@ -5154,14 +5163,12 @@ static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, ret = ticket->error;if (!list_empty (&ticket->list))list_del_init (&ticket->list); -if (ticket->bytes && ticket->bytes < orig_bytes) { -u64 num_bytes = orig_bytes - ticket->bytes; - space_info->bytes_may_use -= num_bytes; -trace_btrfs_space_reservation (fs_info,"space_info" , - space_info->flags, num_bytes,0 ); - } +if (ticket->bytes && ticket->bytes < ticket->orig_bytes) + reclaim_bytes = ticket->orig_bytes - ticket->bytes;spin_unlock (&space_info->lock); +if (reclaim_bytes) +space_info_add_old_bytes (fs_info, space_info, reclaim_bytes);return ret; }
@@ -5187,6 +5194,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, {struct reserve_ticket ticket;u64 used; +u64 reclaim_bytes =0 ;int ret =0 ;ASSERT (orig_bytes);
@@ -5222,6 +5230,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info,* the list and we will do our own flushing further down. */ if (ret && flush !=BTRFS_RESERVE_NO_FLUSH ) { + ticket.orig_bytes = orig_bytes; ticket.bytes = orig_bytes; ticket.error =0 ;init_waitqueue_head (&ticket.wait);
@@ -5262,25 +5271,21 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info,return ret;if (flush ==BTRFS_RESERVE_FLUSH_ALL ) -return wait_reserve_ticket (fs_info, space_info, &ticket, - orig_bytes); +return wait_reserve_ticket (fs_info, space_info, &ticket); ret =0 ;priority_reclaim_metadata_space (fs_info, space_info, &ticket);spin_lock (&space_info->lock);if (ticket.bytes) { -if (ticket.bytes < orig_bytes) { -u64 num_bytes = orig_bytes - ticket.bytes; - space_info->bytes_may_use -= num_bytes; -trace_btrfs_space_reservation (fs_info,"space_info" , - space_info->flags, - num_bytes,0 ); - - } +if (ticket.bytes < orig_bytes) + reclaim_bytes = orig_bytes - ticket.bytes;list_del_init (&ticket.list); ret = -ENOSPC ; }spin_unlock (&space_info->lock); + +if (reclaim_bytes) +space_info_add_old_bytes (fs_info, space_info, reclaim_bytes);ASSERT (list_empty (&ticket.list));return ret; }
Signature · 1 line
2.14.3
PATCH v4 16/42 btrfs: loop in inode_rsv_refill
In reply to Josef BacikWith severe fragmentation we can end up with our inode rsv size being huge during writeout, which would cause us to need to make very large metadata reservations. However we may not actually need that much once writeout is complete. So instead try to make our reservation, and if we couldn't make it re-calculate our new reservation size and try again. If our reservation size doesn't change between tries then we know we are actually out of space and can error out.
--- fs/btrfs/extent-tree.c | 56 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 15 deletions(-)
fs/btrfs/extent-tree.c +41 -15
@@ -5766,6 +5766,21 @@ int btrfs_block_rsv_refill(struct btrfs_root *root,return ret; } +static inline void __get_refill_bytes (struct btrfs_block_rsv *block_rsv, +u64 *metadata_bytes,u64 *qgroup_bytes) +{ + *metadata_bytes =0 ; + *qgroup_bytes =0 ; + +spin_lock (&block_rsv->lock); +if (block_rsv->reserved < block_rsv->size) + *metadata_bytes = block_rsv->size - block_rsv->reserved; +if (block_rsv->qgroup_rsv_reserved < block_rsv->qgroup_rsv_size) + *qgroup_bytes = block_rsv->qgroup_rsv_size - + block_rsv->qgroup_rsv_reserved; +spin_unlock (&block_rsv->lock); +} +/** * btrfs_inode_rsv_refill - refill the inode block rsv. * @inode - the inode we are refilling.
@@ -5781,25 +5796,37 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, {struct btrfs_root *root = inode->root;struct btrfs_block_rsv *block_rsv = &inode->block_rsv; -u64 num_bytes =0 ; +u64 num_bytes =0 , last =0 ;u64 qgroup_num_bytes =0 ;int ret = -ENOSPC ; -spin_lock (&block_rsv->lock); -if (block_rsv->reserved < block_rsv->size) - num_bytes = block_rsv->size - block_rsv->reserved; -if (block_rsv->qgroup_rsv_reserved < block_rsv->qgroup_rsv_size) - qgroup_num_bytes = block_rsv->qgroup_rsv_size - - block_rsv->qgroup_rsv_reserved; -spin_unlock (&block_rsv->lock); - +__get_refill_bytes (block_rsv, &num_bytes, &qgroup_num_bytes);if (num_bytes ==0 )return 0 ; - ret =btrfs_qgroup_reserve_meta_prealloc (root, qgroup_num_bytes, true); -if (ret) -return ret; - ret =reserve_metadata_bytes (root, block_rsv, num_bytes, flush); +do { + ret =btrfs_qgroup_reserve_meta_prealloc (root, qgroup_num_bytes, true); +if (ret) +return ret; + ret =reserve_metadata_bytes (root, block_rsv, num_bytes, flush); +if (ret) { +btrfs_qgroup_free_meta_prealloc (root, qgroup_num_bytes); + last = num_bytes; +/* +* If we are fragmented we can end up with a lot of +* outstanding extents which will make our size be much +* larger than our reserved amount. If we happen to +* try to do a reservation here that may result in us +* trying to do a pretty hefty reservation, which we may +* not need once delalloc flushing happens. If this is +* the case try and do the reserve again. +*/ +if (flush ==BTRFS_RESERVE_FLUSH_ALL ) +__get_refill_bytes (block_rsv, &num_bytes, + &qgroup_num_bytes); + } + }while (ret && last != num_bytes); +if (!ret) {block_rsv_add_bytes (block_rsv, num_bytes,0 ); trace_btrfs_space_reservation(root->fs_info, "delalloc",
@@ -5809,8 +5836,7 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode,spin_lock (&block_rsv->lock); block_rsv->qgroup_rsv_reserved += qgroup_num_bytes;spin_unlock (&block_rsv->lock); - }else -btrfs_qgroup_free_meta_prealloc (root, qgroup_num_bytes); + }return ret; }
Signature · 1 line
2.14.3
PATCH v4 17/42 btrfs: run delayed iputs before committing
In reply to Josef BacikDelayed iputs means we can have final iputs of deleted inodes in the queue, which could potentially generate a lot of pinned space that could be free'd. So before we decide to commit the transaction for ENOPSC reasons, run the delayed iputs so that any potential space is free'd up. If there is and we freed enough we can then commit the transaction and potentially be able to make our reservation.
--- fs/btrfs/extent-tree.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/btrfs/extent-tree.c +9 -0
@@ -4823,6 +4823,15 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ; +/* +* If we have pending delayed iputs then we could free up a bunch of +* pinned space, so make sure we run the iputs before we do our pinned +* bytes check below. +*/ +mutex_lock (&fs_info->cleaner_delayed_iput_mutex); +btrfs_run_delayed_iputs (fs_info); +mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); + trans =btrfs_join_transaction (fs_info->extent_root);if (IS_ERR (trans))return -ENOSPC ;
Signature · 1 line
2.14.3
PATCH v4 18/42 btrfs: move the dio_sem higher up the callchain
In reply to Josef BacikWe're getting a lockdep splat because we take the dio_sem under the log_mutex. What we really need is to protect fsync() from logging an extent map for an extent we never waited on higher up, so just guard the whole thing with dio_sem.
======================================================
WARNING: possible circular locking dependency detected 4.18.0-rc4-xfstests-00025-g5de5edbaf1d4 #411 Not tainted
------------------------------------------------------
aio-dio-invalid/30928 is trying to acquire lock:0000000092621cfd (&mm->mmap_sem){++++}, at: get_user_pages_unlocked+0x5a/0x1e0
but task is already holding lock:00000000cefe6b35 (&ei->dio_sem){++++}, at: btrfs_direct_IO+0x3be/0x400
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #5 (&ei->dio_sem){++++}:
lock_acquire+0xbd/0x220
down_write+0x51/0xb0
btrfs_log_changed_extents+0x80/0xa40
btrfs_log_inode+0xbaf/0x1000
btrfs_log_inode_parent+0x26f/0xa80
btrfs_log_dentry_safe+0x50/0x70
btrfs_sync_file+0x357/0x540
do_fsync+0x38/0x60
__ia32_sys_fdatasync+0x12/0x20
do_fast_syscall_32+0x9a/0x2f0
entry_SYSENTER_compat+0x84/0x96-> #4 (&ei->log_mutex){+.+.}:
lock_acquire+0xbd/0x220
__mutex_lock+0x86/0xa10
btrfs_record_unlink_dir+0x2a/0xa0
btrfs_unlink+0x5a/0xc0
vfs_unlink+0xb1/0x1a0
do_unlinkat+0x264/0x2b0
do_fast_syscall_32+0x9a/0x2f0
entry_SYSENTER_compat+0x84/0x96-> #3 (sb_internal#2){.+.+}:
lock_acquire+0xbd/0x220
__sb_start_write+0x14d/0x230
start_transaction+0x3e6/0x590
btrfs_evict_inode+0x475/0x640
evict+0xbf/0x1b0
btrfs_run_delayed_iputs+0x6c/0x90
cleaner_kthread+0x124/0x1a0
kthread+0x106/0x140
ret_from_fork+0x3a/0x50-> #2 (&fs_info->cleaner_delayed_iput_mutex){+.+.}:
lock_acquire+0xbd/0x220
__mutex_lock+0x86/0xa10
btrfs_alloc_data_chunk_ondemand+0x197/0x530
btrfs_check_data_free_space+0x4c/0x90
btrfs_delalloc_reserve_space+0x20/0x60
btrfs_page_mkwrite+0x87/0x520
do_page_mkwrite+0x31/0xa0
__handle_mm_fault+0x799/0xb00
handle_mm_fault+0x7c/0xe0
__do_page_fault+0x1d3/0x4a0
async_page_fault+0x1e/0x30-> #1 (sb_pagefaults){.+.+}:
lock_acquire+0xbd/0x220
__sb_start_write+0x14d/0x230
btrfs_page_mkwrite+0x6a/0x520
do_page_mkwrite+0x31/0xa0
__handle_mm_fault+0x799/0xb00
handle_mm_fault+0x7c/0xe0
__do_page_fault+0x1d3/0x4a0
async_page_fault+0x1e/0x30-> #0 (&mm->mmap_sem){++++}:
__lock_acquire+0x42e/0x7a0
lock_acquire+0xbd/0x220
down_read+0x48/0xb0
get_user_pages_unlocked+0x5a/0x1e0
get_user_pages_fast+0xa4/0x150
iov_iter_get_pages+0xc3/0x340
do_direct_IO+0xf93/0x1d70
__blockdev_direct_IO+0x32d/0x1c20
btrfs_direct_IO+0x227/0x400
generic_file_direct_write+0xcf/0x180
btrfs_file_write_iter+0x308/0x58c
aio_write+0xf8/0x1d0
io_submit_one+0x3a9/0x620
__ia32_compat_sys_io_submit+0xb2/0x270
do_int80_syscall_32+0x5b/0x1a0
entry_INT80_compat+0x88/0xa0other info that might help us debug this:
Chain exists of: &mm->mmap_sem --> &ei->log_mutex --> &ei->dio_sem Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&ei->dio_sem);
lock(&ei->log_mutex);
lock(&ei->dio_sem); lock(&mm->mmap_sem);
*** DEADLOCK ***
1 lock held by aio-dio-invalid/30928:
#0: 00000000cefe6b35 (&ei->dio_sem){++++}, at: btrfs_direct_IO+0x3be/0x400
stack backtrace: CPU: 0 PID: 30928 Comm: aio-dio-invalid Not tainted 4.18.0-rc4-xfstests-00025-g5de5edbaf1d4 #411 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014 Call Trace: dump_stack+0x7c/0xbb print_circular_bug.isra.37+0x297/0x2a4 check_prev_add.constprop.45+0x781/0x7a0 ? __lock_acquire+0x42e/0x7a0 validate_chain.isra.41+0x7f0/0xb00 __lock_acquire+0x42e/0x7a0 lock_acquire+0xbd/0x220 ? get_user_pages_unlocked+0x5a/0x1e0 down_read+0x48/0xb0 ? get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_unlocked+0x5a/0x1e0 get_user_pages_fast+0xa4/0x150 iov_iter_get_pages+0xc3/0x340 do_direct_IO+0xf93/0x1d70 ? __alloc_workqueue_key+0x358/0x490 ? __blockdev_direct_IO+0x14b/0x1c20 __blockdev_direct_IO+0x32d/0x1c20 ? btrfs_run_delalloc_work+0x40/0x40 ? can_nocow_extent+0x490/0x490 ? kvm_clock_read+0x1f/0x30 ? can_nocow_extent+0x490/0x490 ? btrfs_run_delalloc_work+0x40/0x40 btrfs_direct_IO+0x227/0x400 ? btrfs_run_delalloc_work+0x40/0x40 generic_file_direct_write+0xcf/0x180 btrfs_file_write_iter+0x308/0x58c aio_write+0xf8/0x1d0 ? kvm_clock_read+0x1f/0x30 ? __might_fault+0x3e/0x90 io_submit_one+0x3a9/0x620 ? io_submit_one+0xe5/0x620 __ia32_compat_sys_io_submit+0xb2/0x270 do_int80_syscall_32+0x5b/0x1a0 entry_INT80_compat+0x88/0xa0
--- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-)
fs/btrfs/file.c +12 -0
@@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out;inode_lock (inode); + +/* +* We take the dio_sem here because the tree log stuff can race with +* lockless dio writes and get an extent map logged for an extent we +* never waited on. We need it this high up for lockdep reasons. +*/ +down_write (&BTRFS_I (inode)->dio_sem); +atomic_inc (&root->log_batch);/*
@@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)*/ ret =btrfs_wait_ordered_range (inode, start, len);if (ret) { +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }
@@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* checked called fsync. */ ret =filemap_check_wb_err (inode->i_mapping, file->f_wb_err); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }
@@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans =btrfs_start_transaction (root,0 );if (IS_ERR (trans)) { ret =PTR_ERR (trans); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }
@@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode);/*
fs/btrfs/tree-log.c +0 -2
@@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans,INIT_LIST_HEAD (&extents); -down_write (&inode->dio_sem);write_lock (&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start;
@@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, }WARN_ON (!list_empty (&extents));write_unlock (&tree->lock); -up_write (&inode->dio_sem);btrfs_release_path (path);if (!ret)
Signature · 1 line
2.14.3
PATCH v4 19/42 btrfs: set max_extent_size properly
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We can't use entry->bytes if our entry is a bitmap entry, we need to use entry->max_extent_size in that case. Fix up all the logic to make this consistent.
--- fs/btrfs/free-space-cache.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)
fs/btrfs/free-space-cache.c +19 -10
@@ -1770,6 +1770,18 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl,return -1 ; } +static void set_max_extent_size (struct btrfs_free_space *entry, +u64 *max_extent_size) +{ +if (entry->bitmap) { +if (entry->max_extent_size > *max_extent_size) + *max_extent_size = entry->max_extent_size; + }else { +if (entry->bytes > *max_extent_size) + *max_extent_size = entry->bytes; + } +} +/* Cache the size of the max extent in bytes */ static struct btrfs_free_space * find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes,
@@ -1791,8 +1803,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes,for (node = &entry->offset_index; node; node =rb_next (node)) { entry = rb_entry(node, struct btrfs_free_space, offset_index);if (entry->bytes < *bytes) { -if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +set_max_extent_size (entry, max_extent_size);continue ; }
@@ -1810,8 +1821,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, }if (entry->bytes < *bytes + align_off) { -if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +set_max_extent_size (entry, max_extent_size);continue ; }
@@ -1823,8 +1833,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, *offset = tmp; *bytes = size;return entry; - }else if (size > *max_extent_size) { - *max_extent_size = size; + }else { +set_max_extent_size (entry, max_extent_size); }continue ; }
@@ -2684,8 +2694,7 @@ static u64 btrfs_alloc_from_bitmap(struct btrfs_block_group_cache *block_group, err =search_bitmap (ctl, entry, &search_start, &search_bytes, true);if (err) { -if (search_bytes > *max_extent_size) - *max_extent_size = search_bytes; +set_max_extent_size (entry, max_extent_size);return 0 ; }
@@ -2722,8 +2731,8 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, entry = rb_entry(node, struct btrfs_free_space, offset_index);while (1 ) { -if (entry->bytes < bytes && entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +if (entry->bytes < bytes) +set_max_extent_size (entry, max_extent_size);if (entry->bytes < bytes || (!entry->bitmap && entry->offset < min_start)) {
Signature · 1 line
2.14.3
Re: [PATCH 19/42] btrfs: set max_extent_size properly
Quoted 8 lines · Josef Bacik
From: Josef Bacik <[email protected]>
We can't use
entry->bytesif our entry is a bitmap entry, we need to useentry->max_extent_sizein that case. Fix up all the logic to make this consistent.Signed-off-by Josef Bacik <[email protected]>
Quoted 82 lines
--- fs/btrfs/free-space-cache.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)fs/btrfs/free-space-cache.c +19
-11@@ -1770,6 +1770,18 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl,return -1 ; } +static void set_max_extent_size (struct btrfs_free_space *entry, +u64 *max_extent_size) +{ +if (entry->bitmap) { +if (entry->max_extent_size > *max_extent_size) + *max_extent_size = entry->max_extent_size; + }else { +if (entry->bytes > *max_extent_size) + *max_extent_size = entry->bytes; + } +} +/* Cache the size of the max extent in bytes */ static struct btrfs_free_space * find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes,@@ -1791,8 +1803,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes,for (node = &entry->offset_index; node; node =rb_next (node)) { entry = rb_entry(node, struct btrfs_free_space, offset_index);if (entry->bytes < *bytes) { -if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +set_max_extent_size (entry, max_extent_size);continue ; }@@ -1810,8 +1821,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, }if (entry->bytes < *bytes + align_off) { -if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +set_max_extent_size (entry, max_extent_size);continue ; }@@ -1823,8 +1833,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, *offset = tmp; *bytes = size;return entry; - }else if (size > *max_extent_size) { - *max_extent_size = size; + }else { +set_max_extent_size (entry, max_extent_size); }continue ; }@@ -2684,8 +2694,7 @@ static u64 btrfs_alloc_from_bitmap(struct btrfs_block_group_cache *block_group, err =search_bitmap (ctl, entry, &search_start, &search_bytes, true);if (err) { -if (search_bytes > *max_extent_size) - *max_extent_size = search_bytes; +set_max_extent_size (entry, max_extent_size);return 0 ; }@@ -2722,8 +2731,8 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, entry = rb_entry(node, struct btrfs_free_space, offset_index);while (1 ) { -if (entry->bytes < bytes && entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +if (entry->bytes < bytes) +set_max_extent_size (entry, max_extent_size);if (entry->bytes < bytes || (!entry->bitmap && entry->offset < min_start)) { --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Re: [PATCH 19/42] btrfs: set max_extent_size properly
In reply to Josef BacikQuoted 16 lines · Josef Bacik
From: Josef Bacik <[email protected]>
We can't use
entry->bytesif our entry is a bitmap entry, we need to useentry->max_extent_sizein that case. Fix up all the logic to make this consistent.Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/free-space-cache.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)fs/btrfs/free-space-cache.c +0
-0
fs/btrfs/free-space-cache.c static int search_bitmap(struct btrfs_free_space_ctl *ctl, · line 1773
Quoted 4 lines
fs/btrfs/free-space-cache.c +1
-0@@ -1770,6 +1770,18 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl,return -1 ; } +static void set_max_extent_size(struct btrfs_free_space *entry,
I find this a bit confusing, in the code it's like
set_max_extent_size(entry, max_extent_size)
so it reads like 'set max extent size of the entry to value', while the function does the opposite. That the 2nd parameter is a pointer does not help either. A function comment explaining that would not help as it's confusing at the callsites.
*max_extent_size = get_max_extent_size(entry, *max_extent_size)
This is more obvious what it does but can't be called an improvement regarding readability. Other ideas?
PATCH v4 20/42 btrfs: don't use ctl->free_space for max_extent_size
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
max_extent_size is supposed to be the largest contiguous range for the space info, and ctl->free_space is the total free space in the block group. We need to keep track of these separately and _only_ use the max_free_space if we don't have a max_extent_size, as that means our original request was too large to search any of the block groups for and therefore wouldn't have a max_extent_size set.
--- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
fs/btrfs/extent-tree.c +5 -2
@@ -7496,6 +7496,7 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,struct btrfs_block_group_cache *block_group =NULL ;u64 search_start =0 ;u64 max_extent_size =0 ; +u64 max_free_space =0 ;u64 empty_cluster =0 ;struct btrfs_space_info *space_info;int loop =0 ;
@@ -7791,8 +7792,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,spin_lock (&ctl->tree_lock);if (ctl->free_space < num_bytes + empty_cluster + empty_size) { -if (ctl->free_space > max_extent_size) - max_extent_size = ctl->free_space; + max_free_space =max (max_free_space, + ctl->free_space);spin_unlock (&ctl->tree_lock); goto loop; }
@@ -7959,6 +7960,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, } out:if (ret == -ENOSPC ) { +if (!max_extent_size) + max_extent_size = max_free_space;spin_lock (&space_info->lock); space_info->max_extent_size = max_extent_size;spin_unlock (&space_info->lock);
Signature · 1 line
2.14.3
Re: [PATCH 20/42] btrfs: don't use ctl->free_space for max_extent_size
Quoted 11 lines · Josef Bacik
From: Josef Bacik <[email protected]>
max_extent_sizeis supposed to be the largest contiguous range for the space info, andctl->free_spaceis the total free space in the block group. We need to keep track of these separately and _only_ use themax_free_spaceif we don't have amax_extent_size, as that means our original request was too large to search any of the block groups for and therefore wouldn't have amax_extent_sizeset.Signed-off-by Josef Bacik <[email protected]>
Quoted 39 lines
--- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)fs/btrfs/extent-tree.c +5
-3@@ -7496,6 +7496,7 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,struct btrfs_block_group_cache *block_group =NULL ;u64 search_start =0 ;u64 max_extent_size =0 ; +u64 max_free_space =0 ;u64 empty_cluster =0 ;struct btrfs_space_info *space_info;int loop =0 ;@@ -7791,8 +7792,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,spin_lock (&ctl->tree_lock);if (ctl->free_space < num_bytes + empty_cluster + empty_size) { -if (ctl->free_space > max_extent_size) - max_extent_size = ctl->free_space; + max_free_space =max (max_free_space, + ctl->free_space);spin_unlock (&ctl->tree_lock); goto loop; }@@ -7959,6 +7960,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, } out:if (ret == -ENOSPC ) { +if (!max_extent_size) + max_extent_size = max_free_space;spin_lock (&space_info->lock); space_info->max_extent_size = max_extent_size;spin_unlock (&space_info->lock); --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
PATCH v4 21/42 btrfs: reset max_extent_size on clear in a bitmap
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We need to clear the max_extent_size when we clear bits from a bitmap since it could have been from the range that contains the max_extent_size.
--- fs/btrfs/free-space-cache.c | 2 ++ 1 file changed, 2 insertions(+)
fs/btrfs/free-space-cache.c +2 -0
@@ -1687,6 +1687,8 @@ static inline void __bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,bitmap_clear (info->bitmap, start, count); info->bytes -= bytes; +if (info->max_extent_size > ctl->unit) + info->max_extent_size =0 ; } static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,
Signature · 1 line
2.14.3
PATCH v4 22/42 btrfs: only run delayed refs if we're committing
In reply to Josef BacikI noticed in a giant dbench run that we spent a lot of time on lock contention while running transaction commit. This is because dbench results in a lot of fsync()'s that do a btrfs_transaction_commit(), and they all run the delayed refs first thing, so they all contend with each other. This leads to seconds of 0 throughput. Change this to only run the delayed refs if we're the ones committing the transaction. This makes the latency go away and we get no more lock contention.
--- fs/btrfs/transaction.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-)
fs/btrfs/transaction.c +9 -15
@@ -1925,15 +1925,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)btrfs_trans_release_metadata (trans); trans->block_rsv =NULL ; -/* make a pass through all the delayed refs we have so far -* any runnings procs may add more while we are here -*/ - ret =btrfs_run_delayed_refs (trans,0 ); -if (ret) { -btrfs_end_transaction (trans); -return ret; - } - cur_trans = trans->transaction;/*
@@ -1946,12 +1937,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)if (!list_empty (&trans->new_bgs))btrfs_create_pending_block_groups (trans); - ret =btrfs_run_delayed_refs (trans,0 ); -if (ret) { -btrfs_end_transaction (trans); -return ret; - } -if (!test_bit (BTRFS_TRANS_DIRTY_BG_RUN , &cur_trans->flags)) {int run_it =0 ;
@@ -2022,6 +2007,15 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)spin_unlock (&fs_info->trans_lock); } +/* +* We are now the only one in the commit area, we can run delayed refs +* without hitting a bunch of lock contention from a lot of people +* trying to commit the transaction at once. +*/ + ret =btrfs_run_delayed_refs (trans,0 ); +if (ret) + goto cleanup_transaction; +extwriter_counter_dec (cur_trans, trans->type); ret =btrfs_start_delalloc_flush (fs_info);
Signature · 1 line
2.14.3
PATCH v4 23/42 btrfs: make sure we create all new bgs
In reply to Josef BacikAllocating new chunks modifies both the extent and chunk tree, which can trigger new chunk allocations. So instead of doing list_for_each_safe, just do while (!list_empty()) so we make sure we don't exit with other pending bg's still on our list.
--- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
fs/btrfs/extent-tree.c +5 -2
@@ -10361,7 +10361,7 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)void btrfs_create_pending_block_groups (struct btrfs_trans_handle *trans) {struct btrfs_fs_info *fs_info = trans->fs_info; -struct btrfs_block_group_cache *block_group, *tmp; +struct btrfs_block_group_cache *block_group;struct btrfs_root *extent_root = fs_info->extent_root;struct btrfs_block_group_item item;struct btrfs_key key;
@@ -10369,7 +10369,10 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)bool can_flush_pending_bgs = trans->can_flush_pending_bgs; trans->can_flush_pending_bgs = false; -list_for_each_entry_safe (block_group, tmp, &trans->new_bgs, bg_list) { +while (!list_empty (&trans->new_bgs)) { + block_group =list_first_entry (&trans->new_bgs, + struct btrfs_block_group_cache, + bg_list);if (ret) goto next;
Signature · 1 line
2.14.3
PATCH v4 24/42 btrfs: assert on non-empty delayed iputs
In reply to Josef BacikI ran into an issue where there was some reference being held on an inode that I couldn't track. This assert wasn't triggered, but it at least rules out we're doing something stupid.
--- fs/btrfs/disk-io.c | 1 + 1 file changed, 1 insertion(+)
fs/btrfs/disk-io.c +1 -0
@@ -3979,6 +3979,7 @@ void close_ctree(struct btrfs_fs_info *fs_info)kthread_stop (fs_info->transaction_kthread);kthread_stop (fs_info->cleaner_kthread); +ASSERT (list_empty (&fs_info->delayed_iputs));set_bit (BTRFS_FS_CLOSING_DONE , &fs_info->flags);btrfs_free_qgroup_config (fs_info);
Signature · 1 line
2.14.3
PATCH v4 25/42 btrfs: pass delayed_refs_root to btrfs_delayed_ref_lock
In reply to Josef BacikWe don't need the trans except to get the delayed_refs_root, so just pass the delayed_refs_root into btrfs_delayed_ref_lock and call it a day.
--- fs/btrfs/delayed-ref.c | 5 +---- fs/btrfs/delayed-ref.h | 2 +- fs/btrfs/extent-tree.c | 2 +- 3 files changed, 3 insertions(+), 6 deletions(-)
fs/btrfs/delayed-ref.c +1 -4
@@ -197,12 +197,9 @@ find_ref_head(struct rb_root *root, u64 bytenr,return NULL ; } -int btrfs_delayed_ref_lock (struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock (struct btrfs_delayed_ref_root *delayed_refs,struct btrfs_delayed_ref_head *head) { -struct btrfs_delayed_ref_root *delayed_refs; - - delayed_refs = &trans->transaction->delayed_refs;lockdep_assert_held (&delayed_refs->lock);if (mutex_trylock (&head->mutex))return 0 ;
fs/btrfs/delayed-ref.h +1 -1
@@ -255,7 +255,7 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *btrfs_find_delayed_ref_head (struct btrfs_delayed_ref_root *delayed_refs,u64 bytenr); -int btrfs_delayed_ref_lock (struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock (struct btrfs_delayed_ref_root *delayed_refs,struct btrfs_delayed_ref_head *head);static inline void btrfs_delayed_ref_unlock (struct btrfs_delayed_ref_head *head) {
fs/btrfs/extent-tree.c +1 -1
@@ -2600,7 +2600,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,/* grab the lock that says we are going to process * all the refs for this head */ - ret =btrfs_delayed_ref_lock (trans, locked_ref); + ret =btrfs_delayed_ref_lock (delayed_refs, locked_ref);spin_unlock (&delayed_refs->lock);/* * we may have dropped the spin lock to get the head
Signature · 1 line
2.14.3
PATCH v4 26/42 btrfs: make btrfs_destroy_delayed_refs use btrfs_delayed_ref_lock
In reply to Josef BacikWe have this open coded in btrfs_destroy_delayed_refs, use the helper instead.
--- fs/btrfs/disk-io.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)
fs/btrfs/disk-io.c +2 -9
@@ -4214,16 +4214,9 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); -if (!mutex_trylock (&head->mutex)) { -refcount_inc (&head->refs); -spin_unlock (&delayed_refs->lock); - -mutex_lock (&head->mutex); -mutex_unlock (&head->mutex); -btrfs_put_delayed_ref_head (head); -spin_lock (&delayed_refs->lock); +if (btrfs_delayed_ref_lock (delayed_refs, head))continue ; - } +spin_lock (&head->lock);while ((n =rb_first (&head->ref_tree)) !=NULL ) { ref = rb_entry(n, struct btrfs_delayed_ref_node,
Signature · 1 line
2.14.3
PATCH v4 27/42 btrfs: make btrfs_destroy_delayed_refs use btrfs_delete_ref_head
In reply to Josef BacikInstead of open coding this stuff use the helper instead.
--- fs/btrfs/disk-io.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
fs/btrfs/disk-io.c +1 -6
@@ -4232,12 +4232,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,if (head->must_insert_reserved) pin_bytes = true;btrfs_free_delayed_extent_op (head->extent_op); - delayed_refs->num_heads--; -if (head->processing ==0 ) - delayed_refs->num_heads_ready--; -atomic_dec (&delayed_refs->num_entries); -rb_erase (&head->href_node, &delayed_refs->href_root); -RB_CLEAR_NODE (&head->href_node); +btrfs_delete_ref_head (delayed_refs, head);spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock);mutex_unlock (&head->mutex);
Signature · 1 line
2.14.3
PATCH v4 28/42 btrfs: handle delayed ref head accounting cleanup in abort
In reply to Josef BacikWe weren't doing any of the accounting cleanup when we aborted transactions. Fix this by making cleanup_ref_head_accounting global and calling it from the abort code, this fixes the issue where our accounting was all wrong after the fs aborts.
--- fs/btrfs/ctree.h | 5 +++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/extent-tree.c | 13 ++++++------- 3 files changed, 12 insertions(+), 7 deletions(-)
fs/btrfs/ctree.h +5 -0
@@ -35,6 +35,7 @@struct btrfs_trans_handle ;struct btrfs_transaction ;struct btrfs_pending_snapshot ; +struct btrfs_delayed_ref_root ;extern struct kmem_cache *btrfs_trans_handle_cachep;extern struct kmem_cache *btrfs_bit_radix_cachep;extern struct kmem_cache *btrfs_path_cachep;
@@ -2623,6 +2624,10 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long count);int btrfs_async_run_delayed_refs (struct btrfs_fs_info *fs_info,unsigned long count,u64 transid,int wait); +void +btrfs_cleanup_ref_head_accounting (struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head);int btrfs_lookup_data_extent (struct btrfs_fs_info *fs_info,u64 start,u64 len); int btrfs_lookup_extent_info(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info, u64 bytenr,
fs/btrfs/disk-io.c +1 -0
@@ -4240,6 +4240,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,if (pin_bytes)btrfs_pin_extent (fs_info, head->bytenr, head->num_bytes,1 ); +btrfs_cleanup_ref_head_accounting (fs_info, delayed_refs, head);btrfs_put_delayed_ref_head (head);cond_resched ();spin_lock (&delayed_refs->lock);
fs/btrfs/extent-tree.c +6 -7
@@ -2475,12 +2475,11 @@ static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans,return ret ? ret :1 ; } -static void cleanup_ref_head_accounting (struct btrfs_trans_handle *trans, -struct btrfs_delayed_ref_head *head) +void +btrfs_cleanup_ref_head_accounting (struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head) { -struct btrfs_fs_info *fs_info = trans->fs_info; -struct btrfs_delayed_ref_root *delayed_refs = - &trans->transaction->delayed_refs;int nr_items =1 ;if (head->total_ref_mod <0 ) {
@@ -2558,7 +2557,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } -cleanup_ref_head_accounting (trans, head); +btrfs_cleanup_ref_head_accounting (fs_info, delayed_refs, head);trace_run_delayed_ref_head (fs_info, head,0 );btrfs_delayed_ref_unlock (head);
@@ -7227,7 +7226,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (head->must_insert_reserved) ret =1 ; -cleanup_ref_head_accounting (trans, head); +btrfs_cleanup_ref_head_accounting (trans->fs_info, delayed_refs, head);mutex_unlock (&head->mutex);btrfs_put_delayed_ref_head (head);return ret;
Signature · 1 line
2.14.3
PATCH v4 29/42 btrfs: call btrfs_create_pending_block_groups unconditionally
In reply to Josef BacikThe first thing we do is loop through the list, this
if (!list_empty())
btrfs_create_pending_block_groups();
thing is just wasted space.
--- fs/btrfs/extent-tree.c | 3 +-- fs/btrfs/transaction.c | 6 ++---- 2 files changed, 3 insertions(+), 6 deletions(-)
fs/btrfs/extent-tree.c +1 -2
@@ -2978,8 +2978,7 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, }if (run_all) { -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); +btrfs_create_pending_block_groups (trans);spin_lock (&delayed_refs->lock); node =rb_first (&delayed_refs->href_root);
fs/btrfs/transaction.c +2 -4
@@ -846,8 +846,7 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,btrfs_trans_release_metadata (trans); trans->block_rsv =NULL ; -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); +btrfs_create_pending_block_groups (trans);btrfs_trans_release_chunk_metadata (trans);
@@ -1934,8 +1933,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing =1 ;smp_wmb (); -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); +btrfs_create_pending_block_groups (trans);if (!test_bit (BTRFS_TRANS_DIRTY_BG_RUN , &cur_trans->flags)) {int run_it =0 ;
Signature · 1 line
2.14.3
PATCH v4 30/42 btrfs: just delete pending bgs if we are aborted
In reply to Josef BacikWe still need to do all of the accounting cleanup for pending block groups if we abort. So set the ret to trans->aborted so if we aborted the cleanup happens and everybody is happy.
--- fs/btrfs/extent-tree.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +7 -1
@@ -10363,9 +10363,15 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)struct btrfs_root *extent_root = fs_info->extent_root;struct btrfs_block_group_item item;struct btrfs_key key; -int ret =0 ; +int ret;bool can_flush_pending_bgs = trans->can_flush_pending_bgs; +/* +* If we aborted the transaction with pending bg's we need to just +* cleanup the list and carry on. +*/ + ret = trans->aborted; + trans->can_flush_pending_bgs = false;while (!list_empty (&trans->new_bgs)) { block_group = list_first_entry(&trans->new_bgs,
Signature · 1 line
2.14.3
PATCH v4 31/42 btrfs: cleanup pending bgs on transaction abort
In reply to Josef BacikWe may abort the transaction during a commit and not have a chance to run the pending bgs stuff, which will leave block groups on our list and cause us accounting issues and leaked memory. Fix this by running the pending bgs when we cleanup a transaction.
--- fs/btrfs/transaction.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/btrfs/transaction.c +4 -0
@@ -2280,6 +2280,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)btrfs_scrub_continue (fs_info); cleanup_transaction:btrfs_trans_release_metadata (trans); +/* This cleans up the pending block groups list properly. */ +if (!trans->aborted) + trans->aborted = ret; +btrfs_create_pending_block_groups (trans);btrfs_trans_release_chunk_metadata (trans); trans->block_rsv =NULL ;btrfs_warn (fs_info,"Skipping commit of aborted transaction." );
Signature · 1 line
2.14.3
PATCH v4 32/42 btrfs: only free reserved extent if we didn't insert it
In reply to Josef BacikWhen we insert the file extent once the ordered extent completes we free the reserved extent reservation as it'll have been migrated to the bytes_used counter. However if we error out after this step we'll still clear the reserved extent reservation, resulting in a negative accounting of the reserved bytes for the block group and space info. Fix this by only doing the free if we didn't successfully insert a file extent for this extent.
--- fs/btrfs/inode.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
fs/btrfs/inode.c +9 -1
@@ -2992,6 +2992,7 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent)bool truncated = false;bool range_locked = false;bool clear_new_delalloc_bytes = false; +bool clear_reserved_extent = true; if (!test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags) &&
@@ -3095,10 +3096,12 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) logical_len, logical_len, compress_type,0 ,0 , BTRFS_FILE_EXTENT_REG); -if (!ret) +if (!ret) { + clear_reserved_extent = false;btrfs_release_delalloc_bytes (fs_info, ordered_extent->start, ordered_extent->disk_len); + } } unpin_extent_cache(&BTRFS_I(inode)->extent_tree, ordered_extent->file_offset, ordered_extent->len,
@@ -3159,8 +3162,13 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent)* wrong we need to return the space for this ordered extent * back to the allocator. We only free the extent in the * truncated case if we didn't write out the extent at all. +* +* If we made it past insert_reserved_file_extent before we +* errored out then we don't need to do this as the accounting +* has already been done. */ if ((ret || !logical_len) && + clear_reserved_extent && !test_bit (BTRFS_ORDERED_NOCOW , &ordered_extent->flags) && !test_bit (BTRFS_ORDERED_PREALLOC , &ordered_extent->flags)) btrfs_free_reserved_extent(fs_info,
Signature · 1 line
2.14.3
Re: [PATCH 32/42] btrfs: only free reserved extent if we didn't insert it
Quoted 11 lines · Josef Bacik
When we insert the file extent once the ordered extent completes we free the reserved extent reservation as it'll have been migrated to the
bytes_usedcounter. However if we error out after this step we'll still clear the reserved extent reservation, resulting in a negative accounting of the reserved bytes for the block group and space info. Fix this by only doing the free if we didn't successfully insert a file extent for this extent.Signed-off-by Josef Bacik <[email protected]>Reviewed-by Omar Sandoval <[email protected]>
Quoted 47 lines
--- fs/btrfs/inode.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)fs/btrfs/inode.c +9
-2@@ -2992,6 +2992,7 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent)bool truncated = false;bool range_locked = false;bool clear_new_delalloc_bytes = false; +bool clear_reserved_extent = true; if (!test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags) &&@@ -3095,10 +3096,12 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) logical_len, logical_len, compress_type,0 ,0 , BTRFS_FILE_EXTENT_REG); -if (!ret) +if (!ret) { + clear_reserved_extent = false;btrfs_release_delalloc_bytes (fs_info, ordered_extent->start, ordered_extent->disk_len); + } } unpin_extent_cache(&BTRFS_I(inode)->extent_tree, ordered_extent->file_offset, ordered_extent->len,@@ -3159,8 +3162,13 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent)* wrong we need to return the space for this ordered extent * back to the allocator. We only free the extent in the * truncated case if we didn't write out the extent at all. +* +* If we made it past insert_reserved_file_extent before we +* errored out then we don't need to do this as the accounting +* has already been done. */ if ((ret || !logical_len) && + clear_reserved_extent && !test_bit (BTRFS_ORDERED_NOCOW , &ordered_extent->flags) && !test_bit (BTRFS_ORDERED_PREALLOC , &ordered_extent->flags)) btrfs_free_reserved_extent(fs_info, --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
PATCH v4 33/42 btrfs: fix insert_reserved error handling
In reply to Josef BacikWe were not handling the reserved byte accounting properly for data references. Metadata was fine, if it errored out the error paths would free the bytes_reserved count and pin the extent, but it even missed one of the error cases. So instead move this handling up into run_one_delayed_ref so we are sure that both cases are properly cleaned up in case of a transaction abort.
--- fs/btrfs/extent-tree.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
fs/btrfs/extent-tree.c +4 -8
@@ -2405,6 +2405,9 @@ static int run_one_delayed_ref(struct btrfs_trans_handle *trans, insert_reserved); elseBUG (); +if (ret && insert_reserved) +btrfs_pin_extent (trans->fs_info, node->bytenr, + node->num_bytes,1 );return ret; }
@@ -8257,21 +8260,14 @@ static int alloc_reserved_tree_block(struct btrfs_trans_handle *trans, } path =btrfs_alloc_path (); -if (!path) { -btrfs_free_and_pin_reserved_extent (fs_info, - extent_key.objectid, - fs_info->nodesize); +if (!path)return -ENOMEM ; - } path->leave_spinning =1 ; ret =btrfs_insert_empty_item (trans, fs_info->extent_root, path, &extent_key, size);if (ret) {btrfs_free_path (path); -btrfs_free_and_pin_reserved_extent (fs_info, - extent_key.objectid, - fs_info->nodesize);return ret; }
Signature · 1 line
2.14.3
Re: [PATCH 33/42] btrfs: fix insert_reserved error handling
Quoted 9 lines · Josef Bacik
We were not handling the reserved byte accounting properly for data references. Metadata was fine, if it errored out the error paths would free the
bytes_reservedcount and pin the extent, but it even missed one of the error cases. So instead move this handling up intorun_one_delayed_refso we are sure that both cases are properly cleaned up in case of a transaction abort.Reviewed-by Nikolay Borisov <[email protected]>Signed-off-by Josef Bacik <[email protected]>
PATCH v4 34/42 btrfs: wait on ordered extents on abort cleanup
In reply to Josef BacikIf we flip read-only before we initiate writeback on all dirty pages for ordered extents we've created then we'll have ordered extents left over on umount, which results in all sorts of bad things happening. Fix this by making sure we wait on ordered extents if we have to do the aborted transaction cleanup stuff.
--- fs/btrfs/disk-io.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/btrfs/disk-io.c +8 -0
@@ -4188,6 +4188,14 @@ static void btrfs_destroy_all_ordered_extents(struct btrfs_fs_info *fs_info)spin_lock (&fs_info->ordered_root_lock); }spin_unlock (&fs_info->ordered_root_lock); + +/* +* We need this here because if we've been flipped read-only we won't +* get sync() from the umount, so we need to make sure any ordered +* extents that haven't had their dirty pages IO start writeout yet +* actually get run and error out properly. +*/ +btrfs_wait_ordered_roots (fs_info,U64_MAX ,0 , (u64)-1 ); } static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,
Signature · 1 line
2.14.3
PATCH v4 35/42 MAINTAINERS: update my email address for btrfs
In reply to Josef BacikMy work email is completely useless, switch it to my personal address so I get emails on a account I actually pay attention to.
--- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
MAINTAINERS +1 -1
@@ -3095,7 +3095,7 @@ F: drivers/gpio/gpio-bt8xx.c BTRFS FILE SYSTEM M: Chris Mason <[email protected]> -M: Josef Bacik <[email protected]> +M: Josef Bacik <[email protected]> M: David Sterba <[email protected]> L: [email protected] W: http://btrfs.wiki.kernel.org/
Signature · 1 line
2.14.3
PATCH v4 36/42 btrfs: wait on caching when putting the bg cache
In reply to Josef BacikWhile testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is because we aren't waiting for the caching threads to be done before freeing everything up, so to fix this make sure we wait on any outstanding caching that's being done before we free up the block group, so we're sure to be done with all IO by the time we get to btrfs_stop_all_workers(). This fixes the panic I was seeing consistently in testing.
------------[ cut here ]------------ kernel BUG at fs/btrfs/volumes.c:6112! SMP PTI Modules linked in: CPU: 1 PID: 27165 Comm: kworker/u4:7 Not tainted 4.16.0-02155-g3553e54a578d-dirty #875 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014 Workqueue: btrfs-cache btrfs_cache_helper RIP: 0010:btrfs_map_bio+0x346/0x370 RSP: 0000:ffffc900061e79d0 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffff880071542e00 RCX: 0000000000533000 RDX: ffff88006bb74380 RSI: 0000000000000008 RDI: ffff880078160000 RBP: 0000000000000001 R08: ffff8800781cd200 R09: 0000000000503000 R10: ffff88006cd21200 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: ffff8800781cd200 R15: ffff880071542e00 FS: 0000000000000000(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000817ffc4 CR3: 0000000078314000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: btree_submit_bio_hook+0x8a/0xd0 submit_one_bio+0x5d/0x80 read_extent_buffer_pages+0x18a/0x320 btree_read_extent_buffer_pages+0xbc/0x200 ? alloc_extent_buffer+0x359/0x3e0 read_tree_block+0x3d/0x60 read_block_for_search.isra.30+0x1a5/0x360 btrfs_search_slot+0x41b/0xa10 btrfs_next_old_leaf+0x212/0x470 caching_thread+0x323/0x490 normal_work_helper+0xc5/0x310 process_one_work+0x141/0x340 worker_thread+0x44/0x3c0 kthread+0xf8/0x130 ? process_one_work+0x340/0x340 ? kthread_bind+0x10/0x10 ret_from_fork+0x35/0x40 Code: ff ff 48 8b 4c 24 28 48 89 de 48 8b 7c 24 08 e8 d1 e5 04 00 89 c3 e9 08 ff ff ff 4d 89 c6 49 89 df e9 27 fe ff ff e8 5a 3a bb ff <0f> 0b 0f 0b e9 57 ff ff ff 48 8b 7c 24 08 4c 89 f9 4c 89 ea 48 RIP: btrfs_map_bio+0x346/0x370 RSP: ffffc900061e79d0 ---[ end trace 827eb13e50846033 ]--- Kernel panic - not syncing: Fatal exception Kernel Offset: disabled ---[ end Kernel panic - not syncing: Fatal exception
--- fs/btrfs/extent-tree.c | 1 + 1 file changed, 1 insertion(+)
fs/btrfs/extent-tree.c +1 -0
@@ -9894,6 +9894,7 @@ void btrfs_put_block_group_cache(struct btrfs_fs_info *info) block_group =btrfs_lookup_first_block_group (info, last);while (block_group) { +wait_block_group_cache_done (block_group);spin_lock (&block_group->lock);if (block_group->iref)break ;
Signature · 1 line
2.14.3
PATCH v4 37/42 btrfs: wakeup cleaner thread when adding delayed iput
In reply to Josef BacikThe cleaner thread usually takes care of delayed iputs, with the exception of the btrfs_end_transaction_throttle path. The cleaner thread only gets woken up every 30 seconds, so instead wake it up to do it's work so that we can free up that space as quickly as possible.
--- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+)
fs/btrfs/inode.c +1 -0
@@ -3323,6 +3323,7 @@ void btrfs_add_delayed_iput(struct inode *inode)ASSERT (list_empty (&binode->delayed_iput));list_add_tail (&binode->delayed_iput, &fs_info->delayed_iputs);spin_unlock (&fs_info->delayed_iput_lock); +wake_up_process (fs_info->cleaner_kthread); }void btrfs_run_delayed_iputs (struct btrfs_fs_info *fs_info)
Signature · 1 line
2.14.3
PATCH v4 38/42 btrfs: be more explicit about allowed flush states
In reply to Josef BacikFor FLUSH_LIMIT flushers we really can only allocate chunks and flush delayed inode items, everything else is problematic. I added a bunch of new states and it lead to weirdness in the FLUSH_LIMIT case because I forgot about how it worked. So instead explicitly declare the states that are ok for flushing with FLUSH_LIMIT and use that for our state machine. Then as we add new things that are safe we can just add them to this list.
--- fs/btrfs/extent-tree.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-)
fs/btrfs/extent-tree.c +10 -11
@@ -5110,12 +5110,18 @@ void btrfs_init_async_reclaim_work(struct work_struct *work)INIT_WORK (work, btrfs_async_reclaim_metadata_space); } +static const enum btrfs_flush_state priority_flush_states[] = { +FLUSH_DELAYED_ITEMS_NR , +FLUSH_DELAYED_ITEMS , +ALLOC_CHUNK , +}; +static void priority_reclaim_metadata_space (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,struct reserve_ticket *ticket) {u64 to_reclaim; -int flush_state =FLUSH_DELAYED_ITEMS_NR ; +int flush_state =0 ;spin_lock (&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info,
@@ -5127,7 +5133,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,spin_unlock (&space_info->lock); do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, + priority_flush_states[flush_state]); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) {
@@ -5135,15 +5142,7 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,return ; }spin_unlock (&space_info->lock); - -/* -* Priority flushers can't wait on delalloc without -* deadlocking. -*/ -if (flush_state ==FLUSH_DELALLOC || - flush_state ==FLUSH_DELALLOC_WAIT ) - flush_state =ALLOC_CHUNK ; - }while (flush_state <COMMIT_TRANS ); + }while (flush_state <ARRAY_SIZE (priority_flush_states)); } static int wait_reserve_ticket(struct btrfs_fs_info *fs_info,
Signature · 1 line
2.14.3
PATCH v4 39/42 btrfs: replace cleaner_delayed_iput_mutex with a waitqueue
In reply to Josef BacikThe throttle path doesn't take cleaner_delayed_iput_mutex, which means we could think we're done flushing iputs in the data space reservation path when we could have a throttler doing an iput. There's no real reason to serialize the delayed iput flushing, so instead of taking the cleaner_delayed_iput_mutex whenever we flush the delayed iputs just replace it with an atomic counter and a waitqueue. This removes the short (or long depending on how big the inode is) window where we think there are no more pending iputs when there really are some.
--- fs/btrfs/ctree.h | 4 +++- fs/btrfs/disk-io.c | 5 ++--- fs/btrfs/extent-tree.c | 9 +++++---- fs/btrfs/inode.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 8 deletions(-)
fs/btrfs/ctree.h +3 -1
@@ -894,7 +894,8 @@ struct btrfs_fs_info {spinlock_t delayed_iput_lock;struct list_head delayed_iputs; -struct mutex cleaner_delayed_iput_mutex; +atomic_t nr_delayed_iputs; +wait_queue_head_t delayed_iputs_wait;/* this protects tree_mod_seq_list */ spinlock_t tree_mod_seq_lock;
@@ -3212,6 +3213,7 @@ int btrfs_orphan_cleanup(struct btrfs_root *root);int btrfs_cont_expand (struct inode *inode,loff_t oldsize,loff_t size);void btrfs_add_delayed_iput (struct inode *inode);void btrfs_run_delayed_iputs (struct btrfs_fs_info *fs_info); +int btrfs_wait_on_delayed_iputs (struct btrfs_fs_info *fs_info);int btrfs_prealloc_file_range (struct inode *inode,int mode,u64 start,u64 num_bytes,u64 min_size,loff_t actual_len,u64 *alloc_hint);
fs/btrfs/disk-io.c +2 -3
@@ -1692,9 +1692,7 @@ static int cleaner_kthread(void *arg) goto sleep; } -mutex_lock (&fs_info->cleaner_delayed_iput_mutex);btrfs_run_delayed_iputs (fs_info); -mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); again =btrfs_clean_one_deleted_snapshot (root);mutex_unlock (&fs_info->cleaner_mutex);
@@ -2677,7 +2675,6 @@ int open_ctree(struct super_block *sb,mutex_init (&fs_info->delete_unused_bgs_mutex);mutex_init (&fs_info->reloc_mutex);mutex_init (&fs_info->delalloc_root_mutex); -mutex_init (&fs_info->cleaner_delayed_iput_mutex);seqlock_init (&fs_info->profiles_lock);INIT_LIST_HEAD (&fs_info->dirty_cowonly_roots);
@@ -2699,6 +2696,7 @@ int open_ctree(struct super_block *sb,atomic_set (&fs_info->defrag_running,0 );atomic_set (&fs_info->qgroup_op_seq,0 );atomic_set (&fs_info->reada_works_cnt,0 ); +atomic_set (&fs_info->nr_delayed_iputs,0 );atomic64_set (&fs_info->tree_mod_seq,0 ); fs_info->sb = sb; fs_info->max_inline =BTRFS_DEFAULT_MAX_INLINE ;
@@ -2776,6 +2774,7 @@ int open_ctree(struct super_block *sb,init_waitqueue_head (&fs_info->transaction_wait);init_waitqueue_head (&fs_info->transaction_blocked_wait);init_waitqueue_head (&fs_info->async_submit_wait); +init_waitqueue_head (&fs_info->delayed_iputs_wait);INIT_LIST_HEAD (&fs_info->pinned_chunks);
fs/btrfs/extent-tree.c +5 -4
@@ -4258,8 +4258,9 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes)* operations. Wait for it to finish so that * more space is released. */ -mutex_lock (&fs_info->cleaner_delayed_iput_mutex); -mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); + ret =btrfs_wait_on_delayed_iputs (fs_info); +if (ret) +return ret; goto again; }else {btrfs_end_transaction (trans);
@@ -4829,9 +4830,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,* pinned space, so make sure we run the iputs before we do our pinned * bytes check below. */ -mutex_lock (&fs_info->cleaner_delayed_iput_mutex);btrfs_run_delayed_iputs (fs_info); -mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); +wait_event (fs_info->delayed_iputs_wait, +atomic_read (&fs_info->nr_delayed_iputs) ==0 ); trans =btrfs_join_transaction (fs_info->extent_root);if (IS_ERR (trans))
fs/btrfs/inode.c +21 -0
@@ -3319,6 +3319,7 @@ void btrfs_add_delayed_iput(struct inode *inode)if (atomic_add_unless (&inode->i_count,-1 ,1 ))return ; +atomic_inc (&fs_info->nr_delayed_iputs);spin_lock (&fs_info->delayed_iput_lock);ASSERT (list_empty (&binode->delayed_iput));list_add_tail (&binode->delayed_iput, &fs_info->delayed_iputs);
@@ -3338,11 +3339,31 @@ void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info)list_del_init (&inode->delayed_iput);spin_unlock (&fs_info->delayed_iput_lock);iput (&inode->vfs_inode); +if (atomic_dec_and_test (&fs_info->nr_delayed_iputs)) +wake_up (&fs_info->delayed_iputs_wait);spin_lock (&fs_info->delayed_iput_lock); }spin_unlock (&fs_info->delayed_iput_lock); } +/** +* btrfs_wait_on_delayed_iputs - wait on the delayed iputs to be done running +* @fs_info - the fs_info for this fs +* @return - EINTR if we were killed, 0 if nothing's pending +* +* This will wait on any delayed iputs that are currently running with KILLABLE +* set. Once they are all done running we will return, unless we are killed in +* which case we return EINTR. +*/ +int btrfs_wait_on_delayed_iputs (struct btrfs_fs_info *fs_info) +{ +int ret =wait_event_killable (fs_info->delayed_iputs_wait, +atomic_read (&fs_info->nr_delayed_iputs) ==0 ); +if (ret) +return -EINTR ; +return 0 ; +} +/* * This creates an orphan entry for the given inode in case something goes wrong * in the middle of an unlink.
Signature · 1 line
2.14.3
PATCH v4 40/42 btrfs: drop min_size from evict_refill_and_join
In reply to Josef BacikWe don't need it, rsv->size is set once and never changes throughout its lifetime, so just use that for the reserve size.
--- fs/btrfs/inode.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
fs/btrfs/inode.c +6 -10
@@ -5339,8 +5339,7 @@ static void evict_inode_truncate_pages(struct inode *inode) }static struct btrfs_trans_handle *evict_refill_and_join (struct btrfs_root *root, -struct btrfs_block_rsv *rsv, -u64 min_size) +struct btrfs_block_rsv *rsv) {struct btrfs_fs_info *fs_info = root->fs_info;struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
@@ -5350,7 +5349,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,struct btrfs_trans_handle *trans;int ret; - ret =btrfs_block_rsv_refill (root, rsv, min_size, + ret =btrfs_block_rsv_refill (root, rsv, rsv->size,BTRFS_RESERVE_FLUSH_LIMIT );if (ret && ++failures >2 ) {
@@ -5368,7 +5367,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,* it. */ if (!btrfs_check_space_for_delayed_refs (fs_info) && - !btrfs_block_rsv_migrate (global_rsv, rsv, min_size,0 )) + !btrfs_block_rsv_migrate (global_rsv, rsv, rsv->size,0 ))return trans;/* If not, commit and try again. */
@@ -5384,7 +5383,6 @@ void btrfs_evict_inode(struct inode *inode)struct btrfs_trans_handle *trans;struct btrfs_root *root =BTRFS_I (inode)->root;struct btrfs_block_rsv *rsv; -u64 min_size;int ret;trace_btrfs_inode_evict (inode);
@@ -5394,8 +5392,6 @@ void btrfs_evict_inode(struct inode *inode)return ; } - min_size =btrfs_calc_trunc_metadata_size (fs_info,1 ); -evict_inode_truncate_pages (inode); if (inode->i_nlink &&
@@ -5428,13 +5424,13 @@ void btrfs_evict_inode(struct inode *inode) rsv =btrfs_alloc_block_rsv (fs_info,BTRFS_BLOCK_RSV_TEMP );if (!rsv) goto no_delete; - rsv->size = min_size; + rsv->size =btrfs_calc_trunc_metadata_size (fs_info,1 ); rsv->failfast =1 ;btrfs_i_size_write (BTRFS_I (inode),0 );while (1 ) { - trans =evict_refill_and_join (root, rsv, min_size); + trans =evict_refill_and_join (root, rsv);if (IS_ERR (trans)) goto free_rsv;
@@ -5459,7 +5455,7 @@ void btrfs_evict_inode(struct inode *inode)* If it turns out that we are dropping too many of these, we might want * to add a mechanism for retrying these after a commit. */ - trans =evict_refill_and_join (root, rsv, min_size); + trans =evict_refill_and_join (root, rsv);if (!IS_ERR (trans)) { trans->block_rsv = rsv;btrfs_orphan_del (trans,BTRFS_I (inode));
Signature · 1 line
2.14.3
PATCH v4 41/42 btrfs: reserve extra space during evict()
In reply to Josef BacikWe could generate a lot of delayed refs in evict but never have any left over space from our block rsv to make up for that fact. So reserve some extra space and give it to the transaction so it can be used to refill the delayed refs rsv every loop through the truncate path.
--- fs/btrfs/inode.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +23 -2
@@ -5343,13 +5343,15 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, {struct btrfs_fs_info *fs_info = root->fs_info;struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; +u64 delayed_refs_extra =btrfs_calc_trans_metadata_size (fs_info,1 );int failures =0 ;for (;;) {struct btrfs_trans_handle *trans;int ret; - ret =btrfs_block_rsv_refill (root, rsv, rsv->size, + ret =btrfs_block_rsv_refill (root, rsv, + rsv->size + delayed_refs_extra,BTRFS_RESERVE_FLUSH_LIMIT );if (ret && ++failures >2 ) {
@@ -5358,9 +5360,28 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,return ERR_PTR (-ENOSPC ); } +/* +* Evict can generate a large amount of delayed refs without +* having a way to add space back since we exhaust our temporary +* block rsv. We aren't allowed to do FLUSH_ALL in this case +* because we could deadlock with so many things in the flushing +* code, so we have to try and hold some extra space to +* compensate for our delayed ref generation. If we can't get +* that space then we need see if we can steal our minimum from +* the global reserve. We will be ratelimited by the amount of +* space we have for the delayed refs rsv, so we'll end up +* committing and trying again. +*/ trans =btrfs_join_transaction (root); -if (IS_ERR (trans) || !ret) +if (IS_ERR (trans) || !ret) { +if (!IS_ERR (trans)) { + trans->block_rsv = &fs_info->trans_block_rsv; + trans->bytes_reserved = delayed_refs_extra; +btrfs_block_rsv_migrate (rsv, trans->block_rsv, + delayed_refs_extra,1 ); + }return trans; + }/* * Try to steal from the global reserve if there is space for
Signature · 1 line
2.14.3
PATCH v4 42/42 btrfs: don't run delayed_iputs in commit
In reply to Josef BacikThis could result in a really bad case where we do something like
evict
evict_refill_and_join
btrfs_commit_transaction
btrfs_run_delayed_iputs
evict
evict_refill_and_join
btrfs_commit_transaction
... forever
We have plenty of other places where we run delayed iputs that are much safer, let those do the work.
--- fs/btrfs/transaction.c | 9 --------- 1 file changed, 9 deletions(-)
fs/btrfs/transaction.c +0 -9
@@ -2265,15 +2265,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)kmem_cache_free (btrfs_trans_handle_cachep, trans); -/* -* If fs has been frozen, we can not handle delayed iputs, otherwise -* it'll result in deadlock about SB_FREEZE_FS. -*/ -if (current != fs_info->transaction_kthread && - current != fs_info->cleaner_kthread && - !test_bit (BTRFS_FS_FROZEN , &fs_info->flags)) -btrfs_run_delayed_iputs (fs_info); -return ret; scrub_continue:
Signature · 1 line
2.14.3
Re: [PATCH 42/42] btrfs: don't run delayed_iputs in commit
Quoted 12 lines · Josef Bacik
This could result in a really bad case where we do something like
evict
evict_refill_and_join
btrfs_commit_transaction
btrfs_run_delayed_iputs
evict
evict_refill_and_join
btrfs_commit_transaction
... forever
Quoted 4 lines
We have plenty of other places where we run delayed iputs that are much safer, let those do the work.
Signed-off-by Josef Bacik <[email protected]>
Great catch!
Quoted 27 lines
--- fs/btrfs/transaction.c | 9 --------- 1 file changed, 9 deletions(-)fs/btrfs/transaction.c +0
-10@@ -2265,15 +2265,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)kmem_cache_free (btrfs_trans_handle_cachep, trans); -/* -* If fs has been frozen, we can not handle delayed iputs, otherwise -* it'll result in deadlock about SB_FREEZE_FS. -*/ -if (current != fs_info->transaction_kthread && - current != fs_info->cleaner_kthread && - !test_bit (BTRFS_FS_FROZEN , &fs_info->flags)) -btrfs_run_delayed_iputs (fs_info); -return ret; scrub_continue: --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”