My current patch queue
Subsystems (2)
v3 cover My current patch queue
v2->v3:
- reworked the truncate/evict throttling, we were still occasionally hitting enospc aborts in production in these paths because we were too aggressive with space usage.
- reworked the delayed iput stuff to be a little less racey and less deadlocky.
- Addressed the comments from Dave and Omar.
- A lot of production testing.
v1->v2:
- addressed all of the issues brought up.
- added more comments.
- split up some patches.
original message:
This is the current queue of things that I've been working on. The main thing these patches are doing is separating out the delayed refs reservations from the global reserve into their own block rsv. We have been consistently hitting issues in production where we abort a transaction because we run out of the global reserve either while running delayed refs or while updating dirty block groups. This is because the math around global reserves is made up bullshit magic that has been tweaked more and more throughout the years. The result is something that is inconsistent across the board and sometimes wrong. So instead we need a way to know exactly how much space we need to keep around in order to satisfy our outstanding delayed refs and our dirty block groups.
Since we don't know how many delayed refs we need at the start of any modification we simply use the nr_items passed into btrfs_start_transaction() as a guess for what we may need. This has the side effect of putting more pressure on the ENOSPC system, but it's pressure we can deal with more intelligently because we always know how much space we have outstanding, instead of guessing with weird global reserve math.
This works similar to every other reservation we have, we reserve the worst case up front, and then at transaction end time we free up any space we didn't actually use for delayed refs.
My performance tests show that we are bit faster now since we can do more intelligent flushing and don't have to fall back on simply committing the transaction in hopes that we have enough space for everything we need to do.
That leads me to the 2nd part of this pull, there's a bunch of fixes around ENOSPC. Because we are a bit faster now there were a bunch of things uncovered in testing, but they seem to be all resolved now.
The final chunk of fixes are around transaction aborts. There were a lot of accounting bugs I was running into while running generic/435, so I fixed a bunch of those up so now it runs cleanly.
I have been running these patches through xfstests on multiple machines for a while, they are pretty solid and ready for wider testing and review. Thanks,
Josef
PATCH v3 1/42 btrfs: add btrfs_delete_ref_head helper
From: Josef Bacik <[email protected]>
We do this dance in cleanup_ref_head and check_ref_cleanup, unify it into a helper and cleanup the calling functions.
--- fs/btrfs/delayed-ref.c | 14 ++++++++++++++ fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/extent-tree.c | 22 +++------------------- 3 files changed, 19 insertions(+), 20 deletions(-)
fs/btrfs/delayed-ref.c +14 -0
@@ -393,6 +393,20 @@ btrfs_select_ref_head(struct btrfs_trans_handle *trans)return head; } +void btrfs_delete_ref_head (struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head) +{ +lockdep_assert_held (&delayed_refs->lock); +lockdep_assert_held (&head->lock); + +rb_erase (&head->href_node, &delayed_refs->href_root); +RB_CLEAR_NODE (&head->href_node); +atomic_dec (&delayed_refs->num_entries); + delayed_refs->num_heads--; +if (head->processing ==0 ) + delayed_refs->num_heads_ready--; +} +/* * Helper to insert the ref_node to the tail or merge with tail. *
fs/btrfs/delayed-ref.h +2 -1
@@ -261,7 +261,8 @@ static inline void btrfs_delayed_ref_unlock(struct btrfs_delayed_ref_head *head) {mutex_unlock (&head->mutex); } - +void btrfs_delete_ref_head (struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head);struct btrfs_delayed_ref_head *btrfs_select_ref_head (struct btrfs_trans_handle *trans);
fs/btrfs/extent-tree.c +3 -19
@@ -2492,12 +2492,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans,spin_unlock (&delayed_refs->lock);return 1 ; } - delayed_refs->num_heads--; -rb_erase (&head->href_node, &delayed_refs->href_root); -RB_CLEAR_NODE (&head->href_node); +btrfs_delete_ref_head (delayed_refs, head);spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock); -atomic_dec (&delayed_refs->num_entries);trace_run_delayed_ref_head (fs_info, head,0 );
@@ -6984,22 +6981,9 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (!mutex_trylock (&head->mutex)) goto out; -/* -* at this point we have a head with no other entries. Go -* ahead and process it. -*/ -rb_erase (&head->href_node, &delayed_refs->href_root); -RB_CLEAR_NODE (&head->href_node); -atomic_dec (&delayed_refs->num_entries); - -/* -* we don't take a ref on the node because we're removing it from the -* tree, so we just steal the ref the tree was holding. -*/ - delayed_refs->num_heads--; -if (head->processing ==0 ) - delayed_refs->num_heads_ready--; +btrfs_delete_ref_head (delayed_refs, head); head->processing =0 ; +spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock);
Signature · 1 line
2.14.3
PATCH v3 2/42 btrfs: add cleanup_ref_head_accounting helper
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We were missing some quota cleanups in check_ref_cleanup, so break the ref head accounting cleanup into a helper and call that from both check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that we don't screw up accounting in the future for other things that we add.
--- fs/btrfs/extent-tree.c | 67 +++++++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 28 deletions(-)
fs/btrfs/extent-tree.c +39 -28
@@ -2461,6 +2461,41 @@ static int cleanup_extent_op(struct btrfs_trans_handle *trans,return ret ? ret :1 ; } +static void cleanup_ref_head_accounting (struct btrfs_trans_handle *trans, +struct btrfs_delayed_ref_head *head) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; + +if (head->total_ref_mod <0 ) { +struct btrfs_space_info *space_info; +u64 flags; + +if (head->is_data) + flags =BTRFS_BLOCK_GROUP_DATA ; +else if (head->is_system) + flags =BTRFS_BLOCK_GROUP_SYSTEM ; +else + flags =BTRFS_BLOCK_GROUP_METADATA ; + space_info =__find_space_info (fs_info, flags); +ASSERT (space_info); +percpu_counter_add_batch (&space_info->total_bytes_pinned, + -head->num_bytes, +BTRFS_TOTAL_BYTES_PINNED_BATCH ); + +if (head->is_data) { +spin_lock (&delayed_refs->lock); + delayed_refs->pending_csums -= head->num_bytes; +spin_unlock (&delayed_refs->lock); + } + } + +/* Also free its reserved qgroup space */ +btrfs_qgroup_free_delayed_ref (fs_info, head->qgroup_ref_root, + head->qgroup_reserved); +} +static int cleanup_ref_head (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *head) {
@@ -2496,31 +2531,6 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans,spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock); -trace_run_delayed_ref_head (fs_info, head,0 ); - -if (head->total_ref_mod <0 ) { -struct btrfs_space_info *space_info; -u64 flags; - -if (head->is_data) - flags =BTRFS_BLOCK_GROUP_DATA ; -else if (head->is_system) - flags =BTRFS_BLOCK_GROUP_SYSTEM ; -else - flags =BTRFS_BLOCK_GROUP_METADATA ; - space_info =__find_space_info (fs_info, flags); -ASSERT (space_info); -percpu_counter_add_batch (&space_info->total_bytes_pinned, - -head->num_bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ); - -if (head->is_data) { -spin_lock (&delayed_refs->lock); - delayed_refs->pending_csums -= head->num_bytes; -spin_unlock (&delayed_refs->lock); - } - } -if (head->must_insert_reserved) {btrfs_pin_extent (fs_info, head->bytenr, head->num_bytes,1 );
@@ -2530,9 +2540,9 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } -/* Also free its reserved qgroup space */ -btrfs_qgroup_free_delayed_ref (fs_info, head->qgroup_ref_root, - head->qgroup_reserved); +cleanup_ref_head_accounting (trans, head); + +trace_run_delayed_ref_head (fs_info, head,0 );btrfs_delayed_ref_unlock (head);btrfs_put_delayed_ref_head (head);return 0 ;
@@ -6991,6 +7001,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (head->must_insert_reserved) ret =1 ; +cleanup_ref_head_accounting (trans, head);mutex_unlock (&head->mutex);btrfs_put_delayed_ref_head (head);return ret;
Signature · 1 line
2.14.3
PATCH v3 3/42 btrfs: cleanup extent_op handling
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
The cleanup_extent_op function actually would run the extent_op if it needed running, which made the name sort of a misnomer. Change it to run_and_cleanup_extent_op, and move the actual cleanup work to cleanup_extent_op so it can be used by check_ref_cleanup() in order to unify the extent op handling.
--- fs/btrfs/extent-tree.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-)
fs/btrfs/extent-tree.c +23 -13
@@ -2442,19 +2442,33 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_refbtrfs_delayed_ref_unlock (head); } -static int cleanup_extent_op (struct btrfs_trans_handle *trans, -struct btrfs_delayed_ref_head *head) +static struct btrfs_delayed_extent_op * +cleanup_extent_op (struct btrfs_trans_handle *trans, +struct btrfs_delayed_ref_head *head) {struct btrfs_delayed_extent_op *extent_op = head->extent_op; -int ret;if (!extent_op) -return 0 ; - head->extent_op =NULL ; +return NULL ; +if (head->must_insert_reserved) { + head->extent_op =NULL ;btrfs_free_delayed_extent_op (extent_op); -return 0 ; +return NULL ; } +return extent_op; +} + +static int run_and_cleanup_extent_op (struct btrfs_trans_handle *trans, +struct btrfs_delayed_ref_head *head) +{ +struct btrfs_delayed_extent_op *extent_op = +cleanup_extent_op (trans, head); +int ret; + +if (!extent_op) +return 0 ; + head->extent_op =NULL ;spin_unlock (&head->lock); ret =run_delayed_extent_op (trans, head, extent_op);btrfs_free_delayed_extent_op (extent_op);
@@ -2506,7 +2520,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, delayed_refs = &trans->transaction->delayed_refs; - ret =cleanup_extent_op (trans, head); + ret =run_and_cleanup_extent_op (trans, head);if (ret <0 ) {unselect_delayed_ref_head (delayed_refs, head);btrfs_debug (fs_info,"run_delayed_extent_op returned %d" , ret);
@@ -6977,12 +6991,8 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (!RB_EMPTY_ROOT (&head->ref_tree)) goto out; -if (head->extent_op) { -if (!head->must_insert_reserved) - goto out; -btrfs_free_delayed_extent_op (head->extent_op); - head->extent_op =NULL ; - } +if (cleanup_extent_op (trans, head) !=NULL ) + goto out;/* * waiting for the lock here would deadlock. If someone else has it
Signature · 1 line
2.14.3
PATCH v3 4/42 btrfs: only track ref_heads in delayed_ref_updates
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We use this number to figure out how many delayed refs to run, but __btrfs_run_delayed_refs really only checks every time we need a new delayed ref head, so we always run at least one ref head completely no matter what the number of items on it. Fix the accounting to only be adjusted when we add/remove a ref head.
--- fs/btrfs/delayed-ref.c | 3 --- 1 file changed, 3 deletions(-)
fs/btrfs/delayed-ref.c +0 -3
@@ -234,8 +234,6 @@ static inline void drop_delayed_ref(struct btrfs_trans_handle *trans, ref->in_tree =0 ;btrfs_put_delayed_ref (ref);atomic_dec (&delayed_refs->num_entries); -if (trans->delayed_ref_updates) - trans->delayed_ref_updates--; } static bool merge_ref(struct btrfs_trans_handle *trans,
@@ -460,7 +458,6 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,if (ref->action ==BTRFS_ADD_DELAYED_REF )list_add_tail (&ref->add_list, &href->ref_add_list);atomic_inc (&root->num_entries); - trans->delayed_ref_updates++;spin_unlock (&href->lock);return ret; }
Signature · 1 line
2.14.3
PATCH v3 5/42 btrfs: only count ref heads run in __btrfs_run_delayed_refs
In reply to Josef BacikWe pick the number of ref's to run based on the number of ref heads, and only make the decision to stop once we've processed entire ref heads, so only count the ref heads we've run and bail once we've hit the number of ref heads we wanted to process.
--- fs/btrfs/extent-tree.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
fs/btrfs/extent-tree.c +1 -4
@@ -2592,6 +2592,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,spin_unlock (&delayed_refs->lock);break ; } + count++;/* grab the lock that says we are going to process * all the refs for this head */
@@ -2605,7 +2606,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,*/ if (ret == -EAGAIN ) { locked_ref =NULL ; - count++;continue ; } }
@@ -2633,7 +2633,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,unselect_delayed_ref_head (delayed_refs, locked_ref); locked_ref =NULL ;cond_resched (); - count++;continue ; }
@@ -2651,7 +2650,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,return ret; } locked_ref =NULL ; - count++;continue ; }
@@ -2702,7 +2700,6 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, }btrfs_put_delayed_ref (ref); - count++;cond_resched (); }
Signature · 1 line
2.14.3
PATCH v3 6/42 btrfs: introduce delayed_refs_rsv
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong.
So instead give them their own block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism.
For now the delayed_refs_rsv will hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via the delayed_refs so that will stay there.
--- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-)
fs/btrfs/ctree.h +19 -8
@@ -452,8 +452,9 @@ struct btrfs_space_info {#define BTRFS_BLOCK_RSV_TRANS 3#define BTRFS_BLOCK_RSV_CHUNK 4#define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8struct btrfs_block_rsv {u64 size;
@@ -794,6 +795,8 @@ struct btrfs_fs_info {struct btrfs_block_rsv chunk_block_rsv;/* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; +/* block reservation for delayed refs */ +struct btrfs_block_rsv delayed_refs_rsv;struct btrfs_block_rsv empty_block_rsv;
@@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info,int btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans,struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);void btrfs_dec_block_group_reservations (struct btrfs_fs_info *fs_info,const u64 start);void btrfs_wait_block_group_reservations (struct btrfs_block_group_cache *bg);
@@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum {enum btrfs_flush_state {FLUSH_DELAYED_ITEMS_NR =1 ,FLUSH_DELAYED_ITEMS =2 , -FLUSH_DELALLOC =3 , -FLUSH_DELALLOC_WAIT =4 , -ALLOC_CHUNK =5 , -COMMIT_TRANS =6 , +FLUSH_DELAYED_REFS_NR =3 , +FLUSH_DELAYED_REFS =4 , +FLUSH_DELALLOC =5 , +FLUSH_DELALLOC_WAIT =6 , +ALLOC_CHUNK =7 , +COMMIT_TRANS =8 , };int btrfs_alloc_data_chunk_ondemand (struct btrfs_inode *inode,u64 bytes);
@@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,void btrfs_block_rsv_release (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *block_rsv,u64 num_bytes); +void btrfs_delayed_refs_rsv_release (struct btrfs_fs_info *fs_info,int nr); +void btrfs_update_delayed_refs_rsv (struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *src, +u64 num_bytes);int btrfs_inc_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_dec_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_put_block_group_cache (struct btrfs_fs_info *info);
fs/btrfs/delayed-ref.c +23 -5
@@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,* existing and update must have the same bytenr */ static noinline void -update_existing_head_ref (struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *existing,struct btrfs_delayed_ref_head *update,int *old_ref_mod_ret) { +struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; +struct btrfs_fs_info *fs_info = trans->fs_info;int old_ref_mod;BUG_ON (existing->is_data != update->is_data);
@@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs,* versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { -if (existing->total_ref_mod >=0 && old_ref_mod <0 ) +u64 csum_items = +btrfs_csum_bytes_to_leaves (fs_info, + existing->num_bytes); + +if (existing->total_ref_mod >=0 && old_ref_mod <0 ) { delayed_refs->pending_csums -= existing->num_bytes; -if (existing->total_ref_mod <0 && old_ref_mod >=0 ) +btrfs_delayed_refs_rsv_release (fs_info, csum_items); + } +if (existing->total_ref_mod <0 && old_ref_mod >=0 ) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } }spin_unlock (&existing->lock); }
@@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); -update_existing_head_ref (delayed_refs, existing, head_ref, +update_existing_head_ref (trans, existing, head_ref, old_ref_mod);/* * we've updated the existing ref, free the newly
@@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, }else {if (old_ref_mod) *old_ref_mod =0 ; -if (head_ref->is_data && head_ref->ref_mod <0 ) +if (head_ref->is_data && head_ref->ref_mod <0 ) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += +btrfs_csum_bytes_to_leaves (trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++;atomic_inc (&delayed_refs->num_entries);
@@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans); trace_add_delayed_tree_ref(fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ?
@@ -866,6 +882,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans); trace_add_delayed_data_ref(trans->fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ?
@@ -903,6 +920,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, NULL, NULL, NULL);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans);return 0 ; }
fs/btrfs/disk-io.c +4 -0
@@ -2692,6 +2692,9 @@ int open_ctree(struct super_block *sb,btrfs_init_block_rsv (&fs_info->empty_block_rsv,BTRFS_BLOCK_RSV_EMPTY );btrfs_init_block_rsv (&fs_info->delayed_block_rsv,BTRFS_BLOCK_RSV_DELOPS ); +btrfs_init_block_rsv (&fs_info->delayed_refs_rsv, +BTRFS_BLOCK_RSV_DELREFS ); +atomic_set (&fs_info->async_delalloc_pages,0 );atomic_set (&fs_info->defrag_running,0 );atomic_set (&fs_info->qgroup_op_seq,0 );
@@ -4419,6 +4422,7 @@ void btrfs_cleanup_dirty_bgs(struct btrfs_transaction *cur_trans,spin_unlock (&cur_trans->dirty_bgs_lock);btrfs_put_block_group (cache); +btrfs_delayed_refs_rsv_release (fs_info,1 );spin_lock (&cur_trans->dirty_bgs_lock); }spin_unlock (&cur_trans->dirty_bgs_lock);
fs/btrfs/extent-tree.c +228 -51
467 lines not on this page · raw message
fs/btrfs/inode.c +1 -1
@@ -5382,7 +5382,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,* Try to steal from the global reserve if there is space for * it. */ -if (!btrfs_check_space_for_delayed_refs (trans, fs_info) && +if (!btrfs_check_space_for_delayed_refs (fs_info) && !btrfs_block_rsv_migrate (global_rsv, rsv, min_size,0 ))return trans;
fs/btrfs/transaction.c +35 -42
140 lines not on this page · raw message
include/trace/events/btrfs.h +2 -0
@@ -1048,6 +1048,8 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_ITEMS, "FLUSH_DELAYED_ITEMS"}, \ { FLUSH_DELALLOC, "FLUSH_DELALLOC"}, \ { FLUSH_DELALLOC_WAIT, "FLUSH_DELALLOC_WAIT"}, \ + { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ + { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ { COMMIT_TRANS, "COMMIT_TRANS"})
Signature · 1 line
2.14.3
Re: [PATCH 06/42] btrfs: introduce delayed_refs_rsv
Quoted 166 lines · Josef Bacik
From: Josef Bacik <[email protected]>
Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a
global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong.So instead give them their own
block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism.For now the
delayed_refs_rsvwill hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via thedelayed_refsso that will stay there.Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-)fs/btrfs/ctree.h +19
-8@@ -452,8 +452,9 @@ struct btrfs_space_info {#define BTRFS_BLOCK_RSV_TRANS 3#define BTRFS_BLOCK_RSV_CHUNK 4#define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8struct btrfs_block_rsv {u64 size;@@ -794,6 +795,8 @@ struct btrfs_fs_info {struct btrfs_block_rsv chunk_block_rsv;/* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; +/* block reservation for delayed refs */ +struct btrfs_block_rsv delayed_refs_rsv;struct btrfs_block_rsv empty_block_rsv;@@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info,int btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans,struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);void btrfs_dec_block_group_reservations (struct btrfs_fs_info *fs_info,const u64 start);void btrfs_wait_block_group_reservations (struct btrfs_block_group_cache *bg);@@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum {enum btrfs_flush_state {FLUSH_DELAYED_ITEMS_NR =1 ,FLUSH_DELAYED_ITEMS =2 , -FLUSH_DELALLOC =3 , -FLUSH_DELALLOC_WAIT =4 , -ALLOC_CHUNK =5 , -COMMIT_TRANS =6 , +FLUSH_DELAYED_REFS_NR =3 , +FLUSH_DELAYED_REFS =4 , +FLUSH_DELALLOC =5 , +FLUSH_DELALLOC_WAIT =6 , +ALLOC_CHUNK =7 , +COMMIT_TRANS =8 , };int btrfs_alloc_data_chunk_ondemand (struct btrfs_inode *inode,u64 bytes);@@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,void btrfs_block_rsv_release (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *block_rsv,u64 num_bytes); +void btrfs_delayed_refs_rsv_release (struct btrfs_fs_info *fs_info,int nr); +void btrfs_update_delayed_refs_rsv (struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *src, +u64 num_bytes);int btrfs_inc_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_dec_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_put_block_group_cache (struct btrfs_fs_info *info);fs/btrfs/delayed-ref.c +20
-5@@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,* existing and update must have the same bytenr */ static noinline void -update_existing_head_ref (struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *existing,struct btrfs_delayed_ref_head *update,int *old_ref_mod_ret) { +struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; +struct btrfs_fs_info *fs_info = trans->fs_info;int old_ref_mod;BUG_ON (existing->is_data != update->is_data);@@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs,* versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { -if (existing->total_ref_mod >=0 && old_ref_mod <0 ) +u64 csum_items = +btrfs_csum_bytes_to_leaves (fs_info, + existing->num_bytes); + +if (existing->total_ref_mod >=0 && old_ref_mod <0 ) { delayed_refs->pending_csums -= existing->num_bytes; -if (existing->total_ref_mod <0 && old_ref_mod >=0 ) +btrfs_delayed_refs_rsv_release (fs_info, csum_items); + } +if (existing->total_ref_mod <0 && old_ref_mod >=0 ) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } }spin_unlock (&existing->lock); }@@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); -update_existing_head_ref (delayed_refs, existing, head_ref, +update_existing_head_ref (trans, existing, head_ref, old_ref_mod);/* * we've updated the existing ref, free the newly @@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, }else {if (old_ref_mod) *old_ref_mod =0 ; -if (head_ref->is_data && head_ref->ref_mod <0 ) +if (head_ref->is_data && head_ref->ref_mod <0 ) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += +btrfs_csum_bytes_to_leaves (trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++;atomic_inc (&delayed_refs->num_entries);
fs/btrfs/delayed-ref.c int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, · line 803
Quoted 4 lines
fs/btrfs/delayed-ref.c +1
-0@@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans);
You haven't adressed my initial point about merging modification of delayed_ref_updates and calling btrfs_update_delayed_refs_rsv into one function otherwise this seems error prone. I don't see why this cannot be made, if there is some reason which I'm missing then explain it.
As it stands this btrfs_updated_delayed_refs_rsv is paired with the modifications made in one of the 2nd level callees:
btrfs_add_delayed_tree_ref
add_delayed_ref_head
update_existing_head_ref
I'd rather have btrfs_update_delayed_refs_rsv renamed to something else with 'inc' in its name and called everytime we modify delayed_ref_update. I'm willing to bet 50 bucks in 6 months time someone will change delayed_ref_updates and will forget to call btrfs_update_delayed_refs_rsv.
WRT locking in update_existing_head_ref we are guaranteed to hold delayed_refs->lock, same thing in add_delayed_extent_op. The only places where we don't hold it is in the bg-related paths. But that's easily solvable by simplying breaking the function down into an internal helper doing the actual work with lockdep_assert_held(delayed_refs) at the top and a "public" api which will be taking the lock and calling the helper. WRT performance you will not be putting that much extra code in the critical section i.e the check + the arithmetic of btrfs_calc_trans_metadata_size.
Quoted 719 lines
fs/btrfs/delayed-ref.c +2
-0trace_add_delayed_tree_ref(fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ?@@ -866,6 +882,7 @@ int btrfs_add_delayed_data_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans); trace_add_delayed_data_ref(trans->fs_info, &ref->node, ref, action == BTRFS_ADD_DELAYED_EXTENT ?@@ -903,6 +920,7 @@ int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, NULL, NULL, NULL);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans);return 0 ; }fs/btrfs/disk-io.c +4
-0@@ -2692,6 +2692,9 @@ int open_ctree(struct super_block *sb,btrfs_init_block_rsv (&fs_info->empty_block_rsv,BTRFS_BLOCK_RSV_EMPTY );btrfs_init_block_rsv (&fs_info->delayed_block_rsv,BTRFS_BLOCK_RSV_DELOPS ); +btrfs_init_block_rsv (&fs_info->delayed_refs_rsv, +BTRFS_BLOCK_RSV_DELREFS ); +atomic_set (&fs_info->async_delalloc_pages,0 );atomic_set (&fs_info->defrag_running,0 );atomic_set (&fs_info->qgroup_op_seq,0 );@@ -4419,6 +4422,7 @@ void btrfs_cleanup_dirty_bgs(struct btrfs_transaction *cur_trans,spin_unlock (&cur_trans->dirty_bgs_lock);btrfs_put_block_group (cache); +btrfs_delayed_refs_rsv_release (fs_info,1 );spin_lock (&cur_trans->dirty_bgs_lock); }spin_unlock (&cur_trans->dirty_bgs_lock);fs/btrfs/extent-tree.c +228
-51@@ -2481,6 +2481,7 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans,struct btrfs_fs_info *fs_info = trans->fs_info;struct btrfs_delayed_ref_root *delayed_refs = &trans->transaction->delayed_refs; +int nr_items =1 ;if (head->total_ref_mod <0 ) {struct btrfs_space_info *space_info;@@ -2502,12 +2503,15 @@ static void cleanup_ref_head_accounting(struct btrfs_trans_handle *trans,spin_lock (&delayed_refs->lock); delayed_refs->pending_csums -= head->num_bytes;spin_unlock (&delayed_refs->lock); + nr_items +=btrfs_csum_bytes_to_leaves (fs_info, + head->num_bytes); } }/* Also free its reserved qgroup space */ btrfs_qgroup_free_delayed_ref (fs_info, head->qgroup_ref_root, head->qgroup_reserved); +btrfs_delayed_refs_rsv_release (fs_info, nr_items); } static int cleanup_ref_head(struct btrfs_trans_handle *trans,@@ -2802,40 +2806,22 @@ u64 btrfs_csum_bytes_to_leaves(struct btrfs_fs_info *fs_info, u64 csum_bytes)return num_csums; } -int btrfs_check_space_for_delayed_refs (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info) +bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info) { -struct btrfs_block_rsv *global_rsv; -u64 num_heads = trans->transaction->delayed_refs.num_heads_ready; -u64 csum_bytes = trans->transaction->delayed_refs.pending_csums; -unsigned int num_dirty_bgs = trans->transaction->num_dirty_bgs; -u64 num_bytes, num_dirty_bgs_bytes; -int ret =0 ; - - num_bytes =btrfs_calc_trans_metadata_size (fs_info,1 ); - num_heads =heads_to_leaves (fs_info, num_heads); -if (num_heads >1 ) - num_bytes += (num_heads -1 ) * fs_info->nodesize; - num_bytes <<=1 ; - num_bytes +=btrfs_csum_bytes_to_leaves (fs_info, csum_bytes) * - fs_info->nodesize; - num_dirty_bgs_bytes =btrfs_calc_trans_metadata_size (fs_info, - num_dirty_bgs); - global_rsv = &fs_info->global_block_rsv; - -/* -* If we can't allocate any more chunks lets make sure we have _lots_ of -* wiggle room since running delayed refs can create more delayed refs. -*/ -if (global_rsv->space_info->full) { - num_dirty_bgs_bytes <<=1 ; - num_bytes <<=1 ; - } +struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; +struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; +u64 reserved; +bool ret = false;spin_lock (&global_rsv->lock); -if (global_rsv->reserved <= num_bytes + num_dirty_bgs_bytes) - ret =1 ; + reserved = global_rsv->reserved;spin_unlock (&global_rsv->lock); + +spin_lock (&delayed_refs_rsv->lock); + reserved += delayed_refs_rsv->reserved; +if (delayed_refs_rsv->size >= reserved) + ret = true; +spin_unlock (&delayed_refs_rsv->lock);return ret; }@@ -2855,7 +2841,7 @@ int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans,if (val >=NSEC_PER_SEC /2 )return 2 ; -return btrfs_check_space_for_delayed_refs (trans, fs_info); +return btrfs_check_space_for_delayed_refs (fs_info) ?1 :0 ; }struct async_delayed_refs {@@ -3610,6 +3596,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans)*/ mutex_lock (&trans->transaction->cache_write_mutex);while (!list_empty (&dirty)) { +bool drop_reserve = true; + cache =list_first_entry (&dirty, struct btrfs_block_group_cache, dirty_list);@@ -3682,6 +3670,7 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans)list_add_tail (&cache->dirty_list, &cur_trans->dirty_bgs);btrfs_get_block_group (cache); + drop_reserve = false; }spin_unlock (&cur_trans->dirty_bgs_lock); }else if (ret) {@@ -3692,6 +3681,8 @@ int btrfs_start_dirty_block_groups(struct btrfs_trans_handle *trans)/* if its not on the io list, we need to put the block group */ if (should_put)btrfs_put_block_group (cache); +if (drop_reserve) +btrfs_delayed_refs_rsv_release (fs_info,1 );if (ret)break ;@@ -3840,6 +3831,7 @@ int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans,/* if its not on the io list, we need to put the block group */ if (should_put)btrfs_put_block_group (cache); +btrfs_delayed_refs_rsv_release (fs_info,1 );spin_lock (&cur_trans->dirty_bgs_lock); }spin_unlock (&cur_trans->dirty_bgs_lock);@@ -4816,8 +4808,10 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, {struct reserve_ticket *ticket =NULL ;struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv; +struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;struct btrfs_trans_handle *trans;u64 bytes; +u64 reclaim_bytes =0 ; trans = (struct btrfs_trans_handle *)current->journal_info;if (trans)@@ -4850,12 +4844,16 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,return -ENOSPC ;spin_lock (&delayed_rsv->lock); -if (delayed_rsv->size > bytes) - bytes =0 ; -else - bytes -= delayed_rsv->size; + reclaim_bytes += delayed_rsv->reserved;spin_unlock (&delayed_rsv->lock); +spin_lock (&delayed_refs_rsv->lock); + reclaim_bytes += delayed_refs_rsv->reserved; +spin_unlock (&delayed_refs_rsv->lock); +if (reclaim_bytes >= bytes) + goto commit; + bytes -= reclaim_bytes; +if (__percpu_counter_compare (&space_info->total_bytes_pinned, bytes,BTRFS_TOTAL_BYTES_PINNED_BATCH ) <0 ) {@@ -4905,6 +4903,20 @@ static void flush_space(struct btrfs_fs_info *fs_info,shrink_delalloc (fs_info, num_bytes *2 , num_bytes, state ==FLUSH_DELALLOC_WAIT );break ; +case FLUSH_DELAYED_REFS_NR : +case FLUSH_DELAYED_REFS : + trans =btrfs_join_transaction (root); +if (IS_ERR (trans)) { + ret =PTR_ERR (trans); +break ; + } +if (state ==FLUSH_DELAYED_REFS_NR ) + nr =calc_reclaim_items_nr (fs_info, num_bytes); +else + nr =0 ; +btrfs_run_delayed_refs (trans, nr); +btrfs_end_transaction (trans); +break ;case ALLOC_CHUNK : trans =btrfs_join_transaction (root);if (IS_ERR (trans)) {@@ -5377,6 +5389,91 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,return 0 ; } +/** +* btrfs_migrate_to_delayed_refs_rsv - transfer bytes to our delayed refs rsv. +* @fs_info - the fs info for our fs. +* @src - the source block rsv to transfer from. +* @num_bytes - the number of bytes to transfer. +* +* This transfers up to the num_bytes amount from the src rsv to the +* delayed_refs_rsv. Any extra bytes are returned to the space info. +*/ +void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *src, +u64 num_bytes) +{ +struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv; +u64 to_free =0 ; + +spin_lock (&src->lock); + src->reserved -= num_bytes; + src->size -= num_bytes; +spin_unlock (&src->lock); + +spin_lock (&delayed_refs_rsv->lock); +if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) { +u64 delta = delayed_refs_rsv->size - + delayed_refs_rsv->reserved; +if (num_bytes > delta) { + to_free = num_bytes - delta; + num_bytes = delta; + } + }else { + to_free = num_bytes; + num_bytes =0 ; + } + +if (num_bytes) + delayed_refs_rsv->reserved += num_bytes; +if (delayed_refs_rsv->reserved >= delayed_refs_rsv->size) + delayed_refs_rsv->full =1 ; +spin_unlock (&delayed_refs_rsv->lock); + +if (num_bytes) +trace_btrfs_space_reservation (fs_info,"delayed_refs_rsv" , +0 , num_bytes,1 ); +if (to_free) +space_info_add_old_bytes (fs_info, delayed_refs_rsv->space_info, + to_free); +} + +/** +* btrfs_throttle_delayed_refs - throttle based on our delayed refs usage. +* @fs_info - the fs_info for our fs. +* @flush - control how we can flush for this reservation. +* +* This will refill the delayed block_rsv up to 1 items size worth of space and +* will return -ENOSPC if we can't make the reservation. +*/ +int btrfs_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +enum btrfs_reserve_flush_enum flush) +{ +struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv; +u64 limit =btrfs_calc_trans_metadata_size (fs_info,1 ); +u64 num_bytes =0 ; +int ret = -ENOSPC ; + +spin_lock (&block_rsv->lock); +if (block_rsv->reserved < block_rsv->size) { + num_bytes = block_rsv->size - block_rsv->reserved; + num_bytes =min (num_bytes, limit); + } +spin_unlock (&block_rsv->lock); + +if (!num_bytes) +return 0 ; + + ret =reserve_metadata_bytes (fs_info->extent_root, block_rsv, + num_bytes, flush); +if (ret) +return ret; +block_rsv_add_bytes (block_rsv, num_bytes,0 ); +trace_btrfs_space_reservation (fs_info,"delayed_refs_rsv" , +0 , num_bytes,1 ); +return 0 ; +} + +/* * This is for space we already have accounted in space_info->bytes_may_use, so * basically when we're returning space from block_rsv's. @@ -5699,6 +5796,31 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode,return ret; } +static u64 __btrfs_block_rsv_release (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *block_rsv, +u64 num_bytes,u64 *qgroup_to_release) +{ +struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; +struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; +struct btrfs_block_rsv *target = delayed_rsv; + +if (target->full || target == block_rsv) + target = global_rsv; + +if (block_rsv->space_info != target->space_info) + target =NULL ; + +return block_rsv_release_bytes (fs_info, block_rsv, target, num_bytes, + qgroup_to_release); +} + +void btrfs_block_rsv_release (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *block_rsv, +u64 num_bytes) +{ +__btrfs_block_rsv_release (fs_info, block_rsv, num_bytes,NULL ); +} +/** * btrfs_inode_rsv_release - release any excessive reservation. * @inode - the inode we need to release from. @@ -5713,7 +5835,6 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode,static void btrfs_inode_rsv_release (struct btrfs_inode *inode,bool qgroup_free) {struct btrfs_fs_info *fs_info = inode->root->fs_info; -struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;struct btrfs_block_rsv *block_rsv = &inode->block_rsv;u64 released =0 ;u64 qgroup_to_release =0 ;@@ -5723,8 +5844,8 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free)* are releasing 0 bytes, and then we'll just get the reservation over * the size free'd. */ - released =block_rsv_release_bytes (fs_info, block_rsv, global_rsv,0 , - &qgroup_to_release); + released =__btrfs_block_rsv_release (fs_info, block_rsv,0 , + &qgroup_to_release);if (released >0 )trace_btrfs_space_reservation (fs_info,"delalloc" ,btrfs_ino (inode), released,0 );@@ -5735,16 +5856,26 @@ static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free) qgroup_to_release); } -void btrfs_block_rsv_release (struct btrfs_fs_info *fs_info, -struct btrfs_block_rsv *block_rsv, -u64 num_bytes) +/** +* btrfs_delayed_refs_rsv_release - release a ref head's reservation. +* @fs_info - the fs_info for our fs. +* @nr - the number of items to drop. +* +* This drops the delayed ref head's count from the delayed refs rsv and free's +* any excess reservation we had. +*/ +void btrfs_delayed_refs_rsv_release (struct btrfs_fs_info *fs_info,int nr) { +struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv;struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; +u64 num_bytes =btrfs_calc_trans_metadata_size (fs_info, nr); +u64 released =0 ; -if (global_rsv == block_rsv || - block_rsv->space_info != global_rsv->space_info) - global_rsv =NULL ; -block_rsv_release_bytes (fs_info, block_rsv, global_rsv, num_bytes,NULL ); + released =block_rsv_release_bytes (fs_info, block_rsv, global_rsv, + num_bytes,NULL ); +if (released) +trace_btrfs_space_reservation (fs_info,"delayed_refs_rsv" , +0 , released,0 ); }static void update_global_block_rsv (struct btrfs_fs_info *fs_info)@@ -5809,9 +5940,10 @@ static void init_global_block_rsv(struct btrfs_fs_info *fs_info) fs_info->trans_block_rsv.space_info = space_info; fs_info->empty_block_rsv.space_info = space_info; fs_info->delayed_block_rsv.space_info = space_info; + fs_info->delayed_refs_rsv.space_info = space_info; - fs_info->extent_root->block_rsv = &fs_info->global_block_rsv; - fs_info->csum_root->block_rsv = &fs_info->global_block_rsv; + fs_info->extent_root->block_rsv = &fs_info->delayed_refs_rsv; + fs_info->csum_root->block_rsv = &fs_info->delayed_refs_rsv; fs_info->dev_root->block_rsv = &fs_info->global_block_rsv; fs_info->tree_root->block_rsv = &fs_info->global_block_rsv;if (fs_info->quota_root)@@ -5831,8 +5963,34 @@ static void release_global_block_rsv(struct btrfs_fs_info *fs_info)WARN_ON (fs_info->chunk_block_rsv.reserved >0 );WARN_ON (fs_info->delayed_block_rsv.size >0 );WARN_ON (fs_info->delayed_block_rsv.reserved >0 ); +WARN_ON (fs_info->delayed_refs_rsv.reserved >0 ); +WARN_ON (fs_info->delayed_refs_rsv.size >0 ); } +/* +* btrfs_update_delayed_refs_rsv - adjust the size of the delayed refs rsv +* @trans - the trans that may have generated delayed refs +* +* This is to be called anytime we may have adjusted trans->delayed_ref_updates, +* it'll calculate the additional size and add it to the delayed_refs_rsv. +*/ +void btrfs_update_delayed_refs_rsv (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv; +u64 num_bytes; + +if (!trans->delayed_ref_updates) +return ; + + num_bytes =btrfs_calc_trans_metadata_size (fs_info, + trans->delayed_ref_updates); +spin_lock (&delayed_rsv->lock); + delayed_rsv->size += num_bytes; + delayed_rsv->full =0 ; +spin_unlock (&delayed_rsv->lock); + trans->delayed_ref_updates =0 ; +}/* * To be called after all the new block groups attached to the transaction @@ -6126,6 +6284,7 @@ static int update_block_group(struct btrfs_trans_handle *trans,u64 old_val;u64 byte_in_group;int factor; +int ret =0 ;/* block accounting for super block */ spin_lock (&info->delalloc_root_lock);@@ -6139,8 +6298,10 @@ static int update_block_group(struct btrfs_trans_handle *trans,while (total) { cache =btrfs_lookup_block_group (info, bytenr); -if (!cache) -return -ENOENT ; +if (!cache) { + ret = -ENOENT ; +break ; + } factor =btrfs_bg_type_to_factor (cache->flags);/* @@ -6199,6 +6360,7 @@ static int update_block_group(struct btrfs_trans_handle *trans,list_add_tail (&cache->dirty_list, &trans->transaction->dirty_bgs); trans->transaction->num_dirty_bgs++; + trans->delayed_ref_updates++;btrfs_get_block_group (cache); }spin_unlock (&trans->transaction->dirty_bgs_lock);@@ -6216,7 +6378,10 @@ static int update_block_group(struct btrfs_trans_handle *trans, total -= num_bytes; bytenr += num_bytes; } -return 0 ; + +/* Modified block groups are accounted for in the delayed_refs_rsv. */ +btrfs_update_delayed_refs_rsv (trans); +return ret; }static u64 first_logical_byte (struct btrfs_fs_info *fs_info,u64 search_start)@@ -8230,7 +8395,12 @@ use_block_rsv(struct btrfs_trans_handle *trans, goto again; } -if (btrfs_test_opt (fs_info,ENOSPC_DEBUG )) { +/* +* The global reserve still exists to save us from ourselves, so don't +* warn_on if we are short on our delayed refs reserve. +*/ +if (block_rsv->type !=BTRFS_BLOCK_RSV_DELREFS && +btrfs_test_opt (fs_info,ENOSPC_DEBUG )) { static DEFINE_RATELIMIT_STATE(_rs, DEFAULT_RATELIMIT_INTERVAL * 10, /*DEFAULT_RATELIMIT_BURST*/ 1);@@ -10146,6 +10316,7 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)add_block_group_free_space (trans, block_group);/* already aborted the transaction if it failed. */ next: +btrfs_delayed_refs_rsv_release (fs_info,1 );list_del_init (&block_group->bg_list); } trans->can_flush_pending_bgs = can_flush_pending_bgs;@@ -10223,6 +10394,8 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,link_block_group (cache);list_add_tail (&cache->bg_list, &trans->new_bgs); + trans->delayed_ref_updates++; +btrfs_update_delayed_refs_rsv (trans);set_avail_alloc_bits (fs_info, type);return 0 ;@@ -10260,6 +10433,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,int factor;struct btrfs_caching_control *caching_ctl =NULL ;bool remove_em; +bool remove_rsv = false; block_group =btrfs_lookup_block_group (fs_info, group_start);BUG_ON (!block_group);@@ -10324,6 +10498,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,if (!list_empty (&block_group->dirty_list)) {list_del_init (&block_group->dirty_list); + remove_rsv = true;btrfs_put_block_group (block_group); }spin_unlock (&trans->transaction->dirty_bgs_lock);@@ -10533,6 +10708,8 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans, ret =btrfs_del_item (trans, root, path); out: +if (remove_rsv) +btrfs_delayed_refs_rsv_release (fs_info,1 );btrfs_free_path (path);return ret; }fs/btrfs/inode.c +1
-1@@ -5382,7 +5382,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,* Try to steal from the global reserve if there is space for * it. */ -if (!btrfs_check_space_for_delayed_refs (trans, fs_info) && +if (!btrfs_check_space_for_delayed_refs (fs_info) && !btrfs_block_rsv_migrate (global_rsv, rsv, min_size,0 ))return trans;fs/btrfs/transaction.c +35
-42@@ -455,7 +455,7 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, bool enforce_qgroups) {struct btrfs_fs_info *fs_info = root->fs_info; - +struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;struct btrfs_trans_handle *h;struct btrfs_transaction *cur_trans;u64 num_bytes =0 ;@@ -484,13 +484,28 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,* the appropriate flushing if need be. */ if (num_items && root != fs_info->chunk_root) { +struct btrfs_block_rsv *rsv = &fs_info->trans_block_rsv; +u64 delayed_refs_bytes =0 ; + qgroup_reserved = num_items * fs_info->nodesize; ret =btrfs_qgroup_reserve_meta_pertrans (root, qgroup_reserved, enforce_qgroups);if (ret)return ERR_PTR (ret); +/* +* We want to reserve all the bytes we may need all at once, so +* we only do 1 enospc flushing cycle per transaction start. We +* accomplish this by simply assuming we'll do 2 x num_items +* worth of delayed refs updates in this trans handle, and +* refill that amount for whatever is missing in the reserve. +*/ num_bytes =btrfs_calc_trans_metadata_size (fs_info, num_items); +if (delayed_refs_rsv->full ==0 ) { + delayed_refs_bytes = num_bytes; + num_bytes <<=1 ; + } +/* * Do the reservation for the relocation root creation */ @@ -499,8 +514,24 @@ start_transaction(struct btrfs_root *root, unsigned int num_items, reloc_reserved = true; } - ret =btrfs_block_rsv_add (root, &fs_info->trans_block_rsv, - num_bytes, flush); + ret =btrfs_block_rsv_add (root, rsv, num_bytes, flush); +if (ret) + goto reserve_fail; +if (delayed_refs_bytes) { +btrfs_migrate_to_delayed_refs_rsv (fs_info, rsv, + delayed_refs_bytes); + num_bytes -= delayed_refs_bytes; + } + }else if (num_items ==0 && flush ==BTRFS_RESERVE_FLUSH_ALL && + !delayed_refs_rsv->full) { +/* +* Some people call with btrfs_start_transaction(root, 0) +* because they can be throttled, but have some other mechanism +* for reserving space. We still want these guys to refill the +* delayed block_rsv so just add 1 items worth of reservation +* here. +*/ + ret =btrfs_throttle_delayed_refs (fs_info, flush);if (ret) goto reserve_fail; }@@ -759,7 +790,7 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) {struct btrfs_fs_info *fs_info = trans->fs_info; -if (btrfs_check_space_for_delayed_refs (trans, fs_info)) +if (btrfs_check_space_for_delayed_refs (fs_info))return 1 ;return !!btrfs_block_rsv_check (&fs_info->global_block_rsv,5 );@@ -768,22 +799,12 @@ static int should_end_transaction(struct btrfs_trans_handle *trans)int btrfs_should_end_transaction (struct btrfs_trans_handle *trans) {struct btrfs_transaction *cur_trans = trans->transaction; -int updates; -int err;smp_mb ();if (cur_trans->state >=TRANS_STATE_BLOCKED || cur_trans->delayed_refs.flushing)return 1 ; - updates = trans->delayed_ref_updates; - trans->delayed_ref_updates =0 ; -if (updates) { - err =btrfs_run_delayed_refs (trans, updates *2 ); -if (err)/* Error code will also eval true */ -return err; - } -return should_end_transaction (trans); }@@ -813,11 +834,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, {struct btrfs_fs_info *info = trans->fs_info;struct btrfs_transaction *cur_trans = trans->transaction; -u64 transid = trans->transid; -unsigned long cur = trans->delayed_ref_updates;int lock = (trans->type !=TRANS_JOIN_NOLOCK );int err =0 ; -int must_run_delayed_refs =0 ;if (refcount_read (&trans->use_count) >1 ) {refcount_dec (&trans->use_count);@@ -828,27 +846,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,btrfs_trans_release_metadata (trans); trans->block_rsv =NULL ; -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); - - trans->delayed_ref_updates =0 ; -if (!trans->sync) { - must_run_delayed_refs = -btrfs_should_throttle_delayed_refs (trans, info); - cur = max_t(unsigned long, cur, 32); - -/* -* don't make the caller wait if they are from a NOLOCK -* or ATTACH transaction, it will deadlock with commit -*/ -if (must_run_delayed_refs ==1 && - (trans->type & (__TRANS_JOIN_NOLOCK | __TRANS_ATTACH))) - must_run_delayed_refs =2 ; - } - -btrfs_trans_release_metadata (trans); - trans->block_rsv =NULL ; -if (!list_empty (&trans->new_bgs))btrfs_create_pending_block_groups (trans);@@ -893,10 +890,6 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, }kmem_cache_free (btrfs_trans_handle_cachep, trans); -if (must_run_delayed_refs) { -btrfs_async_run_delayed_refs (info, cur, transid, - must_run_delayed_refs ==1 ); - }return err; }include/trace/events/btrfs.h +2
-0@@ -1048,6 +1048,8 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_ITEMS, "FLUSH_DELAYED_ITEMS"}, \ { FLUSH_DELALLOC, "FLUSH_DELALLOC"}, \ { FLUSH_DELALLOC_WAIT, "FLUSH_DELALLOC_WAIT"}, \ + { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ + { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ { COMMIT_TRANS, "COMMIT_TRANS"})
Re: [PATCH 06/42] btrfs: introduce delayed_refs_rsv
Quoted 186 lines · Nikolay Borisov
Quoted 166 lines · Josef Bacik
From: Josef Bacik <[email protected]>
Traditionally we've had voodoo in btrfs to account for the space that delayed refs may take up by having a
global_block_rsv. This works most of the time, except when it doesn't. We've had issues reported and seen in production where sometimes the global reserve is exhausted during transaction commit before we can run all of our delayed refs, resulting in an aborted transaction. Because of this voodoo we have equally dubious flushing semantics around throttling delayed refs which we often get wrong.So instead give them their own
block_rsv. This way we can always know exactly how much outstanding space we need for delayed refs. This allows us to make sure we are constantly filling that reservation up with space, and allows us to put more precise pressure on the enospc system. Instead of doing math to see if its a good time to throttle, the normal enospc code will be invoked if we have a lot of delayed refs pending, and they will be run via the normal flushing mechanism.For now the
delayed_refs_rsvwill hold the reservations for the delayed refs, the block group updates, and deleting csums. We could have a separate rsv for the block group updates, but the csum deletion stuff is still handled via thedelayed_refsso that will stay there.Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/ctree.h | 27 +++-- fs/btrfs/delayed-ref.c | 28 ++++- fs/btrfs/disk-io.c | 4 + fs/btrfs/extent-tree.c | 279 +++++++++++++++++++++++++++++++++++-------- fs/btrfs/inode.c | 2 +- fs/btrfs/transaction.c | 77 ++++++------ include/trace/events/btrfs.h | 2 + 7 files changed, 312 insertions(+), 107 deletions(-)fs/btrfs/ctree.h +19
-8@@ -452,8 +452,9 @@ struct btrfs_space_info {#define BTRFS_BLOCK_RSV_TRANS 3#define BTRFS_BLOCK_RSV_CHUNK 4#define BTRFS_BLOCK_RSV_DELOPS 5 -#define BTRFS_BLOCK_RSV_EMPTY 6 -#define BTRFS_BLOCK_RSV_TEMP 7 +#define BTRFS_BLOCK_RSV_DELREFS 6 +#define BTRFS_BLOCK_RSV_EMPTY 7 +#define BTRFS_BLOCK_RSV_TEMP 8struct btrfs_block_rsv {u64 size;@@ -794,6 +795,8 @@ struct btrfs_fs_info {struct btrfs_block_rsv chunk_block_rsv;/* block reservation for delayed operations */ struct btrfs_block_rsv delayed_block_rsv; +/* block reservation for delayed refs */ +struct btrfs_block_rsv delayed_refs_rsv;struct btrfs_block_rsv empty_block_rsv;@@ -2608,8 +2611,7 @@ static inline u64 btrfs_calc_trunc_metadata_size(struct btrfs_fs_info *fs_info,int btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans,struct btrfs_fs_info *fs_info); -int btrfs_check_space_for_delayed_refs (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info); +bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);void btrfs_dec_block_group_reservations (struct btrfs_fs_info *fs_info,const u64 start);void btrfs_wait_block_group_reservations (struct btrfs_block_group_cache *bg);@@ -2723,10 +2725,12 @@ enum btrfs_reserve_flush_enum {enum btrfs_flush_state {FLUSH_DELAYED_ITEMS_NR =1 ,FLUSH_DELAYED_ITEMS =2 , -FLUSH_DELALLOC =3 , -FLUSH_DELALLOC_WAIT =4 , -ALLOC_CHUNK =5 , -COMMIT_TRANS =6 , +FLUSH_DELAYED_REFS_NR =3 , +FLUSH_DELAYED_REFS =4 , +FLUSH_DELALLOC =5 , +FLUSH_DELALLOC_WAIT =6 , +ALLOC_CHUNK =7 , +COMMIT_TRANS =8 , };int btrfs_alloc_data_chunk_ondemand (struct btrfs_inode *inode,u64 bytes);@@ -2777,6 +2781,13 @@ int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,void btrfs_block_rsv_release (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *block_rsv,u64 num_bytes); +void btrfs_delayed_refs_rsv_release (struct btrfs_fs_info *fs_info,int nr); +void btrfs_update_delayed_refs_rsv (struct btrfs_trans_handle *trans); +int btrfs_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +enum btrfs_reserve_flush_enum flush); +void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info, +struct btrfs_block_rsv *src, +u64 num_bytes);int btrfs_inc_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_dec_block_group_ro (struct btrfs_block_group_cache *cache);void btrfs_put_block_group_cache (struct btrfs_fs_info *info);fs/btrfs/delayed-ref.c +20
-5@@ -467,11 +467,14 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,* existing and update must have the same bytenr */ static noinline void -update_existing_head_ref (struct btrfs_delayed_ref_root *delayed_refs, +update_existing_head_ref (struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *existing,struct btrfs_delayed_ref_head *update,int *old_ref_mod_ret) { +struct btrfs_delayed_ref_root *delayed_refs = + &trans->transaction->delayed_refs; +struct btrfs_fs_info *fs_info = trans->fs_info;int old_ref_mod;BUG_ON (existing->is_data != update->is_data);@@ -529,10 +532,18 @@ update_existing_head_ref(struct btrfs_delayed_ref_root *delayed_refs,* versa we need to make sure to adjust pending_csums accordingly. */ if (existing->is_data) { -if (existing->total_ref_mod >=0 && old_ref_mod <0 ) +u64 csum_items = +btrfs_csum_bytes_to_leaves (fs_info, + existing->num_bytes); + +if (existing->total_ref_mod >=0 && old_ref_mod <0 ) { delayed_refs->pending_csums -= existing->num_bytes; -if (existing->total_ref_mod <0 && old_ref_mod >=0 ) +btrfs_delayed_refs_rsv_release (fs_info, csum_items); + } +if (existing->total_ref_mod <0 && old_ref_mod >=0 ) { delayed_refs->pending_csums += existing->num_bytes; + trans->delayed_ref_updates += csum_items; + } }spin_unlock (&existing->lock); }@@ -638,7 +649,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, && head_ref->qgroup_reserved && existing->qgroup_ref_root && existing->qgroup_reserved); -update_existing_head_ref (delayed_refs, existing, head_ref, +update_existing_head_ref (trans, existing, head_ref, old_ref_mod);/* * we've updated the existing ref, free the newly @@ -649,8 +660,12 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, }else {if (old_ref_mod) *old_ref_mod =0 ; -if (head_ref->is_data && head_ref->ref_mod <0 ) +if (head_ref->is_data && head_ref->ref_mod <0 ) { delayed_refs->pending_csums += head_ref->num_bytes; + trans->delayed_ref_updates += +btrfs_csum_bytes_to_leaves (trans->fs_info, + head_ref->num_bytes); + } delayed_refs->num_heads++; delayed_refs->num_heads_ready++;atomic_inc (&delayed_refs->num_entries);fs/btrfs/delayed-ref.c int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, · line 803
Quoted 4 lines
fs/btrfs/delayed-ref.c +1
-0@@ -785,6 +800,7 @@ int btrfs_add_delayed_tree_ref(struct btrfs_trans_handle *trans, ret =insert_delayed_ref (trans, delayed_refs, head_ref, &ref->node);spin_unlock (&delayed_refs->lock); +btrfs_update_delayed_refs_rsv (trans);You haven't adressed my initial point about merging modification of
delayed_ref_updatesand callingbtrfs_update_delayed_refs_rsvinto one function otherwise this seems error prone. I don't see why this cannot be made, if there is some reason which I'm missing then explain it.As it stands this
btrfs_updated_delayed_refs_rsvis paired with the modifications made in one of the 2nd level callees:
btrfs_add_delayed_tree_ref
add_delayed_ref_head
update_existing_head_ref
Quoted 6 lines
I'd rather have
btrfs_update_delayed_refs_rsvrenamed to something else with 'inc' in its name and called everytime we modifydelayed_ref_update. I'm willing to bet 50 bucks in 6 months time someone will changedelayed_ref_updatesand will forget to callbtrfs_update_delayed_refs_rsv.
Because we have to take the delayed_refs_rsv lock in this helper, I want to take it as little as possible since it is a fs wide lock, so I want to batch it. There's no reason to change it. Thanks,
Josef
PATCH v3 7/42 btrfs: check if free bgs for commit
In reply to Josef Bacikmay_commit_transaction will skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space.
--- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-)
fs/btrfs/extent-tree.c +19 -14
@@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ; -/* See if there is enough pinned space to make this reservation */ -if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) + trans =btrfs_join_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +return -ENOSPC ; + +/* +* See if there is enough pinned space to make this reservation, or if +* we have bg's that are going to be freed, allowing us to possibly do a +* chunk allocation the next loop through. +*/ +if (test_bit (BTRFS_TRANS_HAVE_FREE_BGS , &trans->transaction->flags) || +__percpu_counter_compare (&space_info->total_bytes_pinned, bytes, +BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) goto commit;/*
@@ -4841,7 +4849,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,* this reservation. */ if (space_info != delayed_rsv->space_info) -return -ENOSPC ; + goto enospc;spin_lock (&delayed_rsv->lock); reclaim_bytes += delayed_rsv->reserved;
@@ -4855,17 +4863,14 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info, bytes -= reclaim_bytes;if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) <0 ) { -return -ENOSPC ; - } - + bytes, +BTRFS_TOTAL_BYTES_PINNED_BATCH ) <0 ) + goto enospc; commit: - trans =btrfs_join_transaction (fs_info->extent_root); -if (IS_ERR (trans)) -return -ENOSPC ; -return btrfs_commit_transaction (trans); +enospc: +btrfs_end_transaction (trans); +return -ENOSPC ; }/*
Signature · 1 line
2.14.3
Re: [PATCH 07/42] btrfs: check if free bgs for commit
Quoted 19 lines · Josef Bacik
may_commit_transactionwill skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space.Signed-off-by Josef Bacik <[email protected]>Reviewed-by Omar Sandoval <[email protected]>--- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-)fs/btrfs/extent-tree.c +0
-0
fs/btrfs/extent-tree.c static int may_commit_transaction(struct btrfs_fs_info *fs_info, · line 4835
Quoted 10 lines
fs/btrfs/extent-tree.c +3
-4@@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ; -/* See if there is enough pinned space to make this reservation */ -if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) + trans =btrfs_join_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +return -ENOSPC ;
Why do you set override the error to ENOSPC, instead of passing PTR_ERR(trans)? There are more reasons why btrfs_join_transaction could
fail: EROFS after error, EDQUOT when quotas say no, ENOMEM from ulist_add, that's just a sample I quickly found.
Re: [PATCH 07/42] btrfs: check if free bgs for commit
Quoted 23 lines · David Sterba
Quoted 22 lines · Josef Bacik
may_commit_transactionwill skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space.Signed-off-by Josef Bacik <[email protected]>Reviewed-by Omar Sandoval <[email protected]>--- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-)fs/btrfs/extent-tree.c +0
-0@@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ;
Quoted 12 lines
fs/btrfs/extent-tree.c static int may_commit_transaction(struct btrfs_fs_info *fs_info, · line 4835
Quoted 7 lines
fs/btrfs/extent-tree.c +3
-4-/* See if there is enough pinned space to make this reservation */ -if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) + trans =btrfs_join_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +return -ENOSPC ;Why do you set override the error to ENOSPC, instead of passing
PTR_ERR(trans)? There are more reasons whybtrfs_join_transactioncould
fail: EROFS after error, EDQUOT when quotas say no, ENOMEM fromulist_add, that's just a sample I quickly found.
(argh I hit r instead of g, sending again)
Because may_commit_transaction is only called during flushing, we don't actually care about the value, just that it failed. Thanks,
Josef
Re: [PATCH 07/42] btrfs: check if free bgs for commit
Quoted 37 lines · Josef Bacik
Quoted 23 lines · David Sterba
Quoted 22 lines · Josef Bacik
may_commit_transactionwill skip committing the transaction if we don't have enough pinned space or if we're trying to find space for a SYSTEM chunk. However if we have pending free block groups in this transaction we still want to commit as we may be able to allocate a chunk to make our reservation. So instead of just returning ENOSPC, check if we have free block groups pending, and if so commit the transaction to allow us to use that free space.Signed-off-by Josef Bacik <[email protected]>Reviewed-by Omar Sandoval <[email protected]>--- fs/btrfs/extent-tree.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-)fs/btrfs/extent-tree.c +0
-0@@ -4830,10 +4830,18 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ;Quoted 12 lines
fs/btrfs/extent-tree.c static int may_commit_transaction(struct btrfs_fs_info *fs_info, · line 4835
Quoted 7 lines
fs/btrfs/extent-tree.c +3
-4-/* See if there is enough pinned space to make this reservation */ -if (__percpu_counter_compare (&space_info->total_bytes_pinned, - bytes, -BTRFS_TOTAL_BYTES_PINNED_BATCH ) >=0 ) + trans =btrfs_join_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +return -ENOSPC ;Why do you set override the error to ENOSPC, instead of passing
PTR_ERR(trans)? There are more reasons whybtrfs_join_transactioncould
fail: EROFS after error, EDQUOT when quotas say no, ENOMEM fromulist_add, that's just a sample I quickly found.
Quoted 4 lines
(argh I hit r instead of g, sending again)
Because
may_commit_transactionis only called during flushing, we don't actually care about the value, just that it failed. Thanks,
No, we do care about error values. If anything, here it's used in the tracepoint as the reason why the flushing failed. Conflating them into one without a good reason is a bad practice I don't want to let spread in the code.
PATCH v3 8/42 btrfs: dump block_rsv whe dumping space info
In reply to Josef BacikFor enospc_debug having the block rsvs is super helpful to see if we've done something wrong.
--- fs/btrfs/extent-tree.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
fs/btrfs/extent-tree.c +15 -0
@@ -7918,6 +7918,15 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,return ret; } +#define DUMP_BLOCK_RSV (fs_info, rsv_name) \ +do { \ + struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \ + spin_lock(&__rsv->lock); \ + btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \ + __rsv->size, __rsv->reserved); \ + spin_unlock(&__rsv->lock); \ +} while (0) +static void dump_space_info (struct btrfs_fs_info *fs_info,struct btrfs_space_info *info,u64 bytes,int dump_block_groups)
@@ -7937,6 +7946,12 @@ static void dump_space_info(struct btrfs_fs_info *fs_info, info->bytes_readonly);spin_unlock (&info->lock); +DUMP_BLOCK_RSV (fs_info, global_block_rsv); +DUMP_BLOCK_RSV (fs_info, trans_block_rsv); +DUMP_BLOCK_RSV (fs_info, chunk_block_rsv); +DUMP_BLOCK_RSV (fs_info, delayed_block_rsv); +DUMP_BLOCK_RSV (fs_info, delayed_refs_rsv); +if (!dump_block_groups)return ;
Signature · 1 line
2.14.3
Re: [PATCH 08/42] btrfs: dump block_rsv whe dumping space info
Quoted 5 lines · Josef Bacik
For
enospc_debughaving the block rsvs is super helpful to see if we've done something wrong.Signed-off-by Josef Bacik <[email protected]>Reviewed-by Omar Sandoval <[email protected]>
PATCH v3 9/42 btrfs: release metadata before running delayed refs
In reply to Josef BacikWe want to release the unused reservation we have since it refills the delayed refs reserve, which will make everything go smoother when running the delayed refs if we're short on our reservation.
--- fs/btrfs/transaction.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/btrfs/transaction.c +3 -3
@@ -1922,6 +1922,9 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)return ret; } +btrfs_trans_release_metadata (trans); + trans->block_rsv =NULL ; +/* make a pass through all the delayed refs we have so far * any runnings procs may add more while we are here */
@@ -1931,9 +1934,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)return ret; } -btrfs_trans_release_metadata (trans); - trans->block_rsv =NULL ; - cur_trans = trans->transaction;/*
Signature · 1 line
2.14.3
PATCH v3 10/42 btrfs: protect space cache inode alloc with nofs
In reply to Josef BacikIf we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail.
--- fs/btrfs/free-space-cache.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/btrfs/free-space-cache.c +8 -0
@@ -10,6 +10,7 @@#include <linux/math64.h> #include <linux/ratelimit.h> #include <linux/error-injection.h> +#include <linux/sched/mm.h> #include "ctree.h" #include "free-space-cache.h" #include "transaction.h"
@@ -47,6 +48,7 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root,struct btrfs_free_space_header *header;struct extent_buffer *leaf;struct inode *inode =NULL ; +unsigned nofs_flag;int ret; key.objectid =BTRFS_FREE_SPACE_OBJECTID ;
@@ -68,7 +70,13 @@ static struct inode *__lookup_free_space_inode(struct btrfs_root *root,btrfs_disk_key_to_cpu (&location, &disk_key);btrfs_release_path (path); +/* +* We are often under a trans handle at this point, so we need to make +* sure NOFS is set to keep us from deadlocking. +*/ + nofs_flag =memalloc_nofs_save (); inode =btrfs_iget (fs_info->sb, &location, root,NULL ); +memalloc_nofs_restore (nofs_flag);if (IS_ERR (inode))return inode;
Signature · 1 line
2.14.3
Re: [PATCH 10/42] btrfs: protect space cache inode alloc with nofs
Quoted 7 lines · Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use
memalloc_nofs_save()in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail.Reviewed-by Omar Sandoval <[email protected]>Signed-off-by Josef Bacik <[email protected]>
PATCH v3 11/42 btrfs: fix truncate throttling
In reply to Josef BacikWe have a bunch of magic to make sure we're throttling delayed refs when truncating a file. Now that we have a delayed refs rsv and a mechanism for refilling that reserve simply use that instead of all of this magic.
--- fs/btrfs/inode.c | 79 ++++++++++++-------------------------------------------- 1 file changed, 17 insertions(+), 62 deletions(-)
fs/btrfs/inode.c +17 -62
@@ -4493,31 +4493,6 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry)return err; } -static int truncate_space_check (struct btrfs_trans_handle *trans, -struct btrfs_root *root, -u64 bytes_deleted) -{ -struct btrfs_fs_info *fs_info = root->fs_info; -int ret; - -/* -* This is only used to apply pressure to the enospc system, we don't -* intend to use this reservation at all. -*/ - bytes_deleted =btrfs_csum_bytes_to_leaves (fs_info, bytes_deleted); - bytes_deleted *= fs_info->nodesize; - ret =btrfs_block_rsv_add (root, &fs_info->trans_block_rsv, - bytes_deleted,BTRFS_RESERVE_NO_FLUSH ); -if (!ret) { -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, - bytes_deleted,1 ); - trans->bytes_reserved += bytes_deleted; - } -return ret; - -} -/* * Return this if we need to call truncate_block for the last bit of the * truncate.
@@ -4562,7 +4537,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,u64 bytes_deleted =0 ;bool be_nice = false;bool should_throttle = false; -bool should_end = false;BUG_ON (new_size >0 && min_type !=BTRFS_EXTENT_DATA_KEY );
@@ -4775,15 +4749,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,btrfs_abort_transaction (trans, ret);break ; } -if (btrfs_should_throttle_delayed_refs (trans, fs_info)) -btrfs_async_run_delayed_refs (fs_info, - trans->delayed_ref_updates *2 , - trans->transid,0 );if (be_nice) { -if (truncate_space_check (trans, root, - extent_num_bytes)) { - should_end = true; - }if (btrfs_should_throttle_delayed_refs (trans, fs_info)) should_throttle = true;
@@ -4795,7 +4761,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,if (path->slots[0 ] ==0 || path->slots[0 ] != pending_del_slot || - should_throttle || should_end) { + should_throttle) {if (pending_del_nr) { ret = btrfs_del_items(trans, root, path, pending_del_slot,
@@ -4807,23 +4773,24 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, pending_del_nr =0 ; }btrfs_release_path (path); -if (should_throttle) { -unsigned long updates = trans->delayed_ref_updates; -if (updates) { - trans->delayed_ref_updates =0 ; - ret =btrfs_run_delayed_refs (trans, - updates *2 ); -if (ret) -break ; - } - } +/* -* if we failed to refill our space rsv, bail out -* and let the transaction restart +* We can generate a lot of delayed refs, so we need to +* throttle every once and a while and make sure we're +* adding enough space to keep up with the work we are +* generating. Since we hold a transaction here we +* can't flush, and we don't want to FLUSH_LIMIT because +* we could have generated too many delayed refs to +* actually allocate, so just bail if we're short and +* let the normal reservation dance happen higher up. */ -if (should_end) { - ret = -EAGAIN ; -break ; +if (should_throttle) { + ret =btrfs_throttle_delayed_refs (fs_info, +BTRFS_RESERVE_NO_FLUSH ); +if (ret) { + ret = -EAGAIN ; +break ; + } } goto search_again; }else {
@@ -4849,18 +4816,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans, }btrfs_free_path (path); - -if (be_nice && bytes_deleted >SZ_32M && (ret >=0 || ret == -EAGAIN )) { -unsigned long updates = trans->delayed_ref_updates; -int err; - -if (updates) { - trans->delayed_ref_updates =0 ; - err =btrfs_run_delayed_refs (trans, updates *2 ); -if (err) - ret = err; - } - }return ret; }
Signature · 1 line
2.14.3
PATCH v3 12/42 btrfs: don't use global rsv for chunk allocation
In reply to Josef BacikWe've done this forever because of the voodoo around knowing how much space we have. However we have better ways of doing this now, and on normal file systems we'll easily have a global reserve of 512MiB, and since metadata chunks are usually 1GiB that means we'll allocate metadata chunks more readily. Instead use the actual used amount when determining if we need to allocate a chunk or not.
--- fs/btrfs/extent-tree.c | 9 --------- 1 file changed, 9 deletions(-)
fs/btrfs/extent-tree.c +0 -9
@@ -4374,21 +4374,12 @@ static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global)static int should_alloc_chunk (struct btrfs_fs_info *fs_info,struct btrfs_space_info *sinfo,int force) { -struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;u64 bytes_used =btrfs_space_info_used (sinfo, false);u64 thresh;if (force ==CHUNK_ALLOC_FORCE )return 1 ; -/* -* We need to take into account the global rsv because for all intents -* and purposes it's used space. Don't worry about locking the -* global_rsv, it doesn't change except when the transaction commits. -*/ -if (sinfo->flags &BTRFS_BLOCK_GROUP_METADATA ) - bytes_used +=calc_global_rsv_need_space (global_rsv); -/* * in limited mode, we want to have some free space up to * about 1% of the FS size.
Signature · 1 line
2.14.3
PATCH v3 13/42 btrfs: add ALLOC_CHUNK_FORCE to the flushing code
In reply to Josef BacikWith my change to no longer take into account the global reserve for metadata allocation chunks we have this side-effect for mixed block group fs'es where we are no longer allocating enough chunks for the data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE step to the flushing state machine. This will only get used if we've already made a full loop through the flushing machinery and tried committing the transaction. If we have then we can try and force a chunk allocation since we likely need it to make progress. This resolves the issues I was seeing with the mixed bg tests in xfstests with my previous patch.
--- fs/btrfs/ctree.h | 3 ++- fs/btrfs/extent-tree.c | 18 +++++++++++++++++- include/trace/events/btrfs.h | 1 + 3 files changed, 20 insertions(+), 2 deletions(-)
fs/btrfs/ctree.h +2 -1
@@ -2730,7 +2730,8 @@ enum btrfs_flush_state {FLUSH_DELALLOC =5 ,FLUSH_DELALLOC_WAIT =6 ,ALLOC_CHUNK =7 , - COMMIT_TRANS = 8, +ALLOC_CHUNK_FORCE =8 , + COMMIT_TRANS = 9, };int btrfs_alloc_data_chunk_ondemand (struct btrfs_inode *inode,u64 bytes);
fs/btrfs/extent-tree.c +17 -1
@@ -4914,6 +4914,7 @@ static void flush_space(struct btrfs_fs_info *fs_info,btrfs_end_transaction (trans);break ;case ALLOC_CHUNK : +case ALLOC_CHUNK_FORCE : trans =btrfs_join_transaction (root);if (IS_ERR (trans)) { ret =PTR_ERR (trans);
@@ -4921,7 +4922,9 @@ static void flush_space(struct btrfs_fs_info *fs_info, } ret =do_chunk_alloc (trans,btrfs_metadata_alloc_profile (fs_info), -CHUNK_ALLOC_NO_FORCE ); + (state ==ALLOC_CHUNK ) ? +CHUNK_ALLOC_NO_FORCE : +CHUNK_ALLOC_FORCE );btrfs_end_transaction (trans);if (ret >0 || ret == -ENOSPC ) ret =0 ;
@@ -5057,6 +5060,19 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) commit_cycles--; } +/* +* We don't want to force a chunk allocation until we've tried +* pretty hard to reclaim space. Think of the case where we +* free'd up a bunch of space and so have a lot of pinned space +* to reclaim. We would rather use that than possibly create a +* underutilized metadata chunk. So if this is our first run +* through the flushing state machine skip ALLOC_CHUNK_FORCE and +* commit the transaction. If nothing has changed the next go +* around then we can force a chunk allocation. +*/ +if (flush_state ==ALLOC_CHUNK_FORCE && !commit_cycles) + flush_state++; +if (flush_state >COMMIT_TRANS ) { commit_cycles++;if (commit_cycles >2 ) {
include/trace/events/btrfs.h +1 -0
@@ -1051,6 +1051,7 @@ TRACE_EVENT(btrfs_trigger_flush, { FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \ { FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \ { ALLOC_CHUNK, "ALLOC_CHUNK"}, \ + { ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE"}, \ { COMMIT_TRANS, "COMMIT_TRANS"}) TRACE_EVENT(btrfs_flush_space,
Signature · 1 line
2.14.3
PATCH v3 14/42 btrfs: reset max_extent_size properly
In reply to Josef BacikIf we use up our block group before allocating a new one we'll easily get a max_extent_size that's set really really low, which will result in a lot of fragmentation. We need to make sure we're resetting the max_extent_size when we add a new chunk or add new space.
--- fs/btrfs/extent-tree.c | 2 ++ 1 file changed, 2 insertions(+)
fs/btrfs/extent-tree.c +2 -0
@@ -4573,6 +4573,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags, goto out; }else { ret =1 ; + space_info->max_extent_size =0 ; } space_info->force_alloc =CHUNK_ALLOC_NO_FORCE ;
@@ -6671,6 +6672,7 @@ static int btrfs_free_reserved_bytes(struct btrfs_block_group_cache *cache, space_info->bytes_readonly += num_bytes; cache->reserved -= num_bytes; space_info->bytes_reserved -= num_bytes; + space_info->max_extent_size =0 ;if (delalloc) cache->delalloc_bytes -= num_bytes;
Signature · 1 line
2.14.3
PATCH v3 15/42 btrfs: don't enospc all tickets on flush failure
In reply to Josef BacikWith the introduction of the per-inode block_rsv it became possible to have really really large reservation requests made because of data fragmentation. Since the ticket stuff assumed that we'd always have relatively small reservation requests it just killed all tickets if we were unable to satisfy the current request. However this is generally not the case anymore. So fix this logic to instead see if we had a ticket that we were able to give some reservation to, and if we were continue the flushing loop again. Likewise we make the tickets use the space_info_add_old_bytes() method of returning what reservation they did receive in hopes that it could satisfy reservations down the line.
--- fs/btrfs/extent-tree.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
fs/btrfs/extent-tree.c +25 -20
@@ -4779,6 +4779,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim, }struct reserve_ticket { +u64 orig_bytes;u64 bytes;int error;struct list_head list;
@@ -5000,7 +5001,7 @@ static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info, !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)); } -static void wake_all_tickets (struct list_head *head) +static bool wake_all_tickets (struct list_head *head) {struct reserve_ticket *ticket;
@@ -5009,7 +5010,10 @@ static void wake_all_tickets(struct list_head *head)list_del_init (&ticket->list); ticket->error = -ENOSPC ;wake_up (&ticket->wait); +if (ticket->bytes != ticket->orig_bytes) +return true; } +return false; }/*
@@ -5077,8 +5081,12 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work)if (flush_state >COMMIT_TRANS ) { commit_cycles++;if (commit_cycles >2 ) { -wake_all_tickets (&space_info->tickets); - space_info->flush =0 ; +if (wake_all_tickets (&space_info->tickets)) { + flush_state =FLUSH_DELAYED_ITEMS_NR ; + commit_cycles--; + }else { + space_info->flush =0 ; + } }else { flush_state =FLUSH_DELAYED_ITEMS_NR ; }
@@ -5130,10 +5138,11 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,static int wait_reserve_ticket (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info, -struct reserve_ticket *ticket,u64 orig_bytes) +struct reserve_ticket *ticket) {DEFINE_WAIT (wait); +u64 reclaim_bytes =0 ;int ret =0 ;spin_lock (&space_info->lock);
@@ -5154,14 +5163,12 @@ static int wait_reserve_ticket(struct btrfs_fs_info *fs_info, ret = ticket->error;if (!list_empty (&ticket->list))list_del_init (&ticket->list); -if (ticket->bytes && ticket->bytes < orig_bytes) { -u64 num_bytes = orig_bytes - ticket->bytes; - space_info->bytes_may_use -= num_bytes; -trace_btrfs_space_reservation (fs_info,"space_info" , - space_info->flags, num_bytes,0 ); - } +if (ticket->bytes && ticket->bytes < ticket->orig_bytes) + reclaim_bytes = ticket->orig_bytes - ticket->bytes;spin_unlock (&space_info->lock); +if (reclaim_bytes) +space_info_add_old_bytes (fs_info, space_info, reclaim_bytes);return ret; }
@@ -5187,6 +5194,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info, {struct reserve_ticket ticket;u64 used; +u64 reclaim_bytes =0 ;int ret =0 ;ASSERT (orig_bytes);
@@ -5222,6 +5230,7 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info,* the list and we will do our own flushing further down. */ if (ret && flush !=BTRFS_RESERVE_NO_FLUSH ) { + ticket.orig_bytes = orig_bytes; ticket.bytes = orig_bytes; ticket.error =0 ;init_waitqueue_head (&ticket.wait);
@@ -5262,25 +5271,21 @@ static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info,return ret;if (flush ==BTRFS_RESERVE_FLUSH_ALL ) -return wait_reserve_ticket (fs_info, space_info, &ticket, - orig_bytes); +return wait_reserve_ticket (fs_info, space_info, &ticket); ret =0 ;priority_reclaim_metadata_space (fs_info, space_info, &ticket);spin_lock (&space_info->lock);if (ticket.bytes) { -if (ticket.bytes < orig_bytes) { -u64 num_bytes = orig_bytes - ticket.bytes; - space_info->bytes_may_use -= num_bytes; -trace_btrfs_space_reservation (fs_info,"space_info" , - space_info->flags, - num_bytes,0 ); - - } +if (ticket.bytes < orig_bytes) + reclaim_bytes = orig_bytes - ticket.bytes;list_del_init (&ticket.list); ret = -ENOSPC ; }spin_unlock (&space_info->lock); + +if (reclaim_bytes) +space_info_add_old_bytes (fs_info, space_info, reclaim_bytes);ASSERT (list_empty (&ticket.list));return ret; }
Signature · 1 line
2.14.3
PATCH v3 16/42 btrfs: loop in inode_rsv_refill
In reply to Josef BacikWith severe fragmentation we can end up with our inode rsv size being huge during writeout, which would cause us to need to make very large metadata reservations. However we may not actually need that much once writeout is complete. So instead try to make our reservation, and if we couldn't make it re-calculate our new reservation size and try again. If our reservation size doesn't change between tries then we know we are actually out of space and can error out.
--- fs/btrfs/extent-tree.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-)
fs/btrfs/extent-tree.c +24 -2
@@ -5781,10 +5781,11 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, {struct btrfs_root *root = inode->root;struct btrfs_block_rsv *block_rsv = &inode->block_rsv; -u64 num_bytes =0 ; +u64 num_bytes =0 , last =0 ;u64 qgroup_num_bytes =0 ;int ret = -ENOSPC ; +again:spin_lock (&block_rsv->lock);if (block_rsv->reserved < block_rsv->size) num_bytes = block_rsv->size - block_rsv->reserved;
@@ -5796,6 +5797,13 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode,if (num_bytes ==0 )return 0 ; +/* +* If our reservation size hasn't changed since the last time we tried +* to make an allocation we can just bail. +*/ +if (last && last == num_bytes) +return -ENOSPC ; + ret =btrfs_qgroup_reserve_meta_prealloc (root, qgroup_num_bytes, true);if (ret)return ret;
@@ -5809,8 +5817,22 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode,spin_lock (&block_rsv->lock); block_rsv->qgroup_rsv_reserved += qgroup_num_bytes;spin_unlock (&block_rsv->lock); - }else + }else {btrfs_qgroup_free_meta_prealloc (root, qgroup_num_bytes); + +/* +* If we are fragmented we can end up with a lot of outstanding +* extents which will make our size be much larger than our +* reserved amount. If we happen to try to do a reservation +* here that may result in us trying to do a pretty hefty +* reservation, which we may not need once delalloc flushing +* happens. If this is the case try and do the reserve again. +*/ +if (flush ==BTRFS_RESERVE_FLUSH_ALL ) { + last = num_bytes; + goto again; + } + }return ret; }
Signature · 1 line
2.14.3
Re: [PATCH 16/42] btrfs: loop in inode_rsv_refill
Quoted 18 lines · Josef Bacik
With severe fragmentation we can end up with our inode rsv size being huge during writeout, which would cause us to need to make very large metadata reservations. However we may not actually need that much once writeout is complete. So instead try to make our reservation, and if we couldn't make it re-calculate our new reservation size and try again. If our reservation size doesn't change between tries then we know we are actually out of space and can error out.
Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/extent-tree.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-)fs/btrfs/extent-tree.c +0
-0
fs/btrfs/extent-tree.c static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, · line 5788
Quoted 9 lines
fs/btrfs/extent-tree.c +2
-1@@ -5781,10 +5781,11 @@ static int btrfs_inode_rsv_refill(struct btrfs_inode *inode, {struct btrfs_root *root = inode->root;struct btrfs_block_rsv *block_rsv = &inode->block_rsv; -u64 num_bytes =0 ; +u64 num_bytes =0 , last =0 ;u64 qgroup_num_bytes =0 ;int ret = -ENOSPC ; +again:
Could this be restructured so there's no new 'goto again;' pattern added?
PATCH v3 17/42 btrfs: run delayed iputs before committing
In reply to Josef BacikDelayed iputs means we can have final iputs of deleted inodes in the queue, which could potentially generate a lot of pinned space that could be free'd. So before we decide to commit the transaction for ENOPSC reasons, run the delayed iputs so that any potential space is free'd up. If there is and we freed enough we can then commit the transaction and potentially be able to make our reservation.
--- fs/btrfs/extent-tree.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/btrfs/extent-tree.c +9 -0
@@ -4823,6 +4823,15 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,if (!bytes)return 0 ; +/* +* If we have pending delayed iputs then we could free up a bunch of +* pinned space, so make sure we run the iputs before we do our pinned +* bytes check below. +*/ +mutex_lock (&fs_info->cleaner_delayed_iput_mutex); +btrfs_run_delayed_iputs (fs_info); +mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); + trans =btrfs_join_transaction (fs_info->extent_root);if (IS_ERR (trans))return -ENOSPC ;
Signature · 1 line
2.14.3
PATCH v3 18/42 btrfs: move the dio_sem higher up the callchain
In reply to Josef BacikWe're getting a lockdep splat because we take the dio_sem under the log_mutex. What we really need is to protect fsync() from logging an extent map for an extent we never waited on higher up, so just guard the whole thing with dio_sem.
--- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-)
fs/btrfs/file.c +12 -0
@@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out;inode_lock (inode); + +/* +* We take the dio_sem here because the tree log stuff can race with +* lockless dio writes and get an extent map logged for an extent we +* never waited on. We need it this high up for lockdep reasons. +*/ +down_write (&BTRFS_I (inode)->dio_sem); +atomic_inc (&root->log_batch);/*
@@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)*/ ret =btrfs_wait_ordered_range (inode, start, len);if (ret) { +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }
@@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* checked called fsync. */ ret =filemap_check_wb_err (inode->i_mapping, file->f_wb_err); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }
@@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans =btrfs_start_transaction (root,0 );if (IS_ERR (trans)) { ret =PTR_ERR (trans); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }
@@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode);/*
fs/btrfs/tree-log.c +0 -2
@@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans,INIT_LIST_HEAD (&extents); -down_write (&inode->dio_sem);write_lock (&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start;
@@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, }WARN_ON (!list_empty (&extents));write_unlock (&tree->lock); -up_write (&inode->dio_sem);btrfs_release_path (path);if (!ret)
Signature · 1 line
2.14.3
Re: [PATCH 18/42] btrfs: move the dio_sem higher up the callchain
Quoted 1 line · Josef Bacik
We're getting a lockdep splat because we take the
dio_semunder the
Can you please add the important bits of the lockdep warning to the changelog? And possibly reference to the test or workload that triggers that.
Quoted 83 lines
log_mutex. What we really need is to protectfsync()from logging an extent map for an extent we never waited on higher up, so just guard the whole thing withdio_sem.Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-)fs/btrfs/file.c +12
-0@@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out;inode_lock (inode); + +/* +* We take the dio_sem here because the tree log stuff can race with +* lockless dio writes and get an extent map logged for an extent we +* never waited on. We need it this high up for lockdep reasons. +*/ +down_write (&BTRFS_I (inode)->dio_sem); +atomic_inc (&root->log_batch);/* @@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)*/ ret =btrfs_wait_ordered_range (inode, start, len);if (ret) { +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }@@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* checked called fsync. */ ret =filemap_check_wb_err (inode->i_mapping, file->f_wb_err); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }@@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans =btrfs_start_transaction (root,0 );if (IS_ERR (trans)) { ret =PTR_ERR (trans); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }@@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode);/* fs/btrfs/tree-log.c +0
-2@@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans,INIT_LIST_HEAD (&extents); -down_write (&inode->dio_sem);write_lock (&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start;@@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, }WARN_ON (!list_empty (&extents));write_unlock (&tree->lock); -up_write (&inode->dio_sem);btrfs_release_path (path);if (!ret)Signature · 1 line
2.14.3
Re: [PATCH 18/42] btrfs: move the dio_sem higher up the callchain
In reply to Josef BacikQuoted 7 lines · Josef Bacik
We're getting a lockdep splat because we take the
dio_semunder thelog_mutex. What we really need is to protectfsync()from logging an extent map for an extent we never waited on higher up, so just guard the whole thing withdio_sem.Signed-off-by Josef Bacik <[email protected]>
Looks good, thanks. However as David said, it would be nice to have a sample trace pasted in the changelog (several fstests test cases trigger this often).
Quoted 79 lines
--- fs/btrfs/file.c | 12 ++++++++++++ fs/btrfs/tree-log.c | 2 -- 2 files changed, 12 insertions(+), 2 deletions(-)fs/btrfs/file.c +12
-0@@ -2079,6 +2079,14 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out;inode_lock (inode); + +/* +* We take the dio_sem here because the tree log stuff can race with +* lockless dio writes and get an extent map logged for an extent we +* never waited on. We need it this high up for lockdep reasons. +*/ +down_write (&BTRFS_I (inode)->dio_sem); +atomic_inc (&root->log_batch);/* @@ -2087,6 +2095,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)*/ ret =btrfs_wait_ordered_range (inode, start, len);if (ret) { +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }@@ -2110,6 +2119,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* checked called fsync. */ ret =filemap_check_wb_err (inode->i_mapping, file->f_wb_err); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }@@ -2128,6 +2138,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) trans =btrfs_start_transaction (root,0 );if (IS_ERR (trans)) { ret =PTR_ERR (trans); +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode); goto out; }@@ -2149,6 +2160,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)* file again, but that will end up using the synchronization * inside btrfs_sync_log to keep things safe. */ +up_write (&BTRFS_I (inode)->dio_sem);inode_unlock (inode);/* fs/btrfs/tree-log.c +0
-3@@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans,INIT_LIST_HEAD (&extents); -down_write (&inode->dio_sem);write_lock (&tree->lock); test_gen = root->fs_info->last_trans_committed; logged_start = start;@@ -4440,7 +4439,6 @@ static int btrfs_log_changed_extents(struct btrfs_trans_handle *trans, }WARN_ON (!list_empty (&extents));write_unlock (&tree->lock); -up_write (&inode->dio_sem);btrfs_release_path (path);if (!ret) --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
PATCH v3 19/42 btrfs: set max_extent_size properly
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We can't use entry->bytes if our entry is a bitmap entry, we need to use entry->max_extent_size in that case. Fix up all the logic to make this consistent.
--- fs/btrfs/free-space-cache.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)
fs/btrfs/free-space-cache.c +19 -10
@@ -1770,6 +1770,18 @@ static int search_bitmap(struct btrfs_free_space_ctl *ctl,return -1 ; } +static void set_max_extent_size (struct btrfs_free_space *entry, +u64 *max_extent_size) +{ +if (entry->bitmap) { +if (entry->max_extent_size > *max_extent_size) + *max_extent_size = entry->max_extent_size; + }else { +if (entry->bytes > *max_extent_size) + *max_extent_size = entry->bytes; + } +} +/* Cache the size of the max extent in bytes */ static struct btrfs_free_space * find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes,
@@ -1791,8 +1803,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes,for (node = &entry->offset_index; node; node =rb_next (node)) { entry = rb_entry(node, struct btrfs_free_space, offset_index);if (entry->bytes < *bytes) { -if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +set_max_extent_size (entry, max_extent_size);continue ; }
@@ -1810,8 +1821,7 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, }if (entry->bytes < *bytes + align_off) { -if (entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +set_max_extent_size (entry, max_extent_size);continue ; }
@@ -1823,8 +1833,8 @@ find_free_space(struct btrfs_free_space_ctl *ctl, u64 *offset, u64 *bytes, *offset = tmp; *bytes = size;return entry; - }else if (size > *max_extent_size) { - *max_extent_size = size; + }else { +set_max_extent_size (entry, max_extent_size); }continue ; }
@@ -2684,8 +2694,7 @@ static u64 btrfs_alloc_from_bitmap(struct btrfs_block_group_cache *block_group, err =search_bitmap (ctl, entry, &search_start, &search_bytes, true);if (err) { -if (search_bytes > *max_extent_size) - *max_extent_size = search_bytes; +set_max_extent_size (entry, max_extent_size);return 0 ; }
@@ -2722,8 +2731,8 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, entry = rb_entry(node, struct btrfs_free_space, offset_index);while (1 ) { -if (entry->bytes < bytes && entry->bytes > *max_extent_size) - *max_extent_size = entry->bytes; +if (entry->bytes < bytes) +set_max_extent_size (entry, max_extent_size);if (entry->bytes < bytes || (!entry->bitmap && entry->offset < min_start)) {
Signature · 1 line
2.14.3
PATCH v3 20/42 btrfs: don't use ctl->free_space for max_extent_size
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
max_extent_size is supposed to be the largest contiguous range for the space info, and ctl->free_space is the total free space in the block group. We need to keep track of these separately and _only_ use the max_free_space if we don't have a max_extent_size, as that means our original request was too large to search any of the block groups for and therefore wouldn't have a max_extent_size set.
--- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
fs/btrfs/extent-tree.c +5 -2
@@ -7492,6 +7492,7 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,struct btrfs_block_group_cache *block_group =NULL ;u64 search_start =0 ;u64 max_extent_size =0 ; +u64 max_free_space =0 ;u64 empty_cluster =0 ;struct btrfs_space_info *space_info;int loop =0 ;
@@ -7787,8 +7788,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info,spin_lock (&ctl->tree_lock);if (ctl->free_space < num_bytes + empty_cluster + empty_size) { -if (ctl->free_space > max_extent_size) - max_extent_size = ctl->free_space; + max_free_space =max (max_free_space, + ctl->free_space);spin_unlock (&ctl->tree_lock); goto loop; }
@@ -7955,6 +7956,8 @@ static noinline int find_free_extent(struct btrfs_fs_info *fs_info, } out:if (ret == -ENOSPC ) { +if (!max_extent_size) + max_extent_size = max_free_space;spin_lock (&space_info->lock); space_info->max_extent_size = max_extent_size;spin_unlock (&space_info->lock);
Signature · 1 line
2.14.3
PATCH v3 21/42 btrfs: reset max_extent_size on clear in a bitmap
In reply to Josef BacikFrom: Josef Bacik <[email protected]>
We need to clear the max_extent_size when we clear bits from a bitmap since it could have been from the range that contains the max_extent_size.
--- fs/btrfs/free-space-cache.c | 2 ++ 1 file changed, 2 insertions(+)
fs/btrfs/free-space-cache.c +2 -0
@@ -1687,6 +1687,8 @@ static inline void __bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,bitmap_clear (info->bitmap, start, count); info->bytes -= bytes; +if (info->max_extent_size > ctl->unit) + info->max_extent_size =0 ; } static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,
Signature · 1 line
2.14.3
PATCH v3 22/42 btrfs: only run delayed refs if we're committing
In reply to Josef BacikI noticed in a giant dbench run that we spent a lot of time on lock contention while running transaction commit. This is because dbench results in a lot of fsync()'s that do a btrfs_transaction_commit(), and they all run the delayed refs first thing, so they all contend with each other. This leads to seconds of 0 throughput. Change this to only run the delayed refs if we're the ones committing the transaction. This makes the latency go away and we get no more lock contention.
--- fs/btrfs/transaction.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-)
fs/btrfs/transaction.c +9 -15
@@ -1925,15 +1925,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)btrfs_trans_release_metadata (trans); trans->block_rsv =NULL ; -/* make a pass through all the delayed refs we have so far -* any runnings procs may add more while we are here -*/ - ret =btrfs_run_delayed_refs (trans,0 ); -if (ret) { -btrfs_end_transaction (trans); -return ret; - } - cur_trans = trans->transaction;/*
@@ -1946,12 +1937,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)if (!list_empty (&trans->new_bgs))btrfs_create_pending_block_groups (trans); - ret =btrfs_run_delayed_refs (trans,0 ); -if (ret) { -btrfs_end_transaction (trans); -return ret; - } -if (!test_bit (BTRFS_TRANS_DIRTY_BG_RUN , &cur_trans->flags)) {int run_it =0 ;
@@ -2022,6 +2007,15 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)spin_unlock (&fs_info->trans_lock); } +/* +* We are now the only one in the commit area, we can run delayed refs +* without hitting a bunch of lock contention from a lot of people +* trying to commit the transaction at once. +*/ + ret =btrfs_run_delayed_refs (trans,0 ); +if (ret) + goto cleanup_transaction; +extwriter_counter_dec (cur_trans, trans->type); ret =btrfs_start_delalloc_flush (fs_info);
Signature · 1 line
2.14.3
PATCH v3 23/42 btrfs: make sure we create all new bgs
In reply to Josef BacikAllocating new chunks modifies both the extent and chunk tree, which can trigger new chunk allocations. So instead of doing list_for_each_safe, just do while (!list_empty()) so we make sure we don't exit with other pending bg's still on our list.
--- fs/btrfs/extent-tree.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
fs/btrfs/extent-tree.c +5 -2
@@ -10357,7 +10357,7 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)void btrfs_create_pending_block_groups (struct btrfs_trans_handle *trans) {struct btrfs_fs_info *fs_info = trans->fs_info; -struct btrfs_block_group_cache *block_group, *tmp; +struct btrfs_block_group_cache *block_group;struct btrfs_root *extent_root = fs_info->extent_root;struct btrfs_block_group_item item;struct btrfs_key key;
@@ -10365,7 +10365,10 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)bool can_flush_pending_bgs = trans->can_flush_pending_bgs; trans->can_flush_pending_bgs = false; -list_for_each_entry_safe (block_group, tmp, &trans->new_bgs, bg_list) { +while (!list_empty (&trans->new_bgs)) { + block_group =list_first_entry (&trans->new_bgs, + struct btrfs_block_group_cache, + bg_list);if (ret) goto next;
Signature · 1 line
2.14.3
Re: [PATCH 23/42] btrfs: make sure we create all new bgs
Quoted 8 lines · Josef Bacik
Allocating new chunks modifies both the extent and chunk tree, which can trigger new chunk allocations. So instead of doing
list_for_each_safe, just do while (!list_empty()) so we make sure we don't exit with other pending bg's still on our list.Reviewed-by Omar Sandoval <[email protected]>Reviewed-by Liu Bo <[email protected]>Signed-off-by Josef Bacik <[email protected]>
PATCH v3 24/42 btrfs: assert on non-empty delayed iputs
In reply to Josef BacikI ran into an issue where there was some reference being held on an inode that I couldn't track. This assert wasn't triggered, but it at least rules out we're doing something stupid.
--- fs/btrfs/disk-io.c | 1 + 1 file changed, 1 insertion(+)
fs/btrfs/disk-io.c +1 -0
@@ -3979,6 +3979,7 @@ void close_ctree(struct btrfs_fs_info *fs_info)kthread_stop (fs_info->transaction_kthread);kthread_stop (fs_info->cleaner_kthread); +ASSERT (list_empty (&fs_info->delayed_iputs));set_bit (BTRFS_FS_CLOSING_DONE , &fs_info->flags);btrfs_free_qgroup_config (fs_info);
Signature · 1 line
2.14.3
Re: [PATCH 24/42] btrfs: assert on non-empty delayed iputs
Quoted 6 lines · Josef Bacik
I ran into an issue where there was some reference being held on an inode that I couldn't track. This assert wasn't triggered, but it at least rules out we're doing something stupid.
Reviewed-by Omar Sandoval <[email protected]>Signed-off-by Josef Bacik <[email protected]>
PATCH v3 25/42 btrfs: pass delayed_refs_root to btrfs_delayed_ref_lock
In reply to Josef BacikWe don't need the trans except to get the delayed_refs_root, so just pass the delayed_refs_root into btrfs_delayed_ref_lock and call it a day.
--- fs/btrfs/delayed-ref.c | 5 +---- fs/btrfs/delayed-ref.h | 2 +- fs/btrfs/extent-tree.c | 2 +- 3 files changed, 3 insertions(+), 6 deletions(-)
fs/btrfs/delayed-ref.c +1 -4
@@ -197,12 +197,9 @@ find_ref_head(struct rb_root *root, u64 bytenr,return NULL ; } -int btrfs_delayed_ref_lock (struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock (struct btrfs_delayed_ref_root *delayed_refs,struct btrfs_delayed_ref_head *head) { -struct btrfs_delayed_ref_root *delayed_refs; - - delayed_refs = &trans->transaction->delayed_refs;lockdep_assert_held (&delayed_refs->lock);if (mutex_trylock (&head->mutex))return 0 ;
fs/btrfs/delayed-ref.h +1 -1
@@ -255,7 +255,7 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,struct btrfs_delayed_ref_head *btrfs_find_delayed_ref_head (struct btrfs_delayed_ref_root *delayed_refs,u64 bytenr); -int btrfs_delayed_ref_lock (struct btrfs_trans_handle *trans, +int btrfs_delayed_ref_lock (struct btrfs_delayed_ref_root *delayed_refs,struct btrfs_delayed_ref_head *head);static inline void btrfs_delayed_ref_unlock (struct btrfs_delayed_ref_head *head) {
fs/btrfs/extent-tree.c +1 -1
@@ -2600,7 +2600,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,/* grab the lock that says we are going to process * all the refs for this head */ - ret =btrfs_delayed_ref_lock (trans, locked_ref); + ret =btrfs_delayed_ref_lock (delayed_refs, locked_ref);spin_unlock (&delayed_refs->lock);/* * we may have dropped the spin lock to get the head
Signature · 1 line
2.14.3
PATCH v3 26/42 btrfs: make btrfs_destroy_delayed_refs use btrfs_delayed_ref_lock
In reply to Josef BacikWe have this open coded in btrfs_destroy_delayed_refs, use the helper instead.
--- fs/btrfs/disk-io.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)
fs/btrfs/disk-io.c +2 -9
@@ -4214,16 +4214,9 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans, head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); -if (!mutex_trylock (&head->mutex)) { -refcount_inc (&head->refs); -spin_unlock (&delayed_refs->lock); - -mutex_lock (&head->mutex); -mutex_unlock (&head->mutex); -btrfs_put_delayed_ref_head (head); -spin_lock (&delayed_refs->lock); +if (btrfs_delayed_ref_lock (delayed_refs, head))continue ; - } +spin_lock (&head->lock);while ((n =rb_first (&head->ref_tree)) !=NULL ) { ref = rb_entry(n, struct btrfs_delayed_ref_node,
Signature · 1 line
2.14.3
PATCH v3 27/42 btrfs: make btrfs_destroy_delayed_refs use btrfs_delete_ref_head
In reply to Josef BacikInstead of open coding this stuff use the helper instead.
--- fs/btrfs/disk-io.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
fs/btrfs/disk-io.c +1 -6
@@ -4232,12 +4232,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,if (head->must_insert_reserved) pin_bytes = true;btrfs_free_delayed_extent_op (head->extent_op); - delayed_refs->num_heads--; -if (head->processing ==0 ) - delayed_refs->num_heads_ready--; -atomic_dec (&delayed_refs->num_entries); -rb_erase (&head->href_node, &delayed_refs->href_root); -RB_CLEAR_NODE (&head->href_node); +btrfs_delete_ref_head (delayed_refs, head);spin_unlock (&head->lock);spin_unlock (&delayed_refs->lock);mutex_unlock (&head->mutex);
Signature · 1 line
2.14.3
PATCH v3 28/42 btrfs: handle delayed ref head accounting cleanup in abort
In reply to Josef BacikWe weren't doing any of the accounting cleanup when we aborted transactions. Fix this by making cleanup_ref_head_accounting global and calling it from the abort code, this fixes the issue where our accounting was all wrong after the fs aborts.
--- fs/btrfs/ctree.h | 5 +++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/extent-tree.c | 13 ++++++------- 3 files changed, 12 insertions(+), 7 deletions(-)
fs/btrfs/ctree.h +5 -0
@@ -35,6 +35,7 @@struct btrfs_trans_handle ;struct btrfs_transaction ;struct btrfs_pending_snapshot ; +struct btrfs_delayed_ref_root ;extern struct kmem_cache *btrfs_trans_handle_cachep;extern struct kmem_cache *btrfs_bit_radix_cachep;extern struct kmem_cache *btrfs_path_cachep;
@@ -2623,6 +2624,10 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long count);int btrfs_async_run_delayed_refs (struct btrfs_fs_info *fs_info,unsigned long count,u64 transid,int wait); +void +btrfs_cleanup_ref_head_accounting (struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head);int btrfs_lookup_data_extent (struct btrfs_fs_info *fs_info,u64 start,u64 len); int btrfs_lookup_extent_info(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info, u64 bytenr,
fs/btrfs/disk-io.c +1 -0
@@ -4240,6 +4240,7 @@ static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,if (pin_bytes)btrfs_pin_extent (fs_info, head->bytenr, head->num_bytes,1 ); +btrfs_cleanup_ref_head_accounting (fs_info, delayed_refs, head);btrfs_put_delayed_ref_head (head);cond_resched ();spin_lock (&delayed_refs->lock);
fs/btrfs/extent-tree.c +6 -7
@@ -2475,12 +2475,11 @@ static int run_and_cleanup_extent_op(struct btrfs_trans_handle *trans,return ret ? ret :1 ; } -static void cleanup_ref_head_accounting (struct btrfs_trans_handle *trans, -struct btrfs_delayed_ref_head *head) +void +btrfs_cleanup_ref_head_accounting (struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_root *delayed_refs, +struct btrfs_delayed_ref_head *head) { -struct btrfs_fs_info *fs_info = trans->fs_info; -struct btrfs_delayed_ref_root *delayed_refs = - &trans->transaction->delayed_refs;int nr_items =1 ;if (head->total_ref_mod <0 ) {
@@ -2558,7 +2557,7 @@ static int cleanup_ref_head(struct btrfs_trans_handle *trans, } } -cleanup_ref_head_accounting (trans, head); +btrfs_cleanup_ref_head_accounting (fs_info, delayed_refs, head);trace_run_delayed_ref_head (fs_info, head,0 );btrfs_delayed_ref_unlock (head);
@@ -7223,7 +7222,7 @@ static noinline int check_ref_cleanup(struct btrfs_trans_handle *trans,if (head->must_insert_reserved) ret =1 ; -cleanup_ref_head_accounting (trans, head); +btrfs_cleanup_ref_head_accounting (trans->fs_info, delayed_refs, head);mutex_unlock (&head->mutex);btrfs_put_delayed_ref_head (head);return ret;
Signature · 1 line
2.14.3
PATCH v3 29/42 btrfs: call btrfs_create_pending_block_groups unconditionally
In reply to Josef BacikThe first thing we do is loop through the list, this
if (!list_empty())
btrfs_create_pending_block_groups();
thing is just wasted space.
--- fs/btrfs/extent-tree.c | 3 +-- fs/btrfs/transaction.c | 6 ++---- 2 files changed, 3 insertions(+), 6 deletions(-)
fs/btrfs/extent-tree.c +1 -2
@@ -2978,8 +2978,7 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, }if (run_all) { -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); +btrfs_create_pending_block_groups (trans);spin_lock (&delayed_refs->lock); node =rb_first (&delayed_refs->href_root);
fs/btrfs/transaction.c +2 -4
@@ -846,8 +846,7 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,btrfs_trans_release_metadata (trans); trans->block_rsv =NULL ; -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); +btrfs_create_pending_block_groups (trans);btrfs_trans_release_chunk_metadata (trans);
@@ -1934,8 +1933,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing =1 ;smp_wmb (); -if (!list_empty (&trans->new_bgs)) -btrfs_create_pending_block_groups (trans); +btrfs_create_pending_block_groups (trans);if (!test_bit (BTRFS_TRANS_DIRTY_BG_RUN , &cur_trans->flags)) {int run_it =0 ;
Signature · 1 line
2.14.3
PATCH v3 30/42 btrfs: just delete pending bgs if we are aborted
In reply to Josef BacikWe still need to do all of the accounting cleanup for pending block groups if we abort. So set the ret to trans->aborted so if we aborted the cleanup happens and everybody is happy.
--- fs/btrfs/extent-tree.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +7 -1
@@ -10359,9 +10359,15 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)struct btrfs_root *extent_root = fs_info->extent_root;struct btrfs_block_group_item item;struct btrfs_key key; -int ret =0 ; +int ret;bool can_flush_pending_bgs = trans->can_flush_pending_bgs; +/* +* If we aborted the transaction with pending bg's we need to just +* cleanup the list and carry on. +*/ + ret = trans->aborted; + trans->can_flush_pending_bgs = false;while (!list_empty (&trans->new_bgs)) { block_group = list_first_entry(&trans->new_bgs,
Signature · 1 line
2.14.3
PATCH v3 31/42 btrfs: cleanup pending bgs on transaction abort
In reply to Josef BacikWe may abort the transaction during a commit and not have a chance to run the pending bgs stuff, which will leave block groups on our list and cause us accounting issues and leaked memory. Fix this by running the pending bgs when we cleanup a transaction.
--- fs/btrfs/transaction.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/btrfs/transaction.c +4 -0
@@ -2280,6 +2280,10 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)btrfs_scrub_continue (fs_info); cleanup_transaction:btrfs_trans_release_metadata (trans); +/* This cleans up the pending block groups list properly. */ +if (!trans->aborted) + trans->aborted = ret; +btrfs_create_pending_block_groups (trans);btrfs_trans_release_chunk_metadata (trans); trans->block_rsv =NULL ;btrfs_warn (fs_info,"Skipping commit of aborted transaction." );
Signature · 1 line
2.14.3
PATCH v3 32/42 btrfs: only free reserved extent if we didn't insert it
In reply to Josef BacikWhen we insert the file extent once the ordered extent completes we free the reserved extent reservation as it'll have been migrated to the bytes_used counter. However if we error out after this step we'll still clear the reserved extent reservation, resulting in a negative accounting of the reserved bytes for the block group and space info. Fix this by only doing the free if we didn't successfully insert a file extent for this extent.
--- fs/btrfs/inode.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
fs/btrfs/inode.c +9 -1
@@ -2992,6 +2992,7 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent)bool truncated = false;bool range_locked = false;bool clear_new_delalloc_bytes = false; +bool clear_reserved_extent = true; if (!test_bit(BTRFS_ORDERED_NOCOW, &ordered_extent->flags) && !test_bit(BTRFS_ORDERED_PREALLOC, &ordered_extent->flags) &&
@@ -3095,10 +3096,12 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) logical_len, logical_len, compress_type,0 ,0 , BTRFS_FILE_EXTENT_REG); -if (!ret) +if (!ret) { + clear_reserved_extent = false;btrfs_release_delalloc_bytes (fs_info, ordered_extent->start, ordered_extent->disk_len); + } } unpin_extent_cache(&BTRFS_I(inode)->extent_tree, ordered_extent->file_offset, ordered_extent->len,
@@ -3159,8 +3162,13 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent)* wrong we need to return the space for this ordered extent * back to the allocator. We only free the extent in the * truncated case if we didn't write out the extent at all. +* +* If we made it past insert_reserved_file_extent before we +* errored out then we don't need to do this as the accounting +* has already been done. */ if ((ret || !logical_len) && + clear_reserved_extent && !test_bit (BTRFS_ORDERED_NOCOW , &ordered_extent->flags) && !test_bit (BTRFS_ORDERED_PREALLOC , &ordered_extent->flags)) btrfs_free_reserved_extent(fs_info,
Signature · 1 line
2.14.3
PATCH v3 33/42 btrfs: fix insert_reserved error handling
In reply to Josef BacikWe were not handling the reserved byte accounting properly for data references. Metadata was fine, if it errored out the error paths would free the bytes_reserved count and pin the extent, but it even missed one of the error cases. So instead move this handling up into run_one_delayed_ref so we are sure that both cases are properly cleaned up in case of a transaction abort.
--- fs/btrfs/extent-tree.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
fs/btrfs/extent-tree.c +4 -8
@@ -2405,6 +2405,9 @@ static int run_one_delayed_ref(struct btrfs_trans_handle *trans, insert_reserved); elseBUG (); +if (ret && insert_reserved) +btrfs_pin_extent (trans->fs_info, node->bytenr, + node->num_bytes,1 );return ret; }
@@ -8253,21 +8256,14 @@ static int alloc_reserved_tree_block(struct btrfs_trans_handle *trans, } path =btrfs_alloc_path (); -if (!path) { -btrfs_free_and_pin_reserved_extent (fs_info, - extent_key.objectid, - fs_info->nodesize); +if (!path)return -ENOMEM ; - } path->leave_spinning =1 ; ret =btrfs_insert_empty_item (trans, fs_info->extent_root, path, &extent_key, size);if (ret) {btrfs_free_path (path); -btrfs_free_and_pin_reserved_extent (fs_info, - extent_key.objectid, - fs_info->nodesize);return ret; }
Signature · 1 line
2.14.3
PATCH v3 34/42 btrfs: wait on ordered extents on abort cleanup
In reply to Josef BacikIf we flip read-only before we initiate writeback on all dirty pages for ordered extents we've created then we'll have ordered extents left over on umount, which results in all sorts of bad things happening. Fix this by making sure we wait on ordered extents if we have to do the aborted transaction cleanup stuff.
--- fs/btrfs/disk-io.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/btrfs/disk-io.c +8 -0
@@ -4188,6 +4188,14 @@ static void btrfs_destroy_all_ordered_extents(struct btrfs_fs_info *fs_info)spin_lock (&fs_info->ordered_root_lock); }spin_unlock (&fs_info->ordered_root_lock); + +/* +* We need this here because if we've been flipped read-only we won't +* get sync() from the umount, so we need to make sure any ordered +* extents that haven't had their dirty pages IO start writeout yet +* actually get run and error out properly. +*/ +btrfs_wait_ordered_roots (fs_info,U64_MAX ,0 , (u64)-1 ); } static int btrfs_destroy_delayed_refs(struct btrfs_transaction *trans,
Signature · 1 line
2.14.3
PATCH v3 35/42 MAINTAINERS: update my email address for btrfs
In reply to Josef BacikMy work email is completely useless, switch it to my personal address so I get emails on a account I actually pay attention to.
--- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
MAINTAINERS +1 -1
@@ -3095,7 +3095,7 @@ F: drivers/gpio/gpio-bt8xx.c BTRFS FILE SYSTEM M: Chris Mason <[email protected]> -M: Josef Bacik <[email protected]> +M: Josef Bacik <[email protected]> M: David Sterba <[email protected]> L: [email protected] W: http://btrfs.wiki.kernel.org/
Signature · 1 line
2.14.3
PATCH v3 36/42 btrfs: wait on caching when putting the bg cache
In reply to Josef BacikWhile testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is because we aren't waiting for the caching threads to be done before freeing everything up, so to fix this make sure we wait on any outstanding caching that's being done before we free up the block group, so we're sure to be done with all IO by the time we get to btrfs_stop_all_workers(). This fixes the panic I was seeing consistently in testing.
--- fs/btrfs/extent-tree.c | 1 + 1 file changed, 1 insertion(+)
fs/btrfs/extent-tree.c +1 -0
@@ -9890,6 +9890,7 @@ void btrfs_put_block_group_cache(struct btrfs_fs_info *info) block_group =btrfs_lookup_first_block_group (info, last);while (block_group) { +wait_block_group_cache_done (block_group);spin_lock (&block_group->lock);if (block_group->iref)break ;
Signature · 1 line
2.14.3
Re: [PATCH 36/42] btrfs: wait on caching when putting the bg cache
Quoted 10 lines · Josef Bacik
While testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is because we aren't waiting for the caching threads to be done before freeing everything up, so to fix this make sure we wait on any outstanding caching that's being done before we free up the block group, so we're sure to be done with all IO by the time we get to
btrfs_stop_all_workers(). This fixes the panic I was seeing consistently in testing.
Can you please attach the stacktrace?
I was running the tests 416/417/418 in a loop for a few days and did not reproduce the crash (in a VM) so this depends on the setup.
PATCH v3 37/42 btrfs: wakeup cleaner thread when adding delayed iput
In reply to Josef BacikThe cleaner thread usually takes care of delayed iputs, with the exception of the btrfs_end_transaction_throttle path. The cleaner thread only gets woken up every 30 seconds, so instead wake it up to do it's work so that we can free up that space as quickly as possible.
--- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+)
fs/btrfs/inode.c +1 -0
@@ -3323,6 +3323,7 @@ void btrfs_add_delayed_iput(struct inode *inode)ASSERT (list_empty (&binode->delayed_iput));list_add_tail (&binode->delayed_iput, &fs_info->delayed_iputs);spin_unlock (&fs_info->delayed_iput_lock); +wake_up_process (fs_info->cleaner_kthread); }void btrfs_run_delayed_iputs (struct btrfs_fs_info *fs_info)
Signature · 1 line
2.14.3
Re: [PATCH 37/42] btrfs: wakeup cleaner thread when adding delayed iput
Quoted 7 lines · Josef Bacik
The cleaner thread usually takes care of delayed iputs, with the exception of the
btrfs_end_transaction_throttlepath. The cleaner thread only gets woken up every 30 seconds, so instead wake it up to do it's work so that we can free up that space as quickly as possible.Signed-off-by Josef Bacik <[email protected]>
Quoted 19 lines
--- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+)fs/btrfs/inode.c +1
-1@@ -3323,6 +3323,7 @@ void btrfs_add_delayed_iput(struct inode *inode)ASSERT (list_empty (&binode->delayed_iput));list_add_tail (&binode->delayed_iput, &fs_info->delayed_iputs);spin_unlock (&fs_info->delayed_iput_lock); +wake_up_process (fs_info->cleaner_kthread); }void btrfs_run_delayed_iputs (struct btrfs_fs_info *fs_info) --2.14.3
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
PATCH v3 38/42 btrfs: be more explicit about allowed flush states
In reply to Josef BacikFor FLUSH_LIMIT flushers we really can only allocate chunks and flush delayed inode items, everything else is problematic. I added a bunch of new states and it lead to weirdness in the FLUSH_LIMIT case because I forgot about how it worked. So instead explicitly declare the states that are ok for flushing with FLUSH_LIMIT and use that for our state machine. Then as we add new things that are safe we can just add them to this list.
--- fs/btrfs/extent-tree.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-)
fs/btrfs/extent-tree.c +10 -11
@@ -5110,12 +5110,18 @@ void btrfs_init_async_reclaim_work(struct work_struct *work)INIT_WORK (work, btrfs_async_reclaim_metadata_space); } +static const enum btrfs_flush_state priority_flush_states[] = { +FLUSH_DELAYED_ITEMS_NR , +FLUSH_DELAYED_ITEMS , +ALLOC_CHUNK , +}; +static void priority_reclaim_metadata_space (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,struct reserve_ticket *ticket) {u64 to_reclaim; -int flush_state =FLUSH_DELAYED_ITEMS_NR ; +int flush_state =0 ;spin_lock (&space_info->lock); to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info,
@@ -5127,7 +5133,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,spin_unlock (&space_info->lock); do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, + priority_flush_states[flush_state]); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) {
@@ -5135,15 +5142,7 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,return ; }spin_unlock (&space_info->lock); - -/* -* Priority flushers can't wait on delalloc without -* deadlocking. -*/ -if (flush_state ==FLUSH_DELALLOC || - flush_state ==FLUSH_DELALLOC_WAIT ) - flush_state =ALLOC_CHUNK ; - }while (flush_state <COMMIT_TRANS ); + }while (flush_state <ARRAY_SIZE (priority_flush_states)); } static int wait_reserve_ticket(struct btrfs_fs_info *fs_info,
Signature · 1 line
2.14.3
PATCH v3 39/42 btrfs: replace cleaner_delayed_iput_mutex with a waitqueue
In reply to Josef BacikThe throttle path doesn't take cleaner_delayed_iput_mutex, which means we could think we're done flushing iputs in the data space reservation path when we could have a throttler doing an iput. There's no real reason to serialize the delayed iput flushing, so instead of taking the cleaner_delayed_iput_mutex whenever we flush the delayed iputs just replace it with an atomic counter and a waitqueue. This removes the short (or long depending on how big the inode is) window where we think there are no more pending iputs when there really are some.
--- fs/btrfs/ctree.h | 4 +++- fs/btrfs/disk-io.c | 5 ++--- fs/btrfs/extent-tree.c | 9 +++++---- fs/btrfs/inode.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 8 deletions(-)
fs/btrfs/ctree.h +3 -1
@@ -894,7 +894,8 @@ struct btrfs_fs_info {spinlock_t delayed_iput_lock;struct list_head delayed_iputs; -struct mutex cleaner_delayed_iput_mutex; +atomic_t nr_delayed_iputs; +wait_queue_head_t delayed_iputs_wait;/* this protects tree_mod_seq_list */ spinlock_t tree_mod_seq_lock;
@@ -3212,6 +3213,7 @@ int btrfs_orphan_cleanup(struct btrfs_root *root);int btrfs_cont_expand (struct inode *inode,loff_t oldsize,loff_t size);void btrfs_add_delayed_iput (struct inode *inode);void btrfs_run_delayed_iputs (struct btrfs_fs_info *fs_info); +int btrfs_wait_on_delayed_iputs (struct btrfs_fs_info *fs_info);int btrfs_prealloc_file_range (struct inode *inode,int mode,u64 start,u64 num_bytes,u64 min_size,loff_t actual_len,u64 *alloc_hint);
fs/btrfs/disk-io.c +2 -3
@@ -1692,9 +1692,7 @@ static int cleaner_kthread(void *arg) goto sleep; } -mutex_lock (&fs_info->cleaner_delayed_iput_mutex);btrfs_run_delayed_iputs (fs_info); -mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); again =btrfs_clean_one_deleted_snapshot (root);mutex_unlock (&fs_info->cleaner_mutex);
@@ -2677,7 +2675,6 @@ int open_ctree(struct super_block *sb,mutex_init (&fs_info->delete_unused_bgs_mutex);mutex_init (&fs_info->reloc_mutex);mutex_init (&fs_info->delalloc_root_mutex); -mutex_init (&fs_info->cleaner_delayed_iput_mutex);seqlock_init (&fs_info->profiles_lock);INIT_LIST_HEAD (&fs_info->dirty_cowonly_roots);
@@ -2699,6 +2696,7 @@ int open_ctree(struct super_block *sb,atomic_set (&fs_info->defrag_running,0 );atomic_set (&fs_info->qgroup_op_seq,0 );atomic_set (&fs_info->reada_works_cnt,0 ); +atomic_set (&fs_info->nr_delayed_iputs,0 );atomic64_set (&fs_info->tree_mod_seq,0 ); fs_info->sb = sb; fs_info->max_inline =BTRFS_DEFAULT_MAX_INLINE ;
@@ -2776,6 +2774,7 @@ int open_ctree(struct super_block *sb,init_waitqueue_head (&fs_info->transaction_wait);init_waitqueue_head (&fs_info->transaction_blocked_wait);init_waitqueue_head (&fs_info->async_submit_wait); +init_waitqueue_head (&fs_info->delayed_iputs_wait);INIT_LIST_HEAD (&fs_info->pinned_chunks);
fs/btrfs/extent-tree.c +5 -4
@@ -4258,8 +4258,9 @@ int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes)* operations. Wait for it to finish so that * more space is released. */ -mutex_lock (&fs_info->cleaner_delayed_iput_mutex); -mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); + ret =btrfs_wait_on_delayed_iputs (fs_info); +if (ret) +return ret; goto again; }else {btrfs_end_transaction (trans);
@@ -4829,9 +4830,9 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,* pinned space, so make sure we run the iputs before we do our pinned * bytes check below. */ -mutex_lock (&fs_info->cleaner_delayed_iput_mutex);btrfs_run_delayed_iputs (fs_info); -mutex_unlock (&fs_info->cleaner_delayed_iput_mutex); +wait_event (fs_info->delayed_iputs_wait, +atomic_read (&fs_info->nr_delayed_iputs) ==0 ); trans =btrfs_join_transaction (fs_info->extent_root);if (IS_ERR (trans))
fs/btrfs/inode.c +21 -0
@@ -3319,6 +3319,7 @@ void btrfs_add_delayed_iput(struct inode *inode)if (atomic_add_unless (&inode->i_count,-1 ,1 ))return ; +atomic_inc (&fs_info->nr_delayed_iputs);spin_lock (&fs_info->delayed_iput_lock);ASSERT (list_empty (&binode->delayed_iput));list_add_tail (&binode->delayed_iput, &fs_info->delayed_iputs);
@@ -3338,11 +3339,31 @@ void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info)list_del_init (&inode->delayed_iput);spin_unlock (&fs_info->delayed_iput_lock);iput (&inode->vfs_inode); +if (atomic_dec_and_test (&fs_info->nr_delayed_iputs)) +wake_up (&fs_info->delayed_iputs_wait);spin_lock (&fs_info->delayed_iput_lock); }spin_unlock (&fs_info->delayed_iput_lock); } +/** +* btrfs_wait_on_delayed_iputs - wait on the delayed iputs to be done running +* @fs_info - the fs_info for this fs +* @return - EINTR if we were killed, 0 if nothing's pending +* +* This will wait on any delayed iputs that are currently running with KILLABLE +* set. Once they are all done running we will return, unless we are killed in +* which case we return EINTR. +*/ +int btrfs_wait_on_delayed_iputs (struct btrfs_fs_info *fs_info) +{ +int ret =wait_event_killable (fs_info->delayed_iputs_wait, +atomic_read (&fs_info->nr_delayed_iputs) ==0 ); +if (ret) +return -EINTR ; +return 0 ; +} +/* * This creates an orphan entry for the given inode in case something goes wrong * in the middle of an unlink.
Signature · 1 line
2.14.3
PATCH v3 40/42 btrfs: drop min_size from evict_refill_and_join
In reply to Josef BacikWe don't need it, rsv->size is set once and never changes throughout its lifetime, so just use that for the reserve size.
--- fs/btrfs/inode.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
fs/btrfs/inode.c +6 -10
@@ -5339,8 +5339,7 @@ static void evict_inode_truncate_pages(struct inode *inode) }static struct btrfs_trans_handle *evict_refill_and_join (struct btrfs_root *root, -struct btrfs_block_rsv *rsv, -u64 min_size) +struct btrfs_block_rsv *rsv) {struct btrfs_fs_info *fs_info = root->fs_info;struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
@@ -5350,7 +5349,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,struct btrfs_trans_handle *trans;int ret; - ret =btrfs_block_rsv_refill (root, rsv, min_size, + ret =btrfs_block_rsv_refill (root, rsv, rsv->size,BTRFS_RESERVE_FLUSH_LIMIT );if (ret && ++failures >2 ) {
@@ -5368,7 +5367,7 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,* it. */ if (!btrfs_check_space_for_delayed_refs (fs_info) && - !btrfs_block_rsv_migrate (global_rsv, rsv, min_size,0 )) + !btrfs_block_rsv_migrate (global_rsv, rsv, rsv->size,0 ))return trans;/* If not, commit and try again. */
@@ -5384,7 +5383,6 @@ void btrfs_evict_inode(struct inode *inode)struct btrfs_trans_handle *trans;struct btrfs_root *root =BTRFS_I (inode)->root;struct btrfs_block_rsv *rsv; -u64 min_size;int ret;trace_btrfs_inode_evict (inode);
@@ -5394,8 +5392,6 @@ void btrfs_evict_inode(struct inode *inode)return ; } - min_size =btrfs_calc_trunc_metadata_size (fs_info,1 ); -evict_inode_truncate_pages (inode); if (inode->i_nlink &&
@@ -5428,13 +5424,13 @@ void btrfs_evict_inode(struct inode *inode) rsv =btrfs_alloc_block_rsv (fs_info,BTRFS_BLOCK_RSV_TEMP );if (!rsv) goto no_delete; - rsv->size = min_size; + rsv->size =btrfs_calc_trunc_metadata_size (fs_info,1 ); rsv->failfast =1 ;btrfs_i_size_write (BTRFS_I (inode),0 );while (1 ) { - trans =evict_refill_and_join (root, rsv, min_size); + trans =evict_refill_and_join (root, rsv);if (IS_ERR (trans)) goto free_rsv;
@@ -5459,7 +5455,7 @@ void btrfs_evict_inode(struct inode *inode)* If it turns out that we are dropping too many of these, we might want * to add a mechanism for retrying these after a commit. */ - trans =evict_refill_and_join (root, rsv, min_size); + trans =evict_refill_and_join (root, rsv);if (!IS_ERR (trans)) { trans->block_rsv = rsv;btrfs_orphan_del (trans,BTRFS_I (inode));
Signature · 1 line
2.14.3
Re: [PATCH 40/42] btrfs: drop min_size from evict_refill_and_join
Quoted 4 lines · Josef Bacik
We don't need it,
rsv->sizeis set once and never changes throughout its lifetime, so just use that for the reserve size.Signed-off-by Josef Bacik <[email protected]>
PATCH v3 41/42 btrfs: reserve extra space during evict()
In reply to Josef BacikWe could generate a lot of delayed refs in evict but never have any left over space from our block rsv to make up for that fact. So reserve some extra space and give it to the transaction so it can be used to refill the delayed refs rsv every loop through the truncate path.
--- fs/btrfs/inode.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +23 -2
@@ -5343,13 +5343,15 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root, {struct btrfs_fs_info *fs_info = root->fs_info;struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv; +u64 delayed_refs_extra =btrfs_calc_trans_metadata_size (fs_info,1 );int failures =0 ;for (;;) {struct btrfs_trans_handle *trans;int ret; - ret =btrfs_block_rsv_refill (root, rsv, rsv->size, + ret =btrfs_block_rsv_refill (root, rsv, + rsv->size + delayed_refs_extra,BTRFS_RESERVE_FLUSH_LIMIT );if (ret && ++failures >2 ) {
@@ -5358,9 +5360,28 @@ static struct btrfs_trans_handle *evict_refill_and_join(struct btrfs_root *root,return ERR_PTR (-ENOSPC ); } +/* +* Evict can generate a large amount of delayed refs without +* having a way to add space back since we exhaust our temporary +* block rsv. We aren't allowed to do FLUSH_ALL in this case +* because we could deadlock with so many things in the flushing +* code, so we have to try and hold some extra space to +* compensate for our delayed ref generation. If we can't get +* that space then we need see if we can steal our minimum from +* the global reserve. We will be ratelimited by the amount of +* space we have for the delayed refs rsv, so we'll end up +* committing and trying again. +*/ trans =btrfs_join_transaction (root); -if (IS_ERR (trans) || !ret) +if (IS_ERR (trans) || !ret) { +if (!IS_ERR (trans)) { + trans->block_rsv = &fs_info->trans_block_rsv; + trans->bytes_reserved = delayed_refs_extra; +btrfs_block_rsv_migrate (rsv, trans->block_rsv, + delayed_refs_extra,1 ); + }return trans; + }/* * Try to steal from the global reserve if there is space for
Signature · 1 line
2.14.3
PATCH v3 42/42 btrfs: don't run delayed_iputs in commit
In reply to Josef BacikThis could result in a really bad case where we do something like
evict
evict_refill_and_join
btrfs_commit_transaction
btrfs_run_delayed_iputs
evict
evict_refill_and_join
btrfs_commit_transaction
... forever
We have plenty of other places where we run delayed iputs that are much safer, let those do the work.
--- fs/btrfs/transaction.c | 9 --------- 1 file changed, 9 deletions(-)
fs/btrfs/transaction.c +0 -9
@@ -2265,15 +2265,6 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans)kmem_cache_free (btrfs_trans_handle_cachep, trans); -/* -* If fs has been frozen, we can not handle delayed iputs, otherwise -* it'll result in deadlock about SB_FREEZE_FS. -*/ -if (current != fs_info->transaction_kthread && - current != fs_info->cleaner_kthread && - !test_bit (BTRFS_FS_FROZEN , &fs_info->flags)) -btrfs_run_delayed_iputs (fs_info); -return ret; scrub_continue:
Signature · 1 line
2.14.3