Throttle delayed refs based on time
cover Throttle delayed refs based on time
Zygo reported a problem on IRC where he was seeing multi-hour long latencies with his test rig with transaction commits. This turned out to be because his test rig runs rsync, balance, snapshot create and delete, dedup, a infinite loop of mkdir/rmdirs, and I'm sure some other horrors I'm forgetting.
When I added the delayed refs reserve, I assumed that the space pressure generated by generating a lot of delayed refs would result in transactions being ended if they needed to be, and thus we no longer needed to throttle delayed refs based on time.
This assumption was wrong, because in Zygo's case he has a multi terabyte file system, so overcommit allows him to generate as many delayed refs as he wants. This meant that he would need to run hundreds of thousands of delayed refs at commit time. To make matters worse, we didn't have a way to stop people from generating more delayed refs, so the transaction commit could be held open indefinitely by balance and snapshot delete. This is how we were getting transaction commits happening every few hours.
The solution to this problem is to bring back the time based delayed ref flushing, and then add the ability for people to throttle themselves on that.
Balance and truncate already had this ability, it only needed to be added to snapshot delete.
I've also added back the async delayed ref flushing, and I've added code to help throttle people when we're generating delayed refs too fast for the system to keep up with them.
This whole patch queue has been running in some form or another on Zygo's awful test bed, and appears to be performing better than it was before I ripped out the original code. Thanks,
Josef
PATCH 1/13 btrfs: use a stable rolling avg for delayed refs avg
From: Zygo Blaxell <[email protected]>
The way we currently calculate the average of delayed refs is very jittery. We do a weighted average, weighing the new average at 3/4 of the previously calculated average.
This jitteriness leads to pretty wild swings in latencies when we are generating a lot of delayed refs. Fix this by smoothing out the delayed ref average calculation with 1000 seconds of data, 1000 refs minimum, with a 0.75 decay rate.
--- fs/btrfs/ctree.h | 7 +++++++ fs/btrfs/disk-io.c | 3 +++ fs/btrfs/extent-tree.c | 19 +++++++++++++++++-- 3 files changed, 27 insertions(+), 2 deletions(-)
fs/btrfs/ctree.h +7 -0
@@ -620,7 +620,14 @@ struct btrfs_fs_info {u64 generation;u64 last_trans_committed; + +/* +* This is for keeping track of how long it takes to run delayed refs so +* that our delayed ref timing doesn't hurt us. +*/ u64 avg_delayed_ref_runtime; +u64 delayed_ref_runtime; +u64 delayed_ref_nr_run;/* * this is updated to the current trans every time a full commit
fs/btrfs/disk-io.c +3 -0
@@ -2734,6 +2734,9 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info) fs_info->tree_mod_log =RB_ROOT ; fs_info->commit_interval =BTRFS_DEFAULT_COMMIT_INTERVAL ; fs_info->avg_delayed_ref_runtime =NSEC_PER_SEC >>6 ;/* div by 64 */ + fs_info->delayed_ref_runtime =NSEC_PER_SEC ; + fs_info->delayed_ref_nr_run =64 ; +/* readahead state */ INIT_RADIX_TREE (&fs_info->reada_tree,GFP_NOFS & ~__GFP_DIRECT_RECLAIM);spin_lock_init (&fs_info->reada_lock);
fs/btrfs/extent-tree.c +17 -2
@@ -2082,8 +2082,23 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,* to avoid large swings in the average. */ spin_lock (&delayed_refs->lock); - avg = fs_info->avg_delayed_ref_runtime *3 + runtime; - fs_info->avg_delayed_ref_runtime = avg >>2 ;/* div by 4 */ + fs_info->delayed_ref_nr_run += actual_count; + fs_info->delayed_ref_runtime += runtime; + avg =div64_u64 (fs_info->delayed_ref_runtime, + fs_info->delayed_ref_nr_run); + +/* +* Once we've built up a fair bit of data, start decaying +* everything by 3/4. +*/ +if (fs_info->delayed_ref_runtime >= (NSEC_PER_SEC *1000ULL ) && + fs_info->delayed_ref_nr_run >1000 ) { + fs_info->delayed_ref_runtime *=3 ; + fs_info->delayed_ref_runtime >>=2 ; + fs_info->delayed_ref_nr_run *=3 ; + fs_info->delayed_ref_nr_run >>=2 ; + } + fs_info->avg_delayed_ref_runtime = avg;spin_unlock (&delayed_refs->lock); }return 0 ;
Signature · 1 line
2.24.1
PATCH 2/13 btrfs: change btrfs_should_throttle_delayed_refs to a bool
In reply to Josef BacikWe don't actually check the specific value from btrfs_should_throttle_delayed_refs anywhere, just return a bool.
--- fs/btrfs/delayed-ref.c | 6 +++--- fs/btrfs/delayed-ref.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
fs/btrfs/delayed-ref.c +3 -3
@@ -50,7 +50,7 @@ bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info)return ret; } -int btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans) +bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans) {u64 num_entries =atomic_read (&trans->transaction->delayed_refs.num_entries);
@@ -61,9 +61,9 @@ int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans) avg_runtime = trans->fs_info->avg_delayed_ref_runtime; val = num_entries * avg_runtime;if (val >=NSEC_PER_SEC ) -return 1 ; +return true;if (val >=NSEC_PER_SEC /2 ) -return 2 ; +return true;return btrfs_check_space_for_delayed_refs (trans->fs_info); }
fs/btrfs/delayed-ref.h +1 -1
@@ -371,7 +371,7 @@ int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *src,u64 num_bytes); -int btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans); +bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans);bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);/*
Signature · 1 line
2.24.1
PATCH 3/13 btrfs: make btrfs_should_throttle_delayed_refs only check run time
In reply to Josef Bacikbtrfs_should_throttle_delayed_refs checks run time of the delayed refs and if there's enough space. However we want to use these two checks independently in the future, so make btrfs_should_throttle_delayed_refs only check the runtime. Then fix the only caller of btrfs_should_throttle_delayed_refs to check the space as well, because we want to throttle truncates on either space or time.
--- fs/btrfs/delayed-ref.c | 3 +-- fs/btrfs/inode.c | 3 ++- 2 files changed, 3 insertions(+), 3 deletions(-)
fs/btrfs/delayed-ref.c +1 -2
@@ -64,8 +64,7 @@ bool btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans)return true;if (val >=NSEC_PER_SEC /2 )return true; - -return btrfs_check_space_for_delayed_refs (trans->fs_info); +return false; }/**
fs/btrfs/inode.c +2 -1
@@ -4349,7 +4349,8 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,break ; }if (be_nice) { -if (btrfs_should_throttle_delayed_refs (trans)) +if (btrfs_should_throttle_delayed_refs (trans) || +btrfs_check_space_for_delayed_refs (fs_info)) should_throttle = true; } }
Signature · 1 line
2.24.1
PATCH 4/13 btrfs: make should_end_transaction check time to run delayed refs
In reply to Josef BacikCurrently snapshot deletion checks to see if it needs to throttle itself before ending a transaction, however this only checks if there's enough space for delayed refs, not how much time it'll take to run those delayed refs. Fix this by checking btrfs_should_throttle_delayed_refs as well, which takes into account how much time it'll take to run delayed refs.
--- fs/btrfs/transaction.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/btrfs/transaction.c +2 -1
@@ -859,7 +859,8 @@ static int should_end_transaction(struct btrfs_trans_handle *trans) {struct btrfs_fs_info *fs_info = trans->fs_info; -if (btrfs_check_space_for_delayed_refs (fs_info)) +if (btrfs_should_throttle_delayed_refs (trans) || +btrfs_check_space_for_delayed_refs (fs_info))return 1 ;return !!btrfs_block_rsv_check (&fs_info->global_block_rsv,5 );
Signature · 1 line
2.24.1
PATCH 5/13 btrfs: squash should_end_transaction
In reply to Josef BacikWe used to call should_end_transaction() in __btrfs_end_transaction, but we no longer do that and it's a tiny function, so squash it into btrfs_should_end_transaction.
--- fs/btrfs/transaction.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-)
fs/btrfs/transaction.c +6 -12
@@ -855,27 +855,21 @@ void btrfs_throttle(struct btrfs_fs_info *fs_info)wait_current_trans (fs_info); } -static int should_end_transaction (struct btrfs_trans_handle *trans) -{ -struct btrfs_fs_info *fs_info = trans->fs_info; - -if (btrfs_should_throttle_delayed_refs (trans) || -btrfs_check_space_for_delayed_refs (fs_info)) -return 1 ; - -return !!btrfs_block_rsv_check (&fs_info->global_block_rsv,5 ); -} -int btrfs_should_end_transaction (struct btrfs_trans_handle *trans) {struct btrfs_transaction *cur_trans = trans->transaction; +struct btrfs_fs_info *fs_info = trans->fs_info;smp_mb ();if (cur_trans->state >=TRANS_STATE_COMMIT_START || cur_trans->delayed_refs.flushing)return 1 ; -return should_end_transaction (trans); +if (btrfs_should_throttle_delayed_refs (trans) || +btrfs_check_space_for_delayed_refs (fs_info)) +return 1 ; + +return !!btrfs_block_rsv_check (&fs_info->global_block_rsv,5 ); }static void btrfs_trans_release_metadata (struct btrfs_trans_handle *trans)
Signature · 1 line
2.24.1
PATCH 6/13 btrfs: add a mode for delayed ref time based throttling
In reply to Josef BacikCurrently we only use btrfs_should_throttle_delayed_refs in the case where we want to pre-emptively stop what we're doing and throttle delayed refs in some way. However we're going to use this function for all transaction ending, so add a flag so we can toggle between the maximum theoretical runtime and our "maybe we should start flushing" runtime.
--- fs/btrfs/delayed-ref.c | 9 +++++---- fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/inode.c | 3 ++- fs/btrfs/transaction.c | 2 +- 4 files changed, 10 insertions(+), 7 deletions(-)
fs/btrfs/delayed-ref.c +5 -4
@@ -50,7 +50,8 @@ bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info)return ret; } -bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans) +bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans, +bool for_throttle) {u64 num_entries =atomic_read (&trans->transaction->delayed_refs.num_entries);
@@ -62,9 +63,9 @@ bool btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans) val = num_entries * avg_runtime;if (val >=NSEC_PER_SEC )return true; -if (val >=NSEC_PER_SEC /2 ) -return true; -return false; +if (!for_throttle) +return false; +return (val >=NSEC_PER_SEC /2 ); }/**
fs/btrfs/delayed-ref.h +2 -1
@@ -371,7 +371,8 @@ int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *src,u64 num_bytes); -bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans); +bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans, +bool for_throttle);bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);/*
fs/btrfs/inode.c +2 -1
@@ -4349,7 +4349,8 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,break ; }if (be_nice) { -if (btrfs_should_throttle_delayed_refs (trans) || +if (btrfs_should_throttle_delayed_refs (trans, + true) ||btrfs_check_space_for_delayed_refs (fs_info)) should_throttle = true; }
fs/btrfs/transaction.c +1 -1
@@ -865,7 +865,7 @@ int btrfs_should_end_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing)return 1 ; -if (btrfs_should_throttle_delayed_refs (trans) || +if (btrfs_should_throttle_delayed_refs (trans, true) ||btrfs_check_space_for_delayed_refs (fs_info))return 1 ;
Signature · 1 line
2.24.1
PATCH 7/13 btrfs: kick off async delayed ref flushing if we are over time budget
In reply to Josef BacikFor very large file systems we cannot rely on the space reservation system to provide enough pressure to flush delayed refs in a timely manner. We have the infrastructure in place to keep track of how much theoretical time it'll take to run our outstanding delayed refs, but unfortunately I ripped all of that out when I added the delayed refs rsv. This code originally was added to address the problem of too many delayed refs building up and thus causing transaction commits to take several minutes to finish.
Fix this by adding back the ability to flush delayed refs based on the time budget for them. We want to limit to around 1 seconds worth of delayed refs to be pending at any given time. In order to keep up with demand we will start the async flusher once we are at the 500ms mark, and the async flusher will attempt to keep us in this ballpark.
--- fs/btrfs/ctree.h | 4 ++++ fs/btrfs/disk-io.c | 3 +++ fs/btrfs/extent-tree.c | 44 ++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/transaction.c | 8 ++++++++ 4 files changed, 59 insertions(+)
fs/btrfs/ctree.h +4 -0
@@ -494,6 +494,7 @@ enum btrfs_orphan_cleanup_state { };void btrfs_init_async_reclaim_work (struct btrfs_fs_info *fs_info); +void btrfs_init_async_delayed_ref_work (struct btrfs_fs_info *fs_info);/* fs_info */ struct reloc_control ;
@@ -924,6 +925,9 @@ struct btrfs_fs_info {struct work_struct async_reclaim_work;struct work_struct async_data_reclaim_work; +/* Used to run delayed refs in the background. */ +struct work_struct async_delayed_ref_work; +spinlock_t unused_bgs_lock;struct list_head unused_bgs;struct mutex unused_bg_unpin_mutex;
fs/btrfs/disk-io.c +3 -0
@@ -2754,6 +2754,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)#endif btrfs_init_balance (fs_info);btrfs_init_async_reclaim_work (fs_info); +btrfs_init_async_delayed_ref_work (fs_info);spin_lock_init (&fs_info->block_group_cache_lock); fs_info->block_group_cache_tree =RB_ROOT ;
@@ -3997,6 +3998,8 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info)*/ kthread_park (fs_info->cleaner_kthread); +cancel_work_sync (&fs_info->async_delayed_ref_work); +/* wait for the qgroup rescan worker to stop */ btrfs_qgroup_wait_for_completion (fs_info, false);
fs/btrfs/extent-tree.c +44 -0
50 lines not on this page · raw message
fs/btrfs/transaction.c +8 -0
@@ -899,6 +899,7 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,struct btrfs_fs_info *info = trans->fs_info;struct btrfs_transaction *cur_trans = trans->transaction;int err =0 ; +bool run_async = false;if (refcount_read (&trans->use_count) >1 ) {refcount_dec (&trans->use_count);
@@ -906,6 +907,9 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,return 0 ; } +if (btrfs_should_throttle_delayed_refs (trans, true)) + run_async = true; +btrfs_trans_release_metadata (trans); trans->block_rsv =NULL ;
@@ -936,6 +940,10 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, err = -EIO ; } +if (run_async && !work_busy (&info->async_delayed_ref_work)) +queue_work (system_unbound_wq, + &info->async_delayed_ref_work); +kmem_cache_free (btrfs_trans_handle_cachep, trans);return err; }
Signature · 1 line
2.24.1
Re: [PATCH 07/13] btrfs: kick off async delayed ref flushing if we are over time budget
Quoted 23 lines · Josef Bacik
For very large file systems we cannot rely on the space reservation system to provide enough pressure to flush delayed refs in a timely manner. We have the infrastructure in place to keep track of how much theoretical time it'll take to run our outstanding delayed refs, but unfortunately I ripped all of that out when I added the delayed refs rsv. This code originally was added to address the problem of too many delayed refs building up and thus causing transaction commits to take several minutes to finish.
Fix this by adding back the ability to flush delayed refs based on the time budget for them. We want to limit to around 1 seconds worth of delayed refs to be pending at any given time. In order to keep up with demand we will start the async flusher once we are at the 500ms mark, and the async flusher will attempt to keep us in this ballpark.
Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/ctree.h | 4 ++++ fs/btrfs/disk-io.c | 3 +++ fs/btrfs/extent-tree.c | 44 ++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/transaction.c | 8 ++++++++ 4 files changed, 59 insertions(+)
<snip>
Quoted 20 lines
fs/btrfs/extent-tree.c +19
-0+static void btrfs_async_run_delayed_refs (struct work_struct *work) +{ +struct btrfs_fs_info *fs_info; +struct btrfs_trans_handle *trans; + + fs_info = container_of(work, struct btrfs_fs_info, + async_delayed_ref_work); + +while (!btrfs_fs_closing (fs_info)) { +unsigned long count; +int ret; + + trans =btrfs_attach_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +break ; + +smp_rmb (); +if (trans->transaction->delayed_refs.flushing) { +btrfs_end_transaction (trans);
fs/btrfs/extent-tree.c int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, · line 2280
Quoted 10 lines
fs/btrfs/extent-tree.c +10
-0+break ; + } + +/* No longer over our threshold, lets bail. */ +if (!btrfs_should_throttle_delayed_refs (trans, true)) { +btrfs_end_transaction (trans); +break ; + } + + count =atomic_read (&trans->transaction->delayed_refs.num_entries);
Don't you want to actually read num_heads_ready rather than num_entries, i.e isn't this introducing the same issues as the one fixed by:
[PATCH 2/5] btrfs: delayed refs pre-flushing should only run the heads we have
Quoted 8 lines
fs/btrfs/extent-tree.c +8
-0+ count >>=2 ; + + ret =btrfs_run_delayed_refs (trans, count); +btrfs_end_transaction (trans); +if (ret <0 ) +break ; + } +}
<snip>
Re: [PATCH 07/13] btrfs: kick off async delayed ref flushing if we are over time budget
In reply to Josef BacikQuoted 23 lines · Josef Bacik
For very large file systems we cannot rely on the space reservation system to provide enough pressure to flush delayed refs in a timely manner. We have the infrastructure in place to keep track of how much theoretical time it'll take to run our outstanding delayed refs, but unfortunately I ripped all of that out when I added the delayed refs rsv. This code originally was added to address the problem of too many delayed refs building up and thus causing transaction commits to take several minutes to finish.
Fix this by adding back the ability to flush delayed refs based on the time budget for them. We want to limit to around 1 seconds worth of delayed refs to be pending at any given time. In order to keep up with demand we will start the async flusher once we are at the 500ms mark, and the async flusher will attempt to keep us in this ballpark.
Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/ctree.h | 4 ++++ fs/btrfs/disk-io.c | 3 +++ fs/btrfs/extent-tree.c | 44 ++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/transaction.c | 8 ++++++++ 4 files changed, 59 insertions(+)
<snip>
Quoted 15 lines
fs/btrfs/extent-tree.c +7
-0@@ -2249,6 +2249,50 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,return 0 ; } +static void btrfs_async_run_delayed_refs (struct work_struct *work) +{ +struct btrfs_fs_info *fs_info; +struct btrfs_trans_handle *trans; + + fs_info = container_of(work, struct btrfs_fs_info, + async_delayed_ref_work);
fs/btrfs/extent-tree.c int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, · line 2268
Quoted 10 lines
fs/btrfs/extent-tree.c +10
-0+ +while (!btrfs_fs_closing (fs_info)) { +unsigned long count; +int ret; + + trans =btrfs_attach_transaction (fs_info->extent_root); +if (IS_ERR (trans)) +break ; + +smp_rmb ();
What is this barrier ordering? IMO its usage is bogus here, because in btrfs_should_end_transaction we use a full barrier and here only an RMB. Further more in btrfs_should_end_transaction we don't have any memory accesses preceding the check of the flushing state. Looking at the callers of btrfs_should_end_transaction I also don't see any ordering guaranteed i.e I think it could be removed altogether. Or perhahps we really want acquire/release semantics e.g. accesses to delayed_refs.flushing should be done via smp_load_acquire/smp_store_release functions?
Quoted 21 lines
fs/btrfs/extent-tree.c +21
-0+if (trans->transaction->delayed_refs.flushing) { +btrfs_end_transaction (trans); +break ; + } + +/* No longer over our threshold, lets bail. */ +if (!btrfs_should_throttle_delayed_refs (trans, true)) { +btrfs_end_transaction (trans); +break ; + } + + count =atomic_read (&trans->transaction->delayed_refs.num_entries); + count >>=2 ; + + ret =btrfs_run_delayed_refs (trans, count); +btrfs_end_transaction (trans); +if (ret <0 ) +break ; + } +} +
<snip>
PATCH 8/13 btrfs: adjust the arguments for btrfs_should_throttle_delayed_refs
In reply to Josef BacikWe want to be able to call this without a trans handle being open, so adjust the arguments to be the fs_info and the delayed_ref_root instead of the trans handle.
--- fs/btrfs/delayed-ref.c | 8 ++++---- fs/btrfs/delayed-ref.h | 3 ++- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/inode.c | 5 +++-- fs/btrfs/transaction.c | 5 +++-- 5 files changed, 13 insertions(+), 10 deletions(-)
fs/btrfs/delayed-ref.c +4 -4
@@ -50,16 +50,16 @@ bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info)return ret; } -bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans, +bool btrfs_should_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_root *delayed_refs,bool for_throttle) { -u64 num_entries = -atomic_read (&trans->transaction->delayed_refs.num_entries); +u64 num_entries =atomic_read (&delayed_refs->num_entries);u64 avg_runtime;u64 val;smp_mb (); - avg_runtime = trans->fs_info->avg_delayed_ref_runtime; + avg_runtime = fs_info->avg_delayed_ref_runtime; val = num_entries * avg_runtime;if (val >=NSEC_PER_SEC )return true;
fs/btrfs/delayed-ref.h +2 -1
@@ -371,7 +371,8 @@ int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,void btrfs_migrate_to_delayed_refs_rsv (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *src,u64 num_bytes); -bool btrfs_should_throttle_delayed_refs (struct btrfs_trans_handle *trans, +bool btrfs_should_throttle_delayed_refs (struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_root *delayed_refs,bool for_throttle);bool btrfs_check_space_for_delayed_refs (struct btrfs_fs_info *fs_info);
fs/btrfs/extent-tree.c +1 -1
@@ -2272,7 +2272,7 @@ static void btrfs_async_run_delayed_refs(struct work_struct *work) }/* No longer over our threshold, lets bail. */ -if (!btrfs_should_throttle_delayed_refs (trans, true)) { +if (!btrfs_should_throttle_delayed_refs (fs_info, &trans->transaction->delayed_refs, true)) {btrfs_end_transaction (trans);break ; }
fs/btrfs/inode.c +3 -2
@@ -4349,8 +4349,9 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,break ; }if (be_nice) { -if (btrfs_should_throttle_delayed_refs (trans, - true) || +if (btrfs_should_throttle_delayed_refs (fs_info, + &trans->transaction->delayed_refs, + true) ||btrfs_check_space_for_delayed_refs (fs_info)) should_throttle = true; }
fs/btrfs/transaction.c +3 -2
@@ -865,7 +865,7 @@ int btrfs_should_end_transaction(struct btrfs_trans_handle *trans) cur_trans->delayed_refs.flushing)return 1 ; -if (btrfs_should_throttle_delayed_refs (trans, true) || +if (btrfs_should_throttle_delayed_refs (fs_info, &cur_trans->delayed_refs, true) ||btrfs_check_space_for_delayed_refs (fs_info))return 1 ;
@@ -907,7 +907,8 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,return 0 ; } -if (btrfs_should_throttle_delayed_refs (trans, true)) +if (btrfs_should_throttle_delayed_refs (info, + &cur_trans->delayed_refs, true)) run_async = true;btrfs_trans_release_metadata (trans);
Signature · 1 line
2.24.1
PATCH 9/13 btrfs: throttle delayed refs based on time
In reply to Josef BacikWe need to make sure we don't generate so many delayed refs that the box gets overwhelmed at commit time. Keep a sequence number of the number of entries run, and if we need to be throttled based on our time constraints wait for the number of delayed refs we added to be run once we end our transaction.
--- fs/btrfs/delayed-ref.c | 6 ++++-- fs/btrfs/delayed-ref.h | 14 ++++++++++++ fs/btrfs/extent-tree.c | 4 +++- fs/btrfs/transaction.c | 49 +++++++++++++++++++++++++++++++++++++++--- fs/btrfs/transaction.h | 16 ++++++++++++++ 5 files changed, 83 insertions(+), 6 deletions(-)
fs/btrfs/delayed-ref.c +4 -2
@@ -424,7 +424,7 @@ static inline void drop_delayed_ref(struct btrfs_trans_handle *trans,list_del (&ref->add_list); ref->in_tree =0 ;btrfs_put_delayed_ref (ref); -atomic_dec (&delayed_refs->num_entries); +btrfs_dec_delayed_ref_entries (delayed_refs); } static bool merge_ref(struct btrfs_trans_handle *trans,
@@ -580,7 +580,7 @@ void btrfs_delete_ref_head(struct btrfs_delayed_ref_root *delayed_refs,rb_erase_cached (&head->href_node, &delayed_refs->href_root);RB_CLEAR_NODE (&head->href_node); -atomic_dec (&delayed_refs->num_entries); +btrfs_dec_delayed_ref_entries (delayed_refs); delayed_refs->num_heads--;if (head->processing ==0 ) delayed_refs->num_heads_ready--;
@@ -639,6 +639,7 @@ static int insert_delayed_ref(struct btrfs_trans_handle *trans,if (ref->action ==BTRFS_ADD_DELAYED_REF )list_add_tail (&ref->add_list, &href->ref_add_list);atomic_inc (&root->num_entries); + trans->total_delayed_refs++;spin_unlock (&href->lock);return ret; }
@@ -843,6 +844,7 @@ add_delayed_ref_head(struct btrfs_trans_handle *trans, delayed_refs->num_heads_ready++;atomic_inc (&delayed_refs->num_entries); trans->delayed_ref_updates++; + trans->total_delayed_refs++; }if (qrecord_inserted_ret) *qrecord_inserted_ret = qrecord_inserted;
fs/btrfs/delayed-ref.h +14 -0
@@ -150,6 +150,13 @@ struct btrfs_delayed_ref_root {*/ atomic_t num_entries; +/* +* How many entries we've run, and a corresponding waitqueue so that we +* can throttle generators appropriately. +*/ +atomic_t entries_run; +wait_queue_head_t wait; +/* total number of head nodes in tree */ unsigned long num_heads;
@@ -391,4 +398,11 @@ btrfs_delayed_node_to_data_ref(struct btrfs_delayed_ref_node *node) return container_of(node, struct btrfs_delayed_data_ref, node); } +static inline void +btrfs_dec_delayed_ref_entries (struct btrfs_delayed_ref_root *delayed_refs) +{ +atomic_dec (&delayed_refs->num_entries); +atomic_inc (&delayed_refs->entries_run); +wake_up (&delayed_refs->wait); +}#endif
fs/btrfs/extent-tree.c +3 -1
@@ -1958,7 +1958,6 @@ static int btrfs_run_delayed_refs_for_head(struct btrfs_trans_handle *trans,default :WARN_ON (1 ); } -atomic_dec (&delayed_refs->num_entries);/* * Record the must_insert_reserved flag before we drop the
@@ -1974,6 +1973,9 @@ static int btrfs_run_delayed_refs_for_head(struct btrfs_trans_handle *trans, ret =run_one_delayed_ref (trans, ref, extent_op, must_insert_reserved); +/* Anybody who's been throttled may be woken up here. */ +btrfs_dec_delayed_ref_entries (delayed_refs); +btrfs_free_delayed_extent_op (extent_op);if (ret) {unselect_delayed_ref_head (delayed_refs, locked_ref);
fs/btrfs/transaction.c +46 -3
100 lines not on this page · raw message
fs/btrfs/transaction.h +16 -0
@@ -109,7 +109,23 @@ struct btrfs_trans_handle {u64 transid;u64 bytes_reserved;u64 chunk_bytes_reserved; + +/* +* This tracks the number of items required for the delayed ref rsv, and +* is used by that code. The accounting is +* +* - 1 per delayed ref head (individual items are not counted). +* - number of csum items that would be inserted for data. +* - block group item updates. +*/ unsigned long delayed_ref_updates; + +/* +* This is the total number of delayed items that we added for this +* trans handle, this is used for the end transaction throttling code. +*/ +unsigned long total_delayed_refs; +struct btrfs_transaction *transaction;struct btrfs_block_rsv *block_rsv;struct btrfs_block_rsv *orig_rsv;
Signature · 1 line
2.24.1
PATCH 10/13 btrfs: handle uncontrolled delayed ref generation
In reply to Josef BacikSome operations can generate way too many delayed refs, resulting in the async flusher being unable to keep up. To deal with this keep track of how often we're needing to throttle the trans handles, and if it's too much increase how many delayed refs they need to wait on each iteration.
--- fs/btrfs/delayed-ref.h | 3 +++ fs/btrfs/transaction.c | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+)
fs/btrfs/delayed-ref.h +3 -0
@@ -157,6 +157,9 @@ struct btrfs_delayed_ref_root {atomic_t entries_run;wait_queue_head_t wait; +atomic_t mult; +time64_t last_adjustment; +/* total number of head nodes in tree */ unsigned long num_heads;
fs/btrfs/transaction.c +21 -0
@@ -308,6 +308,7 @@ static noinline int join_transaction(struct btrfs_fs_info *fs_info, cur_trans->delayed_refs.dirty_extent_root =RB_ROOT ;atomic_set (&cur_trans->delayed_refs.num_entries,0 );atomic_set (&cur_trans->delayed_refs.entries_run,0 ); +atomic_set (&cur_trans->delayed_refs.mult,1 );init_waitqueue_head (&cur_trans->delayed_refs.wait);/*
@@ -902,6 +903,17 @@ btrfs_throttle_for_delayed_refs(struct btrfs_fs_info *fs_info, { unsigned long threshold = max(refs, 1UL) + atomic_read(&delayed_refs->entries_run); + time64_t start = ktime_get_seconds(); + + spin_lock(&delayed_refs->lock); + if (delayed_refs->last_adjustment - start >= 1) { + if (delayed_refs->last_adjustment) + atomic_inc(&delayed_refs->mult); + delayed_refs->last_adjustment = start; + } + spin_unlock(&delayed_refs->lock); + refs *= atomic_read(&delayed_refs->mult); + wait_event_interruptible(delayed_refs->wait, (atomic_read(&delayed_refs->entries_run) >= threshold) || !btrfs_should_throttle_delayed_refs(fs_info, delayed_refs,
@@ -973,6 +985,15 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans, err = -EIO ; } +if (!throttle_delayed_refs &&atomic_read (&cur_trans->delayed_refs.mult) >1 ) { +time64_t start =ktime_get_seconds (); +spin_lock (&cur_trans->delayed_refs.lock); +if ((start - cur_trans->delayed_refs.last_adjustment) >=1 ) { +atomic_dec (&cur_trans->delayed_refs.mult); + cur_trans->delayed_refs.last_adjustment = start; + } +spin_unlock (&cur_trans->delayed_refs.lock); + }if (run_async && !work_busy (&info->async_delayed_ref_work))queue_work (system_unbound_wq, &info->async_delayed_ref_work);
Signature · 1 line
2.24.1
PATCH 11/13 btrfs: check delayed refs while relocating
In reply to Josef BacikRelocation can generate a serious amount of delayed refs, so we need to throttle the relocation sometimes in order to keep up with the async flusher. We already have a mechanism to start over because of ENOSPC, simply add delayed ref counts to the check as well.
--- fs/btrfs/relocation.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
fs/btrfs/relocation.c +10 -0
@@ -2797,6 +2797,16 @@ static int reserve_metadata_space(struct btrfs_trans_handle *trans,int ret;u64 tmp; +/* +* If we're generating too many delayed refs we should bail and allow +* the delayed ref throttling stuff to catch up. +*/ +if (btrfs_check_space_for_delayed_refs (fs_info) || +btrfs_should_throttle_delayed_refs (fs_info, + &trans->transaction->delayed_refs, + true)) +return -EAGAIN ; + num_bytes =calcu_metadata_size (rc, node,1 ) *2 ; trans->block_rsv = rc->block_rsv;
Signature · 1 line
2.24.1
PATCH 12/13 btrfs: throttle truncate for delayed ref generation
In reply to Josef BacikTruncates can generate a lot of delayed refs, and if we're over our time limit or we're already flushing we should just bail so the appropriate action can be taken.
--- fs/btrfs/inode.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
fs/btrfs/inode.c +15 -0
@@ -4386,6 +4386,21 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,* let the normal reservation dance happen higher up. */ if (should_throttle) { +struct btrfs_transaction *cur_trans = + trans->transaction; + +/* +* If we're over time, or we're flushing, go +* ahead and break out so that we can let +* everybody catch up. +*/ +if (btrfs_should_throttle_delayed_refs (fs_info, + &cur_trans->delayed_refs, true) || + cur_trans->delayed_refs.flushing) { + ret = -EAGAIN ; +break ; + } + ret =btrfs_delayed_refs_rsv_refill (fs_info,BTRFS_RESERVE_NO_FLUSH );if (ret) {
Signature · 1 line
2.24.1
PATCH 13/13 btrfs: throttle snapshot delete on delayed refs
In reply to Josef BacikOne of the largest generators of delayed refs is snapshot delete. This is because we'll walk down to a shared node/leaf and drop all of the references to the lower layer in that node/leaf. With our default nodesize of 16kib this can be hundreds of delayed refs, which can easily put us over our threshold for running delayed refs.
Instead check and see if we need to throttle ourselves, and if we do break out with -EAGAIN. If this happens we do not want to do the walk_up_tree because we need to keep processing the node we're on.
We also have to get rid of our BUG_ON(drop_level == 0) everywhere, because we can actually stop at the 0 level. Since we already have the ability to restart snapshot deletions from an arbitrary key this works out fine.
--- fs/btrfs/extent-tree.c | 39 ++++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-)
fs/btrfs/extent-tree.c +30 -9
@@ -4662,6 +4662,7 @@ struct walk_control {int reada_slot;int reada_count;int restarted; +int drop_subtree; };#define DROP_REFERENCE 1
@@ -4766,6 +4767,21 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans,u64 flag =BTRFS_BLOCK_FLAG_FULL_BACKREF ;int ret; +/* +* We only want to break if we aren't yet at the end of our leaf/node. +* The reason for this is if we're at DROP_REFERENCE we'll grab the +* current slot's key for the drop_progress. If we're at the end this +* will obviously go wrong. We are also not going to generate many more +* delayed refs at this point, so allowing us to continue will not hurt +* us. +*/ +if (!wc->drop_subtree && + (path->slots[level] <btrfs_header_nritems (path->nodes[level])) && +btrfs_should_throttle_delayed_refs (fs_info, + &trans->transaction->delayed_refs, + true)) +return -EAGAIN ; +if (wc->stage ==UPDATE_BACKREF &&btrfs_header_owner (eb) != root->root_key.objectid)return 1 ;
@@ -5198,6 +5214,8 @@ static noinline int walk_down_tree(struct btrfs_trans_handle *trans, ret =walk_down_proc (trans, root, path, wc, lookup_info);if (ret >0 )break ; +if (ret <0 ) +return ret;if (level ==0 )break ;
@@ -5332,7 +5350,6 @@ int btrfs_drop_snapshot(struct btrfs_root *root, sizeof(wc->update_progress)); level = root_item->drop_level; -BUG_ON (level ==0 ); path->lowest_level = level; ret =btrfs_search_slot (NULL , root, &key, path,0 ,0 ); path->lowest_level =0 ;
@@ -5381,19 +5398,23 @@ int btrfs_drop_snapshot(struct btrfs_root *root, wc->update_ref = update_ref; wc->keep_locks =0 ; wc->reada_count =BTRFS_NODEPTRS_PER_BLOCK (fs_info); + wc->drop_subtree =0 ;while (1 ) { ret =walk_down_tree (trans, root, path, wc); -if (ret <0 ) { - err = ret; -break ; - } - - ret =walk_up_tree (trans, root, path, wc,BTRFS_MAX_LEVEL ); -if (ret <0 ) { +if (ret <0 && ret != -EAGAIN ) { err = ret;break ; + }else if (ret != -EAGAIN ) { + ret =walk_up_tree (trans, root, path, wc, +BTRFS_MAX_LEVEL ); +if (ret <0 ) { + err = ret; +break ; + } + }else { + ret =0 ; }if (ret >0 ) {
@@ -5411,7 +5432,6 @@ int btrfs_drop_snapshot(struct btrfs_root *root, &wc->drop_progress); root_item->drop_level = wc->drop_level; -BUG_ON (wc->level ==0 );if (btrfs_should_end_transaction (trans) || (!for_reloc &&btrfs_need_cleaner_sleep (fs_info))) { ret = btrfs_update_root(trans, tree_root,
@@ -5544,6 +5564,7 @@ int btrfs_drop_subtree(struct btrfs_trans_handle *trans, wc->stage =DROP_REFERENCE ; wc->update_ref =0 ; wc->keep_locks =1 ; + wc->drop_subtree =1 ; wc->reada_count =BTRFS_NODEPTRS_PER_BLOCK (fs_info);while (1 ) {
Signature · 1 line
2.24.1