Improve preemptive ENOSPC flushing
Subsystems (2)
v3 cover Improve preemptive ENOSPC flushing
v2->v3:
- Added a cleanup to make sure we pass the right enum around for flush_space().
- Fixed a problem reported by a clang build where I used the wrong enum in the preemptive flushing for flush_space().
- Fixed up some nits pointed out by Nikolay.
v1->v2:
- Added a FORCE_COMMIT_TRANS flush operation so we can keep the flush_space stuff consistent and get all the normal tracepoints.
- Renamed fs_info->dio_bytes to ->ordered_bytes and changed it to count all ordered extents that were pending, not just DIO ordered extents that were pending.
- Reworked the clamping to not apply if we're not doing a lot of delalloc reservations.
- Reworked the preempt flushing loop to be more straightforward.
- Fixed the need_preemptive_flushing() helper to take into account DIO heavy workloads.
--- Original email ---
A while ago Nikolay started digging into a problem where they were seeing an around 20% regression on random writes, and he bisected it down to
btrfs: don't end the transaction for delayed refs in throttle
However this wasn't actually the cause of the problem.
This patch removed the code that would preemptively end the transactions if we were low on space. Because we had just introduced the ticketing code, this was no longer necessary and was causing a lot of transaction commits.
And in Nikolay's testing he validated this, we would see like 100x more transaction commits without that patch than with it, but the write regression clearly appeared when this patch was applied.
The root cause of this is that the transaction commits were essentially happening so quickly that we didn't end up needing to wait on space in the ENOSPC ticketing code as much, and thus were able to write pretty quickly. With this gone, we now were getting a sawtoothy sort of behavior where we'd run up, stop while we flushed metadata space, run some more, stop again etc.
When I implemented the ticketing infrastructure, I was trying to get us out of excessively flushing space because we would sometimes over create block groups, and thus short circuited flushing if we no longer had tickets. This had the side effect of breaking the preemptive flushing code, where we attempted to flush space in the background before we were forced to wait for space.
Enter this patchset. We still have some of this preemption logic sprinkled everywhere, so I've separated it out of the normal ticketed flushing code, and made preemptive flushing it's own thing.
The preemptive flushing logic is more specialized than the standard flushing logic. It attempts to flush in whichever pool has the highest usage. This means that if most of our space is tied up in pinned extents, we'll commit the transaction. If most of the space is tied up in delalloc, we'll flush delalloc, etc.
To test this out I used the fio job that Nikolay used, this needs to be adjusted so the overall IO size is at least 2x the RAM size for the box you are testing
fio --direct=0 --ioengine=sync --thread --directory=/mnt/test --invalidate=1 \
--group_reporting=1 --runtime=300 --fallocate=none --ramp_time=10 \
--name=RandomWrites-async-64512-4k-4 --new_group --rw=randwrite \
--size=2g --numjobs=4 --bs=4k --fsync_on_close=0 --end_fsync=0 \
--filename_format=FioWorkloads.\$jobnumI got the following results
misc-next:Josefbw=13.4MiB/s (14.0MB/s), 13.4MiB/s-13.4MiB/s (14.0MB/s-14.0MB/s), io=4015MiB (4210MB), run=300323-300323msec pre-throttling:Josefbw=16.9MiB/s (17.7MB/s), 16.9MiB/s-16.9MiB/s (17.7MB/s-17.7MB/s), io=5068MiB (5314MB), run=300069-300069msec my patches:Josefbw=18.0MiB/s (18.9MB/s), 18.0MiB/s-18.0MiB/s (18.9MB/s-18.9MB/s), io=5403MiB (5666MB), run=300001-300001msec
Thanks,
Josef
Josef Bacik (12):
btrfs: make flush_space take a enum btrfs_flush_state instead of int
btrfs: add a trace point for reserve tickets
btrfs: track ordered bytes instead of just dio ordered bytes
btrfs: introduce a FORCE_COMMIT_TRANS flush operation
btrfs: improve preemptive background space flushing
btrfs: rename need_do_async_reclaim
btrfs: check reclaim_size in need_preemptive_reclaim
btrfs: rework btrfs_calc_reclaim_metadata_size
btrfs: simplify the logic in need_preemptive_flushing
btrfs: implement space clamping for preemptive flushing
btrfs: adjust the flush trace point to include the source
btrfs: add a trace class for dumping the current ENOSPC state
fs/btrfs/ctree.h | 4 +- fs/btrfs/disk-io.c | 9 +- fs/btrfs/ordered-data.c | 13 +- fs/btrfs/space-info.c | 274 ++++++++++++++++++++++++++++------- fs/btrfs/space-info.h | 3 + include/trace/events/btrfs.h | 104 ++++++++++++- 6 files changed, 340 insertions(+), 67 deletions(-)
Signature · 1 line
2.26.2
PATCH v3 1/12 btrfs: make flush_space take a enum btrfs_flush_state instead of int
I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into flush_space. Change the argument to be explicitly the enum we're expecting to make everything consistent. Maybe eventually gcc will catch errors like this.
--- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/space-info.c +2 -2
@@ -667,7 +667,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,*/ static void flush_space (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,u64 num_bytes, -int state) +enum btrfs_flush_state state) {struct btrfs_root *root = fs_info->extent_root;struct btrfs_trans_handle *trans;
@@ -920,7 +920,7 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work)struct btrfs_fs_info *fs_info;struct btrfs_space_info *space_info;u64 to_reclaim; -int flush_state; +enum btrfs_flush_state flush_state;int commit_cycles =0 ;u64 last_tickets_id;
Signature · 1 line
2.26.2
Re: [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int
Quoted 7 lines · Josef Bacik
I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into
flush_space. Change the argument to be explicitly the enum we're expecting to make everything consistent. Maybe eventually gcc will catch errors like this.Signed-off-by Josef Bacik <[email protected]>
Re: [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int
In reply to Josef BacikQuoted 5 lines · Josef Bacik
I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into
flush_space. Change the argument to be explicitly the enum we're expecting to make everything consistent. Maybe eventually gcc will catch errors like this.
I can't find any such public report and none of the clang warnings seem to be specific about that. Local run with clang 11 does not produce any warning.
Quoted 20 lines
Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/btrfs/space-info.c +1
-1@@ -667,7 +667,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,*/ static void flush_space (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,u64 num_bytes, -int state) +enum btrfs_flush_state state) {struct btrfs_root *root = fs_info->extent_root;struct btrfs_trans_handle *trans;
fs/btrfs/space-info.c static void btrfs_async_reclaim_metadata_space(struct work_struct *work) · line 923
Quoted 5 lines
fs/btrfs/space-info.c +1
-1@@ -920,7 +920,7 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work)struct btrfs_fs_info *fs_info;struct btrfs_space_info *space_info;u64 to_reclaim; -int flush_state; +enum btrfs_flush_state flush_state;
The pattern with int instead of the enum is also in priority_reclaim_metadata_space so I'll add it. If you find the report we can add the warning among the newly enabled warnings to Makefile.
Quoted 5 lines
fs/btrfs/space-info.c +0
-0int commit_cycles =0 ;u64 last_tickets_id;Signature · 1 line
2.26.2
Re: [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int
Quoted 11 lines · David Sterba
Quoted 5 lines · Josef Bacik
I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into
flush_space. Change the argument to be explicitly the enum we're expecting to make everything consistent. Maybe eventually gcc will catch errors like this.I can't find any such public report and none of the clang warnings seem to be specific about that. Local run with clang 11 does not produce any warning.
IDK, it was a private email just to me with the following output from clang
s/btrfs/space-info.c:1115:12: warning: implicit conversion from enumeration type 'enum btrfs_flush_state' to different enumeration type 'enum btrfs_reserve_flush_enum' [-Wenum-conversion]
flush = FLUSH_DELALLOC;
~ ^~~~~~~~~~~~~~
fs/btrfs/space-info.c:1120:12: warning: implicit conversion from
enumeration type 'enum btrfs_flush_state' to different enumeration type
'enum btrfs_reserve_flush_enum' [-Wenum-conversion]
flush = FORCE_COMMIT_TRANS;
~ ^~~~~~~~~~~~~~~~~~
fs/btrfs/space-info.c:1124:12: warning: implicit conversion from
enumeration type 'enum btrfs_flush_state' to different enumeration type
'enum btrfs_reserve_flush_enum' [-Wenum-conversion]
flush = FLUSH_DELAYED_ITEMS_NR;
~ ^~~~~~~~~~~~~~~~~~~~~~
fs/btrfs/space-info.c:1127:12: warning: implicit conversion from
enumeration type 'enum btrfs_flush_state' to different enumeration type
'enum btrfs_reserve_flush_enum' [-Wenum-conversion]
flush = FLUSH_DELAYED_REFS_NR;
~ ^~~~~~~~~~~~~~~~~~~~~I figure it made sense, might as well fix it. Do we have that option for our shiny new -W build options? Thanks,
Josef
Re: [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int
Quoted 36 lines · Josef Bacik
Quoted 11 lines · David Sterba
Quoted 5 lines · Josef Bacik
I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into
flush_space. Change the argument to be explicitly the enum we're expecting to make everything consistent. Maybe eventually gcc will catch errors like this.I can't find any such public report and none of the clang warnings seem to be specific about that. Local run with clang 11 does not produce any warning.
IDK, it was a private email just to me with the following output from clang
s/btrfs/space-info.c:1115:12: warning: implicit conversion from enumeration type 'enumbtrfs_flush_state' to different enumeration type 'enumbtrfs_reserve_flush_enum' [-Wenum-conversion]flush = FLUSH_DELALLOC; ~ ^~~~~~~~~~~~~~ fs/btrfs/space-info.c:1120:12: warning: implicit conversion from enumeration type 'enum btrfs_flush_state' to different enumeration type 'enum btrfs_reserve_flush_enum' [-Wenum-conversion] flush = FORCE_COMMIT_TRANS; ~ ^~~~~~~~~~~~~~~~~~ fs/btrfs/space-info.c:1124:12: warning: implicit conversion from enumeration type 'enum btrfs_flush_state' to different enumeration type 'enum btrfs_reserve_flush_enum' [-Wenum-conversion] flush = FLUSH_DELAYED_ITEMS_NR; ~ ^~~~~~~~~~~~~~~~~~~~~~ fs/btrfs/space-info.c:1127:12: warning: implicit conversion from enumeration type 'enum btrfs_flush_state' to different enumeration type 'enum btrfs_reserve_flush_enum' [-Wenum-conversion] flush = FLUSH_DELAYED_REFS_NR; ~ ^~~~~~~~~~~~~~~~~~~~~
Quoted 2 lines
I figure it made sense, might as well fix it. Do we have that option for our shiny new -W build options? Thanks,
We can add -Wenum-conversion sure, but neither gcc (10.2.1) nor clang (11.0.1) reproduces the warning for me so I wonder how useful it would be. I've also tried adding -Wextra and -Wall and the build is clean.
Per gcc documentation, -Wenum-conversion is not exactly what the code did:
"Warn when a value of enumerated type is implicitly converted to a
different enumerated type. This warning is enabled by -Wextra."
And now that I read the warning carefuly it does not apply to our code, "btrfs_reserve_flush_enum = btrfs_flush_state" does not happen so it must have been some older revision of the code.
Changing
@@ -1073,7 +1073,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)
spin_lock(&space_info->lock);
while (need_preemptive_reclaim(fs_info, space_info)) {- enum btrfs_flush_state flush;
+ enum btrfs_reserve_flush_enum flush;
---
gcc says (on misc-next without this patchset)
fs/btrfs/space-info.c: In function ‘btrfs_preempt_reclaim_metadata_space’:fs/btrfs/space-info.c:1112:10: warning: implicit conversion from ‘enum btrfs_flush_state’ to ‘enum btrfs_reserve_flush_enum’ [-Wenum-conversion]
1112 | flush = FLUSH_DELALLOC;
| ^
CC [M] fs/btrfs/block-group.ofs/btrfs/space-info.c:1117:10: warning: implicit conversion from ‘enum btrfs_flush_state’ to ‘enum btrfs_reserve_flush_enum’ [-Wenum-conversion]
1117 | flush = FORCE_COMMIT_TRANS;
| ^fs/btrfs/space-info.c:1121:10: warning: implicit conversion from ‘enum btrfs_flush_state’ to ‘enum btrfs_reserve_flush_enum’ [-Wenum-conversion]
1121 | flush = FLUSH_DELAYED_ITEMS_NR;
| ^fs/btrfs/space-info.c:1124:10: warning: implicit conversion from ‘enum btrfs_flush_state’ to ‘enum btrfs_reserve_flush_enum’ [-Wenum-conversion]
1124 | flush = FLUSH_DELAYED_REFS_NR;
| ^fs/btrfs/space-info.c:1135:48: warning: implicit conversion from ‘enum btrfs_reserve_flush_enum’ to ‘enum btrfs_flush_state’ [-Wenum-conversion]
1135 | flush_space(fs_info, space_info, to_reclaim, flush, true);
| ---
So you weren't fixing the reported problem by this patch. Assigning enum to an int should be natural so thre's no reason to fix that but for clarity I don't mind, the patch stays.
PATCH v3 2/12 btrfs: add a trace point for reserve tickets
In reply to Josef BacikWhile debugging a ENOSPC related performance problem I needed to see the time difference between start and end of a reserve ticket, so add a trace point to report when we handle a reserve ticket.
I opted to spit out start_ns itself without calculating the difference because there could be a gap between enabling the tracpoint and setting start_ns. Doing it this way allows us to filter on 0 start_ns so we don't get bogus entries, and we can easily calculate the time difference with bpftrace or something else.
--- fs/btrfs/space-info.c | 10 +++++++++- include/trace/events/btrfs.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-)
fs/btrfs/space-info.c +9 -1
@@ -1224,6 +1224,7 @@ static void wait_reserve_ticket(struct btrfs_fs_info *fs_info,static int handle_reserve_ticket (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,struct reserve_ticket *ticket, +u64 start_ns,u64 orig_bytes,enum btrfs_reserve_flush_enum flush) {int ret;
@@ -1279,6 +1280,8 @@ static int handle_reserve_ticket(struct btrfs_fs_info *fs_info,* space wasn't reserved at all). */ ASSERT (!(ticket->bytes ==0 && ticket->error)); +trace_btrfs_reserve_ticket (fs_info, space_info->flags, orig_bytes, + start_ns, flush, ticket->error);return ret; }
@@ -1312,6 +1315,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, {struct work_struct *async_work;struct reserve_ticket ticket; +u64 start_ns =0 ;u64 used;int ret =0 ;bool pending_tickets;
@@ -1364,6 +1368,9 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info, space_info->reclaim_size += ticket.bytes;init_waitqueue_head (&ticket.wait); ticket.steal = (flush ==BTRFS_RESERVE_FLUSH_ALL_STEAL ); +if (trace_btrfs_reserve_ticket_enabled ()) + start_ns =ktime_get_ns (); +if (flush ==BTRFS_RESERVE_FLUSH_ALL || flush ==BTRFS_RESERVE_FLUSH_ALL_STEAL || flush ==BTRFS_RESERVE_FLUSH_DATA ) {
@@ -1400,7 +1407,8 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info,if (!ret || flush ==BTRFS_RESERVE_NO_FLUSH )return ret; -return handle_reserve_ticket (fs_info, space_info, &ticket, flush); +return handle_reserve_ticket (fs_info, space_info, &ticket, start_ns, + orig_bytes, flush); }/**
include/trace/events/btrfs.h +29 -0
@@ -2025,6 +2025,35 @@ TRACE_EVENT(btrfs_convert_extent_bit, __print_flags(__entry->clear_bits, "|", EXTENT_FLAGS)) ); +TRACE_EVENT (btrfs_reserve_ticket, + TP_PROTO(const struct btrfs_fs_info *fs_info, u64 flags, u64 bytes, + u64 start_ns, int flush, int error), + +TP_ARGS (fs_info, flags, bytes, start_ns, flush, error), + +TP_STRUCT__entry_btrfs ( + __field( u64, flags ) + __field( u64, bytes ) + __field( u64, start_ns ) + __field( int, flush ) +__field ( int, error ) + ), + +TP_fast_assign_btrfs (fs_info, + __entry->flags = flags; + __entry->bytes = bytes; + __entry->start_ns = start_ns; + __entry->flush = flush; + __entry->error = error; + ), + +TP_printk_btrfs ("flags=%s bytes=%llu start_ns=%llu flush=%s error=%d" , +__print_flags (__entry->flags,"|" ,BTRFS_GROUP_FLAGS ), + __entry->bytes, __entry->start_ns, +__print_symbolic (__entry->flush,FLUSH_ACTIONS ), + __entry->error) +); + DECLARE_EVENT_CLASS(btrfs_sleep_tree_lock, TP_PROTO(const struct extent_buffer *eb, u64 start_ns),
Signature · 1 line
2.26.2
Re: [PATCH v3 02/12] btrfs: add a trace point for reserve tickets
Quoted 22 lines · Josef Bacik
While debugging a ENOSPC related performance problem I needed to see the time difference between start and end of a reserve ticket, so add a trace point to report when we handle a reserve ticket.
I opted to spit out
start_nsitself without calculating the difference because there could be a gap between enabling the tracpoint and settingstart_ns. Doing it this way allows us to filter on 0start_nsso we don't get bogus entries, and we can easily calculate the time difference with bpftrace or something else.Reviewed-by Nikolay Borisov <[email protected]>Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/space-info.c | 10 +++++++++- include/trace/events/btrfs.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-)fs/btrfs/space-info.c +0
-0
fs/btrfs/space-info.c static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, · line 1227
Quoted 4 lines
fs/btrfs/space-info.c +1
-0@@ -1224,6 +1224,7 @@ static void wait_reserve_ticket(struct btrfs_fs_info *fs_info, static int handle_reserve_ticket(struct btrfs_fs_info *fs_info, struct btrfs_space_info *space_info, struct reserve_ticket *ticket, + u64 start_ns, u64 orig_bytes,
Added
+ * @start_ns: timestamp when the reservation started
+ * @orig_bytes: amount of bytes originally reserved
caught by the recently added kdoc verification.
Quoted 1 line
fs/btrfs/space-info.c +0
-0enum btrfs_reserve_flush_enum flush)
PATCH v3 3/12 btrfs: track ordered bytes instead of just dio ordered bytes
In reply to Josef BacikWe track dio_bytes because the shrink delalloc code needs to know if we have more DIO in flight than we have normal buffered IO. The reason for this is because we can't "flush" DIO, we have to just wait on the ordered extents to finish.
However this is true of all ordered extents. If we have more ordered space outstanding than dirty pages we should be waiting on ordered extents. We already are ok on this front technically, because we always do a FLUSH_DELALLOC_WAIT loop, but I want to use the ordered counter in the preemptive flushing code as well, so change this to count all ordered bytes instead of just DIO ordered bytes.
--- fs/btrfs/ctree.h | 2 +- fs/btrfs/disk-io.c | 8 ++++---- fs/btrfs/ordered-data.c | 13 ++++++------- fs/btrfs/space-info.c | 18 +++++++----------- 4 files changed, 18 insertions(+), 23 deletions(-)
fs/btrfs/ctree.h +1 -1
@@ -790,7 +790,7 @@ struct btrfs_fs_info {/* used to keep from writing metadata until there is a nice batch */ struct percpu_counter dirty_metadata_bytes;struct percpu_counter delalloc_bytes; -struct percpu_counter dio_bytes; +struct percpu_counter ordered_bytes;s32 dirty_metadata_batch;s32 delalloc_batch;
fs/btrfs/disk-io.c +4 -4
@@ -1466,7 +1466,7 @@ void btrfs_free_fs_info(struct btrfs_fs_info *fs_info) {percpu_counter_destroy (&fs_info->dirty_metadata_bytes);percpu_counter_destroy (&fs_info->delalloc_bytes); -percpu_counter_destroy (&fs_info->dio_bytes); +percpu_counter_destroy (&fs_info->ordered_bytes);percpu_counter_destroy (&fs_info->dev_replace.bio_counter);btrfs_free_csum_hash (fs_info);btrfs_free_stripe_hash_table (fs_info);
@@ -2748,7 +2748,7 @@ static int init_mount_fs_info(struct btrfs_fs_info *fs_info, struct super_block sb->s_blocksize =BTRFS_BDEV_BLOCKSIZE ; sb->s_blocksize_bits =blksize_bits (BTRFS_BDEV_BLOCKSIZE ); - ret =percpu_counter_init (&fs_info->dio_bytes,0 ,GFP_KERNEL ); + ret =percpu_counter_init (&fs_info->ordered_bytes,0 ,GFP_KERNEL );if (ret)return ret;
@@ -4055,9 +4055,9 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) percpu_counter_sum(&fs_info->delalloc_bytes)); } -if (percpu_counter_sum (&fs_info->dio_bytes)) +if (percpu_counter_sum (&fs_info->ordered_bytes))btrfs_info (fs_info,"at unmount dio bytes count %lld" , -percpu_counter_sum (&fs_info->dio_bytes)); +percpu_counter_sum (&fs_info->ordered_bytes));btrfs_sysfs_remove_mounted (fs_info);btrfs_sysfs_remove_fsid (fs_info->fs_devices);
fs/btrfs/ordered-data.c +6 -7
@@ -202,11 +202,11 @@ static int __btrfs_add_ordered_extent(struct btrfs_inode *inode, u64 file_offsetif (type !=BTRFS_ORDERED_IO_DONE && type !=BTRFS_ORDERED_COMPLETE )set_bit (type, &entry->flags); -if (dio) { -percpu_counter_add_batch (&fs_info->dio_bytes, num_bytes, - fs_info->delalloc_batch); +percpu_counter_add_batch (&fs_info->ordered_bytes, num_bytes, + fs_info->delalloc_batch); + +if (dio)set_bit (BTRFS_ORDERED_DIRECT , &entry->flags); - }/* one ref for the tree */ refcount_set (&entry->refs,1 );
@@ -480,9 +480,8 @@ void btrfs_remove_ordered_extent(struct btrfs_inode *btrfs_inode,btrfs_delalloc_release_metadata (btrfs_inode, entry->num_bytes, false); -if (test_bit (BTRFS_ORDERED_DIRECT , &entry->flags)) -percpu_counter_add_batch (&fs_info->dio_bytes, -entry->num_bytes, - fs_info->delalloc_batch); +percpu_counter_add_batch (&fs_info->ordered_bytes, -entry->num_bytes, + fs_info->delalloc_batch); tree = &btrfs_inode->ordered_tree;spin_lock_irq (&tree->lock);
fs/btrfs/space-info.c +7 -11
@@ -489,7 +489,7 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, {struct btrfs_trans_handle *trans;u64 delalloc_bytes; -u64 dio_bytes; +u64 ordered_bytes;u64 items;long time_left;int loops;
@@ -513,25 +513,20 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, delalloc_bytes =percpu_counter_sum_positive ( &fs_info->delalloc_bytes); - dio_bytes =percpu_counter_sum_positive (&fs_info->dio_bytes); -if (delalloc_bytes ==0 && dio_bytes ==0 ) { -if (trans) -return ; -if (wait_ordered) -btrfs_wait_ordered_roots (fs_info, items,0 , (u64)-1 ); + ordered_bytes =percpu_counter_sum_positive (&fs_info->ordered_bytes); +if (delalloc_bytes ==0 && ordered_bytes ==0 )return ; - }/* * If we are doing more ordered than delalloc we need to just wait on * ordered extents, otherwise we'll waste time trying to flush delalloc * that likely won't give us the space back we need. */ -if (dio_bytes > delalloc_bytes) +if (ordered_bytes > delalloc_bytes) wait_ordered = true; loops =0 ; -while ((delalloc_bytes || dio_bytes) && loops <3 ) { +while ((delalloc_bytes || ordered_bytes) && loops <3 ) {btrfs_start_delalloc_roots (fs_info, items); loops++;
@@ -553,7 +548,8 @@ static void shrink_delalloc(struct btrfs_fs_info *fs_info, delalloc_bytes =percpu_counter_sum_positive ( &fs_info->delalloc_bytes); - dio_bytes =percpu_counter_sum_positive (&fs_info->dio_bytes); + ordered_bytes =percpu_counter_sum_positive ( + &fs_info->ordered_bytes); } }
Signature · 1 line
2.26.2
Re: [PATCH v3 03/12] btrfs: track ordered bytes instead of just dio ordered bytes
Quoted 5 lines · Josef Bacik
We track
dio_bytesbecause the shrink delalloc code needs to know if we have more DIO in flight than we have normal buffered IO. The reason for this is because we can't "flush" DIO, we have to just wait on the ordered extents to finish.
Quoted 8 lines
However this is true of all ordered extents. If we have more ordered space outstanding than dirty pages we should be waiting on ordered extents. We already are ok on this front technically, because we always do a
FLUSH_DELALLOC_WAITloop, but I want to use the ordered counter in the preemptive flushing code as well, so change this to count all ordered bytes instead of just DIO ordered bytes.Signed-off-by Josef Bacik <[email protected]>
PATCH v3 4/12 btrfs: introduce a FORCE_COMMIT_TRANS flush operation
In reply to Josef BacikSole-y for preemptive flushing, we want to be able to force the transaction commit without any of the ambiguity of may_commit_transaction(). This is because may_commit_transaction() checks tickets and such, and in preemptive flushing we already know it'll be helpful, so use this to keep the code nice and clean and straightforward.
--- fs/btrfs/ctree.h | 1 + fs/btrfs/space-info.c | 8 ++++++++ include/trace/events/btrfs.h | 3 ++- 3 files changed, 11 insertions(+), 1 deletion(-)
fs/btrfs/ctree.h +1 -0
@@ -2654,6 +2654,7 @@ enum btrfs_flush_state {ALLOC_CHUNK_FORCE =8 ,RUN_DELAYED_IPUTS =9 ,COMMIT_TRANS =10 , + FORCE_COMMIT_TRANS = 11, }; int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,
fs/btrfs/space-info.c +8 -0
@@ -732,6 +732,14 @@ static void flush_space(struct btrfs_fs_info *fs_info,case COMMIT_TRANS : ret =may_commit_transaction (fs_info, space_info);break ; +case FORCE_COMMIT_TRANS : + trans =btrfs_join_transaction (root); +if (IS_ERR (trans)) { + ret =PTR_ERR (trans); +break ; + } + ret =btrfs_commit_transaction (trans); +break ;default : ret = -ENOSPC ;break ;
include/trace/events/btrfs.h +2 -1
@@ -99,7 +99,8 @@ struct btrfs_space_info; EM( ALLOC_CHUNK, "ALLOC_CHUNK") \ EM( ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE") \ EM( RUN_DELAYED_IPUTS, "RUN_DELAYED_IPUTS") \ -EMe (COMMIT_TRANS ,"COMMIT_TRANS" ) + EM(COMMIT_TRANS, "COMMIT_TRANS") \ +EMe (FORCE_COMMIT_TRANS ,"FORCE_COMMIT_TRANS" )/* * First define the enums in the above macros to be exported to userspace via
Signature · 1 line
2.26.2
Re: [PATCH v3 04/12] btrfs: introduce a FORCE_COMMIT_TRANS flush operation
Quoted 8 lines · Josef Bacik
Sole-y for preemptive flushing, we want to be able to force the transaction commit without any of the ambiguity of
may_commit_transaction(). This is becausemay_commit_transaction()checks tickets and such, and in preemptive flushing we already know it'll be helpful, so use this to keep the code nice and clean and straightforward.Signed-off-by Josef Bacik <[email protected]>
Re: [PATCH v3 04/12] btrfs: introduce a FORCE_COMMIT_TRANS flush operation
In reply to Josef BacikQuoted 19 lines · Josef Bacik
Sole-y for preemptive flushing, we want to be able to force the transaction commit without any of the ambiguity of
may_commit_transaction(). This is becausemay_commit_transaction()checks tickets and such, and in preemptive flushing we already know it'll be helpful, so use this to keep the code nice and clean and straightforward.Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/ctree.h | 1 + fs/btrfs/space-info.c | 8 ++++++++ include/trace/events/btrfs.h | 3 ++- 3 files changed, 11 insertions(+), 1 deletion(-)fs/btrfs/ctree.h +0
-0
fs/btrfs/ctree.h enum btrfs_flush_state { · line 2657
Quoted 4 lines
fs/btrfs/ctree.h +1
-0@@ -2654,6 +2654,7 @@ enum btrfs_flush_state {ALLOC_CHUNK_FORCE =8 ,RUN_DELAYED_IPUTS =9 ,COMMIT_TRANS =10 , + FORCE_COMMIT_TRANS = 11,
This new state is not documented in space-info.c (before btrfs_space_info_used).
Re: [PATCH v3 04/12] btrfs: introduce a FORCE_COMMIT_TRANS flush operation
Quoted 15 lines · David Sterba
Quoted 14 lines · Josef Bacik
Sole-y for preemptive flushing, we want to be able to force the transaction commit without any of the ambiguity of
may_commit_transaction(). This is becausemay_commit_transaction()checks tickets and such, and in preemptive flushing we already know it'll be helpful, so use this to keep the code nice and clean and straightforward.Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/ctree.h | 1 + fs/btrfs/space-info.c | 8 ++++++++ include/trace/events/btrfs.h | 3 ++- 3 files changed, 11 insertions(+), 1 deletion(-)
Quoted 12 lines
fs/btrfs/ctree.h enum btrfs_flush_state { · line 2657
Quoted 9 lines
fs/btrfs/ctree.h +1
-0@@ -2654,6 +2654,7 @@ enum btrfs_flush_state {ALLOC_CHUNK_FORCE =8 ,RUN_DELAYED_IPUTS =9 ,COMMIT_TRANS =10 , + FORCE_COMMIT_TRANS = 11,This new state is not documented in
space-info.c(beforebtrfs_space_info_used).
I've used to changelog and turned it to the comment.
PATCH v3 5/12 btrfs: improve preemptive background space flushing
In reply to Josef BacikCurrently if we ever have to flush space because we do not have enough we allocate a ticket and attach it to the space_info, and then systematically flush things in the file system that hold space reservations until our space is reclaimed.
However this has a latency cost, we must go to sleep and wait for the flushing to make progress before we are woken up and allowed to continue doing our work.
In order to address that we used to kick off the async worker to flush space preemptively, so that we could be reclaiming space hopefully before any tasks needed to stop and wait for space to reclaim.
When I introduced the ticketed ENOSPC stuff this broke slightly in the fact that we were using tickets to indicate if we were done flushing. No tickets, no more flushing. However this meant that we essentially never preemptively flushed. This caused a write performance regression that Nikolay noticed in an unrelated patch that removed the committing of the transaction during btrfs_end_transaction.
The behavior that happened pre that patch was btrfs_end_transaction() would see that we were low on space, and it would commit the transaction. This was bad because in this particular case you could end up with thousands and thousands of transactions being committed during the 5 minute reproducer. With the patch to remove this behavior you got much more sane transaction commits, but we ended up slower because we would write for a while, flush, write for a while, flush again.
To address this we need to reinstate a preemptive flushing mechanism. However it is distinctly different from our ticketing flushing in that it doesn't have tickets to base it's decisions on. Instead of bolting this logic into our existing flushing work, add another worker to handle this preemptive flushing. Here we will attempt to be slightly intelligent about the things that we flushing, attempting to balance between whichever pool is taking up the most space.
--- fs/btrfs/ctree.h | 1 + fs/btrfs/disk-io.c | 1 + fs/btrfs/space-info.c | 101 +++++++++++++++++++++++++++++++++++++++++- 3 files changed, 101 insertions(+), 2 deletions(-)
fs/btrfs/ctree.h +1 -0
@@ -922,6 +922,7 @@ struct btrfs_fs_info {/* Used to reclaim the metadata space in the background. */ struct work_struct async_reclaim_work;struct work_struct async_data_reclaim_work; +struct work_struct preempt_reclaim_work;spinlock_t unused_bgs_lock;struct list_head unused_bgs;
fs/btrfs/disk-io.c +1 -0
@@ -4003,6 +4003,7 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info)cancel_work_sync (&fs_info->async_reclaim_work);cancel_work_sync (&fs_info->async_data_reclaim_work); +cancel_work_sync (&fs_info->preempt_reclaim_work);/* Cancel or finish ongoing discard work */ btrfs_discard_cleanup (fs_info);
fs/btrfs/space-info.c +99 -2
122 lines not on this page · raw message
Signature · 1 line
2.26.2
Re: [PATCH v3 05/12] btrfs: improve preemptive background space flushing
Quoted 36 lines · Josef Bacik
Currently if we ever have to flush space because we do not have enough we allocate a ticket and attach it to the
space_info, and then systematically flush things in the file system that hold space reservations until our space is reclaimed.However this has a latency cost, we must go to sleep and wait for the flushing to make progress before we are woken up and allowed to continue doing our work.
In order to address that we used to kick off the async worker to flush space preemptively, so that we could be reclaiming space hopefully before any tasks needed to stop and wait for space to reclaim.
When I introduced the ticketed ENOSPC stuff this broke slightly in the fact that we were using tickets to indicate if we were done flushing. No tickets, no more flushing. However this meant that we essentially never preemptively flushed. This caused a write performance regression that Nikolay noticed in an unrelated patch that removed the committing of the transaction during
btrfs_end_transaction.The behavior that happened pre that patch was
btrfs_end_transaction()would see that we were low on space, and it would commit the transaction. This was bad because in this particular case you could end up with thousands and thousands of transactions being committed during the 5 minute reproducer. With the patch to remove this behavior you got much more sane transaction commits, but we ended up slower because we would write for a while, flush, write for a while, flush again.To address this we need to reinstate a preemptive flushing mechanism. However it is distinctly different from our ticketing flushing in that it doesn't have tickets to base it's decisions on. Instead of bolting this logic into our existing flushing work, add another worker to handle this preemptive flushing. Here we will attempt to be slightly intelligent about the things that we flushing, attempting to balance between whichever pool is taking up the most space.
Quoted 1 line
Signed-off-by Josef Bacik <[email protected]>
PATCH v3 6/12 btrfs: rename need_do_async_reclaim
In reply to Josef BacikAll of our normal flushing is asynchronous reclaim, so this helper is poorly named. This is more checking if we need to preemptively flush space, so rename it to need_preemptive_reclaim.
--- fs/btrfs/space-info.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
fs/btrfs/space-info.c +5 -5
@@ -799,9 +799,9 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info,return to_reclaim; } -static inline int need_do_async_reclaim (struct btrfs_fs_info *fs_info, -struct btrfs_space_info *space_info, -u64 used) +static inline bool need_preemptive_reclaim (struct btrfs_fs_info *fs_info, +struct btrfs_space_info *space_info, +u64 used) {u64 thresh =div_factor_fine (space_info->total_bytes,98 );
@@ -1019,7 +1019,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)spin_lock (&space_info->lock); used =btrfs_space_info_used (space_info, true); -while (need_do_async_reclaim (fs_info, space_info, used)) { +while (need_preemptive_reclaim (fs_info, space_info, used)) {enum btrfs_flush_state flush;u64 delalloc_size =0 ;u64 to_reclaim, block_rsv_size;
@@ -1496,7 +1496,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info,* the async reclaim as we will panic. */ if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags) && -need_do_async_reclaim (fs_info, space_info, used) && +need_preemptive_reclaim (fs_info, space_info, used) && !work_busy (&fs_info->preempt_reclaim_work)) {trace_btrfs_trigger_flush (fs_info, space_info->flags, orig_bytes, flush,"preempt" );
Signature · 1 line
2.26.2
Re: [PATCH v3 06/12] btrfs: rename need_do_async_reclaim
Quoted 15 lines · Josef Bacik
All of our normal flushing is asynchronous reclaim, so this helper is poorly named. This is more checking if we need to preemptively flush space, so rename it to
need_preemptive_reclaim.Reviewed-by Nikolay Borisov <[email protected]>Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/space-info.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/btrfs/space-info.c +0
-0
fs/btrfs/space-info.c btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info, · line 802
Quoted 7 lines
fs/btrfs/space-info.c +1
-3@@ -799,9 +799,9 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info,return to_reclaim; } -static inline int need_do_async_reclaim (struct btrfs_fs_info *fs_info, -struct btrfs_space_info *space_info, -u64 used) +static inline bool need_preemptive_reclaim(struct btrfs_fs_info *fs_info,
Following patches add more code to that function so it's not really suitable for 'static inline', I'd rather drop it.
PATCH v3 7/12 btrfs: check reclaim_size in need_preemptive_reclaim
In reply to Josef BacikIf we're flushing space for tickets then we have space_info->reclaim_size set and we do not need to do background reclaim.
--- fs/btrfs/space-info.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/btrfs/space-info.c +7 -0
@@ -809,6 +809,13 @@ static inline bool need_preemptive_reclaim(struct btrfs_fs_info *fs_info,if ((space_info->bytes_used + space_info->bytes_reserved) >= thresh)return 0 ; +/* +* We have tickets queued, bail so we don't compete with the async +* flushers. +*/ +if (space_info->reclaim_size) +return 0 ; +if (!btrfs_calc_reclaim_metadata_size (fs_info, space_info))return 0 ;
Signature · 1 line
2.26.2
PATCH v3 8/12 btrfs: rework btrfs_calc_reclaim_metadata_size
In reply to Josef BacikCurrently btrfs_calc_reclaim_metadata_size does two things, it returns the space currently required for flushing by the tickets, and if there are no tickets it calculates a value for the preemptive flushing.
However for the normal ticketed flushing we really only care about the space required for tickets. We will accidentally come in and flush one time, but as soon as we see there are no tickets we bail out of our flushing.
Fix this by making btrfs_calc_reclaim_metadata_size really only tell us what is required for flushing if we have people waiting on space. Then move the preemptive flushing logic into need_preemptive_reclaim(). We ignore btrfs_calc_reclaim_metadata_size() in need_preemptive_reclaim() because if we are in this path then we made our reservation and there are not pending tickets currently, so we do not need to check it, simply do the fuzzy logic to check if we're getting low on space.
--- fs/btrfs/space-info.c | 44 ++++++++++++++++++++----------------------- 1 file changed, 20 insertions(+), 24 deletions(-)
fs/btrfs/space-info.c +20 -24
@@ -756,7 +756,6 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info, {u64 used;u64 avail; -u64 expected;u64 to_reclaim = space_info->reclaim_size;lockdep_assert_held (&space_info->lock);
@@ -774,28 +773,6 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info,if (space_info->total_bytes + avail < used) to_reclaim += used - (space_info->total_bytes + avail); -if (to_reclaim) -return to_reclaim; - - to_reclaim =min_t (u64,num_online_cpus () *SZ_1M ,SZ_16M ); -if (btrfs_can_overcommit (fs_info, space_info, to_reclaim, -BTRFS_RESERVE_FLUSH_ALL )) -return 0 ; - - used =btrfs_space_info_used (space_info, true); - -if (btrfs_can_overcommit (fs_info, space_info,SZ_1M , -BTRFS_RESERVE_FLUSH_ALL )) - expected =div_factor_fine (space_info->total_bytes,95 ); -else - expected =div_factor_fine (space_info->total_bytes,90 ); - -if (used > expected) - to_reclaim = used - expected; -else - to_reclaim =0 ; - to_reclaim =min (to_reclaim, space_info->bytes_may_use + - space_info->bytes_reserved);return to_reclaim; }
@@ -804,6 +781,7 @@ static inline bool need_preemptive_reclaim(struct btrfs_fs_info *fs_info, u64 used) {u64 thresh =div_factor_fine (space_info->total_bytes,98 ); +u64 to_reclaim, expected;/* If we're just plain full then async reclaim just slows us down. */ if ((space_info->bytes_used + space_info->bytes_reserved) >= thresh)
@@ -816,7 +794,25 @@ static inline bool need_preemptive_reclaim(struct btrfs_fs_info *fs_info,if (space_info->reclaim_size)return 0 ; -if (!btrfs_calc_reclaim_metadata_size (fs_info, space_info)) + to_reclaim =min_t (u64,num_online_cpus () *SZ_1M ,SZ_16M ); +if (btrfs_can_overcommit (fs_info, space_info, to_reclaim, +BTRFS_RESERVE_FLUSH_ALL )) +return 0 ; + + used =btrfs_space_info_used (space_info, true); +if (btrfs_can_overcommit (fs_info, space_info,SZ_1M , +BTRFS_RESERVE_FLUSH_ALL )) + expected =div_factor_fine (space_info->total_bytes,95 ); +else + expected =div_factor_fine (space_info->total_bytes,90 ); + +if (used > expected) + to_reclaim = used - expected; +else + to_reclaim =0 ; + to_reclaim =min (to_reclaim, space_info->bytes_may_use + + space_info->bytes_reserved); +if (!to_reclaim)return 0 ; return (used >= thresh && !btrfs_fs_closing(fs_info) &&
Signature · 1 line
2.26.2
PATCH v3 9/12 btrfs: simplify the logic in need_preemptive_flushing
In reply to Josef BacikA lot of this was added all in one go with no explanation, and is a bit unwieldy and confusing. Simplify the logic to start preemptive flushing if we've reserved more than half of our available free space.
--- fs/btrfs/space-info.c | 73 ++++++++++++++++++++++++++++--------------- 1 file changed, 48 insertions(+), 25 deletions(-)
fs/btrfs/space-info.c +48 -25
@@ -777,11 +777,11 @@ btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info, }static inline bool need_preemptive_reclaim (struct btrfs_fs_info *fs_info, -struct btrfs_space_info *space_info, -u64 used) +struct btrfs_space_info *space_info) { +u64 ordered, delalloc;u64 thresh =div_factor_fine (space_info->total_bytes,98 ); -u64 to_reclaim, expected; +u64 used;/* If we're just plain full then async reclaim just slows us down. */ if ((space_info->bytes_used + space_info->bytes_reserved) >= thresh)
@@ -794,26 +794,52 @@ static inline bool need_preemptive_reclaim(struct btrfs_fs_info *fs_info,if (space_info->reclaim_size)return 0 ; - to_reclaim =min_t (u64,num_online_cpus () *SZ_1M ,SZ_16M ); -if (btrfs_can_overcommit (fs_info, space_info, to_reclaim, -BTRFS_RESERVE_FLUSH_ALL )) -return 0 ; +/* +* If we have over half of the free space occupied by reservations or +* pinned then we want to start flushing. +* +* We do not do the traditional thing here, which is to say +* +* if (used >= ((total_bytes + avail) >> 1)) +* return 1; +* +* because this doesn't quite work how we want. If we had more than 50% +* of the space_info used by bytes_used and we had 0 available we'd just +* constantly run the background flusher. Instead we want it to kick in +* if our reclaimable space exceeds 50% of our available free space. +*/ + thresh =calc_available_free_space (fs_info, space_info, +BTRFS_RESERVE_FLUSH_ALL ); + thresh += (space_info->total_bytes - space_info->bytes_used - + space_info->bytes_reserved - space_info->bytes_readonly); + thresh >>=1 ; - used =btrfs_space_info_used (space_info, true); -if (btrfs_can_overcommit (fs_info, space_info,SZ_1M , -BTRFS_RESERVE_FLUSH_ALL )) - expected =div_factor_fine (space_info->total_bytes,95 ); -else - expected =div_factor_fine (space_info->total_bytes,90 ); + used = space_info->bytes_pinned; -if (used > expected) - to_reclaim = used - expected; +/* +* If we have more ordered bytes than delalloc bytes then we're either +* doing a lot of DIO, or we simply don't have a lot of delalloc waiting +* around. Preemptive flushing is only useful in that it can free up +* space before tickets need to wait for things to finish. In the case +* of ordered extents, preemptively waiting on ordered extents gets us +* nothing, if our reservations are tied up in ordered extents we'll +* simply have to slow down writers by forcing them to wait on ordered +* extents. +* +* In the case that ordered is larger than delalloc, only include the +* block reserves that we would actually be able to directly reclaim +* from. In this case if we're heavy on metadata operations this will +* clearly be heavy enough to warrant preemptive flushing. In the case +* of heavy DIO or ordered reservations, preemptive flushing will just +* waste time and cause us to slow down. +*/ + ordered =percpu_counter_sum_positive (&fs_info->ordered_bytes); + delalloc =percpu_counter_sum_positive (&fs_info->delalloc_bytes); +if (ordered >= delalloc) + used += fs_info->delayed_refs_rsv.reserved + + fs_info->delayed_block_rsv.reserved;else - to_reclaim =0 ; - to_reclaim =min (to_reclaim, space_info->bytes_may_use + - space_info->bytes_reserved); -if (!to_reclaim) -return 0 ; + used += space_info->bytes_may_use;return (used >= thresh && !btrfs_fs_closing (fs_info) && !test_bit (BTRFS_FS_STATE_REMOUNTING , &fs_info->fs_state));
@@ -1010,7 +1036,6 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)struct btrfs_block_rsv *delayed_refs_rsv;struct btrfs_block_rsv *global_rsv;struct btrfs_block_rsv *trans_rsv; -u64 used; fs_info = container_of(work, struct btrfs_fs_info, preempt_reclaim_work);
@@ -1021,8 +1046,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work) trans_rsv = &fs_info->trans_block_rsv;spin_lock (&space_info->lock); - used =btrfs_space_info_used (space_info, true); -while (need_preemptive_reclaim (fs_info, space_info, used)) { +while (need_preemptive_reclaim (fs_info, space_info)) {enum btrfs_flush_state flush;u64 delalloc_size =0 ;u64 to_reclaim, block_rsv_size;
@@ -1084,7 +1108,6 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)flush_space (fs_info, space_info, to_reclaim, flush);cond_resched ();spin_lock (&space_info->lock); - used =btrfs_space_info_used (space_info, true); }spin_unlock (&space_info->lock); }
@@ -1499,7 +1522,7 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info,* the async reclaim as we will panic. */ if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags) && -need_preemptive_reclaim (fs_info, space_info, used) && +need_preemptive_reclaim (fs_info, space_info) && !work_busy (&fs_info->preempt_reclaim_work)) {trace_btrfs_trigger_flush (fs_info, space_info->flags, orig_bytes, flush,"preempt" );
Signature · 1 line
2.26.2
Re: [PATCH v3 09/12] btrfs: simplify the logic in need_preemptive_flushing
Quoted 5 lines · Josef Bacik
A lot of this was added all in one go with no explanation, and is a bit unwieldy and confusing. Simplify the logic to start preemptive flushing if we've reserved more than half of our available free space.
Signed-off-by Josef Bacik <[email protected]>
PATCH v3 10/12 btrfs: implement space clamping for preemptive flushing
In reply to Josef BacikStarting preemptive flushing at 50% of available free space is a good start, but some workloads are particularly abusive and can quickly overwhelm the preemptive flushing code and drive us into using tickets.
Handle this by clamping down on our threshold for starting and continuing to run preemptive flushing. This is particularly important for our overcommit case, as we can really drive the file system into overages and then it's more difficult to pull it back as we start to actually fill up the file system.
The clamping is essentially 2^CLAMP, but we start at 1 so whatever we calculate for overcommit is the baseline.
--- fs/btrfs/space-info.c | 38 ++++++++++++++++++++++++++++++++++++-- fs/btrfs/space-info.h | 3 +++ 2 files changed, 39 insertions(+), 2 deletions(-)
fs/btrfs/space-info.c +36 -2
@@ -206,6 +206,7 @@ static int create_space_info(struct btrfs_fs_info *info, u64 flags)INIT_LIST_HEAD (&space_info->ro_bgs);INIT_LIST_HEAD (&space_info->tickets);INIT_LIST_HEAD (&space_info->priority_tickets); + space_info->clamp =1 ; ret =btrfs_sysfs_add_space_info_type (info, space_info);if (ret)
@@ -806,13 +807,13 @@ static inline bool need_preemptive_reclaim(struct btrfs_fs_info *fs_info,* because this doesn't quite work how we want. If we had more than 50% * of the space_info used by bytes_used and we had 0 available we'd just * constantly run the background flusher. Instead we want it to kick in -* if our reclaimable space exceeds 50% of our available free space. +* if our reclaimable space exceeds our clamped free space. */ thresh =calc_available_free_space (fs_info, space_info,BTRFS_RESERVE_FLUSH_ALL ); thresh += (space_info->total_bytes - space_info->bytes_used - space_info->bytes_reserved - space_info->bytes_readonly); - thresh >>=1 ; + thresh >>= space_info->clamp; used = space_info->bytes_pinned;
@@ -1036,6 +1037,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)struct btrfs_block_rsv *delayed_refs_rsv;struct btrfs_block_rsv *global_rsv;struct btrfs_block_rsv *trans_rsv; +int loops =0 ; fs_info = container_of(work, struct btrfs_fs_info, preempt_reclaim_work);
@@ -1052,6 +1054,8 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)u64 to_reclaim, block_rsv_size;u64 global_rsv_size = global_rsv->reserved; + loops++; +/* * We don't have a precise counter for the metadata being * reserved for delalloc, so we'll approximate it by subtracting
@@ -1109,6 +1113,10 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)cond_resched ();spin_lock (&space_info->lock); } + +/* We only went through once, back off our clamping. */ +if (loops ==1 && !space_info->reclaim_size) + space_info->clamp =max (1 , space_info->clamp -1 );spin_unlock (&space_info->lock); }
@@ -1422,6 +1430,24 @@ static inline bool is_normal_flushing(enum btrfs_reserve_flush_enum flush) (flush ==BTRFS_RESERVE_FLUSH_ALL_STEAL ); } +static inline void maybe_clamp_preempt (struct btrfs_fs_info *fs_info, +struct btrfs_space_info *space_info) +{ +u64 ordered =percpu_counter_sum_positive (&fs_info->ordered_bytes); +u64 delalloc =percpu_counter_sum_positive (&fs_info->delalloc_bytes); + +/* +* If we're heavy on ordered operations then clamping won't help us. We +* need to clamp specifically to keep up with dirty'ing buffered +* writers, because there's not a 1:1 correlation of writing delalloc +* and freeing space, like there is with flushing delayed refs or +* delayed nodes. If we're already more ordered than delalloc then +* we're keeping up, otherwise we aren't and should probably clamp. +*/ +if (ordered < delalloc) + space_info->clamp =min (space_info->clamp +1 ,8 ); +} +/** * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space * @root - the root we're allocating for
@@ -1514,6 +1540,14 @@ static int __reserve_bytes(struct btrfs_fs_info *fs_info,list_add_tail (&ticket.list, &space_info->priority_tickets); } + +/* +* We were forced to add a reserve ticket, so our preemptive +* flushing is unable to keep up. Clamp down on the threshold +* for the preemptive flushing in order to keep up with the +* workload. +*/ +maybe_clamp_preempt (fs_info, space_info); }else if (!ret && space_info->flags &BTRFS_BLOCK_GROUP_METADATA ) { used += orig_bytes;/*
fs/btrfs/space-info.h +3 -0
@@ -22,6 +22,9 @@ struct btrfs_space_info {the space info if we had an ENOSPC in the allocator. */ +int clamp;/* Used to scale our threshold for preemptive +flushing. */ + unsigned int full:1; /* indicates that we cannot allocate any morechunks for this space */ unsigned int chunk_alloc:1; /* set if we are allocating a chunk */
Signature · 1 line
2.26.2
Re: [PATCH v3 10/12] btrfs: implement space clamping for preemptive flushing
Quoted 6 lines · Josef Bacik
fs/btrfs/space-info.c +6
-0+static inline void maybe_clamp_preempt (struct btrfs_fs_info *fs_info, +struct btrfs_space_info *space_info) +{ +u64 ordered =percpu_counter_sum_positive (&fs_info->ordered_bytes); +u64 delalloc =percpu_counter_sum_positive (&fs_info->delalloc_bytes); +
fs/btrfs/space-info.c static inline bool is_normal_flushing(enum btrfs_reserve_flush_enum flush) · line 1448
Quoted 10 lines
fs/btrfs/space-info.c +10
-0+/* +* If we're heavy on ordered operations then clamping won't help us. We +* need to clamp specifically to keep up with dirty'ing buffered +* writers, because there's not a 1:1 correlation of writing delalloc +* and freeing space, like there is with flushing delayed refs or +* delayed nodes. If we're already more ordered than delalloc then +* we're keeping up, otherwise we aren't and should probably clamp. +*/ +if (ordered < delalloc) + space_info->clamp =min (space_info->clamp +1 ,8 );
So the divisor will go up to 2^8 = 256, is that intentional?
fs/btrfs/space-info.h struct btrfs_space_info { · line 26
Quoted 8 lines
fs/btrfs/space-info.h +2
-0@@ -22,6 +22,9 @@ struct btrfs_space_info {the space info if we had an ENOSPC in the allocator. */ +int clamp;/* Used to scale our threshold for preemptive +flushing. */
Struct comments should go on the line before so we don't have this awkward formatting but the rest of the struct has that so let it be. One thing I'm missing is that it's power of two, I'll add it there.
Quoted 6 lines
fs/btrfs/space-info.h +1
-0+ unsigned int full:1; /* indicates that we cannot allocate any morechunks for this space */ unsigned int chunk_alloc:1; /* set if we are allocating a chunk */Signature · 1 line
2.26.2
PATCH v3 11/12 btrfs: adjust the flush trace point to include the source
In reply to Josef BacikSince we have normal ticketed flushing and preemptive flushing, adjust the tracepoint so that we know the source of the flushing action to make it easier to debug problems.
--- fs/btrfs/space-info.c | 20 ++++++++++++-------- include/trace/events/btrfs.h | 10 ++++++---- 2 files changed, 18 insertions(+), 12 deletions(-)
fs/btrfs/space-info.c +12 -8
@@ -664,7 +664,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,*/ static void flush_space (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,u64 num_bytes, -enum btrfs_flush_state state) +enum btrfs_flush_state state,bool for_preempt) {struct btrfs_root *root = fs_info->extent_root;struct btrfs_trans_handle *trans;
@@ -747,7 +747,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, }trace_btrfs_flush_space (fs_info, space_info->flags, num_bytes, state, - ret); + ret, for_preempt);return ; }
@@ -973,7 +973,8 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) flush_state =FLUSH_DELAYED_ITEMS_NR ; do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, flush_state, + false); spin_lock(&space_info->lock); if (list_empty(&space_info->tickets)) { space_info->flush = 0;
@@ -1109,7 +1110,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work) to_reclaim >>=2 ;if (!to_reclaim) to_reclaim =btrfs_calc_insert_metadata_size (fs_info,1 ); -flush_space (fs_info, space_info, to_reclaim, flush); +flush_space (fs_info, space_info, to_reclaim, flush, true);cond_resched ();spin_lock (&space_info->lock); }
@@ -1200,7 +1201,8 @@ static void btrfs_async_reclaim_data_space(struct work_struct *work)spin_unlock (&space_info->lock);while (!space_info->full) { -flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE ); +flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE , + false);spin_lock (&space_info->lock);if (list_empty (&space_info->tickets)) { space_info->flush =0 ;
@@ -1213,7 +1215,7 @@ static void btrfs_async_reclaim_data_space(struct work_struct *work)while (flush_state <ARRAY_SIZE (data_flush_states)) {flush_space (fs_info, space_info,U64_MAX , - data_flush_states[flush_state]); + data_flush_states[flush_state], false);spin_lock (&space_info->lock);if (list_empty (&space_info->tickets)) { space_info->flush =0 ;
@@ -1286,7 +1288,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, flush_state =0 ; do { - flush_space(fs_info, space_info, to_reclaim, states[flush_state]); + flush_space(fs_info, space_info, to_reclaim, states[flush_state], + false); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) {
@@ -1302,7 +1305,8 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, struct reserve_ticket *ticket) {while (!space_info->full) { -flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE ); +flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE , + false);spin_lock (&space_info->lock);if (ticket->bytes ==0 ) {spin_unlock (&space_info->lock);
include/trace/events/btrfs.h +6 -4
@@ -1112,15 +1112,16 @@ TRACE_EVENT(btrfs_trigger_flush, TRACE_EVENT(btrfs_flush_space, TP_PROTO(const struct btrfs_fs_info *fs_info, u64 flags, u64 num_bytes, - int state, int ret), + int state, int ret, int for_preempt), - TP_ARGS(fs_info, flags, num_bytes, state, ret), + TP_ARGS(fs_info, flags, num_bytes, state, ret, for_preempt), TP_STRUCT__entry_btrfs( __field( u64, flags ) __field( u64, num_bytes ) __field( int, state ) __field( int, ret ) + __field( int, for_preempt ) ), TP_fast_assign_btrfs(fs_info,
@@ -1128,15 +1129,16 @@ TRACE_EVENT(btrfs_flush_space, __entry->num_bytes = num_bytes; __entry->state = state; __entry->ret = ret; + __entry->for_preempt = for_preempt; ), -TP_printk_btrfs ("state=%d(%s) flags=%llu(%s) num_bytes=%llu ret=%d" , +TP_printk_btrfs ("state=%d(%s) flags=%llu(%s) num_bytes=%llu ret=%d for_preempt=%d" , __entry->state,__print_symbolic (__entry->state,FLUSH_STATES ), __entry->flags,__print_flags ((unsigned long )__entry->flags,"|" ,BTRFS_GROUP_FLAGS ), - __entry->num_bytes, __entry->ret) + __entry->num_bytes, __entry->ret, __entry->for_preempt) ); DECLARE_EVENT_CLASS(btrfs__reserved_extent,
Signature · 1 line
2.26.2
Re: [PATCH v3 11/12] btrfs: adjust the flush trace point to include the source
Quoted 96 lines · Josef Bacik
Since we have normal ticketed flushing and preemptive flushing, adjust the tracepoint so that we know the source of the flushing action to make it easier to debug problems.
Reviewed-by Nikolay Borisov <[email protected]>Signed-off-by Josef Bacik <[email protected]>--- fs/btrfs/space-info.c | 20 ++++++++++++-------- include/trace/events/btrfs.h | 10 ++++++---- 2 files changed, 18 insertions(+), 12 deletions(-)fs/btrfs/space-info.c +12
-8@@ -664,7 +664,7 @@ static int may_commit_transaction(struct btrfs_fs_info *fs_info,*/ static void flush_space (struct btrfs_fs_info *fs_info,struct btrfs_space_info *space_info,u64 num_bytes, -enum btrfs_flush_state state) +enum btrfs_flush_state state,bool for_preempt) {struct btrfs_root *root = fs_info->extent_root;struct btrfs_trans_handle *trans;@@ -747,7 +747,7 @@ static void flush_space(struct btrfs_fs_info *fs_info, }trace_btrfs_flush_space (fs_info, space_info->flags, num_bytes, state, - ret); + ret, for_preempt);return ; }@@ -973,7 +973,8 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work) flush_state =FLUSH_DELAYED_ITEMS_NR ; do { - flush_space(fs_info, space_info, to_reclaim, flush_state); + flush_space(fs_info, space_info, to_reclaim, flush_state, + false); spin_lock(&space_info->lock); if (list_empty(&space_info->tickets)) { space_info->flush = 0;@@ -1109,7 +1110,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work) to_reclaim >>=2 ;if (!to_reclaim) to_reclaim =btrfs_calc_insert_metadata_size (fs_info,1 ); -flush_space (fs_info, space_info, to_reclaim, flush); +flush_space (fs_info, space_info, to_reclaim, flush, true);cond_resched ();spin_lock (&space_info->lock); }@@ -1200,7 +1201,8 @@ static void btrfs_async_reclaim_data_space(struct work_struct *work)spin_unlock (&space_info->lock);while (!space_info->full) { -flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE ); +flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE , + false);spin_lock (&space_info->lock);if (list_empty (&space_info->tickets)) { space_info->flush =0 ;@@ -1213,7 +1215,7 @@ static void btrfs_async_reclaim_data_space(struct work_struct *work)while (flush_state <ARRAY_SIZE (data_flush_states)) {flush_space (fs_info, space_info,U64_MAX , - data_flush_states[flush_state]); + data_flush_states[flush_state], false);spin_lock (&space_info->lock);if (list_empty (&space_info->tickets)) { space_info->flush =0 ;@@ -1286,7 +1288,8 @@ static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info, flush_state =0 ; do { - flush_space(fs_info, space_info, to_reclaim, states[flush_state]); + flush_space(fs_info, space_info, to_reclaim, states[flush_state], + false); flush_state++; spin_lock(&space_info->lock); if (ticket->bytes == 0) {@@ -1302,7 +1305,8 @@ static void priority_reclaim_data_space(struct btrfs_fs_info *fs_info, struct reserve_ticket *ticket) {while (!space_info->full) { -flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE ); +flush_space (fs_info, space_info,U64_MAX ,ALLOC_CHUNK_FORCE , + false);spin_lock (&space_info->lock);if (ticket->bytes ==0 ) {spin_unlock (&space_info->lock);include/trace/events/btrfs.h +0
-0
include/trace/events/btrfs.h TRACE_EVENT(btrfs_trigger_flush, · line 1115
Quoted 5 lines
include/trace/events/btrfs.h +1
-1@@ -1112,15 +1112,16 @@ TRACE_EVENT(btrfs_trigger_flush, TRACE_EVENT(btrfs_flush_space, TP_PROTO(const struct btrfs_fs_info *fs_info, u64 flags, u64 num_bytes, - int state, int ret), + int state, int ret, int for_preempt),
for_preempt should be bool
Quoted 34 lines
include/trace/events/btrfs.h +5
-3-TP_ARGS (fs_info, flags, num_bytes, state, ret), + TP_ARGS(fs_info, flags, num_bytes, state, ret, for_preempt),TP_STRUCT__entry_btrfs (__field (u64 ,flags )__field ( u64, num_bytes )__field ( int, state )__field ( int, ret ) +__field ( int, for_preempt ) ), TP_fast_assign_btrfs(fs_info,@@ -1128,15 +1129,16 @@ TRACE_EVENT(btrfs_flush_space, __entry->num_bytes = num_bytes; __entry->state = state; __entry->ret = ret; + __entry->for_preempt = for_preempt; ), -TP_printk_btrfs ("state=%d(%s) flags=%llu(%s) num_bytes=%llu ret=%d" , +TP_printk_btrfs ("state=%d(%s) flags=%llu(%s) num_bytes=%llu ret=%d for_preempt=%d" , __entry->state,__print_symbolic (__entry->state,FLUSH_STATES ), __entry->flags,__print_flags ((unsigned long )__entry->flags,"|" ,BTRFS_GROUP_FLAGS ), - __entry->num_bytes, __entry->ret) + __entry->num_bytes, __entry->ret, __entry->for_preempt) ); DECLARE_EVENT_CLASS(btrfs__reserved_extent,Signature · 1 line
2.26.2
PATCH v3 12/12 btrfs: add a trace class for dumping the current ENOSPC state
In reply to Josef BacikOften when I'm debugging ENOSPC related issues I have to resort to printing the entire ENOSPC state with trace_printk() in different spots. This gets pretty annoying, so add a trace state that does this for us. Then add a trace point at the end of preemptive flushing so you can see the state of the space_info when we decide to exit preemptive flushing. This helped me figure out we weren't kicking in the preemptive flushing soon enough.
--- fs/btrfs/space-info.c | 1 + include/trace/events/btrfs.h | 62 ++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+)
fs/btrfs/space-info.c +1 -0
@@ -1118,6 +1118,7 @@ static void btrfs_preempt_reclaim_metadata_space(struct work_struct *work)/* We only went through once, back off our clamping. */ if (loops ==1 && !space_info->reclaim_size) space_info->clamp =max (1 , space_info->clamp -1 ); +trace_btrfs_done_preemptive_reclaim (fs_info, space_info);spin_unlock (&space_info->lock); }
include/trace/events/btrfs.h +62 -0
68 lines not on this page · raw message
Signature · 1 line
2.26.2