btrfs: async discard support
Subsystems (4)
Dennis Zhou
maintainerv2 cover btrfs: async discard support
Hello,
The RFC [1] went a lot better than expected, so let's call this v2.
Changes:
- I don't believe there are any functional changes to the code, but there are plenty of changes for readability and a lot of comments added.
- Split out the sysfs parts into their own patches as it was a bit of work to put discard/ under debug/.
- Switched over any flags to be enums. All the variables are now of appropriate type rather than being all atomics.
- Dropped 0015 [2] from v1, load block groups on mount. This is in favor of reading everything in as clean and running fstrim as needed.
- Misc. bug fixes.
- I changed the block_group discard delay from 300s to 120s. It is just to give the allocators a chance to reuse the LBA before letting async discard take a crack at it.
Data:
On a number of webservers, I collected data every minute accounting the time we spent in btrfs_finish_extent_commit() (col. 1) and in btrfs_commit_transaction() (col. 2). btrfs_finish_extent_commit() is where we discard extents synchronously before returning them to the free space cache.
discard=sync:
p99 total per minute p99 total per minute
Drive | extent_commit() (ms) | commit_trans() (ms)
---------------------------------------------------------------
Drive A | 434 | 1170
Drive B | 880 | 2330
Drive C | 2943 | 3920
Drive D | 4763 | 5701
discard=async:
p99 total per minute p99 total per minute
Drive | extent_commit() (ms) | commit_trans() (ms)
--------------------------------------------------------------
Drive A | 134 | 956
Drive B | 64 | 1972
Drive C | 59 | 1032
Drive D | 62 | 1200
While it's not great that the stats are cumulative over 1m, all of these servers are running the same workload and and the delta between the two are substantial. We are spending significantly less time in btrfs_finish_extent_commit() which is responsible for discarding.
From v1:
--------
Discard is an operation that allows for the filesystem to communicate with underlying ssds that a lba region is no longer needed. This gives the drive the more information as it tries to manage the available free space to minimize write amplification. However, discard hasn't been given the most tlc. Discard is a problematic command because a drive's physical block management is more or less a black box to us and the effects of any particular discard aren't necessarily limited the lifetime of a command.
Currently, btrfs handles discarding synchronously during transaction commit. This problematically can delay transaction commit based on the amount of space that needs to be trimmed and the efficacy of the discard operation for a particular drive.
This series introduces async discarding, which removes discard from the transaction commit path. While every SSD has the choice of implementing trim support different, we strive here to do the right thing. The idea hinges on recognizing that write amplification really only kicks in once we're really low on free space. As long as we trim enough to keep a large enough pool of free space, in theory this should minimize the cost of issuing discards on a workload and have limited cost overhead in write amplification.
With async discard, we try to emphasize discarding larger regions and reusing the lba (implicit discard). The first is done by using the free space cache to maintain discard state and thus allows us to get coalescing for fairly cheap. A background workqueue is used to scan over an LRU kept list of the block groups. It then uses filters to determine what to discard next hence giving priority to larger discards. While reusing an lba isn't explicitly attempted, it happens implicitly via find_free_extent() which if it happens to find a dirty extent, will grant us reuse of the lba. Additionally, async discarding skips metadata block groups as these should see a fairly high turnover as btrfs is a self-packing filesystem being stingy with allocating new block groups until necessary.
Preliminary results seem promising as when a lot of freeing is going on, the discarding is delayed allowing for reuse which translates to less discarding (in addition to the slower discarding). This has shown a reduction in p90 and p99 read latencies on a test on our webservers.
I am currently working on tuning the rate at which it discards in the background. I am doing this by evaluating other workloads and drives. The iops and bps rate limits are fairly aggressive right now as my basic survey of a few drives noted that the trim command itself is a significant part of the overhead. So optimizing for larger trims is the right thing to do.
This series contains the following 22 patches and is on top of btrfs-devel#master 3b7c59a1950c:
0001-bitmap-genericize-percpu-bitmap-region-iterators.patch 0002-btrfs-rename-DISCARD-opt-to-DISCARD_SYNC.patch 0003-btrfs-keep-track-of-which-extents-have-been-discarde.patch 0004-btrfs-keep-track-of-cleanliness-of-the-bitmap.patch 0005-btrfs-add-the-beginning-of-async-discard-discard-wor.patch 0006-btrfs-handle-empty-block_group-removal.patch 0007-btrfs-discard-one-region-at-a-time-in-async-discard.patch 0008-btrfs-add-removal-calls-for-sysfs-debug.patch 0009-btrfs-make-UUID-debug-have-its-own-kobject.patch 0010-btrfs-add-discard-sysfs-directory.patch 0011-btrfs-track-discardable-extents-for-async-discard.patch 0012-btrfs-keep-track-of-discardable_bytes.patch 0013-btrfs-calculate-discard-delay-based-on-number-of-ext.patch 0014-btrfs-add-bps-discard-rate-limit.patch 0015-btrfs-limit-max-discard-size-for-async-discard.patch 0016-btrfs-make-max-async-discard-size-tunable.patch 0017-btrfs-have-multiple-discard-lists.patch 0018-btrfs-only-keep-track-of-data-extents-for-async-disc.patch 0019-btrfs-keep-track-of-discard-reuse-stats.patch 0020-btrfs-add-async-discard-header.patch 0021-btrfs-increase-the-metadata-allowance-for-the-free_s.patch 0022-btrfs-make-smaller-extents-more-likely-to-go-into-bi.patch
0001 exports percpu's bitmap iterators for eventual use in 0011. 0002 renames DISCARD to DISCARD_SYNC. 0003 and 0004 adds discard tracking to the free space cache. 0005-0007 adds the core of async discard support. 0008-0010 modify debug sysfs to allow for adding discard/ under debug/. 0011-0016 fiddle with stats and operation limits. 0018 makes async discarding only track data block groups. 0019 adds reuse stats. 0020 adds an explanation header to discard.c. 0021 and 0022 modify the free space cache metadata allowance, add a bitmap -> extent path and makes us more likely to put smaller extents into the bitmaps.
diffstats below:
Dennis Zhou (22):
bitmap: genericize percpu bitmap region iterators
btrfs: rename DISCARD opt to DISCARD_SYNC
btrfs: keep track of which extents have been discarded
btrfs: keep track of cleanliness of the bitmap
btrfs: add the beginning of async discard, discard workqueue
btrfs: handle empty block_group removal
btrfs: discard one region at a time in async discard
btrfs: add removal calls for sysfs debug/
btrfs: make UUID/debug have its own kobject
btrfs: add discard sysfs directory
btrfs: track discardable extents for async discard
btrfs: keep track of discardable_bytes
btrfs: calculate discard delay based on number of extents
btrfs: add bps discard rate limit
btrfs: limit max discard size for async discard
btrfs: make max async discard size tunable
btrfs: have multiple discard lists
btrfs: only keep track of data extents for async discard
btrfs: keep track of discard reuse stats
btrfs: add async discard header
btrfs: increase the metadata allowance for the free_space_cache
btrfs: make smaller extents more likely to go into bitmaps
fs/btrfs/Makefile | 2 +- fs/btrfs/block-group.c | 56 ++- fs/btrfs/block-group.h | 30 ++ fs/btrfs/ctree.h | 52 ++- fs/btrfs/discard.c | 666 ++++++++++++++++++++++++++++++++++++ fs/btrfs/discard.h | 48 +++ fs/btrfs/disk-io.c | 15 +- fs/btrfs/extent-tree.c | 23 +- fs/btrfs/free-space-cache.c | 587 ++++++++++++++++++++++++++----- fs/btrfs/free-space-cache.h | 39 ++- fs/btrfs/inode-map.c | 13 +- fs/btrfs/scrub.c | 7 +- fs/btrfs/super.c | 39 ++- fs/btrfs/sysfs.c | 203 ++++++++++- include/linux/bitmap.h | 35 ++ mm/percpu.c | 61 +--- 16 files changed, 1723 insertions(+), 153 deletions(-) create mode 100644 fs/btrfs/discard.c create mode 100644 fs/btrfs/discard.h
Thanks,
Dennis
Dennis Zhou
maintainerPATCH v2 1/22 bitmap: genericize percpu bitmap region iterators
Bitmaps are fairly popular for their space efficiency, but we don't have generic iterators available. Make percpu's bitmap region iterators available to everyone.
--- include/linux/bitmap.h | 35 ++++++++++++++++++++++++ mm/percpu.c | 61 +++++++++++------------------------------- 2 files changed, 51 insertions(+), 45 deletions(-)
include/linux/bitmap.h +35 -0
@@ -438,6 +438,41 @@ static inline int bitmap_parse(const char *buf, unsigned int buflen,return __bitmap_parse (buf, buflen,0 , maskp, nmaskbits); } +static inline void bitmap_next_clear_region (unsigned long *bitmap, +unsigned int *rs,unsigned int *re, +unsigned int end) +{ + *rs =find_next_zero_bit (bitmap, end, *rs); + *re =find_next_bit (bitmap, end, *rs +1 ); +} + +static inline void bitmap_next_set_region (unsigned long *bitmap, +unsigned int *rs,unsigned int *re, +unsigned int end) +{ + *rs =find_next_bit (bitmap, end, *rs); + *re =find_next_zero_bit (bitmap, end, *rs +1 ); +} + +/* +* Bitmap region iterators. Iterates over the bitmap between [@start, @end). +* @rs and @re should be integer variables and will be set to start and end +* index of the current clear or set region. +*/ +#define bitmap_for_each_clear_region (bitmap, rs, re, start, end) \ + for ((rs) = (start), \ + bitmap_next_clear_region((bitmap), &(rs), &(re), (end)); \ + (rs) < (re); \ + (rs) = (re) + 1, \ + bitmap_next_clear_region((bitmap), &(rs), &(re), (end))) + +#define bitmap_for_each_set_region (bitmap, rs, re, start, end) \ + for ((rs) = (start), \ + bitmap_next_set_region((bitmap), &(rs), &(re), (end)); \ + (rs) < (re); \ + (rs) = (re) + 1, \ + bitmap_next_set_region((bitmap), &(rs), &(re), (end))) +/** * BITMAP_FROM_U64() - Represent u64 value in the format suitable for bitmap. * @n: u64 value
mm/percpu.c +16 -45
127 lines not on this page · raw message
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 2/22 btrfs: rename DISCARD opt to DISCARD_SYNC
In reply to Dennis ZhouThis series introduces async discard which will use the flag DISCARD_ASYNC, so rename the original flag to DISCARD_SYNC as it is synchronously done in transaction commit.
--- fs/btrfs/block-group.c | 2 +- fs/btrfs/ctree.h | 2 +- fs/btrfs/extent-tree.c | 4 ++-- fs/btrfs/super.c | 8 ++++---- 4 files changed, 8 insertions(+), 8 deletions(-)
fs/btrfs/block-group.c +1 -1
@@ -1365,7 +1365,7 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)spin_unlock (&space_info->lock);/* DISCARD can flip during remount */ - trimming =btrfs_test_opt (fs_info,DISCARD ); + trimming =btrfs_test_opt (fs_info,DISCARD_SYNC );/* Implicit trim during transaction commit. */ if (trimming)
fs/btrfs/ctree.h +1 -1
@@ -1171,7 +1171,7 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info)#define BTRFS_MOUNT_FLUSHONCOMMIT (1 << 7)#define BTRFS_MOUNT_SSD_SPREAD (1 << 8)#define BTRFS_MOUNT_NOSSD (1 << 9) -#define BTRFS_MOUNT_DISCARD (1 << 10) +#define BTRFS_MOUNT_DISCARD_SYNC (1 << 10)#define BTRFS_MOUNT_FORCE_COMPRESS (1 << 11)#define BTRFS_MOUNT_SPACE_CACHE (1 << 12)#define BTRFS_MOUNT_CLEAR_CACHE (1 << 13)
fs/btrfs/extent-tree.c +2 -2
@@ -2903,7 +2903,7 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)break ; } -if (btrfs_test_opt (fs_info,DISCARD )) +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) ret =btrfs_discard_extent (fs_info, start, end +1 - start,NULL );
@@ -4146,7 +4146,7 @@ static int __btrfs_free_reserved_extent(struct btrfs_fs_info *fs_info,if (pin)pin_down_extent (cache, start, len,1 );else { -if (btrfs_test_opt (fs_info,DISCARD )) +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) ret =btrfs_discard_extent (fs_info, start, len,NULL );btrfs_add_free_space (cache, start, len);btrfs_free_reserved_bytes (cache, len, delalloc);
fs/btrfs/super.c +4 -4
@@ -695,11 +695,11 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options, info->metadata_ratio);break ;case Opt_discard: -btrfs_set_and_info (info,DISCARD , -"turning on discard" ); +btrfs_set_and_info (info,DISCARD_SYNC , +"turning on sync discard" );break ;case Opt_nodiscard: -btrfs_clear_and_info (info,DISCARD , +btrfs_clear_and_info (info,DISCARD_SYNC ,"turning off discard" );break ;case Opt_space_cache:
@@ -1322,7 +1322,7 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)seq_puts (seq,",nologreplay" );if (btrfs_test_opt (info,FLUSHONCOMMIT ))seq_puts (seq,",flushoncommit" ); -if (btrfs_test_opt (info,DISCARD )) +if (btrfs_test_opt (info,DISCARD_SYNC ))seq_puts (seq,",discard" );if (!(info->sb->s_flags &SB_POSIXACL ))seq_puts (seq,",noacl" );
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 3/22 btrfs: keep track of which extents have been discarded
In reply to Dennis ZhouAsync discard will use the free space cache as backing knowledge for which extents to discard. This patch plumbs knowledge about which extents need to be discarded into the free space cache from unpin_extent_range().
An untrimmed extent can merge with everything as this is a new region. Absorbing trimmed extents is a tradeoff to for greater coalescing which makes life better for find_free_extent(). Additionally, it seems the size of a trim isn't as problematic as the trim io itself.
When reading in the free space cache from disk, if sync is set, mark all extents as trimmed. The current code ensures at transaction commit that all free space is trimmed when sync is set, so this reflects that.
--- fs/btrfs/extent-tree.c | 15 +++++++--- fs/btrfs/free-space-cache.c | 60 ++++++++++++++++++++++++++++++++----- fs/btrfs/free-space-cache.h | 17 ++++++++++- fs/btrfs/inode-map.c | 13 ++++---- 4 files changed, 87 insertions(+), 18 deletions(-)
fs/btrfs/extent-tree.c +11 -4
@@ -2783,6 +2783,7 @@ fetch_cluster_info(struct btrfs_fs_info *fs_info,static int unpin_extent_range (struct btrfs_fs_info *fs_info,u64 start,u64 end, +enum btrfs_trim_state trim_state,const bool return_free_space) {struct btrfs_block_group_cache *cache =NULL ;
@@ -2816,7 +2817,9 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info,if (start < cache->last_byte_to_unpin) { len =min (len, cache->last_byte_to_unpin - start);if (return_free_space) -btrfs_add_free_space (cache, start, len); +__btrfs_add_free_space (fs_info, + cache->free_space_ctl, + start, len, trim_state); } start += len;
@@ -2894,6 +2897,7 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)while (!trans->aborted) {struct extent_state *cached_state =NULL ; +enum btrfs_trim_state trim_state =BTRFS_TRIM_STATE_UNTRIMMED ;mutex_lock (&fs_info->unused_bg_unpin_mutex); ret = find_first_extent_bit(unpin, 0, &start, &end,
@@ -2903,12 +2907,14 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)break ; } -if (btrfs_test_opt (fs_info,DISCARD_SYNC )) +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) { ret =btrfs_discard_extent (fs_info, start, end +1 - start,NULL ); + trim_state =BTRFS_TRIM_STATE_TRIMMED ; + }clear_extent_dirty (unpin, start, end, &cached_state); -unpin_extent_range (fs_info, start, end, true); +unpin_extent_range (fs_info, start, end, trim_state, true);mutex_unlock (&fs_info->unused_bg_unpin_mutex);free_extent_state (cached_state);cond_resched ();
@@ -5512,7 +5518,8 @@ u64 btrfs_account_ro_block_groups_free_space(struct btrfs_space_info *sinfo)int btrfs_error_unpin_extent_range (struct btrfs_fs_info *fs_info,u64 start,u64 end) { -return unpin_extent_range (fs_info, start, end, false); +return unpin_extent_range (fs_info, start, end, +BTRFS_TRIM_STATE_UNTRIMMED , false); }/*
fs/btrfs/free-space-cache.c +53 -7
145 lines not on this page · raw message
fs/btrfs/free-space-cache.h +16 -1
@@ -6,6 +6,14 @@ #ifndef BTRFS_FREE_SPACE_CACHE_H #define BTRFS_FREE_SPACE_CACHE_H +/* + * This is the trim state of an extent or bitmap. + */ +enum btrfs_trim_state { + BTRFS_TRIM_STATE_TRIMMED, + BTRFS_TRIM_STATE_UNTRIMMED, +}; + struct btrfs_free_space { struct rb_node offset_index; u64 offset;
@@ -13,8 +21,14 @@ struct btrfs_free_space {u64 max_extent_size;unsigned long *bitmap;struct list_head list; +enum btrfs_trim_state trim_state; }; +static inline bool btrfs_free_space_trimmed (struct btrfs_free_space *info) +{ +return (info->trim_state ==BTRFS_TRIM_STATE_TRIMMED ); +} +struct btrfs_free_space_ctl {spinlock_t tree_lock;struct rb_root free_space_offset;
@@ -84,7 +98,8 @@ int btrfs_write_out_ino_cache(struct btrfs_root *root,void btrfs_init_free_space_ctl (struct btrfs_block_group_cache *block_group);int __btrfs_add_free_space (struct btrfs_fs_info *fs_info,struct btrfs_free_space_ctl *ctl, -u64 bytenr,u64 size); +u64 bytenr,u64 size, +enum btrfs_trim_state trim_state);int btrfs_add_free_space (struct btrfs_block_group_cache *block_group,u64 bytenr,u64 size); int btrfs_remove_free_space(struct btrfs_block_group_cache *block_group,
fs/btrfs/inode-map.c +7 -6
@@ -107,7 +107,7 @@ static int caching_kthread(void *data)if (last != (u64)-1 && last +1 != key.objectid) {__btrfs_add_free_space (fs_info, ctl, last +1 , - key.objectid - last -1 ); + key.objectid - last -1 ,0 );wake_up (&root->ino_cache_wait); }
@@ -118,7 +118,7 @@ static int caching_kthread(void *data)if (last < root->highest_objectid -1 ) {__btrfs_add_free_space (fs_info, ctl, last +1 , - root->highest_objectid - last -1 ); + root->highest_objectid - last -1 ,0 ); }spin_lock (&root->ino_cache_lock);
@@ -175,7 +175,8 @@ static void start_caching(struct btrfs_root *root) ret =btrfs_find_free_objectid (root, &objectid);if (!ret && objectid <=BTRFS_LAST_FREE_OBJECTID ) {__btrfs_add_free_space (fs_info, ctl, objectid, -BTRFS_LAST_FREE_OBJECTID - objectid +1 ); +BTRFS_LAST_FREE_OBJECTID - objectid +1 , +0 );wake_up (&root->ino_cache_wait); }
@@ -221,7 +222,7 @@ void btrfs_return_ino(struct btrfs_root *root, u64 objectid)return ; again:if (root->ino_cache_state ==BTRFS_CACHE_FINISHED ) { -__btrfs_add_free_space (fs_info, pinned, objectid,1 ); +__btrfs_add_free_space (fs_info, pinned, objectid,1 ,0 ); }else {down_write (&fs_info->commit_root_sem);spin_lock (&root->ino_cache_lock);
@@ -234,7 +235,7 @@ void btrfs_return_ino(struct btrfs_root *root, u64 objectid)start_caching (root); -__btrfs_add_free_space (fs_info, pinned, objectid,1 ); +__btrfs_add_free_space (fs_info, pinned, objectid,1 ,0 );up_write (&fs_info->commit_root_sem); }
@@ -281,7 +282,7 @@ void btrfs_unpin_free_ino(struct btrfs_root *root)spin_unlock (rbroot_lock);if (count)__btrfs_add_free_space (root->fs_info, ctl, - info->offset, count); + info->offset, count,0 );kmem_cache_free (btrfs_free_space_cachep, info); } }
Signature · 1 line
2.17.1
Re: [PATCH 03/22] btrfs: keep track of which extents have been discarded
Quoted 86 lines · Dennis Zhou
Async discard will use the free space cache as backing knowledge for which extents to discard. This patch plumbs knowledge about which extents need to be discarded into the free space cache from
unpin_extent_range().An untrimmed extent can merge with everything as this is a new region. Absorbing trimmed extents is a tradeoff to for greater coalescing which makes life better for
find_free_extent(). Additionally, it seems the size of a trim isn't as problematic as the trim io itself.When reading in the free space cache from disk, if sync is set, mark all extents as trimmed. The current code ensures at transaction commit that all free space is trimmed when sync is set, so this reflects that.
Signed-off-by Dennis Zhou <[email protected]>--- fs/btrfs/extent-tree.c | 15 +++++++--- fs/btrfs/free-space-cache.c | 60 ++++++++++++++++++++++++++++++++----- fs/btrfs/free-space-cache.h | 17 ++++++++++- fs/btrfs/inode-map.c | 13 ++++---- 4 files changed, 87 insertions(+), 18 deletions(-)fs/btrfs/extent-tree.c +11
-4@@ -2783,6 +2783,7 @@ fetch_cluster_info(struct btrfs_fs_info *fs_info,static int unpin_extent_range (struct btrfs_fs_info *fs_info,u64 start,u64 end, +enum btrfs_trim_state trim_state,const bool return_free_space) {struct btrfs_block_group_cache *cache =NULL ;@@ -2816,7 +2817,9 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info,if (start < cache->last_byte_to_unpin) { len =min (len, cache->last_byte_to_unpin - start);if (return_free_space) -btrfs_add_free_space (cache, start, len); +__btrfs_add_free_space (fs_info, + cache->free_space_ctl, + start, len, trim_state); } start += len;@@ -2894,6 +2897,7 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)while (!trans->aborted) {struct extent_state *cached_state =NULL ; +enum btrfs_trim_state trim_state =BTRFS_TRIM_STATE_UNTRIMMED ;mutex_lock (&fs_info->unused_bg_unpin_mutex); ret = find_first_extent_bit(unpin, 0, &start, &end,@@ -2903,12 +2907,14 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)break ; } -if (btrfs_test_opt (fs_info,DISCARD_SYNC )) +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) { ret =btrfs_discard_extent (fs_info, start, end +1 - start,NULL ); + trim_state =BTRFS_TRIM_STATE_TRIMMED ; + }clear_extent_dirty (unpin, start, end, &cached_state); -unpin_extent_range (fs_info, start, end, true); +unpin_extent_range (fs_info, start, end, trim_state, true);mutex_unlock (&fs_info->unused_bg_unpin_mutex);free_extent_state (cached_state);cond_resched ();@@ -5512,7 +5518,8 @@ u64 btrfs_account_ro_block_groups_free_space(struct btrfs_space_info *sinfo)int btrfs_error_unpin_extent_range (struct btrfs_fs_info *fs_info,u64 start,u64 end) { -return unpin_extent_range (fs_info, start, end, false); +return unpin_extent_range (fs_info, start, end, +BTRFS_TRIM_STATE_UNTRIMMED , false); }/* fs/btrfs/free-space-cache.c +0
-0@@ -747,6 +747,14 @@ static int __load_free_space_cache(struct btrfs_root *root, struct inode *inode, goto free_cache;
fs/btrfs/free-space-cache.c static int __load_free_space_cache(struct btrfs_root *root, struct inode *inode, · line 757
Quoted 10 lines
fs/btrfs/free-space-cache.c +8
-0} +/* +* Sync discard ensures that the free space cache is always +* trimmed. So when reading this in, the state should reflect +* that. +*/ +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) + e->trim_state =BTRFS_TRIM_STATE_TRIMMED ; +
BTRFS_TRIM_STATE_TRIMMED == 0, so if we don't have DISCARD_SYNC set, we'll still end up with TRIMMED. Which means we'll end up with weirdness later because unpinned extents will come back as UNTRIMMED in the !DISCARD_SYNC case. I'm thinking about spinning rust here where we'll never have discard, so we should either always treat the free space as trimmed, or always treat it as untrimmed, so we don't end up with different allocator behavior for the undiscardable devices. Thanks,
Josef
Dennis Zhou
maintainerRe: [PATCH 03/22] btrfs: keep track of which extents have been discarded
Quoted 89 lines · Josef Bacik
Quoted 88 lines · Dennis Zhou
Async discard will use the free space cache as backing knowledge for which extents to discard. This patch plumbs knowledge about which extents need to be discarded into the free space cache from
unpin_extent_range().An untrimmed extent can merge with everything as this is a new region. Absorbing trimmed extents is a tradeoff to for greater coalescing which makes life better for
find_free_extent(). Additionally, it seems the size of a trim isn't as problematic as the trim io itself.When reading in the free space cache from disk, if sync is set, mark all extents as trimmed. The current code ensures at transaction commit that all free space is trimmed when sync is set, so this reflects that.
Signed-off-by Dennis Zhou <[email protected]>--- fs/btrfs/extent-tree.c | 15 +++++++--- fs/btrfs/free-space-cache.c | 60 ++++++++++++++++++++++++++++++++----- fs/btrfs/free-space-cache.h | 17 ++++++++++- fs/btrfs/inode-map.c | 13 ++++---- 4 files changed, 87 insertions(+), 18 deletions(-)fs/btrfs/extent-tree.c +11
-4@@ -2783,6 +2783,7 @@ fetch_cluster_info(struct btrfs_fs_info *fs_info,static int unpin_extent_range (struct btrfs_fs_info *fs_info,u64 start,u64 end, +enum btrfs_trim_state trim_state,const bool return_free_space) {struct btrfs_block_group_cache *cache =NULL ;@@ -2816,7 +2817,9 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info,if (start < cache->last_byte_to_unpin) { len =min (len, cache->last_byte_to_unpin - start);if (return_free_space) -btrfs_add_free_space (cache, start, len); +__btrfs_add_free_space (fs_info, + cache->free_space_ctl, + start, len, trim_state); } start += len;@@ -2894,6 +2897,7 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)while (!trans->aborted) {struct extent_state *cached_state =NULL ; +enum btrfs_trim_state trim_state =BTRFS_TRIM_STATE_UNTRIMMED ;mutex_lock (&fs_info->unused_bg_unpin_mutex); ret = find_first_extent_bit(unpin, 0, &start, &end,@@ -2903,12 +2907,14 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)break ; } -if (btrfs_test_opt (fs_info,DISCARD_SYNC )) +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) { ret =btrfs_discard_extent (fs_info, start, end +1 - start,NULL ); + trim_state =BTRFS_TRIM_STATE_TRIMMED ; + }clear_extent_dirty (unpin, start, end, &cached_state); -unpin_extent_range (fs_info, start, end, true); +unpin_extent_range (fs_info, start, end, trim_state, true);mutex_unlock (&fs_info->unused_bg_unpin_mutex);free_extent_state (cached_state);cond_resched ();@@ -5512,7 +5518,8 @@ u64 btrfs_account_ro_block_groups_free_space(struct btrfs_space_info *sinfo)int btrfs_error_unpin_extent_range (struct btrfs_fs_info *fs_info,u64 start,u64 end) { -return unpin_extent_range (fs_info, start, end, false); +return unpin_extent_range (fs_info, start, end, +BTRFS_TRIM_STATE_UNTRIMMED , false); }/* fs/btrfs/free-space-cache.c +0
-0@@ -747,6 +747,14 @@ static int __load_free_space_cache(struct btrfs_root *root, struct inode *inode, goto free_cache; }
Quoted 17 lines
fs/btrfs/free-space-cache.c static int __load_free_space_cache(struct btrfs_root *root, struct inode *inode, · line 757
Quoted 8 lines
fs/btrfs/free-space-cache.c +8
-0+/* +* Sync discard ensures that the free space cache is always +* trimmed. So when reading this in, the state should reflect +* that. +*/ +if (btrfs_test_opt (fs_info,DISCARD_SYNC )) + e->trim_state =BTRFS_TRIM_STATE_TRIMMED ; +
BTRFS_TRIM_STATE_TRIMMED== 0, so if we don't haveDISCARD_SYNCset, we'll still end up with TRIMMED. Which means we'll end up with weirdness later because unpinned extents will come back as UNTRIMMED in the !DISCARD_SYNC case. I'm thinking about spinning rust here where we'll never have discard, so we should either always treat the free space as trimmed, or always treat it as untrimmed, so we don't end up with different allocator behavior for the undiscardable devices. Thanks,
Ah, that is a good catch. I think having everything be treated as untrimmed should be fine. I moved BTRFS_TRIM_STATE_UNTRIMMED to be 0 in the enum.
Thanks,
Dennis
Dennis Zhou
maintainerPATCH v2 4/22 btrfs: keep track of cleanliness of the bitmap
In reply to Dennis ZhouThere is a cap in btrfs in the amount of free extents that a block group can have. When it surpasses that threshold, future extents are placed into bitmaps. Instead of keeping track of if a certain bit is trimmed or not in a second bitmap, keep track of the relative state of the bitmap.
With async discard, trimming bitmaps becomes a more frequent operation. As a trade off with simplicity, we keep track of if discarding a bitmap is in progress. If we fully scan a bitmap and trim as necessary, the bitmap is marked clean. This has some caveats as the min block size may skip over regions deemed too small. But this should be a reasonable trade off rather than keeping a second bitmap and making allocation paths more complex. The downside is we may overtrim, but ideally the min block size should prevent us from doing that too often and getting stuck trimming pathological cases.
BTRFS_TRIM_STATE_TRIMMING is added to indicate a bitmap is in the process of being trimmed. If additional free space is added to that bitmap, the bit is cleared. A bitmap will be marked BTRFS_TRIM_STATE_TRIMMED if the trimming code was able to reach the end of it and the former is still set.
--- fs/btrfs/free-space-cache.c | 89 +++++++++++++++++++++++++++++++++---- fs/btrfs/free-space-cache.h | 12 +++++ 2 files changed, 92 insertions(+), 9 deletions(-)
fs/btrfs/free-space-cache.c +80 -9
@@ -1975,11 +1975,18 @@ static noinline int remove_from_bitmap(struct btrfs_free_space_ctl *ctl,static u64 add_bytes_to_bitmap (struct btrfs_free_space_ctl *ctl,struct btrfs_free_space *info,u64 offset, -u64 bytes) +u64 bytes,enum btrfs_trim_state trim_state) {u64 bytes_to_set =0 ;u64 end; +/* +* This is a tradeoff to make bitmap trim state minimal. We mark the +* whole bitmap untrimmed if at any point we add untrimmed regions. +*/ +if (trim_state ==BTRFS_TRIM_STATE_UNTRIMMED ) + info->trim_state =BTRFS_TRIM_STATE_UNTRIMMED ; + end = info->offset + (u64)(BITS_PER_BITMAP * ctl->unit); bytes_to_set =min (end - offset, bytes);
@@ -2054,10 +2061,12 @@ static int insert_into_bitmap(struct btrfs_free_space_ctl *ctl,struct btrfs_block_group_cache *block_group =NULL ;int added =0 ;u64 bytes, offset, bytes_added; +enum btrfs_trim_state trim_state;int ret; bytes = info->bytes; offset = info->offset; + trim_state = info->trim_state;if (!ctl->op->use_bitmap (ctl, info))return 0 ;
@@ -2092,8 +2101,8 @@ static int insert_into_bitmap(struct btrfs_free_space_ctl *ctl, }if (entry->offset ==offset_to_bitmap (ctl, offset)) { - bytes_added =add_bytes_to_bitmap (ctl, entry, - offset, bytes); + bytes_added =add_bytes_to_bitmap (ctl, entry, offset, + bytes, trim_state); bytes -= bytes_added; offset += bytes_added; }
@@ -2112,7 +2121,8 @@ static int insert_into_bitmap(struct btrfs_free_space_ctl *ctl, goto new_bitmap; } - bytes_added =add_bytes_to_bitmap (ctl, bitmap_info, offset, bytes); + bytes_added =add_bytes_to_bitmap (ctl, bitmap_info, offset, bytes, + trim_state); bytes -= bytes_added; offset += bytes_added; added =0 ;
@@ -2146,6 +2156,7 @@ static int insert_into_bitmap(struct btrfs_free_space_ctl *ctl,/* allocate the bitmap */ info->bitmap =kmem_cache_zalloc (btrfs_free_space_bitmap_cachep,GFP_NOFS ); + info->trim_state =BTRFS_TRIM_STATE_TRIMMED ;spin_lock (&ctl->tree_lock);if (!info->bitmap) { ret = -ENOMEM ;
@@ -3317,6 +3328,39 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group,return ret; } +/* +* If we break out of trimming a bitmap prematurely, we should reset the +* trimming bit. In a rather contrieved case, it's possible to race here so +* reset the state to BTRFS_TRIM_STATE_UNTRIMMED. +* +* start = start of bitmap +* end = near end of bitmap +* +* Thread 1: Thread 2: +* trim_bitmaps(start) +* trim_bitmaps(end) +* end_trimming_bitmap() +* reset_trimming_bitmap() +*/ +static void reset_trimming_bitmap (struct btrfs_free_space_ctl *ctl,u64 offset) +{ +struct btrfs_free_space *entry; + +spin_lock (&ctl->tree_lock); + + entry =tree_search_offset (ctl, offset,1 ,0 ); +if (entry) + entry->trim_state =BTRFS_TRIM_STATE_UNTRIMMED ; + +spin_unlock (&ctl->tree_lock); +} + +static void end_trimming_bitmap (struct btrfs_free_space *entry) +{ +if (btrfs_free_space_trimming_bitmap (entry)) + entry->trim_state =BTRFS_TRIM_STATE_TRIMMED ; +} +static int trim_bitmaps (struct btrfs_block_group_cache *block_group,u64 *total_trimmed,u64 start,u64 end,u64 minlen) {
@@ -3341,16 +3385,33 @@ static int trim_bitmaps(struct btrfs_block_group_cache *block_group, } entry =tree_search_offset (ctl, offset,1 ,0 ); -if (!entry) { +if (!entry ||btrfs_free_space_trimmed (entry)) {spin_unlock (&ctl->tree_lock);mutex_unlock (&ctl->cache_writeout_mutex); next_bitmap = true; goto next; } +/* +* Async discard bitmap trimming begins at by setting the start +* to be key.objectid and the offset_to_bitmap() aligns to the +* start of the bitmap. This lets us know we are fully +* scanning the bitmap rather than only some portion of it. +*/ +if (start == offset) + entry->trim_state =BTRFS_TRIM_STATE_TRIMMING ; + bytes = minlen; ret2 =search_bitmap (ctl, entry, &start, &bytes, false);if (ret2 || start >= end) { +/* +* This keeps the invariant that all bytes are trimmed +* if BTRFS_TRIM_STATE_TRIMMED is set on a bitmap. +*/ +if (ret2 && !minlen) +end_trimming_bitmap (entry); +else + entry->trim_state =BTRFS_TRIM_STATE_UNTRIMMED ;spin_unlock (&ctl->tree_lock);mutex_unlock (&ctl->cache_writeout_mutex); next_bitmap = true;
@@ -3359,6 +3420,7 @@ static int trim_bitmaps(struct btrfs_block_group_cache *block_group, bytes =min (bytes, end - start);if (bytes < minlen) { + entry->trim_state =BTRFS_TRIM_STATE_UNTRIMMED ;spin_unlock (&ctl->tree_lock);mutex_unlock (&ctl->cache_writeout_mutex); goto next;
@@ -3376,18 +3438,21 @@ static int trim_bitmaps(struct btrfs_block_group_cache *block_group, ret =do_trimming (block_group, total_trimmed, start, bytes, start, bytes, &trim_entry); -if (ret) +if (ret) { +reset_trimming_bitmap (ctl, offset);break ; + } next:if (next_bitmap) { offset +=BITS_PER_BITMAP * ctl->unit; + start = offset; }else { start += bytes; -if (start >= offset +BITS_PER_BITMAP * ctl->unit) - offset +=BITS_PER_BITMAP * ctl->unit; }if (fatal_signal_pending (current)) { +if (start != offset) +reset_trimming_bitmap (ctl, offset); ret = -ERESTARTSYS ;break ; }
@@ -3441,6 +3506,7 @@ void btrfs_put_block_group_trimming(struct btrfs_block_group_cache *block_group)int btrfs_trim_block_group (struct btrfs_block_group_cache *block_group,u64 *trimmed,u64 start,u64 end,u64 minlen) { +struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl;int ret; *trimmed =0 ;
@@ -3458,6 +3524,9 @@ int btrfs_trim_block_group(struct btrfs_block_group_cache *block_group, goto out; ret =trim_bitmaps (block_group, trimmed, start, end, minlen); +/* If we ended in the middle of a bitmap, reset the trimming flag. */ +if (end % (BITS_PER_BITMAP * ctl->unit)) +reset_trimming_bitmap (ctl,offset_to_bitmap (ctl, end)); out:btrfs_put_block_group_trimming (block_group);return ret;
@@ -3642,6 +3711,7 @@ int test_add_free_space_entry(struct btrfs_block_group_cache *cache,struct btrfs_free_space_ctl *ctl = cache->free_space_ctl;struct btrfs_free_space *info =NULL , *bitmap_info;void *map =NULL ; +enum btrfs_trim_state trim_state =BTRFS_TRIM_STATE_TRIMMED ;u64 bytes_added;int ret;
@@ -3683,7 +3753,8 @@ int test_add_free_space_entry(struct btrfs_block_group_cache *cache, info =NULL ; } - bytes_added =add_bytes_to_bitmap (ctl, bitmap_info, offset, bytes); + bytes_added =add_bytes_to_bitmap (ctl, bitmap_info, offset, bytes, + trim_state); bytes -= bytes_added; offset += bytes_added;
fs/btrfs/free-space-cache.h +12 -0
@@ -8,10 +8,16 @@/* * This is the trim state of an extent or bitmap. +* +* BTRFS_TRIM_STATE_TRIMMING is special and used to maintain the state of a +* bitmap as we may need several trims to fully trim a single bitmap entry. +* This is reset should any free space other than trimmed space is added to the +* bitmap. */ enum btrfs_trim_state {BTRFS_TRIM_STATE_TRIMMED ,BTRFS_TRIM_STATE_UNTRIMMED , +BTRFS_TRIM_STATE_TRIMMING , };struct btrfs_free_space {
@@ -29,6 +35,12 @@ static inline bool btrfs_free_space_trimmed(struct btrfs_free_space *info)return (info->trim_state ==BTRFS_TRIM_STATE_TRIMMED ); } +static inline bool btrfs_free_space_trimming_bitmap ( +struct btrfs_free_space *info) +{ +return (info->trim_state ==BTRFS_TRIM_STATE_TRIMMING ); +} +struct btrfs_free_space_ctl {spinlock_t tree_lock;struct rb_root free_space_offset;
Signature · 1 line
2.17.1
Re: [PATCH 04/22] btrfs: keep track of cleanliness of the bitmap
Quoted 15 lines · Dennis Zhou
There is a cap in btrfs in the amount of free extents that a block group can have. When it surpasses that threshold, future extents are placed into bitmaps. Instead of keeping track of if a certain bit is trimmed or not in a second bitmap, keep track of the relative state of the bitmap.
With async discard, trimming bitmaps becomes a more frequent operation. As a trade off with simplicity, we keep track of if discarding a bitmap is in progress. If we fully scan a bitmap and trim as necessary, the bitmap is marked clean. This has some caveats as the min block size may skip over regions deemed too small. But this should be a reasonable trade off rather than keeping a second bitmap and making allocation paths more complex. The downside is we may overtrim, but ideally the min block size should prevent us from doing that too often and getting stuck trimming pathological cases.
Quoted 8 lines
BTRFS_TRIM_STATE_TRIMMINGis added to indicate a bitmap is in the process of being trimmed. If additional free space is added to that bitmap, the bit is cleared. A bitmap will be markedBTRFS_TRIM_STATE_TRIMMEDif the trimming code was able to reach the end of it and the former is still set.Signed-off-by Dennis Zhou <[email protected]>---
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 5/22 btrfs: add the beginning of async discard, discard workqueue
In reply to Dennis ZhouWhen discard is enabled, everytime a pinned extent is released back to the block_group's free space cache, a discard is issued for the extent. This is an overeager approach when it comes to discarding and helping the SSD maintain enough free space to prevent severe garbage collection situations.
This adds the beginning of async discard. Instead of issuing a discard prior to returning it to the free space, it is just marked as untrimmed. The block_group is then added to a LRU which then feeds into a workqueue to issue discards at a much slower rate. Full discarding of unused block groups is still done and will be address in a future patch in this series.
For now, we don't persist the discard state of extents and bitmaps. Therefore, our failure recovery mode will be to consider extents untrimmed. This lets us handle failure and unmounting as one in the same.
--- fs/btrfs/Makefile | 2 +- fs/btrfs/block-group.c | 4 + fs/btrfs/block-group.h | 9 ++ fs/btrfs/ctree.h | 21 +++ fs/btrfs/discard.c | 274 ++++++++++++++++++++++++++++++++++++ fs/btrfs/discard.h | 28 ++++ fs/btrfs/disk-io.c | 15 +- fs/btrfs/extent-tree.c | 4 + fs/btrfs/free-space-cache.c | 35 ++++- fs/btrfs/super.c | 35 ++++- 10 files changed, 417 insertions(+), 10 deletions(-) create mode 100644 fs/btrfs/discard.c create mode 100644 fs/btrfs/discard.h
fs/btrfs/Makefile +1 -1
@@ -11,7 +11,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \ compression.o delayed-ref.o relocation.o delayed-inode.o scrub.o \ reada.o backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \ uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \ - block-rsv.o delalloc-space.o block-group.o + block-rsv.o delalloc-space.o block-group.o discard.o btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
fs/btrfs/block-group.c +4 -0
@@ -14,6 +14,7 @@#include "sysfs.h" #include "tree-log.h" #include "delalloc-space.h" +#include "discard.h" /* * Return target flags in extended format or 0 if restripe for this chunk_type
@@ -1273,6 +1274,8 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info) }spin_unlock (&fs_info->unused_bgs_lock); +btrfs_discard_cancel_work (&fs_info->discard_ctl, block_group); +mutex_lock (&fs_info->delete_unused_bgs_mutex);/* Don't want to race with allocators so take the groups_sem */
@@ -1622,6 +1625,7 @@ static struct btrfs_block_group_cache *btrfs_create_block_group_cache(INIT_LIST_HEAD (&cache->cluster_list);INIT_LIST_HEAD (&cache->bg_list);INIT_LIST_HEAD (&cache->ro_list); +INIT_LIST_HEAD (&cache->discard_list);INIT_LIST_HEAD (&cache->dirty_list);INIT_LIST_HEAD (&cache->io_list);btrfs_init_free_space_ctl (cache);
fs/btrfs/block-group.h +9 -0
@@ -115,7 +115,11 @@ struct btrfs_block_group_cache {/* For read-only block groups */ struct list_head ro_list; +/* For discard operations */ atomic_t trimming; +struct list_head discard_list; +int discard_index; +u64 discard_eligible_time;/* For dirty block groups */ struct list_head dirty_list;
@@ -157,6 +161,11 @@ struct btrfs_block_group_cache {struct btrfs_full_stripe_locks_tree full_stripe_locks_root; }; +static inline u64 btrfs_block_group_end (struct btrfs_block_group_cache *cache) +{ +return (cache->key.objectid + cache->key.offset); +} +#ifdef CONFIG_BTRFS_DEBUG static inline int btrfs_should_fragment_free_space (struct btrfs_block_group_cache *block_group)
fs/btrfs/ctree.h +21 -0
@@ -438,6 +438,21 @@ struct btrfs_full_stripe_locks_tree {struct mutex lock; }; +/* Discard control. */ +/* +* Async discard uses multiple lists to differentiate the discard filter +* parameters. +*/ +#define BTRFS_NR_DISCARD_LISTS 1 + +struct btrfs_discard_ctl { +struct workqueue_struct *discard_workers; +struct delayed_work work; +spinlock_t lock; +struct btrfs_block_group_cache *cache; +struct list_head discard_list[BTRFS_NR_DISCARD_LISTS ]; +}; +/* delayed seq elem */ struct seq_list {struct list_head list;
@@ -524,6 +539,9 @@ enum {* so we don't need to offload checksums to workqueues. */ BTRFS_FS_CSUM_IMPL_FAST , + +/* Indicate that the discard workqueue can service discards. */ + BTRFS_FS_DISCARD_RUNNING, }; struct btrfs_fs_info {
@@ -817,6 +835,8 @@ struct btrfs_fs_info {struct btrfs_workqueue *scrub_wr_completion_workers;struct btrfs_workqueue *scrub_parity_workers; +struct btrfs_discard_ctl discard_ctl; +#ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY u32 check_integrity_print_mask;#endif
@@ -1190,6 +1210,7 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info)#define BTRFS_MOUNT_FREE_SPACE_TREE (1 << 26)#define BTRFS_MOUNT_NOLOGREPLAY (1 << 27)#define BTRFS_MOUNT_REF_VERIFY (1 << 28) +#define BTRFS_MOUNT_DISCARD_ASYNC (1 << 29)#define BTRFS_DEFAULT_COMMIT_INTERVAL (30)#define BTRFS_DEFAULT_MAX_INLINE (2048)
fs/btrfs/discard.c new file +274 -0
274 lines not on this page · raw message
fs/btrfs/discard.h new file +28 -0
@@ -0,0 +1,28 @@ +/* +* Copyright (C) 2019 Facebook. All rights reserved. +*/ + +#ifndef BTRFS_DISCARD_H +#define BTRFS_DISCARD_H + +struct btrfs_fs_info ; +struct btrfs_discard_ctl ; +struct btrfs_block_group_cache ; + +void btrfs_add_to_discard_list (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache); + +void btrfs_discard_cancel_work (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache); +void btrfs_discard_queue_work (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache); +void btrfs_discard_schedule_work (struct btrfs_discard_ctl *discard_ctl, +bool override); +bool btrfs_run_discard_work (struct btrfs_discard_ctl *discard_ctl); + +void btrfs_discard_resume (struct btrfs_fs_info *fs_info); +void btrfs_discard_stop (struct btrfs_fs_info *fs_info); +void btrfs_discard_init (struct btrfs_fs_info *fs_info); +void btrfs_discard_cleanup (struct btrfs_fs_info *fs_info); + +#endif
fs/btrfs/disk-io.c +14 -1
@@ -41,6 +41,7 @@#include "tree-checker.h" #include "ref-verify.h" #include "block-group.h" +#include "discard.h" #define BTRFS_SUPER_FLAG_SUPP (BTRFS_HEADER_FLAG_WRITTEN |\ BTRFS_HEADER_FLAG_RELOC |\
@@ -2009,6 +2010,8 @@ static void btrfs_stop_all_workers(struct btrfs_fs_info *fs_info)btrfs_destroy_workqueue (fs_info->flush_workers);btrfs_destroy_workqueue (fs_info->qgroup_rescan_workers);btrfs_destroy_workqueue (fs_info->extent_workers); +if (fs_info->discard_ctl.discard_workers) +destroy_workqueue (fs_info->discard_ctl.discard_workers);/* * Now that all other work queues are destroyed, we can safely destroy * the queues used for metadata I/O, since tasks from those other work
@@ -2218,6 +2221,8 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info,btrfs_alloc_workqueue (fs_info,"extent-refs" , flags,min_t (u64, fs_devices->num_devices, max_active),8 ); + fs_info->discard_ctl.discard_workers = +alloc_workqueue ("btrfs_discard" ,WQ_UNBOUND |WQ_FREEZABLE ,1 ); if (!(fs_info->workers && fs_info->delalloc_workers && fs_info->submit_workers && fs_info->flush_workers &&
@@ -2229,7 +2234,8 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info, fs_info->caching_workers && fs_info->readahead_workers && fs_info->fixup_workers && fs_info->delayed_workers && fs_info->extent_workers && - fs_info->qgroup_rescan_workers)) { + fs_info->qgroup_rescan_workers && + fs_info->discard_ctl.discard_workers)) { return -ENOMEM; }
@@ -2772,6 +2778,8 @@ int open_ctree(struct super_block *sb,btrfs_init_dev_replace_locks (fs_info);btrfs_init_qgroup (fs_info); +btrfs_discard_init (fs_info); +btrfs_init_free_cluster (&fs_info->meta_alloc_cluster);btrfs_init_free_cluster (&fs_info->data_alloc_cluster);
@@ -3284,6 +3292,8 @@ int open_ctree(struct super_block *sb,btrfs_qgroup_rescan_resume (fs_info); +btrfs_discard_resume (fs_info); +if (!fs_info->uuid_root) {btrfs_info (fs_info,"creating UUID tree" ); ret =btrfs_create_uuid_tree (fs_info);
@@ -3993,6 +4003,9 @@ void close_ctree(struct btrfs_fs_info *fs_info)*/ kthread_park (fs_info->cleaner_kthread); +/* cancel or finish ongoing work */ +btrfs_discard_cleanup (fs_info); +/* wait for the qgroup rescan worker to stop */ btrfs_qgroup_wait_for_completion (fs_info, false);
fs/btrfs/extent-tree.c +4 -0
@@ -32,6 +32,7 @@#include "block-rsv.h" #include "delalloc-space.h" #include "block-group.h" +#include "discard.h" #undef SCRAMBLE_DELAYED_REFS
@@ -2920,6 +2921,9 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)cond_resched (); } +if (btrfs_test_opt (fs_info,DISCARD_ASYNC )) +btrfs_discard_schedule_work (&fs_info->discard_ctl, true); +/* * Transaction is finished. We don't need the lock anymore. We * do need to clean up the block groups in case of a transaction
fs/btrfs/free-space-cache.c +29 -6
@@ -21,6 +21,7 @@#include "space-info.h" #include "delalloc-space.h" #include "block-group.h" +#include "discard.h" #define BITS_PER_BITMAP (PAGE_SIZE * 8UL)#define MAX_CACHE_BYTES_PER_GIG SZ_32K
@@ -750,9 +751,11 @@ static int __load_free_space_cache(struct btrfs_root *root, struct inode *inode,/* * Sync discard ensures that the free space cache is always * trimmed. So when reading this in, the state should reflect -* that. +* that. We also do this for async as a stop gap for lack of +* persistence. */ -if (btrfs_test_opt (fs_info,DISCARD_SYNC )) +if (btrfs_test_opt (fs_info,DISCARD_SYNC ) || +btrfs_test_opt (fs_info,DISCARD_ASYNC )) e->trim_state =BTRFS_TRIM_STATE_TRIMMED ;if (!e->bytes) {
@@ -2379,6 +2382,7 @@ int __btrfs_add_free_space(struct btrfs_fs_info *fs_info, u64 offset, u64 bytes, enum btrfs_trim_state trim_state) { + struct btrfs_block_group_cache *cache = ctl->private;struct btrfs_free_space *info;int ret =0 ;
@@ -2428,6 +2432,9 @@ int __btrfs_add_free_space(struct btrfs_fs_info *fs_info,ASSERT (ret != -EEXIST ); } +if (trim_state !=BTRFS_TRIM_STATE_TRIMMED ) +btrfs_discard_queue_work (&fs_info->discard_ctl, cache); +return ret; }
@@ -3201,6 +3208,7 @@ void btrfs_init_free_cluster(struct btrfs_free_cluster *cluster)static int do_trimming (struct btrfs_block_group_cache *block_group,u64 *total_trimmed,u64 start,u64 bytes,u64 reserved_start,u64 reserved_bytes, +enum btrfs_trim_state reserved_trim_state,struct btrfs_trim_range *trim_entry) {struct btrfs_space_info *space_info = block_group->space_info;
@@ -3208,6 +3216,9 @@ static int do_trimming(struct btrfs_block_group_cache *block_group,struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl;int ret;int update =0 ; +u64 end = start + bytes; +u64 reserved_end = reserved_start + reserved_bytes; +enum btrfs_trim_state trim_state;u64 trimmed =0 ;spin_lock (&space_info->lock);
@@ -3221,11 +3232,20 @@ static int do_trimming(struct btrfs_block_group_cache *block_group,spin_unlock (&space_info->lock); ret =btrfs_discard_extent (fs_info, start, bytes, &trimmed); -if (!ret) +if (!ret) { *total_trimmed += trimmed; + trim_state =BTRFS_TRIM_STATE_TRIMMED ; + }mutex_lock (&ctl->cache_writeout_mutex); -btrfs_add_free_space (block_group, reserved_start, reserved_bytes); +if (reserved_start < start) +__btrfs_add_free_space (fs_info, ctl, reserved_start, + start - reserved_start, + reserved_trim_state); +if (start + bytes < reserved_start + reserved_bytes) +__btrfs_add_free_space (fs_info, ctl, end, reserved_end - end, + reserved_trim_state); +__btrfs_add_free_space (fs_info, ctl, start, bytes, trim_state);list_del (&trim_entry->list);mutex_unlock (&ctl->cache_writeout_mutex);
@@ -3252,6 +3272,7 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group,int ret =0 ;u64 extent_start;u64 extent_bytes; +enum btrfs_trim_state extent_trim_state;u64 bytes;while (start < end) {
@@ -3293,6 +3314,7 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group, extent_start = entry->offset; extent_bytes = entry->bytes; + extent_trim_state = entry->trim_state; start =max (start, extent_start); bytes =min (extent_start + extent_bytes, end) - start;if (bytes < minlen) {
@@ -3311,7 +3333,8 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group,mutex_unlock (&ctl->cache_writeout_mutex); ret =do_trimming (block_group, total_trimmed, start, bytes, - extent_start, extent_bytes, &trim_entry); + extent_start, extent_bytes, extent_trim_state, + &trim_entry);if (ret)break ; next:
@@ -3437,7 +3460,7 @@ static int trim_bitmaps(struct btrfs_block_group_cache *block_group,mutex_unlock (&ctl->cache_writeout_mutex); ret =do_trimming (block_group, total_trimmed, start, bytes, - start, bytes, &trim_entry); + start, bytes,0 , &trim_entry);if (ret) {reset_trimming_bitmap (ctl, offset);break ;
fs/btrfs/super.c +33 -2
@@ -46,6 +46,7 @@#include "sysfs.h" #include "tests/btrfs-tests.h" #include "block-group.h" +#include "discard.h" #include "qgroup.h" #define CREATE_TRACE_POINTS
@@ -146,6 +147,8 @@ void __btrfs_handle_fs_error(struct btrfs_fs_info *fs_info, const char *functionif (sb_rdonly (sb))return ; +btrfs_discard_stop (fs_info); +/* btrfs handle error by forcing the filesystem readonly */ sb->s_flags |=SB_RDONLY ;btrfs_info (fs_info,"forced readonly" );
@@ -313,6 +316,7 @@ enum { Opt_datasum, Opt_nodatasum, Opt_defrag, Opt_nodefrag, Opt_discard, Opt_nodiscard, + Opt_discard_mode, Opt_nologreplay, Opt_norecovery, Opt_ratio,
@@ -376,6 +380,7 @@ static const match_table_t tokens = { {Opt_nodefrag, "noautodefrag"}, {Opt_discard, "discard"}, {Opt_nodiscard, "nodiscard"}, + {Opt_discard_mode, "discard=%s"}, {Opt_nologreplay, "nologreplay"}, {Opt_norecovery, "norecovery"}, {Opt_ratio, "metadata_ratio=%u"},
@@ -695,12 +700,26 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options, info->metadata_ratio);break ;case Opt_discard: -btrfs_set_and_info (info,DISCARD_SYNC , -"turning on sync discard" ); +case Opt_discard_mode: +if (token == Opt_discard || +strcmp (args[0 ].from,"sync" ) ==0 ) { +btrfs_clear_opt (info->mount_opt,DISCARD_ASYNC ); +btrfs_set_and_info (info,DISCARD_SYNC , +"turning on sync discard" ); + }else if (strcmp (args[0 ].from,"async" ) ==0 ) { +btrfs_clear_opt (info->mount_opt,DISCARD_SYNC ); +btrfs_set_and_info (info,DISCARD_ASYNC , +"turning on async discard" ); + }else { + ret = -EINVAL ; + goto out; + }break ;case Opt_nodiscard:btrfs_clear_and_info (info,DISCARD_SYNC ,"turning off discard" ); +btrfs_clear_and_info (info,DISCARD_ASYNC , +"turning off async discard" );break ;case Opt_space_cache:case Opt_space_cache_version:
@@ -1324,6 +1343,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)seq_puts (seq,",flushoncommit" );if (btrfs_test_opt (info,DISCARD_SYNC ))seq_puts (seq,",discard" ); +if (btrfs_test_opt (info,DISCARD_ASYNC )) +seq_puts (seq,",discard=async" );if (!(info->sb->s_flags &SB_POSIXACL ))seq_puts (seq,",noacl" );if (btrfs_test_opt (info,SPACE_CACHE ))
@@ -1714,6 +1735,14 @@ static inline void btrfs_remount_cleanup(struct btrfs_fs_info *fs_info,btrfs_cleanup_defrag_inodes (fs_info); } +/* If we toggled discard async. */ +if (!btrfs_raw_test_opt (old_opts,DISCARD_ASYNC ) && +btrfs_test_opt (fs_info,DISCARD_ASYNC )) +btrfs_discard_resume (fs_info); +else if (btrfs_raw_test_opt (old_opts,DISCARD_ASYNC ) && + !btrfs_test_opt (fs_info,DISCARD_ASYNC )) +btrfs_discard_cleanup (fs_info); +clear_bit (BTRFS_FS_STATE_REMOUNTING , &fs_info->fs_state); }
@@ -1761,6 +1790,8 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)*/ cancel_work_sync (&fs_info->async_reclaim_work); +btrfs_discard_cleanup (fs_info); +/* wait for the uuid_scan task to finish */ down (&fs_info->uuid_tree_rescan_sem);/* avoid complains from lockdep et al. */
Signature · 1 line
2.17.1
Re: [PATCH 05/22] btrfs: add the beginning of async discard, discard workqueue
fs/btrfs/discard.c line 3
Quoted 6 lines · Dennis Zhou
fs/btrfs/discard.c new file +3
-0@@ -0,0 +1,274 @@ +/* +* Copyright (C) 2019 Facebook. All rights reserved. +*/
It's the other way around, SPDX should be there,
https://btrfs.wiki.kernel.org/index.php/Developer%27s_FAQ#Copyright_notices_in_files.2C_SPDX
Dennis Zhou
maintainerPATCH v2 6/22 btrfs: handle empty block_group removal
In reply to Dennis Zhoublock_group removal is a little tricky. It can race with the extent allocator, the cleaner thread, and balancing. The current path is for a block_group to be added to the unused_bgs list. Then, when the cleaner thread comes around, it starts a transaction and then proceeds with removing the block_group. Extents that are pinned are subsequently removed from the pinned trees and then eventually a discard is issued for the entire block_group.
Async discard introduces another player into the game, the discard workqueue. While it has none of the racing issues, the new problem is ensuring we don't leave free space untrimmed prior to forgetting the block_group. This is handled by placing fully free block_groups on a separate discard queue. This is necessary to maintain discarding order as in the future we will slowly trim even fully free block_groups. The ordering helps us make progress on the same block_group rather than say the last fully freed block_group or needing to search through the fully freed block groups at the beginning of a list and insert after.
The new order of events is a fully freed block group gets placed on the unused discard queue first. Once it's processed, it will be placed on the unusued_bgs list and then the original sequence of events will happen, just without the final whole block_group discard.
The mount flags can change when processing unused_bgs, so when flipping from DISCARD to DISCARD_ASYNC, the unused_bgs must be punted to the discard_list to be trimmed. If we flip off DISCARD_ASYNC, we punt free block groups on the discard_list to the unused_bg queue which will do the final discard for us.
--- fs/btrfs/block-group.c | 50 ++++++++++++++-- fs/btrfs/ctree.h | 9 ++- fs/btrfs/discard.c | 112 +++++++++++++++++++++++++++++++++++- fs/btrfs/discard.h | 6 ++ fs/btrfs/free-space-cache.c | 36 ++++++++++++ fs/btrfs/free-space-cache.h | 1 + fs/btrfs/scrub.c | 7 ++- 7 files changed, 211 insertions(+), 10 deletions(-)
fs/btrfs/block-group.c +46 -4
@@ -1251,6 +1251,7 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)struct btrfs_block_group_cache *block_group;struct btrfs_space_info *space_info;struct btrfs_trans_handle *trans; +bool async_trim_enabled =btrfs_test_opt (fs_info,DISCARD_ASYNC );int ret =0 ;if (!test_bit (BTRFS_FS_OPEN , &fs_info->flags))
@@ -1260,6 +1261,7 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)while (!list_empty (&fs_info->unused_bgs)) {u64 start, end;int trimming; +bool async_trimmed; block_group = list_first_entry(&fs_info->unused_bgs, struct btrfs_block_group_cache,
@@ -1281,10 +1283,24 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)/* Don't want to race with allocators so take the groups_sem */ down_write (&space_info->groups_sem);spin_lock (&block_group->lock); + +/* +* Async discard moves the final block group discard to be prior +* to the unused_bgs code path. Therefore, if it's not fully +* trimmed, punt it back to the async discard lists. +*/ + async_trimmed = (!btrfs_test_opt (fs_info,DISCARD_ASYNC ) || +btrfs_is_free_space_trimmed (block_group)); +if (block_group->reserved || block_group->pinned ||btrfs_block_group_used (&block_group->item) || block_group->ro || -list_is_singular (&block_group->list)) { +list_is_singular (&block_group->list) || + !async_trimmed) { +/* Requeue if we failed because of async discard. */ +if (!async_trimmed) +btrfs_discard_queue_work (&fs_info->discard_ctl, + block_group);/* * We want to bail if we made new allocations or have * outstanding allocations in this block group. We do
@@ -1367,6 +1383,17 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)spin_unlock (&block_group->lock);spin_unlock (&space_info->lock); +/* +* The normal path here is an unused block group is passed here, +* then trimming is handled in the transaction commit path. +* Async discard interposes before this to do the trimming +* before coming down the unused block group path as trimming +* will no longer be done later in the transaction commit path. +*/ +if (!async_trim_enabled && +btrfs_test_opt (fs_info,DISCARD_ASYNC )) + goto flip_async; +/* DISCARD can flip during remount */ trimming =btrfs_test_opt (fs_info,DISCARD_SYNC );
@@ -1411,6 +1438,13 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)spin_lock (&fs_info->unused_bgs_lock); }spin_unlock (&fs_info->unused_bgs_lock); +return ; + +flip_async: +btrfs_end_transaction (trans); +mutex_unlock (&fs_info->delete_unused_bgs_mutex); +btrfs_put_block_group (block_group); +btrfs_discard_punt_unused_bgs_list (fs_info); }void btrfs_mark_bg_unused (struct btrfs_block_group_cache *bg)
@@ -1618,6 +1652,8 @@ static struct btrfs_block_group_cache *btrfs_create_block_group_cache( cache->full_stripe_len =btrfs_full_stripe_len (fs_info, start);set_free_space_tree_thresholds (cache); + cache->discard_index =BTRFS_DISCARD_INDEX_UNUSED ; +atomic_set (&cache->count,1 );spin_lock_init (&cache->lock);init_rwsem (&cache->data_rwsem);
@@ -1829,7 +1865,11 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)inc_block_group_ro (cache,1 ); }else if (btrfs_block_group_used (&cache->item) ==0 ) {ASSERT (list_empty (&cache->bg_list)); -btrfs_mark_bg_unused (cache); +if (btrfs_test_opt (info,DISCARD_ASYNC )) +btrfs_add_to_discard_unused_list ( + &info->discard_ctl, cache); +else +btrfs_mark_bg_unused (cache); } }
@@ -2724,8 +2764,10 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,* dirty list to avoid races between cleaner kthread and space * cache writeout. */ -if (!alloc && old_val ==0 ) -btrfs_mark_bg_unused (cache); +if (!alloc && old_val ==0 ) { +if (!btrfs_test_opt (info,DISCARD_ASYNC )) +btrfs_mark_bg_unused (cache); + }btrfs_put_block_group (cache); total -= num_bytes;
fs/btrfs/ctree.h +7 -2
@@ -441,9 +441,14 @@ struct btrfs_full_stripe_locks_tree {/* Discard control. */ /* * Async discard uses multiple lists to differentiate the discard filter -* parameters. +* parameters. Index 0 is for completely free block groups where we need to +* ensure the entire block group is trimmed without being lossy. Indices +* afterwards represent monotonically decreasing discard filter sizes to +* prioritize what should be discarded next. */ -#define BTRFS_NR_DISCARD_LISTS 1 +#define BTRFS_NR_DISCARD_LISTS 2 +#define BTRFS_DISCARD_INDEX_UNUSED 0 +#define BTRFS_DISCARD_INDEX_START 1struct btrfs_discard_ctl {struct workqueue_struct *discard_workers;
fs/btrfs/discard.c +109 -3
160 lines not on this page · raw message
fs/btrfs/discard.h +6 -0
@@ -9,9 +9,13 @@ struct btrfs_fs_info;struct btrfs_discard_ctl ;struct btrfs_block_group_cache ; +/* List operations. */ void btrfs_add_to_discard_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache); +void btrfs_add_to_discard_unused_list (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache); +/* Work operations. */ void btrfs_discard_cancel_work (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache); void btrfs_discard_queue_work(struct btrfs_discard_ctl *discard_ctl,
@@ -20,6 +24,8 @@ void btrfs_discard_schedule_work(struct btrfs_discard_ctl *discard_ctl, bool override);bool btrfs_run_discard_work (struct btrfs_discard_ctl *discard_ctl); +/* Setup/Cleanup operations. */ +void btrfs_discard_punt_unused_bgs_list (struct btrfs_fs_info *fs_info);void btrfs_discard_resume (struct btrfs_fs_info *fs_info);void btrfs_discard_stop (struct btrfs_fs_info *fs_info);void btrfs_discard_init (struct btrfs_fs_info *fs_info);
fs/btrfs/free-space-cache.c +36 -0
@@ -2681,6 +2681,37 @@ void btrfs_remove_free_space_cache(struct btrfs_block_group_cache *block_group) } +/** +* btrfs_is_free_space_trimmed - see if everything is trimmed +* @cache: block_group of interest +* +* Walk the @cache's free space rb_tree to determine if everything is trimmed. +*/ +bool btrfs_is_free_space_trimmed (struct btrfs_block_group_cache *cache) +{ +struct btrfs_free_space_ctl *ctl = cache->free_space_ctl; +struct btrfs_free_space *info; +struct rb_node *node; +bool ret = true; + +spin_lock (&ctl->tree_lock); + node =rb_first (&ctl->free_space_offset); + +while (node) { + info = rb_entry(node, struct btrfs_free_space, offset_index); + +if (!btrfs_free_space_trimmed (info)) { + ret = false; +break ; + } + + node =rb_next (node); + } + +spin_unlock (&ctl->tree_lock); +return ret; +} +u64 btrfs_find_space_for_alloc (struct btrfs_block_group_cache *block_group,u64 offset,u64 bytes,u64 empty_size,u64 *max_extent_size)
@@ -2767,6 +2798,9 @@ int btrfs_return_cluster_to_free_space( ret =__btrfs_return_cluster_to_free_space (block_group, cluster);spin_unlock (&ctl->tree_lock); +btrfs_discard_queue_work (&block_group->fs_info->discard_ctl, + block_group); +/* finally drop our ref */ btrfs_put_block_group (block_group);return ret;
@@ -3125,6 +3159,7 @@ int btrfs_find_space_cluster(struct btrfs_block_group_cache *block_group,u64 min_bytes;u64 cont1_bytes;int ret; +bool found_cluster = false;/* * Choose the minimum extent size we'll require for this
@@ -3177,6 +3212,7 @@ int btrfs_find_space_cluster(struct btrfs_block_group_cache *block_group,list_del_init (&entry->list);if (!ret) { + found_cluster = true;atomic_inc (&block_group->count);list_add_tail (&cluster->block_group_list, &block_group->cluster_list);
fs/btrfs/free-space-cache.h +1 -0
@@ -119,6 +119,7 @@ int btrfs_remove_free_space(struct btrfs_block_group_cache *block_group,void __btrfs_remove_free_space_cache (struct btrfs_free_space_ctl *ctl);void btrfs_remove_free_space_cache (struct btrfs_block_group_cache *block_group); +bool btrfs_is_free_space_trimmed (struct btrfs_block_group_cache *cache);u64 btrfs_find_space_for_alloc (struct btrfs_block_group_cache *block_group,u64 offset,u64 bytes,u64 empty_size,u64 *max_extent_size);
fs/btrfs/scrub.c +6 -1
@@ -8,6 +8,7 @@#include <linux/sched/mm.h> #include <crypto/hash.h> #include "ctree.h" +#include "discard.h" #include "volumes.h" #include "disk-io.h" #include "ordered-data.h"
@@ -3683,7 +3684,11 @@ int scrub_enumerate_chunks(struct scrub_ctx *sctx,if (!cache->removed && !cache->ro && cache->reserved ==0 &&btrfs_block_group_used (&cache->item) ==0 ) {spin_unlock (&cache->lock); -btrfs_mark_bg_unused (cache); +if (btrfs_test_opt (fs_info,DISCARD_ASYNC )) +btrfs_add_to_discard_unused_list ( + &fs_info->discard_ctl, cache); +else +btrfs_mark_bg_unused (cache); }else {spin_unlock (&cache->lock); }
Signature · 1 line
2.17.1
Re: [PATCH 06/22] btrfs: handle empty block_group removal
Quoted 402 lines · Dennis Zhou
block_groupremoval is a little tricky. It can race with the extent allocator, the cleaner thread, and balancing. The current path is for ablock_groupto be added to theunused_bgslist. Then, when the cleaner thread comes around, it starts a transaction and then proceeds with removing theblock_group. Extents that are pinned are subsequently removed from the pinned trees and then eventually a discard is issued for the entireblock_group.Async discard introduces another player into the game, the discard workqueue. While it has none of the racing issues, the new problem is ensuring we don't leave free space untrimmed prior to forgetting the
block_group. This is handled by placing fully freeblock_groupson a separate discard queue. This is necessary to maintain discarding order as in the future we will slowly trim even fully freeblock_groups. The ordering helps us make progress on the sameblock_grouprather than say the last fully freedblock_groupor needing to search through the fully freed block groups at the beginning of a list and insert after.The new order of events is a fully freed block group gets placed on the unused discard queue first. Once it's processed, it will be placed on the
unusued_bgslist and then the original sequence of events will happen, just without the final wholeblock_groupdiscard.The mount flags can change when processing
unused_bgs, so when flipping from DISCARD toDISCARD_ASYNC, theunused_bgsmust be punted to thediscard_listto be trimmed. If we flip offDISCARD_ASYNC, we punt free block groups on thediscard_listto theunused_bgqueue which will do the final discard for us.Signed-off-by Dennis Zhou <[email protected]>--- fs/btrfs/block-group.c | 50 ++++++++++++++-- fs/btrfs/ctree.h | 9 ++- fs/btrfs/discard.c | 112 +++++++++++++++++++++++++++++++++++- fs/btrfs/discard.h | 6 ++ fs/btrfs/free-space-cache.c | 36 ++++++++++++ fs/btrfs/free-space-cache.h | 1 + fs/btrfs/scrub.c | 7 ++- 7 files changed, 211 insertions(+), 10 deletions(-)fs/btrfs/block-group.c +46
-4@@ -1251,6 +1251,7 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)struct btrfs_block_group_cache *block_group;struct btrfs_space_info *space_info;struct btrfs_trans_handle *trans; +bool async_trim_enabled =btrfs_test_opt (fs_info,DISCARD_ASYNC );int ret =0 ;if (!test_bit (BTRFS_FS_OPEN , &fs_info->flags))@@ -1260,6 +1261,7 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)while (!list_empty (&fs_info->unused_bgs)) {u64 start, end;int trimming; +bool async_trimmed; block_group = list_first_entry(&fs_info->unused_bgs, struct btrfs_block_group_cache,@@ -1281,10 +1283,24 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)/* Don't want to race with allocators so take the groups_sem */ down_write (&space_info->groups_sem);spin_lock (&block_group->lock); + +/* +* Async discard moves the final block group discard to be prior +* to the unused_bgs code path. Therefore, if it's not fully +* trimmed, punt it back to the async discard lists. +*/ + async_trimmed = (!btrfs_test_opt (fs_info,DISCARD_ASYNC ) || +btrfs_is_free_space_trimmed (block_group)); +if (block_group->reserved || block_group->pinned ||btrfs_block_group_used (&block_group->item) || block_group->ro || -list_is_singular (&block_group->list)) { +list_is_singular (&block_group->list) || + !async_trimmed) { +/* Requeue if we failed because of async discard. */ +if (!async_trimmed) +btrfs_discard_queue_work (&fs_info->discard_ctl, + block_group);/* * We want to bail if we made new allocations or have * outstanding allocations in this block group. We do @@ -1367,6 +1383,17 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)spin_unlock (&block_group->lock);spin_unlock (&space_info->lock); +/* +* The normal path here is an unused block group is passed here, +* then trimming is handled in the transaction commit path. +* Async discard interposes before this to do the trimming +* before coming down the unused block group path as trimming +* will no longer be done later in the transaction commit path. +*/ +if (!async_trim_enabled && +btrfs_test_opt (fs_info,DISCARD_ASYNC )) + goto flip_async; +/* DISCARD can flip during remount */ trimming =btrfs_test_opt (fs_info,DISCARD_SYNC );@@ -1411,6 +1438,13 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)spin_lock (&fs_info->unused_bgs_lock); }spin_unlock (&fs_info->unused_bgs_lock); +return ; + +flip_async: +btrfs_end_transaction (trans); +mutex_unlock (&fs_info->delete_unused_bgs_mutex); +btrfs_put_block_group (block_group); +btrfs_discard_punt_unused_bgs_list (fs_info); }void btrfs_mark_bg_unused (struct btrfs_block_group_cache *bg)@@ -1618,6 +1652,8 @@ static struct btrfs_block_group_cache *btrfs_create_block_group_cache( cache->full_stripe_len =btrfs_full_stripe_len (fs_info, start);set_free_space_tree_thresholds (cache); + cache->discard_index =BTRFS_DISCARD_INDEX_UNUSED ; +atomic_set (&cache->count,1 );spin_lock_init (&cache->lock);init_rwsem (&cache->data_rwsem);@@ -1829,7 +1865,11 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)inc_block_group_ro (cache,1 ); }else if (btrfs_block_group_used (&cache->item) ==0 ) {ASSERT (list_empty (&cache->bg_list)); -btrfs_mark_bg_unused (cache); +if (btrfs_test_opt (info,DISCARD_ASYNC )) +btrfs_add_to_discard_unused_list ( + &info->discard_ctl, cache); +else +btrfs_mark_bg_unused (cache); } }@@ -2724,8 +2764,10 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,* dirty list to avoid races between cleaner kthread and space * cache writeout. */ -if (!alloc && old_val ==0 ) -btrfs_mark_bg_unused (cache); +if (!alloc && old_val ==0 ) { +if (!btrfs_test_opt (info,DISCARD_ASYNC )) +btrfs_mark_bg_unused (cache); + }btrfs_put_block_group (cache); total -= num_bytes;fs/btrfs/ctree.h +7
-2@@ -441,9 +441,14 @@ struct btrfs_full_stripe_locks_tree {/* Discard control. */ /* * Async discard uses multiple lists to differentiate the discard filter -* parameters. +* parameters. Index 0 is for completely free block groups where we need to +* ensure the entire block group is trimmed without being lossy. Indices +* afterwards represent monotonically decreasing discard filter sizes to +* prioritize what should be discarded next. */ -#define BTRFS_NR_DISCARD_LISTS 1 +#define BTRFS_NR_DISCARD_LISTS 2 +#define BTRFS_DISCARD_INDEX_UNUSED 0 +#define BTRFS_DISCARD_INDEX_START 1struct btrfs_discard_ctl {struct workqueue_struct *discard_workers;fs/btrfs/discard.c +109
-3@@ -28,9 +28,13 @@ void btrfs_add_to_discard_list(struct btrfs_discard_ctl *discard_ctl, {spin_lock (&discard_ctl->lock); -if (list_empty (&cache->discard_list)) +if (list_empty (&cache->discard_list) || + cache->discard_index ==BTRFS_DISCARD_INDEX_UNUSED ) { +if (cache->discard_index ==BTRFS_DISCARD_INDEX_UNUSED ) + cache->discard_index =BTRFS_DISCARD_INDEX_START ; cache->discard_eligible_time = (ktime_get_ns () +BTRFS_DISCARD_DELAY ); + }list_move_tail (&cache->discard_list,btrfs_get_discard_list (discard_ctl, cache));@@ -38,6 +42,22 @@ void btrfs_add_to_discard_list(struct btrfs_discard_ctl *discard_ctl,spin_unlock (&discard_ctl->lock); } +void btrfs_add_to_discard_unused_list (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache) +{ +spin_lock (&discard_ctl->lock); + +if (!list_empty (&cache->discard_list)) +list_del_init (&cache->discard_list); + + cache->discard_index =BTRFS_DISCARD_INDEX_UNUSED ; + cache->discard_eligible_time =ktime_get_ns (); +list_add_tail (&cache->discard_list, + &discard_ctl->discard_list[BTRFS_DISCARD_INDEX_UNUSED ]); + +spin_unlock (&discard_ctl->lock); +} +static bool remove_from_discard_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache) {@@ -152,7 +172,11 @@ void btrfs_discard_queue_work(struct btrfs_discard_ctl *discard_ctl,if (!cache || !btrfs_test_opt (cache->fs_info,DISCARD_ASYNC ))return ; -btrfs_add_to_discard_list (discard_ctl, cache); +if (btrfs_block_group_used (&cache->item) ==0 ) +btrfs_add_to_discard_unused_list (discard_ctl, cache); +else +btrfs_add_to_discard_list (discard_ctl, cache); +if (!delayed_work_pending (&discard_ctl->work))btrfs_discard_schedule_work (discard_ctl, false); }@@ -197,6 +221,27 @@ void btrfs_discard_schedule_work(struct btrfs_discard_ctl *discard_ctl,spin_unlock (&discard_ctl->lock); } +/** +* btrfs_finish_discard_pass - determine next step of a block_group +* +* This determines the next step for a block group after it's finished going +* through a pass on a discard list. If it is unused and fully trimmed, we can +* mark it unused and send it to the unused_bgs path. Otherwise, pass it onto +* the appropriate filter list or let it fall off. +*/ +static void btrfs_finish_discard_pass (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache) +{ +remove_from_discard_list (discard_ctl, cache); + +if (btrfs_block_group_used (&cache->item) ==0 ) { +if (btrfs_is_free_space_trimmed (cache)) +btrfs_mark_bg_unused (cache); +else +btrfs_add_to_discard_unused_list (discard_ctl, cache); + } +} +/** * btrfs_discard_workfn - discard work function * @work: work @@ -218,7 +263,7 @@ static void btrfs_discard_workfn(struct work_struct *work)btrfs_trim_block_group (cache, &trimmed, cache->key.objectid,btrfs_block_group_end (cache),0 ); -remove_from_discard_list (discard_ctl, cache); +btrfs_finish_discard_pass (discard_ctl, cache);btrfs_discard_schedule_work (discard_ctl, false); }@@ -239,6 +284,63 @@ bool btrfs_run_discard_work(struct btrfs_discard_ctl *discard_ctl) test_bit(BTRFS_FS_DISCARD_RUNNING, &fs_info->flags)); } +/** +* btrfs_discard_punt_unused_bgs_list - punt unused_bgs list to discard lists +* @fs_info: fs_info of interest +* +* The unused_bgs list needs to be punted to the discard lists because the +* order of operations is changed. In the normal sychronous discard path, the +* block groups are trimmed via a single large trim in transaction commit. This +* is ultimately what we are trying to avoid with asynchronous discard. Thus, +* it must be done before going down the unused_bgs path. +*/ +void btrfs_discard_punt_unused_bgs_list (struct btrfs_fs_info *fs_info) +{ +struct btrfs_block_group_cache *cache, *next; + +spin_lock (&fs_info->unused_bgs_lock); + +/* We enabled async discard, so punt all to the queue. */ +list_for_each_entry_safe (cache, next, &fs_info->unused_bgs, bg_list) { +list_del_init (&cache->bg_list); +btrfs_add_to_discard_unused_list (&fs_info->discard_ctl, cache); + } + +spin_unlock (&fs_info->unused_bgs_lock); +} + +/** +* btrfs_discard_purge_list - purge discard lists +* @discard_ctl: discard control +* +* If we are disabling async discard, we may have intercepted block groups that +* are completely free and ready for the unused_bgs path. As discarding will +* now happen in transaction commit or not at all, we can safely mark the +* corresponding block groups as unused and they will be sent on their merry +* way to the unused_bgs list. +*/ +static void btrfs_discard_purge_list (struct btrfs_discard_ctl *discard_ctl) +{ +struct btrfs_block_group_cache *cache, *next; +int i; + +spin_lock (&discard_ctl->lock); + +for (i =0 ; i <BTRFS_NR_DISCARD_LISTS ; i++) { +list_for_each_entry_safe (cache, next, + &discard_ctl->discard_list[i], + discard_list) { +list_del_init (&cache->discard_list); +spin_unlock (&discard_ctl->lock); +if (btrfs_block_group_used (&cache->item) ==0 ) +btrfs_mark_bg_unused (cache); +spin_lock (&discard_ctl->lock); + } + } + +spin_unlock (&discard_ctl->lock); +} +void btrfs_discard_resume (struct btrfs_fs_info *fs_info) {if (!btrfs_test_opt (fs_info,DISCARD_ASYNC )) {@@ -246,6 +348,8 @@ void btrfs_discard_resume(struct btrfs_fs_info *fs_info)return ; } +btrfs_discard_punt_unused_bgs_list (fs_info); +set_bit (BTRFS_FS_DISCARD_RUNNING , &fs_info->flags); }@@ -271,4 +375,6 @@ void btrfs_discard_cleanup(struct btrfs_fs_info *fs_info) {btrfs_discard_stop (fs_info);cancel_delayed_work_sync (&fs_info->discard_ctl.work); + +btrfs_discard_purge_list (&fs_info->discard_ctl); }fs/btrfs/discard.h +6
-0@@ -9,9 +9,13 @@ struct btrfs_fs_info;struct btrfs_discard_ctl ;struct btrfs_block_group_cache ; +/* List operations. */ void btrfs_add_to_discard_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache); +void btrfs_add_to_discard_unused_list (struct btrfs_discard_ctl *discard_ctl, +struct btrfs_block_group_cache *cache); +/* Work operations. */ void btrfs_discard_cancel_work (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache); void btrfs_discard_queue_work(struct btrfs_discard_ctl *discard_ctl,@@ -20,6 +24,8 @@ void btrfs_discard_schedule_work(struct btrfs_discard_ctl *discard_ctl, bool override);bool btrfs_run_discard_work (struct btrfs_discard_ctl *discard_ctl); +/* Setup/Cleanup operations. */ +void btrfs_discard_punt_unused_bgs_list (struct btrfs_fs_info *fs_info);void btrfs_discard_resume (struct btrfs_fs_info *fs_info);void btrfs_discard_stop (struct btrfs_fs_info *fs_info);void btrfs_discard_init (struct btrfs_fs_info *fs_info);fs/btrfs/free-space-cache.c +21
-0@@ -2681,6 +2681,37 @@ void btrfs_remove_free_space_cache(struct btrfs_block_group_cache *block_group) } +/** +* btrfs_is_free_space_trimmed - see if everything is trimmed +* @cache: block_group of interest +* +* Walk the @cache's free space rb_tree to determine if everything is trimmed. +*/ +bool btrfs_is_free_space_trimmed (struct btrfs_block_group_cache *cache) +{ +struct btrfs_free_space_ctl *ctl = cache->free_space_ctl; +struct btrfs_free_space *info; +struct rb_node *node; +bool ret = true; + +spin_lock (&ctl->tree_lock); + node =rb_first (&ctl->free_space_offset); + +while (node) { + info = rb_entry(node, struct btrfs_free_space, offset_index); + +if (!btrfs_free_space_trimmed (info)) { + ret = false;
fs/btrfs/free-space-cache.c void btrfs_remove_free_space_cache(struct btrfs_block_group_cache *block_group) · line 2714
Quoted 10 lines
fs/btrfs/free-space-cache.c +10
-0+break ; + } + + node =rb_next (node); + } + +spin_unlock (&ctl->tree_lock); +return ret; +} +
I'm not a fan of this, but it's only appears to be called when btrfs_block_group_item_used() == 0, so there should only be at most 2 entries in general. However this could be fixed up to keep track of untrimmed ytes in free_space_ctl. Not a big enough deal for right now, but maybe if you feel squirrely in the future. Otherwise
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 7/22 btrfs: discard one region at a time in async discard
In reply to Dennis ZhouThe prior two patches added discarding via a background workqueue. This just piggybacked off of the fstrim code to trim the whole block at once. Well inevitably this is worse performance wise and will aggressively overtrim. But it was nice to plumb the other infrastructure to keep the patches easier to review.
This adds the real goal of this series which is discarding slowly (ie a slow long running fstrim). The discarding is split into two phases, extents and then bitmaps. The reason for this is two fold. First, the bitmap regions overlap the extent regions. Second, discarding the extents first will let the newly trimmed bitmaps have the highest chance of coalescing when being readded to the free space cache.
--- fs/btrfs/block-group.h | 15 +++++ fs/btrfs/discard.c | 79 ++++++++++++++++++---- fs/btrfs/free-space-cache.c | 130 ++++++++++++++++++++++++++++-------- fs/btrfs/free-space-cache.h | 6 ++ 4 files changed, 188 insertions(+), 42 deletions(-)
fs/btrfs/block-group.h +15 -0
@@ -12,6 +12,19 @@ enum btrfs_disk_cache_state { BTRFS_DC_SETUP, }; +/* +* This describes the state of the block_group for async discard. This is due +* to the two pass nature of it where extent discarding is prioritized over +* bitmap discarding. BTRFS_DISCARD_RESET_CURSOR is set when we are resetting +* between lists to prevent contention for discard state variables +* (eg discard_cursor). +*/ +enum btrfs_discard_state { +BTRFS_DISCARD_EXTENTS , +BTRFS_DISCARD_BITMAPS , +BTRFS_DISCARD_RESET_CURSOR , +}; +/* * Control flags for do_chunk_alloc's force field CHUNK_ALLOC_NO_FORCE means to * only allocate a chunk if we really need one.
@@ -120,6 +133,8 @@ struct btrfs_block_group_cache {struct list_head discard_list;int discard_index;u64 discard_eligible_time; +u64 discard_cursor; +enum btrfs_discard_state discard_state;/* For dirty block groups */ struct list_head dirty_list;
fs/btrfs/discard.c +66 -13
140 lines not on this page · raw message
fs/btrfs/free-space-cache.c +101 -29
224 lines not on this page · raw message
fs/btrfs/free-space-cache.h +6 -0
@@ -138,6 +138,12 @@ int btrfs_return_cluster_to_free_space( struct btrfs_free_cluster *cluster);int btrfs_trim_block_group (struct btrfs_block_group_cache *block_group,u64 *trimmed,u64 start,u64 end,u64 minlen); +int btrfs_trim_block_group_extents (struct btrfs_block_group_cache *block_group, +u64 *trimmed,u64 start,u64 end,u64 minlen, +bool async); +int btrfs_trim_block_group_bitmaps (struct btrfs_block_group_cache *block_group, +u64 *trimmed,u64 start,u64 end,u64 minlen, +bool async);/* Support functions for running our sanity tests */ #ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
Signature · 1 line
2.17.1
Re: [PATCH 07/22] btrfs: discard one region at a time in async discard
Quoted 6 lines · Dennis Zhou
The prior two patches added discarding via a background workqueue. This just piggybacked off of the fstrim code to trim the whole block at once. Well inevitably this is worse performance wise and will aggressively overtrim. But it was nice to plumb the other infrastructure to keep the patches easier to review.
Quoted 8 lines
This adds the real goal of this series which is discarding slowly (ie a slow long running fstrim). The discarding is split into two phases, extents and then bitmaps. The reason for this is two fold. First, the bitmap regions overlap the extent regions. Second, discarding the extents first will let the newly trimmed bitmaps have the highest chance of coalescing when being readded to the free space cache.
Signed-off-by Dennis Zhou <[email protected]>
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 8/22 btrfs: add removal calls for sysfs debug/
In reply to Dennis ZhouWe probably should call sysfs_remove_group() on debug/.
--- fs/btrfs/sysfs.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/btrfs/sysfs.c +7 -0
@@ -732,6 +732,10 @@ void btrfs_sysfs_remove_mounted(struct btrfs_fs_info *fs_info)kobject_del (fs_info->space_info_kobj);kobject_put (fs_info->space_info_kobj); } +#ifdef CONFIG_BTRFS_DEBUG +sysfs_remove_group (&fs_info->fs_devices->fsid_kobj, + &btrfs_debug_feature_attr_group); +#endif addrm_unknown_feature_attrs (fs_info, false);sysfs_remove_group (&fs_info->fs_devices->fsid_kobj, &btrfs_feature_attr_group);sysfs_remove_files (&fs_info->fs_devices->fsid_kobj, btrfs_attrs);
@@ -1170,6 +1174,9 @@ void __cold btrfs_exit_sysfs(void)sysfs_unmerge_group (&btrfs_kset->kobj, &btrfs_static_feature_attr_group);sysfs_remove_group (&btrfs_kset->kobj, &btrfs_feature_attr_group); +#ifdef CONFIG_BTRFS_DEBUG +sysfs_remove_group (&btrfs_kset->kobj, &btrfs_debug_feature_attr_group); +#endif kset_unregister (btrfs_kset); }
Signature · 1 line
2.17.1
Re: [PATCH 08/22] btrfs: add removal calls for sysfs debug/
Quoted 3 lines · Dennis Zhou
We probably should call
sysfs_remove_group()on debug/.Signed-off-by Dennis Zhou <[email protected]>
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 9/22 btrfs: make UUID/debug have its own kobject
In reply to Dennis ZhouBtrfs only allowed attributes to be exposed in debug/. Let's let other groups be created by making debug its own kobject.
This also makes the per-fs debug options separate from the global features mount attributes. This seems to be needed as sysfs_create_files() requires const struct attribute * while sysfs_create_group() can take struct attribute *. This seems nicer as per file system, you'll probably use to_fs_info().
--- fs/btrfs/ctree.h | 4 ++++ fs/btrfs/sysfs.c | 20 ++++++++++++++++---- 2 files changed, 20 insertions(+), 4 deletions(-)
fs/btrfs/ctree.h +4 -0
@@ -928,6 +928,10 @@ struct btrfs_fs_info {spinlock_t ref_verify_lock;struct rb_root block_tree;#endif + +#ifdef CONFIG_BTRFS_DEBUG +struct kobject *debug_kobj; +#endif };static inline struct btrfs_fs_info *btrfs_sb (struct super_block *sb)
fs/btrfs/sysfs.c +16 -4
@@ -319,6 +319,10 @@ static const struct attribute_group btrfs_static_feature_attr_group = {* /sys/fs/btrfs/debug - applies to module or all filesystems * /sys/fs/btrfs/UUID - applies only to the given filesystem */ +static const struct attribute *btrfs_debug_mount_attrs[] = { +NULL , +}; +static struct attribute *btrfs_debug_feature_attrs[] = {NULL };
@@ -733,8 +737,12 @@ void btrfs_sysfs_remove_mounted(struct btrfs_fs_info *fs_info)kobject_put (fs_info->space_info_kobj); }#ifdef CONFIG_BTRFS_DEBUG -sysfs_remove_group (&fs_info->fs_devices->fsid_kobj, - &btrfs_debug_feature_attr_group); +if (fs_info->debug_kobj) { +sysfs_remove_files (fs_info->debug_kobj, + btrfs_debug_mount_attrs); +kobject_del (fs_info->debug_kobj); +kobject_put (fs_info->debug_kobj); + }#endif addrm_unknown_feature_attrs (fs_info, false);sysfs_remove_group (&fs_info->fs_devices->fsid_kobj, &btrfs_feature_attr_group);
@@ -1076,8 +1084,12 @@ int btrfs_sysfs_add_mounted(struct btrfs_fs_info *fs_info) goto failure;#ifdef CONFIG_BTRFS_DEBUG - error =sysfs_create_group (fsid_kobj, - &btrfs_debug_feature_attr_group); + fs_info->debug_kobj =kobject_create_and_add ("debug" , fsid_kobj); +if (!fs_info->debug_kobj) + goto failure; + + error =sysfs_create_files (fs_info->debug_kobj, + btrfs_debug_mount_attrs);if (error) goto failure;#endif
Signature · 1 line
2.17.1
Re: [PATCH 09/22] btrfs: make UUID/debug have its own kobject
Quoted 10 lines · Dennis Zhou
Btrfs only allowed attributes to be exposed in debug/. Let's let other groups be created by making debug its own kobject.
This also makes the per-fs debug options separate from the global features mount attributes. This seems to be needed as
sysfs_create_files()requires const struct attribute * whilesysfs_create_group()can take struct attribute *. This seems nicer as per file system, you'll probably useto_fs_info().Signed-off-by Dennis Zhou <[email protected]>
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 10/22 btrfs: add discard sysfs directory
In reply to Dennis ZhouSetup sysfs directory for discard stats + tunables.
--- fs/btrfs/ctree.h | 1 + fs/btrfs/sysfs.c | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+)
fs/btrfs/ctree.h +1 -0
@@ -931,6 +931,7 @@ struct btrfs_fs_info {#ifdef CONFIG_BTRFS_DEBUG struct kobject *debug_kobj; +struct kobject *discard_kobj;#endif };
fs/btrfs/sysfs.c +24 -0
@@ -313,6 +313,13 @@ static const struct attribute_group btrfs_static_feature_attr_group = {#ifdef CONFIG_BTRFS_DEBUG +/* +* Discard statistics and tunables. +*/ +static const struct attribute *discard_attrs[] = { +NULL , +}; +/* * Runtime debugging exported via sysfs *
@@ -737,6 +744,11 @@ void btrfs_sysfs_remove_mounted(struct btrfs_fs_info *fs_info)kobject_put (fs_info->space_info_kobj); }#ifdef CONFIG_BTRFS_DEBUG +if (fs_info->discard_kobj) { +sysfs_remove_files (fs_info->discard_kobj, discard_attrs); +kobject_del (fs_info->discard_kobj); +kobject_put (fs_info->discard_kobj); + }if (fs_info->debug_kobj) {sysfs_remove_files (fs_info->debug_kobj, btrfs_debug_mount_attrs);
@@ -1092,6 +1104,18 @@ int btrfs_sysfs_add_mounted(struct btrfs_fs_info *fs_info) btrfs_debug_mount_attrs);if (error) goto failure; + +/* Discard directory. */ + fs_info->discard_kobj =kobject_create_and_add ("discard" , + fs_info->debug_kobj); +if (!fs_info->discard_kobj) { + error = -ENOMEM ; + goto failure; + } + + error =sysfs_create_files (fs_info->discard_kobj, discard_attrs); +if (error) + goto failure;#endif error =addrm_unknown_feature_attrs (fs_info, true);
Signature · 1 line
2.17.1
Re: [PATCH 10/22] btrfs: add discard sysfs directory
Quoted 3 lines · Dennis Zhou
Setup sysfs directory for discard stats + tunables.
Signed-off-by Dennis Zhou <[email protected]>
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 11/22 btrfs: track discardable extents for async discard
In reply to Dennis ZhouThe number of discardable extents will serve as the rate limiting metric for how often we should discard. This keeps track of discardable extents in the free space caches by maintaining deltas and propagating them to the global count.
--- fs/btrfs/ctree.h | 10 ++++ fs/btrfs/discard.c | 31 +++++++++++ fs/btrfs/discard.h | 4 ++ fs/btrfs/free-space-cache.c | 106 +++++++++++++++++++++++++++++++++--- fs/btrfs/free-space-cache.h | 2 + fs/btrfs/sysfs.c | 15 +++++ 6 files changed, 159 insertions(+), 9 deletions(-)
fs/btrfs/ctree.h +10 -0
@@ -100,6 +100,15 @@ struct btrfs_ref;#define BTRFS_MAX_EXTENT_SIZE SZ_128M +/* +* Deltas are an effective way to populate global statistics. Give macro names +* to make it clear what we're doing. An example is discard_extents in +* btrfs_free_space_ctl. +*/ +#define BTRFS_STAT_NR_ENTRIES 2 +#define BTRFS_STAT_CURR 0 +#define BTRFS_STAT_PREV 1 +/* * Count how many BTRFS_MAX_EXTENT_SIZE cover the @size
@@ -456,6 +465,7 @@ struct btrfs_discard_ctl {spinlock_t lock;struct btrfs_block_group_cache *cache;struct list_head discard_list[BTRFS_NR_DISCARD_LISTS ]; +atomic_t discardable_extents; };/* delayed seq elem */
fs/btrfs/discard.c +31 -0
@@ -337,6 +337,35 @@ bool btrfs_run_discard_work(struct btrfs_discard_ctl *discard_ctl) test_bit(BTRFS_FS_DISCARD_RUNNING, &fs_info->flags)); } +/** +* btrfs_discard_update_discardable - propagate discard counters +* @cache: block_group of interest +* @ctl: free_space_ctl of @cache +* +* This propagates deltas of counters up to the discard_ctl. It maintains a +* current counter and a previous counter passing the delta up to the global +* stat. Then the current counter value becomes the previous counter value. +*/ +void btrfs_discard_update_discardable (struct btrfs_block_group_cache *cache, +struct btrfs_free_space_ctl *ctl) +{ +struct btrfs_discard_ctl *discard_ctl; +s32 extents_delta; + +if (!cache || !btrfs_test_opt (cache->fs_info,DISCARD_ASYNC )) +return ; + + discard_ctl = &cache->fs_info->discard_ctl; + + extents_delta = (ctl->discardable_extents[BTRFS_STAT_CURR ] - + ctl->discardable_extents[BTRFS_STAT_PREV ]); +if (extents_delta) { +atomic_add (extents_delta, &discard_ctl->discardable_extents); + ctl->discardable_extents[BTRFS_STAT_PREV ] = + ctl->discardable_extents[BTRFS_STAT_CURR ]; + } +} +/** * btrfs_discard_punt_unused_bgs_list - punt unused_bgs list to discard lists * @fs_info: fs_info of interest
@@ -422,6 +451,8 @@ void btrfs_discard_init(struct btrfs_fs_info *fs_info)for (i =0 ; i <BTRFS_NR_DISCARD_LISTS ; i++)INIT_LIST_HEAD (&discard_ctl->discard_list[i]); + +atomic_set (&discard_ctl->discardable_extents,0 ); }void btrfs_discard_cleanup (struct btrfs_fs_info *fs_info)
fs/btrfs/discard.h +4 -0
@@ -24,6 +24,10 @@ void btrfs_discard_schedule_work(struct btrfs_discard_ctl *discard_ctl, bool override);bool btrfs_run_discard_work (struct btrfs_discard_ctl *discard_ctl); +/* Update operations. */ +void btrfs_discard_update_discardable (struct btrfs_block_group_cache *cache, +struct btrfs_free_space_ctl *ctl); +/* Setup/Cleanup operations. */ void btrfs_discard_punt_unused_bgs_list (struct btrfs_fs_info *fs_info);void btrfs_discard_resume (struct btrfs_fs_info *fs_info);
fs/btrfs/free-space-cache.c +97 -9
240 lines not on this page · raw message
fs/btrfs/free-space-cache.h +2 -0
@@ -28,6 +28,7 @@ struct btrfs_free_space {unsigned long *bitmap;struct list_head list;enum btrfs_trim_state trim_state; +s32 bitmap_extents; };static inline bool btrfs_free_space_trimmed (struct btrfs_free_space *info)
@@ -50,6 +51,7 @@ struct btrfs_free_space_ctl {int total_bitmaps;int unit;u64 start; +s32 discardable_extents[BTRFS_STAT_NR_ENTRIES ];const struct btrfs_free_space_op *op;void *private;struct mutex cache_writeout_mutex;
fs/btrfs/sysfs.c +15 -0
@@ -11,6 +11,7 @@#include <linux/bug.h> #include "ctree.h" +#include "discard.h" #include "disk-io.h" #include "transaction.h" #include "sysfs.h"
@@ -316,7 +317,21 @@ static const struct attribute_group btrfs_static_feature_attr_group = {/* * Discard statistics and tunables. */ +#define discard_to_fs_info (_kobj) to_fs_info((_kobj)->parent->parent) + +static ssize_t btrfs_discardable_extents_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%d\n" , +atomic_read (&fs_info->discard_ctl.discardable_extents)); +} +BTRFS_ATTR (discard, discardable_extents, btrfs_discardable_extents_show); +static const struct attribute *discard_attrs[] = { +BTRFS_ATTR_PTR (discard, discardable_extents),NULL , };
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 12/22 btrfs: keep track of discardable_bytes
In reply to Dennis ZhouKeep track of this metric so that we can understand how ahead or behind we are in discarding rate.
--- fs/btrfs/ctree.h | 1 + fs/btrfs/discard.c | 10 +++++++++ fs/btrfs/free-space-cache.c | 41 +++++++++++++++++++++++++++++-------- fs/btrfs/free-space-cache.h | 1 + fs/btrfs/sysfs.c | 12 +++++++++++ 5 files changed, 56 insertions(+), 9 deletions(-)
fs/btrfs/ctree.h +1 -0
@@ -466,6 +466,7 @@ struct btrfs_discard_ctl {struct btrfs_block_group_cache *cache;struct list_head discard_list[BTRFS_NR_DISCARD_LISTS ];atomic_t discardable_extents; +atomic64_t discardable_bytes; };/* delayed seq elem */
fs/btrfs/discard.c +10 -0
@@ -351,6 +351,7 @@ void btrfs_discard_update_discardable(struct btrfs_block_group_cache *cache, {struct btrfs_discard_ctl *discard_ctl;s32 extents_delta; +s64 bytes_delta;if (!cache || !btrfs_test_opt (cache->fs_info,DISCARD_ASYNC ))return ;
@@ -364,6 +365,14 @@ void btrfs_discard_update_discardable(struct btrfs_block_group_cache *cache, ctl->discardable_extents[BTRFS_STAT_PREV ] = ctl->discardable_extents[BTRFS_STAT_CURR ]; } + + bytes_delta = (ctl->discardable_bytes[BTRFS_STAT_CURR ] - + ctl->discardable_bytes[BTRFS_STAT_PREV ]); +if (bytes_delta) { +atomic64_add (bytes_delta, &discard_ctl->discardable_bytes); + ctl->discardable_bytes[BTRFS_STAT_PREV ] = + ctl->discardable_bytes[BTRFS_STAT_CURR ]; + } }/**
@@ -453,6 +462,7 @@ void btrfs_discard_init(struct btrfs_fs_info *fs_info)INIT_LIST_HEAD (&discard_ctl->discard_list[i]);atomic_set (&discard_ctl->discardable_extents,0 ); +atomic64_set (&discard_ctl->discardable_bytes,0 ); }void btrfs_discard_cleanup (struct btrfs_fs_info *fs_info)
fs/btrfs/free-space-cache.c +32 -9
118 lines not on this page · raw message
fs/btrfs/free-space-cache.h +1 -0
@@ -52,6 +52,7 @@ struct btrfs_free_space_ctl {int unit;u64 start;s32 discardable_extents[BTRFS_STAT_NR_ENTRIES ]; +s64 discardable_bytes[BTRFS_STAT_NR_ENTRIES ];const struct btrfs_free_space_op *op;void *private;struct mutex cache_writeout_mutex;
fs/btrfs/sysfs.c +12 -0
@@ -330,8 +330,20 @@ static ssize_t btrfs_discardable_extents_show(struct kobject *kobj, }BTRFS_ATTR (discard, discardable_extents, btrfs_discardable_extents_show); +static ssize_t btrfs_discardable_bytes_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%lld\n" , +atomic64_read (&fs_info->discard_ctl.discardable_bytes)); +} +BTRFS_ATTR (discard, discardable_bytes, btrfs_discardable_bytes_show); +static const struct attribute *discard_attrs[] = {BTRFS_ATTR_PTR (discard, discardable_extents), +BTRFS_ATTR_PTR (discard, discardable_bytes),NULL , };
Signature · 1 line
2.17.1
Re: [PATCH 12/22] btrfs: keep track of discardable_bytes
Quoted 3 lines · Dennis Zhou
Keep track of this metric so that we can understand how ahead or behind we are in discarding rate.
If these are only going to be exposed via a debug sysfs, then it makes sense to only track it if we have DEBUG set. Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 13/22 btrfs: calculate discard delay based on number of extents
In reply to Dennis ZhouUse the number of discardable extents to help guide our discard delay interval. This value is reevaluated every transaction commit.
--- fs/btrfs/ctree.h | 2 ++ fs/btrfs/discard.c | 52 ++++++++++++++++++++++++++++++++++++++---- fs/btrfs/discard.h | 1 + fs/btrfs/extent-tree.c | 4 +++- fs/btrfs/sysfs.c | 31 +++++++++++++++++++++++++ 5 files changed, 85 insertions(+), 5 deletions(-)
fs/btrfs/ctree.h +2 -0
@@ -467,6 +467,8 @@ struct btrfs_discard_ctl {struct list_head discard_list[BTRFS_NR_DISCARD_LISTS ];atomic_t discardable_extents;atomic64_t discardable_bytes; +u32 delay; +u32 iops_limit; };/* delayed seq elem */
fs/btrfs/discard.c +48 -4
77 lines not on this page · raw message
fs/btrfs/discard.h +1 -0
@@ -25,6 +25,7 @@ void btrfs_discard_schedule_work(struct btrfs_discard_ctl *discard_ctl,bool btrfs_run_discard_work (struct btrfs_discard_ctl *discard_ctl);/* Update operations. */ +void btrfs_discard_calc_delay (struct btrfs_discard_ctl *discard_ctl);void btrfs_discard_update_discardable (struct btrfs_block_group_cache *cache,struct btrfs_free_space_ctl *ctl);
fs/btrfs/extent-tree.c +3 -1
@@ -2921,8 +2921,10 @@ int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)cond_resched (); } -if (btrfs_test_opt (fs_info,DISCARD_ASYNC )) +if (btrfs_test_opt (fs_info,DISCARD_ASYNC )) { +btrfs_discard_calc_delay (&fs_info->discard_ctl);btrfs_discard_schedule_work (&fs_info->discard_ctl, true); + }/* * Transaction is finished. We don't need the lock anymore. We
fs/btrfs/sysfs.c +31 -0
@@ -341,9 +341,40 @@ static ssize_t btrfs_discardable_bytes_show(struct kobject *kobj, }BTRFS_ATTR (discard, discardable_bytes, btrfs_discardable_bytes_show); +static ssize_t btrfs_discard_iops_limit_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%u\n" , +READ_ONCE (fs_info->discard_ctl.iops_limit)); +} + +static ssize_t btrfs_discard_iops_limit_store (struct kobject *kobj, +struct kobj_attribute *a, +const char *buf,size_t len) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); +struct btrfs_discard_ctl *discard_ctl = &fs_info->discard_ctl; +u32 iops_limit; +int ret; + + ret =kstrtou32 (buf,10 , &iops_limit); +if (ret) +return -EINVAL ; + +WRITE_ONCE (discard_ctl->iops_limit, iops_limit); + +return len; +} +BTRFS_ATTR_RW (discard, iops_limit, btrfs_discard_iops_limit_show, + btrfs_discard_iops_limit_store); +static const struct attribute *discard_attrs[] = {BTRFS_ATTR_PTR (discard, discardable_extents),BTRFS_ATTR_PTR (discard, discardable_bytes), +BTRFS_ATTR_PTR (discard, iops_limit),NULL , };
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 14/22 btrfs: add bps discard rate limit
In reply to Dennis ZhouProvide an ability to rate limit based on mbps in addition to the iops delay calculated from number of discardable extents.
--- fs/btrfs/ctree.h | 2 ++ fs/btrfs/discard.c | 17 +++++++++++++++++ fs/btrfs/sysfs.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)
fs/btrfs/ctree.h +2 -0
@@ -465,10 +465,12 @@ struct btrfs_discard_ctl {spinlock_t lock;struct btrfs_block_group_cache *cache;struct list_head discard_list[BTRFS_NR_DISCARD_LISTS ]; +u64 prev_discard;atomic_t discardable_extents;atomic64_t discardable_bytes;u32 delay;u32 iops_limit; +u64 bps_limit; };/* delayed seq elem */
fs/btrfs/discard.c +17 -0
@@ -238,6 +238,19 @@ void btrfs_discard_schedule_work(struct btrfs_discard_ctl *discard_ctl, cache =find_next_cache (discard_ctl, now);if (cache) {u32 delay = discard_ctl->delay; +u64 bps_limit =READ_ONCE (discard_ctl->bps_limit); + +/* +* A single delayed workqueue item is responsible for +* discarding, so we can manage the bytes rate limit by keeping +* track of the previous discard. +*/ +if (bps_limit && discard_ctl->prev_discard) { +u64 bps_delay = (MSEC_PER_SEC * + discard_ctl->prev_discard / bps_limit); + + delay =max_t (u64, delay,msecs_to_jiffies (bps_delay)); + }/* * This timeout is to hopefully prevent immediate discarding
@@ -312,6 +325,8 @@ static void btrfs_discard_workfn(struct work_struct *work)btrfs_block_group_end (cache), 0, true); + discard_ctl->prev_discard = trimmed; +/* Determine next steps for a block_group. */ if (cache->discard_cursor >=btrfs_block_group_end (cache)) {if (discard_state ==BTRFS_DISCARD_BITMAPS ) {
@@ -503,10 +518,12 @@ void btrfs_discard_init(struct btrfs_fs_info *fs_info)for (i =0 ; i <BTRFS_NR_DISCARD_LISTS ; i++)INIT_LIST_HEAD (&discard_ctl->discard_list[i]); + discard_ctl->prev_discard =0 ;atomic_set (&discard_ctl->discardable_extents,0 );atomic64_set (&discard_ctl->discardable_bytes,0 ); discard_ctl->delay =BTRFS_DISCARD_MAX_DELAY ; discard_ctl->iops_limit =BTRFS_DISCARD_MAX_IOPS ; + discard_ctl->bps_limit =0 ; }void btrfs_discard_cleanup (struct btrfs_fs_info *fs_info)
fs/btrfs/sysfs.c +31 -0
@@ -371,10 +371,41 @@ static ssize_t btrfs_discard_iops_limit_store(struct kobject *kobj,BTRFS_ATTR_RW (discard, iops_limit, btrfs_discard_iops_limit_show, btrfs_discard_iops_limit_store); +static ssize_t btrfs_discard_bps_limit_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%llu\n" , +READ_ONCE (fs_info->discard_ctl.bps_limit)); +} + +static ssize_t btrfs_discard_bps_limit_store (struct kobject *kobj, +struct kobj_attribute *a, +const char *buf,size_t len) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); +struct btrfs_discard_ctl *discard_ctl = &fs_info->discard_ctl; +u64 bps_limit; +int ret; + + ret =kstrtou64 (buf,10 , &bps_limit); +if (ret) +return -EINVAL ; + +WRITE_ONCE (discard_ctl->bps_limit, bps_limit); + +return len; +} +BTRFS_ATTR_RW (discard, bps_limit, btrfs_discard_bps_limit_show, + btrfs_discard_bps_limit_store); +static const struct attribute *discard_attrs[] = {BTRFS_ATTR_PTR (discard, discardable_extents),BTRFS_ATTR_PTR (discard, discardable_bytes),BTRFS_ATTR_PTR (discard, iops_limit), +BTRFS_ATTR_PTR (discard, bps_limit),NULL , };
Signature · 1 line
2.17.1
Re: [PATCH 14/22] btrfs: add bps discard rate limit
Quoted 4 lines · Dennis Zhou
Provide an ability to rate limit based on mbps in addition to the iops delay calculated from number of discardable extents.
Signed-off-by Dennis Zhou <[email protected]>
I'm sort of confused, are we hiding the ability to set the iops limit and bps limit behind BTRFS_DEBUG for a reason? I get the stats and stuff, but this bit is confusing. We should probably at least let these things be configurable normally, and keep the stats as debug. Thanks,
Josef
Re: [PATCH 14/22] btrfs: add bps discard rate limit
Quoted 6 lines · Josef Bacik
Quoted 2 lines · Dennis Zhou
Provide an ability to rate limit based on mbps in addition to the iops delay calculated from number of discardable extents.
I'm sort of confused, are we hiding the ability to set the iops limit and bps limit behind
BTRFS_DEBUGfor a reason?
IMHO the iops and ratelimiting should tune itself and no extra knobs should be necessary because this depends on the underlying devices and current load. The user set parameters can be wrong the moment it's done.
Before we find the right way to tune it, the sysfs knobs are there to make it easy for someobody familiar with the internals of the async discard working to experiment.
If we'd still want something for regular users, I'd suggest to add some sort of high-level strategies, like bursty/batched/conservative/slow/idk. But this would anyway build on existing default behaviour.
Dennis Zhou
maintainerPATCH v2 15/22 btrfs: limit max discard size for async discard
In reply to Dennis ZhouThrottle the maximum size of a discard so that we can provide an upper bound for the rate of async discard. While the block layer is able to split discards into the appropriate sized discards, we want to be able to account more accurately the rate at which we are consuming ncq slots as well as limit the upper bound of work for a discard.
--- fs/btrfs/discard.h | 5 ++++ fs/btrfs/free-space-cache.c | 48 +++++++++++++++++++++++++++---------- 2 files changed, 41 insertions(+), 12 deletions(-)
fs/btrfs/discard.h +5 -0
@@ -5,10 +5,15 @@#ifndef BTRFS_DISCARD_H #define BTRFS_DISCARD_H +#include <linux/sizes.h> +struct btrfs_fs_info ;struct btrfs_discard_ctl ;struct btrfs_block_group_cache ; +/* Discard size limits. */ +#define BTRFS_ASYNC_DISCARD_MAX_SIZE (SZ_64M) +/* List operations. */ void btrfs_add_to_discard_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache);
fs/btrfs/free-space-cache.c +36 -12
61 lines not on this page · raw message
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 16/22 btrfs: make max async discard size tunable
In reply to Dennis ZhouExpose max_discard_size as a tunable via sysfs.
--- fs/btrfs/ctree.h | 1 + fs/btrfs/discard.c | 1 + fs/btrfs/free-space-cache.c | 19 ++++++++++++------- fs/btrfs/sysfs.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 45 insertions(+), 7 deletions(-)
fs/btrfs/ctree.h +1 -0
@@ -468,6 +468,7 @@ struct btrfs_discard_ctl {u64 prev_discard;atomic_t discardable_extents;atomic64_t discardable_bytes; +u64 max_discard_size;u32 delay;u32 iops_limit;u64 bps_limit;
fs/btrfs/discard.c +1 -0
@@ -521,6 +521,7 @@ void btrfs_discard_init(struct btrfs_fs_info *fs_info) discard_ctl->prev_discard =0 ;atomic_set (&discard_ctl->discardable_extents,0 );atomic64_set (&discard_ctl->discardable_bytes,0 ); + discard_ctl->max_discard_size =BTRFS_ASYNC_DISCARD_MAX_SIZE ; discard_ctl->delay =BTRFS_DISCARD_MAX_DELAY ; discard_ctl->iops_limit =BTRFS_DISCARD_MAX_IOPS ; discard_ctl->bps_limit =0 ;
fs/btrfs/free-space-cache.c +12 -7
@@ -3406,6 +3406,8 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group, u64 *total_trimmed, u64 start, u64 end, u64 minlen, bool async) { + struct btrfs_discard_ctl *discard_ctl = + &block_group->fs_info->discard_ctl;struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl;struct btrfs_free_space *entry;struct rb_node *node;
@@ -3414,6 +3416,7 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group,u64 extent_bytes;enum btrfs_trim_state extent_trim_state;u64 bytes; +u64 max_discard_size =READ_ONCE (discard_ctl->max_discard_size);while (start < end) {struct btrfs_trim_range trim_entry;
@@ -3450,11 +3453,10 @@ static int trim_no_bitmap(struct btrfs_block_group_cache *block_group, goto next; }unlink_free_space (ctl, entry); -if (bytes >BTRFS_ASYNC_DISCARD_MAX_SIZE ) { - bytes = extent_bytes = -BTRFS_ASYNC_DISCARD_MAX_SIZE ; - entry->offset +=BTRFS_ASYNC_DISCARD_MAX_SIZE ; - entry->bytes -=BTRFS_ASYNC_DISCARD_MAX_SIZE ; +if (max_discard_size && bytes > max_discard_size) { + bytes = extent_bytes = max_discard_size; + entry->offset += max_discard_size; + entry->bytes -= max_discard_size;link_free_space (ctl, entry); }else {kmem_cache_free (btrfs_free_space_cachep, entry);
@@ -3563,12 +3565,15 @@ static int trim_bitmaps(struct btrfs_block_group_cache *block_group, u64 *total_trimmed, u64 start, u64 end, u64 minlen, bool async) { + struct btrfs_discard_ctl *discard_ctl = + &block_group->fs_info->discard_ctl;struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl;struct btrfs_free_space *entry;int ret =0 ;int ret2;u64 bytes;u64 offset =offset_to_bitmap (ctl, start); +u64 max_discard_size =READ_ONCE (discard_ctl->max_discard_size);while (offset < end) {bool next_bitmap = false;
@@ -3638,8 +3643,8 @@ static int trim_bitmaps(struct btrfs_block_group_cache *block_group, goto next; } -if (async && bytes >BTRFS_ASYNC_DISCARD_MAX_SIZE ) - bytes =BTRFS_ASYNC_DISCARD_MAX_SIZE ; +if (async && max_discard_size && bytes > max_discard_size) + bytes = max_discard_size;bitmap_clear_bits (ctl, entry, start, bytes);if (entry->bytes ==0 )
fs/btrfs/sysfs.c +31 -0
@@ -401,11 +401,42 @@ static ssize_t btrfs_discard_bps_limit_store(struct kobject *kobj,BTRFS_ATTR_RW (discard, bps_limit, btrfs_discard_bps_limit_show, btrfs_discard_bps_limit_store); +static ssize_t btrfs_discard_max_discard_size_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%llu\n" , +READ_ONCE (fs_info->discard_ctl.max_discard_size)); +} + +static ssize_t btrfs_discard_max_discard_size_store (struct kobject *kobj, +struct kobj_attribute *a, +const char *buf,size_t len) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); +struct btrfs_discard_ctl *discard_ctl = &fs_info->discard_ctl; +u64 max_discard_size; +int ret; + + ret =kstrtou64 (buf,10 , &max_discard_size); +if (ret) +return -EINVAL ; + +WRITE_ONCE (discard_ctl->max_discard_size, max_discard_size); + +return len; +} +BTRFS_ATTR_RW (discard, max_discard_size, btrfs_discard_max_discard_size_show, + btrfs_discard_max_discard_size_store); +static const struct attribute *discard_attrs[] = {BTRFS_ATTR_PTR (discard, discardable_extents),BTRFS_ATTR_PTR (discard, discardable_bytes),BTRFS_ATTR_PTR (discard, iops_limit),BTRFS_ATTR_PTR (discard, bps_limit), +BTRFS_ATTR_PTR (discard, max_discard_size),NULL , };
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 17/22 btrfs: have multiple discard lists
In reply to Dennis ZhouNon-block group destruction discarding currently only had a single list with no minimum discard length. This can lead to caravaning more meaningful discards behind a heavily fragmented block group.
This adds support for multiple lists with minimum discard lengths to prevent the caravan effect. We promote block groups back up when we exceed the BTRFS_ASYNC_DISCARD_MAX_FILTER size, currently we support only 2 lists with filters of 1MB and 32KB respectively.
--- fs/btrfs/ctree.h | 2 +- fs/btrfs/discard.c | 98 ++++++++++++++++++++++++++++++++++--- fs/btrfs/discard.h | 4 ++ fs/btrfs/free-space-cache.c | 53 +++++++++++++++----- fs/btrfs/free-space-cache.h | 2 +- 5 files changed, 136 insertions(+), 23 deletions(-)
fs/btrfs/ctree.h +1 -1
@@ -455,7 +455,7 @@ struct btrfs_full_stripe_locks_tree {* afterwards represent monotonically decreasing discard filter sizes to * prioritize what should be discarded next. */ -#define BTRFS_NR_DISCARD_LISTS 2 +#define BTRFS_NR_DISCARD_LISTS 3#define BTRFS_DISCARD_INDEX_UNUSED 0#define BTRFS_DISCARD_INDEX_START 1
fs/btrfs/discard.c +90 -8
155 lines not on this page · raw message
fs/btrfs/discard.h +4 -0
@@ -13,12 +13,16 @@ struct btrfs_block_group_cache;/* Discard size limits. */ #define BTRFS_ASYNC_DISCARD_MAX_SIZE (SZ_64M) +#define BTRFS_ASYNC_DISCARD_MAX_FILTER (SZ_1M) +#define BTRFS_ASYNC_DISCARD_MIN_FILTER (SZ_32K)/* List operations. */ void btrfs_add_to_discard_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache);void btrfs_add_to_discard_unused_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache); +void btrfs_discard_check_filter (struct btrfs_block_group_cache *cache, +u64 bytes);/* Work operations. */ void btrfs_discard_cancel_work(struct btrfs_discard_ctl *discard_ctl,
fs/btrfs/free-space-cache.c +40 -13
126 lines not on this page · raw message
fs/btrfs/free-space-cache.h +1 -1
@@ -146,7 +146,7 @@ int btrfs_trim_block_group_extents(struct btrfs_block_group_cache *block_group, bool async);int btrfs_trim_block_group_bitmaps (struct btrfs_block_group_cache *block_group,u64 *trimmed,u64 start,u64 end,u64 minlen, -bool async); +u64 maxlen,bool async);/* Support functions for running our sanity tests */ #ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
Signature · 1 line
2.17.1
Re: [PATCH 17/22] btrfs: have multiple discard lists
Quoted 10 lines · Dennis Zhou
Non-block group destruction discarding currently only had a single list with no minimum discard length. This can lead to caravaning more meaningful discards behind a heavily fragmented block group.
This adds support for multiple lists with minimum discard lengths to prevent the caravan effect. We promote block groups back up when we exceed the
BTRFS_ASYNC_DISCARD_MAX_FILTERsize, currently we support only 2 lists with filters of 1MB and 32KB respectively.Signed-off-by Dennis Zhou <[email protected]>
Thanks,
Josef
Dennis Zhou
maintainerPATCH v2 18/22 btrfs: only keep track of data extents for async discard
In reply to Dennis ZhouAs mentioned earlier, discarding data can be done either by issuing an explicit discard or implicitly by reusing the LBA. Metadata chunks see much more frequent reuse due to well it being metadata. So instead of explicitly discarding metadata blocks, just leave them be and let the latter implicit discarding be done for them.
--- fs/btrfs/block-group.h | 6 ++++++ fs/btrfs/discard.c | 11 +++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-)
fs/btrfs/block-group.h +6 -0
@@ -181,6 +181,12 @@ static inline u64 btrfs_block_group_end(struct btrfs_block_group_cache *cache)return (cache->key.objectid + cache->key.offset); } +static inline bool btrfs_is_block_group_data ( +struct btrfs_block_group_cache *cache) +{ +return (cache->flags &BTRFS_BLOCK_GROUP_DATA ); +} +#ifdef CONFIG_BTRFS_DEBUG static inline int btrfs_should_fragment_free_space (struct btrfs_block_group_cache *block_group)
fs/btrfs/discard.c +9 -2
@@ -51,6 +51,9 @@ static void __btrfs_add_to_discard_list(struct btrfs_discard_ctl *discard_ctl,void btrfs_add_to_discard_list (struct btrfs_discard_ctl *discard_ctl,struct btrfs_block_group_cache *cache) { +if (!btrfs_is_block_group_data (cache)) +return ; +spin_lock (&discard_ctl->lock);__btrfs_add_to_discard_list (discard_ctl, cache);
@@ -161,7 +164,10 @@ static struct btrfs_block_group_cache *peek_discard_list(if (cache && now > cache->discard_eligible_time) {if (cache->discard_index ==BTRFS_DISCARD_INDEX_UNUSED &&btrfs_block_group_used (&cache->item) !=0 ) { -__btrfs_add_to_discard_list (discard_ctl, cache); +if (btrfs_is_block_group_data (cache)) +__btrfs_add_to_discard_list (discard_ctl, cache); +else +list_del_init (&cache->discard_list); goto again; }if (cache->discard_state ==BTRFS_DISCARD_RESET_CURSOR ) {
@@ -492,7 +498,8 @@ void btrfs_discard_update_discardable(struct btrfs_block_group_cache *cache,s32 extents_delta;s64 bytes_delta; -if (!cache || !btrfs_test_opt (cache->fs_info,DISCARD_ASYNC )) +if (!cache || !btrfs_test_opt (cache->fs_info,DISCARD_ASYNC ) || + !btrfs_is_block_group_data (cache))return ; discard_ctl = &cache->fs_info->discard_ctl;
Signature · 1 line
2.17.1
Re: [PATCH 18/22] btrfs: only keep track of data extents for async discard
Quoted 6 lines · Dennis Zhou
As mentioned earlier, discarding data can be done either by issuing an explicit discard or implicitly by reusing the LBA. Metadata chunks see much more frequent reuse due to well it being metadata. So instead of explicitly discarding metadata blocks, just leave them be and let the latter implicit discarding be done for them.
Hmm now that I look at this, it seems like we won't even discard empty metadata block groups now, right? Or am I missing something? Thanks,
Josef
Dennis Zhou
maintainerRe: [PATCH 18/22] btrfs: only keep track of data extents for async discard
Quoted 11 lines · Josef Bacik
Quoted 6 lines · Dennis Zhou
As mentioned earlier, discarding data can be done either by issuing an explicit discard or implicitly by reusing the LBA. Metadata chunks see much more frequent reuse due to well it being metadata. So instead of explicitly discarding metadata blocks, just leave them be and let the latter implicit discarding be done for them.
Hmm now that I look at this, it seems like we won't even discard empty metadata block groups now, right? Or am I missing something? Thanks,
Empty block groups go through btrfs_add_to_discard_unused_list() which skips that check. So metadata blocks will be discarded here from btrfs_discard_queue_work() which should be called from __btrfs_add_free_space().
We should just skip discarding metadata blocks while they are being used.
Thanks,
Dennis
Re: [PATCH 18/22] btrfs: only keep track of data extents for async discard
Quoted 13 lines · Dennis Zhou
Quoted 11 lines · Josef Bacik
Quoted 6 lines · Dennis Zhou
As mentioned earlier, discarding data can be done either by issuing an explicit discard or implicitly by reusing the LBA. Metadata chunks see much more frequent reuse due to well it being metadata. So instead of explicitly discarding metadata blocks, just leave them be and let the latter implicit discarding be done for them.
Hmm now that I look at this, it seems like we won't even discard empty metadata block groups now, right? Or am I missing something? Thanks,
Quoted 7 lines
Empty block groups go through
btrfs_add_to_discard_unused_list()which skips that check. So metadata blocks will be discarded here frombtrfs_discard_queue_work()which should be called from__btrfs_add_free_space().We should just skip discarding metadata blocks while they are being used.
I believe discarding metadata blocks while in use is called corruption :) But I understand what you mean.
Dennis Zhou
maintainerPATCH v2 19/22 btrfs: keep track of discard reuse stats
In reply to Dennis ZhouKeep track of how much we are discarding and how often we are reusing with async discard.
--- fs/btrfs/ctree.h | 3 +++ fs/btrfs/discard.c | 7 +++++++ fs/btrfs/free-space-cache.c | 14 ++++++++++++++ fs/btrfs/sysfs.c | 36 ++++++++++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+)
fs/btrfs/ctree.h +3 -0
@@ -472,6 +472,9 @@ struct btrfs_discard_ctl {u32 delay;u32 iops_limit;u64 bps_limit; +u64 discard_extent_bytes; +u64 discard_bitmap_bytes; +atomic64_t discard_bytes_saved; };/* delayed seq elem */
fs/btrfs/discard.c +7 -0
@@ -406,11 +406,15 @@ static void btrfs_discard_workfn(struct work_struct *work) cache->discard_cursor,btrfs_block_group_end (cache), minlen, maxlen, true); +WRITE_ONCE (discard_ctl->discard_bitmap_bytes, + discard_ctl->discard_bitmap_bytes + trimmed); }else {btrfs_trim_block_group_extents (cache, &trimmed, cache->discard_cursor,btrfs_block_group_end (cache), minlen, true); +WRITE_ONCE (discard_ctl->discard_extent_bytes, + discard_ctl->discard_extent_bytes + trimmed); } discard_ctl->prev_discard = trimmed;
@@ -614,6 +618,9 @@ void btrfs_discard_init(struct btrfs_fs_info *fs_info) discard_ctl->delay =BTRFS_DISCARD_MAX_DELAY ; discard_ctl->iops_limit =BTRFS_DISCARD_MAX_IOPS ; discard_ctl->bps_limit =0 ; + discard_ctl->discard_extent_bytes =0 ; + discard_ctl->discard_bitmap_bytes =0 ; +atomic64_set (&discard_ctl->discard_bytes_saved,0 ); }void btrfs_discard_cleanup (struct btrfs_fs_info *fs_info)
fs/btrfs/free-space-cache.c +14 -0
@@ -2817,6 +2817,8 @@ u64 btrfs_find_space_for_alloc(struct btrfs_block_group_cache *block_group, u64 *max_extent_size) {struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl; +struct btrfs_discard_ctl *discard_ctl = + &block_group->fs_info->discard_ctl;struct btrfs_free_space *entry =NULL ;u64 bytes_search = bytes + empty_size;u64 ret =0 ;
@@ -2833,6 +2835,10 @@ u64 btrfs_find_space_for_alloc(struct btrfs_block_group_cache *block_group, ret = offset;if (entry->bitmap) {bitmap_clear_bits (ctl, entry, offset, bytes); + +if (!btrfs_free_space_trimmed (entry)) +atomic64_add (bytes, &discard_ctl->discard_bytes_saved); +if (!entry->bytes)free_bitmap (ctl, entry); }else {
@@ -2841,6 +2847,9 @@ u64 btrfs_find_space_for_alloc(struct btrfs_block_group_cache *block_group, align_gap = entry->offset; align_gap_trim_state = entry->trim_state; +if (!btrfs_free_space_trimmed (entry)) +atomic64_add (bytes, &discard_ctl->discard_bytes_saved); + entry->offset = offset + bytes;WARN_ON (entry->bytes < bytes + align_gap_len);
@@ -2945,6 +2954,8 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group, u64 min_start, u64 *max_extent_size) { struct btrfs_free_space_ctl *ctl = block_group->free_space_ctl; +struct btrfs_discard_ctl *discard_ctl = + &block_group->fs_info->discard_ctl;struct btrfs_free_space *entry =NULL ;struct rb_node *node;u64 ret =0 ;
@@ -3009,6 +3020,9 @@ u64 btrfs_alloc_from_cluster(struct btrfs_block_group_cache *block_group,spin_lock (&ctl->tree_lock); +if (!btrfs_free_space_trimmed (entry)) +atomic64_add (bytes, &discard_ctl->discard_bytes_saved); + ctl->free_space -= bytes;if (!entry->bitmap && !btrfs_free_space_trimmed (entry)) ctl->discardable_bytes[BTRFS_STAT_CURR ] -= bytes;
fs/btrfs/sysfs.c +36 -0
@@ -431,12 +431,48 @@ static ssize_t btrfs_discard_max_discard_size_store(struct kobject *kobj,BTRFS_ATTR_RW (discard, max_discard_size, btrfs_discard_max_discard_size_show, btrfs_discard_max_discard_size_store); +static ssize_t btrfs_discard_extent_bytes_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%lld\n" , +READ_ONCE (fs_info->discard_ctl.discard_extent_bytes)); +} +BTRFS_ATTR (discard, discard_extent_bytes, btrfs_discard_extent_bytes_show); + +static ssize_t btrfs_discard_bitmap_bytes_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%lld\n" , +READ_ONCE (fs_info->discard_ctl.discard_bitmap_bytes)); +} +BTRFS_ATTR (discard, discard_bitmap_bytes, btrfs_discard_bitmap_bytes_show); + +static ssize_t btrfs_discard_bytes_saved_show (struct kobject *kobj, +struct kobj_attribute *a, +char *buf) +{ +struct btrfs_fs_info *fs_info =discard_to_fs_info (kobj); + +return snprintf (buf,PAGE_SIZE ,"%lld\n" , +atomic64_read (&fs_info->discard_ctl.discard_bytes_saved)); +} +BTRFS_ATTR (discard, discard_bytes_saved, btrfs_discard_bytes_saved_show); +static const struct attribute *discard_attrs[] = {BTRFS_ATTR_PTR (discard, discardable_extents),BTRFS_ATTR_PTR (discard, discardable_bytes),BTRFS_ATTR_PTR (discard, iops_limit),BTRFS_ATTR_PTR (discard, bps_limit),BTRFS_ATTR_PTR (discard, max_discard_size), +BTRFS_ATTR_PTR (discard, discard_extent_bytes), +BTRFS_ATTR_PTR (discard, discard_bitmap_bytes), +BTRFS_ATTR_PTR (discard, discard_bytes_saved),NULL , };
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 20/22 btrfs: add async discard header
In reply to Dennis ZhouGive a brief overview for how async discard is implemented.
--- fs/btrfs/discard.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
fs/btrfs/discard.c +34 -0
@@ -1,5 +1,39 @@/* * Copyright (C) 2019 Facebook. All rights reserved. +* +* This contains the logic to handle async discard. +* +* Async discard manages trimming of free space outside of transaction commit. +* Discarding is done by managing the block_groups on a LRU list based on free +* space recency. Two passes are used to first prioritize discarding extents +* and then allow for trimming in the bitmap the best opportunity to coalesce. +* The block_groups are maintained on multiple lists to allow for multiple +* passes with different discard filter requirements. A delayed work item is +* used to manage discarding with timeout determined by a max of the delay +* incurred by the iops rate limit, byte rate limit, and the timeout of max +* delay of BTRFS_DISCARD_MAX_DELAY. +* +* The first list is special to manage discarding of fully free block groups. +* This is necessary because we issue a final trim for a full free block group +* after forgetting it. When a block group becomes unused, instead of directly +* being added to the unused_bgs list, we add it to this first list. Then +* from there, if it becomes fully discarded, we place it onto the unused_bgs +* list. +* +* The in-memory free space cache serves as the backing state for discard. +* Consequently this means there is no persistence. We opt to load all the +* block groups in as not discarded, so the mount case degenerates to the +* crashing case. +* +* As the free space cache uses bitmaps, there exists a tradeoff between +* ease/efficiency for find_free_extent() and the accuracy of discard state. +* Here we opt to let untrimmed regions merge with everything while only letting +* trimmed regions merge with other trimmed regions. This can cause +* overtrimming, but the coalescing benefit seems to be worth it. Additionally, +* bitmap state is tracked as a whole. If we're able to fully trim a bitmap, +* the trimmed flag is set on the bitmap. Otherwise, if an allocation comes in, +* this resets the state and we will retry trimming the whole bitmap. This is a +* tradeoff between discard state accuracy and the cost of accounting. */ #include <linux/jiffies.h>
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 21/22 btrfs: increase the metadata allowance for the free_space_cache
In reply to Dennis ZhouCurrently, there is no way for the free space cache to recover from being serviced by purely bitmaps because the extent threshold is set to 0 in recalculate_thresholds() when we surpass the metadata allowance.
This adds a recovery mechanism by keeping large extents out of the bitmaps and increases the metadata upper bound to 64KB. The recovery mechanism bypasses this upper bound, thus making it a soft upper bound. But, with the bypass being 1MB or greater, it shouldn't add unbounded overhead.
--- fs/btrfs/free-space-cache.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-)
fs/btrfs/free-space-cache.c +11 -15
@@ -24,7 +24,8 @@#include "discard.h" #define BITS_PER_BITMAP (PAGE_SIZE * 8UL) -#define MAX_CACHE_BYTES_PER_GIG SZ_32K +#define MAX_CACHE_BYTES_PER_GIG SZ_64K +#define FORCE_EXTENT_THRESHOLD SZ_1Mstruct btrfs_trim_range {u64 start;
@@ -1691,26 +1692,17 @@ static void recalculate_thresholds(struct btrfs_free_space_ctl *ctl)ASSERT (ctl->total_bitmaps <= max_bitmaps);/* -* The goal is to keep the total amount of memory used per 1gb of space -* at or below 32k, so we need to adjust how much memory we allow to be -* used by extent based free space tracking +* We are trying to keep the total amount of memory used per 1gb of +* space to be MAX_CACHE_BYTES_PER_GIG. However, with a reclamation +* mechanism of pulling extents >= FORCE_EXTENT_THRESHOLD out of +* bitmaps, we may end up using more memory than this. */ if (size <SZ_1G ) max_bytes =MAX_CACHE_BYTES_PER_GIG ;else max_bytes =MAX_CACHE_BYTES_PER_GIG *div_u64 (size,SZ_1G ); -/* -* we want to account for 1 more bitmap than what we have so we can make -* sure we don't go over our overall goal of MAX_CACHE_BYTES_PER_GIG as -* we add more bitmaps. -*/ - bitmap_bytes = (ctl->total_bitmaps +1 ) * ctl->unit; - -if (bitmap_bytes >= max_bytes) { - ctl->extents_thresh =0 ; -return ; - } + bitmap_bytes = ctl->total_bitmaps * ctl->unit;/* * we want the extent entry threshold to always be at most 1/2 the max
@@ -2096,6 +2088,10 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl, forced = true;#endif +/* This is a way to reclaim large regions from the bitmaps. */ +if (!forced && info->bytes >=FORCE_EXTENT_THRESHOLD ) +return false; +/* * If we are below the extents threshold then we can add this as an * extent, and don't have to deal with the bitmap
Signature · 1 line
2.17.1
Dennis Zhou
maintainerPATCH v2 22/22 btrfs: make smaller extents more likely to go into bitmaps
In reply to Dennis ZhouIt's less than ideal for small extents to eat into our extent budget, so force extents <= 32KB into the bitmaps save for the first handful.
--- fs/btrfs/free-space-cache.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/free-space-cache.c +2 -2
@@ -2104,8 +2104,8 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl,* of cache left then go ahead an dadd them, no sense in adding * the overhead of a bitmap if we don't have to. */ -if (info->bytes <= fs_info->sectorsize *4 ) { -if (ctl->free_extents *2 <= ctl->extents_thresh) +if (info->bytes <= fs_info->sectorsize *8 ) { +if (ctl->free_extents *3 <= ctl->extents_thresh)return false; }else {return false;
Signature · 1 line
2.17.1
Quoted 25 lines · Dennis Zhou
Data:
On a number of webservers, I collected data every minute accounting the time we spent inbtrfs_finish_extent_commit()(col. 1) and inbtrfs_commit_transaction()(col. 2).btrfs_finish_extent_commit()is where we discard extents synchronously before returning them to the free space cache.discard=sync:
p99 total per minute p99 total per minute
Drive |extent_commit()(ms) |commit_trans()(ms)
---------------------------------------------------------------
Drive A | 434 | 1170
Drive B | 880 | 2330
Drive C | 2943 | 3920
Drive D | 4763 | 5701discard=async:
p99 total per minute p99 total per minute
Drive |extent_commit()(ms) |commit_trans()(ms)
--------------------------------------------------------------
Drive A | 134 | 956
Drive B | 64 | 1972
Drive C | 59 | 1032
Drive D | 62 | 1200
Quoted 4 lines
While it's not great that the stats are cumulative over 1m, all of these servers are running the same workload and and the delta between the two are substantial. We are spending significantly less time in
btrfs_finish_extent_commit()which is responsible for discarding.
Nice, thanks for getting the numbers. I think we can slap them to the patch that introduceds async discard which is 5/22.
I went through the patchset, looks good, though I'm sure I missed some small things, it's a lot of code. I'm going to add it first thing to 5.6 patch queue. There are some minor conflicts with patches in 5.5 queue, so this will need to be sorted. I've seen some feedback to address, it would be IMHO best to resend the whole series. Thanks.