btrfs: fixes and cleanups around extent maps
cover btrfs: fixes and cleanups around extent maps
From: Filipe Manana <[email protected]>
The following patchset fixes a bug related to dropping extent maps that can make an fsync miss a new extent, does several cleanups and some small performance improvements when dropping and searching for extent maps as well as when flushing delalloc in COW mode. These came out while working on some upcoming changes for fiemap, but since they are really independent, I'm sending them as a separate patchset.
The last patch in the series has a test and results in its changelog.
Filipe Manana (13):
btrfs: fix missed extent on fsync after dropping extent maps
btrfs: move btrfs_drop_extent_cache() to extent_map.c
btrfs: use extent_map_end() at btrfs_drop_extent_map_range()
btrfs: use cond_resched_rwlock_write() during inode eviction
btrfs: move open coded extent map tree deletion out of inode eviction
btrfs: add helper to replace extent map range with a new extent map
btrfs: remove the refcount warning/check at free_extent_map()
btrfs: remove unnecessary extent map initializations
btrfs: assert tree is locked when clearing extent map from logging
btrfs: remove unnecessary NULL pointer checks when searching extent maps
btrfs: remove unnecessary next extent map search
btrfs: avoid pointless extent map tree search when flushing delalloc
btrfs: drop extent map range more efficiently
fs/btrfs/ctree.h | 2 - fs/btrfs/extent_map.c | 343 ++++++++++++++++++++++++++++++++--- fs/btrfs/extent_map.h | 8 + fs/btrfs/file.c | 184 ++----------------- fs/btrfs/free-space-cache.c | 2 +- fs/btrfs/inode.c | 101 +++-------- fs/btrfs/relocation.c | 20 +- fs/btrfs/tests/inode-tests.c | 2 +- 8 files changed, 373 insertions(+), 289 deletions(-)
Signature · 1 line
2.35.1
PATCH 1/13 btrfs: fix missed extent on fsync after dropping extent maps
From: Filipe Manana <[email protected]>
When dropping extent maps for a range, through btrfs_drop_extent_cache(), if we find an extent map that starts before our target range and/or ends before the target range, and we are not able to allocate extent maps for splitting that extent map, then we don't fail and simply remove the entire extent map from the inode's extent map tree.
This is generally fine, because in case anyone needs to access the extent map, it can just load it again later from the respective file extent item(s) in the subvolume btree. However, if that extent map is new and is in the list of modified extents, then a fast fsync will miss the parts of the extent that were outside our range (that needed to be split), therefore not logging them. Fix that by marking the inode for a full fsync. This issue was introduced after removing BUG_ON()s triggered when the split extent map allocations failed, done by commit 7014cdb49305ed ("Btrfs: btrfs_drop_extent_cache should never fail"), back in 2012, and the fast fsync path already existed but was very recent.
Also, in the case where we could allocate extent maps for the split operations but then fail to add a split extent map to the tree, mark the inode for a full fsync as well. This is not supposed to ever fail, and we assert that, but in case assertions are disabled (CONFIG_BTRFS_ASSERT is not set), it's the correct thing to do to make sure a fast fsync will not miss a new extent.
--- fs/btrfs/file.c | 58 +++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 46 insertions(+), 12 deletions(-)
fs/btrfs/file.c +46 -12
@@ -523,6 +523,7 @@ void btrfs_drop_extent_cache(struct btrfs_inode *inode, u64 start, u64 end, testend =0 ; }while (1 ) { +bool ends_after_range = false;int no_splits =0 ; modified = false;
@@ -539,10 +540,12 @@ void btrfs_drop_extent_cache(struct btrfs_inode *inode, u64 start, u64 end,write_unlock (&em_tree->lock);break ; } +if (testend && em->start + em->len > start + len) + ends_after_range = true; flags = em->flags; gen = em->generation;if (skip_pinned &&test_bit (EXTENT_FLAG_PINNED , &em->flags)) { -if (testend && em->start + em->len >= start + len) { +if (ends_after_range) {free_extent_map (em);write_unlock (&em_tree->lock);break ;
@@ -592,7 +595,7 @@ void btrfs_drop_extent_cache(struct btrfs_inode *inode, u64 start, u64 end, split = split2; split2 =NULL ; } -if (testend && em->start + em->len > start + len) { +if (ends_after_range) {u64 diff = start + len - em->start; split->start = start + len;
@@ -630,14 +633,42 @@ void btrfs_drop_extent_cache(struct btrfs_inode *inode, u64 start, u64 end, }else { ret =add_extent_mapping (em_tree, split, modified); -ASSERT (ret ==0 );/* Logic error */ +/* Logic error, shouldn't happen. */ +ASSERT (ret ==0 ); +if (WARN_ON (ret !=0 ) && modified) +btrfs_set_inode_full_sync (inode); }free_extent_map (split); split =NULL ; } next: -if (extent_map_in_tree (em)) +if (extent_map_in_tree (em)) { +/* +* If the extent map is still in the tree it means that +* either of the following is true: +* +* 1) It fits entirely in our range (doesn't end beyond +* it or starts before it); +* +* 2) It starts before our range and/or ends after our +* range, and we were not able to allocate the extent +* maps for split operations, @split and @split2. +* +* If we are at case 2) then we just remove the entire +* extent map - this is fine since if anyone needs it to +* access the subranges outside our range, will just +* load it again from the subvolume tree's file extent +* item. However if the extent map was in the list of +* modified extents, then we must mark the inode for a +* full fsync, otherwise a fast fsync will miss this +* extent if it's new and needs to be logged. +*/ +if ((em->start < start || ends_after_range) && modified) { +ASSERT (no_splits); +btrfs_set_inode_full_sync (inode); + }remove_extent_mapping (em_tree, em); + }write_unlock (&em_tree->lock);/* once for us */
@@ -2199,14 +2230,6 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)atomic_inc (&root->log_batch); -/* -* Always check for the full sync flag while holding the inode's lock, -* to avoid races with other tasks. The flag must be either set all the -* time during logging or always off all the time while logging. -*/ - full_sync =test_bit (BTRFS_INODE_NEEDS_FULL_SYNC , - &BTRFS_I (inode)->runtime_flags); -/* * Before we acquired the inode's lock and the mmap lock, someone may * have dirtied more pages in the target range. We need to make sure
@@ -2231,6 +2254,17 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out; } +/* +* Always check for the full sync flag while holding the inode's lock, +* to avoid races with other tasks. The flag must be either set all the +* time during logging or always off all the time while logging. +* We check the flag here after starting delalloc above, because when +* running delalloc the full sync flag may be set if we need to drop +* extra extent map ranges due to temporary memory allocation failures. +*/ + full_sync =test_bit (BTRFS_INODE_NEEDS_FULL_SYNC , + &BTRFS_I (inode)->runtime_flags); +/* * We have to do this here to avoid the priority inversion of waiting on * IO of a lower priority task while holding a transaction open.
Signature · 1 line
2.35.1
PATCH 3/13 btrfs: use extent_map_end() at btrfs_drop_extent_map_range()
In reply to [email protected]From: Filipe Manana <[email protected]>
Instead of open coding the end offset calculation of an extent map, use the helper extent_map_end() and cache its result in a local variable, since it's used several times.
--- fs/btrfs/extent_map.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
fs/btrfs/extent_map.c +6 -4
@@ -690,6 +690,7 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end, }while (1 ) {struct extent_map *em; +u64 em_end;u64 gen;unsigned long flags;bool ends_after_range = false;
@@ -710,7 +711,8 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,write_unlock (&em_tree->lock);break ; } -if (testend && em->start + em->len > start + len) + em_end =extent_map_end (em); +if (testend && em_end > start + len) ends_after_range = true;if (skip_pinned &&test_bit (EXTENT_FLAG_PINNED , &em->flags)) {if (ends_after_range) {
@@ -718,9 +720,9 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,write_unlock (&em_tree->lock);break ; } - start = em->start + em->len; + start = em_end;if (testend) - len = start + len - (em->start + em->len); + len = start + len - em_end;free_extent_map (em);write_unlock (&em_tree->lock);continue ;
@@ -767,7 +769,7 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end, }if (ends_after_range) { split->start = start + len; - split->len = em->start + em->len - (start + len); + split->len = em_end - (start + len); split->block_start = em->block_start; split->flags = flags; split->compress_type = em->compress_type;
Signature · 1 line
2.35.1
PATCH 2/13 btrfs: move btrfs_drop_extent_cache() to extent_map.c
In reply to [email protected]From: Filipe Manana <[email protected]>
The function btrfs_drop_extent_cache() doesn't really belong at file.c because what it does is drop a range of extent maps for a file range. It directly allocates and manipulates extent maps, by dropping, spliting and replacing them in an extent map tree, so it should be located at extent_map.c, where all manipulations of an extent map tree and its extent maps are supposed to be done.
So move it out of file.c and into extent_map.c. Additionally do the following changes:
1) Rename it into btrfs_drop_extent_map_range(), as this makes it more clear about what it does. The term "cache" is a bit confusing as it's not widely used, "extent maps" or "extent mapping" is much more common;
2) Change its 'skip_pinned' argument from int to bool;
3) Turn several of its local variables from int to bool, since they are used as booleans;
4) Move the declaration of some variables out of the function's main scope and into the scopes where they are used;
5) Remove pointless assignment of false to 'modified' early in the while loop, as later that variable is set and it's not used before that second assignment;
6) Remove checks for NULL before calling free_extent_map().
--- fs/btrfs/ctree.h | 2 - fs/btrfs/extent_map.c | 192 +++++++++++++++++++++++++++++++++++ fs/btrfs/extent_map.h | 5 + fs/btrfs/file.c | 190 +--------------------------------- fs/btrfs/free-space-cache.c | 2 +- fs/btrfs/inode.c | 59 ++++++----- fs/btrfs/relocation.c | 8 +- fs/btrfs/tests/inode-tests.c | 2 +- 8 files changed, 237 insertions(+), 223 deletions(-)
fs/btrfs/ctree.h +0 -2
@@ -3534,8 +3534,6 @@ int btrfs_add_inode_defrag(struct btrfs_trans_handle *trans,int btrfs_run_defrag_inodes (struct btrfs_fs_info *fs_info);void btrfs_cleanup_defrag_inodes (struct btrfs_fs_info *fs_info);int btrfs_sync_file (struct file *file,loff_t start,loff_t end,int datasync); -void btrfs_drop_extent_cache (struct btrfs_inode *inode,u64 start,u64 end, -int skip_pinned);extern const struct file_operations btrfs_file_operations; int btrfs_drop_extents(struct btrfs_trans_handle *trans, struct btrfs_root *root, struct btrfs_inode *inode,
fs/btrfs/extent_map.c +192 -0
201 lines not on this page · raw message
fs/btrfs/extent_map.h +5 -0
@@ -63,6 +63,8 @@ struct extent_map_tree {rwlock_t lock; }; +struct btrfs_inode ; +static inline int extent_map_in_tree (const struct extent_map *em) {return !RB_EMPTY_NODE (&em->rb_node);
@@ -104,5 +106,8 @@ struct extent_map *search_extent_mapping(struct extent_map_tree *tree,int btrfs_add_extent_mapping (struct btrfs_fs_info *fs_info,struct extent_map_tree *em_tree,struct extent_map **em_in,u64 start,u64 len); +void btrfs_drop_extent_map_range (struct btrfs_inode *inode, +u64 start,u64 end, +bool skip_pinned);#endif
fs/btrfs/file.c +3 -187
214 lines not on this page · raw message
fs/btrfs/free-space-cache.c +1 -1
@@ -349,7 +349,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,truncate_pagecache (vfs_inode,0 );lock_extent (&inode->io_tree,0 , (u64)-1 , &cached_state); -btrfs_drop_extent_cache (inode,0 , (u64)-1 ,0 ); +btrfs_drop_extent_map_range (inode,0 , (u64)-1 , false);/* * We skip the throttling logic for free space cache inodes, so we don't
fs/btrfs/inode.c +31 -28
155 lines not on this page · raw message
fs/btrfs/relocation.c +4 -4
@@ -1124,8 +1124,8 @@ int replace_file_extents(struct btrfs_trans_handle *trans,if (!ret)continue ; -btrfs_drop_extent_cache (BTRFS_I (inode), - key.offset, end,1 ); +btrfs_drop_extent_map_range (BTRFS_I (inode), + key.offset, end, true);unlock_extent (&BTRFS_I (inode)->io_tree, key.offset, end,NULL ); }
@@ -1567,7 +1567,7 @@ static int invalidate_extent_cache(struct btrfs_root *root,/* the lock_extent waits for read_folio to complete */ lock_extent (&BTRFS_I (inode)->io_tree, start, end,NULL ); -btrfs_drop_extent_cache (BTRFS_I (inode), start, end,1 ); +btrfs_drop_extent_map_range (BTRFS_I (inode), start, end, true);unlock_extent (&BTRFS_I (inode)->io_tree, start, end,NULL ); }return 0 ;
@@ -2913,7 +2913,7 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inodfree_extent_map (em);break ; } -btrfs_drop_extent_cache (BTRFS_I (inode), start, end,0 ); +btrfs_drop_extent_map_range (BTRFS_I (inode), start, end, false); }unlock_extent (&BTRFS_I (inode)->io_tree, start, end,NULL );return ret;
fs/btrfs/tests/inode-tests.c +1 -1
@@ -267,7 +267,7 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize) goto out; }free_extent_map (em); -btrfs_drop_extent_cache (BTRFS_I (inode),0 , (u64)-1 ,0 ); +btrfs_drop_extent_map_range (BTRFS_I (inode),0 , (u64)-1 , false);/* * All of the magic numbers are based on the mapping setup in
Signature · 1 line
2.35.1
PATCH 4/13 btrfs: use cond_resched_rwlock_write() during inode eviction
In reply to [email protected]From: Filipe Manana <[email protected]>
At evict_inode_truncate_pages(), instead of manually checking if rescheduling is needed, then unlock the extent map tree, reschedule and then write lock again the tree, use the helper cond_resched_rwlock_write() which does all that.
--- fs/btrfs/inode.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
fs/btrfs/inode.c +1 -5
@@ -5295,11 +5295,7 @@ static void evict_inode_truncate_pages(struct inode *inode)clear_bit (EXTENT_FLAG_LOGGING , &em->flags);remove_extent_mapping (map_tree, em);free_extent_map (em); -if (need_resched ()) { -write_unlock (&map_tree->lock); -cond_resched (); -write_lock (&map_tree->lock); - } +cond_resched_rwlock_write (&map_tree->lock); }write_unlock (&map_tree->lock);
Signature · 1 line
2.35.1
PATCH 5/13 btrfs: move open coded extent map tree deletion out of inode eviction
In reply to [email protected]From: Filipe Manana <[email protected]>
Move the loop that removes all the extent maps from the inode's extent map tree during inode eviction out of inode.c and into extent_map.c, to btrfs_drop_extent_map_range(). Anything manipulating extent maps or the extent map tree should be in extent_map.c.
--- fs/btrfs/extent_map.c | 27 +++++++++++++++++++++++++++ fs/btrfs/inode.c | 15 +-------------- 2 files changed, 28 insertions(+), 14 deletions(-)
fs/btrfs/extent_map.c +27 -0
@@ -660,6 +660,29 @@ int btrfs_add_extent_mapping(struct btrfs_fs_info *fs_info,return ret; } +/* +* Drop all extent maps from a tree in the fastest possible way, rescheduling +* if needed. This avoids searching the tree, from the root down to the first +* extent map, before each deletion. +*/ +static void drop_all_extent_maps_fast (struct extent_map_tree *tree) +{ +write_lock (&tree->lock); +while (!RB_EMPTY_ROOT (&tree->map.rb_root)) { +struct extent_map *em; +struct rb_node *node; + + node =rb_first_cached (&tree->map); + em = rb_entry(node, struct extent_map, rb_node); +clear_bit (EXTENT_FLAG_PINNED , &em->flags); +clear_bit (EXTENT_FLAG_LOGGING , &em->flags); +remove_extent_mapping (tree, em); +free_extent_map (em); +cond_resched_rwlock_write (&tree->lock); + } +write_unlock (&tree->lock); +} +/* * Drop all extent maps in a given range. *
@@ -685,6 +708,10 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,WARN_ON (end < start);if (end == (u64)-1 ) { +if (start ==0 && !skip_pinned) { +drop_all_extent_maps_fast (em_tree); +return ; + } len = (u64)-1 ; testend = false; }
fs/btrfs/inode.c +1 -14
@@ -5279,25 +5279,12 @@ static int btrfs_setattr(struct user_namespace *mnt_userns, struct dentry *dentrstatic void evict_inode_truncate_pages (struct inode *inode) {struct extent_io_tree *io_tree = &BTRFS_I (inode)->io_tree; -struct extent_map_tree *map_tree = &BTRFS_I (inode)->extent_tree;struct rb_node *node;ASSERT (inode->i_state &I_FREEING );truncate_inode_pages_final (&inode->i_data); -write_lock (&map_tree->lock); -while (!RB_EMPTY_ROOT (&map_tree->map.rb_root)) { -struct extent_map *em; - - node =rb_first_cached (&map_tree->map); - em = rb_entry(node, struct extent_map, rb_node); -clear_bit (EXTENT_FLAG_PINNED , &em->flags); -clear_bit (EXTENT_FLAG_LOGGING , &em->flags); -remove_extent_mapping (map_tree, em); -free_extent_map (em); -cond_resched_rwlock_write (&map_tree->lock); - } -write_unlock (&map_tree->lock); +btrfs_drop_extent_map_range (BTRFS_I (inode),0 , (u64)-1 , false);/* * Keep looping until we have no more ranges in the io tree.
Signature · 1 line
2.35.1
PATCH 7/13 btrfs: remove the refcount warning/check at free_extent_map()
In reply to [email protected]From: Filipe Manana <[email protected]>
At free_extent_map(), it's pointless to have a WARN_ON() to check if the refcount of the extent map is zero. Such check is already done by the refcount_t module and refcount_dec_and_test(), which loudly complains if we try to decrement a reference count that is currently 0.
The WARN_ON() dates back to the time when used a regular atomic_t type for the reference counter, before we switched to the refcount_t type. The main goal of the refcount_t type/module is precisely to catch such types of bugs and loudly complain if they happen.
--- fs/btrfs/extent_map.c | 1 - 1 file changed, 1 deletion(-)
fs/btrfs/extent_map.c +0 -1
@@ -74,7 +74,6 @@ void free_extent_map(struct extent_map *em) {if (!em)return ; -WARN_ON (refcount_read (&em->refs) ==0 );if (refcount_dec_and_test (&em->refs)) {WARN_ON (extent_map_in_tree (em));WARN_ON (!list_empty (&em->list));
Signature · 1 line
2.35.1
PATCH 6/13 btrfs: add helper to replace extent map range with a new extent map
In reply to [email protected]From: Filipe Manana <[email protected]>
We have several places that need to drop all the extent maps in a given file range and then add a new extent map for that range. Currently they call btrfs_drop_extent_map_range() to delete all extent maps in the range and then keep trying to add the new extent map in a loop that keeps retrying while the insertion of the new extent map fails with -EEXIST.
So instead of repeating this logic, add a helper to extent_map.c that does these steps and name it btrfs_replace_extent_map_range(). Also add a comment about why the retry loop is necessary.
--- fs/btrfs/extent_map.c | 41 +++++++++++++++++++++++++++++++++++ fs/btrfs/extent_map.h | 3 +++ fs/btrfs/file.c | 8 +------ fs/btrfs/inode.c | 50 +++++++------------------------------------ fs/btrfs/relocation.c | 14 +++--------- 5 files changed, 56 insertions(+), 60 deletions(-)
fs/btrfs/extent_map.c +41 -0
@@ -879,3 +879,44 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,free_extent_map (split);free_extent_map (split2); } + +/* +* Replace a range in the inode's extent map tree with a new extent map. +* +* @inode: The target inode. +* @new_em: The new extent map to add to the inode's extent map tree. +* @modified: Indicate if the new extent map should be added to the list of +* modified extents (for fast fsync tracking). +* +* Drops all the extent maps in the inode's extent map tree that intersect the +* range of the new extent map and adds the new extent map to the tree. +* The caller should have locked an appropriate file range in the inode's io +* tree before calling this function. +*/ +int btrfs_replace_extent_map_range (struct btrfs_inode *inode, +struct extent_map *new_em, +bool modified) +{ +const u64 end = new_em->start + new_em->len -1 ; +struct extent_map_tree *tree = &inode->extent_tree; +int ret; + +ASSERT (!extent_map_in_tree (new_em)); + +/* +* The caller has locked an appropriate file range in the inode's io +* tree, but getting -EEXIST when adding the new extent map can still +* happen in case there are extents that partially cover the range, and +* this is due to two tasks operating on different parts of the extent. +* See commit 18e83ac75bfe67 ("Btrfs: fix unexpected EEXIST from +* btrfs_get_extent") for an example and details. +*/ +do { +btrfs_drop_extent_map_range (inode, new_em->start, end, false); +write_lock (&tree->lock); + ret =add_extent_mapping (tree, new_em, modified); +write_unlock (&tree->lock); + }while (ret == -EEXIST ); + +return ret; +}
fs/btrfs/extent_map.h +3 -0
@@ -109,5 +109,8 @@ int btrfs_add_extent_mapping(struct btrfs_fs_info *fs_info,void btrfs_drop_extent_map_range (struct btrfs_inode *inode,u64 start,u64 end,bool skip_pinned); +int btrfs_replace_extent_map_range (struct btrfs_inode *inode, +struct extent_map *new_em, +bool modified);#endif
fs/btrfs/file.c +1 -7
@@ -2298,7 +2298,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,struct extent_buffer *leaf;struct btrfs_file_extent_item *fi;struct extent_map *hole_em; -struct extent_map_tree *em_tree = &inode->extent_tree;struct btrfs_key key;int ret;
@@ -2379,12 +2378,7 @@ static int fill_holes(struct btrfs_trans_handle *trans, hole_em->compress_type =BTRFS_COMPRESS_NONE ; hole_em->generation = trans->transid; -do { -btrfs_drop_extent_map_range (inode, offset, end -1 , false); -write_lock (&em_tree->lock); - ret =add_extent_mapping (em_tree, hole_em,1 ); -write_unlock (&em_tree->lock); - }while (ret == -EEXIST ); + ret =btrfs_replace_extent_map_range (inode, hole_em, true);free_extent_map (hole_em);if (ret)btrfs_set_inode_full_sync (inode);
fs/btrfs/inode.c +8 -42
108 lines not on this page · raw message
fs/btrfs/relocation.c +3 -11
@@ -2890,7 +2890,6 @@ static noinline_for_stack int prealloc_file_extent_cluster( static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inode,u64 start,u64 end,u64 block_start) { -struct extent_map_tree *em_tree = &BTRFS_I (inode)->extent_tree;struct extent_map *em;int ret =0 ;
@@ -2905,17 +2904,10 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inodset_bit (EXTENT_FLAG_PINNED , &em->flags);lock_extent (&BTRFS_I (inode)->io_tree, start, end,NULL ); -while (1 ) { -write_lock (&em_tree->lock); - ret =add_extent_mapping (em_tree, em,0 ); -write_unlock (&em_tree->lock); -if (ret != -EEXIST ) { -free_extent_map (em); -break ; - } -btrfs_drop_extent_map_range (BTRFS_I (inode), start, end, false); - } + ret =btrfs_replace_extent_map_range (BTRFS_I (inode), em, false);unlock_extent (&BTRFS_I (inode)->io_tree, start, end,NULL ); +free_extent_map (em); +return ret; }
Signature · 1 line
2.35.1
PATCH 8/13 btrfs: remove unnecessary extent map initializations
In reply to [email protected]From: Filipe Manana <[email protected]>
When allocating an extent map, we use kmem_cache_zalloc() which guarantees the returned memory is initialized to zeroes, therefore it's pointless to initialize the generation and flags of the extent map to zero again.
Remove those initializations, as they are pointless and slightly increase the object text size.
Before removing them:
$ size fs/btrfs/extent_map.o
text data bss dec hex filename
9241 274 24 9539 2543 fs/btrfs/extent_map.oAfter removing them:
$ size fs/btrfs/extent_map.o
text data bss dec hex filename
9209 274 24 9507 2523 fs/btrfs/extent_map.o--- fs/btrfs/extent_map.c | 2 -- 1 file changed, 2 deletions(-)
fs/btrfs/extent_map.c +0 -2
@@ -55,9 +55,7 @@ struct extent_map *alloc_extent_map(void)if (!em)return NULL ;RB_CLEAR_NODE (&em->rb_node); - em->flags =0 ; em->compress_type =BTRFS_COMPRESS_NONE ; - em->generation =0 ;refcount_set (&em->refs,1 );INIT_LIST_HEAD (&em->list);return em;
Signature · 1 line
2.35.1
PATCH 9/13 btrfs: assert tree is locked when clearing extent map from logging
In reply to [email protected]From: Filipe Manana <[email protected]>
When calling clear_em_logging() we should have a write lock on the extent map tree, as we will try to merge the extent map with the previous and next ones in the tree. So assert that we have a write lock.
--- fs/btrfs/extent_map.c | 2 ++ 1 file changed, 2 insertions(+)
fs/btrfs/extent_map.c +2 -0
@@ -334,6 +334,8 @@ int unpin_extent_cache(struct extent_map_tree *tree, u64 start, u64 len,void clear_em_logging (struct extent_map_tree *tree,struct extent_map *em) { +lockdep_assert_held_write (&tree->lock); +clear_bit (EXTENT_FLAG_LOGGING , &em->flags);if (extent_map_in_tree (em))try_merge_map (tree, em);
Signature · 1 line
2.35.1
PATCH 10/13 btrfs: remove unnecessary NULL pointer checks when searching extent maps
In reply to [email protected]From: Filipe Manana <[email protected]>
The previous and next pointer arguments passed to __tree_search() are never NULL as the only caller of this function, __lookup_extent_mapping(), always passes the address of two on stack pointers. So remove the NULL checks.
--- fs/btrfs/extent_map.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-)
fs/btrfs/extent_map.c +11 -14
@@ -163,24 +163,21 @@ static struct rb_node *__tree_search(struct rb_root *root, u64 offset,return n; } -if (prev_ret) { - orig_prev = prev; -while (prev && offset >=extent_map_end (prev_entry)) { - prev =rb_next (prev); - prev_entry = rb_entry(prev, struct extent_map, rb_node); - } - *prev_ret = prev; - prev = orig_prev; + orig_prev = prev; +while (prev && offset >=extent_map_end (prev_entry)) { + prev =rb_next (prev); + prev_entry = rb_entry(prev, struct extent_map, rb_node); } + *prev_ret = prev; + prev = orig_prev; -if (next_ret) { + prev_entry = rb_entry(prev, struct extent_map, rb_node); +while (prev && offset < prev_entry->start) { + prev =rb_prev (prev); prev_entry = rb_entry(prev, struct extent_map, rb_node); -while (prev && offset < prev_entry->start) { - prev =rb_prev (prev); - prev_entry = rb_entry(prev, struct extent_map, rb_node); - } - *next_ret = prev; } + *next_ret = prev; +return NULL ; }
Signature · 1 line
2.35.1
PATCH 12/13 btrfs: avoid pointless extent map tree search when flushing delalloc
In reply to [email protected]From: Filipe Manana <[email protected]>
When flushing delalloc, in COW mode at cow_file_range(), before entering the loop that allocates extents and creates ordered extents, we do a call to btrfs_drop_extent_map_range() for the whole range. This is pointless because in the loop we call create_io_em(), which will also call btrfs_drop_extent_map_range() before inserting the new extent map.
So remove that call at cow_file_range() not only because it is not needed, but also because it will make the btrfs_drop_extent_map_range() calls made from create_io_em() waste time searching the extent map tree, and that tree can be large for files with many extents. It also makes us waste time at btrfs_drop_extent_map_range() allocating and freeing the split extent maps for nothing.
--- fs/btrfs/inode.c | 1 - 1 file changed, 1 deletion(-)
fs/btrfs/inode.c +0 -1
@@ -1254,7 +1254,6 @@ static noinline int cow_file_range(struct btrfs_inode *inode, } alloc_hint =get_extent_allocation_hint (inode, start, num_bytes); -btrfs_drop_extent_map_range (inode, start, start + num_bytes -1 , false);/* * Relocation relies on the relocated extents to have exactly the same
Signature · 1 line
2.35.1
PATCH 11/13 btrfs: remove unnecessary next extent map search
In reply to [email protected]From: Filipe Manana <[email protected]>
At __tree_search(), and its single caller __lookup_extent_mapping(), there is no point in finding the next extent map that starts after the search offset if we were able to find the previous extent map that ends before our search offset, because __lookup_extent_mapping() ignores the next acceptable extent map if we were able to find the previous one.
So just return immediately if we were able to find the previous extent map, therefore avoiding wasting time iterating the tree looking for the next extent map which will not be used by __lookup_extent_mapping().
--- fs/btrfs/extent_map.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-)
fs/btrfs/extent_map.c +16 -12
@@ -141,8 +141,7 @@ static int tree_insert(struct rb_root_cached *root, struct extent_map *em)* it can't be found, try to find some neighboring extents */ static struct rb_node *__tree_search (struct rb_root *root,u64 offset, -struct rb_node **prev_ret, -struct rb_node **next_ret) +struct rb_node **prev_or_next_ret) {struct rb_node *n = root->rb_node;struct rb_node *prev =NULL ;
@@ -168,15 +167,23 @@ static struct rb_node *__tree_search(struct rb_root *root, u64 offset, prev =rb_next (prev); prev_entry = rb_entry(prev, struct extent_map, rb_node); } - *prev_ret = prev; - prev = orig_prev; +/* +* Previous extent map found, return as in this case the caller does not +* care about the next one. +*/ +if (prev) { + *prev_or_next_ret = prev; +return NULL ; + } + + prev = orig_prev; prev_entry = rb_entry(prev, struct extent_map, rb_node);while (prev && offset < prev_entry->start) { prev =rb_prev (prev); prev_entry = rb_entry(prev, struct extent_map, rb_node); } - *next_ret = prev; + *prev_or_next_ret = prev;return NULL ; }
@@ -422,16 +429,13 @@ __lookup_extent_mapping(struct extent_map_tree *tree, {struct extent_map *em;struct rb_node *rb_node; -struct rb_node *prev =NULL ; -struct rb_node *next =NULL ; +struct rb_node *prev_or_next =NULL ;u64 end =range_end (start, len); - rb_node =__tree_search (&tree->map.rb_root, start, &prev, &next); + rb_node =__tree_search (&tree->map.rb_root, start, &prev_or_next);if (!rb_node) { -if (prev) - rb_node = prev; -else if (next) - rb_node = next; +if (prev_or_next) + rb_node = prev_or_next;else return NULL ; }
Signature · 1 line
2.35.1
PATCH 13/13 btrfs: drop extent map range more efficiently
In reply to [email protected]From: Filipe Manana <[email protected]>
Currently when dropping extent maps for a file range, through btrfs_drop_extent_map_range(), we do the following non-optimal things:
1) We lookup for extent maps one by one, always starting the search from the root of the extent map tree. This is not efficient if we have multiple extent maps in the range;
2) We check on every iteration if we have the 'split' and 'split2' spare extent maps in case we need to split an extent map that intersects our range but also crosses its boundaries (to the left, to the right or both cases). If our target range is for example:
[2M, 8M)
And we have 3 extents maps in the range:
[1M, 3M) [3M, 6M) [6M, 10M[
The on the first iteration we allocate two extent maps for 'split' and 'split2', and use the 'split' to split the first extent map, so after the split we set 'split' to 'split2' and then set 'split2' to NULL.
On the second iteration, we don't need to split the second extent map, but because 'split2' is now NULL, we allocate a new extent map for 'split2'.
On the third iteration we need to split the third extent map, so we use the extent map pointed by 'split'.
So we ended up allocating 3 extent maps for splitting, but all we needed was 2 extent maps. We never need to allocate more than 2, because extent maps that need to be split are always the first one and the last one in the target range.
Improve on this by:
1) Using rb_next() to move on to the next extent map. This results in iterating over less nodes of the tree and it does not require comparing the ranges of nodes to our start/end offset;
2) Allocate the 2 extent maps for splitting before entering the loop and never allocate more than 2. In practice it's very rare to have the combination of both extent map allocations fail, since we have a dedicated slab for extent maps, and also have the need to split two extent maps.
This patch is part of a patchset comprised of the following patches:
btrfs: fix missed extent on fsync after dropping extent maps
btrfs: move btrfs_drop_extent_cache() to extent_map.c
btrfs: use extent_map_end() at btrfs_drop_extent_map_range()
btrfs: use cond_resched_rwlock_write() during inode eviction
btrfs: move open coded extent map tree deletion out of inode eviction
btrfs: add helper to replace extent map range with a new extent map
btrfs: remove the refcount warning/check at free_extent_map()
btrfs: remove unnecessary extent map initializations
btrfs: assert tree is locked when clearing extent map from logging
btrfs: remove unnecessary NULL pointer checks when searching extent maps
btrfs: remove unnecessary next extent map search
btrfs: avoid pointless extent map tree search when flushing delalloc
btrfs: drop extent map range more efficiently
And the following fio test was done before and after applying the whole patchset, on a non-debug kernel (Debian's default kernel config) on a 12 cores Intel box with 64G of ram:
$ cat test.sh #!/bin/bash
DEV=/dev/nvme0n1
MNT=/mnt/nvme0n1
MOUNT_OPTIONS="-o ssd"
MKFS_OPTIONS="-R free-space-tree -O no-holes"
cat <<EOF > /tmp/fio-job.ini
[writers]
rw=randwrite
fsync=8
fallocate=none
group_reporting=1
direct=0 bssplit=4k/20:8k/20:16k/20:32k/10:64k/10:128k/5:256k/5:512k/5:1m/5 ioengine=psync
filesize=2G
runtime=300
time_based
directory=$MNT
numjobs=8
thread
EOF
echo performance | \
tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor echo
echo "Using config:"
echo
cat /tmp/fio-job.ini
echo
umount $MNT &> /dev/null
mkfs.btrfs -f $MKFS_OPTIONS $DEV
mount $MOUNT_OPTIONS $DEV $MNT
fio /tmp/fio-job.ini
umount $MNT
Result before applying the patchset:
WRITE: bw=197MiB/s (206MB/s), 197MiB/s-197MiB/s (206MB/s-206MB/s), io=57.7GiB (61.9GB), run=300188-300188msec
Result after applying the patchset:
WRITE: bw=203MiB/s (213MB/s), 203MiB/s-203MiB/s (213MB/s-213MB/s), io=59.5GiB (63.9GB), run=300019-300019msec
--- fs/btrfs/extent_map.c | 119 ++++++++++++++++++++++++++---------------- 1 file changed, 74 insertions(+), 45 deletions(-)
fs/btrfs/extent_map.c +74 -45
@@ -700,11 +700,11 @@ static void drop_all_extent_maps_fast(struct extent_map_tree *tree)void btrfs_drop_extent_map_range (struct btrfs_inode *inode,u64 start,u64 end,bool skip_pinned) { -struct extent_map *split =NULL ; -struct extent_map *split2 =NULL ; +struct extent_map *split; +struct extent_map *split2; +struct extent_map *em;struct extent_map_tree *em_tree = &inode->extent_tree;u64 len = end - start +1 ; -bool testend = true;WARN_ON (end < start);if (end == (u64)-1 ) {
@@ -713,57 +713,73 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,return ; } len = (u64)-1 ; - testend = false; + }else { +/* Make end offset exclusive for use in the loop below. */ + end++; } -while (1 ) { -struct extent_map *em; -u64 em_end; + +/* +* It's ok if we fail to allocate the extent maps, see the comment near +* the bottom of the loop below. We only need two spare extent maps in +* the worst case, where the first extent map that intersects our range +* starts before the range and the last extent map that intersects our +* range ends after our range (and they might be the same extent map), +* because we need to split those two extent maps at the boundaries. +*/ + split =alloc_extent_map (); + split2 =alloc_extent_map (); + +write_lock (&em_tree->lock); + em =lookup_extent_mapping (em_tree, start, len); + +while (em) { +/* extent_map_end() returns exclusive value (last byte + 1). */ +const u64 em_end =extent_map_end (em); +struct extent_map *next_em =NULL ;u64 gen;unsigned long flags; -bool ends_after_range = false; -bool no_splits = false;bool modified;bool compressed; -if (!split) - split =alloc_extent_map (); -if (!split2) - split2 =alloc_extent_map (); -if (!split || !split2) - no_splits = true; - -write_lock (&em_tree->lock); - em =lookup_extent_mapping (em_tree, start, len); -if (!em) { -write_unlock (&em_tree->lock); -break ; +if (em_end < end) { + next_em =next_extent_map (em); +if (next_em) { +if (next_em->start < end) +refcount_inc (&next_em->refs); +else + next_em =NULL ; + } } - em_end =extent_map_end (em); -if (testend && em_end > start + len) - ends_after_range = true; +if (skip_pinned &&test_bit (EXTENT_FLAG_PINNED , &em->flags)) { -if (ends_after_range) { -free_extent_map (em); -write_unlock (&em_tree->lock); -break ; - } start = em_end; -if (testend) +if (end != (u64)-1 ) len = start + len - em_end; -free_extent_map (em); -write_unlock (&em_tree->lock); -continue ; + goto next; } - flags = em->flags; - gen = em->generation; - compressed =test_bit (EXTENT_FLAG_COMPRESSED , &em->flags); +clear_bit (EXTENT_FLAG_PINNED , &em->flags);clear_bit (EXTENT_FLAG_LOGGING , &flags); modified = !list_empty (&em->list); -if (no_splits) - goto next; + +/* +* The extent map does not cross our target range, so no need to +* split it, we can remove it directly. +*/ +if (em->start >= start && em_end <= end) + goto remove_em; + + flags = em->flags; + gen = em->generation; + compressed =test_bit (EXTENT_FLAG_COMPRESSED , &em->flags);if (em->start < start) { +if (!split) { + split = split2; + split2 =NULL ; +if (!split) + goto remove_em; + } split->start = em->start; split->len = start - em->start;
@@ -794,7 +810,13 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end, split = split2; split2 =NULL ; } -if (ends_after_range) { +if (em_end > end) { +if (!split) { + split = split2; + split2 =NULL ; +if (!split) + goto remove_em; + } split->start = start + len; split->len = em_end - (start + len); split->block_start = em->block_start;
@@ -840,7 +862,7 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,free_extent_map (split); split =NULL ; } -next: +remove_em:if (extent_map_in_tree (em)) {/* * If the extent map is still in the tree it means that
@@ -862,20 +884,27 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,* full fsync, otherwise a fast fsync will miss this * extent if it's new and needs to be logged. */ -if ((em->start < start || ends_after_range) && modified) { -ASSERT (no_splits); +if ((em->start < start || em_end > end) && modified) { +ASSERT (!split);btrfs_set_inode_full_sync (inode); }remove_extent_mapping (em_tree, em); } -write_unlock (&em_tree->lock); -/* Once for us. */ +/* +* Once for the tree reference (we replaced or removed the +* extent map from the tree). +*/ free_extent_map (em); -/* And once for the tree. */ +next: +/* Once for us (for our lookup reference). */ free_extent_map (em); + + em = next_em; } +write_unlock (&em_tree->lock); +free_extent_map (split);free_extent_map (split2); }
Signature · 1 line
2.35.1
Re: [PATCH 01/13] btrfs: fix missed extent on fsync after dropping extent maps
In reply to [email protected]fs/btrfs/file.c void btrfs_drop_extent_cache(struct btrfs_inode *inode, u64 start, u64 end, · line 543
Quoted 10 lines · On 19/09/2022 22:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
When dropping extent maps for a range, through
btrfs_drop_extent_cache(), if we find an extent map that starts before our target range and/or ends before the target range, and we are not able to allocate extent maps for splitting that extent map, then we don't fail and simply remove the entire extent map from the inode's extent map tree.fs/btrfs/file.c +1
-0+if (testend && em->start + em->len > start + len)
%len comes from
u64 len = end - start + 1;
IMO >= was correct here; because including %start + %len is already after the range as in the original code. No?
Quoted 9 lines
fs/btrfs/file.c +2
-1+ ends_after_range = true; flags = em->flags; gen = em->generation;if (skip_pinned &&test_bit (EXTENT_FLAG_PINNED , &em->flags)) { -if (testend && em->start + em->len >= start + len) { +if (ends_after_range) {free_extent_map (em);write_unlock (&em_tree->lock);break ;
Re: [PATCH 01/13] btrfs: fix missed extent on fsync after dropping extent maps
Quoted 11 lines · Anand Jain
Quoted 7 lines · On 19/09/2022 22:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
When dropping extent maps for a range, through
btrfs_drop_extent_cache(), if we find an extent map that starts before our target range and/or ends before the target range, and we are not able to allocate extent maps for splitting that extent map, then we don't fail and simply remove the entire extent map from the inode's extent map tree.
Quoted 8 lines
Quoted 1 line
+ if (testend &&
em->start+em->len> start + len)%len comes from
u64 len = end - start + 1;IMO >= was correct here; because including %start + %len is already after the range as in the original code. No?
No, > is the correct thing to do. It only matters if the extent map's range ends after our range.
Try the math with a simple example, with a start of 0 and a length of 4096 (end offset if 4095), and you'll see.
Thanks.
Quoted 10 lines
Quoted 10 lines
fs/btrfs/file.c +2
-1+ ends_after_range = true; flags = em->flags; gen = em->generation;if (skip_pinned &&test_bit (EXTENT_FLAG_PINNED , &em->flags)) { -if (testend && em->start + em->len >= start + len) { +if (ends_after_range) {free_extent_map (em);write_unlock (&em_tree->lock);break ;
Re: [PATCH 01/13] btrfs: fix missed extent on fsync after dropping extent maps
Quoted 16 lines · Filipe Manana
Quoted 15 lines · Anand Jain
Quoted 10 lines · On 19/09/2022 22:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
When dropping extent maps for a range, through
btrfs_drop_extent_cache(), if we find an extent map that starts before our target range and/or ends before the target range, and we are not able to allocate extent maps for splitting that extent map, then we don't fail and simply remove the entire extent map from the inode's extent map tree.+ if (testend &&
em->start+em->len> start + len)%len comes from
Quoted 7 lines
Quoted 4 lines
u64 len = end - start + 1;IMO >= was correct here; because including %start + %len is already after the range as in the original code. No?
No, > is the correct thing to do. It only matters if the extent map's range ends after our range.
Got it.
Quoted 2 lines
Try the math with a simple example, with a start of 0 and a length of 4096 (end offset if 4095), and you'll see.
Thanks.
Quoted 12 lines
Thanks.
Quoted 10 lines
Quoted 10 lines
fs/btrfs/file.c +2
-1+ ends_after_range = true; flags = em->flags; gen = em->generation;if (skip_pinned &&test_bit (EXTENT_FLAG_PINNED , &em->flags)) { -if (testend && em->start + em->len >= start + len) { +if (ends_after_range) {free_extent_map (em);write_unlock (&em_tree->lock);break ;
Re: [PATCH 01/13] btrfs: fix missed extent on fsync after dropping extent maps
In reply to [email protected]Quoted 19 lines · On 19/09/2022 22:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
When dropping extent maps for a range, through
btrfs_drop_extent_cache(), if we find an extent map that starts before our target range and/or ends before the target range, and we are not able to allocate extent maps for splitting that extent map, then we don't fail and simply remove the entire extent map from the inode's extent map tree.This is generally fine, because in case anyone needs to access the extent map, it can just load it again later from the respective file extent item(s) in the subvolume btree. However, if that extent map is new and is in the list of modified extents, then a fast fsync will miss the parts of the extent that were outside our range (that needed to be split), therefore not logging them. Fix that by marking the inode for a full fsync. This issue was introduced after removing BUG_ON()s triggered when the split extent map allocations failed, done by commit
7014cdb49305ed("Btrfs:btrfs_drop_extent_cacheshould never fail"), back in 2012, and the fast fsync path already existed but was very recent.
Quoted 8 lines
Also, in the case where we could allocate extent maps for the split operations but then fail to add a split extent map to the tree, mark the inode for a full fsync as well. This is not supposed to ever fail, and we assert that, but in case assertions are disabled (
CONFIG_BTRFS_ASSERTis not set), it's the correct thing to do to make sure a fast fsync will not miss a new extent.Signed-off-by Filipe Manana <[email protected]>
Re: [PATCH 10/13] btrfs: remove unnecessary NULL pointer checks when searching extent maps
In reply to [email protected]Quoted 17 lines · On Mon, Sep 19, 2022 at 03:06:37PM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
The previous and next pointer arguments passed to
__tree_search()are never NULL as the only caller of this function,__lookup_extent_mapping(), always passes the address of two on stack pointers. So remove the NULL checks.Signed-off-by Filipe Manana <[email protected]>--- fs/btrfs/extent_map.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-)fs/btrfs/extent_map.c +0
-0
fs/btrfs/extent_map.c static struct rb_node *__tree_search(struct rb_root *root, u64 offset, · line 164
Quoted 2 lines
fs/btrfs/extent_map.c +0
-0@@ -163,24 +163,21 @@ static struct rb_node *__tree_search(struct rb_root *root, u64 offset,return n; }
I've added asserts to verify the pointers are not accidentaly NULL.
Quoted 10 lines · On Mon, Sep 19, 2022 at 03:06:27PM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
The following patchset fixes a bug related to dropping extent maps that can make an fsync miss a new extent, does several cleanups and some small performance improvements when dropping and searching for extent maps as well as when flushing delalloc in COW mode. These came out while working on some upcoming changes for fiemap, but since they are really independent, I'm sending them as a separate patchset.
The last patch in the series has a test and results in its changelog.
Quoted 14 lines
Filipe Manana (13):
btrfs: fix missed extent on fsync after dropping extent maps
btrfs: movebtrfs_drop_extent_cache()toextent_map.c
btrfs: useextent_map_end()atbtrfs_drop_extent_map_range()
btrfs: usecond_resched_rwlock_write()during inode eviction
btrfs: move open coded extent map tree deletion out of inode eviction
btrfs: add helper to replace extent map range with a new extent map
btrfs: remove the refcount warning/check atfree_extent_map()
btrfs: remove unnecessary extent map initializations
btrfs: assert tree is locked when clearing extent map from logging
btrfs: remove unnecessary NULL pointer checks when searching extent maps
btrfs: remove unnecessary next extent map search
btrfs: avoid pointless extent map tree search when flushing delalloc
btrfs: drop extent map range more efficiently
Added to misc-next, namely for the first fix but the cleanups seem safe and the last patch with the performance improvement will be another one for the 6.1. Thanks.