extent buffer dirty cleanups
v3 cover extent buffer dirty cleanups
v2->v3:
- Reworked the logic around clearing dirty on extent buffers not in our transaction. We do need this for update_ref_for_cow in some cases that I didn't account for. I've expanded the logic for this make it more idiot proof, and adjusted all of the patches accordingly.
v1->v2:
- Added "btrfs: do not call btrfs_clean_tree_block in update_ref_for_cow", Qu noticed some corruption with the original patchset, this turned out to be because we were clearing the extent buffer dirty at cow time, which doesn't make sense as we're not free'ing the current block in our current transaction.
--- Original email ---
Hello,
While sync'ing ctree.c to btrfs-progs I noticed we have some oddities when it comes to how we deal with the extent buffer being dirty. We have btrfs_clean_tree_block, which is sort of meant to be run against extent buffers we've modified in this transaction. However we have some other places where we've open coded the same work without the generation check. This makes it kind of confusing, and is inconsistent with how we deal with the fs_info->dirty_metadata_bytes.
So clean this stuff up so we have one helper we use for setting the extent buffer dirty (btrfs_mark_buffer_dirty) and one for clearing dirty (btrfs_clear_buffer_dirty). This makes everything more consistent and clean across the board. I've additionally cleaned up a random writeback thing we had in tree-log that I noticed while doing these cleanups. Thanks,
Josef
Josef Bacik (7):
btrfs: always lock the block before calling btrfs_clean_tree_block
btrfs: add trans argument to btrfs_clean_tree_block
btrfs: replace clearing extent buffer dirty bit with btrfs_clean_block
btrfs: do not increment dirty_metadata_bytes in set_btree_ioerr
btrfs: rename btrfs_clean_tree_block => btrfs_clear_buffer_dirty
btrfs: combine btrfs_clear_buffer_dirty and clear_extent_buffer_dirty
btrfs: remove btrfs_wait_tree_block_writeback
fs/btrfs/ctree.c | 31 +++++++++++++++-------------- fs/btrfs/disk-io.c | 25 +++++------------------- fs/btrfs/disk-io.h | 3 ++- fs/btrfs/extent-tree.c | 9 ++++----- fs/btrfs/extent_io.c | 23 ++++++++++++++-------- fs/btrfs/extent_io.h | 5 ++++- fs/btrfs/free-space-tree.c | 2 +- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 2 +- fs/btrfs/tree-log.c | 40 ++++++++++++++------------------------ 10 files changed, 64 insertions(+), 78 deletions(-)
Signature · 1 line
2.26.3
PATCH v3 1/7 btrfs: always lock the block before calling btrfs_clean_tree_block
We want to clean up the dirty handling for extent buffers so it's a little more consistent, so skip the check for generation == transid and simply always lock the extent buffer before calling btrfs_clean_tree_block.
--- fs/btrfs/extent-tree.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/extent-tree.c +1 -2
@@ -5534,8 +5534,7 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans, } }/* make block locked assertion in btrfs_clean_tree_block happy */ -if (!path->locks[level] && -btrfs_header_generation (eb) == trans->transid) { +if (!path->locks[level]) {btrfs_tree_lock (eb); path->locks[level] =BTRFS_WRITE_LOCK ; }
Signature · 1 line
2.26.3
PATCH v3 2/7 btrfs: add trans argument to btrfs_clean_tree_block
In reply to Josef BacikWe check the header generation in the extent buffer against the current running transaction id to see if it's safe to clear DIRTY on this buffer. Generally speaking if we're clearing the buffer dirty we're holding the transaction open, but in the case of cleaning up an aborted transaction we don't, so we have extra checks in that path to check the transid. To allow for a future cleanup go ahead and pass in the trans handle so we don't have to rely on ->running_transaction being set.
--- fs/btrfs/ctree.c | 31 ++++++++++++++++--------------- fs/btrfs/disk-io.c | 6 +++--- fs/btrfs/disk-io.h | 3 ++- fs/btrfs/extent-tree.c | 4 ++-- fs/btrfs/free-space-tree.c | 2 +- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 2 +- fs/btrfs/tree-log.c | 6 +++--- 8 files changed, 29 insertions(+), 27 deletions(-)
fs/btrfs/ctree.c +16 -15
@@ -459,7 +459,7 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,if (ret)return ret; } -btrfs_clean_tree_block (buf); +btrfs_clean_tree_block (trans, buf); *last_ref =1 ; }return 0 ;
@@ -1026,7 +1026,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans, path->locks[level] =0 ; path->nodes[level] =NULL ; -btrfs_clean_tree_block (mid); +btrfs_clean_tree_block (trans, mid);btrfs_tree_unlock (mid);/* once for the path */ free_extent_buffer (mid);
@@ -1087,7 +1087,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans,if (wret <0 && wret != -ENOSPC ) ret = wret;if (btrfs_header_nritems (right) ==0 ) { -btrfs_clean_tree_block (right); +btrfs_clean_tree_block (trans, right);btrfs_tree_unlock (right);del_ptr (root, path, level +1 , pslot +1 );root_sub_used (root, right->len);
@@ -1133,7 +1133,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans,BUG_ON (wret ==1 ); }if (btrfs_header_nritems (mid) ==0 ) { -btrfs_clean_tree_block (mid); +btrfs_clean_tree_block (trans, mid);btrfs_tree_unlock (mid);del_ptr (root, path, level +1 , pslot);root_sub_used (root, mid->len);
@@ -3008,7 +3008,8 @@ noinline int btrfs_leaf_free_space(struct extent_buffer *leaf)* min slot controls the lowest index we're willing to push to the * right. We'll push up to and including min_slot, but no lower */ -static noinline int __push_leaf_right(struct btrfs_path *path, +static noinline int __push_leaf_right(struct btrfs_trans_handle *trans, + struct btrfs_path *path, int data_size, int empty, struct extent_buffer *right, int free_space, u32 left_nritems,
@@ -3106,7 +3107,7 @@ static noinline int __push_leaf_right(struct btrfs_path *path,if (left_nritems)btrfs_mark_buffer_dirty (left);else -btrfs_clean_tree_block (left); +btrfs_clean_tree_block (trans, left);btrfs_mark_buffer_dirty (right);
@@ -3118,7 +3119,7 @@ static noinline int __push_leaf_right(struct btrfs_path *path,if (path->slots[0 ] >= left_nritems) { path->slots[0 ] -= left_nritems;if (btrfs_header_nritems (path->nodes[0 ]) ==0 ) -btrfs_clean_tree_block (path->nodes[0 ]); +btrfs_clean_tree_block (trans, path->nodes[0 ]);btrfs_tree_unlock (path->nodes[0 ]);free_extent_buffer (path->nodes[0 ]); path->nodes[0 ] = right;
@@ -3210,8 +3211,8 @@ static int push_leaf_right(struct btrfs_trans_handle *trans, struct btrfs_rootreturn 0 ; } -return __push_leaf_right (path, min_data_size, empty, - right, free_space, left_nritems, min_slot); +return __push_leaf_right (trans, path, min_data_size, empty, right, + free_space, left_nritems, min_slot); out_unlock:btrfs_tree_unlock (right);free_extent_buffer (right);
@@ -3226,7 +3227,8 @@ static int push_leaf_right(struct btrfs_trans_handle *trans, struct btrfs_root* item at 'max_slot' won't be touched. Use (u32)-1 to make us do all the * items */ -static noinline int __push_leaf_left(struct btrfs_path *path, int data_size, +static noinline int __push_leaf_left(struct btrfs_trans_handle *trans, +struct btrfs_path *path,int data_size,int empty,struct extent_buffer *left,int free_space,u32 right_nritems,u32 max_slot)
@@ -3330,7 +3332,7 @@ static noinline int __push_leaf_left(struct btrfs_path *path, int data_size,if (right_nritems)btrfs_mark_buffer_dirty (right);else -btrfs_clean_tree_block (right); +btrfs_clean_tree_block (trans, right);btrfs_item_key (right, &disk_key,0 );fixup_low_keys (path, &disk_key,1 );
@@ -3416,9 +3418,8 @@ static int push_leaf_left(struct btrfs_trans_handle *trans, struct btrfs_root ret = -EUCLEAN ; goto out; } -return __push_leaf_left (path, min_data_size, - empty, left, free_space, right_nritems, - max_slot); +return __push_leaf_left (trans, path, min_data_size, empty, left, + free_space, right_nritems, max_slot); out:btrfs_tree_unlock (left);free_extent_buffer (left);
@@ -4367,7 +4368,7 @@ int btrfs_del_items(struct btrfs_trans_handle *trans, struct btrfs_root *root,if (leaf == root->node) {btrfs_set_header_level (leaf,0 ); }else { -btrfs_clean_tree_block (leaf); +btrfs_clean_tree_block (trans, leaf);btrfs_del_leaf (trans, root, path, leaf); } }else {
fs/btrfs/disk-io.c +3 -3
@@ -965,11 +965,11 @@ struct extent_buffer *read_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr, } -void btrfs_clean_tree_block (struct extent_buffer *buf) +void btrfs_clean_tree_block (struct btrfs_trans_handle *trans, +struct extent_buffer *buf) {struct btrfs_fs_info *fs_info = buf->fs_info; -if (btrfs_header_generation (buf) == - fs_info->running_transaction->transid) { +if (btrfs_header_generation (buf) == trans->transid) {btrfs_assert_tree_write_locked (buf);if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , &buf->bflags)) {
fs/btrfs/disk-io.h +2 -1
@@ -39,7 +39,8 @@ struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr, u64 owner_root, int level); -void btrfs_clean_tree_block (struct extent_buffer *buf); +void btrfs_clean_tree_block (struct btrfs_trans_handle *trans, +struct extent_buffer *buf);void btrfs_clear_oneshot_options (struct btrfs_fs_info *fs_info);int btrfs_start_pre_rw_mount (struct btrfs_fs_info *fs_info); int btrfs_check_super_csum(struct btrfs_fs_info *fs_info,
fs/btrfs/extent-tree.c +2 -2
@@ -4908,7 +4908,7 @@ btrfs_init_new_buffer(struct btrfs_trans_handle *trans, struct btrfs_root *root,btrfs_set_buffer_lockdep_class (lockdep_owner, buf, level);__btrfs_tree_lock (buf, nest); -btrfs_clean_tree_block (buf); +btrfs_clean_tree_block (trans, buf);clear_bit (EXTENT_BUFFER_STALE , &buf->bflags);clear_bit (EXTENT_BUFFER_NO_CHECK , &buf->bflags);
@@ -5538,7 +5538,7 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans,btrfs_tree_lock (eb); path->locks[level] =BTRFS_WRITE_LOCK ; } -btrfs_clean_tree_block (eb); +btrfs_clean_tree_block (trans, eb); }if (eb == root->node) {
fs/btrfs/free-space-tree.c +1 -1
@@ -1283,7 +1283,7 @@ int btrfs_clear_free_space_tree(struct btrfs_fs_info *fs_info)list_del (&free_space_root->dirty_list);btrfs_tree_lock (free_space_root->node); -btrfs_clean_tree_block (free_space_root->node); +btrfs_clean_tree_block (trans, free_space_root->node);btrfs_tree_unlock (free_space_root->node);btrfs_free_tree_block (trans,btrfs_root_id (free_space_root), free_space_root->node,0 ,1 );
fs/btrfs/ioctl.c +1 -1
@@ -707,7 +707,7 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,* exists). */ btrfs_tree_lock (leaf); -btrfs_clean_tree_block (leaf); +btrfs_clean_tree_block (trans, leaf);btrfs_tree_unlock (leaf);btrfs_free_tree_block (trans, objectid, leaf,0 ,1 );free_extent_buffer (leaf);
fs/btrfs/qgroup.c +1 -1
@@ -1303,7 +1303,7 @@ int btrfs_quota_disable(struct btrfs_fs_info *fs_info)list_del ("a_root->dirty_list);btrfs_tree_lock (quota_root->node); -btrfs_clean_tree_block (quota_root->node); +btrfs_clean_tree_block (trans, quota_root->node);btrfs_tree_unlock (quota_root->node);btrfs_free_tree_block (trans,btrfs_root_id (quota_root), quota_root->node,0 ,1 );
fs/btrfs/tree-log.c +3 -3
@@ -2637,7 +2637,7 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,if (trans) {btrfs_tree_lock (next); -btrfs_clean_tree_block (next); +btrfs_clean_tree_block (trans, next);btrfs_wait_tree_block_writeback (next);btrfs_tree_unlock (next); ret = btrfs_pin_reserved_extent(trans,
@@ -2707,7 +2707,7 @@ static noinline int walk_up_log_tree(struct btrfs_trans_handle *trans,if (trans) {btrfs_tree_lock (next); -btrfs_clean_tree_block (next); +btrfs_clean_tree_block (trans, next);btrfs_wait_tree_block_writeback (next);btrfs_tree_unlock (next); ret = btrfs_pin_reserved_extent(trans,
@@ -2790,7 +2790,7 @@ static int walk_log_tree(struct btrfs_trans_handle *trans,if (trans) {btrfs_tree_lock (next); -btrfs_clean_tree_block (next); +btrfs_clean_tree_block (trans, next);btrfs_wait_tree_block_writeback (next);btrfs_tree_unlock (next); ret = btrfs_pin_reserved_extent(trans,
Signature · 1 line
2.26.3
PATCH v3 3/7 btrfs: replace clearing extent buffer dirty bit with btrfs_clean_block
In reply to Josef BacikNow that we're passing in the trans into btrfs_clean_tree_block, we can easily roll in the handling of the !trans case and replace all occurrences of
if (test_and_clear_bit(EXTENT_BUFFER_DIRTY, &eb->bflags))
clear_extent_buffer_dirty(eb);
with
btrfs_tree_lock(eb);btrfs_clean_tree_block(eb);btrfs_tree_unlock(eb);
We need the lock because if we are actually dirty we need to make sure we aren't racing with anything that's starting writeout currently. This also makes sure that we're accounting fs_info->dirty_metadata_bytes appropriately.
--- fs/btrfs/disk-io.c | 11 ++++++----- fs/btrfs/tree-log.c | 34 +++++++++++++++------------------- 2 files changed, 21 insertions(+), 24 deletions(-)
fs/btrfs/disk-io.c +6 -5
@@ -969,7 +969,7 @@ void btrfs_clean_tree_block(struct btrfs_trans_handle *trans, struct extent_buffer *buf) {struct btrfs_fs_info *fs_info = buf->fs_info; -if (btrfs_header_generation (buf) == trans->transid) { +if (!trans ||btrfs_header_generation (buf) == trans->transid) {btrfs_assert_tree_write_locked (buf);if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , &buf->bflags)) {
@@ -5077,11 +5077,12 @@ static int btrfs_destroy_marked_extents(struct btrfs_fs_info *fs_info, start += fs_info->nodesize;if (!eb)continue ; -wait_on_extent_buffer_writeback (eb); -if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , - &eb->bflags)) -clear_extent_buffer_dirty (eb); +btrfs_tree_lock (eb); +wait_on_extent_buffer_writeback (eb); +btrfs_clean_tree_block (NULL , eb); +btrfs_tree_unlock (eb); +free_extent_buffer_stale (eb); } }
fs/btrfs/tree-log.c +15 -19
@@ -2635,11 +2635,12 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,return ret; } +btrfs_tree_lock (next); +btrfs_clean_tree_block (trans, next); +btrfs_wait_tree_block_writeback (next); +btrfs_tree_unlock (next); +if (trans) { -btrfs_tree_lock (next); -btrfs_clean_tree_block (trans, next); -btrfs_wait_tree_block_writeback (next); -btrfs_tree_unlock (next); ret =btrfs_pin_reserved_extent (trans, bytenr, blocksize);if (ret) {
@@ -2649,8 +2650,6 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,btrfs_redirty_list_add ( trans->transaction, next); }else { -if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , &next->bflags)) -clear_extent_buffer_dirty (next);unaccount_log_buffer (fs_info, bytenr); } }
@@ -2705,11 +2704,12 @@ static noinline int walk_up_log_tree(struct btrfs_trans_handle *trans, next = path->nodes[*level]; +btrfs_tree_lock (next); +btrfs_clean_tree_block (trans, next); +btrfs_wait_tree_block_writeback (next); +btrfs_tree_unlock (next); +if (trans) { -btrfs_tree_lock (next); -btrfs_clean_tree_block (trans, next); -btrfs_wait_tree_block_writeback (next); -btrfs_tree_unlock (next); ret =btrfs_pin_reserved_extent (trans, path->nodes[*level]->start, path->nodes[*level]->len);
@@ -2718,9 +2718,6 @@ static noinline int walk_up_log_tree(struct btrfs_trans_handle *trans,btrfs_redirty_list_add (trans->transaction, next); }else { -if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , &next->bflags)) -clear_extent_buffer_dirty (next); -unaccount_log_buffer (fs_info, path->nodes[*level]->start); }
@@ -2788,19 +2785,18 @@ static int walk_log_tree(struct btrfs_trans_handle *trans, next = path->nodes[orig_level]; +btrfs_tree_lock (next); +btrfs_clean_tree_block (trans, next); +btrfs_wait_tree_block_writeback (next); +btrfs_tree_unlock (next); +if (trans) { -btrfs_tree_lock (next); -btrfs_clean_tree_block (trans, next); -btrfs_wait_tree_block_writeback (next); -btrfs_tree_unlock (next); ret =btrfs_pin_reserved_extent (trans, next->start, next->len);if (ret) goto out;btrfs_redirty_list_add (trans->transaction, next); }else { -if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , &next->bflags)) -clear_extent_buffer_dirty (next);unaccount_log_buffer (fs_info, next->start); } }
Signature · 1 line
2.26.3
PATCH v3 4/7 btrfs: do not increment dirty_metadata_bytes in set_btree_ioerr
In reply to Josef BacikWe only add if we set the extent buffer dirty, and we subtract when we clear the extent buffer dirty. If we end up in set_btree_ioerr we have already cleared the buffer dirty, and we aren't resetting dirty on the extent buffer, so this is simply wrong.
--- fs/btrfs/extent_io.c | 7 ------- 1 file changed, 7 deletions(-)
fs/btrfs/extent_io.c +0 -7
@@ -2467,13 +2467,6 @@ static void set_btree_ioerr(struct page *page, struct extent_buffer *eb)*/ mapping_set_error (page->mapping, -EIO ); -/* -* If we error out, we should add back the dirty_metadata_bytes -* to make it consistent. -*/ -percpu_counter_add_batch (&fs_info->dirty_metadata_bytes, - eb->len, fs_info->dirty_metadata_batch); -/* * If writeback for a btree extent that doesn't belong to a log tree * failed, increment the counter transaction->eb_write_errors.
Signature · 1 line
2.26.3
PATCH v3 5/7 btrfs: rename btrfs_clean_tree_block => btrfs_clear_buffer_dirty
In reply to Josef Bacikbtrfs_clean_tree_block is a misnomer, it's just clear_extent_buffer_dirty with some extra accounting around it. Rename this to btrfs_clear_buffer_dirty to make it more clear it belongs with it's setter, btrfs_mark_buffer_dirty.
--- fs/btrfs/ctree.c | 16 ++++++++-------- fs/btrfs/disk-io.c | 6 +++--- fs/btrfs/disk-io.h | 4 ++-- fs/btrfs/extent-tree.c | 6 +++--- fs/btrfs/free-space-tree.c | 2 +- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 2 +- fs/btrfs/tree-log.c | 6 +++--- 8 files changed, 22 insertions(+), 22 deletions(-)
fs/btrfs/ctree.c +8 -8
@@ -459,7 +459,7 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,if (ret)return ret; } -btrfs_clean_tree_block (trans, buf); +btrfs_clear_buffer_dirty (trans, buf); *last_ref =1 ; }return 0 ;
@@ -1026,7 +1026,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans, path->locks[level] =0 ; path->nodes[level] =NULL ; -btrfs_clean_tree_block (trans, mid); +btrfs_clear_buffer_dirty (trans, mid);btrfs_tree_unlock (mid);/* once for the path */ free_extent_buffer (mid);
@@ -1087,7 +1087,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans,if (wret <0 && wret != -ENOSPC ) ret = wret;if (btrfs_header_nritems (right) ==0 ) { -btrfs_clean_tree_block (trans, right); +btrfs_clear_buffer_dirty (trans, right);btrfs_tree_unlock (right);del_ptr (root, path, level +1 , pslot +1 );root_sub_used (root, right->len);
@@ -1133,7 +1133,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans,BUG_ON (wret ==1 ); }if (btrfs_header_nritems (mid) ==0 ) { -btrfs_clean_tree_block (trans, mid); +btrfs_clear_buffer_dirty (trans, mid);btrfs_tree_unlock (mid);del_ptr (root, path, level +1 , pslot);root_sub_used (root, mid->len);
@@ -3107,7 +3107,7 @@ static noinline int __push_leaf_right(struct btrfs_trans_handle *trans,if (left_nritems)btrfs_mark_buffer_dirty (left);else -btrfs_clean_tree_block (trans, left); +btrfs_clear_buffer_dirty (trans, left);btrfs_mark_buffer_dirty (right);
@@ -3119,7 +3119,7 @@ static noinline int __push_leaf_right(struct btrfs_trans_handle *trans,if (path->slots[0 ] >= left_nritems) { path->slots[0 ] -= left_nritems;if (btrfs_header_nritems (path->nodes[0 ]) ==0 ) -btrfs_clean_tree_block (trans, path->nodes[0 ]); +btrfs_clear_buffer_dirty (trans, path->nodes[0 ]);btrfs_tree_unlock (path->nodes[0 ]);free_extent_buffer (path->nodes[0 ]); path->nodes[0 ] = right;
@@ -3332,7 +3332,7 @@ static noinline int __push_leaf_left(struct btrfs_trans_handle *trans,if (right_nritems)btrfs_mark_buffer_dirty (right);else -btrfs_clean_tree_block (trans, right); +btrfs_clear_buffer_dirty (trans, right);btrfs_item_key (right, &disk_key,0 );fixup_low_keys (path, &disk_key,1 );
@@ -4368,7 +4368,7 @@ int btrfs_del_items(struct btrfs_trans_handle *trans, struct btrfs_root *root,if (leaf == root->node) {btrfs_set_header_level (leaf,0 ); }else { -btrfs_clean_tree_block (trans, leaf); +btrfs_clear_buffer_dirty (trans, leaf);btrfs_del_leaf (trans, root, path, leaf); } }else {
fs/btrfs/disk-io.c +3 -3
@@ -965,8 +965,8 @@ struct extent_buffer *read_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr, } -void btrfs_clean_tree_block (struct btrfs_trans_handle *trans, -struct extent_buffer *buf) +void btrfs_clear_buffer_dirty (struct btrfs_trans_handle *trans, +struct extent_buffer *buf) {struct btrfs_fs_info *fs_info = buf->fs_info;if (!trans ||btrfs_header_generation (buf) == trans->transid) {
@@ -5080,7 +5080,7 @@ static int btrfs_destroy_marked_extents(struct btrfs_fs_info *fs_info,btrfs_tree_lock (eb);wait_on_extent_buffer_writeback (eb); -btrfs_clean_tree_block (NULL , eb); +btrfs_clear_buffer_dirty (NULL , eb);btrfs_tree_unlock (eb);free_extent_buffer_stale (eb);
fs/btrfs/disk-io.h +2 -2
@@ -39,8 +39,8 @@ struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr, u64 owner_root, int level); -void btrfs_clean_tree_block (struct btrfs_trans_handle *trans, -struct extent_buffer *buf); +void btrfs_clear_buffer_dirty (struct btrfs_trans_handle *trans, +struct extent_buffer *buf);void btrfs_clear_oneshot_options (struct btrfs_fs_info *fs_info);int btrfs_start_pre_rw_mount (struct btrfs_fs_info *fs_info); int btrfs_check_super_csum(struct btrfs_fs_info *fs_info,
fs/btrfs/extent-tree.c +3 -3
@@ -4908,7 +4908,7 @@ btrfs_init_new_buffer(struct btrfs_trans_handle *trans, struct btrfs_root *root,btrfs_set_buffer_lockdep_class (lockdep_owner, buf, level);__btrfs_tree_lock (buf, nest); -btrfs_clean_tree_block (trans, buf); +btrfs_clear_buffer_dirty (trans, buf);clear_bit (EXTENT_BUFFER_STALE , &buf->bflags);clear_bit (EXTENT_BUFFER_NO_CHECK , &buf->bflags);
@@ -5533,12 +5533,12 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans, } } } -/* make block locked assertion in btrfs_clean_tree_block happy */ +/* make block locked assertion in btrfs_clear_buffer_dirty happy */ if (!path->locks[level]) {btrfs_tree_lock (eb); path->locks[level] =BTRFS_WRITE_LOCK ; } -btrfs_clean_tree_block (trans, eb); +btrfs_clear_buffer_dirty (trans, eb); }if (eb == root->node) {
fs/btrfs/free-space-tree.c +1 -1
@@ -1283,7 +1283,7 @@ int btrfs_clear_free_space_tree(struct btrfs_fs_info *fs_info)list_del (&free_space_root->dirty_list);btrfs_tree_lock (free_space_root->node); -btrfs_clean_tree_block (trans, free_space_root->node); +btrfs_clear_buffer_dirty (trans, free_space_root->node);btrfs_tree_unlock (free_space_root->node);btrfs_free_tree_block (trans,btrfs_root_id (free_space_root), free_space_root->node,0 ,1 );
fs/btrfs/ioctl.c +1 -1
@@ -707,7 +707,7 @@ static noinline int create_subvol(struct user_namespace *mnt_userns,* exists). */ btrfs_tree_lock (leaf); -btrfs_clean_tree_block (trans, leaf); +btrfs_clear_buffer_dirty (trans, leaf);btrfs_tree_unlock (leaf);btrfs_free_tree_block (trans, objectid, leaf,0 ,1 );free_extent_buffer (leaf);
fs/btrfs/qgroup.c +1 -1
@@ -1303,7 +1303,7 @@ int btrfs_quota_disable(struct btrfs_fs_info *fs_info)list_del ("a_root->dirty_list);btrfs_tree_lock (quota_root->node); -btrfs_clean_tree_block (trans, quota_root->node); +btrfs_clear_buffer_dirty (trans, quota_root->node);btrfs_tree_unlock (quota_root->node);btrfs_free_tree_block (trans,btrfs_root_id (quota_root), quota_root->node,0 ,1 );
fs/btrfs/tree-log.c +3 -3
@@ -2636,7 +2636,7 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans, }btrfs_tree_lock (next); -btrfs_clean_tree_block (trans, next); +btrfs_clear_buffer_dirty (trans, next);btrfs_wait_tree_block_writeback (next);btrfs_tree_unlock (next);
@@ -2705,7 +2705,7 @@ static noinline int walk_up_log_tree(struct btrfs_trans_handle *trans, next = path->nodes[*level];btrfs_tree_lock (next); -btrfs_clean_tree_block (trans, next); +btrfs_clear_buffer_dirty (trans, next);btrfs_wait_tree_block_writeback (next);btrfs_tree_unlock (next);
@@ -2786,7 +2786,7 @@ static int walk_log_tree(struct btrfs_trans_handle *trans, next = path->nodes[orig_level];btrfs_tree_lock (next); -btrfs_clean_tree_block (trans, next); +btrfs_clear_buffer_dirty (trans, next);btrfs_wait_tree_block_writeback (next);btrfs_tree_unlock (next);
Signature · 1 line
2.26.3
PATCH v3 6/7 btrfs: combine btrfs_clear_buffer_dirty and clear_extent_buffer_dirty
In reply to Josef Bacikbtrfs_clear_buffer_dirty just does the test_clear_bit() and then calls clear_extent_buffer_dirty and does the dirty metadata accounting. Combine this into clear_extent_buffer_dirty and make the result btrfs_clear_buffer_dirty.
--- fs/btrfs/disk-io.c | 16 ---------------- fs/btrfs/extent_io.c | 16 +++++++++++++++- fs/btrfs/extent_io.h | 5 ++++- 3 files changed, 19 insertions(+), 18 deletions(-)
fs/btrfs/disk-io.c +0 -16
@@ -965,22 +965,6 @@ struct extent_buffer *read_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr, } -void btrfs_clear_buffer_dirty (struct btrfs_trans_handle *trans, -struct extent_buffer *buf) -{ -struct btrfs_fs_info *fs_info = buf->fs_info; -if (!trans ||btrfs_header_generation (buf) == trans->transid) { -btrfs_assert_tree_write_locked (buf); - -if (test_and_clear_bit (EXTENT_BUFFER_DIRTY , &buf->bflags)) { -percpu_counter_add_batch (&fs_info->dirty_metadata_bytes, - -buf->len, - fs_info->dirty_metadata_batch); -clear_extent_buffer_dirty (buf); - } - } -} -static void __setup_root (struct btrfs_root *root,struct btrfs_fs_info *fs_info,u64 objectid) {
fs/btrfs/extent_io.c +15 -1
@@ -36,6 +36,7 @@#include "file.h" #include "dev-replace.h" #include "super.h" +#include "transaction.h" static struct kmem_cache *extent_buffer_cache;
@@ -4813,12 +4814,25 @@ static void clear_subpage_extent_buffer_dirty(const struct extent_buffer *eb)WARN_ON (atomic_read (&eb->refs) ==0 ); } -void clear_extent_buffer_dirty (const struct extent_buffer *eb) +void btrfs_clear_buffer_dirty (struct btrfs_trans_handle *trans, +struct extent_buffer *eb) { +struct btrfs_fs_info *fs_info = eb->fs_info;int i;int num_pages;struct page *page; +btrfs_assert_tree_write_locked (eb); + +if (trans &&btrfs_header_generation (eb) != trans->transid) +return ; + +if (!test_and_clear_bit (EXTENT_BUFFER_DIRTY , &eb->bflags)) +return ; + +percpu_counter_add_batch (&fs_info->dirty_metadata_bytes, -eb->len, + fs_info->dirty_metadata_batch); +if (eb->fs_info->nodesize <PAGE_SIZE )return clear_subpage_extent_buffer_dirty (eb);
fs/btrfs/extent_io.h +4 -1
@@ -11,6 +11,8 @@#include "ulist.h" #include "misc.h" +struct btrfs_trans_handle ; +enum {EXTENT_BUFFER_UPTODATE ,EXTENT_BUFFER_DIRTY ,
@@ -261,7 +263,6 @@ void extent_buffer_bitmap_set(const struct extent_buffer *eb, unsigned long starvoid extent_buffer_bitmap_clear (const struct extent_buffer *eb,unsigned long start,unsigned long pos,unsigned long len); -void clear_extent_buffer_dirty (const struct extent_buffer *eb);bool set_extent_buffer_dirty (struct extent_buffer *eb);void set_extent_buffer_uptodate (struct extent_buffer *eb);void clear_extent_buffer_uptodate (struct extent_buffer *eb);
@@ -273,6 +274,8 @@ void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end, u32 bits_to_clear, unsigned long page_ops);int extent_invalidate_folio (struct extent_io_tree *tree,struct folio *folio,size_t offset); +void btrfs_clear_buffer_dirty (struct btrfs_trans_handle *trans, +struct extent_buffer *buf);int btrfs_alloc_page_array (unsigned int nr_pages,struct page **page_array);
Signature · 1 line
2.26.3
PATCH v3 7/7 btrfs: remove btrfs_wait_tree_block_writeback
In reply to Josef BacikThis is used in the tree-log code and is a holdover from previous iterations of extent buffer writeback. We can simply use wait_on_extent_buffer_writeback here, and remove btrfs_wait_tree_block_writeback completely.
--- fs/btrfs/tree-log.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
fs/btrfs/tree-log.c +3 -9
@@ -278,12 +278,6 @@ void btrfs_end_log_trans(struct btrfs_root *root) } } -static void btrfs_wait_tree_block_writeback (struct extent_buffer *buf) -{ -filemap_fdatawait_range (buf->pages[0 ]->mapping, - buf->start, buf->start + buf->len -1 ); -} -/* * the walk control struct is used to pass state down the chain when * processing the log tree. The stage field tells us which part
@@ -2637,7 +2631,7 @@ static noinline int walk_down_log_tree(struct btrfs_trans_handle *trans,btrfs_tree_lock (next);btrfs_clear_buffer_dirty (trans, next); -btrfs_wait_tree_block_writeback (next); +wait_on_extent_buffer_writeback (next);btrfs_tree_unlock (next);if (trans) {
@@ -2706,7 +2700,7 @@ static noinline int walk_up_log_tree(struct btrfs_trans_handle *trans,btrfs_tree_lock (next);btrfs_clear_buffer_dirty (trans, next); -btrfs_wait_tree_block_writeback (next); +wait_on_extent_buffer_writeback (next);btrfs_tree_unlock (next);if (trans) {
@@ -2787,7 +2781,7 @@ static int walk_log_tree(struct btrfs_trans_handle *trans,btrfs_tree_lock (next);btrfs_clear_buffer_dirty (trans, next); -btrfs_wait_tree_block_writeback (next); +wait_on_extent_buffer_writeback (next);btrfs_tree_unlock (next);if (trans) {
Signature · 1 line
2.26.3
Hi,
Quoted 5 lines
v2->v3:
- Reworked the logic around clearing dirty on extent buffers not in our transaction. We do need this forupdate_ref_for_cowin some cases that I didn't account for. I've expanded the logic for this make it more idiot proof, and adjusted all of the patches accordingly.
The issue reported for v2 have been fixed. Thanks a lot.
no regression have been found for these v3 patches here.
Best Regards
Wang Yugui ([email protected])
2023/01/27
Quoted 49 lines
v1->v2:
- Added "btrfs: do not callbtrfs_clean_tree_blockinupdate_ref_for_cow", Qu noticed some corruption with the original patchset, this turned out to be because we were clearing the extent buffer dirty at cow time, which doesn't make sense as we're not free'ing the current block in our current transaction.--- Original email ---
Hello,While sync'ing
ctree.cto btrfs-progs I noticed we have some oddities when it comes to how we deal with the extent buffer being dirty. We havebtrfs_clean_tree_block, which is sort of meant to be run against extent buffers we've modified in this transaction. However we have some other places where we've open coded the same work without the generation check. This makes it kind of confusing, and is inconsistent with how we deal with thefs_info->dirty_metadata_bytes.So clean this stuff up so we have one helper we use for setting the extent buffer dirty (
btrfs_mark_buffer_dirty) and one for clearing dirty (btrfs_clear_buffer_dirty). This makes everything more consistent and clean across the board. I've additionally cleaned up a random writeback thing we had in tree-log that I noticed while doing these cleanups. Thanks,Josef
Josef Bacik (7):
btrfs: always lock the block before callingbtrfs_clean_tree_block
btrfs: add trans argument tobtrfs_clean_tree_block
btrfs: replace clearing extent buffer dirty bit withbtrfs_clean_block
btrfs: do not incrementdirty_metadata_bytesinset_btree_ioerr
btrfs: renamebtrfs_clean_tree_block=>btrfs_clear_buffer_dirty
btrfs: combinebtrfs_clear_buffer_dirtyandclear_extent_buffer_dirty
btrfs: removebtrfs_wait_tree_block_writebackfs/btrfs/ctree.c | 31 +++++++++++++++-------------- fs/btrfs/disk-io.c | 25 +++++------------------- fs/btrfs/disk-io.h | 3 ++- fs/btrfs/extent-tree.c | 9 ++++----- fs/btrfs/extent_io.c | 23 ++++++++++++++-------- fs/btrfs/extent_io.h | 5 ++++- fs/btrfs/free-space-tree.c | 2 +- fs/btrfs/ioctl.c | 2 +- fs/btrfs/qgroup.c | 2 +- fs/btrfs/tree-log.c | 40 ++++++++++++++------------------------ 10 files changed, 64 insertions(+), 78 deletions(-)Signature · 1 line
2.26.3
Quoted 5 lines · Josef Bacik
v2->v3:
- Reworked the logic around clearing dirty on extent buffers not in our transaction. We do need this forupdate_ref_for_cowin some cases that I didn't account for. I've expanded the logic for this make it more idiot proof, and adjusted all of the patches accordingly.
Added to misc-next, thanks.