btrfs: snapshot delete cleanups
v2 cover btrfs: snapshot delete cleanups
v1->v2:
- Simply removed the btrfs_check_eb_owner() calls as per Qu's suggestion.
- Made the 0 reference count error be more verbose as per Dave's suggestion.
--- Original email ---
Hello,
In light of the recent fix for snapshot delete I looked around at the code to see if it could be cleaned up. I still feel like this could be reworked to make the two stages clearer, but this series brings a lot of cleanups and re-factoring as well as comments and documentation that hopefully make this code easier for others to work in. I've broken up the do_walk_down() function into discreet peices that are better documented and describe their use. I've also taken the opportunity to remove a bunch of BUG_ON()'s in this code. I've run this through the CI a few times as I made a couple of errors, but it's passing cleanly now. Thanks,
Josef
Josef Bacik (15):
btrfs: don't do find_extent_buffer in do_walk_down
btrfs: remove all extra btrfs_check_eb_owner() calls
btrfs: use btrfs_read_extent_buffer in do_walk_down
btrfs: push lookup_info into walk_control
btrfs: move the eb uptodate code into it's own helper
btrfs: remove need_account in do_walk_down
btrfs: unify logic to decide if we need to walk down into a node
btrfs: extract the reference dropping code into it's own helper
btrfs: don't BUG_ON ENOMEM in walk_down_proc
btrfs: handle errors from ref mods during UPDATE_BACKREF
btrfs: replace BUG_ON with ASSERT in walk_down_proc
btrfs: clean up our handling of refs == 0 in snapshot delete
btrfs: convert correctness BUG_ON()'s to ASSERT()'s in walk_up_proc
btrfs: handle errors from btrfs_dec_ref properly
btrfs: add documentation around snapshot delete
fs/btrfs/ctree.c | 7 +- fs/btrfs/disk-io.c | 4 - fs/btrfs/extent-tree.c | 514 +++++++++++++++++++++++++++-------------- 3 files changed, 342 insertions(+), 183 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/15 btrfs: don't do find_extent_buffer in do_walk_down
We do find_extent_buffer(), and then if we don't find the eb in cache we call btrfs_find_create_tree_block(), which calls find_extent_buffer() first and then allocates the extent buffer.
The reason we're doing this is because if we don't find the extent buffer in cache we set reada = 1. However this doesn't matter, because lower down we only trigger reada if !btrfs_buffer_uptodate(eb), which is what the case would be if we didn't find the extent buffer in cache and had to allocate it.
Clean this up to simply call btrfs_find_create_tree_block(), and then use the fact that we're having to read the extent buffer off of disk to go ahead and kick off readahead.
--- fs/btrfs/extent-tree.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
fs/btrfs/extent-tree.c +6 -10
@@ -5433,7 +5433,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,struct btrfs_key key;struct extent_buffer *next;int level = wc->level; -int reada =0 ;int ret =0 ;bool need_account = false;
@@ -5459,14 +5458,11 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,btrfs_node_key_to_cpu (path->nodes[level], &check.first_key, path->slots[level]); - next =find_extent_buffer (fs_info, bytenr); -if (!next) { - next =btrfs_find_create_tree_block (fs_info, bytenr, -btrfs_root_id (root), level -1 ); -if (IS_ERR (next)) -return PTR_ERR (next); - reada =1 ; - } + next =btrfs_find_create_tree_block (fs_info, bytenr, +btrfs_root_id (root), level -1 ); +if (IS_ERR (next)) +return PTR_ERR (next); +btrfs_tree_lock (next); ret = btrfs_lookup_extent_info(trans, fs_info, bytenr, level - 1, 1,
@@ -5517,7 +5513,7 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, }if (!next) { -if (reada && level ==1 ) +if (level ==1 )reada_walk_down (trans, root, wc, path); next =read_tree_block (fs_info, bytenr, &check);if (IS_ERR (next)) {
Signature · 1 line
2.43.0
PATCH v2 2/15 btrfs: remove all extra btrfs_check_eb_owner() calls
In reply to Josef BacikCurrently we have a handful of btrfs_check_eb_owner() calls in various places and helpers that read extent buffers. However we call this in the endio handler for every metadata block, so these extra checks are unnecessary, simply remove them from everywhere except the endio handler.
--- fs/btrfs/ctree.c | 7 +------ fs/btrfs/disk-io.c | 4 ---- 2 files changed, 1 insertion(+), 10 deletions(-)
fs/btrfs/ctree.c +1 -6
@@ -1551,12 +1551,7 @@ read_block_for_search(struct btrfs_root *root, struct btrfs_path *p,if (ret) {free_extent_buffer (tmp);btrfs_release_path (p); -return -EIO ; - } -if (btrfs_check_eb_owner (tmp,btrfs_root_id (root))) { -free_extent_buffer (tmp); -btrfs_release_path (p); -return -EUCLEAN ; +return ret; }if (unlock_up)
fs/btrfs/disk-io.c +0 -4
@@ -635,10 +635,6 @@ struct extent_buffer *read_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr,free_extent_buffer_stale (buf);return ERR_PTR (ret); } -if (btrfs_check_eb_owner (buf, check->owner_root)) { -free_extent_buffer_stale (buf); -return ERR_PTR (-EUCLEAN ); - }return buf; }
Signature · 1 line
2.43.0
PATCH v2 3/15 btrfs: use btrfs_read_extent_buffer in do_walk_down
In reply to Josef BacikCurrently if our extent buffer isn't uptodate we will drop the lock, free it, and then call read_tree_block() for the bytenr. This is inefficient, we already have the extent buffer, we can simply call btrfs_read_extent_buffer().
Collapse these two cases down into one if statement, if we are not uptodate we can drop the lock, trigger readahead, and do the read using btrfs_read_extent_buffer(), and carry on.
--- fs/btrfs/extent-tree.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
fs/btrfs/extent-tree.c +4 -11
@@ -5507,22 +5507,15 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,if (!btrfs_buffer_uptodate (next, generation,0 )) {btrfs_tree_unlock (next); -free_extent_buffer (next); - next =NULL ; - *lookup_info =1 ; - } - -if (!next) {if (level ==1 )reada_walk_down (trans, root, wc, path); - next =read_tree_block (fs_info, bytenr, &check); -if (IS_ERR (next)) { -return PTR_ERR (next); - }else if (!extent_buffer_uptodate (next)) { + ret =btrfs_read_extent_buffer (next, &check); +if (ret) {free_extent_buffer (next); -return -EIO ; +return ret; }btrfs_tree_lock (next); + *lookup_info =1 ; } level--;
Signature · 1 line
2.43.0
PATCH v2 4/15 btrfs: push lookup_info into walk_control
In reply to Josef BacikInstead of using a flag we're passing around everywhere, add a field to walk_control that we're already passing around everywhere and use that instead.
--- fs/btrfs/extent-tree.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-)
fs/btrfs/extent-tree.c +11 -10
@@ -5220,6 +5220,7 @@ struct walk_control {int reada_slot;int reada_count;int restarted; +int lookup_info; };#define DROP_REFERENCE 1
@@ -5316,7 +5317,7 @@ static noinline void reada_walk_down(struct btrfs_trans_handle *trans, static noinline int walk_down_proc(struct btrfs_trans_handle *trans,struct btrfs_root *root,struct btrfs_path *path, -struct walk_control *wc,int lookup_info) +struct walk_control *wc) {struct btrfs_fs_info *fs_info = root->fs_info;int level = wc->level;
@@ -5331,7 +5332,7 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans,* when reference count of tree block is 1, it won't increase * again. once full backref flag is set, we never clear it. */ -if (lookup_info && +if (wc->lookup_info && ((wc->stage ==DROP_REFERENCE && wc->refs[level] !=1 ) || (wc->stage ==UPDATE_BACKREF && !(wc->flags[level] & flag)))) {BUG_ON (!path->locks[level]);
@@ -5423,7 +5424,7 @@ static int check_ref_exists(struct btrfs_trans_handle *trans, static noinline int do_walk_down(struct btrfs_trans_handle *trans,struct btrfs_root *root,struct btrfs_path *path, -struct walk_control *wc,int *lookup_info) +struct walk_control *wc) {struct btrfs_fs_info *fs_info = root->fs_info;u64 bytenr;
@@ -5445,7 +5446,7 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,*/ if (wc->stage ==UPDATE_BACKREF && generation <= root->root_key.offset) { - *lookup_info =1 ; + wc->lookup_info =1 ;return 1 ; }
@@ -5477,7 +5478,7 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, ret = -EIO ; goto out_unlock; } - *lookup_info =0 ; + wc->lookup_info =0 ;if (wc->stage ==DROP_REFERENCE ) {if (wc->refs[level -1 ] >1 ) {
@@ -5515,7 +5516,7 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,return ret; }btrfs_tree_lock (next); - *lookup_info =1 ; + wc->lookup_info =1 ; } level--;
@@ -5604,7 +5605,7 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, goto out_unlock; } no_delete: - *lookup_info =1 ; + wc->lookup_info =1 ; ret =1 ; out_unlock:
@@ -5738,11 +5739,11 @@ static noinline int walk_down_tree(struct btrfs_trans_handle *trans, struct walk_control *wc) {int level = wc->level; -int lookup_info =1 ;int ret =0 ; + wc->lookup_info =1 ;while (level >=0 ) { - ret =walk_down_proc (trans, root, path, wc, lookup_info); + ret =walk_down_proc (trans, root, path, wc);if (ret)break ;
@@ -5753,7 +5754,7 @@ static noinline int walk_down_tree(struct btrfs_trans_handle *trans, btrfs_header_nritems(path->nodes[level])) break; - ret =do_walk_down (trans, root, path, wc, &lookup_info); + ret =do_walk_down (trans, root, path, wc);if (ret >0 ) { path->slots[level]++;continue ;
Signature · 1 line
2.43.0
PATCH v2 5/15 btrfs: move the eb uptodate code into it's own helper
In reply to Josef Bacikdo_walk_down() already has a bunch of things going on, and there's a bit of code related to reading in the next eb if we decide we need it. Move this code off into it's own helper to clean up do_walk_down() a little bit.
--- fs/btrfs/extent-tree.c | 68 +++++++++++++++++++++++++++++------------- 1 file changed, 48 insertions(+), 20 deletions(-)
fs/btrfs/extent-tree.c +48 -20
@@ -5408,6 +5408,51 @@ static int check_ref_exists(struct btrfs_trans_handle *trans,return 1 ; } +/* +* We may not have an uptodate block, so if we are going to walk down into this +* block we need to drop the lock, read it off of the disk, re-lock it and +* return to continue dropping the snapshot. +*/ +static int check_next_block_uptodate (struct btrfs_trans_handle *trans, +struct btrfs_root *root, +struct btrfs_path *path, +struct walk_control *wc, +struct extent_buffer *next) +{ +struct btrfs_tree_parent_check check = {0 }; +u64 generation; +int level = wc->level; +int ret; + +btrfs_assert_tree_write_locked (next); + + generation =btrfs_node_ptr_generation (path->nodes[level], + path->slots[level]); + +if (btrfs_buffer_uptodate (next, generation,0 )) +return 0 ; + + check.level = level -1 ; + check.transid = generation; + check.owner_root =btrfs_root_id (root); + check.has_first_key = true; +btrfs_node_key_to_cpu (path->nodes[level], &check.first_key, + path->slots[level]); + +btrfs_tree_unlock (next); +if (level ==1 ) +reada_walk_down (trans, root, wc, path); + ret =btrfs_read_extent_buffer (next, &check); +if (ret) { +free_extent_buffer (next); +return ret; + } +btrfs_tree_lock (next); + wc->lookup_info =1 ; +return 0 ; +} + +/* * helper to process tree block pointer. *
@@ -5430,7 +5475,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,u64 bytenr;u64 generation;u64 owner_root =0 ; -struct btrfs_tree_parent_check check = {0 };struct btrfs_key key;struct extent_buffer *next;int level = wc->level;
@@ -5452,13 +5496,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, bytenr =btrfs_node_blockptr (path->nodes[level], path->slots[level]); - check.level = level -1 ; - check.transid = generation; - check.owner_root =btrfs_root_id (root); - check.has_first_key = true; -btrfs_node_key_to_cpu (path->nodes[level], &check.first_key, - path->slots[level]); - next =btrfs_find_create_tree_block (fs_info, bytenr,btrfs_root_id (root), level -1 );if (IS_ERR (next))
@@ -5506,18 +5543,9 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, goto skip; } -if (!btrfs_buffer_uptodate (next, generation,0 )) { -btrfs_tree_unlock (next); -if (level ==1 ) -reada_walk_down (trans, root, wc, path); - ret =btrfs_read_extent_buffer (next, &check); -if (ret) { -free_extent_buffer (next); -return ret; - } -btrfs_tree_lock (next); - wc->lookup_info =1 ; - } + ret =check_next_block_uptodate (trans, root, path, wc, next); +if (ret) +return ret; level--;ASSERT (level ==btrfs_header_level (next));
Signature · 1 line
2.43.0
PATCH v2 6/15 btrfs: remove need_account in do_walk_down
In reply to Josef BacikWe only set this if wc->refs[level - 1] > 1, and we check this way up above where we need it because the first thing we do before dropping our refs is reset wc->refs[level - 1] to 0. Re-order re-setting of wc->refs to after our drop logic, and then remove the need_account variable and simply check wc->refs[level - 1] directly instead of using a variable.
--- fs/btrfs/extent-tree.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
fs/btrfs/extent-tree.c +4 -5
@@ -5479,7 +5479,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,struct extent_buffer *next;int level = wc->level;int ret =0 ; -bool need_account = false; generation =btrfs_node_ptr_generation (path->nodes[level], path->slots[level]);
@@ -5519,7 +5518,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,if (wc->stage ==DROP_REFERENCE ) {if (wc->refs[level -1 ] >1 ) { - need_account = true;if (level ==1 && (wc->flags[0 ] &BTRFS_BLOCK_FLAG_FULL_BACKREF )) goto skip;
@@ -5562,8 +5560,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, wc->reada_slot =0 ;return 0 ; skip: - wc->refs[level -1 ] =0 ; - wc->flags[level -1 ] =0 ;if (wc->stage ==DROP_REFERENCE ) {struct btrfs_ref ref = { .action =BTRFS_DROP_DELAYED_REF ,
@@ -5608,7 +5604,8 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,* already accounted them at merge time (replace_path), * thus we could skip expensive subtree trace here. */ -if (btrfs_root_id (root) !=BTRFS_TREE_RELOC_OBJECTID && need_account) { +if (btrfs_root_id (root) !=BTRFS_TREE_RELOC_OBJECTID && + wc->refs[level -1 ] >1 ) { ret =btrfs_qgroup_trace_subtree (trans, next, generation, level -1 );if (ret) {
@@ -5633,6 +5630,8 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, goto out_unlock; } no_delete: + wc->refs[level -1 ] =0 ; + wc->flags[level -1 ] =0 ; wc->lookup_info =1 ; ret =1 ;
Signature · 1 line
2.43.0
PATCH v2 7/15 btrfs: unify logic to decide if we need to walk down into a node
In reply to Josef BacikWe currently duplicate the logic for walking into a node during snapshot delete. In one case it is during the actual delete, and in the other we use it for deciding if we should reada the block or not.
Extract this code into it's own helper and comment fully what we're doing, and then update the two users to use the new helper.
--- fs/btrfs/extent-tree.c | 135 +++++++++++++++++++++++++++-------------- 1 file changed, 91 insertions(+), 44 deletions(-)
fs/btrfs/extent-tree.c +91 -44
@@ -5226,6 +5226,81 @@ struct walk_control {#define DROP_REFERENCE 1#define UPDATE_BACKREF 2 +/* +* Decide if we need to walk down into this node to adjust the references. +* +* @root - the root we are currently deleting. +* @wc - the walk control for this deletion. +* @eb - the parent eb that we're currently visiting. +* @refs - the number of refs for wc->level - 1. +* @flags - the flags for wc->level - 1. +* @slot - the slot in the eb that we're currently checking. +* +* This is meant to be called when we're evaluating if a node we point to at +* wc->level should be read and walked into, or if we can simply delete our +* reference to it. We return true if we should walk into the node, false if we +* can skip it. +* +* We have assertions in here to make sure this is called correctly. We assume +* that sanity checking on the blocks read to this point has been done, so any +* corrupted file systems must have been caught before calling this function. +*/ +static bool visit_node_for_delete (struct btrfs_root *root, +struct walk_control *wc, +struct extent_buffer *eb, +u64 refs,u64 flags,int slot) +{ +struct btrfs_key key; +u64 generation; +int level = wc->level; + +ASSERT (level >0 ); +ASSERT (wc->refs[level -1 ] >0 ); + +/* +* The update backref stage we only want to skip if we already have +* FULL_BACKREF set, otherwise we need to read. +*/ +if (wc->stage ==UPDATE_BACKREF ) { +if (level ==1 && flags &BTRFS_BLOCK_FLAG_FULL_BACKREF ) +return false; +return true; + } + +/* +* We're the last ref on this block, we must walk into it and process +* any refs it's pointing at. +*/ +if (wc->refs[level -1 ] ==1 ) +return true; + +/* +* If we're already FULL_BACKREF then we know we can just drop our +* current reference. +*/ +if (level ==1 && flags &BTRFS_BLOCK_FLAG_FULL_BACKREF ) +return false; + +/* +* This block is older than our creation generation, we can drop our +* reference to it. +*/ + generation =btrfs_node_ptr_generation (eb, slot); +if (!wc->update_ref || generation <= root->root_key.offset) +return false; + +/* +* This block was processed from a previous snapshot deletion run, we +* can skip it. +*/ +btrfs_node_key_to_cpu (eb, &key, slot); +if (btrfs_comp_cpu_keys (&key, &wc->update_progress) <0 ) +return false; + +/* All other cases we need to wander into the node. */ +return true; +} + static noinline void reada_walk_down(struct btrfs_trans_handle *trans, struct btrfs_root *root, struct walk_control *wc,
@@ -5237,7 +5312,6 @@ static noinline void reada_walk_down(struct btrfs_trans_handle *trans,u64 refs;u64 flags;u32 nritems; -struct btrfs_key key;struct extent_buffer *eb;int ret;int slot;
@@ -5279,26 +5353,9 @@ static noinline void reada_walk_down(struct btrfs_trans_handle *trans,continue ;BUG_ON (refs ==0 ); -if (wc->stage ==DROP_REFERENCE ) { -if (refs ==1 ) - goto reada; - -if (wc->level ==1 && - (flags &BTRFS_BLOCK_FLAG_FULL_BACKREF )) -continue ; -if (!wc->update_ref || - generation <= root->root_key.offset) -continue ; -btrfs_node_key_to_cpu (eb, &key, slot); - ret =btrfs_comp_cpu_keys (&key, - &wc->update_progress); -if (ret <0 ) -continue ; - }else { -if (wc->level ==1 && - (flags &BTRFS_BLOCK_FLAG_FULL_BACKREF )) -continue ; - } +/* If we don't need to visit this node don't reada. */ +if (!visit_node_for_delete (root, wc, eb, refs, flags, slot)) +continue ; reada:btrfs_readahead_node_child (eb, slot); nread++;
@@ -5475,7 +5532,6 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,u64 bytenr;u64 generation;u64 owner_root =0 ; -struct btrfs_key key;struct extent_buffer *next;int level = wc->level;int ret =0 ;
@@ -5516,29 +5572,20 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, } wc->lookup_info =0 ; -if (wc->stage ==DROP_REFERENCE ) { -if (wc->refs[level -1 ] >1 ) { -if (level ==1 && - (wc->flags[0 ] &BTRFS_BLOCK_FLAG_FULL_BACKREF )) - goto skip; +/* If we don't have to walk into this node skip it. */ +if (!visit_node_for_delete (root, wc, path->nodes[level], + wc->refs[level -1 ], wc->flags[level -1 ], + path->slots[level])) + goto skip; -if (!wc->update_ref || - generation <= root->root_key.offset) - goto skip; - -btrfs_node_key_to_cpu (path->nodes[level], &key, - path->slots[level]); - ret =btrfs_comp_cpu_keys (&key, &wc->update_progress); -if (ret <0 ) - goto skip; - - wc->stage =UPDATE_BACKREF ; - wc->shared_level = level -1 ; - } - }else { -if (level ==1 && - (wc->flags[0 ] &BTRFS_BLOCK_FLAG_FULL_BACKREF )) - goto skip; +/* +* We have to walk down into this node, and if we're currently at the +* DROP_REFERNCE stage and this block is shared then we need to switch +* to the UPDATE_BACKREF stage in order to convert to FULL_BACKREF. +*/ +if (wc->stage ==DROP_REFERENCE && wc->refs[level -1 ] >1 ) { + wc->stage =UPDATE_BACKREF ; + wc->shared_level = level -1 ; } ret =check_next_block_uptodate (trans, root, path, wc, next);
Signature · 1 line
2.43.0
PATCH v2 8/15 btrfs: extract the reference dropping code into it's own helper
In reply to Josef BacikThis is a big chunk of code in do_walk_down that will conditionally remove the reference for the child block we're currently evaluating. Extract it out into it's own helper and call that from do_walk_down instead.
--- fs/btrfs/extent-tree.c | 162 +++++++++++++++++++++++------------------ 1 file changed, 92 insertions(+), 70 deletions(-)
fs/btrfs/extent-tree.c +92 -70
@@ -5509,6 +5509,95 @@ static int check_next_block_uptodate(struct btrfs_trans_handle *trans,return 0 ; } +/* +* If we determine that we don't have to visit wc->level - 1 then we need to +* determine if we can drop our reference. +* +* If we are UPDATE_BACKREF then we will not, we need to update our backrefs. +* +* If we are DROP_REFERENCE this will figure out if we need to drop our current +* reference, skipping it if we dropped it from a previous incompleted drop, or +* dropping it if we still have a reference to it. +*/ +static int maybe_drop_reference (struct btrfs_trans_handle *trans, +struct btrfs_root *root, +struct btrfs_path *path, +struct walk_control *wc, +struct extent_buffer *next, +u64 owner_root) +{ +struct btrfs_ref ref = { + .action =BTRFS_DROP_DELAYED_REF , + .bytenr = next->start, + .num_bytes = root->fs_info->nodesize, + .owning_root = owner_root, + .ref_root =btrfs_root_id (root), + }; +int level = wc->level; +int ret; + +/* We are UPDATE_BACKREF, we're not dropping anything. */ +if (wc->stage ==UPDATE_BACKREF ) +return 0 ; + +if (wc->flags[level] &BTRFS_BLOCK_FLAG_FULL_BACKREF ) { + ref.parent = path->nodes[level]->start; + }else { +ASSERT (btrfs_root_id (root) == +btrfs_header_owner (path->nodes[level])); +if (btrfs_root_id (root) != +btrfs_header_owner (path->nodes[level])) { +btrfs_err (root->fs_info, +"mismatched block owner" ); +return -EIO ; + } + } + +/* +* If we had a drop_progress we need to verify the refs are set as +* expected. If we find our ref then we know that from here on out +* everything should be correct, and we can clear the +* ->restarted flag. +*/ +if (wc->restarted) { + ret =check_ref_exists (trans, root, next->start, ref.parent, + level -1 ); +if (ret <=0 ) +return ret; + ret =0 ; + wc->restarted =0 ; + } + +/* +* Reloc tree doesn't contribute to qgroup numbers, and we have already +* accounted them at merge time (replace_path), thus we could skip +* expensive subtree trace here. +*/ +if (btrfs_root_id (root) !=BTRFS_TREE_RELOC_OBJECTID && + wc->refs[level -1 ] >1 ) { +u64 generation =btrfs_node_ptr_generation (path->nodes[level], + path->slots[level]); + + ret =btrfs_qgroup_trace_subtree (trans, next, generation, level -1 ); +if (ret) { +btrfs_err_rl (root->fs_info, +"Error %d accounting shared subtree. Quota is out of sync, rescan required." , + ret); + } + } + +/* +* We need to update the next key in our walk control so we can +* update the drop_progress key accordingly. We don't care if +* find_next_key doesn't find a key because that means we're at +* the end and are going to clean up now. +*/ + wc->drop_level = level; +find_next_key (path, level, &wc->drop_progress); + +btrfs_init_tree_ref (&ref, level -1 ,0 , false); +return btrfs_free_extent (trans, &ref); +}/* * helper to process tree block pointer.
@@ -5607,76 +5696,9 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, wc->reada_slot =0 ;return 0 ; skip: -if (wc->stage ==DROP_REFERENCE ) { -struct btrfs_ref ref = { - .action =BTRFS_DROP_DELAYED_REF , - .bytenr = bytenr, - .num_bytes = fs_info->nodesize, - .owning_root = owner_root, - .ref_root =btrfs_root_id (root), - }; -if (wc->flags[level] &BTRFS_BLOCK_FLAG_FULL_BACKREF ) { - ref.parent = path->nodes[level]->start; - }else { -ASSERT (btrfs_root_id (root) == -btrfs_header_owner (path->nodes[level])); -if (btrfs_root_id (root) != -btrfs_header_owner (path->nodes[level])) { -btrfs_err (root->fs_info, -"mismatched block owner" ); - ret = -EIO ; - goto out_unlock; - } - } - -/* -* If we had a drop_progress we need to verify the refs are set -* as expected. If we find our ref then we know that from here -* on out everything should be correct, and we can clear the -* ->restarted flag. -*/ -if (wc->restarted) { - ret =check_ref_exists (trans, root, bytenr, ref.parent, - level -1 ); -if (ret <0 ) - goto out_unlock; -if (ret ==0 ) - goto no_delete; - ret =0 ; - wc->restarted =0 ; - } - -/* -* Reloc tree doesn't contribute to qgroup numbers, and we have -* already accounted them at merge time (replace_path), -* thus we could skip expensive subtree trace here. -*/ -if (btrfs_root_id (root) !=BTRFS_TREE_RELOC_OBJECTID && - wc->refs[level -1 ] >1 ) { - ret =btrfs_qgroup_trace_subtree (trans, next, - generation, level -1 ); -if (ret) { -btrfs_err_rl (fs_info, -"Error %d accounting shared subtree. Quota is out of sync, rescan required." , - ret); - } - } - -/* -* We need to update the next key in our walk control so we can -* update the drop_progress key accordingly. We don't care if -* find_next_key doesn't find a key because that means we're at -* the end and are going to clean up now. -*/ - wc->drop_level = level; -find_next_key (path, level, &wc->drop_progress); - -btrfs_init_tree_ref (&ref, level -1 ,0 , false); - ret =btrfs_free_extent (trans, &ref); -if (ret) - goto out_unlock; - } -no_delete: + ret =maybe_drop_reference (trans, root, path, wc, next, owner_root); +if (ret) + goto out_unlock; wc->refs[level -1 ] =0 ; wc->flags[level -1 ] =0 ; wc->lookup_info =1 ;
Signature · 1 line
2.43.0
PATCH v2 9/15 btrfs: don't BUG_ON ENOMEM in walk_down_proc
In reply to Josef BacikWe handle errors here properly, ENOMEM isn't fatal, return the error.
--- fs/btrfs/extent-tree.c | 1 - 1 file changed, 1 deletion(-)
fs/btrfs/extent-tree.c +0 -1
@@ -5398,7 +5398,6 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans, &wc->refs[level], &wc->flags[level], NULL); -BUG_ON (ret == -ENOMEM );if (ret)return ret;BUG_ON (wc->refs[level] ==0 );
Signature · 1 line
2.43.0
PATCH v2 10/15 btrfs: handle errors from ref mods during UPDATE_BACKREF
In reply to Josef BacikWe have blanket BUG_ON(ret) after every one of these reference mod attempts, which is just incorrect. If we encounter any errors during walk_down_tree() we will abort, so abort on any one of these failures.
--- fs/btrfs/extent-tree.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
fs/btrfs/extent-tree.c +12 -3
@@ -5418,11 +5418,20 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans,if (!(wc->flags[level] & flag)) {BUG_ON (!path->locks[level]); ret =btrfs_inc_ref (trans, root, eb,1 ); -BUG_ON (ret);/* -ENOMEM */ +if (ret) { +btrfs_abort_transaction (trans, ret); +return ret; + } ret =btrfs_dec_ref (trans, root, eb,0 ); -BUG_ON (ret);/* -ENOMEM */ +if (ret) { +btrfs_abort_transaction (trans, ret); +return ret; + } ret =btrfs_set_disk_extent_flags (trans, eb, flag); -BUG_ON (ret);/* -ENOMEM */ +if (ret) { +btrfs_abort_transaction (trans, ret); +return ret; + } wc->flags[level] |= flag; }
Signature · 1 line
2.43.0
PATCH v2 11/15 btrfs: replace BUG_ON with ASSERT in walk_down_proc
In reply to Josef BacikWe have a couple of areas where we check to make sure the tree block is locked before looking up or messing with references. This is old code so it has this as BUG_ON(). Convert this to ASSERT() for developers.
--- fs/btrfs/extent-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/extent-tree.c +2 -2
@@ -5392,7 +5392,7 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans,if (wc->lookup_info && ((wc->stage ==DROP_REFERENCE && wc->refs[level] !=1 ) || (wc->stage ==UPDATE_BACKREF && !(wc->flags[level] & flag)))) { -BUG_ON (!path->locks[level]); +ASSERT (path->locks[level]); ret = btrfs_lookup_extent_info(trans, fs_info, eb->start, level, 1, &wc->refs[level],
@@ -5416,7 +5416,7 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans,/* wc->stage == UPDATE_BACKREF */ if (!(wc->flags[level] & flag)) { -BUG_ON (!path->locks[level]); +ASSERT (path->locks[level]); ret =btrfs_inc_ref (trans, root, eb,1 );if (ret) {btrfs_abort_transaction (trans, ret);
Signature · 1 line
2.43.0
PATCH v2 12/15 btrfs: clean up our handling of refs == 0 in snapshot delete
In reply to Josef BacikIn reada we BUG_ON(refs == 0), which could be unkind since we aren't holding a lock on the extent leaf and thus could get a transient incorrect answer. In walk_down_proc we also BUG_ON(refs == 0), which could happen if we have extent tree corruption. Change that to return -EUCLEAN. In do_walk_down() we catch this case and handle it correctly, however we return -EIO, which -EUCLEAN is a more appropriate error code. Finally in walk_up_proc we have the same BUG_ON(refs == 0), so convert that to proper error handling. Also adjust the error message so we can actually do something with the information.
--- fs/btrfs/extent-tree.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-)
fs/btrfs/extent-tree.c +24 -5
@@ -5351,7 +5351,15 @@ static noinline void reada_walk_down(struct btrfs_trans_handle *trans,/* We don't care about errors in readahead. */ if (ret <0 )continue ; -BUG_ON (refs ==0 ); + +/* +* This could be racey, it's conceivable that we raced and end +* up with a bogus refs count, if that's the case just skip, if +* we are actually corrupt we will notice when we look up +* everything again with our locks. +*/ +if (refs ==0 ) +continue ;/* If we don't need to visit this node don't reada. */ if (!visit_node_for_delete (root, wc, eb, refs, flags, slot))
@@ -5400,7 +5408,12 @@ static noinline int walk_down_proc(struct btrfs_trans_handle *trans, NULL);if (ret)return ret; -BUG_ON (wc->refs[level] ==0 ); +if (unlikely (wc->refs[level] ==0 )) { +btrfs_err (fs_info, +"bytenr %llu has 0 references, expect > 0" , + eb->start); +return -EUCLEAN ; + } }if (wc->stage ==DROP_REFERENCE ) {
@@ -5663,8 +5676,9 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans, goto out_unlock;if (unlikely (wc->refs[level -1 ] ==0 )) { -btrfs_err (fs_info,"Missing references." ); - ret = -EIO ; +btrfs_err (fs_info,"bytenr %llu has 0 references, expect > 0" , + bytenr); + ret = -EUCLEAN ; goto out_unlock; } wc->lookup_info =0 ;
@@ -5775,7 +5789,12 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans, path->locks[level] =0 ;return ret; } -BUG_ON (wc->refs[level] ==0 ); +if (unlikely (wc->refs[level] ==0 )) { +btrfs_err (fs_info, +"bytenr %llu has 0 references, expect > 0" , + eb->start); +return -EUCLEAN ; + }if (wc->refs[level] ==1 ) {btrfs_tree_unlock_rw (eb, path->locks[level]); path->locks[level] =0 ;
Signature · 1 line
2.43.0
Re: [PATCH v2 12/15] btrfs: clean up our handling of refs == 0 in snapshot delete
fs/btrfs/extent-tree.c static noinline int walk_up_proc(struct btrfs_trans_handle *trans, · line 5792
Quoted 6 lines · Josef Bacik
fs/btrfs/extent-tree.c +1
-1@@ -5775,7 +5789,12 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans, path->locks[level] =0 ;return ret; } -BUG_ON (wc->refs[level] ==0 ); +if (unlikely (wc->refs[level] ==0 )) {
This still lacks the tree unlock as reported by Dan Carpenter in
https://lore.kernel.org/all/[email protected]/
Quoted 8 lines
fs/btrfs/extent-tree.c +5
-0+btrfs_err (fs_info, +"bytenr %llu has 0 references, expect > 0" , + eb->start); +return -EUCLEAN ; + }if (wc->refs[level] ==1 ) {btrfs_tree_unlock_rw (eb, path->locks[level]); path->locks[level] =0 ;
PATCH v2 13/15 btrfs: convert correctness BUG_ON()'s to ASSERT()'s in walk_up_proc
In reply to Josef BacikIn walk_up_proc we have several sanity checks that should only trip if the programmer made a mistake. Convert these to ASSERT()'s instead of BUG_ON()'s.
--- fs/btrfs/extent-tree.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/btrfs/extent-tree.c +3 -3
@@ -5757,7 +5757,7 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans,u64 parent =0 ;if (wc->stage ==UPDATE_BACKREF ) { -BUG_ON (wc->shared_level < level); +ASSERT (wc->shared_level >= level);if (level < wc->shared_level) goto out;
@@ -5775,7 +5775,7 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans,* count is one. */ if (!path->locks[level]) { -BUG_ON (level ==0 ); +ASSERT (level >0 );btrfs_tree_lock (eb); path->locks[level] =BTRFS_WRITE_LOCK ;
@@ -5804,7 +5804,7 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans, }/* wc->stage == DROP_REFERENCE */ -BUG_ON (wc->refs[level] >1 && !path->locks[level]); +ASSERT (path->locks[level] || wc->refs[level] ==1 );if (wc->refs[level] ==1 ) {if (level ==0 ) {
Signature · 1 line
2.43.0
PATCH v2 14/15 btrfs: handle errors from btrfs_dec_ref properly
In reply to Josef BacikIn walk_up_proc() we BUG_ON(ret) from btrfs_dec_ref(). This is incorrect, we have proper error handling here, return the error.
--- fs/btrfs/extent-tree.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +4 -1
@@ -5812,7 +5812,10 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans, ret =btrfs_dec_ref (trans, root, eb,1 ); else ret =btrfs_dec_ref (trans, root, eb,0 ); -BUG_ON (ret);/* -ENOMEM */ +if (ret) { +btrfs_abort_transaction (trans, ret); +return ret; + }if (is_fstree (btrfs_root_id (root))) { ret =btrfs_qgroup_trace_leaf_items (trans, eb);if (ret) {
Signature · 1 line
2.43.0
PATCH v2 15/15 btrfs: add documentation around snapshot delete
In reply to Josef BacikSnapshot delete has some complicated looking code that is weirdly subtle at times. I've cleaned it up the best I can without re-writing it, but there are still a lot of details that are non-obvious. Add a bunch of comments to the main parts of the code to help future developers better understand the mechanics of snapshot deletion.
--- fs/btrfs/extent-tree.c | 52 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
fs/btrfs/extent-tree.c +52 -0
@@ -5223,7 +5223,20 @@ struct walk_control {int lookup_info; }; +/* +* This is our normal stage. We are traversing blocks the current snapshot owns +* and we are dropping any of our references to any children we are able to, and +* then freeing the block once we've processed all of the children. +*/ #define DROP_REFERENCE 1 + +/* +* We enter this stage when we have to walk into a child block (meaning we can't +* simply drop our reference to it from our current parent node) and there are +* more than one reference on it. If we are the owner of any of the children +* blocks from the current parent node then we have to do the FULL_BACKREF dance +* on them in order to drop our normal ref and add the shared ref. +*/ #define UPDATE_BACKREF 2/*
@@ -5859,6 +5872,27 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans,return -EUCLEAN ; } +/* +* walk_down_tree consists of two steps. +* +* walk_down_proc(). Look up the reference count and reference of our current +* wc->level. At this point path->nodes[wc->level] should be populated and +* uptodate, and in most cases should already be locked. If we are in +* DROP_REFERENCE and our refcount is > 1 then we've entered a shared node and +* we can walk back up the tree. If we are UPDATE_BACKREF we have to set +* FULL_BACKREF on this node if it's not already set, and then do the +* FULL_BACKREF conversion dance, which is to drop the root reference and add +* the shared reference to all of this nodes children. +* +* do_walk_down(). This is where we actually start iterating on the children of +* our current path->nodes[wc->level]. For DROP_REFERENCE that means dropping +* our reference to the children that return false from visit_node_for_delete(), +* which has various conditions where we know we can just drop our reference +* without visiting the node. For UPDATE_BACKREF we will skip any children that +* visit_node_for_delete() returns false for, only walking down when necessary. +* The bulk of the work for UPDATE_BACKREF occurs in the walk_up_tree() part of +* snapshot deletion. +*/ static noinline int walk_down_tree(struct btrfs_trans_handle *trans, struct btrfs_root *root, struct btrfs_path *path,
@@ -5891,6 +5925,24 @@ static noinline int walk_down_tree(struct btrfs_trans_handle *trans,return (ret ==1 ) ?0 : ret; } +/* +* walk_up_tree is responsible for making sure we visit every slot on our +* current node, and if we're at the end of that node then we call +* walk_up_proc() on our current node which will do one of a few things based on +* our stage. +* +* UPDATE_BACKREF. If we wc->level is currently less than our wc->shared_level +* then we need to walk back up the tree, and then going back down into the +* other slots via walk_down_tree to update any other children from our original +* wc->shared_level. Once we're at or above our wc->shared_level we can switch +* back to DROP_REFERENCE, lookup the current nodes refs and flags, and carry +* on. +* +* DROP_REFERENCE. If our refs == 1 then we're going to free this tree block. +* If we're level 0 then we need to btrfs_dec_ref() on all of the data extents +* in our current leaf. After that we call btrfs_free_tree_block() on the +* current node and walk up to the next node to walk down the next slot. +*/ static noinline int walk_up_tree(struct btrfs_trans_handle *trans, struct btrfs_root *root, struct btrfs_path *path,
Signature · 1 line
2.43.0