btrfs-progs: Handle error properly in btrfs_commit_transaction()
cover btrfs-progs: Handle error properly in btrfs_commit_transaction()
This patchset can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/commit_failure_cleanup
This patchset will address the BUG_ON() triggered in fuzz-tests/003 and fuzz-tests/009.
For proper error handling in btrfs_commit_transaction(), we need a way to clean up per-transaction data properly.
Currently we only have delayed refs which are per-transaction, so introduce a new function, btrfs_destroy_delayed_refs() to cleanup delayed refs.
Now btrfs_commit_transaction() can error out gracefully with proper cleanup.
Patch 1 and 2 are just some minor cleanups found when crafting the 3rd patch.
Qu Wenruo (3):
btrfs-progs: Remove the dead branch in btrfs_run_delayed_refs()
btrfs-progs: Refactor btrfs_finish_extent_commit()
btrfs-progs: Handle error properly in btrfs_commit_transaction()
ctree.h | 4 +--- delayed-ref.c | 24 ++++++++++++++++++++++++ delayed-ref.h | 5 +++++ extent-tree.c | 22 +++++++++------------- transaction.c | 23 +++++++++++++++-------- 5 files changed, 54 insertions(+), 24 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/3 btrfs-progs: Remove the dead branch in btrfs_run_delayed_refs()
cleanup_ref_head() will only return 0 or 1, no way to return minutes value.
So remove the dead branch.
--- extent-tree.c | 2 -- 1 file changed, 2 deletions(-)
extent-tree.c +0 -2
@@ -4235,8 +4235,6 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long nr)/* We dropped our lock, we need to loop. */ ret =0 ;continue ; - }else if (ret) { -return ret; } locked_ref =NULL ;continue ;
Signature · 1 line
2.21.0
Re: [PATCH 1/3] btrfs-progs: Remove the dead branch in btrfs_run_delayed_refs()
Quoted 6 lines · Qu Wenruo
cleanup_ref_head()will only return 0 or 1, no way to return minutes value.So remove the dead branch.
Signed-off-by Qu Wenruo <[email protected]>
This is a left-over from the kernel copy, due to the kernel version able to return < 0 from, say, run_and_cleanup_extent_op.
Quoted 18 lines
--- extent-tree.c | 2 -- 1 file changed, 2 deletions(-)extent-tree.c +0
-2@@ -4235,8 +4235,6 @@ int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans, unsigned long nr)/* We dropped our lock, we need to loop. */ ret =0 ;continue ; - }else if (ret) { -return ret; } locked_ref =NULL ;continue ;
PATCH 2/3 btrfs-progs: Refactor btrfs_finish_extent_commit()
In reply to Qu WenruoThis patch will refactor btrfs_finish_extent_commit() by:
- Make it return void
There is no failure pattern for btrfs_finish_extent_commit(), thus it always return 0. And the caller doesn't care about the return value. So no need to return int.
- Remove @root and @unpin parameters
@root is only used to extract fs_info, which can be extracted from transaction handler already.
@unpin is always fs_info->pinned_extents.
All these parameters can be extracted from @trans, no need to pass them.
--- ctree.h | 4 +--- extent-tree.c | 14 ++++++-------- transaction.c | 3 +-- 3 files changed, 8 insertions(+), 13 deletions(-)
ctree.h +1 -3
@@ -2516,9 +2516,7 @@ int btrfs_free_extent(struct btrfs_trans_handle *trans,struct btrfs_root *root, u64 bytenr, u64 num_bytes, u64 parent, u64 root_objectid, u64 owner, u64 offset); -int btrfs_finish_extent_commit (struct btrfs_trans_handle *trans, -struct btrfs_root *root, -struct extent_io_tree *unpin); +void btrfs_finish_extent_commit (struct btrfs_trans_handle *trans); int btrfs_inc_extent_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root, u64 bytenr, u64 num_bytes, u64 parent,
extent-tree.c +6 -8
@@ -1977,27 +1977,25 @@ next:return 0 ; } -int btrfs_finish_extent_commit (struct btrfs_trans_handle *trans, -struct btrfs_root *root, -struct extent_io_tree *unpin) +void btrfs_finish_extent_commit (struct btrfs_trans_handle *trans) {u64 start;u64 end;int ret; -struct extent_io_tree *free_space_cache; - free_space_cache = &root->fs_info->free_space_cache; +struct btrfs_fs_info *fs_info = trans->fs_info; +struct extent_io_tree *free_space_cache = &fs_info->free_space_cache; +struct extent_io_tree *pinned_extents = &fs_info->pinned_extents;while (1 ) { - ret =find_first_extent_bit (unpin,0 , &start, &end, + ret =find_first_extent_bit (pinned_extents,0 , &start, &end,EXTENT_DIRTY );if (ret)break ;update_pinned_extents (trans->fs_info, start, end +1 - start,0 ); -clear_extent_dirty (unpin, start, end); +clear_extent_dirty (pinned_extents, start, end);set_extent_dirty (free_space_cache, start, end); } -return 0 ; } static int pin_down_bytes(struct btrfs_trans_handle *trans, u64 bytenr,
transaction.c +1 -2
@@ -198,8 +198,7 @@ commit_tree:if (ret <0 ) goto out; ret =write_ctree_super (trans); -btrfs_finish_extent_commit (trans, fs_info->extent_root, - &fs_info->pinned_extents); +btrfs_finish_extent_commit (trans);kfree (trans);free_extent_buffer (root->commit_root); root->commit_root =NULL ;
Signature · 1 line
2.21.0
Re: [PATCH 2/3] btrfs-progs: Refactor btrfs_finish_extent_commit()
Quoted 6 lines · Qu Wenruo
This patch will refactor
btrfs_finish_extent_commit()by:
- Make it return void
There is no failure pattern forbtrfs_finish_extent_commit(), thus it always return 0. And the caller doesn't care about the return value. So no need to return int.
Quoted 8 lines
- Remove @root and @unpin parameters
@root is only used to extractfs_info, which can be extracted from transaction handler already.
@unpin is alwaysfs_info->pinned_extents.
All these parameters can be extracted from @trans, no need to pass them.Signed-off-by Qu Wenruo <[email protected]>
This also makes the function's signature identical to its kernel counterpart.
Quoted 74 lines
--- ctree.h | 4 +--- extent-tree.c | 14 ++++++-------- transaction.c | 3 +-- 3 files changed, 8 insertions(+), 13 deletions(-)ctree.h +1
-3@@ -2516,9 +2516,7 @@ int btrfs_free_extent(struct btrfs_trans_handle *trans,struct btrfs_root *root, u64 bytenr, u64 num_bytes, u64 parent, u64 root_objectid, u64 owner, u64 offset); -int btrfs_finish_extent_commit (struct btrfs_trans_handle *trans, -struct btrfs_root *root, -struct extent_io_tree *unpin); +void btrfs_finish_extent_commit (struct btrfs_trans_handle *trans); int btrfs_inc_extent_ref(struct btrfs_trans_handle *trans, struct btrfs_root *root, u64 bytenr, u64 num_bytes, u64 parent,extent-tree.c +6
-8@@ -1977,27 +1977,25 @@ next:return 0 ; } -int btrfs_finish_extent_commit (struct btrfs_trans_handle *trans, -struct btrfs_root *root, -struct extent_io_tree *unpin) +void btrfs_finish_extent_commit (struct btrfs_trans_handle *trans) {u64 start;u64 end;int ret; -struct extent_io_tree *free_space_cache; - free_space_cache = &root->fs_info->free_space_cache; +struct btrfs_fs_info *fs_info = trans->fs_info; +struct extent_io_tree *free_space_cache = &fs_info->free_space_cache; +struct extent_io_tree *pinned_extents = &fs_info->pinned_extents;while (1 ) { - ret =find_first_extent_bit (unpin,0 , &start, &end, + ret =find_first_extent_bit (pinned_extents,0 , &start, &end,EXTENT_DIRTY );if (ret)break ;update_pinned_extents (trans->fs_info, start, end +1 - start,0 ); -clear_extent_dirty (unpin, start, end); +clear_extent_dirty (pinned_extents, start, end);set_extent_dirty (free_space_cache, start, end); } -return 0 ; } static int pin_down_bytes(struct btrfs_trans_handle *trans, u64 bytenr,transaction.c +1
-2@@ -198,8 +198,7 @@ commit_tree:if (ret <0 ) goto out; ret =write_ctree_super (trans); -btrfs_finish_extent_commit (trans, fs_info->extent_root, - &fs_info->pinned_extents); +btrfs_finish_extent_commit (trans);kfree (trans);free_extent_buffer (root->commit_root); root->commit_root =NULL ;
PATCH 3/3 btrfs-progs: Handle error properly in btrfs_commit_transaction()
In reply to Qu Wenruo[BUG]
When running fuzz-tests/003 and fuzz-tests/009, btrfs-progs will crash due to BUG_ON().
[CAUSE]
We abused BUG_ON() in btrfs_commit_transaction(), which is one of the most error prone function for fuzzed images.
Currently to cleanup the aborted transaction, we only need to clean up the only per-transaction data: delayed refs.
This patch will introduce a new function, btrfs_destroy_delayed_refs() to cleanup delayed refs when we failed to commit transaction.
With that function, we will gently destroy per-trans delayed ref, and remove the BUG_ON()s in btrfs_commit_transaction().
--- delayed-ref.c | 24 ++++++++++++++++++++++++ delayed-ref.h | 5 +++++ extent-tree.c | 6 +++--- transaction.c | 20 ++++++++++++++------ 4 files changed, 46 insertions(+), 9 deletions(-)
delayed-ref.c +24 -0
@@ -605,3 +605,27 @@ free_ref:return -ENOMEM ; } + +void btrfs_destroy_delayed_refs (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +struct rb_node *node; +struct btrfs_delayed_ref_root *delayed_refs; + + delayed_refs = &trans->delayed_refs; +if (RB_EMPTY_ROOT (&delayed_refs->href_root)) +return ; +while ((node =rb_first (&delayed_refs->href_root)) !=NULL ) { +struct btrfs_delayed_ref_head *head; +struct btrfs_delayed_ref_node *ref; +struct rb_node *n; + + head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); +while ((n =rb_first (&head->ref_tree)) !=NULL ) { + ref = rb_entry(n, struct btrfs_delayed_ref_node, + ref_node); +drop_delayed_ref (trans, delayed_refs, head, ref); + } +ASSERT (cleanup_ref_head (trans, fs_info, head) ==0 ); + } +}
delayed-ref.h +5 -0
@@ -205,4 +205,9 @@ btrfs_delayed_node_to_tree_ref(struct btrfs_delayed_ref_node *node) { return container_of(node, struct btrfs_delayed_tree_ref, node); } + +int cleanup_ref_head (struct btrfs_trans_handle *trans, +struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_head *head); +void btrfs_destroy_delayed_refs (struct btrfs_trans_handle *trans);#endif
extent-tree.c +3 -3
@@ -4085,9 +4085,9 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_ref delayed_refs->num_heads_ready++; } -static int cleanup_ref_head (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info, -struct btrfs_delayed_ref_head *head) +int cleanup_ref_head (struct btrfs_trans_handle *trans, +struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_head *head) {struct btrfs_delayed_ref_root *delayed_refs;
transaction.c +14 -6
@@ -17,6 +17,7 @@#include "kerncompat.h" #include "disk-io.h" #include "transaction.h" +#include "delayed-ref.h" #include "messages.h"
@@ -165,7 +166,8 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,* consistent */ ret =btrfs_run_delayed_refs (trans,-1 ); -BUG_ON (ret); +if (ret <0 ) + goto error;if (root->commit_root == root->node) goto commit_tree;
@@ -182,21 +184,24 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans, root->root_item.level =btrfs_header_level (root->node); ret =btrfs_update_root (trans, root->fs_info->tree_root, &root->root_key, &root->root_item); -BUG_ON (ret); +if (ret <0 ) + goto error; commit_tree: ret =commit_tree_roots (trans, fs_info); -BUG_ON (ret); +if (ret <0 ) + goto error;/* * Ensure that all committed roots are properly accounted in the * extent tree */ ret =btrfs_run_delayed_refs (trans,-1 ); -BUG_ON (ret); +if (ret <0 ) + goto error;btrfs_write_dirty_block_groups (trans);__commit_transaction (trans, root);if (ret <0 ) - goto out; + goto error; ret =write_ctree_super (trans);btrfs_finish_extent_commit (trans);kfree (trans);
@@ -204,7 +209,10 @@ commit_tree: root->commit_root =NULL ; fs_info->running_transaction =NULL ; fs_info->last_trans_committed = transid; -out: +return ret; +error: +btrfs_destroy_delayed_refs (trans); +free (trans);return ret; }
Signature · 1 line
2.21.0
Quoted 11 lines · Qu Wenruo
[BUG]
When running fuzz-tests/003 and fuzz-tests/009, btrfs-progs will crash due toBUG_ON().[CAUSE]
We abusedBUG_ON()inbtrfs_commit_transaction(), which is one of the most error prone function for fuzzed images.Currently to cleanup the aborted transaction, we only need to clean up the only per-transaction data: delayed refs.
Quoted 7 lines
This patch will introduce a new function,
btrfs_destroy_delayed_refs()to cleanup delayed refs when we failed to commit transaction.With that function, we will gently destroy per-trans delayed ref, and remove the BUG_ON()s in
btrfs_commit_transaction().Signed-off-by Qu Wenruo <[email protected]>
Though see below for a suggestion of further cleanup
Quoted 58 lines
--- delayed-ref.c | 24 ++++++++++++++++++++++++ delayed-ref.h | 5 +++++ extent-tree.c | 6 +++--- transaction.c | 20 ++++++++++++++------ 4 files changed, 46 insertions(+), 9 deletions(-)delayed-ref.c +24
-0@@ -605,3 +605,27 @@ free_ref:return -ENOMEM ; } + +void btrfs_destroy_delayed_refs (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +struct rb_node *node; +struct btrfs_delayed_ref_root *delayed_refs; + + delayed_refs = &trans->delayed_refs; +if (RB_EMPTY_ROOT (&delayed_refs->href_root)) +return ; +while ((node =rb_first (&delayed_refs->href_root)) !=NULL ) { +struct btrfs_delayed_ref_head *head; +struct btrfs_delayed_ref_node *ref; +struct rb_node *n; + + head = rb_entry(node, struct btrfs_delayed_ref_head, href_node); +while ((n =rb_first (&head->ref_tree)) !=NULL ) { + ref = rb_entry(n, struct btrfs_delayed_ref_node, + ref_node); +drop_delayed_ref (trans, delayed_refs, head, ref); + } +ASSERT (cleanup_ref_head (trans, fs_info, head) ==0 ); + } +}delayed-ref.h +5
-0@@ -205,4 +205,9 @@ btrfs_delayed_node_to_tree_ref(struct btrfs_delayed_ref_node *node) { return container_of(node, struct btrfs_delayed_tree_ref, node); } + +int cleanup_ref_head (struct btrfs_trans_handle *trans, +struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_head *head); +void btrfs_destroy_delayed_refs (struct btrfs_trans_handle *trans);#endif extent-tree.c +0
-0
extent-tree.c static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_ref · line 4090
Quoted 9 lines
extent-tree.c +3
-3@@ -4085,9 +4085,9 @@ static void unselect_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_ref delayed_refs->num_heads_ready++; } -static int cleanup_ref_head (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info, -struct btrfs_delayed_ref_head *head) +int cleanup_ref_head (struct btrfs_trans_handle *trans, +struct btrfs_fs_info *fs_info, +struct btrfs_delayed_ref_head *head)
You can create a followup patch that removes the fs_info argument from cleanup_ref_head to make it identical to the kernel version.
Quoted 67 lines
extent-tree.c +0
-0{struct btrfs_delayed_ref_root *delayed_refs;transaction.c +14
-6@@ -17,6 +17,7 @@#include "kerncompat.h" #include "disk-io.h" #include "transaction.h" +#include "delayed-ref.h" #include "messages.h" @@ -165,7 +166,8 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans,* consistent */ ret =btrfs_run_delayed_refs (trans,-1 ); -BUG_ON (ret); +if (ret <0 ) + goto error;if (root->commit_root == root->node) goto commit_tree;@@ -182,21 +184,24 @@ int btrfs_commit_transaction(struct btrfs_trans_handle *trans, root->root_item.level =btrfs_header_level (root->node); ret =btrfs_update_root (trans, root->fs_info->tree_root, &root->root_key, &root->root_item); -BUG_ON (ret); +if (ret <0 ) + goto error; commit_tree: ret =commit_tree_roots (trans, fs_info); -BUG_ON (ret); +if (ret <0 ) + goto error;/* * Ensure that all committed roots are properly accounted in the * extent tree */ ret =btrfs_run_delayed_refs (trans,-1 ); -BUG_ON (ret); +if (ret <0 ) + goto error;btrfs_write_dirty_block_groups (trans);__commit_transaction (trans, root);if (ret <0 ) - goto out; + goto error; ret =write_ctree_super (trans);btrfs_finish_extent_commit (trans);kfree (trans);@@ -204,7 +209,10 @@ commit_tree: root->commit_root =NULL ; fs_info->running_transaction =NULL ; fs_info->last_trans_committed = transid; -out: +return ret; +error: +btrfs_destroy_delayed_refs (trans); +free (trans);return ret; }
Quoted 16 lines · Qu Wenruo
This patchset can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/commit_failure_cleanupThis patchset will address the
BUG_ON()triggered in fuzz-tests/003 and fuzz-tests/009.For proper error handling in
btrfs_commit_transaction(), we need a way to clean up per-transaction data properly.Currently we only have delayed refs which are per-transaction, so introduce a new function,
btrfs_destroy_delayed_refs()to cleanup delayed refs.Now
btrfs_commit_transaction()can error out gracefully with proper cleanup.
Quoted 7 lines
Patch 1 and 2 are just some minor cleanups found when crafting the 3rd patch.
Qu Wenruo (3):
btrfs-progs: Remove the dead branch inbtrfs_run_delayed_refs()
btrfs-progs: Refactorbtrfs_finish_extent_commit()
btrfs-progs: Handle error properly inbtrfs_commit_transaction()
Added to devel, thanks.