Skip to content

Cleanup error handling in relocation

Josef Bacik linux-btrfs 8 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v8 Latest39 patches 2 repliesIn review
  2. v738 patches 6 repliesSuperseded
  3. v652 patches 2 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v552 patches3 repliesSuperseded
v453 patches9 repliesSuperseded
v354 patches59 repliesSuperseded
v242 patches13 repliesSuperseded
v142 patches6 repliesSuperseded

Changes in v8 from the cover letter

- Reordered some of the patches, so that the callers of functions that added new error cases were fixed first, and then added the new error handler.
- Moved a few of the ASSERT(0) to where they made sense. Left the others that are in functions that can have multiple cases of the same error that indicates a logic error.

Patches in v8 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/39btrfs: convert some BUG_ON()'s to ASSERT()'s in do_relocationfs/btrfs/relocation.c+16 -30 replies
2/39btrfs: convert BUG_ON()'s in relocate_tree_blockfs/btrfs/relocation.c+22 -20 replies
3/39btrfs: handle errors from select_reloc_root()fs/btrfs/relocation.c+12 -30 replies
4/39btrfs: convert BUG_ON()'s in select_reloc_root() to proper errorsfs/btrfs/relocation.c+43 -40 replies
5/39btrfs: check record_root_in_trans related failures in select_reloc_rootfs/btrfs/relocation.c+7 -20 replies
6/39btrfs: do proper error handling in record_reloc_root_in_transfs/btrfs/relocation.c+21 -20 replies
7/39btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename_exchangefs/btrfs/inode.c+5 -20 replies
8/39btrfs: handle btrfs_record_root_in_trans failure in btrfs_renamefs/btrfs/inode.c+5 -20 replies
9/39btrfs: handle btrfs_record_root_in_trans failure in btrfs_delete_subvolumefs/btrfs/inode.c+5 -10 replies
10/39btrfs: handle btrfs_record_root_in_trans failure in btrfs_recover_log_treesfs/btrfs/tree-log.c+6 -20 replies
11/39btrfs: handle btrfs_record_root_in_trans failure in create_subvolfs/btrfs/ioctl.c+6 -10 replies
12/39btrfs: btrfs: handle btrfs_record_root_in_trans failure in relocate_tree_blockfs/btrfs/relocation.c+3 -10 replies
13/39btrfs: handle btrfs_record_root_in_trans failure in start_transactionfs/btrfs/transaction.c+5 -10 replies
14/39btrfs: handle record_root_in_trans failure in qgroup_account_snapshotfs/btrfs/transaction.c+4 -20 replies
15/39btrfs: handle record_root_in_trans failure in btrfs_record_root_in_transfs/btrfs/transaction.c+3 -20 replies
16/39btrfs: handle record_root_in_trans failure in create_pending_snapshotfs/btrfs/transaction.c+8 -30 replies
17/39btrfs: return an error from btrfs_record_root_in_transfs/btrfs/transaction.c+3 -20 replies
18/39btrfs: have proper error handling in btrfs_init_reloc_rootfs/btrfs/relocation.c+6 -10 replies
19/39btrfs: do proper error handling in create_reloc_rootfs/btrfs/relocation.c+28 -60 replies
20/39btrfs: validate ->reloc_root after recording root in transfs/btrfs/relocation.c+15 -00 replies
21/39btrfs: handle btrfs_update_reloc_root failure in commit_fs_rootsfs/btrfs/transaction.c+3 -10 replies
22/39btrfs: change insert_dirty_subvol to return errorsfs/btrfs/relocation.c+9 -50 replies
23/39btrfs: handle btrfs_update_reloc_root failure in insert_dirty_subvolfs/btrfs/relocation.c+4 -10 replies
24/39btrfs: handle btrfs_update_reloc_root failure in prepare_to_mergefs/btrfs/relocation.c+12 -10 replies
25/39btrfs: do proper error handling in btrfs_update_reloc_rootfs/btrfs/relocation.c+2 -40 replies
26/39btrfs: convert logic BUG_ON()'s in replace_path to ASSERT()'sfs/btrfs/relocation.c+3 -30 replies
27/39btrfs: handle btrfs_cow_block errors in replace_pathfs/btrfs/relocation.c+10 -20 replies
28/39btrfs: handle btrfs_search_slot failure in replace_pathfs/btrfs/relocation.c+2 -11 reply
29/39btrfs: handle errors in reference count manipulation in replace_pathfs/btrfs/relocation.c+16 -40 replies
30/39btrfs: handle extent reference errors in do_relocationfs/btrfs/relocation.c+5 -40 replies
31/39btrfs: check for BTRFS_BLOCK_FLAG_FULL_BACKREF being set improperlyfs/btrfs/tree-checker.c+5 -00 replies
32/39btrfs: remove the extent item sanity checks in relocate_block_groupfs/btrfs/relocation.c+1 -280 replies
33/39btrfs: do proper error handling in create_reloc_inodefs/btrfs/relocation.c+37 -20 replies
34/39btrfs: handle __add_reloc_root failures in btrfs_recover_relocationfs/btrfs/relocation.c+10 -20 replies
35/39btrfs: do not panic in __add_reloc_rootfs/btrfs/relocation.c+5 -10 replies
36/39btrfs: cleanup error handling in prepare_to_mergefs/btrfs/relocation.c+8 -20 replies
37/39btrfs: handle extent corruption with select_one_root properlyfs/btrfs/relocation.c+16 -30 replies
38/39btrfs: do proper error handling in merge_reloc_rootsfs/btrfs/relocation.c+23 -20 replies
39/39btrfs: check return value of btrfs_commit_transaction in relocationfs/btrfs/relocation.c+5 -40 replies

Reviews trailers per patch

22 of 39 patches have a Reviewed-by or Acked-by.

PatchQu WenruoJohannes Thumshirn
1/39R-b
2/39R-b
3/39
4/39
5/39R-b
6/39
7/39R-b
8/39
9/39R-b
10/39R-b
11/39
12/39R-b
13/39R-b
14/39R-b
15/39
16/39R-b
17/39R-b R-b
18/39
19/39
20/39
21/39R-b
22/39
23/39
24/39
25/39R-b
26/39R-b
27/39R-b
28/39R-b
29/39R-b
30/39
31/39R-b
32/39R-b
33/39
34/39R-b
35/39
36/39R-b
37/39
38/39
39/39R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v8 thread

Cover letter [PATCH v8 00/39] Cleanup error handling in relocation

v7->v8:
- Reordered some of the patches, so that the callers of functions that added new error cases were fixed first, and then added the new error handler.
- Moved a few of the ASSERT(0) to where they made sense. Left the others that are in functions that can have multiple cases of the same error that indicates a logic error.

v6->v7:
- Broke up the series into 3 series, 1 for cosmetic things, 1 for all the major issues (including those reported on v6 of this set), and this new set which is solely the error handling related patches for relocation. It's still a lot of patches, sorry about that.

v5->v6:
- Reworked "btrfs: handle errors from select_reloc_root()" because Zygo reported hitting an ASSERT(ret != -ENOENT) during his testing. This was because I changed select_reloc_root() to return -ENOENT if we happened to race with somebody else who failed to init the reloc root, however we had an ASSERT() to check for this because it indicated corruption. I modified that patch to move the ASSERT() to where the problem actually is, so select_reloc_root() can return whatever error and it'll pass it along. I also removed Qu's reviewed-by for the patch because of the change.

v4->v5:
- Dropped "btrfs: fix error handling in commit_fs_roots" as it was merged.
- Fixed an ASSERT() that happened during relocation recovery that Zygo reported, I moved the error condition out of another condition which broke recovery if we had deleted subvols pending with relocation.

v3->v4:
- Squashed the __add_reloc_root error handling patches in btrfs_recover_relocation as they were small and in the same function.
- Squashed the record_root_in_trans failure handling patches for select_reloc_root as they were small and in the same function.
- Added a new patch to address an existing error handling problem with subvol creation.
- Fixed up the various cases that Qu noticed where I got things wrong, cleaning up a leaked root extent ref, a leaked inode item, and where I accidentally stopped dealing with errors from btrfs_drop_subtree.
- Reworked a bunch of the ASSERT()'s to do ASSERT(0) in their respective if statements.
- Added reviewed-bys.

v2->v3:
- A lot of extra patches fixing various things that I encountered while debugging the corruption problem that was uncovered by these patches.
- Fixed the panic that Zygo was seeing and other issues.
- Fixed up the comments from Nikolay and Filipe.

A slight note, the first set of patches could probably be taken now, and in fact

btrfs: fix error handling in commit_fs_roots

Was sent earlier this week and is very important and needs to be reviewed and merged ASAP. The following are safe and could be merged outside of the rest of this series

btrfs: allow error injection for btrfs_search_slot and btrfs_cow_block
btrfs: fix lockdep splat in btrfs_recover_relocation
btrfs: keep track of the root owner for relocation reads
btrfs: noinline btrfs_should_cancel_balance
btrfs: do not cleanup upper nodes in btrfs_backref_cleanup_node
btrfs: pass down the tree block level through ref-verify
btrfs: make sure owner is set in ref-verify
btrfs: don't clear ret in btrfs_start_dirty_block_groups

The rest obviously are all around the actual error handling.

v1->v2:
- fixed a bug where I accidentally dropped reading flags in relocate_block_group when I dropped the extra checks that we handle in the tree checker.

--- Original message ---
Hello,

Relocation is the last place that is not able to handle errors at all, which results in all sorts of lovely panics if you encounter corruptions or IO errors. I'm going to start cleaning up relocation, but before I move code around I want the error handling to be somewhat sane, so I'm not changing behavior and error handling at the same time.

These patches are purely about error handling, there is no behavior changing other than returning errors up the chain properly. There is a lot of room for follow up cleanups, which will happen next. However I wanted to get this series done today and out so we could get it merged ASAP, and then the follow up cleanups can happen later as they are less important and less critical.

The only exception to the above is the patch to add the error injection sites for btrfs_cow_block and btrfs_search_slot, and a lockdep fix that I discovered while running my tests, those are the first two patches in the series.

I tested this with my error injection stress test, where I keep track of all stack traces that have been tested and only inject errors when we have a new stack trace, which means I should have covered all of the various error conditions. With this patchset I'm no longer panicing while stressing the error conditions. Thanks,

Josef

Josef Bacik (39):
btrfs: convert some BUG_ON()'s to ASSERT()'s in do_relocation
btrfs: convert BUG_ON()'s in relocate_tree_block
btrfs: handle errors from select_reloc_root()
btrfs: convert BUG_ON()'s in select_reloc_root() to proper errors
btrfs: check record_root_in_trans related failures in
select_reloc_root
btrfs: do proper error handling in record_reloc_root_in_trans
btrfs: handle btrfs_record_root_in_trans failure in
btrfs_rename_exchange
btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename
btrfs: handle btrfs_record_root_in_trans failure in
btrfs_delete_subvolume
btrfs: handle btrfs_record_root_in_trans failure in
btrfs_recover_log_trees
btrfs: handle btrfs_record_root_in_trans failure in create_subvol
btrfs: btrfs: handle btrfs_record_root_in_trans failure in
relocate_tree_block
btrfs: handle btrfs_record_root_in_trans failure in start_transaction
btrfs: handle record_root_in_trans failure in qgroup_account_snapshot
btrfs: handle record_root_in_trans failure in
btrfs_record_root_in_trans
btrfs: handle record_root_in_trans failure in create_pending_snapshot
btrfs: return an error from btrfs_record_root_in_trans
btrfs: have proper error handling in btrfs_init_reloc_root
btrfs: do proper error handling in create_reloc_root
btrfs: validate ->reloc_root after recording root in trans
btrfs: handle btrfs_update_reloc_root failure in commit_fs_roots
btrfs: change insert_dirty_subvol to return errors
btrfs: handle btrfs_update_reloc_root failure in insert_dirty_subvol
btrfs: handle btrfs_update_reloc_root failure in prepare_to_merge
btrfs: do proper error handling in btrfs_update_reloc_root
btrfs: convert logic BUG_ON()'s in replace_path to ASSERT()'s
btrfs: handle btrfs_cow_block errors in replace_path
btrfs: handle btrfs_search_slot failure in replace_path
btrfs: handle errors in reference count manipulation in replace_path
btrfs: handle extent reference errors in do_relocation
btrfs: check for BTRFS_BLOCK_FLAG_FULL_BACKREF being set improperly
btrfs: remove the extent item sanity checks in relocate_block_group
btrfs: do proper error handling in create_reloc_inode
btrfs: handle __add_reloc_root failures in btrfs_recover_relocation
btrfs: do not panic in __add_reloc_root
btrfs: cleanup error handling in prepare_to_merge
btrfs: handle extent corruption with select_one_root properly
btrfs: do proper error handling in merge_reloc_roots
btrfs: check return value of btrfs_commit_transaction in relocation

 fs/btrfs/inode.c        |  20 +-
 fs/btrfs/ioctl.c        |   7 +-
 fs/btrfs/relocation.c   | 434 +++++++++++++++++++++++++++++++---------
 fs/btrfs/transaction.c  |  37 +++-
 fs/btrfs/tree-checker.c |   5 +
 fs/btrfs/tree-log.c     |   8 +-
 6 files changed, 399 insertions(+), 112 deletions(-)
Signature · 1 line
2.26.2