btrfs-progs: check: add the ability to repair extent item generation corruption
v2 cover btrfs-progs: check: add the ability to repair extent item generation corruption
Although we have introduced the check ability to detect bad extent item generation, there is no repair ability.
I thought it would be rare to hit, but real world cases prove I'm a total idiot.
So this patchset will add the ability to repair, for both lowmem mode and original mode, along with enhanced test images.
There is also a bug fix for original mode, which fails to detect such problem if it's a tree block.
Changelog:
v2:
- Fix a type in the subject of the 4th patch
- Fix a bracket for for a logical and and bit and
The old code is fine and bit and has higher priority, but the bracket is intended to make that higher priority more obvious.
Qu Wenruo (4):
btrfs-progs: check/lowmem: add the ability to repair extent item
generation
btrfs-progs: check/original: don't reset extent generation for
check_block()
btrfs-progs: check/original: add the ability to repair extent item
generation
btrfs-progs: tests/fsck: enhance invalid extent item generation test
cases
check/main.c | 77 ++++++++++++++-
check/mode-common.c | 59 ++++++++++++
check/mode-common.h | 3 +
check/mode-lowmem.c | 89 ++++++++++++++++--
...xz => bad_extent_item_gen_for_data.img.xz} | Bin
.../bad_extent_item_gen_for_tree_block.img.xz | Bin 0 -> 1804 bytes
.../test.sh | 19 ----
7 files changed, 216 insertions(+), 31 deletions(-)
rename tests/fsck-tests/044-invalid-extent-item-generation/{bad_extent_item_gen.img.xz => bad_extent_item_gen_for_data.img.xz} (100%)
create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen_for_tree_block.img.xz
delete mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.shSignature · 1 line
2.28.0
PATCH v2 1/4 btrfs-progs: check/lowmem: add the ability to repair extent item generation
There is an internal report about bad extent item generation triggering tree-checker.
This patch will add the repair ability to btrfs check --mode=lowmem mode, by resetting the generation field of extent item.
Currently the correct generation for tree block is fetched from its header, while for data extent it uses transid as fallback.
--- check/mode-common.c | 59 ++++++++++++++++++++++++++++++ check/mode-common.h | 3 ++ check/mode-lowmem.c | 89 +++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 144 insertions(+), 7 deletions(-)
check/mode-common.c +59 -0
@@ -1195,3 +1195,62 @@ int recow_extent_buffer(struct btrfs_root *root, struct extent_buffer *eb)btrfs_release_path (&path);return ret; } + +/* +* Try to get a correct extent item geneartion. +* Return 0 if we get a correct generation. +* Return <0 if we failed to get one. +*/ +int get_extent_item_generation (struct btrfs_fs_info *fs_info, +u64 bytenr,u64 *gen_ret) +{ +struct btrfs_root *root = fs_info->extent_root; +struct btrfs_extent_item *ei; +struct btrfs_path path; +struct btrfs_key key; +int ret; + + key.objectid = bytenr; + key.type =BTRFS_METADATA_ITEM_KEY ; + key.offset = (u64)-1 ; + +btrfs_init_path (&path); + ret =btrfs_search_slot (NULL , root, &key, &path,0 ,0 ); +/* Not possible */ +if (ret ==0 ) + ret = -EUCLEAN ; +if (ret <0 ) + goto out; + ret =btrfs_previous_extent_item (root, &path, bytenr); +if (ret >0 ) + ret = -ENOENT ; +if (ret <0 ) + goto out; + + ei =btrfs_item_ptr (path.nodes[0 ], path.slots[0 ], + struct btrfs_extent_item); + +if (btrfs_extent_flags (path.nodes[0 ], ei) & +BTRFS_EXTENT_FLAG_TREE_BLOCK ) { +struct extent_buffer *eb; + + eb =read_tree_block (fs_info, bytenr,0 ); +if (extent_buffer_uptodate (eb)) { + *gen_ret =btrfs_header_generation (eb); + ret =0 ; + }else { + ret = -EIO ; + } +free_extent_buffer (eb); + }else { +/* +* TODO: Grab proper data generation for data extents. +* But this is not an urgent objective, as we can still +* use transaction id as fall back +*/ + ret = -ENOTSUP ; + } +out: +btrfs_release_path (&path); +return ret; +}
check/mode-common.h +3 -0
@@ -173,4 +173,7 @@ static inline u32 btrfs_type_to_imode(u8 type) return imode_by_btrfs_type[(type)]; } + +int get_extent_item_generation (struct btrfs_fs_info *fs_info, +u64 bytenr,u64 *gen_ret);#endif
check/mode-lowmem.c +82 -7
135 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v2 2/4 btrfs-progs: check/original: don't reset extent generation for check_block()
In reply to Qu WenruoIn check_block(), we unconditionally reset extent_record::generation.
This is in fact pretty correct, but this makes original mode fail to detect bad extent item geneartion.
So change to behavior to set the geneartion if and only if the tree block generation is higher.
--- check/main.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
check/main.c +2 -1
@@ -4440,7 +4440,8 @@ static int check_block(struct btrfs_root *root,if (!cache)return 1 ; rec = container_of(cache, struct extent_record, cache); - rec->generation =btrfs_header_generation (buf); +if (rec->generation <btrfs_header_generation (buf)) + rec->generation =btrfs_header_generation (buf); level =btrfs_header_level (buf);if (btrfs_header_nritems (buf) >0 ) {
Signature · 1 line
2.28.0
PATCH v2 3/4 btrfs-progs: check/original: add the ability to repair extent item generation
In reply to Qu WenruoThis is pretty much the same as the lowmem mode, it will try to reset the extent item generation using either the tree block generation or current transid.
--- check/main.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 70 insertions(+), 4 deletions(-)
check/main.c +70 -4
@@ -7914,6 +7914,63 @@ static int record_unaligned_extent_rec(struct btrfs_fs_info *fs_info,return ret; } +static int repair_extent_item_generation (struct btrfs_fs_info *fs_info, +struct extent_record *rec) +{ +struct btrfs_trans_handle *trans; +struct btrfs_path path; +struct btrfs_key key; +struct btrfs_extent_item *ei; +struct btrfs_root *extent_root = fs_info->extent_root; +u64 new_gen =0 ;; +int ret; + + key.objectid = rec->start; + key.type =BTRFS_METADATA_ITEM_KEY ; + key.offset = (u64)-1 ; + +get_extent_item_generation (fs_info, rec->start, &new_gen); + trans =btrfs_start_transaction (extent_root,1 ); +if (IS_ERR (trans)) { + ret =PTR_ERR (trans); + errno = -ret; +error ("failed to start transaction: %m" ); +return ret; + } +btrfs_init_path (&path); + ret =btrfs_search_slot (trans, extent_root, &key, &path,0 ,1 ); +/* Not possible */ +if (ret ==0 ) + ret = -EUCLEAN ; +if (ret <0 ) + goto out; + ret =btrfs_previous_extent_item (extent_root, &path, rec->start); +if (ret >0 ) + ret = -ENOENT ; +if (ret <0 ) + goto out; + +if (!new_gen) + new_gen = trans->transid; + ei =btrfs_item_ptr (path.nodes[0 ], path.slots[0 ], + struct btrfs_extent_item); +btrfs_set_extent_generation (path.nodes[0 ], ei, new_gen); + ret =btrfs_commit_transaction (trans, extent_root); +if (ret <0 ) { + errno = -ret; +error ("failed to commit transaction: %m" ); + goto out; + } +printf ("Reset extent item (%llu) generation to %llu\n" , + key.objectid, new_gen); + rec->generation = new_gen; +out: +btrfs_release_path (&path); +if (ret <0 ) +btrfs_abort_transaction (trans, ret); +return ret; +} +static int check_extent_refs (struct btrfs_root *root,struct cache_tree *extent_cache) {
@@ -8004,11 +8061,20 @@ static int check_extent_refs(struct btrfs_root *root, }if (rec->generation > super_gen +1 ) { -error ( +bool repaired = false; +if (repair) { + ret =repair_extent_item_generation ( + root->fs_info, rec); +if (ret ==0 ) + repaired = true; + } +if (!repaired) { +error ("invalid generation for extent %llu, have %llu expect (0, %llu]" , - rec->start, rec->generation, - super_gen +1 ); - cur_err =1 ; + rec->start, rec->generation, + super_gen +1 ); + cur_err =1 ; + } }if (rec->refs != rec->extent_item_refs) { fprintf(stderr, "ref mismatch on [%llu %llu] ",
Signature · 1 line
2.28.0
PATCH v2 4/4 btrfs-progs: tests/fsck: enhance invalid extent item generation test cases
In reply to Qu WenruoThis patch will:
- Add a new test image for fsck/044
This new image has a corrupted extent item generation for tree block. This image can expose a bug in original mode, which can't detect the problem.
This image also utilize the tree block generation detection code, which the existing image doesn't.
- Rename the existing image
To reflect the fact that the existing one is only for data extent.
- Remove the test.sh
So that the generic path will test both detection and repair.
---
...xz => bad_extent_item_gen_for_data.img.xz} | Bin
.../bad_extent_item_gen_for_tree_block.img.xz | Bin 0 -> 1804 bytes
.../test.sh | 19 ------------------
3 files changed, 19 deletions(-)
rename tests/fsck-tests/044-invalid-extent-item-generation/{bad_extent_item_gen.img.xz => bad_extent_item_gen_for_data.img.xz} (100%)
create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen_for_tree_block.img.xz
delete mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.shtests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen_for_data.img.xz renamed from tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz +0 -0
tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen_for_tree_block.img.xz new file binary +0 -0
tests/fsck-tests/044-invalid-extent-item-generation/test.sh deleted +0 -19
@@ -1,19 +0,0 @@ -#!/bin/bash -# -# Due to a bug in --init-extent-tree option, we may create bad generation -# number for data extents. -# -# This test case will ensure btrfs check can at least detect such problem, -# just like kernel tree-checker. - -source "$TEST_TOP/common" - -check_prereq btrfs - -check_image() { - run_mustfail \ - "btrfs check failed to detect invalid extent item generation" \ - "$TOP/btrfs" check "$1" -} - -check_all_images
Signature · 1 line
2.28.0
Quoted 19 lines · Qu Wenruo
Although we have introduced the check ability to detect bad extent item generation, there is no repair ability.
I thought it would be rare to hit, but real world cases prove I'm a total idiot.
So this patchset will add the ability to repair, for both lowmem mode and original mode, along with enhanced test images.
There is also a bug fix for original mode, which fails to detect such problem if it's a tree block.
Changelog:
v2:
- Fix a type in the subject of the 4th patch
- Fix a bracket for for a logical and and bit and
The old code is fine and bit and has higher priority, but the bracket is intended to make that higher priority more obvious.
Quoted 9 lines
Qu Wenruo (4):
btrfs-progs: check/lowmem: add the ability to repair extent item
generation
btrfs-progs: check/original: don't reset extent generation for
check_block()
btrfs-progs: check/original: add the ability to repair extent item
generation
btrfs-progs: tests/fsck: enhance invalid extent item generation test
cases
Added to devel, thanks.