btrfs-progs: check: add the ability to detect and report mixed inline and regular data extents
cover btrfs-progs: check: add the ability to detect and report mixed inline and regular data extents
Btrfs check original mode can't detect mixed inline and regular data extents at all, while lowmem mode reports the problem as hole gap, not to the point.
This patchset will add the ability to detect such problem, with a test image dumped from subpage branch (with the inline extent disable patch reverted).
The patchset is here to detect such problem exposed during subpage development, while also acts as the final safenet to catch such mixed types bug.
Qu Wenruo (4):
btrfs-progs: check/original: add the "0x" prefix for hex error number
btrfs-progs: check/original: detect and report mixed inline and
regular data extents
btrfs-progs: check/lowmem: detect and report mixed inline and regular
extents properly
btrfs-progs: fsck-tests: add test image for mixed inline and regular
data extents
check/main.c | 7 +++- check/mode-lowmem.c | 31 +++++++++++++++--- check/mode-original.h | 2 ++ .../047-mixed-extent-types/default.img.xz | Bin 0 -> 2112 bytes .../fsck-tests/047-mixed-extent-types/test.sh | 19 +++++++++++ 5 files changed, 54 insertions(+), 5 deletions(-) create mode 100644 tests/fsck-tests/047-mixed-extent-types/default.img.xz create mode 100755 tests/fsck-tests/047-mixed-extent-types/test.sh
Signature · 1 line
2.31.1
PATCH 1/4 btrfs-progs: check/original: add the "0x" prefix for hex error number
In print_inode_error(), it print the error number in hex, but without "0x" prefix.
Just add the prefix.
--- check/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
check/main.c +1 -1
@@ -578,7 +578,7 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec) root_objectid = root->root_key.offset;fprintf (stderr,"reloc" ); } -fprintf (stderr,"root %llu inode %llu errors %x" , +fprintf (stderr,"root %llu inode %llu errors 0x%x" , (unsigned long long ) root_objectid, (unsigned long long ) rec->ino, rec->errors);
Signature · 1 line
2.31.1
PATCH 2/4 btrfs-progs: check/original: detect and report mixed inline and regular data extents
In reply to Qu WenruoWhen a btrfs filesystem has mixed inline and regular data extents, btrfs check original mode won't detect it as an error.
Considering how much effort we have done just to avoid such cases, we really want btrfs check to detect such problem.
This error detection is even more important for the incoming btrfs subpage support, as subpage data rw support can cause such problem much easier.
So this patch will just add such ability to original mode.
--- check/main.c | 5 +++++ check/mode-original.h | 2 ++ 2 files changed, 7 insertions(+)
check/main.c +5 -0
@@ -621,6 +621,8 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec) rec->imode & ~07777);if (errors &I_ERR_INVALID_GEN )fprintf (stderr,", invalid inode generation or transid" ); +if (errors &I_ERR_MIXED_EXTENTS ) +fprintf (stderr,", mixed regular and inline extents" );fprintf (stderr,"\n" );/* Print the holes if needed */
@@ -1583,6 +1585,7 @@ static int process_file_extent(struct btrfs_root *root,if (extent_type ==BTRFS_FILE_EXTENT_INLINE ) {struct btrfs_item *item =btrfs_item_nr (slot); + rec->found_inline_extent =1 ; num_bytes =btrfs_file_extent_ram_bytes (eb, fi);if (num_bytes ==0 ) rec->errors |=I_ERR_BAD_FILE_EXTENT ;
@@ -1602,6 +1605,8 @@ static int process_file_extent(struct btrfs_root *root, num_bytes = (num_bytes + mask) & ~mask; }else if (extent_type ==BTRFS_FILE_EXTENT_REG || extent_type ==BTRFS_FILE_EXTENT_PREALLOC ) { +if (rec->found_inline_extent) + rec->errors |=I_ERR_MIXED_EXTENTS ; num_bytes =btrfs_file_extent_num_bytes (eb, fi); disk_bytenr =btrfs_file_extent_disk_bytenr (eb, fi); extent_offset =btrfs_file_extent_offset (eb, fi);
check/mode-original.h +2 -0
@@ -186,6 +186,7 @@ struct unaligned_extent_rec_t {#define I_ERR_MISMATCH_DIR_HASH (1 << 18)#define I_ERR_INVALID_IMODE (1 << 19)#define I_ERR_INVALID_GEN (1 << 20) +#define I_ERR_MIXED_EXTENTS (1 << 21)struct inode_record {struct list_head backrefs;
@@ -197,6 +198,7 @@ struct inode_record { unsigned int found_csum_item:1; unsigned int some_csum_missing:1; unsigned int nodatasum:1; + unsigned int found_inline_extent:1;int errors;struct list_head unaligned_extent_recs;
Signature · 1 line
2.31.1
PATCH 3/4 btrfs-progs: check/lowmem: detect and report mixed inline and regular extents properly
In reply to Qu Wenruo[BUG]
When lowmem mode check is executed on a filesystem with mixed inline and regular extent, it reports the error as file extent gap:
[4/7] checking fs roots
ERROR: root 5 EXTENT_DATA[257 4096] gap exists, expected: EXTENT_DATA[257 707]
ERROR: errors found in fs roots
found 135168 bytes used, error(s) found
Not detecting the mixed extent types.
[CAUSE]
The offending fs has the following data extents:
item 9 key (257 INODE_REF 256) itemoff 15703 itemsize 14
index 2 namelen 4 name: file
item 10 key (257 EXTENT_DATA 0) itemoff 14975 itemsize 728
generation 7 type 0 (inline)
inline extent data size 707 ram_bytes 707 compression 0 (none)
item 11 key (257 EXTENT_DATA 4096) itemoff 14922 itemsize 53
generation 7 type 2 (prealloc)
prealloc data disk byte 102457344 nr 4096 prealloc data offset 0 nr 4096
The mixed extent type error is obvious, but for inline extent, check_file_extent_inline() will increase the extent_end by inline size.
However for such mixed case, the regular extent will only start sectorsize, leaving lowmem check only to report a hole gap.
[FIX]
This patch will fix it by:
- Introduce @found_inline parameter to detect mixed extent types
- Increase @end by sectorsize for check_file_extent_inline()
--- check/mode-lowmem.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-)
check/mode-lowmem.c +27 -4
@@ -2003,7 +2003,12 @@ static int check_file_extent_inline(struct btrfs_root *root, } } } - *end += extent_num_bytes; +/* +* For inline extent, we expect no more extents. But if there are +* regular extents after inlined extent, they should start at +* sectorsize, not the inline size. +*/ + *end += root->fs_info->sectorsize; *size += extent_num_bytes;return err;
@@ -2022,7 +2027,7 @@ static int check_file_extent_inline(struct btrfs_root *root,*/ static int check_file_extent (struct btrfs_root *root,struct btrfs_path *path,unsigned int nodatasum,u64 isize,u64 *size, -u64 *end) +u64 *end,bool *found_inline) {struct btrfs_file_extent_item *fi;struct btrfs_key fkey;
@@ -2058,9 +2063,25 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, }/* Check inline extent */ -if (extent_type ==BTRFS_FILE_EXTENT_INLINE ) +if (extent_type ==BTRFS_FILE_EXTENT_INLINE ) { + *found_inline = true;return check_file_extent_inline (root, path, size, end); + } +/* +* Here we hit a regular/prealloc file extent, if this inode already +* has an inline extent, we have mixed inline and regular extents. +*/ +if (*found_inline) { + err |=FILE_EXTENT_ERROR ; +error ( +"root %llu inode %llu has mixed inline and regular extents" , + root->objectid, fkey.objectid); +/* +* We still want to continue checking, as mixed inline and +* regular is just a minor problem. +*/ + }/* Check REG_EXTENT/PREALLOC_EXTENT */ gen =btrfs_file_extent_generation (node, fi); disk_bytenr =btrfs_file_extent_disk_bytenr (node, fi);
@@ -2606,6 +2627,7 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)int slot;int ret;int err =0 ; +bool found_inline = false;char namebuf[BTRFS_NAME_LEN ] = {0 };u32 name_len =0 ;
@@ -2726,7 +2748,8 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path) key.offset); } ret =check_file_extent (root, path, nodatasum, isize, - &extent_size, &extent_end); + &extent_size, &extent_end, + &found_inline); err |= ret;break ;case BTRFS_XATTR_ITEM_KEY :
Signature · 1 line
2.31.1
PATCH 4/4 btrfs-progs: fsck-tests: add test image for mixed inline and regular data extents
In reply to Qu WenruoTo make sure btrfs check can detect such problem.
The image is dumped by using subpage data rw branch, with inline extent disable patch reverted, running with 4K sectorsize on 64K page size ARM board, with the following command on an empty btrfs filesystem:
fsx -q -l 262144 -o 65536 -S 191110531 -N 9057 -R -W $mnt/file
--- .../047-mixed-extent-types/default.img.xz | Bin 0 -> 2112 bytes .../fsck-tests/047-mixed-extent-types/test.sh | 19 ++++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 tests/fsck-tests/047-mixed-extent-types/default.img.xz create mode 100755 tests/fsck-tests/047-mixed-extent-types/test.sh
tests/fsck-tests/047-mixed-extent-types/default.img.xz new file binary +0 -0
tests/fsck-tests/047-mixed-extent-types/test.sh new file +19 -0
@@ -0,0 +1,19 @@ +#!/bin/bash +# +# Verify that check can detect inline and regular extents inside the same inode +# +# This problem takes us quite much effort in the past, and subpage data write +# can cause such problem more easily. +# +# Make sure btrfs check can at least detect such error + +source "$TEST_TOP/common" + +check_prereq btrfs + +check_image() { + run_mustfail "mixed extent types not detected" \ + "$TOP/btrfs" check "$1" +} + +check_all_images
Signature · 1 line
2.31.1
Quoted 12 lines · Qu Wenruo
Btrfs check original mode can't detect mixed inline and regular data extents at all, while lowmem mode reports the problem as hole gap, not to the point.
This patchset will add the ability to detect such problem, with a test image dumped from subpage branch (with the inline extent disable patch reverted).
The patchset is here to detect such problem exposed during subpage development, while also acts as the final safenet to catch such mixed types bug.
Quoted 13 lines
Qu Wenruo (4):
btrfs-progs: check/original: add the "0x" prefix for hex error number
btrfs-progs: check/original: detect and report mixed inline and
regular data extents
btrfs-progs: check/lowmem: detect and report mixed inline and regular
extents properly
btrfs-progs: fsck-tests: add test image for mixed inline and regular
data extents
For the series,
--
Su
Quoted 12 lines
check/main.c | 7 +++- check/mode-lowmem.c | 31 +++++++++++++++--- check/mode-original.h | 2 ++ .../047-mixed-extent-types/default.img.xz | Bin 0 -> 2112 bytes .../fsck-tests/047-mixed-extent-types/test.sh | 19 +++++++++++ 5 files changed, 54 insertions(+), 5 deletions(-) create mode 100644 tests/fsck-tests/047-mixed-extent-types/default.img.xz create mode 100755
tests/fsck-tests/047-mixed-extent-types/test.sh
Quoted 3 lines · Qu Wenruo
Btrfs check original mode can't detect mixed inline and regular data extents at all, while lowmem mode reports the problem as hole gap, not to the point.
Please discard the patchset.
We don't yet have an agreement on whether inline extent should be exclusive, and since kernel can handle it without any problem, it isn't something that urgent yet.
Furthermore the fix would be way more complex than I thought, as even on x86 we could reproduce such inline + regular extents, it's not that worthy yet.
Thanks,
Qu
Quoted 27 lines
This patchset will add the ability to detect such problem, with a test image dumped from subpage branch (with the inline extent disable patch reverted).
The patchset is here to detect such problem exposed during subpage development, while also acts as the final safenet to catch such mixed types bug.
Qu Wenruo (4):
btrfs-progs: check/original: add the "0x" prefix for hex error number
btrfs-progs: check/original: detect and report mixed inline and
regular data extents
btrfs-progs: check/lowmem: detect and report mixed inline and regular
extents properly
btrfs-progs: fsck-tests: add test image for mixed inline and regular
data extentscheck/main.c | 7 +++- check/mode-lowmem.c | 31 +++++++++++++++--- check/mode-original.h | 2 ++ .../047-mixed-extent-types/default.img.xz | Bin 0 -> 2112 bytes .../fsck-tests/047-mixed-extent-types/test.sh | 19 +++++++++++ 5 files changed, 54 insertions(+), 5 deletions(-)create mode 100644 tests/fsck-tests/047-mixed-extent-types/default.img.xz create mode 100755
tests/fsck-tests/047-mixed-extent-types/test.sh