fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly
PATCH 1/1 fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly
[BUG]
When passing a btrfs with NO_HOLE feature to U-boot, and if one file contains holes, then the hash of the file is not correct in U-boot:
# mkfs.btrfs -f test.img # Since v5.15, mkfs defaults to NO_HOLES # mount test.img /mnt/btrfs # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file # md5sum /mnt/btrfs/file 277f3840b275c74d01e979ea9d75ac19 /mnt/btrfs/file # umount /mnt/btrfs # ./u-boot => host bind 0 /home/adam/test.img => ls host 0 < > 12288 Mon Dec 27 05:35:23 2021 file => load host 0 0x1000000 file 12288 bytes read in 0 ms => md5sum 0x1000000 0x3000 md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1
The md5sum doesn't match at all.
[CAUSE]
In U-boot btrfs implementation, the function btrfs_read_file() has the following iteration for file extent iteration:
/* Read the aligned part */
while (cur < aligned_end) {
ret = lookup_data_extent(root, &path, ino, cur, &next_offset);
if (ret < 0)
goto out;
if (ret > 0) {
/* No next, direct exit */
if (!next_offset) {
ret = 0;
goto out;
}
}
/* Read file extent */But for NO_HOLES features, hole extents will not have any extent item for it.
Thus if @cur is at a hole, lookup_data_extent() will just return >0, and update @next_offset.
But we still believe there is some data to read for @cur for ret > 0 case, causing we read extent data from the next file extent.
This means, what we do for above NO_HOLES btrfs is:
- Read 4K data from disk to file offset [0, 4K) So far the data is still correct
- Read 4K data from disk to file offset [4K, 8K)
We didn't skip the 4K hole, but read the data at file offset [8K, 12K) into file offset [4K, 8K).
This causes the checksum mismatch.
[FIX]
Add extra check to skip to the next non-hole range after lookup_data_extent().
---
This bug exposed another missing link, that we don't have good test coverage in U-boot btrfs.
This is partially caused by the fact that, btrfs-progs code is not designed to read file contents, but just to check the cross-reference (aka, btrfs-check).
If we really only want read-only support in U-boot, and don't ever plan to add write support, then I'd say the btrfs-fuse project
(https://github.com/adam900710/btrfs-fuse/) is more suitable for U-boot.
As that project already has full fs content verification selftest along with extra multi-device recovery tests.
And shares the same code style between btrfs-progs/kernel.
--- fs/btrfs/inode.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/btrfs/inode.c +8 -0
@@ -717,6 +717,14 @@ int btrfs_file_read(struct btrfs_root *root, u64 ino, u64 file_offset, u64 len, ret =0 ; goto out; } +/* +* Find a extent gap, mostly caused by NO_HOLE feature. +* Just to next offset directly. +*/ +if (next_offset > cur) { + cur = next_offset; +continue ; + } } fi =btrfs_item_ptr (path.nodes[0 ], path.slots[0 ], struct btrfs_file_extent_item);
Signature · 1 line
2.34.1
Quoted 70 lines · Qu Wenruo
[BUG]
When passing a btrfs withNO_HOLEfeature to U-boot, and if one file contains holes, then the hash of the file is not correct in U-boot:# mkfs.btrfs -f test.img # Since v5.15, mkfs defaults to NO_HOLES # mount test.img /mnt/btrfs # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file # md5sum /mnt/btrfs/file 277f3840b275c74d01e979ea9d75ac19 /mnt/btrfs/file # umount /mnt/btrfs # ./u-boot => host bind 0 /home/adam/test.img => ls host 0 < > 12288 Mon Dec 27 05:35:23 2021 file => load host 0 0x1000000 file 12288 bytes read in 0 ms => md5sum 0x1000000 0x3000 md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1The md5sum doesn't match at all.
[CAUSE]
In U-boot btrfs implementation, the functionbtrfs_read_file()has the following iteration for file extent iteration:/* Read the aligned part */ while (cur < aligned_end) { ret = lookup_data_extent(root, &path, ino, cur, &next_offset); if (ret < 0) goto out; if (ret > 0) { /* No next, direct exit */ if (!next_offset) { ret = 0; goto out; } } /* Read file extent */But for
NO_HOLESfeatures, hole extents will not have any extent item for it.
Thus if @cur is at a hole,lookup_data_extent()will just return >0, and update @next_offset.But we still believe there is some data to read for @cur for ret > 0 case, causing we read extent data from the next file extent.
This means, what we do for above
NO_HOLESbtrfs is:
- Read 4K data from disk to file offset [0, 4K) So far the data is still correct- Read 4K data from disk to file offset [4K, 8K)
We didn't skip the 4K hole, but read the data at file offset [8K, 12K) into file offset [4K, 8K).This causes the checksum mismatch.
[FIX]
Add extra check to skip to the next non-hole range afterlookup_data_extent().Signed-off-by Qu Wenruo <[email protected]>---
This bug exposed another missing link, that we don't have good test coverage in U-boot btrfs.This is partially caused by the fact that, btrfs-progs code is not designed to read file contents, but just to check the cross-reference (aka, btrfs-check).
Quoted 7 lines
If we really only want read-only support in U-boot, and don't ever plan to add write support, then I'd say the btrfs-fuse project
(https://github.com/adam900710/btrfs-fuse/) is more suitable for U-boot.As that project already has full fs content verification selftest along with extra multi-device recovery tests.
And shares the same code style between btrfs-progs/kernel.
OK, things are not that bad.
In fact, the btrfs_read_file() implementation in btrfs-fuse has the same naming, same lookup_file_extent() (just a little naming different than lookup_data_extent()), same parameter list.
Just without the unaligned sector handling (handled by FUSE, and it may also be unnecessary for U-boot too), and already have the correct handling for lookup_file_extent(), thanks to the selftest.
So this already means, it can be pretty easy for U-boot to take code from btrfs-fuse part by part, without huge refactor again.
Thanks,
Qu
Quoted 23 lines
--- fs/btrfs/inode.c | 8 ++++++++ 1 file changed, 8 insertions(+)fs/btrfs/inode.c +8
-0@@ -717,6 +717,14 @@ int btrfs_file_read(struct btrfs_root *root, u64 ino, u64 file_offset, u64 len, ret =0 ; goto out; } +/* +* Find a extent gap, mostly caused by NO_HOLE feature. +* Just to next offset directly. +*/ +if (next_offset > cur) { + cur = next_offset; +continue ; + } } fi =btrfs_item_ptr (path.nodes[0 ], path.slots[0 ], struct btrfs_file_extent_item);
Quoted 55 lines · Qu Wenruo
[BUG]
When passing a btrfs withNO_HOLEfeature to U-boot, and if one file contains holes, then the hash of the file is not correct in U-boot:# mkfs.btrfs -f test.img # Since v5.15, mkfs defaults to NO_HOLES # mount test.img /mnt/btrfs # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file # md5sum /mnt/btrfs/file 277f3840b275c74d01e979ea9d75ac19 /mnt/btrfs/file # umount /mnt/btrfs # ./u-boot => host bind 0 /home/adam/test.img => ls host 0 < > 12288 Mon Dec 27 05:35:23 2021 file => load host 0 0x1000000 file 12288 bytes read in 0 ms => md5sum 0x1000000 0x3000 md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1The md5sum doesn't match at all.
[CAUSE]
In U-boot btrfs implementation, the functionbtrfs_read_file()has the following iteration for file extent iteration:/* Read the aligned part */ while (cur < aligned_end) { ret = lookup_data_extent(root, &path, ino, cur, &next_offset); if (ret < 0) goto out; if (ret > 0) { /* No next, direct exit */ if (!next_offset) { ret = 0; goto out; } } /* Read file extent */But for
NO_HOLESfeatures, hole extents will not have any extent item for it.
Thus if @cur is at a hole,lookup_data_extent()will just return >0, and update @next_offset.But we still believe there is some data to read for @cur for ret > 0 case, causing we read extent data from the next file extent.
This means, what we do for above
NO_HOLESbtrfs is:
- Read 4K data from disk to file offset [0, 4K) So far the data is still correct- Read 4K data from disk to file offset [4K, 8K)
We didn't skip the 4K hole, but read the data at file offset [8K, 12K) into file offset [4K, 8K).
Quoted 7 lines
This causes the checksum mismatch.
[FIX]
Add extra check to skip to the next non-hole range afterlookup_data_extent().Signed-off-by Qu Wenruo <[email protected]>
Applied to u-boot/master, thanks!
Signature · 1 line
Tom