btrfs-progs: add RAID56 rebuild ability at read time
cover btrfs-progs: add RAID56 rebuild ability at read time
This branch can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/raid56_rebuild
Please note that, the last time I check devel branch, the RAID56 warning patch is not yet merged.
So this is based on the latest devel branch from github.
And since we're adding proper RAID56 repair ability, there is no need
for the patchset "btrfs-progs: check: avoid false alerts for --check-data-csum on RAID56"
This patchset is mostly to add the ability to properly read data/metadata for RAID56 when mirror_num > 1.
[PROBLEMS]
There are several for current btrfs-progs:
- No RAID56 rebuild ability
Thus for any mirror > 1 on RAID56, we will read the parity data straight, causing no way to recover.
- No unified logical read/write entrance
We have various different functions, for read we have read_data_from_disk(), read_extent_data() and read_extent_from_disk().
Unlike kernel, we have no btrfs_map_bio() to address everything.
This makes adding RAID56 rebuild even hard.
[FIXES]
To address the problem, this patchset will:
- Cleanup and refactors
Mostly to unify the logical read/write entrance.
In patch 1~6, we will unify the logical read write entrances to only two functions:
* read_extent_from_disk()
* write_extent_to_disk()
They will do the chunk mapping and stripe splitting.
- Add RAID56 read rebuild ability
We already have RAID56 RMW for write path. Just add a new helper for mirror > 1 read on RAID56.
- A new test case
In fact the same test case from previous RAID56 warning patchset.
Qu Wenruo (8):
btrfs-progs: remove the unnecessary BTRFS_SUPER_INFO_OFFSET path for
tree block read
btrfs-progs: extract metadata restore read code into its own helper
btrfs-progs: don't use write_extent_to_disk() directly
btrfs-progs: use write_data_to_disk() to replace
write_extent_to_disk()
btrfs-progs: use read_data_from_disk() to replace
read_extent_from_disk() and replace read_extent_data()
btrfs-progs: remove extent_buffer::fd and extent_buffer::dev_bytes
btrfs-progs: allow read_data_from_disk() to rebuild RAID56 using P/Q
btrfs-progs: tests/fsck: add test case for data csum check on raid5
btrfs-corrupt-block.c | 38 +--- btrfs-map-logical.c | 5 +- btrfstune.c | 3 +- check/main.c | 2 +- check/mode-common.c | 4 +- cmds/restore.c | 4 +- image/main.c | 2 +- kernel-shared/ctree.c | 5 +- kernel-shared/disk-io.c | 140 +++++-------- kernel-shared/disk-io.h | 2 - kernel-shared/extent_io.c | 188 ++++++++++++------ kernel-shared/extent_io.h | 9 +- kernel-shared/file.c | 20 +- kernel-shared/free-space-cache.c | 20 +- kernel-shared/volumes.c | 39 ++-- kernel-shared/volumes.h | 1 + .../056-raid56-false-alerts/test.sh | 31 +++ 17 files changed, 277 insertions(+), 236 deletions(-) create mode 100755 tests/fsck-tests/056-raid56-false-alerts/test.sh
Signature · 1 line
2.35.1
PATCH 1/8 btrfs-progs: remove the unnecessary BTRFS_SUPER_INFO_OFFSET path for tree block read
We used to use read_whole_eb() to read super block, but it's no longer the case (so long that I can not even find out which commit did the conversion).
Thus there is no need for read_whole_eb() to handle super block read anymore.
--- kernel-shared/disk-io.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
kernel-shared/disk-io.c +1 -2
@@ -302,8 +302,7 @@ int read_whole_eb(struct btrfs_fs_info *info, struct extent_buffer *eb, int mirr read_len = bytes_left; device =NULL ; -if (!info->on_restoring && - eb->start !=BTRFS_SUPER_INFO_OFFSET ) { +if (!info->on_restoring) { ret =btrfs_map_block (info,READ , eb->start + offset, &read_len, &multi, mirror,NULL );if (ret) {
Signature · 1 line
2.35.1
PATCH 8/8 btrfs-progs: tests/fsck: add test case for data csum check on raid5
In reply to Qu WenruoPreviously 'btrfs check --check-data-csum' will report tons of false alerts for RAID56.
Add a test case to make sure with the new RAID56 rebuild ability, there should be no false alerts.
--- .../056-raid56-false-alerts/test.sh | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100755 tests/fsck-tests/056-raid56-false-alerts/test.sh
tests/fsck-tests/056-raid56-false-alerts/test.sh new file +31 -0
@@ -0,0 +1,31 @@ +#!/bin/bash +# +# Make sure "btrfs check --check-data-csum" won't report false alerts on RAID56 +# data. +# + +source "$TEST_TOP/common" + +check_prereq btrfs +check_prereq mkfs.btrfs +check_global_prereq losetup + +setup_loopdevs 3 +prepare_loopdevs +dev1=${loopdevs[1]} +TEST_DEV=$dev1 + +setup_root_helper + +run_check $SUDO_HELPERS "$TOP/mkfs.btrfs" -f -m raid1 -d raid5 ${loopdevs[@]} +run_check_mount_test_dev + +run_check $SUDO_HELPER dd if=/dev/urandom of="$TEST_MNT/file" bs=16K count=1 \ + status=noxfer > /dev/null 2>&1 + +run_check_umount_test_dev + +# Check data csum should not report false alerts +run_check "$TOP/btrfs" check --check-data-csum "$dev1" + +cleanup_loopdevs
Signature · 1 line
2.35.1
PATCH 2/8 btrfs-progs: extract metadata restore read code into its own helper
In reply to Qu WenruoFor metadata restore, our logical address is mapped to a single device with logical address 1:1 mapped to device physical address.
Move this part of code into a helper, this will make later extent buffer read path refactoer much easier.
--- kernel-shared/disk-io.c | 69 +++++++++++++++++++++++++---------------- 1 file changed, 42 insertions(+), 27 deletions(-)
kernel-shared/disk-io.c +42 -27
@@ -288,6 +288,30 @@ out: } +static int read_on_restore (struct extent_buffer *eb) +{ +struct btrfs_fs_info *fs_info = eb->fs_info; +struct btrfs_device *device; +int ret; + +/* +* For on_restoring mode, there should be only one device, and logical +* address is mapped 1:1 to device physical offset. +*/ +list_for_each_entry (device, &fs_info->fs_devices->devices, dev_list) { +if (device->devid ==1 ) +break ; + } + device->total_ios++; + + ret =btrfs_pread (device->fd, eb->data, eb->len, eb->start, + eb->fs_info->zoned); +if (ret != eb->len) + ret = -EIO ; +else + ret =0 ; +return ret; +}int read_whole_eb (struct btrfs_fs_info *info,struct extent_buffer *eb,int mirror) {
@@ -302,38 +326,29 @@ int read_whole_eb(struct btrfs_fs_info *info, struct extent_buffer *eb, int mirr read_len = bytes_left; device =NULL ; -if (!info->on_restoring) { - ret =btrfs_map_block (info,READ , eb->start + offset, - &read_len, &multi, mirror,NULL ); -if (ret) { -printk ("Couldn't map the block %llu\n" , eb->start + offset); -kfree (multi); -return -EIO ; - } - device = multi->stripes[0 ].dev; - -if (device->fd <=0 ) { -kfree (multi); -return -EIO ; - } +if (info->on_restoring) +return read_on_restore (eb); - eb->fd = device->fd; - device->total_ios++; - eb->dev_bytenr = multi->stripes[0 ].physical; + ret =btrfs_map_block (info,READ , eb->start + offset, + &read_len, &multi, mirror,NULL ); +if (ret) { +printk ("Couldn't map the block %llu\n" , eb->start + offset);kfree (multi); - multi =NULL ; - }else { -/* special case for restore metadump */ -list_for_each_entry (device, &info->fs_devices->devices, dev_list) { -if (device->devid ==1 ) -break ; - } +return -EIO ; + } + device = multi->stripes[0 ].dev; - eb->fd = device->fd; - eb->dev_bytenr = eb->start; - device->total_ios++; +if (device->fd <=0 ) { +kfree (multi); +return -EIO ; } + eb->fd = device->fd; + device->total_ios++; + eb->dev_bytenr = multi->stripes[0 ].physical; +kfree (multi); + multi =NULL ; +if (read_len > bytes_left) read_len = bytes_left;
Signature · 1 line
2.35.1
PATCH 4/8 btrfs-progs: use write_data_to_disk() to replace write_extent_to_disk()
In reply to Qu WenruoFunction write_extent_to_disk() is just writing the content of a tree block to disk.
It can not handle RAID56, and its work is the same as write_data_to_disk().
Thus we can replace write_extent_to_disk() with write_data_to_disk() easily.
There is only one special call site in write_raid56_with_parity(), which can easily be replace with btrfs_pwrite() directly.
This reduce the write entrance, and make later eb::fd removal easier.
--- kernel-shared/disk-io.c | 29 +++++++++++++++-------------- kernel-shared/extent_io.c | 17 +---------------- kernel-shared/extent_io.h | 1 - kernel-shared/volumes.c | 4 +++- 4 files changed, 19 insertions(+), 32 deletions(-)
kernel-shared/disk-io.c +15 -14
@@ -510,12 +510,12 @@ err:int write_and_map_eb (struct btrfs_fs_info *fs_info,struct extent_buffer *eb) {int ret; -int dev_nr; +int mirror_num; +int max_mirror;u64 length;u64 *raid_map =NULL ;struct btrfs_multi_bio *multi =NULL ; - dev_nr =0 ; length = eb->len; ret =btrfs_map_block (fs_info,WRITE , eb->start, &length, &multi,0 , &raid_map);
@@ -526,6 +526,7 @@ int write_and_map_eb(struct btrfs_fs_info *fs_info, struct extent_buffer *eb) goto out; } +/* RAID56 write back need RMW */ if (raid_map) { ret =write_raid56_with_parity (fs_info, eb, multi, length, raid_map);
@@ -534,28 +535,28 @@ int write_and_map_eb(struct btrfs_fs_info *fs_info, struct extent_buffer *eb)error ("failed to write raid56 stripe for bytenr %llu length %llu: %m" , eb->start, length); - goto out; } - }else while (dev_nr < multi->num_stripes) { - eb->fd = multi->stripes[dev_nr].dev->fd; - eb->dev_bytenr = multi->stripes[dev_nr].physical; - multi->stripes[dev_nr].dev->total_ios++; - dev_nr++; - ret =write_extent_to_disk (eb); + goto out; + } + +/* For non-RAID56, we just writeback data to each mirror */ + max_mirror =btrfs_num_copies (fs_info, eb->start, eb->len); +for (mirror_num =1 ; mirror_num <= max_mirror; mirror_num++) { + ret =write_data_to_disk (fs_info, eb->data, eb->start, eb->len, + mirror_num);if (ret <0 ) { errno = -ret;error ( -"failed to write bytenr %llu length %u devid %llu dev_bytenr %llu: %m" , - eb->start, eb->len, - multi->stripes[dev_nr].dev->devid, - eb->dev_bytenr); +"failed to write bytenr %llu length %u to mirror %d: %m" , + eb->start, eb->len, mirror_num); goto out; } } + out:kfree (raid_map);kfree (multi); -return 0 ; +return ret; } int write_tree_block(struct btrfs_trans_handle *trans,
kernel-shared/extent_io.c +1 -16
@@ -808,22 +808,6 @@ out:return ret; } -int write_extent_to_disk (struct extent_buffer *eb) -{ -int ret; - ret =btrfs_pwrite (eb->fd, eb->data, eb->len, eb->dev_bytenr, - eb->fs_info->zoned); -if (ret <0 ) - goto out; -if (ret != eb->len) { - ret = -EIO ; - goto out; - } - ret =0 ; -out: -return ret; -} -int read_data_from_disk (struct btrfs_fs_info *info,void *buf,u64 offset,u64 bytes,int mirror) {
@@ -934,6 +918,7 @@ int write_data_to_disk(struct btrfs_fs_info *info, void *buf, u64 offset, dev_bytenr = multi->stripes[dev_nr].physical; this_len =min (this_len, bytes_left); dev_nr++; + device->total_ios++; ret =btrfs_pwrite (device->fd, buf + total_write, this_len, dev_bytenr, info->zoned);
kernel-shared/extent_io.h +0 -1
@@ -152,7 +152,6 @@ void free_extent_buffer(struct extent_buffer *eb);void free_extent_buffer_nocache (struct extent_buffer *eb);int read_extent_from_disk (struct extent_buffer *eb,unsigned long offset,unsigned long len); -int write_extent_to_disk (struct extent_buffer *eb);int memcmp_extent_buffer (const struct extent_buffer *eb,const void *ptrv,unsigned long start,unsigned long len); void read_extent_buffer(const struct extent_buffer *eb, void *dst,
kernel-shared/volumes.c +3 -1
@@ -30,6 +30,7 @@#include "kernel-shared/volumes.h" #include "zoned.h" #include "common/utils.h" +#include "common/device-utils.h" #include "kernel-lib/raid56.h" const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES ] = {
@@ -2719,7 +2720,8 @@ int write_raid56_with_parity(struct btrfs_fs_info *info, }for (i =0 ; i < multi->num_stripes; i++) { - ret =write_extent_to_disk (ebs[i]); + ret =btrfs_pwrite (ebs[i]->fd, ebs[i]->data, ebs[i]->len, + ebs[i]->dev_bytenr, info->zoned);if (ret <0 ) goto out_free_split; }
Signature · 1 line
2.35.1
PATCH 6/8 btrfs-progs: remove extent_buffer::fd and extent_buffer::dev_bytes
In reply to Qu WenruoThose two members are a shortcut for non-RAID56 profiles.
But we should not use such shortcut, and move all our logical address read/write to the unified read_data_from_disk()/write_data_to_disk().
With previous refactors, now we're safe to remove them.
--- btrfs-corrupt-block.c | 5 ++--- kernel-shared/ctree.c | 5 ++--- kernel-shared/extent_io.c | 2 -- kernel-shared/extent_io.h | 2 -- kernel-shared/volumes.c | 12 +++--------- 5 files changed, 7 insertions(+), 19 deletions(-)
btrfs-corrupt-block.c +2 -3
@@ -55,7 +55,7 @@ static int debug_corrupt_block(struct extent_buffer *eb,if (ret <0 ) { errno = -ret;error ("cannot read eb bytenr %llu: %m" , - eb->dev_bytenr); + eb->start);return ret; } printf("corrupting %llu copy %d\n", eb->start,
@@ -65,10 +65,9 @@ static int debug_corrupt_block(struct extent_buffer *eb,if (ret <0 ) { errno = -ret;error ("cannot write eb bytenr %llu: %m" , - eb->dev_bytenr); + eb->start);return ret; } -fsync (eb->fd); } num_copies = btrfs_num_copies(root->fs_info, eb->start,
kernel-shared/ctree.c +2 -3
@@ -590,10 +590,9 @@ static void generic_err(const struct extent_buffer *buf, int slot, {va_list args; -fprintf (stderr,"corrupt %s: root=%lld block=%llu physical=%llu slot=%d, " , +fprintf (stderr,"corrupt %s: root=%lld block=%llu slot=%d, " ,btrfs_header_level (buf) ==0 ?"leaf" :"node" , -btrfs_header_owner (buf),btrfs_header_bytenr (buf), - buf->dev_bytenr, slot); +btrfs_header_owner (buf),btrfs_header_bytenr (buf), slot);va_start (args, fmt);vfprintf (stderr, fmt, args);va_end (args);
kernel-shared/extent_io.c +0 -2
@@ -616,8 +616,6 @@ static struct extent_buffer *__alloc_extent_buffer(struct btrfs_fs_info *info, eb->len = blocksize; eb->refs =1 ; eb->flags =0 ; - eb->fd =-1 ; - eb->dev_bytenr = (u64)-1 ; eb->cache_node.start = bytenr; eb->cache_node.size = blocksize; eb->fs_info = info;
kernel-shared/extent_io.h +0 -2
@@ -88,13 +88,11 @@ struct extent_state {struct extent_buffer {struct cache_extent cache_node;u64 start; -u64 dev_bytenr;struct list_head lru;struct list_head recow;u32 len;int refs;u32 flags; -int fd;struct btrfs_fs_info *fs_info;char data[] __attribute__((aligned (8 ))); };
kernel-shared/volumes.c +3 -9
@@ -2620,8 +2620,6 @@ static int split_eb_for_raid56(struct btrfs_fs_info *info, eb->len = stripe_len; eb->refs =1 ; eb->flags =0 ; - eb->fd =-1 ; - eb->dev_bytenr = (u64)-1 ; eb->fs_info = info; this_eb_start = raid_map[i];
@@ -2676,9 +2674,6 @@ int write_raid56_with_parity(struct btrfs_fs_info *info,for (i =0 ; i < multi->num_stripes; i++) {struct extent_buffer *new_eb;if (raid_map[i] <BTRFS_RAID5_P_STRIPE ) { - ebs[i]->dev_bytenr = multi->stripes[i].physical; - ebs[i]->fd = multi->stripes[i].dev->fd; - multi->stripes[i].dev->total_ios++;if (ebs[i]->start != raid_map[i]) { ret = -EINVAL ; goto out_free_split;
@@ -2690,8 +2685,6 @@ int write_raid56_with_parity(struct btrfs_fs_info *info, ret = -ENOMEM ; goto out_free_split; } - new_eb->dev_bytenr = multi->stripes[i].physical; - new_eb->fd = multi->stripes[i].dev->fd; multi->stripes[i].dev->total_ios++; new_eb->len = stripe_len; new_eb->fs_info = info;
@@ -2720,8 +2713,9 @@ int write_raid56_with_parity(struct btrfs_fs_info *info, }for (i =0 ; i < multi->num_stripes; i++) { - ret =btrfs_pwrite (ebs[i]->fd, ebs[i]->data, ebs[i]->len, - ebs[i]->dev_bytenr, info->zoned); + multi->stripes[i].dev->total_ios++; + ret =btrfs_pwrite (multi->stripes[i].dev->fd, ebs[i]->data, ebs[i]->len, + multi->stripes[i].physical, info->zoned);if (ret <0 ) goto out_free_split; }
Signature · 1 line
2.35.1
PATCH 7/8 btrfs-progs: allow read_data_from_disk() to rebuild RAID56 using P/Q
In reply to Qu WenruoThis new ability is added by:
- Allow btrfs_map_block() to return the chunk type This makes later work much easier
- Only reset stripe offset inside btrfs_map_block() when needed Currently if @raid_map is not NULL, btrfs_map_block() will consider this call is for WRITE and will reset stripe offset.
This is no longer the case, as for RAID56 read with mirror_num 1/0, we will still call btrfs_map_block() with non-NULL raid_map.
Add a small check to make sure we won't reset stripe offset for mirror 1/0 read.
- Add new helper read_raid56() to handle rebuild
We will read the full stripe (including all data and P/Q stripes) do the rebuild, then only copy the refered part to the caller.
There is a catch for RAID6, we have no way to exhaust all combination, so the current repair will assume the mirror = 0 data is corrupted, then try to find a missing device.
But if no missing device can be found, it will assume P is corrupted. This is just a guess, and can to totally wrong, but we have no better idea.
Now btrfs-progs have full read ability for RAID56.
--- kernel-shared/extent_io.c | 114 +++++++++++++++++++++++++++++++++++++- kernel-shared/volumes.c | 27 +++++---- kernel-shared/volumes.h | 1 + 3 files changed, 128 insertions(+), 14 deletions(-)
kernel-shared/extent_io.c +112 -2
@@ -26,6 +26,7 @@#include "kerncompat.h" #include "kernel-shared/extent_io.h" #include "kernel-lib/list.h" +#include "kernel-lib/raid56.h" #include "kernel-shared/ctree.h" #include "kernel-shared/volumes.h" #include "kernel-shared/disk-io.h"
@@ -788,23 +789,131 @@ struct extent_buffer *alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info,return ret; } +static int read_raid56 (struct btrfs_fs_info *fs_info,void *buf,u64 logical, +u64 len,int mirror,struct btrfs_multi_bio *multi, +u64 *raid_map) +{ +const int num_stripes = multi->num_stripes; +const u64 full_stripe_start = raid_map[0 ]; +void **pointers =NULL ; +int failed_a =-1 ; +int failed_b =-1 ; +int i; +int ret; + +/* Only read repair should go this path */ +ASSERT (mirror >1 ); +ASSERT (raid_map); + +/* The read length should be inside one stripe */ +ASSERT (len <=BTRFS_STRIPE_LEN ); + + pointers =calloc (num_stripes,sizeof (void *)); +if (!pointers) { + ret = -ENOMEM ; + goto out; + } +/* Allocate memory for the full stripe */ +for (i =0 ; i < num_stripes; i++) { + pointers[i] =malloc (BTRFS_STRIPE_LEN ); +if (!pointers[i]) { + ret = -ENOMEM ; + goto out; + } + } + +/* +* Read the full stripe. +* +* The stripes in @multi is not rotated, thus can be used to read from +* disk directly. +*/ +for (i =0 ; i < num_stripes; i++) { + ret =btrfs_pread (multi->stripes[i].dev->fd, pointers[i], +BTRFS_STRIPE_LEN , multi->stripes[i].physical, + fs_info->zoned); +if (ret <BTRFS_STRIPE_LEN ) { + ret = -EIO ; + goto out; + } + } + +/* +* Get the failed index. +* +* Since we're reading using mirror_num > 1 already, it means the data +* stripe where @logical lies in is definitely corrupted. +*/ + failed_a = (logical - full_stripe_start) /BTRFS_STRIPE_LEN ; + +/* +* For RAID6, we don't have good way to exhaust all the combinations, +* so here we can only go through the map to see if we have missing devices. +*/ +if (multi->type &BTRFS_BLOCK_GROUP_RAID6 ) { +for (i =0 ; i < num_stripes; i++) { +/* Skip failed_a, as it's already marked failed */ +if (i == failed_a) +continue ; +/* Missing dev */ +if (multi->stripes[i].dev->fd ==-1 ) { + failed_b = i; +break ; + } + } +/* +* No missing device, we have no better idea, default to P +* corruption +*/ +if (failed_b <0 ) + failed_b = num_stripes -2 ; + } + +/* Rebuild the full stripe */ + ret =raid56_recov (num_stripes,BTRFS_STRIPE_LEN , multi->type, + failed_a, failed_b, pointers); +ASSERT (ret ==0 ); + +/* Now copy the data back to original buf */ +memcpy (buf, pointers[failed_a] + (logical - full_stripe_start) % +BTRFS_STRIPE_LEN , len); + ret =0 ; +out: +for (i =0 ; i < num_stripes; i++) +free (pointers[i]); +free (pointers); +return ret; +} +int read_data_from_disk (struct btrfs_fs_info *info,void *buf,u64 logical,u64 *len,int mirror) {struct btrfs_multi_bio *multi =NULL ;struct btrfs_device *device;u64 read_len = *len; +u64 *raid_map =NULL ;int ret; ret =btrfs_map_block (info,READ , logical, &read_len, &multi, mirror, -NULL ); + &raid_map);if (ret) {fprintf (stderr,"Couldn't map the block %llu\n" , logical);return -EIO ; } + read_len =min (*len, read_len); + +/* We need to rebuild from P/Q */ +if (mirror >1 && multi->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { + ret =read_raid56 (info, buf, logical, read_len, mirror, multi, + raid_map); +free (multi); +free (raid_map); + *len = read_len; +return ret; + } +free (raid_map); device = multi->stripes[0 ].dev; - read_len =min (*len, read_len);if (device->fd <=0 ) {kfree (multi);return -EIO ;
@@ -824,6 +933,7 @@ int read_data_from_disk(struct btrfs_fs_info *info, void *buf, u64 logical, logical, ret, read_len);return -EIO ; } + *len = read_len;return 0 ; }
kernel-shared/volumes.c +15 -12
@@ -1805,6 +1805,7 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw,int stripes_required =1 ;int stripe_index;int i; +bool need_raid_map = false;struct btrfs_multi_bio *multi =NULL ;if (multi_ret && rw ==READ ) {
@@ -1842,17 +1843,18 @@ again: }if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK && multi_ret && ((rw &WRITE ) || mirror_num >1 ) && raid_map_ret) { -/* RAID[56] write or recovery. Return all stripes */ - stripes_required = map->num_stripes; - -/* Only allocate the map if we've already got a large enough multi_ret */ -if (stripes_allocated >= stripes_required) { - raid_map =kmalloc (sizeof (u64) * map->num_stripes,GFP_NOFS ); -if (!raid_map) { -kfree (multi); -return -ENOMEM ; - } - } + need_raid_map = true; +/* RAID[56] write or recovery. Return all stripes */ + stripes_required = map->num_stripes; + +/* Only allocate the map if we've already got a large enough multi_ret */ +if (stripes_allocated >= stripes_required) { + raid_map =kmalloc (sizeof (u64) * map->num_stripes,GFP_NOFS ); +if (!raid_map) { +kfree (multi); +return -ENOMEM ; + } + } }/* if our multi bio struct is too small, back off and try again */
@@ -1890,6 +1892,7 @@ again: goto out; multi->num_stripes =1 ; + multi->type = map->type; stripe_index =0 ;if (map->type &BTRFS_BLOCK_GROUP_RAID1_MASK ) {if (rw ==WRITE )
@@ -1916,7 +1919,7 @@ again:else if (mirror_num) stripe_index = mirror_num -1 ; }else if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { -if (raid_map) { +if (need_raid_map && raid_map) {int rot;u64 tmp;u64 raid56_full_stripe_start;
kernel-shared/volumes.h +1 -0
@@ -106,6 +106,7 @@ struct btrfs_bio_stripe { };struct btrfs_multi_bio { +u64 type;int error;int num_stripes;struct btrfs_bio_stripe stripes[];
Signature · 1 line
2.35.1
PATCH 3/8 btrfs-progs: don't use write_extent_to_disk() directly
In reply to Qu WenruoThere are two call sites using write_extent_to_disk() directly:
- debug_corrupt_block() in btrfs-corrupt-block.c
- corrupt_keys() in btrfs-corrupt-block.c
The problem of write_extent_to_disk() is, it can only handle plain profiles (All profiles except P/Q stripes of RAID56).
Calling it directly can corrupted RAID56 P/Q, and in the future we're going to remove eb::fd/eb::dev_bytes, so remove such call sites with write_and_map_eb().
--- btrfs-corrupt-block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
btrfs-corrupt-block.c +2 -2
@@ -76,7 +76,7 @@ static int debug_corrupt_block(struct extent_buffer *eb,printf ("corrupting %llu copy %d\n" , eb->start, mirror_num);memset (eb->data,0 , eb->len); - ret =write_extent_to_disk (eb); + ret =write_and_map_eb (eb->fs_info, eb);if (ret <0 ) { errno = -ret; error("cannot write eb bytenr %llu: %m",
@@ -159,7 +159,7 @@ static void corrupt_keys(struct btrfs_trans_handle *trans,u16 csum_type = fs_info->csum_type;csum_tree_block_size (eb, csum_size,0 , csum_type); -write_extent_to_disk (eb); +write_and_map_eb (eb->fs_info, eb); } }
Signature · 1 line
2.35.1
PATCH 5/8 btrfs-progs: use read_data_from_disk() to replace read_extent_from_disk() and replace read_extent_data()
In reply to Qu WenruoThe function read_extent_from_disk() is only a wrapper to read tree block.
And read_extent_data() is just a while loop to eliminate short read caused by stripe boundary.
In fact, a lot of call sites of read_extent_data() are either reading metadata (thus no possible short read) or doing extra loop by themselves.
This patch will replace those two functions with read_data_from_disk(), making it the only entrance for data/metadata read.
And update read_data_from_disk() to return the read bytes, so caller can do a simple while loop.
For the few callers of read_extent_data(), open-code a small while loop for them.
This will allow later RAID56 read repair using P/Q much easier.
--- btrfs-corrupt-block.c | 29 +++-------- btrfs-map-logical.c | 5 +- btrfstune.c | 3 +- check/main.c | 2 +- check/mode-common.c | 4 +- cmds/restore.c | 4 +- image/main.c | 2 +- kernel-shared/disk-io.c | 73 +++------------------------- kernel-shared/disk-io.h | 2 - kernel-shared/extent_io.c | 83 +++++++++++--------------------- kernel-shared/extent_io.h | 6 +-- kernel-shared/file.c | 20 ++++---- kernel-shared/free-space-cache.c | 20 ++++++-- 13 files changed, 80 insertions(+), 173 deletions(-)
btrfs-corrupt-block.c +7 -22
@@ -40,33 +40,18 @@ static int debug_corrupt_block(struct extent_buffer *eb, struct btrfs_root *root, u64 bytenr, u32 blocksize, u64 copy) { int ret; -u64 length; -struct btrfs_multi_bio *multi =NULL ; -struct btrfs_device *device;int num_copies;int mirror_num =1 ; - length = blocksize;while (1 ) { - ret =btrfs_map_block (root->fs_info,READ , eb->start, &length, - &multi, mirror_num,NULL ); -if (ret) { -error ("cannot map block %llu length %llu mirror %d: %d" , - eb->start, length, mirror_num, ret); -return ret; - } - device = multi->stripes[0 ].dev; - eb->fd = device->fd; - device->total_ios++; - eb->dev_bytenr = multi->stripes[0 ].physical; - -fprintf (stdout, -"mirror %d logical %llu physical %llu device %s\n" , - mirror_num, bytenr, eb->dev_bytenr, device->name); -free (multi); -if (!copy || mirror_num == copy) { - ret =read_extent_from_disk (eb,0 , eb->len); +u64 read_len = eb->len; + + ret =read_data_from_disk (eb->fs_info, eb->data, + eb->start, &read_len, + mirror_num); +if (read_len < eb->len) + ret = -EIO ;if (ret <0 ) { errno = -ret; error("cannot read eb bytenr %llu: %m",
btrfs-map-logical.c +3 -2
@@ -173,8 +173,9 @@ static int write_extent_content(struct btrfs_fs_info *fs_info, int out_fd,while (cur_offset < length) { cur_len =min_t (u64, length - cur_offset,BUFFER_SIZE ); - ret =read_extent_data (fs_info, buffer, - logical + cur_offset, &cur_len, mirror); + ret =read_data_from_disk (fs_info, buffer, + logical + cur_offset, &cur_len, + mirror);if (ret <0 ) { errno = -ret; fprintf(stderr,
btrfstune.c +2 -1
@@ -333,7 +333,8 @@ static int populate_csum(struct btrfs_trans_handle *trans,while (offset < len) { sectorsize = fs_info->sectorsize; - ret =read_extent_data (fs_info, buf, start + offset, §orsize,0 ); + ret =read_data_from_disk (fs_info, buf, start + offset, + §orsize,0 );if (ret)break ; ret = btrfs_csum_file_block(trans, start + len, start + offset,
check/main.c +1 -1
@@ -5757,7 +5757,7 @@ static int check_extent_csums(struct btrfs_root *root, u64 bytenr,for (mirror =1 ; mirror <= num_copies; mirror++) { read_len = num_bytes - offset;/* read as much space once a time */ - ret =read_extent_data (gfs_info, (char *)data + offset, + ret =read_data_from_disk (gfs_info, (char *)data + offset, bytenr + offset, &read_len, mirror);if (ret) goto out;
check/mode-common.c +2 -2
@@ -1202,8 +1202,8 @@ static int populate_csum(struct btrfs_trans_handle *trans,while (offset < len) { sectorsize = gfs_info->sectorsize; - ret =read_extent_data (gfs_info, buf, start + offset, - §orsize,0 ); + ret =read_data_from_disk (gfs_info, buf, start + offset, + §orsize,0 );if (ret)break ; ret = btrfs_csum_file_block(trans, start + len, start + offset,
cmds/restore.c +2 -2
@@ -407,8 +407,8 @@ again: cur = bytenr;while (cur < bytenr + size_left) { length = bytenr + size_left - cur; - ret =read_extent_data (root->fs_info, inbuf + cur - bytenr, cur, - &length, mirror_num); + ret =read_data_from_disk (root->fs_info, inbuf + cur - bytenr, cur, + &length, mirror_num);if (ret <0 ) { mirror_num++;if (mirror_num > num_copies) {
image/main.c +1 -1
@@ -615,7 +615,7 @@ static int read_data_extent(struct metadump_struct *md,for (cur_mirror =1 ; cur_mirror <= num_copies; cur_mirror++) {while (bytes_left) { read_len = bytes_left; - ret =read_extent_data (fs_info, + ret =read_data_from_disk (fs_info, (char *)(async->buffer + offset), logical, &read_len, cur_mirror);if (ret <0 )
kernel-shared/disk-io.c +6 -67
93 lines not on this page · raw message
kernel-shared/disk-io.h +0 -2
@@ -141,8 +141,6 @@ int read_whole_eb(struct btrfs_fs_info *info, struct extent_buffer *eb, int mirrstruct extent_buffer *read_tree_block (struct btrfs_fs_info *fs_info,u64 bytenr,u64 parent_transid); -int read_extent_data (struct btrfs_fs_info *fs_info,char *data,u64 logical, -u64 *len,int mirror);void readahead_tree_block (struct btrfs_fs_info *fs_info,u64 bytenr,u64 parent_transid); struct extent_buffer* btrfs_find_create_tree_block(
kernel-shared/extent_io.c +28 -55
103 lines not on this page · raw message
kernel-shared/extent_io.h +2 -4
@@ -150,8 +150,6 @@ struct extent_buffer *alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, u64 bytenr, u32 blocksize);void free_extent_buffer (struct extent_buffer *eb);void free_extent_buffer_nocache (struct extent_buffer *eb); -int read_extent_from_disk (struct extent_buffer *eb, -unsigned long offset,unsigned long len);int memcmp_extent_buffer (const struct extent_buffer *eb,const void *ptrv,unsigned long start,unsigned long len); void read_extent_buffer(const struct extent_buffer *eb, void *dst,
@@ -169,8 +167,8 @@ int extent_buffer_test_bit(struct extent_buffer *eb, unsigned long start, unsigned long nr);int set_extent_buffer_dirty (struct extent_buffer *eb);int clear_extent_buffer_dirty (struct extent_buffer *eb); -int read_data_from_disk (struct btrfs_fs_info *info,void *buf,u64 offset, -u64 bytes,int mirror); +int read_data_from_disk (struct btrfs_fs_info *info,void *buf,u64 logical, +u64 *len,int mirror);int write_data_to_disk (struct btrfs_fs_info *info,void *buf,u64 offset,u64 bytes,int mirror); void extent_buffer_bitmap_clear(struct extent_buffer *eb, unsigned long start,
kernel-shared/file.c +11 -9
@@ -225,11 +225,11 @@ int btrfs_read_file(struct btrfs_root *root, u64 ino, u64 start, int len,memset (dest,0 , len);while (1 ) {struct btrfs_file_extent_item *fi; +u64 offset =0 ;u64 extent_start;u64 extent_len;u64 read_start;u64 read_len; -u64 read_len_ret;u64 disk_bytenr; leaf = path.nodes[0 ];
@@ -282,14 +282,16 @@ int btrfs_read_file(struct btrfs_root *root, u64 ino, u64 start, int len, disk_bytenr =btrfs_file_extent_disk_bytenr (leaf, fi) +btrfs_file_extent_offset (leaf, fi); - read_len_ret = read_len; - ret =read_extent_data (fs_info, dest + read_start - start, disk_bytenr, - &read_len_ret,0 ); -if (ret <0 ) -break ; -/* Short read, something went wrong */ -if (read_len_ret != read_len) -return -EIO ; +while (offset < read_len) { +u64 read_len_ret = read_len - offset; + + ret =read_data_from_disk (fs_info, + dest + read_start - start + offset, + disk_bytenr + offset, &read_len_ret,0 ); +if (ret <0 ) + goto out; + offset += read_len_ret; + } read += read_len; next: ret =btrfs_next_item (root, &path);
kernel-shared/free-space-cache.c +15 -5
@@ -118,6 +118,8 @@ static int io_ctl_prepare_pages(struct io_ctl *io_ctl, struct btrfs_root *root, }while (total_read < io_ctl->total_size) { +u64 offset =0 ; +if (path->slots[0 ] >=btrfs_header_nritems (path->nodes[0 ])) { ret =btrfs_next_leaf (root, path);if (ret) {
@@ -150,11 +152,19 @@ static int io_ctl_prepare_pages(struct io_ctl *io_ctl, struct btrfs_root *root, bytenr =btrfs_file_extent_disk_bytenr (leaf, fi) +btrfs_file_extent_offset (leaf, fi); len =btrfs_file_extent_num_bytes (leaf, fi); - ret =read_data_from_disk (root->fs_info, - io_ctl->buffer + key.offset, bytenr, - len,0 ); -if (ret) -break ; +while (offset < len) { +u64 read_len = len - offset; + + ret =read_data_from_disk (root->fs_info, + io_ctl->buffer + key.offset + offset, + bytenr + offset, + &read_len,0 ); +if (ret <0 ) { +btrfs_release_path (path); +return ret; + } + offset += read_len; + } total_read += len; path->slots[0 ]++; }
Signature · 1 line
2.35.1
Quoted 5 lines · Qu Wenruo
This branch can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/raid56_rebuildPlease note that, the last time I check devel branch, the RAID56 warning patch is not yet merged.
It was there but I had not the branch pushed.
Quoted 4 lines
So this is based on the latest devel branch from github.
And since we're adding proper RAID56 repair ability, there is no need
for the patchset "btrfs-progs: check: avoid false alerts for --check-data-csum on RAID56"
OK so I'll replace it with this patchset.
Quoted 2 lines · Qu Wenruo
This branch can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/raid56_rebuild
The mkfs tests fail in 001 with the following last command:
====== RUN CHECK root_helper .../mkfs.btrfs -f -d raid5 -m raid5 /dev/loop0 /dev/loop1 /dev/loop2 /dev/loop3
WARNING: RAID5/6 support has known problems is strongly discouraged
to be used besides testing or evaluation.
kernel-shared/transaction.c:155: __commit_transaction: BUG_ON ret triggered, value 65536 .../mkfs.btrfs(__commit_transaction+0x197)[0x43a107] /...mkfs.btrfs(btrfs_commit_transaction+0x13b)[0x43a26b] /...mkfs.btrfs(main+0x174b)[0x40f68d]
/lib64/libc.so.6(+0x2d540)[0x7f969b5ea540] /lib64/libc.so.6(__libc_start_main+0x7c)[0x7f969b5ea5ec] /...mkfs.btrfs(_start+0x25)[0x40d965]
/...tests/common: line 540: 29172 Aborted sudo -n "$@"
failed: root_helper .../mkfs.btrfs -f -d raid5 -m raid5 /dev/loop0 /dev/loop1 /dev/loop2 /dev/loop3 test failed for case 001-basic-profiles
Quoted 10 lines · David Sterba
Quoted 2 lines · Qu Wenruo
This branch can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/raid56_rebuildThe mkfs tests fail in 001 with the following last command:
====== RUN CHECK
root_helper.../mkfs.btrfs -f -d raid5 -m raid5/dev/loop0/dev/loop1/dev/loop2/dev/loop3
WARNING: RAID5/6 support has known problems is strongly discouraged
to be used besides testing or evaluation.
Quoted 10 lines
kernel-shared/transaction.c:155:__commit_transaction:BUG_ONrettriggered, value 65536 .../mkfs.btrfs(__commit_transaction+0x197)[0x43a107] /...mkfs.btrfs(btrfs_commit_transaction+0x13b)[0x43a26b] /...mkfs.btrfs(main+0x174b)[0x40f68d]
/lib64/libc.so.6(+0x2d540)[0x7f969b5ea540] /lib64/libc.so.6(__libc_start_main+0x7c)[0x7f969b5ea5ec] /...mkfs.btrfs(_start+0x25)[0x40d965]
/...tests/common: line 540: 29172 Aborted sudo -n "$@"
failed:root_helper.../mkfs.btrfs -f -d raid5 -m raid5/dev/loop0/dev/loop1/dev/loop2/dev/loop3test failed for case 001-basic-profiles
It's caused by patch [PATCH 4/8] btrfs-progs: use write_data_to_disk() to replace write_extent_to_disk()
Quoted 22 lines · David Sterba
Quoted 10 lines · David Sterba
Quoted 2 lines · Qu Wenruo
This branch can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/raid56_rebuildThe mkfs tests fail in 001 with the following last command:
====== RUN CHECK
root_helper.../mkfs.btrfs -f -d raid5 -m raid5/dev/loop0/dev/loop1/dev/loop2/dev/loop3
WARNING: RAID5/6 support has known problems is strongly discouraged
to be used besides testing or evaluation.Quoted 10 lines
kernel-shared/transaction.c:155:__commit_transaction:BUG_ONrettriggered, value 65536 .../mkfs.btrfs(__commit_transaction+0x197)[0x43a107] /...mkfs.btrfs(btrfs_commit_transaction+0x13b)[0x43a26b] /...mkfs.btrfs(main+0x174b)[0x40f68d]
/lib64/libc.so.6(+0x2d540)[0x7f969b5ea540] /lib64/libc.so.6(__libc_start_main+0x7c)[0x7f969b5ea5ec] /...mkfs.btrfs(_start+0x25)[0x40d965]
/...tests/common: line 540: 29172 Aborted sudo -n "$@"
failed:root_helper.../mkfs.btrfs -f -d raid5 -m raid5/dev/loop0/dev/loop1/dev/loop2/dev/loop3test failed for case 001-basic-profiles
Quoted 2 lines
It's caused by patch [PATCH 4/8] btrfs-progs: use
write_data_to_disk()to replacewrite_extent_to_disk()
That's a known one, I forgot to change the return value back to 0 for write_extent_to_disk().
Do I need to update the whole series or just send an update based on this?
Thanks,
Qu