btrfs: read-repair rework based on bitmap
Subsystems (2)
v2 cover btrfs: read-repair rework based on bitmap
This is the bitmap version revivied, and based on Christoph's cleanup series.
The branch can be feteched from my repo:
https://github.com/adam900710/linux/tree/read_repair
Changelog:
v2:
- Still go bitmap version to get batched submission
- Instead of using the old failed bio to hold all the pages, here we use an oridnary bio to hold only the corrupted sectors.
- Unlike previous bitmap version, this time no memory allocation The needed bitmap will be on-stack, and only limit to 32bits.
- Go synchronous submission for read and write
- Only use bitmap for the corrupted range.
Unlike previous version, which allocate bio for the whole bio. That's a waste of memory.
- btrfs_read_repair_sector() will submit the existing repair early For uncontinuous new sector or if we have reached size limit (due to bitmap size limit, now it's 128K for 4K sectorsize).
This allow us to use fixed bitmap size.
The core function is still btrfs_read_repair_finish().
But now, btrfs_read_repair_finish() either get called without any corrupted sectors, or it will only face a continuous range of corrupted sectors.
Then we handle the range by iterating all the remaining mirrors.
For each mirror we go:
1) Try to add current bad sector into our io_bio.
If our io_bio is not continuous, we just submit current io_bio and wait for it.
Then add the new sector into it.
2) If the io_bio is not empty, submit it.
By 1) and 2), we will read all bad sectors from the new mirror.
3) Check if the data is fine and update our ctrl::bad_bitmap
We either end with all sectors repaired, or all mirrors exhausted.
The advantage of bitmap method is, we only try at most (num_copies - 1) times, no matter the corruption pattern.
On the other hand, for the worst case we're still doing sector by sector repair.
For the best case (aka, continuous corruption cases), we still do batched bio submission, thus still way better than sector-by-sector repair.
Furthermore, all loops in the code are regular for() loops, no hacking on how we loop.
But I have to admit, even the repair_from_mirror() and btrfs_read_repair_finish() is super easy to read, the details on bio page adding and submission are all hidden into io_add_or_submit() and btrfs_read_repair_add_sector().
Although it has better worst case performance, it's no better than sector-by-sector repair in worst case scenario.
Christoph Hellwig (1):
btrfs: add a btrfs_map_bio_wait helper
Qu Wenruo (6):
btrfs: save the original bi_iter into btrfs_bio for buffered read
btrfs: make repair_io_failure available outside of extent_io.c
btrfs: introduce new read-repair infrastructure
btrfs: make buffered read path to use the new read repair
infrastructure
btrfs: make direct io read path to use the new read repair
infrastructure
btrfs: remove io_failure_record infrastructure completely
fs/btrfs/Makefile | 2 +- fs/btrfs/btrfs_inode.h | 5 - fs/btrfs/extent-io-tree.h | 15 -- fs/btrfs/extent_io.c | 436 +++-------------------------------- fs/btrfs/extent_io.h | 28 +-- fs/btrfs/inode.c | 60 ++--- fs/btrfs/read-repair.c | 328 ++++++++++++++++++++++++++ fs/btrfs/read-repair.h | 48 ++++ fs/btrfs/volumes.c | 21 ++ fs/btrfs/volumes.h | 2 + include/trace/events/btrfs.h | 1 - 11 files changed, 458 insertions(+), 488 deletions(-) create mode 100644 fs/btrfs/read-repair.c create mode 100644 fs/btrfs/read-repair.h
Signature · 1 line
2.36.1
PATCH v2 1/7 btrfs: save the original bi_iter into btrfs_bio for buffered read
Although we have btrfs_bio::iter, it currently have very limited usage:
- RAID56
Which is not needed at all
- btrfs_bio_clone()
This is used mostly for direct IO.
For the incoming read repair patches, we want to grab the original logical bytenr, and be able to iterate the range of the bio (no matter if it's cloned).
So this patch will also save btrfs_bio::iter for buffered read bios at submit_one_bio().
And for the sake of consistency, also save the btrfs_bio::iter for direct IO at btrfs_submit_dio_bio().
The reason that we didn't save the iter in btrfs_map_bio() is, btrfs_map_bio() is going to handle various bios, with or without btrfs_bio bioset.
And we want to keep btrfs_map_bio() to handle and only handle plain bios without bother the bioset.
--- fs/btrfs/extent_io.c | 7 +++++++ fs/btrfs/inode.c | 2 ++ 2 files changed, 9 insertions(+)
fs/btrfs/extent_io.c +7 -0
@@ -188,6 +188,13 @@ static void submit_one_bio(struct bio *bio, int mirror_num,/* Caller should ensure the bio has at least some range added */ ASSERT (bio->bi_iter.bi_size); +/* +* Save the original bi_iter for read bios, as read repair wants the +* orignial logical bytenr. +*/ +if (bio_op (bio) ==REQ_OP_READ ) +btrfs_bio (bio)->iter = bio->bi_iter; +if (is_data_inode (tree->private_data))btrfs_submit_data_bio (tree->private_data, bio, mirror_num, compress_type);
fs/btrfs/inode.c +2 -0
@@ -7974,6 +7974,8 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio, ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA );if (ret) goto err; +/* Check submit_one_bio() for the reason. */ +btrfs_bio (bio)->iter = bio->bi_iter; }if (BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM )
Signature · 1 line
2.36.1
PATCH v2 2/7 btrfs: make repair_io_failure available outside of extent_io.c
In reply to Qu WenruoRemove the static so that the function can be used by the new read repair code, and give it a btrfs_ prefix.
--- fs/btrfs/extent_io.c | 19 ++++++++++--------- fs/btrfs/extent_io.h | 3 +++ 2 files changed, 13 insertions(+), 9 deletions(-)
fs/btrfs/extent_io.c +10 -9
@@ -2321,9 +2321,9 @@ int free_io_failure(struct extent_io_tree *failure_tree,* currently, there can be no more than two copies of every data bit. thus, * exactly one rewrite is required. */ -static int repair_io_failure (struct btrfs_fs_info *fs_info,u64 ino,u64 start, -u64 length,u64 logical,struct page *page, -unsigned int pg_offset,int mirror_num) +int btrfs_repair_io_failure (struct btrfs_fs_info *fs_info,u64 ino,u64 start, +u64 length,u64 logical,struct page *page, +unsigned int pg_offset,int mirror_num) {struct btrfs_device *dev;struct bio_vec bvec;
@@ -2415,8 +2415,9 @@ int btrfs_repair_eb_io_failure(const struct extent_buffer *eb, int mirror_num)for (i =0 ; i < num_pages; i++) {struct page *p = eb->pages[i]; - ret =repair_io_failure (fs_info,0 , start,PAGE_SIZE , start, p, - start -page_offset (p), mirror_num); + ret =btrfs_repair_io_failure (fs_info,0 , start,PAGE_SIZE , + start, p, start -page_offset (p), + mirror_num);if (ret)break ; start +=PAGE_SIZE ;
@@ -2466,9 +2467,9 @@ int clean_io_failure(struct btrfs_fs_info *fs_info, num_copies =btrfs_num_copies (fs_info, failrec->logical, failrec->len);if (num_copies >1 ) { -repair_io_failure (fs_info, ino, start, failrec->len, - failrec->logical, page, pg_offset, - failrec->failed_mirror); +btrfs_repair_io_failure (fs_info, ino, start, + failrec->len, failrec->logical, + page, pg_offset, failrec->failed_mirror); } }
@@ -2626,7 +2627,7 @@ static bool btrfs_check_repairable(struct inode *inode,* * Since we're only doing repair for one sector, we only need to get * a good copy of the failed sector and if we succeed, we have setup -* everything for repair_io_failure to do the rest for us. +* everything for btrfs_repair_io_failure() to do the rest for us. */ ASSERT (failed_mirror); failrec->failed_mirror = failed_mirror;
fs/btrfs/extent_io.h +3 -0
@@ -276,6 +276,9 @@ int btrfs_repair_one_sector(struct inode *inode, struct page *page, unsigned int pgoff, u64 start, int failed_mirror, submit_bio_hook_t *submit_bio_hook); +int btrfs_repair_io_failure (struct btrfs_fs_info *fs_info,u64 ino,u64 start, +u64 length,u64 logical,struct page *page, +unsigned int pg_offset,int mirror_num);#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS bool find_lock_delalloc_range(struct inode *inode,
Signature · 1 line
2.36.1
PATCH v2 3/7 btrfs: add a btrfs_map_bio_wait helper
In reply to Qu WenruoFrom: Christoph Hellwig <[email protected]>
This helpers works like submit_bio_wait, but goes through the btrfs bio mapping using btrfs_map_bio.
--- fs/btrfs/volumes.c | 21 +++++++++++++++++++++ fs/btrfs/volumes.h | 2 ++ 2 files changed, 23 insertions(+)
fs/btrfs/volumes.c +21 -0
@@ -6818,6 +6818,27 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,return BLK_STS_OK ; } +static void btrfs_end_io_sync (struct bio *bio) +{ +complete (bio->bi_private); +} + +blk_status_t btrfs_map_bio_wait (struct btrfs_fs_info *fs_info,struct bio *bio, +int mirror) +{ +DECLARE_COMPLETION_ONSTACK (done); +blk_status_t ret; + + bio->bi_private = &done; + bio->bi_end_io = btrfs_end_io_sync; + ret =btrfs_map_bio (fs_info, bio, mirror); +if (ret) +return ret; + +wait_for_completion_io (&done); +return bio->bi_status; +} +static bool dev_args_match_fs_devices (const struct btrfs_dev_lookup_args *args,const struct btrfs_fs_devices *fs_devices) {
fs/btrfs/volumes.h +2 -0
@@ -555,6 +555,8 @@ struct btrfs_block_group *btrfs_create_chunk(struct btrfs_trans_handle *trans,void btrfs_mapping_tree_free (struct extent_map_tree *tree);blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num); +blk_status_t btrfs_map_bio_wait (struct btrfs_fs_info *fs_info,struct bio *bio, +int mirror);int btrfs_open_devices (struct btrfs_fs_devices *fs_devices,fmode_t flags,void *holder); struct btrfs_device *btrfs_scan_one_device(const char *path,
Signature · 1 line
2.36.1
PATCH v2 4/7 btrfs: introduce new read-repair infrastructure
In reply to Qu WenruoThe new read repair infrastructure is consist of the following 3 parts:
- btrfs_read_repair_ctrl
Record a continous corrupted range.
Will mostly be on-stack structure for the top-level endio function.
- btrfs_read_repair_add_sector()
This function is called each time we hit a bad sector.
This function itself will check if the bad sector can be merged with the existing bad range.
If not, call btrfs_read_repair_finish() to finish the current range first, and then add the new sector into the now empty btrfs_read_repair_ctrl.
Will return -EIO if there is any range failed to be repaired.
- btrfs_read_repair_finish()
This function should be called before the endio function exit.
This function will iterate through all the mirrors, trying to grab the correct data.
If we grabbed a correct sector, we will queue it for later writeback into the bad mirror.
To hold the original bad sectors, we have two bios, one named @bad_sectors. Although it's a bio, we only utilize the bio_vec infrastructure to hold all the initial bad sectors.
It's the @io_bio we really utilize to submit new read and write.
For io_bio, the usage is pretty much the same asbtrfs_read_repair_add_sector().
If we can merge the target sector, then that's the best case. If not, then we submit the current @io_bio, wait for it, and allocate a new bio for the next usage.
--- fs/btrfs/Makefile | 2 +- fs/btrfs/extent_io.c | 2 +- fs/btrfs/extent_io.h | 1 + fs/btrfs/read-repair.c | 328 +++++++++++++++++++++++++++++++++++++++++ fs/btrfs/read-repair.h | 48 ++++++ 5 files changed, 379 insertions(+), 2 deletions(-) create mode 100644 fs/btrfs/read-repair.c create mode 100644 fs/btrfs/read-repair.h
fs/btrfs/Makefile +1 -1
@@ -31,7 +31,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \ backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \ uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \ block-rsv.o delalloc-space.o block-group.o discard.o reflink.o \ - subpage.o tree-mod-log.o + subpage.o tree-mod-log.o read-repair.o btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
fs/btrfs/extent_io.c +1 -1
@@ -2735,7 +2735,7 @@ static void end_page_read(struct page *page, bool uptodate, u64 start, u32 len)btrfs_subpage_end_reader (fs_info, page, start, len); } -static void end_sector_io (struct page *page,u64 offset,bool uptodate) +void end_sector_io (struct page *page,u64 offset,bool uptodate) {struct inode *inode = page->mapping->host;u32 sectorsize =btrfs_sb (inode->i_sb)->sectorsize;
fs/btrfs/extent_io.h +1 -0
@@ -250,6 +250,7 @@ struct bio *btrfs_bio_alloc(unsigned int nr_iovecs);struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio);struct bio *btrfs_bio_clone_partial (struct bio *orig,u64 offset,u64 size); +void end_sector_io (struct page *page,u64 offset,bool uptodate);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num);
fs/btrfs/read-repair.c new file +328 -0
328 lines not on this page · raw message
fs/btrfs/read-repair.h new file +48 -0
48 lines not on this page · raw message
Signature · 1 line
2.36.1
PATCH v2 5/7 btrfs: make buffered read path to use the new read repair infrastructure
In reply to Qu WenruoWe only need to prepare the logical bytenr and file offset of the corrupted sector, and pass it to btrfs_read_repair_add_sector().
And call btrfs_read_repair_finish() before we free the csum.
--- fs/btrfs/extent_io.c | 44 ++++++++++++++++++++------------------------ 1 file changed, 20 insertions(+), 24 deletions(-)
fs/btrfs/extent_io.c +20 -24
@@ -30,6 +30,7 @@#include "zoned.h" #include "block-group.h" #include "compression.h" +#include "read-repair.h" static struct kmem_cache *extent_state_cache;static struct kmem_cache *extent_buffer_cache;
@@ -2749,13 +2750,15 @@ void end_sector_io(struct page *page, u64 offset, bool uptodate) offset + sectorsize - 1, &cached); } -static void submit_data_read_repair (struct inode *inode,struct bio *failed_bio, +static void submit_data_read_repair (struct inode *inode, +struct btrfs_read_repair_ctrl *ctrl,struct bio *failed_bio,u32 bio_offset,const struct bio_vec *bvec,int failed_mirror,unsigned int error_bitmap) {const unsigned int pgoff = bvec->bv_offset;struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);struct page *page = bvec->bv_page; +struct btrfs_bio *failed_bbio =btrfs_bio (failed_bio);const u64 start =page_offset (bvec->bv_page) + bvec->bv_offset;const u64 end = start + bvec->bv_len -1 ;const u32 sectorsize = fs_info->sectorsize;
@@ -2780,7 +2783,9 @@ static void submit_data_read_repair(struct inode *inode, struct bio *failed_bio,for (i =0 ; i < nr_bits; i++) {const unsigned int offset = i * sectorsize;bool uptodate = false; -int ret; +u64 logical = (failed_bbio->iter.bi_sector <<SECTOR_SHIFT ) + + bio_offset + offset; +u8 *csum =NULL ;if (!(error_bitmap & (1U << i))) {/*
@@ -2788,28 +2793,17 @@ static void submit_data_read_repair(struct inode *inode, struct bio *failed_bio,* and unlock the range. */ uptodate = true; - goto next; - } - - ret =btrfs_repair_one_sector (inode, failed_bio, - bio_offset + offset, - page, pgoff + offset, start + offset, - failed_mirror, btrfs_submit_data_bio); -if (!ret) { -/* -* We have submitted the read repair, the page release -* will be handled by the endio function of the -* submitted repair bio. -* Thus we don't need to do any thing here. -*/ +end_sector_io (page, start + offset, uptodate);continue ; } -/* -* Continue on failed repair, otherwise the remaining sectors -* will not be properly unlocked. -*/ -next: -end_sector_io (page, start + offset, uptodate); +if (failed_bbio->csum) + csum =btrfs_csum_ptr (fs_info, failed_bbio->csum, + bio_offset + offset); + +/* The function will release the page if hit failure. */ +btrfs_read_repair_add_sector (inode, ctrl, page, pgoff + offset, + logical, start + offset, csum, + failed_mirror, false); } }
@@ -3017,6 +3011,7 @@ static void end_bio_extent_readpage(struct bio *bio)struct btrfs_bio *bbio =btrfs_bio (bio);struct extent_io_tree *tree, *failure_tree;struct processed_extent processed = {0 }; +struct btrfs_read_repair_ctrl ctrl = {0 };/* * The offset to the beginning of a bio, since one bio can never be * larger than UINT_MAX, u32 here is enough.
@@ -3126,8 +3121,8 @@ static void end_bio_extent_readpage(struct bio *bio)* submit_data_read_repair() will handle all the good * and bad sectors, we just continue to the next bvec. */ -submit_data_read_repair (inode, bio, bio_offset, bvec, - mirror, error_bitmap); +submit_data_read_repair (inode, &ctrl, bio, bio_offset, + bvec, mirror, error_bitmap); }else {/* Update page status and unlock */ end_page_read (page, uptodate, start, len);
@@ -3142,6 +3137,7 @@ static void end_bio_extent_readpage(struct bio *bio) }/* Release the last extent */ endio_readpage_release_extent (&processed,NULL ,0 ,0 , false); +btrfs_read_repair_finish (&ctrl);btrfs_bio_free_csum (bbio);bio_put (bio); }
Signature · 1 line
2.36.1
PATCH v2 6/7 btrfs: make direct io read path to use the new read repair infrastructure
In reply to Qu WenruoBy doing this, we can also remove the function submit_dio_repair_bio().
--- fs/btrfs/inode.c | 51 +++++++++++++++++------------------------------- 1 file changed, 18 insertions(+), 33 deletions(-)
fs/btrfs/inode.c +18 -33
@@ -55,6 +55,7 @@#include "zoned.h" #include "subpage.h" #include "inode-item.h" +#include "read-repair.h" struct btrfs_iget_args {u64 ino;
@@ -7863,58 +7864,42 @@ static void btrfs_dio_private_put(struct btrfs_dio_private *dip)bio_endio (&dip->bio); } -static void submit_dio_repair_bio (struct inode *inode,struct bio *bio, -int mirror_num, -enum btrfs_compression_type compress_type) -{ -struct btrfs_dio_private *dip = bio->bi_private; -struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); - -BUG_ON (bio_op (bio) ==REQ_OP_WRITE ); - -if (btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA )) -return ; - -refcount_inc (&dip->refs); -if (btrfs_map_bio (fs_info, bio, mirror_num)) -refcount_dec (&dip->refs); -} -static blk_status_t btrfs_check_read_dio_bio (struct btrfs_dio_private *dip,struct btrfs_bio *bbio,const bool uptodate) {struct inode *inode = dip->inode;struct btrfs_fs_info *fs_info =BTRFS_I (inode)->root->fs_info; -struct extent_io_tree *failure_tree = &BTRFS_I (inode)->io_failure_tree; -struct extent_io_tree *io_tree = &BTRFS_I (inode)->io_tree; +struct btrfs_read_repair_ctrl ctrl = {0 };const bool csum = !(BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM );blk_status_t err =BLK_STS_OK ;struct bvec_iter iter;struct bio_vec bv; +int ret;u32 offset;btrfs_bio_for_each_sector (fs_info, bv, bbio, iter, offset) {u64 start = bbio->file_offset + offset; -if (uptodate && - (!csum || !check_data_csum (inode, bbio, offset, bv.bv_page, - bv.bv_offset, start))) { -clean_io_failure (fs_info, failure_tree, io_tree, start, - bv.bv_page,btrfs_ino (BTRFS_I (inode)), - bv.bv_offset); - }else { -int ret; - - ret =btrfs_repair_one_sector (inode, &bbio->bio, offset, - bv.bv_page, bv.bv_offset, start, - bbio->mirror_num, - submit_dio_repair_bio); +if (!uptodate || + (csum &&check_data_csum (inode, bbio, offset, bv.bv_page, + bv.bv_offset, start))) { +u64 logical = (bbio->iter.bi_sector <<SECTOR_SHIFT ) + + offset; +u8 *csum =NULL ; + +if (bbio->csum) + csum =btrfs_csum_ptr (fs_info, bbio->csum, offset); + ret =btrfs_read_repair_add_sector (inode, &ctrl, + bv.bv_page, bv.bv_offset, logical, + start, csum, bbio->mirror_num, true);if (ret) err =errno_to_blk_status (ret); } } - + ret =btrfs_read_repair_finish (&ctrl); +if (ret) + err =errno_to_blk_status (ret);return err; }
Signature · 1 line
2.36.1
PATCH v2 7/7 btrfs: remove io_failure_record infrastructure completely
In reply to Qu WenruoSince our read repair are always handled by btrfs_read_repair_ctrl, which only has the lifespan inside endio function.
This means we no longer needs to record which range and its mirror number for failure.
Now if we failed to read some data page, we have already tried every mirrors we have, thus no need to record the failed range.
Thus this patch can remove the whole io_failure_record structure and its related functions.
--- fs/btrfs/btrfs_inode.h | 5 - fs/btrfs/extent-io-tree.h | 15 -- fs/btrfs/extent_io.c | 372 ----------------------------------- fs/btrfs/extent_io.h | 24 --- fs/btrfs/inode.c | 7 - include/trace/events/btrfs.h | 1 - 6 files changed, 424 deletions(-)
fs/btrfs/btrfs_inode.h +0 -5
@@ -91,11 +91,6 @@ struct btrfs_inode {/* the io_tree does range state (DIRTY, LOCKED etc) */ struct extent_io_tree io_tree; -/* special utility tree used to record which mirrors have already been -* tried when checksums fail for a given block -*/ -struct extent_io_tree io_failure_tree; -/* * Keep track of where the inode has extent items mapped in order to * make sure the i_size adjustments are accurate
fs/btrfs/extent-io-tree.h +0 -15
@@ -56,7 +56,6 @@ enum {IO_TREE_FS_EXCLUDED_EXTENTS ,IO_TREE_BTREE_INODE_IO ,IO_TREE_INODE_IO , -IO_TREE_INODE_IO_FAILURE ,IO_TREE_RELOC_BLOCKS ,IO_TREE_TRANS_DIRTY_PAGES , IO_TREE_ROOT_DIRTY_LOG_PAGES,
@@ -250,18 +249,4 @@ bool btrfs_find_delalloc_range(struct extent_io_tree *tree, u64 *start, u64 *end, u64 max_bytes, struct extent_state **cached_state); -/* This should be reworked in the future and put elsewhere. */ -struct io_failure_record *get_state_failrec (struct extent_io_tree *tree,u64 start); -int set_state_failrec (struct extent_io_tree *tree,u64 start, -struct io_failure_record *failrec); -void btrfs_free_io_failure_record (struct btrfs_inode *inode,u64 start, -u64 end); -int free_io_failure (struct extent_io_tree *failure_tree, -struct extent_io_tree *io_tree, -struct io_failure_record *rec); -int clean_io_failure (struct btrfs_fs_info *fs_info, -struct extent_io_tree *failure_tree, -struct extent_io_tree *io_tree,u64 start, -struct page *page,u64 ino,unsigned int pg_offset); -#endif /* BTRFS_EXTENT_IO_TREE_H */
fs/btrfs/extent_io.c +0 -372
408 lines not on this page · raw message
fs/btrfs/extent_io.h +0 -24
@@ -61,7 +61,6 @@ struct btrfs_root;struct btrfs_inode ;struct btrfs_io_bio ;struct btrfs_fs_info ; -struct io_failure_record ;struct extent_io_tree ; typedef void (submit_bio_hook_t)(struct inode *inode, struct bio *bio,
@@ -254,29 +253,6 @@ void end_sector_io(struct page *page, u64 offset, bool uptodate);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num); -/* -* When IO fails, either with EIO or csum verification fails, we -* try other mirrors that might have a good copy of the data. This -* io_failure_record is used to record state as we go through all the -* mirrors. If another mirror has good data, the sector is set up to date -* and things continue. If a good mirror can't be found, the original -* bio end_io callback is called to indicate things have failed. -*/ -struct io_failure_record { -struct page *page; -u64 start; -u64 len; -u64 logical; -enum btrfs_compression_type compress_type; -int this_mirror; -int failed_mirror; -}; - -int btrfs_repair_one_sector (struct inode *inode, -struct bio *failed_bio,u32 bio_offset, -struct page *page,unsigned int pgoff, -u64 start,int failed_mirror, -submit_bio_hook_t *submit_bio_hook);int btrfs_repair_io_failure (struct btrfs_fs_info *fs_info,u64 ino,u64 start,u64 length,u64 logical,struct page *page,unsigned int pg_offset,int mirror_num);
fs/btrfs/inode.c +0 -7
@@ -3143,8 +3143,6 @@ static int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent) ordered_extent->disk_num_bytes); } -btrfs_free_io_failure_record (inode, start, end); -if (test_bit (BTRFS_ORDERED_TRUNCATED , &ordered_extent->flags)) { truncated = true; logical_len = ordered_extent->truncated_len;
@@ -5355,8 +5353,6 @@ void btrfs_evict_inode(struct inode *inode)if (is_bad_inode (inode)) goto no_delete; -btrfs_free_io_failure_record (BTRFS_I (inode),0 , (u64)-1 ); -if (test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) goto no_delete;
@@ -8847,12 +8843,9 @@ struct inode *btrfs_alloc_inode(struct super_block *sb) inode = &ei->vfs_inode;extent_map_tree_init (&ei->extent_tree);extent_io_tree_init (fs_info, &ei->io_tree,IO_TREE_INODE_IO , inode); -extent_io_tree_init (fs_info, &ei->io_failure_tree, -IO_TREE_INODE_IO_FAILURE , inode);extent_io_tree_init (fs_info, &ei->file_extent_tree,IO_TREE_INODE_FILE_EXTENT , inode); ei->io_tree.track_uptodate = true; - ei->io_failure_tree.track_uptodate = true;atomic_set (&ei->sync_writers,0 );mutex_init (&ei->log_mutex);btrfs_ordered_inode_tree_init (&ei->ordered_tree);
include/trace/events/btrfs.h +0 -1
@@ -82,7 +82,6 @@ struct btrfs_space_info; EM( IO_TREE_FS_EXCLUDED_EXTENTS, "EXCLUDED_EXTENTS") \ EM( IO_TREE_BTREE_INODE_IO, "BTREE_INODE_IO") \ EM( IO_TREE_INODE_IO, "INODE_IO") \ - EM( IO_TREE_INODE_IO_FAILURE, "INODE_IO_FAILURE") \ EM( IO_TREE_RELOC_BLOCKS, "RELOC_BLOCKS") \ EM( IO_TREE_TRANS_DIRTY_PAGES, "TRANS_DIRTY_PAGES") \EM (IO_TREE_ROOT_DIRTY_LOG_PAGES ,"ROOT_DIRTY_LOG_PAGES" ) \
Signature · 1 line
2.36.1
Quoted 2 lines · Qu Wenruo
This is the bitmap version revivied, and based on Christoph's cleanup series.
So this is another patchset for raid repair, it may take me some time understand the different approaches and discussions.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
In reply to Qu WenruoQuoted 2 lines · Qu Wenruo
The new read repair infrastructure is consist of the following 3 parts:
[...]
Quoted 14 lines
fs/btrfs/read-repair.c new file +14
-0+static void io_add_or_submit (struct btrfs_read_repair_ctrl *ctrl,int mirror, +u64 logical,struct page *page,unsigned int pgoff, +int opf) +{ +struct btrfs_fs_info *fs_info =btrfs_sb (ctrl->inode->i_sb); +struct bio *io_bio = ctrl->io_bio; + +/* Uninitialized. */ +if (io_bio->bi_iter.bi_sector ==0 ) { +ASSERT (io_bio->bi_iter.bi_size ==0 ); + io_bio->bi_iter.bi_sector = logical >>SECTOR_SHIFT ; + io_bio->bi_opf = opf; +bio_add_page (io_bio, page, fs_info->sectorsize, pgoff); +return ;
fs/btrfs/read-repair.c line 165
Quoted 10 lines
fs/btrfs/read-repair.c new file +10
-0+ } + +/* Continuous, add the page */ +if ((io_bio->bi_iter.bi_sector <<SECTOR_SHIFT ) + + io_bio->bi_iter.bi_size == logical) { +bio_add_page (io_bio, page, fs_info->sectorsize, pgoff); +return ; + } + +/* Not continuous, submit first. */
Hi Christoph, I'm pretty sure the non-continuous bio problem is here for all of our attempts to rework read-repair.
I'm wondering if there is some "dummy" page provided from block layer that we can utilize?
E.g. We have the following checker pattern:
mirror 1 |X|X|X|X| mirror 2 |X| |X| | mirror 3 | |X| |X|
After reading all the 4 sectors from mirror 2, we know the 2nd and 4th are good and should not need to be-read.
Then reading mirror 3 needs us to submit two bios.
But if we have some "dummy" pages, and added into the bio for sector 2 and 4, we only need one bio submission.
Is there such convenient page for us to utilize? Or we have to assign it globally?
Thanks,
Qu
Quoted 217 lines
fs/btrfs/read-repair.c new file +163
-0+io_bio_submit (ctrl, mirror, opf); + io_bio = ctrl->io_bio; + io_bio->bi_iter.bi_sector = logical >>SECTOR_SHIFT ; +bio_add_page (io_bio, page, fs_info->sectorsize, pgoff); +} + +static void writeback_good_mirror (struct btrfs_read_repair_ctrl *ctrl, +int mirror,u64 logical, +struct page *page,unsigned int pgoff) +{ +struct btrfs_fs_info *fs_info =btrfs_sb (ctrl->inode->i_sb); +struct bio *io_bio = ctrl->io_bio; + + +if (btrfs_repair_one_zone (fs_info, ctrl->logical)) +return ; + +/* +* For RAID56, we can not just write the bad data back, as +* any write will trigger RMW and read back the corrrupted +* on-disk stripe, causing further damage. +* So here we do special repair for raid56. +* +* And unfortunately, this repair is very low level and not +* compatible with the rest of the mirror based repair. +* So it's still done in synchronous mode using +* btrfs_repair_io_failure(). +*/ +if (ctrl->is_raid56) { +const u64 file_offset = logical - ctrl->logical + + ctrl->file_offset; +btrfs_repair_io_failure (fs_info, +btrfs_ino (BTRFS_I (ctrl->inode)), file_offset, + fs_info->sectorsize, logical, page, pgoff, + mirror); +return ; + } + +ASSERT (io_bio); +io_add_or_submit (ctrl, mirror, logical, page, pgoff,REQ_OP_WRITE ); +} + +static void repair_from_mirror (struct btrfs_read_repair_ctrl *ctrl,int mirror) +{ +struct btrfs_fs_info *fs_info =btrfs_sb (ctrl->inode->i_sb); +struct bvec_iter iter; +struct bio_vec bv; +unsigned long old_bitmap = ctrl->bad_bitmap; +const int prev_mirror =get_prev_mirror (mirror, ctrl->num_copies); +int nr_sector; +u32 offset; +int ret; + +/* +* Reset the io_bio logial bytenr so later io_add_or_submit() can do +* correct check on the logical bytenr. +*/ + ctrl->io_bio->bi_iter.bi_sector =0 ; + +/* Add all bad sectors into io_bio. */ +bio_for_each_sector (fs_info, bv, ctrl->bad_sectors, iter, offset) { +u64 logical = ctrl->logical + offset; + + nr_sector = offset >> fs_info->sectorsize_bits; + +/* Good sectors, no need to handle. */ +if (!test_bit (nr_sector, &ctrl->bad_bitmap)) +continue ; + +io_add_or_submit (ctrl, mirror, logical, bv.bv_page, + bv.bv_offset,REQ_OP_READ |REQ_SYNC ); + } +io_bio_submit (ctrl, mirror,REQ_OP_READ |REQ_SYNC ); + +/* Check the newly read data. */ +bio_for_each_sector (fs_info, bv, ctrl->bad_sectors, iter, offset) { +u8 *csum_expected; +u8 csum[BTRFS_CSUM_SIZE ]; + + nr_sector = offset >> fs_info->sectorsize_bits; + +/* Originally good sector or read failed, skip. */ +if (!test_bit (nr_sector, &old_bitmap) || +test_bit (nr_sector, &ctrl->bad_bitmap)) +continue ; + +/* No data csum, only need to repair. */ +if (!ctrl->csum) + goto repair; + +/* +* The remaining case is successful read with csum, need +* recheck the csum. +*/ + csum_expected =btrfs_csum_ptr (fs_info, ctrl->csum, offset); + ret =btrfs_check_sector_csum (fs_info, bv.bv_page, + bv.bv_offset, csum, csum_expected); +if (ret) { +set_bit (nr_sector, &ctrl->bad_bitmap); +continue ; + } +repair: +/* +* This sector is properly fixed, write it back to previous +* bad mirror. +*/ +writeback_good_mirror (ctrl, prev_mirror, ctrl->logical + offset, + bv.bv_page, bv.bv_offset); + } +/* Submit the last write bio. */ +io_bio_submit (ctrl, mirror,REQ_OP_WRITE ); +} + +int btrfs_read_repair_finish (struct btrfs_read_repair_ctrl *ctrl) +{ +struct btrfs_fs_info *fs_info; +struct bvec_iter iter; +struct bio_vec bv; +u32 offset; +int nr_sectors; +int mirror; +int ret = -EIO ; + +if (!ctrl->inode) +return 0 ; + + fs_info =btrfs_sb (ctrl->inode->i_sb); + nr_sectors = ctrl->len >> fs_info->sectorsize_bits; +ASSERT (ctrl->len); +/* All sectors should be bad initially. */ +ASSERT (find_first_zero_bit (&ctrl->bad_bitmap, nr_sectors) == nr_sectors); + +for (mirror =get_next_mirror (ctrl->failed_mirror, ctrl->num_copies); + mirror != ctrl->failed_mirror; + mirror =get_next_mirror (mirror, ctrl->num_copies)) { +repair_from_mirror (ctrl, mirror); + +/* All repaired*/ +if (find_first_bit (&ctrl->bad_bitmap, nr_sectors) == nr_sectors) { + ret =0 ; +break ; + } + } + +/* DIO doesn't need any page status/extent update.*/ +if (!ctrl->is_dio) { +/* Unlock all the pages and unlock the extent range. */ +bio_for_each_sector (fs_info, bv, ctrl->bad_sectors, iter, + offset) { +bool uptodate = !test_bit (offset >> + fs_info->sectorsize_bits, + &ctrl->bad_bitmap); + +end_sector_io (bv.bv_page, ctrl->file_offset + offset, + uptodate); + } + } +bio_put (ctrl->bad_sectors); +if (ctrl->io_bio) +bio_put (ctrl->io_bio); +memset (ctrl,0 ,sizeof (*ctrl)); +return ret; +}fs/btrfs/read-repair.h new file +48
-0@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef BTRFS_READ_REPAIR_H +#define BTRFS_READ_REPAIR_H + +#include <linux/blk_types.h> +#include <linux/fs.h> + +struct btrfs_read_repair_ctrl { +struct inode *inode; + +/* The logical bytenr of the firts corrupted sector. */ +u64 logical; + +/* The file offset of the first corrupted sector. */ +u64 file_offset; + +/* The checksum for the corrupted sectors. */ +u8 *csum; + +/* Current length of the corrupted range. */ +u32 len; + +int failed_mirror; +int num_copies; +unsigned long bad_bitmap; +bool is_raid56; +bool is_dio; + +/* This is only to hold all the initial bad continuous sectors. */ +struct bio *bad_sectors; + +/* +* The bio we use to do the real IO. +* This bio has to be btrfs_bio, as btrfs_map_bio() will utilize +* btrfs_bio()->device. +*/ +struct bio *io_bio; +}; + +int btrfs_read_repair_add_sector (struct inode *inode, +struct btrfs_read_repair_ctrl *ctrl, +struct page *page,unsigned int pgoff, +u64 logical,u64 file_offset,u8 *csum, +int failed_mirror,bool is_dio); +int btrfs_read_repair_finish (struct btrfs_read_repair_ctrl *ctrl); + +#endif
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 2 lines · Qu Wenruo
Hi Christoph, I'm pretty sure the non-continuous bio problem is here for all of our attempts to rework read-repair.
Why is it a problem? Multiple discontiguous errors in the same bio are a very unusual error pattern. We need to handle it obviously, but it doesn't need to be optimized as it is so rare. The most common error pattern is that the entire read will return an error, followed by a single corrupted sector.
Quoted 2 lines
I'm wondering if there is some "dummy" page provided from block layer that we can utilize?
For reads nvme (and a few SCSI HBAs) support a bit bucket SGL for reads that discard parts of the data. Right now upstream none of this is supported, altough Keith has been looking into it (for a rather different use case) in nvme. This does not help with writes, never mind the fact that I would not want to use exotic and barely tested code and hardware features for a non time critical and rarely used error handling path..
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 9 lines · Christoph Hellwig
Quoted 2 lines · Qu Wenruo
Hi Christoph, I'm pretty sure the non-continuous bio problem is here for all of our attempts to rework read-repair.
Why is it a problem? Multiple discontiguous errors in the same bio are a very unusual error pattern. We need to handle it obviously, but it doesn't need to be optimized as it is so rare. The most common error pattern is that the entire read will return an error, followed by a single corrupted sector.
Rare case doesn't mean it won't happen.
We still need to address it anyway.
Furthermore, if we can submit one bio to read the whole mirror range, without putting the corrupted data into our repaired data, it also means we will have read at most (num_copies - 1) times, without resetting the initial mirror.
Quoted 10 lines
Quoted 3 lines
I'm wondering if there is some "dummy" page provided from block layer that we can utilize?
For reads nvme (and a few SCSI HBAs) support a bit bucket SGL for reads that discard parts of the data. Right now upstream none of this is supported, altough Keith has been looking into it (for a rather different use case) in nvme. This does not help with writes, never mind the fact that I would not want to use exotic and barely tested code and hardware features for a non time critical and rarely used error handling path..
I'm not purposing the SGL method, but still do a full range read, the only difference is, the page range we don't care will be written to some dust bin page, and only the range we care will be put into the real pages.
E.g. we allocate a dedicated page per-fs (or even for the whole btrfs module) as a dustbin page.
When we don't want to read some range, we just add that page into the bio (this means we may put the same page into the bio several times, and the page may be utilized by several different bios at the same time). And submit the bio.
I'm not sure the current code base can handle the case though.
For write, it's pretty simple, we only writeback the whole correct range. If we didn't recover the full corrupted range, we just don't do writeback.
Thanks,
Qu
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 3 lines · Qu Wenruo
Rare case doesn't mean it won't happen.
We still need to address it anyway.
address != build overly complicated code to optimize for it
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Qu Wenruo
Rare case doesn't mean it won't happen.
We still need to address it anyway.
address != build overly complicated code to optimize for it
Well, using seemly simple code, but can lead to read way more loops and way more data to read, is neither a good way.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 4 lines · Qu Wenruo
Quoted 1 line · Christoph Hellwig
On Thu, May 26, 2022 at 03:37:47PM +0800, Qu Wenruo wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Rare case doesn't mean it won't happen.
We still need to address it anyway.
address != build overly complicated code to optimize for it
Well, using seemly simple code, but can lead to read way more loops and way more data to read, is neither a good way.
Again, having checkered corruption is an extremely unlikely event. I'd rather deal with it by doing more reads than code complexity.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 9 lines · Christoph Hellwig
Quoted 8 lines · Qu Wenruo
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Qu Wenruo
Rare case doesn't mean it won't happen.
We still need to address it anyway.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
address != build overly complicated code to optimize for it
Well, using seemly simple code, but can lead to read way more loops and way more data to read, is neither a good way.
Again, having checkered corruption is an extremely unlikely event. I'd rather deal with it by doing more reads than code complexity.
Then it can be said to almost all ENOSPC error handling code. It's less than 1% chance, but we spend over 10% code for it.
And if you really want to go that path, I see no reason why we didn't go sector-by-sector repair.
Furthermore if "more reads" means over 10 times the amount we need, I strongly doubt if it's sane.
Just the same RAID1C3, mirror 1 all corrupted, mirror 2 and 3 checker pattern, fill it into a 4MiB range, and try run your version of code starting with mirror 1, and see how many loops we need to go, especially how many times we need to read mirror 1 unnecessarily.
Thanks,
Qu
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 1 line · Qu Wenruo
Then it can be said to almost all ENOSPC error handling code.
ENOSPC is a lot more common.
Quoted 4 lines
It's less than 1% chance, but we spend over 10% code for it.
And if you really want to go that path, I see no reason why we didn't go sector-by-sector repair.
Because that really sucks for the case where the whole I/O fails. Which is the common failure scenario.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 4 lines · Christoph Hellwig
Quoted 1 line · Qu Wenruo
Then it can be said to almost all ENOSPC error handling code.
ENOSPC is a lot more common.
Sorry, I mean ENOMEM.
Quoted 8 lines
Quoted 5 lines
It's less than 1% chance, but we spend over 10% code for it.
And if you really want to go that path, I see no reason why we didn't go sector-by-sector repair.
Because that really sucks for the case where the whole I/O fails. Which is the common failure scenario.
But it's just a performance problem, which is not that critical.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 13 lines · Qu Wenruo
Quoted 4 lines · Christoph Hellwig
Quoted 1 line · Qu Wenruo
Then it can be said to almost all ENOSPC error handling code.
ENOSPC is a lot more common.
Sorry, I mean ENOMEM.
Quoted 3 lines
Quoted 3 lines
It's less than 1% chance, but we spend over 10% code for it.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
And if you really want to go that path, I see no reason why we didn't go sector-by-sector repair.
Because that really sucks for the case where the whole I/O fails. Which is the common failure scenario.
But it's just a performance problem, which is not that critical.
I'm officially lost now.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 17 lines · Christoph Hellwig
Quoted 16 lines · Qu Wenruo
Quoted 4 lines · Christoph Hellwig
Quoted 1 line · Qu Wenruo
Then it can be said to almost all ENOSPC error handling code.
ENOSPC is a lot more common.
Sorry, I mean ENOMEM.
Quoted 6 lines
Quoted 5 lines
It's less than 1% chance, but we spend over 10% code for it.
And if you really want to go that path, I see no reason why we didn't go sector-by-sector repair.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Because that really sucks for the case where the whole I/O fails. Which is the common failure scenario.
But it's just a performance problem, which is not that critical.
I'm officially lost now.
Why? If you care so much about the code simplicity, sector-by-sector is the best.
If you care so much about the performance, the latest bitmap is the best, no matter if it's the worst checker patter or not.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 11 lines · Qu Wenruo
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Because that really sucks for the case where the whole I/O fails. Which is the common failure scenario.
But it's just a performance problem, which is not that critical.
I'm officially lost now.
Why? If you care so much about the code simplicity, sector-by-sector is the best.
If you care so much about the performance, the latest bitmap is the best, no matter if it's the worst checker patter or not.
Because you tell me that handling the most common and important case in read repair is just a performance issue, which you keep arguing for micro-optimizing a corner case. And not, for the case of failing a large bio (which arguably can only happen for buffered I/O at the moment, but that is another thing to look into) the bitmaps will only help you for up to 64 sectors. Way better than just doing a single sector synchronous I/O but not exactly nice while still being a fair amount of code compared to just doing variable sized synchronous I/O.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Qu Wenruo
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Because that really sucks for the case where the whole I/O fails. Which is the common failure scenario.
But it's just a performance problem, which is not that critical.
I'm officially lost now.
Quoted 8 lines
Quoted 4 lines
Why? If you care so much about the code simplicity, sector-by-sector is the best.
If you care so much about the performance, the latest bitmap is the best, no matter if it's the worst checker patter or not.Because you tell me that handling the most common and important case in read repair is just a performance issue, which you keep arguing for micro-optimizing a corner case.
Because I'm fine either way, but not fine with the middle ground.
I purposed both versions, to fulfill the different requirements. The tradeoff is not avoidable, we have to choose the poison.
If we want to go code simplicity, then my argument to support sector-by-sector is, the repair is already a cold path (the same argument you go on the checker pattern), thus overall the performance drop is not that critical.
If we want to go the best perf (even corruption is already a corner case), then there is the bitmap version, handling all cases, at the cost of complex code.
If you can find a simpler version, and still handle the checker pattern sanely (aka, without reading the same bad mirror again and again), sure, I'm always fine to go that version.
Otherwise sector-by-sector or bitmap seems a more sane choice to do between.
Thanks,
Qu
Quoted 7 lines
And not, for the case of failing a
large bio (which arguably can only happen for buffered I/O at the moment, but that is another thing to look into) the bitmaps will only help you for up to 64 sectors. Way better than just doing a single sector synchronous I/O but not exactly nice while still being a fair amount of code compared to just doing variable sized synchronous I/O.
Re: [PATCH v2 4/7] btrfs: introduce new read-repair infrastructure
Quoted 5 lines · Qu Wenruo
Quoted 3 lines
Because you tell me that handling the most common and important case in read repair is just a performance issue, which you keep arguing for micro-optimizing a corner case.
Because I'm fine either way, but not fine with the middle ground.
Where the middle ground is optimizig for the common case while handling the non-common cases in a non-optimized way?