RFC: cleanup btrfs bio handling
Subsystems (3)
Hi all,
this series started out as an attempt to move btrfs to use the new as of 5.18 bio interface, which then turned into cleaning up a lot of the surrounding areas. It can be roughly divided into 4 sub-series:
- patches 1 to 4 are bug fixes for bugs found during code inspection. It might be a good idea if experienced btrfs developers could help review them or correct me if I misunderstood something
- patches 5 to 22 are general cleanups on how bios are used and surrounding code
- patches 23 to 29 clean up various extra memory allocations in the bio I/O path. With I/Os that go to a single device (like all reads) only need the btrfs_bio memory allocation and not additional object.
- patches 30 to 40 integrate the btrfs dio code more tightly with iomap and avoid the extra dio_private allocation and bio clone
All this is pretty rough. It survices a xfstests auto group run on a default file system config, though.
The tree is based on Jens' for-next tree as it started with the bio cleanups, and will need a rebase once 5.18-rc1 is out.
A git tree is available here:
git://git.infradead.org/users/hch/misc.git btrfs-bio-cleanup
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/btrfs-bio-cleanup
Diffstat:
fs/btrfs/btrfs_inode.h | 25 - fs/btrfs/check-integrity.c | 165 +++++------ fs/btrfs/check-integrity.h | 8 fs/btrfs/compression.c | 54 +-- fs/btrfs/ctree.h | 6 fs/btrfs/disk-io.c | 272 +++--------------- fs/btrfs/disk-io.h | 21 - fs/btrfs/extent_io.c | 210 ++++++-------- fs/btrfs/extent_io.h | 17 - fs/btrfs/file.c | 6 fs/btrfs/inode.c | 661 ++++++++++++++++++--------------------------- fs/btrfs/raid56.c | 156 ++++------ fs/btrfs/scrub.c | 92 ++---- fs/btrfs/super.c | 11 fs/btrfs/volumes.c | 392 +++++++++++++++----------- fs/btrfs/volumes.h | 60 ++-- fs/iomap/direct-io.c | 29 + fs/iomap/iter.c | 13 include/linux/iomap.h | 23 + 19 files changed, 963 insertions(+), 1258 deletions(-)
PATCH 1/40 btrfs: fix submission hook error handling in btrfs_repair_one_sector
btrfs_repair_one_sector just wants to free the bio if submit_bio_hook fails, but btrfs_submit_data_bio will call bio_endio which will call into the submitter of the original bio and free the bio there as well.
Move the bio_endio calls from btrfs_submit_data_bio and btrfs_submit_metadata_bio into submit_one_bio to fix this double free.
--- fs/btrfs/disk-io.c | 2 -- fs/btrfs/extent_io.c | 4 ++++ fs/btrfs/inode.c | 4 ---- 3 files changed, 4 insertions(+), 6 deletions(-)
fs/btrfs/disk-io.c +0 -2
@@ -941,8 +941,6 @@ blk_status_t btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio,return 0 ; out_w_error: - bio->bi_status = ret; -bio_endio (bio);return ret; }
fs/btrfs/extent_io.c +4 -0
@@ -181,6 +181,10 @@ int __must_check submit_one_bio(struct bio *bio, int mirror_num, ret =btrfs_submit_metadata_bio (tree->private_data, bio, mirror_num, bio_flags); +if (ret) { + bio->bi_status = ret; +bio_endio (bio); + }return blk_status_to_errno (ret); }
fs/btrfs/inode.c +0 -4
@@ -2571,10 +2571,6 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, ret =btrfs_map_bio (fs_info, bio, mirror_num); out: -if (ret) { - bio->bi_status = ret; -bio_endio (bio); - }return ret; }
Signature · 1 line
2.30.2
PATCH 2/40 btrfs: fix direct I/O read repair for split bios
In reply to Christoph HellwigWhen a bio is split in btrfs_submit_direct, dip->file_offset contains the file offset for the first bio. But this means the start value used in btrfs_check_read_dio_bio is incorrect for subsequent bios. Add a file_offset field to struct btrfs_bio to pass along the correct offset.
Given that check_data_csum only uses start of an error message this means problems with this miscalculation will only show up when I/O fails or checksums mismatch.
--- fs/btrfs/extent_io.c | 1 + fs/btrfs/inode.c | 13 +++++-------- fs/btrfs/volumes.h | 3 +++ 3 files changed, 9 insertions(+), 8 deletions(-)
fs/btrfs/extent_io.c +1 -0
@@ -2662,6 +2662,7 @@ int btrfs_repair_one_sector(struct inode *inode, repair_bio =btrfs_bio_alloc (1 ); repair_bbio =btrfs_bio (repair_bio); + repair_bbio->file_offset = start; repair_bio->bi_opf =REQ_OP_READ ; repair_bio->bi_end_io = failed_bio->bi_end_io; repair_bio->bi_iter.bi_sector = failrec->logical >>9 ;
fs/btrfs/inode.c +5 -8
@@ -7773,8 +7773,6 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip,const bool csum = !(BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM );struct bio_vec bvec;struct bvec_iter iter; -const u64 orig_file_offset = dip->file_offset; -u64 start = orig_file_offset;u32 bio_offset =0 ;blk_status_t err =BLK_STS_OK ;
@@ -7784,6 +7782,8 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, nr_sectors = BTRFS_BYTES_TO_BLKS(fs_info, bvec.bv_len); pgoff = bvec.bv_offset; for (i = 0; i < nr_sectors; i++) { + u64 start = bbio->file_offset + bio_offset; + ASSERT(pgoff < PAGE_SIZE); if (uptodate && (!csum || !check_data_csum(inode, bbio,
@@ -7796,17 +7796,13 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, }else {int ret; -ASSERT ((start - orig_file_offset) <UINT_MAX ); - ret =btrfs_repair_one_sector (inode, - &bbio->bio, - start - orig_file_offset, - bvec.bv_page, pgoff, + ret =btrfs_repair_one_sector (inode, &bbio->bio, + bio_offset, bvec.bv_page, pgoff, start, bbio->mirror_num, submit_dio_repair_bio);if (ret) err =errno_to_blk_status (ret); } - start += sectorsize;ASSERT (bio_offset + sectorsize > bio_offset); bio_offset += sectorsize; pgoff += sectorsize;
@@ -8009,6 +8005,7 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, bio =btrfs_bio_clone_partial (dio_bio, clone_offset, clone_len); bio->bi_private = dip; bio->bi_end_io = btrfs_end_dio_bio; +btrfs_bio (bio)->file_offset = file_offset;if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { status = extract_ordered_extent(BTRFS_I(inode), bio,
fs/btrfs/volumes.h +3 -0
@@ -323,6 +323,9 @@ struct btrfs_fs_devices {struct btrfs_bio {unsigned int mirror_num; +/* for direct I/O */ +u64 file_offset; +/* @device is for stripe IO submission. */ struct btrfs_device *device;u8 *csum;
Signature · 1 line
2.30.2
Re: [PATCH 02/40] btrfs: fix direct I/O read repair for split bios
Quoted 9 lines · Christoph Hellwig
When a bio is split in
btrfs_submit_direct,dip->file_offsetcontains the file offset for the first bio. But this means the start value used inbtrfs_check_read_dio_biois incorrect for subsequent bios. Add afile_offsetfield to structbtrfs_bioto pass along the correct offset.Given that
check_data_csumonly uses start of an error message this means problems with this miscalculation will only show up when I/O fails or checksums mismatch. >Signed-off-by Christoph Hellwig <[email protected]>
Personally speaking, I really hate to add DIO specific value into btrfs_bio.
Hopes we can later turn that btrfs_bio::file_offset into some union for other usages.
Despite the extra memory usage, it looks good.
Thanks,
Qu
Quoted 82 lines
--- fs/btrfs/extent_io.c | 1 + fs/btrfs/inode.c | 13 +++++-------- fs/btrfs/volumes.h | 3 +++ 3 files changed, 9 insertions(+), 8 deletions(-)fs/btrfs/extent_io.c +1
-0@@ -2662,6 +2662,7 @@ int btrfs_repair_one_sector(struct inode *inode, repair_bio =btrfs_bio_alloc (1 ); repair_bbio =btrfs_bio (repair_bio); + repair_bbio->file_offset = start; repair_bio->bi_opf =REQ_OP_READ ; repair_bio->bi_end_io = failed_bio->bi_end_io; repair_bio->bi_iter.bi_sector = failrec->logical >>9 ;fs/btrfs/inode.c +5
-8@@ -7773,8 +7773,6 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip,const bool csum = !(BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM );struct bio_vec bvec;struct bvec_iter iter; -const u64 orig_file_offset = dip->file_offset; -u64 start = orig_file_offset;u32 bio_offset =0 ;blk_status_t err =BLK_STS_OK ;@@ -7784,6 +7782,8 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, nr_sectors = BTRFS_BYTES_TO_BLKS(fs_info, bvec.bv_len); pgoff = bvec.bv_offset; for (i = 0; i < nr_sectors; i++) { + u64 start = bbio->file_offset + bio_offset; + ASSERT(pgoff < PAGE_SIZE); if (uptodate && (!csum || !check_data_csum(inode, bbio,@@ -7796,17 +7796,13 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, }else {int ret; -ASSERT ((start - orig_file_offset) <UINT_MAX ); - ret =btrfs_repair_one_sector (inode, - &bbio->bio, - start - orig_file_offset, - bvec.bv_page, pgoff, + ret =btrfs_repair_one_sector (inode, &bbio->bio, + bio_offset, bvec.bv_page, pgoff, start, bbio->mirror_num, submit_dio_repair_bio);if (ret) err =errno_to_blk_status (ret); } - start += sectorsize;ASSERT (bio_offset + sectorsize > bio_offset); bio_offset += sectorsize; pgoff += sectorsize;@@ -8009,6 +8005,7 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, bio =btrfs_bio_clone_partial (dio_bio, clone_offset, clone_len); bio->bi_private = dip; bio->bi_end_io = btrfs_end_dio_bio; +btrfs_bio (bio)->file_offset = file_offset;if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { status = extract_ordered_extent(BTRFS_I(inode), bio,fs/btrfs/volumes.h +3
-0@@ -323,6 +323,9 @@ struct btrfs_fs_devices {struct btrfs_bio {unsigned int mirror_num; +/* for direct I/O */ +u64 file_offset; +/* @device is for stripe IO submission. */ struct btrfs_device *device;u8 *csum;
Re: [PATCH 02/40] btrfs: fix direct I/O read repair for split bios
Quoted 4 lines · Qu Wenruo
Personally speaking, I really hate to add DIO specific value into
btrfs_bio.Hopes we can later turn that
btrfs_bio::file_offsetinto some union for other usages.
Agreed. There is two venues we could explore:
- ceative use of unions. Especially with some of my later patches adding more fields to struct btrfs_bio this could become possible.
- adding a btrfs_dio_bio that contains a btrfs_bio. By the end of this series another field (repair_refs) is added, and iter is only used for direct I/O, so this might be worthwhile. But then again I think we could eventually kill off iter as well.
So we should eventually do something, but for a non-invasive fix like this just adding the field for now seems like the most safe approach.
PATCH 3/40 btrfs: fix direct I/O writes for split bios on zoned devices
In reply to Christoph HellwigWhen a bio is split in btrfs_submit_direct, dip->file_offset contains the file offset for the first bio. But this means the start value used in btrfs_end_dio_bio to record the write location for zone devices is icorrect for subsequent bios.
--- fs/btrfs/inode.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +3 -2
@@ -7829,6 +7829,7 @@ static blk_status_t btrfs_submit_bio_start_direct_io(struct inode *inode,static void btrfs_end_dio_bio (struct bio *bio) {struct btrfs_dio_private *dip = bio->bi_private; +struct btrfs_bio *bbio =btrfs_bio (bio);blk_status_t err = bio->bi_status;if (err)
@@ -7839,12 +7840,12 @@ static void btrfs_end_dio_bio(struct bio *bio) bio->bi_iter.bi_size, err);if (bio_op (bio) ==REQ_OP_READ ) - err =btrfs_check_read_dio_bio (dip,btrfs_bio (bio), !err); + err =btrfs_check_read_dio_bio (dip, bbio, !err);if (err) dip->dio_bio->bi_status = err; -btrfs_record_physical_zoned (dip->inode, dip->file_offset, bio); +btrfs_record_physical_zoned (dip->inode, bbio->file_offset, bio);bio_put (bio);btrfs_dio_private_put (dip);
Signature · 1 line
2.30.2
Re: [PATCH 03/40] btrfs: fix direct I/O writes for split bios on zoned devices
Quoted 6 lines · Christoph Hellwig
When a bio is split in
btrfs_submit_direct,dip->file_offsetcontains the file offset for the first bio. But this means the start value used inbtrfs_end_dio_bioto record the write location for zone devices is icorrect for subsequent bios.Signed-off-by Christoph Hellwig <[email protected]>
Maybe better to be folded with previous patch?
It looks good to me though.
Thanks,
Qu
Quoted 31 lines
--- fs/btrfs/inode.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)fs/btrfs/inode.c +3
-2@@ -7829,6 +7829,7 @@ static blk_status_t btrfs_submit_bio_start_direct_io(struct inode *inode,static void btrfs_end_dio_bio (struct bio *bio) {struct btrfs_dio_private *dip = bio->bi_private; +struct btrfs_bio *bbio =btrfs_bio (bio);blk_status_t err = bio->bi_status;if (err)@@ -7839,12 +7840,12 @@ static void btrfs_end_dio_bio(struct bio *bio) bio->bi_iter.bi_size, err);if (bio_op (bio) ==REQ_OP_READ ) - err =btrfs_check_read_dio_bio (dip,btrfs_bio (bio), !err); + err =btrfs_check_read_dio_bio (dip, bbio, !err);if (err) dip->dio_bio->bi_status = err; -btrfs_record_physical_zoned (dip->inode, dip->file_offset, bio); +btrfs_record_physical_zoned (dip->inode, bbio->file_offset, bio);bio_put (bio);btrfs_dio_private_put (dip);
Re: [PATCH 03/40] btrfs: fix direct I/O writes for split bios on zoned devices
Quoted 8 lines · Qu Wenruo
Quoted 6 lines
When a bio is split in
btrfs_submit_direct,dip->file_offsetcontains the file offset for the first bio. But this means the start value used inbtrfs_end_dio_bioto record the write location for zone devices is icorrect for subsequent bios.Signed-off-by Christoph Hellwig <[email protected]>Maybe better to be folded with previous patch?
Well, it fixes a separate issue.
PATCH 4/40 btrfs: fix and document the zoned device choice in alloc_new_bio
In reply to Christoph HellwigZone Append bios only need a valid block device in struct bio, but not the device in the btrfs_bio. Use the information from btrfs_zoned_get_device to set up bi_bdev and fix zoned writes on multi-device file system with non-homogeneous capabilities and remove the pointless btrfs_bio.device assignment.
Add big fat comments explaining what is going on here.
--- fs/btrfs/extent_io.c | 43 ++++++++++++++++++++++++++++--------------- 1 file changed, 28 insertions(+), 15 deletions(-)
fs/btrfs/extent_io.c +28 -15
@@ -3330,24 +3330,37 @@ static int alloc_new_bio(struct btrfs_inode *inode, ret =calc_bio_boundaries (bio_ctrl, inode, file_offset);if (ret <0 ) goto error; -if (wbc) { -struct block_device *bdev; - bdev = fs_info->fs_devices->latest_dev->bdev; -bio_set_dev (bio, bdev); -wbc_init_bio (wbc, bio); - } -if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { -struct btrfs_device *device; +if (wbc) { +/* +* For Zone append we need the correct block_device that we are +* going to write to set in the bio to be able to respect the +* hardware limitation. Look it up here: +*/ +if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { +struct btrfs_device *dev; + + dev =btrfs_zoned_get_device (fs_info, disk_bytenr, + fs_info->sectorsize); +if (IS_ERR (dev)) { + ret =PTR_ERR (dev); + goto error; + } - device =btrfs_zoned_get_device (fs_info, disk_bytenr, - fs_info->sectorsize); -if (IS_ERR (device)) { - ret =PTR_ERR (device); - goto error; +bio_set_dev (bio, dev->bdev); + }else { +/* +* Otherwise pick the last added device to support +* cgroup writeback. For multi-device file systems this +* means blk-cgroup policies have to always be set on the +* last added/replaced device. This is a bit odd but has +* been like that for a long time. +*/ +bio_set_dev (bio, fs_info->fs_devices->latest_dev->bdev); } - -btrfs_bio (bio)->device = device; +wbc_init_bio (wbc, bio); + }else { +ASSERT (bio_op (bio) !=REQ_OP_ZONE_APPEND ); }return 0 ; error:
Signature · 1 line
2.30.2
PATCH 5/40 btrfs: refactor __btrfsic_submit_bio
In reply to Christoph HellwigSplit out two helpers to mak __btrfsic_submit_bio more readable.
--- fs/btrfs/check-integrity.c | 150 +++++++++++++++++++------------------ 1 file changed, 78 insertions(+), 72 deletions(-)
fs/btrfs/check-integrity.c +78 -72
@@ -2635,6 +2635,74 @@ static struct btrfsic_dev_state *btrfsic_dev_state_lookup(dev_t dev) &btrfsic_dev_state_hashtable); } +static void btrfsic_check_write_bio (struct bio *bio, +struct btrfsic_dev_state *dev_state) +{ +unsigned int segs =bio_segments (bio); +u64 dev_bytenr =512 * bio->bi_iter.bi_sector; +u64 cur_bytenr = dev_bytenr; +struct bvec_iter iter; +struct bio_vec bvec; +char **mapped_datav; +int bio_is_patched =0 ; +int i =0 ; + +if (dev_state->state->print_mask &BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH ) +pr_info ("submit_bio(rw=%d,0x%x, bi_vcnt=%u, bi_sector=%llu (bytenr %llu), bi_bdev=%p)\n" , +bio_op (bio), bio->bi_opf, segs, + bio->bi_iter.bi_sector, dev_bytenr, bio->bi_bdev); + + mapped_datav =kmalloc_array (segs,sizeof (*mapped_datav),GFP_NOFS ); +if (!mapped_datav) +return ; + +bio_for_each_segment (bvec, bio, iter) { +BUG_ON (bvec.bv_len !=PAGE_SIZE ); + mapped_datav[i] =page_address (bvec.bv_page); + i++; + +if (dev_state->state->print_mask & +BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH_VERBOSE ) +pr_info ("#%u: bytenr=%llu, len=%u, offset=%u\n" , + i, cur_bytenr, bvec.bv_len, bvec.bv_offset); + cur_bytenr += bvec.bv_len; + } + +btrfsic_process_written_block (dev_state, dev_bytenr, mapped_datav, segs, + bio, &bio_is_patched, bio->bi_opf); +kfree (mapped_datav); +} + +static void btrfsic_check_flush_bio (struct bio *bio, +struct btrfsic_dev_state *dev_state) +{ +if (dev_state->state->print_mask &BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH ) +pr_info ("submit_bio(rw=%d,0x%x FLUSH, bdev=%p)\n" , +bio_op (bio), bio->bi_opf, bio->bi_bdev); + +if (dev_state->dummy_block_for_bio_bh_flush.is_iodone) { +struct btrfsic_block *const block = + &dev_state->dummy_block_for_bio_bh_flush; + + block->is_iodone =0 ; + block->never_written =0 ; + block->iodone_w_error =0 ; + block->flush_gen = dev_state->last_flush_gen +1 ; + block->submit_bio_bh_rw = bio->bi_opf; + block->orig_bio_private = bio->bi_private; + block->orig_bio_end_io = bio->bi_end_io; + block->next_in_same_bio =NULL ; + bio->bi_private = block; + bio->bi_end_io = btrfsic_bio_end_io; + }else if ((dev_state->state->print_mask & + (BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH | +BTRFSIC_PRINT_MASK_VERBOSE ))) { +pr_info ( +"btrfsic_submit_bio(%pg) with FLUSH but dummy block already in use (ignored)!\n" , + dev_state->bdev); + } +} +static void __btrfsic_submit_bio (struct bio *bio) {struct btrfsic_dev_state *dev_state;
@@ -2642,80 +2710,18 @@ static void __btrfsic_submit_bio(struct bio *bio)if (!btrfsic_is_initialized)return ; -mutex_lock (&btrfsic_mutex); -/* since btrfsic_submit_bio() is also called before -* btrfsic_mount(), this might return NULL */ +/* +* We can be called before btrfsic_mount, so there might not be a +* dev_state. +*/ dev_state =btrfsic_dev_state_lookup (bio->bi_bdev->bd_dev); -if (NULL != dev_state && - (bio_op (bio) ==REQ_OP_WRITE ) &&bio_has_data (bio)) { -int i =0 ; -u64 dev_bytenr; -u64 cur_bytenr; -struct bio_vec bvec; -struct bvec_iter iter; -int bio_is_patched; -char **mapped_datav; -unsigned int segs =bio_segments (bio); - - dev_bytenr =512 * bio->bi_iter.bi_sector; - bio_is_patched =0 ; -if (dev_state->state->print_mask & -BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH ) -pr_info ("submit_bio(rw=%d,0x%x, bi_vcnt=%u, bi_sector=%llu (bytenr %llu), bi_bdev=%p)\n" , -bio_op (bio), bio->bi_opf, segs, - bio->bi_iter.bi_sector, dev_bytenr, bio->bi_bdev); - - mapped_datav =kmalloc_array (segs, -sizeof (*mapped_datav),GFP_NOFS ); -if (!mapped_datav) - goto leave; - cur_bytenr = dev_bytenr; - -bio_for_each_segment (bvec, bio, iter) { -BUG_ON (bvec.bv_len !=PAGE_SIZE ); - mapped_datav[i] =page_address (bvec.bv_page); - i++; - -if (dev_state->state->print_mask & -BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH_VERBOSE ) -pr_info ("#%u: bytenr=%llu, len=%u, offset=%u\n" , - i, cur_bytenr, bvec.bv_len, bvec.bv_offset); - cur_bytenr += bvec.bv_len; - } -btrfsic_process_written_block (dev_state, dev_bytenr, - mapped_datav, segs, - bio, &bio_is_patched, - bio->bi_opf); -kfree (mapped_datav); - }else if (NULL != dev_state && (bio->bi_opf &REQ_PREFLUSH )) { -if (dev_state->state->print_mask & -BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH ) -pr_info ("submit_bio(rw=%d,0x%x FLUSH, bdev=%p)\n" , -bio_op (bio), bio->bi_opf, bio->bi_bdev); -if (!dev_state->dummy_block_for_bio_bh_flush.is_iodone) { -if ((dev_state->state->print_mask & - (BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH | -BTRFSIC_PRINT_MASK_VERBOSE ))) -pr_info ( -"btrfsic_submit_bio(%pg) with FLUSH but dummy block already in use (ignored)!\n" , - dev_state->bdev); - }else { -struct btrfsic_block *const block = - &dev_state->dummy_block_for_bio_bh_flush; - - block->is_iodone =0 ; - block->never_written =0 ; - block->iodone_w_error =0 ; - block->flush_gen = dev_state->last_flush_gen +1 ; - block->submit_bio_bh_rw = bio->bi_opf; - block->orig_bio_private = bio->bi_private; - block->orig_bio_end_io = bio->bi_end_io; - block->next_in_same_bio =NULL ; - bio->bi_private = block; - bio->bi_end_io = btrfsic_bio_end_io; - } +mutex_lock (&btrfsic_mutex); +if (dev_state) { +if (bio_op (bio) ==REQ_OP_WRITE &&bio_has_data (bio)) +btrfsic_check_write_bio (bio, dev_state); +else if (bio->bi_opf &REQ_PREFLUSH ) +btrfsic_check_flush_bio (bio, dev_state); } -leave:mutex_unlock (&btrfsic_mutex); }
Signature · 1 line
2.30.2
PATCH 6/40 btrfs: split submit_bio from btrfsic checking
In reply to Christoph HellwigRequire a separate call to the integrity checking helpers from the actual bio submission.
--- fs/btrfs/check-integrity.c | 14 +------------- fs/btrfs/check-integrity.h | 8 ++++---- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/extent_io.c | 3 ++- fs/btrfs/scrub.c | 12 ++++++++---- fs/btrfs/volumes.c | 3 ++- 6 files changed, 21 insertions(+), 25 deletions(-)
fs/btrfs/check-integrity.c +1 -13
@@ -2703,7 +2703,7 @@ static void btrfsic_check_flush_bio(struct bio *bio, } } -static void __btrfsic_submit_bio (struct bio *bio) +void btrfsic_check_bio (struct bio *bio) {struct btrfsic_dev_state *dev_state;
@@ -2725,18 +2725,6 @@ static void __btrfsic_submit_bio(struct bio *bio)mutex_unlock (&btrfsic_mutex); } -void btrfsic_submit_bio (struct bio *bio) -{ -__btrfsic_submit_bio (bio); -submit_bio (bio); -} - -int btrfsic_submit_bio_wait (struct bio *bio) -{ -__btrfsic_submit_bio (bio); -return submit_bio_wait (bio); -} -int btrfsic_mount (struct btrfs_fs_info *fs_info,struct btrfs_fs_devices *fs_devices,int including_extent_data,u32 print_mask)
fs/btrfs/check-integrity.h +4 -4
@@ -7,11 +7,11 @@#define BTRFS_CHECK_INTEGRITY_H #ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY -void btrfsic_submit_bio (struct bio *bio); -int btrfsic_submit_bio_wait (struct bio *bio); +void btrfsic_check_bio (struct bio *bio);#else -#define btrfsic_submit_bio submit_bio -#define btrfsic_submit_bio_wait submit_bio_wait +static inline void btrfsic_check_bio (struct bio *bio) +{ +}#endif int btrfsic_mount(struct btrfs_fs_info *fs_info,
fs/btrfs/disk-io.c +4 -2
@@ -4048,7 +4048,8 @@ static int write_dev_supers(struct btrfs_device *device,if (i ==0 && !btrfs_test_opt (device->fs_info,NOBARRIER )) bio->bi_opf |=REQ_FUA ; -btrfsic_submit_bio (bio); +btrfsic_check_bio (bio); +submit_bio (bio);if (btrfs_advance_sb_log (device, i)) errors++;
@@ -4161,7 +4162,8 @@ static void write_dev_flush(struct btrfs_device *device)init_completion (&device->flush_wait); bio->bi_private = &device->flush_wait; -btrfsic_submit_bio (bio); +btrfsic_check_bio (bio); +submit_bio (bio);set_bit (BTRFS_DEV_STATE_FLUSH_SENT , &device->dev_state); }
fs/btrfs/extent_io.c +2 -1
@@ -2370,7 +2370,8 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start, bio->bi_opf =REQ_OP_WRITE |REQ_SYNC ;bio_add_page (bio, page, length, pg_offset); -if (btrfsic_submit_bio_wait (bio)) { +btrfsic_check_bio (bio); +if (submit_bio_wait (bio)) {/* try to remap that extent elsewhere? */ btrfs_bio_counter_dec (fs_info);bio_put (bio);
fs/btrfs/scrub.c +8 -4
@@ -1479,7 +1479,8 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info, bio->bi_iter.bi_sector = spage->physical >>9 ; bio->bi_opf =REQ_OP_READ ; -if (btrfsic_submit_bio_wait (bio)) { +btrfsic_check_bio (bio); +if (submit_bio_wait (bio)) { spage->io_error =1 ; sblock->no_io_error_seen =0 ; }
@@ -1565,7 +1566,8 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad,return -EIO ; } -if (btrfsic_submit_bio_wait (bio)) { +btrfsic_check_bio (bio); +if (submit_bio_wait (bio)) {btrfs_dev_stat_inc_and_print (spage_bad->dev,BTRFS_DEV_STAT_WRITE_ERRS );atomic64_inc (&fs_info->dev_replace.num_write_errors);
@@ -1723,7 +1725,8 @@ static void scrub_wr_submit(struct scrub_ctx *sctx)* orders the requests before sending them to the driver which * doubled the write performance on spinning disks when measured * with Linux 3.5 */ -btrfsic_submit_bio (sbio->bio); +btrfsic_check_bio (sbio->bio); +submit_bio (sbio->bio);if (btrfs_is_zoned (sctx->fs_info)) sctx->write_pointer = sbio->physical + sbio->page_count *
@@ -2057,7 +2060,8 @@ static void scrub_submit(struct scrub_ctx *sctx) sbio = sctx->bios[sctx->curr]; sctx->curr =-1 ;scrub_pending_bio_inc (sctx); -btrfsic_submit_bio (sbio->bio); +btrfsic_check_bio (sbio->bio); +submit_bio (sbio->bio); } static int scrub_add_page_to_rd_bio(struct scrub_ctx *sctx,
fs/btrfs/volumes.c +2 -1
@@ -6755,7 +6755,8 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,btrfs_bio_counter_inc_noblocked (fs_info); -btrfsic_submit_bio (bio); +btrfsic_check_bio (bio); +submit_bio (bio); }static void bioc_error (struct btrfs_io_context *bioc,struct bio *bio,u64 logical)
Signature · 1 line
2.30.2
Re: [PATCH 06/40] btrfs: split submit_bio from btrfsic checking
Quoted 2 lines · Christoph Hellwig
Require a separate call to the integrity checking helpers from the actual bio submission.
All-in for this!
I can't remember how many times I feel embarrassed when the bio submission is done inside a function which by its name should only do sanity checks.
Thanks,
Qu
Quoted 159 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/check-integrity.c | 14 +------------- fs/btrfs/check-integrity.h | 8 ++++---- fs/btrfs/disk-io.c | 6 ++++-- fs/btrfs/extent_io.c | 3 ++- fs/btrfs/scrub.c | 12 ++++++++---- fs/btrfs/volumes.c | 3 ++- 6 files changed, 21 insertions(+), 25 deletions(-)fs/btrfs/check-integrity.c +1
-13@@ -2703,7 +2703,7 @@ static void btrfsic_check_flush_bio(struct bio *bio, } } -static void __btrfsic_submit_bio (struct bio *bio) +void btrfsic_check_bio (struct bio *bio) {struct btrfsic_dev_state *dev_state;@@ -2725,18 +2725,6 @@ static void __btrfsic_submit_bio(struct bio *bio)mutex_unlock (&btrfsic_mutex); } -void btrfsic_submit_bio (struct bio *bio) -{ -__btrfsic_submit_bio (bio); -submit_bio (bio); -} - -int btrfsic_submit_bio_wait (struct bio *bio) -{ -__btrfsic_submit_bio (bio); -return submit_bio_wait (bio); -} -int btrfsic_mount (struct btrfs_fs_info *fs_info,struct btrfs_fs_devices *fs_devices,int including_extent_data,u32 print_mask)fs/btrfs/check-integrity.h +4
-4@@ -7,11 +7,11 @@#define BTRFS_CHECK_INTEGRITY_H #ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY -void btrfsic_submit_bio (struct bio *bio); -int btrfsic_submit_bio_wait (struct bio *bio); +void btrfsic_check_bio (struct bio *bio);#else -#define btrfsic_submit_bio submit_bio -#define btrfsic_submit_bio_wait submit_bio_wait +static inline void btrfsic_check_bio (struct bio *bio) +{ +}#endif int btrfsic_mount(struct btrfs_fs_info *fs_info,fs/btrfs/disk-io.c +4
-2@@ -4048,7 +4048,8 @@ static int write_dev_supers(struct btrfs_device *device,if (i ==0 && !btrfs_test_opt (device->fs_info,NOBARRIER )) bio->bi_opf |=REQ_FUA ; -btrfsic_submit_bio (bio); +btrfsic_check_bio (bio); +submit_bio (bio);if (btrfs_advance_sb_log (device, i)) errors++;@@ -4161,7 +4162,8 @@ static void write_dev_flush(struct btrfs_device *device)init_completion (&device->flush_wait); bio->bi_private = &device->flush_wait; -btrfsic_submit_bio (bio); +btrfsic_check_bio (bio); +submit_bio (bio);set_bit (BTRFS_DEV_STATE_FLUSH_SENT , &device->dev_state); }fs/btrfs/extent_io.c +2
-1@@ -2370,7 +2370,8 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start, bio->bi_opf =REQ_OP_WRITE |REQ_SYNC ;bio_add_page (bio, page, length, pg_offset); -if (btrfsic_submit_bio_wait (bio)) { +btrfsic_check_bio (bio); +if (submit_bio_wait (bio)) {/* try to remap that extent elsewhere? */ btrfs_bio_counter_dec (fs_info);bio_put (bio);fs/btrfs/scrub.c +8
-4@@ -1479,7 +1479,8 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info, bio->bi_iter.bi_sector = spage->physical >>9 ; bio->bi_opf =REQ_OP_READ ; -if (btrfsic_submit_bio_wait (bio)) { +btrfsic_check_bio (bio); +if (submit_bio_wait (bio)) { spage->io_error =1 ; sblock->no_io_error_seen =0 ; }@@ -1565,7 +1566,8 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad,return -EIO ; } -if (btrfsic_submit_bio_wait (bio)) { +btrfsic_check_bio (bio); +if (submit_bio_wait (bio)) {btrfs_dev_stat_inc_and_print (spage_bad->dev,BTRFS_DEV_STAT_WRITE_ERRS );atomic64_inc (&fs_info->dev_replace.num_write_errors);@@ -1723,7 +1725,8 @@ static void scrub_wr_submit(struct scrub_ctx *sctx)* orders the requests before sending them to the driver which * doubled the write performance on spinning disks when measured * with Linux 3.5 */ -btrfsic_submit_bio (sbio->bio); +btrfsic_check_bio (sbio->bio); +submit_bio (sbio->bio);if (btrfs_is_zoned (sctx->fs_info)) sctx->write_pointer = sbio->physical + sbio->page_count *@@ -2057,7 +2060,8 @@ static void scrub_submit(struct scrub_ctx *sctx) sbio = sctx->bios[sctx->curr]; sctx->curr =-1 ;scrub_pending_bio_inc (sctx); -btrfsic_submit_bio (sbio->bio); +btrfsic_check_bio (sbio->bio); +submit_bio (sbio->bio); } static int scrub_add_page_to_rd_bio(struct scrub_ctx *sctx,fs/btrfs/volumes.c +2
-1@@ -6755,7 +6755,8 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,btrfs_bio_counter_inc_noblocked (fs_info); -btrfsic_submit_bio (bio); +btrfsic_check_bio (bio); +submit_bio (bio); }static void bioc_error (struct btrfs_io_context *bioc,struct bio *bio,u64 logical)
PATCH 7/40 btrfs: simplify btrfsic_read_block
In reply to Christoph Hellwigbtrfsic_read_block does not need the btrfs_bio structure, so switch to plain bio_alloc.
--- fs/btrfs/check-integrity.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
fs/btrfs/check-integrity.c +2 -3
@@ -1563,10 +1563,9 @@ static int btrfsic_read_block(struct btrfsic_state *state,struct bio *bio;unsigned int j; - bio =btrfs_bio_alloc (num_pages - i); -bio_set_dev (bio, block_ctx->dev->bdev); + bio =bio_alloc (block_ctx->dev->bdev, num_pages - i, +REQ_OP_READ ,GFP_NOFS ); bio->bi_iter.bi_sector = dev_bytenr >>9 ; - bio->bi_opf =REQ_OP_READ ;for (j = i; j < num_pages; j++) { ret = bio_add_page(bio, block_ctx->pagev[j],
Signature · 1 line
2.30.2
PATCH 8/40 btrfs: simplify repair_io_failure
In reply to Christoph HellwigThe I/O in repair_io_failue is synchronous and doesn't need a btrfs_bio, so just use an on-stack bio. Also cleanup the error handling to use goto labels and not discard the actual return values.
--- fs/btrfs/extent_io.c | 52 ++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 28 deletions(-)
fs/btrfs/extent_io.c +24 -28
@@ -2307,12 +2307,13 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start, u64 length, u64 logical, struct page *page, unsigned int pg_offset, int mirror_num) { - struct bio *bio; struct btrfs_device *dev; +struct bio_vec bvec; +struct bio bio;u64 map_length =0 ;u64 sector;struct btrfs_io_context *bioc =NULL ; -int ret; +int ret =0 ;ASSERT (!(fs_info->sb->s_flags &SB_RDONLY ));BUG_ON (!mirror_num);
@@ -2320,8 +2321,6 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start,if (btrfs_repair_one_zone (fs_info, logical))return 0 ; - bio =btrfs_bio_alloc (1 ); - bio->bi_iter.bi_size =0 ; map_length = length;/*
@@ -2339,53 +2338,50 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start,*/ ret =btrfs_map_block (fs_info,BTRFS_MAP_READ , logical, &map_length, &bioc,0 ); -if (ret) { -btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return -EIO ; - } +if (ret) + goto out_counter_dec;ASSERT (bioc->mirror_num ==1 ); }else { ret =btrfs_map_block (fs_info,BTRFS_MAP_WRITE , logical, &map_length, &bioc, mirror_num); -if (ret) { -btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return -EIO ; - } +if (ret) + goto out_counter_dec;BUG_ON (mirror_num != bioc->mirror_num); } sector = bioc->stripes[bioc->mirror_num -1 ].physical >>9 ; - bio->bi_iter.bi_sector = sector; dev = bioc->stripes[bioc->mirror_num -1 ].dev;btrfs_put_bioc (bioc); +if (!dev || !dev->bdev || !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state)) { -btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return -EIO ; + ret = -EIO ; + goto out_counter_dec; } -bio_set_dev (bio, dev->bdev); - bio->bi_opf =REQ_OP_WRITE |REQ_SYNC ; -bio_add_page (bio, page, length, pg_offset); -btrfsic_check_bio (bio); -if (submit_bio_wait (bio)) { +bio_init (&bio, dev->bdev, &bvec,1 ,REQ_OP_WRITE |REQ_SYNC ); + bio.bi_iter.bi_sector = sector; +__bio_add_page (&bio, page, length, pg_offset); + +btrfsic_check_bio (&bio); + ret =submit_bio_wait (&bio); +if (ret) {/* try to remap that extent elsewhere? */ -btrfs_bio_counter_dec (fs_info); -bio_put (bio);btrfs_dev_stat_inc_and_print (dev,BTRFS_DEV_STAT_WRITE_ERRS ); -return -EIO ; + goto out_bio_uninit; }btrfs_info_rl_in_rcu (fs_info,"read error corrected: ino %llu off %llu (dev %s sector %llu)" , ino, start,rcu_str_deref (dev->name), sector); + ret =0 ; + +out_bio_uninit: +bio_uninit (&bio); +out_counter_dec:btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return 0 ; +return ret; }int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num)
Signature · 1 line
2.30.2
Re: [PATCH 08/40] btrfs: simplify repair_io_failure
Quoted 3 lines · Christoph Hellwig
The I/O in
repair_io_failueis synchronous and doesn't need abtrfs_bio, so just use an on-stack bio. Also cleanup the error handling to use goto labels and not discard the actual return values.
Didn't even know we can do on-stack bio.
Looks good to me.
Thanks,
Qu
Quoted 110 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/extent_io.c | 52 ++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 28 deletions(-)fs/btrfs/extent_io.c +24
-28@@ -2307,12 +2307,13 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start, u64 length, u64 logical, struct page *page, unsigned int pg_offset, int mirror_num) { - struct bio *bio; struct btrfs_device *dev; +struct bio_vec bvec; +struct bio bio;u64 map_length =0 ;u64 sector;struct btrfs_io_context *bioc =NULL ; -int ret; +int ret =0 ;ASSERT (!(fs_info->sb->s_flags &SB_RDONLY ));BUG_ON (!mirror_num);@@ -2320,8 +2321,6 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start,if (btrfs_repair_one_zone (fs_info, logical))return 0 ; - bio =btrfs_bio_alloc (1 ); - bio->bi_iter.bi_size =0 ; map_length = length;/* @@ -2339,53 +2338,50 @@ static int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start,*/ ret =btrfs_map_block (fs_info,BTRFS_MAP_READ , logical, &map_length, &bioc,0 ); -if (ret) { -btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return -EIO ; - } +if (ret) + goto out_counter_dec;ASSERT (bioc->mirror_num ==1 ); }else { ret =btrfs_map_block (fs_info,BTRFS_MAP_WRITE , logical, &map_length, &bioc, mirror_num); -if (ret) { -btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return -EIO ; - } +if (ret) + goto out_counter_dec;BUG_ON (mirror_num != bioc->mirror_num); } sector = bioc->stripes[bioc->mirror_num -1 ].physical >>9 ; - bio->bi_iter.bi_sector = sector; dev = bioc->stripes[bioc->mirror_num -1 ].dev;btrfs_put_bioc (bioc); +if (!dev || !dev->bdev || !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state)) { -btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return -EIO ; + ret = -EIO ; + goto out_counter_dec; } -bio_set_dev (bio, dev->bdev); - bio->bi_opf =REQ_OP_WRITE |REQ_SYNC ; -bio_add_page (bio, page, length, pg_offset); -btrfsic_check_bio (bio); -if (submit_bio_wait (bio)) { +bio_init (&bio, dev->bdev, &bvec,1 ,REQ_OP_WRITE |REQ_SYNC ); + bio.bi_iter.bi_sector = sector; +__bio_add_page (&bio, page, length, pg_offset); + +btrfsic_check_bio (&bio); + ret =submit_bio_wait (&bio); +if (ret) {/* try to remap that extent elsewhere? */ -btrfs_bio_counter_dec (fs_info); -bio_put (bio);btrfs_dev_stat_inc_and_print (dev,BTRFS_DEV_STAT_WRITE_ERRS ); -return -EIO ; + goto out_bio_uninit; }btrfs_info_rl_in_rcu (fs_info,"read error corrected: ino %llu off %llu (dev %s sector %llu)" , ino, start,rcu_str_deref (dev->name), sector); + ret =0 ; + +out_bio_uninit: +bio_uninit (&bio); +out_counter_dec:btrfs_bio_counter_dec (fs_info); -bio_put (bio); -return 0 ; +return ret; }int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num)
PATCH 9/40 btrfs: simplify scrub_recheck_block
In reply to Christoph HellwigThe I/O in repair_io_failue is synchronous and doesn't need a btrfs_bio, so just use an on-stack bio.
--- fs/btrfs/scrub.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
fs/btrfs/scrub.c +8 -10
@@ -1462,8 +1462,9 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info,return scrub_recheck_block_on_raid56 (fs_info, sblock);for (page_num =0 ; page_num < sblock->page_count; page_num++) { -struct bio *bio;struct scrub_page *spage = sblock->pagev[page_num]; +struct bio bio; +struct bio_vec bvec;if (spage->dev->bdev ==NULL ) { spage->io_error =1 ;
@@ -1472,20 +1473,17 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info, }WARN_ON (!spage->page); - bio =btrfs_bio_alloc (1 ); -bio_set_dev (bio, spage->dev->bdev); - -bio_add_page (bio, spage->page, fs_info->sectorsize,0 ); - bio->bi_iter.bi_sector = spage->physical >>9 ; - bio->bi_opf =REQ_OP_READ ; +bio_init (&bio, spage->dev->bdev, &bvec,1 ,REQ_OP_READ ); +__bio_add_page (&bio, spage->page, fs_info->sectorsize,0 ); + bio.bi_iter.bi_sector = spage->physical >>9 ; -btrfsic_check_bio (bio); -if (submit_bio_wait (bio)) { +btrfsic_check_bio (&bio); +if (submit_bio_wait (&bio)) { spage->io_error =1 ; sblock->no_io_error_seen =0 ; } -bio_put (bio); +bio_uninit (&bio); }if (sblock->no_io_error_seen)
Signature · 1 line
2.30.2
Re: [PATCH 09/40] btrfs: simplify scrub_recheck_block
Quoted 2 lines · Christoph Hellwig
The I/O in
repair_io_failueis synchronous and doesn't need abtrfs_bio, so just use an on-stack bio.
Just an unrelated question below.
Quoted 23 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/scrub.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)fs/btrfs/scrub.c +2
-1@@ -1462,8 +1462,9 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info,return scrub_recheck_block_on_raid56 (fs_info, sblock);for (page_num =0 ; page_num < sblock->page_count; page_num++) { -struct bio *bio;struct scrub_page *spage = sblock->pagev[page_num]; +struct bio bio; +struct bio_vec bvec;if (spage->dev->bdev ==NULL ) { spage->io_error =1 ;@@ -1472,20 +1473,17 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info, }
fs/btrfs/scrub.c static void scrub_recheck_block(struct btrfs_fs_info *fs_info, · line 1477
Quoted 10 lines
fs/btrfs/scrub.c +2
-6WARN_ON (!spage->page); - bio =btrfs_bio_alloc (1 ); -bio_set_dev (bio, spage->dev->bdev); - -bio_add_page (bio, spage->page, fs_info->sectorsize,0 ); - bio->bi_iter.bi_sector = spage->physical >>9 ; - bio->bi_opf =REQ_OP_READ ; +bio_init (&bio, spage->dev->bdev, &bvec,1 ,REQ_OP_READ ); +__bio_add_page (&bio, spage->page, fs_info->sectorsize,0 );
Can we make the naming for __bio_add_page() better?
With more on-stack bio usage, such __bio_add_page() is really a little embarrassing.
Thanks,
Qu
Quoted 15 lines
fs/btrfs/scrub.c +4
-3+ bio.bi_iter.bi_sector = spage->physical >>9 ; -btrfsic_check_bio (bio); -if (submit_bio_wait (bio)) { +btrfsic_check_bio (&bio); +if (submit_bio_wait (&bio)) { spage->io_error =1 ; sblock->no_io_error_seen =0 ; } -bio_put (bio); +bio_uninit (&bio); }if (sblock->no_io_error_seen)
Re: [PATCH 09/40] btrfs: simplify scrub_recheck_block
Quoted 14 lines · Qu Wenruo
fs/btrfs/scrub.c static void scrub_recheck_block(struct btrfs_fs_info *fs_info, · line 1477
Quoted 11 lines
} WARN_ON(!spage->page);fs/btrfs/scrub.c +2
-6- bio =btrfs_bio_alloc (1 ); -bio_set_dev (bio, spage->dev->bdev); - -bio_add_page (bio, spage->page, fs_info->sectorsize,0 ); - bio->bi_iter.bi_sector = spage->physical >>9 ; - bio->bi_opf =REQ_OP_READ ; +bio_init (&bio, spage->dev->bdev, &bvec,1 ,REQ_OP_READ ); +__bio_add_page (&bio, spage->page, fs_info->sectorsize,0 );Can we make the naming for
__bio_add_page()better?
Quoted 2 lines
With more on-stack bio usage, such
__bio_add_page()is really a little embarrassing.
__bio_add_page is really just a micro-optimize version of __bio_add_page for sinle page users like this. To be honest we should probably just stop using it and I should not have added it here.
PATCH 10/40 btrfs: simplify scrub_repair_page_from_good_copy
In reply to Christoph HellwigThe I/O in repair_io_failue is synchronous and doesn't need a btrfs_bio, so just use an on-stack bio.
--- fs/btrfs/scrub.c | 23 +++++++++-------------- 1 file changed, 9 insertions(+), 14 deletions(-)
fs/btrfs/scrub.c +9 -14
@@ -1544,7 +1544,8 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad,BUG_ON (spage_good->page ==NULL );if (force_write || sblock_bad->header_error || sblock_bad->checksum_error || spage_bad->io_error) { -struct bio *bio; +struct bio bio; +struct bio_vec bvec;int ret;if (!spage_bad->dev->bdev) {
@@ -1553,26 +1554,20 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad,return -EIO ; } - bio =btrfs_bio_alloc (1 ); -bio_set_dev (bio, spage_bad->dev->bdev); - bio->bi_iter.bi_sector = spage_bad->physical >>9 ; - bio->bi_opf =REQ_OP_WRITE ; +bio_init (&bio, spage_bad->dev->bdev, &bvec,1 ,REQ_OP_WRITE ); + bio.bi_iter.bi_sector = spage_bad->physical >>9 ; +__bio_add_page (&bio, spage_good->page, sectorsize,0 ); - ret =bio_add_page (bio, spage_good->page, sectorsize,0 ); -if (ret != sectorsize) { -bio_put (bio); -return -EIO ; - } +btrfsic_check_bio (&bio); + ret =submit_bio_wait (&bio); +bio_uninit (&bio); -btrfsic_check_bio (bio); -if (submit_bio_wait (bio)) { +if (ret) {btrfs_dev_stat_inc_and_print (spage_bad->dev,BTRFS_DEV_STAT_WRITE_ERRS );atomic64_inc (&fs_info->dev_replace.num_write_errors); -bio_put (bio);return -EIO ; } -bio_put (bio); }return 0 ;
Signature · 1 line
2.30.2
Re: [PATCH 10/40] btrfs: simplify scrub_repair_page_from_good_copy
Quoted 4 lines · Christoph Hellwig
The I/O in
repair_io_failueis synchronous and doesn't need abtrfs_bio, so just use an on-stack bio.Signed-off-by Christoph Hellwig <[email protected]>
Thanks,
Qu
Quoted 52 lines
--- fs/btrfs/scrub.c | 23 +++++++++-------------- 1 file changed, 9 insertions(+), 14 deletions(-)fs/btrfs/scrub.c +9
-14@@ -1544,7 +1544,8 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad,BUG_ON (spage_good->page ==NULL );if (force_write || sblock_bad->header_error || sblock_bad->checksum_error || spage_bad->io_error) { -struct bio *bio; +struct bio bio; +struct bio_vec bvec;int ret;if (!spage_bad->dev->bdev) {@@ -1553,26 +1554,20 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad,return -EIO ; } - bio =btrfs_bio_alloc (1 ); -bio_set_dev (bio, spage_bad->dev->bdev); - bio->bi_iter.bi_sector = spage_bad->physical >>9 ; - bio->bi_opf =REQ_OP_WRITE ; +bio_init (&bio, spage_bad->dev->bdev, &bvec,1 ,REQ_OP_WRITE ); + bio.bi_iter.bi_sector = spage_bad->physical >>9 ; +__bio_add_page (&bio, spage_good->page, sectorsize,0 ); - ret =bio_add_page (bio, spage_good->page, sectorsize,0 ); -if (ret != sectorsize) { -bio_put (bio); -return -EIO ; - } +btrfsic_check_bio (&bio); + ret =submit_bio_wait (&bio); +bio_uninit (&bio); -btrfsic_check_bio (bio); -if (submit_bio_wait (bio)) { +if (ret) {btrfs_dev_stat_inc_and_print (spage_bad->dev,BTRFS_DEV_STAT_WRITE_ERRS );atomic64_inc (&fs_info->dev_replace.num_write_errors); -bio_put (bio);return -EIO ; } -bio_put (bio); }return 0 ;
PATCH 11/40 btrfs: move the call to bio_set_dev out of submit_stripe_bio
In reply to Christoph HellwigPrepare for additional refactoring.
--- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/volumes.c +1 -1
@@ -6751,7 +6751,6 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,bio_op (bio), bio->bi_opf, bio->bi_iter.bi_sector, (unsigned long )dev->bdev->bd_dev,rcu_str_deref (dev->name), dev->devid, bio->bi_iter.bi_size); -bio_set_dev (bio, dev->bdev);btrfs_bio_counter_inc_noblocked (fs_info);
@@ -6843,6 +6842,7 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,else bio = first_bio; +bio_set_dev (bio, dev->bdev);submit_stripe_bio (bioc, bio, bioc->stripes[dev_nr].physical, dev); }btrfs_bio_counter_dec (fs_info);
Signature · 1 line
2.30.2
PATCH 12/40 btrfs: pass a block_device to btrfs_bio_clone
In reply to Christoph HellwigPass the block_device to bio_alloc_clone instead of setting it later.
--- fs/btrfs/extent_io.c | 4 ++-- fs/btrfs/extent_io.h | 2 +- fs/btrfs/volumes.c | 9 +++++---- 3 files changed, 8 insertions(+), 7 deletions(-)
fs/btrfs/extent_io.c +2 -2
@@ -3150,13 +3150,13 @@ struct bio *btrfs_bio_alloc(unsigned int nr_iovecs)return bio; } -struct bio *btrfs_bio_clone (struct bio *bio) +struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio) {struct btrfs_bio *bbio;struct bio *new;/* Bio allocation backed by a bioset does not fail */ - new =bio_alloc_clone (bio->bi_bdev, bio,GFP_NOFS , &btrfs_bioset); + new =bio_alloc_clone (bdev, bio,GFP_NOFS , &btrfs_bioset); bbio =btrfs_bio (new);btrfs_bio_init (bbio); bbio->iter = bio->bi_iter;
fs/btrfs/extent_io.h +1 -1
@@ -278,7 +278,7 @@ void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end,struct page *locked_page, u32 bits_to_clear, unsigned long page_ops);struct bio *btrfs_bio_alloc (unsigned int nr_iovecs); -struct bio *btrfs_bio_clone (struct bio *bio); +struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio);struct bio *btrfs_bio_clone_partial (struct bio *orig,u64 offset,u64 size);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);
fs/btrfs/volumes.c +5 -4
@@ -6837,12 +6837,13 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,continue ; } -if (dev_nr < total_devs -1 ) - bio =btrfs_bio_clone (first_bio); -else +if (dev_nr < total_devs -1 ) { + bio =btrfs_bio_clone (dev->bdev, first_bio); + }else { bio = first_bio; +bio_set_dev (bio, dev->bdev); + } -bio_set_dev (bio, dev->bdev);submit_stripe_bio (bioc, bio, bioc->stripes[dev_nr].physical, dev); }btrfs_bio_counter_dec (fs_info);
Signature · 1 line
2.30.2
PATCH 13/40 btrfs: initialize ->bi_opf and ->bi_private in rbio_add_io_page
In reply to Christoph HellwigPrepare for further refactoring by moving this initialization to a single place.
--- fs/btrfs/raid56.c | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-)
fs/btrfs/raid56.c +18 -20
@@ -1069,7 +1069,8 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,struct page *page, int stripe_nr, unsigned long page_index, - unsigned long bio_max_len) + unsigned long bio_max_len, + unsigned int opf) { struct bio *last = bio_list->tail;int ret;
@@ -1106,7 +1107,9 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio,btrfs_bio (bio)->device = stripe->dev; bio->bi_iter.bi_size =0 ;bio_set_dev (bio, stripe->dev->bdev); + bio->bi_opf = opf; bio->bi_iter.bi_sector = disk_start >>9 ; + bio->bi_private = rbio;bio_add_page (bio, page,PAGE_SIZE ,0 );bio_list_add (bio_list, bio);
@@ -1275,7 +1278,8 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio) } ret =rbio_add_io_page (rbio, &bio_list, - page, stripe, pagenr, rbio->stripe_len); + page, stripe, pagenr, rbio->stripe_len, +REQ_OP_WRITE );if (ret) goto cleanup; }
@@ -1300,7 +1304,8 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio) ret =rbio_add_io_page (rbio, &bio_list, page, rbio->bioc->tgtdev_map[stripe], - pagenr, rbio->stripe_len); + pagenr, rbio->stripe_len, +REQ_OP_WRITE );if (ret) goto cleanup; }
@@ -1311,9 +1316,7 @@ static noinline void finish_rmw(struct btrfs_raid_bio *rbio)BUG_ON (atomic_read (&rbio->stripes_pending) ==0 );while ((bio =bio_list_pop (&bio_list))) { - bio->bi_private = rbio; bio->bi_end_io = raid_write_end_io; - bio->bi_opf =REQ_OP_WRITE ;submit_bio (bio); }
@@ -1517,7 +1520,8 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)continue ; ret =rbio_add_io_page (rbio, &bio_list, page, - stripe, pagenr, rbio->stripe_len); + stripe, pagenr, rbio->stripe_len, +REQ_OP_READ );if (ret) goto cleanup; }
@@ -1540,9 +1544,7 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio)*/ atomic_set (&rbio->stripes_pending, bios_to_read);while ((bio =bio_list_pop (&bio_list))) { - bio->bi_private = rbio; bio->bi_end_io = raid_rmw_end_io; - bio->bi_opf =REQ_OP_READ ;btrfs_bio_wq_end_io (rbio->bioc->fs_info, bio,BTRFS_WQ_ENDIO_RAID56 );
@@ -2059,7 +2061,8 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio) ret =rbio_add_io_page (rbio, &bio_list,rbio_stripe_page (rbio, stripe, pagenr), - stripe, pagenr, rbio->stripe_len); + stripe, pagenr, rbio->stripe_len, +REQ_OP_READ );if (ret <0 ) goto cleanup; }
@@ -2086,9 +2089,7 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio)*/ atomic_set (&rbio->stripes_pending, bios_to_read);while ((bio =bio_list_pop (&bio_list))) { - bio->bi_private = rbio; bio->bi_end_io = raid_recover_end_io; - bio->bi_opf =REQ_OP_READ ;btrfs_bio_wq_end_io (rbio->bioc->fs_info, bio,BTRFS_WQ_ENDIO_RAID56 );
@@ -2419,8 +2420,8 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,struct page *page; page =rbio_stripe_page (rbio, rbio->scrubp, pagenr); - ret =rbio_add_io_page (rbio, &bio_list, - page, rbio->scrubp, pagenr, rbio->stripe_len); + ret =rbio_add_io_page (rbio, &bio_list, page, rbio->scrubp, + pagenr, rbio->stripe_len,REQ_OP_WRITE );if (ret) goto cleanup; }
@@ -2434,7 +2435,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio, page =rbio_stripe_page (rbio, rbio->scrubp, pagenr); ret =rbio_add_io_page (rbio, &bio_list, page, bioc->tgtdev_map[rbio->scrubp], - pagenr, rbio->stripe_len); + pagenr, rbio->stripe_len,REQ_OP_WRITE );if (ret) goto cleanup; }
@@ -2450,9 +2451,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,atomic_set (&rbio->stripes_pending, nr_data);while ((bio =bio_list_pop (&bio_list))) { - bio->bi_private = rbio; bio->bi_end_io = raid_write_end_io; - bio->bi_opf =REQ_OP_WRITE ;submit_bio (bio); }
@@ -2604,8 +2603,9 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)if (PageUptodate (page))continue ; - ret =rbio_add_io_page (rbio, &bio_list, page, - stripe, pagenr, rbio->stripe_len); + ret =rbio_add_io_page (rbio, &bio_list, page, stripe, + pagenr, rbio->stripe_len, +REQ_OP_READ );if (ret) goto cleanup; }
@@ -2628,9 +2628,7 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio)*/ atomic_set (&rbio->stripes_pending, bios_to_read);while ((bio =bio_list_pop (&bio_list))) { - bio->bi_private = rbio; bio->bi_end_io = raid56_parity_scrub_end_io; - bio->bi_opf =REQ_OP_READ ;btrfs_bio_wq_end_io (rbio->bioc->fs_info, bio,BTRFS_WQ_ENDIO_RAID56 );
Signature · 1 line
2.30.2
PATCH 14/40 btrfs: don't allocate a btrfs_bio for raid56 per-stripe bios
In reply to Christoph HellwigExcept for the spurious initialization of ->device just after allocation nothing uses the btrfs_bio, so just allocate a normal bio without extra data.
--- fs/btrfs/raid56.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
fs/btrfs/raid56.c +2 -5
@@ -1103,11 +1103,8 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio, }/* put a new bio on the list */ - bio =btrfs_bio_alloc (bio_max_len >>PAGE_SHIFT ?:1 ); -btrfs_bio (bio)->device = stripe->dev; - bio->bi_iter.bi_size =0 ; -bio_set_dev (bio, stripe->dev->bdev); - bio->bi_opf = opf; + bio =bio_alloc (stripe->dev->bdev,max (bio_max_len >>PAGE_SHIFT ,1UL ), + opf,GFP_NOFS ); bio->bi_iter.bi_sector = disk_start >>9 ; bio->bi_private = rbio;
Signature · 1 line
2.30.2
Re: [PATCH 14/40] btrfs: don't allocate a btrfs_bio for raid56 per-stripe bios
Quoted 5 lines · Christoph Hellwig
Except for the spurious initialization of ->device just after allocation nothing uses the
btrfs_bio, so just allocate a normal bio without extra data.Signed-off-by Christoph Hellwig <[email protected]>
RAID56 layer doesn't need to bother special things like mirror_num nor checksum, as it's at a lower layer under logical layer.
So this is completely fine, and save quite some cleanup I'm going to do related to RAID56.
Quoted 22 lines
--- fs/btrfs/raid56.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)fs/btrfs/raid56.c +2
-5@@ -1103,11 +1103,8 @@ static int rbio_add_io_page(struct btrfs_raid_bio *rbio, }/* put a new bio on the list */ - bio =btrfs_bio_alloc (bio_max_len >>PAGE_SHIFT ?:1 ); -btrfs_bio (bio)->device = stripe->dev; - bio->bi_iter.bi_size =0 ; -bio_set_dev (bio, stripe->dev->bdev); - bio->bi_opf = opf; + bio =bio_alloc (stripe->dev->bdev,max (bio_max_len >>PAGE_SHIFT ,1UL ), + opf,GFP_NOFS ); bio->bi_iter.bi_sector = disk_start >>9 ; bio->bi_private = rbio;
PATCH 15/40 btrfs: don't allocate a btrfs_bio for scrub bios
In reply to Christoph HellwigAll the scrub bios go straight to the block device or the raid56 code, none of which looks at the btrfs_bio.
--- fs/btrfs/scrub.c | 47 ++++++++++++++++++----------------------------- 1 file changed, 18 insertions(+), 29 deletions(-)
fs/btrfs/scrub.c +18 -29
@@ -1415,8 +1415,8 @@ static void scrub_recheck_block_on_raid56(struct btrfs_fs_info *fs_info,if (!first_page->dev->bdev) goto out; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); -bio_set_dev (bio, first_page->dev->bdev); + bio =bio_alloc (first_page->dev->bdev,BIO_MAX_VECS ,REQ_OP_READ , +GFP_NOFS );for (page_num =0 ; page_num < sblock->page_count; page_num++) {struct scrub_page *spage = sblock->pagev[page_num];
@@ -1649,8 +1649,6 @@ static int scrub_add_page_to_wr_bio(struct scrub_ctx *sctx, } sbio = sctx->wr_curr_bio;if (sbio->page_count ==0 ) { -struct bio *bio; - ret =fill_writer_pointer_gap (sctx, spage->physical_for_dev_replace);if (ret) {
@@ -1661,17 +1659,14 @@ static int scrub_add_page_to_wr_bio(struct scrub_ctx *sctx, sbio->physical = spage->physical_for_dev_replace; sbio->logical = spage->logical; sbio->dev = sctx->wr_tgtdev; - bio = sbio->bio; -if (!bio) { - bio =btrfs_bio_alloc (sctx->pages_per_bio); - sbio->bio = bio; +if (!sbio->bio) { + sbio->bio =bio_alloc (sbio->dev->bdev, + sctx->pages_per_bio, +REQ_OP_WRITE ,GFP_NOFS ); } - - bio->bi_private = sbio; - bio->bi_end_io = scrub_wr_bio_end_io; -bio_set_dev (bio, sbio->dev->bdev); - bio->bi_iter.bi_sector = sbio->physical >>9 ; - bio->bi_opf =REQ_OP_WRITE ; + sbio->bio->bi_private = sbio; + sbio->bio->bi_end_io = scrub_wr_bio_end_io; + sbio->bio->bi_iter.bi_sector = sbio->physical >>9 ; sbio->status =0 ; } else if (sbio->physical + sbio->page_count * sectorsize != spage->physical_for_dev_replace ||
@@ -1712,7 +1707,6 @@ static void scrub_wr_submit(struct scrub_ctx *sctx) sbio = sctx->wr_curr_bio; sctx->wr_curr_bio =NULL ; -WARN_ON (!sbio->bio->bi_bdev);scrub_pending_bio_inc (sctx);/* process all writes in a single worker thread. Then the block layer * orders the requests before sending them to the driver which
@@ -2084,22 +2078,17 @@ static int scrub_add_page_to_rd_bio(struct scrub_ctx *sctx, } sbio = sctx->bios[sctx->curr];if (sbio->page_count ==0 ) { -struct bio *bio; - sbio->physical = spage->physical; sbio->logical = spage->logical; sbio->dev = spage->dev; - bio = sbio->bio; -if (!bio) { - bio =btrfs_bio_alloc (sctx->pages_per_bio); - sbio->bio = bio; +if (!sbio->bio) { + sbio->bio =bio_alloc (sbio->dev->bdev, + sctx->pages_per_bio, +REQ_OP_READ ,GFP_NOFS ); } - - bio->bi_private = sbio; - bio->bi_end_io = scrub_bio_end_io; -bio_set_dev (bio, sbio->dev->bdev); - bio->bi_iter.bi_sector = sbio->physical >>9 ; - bio->bi_opf =REQ_OP_READ ; + sbio->bio->bi_private = sbio; + sbio->bio->bi_end_io = scrub_bio_end_io; + sbio->bio->bi_iter.bi_sector = sbio->physical >>9 ; sbio->status =0 ; } else if (sbio->physical + sbio->page_count * sectorsize != spage->physical ||
@@ -2215,7 +2204,7 @@ static void scrub_missing_raid56_pages(struct scrub_block *sblock) goto bioc_out; } - bio =btrfs_bio_alloc (BIO_MAX_VECS ); + bio =bio_alloc (NULL ,BIO_MAX_VECS ,REQ_OP_READ ,GFP_NOFS ); bio->bi_iter.bi_sector = logical >>9 ; bio->bi_private = sblock; bio->bi_end_io = scrub_missing_raid56_end_io;
@@ -2831,7 +2820,7 @@ static void scrub_parity_check_and_repair(struct scrub_parity *sparity)if (ret || !bioc || !bioc->raid_map) goto bioc_out; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); + bio =bio_alloc (NULL ,BIO_MAX_VECS ,REQ_OP_READ ,GFP_NOFS ); bio->bi_iter.bi_sector = sparity->logic_start >>9 ; bio->bi_private = sparity; bio->bi_end_io = scrub_parity_bio_endio;
Signature · 1 line
2.30.2
Re: [PATCH 15/40] btrfs: don't allocate a btrfs_bio for scrub bios
Quoted 2 lines · Christoph Hellwig
All the scrub bios go straight to the block device or the raid56 code, none of which looks at the
btrfs_bio.
Exactly!
Thanks,
Qu
Quoted 111 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/scrub.c | 47 ++++++++++++++++++----------------------------- 1 file changed, 18 insertions(+), 29 deletions(-)fs/btrfs/scrub.c +18
-29@@ -1415,8 +1415,8 @@ static void scrub_recheck_block_on_raid56(struct btrfs_fs_info *fs_info,if (!first_page->dev->bdev) goto out; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); -bio_set_dev (bio, first_page->dev->bdev); + bio =bio_alloc (first_page->dev->bdev,BIO_MAX_VECS ,REQ_OP_READ , +GFP_NOFS );for (page_num =0 ; page_num < sblock->page_count; page_num++) {struct scrub_page *spage = sblock->pagev[page_num];@@ -1649,8 +1649,6 @@ static int scrub_add_page_to_wr_bio(struct scrub_ctx *sctx, } sbio = sctx->wr_curr_bio;if (sbio->page_count ==0 ) { -struct bio *bio; - ret =fill_writer_pointer_gap (sctx, spage->physical_for_dev_replace);if (ret) {@@ -1661,17 +1659,14 @@ static int scrub_add_page_to_wr_bio(struct scrub_ctx *sctx, sbio->physical = spage->physical_for_dev_replace; sbio->logical = spage->logical; sbio->dev = sctx->wr_tgtdev; - bio = sbio->bio; -if (!bio) { - bio =btrfs_bio_alloc (sctx->pages_per_bio); - sbio->bio = bio; +if (!sbio->bio) { + sbio->bio =bio_alloc (sbio->dev->bdev, + sctx->pages_per_bio, +REQ_OP_WRITE ,GFP_NOFS ); } - - bio->bi_private = sbio; - bio->bi_end_io = scrub_wr_bio_end_io; -bio_set_dev (bio, sbio->dev->bdev); - bio->bi_iter.bi_sector = sbio->physical >>9 ; - bio->bi_opf =REQ_OP_WRITE ; + sbio->bio->bi_private = sbio; + sbio->bio->bi_end_io = scrub_wr_bio_end_io; + sbio->bio->bi_iter.bi_sector = sbio->physical >>9 ; sbio->status =0 ; } else if (sbio->physical + sbio->page_count * sectorsize != spage->physical_for_dev_replace ||@@ -1712,7 +1707,6 @@ static void scrub_wr_submit(struct scrub_ctx *sctx) sbio = sctx->wr_curr_bio; sctx->wr_curr_bio =NULL ; -WARN_ON (!sbio->bio->bi_bdev);scrub_pending_bio_inc (sctx);/* process all writes in a single worker thread. Then the block layer * orders the requests before sending them to the driver which @@ -2084,22 +2078,17 @@ static int scrub_add_page_to_rd_bio(struct scrub_ctx *sctx, } sbio = sctx->bios[sctx->curr];if (sbio->page_count ==0 ) { -struct bio *bio; - sbio->physical = spage->physical; sbio->logical = spage->logical; sbio->dev = spage->dev; - bio = sbio->bio; -if (!bio) { - bio =btrfs_bio_alloc (sctx->pages_per_bio); - sbio->bio = bio; +if (!sbio->bio) { + sbio->bio =bio_alloc (sbio->dev->bdev, + sctx->pages_per_bio, +REQ_OP_READ ,GFP_NOFS ); } - - bio->bi_private = sbio; - bio->bi_end_io = scrub_bio_end_io; -bio_set_dev (bio, sbio->dev->bdev); - bio->bi_iter.bi_sector = sbio->physical >>9 ; - bio->bi_opf =REQ_OP_READ ; + sbio->bio->bi_private = sbio; + sbio->bio->bi_end_io = scrub_bio_end_io; + sbio->bio->bi_iter.bi_sector = sbio->physical >>9 ; sbio->status =0 ; } else if (sbio->physical + sbio->page_count * sectorsize != spage->physical ||@@ -2215,7 +2204,7 @@ static void scrub_missing_raid56_pages(struct scrub_block *sblock) goto bioc_out; } - bio =btrfs_bio_alloc (BIO_MAX_VECS ); + bio =bio_alloc (NULL ,BIO_MAX_VECS ,REQ_OP_READ ,GFP_NOFS ); bio->bi_iter.bi_sector = logical >>9 ; bio->bi_private = sblock; bio->bi_end_io = scrub_missing_raid56_end_io;@@ -2831,7 +2820,7 @@ static void scrub_parity_check_and_repair(struct scrub_parity *sparity)if (ret || !bioc || !bioc->raid_map) goto bioc_out; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); + bio =bio_alloc (NULL ,BIO_MAX_VECS ,REQ_OP_READ ,GFP_NOFS ); bio->bi_iter.bi_sector = sparity->logic_start >>9 ; bio->bi_private = sparity; bio->bi_end_io = scrub_parity_bio_endio;
PATCH 16/40 btrfs: stop using the btrfs_bio saved iter in index_rbio_pages
In reply to Christoph HellwigThe bios added to ->bio_list are the original bios fed into btrfs_map_bio, which are never advanced. Just use the iter in the bio itself.
--- fs/btrfs/raid56.c | 3 --- 1 file changed, 3 deletions(-)
fs/btrfs/raid56.c +0 -3
@@ -1155,9 +1155,6 @@ static void index_rbio_pages(struct btrfs_raid_bio *rbio) stripe_offset = start - rbio->bioc->raid_map[0 ]; page_index = stripe_offset >>PAGE_SHIFT ; -if (bio_flagged (bio,BIO_CLONED )) - bio->bi_iter =btrfs_bio (bio)->iter; -bio_for_each_segment (bvec, bio, iter) { rbio->bio_pages[page_index + i] = bvec.bv_page; i++;
Signature · 1 line
2.30.2
PATCH 17/40 btrfs: remove the submit_bio_hook argument to submit_read_repair
In reply to Christoph Hellwigsubmit_bio_hooks is always set to btrfs_submit_data_bio, so just remove it.
--- fs/btrfs/extent_io.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
fs/btrfs/extent_io.c +3 -5
@@ -2721,8 +2721,7 @@ static blk_status_t submit_read_repair(struct inode *inode, struct bio *failed_bio, u32 bio_offset, struct page *page, unsigned int pgoff, u64 start, u64 end, int failed_mirror, - unsigned int error_bitmap, - submit_bio_hook_t *submit_bio_hook) + unsigned int error_bitmap) { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);const u32 sectorsize = fs_info->sectorsize;
@@ -2760,7 +2759,7 @@ static blk_status_t submit_read_repair(struct inode *inode, ret =btrfs_repair_one_sector (inode, failed_bio, bio_offset + offset, page, pgoff + offset, start + offset, - failed_mirror, submit_bio_hook); + failed_mirror, btrfs_submit_data_bio);if (!ret) {/* * We have submitted the read repair, the page release
@@ -3075,8 +3074,7 @@ static void end_bio_extent_readpage(struct bio *bio)*/ submit_read_repair (inode, bio, bio_offset, page, start -page_offset (page), start, - end, mirror, error_bitmap, - btrfs_submit_data_bio); + end, mirror, error_bitmap);ASSERT (bio_offset + len > bio_offset); bio_offset += len;
Signature · 1 line
2.30.2
Re: [PATCH 17/40] btrfs: remove the submit_bio_hook argument to submit_read_repair
Quoted 3 lines · Christoph Hellwig
submit_bio_hooksis always set tobtrfs_submit_data_bio, so just remove it.
The same as my recent cleanup for it.
Although I did extra renaming as submit_read_repair() only works for data read.
Thanks,
Qu
Quoted 38 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/extent_io.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)fs/btrfs/extent_io.c +3
-5@@ -2721,8 +2721,7 @@ static blk_status_t submit_read_repair(struct inode *inode, struct bio *failed_bio, u32 bio_offset, struct page *page, unsigned int pgoff, u64 start, u64 end, int failed_mirror, - unsigned int error_bitmap, - submit_bio_hook_t *submit_bio_hook) + unsigned int error_bitmap) { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);const u32 sectorsize = fs_info->sectorsize;@@ -2760,7 +2759,7 @@ static blk_status_t submit_read_repair(struct inode *inode, ret =btrfs_repair_one_sector (inode, failed_bio, bio_offset + offset, page, pgoff + offset, start + offset, - failed_mirror, submit_bio_hook); + failed_mirror, btrfs_submit_data_bio);if (!ret) {/* * We have submitted the read repair, the page release @@ -3075,8 +3074,7 @@ static void end_bio_extent_readpage(struct bio *bio)*/ submit_read_repair (inode, bio, bio_offset, page, start -page_offset (page), start, - end, mirror, error_bitmap, - btrfs_submit_data_bio); + end, mirror, error_bitmap);ASSERT (bio_offset + len > bio_offset); bio_offset += len;
Re: [PATCH 17/40] btrfs: remove the submit_bio_hook argument to submit_read_repair
Quoted 7 lines · Qu Wenruo
Quoted 3 lines · Christoph Hellwig
submit_bio_hooksis always set tobtrfs_submit_data_bio, so just remove it.
Quoted 8 lines
The same as my recent cleanup for it.
Although I did extra renaming as
submit_read_repair()only works for data read.Reviewed-by Qu Wenruo <[email protected]>
I'm fine doing either version. This was just getting in the way of other repair changes, which is why we probbaly both came up with it.
PATCH 18/40 btrfs: move more work into btrfs_end_bioc
In reply to Christoph HellwigAssign ->mirror_num and ->bi_status in btrfs_end_bioc instead of duplicating the logic in the callers. Also remove the bio argument as it always must be bioc->orig_bio and the now pointless bioc_error that did nothing but assign bi_sector to the same value just sampled in the caller.
--- fs/btrfs/volumes.c | 68 ++++++++++++++-------------------------------- 1 file changed, 20 insertions(+), 48 deletions(-)
fs/btrfs/volumes.c +20 -48
@@ -6659,19 +6659,29 @@ int btrfs_map_sblock(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,return __btrfs_map_block (fs_info, op, logical, length, bioc_ret,0 ,1 ); } -static inline void btrfs_end_bioc (struct btrfs_io_context *bioc,struct bio *bio) +static inline void btrfs_end_bioc (struct btrfs_io_context *bioc) { +struct bio *bio = bioc->orig_bio; + +btrfs_bio (bio)->mirror_num = bioc->mirror_num; bio->bi_private = bioc->private; bio->bi_end_io = bioc->end_io; -bio_endio (bio); +/* +* Only send an error to the higher layers if it is beyond the tolerance +* threshold. +*/ +if (atomic_read (&bioc->error) > bioc->max_errors) + bio->bi_status =BLK_STS_IOERR ; +else + bio->bi_status =BLK_STS_OK ; +bio_endio (bio);btrfs_put_bioc (bioc); }static void btrfs_end_bio (struct bio *bio) {struct btrfs_io_context *bioc = bio->bi_private; -int is_orig_bio =0 ;if (bio->bi_status) {atomic_inc (&bioc->error);
@@ -6692,35 +6702,12 @@ static void btrfs_end_bio(struct bio *bio) } } -if (bio == bioc->orig_bio) - is_orig_bio =1 ; +if (bio != bioc->orig_bio) +bio_put (bio);btrfs_bio_counter_dec (bioc->fs_info); - -if (atomic_dec_and_test (&bioc->stripes_pending)) { -if (!is_orig_bio) { -bio_put (bio); - bio = bioc->orig_bio; - } - -btrfs_bio (bio)->mirror_num = bioc->mirror_num; -/* only send an error to the higher layers if it is -* beyond the tolerance of the btrfs bio -*/ -if (atomic_read (&bioc->error) > bioc->max_errors) { - bio->bi_status =BLK_STS_IOERR ; - }else { -/* -* this bio is actually up to date, we didn't -* go over the max number of errors -*/ - bio->bi_status =BLK_STS_OK ; - } - -btrfs_end_bioc (bioc, bio); - }else if (!is_orig_bio) { -bio_put (bio); - } +if (atomic_dec_and_test (&bioc->stripes_pending)) +btrfs_end_bioc (bioc); } static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,
@@ -6758,23 +6745,6 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,submit_bio (bio); } -static void bioc_error (struct btrfs_io_context *bioc,struct bio *bio,u64 logical) -{ -atomic_inc (&bioc->error); -if (atomic_dec_and_test (&bioc->stripes_pending)) { -/* Should be the original bio. */ -WARN_ON (bio != bioc->orig_bio); - -btrfs_bio (bio)->mirror_num = bioc->mirror_num; - bio->bi_iter.bi_sector = logical >>9 ; -if (atomic_read (&bioc->error) > bioc->max_errors) - bio->bi_status =BLK_STS_IOERR ; -else - bio->bi_status =BLK_STS_OK ; -btrfs_end_bioc (bioc, bio); - } -} -blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num) {
@@ -6833,7 +6803,9 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio, &dev->dev_state) || (btrfs_op(first_bio) == BTRFS_MAP_WRITE && !test_bit(BTRFS_DEV_STATE_WRITEABLE, &dev->dev_state))) { -bioc_error (bioc, first_bio, logical); +atomic_inc (&bioc->error); +if (atomic_dec_and_test (&bioc->stripes_pending)) +btrfs_end_bioc (bioc);continue ; }
Signature · 1 line
2.30.2
Re: [PATCH 18/40] btrfs: move more work into btrfs_end_bioc
Quoted 5 lines · Christoph Hellwig
Assign ->mirror_num and ->bi_status in
btrfs_end_biocinstead of duplicating the logic in the callers. Also remove the bio argument as it always must bebioc->orig_bioand the now pointlessbioc_errorthat did nothing but assignbi_sectorto the same value just sampled in the caller.
It may be better to rename @first_bio or the @bio parameter, as it takes me several seconds to realize that @bio get reused for RAID1*/DUP bio cloned submission.
Thanks,
Qu
Quoted 118 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/volumes.c | 68 ++++++++++++++-------------------------------- 1 file changed, 20 insertions(+), 48 deletions(-)fs/btrfs/volumes.c +20
-48@@ -6659,19 +6659,29 @@ int btrfs_map_sblock(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,return __btrfs_map_block (fs_info, op, logical, length, bioc_ret,0 ,1 ); } -static inline void btrfs_end_bioc (struct btrfs_io_context *bioc,struct bio *bio) +static inline void btrfs_end_bioc (struct btrfs_io_context *bioc) { +struct bio *bio = bioc->orig_bio; + +btrfs_bio (bio)->mirror_num = bioc->mirror_num; bio->bi_private = bioc->private; bio->bi_end_io = bioc->end_io; -bio_endio (bio); +/* +* Only send an error to the higher layers if it is beyond the tolerance +* threshold. +*/ +if (atomic_read (&bioc->error) > bioc->max_errors) + bio->bi_status =BLK_STS_IOERR ; +else + bio->bi_status =BLK_STS_OK ; +bio_endio (bio);btrfs_put_bioc (bioc); }static void btrfs_end_bio (struct bio *bio) {struct btrfs_io_context *bioc = bio->bi_private; -int is_orig_bio =0 ;if (bio->bi_status) {atomic_inc (&bioc->error);@@ -6692,35 +6702,12 @@ static void btrfs_end_bio(struct bio *bio) } } -if (bio == bioc->orig_bio) - is_orig_bio =1 ; +if (bio != bioc->orig_bio) +bio_put (bio);btrfs_bio_counter_dec (bioc->fs_info); - -if (atomic_dec_and_test (&bioc->stripes_pending)) { -if (!is_orig_bio) { -bio_put (bio); - bio = bioc->orig_bio; - } - -btrfs_bio (bio)->mirror_num = bioc->mirror_num; -/* only send an error to the higher layers if it is -* beyond the tolerance of the btrfs bio -*/ -if (atomic_read (&bioc->error) > bioc->max_errors) { - bio->bi_status =BLK_STS_IOERR ; - }else { -/* -* this bio is actually up to date, we didn't -* go over the max number of errors -*/ - bio->bi_status =BLK_STS_OK ; - } - -btrfs_end_bioc (bioc, bio); - }else if (!is_orig_bio) { -bio_put (bio); - } +if (atomic_dec_and_test (&bioc->stripes_pending)) +btrfs_end_bioc (bioc); } static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,@@ -6758,23 +6745,6 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,submit_bio (bio); } -static void bioc_error (struct btrfs_io_context *bioc,struct bio *bio,u64 logical) -{ -atomic_inc (&bioc->error); -if (atomic_dec_and_test (&bioc->stripes_pending)) { -/* Should be the original bio. */ -WARN_ON (bio != bioc->orig_bio); - -btrfs_bio (bio)->mirror_num = bioc->mirror_num; - bio->bi_iter.bi_sector = logical >>9 ; -if (atomic_read (&bioc->error) > bioc->max_errors) - bio->bi_status =BLK_STS_IOERR ; -else - bio->bi_status =BLK_STS_OK ; -btrfs_end_bioc (bioc, bio); - } -} -blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num) {@@ -6833,7 +6803,9 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio, &dev->dev_state) || (btrfs_op(first_bio) == BTRFS_MAP_WRITE && !test_bit(BTRFS_DEV_STATE_WRITEABLE, &dev->dev_state))) { -bioc_error (bioc, first_bio, logical); +atomic_inc (&bioc->error); +if (atomic_dec_and_test (&bioc->stripes_pending)) +btrfs_end_bioc (bioc);continue ; }
PATCH 19/40 btrfs: defer I/O completion based on the btrfs_raid_bio
In reply to Christoph HellwigInstead of attaching a an extra allocation an indirect call to each low-level bio issued by the RAID code, add a btrfs_work to struct btrfs_raid_bio and only defer the per-rbio completion action. The per-bio action for all the I/Os are trivial and can be safely done from interrupt context.
As a nice side effect this also allows sharing the boilerplate code for the per-bio completion.
--- fs/btrfs/disk-io.c | 6 +-- fs/btrfs/disk-io.h | 1 - fs/btrfs/raid56.c | 110 +++++++++++++++++++-------------------------- 3 files changed, 46 insertions(+), 71 deletions(-)
fs/btrfs/disk-io.c +1 -5
@@ -740,14 +740,10 @@ static void end_workqueue_bio(struct bio *bio) wq = fs_info->endio_meta_write_workers; else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_FREE_SPACE) wq = fs_info->endio_freespace_worker; - else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_RAID56) - wq = fs_info->endio_raid56_workers; else wq = fs_info->endio_write_workers; }else { -if (end_io_wq->metadata ==BTRFS_WQ_ENDIO_RAID56 ) - wq = fs_info->endio_raid56_workers; -else if (end_io_wq->metadata) +if (end_io_wq->metadata) wq = fs_info->endio_meta_workers;else wq = fs_info->endio_workers;
fs/btrfs/disk-io.h +0 -1
@@ -21,7 +21,6 @@ enum btrfs_wq_endio_type {BTRFS_WQ_ENDIO_DATA ,BTRFS_WQ_ENDIO_METADATA , BTRFS_WQ_ENDIO_FREE_SPACE, - BTRFS_WQ_ENDIO_RAID56, };static inline u64 btrfs_sb_offset (int mirror)
fs/btrfs/raid56.c +45 -65
177 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 20/40 btrfs: cleanup btrfs_submit_metadata_bio
In reply to Christoph HellwigRemove the unused bio_flags argument and clean up the code flow to be straight forward.
--- fs/btrfs/disk-io.c | 42 ++++++++++++++++-------------------------- fs/btrfs/disk-io.h | 2 +- fs/btrfs/extent_io.c | 2 +- 3 files changed, 18 insertions(+), 28 deletions(-)
fs/btrfs/disk-io.c +16 -26
@@ -890,6 +890,10 @@ static blk_status_t btree_submit_bio_start(struct inode *inode, struct bio *bio,return btree_csum_one_bio (bio); } +/* +* Check if metadata writes should be submitted by async threads so that +* checksumming can happen in parallel across all CPUs. +*/ static bool should_async_write (struct btrfs_fs_info *fs_info,struct btrfs_inode *bi) {
@@ -903,41 +907,27 @@ static bool should_async_write(struct btrfs_fs_info *fs_info, }blk_status_t btrfs_submit_metadata_bio (struct inode *inode,struct bio *bio, -int mirror_num,unsigned long bio_flags) +int mirror_num) {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);blk_status_t ret; -if (btrfs_op (bio) !=BTRFS_MAP_WRITE ) { -/* -* called for a read, do the setup so that checksum validation -* can happen in the async kernel threads -*/ - ret =btrfs_bio_wq_end_io (fs_info, bio, -BTRFS_WQ_ENDIO_METADATA ); -if (ret) - goto out_w_error; - ret =btrfs_map_bio (fs_info, bio, mirror_num); - }else if (!should_async_write (fs_info,BTRFS_I (inode))) { +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if (should_async_write (fs_info,BTRFS_I (inode))) +return btrfs_wq_submit_bio (inode, bio, mirror_num,0 ,0 , + btree_submit_bio_start); ret =btree_csum_one_bio (bio);if (ret) - goto out_w_error; - ret =btrfs_map_bio (fs_info, bio, mirror_num); +return ret; }else { -/* -* kthread helpers are used to submit writes so that -* checksumming can happen in parallel across all CPUs -*/ - ret =btrfs_wq_submit_bio (inode, bio, mirror_num,0 , -0 , btree_submit_bio_start); +/* checksum validation should happen in async threads: */ + ret =btrfs_bio_wq_end_io (fs_info, bio, +BTRFS_WQ_ENDIO_METADATA ); +if (ret) +return ret; } -if (ret) - goto out_w_error; -return 0 ; - -out_w_error: -return ret; +return btrfs_map_bio (fs_info, bio, mirror_num); }#ifdef CONFIG_MIGRATION
fs/btrfs/disk-io.h +1 -1
@@ -87,7 +87,7 @@ int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio, struct page *page, u64 start, u64 end, int mirror);blk_status_t btrfs_submit_metadata_bio (struct inode *inode,struct bio *bio, -int mirror_num,unsigned long bio_flags); +int mirror_num);#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS struct btrfs_root *btrfs_alloc_dummy_root (struct btrfs_fs_info *fs_info);#endif
fs/btrfs/extent_io.c +1 -1
@@ -179,7 +179,7 @@ int __must_check submit_one_bio(struct bio *bio, int mirror_num, bio_flags); else ret =btrfs_submit_metadata_bio (tree->private_data, bio, - mirror_num, bio_flags); + mirror_num);if (ret) { bio->bi_status = ret;
Signature · 1 line
2.30.2
Re: [PATCH 20/40] btrfs: cleanup btrfs_submit_metadata_bio
Quoted 2 lines · Christoph Hellwig
Remove the unused
bio_flagsargument and clean up the code flow to be straight forward.
That flag is a legacy when we use a function pointer for both data and metadata bio submission.
After commit 953651eb308f ("btrfs: factor out helper adding a page to bio") and commit 1b36294a6cd5 ("btrfs: call submit_bio_hook directly for metadata pages") we get rid of the hook, and no longer needs the extra @bio_flags.
Thanks,
Qu
Quoted 103 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/disk-io.c | 42 ++++++++++++++++-------------------------- fs/btrfs/disk-io.h | 2 +- fs/btrfs/extent_io.c | 2 +- 3 files changed, 18 insertions(+), 28 deletions(-)fs/btrfs/disk-io.c +16
-26@@ -890,6 +890,10 @@ static blk_status_t btree_submit_bio_start(struct inode *inode, struct bio *bio,return btree_csum_one_bio (bio); } +/* +* Check if metadata writes should be submitted by async threads so that +* checksumming can happen in parallel across all CPUs. +*/ static bool should_async_write (struct btrfs_fs_info *fs_info,struct btrfs_inode *bi) {@@ -903,41 +907,27 @@ static bool should_async_write(struct btrfs_fs_info *fs_info, }blk_status_t btrfs_submit_metadata_bio (struct inode *inode,struct bio *bio, -int mirror_num,unsigned long bio_flags) +int mirror_num) {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);blk_status_t ret; -if (btrfs_op (bio) !=BTRFS_MAP_WRITE ) { -/* -* called for a read, do the setup so that checksum validation -* can happen in the async kernel threads -*/ - ret =btrfs_bio_wq_end_io (fs_info, bio, -BTRFS_WQ_ENDIO_METADATA ); -if (ret) - goto out_w_error; - ret =btrfs_map_bio (fs_info, bio, mirror_num); - }else if (!should_async_write (fs_info,BTRFS_I (inode))) { +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if (should_async_write (fs_info,BTRFS_I (inode))) +return btrfs_wq_submit_bio (inode, bio, mirror_num,0 ,0 , + btree_submit_bio_start); ret =btree_csum_one_bio (bio);if (ret) - goto out_w_error; - ret =btrfs_map_bio (fs_info, bio, mirror_num); +return ret; }else { -/* -* kthread helpers are used to submit writes so that -* checksumming can happen in parallel across all CPUs -*/ - ret =btrfs_wq_submit_bio (inode, bio, mirror_num,0 , -0 , btree_submit_bio_start); +/* checksum validation should happen in async threads: */ + ret =btrfs_bio_wq_end_io (fs_info, bio, +BTRFS_WQ_ENDIO_METADATA ); +if (ret) +return ret; } -if (ret) - goto out_w_error; -return 0 ; - -out_w_error: -return ret; +return btrfs_map_bio (fs_info, bio, mirror_num); }#ifdef CONFIG_MIGRATION fs/btrfs/disk-io.h +1
-1@@ -87,7 +87,7 @@ int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio, struct page *page, u64 start, u64 end, int mirror);blk_status_t btrfs_submit_metadata_bio (struct inode *inode,struct bio *bio, -int mirror_num,unsigned long bio_flags); +int mirror_num);#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS struct btrfs_root *btrfs_alloc_dummy_root (struct btrfs_fs_info *fs_info);#endif fs/btrfs/extent_io.c +1
-1@@ -179,7 +179,7 @@ int __must_check submit_one_bio(struct bio *bio, int mirror_num, bio_flags); else ret =btrfs_submit_metadata_bio (tree->private_data, bio, - mirror_num, bio_flags); + mirror_num);if (ret) { bio->bi_status = ret;
PATCH 21/40 btrfs: cleanup btrfs_submit_data_bio
In reply to Christoph HellwigClean up the code flow to be straight forward.
--- fs/btrfs/inode.c | 85 +++++++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 48 deletions(-)
fs/btrfs/inode.c +37 -48
@@ -2511,67 +2511,56 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); -struct btrfs_root *root =BTRFS_I (inode)->root; -enum btrfs_wq_endio_type metadata =BTRFS_WQ_ENDIO_DATA ; -blk_status_t ret =0 ; -int skip_sum; -int async = !atomic_read (&BTRFS_I (inode)->sync_writers); - - skip_sum = (BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM ) || -test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state); - -if (btrfs_is_free_space_inode (BTRFS_I (inode))) - metadata =BTRFS_WQ_ENDIO_FREE_SPACE ; +struct btrfs_inode *bi =BTRFS_I (inode); +blk_status_t ret;if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { -struct page *page =bio_first_bvec_all (bio)->bv_page; -loff_t file_offset =page_offset (page); - - ret =extract_ordered_extent (BTRFS_I (inode), bio, file_offset); + ret =extract_ordered_extent (bi, bio, +page_offset (bio_first_bvec_all (bio)->bv_page));if (ret) - goto out; +return ret; } -if (btrfs_op (bio) !=BTRFS_MAP_WRITE ) { +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if ((bi->flags &BTRFS_INODE_NODATASUM ) || +test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state)) + goto mapit; + +if (!atomic_read (&bi->sync_writers)) { +/* csum items have already been cloned */ +if (btrfs_is_data_reloc_root (bi->root)) + goto mapit; +return btrfs_wq_submit_bio (inode, bio, mirror_num, bio_flags, +0 , btrfs_submit_bio_start); + } + ret =btrfs_csum_one_bio (bi, bio,0 ,0 ); +if (ret) +return ret; + }else { +enum btrfs_wq_endio_type metadata =BTRFS_WQ_ENDIO_DATA ; + +if (btrfs_is_free_space_inode (bi)) + metadata =BTRFS_WQ_ENDIO_FREE_SPACE ; + ret =btrfs_bio_wq_end_io (fs_info, bio, metadata);if (ret) - goto out; +return ret; -if (bio_flags &EXTENT_BIO_COMPRESSED ) { - ret =btrfs_submit_compressed_read (inode, bio, +if (bio_flags &EXTENT_BIO_COMPRESSED ) +return btrfs_submit_compressed_read (inode, bio, mirror_num, bio_flags); - goto out; - }else { -/* -* Lookup bio sums does extra checks around whether we -* need to csum or not, which is why we ignore skip_sum -* here. -*/ - ret =btrfs_lookup_bio_sums (inode, bio,NULL ); -if (ret) - goto out; - } - goto mapit; - }else if (async && !skip_sum) { -/* csum items have already been cloned */ -if (btrfs_is_data_reloc_root (root)) - goto mapit; -/* we're doing a write, do the async checksumming */ - ret =btrfs_wq_submit_bio (inode, bio, mirror_num, bio_flags, -0 , btrfs_submit_bio_start); - goto out; - }else if (!skip_sum) { - ret =btrfs_csum_one_bio (BTRFS_I (inode), bio,0 ,0 ); + +/* +* Lookup bio sums does extra checks around whether we need to +* csum or not, which is why we ignore skip_sum here. +*/ + ret =btrfs_lookup_bio_sums (inode, bio,NULL );if (ret) - goto out; +return ret; } - mapit: - ret =btrfs_map_bio (fs_info, bio, mirror_num); - -out: -return ret; +return btrfs_map_bio (fs_info, bio, mirror_num); }/*
Signature · 1 line
2.30.2
Re: [PATCH 21/40] btrfs: cleanup btrfs_submit_data_bio
Quoted 47 lines · Christoph Hellwig
Clean up the code flow to be straight forward.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/inode.c | 85 +++++++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 48 deletions(-)fs/btrfs/inode.c +10
-17@@ -2511,67 +2511,56 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); -struct btrfs_root *root =BTRFS_I (inode)->root; -enum btrfs_wq_endio_type metadata =BTRFS_WQ_ENDIO_DATA ; -blk_status_t ret =0 ; -int skip_sum; -int async = !atomic_read (&BTRFS_I (inode)->sync_writers); - - skip_sum = (BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM ) || -test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state); - -if (btrfs_is_free_space_inode (BTRFS_I (inode))) - metadata =BTRFS_WQ_ENDIO_FREE_SPACE ; +struct btrfs_inode *bi =BTRFS_I (inode); +blk_status_t ret;if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { -struct page *page =bio_first_bvec_all (bio)->bv_page; -loff_t file_offset =page_offset (page); - - ret =extract_ordered_extent (BTRFS_I (inode), bio, file_offset); + ret =extract_ordered_extent (bi, bio, +page_offset (bio_first_bvec_all (bio)->bv_page));if (ret) - goto out; +return ret; } -if (btrfs_op (bio) !=BTRFS_MAP_WRITE ) { +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if ((bi->flags &BTRFS_INODE_NODATASUM ) || +test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state)) + goto mapit; +
fs/btrfs/inode.c blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, · line 2538
Quoted 10 lines
fs/btrfs/inode.c +10
-0+if (!atomic_read (&bi->sync_writers)) { +/* csum items have already been cloned */ +if (btrfs_is_data_reloc_root (bi->root)) + goto mapit; +return btrfs_wq_submit_bio (inode, bio, mirror_num, bio_flags, +0 , btrfs_submit_bio_start); + } + ret =btrfs_csum_one_bio (bi, bio,0 ,0 ); +if (ret) +return ret;
Previously we would also call bio_endio() on the bio, do we miss the endio call on it?
Quoted 8 lines
fs/btrfs/inode.c +6
-0+ }else { +enum btrfs_wq_endio_type metadata =BTRFS_WQ_ENDIO_DATA ; + +if (btrfs_is_free_space_inode (bi)) + metadata =BTRFS_WQ_ENDIO_FREE_SPACE ; + ret =btrfs_bio_wq_end_io (fs_info, bio, metadata);if (ret)
fs/btrfs/inode.c blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, · line 2544
Quoted 10 lines
fs/btrfs/inode.c +3
-4- goto out; +return ret; -if (bio_flags &EXTENT_BIO_COMPRESSED ) { - ret =btrfs_submit_compressed_read (inode, bio, +if (bio_flags &EXTENT_BIO_COMPRESSED ) +return btrfs_submit_compressed_read (inode, bio, mirror_num, bio_flags); - goto out;
Previously the out tag also ends the io for the bio. Now we don't.
Quoted 20 lines
fs/btrfs/inode.c +0
-20- }else { -/* -* Lookup bio sums does extra checks around whether we -* need to csum or not, which is why we ignore skip_sum -* here. -*/ - ret =btrfs_lookup_bio_sums (inode, bio,NULL ); -if (ret) - goto out; - } - goto mapit; - }else if (async && !skip_sum) { -/* csum items have already been cloned */ -if (btrfs_is_data_reloc_root (root)) - goto mapit; -/* we're doing a write, do the async checksumming */ - ret =btrfs_wq_submit_bio (inode, bio, mirror_num, bio_flags, -0 , btrfs_submit_bio_start); - goto out; - }else if (!skip_sum) {
fs/btrfs/inode.c blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, · line 2560
Quoted 10 lines
fs/btrfs/inode.c +7
-2- ret =btrfs_csum_one_bio (BTRFS_I (inode), bio,0 ,0 ); + +/* +* Lookup bio sums does extra checks around whether we need to +* csum or not, which is why we ignore skip_sum here. +*/ + ret =btrfs_lookup_bio_sums (inode, bio,NULL );if (ret) - goto out; +return ret;
The same missing endio call for error path.
Thanks,
Qu
Quoted 11 lines
fs/btrfs/inode.c +1
-5} - mapit: - ret =btrfs_map_bio (fs_info, bio, mirror_num); - -out: -return ret; +return btrfs_map_bio (fs_info, bio, mirror_num); }/*
Re: [PATCH 21/40] btrfs: cleanup btrfs_submit_data_bio
Quoted 2 lines · Qu Wenruo
Previously we would also call
bio_endio()on the bio, do we miss the endio call on it?
The first patch in the series moves the bio_endio call into one of the callers of this function instead. That being said mainline has a different fix from Josef that makes the endio call on errors conditional so this part of the series will require some rework.
PATCH 22/40 btrfs: cleanup btrfs_submit_dio_bio
In reply to Christoph HellwigRemove the pointless goto just to return err and clean up the code flow to be a little more straight forward.
--- fs/btrfs/inode.c | 59 ++++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 32 deletions(-)
fs/btrfs/inode.c +27 -32
@@ -7844,47 +7844,42 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio, struct inode *inode, u64 file_offset, int async_submit) { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); +struct btrfs_inode *bi =BTRFS_I (inode);struct btrfs_dio_private *dip = bio->bi_private; -bool write =btrfs_op (bio) ==BTRFS_MAP_WRITE ;blk_status_t ret; -/* Check btrfs_submit_bio_hook() for rules about async submit. */ -if (async_submit) - async_submit = !atomic_read (&BTRFS_I (inode)->sync_writers); +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if (!(bi->flags &BTRFS_INODE_NODATASUM )) { +/* See btrfs_submit_data_bio for async submit rules */ +if (async_submit && !atomic_read (&bi->sync_writers)) +return btrfs_wq_submit_bio (inode, bio,0 ,0 , + file_offset, + btrfs_submit_bio_start_direct_io); -if (!write) { +/* +* If we aren't doing async submit, calculate the csum of the +* bio now. +*/ + ret =btrfs_csum_one_bio (bi, bio, file_offset,1 ); +if (ret) +return ret; + } + }else { ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA );if (ret) - goto err; - } - -if (BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM ) - goto map; +return ret; -if (write && async_submit) { - ret =btrfs_wq_submit_bio (inode, bio,0 ,0 , file_offset, - btrfs_submit_bio_start_direct_io); - goto err; - }else if (write) { -/* -* If we aren't doing async submit, calculate the csum of the -* bio now. -*/ - ret =btrfs_csum_one_bio (BTRFS_I (inode), bio, file_offset,1 ); -if (ret) - goto err; - }else { -u64 csum_offset; +if (!(bi->flags &BTRFS_INODE_NODATASUM )) { +u64 csum_offset; - csum_offset = file_offset - dip->file_offset; - csum_offset >>= fs_info->sectorsize_bits; - csum_offset *= fs_info->csum_size; -btrfs_bio (bio)->csum = dip->csums + csum_offset; + csum_offset = file_offset - dip->file_offset; + csum_offset >>= fs_info->sectorsize_bits; + csum_offset *= fs_info->csum_size; +btrfs_bio (bio)->csum = dip->csums + csum_offset; + } } -map: - ret =btrfs_map_bio (fs_info, bio,0 ); -err: -return ret; + +return btrfs_map_bio (fs_info, bio,0 ); }/*
Signature · 1 line
2.30.2
Re: [PATCH 22/40] btrfs: cleanup btrfs_submit_dio_bio
Quoted 74 lines · Christoph Hellwig
Remove the pointless goto just to return err and clean up the code flow to be a little more straight forward.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/inode.c | 59 ++++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 32 deletions(-)fs/btrfs/inode.c +22
-28@@ -7844,47 +7844,42 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio, struct inode *inode, u64 file_offset, int async_submit) { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); +struct btrfs_inode *bi =BTRFS_I (inode);struct btrfs_dio_private *dip = bio->bi_private; -bool write =btrfs_op (bio) ==BTRFS_MAP_WRITE ;blk_status_t ret; -/* Check btrfs_submit_bio_hook() for rules about async submit. */ -if (async_submit) - async_submit = !atomic_read (&BTRFS_I (inode)->sync_writers); +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if (!(bi->flags &BTRFS_INODE_NODATASUM )) { +/* See btrfs_submit_data_bio for async submit rules */ +if (async_submit && !atomic_read (&bi->sync_writers)) +return btrfs_wq_submit_bio (inode, bio,0 ,0 , + file_offset, + btrfs_submit_bio_start_direct_io); -if (!write) { +/* +* If we aren't doing async submit, calculate the csum of the +* bio now. +*/ + ret =btrfs_csum_one_bio (bi, bio, file_offset,1 ); +if (ret) +return ret; + } + }else { ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA );if (ret) - goto err; - } - -if (BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM ) - goto map; +return ret; -if (write && async_submit) { - ret =btrfs_wq_submit_bio (inode, bio,0 ,0 , file_offset, - btrfs_submit_bio_start_direct_io); - goto err; - }else if (write) { -/* -* If we aren't doing async submit, calculate the csum of the -* bio now. -*/ - ret =btrfs_csum_one_bio (BTRFS_I (inode), bio, file_offset,1 ); -if (ret) - goto err; - }else { -u64 csum_offset; +if (!(bi->flags &BTRFS_INODE_NODATASUM )) { +u64 csum_offset; - csum_offset = file_offset - dip->file_offset; - csum_offset >>= fs_info->sectorsize_bits; - csum_offset *= fs_info->csum_size; -btrfs_bio (bio)->csum = dip->csums + csum_offset; + csum_offset = file_offset - dip->file_offset; + csum_offset >>= fs_info->sectorsize_bits;
fs/btrfs/inode.c static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio, · line 7882
Quoted 10 lines
fs/btrfs/inode.c +5
-4+ csum_offset *= fs_info->csum_size; +btrfs_bio (bio)->csum = dip->csums + csum_offset; + } } -map: - ret =btrfs_map_bio (fs_info, bio,0 ); -err: -return ret; + +return btrfs_map_bio (fs_info, bio,0 );
Can we just put btrfs_map_bio() call into each read/write branch?
In fact it's the shared single btrfs_map_bio() still requires us to use if () {} else {}.
I manually checked the code, it looks fine to me.
Although related to btrfs_op(bio), personally I would put some extra ASSERT()s to make sure in this function we only got either MAP_WRITE or MAP_READ, no other values allowed.
Thanks,
Qu
Quoted 3 lines
fs/btrfs/inode.c +0
-0}/*
Re: [PATCH 22/40] btrfs: cleanup btrfs_submit_dio_bio
Quoted 1 line · Qu Wenruo
Can we just put
btrfs_map_bio()call into each read/write branch?
We could - that being said I find this version a lot cleaner as it has a self-explanatory flow.
PATCH 23/40 btrfs: store an inode pointer in struct btrfs_bio
In reply to Christoph HellwigAll the I/O going through the btrfs_bio based path are associated with an inode. Add a pointer to it to simplify a few things soon. Also pass the bio operation to btrfs_bio_alloc given that we have to touch it anyway.
--- fs/btrfs/compression.c | 4 +--- fs/btrfs/extent_io.c | 23 ++++++++++++----------- fs/btrfs/extent_io.h | 6 ++++-- fs/btrfs/inode.c | 3 ++- fs/btrfs/volumes.h | 2 ++ 5 files changed, 21 insertions(+), 17 deletions(-)
fs/btrfs/compression.c +1 -3
@@ -464,10 +464,8 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_bytestruct bio *bio;int ret; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); - + bio =btrfs_bio_alloc (cb->inode,BIO_MAX_VECS , opf); bio->bi_iter.bi_sector = disk_bytenr >>SECTOR_SHIFT ; - bio->bi_opf = opf; bio->bi_private = cb; bio->bi_end_io = endio_func;
fs/btrfs/extent_io.c +12 -11
@@ -2657,10 +2657,9 @@ int btrfs_repair_one_sector(struct inode *inode,return -EIO ; } - repair_bio =btrfs_bio_alloc (1 ); + repair_bio =btrfs_bio_alloc (inode,1 ,REQ_OP_READ ); repair_bbio =btrfs_bio (repair_bio); repair_bbio->file_offset = start; - repair_bio->bi_opf =REQ_OP_READ ; repair_bio->bi_end_io = failed_bio->bi_end_io; repair_bio->bi_iter.bi_sector = failrec->logical >>9 ; repair_bio->bi_private = failed_bio->bi_private;
@@ -3128,9 +3127,10 @@ static void end_bio_extent_readpage(struct bio *bio)* new bio by bio_alloc_bioset as it does not initialize the bytes outside of * 'bio' because use of __GFP_ZERO is not supported. */ -static inline void btrfs_bio_init (struct btrfs_bio *bbio) +static inline void btrfs_bio_init (struct btrfs_bio *bbio,struct inode *inode) {memset (bbio,0 , offsetof(struct btrfs_bio , bio)); + bbio->inode = inode; }/*
@@ -3138,13 +3138,14 @@ static inline void btrfs_bio_init(struct btrfs_bio *bbio)* * The bio allocation is backed by bioset and does not fail. */ -struct bio *btrfs_bio_alloc (unsigned int nr_iovecs) +struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs, +unsigned int opf) {struct bio *bio;ASSERT (0 < nr_iovecs && nr_iovecs <=BIO_MAX_VECS ); - bio =bio_alloc_bioset (NULL , nr_iovecs,0 ,GFP_NOFS , &btrfs_bioset); -btrfs_bio_init (btrfs_bio (bio)); + bio =bio_alloc_bioset (NULL , nr_iovecs, opf,GFP_NOFS , &btrfs_bioset); +btrfs_bio_init (btrfs_bio (bio), inode);return bio; }
@@ -3156,12 +3157,13 @@ struct bio *btrfs_bio_clone(struct block_device *bdev, struct bio *bio)/* Bio allocation backed by a bioset does not fail */ new =bio_alloc_clone (bdev, bio,GFP_NOFS , &btrfs_bioset); bbio =btrfs_bio (new); -btrfs_bio_init (bbio); +btrfs_bio_init (btrfs_bio (new),btrfs_bio (bio)->inode); bbio->iter = bio->bi_iter;return new; } -struct bio *btrfs_bio_clone_partial (struct bio *orig,u64 offset,u64 size) +struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, +u64 offset,u64 size) {struct bio *bio;struct btrfs_bio *bbio;
@@ -3173,7 +3175,7 @@ struct bio *btrfs_bio_clone_partial(struct bio *orig, u64 offset, u64 size)ASSERT (bio); bbio =btrfs_bio (bio); -btrfs_bio_init (bbio); +btrfs_bio_init (btrfs_bio (bio), inode);bio_trim (bio, offset >>9 , size >>9 ); bbio->iter = bio->bi_iter;
@@ -3308,7 +3310,7 @@ static int alloc_new_bio(struct btrfs_inode *inode,struct bio *bio;int ret; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); + bio =btrfs_bio_alloc (&inode->vfs_inode,BIO_MAX_VECS , opf);/* * For compressed page range, its disk_bytenr is always @disk_bytenr * passed in, no matter if we have added any range into previous bio.
@@ -3321,7 +3323,6 @@ static int alloc_new_bio(struct btrfs_inode *inode, bio_ctrl->bio_flags = bio_flags; bio->bi_end_io = end_io_func; bio->bi_private = &inode->io_tree; - bio->bi_opf = opf; ret =calc_bio_boundaries (bio_ctrl, inode, file_offset);if (ret <0 ) goto error;
fs/btrfs/extent_io.h +4 -2
@@ -277,9 +277,11 @@ void extent_range_redirty_for_io(struct inode *inode, u64 start, u64 end);void extent_clear_unlock_delalloc (struct btrfs_inode *inode,u64 start,u64 end,struct page *locked_page,u32 bits_to_clear,unsigned long page_ops); -struct bio *btrfs_bio_alloc (unsigned int nr_iovecs); +struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs, +unsigned int opf);struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio); -struct bio *btrfs_bio_clone_partial (struct bio *orig,u64 offset,u64 size); +struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, +u64 offset,u64 size);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num);
fs/btrfs/inode.c +2 -1
@@ -7987,7 +7987,8 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,* This will never fail as it's passing GPF_NOFS and * the allocation is backed by btrfs_bioset. */ - bio =btrfs_bio_clone_partial (dio_bio, clone_offset, clone_len); + bio =btrfs_bio_clone_partial (inode, dio_bio, clone_offset, + clone_len); bio->bi_private = dip; bio->bi_end_io = btrfs_end_dio_bio;btrfs_bio (bio)->file_offset = file_offset;
fs/btrfs/volumes.h +2 -0
@@ -321,6 +321,8 @@ struct btrfs_fs_devices {* Mostly for btrfs specific features like csum and mirror_num. */ struct btrfs_bio { +struct inode *inode; +unsigned int mirror_num;/* for direct I/O */
Signature · 1 line
2.30.2
Re: [PATCH 23/40] btrfs: store an inode pointer in struct btrfs_bio
Quoted 5 lines · Christoph Hellwig
All the I/O going through the
btrfs_biobased path are associated with an inode. Add a pointer to it to simplify a few things soon. Also pass the bio operation tobtrfs_bio_allocgiven that we have to touch it anyway.Signed-off-by Christoph Hellwig <[email protected]>
Something I want to avoid is to futher increasing the size of btrfs_bio.
For buffered uncompressed IO, we can grab the inode from the first page. For direct IO we have bio->bi_private (btrfs_dio_private). For compressed IO, it's bio->bi_private again (compressed_bio).
Do the saved code lines really validate the memory usage for all bios?
Thanks,
Qu
Quoted 157 lines
--- fs/btrfs/compression.c | 4 +--- fs/btrfs/extent_io.c | 23 ++++++++++++----------- fs/btrfs/extent_io.h | 6 ++++-- fs/btrfs/inode.c | 3 ++- fs/btrfs/volumes.h | 2 ++ 5 files changed, 21 insertions(+), 17 deletions(-)fs/btrfs/compression.c +1
-3@@ -464,10 +464,8 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_bytestruct bio *bio;int ret; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); - + bio =btrfs_bio_alloc (cb->inode,BIO_MAX_VECS , opf); bio->bi_iter.bi_sector = disk_bytenr >>SECTOR_SHIFT ; - bio->bi_opf = opf; bio->bi_private = cb; bio->bi_end_io = endio_func;fs/btrfs/extent_io.c +12
-11@@ -2657,10 +2657,9 @@ int btrfs_repair_one_sector(struct inode *inode,return -EIO ; } - repair_bio =btrfs_bio_alloc (1 ); + repair_bio =btrfs_bio_alloc (inode,1 ,REQ_OP_READ ); repair_bbio =btrfs_bio (repair_bio); repair_bbio->file_offset = start; - repair_bio->bi_opf =REQ_OP_READ ; repair_bio->bi_end_io = failed_bio->bi_end_io; repair_bio->bi_iter.bi_sector = failrec->logical >>9 ; repair_bio->bi_private = failed_bio->bi_private;@@ -3128,9 +3127,10 @@ static void end_bio_extent_readpage(struct bio *bio)* new bio by bio_alloc_bioset as it does not initialize the bytes outside of * 'bio' because use of __GFP_ZERO is not supported. */ -static inline void btrfs_bio_init (struct btrfs_bio *bbio) +static inline void btrfs_bio_init (struct btrfs_bio *bbio,struct inode *inode) {memset (bbio,0 , offsetof(struct btrfs_bio , bio)); + bbio->inode = inode; }/* @@ -3138,13 +3138,14 @@ static inline void btrfs_bio_init(struct btrfs_bio *bbio)* * The bio allocation is backed by bioset and does not fail. */ -struct bio *btrfs_bio_alloc (unsigned int nr_iovecs) +struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs, +unsigned int opf) {struct bio *bio;ASSERT (0 < nr_iovecs && nr_iovecs <=BIO_MAX_VECS ); - bio =bio_alloc_bioset (NULL , nr_iovecs,0 ,GFP_NOFS , &btrfs_bioset); -btrfs_bio_init (btrfs_bio (bio)); + bio =bio_alloc_bioset (NULL , nr_iovecs, opf,GFP_NOFS , &btrfs_bioset); +btrfs_bio_init (btrfs_bio (bio), inode);return bio; }@@ -3156,12 +3157,13 @@ struct bio *btrfs_bio_clone(struct block_device *bdev, struct bio *bio)/* Bio allocation backed by a bioset does not fail */ new =bio_alloc_clone (bdev, bio,GFP_NOFS , &btrfs_bioset); bbio =btrfs_bio (new); -btrfs_bio_init (bbio); +btrfs_bio_init (btrfs_bio (new),btrfs_bio (bio)->inode); bbio->iter = bio->bi_iter;return new; } -struct bio *btrfs_bio_clone_partial (struct bio *orig,u64 offset,u64 size) +struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, +u64 offset,u64 size) {struct bio *bio;struct btrfs_bio *bbio;@@ -3173,7 +3175,7 @@ struct bio *btrfs_bio_clone_partial(struct bio *orig, u64 offset, u64 size)ASSERT (bio); bbio =btrfs_bio (bio); -btrfs_bio_init (bbio); +btrfs_bio_init (btrfs_bio (bio), inode);bio_trim (bio, offset >>9 , size >>9 ); bbio->iter = bio->bi_iter;@@ -3308,7 +3310,7 @@ static int alloc_new_bio(struct btrfs_inode *inode,struct bio *bio;int ret; - bio =btrfs_bio_alloc (BIO_MAX_VECS ); + bio =btrfs_bio_alloc (&inode->vfs_inode,BIO_MAX_VECS , opf);/* * For compressed page range, its disk_bytenr is always @disk_bytenr * passed in, no matter if we have added any range into previous bio. @@ -3321,7 +3323,6 @@ static int alloc_new_bio(struct btrfs_inode *inode, bio_ctrl->bio_flags = bio_flags; bio->bi_end_io = end_io_func; bio->bi_private = &inode->io_tree; - bio->bi_opf = opf; ret =calc_bio_boundaries (bio_ctrl, inode, file_offset);if (ret <0 ) goto error;fs/btrfs/extent_io.h +4
-2@@ -277,9 +277,11 @@ void extent_range_redirty_for_io(struct inode *inode, u64 start, u64 end);void extent_clear_unlock_delalloc (struct btrfs_inode *inode,u64 start,u64 end,struct page *locked_page,u32 bits_to_clear,unsigned long page_ops); -struct bio *btrfs_bio_alloc (unsigned int nr_iovecs); +struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs, +unsigned int opf);struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio); -struct bio *btrfs_bio_clone_partial (struct bio *orig,u64 offset,u64 size); +struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, +u64 offset,u64 size);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num);fs/btrfs/inode.c +2
-1@@ -7987,7 +7987,8 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,* This will never fail as it's passing GPF_NOFS and * the allocation is backed by btrfs_bioset. */ - bio =btrfs_bio_clone_partial (dio_bio, clone_offset, clone_len); + bio =btrfs_bio_clone_partial (inode, dio_bio, clone_offset, + clone_len); bio->bi_private = dip; bio->bi_end_io = btrfs_end_dio_bio;btrfs_bio (bio)->file_offset = file_offset;fs/btrfs/volumes.h +2
-0@@ -321,6 +321,8 @@ struct btrfs_fs_devices {* Mostly for btrfs specific features like csum and mirror_num. */ struct btrfs_bio { +struct inode *inode; +unsigned int mirror_num;/* for direct I/O */
Re: [PATCH 23/40] btrfs: store an inode pointer in struct btrfs_bio
Quoted 7 lines · Qu Wenruo
Something I want to avoid is to futher increasing the size of
btrfs_bio.For buffered uncompressed IO, we can grab the inode from the first page. For direct IO we have
bio->bi_private(btrfs_dio_private). For compressed IO, it'sbio->bi_privateagain (compressed_bio).Do the saved code lines really validate the memory usage for all bios?
This isn't about the saved lines. It allows to remove the async submit and completion container structures that both point to an inode, and later on the dio_private structure. So overall it actually is a major memory saving.
PATCH 24/40 btrfs: remove btrfs_end_io_wq
In reply to Christoph HellwigAvoid the extra allocation for all read bios by embedding a btrfs_work and I/O end type into the btrfs_bio structure.
--- fs/btrfs/compression.c | 24 +++------ fs/btrfs/ctree.h | 1 - fs/btrfs/disk-io.c | 112 +---------------------------------------- fs/btrfs/disk-io.h | 10 ---- fs/btrfs/inode.c | 19 +++---- fs/btrfs/super.c | 11 +--- fs/btrfs/volumes.c | 44 ++++++++++++++-- fs/btrfs/volumes.h | 11 ++++ 8 files changed, 66 insertions(+), 166 deletions(-)
fs/btrfs/compression.c +8 -16
@@ -423,20 +423,6 @@ static void end_compressed_bio_write(struct bio *bio)bio_put (bio); } -static blk_status_t submit_compressed_bio (struct btrfs_fs_info *fs_info, -struct compressed_bio *cb, -struct bio *bio,int mirror_num) -{ -blk_status_t ret; - -ASSERT (bio->bi_iter.bi_size); - ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA ); -if (ret) -return ret; - ret =btrfs_map_bio (fs_info, bio, mirror_num); -return ret; -} -/* * Allocate a compressed_bio, which will be used to read/write on-disk * (aka, compressed) * data.
@@ -468,6 +454,10 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_byte bio->bi_iter.bi_sector = disk_bytenr >>SECTOR_SHIFT ; bio->bi_private = cb; bio->bi_end_io = endio_func; +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_WRITE ; +else +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ; em =btrfs_get_chunk_map (fs_info, disk_bytenr, fs_info->sectorsize);if (IS_ERR (em)) {
@@ -594,7 +584,8 @@ blk_status_t btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start, goto finish_cb; } - ret =submit_compressed_bio (fs_info, cb, bio,0 ); +ASSERT (bio->bi_iter.bi_size); + ret =btrfs_map_bio (fs_info, bio,0 );if (ret) goto finish_cb; bio =NULL ;
@@ -930,7 +921,8 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, fs_info->sectorsize); sums += fs_info->csum_size * nr_sectors; - ret =submit_compressed_bio (fs_info, cb, comp_bio, mirror_num); +ASSERT (comp_bio->bi_iter.bi_size); + ret =btrfs_map_bio (fs_info, comp_bio, mirror_num);if (ret) goto finish_cb; comp_bio =NULL ;
fs/btrfs/ctree.h +0 -1
@@ -823,7 +823,6 @@ struct btrfs_fs_info {struct btrfs_workqueue *endio_meta_workers;struct btrfs_workqueue *endio_raid56_workers;struct btrfs_workqueue *rmw_workers; -struct btrfs_workqueue *endio_meta_write_workers;struct btrfs_workqueue *endio_write_workers;struct btrfs_workqueue *endio_freespace_worker;struct btrfs_workqueue *caching_workers;
fs/btrfs/disk-io.c +1 -111
160 lines not on this page · raw message
fs/btrfs/disk-io.h +0 -10
@@ -17,12 +17,6 @@*/ #define BTRFS_BDEV_BLOCKSIZE (4096) -enum btrfs_wq_endio_type { -BTRFS_WQ_ENDIO_DATA , -BTRFS_WQ_ENDIO_METADATA , -BTRFS_WQ_ENDIO_FREE_SPACE , -}; -static inline u64 btrfs_sb_offset (int mirror) {u64 start =SZ_16K ;
@@ -119,8 +113,6 @@ int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid, int atomic);int btrfs_read_buffer (struct extent_buffer *buf,u64 parent_transid,int level,struct btrfs_key *first_key); -blk_status_t btrfs_bio_wq_end_io (struct btrfs_fs_info *info,struct bio *bio, -enum btrfs_wq_endio_type metadata); blk_status_t btrfs_wq_submit_bio(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 dio_file_offset,
@@ -144,8 +136,6 @@ int btree_lock_page_hook(struct page *page, void *data,int btrfs_get_num_tolerated_disk_barrier_failures (u64 flags);int btrfs_get_free_objectid (struct btrfs_root *root,u64 *objectid);int btrfs_init_root_free_objectid (struct btrfs_root *root); -int __init btrfs_end_io_wq_init (void ); -void __cold btrfs_end_io_wq_exit (void );#ifdef CONFIG_DEBUG_LOCK_ALLOC void btrfs_set_buffer_lockdep_class(u64 objectid,
fs/btrfs/inode.c +6 -13
@@ -2512,6 +2512,7 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);struct btrfs_inode *bi =BTRFS_I (inode); +struct btrfs_bio *bbio =btrfs_bio (bio);blk_status_t ret;if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) {
@@ -2537,14 +2538,10 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio,if (ret)return ret; }else { -enum btrfs_wq_endio_type metadata =BTRFS_WQ_ENDIO_DATA ; -if (btrfs_is_free_space_inode (bi)) - metadata =BTRFS_WQ_ENDIO_FREE_SPACE ; - - ret =btrfs_bio_wq_end_io (fs_info, bio, metadata); -if (ret) -return ret; + bbio->end_io_type =BTRFS_ENDIO_WQ_FREE_SPACE_READ ; +else + bbio->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ;if (bio_flags &EXTENT_BIO_COMPRESSED ) return btrfs_submit_compressed_read(inode, bio,
@@ -7739,9 +7736,7 @@ static blk_status_t submit_dio_repair_bio(struct inode *inode, struct bio *bio,BUG_ON (bio_op (bio) ==REQ_OP_WRITE ); - ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA ); -if (ret) -return ret; +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_WRITE ;refcount_inc (&dip->refs); ret =btrfs_map_bio (fs_info, bio, mirror_num);
@@ -7865,9 +7860,7 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio,return ret; } }else { - ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA ); -if (ret) -return ret; +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ;if (!(bi->flags &BTRFS_INODE_NODATASUM )) {u64 csum_offset;
fs/btrfs/super.c +1 -10
@@ -1835,8 +1835,6 @@ static void btrfs_resize_thread_pool(struct btrfs_fs_info *fs_info,btrfs_workqueue_set_max (fs_info->caching_workers, new_pool_size);btrfs_workqueue_set_max (fs_info->endio_workers, new_pool_size);btrfs_workqueue_set_max (fs_info->endio_meta_workers, new_pool_size); -btrfs_workqueue_set_max (fs_info->endio_meta_write_workers, - new_pool_size);btrfs_workqueue_set_max (fs_info->endio_write_workers, new_pool_size);btrfs_workqueue_set_max (fs_info->endio_freespace_worker, new_pool_size);btrfs_workqueue_set_max (fs_info->delayed_workers, new_pool_size);
@@ -2593,13 +2591,9 @@ static int __init init_btrfs_fs(void)if (err) goto free_delayed_ref; - err =btrfs_end_io_wq_init (); -if (err) - goto free_prelim_ref; - err =btrfs_interface_init ();if (err) - goto free_end_io_wq; + goto free_prelim_ref;btrfs_print_mod_info ();
@@ -2615,8 +2609,6 @@ static int __init init_btrfs_fs(void) unregister_ioctl:btrfs_interface_exit (); -free_end_io_wq: -btrfs_end_io_wq_exit (); free_prelim_ref:btrfs_prelim_ref_exit (); free_delayed_ref:
@@ -2654,7 +2646,6 @@ static void __exit exit_btrfs_fs(void)extent_state_cache_exit ();extent_io_exit ();btrfs_interface_exit (); -btrfs_end_io_wq_exit ();unregister_filesystem (&btrfs_fs_type);btrfs_exit_sysfs ();btrfs_cleanup_fs_uuids ();
fs/btrfs/volumes.c +39 -5
71 lines not on this page · raw message
fs/btrfs/volumes.h +11 -0
@@ -315,6 +315,14 @@ struct btrfs_fs_devices { - 2 * sizeof(struct btrfs_chunk)) \ / sizeof(struct btrfs_stripe) + 1) +enum btrfs_endio_type { +BTRFS_ENDIO_NONE =0 , +BTRFS_ENDIO_WQ_DATA_READ , +BTRFS_ENDIO_WQ_DATA_WRITE , +BTRFS_ENDIO_WQ_METADATA_READ , +BTRFS_ENDIO_WQ_FREE_SPACE_READ , +}; +/* * Additional info to pass along bio. *
@@ -324,6 +332,9 @@ struct btrfs_bio {struct inode *inode;unsigned int mirror_num; + +enum btrfs_endio_type end_io_type; +struct btrfs_work work;/* for direct I/O */ u64 file_offset;
Signature · 1 line
2.30.2
Re: [PATCH 24/40] btrfs: remove btrfs_end_io_wq
Quoted 4 lines · Christoph Hellwig
Avoid the extra allocation for all read bios by embedding a
btrfs_workand I/O end type into thebtrfs_biostructure.Signed-off-by Christoph Hellwig <[email protected]>
Do we really need to bump the size of btrfs_bio furthermore?
Especially btrfs_bio is allocated for each 64K stripe...
Thanks,
Qu
Quoted 299 lines
--- fs/btrfs/compression.c | 24 +++------ fs/btrfs/ctree.h | 1 - fs/btrfs/disk-io.c | 112 +---------------------------------------- fs/btrfs/disk-io.h | 10 ---- fs/btrfs/inode.c | 19 +++---- fs/btrfs/super.c | 11 +--- fs/btrfs/volumes.c | 44 ++++++++++++++-- fs/btrfs/volumes.h | 11 ++++ 8 files changed, 66 insertions(+), 166 deletions(-)fs/btrfs/compression.c +8
-16@@ -423,20 +423,6 @@ static void end_compressed_bio_write(struct bio *bio)bio_put (bio); } -static blk_status_t submit_compressed_bio (struct btrfs_fs_info *fs_info, -struct compressed_bio *cb, -struct bio *bio,int mirror_num) -{ -blk_status_t ret; - -ASSERT (bio->bi_iter.bi_size); - ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA ); -if (ret) -return ret; - ret =btrfs_map_bio (fs_info, bio, mirror_num); -return ret; -} -/* * Allocate a compressed_bio, which will be used to read/write on-disk * (aka, compressed) * data. @@ -468,6 +454,10 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_byte bio->bi_iter.bi_sector = disk_bytenr >>SECTOR_SHIFT ; bio->bi_private = cb; bio->bi_end_io = endio_func; +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_WRITE ; +else +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ; em =btrfs_get_chunk_map (fs_info, disk_bytenr, fs_info->sectorsize);if (IS_ERR (em)) {@@ -594,7 +584,8 @@ blk_status_t btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start, goto finish_cb; } - ret =submit_compressed_bio (fs_info, cb, bio,0 ); +ASSERT (bio->bi_iter.bi_size); + ret =btrfs_map_bio (fs_info, bio,0 );if (ret) goto finish_cb; bio =NULL ;@@ -930,7 +921,8 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, fs_info->sectorsize); sums += fs_info->csum_size * nr_sectors; - ret =submit_compressed_bio (fs_info, cb, comp_bio, mirror_num); +ASSERT (comp_bio->bi_iter.bi_size); + ret =btrfs_map_bio (fs_info, comp_bio, mirror_num);if (ret) goto finish_cb; comp_bio =NULL ;fs/btrfs/ctree.h +0
-1@@ -823,7 +823,6 @@ struct btrfs_fs_info {struct btrfs_workqueue *endio_meta_workers;struct btrfs_workqueue *endio_raid56_workers;struct btrfs_workqueue *rmw_workers; -struct btrfs_workqueue *endio_meta_write_workers;struct btrfs_workqueue *endio_write_workers;struct btrfs_workqueue *endio_freespace_worker;struct btrfs_workqueue *caching_workers;fs/btrfs/disk-io.c +1
-111@@ -51,7 +51,6 @@ BTRFS_SUPER_FLAG_METADUMP |\ BTRFS_SUPER_FLAG_METADUMP_V2) -static void end_workqueue_fn (struct btrfs_work *work);static void btrfs_destroy_ordered_extents (struct btrfs_root *root);static int btrfs_destroy_delayed_refs (struct btrfs_transaction *trans,struct btrfs_fs_info *fs_info);@@ -64,40 +63,6 @@ static int btrfs_destroy_pinned_extent(struct btrfs_fs_info *fs_info,static int btrfs_cleanup_transaction (struct btrfs_fs_info *fs_info);static void btrfs_error_commit_super (struct btrfs_fs_info *fs_info); -/* -* btrfs_end_io_wq structs are used to do processing in task context when an IO -* is complete. This is used during reads to verify checksums, and it is used -* by writes to insert metadata for new file extents after IO is complete. -*/ -struct btrfs_end_io_wq { -struct bio *bio; -bio_end_io_t *end_io; -void *private; -struct btrfs_fs_info *info; -blk_status_t status; -enum btrfs_wq_endio_type metadata; -struct btrfs_work work; -}; - -static struct kmem_cache *btrfs_end_io_wq_cache; - -int __init btrfs_end_io_wq_init (void ) -{ - btrfs_end_io_wq_cache =kmem_cache_create ("btrfs_end_io_wq" , -sizeof (struct btrfs_end_io_wq ), -0 , -SLAB_MEM_SPREAD , -NULL ); -if (!btrfs_end_io_wq_cache) -return -ENOMEM ; -return 0 ; -} - -void __cold btrfs_end_io_wq_exit (void ) -{ -kmem_cache_destroy (btrfs_end_io_wq_cache); -} -static void btrfs_free_csum_hash (struct btrfs_fs_info *fs_info) {if (fs_info->csum_shash)@@ -726,54 +691,6 @@ int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio,return ret; } -static void end_workqueue_bio (struct bio *bio) -{ -struct btrfs_end_io_wq *end_io_wq = bio->bi_private; -struct btrfs_fs_info *fs_info; -struct btrfs_workqueue *wq; - - fs_info = end_io_wq->info; - end_io_wq->status = bio->bi_status; - -if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { -if (end_io_wq->metadata ==BTRFS_WQ_ENDIO_METADATA ) - wq = fs_info->endio_meta_write_workers; -else if (end_io_wq->metadata ==BTRFS_WQ_ENDIO_FREE_SPACE ) - wq = fs_info->endio_freespace_worker; -else - wq = fs_info->endio_write_workers; - }else { -if (end_io_wq->metadata) - wq = fs_info->endio_meta_workers; -else - wq = fs_info->endio_workers; - } - -btrfs_init_work (&end_io_wq->work, end_workqueue_fn,NULL ,NULL ); -btrfs_queue_work (wq, &end_io_wq->work); -} - -blk_status_t btrfs_bio_wq_end_io (struct btrfs_fs_info *info,struct bio *bio, -enum btrfs_wq_endio_type metadata) -{ -struct btrfs_end_io_wq *end_io_wq; - - end_io_wq =kmem_cache_alloc (btrfs_end_io_wq_cache,GFP_NOFS ); -if (!end_io_wq) -return BLK_STS_RESOURCE ; - - end_io_wq->private = bio->bi_private; - end_io_wq->end_io = bio->bi_end_io; - end_io_wq->info = info; - end_io_wq->status =0 ; - end_io_wq->bio = bio; - end_io_wq->metadata = metadata; - - bio->bi_private = end_io_wq; - bio->bi_end_io = end_workqueue_bio; -return 0 ; -} -static void run_one_async_start (struct btrfs_work *work) {struct async_submit_bio *async;@@ -921,10 +838,7 @@ blk_status_t btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio,return ret; }else {/* checksum validation should happen in async threads: */ - ret =btrfs_bio_wq_end_io (fs_info, bio, -BTRFS_WQ_ENDIO_METADATA ); -if (ret) -return ret; +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_METADATA_READ ; }return btrfs_map_bio (fs_info, bio, mirror_num);@@ -1888,25 +1802,6 @@ struct btrfs_root *btrfs_get_fs_root_commit_root(struct btrfs_fs_info *fs_info,return root; } -/* -* called by the kthread helper functions to finally call the bio end_io -* functions. This is where read checksum verification actually happens -*/ -static void end_workqueue_fn (struct btrfs_work *work) -{ -struct bio *bio; -struct btrfs_end_io_wq *end_io_wq; - - end_io_wq = container_of(work, struct btrfs_end_io_wq, work); - bio = end_io_wq->bio; - - bio->bi_status = end_io_wq->status; - bio->bi_private = end_io_wq->private; - bio->bi_end_io = end_io_wq->end_io; -bio_endio (bio); -kmem_cache_free (btrfs_end_io_wq_cache, end_io_wq); -} -static int cleaner_kthread (void *arg) {struct btrfs_root *root = arg;@@ -2219,7 +2114,6 @@ static void btrfs_stop_all_workers(struct btrfs_fs_info *fs_info)* queues can do metadata I/O operations. */ btrfs_destroy_workqueue (fs_info->endio_meta_workers); -btrfs_destroy_workqueue (fs_info->endio_meta_write_workers); }static void free_root_extent_buffers (struct btrfs_root *root)@@ -2404,9 +2298,6 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info) fs_info->endio_meta_workers =btrfs_alloc_workqueue (fs_info,"endio-meta" , flags, max_active,4 ); - fs_info->endio_meta_write_workers = -btrfs_alloc_workqueue (fs_info,"endio-meta-write" , flags, - max_active,2 ); fs_info->endio_raid56_workers =btrfs_alloc_workqueue (fs_info,"endio-raid56" , flags, max_active,4 );@@ -2429,7 +2320,6 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info) if (!(fs_info->workers && fs_info->delalloc_workers && fs_info->flush_workers && fs_info->endio_workers && fs_info->endio_meta_workers && - fs_info->endio_meta_write_workers && fs_info->endio_write_workers && fs_info->endio_raid56_workers && fs_info->endio_freespace_worker && fs_info->rmw_workers && fs_info->caching_workers && fs_info->fixup_workers &&fs/btrfs/disk-io.h +0
-10@@ -17,12 +17,6 @@*/ #define BTRFS_BDEV_BLOCKSIZE (4096) -enum btrfs_wq_endio_type { -BTRFS_WQ_ENDIO_DATA , -BTRFS_WQ_ENDIO_METADATA , -BTRFS_WQ_ENDIO_FREE_SPACE , -}; -static inline u64 btrfs_sb_offset (int mirror) {u64 start =SZ_16K ;@@ -119,8 +113,6 @@ int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid, int atomic);int btrfs_read_buffer (struct extent_buffer *buf,u64 parent_transid,int level,struct btrfs_key *first_key); -blk_status_t btrfs_bio_wq_end_io (struct btrfs_fs_info *info,struct bio *bio, -enum btrfs_wq_endio_type metadata); blk_status_t btrfs_wq_submit_bio(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 dio_file_offset,@@ -144,8 +136,6 @@ int btree_lock_page_hook(struct page *page, void *data,int btrfs_get_num_tolerated_disk_barrier_failures (u64 flags);int btrfs_get_free_objectid (struct btrfs_root *root,u64 *objectid);int btrfs_init_root_free_objectid (struct btrfs_root *root); -int __init btrfs_end_io_wq_init (void ); -void __cold btrfs_end_io_wq_exit (void );#ifdef CONFIG_DEBUG_LOCK_ALLOC void btrfs_set_buffer_lockdep_class(u64 objectid,fs/btrfs/inode.c +1
-0@@ -2512,6 +2512,7 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);struct btrfs_inode *bi =BTRFS_I (inode); +struct btrfs_bio *bbio =btrfs_bio (bio);blk_status_t ret;if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) {
fs/btrfs/inode.c blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio, · line 2541
Quoted 5 lines
fs/btrfs/inode.c +0
-2@@ -2537,14 +2538,10 @@ blk_status_t btrfs_submit_data_bio(struct inode *inode, struct bio *bio,if (ret)return ret; }else { -enum btrfs_wq_endio_type metadata =BTRFS_WQ_ENDIO_DATA ; -embedding a btrfs_work
and I/O end type
Quoted 187 lines
fs/btrfs/inode.c +5
-11if (btrfs_is_free_space_inode (bi)) - metadata =BTRFS_WQ_ENDIO_FREE_SPACE ; - - ret =btrfs_bio_wq_end_io (fs_info, bio, metadata); -if (ret) -return ret; + bbio->end_io_type =BTRFS_ENDIO_WQ_FREE_SPACE_READ ; +else + bbio->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ;if (bio_flags &EXTENT_BIO_COMPRESSED ) return btrfs_submit_compressed_read(inode, bio,@@ -7739,9 +7736,7 @@ static blk_status_t submit_dio_repair_bio(struct inode *inode, struct bio *bio,BUG_ON (bio_op (bio) ==REQ_OP_WRITE ); - ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA ); -if (ret) -return ret; +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_WRITE ;refcount_inc (&dip->refs); ret =btrfs_map_bio (fs_info, bio, mirror_num);@@ -7865,9 +7860,7 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio,return ret; } }else { - ret =btrfs_bio_wq_end_io (fs_info, bio,BTRFS_WQ_ENDIO_DATA ); -if (ret) -return ret; +btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ;if (!(bi->flags &BTRFS_INODE_NODATASUM )) {u64 csum_offset;fs/btrfs/super.c +1
-10@@ -1835,8 +1835,6 @@ static void btrfs_resize_thread_pool(struct btrfs_fs_info *fs_info,btrfs_workqueue_set_max (fs_info->caching_workers, new_pool_size);btrfs_workqueue_set_max (fs_info->endio_workers, new_pool_size);btrfs_workqueue_set_max (fs_info->endio_meta_workers, new_pool_size); -btrfs_workqueue_set_max (fs_info->endio_meta_write_workers, - new_pool_size);btrfs_workqueue_set_max (fs_info->endio_write_workers, new_pool_size);btrfs_workqueue_set_max (fs_info->endio_freespace_worker, new_pool_size);btrfs_workqueue_set_max (fs_info->delayed_workers, new_pool_size);@@ -2593,13 +2591,9 @@ static int __init init_btrfs_fs(void)if (err) goto free_delayed_ref; - err =btrfs_end_io_wq_init (); -if (err) - goto free_prelim_ref; - err =btrfs_interface_init ();if (err) - goto free_end_io_wq; + goto free_prelim_ref;btrfs_print_mod_info ();@@ -2615,8 +2609,6 @@ static int __init init_btrfs_fs(void) unregister_ioctl:btrfs_interface_exit (); -free_end_io_wq: -btrfs_end_io_wq_exit (); free_prelim_ref:btrfs_prelim_ref_exit (); free_delayed_ref:@@ -2654,7 +2646,6 @@ static void __exit exit_btrfs_fs(void)extent_state_cache_exit ();extent_io_exit ();btrfs_interface_exit (); -btrfs_end_io_wq_exit ();unregister_filesystem (&btrfs_fs_type);btrfs_exit_sysfs ();btrfs_cleanup_fs_uuids ();fs/btrfs/volumes.c +39
-5@@ -6659,11 +6659,38 @@ int btrfs_map_sblock(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,return __btrfs_map_block (fs_info, op, logical, length, bioc_ret,0 ,1 ); } -static inline void btrfs_end_bioc (struct btrfs_io_context *bioc) +static struct btrfs_workqueue *btrfs_end_io_wq (struct btrfs_io_context *bioc) { +struct btrfs_fs_info *fs_info = bioc->fs_info; + +switch (btrfs_bio (bioc->orig_bio)->end_io_type) { +case BTRFS_ENDIO_WQ_DATA_READ : +return fs_info->endio_workers; +case BTRFS_ENDIO_WQ_DATA_WRITE : +return fs_info->endio_write_workers; +case BTRFS_ENDIO_WQ_METADATA_READ : +return fs_info->endio_meta_workers; +case BTRFS_ENDIO_WQ_FREE_SPACE_READ : +return fs_info->endio_freespace_worker; +default : +return NULL ; + } +} + +static void btrfs_end_bio_work (struct btrfs_work *work) +{ + struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, work); + +bio_endio (&bbio->bio); +} + +static void btrfs_end_bioc (struct btrfs_io_context *bioc,bool async) +{ +struct btrfs_workqueue *wq = async ?btrfs_end_io_wq (bioc) :NULL ;struct bio *bio = bioc->orig_bio; +struct btrfs_bio *bbio =btrfs_bio (bio); -btrfs_bio (bio)->mirror_num = bioc->mirror_num; + bbio->mirror_num = bioc->mirror_num; bio->bi_private = bioc->private; bio->bi_end_io = bioc->end_io;@@ -6675,7 +6702,14 @@ static inline void btrfs_end_bioc(struct btrfs_io_context *bioc) bio->bi_status =BLK_STS_IOERR ; else bio->bi_status = BLK_STS_OK; -bio_endio (bio); + +if (wq) { +btrfs_init_work (&bbio->work, btrfs_end_bio_work,NULL ,NULL ); +btrfs_queue_work (wq, &bbio->work); + }else { +bio_endio (bio); + } +btrfs_put_bioc (bioc); }@@ -6707,7 +6741,7 @@ static void btrfs_end_bio(struct bio *bio)btrfs_bio_counter_dec (bioc->fs_info);if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc); +btrfs_end_bioc (bioc, true); } static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,@@ -6805,7 +6839,7 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio, !test_bit(BTRFS_DEV_STATE_WRITEABLE, &dev->dev_state))) {atomic_inc (&bioc->error);if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc); +btrfs_end_bioc (bioc, false);continue ; }fs/btrfs/volumes.h +11
-0@@ -315,6 +315,14 @@ struct btrfs_fs_devices { - 2 * sizeof(struct btrfs_chunk)) \ / sizeof(struct btrfs_stripe) + 1) +enum btrfs_endio_type { +BTRFS_ENDIO_NONE =0 , +BTRFS_ENDIO_WQ_DATA_READ , +BTRFS_ENDIO_WQ_DATA_WRITE , +BTRFS_ENDIO_WQ_METADATA_READ , +BTRFS_ENDIO_WQ_FREE_SPACE_READ , +}; +/* * Additional info to pass along bio. * @@ -324,6 +332,9 @@ struct btrfs_bio {struct inode *inode;unsigned int mirror_num; + +enum btrfs_endio_type end_io_type; +struct btrfs_work work;/* for direct I/O */ u64 file_offset;
Re: [PATCH 24/40] btrfs: remove btrfs_end_io_wq
Quoted 11 lines · Qu Wenruo
Quoted 4 lines · Christoph Hellwig
Avoid the extra allocation for all read bios by embedding a
btrfs_workand I/O end type into thebtrfs_biostructure.Signed-off-by Christoph Hellwig <[email protected]>Do we really need to bump the size of
btrfs_biofurthermore?Especially
btrfs_biois allocated for each 64K stripe...
One of the async submission or completion contexts is allocated for almost every btrfs_bio. So overall this reduceѕ the memory usage (at least together with the rest of the series).
PATCH 25/40 btrfs: remove btrfs_wq_submit_bio
In reply to Christoph HellwigReuse the btrfs_work in struct btrfs_bio for asynchronous submission and remove the extra allocation for async write bios.
--- fs/btrfs/disk-io.c | 122 +++++++++++++-------------------------------- fs/btrfs/disk-io.h | 8 +-- fs/btrfs/inode.c | 42 +++++++++------- 3 files changed, 62 insertions(+), 110 deletions(-)
fs/btrfs/disk-io.c +36 -86
@@ -69,23 +69,6 @@ static void btrfs_free_csum_hash(struct btrfs_fs_info *fs_info)crypto_free_shash (fs_info->csum_shash); } -/* -* async submit bios are used to offload expensive checksumming -* onto the worker threads. They checksum file and metadata bios -* just before they are sent down the IO stack. -*/ -struct async_submit_bio { -struct inode *inode; -struct bio *bio; -extent_submit_bio_start_t *submit_bio_start; -int mirror_num; - -/* Optional parameter for submit_bio_start used by direct io */ -u64 dio_file_offset; -struct btrfs_work work; -blk_status_t status; -}; -/* * Lockdep class keys for extent_buffer->lock's in this root. For a given * eb, the lockdep key is determined by the btrfs_root it belongs to and
@@ -691,18 +674,6 @@ int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio,return ret; } -static void run_one_async_start (struct btrfs_work *work) -{ -struct async_submit_bio *async; -blk_status_t ret; - - async = container_of(work, struct async_submit_bio, work); - ret = async->submit_bio_start (async->inode, async->bio, - async->dio_file_offset); -if (ret) - async->status = ret; -} -/* * In order to insert checksums into the metadata in large chunks, we wait * until bio submission time. All the pages in the bio are checksummed and
@@ -711,72 +682,51 @@ static void run_one_async_start(struct btrfs_work *work)* At IO completion time the csums attached on the ordered extent record are * inserted into the tree. */ -static void run_one_async_done (struct btrfs_work *work) +static void btrfs_submit_bio_work (struct btrfs_work *work) { -struct async_submit_bio *async; -struct inode *inode; + struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, work); +struct btrfs_fs_info *fs_info =btrfs_sb (bbio->inode->i_sb); +struct bio *bio = &bbio->bio;blk_status_t ret; - async = container_of(work, struct async_submit_bio, work); - inode = async->inode; +/* Ensure the bio doesn't go away while linked into the workqueue */ +bio_get (bio);/* If an error occurred we just want to clean up the bio and move on */ -if (async->status) { - async->bio->bi_status = async->status; -bio_endio (async->bio); +if (bio->bi_status) { +bio_endio (bio);return ; }/* -* All of the bios that pass through here are from async helpers. -* Use REQ_CGROUP_PUNT to issue them from the owning cgroup's context. -* This changes nothing when cgroups aren't in use. +* Use REQ_CGROUP_PUNT to issue the bio from the owning cgroup's +* context. This changes nothing when cgroups aren't in use. */ - async->bio->bi_opf |=REQ_CGROUP_PUNT ; - ret =btrfs_map_bio (btrfs_sb (inode->i_sb), async->bio, async->mirror_num); + bio->bi_opf |=REQ_CGROUP_PUNT ; + ret =btrfs_map_bio (fs_info, bio, bbio->mirror_num);if (ret) { - async->bio->bi_status = ret; -bio_endio (async->bio); + bio->bi_status = ret; +bio_endio (bio); } } -static void run_one_async_free (struct btrfs_work *work) +static void btrfs_submit_bio_done (struct btrfs_work *work) { -struct async_submit_bio *async; + struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, work); - async = container_of(work, struct async_submit_bio, work); -kfree (async); +bio_put (&bbio->bio); } -blk_status_t btrfs_wq_submit_bio (struct inode *inode,struct bio *bio, -int mirror_num,unsigned long bio_flags, -u64 dio_file_offset, -extent_submit_bio_start_t *submit_bio_start) +void btrfs_submit_bio_async (struct btrfs_bio *bbio, +void (*start)(struct btrfs_work *work)) { -struct btrfs_fs_info *fs_info =BTRFS_I (inode)->root->fs_info; -struct async_submit_bio *async; +ASSERT (bbio->end_io_type ==BTRFS_ENDIO_NONE ); - async =kmalloc (sizeof (*async),GFP_NOFS ); -if (!async) -return BLK_STS_RESOURCE ; - - async->inode = inode; - async->bio = bio; - async->mirror_num = mirror_num; - async->submit_bio_start = submit_bio_start; - -btrfs_init_work (&async->work, run_one_async_start, run_one_async_done, - run_one_async_free); - - async->dio_file_offset = dio_file_offset; - - async->status =0 ; - -if (op_is_sync (bio->bi_opf)) -btrfs_set_work_high_priority (&async->work); - -btrfs_queue_work (fs_info->workers, &async->work); -return 0 ; +btrfs_init_work (&bbio->work, start, btrfs_submit_bio_work, + btrfs_submit_bio_done); +if (op_is_sync (bbio->bio.bi_opf)) +btrfs_set_work_high_priority (&bbio->work); +btrfs_queue_work (btrfs_sb (bbio->inode->i_sb)->workers, &bbio->work); }static blk_status_t btree_csum_one_bio (struct bio *bio)
@@ -797,14 +747,11 @@ static blk_status_t btree_csum_one_bio(struct bio *bio)return errno_to_blk_status (ret); } -static blk_status_t btree_submit_bio_start (struct inode *inode,struct bio *bio, -u64 dio_file_offset) +static void btree_submit_bio_start (struct btrfs_work *work) { -/* -* when we're called for a write, we're already in the async -* submission context. Just jump into btrfs_map_bio -*/ -return btree_csum_one_bio (bio); + struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, work); + + bbio->bio.bi_status =btree_csum_one_bio (&bbio->bio); }/*
@@ -827,18 +774,21 @@ blk_status_t btrfs_submit_metadata_bio(struct inode *inode, struct bio *bio, int mirror_num) {struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); +struct btrfs_bio *bbio =btrfs_bio (bio);blk_status_t ret;if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { -if (should_async_write (fs_info,BTRFS_I (inode))) -return btrfs_wq_submit_bio (inode, bio, mirror_num,0 ,0 , - btree_submit_bio_start); +if (should_async_write (fs_info,BTRFS_I (inode))) { + bbio->mirror_num = mirror_num; +btrfs_submit_bio_async (bbio, btree_submit_bio_start); +return BLK_STS_OK ; + } ret =btree_csum_one_bio (bio);if (ret)return ret; }else {/* checksum validation should happen in async threads: */ -btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_METADATA_READ ; + bbio->end_io_type =BTRFS_ENDIO_WQ_METADATA_READ ; }return btrfs_map_bio (fs_info, bio, mirror_num);
fs/btrfs/disk-io.h +2 -6
@@ -113,12 +113,8 @@ int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid, int atomic);int btrfs_read_buffer (struct extent_buffer *buf,u64 parent_transid,int level,struct btrfs_key *first_key); -blk_status_t btrfs_wq_submit_bio (struct inode *inode,struct bio *bio, -int mirror_num,unsigned long bio_flags, -u64 dio_file_offset, -extent_submit_bio_start_t *submit_bio_start); -blk_status_t btrfs_submit_bio_done (void *private_data,struct bio *bio, -int mirror_num); +void btrfs_submit_bio_async (struct btrfs_bio *bbio, +void (*start)(struct btrfs_work *work));int btrfs_alloc_log_tree_node (struct btrfs_trans_handle *trans,struct btrfs_root *root); int btrfs_init_log_root_tree(struct btrfs_trans_handle *trans,
fs/btrfs/inode.c +24 -18
82 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 26/40 btrfs: refactor btrfs_map_bio
In reply to Christoph HellwigUse a label for common cleanup, untangle the conditionals for parity RAID and move all per-stripe handling into submit_stripe_bio.
--- fs/btrfs/volumes.c | 88 ++++++++++++++++++++++------------------------ 1 file changed, 42 insertions(+), 46 deletions(-)
fs/btrfs/volumes.c +42 -46
@@ -6744,10 +6744,30 @@ static void btrfs_end_bio(struct bio *bio)btrfs_end_bioc (bioc, true); } -static void submit_stripe_bio (struct btrfs_io_context *bioc,struct bio *bio, -u64 physical,struct btrfs_device *dev) +static void submit_stripe_bio (struct btrfs_io_context *bioc, +struct bio *orig_bio,int dev_nr,bool clone) {struct btrfs_fs_info *fs_info = bioc->fs_info; +struct btrfs_device *dev = bioc->stripes[dev_nr].dev; +u64 physical = bioc->stripes[dev_nr].physical; +struct bio *bio; + +if (!dev || !dev->bdev || +test_bit (BTRFS_DEV_STATE_MISSING , &dev->dev_state) || + (btrfs_op (orig_bio) ==BTRFS_MAP_WRITE && + !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state))) { +atomic_inc (&bioc->error); +if (atomic_dec_and_test (&bioc->stripes_pending)) +btrfs_end_bioc (bioc, false); +return ; + } + +if (clone) { + bio =btrfs_bio_clone (dev->bdev, orig_bio); + }else { + bio = orig_bio; +bio_set_dev (bio, dev->bdev); + } bio->bi_private = bioc;btrfs_bio (bio)->device = dev;
@@ -6782,46 +6802,40 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num) { -struct btrfs_device *dev; -struct bio *first_bio = bio;u64 logical = bio->bi_iter.bi_sector <<9 ; -u64 length =0 ; -u64 map_length; +u64 length = bio->bi_iter.bi_size; +u64 map_length = length;int ret;int dev_nr;int total_devs;struct btrfs_io_context *bioc =NULL ; - length = bio->bi_iter.bi_size; - map_length = length; -btrfs_bio_counter_inc_blocked (fs_info); ret =__btrfs_map_block (fs_info,btrfs_op (bio), logical, &map_length, &bioc, mirror_num,1 ); -if (ret) { -btrfs_bio_counter_dec (fs_info); -return errno_to_blk_status (ret); - } +if (ret) + goto out_dec; total_devs = bioc->num_stripes; - bioc->orig_bio = first_bio; - bioc->private = first_bio->bi_private; - bioc->end_io = first_bio->bi_end_io; + bioc->orig_bio = bio; + bioc->private = bio->bi_private; + bioc->end_io = bio->bi_end_io;atomic_set (&bioc->stripes_pending, bioc->num_stripes); -if ((bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ) && - ((btrfs_op (bio) ==BTRFS_MAP_WRITE ) || (mirror_num >1 ))) { -/* In this case, map_length has been set to the length of -a single stripe; not the whole write */ +if (bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { +/* +* In this case, map_length has been set to the length of a +* single stripe; not the whole write. +*/ if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { ret =raid56_parity_write (bio, bioc, map_length); - }else { + goto out_dec; + } +if (mirror_num >1 ) { ret =raid56_parity_recover (bio, bioc, map_length, mirror_num,1 ); + goto out_dec; } - -btrfs_bio_counter_dec (fs_info); -return errno_to_blk_status (ret); }if (map_length < length) {
@@ -6831,29 +6845,11 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,BUG (); } -for (dev_nr =0 ; dev_nr < total_devs; dev_nr++) { - dev = bioc->stripes[dev_nr].dev; -if (!dev || !dev->bdev ||test_bit (BTRFS_DEV_STATE_MISSING , - &dev->dev_state) || - (btrfs_op (first_bio) ==BTRFS_MAP_WRITE && - !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state))) { -atomic_inc (&bioc->error); -if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc, false); -continue ; - } - -if (dev_nr < total_devs -1 ) { - bio =btrfs_bio_clone (dev->bdev, first_bio); - }else { - bio = first_bio; -bio_set_dev (bio, dev->bdev); - } - -submit_stripe_bio (bioc, bio, bioc->stripes[dev_nr].physical, dev); - } +for (dev_nr =0 ; dev_nr < total_devs; dev_nr++) +submit_stripe_bio (bioc, bio, dev_nr, dev_nr < total_devs -1 ); +out_dec:btrfs_bio_counter_dec (fs_info); -return BLK_STS_OK ; +return errno_to_blk_status (ret); } static bool dev_args_match_fs_devices(const struct btrfs_dev_lookup_args *args,
Signature · 1 line
2.30.2
Re: [PATCH 26/40] btrfs: refactor btrfs_map_bio
Quoted 95 lines · Christoph Hellwig
Use a label for common cleanup, untangle the conditionals for parity RAID and move all per-stripe handling into
submit_stripe_bio.Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/volumes.c | 88 ++++++++++++++++++++++------------------------ 1 file changed, 42 insertions(+), 46 deletions(-)fs/btrfs/volumes.c +34
-21@@ -6744,10 +6744,30 @@ static void btrfs_end_bio(struct bio *bio)btrfs_end_bioc (bioc, true); } -static void submit_stripe_bio (struct btrfs_io_context *bioc,struct bio *bio, -u64 physical,struct btrfs_device *dev) +static void submit_stripe_bio (struct btrfs_io_context *bioc, +struct bio *orig_bio,int dev_nr,bool clone) {struct btrfs_fs_info *fs_info = bioc->fs_info; +struct btrfs_device *dev = bioc->stripes[dev_nr].dev; +u64 physical = bioc->stripes[dev_nr].physical; +struct bio *bio; + +if (!dev || !dev->bdev || +test_bit (BTRFS_DEV_STATE_MISSING , &dev->dev_state) || + (btrfs_op (orig_bio) ==BTRFS_MAP_WRITE && + !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state))) { +atomic_inc (&bioc->error); +if (atomic_dec_and_test (&bioc->stripes_pending)) +btrfs_end_bioc (bioc, false); +return ; + } + +if (clone) { + bio =btrfs_bio_clone (dev->bdev, orig_bio); + }else { + bio = orig_bio; +bio_set_dev (bio, dev->bdev); + } bio->bi_private = bioc;btrfs_bio (bio)->device = dev;@@ -6782,46 +6802,40 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio,blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num) { -struct btrfs_device *dev; -struct bio *first_bio = bio;u64 logical = bio->bi_iter.bi_sector <<9 ; -u64 length =0 ; -u64 map_length; +u64 length = bio->bi_iter.bi_size; +u64 map_length = length;int ret;int dev_nr;int total_devs;struct btrfs_io_context *bioc =NULL ; - length = bio->bi_iter.bi_size; - map_length = length; -btrfs_bio_counter_inc_blocked (fs_info); ret =__btrfs_map_block (fs_info,btrfs_op (bio), logical, &map_length, &bioc, mirror_num,1 ); -if (ret) { -btrfs_bio_counter_dec (fs_info); -return errno_to_blk_status (ret); - } +if (ret) + goto out_dec; total_devs = bioc->num_stripes; - bioc->orig_bio = first_bio; - bioc->private = first_bio->bi_private; - bioc->end_io = first_bio->bi_end_io; + bioc->orig_bio = bio; + bioc->private = bio->bi_private; + bioc->end_io = bio->bi_end_io;atomic_set (&bioc->stripes_pending, bioc->num_stripes); -if ((bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ) && - ((btrfs_op (bio) ==BTRFS_MAP_WRITE ) || (mirror_num >1 ))) { -/* In this case, map_length has been set to the length of -a single stripe; not the whole write */ +if (bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { +/* +* In this case, map_length has been set to the length of a +* single stripe; not the whole write. +*/ if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { ret =raid56_parity_write (bio, bioc, map_length); - }else {
fs/btrfs/volumes.c static void submit_stripe_bio(struct btrfs_io_context *bioc, struct bio *bio, · line 6824
Quoted 10 lines
fs/btrfs/volumes.c +4
-3+ goto out_dec; + } +if (mirror_num >1 ) { ret =raid56_parity_recover (bio, bioc, map_length, mirror_num,1 ); + goto out_dec; } - -btrfs_bio_counter_dec (fs_info); -return errno_to_blk_status (ret);
Can we add some extra comment on the fall through cases?
The read path still needs to go through the regular routine.
Otherwise looks good to me.
Thanks,
Qu
Quoted 37 lines
fs/btrfs/volumes.c +4
-22}if (map_length < length) {@@ -6831,29 +6845,11 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,BUG (); } -for (dev_nr =0 ; dev_nr < total_devs; dev_nr++) { - dev = bioc->stripes[dev_nr].dev; -if (!dev || !dev->bdev ||test_bit (BTRFS_DEV_STATE_MISSING , - &dev->dev_state) || - (btrfs_op (first_bio) ==BTRFS_MAP_WRITE && - !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state))) { -atomic_inc (&bioc->error); -if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc, false); -continue ; - } - -if (dev_nr < total_devs -1 ) { - bio =btrfs_bio_clone (dev->bdev, first_bio); - }else { - bio = first_bio; -bio_set_dev (bio, dev->bdev); - } - -submit_stripe_bio (bioc, bio, bioc->stripes[dev_nr].physical, dev); - } +for (dev_nr =0 ; dev_nr < total_devs; dev_nr++) +submit_stripe_bio (bioc, bio, dev_nr, dev_nr < total_devs -1 ); +out_dec:btrfs_bio_counter_dec (fs_info); -return BLK_STS_OK ; +return errno_to_blk_status (ret); } static bool dev_args_match_fs_devices(const struct btrfs_dev_lookup_args *args,
PATCH 27/40 btrfs: clean up the raid map handling __btrfs_map_block
In reply to Christoph HellwigClear need_raid_map early instead of repeating the same conditional over and over.
--- fs/btrfs/volumes.c | 60 ++++++++++++++++++++++------------------------ 1 file changed, 29 insertions(+), 31 deletions(-)
fs/btrfs/volumes.c +29 -31
@@ -6435,6 +6435,10 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, map = em->map_lookup; +if (!(map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) || + (!need_full_stripe (op) && mirror_num <=1 )) + need_raid_map =0 ; + *length = geom.len; stripe_len = geom.stripe_len; stripe_nr = geom.stripe_nr;
@@ -6509,37 +6513,32 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, dev_replace_is_ongoing); mirror_num = stripe_index - old_stripe_index +1 ; } + }else if (need_raid_map) { +/* push stripe_nr back to the start of the full stripe */ + stripe_nr =div64_u64 (raid56_full_stripe_start, + stripe_len * data_stripes); - }else if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { -if (need_raid_map && (need_full_stripe (op) || mirror_num >1 )) { -/* push stripe_nr back to the start of the full stripe */ - stripe_nr =div64_u64 (raid56_full_stripe_start, - stripe_len * data_stripes); - -/* RAID[56] write or recovery. Return all stripes */ - num_stripes = map->num_stripes; - max_errors =nr_parity_stripes (map); - - *length = map->stripe_len; - stripe_index =0 ; - stripe_offset =0 ; - }else { -/* -* Mirror #0 or #1 means the original data block. -* Mirror #2 is RAID5 parity block. -* Mirror #3 is RAID6 Q block. -*/ - stripe_nr =div_u64_rem (stripe_nr, - data_stripes, &stripe_index); -if (mirror_num >1 ) - stripe_index = data_stripes + mirror_num -2 ; +/* RAID[56] write or recovery. Return all stripes */ + num_stripes = map->num_stripes; + max_errors =nr_parity_stripes (map); -/* We distribute the parity blocks across stripes */ -div_u64_rem (stripe_nr + stripe_index, map->num_stripes, - &stripe_index); -if (!need_full_stripe (op) && mirror_num <=1 ) - mirror_num =1 ; - } + *length = map->stripe_len; + stripe_index =0 ; + stripe_offset =0 ; + }else if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { +/* +* Mirror #0 or #1 means the original data block. +* Mirror #2 is RAID5 parity block. +* Mirror #3 is RAID6 Q block. +*/ + stripe_nr =div_u64_rem (stripe_nr, data_stripes, &stripe_index); +if (mirror_num >1 ) + stripe_index = data_stripes + mirror_num -2 ; +/* We distribute the parity blocks across stripes */ +div_u64_rem (stripe_nr + stripe_index, map->num_stripes, + &stripe_index); +if (!need_full_stripe (op) && mirror_num <=1 ) + mirror_num =1 ; }else {/* * after this, stripe_nr is the number of stripes on this
@@ -6581,8 +6580,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, }/* Build raid_map */ -if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK && need_raid_map && - (need_full_stripe (op) || mirror_num >1 )) { +if (need_raid_map) {u64 tmp;unsigned rot;
Signature · 1 line
2.30.2
Re: [PATCH 27/40] btrfs: clean up the raid map handling __btrfs_map_block
Quoted 2 lines · Christoph Hellwig
Clear
need_raid_mapearly instead of repeating the same conditional over and over.
I had a more comprehensive cleanup, but only for scrub:
https://lore.kernel.org/linux-btrfs/[email protected]/
All profiles are split into 3 categories:
- Simple mirror
Single, DUP, RAID1*.
- Simple stripe
RAID0 and RAID10
Inside each data stripe, it's just simple mirror then.
- RAID56
And for mirror_num == 0/1 cases, inside one data stripe it's still simple mirror.
Maybe we can follow the same ideas here?
Thanks,
Qu
Quoted 93 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/volumes.c | 60 ++++++++++++++++++++++------------------------ 1 file changed, 29 insertions(+), 31 deletions(-)fs/btrfs/volumes.c +29
-31@@ -6435,6 +6435,10 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, map = em->map_lookup; +if (!(map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) || + (!need_full_stripe (op) && mirror_num <=1 )) + need_raid_map =0 ; + *length = geom.len; stripe_len = geom.stripe_len; stripe_nr = geom.stripe_nr;@@ -6509,37 +6513,32 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, dev_replace_is_ongoing); mirror_num = stripe_index - old_stripe_index +1 ; } + }else if (need_raid_map) { +/* push stripe_nr back to the start of the full stripe */ + stripe_nr =div64_u64 (raid56_full_stripe_start, + stripe_len * data_stripes); - }else if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { -if (need_raid_map && (need_full_stripe (op) || mirror_num >1 )) { -/* push stripe_nr back to the start of the full stripe */ - stripe_nr =div64_u64 (raid56_full_stripe_start, - stripe_len * data_stripes); - -/* RAID[56] write or recovery. Return all stripes */ - num_stripes = map->num_stripes; - max_errors =nr_parity_stripes (map); - - *length = map->stripe_len; - stripe_index =0 ; - stripe_offset =0 ; - }else { -/* -* Mirror #0 or #1 means the original data block. -* Mirror #2 is RAID5 parity block. -* Mirror #3 is RAID6 Q block. -*/ - stripe_nr =div_u64_rem (stripe_nr, - data_stripes, &stripe_index); -if (mirror_num >1 ) - stripe_index = data_stripes + mirror_num -2 ; +/* RAID[56] write or recovery. Return all stripes */ + num_stripes = map->num_stripes; + max_errors =nr_parity_stripes (map); -/* We distribute the parity blocks across stripes */ -div_u64_rem (stripe_nr + stripe_index, map->num_stripes, - &stripe_index); -if (!need_full_stripe (op) && mirror_num <=1 ) - mirror_num =1 ; - } + *length = map->stripe_len; + stripe_index =0 ; + stripe_offset =0 ; + }else if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK ) { +/* +* Mirror #0 or #1 means the original data block. +* Mirror #2 is RAID5 parity block. +* Mirror #3 is RAID6 Q block. +*/ + stripe_nr =div_u64_rem (stripe_nr, data_stripes, &stripe_index); +if (mirror_num >1 ) + stripe_index = data_stripes + mirror_num -2 ; +/* We distribute the parity blocks across stripes */ +div_u64_rem (stripe_nr + stripe_index, map->num_stripes, + &stripe_index); +if (!need_full_stripe (op) && mirror_num <=1 ) + mirror_num =1 ; }else {/* * after this, stripe_nr is the number of stripes on this @@ -6581,8 +6580,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, }/* Build raid_map */ -if (map->type &BTRFS_BLOCK_GROUP_RAID56_MASK && need_raid_map && - (need_full_stripe (op) || mirror_num >1 )) { +if (need_raid_map) {u64 tmp;unsigned rot;
Re: [PATCH 27/40] btrfs: clean up the raid map handling __btrfs_map_block
Quoted 8 lines · Qu Wenruo
Quoted 2 lines · Christoph Hellwig
Clear
need_raid_mapearly instead of repeating the same conditional over and over.I had a more comprehensive cleanup, but only for scrub:
https://lore.kernel.org/linux-btrfs/[email protected]/
I'll take a look. I mostly wanted to avoid checking the same conditional yet again for fast path added later in this series.
PATCH 28/40 btrfs: do not allocate a btrfs_io_context in btrfs_map_bio
In reply to Christoph HellwigThere is very little of the I/O context that is actually needed for issuing a bio. Add the few needed fields to struct btrfs_bio instead.
The stripes array is still allocated on demand when more than a single I/O is needed, but for single leg I/O (e.g. all reads) there is no additional memory allocation now.
--- fs/btrfs/volumes.c | 147 ++++++++++++++++++++++++++++----------------- fs/btrfs/volumes.h | 20 ++++-- 2 files changed, 107 insertions(+), 60 deletions(-)
fs/btrfs/volumes.c +92 -55
@@ -253,10 +253,9 @@ static int btrfs_relocate_sys_chunks(struct btrfs_fs_info *fs_info);static void btrfs_dev_stat_print_on_error (struct btrfs_device *dev);static void btrfs_dev_stat_print_on_load (struct btrfs_device *device);static int __btrfs_map_block (struct btrfs_fs_info *fs_info, -enum btrfs_map_op op, -u64 logical,u64 *length, -struct btrfs_io_context **bioc_ret, -int mirror_num,int need_raid_map); +enum btrfs_map_op op,u64 logical,u64 *length, +struct btrfs_io_context **bioc_ret,struct btrfs_bio *bbio, +int mirror_num,int need_raid_map);/* * Device locking
@@ -5926,7 +5925,6 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_sizeof (u64) * (total_stripes), GFP_NOFS|__GFP_NOFAIL); -atomic_set (&bioc->error,0 );refcount_set (&bioc->refs,1 ); bioc->fs_info = fs_info;
@@ -6128,7 +6126,7 @@ static int get_extra_mirror_from_replace(struct btrfs_fs_info *fs_info,int ret =0 ; ret =__btrfs_map_block (fs_info,BTRFS_MAP_GET_READ_MIRRORS , - logical, &length, &bioc,0 ,0 ); + logical, &length, &bioc,NULL ,0 ,0 );if (ret) {ASSERT (bioc ==NULL );return ret;
@@ -6397,10 +6395,9 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em, }static int __btrfs_map_block (struct btrfs_fs_info *fs_info, -enum btrfs_map_op op, -u64 logical,u64 *length, -struct btrfs_io_context **bioc_ret, -int mirror_num,int need_raid_map) +enum btrfs_map_op op,u64 logical,u64 *length, +struct btrfs_io_context **bioc_ret,struct btrfs_bio *bbio, +int mirror_num,int need_raid_map) {struct extent_map *em;struct map_lookup *map;
@@ -6566,6 +6563,48 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, tgtdev_indexes = num_stripes; } +if (need_full_stripe (op)) + max_errors =btrfs_chunk_max_errors (map); + +if (bbio && !need_raid_map) { +int replacement_idx = num_stripes; + +if (num_alloc_stripes >1 ) { + bbio->stripes =kmalloc_array (num_alloc_stripes, +sizeof (*bbio->stripes), +GFP_NOFS | __GFP_NOFAIL); + }else { + bbio->stripes = &bbio->__stripe; + } + +atomic_set (&bbio->stripes_pending, num_stripes); +for (i =0 ; i < num_stripes; i++) { +struct btrfs_bio_stripe *s = &bbio->stripes[i]; + + s->physical = map->stripes[stripe_index].physical + + stripe_offset + stripe_nr * map->stripe_len; + s->dev = map->stripes[stripe_index].dev; + stripe_index++; + +if (op ==BTRFS_MAP_WRITE && dev_replace_is_ongoing && + dev_replace->tgtdev && + !is_block_group_to_copy (fs_info, logical) && + s->dev->devid == dev_replace->srcdev->devid) { +struct btrfs_bio_stripe *r = + &bbio->stripes[replacement_idx++]; + + r->physical = s->physical; + r->dev = dev_replace->tgtdev; + max_errors++; +atomic_inc (&bbio->stripes_pending); + } + } + + bbio->max_errors = max_errors; + bbio->mirror_num = mirror_num; + goto out; + } + bioc =alloc_btrfs_io_context (fs_info, num_alloc_stripes, tgtdev_indexes);if (!bioc) { ret = -ENOMEM ;
@@ -6601,9 +6640,6 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info,sort_parity_stripes (bioc, num_stripes); } -if (need_full_stripe (op)) - max_errors =btrfs_chunk_max_errors (map); -if (dev_replace_is_ongoing && dev_replace->tgtdev !=NULL &&need_full_stripe (op)) { handle_ops_on_dev_replace(op, &bioc, dev_replace, logical,
@@ -6646,7 +6682,7 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, length, bioc_ret);return __btrfs_map_block (fs_info, op, logical, length, bioc_ret, - mirror_num,0 ); +NULL , mirror_num,0 ); }/* For Scrub/replace */
@@ -6654,14 +6690,15 @@ int btrfs_map_sblock(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, u64 logical, u64 *length, struct btrfs_io_context **bioc_ret) { - return __btrfs_map_block(fs_info, op, logical, length, bioc_ret, 0, 1); + return __btrfs_map_block(fs_info, op, logical, length, bioc_ret, NULL, + 0, 1); } -static struct btrfs_workqueue *btrfs_end_io_wq (struct btrfs_io_context *bioc) +static struct btrfs_workqueue *btrfs_end_io_wq (struct btrfs_bio *bbio) { -struct btrfs_fs_info *fs_info = bioc->fs_info; +struct btrfs_fs_info *fs_info =btrfs_sb (bbio->inode->i_sb); -switch (btrfs_bio (bioc->orig_bio)->end_io_type) { +switch (bbio->end_io_type) {case BTRFS_ENDIO_WQ_DATA_READ :return fs_info->endio_workers;case BTRFS_ENDIO_WQ_DATA_WRITE :
@@ -6682,21 +6719,22 @@ static void btrfs_end_bio_work(struct btrfs_work *work)bio_endio (&bbio->bio); } -static void btrfs_end_bioc (struct btrfs_io_context *bioc,bool async) +static void btrfs_end_bbio (struct btrfs_bio *bbio,bool async) { -struct btrfs_workqueue *wq = async ?btrfs_end_io_wq (bioc) :NULL ; -struct bio *bio = bioc->orig_bio; -struct btrfs_bio *bbio =btrfs_bio (bio); +struct btrfs_workqueue *wq = async ?btrfs_end_io_wq (bbio) :NULL ; +struct bio *bio = &bbio->bio; - bbio->mirror_num = bioc->mirror_num; - bio->bi_private = bioc->private; - bio->bi_end_io = bioc->end_io; + bio->bi_private = bbio->private; + bio->bi_end_io = bbio->end_io; + +if (bbio->stripes != &bbio->__stripe) +kfree (bbio->stripes);/* * Only send an error to the higher layers if it is beyond the tolerance * threshold. */ -if (atomic_read (&bioc->error) > bioc->max_errors) +if (atomic_read (&bbio->error) > bbio->max_errors) bio->bi_status =BLK_STS_IOERR ;else bio->bi_status =BLK_STS_OK ;
@@ -6707,16 +6745,14 @@ static void btrfs_end_bioc(struct btrfs_io_context *bioc, bool async) }else {bio_endio (bio); } - -btrfs_put_bioc (bioc); }static void btrfs_end_bio (struct bio *bio) { -struct btrfs_io_context *bioc = bio->bi_private; +struct btrfs_bio *bbio = bio->bi_private;if (bio->bi_status) { -atomic_inc (&bioc->error); +atomic_inc (&bbio->error);if (bio->bi_status ==BLK_STS_IOERR || bio->bi_status ==BLK_STS_TARGET ) {struct btrfs_device *dev =btrfs_bio (bio)->device;
@@ -6734,40 +6770,39 @@ static void btrfs_end_bio(struct bio *bio) } } -if (bio != bioc->orig_bio) +if (bio != &bbio->bio)bio_put (bio); -btrfs_bio_counter_dec (bioc->fs_info); -if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc, true); +btrfs_bio_counter_dec (btrfs_sb (bbio->inode->i_sb)); +if (atomic_dec_and_test (&bbio->stripes_pending)) +btrfs_end_bbio (bbio, true); } -static void submit_stripe_bio (struct btrfs_io_context *bioc, -struct bio *orig_bio,int dev_nr,bool clone) +static void submit_stripe_bio (struct btrfs_bio *bbio,int dev_nr,bool clone) { -struct btrfs_fs_info *fs_info = bioc->fs_info; -struct btrfs_device *dev = bioc->stripes[dev_nr].dev; -u64 physical = bioc->stripes[dev_nr].physical; +struct btrfs_fs_info *fs_info =btrfs_sb (bbio->inode->i_sb); +struct btrfs_device *dev = bbio->stripes[dev_nr].dev; +u64 physical = bbio->stripes[dev_nr].physical;struct bio *bio;if (!dev || !dev->bdev ||test_bit (BTRFS_DEV_STATE_MISSING , &dev->dev_state) || - (btrfs_op (orig_bio) ==BTRFS_MAP_WRITE && + (btrfs_op (&bbio->bio) ==BTRFS_MAP_WRITE && !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state))) { -atomic_inc (&bioc->error); -if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc, false); +atomic_inc (&bbio->error); +if (atomic_dec_and_test (&bbio->stripes_pending)) +btrfs_end_bbio (bbio, false);return ; }if (clone) { - bio =btrfs_bio_clone (dev->bdev, orig_bio); + bio =btrfs_bio_clone (dev->bdev, &bbio->bio); }else { - bio = orig_bio; + bio = &bbio->bio;bio_set_dev (bio, dev->bdev); } - bio->bi_private = bioc; + bio->bi_private = bbio;btrfs_bio (bio)->device = dev; bio->bi_end_io = btrfs_end_bio; bio->bi_iter.bi_sector = physical >>9 ;
@@ -6800,6 +6835,7 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc,blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num) { +struct btrfs_bio *bbio =btrfs_bio (bio);u64 logical = bio->bi_iter.bi_sector <<9 ;u64 length = bio->bi_iter.bi_size;u64 map_length = length;
@@ -6809,18 +6845,17 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,struct btrfs_io_context *bioc =NULL ;btrfs_bio_counter_inc_blocked (fs_info); - ret =__btrfs_map_block (fs_info,btrfs_op (bio), logical, - &map_length, &bioc, mirror_num,1 ); + ret =__btrfs_map_block (fs_info,btrfs_op (bio), logical, &map_length, + &bioc, bbio, mirror_num,1 );if (ret) goto out_dec; - total_devs = bioc->num_stripes; - bioc->orig_bio = bio; - bioc->private = bio->bi_private; - bioc->end_io = bio->bi_end_io; -atomic_set (&bioc->stripes_pending, bioc->num_stripes); + bbio->private = bio->bi_private; + bbio->end_io = bio->bi_end_io; + +if (bioc) { +ASSERT (bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ); -if (bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ) {/* * In this case, map_length has been set to the length of a * single stripe; not the whole write.
@@ -6834,6 +6869,7 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio, mirror_num, 1); goto out_dec; } +ASSERT (0 ); }if (map_length < length) {
@@ -6843,8 +6879,9 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,BUG (); } + total_devs =atomic_read (&bbio->stripes_pending);for (dev_nr =0 ; dev_nr < total_devs; dev_nr++) -submit_stripe_bio (bioc, bio, dev_nr, dev_nr < total_devs -1 ); +submit_stripe_bio (bbio, dev_nr, dev_nr < total_devs -1 ); out_dec:btrfs_bio_counter_dec (fs_info);return errno_to_blk_status (ret);
fs/btrfs/volumes.h +15 -5
@@ -323,6 +323,11 @@ enum btrfs_endio_type { BTRFS_ENDIO_WQ_FREE_SPACE_READ, }; +struct btrfs_bio_stripe { +struct btrfs_device *dev; +u64 physical; +}; +/* * Additional info to pass along bio. *
@@ -333,6 +338,16 @@ struct btrfs_bio {unsigned int mirror_num; +atomic_t stripes_pending; +atomic_t error; +int max_errors; + +struct btrfs_bio_stripe *stripes; +struct btrfs_bio_stripe __stripe; + +bio_end_io_t *end_io; +void *private; +enum btrfs_endio_type end_io_type;struct btrfs_work work;
@@ -389,13 +404,8 @@ struct btrfs_io_stripe {*/ struct btrfs_io_context {refcount_t refs; -atomic_t stripes_pending;struct btrfs_fs_info *fs_info;u64 map_type;/* get from map_lookup->type */ -bio_end_io_t *end_io; -struct bio *orig_bio; -void *private; -atomic_t error;int max_errors;int num_stripes;int mirror_num;
Signature · 1 line
2.30.2
Re: [PATCH 28/40] btrfs: do not allocate a btrfs_io_context in btrfs_map_bio
Quoted 8 lines · Christoph Hellwig
There is very little of the I/O context that is actually needed for issuing a bio. Add the few needed fields to struct
btrfs_bioinstead.The stripes array is still allocated on demand when more than a single I/O is needed, but for single leg I/O (e.g. all reads) there is no additional memory allocation now.
Signed-off-by Christoph Hellwig <[email protected]>
Really not a fan of enlarging btrfs_bio again and again.
Especially for the btrfs_bio_stripe and btrfs_bio_stripe * part.
Considering how many bytes we waste for SINGLE profile, now we need an extra pointer which we will never really use.
Thanks,
Qu
Quoted 348 lines
--- fs/btrfs/volumes.c | 147 ++++++++++++++++++++++++++++----------------- fs/btrfs/volumes.h | 20 ++++-- 2 files changed, 107 insertions(+), 60 deletions(-)fs/btrfs/volumes.c +92
-55@@ -253,10 +253,9 @@ static int btrfs_relocate_sys_chunks(struct btrfs_fs_info *fs_info);static void btrfs_dev_stat_print_on_error (struct btrfs_device *dev);static void btrfs_dev_stat_print_on_load (struct btrfs_device *device);static int __btrfs_map_block (struct btrfs_fs_info *fs_info, -enum btrfs_map_op op, -u64 logical,u64 *length, -struct btrfs_io_context **bioc_ret, -int mirror_num,int need_raid_map); +enum btrfs_map_op op,u64 logical,u64 *length, +struct btrfs_io_context **bioc_ret,struct btrfs_bio *bbio, +int mirror_num,int need_raid_map);/* * Device locking @@ -5926,7 +5925,6 @@ static struct btrfs_io_context *alloc_btrfs_io_context(struct btrfs_fs_info *fs_sizeof (u64) * (total_stripes), GFP_NOFS|__GFP_NOFAIL); -atomic_set (&bioc->error,0 );refcount_set (&bioc->refs,1 ); bioc->fs_info = fs_info;@@ -6128,7 +6126,7 @@ static int get_extra_mirror_from_replace(struct btrfs_fs_info *fs_info,int ret =0 ; ret =__btrfs_map_block (fs_info,BTRFS_MAP_GET_READ_MIRRORS , - logical, &length, &bioc,0 ,0 ); + logical, &length, &bioc,NULL ,0 ,0 );if (ret) {ASSERT (bioc ==NULL );return ret;@@ -6397,10 +6395,9 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em, }static int __btrfs_map_block (struct btrfs_fs_info *fs_info, -enum btrfs_map_op op, -u64 logical,u64 *length, -struct btrfs_io_context **bioc_ret, -int mirror_num,int need_raid_map) +enum btrfs_map_op op,u64 logical,u64 *length, +struct btrfs_io_context **bioc_ret,struct btrfs_bio *bbio, +int mirror_num,int need_raid_map) {struct extent_map *em;struct map_lookup *map;@@ -6566,6 +6563,48 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, tgtdev_indexes = num_stripes; } +if (need_full_stripe (op)) + max_errors =btrfs_chunk_max_errors (map); + +if (bbio && !need_raid_map) { +int replacement_idx = num_stripes; + +if (num_alloc_stripes >1 ) { + bbio->stripes =kmalloc_array (num_alloc_stripes, +sizeof (*bbio->stripes), +GFP_NOFS | __GFP_NOFAIL); + }else { + bbio->stripes = &bbio->__stripe; + } + +atomic_set (&bbio->stripes_pending, num_stripes); +for (i =0 ; i < num_stripes; i++) { +struct btrfs_bio_stripe *s = &bbio->stripes[i]; + + s->physical = map->stripes[stripe_index].physical + + stripe_offset + stripe_nr * map->stripe_len; + s->dev = map->stripes[stripe_index].dev; + stripe_index++; + +if (op ==BTRFS_MAP_WRITE && dev_replace_is_ongoing && + dev_replace->tgtdev && + !is_block_group_to_copy (fs_info, logical) && + s->dev->devid == dev_replace->srcdev->devid) { +struct btrfs_bio_stripe *r = + &bbio->stripes[replacement_idx++]; + + r->physical = s->physical; + r->dev = dev_replace->tgtdev; + max_errors++; +atomic_inc (&bbio->stripes_pending); + } + } + + bbio->max_errors = max_errors; + bbio->mirror_num = mirror_num; + goto out; + } + bioc =alloc_btrfs_io_context (fs_info, num_alloc_stripes, tgtdev_indexes);if (!bioc) { ret = -ENOMEM ;@@ -6601,9 +6640,6 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info,sort_parity_stripes (bioc, num_stripes); } -if (need_full_stripe (op)) - max_errors =btrfs_chunk_max_errors (map); -if (dev_replace_is_ongoing && dev_replace->tgtdev !=NULL &&need_full_stripe (op)) { handle_ops_on_dev_replace(op, &bioc, dev_replace, logical,@@ -6646,7 +6682,7 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, length, bioc_ret);return __btrfs_map_block (fs_info, op, logical, length, bioc_ret, - mirror_num,0 ); +NULL , mirror_num,0 ); }/* For Scrub/replace */ @@ -6654,14 +6690,15 @@ int btrfs_map_sblock(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, u64 logical, u64 *length, struct btrfs_io_context **bioc_ret) { - return __btrfs_map_block(fs_info, op, logical, length, bioc_ret, 0, 1); + return __btrfs_map_block(fs_info, op, logical, length, bioc_ret, NULL, + 0, 1); } -static struct btrfs_workqueue *btrfs_end_io_wq (struct btrfs_io_context *bioc) +static struct btrfs_workqueue *btrfs_end_io_wq (struct btrfs_bio *bbio) { -struct btrfs_fs_info *fs_info = bioc->fs_info; +struct btrfs_fs_info *fs_info =btrfs_sb (bbio->inode->i_sb); -switch (btrfs_bio (bioc->orig_bio)->end_io_type) { +switch (bbio->end_io_type) {case BTRFS_ENDIO_WQ_DATA_READ :return fs_info->endio_workers;case BTRFS_ENDIO_WQ_DATA_WRITE :@@ -6682,21 +6719,22 @@ static void btrfs_end_bio_work(struct btrfs_work *work)bio_endio (&bbio->bio); } -static void btrfs_end_bioc (struct btrfs_io_context *bioc,bool async) +static void btrfs_end_bbio (struct btrfs_bio *bbio,bool async) { -struct btrfs_workqueue *wq = async ?btrfs_end_io_wq (bioc) :NULL ; -struct bio *bio = bioc->orig_bio; -struct btrfs_bio *bbio =btrfs_bio (bio); +struct btrfs_workqueue *wq = async ?btrfs_end_io_wq (bbio) :NULL ; +struct bio *bio = &bbio->bio; - bbio->mirror_num = bioc->mirror_num; - bio->bi_private = bioc->private; - bio->bi_end_io = bioc->end_io; + bio->bi_private = bbio->private; + bio->bi_end_io = bbio->end_io; + +if (bbio->stripes != &bbio->__stripe) +kfree (bbio->stripes);/* * Only send an error to the higher layers if it is beyond the tolerance * threshold. */ -if (atomic_read (&bioc->error) > bioc->max_errors) +if (atomic_read (&bbio->error) > bbio->max_errors) bio->bi_status =BLK_STS_IOERR ;else bio->bi_status =BLK_STS_OK ;@@ -6707,16 +6745,14 @@ static void btrfs_end_bioc(struct btrfs_io_context *bioc, bool async) }else {bio_endio (bio); } - -btrfs_put_bioc (bioc); }static void btrfs_end_bio (struct bio *bio) { -struct btrfs_io_context *bioc = bio->bi_private; +struct btrfs_bio *bbio = bio->bi_private;if (bio->bi_status) { -atomic_inc (&bioc->error); +atomic_inc (&bbio->error);if (bio->bi_status ==BLK_STS_IOERR || bio->bi_status ==BLK_STS_TARGET ) {struct btrfs_device *dev =btrfs_bio (bio)->device;@@ -6734,40 +6770,39 @@ static void btrfs_end_bio(struct bio *bio) } } -if (bio != bioc->orig_bio) +if (bio != &bbio->bio)bio_put (bio); -btrfs_bio_counter_dec (bioc->fs_info); -if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc, true); +btrfs_bio_counter_dec (btrfs_sb (bbio->inode->i_sb)); +if (atomic_dec_and_test (&bbio->stripes_pending)) +btrfs_end_bbio (bbio, true); } -static void submit_stripe_bio (struct btrfs_io_context *bioc, -struct bio *orig_bio,int dev_nr,bool clone) +static void submit_stripe_bio (struct btrfs_bio *bbio,int dev_nr,bool clone) { -struct btrfs_fs_info *fs_info = bioc->fs_info; -struct btrfs_device *dev = bioc->stripes[dev_nr].dev; -u64 physical = bioc->stripes[dev_nr].physical; +struct btrfs_fs_info *fs_info =btrfs_sb (bbio->inode->i_sb); +struct btrfs_device *dev = bbio->stripes[dev_nr].dev; +u64 physical = bbio->stripes[dev_nr].physical;struct bio *bio;if (!dev || !dev->bdev ||test_bit (BTRFS_DEV_STATE_MISSING , &dev->dev_state) || - (btrfs_op (orig_bio) ==BTRFS_MAP_WRITE && + (btrfs_op (&bbio->bio) ==BTRFS_MAP_WRITE && !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state))) { -atomic_inc (&bioc->error); -if (atomic_dec_and_test (&bioc->stripes_pending)) -btrfs_end_bioc (bioc, false); +atomic_inc (&bbio->error); +if (atomic_dec_and_test (&bbio->stripes_pending)) +btrfs_end_bbio (bbio, false);return ; }if (clone) { - bio =btrfs_bio_clone (dev->bdev, orig_bio); + bio =btrfs_bio_clone (dev->bdev, &bbio->bio); }else { - bio = orig_bio; + bio = &bbio->bio;bio_set_dev (bio, dev->bdev); } - bio->bi_private = bioc; + bio->bi_private = bbio;btrfs_bio (bio)->device = dev; bio->bi_end_io = btrfs_end_bio; bio->bi_iter.bi_sector = physical >>9 ;@@ -6800,6 +6835,7 @@ static void submit_stripe_bio(struct btrfs_io_context *bioc,blk_status_t btrfs_map_bio (struct btrfs_fs_info *fs_info,struct bio *bio,int mirror_num) { +struct btrfs_bio *bbio =btrfs_bio (bio);u64 logical = bio->bi_iter.bi_sector <<9 ;u64 length = bio->bi_iter.bi_size;u64 map_length = length;@@ -6809,18 +6845,17 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,struct btrfs_io_context *bioc =NULL ;btrfs_bio_counter_inc_blocked (fs_info); - ret =__btrfs_map_block (fs_info,btrfs_op (bio), logical, - &map_length, &bioc, mirror_num,1 ); + ret =__btrfs_map_block (fs_info,btrfs_op (bio), logical, &map_length, + &bioc, bbio, mirror_num,1 );if (ret) goto out_dec; - total_devs = bioc->num_stripes; - bioc->orig_bio = bio; - bioc->private = bio->bi_private; - bioc->end_io = bio->bi_end_io; -atomic_set (&bioc->stripes_pending, bioc->num_stripes); + bbio->private = bio->bi_private; + bbio->end_io = bio->bi_end_io; + +if (bioc) { +ASSERT (bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ); -if (bioc->map_type &BTRFS_BLOCK_GROUP_RAID56_MASK ) {/* * In this case, map_length has been set to the length of a * single stripe; not the whole write. @@ -6834,6 +6869,7 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio, mirror_num, 1); goto out_dec; } +ASSERT (0 ); }if (map_length < length) {@@ -6843,8 +6879,9 @@ blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,BUG (); } + total_devs =atomic_read (&bbio->stripes_pending);for (dev_nr =0 ; dev_nr < total_devs; dev_nr++) -submit_stripe_bio (bioc, bio, dev_nr, dev_nr < total_devs -1 ); +submit_stripe_bio (bbio, dev_nr, dev_nr < total_devs -1 ); out_dec:btrfs_bio_counter_dec (fs_info);return errno_to_blk_status (ret);fs/btrfs/volumes.h +15
-5@@ -323,6 +323,11 @@ enum btrfs_endio_type { BTRFS_ENDIO_WQ_FREE_SPACE_READ, }; +struct btrfs_bio_stripe { +struct btrfs_device *dev; +u64 physical; +}; +/* * Additional info to pass along bio. * @@ -333,6 +338,16 @@ struct btrfs_bio {unsigned int mirror_num; +atomic_t stripes_pending; +atomic_t error; +int max_errors; + +struct btrfs_bio_stripe *stripes; +struct btrfs_bio_stripe __stripe; + +bio_end_io_t *end_io; +void *private; +enum btrfs_endio_type end_io_type;struct btrfs_work work;@@ -389,13 +404,8 @@ struct btrfs_io_stripe {*/ struct btrfs_io_context {refcount_t refs; -atomic_t stripes_pending;struct btrfs_fs_info *fs_info;u64 map_type;/* get from map_lookup->type */ -bio_end_io_t *end_io; -struct bio *orig_bio; -void *private; -atomic_t error;int max_errors;int num_stripes;int mirror_num;
Re: [PATCH 28/40] btrfs: do not allocate a btrfs_io_context in btrfs_map_bio
Quoted 6 lines · Qu Wenruo
Really not a fan of enlarging
btrfs_bioagain and again.Especially for the
btrfs_bio_stripeandbtrfs_bio_stripe* part.Considering how many bytes we waste for SINGLE profile, now we need an extra pointer which we will never really use.
How do we waste memory? We stop allocating the btrfs_io_context now which can be quite big.
Re: [PATCH 28/40] btrfs: do not allocate a btrfs_io_context in btrfs_map_bio
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Qu Wenruo
Really not a fan of enlarging
btrfs_bioagain and again.Especially for the
btrfs_bio_stripeandbtrfs_bio_stripe* part.Considering how many bytes we waste for SINGLE profile, now we need an extra pointer which we will never really use.
How do we waste memory? We stop allocating the
btrfs_io_contextnow which can be quite big.
Doesn't we waste the embedded __stripe if we choose to use the pointer one? And vice versus.
And for SINGLE profile, we don't really need btrfs_bio_stripe at all, we can fast-path just setting bdev and bi_sector, and submit without even overriding its endio/private.
Thanks,
Qu
Re: [PATCH 28/40] btrfs: do not allocate a btrfs_io_context in btrfs_map_bio
Quoted 9 lines · Qu Wenruo
Quoted 2 lines
How do we waste memory? We stop allocating the
btrfs_io_contextnow which can be quite big.Doesn't we waste the embedded
__stripeif we choose to use the pointer one? And vice versus.And for SINGLE profile, we don't really need
btrfs_bio_stripeat all, we can fast-path just setting bdev andbi_sector, and submit without even overriding its endio/private.
Yes, the 16 byes of the embedded stripe is wasted for I/O that doesn't use it. But all reads use it, which is typically the majority of all I/O and the most performance critical one. If the "waste" is a concern we can split out a separate btrfs_read_bio. Chance are that it will just use slack by the time this all settles with a bunch of other members that can be removed or unioned.
PATCH 29/40 btrfs: do not allocate a btrfs_bio for low-level bios
In reply to Christoph HellwigThe bios submitted from btrfs_map_bio don't interact with the rest of btrfs. The only btrfs_bio field is the device. Add a bbio backpointer pointer to struct btrfs_bio_stripe so that the private data can point to the stripe and just use a normal bio allocation for them.
--- fs/btrfs/extent_io.c | 13 ------------- fs/btrfs/extent_io.h | 1 - fs/btrfs/volumes.c | 21 +++++++++++---------- fs/btrfs/volumes.h | 5 ++++- 4 files changed, 15 insertions(+), 25 deletions(-)
fs/btrfs/extent_io.c +0 -13
@@ -3149,19 +3149,6 @@ struct bio *btrfs_bio_alloc(struct inode *inode, unsigned int nr_iovecs,return bio; } -struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio) -{ -struct btrfs_bio *bbio; -struct bio *new; - -/* Bio allocation backed by a bioset does not fail */ - new =bio_alloc_clone (bdev, bio,GFP_NOFS , &btrfs_bioset); - bbio =btrfs_bio (new); -btrfs_bio_init (btrfs_bio (new),btrfs_bio (bio)->inode); - bbio->iter = bio->bi_iter; -return new; -} -struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig,u64 offset,u64 size) {
fs/btrfs/extent_io.h +0 -1
@@ -279,7 +279,6 @@ void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end, u32 bits_to_clear, unsigned long page_ops);struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs,unsigned int opf); -struct bio *btrfs_bio_clone (struct block_device *bdev,struct bio *bio);struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig,u64 offset,u64 size);
fs/btrfs/volumes.c +11 -10
@@ -6749,23 +6749,21 @@ static void btrfs_end_bbio(struct btrfs_bio *bbio, bool async)static void btrfs_end_bio (struct bio *bio) { -struct btrfs_bio *bbio = bio->bi_private; +struct btrfs_bio_stripe *stripe = bio->bi_private; +struct btrfs_bio *bbio = stripe->bbio;if (bio->bi_status) {atomic_inc (&bbio->error);if (bio->bi_status ==BLK_STS_IOERR || bio->bi_status ==BLK_STS_TARGET ) { -struct btrfs_device *dev =btrfs_bio (bio)->device; - -ASSERT (dev->bdev);if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) -btrfs_dev_stat_inc_and_print (dev, +btrfs_dev_stat_inc_and_print (stripe->dev,BTRFS_DEV_STAT_WRITE_ERRS );else if (!(bio->bi_opf &REQ_RAHEAD )) -btrfs_dev_stat_inc_and_print (dev, +btrfs_dev_stat_inc_and_print (stripe->dev,BTRFS_DEV_STAT_READ_ERRS );if (bio->bi_opf &REQ_PREFLUSH ) -btrfs_dev_stat_inc_and_print (dev, +btrfs_dev_stat_inc_and_print (stripe->dev,BTRFS_DEV_STAT_FLUSH_ERRS ); } }
@@ -6796,14 +6794,17 @@ static void submit_stripe_bio(struct btrfs_bio *bbio, int dev_nr, bool clone) }if (clone) { - bio =btrfs_bio_clone (dev->bdev, &bbio->bio); + bio =bio_alloc_clone (dev->bdev, &bbio->bio,GFP_NOFS , + &fs_bio_set); }else { bio = &bbio->bio;bio_set_dev (bio, dev->bdev); +btrfs_bio (bio)->device = dev; } - bio->bi_private = bbio; -btrfs_bio (bio)->device = dev; + bbio->stripes[dev_nr].bbio = bbio; + bio->bi_private = &bbio->stripes[dev_nr]; + bio->bi_end_io = btrfs_end_bio; bio->bi_iter.bi_sector = physical >>9 ;/*
fs/btrfs/volumes.h +4 -1
@@ -325,7 +325,10 @@ enum btrfs_endio_type {struct btrfs_bio_stripe {struct btrfs_device *dev; -u64 physical; +union { +u64 physical;/* block mapping */ +struct btrfs_bio *bbio;/* for end I/O */ + }; };/*
Signature · 1 line
2.30.2
PATCH 30/40 iomap: add per-iomap_iter private data
In reply to Christoph HellwigAllow the file system to keep state for all iterations. For now only wire it up for direct I/O as there is an immediate need for it.
--- fs/iomap/direct-io.c | 3 +++ include/linux/iomap.h | 1 + 2 files changed, 4 insertions(+)
fs/iomap/direct-io.c +3 -0
@@ -477,6 +477,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, .pos = iocb->ki_pos, .len = iov_iter_count(iter), .flags = IOMAP_DIRECT, + .private = iocb->private, };loff_t end = iomi.pos + iomi.len -1 , ret =0 ; bool wait_for_completion =
@@ -504,6 +505,8 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, dio->submit.waiter = current; dio->submit.poll_bio =NULL ; +WRITE_ONCE (iocb->private,NULL ); +if (iov_iter_rw (iter) ==READ ) {if (iomi.pos >= dio->i_size) goto out_free_dio;
include/linux/iomap.h +1 -0
@@ -188,6 +188,7 @@ struct iomap_iter {unsigned flags;struct iomap iomap;struct iomap srcmap; +void *private; };int iomap_iter (struct iomap_iter *iter,const struct iomap_ops *ops);
Signature · 1 line
2.30.2
PATCH 31/40 iomap: add a new ->iomap_iter operation
In reply to Christoph HellwigThis new operation combines ->iomap_beging, ->iomap_end and the actual advancing of the iter into a new operation. Matthew Wilcox originally proposed this kind of interface to eventually allow inlining most of the iteration and avoid an indirect call. But it also allows for more control in the file system to e.g. keep a little more state over multiple iterations, which is something we'll need to improve the btrfs direct I/O code.
--- fs/iomap/iter.c | 13 +++++++++---- include/linux/iomap.h | 17 +++++++++++++++++ 2 files changed, 26 insertions(+), 4 deletions(-)
fs/iomap/iter.c +9 -4
@@ -7,7 +7,7 @@#include <linux/iomap.h> #include "trace.h" -static inline int iomap_iter_advance (struct iomap_iter *iter) +int iomap_iter_advance (struct iomap_iter *iter) {/* handle the previous iteration (if any) */ if (iter->iomap.length) {
@@ -27,8 +27,9 @@ static inline int iomap_iter_advance(struct iomap_iter *iter)memset (&iter->srcmap,0 ,sizeof (iter->srcmap));return 1 ; } +EXPORT_SYMBOL_GPL (iomap_iter_advance); -static inline void iomap_iter_done (struct iomap_iter *iter) +void iomap_iter_done (struct iomap_iter *iter) {WARN_ON_ONCE (iter->iomap.offset > iter->pos);WARN_ON_ONCE (iter->iomap.length ==0 );
@@ -38,6 +39,7 @@ static inline void iomap_iter_done(struct iomap_iter *iter)if (iter->srcmap.type !=IOMAP_HOLE )trace_iomap_iter_srcmap (iter->inode, &iter->srcmap); } +EXPORT_SYMBOL_GPL (iomap_iter_done);/** * iomap_iter - iterate over a ranges in a file
@@ -58,10 +60,13 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops) {int ret; +if (ops->iomap_iter) +return ops->iomap_iter (iter); +if (iter->iomap.length && ops->iomap_end) { ret = ops->iomap_end (iter->inode, iter->pos,iomap_length (iter), - iter->processed >0 ? iter->processed :0 , - iter->flags, &iter->iomap); +iomap_processed (iter), iter->flags, + &iter->iomap);if (ret <0 && !iter->processed)return ret; }
include/linux/iomap.h +17 -0
@@ -14,6 +14,7 @@ struct address_space;struct fiemap_extent_info ;struct inode ;struct iomap_dio ; +struct iomap_iter ;struct iomap_writepage_ctx ;struct iov_iter ;struct kiocb ;
@@ -148,6 +149,8 @@ struct iomap_page_ops {#endif /* CONFIG_FS_DAX */ struct iomap_ops { +int (*iomap_iter)(struct iomap_iter *iter); +/* * Return the existing mapping at pos, or reserve space starting at * pos for up to length, as long as we can do it as a single mapping.
@@ -208,6 +211,17 @@ static inline u64 iomap_length(const struct iomap_iter *iter)return min (iter->len, end - iter->pos); } +/** +* iomap_length - amount of data processed by the previous iomap iteration +* @iter: iteration structure +*/ +static inline u64 iomap_processed (const struct iomap_iter *iter) +{ +if (iter->processed <=0 ) +return 0 ; +return iter->processed; +} +/** * iomap_iter_srcmap - return the source map for the current iomap iteration * @i: iteration structure
@@ -224,6 +238,9 @@ static inline const struct iomap *iomap_iter_srcmap(const struct iomap_iter *i)return &i->iomap; } +int iomap_iter_advance (struct iomap_iter *iter); +void iomap_iter_done (struct iomap_iter *iter); +ssize_t iomap_file_buffered_write (struct kiocb *iocb,struct iov_iter *from,const struct iomap_ops *ops);int iomap_readpage (struct page *page,const struct iomap_ops *ops);
Signature · 1 line
2.30.2
PATCH 32/40 iomap: optionally allocate dio bios from a file system bio_set
In reply to Christoph HellwigAllow the file system to provide a specific bio_set for allocating direct I/O bios. This will allow file systems that use the ->submit_io hook to stash away additional information for file system use.
To make use of this additional space for information the file system also needs to hook into the completion path and thus override the ->bi_end_io callback. Export iomap_dio_bio_end_io so that the file system can call back into the core iomap direct I/O code after doing so.
--- fs/iomap/direct-io.c | 18 ++++++++++++++---- include/linux/iomap.h | 3 +++ 2 files changed, 17 insertions(+), 4 deletions(-)
fs/iomap/direct-io.c +14 -4
@@ -50,6 +50,15 @@ struct iomap_dio { }; }; +static struct bio *iomap_dio_alloc_bio (const struct iomap_iter *iter, +struct iomap_dio *dio,unsigned short nr_vecs,unsigned int opf) +{ +if (dio->dops && dio->dops->bio_set) +return bio_alloc_bioset (iter->iomap.bdev, nr_vecs, opf, +GFP_KERNEL , dio->dops->bio_set); +return bio_alloc (iter->iomap.bdev, nr_vecs, opf,GFP_KERNEL ); +} +static void iomap_dio_submit_bio (const struct iomap_iter *iter,struct iomap_dio *dio,struct bio *bio,loff_t pos) {
@@ -143,7 +152,7 @@ static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret)cmpxchg (&dio->error,0 , ret); } -static void iomap_dio_bio_end_io (struct bio *bio) +void iomap_dio_bio_end_io (struct bio *bio) {struct iomap_dio *dio = bio->bi_private;bool should_dirty = (dio->flags &IOMAP_DIO_DIRTY );
@@ -175,15 +184,16 @@ static void iomap_dio_bio_end_io(struct bio *bio)bio_put (bio); } } +EXPORT_SYMBOL_GPL (iomap_dio_bio_end_io);static void iomap_dio_zero (const struct iomap_iter *iter,struct iomap_dio *dio,loff_t pos,unsigned len) {struct page *page =ZERO_PAGE (0 ); -int flags =REQ_SYNC |REQ_IDLE ;struct bio *bio; - bio =bio_alloc (iter->iomap.bdev,1 ,REQ_OP_WRITE | flags,GFP_KERNEL ); + bio =iomap_dio_alloc_bio (iter, dio,1 , +REQ_OP_WRITE |REQ_SYNC |REQ_IDLE ); bio->bi_iter.bi_sector =iomap_sector (&iter->iomap, pos); bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io;
@@ -307,7 +317,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, goto out; } - bio =bio_alloc (iomap->bdev, nr_pages, bio_opf,GFP_KERNEL ); + bio =iomap_dio_alloc_bio (iter, dio, nr_pages, bio_opf); bio->bi_iter.bi_sector =iomap_sector (iomap, pos); bio->bi_ioprio = dio->iocb->ki_ioprio; bio->bi_private = dio;
include/linux/iomap.h +3 -0
@@ -341,6 +341,8 @@ struct iomap_dio_ops { unsigned flags);void (*submit_io)(const struct iomap_iter *iter,struct bio *bio,loff_t file_offset); + +struct bio_set *bio_set; };/*
@@ -370,6 +372,7 @@ struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, unsigned int dio_flags, size_t done_before);ssize_t iomap_dio_complete (struct iomap_dio *dio); +void iomap_dio_bio_end_io (struct bio *bio);#ifdef CONFIG_SWAP struct file ;
Signature · 1 line
2.30.2
PATCH 33/40 iomap: add a hint to ->submit_io if there is more I/O coming
In reply to Christoph HellwigBtrfs would like to optimize checksum offloading to threads depending on if there is more I/O to come. Pass that information to the ->submit_io method.
--- fs/btrfs/inode.c | 2 +- fs/iomap/direct-io.c | 8 ++++---- include/linux/iomap.h | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-)
fs/btrfs/inode.c +1 -1
@@ -7917,7 +7917,7 @@ static struct btrfs_dio_private *btrfs_create_dio_private(struct bio *dio_bio, }static void btrfs_submit_direct (const struct iomap_iter *iter, -struct bio *dio_bio,loff_t file_offset) +struct bio *dio_bio,loff_t file_offset,bool more) {struct inode *inode = iter->inode;const bool write = (btrfs_op (dio_bio) ==BTRFS_MAP_WRITE );
fs/iomap/direct-io.c +4 -4
@@ -60,7 +60,7 @@ static struct bio *iomap_dio_alloc_bio(const struct iomap_iter *iter, }static void iomap_dio_submit_bio (const struct iomap_iter *iter, -struct iomap_dio *dio,struct bio *bio,loff_t pos) +struct iomap_dio *dio,struct bio *bio,loff_t pos,bool more) {atomic_inc (&dio->ref);
@@ -70,7 +70,7 @@ static void iomap_dio_submit_bio(const struct iomap_iter *iter, }if (dio->dops && dio->dops->submit_io) - dio->dops->submit_io (iter, bio, pos); + dio->dops->submit_io (iter, bio, pos, more);else submit_bio (bio); }
@@ -200,7 +200,7 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,get_page (page);__bio_add_page (bio, page, len,0 ); -iomap_dio_submit_bio (iter, dio, bio, pos); +iomap_dio_submit_bio (iter, dio, bio, pos, false); }/*
@@ -353,7 +353,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,*/ if (nr_pages) dio->iocb->ki_flags &= ~IOCB_HIPRI ; -iomap_dio_submit_bio (iter, dio, bio, pos); +iomap_dio_submit_bio (iter, dio, bio, pos, nr_pages); pos += n; }while (nr_pages);
include/linux/iomap.h +1 -1
@@ -340,7 +340,7 @@ struct iomap_dio_ops {int (*end_io)(struct kiocb *iocb,ssize_t size,int error,unsigned flags);void (*submit_io)(const struct iomap_iter *iter,struct bio *bio, -loff_t file_offset); +loff_t file_offset,bool more);struct bio_set *bio_set; };
Signature · 1 line
2.30.2
PATCH 34/40 btrfs: add a btrfs_dio_rw wrapper
In reply to Christoph HellwigAdd a wrapper around iomap_dio_rw that keeps the direct I/O internals isolated in inode.c.
--- fs/btrfs/ctree.h | 4 ++-- fs/btrfs/file.c | 6 ++---- fs/btrfs/inode.c | 11 +++++++++-- 3 files changed, 13 insertions(+), 8 deletions(-)
fs/btrfs/ctree.h +2 -2
@@ -3255,9 +3255,9 @@ int btrfs_writepage_cow_fixup(struct page *page);void btrfs_writepage_endio_finish_ordered (struct btrfs_inode *inode,struct page *page,u64 start,u64 end,bool uptodate); +ssize_t btrfs_dio_rw (struct kiocb *iocb,struct iov_iter *iter, +size_t done_before);extern const struct dentry_operations btrfs_dentry_operations; -extern const struct iomap_ops btrfs_dio_iomap_ops; -extern const struct iomap_dio_ops btrfs_dio_ops;/* Inode locking type flags, by default the exclusive lock is taken */ #define BTRFS_ILOCK_SHARED (1U << 0)
fs/btrfs/file.c +2 -4
@@ -1967,8 +1967,7 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from)*/ again: from->nofault = true; - err =iomap_dio_rw (iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, -IOMAP_DIO_PARTIAL , written); + err =btrfs_dio_rw (iocb, from, written); from->nofault = false;/* No increment (+=) because iomap returns a cumulative value. */
@@ -3719,8 +3718,7 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to)*/ pagefault_disable (); to->nofault = true; - ret =iomap_dio_rw (iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, -IOMAP_DIO_PARTIAL , read); + ret =btrfs_dio_rw (iocb, to, read); to->nofault = false;pagefault_enable ();
fs/btrfs/inode.c +9 -2
@@ -8050,15 +8050,22 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,btrfs_dio_private_put (dip); } -const struct iomap_ops btrfs_dio_iomap_ops = { +static const struct iomap_ops btrfs_dio_iomap_ops = { .iomap_begin = btrfs_dio_iomap_begin, .iomap_end = btrfs_dio_iomap_end, }; -const struct iomap_dio_ops btrfs_dio_ops = { +static const struct iomap_dio_ops btrfs_dio_ops = { .submit_io = btrfs_submit_direct, }; +ssize_t btrfs_dio_rw (struct kiocb *iocb,struct iov_iter *iter, +size_t done_before) +{ +return iomap_dio_rw (iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops, +IOMAP_DIO_PARTIAL , done_before); +} +static int btrfs_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo,u64 start,u64 len) {
Signature · 1 line
2.30.2
PATCH 35/40 btrfs: allocate dio_data on stack
In reply to Christoph HellwigMake use of the new iomap_iter->private field to avoid a memory allocation per iomap range.
--- fs/btrfs/inode.c | 36 ++++++++++++++---------------------- 1 file changed, 14 insertions(+), 22 deletions(-)
fs/btrfs/inode.c +14 -22
@@ -7511,10 +7511,11 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start, loff_t length, unsigned int flags, struct iomap *iomap, struct iomap *srcmap) { + struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap);struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);struct extent_map *em;struct extent_state *cached_state =NULL ; -struct btrfs_dio_data *dio_data =NULL ; +struct btrfs_dio_data *dio_data = iter->private;u64 lockstart, lockend;const bool write = !!(flags &IOMAP_WRITE );int ret =0 ;
@@ -7541,21 +7542,15 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start,return ret; } - dio_data =kzalloc (sizeof (*dio_data),GFP_NOFS ); -if (!dio_data) -return -ENOMEM ; - - iomap->private = dio_data; - + dio_data->submitted =0 ; + dio_data->data_reserved =NULL ;/* * If this errors out it's because we couldn't invalidate pagecache for * this range and we need to fallback to buffered. */ -if (lock_extent_direct (inode, lockstart, lockend, &cached_state, write)) { - ret = -ENOTBLK ; - goto err; - } +if (lock_extent_direct (inode, lockstart, lockend, &cached_state, write)) +return -ENOTBLK ; em =btrfs_get_extent (BTRFS_I (inode),NULL ,0 , start, len);if (IS_ERR (em)) {
@@ -7664,24 +7659,22 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start, unlock_err:unlock_extent_cached (&BTRFS_I (inode)->io_tree, lockstart, lockend, &cached_state); -err: -kfree (dio_data); -return ret; }static int btrfs_dio_iomap_end (struct inode *inode,loff_t pos,loff_t length,ssize_t written,unsigned int flags,struct iomap *iomap) { -int ret =0 ; -struct btrfs_dio_data *dio_data = iomap->private; + struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap); +struct btrfs_dio_data *dio_data = iter->private;size_t submitted = dio_data->submitted;const bool write = !!(flags &IOMAP_WRITE ); +int ret =0 ;if (!write && (iomap->type ==IOMAP_HOLE )) {/* If reading from a hole, unlock and return */ unlock_extent (&BTRFS_I (inode)->io_tree, pos, pos + length -1 ); - goto out; +return 0 ; }if (submitted < length) {
@@ -7698,10 +7691,6 @@ static int btrfs_dio_iomap_end(struct inode *inode, loff_t pos, loff_t length,if (write)extent_changeset_free (dio_data->data_reserved); -out: -kfree (dio_data); - iomap->private =NULL ; -return ret; }
@@ -7935,7 +7924,7 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,int ret;blk_status_t status;struct btrfs_io_geometry geom; -struct btrfs_dio_data *dio_data = iter->iomap.private; +struct btrfs_dio_data *dio_data = iter->private;struct extent_map *em =NULL ; dip =btrfs_create_dio_private (dio_bio, inode, file_offset);
@@ -8062,6 +8051,9 @@ static const struct iomap_dio_ops btrfs_dio_ops = {ssize_t btrfs_dio_rw (struct kiocb *iocb,struct iov_iter *iter,size_t done_before) { +struct btrfs_dio_data data; + + iocb->private = &data;return iomap_dio_rw (iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops,IOMAP_DIO_PARTIAL , done_before); }
Signature · 1 line
2.30.2
PATCH 36/40 btrfs: implement ->iomap_iter
In reply to Christoph HellwigSwitch from the separate ->iomap_begin and ->iomap_end methods to ->iomap_iter to allow or greater control over the iteration in subsequent patches.
--- fs/btrfs/inode.c | 69 ++++++++++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 25 deletions(-)
fs/btrfs/inode.c +44 -25
@@ -7507,17 +7507,18 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,return ret; } -static int btrfs_dio_iomap_begin (struct inode *inode,loff_t start, -loff_t length,unsigned int flags,struct iomap *iomap, -struct iomap *srcmap) +static int btrfs_dio_iomap_begin (struct iomap_iter *iter) { - struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap); +struct inode *inode = iter->inode;struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); +loff_t start = iter->pos; +loff_t length = iter->len; +struct iomap *iomap = &iter->iomap;struct extent_map *em;struct extent_state *cached_state =NULL ;struct btrfs_dio_data *dio_data = iter->private;u64 lockstart, lockend; -const bool write = !!(flags &IOMAP_WRITE ); +bool write = (iter->flags &IOMAP_WRITE );int ret =0 ;u64 len = length;bool unlock_extents = false;
@@ -7602,7 +7603,7 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start,* which we return back to our caller - we should only return EIOCBQUEUED * after we have submitted bios for all the extents in the range. */ -if ((flags &IOMAP_NOWAIT ) && len < length) { +if ((iter->flags &IOMAP_NOWAIT ) && len < length) {free_extent_map (em); ret = -EAGAIN ; goto unlock_err;
@@ -7662,30 +7663,28 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start,return ret; } -static int btrfs_dio_iomap_end (struct inode *inode,loff_t pos,loff_t length, -ssize_t written,unsigned int flags,struct iomap *iomap) +static int btrfs_dio_iomap_end (struct iomap_iter *iter) { - struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap);struct btrfs_dio_data *dio_data = iter->private; -size_t submitted = dio_data->submitted; -const bool write = !!(flags &IOMAP_WRITE ); +struct btrfs_inode *bi =BTRFS_I (iter->inode); +bool write = (iter->flags &IOMAP_WRITE ); +loff_t length =iomap_length (iter); +loff_t pos = iter->pos;int ret =0 ; -if (!write && (iomap->type ==IOMAP_HOLE )) { +if (!write && iter->iomap.type ==IOMAP_HOLE ) {/* If reading from a hole, unlock and return */ -unlock_extent (&BTRFS_I (inode)->io_tree, pos, pos + length -1 ); +unlock_extent (&bi->io_tree, pos, pos + length -1 );return 0 ; } -if (submitted < length) { - pos += submitted; - length -= submitted; +if (dio_data->submitted < length) { + pos += dio_data->submitted; + length -= dio_data->submitted;if (write) -__endio_write_update_ordered (BTRFS_I (inode), pos, - length, false); +__endio_write_update_ordered (bi, pos, length, false);else -unlock_extent (&BTRFS_I (inode)->io_tree, pos, - pos + length -1 ); +unlock_extent (&bi->io_tree, pos, pos + length -1 ); ret = -ENOTBLK ; }
@@ -7694,6 +7693,31 @@ static int btrfs_dio_iomap_end(struct inode *inode, loff_t pos, loff_t length,return ret; } +static int btrfs_dio_iomap_iter (struct iomap_iter *iter) +{ +int ret; + +if (iter->iomap.length) { + ret =btrfs_dio_iomap_end (iter); +if (ret <0 && !iter->processed) +return ret; + } + + ret =iomap_iter_advance (iter); +if (ret <=0 ) +return ret; + + ret =btrfs_dio_iomap_begin (iter); +if (ret <0 ) +return ret; +iomap_iter_done (iter); +return 1 ; +} + +static const struct iomap_ops btrfs_dio_iomap_ops = { + .iomap_iter = btrfs_dio_iomap_iter, +}; +static void btrfs_dio_private_put (struct btrfs_dio_private *dip) {/*
@@ -8039,11 +8063,6 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,btrfs_dio_private_put (dip); } -static const struct iomap_ops btrfs_dio_iomap_ops = { - .iomap_begin = btrfs_dio_iomap_begin, - .iomap_end = btrfs_dio_iomap_end, -}; -static const struct iomap_dio_ops btrfs_dio_ops = { .submit_io = btrfs_submit_direct, };
Signature · 1 line
2.30.2
PATCH 37/40 btrfs: add a btrfs_get_stripe_info helper
In reply to Christoph Hellwig--- fs/btrfs/compression.c | 26 ++++++++----------------- fs/btrfs/extent_io.c | 24 ++++++++--------------- fs/btrfs/inode.c | 32 ++++++++++-------------------- fs/btrfs/volumes.c | 44 +++++++++++++++++++++++++++++++++++++++--- fs/btrfs/volumes.h | 20 ++----------------- 5 files changed, 69 insertions(+), 77 deletions(-)
fs/btrfs/compression.c +8 -18
@@ -445,10 +445,9 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_byte u64 *next_stripe_start) {struct btrfs_fs_info *fs_info =btrfs_sb (cb->inode->i_sb); -struct btrfs_io_geometry geom; -struct extent_map *em; +struct block_device *bdev;struct bio *bio; -int ret; +u64 len; bio =btrfs_bio_alloc (cb->inode,BIO_MAX_VECS , opf); bio->bi_iter.bi_sector = disk_bytenr >>SECTOR_SHIFT ;
@@ -459,23 +458,14 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_byteelse btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ; - em =btrfs_get_chunk_map (fs_info, disk_bytenr, fs_info->sectorsize); -if (IS_ERR (em)) { -bio_put (bio); -return ERR_CAST (em); - } + bdev =btrfs_get_stripe_info (fs_info,btrfs_op (bio), disk_bytenr, + fs_info->sectorsize, &len); +if (IS_ERR (bdev)) +return ERR_CAST (bdev);if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) -bio_set_dev (bio, em->map_lookup->stripes[0 ].dev->bdev); - - ret =btrfs_get_io_geometry (fs_info, em,btrfs_op (bio), disk_bytenr, &geom); -free_extent_map (em); -if (ret <0 ) { -bio_put (bio); -return ERR_PTR (ret); - } - *next_stripe_start = disk_bytenr + geom.len; - +bio_set_dev (bio, bdev); + *next_stripe_start = disk_bytenr + len;return bio; }
fs/btrfs/extent_io.c +8 -16
@@ -3235,11 +3235,10 @@ static int calc_bio_boundaries(struct btrfs_bio_ctrl *bio_ctrl, struct btrfs_inode *inode, u64 file_offset) {struct btrfs_fs_info *fs_info = inode->root->fs_info; -struct btrfs_io_geometry geom;struct btrfs_ordered_extent *ordered; -struct extent_map *em;u64 logical = (bio_ctrl->bio->bi_iter.bi_sector <<SECTOR_SHIFT ); -int ret; +struct block_device *bdev; +u64 len;/* * Pages for compressed extent are never submitted to disk directly,
@@ -3253,19 +3252,12 @@ static int calc_bio_boundaries(struct btrfs_bio_ctrl *bio_ctrl, bio_ctrl->len_to_stripe_boundary =U32_MAX ;return 0 ; } - em =btrfs_get_chunk_map (fs_info, logical, fs_info->sectorsize); -if (IS_ERR (em)) -return PTR_ERR (em); - ret =btrfs_get_io_geometry (fs_info, em,btrfs_op (bio_ctrl->bio), - logical, &geom); -free_extent_map (em); -if (ret <0 ) { -return ret; - } -if (geom.len >U32_MAX ) - bio_ctrl->len_to_stripe_boundary =U32_MAX ; -else - bio_ctrl->len_to_stripe_boundary = (u32 )geom.len; + + bdev =btrfs_get_stripe_info (fs_info,btrfs_op (bio_ctrl->bio), logical, + fs_info->sectorsize, &len); +if (IS_ERR (bdev)) +return PTR_ERR (bdev); + bio_ctrl->len_to_stripe_boundary =min (len, (u64 )U32_MAX );if (bio_op (bio_ctrl->bio) !=REQ_OP_ZONE_APPEND ) { bio_ctrl->len_to_oe_boundary =U32_MAX ;
fs/btrfs/inode.c +10 -22
@@ -7944,12 +7944,9 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,u64 submit_len;u64 clone_offset =0 ;u64 clone_len; -u64 logical; -int ret;blk_status_t status; -struct btrfs_io_geometry geom;struct btrfs_dio_data *dio_data = iter->private; -struct extent_map *em =NULL ; +u64 len; dip =btrfs_create_dio_private (dio_bio, inode, file_offset);if (!dip) {
@@ -7978,21 +7975,16 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, submit_len = dio_bio->bi_iter.bi_size; do { - logical = start_sector << 9; - em = btrfs_get_chunk_map(fs_info, logical, submit_len); - if (IS_ERR(em)) { - status = errno_to_blk_status(PTR_ERR(em)); - em = NULL; - goto out_err_em; - } - ret = btrfs_get_io_geometry(fs_info, em, btrfs_op(dio_bio), - logical, &geom); - if (ret) { - status = errno_to_blk_status(ret); - goto out_err_em; + struct block_device *bdev; + + bdev = btrfs_get_stripe_info(fs_info, btrfs_op(dio_bio), + start_sector << 9, submit_len, &len); + if (IS_ERR(bdev)) { + status = errno_to_blk_status(PTR_ERR(bdev)); + goto out_err; } - clone_len = min(submit_len, geom.len); + clone_len = min(submit_len, len); ASSERT(clone_len <= UINT_MAX); /*
@@ -8044,20 +8036,16 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,bio_put (bio);if (submit_len >0 )refcount_dec (&dip->refs); - goto out_err_em; + goto out_err; } dio_data->submitted += clone_len; clone_offset += clone_len; start_sector += clone_len >>9 ; file_offset += clone_len; - -free_extent_map (em); }while (submit_len >0 );return ; -out_err_em: -free_extent_map (em); out_err: dip->dio_bio->bi_status = status;btrfs_dio_private_put (dip);
fs/btrfs/volumes.c +41 -3
62 lines not on this page · raw message
fs/btrfs/volumes.h +2 -18
@@ -17,21 +17,6 @@ extern struct mutex uuid_mutex;#define BTRFS_STRIPE_LEN SZ_64K -struct btrfs_io_geometry { -/* remaining bytes before crossing a stripe */ -u64 len; -/* offset of logical address in chunk */ -u64 offset; -/* length of single IO stripe */ -u64 stripe_len; -/* number of stripe where address falls */ -u64 stripe_nr; -/* offset of address in stripe */ -u64 stripe_offset; -/* offset of raid56 stripe into the chunk */ -u64 raid56_stripe_offset; -}; -/* * Use sequence counter to get consistent device stat data on * 32-bit processors.
@@ -520,9 +505,8 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,int btrfs_map_sblock (struct btrfs_fs_info *fs_info,enum btrfs_map_op op,u64 logical,u64 *length,struct btrfs_io_context **bioc_ret); -int btrfs_get_io_geometry (struct btrfs_fs_info *fs_info,struct extent_map *map, -enum btrfs_map_op op,u64 logical, -struct btrfs_io_geometry *io_geom); +struct block_device *btrfs_get_stripe_info (struct btrfs_fs_info *fs_info, +enum btrfs_map_op op,u64 logical,u64 length,u64 *lenp);int btrfs_read_sys_array (struct btrfs_fs_info *fs_info);int btrfs_read_chunk_tree (struct btrfs_fs_info *fs_info); struct btrfs_block_group *btrfs_create_chunk(struct btrfs_trans_handle *trans,
Signature · 1 line
2.30.2
Re: [PATCH 37/40] btrfs: add a btrfs_get_stripe_info helper
Quoted 1 line · Christoph Hellwig
---
Thanks,
Qu
Quoted 275 lines
fs/btrfs/compression.c | 26 ++++++++----------------- fs/btrfs/extent_io.c | 24 ++++++++--------------- fs/btrfs/inode.c | 32 ++++++++++-------------------- fs/btrfs/volumes.c | 44 +++++++++++++++++++++++++++++++++++++++--- fs/btrfs/volumes.h | 20 ++----------------- 5 files changed, 69 insertions(+), 77 deletions(-)fs/btrfs/compression.c +8
-18@@ -445,10 +445,9 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_byte u64 *next_stripe_start) {struct btrfs_fs_info *fs_info =btrfs_sb (cb->inode->i_sb); -struct btrfs_io_geometry geom; -struct extent_map *em; +struct block_device *bdev;struct bio *bio; -int ret; +u64 len; bio =btrfs_bio_alloc (cb->inode,BIO_MAX_VECS , opf); bio->bi_iter.bi_sector = disk_bytenr >>SECTOR_SHIFT ;@@ -459,23 +458,14 @@ static struct bio *alloc_compressed_bio(struct compressed_bio *cb, u64 disk_byteelse btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ; - em =btrfs_get_chunk_map (fs_info, disk_bytenr, fs_info->sectorsize); -if (IS_ERR (em)) { -bio_put (bio); -return ERR_CAST (em); - } + bdev =btrfs_get_stripe_info (fs_info,btrfs_op (bio), disk_bytenr, + fs_info->sectorsize, &len); +if (IS_ERR (bdev)) +return ERR_CAST (bdev);if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) -bio_set_dev (bio, em->map_lookup->stripes[0 ].dev->bdev); - - ret =btrfs_get_io_geometry (fs_info, em,btrfs_op (bio), disk_bytenr, &geom); -free_extent_map (em); -if (ret <0 ) { -bio_put (bio); -return ERR_PTR (ret); - } - *next_stripe_start = disk_bytenr + geom.len; - +bio_set_dev (bio, bdev); + *next_stripe_start = disk_bytenr + len;return bio; }fs/btrfs/extent_io.c +8
-16@@ -3235,11 +3235,10 @@ static int calc_bio_boundaries(struct btrfs_bio_ctrl *bio_ctrl, struct btrfs_inode *inode, u64 file_offset) {struct btrfs_fs_info *fs_info = inode->root->fs_info; -struct btrfs_io_geometry geom;struct btrfs_ordered_extent *ordered; -struct extent_map *em;u64 logical = (bio_ctrl->bio->bi_iter.bi_sector <<SECTOR_SHIFT ); -int ret; +struct block_device *bdev; +u64 len;/* * Pages for compressed extent are never submitted to disk directly, @@ -3253,19 +3252,12 @@ static int calc_bio_boundaries(struct btrfs_bio_ctrl *bio_ctrl, bio_ctrl->len_to_stripe_boundary =U32_MAX ;return 0 ; } - em =btrfs_get_chunk_map (fs_info, logical, fs_info->sectorsize); -if (IS_ERR (em)) -return PTR_ERR (em); - ret =btrfs_get_io_geometry (fs_info, em,btrfs_op (bio_ctrl->bio), - logical, &geom); -free_extent_map (em); -if (ret <0 ) { -return ret; - } -if (geom.len >U32_MAX ) - bio_ctrl->len_to_stripe_boundary =U32_MAX ; -else - bio_ctrl->len_to_stripe_boundary = (u32 )geom.len; + + bdev =btrfs_get_stripe_info (fs_info,btrfs_op (bio_ctrl->bio), logical, + fs_info->sectorsize, &len); +if (IS_ERR (bdev)) +return PTR_ERR (bdev); + bio_ctrl->len_to_stripe_boundary =min (len, (u64 )U32_MAX );if (bio_op (bio_ctrl->bio) !=REQ_OP_ZONE_APPEND ) { bio_ctrl->len_to_oe_boundary =U32_MAX ;fs/btrfs/inode.c +10
-22@@ -7944,12 +7944,9 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,u64 submit_len;u64 clone_offset =0 ;u64 clone_len; -u64 logical; -int ret;blk_status_t status; -struct btrfs_io_geometry geom;struct btrfs_dio_data *dio_data = iter->private; -struct extent_map *em =NULL ; +u64 len; dip =btrfs_create_dio_private (dio_bio, inode, file_offset);if (!dip) {@@ -7978,21 +7975,16 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, submit_len = dio_bio->bi_iter.bi_size; do { - logical = start_sector << 9; - em = btrfs_get_chunk_map(fs_info, logical, submit_len); - if (IS_ERR(em)) { - status = errno_to_blk_status(PTR_ERR(em)); - em = NULL; - goto out_err_em; - } - ret = btrfs_get_io_geometry(fs_info, em, btrfs_op(dio_bio), - logical, &geom); - if (ret) { - status = errno_to_blk_status(ret); - goto out_err_em; + struct block_device *bdev; + + bdev = btrfs_get_stripe_info(fs_info, btrfs_op(dio_bio), + start_sector << 9, submit_len, &len); + if (IS_ERR(bdev)) { + status = errno_to_blk_status(PTR_ERR(bdev)); + goto out_err; } - clone_len = min(submit_len, geom.len); + clone_len = min(submit_len, len); ASSERT(clone_len <= UINT_MAX); /*@@ -8044,20 +8036,16 @@ static void btrfs_submit_direct(const struct iomap_iter *iter,bio_put (bio);if (submit_len >0 )refcount_dec (&dip->refs); - goto out_err_em; + goto out_err; } dio_data->submitted += clone_len; clone_offset += clone_len; start_sector += clone_len >>9 ; file_offset += clone_len; - -free_extent_map (em); }while (submit_len >0 );return ; -out_err_em: -free_extent_map (em); out_err: dip->dio_bio->bi_status = status;btrfs_dio_private_put (dip);fs/btrfs/volumes.c +41
-3@@ -6301,6 +6301,21 @@ static bool need_full_stripe(enum btrfs_map_op op)return (op ==BTRFS_MAP_WRITE || op ==BTRFS_MAP_GET_READ_MIRRORS ); } +struct btrfs_io_geometry { +/* remaining bytes before crossing a stripe */ +u64 len; +/* offset of logical address in chunk */ +u64 offset; +/* length of single IO stripe */ +u64 stripe_len; +/* number of stripe where address falls */ +u64 stripe_nr; +/* offset of address in stripe */ +u64 stripe_offset; +/* offset of raid56 stripe into the chunk */ +u64 raid56_stripe_offset; +}; +/* * Calculate the geometry of a particular (address, len) tuple. This * information is used to calculate how big a particular bio can get before it @@ -6315,9 +6330,10 @@ static bool need_full_stripe(enum btrfs_map_op op)* Returns < 0 in case a chunk for the given logical address cannot be found, * usually shouldn't happen unless @logical is corrupted, 0 otherwise. */ -int btrfs_get_io_geometry (struct btrfs_fs_info *fs_info,struct extent_map *em, -enum btrfs_map_op op,u64 logical, -struct btrfs_io_geometry *io_geom) +static int btrfs_get_io_geometry (struct btrfs_fs_info *fs_info, +struct extent_map *em, +enum btrfs_map_op op,u64 logical, +struct btrfs_io_geometry *io_geom) {struct map_lookup *map;u64 len;@@ -6394,6 +6410,28 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,return 0 ; } +struct block_device *btrfs_get_stripe_info (struct btrfs_fs_info *fs_info, +enum btrfs_map_op op,u64 logical,u64 len,u64 *lenp) +{ +struct btrfs_io_geometry geom; +struct block_device *bdev; +struct extent_map *em; +int ret; + + em =btrfs_get_chunk_map (fs_info, logical, len); +if (IS_ERR (em)) +return ERR_CAST (em); + + bdev = em->map_lookup->stripes[0 ].dev->bdev; + + ret =btrfs_get_io_geometry (fs_info, em, op, logical, &geom); +free_extent_map (em); +if (ret <0 ) +return ERR_PTR (ret); + *lenp = geom.len; +return bdev; +} + static int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op, u64 logical, u64 *length, struct btrfs_io_context **bioc_ret, struct btrfs_bio *bbio,fs/btrfs/volumes.h +2
-18@@ -17,21 +17,6 @@ extern struct mutex uuid_mutex;#define BTRFS_STRIPE_LEN SZ_64K -struct btrfs_io_geometry { -/* remaining bytes before crossing a stripe */ -u64 len; -/* offset of logical address in chunk */ -u64 offset; -/* length of single IO stripe */ -u64 stripe_len; -/* number of stripe where address falls */ -u64 stripe_nr; -/* offset of address in stripe */ -u64 stripe_offset; -/* offset of raid56 stripe into the chunk */ -u64 raid56_stripe_offset; -}; -/* * Use sequence counter to get consistent device stat data on * 32-bit processors. @@ -520,9 +505,8 @@ int btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,int btrfs_map_sblock (struct btrfs_fs_info *fs_info,enum btrfs_map_op op,u64 logical,u64 *length,struct btrfs_io_context **bioc_ret); -int btrfs_get_io_geometry (struct btrfs_fs_info *fs_info,struct extent_map *map, -enum btrfs_map_op op,u64 logical, -struct btrfs_io_geometry *io_geom); +struct block_device *btrfs_get_stripe_info (struct btrfs_fs_info *fs_info, +enum btrfs_map_op op,u64 logical,u64 length,u64 *lenp);int btrfs_read_sys_array (struct btrfs_fs_info *fs_info);int btrfs_read_chunk_tree (struct btrfs_fs_info *fs_info); struct btrfs_block_group *btrfs_create_chunk(struct btrfs_trans_handle *trans,
PATCH 38/40 btrfs: return a blk_status_t from btrfs_repair_one_sector
In reply to Christoph HellwigThis is what the submit hook returns and what the callers want anyway.
--- fs/btrfs/extent_io.c | 14 +++++++------- fs/btrfs/extent_io.h | 2 +- fs/btrfs/inode.c | 4 ++-- 3 files changed, 10 insertions(+), 10 deletions(-)
fs/btrfs/extent_io.c +7 -7
@@ -2626,7 +2626,7 @@ static bool btrfs_check_repairable(struct inode *inode,return true; } -int btrfs_repair_one_sector(struct inode *inode, +blk_status_t btrfs_repair_one_sector(struct inode *inode, struct bio *failed_bio, u32 bio_offset, struct page *page, unsigned int pgoff, u64 start, int failed_mirror,
@@ -2649,12 +2649,12 @@ int btrfs_repair_one_sector(struct inode *inode, failrec =btrfs_get_io_failure_record (inode, start);if (IS_ERR (failrec)) -return PTR_ERR (failrec); +return errno_to_blk_status (PTR_ERR (failrec));if (!btrfs_check_repairable (inode, failrec, failed_mirror)) {free_io_failure (failure_tree, tree, failrec); -return -EIO ; +return BLK_STS_IOERR ; } repair_bio =btrfs_bio_alloc (inode,1 ,REQ_OP_READ );
@@ -2685,7 +2685,7 @@ int btrfs_repair_one_sector(struct inode *inode,free_io_failure (failure_tree, tree, failrec);bio_put (repair_bio); } -return blk_status_to_errno (status); +return status; }static void end_page_read (struct page *page,bool uptodate,u64 start,u32 len)
@@ -2725,7 +2725,7 @@ static blk_status_t submit_read_repair(struct inode *inode,struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);const u32 sectorsize = fs_info->sectorsize;const int nr_bits = (end +1 - start) >> fs_info->sectorsize_bits; -int error =0 ; +blk_status_t error =BLK_STS_OK ;int i;BUG_ON (bio_op (failed_bio) ==REQ_OP_WRITE );
@@ -2744,7 +2744,7 @@ static blk_status_t submit_read_repair(struct inode *inode,const unsigned int offset = i * sectorsize;struct extent_state *cached =NULL ;bool uptodate = false; -int ret; +blk_status_t ret;if (!(error_bitmap & (1U << i))) {/*
@@ -2786,7 +2786,7 @@ static blk_status_t submit_read_repair(struct inode *inode, start + offset + sectorsize -1 , &cached); } -return errno_to_blk_status (error); +return error; }/* lots and lots of room for performance fixes in the end_bio funcs */
fs/btrfs/extent_io.h +1 -1
@@ -303,7 +303,7 @@ struct io_failure_record {int failed_mirror; }; -int btrfs_repair_one_sector(struct inode *inode, +blk_status_t btrfs_repair_one_sector(struct inode *inode, struct bio *failed_bio, u32 bio_offset, struct page *page, unsigned int pgoff, u64 start, int failed_mirror,
fs/btrfs/inode.c +2 -2
@@ -7794,14 +7794,14 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip,btrfs_ino (BTRFS_I (inode)), pgoff); }else { -int ret; +blk_status_t ret; ret =btrfs_repair_one_sector (inode, &bbio->bio, bio_offset, bvec.bv_page, pgoff, start, bbio->mirror_num, submit_dio_repair_bio);if (ret) - err =errno_to_blk_status (ret); + err = ret; }ASSERT (bio_offset + sectorsize > bio_offset); bio_offset += sectorsize;
Signature · 1 line
2.30.2
PATCH 39/40 btrfs: pass private data end end_io handler to btrfs_repair_one_sector
In reply to Christoph HellwigAllow the caller to control what happens when the repair bio completes. This will be needed streamline the direct I/O path.
--- fs/btrfs/extent_io.c | 15 ++++++++------- fs/btrfs/extent_io.h | 8 ++++---- fs/btrfs/inode.c | 4 +++- 3 files changed, 15 insertions(+), 12 deletions(-)
fs/btrfs/extent_io.c +8 -7
@@ -2627,10 +2627,10 @@ static bool btrfs_check_repairable(struct inode *inode, }blk_status_t btrfs_repair_one_sector (struct inode *inode, -struct bio *failed_bio,u32 bio_offset, -struct page *page,unsigned int pgoff, -u64 start,int failed_mirror, -submit_bio_hook_t *submit_bio_hook) +struct bio *failed_bio,u32 bio_offset,struct page *page, +unsigned int pgoff,u64 start,int failed_mirror, +submit_bio_hook_t *submit_bio_hook, +void *bi_private,void (*bi_end_io)(struct bio *bio)) {struct io_failure_record *failrec;struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);
@@ -2660,9 +2660,9 @@ blk_status_t btrfs_repair_one_sector(struct inode *inode, repair_bio =btrfs_bio_alloc (inode,1 ,REQ_OP_READ ); repair_bbio =btrfs_bio (repair_bio); repair_bbio->file_offset = start; - repair_bio->bi_end_io = failed_bio->bi_end_io; repair_bio->bi_iter.bi_sector = failrec->logical >>9 ; - repair_bio->bi_private = failed_bio->bi_private; + repair_bio->bi_private = bi_private; + repair_bio->bi_end_io = bi_end_io;if (failed_bbio->csum) {const u32 csum_size = fs_info->csum_size;
@@ -2758,7 +2758,8 @@ static blk_status_t submit_read_repair(struct inode *inode, ret =btrfs_repair_one_sector (inode, failed_bio, bio_offset + offset, page, pgoff + offset, start + offset, - failed_mirror, btrfs_submit_data_bio); + failed_mirror, btrfs_submit_data_bio, + failed_bio->bi_private, failed_bio->bi_end_io);if (!ret) {/* * We have submitted the read repair, the page release
fs/btrfs/extent_io.h +4 -4
@@ -304,10 +304,10 @@ struct io_failure_record { };blk_status_t btrfs_repair_one_sector (struct inode *inode, -struct bio *failed_bio,u32 bio_offset, -struct page *page,unsigned int pgoff, -u64 start,int failed_mirror, -submit_bio_hook_t *submit_bio_hook); +struct bio *failed_bio,u32 bio_offset,struct page *page, +unsigned int pgoff,u64 start,int failed_mirror, +submit_bio_hook_t *submit_bio_hook, +void *bi_private,void (*bi_end_io)(struct bio *bio));#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS bool find_lock_delalloc_range(struct inode *inode,
fs/btrfs/inode.c +3 -1
@@ -7799,7 +7799,9 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, ret =btrfs_repair_one_sector (inode, &bbio->bio, bio_offset, bvec.bv_page, pgoff, start, bbio->mirror_num, - submit_dio_repair_bio); + submit_dio_repair_bio, + bbio->bio.bi_private, + bbio->bio.bi_end_io);if (ret) err = ret; }
Signature · 1 line
2.30.2
Re: [PATCH 39/40] btrfs: pass private data end end_io handler to btrfs_repair_one_sector
Quoted 16 lines · Christoph Hellwig
Allow the caller to control what happens when the repair bio completes. This will be needed streamline the direct I/O path.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/btrfs/extent_io.c | 15 ++++++++------- fs/btrfs/extent_io.h | 8 ++++---- fs/btrfs/inode.c | 4 +++- 3 files changed, 15 insertions(+), 12 deletions(-)fs/btrfs/extent_io.c +0
-0@@ -2627,10 +2627,10 @@ static bool btrfs_check_repairable(struct inode *inode, }
fs/btrfs/extent_io.c static bool btrfs_check_repairable(struct inode *inode, · line 2633
Quoted 10 lines
fs/btrfs/extent_io.c +4
-4blk_status_t btrfs_repair_one_sector (struct inode *inode, -struct bio *failed_bio,u32 bio_offset, -struct page *page,unsigned int pgoff, -u64 start,int failed_mirror, -submit_bio_hook_t *submit_bio_hook) +struct bio *failed_bio,u32 bio_offset,struct page *page, +unsigned int pgoff,u64 start,int failed_mirror, +submit_bio_hook_t *submit_bio_hook, +void *bi_private,void (*bi_end_io)(struct bio *bio))
Not a big fan of extra parameters for a function which already had enough...
And I always have a question on repair (aka read from extra copy).
Can't we just make the repair part to be synchronous? Instead of putting everything into another endio call back, and wait for the read and re-check in the same context.
That would definitely streamline the workload way more than this.
And I don't think users would complain about btrfs is slow on reading when correcting the corrupted data.
Thanks,
Qu
Quoted 59 lines
fs/btrfs/extent_io.c +4
-3{struct io_failure_record *failrec;struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);@@ -2660,9 +2660,9 @@ blk_status_t btrfs_repair_one_sector(struct inode *inode, repair_bio =btrfs_bio_alloc (inode,1 ,REQ_OP_READ ); repair_bbio =btrfs_bio (repair_bio); repair_bbio->file_offset = start; - repair_bio->bi_end_io = failed_bio->bi_end_io; repair_bio->bi_iter.bi_sector = failrec->logical >>9 ; - repair_bio->bi_private = failed_bio->bi_private; + repair_bio->bi_private = bi_private; + repair_bio->bi_end_io = bi_end_io;if (failed_bbio->csum) {const u32 csum_size = fs_info->csum_size;@@ -2758,7 +2758,8 @@ static blk_status_t submit_read_repair(struct inode *inode, ret =btrfs_repair_one_sector (inode, failed_bio, bio_offset + offset, page, pgoff + offset, start + offset, - failed_mirror, btrfs_submit_data_bio); + failed_mirror, btrfs_submit_data_bio, + failed_bio->bi_private, failed_bio->bi_end_io);if (!ret) {/* * We have submitted the read repair, the page release fs/btrfs/extent_io.h +4
-4@@ -304,10 +304,10 @@ struct io_failure_record { };blk_status_t btrfs_repair_one_sector (struct inode *inode, -struct bio *failed_bio,u32 bio_offset, -struct page *page,unsigned int pgoff, -u64 start,int failed_mirror, -submit_bio_hook_t *submit_bio_hook); +struct bio *failed_bio,u32 bio_offset,struct page *page, +unsigned int pgoff,u64 start,int failed_mirror, +submit_bio_hook_t *submit_bio_hook, +void *bi_private,void (*bi_end_io)(struct bio *bio));#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS bool find_lock_delalloc_range(struct inode *inode,fs/btrfs/inode.c +3
-1@@ -7799,7 +7799,9 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, ret =btrfs_repair_one_sector (inode, &bbio->bio, bio_offset, bvec.bv_page, pgoff, start, bbio->mirror_num, - submit_dio_repair_bio); + submit_dio_repair_bio, + bbio->bio.bi_private, + bbio->bio.bi_end_io);if (ret) err = ret; }
Re: [PATCH 39/40] btrfs: pass private data end end_io handler to btrfs_repair_one_sector
Quoted 5 lines · Qu Wenruo
Not a big fan of extra parameters for a function which already had enough...
And I always have a question on repair (aka read from extra copy).
Can't we just make the repair part to be synchronous?
I'll defer that to people who know the btrfs code better. This basically means we will block for a long time in the end_io workqueue, which could have adverse consequences.
It would howerver simply a lot of things in a very nice way.
Re: [PATCH 39/40] btrfs: pass private data end end_io handler to btrfs_repair_one_sector
In reply to Christoph Hellwigfs/btrfs/extent_io.c static bool btrfs_check_repairable(struct inode *inode, · line 2633
Quoted 2 lines
fs/btrfs/extent_io.c +1
-0+ void *bi_private, void (*bi_end_io)(struct bio *bio))
Maybe the last parameter can just be "bio_end_io_t bi_end_io"?
PATCH 40/40 btrfs: use the iomap direct I/O bio directly
In reply to Christoph HellwigMake the iomap code allocate btrfs dios by setting the bio_set field, and then feed these directly into btrfs_map_dio.
For this to work iomap_begin needs to report a range that only contains a single chunk, and thus is changed to a two level iteration.
This needs another extra field in struct btrfs_dio. We culd overlay it with other fields not used after I/O submittion, or split out a new btrfs_dio_bio for the file_offset, iter and repair_refs, but compared to the overall saving of the series this is a minor detail.
The per-iomap csum lookup is gone for now as well. At least for small I/Os this just creates a lot of overhead, but for large I/O we could look into optimizing this in one for or another, but I'd love to see a reproducer where it actually matters first. With the state as of this patch the direct I/O bio submission is so close to the buffered one that they could be unified with very little work, so diverging again would be a bit counterproductive. OTOH if the optimization is indeed very useful we should do it in a way that also works for buffered reads.
--- fs/btrfs/btrfs_inode.h | 25 --- fs/btrfs/ctree.h | 1 - fs/btrfs/extent_io.c | 22 +- fs/btrfs/extent_io.h | 4 +- fs/btrfs/inode.c | 451 ++++++++++++++++------------------------- fs/btrfs/volumes.h | 1 + 6 files changed, 184 insertions(+), 320 deletions(-)
fs/btrfs/btrfs_inode.h +0 -25
@@ -346,31 +346,6 @@ static inline bool btrfs_inode_in_log(struct btrfs_inode *inode, u64 generation)return ret; } -struct btrfs_dio_private { -struct inode *inode; - -/* -* Since DIO can use anonymous page, we cannot use page_offset() to -* grab the file offset, thus need a dedicated member for file offset. -*/ -u64 file_offset; -u64 disk_bytenr; -/* Used for bio::bi_size */ -u32 bytes; - -/* -* References to this structure. There is one reference per in-flight -* bio plus one while we're still setting up. -*/ -refcount_t refs; - -/* dio_bio came from fs/direct-io.c */ -struct bio *dio_bio; - -/* Array of checksums */ -u8 csums[]; -}; -/* * btrfs_inode_item stores flags in a u64, btrfs_inode stores them in two * separate u32s. These two functions convert between the two representations.
fs/btrfs/ctree.h +0 -1
@@ -3136,7 +3136,6 @@ int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,int btrfs_find_orphan_item (struct btrfs_root *root,u64 offset);/* file-item.c */ -struct btrfs_dio_private ;int btrfs_del_csums (struct btrfs_trans_handle *trans,struct btrfs_root *root,u64 bytenr,u64 len);blk_status_t btrfs_lookup_bio_sums (struct inode *inode,struct bio *bio,u8 *dst);
fs/btrfs/extent_io.c +1 -21
@@ -31,7 +31,7 @@static struct kmem_cache *extent_state_cache;static struct kmem_cache *extent_buffer_cache; -static struct bio_set btrfs_bioset; +struct bio_set btrfs_bioset;static inline bool extent_state_in_tree (const struct extent_state *state) {
@@ -3150,26 +3150,6 @@ struct bio *btrfs_bio_alloc(struct inode *inode, unsigned int nr_iovecs,return bio; } -struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, -u64 offset,u64 size) -{ -struct bio *bio; -struct btrfs_bio *bbio; - -ASSERT (offset <=UINT_MAX && size <=UINT_MAX ); - -/* this will never fail when it's backed by a bioset */ - bio =bio_alloc_clone (orig->bi_bdev, orig,GFP_NOFS , &btrfs_bioset); -ASSERT (bio); - - bbio =btrfs_bio (bio); -btrfs_bio_init (btrfs_bio (bio), inode); - -bio_trim (bio, offset >>9 , size >>9 ); - bbio->iter = bio->bi_iter; -return bio; -} -/** * Attempt to add a page to bio *
fs/btrfs/extent_io.h +2 -2
@@ -279,8 +279,6 @@ void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end, u32 bits_to_clear, unsigned long page_ops);struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs,unsigned int opf); -struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, -u64 offset,u64 size);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num);
@@ -323,4 +321,6 @@ void btrfs_extent_buffer_leak_debug_check(struct btrfs_fs_info *fs_info);#define btrfs_extent_buffer_leak_debug_check (fs_info) do {} while (0)#endif +extern struct bio_set btrfs_bioset; +#endif
fs/btrfs/inode.c +180 -271
609 lines not on this page · raw message
fs/btrfs/volumes.h +1 -0
@@ -341,6 +341,7 @@ struct btrfs_bio {/* for direct I/O */ u64 file_offset; +atomic_t repair_refs;/* @device is for stripe IO submission. */ struct btrfs_device *device;
Signature · 1 line
2.30.2
Re: [PATCH 40/40] btrfs: use the iomap direct I/O bio directly
Quoted 21 lines · Christoph Hellwig
Make the iomap code allocate btrfs dios by setting the
bio_setfield, and then feed these directly intobtrfs_map_dio.For this to work
iomap_beginneeds to report a range that only contains a single chunk, and thus is changed to a two level iteration.This needs another extra field in struct
btrfs_dio. We culd overlay it with other fields not used after I/O submittion, or split out a newbtrfs_dio_biofor thefile_offset, iter andrepair_refs, but compared to the overall saving of the series this is a minor detail.The per-iomap csum lookup is gone for now as well. At least for small I/Os this just creates a lot of overhead, but for large I/O we could look into optimizing this in one for or another, but I'd love to see a reproducer where it actually matters first. With the state as of this patch the direct I/O bio submission is so close to the buffered one that they could be unified with very little work, so diverging again would be a bit counterproductive. OTOH if the optimization is indeed very useful we should do it in a way that also works for buffered reads.
Quoted 1 line
Signed-off-by Christoph Hellwig <[email protected]>
Not familar with iomap thus I can be totally wrong, but isn't the idea of iomap to separate more code from fs?
My (mostly poor) understanding of iomap is, iomap just crafting a bio (with the help from fs callbacks).
For btrfs or stacked drivers, they can do whatever to split/clone the bio, as long as the original bio can be fullfiled, how btrfs/stacked drivers do the work is their own business.
I'm really not sure if it's a good idea to expose btrfs internal bio_set just for iomap.
Personally speaking I didn't see much problem of cloning an iomap bio, it only causes extra memory of btrfs_bio, which is pretty small previously.
Or did I miss something specific to iomap?
Thanks,
Qu
Quoted 752 lines
--- fs/btrfs/btrfs_inode.h | 25 --- fs/btrfs/ctree.h | 1 - fs/btrfs/extent_io.c | 22 +- fs/btrfs/extent_io.h | 4 +- fs/btrfs/inode.c | 451 ++++++++++++++++------------------------- fs/btrfs/volumes.h | 1 + 6 files changed, 184 insertions(+), 320 deletions(-)fs/btrfs/btrfs_inode.h +0
-25@@ -346,31 +346,6 @@ static inline bool btrfs_inode_in_log(struct btrfs_inode *inode, u64 generation)return ret; } -struct btrfs_dio_private { -struct inode *inode; - -/* -* Since DIO can use anonymous page, we cannot use page_offset() to -* grab the file offset, thus need a dedicated member for file offset. -*/ -u64 file_offset; -u64 disk_bytenr; -/* Used for bio::bi_size */ -u32 bytes; - -/* -* References to this structure. There is one reference per in-flight -* bio plus one while we're still setting up. -*/ -refcount_t refs; - -/* dio_bio came from fs/direct-io.c */ -struct bio *dio_bio; - -/* Array of checksums */ -u8 csums[]; -}; -/* * btrfs_inode_item stores flags in a u64, btrfs_inode stores them in two * separate u32s. These two functions convert between the two representations. fs/btrfs/ctree.h +0
-1@@ -3136,7 +3136,6 @@ int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,int btrfs_find_orphan_item (struct btrfs_root *root,u64 offset);/* file-item.c */ -struct btrfs_dio_private ;int btrfs_del_csums (struct btrfs_trans_handle *trans,struct btrfs_root *root,u64 bytenr,u64 len);blk_status_t btrfs_lookup_bio_sums (struct inode *inode,struct bio *bio,u8 *dst);fs/btrfs/extent_io.c +1
-21@@ -31,7 +31,7 @@static struct kmem_cache *extent_state_cache;static struct kmem_cache *extent_buffer_cache; -static struct bio_set btrfs_bioset; +struct bio_set btrfs_bioset;static inline bool extent_state_in_tree (const struct extent_state *state) {@@ -3150,26 +3150,6 @@ struct bio *btrfs_bio_alloc(struct inode *inode, unsigned int nr_iovecs,return bio; } -struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, -u64 offset,u64 size) -{ -struct bio *bio; -struct btrfs_bio *bbio; - -ASSERT (offset <=UINT_MAX && size <=UINT_MAX ); - -/* this will never fail when it's backed by a bioset */ - bio =bio_alloc_clone (orig->bi_bdev, orig,GFP_NOFS , &btrfs_bioset); -ASSERT (bio); - - bbio =btrfs_bio (bio); -btrfs_bio_init (btrfs_bio (bio), inode); - -bio_trim (bio, offset >>9 , size >>9 ); - bbio->iter = bio->bi_iter; -return bio; -} -/** * Attempt to add a page to bio * fs/btrfs/extent_io.h +2
-2@@ -279,8 +279,6 @@ void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end, u32 bits_to_clear, unsigned long page_ops);struct bio *btrfs_bio_alloc (struct inode *inode,unsigned int nr_iovecs,unsigned int opf); -struct bio *btrfs_bio_clone_partial (struct inode *inode,struct bio *orig, -u64 offset,u64 size);void end_extent_writepage (struct page *page,int err,u64 start,u64 end);int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num);@@ -323,4 +321,6 @@ void btrfs_extent_buffer_leak_debug_check(struct btrfs_fs_info *fs_info);#define btrfs_extent_buffer_leak_debug_check (fs_info) do {} while (0)#endif +extern struct bio_set btrfs_bioset; +#endif fs/btrfs/inode.c +180
-271@@ -62,8 +62,8 @@ struct btrfs_iget_args { };struct btrfs_dio_data { -ssize_t submitted;struct extent_changeset *data_reserved; +struct iomap extent; };static const struct inode_operations btrfs_dir_inode_operations;@@ -7507,16 +7507,16 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,return ret; } -static int btrfs_dio_iomap_begin (struct iomap_iter *iter) +static int btrfs_dio_iomap_begin_extent (struct iomap_iter *iter) {struct inode *inode = iter->inode;struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);loff_t start = iter->pos;loff_t length = iter->len; -struct iomap *iomap = &iter->iomap;struct extent_map *em;struct extent_state *cached_state =NULL ;struct btrfs_dio_data *dio_data = iter->private; +struct iomap *iomap = &dio_data->extent;u64 lockstart, lockend;bool write = (iter->flags &IOMAP_WRITE );int ret =0 ;@@ -7543,7 +7543,6 @@ static int btrfs_dio_iomap_begin(struct iomap_iter *iter)return ret; } - dio_data->submitted =0 ; dio_data->data_reserved =NULL ;/* @@ -7647,14 +7646,12 @@ static int btrfs_dio_iomap_begin(struct iomap_iter *iter) iomap->type =IOMAP_MAPPED ; } iomap->offset = start; - iomap->bdev = fs_info->fs_devices->latest_dev->bdev; iomap->length = len;if (write &&btrfs_use_zone_append (BTRFS_I (inode), em->block_start)) iomap->flags |=IOMAP_F_ZONE_APPEND ;free_extent_map (em); -return 0 ; unlock_err:@@ -7663,53 +7660,95 @@ static int btrfs_dio_iomap_begin(struct iomap_iter *iter)return ret; } -static int btrfs_dio_iomap_end (struct iomap_iter *iter) +static void btrfs_dio_unlock_remaining_extent (struct btrfs_inode *bi, +u64 pos,u64 len,u64 processed,bool write) { -struct btrfs_dio_data *dio_data = iter->private; +if (write) +__endio_write_update_ordered (bi, pos + processed, + len - processed, false); +else +unlock_extent (&bi->io_tree, pos + processed, + pos + len -1 ); +} + +static int btrfs_dio_iomap_begin_chunk (struct iomap_iter *iter) +{ +struct btrfs_fs_info *fs_info =btrfs_sb (iter->inode->i_sb);struct btrfs_inode *bi =BTRFS_I (iter->inode); -bool write = (iter->flags &IOMAP_WRITE ); -loff_t length =iomap_length (iter); -loff_t pos = iter->pos; -int ret =0 ; +struct btrfs_dio_data *dio_data = iter->private; +struct block_device *bdev; +u64 len; + + iter->iomap = dio_data->extent; -if (!write && iter->iomap.type ==IOMAP_HOLE ) { -/* If reading from a hole, unlock and return */ -unlock_extent (&bi->io_tree, pos, pos + length -1 ); +if (dio_data->extent.type !=IOMAP_MAPPED )return 0 ; + + bdev =btrfs_get_stripe_info (fs_info, (iter->flags &IOMAP_WRITE ) ? +BTRFS_MAP_WRITE :BTRFS_MAP_READ , + iter->iomap.addr, iter->iomap.length, &len); +if (WARN_ON_ONCE (IS_ERR (bdev))) { +btrfs_dio_unlock_remaining_extent (bi, dio_data->extent.offset, + dio_data->extent.length,0 , + iter->flags &IOMAP_WRITE ); +return PTR_ERR (bdev); } -if (dio_data->submitted < length) { - pos += dio_data->submitted; - length -= dio_data->submitted; -if (write) -__endio_write_update_ordered (bi, pos, length, false); -else -unlock_extent (&bi->io_tree, pos, pos + length -1 ); - ret = -ENOTBLK ; + iter->iomap.bdev = bdev; + iter->iomap.length =min (iter->iomap.length, len); +return 0 ; +} + +static bool btrfs_dio_iomap_end (struct iomap_iter *iter) +{ +struct btrfs_inode *bi =BTRFS_I (iter->inode); +struct btrfs_dio_data *dio_data = iter->private; +struct iomap *extent = &dio_data->extent; +loff_t processed =iomap_processed (iter); +loff_t length =iomap_length (iter); + +if (iter->iomap.type ==IOMAP_HOLE ) { +ASSERT (!(iter->flags &IOMAP_WRITE )); + +/* If reading from a hole, unlock the whole range here */ +unlock_extent (&bi->io_tree, iter->pos, iter->pos + length -1 ); + }else if (processed < length) { +btrfs_dio_unlock_remaining_extent (bi, extent->offset, + extent->length, processed, + iter->flags &IOMAP_WRITE ); + }else if (iter->pos + processed < extent->offset + extent->length) { + extent->offset += processed; + extent->addr += processed; + extent->length -= processed; +return true; } -if (write) +if (iter->flags &IOMAP_WRITE )extent_changeset_free (dio_data->data_reserved); -return ret; +return false; }static int btrfs_dio_iomap_iter (struct iomap_iter *iter) { +bool keep_extent = false;int ret; -if (iter->iomap.length) { - ret =btrfs_dio_iomap_end (iter); -if (ret <0 && !iter->processed) -return ret; - } +if (iter->iomap.length) + keep_extent =btrfs_dio_iomap_end (iter); ret =iomap_iter_advance (iter);if (ret <=0 )return ret; - ret =btrfs_dio_iomap_begin (iter); -if (ret <0 ) +if (!keep_extent) { + ret =btrfs_dio_iomap_begin_extent (iter); +if (ret <0 ) +return ret; + } + ret =btrfs_dio_iomap_begin_chunk (iter); +if (ret <0 )return ret; +iomap_iter_done (iter);return 1 ; }@@ -7718,54 +7757,40 @@ static const struct iomap_ops btrfs_dio_iomap_ops = { .iomap_iter = btrfs_dio_iomap_iter, }; -static void btrfs_dio_private_put (struct btrfs_dio_private *dip) +static void btrfs_end_read_dio_bio (struct btrfs_bio *bbio, +struct btrfs_bio *main_bbio); + +static void btrfs_dio_repair_end_io (struct bio *bio) { -/* -* This implies a barrier so that stores to dio_bio->bi_status before -* this and loads of dio_bio->bi_status after this are fully ordered. -*/ -if (!refcount_dec_and_test (&dip->refs)) -return ; +struct btrfs_bio *bbio =btrfs_bio (bio); +struct btrfs_inode *bi =BTRFS_I (bbio->inode); +struct btrfs_bio *failed_bbio = bio->bi_private; -if (btrfs_op (dip->dio_bio) ==BTRFS_MAP_WRITE ) { -__endio_write_update_ordered (BTRFS_I (dip->inode), - dip->file_offset, - dip->bytes, - !dip->dio_bio->bi_status); - }else { -unlock_extent (&BTRFS_I (dip->inode)->io_tree, - dip->file_offset, - dip->file_offset + dip->bytes -1 ); +if (bio->bi_status) { +btrfs_warn (bi->root->fs_info, +"direct IO failed ino %llu rw %d,%u sector %#Lx len %u err no %d" , +btrfs_ino (bi),bio_op (bio), bio->bi_opf, + bio->bi_iter.bi_sector, bio->bi_iter.bi_size, + bio->bi_status); } +btrfs_end_read_dio_bio (bbio, failed_bbio); -bio_endio (dip->dio_bio); -kfree (dip); +bio_put (bio); }static blk_status_t submit_dio_repair_bio (struct inode *inode,struct bio *bio,int mirror_num,unsigned long bio_flags) { -struct btrfs_dio_private *dip = bio->bi_private; -struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); -blk_status_t ret; -BUG_ON (bio_op (bio) ==REQ_OP_WRITE ); -btrfs_bio (bio)->end_io_type =BTRFS_ENDIO_WQ_DATA_WRITE ; - -refcount_inc (&dip->refs); - ret =btrfs_map_bio (fs_info, bio, mirror_num); -if (ret) -refcount_dec (&dip->refs); -return ret; +return btrfs_map_bio (btrfs_sb (inode->i_sb), bio, mirror_num); } -static blk_status_t btrfs_check_read_dio_bio (struct btrfs_dio_private *dip, -struct btrfs_bio *bbio, -const bool uptodate) +static void btrfs_end_read_dio_bio (struct btrfs_bio *this_bbio, +struct btrfs_bio *main_bbio) { -struct inode *inode = dip->inode; +struct inode *inode = main_bbio->inode;struct btrfs_fs_info *fs_info =BTRFS_I (inode)->root->fs_info;const u32 sectorsize = fs_info->sectorsize;struct extent_io_tree *failure_tree = &BTRFS_I (inode)->io_failure_tree;@@ -7773,20 +7798,22 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip,const bool csum = !(BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM );struct bio_vec bvec;struct bvec_iter iter; +bool uptodate = !this_bbio->bio.bi_status;u32 bio_offset =0 ; -blk_status_t err =BLK_STS_OK ; -__bio_for_each_segment (bvec, &bbio->bio, iter, bbio->iter) { + main_bbio->bio.bi_status =BLK_STS_OK ; + +__bio_for_each_segment (bvec, &this_bbio->bio, iter, this_bbio->iter) {unsigned int i, nr_sectors, pgoff; nr_sectors =BTRFS_BYTES_TO_BLKS (fs_info, bvec.bv_len); pgoff = bvec.bv_offset;for (i =0 ; i < nr_sectors; i++) { -u64 start = bbio->file_offset + bio_offset; +u64 start = this_bbio->file_offset + bio_offset;ASSERT (pgoff <PAGE_SIZE );if (uptodate && - (!csum || !check_data_csum (inode, bbio, + (!csum || !check_data_csum (inode, this_bbio, bio_offset, bvec.bv_page, pgoff, start))) { clean_io_failure(fs_info, failure_tree, io_tree,@@ -7796,21 +7823,56 @@ static blk_status_t btrfs_check_read_dio_bio(struct btrfs_dio_private *dip, }else {blk_status_t ret; - ret =btrfs_repair_one_sector (inode, &bbio->bio, - bio_offset, bvec.bv_page, pgoff, - start, bbio->mirror_num, +atomic_inc (&main_bbio->repair_refs); + ret =btrfs_repair_one_sector (inode, + &this_bbio->bio, bio_offset, + bvec.bv_page, pgoff, start, + this_bbio->mirror_num, submit_dio_repair_bio, - bbio->bio.bi_private, - bbio->bio.bi_end_io); -if (ret) - err = ret; + main_bbio, + btrfs_dio_repair_end_io); +if (ret) { + main_bbio->bio.bi_status = ret; +atomic_dec (&main_bbio->repair_refs); + } }ASSERT (bio_offset + sectorsize > bio_offset); bio_offset += sectorsize; pgoff += sectorsize; } } -return err; + +if (atomic_dec_and_test (&main_bbio->repair_refs)) { +unlock_extent (&BTRFS_I (inode)->io_tree, main_bbio->file_offset, + main_bbio->file_offset + main_bbio->iter.bi_size -1 ); +iomap_dio_bio_end_io (&main_bbio->bio); + } +} + +static void btrfs_dio_bio_end_io (struct bio *bio) +{ +struct btrfs_bio *bbio =btrfs_bio (bio); +struct btrfs_inode *bi =BTRFS_I (bbio->inode); + +if (bio->bi_status) { +btrfs_warn (bi->root->fs_info, +"direct IO failed ino %llu rw %d,%u sector %#Lx len %u err no %d" , +btrfs_ino (bi),bio_op (bio), bio->bi_opf, + bio->bi_iter.bi_sector, bio->bi_iter.bi_size, + bio->bi_status); + } + +if (bio_op (bio) ==REQ_OP_READ ) { +atomic_set (&bbio->repair_refs,1 ); +btrfs_end_read_dio_bio (bbio, bbio); + }else { +btrfs_record_physical_zoned (bbio->inode, bbio->file_offset, + bio); +__endio_write_update_ordered (bi, bbio->file_offset, + bbio->iter.bi_size, + !bio->bi_status); +iomap_dio_bio_end_io (bio); + } } static void __endio_write_update_ordered(struct btrfs_inode *inode,@@ -7829,47 +7891,47 @@ static void btrfs_submit_bio_start_direct_io(struct btrfs_work *work) &bbio->bio, bbio->file_offset, 1); } -static void btrfs_end_dio_bio (struct bio *bio) +/* +* If we are submitting more than one bio, submit them all asynchronously. The +* exception is RAID 5 or 6, as asynchronous checksums make it difficult to +* collect full stripe writes. +*/ +static bool btrfs_dio_allow_async_write (struct btrfs_fs_info *fs_info, +struct btrfs_inode *bi) { -struct btrfs_dio_private *dip = bio->bi_private; -struct btrfs_bio *bbio =btrfs_bio (bio); -blk_status_t err = bio->bi_status; - -if (err) -btrfs_warn (BTRFS_I (dip->inode)->root->fs_info, -"direct IO failed ino %llu rw %d,%u sector %#Lx len %u err no %d" , -btrfs_ino (BTRFS_I (dip->inode)),bio_op (bio), - bio->bi_opf, bio->bi_iter.bi_sector, - bio->bi_iter.bi_size, err); - -if (bio_op (bio) ==REQ_OP_READ ) - err =btrfs_check_read_dio_bio (dip, bbio, !err); - -if (err) - dip->dio_bio->bi_status = err; - -btrfs_record_physical_zoned (dip->inode, bbio->file_offset, bio); - -bio_put (bio); -btrfs_dio_private_put (dip); +if (btrfs_data_alloc_profile (fs_info) &BTRFS_BLOCK_GROUP_RAID56_MASK ) +return false; +if (atomic_read (&bi->sync_writers)) +return false; +return true; } -static inline blk_status_t btrfs_submit_dio_bio (struct bio *bio, -struct inode *inode,u64 file_offset,int async_submit) +static void btrfs_dio_submit_io (const struct iomap_iter *iter, +struct bio *bio,loff_t file_offset,bool more) { -struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); -struct btrfs_inode *bi =BTRFS_I (inode); -struct btrfs_dio_private *dip = bio->bi_private; +struct btrfs_fs_info *fs_info =btrfs_sb (iter->inode->i_sb); +struct btrfs_inode *bi =BTRFS_I (iter->inode);struct btrfs_bio *bbio =btrfs_bio (bio);blk_status_t ret; +memset (bbio,0 , offsetof(struct btrfs_bio , bio)); + bbio->inode = iter->inode; + bbio->file_offset = file_offset; + bbio->iter = bio->bi_iter; + bio->bi_end_io = btrfs_dio_bio_end_io; +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { + ret =extract_ordered_extent (bi, bio, file_offset); +if (ret) + goto out_err; + } +if (!(bi->flags &BTRFS_INODE_NODATASUM )) { -/* See btrfs_submit_data_bio for async submit rules */ -if (async_submit && !atomic_read (&bi->sync_writers)) { +if (more &&btrfs_dio_allow_async_write (fs_info, bi)) {btrfs_submit_bio_async (bbio, btrfs_submit_bio_start_direct_io); -return BLK_STS_OK ; +return ; }/* @@ -7878,189 +7940,36 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio,*/ ret =btrfs_csum_one_bio (bi, bio, file_offset,1 );if (ret) -return ret; + goto out_err; } }else { bbio->end_io_type =BTRFS_ENDIO_WQ_DATA_READ ; -if (!(bi->flags &BTRFS_INODE_NODATASUM )) { -u64 csum_offset; - - csum_offset = file_offset - dip->file_offset; - csum_offset >>= fs_info->sectorsize_bits; - csum_offset *= fs_info->csum_size; -btrfs_bio (bio)->csum = dip->csums + csum_offset; - } - } - -return btrfs_map_bio (fs_info, bio,0 ); -} - -/* -* If this succeeds, the btrfs_dio_private is responsible for cleaning up locked -* or ordered extents whether or not we submit any bios. -*/ -static struct btrfs_dio_private *btrfs_create_dio_private (struct bio *dio_bio, -struct inode *inode, -loff_t file_offset) -{ -const bool write = (btrfs_op (dio_bio) ==BTRFS_MAP_WRITE ); -const bool csum = !(BTRFS_I (inode)->flags &BTRFS_INODE_NODATASUM ); -size_t dip_size; -struct btrfs_dio_private *dip; - - dip_size =sizeof (*dip); -if (!write && csum) { -struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); -size_t nblocks; - - nblocks = dio_bio->bi_iter.bi_size >> fs_info->sectorsize_bits; - dip_size += fs_info->csum_size * nblocks; - } - - dip =kzalloc (dip_size,GFP_NOFS ); -if (!dip) -return NULL ; - - dip->inode = inode; - dip->file_offset = file_offset; - dip->bytes = dio_bio->bi_iter.bi_size; - dip->disk_bytenr = dio_bio->bi_iter.bi_sector <<9 ; - dip->dio_bio = dio_bio; -refcount_set (&dip->refs,1 ); -return dip; -} - -static void btrfs_submit_direct (const struct iomap_iter *iter, -struct bio *dio_bio,loff_t file_offset,bool more) -{ -struct inode *inode = iter->inode; -const bool write = (btrfs_op (dio_bio) ==BTRFS_MAP_WRITE ); -struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb); -const bool raid56 = (btrfs_data_alloc_profile (fs_info) & -BTRFS_BLOCK_GROUP_RAID56_MASK ); -struct btrfs_dio_private *dip; -struct bio *bio; -u64 start_sector; -int async_submit =0 ; -u64 submit_len; -u64 clone_offset =0 ; -u64 clone_len; -blk_status_t status; -struct btrfs_dio_data *dio_data = iter->private; -u64 len; - - dip =btrfs_create_dio_private (dio_bio, inode, file_offset); -if (!dip) { -if (!write) { -unlock_extent (&BTRFS_I (inode)->io_tree, file_offset, - file_offset + dio_bio->bi_iter.bi_size -1 ); - } - dio_bio->bi_status =BLK_STS_RESOURCE ; -bio_endio (dio_bio); -return ; - } - -if (!write) { -/* -* Load the csums up front to reduce csum tree searches and -* contention when submitting bios. -* -* If we have csums disabled this will do nothing. -*/ - status =btrfs_lookup_bio_sums (inode, dio_bio, dip->csums); -if (status !=BLK_STS_OK ) + ret =btrfs_lookup_bio_sums (iter->inode, bio,NULL ); +if (ret) goto out_err; } - start_sector = dio_bio->bi_iter.bi_sector; - submit_len = dio_bio->bi_iter.bi_size; - -do { -struct block_device *bdev; - - bdev =btrfs_get_stripe_info (fs_info,btrfs_op (dio_bio), - start_sector <<9 , submit_len, &len); -if (IS_ERR (bdev)) { - status =errno_to_blk_status (PTR_ERR (bdev)); - goto out_err; - } - - clone_len =min (submit_len, len); -ASSERT (clone_len <=UINT_MAX ); - -/* -* This will never fail as it's passing GPF_NOFS and -* the allocation is backed by btrfs_bioset. -*/ - bio =btrfs_bio_clone_partial (inode, dio_bio, clone_offset, - clone_len); - bio->bi_private = dip; - bio->bi_end_io = btrfs_end_dio_bio; -btrfs_bio (bio)->file_offset = file_offset; - -if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) { - status =extract_ordered_extent (BTRFS_I (inode), bio, - file_offset); -if (status) { -bio_put (bio); - goto out_err; - } - } - -ASSERT (submit_len >= clone_len); - submit_len -= clone_len; - -/* -* Increase the count before we submit the bio so we know -* the end IO handler won't happen before we increase the -* count. Otherwise, the dip might get freed before we're -* done setting it up. -* -* We transfer the initial reference to the last bio, so we -* don't need to increment the reference count for the last one. -*/ -if (submit_len >0 ) { -refcount_inc (&dip->refs); -/* -* If we are submitting more than one bio, submit them -* all asynchronously. The exception is RAID 5 or 6, as -* asynchronous checksums make it difficult to collect -* full stripe writes. -*/ -if (!raid56) - async_submit =1 ; - } - - status =btrfs_submit_dio_bio (bio, inode, file_offset, - async_submit); -if (status) { -bio_put (bio); -if (submit_len >0 ) -refcount_dec (&dip->refs); - goto out_err; - } + ret =btrfs_map_bio (fs_info, bio,0 ); +if (ret) + goto out_err; - dio_data->submitted += clone_len; - clone_offset += clone_len; - start_sector += clone_len >>9 ; - file_offset += clone_len; - }while (submit_len >0 );return ; out_err: - dip->dio_bio->bi_status = status; -btrfs_dio_private_put (dip); + bio->bi_status = ret; +bio_endio (bio); }static const struct iomap_dio_ops btrfs_dio_ops = { - .submit_io = btrfs_submit_direct, + .submit_io = btrfs_dio_submit_io, + .bio_set = &btrfs_bioset, };ssize_t btrfs_dio_rw (struct kiocb *iocb,struct iov_iter *iter,size_t done_before) { -struct btrfs_dio_data data; +struct btrfs_dio_data data = {}; iocb->private = &data; return iomap_dio_rw(iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops,fs/btrfs/volumes.h +1
-0@@ -341,6 +341,7 @@ struct btrfs_bio {/* for direct I/O */ u64 file_offset; +atomic_t repair_refs;/* @device is for stripe IO submission. */ struct btrfs_device *device;
Re: [PATCH 40/40] btrfs: use the iomap direct I/O bio directly
Quoted 2 lines · Qu Wenruo
Not familar with iomap thus I can be totally wrong, but isn't the idea of iomap to separate more code from fs?
Well, to share more code, which requires a certain abstraction, yes.
Quoted 2 lines
I'm really not sure if it's a good idea to expose btrfs internal
bio_setjust for iomap.
We don't. iomap still purely operates on the generic bio. It just allocates additional space for btrfs to use after ->submit_io is called. Just like how e.g. VFS inodes can come with extra space for file system use.
Quoted 2 lines
Personally speaking I didn't see much problem of cloning an iomap bio, it only causes extra memory of
btrfs_bio, which is pretty small previously.
It is yet another pointless memory allocation in something considered very much a fast path.
Re: [PATCH 40/40] btrfs: use the iomap direct I/O bio directly
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Qu Wenruo
Not familar with iomap thus I can be totally wrong, but isn't the idea of iomap to separate more code from fs?
Well, to share more code, which requires a certain abstraction, yes.
Quoted 7 lines
Quoted 2 lines
I'm really not sure if it's a good idea to expose btrfs internal
bio_setjust for iomap.We don't. iomap still purely operates on the generic bio. It just allocates additional space for btrfs to use after ->submit_io is called. Just like how e.g. VFS inodes can come with extra space for file system use.
OK, it's just higher layer pre-allocates those structures.
A little curious if there will be other users of this other than btrfs.
I guess for XFS/EXT4 they don't need any extra space and can just submit the generic bio directly to their devices?
Quoted 6 lines
Quoted 3 lines
Personally speaking I didn't see much problem of cloning an iomap bio, it only causes extra memory of
btrfs_bio, which is pretty small previously.It is yet another pointless memory allocation in something considered very much a fast path.
Another concern is, this behavior mostly means we don't split the generic bio.
Or we still need to allocate memory for the btrfs specific memory for the new bio.
Thanks,
Qu
Re: [PATCH 40/40] btrfs: use the iomap direct I/O bio directly
Quoted 4 lines · Qu Wenruo
A little curious if there will be other users of this other than btrfs.
I guess for XFS/EXT4 they don't need any extra space and can just submit the generic bio directly to their devices?
For normal I/O, yes. But if we want to use Zone Append we'll basically need to use this kind of hook everywhere.
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Personally speaking I didn't see much problem of cloning an iomap bio, it only causes extra memory of
btrfs_bio, which is pretty small previously.It is yet another pointless memory allocation in something considered very much a fast path.
Another concern is, this behavior mostly means we don't split the generic bio.
Or we still need to allocate memory for the btrfs specific memory for the new bio.
With the current series we never split it, yes. I'm relatively new to btrfs, so why would we want to split the bio anyway?
As far as I can tell this is only done for parity raid, and maybe we could actually do the split for those with just this scheme.
I.e. do what you are doing in your series in btrfs_map_bio and allow to clone partial bios there, which should still be possible threre. We'd still need the high-level btrfs_bio to contain the mapping and various end I/O infrastructure like the btrfs_work.
Re: [PATCH 40/40] btrfs: use the iomap direct I/O bio directly
Quoted 15 lines · Christoph Hellwig
Quoted 4 lines · Qu Wenruo
A little curious if there will be other users of this other than btrfs.
I guess for XFS/EXT4 they don't need any extra space and can just submit the generic bio directly to their devices?
For normal I/O, yes. But if we want to use Zone Append we'll basically need to use this kind of hook everywhere.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
Personally speaking I didn't see much problem of cloning an iomap bio, it only causes extra memory of
btrfs_bio, which is pretty small previously.It is yet another pointless memory allocation in something considered very much a fast path.
Quoted 7 lines
Quoted 4 lines
Another concern is, this behavior mostly means we don't split the generic bio.
Or we still need to allocate memory for the btrfs specific memory for the new bio.With the current series we never split it, yes. I'm relatively new to btrfs, so why would we want to split the bio anyway?
Two reasons, one is to make iomap call backs easier, they won't need to bother the stripe boundary anymore if we can do the bio split inside btrfs. (Also why I want to get rid of the zone boundary check, but not any clue on that yet)
Another one is purely design pattern.
We want better layer separation, things like stripe boundary should belong to chunk layer, thus split should also happen when we map the bio, not the read/write path.
This is exactly how the stack drivers do, and I really hope to follow the path.
Quoted 7 lines
As far as I can tell this is only done for parity raid, and maybe we could actually do the split for those with just this scheme.
I.e. do what you are doing in your series in
btrfs_map_bioand allow to clone partial bios there, which should still be possible threre.
I guess yes, we are probably still able to do that.
Just to mention we can not really get rid of the memory allocation.
Quoted 3 lines
We'd still need the high-level
btrfs_bioto contain the
mapping and various end I/O infrastructure like thebtrfs_work.
Then it looks fine to me now.
Thanks,
Qu
Quoted 5 lines · Christoph Hellwig
All this is pretty rough. It survices a xfstests auto group run on a default file system config, though.
The tree is based on Jens' for-next tree as it started with the bio cleanups, and will need a rebase once 5.18-rc1 is out.
JFYI I've run this through xfstests on zoned null_blk here as well and it's looking good so far. I.e. no noticeable regressions found.
I'll look deeper into the series (just had a quick fly over by now) tomorrow with fresh eyes.