bio splitting cleanups v2
Subsystems (9)
Christoph Hellwig
maintainerHi Jens,
this series has two parts: the first part moves the ->bio_split bio_set to the gendisk as it only is used for file system style I/O.
The other patches reshuffle the bio splitting code so that in the future blk_bio_segment_split can be used to split REQ_OP_ZONE_APPEND bios under file system / remapping driver control. I plan to use that in btrfs in the next merge window.
Changes since v1:
- drop a bogus patch
- fix a comment typo
- fix a commit log typo
- clean up the blk_queue_split calling convention and name
Diffstat:
block/bio-integrity.c | 2 block/bio.c | 2 block/blk-core.c | 9 -- block/blk-merge.c | 186 +++++++++++++++++++++--------------------- block/blk-mq.c | 6 - block/blk-sysfs.c | 2 block/blk.h | 47 ++++------ block/bounce.c | 26 ++--- block/genhd.c | 8 + drivers/block/drbd/drbd_req.c | 2 drivers/block/pktcdvd.c | 2 drivers/block/ps3vram.c | 2 drivers/md/dm.c | 8 - drivers/md/md.c | 2 drivers/nvme/host/multipath.c | 2 drivers/s390/block/dcssblk.c | 2 include/linux/blkdev.h | 5 - 17 files changed, 154 insertions(+), 159 deletions(-)
Christoph Hellwig
maintainerPATCH 1/6 block: change the blk_queue_split calling convention
The double indirect bio leads to somewhat suboptimal code generation. Instead return the (original or split) bio, and make sure the request_queue arguments to the lower level helpers is passed after the bio to avoid constant reshuffling of the argument passing registers.
Also give it and the helpers used to implement it more descriptive names.
--- block/blk-merge.c | 99 +++++++++++++++++------------------ block/blk-mq.c | 4 +- block/blk.h | 6 +-- drivers/block/drbd/drbd_req.c | 2 +- drivers/block/pktcdvd.c | 2 +- drivers/block/ps3vram.c | 2 +- drivers/md/dm.c | 6 +-- drivers/md/md.c | 2 +- drivers/nvme/host/multipath.c | 2 +- drivers/s390/block/dcssblk.c | 2 +- include/linux/blkdev.h | 2 +- 11 files changed, 63 insertions(+), 66 deletions(-)
block/blk-merge.c +48 -51
@@ -95,10 +95,8 @@ static inline bool req_gap_front_merge(struct request *req, struct bio *bio)return bio_will_gap (req->q,NULL , bio, req->bio); } -static struct bio *blk_bio_discard_split (struct request_queue *q, -struct bio *bio, -struct bio_set *bs, -unsigned *nsegs) +static struct bio *bio_split_discard (struct bio *bio,struct request_queue *q, +unsigned *nsegs,struct bio_set *bs) {unsigned int max_discard_sectors, granularity;int alignment;
@@ -139,8 +137,8 @@ static struct bio *blk_bio_discard_split(struct request_queue *q,return bio_split (bio, split_sectors,GFP_NOIO , bs); } -static struct bio *blk_bio_write_zeroes_split (struct request_queue *q, -struct bio *bio,struct bio_set *bs,unsigned *nsegs) +static struct bio *bio_split_write_zeroes (struct bio *bio, +struct request_queue *q,unsigned *nsegs,struct bio_set *bs) { *nsegs =0 ;
@@ -161,8 +159,8 @@ static struct bio *blk_bio_write_zeroes_split(struct request_queue *q,* requests that are submitted to a block device if the start of a bio is not * aligned to a physical block boundary. */ -static inline unsigned get_max_io_size (struct request_queue *q, -struct bio *bio) +static inline unsigned get_max_io_size (struct bio *bio, +struct request_queue *q) {unsigned pbs =queue_physical_block_size (q) >>SECTOR_SHIFT ;unsigned lbs =queue_logical_block_size (q) >>SECTOR_SHIFT ;
@@ -247,16 +245,16 @@ static bool bvec_split_segs(const struct request_queue *q, }/** -* blk_bio_segment_split - split a bio in two bios -* @q: [in] request queue pointer +* bio_split_rw - split a bio in two bios * @bio: [in] bio to be split -* @bs: [in] bio set to allocate the clone from +* @q: [in] request queue pointer * @segs: [out] number of segments in the bio with the first half of the sectors +* @bs: [in] bio set to allocate the clone from * * Clone @bio, update the bi_iter of the clone to represent the first sectors * of @bio and update @bio->bi_iter to represent the remaining sectors. The * following is guaranteed for the cloned bio: -* - That it has at most get_max_io_size(@q, @bio) sectors. +* - That it has at most get_max_io_size(@bio, @q) sectors. * - That it has at most queue_max_segments(@q) segments. * * Except for discard requests the cloned bio will point at the bi_io_vec of
@@ -265,15 +263,13 @@ static bool bvec_split_segs(const struct request_queue *q,* responsible for ensuring that @bs is only destroyed after processing of the * split bio has finished. */ -static struct bio *blk_bio_segment_split (struct request_queue *q, -struct bio *bio, -struct bio_set *bs, -unsigned *segs) +static struct bio *bio_split_rw (struct bio *bio,struct request_queue *q, +unsigned *segs,struct bio_set *bs) {struct bio_vec bv, bvprv, *bvprvp =NULL ;struct bvec_iter iter;unsigned nsegs =0 , bytes =0 ; -const unsigned max_bytes =get_max_io_size (q, bio) <<9 ; +const unsigned max_bytes =get_max_io_size (bio, q) <<9 ;const unsigned max_segs =queue_max_segments (q);bio_for_each_bvec (bv, bio, iter) {
@@ -320,67 +316,68 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, }/** -* __blk_queue_split - split a bio and submit the second half -* @q: [in] request_queue new bio is being queued at -* @bio: [in, out] bio to be split -* @nr_segs: [out] number of segments in the first bio +* __bio_split_to_limits - split a bio to fit the queue limits +* @bio: bio to be split +* @q: request_queue new bio is being queued at +* @nr_segs: returns the number of segments in the returned bio * -* Split a bio into two bios, chain the two bios, submit the second half and -* store a pointer to the first half in *@bio. If the second bio is still too -* big it will be split by a recursive call to this function. Since this -* function may allocate a new bio from q->bio_split, it is the responsibility -* of the caller to ensure that q->bio_split is only released after processing -* of the split bio has finished. +* Check if @bio needs splitting based on the queue limits, and if so split off +* a bio fitting the limits from the beginning of @bio and return it. @bio is +* shortened to the remainder and re-submitted. +* +* The split bio is allocated from @q->bio_split, which is provided by the +* block layer. */ -void __blk_queue_split (struct request_queue *q,struct bio **bio, +struct bio *__bio_split_to_limits (struct bio *bio,struct request_queue *q,unsigned int *nr_segs) { -struct bio *split =NULL ; +struct bio *split; -switch (bio_op (*bio)) { +switch (bio_op (bio)) {case REQ_OP_DISCARD :case REQ_OP_SECURE_ERASE : - split =blk_bio_discard_split (q, *bio, &q->bio_split, nr_segs); + split =bio_split_discard (bio, q, nr_segs, &q->bio_split);break ;case REQ_OP_WRITE_ZEROES : - split =blk_bio_write_zeroes_split (q, *bio, &q->bio_split, - nr_segs); + split =bio_split_write_zeroes (bio, q, nr_segs, &q->bio_split);break ;default : - split =blk_bio_segment_split (q, *bio, &q->bio_split, nr_segs); + split =bio_split_rw (bio, q, nr_segs, &q->bio_split);break ; }if (split) {/* there isn't chance to merge the splitted bio */ split->bi_opf |=REQ_NOMERGE ; - -bio_chain (split, *bio); -trace_block_split (split, (*bio)->bi_iter.bi_sector); -submit_bio_noacct (*bio); - *bio = split; +bio_chain (split, bio); +trace_block_split (split, bio->bi_iter.bi_sector); +submit_bio_noacct (bio); +return split; } +return bio; }/** -* blk_queue_split - split a bio and submit the second half -* @bio: [in, out] bio to be split +* bio_split_to_limits - split a bio to fit the queue limits +* @bio: bio to be split +* +* Check if @bio needs splitting based on the queue limits of @bio->bi_bdev, and +* if so split off a bio fitting the limits from the beginning of @bio and +* return it. @bio is shortened to the remainder and re-submitted. * -* Split a bio into two bios, chains the two bios, submit the second half and -* store a pointer to the first half in *@bio. Since this function may allocate -* a new bio from q->bio_split, it is the responsibility of the caller to ensure -* that q->bio_split is only released after processing of the split bio has -* finished. +* The split bio is allocated from @q->bio_split, which is provided by the +* block layer. */ -void blk_queue_split (struct bio **bio) +struct bio *bio_split_to_limits (struct bio *bio) { -struct request_queue *q =bdev_get_queue ((*bio)->bi_bdev); +struct request_queue *q =bdev_get_queue (bio->bi_bdev);unsigned int nr_segs; -if (blk_may_split (q, *bio)) -__blk_queue_split (q, bio, &nr_segs); +if (bio_may_exceed_limits (bio, q)) +return __bio_split_to_limits (bio, q, &nr_segs); +return bio; } -EXPORT_SYMBOL (blk_queue_split); +EXPORT_SYMBOL (bio_split_to_limits);unsigned int blk_recalc_rq_segments (struct request *rq) {
block/blk-mq.c +2 -2
@@ -2816,8 +2816,8 @@ void blk_mq_submit_bio(struct bio *bio)blk_status_t ret;blk_queue_bounce (q, &bio); -if (blk_may_split (q, bio)) -__blk_queue_split (q, &bio, &nr_segs); +if (bio_may_exceed_limits (bio, q)) + bio =__bio_split_to_limits (bio, q, &nr_segs);if (!bio_integrity_prep (bio))return ;
block/blk.h +3 -3
@@ -288,7 +288,7 @@ ssize_t part_timeout_show(struct device *, struct device_attribute *, char *);ssize_t part_timeout_store (struct device *,struct device_attribute *,const char *,size_t ); -static inline bool blk_may_split (struct request_queue *q,struct bio *bio) +static inline bool bio_may_exceed_limits (struct bio *bio,struct request_queue *q) {switch (bio_op (bio)) {case REQ_OP_DISCARD :
@@ -311,8 +311,8 @@ static inline bool blk_may_split(struct request_queue *q, struct bio *bio) bio->bi_io_vec->bv_len + bio->bi_io_vec->bv_offset >PAGE_SIZE ; } -void __blk_queue_split (struct request_queue *q,struct bio **bio, -unsigned int *nr_segs); +struct bio *__bio_split_to_limits (struct bio *bio,struct request_queue *q, +unsigned int *nr_segs);int ll_back_merge_fn (struct request *req,struct bio *bio,unsigned int nr_segs); bool blk_attempt_req_merge(struct request_queue *q, struct request *rq,
drivers/block/drbd/drbd_req.c +1 -1
@@ -1608,7 +1608,7 @@ void drbd_submit_bio(struct bio *bio) {struct drbd_device *device = bio->bi_bdev->bd_disk->private_data; -blk_queue_split (&bio); + bio =bio_split_to_limits (bio);/* * what we "blindly" assume:
drivers/block/pktcdvd.c +1 -1
@@ -2399,7 +2399,7 @@ static void pkt_submit_bio(struct bio *bio)struct pktcdvd_device *pd = bio->bi_bdev->bd_disk->queue->queuedata;struct bio *split; -blk_queue_split (&bio); + bio =bio_split_to_limits (bio); pkt_dbg(2, pd, "start = %6llx stop = %6llx\n", (unsigned long long)bio->bi_iter.bi_sector,
drivers/block/ps3vram.c +1 -1
@@ -586,7 +586,7 @@ static void ps3vram_submit_bio(struct bio *bio)dev_dbg (&dev->core,"%s\n" , __func__); -blk_queue_split (&bio); + bio =bio_split_to_limits (bio);spin_lock_irq (&priv->lock); busy = !bio_list_empty (&priv->list);
drivers/md/dm.c +3 -3
@@ -1091,7 +1091,7 @@ static sector_t max_io_len(struct dm_target *ti, sector_t sector)* Does the target need to split IO even further? * - varied (per target) IO splitting is a tenet of DM; this * explains why stacked chunk_sectors based splitting via -* blk_queue_split() isn't possible here. +* bio_split_to_limits() isn't possible here. */ if (!ti->max_io_len)return len;
@@ -1669,10 +1669,10 @@ static void dm_split_and_process_bio(struct mapped_device *md, is_abnormal =is_abnormal_io (bio);if (unlikely (is_abnormal)) {/* -* Use blk_queue_split() for abnormal IO (e.g. discard, etc) +* Use bio_split_to_limits() for abnormal IO (e.g. discard, etc) * otherwise associated queue_limits won't be imposed. */ -blk_queue_split (&bio); + bio =bio_split_to_limits (bio); }init_clone_info (&ci, md, map, bio, is_abnormal);
drivers/md/md.c +1 -1
@@ -464,7 +464,7 @@ static void md_submit_bio(struct bio *bio)return ; } -blk_queue_split (&bio); + bio =bio_split_to_limits (bio);if (mddev->ro ==1 &&unlikely (rw ==WRITE )) {if (bio_sectors (bio) !=0 )
drivers/nvme/host/multipath.c +1 -1
@@ -346,7 +346,7 @@ static void nvme_ns_head_submit_bio(struct bio *bio)* different queue via blk_steal_bios(), so we need to use the bio_split * pool from the original queue to allocate the bvecs from. */ -blk_queue_split (&bio); + bio =bio_split_to_limits (bio); srcu_idx =srcu_read_lock (&head->srcu); ns =nvme_find_path (head);
drivers/s390/block/dcssblk.c +1 -1
@@ -863,7 +863,7 @@ dcssblk_submit_bio(struct bio *bio)unsigned long source_addr;unsigned long bytes_done; -blk_queue_split (&bio); + bio =bio_split_to_limits (bio); bytes_done =0 ; dev_info = bio->bi_bdev->bd_disk->private_data;
include/linux/blkdev.h +1 -1
@@ -864,9 +864,9 @@ void blk_request_module(dev_t devt);extern int blk_register_queue (struct gendisk *disk);extern void blk_unregister_queue (struct gendisk *disk);void submit_bio_noacct (struct bio *bio); +struct bio *bio_split_to_limits (struct bio *bio);extern int blk_lld_busy (struct request_queue *q); -extern void blk_queue_split (struct bio **);extern int blk_queue_enter (struct request_queue *q,blk_mq_req_flags_t flags);extern void blk_queue_exit (struct request_queue *q);extern void blk_sync_queue (struct request_queue *q);
Signature · 1 line
2.30.2
Christoph Hellwig
maintainerPATCH 2/6 block: change the blk_queue_bounce calling convention
In reply to Christoph HellwigThe double indirect bio leads to somewhat suboptimal code generation. Instead return the (original or split) bio, and make sure the request_queue arguments to the lower level helpers is passed after the bio to avoid constant reshuffling of the argument passing registers.
--- block/blk-mq.c | 2 +- block/blk.h | 10 ++++++---- block/bounce.c | 26 +++++++++++++------------- 3 files changed, 20 insertions(+), 18 deletions(-)
block/blk-mq.c +1 -1
@@ -2815,7 +2815,7 @@ void blk_mq_submit_bio(struct bio *bio)unsigned int nr_segs =1 ;blk_status_t ret; -blk_queue_bounce (q, &bio); + bio =blk_queue_bounce (bio, q);if (bio_may_exceed_limits (bio, q)) bio =__bio_split_to_limits (bio, q, &nr_segs);
block/blk.h +6 -4
@@ -378,7 +378,7 @@ static inline void blk_throtl_bio_endio(struct bio *bio) { }static inline void blk_throtl_stat_add (struct request *rq,u64 time) { }#endif -void __blk_queue_bounce (struct request_queue *q,struct bio **bio); +struct bio *__blk_queue_bounce (struct bio *bio,struct request_queue *q);static inline bool blk_queue_may_bounce (struct request_queue *q) {
@@ -387,10 +387,12 @@ static inline bool blk_queue_may_bounce(struct request_queue *q) max_low_pfn >= max_pfn; } -static inline void blk_queue_bounce (struct request_queue *q,struct bio **bio) +static inline struct bio *blk_queue_bounce (struct bio *bio, +struct request_queue *q) { -if (unlikely (blk_queue_may_bounce (q) &&bio_has_data (*bio))) -__blk_queue_bounce (q, bio); +if (unlikely (blk_queue_may_bounce (q) &&bio_has_data (bio))) +return __blk_queue_bounce (bio, q); +return bio; }#ifdef CONFIG_BLK_CGROUP_IOLATENCY
block/bounce.c +13 -13
@@ -199,24 +199,24 @@ static struct bio *bounce_clone_bio(struct bio *bio_src)return NULL ; } -void __blk_queue_bounce (struct request_queue *q,struct bio **bio_orig) +struct bio *__blk_queue_bounce (struct bio *bio_orig,struct request_queue *q) {struct bio *bio; -int rw =bio_data_dir (*bio_orig); +int rw =bio_data_dir (bio_orig);struct bio_vec *to, from;struct bvec_iter iter;unsigned i =0 , bytes =0 ;bool bounce = false;int sectors; -bio_for_each_segment (from, *bio_orig, iter) { +bio_for_each_segment (from, bio_orig, iter) {if (i++ <BIO_MAX_VECS ) bytes += from.bv_len;if (PageHighMem (from.bv_page)) bounce = true; }if (!bounce) -return ; +return bio_orig;/* * Individual bvecs might not be logical block aligned. Round down
@@ -225,13 +225,13 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)*/ sectors =ALIGN_DOWN (bytes,queue_logical_block_size (q)) >>SECTOR_SHIFT ; -if (sectors <bio_sectors (*bio_orig)) { - bio =bio_split (*bio_orig, sectors,GFP_NOIO , &bounce_bio_split); -bio_chain (bio, *bio_orig); -submit_bio_noacct (*bio_orig); - *bio_orig = bio; +if (sectors <bio_sectors (bio_orig)) { + bio =bio_split (bio_orig, sectors,GFP_NOIO , &bounce_bio_split); +bio_chain (bio, bio_orig); +submit_bio_noacct (bio_orig); + bio_orig = bio; } - bio =bounce_clone_bio (*bio_orig); + bio =bounce_clone_bio (bio_orig);/* * Bvec table can't be updated by bio_for_each_segment_all(),
@@ -254,7 +254,7 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig) to->bv_page = bounce_page; } -trace_block_bio_bounce (*bio_orig); +trace_block_bio_bounce (bio_orig); bio->bi_flags |= (1 <<BIO_BOUNCED );
@@ -263,6 +263,6 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)else bio->bi_end_io = bounce_end_io_write; - bio->bi_private = *bio_orig; - *bio_orig = bio; + bio->bi_private = bio_orig; +return bio; }
Signature · 1 line
2.30.2
Christoph Hellwig
maintainerPATCH 3/6 block: move ->bio_split to the gendisk
In reply to Christoph HellwigOnly non-passthrough requests are split by the block layer and use the ->bio_split bio_set. Move it from the request_queue to the gendisk.
--- block/blk-core.c | 9 +-------- block/blk-merge.c | 7 ++++--- block/blk-sysfs.c | 2 -- block/genhd.c | 8 +++++++- drivers/md/dm.c | 2 +- include/linux/blkdev.h | 3 ++- 6 files changed, 15 insertions(+), 16 deletions(-)
block/blk-core.c +1 -8
@@ -377,7 +377,6 @@ static void blk_timeout_work(struct work_struct *work)struct request_queue *blk_alloc_queue (int node_id,bool alloc_srcu) {struct request_queue *q; -int ret; q =kmem_cache_alloc_node (blk_get_queue_kmem_cache (alloc_srcu),GFP_KERNEL | __GFP_ZERO, node_id);
@@ -396,13 +395,9 @@ struct request_queue *blk_alloc_queue(int node_id, bool alloc_srcu)if (q->id <0 ) goto fail_srcu; - ret =bioset_init (&q->bio_split,BIO_POOL_SIZE ,0 ,0 ); -if (ret) - goto fail_id; - q->stats =blk_alloc_queue_stats ();if (!q->stats) - goto fail_split; + goto fail_id; q->node = node_id;
@@ -439,8 +434,6 @@ struct request_queue *blk_alloc_queue(int node_id, bool alloc_srcu) fail_stats:blk_free_queue_stats (q->stats); -fail_split: -bioset_exit (&q->bio_split); fail_id:ida_free (&blk_queue_ida, q->id); fail_srcu:
block/blk-merge.c +4 -3
@@ -331,18 +331,19 @@ static struct bio *bio_split_rw(struct bio *bio, struct request_queue *q,struct bio *__bio_split_to_limits (struct bio *bio,struct request_queue *q,unsigned int *nr_segs) { +struct bio_set *bs = &bio->bi_bdev->bd_disk->bio_split;struct bio *split;switch (bio_op (bio)) {case REQ_OP_DISCARD :case REQ_OP_SECURE_ERASE : - split =bio_split_discard (bio, q, nr_segs, &q->bio_split); + split =bio_split_discard (bio, q, nr_segs, bs);break ;case REQ_OP_WRITE_ZEROES : - split =bio_split_write_zeroes (bio, q, nr_segs, &q->bio_split); + split =bio_split_write_zeroes (bio, q, nr_segs, bs);break ;default : - split =bio_split_rw (bio, q, nr_segs, &q->bio_split); + split =bio_split_rw (bio, q, nr_segs, bs);break ; }
block/blk-sysfs.c +0 -2
@@ -779,8 +779,6 @@ static void blk_release_queue(struct kobject *kobj)if (queue_is_mq (q))blk_mq_release (q); -bioset_exit (&q->bio_split); -if (blk_queue_has_srcu (q))cleanup_srcu_struct (q->srcu);
block/genhd.c +7 -1
@@ -1151,6 +1151,7 @@ static void disk_release(struct device *dev)blk_mq_exit_queue (disk->queue);blkcg_exit_queue (disk->queue); +bioset_exit (&disk->bio_split);disk_release_events (disk);kfree (disk->random);
@@ -1342,9 +1343,12 @@ struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id,if (!disk) goto out_put_queue; +if (bioset_init (&disk->bio_split,BIO_POOL_SIZE ,0 ,0 )) + goto out_free_disk; + disk->bdi =bdi_alloc (node_id);if (!disk->bdi) - goto out_free_disk; + goto out_free_bioset;/* bdev_alloc() might need the queue, set before the first call */ disk->queue = q;
@@ -1382,6 +1386,8 @@ struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id,iput (disk->part0->bd_inode); out_free_bdi:bdi_put (disk->bdi); +out_free_bioset: +bioset_exit (&disk->bio_split); out_free_disk:kfree (disk); out_put_queue:
drivers/md/dm.c +1 -1
@@ -1693,7 +1693,7 @@ static void dm_split_and_process_bio(struct mapped_device *md,*/ WARN_ON_ONCE (!dm_io_flagged (io,DM_IO_WAS_SPLIT )); io->split_bio =bio_split (bio, io->sectors,GFP_NOIO , - &md->queue->bio_split); + &md->disk->bio_split);bio_chain (io->split_bio, bio);trace_block_split (io->split_bio, bio->bi_iter.bi_sector);submit_bio_noacct (bio);
include/linux/blkdev.h +2 -1
@@ -140,6 +140,8 @@ struct gendisk {struct request_queue *queue;void *private_data; +struct bio_set bio_split; +int flags;unsigned long state;#define GD_NEED_PART_SCAN 0
@@ -531,7 +533,6 @@ struct request_queue {struct blk_mq_tag_set *tag_set;struct list_head tag_set_list; -struct bio_set bio_split;struct dentry *debugfs_dir;struct dentry *sched_debugfs_dir;
Signature · 1 line
2.30.2
Christoph Hellwig
maintainerPATCH 4/6 block: move the call to get_max_io_size out of blk_bio_segment_split
In reply to Christoph HellwigPrepare for reusing blk_bio_segment_split for (file system controlled) splits of REQ_OP_ZONE_APPEND bios by letting the caller control the maximum size of the bio.
--- block/blk-merge.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
block/blk-merge.c +5 -4
@@ -250,11 +250,12 @@ static bool bvec_split_segs(const struct request_queue *q,* @q: [in] request queue pointer * @segs: [out] number of segments in the bio with the first half of the sectors * @bs: [in] bio set to allocate the clone from +* @max_bytes: [in] maximum number of bytes per bio * * Clone @bio, update the bi_iter of the clone to represent the first sectors * of @bio and update @bio->bi_iter to represent the remaining sectors. The * following is guaranteed for the cloned bio: -* - That it has at most get_max_io_size(@bio, @q) sectors. +* - That it has at most @max_bytes worth of data * - That it has at most queue_max_segments(@q) segments. * * Except for discard requests the cloned bio will point at the bi_io_vec of
@@ -264,12 +265,11 @@ static bool bvec_split_segs(const struct request_queue *q,* split bio has finished. */ static struct bio *bio_split_rw (struct bio *bio,struct request_queue *q, -unsigned *segs,struct bio_set *bs) +unsigned *segs,struct bio_set *bs,unsigned max_bytes) {struct bio_vec bv, bvprv, *bvprvp =NULL ;struct bvec_iter iter;unsigned nsegs =0 , bytes =0 ; -const unsigned max_bytes =get_max_io_size (bio, q) <<9 ;const unsigned max_segs =queue_max_segments (q);bio_for_each_bvec (bv, bio, iter) {
@@ -343,7 +343,8 @@ struct bio *__bio_split_to_limits(struct bio *bio, struct request_queue *q, split =bio_split_write_zeroes (bio, q, nr_segs, bs);break ;default : - split =bio_split_rw (bio, q, nr_segs, bs); + split =bio_split_rw (bio, q, nr_segs, bs, +get_max_io_size (bio, q) <<SECTOR_SHIFT );break ; }
Signature · 1 line
2.30.2
Christoph Hellwig
maintainerPATCH 5/6 block: move bio_allowed_max_sectors to blk-merge.c
In reply to Christoph HellwigMove this helper into the only file where it is used.
--- block/blk-merge.c | 10 ++++++++++ block/blk.h | 10 ---------- 2 files changed, 10 insertions(+), 10 deletions(-)
block/blk-merge.c +10 -0
@@ -95,6 +95,16 @@ static inline bool req_gap_front_merge(struct request *req, struct bio *bio)return bio_will_gap (req->q,NULL , bio, req->bio); } +/* +* The max size one bio can handle is UINT_MAX becasue bvec_iter.bi_size +* is defined as 'unsigned int', meantime it has to be aligned to with the +* logical block size, which is the minimum accepted unit by hardware. +*/ +static unsigned int bio_allowed_max_sectors (struct request_queue *q) +{ +return round_down (UINT_MAX ,queue_logical_block_size (q)) >>9 ; +} +static struct bio *bio_split_discard (struct bio *bio,struct request_queue *q,unsigned *nsegs,struct bio_set *bs) {
block/blk.h +0 -10
@@ -344,16 +344,6 @@ static inline void req_set_nomerge(struct request_queue *q, struct request *req) q->last_merge =NULL ; } -/* -* The max size one bio can handle is UINT_MAX becasue bvec_iter.bi_size -* is defined as 'unsigned int', meantime it has to aligned to with logical -* block size which is the minimum accepted unit by hardware. -*/ -static inline unsigned int bio_allowed_max_sectors (struct request_queue *q) -{ -return round_down (UINT_MAX ,queue_logical_block_size (q)) >>9 ; -} -/* * Internal io_context interface */
Signature · 1 line
2.30.2
Christoph Hellwig
maintainerPATCH 6/6 block: pass struct queue_limits to the bio splitting helpers
In reply to Christoph HellwigAllow using the splitting helpers on just a queue_limits instead of a full request_queue structure. This will eventually allow file systems or remapping drivers to split REQ_OP_ZONE_APPEND bios based on limits calculated as the minimum common capabilities over multiple devices.
--- block/bio-integrity.c | 2 +- block/bio.c | 2 +- block/blk-merge.c | 107 ++++++++++++++++++++---------------------- block/blk-mq.c | 4 +- block/blk.h | 25 +++++----- 5 files changed, 68 insertions(+), 72 deletions(-)
block/bio-integrity.c +1 -1
@@ -134,7 +134,7 @@ int bio_integrity_add_page(struct bio *bio, struct page *page, iv = bip->bip_vec + bip->bip_vcnt;if (bip->bip_vcnt && -bvec_gap_to_prev (bdev_get_queue (bio->bi_bdev), +bvec_gap_to_prev (&bdev_get_queue (bio->bi_bdev)->limits, &bip->bip_vec[bip->bip_vcnt -1 ], offset))return 0 ;
block/bio.c +1 -1
@@ -965,7 +965,7 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio,* would create a gap, disallow it. */ bvec = &bio->bi_io_vec[bio->bi_vcnt -1 ]; -if (bvec_gap_to_prev (q, bvec, offset)) +if (bvec_gap_to_prev (&q->limits, bvec, offset))return 0 ; }
block/blk-merge.c +50 -57
270 lines not on this page · raw message
block/blk-mq.c +2 -2
@@ -2816,8 +2816,8 @@ void blk_mq_submit_bio(struct bio *bio)blk_status_t ret; bio =blk_queue_bounce (bio, q); -if (bio_may_exceed_limits (bio, q)) - bio =__bio_split_to_limits (bio, q, &nr_segs); +if (bio_may_exceed_limits (bio, &q->limits)) + bio =__bio_split_to_limits (bio, &q->limits, &nr_segs);if (!bio_integrity_prep (bio))return ;
block/blk.h +14 -11
@@ -97,23 +97,23 @@ static inline bool biovec_phys_mergeable(struct request_queue *q,return true; } -static inline bool __bvec_gap_to_prev (struct request_queue *q, +static inline bool __bvec_gap_to_prev (struct queue_limits *lim,struct bio_vec *bprv,unsigned int offset) { -return (offset &queue_virt_boundary (q)) || - ((bprv->bv_offset + bprv->bv_len) &queue_virt_boundary (q)); +return (offset & lim->virt_boundary_mask) || + ((bprv->bv_offset + bprv->bv_len) & lim->virt_boundary_mask); }/* * Check if adding a bio_vec after bprv with offset would create a gap in * the SG list. Most drivers don't care about this, but some do. */ -static inline bool bvec_gap_to_prev (struct request_queue *q, +static inline bool bvec_gap_to_prev (struct queue_limits *lim,struct bio_vec *bprv,unsigned int offset) { -if (!queue_virt_boundary (q)) +if (!lim->virt_boundary_mask)return false; -return __bvec_gap_to_prev (q, bprv, offset); +return __bvec_gap_to_prev (lim, bprv, offset); }static inline bool rq_mergeable (struct request *rq)
@@ -189,7 +189,8 @@ static inline bool integrity_req_gap_back_merge(struct request *req,struct bio_integrity_payload *bip =bio_integrity (req->bio);struct bio_integrity_payload *bip_next =bio_integrity (next); -return bvec_gap_to_prev (req->q, &bip->bip_vec[bip->bip_vcnt -1 ], +return bvec_gap_to_prev (&req->q->limits, + &bip->bip_vec[bip->bip_vcnt -1 ], bip_next->bip_vec[0 ].bv_offset); }
@@ -199,7 +200,8 @@ static inline bool integrity_req_gap_front_merge(struct request *req,struct bio_integrity_payload *bip =bio_integrity (bio);struct bio_integrity_payload *bip_next =bio_integrity (req->bio); -return bvec_gap_to_prev (req->q, &bip->bip_vec[bip->bip_vcnt -1 ], +return bvec_gap_to_prev (&req->q->limits, + &bip->bip_vec[bip->bip_vcnt -1 ], bip_next->bip_vec[0 ].bv_offset); }
@@ -288,7 +290,8 @@ ssize_t part_timeout_show(struct device *, struct device_attribute *, char *);ssize_t part_timeout_store (struct device *,struct device_attribute *,const char *,size_t ); -static inline bool bio_may_exceed_limits (struct bio *bio,struct request_queue *q) +static inline bool bio_may_exceed_limits (struct bio *bio, +struct queue_limits *lim) {switch (bio_op (bio)) {case REQ_OP_DISCARD :
@@ -307,11 +310,11 @@ static inline bool bio_may_exceed_limits(struct bio *bio, struct request_queue ** to the performance impact of cloned bios themselves the loop below * doesn't matter anyway. */ -return q->limits.chunk_sectors || bio->bi_vcnt !=1 || +return lim->chunk_sectors || bio->bi_vcnt !=1 || bio->bi_io_vec->bv_len + bio->bi_io_vec->bv_offset >PAGE_SIZE ; } -struct bio *__bio_split_to_limits (struct bio *bio,struct request_queue *q, +struct bio *__bio_split_to_limits (struct bio *bio,struct queue_limits *lim,unsigned int *nr_segs);int ll_back_merge_fn (struct request *req,struct bio *bio,unsigned int nr_segs);
Signature · 1 line
2.30.2
Jens Axboe
maintainerQuoted 9 lines · Christoph Hellwig
Hi Jens,
this series has two parts: the first part moves the ->bio_split
bio_setto the gendisk as it only is used for file system style I/O.The other patches reshuffle the bio splitting code so that in the future
blk_bio_segment_splitcan be used to splitREQ_OP_ZONE_APPENDbios under file system / remapping driver control. I plan to use that in btrfs in the next merge window.
Since this clashes with opf changes from Bart, would you mind spinning them against the for-5.20/drivers-post branch?
Signature · 1 line
Jens Axboe
Christoph Hellwig
maintainerQuoted 2 lines · Jens Axboe
Since this clashes with opf changes from Bart, would you mind spinning them against the for-5.20/drivers-post branch?
Actually. The conflict seems to be that for-5.20/drivers-post still has the blkcg_bio_issue_init in __blk_queue_split that went away in for-5.20/block. So while I can resend against drivers-post, that seems somewhat counterproductive.
Jens Axboe
maintainerQuoted 8 lines · Christoph Hellwig
Quoted 2 lines · Jens Axboe
Since this clashes with opf changes from Bart, would you mind spinning them against the for-5.20/drivers-post branch?
Actually. The conflict seems to be that for-5.20/drivers-post still has the
blkcg_bio_issue_initin__blk_queue_splitthat went away in for-5.20/block. So while I can resend against drivers-post, that seems somewhat counterproductive.
Can you make sure the series applies to my for-next? Then I'll create a temp branch for it.
Signature · 1 line
Jens Axboe