direct-io dma alignment
Subsystems (8)
v6 cover direct-io dma alignment
From: Keith Busch <[email protected]>
The previous version is available here:
https://lore.kernel.org/linux-block/[email protected]/T/#m0a93b6392038aad6144e066fb5ada2cbf316f78e
Changes from the previous are all trivial changes:
Fixed a typo, s/bvev_/bdev_/
Updated commit messages and reviews
Updated code comments
Added a new comment for request_queue dma_alignement expressly documenting the consequences of setting this. All existing users of this attribute were checked to ensure they are on the correct side of those consequences.
Keith Busch (11):
block: fix infinite loop for invalid zone append block/bio: remove duplicate append pages code
block: export dma_alignment attribute
block: introduce bdev_dma_alignment helper
block: add a helper function for dio alignment block/merge: count bytes instead of sectors block/bounce: count bytes instead of sectors
iov: introduce iov_iter_aligned
block: introduce bdev_iter_is_aligned helper
block: relax direct io memory alignment
iomap: add support for dma aligned direct-io
Documentation/ABI/stable/sysfs-block | 9 +++ block/bio.c | 114 ++++++++++++--------------- block/blk-merge.c | 41 ++++++---- block/blk-sysfs.c | 7 ++ block/bounce.c | 13 ++- block/fops.c | 16 ++-- fs/iomap/direct-io.c | 4 +- include/linux/blkdev.h | 17 ++++ include/linux/uio.h | 2 + lib/iov_iter.c | 92 +++++++++++++++++++++ 10 files changed, 224 insertions(+), 91 deletions(-)
Signature · 1 line
2.30.2
PATCH v6 1/11 block: fix infinite loop for invalid zone append
From: Keith Busch <[email protected]>
Returning 0 early from __bio_iov_append_get_pages() for the max_append_sectors warning just creates an infinite loop since 0 means success, and the bio will never fill from the unadvancing iov_iter. We could turn the return into an error value, but it will already be turned into an error value later on, so just remove the warning. Clearly no one ever hit it anyway.
--- block/bio.c | 3 --- 1 file changed, 3 deletions(-)
block/bio.c +0 -3
@@ -1229,9 +1229,6 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter)size_t offset;int ret =0 ; -if (WARN_ON_ONCE (!max_append_sectors)) -return 0 ; -/* * Move page array up in the allocated memory for the bio vecs as far as * possible so that we can start filling biovecs from the beginning
Signature · 1 line
2.30.2
PATCH v6 2/11 block/bio: remove duplicate append pages code
In reply to Keith BuschFrom: Keith Busch <[email protected]>
The getting pages setup for zone append and normal IO are identical. Use common code for each.
--- block/bio.c | 102 ++++++++++++++++++++++------------------------------ 1 file changed, 42 insertions(+), 60 deletions(-)
block/bio.c +42 -60
@@ -1159,6 +1159,37 @@ static void bio_put_pages(struct page **pages, size_t size, size_t off)put_page (pages[i]); } +static int bio_iov_add_page (struct bio *bio,struct page *page, +unsigned int len,unsigned int offset) +{ +bool same_page = false; + +if (!__bio_try_merge_page (bio, page, len, offset, &same_page)) { +if (WARN_ON_ONCE (bio_full (bio, len))) +return -EINVAL ; +__bio_add_page (bio, page, len, offset); +return 0 ; + } + +if (same_page) +put_page (page); +return 0 ; +} + +static int bio_iov_add_zone_append_page (struct bio *bio,struct page *page, +unsigned int len,unsigned int offset) +{ +struct request_queue *q =bdev_get_queue (bio->bi_bdev); +bool same_page = false; + +if (bio_add_hw_page (q, bio, page, len, offset, +queue_max_zone_append_sectors (q), &same_page) != len) +return -EINVAL ; +if (same_page) +put_page (page); +return 0 ; +} +#define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *))/**
@@ -1177,7 +1208,6 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)unsigned short entries_left = bio->bi_max_vecs - bio->bi_vcnt;struct bio_vec *bv = bio->bi_io_vec + bio->bi_vcnt;struct page **pages = (struct page **)bv; -bool same_page = false;ssize_t size, left;unsigned len, i;size_t offset;
@@ -1186,7 +1216,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)* Move page array up in the allocated memory for the bio vecs as far as * possible so that we can start filling biovecs from the beginning * without overwriting the temporary page array. -*/ +*/ BUILD_BUG_ON (PAGE_PTRS_PER_BVEC <2 ); pages += entries_left * (PAGE_PTRS_PER_BVEC -1 );
@@ -1196,18 +1226,18 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)for (left = size, i =0 ; left >0 ; left -= len, i++) {struct page *page = pages[i]; +int ret; len =min_t (size_t,PAGE_SIZE - offset, left); +if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) + ret =bio_iov_add_zone_append_page (bio, page, len, + offset); +else + ret =bio_iov_add_page (bio, page, len, offset); -if (__bio_try_merge_page (bio, page, len, offset, &same_page)) { -if (same_page) -put_page (page); - }else { -if (WARN_ON_ONCE (bio_full (bio, len))) { -bio_put_pages (pages + i, left, offset); -return -EINVAL ; - } -__bio_add_page (bio, page, len, offset); +if (ret) { +bio_put_pages (pages + i, left, offset); +return ret; } offset =0 ; }
@@ -1216,51 +1246,6 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)return 0 ; } -static int __bio_iov_append_get_pages (struct bio *bio,struct iov_iter *iter) -{ -unsigned short nr_pages = bio->bi_max_vecs - bio->bi_vcnt; -unsigned short entries_left = bio->bi_max_vecs - bio->bi_vcnt; -struct request_queue *q =bdev_get_queue (bio->bi_bdev); -unsigned int max_append_sectors =queue_max_zone_append_sectors (q); -struct bio_vec *bv = bio->bi_io_vec + bio->bi_vcnt; -struct page **pages = (struct page **)bv; -ssize_t size, left; -unsigned len, i; -size_t offset; -int ret =0 ; - -/* -* Move page array up in the allocated memory for the bio vecs as far as -* possible so that we can start filling biovecs from the beginning -* without overwriting the temporary page array. -*/ -BUILD_BUG_ON (PAGE_PTRS_PER_BVEC <2 ); - pages += entries_left * (PAGE_PTRS_PER_BVEC -1 ); - - size =iov_iter_get_pages (iter, pages,LONG_MAX , nr_pages, &offset); -if (unlikely (size <=0 )) -return size ? size : -EFAULT ; - -for (left = size, i =0 ; left >0 ; left -= len, i++) { -struct page *page = pages[i]; -bool same_page = false; - - len =min_t (size_t,PAGE_SIZE - offset, left); -if (bio_add_hw_page (q, bio, page, len, offset, - max_append_sectors, &same_page) != len) { -bio_put_pages (pages + i, left, offset); - ret = -EINVAL ; -break ; - } -if (same_page) -put_page (page); - offset =0 ; - } - -iov_iter_advance (iter, size - left); -return ret; -} -/** * bio_iov_iter_get_pages - add user or kernel pages to a bio * @bio: bio to add pages to
@@ -1295,10 +1280,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) }do { -if (bio_op (bio) ==REQ_OP_ZONE_APPEND ) - ret =__bio_iov_append_get_pages (bio, iter); -else - ret =__bio_iov_iter_get_pages (bio, iter); + ret =__bio_iov_iter_get_pages (bio, iter); }while (!ret &&iov_iter_count (iter) && !bio_full (bio,0 ));/* don't account direct I/O as memory stall */
Signature · 1 line
2.30.2
PATCH v6 3/11 block: export dma_alignment attribute
In reply to Keith BuschFrom: Keith Busch <[email protected]>
User space may want to know how to align their buffers to avoid bouncing. Export the queue attribute.
--- Documentation/ABI/stable/sysfs-block | 9 +++++++++ block/blk-sysfs.c | 7 +++++++ 2 files changed, 16 insertions(+)
Documentation/ABI/stable/sysfs-block +9 -0
@@ -260,6 +260,15 @@ Description: for discards, and don't read this file. +What: /sys/block/<disk>/queue/dma_alignment +Date: May 2022 +Contact: [email protected] +Description: + Reports the alignment that user space addresses must have to be + used for raw block device access with O_DIRECT and other driver + specific passthrough mechanisms. + + What: /sys/block/<disk>/queue/fua Date: May 2018 Contact: [email protected]
block/blk-sysfs.c +7 -0
@@ -274,6 +274,11 @@ static ssize_t queue_virt_boundary_mask_show(struct request_queue *q, char *pagereturn queue_var_show (q->limits.virt_boundary_mask, page); } +static ssize_t queue_dma_alignment_show (struct request_queue *q,char *page) +{ +return queue_var_show (queue_dma_alignment (q), page); +} +#define QUEUE_SYSFS_BIT_FNS (name, flag, neg) \ static ssize_t \ queue_##name##_show(struct request_queue *q, char *page) \
@@ -606,6 +611,7 @@ QUEUE_RO_ENTRY(queue_dax, "dax");QUEUE_RW_ENTRY (queue_io_timeout,"io_timeout" );QUEUE_RW_ENTRY (queue_wb_lat,"wbt_lat_usec" );QUEUE_RO_ENTRY (queue_virt_boundary_mask,"virt_boundary_mask" ); +QUEUE_RO_ENTRY (queue_dma_alignment,"dma_alignment" );#ifdef CONFIG_BLK_DEV_THROTTLING_LOW QUEUE_RW_ENTRY (blk_throtl_sample_time,"throttle_sample_time" );
@@ -667,6 +673,7 @@ static struct attribute *queue_attrs[] = { &blk_throtl_sample_time_entry.attr, #endif &queue_virt_boundary_mask_entry.attr, + &queue_dma_alignment_entry.attr, NULL, };
Signature · 1 line
2.30.2
PATCH v6 4/11 block: introduce bdev_dma_alignment helper
In reply to Keith BuschFrom: Keith Busch <[email protected]>
Preparing for upcoming dma_alignment users that have a block_device, but don't need the request_queue.
--- include/linux/blkdev.h | 5 +++++ 1 file changed, 5 insertions(+)
include/linux/blkdev.h +5 -0
@@ -1366,6 +1366,11 @@ static inline int queue_dma_alignment(const struct request_queue *q)return q ? q->dma_alignment :511 ; } +static inline unsigned int bdev_dma_alignment (struct block_device *bdev) +{ +return queue_dma_alignment (bdev_get_queue (bdev)); +} +static inline int blk_rq_aligned (struct request_queue *q,unsigned long addr,unsigned int len) {
Signature · 1 line
2.30.2
PATCH v6 5/11 block: add a helper function for dio alignment
In reply to Keith BuschFrom: Keith Busch <[email protected]>
This will make it easier to add more complex acceptable alignment criteria in the future.
--- block/fops.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
block/fops.c +10 -6
@@ -42,6 +42,13 @@ static unsigned int dio_bio_write_op(struct kiocb *iocb)return op; } +static bool blkdev_dio_unaligned (struct block_device *bdev,loff_t pos, +struct iov_iter *iter) +{ +return ((pos |iov_iter_alignment (iter)) & + (bdev_logical_block_size (bdev) -1 )); +} +#define DIO_INLINE_BIO_VECS 4 static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb,
@@ -54,8 +61,7 @@ static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb,struct bio bio;ssize_t ret; -if ((pos |iov_iter_alignment (iter)) & - (bdev_logical_block_size (bdev) -1 )) +if (blkdev_dio_unaligned (bdev, pos, iter))return -EINVAL ;if (nr_pages <=DIO_INLINE_BIO_VECS )
@@ -173,8 +179,7 @@ static ssize_t __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter,loff_t pos = iocb->ki_pos;int ret =0 ; -if ((pos |iov_iter_alignment (iter)) & - (bdev_logical_block_size (bdev) -1 )) +if (blkdev_dio_unaligned (bdev, pos, iter))return -EINVAL ;if (iocb->ki_flags &IOCB_ALLOC_CACHE )
@@ -298,8 +303,7 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb,loff_t pos = iocb->ki_pos;int ret =0 ; -if ((pos |iov_iter_alignment (iter)) & - (bdev_logical_block_size (bdev) -1 )) +if (blkdev_dio_unaligned (bdev, pos, iter))return -EINVAL ;if (iocb->ki_flags &IOCB_ALLOC_CACHE )
Signature · 1 line
2.30.2
Re: [PATCHv6 05/11] block: add a helper function for dio alignment
block/fops.c static unsigned int dio_bio_write_op(struct kiocb *iocb) · line 50
Quoted 6 lines · Keith Busch
block/fops.c +6
-0+static bool blkdev_dio_unaligned (struct block_device *bdev,loff_t pos, +struct iov_iter *iter) +{ +return ((pos |iov_iter_alignment (iter)) & + (bdev_logical_block_size (bdev) -1 )); +}
A nit: if you have to repost this patch series, please leave out the outer parentheses from the return statement. Otherwise this patch looks good to me.
Thanks,
Bart.
PATCH v6 6/11 block/merge: count bytes instead of sectors
In reply to Keith BuschFrom: Keith Busch <[email protected]>
Individual bv_len's may not be a sector size.
--- block/blk-merge.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-)
block/blk-merge.c +24 -17
@@ -201,11 +201,11 @@ static inline unsigned get_max_segment_size(const struct request_queue *q,* @nsegs: [in,out] Number of segments in the bio being built. Incremented * by the number of segments from @bv that may be appended to that * bio without exceeding @max_segs -* @sectors: [in,out] Number of sectors in the bio being built. Incremented -* by the number of sectors from @bv that may be appended to that -* bio without exceeding @max_sectors +* @bytes: [in,out] Number of bytes in the bio being built. Incremented +* by the number of bytes from @bv that may be appended to that +* bio without exceeding @max_bytes * @max_segs: [in] upper bound for *@nsegs -* @max_sectors: [in] upper bound for *@sectors +* @max_bytes: [in] upper bound for *@bytes * * When splitting a bio, it can happen that a bvec is encountered that is too * big to fit in a single segment and hence that it has to be split in the
@@ -216,10 +216,10 @@ static inline unsigned get_max_segment_size(const struct request_queue *q,*/ static bool bvec_split_segs (const struct request_queue *q,const struct bio_vec *bv,unsigned *nsegs, -unsigned *sectors,unsigned max_segs, -unsigned max_sectors) +unsigned *bytes,unsigned max_segs, +unsigned max_bytes) { -unsigned max_len = (min (max_sectors,UINT_MAX >>9 ) - *sectors) <<9 ; +unsigned max_len =min (max_bytes,UINT_MAX ) - *bytes;unsigned len =min (bv->bv_len, max_len);unsigned total_len =0 ;unsigned seg_size =0 ;
@@ -237,7 +237,7 @@ static bool bvec_split_segs(const struct request_queue *q,break ; } - *sectors += total_len >>9 ; + *bytes += total_len;/* tell the caller to split the bvec if it is too big to fit */ return len >0 || bv->bv_len > max_len;
@@ -269,8 +269,8 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, {struct bio_vec bv, bvprv, *bvprvp =NULL ;struct bvec_iter iter; -unsigned nsegs =0 , sectors =0 ; -const unsigned max_sectors =get_max_io_size (q, bio); +unsigned nsegs =0 , bytes =0 ; +const unsigned max_bytes =get_max_io_size (q, bio) <<9 ;const unsigned max_segs =queue_max_segments (q);bio_for_each_bvec (bv, bio, iter) {
@@ -282,12 +282,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, goto split;if (nsegs < max_segs && - sectors + (bv.bv_len >>9 ) <= max_sectors && + bytes + bv.bv_len <= max_bytes && bv.bv_offset + bv.bv_len <=PAGE_SIZE ) { nsegs++; - sectors += bv.bv_len >>9 ; - }else if (bvec_split_segs (q, &bv, &nsegs, §ors, max_segs, - max_sectors)) { + bytes += bv.bv_len; + }else if (bvec_split_segs (q, &bv, &nsegs, &bytes, max_segs, + max_bytes)) { goto split; }
@@ -300,13 +300,20 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, split: *segs = nsegs; +/* +* Individual bvecs might not be logical block aligned. Round down the +* split size so that each bio is properly block size aligned, even if +* we do not use the full hardware limits. +*/ + bytes =ALIGN_DOWN (bytes,queue_logical_block_size (q)); +/* * Bio splitting may cause subtle trouble such as hang when doing sync * iopoll in direct IO routine. Given performance gain of iopoll for * big IO can be trival, disable iopoll when split needed. */ bio_clear_polled (bio); -return bio_split (bio, sectors,GFP_NOIO , bs); +return bio_split (bio, bytes >>SECTOR_SHIFT ,GFP_NOIO , bs); }/**
@@ -375,7 +382,7 @@ EXPORT_SYMBOL(blk_queue_split);unsigned int blk_recalc_rq_segments (struct request *rq) {unsigned int nr_phys_segs =0 ; -unsigned int nr_sectors =0 ; +unsigned int bytes =0 ;struct req_iterator iter;struct bio_vec bv;
@@ -398,7 +405,7 @@ unsigned int blk_recalc_rq_segments(struct request *rq) }rq_for_each_bvec (bv, rq, iter) -bvec_split_segs (rq->q, &bv, &nr_phys_segs, &nr_sectors, +bvec_split_segs (rq->q, &bv, &nr_phys_segs, &bytes,UINT_MAX ,UINT_MAX );return nr_phys_segs; }
Signature · 1 line
2.30.2
Re: [PATCHv6 06/11] block/merge: count bytes instead of sectors
block/blk-merge.c static struct bio *blk_bio_segment_split(struct request_queue *q, · line 273
Quoted 8 lines · Keith Busch
block/blk-merge.c +2
-2@@ -269,8 +269,8 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, {struct bio_vec bv, bvprv, *bvprvp =NULL ;struct bvec_iter iter; -unsigned nsegs =0 , sectors =0 ; -const unsigned max_sectors =get_max_io_size (q, bio); +unsigned nsegs =0 , bytes =0 ; +const unsigned max_bytes =get_max_io_size (q, bio) <<9 ;
How about using SECTOR_SHIFT instead of 9?
Thanks,
Bart.
PATCH v6 7/11 block/bounce: count bytes instead of sectors
In reply to Keith BuschFrom: Keith Busch <[email protected]>
Individual bv_len's may not be a sector size.
--- block/bounce.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
block/bounce.c +10 -3
@@ -205,19 +205,26 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)int rw =bio_data_dir (*bio_orig);struct bio_vec *to, from;struct bvec_iter iter; -unsigned i =0 ; +unsigned i =0 , bytes =0 ;bool bounce = false; -int sectors =0 ; +int sectors;bio_for_each_segment (from, *bio_orig, iter) {if (i++ <BIO_MAX_VECS ) - sectors += from.bv_len >>9 ; + bytes += from.bv_len;if (PageHighMem (from.bv_page)) bounce = true; }if (!bounce)return ; +/* +* Individual bvecs might not be logical block aligned. Round down +* the split size so that each bio is properly block size aligned, +* even if we do not use the full hardware limits. +*/ + sectors =ALIGN_DOWN (bytes,queue_logical_block_size (q)) >> +SECTOR_SHIFT ;if (sectors <bio_sectors (*bio_orig)) { bio =bio_split (*bio_orig, sectors,GFP_NOIO , &bounce_bio_split);bio_chain (bio, *bio_orig);
Signature · 1 line
2.30.2
Re: [PATCHv6 07/11] block/bounce: count bytes instead of sectors
Quoted 7 lines · Keith Busch
From: Keith Busch <[email protected]>
Individual bv_len's may not be a sector size.
Signed-off-by Keith Busch <[email protected]>Reviewed-by Damien Le Moal <[email protected]>Reviewed-by Pankaj Raghav <[email protected]>
Looks good:
Re: [PATCHv6 07/11] block/bounce: count bytes instead of sectors
In reply to Keith BuschQuoted 35 lines · Keith Busch
From: Keith Busch <[email protected]>
Individual bv_len's may not be a sector size.
Signed-off-by Keith Busch <[email protected]>Reviewed-by Damien Le Moal <[email protected]>Reviewed-by Pankaj Raghav <[email protected]>--- block/bounce.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)block/bounce.c +3
-3@@ -205,19 +205,26 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)int rw =bio_data_dir (*bio_orig);struct bio_vec *to, from;struct bvec_iter iter; -unsigned i =0 ; +unsigned i =0 , bytes =0 ;bool bounce = false; -int sectors =0 ; +int sectors;bio_for_each_segment (from, *bio_orig, iter) {if (i++ <BIO_MAX_VECS ) - sectors += from.bv_len >>9 ; + bytes += from.bv_len;if (PageHighMem (from.bv_page)) bounce = true; }if (!bounce)return ;
block/bounce.c void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig) · line 230
Quoted 10 lines
block/bounce.c +7
-0+/* +* Individual bvecs might not be logical block aligned. Round down +* the split size so that each bio is properly block size aligned, +* even if we do not use the full hardware limits. +*/ + sectors =ALIGN_DOWN (bytes,queue_logical_block_size (q)) >> +SECTOR_SHIFT ;if (sectors <bio_sectors (*bio_orig)) { bio =bio_split (*bio_orig, sectors,GFP_NOIO , &bounce_bio_split);bio_chain (bio, *bio_orig);
Do I see correctly that there are two changes in this patch: counting bytes instead of sectors and also splitting at logical block boundaries instead of a 512-byte boundary? Should this patch perhaps be split?
Thanks,
Bart.
Keith Busch
maintainerRe: [PATCHv6 07/11] block/bounce: count bytes instead of sectors
Quoted 40 lines · Bart Van Assche
Quoted 39 lines · Keith Busch
From: Keith Busch <[email protected]>
Individual bv_len's may not be a sector size.
Signed-off-by Keith Busch <[email protected]>Reviewed-by Damien Le Moal <[email protected]>Reviewed-by Pankaj Raghav <[email protected]>--- block/bounce.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)block/bounce.c +9
-3@@ -205,19 +205,26 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)int rw =bio_data_dir (*bio_orig);struct bio_vec *to, from;struct bvec_iter iter; -unsigned i =0 ; +unsigned i =0 , bytes =0 ;bool bounce = false; -int sectors =0 ; +int sectors;bio_for_each_segment (from, *bio_orig, iter) {if (i++ <BIO_MAX_VECS ) - sectors += from.bv_len >>9 ; + bytes += from.bv_len;if (PageHighMem (from.bv_page)) bounce = true; }if (!bounce)return ; +/* +* Individual bvecs might not be logical block aligned. Round down +* the split size so that each bio is properly block size aligned, +* even if we do not use the full hardware limits. +*/ + sectors = ALIGN_DOWN(bytes, queue_logical_block_size(q)) >>
Quoted 8 lines
block/bounce.c void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig) · line 230
Quoted 4 lines
block/bounce.c +1
-0+SECTOR_SHIFT ;if (sectors <bio_sectors (*bio_orig)) { bio =bio_split (*bio_orig, sectors,GFP_NOIO , &bounce_bio_split);bio_chain (bio, *bio_orig);Do I see correctly that there are two changes in this patch: counting bytes instead of sectors and also splitting at logical block boundaries instead of a 512-byte boundary? Should this patch perhaps be split?
The code previously would only split on a bvec boundary. All bvecs were logical block sized, so that part is not changing. We just needed to be able to split mid-bvec since the series enables unaligned offsets to match hardware capabilities.
PATCH v6 8/11 iov: introduce iov_iter_aligned
In reply to Keith BuschFrom: Keith Busch <[email protected]>
The existing iov_iter_alignment() function returns the logical OR of address and length. For cases where address and length need to be considered separately, introduce a helper function that a caller can specificy length and address masks that indicate if the iov is unaligned.
--- include/linux/uio.h | 2 + lib/iov_iter.c | 92 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+)
include/linux/uio.h +2 -0
@@ -219,6 +219,8 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes, struct iov_iter *i);#endif size_t iov_iter_zero (size_t bytes,struct iov_iter *); +bool iov_iter_is_aligned (const struct iov_iter *i,unsigned addr_mask, +unsigned len_mask);unsigned long iov_iter_alignment (const struct iov_iter *i);unsigned long iov_iter_gap_alignment (const struct iov_iter *i); void iov_iter_init(struct iov_iter *i, unsigned int direction, const struct iovec *iov,
lib/iov_iter.c +92 -0
98 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v6 9/11 block: introduce bdev_iter_is_aligned helper
In reply to Keith BuschFrom: Keith Busch <[email protected]>
Provide a convenient function for this repeatable coding pattern.
--- include/linux/blkdev.h | 7 +++++++ 1 file changed, 7 insertions(+)
include/linux/blkdev.h +7 -0
@@ -1371,6 +1371,13 @@ static inline unsigned int bdev_dma_alignment(struct block_device *bdev)return queue_dma_alignment (bdev_get_queue (bdev)); } +static inline bool bdev_iter_is_aligned (struct block_device *bdev, +struct iov_iter *iter) +{ +return iov_iter_is_aligned (iter,bdev_dma_alignment (bdev), +bdev_logical_block_size (bdev) -1 ); +} +static inline int blk_rq_aligned (struct request_queue *q,unsigned long addr,unsigned int len) {
Signature · 1 line
2.30.2
PATCH v6 10/11 block: relax direct io memory alignment
In reply to Keith BuschFrom: Keith Busch <[email protected]>
Use the address alignment requirements from the block_device for direct io instead of requiring addresses be aligned to the block size. User space can discover the alignment requirements from the dma_alignment queue attribute.
User space can specify any hardware compatible DMA offset for each segment, but every segment length is still required to be a multiple of the block size.
--- block/bio.c | 9 +++++++++ block/fops.c | 4 ++-- include/linux/blkdev.h | 5 +++++ 3 files changed, 16 insertions(+), 2 deletions(-)
block/bio.c +9 -0
@@ -1220,7 +1220,16 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)BUILD_BUG_ON (PAGE_PTRS_PER_BVEC <2 ); pages += entries_left * (PAGE_PTRS_PER_BVEC -1 ); +/* +* Each segment in the iov is required to be a block size multiple. +* However, we may not be able to get the entire segment if it spans +* more pages than bi_max_vecs allows, so we have to ALIGN_DOWN the +* result to ensure the bio's total size is correct. The remainder of +* the iov data will be picked up in the next bio iteration. +*/ size =iov_iter_get_pages (iter, pages,LONG_MAX , nr_pages, &offset); +if (size >0 ) + size =ALIGN_DOWN (size,bdev_logical_block_size (bio->bi_bdev));if (unlikely (size <=0 ))return size ? size : -EFAULT ;
block/fops.c +2 -2
@@ -45,8 +45,8 @@ static unsigned int dio_bio_write_op(struct kiocb *iocb)static bool blkdev_dio_unaligned (struct block_device *bdev,loff_t pos,struct iov_iter *iter) { -return ((pos |iov_iter_alignment (iter)) & - (bdev_logical_block_size (bdev) -1 )); +return pos & (bdev_logical_block_size (bdev) -1 ) || + !bdev_iter_is_aligned (bdev, iter); }#define DIO_INLINE_BIO_VECS 4
include/linux/blkdev.h +5 -0
@@ -425,6 +425,11 @@ struct request_queue {unsigned long nr_requests;/* Max # of requests */ unsigned int dma_pad_mask; +/* +* Drivers that set dma_alignment to less than 511 must be prepared to +* handle individual bvec's that are not a multiple of a SECTOR_SIZE +* due to possible offsets. +*/ unsigned int dma_alignment;#ifdef CONFIG_BLK_INLINE_ENCRYPTION
Signature · 1 line
2.30.2
PATCH v6 11/11 iomap: add support for dma aligned direct-io
In reply to Keith BuschFrom: Keith Busch <[email protected]>
Use the address alignment requirements from the block_device for direct io instead of requiring addresses be aligned to the block size.
--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/iomap/direct-io.c +2 -2
@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;
@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {
Signature · 1 line
2.30.2
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 5 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct
io instead of requiring addresses be aligned to the block size.
Hi Keith,
Our s390 PV guests recently started failing to boot from a -next host, and git blame brought me here.
As near as I have been able to tell, we start tripping up on this code from patch 9 [1] that gets invoked with this patch:
Quoted 7 lines
for (k = 0; k < i->nr_segs; k++, skip = 0) {
size_t len = i->iov[k].iov_len - skip;
if (len > size)
len = size;
if (len & len_mask)
return false;The iovec we're failing on has two segments, one with a len of x200 (and base of x...000) and another with a len of xe00 (and a base of x...200), while len_mask is of course xfff.
So before I go any further on what we might have broken, do you happen to have any suggestions what might be going on here, or something I should try?
Thanks,
Eric
[1] https://lore.kernel.org/r/[email protected]/
Quoted 32 lines
Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const structiomap_iter *iter, struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);fs/iomap/direct-io.c +2
-1loff_t pos = iter->pos;unsigned int bio_opf;@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const structiomap_iter *iter, size_t copied = 0;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (
iomap->type==IOMAP_UNWRITTEN) {
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 23 lines · Eric Farman
Quoted 5 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct
io instead of requiring addresses be aligned to the block size.Hi Keith,
Our s390 PV guests recently started failing to boot from a -next host, and git blame brought me here.
As near as I have been able to tell, we start tripping up on this code from patch 9 [1] that gets invoked with this patch:
Quoted 7 lines
for (k = 0; k < i->nr_segs; k++, skip = 0) { size_t len = i->iov[k].iov_len - skip; if (len > size) len = size; if (len & len_mask) return false;
Quoted 7 lines
The iovec we're failing on has two segments, one with a len of x200 (and base of x...000) and another with a len of xe00 (and a base of x...200), while
len_maskis of course xfff.So before I go any further on what we might have broken, do you happen to have any suggestions what might be going on here, or something I should try?
Thanks for the notice, sorry for the trouble. This check wasn't intended to have any difference from the previous code with respect to the vector lengths.
Could you tell me if you're accessing this through the block device direct-io, or through iomap filesystem?
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 32 lines · Keith Busch
Quoted 23 lines · Eric Farman
Quoted 5 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct
io instead of requiring addresses be aligned to the block size.Hi Keith,
Our s390 PV guests recently started failing to boot from a -next host, and git blame brought me here.
As near as I have been able to tell, we start tripping up on this code from patch 9 [1] that gets invoked with this patch:
Quoted 7 lines
for (k = 0; k < i->nr_segs; k++, skip = 0) { size_t len = i->iov[k].iov_len - skip; if (len > size) len = size; if (len & len_mask) return false;Quoted 7 lines
The iovec we're failing on has two segments, one with a len of x200 (and base of x...000) and another with a len of xe00 (and a base of x...200), while
len_maskis of course xfff.So before I go any further on what we might have broken, do you happen to have any suggestions what might be going on here, or something I should try?
Quoted 5 lines
Thanks for the notice, sorry for the trouble. This check wasn't intended to have any difference from the previous code with respect to the vector lengths.
Could you tell me if you're accessing this through the block device direct-io, or through iomap filesystem?
If using iomap, the previous check was this:
unsigned int blkbits = blksize_bits(bdev_logical_block_size(iomap->bdev)); unsigned int align = iov_iter_alignment(dio->submit.iter); ... if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;
And if raw block device, it was this:
if ((pos | iov_iter_alignment(iter)) & (bdev_logical_block_size(bdev) - 1)) return -EINVAL;
The result of "iov_iter_alignment()" would include "0xe00 | 0x200" in your example, and checked against 0xfff should have been failing prior to this patch. Unless I'm missing something...
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 40 lines · Keith Busch
Quoted 39 lines · Keith Busch
Quoted 37 lines · Eric Farman
Quoted 6 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor
direct
io instead of requiring addresses be aligned to the block size.Hi Keith,
Our s390 PV guests recently started failing to boot from a -next host,
and git blame brought me here.As near as I have been able to tell, we start tripping up on this code
from patch 9 [1] that gets invoked with this patch:Quoted 7 lines
for (k = 0; k < i->nr_segs; k++, skip = 0) { size_t len = i->iov[k].iov_len - skip; if (len > size) len = size; if (len & len_mask) return false;The iovec we're failing on has two segments, one with a len of x200
(and base of x...000) and another with a len of xe00 (and a base of
x...200), whilelen_maskis of course xfff.So before I go any further on what we might have broken, do you happen
to have any suggestions what might be going on here, or something I
should try?
Quoted 8 lines
Quoted 8 lines
Thanks for the notice, sorry for the trouble. This check wasn't intended to
have any difference from the previous code with respect to the vector lengths.Could you tell me if you're accessing this through the block device direct-io,
or through iomap filesystem?
Reasonably certain the failure's on iomap. I'd reverted the subject patch from next-20220622 and got things in working order.
Quoted 11 lines
If using iomap, the previous check was this:
unsigned int blkbits =
blksize_bits(bdev_logical_block_size(iomap->bdev)); unsigned int align = iov_iter_alignment(dio->submit.iter); ... if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;
...
Quoted 5 lines
The result of "
iov_iter_alignment()" would include "0xe00|0x200" in your
example, and checked against0xfffshould have been failing prior to this
patch. Unless I'm missing something...
Nope, you're not. I didn't look back at what the old check was doing, just saw "0xe00 and 0x200" and thought "oh there's one page" instead of noting the code was or'ing them. My bad.
That was the last entry in my trace before the guest gave up, as everything else through this code up to that point seemed okay. I'll pick up the working case and see if I can get a clearer picture between the two.
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 83 lines · Eric Farman
Quoted 50 lines · Keith Busch
Quoted 49 lines · Keith Busch
Quoted 36 lines · Eric Farman
Quoted 7 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor
direct
io instead of requiring addresses be aligned to the block size.Hi Keith,
Our s390 PV guests recently started failing to boot from a -next
host,
and git blame brought me here.As near as I have been able to tell, we start tripping up on this
code
from patch 9 [1] that gets invoked with this patch:Quoted 7 lines
for (k = 0; k < i->nr_segs; k++, skip = 0) { size_t len = i->iov[k].iov_len - skip; if (len > size) len = size; if (len & len_mask) return false;The iovec we're failing on has two segments, one with a len of x200
(and base of x...000) and another with a len of xe00 (and a base
of
x...200), whilelen_maskis of course xfff.Quoted 6 lines
So before I go any further on what we might have broken, do you happen
to have any suggestions what might be going on here, or something
I
should try?Thanks for the notice, sorry for the trouble. This check wasn't intended to
have any difference from the previous code with respect to the vector lengths.Quoted 4 lines
Quoted 4 lines
Could you tell me if you're accessing this through the block device
direct-io,
or through iomap filesystem?Reasonably certain the failure's on iomap. I'd reverted the subject patch from next-20220622 and got things in working order.
Quoted 10 lines
If using iomap, the previous check was this:
unsigned int blkbits =
blksize_bits(bdev_logical_block_size(iomap->bdev)); unsigned int align = iov_iter_alignment(dio->submit.iter); ... if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;...
Quoted 7 lines
The result of "
iov_iter_alignment()" would include "0xe00|0x200" in
your
example, and checked against0xfffshould have been failing prior to
this
patch. Unless I'm missing something...Nope, you're not. I didn't look back at what the old check was doing, just saw "
0xe00and0x200" and thought "oh there's one page" instead of
noting the code was or'ing them. My bad.
Quoted 5 lines
That was the last entry in my trace before the guest gave up, as everything else through this code up to that point seemed okay. I'll pick up the working case and see if I can get a clearer picture between
the two.
Looking over the trace again, I realize I did dump iov_iter_alignment() as a comparator, and I see one pass through that had a non-zero response but bdev_iter_is_aligned() returned true...
count = x1000iov_offset = x0nr_segs = 1iov_len = x1000 (len_mask = xfff)iov_base = x...200 (addr_mask = x1ff)
That particular pass through is in the middle of the stuff it tried to do, so I don't know if that's the cause or not but it strikes me as unusual. Will look into that tomorrow and report back.
Quoted 1 line
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 108 lines · Eric Farman
Quoted 89 lines · Eric Farman
Quoted 53 lines · Keith Busch
Quoted 52 lines · Keith Busch
Quoted 38 lines · Eric Farman
Quoted 8 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_device
for
direct
io instead of requiring addresses be aligned to the block size.Hi Keith,
Our s390 PV guests recently started failing to boot from a -next
host,
and git blame brought me here.As near as I have been able to tell, we start tripping up on this
code
from patch 9 [1] that gets invoked with this patch:Quoted 7 lines
for (k = 0; k < i->nr_segs; k++, skip = 0) { size_t len = i->iov[k].iov_len - skip; if (len > size) len = size; if (len & len_mask) return false;The iovec we're failing on has two segments, one with a len of
x200
(and base of x...000) and another with a len of xe00 (and a base
of
x...200), whilelen_maskis of course xfff.Quoted 7 lines
So before I go any further on what we might have broken, do you
happen
to have any suggestions what might be going on here, or something
I
should try?Thanks for the notice, sorry for the trouble. This check wasn't intended to
have any difference from the previous code with respect to the vector lengths.Quoted 4 lines
Quoted 4 lines
Could you tell me if you're accessing this through the block device
direct-io,
or through iomap filesystem?Reasonably certain the failure's on iomap. I'd reverted the subject patch from next-20220622 and got things in working order.
Quoted 10 lines
If using iomap, the previous check was this:
unsigned int blkbits =
blksize_bits(bdev_logical_block_size(iomap->bdev)); unsigned int align = iov_iter_alignment(dio->submit.iter); ... if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;...
Quoted 8 lines
The result of "
iov_iter_alignment()" would include "0xe00|0x200"
in
your
example, and checked against0xfffshould have been failing prior to
this
patch. Unless I'm missing something...Nope, you're not. I didn't look back at what the old check was doing,
just saw "0xe00and0x200" and thought "oh there's one page" instead
of
noting the code was or'ing them. My bad.Quoted 6 lines
That was the last entry in my trace before the guest gave up, as everything else through this code up to that point seemed okay. I'll
pick up the working case and see if I can get a clearer picture between
the two.Looking over the trace again, I realize I did dump
iov_iter_alignment()
as a comparator, and I see one pass through that had a non-zero response butbdev_iter_is_aligned()returned true...count = x1000
iov_offset= x0nr_segs= 1iov_len= x1000 (len_mask= xfff)iov_base= x...200 (addr_mask= x1ff)
Quoted 5 lines
That particular pass through is in the middle of the stuff it tried to
do, so I don't know if that's the cause or not but it strikes me as unusual. Will look into that tomorrow and report back.
Apologies, it took me an extra day to get back to this, but it is indeed this pass through that's causing our boot failures. I note that the old code (in iomap_dio_bio_iter), did:
if ((pos | length | align) & ((1 << blkbits) - 1))
return -EINVAL;With blkbits equal to 12, the resulting mask was 0x0fff against an align value (from iov_iter_alignment) of x200 kicks us out.
The new code (in iov_iter_aligned_iovec), meanwhile, compares this:
if ((unsigned long)(i->iov[k].iov_base + skip) &
addr_mask)
return false;iov_base (and the output of the old iov_iter_aligned_iovec() routine) is x200, but since addr_mask is x1ff this check provides a different response than it used to.
To check this, I changed the comparator to len_mask (almost certainly not the right answer since addr_mask is then unused, but it was good for a quick test), and our PV guests are able to boot again with -next running in the host.
Thanks,
Eric
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 17 lines · Eric Farman
Apologies, it took me an extra day to get back to this, but it is indeed this pass through that's causing our boot failures. I note that the old code (in
iomap_dio_bio_iter), did:if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;With blkbits equal to 12, the resulting mask was
0x0fffagainst an align value (fromiov_iter_alignment) of x200 kicks us out.The new code (in
iov_iter_aligned_iovec), meanwhile, compares this:if ((unsigned long)(i->iov[k].iov_base + skip) & addr_mask) return false;
Quoted 8 lines
iov_base(and the output of the oldiov_iter_aligned_iovec()routine) is x200, but sinceaddr_maskis x1ff this check provides a different response than it used to.To check this, I changed the comparator to
len_mask(almost certainly not the right answer sinceaddr_maskis then unused, but it was good for a quick test), and our PV guests are able to boot again with -next running in the host.
This raises more questions for me. It sounds like your process used to get an EINVAL error, and it wants to continue getting an EINVAL error instead of letting the direct-io request proceed. Is that correct? If so, could you provide more details on what issue occurs with dispatching this request?
If you really need to restrict address' alignment to the storage's logical block size, I think your storage driver needs to set the dma_alignment queue limit to that value.
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 22 lines · On Mon, 27 Jun 2022 09:36:56 -0600 Keith Busch <[email protected]> wrote:
Quoted 21 lines · Eric Farman
Apologies, it took me an extra day to get back to this, but it is indeed this pass through that's causing our boot failures. I note that the old code (in
iomap_dio_bio_iter), did:if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;With blkbits equal to 12, the resulting mask was
0x0fffagainst an align value (fromiov_iter_alignment) of x200 kicks us out.The new code (in
iov_iter_aligned_iovec), meanwhile, compares this:if ((unsigned long)(i->iov[k].iov_base + skip) & addr_mask) return false;
iov_base(and the output of the oldiov_iter_aligned_iovec()routine) is x200, but sinceaddr_maskis x1ff this check provides a different response than it used to.
Quoted 8 lines
Quoted 4 lines
To check this, I changed the comparator to
len_mask(almost certainly not the right answer sinceaddr_maskis then unused, but it was good for a quick test), and our PV guests are able to boot again with -next running in the host.This raises more questions for me. It sounds like your process used to get an EINVAL error, and it wants to continue getting an EINVAL error instead of letting the direct-io request proceed. Is that correct?
Is my understanding as well. But I'm not familiar enough with the code to tell where and how that -EINVAL gets handled.
BTW let me just point out that the bounce buffering via swiotlb needed for PV is not unlikely to mess up the alignment of things. But I'm not sure if that is relevant here.
Regards,
Halil
Quoted 6 lines
If so, could you
provide more details on what issue occurs with dispatching this request?If you really need to restrict address' alignment to the storage's logical block size, I think your storage driver needs to set the
dma_alignmentqueue limit to that value.
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 47 lines · Halil Pasic
Quoted 34 lines · On Mon, 27 Jun 2022 09:36:56 -0600 Keith Busch <[email protected]> wrote:
Quoted 25 lines · Eric Farman
Apologies, it took me an extra day to get back to this, but it is indeed this pass through that's causing our boot failures. I note that
the old code (iniomap_dio_bio_iter), did:if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;With blkbits equal to 12, the resulting mask was
0x0fffagainst an
align value (fromiov_iter_alignment) of x200 kicks us out.The new code (in
iov_iter_aligned_iovec), meanwhile, compares this:if ((unsigned long)(i->iov[k].iov_base + skip) & addr_mask) return false;
iov_base(and the output of the oldiov_iter_aligned_iovec()routine)
is x200, but sinceaddr_maskis x1ff this check provides a different
response than it used to.Quoted 7 lines
To check this, I changed the comparator to
len_mask(almost certainly
not the right answer sinceaddr_maskis then unused, but it was good
for a quick test), and our PV guests are able to boot again with -next
running in the host.Quoted 5 lines
This raises more questions for me. It sounds like your process used to get an
EINVAL error, and it wants to continue getting an EINVAL error instead of
letting the direct-io request proceed. Is that correct?Is my understanding as well. But I'm not familiar enough with the code to
tell where and how that-EINVALgets handled.
Quoted 12 lines
BTW let me just point out that the bounce buffering via swiotlb needed
for PV is not unlikely to mess up the alignment of things. But I'm not
sure if that is relevant here.Regards,
HalilQuoted 3 lines
If so, could you
provide more details on what issue occurs with dispatching this request?
This error occurs reading the initial boot record for a guest, stating QEMU was unable to read block zero from the device. The code that complains doesn't appear to have anything that says "oh, got EINVAL, try it this other way" but I haven't chased down if/where something in between is expecting that and handling it in some unique way. I -think-
I have an easier reproducer now, so maybe I'd be able to get a better
answer to this question.
Quoted 6 lines
Quoted 6 lines
If you really need to restrict address' alignment to the storage's logical
block size, I think your storage driver needs to set thedma_alignmentqueue
limit to that value.
It's possible that there's a problem in the virtio stack here, but the failing configuration is a qcow image on the host rootfs, so it's not using any distinct driver. The bdev request queue that ends up being used is the same allocated out of blk_alloc_queue, so changing dma_alignment there wouldn't work.
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 42 lines · Eric Farman
Quoted 41 lines · Halil Pasic
Quoted 38 lines · On Mon, 27 Jun 2022 09:36:56 -0600 Keith Busch <[email protected]> wrote:
Quoted 36 lines · Eric Farman
Apologies, it took me an extra day to get back to this, but it is
indeed this pass through that's causing our boot failures. I note
that
the old code (iniomap_dio_bio_iter), did:if ((pos | length | align) & ((1 << blkbits) - 1)) return -EINVAL;With blkbits equal to 12, the resulting mask was
0x0fffagainst an
align value (fromiov_iter_alignment) of x200 kicks us out.The new code (in
iov_iter_aligned_iovec), meanwhile, compares this:if ((unsigned long)(i->iov[k].iov_base + skip) & addr_mask) return false;
iov_base(and the output of the oldiov_iter_aligned_iovec()routine)
is x200, but sinceaddr_maskis x1ff this check provides a different
response than it used to.To check this, I changed the comparator to
len_mask(almost certainly
not the right answer sinceaddr_maskis then unused, but it was good
for a quick test), and our PV guests are able to boot again with
-next
running in the host.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
This raises more questions for me. It sounds like your process used
to get an
EINVAL error, and it wants to continue getting an EINVAL error instead of
letting the direct-io request proceed. Is that correct?
Sort of. In the working case, I see a set of iovecs come through with different counts:
base count
0000 0001
0000 0200
0000 0400
0000 0800
0000 1000
0001 1000
0200 1000 << Change occurs here
0400 1000
0800 1000
1000 1000
EINVAL was being returned for any of these iovecs except the page- aligned ones. Once the x200 request returns 0, the remainder of the above list was skipped and the requests continue elsewhere on the file.
Still not sure how our request is getting us into this process. We're simply asking to read a single block, but that's somewhere within an image file.
Quoted 10 lines
Quoted 10 lines
Is my understanding as well. But I'm not familiar enough with the code to
tell where and how that-EINVALgets handled.BTW let me just point out that the bounce buffering via swiotlb needed
for PV is not unlikely to mess up the alignment of things. But I'm not
sure if that is relevant here.
It's true that PV guests were the first to trip over this, but I've since been able to reproduce this with a normal guest. So long as the image file is connected with cache.direct=true, it's unbootable. That should absolve the swiotlb bits from being at fault here.
Quoted 27 lines
Quoted 7 lines
Regards,
HalilQuoted 3 lines
If so, could you
provide more details on what issue occurs with dispatching this request?This error occurs reading the initial boot record for a guest, stating
QEMU was unable to read block zero from the device. The code that complains doesn't appear to have anything that says "oh, got EINVAL, try it this other way" but I haven't chased down if/where something in
between is expecting that and handling it in some unique way. I -think-
I have an easier reproducer now, so maybe I'd be able to get a
better
answer to this question.Quoted 6 lines
Quoted 6 lines
If you really need to restrict address' alignment to the storage's
logical
block size, I think your storage driver needs to set thedma_alignmentqueue
limit to that value.
Quoted 3 lines
It's possible that there's a problem in the virtio stack here, but the
failing configuration is a qcow image on the host rootfs
(on an ext4 filesystem)
Quoted 4 lines
, so it's not
using any distinct driver. The bdev request queue that ends up being used is the same allocated out ofblk_alloc_queue, so changingdma_alignmentthere wouldn't work.
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 15 lines · Eric Farman
Sort of. In the working case, I see a set of iovecs come through with different counts:
base count
0000 0001
0000 0200
0000 0400
0000 0800
0000 1000
0001 1000
0200 1000 << Change occurs here
0400 1000
0800 1000
1000 1000
Quoted 7 lines
EINVAL was being returned for any of these iovecs except the page- aligned ones. Once the x200 request returns 0, the remainder of the above list was skipped and the requests continue elsewhere on the file.
Still not sure how our request is getting us into this process. We're simply asking to read a single block, but that's somewhere within an image file.
I thought this was sounding like some kind of corruption. I tested ext4 on various qemu devices with 4k logical block sizes, and it all looks okay there.
What block driver are you observing this with?
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 28 lines · Keith Busch
Quoted 21 lines · Eric Farman
Sort of. In the working case, I see a set of iovecs come through with
different counts:base count
0000 0001
0000 0200
0000 0400
0000 0800
0000 1000
0001 1000
0200 1000 << Change occurs here
0400 1000
0800 1000
1000 1000EINVAL was being returned for any of these iovecs except the page- aligned ones. Once the x200 request returns 0, the remainder of the above list was skipped and the requests continue elsewhere on the file.
Quoted 5 lines
Still not sure how our request is getting us into this process. We're
simply asking to read a single block, but that's somewhere within an
image file.
Quoted 6 lines
I thought this was sounding like some kind of corruption. I tested ext4 on
various qemu devices with 4k logical block sizes, and it all looks okay there.What block driver are you observing this with?
s390 dasd
This made me think to change my rootfs, and of course the problem goes away once on something like a SCSI volume.
So crawling through the dasd (instead of virtio) driver and I finally find the point where a change to dma_alignment (which you mentioned earlier) would actually fit.
Such a change fixes this for me, so I'll run it by our DASD guys. Thanks for your help and patience.
Eric
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 11 lines · Eric Farman
s390 dasd
This made me think to change my rootfs, and of course the problem goes away once on something like a SCSI volume.
So crawling through the dasd (instead of virtio) driver and I finally find the point where a change to
dma_alignment(which you mentioned earlier) would actually fit.Such a change fixes this for me, so I'll run it by our DASD guys. Thanks for your help and patience.
I'm assuming there's some driver or device requirement that's making this necessary. Is the below driver change what you're looking for? If so, I think you might want this regardless of this direct-io patch just because other interfaces like blk_rq_map_user_iov() and blk_rq_aligned() align to it.
---
drivers/s390/block/dasd_fba.c +1 -0
@@ -780,6 +780,7 @@ static void dasd_fba_setup_blk_queue(struct dasd_block *block)/* With page sized segments each segment can be translated into one idaw/tidaw */ blk_queue_max_segment_size (q,PAGE_SIZE );blk_queue_segment_boundary (q,PAGE_SIZE -1 ); +blk_queue_dma_alignment (q,PAGE_SIZE -1 ); q->limits.discard_granularity = logical_block_size;
--
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 12 lines · Keith Busch
Quoted 11 lines · Eric Farman
s390 dasd
This made me think to change my rootfs, and of course the problem goes
away once on something like a SCSI volume.So crawling through the dasd (instead of virtio) driver and I finally
find the point where a change todma_alignment(which you mentioned earlier) would actually fit.
Quoted 6 lines
Quoted 2 lines
Such a change fixes this for me, so I'll run it by our DASD guys. Thanks for your help and patience.
I'm assuming there's some driver or device requirement that's making this
necessary. Is the below driver change what you're looking for?
Yup, that's exactly what I have (in dasd_eckd.c) and indeed gets things working again. Need to scrounge up some FBA volumes to test that configuration and the change there.
Quoted 5 lines
If so, I think
you might want this regardless of this direct-io patch just because other
interfaces likeblk_rq_map_user_iov()andblk_rq_aligned()align to it.
Good point.
Quoted 18 lines
---
drivers/s390/block/dasd_fba.c +0
-0@@ -780,6 +780,7 @@ static void dasd_fba_setup_blk_queue(struct dasd_block *block)/* With page sized segments each segment can be translated into one idaw/tidaw */
drivers/s390/block/dasd_fba.c +1
-1blk_queue_max_segment_size (q,PAGE_SIZE );blk_queue_segment_boundary (q,PAGE_SIZE -1 ); +blk_queue_dma_alignment (q,PAGE_SIZE -1 ); q->limits.discard_granularity = logical_block_size; --
Christian Borntraeger
reviewerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
In reply to Keith BuschCCing Stefan, Jan.
Below is necessary (in dasd-eckd.c) to make kvm boot working again (and this seems to be the right thing anyway).
Am 29.06.22 um 21:07 schrieb Keith Busch:
Quoted 32 lines · Eric Farman
Quoted 11 lines
s390 dasd
This made me think to change my rootfs, and of course the problem goes away once on something like a SCSI volume.
So crawling through the dasd (instead of virtio) driver and I finally find the point where a change to
dma_alignment(which you mentioned earlier) would actually fit.Such a change fixes this for me, so I'll run it by our DASD guys. Thanks for your help and patience.
I'm assuming there's some driver or device requirement that's making this necessary. Is the below driver change what you're looking for? If so, I think you might want this regardless of this direct-io patch just because other interfaces like
blk_rq_map_user_iov()andblk_rq_aligned()align to it.---
drivers/s390/block/dasd_fba.c +1
-1@@ -780,6 +780,7 @@ static void dasd_fba_setup_blk_queue(struct dasd_block *block)/* With page sized segments each segment can be translated into one idaw/tidaw */ blk_queue_max_segment_size (q,PAGE_SIZE );blk_queue_segment_boundary (q,PAGE_SIZE -1 ); +blk_queue_dma_alignment (q,PAGE_SIZE -1 ); q->limits.discard_granularity = logical_block_size; --
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
In reply to Keith Busch[+f2fs list and maintainers]
Quoted 24 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;
fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 9 lines
fs/iomap/direct-io.c +2
-1@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {
I noticed that this patch is going to break the following logic in f2fs_should_use_dio() in fs/f2fs/file.c:
/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;
So, f2fs assumes that __iomap_dio_rw() returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.
Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?
(Note: if you blame the above code, it was written by me. But I was just preserving the existing behavior; I don't know the original motivation.)
- Eric
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 63 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 24 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 9 lines
fs/iomap/direct-io.c +2
-1@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {I noticed that this patch is going to break the following logic in
f2fs_should_use_dio()infs/f2fs/file.c:/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;So, f2fs assumes that
__iomap_dio_rw()returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.
Quoted 5 lines
Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?
It looks like f2fs just falls back to buffered IO for this condition without reaching the new code in iomap_dio_bio_iter(). btrfs does the same thing (check_direct_IO()). Is that a problem?
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 64 lines · Keith Busch
Quoted 63 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 24 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 9 lines
fs/iomap/direct-io.c +2
-1@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {I noticed that this patch is going to break the following logic in
f2fs_should_use_dio()infs/f2fs/file.c:/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;So, f2fs assumes that
__iomap_dio_rw()returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.
Quoted 8 lines
Quoted 5 lines
Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?It looks like f2fs just falls back to buffered IO for this condition without reaching the new code in
iomap_dio_bio_iter().
No. It's a bit subtle, so read the code and what I'm saying carefully. f2fs only supports 4K aligned DIO and normally falls back to buffered I/O; however, for DIO that is *very* misaligned (not even LBS aligned) it returns EINVAL instead. And it relies on __iomap_dio_rw() returning that EINVAL.
Relying on __iomap_dio_rw() in that way is definitely a bad design on f2fs's part (and I messed that up when switching f2fs from fs/direct-io.c to iomap). The obvious fix is to just have f2fs do the LBS alignment check itself.
But I think that f2fs shouldn't have different behavior for different levels of misalignment in the first place, so I was wondering if anyone had any thoughts on which behavior (EINVAL or fallback to buffered I/O) should be standardized on in all cases, at least for f2fs. There was some discussion about this sort of thing for ext4 several years ago on the thread
https://lore.kernel.org/linux-ext4/[email protected]/T/#u,
but it didn't really reach a conclusion. I'm wondering if the f2fs maintainers have any thoughts about why the f2fs behavior is as it is. I.e. is it just accidental, or are there specific reasons...
- Eric
Keith Busch
maintainerRe: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 71 lines · Eric Biggers
Quoted 70 lines · Keith Busch
Quoted 68 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 24 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 9 lines
fs/iomap/direct-io.c +2
-1@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {I noticed that this patch is going to break the following logic in
f2fs_should_use_dio()infs/f2fs/file.c:/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;So, f2fs assumes that
__iomap_dio_rw()returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?
Quoted 7 lines
Quoted 2 lines
It looks like f2fs just falls back to buffered IO for this condition without reaching the new code in
iomap_dio_bio_iter().No. It's a bit subtle, so read the code and what I'm saying carefully. f2fs only supports 4K aligned DIO and normally falls back to buffered I/O; however, for DIO that is *very* misaligned (not even LBS aligned) it returns EINVAL instead. And it relies on
__iomap_dio_rw()returning that EINVAL.
Okay, I understand the code flow now.
I tested f2fs direct io with every possible alignment, and it is successful for all hardware dma supported alignments and continues to return EINVAL for unaligned. Is the concern that it's now returning success in scenarios that used to fail? Or is there some other 4k f2fs constraint that I haven't found yet?
Quoted 15 lines
Relying on
__iomap_dio_rw()in that way is definitely a bad design on f2fs's part (and I messed that up when switching f2fs fromfs/direct-io.cto iomap). The obvious fix is to just have f2fs do the LBS alignment check itself.But I think that f2fs shouldn't have different behavior for different levels of misalignment in the first place, so I was wondering if anyone had any thoughts on which behavior (EINVAL or fallback to buffered I/O) should be standardized on in all cases, at least for f2fs. There was some discussion about this sort of thing for ext4 several years ago on the thread
https://lore.kernel.org/linux-ext4/[email protected]/T/#u,
but it didn't really reach a conclusion. I'm wondering if the f2fs maintainers have any thoughts about why the f2fs behavior is as it is. I.e. is it just accidental, or are there specific reasons...- Eric
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 80 lines · Keith Busch
Quoted 71 lines · Eric Biggers
Quoted 70 lines · Keith Busch
Quoted 68 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 24 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 9 lines
fs/iomap/direct-io.c +2
-1@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {I noticed that this patch is going to break the following logic in
f2fs_should_use_dio()infs/f2fs/file.c:/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;So, f2fs assumes that
__iomap_dio_rw()returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?Quoted 7 lines
Quoted 2 lines
It looks like f2fs just falls back to buffered IO for this condition without reaching the new code in
iomap_dio_bio_iter().No. It's a bit subtle, so read the code and what I'm saying carefully. f2fs only supports 4K aligned DIO and normally falls back to buffered I/O; however, for DIO that is *very* misaligned (not even LBS aligned) it returns EINVAL instead. And it relies on
__iomap_dio_rw()returning that EINVAL.
Quoted 7 lines
Okay, I understand the code flow now.
I tested f2fs direct io with every possible alignment, and it is successful for all hardware dma supported alignments and continues to return EINVAL for unaligned. Is the concern that it's now returning success in scenarios that used to fail? Or is there some other 4k f2fs constraint that I haven't found yet?
f2fs has a lot of different options, and edge cases that only occur occasionally such as data blocks being moved by garbage collection. So just because misaligned DIO appears to work doesn't necessarily mean that it actually works. Maybe the f2fs developers can elaborate on the reason that f2fs always requires 4K alignment for DIO.
- Eric
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
In reply to Eric BiggersQuoted 83 lines · Eric Biggers
Quoted 64 lines · Keith Busch
Quoted 63 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 24 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 9 lines
fs/iomap/direct-io.c +2
-1@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count; -if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {I noticed that this patch is going to break the following logic in
f2fs_should_use_dio()infs/f2fs/file.c:/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;So, f2fs assumes that
__iomap_dio_rw()returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.Quoted 8 lines
Quoted 5 lines
Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?It looks like f2fs just falls back to buffered IO for this condition without reaching the new code in
iomap_dio_bio_iter().No. It's a bit subtle, so read the code and what I'm saying carefully. f2fs only supports 4K aligned DIO and normally falls back to buffered I/O; however, for DIO that is *very* misaligned (not even LBS aligned) it returns EINVAL instead. And it relies on
__iomap_dio_rw()returning that EINVAL.Relying on
__iomap_dio_rw()in that way is definitely a bad design on f2fs's part (and I messed that up when switching f2fs fromfs/direct-io.cto iomap). The obvious fix is to just have f2fs do the LBS alignment check itself.
Quoted 9 lines
But I think that f2fs shouldn't have different behavior for different levels of misalignment in the first place, so I was wondering if anyone had any thoughts on which behavior (EINVAL or fallback to buffered I/O) should be standardized on in all cases, at least for f2fs. There was some discussion about this sort of thing for ext4 several years ago on the thread
https://lore.kernel.org/linux-ext4/[email protected]/T/#u,
but it didn't really reach a conclusion. I'm wondering if the f2fs maintainers have any thoughts about why the f2fs behavior is as it is. I.e. is it just accidental, or are there specific reasons...
If there's a generic way to deal with this, I have no objection to follow it. Initially, I remember I was trying to match the ext4 rule, but at some point, I lost the track.
Quoted 2 lines
- Eric
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
In reply to Eric BiggersQuoted 30 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 27 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count;
Quoted 3 lines
fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 256
Quoted 3 lines
fs/iomap/direct-io.c +2
-1-if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))
How does this change intersect with "make statx() return DIO alignment information" ? Will the new STATX_DIOALIGN implementations have to be adjusted to set stx_dio_mem_align = bdev_dma_alignment(...)?
I'm guessing the answer is yes, but I haven't seen any patches on the list to do that, but more and more these days email behaves like a flood of UDP traffic... :(
--D
Quoted 40 lines
fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 259
Quoted 3 lines
fs/iomap/direct-io.c +0
-0return -EINVAL ;if (iomap->type ==IOMAP_UNWRITTEN ) {I noticed that this patch is going to break the following logic in
f2fs_should_use_dio()infs/f2fs/file.c:/* * Direct I/O not aligned to the disk's logical_block_size will be * attempted, but will fail with -EINVAL. * * f2fs additionally requires that direct I/O be aligned to the * filesystem block size, which is often a stricter requirement. * However, f2fs traditionally falls back to buffered I/O on requests * that are logical_block_size-aligned but not fs-block aligned. * * The below logic implements this behavior. */ align = iocb->ki_pos | iov_iter_alignment(iter); if (!IS_ALIGNED(align, i_blocksize(inode)) && IS_ALIGNED(align, bdev_logical_block_size(inode->i_sb->s_bdev))) return false; return true;So, f2fs assumes that
__iomap_dio_rw()returns an error if the I/O isn't logical block aligned. This patch changes that. The result is that DIO will sometimes proceed in cases where the I/O doesn't have the fs block alignment required by f2fs for all DIO.Does anyone have any thoughts about what f2fs should be doing here? I think it's weird that f2fs has different behaviors for different degrees of
misalignment: fail with EINVAL if not logical block aligned, else fallback to buffered I/O if not fs block aligned. I think it should be one convention or the other. Any opinions about which one it should be?(Note: if you blame the above code, it was written by me. But I was just preserving the existing behavior; I don't know the original motivation.)
- Eric
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 39 lines · Darrick J. Wong
Quoted 30 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 27 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count;Quoted 3 lines
fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 256
Quoted 3 lines
fs/iomap/direct-io.c +2
-1-if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))How does this change intersect with "make
statx()return DIO alignment information" ? Will the newSTATX_DIOALIGNimplementations have to be adjusted to setstx_dio_mem_align=bdev_dma_alignment(...)?
Quoted 4 lines
I'm guessing the answer is yes, but I haven't seen any patches on the list to do that, but more and more these days email behaves like a flood of UDP traffic... :(
Yes. I haven't done that in the STATX_DIOALIGN patchset yet because I've been basing it on upstream, which doesn't yet have this iomap patch. I haven't been expecting STATX_DIOALIGN to make 5.20, given that it's a new UAPI that needs time to be properly reviewed, plus I've just been busy with other things. So I've been planning to make the above change after this patch lands upstream.
- Eric
Re: [PATCHv6 11/11] iomap: add support for dma aligned direct-io
Quoted 45 lines · Eric Biggers
Quoted 39 lines · Darrick J. Wong
Quoted 30 lines · Eric Biggers
[+f2fs list and maintainers]
Quoted 27 lines · Keith Busch
From: Keith Busch <[email protected]>
Use the address alignment requirements from the
block_devicefor direct io instead of requiring addresses be aligned to the block size.Signed-off-by Keith Busch <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/iomap/direct-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/iomap/direct-io.c +0
-1@@ -242,7 +242,6 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,struct inode *inode = iter->inode;unsigned int blkbits =blksize_bits (bdev_logical_block_size (iomap->bdev));unsigned int fs_block_size =i_blocksize (inode), pad; -unsigned int align =iov_iter_alignment (dio->submit.iter);loff_t length =iomap_length (iter);loff_t pos = iter->pos;unsigned int bio_opf;@@ -253,7 +252,8 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,size_t copied =0 ;size_t orig_count;Quoted 3 lines
fs/iomap/direct-io.c static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, · line 256
Quoted 3 lines
fs/iomap/direct-io.c +2
-1-if ((pos | length | align) & ((1 << blkbits) -1 )) +if ((pos | length) & ((1 << blkbits) -1 ) || + !bdev_iter_is_aligned (iomap->bdev, dio->submit.iter))How does this change intersect with "make
statx()return DIO alignment information" ? Will the newSTATX_DIOALIGNimplementations have to be adjusted to setstx_dio_mem_align=bdev_dma_alignment(...)?Quoted 4 lines
I'm guessing the answer is yes, but I haven't seen any patches on the list to do that, but more and more these days email behaves like a flood of UDP traffic... :(
Quoted 5 lines
Yes. I haven't done that in the
STATX_DIOALIGNpatchset yet because I've been basing it on upstream, which doesn't yet have this iomap patch. I haven't been expectingSTATX_DIOALIGNto make 5.20, given that it's a new UAPI that needs time to be properly reviewed, plus I've just been busy with other things. So I've been planning to make the above change after this patch lands upstream.
<nod> Ok, I'm looking forward to it. Thank you for your work on statx! :)
--D
Quoted 1 line
- Eric
Quoted 9 lines · Keith Busch
From: Keith Busch <[email protected]>
The previous version is available here:
https://lore.kernel.org/linux-block/[email protected]/T/#m0a93b6392038aad6144e066fb5ada2cbf316f78e
Changes from the previous are all trivial changes:
[...]
Applied, thanks!
[01/11] block: fix infinite loop for invalid zone append
commit: 1180b55c93f6b060ad930db151fe6d2b425f9215
[02/11] block/bio: remove duplicate append pages code
commit: 7a2b81b95a89e578343b1c944ddd64d1b14ee49a
[03/11] block: export dma_alignment attribute
commit: 5f507439f051daaa1e3273ff536afda3ad1f1505
[04/11] block: introduce bdev_dma_alignment helper
commit: 24b10a6e0bc22619535b0ed982b7735910981661
[05/11] block: add a helper function for dio alignment
commit: 8a39418810a65f0bcbe559261ef011fe0e298eeb
[06/11] block/merge: count bytes instead of sectors
commit: 4ff782f24a4cad4b033d0f4f6e38cd50e0d463b0
[07/11] block/bounce: count bytes instead of sectors
commit: 4b5310470e72d77c9b52f8544b98aa8cf77d956f
[08/11] iov: introduce iov_iter_aligned
commit: ab7c0c3abb2e5fa15655e4b87bb7b937ca7e18c3
[09/11] block: introduce bdev_iter_is_aligned helper
commit: 72230944b7a53280c1f351a0d5cafed12732ec21
[10/11] block: relax direct io memory alignment
commit: 84f970d415ef4d048e664ac308792eb93d0152fc
[11/11] iomap: add support for dma aligned direct-io
commit: 40e11e7a6cc74f11b5ca23ceefec7c84af5c4c73Best regards,
Signature · 1 line
Jens Axboe