support file system generated / verified integrity information
Subsystems (7)
Hi all,
this series adds support to generate and verify integrity information (aka T10 PI) in the file system, instead of the automatic below the covers support that is currently used.
There two reasons for this:
a) to increase the protection enveloped. Right now this is just a minor step from the bottom of the block layer to the file system, but it is required to support io_uring integrity data passthrough in the file system similar to the currently existing support for block devices, which will follow next. It also allows the file system to directly see the integrity error and act upon in, e.g. when using RAID either integrated (as in btrfs) or by supporting reading redundant copies through the block layer.
b) to make the PI processing more efficient. This is primarily a concern for reads, where the block layer auto PI has to schedule a work item for each bio, and the file system them has to do it again for bounce buffering. Additionally the current iomap post-I/O workqueue handling is a lot more efficient by supporting merging and avoiding workqueue scheduling storms.
The implementation is based on refactoring the existing block layer PI code to be reusable for this use case, and then adding relatively small wrappers for the file system use case. These are then used in iomap to implement the semantics, and wired up in XFS with a small amount of glue code.
Compared to the baseline (iomap-bounce branch), this does not change performance for writes, but increases read performance up to 15% for 4k I/O, with the benefit decreasing with larger I/O sizes as even the baseline maxes out the device quickly. But even for larger I/O sizes, the number of context switches decreases and cpu usage sees a minor decrease.
The patches both require the recent integrity changes in the block tree and the direct I/O bounce buffering series I previously sent out, so you're probably better off just using the git tree instead of applying the patches.
Git tree:
git://git.infradead.org/users/hch/misc.git iomap-pi
Gitweb:
https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/iomap-pi
Diffstat:
block/Makefile | 2 block/bio-integrity-auto.c | 80 +++--------------------- block/bio-integrity-fs.c | 81 +++++++++++++++++++++++++ block/bio-integrity.c | 64 +++++++++++++++++++ block/bio.c | 17 +++-- block/blk-mq.c | 6 + block/blk-settings.c | 13 ---- block/blk.h | 6 + block/t10-pi.c | 12 +-- drivers/nvdimm/btt.c | 6 + fs/fuse/file.c | 5 - fs/iomap/bio.c | 135 ++++++++++++++++++++++++++++-------------- fs/iomap/buffered-io.c | 8 +- fs/iomap/direct-io.c | 15 ++++ fs/iomap/internal.h | 14 ++++ fs/iomap/ioend.c | 28 +++++++- fs/xfs/xfs_aops.c | 47 +++++++++++++- fs/xfs/xfs_iomap.c | 9 +- include/linux/bio-integrity.h | 12 ++- include/linux/bio.h | 2 include/linux/blk-integrity.h | 21 ++++-- include/linux/blkdev.h | 34 ++++++++-- include/linux/iomap.h | 20 +++++- 23 files changed, 456 insertions(+), 181 deletions(-)
PATCH 1/15 block: factor out a bio_integrity_action helper
Split the logic to see if a bio needs integrity metadata from bio_integrity_prep into a reusable helper than can be called from file system code.
--- block/bio-integrity-auto.c | 64 +++++------------------------------ block/bio-integrity.c | 48 ++++++++++++++++++++++++++ block/blk-mq.c | 6 ++-- drivers/nvdimm/btt.c | 6 ++-- include/linux/bio-integrity.h | 5 ++- include/linux/blk-integrity.h | 16 +++++++++ 6 files changed, 82 insertions(+), 63 deletions(-)
block/bio-integrity-auto.c +8 -56
@@ -50,11 +50,6 @@ static bool bip_should_check(struct bio_integrity_payload *bip)return bip->bip_flags &BIP_CHECK_FLAGS ; } -static bool bi_offload_capable (struct blk_integrity *bi) -{ -return bi->metadata_size == bi->pi_tuple_size; -} -/** * __bio_integrity_endio - Integrity I/O completion function * @bio: Protected bio
@@ -84,69 +79,27 @@ bool __bio_integrity_endio(struct bio *bio)/** * bio_integrity_prep - Prepare bio for integrity I/O * @bio: bio to prepare +* @action: preparation action needed +* +* Allocate the integrity payload. For writes, generate the integrity metadata +* and for reads, setup the completion handler to verify the metadata. * -* Checks if the bio already has an integrity payload attached. If it does, the -* payload has been generated by another kernel subsystem, and we just pass it -* through. -* Otherwise allocates integrity payload and for writes the integrity metadata -* will be generated. For reads, the completion handler will verify the -* metadata. +* This is used for bios that do not have user integrity payloads attached. */ -bool bio_integrity_prep (struct bio *bio) +void bio_integrity_prep (struct bio *bio,unsigned int action) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_data *bid; -bool set_flags = true; -gfp_t gfp =GFP_NOIO ; - -if (!bi) -return true; - -if (!bio_sectors (bio)) -return true; - -/* Already protected? */ -if (bio_integrity (bio)) -return true; - -switch (bio_op (bio)) { -case REQ_OP_READ : -if (bi->flags &BLK_INTEGRITY_NOVERIFY ) { -if (bi_offload_capable (bi)) -return true; - set_flags = false; - } -break ; -case REQ_OP_WRITE : -/* -* Zero the memory allocated to not leak uninitialized kernel -* memory to disk for non-integrity metadata where nothing else -* initializes the memory. -*/ -if (bi->flags &BLK_INTEGRITY_NOGENERATE ) { -if (bi_offload_capable (bi)) -return true; - set_flags = false; - gfp |= __GFP_ZERO; - }else if (bi->metadata_size > bi->pi_tuple_size) - gfp |= __GFP_ZERO; -break ; -default : -return true; - } - -if (WARN_ON_ONCE (bio_has_crypt_ctx (bio))) -return true; bid =mempool_alloc (&bid_pool,GFP_NOIO );bio_integrity_init (bio, &bid->bip, &bid->bvec,1 ); bid->bio = bio; bid->bip.bip_flags |=BIP_BLOCK_INTEGRITY ; -bio_integrity_alloc_buf (bio, gfp & __GFP_ZERO); +bio_integrity_alloc_buf (bio, action &BI_ACT_ZERO );bip_set_seed (&bid->bip, bio->bi_iter.bi_sector); -if (set_flags) { +if (action &BI_ACT_CHECK ) {if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP ) bid->bip.bip_flags |=BIP_IP_CHECKSUM ;if (bi->csum_type)
@@ -160,7 +113,6 @@ bool bio_integrity_prep(struct bio *bio)blk_integrity_generate (bio); else bid->saved_bio_iter = bio->bi_iter; -return true; }EXPORT_SYMBOL (bio_integrity_prep);
block/bio-integrity.c +48 -0
60 lines not on this page · raw message
block/blk-mq.c +4 -2
@@ -3133,6 +3133,7 @@ void blk_mq_submit_bio(struct bio *bio)struct request_queue *q =bdev_get_queue (bio->bi_bdev);struct blk_plug *plug = current->plug;const int is_sync =op_is_sync (bio->bi_opf); +unsigned int integrity_action;struct blk_mq_hw_ctx *hctx;unsigned int nr_segs;struct request *rq;
@@ -3185,8 +3186,9 @@ void blk_mq_submit_bio(struct bio *bio)if (!bio) goto queue_exit; -if (!bio_integrity_prep (bio)) - goto queue_exit; + integrity_action =bio_integrity_action (bio); +if (integrity_action) +bio_integrity_prep (bio, integrity_action);blk_mq_bio_issue_init (q, bio);if (blk_mq_attempt_bio_merge (q, bio, nr_segs))
drivers/nvdimm/btt.c +4 -2
@@ -1437,14 +1437,16 @@ static void btt_submit_bio(struct bio *bio) {struct bio_integrity_payload *bip =bio_integrity (bio);struct btt *btt = bio->bi_bdev->bd_disk->private_data; +unsigned int integrity_action;struct bvec_iter iter;unsigned long start;struct bio_vec bvec;int err =0 ;bool do_acct; -if (!bio_integrity_prep (bio)) -return ; + integrity_action =bio_integrity_action (bio); +if (integrity_action) +bio_integrity_prep (bio, integrity_action); do_acct =blk_queue_io_stat (bio->bi_bdev->bd_disk->queue);if (do_acct)
include/linux/bio-integrity.h +2 -3
@@ -78,7 +78,7 @@ int bio_integrity_add_page(struct bio *bio, struct page *page, unsigned int len,int bio_integrity_map_user (struct bio *bio,struct iov_iter *iter);int bio_integrity_map_iter (struct bio *bio,struct uio_meta *meta);void bio_integrity_unmap_user (struct bio *bio); -bool bio_integrity_prep (struct bio *bio); +void bio_integrity_prep (struct bio *bio,unsigned int action);void bio_integrity_advance (struct bio *bio,unsigned int bytes_done);void bio_integrity_trim (struct bio *bio);int bio_integrity_clone (struct bio *bio,struct bio *bio_src,gfp_t gfp_mask);
@@ -104,9 +104,8 @@ static inline void bio_integrity_unmap_user(struct bio *bio) { } -static inline bool bio_integrity_prep (struct bio *bio) +static inline void bio_integrity_prep (struct bio *bio,unsigned int action) { -return true; } static inline int bio_integrity_clone(struct bio *bio, struct bio *bio_src,
include/linux/blk-integrity.h +16 -0
@@ -180,4 +180,20 @@ static inline struct bio_vec rq_integrity_vec(struct request *rq) }#endif /* CONFIG_BLK_DEV_INTEGRITY */ +enum bio_integrity_action { +BI_ACT_BUFFER = (1u <<0 ),/* allocate buffer */ +BI_ACT_CHECK = (1u <<1 ),/* generate / verify PI */ +BI_ACT_ZERO = (1u <<2 ),/* zero buffer */ +}; + +unsigned int __bio_integrity_action (struct bio *bio); +static inline unsigned int bio_integrity_action (struct bio *bio) +{ +if (!blk_get_integrity (bio->bi_bdev->bd_disk)) +return 0 ; +if (bio_integrity (bio)) +return 0 ; +return __bio_integrity_action (bio); +} +#endif /* _LINUX_BLK_INTEGRITY_H */
Signature · 1 line
2.47.3
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
Quoted 31 lines · Christoph Hellwig
Split the logic to see if a bio needs integrity metadata from
bio_integrity_prepinto a reusable helper than can be called from file system code.Signed-off-by Christoph Hellwig <[email protected]>--- block/bio-integrity-auto.c | 64 +++++------------------------------ block/bio-integrity.c | 48 ++++++++++++++++++++++++++ block/blk-mq.c | 6 ++-- drivers/nvdimm/btt.c | 6 ++-- include/linux/bio-integrity.h | 5 ++- include/linux/blk-integrity.h | 16 +++++++++ 6 files changed, 82 insertions(+), 63 deletions(-)block/bio-integrity-auto.c +0
-5@@ -50,11 +50,6 @@ static bool bip_should_check(struct bio_integrity_payload *bip)return bip->bip_flags &BIP_CHECK_FLAGS ; } -static bool bi_offload_capable (struct blk_integrity *bi) -{ -return bi->metadata_size == bi->pi_tuple_size; -} -/** * __bio_integrity_endio - Integrity I/O completion function * @bio: Protected bio
block/bio-integrity-auto.c bool __bio_integrity_endio(struct bio *bio) · line 82
Quoted 4 lines
block/bio-integrity-auto.c +1
-0@@ -84,69 +79,27 @@ bool __bio_integrity_endio(struct bio *bio)/** * bio_integrity_prep - Prepare bio for integrity I/O * @bio: bio to prepare +* @action: preparation action needed
What is @action? Is it a bitset of BI_ACT_* values? If yes, then can the comment please say that explicitly?
Quoted 60 lines
block/bio-integrity-auto.c +5
-48+* +* Allocate the integrity payload. For writes, generate the integrity metadata +* and for reads, setup the completion handler to verify the metadata. * -* Checks if the bio already has an integrity payload attached. If it does, the -* payload has been generated by another kernel subsystem, and we just pass it -* through. -* Otherwise allocates integrity payload and for writes the integrity metadata -* will be generated. For reads, the completion handler will verify the -* metadata. +* This is used for bios that do not have user integrity payloads attached. */ -bool bio_integrity_prep (struct bio *bio) +void bio_integrity_prep (struct bio *bio,unsigned int action) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_data *bid; -bool set_flags = true; -gfp_t gfp =GFP_NOIO ; - -if (!bi) -return true; - -if (!bio_sectors (bio)) -return true; - -/* Already protected? */ -if (bio_integrity (bio)) -return true; - -switch (bio_op (bio)) { -case REQ_OP_READ : -if (bi->flags &BLK_INTEGRITY_NOVERIFY ) { -if (bi_offload_capable (bi)) -return true; - set_flags = false; - } -break ; -case REQ_OP_WRITE : -/* -* Zero the memory allocated to not leak uninitialized kernel -* memory to disk for non-integrity metadata where nothing else -* initializes the memory. -*/ -if (bi->flags &BLK_INTEGRITY_NOGENERATE ) { -if (bi_offload_capable (bi)) -return true; - set_flags = false; - gfp |= __GFP_ZERO; - }else if (bi->metadata_size > bi->pi_tuple_size) - gfp |= __GFP_ZERO; -break ; -default : -return true; - } - -if (WARN_ON_ONCE (bio_has_crypt_ctx (bio))) -return true; bid =mempool_alloc (&bid_pool,GFP_NOIO );
block/bio-integrity-auto.c bool __bio_integrity_endio(struct bio *bio) · line 102
Quoted 10 lines
block/bio-integrity-auto.c +2
-2bio_integrity_init (bio, &bid->bip, &bid->bvec,1 ); bid->bio = bio; bid->bip.bip_flags |=BIP_BLOCK_INTEGRITY ; -bio_integrity_alloc_buf (bio, gfp & __GFP_ZERO); +bio_integrity_alloc_buf (bio, action &BI_ACT_ZERO );bip_set_seed (&bid->bip, bio->bi_iter.bi_sector); -if (set_flags) { +if (action &BI_ACT_CHECK ) {
Yes, I suppose it is a bitset.
--D
Quoted 24 lines
block/bio-integrity-auto.c +0
-1if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP ) bid->bip.bip_flags |=BIP_IP_CHECKSUM ;if (bi->csum_type)@@ -160,7 +113,6 @@ bool bio_integrity_prep(struct bio *bio)blk_integrity_generate (bio); else bid->saved_bio_iter = bio->bi_iter; -return true; }EXPORT_SYMBOL (bio_integrity_prep);block/bio-integrity.c +1
-0@@ -7,6 +7,7 @@*/ #include <linux/blk-integrity.h> +#include <linux/t10-pi.h> #include "blk.h" struct bio_integrity_alloc {
block/bio-integrity.c struct bio_integrity_alloc { · line 23
Quoted 7 lines
block/bio-integrity.c +4
-0@@ -16,6 +17,53 @@ struct bio_integrity_alloc {static mempool_t integrity_buf_pool; +static bool bi_offload_capable (struct blk_integrity *bi) +{ +return bi->metadata_size == bi->pi_tuple_size; +}
Just out of curiosity, what happens if metadata_size > pi_tuple_size?
Can it be the case that metadata_size < pi_tuple_size?
block/bio-integrity.c struct bio_integrity_alloc { · line 25
Quoted 1 line
block/bio-integrity.c +1
-0+unsigned int __bio_integrity_action (struct bio *bio)
Hrm, this function returns a bitset of BI_ACT_* flags, doesn't it?
Would be kinda nice if a comment could say that.
Quoted 15 lines
block/bio-integrity.c +15
-0+{ +struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk); + +if (WARN_ON_ONCE (bio_has_crypt_ctx (bio))) +return 0 ; + +switch (bio_op (bio)) { +case REQ_OP_READ : +if (bi->flags &BLK_INTEGRITY_NOVERIFY ) { +if (bi_offload_capable (bi)) +return 0 ; +return BI_ACT_BUFFER ; + } +return BI_ACT_BUFFER |BI_ACT_CHECK ; +case REQ_OP_WRITE :
block/bio-integrity.c struct bio_integrity_alloc { · line 50
Quoted 10 lines
block/bio-integrity.c +10
-0+/* +* Flush masquerading as write? +*/ +if (!bio_sectors (bio)) +return 0 ; + +/* +* Zero the memory allocated to not leak uninitialized kernel +* memory to disk for non-integrity metadata where nothing else +* initializes the memory.
Er... does someone initialize it eventually? Such as the filesystem? Or maybe an io_uring caller?
block/bio-integrity.c struct bio_integrity_alloc { · line 60
Quoted 10 lines
block/bio-integrity.c +10
-0+*/ +if (bi->flags &BLK_INTEGRITY_NOGENERATE ) { +if (bi_offload_capable (bi)) +return 0 ; +return BI_ACT_BUFFER |BI_ACT_ZERO ; + } + +if (bi->metadata_size > bi->pi_tuple_size) +return BI_ACT_BUFFER |BI_ACT_CHECK |BI_ACT_ZERO ; +return BI_ACT_BUFFER |BI_ACT_CHECK ;
"check" feels like a weird name for a write, where we're generating the PI information. It really means "block layer takes care of PI generation and validation", right? As opposed to whichever upper layer is using the block device?
BI_ACT_YOUDOIT <snerk>
How about BI_ACT_BDEV /* block layer checks/validates PI */
Everything else in here looks reasonable.
--D
Quoted 109 lines
block/bio-integrity.c +6
-0+default : +return 0 ; + } +} +EXPORT_SYMBOL_GPL (__bio_integrity_action); +void bio_integrity_alloc_buf (struct bio *bio,bool zero_buffer) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);block/blk-mq.c +4
-2@@ -3133,6 +3133,7 @@ void blk_mq_submit_bio(struct bio *bio)struct request_queue *q =bdev_get_queue (bio->bi_bdev);struct blk_plug *plug = current->plug;const int is_sync =op_is_sync (bio->bi_opf); +unsigned int integrity_action;struct blk_mq_hw_ctx *hctx;unsigned int nr_segs;struct request *rq;@@ -3185,8 +3186,9 @@ void blk_mq_submit_bio(struct bio *bio)if (!bio) goto queue_exit; -if (!bio_integrity_prep (bio)) - goto queue_exit; + integrity_action =bio_integrity_action (bio); +if (integrity_action) +bio_integrity_prep (bio, integrity_action);blk_mq_bio_issue_init (q, bio);if (blk_mq_attempt_bio_merge (q, bio, nr_segs))drivers/nvdimm/btt.c +4
-2@@ -1437,14 +1437,16 @@ static void btt_submit_bio(struct bio *bio) {struct bio_integrity_payload *bip =bio_integrity (bio);struct btt *btt = bio->bi_bdev->bd_disk->private_data; +unsigned int integrity_action;struct bvec_iter iter;unsigned long start;struct bio_vec bvec;int err =0 ;bool do_acct; -if (!bio_integrity_prep (bio)) -return ; + integrity_action =bio_integrity_action (bio); +if (integrity_action) +bio_integrity_prep (bio, integrity_action); do_acct =blk_queue_io_stat (bio->bi_bdev->bd_disk->queue);if (do_acct)include/linux/bio-integrity.h +2
-3@@ -78,7 +78,7 @@ int bio_integrity_add_page(struct bio *bio, struct page *page, unsigned int len,int bio_integrity_map_user (struct bio *bio,struct iov_iter *iter);int bio_integrity_map_iter (struct bio *bio,struct uio_meta *meta);void bio_integrity_unmap_user (struct bio *bio); -bool bio_integrity_prep (struct bio *bio); +void bio_integrity_prep (struct bio *bio,unsigned int action);void bio_integrity_advance (struct bio *bio,unsigned int bytes_done);void bio_integrity_trim (struct bio *bio);int bio_integrity_clone (struct bio *bio,struct bio *bio_src,gfp_t gfp_mask);@@ -104,9 +104,8 @@ static inline void bio_integrity_unmap_user(struct bio *bio) { } -static inline bool bio_integrity_prep (struct bio *bio) +static inline void bio_integrity_prep (struct bio *bio,unsigned int action) { -return true; } static inline int bio_integrity_clone(struct bio *bio, struct bio *bio_src,include/linux/blk-integrity.h +16
-0@@ -180,4 +180,20 @@ static inline struct bio_vec rq_integrity_vec(struct request *rq) }#endif /* CONFIG_BLK_DEV_INTEGRITY */ +enum bio_integrity_action { +BI_ACT_BUFFER = (1u <<0 ),/* allocate buffer */ +BI_ACT_CHECK = (1u <<1 ),/* generate / verify PI */ +BI_ACT_ZERO = (1u <<2 ),/* zero buffer */ +}; + +unsigned int __bio_integrity_action (struct bio *bio); +static inline unsigned int bio_integrity_action (struct bio *bio) +{ +if (!blk_get_integrity (bio->bi_bdev->bd_disk)) +return 0 ; +if (bio_integrity (bio)) +return 0 ; +return __bio_integrity_action (bio); +} +#endif /* _LINUX_BLK_INTEGRITY_H */ Signature · 1 line
2.47.3
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
Quoted 6 lines · Darrick J. Wong
Quoted 4 lines
/**
*bio_integrity_prep- Prepare bio for integrity I/O
* @bio: bio to prepare
+ * @action: preparation action neededWhat is @action?
Yes.
Quoted 2 lines
Is it a bitset of BI_ACT_* values? If yes, then can the comment please say that explicitly?
Is this good enough?
* @action: preparation action needed (BI_ACT_*)
Quoted 6 lines
block/bio-integrity.c struct bio_integrity_alloc { · line 23
Quoted 4 lines
block/bio-integrity.c +4
-0+static bool bi_offload_capable (struct blk_integrity *bi) +{ +return bi->metadata_size == bi->pi_tuple_size; +}Just out of curiosity, what happens if
metadata_size>pi_tuple_size?
Then we still have to provide a buffer as the automatic insert/strip doesn't work. (I find the offload name rather confusing for this)
Quoted 1 line
Can it be the case that
metadata_size<pi_tuple_size?
No. See blk_validate_integrity_limits:
if (bi->pi_offset + bi->pi_tuple_size > bi->metadata_size) {
pr_warn("pi_offset (%u) + pi_tuple_size (%u) exceeds metadata_size (%u)\n",
bi->pi_offset, bi->pi_tuple_size,
bi->metadata_size);
return -EINVAL;
}Quoted 6 lines
Quoted 2 lines
+unsigned int
__bio_integrity_action(struct bio *bio)Hrm, this function returns a bitset of BI_ACT_* flags, doesn't it?
Would be kinda nice if a comment could say that.
Is this ok?
/**
* bio_integrity_action - return the integrity action needed for a bio
* @bio: bio to operate on
*
* Returns the mask of integrity actions (BI_ACT_*) that need to be performed
* for @bio.
*/
Quoted 7 lines
block/bio-integrity.c struct bio_integrity_alloc { · line 50
Quoted 4 lines
block/bio-integrity.c +4
-0+/* +* Zero the memory allocated to not leak uninitialized kernel +* memory to disk for non-integrity metadata where nothing else +* initializes the memory. Er... does someone initialize it eventually? Such as the filesystem? Or maybe an
io_uringcaller?
For integrity metadata? The code called later fills it out. But it doesn't fill non-integrity metadata, so we need to zero it.
Quoted 16 lines
block/bio-integrity.c struct bio_integrity_alloc { · line 60
Quoted 10 lines
block/bio-integrity.c +10
-0+*/ +if (bi->flags &BLK_INTEGRITY_NOGENERATE ) { +if (bi_offload_capable (bi)) +return 0 ; +return BI_ACT_BUFFER |BI_ACT_ZERO ; + } + +if (bi->metadata_size > bi->pi_tuple_size) +return BI_ACT_BUFFER |BI_ACT_CHECK |BI_ACT_ZERO ; +return BI_ACT_BUFFER |BI_ACT_CHECK ;"check" feels like a weird name for a write, where we're generating the PI information. It really means "block layer takes care of PI generation and validation", right? As opposed to whichever upper layer is using the block device?
Quoted 3 lines
BI_ACT_YOUDOIT<snerk>How about
BI_ACT_BDEV/* block layer checks/validates PI */
I think BI_ACT_BDEV is not very useful. Check is supposed to include generate and verify, but I'm not sure how we could word this in a nice way.
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
Quoted 4 lines
/**
*bio_integrity_prep- Prepare bio for integrity I/O
* @bio: bio to prepare
+ * @action: preparation action neededWhat is @action?
Quoted 8 lines
Yes.
Quoted 2 lines
Is it a bitset of BI_ACT_* values? If yes, then can the comment please say that explicitly?
Is this good enough?
* @action: preparation action needed (BI_ACT_*)
Yes.
Quoted 31 lines
Quoted 6 lines
block/bio-integrity.c struct bio_integrity_alloc { · line 23
Quoted 4 lines
block/bio-integrity.c +4
-0+static bool bi_offload_capable (struct blk_integrity *bi) +{ +return bi->metadata_size == bi->pi_tuple_size; +}Just out of curiosity, what happens if
metadata_size>pi_tuple_size?Then we still have to provide a buffer as the automatic insert/strip doesn't work. (I find the offload name rather confusing for this)
Quoted 1 line
Can it be the case that
metadata_size<pi_tuple_size?No. See
blk_validate_integrity_limits:if (bi->pi_offset + bi->pi_tuple_size > bi->metadata_size) { pr_warn("pi_offset (%u) + pi_tuple_size (%u) exceeds metadata_size (%u)\n", bi->pi_offset, bi->pi_tuple_size, bi->metadata_size); return -EINVAL; }Quoted 6 lines
Quoted 2 lines
+unsigned int
__bio_integrity_action(struct bio *bio)Hrm, this function returns a bitset of BI_ACT_* flags, doesn't it?
Would be kinda nice if a comment could say that.
Is this ok?
Quoted 7 lines
/**
*bio_integrity_action- return the integrity action needed for a bio
* @bio: bio to operate on
*
* Returns the mask of integrity actions (BI_ACT_*) that need to be performed
* for @bio.
*/
Excellent!
Quoted 11 lines
Quoted 8 lines
block/bio-integrity.c struct bio_integrity_alloc { · line 50
Quoted 5 lines
block/bio-integrity.c +4
-0+/* +* Zero the memory allocated to not leak uninitialized kernel +* memory to disk for non-integrity metadata where nothing else +* initializes the memory. Er... does someone initialize it eventually? Such as the filesystem? Or maybe an
io_uringcaller?For integrity metadata? The code called later fills it out. But it doesn't fill non-integrity metadata, so we need to zero it.
Ahhh, right, the app tag or whatever?
Quoted 16 lines
Quoted 16 lines
block/bio-integrity.c struct bio_integrity_alloc { · line 60
Quoted 10 lines
block/bio-integrity.c +10
-0+*/ +if (bi->flags &BLK_INTEGRITY_NOGENERATE ) { +if (bi_offload_capable (bi)) +return 0 ; +return BI_ACT_BUFFER |BI_ACT_ZERO ; + } + +if (bi->metadata_size > bi->pi_tuple_size) +return BI_ACT_BUFFER |BI_ACT_CHECK |BI_ACT_ZERO ; +return BI_ACT_BUFFER |BI_ACT_CHECK ;"check" feels like a weird name for a write, where we're generating the PI information. It really means "block layer takes care of PI generation and validation", right? As opposed to whichever upper layer is using the block device?
Quoted 7 lines
Quoted 3 lines
BI_ACT_YOUDOIT<snerk>How about
BI_ACT_BDEV/* block layer checks/validates PI */I think
BI_ACT_BDEVis not very useful. Check is supposed to include generate and verify, but I'm not sure how we could word this in a nice way.
Me neither. If nobody else here comes up with a better suggestion then I guess BI_ACT_CHECK + its comment wins.
--D
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
Quoted 7 lines · Darrick J. Wong
Quoted 5 lines
Quoted 2 lines
Er... does someone initialize it eventually? Such as the filesystem? Or maybe an
io_uringcaller?For integrity metadata? The code called later fills it out. But it doesn't fill non-integrity metadata, so we need to zero it.
Ahhh, right, the app tag or whatever?
No, the app_tag is zeroed by the PI code itself. NVMe added the concept of generic user specified metdata. That metadata could either be only the PI tuple, contain the PI tuple inside of it, or not contain any PI tuple at all. This non-PI part is what needs explicit zeroing here.
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
In reply to Christoph Hellwigblock/bio-integrity.c struct bio_integrity_alloc { · line 46
Quoted 7 lines · Christoph Hellwig
block/bio-integrity.c +7
-0+case REQ_OP_WRITE : +/* +* Flush masquerading as write? +*/ +if (!bio_sectors (bio)) +return 0 ; +
Earlier this check was happening for REQ_OP_READ too but not sure whether that was superfluous or actually filtering out zero-sector reads. For everything else,
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
In reply to Christoph HellwigChristoph,
Quoted 3 lines
Split the logic to see if a bio needs integrity metadata from
bio_integrity_prepinto a reusable helper than can be called from file system code.
Looks fine with Darrick's requested clarifications.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 01/15] block: factor out a bio_integrity_action helper
In reply to Christoph HellwigPATCH 2/15 block: factor out a bio_integrity_setup_default helper
In reply to Christoph HellwigAdd a helper to set the seed and check flag based on useful defaults from the profile.
Note that this includes a small behavior change, as we ow only sets the seed if any action is set, which is fine as nothing will look at it.
--- block/bio-integrity-auto.c | 14 ++------------ block/bio-integrity.c | 16 ++++++++++++++++ include/linux/bio-integrity.h | 1 + 3 files changed, 19 insertions(+), 12 deletions(-)
block/bio-integrity-auto.c +2 -12
@@ -88,7 +88,6 @@ bool __bio_integrity_endio(struct bio *bio)*/ void bio_integrity_prep (struct bio *bio,unsigned int action) { -struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_data *bid; bid =mempool_alloc (&bid_pool,GFP_NOIO );
@@ -96,17 +95,8 @@ void bio_integrity_prep(struct bio *bio, unsigned int action) bid->bio = bio; bid->bip.bip_flags |=BIP_BLOCK_INTEGRITY ;bio_integrity_alloc_buf (bio, action &BI_ACT_ZERO ); - -bip_set_seed (&bid->bip, bio->bi_iter.bi_sector); - -if (action &BI_ACT_CHECK ) { -if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP ) - bid->bip.bip_flags |=BIP_IP_CHECKSUM ; -if (bi->csum_type) - bid->bip.bip_flags |=BIP_CHECK_GUARD ; -if (bi->flags &BLK_INTEGRITY_REF_TAG ) - bid->bip.bip_flags |=BIP_CHECK_REFTAG ; - } +if (action &BI_ACT_CHECK ) +bio_integrity_setup_default (bio);/* Auto-generate integrity metadata if this is a write */ if (bio_data_dir (bio) ==WRITE &&bip_should_check (&bid->bip))
block/bio-integrity.c +16 -0
@@ -101,6 +101,22 @@ void bio_integrity_free_buf(struct bio_integrity_payload *bip)kfree (bvec_virt (bv)); } +void bio_integrity_setup_default (struct bio *bio) +{ +struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk); +struct bio_integrity_payload *bip =bio_integrity (bio); + +bip_set_seed (bip, bio->bi_iter.bi_sector); + +if (bi->csum_type) { + bip->bip_flags |=BIP_CHECK_GUARD ; +if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP ) + bip->bip_flags |=BIP_IP_CHECKSUM ; + } +if (bi->flags &BLK_INTEGRITY_REF_TAG ) + bip->bip_flags |=BIP_CHECK_REFTAG ; +} +/** * bio_integrity_free - Free bio integrity payload * @bio: bio containing bip to be freed
include/linux/bio-integrity.h +1 -0
@@ -143,5 +143,6 @@ static inline int bio_integrity_add_page(struct bio *bio, struct page *page,void bio_integrity_alloc_buf (struct bio *bio,bool zero_buffer);void bio_integrity_free_buf (struct bio_integrity_payload *bip); +void bio_integrity_setup_default (struct bio *bio);#endif /* _LINUX_BIO_INTEGRITY_H */
Signature · 1 line
2.47.3
Re: [PATCH 02/15] block: factor out a bio_integrity_setup_default helper
Quoted 4 lines · Christoph Hellwig
Add a helper to set the seed and check flag based on useful defaults from the profile.
Note that this includes a small behavior change, as we ow only sets the
"...as we only set the seed if..." ?
Quoted 49 lines
seed if any action is set, which is fine as nothing will look at it.
Signed-off-by Christoph Hellwig <[email protected]>--- block/bio-integrity-auto.c | 14 ++------------ block/bio-integrity.c | 16 ++++++++++++++++ include/linux/bio-integrity.h | 1 + 3 files changed, 19 insertions(+), 12 deletions(-)block/bio-integrity-auto.c +2
-12@@ -88,7 +88,6 @@ bool __bio_integrity_endio(struct bio *bio)*/ void bio_integrity_prep (struct bio *bio,unsigned int action) { -struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_data *bid; bid =mempool_alloc (&bid_pool,GFP_NOIO );@@ -96,17 +95,8 @@ void bio_integrity_prep(struct bio *bio, unsigned int action) bid->bio = bio; bid->bip.bip_flags |=BIP_BLOCK_INTEGRITY ;bio_integrity_alloc_buf (bio, action &BI_ACT_ZERO ); - -bip_set_seed (&bid->bip, bio->bi_iter.bi_sector); - -if (action &BI_ACT_CHECK ) { -if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP ) - bid->bip.bip_flags |=BIP_IP_CHECKSUM ; -if (bi->csum_type) - bid->bip.bip_flags |=BIP_CHECK_GUARD ; -if (bi->flags &BLK_INTEGRITY_REF_TAG ) - bid->bip.bip_flags |=BIP_CHECK_REFTAG ; - } +if (action &BI_ACT_CHECK ) +bio_integrity_setup_default (bio);/* Auto-generate integrity metadata if this is a write */ if (bio_data_dir (bio) ==WRITE &&bip_should_check (&bid->bip))block/bio-integrity.c +0
-0@@ -101,6 +101,22 @@ void bio_integrity_free_buf(struct bio_integrity_payload *bip)kfree (bvec_virt (bv)); }
block/bio-integrity.c void bio_integrity_free_buf(struct bio_integrity_payload *bip) · line 113
Quoted 10 lines
block/bio-integrity.c +10
-0+void bio_integrity_setup_default (struct bio *bio) +{ +struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk); +struct bio_integrity_payload *bip =bio_integrity (bio); + +bip_set_seed (bip, bio->bi_iter.bi_sector); + +if (bi->csum_type) { + bip->bip_flags |=BIP_CHECK_GUARD ; +if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP )
/me wonders if this should be a switch, but it'd be a pretty lame one.
switch (bi->csum_type) {
case BLK_INTEGRITY_CSUM_NONE:
break;
case BLK_INTEGRITY_CSUM_IP:
bip->bip_flags |= BIP_IP_CHECKSUM;
fallthrough;
case BLK_INTEGRITY_CSUM_CRC:
case BLK_INTEGRITY_CSUM_CRC64:
bip->bip_flags |= BIP_CHECK_GUARD;
break;
}<shrug> I'll let you decide if you want slightly better typechecking.
With the commit message fixed,
--D
Quoted 24 lines
block/bio-integrity.c +6
-0+ bip->bip_flags |=BIP_IP_CHECKSUM ; + } +if (bi->flags &BLK_INTEGRITY_REF_TAG ) + bip->bip_flags |=BIP_CHECK_REFTAG ; +} +/** * bio_integrity_free - Free bio integrity payload * @bio: bio containing bip to be freed include/linux/bio-integrity.h +1
-0@@ -143,5 +143,6 @@ static inline int bio_integrity_add_page(struct bio *bio, struct page *page,void bio_integrity_alloc_buf (struct bio *bio,bool zero_buffer);void bio_integrity_free_buf (struct bio_integrity_payload *bip); +void bio_integrity_setup_default (struct bio *bio);#endif /* _LINUX_BIO_INTEGRITY_H */ Signature · 1 line
2.47.3
Re: [PATCH 02/15] block: factor out a bio_integrity_setup_default helper
Quoted 7 lines · Darrick J. Wong
Quoted 4 lines · Christoph Hellwig
Add a helper to set the seed and check flag based on useful defaults from the profile.
Note that this includes a small behavior change, as we ow only sets the
"...as we only set the seed if..." ?
Yes.
Quoted 13 lines
block/bio-integrity.c void bio_integrity_free_buf(struct bio_integrity_payload *bip) · line 113
Quoted 10 lines
block/bio-integrity.c +10
-0+void bio_integrity_setup_default (struct bio *bio) +{ +struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk); +struct bio_integrity_payload *bip =bio_integrity (bio); + +bip_set_seed (bip, bio->bi_iter.bi_sector); + +if (bi->csum_type) { + bip->bip_flags |=BIP_CHECK_GUARD ; +if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP )/me wonders if this should be a switch, but it'd be a pretty lame one.
Quoted 11 lines
switch (bi->csum_type) {
case BLK_INTEGRITY_CSUM_NONE:
break;
case BLK_INTEGRITY_CSUM_IP:
bip->bip_flags |= BIP_IP_CHECKSUM;
fallthrough;
case BLK_INTEGRITY_CSUM_CRC:
case BLK_INTEGRITY_CSUM_CRC64:
bip->bip_flags |= BIP_CHECK_GUARD;
break;
}I don't really think that's a good idea here. BIP_IP_CHECKSUM is a really a special snowflake for SCSI HBA (and not even actual device) usage, so it should be treated like a special snowflake with the if. I sincerely hope no new special snowflakes will show up for the checksums in the future.
Re: [PATCH 02/15] block: factor out a bio_integrity_setup_default helper
Quoted 36 lines · Christoph Hellwig
Quoted 7 lines · Darrick J. Wong
Quoted 4 lines · Christoph Hellwig
Add a helper to set the seed and check flag based on useful defaults from the profile.
Note that this includes a small behavior change, as we ow only sets the
"...as we only set the seed if..." ?
Yes.
Quoted 13 lines
block/bio-integrity.c void bio_integrity_free_buf(struct bio_integrity_payload *bip) · line 113
Quoted 10 lines
block/bio-integrity.c +10
-0+void bio_integrity_setup_default (struct bio *bio) +{ +struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk); +struct bio_integrity_payload *bip =bio_integrity (bio); + +bip_set_seed (bip, bio->bi_iter.bi_sector); + +if (bi->csum_type) { + bip->bip_flags |=BIP_CHECK_GUARD ; +if (bi->csum_type ==BLK_INTEGRITY_CSUM_IP )/me wonders if this should be a switch, but it'd be a pretty lame one.
Quoted 11 lines
switch (bi->csum_type) { case BLK_INTEGRITY_CSUM_NONE: break; case BLK_INTEGRITY_CSUM_IP: bip->bip_flags |= BIP_IP_CHECKSUM; fallthrough; case BLK_INTEGRITY_CSUM_CRC: case BLK_INTEGRITY_CSUM_CRC64: bip->bip_flags |= BIP_CHECK_GUARD; break; }
Quoted 5 lines
I don't really think that's a good idea here.
BIP_IP_CHECKSUMis a really a special snowflake for SCSI HBA (and not even actual device) usage, so it should be treated like a special snowflake with the if. I sincerely hope no new special snowflakes will show up for the checksums in the future.
Fair enough. The BIP_* flags encoding is ... odd. :)
--D
Re: [PATCH 02/15] block: factor out a bio_integrity_setup_default helper
In reply to Christoph HellwigQuoted 1 line · Christoph Hellwig
Note that this includes a small behavior change, as we ow only sets the
with commit wording fixed,
Re: [PATCH 02/15] block: factor out a bio_integrity_setup_default helper
In reply to Christoph HellwigChristoph,
Quoted 6 lines
Add a helper to set the seed and check flag based on useful defaults from the profile.
Note that this includes a small behavior change, as we ow only sets the seed if any action is set, which is fine as nothing will look at it.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 02/15] block: factor out a bio_integrity_setup_default helper
In reply to Christoph HellwigPATCH 3/15 block: add a bdev_has_integrity_csum helper
In reply to Christoph HellwigFactor out a helper to see if the block device has an integrity checksum from bdev_stable_writes so that it can be reused for other checks.
--- include/linux/blkdev.h | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
include/linux/blkdev.h +10 -6
@@ -1472,14 +1472,18 @@ static inline bool bdev_synchronous(struct block_device *bdev)return bdev->bd_disk->queue->limits.features &BLK_FEAT_SYNCHRONOUS ; } -static inline bool bdev_stable_writes (struct block_device *bdev) +static inline bool bdev_has_integrity_csum (struct block_device *bdev) { -struct request_queue *q =bdev_get_queue (bdev); +struct queue_limits *lim =bdev_limits (bdev); -if (IS_ENABLED (CONFIG_BLK_DEV_INTEGRITY ) && - q->limits.integrity.csum_type !=BLK_INTEGRITY_CSUM_NONE ) -return true; -return q->limits.features &BLK_FEAT_STABLE_WRITES ; +return IS_ENABLED (CONFIG_BLK_DEV_INTEGRITY ) && + lim->integrity.csum_type !=BLK_INTEGRITY_CSUM_NONE ; +} + +static inline bool bdev_stable_writes (struct block_device *bdev) +{ +return bdev_has_integrity_csum (bdev) || + (bdev_limits (bdev)->features &BLK_FEAT_STABLE_WRITES ); }static inline bool blk_queue_write_cache (struct request_queue *q)
Signature · 1 line
2.47.3
Re: [PATCH 03/15] block: add a bdev_has_integrity_csum helper
Quoted 4 lines · Christoph Hellwig
Factor out a helper to see if the block device has an integrity checksum from
bdev_stable_writesso that it can be reused for other checks.Signed-off-by Christoph Hellwig <[email protected]>
This looks like a straightforward splitting of responsibilities, so
--D
Quoted 37 lines
--- include/linux/blkdev.h | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)include/linux/blkdev.h +10
-6@@ -1472,14 +1472,18 @@ static inline bool bdev_synchronous(struct block_device *bdev)return bdev->bd_disk->queue->limits.features &BLK_FEAT_SYNCHRONOUS ; } -static inline bool bdev_stable_writes (struct block_device *bdev) +static inline bool bdev_has_integrity_csum (struct block_device *bdev) { -struct request_queue *q =bdev_get_queue (bdev); +struct queue_limits *lim =bdev_limits (bdev); -if (IS_ENABLED (CONFIG_BLK_DEV_INTEGRITY ) && - q->limits.integrity.csum_type !=BLK_INTEGRITY_CSUM_NONE ) -return true; -return q->limits.features &BLK_FEAT_STABLE_WRITES ; +return IS_ENABLED (CONFIG_BLK_DEV_INTEGRITY ) && + lim->integrity.csum_type !=BLK_INTEGRITY_CSUM_NONE ; +} + +static inline bool bdev_stable_writes (struct block_device *bdev) +{ +return bdev_has_integrity_csum (bdev) || + (bdev_limits (bdev)->features &BLK_FEAT_STABLE_WRITES ); }static inline bool blk_queue_write_cache (struct request_queue *q)Signature · 1 line
2.47.3
Re: [PATCH 03/15] block: add a bdev_has_integrity_csum helper
In reply to Christoph HellwigRe: [PATCH 03/15] block: add a bdev_has_integrity_csum helper
In reply to Christoph HellwigChristoph,
Quoted 2 lines
Factor out a helper to see if the block device has an integrity checksum from
bdev_stable_writesso that it can be reused for other checks.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 03/15] block: add a bdev_has_integrity_csum helper
In reply to Christoph HellwigPATCH 4/15 block: prepare generation / verification helpers for fs usage
In reply to Christoph HellwigReturn the status from verify instead of directly stashing it in the bio, and rename the helpers to use the usual bio_ prefix for things operating on a bio.
--- block/bio-integrity-auto.c | 4 ++-- block/blk.h | 6 ++++-- block/t10-pi.c | 12 ++++++------ 3 files changed, 12 insertions(+), 10 deletions(-)
block/bio-integrity-auto.c +2 -2
@@ -39,7 +39,7 @@ static void bio_integrity_verify_fn(struct work_struct *work) container_of(work, struct bio_integrity_data, work);struct bio *bio = bid->bio; -blk_integrity_verify_iter (bio, &bid->saved_bio_iter); + bio->bi_status =bio_integrity_verify (bio, &bid->saved_bio_iter);bio_integrity_finish (bid);bio_endio (bio); }
@@ -100,7 +100,7 @@ void bio_integrity_prep(struct bio *bio, unsigned int action)/* Auto-generate integrity metadata if this is a write */ if (bio_data_dir (bio) ==WRITE &&bip_should_check (&bid->bip)) -blk_integrity_generate (bio); +bio_integrity_generate (bio);else bid->saved_bio_iter = bio->bi_iter; }
block/blk.h +4 -2
@@ -695,8 +695,10 @@ int bdev_open(struct block_device *bdev, blk_mode_t mode, void *holder, const struct blk_holder_ops *hops, struct file *bdev_file);int bdev_permission (dev_t dev,blk_mode_t mode,void *holder); -void blk_integrity_generate (struct bio *bio); -void blk_integrity_verify_iter (struct bio *bio,struct bvec_iter *saved_iter); +void bio_integrity_generate (struct bio *bio); +blk_status_t bio_integrity_verify (struct bio *bio, +struct bvec_iter *saved_iter); +void blk_integrity_prepare (struct request *rq);void blk_integrity_complete (struct request *rq,unsigned int nr_bytes);
block/t10-pi.c +6 -6
@@ -372,7 +372,7 @@ static void ext_pi_type1_complete(struct request *rq, unsigned int nr_bytes) } } -void blk_integrity_generate (struct bio *bio) +void bio_integrity_generate (struct bio *bio) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_payload *bip =bio_integrity (bio);
@@ -404,7 +404,7 @@ void blk_integrity_generate(struct bio *bio) } } -void blk_integrity_verify_iter (struct bio *bio,struct bvec_iter *saved_iter) +blk_status_t bio_integrity_verify (struct bio *bio,struct bvec_iter *saved_iter) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_payload *bip =bio_integrity (bio);
@@ -439,11 +439,11 @@ void blk_integrity_verify_iter(struct bio *bio, struct bvec_iter *saved_iter) }kunmap_local (kaddr); -if (ret) { - bio->bi_status = ret; -return ; - } +if (ret) +return ret; } + +return BLK_STS_OK ; }void blk_integrity_prepare (struct request *rq)
Signature · 1 line
2.47.3
Re: [PATCH 04/15] block: prepare generation / verification helpers for fs usage
Quoted 5 lines · Christoph Hellwig
Return the status from verify instead of directly stashing it in the bio, and rename the helpers to use the usual
bio_prefix for things operating on a bio.Signed-off-by Christoph Hellwig <[email protected]>
LGTM,
--D
Quoted 87 lines
--- block/bio-integrity-auto.c | 4 ++-- block/blk.h | 6 ++++-- block/t10-pi.c | 12 ++++++------ 3 files changed, 12 insertions(+), 10 deletions(-)block/bio-integrity-auto.c +2
-2@@ -39,7 +39,7 @@ static void bio_integrity_verify_fn(struct work_struct *work) container_of(work, struct bio_integrity_data, work);struct bio *bio = bid->bio; -blk_integrity_verify_iter (bio, &bid->saved_bio_iter); + bio->bi_status =bio_integrity_verify (bio, &bid->saved_bio_iter);bio_integrity_finish (bid);bio_endio (bio); }@@ -100,7 +100,7 @@ void bio_integrity_prep(struct bio *bio, unsigned int action)/* Auto-generate integrity metadata if this is a write */ if (bio_data_dir (bio) ==WRITE &&bip_should_check (&bid->bip)) -blk_integrity_generate (bio); +bio_integrity_generate (bio);else bid->saved_bio_iter = bio->bi_iter; }block/blk.h +4
-2@@ -695,8 +695,10 @@ int bdev_open(struct block_device *bdev, blk_mode_t mode, void *holder, const struct blk_holder_ops *hops, struct file *bdev_file);int bdev_permission (dev_t dev,blk_mode_t mode,void *holder); -void blk_integrity_generate (struct bio *bio); -void blk_integrity_verify_iter (struct bio *bio,struct bvec_iter *saved_iter); +void bio_integrity_generate (struct bio *bio); +blk_status_t bio_integrity_verify (struct bio *bio, +struct bvec_iter *saved_iter); +void blk_integrity_prepare (struct request *rq);void blk_integrity_complete (struct request *rq,unsigned int nr_bytes);block/t10-pi.c +6
-6@@ -372,7 +372,7 @@ static void ext_pi_type1_complete(struct request *rq, unsigned int nr_bytes) } } -void blk_integrity_generate (struct bio *bio) +void bio_integrity_generate (struct bio *bio) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_payload *bip =bio_integrity (bio);@@ -404,7 +404,7 @@ void blk_integrity_generate(struct bio *bio) } } -void blk_integrity_verify_iter (struct bio *bio,struct bvec_iter *saved_iter) +blk_status_t bio_integrity_verify (struct bio *bio,struct bvec_iter *saved_iter) {struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk);struct bio_integrity_payload *bip =bio_integrity (bio);@@ -439,11 +439,11 @@ void blk_integrity_verify_iter(struct bio *bio, struct bvec_iter *saved_iter) }kunmap_local (kaddr); -if (ret) { - bio->bi_status = ret; -return ; - } +if (ret) +return ret; } + +return BLK_STS_OK ; }void blk_integrity_prepare (struct request *rq)Signature · 1 line
2.47.3
Re: [PATCH 04/15] block: prepare generation / verification helpers for fs usage
In reply to Christoph HellwigRe: [PATCH 04/15] block: prepare generation / verification helpers for fs usage
In reply to Christoph HellwigChristoph,
Quoted 3 lines
Return the status from verify instead of directly stashing it in the bio, and rename the helpers to use the usual
bio_prefix for things operating on a bio.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 04/15] block: prepare generation / verification helpers for fs usage
In reply to Christoph HellwigPATCH 5/15 block: make max_integrity_io_size public
In reply to Christoph HellwigFile systems that generate integrity will need this, so move it out of the block private or blk-mq specific headers.
--- block/blk-settings.c | 13 ------------- include/linux/blk-integrity.h | 5 ----- include/linux/blkdev.h | 18 ++++++++++++++++++ 3 files changed, 18 insertions(+), 18 deletions(-)
block/blk-settings.c +0 -13
@@ -123,19 +123,6 @@ static int blk_validate_zoned_limits(struct queue_limits *lim)return 0 ; } -/* -* Maximum size of I/O that needs a block layer integrity buffer. Limited -* by the number of intervals for which we can fit the integrity buffer into -* the buffer size. Because the buffer is a single segment it is also limited -* by the maximum segment size. -*/ -static inline unsigned int max_integrity_io_size (struct queue_limits *lim) -{ - return min_t(unsigned int, lim->max_segment_size, - (BLK_INTEGRITY_MAX_SIZE / lim->integrity.metadata_size) << - lim->integrity.interval_exp); -} -static int blk_validate_integrity_limits (struct queue_limits *lim) {struct blk_integrity *bi = &lim->integrity;
include/linux/blk-integrity.h +0 -5
@@ -8,11 +8,6 @@struct request ; -/* -* Maximum contiguous integrity buffer allocation. -*/ -#define BLK_INTEGRITY_MAX_SIZE SZ_2M -enum blk_integrity_flags {BLK_INTEGRITY_NOVERIFY =1 <<0 ,BLK_INTEGRITY_NOGENERATE =1 <<1 ,
include/linux/blkdev.h +18 -0
@@ -1875,6 +1875,24 @@ static inline int bio_split_rw_at(struct bio *bio, return bio_split_io_at(bio, lim, segs, max_bytes, lim->dma_alignment); } +/* +* Maximum contiguous integrity buffer allocation. +*/ +#define BLK_INTEGRITY_MAX_SIZE SZ_2M + +/* +* Maximum size of I/O that needs a block layer integrity buffer. Limited +* by the number of intervals for which we can fit the integrity buffer into +* the buffer size. Because the buffer is a single segment it is also limited +* by the maximum segment size. +*/ +static inline unsigned int max_integrity_io_size (struct queue_limits *lim) +{ + return min_t(unsigned int, lim->max_segment_size, + (BLK_INTEGRITY_MAX_SIZE / lim->integrity.metadata_size) << + lim->integrity.interval_exp); +} +#define DEFINE_IO_COMP_BATCH (name) struct io_comp_batch name = { }#endif /* _LINUX_BLKDEV_H */
Signature · 1 line
2.47.3
Re: [PATCH 05/15] block: make max_integrity_io_size public
Quoted 4 lines · Christoph Hellwig
File systems that generate integrity will need this, so move it out of the block private or blk-mq specific headers.
Signed-off-by Christoph Hellwig <[email protected]>
Seems reasonable to me given what I saw in the rest of the patchset...
--D
Quoted 79 lines
--- block/blk-settings.c | 13 ------------- include/linux/blk-integrity.h | 5 ----- include/linux/blkdev.h | 18 ++++++++++++++++++ 3 files changed, 18 insertions(+), 18 deletions(-)block/blk-settings.c +0
-13@@ -123,19 +123,6 @@ static int blk_validate_zoned_limits(struct queue_limits *lim)return 0 ; } -/* -* Maximum size of I/O that needs a block layer integrity buffer. Limited -* by the number of intervals for which we can fit the integrity buffer into -* the buffer size. Because the buffer is a single segment it is also limited -* by the maximum segment size. -*/ -static inline unsigned int max_integrity_io_size (struct queue_limits *lim) -{ - return min_t(unsigned int, lim->max_segment_size, - (BLK_INTEGRITY_MAX_SIZE / lim->integrity.metadata_size) << - lim->integrity.interval_exp); -} -static int blk_validate_integrity_limits (struct queue_limits *lim) {struct blk_integrity *bi = &lim->integrity;include/linux/blk-integrity.h +0
-5@@ -8,11 +8,6 @@struct request ; -/* -* Maximum contiguous integrity buffer allocation. -*/ -#define BLK_INTEGRITY_MAX_SIZE SZ_2M -enum blk_integrity_flags {BLK_INTEGRITY_NOVERIFY =1 <<0 ,BLK_INTEGRITY_NOGENERATE =1 <<1 ,include/linux/blkdev.h +18
-0@@ -1875,6 +1875,24 @@ static inline int bio_split_rw_at(struct bio *bio, return bio_split_io_at(bio, lim, segs, max_bytes, lim->dma_alignment); } +/* +* Maximum contiguous integrity buffer allocation. +*/ +#define BLK_INTEGRITY_MAX_SIZE SZ_2M + +/* +* Maximum size of I/O that needs a block layer integrity buffer. Limited +* by the number of intervals for which we can fit the integrity buffer into +* the buffer size. Because the buffer is a single segment it is also limited +* by the maximum segment size. +*/ +static inline unsigned int max_integrity_io_size (struct queue_limits *lim) +{ + return min_t(unsigned int, lim->max_segment_size, + (BLK_INTEGRITY_MAX_SIZE / lim->integrity.metadata_size) << + lim->integrity.interval_exp); +} +#define DEFINE_IO_COMP_BATCH (name) struct io_comp_batch name = { }#endif /* _LINUX_BLKDEV_H */ Signature · 1 line
2.47.3
Re: [PATCH 05/15] block: make max_integrity_io_size public
In reply to Christoph HellwigRe: [PATCH 05/15] block: make max_integrity_io_size public
In reply to Christoph HellwigChristoph,
Quoted 2 lines
File systems that generate integrity will need this, so move it out of the block private or blk-mq specific headers.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 05/15] block: make max_integrity_io_size public
In reply to Christoph HellwigPATCH 6/15 block: add fs_bio_integrity helpers
In reply to Christoph HellwigAdd a set of helpers for file system initiated integrity information. These include mempool backed allocations and verifying based on a passed in sector and size which is often available from file system completion routines.
--- block/Makefile | 2 +- block/bio-integrity-fs.c | 81 +++++++++++++++++++++++++++++++++++ include/linux/bio-integrity.h | 6 +++ 3 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 block/bio-integrity-fs.c
block/Makefile +1 -1
@@ -26,7 +26,7 @@ bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o obj-$(CONFIG_IOSCHED_BFQ) += bfq.o obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o blk-integrity.o t10-pi.o \ - bio-integrity-auto.o + bio-integrity-auto.o bio-integrity-fs.o obj-$(CONFIG_BLK_DEV_ZONED) += blk-zoned.o obj-$(CONFIG_BLK_WBT) += blk-wbt.o obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o
block/bio-integrity-fs.c new file +81 -0
81 lines not on this page · raw message
include/linux/bio-integrity.h +6 -0
@@ -145,4 +145,10 @@ void bio_integrity_alloc_buf(struct bio *bio, bool zero_buffer);void bio_integrity_free_buf (struct bio_integrity_payload *bip);void bio_integrity_setup_default (struct bio *bio); +void fs_bio_integrity_alloc (struct bio *bio); +void fs_bio_integrity_free (struct bio *bio); +void fs_bio_integrity_generate (struct bio *bio); +int fs_bio_integrity_verify (struct bio *bio,sector_t sector, +unsigned int size); +#endif /* _LINUX_BIO_INTEGRITY_H */
Signature · 1 line
2.47.3
Re: [PATCH 06/15] block: add fs_bio_integrity helpers
Quoted 81 lines · Christoph Hellwig
Add a set of helpers for file system initiated integrity information. These include mempool backed allocations and verifying based on a passed in sector and size which is often available from file system completion routines.
Signed-off-by Christoph Hellwig <[email protected]>--- block/Makefile | 2 +- block/bio-integrity-fs.c | 81 +++++++++++++++++++++++++++++++++++ include/linux/bio-integrity.h | 6 +++ 3 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 block/bio-integrity-fs.cblock/Makefile +1
-1@@ -26,7 +26,7 @@ bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o obj-$(CONFIG_IOSCHED_BFQ) += bfq.o obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o blk-integrity.o t10-pi.o \ - bio-integrity-auto.o + bio-integrity-auto.o bio-integrity-fs.o obj-$(CONFIG_BLK_DEV_ZONED) += blk-zoned.o obj-$(CONFIG_BLK_WBT) += blk-wbt.o obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.oblock/bio-integrity-fs.c new file +49
-0@@ -0,0 +1,81 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2025 Christoph Hellwig. +*/ +#include <linux/blk-integrity.h> +#include <linux/bio-integrity.h> +#include "blk.h" + +struct fs_bio_integrity_buf { +struct bio_integrity_payload bip; +struct bio_vec bvec; +}; + +static struct kmem_cache *fs_bio_integrity_cache; +static mempool_t fs_bio_integrity_pool; + +void fs_bio_integrity_alloc (struct bio *bio) +{ +struct fs_bio_integrity_buf *iib; +unsigned int action; + + action =bio_integrity_action (bio); +if (!action) +return ; + + iib =mempool_alloc (&fs_bio_integrity_pool,GFP_NOIO ); +bio_integrity_init (bio, &iib->bip, &iib->bvec,1 ); + +bio_integrity_alloc_buf (bio, action &BI_ACT_ZERO ); +if (action &BI_ACT_CHECK ) +bio_integrity_setup_default (bio); +} + +void fs_bio_integrity_free (struct bio *bio) +{ +struct bio_integrity_payload *bip =bio_integrity (bio); + +bio_integrity_free_buf (bip); + mempool_free(container_of(bip, struct fs_bio_integrity_buf, bip), + &fs_bio_integrity_pool); + + bio->bi_integrity =NULL ; + bio->bi_opf &= ~REQ_INTEGRITY ; +} + +void fs_bio_integrity_generate (struct bio *bio) +{ +fs_bio_integrity_alloc (bio); +bio_integrity_generate (bio);
block/bio-integrity-fs.c line 59
Quoted 10 lines
block/bio-integrity-fs.c new file +10
-0+} +EXPORT_SYMBOL_GPL (fs_bio_integrity_generate); + +int fs_bio_integrity_verify (struct bio *bio,sector_t sector,unsigned int size) +{ +struct blk_integrity *bi =blk_get_integrity (bio->bi_bdev->bd_disk); +struct bio_integrity_payload *bip =bio_integrity (bio); + +/* +* Reinitialize bip->bit_iter.
s/bit_iter/bip_iter/ ?
With that fixed, this looks fine to me;
--D
Quoted 41 lines
block/bio-integrity-fs.c new file +22
-0+* +* This is for use in the submitter after the driver is done with the +* bio. Requires the submitter to remember the sector and the size. +*/ + +memset (&bip->bip_iter,0 ,sizeof (bip->bip_iter)); + bip->bip_iter.bi_sector = sector; + bip->bip_iter.bi_size =bio_integrity_bytes (bi, size >>SECTOR_SHIFT ); +return blk_status_to_errno (bio_integrity_verify (bio, &bip->bip_iter)); +} + +static int __init fs_bio_integrity_init (void ) +{ + fs_bio_integrity_cache =kmem_cache_create ("fs_bio_integrity" , +sizeof (struct fs_bio_integrity_buf ),0 , +SLAB_HWCACHE_ALIGN |SLAB_PANIC ,NULL ); +if (mempool_init_slab_pool (&fs_bio_integrity_pool,BIO_POOL_SIZE , + fs_bio_integrity_cache)) +panic ("fs_bio_integrity: can't create pool\n" ); +return 0 ; +} +fs_initcall (fs_bio_integrity_init);include/linux/bio-integrity.h +6
-0@@ -145,4 +145,10 @@ void bio_integrity_alloc_buf(struct bio *bio, bool zero_buffer);void bio_integrity_free_buf (struct bio_integrity_payload *bip);void bio_integrity_setup_default (struct bio *bio); +void fs_bio_integrity_alloc (struct bio *bio); +void fs_bio_integrity_free (struct bio *bio); +void fs_bio_integrity_generate (struct bio *bio); +int fs_bio_integrity_verify (struct bio *bio,sector_t sector, +unsigned int size); +#endif /* _LINUX_BIO_INTEGRITY_H */ Signature · 1 line
2.47.3
Re: [PATCH 06/15] block: add fs_bio_integrity helpers
In reply to Christoph Hellwigblock/bio-integrity-fs.c line 24
Quoted 8 lines · Christoph Hellwig
block/bio-integrity-fs.c new file +8
-0+void fs_bio_integrity_alloc (struct bio *bio) +{ +struct fs_bio_integrity_buf *iib; +unsigned int action; + + action =bio_integrity_action (bio); +if (!action) +return ;
So this may return from here, but <below>
Quoted 14 lines
block/bio-integrity-fs.c new file +14
-0+ + iib =mempool_alloc (&fs_bio_integrity_pool,GFP_NOIO ); +bio_integrity_init (bio, &iib->bip, &iib->bvec,1 ); + +bio_integrity_alloc_buf (bio, action &BI_ACT_ZERO ); +if (action &BI_ACT_CHECK ) +bio_integrity_setup_default (bio); +} + +void fs_bio_integrity_free (struct bio *bio) +{ +struct bio_integrity_payload *bip =bio_integrity (bio); + +bio_integrity_free_buf (bip);
block/bio-integrity-fs.c line 48
Quoted 10 lines
block/bio-integrity-fs.c new file +10
-0+ mempool_free(container_of(bip, struct fs_bio_integrity_buf, bip), + &fs_bio_integrity_pool); + + bio->bi_integrity =NULL ; + bio->bi_opf &= ~REQ_INTEGRITY ; +} + +void fs_bio_integrity_generate (struct bio *bio) +{ +fs_bio_integrity_alloc (bio);
no check here. A potential null pointer deference in the next line as bio has no bip?
Quoted 2 lines
(quoted diff) +2
-0+ bio_integrity_generate(bio); +}
Re: [PATCH 06/15] block: add fs_bio_integrity helpers
Quoted 12 lines · Kanchan Joshi
block/bio-integrity-fs.c line 24
Quoted 8 lines · Christoph Hellwig
block/bio-integrity-fs.c new file +8
-0+void fs_bio_integrity_alloc (struct bio *bio) +{ +struct fs_bio_integrity_buf *iib; +unsigned int action; + + action =bio_integrity_action (bio); +if (!action) +return ;So this may return from here, but <below>
Quoted 8 lines
block/bio-integrity-fs.c line 48
Quoted 3 lines
block/bio-integrity-fs.c new file +3
-0+void fs_bio_integrity_generate (struct bio *bio) +{ +fs_bio_integrity_alloc (bio);no check here. A potential null pointer deference in the next line as bio has no bip?
Review
Quoted 2 lines
(quoted diff) +2
-0+ bio_integrity_generate(bio); +}
fs_bio_integrity_alloc is only called when the device has PI metadata with checksums. So this case can't really happen. That being said, handling it in one case and not the other seems suboptimal and confusing.
Re: [PATCH 06/15] block: add fs_bio_integrity helpers
In reply to Christoph HellwigChristoph,
Quoted 4 lines
Add a set of helpers for file system initiated integrity information. These include mempool backed allocations and verifying based on a passed in sector and size which is often available from file system completion routines.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 06/15] block: add fs_bio_integrity helpers
In reply to Christoph HellwigPATCH 7/15 block: pass a maxlen argument to bio_iov_iter_bounce
In reply to Christoph HellwigAllow the file system to limit the size processed in a single bounce operation. This is needed when generating integrity data so that the size of a single integrity segment can't overflow.
--- block/bio.c | 17 ++++++++++------- fs/iomap/direct-io.c | 2 +- include/linux/bio.h | 2 +- 3 files changed, 12 insertions(+), 9 deletions(-)
block/bio.c +10 -7
@@ -1293,9 +1293,10 @@ static void bio_free_folios(struct bio *bio) } } -static int bio_iov_iter_bounce_write (struct bio *bio,struct iov_iter *iter) +static int bio_iov_iter_bounce_write (struct bio *bio,struct iov_iter *iter, +size_t maxlen) { -size_t total_len =iov_iter_count (iter); +size_t total_len =min (maxlen,iov_iter_count (iter));if (WARN_ON_ONCE (bio_flagged (bio,BIO_CLONED )))return -EINVAL ;
@@ -1333,9 +1334,10 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter)return 0 ; } -static int bio_iov_iter_bounce_read (struct bio *bio,struct iov_iter *iter) +static int bio_iov_iter_bounce_read (struct bio *bio,struct iov_iter *iter, +size_t maxlen) { -size_t len =min (iov_iter_count (iter),SZ_1M ); +size_t len =min3 (iov_iter_count (iter), maxlen,SZ_1M );struct folio *folio; folio =folio_alloc_greedy (GFP_KERNEL , &len);
@@ -1372,6 +1374,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter)* bio_iov_iter_bounce - bounce buffer data from an iter into a bio * @bio: bio to send * @iter: iter to read from / write into +* @maxlen: maximum size to bounce * * Helper for direct I/O implementations that need to bounce buffer because * we need to checksum the data or perform other operations that require
@@ -1379,11 +1382,11 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter)* copies the data into it. Needs to be paired with bio_iov_iter_unbounce() * called on completion. */ -int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter) +int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter,size_t maxlen) {if (op_is_write (bio_op (bio))) -return bio_iov_iter_bounce_write (bio, iter); -return bio_iov_iter_bounce_read (bio, iter); +return bio_iov_iter_bounce_write (bio, iter, maxlen); +return bio_iov_iter_bounce_read (bio, iter, maxlen); }static void bvec_unpin (struct bio_vec *bv,bool mark_dirty)
fs/iomap/direct-io.c +1 -1
@@ -326,7 +326,7 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, bio->bi_end_io = iomap_dio_bio_end_io;if (dio->flags &IOMAP_DIO_BOUNCE ) - ret =bio_iov_iter_bounce (bio, dio->submit.iter); + ret =bio_iov_iter_bounce (bio, dio->submit.iter,UINT_MAX );else ret =bio_iov_iter_get_pages (bio, dio->submit.iter, alignment -1 );
include/linux/bio.h +1 -1
@@ -479,7 +479,7 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty);extern void bio_set_pages_dirty (struct bio *bio);extern void bio_check_pages_dirty (struct bio *bio); -int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter); +int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter,size_t maxlen);void bio_iov_iter_unbounce (struct bio *bio,bool is_error,bool mark_dirty); extern void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,
Signature · 1 line
2.47.3
Re: [PATCH 07/15] block: pass a maxlen argument to bio_iov_iter_bounce
Quoted 69 lines · Christoph Hellwig
Allow the file system to limit the size processed in a single bounce operation. This is needed when generating integrity data so that the size of a single integrity segment can't overflow.
Signed-off-by Christoph Hellwig <[email protected]>--- block/bio.c | 17 ++++++++++------- fs/iomap/direct-io.c | 2 +- include/linux/bio.h | 2 +- 3 files changed, 12 insertions(+), 9 deletions(-)block/bio.c +10
-7@@ -1293,9 +1293,10 @@ static void bio_free_folios(struct bio *bio) } } -static int bio_iov_iter_bounce_write (struct bio *bio,struct iov_iter *iter) +static int bio_iov_iter_bounce_write (struct bio *bio,struct iov_iter *iter, +size_t maxlen) { -size_t total_len =iov_iter_count (iter); +size_t total_len =min (maxlen,iov_iter_count (iter));if (WARN_ON_ONCE (bio_flagged (bio,BIO_CLONED )))return -EINVAL ;@@ -1333,9 +1334,10 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter)return 0 ; } -static int bio_iov_iter_bounce_read (struct bio *bio,struct iov_iter *iter) +static int bio_iov_iter_bounce_read (struct bio *bio,struct iov_iter *iter, +size_t maxlen) { -size_t len =min (iov_iter_count (iter),SZ_1M ); +size_t len =min3 (iov_iter_count (iter), maxlen,SZ_1M );struct folio *folio; folio =folio_alloc_greedy (GFP_KERNEL , &len);@@ -1372,6 +1374,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter)* bio_iov_iter_bounce - bounce buffer data from an iter into a bio * @bio: bio to send * @iter: iter to read from / write into +* @maxlen: maximum size to bounce * * Helper for direct I/O implementations that need to bounce buffer because * we need to checksum the data or perform other operations that require @@ -1379,11 +1382,11 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter)* copies the data into it. Needs to be paired with bio_iov_iter_unbounce() * called on completion. */ -int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter) +int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter,size_t maxlen) {if (op_is_write (bio_op (bio))) -return bio_iov_iter_bounce_write (bio, iter); -return bio_iov_iter_bounce_read (bio, iter); +return bio_iov_iter_bounce_write (bio, iter, maxlen); +return bio_iov_iter_bounce_read (bio, iter, maxlen); }static void bvec_unpin (struct bio_vec *bv,bool mark_dirty)fs/iomap/direct-io.c +0
-0
fs/iomap/direct-io.c static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, · line 329
Quoted 5 lines
fs/iomap/direct-io.c +1
-1@@ -326,7 +326,7 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, bio->bi_end_io = iomap_dio_bio_end_io;if (dio->flags &IOMAP_DIO_BOUNCE ) - ret =bio_iov_iter_bounce (bio, dio->submit.iter); + ret =bio_iov_iter_bounce (bio, dio->submit.iter,UINT_MAX );
Nitpicking here, but shouldn't this be SIZE_MAX?
--D
Quoted 20 lines
fs/iomap/direct-io.c +0
-0else ret =bio_iov_iter_get_pages (bio, dio->submit.iter, alignment -1 );include/linux/bio.h +1
-1@@ -479,7 +479,7 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty);extern void bio_set_pages_dirty (struct bio *bio);extern void bio_check_pages_dirty (struct bio *bio); -int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter); +int bio_iov_iter_bounce (struct bio *bio,struct iov_iter *iter,size_t maxlen);void bio_iov_iter_unbounce (struct bio *bio,bool is_error,bool mark_dirty); extern void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,Signature · 1 line
2.47.3
Re: [PATCH 07/15] block: pass a maxlen argument to bio_iov_iter_bounce
Quoted 5 lines · Darrick J. Wong
fs/iomap/direct-io.c static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, · line 329
Quoted 3 lines
fs/iomap/direct-io.c +1
-1if (dio->flags &IOMAP_DIO_BOUNCE ) - ret =bio_iov_iter_bounce (bio, dio->submit.iter); + ret =bio_iov_iter_bounce (bio, dio->submit.iter,UINT_MAX );Nitpicking here, but shouldn't this be
SIZE_MAX?
bi_size can't store more than UINT_MAX, so I think this is correct.
Darrick J. Wong
Reviewed-byRe: [PATCH 07/15] block: pass a maxlen argument to bio_iov_iter_bounce
Quoted 8 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
fs/iomap/direct-io.c static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, · line 329
Quoted 3 lines
fs/iomap/direct-io.c +1
-1if (dio->flags &IOMAP_DIO_BOUNCE ) - ret =bio_iov_iter_bounce (bio, dio->submit.iter); + ret =bio_iov_iter_bounce (bio, dio->submit.iter,UINT_MAX );Nitpicking here, but shouldn't this be
SIZE_MAX?
bi_sizecan't store more thanUINT_MAX, so I think this is correct.
Fair enough, I was just surprised that this later gets turned into a min() involving SIZE_MAX.
Practically speaking, they're the same value so it doesn't matter much.
--D
Re: [PATCH 07/15] block: pass a maxlen argument to bio_iov_iter_bounce
In reply to Christoph HellwigChristoph,
Quoted 3 lines
Allow the file system to limit the size processed in a single bounce operation. This is needed when generating integrity data so that the size of a single integrity segment can't overflow.
Signature · 1 line
Martin K. Petersen
Re: [PATCH 07/15] block: pass a maxlen argument to bio_iov_iter_bounce
In reply to Christoph HellwigPATCH 8/15 iomap: refactor iomap_bio_read_folio_range
In reply to Christoph HellwigSplit out the logic to allocate a new bio and only keep the fast path that adds more data to an existing bio in iomap_bio_read_folio_range.
--- fs/iomap/bio.c | 69 +++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 32 deletions(-)
fs/iomap/bio.c +37 -32
@@ -26,45 +26,50 @@ static void iomap_bio_submit_read(struct iomap_read_folio_ctx *ctx)submit_bio (bio); } -static int iomap_bio_read_folio_range (const struct iomap_iter *iter, +static void iomap_read_alloc_bio (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx,size_t plen) { -struct folio *folio = ctx->cur_folio;const struct iomap *iomap = &iter->iomap; -loff_t pos = iter->pos; -size_t poff =offset_in_folio (folio, pos); -loff_t length =iomap_length (iter); -sector_t sector; +unsigned int nr_vecs =DIV_ROUND_UP (iomap_length (iter),PAGE_SIZE ); +struct folio *folio = ctx->cur_folio; +gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL ); +gfp_t orig_gfp = gfp;struct bio *bio = ctx->read_ctx; - sector =iomap_sector (iomap, pos); -if (!bio ||bio_end_sector (bio) != sector || - !bio_add_folio (bio, folio, plen, poff)) { -gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL ); -gfp_t orig_gfp = gfp; -unsigned int nr_vecs =DIV_ROUND_UP (length,PAGE_SIZE ); +if (bio) +submit_bio (bio); + +/* Same as readahead_gfp_mask: */ +if (ctx->rac) + gfp |= __GFP_NORETRY | __GFP_NOWARN; -if (bio) -submit_bio (bio); +/* +* If the bio_alloc fails, try it again for a single page to avoid +* having to deal with partial page reads. This emulates what +* do_mpage_read_folio does. +*/ + bio =bio_alloc (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , gfp); +if (!bio) + bio =bio_alloc (iomap->bdev,1 ,REQ_OP_READ , orig_gfp); +if (ctx->rac) + bio->bi_opf |=REQ_RAHEAD ; + bio->bi_iter.bi_sector =iomap_sector (iomap, iter->pos); + bio->bi_end_io = iomap_read_end_io; +bio_add_folio_nofail (bio, folio, plen, +offset_in_folio (folio, iter->pos)); + ctx->read_ctx = bio; +} + +static int iomap_bio_read_folio_range (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx,size_t plen) +{ +struct folio *folio = ctx->cur_folio; +struct bio *bio = ctx->read_ctx; -if (ctx->rac)/* same as readahead_gfp_mask */ - gfp |= __GFP_NORETRY | __GFP_NOWARN; - bio =bio_alloc (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , - gfp); -/* -* If the bio_alloc fails, try it again for a single page to -* avoid having to deal with partial page reads. This emulates -* what do_mpage_read_folio does. -*/ -if (!bio) - bio =bio_alloc (iomap->bdev,1 ,REQ_OP_READ , orig_gfp); -if (ctx->rac) - bio->bi_opf |=REQ_RAHEAD ; - bio->bi_iter.bi_sector = sector; - bio->bi_end_io = iomap_read_end_io; -bio_add_folio_nofail (bio, folio, plen, poff); - ctx->read_ctx = bio; - } +if (!bio || +bio_end_sector (bio) !=iomap_sector (&iter->iomap, iter->pos) || + !bio_add_folio (bio, folio, plen,offset_in_folio (folio, iter->pos))) +iomap_read_alloc_bio (iter, ctx, plen);return 0 ; }
Signature · 1 line
2.47.3
Re: [PATCH 08/15] iomap: refactor iomap_bio_read_folio_range
Quoted 4 lines · Christoph Hellwig
Split out the logic to allocate a new bio and only keep the fast path that adds more data to an existing bio in
iomap_bio_read_folio_range.Signed-off-by Christoph Hellwig <[email protected]>
Looks like a simple function split to me...
--D
Quoted 95 lines
--- fs/iomap/bio.c | 69 +++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 32 deletions(-)fs/iomap/bio.c +37
-32@@ -26,45 +26,50 @@ static void iomap_bio_submit_read(struct iomap_read_folio_ctx *ctx)submit_bio (bio); } -static int iomap_bio_read_folio_range (const struct iomap_iter *iter, +static void iomap_read_alloc_bio (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx,size_t plen) { -struct folio *folio = ctx->cur_folio;const struct iomap *iomap = &iter->iomap; -loff_t pos = iter->pos; -size_t poff =offset_in_folio (folio, pos); -loff_t length =iomap_length (iter); -sector_t sector; +unsigned int nr_vecs =DIV_ROUND_UP (iomap_length (iter),PAGE_SIZE ); +struct folio *folio = ctx->cur_folio; +gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL ); +gfp_t orig_gfp = gfp;struct bio *bio = ctx->read_ctx; - sector =iomap_sector (iomap, pos); -if (!bio ||bio_end_sector (bio) != sector || - !bio_add_folio (bio, folio, plen, poff)) { -gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL ); -gfp_t orig_gfp = gfp; -unsigned int nr_vecs =DIV_ROUND_UP (length,PAGE_SIZE ); +if (bio) +submit_bio (bio); + +/* Same as readahead_gfp_mask: */ +if (ctx->rac) + gfp |= __GFP_NORETRY | __GFP_NOWARN; -if (bio) -submit_bio (bio); +/* +* If the bio_alloc fails, try it again for a single page to avoid +* having to deal with partial page reads. This emulates what +* do_mpage_read_folio does. +*/ + bio =bio_alloc (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , gfp); +if (!bio) + bio =bio_alloc (iomap->bdev,1 ,REQ_OP_READ , orig_gfp); +if (ctx->rac) + bio->bi_opf |=REQ_RAHEAD ; + bio->bi_iter.bi_sector =iomap_sector (iomap, iter->pos); + bio->bi_end_io = iomap_read_end_io; +bio_add_folio_nofail (bio, folio, plen, +offset_in_folio (folio, iter->pos)); + ctx->read_ctx = bio; +} + +static int iomap_bio_read_folio_range (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx,size_t plen) +{ +struct folio *folio = ctx->cur_folio; +struct bio *bio = ctx->read_ctx; -if (ctx->rac)/* same as readahead_gfp_mask */ - gfp |= __GFP_NORETRY | __GFP_NOWARN; - bio =bio_alloc (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , - gfp); -/* -* If the bio_alloc fails, try it again for a single page to -* avoid having to deal with partial page reads. This emulates -* what do_mpage_read_folio does. -*/ -if (!bio) - bio =bio_alloc (iomap->bdev,1 ,REQ_OP_READ , orig_gfp); -if (ctx->rac) - bio->bi_opf |=REQ_RAHEAD ; - bio->bi_iter.bi_sector = sector; - bio->bi_end_io = iomap_read_end_io; -bio_add_folio_nofail (bio, folio, plen, poff); - ctx->read_ctx = bio; - } +if (!bio || +bio_end_sector (bio) !=iomap_sector (&iter->iomap, iter->pos) || + !bio_add_folio (bio, folio, plen,offset_in_folio (folio, iter->pos))) +iomap_read_alloc_bio (iter, ctx, plen);return 0 ; }Signature · 1 line
2.47.3
PATCH 9/15 iomap: pass the iomap_iter to ->submit_read
In reply to Christoph HellwigThis provides additional context for file systems.
Rename the fuse instance to match the method name while we're at it.
--- fs/fuse/file.c | 5 +++-- fs/iomap/bio.c | 3 ++- fs/iomap/buffered-io.c | 4 ++-- include/linux/iomap.h | 3 ++- 4 files changed, 9 insertions(+), 6 deletions(-)
fs/fuse/file.c +3 -2
@@ -947,7 +947,8 @@ static int fuse_iomap_read_folio_range_async(const struct iomap_iter *iter,return ret; } -static void fuse_iomap_read_submit (struct iomap_read_folio_ctx *ctx) +static void fuse_iomap_submit_read (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx) {struct fuse_fill_read_data *data = ctx->read_ctx;
@@ -958,7 +959,7 @@ static void fuse_iomap_read_submit(struct iomap_read_folio_ctx *ctx)static const struct iomap_read_ops fuse_iomap_read_ops = { .read_folio_range = fuse_iomap_read_folio_range_async, - .submit_read = fuse_iomap_read_submit, + .submit_read = fuse_iomap_submit_read, };static int fuse_read_folio (struct file *file,struct folio *folio)
fs/iomap/bio.c +2 -1
@@ -18,7 +18,8 @@ static void iomap_read_end_io(struct bio *bio)bio_put (bio); } -static void iomap_bio_submit_read (struct iomap_read_folio_ctx *ctx) +static void iomap_bio_submit_read (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx) {struct bio *bio = ctx->read_ctx;
fs/iomap/buffered-io.c +2 -2
@@ -573,7 +573,7 @@ void iomap_read_folio(const struct iomap_ops *ops, &bytes_submitted);if (ctx->ops->submit_read) - ctx->ops->submit_read (ctx); + ctx->ops->submit_read (&iter, ctx);iomap_read_end (folio, bytes_submitted); }
@@ -637,7 +637,7 @@ void iomap_readahead(const struct iomap_ops *ops, &cur_bytes_submitted);if (ctx->ops->submit_read) - ctx->ops->submit_read (ctx); + ctx->ops->submit_read (&iter, ctx);if (ctx->cur_folio)iomap_read_end (ctx->cur_folio, cur_bytes_submitted);
include/linux/iomap.h +2 -1
@@ -512,7 +512,8 @@ struct iomap_read_ops {* * This is optional. */ -void (*submit_read)(struct iomap_read_folio_ctx *ctx); +void (*submit_read)(const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx); };/*
Signature · 1 line
2.47.3
Re: [PATCH 09/15] iomap: pass the iomap_iter to ->submit_read
Quoted 5 lines · Christoph Hellwig
This provides additional context for file systems.
Rename the fuse instance to match the method name while we're at it.
Signed-off-by Christoph Hellwig <[email protected]>
Looks fine, will probably have more comments when we start accessing the iter...
--D
Quoted 84 lines
--- fs/fuse/file.c | 5 +++-- fs/iomap/bio.c | 3 ++- fs/iomap/buffered-io.c | 4 ++-- include/linux/iomap.h | 3 ++- 4 files changed, 9 insertions(+), 6 deletions(-)fs/fuse/file.c +3
-2@@ -947,7 +947,8 @@ static int fuse_iomap_read_folio_range_async(const struct iomap_iter *iter,return ret; } -static void fuse_iomap_read_submit (struct iomap_read_folio_ctx *ctx) +static void fuse_iomap_submit_read (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx) {struct fuse_fill_read_data *data = ctx->read_ctx;@@ -958,7 +959,7 @@ static void fuse_iomap_read_submit(struct iomap_read_folio_ctx *ctx)static const struct iomap_read_ops fuse_iomap_read_ops = { .read_folio_range = fuse_iomap_read_folio_range_async, - .submit_read = fuse_iomap_read_submit, + .submit_read = fuse_iomap_submit_read, };static int fuse_read_folio (struct file *file,struct folio *folio)fs/iomap/bio.c +2
-1@@ -18,7 +18,8 @@ static void iomap_read_end_io(struct bio *bio)bio_put (bio); } -static void iomap_bio_submit_read (struct iomap_read_folio_ctx *ctx) +static void iomap_bio_submit_read (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx) {struct bio *bio = ctx->read_ctx;fs/iomap/buffered-io.c +2
-2@@ -573,7 +573,7 @@ void iomap_read_folio(const struct iomap_ops *ops, &bytes_submitted);if (ctx->ops->submit_read) - ctx->ops->submit_read (ctx); + ctx->ops->submit_read (&iter, ctx);iomap_read_end (folio, bytes_submitted); }@@ -637,7 +637,7 @@ void iomap_readahead(const struct iomap_ops *ops, &cur_bytes_submitted);if (ctx->ops->submit_read) - ctx->ops->submit_read (ctx); + ctx->ops->submit_read (&iter, ctx);if (ctx->cur_folio)iomap_read_end (ctx->cur_folio, cur_bytes_submitted);include/linux/iomap.h +2
-1@@ -512,7 +512,8 @@ struct iomap_read_ops {* * This is optional. */ -void (*submit_read)(struct iomap_read_folio_ctx *ctx); +void (*submit_read)(const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx); };/* Signature · 1 line
2.47.3
PATCH 10/15 iomap: only call into ->submit_read when there is a read_ctx
In reply to Christoph HellwigMove the NULL check into the callers to simplify the callees. Not sure how fuse worked before, given that it was missing the NULL check.
--- fs/iomap/bio.c | 5 +---- fs/iomap/buffered-io.c | 4 ++-- 2 files changed, 3 insertions(+), 6 deletions(-)
fs/iomap/bio.c +1 -4
@@ -21,10 +21,7 @@ static void iomap_read_end_io(struct bio *bio)static void iomap_bio_submit_read (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx) { -struct bio *bio = ctx->read_ctx; - -if (bio) -submit_bio (bio); +submit_bio (ctx->read_ctx); } static void iomap_read_alloc_bio(const struct iomap_iter *iter,
fs/iomap/buffered-io.c +2 -2
@@ -572,7 +572,7 @@ void iomap_read_folio(const struct iomap_ops *ops, iter.status =iomap_read_folio_iter (&iter, ctx, &bytes_submitted); -if (ctx->ops->submit_read) +if (ctx->read_ctx && ctx->ops->submit_read) ctx->ops->submit_read (&iter, ctx);iomap_read_end (folio, bytes_submitted);
@@ -636,7 +636,7 @@ void iomap_readahead(const struct iomap_ops *ops, iter.status =iomap_readahead_iter (&iter, ctx, &cur_bytes_submitted); -if (ctx->ops->submit_read) +if (ctx->read_ctx && ctx->ops->submit_read) ctx->ops->submit_read (&iter, ctx);if (ctx->cur_folio)
Signature · 1 line
2.47.3
Re: [PATCH 10/15] iomap: only call into ->submit_read when there is a read_ctx
Quoted 2 lines · Christoph Hellwig
Move the NULL check into the callers to simplify the callees. Not sure how fuse worked before, given that it was missing the NULL check.
Let's cc Joanne to find out then...? [done]
--D
Quoted 48 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/bio.c | 5 +---- fs/iomap/buffered-io.c | 4 ++-- 2 files changed, 3 insertions(+), 6 deletions(-)fs/iomap/bio.c +1
-4@@ -21,10 +21,7 @@ static void iomap_read_end_io(struct bio *bio)static void iomap_bio_submit_read (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx) { -struct bio *bio = ctx->read_ctx; - -if (bio) -submit_bio (bio); +submit_bio (ctx->read_ctx); } static void iomap_read_alloc_bio(const struct iomap_iter *iter,fs/iomap/buffered-io.c +2
-2@@ -572,7 +572,7 @@ void iomap_read_folio(const struct iomap_ops *ops, iter.status =iomap_read_folio_iter (&iter, ctx, &bytes_submitted); -if (ctx->ops->submit_read) +if (ctx->read_ctx && ctx->ops->submit_read) ctx->ops->submit_read (&iter, ctx);iomap_read_end (folio, bytes_submitted);@@ -636,7 +636,7 @@ void iomap_readahead(const struct iomap_ops *ops, iter.status =iomap_readahead_iter (&iter, ctx, &cur_bytes_submitted); -if (ctx->ops->submit_read) +if (ctx->read_ctx && ctx->ops->submit_read) ctx->ops->submit_read (&iter, ctx);if (ctx->cur_folio)Signature · 1 line
2.47.3
Joanne Koong
Reviewed-byRe: [PATCH 10/15] iomap: only call into ->submit_read when there is a read_ctx
Quoted 4 lines · Darrick J. Wong
Quoted 2 lines · Christoph Hellwig
Move the NULL check into the callers to simplify the callees. Not sure how fuse worked before, given that it was missing the NULL check.
In fuse, ctx->read_ctx is always valid. It gets initialized to point to a local struct before it calls into iomap_read_folio()/iomap_readahead()
Quoted 39 lines
Let's cc Joanne to find out then...? [done]
--D
Quoted 34 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/bio.c | 5 +---- fs/iomap/buffered-io.c | 4 ++-- 2 files changed, 3 insertions(+), 6 deletions(-)fs/iomap/bio.c +1
-4@@ -21,10 +21,7 @@ static void iomap_read_end_io(struct bio *bio)static void iomap_bio_submit_read (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx) { -struct bio *bio = ctx->read_ctx; - -if (bio) -submit_bio (bio); +submit_bio (ctx->read_ctx); } static void iomap_read_alloc_bio(const struct iomap_iter *iter,fs/iomap/buffered-io.c +1
-1@@ -572,7 +572,7 @@ void iomap_read_folio(const struct iomap_ops *ops, iter.status =iomap_read_folio_iter (&iter, ctx, &bytes_submitted); -if (ctx->ops->submit_read) +if (ctx->read_ctx && ctx->ops->submit_read) ctx->ops->submit_read (&iter, ctx);
Quoted 8 lines
fs/iomap/buffered-io.c void iomap_readahead(const struct iomap_ops *ops, · line 640
Quoted 8 lines
fs/iomap/buffered-io.c +1
-1iomap_read_end (folio, bytes_submitted);@@ -636,7 +636,7 @@ void iomap_readahead(const struct iomap_ops *ops, iter.status =iomap_readahead_iter (&iter, ctx, &cur_bytes_submitted); -if (ctx->ops->submit_read) +if (ctx->read_ctx && ctx->ops->submit_read) ctx->ops->submit_read (&iter, ctx);
I wonder if it makes sense to just have submit_read() take in the void *read_ctx directly instead of it taking in the entire struct iomap_read_folio_ctx. afaict, the other fields aren't used/necessary.
This patch as-is makes sense to me though.
Thanks,
Joanne
Quoted 6 lines
Quoted 6 lines
fs/iomap/buffered-io.c +0
-1if (ctx->cur_folio) --2.47.3
Re: [PATCH 10/15] iomap: only call into ->submit_read when there is a read_ctx
Quoted 9 lines · Joanne Koong
Quoted 4 lines · Darrick J. Wong
Quoted 2 lines · Christoph Hellwig
Move the NULL check into the callers to simplify the callees. Not sure how fuse worked before, given that it was missing the NULL check.
In fuse,
ctx->read_ctxis always valid. It gets initialized to point to a local struct before it calls into iomap_read_folio()/iomap_readahead()
Ok, thanks.
Darrick J. Wong
Reviewed-byRe: [PATCH 10/15] iomap: only call into ->submit_read when there is a read_ctx
Quoted 4 lines · Christoph Hellwig
Quoted 3 lines · Joanne Koong
Quoted 2 lines · Darrick J. Wong
On Wed, Jan 21, 2026 at 07:43:18AM +0100, Christoph Hellwig wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Move the NULL check into the callers to simplify the callees. Not sure how fuse worked before, given that it was missing the NULL check.
In fuse,
ctx->read_ctxis always valid. It gets initialized to point to a local struct before it calls into iomap_read_folio()/iomap_readahead()Ok, thanks.
Looks good to me then,
--D
PATCH 11/15 iomap: allow file systems to hook into buffered read bio submission
In reply to Christoph HellwigFiles ystems such as btrfs have additional operations with bios such as verifying data checksums. Allow file systems to hook into submission of the bio to allow for this processing by replacing the direct submit_bio call in iomap_read_alloc_bio with a call into ->submit_read and exporting iomap_read_alloc_bio. Also add a new field to struct iomap_read_folio_ctx to track the file logic offset of the current read context.
Based on a patch from Goldwyn Rodrigues <[email protected]>.
--- fs/iomap/bio.c | 15 +++++++++------ include/linux/iomap.h | 4 ++++ 2 files changed, 13 insertions(+), 6 deletions(-)
fs/iomap/bio.c +9 -6
@@ -32,10 +32,11 @@ static void iomap_read_alloc_bio(const struct iomap_iter *iter,struct folio *folio = ctx->cur_folio;gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL );gfp_t orig_gfp = gfp; -struct bio *bio = ctx->read_ctx; +struct bio *bio; -if (bio) -submit_bio (bio); +/* Submit the existing range if there was one. */ +if (ctx->read_ctx) + ctx->ops->submit_read (iter, ctx);/* Same as readahead_gfp_mask: */ if (ctx->rac)
@@ -56,9 +57,10 @@ static void iomap_read_alloc_bio(const struct iomap_iter *iter,bio_add_folio_nofail (bio, folio, plen,offset_in_folio (folio, iter->pos)); ctx->read_ctx = bio; + ctx->read_ctx_file_offset = iter->pos; } -static int iomap_bio_read_folio_range (const struct iomap_iter *iter, +int iomap_bio_read_folio_range (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx,size_t plen) {struct folio *folio = ctx->cur_folio;
@@ -70,10 +72,11 @@ static int iomap_bio_read_folio_range(const struct iomap_iter *iter,iomap_read_alloc_bio (iter, ctx, plen);return 0 ; } +EXPORT_SYMBOL_GPL (iomap_bio_read_folio_range);const struct iomap_read_ops iomap_bio_read_ops = { - .read_folio_range = iomap_bio_read_folio_range, - .submit_read = iomap_bio_submit_read, + .read_folio_range = iomap_bio_read_folio_range, + .submit_read = iomap_bio_submit_read, };EXPORT_SYMBOL_GPL (iomap_bio_read_ops);
include/linux/iomap.h +4 -0
@@ -493,6 +493,7 @@ struct iomap_read_folio_ctx {struct folio *cur_folio;struct readahead_control *rac;void *read_ctx; +loff_t read_ctx_file_offset; };struct iomap_read_ops {
@@ -599,6 +600,9 @@ int iomap_swapfile_activate(struct swap_info_struct *sis,extern struct bio_set iomap_ioend_bioset;#ifdef CONFIG_BLOCK +int iomap_bio_read_folio_range (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx,size_t plen); +extern const struct iomap_read_ops iomap_bio_read_ops; static inline void iomap_bio_read_folio(struct folio *folio,
Signature · 1 line
2.47.3
Re: [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission
Quoted 1 line · Christoph Hellwig
Files ystems such as btrfs have additional operations with bios such as
Filesystems
Quoted 6 lines
verifying data checksums. Allow file systems to hook into submission of the bio to allow for this processing by replacing the direct
submit_biocall iniomap_read_alloc_biowith a call into ->submit_read and exportingiomap_read_alloc_bio. Also add a new field to structiomap_read_folio_ctxto track the file logic offset of the current read context.
Basically you're enabling filesystems to know what's the offset of a read bio that iomap is about to start? I guess that enables btrfs to stash that info somewhere so that when the bio completes, it can go look up the checksum or something, and compare?
Or for PI the filesystem can do the PI validation itself and if that fails, decide if it's going to do something evil like ask another replica to try reading the information?
(Not that iomap handles overlapping redundant mappings at all...)
--D
Quoted 79 lines
Based on a patch from Goldwyn Rodrigues <[email protected]>.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/bio.c | 15 +++++++++------ include/linux/iomap.h | 4 ++++ 2 files changed, 13 insertions(+), 6 deletions(-)fs/iomap/bio.c +9
-6@@ -32,10 +32,11 @@ static void iomap_read_alloc_bio(const struct iomap_iter *iter,struct folio *folio = ctx->cur_folio;gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL );gfp_t orig_gfp = gfp; -struct bio *bio = ctx->read_ctx; +struct bio *bio; -if (bio) -submit_bio (bio); +/* Submit the existing range if there was one. */ +if (ctx->read_ctx) + ctx->ops->submit_read (iter, ctx);/* Same as readahead_gfp_mask: */ if (ctx->rac)@@ -56,9 +57,10 @@ static void iomap_read_alloc_bio(const struct iomap_iter *iter,bio_add_folio_nofail (bio, folio, plen,offset_in_folio (folio, iter->pos)); ctx->read_ctx = bio; + ctx->read_ctx_file_offset = iter->pos; } -static int iomap_bio_read_folio_range (const struct iomap_iter *iter, +int iomap_bio_read_folio_range (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx,size_t plen) {struct folio *folio = ctx->cur_folio;@@ -70,10 +72,11 @@ static int iomap_bio_read_folio_range(const struct iomap_iter *iter,iomap_read_alloc_bio (iter, ctx, plen);return 0 ; } +EXPORT_SYMBOL_GPL (iomap_bio_read_folio_range);const struct iomap_read_ops iomap_bio_read_ops = { - .read_folio_range = iomap_bio_read_folio_range, - .submit_read = iomap_bio_submit_read, + .read_folio_range = iomap_bio_read_folio_range, + .submit_read = iomap_bio_submit_read, };EXPORT_SYMBOL_GPL (iomap_bio_read_ops);include/linux/iomap.h +4
-0@@ -493,6 +493,7 @@ struct iomap_read_folio_ctx {struct folio *cur_folio;struct readahead_control *rac;void *read_ctx; +loff_t read_ctx_file_offset; };struct iomap_read_ops {@@ -599,6 +600,9 @@ int iomap_swapfile_activate(struct swap_info_struct *sis,extern struct bio_set iomap_ioend_bioset;#ifdef CONFIG_BLOCK +int iomap_bio_read_folio_range (const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx,size_t plen); +extern const struct iomap_read_ops iomap_bio_read_ops; static inline void iomap_bio_read_folio(struct folio *folio,Signature · 1 line
2.47.3
Re: [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission
Quoted 9 lines · Darrick J. Wong
Quoted 6 lines
verifying data checksums. Allow file systems to hook into submission of the bio to allow for this processing by replacing the direct
submit_biocall iniomap_read_alloc_biowith a call into ->submit_read and exportingiomap_read_alloc_bio. Also add a new field to structiomap_read_folio_ctxto track the file logic offset of the current read context.Basically you're enabling filesystems to know what's the offset of a read bio that iomap is about to start?
Yes.
Quoted 3 lines
I guess that enables btrfs to
stash that info somewhere so that when the bio completes, it can go look up the checksum or something, and compare?
Yes, where something in this series is the ioend that the bio is embedded into.
Quoted 3 lines
Or for PI the filesystem can do the PI validation itself and if that fails, decide if it's going to do something evil like ask another replica to try reading the information?
Yeah. If your file system can do it. Nothing in this series does that, though.
Darrick J. Wong
Reviewed-byRe: [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission
Quoted 20 lines · Christoph Hellwig
Quoted 9 lines · Darrick J. Wong
Quoted 6 lines
verifying data checksums. Allow file systems to hook into submission of the bio to allow for this processing by replacing the direct
submit_biocall iniomap_read_alloc_biowith a call into ->submit_read and exportingiomap_read_alloc_bio. Also add a new field to structiomap_read_folio_ctxto track the file logic offset of the current read context.Basically you're enabling filesystems to know what's the offset of a read bio that iomap is about to start?
Yes.
Quoted 3 lines
I guess that enables btrfs to
stash that info somewhere so that when the bio completes, it can go look up the checksum or something, and compare?Yes, where something in this series is the ioend that the bio is embedded into.
Quoted 6 lines
Quoted 3 lines
Or for PI the filesystem can do the PI validation itself and if that fails, decide if it's going to do something evil like ask another replica to try reading the information?
Yeah. If your file system can do it. Nothing in this series does that, though.
<nod> and at this point we're never going to circle back to that project either. :/
In the meantime I think I've understood this patch well enough to say
--D
PATCH 12/15 iomap: add a bioset pointer to iomap_read_folio_ops
In reply to Christoph HellwigOptionally allocate the bio from the bioset provided in iomap_read_folio_ops. If no bioset is provided, fs_bio_set is still used, which is the standard bioset for file systems.
Based on a patch from Goldwyn Rodrigues <[email protected]>.
--- fs/iomap/bio.c | 14 ++++++++++++-- include/linux/iomap.h | 6 ++++++ 2 files changed, 18 insertions(+), 2 deletions(-)
fs/iomap/bio.c +12 -2
@@ -24,11 +24,19 @@ static void iomap_bio_submit_read(const struct iomap_iter *iter,submit_bio (ctx->read_ctx); } +static struct bio_set *iomap_read_bio_set (struct iomap_read_folio_ctx *ctx) +{ +if (ctx->ops && ctx->ops->bio_set) +return ctx->ops->bio_set; +return &fs_bio_set; +} +static void iomap_read_alloc_bio (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx,size_t plen) {const struct iomap *iomap = &iter->iomap;unsigned int nr_vecs =DIV_ROUND_UP (iomap_length (iter),PAGE_SIZE ); +struct bio_set *bio_set =iomap_read_bio_set (ctx);struct folio *folio = ctx->cur_folio;gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL );gfp_t orig_gfp = gfp;
@@ -47,9 +55,11 @@ static void iomap_read_alloc_bio(const struct iomap_iter *iter,* having to deal with partial page reads. This emulates what * do_mpage_read_folio does. */ - bio =bio_alloc (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , gfp); + bio =bio_alloc_bioset (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , + gfp, bio_set);if (!bio) - bio =bio_alloc (iomap->bdev,1 ,REQ_OP_READ , orig_gfp); + bio =bio_alloc_bioset (iomap->bdev,1 ,REQ_OP_READ , orig_gfp, + bio_set);if (ctx->rac) bio->bi_opf |=REQ_RAHEAD ; bio->bi_iter.bi_sector =iomap_sector (iomap, iter->pos);
include/linux/iomap.h +6 -0
@@ -515,6 +515,12 @@ struct iomap_read_ops {*/ void (*submit_read)(const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx); + +/* +* Optional, allows filesystem to specify own bio_set, so new bio's +* can be allocated from the provided bio_set. +*/ +struct bio_set *bio_set; };/*
Signature · 1 line
2.47.3
Re: [PATCH 12/15] iomap: add a bioset pointer to iomap_read_folio_ops
Quoted 7 lines · Christoph Hellwig
Optionally allocate the bio from the bioset provided in
iomap_read_folio_ops. If no bioset is provided,fs_bio_setis still used, which is the standard bioset for file systems.Based on a patch from Goldwyn Rodrigues <[email protected]>.
Signed-off-by Christoph Hellwig <[email protected]>
Looks reasonable,
--D
Quoted 64 lines
--- fs/iomap/bio.c | 14 ++++++++++++-- include/linux/iomap.h | 6 ++++++ 2 files changed, 18 insertions(+), 2 deletions(-)fs/iomap/bio.c +12
-2@@ -24,11 +24,19 @@ static void iomap_bio_submit_read(const struct iomap_iter *iter,submit_bio (ctx->read_ctx); } +static struct bio_set *iomap_read_bio_set (struct iomap_read_folio_ctx *ctx) +{ +if (ctx->ops && ctx->ops->bio_set) +return ctx->ops->bio_set; +return &fs_bio_set; +} +static void iomap_read_alloc_bio (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx,size_t plen) {const struct iomap *iomap = &iter->iomap;unsigned int nr_vecs =DIV_ROUND_UP (iomap_length (iter),PAGE_SIZE ); +struct bio_set *bio_set =iomap_read_bio_set (ctx);struct folio *folio = ctx->cur_folio;gfp_t gfp =mapping_gfp_constraint (folio->mapping,GFP_KERNEL );gfp_t orig_gfp = gfp;@@ -47,9 +55,11 @@ static void iomap_read_alloc_bio(const struct iomap_iter *iter,* having to deal with partial page reads. This emulates what * do_mpage_read_folio does. */ - bio =bio_alloc (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , gfp); + bio =bio_alloc_bioset (iomap->bdev,bio_max_segs (nr_vecs),REQ_OP_READ , + gfp, bio_set);if (!bio) - bio =bio_alloc (iomap->bdev,1 ,REQ_OP_READ , orig_gfp); + bio =bio_alloc_bioset (iomap->bdev,1 ,REQ_OP_READ , orig_gfp, + bio_set);if (ctx->rac) bio->bi_opf |=REQ_RAHEAD ; bio->bi_iter.bi_sector =iomap_sector (iomap, iter->pos);include/linux/iomap.h +6
-0@@ -515,6 +515,12 @@ struct iomap_read_ops {*/ void (*submit_read)(const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx); + +/* +* Optional, allows filesystem to specify own bio_set, so new bio's +* can be allocated from the provided bio_set. +*/ +struct bio_set *bio_set; };/* Signature · 1 line
2.47.3
PATCH 13/15 iomap: support ioends for buffered reads
In reply to Christoph HellwigSupport using the ioend structure to defer I/O completion for buffered reads by calling into the buffered read I/O completion handler from iomap_finish_ioend.
--- fs/iomap/bio.c | 19 ++++++++++++++++--- fs/iomap/internal.h | 1 + fs/iomap/ioend.c | 8 +++++--- 3 files changed, 22 insertions(+), 6 deletions(-)
fs/iomap/bio.c +16 -3
@@ -8,14 +8,27 @@#include "internal.h" #include "trace.h" -static void iomap_read_end_io (struct bio *bio) +static u32 __iomap_read_end_io (struct bio *bio,int error) { -int error =blk_status_to_errno (bio->bi_status);struct folio_iter fi; +u32 folio_count =0 ; -bio_for_each_folio_all (fi, bio) +bio_for_each_folio_all (fi, bio) {iomap_finish_folio_read (fi.folio, fi.offset, fi.length, error); + folio_count++; + }bio_put (bio); +return folio_count; +} + +static void iomap_read_end_io (struct bio *bio) +{ +__iomap_read_end_io (bio,blk_status_to_errno (bio->bi_status)); +} + +u32 iomap_finish_ioend_buffered_read (struct iomap_ioend *ioend) +{ +return __iomap_read_end_io (&ioend->io_bio, ioend->io_error); } static void iomap_bio_submit_read(const struct iomap_iter *iter,
fs/iomap/internal.h +1 -0
@@ -4,6 +4,7 @@#define IOEND_BATCH_SIZE 4096 +u32 iomap_finish_ioend_buffered_read (struct iomap_ioend *ioend);u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend);#ifdef CONFIG_BLOCK
fs/iomap/ioend.c +5 -3
@@ -36,7 +36,7 @@ EXPORT_SYMBOL_GPL(iomap_init_ioend);* state, release holds on bios, and finally free up memory. Do not use the * ioend after this. */ -static u32 iomap_finish_ioend_buffered (struct iomap_ioend *ioend) +static u32 iomap_finish_ioend_buffered_write (struct iomap_ioend *ioend) {struct inode *inode = ioend->io_inode;struct bio *bio = &ioend->io_bio;
@@ -68,7 +68,7 @@ static void ioend_writeback_end_bio(struct bio *bio)struct iomap_ioend *ioend =iomap_ioend_from_bio (bio); ioend->io_error =blk_status_to_errno (bio->bi_status); -iomap_finish_ioend_buffered (ioend); +iomap_finish_ioend_buffered_write (ioend); }/*
@@ -260,7 +260,9 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)return 0 ;if (ioend->io_flags &IOMAP_IOEND_DIRECT )return iomap_finish_ioend_direct (ioend); -return iomap_finish_ioend_buffered (ioend); +if (bio_op (&ioend->io_bio) ==REQ_OP_READ ) +return iomap_finish_ioend_buffered_read (ioend); +return iomap_finish_ioend_buffered_write (ioend); }/*
Signature · 1 line
2.47.3
Re: [PATCH 13/15] iomap: support ioends for buffered reads
Quoted 5 lines · Christoph Hellwig
Support using the ioend structure to defer I/O completion for buffered reads by calling into the buffered read I/O completion handler from
iomap_finish_ioend.Signed-off-by Christoph Hellwig <[email protected]>
Seems simple enough...
--D
Quoted 90 lines
--- fs/iomap/bio.c | 19 ++++++++++++++++--- fs/iomap/internal.h | 1 + fs/iomap/ioend.c | 8 +++++--- 3 files changed, 22 insertions(+), 6 deletions(-)fs/iomap/bio.c +16
-3@@ -8,14 +8,27 @@#include "internal.h" #include "trace.h" -static void iomap_read_end_io (struct bio *bio) +static u32 __iomap_read_end_io (struct bio *bio,int error) { -int error =blk_status_to_errno (bio->bi_status);struct folio_iter fi; +u32 folio_count =0 ; -bio_for_each_folio_all (fi, bio) +bio_for_each_folio_all (fi, bio) {iomap_finish_folio_read (fi.folio, fi.offset, fi.length, error); + folio_count++; + }bio_put (bio); +return folio_count; +} + +static void iomap_read_end_io (struct bio *bio) +{ +__iomap_read_end_io (bio,blk_status_to_errno (bio->bi_status)); +} + +u32 iomap_finish_ioend_buffered_read (struct iomap_ioend *ioend) +{ +return __iomap_read_end_io (&ioend->io_bio, ioend->io_error); } static void iomap_bio_submit_read(const struct iomap_iter *iter,fs/iomap/internal.h +1
-0@@ -4,6 +4,7 @@#define IOEND_BATCH_SIZE 4096 +u32 iomap_finish_ioend_buffered_read (struct iomap_ioend *ioend);u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend);#ifdef CONFIG_BLOCK fs/iomap/ioend.c +5
-3@@ -36,7 +36,7 @@ EXPORT_SYMBOL_GPL(iomap_init_ioend);* state, release holds on bios, and finally free up memory. Do not use the * ioend after this. */ -static u32 iomap_finish_ioend_buffered (struct iomap_ioend *ioend) +static u32 iomap_finish_ioend_buffered_write (struct iomap_ioend *ioend) {struct inode *inode = ioend->io_inode;struct bio *bio = &ioend->io_bio;@@ -68,7 +68,7 @@ static void ioend_writeback_end_bio(struct bio *bio)struct iomap_ioend *ioend =iomap_ioend_from_bio (bio); ioend->io_error =blk_status_to_errno (bio->bi_status); -iomap_finish_ioend_buffered (ioend); +iomap_finish_ioend_buffered_write (ioend); }/* @@ -260,7 +260,9 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)return 0 ;if (ioend->io_flags &IOMAP_IOEND_DIRECT )return iomap_finish_ioend_direct (ioend); -return iomap_finish_ioend_buffered (ioend); +if (bio_op (&ioend->io_bio) ==REQ_OP_READ ) +return iomap_finish_ioend_buffered_read (ioend); +return iomap_finish_ioend_buffered_write (ioend); }/* Signature · 1 line
2.47.3
PATCH 14/15 iomap: support T10 protection information
In reply to Christoph HellwigAdd support for generating / verifying protection information in iomap. This is done by hooking into the bio submission and then using the generic PI helpers. Compared to just using the block layer auto PI this extends the protection envelope and also prepares for eventually passing through PI from userspace at least for direct I/O.
To generate or verify PI, the file system needs to set the IOMAP_F_INTEGRITY flag on the iomap for the request, and ensure the ioends are used for all integrity I/O. Additionally the file system should defer read I/O completions to user context so that the guard tag validation isn't run from interrupt context.
--- fs/iomap/bio.c | 24 +++++++++++++++++++++--- fs/iomap/direct-io.c | 15 ++++++++++++++- fs/iomap/internal.h | 13 +++++++++++++ fs/iomap/ioend.c | 20 ++++++++++++++++++-- include/linux/iomap.h | 7 +++++++ 5 files changed, 73 insertions(+), 6 deletions(-)
fs/iomap/bio.c +21 -3
@@ -3,6 +3,7 @@* Copyright (C) 2010 Red Hat, Inc. * Copyright (C) 2016-2023 Christoph Hellwig. */ +#include <linux/bio-integrity.h> #include <linux/iomap.h> #include <linux/pagemap.h> #include "internal.h"
@@ -17,6 +18,8 @@ static u32 __iomap_read_end_io(struct bio *bio, int error)iomap_finish_folio_read (fi.folio, fi.offset, fi.length, error); folio_count++; } +if (bio_integrity (bio)) +fs_bio_integrity_free (bio);bio_put (bio);return folio_count; }
@@ -34,7 +37,11 @@ u32 iomap_finish_ioend_buffered_read(struct iomap_ioend *ioend)static void iomap_bio_submit_read (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx) { -submit_bio (ctx->read_ctx); +struct bio *bio = ctx->read_ctx; + +if (iter->iomap.flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_alloc (bio); +submit_bio (bio); }static struct bio_set *iomap_read_bio_set (struct iomap_read_folio_ctx *ctx)
@@ -91,6 +98,7 @@ int iomap_bio_read_folio_range(const struct iomap_iter *iter,if (!bio ||bio_end_sector (bio) !=iomap_sector (&iter->iomap, iter->pos) || + bio->bi_iter.bi_size >iomap_max_bio_size (&iter->iomap) - plen || !bio_add_folio (bio, folio, plen,offset_in_folio (folio, iter->pos)))iomap_read_alloc_bio (iter, ctx, plen);return 0 ;
@@ -107,11 +115,21 @@ int iomap_bio_read_folio_range_sync(const struct iomap_iter *iter, struct folio *folio, loff_t pos, size_t len) { const struct iomap *srcmap = iomap_iter_srcmap(iter); +sector_t sector =iomap_sector (srcmap, pos);struct bio_vec bvec;struct bio bio; +int error;bio_init (&bio, srcmap->bdev, &bvec,1 ,REQ_OP_READ ); - bio.bi_iter.bi_sector =iomap_sector (srcmap, pos); + bio.bi_iter.bi_sector = sector;bio_add_folio_nofail (&bio, folio, len,offset_in_folio (folio, pos)); -return submit_bio_wait (&bio); +if (srcmap->flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_alloc (&bio); + error =submit_bio_wait (&bio); +if (srcmap->flags &IOMAP_F_INTEGRITY ) { +if (!error) + error =fs_bio_integrity_verify (&bio, sector, len); +fs_bio_integrity_free (&bio); + } +return error; }
fs/iomap/direct-io.c +14 -1
@@ -3,6 +3,7 @@* Copyright (C) 2010 Red Hat, Inc. * Copyright (c) 2016-2025 Christoph Hellwig. */ +#include <linux/bio-integrity.h> #include <linux/blk-crypto.h> #include <linux/fscrypt.h> #include <linux/pagemap.h>
@@ -215,6 +216,9 @@ static void __iomap_dio_bio_end_io(struct bio *bio, bool inline_completion) {struct iomap_dio *dio = bio->bi_private; +if (bio_integrity (bio)) +fs_bio_integrity_free (bio); +if (dio->flags &IOMAP_DIO_BOUNCE ) {bio_iov_iter_unbounce (bio, !!dio->error, dio->flags &IOMAP_DIO_USER_BACKED );
@@ -325,8 +329,10 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; +if (dio->flags &IOMAP_DIO_BOUNCE ) - ret =bio_iov_iter_bounce (bio, dio->submit.iter,UINT_MAX ); + ret =bio_iov_iter_bounce (bio, dio->submit.iter, +iomap_max_bio_size (&iter->iomap));else ret =bio_iov_iter_get_pages (bio, dio->submit.iter, alignment -1 );
@@ -343,6 +349,13 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, goto out_put_bio; } +if (iter->iomap.flags &IOMAP_F_INTEGRITY ) { +if (dio->flags &IOMAP_DIO_WRITE ) +fs_bio_integrity_generate (bio); +else +fs_bio_integrity_alloc (bio); + } +if (dio->flags &IOMAP_DIO_WRITE )task_io_account_write (ret); else if ((dio->flags & IOMAP_DIO_USER_BACKED) &&
fs/iomap/internal.h +13 -0
@@ -4,6 +4,19 @@#define IOEND_BATCH_SIZE 4096 +/* +* Normally we can build bios as big as the data structure supports. +* +* But for integrity protected I/O we need to respect the maximum size of the +* single contiguous allocation for the integrity buffer. +*/ +static inline size_t iomap_max_bio_size (const struct iomap *iomap) +{ +if (iomap->flags &IOMAP_F_INTEGRITY ) +return max_integrity_io_size (bdev_limits (iomap->bdev)); +return SIZE_MAX ; +} +u32 iomap_finish_ioend_buffered_read (struct iomap_ioend *ioend);u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend);
fs/iomap/ioend.c +18 -2
@@ -2,6 +2,7 @@/* * Copyright (c) 2016-2025 Christoph Hellwig. */ +#include <linux/bio-integrity.h> #include <linux/iomap.h> #include <linux/list_sort.h> #include <linux/pagemap.h>
@@ -59,6 +60,8 @@ static u32 iomap_finish_ioend_buffered_write(struct iomap_ioend *ioend) folio_count++; } +if (bio_integrity (bio)) +fs_bio_integrity_free (bio);bio_put (bio);/* frees the ioend */ return folio_count; }
@@ -92,6 +95,8 @@ int iomap_ioend_writeback_submit(struct iomap_writepage_ctx *wpc, int error)return error; } +if (wpc->iomap.flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_generate (&ioend->io_bio);submit_bio (&ioend->io_bio);return 0 ; }
@@ -113,10 +118,13 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, }static bool iomap_can_add_to_ioend (struct iomap_writepage_ctx *wpc,loff_t pos, -u16 ioend_flags) +unsigned int map_len,u16 ioend_flags) {struct iomap_ioend *ioend = wpc->wb_ctx; +if (ioend->io_bio.bi_iter.bi_size > +iomap_max_bio_size (&wpc->iomap) - map_len) +return false;if (ioend_flags &IOMAP_IOEND_BOUNDARY )return false; if ((ioend_flags & IOMAP_IOEND_NOMERGE_FLAGS) !=
@@ -181,7 +189,7 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,if (pos == wpc->iomap.offset && (wpc->iomap.flags &IOMAP_F_BOUNDARY )) ioend_flags |=IOMAP_IOEND_BOUNDARY ; -if (!ioend || !iomap_can_add_to_ioend (wpc, pos, ioend_flags)) { +if (!ioend || !iomap_can_add_to_ioend (wpc, pos, map_len, ioend_flags)) { new_ioend:if (ioend) { error = wpc->ops->writeback_submit (wpc,0 );
@@ -258,6 +266,14 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)if (!atomic_dec_and_test (&ioend->io_remaining))return 0 ; + +if (!ioend->io_error && +bio_integrity (&ioend->io_bio) && +bio_op (&ioend->io_bio) ==REQ_OP_READ ) { + ioend->io_error =fs_bio_integrity_verify (&ioend->io_bio, + ioend->io_sector, ioend->io_size); + } +if (ioend->io_flags &IOMAP_IOEND_DIRECT )return iomap_finish_ioend_direct (ioend);if (bio_op (&ioend->io_bio) ==REQ_OP_READ )
include/linux/iomap.h +7 -0
@@ -65,6 +65,8 @@ struct vm_fault;* * IOMAP_F_ATOMIC_BIO indicates that (write) I/O will be issued as an atomic * bio, i.e. set REQ_ATOMIC. +* +* IOMAP_F_INTEGRITY indicates that the filesystems handles integrity metadata. */ #define IOMAP_F_NEW (1U << 0)#define IOMAP_F_DIRTY (1U << 1)
@@ -79,6 +81,11 @@ struct vm_fault;#define IOMAP_F_BOUNDARY (1U << 6)#define IOMAP_F_ANON_WRITE (1U << 7)#define IOMAP_F_ATOMIC_BIO (1U << 8) +#ifdef CONFIG_BLK_DEV_INTEGRITY +#define IOMAP_F_INTEGRITY (1U << 9) +#else +#define IOMAP_F_INTEGRITY 0 +#endif /* CONFIG_BLK_DEV_INTEGRITY */ /* * Flag reserved for file system specific usage
Signature · 1 line
2.47.3
Re: [PATCH 14/15] iomap: support T10 protection information
Quoted 10 lines · Christoph Hellwig
Add support for generating / verifying protection information in iomap. This is done by hooking into the bio submission and then using the generic PI helpers. Compared to just using the block layer auto PI this extends the protection envelope and also prepares for eventually passing through PI from userspace at least for direct I/O.
To generate or verify PI, the file system needs to set the
IOMAP_F_INTEGRITYflag on the iomap for the request, and ensure the ioends are used for all integrity I/O. Additionally the file system should defer read I/O completions to user context so that the guard
must ?
Quoted 184 lines
tag validation isn't run from interrupt context.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/bio.c | 24 +++++++++++++++++++++--- fs/iomap/direct-io.c | 15 ++++++++++++++- fs/iomap/internal.h | 13 +++++++++++++ fs/iomap/ioend.c | 20 ++++++++++++++++++-- include/linux/iomap.h | 7 +++++++ 5 files changed, 73 insertions(+), 6 deletions(-)fs/iomap/bio.c +21
-3@@ -3,6 +3,7 @@* Copyright (C) 2010 Red Hat, Inc. * Copyright (C) 2016-2023 Christoph Hellwig. */ +#include <linux/bio-integrity.h> #include <linux/iomap.h> #include <linux/pagemap.h> #include "internal.h" @@ -17,6 +18,8 @@ static u32 __iomap_read_end_io(struct bio *bio, int error)iomap_finish_folio_read (fi.folio, fi.offset, fi.length, error); folio_count++; } +if (bio_integrity (bio)) +fs_bio_integrity_free (bio);bio_put (bio);return folio_count; }@@ -34,7 +37,11 @@ u32 iomap_finish_ioend_buffered_read(struct iomap_ioend *ioend)static void iomap_bio_submit_read (const struct iomap_iter *iter,struct iomap_read_folio_ctx *ctx) { -submit_bio (ctx->read_ctx); +struct bio *bio = ctx->read_ctx; + +if (iter->iomap.flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_alloc (bio); +submit_bio (bio); }static struct bio_set *iomap_read_bio_set (struct iomap_read_folio_ctx *ctx)@@ -91,6 +98,7 @@ int iomap_bio_read_folio_range(const struct iomap_iter *iter,if (!bio ||bio_end_sector (bio) !=iomap_sector (&iter->iomap, iter->pos) || + bio->bi_iter.bi_size >iomap_max_bio_size (&iter->iomap) - plen || !bio_add_folio (bio, folio, plen,offset_in_folio (folio, iter->pos)))iomap_read_alloc_bio (iter, ctx, plen);return 0 ;@@ -107,11 +115,21 @@ int iomap_bio_read_folio_range_sync(const struct iomap_iter *iter, struct folio *folio, loff_t pos, size_t len) { const struct iomap *srcmap = iomap_iter_srcmap(iter); +sector_t sector =iomap_sector (srcmap, pos);struct bio_vec bvec;struct bio bio; +int error;bio_init (&bio, srcmap->bdev, &bvec,1 ,REQ_OP_READ ); - bio.bi_iter.bi_sector =iomap_sector (srcmap, pos); + bio.bi_iter.bi_sector = sector;bio_add_folio_nofail (&bio, folio, len,offset_in_folio (folio, pos)); -return submit_bio_wait (&bio); +if (srcmap->flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_alloc (&bio); + error =submit_bio_wait (&bio); +if (srcmap->flags &IOMAP_F_INTEGRITY ) { +if (!error) + error =fs_bio_integrity_verify (&bio, sector, len); +fs_bio_integrity_free (&bio); + } +return error; }fs/iomap/direct-io.c +14
-1@@ -3,6 +3,7 @@* Copyright (C) 2010 Red Hat, Inc. * Copyright (c) 2016-2025 Christoph Hellwig. */ +#include <linux/bio-integrity.h> #include <linux/blk-crypto.h> #include <linux/fscrypt.h> #include <linux/pagemap.h> @@ -215,6 +216,9 @@ static void __iomap_dio_bio_end_io(struct bio *bio, bool inline_completion) {struct iomap_dio *dio = bio->bi_private; +if (bio_integrity (bio)) +fs_bio_integrity_free (bio); +if (dio->flags &IOMAP_DIO_BOUNCE ) {bio_iov_iter_unbounce (bio, !!dio->error, dio->flags &IOMAP_DIO_USER_BACKED );@@ -325,8 +329,10 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; +if (dio->flags &IOMAP_DIO_BOUNCE ) - ret =bio_iov_iter_bounce (bio, dio->submit.iter,UINT_MAX ); + ret =bio_iov_iter_bounce (bio, dio->submit.iter, +iomap_max_bio_size (&iter->iomap));else ret =bio_iov_iter_get_pages (bio, dio->submit.iter, alignment -1 );@@ -343,6 +349,13 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter, goto out_put_bio; } +if (iter->iomap.flags &IOMAP_F_INTEGRITY ) { +if (dio->flags &IOMAP_DIO_WRITE ) +fs_bio_integrity_generate (bio); +else +fs_bio_integrity_alloc (bio); + } +if (dio->flags &IOMAP_DIO_WRITE )task_io_account_write (ret); else if ((dio->flags & IOMAP_DIO_USER_BACKED) &&fs/iomap/internal.h +13
-0@@ -4,6 +4,19 @@#define IOEND_BATCH_SIZE 4096 +/* +* Normally we can build bios as big as the data structure supports. +* +* But for integrity protected I/O we need to respect the maximum size of the +* single contiguous allocation for the integrity buffer. +*/ +static inline size_t iomap_max_bio_size (const struct iomap *iomap) +{ +if (iomap->flags &IOMAP_F_INTEGRITY ) +return max_integrity_io_size (bdev_limits (iomap->bdev)); +return SIZE_MAX ; +} +u32 iomap_finish_ioend_buffered_read (struct iomap_ioend *ioend);u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend);fs/iomap/ioend.c +5
-1@@ -2,6 +2,7 @@/* * Copyright (c) 2016-2025 Christoph Hellwig. */ +#include <linux/bio-integrity.h> #include <linux/iomap.h> #include <linux/list_sort.h> #include <linux/pagemap.h> @@ -59,6 +60,8 @@ static u32 iomap_finish_ioend_buffered_write(struct iomap_ioend *ioend) folio_count++; } +if (bio_integrity (bio)) +fs_bio_integrity_free (bio);bio_put (bio);/* frees the ioend */ return folio_count; }@@ -92,6 +95,8 @@ int iomap_ioend_writeback_submit(struct iomap_writepage_ctx *wpc, int error)return error; } +if (wpc->iomap.flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_generate (&ioend->io_bio);submit_bio (&ioend->io_bio);return 0 ; }@@ -113,10 +118,13 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, } static bool iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t pos, -u16 ioend_flags)
fs/iomap/ioend.c static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, · line 130
Quoted 10 lines
fs/iomap/ioend.c +4
-0+ unsigned int map_len, u16 ioend_flags) {struct iomap_ioend *ioend = wpc->wb_ctx; +if (ioend->io_bio.bi_iter.bi_size > +iomap_max_bio_size (&wpc->iomap) - map_len) +return false;if (ioend_flags &IOMAP_IOEND_BOUNDARY )return false; if ((ioend_flags & IOMAP_IOEND_NOMERGE_FLAGS) !=
Unrelated question: should iomap_can_add_to_ioend return false if it did an IOMAP_F_ANON_WRITE and the bdevs aren't the same, even if the sectors match? Currently not a problem for XFS, but some day we might want to have a file that maps to zones on different devices.
The changes here otherwise look ok to me.
--D
Quoted 53 lines
fs/iomap/ioend.c +9
-1@@ -181,7 +189,7 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,if (pos == wpc->iomap.offset && (wpc->iomap.flags &IOMAP_F_BOUNDARY )) ioend_flags |=IOMAP_IOEND_BOUNDARY ; -if (!ioend || !iomap_can_add_to_ioend (wpc, pos, ioend_flags)) { +if (!ioend || !iomap_can_add_to_ioend (wpc, pos, map_len, ioend_flags)) { new_ioend:if (ioend) { error = wpc->ops->writeback_submit (wpc,0 );@@ -258,6 +266,14 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)if (!atomic_dec_and_test (&ioend->io_remaining))return 0 ; + +if (!ioend->io_error && +bio_integrity (&ioend->io_bio) && +bio_op (&ioend->io_bio) ==REQ_OP_READ ) { + ioend->io_error =fs_bio_integrity_verify (&ioend->io_bio, + ioend->io_sector, ioend->io_size); + } +if (ioend->io_flags &IOMAP_IOEND_DIRECT )return iomap_finish_ioend_direct (ioend);if (bio_op (&ioend->io_bio) ==REQ_OP_READ )include/linux/iomap.h +7
-0@@ -65,6 +65,8 @@ struct vm_fault;* * IOMAP_F_ATOMIC_BIO indicates that (write) I/O will be issued as an atomic * bio, i.e. set REQ_ATOMIC. +* +* IOMAP_F_INTEGRITY indicates that the filesystems handles integrity metadata. */ #define IOMAP_F_NEW (1U << 0)#define IOMAP_F_DIRTY (1U << 1)@@ -79,6 +81,11 @@ struct vm_fault;#define IOMAP_F_BOUNDARY (1U << 6)#define IOMAP_F_ANON_WRITE (1U << 7)#define IOMAP_F_ATOMIC_BIO (1U << 8) +#ifdef CONFIG_BLK_DEV_INTEGRITY +#define IOMAP_F_INTEGRITY (1U << 9) +#else +#define IOMAP_F_INTEGRITY 0 +#endif /* CONFIG_BLK_DEV_INTEGRITY */ /* * Flag reserved for file system specific usage Signature · 1 line
2.47.3
Re: [PATCH 14/15] iomap: support T10 protection information
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines · Christoph Hellwig
Add support for generating / verifying protection information in iomap. This is done by hooking into the bio submission and then using the generic PI helpers. Compared to just using the block layer auto PI this extends the protection envelope and also prepares for eventually passing through PI from userspace at least for direct I/O.
Quoted 6 lines
Quoted 4 lines
To generate or verify PI, the file system needs to set the
IOMAP_F_INTEGRITYflag on the iomap for the request, and ensure the ioends are used for all integrity I/O. Additionally the file system should defer read I/O completions to user context so that the guardmust ?
Well, the copy isn't actually blocking. So a small copy might actually work from hardirq context, but you're not going to make friends with anyone caring about latency. I guess that means I should upgrade this to a "must" :)
Quoted 14 lines
fs/iomap/ioend.c static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, · line 130
Quoted 9 lines
{
fs/iomap/ioend.c +3
-0struct iomap_ioend *ioend = wpc->wb_ctx; +if (ioend->io_bio.bi_iter.bi_size > +iomap_max_bio_size (&wpc->iomap) - map_len) +return false;if (ioend_flags &IOMAP_IOEND_BOUNDARY )return false; if ((ioend_flags & IOMAP_IOEND_NOMERGE_FLAGS) !=Unrelated question: should
iomap_can_add_to_ioendreturn false if it did anIOMAP_F_ANON_WRITEand the bdevs aren't the same, even if the sectors match? Currently not a problem for XFS, but some day we might want to have a file that maps to zones on different devices.
For IOMAP_F_ANON_WRITE the bdev doesn't really matter at this level, as it applies the actual mapping is done below. So the bdev is really just a placeholder here.
PATCH 15/15 xfs: support T10 protection information
In reply to Christoph HellwigAdd support for generating / verifying protection information in the file system. This is largely done by simply setting the IOMAP_F_INTEGRITY flag and letting iomap do all of the work. XFS just has to ensure that the data read completions for integrity data are run from user context.
For zoned writeback, XFS also has to generate the integrity data itself as the zoned writeback path is not using the generic writeback_submit implementation.
--- fs/xfs/xfs_aops.c | 47 ++++++++++++++++++++++++++++++++++++++++++---- fs/xfs/xfs_iomap.c | 9 ++++++--- 2 files changed, 49 insertions(+), 7 deletions(-)
fs/xfs/xfs_aops.c +43 -4
@@ -22,6 +22,7 @@#include "xfs_icache.h" #include "xfs_zone_alloc.h" #include "xfs_rtgroup.h" +#include <linux/bio-integrity.h> struct xfs_writepage_ctx {struct iomap_writepage_ctx ctx;
@@ -661,6 +662,8 @@ xfs_zoned_writeback_submit(bio_endio (&ioend->io_bio);return error; } +if (wpc->iomap.flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_generate (&ioend->io_bio);xfs_zone_alloc_and_submit (ioend, &XFS_ZWPC (wpc)->open_zone);return 0 ; }
@@ -741,12 +744,43 @@ xfs_vm_bmap(return iomap_bmap (mapping, block, &xfs_read_iomap_ops); } +static void +xfs_bio_submit_read ( +const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx) +{ +struct bio *bio = ctx->read_ctx; + +/* delay read completions to the ioend workqueue */ +iomap_init_ioend (iter->inode, bio, ctx->read_ctx_file_offset,0 ); + bio->bi_end_io = xfs_end_bio; +submit_bio (bio); +} + +static const struct iomap_read_ops xfs_bio_read_integrity_ops = { + .read_folio_range = iomap_bio_read_folio_range, + .submit_read = xfs_bio_submit_read, + .bio_set = &iomap_ioend_bioset, +}; + +static inline const struct iomap_read_ops *bio_read_ops (struct xfs_inode *ip) +{ +if (bdev_has_integrity_csum (xfs_inode_buftarg (ip)->bt_bdev)) +return &xfs_bio_read_integrity_ops; +return &iomap_bio_read_ops; +} + STATIC intxfs_vm_read_folio ( -struct file *unused, -struct folio *folio) +struct file *file, +struct folio *folio) { -iomap_bio_read_folio (folio, &xfs_read_iomap_ops); +struct iomap_read_folio_ctx ctx = { + .cur_folio = folio, + .ops =bio_read_ops (XFS_I (file->f_mapping->host)), + }; + +iomap_read_folio (&xfs_read_iomap_ops, &ctx);return 0 ; }
@@ -754,7 +788,12 @@ STATIC voidxfs_vm_readahead ( struct readahead_control *rac) { -iomap_bio_readahead (rac, &xfs_read_iomap_ops); +struct iomap_read_folio_ctx ctx = { + .rac = rac, + .ops =bio_read_ops (XFS_I (rac->mapping->host)), + }; + +iomap_readahead (&xfs_read_iomap_ops, &ctx); } static int
fs/xfs/xfs_iomap.c +6 -3
@@ -143,11 +143,14 @@ xfs_bmbt_to_iomap( } iomap->offset =XFS_FSB_TO_B (mp, imap->br_startoff); iomap->length =XFS_FSB_TO_B (mp, imap->br_blockcount); -if (mapping_flags &IOMAP_DAX ) + iomap->flags = iomap_flags; +if (mapping_flags &IOMAP_DAX ) { iomap->dax_dev = target->bt_daxdev; -else + }else { iomap->bdev = target->bt_bdev; - iomap->flags = iomap_flags; +if (bdev_has_integrity_csum (iomap->bdev)) + iomap->flags |=IOMAP_F_INTEGRITY ; + }/* * If the inode is dirty for datasync purposes, let iomap know so it
Signature · 1 line
2.47.3
Re: [PATCH 15/15] xfs: support T10 protection information
Quoted 50 lines · Christoph Hellwig
Add support for generating / verifying protection information in the file system. This is largely done by simply setting the
IOMAP_F_INTEGRITYflag and letting iomap do all of the work. XFS just has to ensure that the data read completions for integrity data are run from user context.For zoned writeback, XFS also has to generate the integrity data itself as the zoned writeback path is not using the generic
writeback_submitimplementation.Signed-off-by Christoph Hellwig <[email protected]>--- fs/xfs/xfs_aops.c | 47 ++++++++++++++++++++++++++++++++++++++++++---- fs/xfs/xfs_iomap.c | 9 ++++++--- 2 files changed, 49 insertions(+), 7 deletions(-)fs/xfs/xfs_aops.c +13
-0@@ -22,6 +22,7 @@#include "xfs_icache.h" #include "xfs_zone_alloc.h" #include "xfs_rtgroup.h" +#include <linux/bio-integrity.h> struct xfs_writepage_ctx {struct iomap_writepage_ctx ctx;@@ -661,6 +662,8 @@ xfs_zoned_writeback_submit(bio_endio (&ioend->io_bio);return error; } +if (wpc->iomap.flags &IOMAP_F_INTEGRITY ) +fs_bio_integrity_generate (&ioend->io_bio);xfs_zone_alloc_and_submit (ioend, &XFS_ZWPC (wpc)->open_zone);return 0 ; }@@ -741,12 +744,43 @@ xfs_vm_bmap(return iomap_bmap (mapping, block, &xfs_read_iomap_ops); } +static void +xfs_bio_submit_read ( +const struct iomap_iter *iter, +struct iomap_read_folio_ctx *ctx) +{ +struct bio *bio = ctx->read_ctx; + +/* delay read completions to the ioend workqueue */ +iomap_init_ioend (iter->inode, bio, ctx->read_ctx_file_offset,0 ); + bio->bi_end_io = xfs_end_bio;
fs/xfs/xfs_aops.c xfs_vm_bmap( · line 766
Quoted 10 lines
fs/xfs/xfs_aops.c +10
-0+submit_bio (bio); +} + +static const struct iomap_read_ops xfs_bio_read_integrity_ops = { + .read_folio_range = iomap_bio_read_folio_range, + .submit_read = xfs_bio_submit_read, + .bio_set = &iomap_ioend_bioset, +}; + +static inline const struct iomap_read_ops *bio_read_ops (struct xfs_inode *ip)
xfs_bio_read_ops to avoid conflicts with the block layer?
Otherwise this looks ok to me, so with that fixed,
--D
Quoted 63 lines
fs/xfs/xfs_aops.c +20
-4+{ +if (bdev_has_integrity_csum (xfs_inode_buftarg (ip)->bt_bdev)) +return &xfs_bio_read_integrity_ops; +return &iomap_bio_read_ops; +} + STATIC intxfs_vm_read_folio ( -struct file *unused, -struct folio *folio) +struct file *file, +struct folio *folio) { -iomap_bio_read_folio (folio, &xfs_read_iomap_ops); +struct iomap_read_folio_ctx ctx = { + .cur_folio = folio, + .ops =bio_read_ops (XFS_I (file->f_mapping->host)), + }; + +iomap_read_folio (&xfs_read_iomap_ops, &ctx);return 0 ; }@@ -754,7 +788,12 @@ STATIC voidxfs_vm_readahead ( struct readahead_control *rac) { -iomap_bio_readahead (rac, &xfs_read_iomap_ops); +struct iomap_read_folio_ctx ctx = { + .rac = rac, + .ops =bio_read_ops (XFS_I (rac->mapping->host)), + }; + +iomap_readahead (&xfs_read_iomap_ops, &ctx); } static intfs/xfs/xfs_iomap.c +6
-3@@ -143,11 +143,14 @@ xfs_bmbt_to_iomap( } iomap->offset =XFS_FSB_TO_B (mp, imap->br_startoff); iomap->length =XFS_FSB_TO_B (mp, imap->br_blockcount); -if (mapping_flags &IOMAP_DAX ) + iomap->flags = iomap_flags; +if (mapping_flags &IOMAP_DAX ) { iomap->dax_dev = target->bt_daxdev; -else + }else { iomap->bdev = target->bt_bdev; - iomap->flags = iomap_flags; +if (bdev_has_integrity_csum (iomap->bdev)) + iomap->flags |=IOMAP_F_INTEGRITY ; + }/* * If the inode is dirty for datasync purposes, let iomap know so it Signature · 1 line
2.47.3
Anuj gupta
Tested-byHi Christoph,
Here are the QD1 latency numbers (in usec)
Intel Optane:
Sequential read
| size | xfs-bounce | xfs-pi | +------+------------+-----------+ | 4k | 13.62 | 7.2 | | 64K | 99.66 | 34.16 | | 1M | 258.88 | 306.23 | +------+-------------------------+
Samsung PM1733
Sequential read
| size | xfs-bounce | xfs-pi | +------+------------+-----------+ | 4k | 118.92 | 91.6 | | 64K | 176.15 | 134.55 | | 1M | 612.67 | 584.84 | +------+-------------------------+
For sequential writes, I did not observe any noticeable difference, which is expected.
The Optane numbers at bs=1M look a bit odd (xfs-pi slightly worse than xfs-bounce), but they are reproducible across multiple runs. Overall, the series shows a clear improvement in read performance, which aligns well with the goal of making PI processing more efficient.
Feel free to add:
Quoted 3 lines · Anuj gupta
Hi Christoph,
Here are the QD1 latency numbers (in usec)
Thanks a lot!
Adding in the baseline numbers, as I wanted to compare those:
Quoted 9 lines
Intel Optane:
Sequential read
| size | baseline | xfs-bounce | xfs-pi | +------+----------+-----------+-----------+ | 4k | 7.18 | 13.62 | 7.20 | | 64K | 36.40 | 99.66 | 34.16 | | 1M | 206.38 | 258.88 | 306.23 | +------+----------+------------+----------+
So for 4k and 64k reads we basically get back to the baseline. The 1M numbers are puzzling, though. I wonder if we need to add WQ_CPU_INTENSIVE or do some other tweaks to the XFS I/O completion workqueue so that we don't overload the scheduler.
Quoted 19 lines · Christoph Hellwig
Quoted 3 lines · Anuj gupta
Hi Christoph,
Here are the QD1 latency numbers (in usec)
Thanks a lot!
Adding in the baseline numbers, as I wanted to compare those:
Quoted 9 lines
Intel Optane:
Sequential read
| size | baseline | xfs-bounce | xfs-pi | +------+----------+-----------+-----------+ | 4k | 7.18 | 13.62 | 7.20 | | 64K | 36.40 | 99.66 | 34.16 | | 1M | 206.38 | 258.88 | 306.23 | +------+----------+------------+----------+
Quoted 4 lines
So for 4k and 64k reads we basically get back to the baseline. The 1M numbers are puzzling, though. I wonder if we need to add
WQ_CPU_INTENSIVEor do some other tweaks to the XFS I/O completion workqueue so that we don't overload the scheduler.
Tried adding WQ_CPU_INTENSIVE[1] but didn't see any change in the 1M latency. Looks like this needs something beyond just workqueue tuning.