RFC: iomap patches for zoned XFS
Subsystems (5)
Hi all,
this series contains the iomap prep work to support zoned XFS.
The biggest changes are:
- an option to reuse the ioend code for direct writes in addition to the current use for buffered writeback, which allows the file system to track completions on a per-bio basis instead of the current end_io callback which operates on the entire I/O.
Note that it might make sense to split the ioend code from buffered-io.c into its own file with this. Let me know what you think of that and I can include it in the next version
- change of the writeback_ops so that the submit_bio call can be done by the file system. Note that btrfs will also need this eventually when it starts using iomap
- helpers to split ioend to the zone append queue_limits that plug into the previous item above.
- a bunch of changes for slightly different merge conditions when using zone append. Note that btrfs wants something similar also for compressed I/O, which might be able to share some code. For now the flags use zone append naming, but we can change that if it gets used elsewhere.
- passing private data to a few more helper
The XFS changes to use this will be posted to the xfs list only to not spam fsdevel too much.
PATCH 1/8 iomap: allow the file system to submit the writeback bios
Change ->prepare_ioend to ->submit_ioend and require file systems that implement it to submit the bio. This is needed for file systems that do their own work on the bios before submitting them to the block layer like btrfs or zoned xfs. To make this easier also pass the writeback context to the method.
--- fs/iomap/buffered-io.c | 10 +++++----- fs/xfs/xfs_aops.c | 13 +++++++++---- include/linux/iomap.h | 12 +++++++----- 3 files changed, 21 insertions(+), 14 deletions(-)
fs/iomap/buffered-io.c +5 -5
@@ -1675,7 +1675,7 @@ static void iomap_writepage_end_bio(struct bio *bio) }/* -* Submit the final bio for an ioend. +* Submit an ioend. * * If @error is non-zero, it means that we have a situation where some part of * the submission process has failed after we've marked pages for writeback.
@@ -1694,14 +1694,14 @@ static int iomap_submit_ioend(struct iomap_writepage_ctx *wpc, int error)* failure happened so that the file system end I/O handler gets called * to clean up. */ -if (wpc->ops->prepare_ioend) - error = wpc->ops->prepare_ioend (wpc->ioend, error); +if (wpc->ops->submit_ioend) + error = wpc->ops->submit_ioend (wpc, error); +else if (!error) +submit_bio (&wpc->ioend->io_bio);if (error) { wpc->ioend->io_bio.bi_status =errno_to_blk_status (error);bio_endio (&wpc->ioend->io_bio); - }else { -submit_bio (&wpc->ioend->io_bio); } wpc->ioend =NULL ;
fs/xfs/xfs_aops.c +9 -4
@@ -395,10 +395,11 @@ xfs_map_blocks( }static int -xfs_prepare_ioend ( -struct iomap_ioend *ioend, +xfs_submit_ioend ( +struct iomap_writepage_ctx *wpc,int status) { +struct iomap_ioend *ioend = wpc->ioend;unsigned int nofs_flag;/*
@@ -420,7 +421,11 @@ xfs_prepare_ioend(if (xfs_ioend_is_append (ioend) || ioend->io_type ==IOMAP_UNWRITTEN || (ioend->io_flags &IOMAP_F_SHARED )) ioend->io_bio.bi_end_io = xfs_end_bio; -return status; + +if (status) +return status; +submit_bio (&ioend->io_bio); +return 0 ; }/*
@@ -462,7 +467,7 @@ xfs_discard_folio(static const struct iomap_writeback_ops xfs_writeback_ops = { .map_blocks = xfs_map_blocks, - .prepare_ioend = xfs_prepare_ioend, + .submit_ioend = xfs_submit_ioend, .discard_folio = xfs_discard_folio, };
include/linux/iomap.h +7 -5
@@ -362,12 +362,14 @@ struct iomap_writeback_ops { loff_t offset, unsigned len);/* -* Optional, allows the file systems to perform actions just before -* submitting the bio and/or override the bio end_io handler for complex -* operations like copy on write extent manipulation or unwritten extent -* conversions. +* Optional, allows the file systems to hook into bio submission, +* including overriding the bi_end_io handler. +* +* Returns 0 if the bio was successfully submitted, or a negative +* error code if status was non-zero or another error happened and +* the bio could not be submitted. */ -int (*prepare_ioend)(struct iomap_ioend *ioend,int status); +int (*submit_ioend)(struct iomap_writepage_ctx *wpc,int status);/* * Optional, allows the file system to discard state on a page where
Signature · 1 line
2.45.2
Re: [PATCH 1/8] iomap: allow the file system to submit the writeback bios
Quoted 5 lines · Christoph Hellwig
Change ->prepare_ioend to ->submit_ioend and require file systems that implement it to submit the bio. This is needed for file systems that do their own work on the bios before submitting them to the block layer like btrfs or zoned xfs. To make this easier also pass the writeback context to the method.
The code changes here are pretty straightforward, but please update Documentation/filesystems/iomap/operations.rst to reflect the new name and the new "submit the bio yourself" behavior expected of the implementation.
--D
Quoted 107 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/buffered-io.c | 10 +++++----- fs/xfs/xfs_aops.c | 13 +++++++++---- include/linux/iomap.h | 12 +++++++----- 3 files changed, 21 insertions(+), 14 deletions(-)fs/iomap/buffered-io.c +5
-5@@ -1675,7 +1675,7 @@ static void iomap_writepage_end_bio(struct bio *bio) }/* -* Submit the final bio for an ioend. +* Submit an ioend. * * If @error is non-zero, it means that we have a situation where some part of * the submission process has failed after we've marked pages for writeback. @@ -1694,14 +1694,14 @@ static int iomap_submit_ioend(struct iomap_writepage_ctx *wpc, int error)* failure happened so that the file system end I/O handler gets called * to clean up. */ -if (wpc->ops->prepare_ioend) - error = wpc->ops->prepare_ioend (wpc->ioend, error); +if (wpc->ops->submit_ioend) + error = wpc->ops->submit_ioend (wpc, error); +else if (!error) +submit_bio (&wpc->ioend->io_bio);if (error) { wpc->ioend->io_bio.bi_status =errno_to_blk_status (error);bio_endio (&wpc->ioend->io_bio); - }else { -submit_bio (&wpc->ioend->io_bio); } wpc->ioend =NULL ;fs/xfs/xfs_aops.c +9
-4@@ -395,10 +395,11 @@ xfs_map_blocks( }static int -xfs_prepare_ioend ( -struct iomap_ioend *ioend, +xfs_submit_ioend ( +struct iomap_writepage_ctx *wpc,int status) { +struct iomap_ioend *ioend = wpc->ioend;unsigned int nofs_flag;/* @@ -420,7 +421,11 @@ xfs_prepare_ioend(if (xfs_ioend_is_append (ioend) || ioend->io_type ==IOMAP_UNWRITTEN || (ioend->io_flags &IOMAP_F_SHARED )) ioend->io_bio.bi_end_io = xfs_end_bio; -return status; + +if (status) +return status; +submit_bio (&ioend->io_bio); +return 0 ; }/* @@ -462,7 +467,7 @@ xfs_discard_folio(static const struct iomap_writeback_ops xfs_writeback_ops = { .map_blocks = xfs_map_blocks, - .prepare_ioend = xfs_prepare_ioend, + .submit_ioend = xfs_submit_ioend, .discard_folio = xfs_discard_folio, };include/linux/iomap.h +7
-5@@ -362,12 +362,14 @@ struct iomap_writeback_ops { loff_t offset, unsigned len);/* -* Optional, allows the file systems to perform actions just before -* submitting the bio and/or override the bio end_io handler for complex -* operations like copy on write extent manipulation or unwritten extent -* conversions. +* Optional, allows the file systems to hook into bio submission, +* including overriding the bi_end_io handler. +* +* Returns 0 if the bio was successfully submitted, or a negative +* error code if status was non-zero or another error happened and +* the bio could not be submitted. */ -int (*prepare_ioend)(struct iomap_ioend *ioend,int status); +int (*submit_ioend)(struct iomap_writepage_ctx *wpc,int status);/* * Optional, allows the file system to discard state on a page where Signature · 1 line
2.45.2
PATCH 2/8 iomap: simplify io_flags and io_type in struct iomap_ioend
In reply to Christoph HellwigThe ioend fields for distinct types of I/O are a bit complicated. Consolidate them into a single io_flag field with it's own flags decoupled from the iomap flags. This also prepares for adding a new flag that is unrelated to both of the iomap namespaces.
--- fs/iomap/buffered-io.c | 39 ++++++++++++++++++++++----------------- fs/xfs/xfs_aops.c | 12 ++++++------ include/linux/iomap.h | 16 ++++++++++++++-- 3 files changed, 42 insertions(+), 25 deletions(-)
fs/iomap/buffered-io.c +22 -17
@@ -1605,13 +1605,10 @@ iomap_ioend_can_merge(struct iomap_ioend *ioend, struct iomap_ioend *next) {if (ioend->io_bio.bi_status != next->io_bio.bi_status)return false; -if (next->io_flags &IOMAP_F_BOUNDARY ) +if (next->io_flags &IOMAP_IOEND_BOUNDARY )return false; -if ((ioend->io_flags &IOMAP_F_SHARED ) ^ - (next->io_flags &IOMAP_F_SHARED )) -return false; -if ((ioend->io_type ==IOMAP_UNWRITTEN ) ^ - (next->io_type ==IOMAP_UNWRITTEN )) +if ((ioend->io_flags &IOMAP_IOEND_NOMERGE_FLAGS ) != + (next->io_flags &IOMAP_IOEND_NOMERGE_FLAGS ))return false;if (ioend->io_offset + ioend->io_size != next->io_offset)return false;
@@ -1709,7 +1706,8 @@ static int iomap_submit_ioend(struct iomap_writepage_ctx *wpc, int error) }static struct iomap_ioend *iomap_alloc_ioend (struct iomap_writepage_ctx *wpc, -struct writeback_control *wbc,struct inode *inode,loff_t pos) +struct writeback_control *wbc,struct inode *inode,loff_t pos, +u16 ioend_flags) {struct iomap_ioend *ioend;struct bio *bio;
@@ -1724,8 +1722,7 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, ioend =iomap_ioend_from_bio (bio);INIT_LIST_HEAD (&ioend->io_list); - ioend->io_type = wpc->iomap.type; - ioend->io_flags = wpc->iomap.flags; + ioend->io_flags = ioend_flags;if (pos > wpc->iomap.offset) wpc->iomap.flags &= ~IOMAP_F_BOUNDARY ; ioend->io_inode = inode;
@@ -1737,14 +1734,13 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc,return ioend; } -static bool iomap_can_add_to_ioend (struct iomap_writepage_ctx *wpc,loff_t pos) +static bool iomap_can_add_to_ioend (struct iomap_writepage_ctx *wpc,loff_t pos, +u16 ioend_flags) { -if (wpc->iomap.offset == pos && (wpc->iomap.flags &IOMAP_F_BOUNDARY )) -return false; -if ((wpc->iomap.flags &IOMAP_F_SHARED ) != - (wpc->ioend->io_flags &IOMAP_F_SHARED )) +if (ioend_flags &IOMAP_IOEND_BOUNDARY )return false; -if (wpc->iomap.type != wpc->ioend->io_type) +if ((ioend_flags &IOMAP_IOEND_NOMERGE_FLAGS ) != + (wpc->ioend->io_flags &IOMAP_IOEND_NOMERGE_FLAGS ))return false;if (pos != wpc->ioend->io_offset + wpc->ioend->io_size)return false;
@@ -1778,14 +1774,23 @@ static int iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, {struct iomap_folio_state *ifs = folio->private;size_t poff =offset_in_folio (folio, pos); +unsigned int ioend_flags =0 ;int error; -if (!wpc->ioend || !iomap_can_add_to_ioend (wpc, pos)) { +if (wpc->iomap.type ==IOMAP_UNWRITTEN ) + ioend_flags |=IOMAP_IOEND_UNWRITTEN ; +if (wpc->iomap.flags &IOMAP_F_SHARED ) + ioend_flags |=IOMAP_IOEND_SHARED ; +if (pos == wpc->iomap.offset && (wpc->iomap.flags &IOMAP_F_BOUNDARY )) + ioend_flags |=IOMAP_IOEND_BOUNDARY ; + +if (!wpc->ioend || !iomap_can_add_to_ioend (wpc, pos, ioend_flags)) { new_ioend: error =iomap_submit_ioend (wpc,0 );if (error)return error; - wpc->ioend =iomap_alloc_ioend (wpc, wbc, inode, pos); + wpc->ioend =iomap_alloc_ioend (wpc, wbc, inode, pos, + ioend_flags); }if (!bio_add_folio (&wpc->ioend->io_bio, folio, len, poff))
fs/xfs/xfs_aops.c +6 -6
@@ -114,7 +114,7 @@ xfs_end_ioend(*/ error =blk_status_to_errno (ioend->io_bio.bi_status);if (unlikely (error)) { -if (ioend->io_flags &IOMAP_F_SHARED ) { +if (ioend->io_flags &IOMAP_IOEND_SHARED ) {xfs_reflink_cancel_cow_range (ip, offset, size, true);xfs_bmap_punch_delalloc_range (ip,XFS_DATA_FORK , offset, offset + size);
@@ -125,9 +125,9 @@ xfs_end_ioend(/* * Success: commit the COW or unwritten blocks if needed. */ -if (ioend->io_flags &IOMAP_F_SHARED ) +if (ioend->io_flags &IOMAP_IOEND_SHARED ) error =xfs_reflink_end_cow (ip, offset, size); -else if (ioend->io_type ==IOMAP_UNWRITTEN ) +else if (ioend->io_flags &IOMAP_IOEND_UNWRITTEN ) error =xfs_iomap_write_unwritten (ip, offset, size, false);if (!error &&xfs_ioend_is_append (ioend))
@@ -410,7 +410,7 @@ xfs_submit_ioend( nofs_flag =memalloc_nofs_save ();/* Convert CoW extents to regular */ -if (!status && (ioend->io_flags &IOMAP_F_SHARED )) { +if (!status && (ioend->io_flags &IOMAP_IOEND_SHARED )) { status =xfs_reflink_convert_cow (XFS_I (ioend->io_inode), ioend->io_offset, ioend->io_size); }
@@ -418,8 +418,8 @@ xfs_submit_ioend(memalloc_nofs_restore (nofs_flag);/* send ioends that might require a transaction to the completion wq */ -if (xfs_ioend_is_append (ioend) || ioend->io_type ==IOMAP_UNWRITTEN || - (ioend->io_flags &IOMAP_F_SHARED )) +if (xfs_ioend_is_append (ioend) || + (ioend->io_flags & (IOMAP_IOEND_UNWRITTEN |IOMAP_IOEND_SHARED ))) ioend->io_bio.bi_end_io = xfs_end_bio;if (status)
include/linux/iomap.h +14 -2
@@ -327,13 +327,25 @@ loff_t iomap_seek_data(struct inode *inode, loff_t offset,sector_t iomap_bmap (struct address_space *mapping,sector_t bno,const struct iomap_ops *ops); +/* +* Flags for iomap_ioend->io_flags. +*/ +/* shared COW extent */ +#define IOMAP_IOEND_SHARED (1U << 0) +/* unwritten extent */ +#define IOMAP_IOEND_UNWRITTEN (1U << 1) +/* don't merge into previous ioend */ +#define IOMAP_IOEND_BOUNDARY (1U << 2) + +#define IOMAP_IOEND_NOMERGE_FLAGS \ + (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN) +/* * Structure for writeback I/O completions. */ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ -u16 io_type; -u16 io_flags;/* IOMAP_F_* */ +u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ size_t io_size;/* size of the extent */ loff_t io_offset;/* offset in the file */
Signature · 1 line
2.45.2
Re: [PATCH 2/8] iomap: simplify io_flags and io_type in struct iomap_ioend
Quoted 154 lines · Christoph Hellwig
The ioend fields for distinct types of I/O are a bit complicated. Consolidate them into a single
io_flagfield with it's own flags decoupled from the iomap flags. This also prepares for adding a new flag that is unrelated to both of the iomap namespaces.Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/buffered-io.c | 39 ++++++++++++++++++++++----------------- fs/xfs/xfs_aops.c | 12 ++++++------ include/linux/iomap.h | 16 ++++++++++++++-- 3 files changed, 42 insertions(+), 25 deletions(-)fs/iomap/buffered-io.c +22
-17@@ -1605,13 +1605,10 @@ iomap_ioend_can_merge(struct iomap_ioend *ioend, struct iomap_ioend *next) {if (ioend->io_bio.bi_status != next->io_bio.bi_status)return false; -if (next->io_flags &IOMAP_F_BOUNDARY ) +if (next->io_flags &IOMAP_IOEND_BOUNDARY )return false; -if ((ioend->io_flags &IOMAP_F_SHARED ) ^ - (next->io_flags &IOMAP_F_SHARED )) -return false; -if ((ioend->io_type ==IOMAP_UNWRITTEN ) ^ - (next->io_type ==IOMAP_UNWRITTEN )) +if ((ioend->io_flags &IOMAP_IOEND_NOMERGE_FLAGS ) != + (next->io_flags &IOMAP_IOEND_NOMERGE_FLAGS ))return false;if (ioend->io_offset + ioend->io_size != next->io_offset)return false;@@ -1709,7 +1706,8 @@ static int iomap_submit_ioend(struct iomap_writepage_ctx *wpc, int error) }static struct iomap_ioend *iomap_alloc_ioend (struct iomap_writepage_ctx *wpc, -struct writeback_control *wbc,struct inode *inode,loff_t pos) +struct writeback_control *wbc,struct inode *inode,loff_t pos, +u16 ioend_flags) {struct iomap_ioend *ioend;struct bio *bio;@@ -1724,8 +1722,7 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, ioend =iomap_ioend_from_bio (bio);INIT_LIST_HEAD (&ioend->io_list); - ioend->io_type = wpc->iomap.type; - ioend->io_flags = wpc->iomap.flags; + ioend->io_flags = ioend_flags;if (pos > wpc->iomap.offset) wpc->iomap.flags &= ~IOMAP_F_BOUNDARY ; ioend->io_inode = inode;@@ -1737,14 +1734,13 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc,return ioend; } -static bool iomap_can_add_to_ioend (struct iomap_writepage_ctx *wpc,loff_t pos) +static bool iomap_can_add_to_ioend (struct iomap_writepage_ctx *wpc,loff_t pos, +u16 ioend_flags) { -if (wpc->iomap.offset == pos && (wpc->iomap.flags &IOMAP_F_BOUNDARY )) -return false; -if ((wpc->iomap.flags &IOMAP_F_SHARED ) != - (wpc->ioend->io_flags &IOMAP_F_SHARED )) +if (ioend_flags &IOMAP_IOEND_BOUNDARY )return false; -if (wpc->iomap.type != wpc->ioend->io_type) +if ((ioend_flags &IOMAP_IOEND_NOMERGE_FLAGS ) != + (wpc->ioend->io_flags &IOMAP_IOEND_NOMERGE_FLAGS ))return false;if (pos != wpc->ioend->io_offset + wpc->ioend->io_size)return false;@@ -1778,14 +1774,23 @@ static int iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, {struct iomap_folio_state *ifs = folio->private;size_t poff =offset_in_folio (folio, pos); +unsigned int ioend_flags =0 ;int error; -if (!wpc->ioend || !iomap_can_add_to_ioend (wpc, pos)) { +if (wpc->iomap.type ==IOMAP_UNWRITTEN ) + ioend_flags |=IOMAP_IOEND_UNWRITTEN ; +if (wpc->iomap.flags &IOMAP_F_SHARED ) + ioend_flags |=IOMAP_IOEND_SHARED ; +if (pos == wpc->iomap.offset && (wpc->iomap.flags &IOMAP_F_BOUNDARY )) + ioend_flags |=IOMAP_IOEND_BOUNDARY ; + +if (!wpc->ioend || !iomap_can_add_to_ioend (wpc, pos, ioend_flags)) { new_ioend: error =iomap_submit_ioend (wpc,0 );if (error)return error; - wpc->ioend =iomap_alloc_ioend (wpc, wbc, inode, pos); + wpc->ioend =iomap_alloc_ioend (wpc, wbc, inode, pos, + ioend_flags); }if (!bio_add_folio (&wpc->ioend->io_bio, folio, len, poff))fs/xfs/xfs_aops.c +6
-6@@ -114,7 +114,7 @@ xfs_end_ioend(*/ error =blk_status_to_errno (ioend->io_bio.bi_status);if (unlikely (error)) { -if (ioend->io_flags &IOMAP_F_SHARED ) { +if (ioend->io_flags &IOMAP_IOEND_SHARED ) {xfs_reflink_cancel_cow_range (ip, offset, size, true);xfs_bmap_punch_delalloc_range (ip,XFS_DATA_FORK , offset, offset + size);@@ -125,9 +125,9 @@ xfs_end_ioend(/* * Success: commit the COW or unwritten blocks if needed. */ -if (ioend->io_flags &IOMAP_F_SHARED ) +if (ioend->io_flags &IOMAP_IOEND_SHARED ) error =xfs_reflink_end_cow (ip, offset, size); -else if (ioend->io_type ==IOMAP_UNWRITTEN ) +else if (ioend->io_flags &IOMAP_IOEND_UNWRITTEN ) error =xfs_iomap_write_unwritten (ip, offset, size, false);if (!error &&xfs_ioend_is_append (ioend))@@ -410,7 +410,7 @@ xfs_submit_ioend( nofs_flag =memalloc_nofs_save ();/* Convert CoW extents to regular */ -if (!status && (ioend->io_flags &IOMAP_F_SHARED )) { +if (!status && (ioend->io_flags &IOMAP_IOEND_SHARED )) { status =xfs_reflink_convert_cow (XFS_I (ioend->io_inode), ioend->io_offset, ioend->io_size); }@@ -418,8 +418,8 @@ xfs_submit_ioend(memalloc_nofs_restore (nofs_flag);/* send ioends that might require a transaction to the completion wq */ -if (xfs_ioend_is_append (ioend) || ioend->io_type ==IOMAP_UNWRITTEN || - (ioend->io_flags &IOMAP_F_SHARED )) +if (xfs_ioend_is_append (ioend) || + (ioend->io_flags & (IOMAP_IOEND_UNWRITTEN |IOMAP_IOEND_SHARED ))) ioend->io_bio.bi_end_io = xfs_end_bio;if (status)include/linux/iomap.h +2
-0@@ -327,13 +327,25 @@ loff_t iomap_seek_data(struct inode *inode, loff_t offset,sector_t iomap_bmap (struct address_space *mapping,sector_t bno,const struct iomap_ops *ops); +/* +* Flags for iomap_ioend->io_flags.
include/linux/iomap.h loff_t iomap_seek_data(struct inode *inode, loff_t offset, · line 341
Quoted 10 lines
include/linux/iomap.h +10
-0+*/ +/* shared COW extent */ +#define IOMAP_IOEND_SHARED (1U << 0) +/* unwritten extent */ +#define IOMAP_IOEND_UNWRITTEN (1U << 1) +/* don't merge into previous ioend */ +#define IOMAP_IOEND_BOUNDARY (1U << 2) + +#define IOMAP_IOEND_NOMERGE_FLAGS \ + (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN)
Hmm. At first I wondered "Why wouldn't BOUNDARY be in here too? It also prevents merging of ioends." Then I remembered that BOUNDARY is an explicit nomerge flag, whereas what NOMERGE_FLAGS provides is that we always split ioends whenever the ioend work changes.
How about a comment?
/* split ioends when the type of completion work changes */ #define IOMAP_IOEND_NOMERGE_FLAGS \
(IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN)
Otherwise this looks fine to me.
--D
Quoted 16 lines
include/linux/iomap.h +2
-2+/* * Structure for writeback I/O completions. */ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ -u16 io_type; -u16 io_flags;/* IOMAP_F_* */ +u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ size_t io_size;/* size of the extent */ loff_t io_offset;/* offset in the file */ Signature · 1 line
2.45.2
Re: [PATCH 2/8] iomap: simplify io_flags and io_type in struct iomap_ioend
Quoted 8 lines · Darrick J. Wong
include/linux/iomap.h loff_t iomap_seek_data(struct inode *inode, loff_t offset, · line 341
Quoted 2 lines
include/linux/iomap.h +2
-0+#define IOMAP_IOEND_NOMERGE_FLAGS \ + (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN)Hmm. At first I wondered "Why wouldn't BOUNDARY be in here too? It also prevents merging of ioends." Then I remembered that BOUNDARY is an explicit nomerge flag, whereas what
NOMERGE_FLAGSprovides is that we always split ioends whenever the ioend work changes.
Quoted 7 lines
How about a comment?
/* split ioends when the type of completion work changes */ #define
IOMAP_IOEND_NOMERGE_FLAGS\
(IOMAP_IOEND_SHARED|IOMAP_IOEND_UNWRITTEN)Otherwise this looks fine to me.
The interesting thing about BOUNDARY is not just that it's explicit, but also that it's one-way. We can merge a non-BOUNDARY flag into the end of a BOUNDARY one, just not a BOUNDARY one into the end of a non-BOUNDARY one.
Quoted 19 lines
--D
Quoted 16 lines
include/linux/iomap.h +2
-2+/* * Structure for writeback I/O completions. */ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ -u16 io_type; -u16 io_flags;/* IOMAP_F_* */ +u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ size_t io_size;/* size of the extent */ loff_t io_offset;/* offset in the file */ Signature · 1 line
2.45.2
---end quoted text---
PATCH 3/8 iomap: add a IOMAP_F_ZONE_APPEND flag
In reply to Christoph HellwigThis doesn't much - just always returns the start block number for each iomap instead of increasing it. This is because we'll keep building bios unconstrained by the hardware limits and just split them in file system submission handler.
Maybe we should find another name for it, because it might be useful for btrfs compressed bio submissions as well, but I can't come up with a good one.
--- fs/iomap/buffered-io.c | 19 ++++++++++++++++--- include/linux/iomap.h | 7 +++++++ 2 files changed, 23 insertions(+), 3 deletions(-)
fs/iomap/buffered-io.c +16 -3
@@ -1744,9 +1744,22 @@ static bool iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t pos,return false;if (pos != wpc->ioend->io_offset + wpc->ioend->io_size)return false; -if (iomap_sector (&wpc->iomap, pos) != -bio_end_sector (&wpc->ioend->io_bio)) -return false; +if (wpc->iomap.flags &IOMAP_F_ZONE_APPEND ) { +/* +* For Zone Append command, bi_sector points to the zone start +* before submission. We can merge all I/O for the same zone. +*/ +if (iomap_sector (&wpc->iomap, pos) != + wpc->ioend->io_bio.bi_iter.bi_sector) +return false; + }else { +/* +* For regular writes, the disk blocks needs to be contiguous. +*/ +if (iomap_sector (&wpc->iomap, pos) != +bio_end_sector (&wpc->ioend->io_bio)) +return false; + }/* * Limit ioend bio chain lengths to minimise IO completion latency. This * also prevents long tight loops ending page writeback on all the
include/linux/iomap.h +7 -0
@@ -56,6 +56,10 @@ struct vm_fault;* * IOMAP_F_BOUNDARY indicates that I/O and I/O completions for this iomap must * never be merged with the mapping before it. +* +* IOMAP_F_ZONE_APPEND indicates that (write) I/O should be done as a zone +* append command for zoned devices. Note that the file system needs to +* override the bi_end_io handler to record the actual written sector. */ #define IOMAP_F_NEW (1U << 0)#define IOMAP_F_DIRTY (1U << 1)
@@ -68,6 +72,7 @@ struct vm_fault;#endif /* CONFIG_BUFFER_HEAD */ #define IOMAP_F_XATTR (1U << 5)#define IOMAP_F_BOUNDARY (1U << 6) +#define IOMAP_F_ZONE_APPEND (1U << 7)/* * Flags set by the core iomap code during operations:
@@ -111,6 +116,8 @@ struct iomap {static inline sector_t iomap_sector (const struct iomap *iomap,loff_t pos) { +if (iomap->flags &IOMAP_F_ZONE_APPEND ) +return iomap->addr >>SECTOR_SHIFT ;return (iomap->addr + pos - iomap->offset) >>SECTOR_SHIFT ; }
Signature · 1 line
2.45.2
Re: [PATCH 3/8] iomap: add a IOMAP_F_ZONE_APPEND flag
Quoted 1 line · Christoph Hellwig
This doesn't much - just always returns the start block number for each
"This doesn't much" - I don't understand the sentence very well. How about:
"Add a new IOMAP_F_ZONE_APPEND flag for the filesystem to indicate that the storage device must inform us where it wrote the data, so that the filesystem can update its own internal mapping metadata. The filesystem should set the starting address of the zone in iomap::addr, and extract the LBA address from the bio during ioend processing. iomap builds bios unconstrained by the hardware limits and will split them in the bio submission handler."
The splitting happens whenever IOMAP_F_BOUNDARY gets set by ->map_blocks, right?
Quoted 7 lines
iomap instead of increasing it. This is because we'll keep building bios unconstrained by the hardware limits and just split them in file system submission handler.
Maybe we should find another name for it, because it might be useful for btrfs compressed bio submissions as well, but I can't come up with a good one.
Since you have to tell the device the starting LBA of the zone you want to write to, I think _ZONE_APPEND is a reasonable name. The code looks correct though.
Quoted 52 lines
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/buffered-io.c | 19 ++++++++++++++++--- include/linux/iomap.h | 7 +++++++ 2 files changed, 23 insertions(+), 3 deletions(-)fs/iomap/buffered-io.c +16
-3@@ -1744,9 +1744,22 @@ static bool iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t pos,return false;if (pos != wpc->ioend->io_offset + wpc->ioend->io_size)return false; -if (iomap_sector (&wpc->iomap, pos) != -bio_end_sector (&wpc->ioend->io_bio)) -return false; +if (wpc->iomap.flags &IOMAP_F_ZONE_APPEND ) { +/* +* For Zone Append command, bi_sector points to the zone start +* before submission. We can merge all I/O for the same zone. +*/ +if (iomap_sector (&wpc->iomap, pos) != + wpc->ioend->io_bio.bi_iter.bi_sector) +return false; + }else { +/* +* For regular writes, the disk blocks needs to be contiguous. +*/ +if (iomap_sector (&wpc->iomap, pos) != +bio_end_sector (&wpc->ioend->io_bio)) +return false; + }/* * Limit ioend bio chain lengths to minimise IO completion latency. This * also prevents long tight loops ending page writeback on all the include/linux/iomap.h +4
-0@@ -56,6 +56,10 @@ struct vm_fault;* * IOMAP_F_BOUNDARY indicates that I/O and I/O completions for this iomap must * never be merged with the mapping before it. +* +* IOMAP_F_ZONE_APPEND indicates that (write) I/O should be done as a zone +* append command for zoned devices. Note that the file system needs to +* override the bi_end_io handler to record the actual written sector. */ #define IOMAP_F_NEW (1U << 0)#define IOMAP_F_DIRTY (1U << 1)
include/linux/iomap.h struct vm_fault; · line 75
Quoted 4 lines
include/linux/iomap.h +1
-0@@ -68,6 +72,7 @@ struct vm_fault;#endif /* CONFIG_BUFFER_HEAD */ #define IOMAP_F_XATTR (1U << 5)#define IOMAP_F_BOUNDARY (1U << 6) +#define IOMAP_F_ZONE_APPEND (1U << 7)
Needs a corresponding update in Documentation/iomap/ before we merge this series.
--D
Quoted 16 lines
include/linux/iomap.h +2
-0/* * Flags set by the core iomap code during operations: @@ -111,6 +116,8 @@ struct iomap {static inline sector_t iomap_sector (const struct iomap *iomap,loff_t pos) { +if (iomap->flags &IOMAP_F_ZONE_APPEND ) +return iomap->addr >>SECTOR_SHIFT ;return (iomap->addr + pos - iomap->offset) >>SECTOR_SHIFT ; }Signature · 1 line
2.45.2
Re: [PATCH 3/8] iomap: add a IOMAP_F_ZONE_APPEND flag
Quoted 6 lines · Darrick J. Wong
Quoted 1 line · Christoph Hellwig
This doesn't much - just always returns the start block number for each
"This doesn't much" - I don't understand the sentence very well. How about:
Quoted 7 lines
"Add a new
IOMAP_F_ZONE_APPENDflag for the filesystem to indicate that the storage device must inform us where it wrote the data, so that the filesystem can update its own internal mapping metadata. The filesystem should set the starting address of the zone iniomap::addr, and extract the LBA address from the bio during ioend processing. iomap builds bios unconstrained by the hardware limits and will split them in the bio submission handler."
Sounds reasonable.
Quoted 2 lines
The splitting happens whenever
IOMAP_F_BOUNDARYgets set by ->map_blocks, right?
The splitting happens when:
(a) we run out of enough space in one zone and have to switch to another one. That then most likely picks an new zone, in which case IOMAP_F_BOUNDARY will be set
(b) if the hardware I/O constraints require splitting the I/O (typically because it's too larger)
Re: [PATCH 3/8] iomap: add a IOMAP_F_ZONE_APPEND flag
In reply to Darrick J. WongQuoted 7 lines · Darrick J. Wong
Quoted 4 lines
#endif /*
CONFIG_BUFFER_HEAD*/
#defineIOMAP_F_XATTR(1U << 5)
#defineIOMAP_F_BOUNDARY(1U << 6)
+#defineIOMAP_F_ZONE_APPEND(1U << 7)Needs a corresponding update in
Documentation/iomap/before we merge this series.
Btw, while doing these updates I start to really despise Documentation/filesystems/iomap/. It basically just duplicates what's alredy in the code comments, just far away from the code to which it belongs in that awkward RST format. I'm not sure what it is supposed to buy us?
Re: [PATCH 3/8] iomap: add a IOMAP_F_ZONE_APPEND flag
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines
#endif /*
CONFIG_BUFFER_HEAD*/
#defineIOMAP_F_XATTR(1U << 5)
#defineIOMAP_F_BOUNDARY(1U << 6)
+#defineIOMAP_F_ZONE_APPEND(1U << 7)
Quoted 8 lines
Quoted 2 lines
Needs a corresponding update in
Documentation/iomap/before we merge this series.Btw, while doing these updates I start to really despise
Documentation/filesystems/iomap/. It basically just duplicates what's alredy in the code comments, just far away from the code to which it belongs in that awkward RST format. I'm not sure what it is supposed to buy us?
The iomap documentation has been helpful for the people working on porting existing filesystems to iomap, because it provides a story for "if you want to implement file operation X, this is how you'd do that in iomap". Seeing as we're coming up on -rc4 now, do you want to just send your iomap patches and I'll go deal with the docs?
That way we have two people who understand what the two new flags mean. :)
--D
Re: [PATCH 3/8] iomap: add a IOMAP_F_ZONE_APPEND flag
Quoted 5 lines · Darrick J. Wong
The iomap documentation has been helpful for the people working on porting existing filesystems to iomap, because it provides a story for "if you want to implement file operation X, this is how you'd do that in iomap". Seeing as we're coming up on -rc4 now, do you want to just send your iomap patches and I'll go deal with the docs?
I've actually writtem (or rahter copy and pasted) the documentation already, that's what made me notice how duplicative it is. But let me shoot out what I currently have.
Re: [PATCH 3/8] iomap: add a IOMAP_F_ZONE_APPEND flag
Quoted 10 lines · Christoph Hellwig
Quoted 5 lines · Darrick J. Wong
The iomap documentation has been helpful for the people working on porting existing filesystems to iomap, because it provides a story for "if you want to implement file operation X, this is how you'd do that in iomap". Seeing as we're coming up on -rc4 now, do you want to just send your iomap patches and I'll go deal with the docs?
I've actually writtem (or rahter copy and pasted) the documentation already, that's what made me notice how duplicative it is. But let me shoot out what I currently have.
Oh, ok. I was planning to repost the rtrmap/reflink series shortly FYI.
--D
PATCH 4/8 iomap: split bios to zone append limits in the submission handlers
In reply to Christoph HellwigProvide helpers for file systems to split bios in the direct I/O and writeback I/O submission handlers.
This Follows btrfs' lead and don't try to build bios to hardware limits for zone append commands, but instead build them as normal unconstrained bios and split them to the hardware limits in the I/O submission handler.
--- fs/iomap/Makefile | 1 + fs/iomap/buffered-io.c | 43 ++++++++++++++----------- fs/iomap/ioend.c | 73 ++++++++++++++++++++++++++++++++++++++++++ include/linux/iomap.h | 9 ++++++ 4 files changed, 108 insertions(+), 18 deletions(-) create mode 100644 fs/iomap/ioend.c
fs/iomap/Makefile +1 -0
@@ -12,6 +12,7 @@ iomap-y += trace.o \ iter.o iomap-$(CONFIG_BLOCK) += buffered-io.o \ direct-io.o \ + ioend.o \ fiemap.o \ seek.o iomap-$(CONFIG_SWAP) += swapfile.o
fs/iomap/buffered-io.c +25 -18
81 lines not on this page · raw message
fs/iomap/ioend.c new file +73 -0
73 lines not on this page · raw message
include/linux/iomap.h +9 -0
@@ -354,6 +354,9 @@ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ +atomic_t io_remaining;/* completetion defer count */ +int io_error;/* stashed away status */ +struct iomap_ioend *io_parent;/* parent for completions */ size_t io_size;/* size of the extent */ loff_t io_offset;/* offset in the file */ sector_t io_sector;/* start sector of ioend */
@@ -404,6 +407,10 @@ struct iomap_writepage_ctx {u32 nr_folios;/* folios added to the ioend */ }; +struct iomap_ioend *iomap_init_ioend (struct inode *inode,struct bio *bio, +loff_t file_offset,u16 flags); +struct iomap_ioend *iomap_split_ioend (struct iomap_ioend *ioend,bool is_append, +unsigned int *alloc_len);void iomap_finish_ioends (struct iomap_ioend *ioend,int error);void iomap_ioend_try_merge (struct iomap_ioend *ioend,struct list_head *more_ioends);
@@ -475,4 +482,6 @@ int iomap_swapfile_activate(struct swap_info_struct *sis,# define iomap_swapfile_activate (sis, swapfile, pagespan, ops) (-EIO)#endif /* CONFIG_SWAP */ +extern struct bio_set iomap_ioend_bioset; +#endif /* LINUX_IOMAP_H */
Signature · 1 line
2.45.2
Re: [PATCH 4/8] iomap: split bios to zone append limits in the submission handlers
Quoted 16 lines · Christoph Hellwig
Provide helpers for file systems to split bios in the direct I/O and writeback I/O submission handlers.
This Follows btrfs' lead and don't try to build bios to hardware limits for zone append commands, but instead build them as normal unconstrained bios and split them to the hardware limits in the I/O submission handler.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/Makefile | 1 + fs/iomap/buffered-io.c | 43 ++++++++++++++----------- fs/iomap/ioend.c | 73 ++++++++++++++++++++++++++++++++++++++++++ include/linux/iomap.h | 9 ++++++ 4 files changed, 108 insertions(+), 18 deletions(-) create mode 100644 fs/iomap/ioend.c
...
fs/iomap/ioend.c
Quoted 5 lines
fs/iomap/ioend.c new file +0
-0
...
It might be useful to add a small comment here to point out this splits from the front of the ioend (i.e. akin to bio_split()), documents the params, and maybe mentions the ioend relationship requirements (i.e. according to bio_split(), the split ioend bio refers to the vectors in the original ioend bio).
Brian
Quoted 85 lines
fs/iomap/ioend.c new file +49
-0+struct iomap_ioend *iomap_split_ioend (struct iomap_ioend *ioend,bool is_append, +unsigned int *alloc_len) +{ +struct bio *bio = &ioend->io_bio; +struct iomap_ioend *split_ioend; +struct bio *split; +int sector_offset; +unsigned int nr_segs; + +if (is_append) { +struct queue_limits *lim =bdev_limits (bio->bi_bdev); + + sector_offset =bio_split_rw_at (bio, lim, &nr_segs, +min (lim->max_zone_append_sectors <<SECTOR_SHIFT , + *alloc_len)); +if (!sector_offset) +return NULL ; + }else { +if (bio->bi_iter.bi_size <= *alloc_len) +return NULL ; + sector_offset = *alloc_len >>SECTOR_SHIFT ; + } + +/* ensure the split ioend is still block size aligned */ + sector_offset =ALIGN_DOWN (sector_offset <<SECTOR_SHIFT , +i_blocksize (ioend->io_inode)) >>SECTOR_SHIFT ; + + split =bio_split (bio, sector_offset,GFP_NOFS , &iomap_ioend_bioset); +if (!split) +return NULL ; + split->bi_private = bio->bi_private; + split->bi_end_io = bio->bi_end_io; + + split_ioend =iomap_init_ioend (ioend->io_inode, split, ioend->io_offset, + ioend->io_flags); + split_ioend->io_parent = ioend; + +atomic_inc (&ioend->io_remaining); + ioend->io_offset += split_ioend->io_size; + ioend->io_size -= split_ioend->io_size; + + split_ioend->io_sector = ioend->io_sector; +if (!is_append) + ioend->io_sector += (split_ioend->io_size >>SECTOR_SHIFT ); + + *alloc_len -= split->bi_iter.bi_size; +return split_ioend; +} +EXPORT_SYMBOL_GPL (iomap_split_ioend);include/linux/iomap.h +9
-0@@ -354,6 +354,9 @@ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ +atomic_t io_remaining;/* completetion defer count */ +int io_error;/* stashed away status */ +struct iomap_ioend *io_parent;/* parent for completions */ size_t io_size;/* size of the extent */ loff_t io_offset;/* offset in the file */ sector_t io_sector;/* start sector of ioend */ @@ -404,6 +407,10 @@ struct iomap_writepage_ctx {u32 nr_folios;/* folios added to the ioend */ }; +struct iomap_ioend *iomap_init_ioend (struct inode *inode,struct bio *bio, +loff_t file_offset,u16 flags); +struct iomap_ioend *iomap_split_ioend (struct iomap_ioend *ioend,bool is_append, +unsigned int *alloc_len);void iomap_finish_ioends (struct iomap_ioend *ioend,int error);void iomap_ioend_try_merge (struct iomap_ioend *ioend,struct list_head *more_ioends);@@ -475,4 +482,6 @@ int iomap_swapfile_activate(struct swap_info_struct *sis,# define iomap_swapfile_activate (sis, swapfile, pagespan, ops) (-EIO)#endif /* CONFIG_SWAP */ +extern struct bio_set iomap_ioend_bioset; +#endif /* LINUX_IOMAP_H */ Signature · 1 line
2.45.2
Re: [PATCH 4/8] iomap: split bios to zone append limits in the submission handlers
Quoted 5 lines · Brian Foster
It might be useful to add a small comment here to point out this splits from the front of the ioend (i.e. akin to
bio_split()), documents the params, and maybe mentions the ioend relationship requirements (i.e. according tobio_split(), the split ioend bio refers to the vectors in the original ioend bio).
Sure, I can add that.
Re: [PATCH 4/8] iomap: split bios to zone append limits in the submission handlers
In reply to Christoph Hellwigfs/iomap/ioend.c line 40
Quoted 7 lines · Christoph Hellwig
fs/iomap/ioend.c new file +7
-0+if (is_append) { +struct queue_limits *lim =bdev_limits (bio->bi_bdev); + + sector_offset =bio_split_rw_at (bio, lim, &nr_segs, +min (lim->max_zone_append_sectors <<SECTOR_SHIFT , + *alloc_len)); +if (!sector_offset)
Should this be:
if (sector_offset <= 0)
Quoted 10 lines
fs/iomap/ioend.c new file +10
-0+return NULL ; + }else { +if (bio->bi_iter.bi_size <= *alloc_len) +return NULL ; + sector_offset = *alloc_len >>SECTOR_SHIFT ; + } + +/* ensure the split ioend is still block size aligned */ + sector_offset =ALIGN_DOWN (sector_offset <<SECTOR_SHIFT , +i_blocksize (ioend->io_inode)) >>SECTOR_SHIFT ;
Re: [PATCH 4/8] iomap: split bios to zone append limits in the submission handlers
Quoted 11 lines · John Garry
fs/iomap/ioend.c line 40
Quoted 7 lines · Christoph Hellwig
fs/iomap/ioend.c new file +7
-0+if (is_append) { +struct queue_limits *lim =bdev_limits (bio->bi_bdev); + + sector_offset =bio_split_rw_at (bio, lim, &nr_segs, +min (lim->max_zone_append_sectors <<SECTOR_SHIFT , + *alloc_len)); +if (!sector_offset)Should this be:
Quoted 1 line
if (sector_offset <= 0)
No support for REQ_ATOMIC and REQ_NOWAIT in this path right now, but we might as well future prove it by checking for a negative error value. But we'll then need to propagate the error as well. I'll see what I can do.
Re: [PATCH 4/8] iomap: split bios to zone append limits in the submission handlers
In reply to Christoph HellwigQuoted 124 lines · Christoph Hellwig
Provide helpers for file systems to split bios in the direct I/O and writeback I/O submission handlers.
This Follows btrfs' lead and don't try to build bios to hardware limits for zone append commands, but instead build them as normal unconstrained bios and split them to the hardware limits in the I/O submission handler.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/Makefile | 1 + fs/iomap/buffered-io.c | 43 ++++++++++++++----------- fs/iomap/ioend.c | 73 ++++++++++++++++++++++++++++++++++++++++++ include/linux/iomap.h | 9 ++++++ 4 files changed, 108 insertions(+), 18 deletions(-) create mode 100644 fs/iomap/ioend.cfs/iomap/Makefile +1
-0@@ -12,6 +12,7 @@ iomap-y += trace.o \ iter.o iomap-$(CONFIG_BLOCK) += buffered-io.o \ direct-io.o \ + ioend.o \ fiemap.o \ seek.o iomap-$(CONFIG_SWAP) += swapfile.ofs/iomap/buffered-io.c +25
-18@@ -40,7 +40,8 @@ struct iomap_folio_state {unsigned long state[]; }; -static struct bio_set iomap_ioend_bioset; +struct bio_set iomap_ioend_bioset; +EXPORT_SYMBOL_GPL (iomap_ioend_bioset);static inline bool ifs_is_fully_uptodate (struct folio *folio,struct iomap_folio_state *ifs)@@ -1539,15 +1540,15 @@ static void iomap_finish_folio_write(struct inode *inode, struct folio *folio,* ioend after this. */ static u32 -iomap_finish_ioend (struct iomap_ioend *ioend,int error) +iomap_finish_ioend_buffered (struct iomap_ioend *ioend) {struct inode *inode = ioend->io_inode;struct bio *bio = &ioend->io_bio;struct folio_iter fi;u32 folio_count =0 ; -if (error) { -mapping_set_error (inode->i_mapping, error); +if (ioend->io_error) { +mapping_set_error (inode->i_mapping, ioend->io_error);if (!bio_flagged (bio,BIO_QUIET )) { pr_err_ratelimited( "%s: writeback error on inode %lu, offset %lld, sector %llu",@@ -1566,6 +1567,24 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error)return folio_count; } +static u32 +iomap_finish_ioend (struct iomap_ioend *ioend,int error) +{ +if (ioend->io_parent) { +struct bio *bio = &ioend->io_bio; + + ioend = ioend->io_parent; +bio_put (bio); + } + +if (error) +cmpxchg (&ioend->io_error,0 , error); + +if (!atomic_dec_and_test (&ioend->io_remaining)) +return 0 ; +return iomap_finish_ioend_buffered (ioend); +} +/* * Ioend completion routine for merged bios. This can only be called from task * contexts as merged ioends can be of unbound length. Hence we have to break up @@ -1709,7 +1728,6 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct inode *inode, loff_t pos, u16 ioend_flags) { - struct iomap_ioend *ioend; struct bio *bio; bio = bio_alloc_bioset(wpc->iomap.bdev, BIO_MAX_VECS,@@ -1717,21 +1735,10 @@ static struct iomap_ioend *iomap_alloc_ioend(struct iomap_writepage_ctx *wpc, GFP_NOFS, &iomap_ioend_bioset); bio->bi_iter.bi_sector =iomap_sector (&wpc->iomap, pos); bio->bi_end_io = iomap_writepage_end_bio; -wbc_init_bio (wbc, bio); bio->bi_write_hint = inode->i_write_hint; - - ioend =iomap_ioend_from_bio (bio); -INIT_LIST_HEAD (&ioend->io_list); - ioend->io_flags = ioend_flags; -if (pos > wpc->iomap.offset) - wpc->iomap.flags &= ~IOMAP_F_BOUNDARY ; - ioend->io_inode = inode; - ioend->io_size =0 ; - ioend->io_offset = pos; - ioend->io_sector = bio->bi_iter.bi_sector; - +wbc_init_bio (wbc, bio); wpc->nr_folios =0 ; -return ioend; +return iomap_init_ioend (inode, bio, pos, ioend_flags); } static bool iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t pos,fs/iomap/ioend.c new file +0
-0
fs/iomap/ioend.c line 9
Quoted 9 lines
fs/iomap/ioend.c new file +9
-0@@ -0,0 +1,73 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2024 Christoph Hellwig. +*/ +#include <linux/iomap.h> + +struct iomap_ioend *iomap_init_ioend (struct inode *inode, +struct bio *bio,loff_t file_offset,u16 flags) +{
Nit: s/flags/ioend_flags/ to be consistent with the previous few patches.
Quoted 6 lines
fs/iomap/ioend.c new file +6
-0+struct iomap_ioend *ioend =iomap_ioend_from_bio (bio); + +atomic_set (&ioend->io_remaining,1 ); + ioend->io_error =0 ; + ioend->io_parent =NULL ; +INIT_LIST_HEAD (&ioend->io_list);
fs/iomap/ioend.c line 25
Quoted 10 lines
fs/iomap/ioend.c new file +10
-0+ ioend->io_flags = flags; + ioend->io_inode = inode; + ioend->io_offset = file_offset; + ioend->io_size = bio->bi_iter.bi_size; + ioend->io_sector = bio->bi_iter.bi_sector; +return ioend; +} +EXPORT_SYMBOL_GPL (iomap_init_ioend); + +struct iomap_ioend *iomap_split_ioend(struct iomap_ioend *ioend, bool is_append,
Can you determine is_append from (ioend->io_flags & ZONE_APPEND)?
Also it's not clear to me what the initial and output state of *alloc_len is supposed to be? I guess you set it to the number of bytes the @ioend covers? And this function either returns NULL and alloc_len untouched; or it returns a new ioend and the number of bytes remaining in the passed-in ioend?
(or, as bfoster said, please improve the comments)
Quoted 53 lines
fs/iomap/ioend.c new file +48
-0+ unsigned int *alloc_len) +{ +struct bio *bio = &ioend->io_bio; +struct iomap_ioend *split_ioend; +struct bio *split; +int sector_offset; +unsigned int nr_segs; + +if (is_append) { +struct queue_limits *lim =bdev_limits (bio->bi_bdev); + + sector_offset =bio_split_rw_at (bio, lim, &nr_segs, +min (lim->max_zone_append_sectors <<SECTOR_SHIFT , + *alloc_len)); +if (!sector_offset) +return NULL ; + }else { +if (bio->bi_iter.bi_size <= *alloc_len) +return NULL ; + sector_offset = *alloc_len >>SECTOR_SHIFT ; + } + +/* ensure the split ioend is still block size aligned */ + sector_offset =ALIGN_DOWN (sector_offset <<SECTOR_SHIFT , +i_blocksize (ioend->io_inode)) >>SECTOR_SHIFT ; + + split =bio_split (bio, sector_offset,GFP_NOFS , &iomap_ioend_bioset); +if (!split) +return NULL ; + split->bi_private = bio->bi_private; + split->bi_end_io = bio->bi_end_io; + + split_ioend =iomap_init_ioend (ioend->io_inode, split, ioend->io_offset, + ioend->io_flags); + split_ioend->io_parent = ioend; + +atomic_inc (&ioend->io_remaining); + ioend->io_offset += split_ioend->io_size; + ioend->io_size -= split_ioend->io_size; + + split_ioend->io_sector = ioend->io_sector; +if (!is_append) + ioend->io_sector += (split_ioend->io_size >>SECTOR_SHIFT ); + + *alloc_len -= split->bi_iter.bi_size; +return split_ioend; +} +EXPORT_SYMBOL_GPL (iomap_split_ioend);include/linux/iomap.h +0
-0
include/linux/iomap.h struct iomap_ioend { · line 359
Quoted 6 lines
include/linux/iomap.h +3
-0@@ -354,6 +354,9 @@ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ +atomic_t io_remaining;/* completetion defer count */ +int io_error;/* stashed away status */ +struct iomap_ioend *io_parent;/* parent for completions */
I guess this means ioends can chain together, sort of like how bios can when you split them?
--D
Quoted 25 lines
include/linux/iomap.h +6
-0size_t io_size;/* size of the extent */ loff_t io_offset;/* offset in the file */ sector_t io_sector;/* start sector of ioend */ @@ -404,6 +407,10 @@ struct iomap_writepage_ctx {u32 nr_folios;/* folios added to the ioend */ }; +struct iomap_ioend *iomap_init_ioend (struct inode *inode,struct bio *bio, +loff_t file_offset,u16 flags); +struct iomap_ioend *iomap_split_ioend (struct iomap_ioend *ioend,bool is_append, +unsigned int *alloc_len);void iomap_finish_ioends (struct iomap_ioend *ioend,int error);void iomap_ioend_try_merge (struct iomap_ioend *ioend,struct list_head *more_ioends);@@ -475,4 +482,6 @@ int iomap_swapfile_activate(struct swap_info_struct *sis,# define iomap_swapfile_activate (sis, swapfile, pagespan, ops) (-EIO)#endif /* CONFIG_SWAP */ +extern struct bio_set iomap_ioend_bioset; +#endif /* LINUX_IOMAP_H */ Signature · 1 line
2.45.2
Re: [PATCH 4/8] iomap: split bios to zone append limits in the submission handlers
Quoted 3 lines · Darrick J. Wong
Quoted 1 line
+struct
iomap_ioend*iomap_split_ioend(structiomap_ioend*ioend, boolis_append,Can you determine
is_appendfrom (ioend->io_flags&ZONE_APPEND)?
That would require us to add that flag first :) As we don't really need that as persistent per-iomap that it's probably not worth it.
Quoted 3 lines
Also it's not clear to me what the initial and output state of
*alloc_lenis supposed to be? I guess you set it to the number of bytes the @ioend covers?
It gets set to the number of blocks that the allocator could find, and iomap_split_ioend decrements the amount of that it used for the ioend returned, which is min(*alloc_len, max_zone_append_sectors) for sequential zones, or *alloc_len for conventional zones.
Quoted 11 lines
include/linux/iomap.h struct iomap_ioend { · line 359
Quoted 8 lines
+++
b/include/linux/iomap.hinclude/linux/iomap.h +3
-0@@ -354,6 +354,9 @@ struct iomap_ioend {struct list_head io_list;/* next ioend in chain */ u16 io_flags;/* IOMAP_IOEND_* */ struct inode *io_inode;/* file being written to */ +atomic_t io_remaining;/* completetion defer count */ +int io_error;/* stashed away status */ +struct iomap_ioend *io_parent;/* parent for completions */ I guess this means ioends can chain together, sort of like how bios can when you split them?
Exactly.
PATCH 5/8 iomap: optionally use ioends for direct I/O
In reply to Christoph Hellwigstruct iomap_ioend currently tracks outstanding buffered writes and has some really nice code in core iomap and XFS to merge contiguous I/Os an defer them to userspace for completion in a very efficient way.
For zoned writes we'll also need a per-bio user context completion to record the written blocks, and the infrastructure for that would look basically like the ioend handling for buffered I/O.
So instead of reinventing the wheel, reuse the existing infrastructure.
--- fs/iomap/buffered-io.c | 3 +++ fs/iomap/direct-io.c | 50 +++++++++++++++++++++++++++++++++++++++++- fs/iomap/internal.h | 7 ++++++ include/linux/iomap.h | 4 +++- 4 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 fs/iomap/internal.h
fs/iomap/buffered-io.c +3 -0
@@ -17,6 +17,7 @@#include <linux/bio.h> #include <linux/sched/signal.h> #include <linux/migrate.h> +#include "internal.h" #include "trace.h" #include "../internal.h"
@@ -1582,6 +1583,8 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error)if (!atomic_dec_and_test (&ioend->io_remaining))return 0 ; +if (ioend->io_flags &IOMAP_IOEND_DIRECT ) +return iomap_finish_ioend_direct (ioend);return iomap_finish_ioend_buffered (ioend); }
fs/iomap/direct-io.c +49 -1
74 lines not on this page · raw message
fs/iomap/internal.h new file +7 -0
@@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _IOMAP_INTERNAL_H +#define _IOMAP_INTERNAL_H 1 + +u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend); + +#endif /* _IOMAP_INTERNAL_H */
include/linux/iomap.h +3 -1
@@ -343,9 +343,11 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,#define IOMAP_IOEND_UNWRITTEN (1U << 1)/* don't merge into previous ioend */ #define IOMAP_IOEND_BOUNDARY (1U << 2) +/* is direct I/O */ +#define IOMAP_IOEND_DIRECT (1U << 3)#define IOMAP_IOEND_NOMERGE_FLAGS \ - (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN) + (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT)/* * Structure for writeback I/O completions.
Signature · 1 line
2.45.2
Re: [PATCH 5/8] iomap: optionally use ioends for direct I/O
Quoted 19 lines · Christoph Hellwig
struct
iomap_ioendcurrently tracks outstanding buffered writes and has some really nice code in core iomap and XFS to merge contiguous I/Os an defer them to userspace for completion in a very efficient way.For zoned writes we'll also need a per-bio user context completion to record the written blocks, and the infrastructure for that would look basically like the ioend handling for buffered I/O.
So instead of reinventing the wheel, reuse the existing infrastructure.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/buffered-io.c | 3 +++ fs/iomap/direct-io.c | 50 +++++++++++++++++++++++++++++++++++++++++- fs/iomap/internal.h | 7 ++++++ include/linux/iomap.h | 4 +++- 4 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 fs/iomap/internal.h
...
Quoted 4 lines
fs/iomap/direct-io.c +0
-0
...
Quoted 36 lines
fs/iomap/direct-io.c +32
-0@@ -163,6 +166,51 @@ static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret)cmpxchg (&dio->error,0 , ret); } +u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend) +{ +struct iomap_dio *dio = ioend->io_bio.bi_private; +bool should_dirty = (dio->flags &IOMAP_DIO_DIRTY ); +struct kiocb *iocb = dio->iocb; +u32 vec_count = ioend->io_bio.bi_vcnt; + +if (ioend->io_error) +iomap_dio_set_error (dio, ioend->io_error); + +if (atomic_dec_and_test (&dio->ref)) { +struct inode *inode =file_inode (iocb->ki_filp); + +if (dio->wait_for_completion) { +struct task_struct *waiter = dio->submit.waiter; + +WRITE_ONCE (dio->submit.waiter,NULL ); +blk_wake_io_task (waiter); + }else if (!inode->i_mapping->nrpages) { +WRITE_ONCE (iocb->private,NULL ); + +/* +* We must never invalidate pages from this thread to +* avoid deadlocks with buffered I/O completions. +* Tough luck if you hit the tiny race with someone +* dirtying the range now. +*/ + dio->flags |=IOMAP_DIO_NO_INVALIDATE ; +iomap_dio_complete_work (&dio->aio.work); + }else { +INIT_WORK (&dio->aio.work, iomap_dio_complete_work); +queue_work (inode->i_sb->s_dio_done_wq, &dio->aio.work);
fs/iomap/direct-io.c static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret) · line 210
Quoted 10 lines
fs/iomap/direct-io.c +10
-0+ } + } + +if (should_dirty) { +bio_check_pages_dirty (&ioend->io_bio); + }else { +bio_release_pages (&ioend->io_bio, false); +bio_put (&ioend->io_bio); + } +
Not that it matters all that much, but I'm a little curious about the reasoning for using vec_count here. AFAICS this correlates to per-folio writeback completions for buffered I/O, but that doesn't seem to apply to direct I/O. Is there a reason to have the caller throttle based on vec_counts, or are we just pulling some non-zero value for consistency sake?
Brian
Quoted 40 lines
fs/iomap/direct-io.c +3
-0+return vec_count; +} +void iomap_dio_bio_end_io (struct bio *bio) {struct iomap_dio *dio = bio->bi_private;fs/iomap/internal.h new file +7
-0@@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _IOMAP_INTERNAL_H +#define _IOMAP_INTERNAL_H 1 + +u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend); + +#endif /* _IOMAP_INTERNAL_H */ include/linux/iomap.h +3
-1@@ -343,9 +343,11 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,#define IOMAP_IOEND_UNWRITTEN (1U << 1)/* don't merge into previous ioend */ #define IOMAP_IOEND_BOUNDARY (1U << 2) +/* is direct I/O */ +#define IOMAP_IOEND_DIRECT (1U << 3)#define IOMAP_IOEND_NOMERGE_FLAGS \ - (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN) + (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT)/* * Structure for writeback I/O completions. Signature · 1 line
2.45.2
Re: [PATCH 5/8] iomap: optionally use ioends for direct I/O
Quoted 38 lines · Brian Foster
Quoted 38 lines
fs/iomap/direct-io.c +38
-0+bool should_dirty = (dio->flags &IOMAP_DIO_DIRTY ); +struct kiocb *iocb = dio->iocb; +u32 vec_count = ioend->io_bio.bi_vcnt; + +if (ioend->io_error) +iomap_dio_set_error (dio, ioend->io_error); + +if (atomic_dec_and_test (&dio->ref)) { +struct inode *inode =file_inode (iocb->ki_filp); + +if (dio->wait_for_completion) { +struct task_struct *waiter = dio->submit.waiter; + +WRITE_ONCE (dio->submit.waiter,NULL ); +blk_wake_io_task (waiter); + }else if (!inode->i_mapping->nrpages) { +WRITE_ONCE (iocb->private,NULL ); + +/* +* We must never invalidate pages from this thread to +* avoid deadlocks with buffered I/O completions. +* Tough luck if you hit the tiny race with someone +* dirtying the range now. +*/ + dio->flags |=IOMAP_DIO_NO_INVALIDATE ; +iomap_dio_complete_work (&dio->aio.work); + }else { +INIT_WORK (&dio->aio.work, iomap_dio_complete_work); +queue_work (inode->i_sb->s_dio_done_wq, &dio->aio.work); + } + } + +if (should_dirty) { +bio_check_pages_dirty (&ioend->io_bio); + }else { +bio_release_pages (&ioend->io_bio, false); +bio_put (&ioend->io_bio); + }
Quoted 8 lines
Quoted 1 line
+
Not that it matters all that much, but I'm a little curious about the reasoning for using
vec_counthere. AFAICS this correlates to per-folio writeback completions for buffered I/O, but that doesn't seem to apply to direct I/O. Is there a reason to have the caller throttle based onvec_counts, or are we just pulling some non-zero value for consistency sake?
So direct I/O also iterates over all folios for the bio, to unpin, and in case of reads dirty all of them.
I wanted to plug something useful into cond_resched condition in the caller. Now number of bvecs isn't the number of folios as we can physically merge outside the folio context, but I think this is about as goot as it gets without changing the block code to return the number of folios processed from __bio_release_pages and bio_check_pages_dirty.
Re: [PATCH 5/8] iomap: optionally use ioends for direct I/O
In reply to Christoph HellwigQuoted 32 lines · Christoph Hellwig
struct
iomap_ioendcurrently tracks outstanding buffered writes and has some really nice code in core iomap and XFS to merge contiguous I/Os an defer them to userspace for completion in a very efficient way.For zoned writes we'll also need a per-bio user context completion to record the written blocks, and the infrastructure for that would look basically like the ioend handling for buffered I/O.
So instead of reinventing the wheel, reuse the existing infrastructure.
Signed-off-by Christoph Hellwig <[email protected]>--- fs/iomap/buffered-io.c | 3 +++ fs/iomap/direct-io.c | 50 +++++++++++++++++++++++++++++++++++++++++- fs/iomap/internal.h | 7 ++++++ include/linux/iomap.h | 4 +++- 4 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 fs/iomap/internal.hfs/iomap/buffered-io.c +1
-0@@ -17,6 +17,7 @@#include <linux/bio.h> #include <linux/sched/signal.h> #include <linux/migrate.h> +#include "internal.h" #include "trace.h" #include "../internal.h"
fs/iomap/buffered-io.c iomap_finish_ioend(struct iomap_ioend *ioend, int error) · line 1589
Quoted 7 lines
fs/iomap/buffered-io.c +2
-0@@ -1582,6 +1583,8 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error)if (!atomic_dec_and_test (&ioend->io_remaining))return 0 ; +if (ioend->io_flags &IOMAP_IOEND_DIRECT ) +return iomap_finish_ioend_direct (ioend);return iomap_finish_ioend_buffered (ioend); }
I'm a little surprised that more of the iomap_ioend* functions didn't end up in ioend.c.
Quoted 50 lines
fs/iomap/direct-io.c +20
-1@@ -12,6 +12,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "internal.h" #include "trace.h" #include "../internal.h" @@ -20,6 +21,7 @@* Private flags for iomap_dio, must not overlap with the public ones in * iomap.h: */ +#define IOMAP_DIO_NO_INVALIDATE (1U << 25)#define IOMAP_DIO_CALLER_COMP (1U << 26)#define IOMAP_DIO_INLINE_COMP (1U << 27)#define IOMAP_DIO_WRITE_THROUGH (1U << 28)@@ -117,7 +119,8 @@ ssize_t iomap_dio_complete(struct iomap_dio *dio)* ->end_io() when necessary, otherwise a racing buffer read would cache * zeros from unwritten extents. */ -if (!dio->error && dio->size && (dio->flags &IOMAP_DIO_WRITE )) +if (!dio->error && dio->size && (dio->flags &IOMAP_DIO_WRITE ) && + !(dio->flags &IOMAP_DIO_NO_INVALIDATE ))kiocb_invalidate_post_direct_write (iocb, dio->size);inode_dio_end (file_inode (iocb->ki_filp));@@ -163,6 +166,51 @@ static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret)cmpxchg (&dio->error,0 , ret); } +u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend) +{ +struct iomap_dio *dio = ioend->io_bio.bi_private; +bool should_dirty = (dio->flags &IOMAP_DIO_DIRTY ); +struct kiocb *iocb = dio->iocb; +u32 vec_count = ioend->io_bio.bi_vcnt; + +if (ioend->io_error) +iomap_dio_set_error (dio, ioend->io_error); + +if (atomic_dec_and_test (&dio->ref)) { +struct inode *inode =file_inode (iocb->ki_filp); + +if (dio->wait_for_completion) { +struct task_struct *waiter = dio->submit.waiter; +
fs/iomap/direct-io.c static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret) · line 194
Quoted 10 lines
fs/iomap/direct-io.c +10
-0+WRITE_ONCE (dio->submit.waiter,NULL ); +blk_wake_io_task (waiter); + }else if (!inode->i_mapping->nrpages) { +WRITE_ONCE (iocb->private,NULL ); + +/* +* We must never invalidate pages from this thread to +* avoid deadlocks with buffered I/O completions. +* Tough luck if you hit the tiny race with someone +* dirtying the range now.
What happens, exactly? Does that mean that the dirty pagecache always survives?
--D
Quoted 56 lines
fs/iomap/direct-io.c +19
-0+*/ + dio->flags |=IOMAP_DIO_NO_INVALIDATE ; +iomap_dio_complete_work (&dio->aio.work); + }else { +INIT_WORK (&dio->aio.work, iomap_dio_complete_work); +queue_work (inode->i_sb->s_dio_done_wq, &dio->aio.work); + } + } + +if (should_dirty) { +bio_check_pages_dirty (&ioend->io_bio); + }else { +bio_release_pages (&ioend->io_bio, false); +bio_put (&ioend->io_bio); + } + +return vec_count; +} +void iomap_dio_bio_end_io (struct bio *bio) {struct iomap_dio *dio = bio->bi_private;fs/iomap/internal.h new file +7
-0@@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _IOMAP_INTERNAL_H +#define _IOMAP_INTERNAL_H 1 + +u32 iomap_finish_ioend_direct (struct iomap_ioend *ioend); + +#endif /* _IOMAP_INTERNAL_H */ include/linux/iomap.h +3
-1@@ -343,9 +343,11 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,#define IOMAP_IOEND_UNWRITTEN (1U << 1)/* don't merge into previous ioend */ #define IOMAP_IOEND_BOUNDARY (1U << 2) +/* is direct I/O */ +#define IOMAP_IOEND_DIRECT (1U << 3)#define IOMAP_IOEND_NOMERGE_FLAGS \ - (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN) + (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT)/* * Structure for writeback I/O completions. Signature · 1 line
2.45.2
Re: [PATCH 5/8] iomap: optionally use ioends for direct I/O
Quoted 8 lines · Darrick J. Wong
fs/iomap/buffered-io.c iomap_finish_ioend(struct iomap_ioend *ioend, int error) · line 1589
Quoted 5 lines
fs/iomap/buffered-io.c +2
-0return 0 ; +if (ioend->io_flags &IOMAP_IOEND_DIRECT ) +return iomap_finish_ioend_direct (ioend);return iomap_finish_ioend_buffered (ioend); }I'm a little surprised that more of the iomap_ioend* functions didn't end up in
ioend.c.
See the cover letter. For development I wanted to avoid churn. Once we have general approval for the concept I'd like to move more code.
Quoted 5 lines
Quoted 5 lines
fs/iomap/direct-io.c +5
-0+WRITE_ONCE (dio->submit.waiter,NULL ); +blk_wake_io_task (waiter); + }else if (!inode->i_mapping->nrpages) { +WRITE_ONCE (iocb->private,NULL ); +
Quoted 8 lines
fs/iomap/direct-io.c static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret) · line 194
Quoted 5 lines
fs/iomap/direct-io.c +5
-0+/* +* We must never invalidate pages from this thread to +* avoid deadlocks with buffered I/O completions. +* Tough luck if you hit the tiny race with someone +* dirtying the range now. What happens, exactly? Does that mean that the dirty pagecache always survives?
Yes.
PATCH 6/8 iomap: pass private data to iomap_page_mkwrite
In reply to Christoph HellwigAllow the file system to pass private data which can be used by the iomap_begin and iomap_end methods through the private pointer in the iomap_iter structure.
--- fs/iomap/buffered-io.c | 4 +++- fs/xfs/xfs_file.c | 3 ++- fs/zonefs/file.c | 2 +- include/linux/iomap.h | 5 ++--- 4 files changed, 8 insertions(+), 6 deletions(-)
fs/iomap/buffered-io.c +3 -1
@@ -1495,11 +1495,13 @@ static loff_t iomap_folio_mkwrite_iter(struct iomap_iter *iter,return length; } -vm_fault_t iomap_page_mkwrite (struct vm_fault *vmf,const struct iomap_ops *ops) +vm_fault_t iomap_page_mkwrite (struct vm_fault *vmf,const struct iomap_ops *ops, +void *private) {struct iomap_iter iter = { .inode =file_inode (vmf->vma->vm_file), .flags =IOMAP_WRITE |IOMAP_FAULT , + .private = private, };struct folio *folio =page_folio (vmf->page);ssize_t ret;
fs/xfs/xfs_file.c +2 -1
@@ -1498,7 +1498,8 @@ xfs_write_fault(if (IS_DAX (inode)) ret =xfs_dax_fault_locked (vmf, order, true);else - ret =iomap_page_mkwrite (vmf, &xfs_buffered_write_iomap_ops); + ret =iomap_page_mkwrite (vmf, &xfs_buffered_write_iomap_ops, +NULL );xfs_iunlock (ip, lock_mode);sb_end_pagefault (inode->i_sb);
fs/zonefs/file.c +1 -1
@@ -299,7 +299,7 @@ static vm_fault_t zonefs_filemap_page_mkwrite(struct vm_fault *vmf)/* Serialize against truncates */ filemap_invalidate_lock_shared (inode->i_mapping); - ret =iomap_page_mkwrite (vmf, &zonefs_write_iomap_ops); + ret =iomap_page_mkwrite (vmf, &zonefs_write_iomap_ops,NULL );filemap_invalidate_unlock_shared (inode->i_mapping);sb_end_pagefault (inode->i_sb);
include/linux/iomap.h +2 -3
@@ -316,9 +316,8 @@ int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, const struct iomap_ops *ops);int iomap_truncate_page (struct inode *inode,loff_t pos,bool *did_zero,const struct iomap_ops *ops); -vm_fault_t iomap_page_mkwrite (struct vm_fault *vmf, -const struct iomap_ops *ops); - +vm_fault_t iomap_page_mkwrite (struct vm_fault *vmf,const struct iomap_ops *ops, +void *private);typedef void (*iomap_punch_t )(struct inode *inode,loff_t offset,loff_t length,struct iomap *iomap); void iomap_write_delalloc_release(struct inode *inode, loff_t start_byte,
Signature · 1 line
2.45.2
PATCH 7/8 iomap: pass private data to iomap_zero_range
In reply to Christoph HellwigAllow the file system to pass private data which can be used by the iomap_begin and iomap_end methods through the private pointer in the iomap_iter structure.
--- fs/gfs2/bmap.c | 3 ++- fs/iomap/buffered-io.c | 6 ++++-- fs/xfs/xfs_iomap.c | 2 +- include/linux/iomap.h | 2 +- 4 files changed, 8 insertions(+), 5 deletions(-)
fs/gfs2/bmap.c +2 -1
@@ -1300,7 +1300,8 @@ static int gfs2_block_zero_range(struct inode *inode, loff_t from, unsigned int length) {BUG_ON (current->journal_info); -return iomap_zero_range (inode, from, length,NULL , &gfs2_iomap_ops); +return iomap_zero_range (inode, from, length,NULL , &gfs2_iomap_ops, +NULL ); }#define GFS2_JTRUNC_REVOKES 8192
fs/iomap/buffered-io.c +4 -2
@@ -1397,13 +1397,14 @@ static loff_t iomap_zero_iter(struct iomap_iter *iter, bool *did_zero)int iomap_zero_range (struct inode *inode,loff_t pos,loff_t len,bool *did_zero, -const struct iomap_ops *ops) +const struct iomap_ops *ops,void *private) {struct iomap_iter iter = { .inode = inode, .pos = pos, .len = len, .flags =IOMAP_ZERO , + .private = private, };struct address_space *mapping = inode->i_mapping;unsigned int blocksize =i_blocksize (inode);
@@ -1471,7 +1472,8 @@ iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero,/* Block boundary? Nothing to do */ if (!off)return 0 ; -return iomap_zero_range (inode, pos, blocksize - off, did_zero, ops); +return iomap_zero_range (inode, pos, blocksize - off, did_zero, ops, +NULL ); }EXPORT_SYMBOL_GPL (iomap_truncate_page);
fs/xfs/xfs_iomap.c +1 -1
@@ -1497,7 +1497,7 @@ xfs_zero_range(return dax_zero_range (inode, pos, len, did_zero, &xfs_dax_write_iomap_ops);return iomap_zero_range (inode, pos, len, did_zero, - &xfs_buffered_write_iomap_ops); + &xfs_buffered_write_iomap_ops,NULL ); }int
include/linux/iomap.h +1 -1
@@ -313,7 +313,7 @@ bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio);int iomap_file_unshare (struct inode *inode,loff_t pos,loff_t len,const struct iomap_ops *ops);int iomap_zero_range (struct inode *inode,loff_t pos,loff_t len, -bool *did_zero,const struct iomap_ops *ops); +bool *did_zero,const struct iomap_ops *ops,void *private);int iomap_truncate_page (struct inode *inode,loff_t pos,bool *did_zero,const struct iomap_ops *ops); vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops,
Signature · 1 line
2.45.2
PATCH 8/8 iomap: pass private data to iomap_truncate_page
In reply to Christoph HellwigAllow the file system to pass private data which can be used by the iomap_begin and iomap_end methods through the private pointer in the iomap_iter structure.
--- fs/iomap/buffered-io.c | 4 ++-- fs/xfs/xfs_iomap.c | 2 +- include/linux/iomap.h | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
fs/iomap/buffered-io.c +2 -2
@@ -1464,7 +1464,7 @@ EXPORT_SYMBOL_GPL(iomap_zero_range);int iomap_truncate_page (struct inode *inode,loff_t pos,bool *did_zero, -const struct iomap_ops *ops) +const struct iomap_ops *ops,void *private) {unsigned int blocksize =i_blocksize (inode);unsigned int off = pos & (blocksize -1 );
@@ -1473,7 +1473,7 @@ iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero,if (!off)return 0 ;return iomap_zero_range (inode, pos, blocksize - off, did_zero, ops, -NULL ); + private); }EXPORT_SYMBOL_GPL (iomap_truncate_page);
fs/xfs/xfs_iomap.c +1 -1
@@ -1512,5 +1512,5 @@ xfs_truncate_page(return dax_truncate_page (inode, pos, did_zero, &xfs_dax_write_iomap_ops);return iomap_truncate_page (inode, pos, did_zero, - &xfs_buffered_write_iomap_ops); + &xfs_buffered_write_iomap_ops,NULL ); }
include/linux/iomap.h +1 -1
@@ -315,7 +315,7 @@ int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,int iomap_zero_range (struct inode *inode,loff_t pos,loff_t len,bool *did_zero,const struct iomap_ops *ops,void *private);int iomap_truncate_page (struct inode *inode,loff_t pos,bool *did_zero, -const struct iomap_ops *ops); +const struct iomap_ops *ops,void *private);vm_fault_t iomap_page_mkwrite (struct vm_fault *vmf,const struct iomap_ops *ops,void *private); typedef void (*iomap_punch_t)(struct inode *inode, loff_t offset, loff_t length,
Signature · 1 line
2.45.2
Re: [PATCH 8/8] iomap: pass private data to iomap_truncate_page
Quoted 5 lines · Christoph Hellwig
Allow the file system to pass private data which can be used by the
iomap_beginandiomap_endmethods through the private pointer in theiomap_iterstructure.Signed-off-by Christoph Hellwig <[email protected]>
Patches 6-8 look obviously correct to me, so for those three:
--D
Quoted 56 lines
--- fs/iomap/buffered-io.c | 4 ++-- fs/xfs/xfs_iomap.c | 2 +- include/linux/iomap.h | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)fs/iomap/buffered-io.c +2
-2@@ -1464,7 +1464,7 @@ EXPORT_SYMBOL_GPL(iomap_zero_range);int iomap_truncate_page (struct inode *inode,loff_t pos,bool *did_zero, -const struct iomap_ops *ops) +const struct iomap_ops *ops,void *private) {unsigned int blocksize =i_blocksize (inode);unsigned int off = pos & (blocksize -1 );@@ -1473,7 +1473,7 @@ iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero,if (!off)return 0 ;return iomap_zero_range (inode, pos, blocksize - off, did_zero, ops, -NULL ); + private); }EXPORT_SYMBOL_GPL (iomap_truncate_page);fs/xfs/xfs_iomap.c +1
-1@@ -1512,5 +1512,5 @@ xfs_truncate_page(return dax_truncate_page (inode, pos, did_zero, &xfs_dax_write_iomap_ops);return iomap_truncate_page (inode, pos, did_zero, - &xfs_buffered_write_iomap_ops); + &xfs_buffered_write_iomap_ops,NULL ); }include/linux/iomap.h +1
-1@@ -315,7 +315,7 @@ int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,int iomap_zero_range (struct inode *inode,loff_t pos,loff_t len,bool *did_zero,const struct iomap_ops *ops,void *private);int iomap_truncate_page (struct inode *inode,loff_t pos,bool *did_zero, -const struct iomap_ops *ops); +const struct iomap_ops *ops,void *private);vm_fault_t iomap_page_mkwrite (struct vm_fault *vmf,const struct iomap_ops *ops,void *private); typedef void (*iomap_punch_t)(struct inode *inode, loff_t offset, loff_t length,Signature · 1 line
2.45.2
Brian Foster
Reviewed-byQuoted 25 lines · Christoph Hellwig
Hi all,
this series contains the iomap prep work to support zoned XFS.
The biggest changes are:
- an option to reuse the ioend code for direct writes in addition to the current use for buffered writeback, which allows the file system to track completions on a per-bio basis instead of the current
end_iocallback which operates on the entire I/O.
Note that it might make sense to split the ioend code frombuffered-io.cinto its own file with this. Let me know what you think of that and I can include it in the next version
- change of thewriteback_opsso that thesubmit_biocall can be done by the file system. Note that btrfs will also need this eventually when it starts using iomap
- helpers to split ioend to the zone appendqueue_limitsthat plug into the previous item above.
- a bunch of changes for slightly different merge conditions when using zone append. Note that btrfs wants something similar also for compressed I/O, which might be able to share some code. For now the flags use zone append naming, but we can change that if it gets used elsewhere.
- passing private data to a few more helper
Quoted 3 lines
The XFS changes to use this will be posted to the xfs list only to not spam fsdevel too much.
Orthogonal to the couple or so questions inline, the series LGTM: