lift the xfs writepage code into iomap v8
Subsystems (4)
Hi Darrick,
this series cleans up the xfs writepage code and then lifts it to fs/iomap/ so that it could be use by other file system.
Note: the submit_ioend hook has not been renamed. Feel free to rename it if you really dislike the name.
Changes since v7:
- fix various commit log typos
- fix up various comments
- move some of the iomap tracepoint additions to an earlier patch
- rebased on top of "iomap: iomap that extends beyond EOF should be marked dirty"
Changes since v6:
- actually add trace.c to the patch
- move back to the old order that massages XFS into shape and then lifts the code to iomap
- cleanup iomap_ioend_compare
- cleanup the add_to_ioend checks
Changes since v5:
- move the tracing code to fs/iomap/trace.[ch]
- fix a bisection issue with the tracing code
- add an assert that xfs_end_io now only gets "complicated" completions
- better document the iomap_writeback_ops methods in iomap.h
Changes since v4:
- rebased on top 5.4-rc1
- drop the addition of list_pop / list_pop_entry
- re-split a few patches to better fit Darricks scheme of keeping the iomap additions separate from the XFS switchover
Changes since v3:
- re-split the pages to add new code to iomap and then switch xfs to it later (Darrick)
Changes since v2:
- rebased to v5.3-rc1
- folded in a few changes from the gfs2 enablement series
Changes since v1:
- rebased to the latest xfs for-next tree
- keep the preallocated transactions for size updates
- rename list_pop to list_pop_entry and related cleanups
- better document the nofs context handling
- document that the iomap tracepoints are not a stable API
PATCH 1/14 iomap: iomap that extends beyond EOF should be marked dirty
From: Dave Chinner <[email protected]>
When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.
However, we don't mark such iomaps as IOMAP_F_DIRTY to indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion when O_DSYNC writes are being used and the hardware supports FUA.
Hence when setting IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will call generic_write_sync() to flush the inode size update to stable storage correctly.
--- fs/ext4/inode.c | 9 ++++++++- fs/xfs/xfs_iomap.c | 7 +++++++ include/linux/iomap.h | 2 ++ 3 files changed, 17 insertions(+), 1 deletion(-)
fs/ext4/inode.c +8 -1
@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,return ret; } +/* +* Writes that span EOF might trigger an IO size update on completion, +* so consider them to be dirty for the purposes of O_DSYNC even if +* there is no other metadata changes being made or are pending here. +*/ iomap->flags =0 ; -if (ext4_inode_datasync_dirty (inode)) +if (ext4_inode_datasync_dirty (inode) || + offset + length >i_size_read (inode)) iomap->flags |=IOMAP_F_DIRTY ; + iomap->bdev = inode->i_sb->s_bdev; iomap->dax_dev = sbi->s_daxdev; iomap->offset = (u64 )first_block << blkbits;
fs/xfs/xfs_iomap.c +7 -0
@@ -1049,6 +1049,13 @@ xfs_file_iomap_begin(trace_xfs_iomap_alloc (ip, offset, length,XFS_DATA_FORK , &imap); out_finish: +/* +* Writes that span EOF might trigger an IO size update on completion, +* so consider them to be dirty for the purposes of O_DSYNC even if +* there is no other metadata changes pending or have been made here. +*/ +if ((flags &IOMAP_WRITE ) && offset + length >i_size_read (inode)) + iomap->flags |=IOMAP_F_DIRTY ;return xfs_bmbt_to_iomap (ip, iomap, &imap, shared); out_found:
include/linux/iomap.h +2 -0
@@ -32,6 +32,8 @@ struct vm_fault;* * IOMAP_F_DIRTY indicates the inode has uncommitted metadata needed to access * written data and requires fdatasync to commit them to persistent storage. +* This needs to take into account metadata changes that *may* be made at IO +* completion, such as file size updates from direct IO. */ #define IOMAP_F_NEW 0x01/* blocks have been newly allocated */ #define IOMAP_F_DIRTY 0x02/* uncommitted metadata */
Signature · 1 line
2.20.1
Re: [PATCH 01/14] iomap: iomap that extends beyond EOF should be marked dirty
Quoted 11 lines · Christoph Hellwig
From: Dave Chinner <[email protected]>
When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.
However, we don't mark such iomaps as
IOMAP_F_DIRTYto indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion whenO_DSYNCwrites are being used and the hardware supports FUA.
Quoted 7 lines
Hence when setting
IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will callgeneric_write_sync()to flush the inode size update to stable storage correctly.Signed-off-by Dave Chinner <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>
Looks ok, but need fixes tag. Also, might it be wise to split off the ext4 section into a separate patch so that it can be backported separately?
--D
Quoted 62 lines
--- fs/ext4/inode.c | 9 ++++++++- fs/xfs/xfs_iomap.c | 7 +++++++ include/linux/iomap.h | 2 ++ 3 files changed, 17 insertions(+), 1 deletion(-)fs/ext4/inode.c +8
-1@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,return ret; } +/* +* Writes that span EOF might trigger an IO size update on completion, +* so consider them to be dirty for the purposes of O_DSYNC even if +* there is no other metadata changes being made or are pending here. +*/ iomap->flags =0 ; -if (ext4_inode_datasync_dirty (inode)) +if (ext4_inode_datasync_dirty (inode) || + offset + length >i_size_read (inode)) iomap->flags |=IOMAP_F_DIRTY ; + iomap->bdev = inode->i_sb->s_bdev; iomap->dax_dev = sbi->s_daxdev; iomap->offset = (u64 )first_block << blkbits;fs/xfs/xfs_iomap.c +7
-0@@ -1049,6 +1049,13 @@ xfs_file_iomap_begin(trace_xfs_iomap_alloc (ip, offset, length,XFS_DATA_FORK , &imap); out_finish: +/* +* Writes that span EOF might trigger an IO size update on completion, +* so consider them to be dirty for the purposes of O_DSYNC even if +* there is no other metadata changes pending or have been made here. +*/ +if ((flags &IOMAP_WRITE ) && offset + length >i_size_read (inode)) + iomap->flags |=IOMAP_F_DIRTY ;return xfs_bmbt_to_iomap (ip, iomap, &imap, shared); out_found:include/linux/iomap.h +2
-0@@ -32,6 +32,8 @@ struct vm_fault;* * IOMAP_F_DIRTY indicates the inode has uncommitted metadata needed to access * written data and requires fdatasync to commit them to persistent storage. +* This needs to take into account metadata changes that *may* be made at IO +* completion, such as file size updates from direct IO. */ #define IOMAP_F_NEW 0x01/* blocks have been newly allocated */ #define IOMAP_F_DIRTY 0x02/* uncommitted metadata */ Signature · 1 line
2.20.1
Re: [PATCH 01/14] iomap: iomap that extends beyond EOF should be marked dirty
Quoted 17 lines · Darrick J. Wong
Quoted 16 lines · Christoph Hellwig
From: Dave Chinner <[email protected]>
When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.
However, we don't mark such iomaps as
IOMAP_F_DIRTYto indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion whenO_DSYNCwrites are being used and the hardware supports FUA.Hence when setting
IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will callgeneric_write_sync()to flush the inode size update to stable storage correctly.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Dave Chinner <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>Looks ok, but need fixes tag. Also, might it be wise to split off the ext4 section into a separate patch so that it can be backported separately?
I 've done a bit more digging on this, and the ext4 part is not needed for DAX as IOMAP_F_DIRTY is only used in the page fault path and hence can't change the file size. As such, this only affects direct IO. Hence the ext4 hunk can be added to the ext4 iomap-dio patchset that is being developed rather than being in this patch.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH 01/14] iomap: iomap that extends beyond EOF should be marked dirty
Quoted 25 lines · Dave Chinner
Quoted 17 lines · Darrick J. Wong
Quoted 16 lines · Christoph Hellwig
From: Dave Chinner <[email protected]>
When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.
However, we don't mark such iomaps as
IOMAP_F_DIRTYto indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion whenO_DSYNCwrites are being used and the hardware supports FUA.Hence when setting
IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will callgeneric_write_sync()to flush the inode size update to stable storage correctly.Quoted 6 lines
Quoted 2 lines
Signed-off-by Dave Chinner <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>Looks ok, but need fixes tag. Also, might it be wise to split off the ext4 section into a separate patch so that it can be backported separately?
Quoted 5 lines
I 've done a bit more digging on this, and the ext4 part is not needed for DAX as
IOMAP_F_DIRTYis only used in the page fault path and hence can't change the file size. As such, this only affects direct IO. Hence the ext4 hunk can be added to the ext4 iomap-dio patchset that is being developed rather than being in this patch.
Noted, thanks Dave. I've incorporated the ext4 related change into my patch series.
--<M>--
Re: [PATCH 01/14] iomap: iomap that extends beyond EOF should be marked dirty
Quoted 26 lines · Matthew Bobrowski
Quoted 25 lines · Dave Chinner
Quoted 23 lines · Darrick J. Wong
Quoted 11 lines · Christoph Hellwig
From: Dave Chinner <[email protected]>
When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.
However, we don't mark such iomaps as
IOMAP_F_DIRTYto indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion whenO_DSYNCwrites are being used and the hardware supports FUA.Quoted 7 lines
Hence when setting
IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will callgeneric_write_sync()to flush the inode size update to stable storage correctly.Signed-off-by Dave Chinner <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>Looks ok, but need fixes tag. Also, might it be wise to split off the ext4 section into a separate patch so that it can be backported separately?
Quoted 8 lines
Quoted 5 lines
I 've done a bit more digging on this, and the ext4 part is not needed for DAX as
IOMAP_F_DIRTYis only used in the page fault path and hence can't change the file size. As such, this only affects direct IO. Hence the ext4 hunk can be added to the ext4 iomap-dio patchset that is being developed rather than being in this patch.Noted, thanks Dave. I've incorporated the ext4 related change into my patch series.
Ok, I've dropped the ext4 hunk from my branch.
--D
Quoted 1 line
--<M>--
Re: [PATCH 01/14] iomap: iomap that extends beyond EOF should be marked dirty
In reply to Darrick J. WongQuoted 3 lines · Darrick J. Wong
Looks ok, but need fixes tag. Also, might it be wise to split off the ext4 section into a separate patch so that it can be backported separately?
I'll let Dave handle all that. I've just pulled it in here as multiple patches conflict with this one.
PATCH 2/14 xfs: initialize iomap->flags in xfs_bmbt_to_iomap
In reply to Christoph HellwigCurrently we don't overwrite the flags field in the iomap in xfs_bmbt_to_iomap. This works fine with 0-initialized iomaps on stack, but is harmful once we want to be able to reuse an iomap in the writeback code. Replace the shared parameter with a set of initial flags an thus ensures the flags field is always reinitialized.
--- fs/xfs/xfs_iomap.c | 30 ++++++++++++++++++------------ fs/xfs/xfs_iomap.h | 2 +- fs/xfs/xfs_pnfs.c | 2 +- 3 files changed, 20 insertions(+), 14 deletions(-)
fs/xfs/xfs_iomap.c +18 -12
@@ -54,7 +54,7 @@ xfs_bmbt_to_iomap(struct xfs_inode *ip, struct iomap *iomap, struct xfs_bmbt_irec *imap, - bool shared) + u16 flags) { struct xfs_mount *mp = ip->i_mount;
@@ -79,12 +79,11 @@ xfs_bmbt_to_iomap( iomap->length =XFS_FSB_TO_B (mp, imap->br_blockcount); iomap->bdev =xfs_find_bdev_for_inode (VFS_I (ip)); iomap->dax_dev =xfs_find_daxdev_for_inode (VFS_I (ip)); + iomap->flags = flags;if (xfs_ipincount (ip) && (ip->i_itemp->ili_fsync_fields & ~XFS_ILOG_TIMESTAMP )) iomap->flags |=IOMAP_F_DIRTY ; -if (shared) - iomap->flags |=IOMAP_F_SHARED ;return 0 ; }
@@ -540,6 +539,7 @@ xfs_file_iomap_begin_delay(struct xfs_iext_cursor icur, ccur;xfs_fsblock_t prealloc_blocks =0 ;bool eof = false, cow_eof = false, shared = false; +u16 iomap_flags =0 ;int whichfork =XFS_DATA_FORK ;int error =0 ;
@@ -707,7 +707,7 @@ xfs_file_iomap_begin_delay(* Flag newly allocated delalloc blocks with IOMAP_F_NEW so we punch * them out if the write happens to fail. */ - iomap->flags |=IOMAP_F_NEW ; + iomap_flags |=IOMAP_F_NEW ;trace_xfs_iomap_alloc (ip, offset, count, whichfork, whichfork ==XFS_DATA_FORK ? &imap : &cmap); done:
@@ -715,14 +715,17 @@ xfs_file_iomap_begin_delay(if (imap.br_startoff > offset_fsb) {xfs_trim_extent (&cmap, offset_fsb, imap.br_startoff - offset_fsb); - error =xfs_bmbt_to_iomap (ip, iomap, &cmap, true); + error =xfs_bmbt_to_iomap (ip, iomap, &cmap, +IOMAP_F_SHARED ); goto out_unlock; }/* ensure we only report blocks we have a reservation for */ xfs_trim_extent (&imap, cmap.br_startoff, cmap.br_blockcount); shared = true; } - error =xfs_bmbt_to_iomap (ip, iomap, &imap, shared); +if (shared) + iomap_flags |=IOMAP_F_SHARED ; + error =xfs_bmbt_to_iomap (ip, iomap, &imap, iomap_flags); out_unlock:xfs_iunlock (ip,XFS_ILOCK_EXCL );return error;
@@ -930,6 +933,7 @@ xfs_file_iomap_begin(xfs_fileoff_t offset_fsb, end_fsb;int nimaps =1 , error =0 ;bool shared = false; +u16 iomap_flags =0 ;unsigned lockmode;if (XFS_FORCED_SHUTDOWN (mp))
@@ -1045,7 +1049,7 @@ xfs_file_iomap_begin(if (error)return error; - iomap->flags |=IOMAP_F_NEW ; + iomap_flags |=IOMAP_F_NEW ;trace_xfs_iomap_alloc (ip, offset, length,XFS_DATA_FORK , &imap); out_finish:
@@ -1055,8 +1059,10 @@ xfs_file_iomap_begin(* there is no other metadata changes pending or have been made here. */ if ((flags &IOMAP_WRITE ) && offset + length >i_size_read (inode)) - iomap->flags |=IOMAP_F_DIRTY ; -return xfs_bmbt_to_iomap (ip, iomap, &imap, shared); + iomap_flags |=IOMAP_F_DIRTY ; +if (shared) + iomap_flags |=IOMAP_F_SHARED ; +return xfs_bmbt_to_iomap (ip, iomap, &imap, iomap_flags); out_found:ASSERT (nimaps);
@@ -1200,7 +1206,7 @@ xfs_seek_iomap_begin(if (data_fsb < cow_fsb + cmap.br_blockcount) end_fsb =min (end_fsb, data_fsb);xfs_trim_extent (&cmap, offset_fsb, end_fsb); - error =xfs_bmbt_to_iomap (ip, iomap, &cmap, true); + error =xfs_bmbt_to_iomap (ip, iomap, &cmap,IOMAP_F_SHARED );/* * This is a COW extent, so we must probe the page cache * because there could be dirty page cache being backed
@@ -1222,7 +1228,7 @@ xfs_seek_iomap_begin( imap.br_state =XFS_EXT_NORM ; done:xfs_trim_extent (&imap, offset_fsb, end_fsb); - error =xfs_bmbt_to_iomap (ip, iomap, &imap, false); + error =xfs_bmbt_to_iomap (ip, iomap, &imap,0 ); out_unlock:xfs_iunlock (ip, lockmode);return error;
@@ -1268,7 +1274,7 @@ xfs_xattr_iomap_begin(if (error)return error;ASSERT (nimaps); -return xfs_bmbt_to_iomap (ip, iomap, &imap, false); +return xfs_bmbt_to_iomap (ip, iomap, &imap,0 ); }const struct iomap_ops xfs_xattr_iomap_ops = {
fs/xfs/xfs_iomap.h +1 -1
@@ -16,7 +16,7 @@ int xfs_iomap_write_direct(struct xfs_inode *, xfs_off_t, size_t,int xfs_iomap_write_unwritten (struct xfs_inode *,xfs_off_t ,xfs_off_t ,bool );int xfs_bmbt_to_iomap (struct xfs_inode *,struct iomap *, -struct xfs_bmbt_irec *,bool shared); +struct xfs_bmbt_irec *,u16 );xfs_extlen_t xfs_eof_alignment (struct xfs_inode *ip,xfs_extlen_t extsize); static inline xfs_filblks_t
fs/xfs/xfs_pnfs.c +1 -1
@@ -178,7 +178,7 @@ xfs_fs_map_blocks( }xfs_iunlock (ip,XFS_IOLOCK_EXCL ); - error =xfs_bmbt_to_iomap (ip, iomap, &imap, false); + error =xfs_bmbt_to_iomap (ip, iomap, &imap,0 ); *device_generation = mp->m_generation;return error; out_unlock:
Signature · 1 line
2.20.1
PATCH 3/14 xfs: set IOMAP_F_NEW more carefully
In reply to Christoph HellwigDon't set IOMAP_F_NEW if we COW over an existing allocated range, as these aren't strictly new allocations. This is required to be able to use IOMAP_F_NEW to zero newly allocated blocks, which is required for the iomap code to fully support file systems that don't do delayed allocations or use unwritten extents.
--- fs/xfs/xfs_iomap.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/xfs_iomap.c +6 -3
@@ -707,9 +707,12 @@ xfs_file_iomap_begin_delay(* Flag newly allocated delalloc blocks with IOMAP_F_NEW so we punch * them out if the write happens to fail. */ - iomap_flags |=IOMAP_F_NEW ; -trace_xfs_iomap_alloc (ip, offset, count, whichfork, - whichfork ==XFS_DATA_FORK ? &imap : &cmap); +if (whichfork ==XFS_DATA_FORK ) { + iomap_flags |=IOMAP_F_NEW ; +trace_xfs_iomap_alloc (ip, offset, count, whichfork, &imap); + }else { +trace_xfs_iomap_alloc (ip, offset, count, whichfork, &cmap); + } done:if (whichfork ==XFS_COW_FORK ) {if (imap.br_startoff > offset_fsb) {
Signature · 1 line
2.20.1
PATCH 4/14 xfs: use a struct iomap in xfs_writepage_ctx
In reply to Christoph HellwigIn preparation for moving the XFS writeback code to fs/iomap.c, switch it to use struct iomap instead of the XFS-specific struct xfs_bmbt_irec.
--- fs/xfs/libxfs/xfs_bmap.c | 14 +++++-- fs/xfs/libxfs/xfs_bmap.h | 3 +- fs/xfs/xfs_aops.c | 82 +++++++++++++++++++--------------------- fs/xfs/xfs_aops.h | 2 +- 4 files changed, 50 insertions(+), 51 deletions(-)
fs/xfs/libxfs/xfs_bmap.c +10 -4
@@ -34,6 +34,7 @@#include "xfs_ag_resv.h" #include "xfs_refcount.h" #include "xfs_icache.h" +#include "xfs_iomap.h" kmem_zone_t *xfs_bmap_free_item_zone;
@@ -4456,16 +4457,21 @@ intxfs_bmapi_convert_delalloc ( struct xfs_inode *ip, int whichfork, - xfs_fileoff_t offset_fsb, - struct xfs_bmbt_irec *imap, + xfs_off_t offset, + struct iomap *iomap, unsigned int *seq) { struct xfs_ifork *ifp = XFS_IFORK_PTR(ip, whichfork);struct xfs_mount *mp = ip->i_mount; +xfs_fileoff_t offset_fsb =XFS_B_TO_FSBT (mp, offset);struct xfs_bmalloca bma = {NULL }; +u16 flags =0 ;struct xfs_trans *tp;int error; +if (whichfork ==XFS_COW_FORK ) + flags |=IOMAP_F_SHARED ; +/* * Space for the extent and indirect blocks was reserved when the * delalloc extent was created so there's no need to do so here.
@@ -4495,7 +4501,7 @@ xfs_bmapi_convert_delalloc(* the extent. Just return the real extent at this offset. */ if (!isnullstartblock (bma.got.br_startblock)) { - *imap = bma.got; +xfs_bmbt_to_iomap (ip, iomap, &bma.got, flags); *seq =READ_ONCE (ifp->if_seq); goto out_trans_cancel; }
@@ -4528,7 +4534,7 @@ xfs_bmapi_convert_delalloc(XFS_STATS_INC (mp, xs_xstrat_quick);ASSERT (!isnullstartblock (bma.got.br_startblock)); - *imap = bma.got; +xfs_bmbt_to_iomap (ip, iomap, &bma.got, flags); *seq =READ_ONCE (ifp->if_seq);if (whichfork ==XFS_COW_FORK )
fs/xfs/libxfs/xfs_bmap.h +1 -2
@@ -228,8 +228,7 @@ int xfs_bmapi_reserve_delalloc(struct xfs_inode *ip, int whichfork, struct xfs_bmbt_irec *got, struct xfs_iext_cursor *cur, int eof);int xfs_bmapi_convert_delalloc (struct xfs_inode *ip,int whichfork, -xfs_fileoff_t offset_fsb,struct xfs_bmbt_irec *imap, -unsigned int *seq); +xfs_off_t offset,struct iomap *iomap,unsigned int *seq); int xfs_bmap_add_extent_unwritten_real(struct xfs_trans *tp, struct xfs_inode *ip, int whichfork, struct xfs_iext_cursor *icur, struct xfs_btree_cur **curp,
fs/xfs/xfs_aops.c +38 -44
209 lines not on this page · raw message
fs/xfs/xfs_aops.h +1 -1
@@ -14,7 +14,7 @@ extern struct bio_set xfs_ioend_bioset;struct xfs_ioend {struct list_head io_list;/* next ioend in chain */ int io_fork;/* inode fork written back */ -xfs_exntst_t io_state;/* extent state */ +u16 io_type;struct inode *io_inode;/* file being written to */ size_t io_size;/* size of the extent */ xfs_off_t io_offset;/* offset in the file */
Signature · 1 line
2.20.1
PATCH 5/14 xfs: refactor the ioend merging code
In reply to Christoph HellwigIntroduce two nicely abstracted helper, which can be moved to the iomap code later. Also use list_first_entry_or_null to simplify the code a bit.
--- fs/xfs/xfs_aops.c | 73 +++++++++++++++++++++++++++-------------------- 1 file changed, 42 insertions(+), 31 deletions(-)
fs/xfs/xfs_aops.c +42 -31
@@ -116,6 +116,22 @@ xfs_destroy_ioend( } } +static void +xfs_destroy_ioends ( +struct xfs_ioend *ioend, +int error) +{ +struct list_head tmp; + +list_replace_init (&ioend->io_list, &tmp); +xfs_destroy_ioend (ioend, error); + while ((ioend = list_first_entry_or_null(&tmp, struct xfs_ioend, + io_list))) { +list_del_init (&ioend->io_list); +xfs_destroy_ioend (ioend, error); + } +} +/* * Fast and loose check if this write could update the on-disk inode size. */
@@ -230,7 +246,6 @@ STATIC voidxfs_end_ioend ( struct xfs_ioend *ioend) { -struct list_head ioend_list;struct xfs_inode *ip =XFS_I (ioend->io_inode);xfs_off_t offset = ioend->io_offset;size_t size = ioend->io_size;
@@ -275,16 +290,7 @@ xfs_end_ioend( done:if (ioend->io_append_trans) error =xfs_setfilesize_ioend (ioend, error); -list_replace_init (&ioend->io_list, &ioend_list); -xfs_destroy_ioend (ioend, error); - -while (!list_empty (&ioend_list)) { - ioend = list_first_entry(&ioend_list, struct xfs_ioend, - io_list); -list_del_init (&ioend->io_list); -xfs_destroy_ioend (ioend, error); - } - +xfs_destroy_ioends (ioend, error);memalloc_nofs_restore (nofs_flag); }
@@ -333,17 +339,18 @@ xfs_ioend_try_merge(struct xfs_ioend *ioend, struct list_head *more_ioends) { - struct xfs_ioend *next_ioend; + struct xfs_ioend *next; + +INIT_LIST_HEAD (&ioend->io_list); -while (!list_empty (more_ioends)) { - next_ioend = list_first_entry(more_ioends, struct xfs_ioend, - io_list); -if (!xfs_ioend_can_merge (ioend, next_ioend)) + while ((next = list_first_entry_or_null(more_ioends, struct xfs_ioend, + io_list))) { +if (!xfs_ioend_can_merge (ioend, next))break ; -list_move_tail (&next_ioend->io_list, &ioend->io_list); - ioend->io_size += next_ioend->io_size; -if (next_ioend->io_append_trans) -xfs_ioend_merge_append_transactions (ioend, next_ioend); +list_move_tail (&next->io_list, &ioend->io_list); + ioend->io_size += next->io_size; +if (next->io_append_trans) +xfs_ioend_merge_append_transactions (ioend, next); } }
@@ -366,29 +373,33 @@ xfs_ioend_compare(return 0 ; } +static void +xfs_sort_ioends ( +struct list_head *ioend_list) +{ +list_sort (NULL , ioend_list, xfs_ioend_compare); +} +/* Finish all pending io completions. */ void xfs_end_io (struct work_struct *work) { -struct xfs_inode *ip; +struct xfs_inode *ip = + container_of(work, struct xfs_inode, i_ioend_work);struct xfs_ioend *ioend; -struct list_head completion_list; +struct list_head tmp;unsigned long flags; - ip = container_of(work, struct xfs_inode, i_ioend_work); -spin_lock_irqsave (&ip->i_ioend_lock, flags); -list_replace_init (&ip->i_ioend_list, &completion_list); +list_replace_init (&ip->i_ioend_list, &tmp);spin_unlock_irqrestore (&ip->i_ioend_lock, flags); -list_sort (NULL , &completion_list, xfs_ioend_compare); - -while (!list_empty (&completion_list)) { - ioend = list_first_entry(&completion_list, struct xfs_ioend, - io_list); +xfs_sort_ioends (&tmp); + while ((ioend = list_first_entry_or_null(&tmp, struct xfs_ioend, + io_list))) {list_del_init (&ioend->io_list); -xfs_ioend_try_merge (ioend, &completion_list); +xfs_ioend_try_merge (ioend, &tmp);xfs_end_ioend (ioend); } }
Signature · 1 line
2.20.1
Re: [PATCH 05/14] xfs: refactor the ioend merging code
Quoted 6 lines · Christoph Hellwig
Introduce two nicely abstracted helper, which can be moved to the iomap code later. Also use
list_first_entry_or_nullto simplify the code a bit.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Dave Chinner <[email protected]>
Looks ok,
--D
Quoted 133 lines
--- fs/xfs/xfs_aops.c | 73 +++++++++++++++++++++++++++-------------------- 1 file changed, 42 insertions(+), 31 deletions(-)fs/xfs/xfs_aops.c +42
-31@@ -116,6 +116,22 @@ xfs_destroy_ioend( } } +static void +xfs_destroy_ioends ( +struct xfs_ioend *ioend, +int error) +{ +struct list_head tmp; + +list_replace_init (&ioend->io_list, &tmp); +xfs_destroy_ioend (ioend, error); + while ((ioend = list_first_entry_or_null(&tmp, struct xfs_ioend, + io_list))) { +list_del_init (&ioend->io_list); +xfs_destroy_ioend (ioend, error); + } +} +/* * Fast and loose check if this write could update the on-disk inode size. */ @@ -230,7 +246,6 @@ STATIC voidxfs_end_ioend ( struct xfs_ioend *ioend) { -struct list_head ioend_list;struct xfs_inode *ip =XFS_I (ioend->io_inode);xfs_off_t offset = ioend->io_offset;size_t size = ioend->io_size;@@ -275,16 +290,7 @@ xfs_end_ioend( done:if (ioend->io_append_trans) error =xfs_setfilesize_ioend (ioend, error); -list_replace_init (&ioend->io_list, &ioend_list); -xfs_destroy_ioend (ioend, error); - -while (!list_empty (&ioend_list)) { - ioend = list_first_entry(&ioend_list, struct xfs_ioend, - io_list); -list_del_init (&ioend->io_list); -xfs_destroy_ioend (ioend, error); - } - +xfs_destroy_ioends (ioend, error);memalloc_nofs_restore (nofs_flag); }@@ -333,17 +339,18 @@ xfs_ioend_try_merge(struct xfs_ioend *ioend, struct list_head *more_ioends) { - struct xfs_ioend *next_ioend; + struct xfs_ioend *next; + +INIT_LIST_HEAD (&ioend->io_list); -while (!list_empty (more_ioends)) { - next_ioend = list_first_entry(more_ioends, struct xfs_ioend, - io_list); -if (!xfs_ioend_can_merge (ioend, next_ioend)) + while ((next = list_first_entry_or_null(more_ioends, struct xfs_ioend, + io_list))) { +if (!xfs_ioend_can_merge (ioend, next))break ; -list_move_tail (&next_ioend->io_list, &ioend->io_list); - ioend->io_size += next_ioend->io_size; -if (next_ioend->io_append_trans) -xfs_ioend_merge_append_transactions (ioend, next_ioend); +list_move_tail (&next->io_list, &ioend->io_list); + ioend->io_size += next->io_size; +if (next->io_append_trans) +xfs_ioend_merge_append_transactions (ioend, next); } }@@ -366,29 +373,33 @@ xfs_ioend_compare(return 0 ; } +static void +xfs_sort_ioends ( +struct list_head *ioend_list) +{ +list_sort (NULL , ioend_list, xfs_ioend_compare); +} +/* Finish all pending io completions. */ void xfs_end_io (struct work_struct *work) { -struct xfs_inode *ip; +struct xfs_inode *ip = + container_of(work, struct xfs_inode, i_ioend_work);struct xfs_ioend *ioend; -struct list_head completion_list; +struct list_head tmp;unsigned long flags; - ip = container_of(work, struct xfs_inode, i_ioend_work); -spin_lock_irqsave (&ip->i_ioend_lock, flags); -list_replace_init (&ip->i_ioend_list, &completion_list); +list_replace_init (&ip->i_ioend_list, &tmp);spin_unlock_irqrestore (&ip->i_ioend_lock, flags); -list_sort (NULL , &completion_list, xfs_ioend_compare); - -while (!list_empty (&completion_list)) { - ioend = list_first_entry(&completion_list, struct xfs_ioend, - io_list); +xfs_sort_ioends (&tmp); + while ((ioend = list_first_entry_or_null(&tmp, struct xfs_ioend, + io_list))) {list_del_init (&ioend->io_list); -xfs_ioend_try_merge (ioend, &completion_list); +xfs_ioend_try_merge (ioend, &tmp);xfs_end_ioend (ioend); } }Signature · 1 line
2.20.1
PATCH 6/14 xfs: turn io_append_trans into an io_private void pointer
In reply to Christoph HellwigIn preparation for moving the ioend structure to common code we need to get rid of the xfs-specific xfs_trans type. Just make it a file system private void pointer instead.
--- fs/xfs/xfs_aops.c | 26 +++++++++++++------------- fs/xfs/xfs_aops.h | 2 +- 2 files changed, 14 insertions(+), 14 deletions(-)
fs/xfs/xfs_aops.c +13 -13
@@ -153,7 +153,7 @@ xfs_setfilesize_trans_alloc(if (error)return error; - ioend->io_append_trans = tp; + ioend->io_private = tp;/* * We may pass freeze protection with a transaction. So tell lockdep
@@ -220,7 +220,7 @@ xfs_setfilesize_ioend( int error) {struct xfs_inode *ip =XFS_I (ioend->io_inode); -struct xfs_trans *tp = ioend->io_append_trans; +struct xfs_trans *tp = ioend->io_private;/* * The transaction may have been allocated in the I/O submission thread,
@@ -285,10 +285,10 @@ xfs_end_ioend(else if (ioend->io_type ==IOMAP_UNWRITTEN ) error =xfs_iomap_write_unwritten (ip, offset, size, false);else -ASSERT (!xfs_ioend_is_append (ioend) || ioend->io_append_trans); +ASSERT (!xfs_ioend_is_append (ioend) || ioend->io_private); done: -if (ioend->io_append_trans) +if (ioend->io_private) error =xfs_setfilesize_ioend (ioend, error);xfs_destroy_ioends (ioend, error);memalloc_nofs_restore (nofs_flag);
@@ -321,13 +321,13 @@ xfs_ioend_can_merge(* as it is guaranteed to be clean. */ static void -xfs_ioend_merge_append_transactions ( +xfs_ioend_merge_private (struct xfs_ioend *ioend,struct xfs_ioend *next) { -if (!ioend->io_append_trans) { - ioend->io_append_trans = next->io_append_trans; - next->io_append_trans =NULL ; +if (!ioend->io_private) { + ioend->io_private = next->io_private; + next->io_private =NULL ; }else {xfs_setfilesize_ioend (next, -ECANCELED ); }
@@ -349,8 +349,8 @@ xfs_ioend_try_merge(break ;list_move_tail (&next->io_list, &ioend->io_list); ioend->io_size += next->io_size; -if (next->io_append_trans) -xfs_ioend_merge_append_transactions (ioend, next); +if (next->io_private) +xfs_ioend_merge_private (ioend, next); } }
@@ -415,7 +415,7 @@ xfs_end_bio( if (ioend->io_fork == XFS_COW_FORK || ioend->io_type == IOMAP_UNWRITTEN || - ioend->io_append_trans != NULL) { + ioend->io_private) { spin_lock_irqsave(&ip->i_ioend_lock, flags); if (list_empty(&ip->i_ioend_list)) WARN_ON_ONCE(!queue_work(mp->m_unwritten_workqueue,
@@ -680,7 +680,7 @@ xfs_submit_ioend( (ioend->io_fork == XFS_COW_FORK || ioend->io_type != IOMAP_UNWRITTEN) && xfs_ioend_is_append(ioend) && - !ioend->io_append_trans) + !ioend->io_private) status =xfs_setfilesize_trans_alloc (ioend);memalloc_nofs_restore (nofs_flag);
@@ -729,7 +729,7 @@ xfs_alloc_ioend( ioend->io_inode = inode; ioend->io_size =0 ; ioend->io_offset = offset; - ioend->io_append_trans =NULL ; + ioend->io_private =NULL ; ioend->io_bio = bio;return ioend; }
fs/xfs/xfs_aops.h +1 -1
@@ -18,7 +18,7 @@ struct xfs_ioend {struct inode *io_inode;/* file being written to */ size_t io_size;/* size of the extent */ xfs_off_t io_offset;/* offset in the file */ -struct xfs_trans *io_append_trans;/* xact. for size update */ +void *io_private;/* file system private data */ struct bio *io_bio;/* bio being built */ struct bio io_inline_bio;/* MUST BE LAST! */ };
Signature · 1 line
2.20.1
PATCH 7/14 xfs: remove the fork fields in the writepage_ctx and ioend
In reply to Christoph HellwigIn preparation for moving the writeback code to iomap.c, replace the XFS-specific COW fork concept with the iomap IOMAP_F_SHARED flag.
--- fs/xfs/xfs_aops.c | 71 ++++++++++++++++++++++++++++++----------------- fs/xfs/xfs_aops.h | 2 +- 2 files changed, 46 insertions(+), 27 deletions(-)
fs/xfs/xfs_aops.c +45 -26
@@ -23,7 +23,6 @@*/ struct xfs_writepage_ctx {struct iomap iomap; -int fork;unsigned int data_seq;unsigned int cow_seq;struct xfs_ioend *ioend;
@@ -272,7 +271,7 @@ xfs_end_ioend(*/ error =blk_status_to_errno (ioend->io_bio->bi_status);if (unlikely (error)) { -if (ioend->io_fork ==XFS_COW_FORK ) +if (ioend->io_flags &IOMAP_F_SHARED )xfs_reflink_cancel_cow_range (ip, offset, size, true); goto done; }
@@ -280,7 +279,7 @@ xfs_end_ioend(/* * Success: commit the COW or unwritten blocks if needed. */ -if (ioend->io_fork ==XFS_COW_FORK ) +if (ioend->io_flags &IOMAP_F_SHARED ) error =xfs_reflink_end_cow (ip, offset, size);else if (ioend->io_type ==IOMAP_UNWRITTEN ) error =xfs_iomap_write_unwritten (ip, offset, size, false);
@@ -304,7 +303,8 @@ xfs_ioend_can_merge( {if (ioend->io_bio->bi_status != next->io_bio->bi_status)return false; -if ((ioend->io_fork ==XFS_COW_FORK ) ^ (next->io_fork ==XFS_COW_FORK )) +if ((ioend->io_flags &IOMAP_F_SHARED ) ^ + (next->io_flags &IOMAP_F_SHARED ))return false;if ((ioend->io_type ==IOMAP_UNWRITTEN ) ^ (next->io_type ==IOMAP_UNWRITTEN ))
@@ -404,6 +404,13 @@ xfs_end_io( } } +static inline bool xfs_ioend_needs_workqueue (struct xfs_ioend *ioend) +{ +return ioend->io_private || + ioend->io_type ==IOMAP_UNWRITTEN || + (ioend->io_flags &IOMAP_F_SHARED ); +} + STATIC voidxfs_end_bio (struct bio *bio)
@@ -413,9 +420,7 @@ xfs_end_bio(struct xfs_mount *mp = ip->i_mount;unsigned long flags; - if (ioend->io_fork == XFS_COW_FORK || - ioend->io_type == IOMAP_UNWRITTEN || - ioend->io_private) { + if (xfs_ioend_needs_workqueue(ioend)) { spin_lock_irqsave(&ip->i_ioend_lock, flags); if (list_empty(&ip->i_ioend_list)) WARN_ON_ONCE(!queue_work(mp->m_unwritten_workqueue,
@@ -444,7 +449,7 @@ xfs_imap_valid(* covers the offset. Be careful to check this first because the caller * can revalidate a COW mapping without updating the data seqno. */ -if (wpc->fork ==XFS_COW_FORK ) +if (wpc->iomap.flags &IOMAP_F_SHARED )return true;/*
@@ -474,6 +479,7 @@ static intxfs_convert_blocks ( struct xfs_writepage_ctx *wpc, struct xfs_inode *ip, + int whichfork, loff_t offset) { int error;
@@ -485,8 +491,8 @@ xfs_convert_blocks(* delalloc extent if free space is sufficiently fragmented. */ do { - error = xfs_bmapi_convert_delalloc(ip, wpc->fork, offset, - &wpc->iomap, wpc->fork == XFS_COW_FORK ? + error = xfs_bmapi_convert_delalloc(ip, whichfork, offset, + &wpc->iomap, whichfork == XFS_COW_FORK ? &wpc->cow_seq : &wpc->data_seq); if (error) return error;
@@ -507,6 +513,7 @@ xfs_map_blocks(xfs_fileoff_t offset_fsb =XFS_B_TO_FSBT (mp, offset);xfs_fileoff_t end_fsb =XFS_B_TO_FSB (mp, offset + count);xfs_fileoff_t cow_fsb =NULLFILEOFF ; +int whichfork =XFS_DATA_FORK ;struct xfs_bmbt_irec imap;struct xfs_iext_cursor icur;int retries =0 ;
@@ -555,7 +562,7 @@ xfs_map_blocks( wpc->cow_seq =READ_ONCE (ip->i_cowfp->if_seq);xfs_iunlock (ip,XFS_ILOCK_SHARED ); - wpc->fork =XFS_COW_FORK ; + whichfork =XFS_COW_FORK ; goto allocate_blocks; }
@@ -578,8 +585,6 @@ xfs_map_blocks( wpc->data_seq =READ_ONCE (ip->i_df.if_seq);xfs_iunlock (ip,XFS_ILOCK_SHARED ); - wpc->fork =XFS_DATA_FORK ; -/* landed in a hole or beyond EOF? */ if (imap.br_startoff > offset_fsb) { imap.br_blockcount = imap.br_startoff - offset_fsb;
@@ -604,10 +609,10 @@ xfs_map_blocks( goto allocate_blocks;xfs_bmbt_to_iomap (ip, &wpc->iomap, &imap,0 ); -trace_xfs_map_blocks_found (ip, offset, count, wpc->fork, &imap); +trace_xfs_map_blocks_found (ip, offset, count, whichfork, &imap);return 0 ; allocate_blocks: - error =xfs_convert_blocks (wpc, ip, offset); + error =xfs_convert_blocks (wpc, ip, whichfork, offset);if (error) {/* * If we failed to find the extent in the COW fork we might have
@@ -616,7 +621,7 @@ xfs_map_blocks(* the former case, but prevent additional retries to avoid * looping forever for the latter case. */ -if (error == -EAGAIN && wpc->fork ==XFS_COW_FORK && !retries++) +if (error == -EAGAIN && whichfork ==XFS_COW_FORK && !retries++) goto retry;ASSERT (error != -EAGAIN );return error;
@@ -627,7 +632,7 @@ xfs_map_blocks(* original delalloc one. Trim the return extent to the next COW * boundary again to force a re-lookup. */ -if (wpc->fork !=XFS_COW_FORK && cow_fsb !=NULLFILEOFF ) { +if (whichfork !=XFS_COW_FORK && cow_fsb !=NULLFILEOFF ) {loff_t cow_offset =XFS_FSB_TO_B (mp, cow_fsb);if (cow_offset < wpc->iomap.offset + wpc->iomap.length)
@@ -636,7 +641,7 @@ xfs_map_blocks(ASSERT (wpc->iomap.offset <= offset);ASSERT (wpc->iomap.offset + wpc->iomap.length > offset); -trace_xfs_map_blocks_alloc (ip, offset, count, wpc->fork, &imap); +trace_xfs_map_blocks_alloc (ip, offset, count, whichfork, &imap);return 0 ; }
@@ -670,14 +675,14 @@ xfs_submit_ioend( nofs_flag =memalloc_nofs_save ();/* Convert CoW extents to regular */ -if (!status && ioend->io_fork ==XFS_COW_FORK ) { +if (!status && (ioend->io_flags &IOMAP_F_SHARED )) { status =xfs_reflink_convert_cow (XFS_I (ioend->io_inode), ioend->io_offset, ioend->io_size); }/* Reserve log space if we might write beyond the on-disk inode size. */ if (!status && - (ioend->io_fork ==XFS_COW_FORK || + ((ioend->io_flags &IOMAP_F_SHARED ) || ioend->io_type !=IOMAP_UNWRITTEN ) &&xfs_ioend_is_append (ioend) && !ioend->io_private)
@@ -724,8 +729,8 @@ xfs_alloc_ioend( ioend = container_of(bio, struct xfs_ioend, io_inline_bio);INIT_LIST_HEAD (&ioend->io_list); - ioend->io_fork = wpc->fork; ioend->io_type = wpc->iomap.type; + ioend->io_flags = wpc->iomap.flags; ioend->io_inode = inode; ioend->io_size =0 ; ioend->io_offset = offset;
@@ -759,6 +764,24 @@ xfs_chain_bio(return new; } +static bool +xfs_can_add_to_ioend ( +struct xfs_writepage_ctx *wpc, +xfs_off_t offset, +sector_t sector) +{ +if ((wpc->iomap.flags &IOMAP_F_SHARED ) != + (wpc->ioend->io_flags &IOMAP_F_SHARED )) +return false; +if (wpc->iomap.type != wpc->ioend->io_type) +return false; +if (offset != wpc->ioend->io_offset + wpc->ioend->io_size) +return false; +if (sector !=bio_end_sector (wpc->ioend->io_bio)) +return false; +return true; +} +/* * Test to see if we have an existing ioend structure that we could append to * first, otherwise finish off the current ioend and start another.
@@ -778,11 +801,7 @@ xfs_add_to_ioend(unsigned poff = offset & (PAGE_SIZE -1 );bool merged, same_page = false; -if (!wpc->ioend || - wpc->fork != wpc->ioend->io_fork || - wpc->iomap.type != wpc->ioend->io_type || - sector !=bio_end_sector (wpc->ioend->io_bio) || - offset != wpc->ioend->io_offset + wpc->ioend->io_size) { +if (!wpc->ioend || !xfs_can_add_to_ioend (wpc, offset, sector)) {if (wpc->ioend)list_add (&wpc->ioend->io_list, iolist); wpc->ioend =xfs_alloc_ioend (inode, wpc, offset, sector, wbc);
fs/xfs/xfs_aops.h +1 -1
@@ -13,8 +13,8 @@ extern struct bio_set xfs_ioend_bioset;*/ struct xfs_ioend {struct list_head io_list;/* next ioend in chain */ -int io_fork;/* inode fork written back */ u16 io_type; +u16 io_flags;/* IOMAP_F_* */ struct inode *io_inode;/* file being written to */ size_t io_size;/* size of the extent */ xfs_off_t io_offset;/* offset in the file */
Signature · 1 line
2.20.1
PATCH 8/14 iomap: zero newly allocated mapped blocks
In reply to Christoph HellwigFile systems like gfs2 don't support delayed allocations or unwritten extents and thus allocate normal mapped blocks to fill holes. To cover the case of such file systems allocating new blocks to fill holes also zero out mapped blocks with the new flag.
--- fs/iomap/buffered-io.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
fs/iomap/buffered-io.c +10 -2
@@ -203,6 +203,14 @@ iomap_read_inline_data(struct inode *inode, struct page *page,SetPageUptodate (page); } +static inline bool iomap_block_needs_zeroing (struct inode *inode, +struct iomap *iomap,loff_t pos) +{ +return iomap->type !=IOMAP_MAPPED || + (iomap->flags &IOMAP_F_NEW ) || + pos >=i_size_read (inode); +} +static loff_t iomap_readpage_actor (struct inode *inode,loff_t pos,loff_t length,void *data,struct iomap *iomap)
@@ -226,7 +234,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data,if (plen ==0 ) goto done; -if (iomap->type !=IOMAP_MAPPED || pos >=i_size_read (inode)) { +if (iomap_block_needs_zeroing (inode, iomap, pos)) {zero_user (page, poff, plen);iomap_set_range_uptodate (page, poff, plen); goto done;
@@ -532,7 +540,7 @@ iomap_read_page_sync(struct inode *inode, loff_t block_start, struct page *page,struct bio_vec bvec;struct bio bio; -if (iomap->type !=IOMAP_MAPPED || block_start >=i_size_read (inode)) { +if (iomap_block_needs_zeroing (inode, iomap, block_start)) {zero_user_segments (page, poff, from, to, poff + plen);iomap_set_range_uptodate (page, poff, plen);return 0 ;
Signature · 1 line
2.20.1
PATCH 9/14 iomap: lift common tracing code from xfs to iomap
In reply to Christoph HellwigLift the xfs code for tracing address space operations to the iomap layer.
--- fs/iomap/Makefile | 16 ++++---- fs/iomap/buffered-io.c | 9 +++++ fs/iomap/trace.c | 12 ++++++ fs/iomap/trace.h | 87 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_aops.c | 27 ++----------- fs/xfs/xfs_trace.h | 26 ------------- 6 files changed, 120 insertions(+), 57 deletions(-) create mode 100644 fs/iomap/trace.c create mode 100644 fs/iomap/trace.h
fs/iomap/Makefile +9 -7
@@ -3,13 +3,15 @@ # Copyright (c) 2019 Oracle. # All Rights Reserved. # -obj-$(CONFIG_FS_IOMAP) += iomap.o -iomap-y += \ - apply.o \ - buffered-io.o \ - direct-io.o \ - fiemap.o \ - seek.o +ccflags-y += -I $(srctree)/$(src) # needed for trace events + +obj-$(CONFIG_FS_IOMAP) += iomap.o +iomap-y += trace.o \ + apply.o \ + buffered-io.o \ + direct-io.o \ + fiemap.o \ + seek.o iomap-$(CONFIG_SWAP) += swapfile.o
fs/iomap/buffered-io.c +9 -0
@@ -16,6 +16,7 @@#include <linux/bio.h> #include <linux/sched/signal.h> #include <linux/migrate.h> +#include "trace.h" #include "../internal.h"
@@ -301,6 +302,8 @@ iomap_readpage(struct page *page, const struct iomap_ops *ops)unsigned poff;loff_t ret; +trace_iomap_readpage (page->mapping->host,1 ); +for (poff =0 ; poff <PAGE_SIZE ; poff += ret) { ret = iomap_apply(inode, page_offset(page) + poff, PAGE_SIZE - poff, 0, ops, &ctx,
@@ -397,6 +400,8 @@ iomap_readpages(struct address_space *mapping, struct list_head *pages, loff_t last = page_offset(list_entry(pages->next, struct page, lru));loff_t length = last - pos +PAGE_SIZE , ret =0 ; +trace_iomap_readpages (mapping->host, nr_pages); +while (length >0 ) { ret =iomap_apply (mapping->host, pos, length,0 , ops, &ctx, iomap_readpages_actor);
@@ -463,6 +468,8 @@ EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate);int iomap_releasepage (struct page *page,gfp_t gfp_mask) { +trace_iomap_releasepage (page->mapping->host, page,0 ,0 ); +/* * mm accommodates an old ext3 case where clean pages might not have had * the dirty bit cleared. Thus, it can send actual dirty pages to
@@ -478,6 +485,8 @@ EXPORT_SYMBOL_GPL(iomap_releasepage);void iomap_invalidatepage (struct page *page,unsigned int offset,unsigned int len) { +trace_iomap_invalidatepage (page->mapping->host, page, offset, len); +/* * If we are invalidating the entire page, clear the dirty state from it * and release it to avoid unnecessary buildup of the LRU.
fs/iomap/trace.c new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2019 Christoph Hellwig +*/ +#include <linux/iomap.h> + +/* +* We include this last to have the helpers above available for the trace +* event implementations. +*/ +#define CREATE_TRACE_POINTS +#include "trace.h"
fs/iomap/trace.h new file +87 -0
87 lines not on this page · raw message
fs/xfs/xfs_aops.c +3 -24
@@ -823,16 +823,6 @@ xfs_add_to_ioend(wbc_account_cgroup_owner (wbc, page, len); } -STATIC void -xfs_vm_invalidatepage ( -struct page *page, -unsigned int offset, -unsigned int length) -{ -trace_xfs_invalidatepage (page->mapping->host, page, offset, length); -iomap_invalidatepage (page, offset, length); -} -/* * If the page has delalloc blocks on it, we need to punch them out before we * invalidate the page. If we don't, we leave a stale delalloc mapping on the
@@ -867,7 +857,7 @@ xfs_aops_discard_page(if (error && !XFS_FORCED_SHUTDOWN (mp))xfs_alert (mp,"page discard unable to remove delalloc mapping." ); out_invalidate: -xfs_vm_invalidatepage (page,0 ,PAGE_SIZE ); +iomap_invalidatepage (page,0 ,PAGE_SIZE ); }/*
@@ -1147,15 +1137,6 @@ xfs_dax_writepages( xfs_find_bdev_for_inode(mapping->host), wbc); } -STATIC int -xfs_vm_releasepage ( -struct page *page, -gfp_t gfp_mask) -{ -trace_xfs_releasepage (page->mapping->host, page,0 ,0 ); -return iomap_releasepage (page, gfp_mask); -} - STATIC sector_t xfs_vm_bmap( struct address_space *mapping,
@@ -1184,7 +1165,6 @@ xfs_vm_readpage(struct file *unused, struct page *page) { - trace_xfs_vm_readpage(page->mapping->host, 1); return iomap_readpage(page, &xfs_iomap_ops); }
@@ -1195,7 +1175,6 @@ xfs_vm_readpages( struct list_head *pages, unsigned nr_pages) { -trace_xfs_vm_readpages (mapping->host, nr_pages);return iomap_readpages (mapping, pages, nr_pages, &xfs_iomap_ops); }
@@ -1215,8 +1194,8 @@ const struct address_space_operations xfs_address_space_operations = { .writepage = xfs_vm_writepage, .writepages = xfs_vm_writepages, .set_page_dirty = iomap_set_page_dirty, - .releasepage = xfs_vm_releasepage, - .invalidatepage = xfs_vm_invalidatepage, + .releasepage = iomap_releasepage, + .invalidatepage = iomap_invalidatepage, .bmap = xfs_vm_bmap, .direct_IO = noop_direct_IO, .migratepage = iomap_migrate_page,
fs/xfs/xfs_trace.h +0 -26
@@ -1197,32 +1197,6 @@ DEFINE_PAGE_EVENT(xfs_writepage);DEFINE_PAGE_EVENT (xfs_releasepage);DEFINE_PAGE_EVENT (xfs_invalidatepage); -DECLARE_EVENT_CLASS (xfs_readpage_class, - TP_PROTO(struct inode *inode, int nr_pages), -TP_ARGS (inode, nr_pages), -TP_STRUCT__entry ( - __field(dev_t, dev) - __field(xfs_ino_t, ino) -__field (int, nr_pages) - ), -TP_fast_assign ( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->nr_pages = nr_pages; - ), -TP_printk ("dev %d:%d ino 0x%llx nr_pages %d" , -MAJOR (__entry->dev),MINOR (__entry->dev), - __entry->ino, - __entry->nr_pages) -) - -#define DEFINE_READPAGE_EVENT (name) \ -DEFINE_EVENT(xfs_readpage_class, name, \ - TP_PROTO(struct inode *inode, int nr_pages), \ - TP_ARGS(inode, nr_pages)) -DEFINE_READPAGE_EVENT (xfs_vm_readpage); -DEFINE_READPAGE_EVENT (xfs_vm_readpages); - DECLARE_EVENT_CLASS(xfs_imap_class, TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count, int whichfork, struct xfs_bmbt_irec *irec),
Signature · 1 line
2.20.1
PATCH 10/14 iomap: lift the xfs writeback code to iomap
In reply to Christoph HellwigTake the xfs writeback code and move it to fs/iomap. A new structure with three methods is added as the abstraction from the generic writeback code to the file system. These methods are used to map blocks, submit an ioend, and cancel a page that encountered an error before it was added to an ioend.
--- fs/iomap/buffered-io.c | 551 ++++++++++++++++++++++++++++++++++- fs/iomap/trace.h | 1 + fs/xfs/xfs_aops.c | 637 ++++------------------------------------- fs/xfs/xfs_aops.h | 17 -- fs/xfs/xfs_super.c | 11 +- fs/xfs/xfs_trace.h | 39 --- include/linux/iomap.h | 59 ++++ 7 files changed, 668 insertions(+), 647 deletions(-)
fs/iomap/buffered-io.c +550 -1
@@ -1,7 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* * Copyright (C) 2010 Red Hat, Inc. -* Copyright (c) 2016-2018 Christoph Hellwig. +* Copyright (C) 2016-2019 Christoph Hellwig. */ #include <linux/module.h> #include <linux/compiler.h>
@@ -12,6 +12,7 @@#include <linux/buffer_head.h> #include <linux/dax.h> #include <linux/writeback.h> +#include <linux/list_sort.h> #include <linux/swap.h> #include <linux/bio.h> #include <linux/sched/signal.h>
@@ -20,6 +21,8 @@#include "../internal.h" +static struct bio_set iomap_ioend_bioset; +static struct iomap_page *iomap_page_create (struct inode *inode,struct page *page) {
@@ -1088,3 +1091,549 @@ vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops)return block_page_mkwrite_return (ret); }EXPORT_SYMBOL_GPL (iomap_page_mkwrite); + +static void +iomap_finish_page_writeback (struct inode *inode,struct bio_vec *bvec, +int error) +{ +struct iomap_page *iop =to_iomap_page (bvec->bv_page); + +if (error) { +SetPageError (bvec->bv_page); +mapping_set_error (inode->i_mapping, -EIO ); + } + +WARN_ON_ONCE (i_blocksize (inode) <PAGE_SIZE && !iop); +WARN_ON_ONCE (iop &&atomic_read (&iop->write_count) <=0 ); + +if (!iop ||atomic_dec_and_test (&iop->write_count)) +end_page_writeback (bvec->bv_page); +} + +/* +* We're now finished for good with this ioend structure. Update the page +* state, release holds on bios, and finally free up memory. Do not use the +* ioend after this. +*/ +static void +iomap_finish_ioend (struct iomap_ioend *ioend,int error) +{ +struct inode *inode = ioend->io_inode; +struct bio *bio = &ioend->io_inline_bio; +struct bio *last = ioend->io_bio, *next; +u64 start = bio->bi_iter.bi_sector; +bool quiet =bio_flagged (bio,BIO_QUIET ); + +for (bio = &ioend->io_inline_bio; bio; bio = next) { +struct bio_vec *bv; +struct bvec_iter_all iter_all; + +/* +* For the last bio, bi_private points to the ioend, so we +* need to explicitly end the iteration here. +*/ +if (bio == last) + next =NULL ; +else + next = bio->bi_private; + +/* walk each page on bio, ending page IO on them */ +bio_for_each_segment_all (bv, bio, iter_all) +iomap_finish_page_writeback (inode, bv, error); +bio_put (bio); + } + +if (unlikely (error && !quiet)) { +printk_ratelimited (KERN_ERR +"%s: writeback error on sector %llu" , + inode->i_sb->s_id, start); + } +} + +void +iomap_finish_ioends (struct iomap_ioend *ioend,int error) +{ +struct list_head tmp; + +list_replace_init (&ioend->io_list, &tmp); +iomap_finish_ioend (ioend, error); + +while (!list_empty (&tmp)) { + ioend = list_first_entry(&tmp, struct iomap_ioend, io_list); +list_del_init (&ioend->io_list); +iomap_finish_ioend (ioend, error); + } +} +EXPORT_SYMBOL_GPL (iomap_finish_ioends); + +/* +* We can merge two adjacent ioends if they have the same set of work to do. +*/ +static bool +iomap_ioend_can_merge (struct iomap_ioend *ioend,struct iomap_ioend *next) +{ +if (ioend->io_bio->bi_status != next->io_bio->bi_status) +return false; +if ((ioend->io_flags &IOMAP_F_SHARED ) ^ + (next->io_flags &IOMAP_F_SHARED )) +return false; +if ((ioend->io_type ==IOMAP_UNWRITTEN ) ^ + (next->io_type ==IOMAP_UNWRITTEN )) +return false; +if (ioend->io_offset + ioend->io_size != next->io_offset) +return false; +return true; +} + +void +iomap_ioend_try_merge (struct iomap_ioend *ioend,struct list_head *more_ioends, +void (*merge_private)(struct iomap_ioend *ioend, +struct iomap_ioend *next)) +{ +struct iomap_ioend *next; + +INIT_LIST_HEAD (&ioend->io_list); + + while ((next = list_first_entry_or_null(more_ioends, struct iomap_ioend, + io_list))) { +if (!iomap_ioend_can_merge (ioend, next)) +break ; +list_move_tail (&next->io_list, &ioend->io_list); + ioend->io_size += next->io_size; +if (next->io_private && merge_private) +merge_private (ioend, next); + } +} +EXPORT_SYMBOL_GPL (iomap_ioend_try_merge); + +static int +iomap_ioend_compare (void *priv,struct list_head *a,struct list_head *b) +{ +struct iomap_ioend *ia, *ib; + + ia = container_of(a, struct iomap_ioend, io_list); + ib = container_of(b, struct iomap_ioend, io_list); +if (ia->io_offset < ib->io_offset) +return -1 ; +else if (ia->io_offset > ib->io_offset) +return 1 ; +return 0 ; +} + +void +iomap_sort_ioends (struct list_head *ioend_list) +{ +list_sort (NULL , ioend_list, iomap_ioend_compare); +} +EXPORT_SYMBOL_GPL (iomap_sort_ioends); + +static void iomap_writepage_end_bio (struct bio *bio) +{ +struct iomap_ioend *ioend = bio->bi_private; + +iomap_finish_ioend (ioend,blk_status_to_errno (bio->bi_status)); +} + +/* +* Submit the final bio for an ioend. +* +* If @error is non-zero, it means that we have a situation where some part of +* the submission process has failed after we have marked paged for writeback +* and unlocked them. In this situation, we need to fail the bio instead of +* submitting it. This typically only happens on a filesystem shutdown. +*/ +static int +iomap_submit_ioend (struct iomap_writepage_ctx *wpc,struct iomap_ioend *ioend, +int error) +{ + ioend->io_bio->bi_private = ioend; + ioend->io_bio->bi_end_io = iomap_writepage_end_bio; + +if (wpc->ops->submit_ioend) + error = wpc->ops->submit_ioend (ioend, error); +if (error) { +/* +* If we are failing the IO now, just mark the ioend with an +* error and finish it. This will run IO completion immediately +* as there is only one reference to the ioend at this point in +* time. +*/ + ioend->io_bio->bi_status =errno_to_blk_status (error); +bio_endio (ioend->io_bio); +return error; + } + +submit_bio (ioend->io_bio); +return 0 ; +} + +static struct iomap_ioend * +iomap_alloc_ioend (struct inode *inode,struct iomap_writepage_ctx *wpc, +loff_t offset,sector_t sector,struct writeback_control *wbc) +{ +struct iomap_ioend *ioend; +struct bio *bio; + + bio =bio_alloc_bioset (GFP_NOFS ,BIO_MAX_PAGES , &iomap_ioend_bioset); +bio_set_dev (bio, wpc->iomap.bdev); + bio->bi_iter.bi_sector = sector; + bio->bi_opf =REQ_OP_WRITE |wbc_to_write_flags (wbc); + bio->bi_write_hint = inode->i_write_hint; +wbc_init_bio (wbc, bio); + + ioend = container_of(bio, struct iomap_ioend, io_inline_bio); +INIT_LIST_HEAD (&ioend->io_list); + ioend->io_type = wpc->iomap.type; + ioend->io_flags = wpc->iomap.flags; + ioend->io_inode = inode; + ioend->io_size =0 ; + ioend->io_offset = offset; + ioend->io_private =NULL ; + ioend->io_bio = bio; +return ioend; +} + +/* +* Allocate a new bio, and chain the old bio to the new one. +* +* Note that we have to do perform the chaining in this unintuitive order +* so that the bi_private linkage is set up in the right direction for the +* traversal in iomap_finish_ioend(). +*/ +static struct bio * +iomap_chain_bio (struct bio *prev) +{ +struct bio *new; + + new =bio_alloc (GFP_NOFS ,BIO_MAX_PAGES ); +bio_copy_dev (new, prev);/* also copies over blkcg information */ + new->bi_iter.bi_sector =bio_end_sector (prev); + new->bi_opf = prev->bi_opf; + new->bi_write_hint = prev->bi_write_hint; + +bio_chain (prev, new); +bio_get (prev);/* for iomap_finish_ioend */ +submit_bio (prev); +return new; +} + +static bool +iomap_can_add_to_ioend (struct iomap_writepage_ctx *wpc,loff_t offset, +sector_t sector) +{ +if ((wpc->iomap.flags &IOMAP_F_SHARED ) != + (wpc->ioend->io_flags &IOMAP_F_SHARED )) +return false; +if (wpc->iomap.type != wpc->ioend->io_type) +return false; +if (offset != wpc->ioend->io_offset + wpc->ioend->io_size) +return false; +if (sector !=bio_end_sector (wpc->ioend->io_bio)) +return false; +return true; +} + +/* +* Test to see if we have an existing ioend structure that we could append to +* first, otherwise finish off the current ioend and start another. +*/ +static void +iomap_add_to_ioend (struct inode *inode,loff_t offset,struct page *page, +struct iomap_page *iop,struct iomap_writepage_ctx *wpc, +struct writeback_control *wbc,struct list_head *iolist) +{ +sector_t sector =iomap_sector (&wpc->iomap, offset); +unsigned len =i_blocksize (inode); +unsigned poff = offset & (PAGE_SIZE -1 ); +bool merged, same_page = false; + +if (!wpc->ioend || !iomap_can_add_to_ioend (wpc, offset, sector)) { +if (wpc->ioend) +list_add (&wpc->ioend->io_list, iolist); + wpc->ioend =iomap_alloc_ioend (inode, wpc, offset, sector, wbc); + } + + merged =__bio_try_merge_page (wpc->ioend->io_bio, page, len, poff, + &same_page); +if (iop && !same_page) +atomic_inc (&iop->write_count); + +if (!merged) { +if (bio_full (wpc->ioend->io_bio, len)) { + wpc->ioend->io_bio = +iomap_chain_bio (wpc->ioend->io_bio); + } +bio_add_page (wpc->ioend->io_bio, page, len, poff); + } + + wpc->ioend->io_size += len; +wbc_account_cgroup_owner (wbc, page, len); +} + +/* +* We implement an immediate ioend submission policy here to avoid needing to +* chain multiple ioends and hence nest mempool allocations which can violate +* forward progress guarantees we need to provide. The current ioend we are +* adding blocks to is cached on the writepage context, and if the new block +* does not append to the cached ioend it will create a new ioend and cache that +* instead. +* +* If a new ioend is created and cached, the old ioend is returned and queued +* locally for submission once the entire page is processed or an error has been +* detected. While ioends are submitted immediately after they are completed, +* batching optimisations are provided by higher level block plugging. +* +* At the end of a writeback pass, there will be a cached ioend remaining on the +* writepage context that the caller will need to submit. +*/ +static int +iomap_writepage_map (struct iomap_writepage_ctx *wpc, +struct writeback_control *wbc,struct inode *inode, +struct page *page,u64 end_offset) +{ +struct iomap_page *iop =to_iomap_page (page); +struct iomap_ioend *ioend, *next; +unsigned len =i_blocksize (inode); +u64 file_offset;/* file offset of page */ +int error =0 , count =0 , i; +LIST_HEAD (submit_list); + +WARN_ON_ONCE (i_blocksize (inode) <PAGE_SIZE && !iop); +WARN_ON_ONCE (iop &&atomic_read (&iop->write_count) !=0 ); + +/* +* Walk through the page to find areas to write back. If we run off the +* end of the current map or find the current map invalid, grab a new +* one. +*/ +for (i =0 , file_offset =page_offset (page); + i < (PAGE_SIZE >> inode->i_blkbits) && file_offset < end_offset; + i++, file_offset += len) { +if (iop && !test_bit (i, iop->uptodate)) +continue ; + + error = wpc->ops->map_blocks (wpc, inode, file_offset); +if (error) +break ; +if (wpc->iomap.type ==IOMAP_HOLE ) +continue ; +iomap_add_to_ioend (inode, file_offset, page, iop, wpc, wbc, + &submit_list); + count++; + } + +WARN_ON_ONCE (!wpc->ioend && !list_empty (&submit_list)); +WARN_ON_ONCE (!PageLocked (page)); +WARN_ON_ONCE (PageWriteback (page)); + +/* +* We cannot cancel the ioend directly here on error. We may have +* already set other pages under writeback and hence we have to run I/O +* completion to mark the error state of the pages under writeback +* appropriately. +*/ +if (unlikely (error)) { +if (!count) { +/* +* If the current page hasn't been added to ioend, it +* won't be affected by I/O completions and we must +* discard and unlock it right here. +*/ +if (wpc->ops->discard_page) + wpc->ops->discard_page (page); +ClearPageUptodate (page); +unlock_page (page); + goto done; + } + +/* +* If the page was not fully cleaned, we need to ensure that the +* higher layers come back to it correctly. That means we need +* to keep the page dirty, and for WB_SYNC_ALL writeback we need +* to ensure the PAGECACHE_TAG_TOWRITE index mark is not removed +* so another attempt to write this page in this writeback sweep +* will be made. +*/ +set_page_writeback_keepwrite (page); + }else { +clear_page_dirty_for_io (page); +set_page_writeback (page); + } + +unlock_page (page); + +/* +* Preserve the original error if there was one, otherwise catch +* submission errors here and propagate into subsequent ioend +* submissions. +*/ +list_for_each_entry_safe (ioend, next, &submit_list, io_list) { +int error2; + +list_del_init (&ioend->io_list); + error2 =iomap_submit_ioend (wpc, ioend, error); +if (error2 && !error) + error = error2; + } + +/* +* We can end up here with no error and nothing to write only if we race +* with a partial page truncate on a sub-page block sized filesystem. +*/ +if (!count) +end_page_writeback (page); +done: +mapping_set_error (page->mapping, error); +return error; +} + +/* +* Write out a dirty page. +* +* For delalloc space on the page we need to allocate space and flush it. +* For unwritten space on the page we need to start the conversion to +* regular allocated space. +*/ +static int +iomap_do_writepage (struct page *page,struct writeback_control *wbc,void *data) +{ +struct iomap_writepage_ctx *wpc = data; +struct inode *inode = page->mapping->host; +pgoff_t end_index; +u64 end_offset; +loff_t offset; + +trace_iomap_writepage (inode, page,0 ,0 ); + +/* +* Refuse to write the page out if we are called from reclaim context. +* +* This avoids stack overflows when called from deeply used stacks in +* random callers for direct reclaim or memcg reclaim. We explicitly +* allow reclaim from kswapd as the stack usage there is relatively low. +* +* This should never happen except in the case of a VM regression so +* warn about it. +*/ +if (WARN_ON_ONCE ((current->flags & (PF_MEMALLOC |PF_KSWAPD )) == +PF_MEMALLOC )) + goto redirty; + +/* +* Given that we do not allow direct reclaim to call us, we should +* never be called in a recursive filesystem reclaim context. +*/ +if (WARN_ON_ONCE (current->flags &PF_MEMALLOC_NOFS )) + goto redirty; + +/* +* Is this page beyond the end of the file? +* +* The page index is less than the end_index, adjust the end_offset +* to the highest offset that this page should represent. +* ----------------------------------------------------- +* | file mapping | <EOF> | +* ----------------------------------------------------- +* | Page ... | Page N-2 | Page N-1 | Page N | | +* ^--------------------------------^----------|-------- +* | desired writeback range | see else | +* ---------------------------------^------------------| +*/ + offset =i_size_read (inode); + end_index = offset >>PAGE_SHIFT ; +if (page->index < end_index) + end_offset = (loff_t)(page->index +1 ) <<PAGE_SHIFT ; +else { +/* +* Check whether the page to write out is beyond or straddles +* i_size or not. +* ------------------------------------------------------- +* | file mapping | <EOF> | +* ------------------------------------------------------- +* | Page ... | Page N-2 | Page N-1 | Page N | Beyond | +* ^--------------------------------^-----------|--------- +* | | Straddles | +* ---------------------------------^-----------|--------| +*/ +unsigned offset_into_page = offset & (PAGE_SIZE -1 ); + +/* +* Skip the page if it is fully outside i_size, e.g. due to a +* truncate operation that is in progress. We must redirty the +* page so that reclaim stops reclaiming it. Otherwise +* iomap_vm_releasepage() is called on it and gets confused. +* +* Note that the end_index is unsigned long, it would overflow +* if the given offset is greater than 16TB on 32-bit system +* and if we do check the page is fully outside i_size or not +* via "if (page->index >= end_index + 1)" as "end_index + 1" +* will be evaluated to 0. Hence this page will be redirtied +* and be written out repeatedly which would result in an +* infinite loop, the user program that perform this operation +* will hang. Instead, we can verify this situation by checking +* if the page to write is totally beyond the i_size or if it's +* offset is just equal to the EOF. +*/ +if (page->index > end_index || + (page->index == end_index && offset_into_page ==0 )) + goto redirty; + +/* +* The page straddles i_size. It must be zeroed out on each +* and every writepage invocation because it may be mmapped. +* "A file is mapped in multiples of the page size. For a file +* that is not a multiple of the page size, the remaining +* memory is zeroed when mapped, and writes to that region are +* not written out to the file." +*/ +zero_user_segment (page, offset_into_page,PAGE_SIZE ); + +/* Adjust the end_offset to the end of file */ + end_offset = offset; + } + +return iomap_writepage_map (wpc, wbc, inode, page, end_offset); + +redirty: +redirty_page_for_writepage (wbc, page); +unlock_page (page); +return 0 ; +} + +int +iomap_writepage (struct page *page,struct writeback_control *wbc, +struct iomap_writepage_ctx *wpc, +const struct iomap_writeback_ops *ops) +{ +int ret; + + wpc->ops = ops; + ret =iomap_do_writepage (page, wbc, wpc); +if (!wpc->ioend) +return ret; +return iomap_submit_ioend (wpc, wpc->ioend, ret); +} +EXPORT_SYMBOL_GPL (iomap_writepage); + +int +iomap_writepages (struct address_space *mapping,struct writeback_control *wbc, +struct iomap_writepage_ctx *wpc, +const struct iomap_writeback_ops *ops) +{ +int ret; + + wpc->ops = ops; + ret =write_cache_pages (mapping, wbc, iomap_do_writepage, wpc); +if (!wpc->ioend) +return ret; +return iomap_submit_ioend (wpc, wpc->ioend, ret); +} +EXPORT_SYMBOL_GPL (iomap_writepages); + +static int __init iomap_init (void ) +{ +return bioset_init (&iomap_ioend_bioset,4 * (PAGE_SIZE /SECTOR_SIZE ), + offsetof(struct iomap_ioend , io_inline_bio), +BIOSET_NEED_BVECS ); +} +fs_initcall (iomap_init);
fs/iomap/trace.h +1 -0
@@ -76,6 +76,7 @@ DEFINE_EVENT(iomap_page_class, name, \ TP_PROTO(struct inode *inode, struct page *page, unsigned long off, \ unsigned int len), \ TP_ARGS(inode, page, off, len)) +DEFINE_PAGE_EVENT(iomap_writepage);DEFINE_PAGE_EVENT (iomap_releasepage);DEFINE_PAGE_EVENT (iomap_invalidatepage);
fs/xfs/xfs_aops.c +57 -580
813 lines not on this page · raw message
fs/xfs/xfs_aops.h +0 -17
@@ -6,23 +6,6 @@#ifndef __XFS_AOPS_H__#define __XFS_AOPS_H__ -extern struct bio_set xfs_ioend_bioset; - -/* -* Structure for buffered I/O completions. -*/ -struct xfs_ioend { -struct list_head io_list;/* next ioend in chain */ -u16 io_type; -u16 io_flags;/* IOMAP_F_* */ -struct inode *io_inode;/* file being written to */ -size_t io_size;/* size of the extent */ -xfs_off_t io_offset;/* offset in the file */ -void *io_private;/* file system private data */ -struct bio *io_bio;/* bio being built */ -struct bio io_inline_bio;/* MUST BE LAST! */ -}; -extern const struct address_space_operations xfs_address_space_operations;extern const struct address_space_operations xfs_dax_aops;
fs/xfs/xfs_super.c +1 -10
@@ -40,7 +40,6 @@#include <linux/parser.h> static const struct super_operations xfs_super_operations; -struct bio_set xfs_ioend_bioset;static struct kset *xfs_kset;/* top-level xfs sysfs dir */ #ifdef DEBUG
@@ -1853,15 +1852,10 @@ MODULE_ALIAS_FS("xfs"); STATIC int __initxfs_init_zones (void ) { -if (bioset_init (&xfs_ioend_bioset,4 * (PAGE_SIZE /SECTOR_SIZE ), - offsetof(struct xfs_ioend , io_inline_bio), -BIOSET_NEED_BVECS )) - goto out; - xfs_log_ticket_zone =kmem_zone_init (sizeof (xlog_ticket_t),"xfs_log_ticket" );if (!xfs_log_ticket_zone) - goto out_free_ioend_bioset; + goto out; xfs_bmap_free_item_zone = kmem_zone_init( sizeof(struct xfs_extent_free_item),
@@ -1996,8 +1990,6 @@ xfs_init_zones(void)kmem_zone_destroy (xfs_bmap_free_item_zone); out_destroy_log_ticket_zone:kmem_zone_destroy (xfs_log_ticket_zone); - out_free_ioend_bioset: -bioset_exit (&xfs_ioend_bioset); out:return -ENOMEM ; }
@@ -2028,7 +2020,6 @@ xfs_destroy_zones(void)kmem_zone_destroy (xfs_btree_cur_zone);kmem_zone_destroy (xfs_bmap_free_item_zone);kmem_zone_destroy (xfs_log_ticket_zone); -bioset_exit (&xfs_ioend_bioset); }STATIC int__init
fs/xfs/xfs_trace.h +0 -39
@@ -1158,45 +1158,6 @@ DEFINE_RW_EVENT(xfs_file_buffered_write);DEFINE_RW_EVENT (xfs_file_direct_write);DEFINE_RW_EVENT (xfs_file_dax_write); -DECLARE_EVENT_CLASS (xfs_page_class, - TP_PROTO(struct inode *inode, struct page *page, unsigned long off, - unsigned int len), -TP_ARGS (inode, page, off, len), -TP_STRUCT__entry ( - __field(dev_t, dev) - __field(xfs_ino_t, ino) - __field(pgoff_t, pgoff) - __field(loff_t, size) - __field(unsigned long, offset) - __field(unsigned int, length) - ), -TP_fast_assign ( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = XFS_I(inode)->i_ino; - __entry->pgoff = page_offset(page); - __entry->size = i_size_read(inode); - __entry->offset = off; - __entry->length = len; - ), -TP_printk ("dev %d:%d ino 0x%llx pgoff 0x%lx size 0x%llx offset %lx " -"length %x" , -MAJOR (__entry->dev),MINOR (__entry->dev), - __entry->ino, - __entry->pgoff, - __entry->size, - __entry->offset, - __entry->length) -) - -#define DEFINE_PAGE_EVENT (name) \ -DEFINE_EVENT(xfs_page_class, name, \ - TP_PROTO(struct inode *inode, struct page *page, unsigned long off, \ - unsigned int len), \ - TP_ARGS(inode, page, off, len)) -DEFINE_PAGE_EVENT (xfs_writepage); -DEFINE_PAGE_EVENT (xfs_releasepage); -DEFINE_PAGE_EVENT (xfs_invalidatepage); - DECLARE_EVENT_CLASS(xfs_imap_class, TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count, int whichfork, struct xfs_bmbt_irec *irec),
include/linux/iomap.h +59 -0
77 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 10/14] iomap: lift the xfs writeback code to iomap
Quoted 7 lines · Christoph Hellwig
Take the xfs writeback code and move it to fs/iomap. A new structure with three methods is added as the abstraction from the generic writeback code to the file system. These methods are used to map blocks, submit an ioend, and cancel a page that encountered an error before it was added to an ioend.
Signed-off-by Christoph Hellwig <[email protected]>
<snip>
Quoted 45 lines
include/linux/iomap.h +23
-0@@ -4,6 +4,7 @@#include <linux/atomic.h> #include <linux/bitmap.h> +#include <linux/blk_types.h> #include <linux/mm.h> #include <linux/types.h> #include <linux/mm_types.h> @@ -12,6 +13,7 @@struct address_space ;struct fiemap_extent_info ;struct inode ; +struct iomap_writepage_ctx ;struct iov_iter ;struct kiocb ;struct page ;@@ -185,6 +187,63 @@ loff_t iomap_seek_data(struct inode *inode, loff_t offset,sector_t iomap_bmap (struct address_space *mapping,sector_t bno,const struct iomap_ops *ops); +/* +* Structure for writeback I/O completions. +*/ +struct iomap_ioend { +struct list_head io_list;/* next ioend in chain */ +u16 io_type; +u16 io_flags;/* IOMAP_F_* */ +struct inode *io_inode;/* file being written to */ +size_t io_size;/* size of the extent */ +loff_t io_offset;/* offset in the file */ +void *io_private;/* file system private data */ +struct bio *io_bio;/* bio being built */ +struct bio io_inline_bio;/* MUST BE LAST! */ +}; + +struct iomap_writeback_ops { +/* +* Required, maps the blocks so that writeback can be performed on +* the range starting at offset. +*/ + int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode *inode,
include/linux/iomap.h loff_t iomap_seek_data(struct inode *inode, loff_t offset, · line 220
Quoted 10 lines
include/linux/iomap.h +10
-0+ loff_t offset); + +/* +* Optional, allows the file systems to perform actions just before +* submitting the bio and/or override the bio end_io handler for complex +* operations like copy on write extent manipulation or unwritten extent +* conversions. +*/ +int (*submit_ioend)(struct iomap_ioend *ioend,int status); +
Looks ok, but I reserve the right to rename this to ->prepare_ioend() or something. :)
--D
Quoted 32 lines
include/linux/iomap.h +26
-0+/* +* Optional, allows the file system to discard state on a page where +* we failed to submit any I/O. +*/ +void (*discard_page)(struct page *page); +}; + +struct iomap_writepage_ctx { +struct iomap iomap; +struct iomap_ioend *ioend; +const struct iomap_writeback_ops *ops; +}; + +void iomap_finish_ioends (struct iomap_ioend *ioend,int error); +void iomap_ioend_try_merge (struct iomap_ioend *ioend, +struct list_head *more_ioends, +void (*merge_private)(struct iomap_ioend *ioend, +struct iomap_ioend *next)); +void iomap_sort_ioends (struct list_head *ioend_list); +int iomap_writepage (struct page *page,struct writeback_control *wbc, +struct iomap_writepage_ctx *wpc, +const struct iomap_writeback_ops *ops); +int iomap_writepages (struct address_space *mapping, +struct writeback_control *wbc,struct iomap_writepage_ctx *wpc, +const struct iomap_writeback_ops *ops); +/* * Flags for direct I/O ->end_io: */ Signature · 1 line
2.20.1
Re: [PATCH 10/14] iomap: lift the xfs writeback code to iomap
In reply to Christoph HellwigQuoted 7 lines · Christoph Hellwig
Take the xfs writeback code and move it to fs/iomap. A new structure with three methods is added as the abstraction from the generic writeback code to the file system. These methods are used to map blocks, submit an ioend, and cancel a page that encountered an error before it was added to an ioend.
Signed-off-by Christoph Hellwig <[email protected]>
With Darrick's renaming of the .submit_ioend method, this looks fine.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 11/14 iomap: warn on inline maps in iomap_writepage_map
In reply to Christoph HellwigAnd inline mapping should never mark the page dirty and thus never end up in writepages. Add a check for that condition and warn if it happens.
--- fs/iomap/buffered-io.c | 2 ++ 1 file changed, 2 insertions(+)
fs/iomap/buffered-io.c +2 -0
@@ -1415,6 +1415,8 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, error = wpc->ops->map_blocks (wpc, inode, file_offset);if (error)break ; +if (WARN_ON_ONCE (wpc->iomap.type ==IOMAP_INLINE )) +continue ;if (wpc->iomap.type ==IOMAP_HOLE )continue ; iomap_add_to_ioend(inode, file_offset, page, iop, wpc, wbc,
Signature · 1 line
2.20.1
PATCH 12/14 iomap: move struct iomap_page out of iomap.h
In reply to Christoph HellwigNow that all the writepage code is in the iomap code there is no need to keep this structure public.
--- fs/iomap/buffered-io.c | 17 +++++++++++++++++ include/linux/iomap.h | 17 ----------------- 2 files changed, 17 insertions(+), 17 deletions(-)
fs/iomap/buffered-io.c +17 -0
@@ -21,6 +21,23 @@#include "../internal.h" +/* +* Structure allocated for each page when block size < PAGE_SIZE to track +* sub-page uptodate status and I/O completions. +*/ +struct iomap_page { +atomic_t read_count; +atomic_t write_count; + DECLARE_BITMAP(uptodate, PAGE_SIZE / 512); +}; + +static inline struct iomap_page *to_iomap_page (struct page *page) +{ +if (page_has_private (page)) +return (struct iomap_page *)page_private (page); +return NULL ; +} +static struct bio_set iomap_ioend_bioset; static struct iomap_page *
include/linux/iomap.h +0 -17
@@ -136,23 +136,6 @@ loff_t iomap_apply(struct inode *inode, loff_t pos, loff_t length, unsigned flags, const struct iomap_ops *ops, void *data, iomap_actor_t actor); -/* -* Structure allocate for each page when block size < PAGE_SIZE to track -* sub-page uptodate status and I/O completions. -*/ -struct iomap_page { -atomic_t read_count; -atomic_t write_count; - DECLARE_BITMAP(uptodate, PAGE_SIZE / 512); -}; - -static inline struct iomap_page *to_iomap_page (struct page *page) -{ -if (page_has_private (page)) -return (struct iomap_page *)page_private (page); -return NULL ; -} -ssize_t iomap_file_buffered_write (struct kiocb *iocb,struct iov_iter *from,const struct iomap_ops *ops);int iomap_readpage (struct page *page,const struct iomap_ops *ops);
Signature · 1 line
2.20.1
PATCH 13/14 iomap: cleanup iomap_ioend_compare
In reply to Christoph HellwigMove the initialization of ia and ib to the declaration line and remove a superflous else.
--- fs/iomap/buffered-io.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
fs/iomap/buffered-io.c +3 -4
@@ -1226,13 +1226,12 @@ EXPORT_SYMBOL_GPL(iomap_ioend_try_merge);static int iomap_ioend_compare (void *priv,struct list_head *a,struct list_head *b) { -struct iomap_ioend *ia, *ib; + struct iomap_ioend *ia = container_of(a, struct iomap_ioend, io_list); + struct iomap_ioend *ib = container_of(b, struct iomap_ioend, io_list); - ia = container_of(a, struct iomap_ioend, io_list); - ib = container_of(b, struct iomap_ioend, io_list);if (ia->io_offset < ib->io_offset)return -1 ; -else if (ia->io_offset > ib->io_offset) +if (ia->io_offset > ib->io_offset)return 1 ;return 0 ; }
Signature · 1 line
2.20.1
PATCH 14/14 iomap: pass a struct page to iomap_finish_page_writeback
In reply to Christoph HellwigNo need to pass the full bio_vec.
--- fs/iomap/buffered-io.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
fs/iomap/buffered-io.c +5 -5
@@ -1110,13 +1110,13 @@ vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops)EXPORT_SYMBOL_GPL (iomap_page_mkwrite);static void -iomap_finish_page_writeback (struct inode *inode,struct bio_vec *bvec, +iomap_finish_page_writeback (struct inode *inode,struct page *page,int error) { -struct iomap_page *iop =to_iomap_page (bvec->bv_page); +struct iomap_page *iop =to_iomap_page (page);if (error) { -SetPageError (bvec->bv_page); +SetPageError (page);mapping_set_error (inode->i_mapping, -EIO ); }
@@ -1124,7 +1124,7 @@ iomap_finish_page_writeback(struct inode *inode, struct bio_vec *bvec,WARN_ON_ONCE (iop &&atomic_read (&iop->write_count) <=0 );if (!iop ||atomic_dec_and_test (&iop->write_count)) -end_page_writeback (bvec->bv_page); +end_page_writeback (page); }/*
@@ -1156,7 +1156,7 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error)/* walk each page on bio, ending page IO on them */ bio_for_each_segment_all (bv, bio, iter_all) -iomap_finish_page_writeback (inode, bv, error); +iomap_finish_page_writeback (inode, bv->bv_page, error);bio_put (bio); }
Signature · 1 line
2.20.1
Re: [PATCH 14/14] iomap: pass a struct page to iomap_finish_page_writeback
Quoted 3 lines · Christoph Hellwig
No need to pass the full
bio_vec.Signed-off-by Christoph Hellwig <[email protected]>
Looks ok,
--D
Quoted 46 lines
--- fs/iomap/buffered-io.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/iomap/buffered-io.c +5
-5@@ -1110,13 +1110,13 @@ vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops)EXPORT_SYMBOL_GPL (iomap_page_mkwrite);static void -iomap_finish_page_writeback (struct inode *inode,struct bio_vec *bvec, +iomap_finish_page_writeback (struct inode *inode,struct page *page,int error) { -struct iomap_page *iop =to_iomap_page (bvec->bv_page); +struct iomap_page *iop =to_iomap_page (page);if (error) { -SetPageError (bvec->bv_page); +SetPageError (page);mapping_set_error (inode->i_mapping, -EIO ); }@@ -1124,7 +1124,7 @@ iomap_finish_page_writeback(struct inode *inode, struct bio_vec *bvec,WARN_ON_ONCE (iop &&atomic_read (&iop->write_count) <=0 );if (!iop ||atomic_dec_and_test (&iop->write_count)) -end_page_writeback (bvec->bv_page); +end_page_writeback (page); }/* @@ -1156,7 +1156,7 @@ iomap_finish_ioend(struct iomap_ioend *ioend, int error)/* walk each page on bio, ending page IO on them */ bio_for_each_segment_all (bv, bio, iter_all) -iomap_finish_page_writeback (inode, bv, error); +iomap_finish_page_writeback (inode, bv->bv_page, error);bio_put (bio); }Signature · 1 line
2.20.1
Quoted 7 lines · Christoph Hellwig
Hi Darrick,
this series cleans up the xfs writepage code and then lifts it to
fs/iomap/so that it could be use by other file system.Note: the
submit_ioendhook has not been renamed. Feel free to rename it if you really dislike the name.
Heh, ok. ->prepare_ioend it is.
Is a v3 of "iomap and xfs COW cleanups" coming soon?
--D
Quoted 41 lines
Changes since v7:
- fix various commit log typos
- fix up various comments
- move some of the iomap tracepoint additions to an earlier patch
- rebased on top of "iomap: iomap that extends beyond EOF should be marked dirty"Changes since v6:
- actually addtrace.cto the patch
- move back to the old order that massages XFS into shape and then lifts the code to iomap
- cleanupiomap_ioend_compare
- cleanup theadd_to_ioendchecksChanges since v5:
- move the tracing code to fs/iomap/trace.[ch]
- fix a bisection issue with the tracing code
- add an assert thatxfs_end_ionow only gets "complicated" completions
- better document theiomap_writeback_opsmethods iniomap.hChanges since v4:
- rebased on top 5.4-rc1
- drop the addition oflist_pop/list_pop_entry
- re-split a few patches to better fit Darricks scheme of keeping the iomap additions separate from the XFS switchoverChanges since v3:
- re-split the pages to add new code to iomap and then switch xfs to it later (Darrick)Changes since v2:
- rebased to v5.3-rc1
- folded in a few changes from the gfs2 enablement seriesChanges since v1:
- rebased to the latest xfs for-next tree
- keep the preallocated transactions for size updates
- renamelist_poptolist_pop_entryand related cleanups
- better document the nofs context handling
- document that the iomap tracepoints are not a stable API