avoid taking the iolock in fsync unless actually needed v2
Hi all,
this series avoids taking the iolock in fsync if there is no dirty metadata.
Changes since v1:
- add a comment explaining the ipincount check
PATCH 1/2 xfs: refactor xfs_file_fsync
Factor out the log syncing logic into two helpers to make the code easier to read and more maintainable.
--- fs/xfs/xfs_file.c | 81 +++++++++++++++++++++++++++++------------------ 1 file changed, 50 insertions(+), 31 deletions(-)
fs/xfs/xfs_file.c +50 -31
@@ -118,6 +118,54 @@ xfs_dir_fsync(return xfs_log_force_inode (ip); } +static xfs_lsn_t +xfs_fsync_lsn ( +struct xfs_inode *ip, +bool datasync) +{ +if (!xfs_ipincount (ip)) +return 0 ; +if (datasync && !(ip->i_itemp->ili_fsync_fields & ~XFS_ILOG_TIMESTAMP )) +return 0 ; +return ip->i_itemp->ili_last_lsn; +} + +/* +* All metadata updates are logged, which means that we just have to flush the +* log up to the latest LSN that touched the inode. +* +* If we have concurrent fsync/fdatasync() calls, we need them to all block on +* the log force before we clear the ili_fsync_fields field. This ensures that +* we don't get a racing sync operation that does not wait for the metadata to +* hit the journal before returning. If we race with clearing ili_fsync_fields, +* then all that will happen is the log force will do nothing as the lsn will +* already be on disk. We can't race with setting ili_fsync_fields because that +* is done under XFS_ILOCK_EXCL, and that can't happen because we hold the lock +* shared until after the ili_fsync_fields is cleared. +*/ +static int +xfs_fsync_flush_log ( +struct xfs_inode *ip, +bool datasync, +int *log_flushed) +{ +int error =0 ; +xfs_lsn_t lsn; + +xfs_ilock (ip,XFS_ILOCK_SHARED ); + lsn =xfs_fsync_lsn (ip, datasync); +if (lsn) { + error =xfs_log_force_lsn (ip->i_mount, lsn,XFS_LOG_SYNC , + log_flushed); + +spin_lock (&ip->i_itemp->ili_lock); + ip->i_itemp->ili_fsync_fields =0 ; +spin_unlock (&ip->i_itemp->ili_lock); + } +xfs_iunlock (ip,XFS_ILOCK_SHARED ); +return error; +} + STATIC int xfs_file_fsync( struct file *file,
@@ -125,13 +173,10 @@ xfs_file_fsync( loff_t end, int datasync) { -struct inode *inode = file->f_mapping->host; -struct xfs_inode *ip =XFS_I (inode); -struct xfs_inode_log_item *iip = ip->i_itemp; +struct xfs_inode *ip =XFS_I (file->f_mapping->host);struct xfs_mount *mp = ip->i_mount;int error =0 ;int log_flushed =0 ; -xfs_lsn_t lsn =0 ;trace_xfs_file_fsync (ip);
@@ -155,33 +200,7 @@ xfs_file_fsync(else if (mp->m_logdev_targp != mp->m_ddev_targp)xfs_blkdev_issue_flush (mp->m_ddev_targp); -/* -* All metadata updates are logged, which means that we just have to -* flush the log up to the latest LSN that touched the inode. If we have -* concurrent fsync/fdatasync() calls, we need them to all block on the -* log force before we clear the ili_fsync_fields field. This ensures -* that we don't get a racing sync operation that does not wait for the -* metadata to hit the journal before returning. If we race with -* clearing the ili_fsync_fields, then all that will happen is the log -* force will do nothing as the lsn will already be on disk. We can't -* race with setting ili_fsync_fields because that is done under -* XFS_ILOCK_EXCL, and that can't happen because we hold the lock shared -* until after the ili_fsync_fields is cleared. -*/ -xfs_ilock (ip,XFS_ILOCK_SHARED ); -if (xfs_ipincount (ip)) { -if (!datasync || - (iip->ili_fsync_fields & ~XFS_ILOG_TIMESTAMP )) - lsn = iip->ili_last_lsn; - } - -if (lsn) { - error =xfs_log_force_lsn (mp, lsn,XFS_LOG_SYNC , &log_flushed); -spin_lock (&iip->ili_lock); - iip->ili_fsync_fields =0 ; -spin_unlock (&iip->ili_lock); - } -xfs_iunlock (ip,XFS_ILOCK_SHARED ); + error =xfs_fsync_flush_log (ip, datasync, &log_flushed);/* * If we only have a single device, and the log force about was
Signature · 1 line
2.29.2
Re: [PATCH 1/2] xfs: refactor xfs_file_fsync
Quoted 5 lines · Christoph Hellwig
Factor out the log syncing logic into two helpers to make the code easier to read and more maintainable.
Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Brian Foster <[email protected]>
LGTM.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 2/2 xfs: reduce ilock acquisitions in xfs_file_fsync
In reply to Christoph HellwigIf the inode is not pinned by the time fsync is called we don't need the ilock to protect against concurrent clearing of ili_fsync_fields as the inode won't need a log flush or clearing of these fields. Not taking the iolock allows for full concurrency of fsync and thus O_DSYNC completions with io_uring/aio write submissions.
--- fs/xfs/xfs_file.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
fs/xfs/xfs_file.c +8 -1
@@ -200,7 +200,14 @@ xfs_file_fsync(else if (mp->m_logdev_targp != mp->m_ddev_targp)xfs_blkdev_issue_flush (mp->m_ddev_targp); - error =xfs_fsync_flush_log (ip, datasync, &log_flushed); +/* +* Any inode that has dirty modifications in the log is pinned. The +* racy check here for a pinned inode while not catch modifications +* that happen concurrently to the fsync call, but fsync semantics +* only require to sync previously completed I/O. +*/ +if (xfs_ipincount (ip)) + error =xfs_fsync_flush_log (ip, datasync, &log_flushed);/* * If we only have a single device, and the log force about was
Signature · 1 line
2.29.2
Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync
Quoted 7 lines · Christoph Hellwig
If the inode is not pinned by the time fsync is called we don't need the ilock to protect against concurrent clearing of
ili_fsync_fieldsas the inode won't need a log flush or clearing of these fields. Not taking the iolock allows for full concurrency of fsync and thusO_DSYNCcompletions withio_uring/aiowrite submissions.Signed-off-by Christoph Hellwig <[email protected]>
Code looks good, so
But it makes me wonder...
That is, we already elide the call to generic_write_sync() in direct IO in the case that the device supports FUA and it's a pure overwrite with no dirty metadata on the inode. Hence for a lot of storage and AIO/io_uring+DIO w/ O_DSYNC workloads we're already eliding this fsync-based lock cycle.
In the case where we can't do a REQ_FUA IO because it is not supported by the device, then don't we really only need a cache flush at IO completion rather than the full generic_write_sync() call path? That would provide this optimisation to all the filesystems using iomap_dio_rw(), not just XFS....
In fact, I wonder if we need to do anything other than just use REQ_FUA unconditionally in iomap for this situation, as the block layer will translate REQ_FUA to a write+post-flush if the device doesn't support FUA writes directly.
You're thoughts on that, Christoph?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync
Quoted 12 lines · Dave Chinner
That is, we already elide the call to
generic_write_sync()in direct IO in the case that the device supports FUA and it's a pure overwrite with no dirty metadata on the inode. Hence for a lot of storage and AIO/io_uring+DIO w/O_DSYNCworkloads we're already eliding this fsync-based lock cycle.In the case where we can't do a
REQ_FUAIO because it is not supported by the device, then don't we really only need a cache flush at IO completion rather than the fullgeneric_write_sync()call path? That would provide this optimisation to all the filesystems usingiomap_dio_rw(), not just XFS....
Quoted 6 lines
In fact, I wonder if we need to do anything other than just use
REQ_FUAunconditionally in iomap for this situation, as the block layer will translateREQ_FUAto a write+post-flush if the device doesn't support FUA writes directly.You're thoughts on that, Christoph?
For the pure overwrite O_DIRECT + O_DSYNC case we'd get away with just a flush. And using REQ_FUA will get us there, so it might be worth a try.
Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines
In fact, I wonder if we need to do anything other than just use
REQ_FUAunconditionally in iomap for this situation, as the block layer will translateREQ_FUAto a write+post-flush if the device doesn't support FUA writes directly.You're thoughts on that, Christoph?
For the pure overwrite
O_DIRECT+O_DSYNCcase we'd get away with just a flush. And usingREQ_FUAwill get us there, so it might be worth a try.
And looking at this a little more, while just using REQ_FUA would work it would be rather suboptimal for many cases, as the block layer flush state machine would do a flush for every bio. So for each O_DIRECT + O_DSYNC write that generates more than one bio we'd grow extra flushes.
Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync
In reply to Christoph HellwigQuoted 16 lines · Christoph Hellwig
If the inode is not pinned by the time fsync is called we don't need the ilock to protect against concurrent clearing of
ili_fsync_fieldsas the inode won't need a log flush or clearing of these fields. Not taking the iolock allows for full concurrency of fsync and thusO_DSYNCcompletions withio_uring/aiowrite submissions.Signed-off-by Christoph Hellwig <[email protected]>--- fs/xfs/xfs_file.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)fs/xfs/xfs_file.c +0
-0
fs/xfs/xfs_file.c xfs_file_fsync( · line 205
Quoted 7 lines
fs/xfs/xfs_file.c +3
-1@@ -200,7 +200,14 @@ xfs_file_fsync(else if (mp->m_logdev_targp != mp->m_ddev_targp)xfs_blkdev_issue_flush (mp->m_ddev_targp); - error =xfs_fsync_flush_log (ip, datasync, &log_flushed); +/* +* Any inode that has dirty modifications in the log is pinned. The +* racy check here for a pinned inode while not catch modifications
s/while/will/ ?
Otherwise looks good:
Quoted 11 lines
fs/xfs/xfs_file.c +5
-0+* that happen concurrently to the fsync call, but fsync semantics +* only require to sync previously completed I/O. +*/ +if (xfs_ipincount (ip)) + error =xfs_fsync_flush_log (ip, datasync, &log_flushed);/* * If we only have a single device, and the log force about was Signature · 1 line
2.29.2
Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync
Quoted 6 lines · Brian Foster
fs/xfs/xfs_file.c xfs_file_fsync( · line 205
Quoted 4 lines
fs/xfs/xfs_file.c +3
-1- error =xfs_fsync_flush_log (ip, datasync, &log_flushed); +/* +* Any inode that has dirty modifications in the log is pinned. The +* racy check here for a pinned inode while not catch modifications s/while/will/ ?
Yes. Darrick, can you fix this up when applying the patch, or do you want me to resend?