Skip to contentSkip to conversation

avoid taking the iolock in fsync unless actually needed v2

PATCH In review Christoph Hellwig 3 participants 9 messages Jan 22, 2021 – Jan 28, 2021 XFS filesystem

lore
Share & tools
lore
#0 lore raw

Hi all,

this series avoids taking the iolock in fsync if there is no dirty metadata.

Changes since v1:
- add a comment explaining the ipincount check

#1 lore raw

PATCH 1/2 xfs: refactor xfs_file_fsync

Factor out the log syncing logic into two helpers to make the code easier to read and more maintainable.

---
 fs/xfs/xfs_file.c | 81 +++++++++++++++++++++++++++++------------------
 1 file changed, 50 insertions(+), 31 deletions(-)
fs/xfs/xfs_file.c +50 -31
index 39695b59dfcc92..588232c77f11e0 100644
@@ -118,6 +118,54 @@ xfs_dir_fsync(
 	return xfs_log_force_inode(ip);
 }
 
+static xfs_lsn_t
+xfs_fsync_lsn(
+	struct xfs_inode	*ip,
+	bool			datasync)
+{
+	if (!xfs_ipincount(ip))
+		return 0;
+	if (datasync && !(ip->i_itemp->ili_fsync_fields & ~XFS_ILOG_TIMESTAMP))
+		return 0;
+	return ip->i_itemp->ili_last_lsn;
+}
+
+/*
+ * All metadata updates are logged, which means that we just have to flush the
+ * log up to the latest LSN that touched the inode.
+ *
+ * If we have concurrent fsync/fdatasync() calls, we need them to all block on
+ * the log force before we clear the ili_fsync_fields field. This ensures that
+ * we don't get a racing sync operation that does not wait for the metadata to
+ * hit the journal before returning.  If we race with clearing ili_fsync_fields,
+ * then all that will happen is the log force will do nothing as the lsn will
+ * already be on disk.  We can't race with setting ili_fsync_fields because that
+ * is done under XFS_ILOCK_EXCL, and that can't happen because we hold the lock
+ * shared until after the ili_fsync_fields is cleared.
+ */
+static  int
+xfs_fsync_flush_log(
+	struct xfs_inode	*ip,
+	bool			datasync,
+	int			*log_flushed)
+{
+	int			error = 0;
+	xfs_lsn_t		lsn;
+
+	xfs_ilock(ip, XFS_ILOCK_SHARED);
+	lsn = xfs_fsync_lsn(ip, datasync);
+	if (lsn) {
+		error = xfs_log_force_lsn(ip->i_mount, lsn, XFS_LOG_SYNC,
+					  log_flushed);
+
+		spin_lock(&ip->i_itemp->ili_lock);
+		ip->i_itemp->ili_fsync_fields = 0;
+		spin_unlock(&ip->i_itemp->ili_lock);
+	}
+	xfs_iunlock(ip, XFS_ILOCK_SHARED);
+	return error;
+}
+
 STATIC int
 xfs_file_fsync(
 	struct file		*file,
@@ -125,13 +173,10 @@ xfs_file_fsync(
 	loff_t			end,
 	int			datasync)
 {
-	struct inode		*inode = file->f_mapping->host;
-	struct xfs_inode	*ip = XFS_I(inode);
-	struct xfs_inode_log_item *iip = ip->i_itemp;
+	struct xfs_inode	*ip = XFS_I(file->f_mapping->host);
 	struct xfs_mount	*mp = ip->i_mount;
 	int			error = 0;
 	int			log_flushed = 0;
-	xfs_lsn_t		lsn = 0;
 
 	trace_xfs_file_fsync(ip);
 
@@ -155,33 +200,7 @@ xfs_file_fsync(
 	else if (mp->m_logdev_targp != mp->m_ddev_targp)
 		xfs_blkdev_issue_flush(mp->m_ddev_targp);
 
-	/*
-	 * All metadata updates are logged, which means that we just have to
-	 * flush the log up to the latest LSN that touched the inode. If we have
-	 * concurrent fsync/fdatasync() calls, we need them to all block on the
-	 * log force before we clear the ili_fsync_fields field. This ensures
-	 * that we don't get a racing sync operation that does not wait for the
-	 * metadata to hit the journal before returning. If we race with
-	 * clearing the ili_fsync_fields, then all that will happen is the log
-	 * force will do nothing as the lsn will already be on disk. We can't
-	 * race with setting ili_fsync_fields because that is done under
-	 * XFS_ILOCK_EXCL, and that can't happen because we hold the lock shared
-	 * until after the ili_fsync_fields is cleared.
-	 */
-	xfs_ilock(ip, XFS_ILOCK_SHARED);
-	if (xfs_ipincount(ip)) {
-		if (!datasync ||
-		    (iip->ili_fsync_fields & ~XFS_ILOG_TIMESTAMP))
-			lsn = iip->ili_last_lsn;
-	}
-
-	if (lsn) {
-		error = xfs_log_force_lsn(mp, lsn, XFS_LOG_SYNC, &log_flushed);
-		spin_lock(&iip->ili_lock);
-		iip->ili_fsync_fields = 0;
-		spin_unlock(&iip->ili_lock);
-	}
-	xfs_iunlock(ip, XFS_ILOCK_SHARED);
+	error = xfs_fsync_flush_log(ip, datasync, &log_flushed);
 
 	/*
 	 * If we only have a single device, and the log force about was
Signature · 1 line
2.29.2
#3 lore raw

PATCH 2/2 xfs: reduce ilock acquisitions in xfs_file_fsync

In reply to Christoph Hellwig

If the inode is not pinned by the time fsync is called we don't need the ilock to protect against concurrent clearing of ili_fsync_fields as the inode won't need a log flush or clearing of these fields. Not taking the iolock allows for full concurrency of fsync and thus O_DSYNC completions with io_uring/aio write submissions.

---
 fs/xfs/xfs_file.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
fs/xfs/xfs_file.c +8 -1
index 588232c77f11e0..ffe2d7c37e26cd 100644
@@ -200,7 +200,14 @@ xfs_file_fsync(
 	else if (mp->m_logdev_targp != mp->m_ddev_targp)
 		xfs_blkdev_issue_flush(mp->m_ddev_targp);
 
-	error = xfs_fsync_flush_log(ip, datasync, &log_flushed);
+	/*
+	 * Any inode that has dirty modifications in the log is pinned.  The
+	 * racy check here for a pinned inode while not catch modifications
+	 * that happen concurrently to the fsync call, but fsync semantics
+	 * only require to sync previously completed I/O.
+	 */
+	if (xfs_ipincount(ip))
+		error = xfs_fsync_flush_log(ip, datasync, &log_flushed);
 
 	/*
 	 * If we only have a single device, and the log force about was
Signature · 1 line
2.29.2
Reviewed-by on v1 2/2
#4 lore raw

Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync

Quoted 7 lines · Christoph Hellwig

If the inode is not pinned by the time fsync is called we don't need the ilock to protect against concurrent clearing of ili_fsync_fields as the inode won't need a log flush or clearing of these fields. Not taking the iolock allows for full concurrency of fsync and thus O_DSYNC completions with io_uring/aio write submissions.

Code looks good, so

But it makes me wonder...

That is, we already elide the call to generic_write_sync() in direct IO in the case that the device supports FUA and it's a pure overwrite with no dirty metadata on the inode. Hence for a lot of storage and AIO/io_uring+DIO w/ O_DSYNC workloads we're already eliding this fsync-based lock cycle.

In the case where we can't do a REQ_FUA IO because it is not supported by the device, then don't we really only need a cache flush at IO completion rather than the full generic_write_sync() call path? That would provide this optimisation to all the filesystems using iomap_dio_rw(), not just XFS....

In fact, I wonder if we need to do anything other than just use REQ_FUA unconditionally in iomap for this situation, as the block layer will translate REQ_FUA to a write+post-flush if the device doesn't support FUA writes directly.

You're thoughts on that, Christoph?

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#5 lore raw

Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync

Quoted 12 lines · Dave Chinner

That is, we already elide the call to generic_write_sync() in direct IO in the case that the device supports FUA and it's a pure overwrite with no dirty metadata on the inode. Hence for a lot of storage and AIO/io_uring+DIO w/ O_DSYNC workloads we're already eliding this fsync-based lock cycle.

In the case where we can't do a REQ_FUA IO because it is not supported by the device, then don't we really only need a cache flush at IO completion rather than the full generic_write_sync() call path? That would provide this optimisation to all the filesystems using iomap_dio_rw(), not just XFS....

Quoted 6 lines

In fact, I wonder if we need to do anything other than just use REQ_FUA unconditionally in iomap for this situation, as the block layer will translate REQ_FUA to a write+post-flush if the device doesn't support FUA writes directly.

You're thoughts on that, Christoph?

For the pure overwrite O_DIRECT + O_DSYNC case we'd get away with just a flush. And using REQ_FUA will get us there, so it might be worth a try.

#6 lore raw

Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync

Quoted 10 lines · Christoph Hellwig
Quoted 6 lines

In fact, I wonder if we need to do anything other than just use REQ_FUA unconditionally in iomap for this situation, as the block layer will translate REQ_FUA to a write+post-flush if the device doesn't support FUA writes directly.

You're thoughts on that, Christoph?

For the pure overwrite O_DIRECT + O_DSYNC case we'd get away with just a flush. And using REQ_FUA will get us there, so it might be worth a try.

And looking at this a little more, while just using REQ_FUA would work it would be rather suboptimal for many cases, as the block layer flush state machine would do a flush for every bio. So for each O_DIRECT + O_DSYNC write that generates more than one bio we'd grow extra flushes.

Reviewed-by on v1 2/2
#7 lore raw

Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync

In reply to Christoph Hellwig
Quoted 16 lines · Christoph Hellwig

If the inode is not pinned by the time fsync is called we don't need the ilock to protect against concurrent clearing of ili_fsync_fields as the inode won't need a log flush or clearing of these fields. Not taking the iolock allows for full concurrency of fsync and thus O_DSYNC completions with io_uring/aio write submissions.

---
 fs/xfs/xfs_file.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
fs/xfs/xfs_file.c +0 -0
index 588232c77f11e0..ffe2d7c37e26cd 100644

fs/xfs/xfs_file.c xfs_file_fsync( · line 205

Quoted 7 lines
fs/xfs/xfs_file.c +3 -1
@@ -200,7 +200,14 @@ xfs_file_fsync(
 	else if (mp->m_logdev_targp != mp->m_ddev_targp)
 		xfs_blkdev_issue_flush(mp->m_ddev_targp);
 
-	error = xfs_fsync_flush_log(ip, datasync, &log_flushed);
+	/*
+	 * Any inode that has dirty modifications in the log is pinned.  The
+	 * racy check here for a pinned inode while not catch modifications

s/while/will/ ?

Otherwise looks good:

Quoted 11 lines
fs/xfs/xfs_file.c +5 -0
+	 * that happen concurrently to the fsync call, but fsync semantics
+	 * only require to sync previously completed I/O.
+	 */
+	if (xfs_ipincount(ip))
+		error = xfs_fsync_flush_log(ip, datasync, &log_flushed);
 
 	/*
 	 * If we only have a single device, and the log force about was
Signature · 1 line
2.29.2
#8 lore raw

Re: [PATCH 2/2] xfs: reduce ilock acquisitions in xfs_file_fsync

Quoted 6 lines · Brian Foster

fs/xfs/xfs_file.c xfs_file_fsync( · line 205

Quoted 4 lines
fs/xfs/xfs_file.c +3 -1
-	error = xfs_fsync_flush_log(ip, datasync, &log_flushed);
+	/*
+	 * Any inode that has dirty modifications in the log is pinned.  The
+	 * racy check here for a pinned inode while not catch modifications

s/while/will/ ?

Yes. Darrick, can you fix this up when applying the patch, or do you want me to resend?