Skip to contentSkip to conversation

iomap: iomap that extends beyond EOF should be marked dirty

PATCH Superseded Dave Chinner 4 participants 9 messages Oct 16, 2019 – Oct 17, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iomap: iomap that extends beyond EOF should be marked dirty

From: Dave Chinner <[email protected]>

When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.

However, we don't mark such iomaps as IOMAP_F_DIRTY to indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion when O_DSYNC writes are beingt used and the hardware supports FUA.

Hence when setting IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will call generic_write_sync() to flush the inode size update to stable storage correctly.

---
 fs/ext4/inode.c       | 9 ++++++++-
 fs/xfs/xfs_iomap.c    | 8 ++++++++
 include/linux/iomap.h | 2 ++
 3 files changed, 18 insertions(+), 1 deletion(-)
fs/ext4/inode.c +8 -1
index 516faa280ced..e9dc52537e5b 100644
@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
 			return ret;
 	}
 
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
 	iomap->flags = 0;
-	if (ext4_inode_datasync_dirty(inode))
+	if (ext4_inode_datasync_dirty(inode) ||
+	    offset + length > i_size_read(inode))
 		iomap->flags |= IOMAP_F_DIRTY;
+
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->dax_dev = sbi->s_daxdev;
 	iomap->offset = (u64)first_block << blkbits;
fs/xfs/xfs_iomap.c +8 -0
index f780e223b118..38be06f19ea2 100644
@@ -722,6 +722,14 @@ xfs_file_iomap_begin_delay(
 		xfs_trim_extent(&imap, cmap.br_startoff, cmap.br_blockcount);
 		shared = true;
 	}
+
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
+	if (offset + count > i_size_read(inode))
+		iomap->flags |= IOMAP_F_DIRTY;
 	error = xfs_bmbt_to_iomap(ip, iomap, &imap, shared);
 out_unlock:
 	xfs_iunlock(ip, XFS_ILOCK_EXCL);
include/linux/iomap.h +2 -0
index 7aa5d6117936..24bd227d59f9 100644
@@ -32,6 +32,8 @@ struct vm_fault;
  *
  * IOMAP_F_DIRTY indicates the inode has uncommitted metadata needed to access
  * written data and requires fdatasync to commit them to persistent storage.
+ * This needs to take into account metadata changes that *may* be made at IO
+ * completion, such as file size updates from direct IO.
  */
 #define IOMAP_F_NEW		0x01	/* blocks have been newly allocated */
 #define IOMAP_F_DIRTY		0x02	/* uncommitted metadata */
Signature · 1 line
2.23.0.rc1
#1 lore raw
Quoted 54 lines · Dave Chinner

From: Dave Chinner <[email protected]>

When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.

However, we don't mark such iomaps as IOMAP_F_DIRTY to indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion when O_DSYNC writes are beingt used and the hardware supports FUA.

Hence when setting IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will call generic_write_sync() to flush the inode size update to stable storage correctly.

---
 fs/ext4/inode.c       | 9 ++++++++-
 fs/xfs/xfs_iomap.c    | 8 ++++++++
 include/linux/iomap.h | 2 ++
 3 files changed, 18 insertions(+), 1 deletion(-)
fs/ext4/inode.c +8 -1
index 516faa280ced..e9dc52537e5b 100644
@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
 			return ret;
 	}
 
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
 	iomap->flags = 0;
-	if (ext4_inode_datasync_dirty(inode))
+	if (ext4_inode_datasync_dirty(inode) ||
+	    offset + length > i_size_read(inode))
 		iomap->flags |= IOMAP_F_DIRTY;
+
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->dax_dev = sbi->s_daxdev;
 	iomap->offset = (u64)first_block << blkbits;
fs/xfs/xfs_iomap.c +1 -0
index f780e223b118..38be06f19ea2 100644
@@ -722,6 +722,14 @@ xfs_file_iomap_begin_delay(
 		xfs_trim_extent(&imap, cmap.br_startoff, cmap.br_blockcount);
 		shared = true;
 	}
+

fs/xfs/xfs_iomap.c xfs_file_iomap_begin_delay( · line 735

Quoted 10 lines
fs/xfs/xfs_iomap.c +7 -0
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
+	if (offset + count > i_size_read(inode))
+		iomap->flags |= IOMAP_F_DIRTY;
 	error = xfs_bmbt_to_iomap(ip, iomap, &imap, shared);
 out_unlock:
 	xfs_iunlock(ip, XFS_ILOCK_EXCL);

Urk, self-nack.

I put this in the wrong function. Too much spaghetti that all looks similar in the XFS iomap code now. Will repost in a minute.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
In reply to Dave Chinner
Quoted 51 lines · Dave Chinner

From: Dave Chinner <[email protected]>

When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.

However, we don't mark such iomaps as IOMAP_F_DIRTY to indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion when O_DSYNC writes are beingt used and the hardware supports FUA.

Hence when setting IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will call generic_write_sync() to flush the inode size update to stable storage correctly.

---
 fs/ext4/inode.c       | 9 ++++++++-
 fs/xfs/xfs_iomap.c    | 8 ++++++++
 include/linux/iomap.h | 2 ++
 3 files changed, 18 insertions(+), 1 deletion(-)
fs/ext4/inode.c +8 -1
index 516faa280ced..e9dc52537e5b 100644
@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
 			return ret;
 	}
 
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
 	iomap->flags = 0;
-	if (ext4_inode_datasync_dirty(inode))
+	if (ext4_inode_datasync_dirty(inode) ||
+	    offset + length > i_size_read(inode))
 		iomap->flags |= IOMAP_F_DIRTY;
+
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->dax_dev = sbi->s_daxdev;
 	iomap->offset = (u64)first_block << blkbits;
fs/xfs/xfs_iomap.c +0 -0
index f780e223b118..38be06f19ea2 100644
@@ -722,6 +722,14 @@ xfs_file_iomap_begin_delay(
 		xfs_trim_extent(&imap, cmap.br_startoff, cmap.br_blockcount);

fs/xfs/xfs_iomap.c xfs_file_iomap_begin_delay( · line 732

Quoted 10 lines
fs/xfs/xfs_iomap.c +8 -0
 		shared = true;
 	}
+
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
+	if (offset + count > i_size_read(inode))
+		iomap->flags |= IOMAP_F_DIRTY;

This ought to be in xfs_direct_write_iomap_begin(), right?

(Hoping to see another rev of Christoph's iomap cleanup series... ;))

--D

Quoted 19 lines
fs/xfs/xfs_iomap.c +0 -0
 	error = xfs_bmbt_to_iomap(ip, iomap, &imap, shared);
 out_unlock:
 	xfs_iunlock(ip, XFS_ILOCK_EXCL);
include/linux/iomap.h +2 -0
index 7aa5d6117936..24bd227d59f9 100644
@@ -32,6 +32,8 @@ struct vm_fault;
  *
  * IOMAP_F_DIRTY indicates the inode has uncommitted metadata needed to access
  * written data and requires fdatasync to commit them to persistent storage.
+ * This needs to take into account metadata changes that *may* be made at IO
+ * completion, such as file size updates from direct IO.
  */
 #define IOMAP_F_NEW		0x01	/* blocks have been newly allocated */
 #define IOMAP_F_DIRTY		0x02	/* uncommitted metadata */
Signature · 1 line
2.23.0.rc1
#3 lore raw
Quoted 11 lines · Darrick J. Wong

fs/xfs/xfs_iomap.c xfs_file_iomap_begin_delay( · line 732

Quoted 7 lines
fs/xfs/xfs_iomap.c +7 -0
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
+	if (offset + count > i_size_read(inode))
+		iomap->flags |= IOMAP_F_DIRTY;

This ought to be in xfs_direct_write_iomap_begin(), right?

(Hoping to see another rev of Christoph's iomap cleanup series... ;))

I need to finish off all the nitpicks on the first iomap series..

Also we'll want this patch in first as it is 5.4 / -stable material, so I'll need to rebase on top of that as well.

#4 lore raw

PATCH v2 1/1 iomap: iomap that extends beyond EOF should be marked dirty

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

When doing a direct IO that spans the current EOF, and there are written blocks beyond EOF that extend beyond the current write, the only metadata update that needs to be done is a file size extension.

However, we don't mark such iomaps as IOMAP_F_DIRTY to indicate that there is IO completion metadata updates required, and hence we may fail to correctly sync file size extensions made in IO completion when O_DSYNC writes are being used and the hardware supports FUA.

Hence when setting IOMAP_F_DIRTY, we need to also take into account whether the iomap spans the current EOF. If it does, then we need to mark it dirty so that IO completion will call generic_write_sync() to flush the inode size update to stable storage correctly.

---
V2: put the XFS code in the right function, tested on non-FUA capable hardware, too.

 fs/ext4/inode.c       | 9 ++++++++-
 fs/xfs/xfs_iomap.c    | 7 +++++++
 include/linux/iomap.h | 2 ++
 3 files changed, 17 insertions(+), 1 deletion(-)
fs/ext4/inode.c +8 -1
index 516faa280ced..e9dc52537e5b 100644
@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
 			return ret;
 	}
 
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
 	iomap->flags = 0;
-	if (ext4_inode_datasync_dirty(inode))
+	if (ext4_inode_datasync_dirty(inode) ||
+	    offset + length > i_size_read(inode))
 		iomap->flags |= IOMAP_F_DIRTY;
+
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->dax_dev = sbi->s_daxdev;
 	iomap->offset = (u64)first_block << blkbits;
fs/xfs/xfs_iomap.c +7 -0
index f780e223b118..32993c2acbd9 100644
@@ -1049,6 +1049,13 @@ xfs_file_iomap_begin(
 	trace_xfs_iomap_alloc(ip, offset, length, XFS_DATA_FORK, &imap);
 
 out_finish:
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes pending or have been made here.
+	 */
+	if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode))
+		iomap->flags |= IOMAP_F_DIRTY;
 	return xfs_bmbt_to_iomap(ip, iomap, &imap, shared);
 
 out_found:
include/linux/iomap.h +2 -0
index 7aa5d6117936..24bd227d59f9 100644
@@ -32,6 +32,8 @@ struct vm_fault;
  *
  * IOMAP_F_DIRTY indicates the inode has uncommitted metadata needed to access
  * written data and requires fdatasync to commit them to persistent storage.
+ * This needs to take into account metadata changes that *may* be made at IO
+ * completion, such as file size updates from direct IO.
  */
 #define IOMAP_F_NEW		0x01	/* blocks have been newly allocated */
 #define IOMAP_F_DIRTY		0x02	/* uncommitted metadata */
maintainer
#6 lore raw
In reply to Dave Chinner
Quoted 12 lines · Dave Chinner
fs/ext4/inode.c +4 -0
index 516faa280ced..e9dc52537e5b 100644
@@ -3523,9 +3523,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
 			return ret;
 	}
 
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.

fs/ext4/inode.c static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, · line 3538

Quoted 10 lines
fs/ext4/inode.c +4 -1
+	 */
 	iomap->flags = 0;
-	if (ext4_inode_datasync_dirty(inode))
+	if (ext4_inode_datasync_dirty(inode) ||
+	    offset + length > i_size_read(inode))
 		iomap->flags |= IOMAP_F_DIRTY;
+
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->dax_dev = sbi->s_daxdev;
 	iomap->offset = (u64)first_block << blkbits;

Ext4 is not currently using iomap for any kind of writing right now, so perhaps this should land via Matthew's patchset?

- Ted

#7 lore raw
Quoted 9 lines · Theodore Y. Ts'o
Quoted 9 lines
fs/ext4/inode.c +7 -1
+	/*
+	 * Writes that span EOF might trigger an IO size update on completion,
+	 * so consider them to be dirty for the purposes of O_DSYNC even if
+	 * there is no other metadata changes being made or are pending here.
+	 */
 	iomap->flags = 0;
-	if (ext4_inode_datasync_dirty(inode))
+	if (ext4_inode_datasync_dirty(inode) ||
+	    offset + length > i_size_read(inode))
Quoted 8 lines

fs/ext4/inode.c static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, · line 3538

Quoted 5 lines
fs/ext4/inode.c +1 -0
 		iomap->flags |= IOMAP_F_DIRTY;
+
 	iomap->bdev = inode->i_sb->s_bdev;
 	iomap->dax_dev = sbi->s_daxdev;
 	iomap->offset = (u64)first_block << blkbits;

Ext4 is not currently using iomap for any kind of writing right now, so perhaps this should land via Matthew's patchset?

It does for DAX, which is one of the consumers of IOMAP_F_DIRTY.

maintainer
#8 lore raw
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines

Ext4 is not currently using iomap for any kind of writing right now, so perhaps this should land via Matthew's patchset?

It does for DAX, which is one of the consumers of IOMAP_F_DIRTY.

Ah, right, I had forgotten about DAX.

- Ted