iomap write invalidation v2
Subsystems (7)
Changes since v1:
- use -ENOTBLK for the direct to buffered I/O fallback everywhere
- document the choice of -ENOTBLK better
- update a comment in XFS
PATCH 1/3 xfs: use ENOTBLK for direct I/O to buffered I/O fallback
This is what the classic fs/direct-io.c implementation and thuse other file systems use.
--- fs/xfs/xfs_file.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/xfs/xfs_file.c +2 -2
@@ -505,7 +505,7 @@ xfs_file_dio_aio_write(*/ if (xfs_is_cow_inode (ip)) {trace_xfs_reflink_bounce_dio_write (ip, iocb->ki_pos, count); -return -EREMCHG ; +return -ENOTBLK ; } iolock =XFS_IOLOCK_EXCL ; }else {
@@ -714,7 +714,7 @@ xfs_file_write_iter(* allow an operation to fall back to buffered mode. */ ret =xfs_file_dio_aio_write (iocb, from); -if (ret != -EREMCHG ) +if (ret != -ENOTBLK )return ret; }
Signature · 1 line
2.27.0
Re: [PATCH 1/3] xfs: use ENOTBLK for direct I/O to buffered I/O fallback
Quoted 4 lines · Christoph Hellwig
This is what the classic
fs/direct-io.cimplementation and thuse other file systems use.Signed-off-by Christoph Hellwig <[email protected]>
Looks ok to me,
--D
Quoted 29 lines
--- fs/xfs/xfs_file.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)fs/xfs/xfs_file.c +2
-2@@ -505,7 +505,7 @@ xfs_file_dio_aio_write(*/ if (xfs_is_cow_inode (ip)) {trace_xfs_reflink_bounce_dio_write (ip, iocb->ki_pos, count); -return -EREMCHG ; +return -ENOTBLK ; } iolock =XFS_IOLOCK_EXCL ; }else {@@ -714,7 +714,7 @@ xfs_file_write_iter(* allow an operation to fall back to buffered mode. */ ret =xfs_file_dio_aio_write (iocb, from); -if (ret != -EREMCHG ) +if (ret != -ENOTBLK )return ret; }Signature · 1 line
2.27.0
PATCH 2/3 iomap: Only invalidate page cache pages on direct IO writes
In reply to Christoph HellwigFrom: Dave Chinner <[email protected]>
The historic requirement for XFS to invalidate cached pages on direct IO reads has been lost in the twisty pages of history - it was inherited from Irix, which implemented page cache invalidation on read as a method of working around problems synchronising page cache state with uncached IO.
XFS has carried this ever since. In the initial linux ports it was necessary to get mmap and DIO to play "ok" together and not immediately corrupt data. This was the state of play until the linux kernel had infrastructure to track unwritten extents and synchronise page faults with allocations and unwritten extent conversions (->page_mkwrite infrastructure). IOws, the page cache invalidation on DIO read was necessary to prevent trivial data corruptions. This didn't solve all the problems, though.
There were peformance problems if we didn't invalidate the entire page cache over the file on read - we couldn't easily determine if the cached pages were over the range of the IO, and invalidation required taking a serialising lock (i_mutex) on the inode. This serialising lock was an issue for XFS, as it was the only exclusive lock in the direct Io read path.
Hence if there were any cached pages, we'd just invalidate the entire file in one go so that subsequent IOs didn't need to take the serialising lock. This was a problem that prevented ranged invalidation from being particularly useful for avoiding the remaining coherency issues. This was solved with the conversion of i_mutex to i_rwsem and the conversion of the XFS inode IO lock to use i_rwsem. Hence we could now just do ranged invalidation and the performance problem went away.
However, page cache invalidation was still needed to serialise sub-page/sub-block zeroing via direct IO against buffered IO because bufferhead state attached to the cached page could get out of whack when direct IOs were issued. We've removed bufferheads from the XFS code, and we don't carry any extent state on the cached pages anymore, and so this problem has gone away, too.
IOWs, it would appear that we don't have any good reason to be invalidating the page cache on DIO reads anymore. Hence remove the invalidation on read because it is unnecessary overhead, not needed to maintain coherency between mmap/buffered access and direct IO anymore, and prevents anyone from using direct IO reads from intentionally invalidating the page cache of a file.
--- fs/iomap/direct-io.c | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-)
fs/iomap/direct-io.c +15 -16
@@ -475,23 +475,22 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (ret) goto out_free_dio; -/* -* Try to invalidate cache pages for the range we're direct -* writing. If this invalidation fails, tough, the write will -* still work, but racing two incompatible write paths is a -* pretty crazy thing to do, so we don't support it 100%. -*/ - ret =invalidate_inode_pages2_range (mapping, - pos >>PAGE_SHIFT , end >>PAGE_SHIFT ); -if (ret) -dio_warn_stale_pagecache (iocb->ki_filp); - ret =0 ; +if (iov_iter_rw (iter) ==WRITE ) { +/* +* Try to invalidate cache pages for the range we are writing. +* If this invalidation fails, tough, the write will still work, +* but racing two incompatible write paths is a pretty crazy +* thing to do, so we don't support it 100%. +*/ +if (invalidate_inode_pages2_range (mapping, pos >>PAGE_SHIFT , + end >>PAGE_SHIFT )) +dio_warn_stale_pagecache (iocb->ki_filp); -if (iov_iter_rw (iter) ==WRITE && !wait_for_completion && - !inode->i_sb->s_dio_done_wq) { - ret =sb_init_dio_done_wq (inode->i_sb); -if (ret <0 ) - goto out_free_dio; +if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { + ret =sb_init_dio_done_wq (inode->i_sb); +if (ret <0 ) + goto out_free_dio; + } }inode_dio_begin (inode);
Signature · 1 line
2.27.0
PATCH 3/3 iomap: fall back to buffered writes for invalidation failures
In reply to Christoph HellwigFailing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)
fs/ext4/file.c +2 -0
@@ -544,6 +544,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret =iomap_dio_rw (iocb, from, iomap_ops, &ext4_dio_write_ops,is_sync_kiocb (iocb) || unaligned_io || extend); +if (ret == -ENOTBLK ) + ret =0 ;if (extend) ret =ext4_handle_inode_extension (inode, offset, ret, count);
fs/gfs2/file.c +2 -1
@@ -835,7 +835,8 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from) ret =iomap_dio_rw (iocb, from, &gfs2_iomap_ops,NULL ,is_sync_kiocb (iocb)); - +if (ret == -ENOTBLK ) + ret =0 ; out:gfs2_glock_dq (&gh); out_uninit:
fs/iomap/direct-io.c +11 -5
@@ -10,6 +10,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "trace.h" #include "../internal.h"
@@ -401,6 +402,9 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,* can be mapped into multiple disjoint IOs and only a subset of the IOs issued * may be pure data writes. In that case, we still need to do a full data sync * completion. +* +* Returns -ENOTBLK In case of a page invalidation invalidation failure for +* writes. The callers needs to fall back to buffered I/O in this case. */ ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
@@ -478,13 +482,15 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (iov_iter_rw (iter) ==WRITE ) {/* * Try to invalidate cache pages for the range we are writing. -* If this invalidation fails, tough, the write will still work, -* but racing two incompatible write paths is a pretty crazy -* thing to do, so we don't support it 100%. +* If this invalidation fails, let the caller fall back to +* buffered I/O. */ if (invalidate_inode_pages2_range (mapping, pos >>PAGE_SHIFT , - end >>PAGE_SHIFT )) -dio_warn_stale_pagecache (iocb->ki_filp); + end >>PAGE_SHIFT )) { +trace_iomap_dio_invalidate_fail (inode, pos, count); + ret = -ENOTBLK ; + goto out_free_dio; + }if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { ret =sb_init_dio_done_wq (inode->i_sb);
fs/iomap/trace.h +1 -0
@@ -74,6 +74,7 @@ DEFINE_EVENT(iomap_range_class, name, \DEFINE_RANGE_EVENT (iomap_writepage);DEFINE_RANGE_EVENT (iomap_releasepage);DEFINE_RANGE_EVENT (iomap_invalidatepage); +DEFINE_RANGE_EVENT (iomap_dio_invalidate_fail);#define IOMAP_TYPE_STRINGS \ { IOMAP_HOLE, "HOLE" }, \
fs/xfs/xfs_file.c +2 -2
@@ -553,8 +553,8 @@ xfs_file_dio_aio_write(xfs_iunlock (ip, iolock);/* -* No fallback to buffered IO on errors for XFS, direct IO will either -* complete fully or fail. +* No fallback to buffered IO after short writes for XFS, direct I/O +* will either complete fully or return an error. */ ASSERT (ret <0 || ret == count);return ret;
fs/zonefs/super.c +5 -2
@@ -786,8 +786,11 @@ static ssize_t zonefs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)if (iocb->ki_pos >=ZONEFS_I (inode)->i_max_size)return -EFBIG ; -if (iocb->ki_flags &IOCB_DIRECT ) -return zonefs_file_dio_write (iocb, from); +if (iocb->ki_flags &IOCB_DIRECT ) { +ssize_t ret =zonefs_file_dio_write (iocb, from); +if (ret != -ENOTBLK ) +return ret; + }return zonefs_file_buffered_write (iocb, from); }
Signature · 1 line
2.27.0
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
Quoted 7 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>
For the iomap and xfs parts,
But I'd still like acks from Ted, Andreas, and Damien for ext4, gfs2, and zonefs, respectively.
(Particularly if anyone was harboring ideas about trying to get this in before 5.10, though I've not yet heard anyone say that explicitly...)
--D
Quoted 127 lines
--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)fs/ext4/file.c +2
-0@@ -544,6 +544,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret =iomap_dio_rw (iocb, from, iomap_ops, &ext4_dio_write_ops,is_sync_kiocb (iocb) || unaligned_io || extend); +if (ret == -ENOTBLK ) + ret =0 ;if (extend) ret =ext4_handle_inode_extension (inode, offset, ret, count);fs/gfs2/file.c +2
-1@@ -835,7 +835,8 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from) ret =iomap_dio_rw (iocb, from, &gfs2_iomap_ops,NULL ,is_sync_kiocb (iocb)); - +if (ret == -ENOTBLK ) + ret =0 ; out:gfs2_glock_dq (&gh); out_uninit:fs/iomap/direct-io.c +11
-5@@ -10,6 +10,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "trace.h" #include "../internal.h" @@ -401,6 +402,9 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,* can be mapped into multiple disjoint IOs and only a subset of the IOs issued * may be pure data writes. In that case, we still need to do a full data sync * completion. +* +* Returns -ENOTBLK In case of a page invalidation invalidation failure for +* writes. The callers needs to fall back to buffered I/O in this case. */ ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,@@ -478,13 +482,15 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (iov_iter_rw (iter) ==WRITE ) {/* * Try to invalidate cache pages for the range we are writing. -* If this invalidation fails, tough, the write will still work, -* but racing two incompatible write paths is a pretty crazy -* thing to do, so we don't support it 100%. +* If this invalidation fails, let the caller fall back to +* buffered I/O. */ if (invalidate_inode_pages2_range (mapping, pos >>PAGE_SHIFT , - end >>PAGE_SHIFT )) -dio_warn_stale_pagecache (iocb->ki_filp); + end >>PAGE_SHIFT )) { +trace_iomap_dio_invalidate_fail (inode, pos, count); + ret = -ENOTBLK ; + goto out_free_dio; + }if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { ret =sb_init_dio_done_wq (inode->i_sb);fs/iomap/trace.h +1
-0@@ -74,6 +74,7 @@ DEFINE_EVENT(iomap_range_class, name, \DEFINE_RANGE_EVENT (iomap_writepage);DEFINE_RANGE_EVENT (iomap_releasepage);DEFINE_RANGE_EVENT (iomap_invalidatepage); +DEFINE_RANGE_EVENT (iomap_dio_invalidate_fail);#define IOMAP_TYPE_STRINGS \ { IOMAP_HOLE, "HOLE" }, \fs/xfs/xfs_file.c +2
-2@@ -553,8 +553,8 @@ xfs_file_dio_aio_write(xfs_iunlock (ip, iolock);/* -* No fallback to buffered IO on errors for XFS, direct IO will either -* complete fully or fail. +* No fallback to buffered IO after short writes for XFS, direct I/O +* will either complete fully or return an error. */ ASSERT (ret <0 || ret == count);return ret;fs/zonefs/super.c +5
-2@@ -786,8 +786,11 @@ static ssize_t zonefs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)if (iocb->ki_pos >=ZONEFS_I (inode)->i_max_size)return -EFBIG ; -if (iocb->ki_flags &IOCB_DIRECT ) -return zonefs_file_dio_write (iocb, from); +if (iocb->ki_flags &IOCB_DIRECT ) { +ssize_t ret =zonefs_file_dio_write (iocb, from); +if (ret != -ENOTBLK ) +return ret; + }return zonefs_file_buffered_write (iocb, from); }Signature · 1 line
2.27.0
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
Quoted 9 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>
Quoted 8 lines
For the iomap and xfs parts,
Reviewed-by Darrick J. Wong <[email protected]>But I'd still like acks from Ted, Andreas, and Damien for ext4, gfs2, and zonefs, respectively.
(Particularly if anyone was harboring ideas about trying to get this in before 5.10, though I've not yet heard anyone say that explicitly...)
Why would we want to wait another whole merge window?
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>For the iomap and xfs parts,
Reviewed-by Darrick J. Wong <[email protected]>
Quoted 7 lines
Quoted 5 lines
But I'd still like acks from Ted, Andreas, and Damien for ext4, gfs2, and zonefs, respectively.
(Particularly if anyone was harboring ideas about trying to get this in before 5.10, though I've not yet heard anyone say that explicitly...)
Why would we want to wait another whole merge window?
Well it /is/ past -rc6, which is a tad late...
OTOH we've been talking about this for 2 months now and most of the actual behavior change is in xfs land so maybe it's fine. :)
--D
Bob Peterson
Acked-byRe: [Cluster-devel] [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
In reply to Darrick J. Wong----- Original Message -----
Quoted 12 lines · Christoph Hellwig
Quoted 7 lines
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>For the iomap and xfs parts,
Reviewed-by Darrick J. Wong <[email protected]>
Quoted 16 lines
But I'd still like acks from Ted, Andreas, and Damien for ext4, gfs2, and zonefs, respectively.
(Particularly if anyone was harboring ideas about trying to get this in before 5.10, though I've not yet heard anyone say that explicitly...)
--D
Quoted 8 lines
--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)
Hi,
I think Andreas is on holiday this week, but the gfs2 portion looks good to me:
For the gfs2 portion:
Regards,
Bob Peterson
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
In reply to Christoph HellwigQuoted 122 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)fs/ext4/file.c +2
-0@@ -544,6 +544,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret =iomap_dio_rw (iocb, from, iomap_ops, &ext4_dio_write_ops,is_sync_kiocb (iocb) || unaligned_io || extend); +if (ret == -ENOTBLK ) + ret =0 ;if (extend) ret =ext4_handle_inode_extension (inode, offset, ret, count);fs/gfs2/file.c +2
-1@@ -835,7 +835,8 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from) ret =iomap_dio_rw (iocb, from, &gfs2_iomap_ops,NULL ,is_sync_kiocb (iocb)); - +if (ret == -ENOTBLK ) + ret =0 ; out:gfs2_glock_dq (&gh); out_uninit:fs/iomap/direct-io.c +11
-5@@ -10,6 +10,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "trace.h" #include "../internal.h" @@ -401,6 +402,9 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,* can be mapped into multiple disjoint IOs and only a subset of the IOs issued * may be pure data writes. In that case, we still need to do a full data sync * completion. +* +* Returns -ENOTBLK In case of a page invalidation invalidation failure for +* writes. The callers needs to fall back to buffered I/O in this case. */ ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,@@ -478,13 +482,15 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (iov_iter_rw (iter) ==WRITE ) {/* * Try to invalidate cache pages for the range we are writing. -* If this invalidation fails, tough, the write will still work, -* but racing two incompatible write paths is a pretty crazy -* thing to do, so we don't support it 100%. +* If this invalidation fails, let the caller fall back to +* buffered I/O. */ if (invalidate_inode_pages2_range (mapping, pos >>PAGE_SHIFT , - end >>PAGE_SHIFT )) -dio_warn_stale_pagecache (iocb->ki_filp); + end >>PAGE_SHIFT )) { +trace_iomap_dio_invalidate_fail (inode, pos, count); + ret = -ENOTBLK ; + goto out_free_dio; + }if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { ret =sb_init_dio_done_wq (inode->i_sb);fs/iomap/trace.h +1
-0@@ -74,6 +74,7 @@ DEFINE_EVENT(iomap_range_class, name, \DEFINE_RANGE_EVENT (iomap_writepage);DEFINE_RANGE_EVENT (iomap_releasepage);DEFINE_RANGE_EVENT (iomap_invalidatepage); +DEFINE_RANGE_EVENT (iomap_dio_invalidate_fail);#define IOMAP_TYPE_STRINGS \ { IOMAP_HOLE, "HOLE" }, \fs/xfs/xfs_file.c +2
-2@@ -553,8 +553,8 @@ xfs_file_dio_aio_write(xfs_iunlock (ip, iolock);/* -* No fallback to buffered IO on errors for XFS, direct IO will either -* complete fully or fail. +* No fallback to buffered IO after short writes for XFS, direct I/O +* will either complete fully or return an error. */ ASSERT (ret <0 || ret == count);return ret;fs/zonefs/super.c +0
-0@@ -786,8 +786,11 @@ static ssize_t zonefs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)if (iocb->ki_pos >=ZONEFS_I (inode)->i_max_size)return -EFBIG ;
fs/zonefs/super.c static ssize_t zonefs_file_write_iter(struct kiocb *iocb, struct iov_iter *from) · line 796
Quoted 10 lines
fs/zonefs/super.c +5
-2-if (iocb->ki_flags &IOCB_DIRECT ) -return zonefs_file_dio_write (iocb, from); +if (iocb->ki_flags &IOCB_DIRECT ) { +ssize_t ret =zonefs_file_dio_write (iocb, from); +if (ret != -ENOTBLK ) +return ret; + }return zonefs_file_buffered_write (iocb, from); }
Looks fine. For zonefs:
Signature · 2 lines
Damien Le Moal Western Digital Research
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
In reply to Christoph HellwigHey Ted,
Could you please review the fs/ext4/ part of this patch (it's the follow-on to the directio discussion I had with you last week) so that I can get this moving for 5.9? Thx,
--D
Quoted 134 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)fs/ext4/file.c +2
-0@@ -544,6 +544,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret =iomap_dio_rw (iocb, from, iomap_ops, &ext4_dio_write_ops,is_sync_kiocb (iocb) || unaligned_io || extend); +if (ret == -ENOTBLK ) + ret =0 ;if (extend) ret =ext4_handle_inode_extension (inode, offset, ret, count);fs/gfs2/file.c +2
-1@@ -835,7 +835,8 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from) ret =iomap_dio_rw (iocb, from, &gfs2_iomap_ops,NULL ,is_sync_kiocb (iocb)); - +if (ret == -ENOTBLK ) + ret =0 ; out:gfs2_glock_dq (&gh); out_uninit:fs/iomap/direct-io.c +11
-5@@ -10,6 +10,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "trace.h" #include "../internal.h" @@ -401,6 +402,9 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,* can be mapped into multiple disjoint IOs and only a subset of the IOs issued * may be pure data writes. In that case, we still need to do a full data sync * completion. +* +* Returns -ENOTBLK In case of a page invalidation invalidation failure for +* writes. The callers needs to fall back to buffered I/O in this case. */ ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,@@ -478,13 +482,15 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (iov_iter_rw (iter) ==WRITE ) {/* * Try to invalidate cache pages for the range we are writing. -* If this invalidation fails, tough, the write will still work, -* but racing two incompatible write paths is a pretty crazy -* thing to do, so we don't support it 100%. +* If this invalidation fails, let the caller fall back to +* buffered I/O. */ if (invalidate_inode_pages2_range (mapping, pos >>PAGE_SHIFT , - end >>PAGE_SHIFT )) -dio_warn_stale_pagecache (iocb->ki_filp); + end >>PAGE_SHIFT )) { +trace_iomap_dio_invalidate_fail (inode, pos, count); + ret = -ENOTBLK ; + goto out_free_dio; + }if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { ret =sb_init_dio_done_wq (inode->i_sb);fs/iomap/trace.h +1
-0@@ -74,6 +74,7 @@ DEFINE_EVENT(iomap_range_class, name, \DEFINE_RANGE_EVENT (iomap_writepage);DEFINE_RANGE_EVENT (iomap_releasepage);DEFINE_RANGE_EVENT (iomap_invalidatepage); +DEFINE_RANGE_EVENT (iomap_dio_invalidate_fail);#define IOMAP_TYPE_STRINGS \ { IOMAP_HOLE, "HOLE" }, \fs/xfs/xfs_file.c +2
-2@@ -553,8 +553,8 @@ xfs_file_dio_aio_write(xfs_iunlock (ip, iolock);/* -* No fallback to buffered IO on errors for XFS, direct IO will either -* complete fully or fail. +* No fallback to buffered IO after short writes for XFS, direct I/O +* will either complete fully or return an error. */ ASSERT (ret <0 || ret == count);return ret;fs/zonefs/super.c +5
-2@@ -786,8 +786,11 @@ static ssize_t zonefs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)if (iocb->ki_pos >=ZONEFS_I (inode)->i_max_size)return -EFBIG ; -if (iocb->ki_flags &IOCB_DIRECT ) -return zonefs_file_dio_write (iocb, from); +if (iocb->ki_flags &IOCB_DIRECT ) { +ssize_t ret =zonefs_file_dio_write (iocb, from); +if (ret != -ENOTBLK ) +return ret; + }return zonefs_file_buffered_write (iocb, from); }Signature · 1 line
2.27.0
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
Quoted 5 lines · Darrick J. Wong
Hey Ted,
Could you please review the
fs/ext4/part of this patch (it's the follow-on to the directio discussion I had with you last week) so that I can get this moving for 5.9? Thx,
- Ted
Re: [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
In reply to Christoph HellwigQuoted 7 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>
mainly for ext4 part.
Quoted 69 lines
--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)fs/ext4/file.c +2
-0@@ -544,6 +544,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret =iomap_dio_rw (iocb, from, iomap_ops, &ext4_dio_write_ops,is_sync_kiocb (iocb) || unaligned_io || extend); +if (ret == -ENOTBLK ) + ret =0 ;if (extend) ret =ext4_handle_inode_extension (inode, offset, ret, count);fs/gfs2/file.c +2
-1@@ -835,7 +835,8 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from) ret =iomap_dio_rw (iocb, from, &gfs2_iomap_ops,NULL ,is_sync_kiocb (iocb)); - +if (ret == -ENOTBLK ) + ret =0 ; out:gfs2_glock_dq (&gh); out_uninit:fs/iomap/direct-io.c +6
-3@@ -10,6 +10,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "trace.h" #include "../internal.h" @@ -401,6 +402,9 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,* can be mapped into multiple disjoint IOs and only a subset of the IOs issued * may be pure data writes. In that case, we still need to do a full data sync * completion. +* +* Returns -ENOTBLK In case of a page invalidation invalidation failure for +* writes. The callers needs to fall back to buffered I/O in this case. */ ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,@@ -478,13 +482,15 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (iov_iter_rw (iter) ==WRITE ) {/* * Try to invalidate cache pages for the range we are writing. -* If this invalidation fails, tough, the write will still work, -* but racing two incompatible write paths is a pretty crazy -* thing to do, so we don't support it 100%. +* If this invalidation fails, let the caller fall back to +* buffered I/O. */ if (invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT,
fs/iomap/direct-io.c iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, · line 496
Quoted 10 lines
fs/iomap/direct-io.c +5
-2- end >> PAGE_SHIFT)) -dio_warn_stale_pagecache (iocb->ki_filp); + end >> PAGE_SHIFT)) { +trace_iomap_dio_invalidate_fail (inode, pos, count); + ret = -ENOTBLK ; + goto out_free_dio; + }if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { ret =sb_init_dio_done_wq (inode->i_sb);
Just as a note. So if the driver returns -ENOTBLK (from ->iomap_end)then iomap considers it as a magic value to fall back to buffered-io and it changes ret=0 And now with this patch, iomap could also return -ENOTBLK if it gets an error while doing above operation and so the driver is free to consider this as a fallback mechanism to buffered-io.
Re: [Cluster-devel] [PATCH 3/3] iomap: fall back to buffered writes for invalidation failures
In reply to Christoph HellwigQuoted 132 lines · Christoph Hellwig
Failing to invalid the page cache means data in incoherent, which is a very bad state for the system. Always fall back to buffered I/O through the page cache if we can't invalidate mappings.
Signed-off-by Christoph Hellwig <[email protected]>Acked-by Dave Chinner <[email protected]>Reviewed-by Goldwyn Rodrigues <[email protected]>--- fs/ext4/file.c | 2 ++ fs/gfs2/file.c | 3 ++- fs/iomap/direct-io.c | 16 +++++++++++----- fs/iomap/trace.h | 1 + fs/xfs/xfs_file.c | 4 ++-- fs/zonefs/super.c | 7 +++++-- 6 files changed, 23 insertions(+), 10 deletions(-)fs/ext4/file.c +2
-0@@ -544,6 +544,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret =iomap_dio_rw (iocb, from, iomap_ops, &ext4_dio_write_ops,is_sync_kiocb (iocb) || unaligned_io || extend); +if (ret == -ENOTBLK ) + ret =0 ;if (extend) ret =ext4_handle_inode_extension (inode, offset, ret, count);fs/gfs2/file.c +2
-1@@ -835,7 +835,8 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from) ret =iomap_dio_rw (iocb, from, &gfs2_iomap_ops,NULL ,is_sync_kiocb (iocb)); - +if (ret == -ENOTBLK ) + ret =0 ; out:gfs2_glock_dq (&gh); out_uninit:fs/iomap/direct-io.c +11
-5@@ -10,6 +10,7 @@#include <linux/backing-dev.h> #include <linux/uio.h> #include <linux/task_io_accounting_ops.h> +#include "trace.h" #include "../internal.h" @@ -401,6 +402,9 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,* can be mapped into multiple disjoint IOs and only a subset of the IOs issued * may be pure data writes. In that case, we still need to do a full data sync * completion. +* +* Returns -ENOTBLK In case of a page invalidation invalidation failure for +* writes. The callers needs to fall back to buffered I/O in this case. */ ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,@@ -478,13 +482,15 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,if (iov_iter_rw (iter) ==WRITE ) {/* * Try to invalidate cache pages for the range we are writing. -* If this invalidation fails, tough, the write will still work, -* but racing two incompatible write paths is a pretty crazy -* thing to do, so we don't support it 100%. +* If this invalidation fails, let the caller fall back to +* buffered I/O. */ if (invalidate_inode_pages2_range (mapping, pos >>PAGE_SHIFT , - end >>PAGE_SHIFT )) -dio_warn_stale_pagecache (iocb->ki_filp); + end >>PAGE_SHIFT )) { +trace_iomap_dio_invalidate_fail (inode, pos, count); + ret = -ENOTBLK ; + goto out_free_dio; + }if (!wait_for_completion && !inode->i_sb->s_dio_done_wq) { ret =sb_init_dio_done_wq (inode->i_sb);fs/iomap/trace.h +1
-0@@ -74,6 +74,7 @@ DEFINE_EVENT(iomap_range_class, name, \DEFINE_RANGE_EVENT (iomap_writepage);DEFINE_RANGE_EVENT (iomap_releasepage);DEFINE_RANGE_EVENT (iomap_invalidatepage); +DEFINE_RANGE_EVENT (iomap_dio_invalidate_fail);#define IOMAP_TYPE_STRINGS \ { IOMAP_HOLE, "HOLE" }, \fs/xfs/xfs_file.c +2
-2@@ -553,8 +553,8 @@ xfs_file_dio_aio_write(xfs_iunlock (ip, iolock);/* -* No fallback to buffered IO on errors for XFS, direct IO will either -* complete fully or fail. +* No fallback to buffered IO after short writes for XFS, direct I/O +* will either complete fully or return an error. */ ASSERT (ret <0 || ret == count);return ret;fs/zonefs/super.c +5
-3@@ -786,8 +786,11 @@ static ssize_t zonefs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)if (iocb->ki_pos >=ZONEFS_I (inode)->i_max_size)return -EFBIG ; -if (iocb->ki_flags &IOCB_DIRECT ) -return zonefs_file_dio_write (iocb, from); +if (iocb->ki_flags &IOCB_DIRECT ) { +ssize_t ret =zonefs_file_dio_write (iocb, from); +if (ret != -ENOTBLK ) +return ret; + }return zonefs_file_buffered_write (iocb, from); } --
Quoted 2 lines
2.27.0
Thanks,
Andreas