ceph: invalidate pages when doing direct/sync writes
PATCH v3 1/1 ceph: invalidate pages when doing direct/sync writes
When doing a direct/sync write, we need to invalidate the page cache in the range being written to. If we don't do this, the cache will include invalid data as we just did a write that avoided the page cache.
--- fs/ceph/file.c | 9 +++++++++ 1 file changed, 9 insertions(+)
Ok, here's a new attempt. After discussion in this thread and on IRC, I think this is the right fix. generic/647 now passes with and without encryption. Thanks!
Changes since v2:
- Invalidation needs to be done after a write
Changes since v1:
- Replaced truncate_inode_pages_range() by invalidate_inode_pages2_range
- Call fscache_invalidate with FSCACHE_INVAL_DIO_WRITE if we're doing DIO
fs/ceph/file.c +9 -0
@@ -1938,6 +1938,15 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; }ceph_clear_error_write (ci); + ret =invalidate_inode_pages2_range ( + inode->i_mapping, + pos >>PAGE_SHIFT , + (pos + len -1 ) >>PAGE_SHIFT ); +if (ret <0 ) { +dout ("invalidate_inode_pages2_range returned %d\n" , + ret); + ret =0 ; + } pos += len; written += len;dout ("sync_write written %d\n" , written);
Quoted 30 lines · Luís Henriques
When doing a direct/sync write, we need to invalidate the page cache in the range being written to. If we don't do this, the cache will include invalid data as we just did a write that avoided the page cache.
Signed-off-by Luís Henriques <[email protected]>--- fs/ceph/file.c | 9 +++++++++ 1 file changed, 9 insertions(+)Ok, here's a new attempt. After discussion in this thread and on IRC, I think this is the right fix. generic/647 now passes with and without encryption. Thanks!
Changes since v2:
- Invalidation needs to be done after a writeChanges since v1:
- Replacedtruncate_inode_pages_range()byinvalidate_inode_pages2_range
- Callfscache_invalidatewithFSCACHE_INVAL_DIO_WRITEif we're doing DIOfs/ceph/file.c +2
-0@@ -1938,6 +1938,15 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; }ceph_clear_error_write (ci); + ret = invalidate_inode_pages2_range( + inode->i_mapping,
fs/ceph/file.c ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, · line 1952
Quoted 10 lines
fs/ceph/file.c +7
-0+ pos >>PAGE_SHIFT , + (pos + len - 1) >> PAGE_SHIFT); +if (ret <0 ) { +dout ("invalidate_inode_pages2_range returned %d\n" , + ret); + ret =0 ; + } pos += len; written += len;dout ("sync_write written %d\n" , written);
Looks good. I suspect we can also remove the invalidate_indode_pages2_range call earlier in this function too. I may roll that into this patch.
I'll give this an xfstests run with fscrypt enabled and see how it does.
Thanks!
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 37 lines · Jeff Layton
Quoted 36 lines · Luís Henriques
When doing a direct/sync write, we need to invalidate the page cache in the range being written to. If we don't do this, the cache will include invalid data as we just did a write that avoided the page cache.
Signed-off-by Luís Henriques <[email protected]>--- fs/ceph/file.c | 9 +++++++++ 1 file changed, 9 insertions(+)Ok, here's a new attempt. After discussion in this thread and on IRC, I think this is the right fix. generic/647 now passes with and without encryption. Thanks!
Changes since v2:
- Invalidation needs to be done after a writeChanges since v1:
- Replacedtruncate_inode_pages_range()byinvalidate_inode_pages2_range
- Callfscache_invalidatewithFSCACHE_INVAL_DIO_WRITEif we're doing DIOfs/ceph/file.c +8
-0@@ -1938,6 +1938,15 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; }ceph_clear_error_write (ci); + ret =invalidate_inode_pages2_range ( + inode->i_mapping, + pos >>PAGE_SHIFT , + (pos + len -1 ) >>PAGE_SHIFT ); +if (ret <0 ) { +dout ("invalidate_inode_pages2_range returned %d\n" , + ret); + ret =0 ;
Quoted 8 lines
fs/ceph/file.c ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, · line 1952
Quoted 4 lines
fs/ceph/file.c +1
-0+ } pos += len; written += len;dout ("sync_write written %d\n" , written);Looks good. I suspect we can also remove the
invalidate_indode_pages2_rangecall earlier in this function too. I may roll that into this patch.
Right, that occurred to me as well but I wasn't really sure that would be safe.
Quoted 1 line
I'll give this an xfstests run with fscrypt enabled and see how it does.
I'll do the same here, I just run a few tests on a vstart cluster. But I'll definitely give it some more testing.
Cheers,
Signature · 1 line
Luís
Quoted 31 lines · Luís Henriques
When doing a direct/sync write, we need to invalidate the page cache in the range being written to. If we don't do this, the cache will include invalid data as we just did a write that avoided the page cache.
Signed-off-by Luís Henriques <[email protected]>--- fs/ceph/file.c | 9 +++++++++ 1 file changed, 9 insertions(+)Ok, here's a new attempt. After discussion in this thread and on IRC, I think this is the right fix. generic/647 now passes with and without encryption. Thanks!
Changes since v2:
- Invalidation needs to be done after a writeChanges since v1:
- Replacedtruncate_inode_pages_range()byinvalidate_inode_pages2_range
- Callfscache_invalidatewithFSCACHE_INVAL_DIO_WRITEif we're doing DIOfs/ceph/file.c +2
-0@@ -1938,6 +1938,15 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; }ceph_clear_error_write (ci); + ret = invalidate_inode_pages2_range( + inode->i_mapping,
fs/ceph/file.c ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, · line 1952
Quoted 10 lines
fs/ceph/file.c +7
-0+ pos >>PAGE_SHIFT , + (pos + len - 1) >> PAGE_SHIFT); +if (ret <0 ) { +dout ("invalidate_inode_pages2_range returned %d\n" , + ret); + ret =0 ; + } pos += len; written += len;dout ("sync_write written %d\n" , written);
LGTM.
Maybe it worth adding a comment to explain why we need this and where the mapping come from ?
Quoted 44 lines · Xiubo Li
Quoted 31 lines · Luís Henriques
When doing a direct/sync write, we need to invalidate the page cache in the range being written to. If we don't do this, the cache will include invalid data as we just did a write that avoided the page cache.
Signed-off-by Luís Henriques <[email protected]>--- fs/ceph/file.c | 9 +++++++++ 1 file changed, 9 insertions(+)Ok, here's a new attempt. After discussion in this thread and on IRC, I think this is the right fix. generic/647 now passes with and without encryption. Thanks!
Changes since v2:
- Invalidation needs to be done after a writeChanges since v1:
- Replacedtruncate_inode_pages_range()byinvalidate_inode_pages2_range
- Callfscache_invalidatewithFSCACHE_INVAL_DIO_WRITEif we're doing DIOfs/ceph/file.c +2
-0@@ -1938,6 +1938,15 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; }ceph_clear_error_write (ci); + ret = invalidate_inode_pages2_range( + inode->i_mapping,fs/ceph/file.c ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, · line 1952
Quoted 10 lines
fs/ceph/file.c +7
-0+ pos >>PAGE_SHIFT , + (pos + len - 1) >> PAGE_SHIFT); +if (ret <0 ) { +dout ("invalidate_inode_pages2_range returned %d\n" , + ret); + ret =0 ; + } pos += len; written += len;dout ("sync_write written %d\n" , written);LGTM.
Quoted 5 lines
Maybe it worth adding a comment to explain why we need this and where the mapping come from ?
Reviewed-by Xiubo Li <[email protected]>
Sure, I'll send out v4 with an extra comment. Thanks.
Cheers,
Signature · 1 line
Luís