f2fs: atomic: fix to not allow GC to pollute atomic_file
Chao Yu
maintainerPATCH v2 2/4 f2fs: atomic: fix to not allow GC to pollute atomic_file
SQLite App GC Thread Shrinker
- f2fs_ioc_start_atomic_write
- f2fs_ioc_commit_atomic_write
- f2fs_commit_atomic_write
- filemap_write_and_wait_range
: write atomic_file's data to cow_inode
echo 3 > drop_caches
- f2fs_gc
- gc_data_segment
- move_data_page
- set_page_dirty
: it may load data of previous
transaction into pagecache.
- f2fs_down_write(&fi->i_gc_rwsem[WRITE])
- __f2fs_commit_atomic_write
- f2fs_up_write(&fi->i_gc_rwsem[WRITE])
During committing atomic_file, GC may be triggered to migrate atomic_file's block, so it may contain data of previous transaction in page cache, we should drop atomic_file's cache once it was migrated by GC.
And also, we should writeback atomic_file and cow_file's data w/ i_gc_rwsem lock held, in order to avoid block address change during __f2fs_commit_atomic_write().
Meahwhile, this patch adds f2fs_wait_on_block_writeback_range() to wait completion of block migration.
---
v2:
- fix error path handling.
fs/f2fs/segment.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)
fs/f2fs/segment.c +23 -4
@@ -236,6 +236,9 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index,return err; } +if (__is_valid_data_blkaddr (dn.data_blkaddr)) +f2fs_wait_on_block_writeback_range (inode, dn.data_blkaddr,1 ); +if (recover) {/* dn.data_blkaddr is always valid */ if (!__is_valid_data_blkaddr (new_addr)) {
@@ -339,6 +342,9 @@ static int __f2fs_commit_atomic_write(struct inode *inode) goto out; } +f2fs_wait_on_block_writeback_range (cow_inode, + blkaddr,1 ); + new =f2fs_kmem_cache_alloc (revoke_entry_slab,GFP_NOFS , true,NULL );
@@ -379,16 +385,29 @@ int f2fs_commit_atomic_write(struct inode *inode)struct f2fs_inode_info *fi =F2FS_I (inode);int err; +f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); + err =filemap_write_and_wait_range (inode->i_mapping,0 ,LLONG_MAX );if (err) -return err; + goto out; -f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); -f2fs_lock_op (sbi); +/* writeback GCing page of cow_inode */ + err =filemap_write_and_wait_range (fi->cow_inode->i_mapping, +0 ,LLONG_MAX ); +if (err) + goto out; - err =__f2fs_commit_atomic_write (inode); +filemap_invalidate_lock (inode->i_mapping); + +/* don't allow clean page loaded by GC to pollute atomic_file */ +truncate_pagecache (inode,0 ); +f2fs_lock_op (sbi); + err =__f2fs_commit_atomic_write (inode);f2fs_unlock_op (sbi); + +filemap_invalidate_unlock (inode->i_mapping); +out:f2fs_up_write (&fi->i_gc_rwsem[WRITE ]);return err;
Signature · 1 line
2.40.1
Hello Chao Yu,
Quoted 11 lines
SQLite App GC Thread Shrinker
-f2fs_ioc_start_atomic_write-
f2fs_ioc_commit_atomic_write
-f2fs_commit_atomic_write
-filemap_write_and_wait_range
: write atomic_file's data tocow_inode
echo 3 >drop_caches
-f2fs_gc
-gc_data_segment
-move_data_page
We modified the code to make GC of the atomic file go to move_data_block in commit b40a2b003709 (f2fs: use meta inode for GC of atomic file). Could you please check if this patch is still necessary?
Thanks,
Quoted 31 lines
-
set_page_dirty
: it may load data of previous
transaction into pagecache.
-f2fs_down_write(&fi->i_gc_rwsem[WRITE])
-__f2fs_commit_atomic_write
-f2fs_up_write(&fi->i_gc_rwsem[WRITE])During committing
atomic_file, GC may be triggered to migrate atomic_file's block, so it may contain data of previous transaction in page cache, we should drop atomic_file's cache once it was migrated by GC.And also, we should writeback
atomic_fileand cow_file's data w/i_gc_rwsemlock held, in order to avoid block address change during__f2fs_commit_atomic_write().Meahwhile, this patch adds
f2fs_wait_on_block_writeback_range()to wait completion of block migration.Fixes 3db1de0e582c ("f2fs: change the current atomic write way")Signed-off-by Chao Yu <[email protected]>---
v2:
- fix error path handling.fs/f2fs/segment.c| 27 +++++++++++++++++++++++----1 file changed, 23 insertions(+), 4 deletions(-)fs/f2fs/segment.c index renamed from fs/f2fs/segment.c +0
-0
3aee71c9f3c6..a43054ab0cf1100644
fs/f2fs/segment.c static int __replace_atomic_write_block(struct inode · line 241
Quoted 6 lines
fs/f2fs/segment.c +2
-0@@ -236,6 +236,9 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index, return err; } +if (__is_valid_data_blkaddr (dn.data_blkaddr)) + f2fs_wait_on_block_writeback_range(inode, dn.data_blkaddr,
1);
Quoted 4 lines
fs/f2fs/segment.c +1
-0+if (recover) {/* dn.data_blkaddr is always valid */ if (!__is_valid_data_blkaddr(new_addr)) { @@ -339,6 +342,9
@@
Quoted 8 lines
static int
__f2fs_commit_atomic_write(struct inode *inode)goto out; }fs/f2fs/segment.c +3
-0+f2fs_wait_on_block_writeback_range (cow_inode, + blkaddr,1 ); +new = f2fs_kmem_cache_alloc(revoke_entry_slab,
GFP_NOFS,
Quoted 45 lines
true, NULL);fs/f2fs/segment.c +17
-5@@ -379,16 +385,29 @@ int f2fs_commit_atomic_write(struct inode *inode)struct f2fs_inode_info *fi =F2FS_I (inode);int err; +f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); + err =filemap_write_and_wait_range (inode->i_mapping,0 ,LLONG_MAX );if (err) -return err; + goto out; -f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); -f2fs_lock_op (sbi); +/* writeback GCing page of cow_inode */ + err =filemap_write_and_wait_range (fi->cow_inode->i_mapping, +0 ,LLONG_MAX ); +if (err) + goto out; - err =__f2fs_commit_atomic_write (inode); +filemap_invalidate_lock (inode->i_mapping); + +/* don't allow clean page loaded by GC to pollute atomic_file */ +truncate_pagecache (inode,0 ); +f2fs_lock_op (sbi); + err =__f2fs_commit_atomic_write (inode);f2fs_unlock_op (sbi); + +filemap_invalidate_unlock (inode->i_mapping); +out:f2fs_up_write (&fi->i_gc_rwsem[WRITE ]);return err; --2.40.1
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Chao Yu
maintainerQuoted 14 lines · Sunmin Jeong
Hello Chao Yu,
Quoted 11 lines
SQLite App GC Thread Shrinker
-f2fs_ioc_start_atomic_write-
f2fs_ioc_commit_atomic_write
-f2fs_commit_atomic_write
-filemap_write_and_wait_range
: write atomic_file's data tocow_inode
echo 3 >drop_caches
-f2fs_gc
-gc_data_segment
-move_data_page
Quoted 3 lines
We modified the code to make GC of the atomic file go to
move_data_blockin commitb40a2b003709(f2fs: use meta inode for GC of atomic file). Could you please check if this patch is still necessary?
Sunmin,
Yes, you're right.
Also, I've checked f2fs/003 of xfstests w/ commit b40a2b003709, it can fix the regression issue as well, so let's drop it.
Thanks for pointing it out. :)
Thanks,
Quoted 101 lines
Thanks,
Quoted 31 lines
-
set_page_dirty
: it may load data of previous
transaction into pagecache.
-f2fs_down_write(&fi->i_gc_rwsem[WRITE])
-__f2fs_commit_atomic_write
-f2fs_up_write(&fi->i_gc_rwsem[WRITE])During committing
atomic_file, GC may be triggered to migrate atomic_file's block, so it may contain data of previous transaction in page cache, we should drop atomic_file's cache once it was migrated by GC.And also, we should writeback
atomic_fileand cow_file's data w/i_gc_rwsemlock held, in order to avoid block address change during__f2fs_commit_atomic_write().Meahwhile, this patch adds
f2fs_wait_on_block_writeback_range()to wait completion of block migration.Fixes 3db1de0e582c ("f2fs: change the current atomic write way")Signed-off-by Chao Yu <[email protected]>---
v2:
- fix error path handling.fs/f2fs/segment.c| 27 +++++++++++++++++++++++----1 file changed, 23 insertions(+), 4 deletions(-)fs/f2fs/segment.c index renamed from fs/f2fs/segment.c +0
-0
3aee71c9f3c6..a43054ab0cf1100644fs/f2fs/segment.c static int __replace_atomic_write_block(struct inode · line 241
Quoted 6 lines
fs/f2fs/segment.c +2
-0@@ -236,6 +236,9 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index, return err; } +if (__is_valid_data_blkaddr (dn.data_blkaddr)) + f2fs_wait_on_block_writeback_range(inode, dn.data_blkaddr,1);
Quoted 4 lines
fs/f2fs/segment.c +1
-0+if (recover) {/* dn.data_blkaddr is always valid */ if (!__is_valid_data_blkaddr(new_addr)) { @@ -339,6 +342,9@@
Quoted 8 lines
static int
__f2fs_commit_atomic_write(struct inode *inode)goto out; }fs/f2fs/segment.c +3
-0+f2fs_wait_on_block_writeback_range (cow_inode, + blkaddr,1 ); +new = f2fs_kmem_cache_alloc(revoke_entry_slab,
GFP_NOFS,Quoted 45 lines
true, NULL);fs/f2fs/segment.c +17
-5@@ -379,16 +385,29 @@ int f2fs_commit_atomic_write(struct inode *inode)struct f2fs_inode_info *fi =F2FS_I (inode);int err; +f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); + err =filemap_write_and_wait_range (inode->i_mapping,0 ,LLONG_MAX );if (err) -return err; + goto out; -f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); -f2fs_lock_op (sbi); +/* writeback GCing page of cow_inode */ + err =filemap_write_and_wait_range (fi->cow_inode->i_mapping, +0 ,LLONG_MAX ); +if (err) + goto out; - err =__f2fs_commit_atomic_write (inode); +filemap_invalidate_lock (inode->i_mapping); + +/* don't allow clean page loaded by GC to pollute atomic_file */ +truncate_pagecache (inode,0 ); +f2fs_lock_op (sbi); + err =__f2fs_commit_atomic_write (inode);f2fs_unlock_op (sbi); + +filemap_invalidate_unlock (inode->i_mapping); +out:f2fs_up_write (&fi->i_gc_rwsem[WRITE ]);return err; --2.40.1
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Jaegeuk Kim
maintainerQuoted 76 lines · Chao Yu
SQLite App GC Thread Shrinker
-f2fs_ioc_start_atomic_write-
f2fs_ioc_commit_atomic_write
-f2fs_commit_atomic_write
-filemap_write_and_wait_range
: write atomic_file's data tocow_inode
echo 3 >drop_caches
-f2fs_gc
-gc_data_segment
-move_data_page
-set_page_dirty
: it may load data of previous
transaction into pagecache.
-f2fs_down_write(&fi->i_gc_rwsem[WRITE])
-__f2fs_commit_atomic_write
-f2fs_up_write(&fi->i_gc_rwsem[WRITE])During committing
atomic_file, GC may be triggered to migrate atomic_file's block, so it may contain data of previous transaction in page cache, we should drop atomic_file's cache once it was migrated by GC.And also, we should writeback
atomic_fileand cow_file's data w/i_gc_rwsemlock held, in order to avoid block address change during__f2fs_commit_atomic_write().Meahwhile, this patch adds
f2fs_wait_on_block_writeback_range()to wait completion of block migration.Fixes 3db1de0e582c ("f2fs: change the current atomic write way")Signed-off-by Chao Yu <[email protected]>---
v2:
- fix error path handling.fs/f2fs/segment.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)fs/f2fs/segment.c +10
-3@@ -236,6 +236,9 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index,return err; } +if (__is_valid_data_blkaddr (dn.data_blkaddr)) +f2fs_wait_on_block_writeback_range (inode, dn.data_blkaddr,1 ); +if (recover) {/* dn.data_blkaddr is always valid */ if (!__is_valid_data_blkaddr (new_addr)) {@@ -339,6 +342,9 @@ static int __f2fs_commit_atomic_write(struct inode *inode) goto out; } +f2fs_wait_on_block_writeback_range (cow_inode, + blkaddr,1 ); + new =f2fs_kmem_cache_alloc (revoke_entry_slab,GFP_NOFS , true,NULL );@@ -379,16 +385,29 @@ int f2fs_commit_atomic_write(struct inode *inode)struct f2fs_inode_info *fi =F2FS_I (inode);int err; +f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); + err =filemap_write_and_wait_range (inode->i_mapping,0 ,LLONG_MAX );if (err) -return err; + goto out; -f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); -f2fs_lock_op (sbi); +/* writeback GCing page of cow_inode */
fs/f2fs/segment.c int f2fs_commit_atomic_write(struct inode *inode) · line 403
Quoted 10 lines
fs/f2fs/segment.c +8
-1+ err =filemap_write_and_wait_range (fi->cow_inode->i_mapping, +0 ,LLONG_MAX ); +if (err) + goto out; - err =__f2fs_commit_atomic_write (inode); +filemap_invalidate_lock (inode->i_mapping); + +/* don't allow clean page loaded by GC to pollute atomic_file */ +truncate_pagecache (inode,0 );
Performance impact?
Quoted 12 lines
fs/f2fs/segment.c +5
-0+f2fs_lock_op (sbi); + err =__f2fs_commit_atomic_write (inode);f2fs_unlock_op (sbi); + +filemap_invalidate_unlock (inode->i_mapping); +out:f2fs_up_write (&fi->i_gc_rwsem[WRITE ]);return err;Signature · 1 line
2.40.1
Chao Yu
maintainerQuoted 82 lines · Jaegeuk Kim
Quoted 81 lines · Chao Yu
SQLite App GC Thread Shrinker
-f2fs_ioc_start_atomic_write-
f2fs_ioc_commit_atomic_write
-f2fs_commit_atomic_write
-filemap_write_and_wait_range
: write atomic_file's data tocow_inode
echo 3 >drop_caches
-f2fs_gc
-gc_data_segment
-move_data_page
-set_page_dirty
: it may load data of previous
transaction into pagecache.
-f2fs_down_write(&fi->i_gc_rwsem[WRITE])
-__f2fs_commit_atomic_write
-f2fs_up_write(&fi->i_gc_rwsem[WRITE])During committing
atomic_file, GC may be triggered to migrate atomic_file's block, so it may contain data of previous transaction in page cache, we should drop atomic_file's cache once it was migrated by GC.And also, we should writeback
atomic_fileand cow_file's data w/i_gc_rwsemlock held, in order to avoid block address change during__f2fs_commit_atomic_write().Meahwhile, this patch adds
f2fs_wait_on_block_writeback_range()to wait completion of block migration.Fixes 3db1de0e582c ("f2fs: change the current atomic write way")Signed-off-by Chao Yu <[email protected]>---
v2:
- fix error path handling.fs/f2fs/segment.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)fs/f2fs/segment.c +14
-3@@ -236,6 +236,9 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index,return err; } +if (__is_valid_data_blkaddr (dn.data_blkaddr)) +f2fs_wait_on_block_writeback_range (inode, dn.data_blkaddr,1 ); +if (recover) {/* dn.data_blkaddr is always valid */ if (!__is_valid_data_blkaddr (new_addr)) {@@ -339,6 +342,9 @@ static int __f2fs_commit_atomic_write(struct inode *inode) goto out; } +f2fs_wait_on_block_writeback_range (cow_inode, + blkaddr,1 ); + new =f2fs_kmem_cache_alloc (revoke_entry_slab,GFP_NOFS , true,NULL );@@ -379,16 +385,29 @@ int f2fs_commit_atomic_write(struct inode *inode)struct f2fs_inode_info *fi =F2FS_I (inode);int err; +f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); + err =filemap_write_and_wait_range (inode->i_mapping,0 ,LLONG_MAX );if (err) -return err; + goto out; -f2fs_down_write (&fi->i_gc_rwsem[WRITE ]); -f2fs_lock_op (sbi); +/* writeback GCing page of cow_inode */ + err =filemap_write_and_wait_range (fi->cow_inode->i_mapping, +0 ,LLONG_MAX ); +if (err) + goto out;
Quoted 7 lines
fs/f2fs/segment.c int f2fs_commit_atomic_write(struct inode *inode) · line 403
Quoted 5 lines
fs/f2fs/segment.c +4
-1- err =__f2fs_commit_atomic_write (inode); +filemap_invalidate_lock (inode->i_mapping); + +/* don't allow clean page loaded by GC to pollute atomic_file */ +truncate_pagecache (inode,0 );Performance impact?
Please drop this patch due to commit b40a2b003709 (f2fs: use meta inode for GC of atomic file) has fixed this issue. Thanks.
Thanks,
Quoted 13 lines
Quoted 13 lines
fs/f2fs/segment.c +5
-0+f2fs_lock_op (sbi); + err =__f2fs_commit_atomic_write (inode);f2fs_unlock_op (sbi); + +filemap_invalidate_unlock (inode->i_mapping); +out:f2fs_up_write (&fi->i_gc_rwsem[WRITE ]);return err;Signature · 1 line
2.40.1