f2fs: do not select same victim right again
Jaegeuk Kim
maintainerPATCH 1/2 f2fs: do not select same victim right again
GC must avoid select the same victim again.
--- fs/f2fs/gc.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/f2fs/gc.c +3 -3
@@ -274,6 +274,9 @@ static unsigned int get_cb_cost(struct f2fs_sb_info *sbi, unsigned int segno)static inline unsigned int get_gc_cost (struct f2fs_sb_info *sbi,unsigned int segno,struct victim_sel_policy *p) { +if (sbi->cur_victim_sec ==GET_SEC_FROM_SEG (sbi, segno)) +return UINT_MAX ; +if (p->alloc_mode ==SSR )return get_seg_entry (sbi, segno)->ckpt_valid_blocks;
@@ -1326,9 +1329,6 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; } -if (gc_type ==FG_GC ) - sbi->cur_victim_sec =NULL_SEGNO ; -if (sync) goto stop;
Signature · 1 line
2.19.0.605.g01d371f741-goog
Jaegeuk Kim
maintainerPATCH 2/2 f2fs: avoid infinite GC loop due to stale atomic files
If committing atomic pages is failed when doing f2fs_do_sync_file(), we can get commited pages but atomic_file being still set like:
- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c
In that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)
[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)
--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
fs/f2fs/file.c +5 -5
@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }
Signature · 1 line
2.19.0.605.g01d371f741-goog
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 43 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true);
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 10 lines
fs/f2fs/file.c +5
-5-if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);
If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear FI_ATOMIC_FILE flag.
So my question is why below logic didn't handle such condition well?
f2fs_gc()
if (has_not_enough_free_secs(sbi, sec_freed, 0)) {
if (skipped_round <= MAX_SKIP_GC_COUNT ||
skipped_round * 2 < round) {
segno = NULL_SEGNO;
goto gc_more;
}
if (first_skipped < last_skipped &&
(last_skipped - first_skipped) >
sbi->skipped_gc_rwsem) {
f2fs_drop_inmem_pages_all(sbi, true);
segno = NULL_SEGNO;
goto gc_more;
}
if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED))
ret = f2fs_write_checkpoint(sbi, &cpc);
}fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2007
Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;
Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.
f2fs_commit_inmem_pages() should have flushed them.
Quoted 5 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()
Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) {
if (skipped_round <= MAX_SKIP_GC_COUNT ||
skipped_round * 2 < round) {
segno = NULL_SEGNO;
goto gc_more;
}
if (first_skipped < last_skipped &&
(last_skipped - first_skipped) >
sbi->skipped_gc_rwsem) {
f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since f2fs_commit_inmem_pages() removed the inode from inmem list.
Quoted 11 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.
Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.
Oh, we failed to flush its nodes.
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear ATOMIC_FILE flag in that two error path? I'm not very clear how sqlite handle such error.
Thanks,
Quoted 34 lines
Quoted 6 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; } if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.Quoted 11 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }.
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 62 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.
Quoted 7 lines
Oh, we failed to flush its nodes.
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages() did that, but not in this case.
Quoted 37 lines
Thanks,
Quoted 34 lines
Quoted 6 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; } if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.Quoted 11 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }.
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.
What I mean is, for any error returned from atomic_commit() interface, should userspace application handle it with consistent way, like trigger f2fs_drop_inmem_pages(), so we don't need to handle it inside atomic_commit().
Quoted 40 lines
Quoted 38 lines
Thanks,
Quoted 34 lines
Quoted 6 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; } if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.Quoted 11 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }.
.
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 69 lines · Chao Yu
Quoted 68 lines · Jaegeuk Kim
Quoted 67 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().
f2fs_ioc_abort_volatile_write() will be triggered.
Quoted 41 lines
Quoted 41 lines
Quoted 39 lines
Thanks,
Quoted 34 lines
Quoted 6 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; } if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.Quoted 11 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }.
.
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 73 lines · Jaegeuk Kim
Quoted 72 lines · Chao Yu
Quoted 71 lines · Jaegeuk Kim
Quoted 69 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().
f2fs_ioc_abort_volatile_write()will be triggered.
If userspace can do this, we can get rid of this patch, or am I missing sth?
- f2fs_ioc_abort_volatile_write
- f2fs_drop_inmem_pages
- clear_inode_flag(inode, FI_ATOMIC_FILE);
- fi->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
- stat_dec_atomic_write(inode);
Quoted 44 lines
Quoted 34 lines
Quoted 34 lines
Quoted 34 lines
Thanks,
Quoted 28 lines
Quoted 6 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; } if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.Quoted 7 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2007
Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }.
.
.
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 76 lines · Chao Yu
Quoted 75 lines · Jaegeuk Kim
Quoted 74 lines · Chao Yu
Quoted 72 lines · Jaegeuk Kim
Quoted 62 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Quoted 7 lines
Oh, we failed to flush its nodes.
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Quoted 51 lines
-
f2fs_ioc_abort_volatile_write
-f2fs_drop_inmem_pages
-clear_inode_flag(inode, FI_ATOMIC_FILE);
- fi->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
-stat_dec_atomic_write(inode);Quoted 44 lines
Quoted 34 lines
Quoted 34 lines
Quoted 34 lines
Thanks,
Quoted 28 lines
Quoted 6 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()Quoted 11 lines
if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; } if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.Quoted 7 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2007
Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }.
.
.
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Actually, we can call this.
--- fs/f2fs/file.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
fs/f2fs/file.c +1 -5
@@ -1997,11 +1997,7 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } +f2fs_drop_inmem_pages (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }
Signature · 1 line
2.19.0.605.g01d371f741-goog
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
In reply to Jaegeuk KimQuoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Yes, however, even atomic_write won't be called sometimes... that's why we add handling logic in f2fs_gc().
Quoted 27 lines
Quoted 27 lines
-
f2fs_ioc_abort_volatile_write
-f2fs_drop_inmem_pages
-clear_inode_flag(inode, FI_ATOMIC_FILE);
- fi->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
-stat_dec_atomic_write(inode);Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
Thanks,
Quoted 13 lines
Quoted 13 lines
So my question is why below logic didn't handle such condition well?
f2fs_gc()if (has_not_enough_free_secs(sbi, sec_freed, 0)) { if (skipped_round <= MAX_SKIP_GC_COUNT || skipped_round * 2 < round) { segno = NULL_SEGNO; goto gc_more; }
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
if (first_skipped < last_skipped && (last_skipped - first_skipped) > sbi->skipped_gc_rwsem) { f2fs_drop_inmem_pages_all(sbi, true);This is doing nothing, since
f2fs_commit_inmem_pages()removed the inode from inmem list.
One possible solution is keeping inode in sbi->inode_lock[ATOMIC_FILE] list until f2fs_do_sync_file() return, then f2fs_drop_inmem_pages_all() can find and drop the inode.
Thanks,
Quoted 20 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
segno = NULL_SEGNO; goto gc_more; } if (gc_type == FG_GC && !is_sbi_flag_set(sbi, SBI_CP_DISABLED)) ret = f2fs_write_checkpoint(sbi, &cpc); }Quoted 4 lines
fs/f2fs/file.c +0
-0}else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
.
.
.
.
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 82 lines · Chao Yu
Quoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Yes, however, even
atomic_writewon't be called sometimes... that's why we add handling logic inf2fs_gc().
We need to clean this up like:
If committing atomic pages is failed when doing f2fs_do_sync_file(), we can get commited pages but atomic_file being still set like:
- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c
In that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)
[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)
So, refactor atomic_write flow like this:
1. start_atomic_write
- add inmem_list and set atomic_file
2. write()
- register it in inmem_pages
3. commit_atomic_write
- if no error, f2fs_drop_inmem_pages()
- f2fs_commit_inmme_pages() failed
: __revoked_inmem_pages() was done
- f2fs_do_sync_file failed
: abort_atomic_write later
4. abort_atomic_write
- f2fs_drop_inmem_pages
5. f2fs_drop_inmem_pages
- clear atomic_file
- remove inmem_list
Based on this change, when GC fails to move block in atomic_file, f2fs_drop_inmem_pages_all() can call f2fs_drop_inmem_pages().
--- fs/f2fs/file.c | 17 +++++++++++------ fs/f2fs/segment.c | 24 +++++------------------- 2 files changed, 16 insertions(+), 25 deletions(-)
fs/f2fs/file.c +11 -6
@@ -1830,6 +1830,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) {struct inode *inode =file_inode (filp);int ret; +struct f2fs_inode_info *fi =F2FS_I (inode);if (!inode_owner_or_capable (inode))return -EACCES ;
@@ -1871,10 +1872,17 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } -set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]); +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ +set_inode_flag (inode,FI_ATOMIC_FILE ); +f2fs_update_time (F2FS_I_SB (inode),REQ_TIME );F2FS_I (inode)->inmem_task = current;stat_inc_atomic_write (inode);
@@ -1912,11 +1920,8 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } +if (!ret) +f2fs_drop_inmem_pages (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }
fs/f2fs/segment.c +5 -19
@@ -186,7 +186,6 @@ bool f2fs_need_SSR(struct f2fs_sb_info *sbi)void f2fs_register_inmem_page (struct inode *inode,struct page *page) {struct f2fs_sb_info *sbi =F2FS_I_SB (inode); -struct f2fs_inode_info *fi =F2FS_I (inode);struct inmem_pages *new;f2fs_trace_pid (page);
@@ -200,15 +199,9 @@ void f2fs_register_inmem_page(struct inode *inode, struct page *page)INIT_LIST_HEAD (&new->list);/* increase reference count with clean state */ -mutex_lock (&fi->inmem_lock);get_page (page);list_add_tail (&new->list, &fi->inmem_pages); -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (list_empty (&fi->inmem_ilist)) -list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);inc_page_count (F2FS_I_SB (inode),F2FS_INMEM_PAGES ); -mutex_unlock (&fi->inmem_lock);trace_f2fs_register_inmem_page (page,INMEM ); }
@@ -330,19 +323,17 @@ void f2fs_drop_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock);__revoke_inmem_pages (inode, &fi->inmem_pages, true, false, true); - -if (list_empty (&fi->inmem_pages)) { -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); - }mutex_unlock (&fi->inmem_lock); }clear_inode_flag (inode,FI_ATOMIC_FILE ); fi->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ;stat_dec_atomic_write (inode); + +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (!list_empty (&fi->inmem_ilist)) +list_del_init (&fi->inmem_ilist); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); }void f2fs_drop_inmem_page (struct inode *inode,struct page *page)
@@ -471,11 +462,6 @@ int f2fs_commit_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock); err =__f2fs_commit_inmem_pages (inode); - -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);mutex_unlock (&fi->inmem_lock);clear_inode_flag (inode,FI_ATOMIC_COMMIT );
Signature · 1 line
2.19.0.605.g01d371f741-goog
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 138 lines · Jaegeuk Kim
Quoted 82 lines · Chao Yu
Quoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp)
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Yes, however, even
atomic_writewon't be called sometimes... that's why we add handling logic inf2fs_gc().We need to clean this up like:
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)So, refactor
atomic_writeflow like this:
1.start_atomic_write
- addinmem_listand setatomic_file2.
write()
- register it ininmem_pages3.
commit_atomic_write
- if no error,f2fs_drop_inmem_pages()
-f2fs_commit_inmme_pages()failed
:__revoked_inmem_pages()was done
-f2fs_do_sync_filefailed
:abort_atomic_writelater
Quoted 6 lines
4.
abort_atomic_write
-f2fs_drop_inmem_pages5.
f2fs_drop_inmem_pages
- clearatomic_file
- removeinmem_list
Looks good.
Quoted 27 lines
Based on this change, when GC fails to move block in
atomic_file,f2fs_drop_inmem_pages_all()can callf2fs_drop_inmem_pages().Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 17 +++++++++++------ fs/f2fs/segment.c | 24 +++++------------------- 2 files changed, 16 insertions(+), 25 deletions(-)fs/f2fs/file.c +1
-1@@ -1830,6 +1830,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) {struct inode *inode =file_inode (filp);int ret; +struct f2fs_inode_info *fi =F2FS_I (inode);if (!inode_owner_or_capable (inode))return -EACCES ;@@ -1871,10 +1872,17 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } -set_inode_flag (inode,FI_ATOMIC_FILE );
fs/f2fs/file.c static int f2fs_ioc_start_atomic_write(struct file *filp) · line 1884
Quoted 10 lines
fs/f2fs/file.c +7
-0clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]); +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ +set_inode_flag (inode,FI_ATOMIC_FILE );
Should call up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]) here? Then in f2fs_gc(), we can check FI_ATOMIC_FILE atomically under i_gc_rwsem.
Thanks,
Quoted 74 lines
fs/f2fs/file.c +3
-5+f2fs_update_time (F2FS_I_SB (inode),REQ_TIME );F2FS_I (inode)->inmem_task = current;stat_inc_atomic_write (inode);@@ -1912,11 +1920,8 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } +if (!ret) +f2fs_drop_inmem_pages (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }fs/f2fs/segment.c +5
-14@@ -186,7 +186,6 @@ bool f2fs_need_SSR(struct f2fs_sb_info *sbi)void f2fs_register_inmem_page (struct inode *inode,struct page *page) {struct f2fs_sb_info *sbi =F2FS_I_SB (inode); -struct f2fs_inode_info *fi =F2FS_I (inode);struct inmem_pages *new;f2fs_trace_pid (page);@@ -200,15 +199,9 @@ void f2fs_register_inmem_page(struct inode *inode, struct page *page)INIT_LIST_HEAD (&new->list);/* increase reference count with clean state */ -mutex_lock (&fi->inmem_lock);get_page (page);list_add_tail (&new->list, &fi->inmem_pages); -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (list_empty (&fi->inmem_ilist)) -list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);inc_page_count (F2FS_I_SB (inode),F2FS_INMEM_PAGES ); -mutex_unlock (&fi->inmem_lock);trace_f2fs_register_inmem_page (page,INMEM ); }@@ -330,19 +323,17 @@ void f2fs_drop_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock);__revoke_inmem_pages (inode, &fi->inmem_pages, true, false, true); - -if (list_empty (&fi->inmem_pages)) { -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); - }mutex_unlock (&fi->inmem_lock); }clear_inode_flag (inode,FI_ATOMIC_FILE ); fi->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ;stat_dec_atomic_write (inode); + +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (!list_empty (&fi->inmem_ilist)) +list_del_init (&fi->inmem_ilist); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); }void f2fs_drop_inmem_page (struct inode *inode,struct page *page)@@ -471,11 +462,6 @@ int f2fs_commit_inmem_pages(struct inode *inode)
fs/f2fs/segment.c int f2fs_commit_inmem_pages(struct inode *inode) · line 467
Quoted 10 lines
fs/f2fs/segment.c +0
-5mutex_lock (&fi->inmem_lock); err =__f2fs_commit_inmem_pages (inode); - -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);mutex_unlock (&fi->inmem_lock);clear_inode_flag (inode,FI_ATOMIC_COMMIT );
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 178 lines · Chao Yu
Quoted 138 lines · Jaegeuk Kim
Quoted 82 lines · Chao Yu
Quoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)fs/f2fs/file.c +0
-0@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out;Quoted 14 lines
fs/f2fs/file.c static int f2fs_ioc_commit_atomic_write(struct file *filp) · line 2004
Quoted 11 lines
fs/f2fs/file.c +5
-5ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } + +/* doesn't need to check error */ +clear_inode_flag (inode,FI_ATOMIC_FILE ); +F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; +stat_dec_atomic_write (inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Yes, however, even
atomic_writewon't be called sometimes... that's why we add handling logic inf2fs_gc().We need to clean this up like:
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)So, refactor
atomic_writeflow like this:
1.start_atomic_write
- addinmem_listand setatomic_file2.
write()
- register it ininmem_pages3.
commit_atomic_write
- if no error,f2fs_drop_inmem_pages()
-f2fs_commit_inmme_pages()failed
:__revoked_inmem_pages()was done
-f2fs_do_sync_filefailed
:abort_atomic_writelaterQuoted 6 lines
4.
abort_atomic_write
-f2fs_drop_inmem_pages5.
f2fs_drop_inmem_pages
- clearatomic_file
- removeinmem_listLooks good.
Quoted 30 lines
Based on this change, when GC fails to move block in
atomic_file,f2fs_drop_inmem_pages_all()can callf2fs_drop_inmem_pages().Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 17 +++++++++++------ fs/f2fs/segment.c | 24 +++++------------------- 2 files changed, 16 insertions(+), 25 deletions(-)fs/f2fs/file.c +1
-1@@ -1830,6 +1830,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) {struct inode *inode =file_inode (filp);int ret; +struct f2fs_inode_info *fi =F2FS_I (inode);if (!inode_owner_or_capable (inode))return -EACCES ;@@ -1871,10 +1872,17 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } -set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]);
Quoted 10 lines
fs/f2fs/file.c static int f2fs_ioc_start_atomic_write(struct file *filp) · line 1884
Quoted 7 lines
fs/f2fs/file.c +7
-0+spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ +set_inode_flag (inode,FI_ATOMIC_FILE );Should call up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]) here? Then in
f2fs_gc(), we can checkFI_ATOMIC_FILEatomically underi_gc_rwsem.
Yup.
If committing atomic pages is failed when doing f2fs_do_sync_file(), we can get commited pages but atomic_file being still set like:
- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c
In that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)
[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)
So, refactor atomic_write flow like this:
1. start_atomic_write
- add inmem_list and set atomic_file
2. write()
- register it in inmem_pages
3. commit_atomic_write
- if no error, f2fs_drop_inmem_pages()
- f2fs_commit_inmme_pages() failed
: __revoked_inmem_pages() was done
- f2fs_do_sync_file failed
: abort_atomic_write later
4. abort_atomic_write
- f2fs_drop_inmem_pages
5. f2fs_drop_inmem_pages
- clear atomic_file
- remove inmem_list
Based on this change, when GC fails to move block in atomic_file, f2fs_drop_inmem_pages_all() can call f2fs_drop_inmem_pages().
--- fs/f2fs/file.c | 15 ++++++++++----- fs/f2fs/segment.c | 27 ++++++--------------------- 2 files changed, 16 insertions(+), 26 deletions(-)
fs/f2fs/file.c +10 -5
@@ -1829,6 +1829,8 @@ static int f2fs_ioc_getversion(struct file *filp, unsigned long arg)static int f2fs_ioc_start_atomic_write (struct file *filp) {struct inode *inode =file_inode (filp); +struct f2fs_inode_info *fi =F2FS_I (inode); +struct f2fs_sb_info *sbi =F2FS_I_SB (inode);int ret;if (!inode_owner_or_capable (inode))
@@ -1871,6 +1873,12 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]);
@@ -1912,11 +1920,8 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } +if (!ret) +f2fs_drop_inmem_pages (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }
fs/f2fs/segment.c +6 -21
@@ -185,8 +185,6 @@ bool f2fs_need_SSR(struct f2fs_sb_info *sbi)void f2fs_register_inmem_page (struct inode *inode,struct page *page) { -struct f2fs_sb_info *sbi =F2FS_I_SB (inode); -struct f2fs_inode_info *fi =F2FS_I (inode);struct inmem_pages *new;f2fs_trace_pid (page);
@@ -200,15 +198,9 @@ void f2fs_register_inmem_page(struct inode *inode, struct page *page)INIT_LIST_HEAD (&new->list);/* increase reference count with clean state */ -mutex_lock (&fi->inmem_lock);get_page (page); -list_add_tail (&new->list, &fi->inmem_pages); -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (list_empty (&fi->inmem_ilist)) -list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); +list_add_tail (&new->list, &F2FS_I (inode)->inmem_pages);inc_page_count (F2FS_I_SB (inode),F2FS_INMEM_PAGES ); -mutex_unlock (&fi->inmem_lock);trace_f2fs_register_inmem_page (page,INMEM ); }
@@ -330,19 +322,17 @@ void f2fs_drop_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock);__revoke_inmem_pages (inode, &fi->inmem_pages, true, false, true); - -if (list_empty (&fi->inmem_pages)) { -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); - }mutex_unlock (&fi->inmem_lock); }clear_inode_flag (inode,FI_ATOMIC_FILE ); fi->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ;stat_dec_atomic_write (inode); + +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (!list_empty (&fi->inmem_ilist)) +list_del_init (&fi->inmem_ilist); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); }void f2fs_drop_inmem_page (struct inode *inode,struct page *page)
@@ -471,11 +461,6 @@ int f2fs_commit_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock); err =__f2fs_commit_inmem_pages (inode); - -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);mutex_unlock (&fi->inmem_lock);clear_inode_flag (inode,FI_ATOMIC_COMMIT );
Signature · 1 line
2.19.0.605.g01d371f741-goog
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 306 lines · Jaegeuk Kim
Quoted 178 lines · Chao Yu
Quoted 138 lines · Jaegeuk Kim
Quoted 82 lines · Chao Yu
Quoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 43 lines · Chao Yu
Quoted 42 lines · Jaegeuk Kim
If committing atomic pages is failed when doing f2fs_do_sync_file(), we can
get commited pages but atomic_file being still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATA
f2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096
f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234
f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATA
f2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096
f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234
f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by: Jaegeuk Kim <[email protected]>
---
fs/f2fs/file.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 7ae2f3bd8c2f..68b6da734e5f 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp)
goto err_out;Quoted 14 lines
Quoted 11 lines
ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true);
- if (!ret) {
- clear_inode_flag(inode, FI_ATOMIC_FILE);
- F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
- stat_dec_atomic_write(inode);
- }
+
+ /* doesn't need to check error */
+ clear_inode_flag(inode, FI_ATOMIC_FILE);
+ F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
+ stat_dec_atomic_write(inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause memory leak when we just clear
FI_ATOMIC_FILEflag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Yes, however, even
atomic_writewon't be called sometimes... that's why we add handling logic inf2fs_gc().We need to clean this up like:
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)So, refactor
atomic_writeflow like this:
1.start_atomic_write
- addinmem_listand setatomic_file2.
write()
- register it ininmem_pages3.
commit_atomic_write
- if no error,f2fs_drop_inmem_pages()
-f2fs_commit_inmme_pages()failed
:__revoked_inmem_pages()was done
-f2fs_do_sync_filefailed
:abort_atomic_writelaterQuoted 6 lines
4.
abort_atomic_write
-f2fs_drop_inmem_pages5.
f2fs_drop_inmem_pages
- clearatomic_file
- removeinmem_listLooks good.
Quoted 30 lines
Based on this change, when GC fails to move block in
atomic_file,f2fs_drop_inmem_pages_all()can callf2fs_drop_inmem_pages().Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 17 +++++++++++------ fs/f2fs/segment.c | 24 +++++------------------- 2 files changed, 16 insertions(+), 25 deletions(-)fs/f2fs/file.c +1
-1@@ -1830,6 +1830,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) {struct inode *inode =file_inode (filp);int ret; +struct f2fs_inode_info *fi =F2FS_I (inode);if (!inode_owner_or_capable (inode))return -EACCES ;@@ -1871,10 +1872,17 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } -set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]);Quoted 10 lines
fs/f2fs/file.c static int f2fs_ioc_start_atomic_write(struct file *filp)
Quoted 7 lines
fs/f2fs/file.c +7
-0+spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ +set_inode_flag (inode,FI_ATOMIC_FILE );Should call up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]) here? Then in
f2fs_gc(), we can checkFI_ATOMIC_FILEatomically underi_gc_rwsem.Yup.
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)So, refactor
atomic_writeflow like this:
1.start_atomic_write
- addinmem_listand setatomic_file2.
write()
- register it ininmem_pages3.
commit_atomic_write
- if no error,f2fs_drop_inmem_pages()
-f2fs_commit_inmme_pages()failed
:__revoked_inmem_pages()was done
-f2fs_do_sync_filefailed
:abort_atomic_writelater4.
abort_atomic_write
-f2fs_drop_inmem_pages5.
f2fs_drop_inmem_pages
- clearatomic_file
- removeinmem_listBased on this change, when GC fails to move block in
atomic_file,f2fs_drop_inmem_pages_all()can callf2fs_drop_inmem_pages().Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 15 ++++++++++----- fs/f2fs/segment.c | 27 ++++++--------------------- 2 files changed, 16 insertions(+), 26 deletions(-)fs/f2fs/file.c +10
-5@@ -1829,6 +1829,8 @@ static int f2fs_ioc_getversion(struct file *filp, unsigned long arg)static int f2fs_ioc_start_atomic_write (struct file *filp) {struct inode *inode =file_inode (filp); +struct f2fs_inode_info *fi =F2FS_I (inode); +struct f2fs_sb_info *sbi =F2FS_I_SB (inode);int ret;if (!inode_owner_or_capable (inode))@@ -1871,6 +1873,12 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]);@@ -1912,11 +1920,8 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } +if (!ret) +f2fs_drop_inmem_pages (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }fs/f2fs/segment.c +0
-2@@ -185,8 +185,6 @@ bool f2fs_need_SSR(struct f2fs_sb_info *sbi)void f2fs_register_inmem_page (struct inode *inode,struct page *page) { -struct f2fs_sb_info *sbi =F2FS_I_SB (inode); -struct f2fs_inode_info *fi =F2FS_I (inode);struct inmem_pages *new;f2fs_trace_pid (page);
fs/f2fs/segment.c void f2fs_register_inmem_page(struct inode *inode, struct page *page) · line 203
Quoted 4 lines
fs/f2fs/segment.c +0
-1@@ -200,15 +198,9 @@ void f2fs_register_inmem_page(struct inode *inode, struct page *page)INIT_LIST_HEAD (&new->list);/* increase reference count with clean state */ -mutex_lock (&fi->inmem_lock);
The lock is used to protect F2FS_I(inode)->inmem_pages list... it should be kept?
Thanks,
Quoted 50 lines
fs/f2fs/segment.c +6
-18get_page (page); -list_add_tail (&new->list, &fi->inmem_pages); -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (list_empty (&fi->inmem_ilist)) -list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); +list_add_tail (&new->list, &F2FS_I (inode)->inmem_pages);inc_page_count (F2FS_I_SB (inode),F2FS_INMEM_PAGES ); -mutex_unlock (&fi->inmem_lock);trace_f2fs_register_inmem_page (page,INMEM ); }@@ -330,19 +322,17 @@ void f2fs_drop_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock);__revoke_inmem_pages (inode, &fi->inmem_pages, true, false, true); - -if (list_empty (&fi->inmem_pages)) { -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); - }mutex_unlock (&fi->inmem_lock); }clear_inode_flag (inode,FI_ATOMIC_FILE ); fi->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ;stat_dec_atomic_write (inode); + +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (!list_empty (&fi->inmem_ilist)) +list_del_init (&fi->inmem_ilist); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); }void f2fs_drop_inmem_page (struct inode *inode,struct page *page)@@ -471,11 +461,6 @@ int f2fs_commit_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock); err =__f2fs_commit_inmem_pages (inode); - -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);mutex_unlock (&fi->inmem_lock);clear_inode_flag (inode,FI_ATOMIC_COMMIT );
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 305 lines · Chao Yu
Quoted 304 lines · Jaegeuk Kim
Quoted 178 lines · Chao Yu
Quoted 138 lines · Jaegeuk Kim
Quoted 82 lines · Chao Yu
Quoted 81 lines · Jaegeuk Kim
Quoted 80 lines · Chao Yu
Quoted 79 lines · Jaegeuk Kim
Quoted 77 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 59 lines · Jaegeuk Kim
Quoted 57 lines · Chao Yu
On 2019/9/9 9:25, Jaegeuk Kim wrote:
If committing atomic pages is failed when doing f2fs_do_sync_file(), we can
get commited pages but atomic_file being still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATA
f2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096
f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234
f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2c
f2fs_submit_page_bio: dev = (253,7), ino = 2, page_index = 0x2359a8, oldaddr = 0x2359a8, newaddr = 0x2359a8, rw = READ(), type = COLD_DATA
f2fs_submit_read_bio: dev = (253,7)/(253,7), rw = READ(), DATA, sector = 18533696, size = 4096
f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1, ofs_unit = 1, pre_victim_secno = 4355, prefree = 0, free = 234
f2fs_iget: dev = (253,7), ino = 6247, pino = 5845, i_mode = 0x81b0, i_size = 319488, i_nlink = 1, i_blocks = 624, i_advise = 0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)Signed-off-by: Jaegeuk Kim <[email protected]>
---
fs/f2fs/file.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 7ae2f3bd8c2f..68b6da734e5f 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1997,11 +1997,11 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp)
goto err_out;ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true);
- if (!ret) {
- clear_inode_flag(inode, FI_ATOMIC_FILE);
- F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
- stat_dec_atomic_write(inode);
- }
+
+ /* doesn't need to check error */
+ clear_inode_flag(inode, FI_ATOMIC_FILE);
+ F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
+ stat_dec_atomic_write(inode);If there are still valid atomic write pages linked in .inmem_pages, it may cause
memory leak when we just clear FI_ATOMIC_FILE flag.Quoted 1 line
f2fs_commit_inmem_pages()should have flushed them.Oh, we failed to flush its nodes.
Quoted 7 lines
Quoted 5 lines
However we won't clear such info if we failed to flush inmen pages, it looks inconsistent.
Any interface needed to drop inmem pages or clear
ATOMIC_FILEflag in that two error path? I'm not very clear how sqlite handle such error.
f2fs_drop_inmem_pages()did that, but not in this case.What I mean is, for any error returned from
atomic_commit()interface, should userspace application handle it with consistent way, like triggerf2fs_drop_inmem_pages(), so we don't need to handle it insideatomic_commit().Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
f2fs_ioc_abort_volatile_write()will be triggered.If userspace can do this, we can get rid of this patch, or am I missing sth?
We don't know when that will come. And, other threads are waiting for GC here.
Yes, however, even
atomic_writewon't be called sometimes... that's why we add handling logic inf2fs_gc().We need to clean this up like:
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)So, refactor
atomic_writeflow like this:
1.start_atomic_write
- addinmem_listand setatomic_file2.
write()
- register it ininmem_pages3.
commit_atomic_write
- if no error,f2fs_drop_inmem_pages()
-f2fs_commit_inmme_pages()failed
:__revoked_inmem_pages()was done
-f2fs_do_sync_filefailed
:abort_atomic_writelaterQuoted 6 lines
4.
abort_atomic_write
-f2fs_drop_inmem_pages5.
f2fs_drop_inmem_pages
- clearatomic_file
- removeinmem_listLooks good.
Quoted 30 lines
Based on this change, when GC fails to move block in
atomic_file,f2fs_drop_inmem_pages_all()can callf2fs_drop_inmem_pages().Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 17 +++++++++++------ fs/f2fs/segment.c | 24 +++++------------------- 2 files changed, 16 insertions(+), 25 deletions(-)fs/f2fs/file.c +1
-1@@ -1830,6 +1830,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) {struct inode *inode =file_inode (filp);int ret; +struct f2fs_inode_info *fi =F2FS_I (inode);if (!inode_owner_or_capable (inode))return -EACCES ;@@ -1871,10 +1872,17 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } -set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]);Quoted 10 lines
fs/f2fs/file.c static int f2fs_ioc_start_atomic_write(struct file *filp)
Quoted 7 lines
fs/f2fs/file.c +7
-0+spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ +set_inode_flag (inode,FI_ATOMIC_FILE );Should call up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]) here? Then in
f2fs_gc(), we can checkFI_ATOMIC_FILEatomically underi_gc_rwsem.Yup.
If committing atomic pages is failed when doing
f2fs_do_sync_file(), we can get commited pages butatomic_filebeing still set like:- inmem: 0, atomic IO: 4 (Max. 10), volatile IO: 0 (Max. 0)
If GC selects this block, we can get an infinite loop like this:
f2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cf2fs_submit_page_bio: dev = (253,7), ino = 2,page_index=0x2359a8, oldaddr =0x2359a8, newaddr =0x2359a8, rw =READ(), type =COLD_DATAf2fs_submit_read_bio: dev = (253,7)/(253,7), rw =READ(), DATA, sector = 18533696, size = 4096f2fs_get_victim: dev = (253,7), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 4355, cost = 1,ofs_unit= 1,pre_victim_secno= 4355, prefree = 0, free = 234f2fs_iget: dev = (253,7), ino = 6247, pino = 5845,i_mode=0x81b0,i_size= 319488,i_nlink= 1,i_blocks= 624,i_advise=0x2cIn that moment, we can observe:
[Before]
Try to move 5084219 blocks (BG: 384508)
- data blocks : 4962373 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4534686 (10)[After]
Try to move 5088973 blocks (BG: 384508)
- data blocks : 4967127 (274483)
- node blocks : 121846 (110025)
Skipped : atomic write 4539440 (10)So, refactor
atomic_writeflow like this:
1.start_atomic_write
- addinmem_listand setatomic_file2.
write()
- register it ininmem_pages3.
commit_atomic_write
- if no error,f2fs_drop_inmem_pages()
-f2fs_commit_inmme_pages()failed
:__revoked_inmem_pages()was done
-f2fs_do_sync_filefailed
:abort_atomic_writelater4.
abort_atomic_write
-f2fs_drop_inmem_pages5.
f2fs_drop_inmem_pages
- clearatomic_file
- removeinmem_listBased on this change, when GC fails to move block in
atomic_file,f2fs_drop_inmem_pages_all()can callf2fs_drop_inmem_pages().Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/file.c | 15 ++++++++++----- fs/f2fs/segment.c | 27 ++++++--------------------- 2 files changed, 16 insertions(+), 26 deletions(-)fs/f2fs/file.c +10
-5@@ -1829,6 +1829,8 @@ static int f2fs_ioc_getversion(struct file *filp, unsigned long arg)static int f2fs_ioc_start_atomic_write (struct file *filp) {struct inode *inode =file_inode (filp); +struct f2fs_inode_info *fi =F2FS_I (inode); +struct f2fs_sb_info *sbi =F2FS_I_SB (inode);int ret;if (!inode_owner_or_capable (inode))@@ -1871,6 +1873,12 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) goto out; } +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (list_empty (&fi->inmem_ilist)) +list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); + +/* add inode in inmem_list first and set atomic_file */ set_inode_flag (inode,FI_ATOMIC_FILE );clear_inode_flag (inode,FI_ATOMIC_REVOKE_REQUEST );up_write (&F2FS_I (inode)->i_gc_rwsem[WRITE ]);@@ -1912,11 +1920,8 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) goto err_out; ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,0 , true); -if (!ret) { -clear_inode_flag (inode,FI_ATOMIC_FILE ); -F2FS_I (inode)->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ; -stat_dec_atomic_write (inode); - } +if (!ret) +f2fs_drop_inmem_pages (inode); }else { ret =f2fs_do_sync_file (filp,0 ,LLONG_MAX ,1 , false); }fs/f2fs/segment.c +0
-2@@ -185,8 +185,6 @@ bool f2fs_need_SSR(struct f2fs_sb_info *sbi)void f2fs_register_inmem_page (struct inode *inode,struct page *page) { -struct f2fs_sb_info *sbi =F2FS_I_SB (inode); -struct f2fs_inode_info *fi =F2FS_I (inode);struct inmem_pages *new;
Quoted 8 lines
fs/f2fs/segment.c void f2fs_register_inmem_page(struct inode *inode, struct page *page) · line 203
Quoted 6 lines
fs/f2fs/segment.c +0
-1f2fs_trace_pid (page);@@ -200,15 +198,9 @@ void f2fs_register_inmem_page(struct inode *inode, struct page *page)INIT_LIST_HEAD (&new->list);/* increase reference count with clean state */ -mutex_lock (&fi->inmem_lock);The lock is used to protect F2FS_I(inode)->inmem_pages list... it should be kept?
Urg.. yup. I added.
Quoted 53 lines
Thanks,
Quoted 50 lines
fs/f2fs/segment.c +6
-18get_page (page); -list_add_tail (&new->list, &fi->inmem_pages); -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (list_empty (&fi->inmem_ilist)) -list_add_tail (&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE ]); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); +list_add_tail (&new->list, &F2FS_I (inode)->inmem_pages);inc_page_count (F2FS_I_SB (inode),F2FS_INMEM_PAGES ); -mutex_unlock (&fi->inmem_lock);trace_f2fs_register_inmem_page (page,INMEM ); }@@ -330,19 +322,17 @@ void f2fs_drop_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock);__revoke_inmem_pages (inode, &fi->inmem_pages, true, false, true); - -if (list_empty (&fi->inmem_pages)) { -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); - }mutex_unlock (&fi->inmem_lock); }clear_inode_flag (inode,FI_ATOMIC_FILE ); fi->i_gc_failures[GC_FAILURE_ATOMIC ] =0 ;stat_dec_atomic_write (inode); + +spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); +if (!list_empty (&fi->inmem_ilist)) +list_del_init (&fi->inmem_ilist); +spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]); }void f2fs_drop_inmem_page (struct inode *inode,struct page *page)@@ -471,11 +461,6 @@ int f2fs_commit_inmem_pages(struct inode *inode)mutex_lock (&fi->inmem_lock); err =__f2fs_commit_inmem_pages (inode); - -spin_lock (&sbi->inode_lock[ATOMIC_FILE ]); -if (!list_empty (&fi->inmem_ilist)) -list_del_init (&fi->inmem_ilist); -spin_unlock (&sbi->inode_lock[ATOMIC_FILE ]);mutex_unlock (&fi->inmem_lock);clear_inode_flag (inode,FI_ATOMIC_COMMIT );
Chao Yu
Reviewed-byRe: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
Quoted 2 lines · Jaegeuk Kim
Quoted 1 line
The lock is used to protect F2FS_I(inode)->inmem_pages list... it should be kept?
Urg.. yup. I added.
Thanks,
Quoted 1 line
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Thanks,
Quoted 31 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/gc.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)fs/f2fs/gc.c +3
-3@@ -274,6 +274,9 @@ static unsigned int get_cb_cost(struct f2fs_sb_info *sbi, unsigned int segno)static inline unsigned int get_gc_cost (struct f2fs_sb_info *sbi,unsigned int segno,struct victim_sel_policy *p) { +if (sbi->cur_victim_sec ==GET_SEC_FROM_SEG (sbi, segno)) +return UINT_MAX ; +if (p->alloc_mode ==SSR )return get_seg_entry (sbi, segno)->ckpt_valid_blocks;@@ -1326,9 +1329,6 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; } -if (gc_type ==FG_GC ) - sbi->cur_victim_sec =NULL_SEGNO ; -if (sync) goto stop;
Jaegeuk Kim
maintainerQuoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by sec_usage_check(). We may be able to avoid this only in the suspicious loop?
--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/f2fs/gc.c +1 -1
@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; } -if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed) sbi->cur_victim_sec =NULL_SEGNO ;if (sync)
Signature · 1 line
2.19.0.605.g01d371f741-goog
Quoted 19 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 5 lines
fs/f2fs/gc.c +1
-1@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; } -if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)
That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync)
Jaegeuk Kim
maintainerQuoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }
Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to is_alive or atomic_file.
Quoted 7 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync)
Quoted 31 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
Quoted 2 lines
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
Jaegeuk Kim
maintainerQuoted 32 lines · Chao Yu
Quoted 31 lines · Jaegeuk Kim
Quoted 29 lines · Chao Yu
Quoted 19 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 5 lines
fs/f2fs/gc.c +1
-1@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; } -if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
Quoted 8 lines
Quoted 2 lines
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.
Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
What I mean is if there is a bug (maybe in is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).
But I guess the benefit of this change is, if FGGC fails to migrate block due to i_gc_rwsem race, selecting another section and later retrying previous one may avoid lock race, right?
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
Jaegeuk Kim
maintainerQuoted 48 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
Quoted 8 lines
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?
In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Quoted 21 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
Quoted 54 lines · Jaegeuk Kim
Quoted 53 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).
Quoted 6 lines
Quoted 3 lines
But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Yup,
One more concern is would this cur_victim_sec remain after FGGC? then BGGC/SSR will always skip the section cur_victim_sec points to.
So could we reset cur_victim_sec in the end of FGGC?
Thanks,
Quoted 24 lines
Quoted 22 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
.
Jaegeuk Kim
maintainerQuoted 59 lines · Chao Yu
Quoted 58 lines · Jaegeuk Kim
Quoted 56 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?
Quoted 7 lines
Quoted 2 lines
In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Yup,
One more concern is would this
cur_victim_secremain after FGGC? then BGGC/SSR will always skip the sectioncur_victim_secpoints to.
Then, we can get another loop before using it by BGGC/SSR.
Quoted 29 lines
So could we reset
cur_victim_secin the end of FGGC?Thanks,
Quoted 24 lines
Quoted 22 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
.
Quoted 63 lines · Jaegeuk Kim
Quoted 62 lines · Chao Yu
Quoted 60 lines · Jaegeuk Kim
Quoted 48 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
Quoted 8 lines
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Quoted 6 lines
Quoted 4 lines
Yup,
One more concern is would this
cur_victim_secremain after FGGC? then BGGC/SSR will always skip the sectioncur_victim_secpoints to.Then, we can get another loop before using it by BGGC/SSR.
I guess I didn't catch your point, do you mean, if we reset it in the end of FGGC, we may encounter the loop during BGGC/SSR?
I meant:
f2fs_gc()
...
+ if (gc_type == FG_GC)
+ sbi->cur_victim_sec = NULL_SEGNO;
mutex_unlock(&sbi->gc_mutex); put_gc_inode(&gc_list);
...
Thanks,
Quoted 32 lines
Quoted 30 lines
So could we reset
cur_victim_secin the end of FGGC?Thanks,
Quoted 24 lines
Quoted 22 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
.
.
Jaegeuk Kim
maintainerQuoted 66 lines · Chao Yu
Quoted 65 lines · Jaegeuk Kim
Quoted 64 lines · Chao Yu
Quoted 54 lines · Jaegeuk Kim
Quoted 53 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).Quoted 6 lines
Quoted 3 lines
But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Yup,
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
One more concern is would this
cur_victim_secremain after FGGC? then BGGC/SSR will always skip the sectioncur_victim_secpoints to.Then, we can get another loop before using it by BGGC/SSR.
I guess I didn't catch your point, do you mean, if we reset it in the end of FGGC, we may encounter the loop during BGGC/SSR?
FGGC failed in a loop and last victim was remained in cur_victim_sec. Next FGGC kicked in and did the same thing again. I don't expect BGGC/SSR wants to select this victim much, since it will have CB policy.
Quoted 48 lines
I meant:
f2fs_gc()
...(quoted diff) +2
-0+ if (gc_type == FG_GC) + sbi->cur_victim_sec = NULL_SEGNO;mutex_unlock(&sbi->gc_mutex); put_gc_inode(&gc_list);...
Thanks,
Quoted 32 lines
Quoted 30 lines
So could we reset
cur_victim_secin the end of FGGC?Thanks,
Quoted 24 lines
Quoted 22 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
.
.
Quoted 70 lines · Jaegeuk Kim
Quoted 69 lines · Chao Yu
Quoted 68 lines · Jaegeuk Kim
Quoted 66 lines · Chao Yu
Quoted 54 lines · Jaegeuk Kim
Quoted 53 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
Quoted 1 line · Jaegeuk Kim
GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently.
Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).Quoted 6 lines
Quoted 3 lines
But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Yup,
One more concern is would this
cur_victim_secremain after FGGC? then BGGC/SSR will always skip the sectioncur_victim_secpoints to.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Then, we can get another loop before using it by BGGC/SSR.
I guess I didn't catch your point, do you mean, if we reset it in the end of FGGC, we may encounter the loop during BGGC/SSR?
FGGC failed in a loop and last victim was remained in
cur_victim_sec.
It won't run into a loop because we keep below condition?
+ if (gc_type == FG_GC && seg_freed)
+ sbi->cur_victim_sec = NULL_SEGNO;
if (sync) goto stop;
I meant add below logic in addition:
+ if (gc_type == FG_GC)
+ sbi->cur_victim_sec = NULL_SEGNO;
mutex_unlock(&sbi->gc_mutex);
Thanks,
Quoted 53 lines
Next FGGC kicked in and did the same thing again. I don't expect BGGC/SSR wants to select this victim much, since it will have CB policy.
Quoted 9 lines
I meant:
f2fs_gc()
...(quoted diff) +2
-0+ if (gc_type == FG_GC) + sbi->cur_victim_sec = NULL_SEGNO;Quoted 39 lines
mutex_unlock(&sbi->gc_mutex); put_gc_inode(&gc_list);...
Thanks,
Quoted 32 lines
Quoted 30 lines
So could we reset
cur_victim_secin the end of FGGC?Thanks,
Quoted 24 lines
Quoted 22 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
.
.
.
Jaegeuk Kim
maintainerQuoted 71 lines · Chao Yu
Quoted 70 lines · Jaegeuk Kim
Quoted 69 lines · Chao Yu
Quoted 68 lines · Jaegeuk Kim
Quoted 66 lines · Chao Yu
Quoted 54 lines · Jaegeuk Kim
Quoted 53 lines · Chao Yu
Quoted 36 lines · Jaegeuk Kim
Quoted 35 lines · Chao Yu
Quoted 33 lines · Jaegeuk Kim
Quoted 23 lines · Chao Yu
Quoted 22 lines · Jaegeuk Kim
Quoted 5 lines · Chao Yu
On 2019/9/9 9:25, Jaegeuk Kim wrote:
GC must avoid select the same victim again.Blocks in previous victim will occupy addition free segment, I doubt after this
change, FGGC may encounter out-of-free space issue more frequently.Hmm, actually this change seems wrong by
sec_usage_check(). We may be able to avoid this only in the suspicious loop?--- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/f2fs/gc.c +0
-0@@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; }Quoted 6 lines
fs/f2fs/gc.c int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, · line 1329
Quoted 2 lines
fs/f2fs/gc.c +1
-1-if (gc_type ==FG_GC ) +if (gc_type ==FG_GC && seg_freed)That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose.
I've checked the Sahitya's patch. So, it seems the problem can happen due to
is_aliveoratomic_file.Quoted 8 lines
Quoted 5 lines
For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks.
How about keeping it as it is to find potential bug.
I think it'd be fine to merge this. Could you check the above scenario in more detail?
I haven't saw this in real scenario yet.
What I mean is if there is a bug (maybe in
is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...).Quoted 6 lines
Quoted 3 lines
But I guess the benefit of this change is, if FGGC fails to migrate block due to
i_gc_rwsemrace, selecting another section and later retrying previous one may avoid lock race, right?In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only.
Yup,
One more concern is would this
cur_victim_secremain after FGGC? then BGGC/SSR will always skip the sectioncur_victim_secpoints to.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Then, we can get another loop before using it by BGGC/SSR.
I guess I didn't catch your point, do you mean, if we reset it in the end of FGGC, we may encounter the loop during BGGC/SSR?
FGGC failed in a loop and last victim was remained in
cur_victim_sec.It won't run into a loop because we keep below condition?
The following FGGC will be likely to select this victim again, which doesn't mean "this loop" but "loop of f2fs_gc".
Quoted 69 lines
(quoted diff) +2
-0+ if (gc_type == FG_GC && seg_freed) + sbi->cur_victim_sec = NULL_SEGNO;if (sync) goto stop;I meant add below logic in addition:
(quoted diff) +2
-0+ if (gc_type == FG_GC) + sbi->cur_victim_sec = NULL_SEGNO;mutex_unlock(&sbi->gc_mutex);Thanks,
Quoted 53 lines
Next FGGC kicked in and did the same thing again. I don't expect BGGC/SSR wants to select this victim much, since it will have CB policy.
Quoted 9 lines
I meant:
f2fs_gc()
...(quoted diff) +2
-0+ if (gc_type == FG_GC) + sbi->cur_victim_sec = NULL_SEGNO;Quoted 39 lines
mutex_unlock(&sbi->gc_mutex); put_gc_inode(&gc_list);...
Thanks,
Quoted 32 lines
Quoted 30 lines
So could we reset
cur_victim_secin the end of FGGC?Thanks,
Quoted 24 lines
Quoted 22 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 13 lines
Thanks,
Quoted 10 lines
Quoted 8 lines
Thanks,
Quoted 4 lines
fs/f2fs/gc.c +0
-0sbi->cur_victim_sec =NULL_SEGNO ;if (sync).
.
.
.
.