Skip to contentSkip to conversation

f2fs: fix FG GC failure when file in victim is pinned

PATCH In review Jiucheng Xu 2 participants 5 messages Jun 20 – Jun 26 F2FS file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 f2fs: fix FG GC failure when file in victim is pinned

When continuous write operations occur in the system, BG GC fails to work. This leads to large dirty_segments and small free_segments. If fallocate() is performed on a pinned file with the allocated space exceeding the free_segment, FG_GC reclamation fails.

The reason is that the file corresponding to the block in the victim is pinned, causing gc_data_segment() to fail. Since the condition sec_freed < gc_control->nr_free_secs isn't satisfied, GC stops, resulting in the failure of f2fs_fallocate() allocation.

Setting gc_control->nr_free_secs = 1 make FG GC continue searching for new victim.

---
 fs/f2fs/file.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/f2fs/file.c +1 -1
index 8acdd94272a0ced448e0ba21635d702cfec10682..3e49a73bbf3a184a314e97bff9509a66c27eac00 100644
@@ -1883,7 +1883,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
 			.init_gc_type = FG_GC,
 			.should_migrate_blocks = false,
 			.err_gc_skipped = true,
-			.nr_free_secs = 0 };
+			.nr_free_secs = 1 };
 	pgoff_t pg_start, pg_end;
 	loff_t new_size;
 	loff_t off_end;

---
base-commit: b51f606aa323d553d786ed681a213f134dc688d6
change-id: 20260620-origin-dev-99cdccc83800

Best regards,

Signature · 1 line
Jiucheng Xu <[email protected]>
maintainer
#1 lore raw
Quoted 9 lines · On 6/20/26 17:34, Jiucheng Xu via B4 Relay wrote:

From: Jiucheng Xu <[email protected]>

When continuous write operations occur in the system, BG GC fails to work. This leads to large dirty_segments and small free_segments. If fallocate() is performed on a pinned file with the allocated space exceeding the free_segment, FG_GC reclamation fails.

The reason is that the file corresponding to the block in the victim is pinned, causing gc_data_segment() to fail. Since the condition sec_freed

Jiucheng,

pinned file should be aligned to section size, why there is fragmented blocks of pinfile locates in dirty sections?

Quoted 5 lines

< gc_control->nr_free_secs isn't satisfied, GC stops, resulting in the failure of f2fs_fallocate() allocation.

Setting gc_control->nr_free_secs = 1 make FG GC continue searching for new victim.

Maybe we can try this instead of changing f2fs_expand_inode_data() logic:
1. call fggc via ioctl or trigger urgent gc via sysfs
2. fallocate on pinfile, goto 1) if it failed

But, anyway, I suspect it's risk, if there is no normal dirty section, FGGC will try to call f2fs_unpin_all_sections(), then migrate dirty section which has pinned blocks, that will cause more damage.

Can you please figure out why pinfile is fragmented first...

Thanks,

Quoted 25 lines
---
  fs/f2fs/file.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
fs/f2fs/file.c +1 -2
index 8acdd94272a0ced448e0ba21635d702cfec10682..3e49a73bbf3a184a314e97bff9509a66c27eac00 100644
@@ -1883,7 +1883,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
  			.init_gc_type = FG_GC,
  			.should_migrate_blocks = false,
  			.err_gc_skipped = true,
-			.nr_free_secs = 0 };
+			.nr_free_secs = 1 };
  	pgoff_t pg_start, pg_end;
  	loff_t new_size;
  	loff_t off_end;
 
---

base-commit: b51f606aa323d553d786ed681a213f134dc688d6
change-id: 20260620-origin-dev-99cdccc83800

Best regards,

#2 lore raw
Quoted 30 lines · Chao Yu

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/ LearnAboutSenderIdentification ]

[ EXTERNAL EMAIL ]

Quoted 9 lines · On 6/20/26 17:34, Jiucheng Xu via B4 Relay wrote:

From: Jiucheng Xu <[email protected]>

When continuous write operations occur in the system, BG GC fails to work. This leads to large dirty_segments and small free_segments. If fallocate() is performed on a pinned file with the allocated space exceeding the free_segment, FG_GC reclamation fails.

The reason is that the file corresponding to the block in the victim is pinned, causing gc_data_segment() to fail. Since the condition sec_freed

Jiucheng,

pinned file should be aligned to section size, why there is fragmented blocks of pinfile locates in dirty sections?

Quoted 5 lines

< gc_control->nr_free_secs isn't satisfied, GC stops, resulting in the failure of f2fs_fallocate() allocation.

Setting gc_control->nr_free_secs = 1 make FG GC continue searching for new victim.

Maybe we can try this instead of changing f2fs_expand_inode_data() logic:
1. call fggc via ioctl or trigger urgent gc via sysfs
2. fallocate on pinfile, goto 1) if it failed

Quoted 5 lines

But, anyway, I suspect it's risk, if there is no normal dirty section, FGGC will try to call f2fs_unpin_all_sections(), then migrate dirty section which has pinned blocks, that will cause more damage.

Can you please figure out why pinfile is fragmented first...

Hi Chao,

Thanks for your feedback. I just do a test to simulate the Android OTA createCowImage failed when dirty_segments is too large.

Simple reproduction steps:
1. In my case, dirty_segments=969, free_segments=96, free space = 1.7G, section:segment = 1:1
2. touch a.bin
3. f2fs_io pinfile set a.bin
4. fallocate -l 1.5G a.bin

And f2fs_fallocate() returns -11.

Although the urgent mode work fine, but it is not very convenient to control since AOPS has a great deal of code that calls fallocate.

I have tried 5 times to fallocate(), but GC always selects the same victim segno. So I think it is an issue.

The trace:
f2fs_get_victim: dev = (254,0), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 624, cost = 19...

f2fs_gc_end: dev = (254,0), ret = 0, seg_freed = 0, sec_freed = 0, nodes = 117, dents = 0, imeta = 98, free_sec:50, free_seg:50, rsv_seg:47, prefree_seg:47

I dump the failed victim segno, block offset in segment and related inode ino:
do_garbage_collect: ino=33902 segno=624 off=132

The dump of pinfile 33902 info:
main_blkaddr [0x 2600 : 9728]
i_inline [0x 61 : 97]
inline shows the file is pinned (0x60)

i_addr[0x9]                             [0x   2f7b2 : 194482]
i_addr[0xa]                             [0x   2f7b3 : 194483]
i_addr[0xb]                             [0x   2f7b4 : 194484]
...
i_addr[0x1d]                            [0x   2f7c6 : 194502]
i_addr[0x1e]                            [0x   465af : 288175]
i_addr[0x1f]                            [0x   465b0 : 288176]
...
i_addr[0x21]                            [0x   465b2 : 288178]
i_addr[0x22]                            [0x   5046d : 328813]
i_addr[0x23]                            [0x   5046e : 328814]
i_addr[0x24]                            [0x   505b6 : 329142]
i_addr[0x25]                            [0x   505b7 : 329143]
i_addr[0x26]                            [0x   505b8 : 329144]
i_addr[0x27]                            [0x   50684 : 329348]
i_addr[0x28]                            [0x   50685 : 329349]

The file is created by logcat,and looks like indeed fragmented.

Calculate the block address:
624 * 512 + 132 + 9728(main_blkaddr) = 329348

So the i_addr[0x27] = 329348 is the failed block.

Regarding "why the pinfile is fragmented" you said, we use v5.15 + android U. Is it that v5.15 lacks some patches for special handling of pinfiles?

Thanks,

Quoted 30 lines

Thanks,

Quoted 26 lines

---
  fs/f2fs/file.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
fs/f2fs/file.c +1 -1
index 8acdd94272a0ced448e0ba21635d702cfec10682..3e49a73bbf3a184a314e97bff9509a66c27eac00 100644
@@ -1883,7 +1883,7 @@ static int f2fs_expand_inode_data(struct inode 
 *inode, loff_t offset,
                       .init_gc_type = FG_GC,
                       .should_migrate_blocks = false,
                       .err_gc_skipped = true,
-                     .nr_free_secs = 0 };
+                     .nr_free_secs = 1 };
       pgoff_t pg_start, pg_end;
       loff_t new_size;

      loff_t off_end;

---
base-commit: b51f606aa323d553d786ed681a213f134dc688d6
change-id: 20260620-origin-dev-99cdccc83800

Best regards,

maintainer
#3 lore raw
Quoted 85 lines · Jiucheng Xu
Quoted 30 lines · Chao Yu

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/ LearnAboutSenderIdentification ]

[ EXTERNAL EMAIL ]

Quoted 9 lines · On 6/20/26 17:34, Jiucheng Xu via B4 Relay wrote:

From: Jiucheng Xu <[email protected]>

When continuous write operations occur in the system, BG GC fails to work. This leads to large dirty_segments and small free_segments. If fallocate() is performed on a pinned file with the allocated space exceeding the free_segment, FG_GC reclamation fails.

The reason is that the file corresponding to the block in the victim is pinned, causing gc_data_segment() to fail. Since the condition sec_freed

Jiucheng,

pinned file should be aligned to section size, why there is fragmented blocks of pinfile locates in dirty sections?

Quoted 5 lines

< gc_control->nr_free_secs isn't satisfied, GC stops, resulting in the failure of f2fs_fallocate() allocation.

Setting gc_control->nr_free_secs = 1 make FG GC continue searching for new victim.

Maybe we can try this instead of changing f2fs_expand_inode_data() logic:
1. call fggc via ioctl or trigger urgent gc via sysfs
2. fallocate on pinfile, goto 1) if it failed

Quoted 5 lines

But, anyway, I suspect it's risk, if there is no normal dirty section, FGGC will try to call f2fs_unpin_all_sections(), then migrate dirty section which has pinned blocks, that will cause more damage.

Can you please figure out why pinfile is fragmented first...

Hi Chao,

Thanks for your feedback. I just do a test to simulate the Android OTA createCowImage failed when dirty_segments is too large.

Simple reproduction steps:
1. In my case, dirty_segments=969, free_segments=96, free space = 1.7G, section:segment = 1:1
2. touch a.bin
3. f2fs_io pinfile set a.bin
4. fallocate -l 1.5G a.bin

And f2fs_fallocate() returns -11.

Although the urgent mode work fine, but it is not very convenient to control since AOPS has a great deal of code that calls fallocate.

I have tried 5 times to fallocate(), but GC always selects the same victim segno. So I think it is an issue.

The trace:
f2fs_get_victim: dev = (254,0), type = No TYPE, policy = (Foreground GC, LFS-mode, Greedy), victim = 624, cost = 19...

f2fs_gc_end: dev = (254,0), ret = 0, seg_freed = 0, sec_freed = 0, nodes = 117, dents = 0, imeta = 98, free_sec:50, free_seg:50, rsv_seg:47, prefree_seg:47

I dump the failed victim segno, block offset in segment and related inode ino:
do_garbage_collect: ino=33902 segno=624 off=132

The dump of pinfile 33902 info:
main_blkaddr                            [0x    2600 : 9728] i_inline                                [0x      61 : 97] inline shows the file is pinned (0x60)
i_addr[0x9]                             [0x   2f7b2 : 194482] i_addr[0xa]                             [0x   2f7b3 : 194483] i_addr[0xb]                             [0x   2f7b4 : 194484]
...
i_addr[0x1d]                            [0x   2f7c6 : 194502] i_addr[0x1e]                            [0x   465af : 288175] i_addr[0x1f]                            [0x   465b0 : 288176]
...
i_addr[0x21]                            [0x   465b2 : 288178] i_addr[0x22]                            [0x   5046d : 328813] i_addr[0x23]                            [0x   5046e : 328814] i_addr[0x24]                            [0x   505b6 : 329142] i_addr[0x25]                            [0x   505b7 : 329143] i_addr[0x26]                            [0x   505b8 : 329144] i_addr[0x27]                            [0x   50684 : 329348] i_addr[0x28]                            [0x   50685 : 329349]

Quoted 8 lines

The file is created by logcat,and looks like indeed fragmented.

Calculate the block address:
624 * 512 + 132 + 9728(main_blkaddr) = 329348

So the i_addr[0x27] = 329348 is the failed block.

Regarding "why the pinfile is fragmented" you said, we use v5.15 + android U. Is it that v5.15 lacks some patches for special handling of pinfiles?

Ah, I think you can unpin log file created by logcat, the flag is added in aosp/1014260 to avoid fragmentation in filesystem (), but it's gone in aosp/43c6d76a

Thanks,

Quoted 34 lines

Thanks,

Quoted 29 lines

Thanks,

Quoted 25 lines

---
  fs/f2fs/file.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
fs/f2fs/file.c +1 -2
index 8acdd94272a0ced448e0ba21635d702cfec10682..3e49a73bbf3a184a314e97bff9509a66c27eac00 100644
@@ -1883,7 +1883,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
                       .init_gc_type = FG_GC,
                       .should_migrate_blocks = false,
                       .err_gc_skipped = true,
-                     .nr_free_secs = 0 };
+                     .nr_free_secs = 1 };
       pgoff_t pg_start, pg_end;
       loff_t new_size;
       loff_t off_end;
 
---

base-commit: b51f606aa323d553d786ed681a213f134dc688d6
change-id: 20260620-origin-dev-99cdccc83800

Best regards,

#4 lore raw
Quoted 6 lines · Chao Yu

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

[ EXTERNAL EMAIL ]

Quoted 1 line · Jiucheng Xu
Quoted 7 lines
Quoted 1 line

Regarding "why the pinfile is fragmented" you said, we use v5.15 + android U. Is it that v5.15 lacks some patches for special handling of pinfiles?

Ah, I think you can unpin log file created by logcat, the flag is added in aosp/1014260 to avoid fragmentation in filesystem (), but it's gone in aosp/43c6d76a

Thanks,

Thanks Chao, I think unpinning the log file can avoid this issue.

Let me ask a bit more.

I wonder why gc_control.nr_free_secs must be 0 in f2fs_expand_inode_data().

static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
loff_t len, int mode)
{
struct f2fs_gc_control gc_control = { .victim_segno = NULL_SEGNO,
.init_gc_type = FG_GC, .should_migrate_blocks = false, .err_gc_skipped = true,
.nr_free_secs = 0 };

Since it has reached such an urgent situation that FG GC needs to be triggered, I think the GC should try the second most suitable victim instead of returning -EAGAIN.

int f2fs_gc(struct f2fs_sb_info *sbi, struct f2fs_gc_control *gc_control)

{
        if (gc_type == FG_GC) {
                sbi->cur_victim_sec = NULL_SEGNO;

                if (has_enough_free_secs(sbi, sec_freed, 0)) {
                        if (!gc_control->no_bg_gc &&
                            total_sec_freed < gc_control->nr_free_secs) 

//If nr_free_secs = 1, GC can try the second most suitable victim.

                                goto go_gc_more;
                        goto stop;
                }

Is there any risk if it is set to 1?

If I'm not mistaken, for pinned files, the allocation in f2fs_expand_inode_data() is also carried out in units of sections. Therefore, it should be reasonable to set nr_free_secs to 1 here.

Welcome your feedback!

Thanks,

Quoted 27 lines
Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
---
   fs/f2fs/file.c | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
fs/f2fs/file.c +1 -2
index 8acdd94272a0ced448e0ba21635d702cfec10682..3e49a73bbf3a184a314e97bff9509a66c27eac00 100644
@@ -1883,7 +1883,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
                       .init_gc_type = FG_GC,
                       .should_migrate_blocks = false,
                       .err_gc_skipped = true,
-                     .nr_free_secs = 0 };
+                     .nr_free_secs = 1 };
       pgoff_t pg_start, pg_end;
       loff_t new_size;
       loff_t off_end;
 
---

base-commit: b51f606aa323d553d786ed681a213f134dc688d6
change-id: 20260620-origin-dev-99cdccc83800

Best regards,