Skip to contentSkip to conversation

f2fs/025: test to do sanity check section type correctly in f2fs GC

PATCH In review Chao Yu 2 participants 3 messages May 18 – May 21

lore
Share & tools
lore

Chao Yu

author
#0 lore raw

PATCH 1/1 f2fs/025: test to do sanity check section type correctly in f2fs GC

Without commit 520760b9f915 ("f2fs: optimize representative type determination in GC"), f2fs GC will report inconsistent segment type in large section issue, and then it will force to shutdown filesystem.

[  768.190903] F2FS-fs (loop51): Inconsistent segment (3) type [1, 0] in SIT and SSA

The reason is f2fs kernel will assume all segment type inside large section is the same, during GC it loads type from one segment and compare it to other segments' type, however due to recovery flow, the chosen segment may has zero valid blocks w/ different segment type, since the segment is invalid(free) one, it will never be migrated, so that we should not treat such state as abnormal condition.

This testcase is created to simulate above condition to see whether f2fs kernel module can handle it correctly

---
 tests/f2fs/025     | 88 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/f2fs/025.out |  2 ++
 2 files changed, 90 insertions(+)
 create mode 100644 tests/f2fs/025
 create mode 100644 tests/f2fs/025.out
tests/f2fs/025 new file +88 -0
new file mode 100644
index 00000000..1eba7158
@@ -0,0 +1,88 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Chao Yu <[email protected]>
+#
+# FS QA Test No. f2fs/025
+#
+# Check whether f2fs will encounter cp_error (Inconsistent segment type)
+# when doing sanity check on type of segments inside large section during
+# garbage collection.
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_fixed_by_kernel_commit 520760b9f915 \
+	"f2fs: optimize representative type determination in GC"
+
+. ./common/filter
+
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+_require_scratch
+_require_xfs_io_command "pwrite"
+_require_xfs_io_command "truncate"
+_require_command "$F2FS_IO_PROG" f2fs_io
+_require_check_dmesg
+
+# Format with 96MB size and 2 segments per section
+_scratch_mkfs_sized $((96 * 1024 * 1024)) "" "-s 2" >> $seqres.full 2>&1
+
+# Mount with mode=lfs
+_scratch_mount -o mode=lfs >> $seqres.full 2>&1
+
+# Create files to fill whole filesystem, then segment type will be changed to node type
+for ((i=0;i<5120;i++)) do
+	touch $SCRATCH_MNT/$i >> $seqres.full 2>&1
+done
+sync
+
+# Remove all files to create free(empty) node segments
+rm -f $SCRATCH_MNT/*
+sync
+
+# Allocate free space so that we have chance to reuse free(empty) node segments
+$XFS_IO_PROG -f -c "pwrite -b 4k 0 1928k" $SCRATCH_MNT/file >> $seqres.full 2>&1
+sync
+
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 16M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 16M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+sync
+
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 8M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 32K" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 2M" -c "fsync" $SCRATCH_MNT/file >> $seqres.full 2>&1
+
+# Shutdown the filesystem without checkpoint
+$F2FS_IO_PROG shutdown 2 $SCRATCH_MNT >> $seqres.full 2>&1
+
+_scratch_unmount >> $seqres.full 2>&1
+
+_scratch_mount -o mode=lfs >> $seqres.full 2>&1
+
+# Run urgent_gc mode to trigger garbage collection
+dev_name=$(_short_dev $SCRATCH_DEV)
+if [ -f /sys/fs/f2fs/$dev_name/gc_urgent ]; then
+	echo 1 > /sys/fs/f2fs/$dev_name/gc_urgent
+fi
+
+# Wait background GC thread to wake up to run and potentially encounter the inconsistency
+sleep 5
+
+_scratch_unmount >> $seqres.full 2>&1
+
+# Check whether the dmesg has the warning indicating the bug
+_check_dmesg_for "F2FS-fs \($dev_name\): Inconsistent segment" && \
+	_fail "F2FS-fs ($dev_name): Inconsistent segment type detected in dmesg!"
+
+echo "Silence is golden"
+status=0
+exit
tests/f2fs/025.out new file +2 -0
new file mode 100644
index 00000000..3d70951e
@@ -0,0 +1,2 @@
+QA output created by 025
+Silence is golden
Signature · 1 line
2.49.0
#1 lore raw
Quoted 59 lines · Chao Yu

Without commit 520760b9f915 ("f2fs: optimize representative type determination in GC"), f2fs GC will report inconsistent segment type in large section issue, and then it will force to shutdown filesystem.

[  768.190903] F2FS-fs (loop51): Inconsistent segment (3) type [1, 0] in SIT and SSA

The reason is f2fs kernel will assume all segment type inside large section is the same, during GC it loads type from one segment and compare it to other segments' type, however due to recovery flow, the chosen segment may has zero valid blocks w/ different segment type, since the segment is invalid(free) one, it will never be migrated, so that we should not treat such state as abnormal condition.

This testcase is created to simulate above condition to see whether f2fs kernel module can handle it correctly

---
 tests/f2fs/025     | 88 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/f2fs/025.out |  2 ++
 2 files changed, 90 insertions(+)
 create mode 100644 tests/f2fs/025
 create mode 100644 tests/f2fs/025.out
tests/f2fs/025 new file +29 -0
new file mode 100644
index 00000000..1eba7158
@@ -0,0 +1,88 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Chao Yu <[email protected]>
+#
+# FS QA Test No. f2fs/025
+#
+# Check whether f2fs will encounter cp_error (Inconsistent segment type)
+# when doing sanity check on type of segments inside large section during
+# garbage collection.
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_fixed_by_kernel_commit 520760b9f915 \
+	"f2fs: optimize representative type determination in GC"
+
+. ./common/filter
+
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+_require_scratch
+_require_xfs_io_command "pwrite"
+_require_xfs_io_command "truncate"
+_require_command "$F2FS_IO_PROG" f2fs_io
+_require_check_dmesg

tests/f2fs/025 line 39

Quoted 10 lines
tests/f2fs/025 new file +10 -0
+
+# Format with 96MB size and 2 segments per section
+_scratch_mkfs_sized $((96 * 1024 * 1024)) "" "-s 2" >> $seqres.full 2>&1
+
+# Mount with mode=lfs
+_scratch_mount -o mode=lfs >> $seqres.full 2>&1
+
+# Create files to fill whole filesystem, then segment type will be changed to node type
+for ((i=0;i<5120;i++)) do
+	touch $SCRATCH_MNT/$i >> $seqres.full 2>&1

5120 * 4k = 20MB

Quoted 12 lines
tests/f2fs/025 new file +12 -0
+done
+sync
+
+# Remove all files to create free(empty) node segments
+rm -f $SCRATCH_MNT/*
+sync
+
+# Allocate free space so that we have chance to reuse free(empty) node segments
+$XFS_IO_PROG -f -c "pwrite -b 4k 0 1928k" $SCRATCH_MNT/file >> $seqres.full 2>&1
+sync
+
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1

tests/f2fs/025 line 61

Quoted 10 lines
tests/f2fs/025 new file +10 -0
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 16M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 16M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+sync
+
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 8M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 32K" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1

1928k + 16M + 16M + 8M + 32K ~= 40M

tests/f2fs/025 line 62

Quoted 1 line
tests/f2fs/025 new file +1 -0
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 2M" -c "fsync" $SCRATCH_MNT/file >> $seqres.full 2>&1

96M - 40M - 20M = 36M

I'm not a f2fs expert, I think you hope to (almost) fill the fs before this 2M pwrite step. If so, I'm wondering can we ensure that other metadata takes ~36 MB of space, thereby guaranteeing that this 2 MB write triggers the reuse of that 1928 KB segment (no matter with any MKFS_OPTIONS)?

Thanks,
Zorro

Quoted 37 lines
tests/f2fs/025 new file +26 -0
+
+# Shutdown the filesystem without checkpoint
+$F2FS_IO_PROG shutdown 2 $SCRATCH_MNT >> $seqres.full 2>&1
+
+_scratch_unmount >> $seqres.full 2>&1
+
+_scratch_mount -o mode=lfs >> $seqres.full 2>&1
+
+# Run urgent_gc mode to trigger garbage collection
+dev_name=$(_short_dev $SCRATCH_DEV)
+if [ -f /sys/fs/f2fs/$dev_name/gc_urgent ]; then
+	echo 1 > /sys/fs/f2fs/$dev_name/gc_urgent
+fi
+
+# Wait background GC thread to wake up to run and potentially encounter the inconsistency
+sleep 5
+
+_scratch_unmount >> $seqres.full 2>&1
+
+# Check whether the dmesg has the warning indicating the bug
+_check_dmesg_for "F2FS-fs \($dev_name\): Inconsistent segment" && \
+	_fail "F2FS-fs ($dev_name): Inconsistent segment type detected in dmesg!"
+
+echo "Silence is golden"
+status=0
+exit
tests/f2fs/025.out new file +2 -0
new file mode 100644
index 00000000..3d70951e
@@ -0,0 +1,2 @@
+QA output created by 025
+Silence is golden
Signature · 1 line
2.49.0
author
#2 lore raw
Quoted 98 lines · Zorro Lang
Quoted 59 lines · Chao Yu

Without commit 520760b9f915 ("f2fs: optimize representative type determination in GC"), f2fs GC will report inconsistent segment type in large section issue, and then it will force to shutdown filesystem.

[  768.190903] F2FS-fs (loop51): Inconsistent segment (3) type [1, 0] in SIT and SSA

The reason is f2fs kernel will assume all segment type inside large section is the same, during GC it loads type from one segment and compare it to other segments' type, however due to recovery flow, the chosen segment may has zero valid blocks w/ different segment type, since the segment is invalid(free) one, it will never be migrated, so that we should not treat such state as abnormal condition.

This testcase is created to simulate above condition to see whether f2fs kernel module can handle it correctly

---
  tests/f2fs/025     | 88 ++++++++++++++++++++++++++++++++++++++++++++++
  tests/f2fs/025.out |  2 ++
  2 files changed, 90 insertions(+)

create mode 100644 tests/f2fs/025
create mode 100644 tests/f2fs/025.out

tests/f2fs/025 new file +29 -0
new file mode 100644
index 00000000..1eba7158
@@ -0,0 +1,88 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2026 Chao Yu <[email protected]>
+#
+# FS QA Test No. f2fs/025
+#
+# Check whether f2fs will encounter cp_error (Inconsistent segment type)
+# when doing sanity check on type of segments inside large section during
+# garbage collection.
+#
+. ./common/preamble
+_begin_fstest auto quick
+
+_fixed_by_kernel_commit 520760b9f915 \
+	"f2fs: optimize representative type determination in GC"
+
+. ./common/filter
+
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+}
+
+_require_scratch
+_require_xfs_io_command "pwrite"
+_require_xfs_io_command "truncate"
+_require_command "$F2FS_IO_PROG" f2fs_io
+_require_check_dmesg

tests/f2fs/025 line 39

Quoted 10 lines
tests/f2fs/025 new file +10 -0
+
+# Format with 96MB size and 2 segments per section
+_scratch_mkfs_sized $((96 * 1024 * 1024)) "" "-s 2" >> $seqres.full 2>&1
+
+# Mount with mode=lfs
+_scratch_mount -o mode=lfs >> $seqres.full 2>&1
+
+# Create files to fill whole filesystem, then segment type will be changed to node type
+for ((i=0;i<5120;i++)) do
+	touch $SCRATCH_MNT/$i >> $seqres.full 2>&1

5120 * 4k = 20MB

Quoted 12 lines
tests/f2fs/025 new file +12 -0
+done
+sync
+
+# Remove all files to create free(empty) node segments
+rm -f $SCRATCH_MNT/*
+sync
+
+# Allocate free space so that we have chance to reuse free(empty) node segments
+$XFS_IO_PROG -f -c "pwrite -b 4k 0 1928k" $SCRATCH_MNT/file >> $seqres.full 2>&1
+sync
+
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1

tests/f2fs/025 line 61

Quoted 10 lines
tests/f2fs/025 new file +10 -0
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 16M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 16M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+sync
+
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 8M" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 32K" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "truncate 0" $SCRATCH_MNT/file >> $seqres.full 2>&1

1928k + 16M + 16M + 8M + 32K ~= 40M

Quoted 6 lines

tests/f2fs/025 line 62

Quoted 1 line
tests/f2fs/025 new file +1 -0
+$XFS_IO_PROG -d -c "pwrite -b 4k 0 2M" -c "fsync" $SCRATCH_MNT/file >> $seqres.full 2>&1

96M - 40M - 20M = 36M

I'm not a f2fs expert, I think you hope to (almost) fill the fs before this 2M pwrite step. If so, I'm wondering can we ensure that other metadata takes ~36 MB

Zorro, thanks for the review.

Actually, we don't need to write metadata to other ~36 MB space, because node log and data log are start from different area of block addresses, more details as below:

warm_node type log starts from beginning of block addresses
https://git.kernel.org/pub/scm/linux/kernel/git/chao/f2fs-tools.git/tree/mkfs/f2fs_format.c#n662

warm_data type log starts from middle of block addresses
https://git.kernel.org/pub/scm/linux/kernel/git/chao/f2fs-tools.git/tree/mkfs/f2fs_format.c#n668

With lfs mode, warm_data log can allocate forward until to the end of block addresses, and then start from beginning of block addresss, then we can expect it reuse the area which were used by node log.

Quoted 2 lines

of space, thereby guaranteeing that this 2 MB write triggers the reuse of that 1928 KB segment (no matter with any MKFS_OPTIONS)?

Oh, I may miss to consider MKFS_OPTIONS and MOUNT_OPTIONS, what do you think of clearing both MKFS_OPTIONS and MOUNT_OPTIONS before the test?

Thanks,

Quoted 41 lines

Thanks,
Zorro

Quoted 37 lines
tests/f2fs/025 new file +26 -0
+
+# Shutdown the filesystem without checkpoint
+$F2FS_IO_PROG shutdown 2 $SCRATCH_MNT >> $seqres.full 2>&1
+
+_scratch_unmount >> $seqres.full 2>&1
+
+_scratch_mount -o mode=lfs >> $seqres.full 2>&1
+
+# Run urgent_gc mode to trigger garbage collection
+dev_name=$(_short_dev $SCRATCH_DEV)
+if [ -f /sys/fs/f2fs/$dev_name/gc_urgent ]; then
+	echo 1 > /sys/fs/f2fs/$dev_name/gc_urgent
+fi
+
+# Wait background GC thread to wake up to run and potentially encounter the inconsistency
+sleep 5
+
+_scratch_unmount >> $seqres.full 2>&1
+
+# Check whether the dmesg has the warning indicating the bug
+_check_dmesg_for "F2FS-fs \($dev_name\): Inconsistent segment" && \
+	_fail "F2FS-fs ($dev_name): Inconsistent segment type detected in dmesg!"
+
+echo "Silence is golden"
+status=0
+exit
tests/f2fs/025.out new file +2 -0
new file mode 100644
index 00000000..3d70951e
@@ -0,0 +1,2 @@
+QA output created by 025
+Silence is golden
Signature · 1 line
2.49.0