Skip to contentSkip to conversation

f2fs/013: test to check potential corruption on atomic_write file

PATCH Superseded Chao Yu 2 participants 4 messages May 6, 2025 – May 7, 2025

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Chao Yu

author
#0 lore raw

PATCH v2 1/1 f2fs/013: test to check potential corruption on atomic_write file

This is a regression testcase to check whether we will handle database inode dirty status correctly:
1. mount f2fs image w/ timeout fault injection option
2. create a regular file, and write data into the file
3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
4. write transaction data to the file
5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
6. meanwhile loop call fsync in parallel
Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic file"), database file may become corrupted after atomic write while there is concurrent dirty inode flush in background.

Cc Jaegeuk Kim <[email protected]>
Cc Daeho Jeong <[email protected]>

---
v2:
- fix description of test steps
- add missing "_require_kernel_config CONFIG_F2FS_FAULT_INJECTION"
- add missing "_require_command "$F2FS_IO_PROG" f2fs_io"
- put correct filesize in the golden output

 tests/f2fs/013     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/f2fs/013.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/f2fs/013
 create mode 100644 tests/f2fs/013.out
tests/f2fs/013 new file +71 -0
new file mode 100755
index 00000000..4cf56840
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Chao Yu.  All Rights Reserved.
+#
+# FS QA Test No. f2fs/013
+#
+# This is a regression testcase to check whether we will handle database
+# inode dirty status correctly:
+# 1. mount f2fs image w/ timeout fault injection option
+# 2. create a regular file, and write data into the file
+# 3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
+# 4. write transaction data to the file
+# 5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
+# 6. meanwhile loop call fsync in parallel
+# Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic
+# file"), database file may become corrupted after atomic write while
+# there is concurrent dirty inode flush in background.
+#
+. ./common/preamble
+_begin_fstest auto quick
+_require_kernel_config CONFIG_F2FS_FAULT_INJECTION
+_require_command "$F2FS_IO_PROG" f2fs_io
+
+_cleanup()
+{
+	[ -n "$atomic_write_pid" ] && kill -9 $atomic_write_pid
+	wait
+	cd /
+	rm -r -f $tmp.*
+}
+
+_fixed_by_kernel_commit f098aeba04c9 \
+	"f2fs: fix to avoid atomicity corruption of atomic file"
+
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+# note that it relis on FAULT_TIMEOUT fault injection support in f2fs
+_scratch_mount "-o fault_injection=1,fault_type=4194304" >> $seqres.full
+
+dbfile=$SCRATCH_MNT/file.db
+
+# initialize database file
+$XFS_IO_PROG -c "pwrite 0 4k" -c "fsync" -f $dbfile >> $seqres.full
+
+# sync filesystem to clear dirty inode
+sync
+
+# start atomic_write on src.db database file and commit transaction
+$F2FS_IO_PROG write 1 0 2 zero atomic_commit $dbfile >> $seqres.full &
+atomic_write_pid=$!
+
+# call fsync to flush dirty inode of database file in parallel
+for ((j=0;j<1000;j++)) do
+	$F2FS_IO_PROG fsync $dbfile >> $seqres.full
+done
+
+wait $atomic_write_pid
+unset atomic_write_pid
+
+# flush dirty data and drop cache
+sync
+echo 3 > /proc/sys/vm/drop_caches
+
+stat $dbfile -c %s
+
+rm $dbfile
+sync
+
+status=0
+exit
tests/f2fs/013.out new file +2 -0
new file mode 100644
index 00000000..d9271f75
@@ -0,0 +1,2 @@
+QA output created by 013
+8192
Signature · 1 line
2.49.0
#1 lore raw
Quoted 62 lines · On Tue, May 6, 2025 at 12:05 AM Chao Yu via Linux-f2fs-devel…

This is a regression testcase to check whether we will handle database inode dirty status correctly:
1. mount f2fs image w/ timeout fault injection option
2. create a regular file, and write data into the file
3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
4. write transaction data to the file
5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
6. meanwhile loop call fsync in parallel
Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic file"), database file may become corrupted after atomic write while there is concurrent dirty inode flush in background.

Cc Jaegeuk Kim <[email protected]>
Cc Daeho Jeong <[email protected]>

---
v2:
- fix description of test steps
- add missing "_require_kernel_config CONFIG_F2FS_FAULT_INJECTION"
- add missing "_require_command "$F2FS_IO_PROG" f2fs_io"
- put correct filesize in the golden output

 tests/f2fs/013     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/f2fs/013.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/f2fs/013
 create mode 100644 tests/f2fs/013.out
tests/f2fs/013 new file +28 -0
new file mode 100755
index 00000000..4cf56840
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Chao Yu.  All Rights Reserved.
+#
+# FS QA Test No. f2fs/013
+#
+# This is a regression testcase to check whether we will handle database
+# inode dirty status correctly:
+# 1. mount f2fs image w/ timeout fault injection option
+# 2. create a regular file, and write data into the file
+# 3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
+# 4. write transaction data to the file
+# 5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
+# 6. meanwhile loop call fsync in parallel
+# Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic
+# file"), database file may become corrupted after atomic write while
+# there is concurrent dirty inode flush in background.
+#
+. ./common/preamble
+_begin_fstest auto quick
+_require_kernel_config CONFIG_F2FS_FAULT_INJECTION
+_require_command "$F2FS_IO_PROG" f2fs_io
+
+_cleanup()
+{
+       [ -n "$atomic_write_pid" ] && kill -9 $atomic_write_pid
+       wait
+       cd /

tests/f2fs/013 line 38

Quoted 10 lines
tests/f2fs/013 new file +10 -0
+       rm -r -f $tmp.*
+}
+
+_fixed_by_kernel_commit f098aeba04c9 \
+       "f2fs: fix to avoid atomicity corruption of atomic file"
+
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+# note that it relis on FAULT_TIMEOUT fault injection support in f2fs

relis -> relies

tests/f2fs/013 line 50

Quoted 12 lines
tests/f2fs/013 new file +12 -0
+_scratch_mount "-o fault_injection=1,fault_type=4194304" >> $seqres.full
+
+dbfile=$SCRATCH_MNT/file.db
+
+# initialize database file
+$XFS_IO_PROG -c "pwrite 0 4k" -c "fsync" -f $dbfile >> $seqres.full
+
+# sync filesystem to clear dirty inode
+sync
+
+# start atomic_write on src.db database file and commit transaction
+$F2FS_IO_PROG write 1 0 2 zero atomic_commit $dbfile >> $seqres.full &

I think we might have some timing issues here like background f2fs_io ends earlier than foreground fsync() loop.

Quoted 38 lines
tests/f2fs/013 new file +21 -0
+atomic_write_pid=$!
+
+# call fsync to flush dirty inode of database file in parallel
+for ((j=0;j<1000;j++)) do
+       $F2FS_IO_PROG fsync $dbfile >> $seqres.full
+done
+
+wait $atomic_write_pid
+unset atomic_write_pid
+
+# flush dirty data and drop cache
+sync
+echo 3 > /proc/sys/vm/drop_caches
+
+stat $dbfile -c %s
+
+rm $dbfile
+sync
+
+status=0
+exit
tests/f2fs/013.out new file +2 -1
new file mode 100644
index 00000000..d9271f75
@@ -0,0 +1,2 @@
+QA output created by 013
+8192
--

2.49.0

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

author
#2 lore raw
Quoted 68 lines · Daeho Jeong
Quoted 66 lines · On Tue, May 6, 2025 at 12:05 AM Chao Yu via Linux-f2fs-devel…

This is a regression testcase to check whether we will handle database inode dirty status correctly:
1. mount f2fs image w/ timeout fault injection option
2. create a regular file, and write data into the file
3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
4. write transaction data to the file
5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
6. meanwhile loop call fsync in parallel
Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic file"), database file may become corrupted after atomic write while there is concurrent dirty inode flush in background.

Cc Jaegeuk Kim <[email protected]>
Cc Daeho Jeong <[email protected]>

---
v2:
- fix description of test steps
- add missing "_require_kernel_config CONFIG_F2FS_FAULT_INJECTION"
- add missing "_require_command "$F2FS_IO_PROG" f2fs_io"
- put correct filesize in the golden output

 tests/f2fs/013     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/f2fs/013.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/f2fs/013
 create mode 100644 tests/f2fs/013.out
tests/f2fs/013 new file +32 -0
new file mode 100755
index 00000000..4cf56840
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Chao Yu.  All Rights Reserved.
+#
+# FS QA Test No. f2fs/013
+#
+# This is a regression testcase to check whether we will handle database
+# inode dirty status correctly:
+# 1. mount f2fs image w/ timeout fault injection option
+# 2. create a regular file, and write data into the file
+# 3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
+# 4. write transaction data to the file
+# 5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
+# 6. meanwhile loop call fsync in parallel
+# Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic
+# file"), database file may become corrupted after atomic write while
+# there is concurrent dirty inode flush in background.
+#
+. ./common/preamble
+_begin_fstest auto quick
+_require_kernel_config CONFIG_F2FS_FAULT_INJECTION
+_require_command "$F2FS_IO_PROG" f2fs_io
+
+_cleanup()
+{
+       [ -n "$atomic_write_pid" ] && kill -9 $atomic_write_pid
+       wait
+       cd /
+       rm -r -f $tmp.*
+}
+
+_fixed_by_kernel_commit f098aeba04c9 \
Quoted 8 lines

tests/f2fs/013 line 38

Quoted 6 lines
tests/f2fs/013 new file +6 -0
+       "f2fs: fix to avoid atomicity corruption of atomic file"
+
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+# note that it relis on FAULT_TIMEOUT fault injection support in f2fs

relis -> relies

Will fix.

Quoted 16 lines

tests/f2fs/013 line 50

Quoted 13 lines
tests/f2fs/013 new file +12 -0
+_scratch_mount "-o fault_injection=1,fault_type=4194304" >> $seqres.full
+
+dbfile=$SCRATCH_MNT/file.db
+
+# initialize database file
+$XFS_IO_PROG -c "pwrite 0 4k" -c "fsync" -f $dbfile >> $seqres.full
+
+# sync filesystem to clear dirty inode
+sync
+
+# start atomic_write on src.db database file and commit transaction
+$F2FS_IO_PROG write 1 0 2 zero atomic_commit $dbfile >> $seqres.full &

I think we might have some timing issues here like background f2fs_io ends earlier than foreground fsync() loop.

What about adding a sysfs entry to control timeout, by default it's 1 second, we can tune it to 5 second for this case?

Thanks,

Quoted 39 lines
Quoted 39 lines
tests/f2fs/013 new file +21 -0
+atomic_write_pid=$!
+
+# call fsync to flush dirty inode of database file in parallel
+for ((j=0;j<1000;j++)) do
+       $F2FS_IO_PROG fsync $dbfile >> $seqres.full
+done
+
+wait $atomic_write_pid
+unset atomic_write_pid
+
+# flush dirty data and drop cache
+sync
+echo 3 > /proc/sys/vm/drop_caches
+
+stat $dbfile -c %s
+
+rm $dbfile
+sync
+
+status=0
+exit
tests/f2fs/013.out new file +2 -1
new file mode 100644
index 00000000..d9271f75
@@ -0,0 +1,2 @@
+QA output created by 013
+8192
--

2.49.0

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#3 lore raw
Quoted 98 lines · Chao Yu
Quoted 68 lines · Daeho Jeong
Quoted 66 lines · On Tue, May 6, 2025 at 12:05 AM Chao Yu via Linux-f2fs-devel…

This is a regression testcase to check whether we will handle database inode dirty status correctly:
1. mount f2fs image w/ timeout fault injection option
2. create a regular file, and write data into the file
3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
4. write transaction data to the file
5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
6. meanwhile loop call fsync in parallel
Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic file"), database file may become corrupted after atomic write while there is concurrent dirty inode flush in background.

Cc Jaegeuk Kim <[email protected]>
Cc Daeho Jeong <[email protected]>

---
v2:
- fix description of test steps
- add missing "_require_kernel_config CONFIG_F2FS_FAULT_INJECTION"
- add missing "_require_command "$F2FS_IO_PROG" f2fs_io"
- put correct filesize in the golden output

 tests/f2fs/013     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/f2fs/013.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/f2fs/013
 create mode 100644 tests/f2fs/013.out
tests/f2fs/013 new file +32 -0
new file mode 100755
index 00000000..4cf56840
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Chao Yu.  All Rights Reserved.
+#
+# FS QA Test No. f2fs/013
+#
+# This is a regression testcase to check whether we will handle database
+# inode dirty status correctly:
+# 1. mount f2fs image w/ timeout fault injection option
+# 2. create a regular file, and write data into the file
+# 3. start transaction on the file (via F2FS_IOC_START_ATOMIC_WRITE)
+# 4. write transaction data to the file
+# 5. commit and end the transaction (via F2FS_IOC_COMMIT_ATOMIC_WRITE)
+# 6. meanwhile loop call fsync in parallel
+# Before f098aeba04c9 ("f2fs: fix to avoid atomicity corruption of atomic
+# file"), database file may become corrupted after atomic write while
+# there is concurrent dirty inode flush in background.
+#
+. ./common/preamble
+_begin_fstest auto quick
+_require_kernel_config CONFIG_F2FS_FAULT_INJECTION
+_require_command "$F2FS_IO_PROG" f2fs_io
+
+_cleanup()
+{
+       [ -n "$atomic_write_pid" ] && kill -9 $atomic_write_pid
+       wait
+       cd /
+       rm -r -f $tmp.*
+}
+
+_fixed_by_kernel_commit f098aeba04c9 \
Quoted 8 lines

tests/f2fs/013 line 38

Quoted 6 lines
tests/f2fs/013 new file +6 -0
+       "f2fs: fix to avoid atomicity corruption of atomic file"
+
+_require_scratch
+
+_scratch_mkfs >> $seqres.full
+# note that it relis on FAULT_TIMEOUT fault injection support in f2fs

relis -> relies

Will fix.

Quoted 16 lines

tests/f2fs/013 line 50

Quoted 13 lines
tests/f2fs/013 new file +12 -0
+_scratch_mount "-o fault_injection=1,fault_type=4194304" >> $seqres.full
+
+dbfile=$SCRATCH_MNT/file.db
+
+# initialize database file
+$XFS_IO_PROG -c "pwrite 0 4k" -c "fsync" -f $dbfile >> $seqres.full
+
+# sync filesystem to clear dirty inode
+sync
+
+# start atomic_write on src.db database file and commit transaction
+$F2FS_IO_PROG write 1 0 2 zero atomic_commit $dbfile >> $seqres.full &

I think we might have some timing issues here like background f2fs_io ends earlier than foreground fsync() loop.

Quoted 2 lines

What about adding a sysfs entry to control timeout, by default it's 1 second, we can tune it to 5 second for this case?

Ah, so TIMEOUT fault will induce 1 second delay in the kernel path? Then, I am okay with that.

Quoted 43 lines

Thanks,

Quoted 39 lines
Quoted 39 lines
tests/f2fs/013 new file +21 -0
+atomic_write_pid=$!
+
+# call fsync to flush dirty inode of database file in parallel
+for ((j=0;j<1000;j++)) do
+       $F2FS_IO_PROG fsync $dbfile >> $seqres.full
+done
+
+wait $atomic_write_pid
+unset atomic_write_pid
+
+# flush dirty data and drop cache
+sync
+echo 3 > /proc/sys/vm/drop_caches
+
+stat $dbfile -c %s
+
+rm $dbfile
+sync
+
+status=0
+exit
tests/f2fs/013.out new file +2 -1
new file mode 100644
index 00000000..d9271f75
@@ -0,0 +1,2 @@
+QA output created by 013
+8192
--

2.49.0

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel