ext4: verify ext4 when the starting block of index and leaf are inconsistent
PATCH 1/1 ext4: verify ext4 when the starting block of index and leaf are inconsistent
From: chenlong <[email protected]>
Insert the extent entry in ext4_ext_insert_extent() (to be inserted at the beginning of the block). In the stage of updating the starting block number of the parent index block, an error happened
in ext4_ext_correct_indexes()->ext4_ext_get_access(), which caused the index update of the parent index node to fail. The ext4_ext_insert_extent() function exits directly and does not roll back the extent entry of the leaf block. Eventually, the extent starting block numbers in the index block and leaf block are inconsistent, triggering bugon
This is a regression test for three kernel commit:
1. 0f2f87d51aebc (ext4: prevent partial update of the extent blocks)
2. 9c6e071913792 (ext4: check for inconsistent extents between index and leaf block)
3. 8dd27fecede55 (ext4: check for out-of-order index extents in ext4_valid_extent_entries())
--- tests/ext4/054 | 97 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/054.out | 2 + 2 files changed, 99 insertions(+) create mode 100755 tests/ext4/054 create mode 100644 tests/ext4/054.out
tests/ext4/054 new file +97 -0
@@ -0,0 +1,97 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Huawei. All Rights Reserved. +# +# FS QA Test 054 +# +# Regression test for kernel commit: +# 1. 0f2f87d51aebc (ext4: prevent partial update of the extent blocks) +# 2. 9c6e071913792 (ext4: check for inconsistent extents between index \ +# and leaf block) +# 3. 8dd27fecede55 (ext4: check for out-of-order index extents in \ +# ext4_valid_extent_entries()) + +. ./common/preamble +_begin_fstest auto quick + +_cleanup() +{ + cd / + rm -f $tmp.* + $UMOUNT_PROG ${IMG_MNT} > /dev/null 2>&1 + rm -f ${IMG_FILE} > /dev/null 2>&1 +} + +# Import common functions +. ./common/filter + +# real QA test starts here +_supported_fs ext4 +_require_test +_require_xfs_io_command "falloc" +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_xfs_io_command "fpunch" +_require_command "$DEBUGFS_PROG" debugfs + +IMG_FILE=${SCRATCH_MNT}/$seq.fs +IMG_MNT=${SCRATCH_MNT}/$seq.mnt +TEST_FILE="${IMG_MNT}/testfile" + +fallocate -l 10M $IMG_FILE +mkdir -p $IMG_MNT || _fail "cannot create loopback mount point" + +$MKFS_EXT4_PROG -F $IMG_FILE >> $seqres.full 2>&1 || _fail "mkfs failed" + +$MOUNT_PROG -t ${FSTYP} ${IMG_FILE} ${IMG_MNT} >/dev/null 2>&1 || \ + _fail "mount failed" +touch $TEST_FILE + +# +# The following steps create an abnormal file system image and +# create a file(testfile). The starting logic block of his second leaf +# extent block is inconsistent with the starting logic block of the +# second idx extent entry in the inode. +# Level Entries Logical Physical Length Flags +# 0/ 1 2/ 2 5632 - 6527 2020 896 +# ^^^^ 指向了leaf block +# 1/ 1 1/ 15 5376 - 5439 7041 - 7104 64 Uninit +# 1/ 1 2/ 15 5632 - 5695 7297 - 7360 64 Uninit +# +for i in {0..50} +do + offset=$((1024 * 128 * i)) + $XFS_IO_PROG -c "falloc $offset $((1024 * 64))" $TEST_FILE >> $seqres.full + offset=$((offset + 1024 * 64)) + $XFS_IO_PROG -c "pwrite $offset $((1024 * 64))" $TEST_FILE >> $seqres.full + $XFS_IO_PROG -c "fsync" $TEST_FILE >> $seqres.full +done + +$XFS_IO_PROG -c "fpunch $((1024 * 5376)) $((1024 * 256))" $TEST_FILE \ + >> $seqres.full +$XFS_IO_PROG -c "fsync" $TEST_FILE >> $seqres.full +$XFS_IO_PROG -c "falloc $((1024 * 5376)) $((1024 * 64))" $TEST_FILE \ + >> $seqres.full +$XFS_IO_PROG -c "fsync" $TEST_FILE + +$UMOUNT_PROG ${IMG_MNT} || _fail "umount failed" + +$DEBUGFS_PROG -w -R "set_inode_field testfile block[6] 0x1600" $IMG_FILE \ + 2>>$seqres.full >> $seqres.full + +# Mount the file system and create a block at a location in the middle +# of 5440-5632 + +$MOUNT_PROG -t ${FSTYP} -o nodelalloc ${IMG_FILE} ${IMG_MNT} \ + >/dev/null 2>&1 || _fail "mount failed" +$XFS_IO_PROG -c "pwrite $((1024 * 5568)) $((1024 * 64))" $TEST_FILE \ + 2>>$seqres.full >> $seqres.full + +# If this BUG is not fixed, when the testcase is executed to this position, +# the kernel will be BUG_ON immediately. + +echo "Silence is golden" + +# success, all done +status=0 +exit
tests/ext4/054.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 054 +Silence is golden
Signature · 1 line
2.17.1
Quoted 11 lines · On Tue, Nov 23, 2021 at 09:37:59AM +0800, [email protected] wrote:
From: chenlong <[email protected]>
Insert the extent entry in
ext4_ext_insert_extent()(to be inserted at the beginning of the block). In the stage of updating the starting block number of the parent index block, an error happened
in ext4_ext_correct_indexes()->ext4_ext_get_access(), which caused the index update of the parent index node to fail. Theext4_ext_insert_extent()function exits directly and does not roll back the extent entry of the leaf block. Eventually, the extent starting block numbers in the index block and leaf block are inconsistent, triggering bugon
Quoted 6 lines
This is a regression test for three kernel commit:
1.0f2f87d51aebc(ext4: prevent partial update of the extent blocks)
2.9c6e071913792(ext4: check for inconsistent extents between index and leaf block)
3.8dd27fecede55(ext4: check for out-of-order index extents inext4_valid_extent_entries())
Missing Signed-off-by tag.
Quoted 41 lines
--- tests/ext4/054 | 97 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/054.out | 2 + 2 files changed, 99 insertions(+) create mode 100755 tests/ext4/054 create mode 100644 tests/ext4/054.outtests/ext4/054 new file +28
-0@@ -0,0 +1,97 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Huawei. All Rights Reserved. +# +# FS QA Test 054 +# +# Regression test for kernel commit: +# 1. 0f2f87d51aebc (ext4: prevent partial update of the extent blocks) +# 2. 9c6e071913792 (ext4: check for inconsistent extents between index \ +# and leaf block) +# 3. 8dd27fecede55 (ext4: check for out-of-order index extents in \ +# ext4_valid_extent_entries()) + +. ./common/preamble +_begin_fstest auto quick + +_cleanup() +{ + cd / + rm -f $tmp.* + $UMOUNT_PROG ${IMG_MNT} > /dev/null 2>&1 + rm -f ${IMG_FILE} > /dev/null 2>&1 +} + +# Import common functions +. ./common/filter + +# real QA test starts here
tests/ext4/054 line 38
Quoted 10 lines
tests/ext4/054 new file +10
-0+_supported_fs ext4 +_require_test +_require_xfs_io_command "falloc" +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_xfs_io_command "fpunch" +_require_command "$DEBUGFS_PROG" debugfs + +IMG_FILE=${SCRATCH_MNT}/$seq.fs +IMG_MNT=${SCRATCH_MNT}/$seq.mnt
SCRATCH_DEV is not required nor mounted, so IMG_FILE and IMG_MNT are created on the root filesystem.
And I don't see why we need to loop back mount an fs image here, does _require_scratch_nocheck then use $SCRATCH_DEV directly work for you?
And if loop device is really needed, _require_loop should be added as well, and use $TEST_DIR to save the fs image and mnt dir, e.g.
IMG_FILE=$TEST_DIR/$seq.fs
IMG_MNT=$TEST_DIR/$seq.mnt
and use _create_loop_device and _destroy_loop_device helpers.
tests/ext4/054 line 41
Quoted 3 lines
tests/ext4/054 new file +3
-0+TEST_FILE="${IMG_MNT}/testfile" + +fallocate -l 10M $IMG_FILE
$XFS_IO_PROG -c "falloc 0 10m" $IMG_FILE
tests/ext4/054 line 47
Quoted 6 lines
tests/ext4/054 new file +6
-0+mkdir -p $IMG_MNT || _fail "cannot create loopback mount point" + +$MKFS_EXT4_PROG -F $IMG_FILE >> $seqres.full 2>&1 || _fail "mkfs failed" + +$MOUNT_PROG -t ${FSTYP} ${IMG_FILE} ${IMG_MNT} >/dev/null 2>&1 || \ + _fail "mount failed"
If use $SCRATCH_DEV, above will simply be
_scratch_mkfs >> $seqres.full 2>&1_scratch_mount
tests/ext4/054 line 52
Quoted 5 lines
tests/ext4/054 new file +5
-0+touch $TEST_FILE + +# +# The following steps create an abnormal file system image and +# create a file(testfile). The starting logic block of his second leaf
^^^^ its ?
tests/ext4/054 line 57
Quoted 5 lines
tests/ext4/054 new file +5
-0+# extent block is inconsistent with the starting logic block of the +# second idx extent entry in the inode. +# Level Entries Logical Physical Length Flags +# 0/ 1 2/ 2 5632 - 6527 2020 896 +# ^^^^ 指向了leaf block
^^^^^ Use English please.
Quoted 18 lines
tests/ext4/054 new file +18
-0+# 1/ 1 1/ 15 5376 - 5439 7041 - 7104 64 Uninit +# 1/ 1 2/ 15 5632 - 5695 7297 - 7360 64 Uninit +# +for i in {0..50} +do + offset=$((1024 * 128 * i)) + $XFS_IO_PROG -c "falloc $offset $((1024 * 64))" $TEST_FILE >> $seqres.full + offset=$((offset + 1024 * 64)) + $XFS_IO_PROG -c "pwrite $offset $((1024 * 64))" $TEST_FILE >> $seqres.full + $XFS_IO_PROG -c "fsync" $TEST_FILE >> $seqres.full +done + +$XFS_IO_PROG -c "fpunch $((1024 * 5376)) $((1024 * 256))" $TEST_FILE \ + >> $seqres.full +$XFS_IO_PROG -c "fsync" $TEST_FILE >> $seqres.full +$XFS_IO_PROG -c "falloc $((1024 * 5376)) $((1024 * 64))" $TEST_FILE \ + >> $seqres.full +$XFS_IO_PROG -c "fsync" $TEST_FILE
tests/ext4/054 line 85
Quoted 10 lines
tests/ext4/054 new file +10
-0+ +$UMOUNT_PROG ${IMG_MNT} || _fail "umount failed" + +$DEBUGFS_PROG -w -R "set_inode_field testfile block[6] 0x1600" $IMG_FILE \ + 2>>$seqres.full >> $seqres.full + +# Mount the file system and create a block at a location in the middle +# of 5440-5632 + +$MOUNT_PROG -t ${FSTYP} -o nodelalloc ${IMG_FILE} ${IMG_MNT} \
Please add comments to explain why nodelalloc is needed.
Thanks,
Eryu
Quoted 22 lines
tests/ext4/054 new file +12
-0+ >/dev/null 2>&1 || _fail "mount failed" +$XFS_IO_PROG -c "pwrite $((1024 * 5568)) $((1024 * 64))" $TEST_FILE \ + 2>>$seqres.full >> $seqres.full + +# If this BUG is not fixed, when the testcase is executed to this position, +# the kernel will be BUG_ON immediately. + +echo "Silence is golden" + +# success, all done +status=0 +exittests/ext4/054.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 054 +Silence is goldenSignature · 1 line
2.17.1
Quoted 12 lines · Eryu Guan
Quoted 11 lines · On Tue, Nov 23, 2021 at 09:37:59AM +0800, [email protected] wrote:
From: chenlong <[email protected]>
Insert the extent entry in
ext4_ext_insert_extent()(to be inserted at the beginning of the block). In the stage of updating the starting block number of the parent index block, an error happened
in ext4_ext_correct_indexes()->ext4_ext_get_access(), which caused the index update of the parent index node to fail. Theext4_ext_insert_extent()function exits directly and does not roll back the extent entry of the leaf block. Eventually, the extent starting block numbers in the index block and leaf block are inconsistent, triggering bugon
Quoted 9 lines
Quoted 6 lines
This is a regression test for three kernel commit:
1.0f2f87d51aebc(ext4: prevent partial update of the extent blocks)
2.9c6e071913792(ext4: check for inconsistent extents between index and leaf block)
3.8dd27fecede55(ext4: check for out-of-order index extents inext4_valid_extent_entries())Missing Signed-off-by tag.
Thanks for your review.
I will add it in the V2 version.
Quoted 58 lines
Quoted 41 lines
--- tests/ext4/054 | 97 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/054.out | 2 + 2 files changed, 99 insertions(+)create mode 100755 tests/ext4/054
create mode 100644 tests/ext4/054.outtests/ext4/054 new file +28
-0@@ -0,0 +1,97 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Huawei. All Rights Reserved. +# +# FS QA Test 054 +# +# Regression test for kernel commit: +# 1. 0f2f87d51aebc (ext4: prevent partial update of the extent blocks) +# 2. 9c6e071913792 (ext4: check for inconsistent extents between index \ +# and leaf block) +# 3. 8dd27fecede55 (ext4: check for out-of-order index extents in \ +# ext4_valid_extent_entries()) + +. ./common/preamble +_begin_fstest auto quick + +_cleanup() +{ + cd / + rm -f $tmp.* + $UMOUNT_PROG ${IMG_MNT} > /dev/null 2>&1 + rm -f ${IMG_FILE} > /dev/null 2>&1 +} + +# Import common functions +. ./common/filter + +# real QA test starts heretests/ext4/054 line 38
Quoted 10 lines
tests/ext4/054 new file +10
-0+_supported_fs ext4 +_require_test +_require_xfs_io_command "falloc" +_require_xfs_io_command "pwrite" +_require_xfs_io_command "fsync" +_require_xfs_io_command "fpunch" +_require_command "$DEBUGFS_PROG" debugfs + +IMG_FILE=${SCRATCH_MNT}/$seq.fs +IMG_MNT=${SCRATCH_MNT}/$seq.mnt
SCRATCH_DEVis not required nor mounted, soIMG_FILEandIMG_MNTare created on the root filesystem.And I don't see why we need to loop back mount an fs image here, does
_require_scratch_nocheckthen use $SCRATCH_DEV directly work for you?
Quoted 8 lines
And if loop device is really needed,
_require_loopshould be added as well, and use $TEST_DIR to save the fs image and mnt dir, e.g.IMG_FILE=$TEST_DIR/$seq.fs
IMG_MNT=$TEST_DIR/$seq.mntand use
_create_loop_deviceand_destroy_loop_devicehelpers.
The original idea was to construct a damaged file system by creating a loop device. In fact, scratch_dev can also achieve the goal. In V2 version, it will be changed to scratch_dev.
Quoted 6 lines
tests/ext4/054 line 41
Quoted 3 lines
tests/ext4/054 new file +3
-0+TEST_FILE="${IMG_MNT}/testfile" + +fallocate -l 10M $IMG_FILE$XFS_IO_PROG -c "falloc 0 10m" $IMG_FILE
use scratch_dev to test, no need to create img_file.
Quoted 9 lines
tests/ext4/054 line 47
Quoted 6 lines
tests/ext4/054 new file +6
-0+mkdir -p $IMG_MNT || _fail "cannot create loopback mount point" + +$MKFS_EXT4_PROG -F $IMG_FILE >> $seqres.full 2>&1 || _fail "mkfs failed" + +$MOUNT_PROG -t ${FSTYP} ${IMG_FILE} ${IMG_MNT} >/dev/null 2>&1 || \ + _fail "mount failed"If use $SCRATCH_DEV, above will simply be
Quoted 3 lines
_scratch_mkfs>> $seqres.full 2>&1_scratch_mount
no need to use loop device.
Quoted 7 lines
tests/ext4/054 line 52
Quoted 5 lines
tests/ext4/054 new file +5
-0+touch $TEST_FILE + +# +# The following steps create an abnormal file system image and +# create a file(testfile). The starting logic block of his second leaf^^^^ its ?
it is more appropriate to use its here.
Quoted 7 lines
tests/ext4/054 line 57
Quoted 5 lines
tests/ext4/054 new file +5
-0+# extent block is inconsistent with the starting logic block of the +# second idx extent entry in the inode. +# Level Entries Logical Physical Length Flags +# 0/ 1 2/ 2 5632 - 6527 2020 896 +# ^^^^ 指向了leaf block^^^^^ Use English please.
sorry, i correct it.
Quoted 31 lines
Quoted 18 lines
tests/ext4/054 new file +18
-0+# 1/ 1 1/ 15 5376 - 5439 7041 - 7104 64 Uninit +# 1/ 1 2/ 15 5632 - 5695 7297 - 7360 64 Uninit +# +for i in {0..50} +do + offset=$((1024 * 128 * i)) + $XFS_IO_PROG -c "falloc $offset $((1024 * 64))" $TEST_FILE >> $seqres.full + offset=$((offset + 1024 * 64)) + $XFS_IO_PROG -c "pwrite $offset $((1024 * 64))" $TEST_FILE >> $seqres.full + $XFS_IO_PROG -c "fsync" $TEST_FILE >> $seqres.full +done + +$XFS_IO_PROG -c "fpunch $((1024 * 5376)) $((1024 * 256))" $TEST_FILE \ + >> $seqres.full +$XFS_IO_PROG -c "fsync" $TEST_FILE >> $seqres.full +$XFS_IO_PROG -c "falloc $((1024 * 5376)) $((1024 * 64))" $TEST_FILE \ + >> $seqres.full +$XFS_IO_PROG -c "fsync" $TEST_FILEtests/ext4/054 line 85
Quoted 10 lines
tests/ext4/054 new file +10
-0+ +$UMOUNT_PROG ${IMG_MNT} || _fail "umount failed" + +$DEBUGFS_PROG -w -R "set_inode_field testfile block[6] 0x1600" $IMG_FILE \ + 2>>$seqres.full >> $seqres.full + +# Mount the file system and create a block at a location in the middle +# of 5440-5632 + +$MOUNT_PROG -t ${FSTYP} -o nodelalloc ${IMG_FILE} ${IMG_MNT} \Please add comments to explain why nodelalloc is needed.
Quoted 3 lines
Thanks,
Eryu
When the delalloc option is enabled when mounting, the block allocation will not be triggered when writing (the maximum waiting time is about 30s). If nodellalloc is used, the block will be allocated when writing. The expected BUG_ON in this use case can be triggered immediately, and the test results can be quickly obtained.
I will add comments in the V2 version.
Quoted 24 lines
Quoted 22 lines
tests/ext4/054 new file +12
-0+ >/dev/null 2>&1 || _fail "mount failed" +$XFS_IO_PROG -c "pwrite $((1024 * 5568)) $((1024 * 64))" $TEST_FILE \ + 2>>$seqres.full >> $seqres.full + +# If this BUG is not fixed, when the testcase is executed to this position, +# the kernel will be BUG_ON immediately. + +echo "Silence is golden" + +# success, all done +status=0 +exittests/ext4/054.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 054 +Silence is goldenSignature · 1 line
2.17.1.