xfs: test the deadlock between the AGI and AGF with RENAME_WHITEOUT
PATCH 1/1 xfs: test the deadlock between the AGI and AGF with RENAME_WHITEOUT
There is ABBA deadlock bug between the AGI and AGF when performing rename() with RENAME_WHITEOUT flag, and add this testcase to make sure the rename() call works well.
--- tests/xfs/512 | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/512.out | 2 ++ tests/xfs/group | 1 + 3 files changed, 103 insertions(+) create mode 100755 tests/xfs/512 create mode 100644 tests/xfs/512.out
tests/xfs/512 new file +100 -0
@@ -0,0 +1,100 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 Tencent. All Rights Reserved. +# +# FS QA Test 512 +# +# Test the ABBA deadlock case between the AGI and AGF When performing +# rename operation with RENAME_WHITEOUT flag. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +rm -f $seqres.full + +# real QA test starts here +_supported_fs xfs +_supported_os Linux +_require_scratch + +# Single AG will cause default xfs_repair to fail. This test need a +# single AG fs, so ignore the check. +_require_scratch_nocheck + +prepare_file() +{ + # create many small files for the rename with RENAME_WHITEOUT + i=0 + while [ $i -le $files ]; do + file=$SCRATCH_MNT/f$i + $XFS_IO_PROG -f -d -c 'pwrite -b 4k 0 4k' $file >/dev/null 2>&1 + let i=$i+1 + done +} + +rename_whiteout() +{ + # create the rename targetdir + renamedir=$SCRATCH_MNT/renamedir + mkdir $renamedir + + # just get a random long name... + longnamepre=FFFsafdsagafsadfagasdjfalskdgakdlsglkasdg + + # now try to do rename with RENAME_WHITEOUT flag + i=0 + while [ $i -le $files ]; do + src/renameat2 -w $SCRATCH_MNT/f$i $renamedir/$longnamepre$i >/dev/null 2>&1 + let i=$i+1 + done +} + +create_file() +{ + # create the targetdir + createdir=$SCRATCH_MNT/createdir + mkdir $createdir + + # try to create file at the same time to hit the deadlock + i=0 + while [ $i -le $files ]; do + file=$createdir/f$i + $XFS_IO_PROG -f -d -c 'pwrite -b 4k 0 4k' $file >/dev/null 2>&1 + let i=$i+1 + done +} + +_scratch_mkfs_xfs -bsize=512 -dagcount=1 >> $seqres.full 2>&1 || + _fail "mkfs failed" +_scratch_mount + +files=250000 + +prepare_file +rename_whiteout & +create_file & + +wait +echo Silence is golden + +# Failure comes in the form of a deadlock. + +# success, all done +status=0 +exit
tests/xfs/512.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 512 +Silence is golden
tests/xfs/group +1 -0
@@ -509,3 +509,4 @@ 509 auto ioctl 510 auto ioctl quick 511 auto quick quota +512 auto rename
Signature · 4 lines
1.8.3.1 -- kaixuxia
Quoted 43 lines · kaixuxia
There is ABBA deadlock bug between the AGI and AGF when performing
rename()withRENAME_WHITEOUTflag, and add this testcase to make sure therename()call works well.Signed-off-by kaixuxia <[email protected]>--- tests/xfs/512 | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/512.out | 2 ++ tests/xfs/group | 1 + 3 files changed, 103 insertions(+) create mode 100755 tests/xfs/512 create mode 100644 tests/xfs/512.outtests/xfs/512 new file +24
-0@@ -0,0 +1,100 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 Tencent. All Rights Reserved. +# +# FS QA Test 512 +# +# Test the ABBA deadlock case between the AGI and AGF When performing +# rename operation with RENAME_WHITEOUT flag. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} +
tests/xfs/512 line 34
Quoted 10 lines
tests/xfs/512 new file +10
-0+# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +rm -f $seqres.full + +# real QA test starts here +_supported_fs xfs +_supported_os Linux +_require_scratch
Only _require_scratch_nocheck is suffiecient.
tests/xfs/512 line 38
Quoted 4 lines
tests/xfs/512 new file +4
-0+ +# Single AG will cause default xfs_repair to fail. This test need a +# single AG fs, so ignore the check. +_require_scratch_nocheck
Also need to
. ./common/rename
...
_requires_renameat2
Also, this test requires RENAME_WHITEOUT, I'd suggest enhance src/renameat2.c to support check for if a given rename flag is supported by kernel, and refactor the checks in generic/02[45] and generic/078 into _requires_renameat2 to use the new functionality. e.g.
# without option, behavior stays unchanged, check for renameat2 syscall # support _requires_renameat2
# check if renameat2 upports RENAME_WHITEOUT flag _requires_renameat2 whiteout
# check if renameat2 upports RENAME_EXCHANGE flag _requires_renameat2 exchange
tests/xfs/512 line 47
Quoted 9 lines
tests/xfs/512 new file +9
-0+ +prepare_file() +{ + # create many small files for the rename with RENAME_WHITEOUT + i=0 + while [ $i -le $files ]; do + file=$SCRATCH_MNT/f$i + $XFS_IO_PROG -f -d -c 'pwrite -b 4k 0 4k' $file >/dev/null 2>&1 + let i=$i+1
Creating 250000 4k files take a long time. Does file content really matters? I guess racing RENAME_WHITEOUT with file creation is enough, is it possible to just create many empty files? e.g.
echo > $file
this saves a lot time.
tests/xfs/512 line 58
Quoted 11 lines
tests/xfs/512 new file +11
-0+ done +} + +rename_whiteout() +{ + # create the rename targetdir + renamedir=$SCRATCH_MNT/renamedir + mkdir $renamedir + + # just get a random long name... + longnamepre=FFFsafdsagafsadfagasdjfalskdgakdlsglkasdg
Better to explain why long file name is required.
Quoted 12 lines
tests/xfs/512 new file +12
-0+ + # now try to do rename with RENAME_WHITEOUT flag + i=0 + while [ $i -le $files ]; do + src/renameat2 -w $SCRATCH_MNT/f$i $renamedir/$longnamepre$i >/dev/null 2>&1 + let i=$i+1 + done +} + +create_file() +{ + # create the targetdir
tests/xfs/512 line 80
Quoted 10 lines
tests/xfs/512 new file +10
-0+ createdir=$SCRATCH_MNT/createdir + mkdir $createdir + + # try to create file at the same time to hit the deadlock + i=0 + while [ $i -le $files ]; do + file=$createdir/f$i + $XFS_IO_PROG -f -d -c 'pwrite -b 4k 0 4k' $file >/dev/null 2>&1 + let i=$i+1 + done
Same here, does creating empty files work?
tests/xfs/512 line 83
Quoted 3 lines
tests/xfs/512 new file +3
-0+} + +_scratch_mkfs_xfs -bsize=512 -dagcount=1 >> $seqres.full 2>&1 ||
This doesn't work because crc is on by default, as crc requires minimum 1k block size. Is 512 block size really needed?
tests/xfs/512 line 87
Quoted 4 lines
tests/xfs/512 new file +4
-0+ _fail "mkfs failed" +_scratch_mount + +files=250000
If we could reduce file number to create, test could run faster as well. Running test with less than 250000 files couldn't reproduce the deadlock?
Thanks,
Eryu
Quoted 35 lines
tests/xfs/512 new file +13
-0+ +prepare_file +rename_whiteout & +create_file & + +wait +echo Silence is golden + +# Failure comes in the form of a deadlock. + +# success, all done +status=0 +exittests/xfs/512.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 512 +Silence is goldentests/xfs/group +1
-0@@ -509,3 +509,4 @@ 509 auto ioctl 510 auto ioctl quick 511 auto quick quota +512 auto renameSignature · 4 lines
1.8.3.1 -- kaixuxia
Quoted 48 lines · Eryu Guan
Quoted 47 lines · kaixuxia
There is ABBA deadlock bug between the AGI and AGF when performing
rename()withRENAME_WHITEOUTflag, and add this testcase to make sure therename()call works well.Signed-off-by kaixuxia <[email protected]>--- tests/xfs/512 | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/512.out | 2 ++ tests/xfs/group | 1 + 3 files changed, 103 insertions(+) create mode 100755 tests/xfs/512 create mode 100644 tests/xfs/512.outtests/xfs/512 new file +28
-0@@ -0,0 +1,100 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 Tencent. All Rights Reserved. +# +# FS QA Test 512 +# +# Test the ABBA deadlock case between the AGI and AGF When performing +# rename operation with RENAME_WHITEOUT flag. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +
Quoted 8 lines
tests/xfs/512 line 34
Quoted 6 lines
tests/xfs/512 new file +6
-0+rm -f $seqres.full + +# real QA test starts here +_supported_fs xfs +_supported_os Linux +_require_scratchOnly
_require_scratch_nocheckis suffiecient.
Yeah, will fix it.
Quoted 22 lines
tests/xfs/512 line 38
Quoted 5 lines
tests/xfs/512 new file +4
-0+ +# Single AG will cause default xfs_repair to fail. This test need a +# single AG fs, so ignore the check. +_require_scratch_nocheckAlso need to
. ./common/rename
...
_requires_renameat2Also, this test requires
RENAME_WHITEOUT, I'd suggest enhancesrc/renameat2.cto support check for if a given rename flag is supported by kernel, and refactor the checks in generic/02[45] and generic/078 into_requires_renameat2to use the new functionality. e.g.# without option, behavior stays unchanged, check for renameat2 syscall # support _requires_renameat2
Quoted 6 lines
# check if renameat2 upports RENAME_WHITEOUT flag _requires_renameat2 whiteout# check if renameat2 upports RENAME_EXCHANGE flag _requires_renameat2 exchange
Right, this testcase requires RENAME_WHITEOUT, it is necessary that check whether the given rename flag is supported by kernel. I will try to refactor the corresponding tests and functions.
Quoted 14 lines
tests/xfs/512 line 47
Quoted 9 lines
tests/xfs/512 new file +9
-0+ +prepare_file() +{ + # create many small files for the rename with RENAME_WHITEOUT + i=0 + while [ $i -le $files ]; do + file=$SCRATCH_MNT/f$i + $XFS_IO_PROG -f -d -c 'pwrite -b 4k 0 4k' $file >/dev/null 2>&1 + let i=$i+1Creating 250000 4k files take a long time. Does file content really matters? I guess racing
RENAME_WHITEOUTwith file creation is enough, is it possible to just create many empty files? e.g.
Quoted 3 lines
echo > $file
this saves a lot time.
Yeah, create empty files is better.
Quoted 15 lines
tests/xfs/512 line 58
Quoted 12 lines
tests/xfs/512 new file +11
-0+ done +} + +rename_whiteout() +{ + # create the rename targetdir + renamedir=$SCRATCH_MNT/renamedir + mkdir $renamedir + + # just get a random long name... + longnamepre=FFFsafdsagafsadfagasdjfalskdgakdlsglkasdgBetter to explain why long file name is required.
Will add the explanation. The long file name will increase the possibility that target_dp allocate new blocks(acquire the AGF lock) to store the filename.
Quoted 16 lines
Quoted 16 lines
tests/xfs/512 new file +16
-0+ + # now try to do rename with RENAME_WHITEOUT flag + i=0 + while [ $i -le $files ]; do + src/renameat2 -w $SCRATCH_MNT/f$i $renamedir/$longnamepre$i >/dev/null 2>&1 + let i=$i+1 + done +} + +create_file() +{ + # create the targetdir + createdir=$SCRATCH_MNT/createdir + mkdir $createdir + + # try to create file at the same time to hit the deadlock
Quoted 8 lines
tests/xfs/512 line 80
Quoted 6 lines
tests/xfs/512 new file +6
-0+ i=0 + while [ $i -le $files ]; do + file=$createdir/f$i + $XFS_IO_PROG -f -d -c 'pwrite -b 4k 0 4k' $file >/dev/null 2>&1 + let i=$i+1 + doneSame here, does creating empty files work?
Yeah.
Quoted 8 lines
tests/xfs/512 line 83
Quoted 4 lines
tests/xfs/512 new file +3
-0+} + +_scratch_mkfs_xfs -bsize=512 -dagcount=1 >> $seqres.full 2>&1 ||This doesn't work because crc is on by default, as crc requires minimum 1k block size. Is 512 block size really needed?
Quoted 8 lines
tests/xfs/512 line 87
Quoted 4 lines
tests/xfs/512 new file +4
-0+ _fail "mkfs failed" +_scratch_mount + +files=250000If we could reduce file number to create, test could run faster as well. Running test with less than 250000 files couldn't reproduce the deadlock?
I used this magic number in my local test environment and reproduced the deadlock every time. Creating empty files would reduce the run time...
Thanks.
Quoted 39 lines
Thanks,
EryuQuoted 35 lines
tests/xfs/512 new file +13
-0+ +prepare_file +rename_whiteout & +create_file & + +wait +echo Silence is golden + +# Failure comes in the form of a deadlock. + +# success, all done +status=0 +exittests/xfs/512.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 512 +Silence is goldentests/xfs/group +1
-0@@ -509,3 +509,4 @@ 509 auto ioctl 510 auto ioctl quick 511 auto quick quota +512 auto renameSignature · 4 lines
1.8.3.1 -- kaixuxia
Signature · 1 line
kaixuxia