xfs/014: try a few times to create speculative preallocations
PATCH 1/1 xfs/014: try a few times to create speculative preallocations
From: Darrick J. Wong <[email protected]>
This test checks that speculative file preallocations are transferred to threads writing other files when space is low. Since we have background threads to clear those preallocations, it's possible that the test program might not get a speculative preallocation on the first try.
This problem has become more pronounced since the introduction of background inode inactivation since userspace no longer has direct control over the timing of file blocks being released from unlinked files. As a result, the author has seen an increase in sporadic warnings from this test about speculative preallocations not appearing.
Therefore, modify the function to try up to five times to create the speculative preallocation before emitting warnings that then cause golden output failures.
--- tests/xfs/014 | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-)
tests/xfs/014 +25 -16
@@ -33,27 +33,36 @@ _cleanup() # failure. _spec_prealloc_file() { - file=$1 + local file=$1 + local prealloc_size=0 + local i=0 - rm -f $file + # Now that we have background garbage collection processes that can be + # triggered by low space/quota conditions, it's possible that we won't + # succeed in creating a speculative preallocation on the first try. + for ((tries = 0; tries < 5 && prealloc_size == 0; tries++)); do + rm -f $file - # a few file extending open-write-close cycles should be enough to - # trigger the fs to retain preallocation. write 256k in 32k intervals to - # be sure - for i in $(seq 0 32768 262144); do - $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + # a few file extending open-write-close cycles should be enough + # to trigger the fs to retain preallocation. write 256k in 32k + # intervals to be sure + for i in $(seq 0 32768 262144); do + $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + done + + # write a 4k aligned amount of data to keep the calculations + # simple + $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full + + size=`_get_filesize $file` + blocks=`stat -c "%b" $file` + blocksize=`stat -c "%B" $file` + + prealloc_size=$((blocks * blocksize - size)) done - # write a 4k aligned amount of data to keep the calculations simple - $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full - - size=`_get_filesize $file` - blocks=`stat -c "%b" $file` - blocksize=`stat -c "%B" $file` - - prealloc_size=$((blocks * blocksize - size)) if [ $prealloc_size -eq 0 ]; then - echo "Warning: No speculative preallocation for $file." \ + echo "Warning: No speculative preallocation for $file after $tries iterations." \ "Check use of the allocsize= mount option." fi
Quoted 53 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This test checks that speculative file preallocations are transferred to threads writing other files when space is low. Since we have background threads to clear those preallocations, it's possible that the test program might not get a speculative preallocation on the first try.
This problem has become more pronounced since the introduction of background inode inactivation since userspace no longer has direct control over the timing of file blocks being released from unlinked files. As a result, the author has seen an increase in sporadic warnings from this test about speculative preallocations not appearing.
Therefore, modify the function to try up to five times to create the speculative preallocation before emitting warnings that then cause golden output failures.
Signed-off-by Darrick J. Wong <[email protected]>--- tests/xfs/014 | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-)tests/xfs/014 +14
-7@@ -33,27 +33,36 @@ _cleanup() # failure. _spec_prealloc_file() { - file=$1 + local file=$1 + local prealloc_size=0 + local i=0 - rm -f $file + # Now that we have background garbage collection processes that can be + # triggered by low space/quota conditions, it's possible that we won't + # succeed in creating a speculative preallocation on the first try. + for ((tries = 0; tries < 5 && prealloc_size == 0; tries++)); do + rm -f $file - # a few file extending open-write-close cycles should be enough to - # trigger the fs to retain preallocation. write 256k in 32k intervals to - # be sure - for i in $(seq 0 32768 262144); do - $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + # a few file extending open-write-close cycles should be enough + # to trigger the fs to retain preallocation. write 256k in 32k + # intervals to be sure + for i in $(seq 0 32768 262144); do + $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + done
tests/xfs/014 _cleanup() · line 61
Quoted 10 lines
tests/xfs/014 +10
-0+ + # write a 4k aligned amount of data to keep the calculations + # simple + $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full + + size=`_get_filesize $file` + blocks=`stat -c "%b" $file` + blocksize=`stat -c "%B" $file` + + prealloc_size=$((blocks * blocksize - size))
So we only try same pwrite operations 5 times, and only check the prealloc_size after 5 times done? Should we break from this loop once prealloc_size > 0?
Thanks,
Zorro
Quoted 17 lines
tests/xfs/014 +1
-9done - # write a 4k aligned amount of data to keep the calculations simple - $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full - - size=`_get_filesize $file` - blocks=`stat -c "%b" $file` - blocksize=`stat -c "%B" $file` - - prealloc_size=$((blocks * blocksize - size)) if [ $prealloc_size -eq 0 ]; then - echo "Warning: No speculative preallocation for $file." \ + echo "Warning: No speculative preallocation for $file after $tries iterations." \ "Check use of the allocsize= mount option." fi
Quoted 59 lines · Zorro Lang
Quoted 58 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This test checks that speculative file preallocations are transferred to threads writing other files when space is low. Since we have background threads to clear those preallocations, it's possible that the test program might not get a speculative preallocation on the first try.
This problem has become more pronounced since the introduction of background inode inactivation since userspace no longer has direct control over the timing of file blocks being released from unlinked files. As a result, the author has seen an increase in sporadic warnings from this test about speculative preallocations not appearing.
Therefore, modify the function to try up to five times to create the speculative preallocation before emitting warnings that then cause golden output failures.
Signed-off-by Darrick J. Wong <[email protected]>--- tests/xfs/014 | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-)tests/xfs/014 +19
-7@@ -33,27 +33,36 @@ _cleanup() # failure. _spec_prealloc_file() { - file=$1 + local file=$1 + local prealloc_size=0 + local i=0 - rm -f $file + # Now that we have background garbage collection processes that can be + # triggered by low space/quota conditions, it's possible that we won't + # succeed in creating a speculative preallocation on the first try. + for ((tries = 0; tries < 5 && prealloc_size == 0; tries++)); do + rm -f $file - # a few file extending open-write-close cycles should be enough to - # trigger the fs to retain preallocation. write 256k in 32k intervals to - # be sure - for i in $(seq 0 32768 262144); do - $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + # a few file extending open-write-close cycles should be enough + # to trigger the fs to retain preallocation. write 256k in 32k + # intervals to be sure + for i in $(seq 0 32768 262144); do + $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + done + + # write a 4k aligned amount of data to keep the calculations + # simple + $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full +
Quoted 8 lines
tests/xfs/014 _cleanup() · line 61
Quoted 5 lines
tests/xfs/014 +5
-0+ size=`_get_filesize $file` + blocks=`stat -c "%b" $file` + blocksize=`stat -c "%B" $file` + + prealloc_size=$((blocks * blocksize - size))So we only try same pwrite operations 5 times, and only check the
prealloc_sizeafter 5 times done? Should we break from this loop onceprealloc_size> 0?
The second clause of the for loop tests for that, does it not?
--D
Quoted 22 lines
Thanks,
ZorroQuoted 17 lines
tests/xfs/014 +1
-9done - # write a 4k aligned amount of data to keep the calculations simple - $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full - - size=`_get_filesize $file` - blocks=`stat -c "%b" $file` - blocksize=`stat -c "%B" $file` - - prealloc_size=$((blocks * blocksize - size)) if [ $prealloc_size -eq 0 ]; then - echo "Warning: No speculative preallocation for $file." \ + echo "Warning: No speculative preallocation for $file after $tries iterations." \ "Check use of the allocsize= mount option." fi
Zorro Lang
Reviewed-byQuoted 69 lines · Darrick J. Wong
Quoted 59 lines · Zorro Lang
Quoted 58 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
This test checks that speculative file preallocations are transferred to threads writing other files when space is low. Since we have background threads to clear those preallocations, it's possible that the test program might not get a speculative preallocation on the first try.
This problem has become more pronounced since the introduction of background inode inactivation since userspace no longer has direct control over the timing of file blocks being released from unlinked files. As a result, the author has seen an increase in sporadic warnings from this test about speculative preallocations not appearing.
Therefore, modify the function to try up to five times to create the speculative preallocation before emitting warnings that then cause golden output failures.
Signed-off-by Darrick J. Wong <[email protected]>--- tests/xfs/014 | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-)tests/xfs/014 +19
-7@@ -33,27 +33,36 @@ _cleanup() # failure. _spec_prealloc_file() { - file=$1 + local file=$1 + local prealloc_size=0 + local i=0 - rm -f $file + # Now that we have background garbage collection processes that can be + # triggered by low space/quota conditions, it's possible that we won't + # succeed in creating a speculative preallocation on the first try. + for ((tries = 0; tries < 5 && prealloc_size == 0; tries++)); do + rm -f $file - # a few file extending open-write-close cycles should be enough to - # trigger the fs to retain preallocation. write 256k in 32k intervals to - # be sure - for i in $(seq 0 32768 262144); do - $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + # a few file extending open-write-close cycles should be enough + # to trigger the fs to retain preallocation. write 256k in 32k + # intervals to be sure + for i in $(seq 0 32768 262144); do + $XFS_IO_PROG -f -c "pwrite $i 32k" $file >> $seqres.full + done + + # write a 4k aligned amount of data to keep the calculations + # simple + $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full +Quoted 8 lines
tests/xfs/014 _cleanup() · line 61
Quoted 5 lines
tests/xfs/014 +5
-0+ size=`_get_filesize $file` + blocks=`stat -c "%b" $file` + blocksize=`stat -c "%B" $file` + + prealloc_size=$((blocks * blocksize - size))So we only try same pwrite operations 5 times, and only check the
prealloc_sizeafter 5 times done? Should we break from this loop onceprealloc_size> 0?
Quoted 1 line
The second clause of the for loop tests for that, does it not?
Oh, yes, sorry I missed the "&& prealloc_size == 0", I thought you just gave it 5 tries :) So this patch is good to me.
Quoted 26 lines
--D
Quoted 22 lines
Thanks,
ZorroQuoted 17 lines
tests/xfs/014 +1
-9done - # write a 4k aligned amount of data to keep the calculations simple - $XFS_IO_PROG -c "pwrite 0 128m" $file >> $seqres.full - - size=`_get_filesize $file` - blocks=`stat -c "%b" $file` - blocksize=`stat -c "%B" $file` - - prealloc_size=$((blocks * blocksize - size)) if [ $prealloc_size -eq 0 ]; then - echo "Warning: No speculative preallocation for $file." \ + echo "Warning: No speculative preallocation for $file after $tries iterations." \ "Check use of the allocsize= mount option." fi