common: Fix file leak in _get_max_file_size
PATCH 1/4 common: Fix file leak in _get_max_file_size
This is obviously mostly problematic for FS lacking support for sparse files.
--- common/rc | 1 + tests/generic/692 | 0 2 files changed, 1 insertion(+) mode change 100644 => 100755 tests/generic/692
common/rc +1 -0
@@ -4637,6 +4637,7 @@ _get_max_file_size() l=$m fi done + rm -f $testfile echo $l }
tests/generic/692 +0 -0
Signature · 1 line
2.37.3
PATCH 2/4 g/299: speed up the test
Move the _require_xfs_io_command "falloc" above the potencially 'expensive' function _get_max_file_size
--- tests/generic/299 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/generic/299 +1 -1
@@ -24,6 +24,7 @@ _require_scratch _require_odirect _require_aio _require_block_device $SCRATCH_DEV +_require_xfs_io_command "falloc" NUM_JOBS=$((4*LOAD_FACTOR)) BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
@@ -96,7 +97,6 @@ filename=buffered-aio-verifier EOF _require_fio $fio_config -_require_xfs_io_command "falloc" _scratch_mkfs >> $seqres.full 2>&1 _scratch_mount
Signature · 1 line
2.37.3
Re: [PATCH 2/4] g/299: speed up the test
Quoted 5 lines · Pavel Reichl
Move the
_require_xfs_io_command"falloc" above the potencially 'expensive' function_get_max_file_sizeSigned-off-by Pavel Reichl <[email protected]>---
It's not for speeding up this test, I think you're trying to notrun this case on a fs which doesn't support fallocate and sparse file but support truncate syscall. Anyway I don't have objection on this patch.
Quoted 26 lines
tests/generic/299 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/generic/299 +1
-1@@ -24,6 +24,7 @@ _require_scratch _require_odirect _require_aio _require_block_device $SCRATCH_DEV +_require_xfs_io_command "falloc" NUM_JOBS=$((4*LOAD_FACTOR)) BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`@@ -96,7 +97,6 @@ filename=buffered-aio-verifier EOF _require_fio $fio_config -_require_xfs_io_command "falloc" _scratch_mkfs >> $seqres.full 2>&1 _scratch_mountSignature · 1 line
2.37.3
PATCH 3/4 generic: cleanup test files
In reply to Pavel ReichlThere seems to be some seek_sanity_testfile files that are not cleaned up and take up space.
-rwxr-xr-x. 1 root root 8.0G Sep 22 13:39 seek_sanity_testfile10
--- tests/generic/285 | 2 +- tests/generic/436 | 2 +- tests/generic/445 | 2 +- tests/generic/448 | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
tests/generic/285 +1 -1
@@ -28,7 +28,7 @@ _require_test_program "seek_sanity_test" # Override the default cleanup function. _cleanup() { - eval "rm -f $BASE_TEST_FILE.*" + rm -f $BASE_TEST_FILE* } _run_seek_sanity_test $BASE_TEST_FILE > $seqres.full 2>&1 ||
tests/generic/436 +1 -1
@@ -16,7 +16,7 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile # Override the default cleanup function. _cleanup() { - rm -f $tmp.* $BASE_TEST_FILE.* + rm -f $tmp.* $BASE_TEST_FILE* } # Import common functions.
tests/generic/445 +1 -1
@@ -16,7 +16,7 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile # Override the default cleanup function. _cleanup() { - rm -f $tmp.* $BASE_TEST_FILE.* + rm -f $tmp.* $BASE_TEST_FILE* } # Import common functions.
tests/generic/448 +1 -1
@@ -15,7 +15,7 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile_$seq _cleanup() { cd / - rm -f $tmp.* $BASE_TEST_FILE + rm -f $tmp.* $BASE_TEST_FILE* } # Import common functions.
Signature · 1 line
2.37.3
Re: [PATCH 3/4] generic: cleanup test files
Quoted 18 lines · Pavel Reichl
There seems to be some
seek_sanity_testfilefiles that are not cleaned up and take up space.-rwxr-xr-x. 1 root root 8.0G Sep 22 13:39
seek_sanity_testfile10Signed-off-by Pavel Reichl <[email protected]>--- tests/generic/285 | 2 +- tests/generic/436 | 2 +- tests/generic/445 | 2 +- tests/generic/448 | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)tests/generic/285 +0
-0
tests/generic/285 _require_test_program "seek_sanity_test" · line 31
Quoted 5 lines
tests/generic/285 +1
-1@@ -28,7 +28,7 @@ _require_test_program "seek_sanity_test" # Override the default cleanup function. _cleanup() { - eval "rm -f $BASE_TEST_FILE.*" + rm -f $BASE_TEST_FILE*
OK, this eval looks not reasonable.
tests/generic/436 BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile
Quoted 8 lines
tests/generic/285 +0
-0} _run_seek_sanity_test $BASE_TEST_FILE > $seqres.full 2>&1 ||tests/generic/436 +0
-0@@ -16,7 +16,7 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile
I'm wondering two questions:
1) If these cases all use $TEST_DIR/seek_sanity_testfile, and don't really remove these seek_sanity_testfile* files, will them affect each testing result? Can we set BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile_$seq at least, to avoid they use same files?
2) If these $BASE_TEST_FILE* files won't take too much space, and won't affect later testing, how about keeping them in $TEST_DIR, then remove these specific _cleanup(). From my testing, after running g/285, g/436, g/445 and g/448, I got [1]. Looks like they only takes ~25M space.
Thanks,
Zorro
[1]
# du -h /mnt/test/ 25M /mnt/test/ # du -h /mnt/test/seek_sanity_testfile* 4.0K /mnt/test/seek_sanity_testfile 0 /mnt/test/seek_sanity_testfile01 4.0K /mnt/test/seek_sanity_testfile02 12K /mnt/test/seek_sanity_testfile03 4.0K /mnt/test/seek_sanity_testfile04 4.0K /mnt/test/seek_sanity_testfile05 8.0K /mnt/test/seek_sanity_testfile06 44K /mnt/test/seek_sanity_testfile07 44K /mnt/test/seek_sanity_testfile08 404K /mnt/test/seek_sanity_testfile09 128K /mnt/test/seek_sanity_testfile10 128K /mnt/test/seek_sanity_testfile11 128K /mnt/test/seek_sanity_testfile12 4.0M /mnt/test/seek_sanity_testfile13 4.0M /mnt/test/seek_sanity_testfile14 12M /mnt/test/seek_sanity_testfile15 4.0M /mnt/test/seek_sanity_testfile16 0 /mnt/test/seek_sanity_testfile17 0 /mnt/test/seek_sanity_testfile_44818
Quoted 40 lines
tests/generic/436 +1
-1# Override the default cleanup function. _cleanup() { - rm -f $tmp.* $BASE_TEST_FILE.* + rm -f $tmp.* $BASE_TEST_FILE* } # Import common functions.tests/generic/445 +1
-1@@ -16,7 +16,7 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile # Override the default cleanup function. _cleanup() { - rm -f $tmp.* $BASE_TEST_FILE.* + rm -f $tmp.* $BASE_TEST_FILE* } # Import common functions.tests/generic/448 +1
-1@@ -15,7 +15,7 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile_$seq _cleanup() { cd / - rm -f $tmp.* $BASE_TEST_FILE + rm -f $tmp.* $BASE_TEST_FILE*}
# Import common functions.
Signature · 1 line
2.37.3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 3/4] generic: cleanup test files
Quoted 10 lines · Pavel Reichl
2) If these $BASE_TEST_FILE* files won't take too much space, and won't affect later testing, how about keeping them in $TEST_DIR, then remove these specific
_cleanup(). From my testing, after running g/285, g/436, g/445 and g/448, I got [1]. Looks like they only takes ~25M space.Hi,
as I mentioned in the commit message - in my case I'm getting:
8.0Gseek_sanity_testfile10
So I think it's good idea to solve it somehow.
OK, you can combine patch 1,3,4 into one patch, due to they all take too much space on filesystem which doesn't support sparse file, so we make sure these files be cleanup.
Re: [PATCH 3/4] generic: cleanup test files
In reply to Zorro LangQuoted 6 lines
I'm wondering two questions:
1) If these cases all use $TEST_DIR/seek_sanity_testfile, and don't really remove these seek_sanity_testfile* files, will them affect each testing result? Can we set BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile_$seq at least, to avoid they use same files?
I'm happy to try to implement this idea.
Re: [PATCH 3/4] generic: cleanup test files
Quoted 7 lines · Pavel Reichl
Quoted 6 lines
I'm wondering two questions:
1) If these cases all use $TEST_DIR/seek_sanity_testfile, and don't really remove these seek_sanity_testfile* files, will them affect each testing result? Can we set BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile_$seq at least, to avoid they use same files?I'm happy to try to implement this idea.
If you need to remove them in cleanup, then the name won't conflict :)
Quoted 1 line
PATCH 4/4 g/394: clean up test files
In reply to Pavel ReichlClean up test files taking space.
-rwxr-xr-x. 1 root root 1.0G Sep 22 14:58 394.1183899 -rwxr-xr-x. 1 root root 1.0G Sep 22 14:58 394.1183899-1
-rwxr-xr-x. 1 root root 0 sep 22 14:58 394.1183899+1
--- tests/generic/394 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/generic/394 +1 -1
@@ -14,7 +14,7 @@ _cleanup() { cd / ulimit -f unlimited - rm -f $tmp.* + rm -f $tmp.* $TEST_DIR/$seq.* } # Import common functions.
Signature · 1 line
2.37.3
Re: [PATCH 4/4] g/394: clean up test files
Quoted 16 lines · Pavel Reichl
Clean up test files taking space.
(quoted diff) +0
-3-rwxr-xr-x. 1 root root 1.0G Sep 22 14:58 394.1183899 -rwxr-xr-x. 1 root root 1.0G Sep 22 14:58 394.1183899-1 -rwxr-xr-x. 1 root root 0 sep 22 14:58 394.1183899+1Signed-off-by Pavel Reichl <[email protected]>--- tests/generic/394 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/generic/394 +0
-0
tests/generic/394 _cleanup() · line 17
Quoted 5 lines
tests/generic/394 +1
-1@@ -14,7 +14,7 @@ _cleanup() { cd / ulimit -f unlimited - rm -f $tmp.* + rm -f $tmp.* $TEST_DIR/$seq.*
I think you can combine this patch with patch 1/4, due to they're for same purpose, all due to some fs (e.g. exfat) doesn't support sparse file, so we need to remove those large files created by truncate. If you find same issue on other places, better to fix them in same patch.
Quoted 6 lines
tests/generic/394 +0
-0} # Import common functions.Signature · 1 line
2.37.3
Quoted 15 lines · Pavel Reichl
This is obviously mostly problematic for FS lacking support for sparse files.
Signed-off-by Pavel Reichl <[email protected]>--- common/rc | 1 + tests/generic/692 | 0 2 files changed, 1 insertion(+) mode change 100644 => 100755 tests/generic/692common/rc +0
-0
common/rc _get_max_file_size() · line 4640
Quoted 4 lines
common/rc +1
-0@@ -4637,6 +4637,7 @@ _get_max_file_size() l=$m fi done + rm -f $testfile
OK, as "truncate" will allocate real space in exfat and other fs not support sparse file, I think we'd better to remove it. If you find more similar situation, please remove them in same one patch :)
tests/generic/692
Quoted 6 lines
common/rc +0
-0echo $l }tests/generic/692 +0
-0
Hmm... is this another fix? Although I don't know if it's worth to be a single patch, if it doesn't cause anything wrong?
Thanks,
Zorro
Quoted 3 lines
Signature · 1 line
2.37.3
Quoted 7 lines
tests/generic/692
Quoted 5 lines
tests/generic/692 +0
-0Hmm... is this another fix? Although I don't know if it's worth to be a single patch, if it doesn't cause anything wrong?
Hmm, sorry about this change, it's not on purpose, I'm not sure what happened here. I'll remove it in the next iteration
Quoted 15 lines · Pavel Reichl
This is obviously mostly problematic for FS lacking support for sparse files.
Signed-off-by Pavel Reichl <[email protected]>--- common/rc | 1 + tests/generic/692 | 0 2 files changed, 1 insertion(+) mode change 100644 => 100755 tests/generic/692common/rc +0
-0
common/rc _get_max_file_size() · line 4642
Quoted 6 lines
common/rc +1
-0@@ -4637,6 +4637,7 @@ _get_max_file_size() l=$m fi done + rm -f $testfile echo $l }
Removing the file is fine, but I didn't have filesystems that lack support for sparse files in mind when I wrote this function. Maybe it should look at $FSTYP first, and only use the generic algorithm as a fallback for unhandled types?
- Eric
Quoted 11 lines · Eric Biggers
Quoted 10 lines · Pavel Reichl
This is obviously mostly problematic for FS lacking support for sparse files.
Signed-off-by Pavel Reichl <[email protected]>--- common/rc | 1 + tests/generic/692 | 0 2 files changed, 1 insertion(+) mode change 100644 => 100755 tests/generic/692
Quoted 15 lines
common/rc _get_max_file_size() · line 4642
Quoted 11 lines
common/rc +1
-0@@ -4637,6 +4637,7 @@ _get_max_file_size() l=$m fi done + rm -f $testfile echo $l }Removing the file is fine, but I didn't have filesystems that lack support for sparse files in mind when I wrote this function. Maybe it should look at $FSTYP first, and only use the generic algorithm as a fallback for unhandled types?
That might be good idea. The current algorithm takes too long time for fs which doesn't support sparse file.
Anyway, this $testfile should be removed. If Pavel would like to change this function likes:
_get_max_file_size()
{
case $FSTYP:
exfat)
# calculate the max file size of exfat according to blocksize, then
echo $max_file_size_of_exfat
;;
# Add ext* if extN forks think it's better
*)
# By default, do the algorithm this function does originally
;;
esac
}he can split this part as a separated patch, then let's review it. Or we can have this patch at first, then let Eric change it when he'd like to do that.
Thanks,
Zorro
Quoted 3 lines
- Eric