Skip to contentSkip to conversation

iotests: Check for the possibility to create large files

PATCH Applied Thomas Huth 3 participants 7 messages Dec 2, 2019 – Dec 9, 2019

v1 of 2 Applied Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover iotests: Check for the possibility to create large files

Travis recently added the possibility to test on ppc64le, arm64 and s390x hosts, too. However, the containers are very restricted there and do not allow the creation of large files, so that the tests 060 and 079 are currently failing there. So let's add some proper checks to these tests first.

Thomas Huth (3):
iotests: Provide a function for checking the creation of huge files
iotests: Skip test 060 if it is not possible to create large files
iotests: Skip test 079 if it is not possible to create large files

 tests/qemu-iotests/005       |  5 +----
 tests/qemu-iotests/060       |  3 +++
 tests/qemu-iotests/079       |  3 +++
 tests/qemu-iotests/220       |  6 ++----
 tests/qemu-iotests/common.rc | 10 ++++++++++
 5 files changed, 19 insertions(+), 8 deletions(-)
Signature · 1 line
2.18.1
#1 lore raw

PATCH 1/3 iotests: Provide a function for checking the creation of huge files

Some tests create huge (but sparse) files, and to be able to run those tests in certain limited environments (like CI containers), we have to check for the possibility to create such files first. Thus let's introduce a common function to check for large files, and replace the already existing checks in the iotests 005 and 220 with this function.

---
 tests/qemu-iotests/005       |  5 +----
 tests/qemu-iotests/220       |  6 ++----
 tests/qemu-iotests/common.rc | 10 ++++++++++
 3 files changed, 13 insertions(+), 8 deletions(-)
tests/qemu-iotests/005 +1 -4
index 58442762fe..b6d03ac37d 100755
@@ -59,10 +59,7 @@ fi
 # Sanity check: For raw, we require a file system that permits the creation
 # of a HUGE (but very sparse) file. Check we can create it before continuing.
 if [ "$IMGFMT" = "raw" ]; then
-    if ! truncate --size=5T "$TEST_IMG"; then
-        _notrun "file system on $TEST_DIR does not support large enough files"
-    fi
-    rm "$TEST_IMG"
+    _require_large_file 5T
 fi
 
 echo
tests/qemu-iotests/220 +2 -4
index 2d62c5dcac..15159270d3 100755
@@ -42,10 +42,8 @@ echo "== Creating huge file =="
 
 # Sanity check: We require a file system that permits the creation
 # of a HUGE (but very sparse) file.  tmpfs works, ext4 does not.
-if ! truncate --size=513T "$TEST_IMG"; then
-    _notrun "file system on $TEST_DIR does not support large enough files"
-fi
-rm "$TEST_IMG"
+_require_large_file 513T
+
 IMGOPTS='cluster_size=2M,refcount_bits=1' _make_test_img 513T
 
 echo "== Populating refcounts =="
tests/qemu-iotests/common.rc +10 -0
index 38e949cf69..91c0217e59 100644
@@ -657,5 +657,15 @@ _require_devices()
     done
 }
 
+# Check that we have a file system that allows huge (but very sparse) files
+#
+_require_large_file()
+{
+    if ! truncate --size="$1" "$TEST_IMG"; then
+        _notrun "file system on $TEST_DIR does not support large enough files"
+    fi
+    rm "$TEST_IMG"
+}
+
 # make sure this script returns success
 true
Signature · 1 line
2.18.1
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] iotests: Provide a function for checking the creation of huge files

Quoted 7 lines · Thomas Huth

Some tests create huge (but sparse) files, and to be able to run those tests in certain limited environments (like CI containers), we have to check for the possibility to create such files first. Thus let's introduce a common function to check for large files, and replace the already existing checks in the iotests 005 and 220 with this function.

Quoted 59 lines
---
 tests/qemu-iotests/005       |  5 +----
 tests/qemu-iotests/220       |  6 ++----
 tests/qemu-iotests/common.rc | 10 ++++++++++
 3 files changed, 13 insertions(+), 8 deletions(-)
tests/qemu-iotests/005 +1 -4
index 58442762fe..b6d03ac37d 100755
@@ -59,10 +59,7 @@ fi
 # Sanity check: For raw, we require a file system that permits the creation
 # of a HUGE (but very sparse) file. Check we can create it before continuing.
 if [ "$IMGFMT" = "raw" ]; then
-    if ! truncate --size=5T "$TEST_IMG"; then
-        _notrun "file system on $TEST_DIR does not support large enough files"
-    fi
-    rm "$TEST_IMG"
+    _require_large_file 5T
 fi
 
 echo
tests/qemu-iotests/220 +2 -4
index 2d62c5dcac..15159270d3 100755
@@ -42,10 +42,8 @@ echo "== Creating huge file =="
 
 # Sanity check: We require a file system that permits the creation
 # of a HUGE (but very sparse) file.  tmpfs works, ext4 does not.
-if ! truncate --size=513T "$TEST_IMG"; then
-    _notrun "file system on $TEST_DIR does not support large enough files"
-fi
-rm "$TEST_IMG"
+_require_large_file 513T
+
 IMGOPTS='cluster_size=2M,refcount_bits=1' _make_test_img 513T
 
 echo "== Populating refcounts =="
tests/qemu-iotests/common.rc +10 -0
index 38e949cf69..91c0217e59 100644
@@ -657,5 +657,15 @@ _require_devices()
     done
 }
 
+# Check that we have a file system that allows huge (but very sparse) files
+#
+_require_large_file()
+{
+    if ! truncate --size="$1" "$TEST_IMG"; then
+        _notrun "file system on $TEST_DIR does not support large enough files"
+    fi
+    rm "$TEST_IMG"
+}
+
 # make sure this script returns success
 true
Signature · 1 line
Alex Bennée
#3 lore raw

PATCH 2/3 iotests: Skip test 060 if it is not possible to create large files

In reply to Thomas Huth

Test 060 fails in the arm64, s390x and ppc64le LXD containers on Travis (which we will hopefully enable in our CI soon). These containers apparently do not allow large files to be created. The repair process in test 060 creates a file of 64 GiB, so test first whether such large files are possible and skip the test if that's not the case.

---
 tests/qemu-iotests/060 | 3 +++
 1 file changed, 3 insertions(+)
tests/qemu-iotests/060 +3 -0
index b91d8321bb..d96f17a484 100755
@@ -49,6 +49,9 @@ _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
 
+# The repair process will create a large file - so check for availability first
+_require_large_file 64G
+
 rt_offset=65536  # 0x10000 (XXX: just an assumption)
 rb_offset=131072 # 0x20000 (XXX: just an assumption)
 l1_offset=196608 # 0x30000 (XXX: just an assumption)
Signature · 1 line
2.18.1
#4 lore raw

PATCH 3/3 iotests: Skip test 079 if it is not possible to create large files

In reply to Thomas Huth

Test 079 fails in the arm64, s390x and ppc64le LXD containers on Travis (which we will hopefully enable in our CI soon). These containers apparently do not allow large files to be created. Test 079 tries to create a 4G sparse file, which is apparently already too big for these containers, so check first whether we can really create such files before executing the test.

---
 tests/qemu-iotests/079 | 3 +++
 1 file changed, 3 insertions(+)
tests/qemu-iotests/079 +3 -0
index 81f0c21f53..78536d3bbf 100755
@@ -39,6 +39,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file nfs
 
+# Some containers (e.g. non-x86 on Travis) do not allow large files
+_require_large_file 4G
+
 echo "=== Check option preallocation and cluster_size ==="
 echo
 cluster_sizes="16384 32768 65536 131072 262144 524288 1048576 2097152 4194304"
Signature · 1 line
2.18.1
#5 lore raw
In reply to Thomas Huth
Quoted 5 lines · Thomas Huth

Travis recently added the possibility to test on ppc64le, arm64 and s390x hosts, too. However, the containers are very restricted there and do not allow the creation of large files, so that the tests 060 and 079 are currently failing there. So let's add some proper checks to these tests first.

These look good to me, do you want them to go via my testing/next or are Kevin and Max going to take it via their tree?

Quoted 12 lines

Thomas Huth (3):
iotests: Provide a function for checking the creation of huge files
iotests: Skip test 060 if it is not possible to create large files
iotests: Skip test 079 if it is not possible to create large files

 tests/qemu-iotests/005       |  5 +----
 tests/qemu-iotests/060       |  3 +++
 tests/qemu-iotests/079       |  3 +++
 tests/qemu-iotests/220       |  6 ++----
 tests/qemu-iotests/common.rc | 10 ++++++++++
 5 files changed, 19 insertions(+), 8 deletions(-)
Signature · 1 line
Alex Bennée
Applied
#6 lore raw
In reply to Thomas Huth

Am 02.12.2019 um 11:16 hat Thomas Huth geschrieben:

Quoted 5 lines

Travis recently added the possibility to test on ppc64le, arm64 and s390x hosts, too. However, the containers are very restricted there and do not allow the creation of large files, so that the tests 060 and 079 are currently failing there. So let's add some proper checks to these tests first.

Thanks, applied to the block branch.

Kevin