Skip to contentSkip to conversation

qemu-iotests improvements

PATCH In review Jeff Cody 4 participants 37 messages Oct 17, 2017 – Oct 19, 2017

lore
Share & tools
lore

Jeff Cody

author
#0 lore raw

v5 cover qemu-iotests improvements

Previous series subject: "qemu-iotests: place output in unique dir"

Changes v4 -> v5:
-------------------

Relatively minor changes:

* line-too-long complaint by patchew

* fixed running from out-of-tree

* fixed 2 other bugs found: one test was forgotten, and some QEMU process FIFOs were left behind.

* output formatting changes

git-backport-diff -r qemu/master.. -u devel-iotests-v4
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/10:[----] [--] 'qemu-iotests: refuse to run if TEST_DIR contains spaces' 002/10:[0012] [FC] 'qemu-iotests: set TEST_DIR to a unique dir for each test' 003/10:[----] [-C] 'qemu-iotests: automatically clean up bash protocol servers' 004/10:[----] [--] 'qemu-iotests: remove file cleanup from bash tests' 005/10:[0064] [FC] 'qemu-iotests: change qemu pid and fd tracking / cleanup' 006/10:[0007] [FC] 'qemu-iotests: make ./check automatically reap QEMU processes' 007/10:[0003] [FC] 'qemu-iotests: run python tests in own subdirectories' 008/10:[0002] [FC] 'qemu-iotests: modify python tests to run from subdir' 009/10:[----] [--] 'qemu-iotests: add option to save temp files on error' 010/10:[0066] [FC] 'qemu-iotests: add support for running multi-threaded iotests'

Changes v3 -> v4:
-------------------

Significant changes from v3. Highlights of the major changes from v3:

* Python tests are properly run in hierarchial sub-directories now (Thanks Stefan, John)

* Protocol servers are cleaned up automatically (Thanks Kevin)

* Prevent running qemu-iotests if TEST_DIR contains spaces

* common.qemu process reaping overhauled, and qemu processes also automatically killed on test conclusion

* multi-thread iotest job support. Here is an example of the speedup:

Previous, single-thread run of everything qcow2:

# ./check -qcow2

[...]

Not run: 045 059 064 070 075 076 077 078 081 083 084 088 092 093 094 101

         106 109 113 116 119 123 128 131 135 136 146 148 149 160 171 173 175

Failures: 191
Failed 1 of 149 tests

real    8m22.077s
user    4m48.177s
sys     1m16.553s

Multi-process run:

# ./check -qcow2 -j 5

[...]

Not run: 045 059 064 070 075 076 077 078 081 083 084 088 092 093 094 101

         106 109 113 116 119 123 128 131 135 136 146 148 149 160 171 173 175

Failures: 183 191
Failed 2 of 149 tests

real    3m7.458s
user    5m29.678s
sys     1m55.007s

(See commit message on patch 10 for why there is an additional test
failure)

git-backport-diff -r qemu/master.. -u devel-iotests-v3
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/10:[down] 'qemu-iotests: refuse to run if TEST_DIR contains spaces' 002/10:[----] [--] 'qemu-iotests: set TEST_DIR to a unique dir for each test' 003/10:[down] 'qemu-iotests: automatically clean up bash protocol servers' 004/10:[0073] [FC] 'qemu-iotests: remove file cleanup from bash tests' 005/10:[down] 'qemu-iotests: change qemu pid and fd tracking / cleanup' 006/10:[down] 'qemu-iotests: make ./check automatically reap QEMU processes' 007/10:[down] 'qemu-iotests: run python tests in own subdirectories' 008/10:[down] 'qemu-iotests: modify python tests to run from subdir' 009/10:[0017] [FC] 'qemu-iotests: add option to save temp files on error' 010/10:[down] 'qemu-iotests: add support for running multi-threaded iotests'

Jeff Cody (10):
qemu-iotests: refuse to run if TEST_DIR contains spaces
qemu-iotests: set TEST_DIR to a unique dir for each test
qemu-iotests: automatically clean up bash protocol servers
qemu-iotests: remove file cleanup from bash tests
qemu-iotests: change qemu pid and fd tracking / cleanup
qemu-iotests: make ./check automatically reap QEMU processes
qemu-iotests: run python tests in own subdirectories
qemu-iotests: modify python tests to run from subdir
qemu-iotests: add option to save temp files on error
qemu-iotests: add support for running multi-threaded iotests

 tests/qemu-iotests/001         |   6 -
 tests/qemu-iotests/002         |   6 -
 tests/qemu-iotests/003         |   6 -
 tests/qemu-iotests/004         |   6 -
 tests/qemu-iotests/005         |   6 -
 tests/qemu-iotests/007         |   7 -
 tests/qemu-iotests/008         |   6 -
 tests/qemu-iotests/009         |   6 -
 tests/qemu-iotests/010         |   6 -
 tests/qemu-iotests/011         |   6 -
 tests/qemu-iotests/012         |   6 -
 tests/qemu-iotests/013         |   6 -
 tests/qemu-iotests/014         |   6 -
 tests/qemu-iotests/015         |   7 -
 tests/qemu-iotests/017         |   6 -
 tests/qemu-iotests/018         |   6 -
 tests/qemu-iotests/019         |   8 -
 tests/qemu-iotests/020         |   8 -
 tests/qemu-iotests/021         |   6 -
 tests/qemu-iotests/022         |   6 -
 tests/qemu-iotests/023         |   6 -
 tests/qemu-iotests/024         |   8 -
 tests/qemu-iotests/025         |   6 -
 tests/qemu-iotests/026         |   7 -
 tests/qemu-iotests/027         |   6 -
 tests/qemu-iotests/028         |   8 -
 tests/qemu-iotests/029         |   7 -
 tests/qemu-iotests/030         |  82 ++++-----
 tests/qemu-iotests/031         |   6 -
 tests/qemu-iotests/032         |   6 -
 tests/qemu-iotests/033         |   6 -
 tests/qemu-iotests/034         |   6 -
 tests/qemu-iotests/035         |   6 -
 tests/qemu-iotests/036         |   6 -
 tests/qemu-iotests/037         |   6 -
 tests/qemu-iotests/038         |   6 -
 tests/qemu-iotests/039         |   6 -
 tests/qemu-iotests/040         | 128 ++++++-------
 tests/qemu-iotests/041         | 333 ++++++++++++++++-----------------
 tests/qemu-iotests/042         |   6 -
 tests/qemu-iotests/043         |   7 -
 tests/qemu-iotests/044         |  11 +-
 tests/qemu-iotests/045         |  42 ++---
 tests/qemu-iotests/046         |   6 -
 tests/qemu-iotests/047         |   6 -
 tests/qemu-iotests/048         |   8 -
 tests/qemu-iotests/048.out     |   1 -
 tests/qemu-iotests/049         |   6 -
 tests/qemu-iotests/050         |   8 -
 tests/qemu-iotests/051         |   6 -
 tests/qemu-iotests/052         |   6 -
 tests/qemu-iotests/053         |   7 -
 tests/qemu-iotests/054         |   6 -
 tests/qemu-iotests/055         |  97 +++++-----
 tests/qemu-iotests/056         |  39 ++--
 tests/qemu-iotests/057         |   4 +-
 tests/qemu-iotests/058         |  47 ++---
 tests/qemu-iotests/059         |   7 -
 tests/qemu-iotests/060         |   6 -
 tests/qemu-iotests/061         |   6 -
 tests/qemu-iotests/062         |   6 -
 tests/qemu-iotests/063         |   7 -
 tests/qemu-iotests/064         |   6 -
 tests/qemu-iotests/065         |  13 +-
 tests/qemu-iotests/066         |   6 -
 tests/qemu-iotests/068         |   6 -
 tests/qemu-iotests/069         |   6 -
 tests/qemu-iotests/070         |   6 -
 tests/qemu-iotests/071         |   6 -
 tests/qemu-iotests/072         |   6 -
 tests/qemu-iotests/073         |   6 -
 tests/qemu-iotests/074         |   9 -
 tests/qemu-iotests/074.out     |   1 -
 tests/qemu-iotests/075         |   6 -
 tests/qemu-iotests/076         |   6 -
 tests/qemu-iotests/077         |   6 -
 tests/qemu-iotests/078         |   6 -
 tests/qemu-iotests/079         |   6 -
 tests/qemu-iotests/080         |   7 -
 tests/qemu-iotests/081         |   8 -
 tests/qemu-iotests/082         |   6 -
 tests/qemu-iotests/083         |   8 -
 tests/qemu-iotests/084         |   6 -
 tests/qemu-iotests/085         |  13 --
 tests/qemu-iotests/086         |   6 -
 tests/qemu-iotests/088         |   7 -
 tests/qemu-iotests/089         |   6 -
 tests/qemu-iotests/090         |   6 -
 tests/qemu-iotests/091         |   8 -
 tests/qemu-iotests/092         |   7 -
 tests/qemu-iotests/094         |   9 -
 tests/qemu-iotests/095         |   8 -
 tests/qemu-iotests/096         |   8 +-
 tests/qemu-iotests/097         |   7 -
 tests/qemu-iotests/098         |   7 -
 tests/qemu-iotests/099         |   6 -
 tests/qemu-iotests/101         |   6 -
 tests/qemu-iotests/102         |   7 -
 tests/qemu-iotests/103         |   6 -
 tests/qemu-iotests/104         |   2 -
 tests/qemu-iotests/105         |   6 -
 tests/qemu-iotests/106         |   6 -
 tests/qemu-iotests/107         |   6 -
 tests/qemu-iotests/108         |   6 -
 tests/qemu-iotests/109         |   8 -
 tests/qemu-iotests/110         |   6 -
 tests/qemu-iotests/111         |   6 -
 tests/qemu-iotests/112         |   6 -
 tests/qemu-iotests/113         |   6 -
 tests/qemu-iotests/114         |   6 -
 tests/qemu-iotests/115         |   6 -
 tests/qemu-iotests/116         |   6 -
 tests/qemu-iotests/117         |   7 -
 tests/qemu-iotests/118         | 200 ++++++++++----------
 tests/qemu-iotests/119         |   6 -
 tests/qemu-iotests/120         |   6 -
 tests/qemu-iotests/121         |   6 -
 tests/qemu-iotests/122         |   7 -
 tests/qemu-iotests/123         |   7 -
 tests/qemu-iotests/124         |  24 +--
 tests/qemu-iotests/125         |   6 -
 tests/qemu-iotests/129         |   6 +-
 tests/qemu-iotests/130         |   7 -
 tests/qemu-iotests/131         |   6 -
 tests/qemu-iotests/132         |  19 +-
 tests/qemu-iotests/133         |   6 -
 tests/qemu-iotests/134         |   6 -
 tests/qemu-iotests/135         |   6 -
 tests/qemu-iotests/136         |   7 +-
 tests/qemu-iotests/137         |   6 -
 tests/qemu-iotests/138         |   6 -
 tests/qemu-iotests/139         |  33 ++--
 tests/qemu-iotests/140         |   8 -
 tests/qemu-iotests/141         |   8 -
 tests/qemu-iotests/142         |   7 -
 tests/qemu-iotests/143         |   7 -
 tests/qemu-iotests/144         |   8 -
 tests/qemu-iotests/145         |   7 -
 tests/qemu-iotests/146         |   7 -
 tests/qemu-iotests/147         |  41 ++---
 tests/qemu-iotests/148         |  33 ++--
 tests/qemu-iotests/150         |   6 -
 tests/qemu-iotests/152         |  21 +--
 tests/qemu-iotests/153         |  12 --
 tests/qemu-iotests/154         |   6 -
 tests/qemu-iotests/155         |  54 +++---
 tests/qemu-iotests/156         |   7 -
 tests/qemu-iotests/157         |   6 -
 tests/qemu-iotests/158         |   6 -
 tests/qemu-iotests/159         |   7 -
 tests/qemu-iotests/160         |   7 -
 tests/qemu-iotests/162         |   7 -
 tests/qemu-iotests/163         |  42 ++---
 tests/qemu-iotests/165         |  11 +-
 tests/qemu-iotests/170         |   7 -
 tests/qemu-iotests/171         |   6 -
 tests/qemu-iotests/172         |   8 -
 tests/qemu-iotests/173         |   8 -
 tests/qemu-iotests/174         |   6 -
 tests/qemu-iotests/175         |   6 -
 tests/qemu-iotests/176         |   7 -
 tests/qemu-iotests/177         |   6 -
 tests/qemu-iotests/178         |   7 -
 tests/qemu-iotests/179         |   7 -
 tests/qemu-iotests/181         |   8 -
 tests/qemu-iotests/182         |   6 -
 tests/qemu-iotests/183         |   9 -
 tests/qemu-iotests/184         |   6 -
 tests/qemu-iotests/185         |   9 -
 tests/qemu-iotests/186         |   6 -
 tests/qemu-iotests/187         |   8 -
 tests/qemu-iotests/188         |   6 -
 tests/qemu-iotests/189         |   6 -
 tests/qemu-iotests/190         |   7 -
 tests/qemu-iotests/191         |  10 -
 tests/qemu-iotests/192         |   6 -
 tests/qemu-iotests/195         |   7 -
 tests/qemu-iotests/197         |   7 -
 tests/qemu-iotests/check       | 409 +++++++++++++++++++++++++++++++----------
 tests/qemu-iotests/common.qemu |  82 +++++++--
 tests/qemu-iotests/common.rc   |  98 ++++++----
 tests/qemu-iotests/iotests.py  |  15 +-
 182 files changed, 1044 insertions(+), 1869 deletions(-)
Signature · 1 line
2.9.5
author
#1 lore raw

PATCH v5 1/10 qemu-iotests: refuse to run if TEST_DIR contains spaces

Currently, not all qemu-iotests work if TEST_DIR has spaces, and they also might not be safe. Refuse to run if TEST_DIR in this case, at least until all tests are fixed sometime in the future.

---
 tests/qemu-iotests/check | 8 ++++++++
 1 file changed, 8 insertions(+)
tests/qemu-iotests/check +8 -0
index e6b6ff7..e2163cc 100755
@@ -102,6 +102,14 @@ if [ -z "$TEST_DIR" ]; then
         TEST_DIR=`pwd`/scratch
 fi
 
+case $TEST_DIR in
+    *[[:blank:]]*)
+        echo "The TEST_DIR pathname '$TEST_DIR' contains whitespace. "
+        echo "This is currently unsupported by qemu-iotests"
+        exit 1
+        ;;
+esac
+
 if [ ! -e "$TEST_DIR" ]; then
         mkdir "$TEST_DIR"
 fi
Signature · 1 line
2.9.5
#2 lore raw

Re: [Qemu-devel] [PATCH v5 01/10] qemu-iotests: refuse to run if TEST_DIR contains spaces

Quoted 14 lines · Jeff Cody

Currently, not all qemu-iotests work if TEST_DIR has spaces, and they also might not be safe. Refuse to run if TEST_DIR in this case, at least until all tests are fixed sometime in the future.

---
 tests/qemu-iotests/check | 8 ++++++++
 1 file changed, 8 insertions(+)
tests/qemu-iotests/check +0 -0
index e6b6ff7..e2163cc 100755

tests/qemu-iotests/check if [ -z "$TEST_DIR" ]; then · line 107

Quoted 6 lines
tests/qemu-iotests/check +3 -0
@@ -102,6 +102,14 @@ if [ -z "$TEST_DIR" ]; then
         TEST_DIR=`pwd`/scratch
 fi
 
+case $TEST_DIR in
+    *[[:blank:]]*)
+        echo "The TEST_DIR pathname '$TEST_DIR' contains whitespace. "

Echoes a trailing space. Do we care?

tests/qemu-iotests/check if [ -z "$TEST_DIR" ]; then · line 108

Quoted 1 line
tests/qemu-iotests/check +1 -0
+        echo "This is currently unsupported by qemu-iotests"

Is it just whitespace, or should we also be wary of other shell metacharacters (such as quotes or glob characters)? In test 197, I did an alternative check for anything that is not alphanumeric, -, or _ (and allowing / between names).

Quoted 8 lines
tests/qemu-iotests/check +4 -0
+        exit 1
+        ;;
+esac
+
 if [ ! -e "$TEST_DIR" ]; then
         mkdir "$TEST_DIR"
 fi
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#3 lore raw

Re: [Qemu-devel] [PATCH v5 01/10] qemu-iotests: refuse to run if TEST_DIR contains spaces

Quoted 18 lines · Eric Blake
Quoted 17 lines · Jeff Cody

Currently, not all qemu-iotests work if TEST_DIR has spaces, and they also might not be safe. Refuse to run if TEST_DIR in this case, at least until all tests are fixed sometime in the future.

---
 tests/qemu-iotests/check | 8 ++++++++
 1 file changed, 8 insertions(+)
tests/qemu-iotests/check +0 -0
index e6b6ff7..e2163cc 100755
@@ -102,6 +102,14 @@ if [ -z "$TEST_DIR" ]; then
         TEST_DIR=`pwd`/scratch
 fi
 
Quoted 6 lines

tests/qemu-iotests/check if [ -z "$TEST_DIR" ]; then · line 107

Quoted 3 lines
tests/qemu-iotests/check +3 -0
+case $TEST_DIR in
+    *[[:blank:]]*)
+        echo "The TEST_DIR pathname '$TEST_DIR' contains whitespace. "

Echoes a trailing space. Do we care?

Heh, spurious whitespcae, a bit ironic, I suppose...

Might as well take care of it with a v6, along with expanding the tests as you hint at below.

Quoted 6 lines

tests/qemu-iotests/check if [ -z "$TEST_DIR" ]; then · line 108

Quoted 1 line
tests/qemu-iotests/check +1 -0
+        echo "This is currently unsupported by qemu-iotests"

Is it just whitespace, or should we also be wary of other shell metacharacters (such as quotes or glob characters)? In test 197, I did an alternative check for anything that is not alphanumeric, -, or _ (and allowing / between names).

I know that '-' works, because for years my work directory has been called "qemu-kvm". I'm not sure about globbing characters like *, or quotes, but probably not a bad idea to exclude them as well (although, someone that uses '*' in their directory names is probably no stranger to data catastrophe!).

Quoted 15 lines
Quoted 9 lines
tests/qemu-iotests/check +4 -0
+        exit 1
+        ;;
+esac
+
 if [ ! -e "$TEST_DIR" ]; then
         mkdir "$TEST_DIR"
 fi
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#4 lore raw

PATCH v5 2/10 qemu-iotests: set TEST_DIR to a unique dir for each test

In reply to Jeff Cody

Right now, all qemu-iotests output data into the same scratch directory, and so each test needs to be responsible for cleaning up its own files.

Have each test use 'scratch/$seq' as its temp directory, so the check script can do simple cleanup of removing the whole temporary directory.

---
 tests/qemu-iotests/check | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)
tests/qemu-iotests/check +17 -4
index e2163cc..5ae34bf 100755
@@ -713,6 +713,7 @@ seq="check"
 
 for seq in $list
 do
+    TEST_DIR_SEQ=$TEST_DIR/$seq
     err=false
     printf %s "$seq"
     if [ -n "$TESTS_REMAINING_LOG" ] ; then
@@ -756,13 +757,23 @@ do
         fi
         export OUTPUT_DIR=$PWD
         if $debug; then
-            (cd "$source_iotests";
+            (
+            export TEST_DIR=$TEST_DIR_SEQ
+            cd "$source_iotests";
+            . ./common.config
+            . ./common.rc
             MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-                    $run_command -d 2>&1 | tee $tmp.out)
+                    $run_command -d 2>&1 | tee $tmp.out
+            )
         else
-            (cd "$source_iotests";
+            (
+            export TEST_DIR=$TEST_DIR_SEQ
+            cd "$source_iotests";
+            . ./common.config
+            . ./common.rc
             MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-                    $run_command >$tmp.out 2>&1)
+                    $run_command >$tmp.out 2>&1
+            )
         fi
         sts=$?
         $timestamp && _timestamp
@@ -826,6 +837,8 @@ do
             fi
         fi
 
+        rm -rf "$TEST_DIR_SEQ"
+
     fi
 
     # come here for each test, except when $showme is true
Signature · 1 line
2.9.5
author
#5 lore raw

PATCH v5 3/10 qemu-iotests: automatically clean up bash protocol servers

In reply to Jeff Cody

For bash tests, this allows 'check' to reap all launch protocol servers / processes, after a test has finished running.

---
 tests/qemu-iotests/check     | 13 +++++++
 tests/qemu-iotests/common.rc | 93 +++++++++++++++++++++++++++++---------------
 2 files changed, 75 insertions(+), 31 deletions(-)
tests/qemu-iotests/check +13 -0
index 5ae34bf..b4ab646 100755
@@ -757,6 +757,8 @@ do
         fi
         export OUTPUT_DIR=$PWD
         if $debug; then
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";
@@ -766,6 +768,8 @@ do
                     $run_command -d 2>&1 | tee $tmp.out
             )
         else
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";
@@ -837,6 +841,15 @@ do
             fi
         fi
 
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        . "$source_iotests/common.config"
+        . "$source_iotests/common.rc"
+
+        _cleanup_protocols
+        )
         rm -rf "$TEST_DIR_SEQ"
 
     fi
tests/qemu-iotests/common.rc +62 -31

121 lines not on this page · raw message

Signature · 1 line
2.9.5
Reviewed-by on v5 3/10
#6 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 2 lines · Jeff Cody

For bash tests, this allows 'check' to reap all launch protocol servers / processes, after a test has finished running.

Nice stuff!

Quoted 6 lines
---
 tests/qemu-iotests/check     | 13 +++++++
 tests/qemu-iotests/common.rc | 93 +++++++++++++++++++++++++++++---------------
 2 files changed, 75 insertions(+), 31 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
#7 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

In reply to Jeff Cody

tests/qemu-iotests/check do · line 764

Quoted 7 lines · Jeff Cody
             )
         else
tests/qemu-iotests/check +2 -0
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";

Where is the missing ")"?

tests/qemu-iotests/check do · line 851

Quoted 12 lines
tests/qemu-iotests/check +8 -0
@@ -837,6 +841,15 @@ do
             fi
         fi
 
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        . "$source_iotests/common.config"
+        . "$source_iotests/common.rc"
+
+        _cleanup_protocols

"check" wasn't including common.rc before this patch, and most of the content of that file doesn't apply to "check". So if we want to move cleanup code to "check" we should remove it from common.rc too.

In general, I think we should strive to have a better separation between tests and harness. This for example could let us write a different harness for the same tests and integrate the tests better into Avocado. Hence I see one advantage and one disadvantage in your series:

* by adding more functionality to "check", it shows that the current separation may fail with a more sophisticated harness (such as the one you are creating here)

* it adds a lot more knowledge of QEMU (especially protocols) in "check", but there is still some unbalance: tests create the images and the protocol servers, but the harness cleans it up. The visible result of this unbalance is for example how multi-process protocol tests can fail when multiple tests try to bind the same address.

I think it's the right direction, but it feels like it's not there yet... Sorry---I know this is not very constructive, but I hope it helps anyway.

Maybe we should actually rewrite "check" in Python. That would force us to think more about the design.

Paolo

author
#8 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 10 lines · Paolo Bonzini

tests/qemu-iotests/check do · line 764

Quoted 7 lines · Jeff Cody
             )
         else
tests/qemu-iotests/check +2 -0
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";

Where is the missing ")"?

It's part of the diff context, not a change itself... it's still there, just not shown in the patch.

Quoted 33 lines

tests/qemu-iotests/check do · line 851

Quoted 13 lines
tests/qemu-iotests/check +8 -0
@@ -837,6 +841,15 @@ do
             fi
         fi
 
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        . "$source_iotests/common.config"
+        . "$source_iotests/common.rc"
+
+        _cleanup_protocols

"check" wasn't including common.rc before this patch, and most of the content of that file doesn't apply to "check". So if we want to move cleanup code to "check" we should remove it from common.rc too.

In general, I think we should strive to have a better separation between tests and harness. This for example could let us write a different harness for the same tests and integrate the tests better into Avocado. Hence I see one advantage and one disadvantage in your series:

* by adding more functionality to "check", it shows that the current separation may fail with a more sophisticated harness (such as the one you are creating here)

* it adds a lot more knowledge of QEMU (especially protocols) in "check", but there is still some unbalance: tests create the images and the protocol servers, but the harness cleans it up. The visible result of this unbalance is for example how multi-process protocol tests can fail when multiple tests try to bind the same address.

Quoted 7 lines

I think it's the right direction, but it feels like it's not there yet... Sorry---I know this is not very constructive, but I hope it helps anyway.

Maybe we should actually rewrite "check" in Python. That would force us to think more about the design.

I think writing a more sophisticated harness in another language, such as Python, has merit. But to clarify, do you mean that as a 'nack' to this series, or as something to be done later, after this series? If this series improves the existing harness, I don't see why to exclude it because a new re-write could be superior (which I don't dispute).

#9 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 48 lines · Jeff Cody
Quoted 9 lines · Paolo Bonzini

tests/qemu-iotests/check do · line 764

Quoted 7 lines · Jeff Cody
             )
         else
tests/qemu-iotests/check +2 -0
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";

Where is the missing ")"?

It's part of the diff context, not a change itself... it's still there, just not shown in the patch.

Quoted 35 lines

tests/qemu-iotests/check do · line 851

Quoted 12 lines
tests/qemu-iotests/check +8 -0
@@ -837,6 +841,15 @@ do
             fi
         fi
 
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        . "$source_iotests/common.config"
+        . "$source_iotests/common.rc"
+
+        _cleanup_protocols

"check" wasn't including common.rc before this patch, and most of the content of that file doesn't apply to "check". So if we want to move cleanup code to "check" we should remove it from common.rc too.

In general, I think we should strive to have a better separation between tests and harness. This for example could let us write a different harness for the same tests and integrate the tests better into Avocado. Hence I see one advantage and one disadvantage in your series:

* by adding more functionality to "check", it shows that the current separation may fail with a more sophisticated harness (such as the one you are creating here)

* it adds a lot more knowledge of QEMU (especially protocols) in "check", but there is still some unbalance: tests create the images and the protocol servers, but the harness cleans it up. The visible result of this unbalance is for example how multi-process protocol tests can fail when multiple tests try to bind the same address.

I think it's the right direction, but it feels like it's not there yet... Sorry---I know this is not very constructive, but I hope it helps anyway.

Quoted 7 lines
Quoted 2 lines

Maybe we should actually rewrite "check" in Python. That would force us to think more about the design.

I think writing a more sophisticated harness in another language, such as Python, has merit. But to clarify, do you mean that as a 'nack' to this series, or as something to be done later, after this series? If this series improves the existing harness, I don't see why to exclude it because a new

Well, I have no idea (hence the "not very constructive" part). I'm only "nacking" the sourcing of common.rc in the check script.

The series improves the harness, but it also sets a very different separation between the tests and the harness (especially WRT the tests cleaning up after themselves). The level of separation would at least be clearer if check didn't include common.rc.

Paolo

author
#10 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 57 lines · Paolo Bonzini
Quoted 48 lines · Jeff Cody
Quoted 9 lines · Paolo Bonzini

tests/qemu-iotests/check do · line 764

Quoted 7 lines · Jeff Cody
             )
         else
tests/qemu-iotests/check +2 -0
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";

Where is the missing ")"?

It's part of the diff context, not a change itself... it's still there, just not shown in the patch.

Quoted 35 lines

tests/qemu-iotests/check do · line 851

Quoted 12 lines
tests/qemu-iotests/check +8 -0
@@ -837,6 +841,15 @@ do
             fi
         fi
 
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        . "$source_iotests/common.config"
+        . "$source_iotests/common.rc"
+
+        _cleanup_protocols

"check" wasn't including common.rc before this patch, and most of the content of that file doesn't apply to "check". So if we want to move cleanup code to "check" we should remove it from common.rc too.

In general, I think we should strive to have a better separation between tests and harness. This for example could let us write a different harness for the same tests and integrate the tests better into Avocado. Hence I see one advantage and one disadvantage in your series:

* by adding more functionality to "check", it shows that the current separation may fail with a more sophisticated harness (such as the one you are creating here)

* it adds a lot more knowledge of QEMU (especially protocols) in "check", but there is still some unbalance: tests create the images and the protocol servers, but the harness cleans it up. The visible result of this unbalance is for example how multi-process protocol tests can fail when multiple tests try to bind the same address.

I think it's the right direction, but it feels like it's not there yet... Sorry---I know this is not very constructive, but I hope it helps anyway.

Quoted 7 lines
Quoted 2 lines

Maybe we should actually rewrite "check" in Python. That would force us to think more about the design.

I think writing a more sophisticated harness in another language, such as Python, has merit. But to clarify, do you mean that as a 'nack' to this series, or as something to be done later, after this series? If this series improves the existing harness, I don't see why to exclude it because a new

Quoted 8 lines

Well, I have no idea (hence the "not very constructive" part). I'm only "nacking" the sourcing of common.rc in the check script.

The series improves the harness, but it also sets a very different separation between the tests and the harness (especially WRT the tests cleaning up after themselves). The level of separation would at least be clearer if check didn't include common.rc.

I can get rid of the common.rc includes prior to running the tests, but this series really requires including common.rc in the spot you mentioned, for automatically cleaning up protocol and QEMU processes.

That auto-cleanup is arguably a big improvement, as it has been relatively common to run across tests that leave processes running in the background.

I agree that it sets up different expectations, but that is at least partly intentional. I don't really want to have to rely on individually written tests to clean up properly. That is ~200 chances (and growing) for a mistake; instead, this series moves that responsibility into a single place to maintain.

#11 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 11 lines · Jeff Cody
Quoted 8 lines

Well, I have no idea (hence the "not very constructive" part). I'm only "nacking" the sourcing of common.rc in the check script.

The series improves the harness, but it also sets a very different separation between the tests and the harness (especially WRT the tests cleaning up after themselves). The level of separation would at least be clearer if check didn't include common.rc.

I can get rid of the common.rc includes prior to running the tests, but this series really requires including common.rc in the spot you mentioned, for automatically cleaning up protocol and QEMU processes.

Understood, but does it have to be common.rc? Can it be a different file? That at least would still make it clear what check is doing (for example it is not launching qemu, which is part of common.rc).

Quoted 8 lines

That auto-cleanup is arguably a big improvement, as it has been relatively common to run across tests that leave processes running in the background.

I agree that it sets up different expectations, but that is at least partly intentional. I don't really want to have to rely on individually written tests to clean up properly. That is ~200 chances (and growing) for a mistake; instead, this series moves that responsibility into a single place to maintain.

Understood, that's also why I'm all but nacking the entire series!

Paolo

author
#12 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · Jeff Cody
Quoted 8 lines

Well, I have no idea (hence the "not very constructive" part). I'm only "nacking" the sourcing of common.rc in the check script.

The series improves the harness, but it also sets a very different separation between the tests and the harness (especially WRT the tests cleaning up after themselves). The level of separation would at least be clearer if check didn't include common.rc.

Quoted 8 lines
Quoted 3 lines

I can get rid of the common.rc includes prior to running the tests, but this series really requires including common.rc in the spot you mentioned, for automatically cleaning up protocol and QEMU processes.

Understood, but does it have to be common.rc? Can it be a different file? That at least would still make it clear what check is doing (for example it is not launching qemu, which is part of common.rc).

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

Quoted 12 lines
Quoted 8 lines

That auto-cleanup is arguably a big improvement, as it has been relatively common to run across tests that leave processes running in the background.

I agree that it sets up different expectations, but that is at least partly intentional. I don't really want to have to rely on individually written tests to clean up properly. That is ~200 chances (and growing) for a mistake; instead, this series moves that responsibility into a single place to maintain.

Understood, that's also why I'm all but nacking the entire series!

Paolo

#13 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 11 lines · Jeff Cody

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

Quoted 5 lines

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

A great start. Which of these are actually needed by the tests (and hence by common.rc) and why?

Paolo

author
#14 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · Jeff Cody

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

Quoted 9 lines
Quoted 5 lines

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

A great start. Which of these are actually needed by the tests (and hence by common.rc) and why?

Some tests are written such that they do intermediate cleanups between multiple internal sub-tests for varying reasons, and so use those cleanup functions as part of their testing. The function _cleanup_test_img effectively calls all the other functions I listed, so they are really all required for the tests, if they choose to call _cleanup_test_img.

And for 'check' to tear everything down to a clean state, it also needs to use the cleanup functions for everything that is not just a file/directory.

-Jeff

#15 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 19 lines · Jeff Cody
Quoted 10 lines
Quoted 10 lines

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

Quoted 8 lines
Quoted 5 lines

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

A great start. Which of these are actually needed by the tests (and hence by common.rc) and why?

Quoted 8 lines

Some tests are written such that they do intermediate cleanups between multiple internal sub-tests for varying reasons, and so use those cleanup functions as part of their testing. The function _cleanup_test_img effectively calls all the other functions I listed, so they are really all required for the tests, if they choose to call _cleanup_test_img.

And for 'check' to tear everything down to a clean state, it also needs to use the cleanup functions for everything that is not just a file/directory.

Do these tests really need the "cleanup protocols" part, because the few that have more than one _cleanup_test_img (059, 066, 070, 084, 146, 171) are either file-only or non-raw, so they should be able to just rebuild the format on top of the same image.

Maybe that's where the separation lies---protocol vs. format, where cleaning up the "file" protocol need not do anything because it's done when removing the test directory. If that's the case, it'd be nice because it might also make it much easier to tackle the issue with parallel tests.

Paolo

author
#16 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 26 lines · Paolo Bonzini
Quoted 25 lines · Jeff Cody
Quoted 10 lines
Quoted 10 lines

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

Quoted 8 lines
Quoted 5 lines

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

A great start. Which of these are actually needed by the tests (and hence by common.rc) and why?

Some tests are written such that they do intermediate cleanups between multiple internal sub-tests for varying reasons, and so use those cleanup functions as part of their testing. The function _cleanup_test_img effectively calls all the other functions I listed, so they are really all required for the tests, if they choose to call _cleanup_test_img.

Quoted 8 lines
Quoted 2 lines

And for 'check' to tear everything down to a clean state, it also needs to use the cleanup functions for everything that is not just a file/directory.

Do these tests really need the "cleanup protocols" part, because the few that have more than one _cleanup_test_img (059, 066, 070, 084, 146, 171) are either file-only or non-raw, so they should be able to just rebuild the format on top of the same image.

Maybe not, but that could just be the nature of what bugs we are testing for at this time. These specific tests may not need to, but it is not inconceivable to have a test that involves bringing up and tearing down a protocol based server some arbitrary number times, to test that specific behavior.

Quoted 6 lines

Maybe that's where the separation lies---protocol vs. format, where cleaning up the "file" protocol need not do anything because it's done when removing the test directory. If that's the case, it'd be nice because it might also make it much easier to tackle the issue with parallel tests.

On final exit, yes, a test needs not remember to remove all of its mouse droppings. But as far as not needing to remove images in intermediate stages of a given test, I think that assumes too much. For instance, qemu-img _should_ be able to rebuild a format on top of the same image. But maybe a test wants to see if that specific functionality actually works as intended, and compares removing and creating an image to rebuilding on top of an image, etc.

#17 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 36 lines · Jeff Cody
Quoted 26 lines · Paolo Bonzini
Quoted 25 lines · Jeff Cody
Quoted 10 lines
Quoted 10 lines

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

Quoted 8 lines
Quoted 5 lines

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

A great start. Which of these are actually needed by the tests (and hence by common.rc) and why?

Some tests are written such that they do intermediate cleanups between multiple internal sub-tests for varying reasons, and so use those cleanup functions as part of their testing. The function _cleanup_test_img effectively calls all the other functions I listed, so they are really all required for the tests, if they choose to call _cleanup_test_img.

Quoted 8 lines
Quoted 2 lines

And for 'check' to tear everything down to a clean state, it also needs to use the cleanup functions for everything that is not just a file/directory.

Do these tests really need the "cleanup protocols" part, because the few that have more than one _cleanup_test_img (059, 066, 070, 084, 146, 171) are either file-only or non-raw, so they should be able to just rebuild the format on top of the same image.

Quoted 5 lines

Maybe not, but that could just be the nature of what bugs we are testing for at this time. These specific tests may not need to, but it is not inconceivable to have a test that involves bringing up and tearing down a protocol based server some arbitrary number times, to test that specific behavior.

Right, but such a test should probably be protocol-specific; it can just use "file" and invoke QEMU_NBD on its own for example, similar to what tests 058 and 162 already do.

For example, it's very different if you delete and recreate a raw image _and_ rerun qemu-nbd, or only do the latter.

Quoted 6 lines
Quoted 5 lines

Maybe that's where the separation lies---protocol vs. format, where cleaning up the "file" protocol need not do anything because it's done when removing the test directory. If that's the case, it'd be nice because it might also make it much easier to tackle the issue with parallel tests.

Quoted 7 lines

On final exit, yes, a test needs not remember to remove all of its mouse droppings. But as far as not needing to remove images in intermediate stages of a given test, I think that assumes too much. For instance, qemu-img _should_ be able to rebuild a format on top of the same image. But maybe a test wants to see if that specific functionality actually works as intended, and compares removing and creating an image to rebuilding on top of an image, etc.

Right, but let's draw a line, does such a test need to support multiple protocols? For example:

- 083 and 143 for example are very much NBD-specific; 083 uses "_supported_proto nbd", while 143 probably should be using either "file" or "nbd".

- only 058 and 162 are running qemu-nbd manually, and they actually start a _new_ NBD protocol server

In addition not many tests do not use _make_test_img, as seen with "git grep -LE '_make_test_img|bin/env python' [0-9][0-9][0-9]". They are:

- 064, 070 and 135 which use the sample image and thus _should_ be using file

- 049, 082, 111 which is creating images with "qemu-img create"; 049 and 111 might actually use nfs too. 150 is using "qemu-img convert", which is also creation more or less

- 083 is NBD-specific because it uses the fault injector

- 113 probably should be more generic than just NBD

- 128 is pretty unique in that it creates a Linux device mapper device (!)

- 143 probably should be using either "file" or "nbd".

- 162 is also a bit unique in that it tests null-co:// and disregards IMGFMT/IMGPROTO

So it does seem that handling the protocol in "check" is a good approximation.

And by the way, the only existing uses of "_supported_proto" are

_supported_proto file
_supported_proto file nfs
_supported_proto file sheepdog rbd nfs
_supported_proto generic
_supported_proto nbd
_supported_proto nfs

so another thing to do might be to change _supported_proto to a feature-based test:

- file/sheepdog/rbd/nfs are those that support resizing (aka "truncate")

- file/nfs are generally those that support "qemu-img create" (plus others that should be "generic" actually, including 090, 103, 107); 150 is "file" because it needs "map" in addition to "create".

- nfs is used in test 173 to test protocol-based images with relative filename backing strings; it probably can run on file too, anyway that's a third important discriminating feature

- nbd is used in a bunch of random tests that can be made more generic (094, 113, 119, 123). Only 083 is NBD-specific because it uses the NBD fault injector, and it actually doesn't use the protocol that is created by the caller.

Since you are doing a lot of whole-directory moves, "_supported" tags could become a separate configuration file, e.g.

----

# all generic, no need to include it
#[001]

[025]
fmt=raw qcow2 qed

# specify feature
proto=+resize

[143]
fmt=generic
proto=nbd
start_protocol=no

[150]
fmt=raw qcow2
proto=+create +map
----

The few tests using start_protocol=no would have to ensure parallel-safe operation themselves. This would also enable a more consistent handling across shell and Python tests.

So, this is why I was wondering whether patches 3/4 kinda paint ourselves in the corner.

So, looking at the patches:

- 1, 2, 7, 8, 9 are definitely good ideas, and should be done _before_ an eventual/hypothetical Python rewrite of "check".

- for 5, 6 I think we should be using shell job control instead in "check" ('set -m')

#! /bin/sh
set -m
# Start a job which leaves two processes behind. By starting it

  # in the background, we can get the leader process's pid in $!
  # That pid is also the process group id of the whole job.
  sh -c 'echo subshell pid is $$; sleep 10 | sleep 15 &' &
  pgrp=$!
  wait
  echo '$! is '$pgrp', killing all processes in that group:'
  pgrep -g $pgrp -a
  kill -TERM -$pgrp
  sleep 1
  echo Leftover processes have been killed:
  ps axo pid,ppid,pgrp,stat,tty,comm|grep sleep

(Python can do the same by setting preexec_fn=os.setpgrp when calling Subprocess.popen; then you can kill the entire group with os.killpg)

- 10 depends on everything before. SAD! ;)

It's a pity to lose the cleanup in patch 4, but I think it's better in the long term if we have a clear idea of the tests' tasks vs. the harness's.

Thanks,

Paolo

author
#18 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 64 lines · Paolo Bonzini
Quoted 36 lines · Jeff Cody
Quoted 26 lines · Paolo Bonzini
Quoted 25 lines · Jeff Cody
Quoted 10 lines
Quoted 10 lines

Here is what we need from common.rc for this series:

_rm_test_img
_cleanup_nbd
_cleanup_vxhs
_cleanup_rbd
_cleanup_sheepdog
_cleanup_protocols
_cleanup_test_img

Quoted 8 lines
Quoted 5 lines

They all have a common theme (cleanup), so I could move them all to a common.cleanup (naming suggestion?) file (which would need to be included by common.rc, as well).

Would this be a strong enough delineation to overcome your concerns?

A great start. Which of these are actually needed by the tests (and hence by common.rc) and why?

Some tests are written such that they do intermediate cleanups between multiple internal sub-tests for varying reasons, and so use those cleanup functions as part of their testing. The function _cleanup_test_img effectively calls all the other functions I listed, so they are really all required for the tests, if they choose to call _cleanup_test_img.

Quoted 8 lines
Quoted 2 lines

And for 'check' to tear everything down to a clean state, it also needs to use the cleanup functions for everything that is not just a file/directory.

Do these tests really need the "cleanup protocols" part, because the few that have more than one _cleanup_test_img (059, 066, 070, 084, 146, 171) are either file-only or non-raw, so they should be able to just rebuild the format on top of the same image.

Quoted 5 lines

Maybe not, but that could just be the nature of what bugs we are testing for at this time. These specific tests may not need to, but it is not inconceivable to have a test that involves bringing up and tearing down a protocol based server some arbitrary number times, to test that specific behavior.

Right, but such a test should probably be protocol-specific; it can just use "file" and invoke QEMU_NBD on its own for example, similar to what tests 058 and 162 already do.

For example, it's very different if you delete and recreate a raw image _and_ rerun qemu-nbd, or only do the latter.

Quoted 6 lines
Quoted 5 lines

Maybe that's where the separation lies---protocol vs. format, where cleaning up the "file" protocol need not do anything because it's done when removing the test directory. If that's the case, it'd be nice because it might also make it much easier to tackle the issue with parallel tests.

Quoted 7 lines

On final exit, yes, a test needs not remember to remove all of its mouse droppings. But as far as not needing to remove images in intermediate stages of a given test, I think that assumes too much. For instance, qemu-img _should_ be able to rebuild a format on top of the same image. But maybe a test wants to see if that specific functionality actually works as intended, and compares removing and creating an image to rebuilding on top of an image, etc.

Quoted 3 lines

Right, but let's draw a line, does such a test need to support multiple protocols? For example:

This is a good question. But, I'm not sure that this is a question this series is trying to answer; one goal of this series is to keep the existing APIs currently in use by tests unchanged.

Quoted 83 lines

- 083 and 143 for example are very much NBD-specific; 083 uses "_supported_proto nbd", while 143 probably should be using either "file" or "nbd".

- only 058 and 162 are running qemu-nbd manually, and they actually start a _new_ NBD protocol server

In addition not many tests do not use _make_test_img, as seen with "git grep -LE '_make_test_img|bin/env python' [0-9][0-9][0-9]". They are:

- 064, 070 and 135 which use the sample image and thus _should_ be using file

- 049, 082, 111 which is creating images with "qemu-img create"; 049 and 111 might actually use nfs too. 150 is using "qemu-img convert", which is also creation more or less

- 083 is NBD-specific because it uses the fault injector

- 113 probably should be more generic than just NBD

- 128 is pretty unique in that it creates a Linux device mapper device (!)

- 143 probably should be using either "file" or "nbd".

- 162 is also a bit unique in that it tests null-co:// and disregards IMGFMT/IMGPROTO

So it does seem that handling the protocol in "check" is a good approximation.

And by the way, the only existing uses of "_supported_proto" are

_supported_proto file
_supported_proto file nfs
_supported_proto file sheepdog rbd nfs
_supported_proto generic
_supported_proto nbd
_supported_proto nfs

so another thing to do might be to change _supported_proto to a feature-based test:

- file/sheepdog/rbd/nfs are those that support resizing (aka "truncate")

- file/nfs are generally those that support "qemu-img create" (plus others that should be "generic" actually, including 090, 103, 107); 150 is "file" because it needs "map" in addition to "create".

- nfs is used in test 173 to test protocol-based images with relative filename backing strings; it probably can run on file too, anyway that's a third important discriminating feature

- nbd is used in a bunch of random tests that can be made more generic (094, 113, 119, 123). Only 083 is NBD-specific because it uses the NBD fault injector, and it actually doesn't use the protocol that is created by the caller.

Since you are doing a lot of whole-directory moves, "_supported" tags could become a separate configuration file, e.g.

----

# all generic, no need to include it
#[001]

[025]
fmt=raw qcow2 qed

# specify feature
proto=+resize

[143]
fmt=generic
proto=nbd
start_protocol=no

[150]
fmt=raw qcow2
proto=+create +map
----

Quoted 7 lines

The few tests using start_protocol=no would have to ensure parallel-safe operation themselves. This would also enable a more consistent handling across shell and Python tests.

So, this is why I was wondering whether patches 3/4 kinda paint ourselves in the corner.

I think this conflates a bit how we'd like to restructure tests in a future harness rewrite, and what this series does. I'm advocating that this series keep the same test APIs intact, and remove the need for tests to perform exit cleanup.

(Patch 10 is just some icing, since patches 1-9 make it fairly easy to add)

If we look at what patches 3 & 4 do:

Patch 3:

- Code movement within common.rc, but doesn't change the API. Tests still just call _cleanup_test_img() as needed.

- It does break apart _cleanup_test_img(), thereby technically creating some new APIs available to future tests:
* _cleanup_nbd()
* _cleanup_vxhs()
* _cleanup_rbd()
* _cleanup_sheepdog()
* _cleanup_protocols()

Maybe these new APIs are a sticking point? If so, perhaps we can mark them (via comments) as internal-only?

- ./check does an extra protocol cleanup check after a test is run, via the new _cleanup_protocols().

As far as existing tests go, no changes yet.

Patch 4:

- Removes test exit cleanup from tests

Now this does change test behavior, as it relies on the harness for file and protocol cleanup at test exit.

This will indeed paint us in a corner if we want a new check.py to not perform the test exit cleanup, and leave test cleanup (either partially or fully) as the responsibility for the tests. [1]

Quoted 4 lines

So, looking at the patches:

- 1, 2, 7, 8, 9 are definitely good ideas, and should be done _before_ an eventual/hypothetical Python rewrite of "check".

Alas, 9 requires 4 (which in turn requires 3). Without 4, there is nothing to keep, as the tests try to remove it all.

Quoted 7 lines

- for 5, 6 I think we should be using shell job control instead in "check" ('set -m')

#! /bin/sh
set -m
# Start a job which leaves two processes behind. By starting it

Quoted 12 lines
  # in the background, we can get the leader process's pid in $!
  # That pid is also the process group id of the whole job.
  sh -c 'echo subshell pid is $$; sleep 10 | sleep 15 &' &
  pgrp=$!
  wait
  echo '$! is '$pgrp', killing all processes in that group:'
  pgrep -g $pgrp -a
  kill -TERM -$pgrp
  sleep 1
  echo Leftover processes have been killed:
  ps axo pid,ppid,pgrp,stat,tty,comm|grep sleep

Existing tests right now use _cleanup_qemu in their tests (outside of final cleanup): 095 109 117 130, etc. So we can do process control differently, but _cleanup_qemu still needs to exist and also clean up other files (such as fifos, close fds, etc..), and provide the same functionality (optional wait-for-completion, etc.), if we are keeping the usage by tests the same.

Quoted 3 lines

(Python can do the same by setting preexec_fn=os.setpgrp when calling Subprocess.popen; then you can kill the entire group with os.killpg)

Yeah - a new check rewrite in a language like python would make process control significantly easier.

Quoted 5 lines

- 10 depends on everything before. SAD! ;)

It's a pity to lose the cleanup in patch 4, but I think it's better in the long term if we have a clear idea of the tests' tasks vs. the harness's.

This is the crux of it all, I suppose.

[1] So on that point: do you think individual tests should be responsible for cleaning up files and processes at test exit? If that answer is a 'yes' to either files or processes, then 3, 4, 6 (and maybe 9) are incompatible with a future redesign with that assumption. FWIW, my thought is that the answer should be "the harness shall perform both file and process cleanup on test exit".

Jeff

#19 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

Quoted 10 lines · Jeff Cody
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Jeff Cody

On final exit, yes, a test needs not remember to remove all of its mouse droppings. But as far as not needing to remove images in intermediate stages of a given test, I think that assumes too much. For instance, qemu-img _should_ be able to rebuild a format on top of the same image. But maybe a test wants to see if that specific functionality actually works as intended, and compares removing and creating an image to rebuilding on top of an image, etc.

Quoted 6 lines
Quoted 3 lines

Right, but let's draw a line, does such a test need to support multiple protocols? For example:

This is a good question. But, I'm not sure that this is a question this series is trying to answer; one goal of this series is to keep the existing APIs currently in use by tests unchanged.

Right, but in order to do so it's also making the line between test and harness unclear, which is something I'd like to avoid (when I looked at it a couple months ago, the line was surprisingly clear apart from some confusion around searching for programs, and separating check vs. common.rc turned out to be very easy).

Quoted 5 lines
Quoted 2 lines

[snip] So, this is why I was wondering whether patches 3/4 kinda paint ourselves in the corner.

I think this conflates a bit how we'd like to restructure tests in a future harness rewrite, and what this series does.

This is true. But this sure is not exactly keeping the test APIs intact. The APIs are intact, but the usage isn't---for example, for patch 9 to work you need to _not_ use _cleanup_test_img in the tests.

Quoted 15 lines

If we look at what patches 3 & 4 do:

Patch 3:

- Code movement within common.rc, but doesn't change the API. Tests still just call _cleanup_test_img() as needed.

- It does break apart _cleanup_test_img(), thereby technically creating some new APIs available to future tests:
* _cleanup_nbd()
* _cleanup_vxhs()
* _cleanup_rbd()
* _cleanup_sheepdog()
* _cleanup_protocols()

Quoted 7 lines

Maybe these new APIs are a sticking point? If so, perhaps we can mark them (via comments) as internal-only?

- ./check does an extra protocol cleanup check after a test is run, via the new _cleanup_protocols().

As far as existing tests go, no changes yet.

Here I'd like to remove _cleanup_test_img as a test API even. Most invocations out of the "trap" are unnecessary. Some (for VMDK) can be changed to _rm_test_img or changed to create a file with a new name (to make patch 9 more effective).

With that change, we can apply patch 4 with no issue.

Quoted 10 lines

Patch 4:

- Removes test exit cleanup from tests

Now this does change test behavior, as it relies on the harness for file and protocol cleanup at test exit.

This will indeed paint us in a corner if we want a new check.py to not perform the test exit cleanup, and leave test cleanup (either partially or fully) as the responsibility for the tests. [1]

I think patch 9 is enough proof that check should perform the test exit cleanup.

But again, the thing I'm worried about is mixing code between check and tests.

Quoted 27 lines
Quoted 4 lines

So, looking at the patches:

- 1, 2, 7, 8, 9 are definitely good ideas, and should be done _before_ an eventual/hypothetical Python rewrite of "check".

Alas, 9 requires 4 (which in turn requires 3). Without 4, there is nothing to keep, as the tests try to remove it all.

Quoted 6 lines

- for 5, 6 I think we should be using shell job control instead in "check" ('set -m')

#! /bin/sh
set -m
# Start a job which leaves two processes behind. By starting it

Quoted 12 lines
  # in the background, we can get the leader process's pid in $!
  # That pid is also the process group id of the whole job.
  sh -c 'echo subshell pid is $$; sleep 10 | sleep 15 &' &
  pgrp=$!
  wait
  echo '$! is '$pgrp', killing all processes in that group:'
  pgrep -g $pgrp -a
  kill -TERM -$pgrp
  sleep 1
  echo Leftover processes have been killed:
  ps axo pid,ppid,pgrp,stat,tty,comm|grep sleep
Quoted 5 lines

Existing tests right now use _cleanup_qemu in their tests (outside of final cleanup): 095 109 117 130, etc. So we can do process control differently, but _cleanup_qemu still needs to exist and also clean up other files (such as fifos, close fds, etc..), and provide the same functionality (optional wait-for-completion, etc.), if we are keeping the usage by tests the same.

Yes, _cleanup_qemu can stay in the tests.

Quoted 6 lines

[1] So on that point: do you think individual tests should be responsible for cleaning up files and processes at test exit? If that answer is a 'yes' to either files or processes, then 3, 4, 6 (and maybe 9) are incompatible with a future redesign with that assumption. FWIW, my thought is that the answer should be "the harness shall perform both file and process cleanup on test exit".

I definitely agree on that, but I that the harness can be a little less
refined: kill the whole process group and rm -rf the whole test directory. Protocols may need a little bit of refinement, but everything else should use OS services and ignore the QEMUness of qemu-iotests (and especially the fact that tests are shell scripts).

Paolo

#20 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

In reply to Paolo Bonzini
Quoted 10 lines · Paolo Bonzini

tests/qemu-iotests/check do · line 764

Quoted 7 lines · Jeff Cody
             )
         else
tests/qemu-iotests/check +2 -0
+            # Do this in a sub-shell, so we are operating on the right
+            # TEST_DIR / QEMU_TEST_DIR
             (
             export TEST_DIR=$TEST_DIR_SEQ
             cd "$source_iotests";

Where is the missing ")"?

This hunk is just adding comments, not '(', so it is already well-paired.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
#21 lore raw

Re: [Qemu-devel] [PATCH v5 03/10] qemu-iotests: automatically clean up bash protocol servers

In reply to Paolo Bonzini
Quoted 2 lines · Paolo Bonzini

Maybe we should actually rewrite "check" in Python. That would force us to think more about the design.

I think that would be great.

Any shell script that's more than 5 lines long is a shell script that should not exist. It is a terrible language for doing anything remotely complex, and the iotest harness easily falls into category.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#22 lore raw

PATCH v5 4/10 qemu-iotests: remove file cleanup from bash tests

In reply to Jeff Cody

All files for a given test are now self-contained in a subdirectory, and therefore the "./check" script can do all file-related cleanup without any help.

This removes file cleanups from the bash tests. The only cleanup left is whatever is needed to kill any spawned processes; e.g. _cleanup_qemu.

---
 tests/qemu-iotests/001     |  6 ------
 tests/qemu-iotests/002     |  6 ------
 tests/qemu-iotests/003     |  6 ------
 tests/qemu-iotests/004     |  6 ------
 tests/qemu-iotests/005     |  6 ------
 tests/qemu-iotests/007     |  7 -------
 tests/qemu-iotests/008     |  6 ------
 tests/qemu-iotests/009     |  6 ------
 tests/qemu-iotests/010     |  6 ------
 tests/qemu-iotests/011     |  6 ------
 tests/qemu-iotests/012     |  6 ------
 tests/qemu-iotests/013     |  6 ------
 tests/qemu-iotests/014     |  6 ------
 tests/qemu-iotests/015     |  7 -------
 tests/qemu-iotests/017     |  6 ------
 tests/qemu-iotests/018     |  6 ------
 tests/qemu-iotests/019     |  8 --------
 tests/qemu-iotests/020     |  8 --------
 tests/qemu-iotests/021     |  6 ------
 tests/qemu-iotests/022     |  6 ------
 tests/qemu-iotests/023     |  6 ------
 tests/qemu-iotests/024     |  8 --------
 tests/qemu-iotests/025     |  6 ------
 tests/qemu-iotests/026     |  7 -------
 tests/qemu-iotests/027     |  6 ------
 tests/qemu-iotests/028     |  8 --------
 tests/qemu-iotests/029     |  7 -------
 tests/qemu-iotests/031     |  6 ------
 tests/qemu-iotests/032     |  6 ------
 tests/qemu-iotests/033     |  6 ------
 tests/qemu-iotests/034     |  6 ------
 tests/qemu-iotests/035     |  6 ------
 tests/qemu-iotests/036     |  6 ------
 tests/qemu-iotests/037     |  6 ------
 tests/qemu-iotests/038     |  6 ------
 tests/qemu-iotests/039     |  6 ------
 tests/qemu-iotests/042     |  6 ------
 tests/qemu-iotests/043     |  7 -------
 tests/qemu-iotests/046     |  6 ------
 tests/qemu-iotests/047     |  6 ------
 tests/qemu-iotests/048     |  8 --------
 tests/qemu-iotests/048.out |  1 -
 tests/qemu-iotests/049     |  6 ------
 tests/qemu-iotests/050     |  8 --------
 tests/qemu-iotests/051     |  6 ------
 tests/qemu-iotests/052     |  6 ------
 tests/qemu-iotests/053     |  7 -------
 tests/qemu-iotests/054     |  6 ------
 tests/qemu-iotests/058     | 47 ++++++++++++++--------------------------------
 tests/qemu-iotests/059     |  7 -------
 tests/qemu-iotests/060     |  6 ------
 tests/qemu-iotests/061     |  6 ------
 tests/qemu-iotests/062     |  6 ------
 tests/qemu-iotests/063     |  7 -------
 tests/qemu-iotests/064     |  6 ------
 tests/qemu-iotests/066     |  6 ------
 tests/qemu-iotests/068     |  6 ------
 tests/qemu-iotests/069     |  6 ------
 tests/qemu-iotests/070     |  6 ------
 tests/qemu-iotests/071     |  6 ------
 tests/qemu-iotests/072     |  6 ------
 tests/qemu-iotests/073     |  6 ------
 tests/qemu-iotests/074     |  9 ---------
 tests/qemu-iotests/074.out |  1 -
 tests/qemu-iotests/075     |  6 ------
 tests/qemu-iotests/076     |  6 ------
 tests/qemu-iotests/077     |  6 ------
 tests/qemu-iotests/078     |  6 ------
 tests/qemu-iotests/079     |  6 ------
 tests/qemu-iotests/080     |  7 -------
 tests/qemu-iotests/081     |  8 --------
 tests/qemu-iotests/082     |  6 ------
 tests/qemu-iotests/083     |  8 --------
 tests/qemu-iotests/084     |  6 ------
 tests/qemu-iotests/085     | 13 +------------
 tests/qemu-iotests/086     |  6 ------
 tests/qemu-iotests/088     |  7 -------
 tests/qemu-iotests/089     |  6 ------
 tests/qemu-iotests/090     |  6 ------
 tests/qemu-iotests/091     |  8 +-------
 tests/qemu-iotests/092     |  7 -------
 tests/qemu-iotests/094     |  9 +--------
 tests/qemu-iotests/095     |  8 +-------
 tests/qemu-iotests/097     |  7 -------
 tests/qemu-iotests/098     |  7 -------
 tests/qemu-iotests/099     |  6 ------
 tests/qemu-iotests/101     |  6 ------
 tests/qemu-iotests/102     |  7 +------
 tests/qemu-iotests/103     |  6 ------
 tests/qemu-iotests/104     |  2 --
 tests/qemu-iotests/105     |  6 ------
 tests/qemu-iotests/106     |  6 ------
 tests/qemu-iotests/107     |  6 ------
 tests/qemu-iotests/108     |  6 ------
 tests/qemu-iotests/109     |  8 +-------
 tests/qemu-iotests/110     |  6 ------
 tests/qemu-iotests/111     |  6 ------
 tests/qemu-iotests/112     |  6 ------
 tests/qemu-iotests/113     |  6 ------
 tests/qemu-iotests/114     |  6 ------
 tests/qemu-iotests/115     |  6 ------
 tests/qemu-iotests/116     |  6 ------
 tests/qemu-iotests/117     |  7 +------
 tests/qemu-iotests/119     |  6 ------
 tests/qemu-iotests/120     |  6 ------
 tests/qemu-iotests/121     |  6 ------
 tests/qemu-iotests/122     |  7 -------
 tests/qemu-iotests/123     |  7 -------
 tests/qemu-iotests/125     |  6 ------
 tests/qemu-iotests/130     |  7 +------
 tests/qemu-iotests/131     |  6 ------
 tests/qemu-iotests/133     |  6 ------
 tests/qemu-iotests/134     |  6 ------
 tests/qemu-iotests/135     |  6 ------
 tests/qemu-iotests/137     |  6 ------
 tests/qemu-iotests/138     |  6 ------
 tests/qemu-iotests/140     |  8 +-------
 tests/qemu-iotests/141     |  8 +-------
 tests/qemu-iotests/142     |  7 -------
 tests/qemu-iotests/143     |  7 +------
 tests/qemu-iotests/144     |  8 +-------
 tests/qemu-iotests/145     |  7 -------
 tests/qemu-iotests/146     |  7 +------
 tests/qemu-iotests/150     |  6 ------
 tests/qemu-iotests/153     | 12 ------------
 tests/qemu-iotests/154     |  6 ------
 tests/qemu-iotests/156     |  7 +------
 tests/qemu-iotests/157     |  6 ------
 tests/qemu-iotests/158     |  6 ------
 tests/qemu-iotests/159     |  7 -------
 tests/qemu-iotests/160     |  7 -------
 tests/qemu-iotests/162     |  7 -------
 tests/qemu-iotests/170     |  7 -------
 tests/qemu-iotests/171     |  6 ------
 tests/qemu-iotests/172     |  8 --------
 tests/qemu-iotests/173     |  8 +-------
 tests/qemu-iotests/174     |  6 ------
 tests/qemu-iotests/175     |  6 ------
 tests/qemu-iotests/176     |  7 -------
 tests/qemu-iotests/177     |  6 ------
 tests/qemu-iotests/178     |  7 -------
 tests/qemu-iotests/179     |  7 -------
 tests/qemu-iotests/181     |  8 +-------
 tests/qemu-iotests/182     |  6 ------
 tests/qemu-iotests/183     |  9 +--------
 tests/qemu-iotests/184     |  6 ------
 tests/qemu-iotests/185     |  9 +--------
 tests/qemu-iotests/186     |  6 ------
 tests/qemu-iotests/187     |  8 --------
 tests/qemu-iotests/188     |  6 ------
 tests/qemu-iotests/189     |  6 ------
 tests/qemu-iotests/190     |  7 -------
 tests/qemu-iotests/191     |  4 ----
 tests/qemu-iotests/192     |  6 ------
 tests/qemu-iotests/195     |  7 -------
 tests/qemu-iotests/197     |  7 -------
 156 files changed, 32 insertions(+), 1023 deletions(-)
tests/qemu-iotests/001 +0 -6
index ffd14e2..6f71879 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/002 +0 -6
index d4f8e91..4c9ff23 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/003 +0 -6
index 19889b9..2648d03 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/004 +0 -6
index 6f2aa3d..99743b4 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/005 +0 -6
index 4447377..23a77e0 100755
@@ -30,12 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/007 +0 -7
index fa543ee..c8068e5 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	true
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/008 +0 -6
index 8e89d74..7f7f8ae 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/009 +0 -6
index 16e4475..e03412a 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/010 +0 -6
index 151dac2..8608848 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/011 +0 -6
index f8d044e..0c0d15f 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/012 +0 -6
index 01a770d..aa9cfb4 100755
@@ -29,12 +29,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/013 +0 -6
index d013f87..0ff9c4d 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/014 +0 -6
index 2ea79e8..3dee9f7 100755
@@ -29,12 +29,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/015 +0 -7
index aaf9c3f..d29a778 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	true
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/017 +0 -6
index 4f9302d..994ea0e 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/018 +0 -6
index 1d39d35..11ad2b1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/019 +0 -8
index 24a789a..3728193 100755
@@ -29,14 +29,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-    rm -f "$TEST_IMG.base"
-    rm -f "$TEST_IMG.orig"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/020 +0 -8
index 7a11110..37cd5d1 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-    rm -f "$TEST_IMG.base"
-    rm -f "$TEST_IMG.orig"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/021 +0 -6
index 11e8ed7..5ab2acb 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/022 +0 -6
index 2452a9f..4870b17 100755
@@ -29,12 +29,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/023 +0 -6
index 497ae1e..5a2bf4e 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/024 +0 -8
index e0d77ce..3e638d9 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	rm -f "$TEST_DIR/t.$IMGFMT.base_old"
-	rm -f "$TEST_DIR/t.$IMGFMT.base_new"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/025 +0 -6
index f5e672e..93634c1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/026 +0 -7
index 7fadfba..dc4ff9f 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-    rm "$TEST_DIR/blkdebug.conf"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/027 +0 -6
index 08593da..c3f9146 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/028 +0 -8
index 97a8869..3176789 100755
@@ -30,14 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f "${TEST_IMG}.copy"
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/029 +0 -7
index 30bab24..981a864 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -f $TEST_IMG.snap
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/031 +0 -6
index 1e08abc..6805c2c 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/032 +0 -6
index 24bcb52..dafcfb8 100755
@@ -29,12 +29,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/033 +0 -6
index 2cdfd13..bf3c59c 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/034 +0 -6
index 1b28bda..830b0db 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/035 +0 -6
index efc38e4..a79a4fd 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/036 +0 -6
index ce638d6..841f347 100755
@@ -30,12 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/037 +0 -6
index c476b82..756fc2c 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/038 +0 -6
index d99a150..cf730ae 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/039 +0 -6
index 1f48339..dd483c4 100755
@@ -30,12 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/042 +0 -6
index a53e7cb..0efeec6 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/043 +0 -7
index 1c6c22d..48b71fe 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG".[123].base
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/046 +0 -6
index f2ebecf..362ed94 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/047 +0 -6
index 1b8f3d4..1850035 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/048 +0 -8
index 9ed04a0..c95bd2f 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 
 status=1        # failure is the default!
 
-_cleanup()
-{
-    echo "Cleanup"
-    _cleanup_test_img
-    rm "${TEST_IMG_FILE2}"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 _compare()
 {
     $QEMU_IMG compare $QEMU_IMG_EXTRA_ARGS "$@" "$TEST_IMG" "${TEST_IMG2}"
tests/qemu-iotests/048.out +0 -1
index 0bcf663..3318eed 100644
@@ -39,4 +39,3 @@ wrote 512/512 bytes at offset 512
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 Content mismatch at offset 512!
 1
-Cleanup
tests/qemu-iotests/049 +0 -6
index df35b6d..00733a6 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/050 +0 -8
index 03b4a5d..e3b6d8c 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.old"
-    rm -f "$TEST_IMG.new"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/051 +0 -6
index dba8816..24da3c5 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/052 +0 -6
index 842eace..fc23fad 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/053 +0 -7
index 2a04f5f..e8b9037 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	rm -f "$TEST_IMG.orig"
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/054 +0 -6
index bf47ef9..e2de997 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/058 +14 -33

73 lines not on this page · raw message

tests/qemu-iotests/059 +0 -7
index a1c34ee..6ccd250 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.qcow2"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/060 +0 -6
index 8e95c45..6532837 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/061 +0 -6
index f5678b1..1346dd9 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/062 +0 -6
index 051fb9f..3a1df0d 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/063 +0 -7
index e4f6ea9..4a9fc44 100755
@@ -28,13 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	rm -f "$TEST_IMG.orig" "$TEST_IMG.raw1" "$TEST_IMG.raw2"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/064 +0 -6
index 5792fbb..211c541 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/066 +0 -6
index 8638217..e811f48 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/068 +0 -6
index e7fca6a..d117ff3 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/069 +0 -6
index 96e55ef..03d5a3c 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/070 +0 -6
index 8d08d74..1220cc1 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/071 +0 -6
index 48b4955..1a8d6e7 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/072 +0 -6
index aa027c7..8aaf3fd 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/073 +0 -6
index 40f85b1..c19ad0d 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/074 +0 -9
index b17866b..d7888ab 100755
@@ -27,15 +27,6 @@ echo "QA output created by $seq"
 
 status=1        # failure is the default!
 
-_cleanup()
-{
-    echo "Cleanup"
-    _cleanup_test_img
-    rm "${TEST_IMG2}"
-    rm -f "$TEST_DIR/blkdebug.conf"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 _compare()
 {
     $QEMU_IMG compare "$@" "$TEST_IMG" "${TEST_IMG2}"
tests/qemu-iotests/074.out +0 -1
index 8fba5ae..fb93965 100644
@@ -15,4 +15,3 @@ qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_D
 qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
 Warning: Image size mismatch!
 4
-Cleanup
tests/qemu-iotests/075 +0 -6
index 770d51c..c2509d1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/076 +0 -6
index ef9e6a4..69bccad 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/077 +0 -6
index d2d2a2d..4a458e1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/078 +0 -6
index f333e9a..22d078e 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/079 +0 -6
index b2e3f74..fba6ec6 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/080 +0 -7
index 55044c7..be548a9 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -f $TEST_IMG.snap
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/081 +0 -8
index da3fb09..2dadc16 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -rf $TEST_DIR/1.raw
-    rm -rf $TEST_DIR/2.raw
-    rm -rf $TEST_DIR/3.raw
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/082 +0 -6
index d5c83d4..aa585db 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/083 +0 -8
index 0306f11..b2d0ce1 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	rm -f nbd.sock
-	rm -f nbd-fault-injector.out
-	rm -f nbd-fault-injector.conf
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/084 +0 -6
index 04f2aa9..b0d42a0 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/085 +1 -12
index 5c7668c..7b69f86 100755
@@ -37,18 +37,7 @@ snapshot_virt1="snapshot-v1.qcow2"
 
 SNAPSHOTS=10
 
-_cleanup()
-{
-    _cleanup_qemu
-    for i in $(seq 1 ${SNAPSHOTS})
-    do
-        rm -f "${TEST_DIR}/${i}-${snapshot_virt0}"
-        rm -f "${TEST_DIR}/${i}-${snapshot_virt1}"
-    done
-    rm -f "${TEST_IMG}" "${TEST_IMG}.1" "${TEST_IMG}.2" "${TEST_IMG}.base"
-
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/086 +0 -6
index cd4494a..4d5f7db 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/088 +0 -7
index b8076f2..72e7c68 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -f $TEST_IMG.snap
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/089 +0 -6
index 9bfe230..75151f1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/090 +0 -6
index 7380503..c3be13c 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/091 +1 -7
index 10ac4a8..c4df2fb 100755
@@ -31,13 +31,7 @@ status=1    # failure is the default!
 
 MIG_FIFO="${TEST_DIR}/migrate"
 
-_cleanup()
-{
-    rm -f "${MIG_FIFO}"
-    _cleanup_qemu
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/092 +0 -7
index 5bbdd07..3b8cf8f 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -f $TEST_IMG.snap
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/094 +1 -8
index 9aa01e3..35e882c 100755
@@ -27,14 +27,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    _cleanup_test_img
-    rm -f "$TEST_DIR/source.$IMGFMT"
-}
-
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/095 +1 -7
index 030adb2..75d60c4 100755
@@ -30,13 +30,7 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm  -f "${TEST_IMG}.base" "${TEST_IMG}.snp1"
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/097 +0 -7
index e22670c..81e6729 100755
@@ -28,13 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    _rm_test_img "$TEST_IMG.itmd"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/098 +0 -7
index b002e96..24d8510 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_DIR/blkdebug.conf"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/099 +0 -6
index caaf58e..d811b34 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/101 +0 -6
index ea53f8b..62faf99 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/102 +1 -6
index d7ad8d9..201c520 100755
@@ -27,12 +27,7 @@ echo "QA output created by $seq"
 here=$PWD
 status=1    # failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and qemu instance handling
 . ./common.rc
tests/qemu-iotests/103 +0 -6
index ecbd8eb..df86d4f 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=$PWD
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/104 +0 -2
index 726d467..f10d311 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-trap "exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/105 +0 -6
index 3db4ce3..b469b1b 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/106 +0 -6
index bfe71f4..e4db5a1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=$PWD
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment and filters
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/107 +0 -6
index d7222dc..d513120 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/108 +0 -6
index 2355d98..8371db4 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/109 +1 -7
index d70b574..d4fca99 100755
@@ -27,13 +27,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f $TEST_IMG.src
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/110 +0 -6
index 9de7369..93dd09f 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/111 +0 -6
index a1c152d..f590d82 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/112 +0 -6
index 28eb9aa..ee3679e 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/113 +0 -6
index 19b68b2..07437ea 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/114 +0 -6
index 5b7dc54..5bfd473 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/115 +0 -6
index 665c2ea..0c9aaad 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/116 +0 -6
index df0172f..e687601 100755
@@ -30,12 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/117 +1 -6
index 6c83461..579cecb 100755
@@ -27,12 +27,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/119 +0 -6
index 4f34fb4..8cdb404 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/120 +0 -6
index f40b97d..bb0e2dd 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/121 +0 -6
index 1307b4e..d75fce1 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/122 +0 -7
index 45b359c..3942604 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -f "$TEST_IMG".[123]
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/123 +0 -7
index b18e3fc..98fd3ce 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$SRC_IMG"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/125 +0 -6
index 9424313..bca5fb5 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=$PWD
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 get_image_size_on_host()
 {
     $QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" | grep "disk size" \
tests/qemu-iotests/130 +1 -6
index e7e43de..4aad4ea 100755
@@ -29,12 +29,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/131 +0 -6
index 94a9ae7..52588ef 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/133 +0 -6
index 9d35a6a..2d0aae9 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/134 +0 -6
index 9914415..147ea2c 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/135 +0 -6
index ce60831..75c160d 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/137 +0 -6
index eb91e51..6068a13 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/138 +0 -6
index 21650d8..9682560 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/140 +1 -7
index f89d0d6..c5e1a5b 100755
@@ -31,13 +31,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    _cleanup_test_img
-    rm -f "$TEST_DIR/nbd"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/141 +1 -7
index 2f9d7b9..cff2319 100755
@@ -27,13 +27,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    _cleanup_test_img
-    rm -f "$TEST_DIR"/{b,m,o}.$IMGFMT
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/142 +0 -7
index 1639c83..65775be 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f $TEST_IMG.snap
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/143 +1 -6
index 5ff1944..b4736aa 100755
@@ -27,12 +27,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f "$TEST_DIR/nbd"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/144 +1 -7
index 00de3c3..b47c561 100755
@@ -32,13 +32,7 @@ status=1	# failure is the default!
 TMP_SNAP1=${TEST_DIR}/tmp.qcow2
 TMP_SNAP2=${TEST_DIR}/tmp2.qcow2
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f "${TEST_IMG}" "${TMP_SNAP1}" "${TMP_SNAP2}"
-}
-
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/145 +0 -7
index c371b3c..930628e 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	true
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/146 +1 -6
index 043711b..99b1e81 100755
@@ -27,12 +27,7 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1    # failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/150 +0 -6
index ee8f637..120920e 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/153 +0 -12
index fa25eb2..42882eb 100755
@@ -28,18 +28,6 @@ here="$PWD"
 tmp=/tmp/$$
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "${TEST_IMG}.base"
-    rm -f "${TEST_IMG}.convert"
-    rm -f "${TEST_IMG}.a"
-    rm -f "${TEST_IMG}.b"
-    rm -f "${TEST_IMG}.c"
-    rm -f "${TEST_IMG}.lnk"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/154 +0 -6
index fde03b0..90e7e65 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/156 +1 -6
index e75dc4d..9dbc6d7 100755
@@ -35,12 +35,7 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f "$TEST_IMG"{,.target}{,.backing,.overlay}
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/157 +0 -6
index 2bf02be..bda53c8 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/158 +0 -6
index 24ac600..a07708e 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/159 +0 -7
index 9b0e1ec..7fcccde 100755
@@ -26,13 +26,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.out"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 . ./common.rc
 . ./common.filter
 . ./common.pattern
tests/qemu-iotests/160 +0 -7
index 5c910e5..48dd25d 100755
@@ -26,13 +26,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.out" "$TEST_IMG.out.dd"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 . ./common.rc
 . ./common.filter
 . ./common.pattern
tests/qemu-iotests/162 +0 -7
index 477a806..cad2bd7 100755
@@ -28,13 +28,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    rm -f "${TEST_DIR}/qemu-nbd.pid"
-    rm -f 42
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/170 +0 -7
index b79359f..5288a2d 100755
@@ -26,13 +26,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.out"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 . ./common.rc
 . ./common.filter
 . ./common.pattern
tests/qemu-iotests/171 +0 -6
index bcfaaf1..2798d2f 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/172 +0 -8
index 02c5f79..d9efd4b 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-    rm -f "$TEST_IMG.2"
-    rm -f "$TEST_IMG.3"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/173 +1 -7
index bdaa092..0c2fde4 100755
@@ -27,13 +27,7 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1    # failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm  -f "${QEMU_TEST_DIR}/image.base" "${QEMU_TEST_DIR}/image.snp1"
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/174 +0 -6
index 552879d..67cd785 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/175 +0 -6
index ca56e82..1407597 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/176 +0 -7
index 950b287..56cd490 100755
@@ -31,13 +31,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    _rm_test_img "$TEST_IMG.itmd"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/177 +0 -6
index f8ed8fb..ed2e8b2 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/178 +0 -7
index 6af52c6..fd583d1 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1    # failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.converted"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/179 +0 -7
index 115944a..f7cf129 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-    rm -f "$TEST_DIR/blkdebug.conf"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/181 +1 -7
index 0c91e8f..18fb133 100755
@@ -29,13 +29,7 @@ status=1	# failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-_cleanup()
-{
-    rm -f "${MIG_SOCKET}"
-	_cleanup_test_img
-    _cleanup_qemu
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/182 +0 -6
index 2e078ce..a842f9f 100755
@@ -28,12 +28,6 @@ here="$PWD"
 tmp=/tmp/$$
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/183 +1 -8
index 20268ff..2e09679 100755
@@ -29,14 +29,7 @@ status=1 # failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-_cleanup()
-{
-    rm -f "${MIG_SOCKET}"
-    rm -f "${TEST_IMG}.dest"
-    _cleanup_test_img
-    _cleanup_qemu
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/184 +0 -6
index 704f38f..2130e64 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/185 +1 -8
index f5b47e4..7269d5e 100755
@@ -29,14 +29,7 @@ status=1 # failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-_cleanup()
-{
-    rm -f "${TEST_IMG}.mid"
-    rm -f "${TEST_IMG}.copy"
-    _cleanup_test_img
-    _cleanup_qemu
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
 
 # get standard environment, filters and checks
 . ./common.rc
tests/qemu-iotests/186 +0 -6
index 44cc01e..27d6530 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/187 +0 -8
index 7bb7833..8b29df3 100755
@@ -27,14 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-    rm -f "$TEST_IMG.2"
-    rm -f "$TEST_IMG.3"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/188 +0 -6
index 83ed03e..48f855d 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/189 +0 -6
index e695475..c5d516d 100755
@@ -27,12 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/190 +0 -7
index 8f808fe..84d20c6 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1    # failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.converted"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/191 +0 -4
index ac2b88f..f97e580 100755
@@ -31,10 +31,6 @@ MIG_SOCKET="${TEST_DIR}/migrate"
 
 _cleanup()
 {
-    rm -f "${TEST_IMG}.mid"
-    rm -f "${TEST_IMG}.ovl2"
-    rm -f "${TEST_IMG}.ovl3"
-    _cleanup_test_img
     _cleanup_qemu
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15
tests/qemu-iotests/192 +0 -6
index 595f0d7..be24bfb 100755
@@ -28,12 +28,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/195 +0 -7
index e7a403d..b826152 100755
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1 # failure is the default!
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$TEST_IMG.mid"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/197 +0 -7
index 887eb4f..df09870 100755
@@ -41,13 +41,6 @@ case "$TEST_DIR" in
         _notrun "Suspicious TEST_DIR='$TEST_DIR', cowardly refusing to run" ;;
 esac
 
-_cleanup()
-{
-    _cleanup_test_img
-    rm -f "$BLKDBG_CONF"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # Test is supported for any backing file; but we force qcow2 for our wrapper.
 _supported_fmt generic
 _supported_proto generic
Signature · 1 line
2.9.5
Reviewed-by on v5 4/10
#23 lore raw

Re: [Qemu-devel] [PATCH v5 04/10] qemu-iotests: remove file cleanup from bash tests

Quoted 11 lines · Jeff Cody

All files for a given test are now self-contained in a subdirectory, and therefore the "./check" script can do all file-related cleanup without any help.

This removes file cleanups from the bash tests. The only cleanup left is whatever is needed to kill any spawned processes; e.g. _cleanup_qemu.

---

 tests/qemu-iotests/197     |  7 -------

Looks like you have rebased to the current state of the tree. There may be a couple more attempting to sneak in before things get merged, but I trust you'll stay on top of that.

Quoted 1 line
 156 files changed, 32 insertions(+), 1023 deletions(-)

Fun diffstat, but took me a while to scan through for any mistakes.

tests/qemu-iotests/007 echo "QA output created by $seq" · line 35

Quoted 11 lines

+++ b/tests/qemu-iotests/007

tests/qemu-iotests/007 +0 -6
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	true
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15

Interesting use of 'true' (why? Since the only call to _cleanup changes exit status to $status anyways, it does not matter what state $? was left in after _cleanup)! Not a problem to see it disappear.

tests/qemu-iotests/028 echo "QA output created by $seq" · line 39

Quoted 12 lines
tests/qemu-iotests/028 +1 -7
+++ b/tests/qemu-iotests/028
@@ -30,14 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f "${TEST_IMG}.copy"
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15

Is this removal of _cleanup_qemu correct, or does it belong in a different patch in the series? [1]

Quoted 7 lines
tests/qemu-iotests/058 +3 -0
+++ b/tests/qemu-iotests/058
@@ -29,23 +29,22 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
+# get standard environment, filters and checks
+. ./common.rc

tests/qemu-iotests/058 echo "QA output created by $seq" · line 43

Quoted 10 lines
tests/qemu-iotests/058 +10 -0
+. ./common.filter
+. ./common.pattern
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+_require_command QEMU_NBD
+# Internal snapshots are (currently) impossible with refcount_bits=1
+_unsupported_imgopts 'refcount_bits=1[^0-9]'
+

Code motion...

Quoted 6 lines
tests/qemu-iotests/058 +0 -2
 nbd_unix_socket=$TEST_DIR/test_qemu_nbd_socket
 nbd_snapshot_img="nbd:unix:$nbd_unix_socket"
 rm -f "${TEST_DIR}/qemu-nbd.pid"
 
-_cleanup_nbd()
-{

tests/qemu-iotests/058 echo "QA output created by $seq" · line 47

Quoted 10 lines
tests/qemu-iotests/058 +0 -10
-    local NBD_SNAPSHOT_PID
-    if [ -f "${TEST_DIR}/qemu-nbd.pid" ]; then
-        read NBD_SNAPSHOT_PID < "${TEST_DIR}/qemu-nbd.pid"
-        rm -f "${TEST_DIR}/qemu-nbd.pid"
-        if [ -n "$NBD_SNAPSHOT_PID" ]; then
-            kill "$NBD_SNAPSHOT_PID"
-        fi
-    fi
-    rm -f "$nbd_unix_socket"
-}

...and dropping redundant code that used to be overridden by the common includes but now would take priority...

Quoted 13 lines
tests/qemu-iotests/058 +1 -1
-
 _wait_for_nbd()
 {
     for ((i = 0; i < 300; i++))
@@ -64,6 +63,7 @@ converted_image=$TEST_IMG.converted
 _export_nbd_snapshot()
 {
     _cleanup_nbd
+    rm -f "$nbd_unix_socket"
     $QEMU_NBD -v -t -k "$nbd_unix_socket" "$TEST_IMG" -l $1 &
     _wait_for_nbd
 }

tests/qemu-iotests/058 _export_nbd_snapshot() · line 74

Quoted 4 lines
tests/qemu-iotests/058 +1 -0
@@ -71,30 +71,11 @@ _export_nbd_snapshot()
 _export_nbd_snapshot1()
 {
     _cleanup_nbd
+    rm -f "$nbd_unix_socket"

...tweaks to account for the change in common ordering,

Quoted 14 lines
tests/qemu-iotests/058 +0 -10
     $QEMU_NBD -v -t -k "$nbd_unix_socket" "$TEST_IMG" -l snapshot.name=$1 &
     _wait_for_nbd
 }
 
-_cleanup()
-{
-    _cleanup_nbd
-    _cleanup_test_img
-    rm -f "$converted_image"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
-# get standard environment, filters and checks
-. ./common.rc

tests/qemu-iotests/058 _export_nbd_snapshot() · line 97

Quoted 10 lines
tests/qemu-iotests/058 +0 -10
-. ./common.filter
-. ./common.pattern
-
-_supported_fmt qcow2
-_supported_proto file
-_supported_os Linux
-_require_command QEMU_NBD
-# Internal snapshots are (currently) impossible with refcount_bits=1
-_unsupported_imgopts 'refcount_bits=1[^0-9]'
-

...and back to code motion. This file is different enough from the usual patterns of cleanups in the rest of the series; should some of this rearrangement be split into a separate patch? But only if you have to respin; if we have nothing else preventing v5 from going in, I don't think we need the review churn.

Quoted 8 lines
tests/qemu-iotests/085 +1 -3
+++ b/tests/qemu-iotests/085
@@ -37,18 +37,7 @@ snapshot_virt1="snapshot-v1.qcow2"
 
 SNAPSHOTS=10
 
-_cleanup()
-{
-    _cleanup_qemu

tests/qemu-iotests/085 snapshot_virt1="snapshot-v1.qcow2" · line 40

Quoted 10 lines
tests/qemu-iotests/085 +1 -9
-    for i in $(seq 1 ${SNAPSHOTS})
-    do
-        rm -f "${TEST_DIR}/${i}-${snapshot_virt0}"
-        rm -f "${TEST_DIR}/${i}-${snapshot_virt1}"
-    done
-    rm -f "${TEST_IMG}" "${TEST_IMG}.1" "${TEST_IMG}.2" "${TEST_IMG}.base"
-
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15

[1] Evidence that the change to 28 may be broken.

tests/qemu-iotests/191 MIG_SOCKET="${TEST_DIR}/migrate" · line 36

Quoted 12 lines
tests/qemu-iotests/191 +1 -4
+++ b/tests/qemu-iotests/191
@@ -31,10 +31,6 @@ MIG_SOCKET="${TEST_DIR}/migrate"
 
 _cleanup()
 {
-    rm -f "${TEST_IMG}.mid"
-    rm -f "${TEST_IMG}.ovl2"
-    rm -f "${TEST_IMG}.ovl3"
-    _cleanup_test_img
     _cleanup_qemu
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15

Elsewhere, you removed the _cleanup function entirely, and changed the trap to directly call _cleanup_qemu. Worth doing here?

It looks like only 28 was broken, and that fix is an obvious pattern, as well as any other minor tweaks you want to make based on my comments. With that correction,

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#24 lore raw

Re: [Qemu-devel] [PATCH v5 04/10] qemu-iotests: remove file cleanup from bash tests

Quoted 42 lines · Eric Blake
Quoted 11 lines · Jeff Cody

All files for a given test are now self-contained in a subdirectory, and therefore the "./check" script can do all file-related cleanup without any help.

This removes file cleanups from the bash tests. The only cleanup left is whatever is needed to kill any spawned processes; e.g. _cleanup_qemu.

---

 tests/qemu-iotests/197     |  7 -------

Looks like you have rebased to the current state of the tree. There may be a couple more attempting to sneak in before things get merged, but I trust you'll stay on top of that.

Quoted 1 line
 156 files changed, 32 insertions(+), 1023 deletions(-)

Fun diffstat, but took me a while to scan through for any mistakes.

tests/qemu-iotests/007 echo "QA output created by $seq" · line 35

Quoted 11 lines

+++ b/tests/qemu-iotests/007

tests/qemu-iotests/007 +0 -6
@@ -27,13 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-_cleanup()
-{
-	_cleanup_test_img
-	true
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15

Interesting use of 'true' (why? Since the only call to _cleanup changes exit status to $status anyways, it does not matter what state $? was left in after _cleanup)! Not a problem to see it disappear.

Quoted 5 lines
tests/qemu-iotests/001 +1 -0
+++ b/tests/qemu-iotests/028
@@ -30,14 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
Quoted 11 lines

tests/qemu-iotests/028 echo "QA output created by $seq" · line 39

Quoted 7 lines
tests/qemu-iotests/028 +0 -7
-_cleanup()
-{
-    _cleanup_qemu
-    rm -f "${TEST_IMG}.copy"
-    _cleanup_test_img
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15

Is this removal of _cleanup_qemu correct, or does it belong in a different patch in the series? [1]

Nice catch. I think I had missed that instance before, and then squashed the change into the wrong patch. It should be in patch 6.

Quoted 82 lines
Quoted 7 lines
tests/qemu-iotests/058 +3 -0
+++ b/tests/qemu-iotests/058
@@ -29,23 +29,22 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
+# get standard environment, filters and checks
+. ./common.rc

tests/qemu-iotests/058 echo "QA output created by $seq" · line 43

Quoted 10 lines
tests/qemu-iotests/058 +10 -0
+. ./common.filter
+. ./common.pattern
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+_require_command QEMU_NBD
+# Internal snapshots are (currently) impossible with refcount_bits=1
+_unsupported_imgopts 'refcount_bits=1[^0-9]'
+

Code motion...

Quoted 6 lines
tests/qemu-iotests/058 +0 -2
 nbd_unix_socket=$TEST_DIR/test_qemu_nbd_socket
 nbd_snapshot_img="nbd:unix:$nbd_unix_socket"
 rm -f "${TEST_DIR}/qemu-nbd.pid"
 
-_cleanup_nbd()
-{

tests/qemu-iotests/058 echo "QA output created by $seq" · line 47

Quoted 10 lines
tests/qemu-iotests/058 +0 -10
-    local NBD_SNAPSHOT_PID
-    if [ -f "${TEST_DIR}/qemu-nbd.pid" ]; then
-        read NBD_SNAPSHOT_PID < "${TEST_DIR}/qemu-nbd.pid"
-        rm -f "${TEST_DIR}/qemu-nbd.pid"
-        if [ -n "$NBD_SNAPSHOT_PID" ]; then
-            kill "$NBD_SNAPSHOT_PID"
-        fi
-    fi
-    rm -f "$nbd_unix_socket"
-}

...and dropping redundant code that used to be overridden by the common includes but now would take priority...

Quoted 13 lines
tests/qemu-iotests/058 +1 -1
-
 _wait_for_nbd()
 {
     for ((i = 0; i < 300; i++))
@@ -64,6 +63,7 @@ converted_image=$TEST_IMG.converted
 _export_nbd_snapshot()
 {
     _cleanup_nbd
+    rm -f "$nbd_unix_socket"
     $QEMU_NBD -v -t -k "$nbd_unix_socket" "$TEST_IMG" -l $1 &
     _wait_for_nbd
 }

tests/qemu-iotests/058 _export_nbd_snapshot() · line 74

Quoted 4 lines
tests/qemu-iotests/058 +1 -0
@@ -71,30 +71,11 @@ _export_nbd_snapshot()
 _export_nbd_snapshot1()
 {
     _cleanup_nbd
+    rm -f "$nbd_unix_socket"

...tweaks to account for the change in common ordering,

Quoted 22 lines
tests/qemu-iotests/058 +0 -18
     $QEMU_NBD -v -t -k "$nbd_unix_socket" "$TEST_IMG" -l snapshot.name=$1 &
     _wait_for_nbd
 }
 
-_cleanup()
-{
-    _cleanup_nbd
-    _cleanup_test_img
-    rm -f "$converted_image"
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
-# get standard environment, filters and checks
-. ./common.rc
-. ./common.filter
-. ./common.pattern
-
-_supported_fmt qcow2
-_supported_proto file
-_supported_os Linux
-_require_command QEMU_NBD
-# Internal snapshots are (currently) impossible with refcount_bits=1
Quoted 9 lines

tests/qemu-iotests/058 _export_nbd_snapshot() · line 97

Quoted 2 lines
tests/qemu-iotests/058 +0 -2
-_unsupported_imgopts 'refcount_bits=1[^0-9]'
-

...and back to code motion. This file is different enough from the usual patterns of cleanups in the rest of the series; should some of this rearrangement be split into a separate patch? But only if you have to respin; if we have nothing else preventing v5 from going in, I don't think we need the review churn.

I'll do that. We'll need a v6, there have been a couple other things you've pointed out, and I've found a couple small bugs as well.

Quoted 24 lines
Quoted 8 lines
tests/qemu-iotests/085 +1 -3
+++ b/tests/qemu-iotests/085
@@ -37,18 +37,7 @@ snapshot_virt1="snapshot-v1.qcow2"
 
 SNAPSHOTS=10
 
-_cleanup()
-{
-    _cleanup_qemu

tests/qemu-iotests/085 snapshot_virt1="snapshot-v1.qcow2" · line 40

Quoted 10 lines
tests/qemu-iotests/085 +1 -9
-    for i in $(seq 1 ${SNAPSHOTS})
-    do
-        rm -f "${TEST_DIR}/${i}-${snapshot_virt0}"
-        rm -f "${TEST_DIR}/${i}-${snapshot_virt1}"
-    done
-    rm -f "${TEST_IMG}" "${TEST_IMG}.1" "${TEST_IMG}.2" "${TEST_IMG}.base"
-
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
+trap "_cleanup_qemu; exit \$status" 0 1 2 3 15

[1] Evidence that the change to 28 may be broken.

Quoted 3 lines
(quoted diff) +1 -0
+++ b/tests/qemu-iotests/191
@@ -31,10 +31,6 @@ MIG_SOCKET="${TEST_DIR}/migrate"
 
Quoted 13 lines

tests/qemu-iotests/191 MIG_SOCKET="${TEST_DIR}/migrate" · line 36

Quoted 9 lines
tests/qemu-iotests/191 +0 -4
 _cleanup()
 {
-    rm -f "${TEST_IMG}.mid"
-    rm -f "${TEST_IMG}.ovl2"
-    rm -f "${TEST_IMG}.ovl3"
-    _cleanup_test_img
     _cleanup_qemu
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15

Elsewhere, you removed the _cleanup function entirely, and changed the trap to directly call _cleanup_qemu. Worth doing here?

Yes, I think so.

Quoted 5 lines

It looks like only 28 was broken, and that fix is an obvious pattern, as well as any other minor tweaks you want to make based on my comments. With that correction,

Thanks!

author
#25 lore raw

PATCH v5 5/10 qemu-iotests: change qemu pid and fd tracking / cleanup

In reply to Jeff Cody

So that later patches can cleanup running qemu processes called from different subshells, track resources to cleanup in pid and fd list files.

In subsquent patches, ./check will kill all QEMU processes launched with the common.qemu framework, and the tests will not need to cleanup manually (unless they want to do so as part of the test, e.g. wait for a process to end such as migration).

---
 tests/qemu-iotests/common.qemu | 82 ++++++++++++++++++++++++++++++++----------
 tests/qemu-iotests/common.rc   |  3 +-
 2 files changed, 64 insertions(+), 21 deletions(-)
tests/qemu-iotests/common.qemu +63 -19
index 7b3052d..35a08a6 100644
@@ -33,6 +33,10 @@ QEMU_FIFO_OUT="${QEMU_TEST_DIR}/qmp-out-$$"
 QEMU_HANDLE=0
 export _QEMU_HANDLE=0
 
+QEMU_PID_LIST="${QEMU_TEST_DIR}"/qemu-pid.lst
+QEMU_OUT_FD_LIST="${QEMU_TEST_DIR}"/qemu-out-fd.lst
+QEMU_IN_FD_LIST="${QEMU_TEST_DIR}"/qemu-in-fd.lst
+QEMU_FIFO_LIST="${QEMU_TEST_DIR}"/qemu-fifo.lst
 # If bash version is >= 4.1, these will be overwritten and dynamic
 # file descriptor values assigned.
 _out_fd=3
@@ -193,6 +197,10 @@ function _launch_qemu()
     QEMU_OUT[${_QEMU_HANDLE}]=${_out_fd}
     QEMU_IN[${_QEMU_HANDLE}]=${_in_fd}
     QEMU_STATUS[${_QEMU_HANDLE}]=0
+    echo ${_out_fd} >> "$QEMU_OUT_FD_LIST"
+    echo ${_in_fd} >> "$QEMU_IN_FD_LIST"
+    echo ${fifo_in} >> "$QEMU_FIFO_LIST"
+    echo ${fifo_out} >> "$QEMU_FIFO_LIST"
 
     if [ "${qemu_comm_method}" == "qmp" ]
     then
@@ -209,35 +217,71 @@ function _launch_qemu()
 
 # Silenty kills the QEMU process
 #
+# This is able to kill and clean up after QEMU processes without the
+# need for any subshell-specific variables, so long as the qemu-pidlist
+# and qemu-out-fd.list and qemu-in-fd.list files are properly maintained.
+#
 # If $wait is set to anything other than the empty string, the process will not
 # be killed but only waited for, and any output will be forwarded to stdout. If
 # $wait is empty, the process will be killed and all output will be suppressed.
 function _cleanup_qemu()
 {
-    # QEMU_PID[], QEMU_IN[], QEMU_OUT[] all use same indices
-    for i in "${!QEMU_OUT[@]}"
+    local fifo_path=
+    local testdir_path=
+
+    if [ ! -e "${QEMU_PID_LIST}" ]; then
+        return
+    fi
+
+    # get line count, and therefore number of processes to kill
+    numproc=$(wc -l "${QEMU_PID_LIST}" | sed 's/\s.*//')
+
+    for i in $(seq 1 $numproc)
     do
         local QEMU_PID
-        if [ -f "${QEMU_TEST_DIR}/qemu-${i}.pid" ]; then
-            read QEMU_PID < "${QEMU_TEST_DIR}/qemu-${i}.pid"
-            rm -f "${QEMU_TEST_DIR}/qemu-${i}.pid"
-            if [ -z "${wait}" ] && [ -n "${QEMU_PID}" ]; then
-                kill -KILL ${QEMU_PID} 2>/dev/null
-            fi
-            if [ -n "${QEMU_PID}" ]; then
-                wait ${QEMU_PID} 2>/dev/null # silent kill
-            fi
+        local OUT_FD
+        local IN_FD
+        j=$(expr $i - 1)
+
+        QEMU_PID=$(tail -n+${i} "${QEMU_PID_LIST}" | head -n1)
+        OUT_FD=$(tail -n+${i} "${QEMU_OUT_FD_LIST}" | head -n1)
+        IN_FD=$(tail -n+${i} "${QEMU_IN_FD_LIST}" | head -n1)
+
+        if [ -z "${wait}" ] && [ -n "${QEMU_PID}" ]; then
+            kill -KILL ${QEMU_PID} 2>/dev/null
+        fi
+        if [ -n "${QEMU_PID}" ]; then
+            wait ${QEMU_PID} 2>/dev/null # silent kill
         fi
 
-        if [ -n "${wait}" ]; then
-            cat <&${QEMU_OUT[$i]} | _filter_testdir | _filter_qemu \
-                                  | _filter_qemu_io | _filter_qmp | _filter_hmp
+        if [ -n "${wait}" ] && [ -n "${OUT_FD}" ]; then
+            cat <&${OUT_FD} | _filter_testdir | _filter_qemu \
+                            | _filter_qemu_io | _filter_qmp | _filter_hmp
+        fi
+
+        if [ -n "${IN_FD}" ]; then
+            eval "exec ${IN_FD}<&-"   # close file descriptors
+        fi
+        if [ -n "${OUT_FD}" ]; then
+            eval "exec ${OUT_FD}<&-"
         fi
-        rm -f "${QEMU_FIFO_IN}_${i}" "${QEMU_FIFO_OUT}_${i}"
-        eval "exec ${QEMU_IN[$i]}<&-"   # close file descriptors
-        eval "exec ${QEMU_OUT[$i]}<&-"
 
-        unset QEMU_IN[$i]
-        unset QEMU_OUT[$i]
+        unset QEMU_IN[$j]
+        unset QEMU_OUT[$j]
     done
+
+    # The FIFOs do not correspond to process entry in the pidlist, so
+    # just clean them up afterwards
+    while read fifo_name
+    do
+        # make sure entries are inside the $TEST_DIR
+        fifo_path=$(dirname -z $(realpath "$fifo_name"))
+        testdir_path=$(realpath "$QEMU_TEST_DIR")
+        if [ "$fifo_path" == "$testdir_path" ]
+        then
+            rm -f "$fifo_name"
+        fi
+    done < "${QEMU_FIFO_LIST}"
+
+    rm -f "${QEMU_PID_LIST}" "${QEMU_OUT_FD_LIST}" "${QEMU_IN_FD_LIST}" "$QEMU_FIFO_LIST}"
 }
tests/qemu-iotests/common.rc +1 -2
index a345ffd..b26b02f 100644
@@ -40,7 +40,6 @@ poke_file()
     printf "$3" | dd "of=$1" bs=1 "seek=$2" conv=notrunc &>/dev/null
 }
 
-
 if ! . ./common.config
     then
     echo "$0: failed to source common.config"
@@ -51,7 +50,7 @@ _qemu_wrapper()
 {
     (
         if [ -n "${QEMU_NEED_PID}" ]; then
-            echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid"
+            echo $BASHPID >> "${QEMU_TEST_DIR}/qemu-pid.lst"
         fi
         exec "$QEMU_PROG" $QEMU_OPTIONS "$@"
     )
Signature · 1 line
2.9.5
author
#26 lore raw

Re: [Qemu-devel] [PATCH v5 05/10] qemu-iotests: change qemu pid and fd tracking / cleanup

Quoted 113 lines · Jeff Cody

So that later patches can cleanup running qemu processes called from different subshells, track resources to cleanup in pid and fd list files.

In subsquent patches, ./check will kill all QEMU processes launched with the common.qemu framework, and the tests will not need to cleanup manually (unless they want to do so as part of the test, e.g. wait for a process to end such as migration).

---
 tests/qemu-iotests/common.qemu | 82 ++++++++++++++++++++++++++++++++----------
 tests/qemu-iotests/common.rc   |  3 +-
 2 files changed, 64 insertions(+), 21 deletions(-)
tests/qemu-iotests/common.qemu +46 -19
index 7b3052d..35a08a6 100644
@@ -33,6 +33,10 @@ QEMU_FIFO_OUT="${QEMU_TEST_DIR}/qmp-out-$$"
 QEMU_HANDLE=0
 export _QEMU_HANDLE=0
 
+QEMU_PID_LIST="${QEMU_TEST_DIR}"/qemu-pid.lst
+QEMU_OUT_FD_LIST="${QEMU_TEST_DIR}"/qemu-out-fd.lst
+QEMU_IN_FD_LIST="${QEMU_TEST_DIR}"/qemu-in-fd.lst
+QEMU_FIFO_LIST="${QEMU_TEST_DIR}"/qemu-fifo.lst
 # If bash version is >= 4.1, these will be overwritten and dynamic
 # file descriptor values assigned.
 _out_fd=3
@@ -193,6 +197,10 @@ function _launch_qemu()
     QEMU_OUT[${_QEMU_HANDLE}]=${_out_fd}
     QEMU_IN[${_QEMU_HANDLE}]=${_in_fd}
     QEMU_STATUS[${_QEMU_HANDLE}]=0
+    echo ${_out_fd} >> "$QEMU_OUT_FD_LIST"
+    echo ${_in_fd} >> "$QEMU_IN_FD_LIST"
+    echo ${fifo_in} >> "$QEMU_FIFO_LIST"
+    echo ${fifo_out} >> "$QEMU_FIFO_LIST"
 
     if [ "${qemu_comm_method}" == "qmp" ]
     then
@@ -209,35 +217,71 @@ function _launch_qemu()
 
 # Silenty kills the QEMU process
 #
+# This is able to kill and clean up after QEMU processes without the
+# need for any subshell-specific variables, so long as the qemu-pidlist
+# and qemu-out-fd.list and qemu-in-fd.list files are properly maintained.
+#
 # If $wait is set to anything other than the empty string, the process will not
 # be killed but only waited for, and any output will be forwarded to stdout. If
 # $wait is empty, the process will be killed and all output will be suppressed.
 function _cleanup_qemu()
 {
-    # QEMU_PID[], QEMU_IN[], QEMU_OUT[] all use same indices
-    for i in "${!QEMU_OUT[@]}"
+    local fifo_path=
+    local testdir_path=
+
+    if [ ! -e "${QEMU_PID_LIST}" ]; then
+        return
+    fi
+
+    # get line count, and therefore number of processes to kill
+    numproc=$(wc -l "${QEMU_PID_LIST}" | sed 's/\s.*//')
+
+    for i in $(seq 1 $numproc)
     do
         local QEMU_PID
-        if [ -f "${QEMU_TEST_DIR}/qemu-${i}.pid" ]; then
-            read QEMU_PID < "${QEMU_TEST_DIR}/qemu-${i}.pid"
-            rm -f "${QEMU_TEST_DIR}/qemu-${i}.pid"
-            if [ -z "${wait}" ] && [ -n "${QEMU_PID}" ]; then
-                kill -KILL ${QEMU_PID} 2>/dev/null
-            fi
-            if [ -n "${QEMU_PID}" ]; then
-                wait ${QEMU_PID} 2>/dev/null # silent kill
-            fi
+        local OUT_FD
+        local IN_FD
+        j=$(expr $i - 1)
+
+        QEMU_PID=$(tail -n+${i} "${QEMU_PID_LIST}" | head -n1)
+        OUT_FD=$(tail -n+${i} "${QEMU_OUT_FD_LIST}" | head -n1)
+        IN_FD=$(tail -n+${i} "${QEMU_IN_FD_LIST}" | head -n1)
+
+        if [ -z "${wait}" ] && [ -n "${QEMU_PID}" ]; then
+            kill -KILL ${QEMU_PID} 2>/dev/null
+        fi
+        if [ -n "${QEMU_PID}" ]; then
+            wait ${QEMU_PID} 2>/dev/null # silent kill
         fi
 
-        if [ -n "${wait}" ]; then
-            cat <&${QEMU_OUT[$i]} | _filter_testdir | _filter_qemu \
-                                  | _filter_qemu_io | _filter_qmp | _filter_hmp
+        if [ -n "${wait}" ] && [ -n "${OUT_FD}" ]; then
+            cat <&${OUT_FD} | _filter_testdir | _filter_qemu \
+                            | _filter_qemu_io | _filter_qmp | _filter_hmp
+        fi
+
+        if [ -n "${IN_FD}" ]; then
+            eval "exec ${IN_FD}<&-"   # close file descriptors
+        fi
+        if [ -n "${OUT_FD}" ]; then
+            eval "exec ${OUT_FD}<&-"
         fi
-        rm -f "${QEMU_FIFO_IN}_${i}" "${QEMU_FIFO_OUT}_${i}"
-        eval "exec ${QEMU_IN[$i]}<&-"   # close file descriptors
-        eval "exec ${QEMU_OUT[$i]}<&-"
 
-        unset QEMU_IN[$i]
-        unset QEMU_OUT[$i]

tests/qemu-iotests/common.qemu function _launch_qemu() · line 278

Quoted 10 lines
tests/qemu-iotests/common.qemu +9 -0
+        unset QEMU_IN[$j]
+        unset QEMU_OUT[$j]
     done
+
+    # The FIFOs do not correspond to process entry in the pidlist, so
+    # just clean them up afterwards
+    while read fifo_name
+    do
+        # make sure entries are inside the $TEST_DIR
+        fifo_path=$(dirname -z $(realpath "$fifo_name"))

Pointing out another issue I noticed after testing on a different machine:
in Bash > 4.4, this generates a warning, which breaks diff stats. The fix is to drop the '-z':

        fifo_path=$(dirname $(realpath "$fifo_name"))
Quoted 33 lines
tests/qemu-iotests/common.qemu +8 -0
+        testdir_path=$(realpath "$QEMU_TEST_DIR")
+        if [ "$fifo_path" == "$testdir_path" ]
+        then
+            rm -f "$fifo_name"
+        fi
+    done < "${QEMU_FIFO_LIST}"
+
+    rm -f "${QEMU_PID_LIST}" "${QEMU_OUT_FD_LIST}" "${QEMU_IN_FD_LIST}" "$QEMU_FIFO_LIST}"
 }
tests/qemu-iotests/common.rc +1 -2
index a345ffd..b26b02f 100644
@@ -40,7 +40,6 @@ poke_file()
     printf "$3" | dd "of=$1" bs=1 "seek=$2" conv=notrunc &>/dev/null
 }
 
-
 if ! . ./common.config
     then
     echo "$0: failed to source common.config"
@@ -51,7 +50,7 @@ _qemu_wrapper()
 {
     (
         if [ -n "${QEMU_NEED_PID}" ]; then
-            echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid"
+            echo $BASHPID >> "${QEMU_TEST_DIR}/qemu-pid.lst"
         fi
         exec "$QEMU_PROG" $QEMU_OPTIONS "$@"
     )
Signature · 1 line
2.9.5
#27 lore raw

Re: [Qemu-devel] [PATCH v5 05/10] qemu-iotests: change qemu pid and fd tracking / cleanup

Quoted 6 lines · Jeff Cody
Quoted 5 lines · Jeff Cody

So that later patches can cleanup running qemu processes called from different subshells, track resources to cleanup in pid and fd list files.

In subsquent patches, ./check will kill all QEMU processes launched with

s/subsquent/subsequent/

Quoted 14 lines

tests/qemu-iotests/common.qemu function _launch_qemu() · line 278

Quoted 13 lines

the common.qemu framework, and the tests will not need to cleanup manually (unless they want to do so as part of the test, e.g. wait for a process to end such as migration).

---

tests/qemu-iotests/common.qemu +6 -0
+    # The FIFOs do not correspond to process entry in the pidlist, so
+    # just clean them up afterwards
+    while read fifo_name
+    do
+        # make sure entries are inside the $TEST_DIR
+        fifo_path=$(dirname -z $(realpath "$fifo_name"))
Quoted 4 lines

Pointing out another issue I noticed after testing on a different machine:
in Bash > 4.4, this generates a warning, which breaks diff stats. The fix is to drop the '-z':

        fifo_path=$(dirname $(realpath "$fifo_name"))

That, and 'dirname -z' is GNU-specific. It is handy for getting the name of a directory that ends in newline (yes, you are evil if you do 'mkdir $"abc\n"', because $(dirname "$dir") will strip that trailing newline and likely trip up a poorly-written script to operate on 'abc' instead). But as we already check for no whitespace in $TESTDIR, we don't have to rely on -z to work around evil directories with a name ending in newline.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
#28 lore raw

Re: [Qemu-devel] [PATCH v5 05/10] qemu-iotests: change qemu pid and fd tracking / cleanup

In reply to Jeff Cody
Quoted 9 lines · Jeff Cody

So that later patches can cleanup running qemu processes called from different subshells, track resources to cleanup in pid and fd list files.

In subsquent patches, ./check will kill all QEMU processes launched with the common.qemu framework, and the tests will not need to cleanup manually (unless they want to do so as part of the test, e.g. wait for a process to end such as migration).

Quoted 5 lines
---
 tests/qemu-iotests/common.qemu | 82 ++++++++++++++++++++++++++++++++----------
 tests/qemu-iotests/common.rc   |  3 +-
 2 files changed, 64 insertions(+), 21 deletions(-)

Modulo the typo in the commit message in response to your other tweaks,

tests/qemu-iotests/common.qemu function _launch_qemu() · line 237

Quoted 2 lines
tests/qemu-iotests/common.qemu +2 -0
+    # get line count, and therefore number of processes to kill
+    numproc=$(wc -l "${QEMU_PID_LIST}" | sed 's/\s.*//')

Instead of having to use sed, just do:

numproc=$(wc -l < "${QEMU_PID_LIST}")

wc is required to behave differently when parsing stdin than when parsing a named file, so you might as well take advantage of it.

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+    for i in $(seq 1 $numproc)

Are we sure seq is always available? Shave a subshell, and avoid the dependency, by using bash's:

for i in {1..$numproc}

tests/qemu-iotests/common.rc poke_file() · line 43

Quoted 7 lines

+++ b/tests/qemu-iotests/common.rc

tests/qemu-iotests/common.rc +0 -1
@@ -40,7 +40,6 @@ poke_file()
     printf "$3" | dd "of=$1" bs=1 "seek=$2" conv=notrunc &>/dev/null
 }
 
-
 if ! . ./common.config

Spurious whitespace change?

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#29 lore raw

PATCH v5 6/10 qemu-iotests: make ./check automatically reap QEMU processes

In reply to Jeff Cody

Check will now take care of cleaning up all QEMU processes started from bash tests using the common.qemu framework.

This also paves the way to added another check option to keep QEMU processes around, in the case of a failed test.

---
 tests/qemu-iotests/085   | 2 --
 tests/qemu-iotests/091   | 2 --
 tests/qemu-iotests/094   | 2 --
 tests/qemu-iotests/095   | 2 --
 tests/qemu-iotests/102   | 2 --
 tests/qemu-iotests/109   | 2 --
 tests/qemu-iotests/117   | 2 --
 tests/qemu-iotests/130   | 2 --
 tests/qemu-iotests/140   | 2 --
 tests/qemu-iotests/141   | 2 --
 tests/qemu-iotests/143   | 2 --
 tests/qemu-iotests/144   | 2 --
 tests/qemu-iotests/146   | 2 --
 tests/qemu-iotests/156   | 2 --
 tests/qemu-iotests/173   | 2 --
 tests/qemu-iotests/181   | 2 --
 tests/qemu-iotests/183   | 2 --
 tests/qemu-iotests/185   | 2 --
 tests/qemu-iotests/191   | 6 ------
 tests/qemu-iotests/check | 7 +++++--
 20 files changed, 5 insertions(+), 44 deletions(-)
tests/qemu-iotests/085 +0 -2
index 7b69f86..283f9a9 100755
@@ -37,8 +37,6 @@ snapshot_virt1="snapshot-v1.qcow2"
 
 SNAPSHOTS=10
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/091 +0 -2
index c4df2fb..cc4c50c 100755
@@ -31,8 +31,6 @@ status=1    # failure is the default!
 
 MIG_FIFO="${TEST_DIR}/migrate"
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/094 +0 -2
index 35e882c..1587550 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/095 +0 -2
index 75d60c4..2891014 100755
@@ -30,8 +30,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/102 +0 -2
index 201c520..2980638 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here=$PWD
 status=1    # failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and qemu instance handling
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/109 +0 -2
index d4fca99..2f6e456 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/117 +0 -2
index 579cecb..a427ee7 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/130 +0 -2
index 4aad4ea..3610738 100755
@@ -29,8 +29,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/140 +0 -2
index c5e1a5b..ec79402 100755
@@ -31,8 +31,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/141 +0 -2
index cff2319..39b75a4 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/143 +0 -2
index b4736aa..e107ae3 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/144 +0 -2
index b47c561..8b55d33 100755
@@ -32,8 +32,6 @@ status=1	# failure is the default!
 TMP_SNAP1=${TEST_DIR}/tmp.qcow2
 TMP_SNAP2=${TEST_DIR}/tmp2.qcow2
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/146 +0 -2
index 99b1e81..7dffe22 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1    # failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/156 +0 -2
index 9dbc6d7..26ebd73 100755
@@ -35,8 +35,6 @@ echo "QA output created by $seq"
 here="$PWD"
 status=1	# failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/173 +0 -2
index 0c2fde4..933cd96 100755
@@ -27,8 +27,6 @@ echo "QA output created by $seq"
 here=`pwd`
 status=1    # failure is the default!
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/181 +0 -2
index 18fb133..e6f7908 100755
@@ -29,8 +29,6 @@ status=1	# failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/183 +0 -2
index 2e09679..cb82d3c 100755
@@ -29,8 +29,6 @@ status=1 # failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/185 +0 -2
index 7269d5e..ef4b1cd 100755
@@ -29,8 +29,6 @@ status=1 # failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-trap "_cleanup_qemu; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/191 +0 -6
index f97e580..7fc1365 100755
@@ -29,12 +29,6 @@ status=1 # failure is the default!
 
 MIG_SOCKET="${TEST_DIR}/migrate"
 
-_cleanup()
-{
-    _cleanup_qemu
-}
-trap "_cleanup; exit \$status" 0 1 2 3 15
-
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
tests/qemu-iotests/check +5 -2
index b4ab646..057ea39 100755
@@ -845,10 +845,13 @@ do
         # TEST_DIR / QEMU_TEST_DIR
         (
         export TEST_DIR=$TEST_DIR_SEQ
-        . "$source_iotests/common.config"
-        . "$source_iotests/common.rc"
+        cd "$source_iotests";
+        . ./common.config
+        . ./common.rc
+        . ./common.qemu
 
         _cleanup_protocols
+        _cleanup_qemu
         )
         rm -rf "$TEST_DIR_SEQ"
 
Signature · 1 line
2.9.5
Reviewed-by on v5 6/10
#30 lore raw

Re: [Qemu-devel] [PATCH v5 06/10] qemu-iotests: make ./check automatically reap QEMU processes

Quoted 9 lines · Jeff Cody

Check will now take care of cleaning up all QEMU processes started from bash tests using the common.qemu framework.

This also paves the way to added another check option to keep QEMU processes around, in the case of a failed test.

---
 tests/qemu-iotests/085   | 2 --

and 28, after fixing things earlier in the series

Quoted 4 lines

+++ b/tests/qemu-iotests/check

tests/qemu-iotests/check +0 -0
@@ -845,10 +845,13 @@ do
         # TEST_DIR / QEMU_TEST_DIR
         (

tests/qemu-iotests/check do · line 854

Quoted 10 lines
tests/qemu-iotests/check +5 -2
         export TEST_DIR=$TEST_DIR_SEQ
-        . "$source_iotests/common.config"
-        . "$source_iotests/common.rc"
+        cd "$source_iotests";
+        . ./common.config
+        . ./common.rc
+        . ./common.qemu
 
         _cleanup_protocols
+        _cleanup_qemu
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#31 lore raw

PATCH v5 7/10 qemu-iotests: run python tests in own subdirectories

In reply to Jeff Cody

This adds the framework to iotests.py to run python iotests in a subdirectory structure, structured like so:

scratch/

├── TestNumber
│   ├── TestClassName
│   │   ├── test_method_name

Prior to this patch, tests were run in a test subdirectory from previous patches in the series, like this:

scratch/
├── TestNumber

However, given the nature of python's unittest framework, additional subdirectories are needed, if we want to insure that we can save intermediate files in case of test failures (as we will do in a subsequent patch) without running the risk of tainting other test methods from the test file.

In python tests using iiotests.QMPTestCase, any reference to 'iotests.test_dir' should be replaced by 'self.workdir'. This may also require changing the scope of path name variables.

---
 tests/qemu-iotests/iotests.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
tests/qemu-iotests/iotests.py +11 -0
index 6f05790..7ff400a 100644
@@ -262,6 +262,17 @@ index_re = re.compile(r'([^\[]+)\[([^\]]+)\]')
 class QMPTestCase(unittest.TestCase):
     '''Abstract base class for QMP test cases'''
 
+    def __init__(self, *args):
+        super(QMPTestCase, self).__init__(*args)
+        self.workdir = os.path.join(test_dir, self.__class__.__name__,
+                                    self._testMethodName)
+        try:
+            os.makedirs(self.workdir)
+        except OSError, error:
+            if error.errno != errno.EEXIST:
+                raise
+        os.chdir(self.workdir)
+
     def dictpath(self, d, path):
         '''Traverse a path in a nested dict'''
         for component in path.split('/'):
Signature · 1 line
2.9.5
author
#32 lore raw

PATCH v5 8/10 qemu-iotests: modify python tests to run from subdir

In reply to Jeff Cody

Also removes any intermediate file deletion; this is no longer needed, as 'check' will take care of that once the test is completed (unless the user requests to save files on failure, and the test fails)

---
 tests/qemu-iotests/030 |  82 ++++++------
 tests/qemu-iotests/040 | 128 +++++++++----------
 tests/qemu-iotests/041 | 333 +++++++++++++++++++++++++------------------------
 tests/qemu-iotests/044 |  11 +-
 tests/qemu-iotests/045 |  42 +++----
 tests/qemu-iotests/055 |  97 +++++++-------
 tests/qemu-iotests/056 |  39 +++---
 tests/qemu-iotests/057 |   4 +-
 tests/qemu-iotests/065 |  13 +-
 tests/qemu-iotests/096 |   8 +-
 tests/qemu-iotests/118 | 200 ++++++++++++++---------------
 tests/qemu-iotests/124 |  24 ++--
 tests/qemu-iotests/129 |   6 +-
 tests/qemu-iotests/132 |  19 ++-
 tests/qemu-iotests/136 |   7 +-
 tests/qemu-iotests/139 |  33 +++--
 tests/qemu-iotests/147 |  41 +++---
 tests/qemu-iotests/148 |  33 +++--
 tests/qemu-iotests/152 |  21 ++--
 tests/qemu-iotests/155 |  54 ++++----
 tests/qemu-iotests/163 |  42 +++----
 tests/qemu-iotests/165 |  11 +-
 22 files changed, 581 insertions(+), 667 deletions(-)
tests/qemu-iotests/030 +36 -46
index 1883894..6e52509 100755
@@ -23,27 +23,24 @@ import os
 import iotests
 from iotests import qemu_img, qemu_io
 
-backing_img = os.path.join(iotests.test_dir, 'backing.img')
-mid_img = os.path.join(iotests.test_dir, 'mid.img')
-test_img = os.path.join(iotests.test_dir, 'test.img')
 
 class TestSingleDrive(iotests.QMPTestCase):
     image_len = 1 * 1024 * 1024 # MB
 
     def setUp(self):
-        iotests.create_image(backing_img, TestSingleDrive.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img)
-        qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 512', backing_img)
-        qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 524288 512', mid_img)
-        self.vm = iotests.VM().add_drive("blkdebug::" + test_img, "backing.node-name=mid")
+        self.backing_img = os.path.join(self.workdir, 'backing.img')
+        self.mid_img = os.path.join(self.workdir, 'mid.img')
+        self.test_img = os.path.join(self.workdir, 'test.img')
+        iotests.create_image(self.backing_img, TestSingleDrive.image_len)
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.backing_img, self.mid_img)
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.mid_img, self.test_img)
+        qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 512', self.backing_img)
+        qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 524288 512', self.mid_img)
+        self.vm = iotests.VM().add_drive("blkdebug::" + self.test_img, "backing.node-name=mid")
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(mid_img)
-        os.remove(backing_img)
 
     def test_stream(self):
         self.assert_no_active_block_jobs()
@@ -56,15 +53,15 @@ class TestSingleDrive(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
         self.vm.shutdown()
 
-        self.assertEqual(qemu_io('-f', 'raw', '-c', 'map', backing_img),
-                         qemu_io('-f', iotests.imgfmt, '-c', 'map', test_img),
+        self.assertEqual(qemu_io('-f', 'raw', '-c', 'map', self.backing_img),
+                         qemu_io('-f', iotests.imgfmt, '-c', 'map', self.test_img),
                          'image file map does not match backing file after streaming')
 
     def test_stream_intermediate(self):
         self.assert_no_active_block_jobs()
 
-        self.assertNotEqual(qemu_io('-f', 'raw', '-rU', '-c', 'map', backing_img),
-                            qemu_io('-f', iotests.imgfmt, '-rU', '-c', 'map', mid_img),
+        self.assertNotEqual(qemu_io('-f', 'raw', '-rU', '-c', 'map', self.backing_img),
+                            qemu_io('-f', iotests.imgfmt, '-rU', '-c', 'map', self.mid_img),
                             'image file map matches backing file before streaming')
 
         result = self.vm.qmp('block-stream', device='mid', job_id='stream-mid')
@@ -75,8 +72,8 @@ class TestSingleDrive(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
         self.vm.shutdown()
 
-        self.assertEqual(qemu_io('-f', 'raw', '-c', 'map', backing_img),
-                         qemu_io('-f', iotests.imgfmt, '-c', 'map', mid_img),
+        self.assertEqual(qemu_io('-f', 'raw', '-c', 'map', self.backing_img),
+                         qemu_io('-f', iotests.imgfmt, '-c', 'map', self.mid_img),
                          'image file map does not match backing file after streaming')
 
     def test_stream_pause(self):
@@ -107,18 +104,18 @@ class TestSingleDrive(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
         self.vm.shutdown()
 
-        self.assertEqual(qemu_io('-f', 'raw', '-c', 'map', backing_img),
-                         qemu_io('-f', iotests.imgfmt, '-c', 'map', test_img),
+        self.assertEqual(qemu_io('-f', 'raw', '-c', 'map', self.backing_img),
+                         qemu_io('-f', iotests.imgfmt, '-c', 'map', self.test_img),
                          'image file map does not match backing file after streaming')
 
     def test_stream_no_op(self):
         self.assert_no_active_block_jobs()
 
         # The image map is empty before the operation
-        empty_map = qemu_io('-f', iotests.imgfmt, '-rU', '-c', 'map', test_img)
+        empty_map = qemu_io('-f', iotests.imgfmt, '-rU', '-c', 'map', self.test_img)
 
         # This is a no-op: no data should ever be copied from the base image
-        result = self.vm.qmp('block-stream', device='drive0', base=mid_img)
+        result = self.vm.qmp('block-stream', device='drive0', base=self.mid_img)
         self.assert_qmp(result, 'return', {})
 
         self.wait_until_completed()
@@ -126,13 +123,13 @@ class TestSingleDrive(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
         self.vm.shutdown()
 
-        self.assertEqual(qemu_io('-f', iotests.imgfmt, '-c', 'map', test_img),
+        self.assertEqual(qemu_io('-f', iotests.imgfmt, '-c', 'map', self.test_img),
                          empty_map, 'image file map changed after a no-op')
 
     def test_stream_partial(self):
         self.assert_no_active_block_jobs()
 
-        result = self.vm.qmp('block-stream', device='drive0', base=backing_img)
+        result = self.vm.qmp('block-stream', device='drive0', base=self.backing_img)
         self.assert_qmp(result, 'return', {})
 
         self.wait_until_completed()
@@ -140,8 +137,8 @@ class TestSingleDrive(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
         self.vm.shutdown()
 
-        self.assertEqual(qemu_io('-f', iotests.imgfmt, '-c', 'map', mid_img),
-                         qemu_io('-f', iotests.imgfmt, '-c', 'map', test_img),
+        self.assertEqual(qemu_io('-f', iotests.imgfmt, '-c', 'map', self.mid_img),
+                         qemu_io('-f', iotests.imgfmt, '-c', 'map', self.test_img),
                          'image file map does not match backing file after streaming')
 
     def test_device_not_found(self):
@@ -157,7 +154,6 @@ class TestParallelOps(iotests.QMPTestCase):
     num_ops = 4 # Number of parallel block-stream operations
     num_imgs = num_ops * 2 + 1
     image_len = num_ops * 1024 * 1024
-    imgs = []
 
     def setUp(self):
         opts = []
@@ -167,7 +163,7 @@ class TestParallelOps(iotests.QMPTestCase):
         for i in range(self.num_imgs):
             img_depth = self.num_imgs - i - 1
             opts.append("backing." * img_depth + "node-name=node%d" % i)
-            self.imgs.append(os.path.join(iotests.test_dir, 'img-%d.img' % i))
+            self.imgs.append(os.path.join(self.workdir, 'img-%d.img' % i))
 
         # Create all images
         iotests.create_image(self.imgs[0], self.image_len)
@@ -192,8 +188,6 @@ class TestParallelOps(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        for img in self.imgs:
-            os.remove(img)
 
     # Test that it's possible to run several block-stream operations
     # in parallel in the same snapshot chain
@@ -395,8 +389,8 @@ class TestQuorum(iotests.QMPTestCase):
 
         # Initialize file names and command-line options
         for i in range(self.num_children):
-            child_img = os.path.join(iotests.test_dir, 'img-%d.img' % i)
-            backing_img = os.path.join(iotests.test_dir, 'backing-%d.img' % i)
+            child_img = os.path.join(self.workdir, 'img-%d.img' % i)
+            backing_img = os.path.join(self.workdir, 'backing-%d.img' % i)
             self.children.append(child_img)
             self.backing.append(backing_img)
             qemu_img('create', '-f', iotests.imgfmt, backing_img, '1M')
@@ -414,10 +408,6 @@ class TestQuorum(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        for img in self.children:
-            os.remove(img)
-        for img in self.backing:
-            os.remove(img)
 
     def test_stream_quorum(self):
         if not iotests.supports_quorum():
@@ -446,6 +436,8 @@ class TestSmallerBackingFile(iotests.QMPTestCase):
     image_len = 2 * backing_len
 
     def setUp(self):
+        backing_img = os.path.join(self.workdir, 'backing.img')
+        test_img = os.path.join(self.workdir, 'test.img')
         iotests.create_image(backing_img, self.backing_len)
         qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img, str(self.image_len))
         self.vm = iotests.VM().add_drive(test_img)
@@ -495,6 +487,8 @@ new_state = "1"
 
 class TestEIO(TestErrors):
     def setUp(self):
+        backing_img = os.path.join(self.workdir, 'backing.img')
+        test_img = os.path.join(self.workdir, 'test.img')
         self.blkdebug_file = backing_img + ".blkdebug"
         iotests.create_image(backing_img, TestErrors.image_len)
         self.create_blkdebug_file(self.blkdebug_file, "read_aio", 5)
@@ -507,9 +501,6 @@ class TestEIO(TestErrors):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
-        os.remove(self.blkdebug_file)
 
     def test_report(self):
         self.assert_no_active_block_jobs()
@@ -637,6 +628,8 @@ class TestEIO(TestErrors):
 
 class TestENOSPC(TestErrors):
     def setUp(self):
+        backing_img = os.path.join(self.workdir, 'backing.img')
+        test_img = os.path.join(self.workdir, 'test.img')
         self.blkdebug_file = backing_img + ".blkdebug"
         iotests.create_image(backing_img, TestErrors.image_len)
         self.create_blkdebug_file(self.blkdebug_file, "read_aio", 28)
@@ -649,9 +642,6 @@ class TestENOSPC(TestErrors):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
-        os.remove(self.blkdebug_file)
 
     def test_enospc(self):
         self.assert_no_active_block_jobs()
@@ -695,6 +685,8 @@ class TestStreamStop(iotests.QMPTestCase):
     image_len = 8 * 1024 * 1024 * 1024 # GB
 
     def setUp(self):
+        backing_img = os.path.join(self.workdir, 'backing.img')
+        test_img = os.path.join(self.workdir, 'test.img')
         qemu_img('create', backing_img, str(TestStreamStop.image_len))
         qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 32M', backing_img)
         qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
@@ -704,8 +696,6 @@ class TestStreamStop(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
 
     def test_stream_stop(self):
         self.assert_no_active_block_jobs()
@@ -724,6 +714,8 @@ class TestSetSpeed(iotests.QMPTestCase):
     image_len = 80 * 1024 * 1024 # MB
 
     def setUp(self):
+        backing_img = os.path.join(self.workdir, 'backing.img')
+        test_img = os.path.join(self.workdir, 'test.img')
         qemu_img('create', backing_img, str(TestSetSpeed.image_len))
         qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 32M', backing_img)
         qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
@@ -733,8 +725,6 @@ class TestSetSpeed(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
 
     # This is a short performance test which is not run by default.
     # Invoke "IMGFMT=qed ./030 TestSetSpeed.perf_test_throughput"
tests/qemu-iotests/040 +58 -70

239 lines not on this page · raw message

tests/qemu-iotests/041 +168 -165

792 lines not on this page · raw message

tests/qemu-iotests/044 +5 -6
index 11ea0f4..c051250 100755
@@ -27,7 +27,6 @@ from iotests import qemu_img, qemu_img_verbose, qemu_io
 import struct
 import subprocess
 
-test_img = os.path.join(iotests.test_dir, 'test.img')
 
 class TestRefcountTableGrowth(iotests.QMPTestCase):
     '''Abstract base class for image mirroring test cases'''
@@ -99,18 +98,18 @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
 
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=512', test_img, '16G')
-        self.preallocate(test_img)
+        self.test_img = os.path.join(self.workdir, 'test.img')
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=512', self.test_img, '16G')
+        self.preallocate(self.test_img)
         pass
 
 
     def tearDown(self):
-        os.remove(test_img)
         pass
 
     def test_grow_refcount_table(self):
-        qemu_io('-c', 'write 3800M 1M', test_img)
-        qemu_img_verbose('check' , test_img)
+        qemu_io('-c', 'write 3800M 1M', self.test_img)
+        qemu_img_verbose('check' , self.test_img)
         pass
 
 if __name__ == '__main__':
tests/qemu-iotests/045 +18 -24

73 lines not on this page · raw message

tests/qemu-iotests/055 +44 -53

277 lines not on this page · raw message

tests/qemu-iotests/056 +17 -22
index 04f2c3c..47da53f 100755
@@ -25,50 +25,45 @@ import os
 import iotests
 from iotests import qemu_img, qemu_io, create_image
 
-backing_img = os.path.join(iotests.test_dir, 'backing.img')
-test_img = os.path.join(iotests.test_dir, 'test.img')
-target_img = os.path.join(iotests.test_dir, 'target.img')
 
 class TestSyncModesNoneAndTop(iotests.QMPTestCase):
     image_len = 64 * 1024 * 1024 # MB
 
     def setUp(self):
-        create_image(backing_img, TestSyncModesNoneAndTop.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
-        qemu_io('-c', 'write -P0x41 0 512', test_img)
-        qemu_io('-c', 'write -P0xd5 1M 32k', test_img)
-        qemu_io('-c', 'write -P0xdc 32M 124k', test_img)
-        qemu_io('-c', 'write -P0xdc 67043328 64k', test_img)
-        self.vm = iotests.VM().add_drive(test_img)
+        self.backing_img = os.path.join(self.workdir, 'backing.img')
+        self.test_img = os.path.join(self.workdir, 'test.img')
+        self.target_img = os.path.join(self.workdir, 'target.img')
+        create_image(self.backing_img, TestSyncModesNoneAndTop.image_len)
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.backing_img, self.test_img)
+        qemu_io('-c', 'write -P0x41 0 512', self.test_img)
+        qemu_io('-c', 'write -P0xd5 1M 32k', self.test_img)
+        qemu_io('-c', 'write -P0xdc 32M 124k', self.test_img)
+        qemu_io('-c', 'write -P0xdc 67043328 64k', self.test_img)
+        self.vm = iotests.VM().add_drive(self.test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
-        try:
-            os.remove(target_img)
-        except OSError:
-            pass
+        pass
 
     def test_complete_top(self):
         self.assert_no_active_block_jobs()
         result = self.vm.qmp('drive-backup', device='drive0', sync='top',
-                             format=iotests.imgfmt, target=target_img)
+                             format=iotests.imgfmt, target=self.target_img)
         self.assert_qmp(result, 'return', {})
 
         self.wait_until_completed(check_offset=False)
 
         self.assert_no_active_block_jobs()
         self.vm.shutdown()
-        self.assertTrue(iotests.compare_images(test_img, target_img),
+        self.assertTrue(iotests.compare_images(self.test_img, self.target_img),
                         'target image does not match source after backup')
 
     def test_cancel_sync_none(self):
         self.assert_no_active_block_jobs()
 
         result = self.vm.qmp('drive-backup', device='drive0',
-                             sync='none', target=target_img)
+                             sync='none', target=self.target_img)
         self.assert_qmp(result, 'return', {})
         time.sleep(1)
         self.vm.hmp_qemu_io('drive0', 'write -P0x5e 0 512')
@@ -80,21 +75,21 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase):
 
         self.vm.shutdown()
         time.sleep(1)
-        self.assertEqual(-1, qemu_io('-c', 'read -P0x41 0 512', target_img).find("verification failed"))
+        self.assertEqual(-1, qemu_io('-c', 'read -P0x41 0 512', self.target_img).find("verification failed"))
 
 class TestBeforeWriteNotifier(iotests.QMPTestCase):
     def setUp(self):
+        self.target_img = os.path.join(self.workdir, 'target.img')
         self.vm = iotests.VM().add_drive_raw("file=blkdebug::null-co://,id=drive0,align=65536,driver=blkdebug")
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(target_img)
 
     def test_before_write_notifier(self):
         self.vm.pause_drive("drive0")
         result = self.vm.qmp('drive-backup', device='drive0',
-                             sync='full', target=target_img,
+                             sync='full', target=self.target_img,
                              format="file", speed=1)
         self.assert_qmp(result, 'return', {})
         result = self.vm.qmp('block-job-pause', device="drive0")
tests/qemu-iotests/057 +1 -3
index 9f0a5a3..00b07d3 100755
@@ -38,7 +38,7 @@ class ImageSnapshotTestCase(iotests.QMPTestCase):
         self.vm = iotests.VM()
         for i in range(0, image_num):
             filename = '%s%d' % (test_img_base_name, i)
-            img = os.path.join(iotests.test_dir, filename)
+            img = os.path.join(self.workdir, filename)
             device = '%s%d' % (test_drv_base_name, i)
             qemu_img('create', '-f', iotests.imgfmt, img, str(self.image_len))
             self.vm.add_drive(img)
@@ -49,8 +49,6 @@ class ImageSnapshotTestCase(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        for dev_expect in self.expect:
-            os.remove(dev_expect['image'])
 
     def createSnapshotInTransaction(self, snapshot_num, abort = False):
         actions = []
tests/qemu-iotests/065 +6 -7
index 72aa970..6c67f7d 100755
@@ -26,19 +26,18 @@ import iotests
 from iotests import qemu_img, qemu_img_pipe
 import unittest
 
-test_img = os.path.join(iotests.test_dir, 'test.img')
 
 class TestImageInfoSpecific(iotests.QMPTestCase):
     '''Abstract base class for ImageInfoSpecific tests'''
+    test_img = ''
 
     def setUp(self):
+        self.test_img = os.path.join(self.workdir, 'test.img')
         if self.img_options is None:
             self.skipTest('Skipping abstract test class')
         qemu_img('create', '-f', iotests.imgfmt, '-o', self.img_options,
-                 test_img, '128K')
+                 self.test_img, '128K')
 
-    def tearDown(self):
-        os.remove(test_img)
 
 class TestQemuImgInfo(TestImageInfoSpecific):
     '''Abstract base class for qemu-img info tests'''
@@ -48,13 +47,13 @@ class TestQemuImgInfo(TestImageInfoSpecific):
     human_compare = None
 
     def test_json(self):
-        data = json.loads(qemu_img_pipe('info', '--output=json', test_img))
+        data = json.loads(qemu_img_pipe('info', '--output=json', self.test_img))
         data = data['format-specific']
         self.assertEqual(data['type'], iotests.imgfmt)
         self.assertEqual(data['data'], self.json_compare)
 
     def test_human(self):
-        data = qemu_img_pipe('info', '--output=human', test_img).split('\n')
+        data = qemu_img_pipe('info', '--output=human', self.test_img).split('\n')
         data = data[(data.index('Format specific information:') + 1)
                     :data.index('')]
         for field in data:
@@ -71,7 +70,7 @@ class TestQMP(TestImageInfoSpecific):
 
     def setUp(self):
         self.TestImageInfoSpecific.setUp(self)
-        self.vm = iotests.VM().add_drive(test_img, self.qemu_options)
+        self.vm = iotests.VM().add_drive(self.test_img, self.qemu_options)
         self.vm.launch()
 
     def tearDown(self):
tests/qemu-iotests/096 +4 -4
index aeeb375..b9388ec 100644
@@ -23,13 +23,15 @@ import iotests
 import os
 
 class TestLiveSnapshot(iotests.QMPTestCase):
-    base_img = os.path.join(iotests.test_dir, 'base.img')
-    target_img = os.path.join(iotests.test_dir, 'target.img')
+    base_img = ''
+    target_img = ''
     group = 'mygroup'
     iops = 6000
     iops_size = 1024
 
     def setUp(self):
+        self.base_img = os.path.join(self.workdir, 'base.img')
+        self.target_img = os.path.join(self.workdir, 'target.img')
         opts = []
         opts.append('node-name=base')
         opts.append('throttling.group=%s' % self.group)
@@ -41,8 +43,6 @@ class TestLiveSnapshot(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(self.base_img)
-        os.remove(self.target_img)
 
     def checkConfig(self, active_layer):
         result = self.vm.qmp('query-block')
tests/qemu-iotests/118 +102 -98

542 lines not on this page · raw message

tests/qemu-iotests/124 +7 -17
index 8e76e62..91ca06d 100644
@@ -54,11 +54,12 @@ def transaction_drive_backup(device, target, **kwargs):
 
 
 class Bitmap:
-    def __init__(self, name, drive):
+    def __init__(self, name, drive, workdir=iotests.test_dir):
         self.name = name
         self.drive = drive
         self.num = 0
         self.backups = list()
+        self.workdir = workdir
 
     def base_target(self):
         return (self.drive['backup'], None)
@@ -67,7 +68,7 @@ class Bitmap:
         if num is None:
             num = self.num
         self.num = num + 1
-        base = os.path.join(iotests.test_dir,
+        base = os.path.join(self.workdir,
                             "%s.%s." % (self.drive['id'], self.name))
         suff = "%i.%s" % (num, self.drive['fmt'])
         target = base + "inc" + suff
@@ -85,12 +86,6 @@ class Bitmap:
             try_remove(image)
         self.num -= 1
 
-    def cleanup(self):
-        for backup in self.backups:
-            for image in backup:
-                try_remove(image)
-
-
 class TestIncrementalBackupBase(iotests.QMPTestCase):
     def __init__(self, *args):
         super(TestIncrementalBackupBase, self).__init__(*args)
@@ -98,7 +93,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase):
         self.files = list()
         self.drives = list()
         self.vm = iotests.VM()
-        self.err_img = os.path.join(iotests.test_dir, 'err.%s' % iotests.imgfmt)
+        self.err_img = os.path.join(self.workdir, 'err.%s' % iotests.imgfmt)
 
 
     def setUp(self):
@@ -118,9 +113,9 @@ class TestIncrementalBackupBase(iotests.QMPTestCase):
 
     def add_node(self, node_id, fmt=iotests.imgfmt, path=None, backup=None):
         if path is None:
-            path = os.path.join(iotests.test_dir, '%s.%s' % (node_id, fmt))
+            path = os.path.join(self.workdir, '%s.%s' % (node_id, fmt))
         if backup is None:
-            backup = os.path.join(iotests.test_dir,
+            backup = os.path.join(self.workdir,
                                   '%s.full.backup.%s' % (node_id, fmt))
 
         self.drives.append({
@@ -196,7 +191,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase):
 
 
     def add_bitmap(self, name, drive, **kwargs):
-        bitmap = Bitmap(name, drive)
+        bitmap = Bitmap(name, drive, self.workdir)
         self.bitmaps.append(bitmap)
         result = self.vm.qmp('block-dirty-bitmap-add', node=drive['id'],
                              name=bitmap.name, **kwargs)
@@ -273,11 +268,6 @@ class TestIncrementalBackupBase(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        for bitmap in self.bitmaps:
-            bitmap.cleanup()
-        for filename in self.files:
-            try_remove(filename)
-
 
 
 class TestIncrementalBackup(TestIncrementalBackupBase):
tests/qemu-iotests/129 +3 -3
index 9e87e1c..8293649 100644
@@ -23,11 +23,11 @@ import iotests
 import time
 
 class TestStopWithBlockJob(iotests.QMPTestCase):
-    test_img = os.path.join(iotests.test_dir, 'test.img')
-    target_img = os.path.join(iotests.test_dir, 'target.img')
-    base_img = os.path.join(iotests.test_dir, 'base.img')
 
     def setUp(self):
+        self.test_img = os.path.join(self.workdir, 'test.img')
+        self.target_img = os.path.join(self.workdir, 'target.img')
+        self.base_img = os.path.join(self.workdir, 'base.img')
         iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G")
         iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img)
         iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img)
tests/qemu-iotests/132 +7 -12
index f53ef6e..3ca6f06 100644
@@ -23,36 +23,31 @@ import os
 import iotests
 from iotests import qemu_img, qemu_io
 
-test_img = os.path.join(iotests.test_dir, 'test.img')
-target_img = os.path.join(iotests.test_dir, 'target.img')
 
 class TestSingleDrive(iotests.QMPTestCase):
     image_len = 2 * 1024 * 1024 # MB
 
     def setUp(self):
+        self.test_img = os.path.join(self.workdir, 'test.img')
+        self.target_img = os.path.join(self.workdir, 'target.img')
         # Write data to the image so we can compare later
-        qemu_img('create', '-f', iotests.imgfmt, test_img, str(TestSingleDrive.image_len))
-        qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 0 2M', test_img)
+        qemu_img('create', '-f', iotests.imgfmt, self.test_img, str(TestSingleDrive.image_len))
+        qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 0 2M', self.test_img)
 
-        self.vm = iotests.VM().add_drive(test_img, 'discard=unmap')
+        self.vm = iotests.VM().add_drive(self.test_img, 'discard=unmap')
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        try:
-            os.remove(target_img)
-        except OSError:
-            pass
 
     def test_mirror_discard(self):
         result = self.vm.qmp('drive-mirror', device='drive0', sync='full',
-                             target=target_img)
+                             target=self.target_img)
         self.assert_qmp(result, 'return', {})
         self.vm.hmp_qemu_io('drive0', 'discard 0 64k')
         self.complete_and_wait('drive0')
         self.vm.shutdown()
-        self.assertTrue(iotests.compare_images(test_img, target_img),
+        self.assertTrue(iotests.compare_images(self.test_img, self.target_img),
                         'target image does not match source after mirroring')
 
 if __name__ == '__main__':
tests/qemu-iotests/136 +3 -4
index 4b99489..8109d63 100644
@@ -27,7 +27,6 @@ nsec_per_sec = 1000000000
 op_latency = nsec_per_sec / 1000 # See qtest_latency_ns in accounting.c
 bad_sector = 8192
 bad_offset = bad_sector * 512
-blkdebug_file = os.path.join(iotests.test_dir, 'blkdebug.conf')
 
 class BlockDeviceStatsTestCase(iotests.QMPTestCase):
     test_img = "null-aio://"
@@ -53,7 +52,7 @@ class BlockDeviceStatsTestCase(iotests.QMPTestCase):
         raise Exception("Device not found for blockstats: %s" % device)
 
     def create_blkdebug_file(self):
-        file = open(blkdebug_file, 'w')
+        file = open(self.blkdebug_file, 'w')
         file.write('''
 [inject-error]
 event = "read_aio"
@@ -68,6 +67,7 @@ sector = "%d"
         file.close()
 
     def setUp(self):
+        self.blkdebug_file = os.path.join(self.workdir, 'blkdebug.conf')
         drive_args = []
         drive_args.append("stats-intervals.0=%d" % interval_length)
         drive_args.append("stats-account-invalid=%s" %
@@ -76,7 +76,7 @@ sector = "%d"
                           (self.account_failed and "on" or "off"))
         self.create_blkdebug_file()
         self.vm = iotests.VM().add_drive('blkdebug:%s:%s' %
-                                         (blkdebug_file, self.test_img),
+                                         (self.blkdebug_file, self.test_img),
                                          ','.join(drive_args))
         self.vm.launch()
         # Set an initial value for the clock
@@ -84,7 +84,6 @@ sector = "%d"
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(blkdebug_file)
 
     def accounted_ops(self, read = False, write = False, flush = False):
         ops = 0
tests/qemu-iotests/139 +15 -18
index f8f0280..7699dd2 100644
@@ -23,8 +23,6 @@ import os
 import iotests
 import time
 
-base_img = os.path.join(iotests.test_dir, 'base.img')
-new_img = os.path.join(iotests.test_dir, 'new.img')
 if iotests.qemu_default_machine == 's390-ccw-virtio':
     default_virtio_blk = 'virtio-blk-ccw'
 else:
@@ -33,7 +31,9 @@ else:
 class TestBlockdevDel(iotests.QMPTestCase):
 
     def setUp(self):
-        iotests.qemu_img('create', '-f', iotests.imgfmt, base_img, '1M')
+        self.base_img = os.path.join(self.workdir, 'base.img')
+        self.new_img = os.path.join(self.workdir, 'new.img')
+        iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, '1M')
         self.vm = iotests.VM()
         if iotests.qemu_default_machine == 's390-ccw-virtio':
             self.vm.add_device("virtio-scsi-ccw,id=virtio-scsi")
@@ -44,9 +44,6 @@ class TestBlockdevDel(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(base_img)
-        if os.path.isfile(new_img):
-            os.remove(new_img)
 
     # Check whether a BlockDriverState exists
     def checkBlockDriverState(self, node, must_exist = True):
@@ -64,7 +61,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
                 'node-name': node,
                 'file': {'driver': 'file',
                          'node-name': file_node,
-                         'filename': base_img}}
+                         'filename': self.base_img}}
         result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
         self.assert_qmp(result, 'return', {})
         self.checkBlockDriverState(node)
@@ -74,12 +71,12 @@ class TestBlockdevDel(iotests.QMPTestCase):
     def addBlockDriverStateOverlay(self, node):
         self.checkBlockDriverState(node, False)
         iotests.qemu_img('create', '-u', '-f', iotests.imgfmt,
-                         '-b', base_img, new_img, '1M')
+                         '-b', self.base_img, self.new_img, '1M')
         opts = {'driver': iotests.imgfmt,
                 'node-name': node,
                 'backing': None,
                 'file': {'driver': 'file',
-                         'filename': new_img}}
+                         'filename': self.new_img}}
         result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
         self.assert_qmp(result, 'return', {})
         self.checkBlockDriverState(node)
@@ -143,7 +140,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
         self.checkBlockDriverState(node)
         self.checkBlockDriverState(overlay, False)
         opts = {'node-name': node,
-                'snapshot-file': new_img,
+                'snapshot-file': self.new_img,
                 'snapshot-node-name': overlay,
                 'format': iotests.imgfmt}
         result = self.vm.qmp('blockdev-snapshot-sync', conv_keys=False, **opts)
@@ -166,7 +163,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
         self.checkBlockDriverState(new_node, False)
         opts = {'device': node,
                 'job-id': node,
-                'target': new_img,
+                'target': self.new_img,
                 'node-name': new_node,
                 'sync': 'top',
                 'format': iotests.imgfmt}
@@ -189,7 +186,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
         image = {'driver': iotests.imgfmt,
                  'node-name': node,
                  'file': {'driver': 'file',
-                          'filename': base_img}}
+                          'filename': self.base_img}}
         opts = {'driver': 'blkdebug',
                 'node-name': debug,
                 'image': image}
@@ -205,15 +202,15 @@ class TestBlockdevDel(iotests.QMPTestCase):
         self.checkBlockDriverState(test, False)
         self.checkBlockDriverState(raw, False)
         self.checkBlockDriverState(blkverify, False)
-        iotests.qemu_img('create', '-f', iotests.imgfmt, new_img, '1M')
+        iotests.qemu_img('create', '-f', iotests.imgfmt, self.new_img, '1M')
         node_0 = {'driver': iotests.imgfmt,
                   'node-name': test,
                   'file': {'driver': 'file',
-                           'filename': base_img}}
+                           'filename': self.base_img}}
         node_1 = {'driver': iotests.imgfmt,
                   'node-name': raw,
                   'file': {'driver': 'file',
-                           'filename': new_img}}
+                           'filename': self.new_img}}
         opts = {'driver': 'blkverify',
                 'node-name': blkverify,
                 'test': node_0,
@@ -229,15 +226,15 @@ class TestBlockdevDel(iotests.QMPTestCase):
         self.checkBlockDriverState(child0, False)
         self.checkBlockDriverState(child1, False)
         self.checkBlockDriverState(quorum, False)
-        iotests.qemu_img('create', '-f', iotests.imgfmt, new_img, '1M')
+        iotests.qemu_img('create', '-f', iotests.imgfmt, self.new_img, '1M')
         child_0 = {'driver': iotests.imgfmt,
                    'node-name': child0,
                    'file': {'driver': 'file',
-                            'filename': base_img}}
+                            'filename': self.base_img}}
         child_1 = {'driver': iotests.imgfmt,
                    'node-name': child1,
                    'file': {'driver': 'file',
-                            'filename': new_img}}
+                            'filename': self.new_img}}
         opts = {'driver': 'quorum',
                 'node-name': quorum,
                 'vote-threshold': 1,
tests/qemu-iotests/147 +18 -23

102 lines not on this page · raw message

tests/qemu-iotests/148 +15 -18
index e01b061..9e66616 100644
@@ -22,14 +22,6 @@
 import os
 import iotests
 
-imgs = (os.path.join(iotests.test_dir, 'quorum0.img'),
-        os.path.join(iotests.test_dir, 'quorum1.img'),
-        os.path.join(iotests.test_dir, 'quorum2.img'))
-
-img_conf = (os.path.join(iotests.test_dir, 'quorum0.conf'),
-            os.path.join(iotests.test_dir, 'quorum1.conf'),
-            os.path.join(iotests.test_dir, 'quorum2.conf'))
-
 event_rate = 1000000000
 sector_size = 512
 offset = 10
@@ -48,15 +40,23 @@ sector = "%d"
         file.close()
 
     def setUp(self):
+        self.imgs = (os.path.join(self.workdir, 'quorum0.img'),
+                     os.path.join(self.workdir, 'quorum1.img'),
+                     os.path.join(self.workdir, 'quorum2.img'))
+
+        self.img_conf = (os.path.join(self.workdir, 'quorum0.conf'),
+                         os.path.join(self.workdir, 'quorum1.conf'),
+                         os.path.join(self.workdir, 'quorum2.conf'))
+
         driveopts = ['driver=quorum', 'vote-threshold=2']
         driveopts.append('read-pattern=%s' % self.read_pattern)
-        for i in range(len(imgs)):
-            iotests.qemu_img('create', '-f', iotests.imgfmt, imgs[i], '1M')
-            self.create_blkdebug_file(img_conf[i], i + offset)
+        for i in range(len(self.imgs)):
+            iotests.qemu_img('create', '-f', iotests.imgfmt, self.imgs[i], '1M')
+            self.create_blkdebug_file(self.img_conf[i], i + offset)
             driveopts.append('children.%d.driver=%s' % (i, iotests.imgfmt))
             driveopts.append('children.%d.file.driver=blkdebug' % i)
-            driveopts.append('children.%d.file.config=%s' % (i, img_conf[i]))
-            driveopts.append('children.%d.file.image.filename=%s' % (i, imgs[i]))
+            driveopts.append('children.%d.file.config=%s' % (i, self.img_conf[i]))
+            driveopts.append('children.%d.file.image.filename=%s' % (i, self.imgs[i]))
             driveopts.append('children.%d.node-name=img%d' % (i, i))
         self.vm = iotests.VM()
         self.vm.add_drive(None, opts = ','.join(driveopts))
@@ -64,9 +64,6 @@ sector = "%d"
 
     def tearDown(self):
         self.vm.shutdown()
-        for i in range(len(imgs)):
-            os.remove(imgs[i])
-            os.remove(img_conf[i])
 
     def do_check_event(self, node, sector = 0):
         if node == None:
@@ -107,7 +104,7 @@ sector = "%d"
 
         # I/O errors in different children: all events are emitted
         delay = 10
-        for i in range(len(imgs)):
+        for i in range(len(self.imgs)):
             self.vm.hmp_qemu_io("drive0", "aio_read %d %d" %
                                 ((offset + i) * sector_size, sector_size))
             self.vm.qtest("clock_step %d" % delay)
@@ -119,7 +116,7 @@ sector = "%d"
 
         # I/O errors in different children: all events are emitted
         delay = 2 * event_rate
-        for i in range(len(imgs)):
+        for i in range(len(self.imgs)):
             self.vm.hmp_qemu_io("drive0", "aio_read %d %d" %
                                 ((offset + i) * sector_size, sector_size))
             self.vm.qtest("clock_step %d" % delay)
tests/qemu-iotests/152 +8 -13
index fec546d..07f713d 100644
@@ -22,39 +22,34 @@ import os
 import iotests
 from iotests import qemu_img
 
-test_img = os.path.join(iotests.test_dir, 'test.img')
-target_img = os.path.join(iotests.test_dir, 'target.img')
 
 class TestUnaligned(iotests.QMPTestCase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, test_img, '512')
-        self.vm = iotests.VM().add_drive(test_img)
+        self.test_img = os.path.join(self.workdir, 'test.img')
+        self.target_img = os.path.join(self.workdir, 'target.img')
+        qemu_img('create', '-f', iotests.imgfmt, self.test_img, '512')
+        self.vm = iotests.VM().add_drive(self.test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        try:
-            os.remove(target_img)
-        except OSError:
-            pass
 
     def test_unaligned(self):
         result = self.vm.qmp('drive-mirror', device='drive0', sync='full',
-                             granularity=65536, target=target_img)
+                             granularity=65536, target=self.target_img)
         self.complete_and_wait()
         self.vm.shutdown()
-        self.assertEqual(iotests.image_size(test_img), iotests.image_size(target_img),
+        self.assertEqual(iotests.image_size(self.test_img), iotests.image_size(self.target_img),
                          "Target size doesn't match source when granularity when unaligend")
 
     def test_unaligned_with_update(self):
         result = self.vm.qmp('drive-mirror', device='drive0', sync='full',
-                             granularity=65536, target=target_img)
+                             granularity=65536, target=self.target_img)
         self.wait_ready()
         self.vm.hmp_qemu_io('drive0', 'write 0 512')
         self.complete_and_wait(wait_ready=False)
         self.vm.shutdown()
-        self.assertEqual(iotests.image_size(test_img), iotests.image_size(target_img),
+        self.assertEqual(iotests.image_size(self.test_img), iotests.image_size(self.target_img),
                          "Target size doesn't match source when granularity when unaligend")
 
 
tests/qemu-iotests/155 +23 -31

130 lines not on this page · raw message

tests/qemu-iotests/163 +19 -23

24 lines not on this page · raw message

tests/qemu-iotests/165 +4 -7
index a3932db..6672fa9 100755

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.9.5

Part of the diff is not shown here. View the raw message.

author
#33 lore raw

PATCH v5 9/10 qemu-iotests: add option to save temp files on error

In reply to Jeff Cody

Now that ./check takes care of cleaning up after each tests, it can also selectively not clean up. Add option to leave all output from tests intact if that test encountered an error.

---
 tests/qemu-iotests/check | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
tests/qemu-iotests/check +12 -1
index 057ea39..a66f7b0 100755
@@ -125,6 +125,7 @@ sortme=false
 expunge=true
 have_test_arg=false
 cachemode=false
+save_on_err=false
 
 tmp="${TEST_DIR}"/$$
 rm -f $tmp.list $tmp.tmp $tmp.sed
@@ -263,6 +264,8 @@ other options
     -o options          -o options to pass to qemu-img create/convert
     -T                  output timestamps
     -c mode             cache mode
+    -s                  save test scratch directory on test failure
+
 
 testlist options
     -g group[,group...]        include tests from these groups
@@ -435,6 +438,10 @@ testlist options
             xgroup=true
             xpand=false
             ;;
+        -s)
+            save_on_err=true
+            xpand=false
+            ;;
         '[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]')
             echo "No tests?"
             status=1
@@ -853,7 +860,11 @@ do
         _cleanup_protocols
         _cleanup_qemu
         )
-        rm -rf "$TEST_DIR_SEQ"
+
+        if [ "$save_on_err" != "true" ] || [ "$err" != "true" ]
+        then
+            rm -rf "$TEST_DIR_SEQ"
+        fi
 
     fi
 
Signature · 1 line
2.9.5
#34 lore raw

Re: [Qemu-devel] [PATCH v5 09/10] qemu-iotests: add option to save temp files on error

Quoted 22 lines · Jeff Cody

Now that ./check takes care of cleaning up after each tests, it can also selectively not clean up. Add option to leave all output from tests intact if that test encountered an error.

---
 tests/qemu-iotests/check | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
tests/qemu-iotests/check +1 -0
index 057ea39..a66f7b0 100755
@@ -125,6 +125,7 @@ sortme=false
 expunge=true
 have_test_arg=false
 cachemode=false
+save_on_err=false
 
 tmp="${TEST_DIR}"/$$
 rm -f $tmp.list $tmp.tmp $tmp.sed

tests/qemu-iotests/check other options · line 267

Quoted 4 lines
tests/qemu-iotests/check +1 -0
@@ -263,6 +264,8 @@ other options
     -o options          -o options to pass to qemu-img create/convert
     -T                  output timestamps
     -c mode             cache mode
+    -s                  save test scratch directory on test failure

Is there an easy way to force all tests to fail (and thus preserve the scratch directory for inspection, even though the test normally would succeed)? It might serve as a useful double-check that you got everything in this series, maybe even worth documenting in the commit messages.

tests/qemu-iotests/check do · line 867

Quoted 5 lines
tests/qemu-iotests/check +5 -0
+
+        if [ "$save_on_err" != "true" ] || [ "$err" != "true" ]
+        then
+            rm -rf "$TEST_DIR_SEQ"
+        fi

I suppose the easiest way is to hack this 'if'.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#35 lore raw

PATCH v5 10/10 qemu-iotests: add support for running multi-threaded iotests

In reply to Jeff Cody

This adds support for running qemu-iotests in an arbitrary number of sub-processes, so that tests can be run in parallel.

This necessarily changes the output format, although it should still be familiar. If you run in a single thread, the output format will largely be the same as before this patch.

To run in more than one process, use the '-j num' option, e.g.:
./check -qcow2 -j 5

Some caveats:

* Some output format options, such as timestamps, are currently not compatible with multiple jobs. If you select multiple jobs, timestamps will be disabled.

* Some tests may be more prone to failure with multiple jobs. This isn't a flaw of multiple jobs per se, but rather of fragile tests. Some tests (181, 183) are very sensitive in timing, and high cpu loads can cause them to fail. It may be worth adding support for 'single-thread only' tests in subsequent patches, that complete designated single-thread jobs at the end.

* Running protocol tests multi-threaded may fail, as multiple tests may try to bind the same address.

If '-j' is not specified, the default is a single iotest being run at a time.

---
 tests/qemu-iotests/check      | 427 +++++++++++++++++++++++++++++-------------
 tests/qemu-iotests/common.rc  |   2 +-
 tests/qemu-iotests/iotests.py |   4 +-
 3 files changed, 297 insertions(+), 136 deletions(-)
tests/qemu-iotests/check +294 -133
index a66f7b0..363617e 100755
@@ -26,6 +26,8 @@ n_bad=0
 bad=""
 notrun=""
 interrupt=true
+TEST_DIR_SEQ="$TEST_DIR"
+MAX_JOBS=1
 
 # by default don't output timestamps
 timestamp=${TIMESTAMP:=false}
@@ -125,6 +127,7 @@ sortme=false
 expunge=true
 have_test_arg=false
 cachemode=false
+multijob=false
 save_on_err=false
 
 tmp="${TEST_DIR}"/$$
@@ -220,6 +223,11 @@ s/ .*//p
         CACHEMODE_IS_DEFAULT=false
         cachemode=false
         continue
+    elif $multijob
+    then
+        MAX_JOBS="$r"
+        multijob=false
+        continue
     fi
 
     xpand=true
@@ -262,9 +270,10 @@ other options
     -misalign           misalign memory allocations
     -n                  show me, do not run tests
     -o options          -o options to pass to qemu-img create/convert
-    -T                  output timestamps
+    -T                  output timestamps, disabled if using '-j'
     -c mode             cache mode
     -s                  save test scratch directory on test failure
+    -j num              run tests in 'num' processes
 
 
 testlist options
@@ -442,6 +451,10 @@ testlist options
             save_on_err=true
             xpand=false
             ;;
+        -j)
+            multijob=true
+            xpand=false
+            ;;
         '[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]')
             echo "No tests?"
             status=1
@@ -506,6 +519,20 @@ BEGIN        { for (t='$start'; t<='$end'; t++) printf "%03d\n",t }' \
 
 done
 
+# No need for multi-process support, and this keeps output simpler
+if $showme
+then
+    MAX_JOBS=1
+fi
+
+# TODO: Change test output format so that this can be useful
+#       with multi-process jobs
+if $timestamp && [ $MAX_JOBS -gt 1 ]
+then
+    echo "Not showing timestamps with multi-job test"
+    timestamp=false
+fi
+
 # Set qemu-io cache mode with $CACHEMODE we have
 QEMU_IO_OPTIONS="$QEMU_IO_OPTIONS --cache $CACHEMODE"
 
@@ -670,8 +697,11 @@ END        { if (NR > 0) {
 
         if [ ! -z "$notrun" ]
         then
-            echo "Not run:$notrun"
-            echo "Not run:$notrun" >>check.log
+            # if run with $MAX_JOBS > 1, this will likely be
+            # out of order
+            notrun=$(echo $notrun|tr " " "\n"|sort|tr "\n" " ")
+            echo "Not run: $notrun"
+            echo "Not run: $notrun" >>check.log
         fi
         if [ ! -z "$n_bad" -a $n_bad != 0 ]
         then
@@ -694,7 +724,7 @@ END        { if (NR > 0) {
     rm -f $tmp.*
 }
 
-trap "_wrapup; exit \$status" 0 1 2 3 15
+trap "_wait_to_finish; _wrapup; exit \$status" 0 1 2 3 15
 
 [ -f $TIMESTAMP_FILE ] || touch $TIMESTAMP_FILE
 
@@ -718,11 +748,235 @@ seq="check"
 
 [ -n "$TESTS_REMAINING_LOG" ] && echo $list > $TESTS_REMAINING_LOG
 
+# Execute actual test.  This will be run in a background process.
+_do_test()
+{
+    local seq=$1
+    local err=false
+    local tmp="${TEST_DIR}/$seq/$seq"
+    local TEST_DIR_SEQ=$TEST_DIR/$seq
+
+    _wallclock > "${TEST_DIR}/$seq.start.clock"
+    $timestamp && printf %s "        [$(date "+%T")]"
+
+    if [ "$(head -n 1 "$source_iotests/$seq")" == "#!/usr/bin/env python" ]; then
+        run_command="$PYTHON $seq"
+    else
+        run_command="./$seq"
+    fi
+    export OUTPUT_DIR=$PWD
+    if $debug; then
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        cd "$source_iotests";
+        . ./common.config
+        . ./common.rc
+            MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
+            $run_command -d 2>&1 | tee $tmp.out
+        )
+    else
+        # Do this in a sub-shell, so we are operating on the right
+        # TEST_DIR / QEMU_TEST_DIR
+        (
+        export TEST_DIR=$TEST_DIR_SEQ
+        cd "$source_iotests";
+        . ./common.config
+        . ./common.rc
+            MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
+            $run_command >$tmp.out 2>&1
+        )
+    fi
+    sts=$?
+    $timestamp && _timestamp
+    _wallclock > "${TEST_DIR}/$seq.stop.clock"
+
+    if [ -f core ]
+    then
+        printf " [dumped core]"
+        mv core $seq.core
+        err=true
+    fi
+
+    if [ ! -f $seq.notrun ]
+    then
+        if [ $sts -ne 0 ]
+        then
+            printf %s " [failed, exit status $sts]" > "$TEST_DIR/$seq.err"
+            err=true
+        fi
+    fi
+
+
+    # Do this in a sub-shell, so we are operating on the right
+    # TEST_DIR / QEMU_TEST_DIR
+    (
+    export TEST_DIR=$TEST_DIR_SEQ
+    cd "$source_iotests";
+    . ./common.config
+    . ./common.rc
+    . ./common.qemu
+
+    _cleanup_protocols
+    _cleanup_qemu
+    )
+
+    if [ "$err" == "true" ]
+    then
+        touch "$TEST_DIR/$seq.err"
+    fi
+}
+
+# Runs after we detect a test has completed
+function _check_results()
+{
+    local seq=$1
+    local err=false
+    local localtmp="${TEST_DIR}/$seq/$seq"
+    local TEST_DIR_SEQ=$TEST_DIR/$seq
+    local success=true
+
+    if [ -f $seq.notrun ]
+    then
+        if [ $MAX_JOBS -gt 1 ]
+        then
+            printf "$seq   "
+        fi
+        $timestamp || printf " [not run] "
+        $timestamp && printf " [not run]" && printf %s "        $seq -- "
+        cat $seq.notrun
+        notrun="$notrun $seq"
+        success=false
+    else
+        reference="$source_iotests/$seq.out"
+        reference_machine="$source_iotests/$seq.$QEMU_DEFAULT_MACHINE.out"
+        if [ -f "$reference_machine" ]; then
+            reference="$reference_machine"
+        fi
+
+        reference_format="$source_iotests/$seq.out.$IMGFMT"
+        if [ -f "$reference_format" ]; then
+            reference="$reference_format"
+        fi
+
+        if [ "$CACHEMODE" = "none" ]; then
+            [ -f "$source_iotests/$seq.out.nocache" ] && reference="$source_iotests/$seq.out.nocache"
+        fi
+
+        if [ ! -f "$reference" ]
+        then
+            echo " - no qualified output"
+        else
+            if diff -w "$reference" $localtmp.out >/dev/null 2>&1
+            then
+                if $err
+                then
+                    :
+                else
+                    stop=$(cat "$TEST_DIR/$seq.stop.clock")
+                    start=$(cat "$TEST_DIR/$seq.start.clock")
+                    echo "$seq `expr $stop - $start`" >>$tmp.time
+                fi
+            else
+                printf " - output mismatch (see $seq.out.bad)"
+                mv $localtmp.out $seq.out.bad
+                $diff -w "$reference" $(realpath $seq.out.bad)
+                err=true
+            fi
+        fi
+    fi
+
+    if [ "$err" == "true" ] || [ -f "$TEST_DIR/$seq.err" ]
+    then
+        if [ $MAX_JOBS -gt 1 ]
+        then
+            printf "\n$seq    [  fail ] "
+        fi
+        if [ -f "$TEST_DIR/$seq.err" ]
+        then
+            cat "$TEST_DIR/$seq.err"
+        fi
+        success=false
+        err=true
+        bad="$bad $seq"
+        n_bad=`expr $n_bad + 1`
+        quick=false
+    fi
+
+
+    if [ "$save_on_err" != "true" ] || [ "$err" != "true" ]
+    then
+        rm -rf "$TEST_DIR_SEQ"
+    fi
+
+    [ -f $seq.notrun ] || try=`expr $try + 1`
+
+    rm -f "$TEST_DIR/$seq.stop.clock" "$TEST_DIR/$seq.start.clock"
+
+    if [ $MAX_JOBS -eq 1 ] || [ "$success" == "false" ]
+    then
+        printf "\n"
+    fi
+}
+
+# Even if interrupted, we want to wait until
+# all tests have completed, so we can properly clean
+# up after them via _check_results
+function _wait_to_finish()
+{
+    while [ $jobs_running -gt 0 ]
+    do
+        i=0
+        while [ $jobs_running -gt 0 ]
+        do
+            job=${job_slots[$i]}
+            if [ $job -gt 0 ]
+            then
+                if [ -z "$(ps -o pid -h -p $job)" ]
+                then
+                    _check_results ${job_seq[$i]}
+                    job_slots[$i]=0
+                    job_seq[$i]=-1
+                    let jobs_running--
+                fi
+            fi
+            let i++
+            let i=`expr $i % $MAX_JOBS`
+            sleep 0.1
+        done
+    done
+}
+
+job_slots=
+
+jobs_running=0
+
+for i in `seq 0 $MAX_JOBS`
+do
+    job_slots[$i]=0
+done
+
+
+# Now iterate of the list of tests
 for seq in $list
 do
-    TEST_DIR_SEQ=$TEST_DIR/$seq
-    err=false
+
+    rm -f $seq.out.bad
+    lasttime=`sed -n -e "/^$seq /s/.* //p" <$TIMESTAMP_FILE`
     printf %s "$seq"
+    if [ "X$lasttime" != X ]; then
+        printf %s " ${lasttime}s ..."
+    else
+        printf "        "        # prettier output with timestamps.
+    fi
+    rm -f core $seq.notrun
+    if [ $MAX_JOBS -gt 1 ]
+    then
+        printf "\n"
+    fi
+
+    err=false
     if [ -n "$TESTS_REMAINING_LOG" ] ; then
         sed -e "s/$seq//" -e 's/  / /' -e 's/^ *//' $TESTS_REMAINING_LOG > $TESTS_REMAINING_LOG.tmp
         mv $TESTS_REMAINING_LOG.tmp $TESTS_REMAINING_LOG
@@ -743,144 +997,51 @@ do
         echo " - no such test?"
         echo "/^$seq\$/d" >>$tmp.expunged
     else
-        # really going to try and run this one
-        #
-        rm -f $seq.out.bad
-        lasttime=`sed -n -e "/^$seq /s/.* //p" <$TIMESTAMP_FILE`
-        if [ "X$lasttime" != X ]; then
-                printf %s " ${lasttime}s ..."
-        else
-                printf "        "        # prettier output with timestamps.
-        fi
-        rm -f core $seq.notrun
 
-        start=`_wallclock`
-        $timestamp && printf %s "        [$(date "+%T")]"
-
-        if [ "$(head -n 1 "$source_iotests/$seq")" == "#!/usr/bin/env python" ]; then
-            run_command="$PYTHON $seq"
-        else
-            run_command="./$seq"
-        fi
-        export OUTPUT_DIR=$PWD
-        if $debug; then
-            # Do this in a sub-shell, so we are operating on the right
-            # TEST_DIR / QEMU_TEST_DIR
-            (
-            export TEST_DIR=$TEST_DIR_SEQ
-            cd "$source_iotests";
-            . ./common.config
-            . ./common.rc
-            MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-                    $run_command -d 2>&1 | tee $tmp.out
-            )
-        else
-            # Do this in a sub-shell, so we are operating on the right
-            # TEST_DIR / QEMU_TEST_DIR
-            (
-            export TEST_DIR=$TEST_DIR_SEQ
-            cd "$source_iotests";
-            . ./common.config
-            . ./common.rc
-            MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(($RANDOM % 255 + 1))} \
-                    $run_command >$tmp.out 2>&1
-            )
-        fi
-        sts=$?
-        $timestamp && _timestamp
-        stop=`_wallclock`
-
-        if [ -f core ]
-        then
-            printf " [dumped core]"
-            mv core $seq.core
-            err=true
-        fi
-
-        if [ -f $seq.notrun ]
+        if [ $MAX_JOBS -eq 1 ]
         then
-            $timestamp || printf " [not run] "
-            $timestamp && echo " [not run]" && printf %s "        $seq -- "
-            cat $seq.notrun
-            notrun="$notrun $seq"
+            # This way we can mimic the same output format before multi-process test
+            # support was added, if we are running one test at a time
+            _do_test $seq
+            _check_results $seq
         else
-            if [ $sts -ne 0 ]
-            then
-                printf %s " [failed, exit status $sts]"
-                err=true
-            fi
-
-            reference="$source_iotests/$seq.out"
-            reference_machine="$source_iotests/$seq.$QEMU_DEFAULT_MACHINE.out"
-            if [ -f "$reference_machine" ]; then
-                reference="$reference_machine"
-            fi
-
-            reference_format="$source_iotests/$seq.out.$IMGFMT"
-            if [ -f "$reference_format" ]; then
-                reference="$reference_format"
-            fi
-
-            if [ "$CACHEMODE" = "none" ]; then
-                [ -f "$source_iotests/$seq.out.nocache" ] && reference="$source_iotests/$seq.out.nocache"
-            fi
-
-            if [ ! -f "$reference" ]
-            then
-                echo " - no qualified output"
-                err=true
-            else
-                if diff -w "$reference" $tmp.out >/dev/null 2>&1
+            # loop until a job slot becomes free
+            i=0
+            while true
+            do
+                job=${job_slots[$i]}
+                if [ $job -le 0 ]
+                then
+                    let jobs_running++
+                    job_seq[$i]=$seq
+                    # Execute the test in the background
+                    _do_test $seq &
+                    job_slots[$i]=$!
+                    break
+                elif [ -z "$(ps -o pid -h -p $job)" ]
                 then
-                    echo ""
-                    if $err
-                    then
-                        :
-                    else
-                        echo "$seq `expr $stop - $start`" >>$tmp.time
-                    fi
-                else
-                    echo " - output mismatch (see $seq.out.bad)"
-                    mv $tmp.out $seq.out.bad
-                    $diff -w "$reference" $(realpath $seq.out.bad)
-                    err=true
+                    # job has completed
+                    _check_results ${job_seq[$i]}
+                    job_slots[$i]=0
+                    job_seq[$i]=-1
+                    let jobs_running--
+                    continue
                 fi
-            fi
-        fi
-
-        # Do this in a sub-shell, so we are operating on the right
-        # TEST_DIR / QEMU_TEST_DIR
-        (
-        export TEST_DIR=$TEST_DIR_SEQ
-        cd "$source_iotests";
-        . ./common.config
-        . ./common.rc
-        . ./common.qemu
 
-        _cleanup_protocols
-        _cleanup_qemu
-        )
+                let i++
+                let i=`expr $i % $MAX_JOBS`
 
-        if [ "$save_on_err" != "true" ] || [ "$err" != "true" ]
-        then
-            rm -rf "$TEST_DIR_SEQ"
+                if [ $i -eq 0 ]
+                then
+                    sleep 0.25
+                fi
+            done
         fi
-
     fi
-
-    # come here for each test, except when $showme is true
-    #
-    if $err
-    then
-        bad="$bad $seq"
-        n_bad=`expr $n_bad + 1`
-        quick=false
-    fi
-    [ -f $seq.notrun ] || try=`expr $try + 1`
-
-    seq="after_$seq"
 done
 
+_wait_to_finish
+
 interrupt=false
 status=`expr $n_bad`
 exit
tests/qemu-iotests/common.rc +1 -1
index b26b02f..273a815 100644
@@ -392,7 +392,7 @@ _img_info()
 #
 _notrun()
 {
-    echo "$*" >"$OUTPUT_DIR/$seq.notrun"
+    printf "$*" >"$OUTPUT_DIR/$seq.notrun"
     echo "$seq not run: $*"
     status=0
     exit
tests/qemu-iotests/iotests.py +2 -2
index 7ff400a..11dfcf5 100644
@@ -427,8 +427,8 @@ def notrun(reason):
     # Each test in qemu-iotests has a number ("seq")
     seq = os.path.basename(sys.argv[0])
 
-    open('%s/%s.notrun' % (output_dir, seq), 'wb').write(reason + '\n')
-    print '%s not run: %s' % (seq, reason)
+    open('%s/%s.notrun' % (output_dir, seq), 'wb').write(reason)
+    print '%s not run: %s\n' % (seq, reason)
     sys.exit(0)
 
 def verify_image_format(supported_fmts=[], unsupported_fmts=[]):
Signature · 1 line
2.9.5
author
#36 lore raw

Re: [Qemu-devel] [PATCH v5 10/10] qemu-iotests: add support for running multi-threaded iotests

Quoted 40 lines · Jeff Cody

This adds support for running qemu-iotests in an arbitrary number of sub-processes, so that tests can be run in parallel.

This necessarily changes the output format, although it should still be familiar. If you run in a single thread, the output format will largely be the same as before this patch.

To run in more than one process, use the '-j num' option, e.g.:
./check -qcow2 -j 5

Some caveats:

* Some output format options, such as timestamps, are currently not compatible with multiple jobs. If you select multiple jobs, timestamps will be disabled.

* Some tests may be more prone to failure with multiple jobs. This isn't a flaw of multiple jobs per se, but rather of fragile tests. Some tests (181, 183) are very sensitive in timing, and high cpu loads can cause them to fail. It may be worth adding support for 'single-thread only' tests in subsequent patches, that complete designated single-thread jobs at the end.

* Running protocol tests multi-threaded may fail, as multiple tests may try to bind the same address.

If '-j' is not specified, the default is a single iotest being run at a time.

---
 tests/qemu-iotests/check      | 427 +++++++++++++++++++++++++++++-------------
 tests/qemu-iotests/common.rc  |   2 +-
 tests/qemu-iotests/iotests.py |   4 +-
 3 files changed, 297 insertions(+), 136 deletions(-)
tests/qemu-iotests/check +0 -0
index a66f7b0..363617e 100755

[...]

Quoted 17 lines
tests/qemu-iotests/check +17 -0
+
+# Even if interrupted, we want to wait until
+# all tests have completed, so we can properly clean
+# up after them via _check_results
+function _wait_to_finish()
+{
+    while [ $jobs_running -gt 0 ]
+    do
+        i=0
+        while [ $jobs_running -gt 0 ]
+        do
+            job=${job_slots[$i]}
+            if [ $job -gt 0 ]
+            then
+                if [ -z "$(ps -o pid -h -p $job)" ]
+                then
+                    _check_results ${job_seq[$i]}

tests/qemu-iotests/check seq="check" · line 948

Quoted 10 lines
tests/qemu-iotests/check +10 -0
+                    job_slots[$i]=0
+                    job_seq[$i]=-1
+                    let jobs_running--
+                fi
+            fi
+            let i++
+            let i=`expr $i % $MAX_JOBS`
+            sleep 0.1
+        done
+    done

Oops. That double while loop, while harmless, is spurious. When I do a v6, I'll remove the outer one when addressing any other review comments.