Skip to contentSkip to conversation

iotests: drop group file

PATCH In review Vladimir Sementsov-Ogievskiy 5 participants 10 messages Mar 24, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iotests: drop group file

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

    grep '^[0-9]\{3\} ' group | while read line; do
        file=$(awk '{print $1}' <<< "$line");
        groups=$(sed -e 's/^... //' <<< "$line");
        awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
        cat tmp > $file;
    done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

check script is also refactored to make it simple to do next cool thing about iotests: allow meaningful names for test-case files.

4. Remove group file.

---

Hi all!

I understand, that it's 5.1 material, and I will have to regenerate generated part and resend, but I just really want to share the idea with you. Hope you like it:)

 docs/devel/testing.rst   |  30 +++-
 tests/qemu-iotests/001   |   1 +
 tests/qemu-iotests/002   |   1 +
 tests/qemu-iotests/003   |   1 +
 tests/qemu-iotests/004   |   1 +
 tests/qemu-iotests/005   |   1 +
 tests/qemu-iotests/007   |   1 +
 tests/qemu-iotests/008   |   1 +
 tests/qemu-iotests/009   |   1 +
 tests/qemu-iotests/010   |   1 +
 tests/qemu-iotests/011   |   1 +
 tests/qemu-iotests/012   |   1 +
 tests/qemu-iotests/013   |   1 +
 tests/qemu-iotests/014   |   1 +
 tests/qemu-iotests/015   |   1 +
 tests/qemu-iotests/017   |   1 +
 tests/qemu-iotests/018   |   1 +
 tests/qemu-iotests/019   |   1 +
 tests/qemu-iotests/020   |   1 +
 tests/qemu-iotests/021   |   1 +
 tests/qemu-iotests/022   |   1 +
 tests/qemu-iotests/023   |   1 +
 tests/qemu-iotests/024   |   1 +
 tests/qemu-iotests/025   |   1 +
 tests/qemu-iotests/026   |   1 +
 tests/qemu-iotests/027   |   1 +
 tests/qemu-iotests/028   |   1 +
 tests/qemu-iotests/029   |   1 +
 tests/qemu-iotests/030   |   1 +
 tests/qemu-iotests/031   |   1 +
 tests/qemu-iotests/032   |   1 +
 tests/qemu-iotests/033   |   1 +
 tests/qemu-iotests/034   |   1 +
 tests/qemu-iotests/035   |   1 +
 tests/qemu-iotests/036   |   1 +
 tests/qemu-iotests/037   |   1 +
 tests/qemu-iotests/038   |   1 +
 tests/qemu-iotests/039   |   1 +
 tests/qemu-iotests/040   |   1 +
 tests/qemu-iotests/041   |   1 +
 tests/qemu-iotests/042   |   1 +
 tests/qemu-iotests/043   |   1 +
 tests/qemu-iotests/044   |   1 +
 tests/qemu-iotests/045   |   1 +
 tests/qemu-iotests/046   |   1 +
 tests/qemu-iotests/047   |   1 +
 tests/qemu-iotests/048   |   1 +
 tests/qemu-iotests/049   |   1 +
 tests/qemu-iotests/050   |   1 +
 tests/qemu-iotests/051   |   1 +
 tests/qemu-iotests/052   |   1 +
 tests/qemu-iotests/053   |   1 +
 tests/qemu-iotests/054   |   1 +
 tests/qemu-iotests/055   |   1 +
 tests/qemu-iotests/056   |   1 +
 tests/qemu-iotests/057   |   1 +
 tests/qemu-iotests/058   |   1 +
 tests/qemu-iotests/059   |   1 +
 tests/qemu-iotests/060   |   1 +
 tests/qemu-iotests/061   |   1 +
 tests/qemu-iotests/062   |   1 +
 tests/qemu-iotests/063   |   1 +
 tests/qemu-iotests/064   |   1 +
 tests/qemu-iotests/065   |   1 +
 tests/qemu-iotests/066   |   1 +
 tests/qemu-iotests/067   |   1 +
 tests/qemu-iotests/068   |   1 +
 tests/qemu-iotests/069   |   1 +
 tests/qemu-iotests/070   |   1 +
 tests/qemu-iotests/071   |   1 +
 tests/qemu-iotests/072   |   1 +
 tests/qemu-iotests/073   |   1 +
 tests/qemu-iotests/074   |   1 +
 tests/qemu-iotests/075   |   1 +
 tests/qemu-iotests/076   |   1 +
 tests/qemu-iotests/077   |   1 +
 tests/qemu-iotests/078   |   1 +
 tests/qemu-iotests/079   |   1 +
 tests/qemu-iotests/080   |   1 +
 tests/qemu-iotests/081   |   1 +
 tests/qemu-iotests/082   |   1 +
 tests/qemu-iotests/083   |   1 +
 tests/qemu-iotests/084   |   1 +
 tests/qemu-iotests/085   |   1 +
 tests/qemu-iotests/086   |   1 +
 tests/qemu-iotests/087   |   1 +
 tests/qemu-iotests/088   |   1 +
 tests/qemu-iotests/089   |   1 +
 tests/qemu-iotests/090   |   1 +
 tests/qemu-iotests/091   |   1 +
 tests/qemu-iotests/092   |   1 +
 tests/qemu-iotests/093   |   1 +
 tests/qemu-iotests/094   |   1 +
 tests/qemu-iotests/095   |   1 +
 tests/qemu-iotests/096   |   1 +
 tests/qemu-iotests/097   |   1 +
 tests/qemu-iotests/098   |   1 +
 tests/qemu-iotests/099   |   1 +
 tests/qemu-iotests/101   |   1 +
 tests/qemu-iotests/102   |   1 +
 tests/qemu-iotests/103   |   1 +
 tests/qemu-iotests/104   |   1 +
 tests/qemu-iotests/105   |   1 +
 tests/qemu-iotests/106   |   1 +
 tests/qemu-iotests/107   |   1 +
 tests/qemu-iotests/108   |   1 +
 tests/qemu-iotests/109   |   1 +
 tests/qemu-iotests/110   |   1 +
 tests/qemu-iotests/111   |   1 +
 tests/qemu-iotests/112   |   1 +
 tests/qemu-iotests/113   |   1 +
 tests/qemu-iotests/114   |   1 +
 tests/qemu-iotests/115   |   1 +
 tests/qemu-iotests/116   |   1 +
 tests/qemu-iotests/117   |   1 +
 tests/qemu-iotests/118   |   1 +
 tests/qemu-iotests/119   |   1 +
 tests/qemu-iotests/120   |   1 +
 tests/qemu-iotests/121   |   1 +
 tests/qemu-iotests/122   |   1 +
 tests/qemu-iotests/123   |   1 +
 tests/qemu-iotests/124   |   1 +
 tests/qemu-iotests/125   |   1 +
 tests/qemu-iotests/126   |   1 +
 tests/qemu-iotests/127   |   1 +
 tests/qemu-iotests/128   |   1 +
 tests/qemu-iotests/129   |   1 +
 tests/qemu-iotests/130   |   1 +
 tests/qemu-iotests/131   |   1 +
 tests/qemu-iotests/132   |   1 +
 tests/qemu-iotests/133   |   1 +
 tests/qemu-iotests/134   |   1 +
 tests/qemu-iotests/135   |   1 +
 tests/qemu-iotests/136   |   1 +
 tests/qemu-iotests/137   |   1 +
 tests/qemu-iotests/138   |   1 +
 tests/qemu-iotests/139   |   1 +
 tests/qemu-iotests/140   |   1 +
 tests/qemu-iotests/141   |   1 +
 tests/qemu-iotests/143   |   1 +
 tests/qemu-iotests/144   |   1 +
 tests/qemu-iotests/145   |   1 +
 tests/qemu-iotests/146   |   1 +
 tests/qemu-iotests/147   |   1 +
 tests/qemu-iotests/148   |   1 +
 tests/qemu-iotests/149   |   1 +
 tests/qemu-iotests/150   |   1 +
 tests/qemu-iotests/151   |   1 +
 tests/qemu-iotests/152   |   1 +
 tests/qemu-iotests/153   |   1 +
 tests/qemu-iotests/154   |   1 +
 tests/qemu-iotests/155   |   1 +
 tests/qemu-iotests/156   |   1 +
 tests/qemu-iotests/157   |   1 +
 tests/qemu-iotests/158   |   1 +
 tests/qemu-iotests/159   |   1 +
 tests/qemu-iotests/160   |   1 +
 tests/qemu-iotests/161   |   1 +
 tests/qemu-iotests/162   |   1 +
 tests/qemu-iotests/163   |   1 +
 tests/qemu-iotests/165   |   1 +
 tests/qemu-iotests/169   |   1 +
 tests/qemu-iotests/170   |   1 +
 tests/qemu-iotests/171   |   1 +
 tests/qemu-iotests/172   |   1 +
 tests/qemu-iotests/173   |   1 +
 tests/qemu-iotests/174   |   1 +
 tests/qemu-iotests/175   |   1 +
 tests/qemu-iotests/176   |   1 +
 tests/qemu-iotests/177   |   1 +
 tests/qemu-iotests/178   |   1 +
 tests/qemu-iotests/179   |   1 +
 tests/qemu-iotests/181   |   1 +
 tests/qemu-iotests/182   |   1 +
 tests/qemu-iotests/183   |   1 +
 tests/qemu-iotests/184   |   1 +
 tests/qemu-iotests/185   |   1 +
 tests/qemu-iotests/186   |   1 +
 tests/qemu-iotests/187   |   1 +
 tests/qemu-iotests/188   |   1 +
 tests/qemu-iotests/189   |   1 +
 tests/qemu-iotests/190   |   1 +
 tests/qemu-iotests/191   |   1 +
 tests/qemu-iotests/192   |   1 +
 tests/qemu-iotests/194   |   1 +
 tests/qemu-iotests/195   |   1 +
 tests/qemu-iotests/196   |   1 +
 tests/qemu-iotests/197   |   1 +
 tests/qemu-iotests/198   |   1 +
 tests/qemu-iotests/199   |   1 +
 tests/qemu-iotests/200   |   1 +
 tests/qemu-iotests/201   |   1 +
 tests/qemu-iotests/202   |   1 +
 tests/qemu-iotests/203   |   1 +
 tests/qemu-iotests/204   |   1 +
 tests/qemu-iotests/205   |   1 +
 tests/qemu-iotests/206   |   1 +
 tests/qemu-iotests/207   |   1 +
 tests/qemu-iotests/208   |   1 +
 tests/qemu-iotests/209   |   1 +
 tests/qemu-iotests/210   |   1 +
 tests/qemu-iotests/211   |   1 +
 tests/qemu-iotests/212   |   1 +
 tests/qemu-iotests/213   |   1 +
 tests/qemu-iotests/214   |   1 +
 tests/qemu-iotests/215   |   1 +
 tests/qemu-iotests/216   |   1 +
 tests/qemu-iotests/217   |   1 +
 tests/qemu-iotests/218   |   1 +
 tests/qemu-iotests/219   |   1 +
 tests/qemu-iotests/220   |   1 +
 tests/qemu-iotests/221   |   1 +
 tests/qemu-iotests/222   |   1 +
 tests/qemu-iotests/223   |   1 +
 tests/qemu-iotests/224   |   1 +
 tests/qemu-iotests/225   |   1 +
 tests/qemu-iotests/226   |   1 +
 tests/qemu-iotests/227   |   1 +
 tests/qemu-iotests/228   |   1 +
 tests/qemu-iotests/229   |   1 +
 tests/qemu-iotests/231   |   1 +
 tests/qemu-iotests/232   |   1 +
 tests/qemu-iotests/233   |   1 +
 tests/qemu-iotests/234   |   1 +
 tests/qemu-iotests/235   |   1 +
 tests/qemu-iotests/236   |   1 +
 tests/qemu-iotests/237   |   1 +
 tests/qemu-iotests/238   |   1 +
 tests/qemu-iotests/239   |   1 +
 tests/qemu-iotests/240   |   1 +
 tests/qemu-iotests/241   |   1 +
 tests/qemu-iotests/242   |   1 +
 tests/qemu-iotests/243   |   1 +
 tests/qemu-iotests/244   |   1 +
 tests/qemu-iotests/245   |   1 +
 tests/qemu-iotests/246   |   1 +
 tests/qemu-iotests/247   |   1 +
 tests/qemu-iotests/248   |   1 +
 tests/qemu-iotests/249   |   1 +
 tests/qemu-iotests/250   |   1 +
 tests/qemu-iotests/251   |   1 +
 tests/qemu-iotests/252   |   1 +
 tests/qemu-iotests/253   |   1 +
 tests/qemu-iotests/254   |   1 +
 tests/qemu-iotests/255   |   1 +
 tests/qemu-iotests/256   |   1 +
 tests/qemu-iotests/257   |   1 +
 tests/qemu-iotests/258   |   1 +
 tests/qemu-iotests/259   |   1 +
 tests/qemu-iotests/260   |   1 +
 tests/qemu-iotests/261   |   1 +
 tests/qemu-iotests/262   |   1 +
 tests/qemu-iotests/263   |   1 +
 tests/qemu-iotests/264   |   1 +
 tests/qemu-iotests/265   |   1 +
 tests/qemu-iotests/266   |   1 +
 tests/qemu-iotests/267   |   1 +
 tests/qemu-iotests/268   |   1 +
 tests/qemu-iotests/270   |   1 +
 tests/qemu-iotests/272   |   1 +
 tests/qemu-iotests/273   |   1 +
 tests/qemu-iotests/277   |   1 +
 tests/qemu-iotests/279   |   1 +
 tests/qemu-iotests/280   |   1 +
 tests/qemu-iotests/281   |   1 +
 tests/qemu-iotests/282   |   1 +
 tests/qemu-iotests/283   |   1 +
 tests/qemu-iotests/284   |   1 +
 tests/qemu-iotests/286   |   1 +
 tests/qemu-iotests/288   |   1 +
 tests/qemu-iotests/check |  22 +--
 tests/qemu-iotests/group | 297 ---------------------------------------
 272 files changed, 311 insertions(+), 307 deletions(-)
 delete mode 100644 tests/qemu-iotests/group
docs/devel/testing.rst +29 -1
index 770a987ea4..fdf3a08fd4 100644
@@ -153,7 +153,7 @@ check-block
 -----------
 
 ``make check-block`` runs a subset of the block layer iotests (the tests that
-are in the "auto" group in ``tests/qemu-iotests/group``).
+are in the "auto" group).
 See the "QEMU iotests" section below for more information.
 
 GCC gcov support
@@ -267,6 +267,34 @@ another application on the host may have locked the file, possibly leading to a
 test failure.  If using such devices are explicitly desired, consider adding
 ``locking=off`` option to disable image locking.
 
+Test case groups
+----------------
+
+Test may belong to some groups, you may define it in the comment inside the
+test. By convention, test groups are listed in the second line of the test
+file, after "#!/..." line, like this:
+
+.. code::
+
+  #!/usr/bin/env python3
+  # group: auto quick
+  #
+  ...
+
+The following groups are defined:
+
+- quick : Tests in this group should finish within some few seconds.
+
+- img : Tests in this group can be used to excercise the qemu-img tool.
+
+- auto : Tests in this group are used during "make check" and should be
+  runnable in any case. That means they should run with every QEMU binary
+  (also non-x86), with every QEMU configuration (i.e. must not fail if
+  an optional feature is not compiled in - but reporting a "skip" is ok),
+  work at least with the qcow2 file format, work with all kind of host
+  filesystems and users (e.g. "nobody" or "root") and must not take too
+  much memory and disk space (since CI pipelines tend to fail otherwise).
+
 .. _docker-ref:
 
 Docker based tests
tests/qemu-iotests/001 +1 -0
index d87a535c33..67ff7e3291 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test simple read/write using plain bdrv_read/bdrv_write
 #
tests/qemu-iotests/002 +1 -0
index 1a0d411df5..5ce1647531 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test simple read/write using plain bdrv_pread/bdrv_pwrite
 #
tests/qemu-iotests/003 +1 -0
index 33eeade0de..03f902a83c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test simple read/write using bdrv_aio_readv/bdrv_aio_writev
 #
tests/qemu-iotests/004 +1 -0
index d308dc4b49..e955579a67 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Make sure we can't read and write outside of the image size.
 #
tests/qemu-iotests/005 +1 -0
index b6d03ac37d..40e64a9a8f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: img auto quick
 #
 # Make sure qemu-img can create 5TB images
 #
tests/qemu-iotests/007 +1 -0
index 160683adf8..936d3f14fb 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: snapshot auto
 #
 # Check for one possible case of qcow2 refcount corruption.
 #
tests/qemu-iotests/008 +1 -0
index 2b81b119bf..fa4990b513 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test simple asynchronous read/write operations.
 #
tests/qemu-iotests/009 +1 -0
index 4dc7d210f9..efa852bad3 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Nolan I qcow2 corruption - incorrectly reports free clusters
 #
tests/qemu-iotests/010 +1 -0
index df809b3088..4ae9027b47 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Nolan II qcow2 corruption - wrong used cluster
 #
tests/qemu-iotests/011 +1 -0
index 57b99ae4a9..5c99ac987f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test for AIO allocation on the same cluster
 #
tests/qemu-iotests/012 +1 -0
index 12957285b3..3a24d2ca8d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto quick
 #
 # Make sure we can open read-only images
 #
tests/qemu-iotests/013 +1 -0
index 5cb9032f16..d39d0cd88b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # qcow2 pattern test, empty and compressed image - 4k cluster patterns
 #
tests/qemu-iotests/014 +1 -0
index e1221c0fff..2d23469332 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # qcow2 pattern test, complex patterns including compression and snapshots
 # Using patterns for 4k cluster size.
tests/qemu-iotests/015 +1 -0
index 4d8effd0ae..40c23235a6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw snapshot
 #
 # Combined test to grow the refcount table and test snapshots.
 #
tests/qemu-iotests/017 +1 -0
index 0a4b854e65..3cab97ad94 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing auto quick
 #
 # Simple backing file reads
 #
tests/qemu-iotests/018 +1 -0
index c69ce09209..a946243450 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing auto quick
 #
 # Merge backing file into test image when converting the image
 #
tests/qemu-iotests/019 +1 -0
index 813a84acac..a1a2065f6a 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing auto quick
 #
 # When using a backing file for the output image in qemu-img convert,
 # the backing file clusters must not copied. The data must still be
tests/qemu-iotests/020 +1 -0
index 20f8f185d0..a4ae2e1017 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing auto quick
 #
 # Commit changes to backing file
 #
tests/qemu-iotests/021 +1 -0
index f888269fd4..0fc89df2fe 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: io auto quick
 #
 # Test handling of invalid patterns arguments to qemu-io
 #
tests/qemu-iotests/022 +1 -0
index 99eb08f57f..a116cfe255 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw snapshot auto
 #
 # Test bdrv_load/save_vmstate using the usual patterns
 #
tests/qemu-iotests/023 +1 -0
index 02ed047820..d19d13ff5d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # qcow2 pattern test with various cluster sizes
 #
tests/qemu-iotests/024 +1 -0
index e2e766241e..de70aeac1c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing auto quick
 #
 # Rebasing COW images
 #
tests/qemu-iotests/025 +1 -0
index e05d833452..0047969d17 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Resizing images
 #
tests/qemu-iotests/026 +1 -0
index b05a4692cf..825e6b5210 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw blkdbg
 #
 # qcow2 error path testing
 #
tests/qemu-iotests/027 +1 -0
index 494be0921f..b279c88f33 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test that sub-cluster allocating writes zero the rest of the cluster
 #
tests/qemu-iotests/028 +1 -0
index 797dae5350..665889297f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing quick
 #
 # Test that backing files can be smaller than the image
 #
tests/qemu-iotests/029 +1 -0
index 2161a4b87a..3a20d2d1ff 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # qcow2 internal snapshots/VM state tests
 #
tests/qemu-iotests/030 +1 -0
index aa911d266a..a9066d4947 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw auto backing
 #
 # Tests for image streaming.
 #
tests/qemu-iotests/031 +1 -0
index 646ecd593f..af167e85be 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test that all qcow2 header extensions survive a header rewrite
 #
tests/qemu-iotests/032 +1 -0
index 8337a4d825..ebbe7cb0ba 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test that AIO requests are drained before an image is closed. This used
 # to segfault because the request coroutine kept running even after the
tests/qemu-iotests/033 +1 -0
index 8b40991d55..da9133c44b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test aligned and misaligned write zeroes operations.
 #
tests/qemu-iotests/034 +1 -0
index da4cea1571..e5d5996aa0 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test bdrv_pwrite_zeroes with backing files (see also 154)
 #
tests/qemu-iotests/035 +1 -0
index d950a0dd1e..0c0c4fdd42 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Let a few AIO requests run in parallel and have them access different L2
 # tables so that the cache has a chance to get used up.
tests/qemu-iotests/036 +1 -0
index 512598421c..5daa745b82 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qcow2 feature bits
 #
tests/qemu-iotests/037 +1 -0
index e6517acbd4..9bcc7695a6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test COW from backing files
 #
tests/qemu-iotests/038 +1 -0
index 707e2d72e9..63dff13827 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test COW from backing files with AIO
 #
tests/qemu-iotests/039 +1 -0
index ddce48ab47..3cddb8a720 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qcow2 lazy refcounts
 #
tests/qemu-iotests/040 +1 -0
index 32c82b4ec6..1a8bd3f613 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw auto
 #
 # Tests for image block commit.
 #
tests/qemu-iotests/041 +1 -0
index 5d67bf14bf..b6a14dc1b2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw auto backing
 #
 # Tests for image mirroring.
 #
tests/qemu-iotests/042 +1 -0
index 0e85b356b7..6cf823fa74 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qemu-img operation on zero size images
 #
tests/qemu-iotests/043 +1 -0
index b102e49208..83b97cccaf 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing
 #
 # Test that qemu-img info --backing-chain detects infinite loops
 #
tests/qemu-iotests/044 +1 -0
index 7e99ea7c68..64b18eb7c8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Tests growing a large refcount table.
 #
tests/qemu-iotests/045 +1 -0
index 5acc89099c..45eb239baa 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests for fdsets and getfd.
 #
tests/qemu-iotests/046 +1 -0
index a066eec605..969008ed20 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto aio quick
 #
 # Test concurrent cluster allocations
 #
tests/qemu-iotests/047 +1 -0
index 4528465fb0..8dd21e0a81 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Regression test for commit b7ab0fea (which was a corruption fix,
 # despite the commit message claiming otherwise)
tests/qemu-iotests/048 +1 -0
index 2af6b74b41..bf8e4bf528 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: img auto quick
 ##
 ## qemu-img compare test
 ##
tests/qemu-iotests/049 +1 -0
index 051a1c79e0..ed6730c4e8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Check qemu-img option parsing
 #
tests/qemu-iotests/050 +1 -0
index cdc5356541..62cc29f756 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test qemu-img rebase with zero clusters
 #
tests/qemu-iotests/051 +1 -0
index 034d3a3250..833aeda9f7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test command line configuration of block devices and driver-specific options
 #
tests/qemu-iotests/052 +1 -0
index 45a140910d..44c7895b80 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test bdrv_read/bdrv_write using BDRV_O_SNAPSHOT
 #
tests/qemu-iotests/053 +1 -0
index 71d299c4f9..9a2958d42d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qemu-img convert when image length is not a multiple of cluster size
 #
tests/qemu-iotests/054 +1 -0
index a8905b60d0..a796f565e0 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test huge qcow2 images
 #
tests/qemu-iotests/055 +1 -0
index 82b9f5f47d..c70664dac6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Tests for drive-backup and blockdev-backup
 #
tests/qemu-iotests/056 +1 -0
index f73fc74457..ffb2077a03 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw backing
 #
 # Tests for drive-backup
 #
tests/qemu-iotests/057 +1 -0
index a8b4bb60e0..b0d431999e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Tests for internal snapshot.
 #
tests/qemu-iotests/058 +1 -0
index d84740ed9f..ce35ff4ee0 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test export internal snapshot by qemu-nbd, convert it by qemu-img.
 #
tests/qemu-iotests/059 +1 -0
index 5438025285..33f5450edd 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for vmdk
 #
tests/qemu-iotests/060 +1 -0
index 043f12904a..4ec86931e1 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for image corruption (overlapping data structures) in qcow2
 #
tests/qemu-iotests/061 +1 -0
index 36b040491f..67355510d8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test case for image option amendment in qcow2.
 #
tests/qemu-iotests/062 +1 -0
index f26b88df9d..321252298d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for snapshotting images with unallocated zero clusters in
 # qcow2
tests/qemu-iotests/063 +1 -0
index c750b3806e..3a44758053 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # test of qemu-img convert -n - convert without creation
 #
tests/qemu-iotests/064 +1 -0
index 90673186ec..71fc575b21 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test VHDX read/write from a sample image created with Hyper-V
 #
tests/qemu-iotests/065 +1 -0
index 6426474271..2d76287d44 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test for additional information emitted by qemu-img info on qcow2
 # images
tests/qemu-iotests/066 +1 -0
index a4ac613f8e..a780ed7ab5 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for preallocated zero clusters in qcow2
 #
tests/qemu-iotests/067 +1 -0
index a63be9cabf..b23f8c65de 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test automatic deletion of BDSes created by -drive/drive_add
 #
tests/qemu-iotests/068 +1 -0
index ccd1a9f1db..03e03508a6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for loading a saved VM state from a qcow2 image
 #
tests/qemu-iotests/069 +1 -0
index b997b127f0..ccb24bc948 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for deleting a backing file
 #
tests/qemu-iotests/070 +1 -0
index cb0f927c16..b181e00f9b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test VHDX log replay from an image with a journal that needs to be
 # replayed
tests/qemu-iotests/071 +1 -0
index 88faebcc1d..1d92f7f4a6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for the QMP blkdebug and blkverify interfaces
 #
tests/qemu-iotests/072 +1 -0
index f0b73e7e65..c492ab8a78 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for nested image formats
 #
tests/qemu-iotests/073 +1 -0
index 23a1bdf890..ccdd465bbf 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test count_contiguous_clusters in qcow2
 #
tests/qemu-iotests/074 +1 -0
index db03edf0b0..c32c94b50d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 ##
 ## qemu-img compare test (qcow2 only ones)
 ##
tests/qemu-iotests/075 +1 -0
index 389d5675fa..ca2ed2a05c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # cloop format input validation tests
 #
tests/qemu-iotests/076 +1 -0
index 0d405ef3f2..1a8927d765 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: io
 #
 # parallels format input validation tests
 #
tests/qemu-iotests/077 +1 -0
index c284952082..fbb90d8036 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test concurrent pread/pwrite
 #
tests/qemu-iotests/078 +1 -0
index 54fc654d8e..0b48b7f137 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # bochs format input validation tests
 #
tests/qemu-iotests/079 +1 -0
index 3642b51feb..e00a3cda2f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test qcow2 preallocation with different cluster_sizes
 #
tests/qemu-iotests/080 +1 -0
index a3d13c414e..e0bdef11f4 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # qcow2 format input validation tests
 #
tests/qemu-iotests/081 +1 -0
index 537d40dfd5..a8185036cb 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test Quorum block driver
 #
tests/qemu-iotests/082 +1 -0
index 3286c2c6db..2259c9d9f7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test qemu-img command line parsing
 #
tests/qemu-iotests/083 +1 -0
index 10fdfc8ebb..bc32b537b2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test NBD client unexpected disconnect
 #
tests/qemu-iotests/084 +1 -0
index c29d7395e9..e51e91a7c8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: img quick
 #
 # Test case for VDI header corruption; image too large, and too many blocks.
 # Also simple test for creating dynamic and static VDI images.
tests/qemu-iotests/085 +1 -0
index 46981dbb64..3f098d4e07 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Live snapshot tests
 #
tests/qemu-iotests/086 +1 -0
index fea1a7bd8a..c055e7bfe1 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qemu-img progress output
 #
tests/qemu-iotests/087 +1 -0
index d6c8613419..4df975d0cd 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test unsupported blockdev-add cases
 #
tests/qemu-iotests/088 +1 -0
index ef1163346c..e3102fe888 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # vpc (VHD) format input validation tests
 #
tests/qemu-iotests/089 +1 -0
index 059ad75e28..1041cde0a1 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for support of JSON filenames
 #
tests/qemu-iotests/090 +1 -0
index 1246e4f910..cda38eeef2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test for discarding compressed clusters on qcow2 images
 #
tests/qemu-iotests/091 +1 -0
index d2a2aca347..988ea12b39 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw migration
 #
 # Live migration test
 #
tests/qemu-iotests/092 +1 -0
index 40ec62b6f1..bfa116d191 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # qcow1 format input validation tests
 #
tests/qemu-iotests/093 +1 -0
index 32ded11430..7745cb04b6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: throttle
 #
 # Tests for IO throttling
 #
tests/qemu-iotests/094 +1 -0
index 2d3e1004d3..a295fb20ef 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for drive-mirror to NBD
 #
tests/qemu-iotests/095 +1 -0
index 155ae86aa7..24c8ca2672 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test for commit of larger active layer
 #
tests/qemu-iotests/096 +1 -0
index 5915f92786..b5d7636bdc 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test that snapshots move the throttling configuration to the active
 # layer
tests/qemu-iotests/097 +1 -0
index 690f3d3ce1..2a95ee0c0f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing
 #
 # Commit changes into backing chains and empty the top image if the
 # backing image is not explicitly specified
tests/qemu-iotests/098 +1 -0
index 1e29d96b3d..2b029a41c8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test qcow2's bdrv_make_empty for images without internal snapshots
 #
tests/qemu-iotests/099 +1 -0
index 65e8e92572..2f1199ce04 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test valid filenames for blkdebug and blkverify representatively for
 # other protocols (such as NBD) when queried
tests/qemu-iotests/101 +1 -0
index a4c1b6366a..4c4a8cea11 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test short file I/O
 #
tests/qemu-iotests/102 +1 -0
index b898df436f..0dd3cee426 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for qemu-io -c map and qemu-img map
 #
tests/qemu-iotests/103 +1 -0
index 8c1ebe0443..5fee8a0119 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for qcow2 metadata cache size specification
 #
tests/qemu-iotests/104 +1 -0
index c70f28a9a1..3ebb74cf6e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test image creation with aligned and unaligned sizes
 #
tests/qemu-iotests/105 +1 -0
index 4d55a2d3ef..d804685110 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Create, read, write big image
 #
tests/qemu-iotests/106 +1 -0
index b5d1ec4078..9e833a105c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test preallocated resize of raw images
 #
tests/qemu-iotests/107 +1 -0
index 268ba27688..86dc27343e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Tests updates of the qcow2 L1 table
 #
tests/qemu-iotests/108 +1 -0
index 5f7076fba4..6b1ddbb5b6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for repairing qcow2 images which cannot be repaired using
 # the on-disk refcount structures
tests/qemu-iotests/109 +1 -0
index a51dd84b3d..cbdaab88dd 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test writing image headers of other formats into raw images
 #
tests/qemu-iotests/110 +1 -0
index 139c02c2cf..4e2a3ce04d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test case for relative backing file names in complex BDS trees
 #
tests/qemu-iotests/111 +1 -0
index 3b43d1bd83..1a3d1ee741 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for non-existing backing file when creating a qcow2 image
 # and not specifying the size
tests/qemu-iotests/112 +1 -0
index 20ff5c224a..7a5b859c81 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test cases for different refcount_bits values
 #
tests/qemu-iotests/113 +1 -0
index f2703a2c50..5dc92b49ca 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for accessing creation options on image formats and
 # protocols not supporting image creation
tests/qemu-iotests/114 +1 -0
index 26104fff6c..a3a0e2c807 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test invalid backing file format in qcow2 images
 #
tests/qemu-iotests/115 +1 -0
index d254b18342..5c3f60bc1b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test case for non-self-referential qcow2 refcount blocks
 #
tests/qemu-iotests/116 +1 -0
index 941b07a1a9..4f40fcb3d2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test error code paths for invalid QED images
 #
tests/qemu-iotests/117 +1 -0
index f37b34f8b1..8210a0f70d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test case for shared BDS between backend trees
 #
tests/qemu-iotests/118 +1 -0
index adc8a848b5..d705a01b78 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test case for the QMP 'change' command and all other associated
 # commands
tests/qemu-iotests/119 +1 -0
index ea6770a484..5770b50045 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # NBD test case for overriding BDRV_O_PROTOCOL by explicitly specifying
 # a driver
tests/qemu-iotests/120 +1 -0
index 2931a7550f..03d5a3ba33 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Non-NBD test cases for overriding BDRV_O_PROTOCOL by explicitly
 # specifying a driver
tests/qemu-iotests/121 +1 -0
index 90ea0db737..0abe27f9f7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test cases for qcow2 refcount table growth
 #
tests/qemu-iotests/122 +1 -0
index f7a3ae684a..5faf9ab034 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test some qemu-img convert cases
 #
tests/qemu-iotests/123 +1 -0
index 01b771c76e..e19111f70d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for qemu-img convert to NBD
 #
tests/qemu-iotests/124 +1 -0
index 3705cbb6b3..3b21bc497f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw backing
 #
 # Tests for incremental drive-backup
 #
tests/qemu-iotests/125 +1 -0
index d510984045..55e45657a6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test preallocated growth of qcow2 images
 #
tests/qemu-iotests/126 +1 -0
index b7fce1e59d..7e4d1b3345 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing
 #
 # Tests handling of colons in filenames (which may be confused with protocol
 # prefixes)
tests/qemu-iotests/127 +1 -0
index a4fc866038..b30a49579f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Test case for mirroring with dataplane
 #
tests/qemu-iotests/128 +1 -0
index 3606c41760..d0e00d24b1 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test that opening O_DIRECT succeeds when image file I/O produces EIO
 #
tests/qemu-iotests/129 +1 -0
index b0da4a5541..0873b4800d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests that "bdrv_drain_all" doesn't drain block jobs
 #
tests/qemu-iotests/130 +1 -0
index 77ad2aa13a..9d50960d66 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test that temporary backing file overrides (on the command line or in
 # blockdev-add) don't replace the original path stored in the image during
tests/qemu-iotests/131 +1 -0
index 27870231cf..d7456cae5b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # parallels format validation tests (created by QEMU)
 #
tests/qemu-iotests/132 +1 -0
index 39ea43067e..367ea08036 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test mirror with unmap
 #
tests/qemu-iotests/133 +1 -0
index 6f7cacc091..602c4b7a96 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto quick
 #
 # Test for reopen
 #
tests/qemu-iotests/134 +1 -0
index 5f0fb86211..30cc53088e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test encrypted read/write using plain bdrv_read/bdrv_write
 #
tests/qemu-iotests/135 +1 -0
index 3b3d1dc2a5..299075b4c9 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test VPC open of image with large Max Table Entries value.
 #
tests/qemu-iotests/136 +1 -0
index d59400c9fc..8fce88bd67 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Tests for block device statistics
 #
tests/qemu-iotests/137 +1 -0
index 7ae86892f7..1cb2fe1653 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test qcow2 reopen
 #
tests/qemu-iotests/138 +1 -0
index 54b01046ad..b99a9bddf6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # General test case for qcow2's image check
 #
tests/qemu-iotests/139 +1 -0
index 7120d3142b..401e8a915d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test cases for the QMP 'blockdev-del' command
 #
tests/qemu-iotests/140 +1 -0
index 8d2ce5d9e3..13bc4ca2d7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for ejecting a BlockBackend with an NBD server attached to it
 #
tests/qemu-iotests/141 +1 -0
index 5192d256e3..38736e16f2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for ejecting BDSs with block jobs still running on them
 #
tests/qemu-iotests/143 +1 -0
index f649b36195..8e2009df26 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto quick
 #
 # Test case for connecting to a non-existing NBD export name
 #
tests/qemu-iotests/144 +1 -0
index 4569ac0b4b..60e9ddd75f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 # Check live snapshot, followed by active commit, and another snapshot.
 #
 # This test is to catch the error case of BZ #1300209:
tests/qemu-iotests/145 +1 -0
index 9427549651..a2ce92516d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test the combination of -incoming and snapshot=on
 #
tests/qemu-iotests/146 +1 -0
index 2e43abddfc..8bc9bbefa3 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test VHD image format creator detection and override
 #
tests/qemu-iotests/147 +1 -0
index d7a9f31089..47dfa62e6b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: img
 #
 # Test case for NBD's blockdev-add interface
 #
tests/qemu-iotests/148 +1 -0
index 90931948e3..35f3861dfc 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test the rate limit of QMP events
 #
tests/qemu-iotests/149 +1 -0
index b4a21bf7b7..e84f0f8e1f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw sudo
 #
 # Copyright (C) 2016 Red Hat, Inc.
 #
tests/qemu-iotests/150 +1 -0
index 3b1f32197a..ac6930ae20 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test that qemu-img convert -S 0 fully allocates the target image
 #
tests/qemu-iotests/151 +1 -0
index f2df72c29c..182f6b5321 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Tests for active mirroring
 #
tests/qemu-iotests/152 +1 -0
index cc2ea09654..4e179c340f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests for drive-mirror with source size unaligned to granularity
 #
tests/qemu-iotests/153 +1 -0
index 2b13111768..ef6a3cfd18 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test image locking
 #
tests/qemu-iotests/154 +1 -0
index d68f66b9e0..83900cc072 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # qcow2 specific bdrv_pwrite_zeroes tests with backing files (complements 034)
 #
tests/qemu-iotests/155 +1 -0
index 571bce9de4..4ac3f86a8b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test whether the backing BDSs are correct after completion of a
 # mirror block job; in "existing" modes (drive-mirror with
tests/qemu-iotests/156 +1 -0
index 5559df63a5..4019ada313 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Tests oVirt-like storage migration:
 #  - Create snapshot
tests/qemu-iotests/157 +1 -0
index 7cbac38099..0dc9ba68d2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test command line configuration of block devices with qdev
 #
tests/qemu-iotests/158 +1 -0
index ba4db6116a..6dcd14aeb2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test encrypted read/write using backing files
 #
tests/qemu-iotests/159 +1 -0
index f9690053a2..4eb476d3a8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # qemu-img dd test with different block sizes
 #
tests/qemu-iotests/160 +1 -0
index 0572b5ae9a..7984a9c6f7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # qemu-img dd test for the skip option
 #
tests/qemu-iotests/161 +1 -0
index f572a19af2..e161fb4679 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test reopening a backing image after block-stream and block-commit
 #
tests/qemu-iotests/162 +1 -0
index c0053ed975..cf17f494d8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test case for specifying runtime options of the wrong type to some
 # block drivers
tests/qemu-iotests/163 +1 -0
index 5a3cc840a5..dedce8ef43 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Tests for shrinking images
 #
tests/qemu-iotests/165 +1 -0
index fb56a769b4..abc4ffadd5 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests for persistent dirty bitmaps.
 #
tests/qemu-iotests/169 +1 -0
index 2c5a132aa3..2d41e1cc03 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick migration
 #
 # Tests for dirty bitmaps migration.
 #
tests/qemu-iotests/170 +1 -0
index 6c8f0e8085..41387e4d66 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # qemu-img dd test
 #
tests/qemu-iotests/171 +1 -0
index 341064a1c6..babfe98753 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test 'offset' and 'size' options of the raw driver. Make sure we can't
 # (or can) read and write outside of the image size.
tests/qemu-iotests/172 +1 -0
index 7195fb895a..4548e3aab9 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto
 #
 # Test floppy configuration
 #
tests/qemu-iotests/173 +1 -0
index ec6d1705e5..9594f3c5ea 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test QAPI commands looking up protocol based images with relative
 # filename backing strings
tests/qemu-iotests/174 +1 -0
index e2f14a38c6..1a141fbc0f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto
 #
 # Test that qemu-io fail with non-zero exit code
 #
tests/qemu-iotests/175 +1 -0
index 020ed8e61f..9bb7ea6936 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test creating raw image preallocation mode
 #
tests/qemu-iotests/176 +1 -0
index 117c8b6954..e5be07d6a6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing
 #
 # Commit changes into backing chains and empty the top image if the
 # backing image is not explicitly specified.
tests/qemu-iotests/177 +1 -0
index 752d29f8ad..b609e0fdfb 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test corner cases with unusual block geometries
 #
tests/qemu-iotests/178 +1 -0
index 7cf0e27154..b911b79915 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: img
 #
 # qemu-img measure sub-command tests
 #
tests/qemu-iotests/179 +1 -0
index 9372dc30ef..ebd48e6ad2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test case for write zeroes with unmap
 #
tests/qemu-iotests/181 +1 -0
index 438c2dcd80..dadf3bf88e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto migration
 #
 # Test postcopy live migration with shared storage
 #
tests/qemu-iotests/182 +1 -0
index 56a2dd58e6..55a0384c08 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test image locking for POSIX locks
 #
tests/qemu-iotests/183 +1 -0
index acdbefa310..16390fe93d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw migration
 #
 # Test old-style block migration (migrate -b)
 #
tests/qemu-iotests/184 +1 -0
index 33dd8d2a4f..1d923f509d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test I/O throttle block filter driver interface
 #
tests/qemu-iotests/185 +1 -0
index fd5e6ebe11..7bc8fe5767 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test exiting qemu while jobs are still running
 #
tests/qemu-iotests/186 +1 -0
index 3ea0442d44..66f2418182 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test 'info block' with all kinds of configurations
 #
tests/qemu-iotests/187 +1 -0
index c6e1dc57a0..46fd5678c7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test switching between read-only and read-write
 #
tests/qemu-iotests/188 +1 -0
index afca44df54..87e902352d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test encrypted read/write using plain bdrv_read/bdrv_write
 #
tests/qemu-iotests/189 +1 -0
index c9ce9d3bed..2f6f8a9309 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test encrypted read/write using backing files
 #
tests/qemu-iotests/190 +1 -0
index 6d41650438..6011be0861 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # qemu-img measure sub-command tests on huge qcow2 files
 #
tests/qemu-iotests/191 +1 -0
index b05db68141..bce1d8445e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test commit block job where top has two parents
 #
tests/qemu-iotests/192 +1 -0
index d2ba55dd90..d809187fca 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test NBD export with -incoming (non-shared storage migration use case from
 # libvirt)
tests/qemu-iotests/194 +1 -0
index 9dc1bd3510..b668120ebb 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw migration quick
 #
 # Copyright (C) 2017 Red Hat, Inc.
 #
tests/qemu-iotests/195 +1 -0
index 48984b7ac1..dab6e0fe9c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test change-backing-file command
 #
tests/qemu-iotests/196 +1 -0
index e8fcf37273..2451515094 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick migration
 #
 # Test clearing unknown autoclear_features flag by qcow2 after
 # migration. This test mimics migration to older qemu.
tests/qemu-iotests/197 +1 -0
index 95f05b0e34..74e751d06f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for copy-on-read into qcow2
 #
tests/qemu-iotests/198 +1 -0
index fb0d5a29d3..b4c498015f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test commit of encrypted qcow2 files
 #
tests/qemu-iotests/199 +1 -0
index 40774eed74..9cb8c55956 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw migration
 #
 # Tests for dirty bitmaps postcopy migration.
 #
tests/qemu-iotests/200 +1 -0
index 59f7854b9f..c72cd72e56 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Block job co-routine race condition test.
 #
tests/qemu-iotests/201 +1 -0
index 483eb189c5..412a6f1834 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw migration
 #
 # Test savevm and loadvm after live migration with postcopy flag
 #
tests/qemu-iotests/202 +1 -0
index 920a8683ef..5f8afc85bd 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Copyright (C) 2017 Red Hat, Inc.
 #
tests/qemu-iotests/203 +1 -0
index 49eff5d405..3e2e35ad6b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw auto migration
 #
 # Copyright (C) 2017 Red Hat, Inc.
 #
tests/qemu-iotests/204 +1 -0
index abb73dc381..e528b796f6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test corner cases with unusual block geometries
 #
tests/qemu-iotests/205 +1 -0
index 43432cb599..c0e107328f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests for qmp command nbd-server-remove.
 #
tests/qemu-iotests/206 +1 -0
index e2b50ae24d..4cf41eca7f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test qcow2 and file image creation
 #
tests/qemu-iotests/207 +1 -0
index 3d9c1208ca..813cfe7a27 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test ssh image creation
 #
tests/qemu-iotests/208 +1 -0
index 1c3fc8c7fd..5ef7bc3caa 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Copyright (C) 2018 Red Hat, Inc.
 #
tests/qemu-iotests/209 +1 -0
index 65c1a1e70a..2dd0bbb905 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests for NBD BLOCK_STATUS extension
 #
tests/qemu-iotests/210 +1 -0
index e49896e23d..f03c13c186 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test luks and file image creation
 #
tests/qemu-iotests/211 +1 -0
index 163994d559..e66737970f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test VDI and file image creation
 #
tests/qemu-iotests/212 +1 -0
index 800f92dd84..b7a53949f8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test parallels and file image creation
 #
tests/qemu-iotests/213 +1 -0
index 1eee45276a..c7b5c0e7e2 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test vhdx and file image creation
 #
tests/qemu-iotests/214 +1 -0
index af677d90b8..4c0e8f2b48 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # Test qcow2 image compression
 #
tests/qemu-iotests/215 +1 -0
index f99bae78c7..d464596f14 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for copy-on-read into qcow2, using the COR filter driver
 #
tests/qemu-iotests/216 +1 -0
index 372f042d3e..7374bede19 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Copy-on-read tests using a COR filter node
 #
tests/qemu-iotests/217 +1 -0
index d89116ccad..df370549f8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # I/O errors when working with internal qcow2 snapshots, and repairing
 # the result
tests/qemu-iotests/218 +1 -0
index 1325ba9eaa..c8cca23161 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # This test covers what happens when a mirror block job is cancelled
 # in various phases of its existence.
tests/qemu-iotests/219 +1 -0
index b8774770c4..9523627451 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Copyright (C) 2018 Red Hat, Inc.
 #
tests/qemu-iotests/220 +1 -0
index a9259b7127..126749c707 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto
 #
 # max limits on compression in huge qcow2 files
 #
tests/qemu-iotests/221 +1 -0
index 0e9096fec7..2db84a3962 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test qemu-img vs. unaligned images
 # (See also 253, which is the O_DIRECT version)
tests/qemu-iotests/222 +1 -0
index bf1718e179..f1ee316abb 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # This test covers the basic fleecing workflow, which provides a
 # point-in-time snapshot of a node that can be queried over NBD.
tests/qemu-iotests/223 +1 -0
index 56fbc5fb09..18ff35ac3a 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test reading dirty bitmap over NBD
 #
tests/qemu-iotests/224 +1 -0
index e91fb26fd8..40e6beaa07 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test json:{} filenames with qemu-internal BDSs
 # (the one of commit, to be precise)
tests/qemu-iotests/225 +1 -0
index c9a334c7e9..c25ac26949 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test vmdk backing file correlation
 #
tests/qemu-iotests/226 +1 -0
index c1e1fb2b1c..6a9adb4a0b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto quick
 #
 # This test covers expected filetypes for the file, host_cdrom and
 # host_device drivers.
tests/qemu-iotests/227 +1 -0
index 637d7c3726..7e45a47ac6 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test query-blockstats with different ways to create a BB
 #
tests/qemu-iotests/228 +1 -0
index 64bc82ee23..5e723d342c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test for when a backing file is considered overridden (thus, a
 # json:{} filename is generated for the overlay) and when it is not
tests/qemu-iotests/229 +1 -0
index 866168b236..9b8ecee2a7 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: auto quick
 #
 # Test for force canceling a running blockjob that is paused in
 # an error state.
tests/qemu-iotests/231 +1 -0
index c0b053ac30..0f66d0ca36 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test legacy and modern option parsing for rbd/ceph.  This will not
 # actually connect to a ceph server, but rather looks for the appropriate
tests/qemu-iotests/232 +1 -0
index 685356ac3b..b30faaa218 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test for auto-read-only
 #
tests/qemu-iotests/233 +1 -0
index a5c17c3963..7ce5764903 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test NBD TLS certificate / authorization integration
 #
tests/qemu-iotests/234 +1 -0
index 324c1549fd..e443fc59cc 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: quick migration
 #
 # Copyright (C) 2018 Red Hat, Inc.
 #
tests/qemu-iotests/235 +1 -0
index 760826128e..7517f78626 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: quick
 #
 # Simple mirror test
 #
tests/qemu-iotests/236 +1 -0
index 8ce927a16c..75b63ad4cf 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: quick
 #
 # Test bitmap merges.
 #
tests/qemu-iotests/237 +1 -0
index 50ba364a3e..4068298f8c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test vmdk and file image creation
 #
tests/qemu-iotests/238 +1 -0
index d4e060228c..662ba94131 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: quick
 #
 # Regression test for throttle group member unregister segfault with iothread
 #
tests/qemu-iotests/239 +1 -0
index b0991ffe59..4f0037148d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test case for dmg
 #
tests/qemu-iotests/240 +1 -0
index 8b4337b58d..d9428e50bc 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # Test hot plugging and unplugging with iothreads
 #
tests/qemu-iotests/241 +1 -0
index 8dae8d39e4..c962c8b607 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test qemu-nbd vs. unaligned images
 #
tests/qemu-iotests/242 +1 -0
index 97617876bc..757ac569bc 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test for qcow2 bitmap printed information
 #
tests/qemu-iotests/243 +1 -0
index a61852f6d9..160ea02538 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test qcow2 preallocation
 #
tests/qemu-iotests/244 +1 -0
index 2ec1815e6f..037a961422 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qcow2 with external data files
 #
tests/qemu-iotests/245 +1 -0
index 1001275a44..9fc1e3e0c8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test cases for the QMP 'x-blockdev-reopen' command
 #
tests/qemu-iotests/246 +1 -0
index 59a216a839..71a17a024d 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test persistent bitmap resizing.
 #
tests/qemu-iotests/247 +1 -0
index 87e37b39e2..70b9a42a33 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test for auto-read-only with commit block job
 #
tests/qemu-iotests/248 +1 -0
index 68c374692e..a4af7b99df 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test resume mirror after auto pause on ENOSPC
 #
tests/qemu-iotests/249 +1 -0
index 2b99c9789e..623acdb97c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test that a backing image is put back in read-only mode after
 # block-commit (both when it fails and when it succeeds).
tests/qemu-iotests/250 +1 -0
index 9bb6b94d74..b9ee43214a 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test big discard in qcow2 shrink
 #
tests/qemu-iotests/251 +1 -0
index 7918ba3559..b0900898fa 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test qemu-img convert --salvage
 #
tests/qemu-iotests/252 +1 -0
index 83280c1715..a6df22ff3f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto backing quick
 #
 # Tests for rebasing COW images that require zero cluster support
 #
tests/qemu-iotests/253 +1 -0
index d88d5afa45..35039d20a8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test qemu-img vs. unaligned images; O_DIRECT version
 # (Originates from 221)
tests/qemu-iotests/254 +1 -0
index ee66c986db..f4965d343b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw backing quick
 #
 # Test external snapshot with bitmap copying and moving.
 #
tests/qemu-iotests/255 +1 -0
index 4a4818bafb..2cb2a98f7c 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test commit job graph modifications while requests are active
 #
tests/qemu-iotests/256 +1 -0
index e34074c83e..a34f7f7785 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw auto quick
 #
 # Test incremental/backup across iothread contexts
 #
tests/qemu-iotests/257 +1 -0
index 004a433b8b..ce1918fa37 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test bitmap-sync backups (incremental, differential, and partials)
 #
tests/qemu-iotests/258 +1 -0
index 091755a45c..d09174912b 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Very specific tests for adjacent commit/stream block jobs
 #
tests/qemu-iotests/259 +1 -0
index 62e29af05f..76cde429c4 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test generic image creation fallback (by using NBD)
 #
tests/qemu-iotests/260 +1 -0
index 30c0de380d..f2de0dbf87 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Tests for temporary external snapshot when we have bitmaps.
 #
tests/qemu-iotests/261 +1 -0
index ddcb04f285..e8e61f8d6e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test case for qcow2's handling of extra data in snapshot table entries
 # (and more generally, how certain cases of broken snapshot tables are
tests/qemu-iotests/262 +1 -0
index 8835dce7be..6a9925bcac 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick migration
 #
 # Copyright (C) 2019 Red Hat, Inc.
 #
tests/qemu-iotests/263 +1 -0
index d2c030fae9..54d009ce24 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test encrypted write that crosses cluster boundary of two unallocated clusters
 # Based on 188
tests/qemu-iotests/264 +1 -0
index 879123a343..9ef7fe31de 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw
 #
 # Test nbd reconnect
 #
tests/qemu-iotests/265 +1 -0
index 00f2ec769e..f796361903 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test reverse-ordered qcow2 writes on a sub-cluster level
 #
tests/qemu-iotests/266 +1 -0
index 91bdf8729e..71ce81d0df 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test VPC and file image creation
 #
tests/qemu-iotests/267 +1 -0
index 3146273eef..4fd18fefdb 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick snapshot
 #
 # Test which nodes are involved in internal snapshots
 #
tests/qemu-iotests/268 +1 -0
index 78c3f4db3a..4fce2ef75f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw auto quick
 #
 # Test write request with required alignment larger than the cluster size
 #
tests/qemu-iotests/270 +1 -0
index b9a12b908c..f49894149f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing quick
 #
 # Test large write to a qcow2 image
 #
tests/qemu-iotests/272 +1 -0
index c2f782d47b..015347a0c9 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test compressed write to a qcow2 image at an offset above 4 GB
 #
tests/qemu-iotests/273 +1 -0
index 00ff79bcf8..e537c0be62 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: backing quick
 #
 # Test multiple blockdev-snapshot calls with 'backing': null
 #
tests/qemu-iotests/277 +1 -0
index 04aa15a3d5..5b5dbd9283 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test NBD client reconnection
 #
tests/qemu-iotests/279 +1 -0
index 30d29b1cb2..14378d99c0 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw backing quick
 #
 # Test qemu-img --backing-chain --image-opts
 #
tests/qemu-iotests/280 +1 -0
index 69288fdd0e..5a3fca906e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw migration quick
 #
 # Copyright (C) 2019 Red Hat, Inc.
 #
tests/qemu-iotests/281 +1 -0
index 0bf973bca6..956698083f 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: rw quick
 #
 # Test cases for blockdev + IOThread interactions
 #
tests/qemu-iotests/282 +1 -0
index 081eb12080..dfc61a65a8 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw img quick
 #
 # Test qemu-img file cleanup for LUKS when using a non-UTF8 secret
 #
tests/qemu-iotests/283 +1 -0
index 55b7cff953..e0eefa685c 100644
@@ -1,4 +1,5 @@
 #!/usr/bin/env python3
+# group: auto quick
 #
 # Test for backup-top filter permission activation failure
 #
tests/qemu-iotests/284 +1 -0
index 071e89b33e..d0e46d064e 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw
 #
 # Test ref count checks on encrypted images
 #
tests/qemu-iotests/286 +1 -0
index f14445ba4a..1fd24d5f74 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: rw quick
 #
 # Test qemu-img snapshot -l
 #
tests/qemu-iotests/288 +1 -0
index 6c62065aef..47aca6592a 100755
@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# group: quick
 #
 # qemu-img measure tests for LUKS images
 #
tests/qemu-iotests/check +13 -9
index f7a2d3d6c3..bbdf22f2a1 100755
@@ -155,6 +155,14 @@ export VALGRIND_QEMU=
 export IMGKEYSECRET=
 export IMGOPTSSYNTAX=false
 
+all_tests() {
+    ls ??? | grep '^[0-9]\{3\}'
+}
+
+find_tests() {
+    all_tests | xargs grep -l "^# group: .*$1"
+}
+
 # Save current tty settings, since an aborting qemu call may leave things
 # screwed up
 STTY_RESTORE=
@@ -168,9 +176,7 @@ do
     if $group
     then
         # arg after -g
-        group_list=$(sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0-9][0-9][0-9].* $r /"'{
-s/ .*//p
-}')
+        group_list=$(find_tests "$r")
         if [ -z "$group_list" ]
         then
             echo "Group \"$r\" is empty or not defined?"
@@ -193,10 +199,8 @@ s/ .*//p
     then
         # arg after -x
         # Populate $tmp.list with all tests
-        awk '/^[0-9]{3,}/ {print $1}' "${source_iotests}/group" > $tmp.list 2>/dev/null
-        group_list=$(sed -n <"$source_iotests/group" -e 's/$/ /' -e "/^[0-9][0-9][0-9].* $r /"'{
-s/ .*//p
-}')
+        all_tests > $tmp.list 2>/dev/null
+        group_list=$(find_tests "$r")
         if [ -z "$group_list" ]
         then
             echo "Group \"$r\" is empty or not defined?"
@@ -504,7 +508,7 @@ testlist options
 BEGIN        { for (t='$start'; t<='$end'; t++) printf "%03d\n",t }' \
         | while read id
         do
-            if grep -s "^$id\( \|\$\)" "$source_iotests/group" >/dev/null
+            if test -f "$id";
             then
                 # in group file ... OK
                 echo $id >>$tmp.list
@@ -566,7 +570,7 @@ else
         touch $tmp.list
     else
         # no test numbers, do everything from group file
-        sed -n -e '/^[0-9][0-9][0-9]*/s/^\([0-9]*\).*/\1/p' <"$source_iotests/group" >$tmp.list
+        all_tests > $tmp.list
     fi
 fi
 
tests/qemu-iotests/group deleted +0 -297

297 lines not on this page · raw message

Signature · 1 line
2.21.0
#1 lore raw
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

  CC      s_normRoundPackToF64.o
  CC      s_addMagsF64.o
  CC      s_subMagsF64.o

make: *** [check-tests/check-block.sh] Error 1
make: *** Waiting for unfinished jobs....
CC s_mulAddF64.o
CC s_normSubnormalExtF80Sig.o
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=2b582229add741f387fcf0a88ff2a3ff', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-ot7uydwt/src/docker-src.2020-03-24-03.54.58.28935:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=2b582229add741f387fcf0a88ff2a3ff
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-ot7uydwt/src'
make: *** [docker-run-test-quick@centos7] Error 2

real 2m41.224s
user 0m7.941s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#2 lore raw
Quoted 35 lines · 24.03.2020 10:57, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

   CC      s_normRoundPackToF64.o
   CC      s_addMagsF64.o
   CC      s_subMagsF64.o

make: *** [check-tests/check-block.sh] Error 1
make: *** Waiting for unfinished jobs....
CC s_mulAddF64.o
CC s_normSubnormalExtF80Sig.o
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=2b582229add741f387fcf0a88ff2a3ff', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-ot7uydwt/src/docker-src.2020-03-24-03.54.58.28935:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=2b582229add741f387fcf0a88ff2a3ff
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-ot7uydwt/src'
make: *** [docker-run-test-quick@centos7] Error 2

real 2m41.224s
user 0m7.941s

Quoted 6 lines

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Doesn't look as something related to my patch.

Signature · 2 lines
Best regards,
Vladimir
#3 lore raw
In reply to Vladimir Sementsov-Ogievskiy

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 23 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

    grep '^[0-9]\{3\} ' group | while read line; do
        file=$(awk '{print $1}' <<< "$line");
        groups=$(sed -e 's/^... //' <<< "$line");
        awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
        cat tmp > $file;
    done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

Quoted 6 lines

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Downstream, I think we still disable a few tests because we're compiling out features that are required for some tests to pass, and deleting the test cases completely would probably move conflicts just to a different place.

So I think we need a method to distuingish an enabled test that is in no group from a disabled test.

Quoted 2 lines

check script is also refactored to make it simple to do next cool thing about iotests: allow meaningful names for test-case files.

This one would also require us to be able to distinguish test case files from arbitrary other files.

tests/qemu-iotests/group line 30

Quoted 9 lines
tests/qemu-iotests/group deleted +0 -9
-#
-# test-group association ... one line per test
-#
-001 rw auto quick
-002 rw auto quick
-003 rw auto
-004 rw auto quick
-005 img auto quick
-# 006 was removed, do not reuse

We lose these comments without a replacement. I wonder whether it's important or if we can think of another way to make sure that numbers aren't reused. (I'm not completely sure any more why we decided that we don't want to reuse numbers. Maybe because of backports?)

Kevin

#4 lore raw
Quoted 37 lines · Kevin Wolf

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 23 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

    grep '^[0-9]\{3\} ' group | while read line; do
        file=$(awk '{print $1}' <<< "$line");
        groups=$(sed -e 's/^... //' <<< "$line");
        awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
        cat tmp > $file;
    done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

Quoted 6 lines

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Quoted 7 lines

Downstream, I think we still disable a few tests because we're compiling out features that are required for some tests to pass, and deleting the test cases completely would probably move conflicts just to a different place.

So I think we need a method to distuingish an enabled test that is in no group from a disabled test.

Can we just have a file "blacklist.local" (which doesn't exist by default) where you list tests to skip locally ?

Or a "group.local" such that any info in this group.local file will replace the default info extracted from the test file ?

Quoted 5 lines
Quoted 2 lines

check script is also refactored to make it simple to do next cool thing about iotests: allow meaningful names for test-case files.

This one would also require us to be able to distinguish test case files from arbitrary other files.

A test-XXXXXXXXX.sh or a XXXXXX.test naming convention ?

Quoted 6 lines
Quoted 6 lines
tests/qemu-iotests/group deleted +0 -6
-#
-# test-group association ... one line per test
-#
-001 rw auto quick
-002 rw auto quick
-003 rw auto
Quoted 8 lines
Quoted 3 lines

-004 rw auto quick
-005 img auto quick
-# 006 was removed, do not reuse

We lose these comments without a replacement. I wonder whether it's important or if we can think of another way to make sure that numbers aren't reused. (I'm not completely sure any more why we decided that we don't want to reuse numbers. Maybe because of backports?)

The key goal of the patch is to avoid conflicts from clashing changes between branches. To full achieve that goal we need to both avoid touching the shared groups file, but more than than, we must avoid creating clashing test filenames.

If we keep using 3-digit filenames then the goal of this patch is not achieved, as the risk of clashes is still higher. IOW, we must switch to a more verbose naming convention to increase the entropy and thus eliminate risk of clashes. If this is done, then the idea of reserving previously used test names ceases to be something to worry about.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw
Quoted 81 lines · Daniel P. Berrangé
Quoted 37 lines · Kevin Wolf

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 23 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

     grep '^[0-9]\{3\} ' group | while read line; do
         file=$(awk '{print $1}' <<< "$line");
         groups=$(sed -e 's/^... //' <<< "$line");
         awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
         cat tmp > $file;
     done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

Quoted 6 lines

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Quoted 7 lines

Downstream, I think we still disable a few tests because we're compiling out features that are required for some tests to pass, and deleting the test cases completely would probably move conflicts just to a different place.

So I think we need a method to distuingish an enabled test that is in no group from a disabled test.

Can we just have a file "blacklist.local" (which doesn't exist by default) where you list tests to skip locally ?

Or a "group.local" such that any info in this group.local file will replace the default info extracted from the test file ?

Quoted 5 lines
Quoted 2 lines

check script is also refactored to make it simple to do next cool thing about iotests: allow meaningful names for test-case files.

This one would also require us to be able to distinguish test case files from arbitrary other files.

A test-XXXXXXXXX.sh or a XXXXXX.test naming convention ?

Quoted 6 lines
Quoted 6 lines
tests/qemu-iotests/group deleted +0 -6
-#
-# test-group association ... one line per test
-#
-001 rw auto quick
-002 rw auto quick
-003 rw auto
Quoted 8 lines
Quoted 3 lines

-004 rw auto quick
-005 img auto quick
-# 006 was removed, do not reuse

We lose these comments without a replacement. I wonder whether it's important or if we can think of another way to make sure that numbers aren't reused. (I'm not completely sure any more why we decided that we don't want to reuse numbers. Maybe because of backports?)

The key goal of the patch is to avoid conflicts from clashing changes between branches. To full achieve that goal we need to both avoid touching the shared groups file, but more than than, we must avoid creating clashing test filenames.

Quoted 6 lines

If we keep using 3-digit filenames then the goal of this patch is not achieved, as the risk of clashes is still higher. IOW, we must switch to a more verbose naming convention to increase the entropy and thus eliminate risk of clashes. If this is done, then the idea of reserving previously used test names ceases to be something to worry about.

Hm. You are right, better to solve all these problems together.

Signature · 2 lines
Best regards,
Vladimir
#6 lore raw
In reply to Kevin Wolf
Quoted 31 lines · Kevin Wolf

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 23 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

     grep '^[0-9]\{3\} ' group | while read line; do
         file=$(awk '{print $1}' <<< "$line");
         groups=$(sed -e 's/^... //' <<< "$line");
         awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
         cat tmp > $file;
     done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

Quoted 6 lines

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

Quoted 5 lines

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Hmm. Probably, you could add it to group "disabled", and run check -x disabled.

Quoted 5 lines

Downstream, I think we still disable a few tests because we're compiling out features that are required for some tests to pass, and deleting the test cases completely would probably move conflicts just to a different place.

I came to fixing all test in downstream, correctly skipping test-cases which needs compiled-out feature. We don't comment out tests in group, all tests should pass or be correctly skipped by check.

Quoted 3 lines

So I think we need a method to distuingish an enabled test that is in no group from a disabled test.

What about "disabled" group?

Quoted 6 lines
Quoted 3 lines

check script is also refactored to make it simple to do next cool thing about iotests: allow meaningful names for test-case files.

This one would also require us to be able to distinguish test case files from arbitrary other files.

I think, something like "all files started with 'test-' prefix"

Quoted 16 lines

tests/qemu-iotests/group line 30

Quoted 10 lines
tests/qemu-iotests/group deleted +0 -9
-#
-# test-group association ... one line per test
-#
-001 rw auto quick
-002 rw auto quick
-003 rw auto
-004 rw auto quick
-005 img auto quick
-# 006 was removed, do not reuse

We lose these comments without a replacement. I wonder whether it's important or if we can think of another way to make sure that numbers aren't reused. (I'm not completely sure any more why we decided that we don't want to reuse numbers. Maybe because of backports?)

Hmm.. Okay, if we had this test in past, than drop, than reuse number for the another test, we possibly may break bisecting.

I can add notes about it into documentation (nobody will read it). Anyway, I don't think it is too bad.

And if we move to textual file-names for tests, we will not reuse old removed numbers anyway.

Signature · 2 lines
Best regards,
Vladimir
#7 lore raw
Quoted 32 lines · Vladimir Sementsov-Ogievskiy
Quoted 31 lines · Kevin Wolf

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 29 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

     grep '^[0-9]\{3\} ' group | while read line; do
         file=$(awk '{print $1}' <<< "$line");
         groups=$(sed -e 's/^... //' <<< "$line");
         awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
         cat tmp > $file;
     done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

Quoted 7 lines
Quoted 5 lines

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Hmm. Probably, you could add it to group "disabled", and run check -x disabled.

As a developer you don't really want to be making changes to git tracked files in order to temporarily skip a test, as then git reports them as modified & you risk accidentally committing throwaway changes.

Better to have a separate groups.local file to record local overrides in a non-tracked file.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#8 lore raw

Am 24.03.2020 um 11:21 hat Daniel P. Berrangé geschrieben:

Quoted 39 lines · Vladimir Sementsov-Ogievskiy
Quoted 38 lines
Quoted 36 lines · Kevin Wolf

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 23 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

     grep '^[0-9]\{3\} ' group | while read line; do
         file=$(awk '{print $1}' <<< "$line");
         groups=$(sed -e 's/^... //' <<< "$line");
         awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
         cat tmp > $file;
     done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

Quoted 6 lines

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Quoted 8 lines
Quoted 1 line

Hmm. Probably, you could add it to group "disabled", and run check -x disabled.

As a developer you don't really want to be making changes to git tracked files in order to temporarily skip a test, as then git reports them as modified & you risk accidentally committing throwaway changes.

Better to have a separate groups.local file to record local overrides in a non-tracked file.

For locally disabling tests, we have that 'expunged' file that nobody knows about and that I just found when we were reformatting the ./check output...

I wouldn't want to use that for disabling the tests in a downstream repository, though, it should stay a local thing even there.

Kevin

#9 lore raw
In reply to Kevin Wolf
Quoted 36 lines · Kevin Wolf

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 23 lines

When sending iotests to upstream or do patch porting from one branch to another we very often have to resolve conflicts in group file, as many absolutely independent features are intersecting by this file. These conflicts are simple, but imagine how much time we all have already spent on resolving them? Let's finally get rid of group file.

This patch transposes group info: instead of collecting it in one file, let each test define its groups by itself.

Several steps are done to achive it:

1. Define groups in test files automatically:

    grep '^[0-9]\{3\} ' group | while read line; do
        file=$(awk '{print $1}' <<< "$line");
        groups=$(sed -e 's/^... //' <<< "$line");
        awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
        cat tmp > $file;
    done

2. Copy groups documentation into docs/devel/testing.rst, which already has a section about iotests.

Quoted 6 lines

3. Modify check script to work without group file.

Here is a logic change: before, even if test do not belong to any group (only iotest 142 currently) it should be defined in group file. Now, test is not forced to define any group. Instead check considers all files with names matching [0-9][0-9][0-9] as tests.

This has both a positive and a negative effect: Now you don't have to modify another file when you add a new test, but it will be picked up automatically. However, if you want to disable a test, you could previously just remove it from groups (or comment it out), and now you have to delete the test instead.

Quoted 7 lines

Downstream, I think we still disable a few tests because we're compiling out features that are required for some tests to pass, and deleting the test cases completely would probably move conflicts just to a different place.

So I think we need a method to distuingish an enabled test that is in no group from a disabled test.

+1 for blacklist.local file

Quoted 6 lines
Quoted 3 lines

check script is also refactored to make it simple to do next cool thing about iotests: allow meaningful names for test-case files.

This one would also require us to be able to distinguish test case files from arbitrary other files.

Quoted 13 lines

tests/qemu-iotests/group line 30

Quoted 9 lines
tests/qemu-iotests/group deleted +0 -9
-#
-# test-group association ... one line per test
-#
-001 rw auto quick
-002 rw auto quick
-003 rw auto
-004 rw auto quick
-005 img auto quick
-# 006 was removed, do not reuse

We lose these comments without a replacement. I wonder whether it's important or if we can think of another way to make sure that numbers aren't reused. (I'm not completely sure any more why we decided that we don't want to reuse numbers. Maybe because of backports?)

we could generate this file with a starter script with proper option.

Den