Skip to contentSkip to conversation

iotests: Check for zstd support

PATCH Superseded Hanna Reitz 3 participants 8 messages Mar 2, 2022 – Mar 23, 2022

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover iotests: Check for zstd support

Hi,

v1 cover letter:

https://lists.nongnu.org/archive/html/qemu-devel/2022-02/msg04592.html

We have two tests (as far as I know) that use compression_type=zstd for qcow2 but do not check whether that is actually supported. Thomas reported this for 065, but it’s also the case for 303.

This series makes 303 be skipped when zstd is not compiled in, and has 065 use zlib for each of its test cases then (it was made to use zstd just to improve on coverage, so using zlib as a fallback is perfectly fine).

v2:
- Add the first patch so that 065 and 303 can use these new iotests.py functions to check for zstd support instead of checking for their own qemu-img create’s output
- Have 065 fall back to zlib instead of skipping zstd test cases

Hanna Reitz (3):
iotests.py: Add supports_qcow2_zstd_compression() iotests/065: Check for zstd support iotests/303: Check for zstd support

 tests/qemu-iotests/065        | 24 ++++++++++++++++++------
 tests/qemu-iotests/303        |  4 +++-
 tests/qemu-iotests/iotests.py | 21 +++++++++++++++++++++
 3 files changed, 42 insertions(+), 7 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v2 1/3 iotests.py: Add supports_qcow2_zstd_compression()

---
 tests/qemu-iotests/iotests.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
tests/qemu-iotests/iotests.py +21 -0
index 6027780180..ce60abcadc 100644
@@ -1377,6 +1377,27 @@ def verify_working_luks():
     if not working:
         notrun(reason)
 
+def supports_qcow2_zstd_compression() -> bool:
+    img_file = f'{test_dir}/qcow2-zstd-test.qcow2'
+    (output, status) = \
+        qemu_img_pipe_and_status('create', '-f', 'qcow2',
+                                 '-o', 'compression_type=zstd',
+                                 img_file, '0')
+    try:
+        os.remove(img_file)
+    except OSError:
+        pass
+
+    if status == 1 and \
+            "'compression-type' does not accept value 'zstd'" in output:
+        return False
+    else:
+        return True
+
+def verify_qcow2_zstd_compression():
+    if not supports_qcow2_zstd_compression():
+        notrun('zstd compression not supported')
+
 def qemu_pipe(*args: str) -> str:
     """
     Run qemu with an option to print something and exit (e.g. a help option).
Signature · 1 line
2.34.1
#2 lore raw

PATCH v2 2/3 iotests/065: Check for zstd support

In reply to Hanna Reitz

Some test cases run in iotest 065 want to run with zstd compression just for added coverage. Run them with zlib if there is no zstd support compiled in.

Fixes 12a936171d71f839dc907ff ("iotest 065: explicit compression type")
---
 tests/qemu-iotests/065 | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
tests/qemu-iotests/065 +18 -6
index f7c1b68dad..d1a72923c9 100755
@@ -24,7 +24,7 @@ import os
 import re
 import json
 import iotests
-from iotests import qemu_img, qemu_img_pipe
+from iotests import qemu_img, qemu_img_pipe, supports_qcow2_zstd_compression
 import unittest
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -96,11 +96,17 @@ class TestQCow2(TestQemuImgInfo):
 
 class TestQCow3NotLazy(TestQemuImgInfo):
     '''Testing a qcow2 version 3 image with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zstd'
+    if supports_qcow2_zstd_compression():
+        compression_type = 'zstd'
+    else:
+        compression_type = 'zlib'
+
+    img_options = 'compat=1.1,lazy_refcounts=off'
+    img_options += f',compression_type={compression_type}'
     json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
                      'refcount-bits': 16, 'corrupt': False,
-                     'compression-type': 'zstd', 'extended-l2': False }
-    human_compare = [ 'compat: 1.1', 'compression type: zstd',
+                     'compression-type': compression_type, 'extended-l2': False }
+    human_compare = [ 'compat: 1.1', f'compression type: {compression_type}',
                       'lazy refcounts: false', 'refcount bits: 16',
                       'corrupt: false', 'extended l2: false' ]
 
@@ -127,11 +133,17 @@ class TestQCow3NotLazyQMP(TestQMP):
 class TestQCow3LazyQMP(TestQMP):
     '''Testing a qcow2 version 3 image with lazy refcounts enabled, opening
        with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zstd'
+    if supports_qcow2_zstd_compression():
+        compression_type = 'zstd'
+    else:
+        compression_type = 'zlib'
+
+    img_options = 'compat=1.1,lazy_refcounts=on'
+    img_options += f',compression_type={compression_type}'
     qemu_options = 'lazy-refcounts=off'
     compare = { 'compat': '1.1', 'lazy-refcounts': True,
                 'refcount-bits': 16, 'corrupt': False,
-                'compression-type': 'zstd', 'extended-l2': False }
+                'compression-type': compression_type, 'extended-l2': False }
 
 TestImageInfoSpecific = None
 TestQemuImgInfo = None
Signature · 1 line
2.34.1
#3 lore raw

PATCH v2 3/3 iotests/303: Check for zstd support

In reply to Hanna Reitz

303 runs two test cases, one of which requires zstd support. Unfortunately, given that this is not a unittest-style test, we cannot easily skip that single case, and instead can only skip the whole test.

(Alternatively, we could split this test into a zlib and a zstd part, but that seems excessive, given that this test is not in auto and thus likely only run by developers who have zstd support compiled in.)

Fixes 677e0bae686e7c670a71d1f ("iotest 303: explicit compression type")
---
 tests/qemu-iotests/303 | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tests/qemu-iotests/303 +3 -1
index 16c2e10827..25c3e7eaaa 100755
@@ -21,10 +21,12 @@
 
 import iotests
 import subprocess
-from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io
+from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io, \
+        verify_qcow2_zstd_compression
 
 iotests.script_initialize(supported_fmts=['qcow2'],
                           unsupported_imgopts=['refcount_bits', 'compat'])
+verify_qcow2_zstd_compression()
 
 disk = file_path('disk')
 chunk = 1024 * 1024
Signature · 1 line
2.34.1
#4 lore raw
In reply to Hanna Reitz

Ping

Quoted 32 lines · Hanna Reitz

Hi,

v1 cover letter:

https://lists.nongnu.org/archive/html/qemu-devel/2022-02/msg04592.html

We have two tests (as far as I know) that use compression_type=zstd for qcow2 but do not check whether that is actually supported. Thomas reported this for 065, but it’s also the case for 303.

This series makes 303 be skipped when zstd is not compiled in, and has 065 use zlib for each of its test cases then (it was made to use zstd just to improve on coverage, so using zlib as a fallback is perfectly fine).

v2:
- Add the first patch so that 065 and 303 can use these new iotests.py functions to check for zstd support instead of checking for their own qemu-img create’s output
- Have 065 fall back to zlib instead of skipping zstd test cases

Hanna Reitz (3):
iotests.py: Add supports_qcow2_zstd_compression() iotests/065: Check for zstd support iotests/303: Check for zstd support

  tests/qemu-iotests/065        | 24 ++++++++++++++++++------
  tests/qemu-iotests/303        |  4 +++-
  tests/qemu-iotests/iotests.py | 21 +++++++++++++++++++++
  3 files changed, 42 insertions(+), 7 deletions(-)
#5 lore raw
Quoted 36 lines · Hanna Reitz

Ping

Quoted 32 lines · Hanna Reitz

Hi,

v1 cover letter:

https://lists.nongnu.org/archive/html/qemu-devel/2022-02/msg04592.html

We have two tests (as far as I know) that use compression_type=zstd for qcow2 but do not check whether that is actually supported.  Thomas reported this for 065, but it’s also the case for 303.

This series makes 303 be skipped when zstd is not compiled in, and has 065 use zlib for each of its test cases then (it was made to use zstd just to improve on coverage, so using zlib as a fallback is perfectly fine).

v2:
- Add the first patch so that 065 and 303 can use these new iotests.py functions to check for zstd support instead of checking for their own qemu-img create’s output
- Have 065 fall back to zlib instead of skipping zstd test cases

Hanna Reitz (3):
   iotests.py: Add supports_qcow2_zstd_compression() iotests/065: Check for zstd support iotests/303: Check for zstd support

  tests/qemu-iotests/065        | 24 ++++++++++++++++++------ tests/qemu-iotests/303        |  4 +++- tests/qemu-iotests/iotests.py | 21 +++++++++++++++++++++

  3 files changed, 42 insertions(+), 7 deletions(-)
Signature · 2 lines
Best regards,
Vladimir
Tested-by on the v2 series
#6 lore raw
In reply to Hanna Reitz
Quoted 27 lines · Hanna Reitz

Hi,

v1 cover letter:

https://lists.nongnu.org/archive/html/qemu-devel/2022-02/msg04592.html

We have two tests (as far as I know) that use compression_type=zstd for qcow2 but do not check whether that is actually supported. Thomas reported this for 065, but it’s also the case for 303.

This series makes 303 be skipped when zstd is not compiled in, and has 065 use zlib for each of its test cases then (it was made to use zstd just to improve on coverage, so using zlib as a fallback is perfectly fine).

v2:
- Add the first patch so that 065 and 303 can use these new iotests.py functions to check for zstd support instead of checking for their own qemu-img create’s output
- Have 065 fall back to zlib instead of skipping zstd test cases

Hanna Reitz (3):
iotests.py: Add supports_qcow2_zstd_compression() iotests/065: Check for zstd support iotests/303: Check for zstd support

Quoted 4 lines
  tests/qemu-iotests/065        | 24 ++++++++++++++++++------
  tests/qemu-iotests/303        |  4 +++-
  tests/qemu-iotests/iotests.py | 21 +++++++++++++++++++++
  3 files changed, 42 insertions(+), 7 deletions(-)

Thanks, this fixes the failures of 065 and 303 on my system!

Series

#7 lore raw
In reply to Hanna Reitz
Quoted 6 lines · Hanna Reitz

Hi,

v1 cover letter:

https://lists.nongnu.org/archive/html/qemu-devel/2022-02/msg04592.html

Quoted 8 lines

We have two tests (as far as I know) that use compression_type=zstd for qcow2 but do not check whether that is actually supported. Thomas reported this for 065, but it’s also the case for 303.

This series makes 303 be skipped when zstd is not compiled in, and has 065 use zlib for each of its test cases then (it was made to use zstd just to improve on coverage, so using zlib as a fallback is perfectly fine).

Thanks a lot for the quick replies to my ping!

Regrettably, I forgot to consider that John’s recent changes would remove qemu_img_pipe_and_status(), which is used in patch 1...  I’m sorry, but I’ll have to send a v3. :/

Hanna