Skip to contentSkip to conversation

iotests: Check for zstd support

PATCH In review Hanna Reitz 2 participants 6 messages Mar 23, 2022

lore
Share & tools
lore
#0 lore raw

v3 cover iotests: Check for zstd support

Hi,

v1 cover letter:
https://lists.nongnu.org/archive/html/qemu-devel/2022-02/msg04592.html

v2 cover letter:
https://lists.nongnu.org/archive/html/qemu-block/2022-03/msg00041.html

Thanks a lot for the review and testing on v2; but I missed that I’d have to send v3 because qemu_img_pipe_and_status() has just been removed, which was used in patch 1 of v2. :/

v3:
- Patch 1: Use qemu_img(check=False) instead of
qemu_img_pipe_and_status()
- Patch 2: Rebase-caused import conflict

git backport-diff against v2:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/3:[0011] [FC] 'iotests.py: Add supports_qcow2_zstd_compression()' 002/3:[0004] [FC] 'iotests/065: Check for zstd support' 003/3:[----] [--] 'iotests/303: Check for zstd support'

Hanna Reitz (3):
iotests.py: Add supports_qcow2_zstd_compression() iotests/065: Check for zstd support iotests/303: Check for zstd support

 tests/qemu-iotests/065        | 24 ++++++++++++++++++------
 tests/qemu-iotests/303        |  4 +++-
 tests/qemu-iotests/iotests.py | 20 ++++++++++++++++++++
 3 files changed, 41 insertions(+), 7 deletions(-)
Signature · 1 line
2.35.1
#1 lore raw

PATCH v3 1/3 iotests.py: Add supports_qcow2_zstd_compression()

---
 tests/qemu-iotests/iotests.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
tests/qemu-iotests/iotests.py +20 -0
index fcec3e51e5..fe10a6cf05 100644
@@ -1471,6 +1471,26 @@ def verify_working_luks():
     if not working:
         notrun(reason)
 
+def supports_qcow2_zstd_compression() -> bool:
+    img_file = f'{test_dir}/qcow2-zstd-test.qcow2'
+    res = qemu_img('create', '-f', 'qcow2', '-o', 'compression_type=zstd',
+                   img_file, '0',
+                   check=False)
+    try:
+        os.remove(img_file)
+    except OSError:
+        pass
+
+    if res.returncode == 1 and \
+            "'compression-type' does not accept value 'zstd'" in res.stdout:
+        return False
+    else:
+        return True
+
+def verify_qcow2_zstd_compression():
+    if not supports_qcow2_zstd_compression():
+        notrun('zstd compression not supported')
+
 def qemu_pipe(*args: str) -> str:
     """
     Run qemu with an option to print something and exit (e.g. a help option).
Signature · 1 line
2.35.1
Reviewed-by on v3 1/3
#2 lore raw

Re: [PATCH v3 1/3] iotests.py: Add supports_qcow2_zstd_compression()

Quoted 11 lines · Hanna Reitz
---
  tests/qemu-iotests/iotests.py | 20 ++++++++++++++++++++
  1 file changed, 20 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index fcec3e51e5..fe10a6cf05 100644

tests/qemu-iotests/iotests.py def verify_working_luks(): · line 1478

Quoted 8 lines
tests/qemu-iotests/iotests.py +5 -0
@@ -1471,6 +1471,26 @@ def verify_working_luks():
      if not working:
          notrun(reason)
 
+def supports_qcow2_zstd_compression() -> bool:
+    img_file = f'{test_dir}/qcow2-zstd-test.qcow2'
+    res = qemu_img('create', '-f', 'qcow2', '-o', 'compression_type=zstd',
+                   img_file, '0',
+                   check=False)

check=False may be squashed into previous line

tests/qemu-iotests/iotests.py def verify_working_luks(): · line 1488

Quoted 10 lines
tests/qemu-iotests/iotests.py +10 -0
+    try:
+        os.remove(img_file)
+    except OSError:
+        pass
+
+    if res.returncode == 1 and \
+            "'compression-type' does not accept value 'zstd'" in res.stdout:
+        return False
+    else:
+        return True

May be:

return not(res.returncode == 1 and
"'compression-type' does not accept value 'zstd'" in res.stdout)

tests/qemu-iotests/iotests.py def verify_working_luks(): · line 1496

Quoted 8 lines
tests/qemu-iotests/iotests.py +5 -0
+
+def verify_qcow2_zstd_compression():
+    if not supports_qcow2_zstd_compression():
+        notrun('zstd compression not supported')
+
  def qemu_pipe(*args: str) -> str:
      """
      Run qemu with an option to print something and exit (e.g. a help option).

anyway:

Signature · 2 lines
Best regards,
Vladimir
#3 lore raw

PATCH v3 2/3 iotests/065: Check for zstd support

In reply to Hanna Reitz

Some test cases run in iotest 065 want to run with zstd compression just for added coverage. Run them with zlib if there is no zstd support compiled in.

Fixes 12a936171d71f839dc907ff ("iotest 065: explicit compression type")
---
 tests/qemu-iotests/065 | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
tests/qemu-iotests/065 +18 -6
index ba94e19349..b724c89c7c 100755
@@ -24,7 +24,7 @@ import os
 import re
 import json
 import iotests
-from iotests import qemu_img, qemu_img_info
+from iotests import qemu_img, qemu_img_info, supports_qcow2_zstd_compression
 import unittest
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -95,11 +95,17 @@ class TestQCow2(TestQemuImgInfo):
 
 class TestQCow3NotLazy(TestQemuImgInfo):
     '''Testing a qcow2 version 3 image with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zstd'
+    if supports_qcow2_zstd_compression():
+        compression_type = 'zstd'
+    else:
+        compression_type = 'zlib'
+
+    img_options = 'compat=1.1,lazy_refcounts=off'
+    img_options += f',compression_type={compression_type}'
     json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
                      'refcount-bits': 16, 'corrupt': False,
-                     'compression-type': 'zstd', 'extended-l2': False }
-    human_compare = [ 'compat: 1.1', 'compression type: zstd',
+                     'compression-type': compression_type, 'extended-l2': False }
+    human_compare = [ 'compat: 1.1', f'compression type: {compression_type}',
                       'lazy refcounts: false', 'refcount bits: 16',
                       'corrupt: false', 'extended l2: false' ]
 
@@ -126,11 +132,17 @@ class TestQCow3NotLazyQMP(TestQMP):
 class TestQCow3LazyQMP(TestQMP):
     '''Testing a qcow2 version 3 image with lazy refcounts enabled, opening
        with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zstd'
+    if supports_qcow2_zstd_compression():
+        compression_type = 'zstd'
+    else:
+        compression_type = 'zlib'
+
+    img_options = 'compat=1.1,lazy_refcounts=on'
+    img_options += f',compression_type={compression_type}'
     qemu_options = 'lazy-refcounts=off'
     compare = { 'compat': '1.1', 'lazy-refcounts': True,
                 'refcount-bits': 16, 'corrupt': False,
-                'compression-type': 'zstd', 'extended-l2': False }
+                'compression-type': compression_type, 'extended-l2': False }
 
 TestImageInfoSpecific = None
 TestQemuImgInfo = None
Signature · 1 line
2.35.1
Reviewed-by on v3 2/3
#4 lore raw

Re: [PATCH v3 2/3] iotests/065: Check for zstd support

Quoted 7 lines · Hanna Reitz

Some test cases run in iotest 065 want to run with zstd compression just for added coverage. Run them with zlib if there is no zstd support compiled in.

Fixes 12a936171d71f839dc907ff ("iotest 065: explicit compression type")
Signature · 2 lines
Best regards,
Vladimir
#5 lore raw

PATCH v3 3/3 iotests/303: Check for zstd support

In reply to Hanna Reitz

303 runs two test cases, one of which requires zstd support. Unfortunately, given that this is not a unittest-style test, we cannot easily skip that single case, and instead can only skip the whole test.

(Alternatively, we could split this test into a zlib and a zstd part, but that seems excessive, given that this test is not in auto and thus likely only run by developers who have zstd support compiled in.)

Fixes 677e0bae686e7c670a71d1f ("iotest 303: explicit compression type")
---
 tests/qemu-iotests/303 | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tests/qemu-iotests/303 +3 -1
index 93aa5ce9b7..40e947f26c 100755
@@ -21,10 +21,12 @@
 
 import iotests
 import subprocess
-from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io
+from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io, \
+        verify_qcow2_zstd_compression
 
 iotests.script_initialize(supported_fmts=['qcow2'],
                           unsupported_imgopts=['refcount_bits', 'compat'])
+verify_qcow2_zstd_compression()
 
 disk = file_path('disk')
 chunk = 1024 * 1024
Signature · 1 line
2.35.1