Skip to contentSkip to conversation

iotests: Honor $IMGOPTS in Python tests

PATCH In review Max Reitz 3 participants 81 messages Oct 1, 2019 – Oct 4, 2019

lore
Share & tools
lore

Max Reitz

author
#0 lore raw

cover iotests: Honor $IMGOPTS in Python tests

First of all: Sorry.

Second:

Based-on: My block branch
(https://github.com/XanClic/qemu.git block)

Based-on: [email protected]
(“iotests: use python logging”)

Based-on: [email protected]
(“iotests: Allow ./check -o data_file”)

Based-on: [email protected]
(“iotests: Selfish patches”)

Based-on: [email protected]
(“block: Skip COR for inactive nodes”)

OK, now:

Hi,

My recent series “iotests: Allow ./check -o data_file” enabled our bash tests to interpret the data_file qcow2 option. It didn’t do anything for Python tests because those currently completely ignore all image format options.

This is where it gets hairy. To do so, we need two things: First of all, whatever way Python tests use to create images needs to interpret $IMGOPTS. Second, when deleting image files, they must not use a plain os.remove(), but a special function that will clean up data files, too.

The heap of patches in this series comes from making the Python tests use these new functions.

Most Python tests just run qemu-img through a helper function that does not care about the exact subcommand to create images. I could add $IMGOPTS support to it, but that doesn’t feel quite right to me, and it wouldn’t reduce the patch count because we still need a special removal function.

This series is structured as follows:
- Patches 1 through 7 add support to handle image files differently from other files (consider $IMGOPTS when creating them, consider data files when deleting them, separate ImagePaths from FilePaths, and so on)

- Patches 8 and 9 add two filters we’ll need in the next range:

- Patches 10 through 13 address some issues in a handful of tests that just need to be changed a little so they can overall work with some format options

- Patch 14 makes all tests pass unsupported_imgopts where there are options that they cannot support.

- Patches 15 through 65 make all Python tests use the new functions introduced in the first 7 patches so they no longer ignore $IMGOPTS.

I felt like this is much better than munching everything together into a single big commit (better to rebase, better to review), and I don’t really like ideas like “Just do five patches that each address ten iotests”.

But I’m still very much open to suggestions on how to combine these many small patches to reduce the overall patch count.

- Patch 66 ensures that Python tests always use the new function to create test images so they won’t bypass $IMGOPTS.

- Patch 67 cleans up. qemu_img_log() is only used for image creation, and I don’t see the point in that. The output is predictable and it is very unlikely to fail. We can see in the bash tests that regularly we basically just filter everything from it anyway.
(So this series replaces log(qemu_img_pipe()) instances by asserting that image creation did not fail.)
((qemu_img_create() obviously no longer has any use after this series.))

After this series, running the iotests with -o compat=0.10, -o refcount_bits=1, and -o 'data_file=$TEST_IMG.data_file' does something sensible even for the Python tests, and it passes.

Max Reitz (67):
iotests.py: Read $IMGOPTS
iotests.py: Add @skip_for_imgopts() iotests.py: Add unsupported_imgopts iotests.py: create_test_image, remove_test_image iotests.py: Add ImagePaths
iotests.py: Add image_path()
iotests.py: Filter data_file in filter_img_info iotests.py: Add filter_json_filename() iotests.py: Add @hide_fields to img_info_log iotests/169: Skip persistent cases for compat=0.10 iotests/224: Filter json:{} from commit command iotests/228: Filter json:{} filenames iotests/242: Hide refcount bit information
iotests: Use unsupported_imgopts in Python tests iotests/030: Honor $IMGOPTS
iotests/040: Honor $IMGOPTS
iotests/041: Honor $IMGOPTS
iotests/044: Honor $IMGOPTS
iotests/045: Honor $IMGOPTS
iotests/055: Honor $IMGOPTS
iotests/056: Honor $IMGOPTS
iotests/057: Honor $IMGOPTS
iotests/065: Honor $IMGOPTS
iotests/096: Honor $IMGOPTS
iotests/118: Honor $IMGOPTS
iotests/124: Honor $IMGOPTS
iotests/129: Honor $IMGOPTS
iotests/132: Honor $IMGOPTS
iotests/139: Honor $IMGOPTS
iotests/147: Honor $IMGOPTS
iotests/148: Honor $IMGOPTS
iotests/151: Honor $IMGOPTS
iotests/152: Honor $IMGOPTS
iotests/155: Honor $IMGOPTS
iotests/163: Honor $IMGOPTS
iotests/165: Honor $IMGOPTS
iotests/169: Honor $IMGOPTS
iotests/194: Honor $IMGOPTS
iotests/196: Honor $IMGOPTS
iotests/199: Honor $IMGOPTS
iotests/202: Honor $IMGOPTS
iotests/203: Honor $IMGOPTS
iotests/205: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/216: Honor $IMGOPTS
iotests/218: Honor $IMGOPTS
iotests/219: Honor $IMGOPTS
iotests/222: Honor $IMGOPTS
iotests/224: Honor $IMGOPTS
iotests/228: Honor $IMGOPTS
iotests/234: Honor $IMGOPTS
iotests/235: Honor $IMGOPTS
iotests/236: Honor $IMGOPTS
iotests/237: Honor $IMGOPTS
iotests/242: Honor $IMGOPTS
iotests/245: Honor $IMGOPTS
iotests/246: Honor $IMGOPTS
iotests/248: Honor $IMGOPTS
iotests/254: Honor $IMGOPTS
iotests/255: Honor $IMGOPTS
iotests/256: Honor $IMGOPTS
iotests/257: Honor $IMGOPTS
iotests/258: Honor $IMGOPTS
iotests/262: Honor $IMGOPTS
iotests.py: Forbid qemu_img*('create', ...) iotests.py: Drop qemu_img_log(), qemu_img_create()

 tests/qemu-iotests/030        |  69 ++++++------
 tests/qemu-iotests/040        |  42 ++++----
 tests/qemu-iotests/041        | 108 +++++++++----------
 tests/qemu-iotests/044        |  11 +-
 tests/qemu-iotests/045        |  26 ++---
 tests/qemu-iotests/055        |  41 +++----
 tests/qemu-iotests/056        |  30 +++---
 tests/qemu-iotests/057        |  10 +-
 tests/qemu-iotests/065        |  21 ++--
 tests/qemu-iotests/096        |   5 +-
 tests/qemu-iotests/118        |  26 ++---
 tests/qemu-iotests/124        |  29 +++--
 tests/qemu-iotests/129        |  11 +-
 tests/qemu-iotests/132        |   6 +-
 tests/qemu-iotests/139        |  15 ++-
 tests/qemu-iotests/147        |  11 +-
 tests/qemu-iotests/148        |   5 +-
 tests/qemu-iotests/151        |  10 +-
 tests/qemu-iotests/152        |   6 +-
 tests/qemu-iotests/155        |  29 +++--
 tests/qemu-iotests/163        |  29 ++---
 tests/qemu-iotests/165        |  10 +-
 tests/qemu-iotests/169        |  23 ++--
 tests/qemu-iotests/194        |   9 +-
 tests/qemu-iotests/196        |  10 +-
 tests/qemu-iotests/199        |  10 +-
 tests/qemu-iotests/202        |   9 +-
 tests/qemu-iotests/203        |   9 +-
 tests/qemu-iotests/205        |   7 +-
 tests/qemu-iotests/206        |   5 +-
 tests/qemu-iotests/208        |   5 +-
 tests/qemu-iotests/209        |   9 +-
 tests/qemu-iotests/216        |  11 +-
 tests/qemu-iotests/218        |   6 +-
 tests/qemu-iotests/219        |   5 +-
 tests/qemu-iotests/222        |  13 +--
 tests/qemu-iotests/224        |  33 +++---
 tests/qemu-iotests/224.out    |   4 +-
 tests/qemu-iotests/228        |  25 ++---
 tests/qemu-iotests/228.out    |   8 +-
 tests/qemu-iotests/234        |   9 +-
 tests/qemu-iotests/235        |   7 +-
 tests/qemu-iotests/236        |   6 +-
 tests/qemu-iotests/237        |  15 +--
 tests/qemu-iotests/237.out    |   6 --
 tests/qemu-iotests/242        |  21 ++--
 tests/qemu-iotests/242.out    |   5 -
 tests/qemu-iotests/245        |  21 ++--
 tests/qemu-iotests/246        |  11 +-
 tests/qemu-iotests/248        |  14 ++-
 tests/qemu-iotests/254        |  10 +-
 tests/qemu-iotests/255        |  20 ++--
 tests/qemu-iotests/255.out    |   8 --
 tests/qemu-iotests/256        |  10 +-
 tests/qemu-iotests/257        |  18 ++--
 tests/qemu-iotests/258        |  16 +--
 tests/qemu-iotests/262        |   5 +-
 tests/qemu-iotests/iotests.py | 197 +++++++++++++++++++++++++++-------
 58 files changed, 654 insertions(+), 496 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH 1/67 iotests.py: Read $IMGOPTS

We do not do anything with yet, but this is the first step.

---
 tests/qemu-iotests/iotests.py | 6 ++++++
 1 file changed, 6 insertions(+)
tests/qemu-iotests/iotests.py +6 -0
index 1c5fce3e9e..7030900807 100644
@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.')
 cachemode = os.environ.get('CACHEMODE')
 qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
+imgopts = os.environ.get('IMGOPTS', '')
+if len(imgopts) == 0:
+    imgopts = []
+else:
+    imgopts = imgopts.split(',')
+
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
Signature · 1 line
2.21.0
Reviewed-by on v1 1/67
#2 lore raw

Re: [PATCH 01/67] iotests.py: Read $IMGOPTS

Quoted 12 lines · Max Reitz

We do not do anything with yet, but this is the first step.

---
 tests/qemu-iotests/iotests.py | 6 ++++++
 1 file changed, 6 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 1c5fce3e9e..7030900807 100644

tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 77

Quoted 9 lines
tests/qemu-iotests/iotests.py +6 -0
@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.')
 cachemode = os.environ.get('CACHEMODE')
 qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
+imgopts = os.environ.get('IMGOPTS', '')
+if len(imgopts) == 0:
+    imgopts = []
+else:
+    imgopts = imgopts.split(',')
+

Sometimes I think about the type of person I'd like to meet, and I ask myself if it's the type of person who would quote pep8, or say things like "idiomatic python" in a code review when the existing form has no technical problem whatsoever.

I wonder what type of person I am.

Well, questions for another day.

tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 80

Quoted 4 lines
tests/qemu-iotests/iotests.py +0 -0
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
#3 lore raw

Re: [PATCH 01/67] iotests.py: Read $IMGOPTS

Quoted 30 lines · John Snow
Quoted 12 lines · Max Reitz

We do not do anything with yet, but this is the first step.

---
  tests/qemu-iotests/iotests.py | 6 ++++++
  1 file changed, 6 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 1c5fce3e9e..7030900807 100644

tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 77

Quoted 9 lines
tests/qemu-iotests/iotests.py +6 -0
@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.')
  cachemode = os.environ.get('CACHEMODE')
  qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
+imgopts = os.environ.get('IMGOPTS', '')
+if len(imgopts) == 0:
+    imgopts = []
+else:
+    imgopts = imgopts.split(',')
+

Sometimes I think about the type of person I'd like to meet, and I ask myself if it's the type of person who would quote pep8, or say things like "idiomatic python" in a code review when the existing form has no technical problem whatsoever.

Quoted 3 lines

I wonder what type of person I am.

Well, questions for another day.

Hope you don't say about quoting, as pep8 "does not make a recommendation for this".

Quoted 8 lines

tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 80

Quoted 5 lines
tests/qemu-iotests/iotests.py +0 -0
  socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
 
  luks_default_secret_object = 'secret,id=keysec0,data=' + \
Signature · 2 lines
Best regards,
Vladimir
Reviewed-by on v1 1/67
#4 lore raw

Re: [PATCH 01/67] iotests.py: Read $IMGOPTS

In reply to Max Reitz
Quoted 15 lines · Max Reitz

We do not do anything with yet, but this is the first step.

---
  tests/qemu-iotests/iotests.py | 6 ++++++
  1 file changed, 6 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 1c5fce3e9e..7030900807 100644
@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.')
  cachemode = os.environ.get('CACHEMODE')
  qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')

tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 80

Quoted 10 lines
tests/qemu-iotests/iotests.py +6 -0
 
+imgopts = os.environ.get('IMGOPTS', '')
+if len(imgopts) == 0:
+    imgopts = []
+else:
+    imgopts = imgopts.split(',')
+
  socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
 
  luks_default_secret_object = 'secret,id=keysec0,data=' + \

Note, that empty sequences are always false, so you may use just "if not imgopts"

Signature · 2 lines
Best regards,
Vladimir
author
#5 lore raw

PATCH 2/67 iotests.py: Add @skip_for_imgopts()

In reply to Max Reitz
---
 tests/qemu-iotests/iotests.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)
tests/qemu-iotests/iotests.py +13 -0
index 7030900807..cdcb62c4ac 100644
@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False):
         return func_wrapper
     return skip_test_decorator
 
+def skip_for_imgopts(unsupported_opts=[]):
+    '''Skip Test Decorator
+       Skips the test if imgopts contains any of the given options'''
+    def skip_test_decorator(func):
+        def func_wrapper(test_case: QMPTestCase, *args, **kwargs):
+            for opt in imgopts:
+                if any(unsupported in opt for unsupported in unsupported_opts):
+                    test_case.case_skip('{}: Option {} is unsupported'.format(
+                                        test_case, opt))
+            return func(test_case, *args, **kwargs)
+        return func_wrapper
+    return skip_test_decorator
+
 def execute_unittest(debug=False):
     """Executes unittests within the calling module."""
 
Signature · 1 line
2.21.0
Reviewed-by on v1 2/67
#6 lore raw

Re: [PATCH 02/67] iotests.py: Add @skip_for_imgopts()

Quoted 10 lines · Max Reitz
---
 tests/qemu-iotests/iotests.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 7030900807..cdcb62c4ac 100644

tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 957

Quoted 8 lines
tests/qemu-iotests/iotests.py +5 -0
@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False):
         return func_wrapper
     return skip_test_decorator
 
+def skip_for_imgopts(unsupported_opts=[]):
+    '''Skip Test Decorator
+       Skips the test if imgopts contains any of the given options'''
+    def skip_test_decorator(func):
+        def func_wrapper(test_case: QMPTestCase, *args, **kwargs):

:D

tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 968

Quoted 12 lines
tests/qemu-iotests/iotests.py +8 -0
+            for opt in imgopts:
+                if any(unsupported in opt for unsupported in unsupported_opts):
+                    test_case.case_skip('{}: Option {} is unsupported'.format(
+                                        test_case, opt))
+            return func(test_case, *args, **kwargs)
+        return func_wrapper
+    return skip_test_decorator
+
 def execute_unittest(debug=False):
     """Executes unittests within the calling module."""
 
#7 lore raw

Re: [PATCH 02/67] iotests.py: Add @skip_for_imgopts()

In reply to Max Reitz
Quoted 10 lines · Max Reitz
---
  tests/qemu-iotests/iotests.py | 13 +++++++++++++
  1 file changed, 13 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 7030900807..cdcb62c4ac 100644

tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 957

Quoted 8 lines
tests/qemu-iotests/iotests.py +5 -0
@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False):
          return func_wrapper
      return skip_test_decorator
 
+def skip_for_imgopts(unsupported_opts=[]):
+    '''Skip Test Decorator
+       Skips the test if imgopts contains any of the given options'''
+    def skip_test_decorator(func):
+        def func_wrapper(test_case: QMPTestCase, *args, **kwargs):

how about

unsup = set(imgopts) & set(unsupported_opts)
if unsup:

    test_case.case_skip('... Options {} are ...', format(..., ', '.join(map(str, unsup)))
Quoted 12 lines
tests/qemu-iotests/iotests.py +8 -0
+            for opt in imgopts:
+                if any(unsupported in opt for unsupported in unsupported_opts):
+                    test_case.case_skip('{}: Option {} is unsupported'.format(
+                                        test_case, opt))
+            return func(test_case, *args, **kwargs)
+        return func_wrapper
+    return skip_test_decorator
+
  def execute_unittest(debug=False):
      """Executes unittests within the calling module."""
 
Signature · 2 lines
Best regards,
Vladimir
author
#8 lore raw

Re: [PATCH 02/67] iotests.py: Add @skip_for_imgopts()

Quoted 22 lines · Vladimir Sementsov-Ogievskiy
Quoted 10 lines · Max Reitz
---
  tests/qemu-iotests/iotests.py | 13 +++++++++++++
  1 file changed, 13 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 7030900807..cdcb62c4ac 100644

tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 957

Quoted 8 lines
tests/qemu-iotests/iotests.py +5 -0
@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False):
          return func_wrapper
      return skip_test_decorator
 
+def skip_for_imgopts(unsupported_opts=[]):
+    '''Skip Test Decorator
+       Skips the test if imgopts contains any of the given options'''
+    def skip_test_decorator(func):
+        def func_wrapper(test_case: QMPTestCase, *args, **kwargs):

how about

Quoted 3 lines

unsup = set(imgopts) & set(unsupported_opts)
if unsup:

    test_case.case_skip('... Options {} are ...', format(..., ', '.join(map(str, unsup)))

Sure.

Max

author
#9 lore raw

PATCH 3/67 iotests.py: Add unsupported_imgopts

In reply to Max Reitz
---
 tests/qemu-iotests/iotests.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +6 -1
index cdcb62c4ac..b5ea424de4 100644
@@ -998,7 +998,8 @@ def execute_setup_common(supported_fmts=[],
                          supported_cache_modes=[],
                          unsupported_fmts=[],
                          supported_protocols=[],
-                         unsupported_protocols=[]):
+                         unsupported_protocols=[],
+                         unsupported_imgopts=[]):
     """
     Perform necessary setup for either script-style or unittest-style tests.
     """
@@ -1016,6 +1017,10 @@ def execute_setup_common(supported_fmts=[],
     verify_platform(supported=supported_platforms)
     verify_cache_mode(supported_cache_modes)
 
+    for opt in imgopts:
+        if any(unsupported in opt for unsupported in unsupported_imgopts):
+            notrun('not suitable for this option: %s' % opt)
+
     debug = '-d' in sys.argv
     if debug:
         sys.argv.remove('-d')
Signature · 1 line
2.21.0
Reviewed-by on v1 3/67
#10 lore raw

Re: [PATCH 03/67] iotests.py: Add unsupported_imgopts

Quoted 20 lines · Max Reitz
---
 tests/qemu-iotests/iotests.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
index cdcb62c4ac..b5ea424de4 100644
@@ -998,7 +998,8 @@ def execute_setup_common(supported_fmts=[],
                          supported_cache_modes=[],
                          unsupported_fmts=[],
                          supported_protocols=[],
-                         unsupported_protocols=[]):
+                         unsupported_protocols=[],
+                         unsupported_imgopts=[]):
     """
     Perform necessary setup for either script-style or unittest-style tests.
     """

tests/qemu-iotests/iotests.py def execute_setup_common(supported_fmts=[], · line 1021

Quoted 5 lines
tests/qemu-iotests/iotests.py +2 -0
@@ -1016,6 +1017,10 @@ def execute_setup_common(supported_fmts=[],
     verify_platform(supported=supported_platforms)
     verify_cache_mode(supported_cache_modes)
 
+    for opt in imgopts:
+        if any(unsupported in opt for unsupported in unsupported_imgopts):

any() is cool and I don't use it as often as I should.

tests/qemu-iotests/iotests.py def execute_setup_common(supported_fmts=[], · line 1026

Quoted 6 lines
tests/qemu-iotests/iotests.py +2 -0
+            notrun('not suitable for this option: %s' % opt)
+
     debug = '-d' in sys.argv
     if debug:
         sys.argv.remove('-d')
#11 lore raw

Re: [PATCH 03/67] iotests.py: Add unsupported_imgopts

In reply to Max Reitz
Quoted 20 lines · Max Reitz
---
  tests/qemu-iotests/iotests.py | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
index cdcb62c4ac..b5ea424de4 100644
@@ -998,7 +998,8 @@ def execute_setup_common(supported_fmts=[],
                           supported_cache_modes=[],
                           unsupported_fmts=[],
                           supported_protocols=[],
-                         unsupported_protocols=[]):
+                         unsupported_protocols=[],
+                         unsupported_imgopts=[]):
      """
      Perform necessary setup for either script-style or unittest-style tests.
      """

tests/qemu-iotests/iotests.py def execute_setup_common(supported_fmts=[], · line 1023

Quoted 7 lines
tests/qemu-iotests/iotests.py +4 -0
@@ -1016,6 +1017,10 @@ def execute_setup_common(supported_fmts=[],
      verify_platform(supported=supported_platforms)
      verify_cache_mode(supported_cache_modes)
 
+    for opt in imgopts:
+        if any(unsupported in opt for unsupported in unsupported_imgopts):
+            notrun('not suitable for this option: %s' % opt)
+

same suggestion

Quoted 4 lines
tests/qemu-iotests/iotests.py +0 -0
      debug = '-d' in sys.argv
      if debug:
          sys.argv.remove('-d')
Signature · 2 lines
Best regards,
Vladimir
author
#12 lore raw

PATCH 4/67 iotests.py: create_test_image, remove_test_image

In reply to Max Reitz

Python tests should use these two new functions instead of qemu_img('create', ...) + os.remove(), so that user-supplied image options are interpreted and handled correctly.

---
 tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
tests/qemu-iotests/iotests.py +56 -0
index b5ea424de4..fce1ab04c9 100644
@@ -122,6 +122,62 @@ def qemu_img_create(*args):
 
     return qemu_img(*args)
 
+def create_test_image(filename, size=None, fmt=imgfmt, opts=[],
+                      backing_file=None, backing_fmt=None,
+                      objects=[], unsafe=False):
+    if fmt == imgfmt:
+        # Only use imgopts for the default format
+        opts = imgopts + opts
+
+    for i, opt in enumerate(opts):
+        if '$TEST_IMG' in opt:
+            opts[i] = opt.replace('$TEST_IMG', filename)
+
+    # default luks support
+    if fmt == 'luks':
+        if not any('key-secret' in opt for opt in opts):
+            opts.append(luks_default_key_secret_opt)
+        objects.append(luks_default_secret_object)
+
+    args = ['create', '-f', fmt]
+
+    if len(opts) > 0:
+        args += ['-o', ','.join(opts)]
+
+    if backing_file is not None:
+        args += ['-b', backing_file]
+
+    if backing_fmt is not None:
+        args += ['-F', backing_fmt]
+
+    if len(objects) > 0:
+        # Generate a [['--object', $obj], [...], ...] list and flatten it
+        args += [arg for objarg in (['--object', obj] for obj in objects) \
+                     for arg in objarg]
+
+    if unsafe:
+        args.append('-u')
+
+    args.append(filename)
+    if size is not None:
+        args.append(str(size))
+
+    return qemu_img(*args)
+
+# Use this to remove images create with create_test_image in the
+# default image format (iotests.imgfmt)
+def remove_test_image(filename):
+    try:
+        os.remove(filename)
+
+        data_file = next(opt.replace('data_file=', '') \
+                            .replace('$TEST_IMG', filename) \
+                         for opt in imgopts if opt.startswith('data_file='))
+
+        os.remove(data_file)
+    except:
+        pass
+
 def qemu_img_verbose(*args):
     '''Run qemu-img without suppressing its output and return the exit code'''
     exitcode = subprocess.call(qemu_img_args + list(args))
Signature · 1 line
2.21.0
#13 lore raw

Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image

Quoted 14 lines · Max Reitz

Python tests should use these two new functions instead of qemu_img('create', ...) + os.remove(), so that user-supplied image options are interpreted and handled correctly.

---
 tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index b5ea424de4..fce1ab04c9 100644

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 127

Quoted 6 lines
tests/qemu-iotests/iotests.py +3 -0
@@ -122,6 +122,62 @@ def qemu_img_create(*args):
 
     return qemu_img(*args)
 
+def create_test_image(filename, size=None, fmt=imgfmt, opts=[],
+                      backing_file=None, backing_fmt=None,
+                      objects=[], unsafe=False):

Python! It's the language that everybody loves and can do no wrong!

Ah, wait, no, maybe the opposite.

You want this:

(..., opts=None, ...):

    opts = opts or []

because, unfortunately, default parameters are bound at definition time and not at call time, so the default list here is like a static local.

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 138

Quoted 11 lines
tests/qemu-iotests/iotests.py +11 -0
+    if fmt == imgfmt:
+        # Only use imgopts for the default format
+        opts = imgopts + opts
+
+    for i, opt in enumerate(opts):
+        if '$TEST_IMG' in opt:
+            opts[i] = opt.replace('$TEST_IMG', filename)
+
+    # default luks support
+    if fmt == 'luks':
+        if not any('key-secret' in opt for opt in opts):

You can write "if not 'key-secret' in opts"

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 139

Quoted 1 line
tests/qemu-iotests/iotests.py +1 -0
+            opts.append(luks_default_key_secret_opt)

And here we might modify that default list.

Quoted 7 lines
tests/qemu-iotests/iotests.py +7 -0
+        objects.append(luks_default_secret_object)
+
+    args = ['create', '-f', fmt]
+
+    if len(opts) > 0:
+        args += ['-o', ','.join(opts)]
+

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 156

Quoted 10 lines
tests/qemu-iotests/iotests.py +10 -0
+    if backing_file is not None:
+        args += ['-b', backing_file]
+
+    if backing_fmt is not None:
+        args += ['-F', backing_fmt]
+
+    if len(objects) > 0:
+        # Generate a [['--object', $obj], [...], ...] list and flatten it
+        args += [arg for objarg in (['--object', obj] for obj in objects) \
+                     for arg in objarg]

I may have mentioned at one point that I love comprehensions, but dislike nested comprehensions. At this point, I think it's far simpler to say:

for obj in objects:

    args.extend(['--object', obj])

or, even shorter:

    args += ['--object', obj]

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 167

Quoted 11 lines
tests/qemu-iotests/iotests.py +11 -0
+
+    if unsafe:
+        args.append('-u')
+
+    args.append(filename)
+    if size is not None:
+        args.append(str(size))
+
+    return qemu_img(*args)
+
+# Use this to remove images create with create_test_image in the

created

and you might as well move the # comment to a """docstring""" while you're here.

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 176

Quoted 9 lines
tests/qemu-iotests/iotests.py +9 -0
+# default image format (iotests.imgfmt)
+def remove_test_image(filename):
+    try:
+        os.remove(filename)
+
+        data_file = next(opt.replace('data_file=', '') \
+                            .replace('$TEST_IMG', filename) \
+                         for opt in imgopts if opt.startswith('data_file='))
+

Learned something today: you can use next() to get the first value from a generator expression.

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 177

Quoted 1 line
tests/qemu-iotests/iotests.py +1 -0
+        os.remove(data_file)

Keep in mind that if the generator expression returns no results, that next() will throw an exception and we won't make it here. That's ok, but,

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 180

Quoted 3 lines
tests/qemu-iotests/iotests.py +3 -0
+    except:
+        pass
+

The unqualified except doesn't help me know which errors you expected and which you didn't.

We have a function like this elsewhere in the python directory:

def remove_if_exists(filename):
try:
os.remove(filename)
except FileNotFoundError:
pass

Can we use that here and remove the try:/except: from this function? It will require you to change the list search to something like this instead:

remove_if_exists(filename)
for opt in (x for x in imgopts if etc):

    data_file = opt.replace('etc', 'etc')
    remove_if_exists(data_file)

to avoid the exception when you call next().

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 183

Quoted 4 lines
tests/qemu-iotests/iotests.py +0 -0
 def qemu_img_verbose(*args):
     '''Run qemu-img without suppressing its output and return the exit code'''
     exitcode = subprocess.call(qemu_img_args + list(args))

My fussiness with the remove() function is just optional picky stuff, but the rest matters, I think.

--js

#14 lore raw

Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image

Quoted 1 line · John Snow

You can write "if not 'key-secret' in opts"

Ah nah, because it's key-secret= whatever and I'm just being too hasty with what I thought is actually in the opts list.

Carry on ...

author
#15 lore raw

Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image

In reply to John Snow
Quoted 28 lines · John Snow
Quoted 14 lines · Max Reitz

Python tests should use these two new functions instead of qemu_img('create', ...) + os.remove(), so that user-supplied image options are interpreted and handled correctly.

---
 tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index b5ea424de4..fce1ab04c9 100644

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 127

Quoted 6 lines
tests/qemu-iotests/iotests.py +3 -0
@@ -122,6 +122,62 @@ def qemu_img_create(*args):
 
     return qemu_img(*args)
 
+def create_test_image(filename, size=None, fmt=imgfmt, opts=[],
+                      backing_file=None, backing_fmt=None,
+                      objects=[], unsafe=False):

Python! It's the language that everybody loves and can do no wrong!

Ah, wait, no, maybe the opposite.

Quoted 7 lines

You want this:

(..., opts=None, ...):

    opts = opts or []

because, unfortunately, default parameters are bound at definition time and not at call time, so the default list here is like a static local.

OK. Interesting.

I suppose the same goes for @objects, then.

Quoted 5 lines
Quoted 5 lines
tests/qemu-iotests/iotests.py +5 -0
+    if fmt == imgfmt:
+        # Only use imgopts for the default format
+        opts = imgopts + opts
+
+    for i, opt in enumerate(opts):
Quoted 8 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 138

Quoted 6 lines
tests/qemu-iotests/iotests.py +6 -0
+        if '$TEST_IMG' in opt:
+            opts[i] = opt.replace('$TEST_IMG', filename)
+
+    # default luks support
+    if fmt == 'luks':
+        if not any('key-secret' in opt for opt in opts):

You can write "if not 'key-secret' in opts"

Oh, that’s recursive?

Quoted 16 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 139

Quoted 1 line
tests/qemu-iotests/iotests.py +1 -0
+            opts.append(luks_default_key_secret_opt)

And here we might modify that default list.

Quoted 12 lines
tests/qemu-iotests/iotests.py +12 -0
+        objects.append(luks_default_secret_object)
+
+    args = ['create', '-f', fmt]
+
+    if len(opts) > 0:
+        args += ['-o', ','.join(opts)]
+
+    if backing_file is not None:
+        args += ['-b', backing_file]
+
+    if backing_fmt is not None:
+        args += ['-F', backing_fmt]
Quoted 8 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 156

Quoted 5 lines
tests/qemu-iotests/iotests.py +5 -0
+
+    if len(objects) > 0:
+        # Generate a [['--object', $obj], [...], ...] list and flatten it
+        args += [arg for objarg in (['--object', obj] for obj in objects) \
+                     for arg in objarg]

I may have mentioned at one point that I love comprehensions, but dislike nested comprehensions.

I can’t remember but I do remember writing this piece of code, being sad that there is no .flatten, and wanting everyone to see the monster that arises.

Quoted 8 lines

At this point, I think it's far simpler
to say:

for obj in objects:

    args.extend(['--object', obj])

or, even shorter:

    args += ['--object', obj]

OK, so now you saw it, I’m glad to make the flattening more flattering to read.

Quoted 17 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 167

Quoted 11 lines
tests/qemu-iotests/iotests.py +11 -0
+
+    if unsafe:
+        args.append('-u')
+
+    args.append(filename)
+    if size is not None:
+        args.append(str(size))
+
+    return qemu_img(*args)
+
+# Use this to remove images create with create_test_image in the

created

and you might as well move the # comment to a """docstring""" while you're here.

Quoted 12 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 176

Quoted 9 lines
tests/qemu-iotests/iotests.py +9 -0
+# default image format (iotests.imgfmt)
+def remove_test_image(filename):
+    try:
+        os.remove(filename)
+
+        data_file = next(opt.replace('data_file=', '') \
+                            .replace('$TEST_IMG', filename) \
+                         for opt in imgopts if opt.startswith('data_file='))
+

Learned something today: you can use next() to get the first value from a generator expression.

I was sad for a bit that Python doesn’t have a find(), but then I noticed this works as well. (Already used extensively in “iotests: Add VM.assert_block_path()” from my “block: Fix check_to_replace_node()” series.)

Quoted 4 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 177

Quoted 1 line
tests/qemu-iotests/iotests.py +1 -0
+        os.remove(data_file)

Keep in mind that if the generator expression returns no results, that next() will throw an exception and we won't make it here. That's ok, but,

I did. If there are no results, it’s good we won’t get here.

This code would be wrong if the next() didn’t throw an exception.

Quoted 6 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 180

Quoted 3 lines
tests/qemu-iotests/iotests.py +3 -0
+    except:
+        pass
+

The unqualified except doesn't help me know which errors you expected and which you didn't.

What I’m expecting: FileNotFound, StopIteration.

But the thing is that I feel like maybe removing a file should always pass, regardless of the exact exception. (I can imagine to be wrong.)

Quoted 7 lines

We have a function like this elsewhere in the python directory:

def remove_if_exists(filename):
try:
os.remove(filename)
except FileNotFoundError:
pass

We do? I can’t find it. I find a _remove_if_exists in machine.py, which I’m not sure whether it’s supposed to be used outside, and it works a bit different, actually (but probably to the same effect).

Quoted 9 lines

Can we use that here and remove the try:/except: from this function? It will require you to change the list search to something like this instead:

remove_if_exists(filename)
for opt in (x for x in imgopts if etc):

    data_file = opt.replace('etc', 'etc')
    remove_if_exists(data_file)

to avoid the exception when you call next().

I don’t know why I’d avoid the exception, though.

This is probably because I don’t like pythonic code, again, but I prefer a next() + exception over a for loop that just iterates once or not at all.

Quoted 7 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 183

Quoted 4 lines
tests/qemu-iotests/iotests.py +0 -0
 def qemu_img_verbose(*args):
     '''Run qemu-img without suppressing its output and return the exit code'''
     exitcode = subprocess.call(qemu_img_args + list(args))

My fussiness with the remove() function is just optional picky stuff, but the rest matters, I think.

OK. Indeed it does!

Max

#16 lore raw

Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image

Quoted 34 lines · Max Reitz
Quoted 33 lines · John Snow
Quoted 14 lines · Max Reitz

Python tests should use these two new functions instead of qemu_img('create', ...) + os.remove(), so that user-supplied image options are interpreted and handled correctly.

---
 tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index b5ea424de4..fce1ab04c9 100644

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 127

Quoted 6 lines
tests/qemu-iotests/iotests.py +3 -0
@@ -122,6 +122,62 @@ def qemu_img_create(*args):
 
     return qemu_img(*args)
 
+def create_test_image(filename, size=None, fmt=imgfmt, opts=[],
+                      backing_file=None, backing_fmt=None,
+                      objects=[], unsafe=False):

Python! It's the language that everybody loves and can do no wrong!

Ah, wait, no, maybe the opposite.

You want this:

(..., opts=None, ...):

    opts = opts or []
Quoted 7 lines
Quoted 2 lines

because, unfortunately, default parameters are bound at definition time and not at call time, so the default list here is like a static local.

OK. Interesting.

I suppose the same goes for @objects, then.

It is by far the WORST thing about Python.

I realize we use this pattern a few places in iotests, but I think it's also usually where we don't modify the list, so it's actually OK, but serves as an example of a bad habit.

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
tests/qemu-iotests/iotests.py +5 -0
+    if fmt == imgfmt:
+        # Only use imgopts for the default format
+        opts = imgopts + opts
+
+    for i, opt in enumerate(opts):
Quoted 8 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 138

Quoted 6 lines
tests/qemu-iotests/iotests.py +6 -0
+        if '$TEST_IMG' in opt:
+            opts[i] = opt.replace('$TEST_IMG', filename)
+
+    # default luks support
+    if fmt == 'luks':
+        if not any('key-secret' in opt for opt in opts):

You can write "if not 'key-secret' in opts"

Quoted 2 lines

Oh, that’s recursive?

No, I was just mistaken about the shape of the data.
You are looking for 'key-secret=XXX', I was thinking that there was a token that was really just 'key-secret'.

What you wrote is correct and good and I am wrong and bad.

Quoted 32 lines
Quoted 16 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 139

Quoted 1 line
tests/qemu-iotests/iotests.py +1 -0
+            opts.append(luks_default_key_secret_opt)

And here we might modify that default list.

Quoted 12 lines
tests/qemu-iotests/iotests.py +12 -0
+        objects.append(luks_default_secret_object)
+
+    args = ['create', '-f', fmt]
+
+    if len(opts) > 0:
+        args += ['-o', ','.join(opts)]
+
+    if backing_file is not None:
+        args += ['-b', backing_file]
+
+    if backing_fmt is not None:
+        args += ['-F', backing_fmt]
Quoted 8 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 156

Quoted 5 lines
tests/qemu-iotests/iotests.py +5 -0
+
+    if len(objects) > 0:
+        # Generate a [['--object', $obj], [...], ...] list and flatten it
+        args += [arg for objarg in (['--object', obj] for obj in objects) \
+                     for arg in objarg]

I may have mentioned at one point that I love comprehensions, but dislike nested comprehensions.

I can’t remember but I do remember writing this piece of code, being sad that there is no .flatten, and wanting everyone to see the monster that arises.

Quoted 3 lines

At this point, I think it's far simpler
to say:

Quoted 9 lines
Quoted 5 lines

for obj in objects:

    args.extend(['--object', obj])

or, even shorter:

    args += ['--object', obj]

OK, so now you saw it, I’m glad to make the flattening more flattering to read.

I am sorry I ever mentioned liking Python. I will accept your punishments.

Quoted 30 lines
Quoted 17 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 167

Quoted 11 lines
tests/qemu-iotests/iotests.py +11 -0
+
+    if unsafe:
+        args.append('-u')
+
+    args.append(filename)
+    if size is not None:
+        args.append(str(size))
+
+    return qemu_img(*args)
+
+# Use this to remove images create with create_test_image in the

created

and you might as well move the # comment to a """docstring""" while you're here.

Quoted 12 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 176

Quoted 9 lines
tests/qemu-iotests/iotests.py +9 -0
+# default image format (iotests.imgfmt)
+def remove_test_image(filename):
+    try:
+        os.remove(filename)
+
+        data_file = next(opt.replace('data_file=', '') \
+                            .replace('$TEST_IMG', filename) \
+                         for opt in imgopts if opt.startswith('data_file='))
+

Learned something today: you can use next() to get the first value from a generator expression.

Quoted 5 lines

I was sad for a bit that Python doesn’t have a find(), but then I noticed this works as well. (Already used extensively in “iotests: Add VM.assert_block_path()” from my “block: Fix check_to_replace_node()” series.)

I honestly tried to rewrite this a few times because it looks so chunky, but realized there isn't ... a great way to do this without implying that you might find more than one result.

You can filter to a new list and assert that the length is one, but that's not less chunky.

Quoted 9 lines
Quoted 4 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 177

Quoted 1 line
tests/qemu-iotests/iotests.py +1 -0
+        os.remove(data_file)

Keep in mind that if the generator expression returns no results, that next() will throw an exception and we won't make it here. That's ok, but,

I did. If there are no results, it’s good we won’t get here.

This code would be wrong if the next() didn’t throw an exception.

It just wasn't clear, because the except is doing the lifting for both the remove and the finding.

Oh well, it's not really important.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
tests/qemu-iotests/iotests.py +3 -0
+    except:
+        pass
+
Quoted 8 lines
Quoted 2 lines

The unqualified except doesn't help me know which errors you expected and which you didn't.

What I’m expecting: FileNotFound, StopIteration.

But the thing is that I feel like maybe removing a file should always pass, regardless of the exact exception. (I can imagine to be wrong.)

I wonder if that's true ... I just don't know what the full set of errors we might get are. I don't really like exception driven code, honestly.

"It's wrong to catch ANY exception because you might suppress errors too broadly."

"It's wrong to be too specific, because you'll miss cases you meant to catch."

Awful.

Anyway, like I said I was just being fiddly because I found this odd to read, but really don't have suggestions that are clearly nicer, so ... carry on.

Quoted 8 lines
Quoted 7 lines

We have a function like this elsewhere in the python directory:

def remove_if_exists(filename):
try:
os.remove(filename)
except FileNotFoundError:
pass

Quoted 4 lines

We do? I can’t find it. I find a _remove_if_exists in machine.py, which I’m not sure whether it’s supposed to be used outside, and it works a bit different, actually (but probably to the same effect).

Yeah, that's the one. Don't worry about plucking it out here for this, just nothing that we do this in a few places. We might want a util eventually that gets it exactly right.

Or not, because what's "exactly right" anyway. Ah, ah, ah.

Quoted 8 lines
Quoted 8 lines

Can we use that here and remove the try:/except: from this function? It will require you to change the list search to something like this instead:

remove_if_exists(filename)
for opt in (x for x in imgopts if etc):

    data_file = opt.replace('etc', 'etc')
    remove_if_exists(data_file)
Quoted 7 lines
Quoted 1 line

to avoid the exception when you call next().

I don’t know why I’d avoid the exception, though.

This is probably because I don’t like pythonic code, again, but I prefer a next() + exception over a for loop that just iterates once or not at all.

Nah, Python people LOVE exceptions. They don't like "bare except" statements, though. I am the weird person in that I like to avoid exceptions whenever it's elegant and pretty to do so.

I find exceptions as normal control flow to be quite hard to deal with; but Pythonistas seem to love it.

Quoted 12 lines
Quoted 7 lines

tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 183

Quoted 4 lines
tests/qemu-iotests/iotests.py +0 -0
 def qemu_img_verbose(*args):
     '''Run qemu-img without suppressing its output and return the exit code'''
     exitcode = subprocess.call(qemu_img_args + list(args))

My fussiness with the remove() function is just optional picky stuff, but the rest matters, I think.

OK. Indeed it does!

Max

author
#17 lore raw

PATCH 5/67 iotests.py: Add ImagePaths

In reply to Max Reitz

create_test_image() must be paired with an ImagePath so that the image is properly cleaned up with remove_test_image() instead of just os.remove().

---
 tests/qemu-iotests/iotests.py | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)
tests/qemu-iotests/iotests.py +30 -2
index fce1ab04c9..5be6ca674c 100644
@@ -440,8 +440,8 @@ class FilePaths(object):
     Use this context manager to generate filenames and ensure that the file
     gets deleted::
 
-        with FilePaths(['test.img']) as img_path:
-            qemu_img('create', img_path, '1G')
+        with FilePaths(['migration.sock']) as migration_sock_path:
+            # Set up and use UNIX socket on migration.sock
         # migration_sock_path is automatically deleted
     """
     def __init__(self, names):
@@ -460,6 +460,23 @@ class FilePaths(object):
             pass
         return False
 
+class ImagePaths(FilePaths):
+    """
+    Same as FilePaths, except it calls remove_test_image() to clean up
+    (which ensures that external data files are cleaned up, too).
+
+    Use this class for test images in the default format
+    (iotests.imgfmt):
+
+    with ImagePaths(['test.img']) as img_path:
+        create_test_image(img_path, '1G')
+    # The test image is automatically cleaned up
+    """
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        for path in self.paths:
+            remove_test_image(path)
+        return False
+
 class FilePath(FilePaths):
     """
     FilePath is a specialization of FilePaths that takes a single filename.
@@ -470,6 +487,17 @@ class FilePath(FilePaths):
     def __enter__(self):
         return self.paths[0]
 
+class ImagePath(ImagePaths):
+    """
+    ImagePath is a specialization of ImagePaths that takes a single
+    filename.
+    """
+    def __init__(self, name):
+        super(ImagePath, self).__init__([name])
+
+    def __enter__(self):
+        return self.paths[0]
+
 def file_path_remover():
     for path in reversed(file_path_remover.paths):
         try:
Signature · 1 line
2.21.0
author
#18 lore raw

PATCH 6/67 iotests.py: Add image_path()

In reply to Max Reitz

Just like we have file_path() as an alternative to FilePath, this is an alternative fo ImagePath.

---
 tests/qemu-iotests/iotests.py | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
tests/qemu-iotests/iotests.py +12 -5
index 5be6ca674c..280e6c2ec2 100644
@@ -505,8 +505,12 @@ def file_path_remover():
         except OSError:
             pass
 
+def image_path_remover():
+    for path in reversed(image_path_remover.paths):
+        remove_test_image(path)
 
-def file_path(*names):
+
+def file_path(*names, remover=file_path_remover):
     ''' Another way to get auto-generated filename that cleans itself up.
 
     Use is as simple as:
@@ -515,19 +519,22 @@ def file_path(*names):
     sock = file_path('socket')
     '''
 
-    if not hasattr(file_path_remover, 'paths'):
-        file_path_remover.paths = []
-        atexit.register(file_path_remover)
+    if not hasattr(remover, 'paths'):
+        remover.paths = []
+        atexit.register(remover)
 
     paths = []
     for name in names:
         filename = file_pattern(name)
         path = os.path.join(test_dir, filename)
-        file_path_remover.paths.append(path)
+        remover.paths.append(path)
         paths.append(path)
 
     return paths[0] if len(paths) == 1 else paths
 
+def image_path(*names):
+    return file_path(*names, remover=image_path_remover)
+
 def remote_filename(path):
     if imgproto == 'file':
         return path
Signature · 1 line
2.21.0
author
#19 lore raw

PATCH 7/67 iotests.py: Filter data_file in filter_img_info

In reply to Max Reitz

If $IMGOPTS contains a data_file reference, let filter_img_info filter all data_file information.

---
 tests/qemu-iotests/iotests.py | 4 ++++
 1 file changed, 4 insertions(+)
tests/qemu-iotests/iotests.py +4 -0
index 280e6c2ec2..fb0a49372d 100644
@@ -380,9 +380,13 @@ def filter_generated_node_ids(msg):
 
 def filter_img_info(output, filename):
     lines = []
+    user_data_file = any('data_file' in opt for opt in imgopts)
     for line in output.split('\n'):
         if 'disk size' in line or 'actual-size' in line:
             continue
+        if user_data_file:
+            if 'data file' in line:
+                continue
         line = line.replace(filename, 'TEST_IMG') \
                    .replace(imgfmt, 'IMGFMT')
         line = re.sub('iters: [0-9]+', 'iters: XXX', line)
Signature · 1 line
2.21.0
author
#20 lore raw

PATCH 8/67 iotests.py: Add filter_json_filename()

In reply to Max Reitz
---
 tests/qemu-iotests/iotests.py | 3 +++
 1 file changed, 3 insertions(+)
tests/qemu-iotests/iotests.py +3 -0
index fb0a49372d..9737dd881b 100644
@@ -378,6 +378,9 @@ def filter_qmp_testfiles(qmsg):
 def filter_generated_node_ids(msg):
     return re.sub("#block[0-9]+", "NODE_NAME", msg)
 
+def filter_json_filename(msg):
+    return re.sub('json:{.*}', 'json:{ /* filtered */ }', msg)
+
 def filter_img_info(output, filename):
     lines = []
     user_data_file = any('data_file' in opt for opt in imgopts)
Signature · 1 line
2.21.0
author
#21 lore raw

PATCH 9/67 iotests.py: Add @hide_fields to img_info_log

In reply to Max Reitz

You can specify fields here that should be hidden from the output.

---
 tests/qemu-iotests/iotests.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
tests/qemu-iotests/iotests.py +9 -2
index 9737dd881b..f3c80ba4e9 100644
@@ -201,7 +201,8 @@ def qemu_img_log(*args):
     log(result, filters=[filter_testfiles])
     return result
 
-def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):
+def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[],
+                 hide_fields=[]):
     args = [ 'info' ]
     if imgopts:
         args.append('--image-opts')
@@ -210,7 +211,13 @@ def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):
     args += extra_args
     args.append(filename)
 
-    output = qemu_img_pipe(*args)
+    output = qemu_img_pipe(*args).splitlines(keepends=True)
+
+    output = [line for line in output
+              if not any(line.strip().startswith(field)
+                         for field in hide_fields)]
+    output = ''.join(output)
+
     if not filter_path:
         filter_path = filename
     log(filter_img_info(output, filter_path))
Signature · 1 line
2.21.0
author
#22 lore raw

PATCH 10/67 iotests/169: Skip persistent cases for compat=0.10

In reply to Max Reitz

compat=0.10 images cannot store persistent dirty bitmaps; let 169 skip all such test cases when compat=0.10 is in the $IMGOPTS.

---
 tests/qemu-iotests/169 | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
tests/qemu-iotests/169 +12 -1
index 8c204caf20..5e978b7b7d 100755
@@ -41,7 +41,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
         self.vm_b.shutdown()
         os.remove(disk_a)
         os.remove(disk_b)
-        os.remove(mig_file)
+        try:
+            os.remove(mig_file)
+        except OSError:
+            pass
 
     def setUp(self):
         qemu_img('create', '-f', iotests.imgfmt, disk_a, size)
@@ -79,6 +82,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
     def do_test_migration_resume_source(self, persistent, migrate_bitmaps):
         granularity = 512
 
+        if persistent and 'compat=0.10' in iotests.imgopts:
+            self.case_skip('compat=0.10 does not support persistent dirty ' +
+                           'bitmaps')
+
         # regions = ((start, count), ...)
         regions = ((0, 0x10000),
                    (0xf0000, 0x10000),
@@ -137,6 +144,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
                           shared_storage):
         granularity = 512
 
+        if persistent and 'compat=0.10' in iotests.imgopts:
+            self.case_skip('compat=0.10 does not support persistent dirty ' +
+                           'bitmaps')
+
         # regions = ((start, count), ...)
         regions = ((0, 0x10000),
                    (0xf0000, 0x10000),
Signature · 1 line
2.21.0
author
#23 lore raw

PATCH 11/67 iotests/224: Filter json:{} from commit command

In reply to Max Reitz

We only care that block-commit works with json:{} filenames, we do not need to see their exact values in the output.

---
 tests/qemu-iotests/224     | 15 ++++++++++++---
 tests/qemu-iotests/224.out |  4 ++--
 2 files changed, 14 insertions(+), 5 deletions(-)
tests/qemu-iotests/224 +12 -3
index d0d0c44104..e227d3ab3c 100755
@@ -22,12 +22,19 @@
 
 import iotests
 from iotests import log, qemu_img, qemu_io_silent, filter_qmp_testfiles, \
-                    filter_qmp_imgfmt
+                    filter_qmp_imgfmt, filter_json_filename
 import json
+import re
 
 # Need backing file support (for arbitrary backing formats)
 iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed'])
 
+def filter_json_top_or_base(qmsg):
+    def _filter(key, value):
+        if key == 'base' or key == 'top':
+            return filter_json_filename(value)
+        return value
+    return iotests.filter_qmp(qmsg, _filter)
 
 # There are two variations of this test:
 # (1) We do not set filter_node_name.  In that case, the commit_top
@@ -104,7 +111,8 @@ for filter_node_name in False, True:
                         top=mid_name,
                         base=base_name,
                         speed=1,
-                        filters=[filter_qmp_testfiles, filter_qmp_imgfmt])
+                        filters=[filter_qmp_testfiles, filter_qmp_imgfmt,
+                                 filter_json_top_or_base])
         else:
             vm.qmp_log('block-commit',
                         job_id='commit',
@@ -112,7 +120,8 @@ for filter_node_name in False, True:
                         top=mid_name,
                         base=base_name,
                         speed=1,
-                        filters=[filter_qmp_testfiles, filter_qmp_imgfmt])
+                        filters=[filter_qmp_testfiles, filter_qmp_imgfmt,
+                                 filter_json_top_or_base])
 
         vm.qmp_log('job-pause', id='commit')
 
tests/qemu-iotests/224.out +2 -2
index 23374a1d29..287d2eac7a 100644
@@ -3,7 +3,7 @@
 
 {"execute": "blockdev-add", "arguments": {"backing": {"backing": {"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}, "node-name": "base"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-mid.img"}, "node-name": "mid"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "top"}}
 {"return": {}}
-{"execute": "block-commit", "arguments": {"base": "json:{\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}", "device": "top", "job-id": "commit", "speed": 1, "top": "json:{\"backing\": {\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-mid.img\"}}"}}
+{"execute": "block-commit", "arguments": {"base": "json:{ /* filtered */ }", "device": "top", "job-id": "commit", "speed": 1, "top": "json:{ /* filtered */ }"}}
 {"return": {}}
 {"execute": "job-pause", "arguments": {"id": "commit"}}
 {"return": {}}
@@ -12,7 +12,7 @@
 
 {"execute": "blockdev-add", "arguments": {"backing": {"backing": {"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}, "node-name": "base"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-mid.img"}, "node-name": "mid"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "top"}}
 {"return": {}}
-{"execute": "block-commit", "arguments": {"base": "json:{\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}", "device": "top", "filter-node-name": "filter_node", "job-id": "commit", "speed": 1, "top": "json:{\"backing\": {\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-mid.img\"}}"}}
+{"execute": "block-commit", "arguments": {"base": "json:{ /* filtered */ }", "device": "top", "filter-node-name": "filter_node", "job-id": "commit", "speed": 1, "top": "json:{ /* filtered */ }"}}
 {"return": {}}
 {"execute": "job-pause", "arguments": {"id": "commit"}}
 {"return": {}}
Signature · 1 line
2.21.0
author
#24 lore raw

PATCH 12/67 iotests/228: Filter json:{} filenames

In reply to Max Reitz

We only care about where we get json:{} filenames (and what the non-json:{} filenames look like), but not about how the json:{} filenames look.

Filter them so we will not get a different output for qcow2 images with external data files.

---
 tests/qemu-iotests/228     | 8 ++++----
 tests/qemu-iotests/228.out | 8 ++++----
 2 files changed, 8 insertions(+), 8 deletions(-)
tests/qemu-iotests/228 +4 -4
index 9785868ab3..e507db4f27 100755
@@ -22,7 +22,7 @@
 
 import iotests
 from iotests import log, qemu_img, filter_testfiles, filter_imgfmt, \
-        filter_qmp_testfiles, filter_qmp_imgfmt
+        filter_qmp_testfiles, filter_qmp_imgfmt, filter_json_filename
 
 # Need backing file and change-backing-file support
 iotests.script_initialize(supported_fmts=['qcow2', 'qed'])
@@ -32,14 +32,14 @@ def log_node_info(node):
     log('')
 
     log('bs->filename: ' + node['image']['filename'],
-        filters=[filter_testfiles, filter_imgfmt])
+        filters=[filter_testfiles, filter_imgfmt, filter_json_filename])
     log('bs->backing_file: ' + node['backing_file'],
-        filters=[filter_testfiles, filter_imgfmt])
+        filters=[filter_testfiles, filter_imgfmt, filter_json_filename])
 
     if 'backing-image' in node['image']:
         log('bs->backing->bs->filename: ' +
             node['image']['backing-image']['filename'],
-            filters=[filter_testfiles, filter_imgfmt])
+            filters=[filter_testfiles, filter_imgfmt, filter_json_filename])
     else:
         log('bs->backing: (none)')
 
tests/qemu-iotests/228.out +4 -4
index 4217df24fe..f8155019d5 100644
@@ -17,7 +17,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img
 {"execute": "change-backing-file", "arguments": {"backing-file": "null-co://", "device": "node0", "image-node-name": "node0"}}
 {"return": {}}
 
-bs->filename: json:{"backing": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}}
+bs->filename: json:{ /* filtered */ }
 bs->backing_file: null-co://
 bs->backing->bs->filename: TEST_DIR/PID-base.img
 
@@ -31,7 +31,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img
 {"execute": "change-backing-file", "arguments": {"backing-file": "file:TEST_DIR/PID-base.img", "device": "node0", "image-node-name": "node0"}}
 {"return": {}}
 
-bs->filename: json:{"backing": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}}
+bs->filename: json:{ /* filtered */ }
 bs->backing_file: file:TEST_DIR/PID-base.img
 bs->backing->bs->filename: TEST_DIR/PID-base.img
 
@@ -54,7 +54,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img
 {"execute": "blockdev-add", "arguments": {"backing": "null", "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "node0"}}
 {"return": {}}
 
-bs->filename: json:{"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}}
+bs->filename: json:{ /* filtered */ }
 bs->backing_file: null-co://
 bs->backing->bs->filename: null-co://
 
@@ -65,7 +65,7 @@ bs->backing->bs->filename: null-co://
 {"execute": "blockdev-add", "arguments": {"backing": null, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "node0"}}
 {"return": {}}
 
-bs->filename: json:{"backing": null, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}}
+bs->filename: json:{ /* filtered */ }
 bs->backing_file: TEST_DIR/PID-base.img
 bs->backing: (none)
 
Signature · 1 line
2.21.0
author
#25 lore raw

PATCH 13/67 iotests/242: Hide refcount bit information

In reply to Max Reitz

It does not matter, and this lets the test work with different refcount widths.

---
 tests/qemu-iotests/242     | 6 +++---
 tests/qemu-iotests/242.out | 5 -----
 2 files changed, 3 insertions(+), 8 deletions(-)
tests/qemu-iotests/242 +3 -3
index 7c2685b4cc..21f69a0d2c 100755
@@ -35,7 +35,7 @@ flag_offset = 0x5000f
 
 def print_bitmap(extra_args):
     log('qemu-img info dump:\n')
-    img_info_log(disk, extra_args=extra_args)
+    img_info_log(disk, extra_args=extra_args, hide_fields=['refcount bits'])
     result = json.loads(qemu_img_pipe('info', '--force-share',
                                       '--output=json', disk))
     if 'bitmaps' in result['format-specific']['data']:
@@ -98,9 +98,9 @@ add_bitmap(1, True, False)
 log('Write an unknown bitmap flag \'{}\' into a new QCOW2 image at offset {}'
     .format(hex(bitmap_flag_unknown), flag_offset))
 toggle_flag(flag_offset)
-img_info_log(disk)
+img_info_log(disk, hide_fields=['refcount bits'])
 toggle_flag(flag_offset)
 log('Unset the unknown bitmap flag \'{}\' in the bitmap directory entry:\n'
     .format(hex(bitmap_flag_unknown)))
-img_info_log(disk)
+img_info_log(disk, hide_fields=['refcount bits'])
 log('Test complete')
tests/qemu-iotests/242.out +0 -5
index 7ac8404d11..112b389241 100644
@@ -13,7 +13,6 @@ cluster_size: 65536
 Format specific information:
     compat: 1.1
     lazy refcounts: false
-    refcount bits: 16
     corrupt: false
 
 No bitmap in JSON format output
@@ -38,7 +37,6 @@ Format specific information:
             flags:
             name: bitmap-1
             granularity: 32768
-    refcount bits: 16
     corrupt: false
 
 The same bitmaps in JSON format:
@@ -75,7 +73,6 @@ Format specific information:
                 [0]: auto
             name: bitmap-2
             granularity: 65536
-    refcount bits: 16
     corrupt: false
 
 The same bitmaps in JSON format:
@@ -117,7 +114,6 @@ Format specific information:
                 [1]: auto
             name: bitmap-2
             granularity: 65536
-    refcount bits: 16
     corrupt: false
 
 The same bitmaps in JSON format:
@@ -160,7 +156,6 @@ Format specific information:
                 [0]: auto
             name: bitmap-0
             granularity: 16384
-    refcount bits: 16
     corrupt: false
 
 Test complete
Signature · 1 line
2.21.0
author
#26 lore raw

PATCH 14/67 iotests: Use unsupported_imgopts in Python tests

In reply to Max Reitz

Set unsupported_imgopts as appropriate for all tests that should make use of it.

---
 tests/qemu-iotests/044 | 5 ++++-
 tests/qemu-iotests/057 | 4 +++-
 tests/qemu-iotests/065 | 4 +++-
 tests/qemu-iotests/163 | 5 ++++-
 tests/qemu-iotests/165 | 4 +++-
 tests/qemu-iotests/196 | 4 +++-
 tests/qemu-iotests/206 | 5 ++++-
 tests/qemu-iotests/222 | 3 ++-
 tests/qemu-iotests/237 | 4 +++-
 tests/qemu-iotests/242 | 4 +++-
 tests/qemu-iotests/245 | 4 +++-
 tests/qemu-iotests/246 | 5 ++++-
 tests/qemu-iotests/248 | 6 +++++-
 tests/qemu-iotests/254 | 4 +++-
 tests/qemu-iotests/257 | 4 +++-
 15 files changed, 50 insertions(+), 15 deletions(-)
tests/qemu-iotests/044 +4 -1
index 8b2afa2a11..97ba98e628 100755
@@ -116,4 +116,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 # These refcount calculations do not work with data_file or
+                 # with any refcount width but the default (of 16)
+                 unsupported_imgopts=['data_file', 'refcount_bits'])
tests/qemu-iotests/057 +3 -1
index 9fbba759b6..f5aa5929dc 100755
@@ -257,4 +257,6 @@ class TestSnapshotDelete(ImageSnapshotTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 # Snapshots do not work with data_file
+                 unsupported_imgopts=['data_file'])
tests/qemu-iotests/065 +3 -1
index 5b21eb96bd..9db9552784 100755
@@ -130,4 +130,6 @@ TestQMP = None
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 # Neither of these options would work with compat=0.10
+                 unsupported_imgopts=['refcount_bits', 'data_file'])
tests/qemu-iotests/163 +4 -1
index d94728e080..92633f1b21 100755
@@ -168,4 +168,7 @@ ShrinkBaseClass = None
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['raw', 'qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 # We want to set our own refcount_bits, and that will
+                 # not work with compat=0.10
+                 unsupported_imgopts=['refcount_bits', 'compat=0.10'])
tests/qemu-iotests/165 +3 -1
index 5650dc7c87..97dd7102c3 100755
@@ -104,4 +104,6 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 # compat=1.1 is needed for dirty bitmaps
+                 unsupported_imgopts=['compat=0.10'])
tests/qemu-iotests/196 +3 -1
index 92fe9244f8..ec4852a19c 100755
@@ -64,4 +64,6 @@ class TestInvalidateAutoclear(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 # compat=1.1 is needed for autoclear flags
+                 unsupported_imgopts=['compat=0.10'])
tests/qemu-iotests/206 +4 -1
index 23ff2f624b..91dd6ee176 100755
@@ -23,7 +23,10 @@
 import iotests
 from iotests import imgfmt
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          # All options are ignored
+                          unsupported_imgopts=['compat=0.10', 'refcount_bits',
+                                               'data_file'])
 
 def blockdev_create(vm, options):
     result = vm.qmp_log('blockdev-create',
tests/qemu-iotests/222 +2 -1
index 6788979ed3..ffa88d4bb9 100644
@@ -25,7 +25,8 @@ import iotests
 from iotests import log, qemu_img, qemu_io, qemu_io_silent
 
 iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk',
-                                          'vhdx', 'raw'])
+                                          'vhdx', 'raw'],
+                          unsupported_imgopts=['streamOptimized'])
 
 patterns = [("0x5d", "0",         "64k"),
             ("0xd5", "1M",        "64k"),
tests/qemu-iotests/237 +3 -1
index 3758ace0bc..34ff4b55db 100755
@@ -24,7 +24,9 @@ import math
 import iotests
 from iotests import imgfmt
 
-iotests.script_initialize(supported_fmts=['vmdk'])
+iotests.script_initialize(supported_fmts=['vmdk'],
+                          # All options are ignored
+                          unsupported_imgopts=['subformat'])
 
 def blockdev_create(vm, options):
     result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
tests/qemu-iotests/242 +3 -1
index 21f69a0d2c..333fb8b56c 100755
@@ -24,7 +24,9 @@ import struct
 from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \
     file_path, img_info_log, log, filter_qemu_io
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          # compat=1.1 is needed for dirty bitmaps
+                          unsupported_imgopts=['compat=0.10'])
 
 disk = file_path('disk')
 chunk = 256 * 1024
tests/qemu-iotests/245 +3 -1
index 50544a7836..72ab2a248d 100644
@@ -1002,4 +1002,6 @@ class TestBlockdevReopen(iotests.QMPTestCase):
 if __name__ == '__main__':
     iotests.activate_logging()
     iotests.main(supported_fmts=["qcow2"],
-                 supported_protocols=["file"])
+                 supported_protocols=["file"],
+                 # reopen options differ with an external data file
+                 unsupported_imgopts=['data_file'])
tests/qemu-iotests/246 +4 -1
index 1d7747d62d..d1364d5901 100755
@@ -22,7 +22,10 @@
 import iotests
 from iotests import log
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          # compat=1.1 is needed for dirty bitmaps
+                          unsupported_imgopts=['compat=0.10'])
+
 size = 64 * 1024 * 1024 * 1024
 gran_small = 32 * 1024
 gran_large = 128 * 1024
tests/qemu-iotests/248 +5 -1
index 781b21b227..0adc4da802 100755
@@ -21,7 +21,11 @@
 import iotests
 from iotests import qemu_img_create, qemu_io, file_path, filter_qmp_testfiles
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          # With an external data file, we would need
+                          # to impose the limit on @data-file instead
+                          # of @file
+                          unsupported_imgopts=['data_file'])
 
 source, target = file_path('source', 'target')
 size = 5 * 1024 * 1024
tests/qemu-iotests/254 +3 -1
index 43b40f4f71..0404faf853 100755
@@ -21,7 +21,9 @@
 import iotests
 from iotests import qemu_img_create, file_path, log
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          # compat=1.1 is needed for dirty bitmaps
+                          unsupported_imgopts=['compat=0.10'])
 
 disk, top = file_path('disk', 'top')
 size = 1024 * 1024
tests/qemu-iotests/257 +3 -1
index de8b45f094..82c10e9b52 100755
@@ -526,4 +526,6 @@ def main():
 
 if __name__ == '__main__':
     iotests.script_main(main, supported_fmts=['qcow2'],
-                        supported_protocols=['file'])
+                        supported_protocols=['file'],
+                        # blkdebug does not work with an external data file
+                        unsupported_imgopts=['data_file'])
Signature · 1 line
2.21.0
author
#27 lore raw

PATCH 15/67 iotests/030: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/030 | 69 ++++++++++++++++++++----------------------
 1 file changed, 32 insertions(+), 37 deletions(-)
tests/qemu-iotests/030 +32 -37
index 01aa96ed16..cddb017496 100755
@@ -21,7 +21,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_io
 
 backing_img = os.path.join(iotests.test_dir, 'backing.img')
 mid_img = os.path.join(iotests.test_dir, 'mid.img')
@@ -32,8 +32,8 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def setUp(self):
         iotests.create_image(backing_img, TestSingleDrive.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img)
+        create_test_image(mid_img, backing_file=backing_img)
+        create_test_image(test_img, backing_file=mid_img)
         qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 512', backing_img)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 524288 512', mid_img)
         self.vm = iotests.VM().add_drive("blkdebug::" + test_img,
@@ -43,8 +43,8 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(mid_img)
+        remove_test_image(test_img)
+        remove_test_image(mid_img)
         os.remove(backing_img)
 
     def test_stream(self):
@@ -155,9 +155,8 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def test_read_only(self):
         # Create a new file that we can attach (we need a read-only top)
-        with iotests.FilePath('ro-top.img') as ro_top_path:
-            qemu_img('create', '-f', iotests.imgfmt, ro_top_path,
-                     str(self.image_len))
+        with iotests.ImagePath('ro-top.img') as ro_top_path:
+            create_test_image(ro_top_path, self.image_len)
 
             result = self.vm.qmp('blockdev-add',
                                  node_name='ro-top',
@@ -198,8 +197,7 @@ class TestParallelOps(iotests.QMPTestCase):
         # Create all images
         iotests.create_image(self.imgs[0], self.image_len)
         for i in range(1, self.num_imgs):
-            qemu_img('create', '-f', iotests.imgfmt,
-                     '-o', 'backing_file=%s' % self.imgs[i-1], self.imgs[i])
+            create_test_image(self.imgs[i], backing_file=self.imgs[i - 1])
 
         # Put data into the images we are copying data from
         odd_img_indexes = [x for x in reversed(range(self.num_imgs)) if x % 2 == 1]
@@ -219,7 +217,7 @@ class TestParallelOps(iotests.QMPTestCase):
     def tearDown(self):
         self.vm.shutdown()
         for img in self.imgs:
-            os.remove(img)
+            remove_test_image(img)
 
     # Test that it's possible to run several block-stream operations
     # in parallel in the same snapshot chain
@@ -539,11 +537,10 @@ class TestQuorum(iotests.QMPTestCase):
             backing_img = os.path.join(iotests.test_dir, 'backing-%d.img' % i)
             self.children.append(child_img)
             self.backing.append(backing_img)
-            qemu_img('create', '-f', iotests.imgfmt, backing_img, '1M')
+            create_test_image(backing_img, '1M')
             qemu_io('-f', iotests.imgfmt,
                     '-c', 'write -P 0x55 0 1024', backing_img)
-            qemu_img('create', '-f', iotests.imgfmt,
-                     '-o', 'backing_file=%s' % backing_img, child_img)
+            create_test_image(child_img, backing_file=backing_img)
             opts.append("children.%d.file.filename=%s" % (i, child_img))
             opts.append("children.%d.node-name=node%d" % (i, i))
 
@@ -555,9 +552,9 @@ class TestQuorum(iotests.QMPTestCase):
     def tearDown(self):
         self.vm.shutdown()
         for img in self.children:
-            os.remove(img)
+            remove_test_image(img)
         for img in self.backing:
-            os.remove(img)
+            remove_test_image(img)
 
     def test_stream_quorum(self):
         if not iotests.supports_quorum():
@@ -587,7 +584,7 @@ class TestSmallerBackingFile(iotests.QMPTestCase):
 
     def setUp(self):
         iotests.create_image(backing_img, self.backing_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img, str(self.image_len))
+        create_test_image(test_img, self.image_len, backing_file=backing_img)
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
@@ -638,16 +635,15 @@ class TestEIO(TestErrors):
         self.blkdebug_file = backing_img + ".blkdebug"
         iotests.create_image(backing_img, TestErrors.image_len)
         self.create_blkdebug_file(self.blkdebug_file, "read_aio", 5)
-        qemu_img('create', '-f', iotests.imgfmt,
-                 '-o', 'backing_file=blkdebug:%s:%s,backing_fmt=raw'
-                       % (self.blkdebug_file, backing_img),
-                 test_img)
+        create_test_image(test_img, backing_fmt='raw',
+                          backing_file=('blkdebug:%s:%s' % (self.blkdebug_file,
+                                                            backing_img)))
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(backing_img)
         os.remove(self.blkdebug_file)
 
@@ -788,16 +784,15 @@ class TestENOSPC(TestErrors):
         self.blkdebug_file = backing_img + ".blkdebug"
         iotests.create_image(backing_img, TestErrors.image_len)
         self.create_blkdebug_file(self.blkdebug_file, "read_aio", 28)
-        qemu_img('create', '-f', iotests.imgfmt,
-                 '-o', 'backing_file=blkdebug:%s:%s,backing_fmt=raw'
-                       % (self.blkdebug_file, backing_img),
-                 test_img)
+        create_test_image(test_img, backing_fmt='raw',
+                          backing_file=('blkdebug:%s:%s' % (self.blkdebug_file,
+                                                            backing_img)))
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(backing_img)
         os.remove(self.blkdebug_file)
 
@@ -848,17 +843,17 @@ class TestStreamStop(iotests.QMPTestCase):
     image_len = 8 * 1024 * 1024 * 1024 # GB
 
     def setUp(self):
-        qemu_img('create', backing_img, str(TestStreamStop.image_len))
-        qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 32M', backing_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(backing_img, TestStreamStop.image_len)
+        qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 32M', backing_img)
+        create_test_image(test_img, backing_file=backing_img)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 32M 32M', test_img)
         self.vm = iotests.VM().add_drive("blkdebug::" + test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
+        remove_test_image(test_img)
+        remove_test_image(backing_img)
 
     def test_stream_stop(self):
         self.assert_no_active_block_jobs()
@@ -879,17 +874,17 @@ class TestSetSpeed(iotests.QMPTestCase):
     image_len = 80 * 1024 * 1024 # MB
 
     def setUp(self):
-        qemu_img('create', backing_img, str(TestSetSpeed.image_len))
-        qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 32M', backing_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(backing_img, TestSetSpeed.image_len)
+        qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 32M', backing_img)
+        create_test_image(test_img, backing_file=backing_img)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 32M 32M', test_img)
         self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
+        remove_test_image(test_img)
+        remove_test_image(backing_img)
 
     # This is a short performance test which is not run by default.
     # Invoke "IMGFMT=qed ./030 TestSetSpeed.perf_test_throughput"
Signature · 1 line
2.21.0
author
#28 lore raw

PATCH 16/67 iotests/040: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/040 | 42 +++++++++++++++++++++---------------------
 1 file changed, 21 insertions(+), 21 deletions(-)
tests/qemu-iotests/040 +21 -21
index 762ad1ebcb..9584eeeb1f 100755
@@ -24,7 +24,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_img, qemu_io
 import struct
 import errno
 
@@ -80,8 +80,8 @@ class TestSingleDrive(ImageCommitTestCase):
 
     def setUp(self):
         iotests.create_image(backing_img, self.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img)
+        create_test_image(mid_img, backing_file=backing_img)
+        create_test_image(test_img, backing_file=mid_img)
         qemu_io('-f', 'raw', '-c', 'write -P 0xab 0 524288', backing_img)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288', mid_img)
         self.vm = iotests.VM().add_drive(test_img, "node-name=top,backing.node-name=mid,backing.backing.node-name=base", interface="none")
@@ -92,8 +92,8 @@ class TestSingleDrive(ImageCommitTestCase):
 
     def tearDown(self):
         self.vm.shutdown(has_quit=self.has_quit)
-        os.remove(test_img)
-        os.remove(mid_img)
+        remove_test_image(test_img)
+        remove_test_image(mid_img)
         os.remove(backing_img)
 
     def test_commit(self):
@@ -303,8 +303,8 @@ class TestRelativePaths(ImageCommitTestCase):
             if exception.errno != errno.EEXIST:
                 raise
         iotests.create_image(self.backing_img_abs, TestRelativePaths.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.backing_img_abs, self.mid_img_abs)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.mid_img_abs, self.test_img)
+        create_test_image(self.mid_img_abs, backing_file=self.backing_img_abs)
+        create_test_image(self.test_img, backing_file=self.mid_img_abs)
         qemu_img('rebase', '-u', '-b', self.backing_img, self.mid_img_abs)
         qemu_img('rebase', '-u', '-b', self.mid_img, self.test_img)
         qemu_io('-f', 'raw', '-c', 'write -P 0xab 0 524288', self.backing_img_abs)
@@ -314,8 +314,8 @@ class TestRelativePaths(ImageCommitTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(self.test_img)
-        os.remove(self.mid_img_abs)
+        remove_test_image(self.test_img)
+        remove_test_image(self.mid_img_abs)
         os.remove(self.backing_img_abs)
         try:
             os.rmdir(os.path.join(iotests.test_dir, self.dir1))
@@ -368,9 +368,9 @@ class TestSetSpeed(ImageCommitTestCase):
     image_len = 80 * 1024 * 1024 # MB
 
     def setUp(self):
-        qemu_img('create', backing_img, str(TestSetSpeed.image_len))
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img)
+        create_test_image(backing_img, TestSetSpeed.image_len)
+        create_test_image(mid_img, backing_file=backing_img)
+        create_test_image(test_img, backing_file=mid_img)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 512', test_img)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288', mid_img)
         self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
@@ -378,9 +378,9 @@ class TestSetSpeed(ImageCommitTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(mid_img)
-        os.remove(backing_img)
+        remove_test_image(test_img)
+        remove_test_image(mid_img)
+        remove_test_image(backing_img)
 
     def test_set_speed(self):
         self.assert_no_active_block_jobs()
@@ -408,9 +408,9 @@ class TestReopenOverlay(ImageCommitTestCase):
 
     def setUp(self):
         iotests.create_image(self.img0, self.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.img0, self.img1)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.img1, self.img2)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.img2, self.img3)
+        create_test_image(self.img1, backing_file=self.img0)
+        create_test_image(self.img2, backing_file=self.img1)
+        create_test_image(self.img3, backing_file=self.img2)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xab 0 128K', self.img1)
         self.vm = iotests.VM().add_drive(self.img3)
         self.vm.launch()
@@ -418,9 +418,9 @@ class TestReopenOverlay(ImageCommitTestCase):
     def tearDown(self):
         self.vm.shutdown()
         os.remove(self.img0)
-        os.remove(self.img1)
-        os.remove(self.img2)
-        os.remove(self.img3)
+        remove_test_image(self.img1)
+        remove_test_image(self.img2)
+        remove_test_image(self.img3)
 
     # This tests what happens when the overlay image of the 'top' node
     # needs to be reopened in read-write mode in order to update the
Signature · 1 line
2.21.0
author
#29 lore raw

PATCH 17/67 iotests/041: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/041 | 108 +++++++++++++++++++----------------------
 1 file changed, 51 insertions(+), 57 deletions(-)
tests/qemu-iotests/041 +51 -57
index 8568426311..5cf2e2cb52 100755
@@ -21,7 +21,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_img, qemu_io
 
 backing_img = os.path.join(iotests.test_dir, 'backing.img')
 target_backing_img = os.path.join(iotests.test_dir, 'target-backing.img')
@@ -41,7 +41,7 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def setUp(self):
         iotests.create_image(backing_img, self.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(test_img, backing_file=backing_img)
         self.vm = iotests.VM().add_drive(test_img, "node-name=top,backing.node-name=base")
         if iotests.qemu_default_machine == 'pc':
             self.vm.add_drive(None, 'media=cdrom', 'ide')
@@ -49,12 +49,9 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(backing_img)
-        try:
-            os.remove(target_img)
-        except OSError:
-            pass
+        remove_test_image(target_img)
 
     def test_complete(self):
         self.assert_no_active_block_jobs()
@@ -138,8 +135,8 @@ class TestSingleDrive(iotests.QMPTestCase):
     def test_small_buffer2(self):
         self.assert_no_active_block_jobs()
 
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=%d,size=%d'
-                        % (self.image_len, self.image_len), target_img)
+        create_test_image(target_img, self.image_len,
+                          opts=['cluster_size=%d' % self.image_len])
         result = self.vm.qmp(self.qmp_cmd, device='drive0', sync='full',
                              buf_size=65536, mode='existing', target=self.qmp_target)
         self.assert_qmp(result, 'return', {})
@@ -154,8 +151,8 @@ class TestSingleDrive(iotests.QMPTestCase):
     def test_large_cluster(self):
         self.assert_no_active_block_jobs()
 
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=%d,backing_file=%s'
-                        % (self.image_len, backing_img), target_img)
+        create_test_image(target_img, backing_file=backing_img,
+                          opts=['cluster_size=%d' % self.image_len])
         result = self.vm.qmp(self.qmp_cmd, device='drive0', sync='full',
                              mode='existing', target=self.qmp_target)
         self.assert_qmp(result, 'return', {})
@@ -227,7 +224,7 @@ class TestSingleBlockdev(TestSingleDrive):
 
     def setUp(self):
         TestSingleDrive.setUp(self)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img)
+        create_test_image(target_img, backing_file=backing_img)
         args = {'driver': iotests.imgfmt,
                 'node-name': self.qmp_target,
                 'file': { 'filename': target_img, 'driver': 'file' } }
@@ -265,24 +262,21 @@ class TestMirrorNoBacking(iotests.QMPTestCase):
 
     def setUp(self):
         iotests.create_image(backing_img, TestMirrorNoBacking.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(test_img, backing_file=backing_img)
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(backing_img)
-        try:
-            os.remove(target_backing_img)
-        except:
-            pass
-        os.remove(target_img)
+        remove_test_image(target_backing_img)
+        remove_test_image(target_img)
 
     def test_complete(self):
         self.assert_no_active_block_jobs()
 
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img)
+        create_test_image(target_img, backing_file=backing_img)
         result = self.vm.qmp('drive-mirror', device='drive0', sync='full',
                              mode='existing', target=target_img)
         self.assert_qmp(result, 'return', {})
@@ -297,7 +291,7 @@ class TestMirrorNoBacking(iotests.QMPTestCase):
     def test_cancel(self):
         self.assert_no_active_block_jobs()
 
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img)
+        create_test_image(target_img, backing_file=backing_img)
         result = self.vm.qmp('drive-mirror', device='drive0', sync='full',
                              mode='existing', target=target_img)
         self.assert_qmp(result, 'return', {})
@@ -313,10 +307,9 @@ class TestMirrorNoBacking(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
 
         # qemu-img create fails if the image is not there
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'size=%d'
-                        %(TestMirrorNoBacking.image_len), target_backing_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=%d,backing_file=%s'
-                        % (TestMirrorNoBacking.image_len, target_backing_img), target_img)
+        create_test_image(target_backing_img, TestMirrorNoBacking.image_len)
+        create_test_image(target_img, backing_file=target_backing_img,
+                          opts=['cluster_size=%d' % TestMirrorNoBacking.image_len])
 
         result = self.vm.qmp('drive-mirror', device='drive0', sync='full',
                              mode='existing', target=target_img)
@@ -335,14 +328,14 @@ class TestMirrorResized(iotests.QMPTestCase):
 
     def setUp(self):
         iotests.create_image(backing_img, TestMirrorResized.backing_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(test_img, backing_file=backing_img)
         qemu_img('resize', test_img, '2M')
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(backing_img)
         try:
             os.remove(target_img)
@@ -407,14 +400,15 @@ new_state = "1"
 ''' % (event, errno, self.MIRROR_GRANULARITY // 512, event, event))
         file.close()
 
+    # blkdebug does not work with an external data file
+    @iotests.skip_for_imgopts(['data_file'])
     def setUp(self):
         self.blkdebug_file = backing_img + ".blkdebug"
         iotests.create_image(backing_img, TestReadErrors.image_len)
         self.create_blkdebug_file(self.blkdebug_file, "read_aio", 5)
-        qemu_img('create', '-f', iotests.imgfmt,
-                 '-o', 'backing_file=blkdebug:%s:%s,backing_fmt=raw'
-                       % (self.blkdebug_file, backing_img),
-                 test_img)
+        create_test_image(test_img, backing_fmt='raw',
+                          backing_file=('blkdebug:%s:%s' % (self.blkdebug_file,
+                                                            backing_img)))
         # Write something for tests that use sync='top'
         qemu_io('-c', 'write %d 512' % (self.MIRROR_GRANULARITY + 65536),
                         test_img)
@@ -423,8 +417,8 @@ new_state = "1"
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(target_img)
+        remove_test_image(test_img)
+        remove_test_image(target_img)
         os.remove(backing_img)
         os.remove(self.blkdebug_file)
 
@@ -483,7 +477,8 @@ new_state = "1"
         # Test COW into the target image.  The first half of the
         # cluster at MIRROR_GRANULARITY has to be copied from
         # backing_img, even though sync='top'.
-        qemu_img('create', '-f', iotests.imgfmt, '-ocluster_size=131072,backing_file=%s' %(backing_img), target_img)
+        create_test_image(target_img, backing_file=backing_img,
+                          opts=['cluster_size=131072'])
         result = self.vm.qmp('drive-mirror', device='drive0', sync='top',
                              on_source_error='ignore',
                              mode='existing', target=target_img)
@@ -572,20 +567,22 @@ new_state = "1"
 ''' % (event, errno, self.MIRROR_GRANULARITY // 512, event, event))
         file.close()
 
+    # blkdebug does not work with an external data file
+    @iotests.skip_for_imgopts(['data_file'])
     def setUp(self):
         self.blkdebug_file = target_img + ".blkdebug"
         iotests.create_image(backing_img, TestWriteErrors.image_len)
         self.create_blkdebug_file(self.blkdebug_file, "write_aio", 5)
-        qemu_img('create', '-f', iotests.imgfmt, '-obacking_file=%s' %(backing_img), test_img)
+        create_test_image(test_img, backing_file=backing_img)
         self.vm = iotests.VM().add_drive(test_img)
         self.target_img = 'blkdebug:%s:%s' % (self.blkdebug_file, target_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-osize=%d' %(TestWriteErrors.image_len), target_img)
+        create_test_image(target_img, TestWriteErrors.image_len)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(target_img)
+        remove_test_image(test_img)
+        remove_test_image(target_img)
         os.remove(backing_img)
         os.remove(self.blkdebug_file)
 
@@ -673,15 +670,15 @@ class TestSetSpeed(iotests.QMPTestCase):
     image_len = 80 * 1024 * 1024 # MB
 
     def setUp(self):
-        qemu_img('create', backing_img, str(TestSetSpeed.image_len))
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(backing_img, TestSetSpeed.image_len)
+        create_test_image(test_img, backing_file=backing_img)
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(backing_img)
+        remove_test_image(test_img)
+        remove_test_image(backing_img)
         os.remove(target_img)
 
     def test_set_speed(self):
@@ -739,8 +736,7 @@ class TestUnbackedSource(iotests.QMPTestCase):
     image_len = 2 * 1024 * 1024 # MB
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, test_img,
-                 str(TestUnbackedSource.image_len))
+        create_test_image(test_img, TestUnbackedSource.image_len)
         self.vm = iotests.VM()
         self.vm.launch()
         result = self.vm.qmp('blockdev-add', node_name='drive0',
@@ -753,8 +749,8 @@ class TestUnbackedSource(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
-        os.remove(target_img)
+        remove_test_image(test_img)
+        remove_test_image(target_img)
 
     def test_absolute_paths_full(self):
         self.assert_no_active_block_jobs()
@@ -784,8 +780,7 @@ class TestUnbackedSource(iotests.QMPTestCase):
         self.assert_no_active_block_jobs()
 
     def test_existing_full(self):
-        qemu_img('create', '-f', iotests.imgfmt, target_img,
-                 str(self.image_len))
+        create_test_image(target_img, self.image_len)
         qemu_io('-c', 'write -P 42 0 64k', target_img)
 
         self.assert_no_active_block_jobs()
@@ -802,8 +797,7 @@ class TestUnbackedSource(iotests.QMPTestCase):
                         'target image does not match source after mirroring')
 
     def test_blockdev_full(self):
-        qemu_img('create', '-f', iotests.imgfmt, target_img,
-                 str(self.image_len))
+        create_test_image(target_img, self.image_len)
         qemu_io('-c', 'write -P 42 0 64k', target_img)
 
         result = self.vm.qmp('blockdev-add', node_name='target',
@@ -834,8 +828,7 @@ class TestGranularity(iotests.QMPTestCase):
     image_len = 10 * 1024 * 1024 # MB
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, test_img,
-                 str(TestGranularity.image_len))
+        create_test_image(test_img, TestGranularity.image_len)
         qemu_io('-c', 'write 0 %d' % (self.image_len),
                 test_img)
         self.vm = iotests.VM().add_drive(test_img)
@@ -845,7 +838,7 @@ class TestGranularity(iotests.QMPTestCase):
         self.vm.shutdown()
         self.assertTrue(iotests.compare_images(test_img, target_img),
                         'target image does not match source after mirroring')
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(target_img)
 
     def test_granularity(self):
@@ -879,8 +872,7 @@ class TestRepairQuorum(iotests.QMPTestCase):
 
         # Add each individual quorum images
         for i in self.IMAGES:
-            qemu_img('create', '-f', iotests.imgfmt, i,
-                     str(TestSingleDrive.image_len))
+            create_test_image(i, TestSingleDrive.image_len)
             # Assign a node name to each quorum image in order to manipulate
             # them
             opts = "node-name=img%i" % self.IMAGES.index(i)
@@ -898,8 +890,10 @@ class TestRepairQuorum(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        for i in self.IMAGES + [ quorum_repair_img, quorum_snapshot_file ]:
-            # Do a try/except because the test may have deleted some images
+        for i in self.IMAGES:
+            remove_test_image(i)
+
+        for i in [ quorum_repair_img, quorum_snapshot_file ]:
             try:
                 os.remove(i)
             except OSError:
Signature · 1 line
2.21.0
author
#30 lore raw

PATCH 18/67 iotests/044: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/044 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/044 +3 -3
index 97ba98e628..a5e13751e1 100755
@@ -23,7 +23,7 @@ import os
 import qcow2
 from qcow2 import QcowHeader
 import iotests
-from iotests import qemu_img, qemu_img_verbose, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_img_verbose, qemu_io
 import struct
 import subprocess
 import sys
@@ -100,13 +100,13 @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
 
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=512', test_img, '16G')
+        create_test_image(test_img, '16G', opts=['cluster_size=512'])
         self.preallocate(test_img)
         pass
 
 
     def tearDown(self):
-        os.remove(test_img)
+        remove_test_image(test_img)
         pass
 
     def test_grow_refcount_table(self):
Signature · 1 line
2.21.0
author
#31 lore raw

PATCH 19/67 iotests/045: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/045 | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
tests/qemu-iotests/045 +13 -13
index 01cc038884..781ceb8d19 100755
@@ -20,7 +20,7 @@
 
 import os
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 image0 = os.path.join(iotests.test_dir, 'image0')
 image1 = os.path.join(iotests.test_dir, 'image1')
@@ -32,11 +32,11 @@ class TestFdSets(iotests.QMPTestCase):
 
     def setUp(self):
         self.vm = iotests.VM()
-        qemu_img('create', '-f', iotests.imgfmt, image0, '128K')
-        qemu_img('create', '-f', iotests.imgfmt, image1, '128K')
-        qemu_img('create', '-f', iotests.imgfmt, image2, '128K')
-        qemu_img('create', '-f', iotests.imgfmt, image3, '128K')
-        qemu_img('create', '-f', iotests.imgfmt, image4, '128K')
+        create_test_image(image0, '128K')
+        create_test_image(image1, '128K')
+        create_test_image(image2, '128K')
+        create_test_image(image3, '128K')
+        create_test_image(image4, '128K')
         self.file0 = open(image0, 'r')
         self.file1 = open(image1, 'w+')
         self.file2 = open(image2, 'r')
@@ -57,11 +57,11 @@ class TestFdSets(iotests.QMPTestCase):
         self.file2.close()
         self.file3.close()
         self.file4.close()
-        os.remove(image0)
-        os.remove(image1)
-        os.remove(image2)
-        os.remove(image3)
-        os.remove(image4)
+        remove_test_image(image0)
+        remove_test_image(image1)
+        remove_test_image(image2)
+        remove_test_image(image3)
+        remove_test_image(image4)
 
     def test_query_fdset(self):
         result = self.vm.qmp('query-fdsets')
@@ -129,7 +129,7 @@ class TestFdSets(iotests.QMPTestCase):
 class TestSCMFd(iotests.QMPTestCase):
     def setUp(self):
         self.vm = iotests.VM()
-        qemu_img('create', '-f', iotests.imgfmt, image0, '128K')
+        create_test_image(image0, '128K')
         # Add an unused monitor, to verify it works fine when two monitor
         # instances present
         self.vm.add_monitor_null()
@@ -137,7 +137,7 @@ class TestSCMFd(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(image0)
+        remove_test_image(image0)
 
     def _send_fd_by_SCM(self):
         ret = self.vm.send_fd_scm(file_path=image0)
Signature · 1 line
2.21.0
author
#32 lore raw

PATCH 20/67 iotests/055: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/055 | 41 +++++++++++++++++++++++------------------
 1 file changed, 23 insertions(+), 18 deletions(-)
tests/qemu-iotests/055 +23 -18
index c732a112d6..e44f18570d 100755
@@ -23,7 +23,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_io
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
 target_img = os.path.join(iotests.test_dir, 'target.img')
@@ -32,7 +32,7 @@ blockdev_target_img = os.path.join(iotests.test_dir, 'blockdev-target.img')
 image_len = 64 * 1024 * 1024 # MB
 
 def setUpModule():
-    qemu_img('create', '-f', iotests.imgfmt, test_img, str(image_len))
+    create_test_image(test_img, image_len)
     qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x11 0 64k', test_img)
     qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x00 64k 128k', test_img)
     qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x22 162k 32k', test_img)
@@ -41,12 +41,12 @@ def setUpModule():
     qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x33 67043328 64k', test_img)
 
 def tearDownModule():
-    os.remove(test_img)
+    remove_test_image(test_img)
 
 
 class TestSingleDrive(iotests.QMPTestCase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
+        create_test_image(blockdev_target_img, image_len)
 
         self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
         self.vm.add_drive(blockdev_target_img, interface="none")
@@ -56,7 +56,7 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(blockdev_target_img)
+        remove_test_image(blockdev_target_img)
         try:
             os.remove(target_img)
         except OSError:
@@ -163,7 +163,7 @@ class TestSingleDrive(iotests.QMPTestCase):
 
 class TestSetSpeed(iotests.QMPTestCase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
+        create_test_image(blockdev_target_img, image_len)
 
         self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
         self.vm.add_drive(blockdev_target_img, interface="none")
@@ -171,7 +171,7 @@ class TestSetSpeed(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(blockdev_target_img)
+        remove_test_image(blockdev_target_img)
         try:
             os.remove(target_img)
         except OSError:
@@ -249,7 +249,7 @@ class TestSetSpeed(iotests.QMPTestCase):
 #       would stall.  Instead, we limit the block job speed here.
 class TestSingleTransaction(iotests.QMPTestCase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
+        create_test_image(blockdev_target_img, image_len)
 
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.add_drive(blockdev_target_img, interface="none")
@@ -259,7 +259,7 @@ class TestSingleTransaction(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(blockdev_target_img)
+        remove_test_image(blockdev_target_img)
         try:
             os.remove(target_img)
         except OSError:
@@ -452,29 +452,34 @@ class TestSingleTransaction(iotests.QMPTestCase):
 
 class TestDriveCompression(iotests.QMPTestCase):
     image_len = 64 * 1024 * 1024 # MB
-    fmt_supports_compression = [{'type': 'qcow2', 'args': ()},
-                                {'type': 'vmdk', 'args': ('-o', 'subformat=streamOptimized')}]
+    fmt_supports_compression = [{'type': 'qcow2', 'opts': []},
+                                {'type': 'vmdk', 'opts': ['subformat=streamOptimized']}]
+
+    # Compression does not work with external data files
+    @iotests.skip_for_imgopts(['data_file'])
+    def setUp(self):
+        pass
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(blockdev_target_img)
+        remove_test_image(blockdev_target_img)
         try:
             os.remove(target_img)
         except OSError:
             pass
 
-    def do_prepare_drives(self, fmt, args, attach_target):
+    def do_prepare_drives(self, fmt, opts, attach_target):
         self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
 
-        qemu_img('create', '-f', fmt, blockdev_target_img,
-                 str(TestDriveCompression.image_len), *args)
+        create_test_image(blockdev_target_img, TestDriveCompression.image_len,
+                          fmt=fmt, opts=opts)
         if attach_target:
             self.vm.add_drive(blockdev_target_img, format=fmt, interface="none")
 
         self.vm.launch()
 
     def do_test_compress_complete(self, cmd, format, attach_target, **args):
-        self.do_prepare_drives(format['type'], format['args'], attach_target)
+        self.do_prepare_drives(format['type'], format['opts'], attach_target)
 
         self.assert_no_active_block_jobs()
 
@@ -499,7 +504,7 @@ class TestDriveCompression(iotests.QMPTestCase):
                                            target='drive1')
 
     def do_test_compress_cancel(self, cmd, format, attach_target, **args):
-        self.do_prepare_drives(format['type'], format['args'], attach_target)
+        self.do_prepare_drives(format['type'], format['opts'], attach_target)
 
         self.assert_no_active_block_jobs()
 
@@ -523,7 +528,7 @@ class TestDriveCompression(iotests.QMPTestCase):
                                          target='drive1')
 
     def do_test_compress_pause(self, cmd, format, attach_target, **args):
-        self.do_prepare_drives(format['type'], format['args'], attach_target)
+        self.do_prepare_drives(format['type'], format['opts'], attach_target)
 
         self.assert_no_active_block_jobs()
 
Signature · 1 line
2.21.0
author
#33 lore raw

PATCH 21/67 iotests/056: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/056 | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)
tests/qemu-iotests/056 +14 -16
index 98c55d8e5a..58b4103cd9 100755
@@ -23,7 +23,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io, create_image
+from iotests import create_test_image, remove_test_image, qemu_io, create_image
 
 backing_img = os.path.join(iotests.test_dir, 'backing.img')
 test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -31,18 +31,12 @@ target_img = os.path.join(iotests.test_dir, 'target.img')
 
 def img_create(img, fmt=iotests.imgfmt, size='64M', **kwargs):
     fullname = os.path.join(iotests.test_dir, '%s.%s' % (img, fmt))
-    optargs = []
-    for k,v in kwargs.items():
-        optargs = optargs + ['-o', '%s=%s' % (k,v)]
-    args = ['create', '-f', fmt] + optargs + [fullname, size]
-    iotests.qemu_img(*args)
+    opts = []
+    for k, v in kwargs.items():
+        opts.append('%s=%s' % (k, v))
+    create_test_image(fullname, size, fmt=fmt, opts=opts)
     return fullname
 
-def try_remove(img):
-    try:
-        os.remove(img)
-    except OSError:
-        pass
 
 def io_write_patterns(img, patterns):
     for pattern in patterns:
@@ -54,7 +48,7 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase):
 
     def setUp(self):
         create_image(backing_img, TestSyncModesNoneAndTop.image_len)
-        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        create_test_image(test_img, backing_file=backing_img)
         qemu_io('-c', 'write -P0x41 0 512', test_img)
         qemu_io('-c', 'write -P0xd5 1M 32k', test_img)
         qemu_io('-c', 'write -P0xdc 32M 124k', test_img)
@@ -64,7 +58,7 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         os.remove(backing_img)
         try:
             os.remove(target_img)
@@ -103,6 +97,8 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase):
         self.assertEqual(-1, qemu_io('-c', 'read -P0x41 0 512', target_img).find("verification failed"))
 
 class TestBeforeWriteNotifier(iotests.QMPTestCase):
+    # blkdebug does not work with an external data file
+    @iotests.skip_for_imgopts(['data_file'])
     def setUp(self):
         self.vm = iotests.VM().add_drive_raw("file=blkdebug::null-co://,id=drive0,align=65536,driver=blkdebug")
         self.vm.launch()
@@ -139,9 +135,9 @@ class BackupTest(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        try_remove(self.test_img)
-        try_remove(self.dest_img)
-        try_remove(self.ref_img)
+        remove_test_image(self.test_img)
+        remove_test_image(self.dest_img)
+        remove_test_image(self.ref_img)
 
     def hmp_io_writes(self, drive, patterns):
         for pattern in patterns:
@@ -268,6 +264,8 @@ class BackupTest(iotests.QMPTestCase):
                                  sync='full', target=self.dest_img,
                                  auto_dismiss=False)
 
+    # blkdebug does not work with an external data file
+    @iotests.skip_for_imgopts(['data_file'])
     def dismissal_failure(self, dismissal_opt):
         res = self.vm.qmp('query-block-jobs')
         self.assert_qmp(res, 'return', [])
Signature · 1 line
2.21.0
author
#34 lore raw

PATCH 22/67 iotests/057: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/057 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/057 +3 -3
index f5aa5929dc..c83f969b09 100755
@@ -23,7 +23,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_io
 
 test_drv_base_name = 'drive'
 
@@ -40,7 +40,7 @@ class ImageSnapshotTestCase(iotests.QMPTestCase):
             filename = '%s%d' % (test_img_base_name, i)
             img = os.path.join(iotests.test_dir, filename)
             device = '%s%d' % (test_drv_base_name, i)
-            qemu_img('create', '-f', iotests.imgfmt, img, str(self.image_len))
+            create_test_image(img, self.image_len)
             self.vm.add_drive(img)
             self.expect.append({'image': img, 'device': device,
                                 'snapshots': [],
@@ -50,7 +50,7 @@ class ImageSnapshotTestCase(iotests.QMPTestCase):
     def tearDown(self):
         self.vm.shutdown()
         for dev_expect in self.expect:
-            os.remove(dev_expect['image'])
+            remove_test_image(dev_expect['image'])
 
     def createSnapshotInTransaction(self, snapshot_num, abort = False):
         actions = []
Signature · 1 line
2.21.0
author
#35 lore raw

PATCH 23/67 iotests/065: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/065 | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
tests/qemu-iotests/065 +8 -9
index 9db9552784..221a87417d 100755
@@ -23,7 +23,7 @@ import os
 import re
 import json
 import iotests
-from iotests import qemu_img, qemu_img_pipe
+from iotests import create_test_image, remove_test_image, qemu_img_pipe
 import unittest
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -34,11 +34,10 @@ class TestImageInfoSpecific(iotests.QMPTestCase):
     def setUp(self):
         if self.img_options is None:
             self.skipTest('Skipping abstract test class')
-        qemu_img('create', '-f', iotests.imgfmt, '-o', self.img_options,
-                 test_img, '128K')
+        create_test_image(test_img, '128K', opts=self.img_options)
 
     def tearDown(self):
-        os.remove(test_img)
+        remove_test_image(test_img)
 
 class TestQemuImgInfo(TestImageInfoSpecific):
     '''Abstract base class for qemu-img info tests'''
@@ -87,13 +86,13 @@ class TestQMP(TestImageInfoSpecific):
 
 class TestQCow2(TestQemuImgInfo):
     '''Testing a qcow2 version 2 image'''
-    img_options = 'compat=0.10'
+    img_options = ['compat=0.10']
     json_compare = { 'compat': '0.10', 'refcount-bits': 16 }
     human_compare = [ 'compat: 0.10', 'refcount bits: 16' ]
 
 class TestQCow3NotLazy(TestQemuImgInfo):
     '''Testing a qcow2 version 3 image with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=off'
+    img_options = ['compat=1.1', 'lazy_refcounts=off']
     json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
                      'refcount-bits': 16, 'corrupt': False }
     human_compare = [ 'compat: 1.1', 'lazy refcounts: false',
@@ -101,7 +100,7 @@ class TestQCow3NotLazy(TestQemuImgInfo):
 
 class TestQCow3Lazy(TestQemuImgInfo):
     '''Testing a qcow2 version 3 image with lazy refcounts enabled'''
-    img_options = 'compat=1.1,lazy_refcounts=on'
+    img_options = ['compat=1.1', 'lazy_refcounts=on']
     json_compare = { 'compat': '1.1', 'lazy-refcounts': True,
                      'refcount-bits': 16, 'corrupt': False }
     human_compare = [ 'compat: 1.1', 'lazy refcounts: true',
@@ -110,7 +109,7 @@ class TestQCow3Lazy(TestQemuImgInfo):
 class TestQCow3NotLazyQMP(TestQMP):
     '''Testing a qcow2 version 3 image with lazy refcounts disabled, opening
        with lazy refcounts enabled'''
-    img_options = 'compat=1.1,lazy_refcounts=off'
+    img_options = ['compat=1.1', 'lazy_refcounts=off']
     qemu_options = 'lazy-refcounts=on'
     compare = { 'compat': '1.1', 'lazy-refcounts': False,
                 'refcount-bits': 16, 'corrupt': False }
@@ -119,7 +118,7 @@ class TestQCow3NotLazyQMP(TestQMP):
 class TestQCow3LazyQMP(TestQMP):
     '''Testing a qcow2 version 3 image with lazy refcounts enabled, opening
        with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=on'
+    img_options = ['compat=1.1', 'lazy_refcounts=on']
     qemu_options = 'lazy-refcounts=off'
     compare = { 'compat': '1.1', 'lazy-refcounts': True,
                 'refcount-bits': 16, 'corrupt': False }
Signature · 1 line
2.21.0
author
#36 lore raw

PATCH 24/67 iotests/096: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/096 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/096 +3 -2
index ab9cb47822..ee0cfbffe8 100755
@@ -20,6 +20,7 @@
 #
 
 import iotests
+from iotests import create_test_image, remove_test_image
 import os
 
 class TestLiveSnapshot(iotests.QMPTestCase):
@@ -35,13 +36,13 @@ class TestLiveSnapshot(iotests.QMPTestCase):
         opts.append('throttling.group=%s' % self.group)
         opts.append('throttling.iops-total=%d' % self.iops)
         opts.append('throttling.iops-size=%d' % self.iops_size)
-        iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, '100M')
+        create_test_image(self.base_img, '100M')
         self.vm = iotests.VM().add_drive(self.base_img, ','.join(opts))
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(self.base_img)
+        remove_test_image(self.base_img)
         os.remove(self.target_img)
 
     def checkConfig(self, active_layer):
Signature · 1 line
2.21.0
author
#37 lore raw

PATCH 25/67 iotests/118: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/118 | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
tests/qemu-iotests/118 +13 -13
index ea0b326ae0..634a0b6445 100755
@@ -23,7 +23,7 @@ import os
 import stat
 import time
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 old_img = os.path.join(iotests.test_dir, 'test0.img')
 new_img = os.path.join(iotests.test_dir, 'test1.img')
@@ -301,8 +301,8 @@ class TestInitiallyFilled(GeneralChangeTestsBaseClass):
     was_empty = False
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, old_img, '1440k')
-        qemu_img('create', '-f', iotests.imgfmt, new_img, '1440k')
+        create_test_image(old_img, '1440k')
+        create_test_image(new_img, '1440k')
         self.vm = iotests.VM()
         if self.use_drive:
             self.vm.add_drive(old_img, 'media=%s' % self.media, 'none')
@@ -320,8 +320,8 @@ class TestInitiallyFilled(GeneralChangeTestsBaseClass):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(old_img)
-        os.remove(new_img)
+        remove_test_image(old_img)
+        remove_test_image(new_img)
 
     def test_insert_on_filled(self):
         result = self.vm.qmp('blockdev-add',
@@ -344,7 +344,7 @@ class TestInitiallyEmpty(GeneralChangeTestsBaseClass):
     was_empty = True
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, new_img, '1440k')
+        create_test_image(new_img, '1440k')
         self.vm = iotests.VM()
         if self.use_drive:
             self.vm.add_drive(None, 'media=%s' % self.media, 'none')
@@ -358,7 +358,7 @@ class TestInitiallyEmpty(GeneralChangeTestsBaseClass):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(new_img)
+        remove_test_image(new_img)
 
     def test_remove_on_empty(self):
         result = self.vm.qmp('blockdev-open-tray', id=self.device_name)
@@ -394,16 +394,16 @@ class TestChangeReadOnly(ChangeBaseClass):
     device_name = 'qdev0'
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, old_img, '1440k')
-        qemu_img('create', '-f', iotests.imgfmt, new_img, '1440k')
+        create_test_image(old_img, '1440k')
+        create_test_image(new_img, '1440k')
         self.vm = iotests.VM()
 
     def tearDown(self):
         self.vm.shutdown()
         os.chmod(old_img, 0o666)
         os.chmod(new_img, 0o666)
-        os.remove(old_img)
-        os.remove(new_img)
+        remove_test_image(old_img)
+        remove_test_image(new_img)
 
     def test_ro_ro_retain(self):
         os.chmod(old_img, 0o444)
@@ -640,7 +640,7 @@ class TestBlockJobsAfterCycle(ChangeBaseClass):
     device_name = 'qdev0'
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, old_img, '1440K')
+        create_test_image(old_img, '1440k')
 
         self.vm = iotests.VM()
         self.vm.add_drive_raw("id=drive0,driver=null-co,if=none")
@@ -674,7 +674,7 @@ class TestBlockJobsAfterCycle(ChangeBaseClass):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(old_img)
+        remove_test_image(old_img)
         try:
             os.remove(new_img)
         except OSError:
Signature · 1 line
2.21.0
author
#38 lore raw

PATCH 26/67 iotests/124: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/124 | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)
tests/qemu-iotests/124 +13 -16
index d3e851e1ae..145dccb2b0 100755
@@ -22,6 +22,7 @@
 
 import os
 import iotests
+from iotests import create_test_image, remove_test_image
 
 
 def io_write_patterns(img, patterns):
@@ -29,13 +30,6 @@ def io_write_patterns(img, patterns):
         iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img)
 
 
-def try_remove(img):
-    try:
-        os.remove(img)
-    except OSError:
-        pass
-
-
 def transaction_action(action, **kwargs):
     return {
         'type': action,
@@ -82,13 +76,13 @@ class Bitmap:
 
     def del_target(self):
         for image in self.backups.pop():
-            try_remove(image)
+            remove_test_image(image)
         self.num -= 1
 
     def cleanup(self):
         for backup in self.backups:
             for image in backup:
-                try_remove(image)
+                remove_test_image(image)
 
 
 class TestIncrementalBackupBase(iotests.QMPTestCase):
@@ -133,15 +127,14 @@ class TestIncrementalBackupBase(iotests.QMPTestCase):
 
     def img_create(self, img, fmt=iotests.imgfmt, size='64M',
                    parent=None, parentFormat=None, **kwargs):
-        optargs = []
-        for k,v in kwargs.items():
-            optargs = optargs + ['-o', '%s=%s' % (k,v)]
-        args = ['create', '-f', fmt] + optargs + [img, size]
+        opts = []
+        for k, v in kwargs.items():
+            opts.append('%s=%s' % (k, v))
         if parent:
             if parentFormat is None:
                 parentFormat = fmt
-            args = args + ['-b', parent, '-F', parentFormat]
-        iotests.qemu_img(*args)
+        create_test_image(img, size, fmt=fmt, backing_file=parent,
+                          backing_fmt=parentFormat, opts=opts)
         self.files.append(img)
 
 
@@ -287,7 +280,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase):
         for bitmap in self.bitmaps:
             bitmap.cleanup()
         for filename in self.files:
-            try_remove(filename)
+            remove_test_image(filename)
 
 
 
@@ -414,6 +407,8 @@ class TestIncrementalBackup(TestIncrementalBackupBase):
         self.check_backups()
 
 
+    # blkdebug does not work with an external data file
+    @iotests.skip_for_imgopts(['data_file'])
     def do_transaction_failure_test(self, race=False):
         # Create a second drive, with pattern:
         drive1 = self.add_node('drive1')
@@ -608,6 +603,8 @@ class TestIncrementalBackup(TestIncrementalBackupBase):
 class TestIncrementalBackupBlkdebug(TestIncrementalBackupBase):
     '''Incremental backup tests that utilize a BlkDebug filter on drive0.'''
 
+    # blkdebug does not work with an external data file
+    @iotests.skip_for_imgopts(['data_file'])
     def setUp(self):
         drive0 = self.add_node('drive0')
         self.img_create(drive0['file'], drive0['fmt'])
Signature · 1 line
2.21.0
author
#39 lore raw

PATCH 27/67 iotests/129: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/129 | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
tests/qemu-iotests/129 +9 -2
index cd6b9e9ce7..379719d85b 100755
@@ -20,6 +20,7 @@
 
 import os
 import iotests
+from iotests import create_test_image, remove_test_image
 import time
 
 class TestStopWithBlockJob(iotests.QMPTestCase):
@@ -28,8 +29,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
     base_img = os.path.join(iotests.test_dir, 'base.img')
 
     def setUp(self):
-        iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G")
-        iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img)
+        create_test_image(self.base_img, '1G')
+        create_test_image(self.test_img, backing_file=self.base_img)
         iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img)
         self.vm = iotests.VM().add_drive(self.test_img)
         self.vm.launch()
@@ -46,6 +47,12 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
         result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
                              **params)
         self.vm.shutdown()
+        remove_test_image(self.test_img)
+        remove_test_image(self.base_img)
+        try:
+            os.remove(self.target_img)
+        except OSError:
+            pass
 
     def do_test_stop(self, cmd, **args):
         """Test 'stop' while block job is running on a throttled drive.
Signature · 1 line
2.21.0
author
#40 lore raw

PATCH 28/67 iotests/132: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/132 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/132 +3 -3
index 0f2a106c81..9c7b773f09 100755
@@ -21,7 +21,7 @@
 import time
 import os
 import iotests
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_io
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
 target_img = os.path.join(iotests.test_dir, 'target.img')
@@ -31,7 +31,7 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def setUp(self):
         # Write data to the image so we can compare later
-        qemu_img('create', '-f', iotests.imgfmt, test_img, str(TestSingleDrive.image_len))
+        create_test_image(test_img, TestSingleDrive.image_len)
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 0 2M', test_img)
 
         self.vm = iotests.VM().add_drive(test_img, 'discard=unmap')
@@ -39,7 +39,7 @@ class TestSingleDrive(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         try:
             os.remove(target_img)
         except OSError:
Signature · 1 line
2.21.0
author
#41 lore raw

PATCH 29/67 iotests/139: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/139 | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
tests/qemu-iotests/139 +7 -8
index cbb5a76530..f4cb553116 100755
@@ -21,6 +21,7 @@
 
 import os
 import iotests
+from iotests import create_test_image, remove_test_image
 import time
 
 base_img = os.path.join(iotests.test_dir, 'base.img')
@@ -33,7 +34,7 @@ else:
 class TestBlockdevDel(iotests.QMPTestCase):
 
     def setUp(self):
-        iotests.qemu_img('create', '-f', iotests.imgfmt, base_img, '1M')
+        create_test_image(base_img, '1M')
         self.vm = iotests.VM()
         self.vm.add_device("{},id=virtio-scsi".format(
             iotests.get_virtio_scsi_device()))
@@ -41,9 +42,8 @@ class TestBlockdevDel(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(base_img)
-        if os.path.isfile(new_img):
-            os.remove(new_img)
+        remove_test_image(base_img)
+        remove_test_image(new_img)
 
     # Check whether a BlockDriverState exists
     def checkBlockDriverState(self, node, must_exist = True):
@@ -70,8 +70,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
     # Add a BlockDriverState that will be used as overlay for the base_img BDS
     def addBlockDriverStateOverlay(self, node):
         self.checkBlockDriverState(node, False)
-        iotests.qemu_img('create', '-u', '-f', iotests.imgfmt,
-                         '-b', base_img, new_img, '1M')
+        create_test_image(new_img, '1M', backing_file=base_img, unsafe=True)
         opts = {'driver': iotests.imgfmt,
                 'node-name': node,
                 'backing': None,
@@ -202,7 +201,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
         self.checkBlockDriverState(test, False)
         self.checkBlockDriverState(raw, False)
         self.checkBlockDriverState(blkverify, False)
-        iotests.qemu_img('create', '-f', iotests.imgfmt, new_img, '1M')
+        create_test_image(new_img, '1M')
         node_0 = {'driver': iotests.imgfmt,
                   'node-name': test,
                   'file': {'driver': 'file',
@@ -226,7 +225,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
         self.checkBlockDriverState(child0, False)
         self.checkBlockDriverState(child1, False)
         self.checkBlockDriverState(quorum, False)
-        iotests.qemu_img('create', '-f', iotests.imgfmt, new_img, '1M')
+        create_test_image(new_img, '1M')
         child_0 = {'driver': iotests.imgfmt,
                    'node-name': child0,
                    'file': {'driver': 'file',
Signature · 1 line
2.21.0
author
#42 lore raw

PATCH 30/67 iotests/147: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/147 | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
tests/qemu-iotests/147 +6 -5
index ab8480b9a4..920e44a65c 100755
@@ -24,7 +24,8 @@ import socket
 import stat
 import time
 import iotests
-from iotests import cachemode, imgfmt, qemu_img, qemu_nbd, qemu_nbd_early_pipe
+from iotests import cachemode, imgfmt, create_test_image, remove_test_image, \
+        qemu_nbd, qemu_nbd_early_pipe
 
 NBD_PORT_START      = 32768
 NBD_PORT_END        = NBD_PORT_START + 1024
@@ -80,13 +81,13 @@ class NBDBlockdevAddBase(iotests.QMPTestCase):
 
 class QemuNBD(NBDBlockdevAddBase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, test_img, '64k')
+        create_test_image(test_img, '64k')
         self.vm = iotests.VM()
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         try:
             os.remove(unix_socket)
         except OSError:
@@ -127,7 +128,7 @@ class QemuNBD(NBDBlockdevAddBase):
 
 class BuiltinNBD(NBDBlockdevAddBase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, test_img, '64k')
+        create_test_image(test_img, '64k')
         self.vm = iotests.VM()
         self.vm.launch()
         self.server = iotests.VM('.server')
@@ -140,7 +141,7 @@ class BuiltinNBD(NBDBlockdevAddBase):
     def tearDown(self):
         self.vm.shutdown()
         self.server.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         try:
             os.remove(unix_socket)
         except OSError:
Signature · 1 line
2.21.0
author
#43 lore raw

PATCH 31/67 iotests/148: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/148 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/148 +3 -2
index 8c11c53cba..a6df0ed983 100755
@@ -21,6 +21,7 @@
 
 import os
 import iotests
+from iotests import create_test_image, remove_test_image
 
 imgs = (os.path.join(iotests.test_dir, 'quorum0.img'),
         os.path.join(iotests.test_dir, 'quorum1.img'),
@@ -51,7 +52,7 @@ sector = "%d"
         driveopts = ['driver=quorum', 'vote-threshold=2']
         driveopts.append('read-pattern=%s' % self.read_pattern)
         for i in range(len(imgs)):
-            iotests.qemu_img('create', '-f', iotests.imgfmt, imgs[i], '1M')
+            create_test_image(imgs[i], '1M')
             self.create_blkdebug_file(img_conf[i], i + offset)
             driveopts.append('children.%d.driver=%s' % (i, iotests.imgfmt))
             driveopts.append('children.%d.file.driver=blkdebug' % i)
@@ -65,7 +66,7 @@ sector = "%d"
     def tearDown(self):
         self.vm.shutdown()
         for i in range(len(imgs)):
-            os.remove(imgs[i])
+            remove_test_image(imgs[i])
             os.remove(img_conf[i])
 
     def do_check_event(self, node, sector = 0):
Signature · 1 line
2.21.0
author
#44 lore raw

PATCH 32/67 iotests/151: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/151 | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/151 +5 -5
index 76ae265cc1..bfe2bbb200 100755
@@ -20,7 +20,7 @@
 
 import os
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 source_img = os.path.join(iotests.test_dir, 'source.' + iotests.imgfmt)
 target_img = os.path.join(iotests.test_dir, 'target.' + iotests.imgfmt)
@@ -30,8 +30,8 @@ class TestActiveMirror(iotests.QMPTestCase):
     potential_writes_in_flight = True
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, source_img, '128M')
-        qemu_img('create', '-f', iotests.imgfmt, target_img, '128M')
+        create_test_image(source_img, '128M')
+        create_test_image(target_img, '128M')
 
         blk_source = {'id': 'source',
                       'if': 'none',
@@ -58,8 +58,8 @@ class TestActiveMirror(iotests.QMPTestCase):
             self.assertTrue(iotests.compare_images(source_img, target_img),
                             'mirror target does not match source')
 
-        os.remove(source_img)
-        os.remove(target_img)
+        remove_test_image(source_img)
+        remove_test_image(target_img)
 
     def doActiveIO(self, sync_source_and_target):
         # Fill the source image
Signature · 1 line
2.21.0
author
#45 lore raw

PATCH 33/67 iotests/152: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/152 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/152 +3 -3
index 732bf5f062..123ca0cd5d 100755
@@ -20,20 +20,20 @@
 
 import os
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
 target_img = os.path.join(iotests.test_dir, 'target.img')
 
 class TestUnaligned(iotests.QMPTestCase):
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, test_img, '512')
+        create_test_image(test_img, 512)
         self.vm = iotests.VM().add_drive(test_img)
         self.vm.launch()
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(test_img)
+        remove_test_image(test_img)
         try:
             os.remove(target_img)
         except OSError:
Signature · 1 line
2.21.0
author
#46 lore raw

PATCH 34/67 iotests/155: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/155 | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)
tests/qemu-iotests/155 +13 -16
index e19485911c..4efe0e2237 100755
@@ -23,7 +23,7 @@
 
 import os
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 back0_img = os.path.join(iotests.test_dir, 'back0.' + iotests.imgfmt)
 back1_img = os.path.join(iotests.test_dir, 'back1.' + iotests.imgfmt)
@@ -51,10 +51,10 @@ class BaseClass(iotests.QMPTestCase):
     target_real_backing = None
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, back0_img, '1440K')
-        qemu_img('create', '-f', iotests.imgfmt, '-b', back0_img, back1_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-b', back1_img, back2_img)
-        qemu_img('create', '-f', iotests.imgfmt, '-b', back2_img, source_img)
+        create_test_image(back0_img, '1440K')
+        create_test_image(back1_img, backing_file=back0_img)
+        create_test_image(back2_img, backing_file=back1_img)
+        create_test_image(source_img, backing_file=back2_img)
 
         self.vm = iotests.VM()
         # Add the BDS via blockdev-add so it stays around after the mirror block
@@ -71,10 +71,10 @@ class BaseClass(iotests.QMPTestCase):
 
         if self.existing:
             if self.target_backing:
-                qemu_img('create', '-f', iotests.imgfmt,
-                         '-b', self.target_backing, target_img, '1440K')
+                create_test_image(target_img, '1440K', \
+                                  backing_file=self.target_backing)
             else:
-                qemu_img('create', '-f', iotests.imgfmt, target_img, '1440K')
+                create_test_image(target_img, '1440K')
 
             if self.cmd == 'blockdev-mirror':
                 options = { 'node-name': 'target',
@@ -89,14 +89,11 @@ class BaseClass(iotests.QMPTestCase):
 
     def tearDown(self):
         self.vm.shutdown()
-        os.remove(source_img)
-        os.remove(back2_img)
-        os.remove(back1_img)
-        os.remove(back0_img)
-        try:
-            os.remove(target_img)
-        except OSError:
-            pass
+        remove_test_image(source_img)
+        remove_test_image(back2_img)
+        remove_test_image(back1_img)
+        remove_test_image(back0_img)
+        remove_test_image(target_img)
 
     def findBlockNode(self, node_name, qdev=None):
         if qdev:
Signature · 1 line
2.21.0
author
#47 lore raw

PATCH 35/67 iotests/163: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/163 | 24 +++++++++++-------------
 1 file changed, 11 insertions(+), 13 deletions(-)
tests/qemu-iotests/163 +11 -13
index 92633f1b21..5c5e23f9fc 100755
@@ -19,7 +19,8 @@
 #
 
 import os, random, iotests, struct, qcow2, sys
-from iotests import qemu_img, qemu_io, image_size
+from iotests import create_test_image, remove_test_image, qemu_img, qemu_io, \
+        image_size
 
 test_img = os.path.join(iotests.test_dir, 'test.img')
 check_img = os.path.join(iotests.test_dir, 'check.img')
@@ -82,22 +83,19 @@ class ShrinkBaseClass(iotests.QMPTestCase):
 
     def setUp(self):
         if iotests.imgfmt == 'raw':
-            qemu_img('create', '-f', iotests.imgfmt, test_img, self.image_len)
-            qemu_img('create', '-f', iotests.imgfmt, check_img,
-                     self.shrink_size)
+            create_test_image(test_img, self.image_len)
+            create_test_image(check_img, self.shrink_size)
         else:
-            qemu_img('create', '-f', iotests.imgfmt,
-                     '-o', 'cluster_size=' + self.cluster_size +
-                     ',refcount_bits=' + self.refcount_bits,
-                     test_img, self.image_len)
-            qemu_img('create', '-f', iotests.imgfmt,
-                     '-o', 'cluster_size=%s'% self.cluster_size,
-                     check_img, self.shrink_size)
+            create_test_image(test_img, self.image_len,
+                              opts=['cluster_size=' + self.cluster_size,
+                                    'refcount_bits=' + self.refcount_bits])
+            create_test_image(check_img, self.shrink_size,
+                              opts=['cluster_size=' + self.cluster_size])
         qemu_io('-c', 'write -P 0xff 0 ' + self.shrink_size, check_img)
 
     def tearDown(self):
-        os.remove(test_img)
-        os.remove(check_img)
+        remove_test_image(test_img)
+        remove_test_image(check_img)
 
     def image_verify(self):
         self.assertEqual(image_size(test_img), image_size(check_img),
Signature · 1 line
2.21.0
author
#48 lore raw

PATCH 36/67 iotests/165: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/165 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/165 +3 -3
index 97dd7102c3..0c92947f72 100755
@@ -22,7 +22,7 @@ from __future__ import print_function
 import os
 import re
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 disk = os.path.join(iotests.test_dir, 'disk')
 disk_size = 0x40000000 # 1G
@@ -37,10 +37,10 @@ regions2 = ((0x10000000, 0x20000),
 class TestPersistentDirtyBitmap(iotests.QMPTestCase):
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, disk, str(disk_size))
+        create_test_image(disk, disk_size)
 
     def tearDown(self):
-        os.remove(disk)
+        remove_test_image(disk)
 
     def mkVm(self):
         return iotests.VM().add_drive(disk)
Signature · 1 line
2.21.0
author
#49 lore raw

PATCH 37/67 iotests/169: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/169 | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/169 +5 -5
index 5e978b7b7d..8d41463567 100755
@@ -24,7 +24,7 @@ import time
 import itertools
 import operator
 import re
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 
 disk_a = os.path.join(iotests.test_dir, 'disk_a')
@@ -39,16 +39,16 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
     def tearDown(self):
         self.vm_a.shutdown()
         self.vm_b.shutdown()
-        os.remove(disk_a)
-        os.remove(disk_b)
+        remove_test_image(disk_a)
+        remove_test_image(disk_b)
         try:
             os.remove(mig_file)
         except OSError:
             pass
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, disk_a, size)
-        qemu_img('create', '-f', iotests.imgfmt, disk_b, size)
+        create_test_image(disk_a, size)
+        create_test_image(disk_b, size)
 
         self.vm_a = iotests.VM(path_suffix='a').add_drive(disk_a)
         self.vm_a.launch()
Signature · 1 line
2.21.0
author
#50 lore raw

PATCH 38/67 iotests/194: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/194 | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/194 +5 -4
index c8aeb6d0e4..70cae2d8ef 100755
@@ -20,19 +20,20 @@
 # Non-shared storage migration test using NBD server and drive-mirror
 
 import iotests
+from iotests import create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2', 'qed', 'raw'])
 
-with iotests.FilePath('source.img') as source_img_path, \
-     iotests.FilePath('dest.img') as dest_img_path, \
+with iotests.ImagePath('source.img') as source_img_path, \
+     iotests.ImagePath('dest.img') as dest_img_path, \
      iotests.FilePath('migration.sock') as migration_sock_path, \
      iotests.FilePath('nbd.sock') as nbd_sock_path, \
      iotests.VM('source') as source_vm, \
      iotests.VM('dest') as dest_vm:
 
     img_size = '1G'
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, source_img_path, img_size)
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, dest_img_path, img_size)
+    create_test_image(source_img_path, img_size)
+    create_test_image(dest_img_path, img_size)
 
     iotests.log('Launching VMs...')
     (source_vm.add_drive(source_img_path)
Signature · 1 line
2.21.0
author
#51 lore raw

PATCH 39/67 iotests/196: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/196 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/196 +3 -3
index ec4852a19c..4d6a5fac05 100755
@@ -21,7 +21,7 @@
 
 import os
 import iotests
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 disk = os.path.join(iotests.test_dir, 'disk')
 migfile = os.path.join(iotests.test_dir, 'migfile')
@@ -31,11 +31,11 @@ class TestInvalidateAutoclear(iotests.QMPTestCase):
     def tearDown(self):
         self.vm_a.shutdown()
         self.vm_b.shutdown()
-        os.remove(disk)
+        remove_test_image(disk)
         os.remove(migfile)
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, disk, '1M')
+        create_test_image(disk, '1M')
 
         self.vm_a = iotests.VM(path_suffix='a').add_drive(disk)
         self.vm_a.launch()
Signature · 1 line
2.21.0
author
#52 lore raw

PATCH 40/67 iotests/199: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/199 | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/199 +5 -5
index a2c8ecab5a..ec028e1149 100755
@@ -21,7 +21,7 @@
 import os
 import iotests
 import time
-from iotests import qemu_img
+from iotests import create_test_image, remove_test_image
 
 disk_a = os.path.join(iotests.test_dir, 'disk_a')
 disk_b = os.path.join(iotests.test_dir, 'disk_b')
@@ -33,14 +33,14 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
     def tearDown(self):
         self.vm_a.shutdown()
         self.vm_b.shutdown()
-        os.remove(disk_a)
-        os.remove(disk_b)
+        remove_test_image(disk_a)
+        remove_test_image(disk_b)
         os.remove(fifo)
 
     def setUp(self):
         os.mkfifo(fifo)
-        qemu_img('create', '-f', iotests.imgfmt, disk_a, size)
-        qemu_img('create', '-f', iotests.imgfmt, disk_b, size)
+        create_test_image(disk_a, size)
+        create_test_image(disk_b, size)
         self.vm_a = iotests.VM(path_suffix='a').add_drive(disk_a)
         self.vm_b = iotests.VM(path_suffix='b').add_drive(disk_b)
         self.vm_b.add_incoming("exec: cat '" + fifo + "'")
Signature · 1 line
2.21.0
author
#53 lore raw

PATCH 41/67 iotests/202: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/202 | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/202 +5 -4
index 1271ac9459..f2a77fbe9b 100755
@@ -23,18 +23,19 @@
 # against regressions.
 
 import iotests
+from iotests import create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
-with iotests.FilePath('disk0.img') as disk0_img_path, \
-     iotests.FilePath('disk1.img') as disk1_img_path, \
+with iotests.ImagePath('disk0.img') as disk0_img_path, \
+     iotests.ImagePath('disk1.img') as disk1_img_path, \
      iotests.FilePath('disk0-snap.img') as disk0_snap_img_path, \
      iotests.FilePath('disk1-snap.img') as disk1_snap_img_path, \
      iotests.VM() as vm:
 
     img_size = '10M'
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk0_img_path, img_size)
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk1_img_path, img_size)
+    create_test_image(disk0_img_path, img_size)
+    create_test_image(disk1_img_path, img_size)
 
     iotests.log('Launching VM...')
     vm.launch()
Signature · 1 line
2.21.0
author
#54 lore raw

PATCH 42/67 iotests/203: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/203 | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/203 +5 -4
index c40fe231ea..ce4bc05e08 100755
@@ -23,16 +23,17 @@
 # BDRV_POLL_WHILE().
 
 import iotests
+from iotests import create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
-with iotests.FilePath('disk0.img') as disk0_img_path, \
-     iotests.FilePath('disk1.img') as disk1_img_path, \
+with iotests.ImagePath('disk0.img') as disk0_img_path, \
+     iotests.ImagePath('disk1.img') as disk1_img_path, \
      iotests.VM() as vm:
 
     img_size = '10M'
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk0_img_path, img_size)
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk1_img_path, img_size)
+    create_test_image(disk0_img_path, img_size)
+    create_test_image(disk1_img_path, img_size)
 
     iotests.log('Launching VM...')
     (vm.add_object('iothread,id=iothread0')
Signature · 1 line
2.21.0
author
#55 lore raw

PATCH 43/67 iotests/205: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/205 | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
tests/qemu-iotests/205 +4 -3
index 76f6c5fa2b..343fbc296e 100755
@@ -22,7 +22,8 @@ import os
 import sys
 import iotests
 import time
-from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive
+from iotests import create_test_image, remove_test_image, \
+        qemu_io, filter_qemu_io, QemuIoInteractive
 
 nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock')
 nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
@@ -31,7 +32,7 @@ disk = os.path.join(iotests.test_dir, 'disk')
 
 class TestNbdServerRemove(iotests.QMPTestCase):
     def setUp(self):
-        qemu_img_create('-f', iotests.imgfmt, disk, '1M')
+        create_test_image(disk, '1M')
 
         self.vm = iotests.VM().add_drive(disk)
         self.vm.launch()
@@ -51,7 +52,7 @@ class TestNbdServerRemove(iotests.QMPTestCase):
     def tearDown(self):
         self.vm.shutdown()
         os.remove(nbd_sock)
-        os.remove(disk)
+        remove_test_image(disk)
 
     def remove_export(self, name, mode=None):
         if mode is None:
Signature · 1 line
2.21.0
author
#56 lore raw

PATCH 44/67 iotests/208: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/208 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/208 +3 -2
index dfce6f9fe4..cd565eea43 100755
@@ -21,16 +21,17 @@
 # blockdev-snapshot-sync.
 
 import iotests
+from iotests import create_test_image
 
 iotests.script_initialize(supported_fmts=['generic'])
 
-with iotests.FilePath('disk.img') as disk_img_path, \
+with iotests.ImagePath('disk.img') as disk_img_path, \
      iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \
      iotests.FilePath('nbd.sock') as nbd_sock_path, \
      iotests.VM() as vm:
 
     img_size = '10M'
-    iotests.qemu_img_create('-f', iotests.imgfmt, disk_img_path, img_size)
+    create_test_image(disk_img_path, img_size)
 
     iotests.log('Launching VM...')
     (vm.add_drive(disk_img_path, 'node-name=drive0-node', interface='none')
Signature · 1 line
2.21.0
author
#57 lore raw

PATCH 45/67 iotests/208: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/209 | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/209 +5 -4
index a77f884166..eb1215afd7 100755
@@ -19,15 +19,16 @@
 #
 
 import iotests
-from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \
-                    file_path
+from iotests import create_test_image, qemu_io, qemu_img_verbose, qemu_nbd, \
+                    file_path, image_path
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
-disk, nbd_sock = file_path('disk', 'nbd-sock')
+nbd_sock = file_path('nbd-sock')
+disk = image_path('disk')
 nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
 
-qemu_img_create('-f', iotests.imgfmt, disk, '1M')
+create_test_image(disk, '1M')
 qemu_io('-f', iotests.imgfmt, '-c', 'write 0 512K', disk)
 
 qemu_nbd('-k', nbd_sock, '-x', 'exp', '-f', iotests.imgfmt, disk)
Signature · 1 line
2.21.0
author
#58 lore raw

PATCH 46/67 iotests/216: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/216 | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
tests/qemu-iotests/216 +5 -6
index 7574bcc09f..18f2611da6 100755
@@ -20,7 +20,7 @@
 # Creator/Owner: Max Reitz <[email protected]>
 
 import iotests
-from iotests import log, qemu_img, qemu_io_silent
+from iotests import log, create_test_image, qemu_io_silent
 
 # Need backing file support
 iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk'])
@@ -42,17 +42,16 @@ log('')
 # A COR filter node, however, can request the proper permissions for
 # its child and therefore is not hit by this issue.
 
-with iotests.FilePath('base.img') as base_img_path, \
-     iotests.FilePath('top.img') as top_img_path, \
+with iotests.ImagePath('base.img') as base_img_path, \
+     iotests.ImagePath('top.img') as top_img_path, \
      iotests.VM() as vm:
 
     log('--- Setting up images ---')
     log('')
 
-    assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
+    assert create_test_image(base_img_path, '64M') == 0
     assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0
-    assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
-                    top_img_path) == 0
+    assert create_test_image(top_img_path, backing_file=base_img_path) == 0
     assert qemu_io_silent(top_img_path,  '-c', 'write -P 2 1M 1M') == 0
 
     log('Done')
Signature · 1 line
2.21.0
author
#59 lore raw

PATCH 47/67 iotests/218: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/218 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/218 +3 -3
index e18e31076b..6f6fe6588e 100755
@@ -27,7 +27,7 @@
 # Creator/Owner: Max Reitz <[email protected]>
 
 import iotests
-from iotests import log, qemu_img, qemu_io_silent
+from iotests import log, create_test_image, qemu_io_silent
 
 iotests.script_initialize(supported_fmts=['qcow2', 'raw'])
 
@@ -142,9 +142,9 @@ log('=== Cancel mirror job from throttled node by quitting ===')
 log('')
 
 with iotests.VM() as vm, \
-     iotests.FilePath('src.img') as src_img_path:
+     iotests.ImagePath('src.img') as src_img_path:
 
-    assert qemu_img('create', '-f', iotests.imgfmt, src_img_path, '64M') == 0
+    assert create_test_image(src_img_path, '64M') == 0
     assert qemu_io_silent('-f', iotests.imgfmt, src_img_path,
                           '-c', 'write -P 42 0M 64M') == 0
 
Signature · 1 line
2.21.0
author
#60 lore raw

PATCH 48/67 iotests/219: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/219 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/219 +3 -2
index 9ae27cb04e..37e3c70aef 100755
@@ -20,6 +20,7 @@
 # Check using the job-* QMP commands with block jobs
 
 import iotests
+from iotests import create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
@@ -177,11 +178,11 @@ def test_job_lifecycle(vm, job, job_args, has_ready=False):
     iotests.log(vm.qmp('query-jobs'))
 
 
-with iotests.FilePath('disk.img') as disk_path, \
+with iotests.ImagePath('disk.img') as disk_path, \
      iotests.FilePath('copy.img') as copy_path, \
      iotests.VM() as vm:
 
-    iotests.qemu_img_create('-f', iotests.imgfmt, disk_path, str(img_size))
+    create_test_image(disk_path, img_size)
     iotests.qemu_io('-c', 'write 0 %i' % (img_size),
                     '-f', iotests.imgfmt, disk_path)
 
Signature · 1 line
2.21.0
author
#61 lore raw

PATCH 49/67 iotests/222: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/222 | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/222 +5 -5
index ffa88d4bb9..7ae76c1be1 100644
@@ -22,7 +22,7 @@
 # Creator/Owner: John Snow <[email protected]>
 
 import iotests
-from iotests import log, qemu_img, qemu_io, qemu_io_silent
+from iotests import log, create_test_image, qemu_io, qemu_io_silent
 
 iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk',
                                           'vhdx', 'raw'],
@@ -46,16 +46,16 @@ remainder = [("0xd5", "0x108000",  "32k"), # Right-end of partial-left [1]
              ("0xdc", "32M",       "32k"), # Left-end of partial-right [2]
              ("0xcd", "0x3ff0000", "64k")] # patterns[3]
 
-with iotests.FilePath('base.img') as base_img_path, \
-     iotests.FilePath('fleece.img') as fleece_img_path, \
+with iotests.ImagePath('base.img') as base_img_path, \
+     iotests.ImagePath('fleece.img') as fleece_img_path, \
      iotests.FilePath('nbd.sock') as nbd_sock_path, \
      iotests.VM() as vm:
 
     log('--- Setting up images ---')
     log('')
 
-    assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
-    assert qemu_img('create', '-f', "qcow2", fleece_img_path, '64M') == 0
+    assert create_test_image(base_img_path, '64M') == 0
+    assert create_test_image(fleece_img_path, '64M', fmt='qcow2') == 0
 
     for p in patterns:
         qemu_io('-f', iotests.imgfmt,
Signature · 1 line
2.21.0
author
#62 lore raw

PATCH 50/67 iotests/224: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/224 | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
tests/qemu-iotests/224 +9 -11
index e227d3ab3c..e9f5729c33 100755
@@ -21,8 +21,9 @@
 # Creator/Owner: Max Reitz <[email protected]>
 
 import iotests
-from iotests import log, qemu_img, qemu_io_silent, filter_qmp_testfiles, \
-                    filter_qmp_imgfmt, filter_json_filename
+from iotests import log, create_test_image, qemu_io_silent, \
+                    filter_qmp_testfiles, filter_qmp_imgfmt, \
+                    filter_json_filename
 import json
 import re
 
@@ -47,17 +48,14 @@ for filter_node_name in False, True:
     log('--- filter_node_name: %s ---' % filter_node_name)
     log('')
 
-    with iotests.FilePath('base.img') as base_img_path, \
-         iotests.FilePath('mid.img') as mid_img_path, \
-         iotests.FilePath('top.img') as top_img_path, \
+    with iotests.ImagePath('base.img') as base_img_path, \
+         iotests.ImagePath('mid.img') as mid_img_path, \
+         iotests.ImagePath('top.img') as top_img_path, \
          iotests.VM() as vm:
 
-        assert qemu_img('create', '-f', iotests.imgfmt,
-                        base_img_path, '64M') == 0
-        assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
-                        mid_img_path) == 0
-        assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path,
-                        top_img_path) == 0
+        assert create_test_image(base_img_path, '64M') == 0
+        assert create_test_image(mid_img_path, backing_file=base_img_path) == 0
+        assert create_test_image(top_img_path, backing_file=mid_img_path) == 0
 
         # Something to commit
         assert qemu_io_silent(mid_img_path, '-c', 'write -P 1 0 1M') == 0
Signature · 1 line
2.21.0
author
#63 lore raw

PATCH 51/67 iotests/228: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/228 | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
tests/qemu-iotests/228 +10 -9
index e507db4f27..ad7966fc8f 100755
@@ -21,8 +21,9 @@
 # Creator/Owner: Max Reitz <[email protected]>
 
 import iotests
-from iotests import log, qemu_img, filter_testfiles, filter_imgfmt, \
-        filter_qmp_testfiles, filter_qmp_imgfmt, filter_json_filename
+from iotests import log, create_test_image, remove_test_image, \
+        filter_testfiles, filter_imgfmt, filter_qmp_testfiles, \
+        filter_qmp_imgfmt, filter_json_filename
 
 # Need backing file and change-backing-file support
 iotests.script_initialize(supported_fmts=['qcow2', 'qed'])
@@ -46,14 +47,14 @@ def log_node_info(node):
     log('')
 
 
-with iotests.FilePath('base.img') as base_img_path, \
-     iotests.FilePath('top.img') as top_img_path, \
+with iotests.ImagePath('base.img') as base_img_path, \
+     iotests.ImagePath('top.img') as top_img_path, \
      iotests.VM() as vm:
 
-    assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
+    assert create_test_image(base_img_path, '64M') == 0
     # Choose a funny way to describe the backing filename
-    assert qemu_img('create', '-f', iotests.imgfmt, '-b',
-                    'file:' + base_img_path, top_img_path) == 0
+    assert create_test_image(top_img_path,
+                             backing_file=('file:' + base_img_path)) == 0
 
     vm.launch()
 
@@ -167,8 +168,8 @@ with iotests.FilePath('base.img') as base_img_path, \
     # (because qemu cannot "canonicalize"/"resolve" the backing
     # filename unless the backing file is opened implicitly with the
     # overlay)
-    assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
-                    top_img_path) == 0
+    remove_test_image(top_img_path)
+    assert create_test_image(top_img_path, backing_file=base_img_path) == 0
 
     # You can only reliably override backing options by using a node
     # reference (or by specifying file.filename, but, well...)
Signature · 1 line
2.21.0
author
#64 lore raw

PATCH 52/67 iotests/234: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/234 | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/234 +5 -4
index 3de6ab2341..f200af3f27 100755
@@ -21,19 +21,20 @@
 # that is built with individually created nodes
 
 import iotests
+from iotests import create_test_image
 import os
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
-with iotests.FilePath('img') as img_path, \
-     iotests.FilePath('backing') as backing_path, \
+with iotests.ImagePath('img') as img_path, \
+     iotests.ImagePath('backing') as backing_path, \
      iotests.FilePath('mig_fifo_a') as fifo_a, \
      iotests.FilePath('mig_fifo_b') as fifo_b, \
      iotests.VM(path_suffix='a') as vm_a, \
      iotests.VM(path_suffix='b') as vm_b:
 
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, backing_path, '64M')
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M')
+    create_test_image(backing_path, '64M')
+    create_test_image(img_path, '64M')
 
     os.mkfifo(fifo_a)
     os.mkfifo(fifo_b)
Signature · 1 line
2.21.0
author
#65 lore raw

PATCH 53/67 iotests/235: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/235 | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
tests/qemu-iotests/235 +3 -4
index 9e88c65b93..3326d205b7 100755
@@ -21,7 +21,7 @@
 import sys
 import os
 import iotests
-from iotests import qemu_img_create, qemu_io, file_path, log
+from iotests import create_test_image, qemu_io, image_path, log
 
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 
@@ -42,11 +42,10 @@ iotests.script_initialize(supported_fmts=['qcow2'])
 
 size = 1 * 1024 * 1024 * 1024
 
-disk = file_path('disk')
+disk = image_path('disk')
 
 # prepare source image
-qemu_img_create('-f', iotests.imgfmt, '-o', 'preallocation=metadata', disk,
-                str(size))
+create_test_image(disk, size, opts=['preallocation=metadata'])
 
 vm = QEMUMachine(iotests.qemu_prog)
 vm.add_args('-machine', 'accel=kvm:tcg')
Signature · 1 line
2.21.0
author
#66 lore raw

PATCH 54/67 iotests/236: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/236 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/236 +3 -3
index b88779eb0b..ef1d732510 100755
@@ -20,7 +20,7 @@
 # [email protected]
 
 import iotests
-from iotests import log
+from iotests import log, create_test_image
 
 iotests.script_initialize(supported_fmts=['generic'])
 size = 64 * 1024 * 1024
@@ -41,11 +41,11 @@ def query_bitmaps(vm):
     return { "bitmaps": { device['device']: device.get('dirty-bitmaps', []) for
                           device in res['return'] } }
 
-with iotests.FilePath('img') as img_path, \
+with iotests.ImagePath('img') as img_path, \
      iotests.VM() as vm:
 
     log('--- Preparing image & VM ---\n')
-    iotests.qemu_img_create('-f', iotests.imgfmt, img_path, str(size))
+    create_test_image(img_path, size)
     vm.add_drive(img_path)
     vm.launch()
 
Signature · 1 line
2.21.0
author
#67 lore raw

PATCH 55/67 iotests/237: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/237     | 11 +++++------
 tests/qemu-iotests/237.out |  6 ------
 2 files changed, 5 insertions(+), 12 deletions(-)
tests/qemu-iotests/237 +5 -6
index 34ff4b55db..179eae87dc 100755
@@ -22,7 +22,7 @@
 
 import math
 import iotests
-from iotests import imgfmt
+from iotests import imgfmt, create_test_image
 
 iotests.script_initialize(supported_fmts=['vmdk'],
                           # All options are ignored
@@ -38,9 +38,9 @@ def blockdev_create(vm, options):
     iotests.log("")
 
 with iotests.FilePath('t.vmdk') as disk_path, \
-     iotests.FilePath('t.vmdk.1') as extent1_path, \
-     iotests.FilePath('t.vmdk.2') as extent2_path, \
-     iotests.FilePath('t.vmdk.3') as extent3_path, \
+     iotests.ImagePath('t.vmdk.1') as extent1_path, \
+     iotests.ImagePath('t.vmdk.2') as extent2_path, \
+     iotests.ImagePath('t.vmdk.3') as extent3_path, \
      iotests.VM() as vm:
 
     #
@@ -175,8 +175,7 @@ with iotests.FilePath('t.vmdk') as disk_path, \
     iotests.log("")
 
     for path in [ extent1_path, extent2_path, extent3_path ]:
-        msg = iotests.qemu_img_pipe('create', '-f', imgfmt, path, '0')
-        iotests.log(msg, [iotests.filter_testfiles])
+        assert create_test_image(path, 0) == 0
 
     vm.add_blockdev('driver=file,filename=%s,node-name=ext1' % (extent1_path))
     vm.add_blockdev('driver=file,filename=%s,node-name=ext2' % (extent2_path))
tests/qemu-iotests/237.out +0 -6
index a8c800bfad..c32b970363 100644
@@ -129,12 +129,6 @@ Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exi
 
 === Other subformats ===
 
-Formatting 'TEST_DIR/PID-t.vmdk.1', fmt=vmdk size=0 compat6=off hwversion=undefined
-
-Formatting 'TEST_DIR/PID-t.vmdk.2', fmt=vmdk size=0 compat6=off hwversion=undefined
-
-Formatting 'TEST_DIR/PID-t.vmdk.3', fmt=vmdk size=0 compat6=off hwversion=undefined
-
 == Missing extent ==
 
 {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "vmdk", "file": "node0", "size": 33554432, "subformat": "monolithicFlat"}}}
Signature · 1 line
2.21.0
author
#68 lore raw

PATCH 56/67 iotests/242: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/242 | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
tests/qemu-iotests/242 +6 -5
index 333fb8b56c..bccd8a9fdf 100755
@@ -21,14 +21,14 @@
 import iotests
 import json
 import struct
-from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \
-    file_path, img_info_log, log, filter_qemu_io
+from iotests import create_test_image, remove_test_image, qemu_io, \
+    qemu_img_pipe, image_path, img_info_log, log, filter_qemu_io
 
 iotests.script_initialize(supported_fmts=['qcow2'],
                           # compat=1.1 is needed for dirty bitmaps
                           unsupported_imgopts=['compat=0.10'])
 
-disk = file_path('disk')
+disk = image_path('disk')
 chunk = 256 * 1024
 bitmap_flag_unknown = 1 << 2
 # flag_offset = 5*cluster_size + flag_offset_in_bitmap_directory_entry
@@ -74,7 +74,7 @@ def toggle_flag(offset):
         f.write(struct.pack("B", toggled))
 
 
-qemu_img_create('-f', iotests.imgfmt, disk, '1M')
+create_test_image(disk, '1M')
 
 for num in range(1, 4):
     disabled = False
@@ -95,7 +95,8 @@ vm.shutdown()
 
 num += 1
 log('\nTest {}'.format(num))
-qemu_img_create('-f', iotests.imgfmt, disk, '1M')
+remove_test_image(disk)
+create_test_image(disk, '1M')
 add_bitmap(1, True, False)
 log('Write an unknown bitmap flag \'{}\' into a new QCOW2 image at offset {}'
     .format(hex(bitmap_flag_unknown), flag_offset))
Signature · 1 line
2.21.0
author
#69 lore raw

PATCH 57/67 iotests/245: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/245 | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
tests/qemu-iotests/245 +9 -8
index 72ab2a248d..af6bc40575 100644
@@ -24,7 +24,7 @@ import re
 import iotests
 import copy
 import json
-from iotests import qemu_img, qemu_io
+from iotests import create_test_image, remove_test_image, qemu_io
 
 hd_path = [
     os.path.join(iotests.test_dir, 'hd0.img'),
@@ -43,9 +43,9 @@ class TestBlockdevReopen(iotests.QMPTestCase):
     total_io_cmds = 0
 
     def setUp(self):
-        qemu_img('create', '-f', iotests.imgfmt, hd_path[0], '3M')
-        qemu_img('create', '-f', iotests.imgfmt, '-b', hd_path[0], hd_path[1])
-        qemu_img('create', '-f', iotests.imgfmt, hd_path[2], '3M')
+        create_test_image(hd_path[0], '3M')
+        create_test_image(hd_path[1], backing_file=hd_path[0])
+        create_test_image(hd_path[2], '3M')
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xa0  0 1M', hd_path[0])
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xa1 1M 1M', hd_path[1])
         qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xa2 2M 1M', hd_path[2])
@@ -55,9 +55,9 @@ class TestBlockdevReopen(iotests.QMPTestCase):
     def tearDown(self):
         self.vm.shutdown()
         self.check_qemu_io_errors()
-        os.remove(hd_path[0])
-        os.remove(hd_path[1])
-        os.remove(hd_path[2])
+        remove_test_image(hd_path[0])
+        remove_test_image(hd_path[1])
+        remove_test_image(hd_path[2])
 
     # The output of qemu-io is not returned by vm.hmp_qemu_io() but
     # it's stored in the log and can only be read when the VM has been
@@ -328,7 +328,8 @@ class TestBlockdevReopen(iotests.QMPTestCase):
 
         # First we create a 2MB raw file, and fill each half with a
         # different value
-        qemu_img('create', '-f', 'raw', hd_path[0], '2M')
+        remove_test_image(hd_path[0])
+        create_test_image(hd_path[0], '2M', fmt='raw')
         qemu_io('-f', 'raw', '-c', 'write -P 0xa0  0 1M', hd_path[0])
         qemu_io('-f', 'raw', '-c', 'write -P 0xa1 1M 1M', hd_path[0])
 
Signature · 1 line
2.21.0
author
#70 lore raw

PATCH 58/67 iotests/246: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/246 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/246 +3 -3
index d1364d5901..2224c61545 100755
@@ -20,7 +20,7 @@
 # [email protected]
 
 import iotests
-from iotests import log
+from iotests import log, create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2'],
                           # compat=1.1 is needed for dirty bitmaps
@@ -35,11 +35,11 @@ def query_bitmaps(vm):
     return { "bitmaps": { device['device']: device.get('dirty-bitmaps', []) for
                           device in res['return'] } }
 
-with iotests.FilePath('img') as img_path, \
+with iotests.ImagePath('img') as img_path, \
      iotests.VM() as vm:
 
     log('--- Preparing image & VM ---\n')
-    iotests.qemu_img_create('-f', iotests.imgfmt, img_path, str(size))
+    create_test_image(img_path, size)
     vm.add_drive(img_path)
 
 
Signature · 1 line
2.21.0
author
#71 lore raw

PATCH 59/67 iotests/248: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/248 | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tests/qemu-iotests/248 +4 -4
index 0adc4da802..90821eaeda 100755
@@ -19,7 +19,7 @@
 #
 
 import iotests
-from iotests import qemu_img_create, qemu_io, file_path, filter_qmp_testfiles
+from iotests import create_test_image, qemu_io, image_path, filter_qmp_testfiles
 
 iotests.script_initialize(supported_fmts=['qcow2'],
                           # With an external data file, we would need
@@ -27,12 +27,12 @@ iotests.script_initialize(supported_fmts=['qcow2'],
                           # of @file
                           unsupported_imgopts=['data_file'])
 
-source, target = file_path('source', 'target')
+source, target = image_path('source', 'target')
 size = 5 * 1024 * 1024
 limit = 2 * 1024 * 1024
 
-qemu_img_create('-f', iotests.imgfmt, source, str(size))
-qemu_img_create('-f', iotests.imgfmt, target, str(size))
+create_test_image(source, size)
+create_test_image(target, size)
 qemu_io('-c', 'write 0 {}'.format(size), source)
 
 # raw format don't like empty files
Signature · 1 line
2.21.0
author
#72 lore raw

PATCH 60/67 iotests/254: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/254 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/254 +3 -3
index 0404faf853..b44771fbc6 100755
@@ -19,16 +19,16 @@
 #
 
 import iotests
-from iotests import qemu_img_create, file_path, log
+from iotests import create_test_image, image_path, log
 
 iotests.script_initialize(supported_fmts=['qcow2'],
                           # compat=1.1 is needed for dirty bitmaps
                           unsupported_imgopts=['compat=0.10'])
 
-disk, top = file_path('disk', 'top')
+disk, top = image_path('disk', 'top')
 size = 1024 * 1024
 
-qemu_img_create('-f', iotests.imgfmt, disk, str(size))
+create_test_image(disk, size)
 
 vm = iotests.VM().add_drive(disk, opts='node-name=base')
 vm.launch()
Signature · 1 line
2.21.0
author
#73 lore raw

PATCH 61/67 iotests/255: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/255     | 20 +++++++++-----------
 tests/qemu-iotests/255.out |  8 --------
 2 files changed, 9 insertions(+), 19 deletions(-)
tests/qemu-iotests/255 +9 -11
index ff16402268..4e745b68fa 100755
@@ -21,7 +21,7 @@
 #
 
 import iotests
-from iotests import imgfmt
+from iotests import imgfmt, create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
@@ -39,8 +39,8 @@ iotests.log('Finishing a commit job with background reads')
 iotests.log('============================================')
 iotests.log('')
 
-with iotests.FilePath('t.qcow2') as disk_path, \
-     iotests.FilePath('t.qcow2.mid') as mid_path, \
+with iotests.ImagePath('t.qcow2') as disk_path, \
+     iotests.ImagePath('t.qcow2.mid') as mid_path, \
      iotests.FilePath('t.qcow2.base') as base_path, \
      iotests.VM() as vm:
 
@@ -48,11 +48,10 @@ with iotests.FilePath('t.qcow2') as disk_path, \
     iotests.log("")
 
     size = 128 * 1024 * 1024
-    size_str = str(size)
 
     iotests.create_image(base_path, size)
-    iotests.qemu_img_log('create', '-f', iotests.imgfmt, mid_path, size_str)
-    iotests.qemu_img_log('create', '-f', iotests.imgfmt, disk_path, size_str)
+    assert create_test_image(mid_path, size) == 0
+    assert create_test_image(disk_path, size) == 0
 
     # Create a backing chain like this:
     # base <- [throttled: bps-read=4096] <- mid <- overlay
@@ -91,18 +90,17 @@ iotests.log('Closing the VM while a job is being cancelled')
 iotests.log('=============================================')
 iotests.log('')
 
-with iotests.FilePath('src.qcow2') as src_path, \
-     iotests.FilePath('dst.qcow2') as dst_path, \
+with iotests.ImagePath('src.qcow2') as src_path, \
+     iotests.ImagePath('dst.qcow2') as dst_path, \
      iotests.VM() as vm:
 
     iotests.log('=== Create images and start VM ===')
     iotests.log('')
 
     size = 128 * 1024 * 1024
-    size_str = str(size)
 
-    iotests.qemu_img_log('create', '-f', iotests.imgfmt, src_path, size_str)
-    iotests.qemu_img_log('create', '-f', iotests.imgfmt, dst_path, size_str)
+    assert create_test_image(src_path, size) == 0
+    assert create_test_image(dst_path, size) == 0
 
     iotests.log(iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write 0 1M',
                                 src_path),
tests/qemu-iotests/255.out +0 -8
index 348909fdef..2e837cbb5f 100644
@@ -3,10 +3,6 @@ Finishing a commit job with background reads
 
 === Create backing chain and start VM ===
 
-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16
-
-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16
-
 === Start background read requests ===
 
 === Run a commit job ===
@@ -23,10 +19,6 @@ Closing the VM while a job is being cancelled
 
 === Create images and start VM ===
 
-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16
-
-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16
-
 wrote 1048576/1048576 bytes at offset 0
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
Signature · 1 line
2.21.0
author
#74 lore raw

PATCH 62/67 iotests/256: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/256 | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/256 +5 -5
index d2f9212e5a..7c78a123ad 100755
@@ -21,13 +21,13 @@
 
 import os
 import iotests
-from iotests import log
+from iotests import log, create_test_image
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 size = 64 * 1024 * 1024
 
-with iotests.FilePath('img0') as img0_path, \
-     iotests.FilePath('img1') as img1_path, \
+with iotests.ImagePath('img0') as img0_path, \
+     iotests.ImagePath('img1') as img1_path, \
      iotests.FilePath('img0-full') as img0_full_path, \
      iotests.FilePath('img1-full') as img1_full_path, \
      iotests.FilePath('img0-incr') as img0_incr_path, \
@@ -62,8 +62,8 @@ with iotests.FilePath('img0') as img0_path, \
     vm.add_object('iothread,id=iothread1')
     vm.add_device('virtio-scsi-pci,id=scsi0,iothread=iothread0')
     vm.add_device('virtio-scsi-pci,id=scsi1,iothread=iothread1')
-    iotests.qemu_img_create('-f', iotests.imgfmt, img0_path, str(size))
-    iotests.qemu_img_create('-f', iotests.imgfmt, img1_path, str(size))
+    create_test_image(img0_path, size)
+    create_test_image(img1_path, size)
     vm.add_drive(img0_path, interface='none')
     vm.add_device('scsi-hd,id=device0,drive=drive0,bus=scsi0.0')
     vm.add_drive(img1_path, interface='none')
Signature · 1 line
2.21.0
author
#75 lore raw

PATCH 63/67 iotests/257: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/257 | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
tests/qemu-iotests/257 +7 -7
index 82c10e9b52..ea38945a71 100755
@@ -23,7 +23,7 @@ import math
 import os
 
 import iotests
-from iotests import log, qemu_img
+from iotests import log, create_test_image, qemu_img
 
 SIZE = 64 * 1024 * 1024
 GRANULARITY = 64 * 1024
@@ -152,7 +152,7 @@ class Drive:
     def img_create(self, fmt, size):
         self.fmt = fmt
         self.size = size
-        iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+        create_test_image(self.path, self.size)
 
     def create_target(self, name, fmt, size):
         basename = os.path.basename(self.path)
@@ -273,10 +273,10 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
                         an incomplete backup. Testing limitations prevent
                         testing competing writes.
     """
-    with iotests.FilePaths(['img', 'bsync1', 'bsync2',
+    with iotests.ImagePath('img') as img_path, \
+        iotests.FilePaths(['bsync1', 'bsync2',
                             'fbackup0', 'fbackup1', 'fbackup2']) as \
-                            (img_path, bsync1, bsync2,
-                             fbackup0, fbackup1, fbackup2), \
+                            (bsync1, bsync2, fbackup0, fbackup1, fbackup2), \
          iotests.VM() as vm:
 
         mode = "Mode {:s}; Bitmap Sync {:s}".format(msync_mode, bsync_mode)
@@ -439,8 +439,8 @@ def test_backup_api():
     """
     Test malformed and prohibited invocations of the backup API.
     """
-    with iotests.FilePaths(['img', 'bsync1']) as \
-         (img_path, backup_path), \
+    with iotests.ImagePath('img') as img_path, \
+         iotests.FilePath('bsync1') as backup_path, \
          iotests.VM() as vm:
 
         log("\n=== API failure tests ===\n")
Signature · 1 line
2.21.0
author
#76 lore raw

PATCH 64/67 iotests/258: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/258 | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
tests/qemu-iotests/258 +8 -8
index 1372522c7a..3e9505d8d4 100755
@@ -20,7 +20,7 @@
 # Creator/Owner: Max Reitz <[email protected]>
 
 import iotests
-from iotests import log, qemu_img, qemu_io_silent, \
+from iotests import log, create_test_image, qemu_io_silent, \
         filter_qmp_testfiles, filter_qmp_imgfmt
 
 # Returns a node for blockdev-add
@@ -65,22 +65,22 @@ def test_concurrent_finish(write_to_stream_node):
     # and the other way around, depending on whether the commit job
     # is finalized before stream completes or not.
 
-    with iotests.FilePath('node4.img') as node4_path, \
-         iotests.FilePath('node3.img') as node3_path, \
-         iotests.FilePath('node2.img') as node2_path, \
-         iotests.FilePath('node1.img') as node1_path, \
-         iotests.FilePath('node0.img') as node0_path, \
+    with iotests.ImagePath('node4.img') as node4_path, \
+         iotests.ImagePath('node3.img') as node3_path, \
+         iotests.ImagePath('node2.img') as node2_path, \
+         iotests.ImagePath('node1.img') as node1_path, \
+         iotests.ImagePath('node0.img') as node0_path, \
          iotests.VM() as vm:
 
         # It is important to use raw for the base layer (so that
         # permissions are just handed through to the protocol layer)
-        assert qemu_img('create', '-f', 'raw', node0_path, '64M') == 0
+        assert create_test_image(node0_path, '64M', fmt='raw') == 0
 
         stream_throttle=None
         commit_throttle=None
 
         for path in [node1_path, node2_path, node3_path, node4_path]:
-            assert qemu_img('create', '-f', iotests.imgfmt, path, '64M') == 0
+            assert create_test_image(path, '64M') == 0
 
         if write_to_stream_node:
             # This is what (most of the time) makes commit finish
Signature · 1 line
2.21.0
author
#77 lore raw

PATCH 65/67 iotests/262: Honor $IMGOPTS

In reply to Max Reitz
---
 tests/qemu-iotests/262 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/262 +3 -2
index 6879665f99..b436fb3e14 100755
@@ -21,11 +21,12 @@
 # iothread just for fun.
 
 import iotests
+from iotests import create_test_image
 import os
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
-with iotests.FilePath('img') as img_path, \
+with iotests.ImagePath('img') as img_path, \
      iotests.FilePath('mig_fifo') as fifo, \
      iotests.VM(path_suffix='a') as vm_a, \
      iotests.VM(path_suffix='b') as vm_b:
@@ -49,7 +50,7 @@ with iotests.FilePath('img') as img_path, \
 
         vm.add_device('virtio-blk,drive=%s,iothread=iothread0' % root)
 
-    iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M')
+    create_test_image(img_path, '64M')
 
     os.mkfifo(fifo)
 
Signature · 1 line
2.21.0
author
#78 lore raw

PATCH 66/67 iotests.py: Forbid qemu_img*('create', ...)

In reply to Max Reitz

All tests should use create_test_image().

---
 tests/qemu-iotests/iotests.py | 36 +++++++++++++++++++++++++++--------
 1 file changed, 28 insertions(+), 8 deletions(-)
tests/qemu-iotests/iotests.py +28 -8
index f3c80ba4e9..2e377b8379 100644
@@ -82,12 +82,20 @@ luks_default_secret_object = 'secret,id=keysec0,data=' + \
 luks_default_key_secret_opt = 'key-secret=keysec0'
 
 
-def qemu_img(*args):
+# @from_create_test_image is purely for use in create_test_image
+def qemu_img(*args, from_create_test_image=False):
     '''Run qemu-img and return the exit code'''
+    args = list(args)
+
+    assert args[0] != 'create' or from_create_test_image, \
+        'Use iotests.create_test_image for image creation'
+
     devnull = open('/dev/null', 'r+')
-    exitcode = subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull)
+    exitcode = subprocess.call(qemu_img_args + args,
+                               stdin=devnull, stdout=devnull)
     if exitcode < 0:
-        sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
+        sys.stderr.write('qemu-img received signal %i: %s\n' %
+                         (-exitcode, ' '.join(qemu_img_args + args)))
     return exitcode
 
 def ordered_qmp(qmsg, conv_keys=True):
@@ -162,7 +170,7 @@ def create_test_image(filename, size=None, fmt=imgfmt, opts=[],
     if size is not None:
         args.append(str(size))
 
-    return qemu_img(*args)
+    return qemu_img(*args, from_create_test_image=True)
 
 # Use this to remove images create with create_test_image in the
 # default image format (iotests.imgfmt)
@@ -180,20 +188,32 @@ def remove_test_image(filename):
 
 def qemu_img_verbose(*args):
     '''Run qemu-img without suppressing its output and return the exit code'''
-    exitcode = subprocess.call(qemu_img_args + list(args))
+    args = list(args)
+
+    assert args[0] != 'create', \
+            'Use iotests.create_test_image for image creation'
+
+    exitcode = subprocess.call(qemu_img_args + args)
     if exitcode < 0:
-        sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
+        sys.stderr.write('qemu-img received signal %i: %s\n' %
+                         (-exitcode, ' '.join(qemu_img_args + args)))
     return exitcode
 
 def qemu_img_pipe(*args):
     '''Run qemu-img and return its output'''
-    subp = subprocess.Popen(qemu_img_args + list(args),
+    args = list(args)
+
+    assert args[0] != 'create', \
+            'Use iotests.create_test_image for image creation'
+
+    subp = subprocess.Popen(qemu_img_args + args,
                             stdout=subprocess.PIPE,
                             stderr=subprocess.STDOUT,
                             universal_newlines=True)
     exitcode = subp.wait()
     if exitcode < 0:
-        sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
+        sys.stderr.write('qemu-img received signal %i: %s\n' %
+                         (-exitcode, ' '.join(qemu_img_args + args)))
     return subp.communicate()[0]
 
 def qemu_img_log(*args):
Signature · 1 line
2.21.0
#79 lore raw
In reply to Max Reitz
Quoted 2 lines · Max Reitz

First of all: Sorry.

Thank you for finding the time to do it.

Quoted 43 lines

Second:

Based-on: My block branch
(https://github.com/XanClic/qemu.git block)

Based-on: [email protected]
(“iotests: use python logging”)

Based-on: [email protected]
(“iotests: Allow ./check -o data_file”)

Based-on: [email protected]
(“iotests: Selfish patches”)

Based-on: [email protected]
(“block: Skip COR for inactive nodes”)

OK, now:

Hi,

My recent series “iotests: Allow ./check -o data_file” enabled our bash tests to interpret the data_file qcow2 option. It didn’t do anything for Python tests because those currently completely ignore all image format options.

This is where it gets hairy. To do so, we need two things: First of all, whatever way Python tests use to create images needs to interpret $IMGOPTS. Second, when deleting image files, they must not use a plain os.remove(), but a special function that will clean up data files, too.

The heap of patches in this series comes from making the Python tests use these new functions.

Most Python tests just run qemu-img through a helper function that does not care about the exact subcommand to create images. I could add $IMGOPTS support to it, but that doesn’t feel quite right to me, and it wouldn’t reduce the patch count because we still need a special removal function.

Quoted 4 lines

This series is structured as follows:
- Patches 1 through 7 add support to handle image files differently from other files (consider $IMGOPTS when creating them, consider data files when deleting them, separate ImagePaths from FilePaths, and so on)

OK, that makes sense. I suppose we've been playing a bit fast and loose with these such things.

Quoted 10 lines

- Patches 8 and 9 add two filters we’ll need in the next range:

- Patches 10 through 13 address some issues in a handful of tests that just need to be changed a little so they can overall work with some format options

- Patch 14 makes all tests pass unsupported_imgopts where there are options that they cannot support.

Quoted 8 lines

- Patches 15 through 65 make all Python tests use the new functions introduced in the first 7 patches so they no longer ignore $IMGOPTS.

I felt like this is much better than munching everything together into a single big commit (better to rebase, better to review), and I don’t really like ideas like “Just do five patches that each address ten iotests”.

This is the right approach, for the exact reasons you specify.

Quoted 3 lines

But I’m still very much open to suggestions on how to combine these many small patches to reduce the overall patch count.

You could group them by release windows, if you really wanted to;

- [...etc...]
- Update iotests added for 3.2
- Update iotests added for 4.0
- Update iotests added for 4.1
- Individual patches thereafter

But maybe that doesn't really solve anything for anyone. If you didn't find a more obvious grouping for these, I'd just leave it alone. I'll get to reviewing them.

Quoted 13 lines

- Patch 66 ensures that Python tests always use the new function to create test images so they won’t bypass $IMGOPTS.

- Patch 67 cleans up. qemu_img_log() is only used for image creation, and I don’t see the point in that. The output is predictable and it is very unlikely to fail. We can see in the bash tests that regularly we basically just filter everything from it anyway.
(So this series replaces log(qemu_img_pipe()) instances by asserting that image creation did not fail.)
((qemu_img_create() obviously no longer has any use after this series.))

Quoted 4 lines

After this series, running the iotests with -o compat=0.10, -o refcount_bits=1, and -o 'data_file=$TEST_IMG.data_file' does something sensible even for the Python tests, and it passes.

No minor accomplishment.

I'll make sure to review at least 1-14, but not before Friday.

Quoted 130 lines

Max Reitz (67):
iotests.py: Read $IMGOPTS
iotests.py: Add @skip_for_imgopts() iotests.py: Add unsupported_imgopts iotests.py: create_test_image, remove_test_image iotests.py: Add ImagePaths
iotests.py: Add image_path()
iotests.py: Filter data_file in filter_img_info iotests.py: Add filter_json_filename() iotests.py: Add @hide_fields to img_info_log iotests/169: Skip persistent cases for compat=0.10 iotests/224: Filter json:{} from commit command iotests/228: Filter json:{} filenames iotests/242: Hide refcount bit information
iotests: Use unsupported_imgopts in Python tests iotests/030: Honor $IMGOPTS
iotests/040: Honor $IMGOPTS
iotests/041: Honor $IMGOPTS
iotests/044: Honor $IMGOPTS
iotests/045: Honor $IMGOPTS
iotests/055: Honor $IMGOPTS
iotests/056: Honor $IMGOPTS
iotests/057: Honor $IMGOPTS
iotests/065: Honor $IMGOPTS
iotests/096: Honor $IMGOPTS
iotests/118: Honor $IMGOPTS
iotests/124: Honor $IMGOPTS
iotests/129: Honor $IMGOPTS
iotests/132: Honor $IMGOPTS
iotests/139: Honor $IMGOPTS
iotests/147: Honor $IMGOPTS
iotests/148: Honor $IMGOPTS
iotests/151: Honor $IMGOPTS
iotests/152: Honor $IMGOPTS
iotests/155: Honor $IMGOPTS
iotests/163: Honor $IMGOPTS
iotests/165: Honor $IMGOPTS
iotests/169: Honor $IMGOPTS
iotests/194: Honor $IMGOPTS
iotests/196: Honor $IMGOPTS
iotests/199: Honor $IMGOPTS
iotests/202: Honor $IMGOPTS
iotests/203: Honor $IMGOPTS
iotests/205: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/216: Honor $IMGOPTS
iotests/218: Honor $IMGOPTS
iotests/219: Honor $IMGOPTS
iotests/222: Honor $IMGOPTS
iotests/224: Honor $IMGOPTS
iotests/228: Honor $IMGOPTS
iotests/234: Honor $IMGOPTS
iotests/235: Honor $IMGOPTS
iotests/236: Honor $IMGOPTS
iotests/237: Honor $IMGOPTS
iotests/242: Honor $IMGOPTS
iotests/245: Honor $IMGOPTS
iotests/246: Honor $IMGOPTS
iotests/248: Honor $IMGOPTS
iotests/254: Honor $IMGOPTS
iotests/255: Honor $IMGOPTS
iotests/256: Honor $IMGOPTS
iotests/257: Honor $IMGOPTS
iotests/258: Honor $IMGOPTS
iotests/262: Honor $IMGOPTS
iotests.py: Forbid qemu_img*('create', ...) iotests.py: Drop qemu_img_log(), qemu_img_create()

 tests/qemu-iotests/030        |  69 ++++++------
 tests/qemu-iotests/040        |  42 ++++----
 tests/qemu-iotests/041        | 108 +++++++++----------
 tests/qemu-iotests/044        |  11 +-
 tests/qemu-iotests/045        |  26 ++---
 tests/qemu-iotests/055        |  41 +++----
 tests/qemu-iotests/056        |  30 +++---
 tests/qemu-iotests/057        |  10 +-
 tests/qemu-iotests/065        |  21 ++--
 tests/qemu-iotests/096        |   5 +-
 tests/qemu-iotests/118        |  26 ++---
 tests/qemu-iotests/124        |  29 +++--
 tests/qemu-iotests/129        |  11 +-
 tests/qemu-iotests/132        |   6 +-
 tests/qemu-iotests/139        |  15 ++-
 tests/qemu-iotests/147        |  11 +-
 tests/qemu-iotests/148        |   5 +-
 tests/qemu-iotests/151        |  10 +-
 tests/qemu-iotests/152        |   6 +-
 tests/qemu-iotests/155        |  29 +++--
 tests/qemu-iotests/163        |  29 ++---
 tests/qemu-iotests/165        |  10 +-
 tests/qemu-iotests/169        |  23 ++--
 tests/qemu-iotests/194        |   9 +-
 tests/qemu-iotests/196        |  10 +-
 tests/qemu-iotests/199        |  10 +-
 tests/qemu-iotests/202        |   9 +-
 tests/qemu-iotests/203        |   9 +-
 tests/qemu-iotests/205        |   7 +-
 tests/qemu-iotests/206        |   5 +-
 tests/qemu-iotests/208        |   5 +-
 tests/qemu-iotests/209        |   9 +-
 tests/qemu-iotests/216        |  11 +-
 tests/qemu-iotests/218        |   6 +-
 tests/qemu-iotests/219        |   5 +-
 tests/qemu-iotests/222        |  13 +--
 tests/qemu-iotests/224        |  33 +++---
 tests/qemu-iotests/224.out    |   4 +-
 tests/qemu-iotests/228        |  25 ++---
 tests/qemu-iotests/228.out    |   8 +-
 tests/qemu-iotests/234        |   9 +-
 tests/qemu-iotests/235        |   7 +-
 tests/qemu-iotests/236        |   6 +-
 tests/qemu-iotests/237        |  15 +--
 tests/qemu-iotests/237.out    |   6 --
 tests/qemu-iotests/242        |  21 ++--
 tests/qemu-iotests/242.out    |   5 -
 tests/qemu-iotests/245        |  21 ++--
 tests/qemu-iotests/246        |  11 +-
 tests/qemu-iotests/248        |  14 ++-
 tests/qemu-iotests/254        |  10 +-
 tests/qemu-iotests/255        |  20 ++--
 tests/qemu-iotests/255.out    |   8 --
 tests/qemu-iotests/256        |  10 +-
 tests/qemu-iotests/257        |  18 ++--
 tests/qemu-iotests/258        |  16 +--
 tests/qemu-iotests/262        |   5 +-
 tests/qemu-iotests/iotests.py | 197 +++++++++++++++++++++++++++-------
 58 files changed, 654 insertions(+), 496 deletions(-)
author
#80 lore raw

PATCH 67/67 iotests.py: Drop qemu_img_log(), qemu_img_create()

In reply to Max Reitz

They are both unused now.

---
 tests/qemu-iotests/iotests.py | 24 ------------------------
 1 file changed, 24 deletions(-)
tests/qemu-iotests/iotests.py +0 -24
index 2e377b8379..2476f3744a 100644
@@ -111,25 +111,6 @@ def ordered_qmp(qmsg, conv_keys=True):
         return od
     return qmsg
 
-def qemu_img_create(*args):
-    args = list(args)
-
-    # default luks support
-    if '-f' in args and args[args.index('-f') + 1] == 'luks':
-        if '-o' in args:
-            i = args.index('-o')
-            if 'key-secret' not in args[i + 1]:
-                args[i + 1].append(luks_default_key_secret_opt)
-                args.insert(i + 2, '--object')
-                args.insert(i + 3, luks_default_secret_object)
-        else:
-            args = ['-o', luks_default_key_secret_opt,
-                    '--object', luks_default_secret_object] + args
-
-    args.insert(0, 'create')
-
-    return qemu_img(*args)
-
 def create_test_image(filename, size=None, fmt=imgfmt, opts=[],
                       backing_file=None, backing_fmt=None,
                       objects=[], unsafe=False):
@@ -216,11 +197,6 @@ def qemu_img_pipe(*args):
                          (-exitcode, ' '.join(qemu_img_args + args)))
     return subp.communicate()[0]
 
-def qemu_img_log(*args):
-    result = qemu_img_pipe(*args)
-    log(result, filters=[filter_testfiles])
-    return result
-
 def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[],
                  hide_fields=[]):
     args = [ 'info' ]
Signature · 1 line
2.21.0