iotests: Honor $IMGOPTS in Python tests
cover iotests: Honor $IMGOPTS in Python tests
First of all: Sorry.
Second:
Based-on: My block branch
(https://github.com/XanClic/qemu.git block)
Based-on: [email protected]
(“iotests: use python logging”)
Based-on: [email protected]
(“iotests: Allow ./check -o data_file”)
Based-on: [email protected]
(“iotests: Selfish patches”)
Based-on: [email protected]
(“block: Skip COR for inactive nodes”)
OK, now:
Hi,
My recent series “iotests: Allow ./check -o data_file” enabled our bash tests to interpret the data_file qcow2 option. It didn’t do anything for Python tests because those currently completely ignore all image format options.
This is where it gets hairy. To do so, we need two things: First of all, whatever way Python tests use to create images needs to interpret $IMGOPTS. Second, when deleting image files, they must not use a plain os.remove(), but a special function that will clean up data files, too.
The heap of patches in this series comes from making the Python tests use these new functions.
Most Python tests just run qemu-img through a helper function that does not care about the exact subcommand to create images. I could add $IMGOPTS support to it, but that doesn’t feel quite right to me, and it wouldn’t reduce the patch count because we still need a special removal function.
This series is structured as follows:
- Patches 1 through 7 add support to handle image files differently from other files (consider $IMGOPTS when creating them, consider data files when deleting them, separate ImagePaths from FilePaths, and so on)
- Patches 8 and 9 add two filters we’ll need in the next range:
- Patches 10 through 13 address some issues in a handful of tests that just need to be changed a little so they can overall work with some format options
- Patch 14 makes all tests pass unsupported_imgopts where there are options that they cannot support.
- Patches 15 through 65 make all Python tests use the new functions introduced in the first 7 patches so they no longer ignore $IMGOPTS.
I felt like this is much better than munching everything together into a single big commit (better to rebase, better to review), and I don’t really like ideas like “Just do five patches that each address ten iotests”.
But I’m still very much open to suggestions on how to combine these many small patches to reduce the overall patch count.
- Patch 66 ensures that Python tests always use the new function to create test images so they won’t bypass $IMGOPTS.
- Patch 67 cleans up. qemu_img_log() is only used for image creation, and I don’t see the point in that. The output is predictable and it is very unlikely to fail. We can see in the bash tests that regularly we basically just filter everything from it anyway.
(So this series replaces log(qemu_img_pipe()) instances by asserting that image creation did not fail.)
((qemu_img_create() obviously no longer has any use after this series.))
After this series, running the iotests with -o compat=0.10, -o refcount_bits=1, and -o 'data_file=$TEST_IMG.data_file' does something sensible even for the Python tests, and it passes.
Max Reitz (67):
iotests.py: Read $IMGOPTS
iotests.py: Add @skip_for_imgopts() iotests.py: Add unsupported_imgopts iotests.py: create_test_image, remove_test_image iotests.py: Add ImagePaths
iotests.py: Add image_path()
iotests.py: Filter data_file in filter_img_info iotests.py: Add filter_json_filename() iotests.py: Add @hide_fields to img_info_log iotests/169: Skip persistent cases for compat=0.10 iotests/224: Filter json:{} from commit command iotests/228: Filter json:{} filenames iotests/242: Hide refcount bit information
iotests: Use unsupported_imgopts in Python tests iotests/030: Honor $IMGOPTS
iotests/040: Honor $IMGOPTS
iotests/041: Honor $IMGOPTS
iotests/044: Honor $IMGOPTS
iotests/045: Honor $IMGOPTS
iotests/055: Honor $IMGOPTS
iotests/056: Honor $IMGOPTS
iotests/057: Honor $IMGOPTS
iotests/065: Honor $IMGOPTS
iotests/096: Honor $IMGOPTS
iotests/118: Honor $IMGOPTS
iotests/124: Honor $IMGOPTS
iotests/129: Honor $IMGOPTS
iotests/132: Honor $IMGOPTS
iotests/139: Honor $IMGOPTS
iotests/147: Honor $IMGOPTS
iotests/148: Honor $IMGOPTS
iotests/151: Honor $IMGOPTS
iotests/152: Honor $IMGOPTS
iotests/155: Honor $IMGOPTS
iotests/163: Honor $IMGOPTS
iotests/165: Honor $IMGOPTS
iotests/169: Honor $IMGOPTS
iotests/194: Honor $IMGOPTS
iotests/196: Honor $IMGOPTS
iotests/199: Honor $IMGOPTS
iotests/202: Honor $IMGOPTS
iotests/203: Honor $IMGOPTS
iotests/205: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/216: Honor $IMGOPTS
iotests/218: Honor $IMGOPTS
iotests/219: Honor $IMGOPTS
iotests/222: Honor $IMGOPTS
iotests/224: Honor $IMGOPTS
iotests/228: Honor $IMGOPTS
iotests/234: Honor $IMGOPTS
iotests/235: Honor $IMGOPTS
iotests/236: Honor $IMGOPTS
iotests/237: Honor $IMGOPTS
iotests/242: Honor $IMGOPTS
iotests/245: Honor $IMGOPTS
iotests/246: Honor $IMGOPTS
iotests/248: Honor $IMGOPTS
iotests/254: Honor $IMGOPTS
iotests/255: Honor $IMGOPTS
iotests/256: Honor $IMGOPTS
iotests/257: Honor $IMGOPTS
iotests/258: Honor $IMGOPTS
iotests/262: Honor $IMGOPTS
iotests.py: Forbid qemu_img*('create', ...) iotests.py: Drop qemu_img_log(), qemu_img_create()
tests/qemu-iotests/030 | 69 ++++++------ tests/qemu-iotests/040 | 42 ++++---- tests/qemu-iotests/041 | 108 +++++++++---------- tests/qemu-iotests/044 | 11 +- tests/qemu-iotests/045 | 26 ++--- tests/qemu-iotests/055 | 41 +++---- tests/qemu-iotests/056 | 30 +++--- tests/qemu-iotests/057 | 10 +- tests/qemu-iotests/065 | 21 ++-- tests/qemu-iotests/096 | 5 +- tests/qemu-iotests/118 | 26 ++--- tests/qemu-iotests/124 | 29 +++-- tests/qemu-iotests/129 | 11 +- tests/qemu-iotests/132 | 6 +- tests/qemu-iotests/139 | 15 ++- tests/qemu-iotests/147 | 11 +- tests/qemu-iotests/148 | 5 +- tests/qemu-iotests/151 | 10 +- tests/qemu-iotests/152 | 6 +- tests/qemu-iotests/155 | 29 +++-- tests/qemu-iotests/163 | 29 ++--- tests/qemu-iotests/165 | 10 +- tests/qemu-iotests/169 | 23 ++-- tests/qemu-iotests/194 | 9 +- tests/qemu-iotests/196 | 10 +- tests/qemu-iotests/199 | 10 +- tests/qemu-iotests/202 | 9 +- tests/qemu-iotests/203 | 9 +- tests/qemu-iotests/205 | 7 +- tests/qemu-iotests/206 | 5 +- tests/qemu-iotests/208 | 5 +- tests/qemu-iotests/209 | 9 +- tests/qemu-iotests/216 | 11 +- tests/qemu-iotests/218 | 6 +- tests/qemu-iotests/219 | 5 +- tests/qemu-iotests/222 | 13 +-- tests/qemu-iotests/224 | 33 +++--- tests/qemu-iotests/224.out | 4 +- tests/qemu-iotests/228 | 25 ++--- tests/qemu-iotests/228.out | 8 +- tests/qemu-iotests/234 | 9 +- tests/qemu-iotests/235 | 7 +- tests/qemu-iotests/236 | 6 +- tests/qemu-iotests/237 | 15 +-- tests/qemu-iotests/237.out | 6 -- tests/qemu-iotests/242 | 21 ++-- tests/qemu-iotests/242.out | 5 - tests/qemu-iotests/245 | 21 ++-- tests/qemu-iotests/246 | 11 +- tests/qemu-iotests/248 | 14 ++- tests/qemu-iotests/254 | 10 +- tests/qemu-iotests/255 | 20 ++-- tests/qemu-iotests/255.out | 8 -- tests/qemu-iotests/256 | 10 +- tests/qemu-iotests/257 | 18 ++-- tests/qemu-iotests/258 | 16 +-- tests/qemu-iotests/262 | 5 +- tests/qemu-iotests/iotests.py | 197 +++++++++++++++++++++++++++------- 58 files changed, 654 insertions(+), 496 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/67 iotests.py: Read $IMGOPTS
We do not do anything with yet, but this is the first step.
--- tests/qemu-iotests/iotests.py | 6 ++++++ 1 file changed, 6 insertions(+)
tests/qemu-iotests/iotests.py +6 -0
@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') +imgopts = os.environ.get('IMGOPTS', '') +if len(imgopts) == 0: + imgopts = [] +else: + imgopts = imgopts.split(',') + socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \
Signature · 1 line
2.21.0
Re: [PATCH 01/67] iotests.py: Read $IMGOPTS
Quoted 12 lines · Max Reitz
We do not do anything with yet, but this is the first step.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 6 ++++++ 1 file changed, 6 insertions(+)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 77
Quoted 9 lines
tests/qemu-iotests/iotests.py +6
-0@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') +imgopts = os.environ.get('IMGOPTS', '') +if len(imgopts) == 0: + imgopts = [] +else: + imgopts = imgopts.split(',') +
Sometimes I think about the type of person I'd like to meet, and I ask myself if it's the type of person who would quote pep8, or say things like "idiomatic python" in a code review when the existing form has no technical problem whatsoever.
I wonder what type of person I am.
Well, questions for another day.
tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 80
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \
Re: [PATCH 01/67] iotests.py: Read $IMGOPTS
Quoted 30 lines · John Snow
Quoted 12 lines · Max Reitz
We do not do anything with yet, but this is the first step.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 6 ++++++ 1 file changed, 6 insertions(+)tests/qemu-iotests/iotests.py +0
-0tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 77
Quoted 9 lines
tests/qemu-iotests/iotests.py +6
-0@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') +imgopts = os.environ.get('IMGOPTS', '') +if len(imgopts) == 0: + imgopts = [] +else: + imgopts = imgopts.split(',') +Sometimes I think about the type of person I'd like to meet, and I ask myself if it's the type of person who would quote pep8, or say things like "idiomatic python" in a code review when the existing form has no technical problem whatsoever.
Quoted 3 lines
I wonder what type of person I am.
Well, questions for another day.
Hope you don't say about quoting, as pep8 "does not make a recommendation for this".
Quoted 8 lines
tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 80
Quoted 5 lines
tests/qemu-iotests/iotests.py +0
-0socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \Reviewed-by John Snow <[email protected]>
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 01/67] iotests.py: Read $IMGOPTS
In reply to Max ReitzQuoted 15 lines · Max Reitz
We do not do anything with yet, but this is the first step.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 6 ++++++ 1 file changed, 6 insertions(+)tests/qemu-iotests/iotests.py +0
-0@@ -69,6 +69,12 @@ output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
tests/qemu-iotests/iotests.py output_dir = os.environ.get('OUTPUT_DIR', '.') · line 80
Quoted 10 lines
tests/qemu-iotests/iotests.py +6
-0+imgopts = os.environ.get('IMGOPTS', '') +if len(imgopts) == 0: + imgopts = [] +else: + imgopts = imgopts.split(',') + socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \
Note, that empty sequences are always false, so you may use just "if not imgopts"
Signature · 2 lines
Best regards, Vladimir
PATCH 2/67 iotests.py: Add @skip_for_imgopts()
In reply to Max Reitz--- tests/qemu-iotests/iotests.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)
tests/qemu-iotests/iotests.py +13 -0
@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False): return func_wrapper return skip_test_decorator +def skip_for_imgopts(unsupported_opts=[]): + '''Skip Test Decorator + Skips the test if imgopts contains any of the given options''' + def skip_test_decorator(func): + def func_wrapper(test_case: QMPTestCase, *args, **kwargs): + for opt in imgopts: + if any(unsupported in opt for unsupported in unsupported_opts): + test_case.case_skip('{}: Option {} is unsupported'.format( + test_case, opt)) + return func(test_case, *args, **kwargs) + return func_wrapper + return skip_test_decorator + def execute_unittest(debug=False): """Executes unittests within the calling module."""
Signature · 1 line
2.21.0
Re: [PATCH 02/67] iotests.py: Add @skip_for_imgopts()
Quoted 10 lines · Max Reitz
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 957
Quoted 8 lines
tests/qemu-iotests/iotests.py +5
-0@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False): return func_wrapper return skip_test_decorator +def skip_for_imgopts(unsupported_opts=[]): + '''Skip Test Decorator + Skips the test if imgopts contains any of the given options''' + def skip_test_decorator(func): + def func_wrapper(test_case: QMPTestCase, *args, **kwargs):
:D
tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 968
Quoted 12 lines
tests/qemu-iotests/iotests.py +8
-0+ for opt in imgopts: + if any(unsupported in opt for unsupported in unsupported_opts): + test_case.case_skip('{}: Option {} is unsupported'.format( + test_case, opt)) + return func(test_case, *args, **kwargs) + return func_wrapper + return skip_test_decorator + def execute_unittest(debug=False): """Executes unittests within the calling module."""
Re: [PATCH 02/67] iotests.py: Add @skip_for_imgopts()
In reply to Max ReitzQuoted 10 lines · Max Reitz
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 957
Quoted 8 lines
tests/qemu-iotests/iotests.py +5
-0@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False): return func_wrapper return skip_test_decorator +def skip_for_imgopts(unsupported_opts=[]): + '''Skip Test Decorator + Skips the test if imgopts contains any of the given options''' + def skip_test_decorator(func): + def func_wrapper(test_case: QMPTestCase, *args, **kwargs):
how about
unsup = set(imgopts) & set(unsupported_opts)
if unsup:
test_case.case_skip('... Options {} are ...', format(..., ', '.join(map(str, unsup)))Quoted 12 lines
tests/qemu-iotests/iotests.py +8
-0+ for opt in imgopts: + if any(unsupported in opt for unsupported in unsupported_opts): + test_case.case_skip('{}: Option {} is unsupported'.format( + test_case, opt)) + return func(test_case, *args, **kwargs) + return func_wrapper + return skip_test_decorator + def execute_unittest(debug=False): """Executes unittests within the calling module."""
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 02/67] iotests.py: Add @skip_for_imgopts()
Quoted 22 lines · Vladimir Sementsov-Ogievskiy
Quoted 10 lines · Max Reitz
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)tests/qemu-iotests/iotests.py +0
-0tests/qemu-iotests/iotests.py def skip_if_unsupported(required_formats=[], read_only=False): · line 957
Quoted 8 lines
tests/qemu-iotests/iotests.py +5
-0@@ -950,6 +950,19 @@ def skip_if_unsupported(required_formats=[], read_only=False): return func_wrapper return skip_test_decorator +def skip_for_imgopts(unsupported_opts=[]): + '''Skip Test Decorator + Skips the test if imgopts contains any of the given options''' + def skip_test_decorator(func): + def func_wrapper(test_case: QMPTestCase, *args, **kwargs):how about
Quoted 3 lines
unsup = set(imgopts) &
set(unsupported_opts)
if unsup:test_case.case_skip('... Options {} are ...', format(..., ', '.join(map(str, unsup)))
Sure.
Max
PATCH 3/67 iotests.py: Add unsupported_imgopts
In reply to Max Reitz--- tests/qemu-iotests/iotests.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +6 -1
@@ -998,7 +998,8 @@ def execute_setup_common(supported_fmts=[], supported_cache_modes=[], unsupported_fmts=[], supported_protocols=[], - unsupported_protocols=[]): + unsupported_protocols=[], + unsupported_imgopts=[]): """ Perform necessary setup for either script-style or unittest-style tests. """
@@ -1016,6 +1017,10 @@ def execute_setup_common(supported_fmts=[], verify_platform(supported=supported_platforms) verify_cache_mode(supported_cache_modes) + for opt in imgopts: + if any(unsupported in opt for unsupported in unsupported_imgopts): + notrun('not suitable for this option: %s' % opt) + debug = '-d' in sys.argv if debug: sys.argv.remove('-d')
Signature · 1 line
2.21.0
Re: [PATCH 03/67] iotests.py: Add unsupported_imgopts
Quoted 20 lines · Max Reitz
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)tests/qemu-iotests/iotests.py +2
-1@@ -998,7 +998,8 @@ def execute_setup_common(supported_fmts=[], supported_cache_modes=[], unsupported_fmts=[], supported_protocols=[], - unsupported_protocols=[]): + unsupported_protocols=[], + unsupported_imgopts=[]): """ Perform necessary setup for either script-style or unittest-style tests. """
tests/qemu-iotests/iotests.py def execute_setup_common(supported_fmts=[], · line 1021
Quoted 5 lines
tests/qemu-iotests/iotests.py +2
-0@@ -1016,6 +1017,10 @@ def execute_setup_common(supported_fmts=[], verify_platform(supported=supported_platforms) verify_cache_mode(supported_cache_modes) + for opt in imgopts: + if any(unsupported in opt for unsupported in unsupported_imgopts):
any() is cool and I don't use it as often as I should.
tests/qemu-iotests/iotests.py def execute_setup_common(supported_fmts=[], · line 1026
Quoted 6 lines
tests/qemu-iotests/iotests.py +2
-0+ notrun('not suitable for this option: %s' % opt) + debug = '-d' in sys.argv if debug: sys.argv.remove('-d')
Re: [PATCH 03/67] iotests.py: Add unsupported_imgopts
In reply to Max ReitzQuoted 20 lines · Max Reitz
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)tests/qemu-iotests/iotests.py +2
-1@@ -998,7 +998,8 @@ def execute_setup_common(supported_fmts=[], supported_cache_modes=[], unsupported_fmts=[], supported_protocols=[], - unsupported_protocols=[]): + unsupported_protocols=[], + unsupported_imgopts=[]): """ Perform necessary setup for either script-style or unittest-style tests. """
tests/qemu-iotests/iotests.py def execute_setup_common(supported_fmts=[], · line 1023
Quoted 7 lines
tests/qemu-iotests/iotests.py +4
-0@@ -1016,6 +1017,10 @@ def execute_setup_common(supported_fmts=[], verify_platform(supported=supported_platforms) verify_cache_mode(supported_cache_modes) + for opt in imgopts: + if any(unsupported in opt for unsupported in unsupported_imgopts): + notrun('not suitable for this option: %s' % opt) +
same suggestion
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0debug = '-d' in sys.argv if debug: sys.argv.remove('-d')
Signature · 2 lines
Best regards, Vladimir
PATCH 4/67 iotests.py: create_test_image, remove_test_image
In reply to Max ReitzPython tests should use these two new functions instead of qemu_img('create', ...) + os.remove(), so that user-supplied image options are interpreted and handled correctly.
--- tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+)
tests/qemu-iotests/iotests.py +56 -0
@@ -122,6 +122,62 @@ def qemu_img_create(*args): return qemu_img(*args) +def create_test_image(filename, size=None, fmt=imgfmt, opts=[], + backing_file=None, backing_fmt=None, + objects=[], unsafe=False): + if fmt == imgfmt: + # Only use imgopts for the default format + opts = imgopts + opts + + for i, opt in enumerate(opts): + if '$TEST_IMG' in opt: + opts[i] = opt.replace('$TEST_IMG', filename) + + # default luks support + if fmt == 'luks': + if not any('key-secret' in opt for opt in opts): + opts.append(luks_default_key_secret_opt) + objects.append(luks_default_secret_object) + + args = ['create', '-f', fmt] + + if len(opts) > 0: + args += ['-o', ','.join(opts)] + + if backing_file is not None: + args += ['-b', backing_file] + + if backing_fmt is not None: + args += ['-F', backing_fmt] + + if len(objects) > 0: + # Generate a [['--object', $obj], [...], ...] list and flatten it + args += [arg for objarg in (['--object', obj] for obj in objects) \ + for arg in objarg] + + if unsafe: + args.append('-u') + + args.append(filename) + if size is not None: + args.append(str(size)) + + return qemu_img(*args) + +# Use this to remove images create with create_test_image in the +# default image format (iotests.imgfmt) +def remove_test_image(filename): + try: + os.remove(filename) + + data_file = next(opt.replace('data_file=', '') \ + .replace('$TEST_IMG', filename) \ + for opt in imgopts if opt.startswith('data_file=')) + + os.remove(data_file) + except: + pass + def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args))
Signature · 1 line
2.21.0
Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image
Quoted 14 lines · Max Reitz
Python tests should use these two new functions instead of
qemu_img('create', ...)+os.remove(), so that user-supplied image options are interpreted and handled correctly.Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 127
Quoted 6 lines
tests/qemu-iotests/iotests.py +3
-0@@ -122,6 +122,62 @@ def qemu_img_create(*args): return qemu_img(*args) +def create_test_image(filename, size=None, fmt=imgfmt, opts=[], + backing_file=None, backing_fmt=None, + objects=[], unsafe=False):
Python! It's the language that everybody loves and can do no wrong!
Ah, wait, no, maybe the opposite.
You want this:
(..., opts=None, ...):
opts = opts or []
because, unfortunately, default parameters are bound at definition time and not at call time, so the default list here is like a static local.
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 138
Quoted 11 lines
tests/qemu-iotests/iotests.py +11
-0+ if fmt == imgfmt: + # Only use imgopts for the default format + opts = imgopts + opts + + for i, opt in enumerate(opts): + if '$TEST_IMG' in opt: + opts[i] = opt.replace('$TEST_IMG', filename) + + # default luks support + if fmt == 'luks': + if not any('key-secret' in opt for opt in opts):
You can write "if not 'key-secret' in opts"
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 139
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+ opts.append(luks_default_key_secret_opt)
And here we might modify that default list.
Quoted 7 lines
tests/qemu-iotests/iotests.py +7
-0+ objects.append(luks_default_secret_object) + + args = ['create', '-f', fmt] + + if len(opts) > 0: + args += ['-o', ','.join(opts)] +
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 156
Quoted 10 lines
tests/qemu-iotests/iotests.py +10
-0+ if backing_file is not None: + args += ['-b', backing_file] + + if backing_fmt is not None: + args += ['-F', backing_fmt] + + if len(objects) > 0: + # Generate a [['--object', $obj], [...], ...] list and flatten it + args += [arg for objarg in (['--object', obj] for obj in objects) \ + for arg in objarg]
I may have mentioned at one point that I love comprehensions, but dislike nested comprehensions. At this point, I think it's far simpler to say:
for obj in objects:
args.extend(['--object', obj])
or, even shorter:
args += ['--object', obj]
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 167
Quoted 11 lines
tests/qemu-iotests/iotests.py +11
-0+ + if unsafe: + args.append('-u') + + args.append(filename) + if size is not None: + args.append(str(size)) + + return qemu_img(*args) + +# Use this to remove images create with create_test_image in the
created
and you might as well move the # comment to a """docstring""" while you're here.
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 176
Quoted 9 lines
tests/qemu-iotests/iotests.py +9
-0+# default image format (iotests.imgfmt) +def remove_test_image(filename): + try: + os.remove(filename) + + data_file = next(opt.replace('data_file=', '') \ + .replace('$TEST_IMG', filename) \ + for opt in imgopts if opt.startswith('data_file=')) +
Learned something today: you can use next() to get the first value from a generator expression.
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 177
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+ os.remove(data_file)
Keep in mind that if the generator expression returns no results, that next() will throw an exception and we won't make it here. That's ok, but,
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 180
Quoted 3 lines
tests/qemu-iotests/iotests.py +3
-0+ except: + pass +
The unqualified except doesn't help me know which errors you expected and which you didn't.
We have a function like this elsewhere in the python directory:
def remove_if_exists(filename):
try:
os.remove(filename)
except FileNotFoundError:
pass
Can we use that here and remove the try:/except: from this function? It will require you to change the list search to something like this instead:
remove_if_exists(filename)
for opt in (x for x in imgopts if etc):
data_file = opt.replace('etc', 'etc')
remove_if_exists(data_file)to avoid the exception when you call next().
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 183
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args))
My fussiness with the remove() function is just optional picky stuff, but the rest matters, I think.
--js
Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image
Quoted 1 line · John Snow
You can write "if not 'key-secret' in opts"
Ah nah, because it's key-secret= whatever and I'm just being too hasty with what I thought is actually in the opts list.
Carry on ...
Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image
In reply to John SnowQuoted 28 lines · John Snow
Quoted 14 lines · Max Reitz
Python tests should use these two new functions instead of
qemu_img('create', ...)+os.remove(), so that user-supplied image options are interpreted and handled correctly.Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+)tests/qemu-iotests/iotests.py +0
-0tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 127
Quoted 6 lines
tests/qemu-iotests/iotests.py +3
-0@@ -122,6 +122,62 @@ def qemu_img_create(*args): return qemu_img(*args) +def create_test_image(filename, size=None, fmt=imgfmt, opts=[], + backing_file=None, backing_fmt=None, + objects=[], unsafe=False):Python! It's the language that everybody loves and can do no wrong!
Ah, wait, no, maybe the opposite.
Quoted 7 lines
You want this:
(..., opts=None, ...):
opts = opts or []because, unfortunately, default parameters are bound at definition time and not at call time, so the default list here is like a static local.
OK. Interesting.
I suppose the same goes for @objects, then.
Quoted 5 lines
Quoted 5 lines
tests/qemu-iotests/iotests.py +5
-0+ if fmt == imgfmt: + # Only use imgopts for the default format + opts = imgopts + opts + + for i, opt in enumerate(opts):
Quoted 8 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 138
Quoted 6 lines
tests/qemu-iotests/iotests.py +6
-0+ if '$TEST_IMG' in opt: + opts[i] = opt.replace('$TEST_IMG', filename) + + # default luks support + if fmt == 'luks': + if not any('key-secret' in opt for opt in opts):You can write "if not 'key-secret' in opts"
Oh, that’s recursive?
Quoted 16 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 139
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+ opts.append(luks_default_key_secret_opt)And here we might modify that default list.
Quoted 12 lines
tests/qemu-iotests/iotests.py +12
-0+ objects.append(luks_default_secret_object) + + args = ['create', '-f', fmt] + + if len(opts) > 0: + args += ['-o', ','.join(opts)] + + if backing_file is not None: + args += ['-b', backing_file] + + if backing_fmt is not None: + args += ['-F', backing_fmt]
Quoted 8 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 156
Quoted 5 lines
tests/qemu-iotests/iotests.py +5
-0+ + if len(objects) > 0: + # Generate a [['--object', $obj], [...], ...] list and flatten it + args += [arg for objarg in (['--object', obj] for obj in objects) \ + for arg in objarg]I may have mentioned at one point that I love comprehensions, but dislike nested comprehensions.
I can’t remember but I do remember writing this piece of code, being sad that there is no .flatten, and wanting everyone to see the monster that arises.
Quoted 8 lines
At this point, I think it's far simpler
to say:for obj in objects:
args.extend(['--object', obj])or, even shorter:
args += ['--object', obj]
OK, so now you saw it, I’m glad to make the flattening more flattering to read.
Quoted 17 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 167
Quoted 11 lines
tests/qemu-iotests/iotests.py +11
-0+ + if unsafe: + args.append('-u') + + args.append(filename) + if size is not None: + args.append(str(size)) + + return qemu_img(*args) + +# Use this to remove images create with create_test_image in thecreated
and you might as well move the # comment to a """docstring""" while you're here.
Quoted 12 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 176
Quoted 9 lines
tests/qemu-iotests/iotests.py +9
-0+# default image format (iotests.imgfmt) +def remove_test_image(filename): + try: + os.remove(filename) + + data_file = next(opt.replace('data_file=', '') \ + .replace('$TEST_IMG', filename) \ + for opt in imgopts if opt.startswith('data_file=')) +Learned something today: you can use
next()to get the first value from a generator expression.
I was sad for a bit that Python doesn’t have a find(), but then I noticed this works as well. (Already used extensively in “iotests: Add VM.assert_block_path()” from my “block: Fix check_to_replace_node()” series.)
Quoted 4 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 177
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+ os.remove(data_file)Keep in mind that if the generator expression returns no results, that
next()will throw an exception and we won't make it here. That's ok, but,
I did. If there are no results, it’s good we won’t get here.
This code would be wrong if the next() didn’t throw an exception.
Quoted 6 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 180
Quoted 3 lines
tests/qemu-iotests/iotests.py +3
-0+ except: + pass +The unqualified except doesn't help me know which errors you expected and which you didn't.
What I’m expecting: FileNotFound, StopIteration.
But the thing is that I feel like maybe removing a file should always pass, regardless of the exact exception. (I can imagine to be wrong.)
Quoted 7 lines
We have a function like this elsewhere in the python directory:
def
remove_if_exists(filename):
try:
os.remove(filename)
except FileNotFoundError:
pass
We do? I can’t find it. I find a _remove_if_exists in machine.py, which I’m not sure whether it’s supposed to be used outside, and it works a bit different, actually (but probably to the same effect).
Quoted 9 lines
Can we use that here and remove the try:/except: from this function? It will require you to change the list search to something like this instead:
remove_if_exists(filename)
for opt in (x for x in imgopts if etc):data_file = opt.replace('etc', 'etc') remove_if_exists(data_file)to avoid the exception when you call
next().
I don’t know why I’d avoid the exception, though.
This is probably because I don’t like pythonic code, again, but I prefer a next() + exception over a for loop that just iterates once or not at all.
Quoted 7 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 183
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args))My fussiness with the
remove()function is just optional picky stuff, but the rest matters, I think.
OK. Indeed it does!
Max
Re: [PATCH 04/67] iotests.py: create_test_image, remove_test_image
Quoted 34 lines · Max Reitz
Quoted 33 lines · John Snow
Quoted 14 lines · Max Reitz
Python tests should use these two new functions instead of
qemu_img('create', ...)+os.remove(), so that user-supplied image options are interpreted and handled correctly.Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 56 +++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+)tests/qemu-iotests/iotests.py +0
-0tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 127
Quoted 6 lines
tests/qemu-iotests/iotests.py +3
-0@@ -122,6 +122,62 @@ def qemu_img_create(*args): return qemu_img(*args) +def create_test_image(filename, size=None, fmt=imgfmt, opts=[], + backing_file=None, backing_fmt=None, + objects=[], unsafe=False):Python! It's the language that everybody loves and can do no wrong!
Ah, wait, no, maybe the opposite.
You want this:
(..., opts=None, ...):
opts = opts or []
Quoted 7 lines
Quoted 2 lines
because, unfortunately, default parameters are bound at definition time and not at call time, so the default list here is like a static local.
OK. Interesting.
I suppose the same goes for @objects, then.
It is by far the WORST thing about Python.
I realize we use this pattern a few places in iotests, but I think it's also usually where we don't modify the list, so it's actually OK, but serves as an example of a bad habit.
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
tests/qemu-iotests/iotests.py +5
-0+ if fmt == imgfmt: + # Only use imgopts for the default format + opts = imgopts + opts + + for i, opt in enumerate(opts):Quoted 8 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 138
Quoted 6 lines
tests/qemu-iotests/iotests.py +6
-0+ if '$TEST_IMG' in opt: + opts[i] = opt.replace('$TEST_IMG', filename) + + # default luks support + if fmt == 'luks': + if not any('key-secret' in opt for opt in opts):You can write "if not 'key-secret' in opts"
Quoted 2 lines
Oh, that’s recursive?
No, I was just mistaken about the shape of the data.
You are looking for 'key-secret=XXX', I was thinking that there was a token that was really just 'key-secret'.
What you wrote is correct and good and I am wrong and bad.
Quoted 32 lines
Quoted 16 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 139
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+ opts.append(luks_default_key_secret_opt)And here we might modify that default list.
Quoted 12 lines
tests/qemu-iotests/iotests.py +12
-0+ objects.append(luks_default_secret_object) + + args = ['create', '-f', fmt] + + if len(opts) > 0: + args += ['-o', ','.join(opts)] + + if backing_file is not None: + args += ['-b', backing_file] + + if backing_fmt is not None: + args += ['-F', backing_fmt]Quoted 8 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 156
Quoted 5 lines
tests/qemu-iotests/iotests.py +5
-0+ + if len(objects) > 0: + # Generate a [['--object', $obj], [...], ...] list and flatten it + args += [arg for objarg in (['--object', obj] for obj in objects) \ + for arg in objarg]I may have mentioned at one point that I love comprehensions, but dislike nested comprehensions.
I can’t remember but I do remember writing this piece of code, being sad that there is no .flatten, and wanting everyone to see the monster that arises.
Quoted 3 lines
At this point, I think it's far simpler
to say:
Quoted 9 lines
Quoted 5 lines
for obj in objects:
args.extend(['--object', obj])or, even shorter:
args += ['--object', obj]OK, so now you saw it, I’m glad to make the flattening more flattering to read.
I am sorry I ever mentioned liking Python. I will accept your punishments.
Quoted 30 lines
Quoted 17 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 167
Quoted 11 lines
tests/qemu-iotests/iotests.py +11
-0+ + if unsafe: + args.append('-u') + + args.append(filename) + if size is not None: + args.append(str(size)) + + return qemu_img(*args) + +# Use this to remove images create with create_test_image in thecreated
and you might as well move the # comment to a """docstring""" while you're here.
Quoted 12 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 176
Quoted 9 lines
tests/qemu-iotests/iotests.py +9
-0+# default image format (iotests.imgfmt) +def remove_test_image(filename): + try: + os.remove(filename) + + data_file = next(opt.replace('data_file=', '') \ + .replace('$TEST_IMG', filename) \ + for opt in imgopts if opt.startswith('data_file=')) +Learned something today: you can use
next()to get the first value from a generator expression.
Quoted 5 lines
I was sad for a bit that Python doesn’t have a
find(), but then I noticed this works as well. (Already used extensively in “iotests: Add VM.assert_block_path()” from my “block: Fix check_to_replace_node()” series.)
I honestly tried to rewrite this a few times because it looks so chunky, but realized there isn't ... a great way to do this without implying that you might find more than one result.
You can filter to a new list and assert that the length is one, but that's not less chunky.
Quoted 9 lines
Quoted 4 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 177
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+ os.remove(data_file)Keep in mind that if the generator expression returns no results, that
next()will throw an exception and we won't make it here. That's ok, but,I did. If there are no results, it’s good we won’t get here.
This code would be wrong if the
next()didn’t throw an exception.
It just wasn't clear, because the except is doing the lifting for both the remove and the finding.
Oh well, it's not really important.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
tests/qemu-iotests/iotests.py +3
-0+ except: + pass +
Quoted 8 lines
Quoted 2 lines
The unqualified except doesn't help me know which errors you expected and which you didn't.
What I’m expecting: FileNotFound, StopIteration.
But the thing is that I feel like maybe removing a file should always pass, regardless of the exact exception. (I can imagine to be wrong.)
I wonder if that's true ... I just don't know what the full set of errors we might get are. I don't really like exception driven code, honestly.
"It's wrong to catch ANY exception because you might suppress errors too broadly."
"It's wrong to be too specific, because you'll miss cases you meant to catch."
Awful.
Anyway, like I said I was just being fiddly because I found this odd to read, but really don't have suggestions that are clearly nicer, so ... carry on.
Quoted 8 lines
Quoted 7 lines
We have a function like this elsewhere in the python directory:
def
remove_if_exists(filename):
try:
os.remove(filename)
except FileNotFoundError:
pass
Quoted 4 lines
We do? I can’t find it. I find a
_remove_if_existsinmachine.py, which I’m not sure whether it’s supposed to be used outside, and it works a bit different, actually (but probably to the same effect).
Yeah, that's the one. Don't worry about plucking it out here for this, just nothing that we do this in a few places. We might want a util eventually that gets it exactly right.
Or not, because what's "exactly right" anyway. Ah, ah, ah.
Quoted 8 lines
Quoted 8 lines
Can we use that here and remove the try:/except: from this function? It will require you to change the list search to something like this instead:
remove_if_exists(filename)
for opt in (x for x in imgopts if etc):data_file = opt.replace('etc', 'etc') remove_if_exists(data_file)
Quoted 7 lines
Quoted 1 line
to avoid the exception when you call
next().I don’t know why I’d avoid the exception, though.
This is probably because I don’t like pythonic code, again, but I prefer a
next()+ exception over a for loop that just iterates once or not at all.
Nah, Python people LOVE exceptions. They don't like "bare except" statements, though. I am the weird person in that I like to avoid exceptions whenever it's elegant and pretty to do so.
I find exceptions as normal control flow to be quite hard to deal with; but Pythonistas seem to love it.
Quoted 12 lines
Quoted 7 lines
tests/qemu-iotests/iotests.py def qemu_img_create(*args): · line 183
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args))My fussiness with the
remove()function is just optional picky stuff, but the rest matters, I think.OK. Indeed it does!
Max
PATCH 5/67 iotests.py: Add ImagePaths
In reply to Max Reitzcreate_test_image() must be paired with an ImagePath so that the image is properly cleaned up with remove_test_image() instead of just os.remove().
--- tests/qemu-iotests/iotests.py | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-)
tests/qemu-iotests/iotests.py +30 -2
@@ -440,8 +440,8 @@ class FilePaths(object): Use this context manager to generate filenames and ensure that the file gets deleted:: - with FilePaths(['test.img']) as img_path: - qemu_img('create', img_path, '1G') + with FilePaths(['migration.sock']) as migration_sock_path: + # Set up and use UNIX socket on migration.sock # migration_sock_path is automatically deleted """ def __init__(self, names):
@@ -460,6 +460,23 @@ class FilePaths(object): pass return False +class ImagePaths(FilePaths): + """ + Same as FilePaths, except it calls remove_test_image() to clean up + (which ensures that external data files are cleaned up, too). + + Use this class for test images in the default format + (iotests.imgfmt): + + with ImagePaths(['test.img']) as img_path: + create_test_image(img_path, '1G') + # The test image is automatically cleaned up + """ + def __exit__(self, exc_type, exc_val, exc_tb): + for path in self.paths: + remove_test_image(path) + return False + class FilePath(FilePaths): """ FilePath is a specialization of FilePaths that takes a single filename.
@@ -470,6 +487,17 @@ class FilePath(FilePaths): def __enter__(self): return self.paths[0] +class ImagePath(ImagePaths): + """ + ImagePath is a specialization of ImagePaths that takes a single + filename. + """ + def __init__(self, name): + super(ImagePath, self).__init__([name]) + + def __enter__(self): + return self.paths[0] + def file_path_remover(): for path in reversed(file_path_remover.paths): try:
Signature · 1 line
2.21.0
PATCH 6/67 iotests.py: Add image_path()
In reply to Max ReitzJust like we have file_path() as an alternative to FilePath, this is an alternative fo ImagePath.
--- tests/qemu-iotests/iotests.py | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-)
tests/qemu-iotests/iotests.py +12 -5
@@ -505,8 +505,12 @@ def file_path_remover(): except OSError: pass +def image_path_remover(): + for path in reversed(image_path_remover.paths): + remove_test_image(path) -def file_path(*names): + +def file_path(*names, remover=file_path_remover): ''' Another way to get auto-generated filename that cleans itself up. Use is as simple as:
@@ -515,19 +519,22 @@ def file_path(*names): sock = file_path('socket') ''' - if not hasattr(file_path_remover, 'paths'): - file_path_remover.paths = [] - atexit.register(file_path_remover) + if not hasattr(remover, 'paths'): + remover.paths = [] + atexit.register(remover) paths = [] for name in names: filename = file_pattern(name) path = os.path.join(test_dir, filename) - file_path_remover.paths.append(path) + remover.paths.append(path) paths.append(path) return paths[0] if len(paths) == 1 else paths +def image_path(*names): + return file_path(*names, remover=image_path_remover) + def remote_filename(path): if imgproto == 'file': return path
Signature · 1 line
2.21.0
PATCH 7/67 iotests.py: Filter data_file in filter_img_info
In reply to Max ReitzIf $IMGOPTS contains a data_file reference, let filter_img_info filter all data_file information.
--- tests/qemu-iotests/iotests.py | 4 ++++ 1 file changed, 4 insertions(+)
tests/qemu-iotests/iotests.py +4 -0
@@ -380,9 +380,13 @@ def filter_generated_node_ids(msg): def filter_img_info(output, filename): lines = [] + user_data_file = any('data_file' in opt for opt in imgopts) for line in output.split('\n'): if 'disk size' in line or 'actual-size' in line: continue + if user_data_file: + if 'data file' in line: + continue line = line.replace(filename, 'TEST_IMG') \ .replace(imgfmt, 'IMGFMT') line = re.sub('iters: [0-9]+', 'iters: XXX', line)
Signature · 1 line
2.21.0
PATCH 8/67 iotests.py: Add filter_json_filename()
In reply to Max Reitz--- tests/qemu-iotests/iotests.py | 3 +++ 1 file changed, 3 insertions(+)
tests/qemu-iotests/iotests.py +3 -0
@@ -378,6 +378,9 @@ def filter_qmp_testfiles(qmsg): def filter_generated_node_ids(msg): return re.sub("#block[0-9]+", "NODE_NAME", msg) +def filter_json_filename(msg): + return re.sub('json:{.*}', 'json:{ /* filtered */ }', msg) + def filter_img_info(output, filename): lines = [] user_data_file = any('data_file' in opt for opt in imgopts)
Signature · 1 line
2.21.0
PATCH 9/67 iotests.py: Add @hide_fields to img_info_log
In reply to Max ReitzYou can specify fields here that should be hidden from the output.
--- tests/qemu-iotests/iotests.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
tests/qemu-iotests/iotests.py +9 -2
@@ -201,7 +201,8 @@ def qemu_img_log(*args): log(result, filters=[filter_testfiles]) return result -def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]): +def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[], + hide_fields=[]): args = [ 'info' ] if imgopts: args.append('--image-opts')
@@ -210,7 +211,13 @@ def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]): args += extra_args args.append(filename) - output = qemu_img_pipe(*args) + output = qemu_img_pipe(*args).splitlines(keepends=True) + + output = [line for line in output + if not any(line.strip().startswith(field) + for field in hide_fields)] + output = ''.join(output) + if not filter_path: filter_path = filename log(filter_img_info(output, filter_path))
Signature · 1 line
2.21.0
PATCH 10/67 iotests/169: Skip persistent cases for compat=0.10
In reply to Max Reitzcompat=0.10 images cannot store persistent dirty bitmaps; let 169 skip all such test cases when compat=0.10 is in the $IMGOPTS.
--- tests/qemu-iotests/169 | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
tests/qemu-iotests/169 +12 -1
@@ -41,7 +41,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.vm_b.shutdown() os.remove(disk_a) os.remove(disk_b) - os.remove(mig_file) + try: + os.remove(mig_file) + except OSError: + pass def setUp(self): qemu_img('create', '-f', iotests.imgfmt, disk_a, size)
@@ -79,6 +82,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): def do_test_migration_resume_source(self, persistent, migrate_bitmaps): granularity = 512 + if persistent and 'compat=0.10' in iotests.imgopts: + self.case_skip('compat=0.10 does not support persistent dirty ' + + 'bitmaps') + # regions = ((start, count), ...) regions = ((0, 0x10000), (0xf0000, 0x10000),
@@ -137,6 +144,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): shared_storage): granularity = 512 + if persistent and 'compat=0.10' in iotests.imgopts: + self.case_skip('compat=0.10 does not support persistent dirty ' + + 'bitmaps') + # regions = ((start, count), ...) regions = ((0, 0x10000), (0xf0000, 0x10000),
Signature · 1 line
2.21.0
PATCH 11/67 iotests/224: Filter json:{} from commit command
In reply to Max ReitzWe only care that block-commit works with json:{} filenames, we do not need to see their exact values in the output.
--- tests/qemu-iotests/224 | 15 ++++++++++++--- tests/qemu-iotests/224.out | 4 ++-- 2 files changed, 14 insertions(+), 5 deletions(-)
tests/qemu-iotests/224 +12 -3
@@ -22,12 +22,19 @@ import iotests from iotests import log, qemu_img, qemu_io_silent, filter_qmp_testfiles, \ - filter_qmp_imgfmt + filter_qmp_imgfmt, filter_json_filename import json +import re # Need backing file support (for arbitrary backing formats) iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed']) +def filter_json_top_or_base(qmsg): + def _filter(key, value): + if key == 'base' or key == 'top': + return filter_json_filename(value) + return value + return iotests.filter_qmp(qmsg, _filter) # There are two variations of this test: # (1) We do not set filter_node_name. In that case, the commit_top
@@ -104,7 +111,8 @@ for filter_node_name in False, True: top=mid_name, base=base_name, speed=1, - filters=[filter_qmp_testfiles, filter_qmp_imgfmt]) + filters=[filter_qmp_testfiles, filter_qmp_imgfmt, + filter_json_top_or_base]) else: vm.qmp_log('block-commit', job_id='commit',
@@ -112,7 +120,8 @@ for filter_node_name in False, True: top=mid_name, base=base_name, speed=1, - filters=[filter_qmp_testfiles, filter_qmp_imgfmt]) + filters=[filter_qmp_testfiles, filter_qmp_imgfmt, + filter_json_top_or_base]) vm.qmp_log('job-pause', id='commit')
tests/qemu-iotests/224.out +2 -2
@@ -3,7 +3,7 @@ {"execute": "blockdev-add", "arguments": {"backing": {"backing": {"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}, "node-name": "base"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-mid.img"}, "node-name": "mid"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "top"}} {"return": {}} -{"execute": "block-commit", "arguments": {"base": "json:{\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}", "device": "top", "job-id": "commit", "speed": 1, "top": "json:{\"backing\": {\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-mid.img\"}}"}} +{"execute": "block-commit", "arguments": {"base": "json:{ /* filtered */ }", "device": "top", "job-id": "commit", "speed": 1, "top": "json:{ /* filtered */ }"}} {"return": {}} {"execute": "job-pause", "arguments": {"id": "commit"}} {"return": {}}
@@ -12,7 +12,7 @@ {"execute": "blockdev-add", "arguments": {"backing": {"backing": {"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}, "node-name": "base"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-mid.img"}, "node-name": "mid"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "top"}} {"return": {}} -{"execute": "block-commit", "arguments": {"base": "json:{\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}", "device": "top", "filter-node-name": "filter_node", "job-id": "commit", "speed": 1, "top": "json:{\"backing\": {\"backing\": {\"driver\": \"null-co\"}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-base.img\"}}, \"driver\": \"IMGFMT\", \"file\": {\"driver\": \"file\", \"filename\": \"TEST_DIR/PID-mid.img\"}}"}} +{"execute": "block-commit", "arguments": {"base": "json:{ /* filtered */ }", "device": "top", "filter-node-name": "filter_node", "job-id": "commit", "speed": 1, "top": "json:{ /* filtered */ }"}} {"return": {}} {"execute": "job-pause", "arguments": {"id": "commit"}} {"return": {}}
Signature · 1 line
2.21.0
PATCH 12/67 iotests/228: Filter json:{} filenames
In reply to Max ReitzWe only care about where we get json:{} filenames (and what the non-json:{} filenames look like), but not about how the json:{} filenames look.
Filter them so we will not get a different output for qcow2 images with external data files.
--- tests/qemu-iotests/228 | 8 ++++---- tests/qemu-iotests/228.out | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-)
tests/qemu-iotests/228 +4 -4
@@ -22,7 +22,7 @@ import iotests from iotests import log, qemu_img, filter_testfiles, filter_imgfmt, \ - filter_qmp_testfiles, filter_qmp_imgfmt + filter_qmp_testfiles, filter_qmp_imgfmt, filter_json_filename # Need backing file and change-backing-file support iotests.script_initialize(supported_fmts=['qcow2', 'qed'])
@@ -32,14 +32,14 @@ def log_node_info(node): log('') log('bs->filename: ' + node['image']['filename'], - filters=[filter_testfiles, filter_imgfmt]) + filters=[filter_testfiles, filter_imgfmt, filter_json_filename]) log('bs->backing_file: ' + node['backing_file'], - filters=[filter_testfiles, filter_imgfmt]) + filters=[filter_testfiles, filter_imgfmt, filter_json_filename]) if 'backing-image' in node['image']: log('bs->backing->bs->filename: ' + node['image']['backing-image']['filename'], - filters=[filter_testfiles, filter_imgfmt]) + filters=[filter_testfiles, filter_imgfmt, filter_json_filename]) else: log('bs->backing: (none)')
tests/qemu-iotests/228.out +4 -4
@@ -17,7 +17,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img {"execute": "change-backing-file", "arguments": {"backing-file": "null-co://", "device": "node0", "image-node-name": "node0"}} {"return": {}} -bs->filename: json:{"backing": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}} +bs->filename: json:{ /* filtered */ } bs->backing_file: null-co:// bs->backing->bs->filename: TEST_DIR/PID-base.img
@@ -31,7 +31,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img {"execute": "change-backing-file", "arguments": {"backing-file": "file:TEST_DIR/PID-base.img", "device": "node0", "image-node-name": "node0"}} {"return": {}} -bs->filename: json:{"backing": {"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-base.img"}}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}} +bs->filename: json:{ /* filtered */ } bs->backing_file: file:TEST_DIR/PID-base.img bs->backing->bs->filename: TEST_DIR/PID-base.img
@@ -54,7 +54,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img {"execute": "blockdev-add", "arguments": {"backing": "null", "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "node0"}} {"return": {}} -bs->filename: json:{"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}} +bs->filename: json:{ /* filtered */ } bs->backing_file: null-co:// bs->backing->bs->filename: null-co://
@@ -65,7 +65,7 @@ bs->backing->bs->filename: null-co:// {"execute": "blockdev-add", "arguments": {"backing": null, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}, "node-name": "node0"}} {"return": {}} -bs->filename: json:{"backing": null, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}} +bs->filename: json:{ /* filtered */ } bs->backing_file: TEST_DIR/PID-base.img bs->backing: (none)
Signature · 1 line
2.21.0
PATCH 13/67 iotests/242: Hide refcount bit information
In reply to Max ReitzIt does not matter, and this lets the test work with different refcount widths.
--- tests/qemu-iotests/242 | 6 +++--- tests/qemu-iotests/242.out | 5 ----- 2 files changed, 3 insertions(+), 8 deletions(-)
tests/qemu-iotests/242 +3 -3
@@ -35,7 +35,7 @@ flag_offset = 0x5000f def print_bitmap(extra_args): log('qemu-img info dump:\n') - img_info_log(disk, extra_args=extra_args) + img_info_log(disk, extra_args=extra_args, hide_fields=['refcount bits']) result = json.loads(qemu_img_pipe('info', '--force-share', '--output=json', disk)) if 'bitmaps' in result['format-specific']['data']:
@@ -98,9 +98,9 @@ add_bitmap(1, True, False) log('Write an unknown bitmap flag \'{}\' into a new QCOW2 image at offset {}' .format(hex(bitmap_flag_unknown), flag_offset)) toggle_flag(flag_offset) -img_info_log(disk) +img_info_log(disk, hide_fields=['refcount bits']) toggle_flag(flag_offset) log('Unset the unknown bitmap flag \'{}\' in the bitmap directory entry:\n' .format(hex(bitmap_flag_unknown))) -img_info_log(disk) +img_info_log(disk, hide_fields=['refcount bits']) log('Test complete')
tests/qemu-iotests/242.out +0 -5
@@ -13,7 +13,6 @@ cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false - refcount bits: 16 corrupt: false No bitmap in JSON format output
@@ -38,7 +37,6 @@ Format specific information: flags: name: bitmap-1 granularity: 32768 - refcount bits: 16 corrupt: false The same bitmaps in JSON format:
@@ -75,7 +73,6 @@ Format specific information: [0]: auto name: bitmap-2 granularity: 65536 - refcount bits: 16 corrupt: false The same bitmaps in JSON format:
@@ -117,7 +114,6 @@ Format specific information: [1]: auto name: bitmap-2 granularity: 65536 - refcount bits: 16 corrupt: false The same bitmaps in JSON format:
@@ -160,7 +156,6 @@ Format specific information: [0]: auto name: bitmap-0 granularity: 16384 - refcount bits: 16 corrupt: false Test complete
Signature · 1 line
2.21.0
PATCH 14/67 iotests: Use unsupported_imgopts in Python tests
In reply to Max ReitzSet unsupported_imgopts as appropriate for all tests that should make use of it.
--- tests/qemu-iotests/044 | 5 ++++- tests/qemu-iotests/057 | 4 +++- tests/qemu-iotests/065 | 4 +++- tests/qemu-iotests/163 | 5 ++++- tests/qemu-iotests/165 | 4 +++- tests/qemu-iotests/196 | 4 +++- tests/qemu-iotests/206 | 5 ++++- tests/qemu-iotests/222 | 3 ++- tests/qemu-iotests/237 | 4 +++- tests/qemu-iotests/242 | 4 +++- tests/qemu-iotests/245 | 4 +++- tests/qemu-iotests/246 | 5 ++++- tests/qemu-iotests/248 | 6 +++++- tests/qemu-iotests/254 | 4 +++- tests/qemu-iotests/257 | 4 +++- 15 files changed, 50 insertions(+), 15 deletions(-)
tests/qemu-iotests/044 +4 -1
@@ -116,4 +116,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): if __name__ == '__main__': iotests.main(supported_fmts=['qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # These refcount calculations do not work with data_file or + # with any refcount width but the default (of 16) + unsupported_imgopts=['data_file', 'refcount_bits'])
tests/qemu-iotests/057 +3 -1
@@ -257,4 +257,6 @@ class TestSnapshotDelete(ImageSnapshotTestCase): if __name__ == '__main__': iotests.main(supported_fmts=['qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # Snapshots do not work with data_file + unsupported_imgopts=['data_file'])
tests/qemu-iotests/065 +3 -1
@@ -130,4 +130,6 @@ TestQMP = None if __name__ == '__main__': iotests.main(supported_fmts=['qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # Neither of these options would work with compat=0.10 + unsupported_imgopts=['refcount_bits', 'data_file'])
tests/qemu-iotests/163 +4 -1
@@ -168,4 +168,7 @@ ShrinkBaseClass = None if __name__ == '__main__': iotests.main(supported_fmts=['raw', 'qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # We want to set our own refcount_bits, and that will + # not work with compat=0.10 + unsupported_imgopts=['refcount_bits', 'compat=0.10'])
tests/qemu-iotests/165 +3 -1
@@ -104,4 +104,6 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase): if __name__ == '__main__': iotests.main(supported_fmts=['qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # compat=1.1 is needed for dirty bitmaps + unsupported_imgopts=['compat=0.10'])
tests/qemu-iotests/196 +3 -1
@@ -64,4 +64,6 @@ class TestInvalidateAutoclear(iotests.QMPTestCase): if __name__ == '__main__': iotests.main(supported_fmts=['qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # compat=1.1 is needed for autoclear flags + unsupported_imgopts=['compat=0.10'])
tests/qemu-iotests/206 +4 -1
@@ -23,7 +23,10 @@ import iotests from iotests import imgfmt -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + # All options are ignored + unsupported_imgopts=['compat=0.10', 'refcount_bits', + 'data_file']) def blockdev_create(vm, options): result = vm.qmp_log('blockdev-create',
tests/qemu-iotests/222 +2 -1
@@ -25,7 +25,8 @@ import iotests from iotests import log, qemu_img, qemu_io, qemu_io_silent iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', - 'vhdx', 'raw']) + 'vhdx', 'raw'], + unsupported_imgopts=['streamOptimized']) patterns = [("0x5d", "0", "64k"), ("0xd5", "1M", "64k"),
tests/qemu-iotests/237 +3 -1
@@ -24,7 +24,9 @@ import math import iotests from iotests import imgfmt -iotests.script_initialize(supported_fmts=['vmdk']) +iotests.script_initialize(supported_fmts=['vmdk'], + # All options are ignored + unsupported_imgopts=['subformat']) def blockdev_create(vm, options): result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
tests/qemu-iotests/242 +3 -1
@@ -24,7 +24,9 @@ import struct from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \ file_path, img_info_log, log, filter_qemu_io -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + # compat=1.1 is needed for dirty bitmaps + unsupported_imgopts=['compat=0.10']) disk = file_path('disk') chunk = 256 * 1024
tests/qemu-iotests/245 +3 -1
@@ -1002,4 +1002,6 @@ class TestBlockdevReopen(iotests.QMPTestCase): if __name__ == '__main__': iotests.activate_logging() iotests.main(supported_fmts=["qcow2"], - supported_protocols=["file"]) + supported_protocols=["file"], + # reopen options differ with an external data file + unsupported_imgopts=['data_file'])
tests/qemu-iotests/246 +4 -1
@@ -22,7 +22,10 @@ import iotests from iotests import log -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + # compat=1.1 is needed for dirty bitmaps + unsupported_imgopts=['compat=0.10']) + size = 64 * 1024 * 1024 * 1024 gran_small = 32 * 1024 gran_large = 128 * 1024
tests/qemu-iotests/248 +5 -1
@@ -21,7 +21,11 @@ import iotests from iotests import qemu_img_create, qemu_io, file_path, filter_qmp_testfiles -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + # With an external data file, we would need + # to impose the limit on @data-file instead + # of @file + unsupported_imgopts=['data_file']) source, target = file_path('source', 'target') size = 5 * 1024 * 1024
tests/qemu-iotests/254 +3 -1
@@ -21,7 +21,9 @@ import iotests from iotests import qemu_img_create, file_path, log -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + # compat=1.1 is needed for dirty bitmaps + unsupported_imgopts=['compat=0.10']) disk, top = file_path('disk', 'top') size = 1024 * 1024
tests/qemu-iotests/257 +3 -1
@@ -526,4 +526,6 @@ def main(): if __name__ == '__main__': iotests.script_main(main, supported_fmts=['qcow2'], - supported_protocols=['file']) + supported_protocols=['file'], + # blkdebug does not work with an external data file + unsupported_imgopts=['data_file'])
Signature · 1 line
2.21.0
PATCH 15/67 iotests/030: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/030 | 69 ++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 37 deletions(-)
tests/qemu-iotests/030 +32 -37
@@ -21,7 +21,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_io backing_img = os.path.join(iotests.test_dir, 'backing.img') mid_img = os.path.join(iotests.test_dir, 'mid.img')
@@ -32,8 +32,8 @@ class TestSingleDrive(iotests.QMPTestCase): def setUp(self): iotests.create_image(backing_img, TestSingleDrive.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img) + create_test_image(mid_img, backing_file=backing_img) + create_test_image(test_img, backing_file=mid_img) qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 512', backing_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 524288 512', mid_img) self.vm = iotests.VM().add_drive("blkdebug::" + test_img,
@@ -43,8 +43,8 @@ class TestSingleDrive(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(mid_img) + remove_test_image(test_img) + remove_test_image(mid_img) os.remove(backing_img) def test_stream(self):
@@ -155,9 +155,8 @@ class TestSingleDrive(iotests.QMPTestCase): def test_read_only(self): # Create a new file that we can attach (we need a read-only top) - with iotests.FilePath('ro-top.img') as ro_top_path: - qemu_img('create', '-f', iotests.imgfmt, ro_top_path, - str(self.image_len)) + with iotests.ImagePath('ro-top.img') as ro_top_path: + create_test_image(ro_top_path, self.image_len) result = self.vm.qmp('blockdev-add', node_name='ro-top',
@@ -198,8 +197,7 @@ class TestParallelOps(iotests.QMPTestCase): # Create all images iotests.create_image(self.imgs[0], self.image_len) for i in range(1, self.num_imgs): - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'backing_file=%s' % self.imgs[i-1], self.imgs[i]) + create_test_image(self.imgs[i], backing_file=self.imgs[i - 1]) # Put data into the images we are copying data from odd_img_indexes = [x for x in reversed(range(self.num_imgs)) if x % 2 == 1]
@@ -219,7 +217,7 @@ class TestParallelOps(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() for img in self.imgs: - os.remove(img) + remove_test_image(img) # Test that it's possible to run several block-stream operations # in parallel in the same snapshot chain
@@ -539,11 +537,10 @@ class TestQuorum(iotests.QMPTestCase): backing_img = os.path.join(iotests.test_dir, 'backing-%d.img' % i) self.children.append(child_img) self.backing.append(backing_img) - qemu_img('create', '-f', iotests.imgfmt, backing_img, '1M') + create_test_image(backing_img, '1M') qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x55 0 1024', backing_img) - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'backing_file=%s' % backing_img, child_img) + create_test_image(child_img, backing_file=backing_img) opts.append("children.%d.file.filename=%s" % (i, child_img)) opts.append("children.%d.node-name=node%d" % (i, i))
@@ -555,9 +552,9 @@ class TestQuorum(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() for img in self.children: - os.remove(img) + remove_test_image(img) for img in self.backing: - os.remove(img) + remove_test_image(img) def test_stream_quorum(self): if not iotests.supports_quorum():
@@ -587,7 +584,7 @@ class TestSmallerBackingFile(iotests.QMPTestCase): def setUp(self): iotests.create_image(backing_img, self.backing_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img, str(self.image_len)) + create_test_image(test_img, self.image_len, backing_file=backing_img) self.vm = iotests.VM().add_drive(test_img) self.vm.launch()
@@ -638,16 +635,15 @@ class TestEIO(TestErrors): self.blkdebug_file = backing_img + ".blkdebug" iotests.create_image(backing_img, TestErrors.image_len) self.create_blkdebug_file(self.blkdebug_file, "read_aio", 5) - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'backing_file=blkdebug:%s:%s,backing_fmt=raw' - % (self.blkdebug_file, backing_img), - test_img) + create_test_image(test_img, backing_fmt='raw', + backing_file=('blkdebug:%s:%s' % (self.blkdebug_file, + backing_img))) self.vm = iotests.VM().add_drive(test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) os.remove(backing_img) os.remove(self.blkdebug_file)
@@ -788,16 +784,15 @@ class TestENOSPC(TestErrors): self.blkdebug_file = backing_img + ".blkdebug" iotests.create_image(backing_img, TestErrors.image_len) self.create_blkdebug_file(self.blkdebug_file, "read_aio", 28) - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'backing_file=blkdebug:%s:%s,backing_fmt=raw' - % (self.blkdebug_file, backing_img), - test_img) + create_test_image(test_img, backing_fmt='raw', + backing_file=('blkdebug:%s:%s' % (self.blkdebug_file, + backing_img))) self.vm = iotests.VM().add_drive(test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) os.remove(backing_img) os.remove(self.blkdebug_file)
@@ -848,17 +843,17 @@ class TestStreamStop(iotests.QMPTestCase): image_len = 8 * 1024 * 1024 * 1024 # GB def setUp(self): - qemu_img('create', backing_img, str(TestStreamStop.image_len)) - qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 32M', backing_img) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(backing_img, TestStreamStop.image_len) + qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 32M', backing_img) + create_test_image(test_img, backing_file=backing_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 32M 32M', test_img) self.vm = iotests.VM().add_drive("blkdebug::" + test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(backing_img) + remove_test_image(test_img) + remove_test_image(backing_img) def test_stream_stop(self): self.assert_no_active_block_jobs()
@@ -879,17 +874,17 @@ class TestSetSpeed(iotests.QMPTestCase): image_len = 80 * 1024 * 1024 # MB def setUp(self): - qemu_img('create', backing_img, str(TestSetSpeed.image_len)) - qemu_io('-f', 'raw', '-c', 'write -P 0x1 0 32M', backing_img) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(backing_img, TestSetSpeed.image_len) + qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 32M', backing_img) + create_test_image(test_img, backing_file=backing_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 32M 32M', test_img) self.vm = iotests.VM().add_drive('blkdebug::' + test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(backing_img) + remove_test_image(test_img) + remove_test_image(backing_img) # This is a short performance test which is not run by default. # Invoke "IMGFMT=qed ./030 TestSetSpeed.perf_test_throughput"
Signature · 1 line
2.21.0
PATCH 16/67 iotests/040: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/040 | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-)
tests/qemu-iotests/040 +21 -21
@@ -24,7 +24,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_img, qemu_io import struct import errno
@@ -80,8 +80,8 @@ class TestSingleDrive(ImageCommitTestCase): def setUp(self): iotests.create_image(backing_img, self.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img) + create_test_image(mid_img, backing_file=backing_img) + create_test_image(test_img, backing_file=mid_img) qemu_io('-f', 'raw', '-c', 'write -P 0xab 0 524288', backing_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288', mid_img) self.vm = iotests.VM().add_drive(test_img, "node-name=top,backing.node-name=mid,backing.backing.node-name=base", interface="none")
@@ -92,8 +92,8 @@ class TestSingleDrive(ImageCommitTestCase): def tearDown(self): self.vm.shutdown(has_quit=self.has_quit) - os.remove(test_img) - os.remove(mid_img) + remove_test_image(test_img) + remove_test_image(mid_img) os.remove(backing_img) def test_commit(self):
@@ -303,8 +303,8 @@ class TestRelativePaths(ImageCommitTestCase): if exception.errno != errno.EEXIST: raise iotests.create_image(self.backing_img_abs, TestRelativePaths.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.backing_img_abs, self.mid_img_abs) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.mid_img_abs, self.test_img) + create_test_image(self.mid_img_abs, backing_file=self.backing_img_abs) + create_test_image(self.test_img, backing_file=self.mid_img_abs) qemu_img('rebase', '-u', '-b', self.backing_img, self.mid_img_abs) qemu_img('rebase', '-u', '-b', self.mid_img, self.test_img) qemu_io('-f', 'raw', '-c', 'write -P 0xab 0 524288', self.backing_img_abs)
@@ -314,8 +314,8 @@ class TestRelativePaths(ImageCommitTestCase): def tearDown(self): self.vm.shutdown() - os.remove(self.test_img) - os.remove(self.mid_img_abs) + remove_test_image(self.test_img) + remove_test_image(self.mid_img_abs) os.remove(self.backing_img_abs) try: os.rmdir(os.path.join(iotests.test_dir, self.dir1))
@@ -368,9 +368,9 @@ class TestSetSpeed(ImageCommitTestCase): image_len = 80 * 1024 * 1024 # MB def setUp(self): - qemu_img('create', backing_img, str(TestSetSpeed.image_len)) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, mid_img) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % mid_img, test_img) + create_test_image(backing_img, TestSetSpeed.image_len) + create_test_image(mid_img, backing_file=backing_img) + create_test_image(test_img, backing_file=mid_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0x1 0 512', test_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288', mid_img) self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
@@ -378,9 +378,9 @@ class TestSetSpeed(ImageCommitTestCase): def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(mid_img) - os.remove(backing_img) + remove_test_image(test_img) + remove_test_image(mid_img) + remove_test_image(backing_img) def test_set_speed(self): self.assert_no_active_block_jobs()
@@ -408,9 +408,9 @@ class TestReopenOverlay(ImageCommitTestCase): def setUp(self): iotests.create_image(self.img0, self.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.img0, self.img1) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.img1, self.img2) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % self.img2, self.img3) + create_test_image(self.img1, backing_file=self.img0) + create_test_image(self.img2, backing_file=self.img1) + create_test_image(self.img3, backing_file=self.img2) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xab 0 128K', self.img1) self.vm = iotests.VM().add_drive(self.img3) self.vm.launch()
@@ -418,9 +418,9 @@ class TestReopenOverlay(ImageCommitTestCase): def tearDown(self): self.vm.shutdown() os.remove(self.img0) - os.remove(self.img1) - os.remove(self.img2) - os.remove(self.img3) + remove_test_image(self.img1) + remove_test_image(self.img2) + remove_test_image(self.img3) # This tests what happens when the overlay image of the 'top' node # needs to be reopened in read-write mode in order to update the
Signature · 1 line
2.21.0
PATCH 17/67 iotests/041: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/041 | 108 +++++++++++++++++++---------------------- 1 file changed, 51 insertions(+), 57 deletions(-)
tests/qemu-iotests/041 +51 -57
@@ -21,7 +21,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_img, qemu_io backing_img = os.path.join(iotests.test_dir, 'backing.img') target_backing_img = os.path.join(iotests.test_dir, 'target-backing.img')
@@ -41,7 +41,7 @@ class TestSingleDrive(iotests.QMPTestCase): def setUp(self): iotests.create_image(backing_img, self.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(test_img, backing_file=backing_img) self.vm = iotests.VM().add_drive(test_img, "node-name=top,backing.node-name=base") if iotests.qemu_default_machine == 'pc': self.vm.add_drive(None, 'media=cdrom', 'ide')
@@ -49,12 +49,9 @@ class TestSingleDrive(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) os.remove(backing_img) - try: - os.remove(target_img) - except OSError: - pass + remove_test_image(target_img) def test_complete(self): self.assert_no_active_block_jobs()
@@ -138,8 +135,8 @@ class TestSingleDrive(iotests.QMPTestCase): def test_small_buffer2(self): self.assert_no_active_block_jobs() - qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=%d,size=%d' - % (self.image_len, self.image_len), target_img) + create_test_image(target_img, self.image_len, + opts=['cluster_size=%d' % self.image_len]) result = self.vm.qmp(self.qmp_cmd, device='drive0', sync='full', buf_size=65536, mode='existing', target=self.qmp_target) self.assert_qmp(result, 'return', {})
@@ -154,8 +151,8 @@ class TestSingleDrive(iotests.QMPTestCase): def test_large_cluster(self): self.assert_no_active_block_jobs() - qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=%d,backing_file=%s' - % (self.image_len, backing_img), target_img) + create_test_image(target_img, backing_file=backing_img, + opts=['cluster_size=%d' % self.image_len]) result = self.vm.qmp(self.qmp_cmd, device='drive0', sync='full', mode='existing', target=self.qmp_target) self.assert_qmp(result, 'return', {})
@@ -227,7 +224,7 @@ class TestSingleBlockdev(TestSingleDrive): def setUp(self): TestSingleDrive.setUp(self) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img) + create_test_image(target_img, backing_file=backing_img) args = {'driver': iotests.imgfmt, 'node-name': self.qmp_target, 'file': { 'filename': target_img, 'driver': 'file' } }
@@ -265,24 +262,21 @@ class TestMirrorNoBacking(iotests.QMPTestCase): def setUp(self): iotests.create_image(backing_img, TestMirrorNoBacking.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(test_img, backing_file=backing_img) self.vm = iotests.VM().add_drive(test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) os.remove(backing_img) - try: - os.remove(target_backing_img) - except: - pass - os.remove(target_img) + remove_test_image(target_backing_img) + remove_test_image(target_img) def test_complete(self): self.assert_no_active_block_jobs() - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img) + create_test_image(target_img, backing_file=backing_img) result = self.vm.qmp('drive-mirror', device='drive0', sync='full', mode='existing', target=target_img) self.assert_qmp(result, 'return', {})
@@ -297,7 +291,7 @@ class TestMirrorNoBacking(iotests.QMPTestCase): def test_cancel(self): self.assert_no_active_block_jobs() - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img) + create_test_image(target_img, backing_file=backing_img) result = self.vm.qmp('drive-mirror', device='drive0', sync='full', mode='existing', target=target_img) self.assert_qmp(result, 'return', {})
@@ -313,10 +307,9 @@ class TestMirrorNoBacking(iotests.QMPTestCase): self.assert_no_active_block_jobs() # qemu-img create fails if the image is not there - qemu_img('create', '-f', iotests.imgfmt, '-o', 'size=%d' - %(TestMirrorNoBacking.image_len), target_backing_img) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=%d,backing_file=%s' - % (TestMirrorNoBacking.image_len, target_backing_img), target_img) + create_test_image(target_backing_img, TestMirrorNoBacking.image_len) + create_test_image(target_img, backing_file=target_backing_img, + opts=['cluster_size=%d' % TestMirrorNoBacking.image_len]) result = self.vm.qmp('drive-mirror', device='drive0', sync='full', mode='existing', target=target_img)
@@ -335,14 +328,14 @@ class TestMirrorResized(iotests.QMPTestCase): def setUp(self): iotests.create_image(backing_img, TestMirrorResized.backing_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(test_img, backing_file=backing_img) qemu_img('resize', test_img, '2M') self.vm = iotests.VM().add_drive(test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) os.remove(backing_img) try: os.remove(target_img)
@@ -407,14 +400,15 @@ new_state = "1" ''' % (event, errno, self.MIRROR_GRANULARITY // 512, event, event)) file.close() + # blkdebug does not work with an external data file + @iotests.skip_for_imgopts(['data_file']) def setUp(self): self.blkdebug_file = backing_img + ".blkdebug" iotests.create_image(backing_img, TestReadErrors.image_len) self.create_blkdebug_file(self.blkdebug_file, "read_aio", 5) - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'backing_file=blkdebug:%s:%s,backing_fmt=raw' - % (self.blkdebug_file, backing_img), - test_img) + create_test_image(test_img, backing_fmt='raw', + backing_file=('blkdebug:%s:%s' % (self.blkdebug_file, + backing_img))) # Write something for tests that use sync='top' qemu_io('-c', 'write %d 512' % (self.MIRROR_GRANULARITY + 65536), test_img)
@@ -423,8 +417,8 @@ new_state = "1" def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(target_img) + remove_test_image(test_img) + remove_test_image(target_img) os.remove(backing_img) os.remove(self.blkdebug_file)
@@ -483,7 +477,8 @@ new_state = "1" # Test COW into the target image. The first half of the # cluster at MIRROR_GRANULARITY has to be copied from # backing_img, even though sync='top'. - qemu_img('create', '-f', iotests.imgfmt, '-ocluster_size=131072,backing_file=%s' %(backing_img), target_img) + create_test_image(target_img, backing_file=backing_img, + opts=['cluster_size=131072']) result = self.vm.qmp('drive-mirror', device='drive0', sync='top', on_source_error='ignore', mode='existing', target=target_img)
@@ -572,20 +567,22 @@ new_state = "1" ''' % (event, errno, self.MIRROR_GRANULARITY // 512, event, event)) file.close() + # blkdebug does not work with an external data file + @iotests.skip_for_imgopts(['data_file']) def setUp(self): self.blkdebug_file = target_img + ".blkdebug" iotests.create_image(backing_img, TestWriteErrors.image_len) self.create_blkdebug_file(self.blkdebug_file, "write_aio", 5) - qemu_img('create', '-f', iotests.imgfmt, '-obacking_file=%s' %(backing_img), test_img) + create_test_image(test_img, backing_file=backing_img) self.vm = iotests.VM().add_drive(test_img) self.target_img = 'blkdebug:%s:%s' % (self.blkdebug_file, target_img) - qemu_img('create', '-f', iotests.imgfmt, '-osize=%d' %(TestWriteErrors.image_len), target_img) + create_test_image(target_img, TestWriteErrors.image_len) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(target_img) + remove_test_image(test_img) + remove_test_image(target_img) os.remove(backing_img) os.remove(self.blkdebug_file)
@@ -673,15 +670,15 @@ class TestSetSpeed(iotests.QMPTestCase): image_len = 80 * 1024 * 1024 # MB def setUp(self): - qemu_img('create', backing_img, str(TestSetSpeed.image_len)) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(backing_img, TestSetSpeed.image_len) + create_test_image(test_img, backing_file=backing_img) self.vm = iotests.VM().add_drive(test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(backing_img) + remove_test_image(test_img) + remove_test_image(backing_img) os.remove(target_img) def test_set_speed(self):
@@ -739,8 +736,7 @@ class TestUnbackedSource(iotests.QMPTestCase): image_len = 2 * 1024 * 1024 # MB def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, test_img, - str(TestUnbackedSource.image_len)) + create_test_image(test_img, TestUnbackedSource.image_len) self.vm = iotests.VM() self.vm.launch() result = self.vm.qmp('blockdev-add', node_name='drive0',
@@ -753,8 +749,8 @@ class TestUnbackedSource(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(test_img) - os.remove(target_img) + remove_test_image(test_img) + remove_test_image(target_img) def test_absolute_paths_full(self): self.assert_no_active_block_jobs()
@@ -784,8 +780,7 @@ class TestUnbackedSource(iotests.QMPTestCase): self.assert_no_active_block_jobs() def test_existing_full(self): - qemu_img('create', '-f', iotests.imgfmt, target_img, - str(self.image_len)) + create_test_image(target_img, self.image_len) qemu_io('-c', 'write -P 42 0 64k', target_img) self.assert_no_active_block_jobs()
@@ -802,8 +797,7 @@ class TestUnbackedSource(iotests.QMPTestCase): 'target image does not match source after mirroring') def test_blockdev_full(self): - qemu_img('create', '-f', iotests.imgfmt, target_img, - str(self.image_len)) + create_test_image(target_img, self.image_len) qemu_io('-c', 'write -P 42 0 64k', target_img) result = self.vm.qmp('blockdev-add', node_name='target',
@@ -834,8 +828,7 @@ class TestGranularity(iotests.QMPTestCase): image_len = 10 * 1024 * 1024 # MB def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, test_img, - str(TestGranularity.image_len)) + create_test_image(test_img, TestGranularity.image_len) qemu_io('-c', 'write 0 %d' % (self.image_len), test_img) self.vm = iotests.VM().add_drive(test_img)
@@ -845,7 +838,7 @@ class TestGranularity(iotests.QMPTestCase): self.vm.shutdown() self.assertTrue(iotests.compare_images(test_img, target_img), 'target image does not match source after mirroring') - os.remove(test_img) + remove_test_image(test_img) os.remove(target_img) def test_granularity(self):
@@ -879,8 +872,7 @@ class TestRepairQuorum(iotests.QMPTestCase): # Add each individual quorum images for i in self.IMAGES: - qemu_img('create', '-f', iotests.imgfmt, i, - str(TestSingleDrive.image_len)) + create_test_image(i, TestSingleDrive.image_len) # Assign a node name to each quorum image in order to manipulate # them opts = "node-name=img%i" % self.IMAGES.index(i)
@@ -898,8 +890,10 @@ class TestRepairQuorum(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - for i in self.IMAGES + [ quorum_repair_img, quorum_snapshot_file ]: - # Do a try/except because the test may have deleted some images + for i in self.IMAGES: + remove_test_image(i) + + for i in [ quorum_repair_img, quorum_snapshot_file ]: try: os.remove(i) except OSError:
Signature · 1 line
2.21.0
PATCH 18/67 iotests/044: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/044 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/044 +3 -3
@@ -23,7 +23,7 @@ import os import qcow2 from qcow2 import QcowHeader import iotests -from iotests import qemu_img, qemu_img_verbose, qemu_io +from iotests import create_test_image, remove_test_image, qemu_img_verbose, qemu_io import struct import subprocess import sys
@@ -100,13 +100,13 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, '-o', 'cluster_size=512', test_img, '16G') + create_test_image(test_img, '16G', opts=['cluster_size=512']) self.preallocate(test_img) pass def tearDown(self): - os.remove(test_img) + remove_test_image(test_img) pass def test_grow_refcount_table(self):
Signature · 1 line
2.21.0
PATCH 19/67 iotests/045: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/045 | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
tests/qemu-iotests/045 +13 -13
@@ -20,7 +20,7 @@ import os import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image image0 = os.path.join(iotests.test_dir, 'image0') image1 = os.path.join(iotests.test_dir, 'image1')
@@ -32,11 +32,11 @@ class TestFdSets(iotests.QMPTestCase): def setUp(self): self.vm = iotests.VM() - qemu_img('create', '-f', iotests.imgfmt, image0, '128K') - qemu_img('create', '-f', iotests.imgfmt, image1, '128K') - qemu_img('create', '-f', iotests.imgfmt, image2, '128K') - qemu_img('create', '-f', iotests.imgfmt, image3, '128K') - qemu_img('create', '-f', iotests.imgfmt, image4, '128K') + create_test_image(image0, '128K') + create_test_image(image1, '128K') + create_test_image(image2, '128K') + create_test_image(image3, '128K') + create_test_image(image4, '128K') self.file0 = open(image0, 'r') self.file1 = open(image1, 'w+') self.file2 = open(image2, 'r')
@@ -57,11 +57,11 @@ class TestFdSets(iotests.QMPTestCase): self.file2.close() self.file3.close() self.file4.close() - os.remove(image0) - os.remove(image1) - os.remove(image2) - os.remove(image3) - os.remove(image4) + remove_test_image(image0) + remove_test_image(image1) + remove_test_image(image2) + remove_test_image(image3) + remove_test_image(image4) def test_query_fdset(self): result = self.vm.qmp('query-fdsets')
@@ -129,7 +129,7 @@ class TestFdSets(iotests.QMPTestCase): class TestSCMFd(iotests.QMPTestCase): def setUp(self): self.vm = iotests.VM() - qemu_img('create', '-f', iotests.imgfmt, image0, '128K') + create_test_image(image0, '128K') # Add an unused monitor, to verify it works fine when two monitor # instances present self.vm.add_monitor_null()
@@ -137,7 +137,7 @@ class TestSCMFd(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(image0) + remove_test_image(image0) def _send_fd_by_SCM(self): ret = self.vm.send_fd_scm(file_path=image0)
Signature · 1 line
2.21.0
PATCH 20/67 iotests/055: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/055 | 41 +++++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 18 deletions(-)
tests/qemu-iotests/055 +23 -18
@@ -23,7 +23,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_io test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img')
@@ -32,7 +32,7 @@ blockdev_target_img = os.path.join(iotests.test_dir, 'blockdev-target.img') image_len = 64 * 1024 * 1024 # MB def setUpModule(): - qemu_img('create', '-f', iotests.imgfmt, test_img, str(image_len)) + create_test_image(test_img, image_len) qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x11 0 64k', test_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x00 64k 128k', test_img) qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x22 162k 32k', test_img)
@@ -41,12 +41,12 @@ def setUpModule(): qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x33 67043328 64k', test_img) def tearDownModule(): - os.remove(test_img) + remove_test_image(test_img) class TestSingleDrive(iotests.QMPTestCase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len)) + create_test_image(blockdev_target_img, image_len) self.vm = iotests.VM().add_drive('blkdebug::' + test_img) self.vm.add_drive(blockdev_target_img, interface="none")
@@ -56,7 +56,7 @@ class TestSingleDrive(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(blockdev_target_img) + remove_test_image(blockdev_target_img) try: os.remove(target_img) except OSError:
@@ -163,7 +163,7 @@ class TestSingleDrive(iotests.QMPTestCase): class TestSetSpeed(iotests.QMPTestCase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len)) + create_test_image(blockdev_target_img, image_len) self.vm = iotests.VM().add_drive('blkdebug::' + test_img) self.vm.add_drive(blockdev_target_img, interface="none")
@@ -171,7 +171,7 @@ class TestSetSpeed(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(blockdev_target_img) + remove_test_image(blockdev_target_img) try: os.remove(target_img) except OSError:
@@ -249,7 +249,7 @@ class TestSetSpeed(iotests.QMPTestCase): # would stall. Instead, we limit the block job speed here. class TestSingleTransaction(iotests.QMPTestCase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len)) + create_test_image(blockdev_target_img, image_len) self.vm = iotests.VM().add_drive(test_img) self.vm.add_drive(blockdev_target_img, interface="none")
@@ -259,7 +259,7 @@ class TestSingleTransaction(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(blockdev_target_img) + remove_test_image(blockdev_target_img) try: os.remove(target_img) except OSError:
@@ -452,29 +452,34 @@ class TestSingleTransaction(iotests.QMPTestCase): class TestDriveCompression(iotests.QMPTestCase): image_len = 64 * 1024 * 1024 # MB - fmt_supports_compression = [{'type': 'qcow2', 'args': ()}, - {'type': 'vmdk', 'args': ('-o', 'subformat=streamOptimized')}] + fmt_supports_compression = [{'type': 'qcow2', 'opts': []}, + {'type': 'vmdk', 'opts': ['subformat=streamOptimized']}] + + # Compression does not work with external data files + @iotests.skip_for_imgopts(['data_file']) + def setUp(self): + pass def tearDown(self): self.vm.shutdown() - os.remove(blockdev_target_img) + remove_test_image(blockdev_target_img) try: os.remove(target_img) except OSError: pass - def do_prepare_drives(self, fmt, args, attach_target): + def do_prepare_drives(self, fmt, opts, attach_target): self.vm = iotests.VM().add_drive('blkdebug::' + test_img) - qemu_img('create', '-f', fmt, blockdev_target_img, - str(TestDriveCompression.image_len), *args) + create_test_image(blockdev_target_img, TestDriveCompression.image_len, + fmt=fmt, opts=opts) if attach_target: self.vm.add_drive(blockdev_target_img, format=fmt, interface="none") self.vm.launch() def do_test_compress_complete(self, cmd, format, attach_target, **args): - self.do_prepare_drives(format['type'], format['args'], attach_target) + self.do_prepare_drives(format['type'], format['opts'], attach_target) self.assert_no_active_block_jobs()
@@ -499,7 +504,7 @@ class TestDriveCompression(iotests.QMPTestCase): target='drive1') def do_test_compress_cancel(self, cmd, format, attach_target, **args): - self.do_prepare_drives(format['type'], format['args'], attach_target) + self.do_prepare_drives(format['type'], format['opts'], attach_target) self.assert_no_active_block_jobs()
@@ -523,7 +528,7 @@ class TestDriveCompression(iotests.QMPTestCase): target='drive1') def do_test_compress_pause(self, cmd, format, attach_target, **args): - self.do_prepare_drives(format['type'], format['args'], attach_target) + self.do_prepare_drives(format['type'], format['opts'], attach_target) self.assert_no_active_block_jobs()
Signature · 1 line
2.21.0
PATCH 21/67 iotests/056: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/056 | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-)
tests/qemu-iotests/056 +14 -16
@@ -23,7 +23,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io, create_image +from iotests import create_test_image, remove_test_image, qemu_io, create_image backing_img = os.path.join(iotests.test_dir, 'backing.img') test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -31,18 +31,12 @@ target_img = os.path.join(iotests.test_dir, 'target.img') def img_create(img, fmt=iotests.imgfmt, size='64M', **kwargs): fullname = os.path.join(iotests.test_dir, '%s.%s' % (img, fmt)) - optargs = [] - for k,v in kwargs.items(): - optargs = optargs + ['-o', '%s=%s' % (k,v)] - args = ['create', '-f', fmt] + optargs + [fullname, size] - iotests.qemu_img(*args) + opts = [] + for k, v in kwargs.items(): + opts.append('%s=%s' % (k, v)) + create_test_image(fullname, size, fmt=fmt, opts=opts) return fullname -def try_remove(img): - try: - os.remove(img) - except OSError: - pass def io_write_patterns(img, patterns): for pattern in patterns:
@@ -54,7 +48,7 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase): def setUp(self): create_image(backing_img, TestSyncModesNoneAndTop.image_len) - qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img) + create_test_image(test_img, backing_file=backing_img) qemu_io('-c', 'write -P0x41 0 512', test_img) qemu_io('-c', 'write -P0xd5 1M 32k', test_img) qemu_io('-c', 'write -P0xdc 32M 124k', test_img)
@@ -64,7 +58,7 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) os.remove(backing_img) try: os.remove(target_img)
@@ -103,6 +97,8 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase): self.assertEqual(-1, qemu_io('-c', 'read -P0x41 0 512', target_img).find("verification failed")) class TestBeforeWriteNotifier(iotests.QMPTestCase): + # blkdebug does not work with an external data file + @iotests.skip_for_imgopts(['data_file']) def setUp(self): self.vm = iotests.VM().add_drive_raw("file=blkdebug::null-co://,id=drive0,align=65536,driver=blkdebug") self.vm.launch()
@@ -139,9 +135,9 @@ class BackupTest(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - try_remove(self.test_img) - try_remove(self.dest_img) - try_remove(self.ref_img) + remove_test_image(self.test_img) + remove_test_image(self.dest_img) + remove_test_image(self.ref_img) def hmp_io_writes(self, drive, patterns): for pattern in patterns:
@@ -268,6 +264,8 @@ class BackupTest(iotests.QMPTestCase): sync='full', target=self.dest_img, auto_dismiss=False) + # blkdebug does not work with an external data file + @iotests.skip_for_imgopts(['data_file']) def dismissal_failure(self, dismissal_opt): res = self.vm.qmp('query-block-jobs') self.assert_qmp(res, 'return', [])
Signature · 1 line
2.21.0
PATCH 22/67 iotests/057: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/057 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/057 +3 -3
@@ -23,7 +23,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_io test_drv_base_name = 'drive'
@@ -40,7 +40,7 @@ class ImageSnapshotTestCase(iotests.QMPTestCase): filename = '%s%d' % (test_img_base_name, i) img = os.path.join(iotests.test_dir, filename) device = '%s%d' % (test_drv_base_name, i) - qemu_img('create', '-f', iotests.imgfmt, img, str(self.image_len)) + create_test_image(img, self.image_len) self.vm.add_drive(img) self.expect.append({'image': img, 'device': device, 'snapshots': [],
@@ -50,7 +50,7 @@ class ImageSnapshotTestCase(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() for dev_expect in self.expect: - os.remove(dev_expect['image']) + remove_test_image(dev_expect['image']) def createSnapshotInTransaction(self, snapshot_num, abort = False): actions = []
Signature · 1 line
2.21.0
PATCH 23/67 iotests/065: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/065 | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-)
tests/qemu-iotests/065 +8 -9
@@ -23,7 +23,7 @@ import os import re import json import iotests -from iotests import qemu_img, qemu_img_pipe +from iotests import create_test_image, remove_test_image, qemu_img_pipe import unittest test_img = os.path.join(iotests.test_dir, 'test.img')
@@ -34,11 +34,10 @@ class TestImageInfoSpecific(iotests.QMPTestCase): def setUp(self): if self.img_options is None: self.skipTest('Skipping abstract test class') - qemu_img('create', '-f', iotests.imgfmt, '-o', self.img_options, - test_img, '128K') + create_test_image(test_img, '128K', opts=self.img_options) def tearDown(self): - os.remove(test_img) + remove_test_image(test_img) class TestQemuImgInfo(TestImageInfoSpecific): '''Abstract base class for qemu-img info tests'''
@@ -87,13 +86,13 @@ class TestQMP(TestImageInfoSpecific): class TestQCow2(TestQemuImgInfo): '''Testing a qcow2 version 2 image''' - img_options = 'compat=0.10' + img_options = ['compat=0.10'] json_compare = { 'compat': '0.10', 'refcount-bits': 16 } human_compare = [ 'compat: 0.10', 'refcount bits: 16' ] class TestQCow3NotLazy(TestQemuImgInfo): '''Testing a qcow2 version 3 image with lazy refcounts disabled''' - img_options = 'compat=1.1,lazy_refcounts=off' + img_options = ['compat=1.1', 'lazy_refcounts=off'] json_compare = { 'compat': '1.1', 'lazy-refcounts': False, 'refcount-bits': 16, 'corrupt': False } human_compare = [ 'compat: 1.1', 'lazy refcounts: false',
@@ -101,7 +100,7 @@ class TestQCow3NotLazy(TestQemuImgInfo): class TestQCow3Lazy(TestQemuImgInfo): '''Testing a qcow2 version 3 image with lazy refcounts enabled''' - img_options = 'compat=1.1,lazy_refcounts=on' + img_options = ['compat=1.1', 'lazy_refcounts=on'] json_compare = { 'compat': '1.1', 'lazy-refcounts': True, 'refcount-bits': 16, 'corrupt': False } human_compare = [ 'compat: 1.1', 'lazy refcounts: true',
@@ -110,7 +109,7 @@ class TestQCow3Lazy(TestQemuImgInfo): class TestQCow3NotLazyQMP(TestQMP): '''Testing a qcow2 version 3 image with lazy refcounts disabled, opening with lazy refcounts enabled''' - img_options = 'compat=1.1,lazy_refcounts=off' + img_options = ['compat=1.1', 'lazy_refcounts=off'] qemu_options = 'lazy-refcounts=on' compare = { 'compat': '1.1', 'lazy-refcounts': False, 'refcount-bits': 16, 'corrupt': False }
@@ -119,7 +118,7 @@ class TestQCow3NotLazyQMP(TestQMP): class TestQCow3LazyQMP(TestQMP): '''Testing a qcow2 version 3 image with lazy refcounts enabled, opening with lazy refcounts disabled''' - img_options = 'compat=1.1,lazy_refcounts=on' + img_options = ['compat=1.1', 'lazy_refcounts=on'] qemu_options = 'lazy-refcounts=off' compare = { 'compat': '1.1', 'lazy-refcounts': True, 'refcount-bits': 16, 'corrupt': False }
Signature · 1 line
2.21.0
PATCH 24/67 iotests/096: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/096 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/096 +3 -2
@@ -20,6 +20,7 @@ # import iotests +from iotests import create_test_image, remove_test_image import os class TestLiveSnapshot(iotests.QMPTestCase):
@@ -35,13 +36,13 @@ class TestLiveSnapshot(iotests.QMPTestCase): opts.append('throttling.group=%s' % self.group) opts.append('throttling.iops-total=%d' % self.iops) opts.append('throttling.iops-size=%d' % self.iops_size) - iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, '100M') + create_test_image(self.base_img, '100M') self.vm = iotests.VM().add_drive(self.base_img, ','.join(opts)) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(self.base_img) + remove_test_image(self.base_img) os.remove(self.target_img) def checkConfig(self, active_layer):
Signature · 1 line
2.21.0
PATCH 25/67 iotests/118: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/118 | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
tests/qemu-iotests/118 +13 -13
@@ -23,7 +23,7 @@ import os import stat import time import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image old_img = os.path.join(iotests.test_dir, 'test0.img') new_img = os.path.join(iotests.test_dir, 'test1.img')
@@ -301,8 +301,8 @@ class TestInitiallyFilled(GeneralChangeTestsBaseClass): was_empty = False def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, old_img, '1440k') - qemu_img('create', '-f', iotests.imgfmt, new_img, '1440k') + create_test_image(old_img, '1440k') + create_test_image(new_img, '1440k') self.vm = iotests.VM() if self.use_drive: self.vm.add_drive(old_img, 'media=%s' % self.media, 'none')
@@ -320,8 +320,8 @@ class TestInitiallyFilled(GeneralChangeTestsBaseClass): def tearDown(self): self.vm.shutdown() - os.remove(old_img) - os.remove(new_img) + remove_test_image(old_img) + remove_test_image(new_img) def test_insert_on_filled(self): result = self.vm.qmp('blockdev-add',
@@ -344,7 +344,7 @@ class TestInitiallyEmpty(GeneralChangeTestsBaseClass): was_empty = True def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, new_img, '1440k') + create_test_image(new_img, '1440k') self.vm = iotests.VM() if self.use_drive: self.vm.add_drive(None, 'media=%s' % self.media, 'none')
@@ -358,7 +358,7 @@ class TestInitiallyEmpty(GeneralChangeTestsBaseClass): def tearDown(self): self.vm.shutdown() - os.remove(new_img) + remove_test_image(new_img) def test_remove_on_empty(self): result = self.vm.qmp('blockdev-open-tray', id=self.device_name)
@@ -394,16 +394,16 @@ class TestChangeReadOnly(ChangeBaseClass): device_name = 'qdev0' def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, old_img, '1440k') - qemu_img('create', '-f', iotests.imgfmt, new_img, '1440k') + create_test_image(old_img, '1440k') + create_test_image(new_img, '1440k') self.vm = iotests.VM() def tearDown(self): self.vm.shutdown() os.chmod(old_img, 0o666) os.chmod(new_img, 0o666) - os.remove(old_img) - os.remove(new_img) + remove_test_image(old_img) + remove_test_image(new_img) def test_ro_ro_retain(self): os.chmod(old_img, 0o444)
@@ -640,7 +640,7 @@ class TestBlockJobsAfterCycle(ChangeBaseClass): device_name = 'qdev0' def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, old_img, '1440K') + create_test_image(old_img, '1440k') self.vm = iotests.VM() self.vm.add_drive_raw("id=drive0,driver=null-co,if=none")
@@ -674,7 +674,7 @@ class TestBlockJobsAfterCycle(ChangeBaseClass): def tearDown(self): self.vm.shutdown() - os.remove(old_img) + remove_test_image(old_img) try: os.remove(new_img) except OSError:
Signature · 1 line
2.21.0
PATCH 26/67 iotests/124: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/124 | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-)
tests/qemu-iotests/124 +13 -16
@@ -22,6 +22,7 @@ import os import iotests +from iotests import create_test_image, remove_test_image def io_write_patterns(img, patterns):
@@ -29,13 +30,6 @@ def io_write_patterns(img, patterns): iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img) -def try_remove(img): - try: - os.remove(img) - except OSError: - pass - - def transaction_action(action, **kwargs): return { 'type': action,
@@ -82,13 +76,13 @@ class Bitmap: def del_target(self): for image in self.backups.pop(): - try_remove(image) + remove_test_image(image) self.num -= 1 def cleanup(self): for backup in self.backups: for image in backup: - try_remove(image) + remove_test_image(image) class TestIncrementalBackupBase(iotests.QMPTestCase):
@@ -133,15 +127,14 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): def img_create(self, img, fmt=iotests.imgfmt, size='64M', parent=None, parentFormat=None, **kwargs): - optargs = [] - for k,v in kwargs.items(): - optargs = optargs + ['-o', '%s=%s' % (k,v)] - args = ['create', '-f', fmt] + optargs + [img, size] + opts = [] + for k, v in kwargs.items(): + opts.append('%s=%s' % (k, v)) if parent: if parentFormat is None: parentFormat = fmt - args = args + ['-b', parent, '-F', parentFormat] - iotests.qemu_img(*args) + create_test_image(img, size, fmt=fmt, backing_file=parent, + backing_fmt=parentFormat, opts=opts) self.files.append(img)
@@ -287,7 +280,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): for bitmap in self.bitmaps: bitmap.cleanup() for filename in self.files: - try_remove(filename) + remove_test_image(filename)
@@ -414,6 +407,8 @@ class TestIncrementalBackup(TestIncrementalBackupBase): self.check_backups() + # blkdebug does not work with an external data file + @iotests.skip_for_imgopts(['data_file']) def do_transaction_failure_test(self, race=False): # Create a second drive, with pattern: drive1 = self.add_node('drive1')
@@ -608,6 +603,8 @@ class TestIncrementalBackup(TestIncrementalBackupBase): class TestIncrementalBackupBlkdebug(TestIncrementalBackupBase): '''Incremental backup tests that utilize a BlkDebug filter on drive0.''' + # blkdebug does not work with an external data file + @iotests.skip_for_imgopts(['data_file']) def setUp(self): drive0 = self.add_node('drive0') self.img_create(drive0['file'], drive0['fmt'])
Signature · 1 line
2.21.0
PATCH 27/67 iotests/129: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/129 | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
tests/qemu-iotests/129 +9 -2
@@ -20,6 +20,7 @@ import os import iotests +from iotests import create_test_image, remove_test_image import time class TestStopWithBlockJob(iotests.QMPTestCase):
@@ -28,8 +29,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): base_img = os.path.join(iotests.test_dir, 'base.img') def setUp(self): - iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G") - iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img) + create_test_image(self.base_img, '1G') + create_test_image(self.test_img, backing_file=self.base_img) iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img) self.vm = iotests.VM().add_drive(self.test_img) self.vm.launch()
@@ -46,6 +47,12 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result = self.vm.qmp("block_set_io_throttle", conv_keys=False, **params) self.vm.shutdown() + remove_test_image(self.test_img) + remove_test_image(self.base_img) + try: + os.remove(self.target_img) + except OSError: + pass def do_test_stop(self, cmd, **args): """Test 'stop' while block job is running on a throttled drive.
Signature · 1 line
2.21.0
PATCH 28/67 iotests/132: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/132 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/132 +3 -3
@@ -21,7 +21,7 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_io test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img')
@@ -31,7 +31,7 @@ class TestSingleDrive(iotests.QMPTestCase): def setUp(self): # Write data to the image so we can compare later - qemu_img('create', '-f', iotests.imgfmt, test_img, str(TestSingleDrive.image_len)) + create_test_image(test_img, TestSingleDrive.image_len) qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 0 2M', test_img) self.vm = iotests.VM().add_drive(test_img, 'discard=unmap')
@@ -39,7 +39,7 @@ class TestSingleDrive(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) try: os.remove(target_img) except OSError:
Signature · 1 line
2.21.0
PATCH 29/67 iotests/139: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/139 | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
tests/qemu-iotests/139 +7 -8
@@ -21,6 +21,7 @@ import os import iotests +from iotests import create_test_image, remove_test_image import time base_img = os.path.join(iotests.test_dir, 'base.img')
@@ -33,7 +34,7 @@ else: class TestBlockdevDel(iotests.QMPTestCase): def setUp(self): - iotests.qemu_img('create', '-f', iotests.imgfmt, base_img, '1M') + create_test_image(base_img, '1M') self.vm = iotests.VM() self.vm.add_device("{},id=virtio-scsi".format( iotests.get_virtio_scsi_device()))
@@ -41,9 +42,8 @@ class TestBlockdevDel(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(base_img) - if os.path.isfile(new_img): - os.remove(new_img) + remove_test_image(base_img) + remove_test_image(new_img) # Check whether a BlockDriverState exists def checkBlockDriverState(self, node, must_exist = True):
@@ -70,8 +70,7 @@ class TestBlockdevDel(iotests.QMPTestCase): # Add a BlockDriverState that will be used as overlay for the base_img BDS def addBlockDriverStateOverlay(self, node): self.checkBlockDriverState(node, False) - iotests.qemu_img('create', '-u', '-f', iotests.imgfmt, - '-b', base_img, new_img, '1M') + create_test_image(new_img, '1M', backing_file=base_img, unsafe=True) opts = {'driver': iotests.imgfmt, 'node-name': node, 'backing': None,
@@ -202,7 +201,7 @@ class TestBlockdevDel(iotests.QMPTestCase): self.checkBlockDriverState(test, False) self.checkBlockDriverState(raw, False) self.checkBlockDriverState(blkverify, False) - iotests.qemu_img('create', '-f', iotests.imgfmt, new_img, '1M') + create_test_image(new_img, '1M') node_0 = {'driver': iotests.imgfmt, 'node-name': test, 'file': {'driver': 'file',
@@ -226,7 +225,7 @@ class TestBlockdevDel(iotests.QMPTestCase): self.checkBlockDriverState(child0, False) self.checkBlockDriverState(child1, False) self.checkBlockDriverState(quorum, False) - iotests.qemu_img('create', '-f', iotests.imgfmt, new_img, '1M') + create_test_image(new_img, '1M') child_0 = {'driver': iotests.imgfmt, 'node-name': child0, 'file': {'driver': 'file',
Signature · 1 line
2.21.0
PATCH 30/67 iotests/147: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/147 | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tests/qemu-iotests/147 +6 -5
@@ -24,7 +24,8 @@ import socket import stat import time import iotests -from iotests import cachemode, imgfmt, qemu_img, qemu_nbd, qemu_nbd_early_pipe +from iotests import cachemode, imgfmt, create_test_image, remove_test_image, \ + qemu_nbd, qemu_nbd_early_pipe NBD_PORT_START = 32768 NBD_PORT_END = NBD_PORT_START + 1024
@@ -80,13 +81,13 @@ class NBDBlockdevAddBase(iotests.QMPTestCase): class QemuNBD(NBDBlockdevAddBase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, test_img, '64k') + create_test_image(test_img, '64k') self.vm = iotests.VM() self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) try: os.remove(unix_socket) except OSError:
@@ -127,7 +128,7 @@ class QemuNBD(NBDBlockdevAddBase): class BuiltinNBD(NBDBlockdevAddBase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, test_img, '64k') + create_test_image(test_img, '64k') self.vm = iotests.VM() self.vm.launch() self.server = iotests.VM('.server')
@@ -140,7 +141,7 @@ class BuiltinNBD(NBDBlockdevAddBase): def tearDown(self): self.vm.shutdown() self.server.shutdown() - os.remove(test_img) + remove_test_image(test_img) try: os.remove(unix_socket) except OSError:
Signature · 1 line
2.21.0
PATCH 31/67 iotests/148: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/148 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/148 +3 -2
@@ -21,6 +21,7 @@ import os import iotests +from iotests import create_test_image, remove_test_image imgs = (os.path.join(iotests.test_dir, 'quorum0.img'), os.path.join(iotests.test_dir, 'quorum1.img'),
@@ -51,7 +52,7 @@ sector = "%d" driveopts = ['driver=quorum', 'vote-threshold=2'] driveopts.append('read-pattern=%s' % self.read_pattern) for i in range(len(imgs)): - iotests.qemu_img('create', '-f', iotests.imgfmt, imgs[i], '1M') + create_test_image(imgs[i], '1M') self.create_blkdebug_file(img_conf[i], i + offset) driveopts.append('children.%d.driver=%s' % (i, iotests.imgfmt)) driveopts.append('children.%d.file.driver=blkdebug' % i)
@@ -65,7 +66,7 @@ sector = "%d" def tearDown(self): self.vm.shutdown() for i in range(len(imgs)): - os.remove(imgs[i]) + remove_test_image(imgs[i]) os.remove(img_conf[i]) def do_check_event(self, node, sector = 0):
Signature · 1 line
2.21.0
PATCH 32/67 iotests/151: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/151 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/151 +5 -5
@@ -20,7 +20,7 @@ import os import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image source_img = os.path.join(iotests.test_dir, 'source.' + iotests.imgfmt) target_img = os.path.join(iotests.test_dir, 'target.' + iotests.imgfmt)
@@ -30,8 +30,8 @@ class TestActiveMirror(iotests.QMPTestCase): potential_writes_in_flight = True def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, source_img, '128M') - qemu_img('create', '-f', iotests.imgfmt, target_img, '128M') + create_test_image(source_img, '128M') + create_test_image(target_img, '128M') blk_source = {'id': 'source', 'if': 'none',
@@ -58,8 +58,8 @@ class TestActiveMirror(iotests.QMPTestCase): self.assertTrue(iotests.compare_images(source_img, target_img), 'mirror target does not match source') - os.remove(source_img) - os.remove(target_img) + remove_test_image(source_img) + remove_test_image(target_img) def doActiveIO(self, sync_source_and_target): # Fill the source image
Signature · 1 line
2.21.0
PATCH 33/67 iotests/152: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/152 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/152 +3 -3
@@ -20,20 +20,20 @@ import os import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img') class TestUnaligned(iotests.QMPTestCase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, test_img, '512') + create_test_image(test_img, 512) self.vm = iotests.VM().add_drive(test_img) self.vm.launch() def tearDown(self): self.vm.shutdown() - os.remove(test_img) + remove_test_image(test_img) try: os.remove(target_img) except OSError:
Signature · 1 line
2.21.0
PATCH 34/67 iotests/155: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/155 | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-)
tests/qemu-iotests/155 +13 -16
@@ -23,7 +23,7 @@ import os import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image back0_img = os.path.join(iotests.test_dir, 'back0.' + iotests.imgfmt) back1_img = os.path.join(iotests.test_dir, 'back1.' + iotests.imgfmt)
@@ -51,10 +51,10 @@ class BaseClass(iotests.QMPTestCase): target_real_backing = None def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, back0_img, '1440K') - qemu_img('create', '-f', iotests.imgfmt, '-b', back0_img, back1_img) - qemu_img('create', '-f', iotests.imgfmt, '-b', back1_img, back2_img) - qemu_img('create', '-f', iotests.imgfmt, '-b', back2_img, source_img) + create_test_image(back0_img, '1440K') + create_test_image(back1_img, backing_file=back0_img) + create_test_image(back2_img, backing_file=back1_img) + create_test_image(source_img, backing_file=back2_img) self.vm = iotests.VM() # Add the BDS via blockdev-add so it stays around after the mirror block
@@ -71,10 +71,10 @@ class BaseClass(iotests.QMPTestCase): if self.existing: if self.target_backing: - qemu_img('create', '-f', iotests.imgfmt, - '-b', self.target_backing, target_img, '1440K') + create_test_image(target_img, '1440K', \ + backing_file=self.target_backing) else: - qemu_img('create', '-f', iotests.imgfmt, target_img, '1440K') + create_test_image(target_img, '1440K') if self.cmd == 'blockdev-mirror': options = { 'node-name': 'target',
@@ -89,14 +89,11 @@ class BaseClass(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - os.remove(source_img) - os.remove(back2_img) - os.remove(back1_img) - os.remove(back0_img) - try: - os.remove(target_img) - except OSError: - pass + remove_test_image(source_img) + remove_test_image(back2_img) + remove_test_image(back1_img) + remove_test_image(back0_img) + remove_test_image(target_img) def findBlockNode(self, node_name, qdev=None): if qdev:
Signature · 1 line
2.21.0
PATCH 35/67 iotests/163: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/163 | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
tests/qemu-iotests/163 +11 -13
@@ -19,7 +19,8 @@ # import os, random, iotests, struct, qcow2, sys -from iotests import qemu_img, qemu_io, image_size +from iotests import create_test_image, remove_test_image, qemu_img, qemu_io, \ + image_size test_img = os.path.join(iotests.test_dir, 'test.img') check_img = os.path.join(iotests.test_dir, 'check.img')
@@ -82,22 +83,19 @@ class ShrinkBaseClass(iotests.QMPTestCase): def setUp(self): if iotests.imgfmt == 'raw': - qemu_img('create', '-f', iotests.imgfmt, test_img, self.image_len) - qemu_img('create', '-f', iotests.imgfmt, check_img, - self.shrink_size) + create_test_image(test_img, self.image_len) + create_test_image(check_img, self.shrink_size) else: - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'cluster_size=' + self.cluster_size + - ',refcount_bits=' + self.refcount_bits, - test_img, self.image_len) - qemu_img('create', '-f', iotests.imgfmt, - '-o', 'cluster_size=%s'% self.cluster_size, - check_img, self.shrink_size) + create_test_image(test_img, self.image_len, + opts=['cluster_size=' + self.cluster_size, + 'refcount_bits=' + self.refcount_bits]) + create_test_image(check_img, self.shrink_size, + opts=['cluster_size=' + self.cluster_size]) qemu_io('-c', 'write -P 0xff 0 ' + self.shrink_size, check_img) def tearDown(self): - os.remove(test_img) - os.remove(check_img) + remove_test_image(test_img) + remove_test_image(check_img) def image_verify(self): self.assertEqual(image_size(test_img), image_size(check_img),
Signature · 1 line
2.21.0
PATCH 36/67 iotests/165: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/165 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/165 +3 -3
@@ -22,7 +22,7 @@ from __future__ import print_function import os import re import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image disk = os.path.join(iotests.test_dir, 'disk') disk_size = 0x40000000 # 1G
@@ -37,10 +37,10 @@ regions2 = ((0x10000000, 0x20000), class TestPersistentDirtyBitmap(iotests.QMPTestCase): def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, disk, str(disk_size)) + create_test_image(disk, disk_size) def tearDown(self): - os.remove(disk) + remove_test_image(disk) def mkVm(self): return iotests.VM().add_drive(disk)
Signature · 1 line
2.21.0
PATCH 37/67 iotests/169: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/169 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/169 +5 -5
@@ -24,7 +24,7 @@ import time import itertools import operator import re -from iotests import qemu_img +from iotests import create_test_image, remove_test_image disk_a = os.path.join(iotests.test_dir, 'disk_a')
@@ -39,16 +39,16 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): def tearDown(self): self.vm_a.shutdown() self.vm_b.shutdown() - os.remove(disk_a) - os.remove(disk_b) + remove_test_image(disk_a) + remove_test_image(disk_b) try: os.remove(mig_file) except OSError: pass def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, disk_a, size) - qemu_img('create', '-f', iotests.imgfmt, disk_b, size) + create_test_image(disk_a, size) + create_test_image(disk_b, size) self.vm_a = iotests.VM(path_suffix='a').add_drive(disk_a) self.vm_a.launch()
Signature · 1 line
2.21.0
PATCH 38/67 iotests/194: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/194 | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/194 +5 -4
@@ -20,19 +20,20 @@ # Non-shared storage migration test using NBD server and drive-mirror import iotests +from iotests import create_test_image iotests.script_initialize(supported_fmts=['qcow2', 'qed', 'raw']) -with iotests.FilePath('source.img') as source_img_path, \ - iotests.FilePath('dest.img') as dest_img_path, \ +with iotests.ImagePath('source.img') as source_img_path, \ + iotests.ImagePath('dest.img') as dest_img_path, \ iotests.FilePath('migration.sock') as migration_sock_path, \ iotests.FilePath('nbd.sock') as nbd_sock_path, \ iotests.VM('source') as source_vm, \ iotests.VM('dest') as dest_vm: img_size = '1G' - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, source_img_path, img_size) - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, dest_img_path, img_size) + create_test_image(source_img_path, img_size) + create_test_image(dest_img_path, img_size) iotests.log('Launching VMs...') (source_vm.add_drive(source_img_path)
Signature · 1 line
2.21.0
PATCH 39/67 iotests/196: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/196 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/196 +3 -3
@@ -21,7 +21,7 @@ import os import iotests -from iotests import qemu_img +from iotests import create_test_image, remove_test_image disk = os.path.join(iotests.test_dir, 'disk') migfile = os.path.join(iotests.test_dir, 'migfile')
@@ -31,11 +31,11 @@ class TestInvalidateAutoclear(iotests.QMPTestCase): def tearDown(self): self.vm_a.shutdown() self.vm_b.shutdown() - os.remove(disk) + remove_test_image(disk) os.remove(migfile) def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, disk, '1M') + create_test_image(disk, '1M') self.vm_a = iotests.VM(path_suffix='a').add_drive(disk) self.vm_a.launch()
Signature · 1 line
2.21.0
PATCH 40/67 iotests/199: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/199 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/199 +5 -5
@@ -21,7 +21,7 @@ import os import iotests import time -from iotests import qemu_img +from iotests import create_test_image, remove_test_image disk_a = os.path.join(iotests.test_dir, 'disk_a') disk_b = os.path.join(iotests.test_dir, 'disk_b')
@@ -33,14 +33,14 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): def tearDown(self): self.vm_a.shutdown() self.vm_b.shutdown() - os.remove(disk_a) - os.remove(disk_b) + remove_test_image(disk_a) + remove_test_image(disk_b) os.remove(fifo) def setUp(self): os.mkfifo(fifo) - qemu_img('create', '-f', iotests.imgfmt, disk_a, size) - qemu_img('create', '-f', iotests.imgfmt, disk_b, size) + create_test_image(disk_a, size) + create_test_image(disk_b, size) self.vm_a = iotests.VM(path_suffix='a').add_drive(disk_a) self.vm_b = iotests.VM(path_suffix='b').add_drive(disk_b) self.vm_b.add_incoming("exec: cat '" + fifo + "'")
Signature · 1 line
2.21.0
PATCH 41/67 iotests/202: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/202 | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/202 +5 -4
@@ -23,18 +23,19 @@ # against regressions. import iotests +from iotests import create_test_image iotests.script_initialize(supported_fmts=['qcow2']) -with iotests.FilePath('disk0.img') as disk0_img_path, \ - iotests.FilePath('disk1.img') as disk1_img_path, \ +with iotests.ImagePath('disk0.img') as disk0_img_path, \ + iotests.ImagePath('disk1.img') as disk1_img_path, \ iotests.FilePath('disk0-snap.img') as disk0_snap_img_path, \ iotests.FilePath('disk1-snap.img') as disk1_snap_img_path, \ iotests.VM() as vm: img_size = '10M' - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk0_img_path, img_size) - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk1_img_path, img_size) + create_test_image(disk0_img_path, img_size) + create_test_image(disk1_img_path, img_size) iotests.log('Launching VM...') vm.launch()
Signature · 1 line
2.21.0
PATCH 42/67 iotests/203: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/203 | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/203 +5 -4
@@ -23,16 +23,17 @@ # BDRV_POLL_WHILE(). import iotests +from iotests import create_test_image iotests.script_initialize(supported_fmts=['qcow2']) -with iotests.FilePath('disk0.img') as disk0_img_path, \ - iotests.FilePath('disk1.img') as disk1_img_path, \ +with iotests.ImagePath('disk0.img') as disk0_img_path, \ + iotests.ImagePath('disk1.img') as disk1_img_path, \ iotests.VM() as vm: img_size = '10M' - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk0_img_path, img_size) - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk1_img_path, img_size) + create_test_image(disk0_img_path, img_size) + create_test_image(disk1_img_path, img_size) iotests.log('Launching VM...') (vm.add_object('iothread,id=iothread0')
Signature · 1 line
2.21.0
PATCH 43/67 iotests/205: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/205 | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
tests/qemu-iotests/205 +4 -3
@@ -22,7 +22,8 @@ import os import sys import iotests import time -from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive +from iotests import create_test_image, remove_test_image, \ + qemu_io, filter_qemu_io, QemuIoInteractive nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock') nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
@@ -31,7 +32,7 @@ disk = os.path.join(iotests.test_dir, 'disk') class TestNbdServerRemove(iotests.QMPTestCase): def setUp(self): - qemu_img_create('-f', iotests.imgfmt, disk, '1M') + create_test_image(disk, '1M') self.vm = iotests.VM().add_drive(disk) self.vm.launch()
@@ -51,7 +52,7 @@ class TestNbdServerRemove(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() os.remove(nbd_sock) - os.remove(disk) + remove_test_image(disk) def remove_export(self, name, mode=None): if mode is None:
Signature · 1 line
2.21.0
PATCH 44/67 iotests/208: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/208 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/208 +3 -2
@@ -21,16 +21,17 @@ # blockdev-snapshot-sync. import iotests +from iotests import create_test_image iotests.script_initialize(supported_fmts=['generic']) -with iotests.FilePath('disk.img') as disk_img_path, \ +with iotests.ImagePath('disk.img') as disk_img_path, \ iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \ iotests.FilePath('nbd.sock') as nbd_sock_path, \ iotests.VM() as vm: img_size = '10M' - iotests.qemu_img_create('-f', iotests.imgfmt, disk_img_path, img_size) + create_test_image(disk_img_path, img_size) iotests.log('Launching VM...') (vm.add_drive(disk_img_path, 'node-name=drive0-node', interface='none')
Signature · 1 line
2.21.0
PATCH 45/67 iotests/208: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/209 | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/209 +5 -4
@@ -19,15 +19,16 @@ # import iotests -from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \ - file_path +from iotests import create_test_image, qemu_io, qemu_img_verbose, qemu_nbd, \ + file_path, image_path iotests.script_initialize(supported_fmts=['qcow2']) -disk, nbd_sock = file_path('disk', 'nbd-sock') +nbd_sock = file_path('nbd-sock') +disk = image_path('disk') nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock -qemu_img_create('-f', iotests.imgfmt, disk, '1M') +create_test_image(disk, '1M') qemu_io('-f', iotests.imgfmt, '-c', 'write 0 512K', disk) qemu_nbd('-k', nbd_sock, '-x', 'exp', '-f', iotests.imgfmt, disk)
Signature · 1 line
2.21.0
PATCH 46/67 iotests/216: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/216 | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
tests/qemu-iotests/216 +5 -6
@@ -20,7 +20,7 @@ # Creator/Owner: Max Reitz <[email protected]> import iotests -from iotests import log, qemu_img, qemu_io_silent +from iotests import log, create_test_image, qemu_io_silent # Need backing file support iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk'])
@@ -42,17 +42,16 @@ log('') # A COR filter node, however, can request the proper permissions for # its child and therefore is not hit by this issue. -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('top.img') as top_img_path, \ +with iotests.ImagePath('base.img') as base_img_path, \ + iotests.ImagePath('top.img') as top_img_path, \ iotests.VM() as vm: log('--- Setting up images ---') log('') - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 + assert create_test_image(base_img_path, '64M') == 0 assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - top_img_path) == 0 + assert create_test_image(top_img_path, backing_file=base_img_path) == 0 assert qemu_io_silent(top_img_path, '-c', 'write -P 2 1M 1M') == 0 log('Done')
Signature · 1 line
2.21.0
PATCH 47/67 iotests/218: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/218 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/218 +3 -3
@@ -27,7 +27,7 @@ # Creator/Owner: Max Reitz <[email protected]> import iotests -from iotests import log, qemu_img, qemu_io_silent +from iotests import log, create_test_image, qemu_io_silent iotests.script_initialize(supported_fmts=['qcow2', 'raw'])
@@ -142,9 +142,9 @@ log('=== Cancel mirror job from throttled node by quitting ===') log('') with iotests.VM() as vm, \ - iotests.FilePath('src.img') as src_img_path: + iotests.ImagePath('src.img') as src_img_path: - assert qemu_img('create', '-f', iotests.imgfmt, src_img_path, '64M') == 0 + assert create_test_image(src_img_path, '64M') == 0 assert qemu_io_silent('-f', iotests.imgfmt, src_img_path, '-c', 'write -P 42 0M 64M') == 0
Signature · 1 line
2.21.0
PATCH 48/67 iotests/219: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/219 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/219 +3 -2
@@ -20,6 +20,7 @@ # Check using the job-* QMP commands with block jobs import iotests +from iotests import create_test_image iotests.script_initialize(supported_fmts=['qcow2'])
@@ -177,11 +178,11 @@ def test_job_lifecycle(vm, job, job_args, has_ready=False): iotests.log(vm.qmp('query-jobs')) -with iotests.FilePath('disk.img') as disk_path, \ +with iotests.ImagePath('disk.img') as disk_path, \ iotests.FilePath('copy.img') as copy_path, \ iotests.VM() as vm: - iotests.qemu_img_create('-f', iotests.imgfmt, disk_path, str(img_size)) + create_test_image(disk_path, img_size) iotests.qemu_io('-c', 'write 0 %i' % (img_size), '-f', iotests.imgfmt, disk_path)
Signature · 1 line
2.21.0
PATCH 49/67 iotests/222: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/222 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/222 +5 -5
@@ -22,7 +22,7 @@ # Creator/Owner: John Snow <[email protected]> import iotests -from iotests import log, qemu_img, qemu_io, qemu_io_silent +from iotests import log, create_test_image, qemu_io, qemu_io_silent iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', 'vhdx', 'raw'],
@@ -46,16 +46,16 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] ("0xdc", "32M", "32k"), # Left-end of partial-right [2] ("0xcd", "0x3ff0000", "64k")] # patterns[3] -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('fleece.img') as fleece_img_path, \ +with iotests.ImagePath('base.img') as base_img_path, \ + iotests.ImagePath('fleece.img') as fleece_img_path, \ iotests.FilePath('nbd.sock') as nbd_sock_path, \ iotests.VM() as vm: log('--- Setting up images ---') log('') - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 - assert qemu_img('create', '-f', "qcow2", fleece_img_path, '64M') == 0 + assert create_test_image(base_img_path, '64M') == 0 + assert create_test_image(fleece_img_path, '64M', fmt='qcow2') == 0 for p in patterns: qemu_io('-f', iotests.imgfmt,
Signature · 1 line
2.21.0
PATCH 50/67 iotests/224: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/224 | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
tests/qemu-iotests/224 +9 -11
@@ -21,8 +21,9 @@ # Creator/Owner: Max Reitz <[email protected]> import iotests -from iotests import log, qemu_img, qemu_io_silent, filter_qmp_testfiles, \ - filter_qmp_imgfmt, filter_json_filename +from iotests import log, create_test_image, qemu_io_silent, \ + filter_qmp_testfiles, filter_qmp_imgfmt, \ + filter_json_filename import json import re
@@ -47,17 +48,14 @@ for filter_node_name in False, True: log('--- filter_node_name: %s ---' % filter_node_name) log('') - with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('mid.img') as mid_img_path, \ - iotests.FilePath('top.img') as top_img_path, \ + with iotests.ImagePath('base.img') as base_img_path, \ + iotests.ImagePath('mid.img') as mid_img_path, \ + iotests.ImagePath('top.img') as top_img_path, \ iotests.VM() as vm: - assert qemu_img('create', '-f', iotests.imgfmt, - base_img_path, '64M') == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - mid_img_path) == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path, - top_img_path) == 0 + assert create_test_image(base_img_path, '64M') == 0 + assert create_test_image(mid_img_path, backing_file=base_img_path) == 0 + assert create_test_image(top_img_path, backing_file=mid_img_path) == 0 # Something to commit assert qemu_io_silent(mid_img_path, '-c', 'write -P 1 0 1M') == 0
Signature · 1 line
2.21.0
PATCH 51/67 iotests/228: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/228 | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
tests/qemu-iotests/228 +10 -9
@@ -21,8 +21,9 @@ # Creator/Owner: Max Reitz <[email protected]> import iotests -from iotests import log, qemu_img, filter_testfiles, filter_imgfmt, \ - filter_qmp_testfiles, filter_qmp_imgfmt, filter_json_filename +from iotests import log, create_test_image, remove_test_image, \ + filter_testfiles, filter_imgfmt, filter_qmp_testfiles, \ + filter_qmp_imgfmt, filter_json_filename # Need backing file and change-backing-file support iotests.script_initialize(supported_fmts=['qcow2', 'qed'])
@@ -46,14 +47,14 @@ def log_node_info(node): log('') -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('top.img') as top_img_path, \ +with iotests.ImagePath('base.img') as base_img_path, \ + iotests.ImagePath('top.img') as top_img_path, \ iotests.VM() as vm: - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 + assert create_test_image(base_img_path, '64M') == 0 # Choose a funny way to describe the backing filename - assert qemu_img('create', '-f', iotests.imgfmt, '-b', - 'file:' + base_img_path, top_img_path) == 0 + assert create_test_image(top_img_path, + backing_file=('file:' + base_img_path)) == 0 vm.launch()
@@ -167,8 +168,8 @@ with iotests.FilePath('base.img') as base_img_path, \ # (because qemu cannot "canonicalize"/"resolve" the backing # filename unless the backing file is opened implicitly with the # overlay) - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - top_img_path) == 0 + remove_test_image(top_img_path) + assert create_test_image(top_img_path, backing_file=base_img_path) == 0 # You can only reliably override backing options by using a node # reference (or by specifying file.filename, but, well...)
Signature · 1 line
2.21.0
PATCH 52/67 iotests/234: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/234 | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/234 +5 -4
@@ -21,19 +21,20 @@ # that is built with individually created nodes import iotests +from iotests import create_test_image import os iotests.script_initialize(supported_fmts=['qcow2']) -with iotests.FilePath('img') as img_path, \ - iotests.FilePath('backing') as backing_path, \ +with iotests.ImagePath('img') as img_path, \ + iotests.ImagePath('backing') as backing_path, \ iotests.FilePath('mig_fifo_a') as fifo_a, \ iotests.FilePath('mig_fifo_b') as fifo_b, \ iotests.VM(path_suffix='a') as vm_a, \ iotests.VM(path_suffix='b') as vm_b: - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, backing_path, '64M') - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M') + create_test_image(backing_path, '64M') + create_test_image(img_path, '64M') os.mkfifo(fifo_a) os.mkfifo(fifo_b)
Signature · 1 line
2.21.0
PATCH 53/67 iotests/235: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/235 | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
tests/qemu-iotests/235 +3 -4
@@ -21,7 +21,7 @@ import sys import os import iotests -from iotests import qemu_img_create, qemu_io, file_path, log +from iotests import create_test_image, qemu_io, image_path, log sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
@@ -42,11 +42,10 @@ iotests.script_initialize(supported_fmts=['qcow2']) size = 1 * 1024 * 1024 * 1024 -disk = file_path('disk') +disk = image_path('disk') # prepare source image -qemu_img_create('-f', iotests.imgfmt, '-o', 'preallocation=metadata', disk, - str(size)) +create_test_image(disk, size, opts=['preallocation=metadata']) vm = QEMUMachine(iotests.qemu_prog) vm.add_args('-machine', 'accel=kvm:tcg')
Signature · 1 line
2.21.0
PATCH 54/67 iotests/236: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/236 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/236 +3 -3
@@ -20,7 +20,7 @@ # [email protected] import iotests -from iotests import log +from iotests import log, create_test_image iotests.script_initialize(supported_fmts=['generic']) size = 64 * 1024 * 1024
@@ -41,11 +41,11 @@ def query_bitmaps(vm): return { "bitmaps": { device['device']: device.get('dirty-bitmaps', []) for device in res['return'] } } -with iotests.FilePath('img') as img_path, \ +with iotests.ImagePath('img') as img_path, \ iotests.VM() as vm: log('--- Preparing image & VM ---\n') - iotests.qemu_img_create('-f', iotests.imgfmt, img_path, str(size)) + create_test_image(img_path, size) vm.add_drive(img_path) vm.launch()
Signature · 1 line
2.21.0
PATCH 55/67 iotests/237: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/237 | 11 +++++------ tests/qemu-iotests/237.out | 6 ------ 2 files changed, 5 insertions(+), 12 deletions(-)
tests/qemu-iotests/237 +5 -6
@@ -22,7 +22,7 @@ import math import iotests -from iotests import imgfmt +from iotests import imgfmt, create_test_image iotests.script_initialize(supported_fmts=['vmdk'], # All options are ignored
@@ -38,9 +38,9 @@ def blockdev_create(vm, options): iotests.log("") with iotests.FilePath('t.vmdk') as disk_path, \ - iotests.FilePath('t.vmdk.1') as extent1_path, \ - iotests.FilePath('t.vmdk.2') as extent2_path, \ - iotests.FilePath('t.vmdk.3') as extent3_path, \ + iotests.ImagePath('t.vmdk.1') as extent1_path, \ + iotests.ImagePath('t.vmdk.2') as extent2_path, \ + iotests.ImagePath('t.vmdk.3') as extent3_path, \ iotests.VM() as vm: #
@@ -175,8 +175,7 @@ with iotests.FilePath('t.vmdk') as disk_path, \ iotests.log("") for path in [ extent1_path, extent2_path, extent3_path ]: - msg = iotests.qemu_img_pipe('create', '-f', imgfmt, path, '0') - iotests.log(msg, [iotests.filter_testfiles]) + assert create_test_image(path, 0) == 0 vm.add_blockdev('driver=file,filename=%s,node-name=ext1' % (extent1_path)) vm.add_blockdev('driver=file,filename=%s,node-name=ext2' % (extent2_path))
tests/qemu-iotests/237.out +0 -6
@@ -129,12 +129,6 @@ Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exi === Other subformats === -Formatting 'TEST_DIR/PID-t.vmdk.1', fmt=vmdk size=0 compat6=off hwversion=undefined - -Formatting 'TEST_DIR/PID-t.vmdk.2', fmt=vmdk size=0 compat6=off hwversion=undefined - -Formatting 'TEST_DIR/PID-t.vmdk.3', fmt=vmdk size=0 compat6=off hwversion=undefined - == Missing extent == {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "vmdk", "file": "node0", "size": 33554432, "subformat": "monolithicFlat"}}}
Signature · 1 line
2.21.0
PATCH 56/67 iotests/242: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/242 | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tests/qemu-iotests/242 +6 -5
@@ -21,14 +21,14 @@ import iotests import json import struct -from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \ - file_path, img_info_log, log, filter_qemu_io +from iotests import create_test_image, remove_test_image, qemu_io, \ + qemu_img_pipe, image_path, img_info_log, log, filter_qemu_io iotests.script_initialize(supported_fmts=['qcow2'], # compat=1.1 is needed for dirty bitmaps unsupported_imgopts=['compat=0.10']) -disk = file_path('disk') +disk = image_path('disk') chunk = 256 * 1024 bitmap_flag_unknown = 1 << 2 # flag_offset = 5*cluster_size + flag_offset_in_bitmap_directory_entry
@@ -74,7 +74,7 @@ def toggle_flag(offset): f.write(struct.pack("B", toggled)) -qemu_img_create('-f', iotests.imgfmt, disk, '1M') +create_test_image(disk, '1M') for num in range(1, 4): disabled = False
@@ -95,7 +95,8 @@ vm.shutdown() num += 1 log('\nTest {}'.format(num)) -qemu_img_create('-f', iotests.imgfmt, disk, '1M') +remove_test_image(disk) +create_test_image(disk, '1M') add_bitmap(1, True, False) log('Write an unknown bitmap flag \'{}\' into a new QCOW2 image at offset {}' .format(hex(bitmap_flag_unknown), flag_offset))
Signature · 1 line
2.21.0
PATCH 57/67 iotests/245: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/245 | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
tests/qemu-iotests/245 +9 -8
@@ -24,7 +24,7 @@ import re import iotests import copy import json -from iotests import qemu_img, qemu_io +from iotests import create_test_image, remove_test_image, qemu_io hd_path = [ os.path.join(iotests.test_dir, 'hd0.img'),
@@ -43,9 +43,9 @@ class TestBlockdevReopen(iotests.QMPTestCase): total_io_cmds = 0 def setUp(self): - qemu_img('create', '-f', iotests.imgfmt, hd_path[0], '3M') - qemu_img('create', '-f', iotests.imgfmt, '-b', hd_path[0], hd_path[1]) - qemu_img('create', '-f', iotests.imgfmt, hd_path[2], '3M') + create_test_image(hd_path[0], '3M') + create_test_image(hd_path[1], backing_file=hd_path[0]) + create_test_image(hd_path[2], '3M') qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xa0 0 1M', hd_path[0]) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xa1 1M 1M', hd_path[1]) qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xa2 2M 1M', hd_path[2])
@@ -55,9 +55,9 @@ class TestBlockdevReopen(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() self.check_qemu_io_errors() - os.remove(hd_path[0]) - os.remove(hd_path[1]) - os.remove(hd_path[2]) + remove_test_image(hd_path[0]) + remove_test_image(hd_path[1]) + remove_test_image(hd_path[2]) # The output of qemu-io is not returned by vm.hmp_qemu_io() but # it's stored in the log and can only be read when the VM has been
@@ -328,7 +328,8 @@ class TestBlockdevReopen(iotests.QMPTestCase): # First we create a 2MB raw file, and fill each half with a # different value - qemu_img('create', '-f', 'raw', hd_path[0], '2M') + remove_test_image(hd_path[0]) + create_test_image(hd_path[0], '2M', fmt='raw') qemu_io('-f', 'raw', '-c', 'write -P 0xa0 0 1M', hd_path[0]) qemu_io('-f', 'raw', '-c', 'write -P 0xa1 1M 1M', hd_path[0])
Signature · 1 line
2.21.0
PATCH 58/67 iotests/246: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/246 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/246 +3 -3
@@ -20,7 +20,7 @@ # [email protected] import iotests -from iotests import log +from iotests import log, create_test_image iotests.script_initialize(supported_fmts=['qcow2'], # compat=1.1 is needed for dirty bitmaps
@@ -35,11 +35,11 @@ def query_bitmaps(vm): return { "bitmaps": { device['device']: device.get('dirty-bitmaps', []) for device in res['return'] } } -with iotests.FilePath('img') as img_path, \ +with iotests.ImagePath('img') as img_path, \ iotests.VM() as vm: log('--- Preparing image & VM ---\n') - iotests.qemu_img_create('-f', iotests.imgfmt, img_path, str(size)) + create_test_image(img_path, size) vm.add_drive(img_path)
Signature · 1 line
2.21.0
PATCH 59/67 iotests/248: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/248 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tests/qemu-iotests/248 +4 -4
@@ -19,7 +19,7 @@ # import iotests -from iotests import qemu_img_create, qemu_io, file_path, filter_qmp_testfiles +from iotests import create_test_image, qemu_io, image_path, filter_qmp_testfiles iotests.script_initialize(supported_fmts=['qcow2'], # With an external data file, we would need
@@ -27,12 +27,12 @@ iotests.script_initialize(supported_fmts=['qcow2'], # of @file unsupported_imgopts=['data_file']) -source, target = file_path('source', 'target') +source, target = image_path('source', 'target') size = 5 * 1024 * 1024 limit = 2 * 1024 * 1024 -qemu_img_create('-f', iotests.imgfmt, source, str(size)) -qemu_img_create('-f', iotests.imgfmt, target, str(size)) +create_test_image(source, size) +create_test_image(target, size) qemu_io('-c', 'write 0 {}'.format(size), source) # raw format don't like empty files
Signature · 1 line
2.21.0
PATCH 60/67 iotests/254: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/254 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/254 +3 -3
@@ -19,16 +19,16 @@ # import iotests -from iotests import qemu_img_create, file_path, log +from iotests import create_test_image, image_path, log iotests.script_initialize(supported_fmts=['qcow2'], # compat=1.1 is needed for dirty bitmaps unsupported_imgopts=['compat=0.10']) -disk, top = file_path('disk', 'top') +disk, top = image_path('disk', 'top') size = 1024 * 1024 -qemu_img_create('-f', iotests.imgfmt, disk, str(size)) +create_test_image(disk, size) vm = iotests.VM().add_drive(disk, opts='node-name=base') vm.launch()
Signature · 1 line
2.21.0
PATCH 61/67 iotests/255: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/255 | 20 +++++++++----------- tests/qemu-iotests/255.out | 8 -------- 2 files changed, 9 insertions(+), 19 deletions(-)
tests/qemu-iotests/255 +9 -11
@@ -21,7 +21,7 @@ # import iotests -from iotests import imgfmt +from iotests import imgfmt, create_test_image iotests.script_initialize(supported_fmts=['qcow2'])
@@ -39,8 +39,8 @@ iotests.log('Finishing a commit job with background reads') iotests.log('============================================') iotests.log('') -with iotests.FilePath('t.qcow2') as disk_path, \ - iotests.FilePath('t.qcow2.mid') as mid_path, \ +with iotests.ImagePath('t.qcow2') as disk_path, \ + iotests.ImagePath('t.qcow2.mid') as mid_path, \ iotests.FilePath('t.qcow2.base') as base_path, \ iotests.VM() as vm:
@@ -48,11 +48,10 @@ with iotests.FilePath('t.qcow2') as disk_path, \ iotests.log("") size = 128 * 1024 * 1024 - size_str = str(size) iotests.create_image(base_path, size) - iotests.qemu_img_log('create', '-f', iotests.imgfmt, mid_path, size_str) - iotests.qemu_img_log('create', '-f', iotests.imgfmt, disk_path, size_str) + assert create_test_image(mid_path, size) == 0 + assert create_test_image(disk_path, size) == 0 # Create a backing chain like this: # base <- [throttled: bps-read=4096] <- mid <- overlay
@@ -91,18 +90,17 @@ iotests.log('Closing the VM while a job is being cancelled') iotests.log('=============================================') iotests.log('') -with iotests.FilePath('src.qcow2') as src_path, \ - iotests.FilePath('dst.qcow2') as dst_path, \ +with iotests.ImagePath('src.qcow2') as src_path, \ + iotests.ImagePath('dst.qcow2') as dst_path, \ iotests.VM() as vm: iotests.log('=== Create images and start VM ===') iotests.log('') size = 128 * 1024 * 1024 - size_str = str(size) - iotests.qemu_img_log('create', '-f', iotests.imgfmt, src_path, size_str) - iotests.qemu_img_log('create', '-f', iotests.imgfmt, dst_path, size_str) + assert create_test_image(src_path, size) == 0 + assert create_test_image(dst_path, size) == 0 iotests.log(iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write 0 1M', src_path),
tests/qemu-iotests/255.out +0 -8
@@ -3,10 +3,6 @@ Finishing a commit job with background reads === Create backing chain and start VM === -Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 - -Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 - === Start background read requests === === Run a commit job ===
@@ -23,10 +19,6 @@ Closing the VM while a job is being cancelled === Create images and start VM === -Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 - -Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 - wrote 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Signature · 1 line
2.21.0
PATCH 62/67 iotests/256: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/256 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tests/qemu-iotests/256 +5 -5
@@ -21,13 +21,13 @@ import os import iotests -from iotests import log +from iotests import log, create_test_image iotests.script_initialize(supported_fmts=['qcow2']) size = 64 * 1024 * 1024 -with iotests.FilePath('img0') as img0_path, \ - iotests.FilePath('img1') as img1_path, \ +with iotests.ImagePath('img0') as img0_path, \ + iotests.ImagePath('img1') as img1_path, \ iotests.FilePath('img0-full') as img0_full_path, \ iotests.FilePath('img1-full') as img1_full_path, \ iotests.FilePath('img0-incr') as img0_incr_path, \
@@ -62,8 +62,8 @@ with iotests.FilePath('img0') as img0_path, \ vm.add_object('iothread,id=iothread1') vm.add_device('virtio-scsi-pci,id=scsi0,iothread=iothread0') vm.add_device('virtio-scsi-pci,id=scsi1,iothread=iothread1') - iotests.qemu_img_create('-f', iotests.imgfmt, img0_path, str(size)) - iotests.qemu_img_create('-f', iotests.imgfmt, img1_path, str(size)) + create_test_image(img0_path, size) + create_test_image(img1_path, size) vm.add_drive(img0_path, interface='none') vm.add_device('scsi-hd,id=device0,drive=drive0,bus=scsi0.0') vm.add_drive(img1_path, interface='none')
Signature · 1 line
2.21.0
PATCH 63/67 iotests/257: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/257 | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tests/qemu-iotests/257 +7 -7
@@ -23,7 +23,7 @@ import math import os import iotests -from iotests import log, qemu_img +from iotests import log, create_test_image, qemu_img SIZE = 64 * 1024 * 1024 GRANULARITY = 64 * 1024
@@ -152,7 +152,7 @@ class Drive: def img_create(self, fmt, size): self.fmt = fmt self.size = size - iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size)) + create_test_image(self.path, self.size) def create_target(self, name, fmt, size): basename = os.path.basename(self.path)
@@ -273,10 +273,10 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None): an incomplete backup. Testing limitations prevent testing competing writes. """ - with iotests.FilePaths(['img', 'bsync1', 'bsync2', + with iotests.ImagePath('img') as img_path, \ + iotests.FilePaths(['bsync1', 'bsync2', 'fbackup0', 'fbackup1', 'fbackup2']) as \ - (img_path, bsync1, bsync2, - fbackup0, fbackup1, fbackup2), \ + (bsync1, bsync2, fbackup0, fbackup1, fbackup2), \ iotests.VM() as vm: mode = "Mode {:s}; Bitmap Sync {:s}".format(msync_mode, bsync_mode)
@@ -439,8 +439,8 @@ def test_backup_api(): """ Test malformed and prohibited invocations of the backup API. """ - with iotests.FilePaths(['img', 'bsync1']) as \ - (img_path, backup_path), \ + with iotests.ImagePath('img') as img_path, \ + iotests.FilePath('bsync1') as backup_path, \ iotests.VM() as vm: log("\n=== API failure tests ===\n")
Signature · 1 line
2.21.0
PATCH 64/67 iotests/258: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/258 | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
tests/qemu-iotests/258 +8 -8
@@ -20,7 +20,7 @@ # Creator/Owner: Max Reitz <[email protected]> import iotests -from iotests import log, qemu_img, qemu_io_silent, \ +from iotests import log, create_test_image, qemu_io_silent, \ filter_qmp_testfiles, filter_qmp_imgfmt # Returns a node for blockdev-add
@@ -65,22 +65,22 @@ def test_concurrent_finish(write_to_stream_node): # and the other way around, depending on whether the commit job # is finalized before stream completes or not. - with iotests.FilePath('node4.img') as node4_path, \ - iotests.FilePath('node3.img') as node3_path, \ - iotests.FilePath('node2.img') as node2_path, \ - iotests.FilePath('node1.img') as node1_path, \ - iotests.FilePath('node0.img') as node0_path, \ + with iotests.ImagePath('node4.img') as node4_path, \ + iotests.ImagePath('node3.img') as node3_path, \ + iotests.ImagePath('node2.img') as node2_path, \ + iotests.ImagePath('node1.img') as node1_path, \ + iotests.ImagePath('node0.img') as node0_path, \ iotests.VM() as vm: # It is important to use raw for the base layer (so that # permissions are just handed through to the protocol layer) - assert qemu_img('create', '-f', 'raw', node0_path, '64M') == 0 + assert create_test_image(node0_path, '64M', fmt='raw') == 0 stream_throttle=None commit_throttle=None for path in [node1_path, node2_path, node3_path, node4_path]: - assert qemu_img('create', '-f', iotests.imgfmt, path, '64M') == 0 + assert create_test_image(path, '64M') == 0 if write_to_stream_node: # This is what (most of the time) makes commit finish
Signature · 1 line
2.21.0
PATCH 65/67 iotests/262: Honor $IMGOPTS
In reply to Max Reitz--- tests/qemu-iotests/262 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/262 +3 -2
@@ -21,11 +21,12 @@ # iothread just for fun. import iotests +from iotests import create_test_image import os iotests.script_initialize(supported_fmts=['qcow2']) -with iotests.FilePath('img') as img_path, \ +with iotests.ImagePath('img') as img_path, \ iotests.FilePath('mig_fifo') as fifo, \ iotests.VM(path_suffix='a') as vm_a, \ iotests.VM(path_suffix='b') as vm_b:
@@ -49,7 +50,7 @@ with iotests.FilePath('img') as img_path, \ vm.add_device('virtio-blk,drive=%s,iothread=iothread0' % root) - iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M') + create_test_image(img_path, '64M') os.mkfifo(fifo)
Signature · 1 line
2.21.0
PATCH 66/67 iotests.py: Forbid qemu_img*('create', ...)
In reply to Max ReitzAll tests should use create_test_image().
--- tests/qemu-iotests/iotests.py | 36 +++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-)
tests/qemu-iotests/iotests.py +28 -8
@@ -82,12 +82,20 @@ luks_default_secret_object = 'secret,id=keysec0,data=' + \ luks_default_key_secret_opt = 'key-secret=keysec0' -def qemu_img(*args): +# @from_create_test_image is purely for use in create_test_image +def qemu_img(*args, from_create_test_image=False): '''Run qemu-img and return the exit code''' + args = list(args) + + assert args[0] != 'create' or from_create_test_image, \ + 'Use iotests.create_test_image for image creation' + devnull = open('/dev/null', 'r+') - exitcode = subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull) + exitcode = subprocess.call(qemu_img_args + args, + stdin=devnull, stdout=devnull) if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' % + (-exitcode, ' '.join(qemu_img_args + args))) return exitcode def ordered_qmp(qmsg, conv_keys=True):
@@ -162,7 +170,7 @@ def create_test_image(filename, size=None, fmt=imgfmt, opts=[], if size is not None: args.append(str(size)) - return qemu_img(*args) + return qemu_img(*args, from_create_test_image=True) # Use this to remove images create with create_test_image in the # default image format (iotests.imgfmt)
@@ -180,20 +188,32 @@ def remove_test_image(filename): def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' - exitcode = subprocess.call(qemu_img_args + list(args)) + args = list(args) + + assert args[0] != 'create', \ + 'Use iotests.create_test_image for image creation' + + exitcode = subprocess.call(qemu_img_args + args) if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' % + (-exitcode, ' '.join(qemu_img_args + args))) return exitcode def qemu_img_pipe(*args): '''Run qemu-img and return its output''' - subp = subprocess.Popen(qemu_img_args + list(args), + args = list(args) + + assert args[0] != 'create', \ + 'Use iotests.create_test_image for image creation' + + subp = subprocess.Popen(qemu_img_args + args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True) exitcode = subp.wait() if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' % + (-exitcode, ' '.join(qemu_img_args + args))) return subp.communicate()[0] def qemu_img_log(*args):
Signature · 1 line
2.21.0
Quoted 2 lines · Max Reitz
First of all: Sorry.
Thank you for finding the time to do it.
Quoted 43 lines
Second:
Based-on: My block branch
(https://github.com/XanClic/qemu.git block)Based-on: [email protected]
(“iotests: use python logging”)Based-on: [email protected]
(“iotests: Allow ./check -o data_file”)Based-on: [email protected]
(“iotests: Selfish patches”)Based-on: [email protected]
(“block: Skip COR for inactive nodes”)OK, now:
Hi,
My recent series “iotests: Allow ./check -o data_file” enabled our bash tests to interpret the
data_fileqcow2 option. It didn’t do anything for Python tests because those currently completely ignore all image format options.This is where it gets hairy. To do so, we need two things: First of all, whatever way Python tests use to create images needs to interpret $IMGOPTS. Second, when deleting image files, they must not use a plain
os.remove(), but a special function that will clean up data files, too.The heap of patches in this series comes from making the Python tests use these new functions.
Most Python tests just run qemu-img through a helper function that does not care about the exact subcommand to create images. I could add $IMGOPTS support to it, but that doesn’t feel quite right to me, and it wouldn’t reduce the patch count because we still need a special removal function.
Quoted 4 lines
This series is structured as follows:
- Patches 1 through 7 add support to handle image files differently from other files (consider $IMGOPTS when creating them, consider data files when deleting them, separate ImagePaths from FilePaths, and so on)
OK, that makes sense. I suppose we've been playing a bit fast and loose with these such things.
Quoted 10 lines
- Patches 8 and 9 add two filters we’ll need in the next range:
- Patches 10 through 13 address some issues in a handful of tests that just need to be changed a little so they can overall work with some format options
- Patch 14 makes all tests pass
unsupported_imgoptswhere there are options that they cannot support.
Quoted 8 lines
- Patches 15 through 65 make all Python tests use the new functions introduced in the first 7 patches so they no longer ignore $IMGOPTS.
I felt like this is much better than munching everything together into a single big commit (better to rebase, better to review), and I don’t really like ideas like “Just do five patches that each address ten iotests”.
This is the right approach, for the exact reasons you specify.
Quoted 3 lines
But I’m still very much open to suggestions on how to combine these many small patches to reduce the overall patch count.
You could group them by release windows, if you really wanted to;
- [...etc...]
- Update iotests added for 3.2
- Update iotests added for 4.0
- Update iotests added for 4.1
- Individual patches thereafter
But maybe that doesn't really solve anything for anyone. If you didn't find a more obvious grouping for these, I'd just leave it alone. I'll get to reviewing them.
Quoted 13 lines
- Patch 66 ensures that Python tests always use the new function to create test images so they won’t bypass $IMGOPTS.
- Patch 67 cleans up.
qemu_img_log()is only used for image creation, and I don’t see the point in that. The output is predictable and it is very unlikely to fail. We can see in the bash tests that regularly we basically just filter everything from it anyway.
(So this series replaces log(qemu_img_pipe()) instances by asserting that image creation did not fail.)
((qemu_img_create()obviously no longer has any use after this series.))
Quoted 4 lines
After this series, running the iotests with -o compat=0.10, -o refcount_bits=1, and -o 'data_file=$TEST_IMG.data_file' does something sensible even for the Python tests, and it passes.
No minor accomplishment.
I'll make sure to review at least 1-14, but not before Friday.
Quoted 130 lines
Max Reitz (67):
iotests.py: Read $IMGOPTS
iotests.py: Add @skip_for_imgopts()iotests.py: Addunsupported_imgoptsiotests.py:create_test_image,remove_test_imageiotests.py: Add ImagePaths
iotests.py: Addimage_path()
iotests.py: Filterdata_fileinfilter_img_infoiotests.py: Addfilter_json_filename()iotests.py: Add @hide_fields toimg_info_logiotests/169: Skip persistent cases for compat=0.10 iotests/224: Filter json:{} from commit command iotests/228: Filter json:{} filenames iotests/242: Hide refcount bit information
iotests: Useunsupported_imgoptsin Python tests iotests/030: Honor $IMGOPTS
iotests/040: Honor $IMGOPTS
iotests/041: Honor $IMGOPTS
iotests/044: Honor $IMGOPTS
iotests/045: Honor $IMGOPTS
iotests/055: Honor $IMGOPTS
iotests/056: Honor $IMGOPTS
iotests/057: Honor $IMGOPTS
iotests/065: Honor $IMGOPTS
iotests/096: Honor $IMGOPTS
iotests/118: Honor $IMGOPTS
iotests/124: Honor $IMGOPTS
iotests/129: Honor $IMGOPTS
iotests/132: Honor $IMGOPTS
iotests/139: Honor $IMGOPTS
iotests/147: Honor $IMGOPTS
iotests/148: Honor $IMGOPTS
iotests/151: Honor $IMGOPTS
iotests/152: Honor $IMGOPTS
iotests/155: Honor $IMGOPTS
iotests/163: Honor $IMGOPTS
iotests/165: Honor $IMGOPTS
iotests/169: Honor $IMGOPTS
iotests/194: Honor $IMGOPTS
iotests/196: Honor $IMGOPTS
iotests/199: Honor $IMGOPTS
iotests/202: Honor $IMGOPTS
iotests/203: Honor $IMGOPTS
iotests/205: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/208: Honor $IMGOPTS
iotests/216: Honor $IMGOPTS
iotests/218: Honor $IMGOPTS
iotests/219: Honor $IMGOPTS
iotests/222: Honor $IMGOPTS
iotests/224: Honor $IMGOPTS
iotests/228: Honor $IMGOPTS
iotests/234: Honor $IMGOPTS
iotests/235: Honor $IMGOPTS
iotests/236: Honor $IMGOPTS
iotests/237: Honor $IMGOPTS
iotests/242: Honor $IMGOPTS
iotests/245: Honor $IMGOPTS
iotests/246: Honor $IMGOPTS
iotests/248: Honor $IMGOPTS
iotests/254: Honor $IMGOPTS
iotests/255: Honor $IMGOPTS
iotests/256: Honor $IMGOPTS
iotests/257: Honor $IMGOPTS
iotests/258: Honor $IMGOPTS
iotests/262: Honor $IMGOPTS
iotests.py: Forbid qemu_img*('create', ...)iotests.py: Dropqemu_img_log(),qemu_img_create()tests/qemu-iotests/030 | 69 ++++++------ tests/qemu-iotests/040 | 42 ++++---- tests/qemu-iotests/041 | 108 +++++++++---------- tests/qemu-iotests/044 | 11 +- tests/qemu-iotests/045 | 26 ++--- tests/qemu-iotests/055 | 41 +++---- tests/qemu-iotests/056 | 30 +++--- tests/qemu-iotests/057 | 10 +- tests/qemu-iotests/065 | 21 ++-- tests/qemu-iotests/096 | 5 +- tests/qemu-iotests/118 | 26 ++--- tests/qemu-iotests/124 | 29 +++-- tests/qemu-iotests/129 | 11 +- tests/qemu-iotests/132 | 6 +- tests/qemu-iotests/139 | 15 ++- tests/qemu-iotests/147 | 11 +- tests/qemu-iotests/148 | 5 +- tests/qemu-iotests/151 | 10 +- tests/qemu-iotests/152 | 6 +- tests/qemu-iotests/155 | 29 +++-- tests/qemu-iotests/163 | 29 ++--- tests/qemu-iotests/165 | 10 +- tests/qemu-iotests/169 | 23 ++-- tests/qemu-iotests/194 | 9 +- tests/qemu-iotests/196 | 10 +- tests/qemu-iotests/199 | 10 +- tests/qemu-iotests/202 | 9 +- tests/qemu-iotests/203 | 9 +- tests/qemu-iotests/205 | 7 +- tests/qemu-iotests/206 | 5 +- tests/qemu-iotests/208 | 5 +- tests/qemu-iotests/209 | 9 +- tests/qemu-iotests/216 | 11 +- tests/qemu-iotests/218 | 6 +- tests/qemu-iotests/219 | 5 +- tests/qemu-iotests/222 | 13 +-- tests/qemu-iotests/224 | 33 +++--- tests/qemu-iotests/224.out | 4 +- tests/qemu-iotests/228 | 25 ++--- tests/qemu-iotests/228.out | 8 +- tests/qemu-iotests/234 | 9 +- tests/qemu-iotests/235 | 7 +- tests/qemu-iotests/236 | 6 +- tests/qemu-iotests/237 | 15 +-- tests/qemu-iotests/237.out | 6 -- tests/qemu-iotests/242 | 21 ++-- tests/qemu-iotests/242.out | 5 - tests/qemu-iotests/245 | 21 ++-- tests/qemu-iotests/246 | 11 +- tests/qemu-iotests/248 | 14 ++- tests/qemu-iotests/254 | 10 +- tests/qemu-iotests/255 | 20 ++-- tests/qemu-iotests/255.out | 8 -- tests/qemu-iotests/256 | 10 +- tests/qemu-iotests/257 | 18 ++-- tests/qemu-iotests/258 | 16 +-- tests/qemu-iotests/262 | 5 +- tests/qemu-iotests/iotests.py | 197 +++++++++++++++++++++++++++------- 58 files changed, 654 insertions(+), 496 deletions(-)
PATCH 67/67 iotests.py: Drop qemu_img_log(), qemu_img_create()
In reply to Max ReitzThey are both unused now.
--- tests/qemu-iotests/iotests.py | 24 ------------------------ 1 file changed, 24 deletions(-)
tests/qemu-iotests/iotests.py +0 -24
@@ -111,25 +111,6 @@ def ordered_qmp(qmsg, conv_keys=True): return od return qmsg -def qemu_img_create(*args): - args = list(args) - - # default luks support - if '-f' in args and args[args.index('-f') + 1] == 'luks': - if '-o' in args: - i = args.index('-o') - if 'key-secret' not in args[i + 1]: - args[i + 1].append(luks_default_key_secret_opt) - args.insert(i + 2, '--object') - args.insert(i + 3, luks_default_secret_object) - else: - args = ['-o', luks_default_key_secret_opt, - '--object', luks_default_secret_object] + args - - args.insert(0, 'create') - - return qemu_img(*args) - def create_test_image(filename, size=None, fmt=imgfmt, opts=[], backing_file=None, backing_fmt=None, objects=[], unsafe=False):
@@ -216,11 +197,6 @@ def qemu_img_pipe(*args): (-exitcode, ' '.join(qemu_img_args + args))) return subp.communicate()[0] -def qemu_img_log(*args): - result = qemu_img_pipe(*args) - log(result, filters=[filter_testfiles]) - return result - def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[], hide_fields=[]): args = [ 'info' ]
Signature · 1 line
2.21.0