iotests: add enhanced debugging info to qemu-img failures
cover iotests: add enhanced debugging info to qemu-img failures
This is kinda-sorta V3-ish of a series I started in response to Thomas Huth's encountering a failure in qemu-img because of missing zstd support. This series changes the qemu_img() function in iotests.py to one that raises an Exception on non-zero return code by default.
Alongside this, the Exception object itself is also augmented so that it prints the stdout/stderr logs to screen if the exception goes unhandled so that failure cases are very obvious and easy to spot in the middle of python tracebacks.
(Test this out yourself: Disable zstd support and then run qcow2 iotest 065 before and after this patchset. It makes a real difference!)
NOTE: I have another 13-ish patches that go the rest of the way and ensure that *every* call to qemu-img goes through this new qemu_img() function, but for the sake of doing the most good in the shortest amount of time, I am sending just the first 5 patches, and the rest will be sent later. I think this is a very good series to get in before freeze so that we have it during the heavy testing season.
John Snow (5):
python/utils: add add_visual_margin() text decoration utility python/utils: add VerboseProcessError
iotests: Remove explicit checks for qemu_img() == 0
iotests: make qemu_img raise on non-zero rc by default
iotests: fortify compare_images() against crashes
python/qemu/utils/__init__.py | 114 ++++++++++++++++++ tests/qemu-iotests/163 | 9 +- tests/qemu-iotests/216 | 6 +- tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/224 | 11 +- tests/qemu-iotests/228 | 12 +- tests/qemu-iotests/257 | 11 +- tests/qemu-iotests/258 | 4 +- tests/qemu-iotests/310 | 14 +-- tests/qemu-iotests/iotests.py | 69 +++++++++-- tests/qemu-iotests/tests/block-status-cache | 3 +- tests/qemu-iotests/tests/image-fleecing | 4 +- .../tests/mirror-ready-cancel-error | 6 +- tests/qemu-iotests/tests/mirror-top-perms | 3 +- .../tests/remove-bitmap-from-backing | 8 +- .../qemu-iotests/tests/stream-error-on-reset | 4 +- 16 files changed, 220 insertions(+), 60 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/5 python/utils: add add_visual_margin() text decoration utility
Quoted 1 line
Quoted 1 line
Quoted 1 line
print(add_visual_margin(msg, width=72, name="Commit Message"))
┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ┃ add_visual_margin() takes a chunk of text and wraps it in a visual ┃ container that force-wraps to a specified width. An optional title ┃ label may be given, and any of the individual glyphs used to draw the ┃ box may be replaced or specified as well. ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
--- python/qemu/utils/__init__.py | 78 +++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+)
python/qemu/utils/__init__.py +78 -0
@@ -15,7 +15,10 @@ # the COPYING file in the top-level directory. # +import os import re +import shutil +import textwrap from typing import Optional # pylint: disable=import-error
@@ -23,6 +26,7 @@ __all__ = ( + 'add_visual_margin', 'get_info_usernet_hostfwd_port', 'kvm_available', 'list_accel',
@@ -43,3 +47,77 @@ def get_info_usernet_hostfwd_port(info_usernet_output: str) -> Optional[int]: if match is not None: return int(match[1]) return None + + +# pylint: disable=too-many-arguments +def add_visual_margin( + content: str = '', + width: Optional[int] = None, + name: Optional[str] = None, + padding: int = 1, + upper_left: str = '┏', + lower_left: str = '┗', + horizontal: str = '━', + vertical: str = '┃', +) -> str: + """ + Decorate and wrap some text with a visual decoration around it. + + This function assumes that the text decoration characters are single + characters that display using a single monospace column. + + ┏━ Example ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + ┃ This is what this function looks like with text content that's + ┃ wrapped to 72 characters. The right-hand margin is left open to + ┃ acommodate the occasional unicode character that might make + ┃ predicting the total "visual" width of a line difficult. This + ┃ provides a visual distinction that's good-enough, though. + ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + + :param content: The text to wrap and decorate. + :param width: + The number of columns to use, including for the decoration + itself. The default (None) uses the the available width of the + current terminal, or a fallback of 72 lines. A negative number + subtracts a fixed-width from the default size. The default obeys + the COLUMNS environment variable, if set. + :param name: A label to apply to the upper-left of the box. + :param padding: How many columns of padding to apply inside. + :param upper_left: Upper-left single-width text decoration character. + :param lower_left: Lower-left single-width text decoration character. + :param horizontal: Horizontal single-width text decoration character. + :param vertical: Vertical single-width text decoration character. + """ + if width is None or width < 0: + avail = shutil.get_terminal_size(fallback=(72, 24))[0] + if width is None: + _width = avail + else: + _width = avail + width + else: + _width = width + + prefix = vertical + (' ' * padding) + + def _bar(name: Optional[str], top: bool = True) -> str: + ret = upper_left if top else lower_left + if name is not None: + ret += f"{horizontal} {name} " + + filler_len = _width - len(ret) + ret += f"{horizontal * filler_len}" + return ret + + def _wrap(line: str) -> str: + return os.linesep.join( + textwrap.wrap( + line, width=_width - padding, initial_indent=prefix, + subsequent_indent=prefix, replace_whitespace=False, + drop_whitespace=True, break_on_hyphens=False) + ) + + return os.linesep.join(( + _bar(name, top=True), + os.linesep.join(_wrap(line) for line in content.splitlines()), + _bar(None, top=False), + ))
Signature · 1 line
2.34.1
Re: [PATCH 1/5] python/utils: add add_visual_margin() text decoration utility
Quoted 7 lines · John Snow
Quoted 1 line
Quoted 1 line
Quoted 1 line
print(add_visual_margin(msg, width=72, name="Commit Message"))┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ┃ add_visual_margin() takes a chunk of text and wraps it in a visual ┃ container that force-wraps to a specified width. An optional title ┃ label may be given, and any of the individual glyphs used to draw the ┃ box may be replaced or specified as well. ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
I see you dropped the right margin compared to earlier versions, but agree that this is still a nice visual indicator, and probably easier to maintain in this form. And it got rid of the weird spacing on the left when the wrap point hit at the wrong time.
python/qemu/utils/__init__.py def get_info_usernet_hostfwd_port(info_usernet_output: str) -> Optional[int]: · line 75
Quoted 12 lines
python/qemu/utils/__init__.py +12
-0+ Decorate and wrap some text with a visual decoration around it. + + This function assumes that the text decoration characters are single + characters that display using a single monospace column. + + ┏━ Example ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + ┃ This is what this function looks like with text content that's + ┃ wrapped to 72 characters. The right-hand margin is left open to + ┃ acommodate the occasional unicode character that might make + ┃ predicting the total "visual" width of a line difficult. This + ┃ provides a visual distinction that's good-enough, though. + ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Yep - hand-waving away Unicode messiness is certainly easiest ;)
[take with a grain of salt - my python is weak. But as you said in the cover letter, it's fairly straightforward to reproduce an environment where you can see it in action for hands-on testing]
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [PATCH 1/5] python/utils: add add_visual_margin() text decoration utility
Quoted 26 lines · Eric Blake
Quoted 7 lines · John Snow
Quoted 1 line
Quoted 1 line
Quoted 1 line
print(add_visual_margin(msg, width=72, name="Commit Message"))┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ┃ add_visual_margin() takes a chunk of text and wraps it in a visual ┃ container that force-wraps to a specified width. An optional title ┃ label may be given, and any of the individual glyphs used to draw the ┃ box may be replaced or specified as well. ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━I see you dropped the right margin compared to earlier versions, but agree that this is still a nice visual indicator, and probably easier to maintain in this form. And it got rid of the weird spacing on the left when the wrap point hit at the wrong time.
python/qemu/utils/__init__.py def get_info_usernet_hostfwd_port(info_usernet_output: str) -> Optional[int]:
Quoted 6 lines
python/qemu/utils/__init__.py +6
-0+ Decorate and wrap some text with a visual decoration around it. + + This function assumes that the text decoration characters are single + characters that display using a single monospace column. + + ┏━ Example━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > + ┃ This is what this function looks like with text content that's > + ┃ wrapped to 72 characters. The right-hand margin is left open to > + ┃ acommodate the occasional unicode character that might make > + ┃ predicting the total "visual" width of a line difficult. This > + ┃ provides a visual distinction that's good-enough, though.
Quoted 5 lines
Quoted 1 line
+
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Yep - hand-waving away Unicode messiness is certainly easiest ;)
I started reading the unicode specification for determining the number of "visual grapheme clusters" in a string, got a headache, and gave up.
(See also the PyPI package "grapheme", which was last updated for unicode 13.0 and appears unmaintained. It's no small amount of code!)
I could write a function that gives a *pretty good estimate*, but I don't want to maintain something like that.
Urwid has functions that give *great* estimates, but I'd like to avoid dependencies.
Leaving the right margin open was the quickest, cleanest hack :) I'm amazed at how hard this is to accomplish.
(See https://hsivonen.fi/string-length/ which ought to be required reading in all CS classes from now on.)
Ultimately I took both this decoration and the exception code into utils as an admission of guilt and a pledge to maintain what I did write ;)
Quoted 6 lines
Reviewed-by Eric Blake <[email protected]>[take with a grain of salt - my python is weak. But as you said in the cover letter, it's fairly straightforward to reproduce an environment where you can see it in action for hands-on testing]
I've long since given up on anyone understanding the Python I write. Eduardo did, so he quit. 😅
Thanks for the reviews.
Quoted 7 lines
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
PATCH 2/5 python/utils: add VerboseProcessError
In reply to John SnowThis adds an Exception that extends the Python stdlib subprocess.CalledProcessError.
The difference is that the str() method of this exception also adds the stdout/stderr logs. In effect, if this exception goes unhandled, Python will print the output in a visually distinct wrapper to the terminal so that it's easy to spot in a sea of traceback information.
--- python/qemu/utils/__init__.py | 36 +++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
python/qemu/utils/__init__.py +36 -0
@@ -18,6 +18,7 @@ import os import re import shutil +from subprocess import CalledProcessError import textwrap from typing import Optional
@@ -26,6 +27,7 @@ __all__ = ( + 'VerboseProcessError', 'add_visual_margin', 'get_info_usernet_hostfwd_port', 'kvm_available',
@@ -121,3 +123,37 @@ def _wrap(line: str) -> str: os.linesep.join(_wrap(line) for line in content.splitlines()), _bar(None, top=False), )) + + +class VerboseProcessError(CalledProcessError): + """ + The same as CalledProcessError, but more verbose. + + This is useful for debugging failed calls during test executions. + The return code, signal (if any), and terminal output will be displayed + on unhandled exceptions. + """ + def summary(self) -> str: + """Return the normal CalledProcessError str() output.""" + return super().__str__() + + def __str__(self) -> str: + lmargin = ' ' + width = -len(lmargin) + sections = [] + + name = 'output' if self.stderr is None else 'stdout' + if self.stdout: + sections.append(add_visual_margin(self.stdout, width, name)) + else: + sections.append(f"{name}: N/A") + + if self.stderr: + sections.append(add_visual_margin(self.stderr, width, 'stderr')) + elif self.stderr is not None: + sections.append("stderr: N/A") + + return os.linesep.join(( + self.summary(), + textwrap.indent(os.linesep.join(sections), prefix=lmargin), + ))
Signature · 1 line
2.34.1
Re: [PATCH 2/5] python/utils: add VerboseProcessError
Quoted 8 lines · John Snow
This adds an Exception that extends the Python stdlib subprocess.CalledProcessError.
The difference is that the
str()method of this exception also adds the stdout/stderr logs. In effect, if this exception goes unhandled, Python will print the output in a visually distinct wrapper to the terminal so that it's easy to spot in a sea of traceback information.
Quoted 5 lines
Signed-off-by John Snow <[email protected]>--- python/qemu/utils/__init__.py | 36 +++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 3/5 iotests: Remove explicit checks for qemu_img() == 0
In reply to John Snowqemu_img() returning zero ought to be the rule, not the exception. Remove all explicit checks against the condition in preparation for making non-zero returns an Exception.
--- tests/qemu-iotests/163 | 9 +++------ tests/qemu-iotests/216 | 6 +++--- tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/224 | 11 +++++------ tests/qemu-iotests/228 | 12 ++++++------ tests/qemu-iotests/257 | 3 +-- tests/qemu-iotests/258 | 4 ++-- tests/qemu-iotests/310 | 14 +++++++------- tests/qemu-iotests/tests/block-status-cache | 3 +-- tests/qemu-iotests/tests/image-fleecing | 4 ++-- tests/qemu-iotests/tests/mirror-ready-cancel-error | 6 ++---- tests/qemu-iotests/tests/mirror-top-perms | 3 +-- .../qemu-iotests/tests/remove-bitmap-from-backing | 8 ++++---- tests/qemu-iotests/tests/stream-error-on-reset | 4 ++-- 14 files changed, 40 insertions(+), 49 deletions(-)
tests/qemu-iotests/163 +3 -6
@@ -107,8 +107,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): if iotests.imgfmt == 'raw': return - self.assertEqual(qemu_img('check', test_img), 0, - "Verifying image corruption") + qemu_img('check', test_img) def test_empty_image(self): qemu_img('resize', '-f', iotests.imgfmt, '--shrink', test_img,
@@ -130,8 +129,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): qemu_img('resize', '-f', iotests.imgfmt, '--shrink', test_img, self.shrink_size) - self.assertEqual(qemu_img("compare", test_img, check_img), 0, - "Verifying image content") + qemu_img("compare", test_img, check_img) self.image_verify()
@@ -146,8 +144,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): qemu_img('resize', '-f', iotests.imgfmt, '--shrink', test_img, self.shrink_size) - self.assertEqual(qemu_img("compare", test_img, check_img), 0, - "Verifying image content") + qemu_img("compare", test_img, check_img) self.image_verify()
tests/qemu-iotests/216 +3 -3
@@ -51,10 +51,10 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Setting up images ---') log('') - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 + qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - '-F', iotests.imgfmt, top_img_path) == 0 + qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, + '-F', iotests.imgfmt, top_img_path) assert qemu_io_silent(top_img_path, '-c', 'write -P 2 1M 1M') == 0 log('Done')
tests/qemu-iotests/218 +1 -1
@@ -145,7 +145,7 @@ log('') with iotests.VM() as vm, \ iotests.FilePath('src.img') as src_img_path: - assert qemu_img('create', '-f', iotests.imgfmt, src_img_path, '64M') == 0 + qemu_img('create', '-f', iotests.imgfmt, src_img_path, '64M') assert qemu_io_silent('-f', iotests.imgfmt, src_img_path, '-c', 'write -P 42 0M 64M') == 0
tests/qemu-iotests/224 +5 -6
@@ -47,12 +47,11 @@ for filter_node_name in False, True: iotests.FilePath('top.img') as top_img_path, \ iotests.VM() as vm: - assert qemu_img('create', '-f', iotests.imgfmt, - base_img_path, '64M') == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - '-F', iotests.imgfmt, mid_img_path) == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path, - '-F', iotests.imgfmt, top_img_path) == 0 + qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') + qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, + '-F', iotests.imgfmt, mid_img_path) + qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path, + '-F', iotests.imgfmt, top_img_path) # Something to commit assert qemu_io_silent(mid_img_path, '-c', 'write -P 1 0 1M') == 0
tests/qemu-iotests/228 +6 -6
@@ -54,11 +54,11 @@ with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('top.img') as top_img_path, \ iotests.VM() as vm: - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 + qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') # Choose a funny way to describe the backing filename - assert qemu_img('create', '-f', iotests.imgfmt, '-b', - 'file:' + base_img_path, '-F', iotests.imgfmt, - top_img_path) == 0 + qemu_img('create', '-f', iotests.imgfmt, '-b', + 'file:' + base_img_path, '-F', iotests.imgfmt, + top_img_path) vm.launch()
@@ -172,8 +172,8 @@ with iotests.FilePath('base.img') as base_img_path, \ # (because qemu cannot "canonicalize"/"resolve" the backing # filename unless the backing file is opened implicitly with the # overlay) - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - '-F', iotests.imgfmt, top_img_path) == 0 + qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, + '-F', iotests.imgfmt, top_img_path) # You can only reliably override backing options by using a node # reference (or by specifying file.filename, but, well...)
tests/qemu-iotests/257 +1 -2
@@ -240,8 +240,7 @@ def compare_images(image, reference, baseimg=None, expected_match=True): """ expected_ret = 0 if expected_match else 1 if baseimg: - assert qemu_img("rebase", "-u", "-b", baseimg, '-F', iotests.imgfmt, - image) == 0 + qemu_img("rebase", "-u", "-b", baseimg, '-F', iotests.imgfmt, image) ret = qemu_img("compare", image, reference) log('qemu_img compare "{:s}" "{:s}" ==> {:s}, {:s}'.format( image, reference,
tests/qemu-iotests/258 +2 -2
@@ -75,13 +75,13 @@ def test_concurrent_finish(write_to_stream_node): # It is important to use raw for the base layer (so that # permissions are just handed through to the protocol layer) - assert qemu_img('create', '-f', 'raw', node0_path, '64M') == 0 + qemu_img('create', '-f', 'raw', node0_path, '64M') stream_throttle=None commit_throttle=None for path in [node1_path, node2_path, node3_path, node4_path]: - assert qemu_img('create', '-f', iotests.imgfmt, path, '64M') == 0 + qemu_img('create', '-f', iotests.imgfmt, path, '64M') if write_to_stream_node: # This is what (most of the time) makes commit finish
tests/qemu-iotests/310 +7 -7
@@ -43,15 +43,15 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Setting up images ---') log('') - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 + qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0 assert qemu_io_silent(base_img_path, '-c', 'write -P 1 3M 1M') == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, - '-F', iotests.imgfmt, mid_img_path) == 0 + qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, + '-F', iotests.imgfmt, mid_img_path) assert qemu_io_silent(mid_img_path, '-c', 'write -P 3 2M 1M') == 0 assert qemu_io_silent(mid_img_path, '-c', 'write -P 3 4M 1M') == 0 - assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path, - '-F', iotests.imgfmt, top_img_path) == 0 + qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path, + '-F', iotests.imgfmt, top_img_path) assert qemu_io_silent(top_img_path, '-c', 'write -P 2 1M 1M') == 0 # 0 1 2 3 4
@@ -105,8 +105,8 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') # Detach backing to check that we can read the data from the top level now - assert qemu_img('rebase', '-u', '-b', '', '-f', iotests.imgfmt, - top_img_path) == 0 + qemu_img('rebase', '-u', '-b', '', '-f', iotests.imgfmt, + top_img_path) assert qemu_io_silent(top_img_path, '-c', 'read -P 0 0 1M') == 0 assert qemu_io_silent(top_img_path, '-c', 'read -P 2 1M 1M') == 0
tests/qemu-iotests/tests/block-status-cache +1 -2
@@ -35,8 +35,7 @@ nbd_sock = os.path.join(iotests.sock_dir, 'nbd.sock') class TestBscWithNbd(iotests.QMPTestCase): def setUp(self) -> None: """Just create an empty image with a read-only NBD server on it""" - assert qemu_img_create('-f', iotests.imgfmt, test_img, - str(image_size)) == 0 + qemu_img_create('-f', iotests.imgfmt, test_img, str(image_size)) # Pass --allocation-depth to enable the qemu:allocation-depth context, # which we are going to query to provoke a block-status inquiry with
tests/qemu-iotests/tests/image-fleecing +2 -2
@@ -53,8 +53,8 @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm): log('--- Setting up images ---') log('') - assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 - assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0 + qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') + qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') for p in patterns: qemu_io('-f', iotests.imgfmt,
tests/qemu-iotests/tests/mirror-ready-cancel-error +2 -4
@@ -31,10 +31,8 @@ target = os.path.join(iotests.test_dir, 'target.img') class TestMirrorReadyCancelError(iotests.QMPTestCase): def setUp(self) -> None: - assert iotests.qemu_img_create('-f', iotests.imgfmt, source, - str(image_size)) == 0 - assert iotests.qemu_img_create('-f', iotests.imgfmt, target, - str(image_size)) == 0 + iotests.qemu_img_create('-f', iotests.imgfmt, source, str(image_size)) + iotests.qemu_img_create('-f', iotests.imgfmt, target, str(image_size)) # Ensure that mirror will copy something before READY so the # target format layer will forward the pre-READY flush to its
tests/qemu-iotests/tests/mirror-top-perms +1 -2
@@ -34,8 +34,7 @@ source = os.path.join(iotests.test_dir, 'source.img') class TestMirrorTopPerms(iotests.QMPTestCase): def setUp(self): - assert qemu_img('create', '-f', iotests.imgfmt, source, - str(image_size)) == 0 + qemu_img('create', '-f', iotests.imgfmt, source, str(image_size)) self.vm = iotests.VM() self.vm.add_drive(source) self.vm.add_blockdev(f'null-co,node-name=null,size={image_size}')
tests/qemu-iotests/tests/remove-bitmap-from-backing +4 -4
@@ -27,11 +27,11 @@ iotests.script_initialize(supported_fmts=['qcow2'], top, base = iotests.file_path('top', 'base') size = '1M' -assert qemu_img_create('-f', iotests.imgfmt, base, size) == 0 -assert qemu_img_create('-f', iotests.imgfmt, '-b', base, - '-F', iotests.imgfmt, top, size) == 0 +qemu_img_create('-f', iotests.imgfmt, base, size) +qemu_img_create('-f', iotests.imgfmt, '-b', base, + '-F', iotests.imgfmt, top, size) -assert qemu_img('bitmap', '--add', base, 'bitmap0') == 0 +qemu_img('bitmap', '--add', base, 'bitmap0') # Just assert that our method of checking bitmaps in the image works. assert 'bitmaps' in qemu_img_pipe('info', base)
tests/qemu-iotests/tests/stream-error-on-reset +2 -2
@@ -54,9 +54,9 @@ class TestStreamErrorOnReset(QMPTestCase): to it will result in an error - top image is attached to a virtio-scsi device """ - assert qemu_img_create('-f', imgfmt, base, str(image_size)) == 0 + qemu_img_create('-f', imgfmt, base, str(image_size)) assert qemu_io_silent('-c', f'write 0 {data_size}', base) == 0 - assert qemu_img_create('-f', imgfmt, top, str(image_size)) == 0 + qemu_img_create('-f', imgfmt, top, str(image_size)) self.vm = iotests.VM() self.vm.add_args('-accel', 'tcg') # Make throttling work properly
Signature · 1 line
2.34.1
PATCH 4/5 iotests: make qemu_img raise on non-zero rc by default
In reply to John Snowre-write qemu_img() as a function that will by default raise a VerboseProcessException (extended from CalledProcessException) on non-zero return codes. This will produce a stack trace that will show the command line arguments and return code from the failed process run.
Users that want something more flexible (there appears to be only one) can use check=False and manage the return themselves. However, when the return code is negative, the Exception will be raised no matter what. This is done under the belief that there's no legitimate reason, even in negative tests, to see a crash from qemu-img.
--- tests/qemu-iotests/257 | 8 ++++-- tests/qemu-iotests/iotests.py | 54 +++++++++++++++++++++++++++++++---- 2 files changed, 53 insertions(+), 9 deletions(-)
tests/qemu-iotests/257 +5 -3
@@ -241,11 +241,13 @@ def compare_images(image, reference, baseimg=None, expected_match=True): expected_ret = 0 if expected_match else 1 if baseimg: qemu_img("rebase", "-u", "-b", baseimg, '-F', iotests.imgfmt, image) - ret = qemu_img("compare", image, reference) + + sub = qemu_img("compare", image, reference, check=False) + log('qemu_img compare "{:s}" "{:s}" ==> {:s}, {:s}'.format( image, reference, - "Identical" if ret == 0 else "Mismatch", - "OK!" if ret == expected_ret else "ERROR!"), + "Identical" if sub.returncode == 0 else "Mismatch", + "OK!" if sub.returncode == expected_ret else "ERROR!"), filters=[iotests.filter_testfiles]) def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
tests/qemu-iotests/iotests.py +48 -6
78 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH 4/5] iotests: make qemu_img raise on non-zero rc by default
Quoted 11 lines · John Snow
re-write
qemu_img()as a function that will by default raise a VerboseProcessException (extended from CalledProcessException) on non-zero return codes. This will produce a stack trace that will show the command line arguments and return code from the failed process run.Users that want something more flexible (there appears to be only one) can use check=False and manage the return themselves. However, when the return code is negative, the Exception will be raised no matter what. This is done under the belief that there's no legitimate reason, even in negative tests, to see a crash from qemu-img.
Quoted 6 lines
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/257 | 8 ++++-- tests/qemu-iotests/iotests.py | 54 +++++++++++++++++++++++++++++++---- 2 files changed, 53 insertions(+), 9 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH 5/5 iotests: fortify compare_images() against crashes
In reply to John SnowFority compare_images() to be more discerning about the status codes it receives. If qemu_img() returns an exit code that implies it didn't actually perform the comparison, treat that as an exceptional circumstance and force the caller to be aware of the peril.
If a negative test is desired (Perhaps to test how qemu_img compare behaves on malformed images, for instance), it is still possible to catch the exception in the test and deal with that circumstance manually.
--- tests/qemu-iotests/iotests.py | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
tests/qemu-iotests/iotests.py +16 -5
@@ -474,11 +474,22 @@ def qemu_nbd_popen(*args): p.kill() p.wait() -def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt): - '''Return True if two image files are identical''' - res = qemu_img('compare', '-f', fmt1, - '-F', fmt2, img1, img2, check=False) - return res.returncode == 0 +def compare_images(img1: str, img2: str, + fmt1: str = imgfmt, fmt2: str = imgfmt) -> bool: + """ + Compare two images with QEMU_IMG; return True if they are identical. + + :raise CalledProcessError: + when qemu-img crashes or returns a status code of anything other + than 0 (identical) or 1 (different). + """ + try: + qemu_img('compare', '-f', fmt1, '-F', fmt2, img1, img2) + return True + except subprocess.CalledProcessError as exc: + if exc.returncode == 1: + return False + raise def create_image(name, size): '''Create a fully-allocated raw image with sector markers'''
Signature · 1 line
2.34.1
Re: [PATCH 5/5] iotests: fortify compare_images() against crashes
Quoted 1 line · John Snow
Fority
compare_images()to be more discerning about the status codes it
Fortify
Quoted 5 lines
receives. If
qemu_img()returns an exit code that implies it didn't actually perform the comparison, treat that as an exceptional circumstance and force the caller to be aware of the peril.If a negative test is desired (Perhaps to test how
qemu_imgcompare
perhaps
Quoted 8 lines
behaves on malformed images, for instance), it is still possible to catch the exception in the test and deal with that circumstance manually.
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org