iotests: use python logging
v8 cover iotests: use python logging
This series uses python logging to enable output conditionally on iotests.log(). We unify an initialization call (which also enables debugging output for those tests with -d) and then make the switch inside of iotests.
It will help alleviate the need to create logged/unlogged versions of all the various helpers we have made.
Also, I got lost and accidentally delinted iotests while I was here. Sorry about that.
V8:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/11:[----] [--] 'iotests: do a light delinting' 002/11:[----] [--] 'iotests: don't use 'format' for drive_add' 003/11:[----] [--] 'iotests: ignore import warnings from pylint' 004/11:[----] [--] 'iotests: replace mutable list default args' 005/11:[----] [--] 'iotests: add pylintrc file' 006/11:[down] 'iotests: drop Python 3.4 compatibility code' 007/11:[0033] [FC] 'iotests: limit line length to 79 chars' 008/11:[----] [--] 'iotests: add script_initialize' 009/11:[----] [--] 'iotest 258: use script_main' 010/11:[----] [--] 'iotests: Mark verify functions as private' 011/11:[0007] [FC] 'iotests: use python logging for iotests.log()'
6: Split out the little drop of Python 3.4 code. (Phil)
7: Change line continuation styles (QEMU Memorial Choir)
11: Rebase changes; remove use_log from more places, adjust test output.
V7:
- All delinting patches are now entirely front-loaded.
- Redid delinting to avoid "correcting" no-else-return statements.
- Moved more mutable list corrections into patch 4, to make it standalone.
- Moved pylintrc up to patch 5. Disabled no-else-return.
- Added patch 6 to require line length checks.
(Some python 3.4 compatibility code is removed as a consequence.)
- Patch 7 changes slightly as a result of patch 4 changes.
- Added some logging explainer into patch 10.
(Patch changes slightly because of patch 6.)
V6:
- It's been so long since V5, let's just look at it anew.
- Dropped patch 1, rebased, added more delinting.
- I'm not touching the supported_platforms thing. Not interested in rehashing that debate.
V5:
- Rebased again
- Allow Python tests to run on any platform
V4:
- Rebased on top of kwolf/block at the behest of mreitz
V3:
- Rebased for 4.1+; now based on main branch.
V2:
- Added all of the other python tests I missed to use script_initialize
- Refactored the common setup as per Ehabkost's suggestion
- Added protocol arguments to common initialization,
but this isn't strictly required.
John Snow (11):
iotests: do a light delinting
iotests: don't use 'format' for drive_add
iotests: ignore import warnings from pylint
iotests: replace mutable list default args
iotests: add pylintrc file
iotests: drop Python 3.4 compatibility code
iotests: limit line length to 79 chars
iotests: add script_initialize
iotest 258: use script_main
iotests: Mark verify functions as private
iotests: use python logging for iotests.log()
tests/qemu-iotests/030 | 4 +- tests/qemu-iotests/055 | 3 +- tests/qemu-iotests/149 | 3 +- tests/qemu-iotests/155 | 2 +- tests/qemu-iotests/194 | 4 +- tests/qemu-iotests/202 | 4 +- tests/qemu-iotests/203 | 4 +- tests/qemu-iotests/206 | 2 +- tests/qemu-iotests/207 | 6 +- tests/qemu-iotests/208 | 2 +- tests/qemu-iotests/209 | 2 +- tests/qemu-iotests/210 | 6 +- tests/qemu-iotests/211 | 6 +- tests/qemu-iotests/212 | 6 +- tests/qemu-iotests/213 | 6 +- tests/qemu-iotests/216 | 4 +- tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/219 | 2 +- tests/qemu-iotests/222 | 7 +- tests/qemu-iotests/224 | 4 +- tests/qemu-iotests/228 | 6 +- tests/qemu-iotests/234 | 4 +- tests/qemu-iotests/235 | 4 +- tests/qemu-iotests/236 | 2 +- tests/qemu-iotests/237 | 2 +- tests/qemu-iotests/238 | 2 + tests/qemu-iotests/242 | 2 +- tests/qemu-iotests/245 | 1 + tests/qemu-iotests/245.out | 24 +-- tests/qemu-iotests/246 | 2 +- tests/qemu-iotests/248 | 2 +- tests/qemu-iotests/254 | 2 +- tests/qemu-iotests/255 | 2 +- tests/qemu-iotests/256 | 2 +- tests/qemu-iotests/258 | 10 +- tests/qemu-iotests/260 | 4 +- tests/qemu-iotests/262 | 4 +- tests/qemu-iotests/264 | 4 +- tests/qemu-iotests/277 | 2 + tests/qemu-iotests/280 | 8 +- tests/qemu-iotests/283 | 4 +- tests/qemu-iotests/iotests.py | 312 ++++++++++++++++++++-------------- tests/qemu-iotests/pylintrc | 26 +++ 43 files changed, 307 insertions(+), 203 deletions(-) create mode 100644 tests/qemu-iotests/pylintrc
Signature · 1 line
2.21.1
PATCH v8 1/11 iotests: do a light delinting
This doesn't fix everything in here, but it does help clean up the pylint report considerably.
This should be 100% style changes only; the intent is to make pylint more useful by working on establishing a baseline for iotests that we can gate against in the future.
--- tests/qemu-iotests/iotests.py | 83 ++++++++++++++++++----------------- 1 file changed, 43 insertions(+), 40 deletions(-)
tests/qemu-iotests/iotests.py +43 -40
@@ -16,11 +16,9 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # -import errno import os import re import subprocess -import string import unittest import sys import struct
@@ -34,7 +32,7 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qtest -assert sys.version_info >= (3,6) +assert sys.version_info >= (3, 6) # This will not work if arguments contain spaces but is necessary if we # want to support the override options that ./check supports.
@@ -138,11 +136,11 @@ def qemu_img_log(*args): return result def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]): - args = [ 'info' ] + args = ['info'] if imgopts: args.append('--image-opts') else: - args += [ '-f', imgfmt ] + args += ['-f', imgfmt] args += extra_args args.append(filename)
@@ -221,7 +219,7 @@ def cmd(self, cmd): # quit command is in close(), '\n' is added automatically assert '\n' not in cmd cmd = cmd.strip() - assert cmd != 'q' and cmd != 'quit' + assert cmd not in ('q', 'quit') self._p.stdin.write(cmd + '\n') self._p.stdin.flush() return self._read_output()
@@ -243,10 +241,8 @@ def qemu_nbd_early_pipe(*args): sys.stderr.write('qemu-nbd received signal %i: %s\n' % (-exitcode, ' '.join(qemu_nbd_args + ['--fork'] + list(args)))) - if exitcode == 0: - return exitcode, '' - else: - return exitcode, subp.communicate()[0] + + return exitcode, subp.communicate()[0] if exitcode else '' def qemu_nbd_popen(*args): '''Run qemu-nbd in daemon mode and return the parent's exit code'''
@@ -310,7 +306,7 @@ def filter_qmp(qmsg, filter_fn): items = qmsg.items() for k, v in items: - if isinstance(v, list) or isinstance(v, dict): + if isinstance(v, (dict, list)): qmsg[k] = filter_qmp(v, filter_fn) else: qmsg[k] = filter_fn(k, v)
@@ -321,7 +317,7 @@ def filter_testfiles(msg): return msg.replace(prefix, 'TEST_DIR/PID-') def filter_qmp_testfiles(qmsg): - def _filter(key, value): + def _filter(_key, value): if is_str(value): return filter_testfiles(value) return value
@@ -347,7 +343,7 @@ def filter_imgfmt(msg): return msg.replace(imgfmt, 'IMGFMT') def filter_qmp_imgfmt(qmsg): - def _filter(key, value): + def _filter(_key, value): if is_str(value): return filter_imgfmt(value) return value
@@ -358,7 +354,7 @@ def log(msg, filters=[], indent=None): If indent is provided, JSON serializable messages are pretty-printed.''' for flt in filters: msg = flt(msg) - if isinstance(msg, dict) or isinstance(msg, list): + if isinstance(msg, (dict, list)): # Python < 3.4 needs to know not to add whitespace when pretty-printing: separators = (', ', ': ') if indent is None else (',', ': ') # Don't sort if it's already sorted
@@ -369,14 +365,14 @@ def log(msg, filters=[], indent=None): print(msg) class Timeout: - def __init__(self, seconds, errmsg = "Timeout"): + def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg def __enter__(self): signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self - def __exit__(self, type, value, traceback): + def __exit__(self, exc_type, value, traceback): signal.setitimer(signal.ITIMER_REAL, 0) return False def timeout(self, signum, frame):
@@ -385,7 +381,7 @@ def timeout(self, signum, frame): def file_pattern(name): return "{0}-{1}".format(os.getpid(), name) -class FilePaths(object): +class FilePaths: """ FilePaths is an auto-generated filename that cleans itself up.
@@ -532,11 +528,11 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command', - command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) def resume_drive(self, drive): self.qmp('human-monitor-command', - command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) + command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command'''
@@ -547,8 +543,8 @@ def flatten_qmp_object(self, obj, output=None, basestr=''): if output is None: output = dict() if isinstance(obj, list): - for i in range(len(obj)): - self.flatten_qmp_object(obj[i], output, basestr + str(i) + '.') + for i, atom in enumerate(obj): + self.flatten_qmp_object(atom, output, basestr + str(i) + '.') elif isinstance(obj, dict): for key in obj: self.flatten_qmp_object(obj[key], output, basestr + key + '.')
@@ -706,9 +702,7 @@ def get_bitmap(self, node_name, bitmap_name, recording=None, bitmaps=None): for bitmap in bitmaps[node_name]: if bitmap.get('name', '') == bitmap_name: - if recording is None: - return bitmap - elif bitmap.get('recording') == recording: + if recording is None or bitmap.get('recording') == recording: return bitmap return None
@@ -759,12 +753,13 @@ def assert_block_path(self, root, path, expected_node, graph=None): assert node is not None, 'Cannot follow path %s%s' % (root, path) try: - node_id = next(edge['child'] for edge in graph['edges'] \ - if edge['parent'] == node['id'] and - edge['name'] == child_name) + node_id = next(edge['child'] for edge in graph['edges'] + if (edge['parent'] == node['id'] and + edge['name'] == child_name)) + + node = next(node for node in graph['nodes'] + if node['id'] == node_id) - node = next(node for node in graph['nodes'] \ - if node['id'] == node_id) except StopIteration: node = None
@@ -782,6 +777,12 @@ def assert_block_path(self, root, path, expected_node, graph=None): class QMPTestCase(unittest.TestCase): '''Abstract base class for QMP test cases''' + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + # Many users of this class set a VM property we rely on heavily + # in the methods below. + self.vm = None + def dictpath(self, d, path): '''Traverse a path in a nested dict''' for component in path.split('/'):
@@ -827,7 +828,7 @@ def assert_qmp(self, d, path, value): else: self.assertEqual(result, value, '"%s" is "%s", expected "%s"' - % (path, str(result), str(value))) + % (path, str(result), str(value))) def assert_no_active_block_jobs(self): result = self.vm.qmp('query-block-jobs')
@@ -837,15 +838,15 @@ def assert_has_block_node(self, node_name=None, file_name=None): """Issue a query-named-block-nodes and assert node_name and/or file_name is present in the result""" def check_equal_or_none(a, b): - return a == None or b == None or a == b + return a is None or b is None or a == b assert node_name or file_name result = self.vm.qmp('query-named-block-nodes') for x in result["return"]: if check_equal_or_none(x.get("node-name"), node_name) and \ check_equal_or_none(x.get("file"), file_name): return - self.assertTrue(False, "Cannot find %s %s in result:\n%s" % \ - (node_name, file_name, result)) + self.fail("Cannot find %s %s in result:\n%s" % + (node_name, file_name, result)) def assert_json_filename_equal(self, json_filename, reference): '''Asserts that the given filename is a json: filename and that its
@@ -894,13 +895,13 @@ def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0, self.assert_qmp(event, 'data/error', error) self.assert_no_active_block_jobs() return event - elif event['event'] == 'JOB_STATUS_CHANGE': + if event['event'] == 'JOB_STATUS_CHANGE': self.assert_qmp(event, 'data/id', drive) def wait_ready(self, drive='drive0'): - '''Wait until a block job BLOCK_JOB_READY event''' - f = {'data': {'type': 'mirror', 'device': drive } } - event = self.vm.event_wait(name='BLOCK_JOB_READY', match=f) + """Wait until a BLOCK_JOB_READY event, and return the event.""" + f = {'data': {'type': 'mirror', 'device': drive}} + return self.vm.event_wait(name='BLOCK_JOB_READY', match=f) def wait_ready_and_cancel(self, drive='drive0'): self.wait_ready(drive=drive)
@@ -929,7 +930,7 @@ def pause_wait(self, job_id='job0'): for job in result['return']: if job['device'] == job_id: found = True - if job['paused'] == True and job['busy'] == False: + if job['paused'] and not job['busy']: return job break assert found
@@ -1026,8 +1027,8 @@ def qemu_pipe(*args): universal_newlines=True) exitcode = subp.wait() if exitcode < 0: - sys.stderr.write('qemu received signal %i: %s\n' % (-exitcode, - ' '.join(args))) + sys.stderr.write('qemu received signal %i: %s\n' % + (-exitcode, ' '.join(args))) return subp.communicate()[0] def supported_formats(read_only=False):
@@ -1059,6 +1060,7 @@ def func_wrapper(test_case: QMPTestCase, *args, **kwargs): if usf_list: test_case.case_skip('{}: formats {} are not whitelisted'.format( test_case, usf_list)) + return None else: return func(test_case, *args, **kwargs) return func_wrapper
@@ -1070,6 +1072,7 @@ def skip_if_user_is_root(func): def func_wrapper(*args, **kwargs): if os.getuid() == 0: case_notrun('{}: cannot be run as root'.format(args[0])) + return None else: return func(*args, **kwargs) return func_wrapper
Signature · 1 line
2.21.1
PATCH v8 2/11 iotests: don't use 'format' for drive_add
In reply to John SnowIt shadows (with a different type) the built-in format. Use something else.
--- tests/qemu-iotests/055 | 3 ++- tests/qemu-iotests/iotests.py | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-)
tests/qemu-iotests/055 +2 -1
@@ -469,7 +469,8 @@ class TestDriveCompression(iotests.QMPTestCase): qemu_img('create', '-f', fmt, blockdev_target_img, str(TestDriveCompression.image_len), *args) if attach_target: - self.vm.add_drive(blockdev_target_img, format=fmt, interface="none") + self.vm.add_drive(blockdev_target_img, + img_format=fmt, interface="none") self.vm.launch()
tests/qemu-iotests/iotests.py +3 -3
@@ -482,21 +482,21 @@ def add_drive_raw(self, opts): self._args.append(opts) return self - def add_drive(self, path, opts='', interface='virtio', format=imgfmt): + def add_drive(self, path, opts='', interface='virtio', img_format=imgfmt): '''Add a virtio-blk drive to the VM''' options = ['if=%s' % interface, 'id=drive%d' % self._num_drives] if path is not None: options.append('file=%s' % path) - options.append('format=%s' % format) + options.append('format=%s' % img_format) options.append('cache=%s' % cachemode) options.append('aio=%s' % aiomode) if opts: options.append(opts) - if format == 'luks' and 'key-secret' not in opts: + if img_format == 'luks' and 'key-secret' not in opts: # default luks support if luks_default_secret_object not in self._args: self.add_object(luks_default_secret_object)
Signature · 1 line
2.21.1
PATCH v8 9/11 iotest 258: use script_main
In reply to John SnowSince this one is nicely factored to use a single entry point, use script_main to run the tests.
--- tests/qemu-iotests/258 | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-)
tests/qemu-iotests/258 +4 -7
@@ -23,12 +23,6 @@ import iotests from iotests import log, qemu_img, qemu_io_silent, \ filter_qmp_testfiles, filter_qmp_imgfmt -# Need backing file and change-backing-file support -iotests.script_initialize( - supported_fmts=['qcow2', 'qed'], - supported_platforms=['linux'], -) - # Returns a node for blockdev-add def node(node_name, path, backing=None, fmt=None, throttle=None): if fmt is None:
@@ -161,4 +155,7 @@ def main(): test_concurrent_finish(False) if __name__ == '__main__': - main() + # Need backing file and change-backing-file support + iotests.script_main(main, + supported_fmts=['qcow2', 'qed'], + supported_platforms=['linux'])
Signature · 1 line
2.21.1
PATCH v8 3/11 iotests: ignore import warnings from pylint
In reply to John SnowThe right way to solve this is to come up with a virtual environment infrastructure that sets all the paths correctly, and/or to create installable python modules that can be imported normally.
That's hard, so just silence this error for now.
--- tests/qemu-iotests/iotests.py | 1 + 1 file changed, 1 insertion(+)
tests/qemu-iotests/iotests.py +1 -0
@@ -29,6 +29,7 @@ import io from collections import OrderedDict +# pylint: disable=import-error, wrong-import-position sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qtest
Signature · 1 line
2.21.1
PATCH v8 4/11 iotests: replace mutable list default args
In reply to John SnowIt's bad hygiene: if we modify this list, it will be modified across all invocations.
(Remaining bad usages are fixed in a subsequent patch which changes the function signature anyway.)
--- tests/qemu-iotests/iotests.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
tests/qemu-iotests/iotests.py +12 -12
@@ -136,7 +136,7 @@ def qemu_img_log(*args): log(result, filters=[filter_testfiles]) return result -def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]): +def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()): args = ['info'] if imgopts: args.append('--image-opts')
@@ -350,7 +350,7 @@ def _filter(_key, value): return value return filter_qmp(qmsg, _filter) -def log(msg, filters=[], indent=None): +def log(msg, filters=(), indent=None): '''Logs either a string message or a JSON serializable message (like QMP). If indent is provided, JSON serializable messages are pretty-printed.''' for flt in filters:
@@ -566,7 +566,7 @@ def get_qmp_events_filtered(self, wait=60.0): result.append(filter_qmp_event(ev)) return result - def qmp_log(self, cmd, filters=[], indent=None, **kwargs): + def qmp_log(self, cmd, filters=(), indent=None, **kwargs): full_cmd = OrderedDict(( ("execute", cmd), ("arguments", ordered_qmp(kwargs))
@@ -970,7 +970,7 @@ def case_notrun(reason): open('%s/%s.casenotrun' % (output_dir, seq), 'a').write( ' [case not run] ' + reason + '\n') -def verify_image_format(supported_fmts=[], unsupported_fmts=[]): +def verify_image_format(supported_fmts=(), unsupported_fmts=()): assert not (supported_fmts and unsupported_fmts) if 'generic' in supported_fmts and \
@@ -984,7 +984,7 @@ def verify_image_format(supported_fmts=[], unsupported_fmts=[]): if not_sup or (imgfmt in unsupported_fmts): notrun('not suitable for this image format: %s' % imgfmt) -def verify_protocol(supported=[], unsupported=[]): +def verify_protocol(supported=(), unsupported=()): assert not (supported and unsupported) if 'generic' in supported:
@@ -1003,11 +1003,11 @@ def verify_platform(supported=None, unsupported=None): if not any((sys.platform.startswith(x) for x in supported)): notrun('not suitable for this OS: %s' % sys.platform) -def verify_cache_mode(supported_cache_modes=[]): +def verify_cache_mode(supported_cache_modes=()): if supported_cache_modes and (cachemode not in supported_cache_modes): notrun('not suitable for this cache mode: %s' % cachemode) -def verify_aio_mode(supported_aio_modes=[]): +def verify_aio_mode(supported_aio_modes=()): if supported_aio_modes and (aiomode not in supported_aio_modes): notrun('not suitable for this aio mode: %s' % aiomode)
@@ -1047,7 +1047,7 @@ def supported_formats(read_only=False): return supported_formats.formats[read_only] -def skip_if_unsupported(required_formats=[], read_only=False): +def skip_if_unsupported(required_formats=(), read_only=False): '''Skip Test Decorator Runs the test if all the required formats are whitelisted''' def skip_test_decorator(func):
@@ -1098,11 +1098,11 @@ def execute_unittest(output, verbosity, debug): sys.stderr.write(out) def execute_test(test_function=None, - supported_fmts=[], + supported_fmts=(), supported_platforms=None, - supported_cache_modes=[], supported_aio_modes={}, - unsupported_fmts=[], supported_protocols=[], - unsupported_protocols=[]): + supported_cache_modes=(), supported_aio_modes=(), + unsupported_fmts=(), supported_protocols=(), + unsupported_protocols=()): """Run either unittest or script-style tests.""" # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
Signature · 1 line
2.21.1
PATCH v8 11/11 iotests: use python logging for iotests.log()
In reply to John SnowWe can turn logging on/off globally instead of per-function.
Remove use_log from run_job, and use python logging to turn on diffable output when we run through a script entry point.
iotest 245 changes output order due to buffering reasons.
An extended note on python logging:
A NullHandler is added to qemu.iotests to stop output from being generated if this code is used as a library without configuring logging. A NullHandler is only needed at the root, so a duplicate handler is not needed for qemu.iotests.diff_io.
When logging is not configured, messages at the 'WARNING' levels or above are printed with default settings. The NullHandler stops this from occurring, which is considered good hygiene for code used as a library.
See https://docs.python.org/3/howto/logging.html#library-config
When logging is actually enabled (always at the behest of an explicit call by a client script), a root logger is implicitly created at the root, which allows messages to propagate upwards and be handled/emitted from the root logger with default settings.
When we want iotest logging, we attach a handler to the qemu.iotests.diff_io logger and disable propagation to avoid possible double-printing.
For more information on python logging infrastructure, I highly recommend downloading the pip package logging_tree, which provides convenient visualizations of the hierarchical logging configuration under different circumstances.
See https://pypi.org/project/logging_tree/ for more information.
--- tests/qemu-iotests/030 | 4 +-- tests/qemu-iotests/155 | 2 +- tests/qemu-iotests/245 | 1 + tests/qemu-iotests/245.out | 24 ++++++++-------- tests/qemu-iotests/iotests.py | 53 ++++++++++++++++++++--------------- 5 files changed, 46 insertions(+), 38 deletions(-)
tests/qemu-iotests/030 +2 -2
@@ -411,8 +411,8 @@ class TestParallelOps(iotests.QMPTestCase): result = self.vm.qmp('block-job-set-speed', device='drive0', speed=0) self.assert_qmp(result, 'return', {}) - self.vm.run_job(job='drive0', auto_dismiss=True, use_log=False) - self.vm.run_job(job='node4', auto_dismiss=True, use_log=False) + self.vm.run_job(job='drive0', auto_dismiss=True) + self.vm.run_job(job='node4', auto_dismiss=True) self.assert_no_active_block_jobs() # Test a block-stream and a block-commit job in parallel
tests/qemu-iotests/155 +1 -1
@@ -188,7 +188,7 @@ class MirrorBaseClass(BaseClass): self.assert_qmp(result, 'return', {}) - self.vm.run_job('mirror-job', use_log=False, auto_finalize=False, + self.vm.run_job('mirror-job', auto_finalize=False, pre_finalize=self.openBacking, auto_dismiss=True) def testFull(self):
tests/qemu-iotests/245 +1 -0
@@ -1027,5 +1027,6 @@ class TestBlockdevReopen(iotests.QMPTestCase): self.run_test_iothreads(None, 'iothread0') if __name__ == '__main__': + iotests.activate_logging() iotests.main(supported_fmts=["qcow2"], supported_protocols=["file"])
tests/qemu-iotests/245.out +12 -12
@@ -1,17 +1,17 @@ +{"execute": "job-finalize", "arguments": {"id": "commit0"}} +{"return": {}} +{"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "job-finalize", "arguments": {"id": "stream0"}} +{"return": {}} +{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"execute": "job-finalize", "arguments": {"id": "stream0"}} +{"return": {}} +{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} ..................... ---------------------------------------------------------------------- Ran 21 tests OK -{"execute": "job-finalize", "arguments": {"id": "commit0"}} -{"return": {}} -{"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed": 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"execute": "job-finalize", "arguments": {"id": "stream0"}} -{"return": {}} -{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"execute": "job-finalize", "arguments": {"id": "stream0"}} -{"return": {}} -{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
tests/qemu-iotests/iotests.py +30 -23
123 lines not on this page · raw message
Signature · 1 line
2.21.1
PATCH v8 5/11 iotests: add pylintrc file
In reply to John SnowThis allows others to get repeatable results with pylint. If you run pylint iotests.py, you should see a 100% pass.
--- tests/qemu-iotests/pylintrc | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 tests/qemu-iotests/pylintrc
tests/qemu-iotests/pylintrc new file +22 -0
@@ -0,0 +1,22 @@ +[MESSAGES CONTROL] + +# Disable the message, report, category or checker with the given id(s). You +# can either give multiple identifiers separated by comma (,) or put this +# option multiple times (only on the command line, not in the configuration +# file where it should appear only once). You can also use "--disable=all" to +# disable everything first and then reenable specific checks. For example, if +# you want to run only the similarities checker, you can use "--disable=all +# --enable=similarities". If you want to run only the classes checker, but have +# no Warning level messages displayed, use "--disable=all --enable=classes +# --disable=W". +disable=invalid-name, + no-else-return, + too-many-lines, + too-few-public-methods, + too-many-arguments, + too-many-locals, + too-many-branches, + too-many-public-methods, + # These are temporary, and should be removed: + missing-docstring, + line-too-long,
Signature · 1 line
2.21.1
PATCH v8 10/11 iotests: Mark verify functions as private
In reply to John SnowMark the verify functions as "private" with a leading underscore, to discourage their use.
(Also, make pending patches not yet using the new entry points fail in a very obvious way.)
--- tests/qemu-iotests/iotests.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
tests/qemu-iotests/iotests.py +10 -10
@@ -988,7 +988,7 @@ def case_notrun(reason): open('%s/%s.casenotrun' % (output_dir, seq), 'a').write( ' [case not run] ' + reason + '\n') -def verify_image_format(supported_fmts=(), unsupported_fmts=()): +def _verify_image_format(supported_fmts=(), unsupported_fmts=()): assert not (supported_fmts and unsupported_fmts) if 'generic' in supported_fmts and \
@@ -1002,7 +1002,7 @@ def verify_image_format(supported_fmts=(), unsupported_fmts=()): if not_sup or (imgfmt in unsupported_fmts): notrun('not suitable for this image format: %s' % imgfmt) -def verify_protocol(supported=(), unsupported=()): +def _verify_protocol(supported=(), unsupported=()): assert not (supported and unsupported) if 'generic' in supported:
@@ -1012,7 +1012,7 @@ def verify_protocol(supported=(), unsupported=()): if not_sup or (imgproto in unsupported): notrun('not suitable for this protocol: %s' % imgproto) -def verify_platform(supported=(), unsupported=()): +def _verify_platform(supported=(), unsupported=()): if any((sys.platform.startswith(x) for x in unsupported)): notrun('not suitable for this OS: %s' % sys.platform)
@@ -1020,11 +1020,11 @@ def verify_platform(supported=(), unsupported=()): if not any((sys.platform.startswith(x) for x in supported)): notrun('not suitable for this OS: %s' % sys.platform) -def verify_cache_mode(supported_cache_modes=()): +def _verify_cache_mode(supported_cache_modes=()): if supported_cache_modes and (cachemode not in supported_cache_modes): notrun('not suitable for this cache mode: %s' % cachemode) -def verify_aio_mode(supported_aio_modes=()): +def _verify_aio_mode(supported_aio_modes=()): if supported_aio_modes and (aiomode not in supported_aio_modes): notrun('not suitable for this aio mode: %s' % aiomode)
@@ -1151,11 +1151,11 @@ def execute_setup_common(supported_fmts: Collection[str] = (), sys.stderr.write('Please run this test via the "check" script\n') sys.exit(os.EX_USAGE) - verify_image_format(supported_fmts, unsupported_fmts) - verify_protocol(supported_protocols, unsupported_protocols) - verify_platform(supported=supported_platforms) - verify_cache_mode(supported_cache_modes) - verify_aio_mode(supported_aio_modes) + _verify_image_format(supported_fmts, unsupported_fmts) + _verify_protocol(supported_protocols, unsupported_protocols) + _verify_platform(supported=supported_platforms) + _verify_cache_mode(supported_cache_modes) + _verify_aio_mode(supported_aio_modes) debug = '-d' in sys.argv if debug:
Signature · 1 line
2.21.1
PATCH v8 6/11 iotests: drop Python 3.4 compatibility code
In reply to John SnowWe no longer need to accommodate 3.4, drop this code. (Also, the line is over 79 characters, so drop it.)
Touch up the docstring a little bit while we're here.
--- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tests/qemu-iotests/iotests.py +7 -7
@@ -22,6 +22,7 @@ import unittest import sys import struct +from typing import Optional import json import signal import logging
@@ -350,18 +351,17 @@ def _filter(_key, value): return value return filter_qmp(qmsg, _filter) -def log(msg, filters=(), indent=None): - '''Logs either a string message or a JSON serializable message (like QMP). - If indent is provided, JSON serializable messages are pretty-printed.''' +def log(msg, filters=(), indent: Optional[int] = None) -> None: + """ + Logs either a string message or a JSON serializable message (like QMP). + If indent is provided, JSON serializable messages are pretty-printed. + """ for flt in filters: msg = flt(msg) if isinstance(msg, (dict, list)): - # Python < 3.4 needs to know not to add whitespace when pretty-printing: - separators = (', ', ': ') if indent is None else (',', ': ') # Don't sort if it's already sorted do_sort = not isinstance(msg, OrderedDict) - print(json.dumps(msg, sort_keys=do_sort, - indent=indent, separators=separators)) + print(json.dumps(msg, sort_keys=do_sort, indent=indent)) else: print(msg)
Signature · 1 line
2.21.1
PATCH v8 8/11 iotests: add script_initialize
In reply to John SnowLike script_main, but doesn't require a single point of entry. Replace all existing initialization sections with this drop-in replacement.
This brings debug support to all existing script-style iotests.
--- tests/qemu-iotests/149 | 3 +- tests/qemu-iotests/194 | 4 +- tests/qemu-iotests/202 | 4 +- tests/qemu-iotests/203 | 4 +- tests/qemu-iotests/206 | 2 +- tests/qemu-iotests/207 | 6 ++- tests/qemu-iotests/208 | 2 +- tests/qemu-iotests/209 | 2 +- tests/qemu-iotests/210 | 6 ++- tests/qemu-iotests/211 | 6 ++- tests/qemu-iotests/212 | 6 ++- tests/qemu-iotests/213 | 6 ++- tests/qemu-iotests/216 | 4 +- tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/219 | 2 +- tests/qemu-iotests/222 | 7 ++-- tests/qemu-iotests/224 | 4 +- tests/qemu-iotests/228 | 6 ++- tests/qemu-iotests/234 | 4 +- tests/qemu-iotests/235 | 4 +- tests/qemu-iotests/236 | 2 +- tests/qemu-iotests/237 | 2 +- tests/qemu-iotests/238 | 2 + tests/qemu-iotests/242 | 2 +- tests/qemu-iotests/246 | 2 +- tests/qemu-iotests/248 | 2 +- tests/qemu-iotests/254 | 2 +- tests/qemu-iotests/255 | 2 +- tests/qemu-iotests/256 | 2 +- tests/qemu-iotests/258 | 7 ++-- tests/qemu-iotests/260 | 4 +- tests/qemu-iotests/262 | 4 +- tests/qemu-iotests/264 | 4 +- tests/qemu-iotests/277 | 2 + tests/qemu-iotests/280 | 8 ++-- tests/qemu-iotests/283 | 4 +- tests/qemu-iotests/iotests.py | 73 +++++++++++++++++++++++------------ 37 files changed, 128 insertions(+), 80 deletions(-)
tests/qemu-iotests/149 +1 -2
@@ -382,8 +382,7 @@ def test_once(config, qemu_img=False): # Obviously we only work with the luks image format -iotests.verify_image_format(supported_fmts=['luks']) -iotests.verify_platform() +iotests.script_initialize(supported_fmts=['luks']) # We need sudo in order to run cryptsetup to create # dm-crypt devices. This is safe to use on any
tests/qemu-iotests/194 +2 -2
@@ -21,8 +21,8 @@ import iotests -iotests.verify_image_format(supported_fmts=['qcow2', 'qed', 'raw']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2', 'qed', 'raw'], + supported_platforms=['linux']) with iotests.FilePath('source.img') as source_img_path, \ iotests.FilePath('dest.img') as dest_img_path, \
tests/qemu-iotests/202 +2 -2
@@ -24,8 +24,8 @@ import iotests -iotests.verify_image_format(supported_fmts=['qcow2']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2'], + supported_platforms=['linux']) with iotests.FilePath('disk0.img') as disk0_img_path, \ iotests.FilePath('disk1.img') as disk1_img_path, \
tests/qemu-iotests/203 +2 -2
@@ -24,8 +24,8 @@ import iotests -iotests.verify_image_format(supported_fmts=['qcow2']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2'], + supported_platforms=['linux']) with iotests.FilePath('disk0.img') as disk0_img_path, \ iotests.FilePath('disk1.img') as disk1_img_path, \
tests/qemu-iotests/206 +1 -1
@@ -23,7 +23,7 @@ import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) with iotests.FilePath('t.qcow2') as disk_path, \ iotests.FilePath('t.qcow2.base') as backing_path, \
tests/qemu-iotests/207 +4 -2
@@ -24,8 +24,10 @@ import iotests import subprocess import re -iotests.verify_image_format(supported_fmts=['raw']) -iotests.verify_protocol(supported=['ssh']) +iotests.script_initialize( + supported_fmts=['raw'], + supported_protocols=['ssh'], +) def filter_hash(qmsg): def _filter(key, value):
tests/qemu-iotests/208 +1 -1
@@ -22,7 +22,7 @@ import iotests -iotests.verify_image_format(supported_fmts=['generic']) +iotests.script_initialize(supported_fmts=['generic']) with iotests.FilePath('disk.img') as disk_img_path, \ iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \
tests/qemu-iotests/209 +1 -1
@@ -22,7 +22,7 @@ import iotests from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \ file_path -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) disk = file_path('disk') nbd_sock = file_path('nbd-sock', base_dir=iotests.sock_dir)
tests/qemu-iotests/210 +4 -2
@@ -23,8 +23,10 @@ import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['luks']) -iotests.verify_protocol(supported=['file']) +iotests.script_initialize( + supported_fmts=['luks'], + supported_protocols=['file'], +) with iotests.FilePath('t.luks') as disk_path, \ iotests.VM() as vm:
tests/qemu-iotests/211 +4 -2
@@ -23,8 +23,10 @@ import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['vdi']) -iotests.verify_protocol(supported=['file']) +iotests.script_initialize( + supported_fmts=['vdi'], + supported_protocols=['file'], +) def blockdev_create(vm, options): error = vm.blockdev_create(options)
tests/qemu-iotests/212 +4 -2
@@ -23,8 +23,10 @@ import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['parallels']) -iotests.verify_protocol(supported=['file']) +iotests.script_initialize( + supported_fmts=['parallels'], + supported_protocols=['file'], +) with iotests.FilePath('t.parallels') as disk_path, \ iotests.VM() as vm:
tests/qemu-iotests/213 +4 -2
@@ -23,8 +23,10 @@ import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['vhdx']) -iotests.verify_protocol(supported=['file']) +iotests.script_initialize( + supported_fmts=['vhdx'], + supported_protocols=['file'], +) with iotests.FilePath('t.vhdx') as disk_path, \ iotests.VM() as vm:
tests/qemu-iotests/216 +2 -2
@@ -23,8 +23,8 @@ import iotests from iotests import log, qemu_img, qemu_io_silent # Need backing file support -iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk'], + supported_platforms=['linux']) log('') log('=== Copy-on-read across nodes ===')
tests/qemu-iotests/218 +1 -1
@@ -29,7 +29,7 @@ import iotests from iotests import log, qemu_img, qemu_io_silent -iotests.verify_image_format(supported_fmts=['qcow2', 'raw']) +iotests.script_initialize(supported_fmts=['qcow2', 'raw']) # Launches the VM, adds two null-co nodes (source and target), and
tests/qemu-iotests/219 +1 -1
@@ -21,7 +21,7 @@ import iotests -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) img_size = 4 * 1024 * 1024
tests/qemu-iotests/222 +4 -3
@@ -24,9 +24,10 @@ import iotests from iotests import log, qemu_img, qemu_io, qemu_io_silent -iotests.verify_platform(['linux']) -iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', - 'vhdx', 'raw']) +iotests.script_initialize( + supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', 'vhdx', 'raw'], + supported_platforms=['linux'], +) patterns = [("0x5d", "0", "64k"), ("0xd5", "1M", "64k"),
tests/qemu-iotests/224 +2 -2
@@ -26,8 +26,8 @@ from iotests import log, qemu_img, qemu_io_silent, filter_qmp_testfiles, \ import json # Need backing file support (for arbitrary backing formats) -iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2', 'qcow', 'qed'], + supported_platforms=['linux']) # There are two variations of this test:
tests/qemu-iotests/228 +4 -2
@@ -25,8 +25,10 @@ from iotests import log, qemu_img, filter_testfiles, filter_imgfmt, \ filter_qmp_testfiles, filter_qmp_imgfmt # Need backing file and change-backing-file support -iotests.verify_image_format(supported_fmts=['qcow2', 'qed']) -iotests.verify_platform(['linux']) +iotests.script_initialize( + supported_fmts=['qcow2', 'qed'], + supported_platforms=['linux'], +) def log_node_info(node):
tests/qemu-iotests/234 +2 -2
@@ -23,8 +23,8 @@ import iotests import os -iotests.verify_image_format(supported_fmts=['qcow2']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2'], + supported_platforms=['linux']) with iotests.FilePath('img') as img_path, \ iotests.FilePath('backing') as backing_path, \
tests/qemu-iotests/235 +2 -2
@@ -27,6 +27,8 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu.machine import QEMUMachine +iotests.script_initialize(supported_fmts=['qcow2']) + # Note: # This test was added to check that mirror dead-lock was fixed (see previous # commit before this test addition).
@@ -40,8 +42,6 @@ from qemu.machine import QEMUMachine size = 1 * 1024 * 1024 * 1024 -iotests.verify_image_format(supported_fmts=['qcow2']) - disk = file_path('disk') # prepare source image
tests/qemu-iotests/236 +1 -1
@@ -22,7 +22,7 @@ import iotests from iotests import log -iotests.verify_image_format(supported_fmts=['generic']) +iotests.script_initialize(supported_fmts=['generic']) size = 64 * 1024 * 1024 granularity = 64 * 1024
tests/qemu-iotests/237 +1 -1
@@ -24,7 +24,7 @@ import math import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['vmdk']) +iotests.script_initialize(supported_fmts=['vmdk']) with iotests.FilePath('t.vmdk') as disk_path, \ iotests.FilePath('t.vmdk.1') as extent1_path, \
tests/qemu-iotests/238 +2 -0
@@ -23,6 +23,8 @@ import os import iotests from iotests import log +iotests.script_initialize() + virtio_scsi_device = iotests.get_virtio_scsi_device() vm = iotests.VM()
tests/qemu-iotests/242 +1 -1
@@ -24,7 +24,7 @@ import struct from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \ file_path, img_info_log, log, filter_qemu_io -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) disk = file_path('disk') chunk = 256 * 1024
tests/qemu-iotests/246 +1 -1
@@ -22,7 +22,7 @@ import iotests from iotests import log -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) size = 64 * 1024 * 1024 * 1024 gran_small = 32 * 1024 gran_large = 128 * 1024
tests/qemu-iotests/248 +1 -1
@@ -21,7 +21,7 @@ import iotests from iotests import qemu_img_create, qemu_io, file_path, filter_qmp_testfiles -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) source, target = file_path('source', 'target') size = 5 * 1024 * 1024
tests/qemu-iotests/254 +1 -1
@@ -21,7 +21,7 @@ import iotests from iotests import qemu_img_create, file_path, log -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) disk, top = file_path('disk', 'top') size = 1024 * 1024
tests/qemu-iotests/255 +1 -1
@@ -23,7 +23,7 @@ import iotests from iotests import imgfmt -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) iotests.log('Finishing a commit job with background reads') iotests.log('============================================')
tests/qemu-iotests/256 +1 -1
@@ -23,7 +23,7 @@ import os import iotests from iotests import log -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2']) size = 64 * 1024 * 1024 with iotests.FilePath('img0') as img0_path, \
tests/qemu-iotests/258 +4 -3
@@ -24,9 +24,10 @@ from iotests import log, qemu_img, qemu_io_silent, \ filter_qmp_testfiles, filter_qmp_imgfmt # Need backing file and change-backing-file support -iotests.verify_image_format(supported_fmts=['qcow2', 'qed']) -iotests.verify_platform(['linux']) - +iotests.script_initialize( + supported_fmts=['qcow2', 'qed'], + supported_platforms=['linux'], +) # Returns a node for blockdev-add def node(node_name, path, backing=None, fmt=None, throttle=None):
tests/qemu-iotests/260 +3 -1
@@ -21,7 +21,9 @@ import iotests from iotests import qemu_img_create, file_path, log, filter_qmp_event -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize( + supported_fmts=['qcow2'] +) base, top = file_path('base', 'top') size = 64 * 1024 * 3
tests/qemu-iotests/262 +2 -2
@@ -23,8 +23,8 @@ import iotests import os -iotests.verify_image_format(supported_fmts=['qcow2']) -iotests.verify_platform(['linux']) +iotests.script_initialize(supported_fmts=['qcow2'], + supported_platforms=['linux']) with iotests.FilePath('img') as img_path, \ iotests.FilePath('mig_fifo') as fifo, \
tests/qemu-iotests/264 +3 -1
@@ -24,7 +24,9 @@ import iotests from iotests import qemu_img_create, qemu_io_silent_check, file_path, \ qemu_nbd_popen, log -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize( + supported_fmts=['qcow2'], +) disk_a, disk_b, nbd_sock = file_path('disk_a', 'disk_b', 'nbd-sock') nbd_uri = 'nbd+unix:///?socket=' + nbd_sock
tests/qemu-iotests/277 +2 -0
@@ -23,6 +23,8 @@ import subprocess import iotests from iotests import file_path, log +iotests.script_initialize() + nbd_sock, conf_file = file_path('nbd-sock', 'nbd-fault-injector.conf')
tests/qemu-iotests/280 +5 -3
@@ -22,9 +22,11 @@ import iotests import os -iotests.verify_image_format(supported_fmts=['qcow2']) -iotests.verify_protocol(supported=['file']) -iotests.verify_platform(['linux']) +iotests.script_initialize( + supported_fmts=['qcow2'], + supported_protocols=['file'], + supported_platforms=['linux'], +) with iotests.FilePath('base') as base_path , \ iotests.FilePath('top') as top_path, \
tests/qemu-iotests/283 +3 -1
@@ -21,7 +21,9 @@ import iotests # The test is unrelated to formats, restrict it to qcow2 to avoid extra runs -iotests.verify_image_format(supported_fmts=['qcow2']) +iotests.script_initialize( + supported_fmts=['qcow2'], +) size = 1024 * 1024
tests/qemu-iotests/iotests.py +48 -25
126 lines not on this page · raw message
Signature · 1 line
2.21.1
PATCH v8 7/11 iotests: limit line length to 79 chars
In reply to John Snow79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
--- tests/qemu-iotests/iotests.py | 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23 deletions(-)
tests/qemu-iotests/iotests.py +42 -22
@@ -77,9 +77,11 @@ def qemu_img(*args): '''Run qemu-img and return the exit code''' devnull = open('/dev/null', 'r+') - exitcode = subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull) + exitcode = subprocess.call(qemu_img_args + list(args), + stdin=devnull, stdout=devnull) if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode def ordered_qmp(qmsg, conv_keys=True):
@@ -118,7 +120,8 @@ def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args)) if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode def qemu_img_pipe(*args):
@@ -129,7 +132,8 @@ def qemu_img_pipe(*args): universal_newlines=True) exitcode = subp.wait() if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args)))) return subp.communicate()[0] def qemu_img_log(*args):
@@ -159,7 +163,8 @@ def qemu_io(*args): universal_newlines=True) exitcode = subp.wait() if exitcode < 0: - sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args))) + sys.stderr.write('qemu-io received signal %i: %s\n' + % (-exitcode, ' '.join(args))) return subp.communicate()[0] def qemu_io_log(*args):
@@ -281,10 +286,13 @@ def filter_test_dir(msg): def filter_win32(msg): return win32_re.sub("", msg) -qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* \([0-9\/.inf]* [EPTGMKiBbytes]*\/sec and [0-9\/.inf]* ops\/sec\)") +qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* " + r"\([0-9\/.inf]* [EPTGMKiBbytes]*\/sec " + r"and [0-9\/.inf]* ops\/sec\)") def filter_qemu_io(msg): msg = filter_win32(msg) - return qemu_io_re.sub("X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)", msg) + return qemu_io_re.sub("X ops; XX:XX:XX.X " + "(XXX YYY/sec and XXX ops/sec)", msg) chown_re = re.compile(r"chown [0-9]+:[0-9]+") def filter_chown(msg):
@@ -336,7 +344,9 @@ def filter_img_info(output, filename): line = line.replace(filename, 'TEST_IMG') \ .replace(imgfmt, 'IMGFMT') line = re.sub('iters: [0-9]+', 'iters: XXX', line) - line = re.sub('uuid: [-a-f0-9]+', 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', line) + line = re.sub('uuid: [-a-f0-9]+', + 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', + line) line = re.sub('cid: [0-9]+', 'cid: XXXXXXXXXX', line) lines.append(line) return '\n'.join(lines)
@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command', - command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive)) def resume_drive(self, drive): self.qmp('human-monitor-command', - command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) + command_line='qemu-io %s "remove_break bp_%s"' + % (drive, drive)) def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command'''
@@ -793,16 +805,18 @@ def dictpath(self, d, path): idx = int(idx) if not isinstance(d, dict) or component not in d: - self.fail('failed path traversal for "%s" in "%s"' % (path, str(d))) + self.fail(f'failed path traversal for "{path}" in "{d}"') d = d[component] if m: if not isinstance(d, list): - self.fail('path component "%s" in "%s" is not a list in "%s"' % (component, path, str(d))) + self.fail(f'path component "{component}" in "{path}" ' + f'is not a list in "{d}"') try: d = d[idx] except IndexError: - self.fail('invalid index "%s" in path "%s" in "%s"' % (idx, path, str(d))) + self.fail(f'invalid index "{idx}" in path "{path}" ' + f'in "{d}"') return d def assert_qmp_absent(self, d, path):
@@ -853,10 +867,13 @@ def assert_json_filename_equal(self, json_filename, reference): '''Asserts that the given filename is a json: filename and that its content is equal to the given reference object''' self.assertEqual(json_filename[:5], 'json:') - self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filename[5:])), - self.vm.flatten_qmp_object(reference)) + self.assertEqual( + self.vm.flatten_qmp_object(json.loads(json_filename[5:])), + self.vm.flatten_qmp_object(reference) + ) - def cancel_and_wait(self, drive='drive0', force=False, resume=False, wait=60.0): + def cancel_and_wait(self, drive='drive0', force=False, + resume=False, wait=60.0): '''Cancel a block job and wait for it to finish, returning the event''' result = self.vm.qmp('block-job-cancel', device=drive, force=force) self.assert_qmp(result, 'return', {})
@@ -880,8 +897,8 @@ def cancel_and_wait(self, drive='drive0', force=False, resume=False, wait=60.0): self.assert_no_active_block_jobs() return result - def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0, - error=None): + def wait_until_completed(self, drive='drive0', check_offset=True, + wait=60.0, error=None): '''Wait for a block job to finish, returning the event''' while True: for event in self.vm.get_qmp_events(wait=wait):
@@ -1020,8 +1037,11 @@ def verify_quorum(): notrun('quorum support missing') def qemu_pipe(*args): - '''Run qemu with an option to print something and exit (e.g. a help option), - and return its output''' + """ + Run qemu with an option to print something and exit (e.g. a help option). + + :return: QEMU's stdout output. + """ args = [qemu_prog] + qemu_opts + list(args) subp = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
@@ -1059,8 +1079,8 @@ def func_wrapper(test_case: QMPTestCase, *args, **kwargs): usf_list = list(set(fmts) - set(supported_formats(read_only))) if usf_list: - test_case.case_skip('{}: formats {} are not whitelisted'.format( - test_case, usf_list)) + msg = f'{test_case}: formats {usf_list} are not whitelisted' + test_case.case_skip(msg) return None else: return func(test_case, *args, **kwargs)
tests/qemu-iotests/pylintrc +5 -1
@@ -19,4 +19,8 @@ disable=invalid-name, too-many-public-methods, # These are temporary, and should be removed: missing-docstring, - line-too-long, + +[FORMAT] + +# Maximum number of characters on a single line. +max-line-length=79
Signature · 1 line
2.21.1
Re: [PATCH v8 03/11] iotests: ignore import warnings from pylint
In reply to John SnowQuoted 17 lines · John Snow
The right way to solve this is to come up with a virtual environment infrastructure that sets all the paths correctly, and/or to create installable python modules that can be imported normally.
That's hard, so just silence this error for now.
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 1 + 1 file changed, 1 insertion(+)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py line 35
Quoted 7 lines
tests/qemu-iotests/iotests.py +1
-0@@ -29,6 +29,7 @@ import io from collections import OrderedDict +# pylint: disable=import-error, wrong-import-position sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qtest
Re: [PATCH v8 07/11] iotests: limit line length to 79 chars
In reply to John SnowQuoted 181 lines · John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23 deletions(-)tests/qemu-iotests/iotests.py +42
-22@@ -77,9 +77,11 @@ def qemu_img(*args): '''Run qemu-img and return the exit code''' devnull = open('/dev/null', 'r+') - exitcode = subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull) + exitcode = subprocess.call(qemu_img_args + list(args), + stdin=devnull, stdout=devnull) if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode def ordered_qmp(qmsg, conv_keys=True):@@ -118,7 +120,8 @@ def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args)) if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode def qemu_img_pipe(*args):@@ -129,7 +132,8 @@ def qemu_img_pipe(*args): universal_newlines=True) exitcode = subp.wait() if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args)))) return subp.communicate()[0] def qemu_img_log(*args):@@ -159,7 +163,8 @@ def qemu_io(*args): universal_newlines=True) exitcode = subp.wait() if exitcode < 0: - sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args))) + sys.stderr.write('qemu-io received signal %i: %s\n' + % (-exitcode, ' '.join(args))) return subp.communicate()[0] def qemu_io_log(*args):@@ -281,10 +286,13 @@ def filter_test_dir(msg): def filter_win32(msg): return win32_re.sub("", msg) -qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* \([0-9\/.inf]* [EPTGMKiBbytes]*\/sec and [0-9\/.inf]* ops\/sec\)") +qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* " + r"\([0-9\/.inf]* [EPTGMKiBbytes]*\/sec " + r"and [0-9\/.inf]* ops\/sec\)") def filter_qemu_io(msg): msg = filter_win32(msg) - return qemu_io_re.sub("X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)", msg) + return qemu_io_re.sub("X ops; XX:XX:XX.X " + "(XXX YYY/sec and XXX ops/sec)", msg) chown_re = re.compile(r"chown [0-9]+:[0-9]+") def filter_chown(msg):@@ -336,7 +344,9 @@ def filter_img_info(output, filename): line = line.replace(filename, 'TEST_IMG') \ .replace(imgfmt, 'IMGFMT') line = re.sub('iters: [0-9]+', 'iters: XXX', line) - line = re.sub('uuid: [-a-f0-9]+', 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', line) + line = re.sub('uuid: [-a-f0-9]+', + 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', + line) line = re.sub('cid: [0-9]+', 'cid: XXXXXXXXXX', line) lines.append(line) return '\n'.join(lines)@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command', - command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive)) def resume_drive(self, drive): self.qmp('human-monitor-command', - command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) + command_line='qemu-io %s "remove_break bp_%s"' + % (drive, drive)) def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command'''@@ -793,16 +805,18 @@ def dictpath(self, d, path): idx = int(idx) if not isinstance(d, dict) or component not in d: - self.fail('failed path traversal for "%s" in "%s"' % (path, str(d))) + self.fail(f'failed path traversal for "{path}" in "{d}"') d = d[component] if m: if not isinstance(d, list): - self.fail('path component "%s" in "%s" is not a list in "%s"' % (component, path, str(d))) + self.fail(f'path component "{component}" in "{path}" ' + f'is not a list in "{d}"') try: d = d[idx] except IndexError: - self.fail('invalid index "%s" in path "%s" in "%s"' % (idx, path, str(d))) + self.fail(f'invalid index "{idx}" in path "{path}" ' + f'in "{d}"') return d def assert_qmp_absent(self, d, path):@@ -853,10 +867,13 @@ def assert_json_filename_equal(self, json_filename, reference): '''Asserts that the given filename is a json: filename and that its content is equal to the given reference object''' self.assertEqual(json_filename[:5], 'json:') - self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filename[5:])), - self.vm.flatten_qmp_object(reference)) + self.assertEqual( + self.vm.flatten_qmp_object(json.loads(json_filename[5:])), + self.vm.flatten_qmp_object(reference) + ) - def cancel_and_wait(self, drive='drive0', force=False, resume=False, wait=60.0): + def cancel_and_wait(self, drive='drive0', force=False, + resume=False, wait=60.0): '''Cancel a block job and wait for it to finish, returning the event''' result = self.vm.qmp('block-job-cancel', device=drive, force=force) self.assert_qmp(result, 'return', {})@@ -880,8 +897,8 @@ def cancel_and_wait(self, drive='drive0', force=False, resume=False, wait=60.0): self.assert_no_active_block_jobs() return result - def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0, - error=None): + def wait_until_completed(self, drive='drive0', check_offset=True, + wait=60.0, error=None): '''Wait for a block job to finish, returning the event''' while True: for event in self.vm.get_qmp_events(wait=wait):@@ -1020,8 +1037,11 @@ def verify_quorum(): notrun('quorum support missing') def qemu_pipe(*args): - '''Run qemu with an option to print something and exit (e.g. a help option), - and return its output''' + """ + Run qemu with an option to print something and exit (e.g. a help option). + + :return: QEMU's stdout output. + """ args = [qemu_prog] + qemu_opts + list(args) subp = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,@@ -1059,8 +1079,8 @@ def func_wrapper(test_case: QMPTestCase, *args, **kwargs): usf_list = list(set(fmts) - set(supported_formats(read_only))) if usf_list: - test_case.case_skip('{}: formats {} are not whitelisted'.format( - test_case, usf_list)) + msg = f'{test_case}: formats {usf_list} are not whitelisted' + test_case.case_skip(msg) return None else: return func(test_case, *args, **kwargs)tests/qemu-iotests/pylintrc +0
-0
tests/qemu-iotests/pylintrc disable=invalid-name, · line 26
Quoted 9 lines
tests/qemu-iotests/pylintrc +5
-1@@ -19,4 +19,8 @@ disable=invalid-name, too-many-public-methods, # These are temporary, and should be removed: missing-docstring, - line-too-long, + +[FORMAT] + +# Maximum number of characters on a single line. +max-line-length=79
Re: [PATCH v8 07/11] iotests: limit line length to 79 chars
Quoted 205 lines · Philippe Mathieu-Daudé
Quoted 204 lines · John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
Signed-off-by John Snow <[email protected]>---
tests/qemu-iotests/iotests.py| 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++-2 files changed, 47 insertions(+), 23 deletions(-)tests/qemu-iotests/iotests.py +0
-0@@ -77,9 +77,11 @@def
qemu_img(*args):
'''Run qemu-img and return the exit code''' devnull =open('/dev/null', 'r+')
- exitcode =subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull)
+ exitcode = subprocess.call(qemu_img_args + list(args), + stdin=devnull, stdout=devnull) if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
+ sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args))))
return exitcode
defordered_qmp(qmsg, conv_keys=True):tests/qemu-iotests/iotests.py +0
-0@@ -118,7 +120,8 @@ def qemu_img_verbose(*args):'''Run qemu-img without suppressing its output and return the exit code'''
exitcode =subprocess.call(qemu_img_args + list(args))if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
+ sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args))))
return exitcode
defqemu_img_pipe(*args):tests/qemu-iotests/iotests.py +0
-0@@ -129,7 +132,8 @@ def qemu_img_pipe(*args): universal_newlines=True) exitcode = subp.wait()if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args))))
+ sys.stderr.write('qemu-img received signal %i: %s\n' + % (-exitcode, ' '.join(qemu_img_args + list(args))))
return subp.communicate()[0]
defqemu_img_log(*args):tests/qemu-iotests/iotests.py +0
-0@@ -159,7 +163,8 @@ def qemu_io(*args): universal_newlines=True) exitcode = subp.wait()if exitcode < 0:
-sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args)))
+ sys.stderr.write('qemu-io received signal %i: %s\n' + % (-exitcode, ' '.join(args))) return subp.communicate()[0]
defqemu_io_log(*args):tests/qemu-iotests/iotests.py +0
-0@@ -281,10 +286,13 @@ def filter_test_dir(msg):def
filter_win32(msg):
returnwin32_re.sub("", msg)
-qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* \([0-9\/.inf]* [EPTGMKiBbytes]*\/sec and [0-9\/.inf]* ops\/sec\)") +qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* "
+ r"\([0-9\/.inf]* [EPTGMKiBbytes]*\/sec " + r"and [0-9\/.inf]* ops\/sec\)") deffilter_qemu_io(msg):
msg =filter_win32(msg)
- returnqemu_io_re.sub("X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)", msg)
+ return qemu_io_re.sub("X ops; XX:XX:XX.X "
+ "(XXX YYY/sec and XXX ops/sec)", msg)chown_re=re.compile(r"chown [0-9]+:[0-9]+")
deffilter_chown(msg):tests/qemu-iotests/iotests.py +3
-1@@ -336,7 +344,9 @@ def filter_img_info(output, filename): line = line.replace(filename, 'TEST_IMG') \ .replace(imgfmt, 'IMGFMT') line = re.sub('iters: [0-9]+', 'iters: XXX', line) - line = re.sub('uuid: [-a-f0-9]+', 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', line) + line = re.sub('uuid: [-a-f0-9]+', + 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', + line) line = re.sub('cid: [0-9]+', 'cid: XXXXXXXXXX', line) lines.append(line)return '\n'.join(lines)
tests/qemu-iotests/iotests.py +0
-0@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio")return
self.qmp('human-monitor-command',
- command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive))
+ command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive))
defresume_drive(self, drive):
self.qmp('human-monitor-command',
- command_line='qemu-io %s "remove_breakbp_%s"' % (drive, drive))
+ command_line='qemu-io %s "remove_breakbp_%s"' + % (drive, drive))
defhmp_qemu_io(self, drive, cmd):
'''Write to a given drive using an HMP command'''tests/qemu-iotests/iotests.py +0
-0@@ -793,16 +805,18 @@ def dictpath(self, d, path): idx = int(idx)if not
isinstance(d, dict)or component not in d:
-self.fail('failed path traversal for "%s" in "%s"' % (path, str(d)))
+self.fail(f'failed path traversal for "{path}" in "{d}"')
d = d[component]
if m:
if notisinstance(d, list):
-self.fail('path component "%s" in "%s" is not a list in "%s"' % (component, path, str(d)))
+ self.fail(f'path component "{component}" in "{path}" '
+ f'is not a list in "{d}"') try:
d = d[idx]
except IndexError:
-self.fail('invalid index "%s" in path "%s" in "%s"' % (idx, path, str(d)))
+ self.fail(f'invalid index "{idx}" in path "{path}" ' + f'in "{d}"')
return d
defassert_qmp_absent(self, d, path):tests/qemu-iotests/iotests.py +0
-0@@ -853,10 +867,13 @@ def assert_json_filename_equal(self,
json_filename, reference):
'''Asserts that the given filename is a json: filename and that its
content is equal to the given reference object'''self.assertEqual(json_filename[:5], 'json:')
- self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filename[5:])),-
self.vm.flatten_qmp_object(reference)) + self.assertEqual(
+ self.vm.flatten_qmp_object(json.loads(json_filename[5:])), +self.vm.flatten_qmp_object(reference)
+ )
- defcancel_and_wait(self, drive='drive0', force=False, resume=False, wait=60.0):
+ def cancel_and_wait(self, drive='drive0', force=False, + resume=False, wait=60.0):
'''Cancel a block job and wait for it to finish, returning the event'''
result =self.vm.qmp('block-job-cancel', device=drive, force=force)
self.assert_qmp(result, 'return', {})tests/qemu-iotests/iotests.py +0
-0@@ -880,8 +897,8 @@ def cancel_and_wait(self, drive='drive0',force=False, resume=False, wait=60.0):
self.assert_no_active_block_jobs()
return result
- def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0,
- error=None):
+ def wait_until_completed(self, drive='drive0', check_offset=True, + wait=60.0, error=None):
'''Wait for a block job to finish, returning the event''' while True:
for event inself.vm.get_qmp_events(wait=wait):tests/qemu-iotests/iotests.py +0
-0@@ -1020,8 +1037,11 @@ def verify_quorum(): notrun('quorum support missing')def
qemu_pipe(*args):
- '''Run qemu with an option to print something and exit (e.g. a help option),
- and return its output'''
+ """
+ Run qemu with an option to print something and exit (e.g. a help option).
+
+ :return: QEMU's stdout output.
+ """
args = [qemu_prog] +qemu_opts+ list(args)
subp = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,tests/qemu-iotests/iotests.py +0
-0@@ -1059,8 +1079,8 @@ def func_wrapper(test_case: QMPTestCase, *args, **kwargs):
usf_list=list(set(fmts) - set(supported_formats(read_only)))
ifusf_list:
- test_case.case_skip('{}: formats {} are not whitelisted'.format(
-test_case,usf_list))
+ msg = f'{test_case}: formats {usf_list} are not whitelisted'
+test_case.case_skip(msg)
return None
else:
returnfunc(test_case, *args, **kwargs)tests/qemu-iotests/pylintrc +0
-1@@ -19,4 +19,8 @@ disable=invalid-name, too-many-public-methods, # These are temporary, and should be removed: missing-docstring, - line-too-long,
Quoted 9 lines
tests/qemu-iotests/pylintrc disable=invalid-name, · line 26
Quoted 6 lines
tests/qemu-iotests/pylintrc +5
-0+ +[FORMAT] + +# Maximum number of characters on a single line. +max-line-length=79Reviewed-by Philippe Mathieu-Daudé <[email protected]>
Thanks :)
Re: [PATCH v8 01/11] iotests: do a light delinting
In reply to John SnowQuoted 7 lines · John Snow
This doesn't fix everything in here, but it does help clean up the pylint report considerably.
This should be 100% style changes only; the intent is to make pylint more useful by working on establishing a baseline for iotests that we can gate against in the future.
Quoted 5 lines
Signed-off-by John Snow <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>--- tests/qemu-iotests/iotests.py | 83 ++++++++++++++++++----------------- 1 file changed, 43 insertions(+), 40 deletions(-)
Re: [PATCH v8 03/11] iotests: ignore import warnings from pylint
In reply to John SnowQuoted 10 lines · John Snow
The right way to solve this is to come up with a virtual environment infrastructure that sets all the paths correctly, and/or to create installable python modules that can be imported normally.
That's hard, so just silence this error for now.
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 1 + 1 file changed, 1 insertion(+)
Re: [PATCH v8 04/11] iotests: replace mutable list default args
In reply to John SnowQuoted 11 lines · John Snow
It's bad hygiene: if we modify this list, it will be modified across all invocations.
(Remaining bad usages are fixed in a subsequent patch which changes the function signature anyway.)
Signed-off-by John Snow <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>--- tests/qemu-iotests/iotests.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
Re: [PATCH v8 05/11] iotests: add pylintrc file
In reply to John SnowQuoted 8 lines · John Snow
This allows others to get repeatable results with pylint. If you run
pylint iotests.py, you should see a 100% pass.Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/pylintrc | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 tests/qemu-iotests/pylintrc
Re: [PATCH v8 06/11] iotests: drop Python 3.4 compatibility code
In reply to John SnowQuoted 1 line · John Snow
We no longer need to accommodate 3.4, drop this code.
Pre-3.4, actually.
Quoted 23 lines
(Also, the line is over 79 characters, so drop it.)
Touch up the docstring a little bit while we're here.
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)tests/qemu-iotests/iotests.py +1
-0@@ -22,6 +22,7 @@ import unittest import sys import struct +from typing import Optional import json import signal import logging@@ -350,18 +351,17 @@ def _filter(_key, value): return value
tests/qemu-iotests/iotests.py def _filter(_key, value): · line 358
Quoted 10 lines
tests/qemu-iotests/iotests.py +5
-3return filter_qmp(qmsg, _filter) -def log(msg, filters=(), indent=None): - '''Logs either a string message or a JSON serializable message (like QMP). - If indent is provided, JSON serializable messages are pretty-printed.''' +def log(msg, filters=(), indent: Optional[int] = None) -> None: + """ + Logs either a string message or a JSON serializable message (like QMP). + If indent is provided, JSON serializable messages are pretty-printed. + """
I feel like I should complain about this unrelated (I think?) change, but I won’t.
Quoted 14 lines
tests/qemu-iotests/iotests.py +1
-4for flt in filters: msg = flt(msg) if isinstance(msg, (dict, list)): - # Python < 3.4 needs to know not to add whitespace when pretty-printing: - separators = (', ', ': ') if indent is None else (',', ': ') # Don't sort if it's already sorted do_sort = not isinstance(msg, OrderedDict) - print(json.dumps(msg, sort_keys=do_sort, - indent=indent, separators=separators)) + print(json.dumps(msg, sort_keys=do_sort, indent=indent)) else: print(msg)
Re: [PATCH v8 07/11] iotests: limit line length to 79 chars
In reply to John SnowQuoted 2 lines · John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
Also for my very GUI-y diff program (kompare).
Quoted 10 lines
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23 deletions(-)tests/qemu-iotests/iotests.py +0
-0
[...]
tests/qemu-iotests/iotests.py def pause_drive(self, drive, event=None): · line 543
Quoted 7 lines
tests/qemu-iotests/iotests.py +2
-1@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command', - command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive))
Can we put this value in a variable instead? I don’t like the % aligning with the parameter name instead of the string value. (I also don’t like how there are no spaces around the assignment =, but around the %, even though the % binds more strongly.)
Quoted 10 lines
tests/qemu-iotests/iotests.py +2
-1def resume_drive(self, drive): self.qmp('human-monitor-command', - command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) + command_line='qemu-io %s "remove_break bp_%s"' + % (drive, drive)) def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command'''
tests/qemu-iotests/iotests.py def dictpath(self, d, path): · line 808
Quoted 5 lines
tests/qemu-iotests/iotests.py +1
-1@@ -793,16 +805,18 @@ def dictpath(self, d, path): idx = int(idx) if not isinstance(d, dict) or component not in d: - self.fail('failed path traversal for "%s" in "%s"' % (path, str(d))) + self.fail(f'failed path traversal for "{path}" in "{d}"')
Do we require 3.6 so that f-strings are guaranteed to work? (I thought we didn’t. I’d be happy to use them.)
Max
Re: [PATCH v8 07/11] iotests: limit line length to 79 chars
Quoted 44 lines · Max Reitz
Quoted 2 lines · John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
Also for my very GUI-y diff program (kompare).
Quoted 10 lines
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23 deletions(-)tests/qemu-iotests/iotests.py +0
-0[...]
tests/qemu-iotests/iotests.py def pause_drive(self, drive, event=None): · line 543
Quoted 7 lines
tests/qemu-iotests/iotests.py +2
-1@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command', - command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive))Can we put this value in a variable instead? I don’t like the % aligning with the parameter name instead of the string value. (I also don’t like how there are no spaces around the assignment =, but around the %, even though the % binds more strongly.)
Quoted 12 lines
tests/qemu-iotests/iotests.py +2
-1def resume_drive(self, drive): self.qmp('human-monitor-command', - command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) + command_line='qemu-io %s "remove_break bp_%s"' + % (drive, drive)) def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command'''@@ -793,16 +805,18 @@ def dictpath(self, d, path): idx = int(idx)
Quoted 6 lines
tests/qemu-iotests/iotests.py def dictpath(self, d, path): · line 808
Quoted 3 lines
tests/qemu-iotests/iotests.py +1
-1if not isinstance(d, dict) or component not in d: - self.fail('failed path traversal for "%s" in "%s"' % (path, str(d))) + self.fail(f'failed path traversal for "{path}" in "{d}"')Do we require 3.6 so that f-strings are guaranteed to work? (I thought we didn’t. I’d be happy to use them.)
Oh. Of course we do. It says so in this file that this whole series is about.
Max
Re: [PATCH v8 11/11] iotests: use python logging for iotests.log()
In reply to John SnowQuoted 9 lines · John Snow
We can turn logging on/off globally instead of per-function.
Remove
use_logfromrun_job, and use python logging to turn on diffable output when we run through a script entry point.iotest 245 changes output order due to buffering reasons.
An extended note on python logging:
Thanks!
Quoted 27 lines
A NullHandler is added to
qemu.ioteststo stop output from being generated if this code is used as a library without configuring logging. A NullHandler is only needed at the root, so a duplicate handler is not needed forqemu.iotests.diff_io.When logging is not configured, messages at the 'WARNING' levels or above are printed with default settings. The NullHandler stops this from occurring, which is considered good hygiene for code used as a library.
See https://docs.python.org/3/howto/logging.html#library-config
When logging is actually enabled (always at the behest of an explicit call by a client script), a root logger is implicitly created at the root, which allows messages to propagate upwards and be handled/emitted from the root logger with default settings.
When we want iotest logging, we attach a handler to the qemu.iotests.diff_io logger and disable propagation to avoid possible double-printing.
For more information on python logging infrastructure, I highly recommend downloading the pip package
logging_tree, which provides convenient visualizations of the hierarchical logging configuration under different circumstances.See https://pypi.org/project/logging_tree/ for more information.
Quoted 8 lines
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/030 | 4 +-- tests/qemu-iotests/155 | 2 +- tests/qemu-iotests/245 | 1 + tests/qemu-iotests/245.out | 24 ++++++++-------- tests/qemu-iotests/iotests.py | 53 ++++++++++++++++++++--------------- 5 files changed, 46 insertions(+), 38 deletions(-)
Re: [PATCH v8 06/11] iotests: drop Python 3.4 compatibility code
In reply to Max ReitzQuoted 30 lines · Max Reitz
Quoted 1 line · John Snow
We no longer need to accommodate 3.4, drop this code.
Pre-3.4, actually.
Quoted 25 lines
(Also, the line is over 79 characters, so drop it.)
Touch up the docstring a little bit while we're here.
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)tests/qemu-iotests/iotests.py +1
-0@@ -22,6 +22,7 @@ import unittest import sys import struct +from typing import Optional import json import signal import logging@@ -350,18 +351,17 @@ def _filter(_key, value): return value return filter_qmp(qmsg, _filter)
Quoted 12 lines
tests/qemu-iotests/iotests.py def _filter(_key, value): · line 358
Quoted 8 lines
tests/qemu-iotests/iotests.py +5
-3-def log(msg, filters=(), indent=None): - '''Logs either a string message or a JSON serializable message (like QMP). - If indent is provided, JSON serializable messages are pretty-printed.''' +def log(msg, filters=(), indent: Optional[int] = None) -> None: + """ + Logs either a string message or a JSON serializable message (like QMP). + If indent is provided, JSON serializable messages are pretty-printed. + """I feel like I should complain about this unrelated (I think?) change, but I won’t.
It just seemed like so little to have in its own little patch, and I wasn't prepared to fix the docstrings in the whole file ...
... It will be a future patch that tidies up this whole file and drops the missing-docstring ignore from pylintrc.
Quoted 2 lines
Reviewed-by Max Reitz <[email protected]>
Thanks.
Quoted 16 lines
Quoted 14 lines
tests/qemu-iotests/iotests.py +1
-4for flt in filters: msg = flt(msg) if isinstance(msg, (dict, list)): - # Python < 3.4 needs to know not to add whitespace when pretty-printing: - separators = (', ', ': ') if indent is None else (',', ': ') # Don't sort if it's already sorted do_sort = not isinstance(msg, OrderedDict) - print(json.dumps(msg, sort_keys=do_sort, - indent=indent, separators=separators)) + print(json.dumps(msg, sort_keys=do_sort, indent=indent)) else: print(msg)
Re: [PATCH v8 07/11] iotests: limit line length to 79 chars
In reply to Max ReitzQuoted 20 lines · Max Reitz
Quoted 19 lines · Max Reitz
Quoted 2 lines · John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
Also for my very GUI-y diff program (kompare).
Quoted 10 lines
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23 deletions(-)tests/qemu-iotests/iotests.py +0
-0[...]
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command',Quoted 10 lines
tests/qemu-iotests/iotests.py def pause_drive(self, drive, event=None): · line 543
Quoted 3 lines
tests/qemu-iotests/iotests.py +2
-1- command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive))Can we put this value in a variable instead? I don’t like the % aligning with the parameter name instead of the string value. (I also don’t like how there are no spaces around the assignment =, but around the %, even though the % binds more strongly.)
Quoted 1 line
I think I had another patch somewhere that added an HMP helper that fixes this issue for this particular instance.
I can send that separately as a follow-up. I think otherwise, unfortunately, "we" "decided" that this indent style is "best".
So I think that this patch is "correct".
(All of the other options for indent styles seemed to be worse visually, or actively go against PEP8. While PEP8 is not a bible, every conscious choice to disregard it generally means you will be fighting a CQA tool at some other point in time. I have therefore adopted a "When in Rome" policy to reduce friction wherever possible with pylint, flake8, mypy, pycharm, and so on.)
((I would prefer we begin to deprecate uses of % and begin using .format() and f-strings wherever possible to help alleviate this kind of syntax, too -- but I must insist that's for another series.))
Quoted 18 lines
Quoted 11 lines
Quoted 11 lines
def
resume_drive(self, drive):
self.qmp('human-monitor-command',tests/qemu-iotests/iotests.py +2
-1- command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive)) + command_line='qemu-io %s "remove_break bp_%s"' + % (drive, drive)) def hmp_qemu_io(self, drive, cmd): '''Write to a given drive using an HMP command'''@@ -793,16 +805,18 @@ def dictpath(self, d, path): idx = int(idx)Quoted 6 lines
tests/qemu-iotests/iotests.py def dictpath(self, d, path): · line 808
Quoted 3 lines
tests/qemu-iotests/iotests.py +1
-1if not isinstance(d, dict) or component not in d: - self.fail('failed path traversal for "%s" in "%s"' % (path, str(d))) + self.fail(f'failed path traversal for "{path}" in "{d}"')Do we require 3.6 so that f-strings are guaranteed to work? (I thought we didn’t. I’d be happy to use them.)
Quoted 3 lines
Oh. Of course we do. It says so in this file that this whole series is about.
Yup. I didn't like the idea of iotests using a newer version of python than our build system, but my concern was not shared, so we get to use f-strings for non-buildtime tools.
End of support for Python 3.5 will be 2020-09-13; so we'll get to use f-strings everywhere else soon, too.
--js
Re: [PATCH v8 07/11] iotests: limit line length to 79 chars
Quoted 38 lines · John Snow
Quoted 20 lines · Max Reitz
Quoted 19 lines · Max Reitz
Quoted 2 lines · John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients.
Also for my very GUI-y diff program (kompare).
Quoted 10 lines
Signed-off-by John Snow <[email protected]>--- tests/qemu-iotests/iotests.py | 64 +++++++++++++++++++++++------------ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23 deletions(-)tests/qemu-iotests/iotests.py +0
-0[...]
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
tests/qemu-iotests/iotests.py +0
-0@@ -529,11 +539,13 @@ def pause_drive(self, drive, event=None): self.pause_drive(drive, "write_aio") return self.qmp('human-monitor-command',Quoted 10 lines
tests/qemu-iotests/iotests.py def pause_drive(self, drive, event=None): · line 543
Quoted 3 lines
tests/qemu-iotests/iotests.py +2
-1- command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive)) + command_line='qemu-io %s "break %s bp_%s"' + % (drive, event, drive))Can we put this value in a variable instead? I don’t like the % aligning with the parameter name instead of the string value. (I also don’t like how there are no spaces around the assignment =, but around the %, even though the % binds more strongly.)
Quoted 1 line
Quoted 7 lines
I think I had another patch somewhere that added an HMP helper that fixes this issue for this particular instance.
I can send that separately as a follow-up. I think otherwise, unfortunately, "we" "decided" that this indent style is "best".
So I think that this patch is "correct".
Perhaps it’s the best if we assume that we had to do it this way, but can’t we just do a
qemu_io_cmd = f'qemu-io {drive} "break {event} bp_{drive}"' self.qmp('human-monitor-command', command_line=qemu_io_cmd)
?
(Or maybe an f-string inline. I was thinking of a separate variable because I wasn’t aware that f-strings would be an option until I got to later hunks of this patch...)
Quoted 11 lines
(All of the other options for indent styles seemed to be worse visually, or actively go against PEP8. While PEP8 is not a bible, every conscious choice to disregard it generally means you will be fighting a CQA tool at some other point in time. I have therefore adopted a "When in Rome" policy to reduce friction wherever possible with pylint, flake8, mypy, pycharm, and so on.)
((I would prefer we begin to deprecate uses of % and begin using .format() and f-strings wherever possible to help alleviate this kind of syntax, too -- but I must insist that's for another series.))
Hm. But you do change something to f-strings below, why not here?
Max