iotests: add JobRunner framework
v4 cover iotests: add JobRunner framework
The basic idea is to make a generic job runtime manager and allow callers to subclass the manager. Then, instead of adding callback arguments to the function all the time, we have à la carte customization of the loop.
To showcase this a little bit, I removed the pre_finalization argument and made existing callers use a custom JobRunner; and then converted test 040 to use this style of job runner.
Is it a simplification? No. Is it cool? Maybe. Did it remove the duplicated job-running code in 040? yes.
As of V4, I'm not sure I really want to pursue this at the moment; it does feel a bit harder to use than the old interface; though it's more "complete". I might look into refining the idea and trying to include it directly in python/qemu; and providing some more 'bespoke' wrappers in iotests that make using it in the test framework a little less cumbersome.
Regardless, I am posting it to the list for archiving for now.
v4:
- Rebased on current master.
- Converted new usages is test 055.
V3:
- Rebased on logging series v8
- Converted 155's new usage of job_run
V2:
- Rebased on logging series; logging conditionals are pretty now.
- Inlined callback login in 257
- No longer based on bitmap-populate job (no test 287)
- Moved super() call to the beginning of test 040's callback
- Added docstrings and type annotations
John Snow (3):
qmp.py: change event_wait to use a dict
iotests: add JobRunner class
iotests: modify test 040 to use JobRunner
python/qemu/machine.py | 10 +- tests/qemu-iotests/040 | 51 +++++---- tests/qemu-iotests/055 | 17 +-- tests/qemu-iotests/155 | 11 +- tests/qemu-iotests/255 | 9 +- tests/qemu-iotests/257 | 54 +++++---- tests/qemu-iotests/260 | 5 +- tests/qemu-iotests/iotests.py | 206 +++++++++++++++++++++++++--------- tests/qemu-iotests/pylintrc | 11 ++ 9 files changed, 258 insertions(+), 116 deletions(-)
Signature · 1 line
2.21.1
PATCH v4 1/3 qmp.py: change event_wait to use a dict
It's easier to work with than a list of tuples, because we can check the keys for membership.
--- python/qemu/machine.py | 10 +++++----- tests/qemu-iotests/040 | 12 ++++++------ tests/qemu-iotests/260 | 5 +++-- tests/qemu-iotests/iotests.py | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-)
python/qemu/machine.py +5 -5
@@ -478,21 +478,21 @@ def event_wait(self, name, timeout=60.0, match=None): timeout: QEMUMonitorProtocol.pull_event timeout parameter. match: Optional match criteria. See event_match for details. """ - return self.events_wait([(name, match)], timeout) + return self.events_wait({name: match}, timeout) def events_wait(self, events, timeout=60.0): """ events_wait waits for and returns a named event from QMP with a timeout. - events: a sequence of (name, match_criteria) tuples. + events: a mapping containing {name: match_criteria}. The match criteria are optional and may be None. See event_match for details. timeout: QEMUMonitorProtocol.pull_event timeout parameter. """ def _match(event): - for name, match in events: - if event['event'] == name and self.event_match(event, match): - return True + name = event['event'] + if name in events: + return self.event_match(event, events[name]) return False # Search cached events
tests/qemu-iotests/040 +6 -6
@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase): def run_job(self, expected_events, error_pauses_job=False): match_device = {'data': {'device': 'job0'}} - events = [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ] + events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + } completed = False log = []
tests/qemu-iotests/260 +3 -2
@@ -67,8 +67,9 @@ def test(persistent, restart): vm.qmp_log('block-commit', device='drive0', top=top, filters=[iotests.filter_qmp_testfiles]) - ev = vm.events_wait((('BLOCK_JOB_READY', None), - ('BLOCK_JOB_COMPLETED', None))) + ev = vm.events_wait({ + 'BLOCK_JOB_READY': None, + 'BLOCK_JOB_COMPLETED': None }) log(filter_qmp_event(ev)) if (ev['event'] == 'BLOCK_JOB_COMPLETED'): vm.shutdown()
tests/qemu-iotests/iotests.py +8 -8
@@ -635,14 +635,14 @@ def run_job(self, job, auto_finalize=True, auto_dismiss=False, """ match_device = {'data': {'device': job}} match_id = {'data': {'id': job}} - events = [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ('BLOCK_JOB_PENDING', match_id), - ('JOB_STATUS_CHANGE', match_id) - ] + events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + 'BLOCK_JOB_PENDING': match_id, + 'JOB_STATUS_CHANGE': match_id, + } error = None while True: ev = filter_qmp_event(self.events_wait(events, timeout=wait))
Signature · 1 line
2.21.1
Re: [PATCH v4 1/3] qmp.py: change event_wait to use a dict
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 10 lines
It's easier to work with than a list of tuples, because we can check the keys for membership.
Signed-off-by John Snow <[email protected]>--- python/qemu/machine.py | 10 +++++----- tests/qemu-iotests/040 | 12 ++++++------ tests/qemu-iotests/260 | 5 +++-- tests/qemu-iotests/iotests.py | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-)
I think you need to convert scripts/simplebench/bench_block_job.py, too.
Quoted 18 lines
python/qemu/machine.py +2
-2@@ -478,21 +478,21 @@ def event_wait(self, name, timeout=60.0, match=None): timeout: QEMUMonitorProtocol.pull_event timeout parameter. match: Optional match criteria. See event_match for details. """ - return self.events_wait([(name, match)], timeout) + return self.events_wait({name: match}, timeout) def events_wait(self, events, timeout=60.0): """ events_wait waits for and returns a named event from QMP with a timeout. - events: a sequence of (name, match_criteria) tuples. + events: a mapping containing {name: match_criteria}. The match criteria are optional and may be None.
python/qemu/machine.py def event_wait(self, name, timeout=60.0, match=None): · line 495
Quoted 10 lines
python/qemu/machine.py +3
-3See event_match for details. timeout: QEMUMonitorProtocol.pull_event timeout parameter. """ def _match(event): - for name, match in events: - if event['event'] == name and self.event_match(event, match): - return True + name = event['event'] + if name in events: + return self.event_match(event, events[name])
This part confused me a bit for a second. Of course, that's probably mostly just me, but I feel 'events' isn't a good name any more when the values of the dict are match strings rather than events.
Quoted 31 lines
python/qemu/machine.py +0
-0return False # Search cached eventstests/qemu-iotests/040 +6
-6@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase): def run_job(self, expected_events, error_pauses_job=False): match_device = {'data': {'device': 'job0'}} - events = [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ] + events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + } completed = False log = []tests/qemu-iotests/260 +0
-0
tests/qemu-iotests/260 def test(persistent, restart): · line 72
Quoted 8 lines
tests/qemu-iotests/260 +3
-2@@ -67,8 +67,9 @@ def test(persistent, restart): vm.qmp_log('block-commit', device='drive0', top=top, filters=[iotests.filter_qmp_testfiles]) - ev = vm.events_wait((('BLOCK_JOB_READY', None), - ('BLOCK_JOB_COMPLETED', None))) + ev = vm.events_wait({ + 'BLOCK_JOB_READY': None, + 'BLOCK_JOB_COMPLETED': None })
So, I'm not sure if this is nitpicking or rather bikeshedding, but having the closing brackets on the next line would be more consistent with the other instances in this patch.
tests/qemu-iotests/260 def test(persistent, restart): · line 75
Quoted 3 lines
tests/qemu-iotests/260 +0
-0log(filter_qmp_event(ev)) if (ev['event'] == 'BLOCK_JOB_COMPLETED'): vm.shutdown()
Kevin
Re: [PATCH v4 1/3] qmp.py: change event_wait to use a dict
Quoted 12 lines · Kevin Wolf
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 10 lines
It's easier to work with than a list of tuples, because we can check the keys for membership.
Signed-off-by John Snow <[email protected]>--- python/qemu/machine.py | 10 +++++----- tests/qemu-iotests/040 | 12 ++++++------ tests/qemu-iotests/260 | 5 +++-- tests/qemu-iotests/iotests.py | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-)
Quoted 1 line
I think you need to convert
scripts/simplebench/bench_block_job.py, too.
Oh, right -- that one is new since I did this. A lot of these scripts need to be moved over into the python/ directory and managed under the same pylint/mypy regime as everything else.
A *ton* of our scripts are in various states of disrepair.
Quoted 25 lines
Quoted 25 lines
python/qemu/machine.py +2
-4@@ -478,21 +478,21 @@ def event_wait(self, name, timeout=60.0, match=None): timeout: QEMUMonitorProtocol.pull_event timeout parameter. match: Optional match criteria. See event_match for details. """ - return self.events_wait([(name, match)], timeout) + return self.events_wait({name: match}, timeout) def events_wait(self, events, timeout=60.0): """ events_wait waits for and returns a named event from QMP with a timeout. - events: a sequence of (name, match_criteria) tuples. + events: a mapping containing {name: match_criteria}. The match criteria are optional and may be None. See event_match for details. timeout: QEMUMonitorProtocol.pull_event timeout parameter. """ def _match(event): - for name, match in events: - if event['event'] == name and self.event_match(event, match):
Quoted 9 lines
python/qemu/machine.py def event_wait(self, name, timeout=60.0, match=None): · line 495
Quoted 4 lines
python/qemu/machine.py +3
-1- return True + name = event['event'] + if name in events: + return self.event_match(event, events[name])This part confused me a bit for a second. Of course, that's probably mostly just me, but I feel 'events' isn't a good name any more when the values of the dict are match strings rather than events.
This is honestly a really bad function. When I was trying to type everything, this one was at the bottom of the pile and it was the worst.
It needs an overhaul.
In my 32 patch series, I left the "match" types as "Any" pretty much everywhere, because it's such a laissez-faire series of functions.
I'll keep the feedback in mind.
Quoted 32 lines
Quoted 32 lines
python/qemu/machine.py +0
-0return False # Search cached eventstests/qemu-iotests/040 +6
-6@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase): def run_job(self, expected_events, error_pauses_job=False): match_device = {'data': {'device': 'job0'}} - events = [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ] + events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + } completed = False log = []tests/qemu-iotests/260 +0
-0@@ -67,8 +67,9 @@ def test(persistent, restart):
Quoted 12 lines
tests/qemu-iotests/260 def test(persistent, restart): · line 72
Quoted 7 lines
tests/qemu-iotests/260 +3
-2vm.qmp_log('block-commit', device='drive0', top=top, filters=[iotests.filter_qmp_testfiles]) - ev = vm.events_wait((('BLOCK_JOB_READY', None), - ('BLOCK_JOB_COMPLETED', None))) + ev = vm.events_wait({ + 'BLOCK_JOB_READY': None, + 'BLOCK_JOB_COMPLETED': None })So, I'm not sure if this is nitpicking or rather bikeshedding, but having the closing brackets on the next line would be more consistent with the other instances in this patch.
Nah, it's fine. I'll clean it up. This is pretty close to an RFC series anyway, so I didn't really polish it.
(Or, I will try to clean it up. I probably won't work on it again in the near term. I think I just wanted to see if this seemed useful in general to people.
As part of maybe moving the python library onto a package, I thought that maybe developing a JobRunner tool would be useful to have in that library. As you can see, I nestled it into iotests.py, though.)
Quoted 6 lines
tests/qemu-iotests/260 def test(persistent, restart): · line 75
Quoted 3 lines
tests/qemu-iotests/260 +0
-0log(filter_qmp_event(ev)) if (ev['event'] == 'BLOCK_JOB_COMPLETED'): vm.shutdown()Kevin
Signature · 1 line
—js
Re: [PATCH v4 1/3] qmp.py: change event_wait to use a dict
Am 14.05.2020 um 17:07 hat John Snow geschrieben:
Quoted 15 lines · Kevin Wolf
Quoted 12 lines
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 10 lines
It's easier to work with than a list of tuples, because we can check the keys for membership.
Signed-off-by John Snow <[email protected]>--- python/qemu/machine.py | 10 +++++----- tests/qemu-iotests/040 | 12 ++++++------ tests/qemu-iotests/260 | 5 +++-- tests/qemu-iotests/iotests.py | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-)
Quoted 7 lines
Quoted 1 line
I think you need to convert
scripts/simplebench/bench_block_job.py, too.Oh, right -- that one is new since I did this. A lot of these scripts need to be moved over into the python/ directory and managed under the same pylint/mypy regime as everything else.
A *ton* of our scripts are in various states of disrepair.
Is python/ actually supposed to have executable files in it? I thought it was more for libraries.
Quoted 35 lines
Quoted 25 lines
Quoted 25 lines
python/qemu/machine.py +2
-4@@ -478,21 +478,21 @@ def event_wait(self, name, timeout=60.0, match=None): timeout: QEMUMonitorProtocol.pull_event timeout parameter. match: Optional match criteria. See event_match for details. """ - return self.events_wait([(name, match)], timeout) + return self.events_wait({name: match}, timeout) def events_wait(self, events, timeout=60.0): """ events_wait waits for and returns a named event from QMP with a timeout. - events: a sequence of (name, match_criteria) tuples. + events: a mapping containing {name: match_criteria}. The match criteria are optional and may be None. See event_match for details. timeout: QEMUMonitorProtocol.pull_event timeout parameter. """ def _match(event): - for name, match in events: - if event['event'] == name and self.event_match(event, match):Quoted 9 lines
python/qemu/machine.py def event_wait(self, name, timeout=60.0, match=None): · line 495
Quoted 4 lines
python/qemu/machine.py +3
-1- return True + name = event['event'] + if name in events: + return self.event_match(event, events[name])This part confused me a bit for a second. Of course, that's probably mostly just me, but I feel 'events' isn't a good name any more when the values of the dict are match strings rather than events.
Quoted 7 lines
This is honestly a really bad function. When I was trying to type everything, this one was at the bottom of the pile and it was the worst.
It needs an overhaul.
In my 32 patch series, I left the "match" types as "Any" pretty much everywhere, because it's such a laissez-faire series of functions.
It would require recursive types, which aren't supported yet. So I guess Any is the best we can do at the moment.
Quoted 47 lines
I'll keep the feedback in mind.
Quoted 32 lines
Quoted 32 lines
python/qemu/machine.py +0
-0return False # Search cached eventstests/qemu-iotests/040 +6
-6@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase): def run_job(self, expected_events, error_pauses_job=False): match_device = {'data': {'device': 'job0'}} - events = [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ] + events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + } completed = False log = []tests/qemu-iotests/260 +0
-0@@ -67,8 +67,9 @@ def test(persistent, restart):Quoted 12 lines
tests/qemu-iotests/260 def test(persistent, restart): · line 72
Quoted 7 lines
tests/qemu-iotests/260 +3
-2vm.qmp_log('block-commit', device='drive0', top=top, filters=[iotests.filter_qmp_testfiles]) - ev = vm.events_wait((('BLOCK_JOB_READY', None), - ('BLOCK_JOB_COMPLETED', None))) + ev = vm.events_wait({ + 'BLOCK_JOB_READY': None, + 'BLOCK_JOB_COMPLETED': None })So, I'm not sure if this is nitpicking or rather bikeshedding, but having the closing brackets on the next line would be more consistent with the other instances in this patch.
Quoted 6 lines
Nah, it's fine. I'll clean it up. This is pretty close to an RFC series anyway, so I didn't really polish it.
(Or, I will try to clean it up. I probably won't work on it again in the near term. I think I just wanted to see if this seemed useful in general to people.
Ah, there isn't much missing for this series, though. We don't have to wait for a fix-the-world series when we can incrementally improve things.
Quoted 3 lines
As part of maybe moving the python library onto a package, I thought that maybe developing a JobRunner tool would be useful to have in that library. As you can see, I nestled it into
iotests.py, though.)
Let's just do that now, we can always move it somewhere else later.
Kevin
Re: [PATCH v4 1/3] qmp.py: change event_wait to use a dict
Quoted 18 lines · Kevin Wolf
Am 14.05.2020 um 17:07 hat John Snow geschrieben:
Quoted 17 lines · Kevin Wolf
Quoted 13 lines
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 10 lines
It's easier to work with than a list of tuples, because we can check the keys for membership.
Signed-off-by John Snow <[email protected]>--- python/qemu/machine.py | 10 +++++----- tests/qemu-iotests/040 | 12 ++++++------ tests/qemu-iotests/260 | 5 +++-- tests/qemu-iotests/iotests.py | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-)I think you need to convert
scripts/simplebench/bench_block_job.py, too.
Quoted 9 lines
Quoted 5 lines
Oh, right -- that one is new since I did this. A lot of these scripts need to be moved over into the python/ directory and managed under the same pylint/mypy regime as everything else.
A *ton* of our scripts are in various states of disrepair.
Is python/ actually supposed to have executable files in it? I thought it was more for libraries.
Welllllllllllllll. At the moment it's library only. but one of the things you can do with a library is define executable entry-points into that library.
If you haven't cast an eye at that 32 patch series yet, it basically creates a structure like this:
Quoted 1 line
./python/qemu/lib/[qmp|machine|qtest|accel].py
qemu/ forms a PEP420 namespace; the idea is to be able to modularly create and independently package subpackages.
qemu/lib forms a proper python package in which there are no executables, just a library, as you say.
My idea is that anything under python/*/ ought to form a properly formatted package. So we could, for instance, have a python/qemu/devtools namespace which packages and collects a bunch of our little scripts.
Then we could make sure we hit them with the same mypy/pylint/flake8/whatever as the core libraries those scripts are using to keep them in sync better.
And, ideally, if they are all using the same kind of paradigms for import and dependency management it will be easier to use them and keep them up to date, etc.
For using them as a developer, you could, say,
cd ~/src/qemu/python
pip3 install --user -e .
and install the source packages to your local environment and then have access to e.g.
Quoted 1 line
qmp-shell
right on your CLI, without having to fuss with PYTHONPATH or anything else. As you update the source repo, you'll get the new versions of the package living in your python environment automatically.
Of course, this maybe has downsides too; so you can always use a virtual environment to adopt a context in which you have these tools. For that,
Quoted 4 lines
pip3 install --user pipenv # or use dnf, or apt, w/e. cd ~/src/qemu/python
pipenv shell
pip install -e .
And from here you'll have the dev package installed to a development venv that you can use.
*cough* anyway, that's wildly off-topic.
Generally, you want to format a library such that you have a callable entry point, maybe named main(). So you'd have some qmp-shell module and it has a main() function.
Then, in the setup.py script, you'd define qemu.lib.qmp_shell:main() as an entry point and give it a name like 'qmp-shell'. When pip/setuptools processes your package installation, it'll create a shim for you in e.g. ~/.local/bin/qmp-shell that will just load the library and execute that entrypoint for you.
I was thinking I'd do this for all of our python scripts so I could spend my energy on a pylint/mypy test infrastructure *once* and *in one place* and then it would be easier to detect regressions for scripts that don't actually run as part of the test suite.
Quoted 38 lines
Quoted 38 lines
Quoted 25 lines
Quoted 25 lines
python/qemu/machine.py +2
-4@@ -478,21 +478,21 @@ def event_wait(self, name, timeout=60.0, match=None): timeout: QEMUMonitorProtocol.pull_event timeout parameter. match: Optional match criteria. See event_match for details. """ - return self.events_wait([(name, match)], timeout) + return self.events_wait({name: match}, timeout) def events_wait(self, events, timeout=60.0): """ events_wait waits for and returns a named event from QMP with a timeout. - events: a sequence of (name, match_criteria) tuples. + events: a mapping containing {name: match_criteria}. The match criteria are optional and may be None. See event_match for details. timeout: QEMUMonitorProtocol.pull_event timeout parameter. """ def _match(event): - for name, match in events: - if event['event'] == name and self.event_match(event, match):Quoted 9 lines
python/qemu/machine.py def event_wait(self, name, timeout=60.0, match=None): · line 495
Quoted 4 lines
python/qemu/machine.py +3
-1- return True + name = event['event'] + if name in events: + return self.event_match(event, events[name])This part confused me a bit for a second. Of course, that's probably mostly just me, but I feel 'events' isn't a good name any more when the values of the dict are match strings rather than events.
This is honestly a really bad function. When I was trying to type everything, this one was at the bottom of the pile and it was the worst.
Quoted 8 lines
Quoted 4 lines
It needs an overhaul.
In my 32 patch series, I left the "match" types as "Any" pretty much everywhere, because it's such a laissez-faire series of functions.
It would require recursive types, which aren't supported yet. So I guess Any is the best we can do at the moment.
It would also do well with a fully schema-validated API if we actually knew exactly what a "QMP Message" was and had some guarantee about exactly what fields and types it had.
This is where a generated validator would really help for type-safe SDK tooling.
... We of course do not have that right now, and might not ever, so yeah, 'Any' type is fine -- but I really want to rewrite this function because it does stick out as ugly and hard to read, I admit :(
Anything that's hard to type -- by the person who wrote most of it -- is going to be hard to use or understand. Time for a rework.
Quoted 50 lines
Quoted 50 lines
I'll keep the feedback in mind.
Quoted 32 lines
Quoted 32 lines
python/qemu/machine.py +0
-0return False # Search cached eventstests/qemu-iotests/040 +6
-6@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase): def run_job(self, expected_events, error_pauses_job=False): match_device = {'data': {'device': 'job0'}} - events = [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ] + events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + } completed = False log = []tests/qemu-iotests/260 +0
-0@@ -67,8 +67,9 @@ def test(persistent, restart):Quoted 12 lines
tests/qemu-iotests/260 def test(persistent, restart): · line 72
Quoted 7 lines
tests/qemu-iotests/260 +3
-2vm.qmp_log('block-commit', device='drive0', top=top, filters=[iotests.filter_qmp_testfiles]) - ev = vm.events_wait((('BLOCK_JOB_READY', None), - ('BLOCK_JOB_COMPLETED', None))) + ev = vm.events_wait({ + 'BLOCK_JOB_READY': None, + 'BLOCK_JOB_COMPLETED': None })So, I'm not sure if this is nitpicking or rather bikeshedding, but having the closing brackets on the next line would be more consistent with the other instances in this patch.
Nah, it's fine. I'll clean it up. This is pretty close to an RFC series anyway, so I didn't really polish it.
Quoted 8 lines
Quoted 3 lines
(Or, I will try to clean it up. I probably won't work on it again in the near term. I think I just wanted to see if this seemed useful in general to people.
Ah, there isn't much missing for this series, though. We don't have to wait for a fix-the-world series when we can incrementally improve things.
Alright, I'll try to hit it halfway -- I spent some time thinking about a "full" job running framework but ran into some dead-ends I wasn't too happy with, and wasn't convinced this was a simplification of any kind.
Still, seeing part of the job running code get duplicated in 040 was a motivation to try and provide some universal job-running monster that would be extensible for nearly any task.
Unfortunately that complexity does generally make the calling sites look worse, so I cooled off on the idea since.
So I did intend this as an RFC, because I'm not really 100% happy with the design.
Quoted 6 lines
Quoted 3 lines
As part of maybe moving the python library onto a package, I thought that maybe developing a JobRunner tool would be useful to have in that library. As you can see, I nestled it into
iotests.py, though.)Let's just do that now, we can always move it somewhere else later.
I assume you mean "Let's just put it in iotests.py for now."
If we do decide to take that 32 patch series to formalize a qemu lib, it will not be hard to start moving things from here to there.
I will probably take an eye to our iotests and see what functionality gets duplicated or used a lot and try to push more things down into the stack where possible.
The QMP event handling stuff in particular seems needlessly split between qmp.py, machine.py, iotests.py, etc.
Quoted 2 lines
Kevin
Re: [PATCH v4 1/3] qmp.py: change event_wait to use a dict
Quoted 21 lines · John Snow
Quoted 4 lines
Quoted 4 lines
Nah, it's fine. I'll clean it up. This is pretty close to an RFC series anyway, so I didn't really polish it.
Quoted 8 lines
Quoted 3 lines
(Or, I will try to clean it up. I probably won't work on it again in the near term. I think I just wanted to see if this seemed useful in general to people.
Ah, there isn't much missing for this series, though. We don't have to wait for a fix-the-world series when we can incrementally improve things.
Alright, I'll try to hit it halfway -- I spent some time thinking about a "full" job running framework but ran into some dead-ends I wasn't too happy with, and wasn't convinced this was a simplification of any kind.
Still, seeing part of the job running code get duplicated in 040 was a motivation to try and provide some universal job-running monster that would be extensible for nearly any task.
Quoted 5 lines
Unfortunately that complexity does generally make the calling sites look worse, so I cooled off on the idea since.
So I did intend this as an RFC, because I'm not really 100% happy with the design.
I noticed that the block-dirty-bitmap-populate series depends on this one; is it going to be simpler for me to fix the few things that Kevin pointed out here, or to wait for you to post a v5 of this series, or to rewrite the iotest in that series to not depend on JobRunner after all?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v4 1/3] qmp.py: change event_wait to use a dict
Quoted 28 lines · Eric Blake
Quoted 26 lines · John Snow
Quoted 10 lines
Quoted 9 lines
Nah, it's fine. I'll clean it up. This is pretty close to an RFC series anyway, so I didn't really polish it.
(Or, I will try to clean it up. I probably won't work on it again in the
near term. I think I just wanted to see if this seemed useful in general
to people.Quoted 4 lines
Ah, there isn't much missing for this series, though. We don't have to wait for a fix-the-world series when we can incrementally improve things.
Alright, I'll try to hit it halfway -- I spent some time thinking about a "full" job running framework but ran into some dead-ends I wasn't too happy with, and wasn't convinced this was a simplification of any kind.
Still, seeing part of the job running code get duplicated in 040 was a motivation to try and provide some universal job-running monster that would be extensible for nearly any task.
Unfortunately that complexity does generally make the calling sites look worse, so I cooled off on the idea since.
Quoted 8 lines
Quoted 2 lines
So I did intend this as an RFC, because I'm not really 100% happy with the design.
I noticed that the block-dirty-bitmap-populate series depends on this one; is it going to be simpler for me to fix the few things that Kevin pointed out here, or to wait for you to post a v5 of this series, or to rewrite the iotest in that series to not depend on JobRunner after all?
It should be pretty trivial (I think) to just rebase the bitpop job on top of mainline QEMU without needing this, I'd recommend doing that.
I started porting the job runner to the standalone qemu package instead and it's going to take me longer to do that than it would be to just not use this patchset for the bitpop test.
If you ping me on IRC tomorrow (Sorry) I can wean the dependency myself.
--js
PATCH v4 2/3 iotests: add JobRunner class
In reply to John SnowThe idea is that instead of increasing the arguments to job_run all the time, create a more general-purpose job runner that can be subclassed to do interesting things with.
pylint note: the 'callbacks' option guards against unused warning arguments in functions designated as callbacks. It does not currently guard against "no-self-use" though; hence a once-off ignore.
mypy note: QapiEvent is only a weak alias; it's fully interchangable with the type it's declared as. In the future, we may wish to tighten these types. For now, this communicates the rough shape of the type and (more importantly) the intent.
--- tests/qemu-iotests/055 | 17 +-- tests/qemu-iotests/155 | 11 +- tests/qemu-iotests/255 | 9 +- tests/qemu-iotests/257 | 54 +++++---- tests/qemu-iotests/iotests.py | 206 +++++++++++++++++++++++++--------- tests/qemu-iotests/pylintrc | 11 ++ 6 files changed, 225 insertions(+), 83 deletions(-)
tests/qemu-iotests/055 +10 -7
@@ -115,19 +115,22 @@ class TestSingleDrive(iotests.QMPTestCase): self.do_test_pause('blockdev-backup', 'drive1', blockdev_target_img) def do_test_resize_blockdev_backup(self, device, node): - def pre_finalize(): - result = self.vm.qmp('block_resize', device=device, size=65536) - self.assert_qmp(result, 'error/class', 'GenericError') - - result = self.vm.qmp('block_resize', node_name=node, size=65536) - self.assert_qmp(result, 'error/class', 'GenericError') + class JobRunner(iotests.TestJobRunner): + def on_pending(self, event): + qmp = self._vm.qmp + result = qmp('block_resize', device=device, size=65536) + self.test.assert_qmp(result, 'error/class', 'GenericError') + result = qmp('block_resize', node_name=node, size=65536) + self.test.assert_qmp(result, 'error/class', 'GenericError') + super().on_pending(event) result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0', target='drive1', sync='full', auto_finalize=False, auto_dismiss=False) self.assert_qmp(result, 'return', {}) - self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize) + runner = JobRunner(self.vm, 'job0', test=self, auto_finalize=False) + runner.run() def test_source_resize_blockdev_backup(self): self.do_test_resize_blockdev_backup('drive0', 'source')
tests/qemu-iotests/155 +9 -2
@@ -163,6 +163,12 @@ class BaseClass(iotests.QMPTestCase): self.assert_qmp_absent(node, 'image/backing-image') +class MirrorJob(iotests.TestJobRunner): + def on_pending(self, event): + self.test.openBacking() + super().on_pending(event) + + # Class variables for controlling its behavior: # # cmd: Mirroring command to execute, either drive-mirror or blockdev-mirror
@@ -188,8 +194,9 @@ class MirrorBaseClass(BaseClass): self.assert_qmp(result, 'return', {}) - self.vm.run_job('mirror-job', auto_finalize=False, - pre_finalize=self.openBacking, auto_dismiss=True) + job = MirrorJob(self.vm, 'mirror-job', test=self, + auto_finalize=False, auto_dismiss=True) + job.run() def testFull(self): self.runMirror('full')
tests/qemu-iotests/255 +7 -2
@@ -71,8 +71,13 @@ with iotests.FilePath('t.qcow2') as disk_path, \ result = vm.qmp_log('block-commit', job_id='job0', auto_finalize=False, device='overlay', top_node='mid') - vm.run_job('job0', auto_finalize=False, pre_finalize=start_requests, - auto_dismiss=True) + class JobRunner(iotests.JobRunner): + def on_pending(self, event): + start_requests() + super().on_pending(event) + + runner = JobRunner(vm, 'job0', auto_finalize=False, auto_dismiss=True) + runner.run() vm.shutdown()
tests/qemu-iotests/257 +32 -22
62 lines not on this page · raw message
tests/qemu-iotests/iotests.py +156 -50
234 lines not on this page · raw message
tests/qemu-iotests/pylintrc +11 -0
@@ -17,9 +17,20 @@ disable=invalid-name, too-many-lines, too-many-locals, too-many-public-methods, + too-many-instance-attributes, # These are temporary, and should be removed: missing-docstring, + +[VARIABLES] + +# List of strings which can identify a callback function by name. A callback +# name must start or end with one of those strings. +callbacks=cb_, + _cb, + on_, + + [FORMAT] # Maximum number of characters on a single line.
Signature · 1 line
2.21.1
Re: [PATCH v4 2/3] iotests: add JobRunner class
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 15 lines
The idea is that instead of increasing the arguments to
job_runall the time, create a more general-purpose job runner that can be subclassed to do interesting things with.pylint note: the 'callbacks' option guards against unused warning arguments in functions designated as callbacks. It does not currently guard against "no-self-use" though; hence a once-off ignore.
mypy note: QapiEvent is only a weak alias; it's fully interchangable with the type it's declared as. In the future, we may wish to tighten these types. For now, this communicates the rough shape of the type and (more importantly) the intent.
Signed-off-by John Snow <[email protected]>
tests/qemu-iotests/iotests.py def remote_filename(path): · line 534
Quoted 9 lines
tests/qemu-iotests/iotests.py +9
-0+ # Listen for these events with these parameters: + self._events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + 'BLOCK_JOB_PENDING': match_id, + 'JOB_STATUS_CHANGE': match_id + }
The old code had a trailing comma here in case we need to add more events later. Anyway:
Re: [PATCH v4 2/3] iotests: add JobRunner class
Quoted 26 lines · Kevin Wolf
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 15 lines
The idea is that instead of increasing the arguments to
job_runall the time, create a more general-purpose job runner that can be subclassed to do interesting things with.pylint note: the 'callbacks' option guards against unused warning arguments in functions designated as callbacks. It does not currently guard against "no-self-use" though; hence a once-off ignore.
mypy note: QapiEvent is only a weak alias; it's fully interchangable with the type it's declared as. In the future, we may wish to tighten these types. For now, this communicates the rough shape of the type and (more importantly) the intent.
Signed-off-by John Snow <[email protected]>tests/qemu-iotests/iotests.py def remote_filename(path): · line 534
Quoted 9 lines
tests/qemu-iotests/iotests.py +9
-0+ # Listen for these events with these parameters: + self._events = { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + 'BLOCK_JOB_PENDING': match_id, + 'JOB_STATUS_CHANGE': match_id + }
Quoted 5 lines
The old code had a trailing comma here in case we need to add more events later. Anyway:
Reviewed-by Kevin Wolf <[email protected]>
Whoops. I favor those too, so I'll put it back.
PATCH v4 3/3 iotests: modify test 040 to use JobRunner
In reply to John SnowInstead of having somewhat reproduced it for itself.
--- tests/qemu-iotests/040 | 51 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 26 deletions(-)
tests/qemu-iotests/040 +25 -26
@@ -483,34 +483,33 @@ class TestErrorHandling(iotests.QMPTestCase): file=('top-dbg' if top_debug else 'top-file'), backing='mid-fmt') + + class TestJobRunner(iotests.JobRunner): + expected_events = ('BLOCK_JOB_COMPLETED', + 'BLOCK_JOB_ERROR', + 'BLOCK_JOB_READY') + + def __init__(self, *args, test, **kwargs): + super().__init__(*args, **kwargs) + self.log = [] + self.test = test + + def on_pause(self, event): + super().on_pause(event) + result = self._vm.qmp('block-job-resume', device=self._id) + self.test.assert_qmp(result, 'return', {}) + + def on_block_job_event(self, event): + if event['event'] not in self.expected_events: + self.test.fail("Unexpected event: %s" % event) + super().on_block_job_event(event) + self.log.append(event) + def run_job(self, expected_events, error_pauses_job=False): - match_device = {'data': {'device': 'job0'}} - events = { - 'BLOCK_JOB_COMPLETED': match_device, - 'BLOCK_JOB_CANCELLED': match_device, - 'BLOCK_JOB_ERROR': match_device, - 'BLOCK_JOB_READY': match_device, - } - - completed = False - log = [] - while not completed: - ev = self.vm.events_wait(events, timeout=5.0) - if ev['event'] == 'BLOCK_JOB_COMPLETED': - completed = True - elif ev['event'] == 'BLOCK_JOB_ERROR': - if error_pauses_job: - result = self.vm.qmp('block-job-resume', device='job0') - self.assert_qmp(result, 'return', {}) - elif ev['event'] == 'BLOCK_JOB_READY': - result = self.vm.qmp('block-job-complete', device='job0') - self.assert_qmp(result, 'return', {}) - else: - self.fail("Unexpected event: %s" % ev) - log.append(iotests.filter_qmp_event(ev)) - + job = self.TestJobRunner(self.vm, 'job0', test=self) + job.run() self.maxDiff = None - self.assertEqual(expected_events, log) + self.assertEqual(expected_events, job.log) def event_error(self, op, action): return {
Signature · 1 line
2.21.1
Re: [PATCH v4 3/3] iotests: modify test 040 to use JobRunner
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 3 lines
Instead of having somewhat reproduced it for itself.
Signed-off-by John Snow <[email protected]>
I think you should pass auto_dismiss=True to the JobRunner, or (probably preferable) change prepare_and_start_job() to start the job with auto_dismiss=False.
Kevin
Re: [PATCH v4 3/3] iotests: modify test 040 to use JobRunner
Quoted 11 lines · Kevin Wolf
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 3 lines
Instead of having somewhat reproduced it for itself.
Signed-off-by John Snow <[email protected]>I think you should pass auto_dismiss=True to the JobRunner, or (probably preferable) change
prepare_and_start_job()to start the job with auto_dismiss=False.Kevin
okay, I'll try that out and see if I like it.
Wild tangents, as is my normal:
I also think it would be neat, in some sense, to provide a job creation abstraction where creating the QMP command in python also creates the runner with the right parameters based on how you initialized it.
I've not given these even a proper three minutes think, but some generalized interface for managing the creation of jobs to use in concert with the job runner would be slick.
(What reminds me of this is needing to remember and understand if I started something with auto_dismiss or not, which jobs it defaults to which for, etc. Streamlining the creation and runner could be slick for faster test-writing in normative cases.)
Re: [PATCH v4 3/3] iotests: modify test 040 to use JobRunner
Am 14.05.2020 um 21:37 hat John Snow geschrieben:
Quoted 20 lines · Kevin Wolf
Quoted 11 lines
Am 14.05.2020 um 04:25 hat John Snow geschrieben:
Quoted 3 lines
Instead of having somewhat reproduced it for itself.
Signed-off-by John Snow <[email protected]>I think you should pass auto_dismiss=True to the JobRunner, or (probably preferable) change
prepare_and_start_job()to start the job with auto_dismiss=False.Kevin
okay, I'll try that out and see if I like it.
Wild tangents, as is my normal:
Quoted 7 lines
I also think it would be neat, in some sense, to provide a job creation abstraction where creating the QMP command in python also creates the runner with the right parameters based on how you initialized it.
I've not given these even a proper three minutes think, but some generalized interface for managing the creation of jobs to use in concert with the job runner would be slick.
JobRunner could have a static method that starts a job (it would take the same options as qmp() and forward everything to qmp(), except that it parses auto_* first) and returns a JobRunner object that you can run later.
Quoted 4 lines
(What reminds me of this is needing to remember and understand if I started something with
auto_dismissor not, which jobs it defaults to which for, etc. Streamlining the creation and runner could be slick for faster test-writing in normative cases.)
Yes.
In general, I think we should keep tests simple (even if that means some duplication) and avoid overengineering testing infrastructure because we could well sink more time there than we'll ever get back, but I guess small and simple wrappers like in this case can't hurt.
Kevin