Skip to content

RFC python/qemu: refactor as installable package

John Snow qemu-devel 1 version Applied

Status from this message: Thanks, applied to my python-next tree:

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest32 patches 52 repliesApplied

Patches in v1 Applied

#Subject and filesSizeRepliesApplied
1/32python/qemu: create qemu.lib modulepython/qemu/__init__.py python/qemu/lib/__init__.py python/qemu/lib/accel.py +16 more+74 -2816 replies
2/32scripts/qmp: Fix shebang and importsscripts/qmp/qmp scripts/qmp/qom-fuse scripts/qmp/qom-get +3 more+21 -91 reply
3/32python//machine.py: remove bare exceptpython/qemu/lib/machine.py+21 -126 replies
4/32python/qemu/lib: delint, add pylintrcpython/qemu/lib/machine.py python/qemu/lib/pylintrc python/qemu/lib/qtest.py+88 -181 reply
5/32python/qemu/lib: delint; add flake8 configpython/qemu/lib/.flake8 python/qemu/lib/accel.py python/qemu/lib/machine.py +1 more+19 -92 replies
6/32python/qemu: formalize as packagepython/README.rst python/setup.py+56 -01 reply
7/32python/qemu: add README.rstpython/qemu/README.rst+8 -00 replies
8/32python/qemu: Add Pipfilepython/Pipfile+11 -00 replies
9/32python/qemu: add pylint to Pipfilepython/Pipfile python/Pipfile.lock+124 -00 replies
10/32python/qemu: Add flake8 to Pipfilepython/Pipfile python/Pipfile.lock+31 -10 replies
11/32python/qemu/lib: remove Python2 style super() callspython/qemu/lib/machine.py python/qemu/lib/qtest.py+8 -92 replies
12/32python/qemu/lib: fix socket.makefile() typingpython/qemu/lib/qmp.py python/qemu/lib/qtest.py+15 -71 reply
13/32python/qemu/lib: Adjust traceback typingpython/qemu/lib/machine.py python/qemu/lib/qmp.py+14 -42 replies
14/32python//qmp.py: use True/False for non/blocking modespython/qemu/lib/qmp.py+2 -22 replies
15/32python//qmp.py: Define common typespython/qemu/lib/qmp.py+18 -00 replies
16/32python//qmp.py: re-absorb MonitorResponseErrorpython/qemu/lib/machine.py python/qemu/lib/qmp.py scripts/render_block_graph.py+18 -181 reply
17/32python//qmp.py: Do not return None from cmd_objpython/qemu/lib/qmp.py+5 -90 replies
18/32python//qmp.py: add casts to JSON deserializationpython/qemu/lib/qmp.py+6 -21 reply
19/32python//qmp.py: add QMPProtocolErrorpython/qemu/lib/qmp.py+10 -01 reply
20/32python//qmp.py: assert sockfile is not Nonepython/qemu/lib/qmp.py+1 -03 replies
21/32python//machine.py: remove logging configurationpython/qemu/lib/machine.py+0 -32 replies
22/32python//machine.py: Fix monitor address typingpython/qemu/lib/machine.py+29 -160 replies
23/32python//machine.py: reorder __init__python/qemu/lib/machine.py+17 -121 reply
24/32python//machine.py: Don't modify state in _base_args()python/qemu/lib/machine.py python/qemu/lib/qtest.py+18 -90 replies
25/32python//machine.py: Handle None events in event_waitpython/qemu/lib/machine.py+5 -10 replies
26/32python//machine.py: use qmp.commandpython/qemu/lib/machine.py+15 -114 replies
27/32python//machine.py: Add _qmp access shimpython/qemu/lib/machine.py+9 -30 replies
28/32python//machine.py: fix _popen accesspython/qemu/lib/machine.py+12 -60 replies
29/32python//qtest.py: Check before accessing _qtestpython/qemu/lib/qtest.py+11 -32 replies
30/32python/qemu/lib: make 'args' style arguments immutablepython/qemu/lib/machine.py python/qemu/lib/qtest.py+29 -170 replies
31/32python/qemu: add mypy to Pipfilepython/Pipfile python/Pipfile.lock+37 -10 replies
32/32python/qemu/lib: Add mypy type annotationspython/qemu/lib/accel.py python/qemu/lib/machine.py python/qemu/lib/qmp.py +1 more+99 -740 replies

Reviews trailers per patch

16 of 32 patches have a Reviewed-by or Acked-by.

PatchPhilippe Mathieu-Daudé
1/32
2/32R-b
3/32R-b
4/32R-b
5/32R-b
6/32A-b
7/32
8/32
9/32
10/32
11/32R-b
12/32R-b
13/32A-b
14/32R-b
15/32
16/32R-b
17/32
18/32A-b
19/32R-b
20/32A-b R-b
21/32R-b
22/32
23/32R-b
24/32
25/32
26/32
27/32
28/32
29/32R-b
30/32
31/32
32/32

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH RFC 00/32] python/qemu: refactor as installable package

Hey, I got lost on my way to the store and I accidentally got 32 patches that convert our python library into something that passes pylint, flake8, and mypy --strict.

...So, a few things:

1. This is just an RFC. The actual design of these libraries really needs adjusted to be more pythonic. In general this means less Optional[T] return types and raising more exceptions. This could be handled later, but we ought to address it before publishing, if we do.

2. We still need to think carefully about how we package this, if we even want to package it, what the license on the top-level package should be, etc.

3. We should consider how to version it. For now, I'm using a lockstep versioning.

4. You can install this package using pip3 or python3 setup.py to a virtual environment or to your real one. From there, any python code in the QEMU tree that imports these modules will work with no sys.path hacking or custom PYTHONPATH exports.

5. You don't have to install it, though. I left all of the usual hacks in place in the rest of the tree so that everything will just keep working exactly as-is for right now. It's just that you COULD install it.

6. Here's a cool trick if you don't know about it yet:

Quoted 2 lines

cd qemu/python/qemu
pip3 install --user -e .

This will install the package in "develop" mode, which installs a forwarder package. When you update your source tree, the installed package stays "up to date" with the most recent edits.

Alright, have fun, stay safe!

John Snow (32):
python/qemu: create qemu.lib module scripts/qmp: Fix shebang and imports python//machine.py: remove bare except python/qemu/lib: delint, add pylintrc python/qemu/lib: delint; add flake8 config python/qemu: formalize as package python/qemu: add README.rst
python/qemu: Add Pipfile
python/qemu: add pylint to Pipfile python/qemu: Add flake8 to Pipfile python/qemu/lib: remove Python2 style super() calls python/qemu/lib: fix socket.makefile() typing python/qemu/lib: Adjust traceback typing python//qmp.py: use True/False for non/blocking modes python//qmp.py: Define common types python//qmp.py: re-absorb MonitorResponseError python//qmp.py: Do not return None from cmd_obj python//qmp.py: add casts to JSON deserialization python//qmp.py: add QMPProtocolError python//qmp.py: assert sockfile is not None python//machine.py: remove logging configuration python//machine.py: Fix monitor address typing python//machine.py: reorder __init__ python//machine.py: Don't modify state in _base_args() python//machine.py: Handle None events in event_wait python//machine.py: use qmp.command python//machine.py: Add _qmp access shim python//machine.py: fix _popen access python//qtest.py: Check before accessing _qtest python/qemu/lib: make 'args' style arguments immutable python/qemu: add mypy to Pipfile python/qemu/lib: Add mypy type annotations

 python/README.rst                         |   6 +
 python/qemu/README.rst                    |   8 +
 python/Pipfile                            |  14 +
 python/Pipfile.lock                       | 187 +++++++++++++
 python/qemu/__init__.py                   |  11 -
 python/qemu/lib/.flake8                   |   2 +
 python/qemu/lib/__init__.py               |  57 ++++
 python/qemu/{ => lib}/accel.py            |  17 +-
 python/qemu/{ => lib}/machine.py          | 320 +++++++++++++---------
 python/qemu/lib/pylintrc                  |  58 ++++
 python/qemu/{ => lib}/qmp.py              | 140 +++++++---
 python/qemu/lib/qtest.py                  | 160 +++++++++++
 python/qemu/qtest.py                      | 119 --------
 python/setup.py                           |  50 ++++
 scripts/device-crash-test                 |   2 +-
 scripts/qmp/qemu-ga-client                |   2 +-
 scripts/qmp/qmp                           |   4 +-
 scripts/qmp/qmp-shell                     |   2 +-
 scripts/qmp/qom-fuse                      |   4 +-
 scripts/qmp/qom-get                       |   6 +-
 scripts/qmp/qom-list                      |   4 +-
 scripts/qmp/qom-set                       |   6 +-
 scripts/qmp/qom-tree                      |   6 +-
 scripts/render_block_graph.py             |   5 +-
 scripts/simplebench/bench_block_job.py    |   4 +-
 tests/acceptance/avocado_qemu/__init__.py |   2 +-
 tests/acceptance/boot_linux.py            |   3 +-
 tests/acceptance/virtio_check_params.py   |   2 +-
 tests/acceptance/virtio_version.py        |   2 +-
 tests/migration/guestperf/engine.py       |   2 +-
 tests/qemu-iotests/235                    |   2 +-
 tests/qemu-iotests/iotests.py             |   2 +-
 tests/vm/basevm.py                        |   6 +-
 33 files changed, 881 insertions(+), 334 deletions(-)
 create mode 100644 python/README.rst
 create mode 100644 python/qemu/README.rst
 create mode 100644 python/Pipfile
 create mode 100644 python/Pipfile.lock
 delete mode 100644 python/qemu/__init__.py
 create mode 100644 python/qemu/lib/.flake8
 create mode 100644 python/qemu/lib/__init__.py
 rename python/qemu/{ => lib}/accel.py (86%)
 rename python/qemu/{ => lib}/machine.py (67%)
 create mode 100644 python/qemu/lib/pylintrc
 rename python/qemu/{ => lib}/qmp.py (70%)
 create mode 100644 python/qemu/lib/qtest.py
 delete mode 100644 python/qemu/qtest.py
 create mode 100755 python/setup.py
Signature · 1 line
2.21.1