Python: packaging cleanups
v3 cover Python: packaging cleanups
This series is a collection of touchups and small fixes to the Python packaging series that was initiated with commit ea1213b7.
It fixes a small handful of annoyances and adjusts some readme files, help text, and so on.
I'd like to make sure this is included prior to the 6.1 RC testing period. (And now, at the very least, patch #01 needs to go in ASAP.)
V3:
001/15:[down] 'python/qom: Do not use 'err' name at module scope' 010/15:[down] 'python: only check qemu/ subdir with flake8' 015/15:[down] 'python: Fix broken ReST docstrings'
- 01: Update library for pylint 2.9.0 which happened ~2hrs ago
- 10: Fix pylint checking stuff it shouldn't (Wainer)
- 15: Also new, it helps me along with Sphinx doc work.
V2:
007/12:[0002] [FC] 'python: update help text for check-tox' 008/12:[down] 'python: Fix .PHONY Make specifiers' 009/12:[down] 'python: add 'make check-dev' invocation'
- 07: Changed '3.6-3.10' to '3.6 - 3.10'.
(Kept Willian's RB)
- 08: New, based on Wainer's observation I had missed 'check-tox'.
- 09: Renamed targets based on Wainer's review.
Status:
[01] python-qom-do-not-use-err-at # [SOB] JS [02] python-expose-typing # [RB] WR,WdSM [SOB] JS [03] python-remove-global-pylint # [RB] WR,WdSM [SOB] JS [04] python-re-lock-pipenv-at # [RB] WR,WdSM [SOB] JS [05] python-readme.rst-touchups # [RB] WR,WdSM [SOB] JS [06] python-add-no-install-usage # [RB] WR,WdSM [SOB] JS [07] python-rename-venv-check # [RB] WR,WdSM [SOB] JS [08] python-update-help-text-for # [RB] WR,WdSM [SOB] JS [09] python-fix-.phony-make # [RB] WR,WdSM [SOB] JS [10] python-only-check-qemu-subdir # [SOB] JS [11] python-add-make-check-venv # [SOB] JS [12] python-update-help-text-on # [RB] WR,WdSM [SOB] JS [13] python-update-help-text-on-1 # [RB] WR,WdSM [SOB] JS [14] python-remove-auto-generated # [RB] WR,WdSM [SOB] JS [15] python-fix-broken-rest # [SOB] JS
John Snow (15):
python/qom: Do not use 'err' name at module scope
python: expose typing information via PEP 561
python: Remove global pylint suppressions
python: Re-lock pipenv at *oldest* supported versions
python: README.rst touchups
python: Add no-install usage instructions
python: rename 'venv-check' target to 'check-pipenv'
python: update help text for check-tox
python: Fix .PHONY Make specifiers
python: only check qemu/ subdir with flake8
python: add 'make check-dev' invocation
python: Update help text on 'make check', 'make develop'
python: Update help text on 'make clean', 'make distclean'
python: remove auto-generated pyproject.toml file
python: Fix broken ReST docstrings
python/README.rst | 47 ++++++++++--- .gitlab-ci.d/static_checks.yml | 2 +- python/Makefile | 89 +++++++++++++++++++------ python/Pipfile.lock | 113 +++++++++++++++----------------- python/qemu/machine/__init__.py | 6 +- python/qemu/machine/machine.py | 6 +- python/qemu/machine/py.typed | 0 python/qemu/machine/qtest.py | 2 + python/qemu/qmp/__init__.py | 1 + python/qemu/qmp/py.typed | 0 python/qemu/qmp/qom.py | 4 +- python/qemu/qmp/qom_common.py | 2 +- python/qemu/utils/accel.py | 2 +- python/qemu/utils/py.typed | 0 python/setup.cfg | 14 ++-- python/tests/flake8.sh | 2 +- 16 files changed, 186 insertions(+), 104 deletions(-) create mode 100644 python/qemu/machine/py.typed create mode 100644 python/qemu/qmp/py.typed create mode 100644 python/qemu/utils/py.typed
Signature · 1 line
2.31.1
PATCH v3 1/15 python/qom: Do not use 'err' name at module scope
Pylint updated to 2.9.0 upstream, adding new warnings for things that re-use the 'err' variable. Luckily, this only breaks the python-check-tox job, which is allowed to fail as a warning.
---
I guess that's good enough evidence that check-pipenv and check-tox both have their place :)
--- python/qemu/qmp/qom.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
python/qemu/qmp/qom.py +2 -2
@@ -38,8 +38,8 @@ try: from .qom_fuse import QOMFuse -except ModuleNotFoundError as err: - if err.name != 'fuse': +except ModuleNotFoundError as _err: + if _err.name != 'fuse': raise else: assert issubclass(QOMFuse, QOMCommand)
Signature · 1 line
2.31.1
Re: [PATCH v3 01/15] python/qom: Do not use 'err' name at module scope
Quoted 8 lines · John Snow
Pylint updated to 2.9.0 upstream, adding new warnings for things that re-use the 'err' variable. Luckily, this only breaks the python-check-tox job, which is allowed to fail as a warning.
Signed-off-by John Snow <[email protected]>---
Quoted 7 lines
I guess that's good enough evidence that check-pipenv and check-tox both have their place :)
Signed-off-by John Snow <[email protected]>--- python/qemu/qmp/qom.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 16 lines
python/qemu/qmp/qom.py +2
-2@@ -38,8 +38,8 @@ try: from .qom_fuse import QOMFuse -except ModuleNotFoundError as err: - if err.name != 'fuse': +except ModuleNotFoundError as _err: + if _err.name != 'fuse': raise else: assert issubclass(QOMFuse, QOMCommand)
Re: [PATCH v3 01/15] python/qom: Do not use 'err' name at module scope
In reply to John SnowQuoted 9 lines · John Snow
Pylint updated to 2.9.0 upstream, adding new warnings for things that re-use the 'err' variable. Luckily, this only breaks the python-check-tox job, which is allowed to fail as a warning.
Signed-off-by John Snow <[email protected]>---
Quoted 8 lines
I guess that's good enough evidence that check-pipenv and check-tox both have their place :)
Signed-off-by John Snow <[email protected]>--- python/qemu/qmp/qom.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
PATCH v3 2/15 python: expose typing information via PEP 561
In reply to John Snowhttps://www.python.org/dev/peps/pep-0561/#specification
Create 'py.typed' files in each subpackage that indicate to mypy that this is a typed module, so that users of any of these packages can use mypy to check their code as well.
Note: Theoretically it's possible to ditch MANIFEST.in in favor of using package_data in setup.cfg, but I genuinely could not figure out how to get it to include things from the *source root* into the *package root*; only how to include things from each subpackage. I tried!
--- python/qemu/machine/py.typed | 0 python/qemu/qmp/py.typed | 0 python/qemu/utils/py.typed | 0 python/setup.cfg | 4 ++++ 4 files changed, 4 insertions(+) create mode 100644 python/qemu/machine/py.typed create mode 100644 python/qemu/qmp/py.typed create mode 100644 python/qemu/utils/py.typed
python/qemu/machine/py.typed new file +0 -0
python/qemu/qmp/py.typed new file +0 -0
python/qemu/utils/py.typed new file +0 -0
python/setup.cfg +4 -0
@@ -19,6 +19,7 @@ classifiers = Programming Language :: Python :: 3.8 Programming Language :: Python :: 3.9 Programming Language :: Python :: 3.10 + Typing :: Typed [options] python_requires = >= 3.6
@@ -27,6 +28,9 @@ packages = qemu.machine qemu.utils +[options.package_data] +* = py.typed + [options.extras_require] # Run `pipenv lock --dev` when changing these requirements. devel =
Signature · 1 line
2.31.1
PATCH v3 3/15 python: Remove global pylint suppressions
In reply to John SnowThese suppressions only apply to a small handful of places. Instead of disabling them globally, disable them just in the cases where we need. The design of the machine class grew quite organically with tons of constructor and class instance variables -- there's little chance of meaningfully refactoring it in the near term, so just suppress the warnings for that class.
--- python/qemu/machine/machine.py | 3 +++ python/qemu/machine/qtest.py | 2 ++ python/setup.cfg | 4 +--- 3 files changed, 6 insertions(+), 3 deletions(-)
python/qemu/machine/machine.py +3 -0
@@ -84,6 +84,7 @@ class QEMUMachine: ... # vm is guaranteed to be shut down here """ + # pylint: disable=too-many-instance-attributes, too-many-public-methods def __init__(self, binary: str,
@@ -111,6 +112,8 @@ def __init__(self, @param console_log: (optional) path to console log file @note: Qemu process is not started until launch() is used. ''' + # pylint: disable=too-many-arguments + # Direct user configuration self._binary = binary
python/qemu/machine/qtest.py +2 -0
@@ -116,6 +116,8 @@ def __init__(self, base_temp_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None, sock_dir: Optional[str] = None): + # pylint: disable=too-many-arguments + if name is None: name = "qemu-%d" % os.getpid() if sock_dir is None:
python/setup.cfg +1 -3
@@ -87,9 +87,7 @@ ignore_missing_imports = True # --enable=similarities". If you want to run only the classes checker, but have # no Warning level messages displayed, use "--disable=all --enable=classes # --disable=W". -disable=too-many-arguments, - too-many-instance-attributes, - too-many-public-methods, +disable= [pylint.basic] # Good variable names which should always be accepted, separated by a comma.
Signature · 1 line
2.31.1
PATCH v3 4/15 python: Re-lock pipenv at *oldest* supported versions
In reply to John Snowtox is already testing the most recent versions. Let's use pipenv to test the oldest versions we claim to support. This matches the stylistic choice to have pipenv always test our oldest supported Python version, 3.6.
The effect of this is that the python-check-pipenv CI job on gitlab will now test against much older versions of these linters, which will help highlight incompatible changes that might otherwise go unnoticed.
Update instructions for adding and bumping versions in setup.cfg. The reason for deleting the line that gets added to Pipfile is largely just to avoid having the version minimums specified in multiple places in config checked into the tree.
(This patch was written by deleting Pipfile and Pipfile.lock, then explicitly installing each dependency manually at a specific version. Then, I restored the prior Pipfile and re-ran pipenv lock --dev --keep-outdated to re-add the qemu dependency back to the pipenv environment while keeping the "old" packages. It's annoying, yes, but I think the improvement to test coverage is worthwhile.)
--- python/Pipfile.lock | 113 +++++++++++++++++++++----------------------- python/setup.cfg | 4 +- 2 files changed, 56 insertions(+), 61 deletions(-)
python/Pipfile.lock +53 -60
@@ -31,19 +31,19 @@ }, "astroid": { "hashes": [ - "sha256:4db03ab5fc3340cf619dbc25e42c2cc3755154ce6009469766d7143d1fc2ee4e", - "sha256:8a398dfce302c13f14bab13e2b14fe385d32b73f4e4853b9bdfb64598baa1975" + "sha256:09bdb456e02564731f8b5957cdd0c98a7f01d2db5e90eb1d794c353c28bfd705", + "sha256:6a8a51f64dae307f6e0c9db752b66a7951e282389d8362cc1d39a56f3feeb31d" ], "markers": "python_version ~= '3.6'", - "version": "==2.5.6" + "version": "==2.6.0" }, "avocado-framework": { "hashes": [ - "sha256:42aa7962df98d6b78d4efd9afa2177226dc630f3d83a2a7d5baf7a0a7da7fa1b", - "sha256:d96ae343abf890e1ef3b3a6af5ce49e35f6bded0715770c4acb325bca555c515" + "sha256:3fca7226d7d164f124af8a741e7fa658ff4345a0738ddc32907631fd688b38ed", + "sha256:48ac254c0ae2ef0c0ceeb38e3d3df0388718eda8f48b3ab55b30b252839f42b1" ], - "markers": "python_version >= '3.6'", - "version": "==88.1" + "index": "pypi", + "version": "==87.0" }, "distlib": { "hashes": [
@@ -61,25 +61,27 @@ }, "flake8": { "hashes": [ - "sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b", - "sha256:bf8fd333346d844f616e8d47905ef3a3384edae6b4e9beb0c5101e25e3110907" + "sha256:6a35f5b8761f45c5513e3405f110a86bea57982c3b75b766ce7b65217abe1670", + "sha256:c01f8a3963b3571a8e6bd7a4063359aff90749e160778e03817cd9b71c9e07d2" ], - "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", - "version": "==3.9.2" + "index": "pypi", + "version": "==3.6.0" }, "fusepy": { "hashes": [ - "sha256:72ff783ec2f43de3ab394e3f7457605bf04c8cf288a2f4068b4cde141d4ee6bd" + "sha256:10f5c7f5414241bffecdc333c4d3a725f1d6605cae6b4eaf86a838ff49cdaf6c", + "sha256:a9f3a3699080ddcf0919fd1eb2cf743e1f5859ca54c2018632f939bdfac269ee" ], - "version": "==3.0.1" + "index": "pypi", + "version": "==2.0.4" }, "importlib-metadata": { "hashes": [ - "sha256:8c501196e49fb9df5df43833bdb1e4328f64847763ec8a50703148b73784d581", - "sha256:d7eb1dea6d6a6086f8be21784cc9e3bcfa55872b52309bc5fad53a8ea444465d" + "sha256:90bb658cdbbf6d1735b6341ce708fc7024a3e14e99ffdc5783edea9f9b077f83", + "sha256:dc15b2969b4ce36305c51eebe62d418ac7791e9a157911d58bfb1f9ccd8e2070" ], "markers": "python_version < '3.8'", - "version": "==4.0.1" + "version": "==1.7.0" }, "importlib-resources": { "hashes": [
@@ -91,11 +93,11 @@ }, "isort": { "hashes": [ - "sha256:0a943902919f65c5684ac4e0154b1ad4fac6dcaa5d9f3426b732f1c8b5419be6", - "sha256:2bb1680aad211e3c9944dbce1d4ba09a989f04e238296c87fe2139faa26d655d" + "sha256:408e4d75d84f51b64d0824894afee44469eba34a4caee621dc53799f80d71ccc", + "sha256:64022dea6a06badfa09b300b4dfe8ba968114a737919e8ed50aea1c288f078aa" ], - "markers": "python_version >= '3.6' and python_version < '4.0'", - "version": "==5.8.0" + "index": "pypi", + "version": "==5.1.2" }, "lazy-object-proxy": { "hashes": [
@@ -134,31 +136,23 @@ }, "mypy": { "hashes": [ - "sha256:0d0a87c0e7e3a9becdfbe936c981d32e5ee0ccda3e0f07e1ef2c3d1a817cf73e", - "sha256:25adde9b862f8f9aac9d2d11971f226bd4c8fbaa89fb76bdadb267ef22d10064", - "sha256:28fb5479c494b1bab244620685e2eb3c3f988d71fd5d64cc753195e8ed53df7c", - "sha256:2f9b3407c58347a452fc0736861593e105139b905cca7d097e413453a1d650b4", - "sha256:33f159443db0829d16f0a8d83d94df3109bb6dd801975fe86bacb9bf71628e97", - "sha256:3f2aca7f68580dc2508289c729bd49ee929a436208d2b2b6aab15745a70a57df", - "sha256:499c798053cdebcaa916eef8cd733e5584b5909f789de856b482cd7d069bdad8", - "sha256:4eec37370483331d13514c3f55f446fc5248d6373e7029a29ecb7b7494851e7a", - "sha256:552a815579aa1e995f39fd05dde6cd378e191b063f031f2acfe73ce9fb7f9e56", - "sha256:5873888fff1c7cf5b71efbe80e0e73153fe9212fafdf8e44adfe4c20ec9f82d7", - "sha256:61a3d5b97955422964be6b3baf05ff2ce7f26f52c85dd88db11d5e03e146a3a6", - "sha256:674e822aa665b9fd75130c6c5f5ed9564a38c6cea6a6432ce47eafb68ee578c5", - "sha256:7ce3175801d0ae5fdfa79b4f0cfed08807af4d075b402b7e294e6aa72af9aa2a", - "sha256:9743c91088d396c1a5a3c9978354b61b0382b4e3c440ce83cf77994a43e8c521", - "sha256:9f94aac67a2045ec719ffe6111df543bac7874cee01f41928f6969756e030564", - "sha256:a26f8ec704e5a7423c8824d425086705e381b4f1dfdef6e3a1edab7ba174ec49", - "sha256:abf7e0c3cf117c44d9285cc6128856106183938c68fd4944763003decdcfeb66", - "sha256:b09669bcda124e83708f34a94606e01b614fa71931d356c1f1a5297ba11f110a", - "sha256:cd07039aa5df222037005b08fbbfd69b3ab0b0bd7a07d7906de75ae52c4e3119", - "sha256:d23e0ea196702d918b60c8288561e722bf437d82cb7ef2edcd98cfa38905d506", - "sha256:d65cc1df038ef55a99e617431f0553cd77763869eebdf9042403e16089fe746c", - "sha256:d7da2e1d5f558c37d6e8c1246f1aec1e7349e4913d8fb3cb289a35de573fe2eb" + "sha256:15b948e1302682e3682f11f50208b726a246ab4e6c1b39f9264a8796bb416aa2", + "sha256:219a3116ecd015f8dca7b5d2c366c973509dfb9a8fc97ef044a36e3da66144a1", + "sha256:3b1fc683fb204c6b4403a1ef23f0b1fac8e4477091585e0c8c54cbdf7d7bb164", + "sha256:3beff56b453b6ef94ecb2996bea101a08f1f8a9771d3cbf4988a61e4d9973761", + "sha256:7687f6455ec3ed7649d1ae574136835a4272b65b3ddcf01ab8704ac65616c5ce", + "sha256:7ec45a70d40ede1ec7ad7f95b3c94c9cf4c186a32f6bacb1795b60abd2f9ef27", + "sha256:86c857510a9b7c3104cf4cde1568f4921762c8f9842e987bc03ed4f160925754", + "sha256:8a627507ef9b307b46a1fea9513d5c98680ba09591253082b4c48697ba05a4ae", + "sha256:8dfb69fbf9f3aeed18afffb15e319ca7f8da9642336348ddd6cab2713ddcf8f9", + "sha256:a34b577cdf6313bf24755f7a0e3f3c326d5c1f4fe7422d1d06498eb25ad0c600", + "sha256:a8ffcd53cb5dfc131850851cc09f1c44689c2812d0beb954d8138d4f5fc17f65", + "sha256:b90928f2d9eb2f33162405f32dde9f6dcead63a0971ca8a1b50eb4ca3e35ceb8", + "sha256:c56ffe22faa2e51054c5f7a3bc70a370939c2ed4de308c690e7949230c995913", + "sha256:f91c7ae919bbc3f96cd5e5b2e786b2b108343d1d7972ea130f7de27fdd547cf3" ], - "markers": "python_version >= '3.5'", - "version": "==0.812" + "index": "pypi", + "version": "==0.770" }, "mypy-extensions": { "hashes": [
@@ -193,27 +187,26 @@ }, "pycodestyle": { "hashes": [ - "sha256:514f76d918fcc0b55c6680472f0a37970994e07bbb80725808c17089be302068", - "sha256:c389c1d06bf7904078ca03399a4816f974a1d590090fecea0c63ec26ebaf1cef" + "sha256:74abc4e221d393ea5ce1f129ea6903209940c1ecd29e002e8c6933c2b21026e0", + "sha256:cbc619d09254895b0d12c2c691e237b2e91e9b2ecf5e84c26b35400f93dcfb83", + "sha256:cbfca99bd594a10f674d0cd97a3d802a1fdef635d4361e1a2658de47ed261e3a" ], - "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", - "version": "==2.7.0" + "version": "==2.4.0" }, "pyflakes": { "hashes": [ - "sha256:7893783d01b8a89811dd72d7dfd4d84ff098e5eed95cfa8905b22bbffe52efc3", - "sha256:f5bc8ecabc05bb9d291eb5203d6810b49040f6ff446a756326104746cc00c1db" + "sha256:9a7662ec724d0120012f6e29d6248ae3727d821bba522a0e6b356eff19126a49", + "sha256:f661252913bc1dbe7fcfcbf0af0db3f42ab65aabd1a6ca68fe5d466bace94dae" ], - "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", - "version": "==2.3.1" + "version": "==2.0.0" }, "pylint": { "hashes": [ - "sha256:586d8fa9b1891f4b725f587ef267abe2a1bad89d6b184520c7f07a253dd6e217", - "sha256:f7e2072654a6b6afdf5e2fb38147d3e2d2d43c89f648637baab63e026481279b" + "sha256:082a6d461b54f90eea49ca90fff4ee8b6e45e8029e5dbd72f6107ef84f3779c0", + "sha256:a01cd675eccf6e25b3bdb42be184eb46aaf89187d612ba0fb5f93328ed6b0fd5" ], - "markers": "python_version ~= '3.6'", - "version": "==2.8.2" + "index": "pypi", + "version": "==2.8.0" }, "pyparsing": { "hashes": [
@@ -245,11 +238,11 @@ }, "tox": { "hashes": [ - "sha256:307a81ddb82bd463971a273f33e9533a24ed22185f27db8ce3386bff27d324e3", - "sha256:b0b5818049a1c1997599d42012a637a33f24c62ab8187223fdd318fa8522637b" + "sha256:c60692d92fe759f46c610ac04c03cf0169432d1ff8e981e8ae63e068d0954fc3", + "sha256:f179cb4043d7dc1339425dd49ab1dd8c916246b0d9173143c1b0af7498a03ab0" ], - "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", - "version": "==3.23.1" + "index": "pypi", + "version": "==3.18.0" }, "typed-ast": { "hashes": [
python/setup.cfg +3 -1
@@ -32,7 +32,9 @@ packages = * = py.typed [options.extras_require] -# Run `pipenv lock --dev` when changing these requirements. +# For the devel group, When adding new dependencies or bumping the minimum +# version, use e.g. "pipenv install --dev pylint==3.0.0". +# Subsequently, edit 'Pipfile' to remove e.g. 'pylint = "==3.0.0'. devel = avocado-framework >= 87.0 flake8 >= 3.6.0
Signature · 1 line
2.31.1
PATCH v3 5/15 python: README.rst touchups
In reply to John SnowClarifying a few points; removing the reference to 'setuptools' because it isn't referenced anywhere else in this document and doesn't really provide any useful information to a Python newcomer.
Adjusting the language elsewhere to be less ambiguous and have fewer run-on sentences.
--- python/README.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
python/README.rst +9 -8
@@ -7,8 +7,7 @@ then by package (e.g. ``qemu/machine``, ``qemu/qmp``, etc). ``setup.py`` is used by ``pip`` to install this tooling to the current environment. ``setup.cfg`` provides the packaging configuration used by -``setup.py`` in a setuptools specific format. You will generally invoke -it by doing one of the following: +``setup.py``. You will generally invoke it by doing one of the following: 1. ``pip3 install .`` will install these packages to your current environment. If you are inside a virtual environment, they will
@@ -17,12 +16,13 @@ it by doing one of the following: 2. ``pip3 install --user .`` will install these packages to your user's local python packages. If you are inside of a virtual environment, - this will fail; you likely want the first invocation above. + this will fail; you want the first invocation above. -If you append the ``-e`` argument, pip will install in "editable" mode; -which installs a version of the package that installs a forwarder -pointing to these files, such that the package always reflects the -latest version in your git tree. +If you append the ``--editable`` or ``-e`` argument to either invocation +above, pip will install in "editable" mode. This installs the package as +a forwarder ("qemu.egg-link") that points to the source tree. In so +doing, the installed package always reflects the latest version in your +source tree. Installing ".[devel]" instead of "." will additionally pull in required packages for testing this package. They are not runtime requirements,
@@ -30,6 +30,7 @@ and are not needed to simply use these libraries. Running ``make develop`` will pull in all testing dependencies and install QEMU in editable mode to the current environment. +(It is a shortcut for ``pip3 install -e .[devel]``.) See `Installing packages using pip and virtual environments <https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/>`_
@@ -39,7 +40,7 @@ for more information. Files in this directory ----------------------- -- ``qemu/`` Python package source directory. +- ``qemu/`` Python 'qemu' namespace package source directory. - ``tests/`` Python package tests directory. - ``avocado.cfg`` Configuration for the Avocado test-runner. Used by ``make check`` et al.
Signature · 1 line
2.31.1
PATCH v3 6/15 python: Add no-install usage instructions
In reply to John SnowIt's not encouraged, but it's legitimate to want to know how to do.
--- python/README.rst | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
python/README.rst +28 -0
@@ -37,6 +37,34 @@ See `Installing packages using pip and virtual environments for more information. +Using these packages without installing them +-------------------------------------------- + +These packages may be used without installing them first, by using one +of two tricks: + +1. Set your PYTHONPATH environment variable to include this source + directory, e.g. ``~/src/qemu/python``. See + https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH + +2. Inside a Python script, use ``sys.path`` to forcibly include a search + path prior to importing the ``qemu`` namespace. See + https://docs.python.org/3/library/sys.html#sys.path + +A strong downside to both approaches is that they generally interfere +with static analysis tools being able to locate and analyze the code +being imported. + +Package installation also normally provides executable console scripts, +so that tools like ``qmp-shell`` are always available via $PATH. To +invoke them without installation, you can invoke e.g.: + +``> PYTHONPATH=~/src/qemu/python python3 -m qemu.qmp.qmp_shell`` + +The mappings between console script name and python module path can be +found in ``setup.cfg``. + + Files in this directory -----------------------
Signature · 1 line
2.31.1
PATCH v3 7/15 python: rename 'venv-check' target to 'check-pipenv'
In reply to John SnowWell, Cleber was right, this is a better name.
In preparation for adding a different kind of virtual environment check (One that simply uses whichever version of Python you happen to have), rename this test 'check-pipenv' so that it matches the CI job 'check-python-pipenv'.
Remove the "If you don't know which test to run" hint, because it's not actually likely you have Python 3.6 installed to be able to run the test. It's still the test I'd most prefer you to run, but it's not the test you are most likely to be able to run.
Rename the 'venv' target to 'pipenv' as well, and move the more pertinent help text under the 'check-pipenv' target.
--- python/README.rst | 2 +- .gitlab-ci.d/static_checks.yml | 2 +- python/Makefile | 21 +++++++++++---------- 3 files changed, 13 insertions(+), 12 deletions(-)
python/README.rst +1 -1
@@ -79,7 +79,7 @@ Files in this directory - ``PACKAGE.rst`` is used as the README file that is visible on PyPI.org. - ``Pipfile`` is used by Pipenv to generate ``Pipfile.lock``. - ``Pipfile.lock`` is a set of pinned package dependencies that this package - is tested under in our CI suite. It is used by ``make venv-check``. + is tested under in our CI suite. It is used by ``make check-pipenv``. - ``README.rst`` you are here! - ``VERSION`` contains the PEP-440 compliant version used to describe this package; it is referenced by ``setup.cfg``.
.gitlab-ci.d/static_checks.yml +1 -1
@@ -30,7 +30,7 @@ check-python-pipenv: stage: test image: $CI_REGISTRY_IMAGE/qemu/python:latest script: - - make -C python venv-check + - make -C python check-pipenv variables: GIT_DEPTH: 1 needs:
python/Makefile +11 -10
@@ -1,15 +1,13 @@ -.PHONY: help venv venv-check check clean distclean develop +.PHONY: help pipenv check-pipenv check clean distclean develop help: @echo "python packaging help:" @echo "" - @echo "make venv: Create pipenv's virtual environment." - @echo " NOTE: Requires Python 3.6 and pipenv." - @echo " Will download packages from PyPI." - @echo " Hint: (On Fedora): 'sudo dnf install python36 pipenv'" - @echo "" - @echo "make venv-check: run linters using pipenv's virtual environment." - @echo " Hint: If you don't know which test to run, run this one!" + @echo "make check-pipenv:" + @echo " Run tests in pipenv's virtual environment." + @echo " These tests use the oldest dependencies." + @echo " Requires: Python 3.6 and pipenv." + @echo " Hint (Fedora): 'sudo dnf install python3.6 pipenv'" @echo "" @echo "make develop: Install deps for 'make check', and" @echo " the qemu libs in editable/development mode."
@@ -18,18 +16,21 @@ help: @echo "" @echo "make check-tox: run linters using multiple python versions." @echo "" + @echo "make pipenv" + @echo " Creates pipenv's virtual environment (.venv)" + @echo "" @echo "make clean: remove package build output." @echo "" @echo "make distclean: remove venv files, qemu package forwarder," @echo " built distribution files, and everything" @echo " from 'make clean'." -venv: .venv +pipenv: .venv .venv: Pipfile.lock @PIPENV_VENV_IN_PROJECT=1 pipenv sync --dev --keep-outdated @touch .venv -venv-check: venv +check-pipenv: pipenv @pipenv run make check develop:
Signature · 1 line
2.31.1
PATCH v3 8/15 python: update help text for check-tox
In reply to John SnowMove it up near the check-pipenv help text, and update it to suggest parity.
(At the time I first added it, I wasn't sure if I would be keeping it, but I've come to appreciate it as it has actually helped uncover bugs I would not have noticed without it. It should stay.)
--- python/Makefile | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
python/Makefile +6 -2
@@ -9,13 +9,17 @@ help: @echo " Requires: Python 3.6 and pipenv." @echo " Hint (Fedora): 'sudo dnf install python3.6 pipenv'" @echo "" + @echo "make check-tox:" + @echo " Run tests against multiple python versions." + @echo " These tests use the newest dependencies." + @echo " Requires: Python 3.6 - 3.10, and tox." + @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.10'" + @echo "" @echo "make develop: Install deps for 'make check', and" @echo " the qemu libs in editable/development mode." @echo "" @echo "make check: run linters using the current environment." @echo "" - @echo "make check-tox: run linters using multiple python versions." - @echo "" @echo "make pipenv" @echo " Creates pipenv's virtual environment (.venv)" @echo ""
Signature · 1 line
2.31.1
PATCH v3 9/15 python: Fix .PHONY Make specifiers
In reply to John SnowI missed the 'check-tox' target. Add that, but split the large .PHONY specifier at the top into its component pieces and move them near the targets they describe so that they're much harder to forget to update.
--- python/Makefile | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
python/Makefile +8 -2
@@ -1,5 +1,4 @@ -.PHONY: help pipenv check-pipenv check clean distclean develop - +.PHONY: help help: @echo "python packaging help:" @echo ""
@@ -29,25 +28,32 @@ help: @echo " built distribution files, and everything" @echo " from 'make clean'." +.PHONY: pipenv pipenv: .venv .venv: Pipfile.lock @PIPENV_VENV_IN_PROJECT=1 pipenv sync --dev --keep-outdated @touch .venv +.PHONY: check-pipenv check-pipenv: pipenv @pipenv run make check +.PHONY: develop develop: pip3 install -e .[devel] +.PHONY: check check: @avocado --config avocado.cfg run tests/ +.PHONY: check-tox check-tox: @tox +.PHONY: clean clean: python3 setup.py clean --all +.PHONY: distclean distclean: clean rm -rf qemu.egg-info/ .venv/ .tox/ dist/
Signature · 1 line
2.31.1
PATCH v3 10/15 python: only check qemu/ subdir with flake8
In reply to John Snowflake8 is a little eager to check everything it can. Limit it to checking inside the qemu namespace directory only. Update setup.cfg now that the exclude patterns are no longer necessary.
--- python/setup.cfg | 2 -- python/tests/flake8.sh | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-)
python/setup.cfg +0 -2
@@ -62,8 +62,6 @@ console_scripts = [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's exclude = __pycache__, - .venv, - .tox, [mypy] strict = True
python/tests/flake8.sh +1 -1
@@ -1,2 +1,2 @@ #!/bin/sh -e -python3 -m flake8 +python3 -m flake8 qemu/
Signature · 1 line
2.31.1
Re: [PATCH v3 10/15] python: only check qemu/ subdir with flake8
Quoted 11 lines · John Snow
flake8 is a little eager to check everything it can. Limit it to checking inside the qemu namespace directory only. Update setup.cfg now that the exclude patterns are no longer necessary.
Signed-off-by John Snow <[email protected]>--- python/setup.cfg | 2 -- python/tests/flake8.sh | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-)
Re: [PATCH v3 10/15] python: only check qemu/ subdir with flake8
In reply to John SnowQuoted 9 lines · John Snow
flake8 is a little eager to check everything it can. Limit it to checking inside the qemu namespace directory only. Update setup.cfg now that the exclude patterns are no longer necessary.
Signed-off-by John Snow <[email protected]>--- python/setup.cfg | 2 -- python/tests/flake8.sh | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-)
How I tested it:
$ make check-dev VENV .dev-venv ACTIVATE .dev-venv INSTALL qemu[devel] .dev-venv ACTIVATE .dev-venv make[1]: Entering directory '/home/wmoschet/src/qemu/python' JOB ID : 41162d4a9e93095abb66c219227b6af27edbcf43 JOB LOG : /home/wmoschet/avocado/job-results/job-2021-06-30T11.50-41162d4/job.log (1/4) tests/flake8.sh: PASS (0.30 s) (2/4) tests/isort.sh: PASS (0.12 s) (3/4) tests/mypy.sh: PASS (0.32 s) (4/4) tests/pylint.sh: PASS (4.46 s) RESULTS : PASS 4 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 8.46 s make[1]: Leaving directory '/home/wmoschet/src/qemu/python'
Quoted 22 lines
python/setup.cfg +0
-2@@ -62,8 +62,6 @@ console_scripts = [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's exclude = __pycache__, - .venv, - .tox, [mypy] strict = Truepython/tests/flake8.sh +1
-1@@ -1,2 +1,2 @@ #!/bin/sh -e -python3 -m flake8 +python3 -m flake8 qemu/
PATCH v3 11/15 python: add 'make check-dev' invocation
In reply to John SnowThis is a *third* way to run the Python tests. Unlike the first two (check-pipenv, check-tox), this version does not require any specific interpreter version -- making it a lot easier to tell people to run it as a quick smoketest prior to submission to GitLab CI.
Summary:
Checked via GitLab CI:
- check-pipenv: tests our oldest python & dependencies
- check-tox: tests newest dependencies on all non-EOL python versions
Executed only incidentally:
- check-dev: tests newest dependencies on whichever python version
('make check' does not set up any environment at all, it just runs the tests in your current environment. All four invocations perform the exact same tests, just in different execution environments.)
--- python/Makefile | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)
python/Makefile +33 -2
@@ -1,3 +1,5 @@ +QEMU_VENV_DIR=.dev-venv + .PHONY: help help: @echo "python packaging help:"
@@ -14,6 +16,11 @@ help: @echo " Requires: Python 3.6 - 3.10, and tox." @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.10'" @echo "" + @echo "make check-dev:" + @echo " Run tests in a venv against your default python3 version." + @echo " These tests use the newest dependencies." + @echo " Requires: Python 3.x" + @echo "" @echo "make develop: Install deps for 'make check', and" @echo " the qemu libs in editable/development mode." @echo ""
@@ -22,6 +29,9 @@ help: @echo "make pipenv" @echo " Creates pipenv's virtual environment (.venv)" @echo "" + @echo "make dev-venv" + @echo " Creates a simple venv for check-dev. ($(QEMU_VENV_DIR))" + @echo "" @echo "make clean: remove package build output." @echo "" @echo "make distclean: remove venv files, qemu package forwarder,"
@@ -38,9 +48,30 @@ pipenv: .venv check-pipenv: pipenv @pipenv run make check +.PHONY: dev-venv +dev-venv: $(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate +$(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate: setup.cfg + @echo "VENV $(QEMU_VENV_DIR)" + @python3 -m venv $(QEMU_VENV_DIR) + @( \ + echo "ACTIVATE $(QEMU_VENV_DIR)"; \ + . $(QEMU_VENV_DIR)/bin/activate; \ + echo "INSTALL qemu[devel] $(QEMU_VENV_DIR)"; \ + make develop 1>/dev/null; \ + ) + @touch $(QEMU_VENV_DIR) + +.PHONY: check-dev +check-dev: dev-venv + @( \ + echo "ACTIVATE $(QEMU_VENV_DIR)"; \ + . $(QEMU_VENV_DIR)/bin/activate; \ + make check; \ + ) + .PHONY: develop develop: - pip3 install -e .[devel] + pip3 install --disable-pip-version-check -e .[devel] .PHONY: check check:
@@ -56,4 +87,4 @@ clean: .PHONY: distclean distclean: clean - rm -rf qemu.egg-info/ .venv/ .tox/ dist/ + rm -rf qemu.egg-info/ .venv/ .tox/ $(QEMU_VENV_DIR) dist/
Signature · 1 line
2.31.1
Re: [PATCH v3 11/15] python: add 'make check-dev' invocation
Quoted 14 lines · John Snow
This is a *third* way to run the Python tests. Unlike the first two (check-pipenv, check-tox), this version does not require any specific interpreter version -- making it a lot easier to tell people to run it as a quick smoketest prior to submission to GitLab CI.
Summary:
Checked via GitLab CI:
- check-pipenv: tests our oldest python & dependencies
- check-tox: tests newest dependencies on all non-EOL python versions
Executed only incidentally:
- check-dev: tests newest dependencies on whichever python version
Quoted 9 lines
('make check' does not set up any environment at all, it just runs the tests in your current environment. All four invocations perform the exact same tests, just in different execution environments.)
Signed-off-by John Snow <[email protected]>--- python/Makefile | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)
Re: [PATCH v3 11/15] python: add 'make check-dev' invocation
Quoted 19 lines · Willian Rampazzo
Quoted 18 lines · John Snow
This is a *third* way to run the Python tests. Unlike the first two (check-pipenv, check-tox), this version does not require any specific interpreter version -- making it a lot easier to tell people to run it as a quick smoketest prior to submission to GitLab CI.
Summary:
Checked via GitLab CI:
- check-pipenv: tests our oldest python & dependencies
- check-tox: tests newest dependencies on all non-EOL python versions
Executed only incidentally:
- check-dev: tests newest dependencies on whichever python version('make check' does not set up any environment at all, it just runs the tests in your current environment. All four invocations perform the exact same tests, just in different execution environments.)
Quoted 9 lines
Quoted 5 lines
Signed-off-by John Snow <[email protected]>--- python/Makefile | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)Reviewed-by Willian Rampazzo <[email protected]>
Thanks. I am squashing in a hotfix here to add .dev-venv to .gitignore, too. Not worth an entire respin for that.
(Assuming that's gonna be A-OK with both of you.)
Wainer dos Santos Moschetta
Acked-byRe: [PATCH v3 11/15] python: add 'make check-dev' invocation
Quoted 36 lines · John Snow
On Wed, Jun 30, 2021 at 10:41 AM Willian Rampazzo <[email protected] <mailto:[email protected]>> wrote:
On Tue, Jun 29, 2021 at 6:43 PM John Snow <[email protected] <mailto:[email protected]>> wrote:
>
> This is a *third* way to run the Python tests. Unlike the first two > (check-pipenv, check-tox), this version does not require any specific
> interpreter version -- making it a lot easier to tell people to run it
> as a quick smoketest prior to submission to GitLab CI. >
> Summary:
>
> Checked via GitLab CI:
> - check-pipenv: tests our oldest python & dependencies > - check-tox: tests newest dependencies on all non-EOL python versions
> Executed only incidentally:
> - check-dev: tests newest dependencies on whichever python version
>
> ('make check' does not set up any environment at all, it just runs the
> tests in your current environment. All four invocations perform the > exact same tests, just in different execution environments.) >
> Signed-off-by: John Snow <[email protected] <mailto:[email protected]>>
> ---
> python/Makefile | 35 +++++++++++++++++++++++++++++++++-- > 1 file changed, 33 insertions(+), 2 deletions(-)
>
Quoted 6 lines
Reviewed-by Willian Rampazzo <[email protected]<mailto:[email protected]>>
Thanks. I am squashing in a hotfix here to add .dev-venv to .gitignore, too. Not worth an entire respin for that. (Assuming that's gonna be A-OK with both of you.)
Willian Rampazzo
Acked-byRe: [PATCH v3 11/15] python: add 'make check-dev' invocation
In reply to John SnowQuoted 30 lines · John Snow
Quoted 26 lines · Willian Rampazzo
Quoted 23 lines · John Snow
This is a *third* way to run the Python tests. Unlike the first two (check-pipenv, check-tox), this version does not require any specific interpreter version -- making it a lot easier to tell people to run it as a quick smoketest prior to submission to GitLab CI.
Summary:
Checked via GitLab CI:
- check-pipenv: tests our oldest python & dependencies
- check-tox: tests newest dependencies on all non-EOL python versions
Executed only incidentally:
- check-dev: tests newest dependencies on whichever python version('make check' does not set up any environment at all, it just runs the tests in your current environment. All four invocations perform the exact same tests, just in different execution environments.)
Signed-off-by John Snow <[email protected]>--- python/Makefile | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)
Quoted 6 lines
Quoted 2 lines
Reviewed-by Willian Rampazzo <[email protected]>Thanks. I am squashing in a hotfix here to add .dev-venv to .gitignore, too. Not worth an entire respin for that. (Assuming that's gonna be A-OK with both of you.)
Re: [PATCH v3 11/15] python: add 'make check-dev' invocation
In reply to John SnowQuoted 13 lines · John Snow
This is a *third* way to run the Python tests. Unlike the first two (check-pipenv, check-tox), this version does not require any specific interpreter version -- making it a lot easier to tell people to run it as a quick smoketest prior to submission to GitLab CI.
Summary:
Checked via GitLab CI:
- check-pipenv: tests our oldest python & dependencies
- check-tox: tests newest dependencies on all non-EOL python versions
Executed only incidentally:
- check-dev: tests newest dependencies on whichever python version
Quoted 8 lines
('make check' does not set up any environment at all, it just runs the tests in your current environment. All four invocations perform the exact same tests, just in different execution environments.)
Signed-off-by John Snow <[email protected]>--- python/Makefile | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)
It was tested as I mentioned on patch 10.
Quoted 71 lines
python/Makefile +33
-2@@ -1,3 +1,5 @@ +QEMU_VENV_DIR=.dev-venv + .PHONY: help help: @echo "python packaging help:"@@ -14,6 +16,11 @@ help: @echo " Requires: Python 3.6 - 3.10, and tox." @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.10'" @echo "" + @echo "make check-dev:" + @echo " Run tests in a venv against your default python3 version." + @echo " These tests use the newest dependencies." + @echo " Requires: Python 3.x" + @echo "" @echo "make develop: Install deps for 'make check', and" @echo " the qemu libs in editable/development mode." @echo ""@@ -22,6 +29,9 @@ help: @echo "make pipenv" @echo " Creates pipenv's virtual environment (.venv)" @echo "" + @echo "make dev-venv" + @echo " Creates a simple venv for check-dev. ($(QEMU_VENV_DIR))" + @echo "" @echo "make clean: remove package build output." @echo "" @echo "make distclean: remove venv files, qemu package forwarder,"@@ -38,9 +48,30 @@ pipenv: .venv check-pipenv: pipenv @pipenv run make check +.PHONY: dev-venv +dev-venv: $(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate +$(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate: setup.cfg + @echo "VENV $(QEMU_VENV_DIR)" + @python3 -m venv $(QEMU_VENV_DIR) + @( \ + echo "ACTIVATE $(QEMU_VENV_DIR)"; \ + . $(QEMU_VENV_DIR)/bin/activate; \ + echo "INSTALL qemu[devel] $(QEMU_VENV_DIR)"; \ + make develop 1>/dev/null; \ + ) + @touch $(QEMU_VENV_DIR) + +.PHONY: check-dev +check-dev: dev-venv + @( \ + echo "ACTIVATE $(QEMU_VENV_DIR)"; \ + . $(QEMU_VENV_DIR)/bin/activate; \ + make check; \ + ) + .PHONY: develop develop: - pip3 install -e .[devel] + pip3 install --disable-pip-version-check -e .[devel] .PHONY: check check:@@ -56,4 +87,4 @@ clean: .PHONY: distclean distclean: clean - rm -rf qemu.egg-info/ .venv/ .tox/ dist/ + rm -rf qemu.egg-info/ .venv/ .tox/ $(QEMU_VENV_DIR) dist/
PATCH v3 12/15 python: Update help text on 'make check', 'make develop'
In reply to John SnowUpdate for visual parity with the other targets.
--- python/Makefile | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
python/Makefile +7 -3
@@ -21,10 +21,14 @@ help: @echo " These tests use the newest dependencies." @echo " Requires: Python 3.x" @echo "" - @echo "make develop: Install deps for 'make check', and" - @echo " the qemu libs in editable/development mode." + @echo "make check:" + @echo " Run tests in your *current environment*." + @echo " Performs no environment setup of any kind." @echo "" - @echo "make check: run linters using the current environment." + @echo "make develop:" + @echo " Install deps needed for for 'make check'," + @echo " and install the qemu package in editable mode." + @echo " (Can be used in or outside of a venv.)" @echo "" @echo "make pipenv" @echo " Creates pipenv's virtual environment (.venv)"
Signature · 1 line
2.31.1
PATCH v3 13/15 python: Update help text on 'make clean', 'make distclean'
In reply to John SnowUpdate for visual parity with all the remaining targets.
--- python/Makefile | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
python/Makefile +7 -4
@@ -36,11 +36,14 @@ help: @echo "make dev-venv" @echo " Creates a simple venv for check-dev. ($(QEMU_VENV_DIR))" @echo "" - @echo "make clean: remove package build output." + @echo "make clean:" + @echo " Remove package build output." @echo "" - @echo "make distclean: remove venv files, qemu package forwarder," - @echo " built distribution files, and everything" - @echo " from 'make clean'." + @echo "make distclean:" + @echo " remove pipenv/venv files, qemu package forwarder," + @echo " built distribution files, and everything from 'make clean'." + @echo "" + @echo -e "Have a nice day ^_^\n" .PHONY: pipenv pipenv: .venv
Signature · 1 line
2.31.1
PATCH v3 14/15 python: remove auto-generated pyproject.toml file
In reply to John SnowFor reasons that at-present escape me, pipenv insists on creating a stub pyproject.toml file. This file is a nuisance, because its mere presence changes the behavior of various tools.
For instance, this stub file will cause "pip install --user -e ." to fail in spectacular fashion with misleading errors. "pip install -e ." works okay, but for some reason pip does not support editable installs to the user directory when using PEP517.
References:
https://github.com/pypa/pip/pull/9990
https://github.com/pypa/pip/issues/7953
As outlined in ea1213b7ccc, it is still too early for us to consider moving to a PEP-517 exclusive package. We must support older distributions, so squash the annoyance for now. (Python 3.6 shipped Dec 2016, PEP517 support showed up in pip sometime in 2019 or so.)
Add 'pyproject.toml' to the 'make clean' target, and also delete it after every pipenv invocation issued by the Makefile.
--- python/Makefile | 2 ++ 1 file changed, 2 insertions(+)
python/Makefile +2 -0
@@ -49,6 +49,7 @@ help: pipenv: .venv .venv: Pipfile.lock @PIPENV_VENV_IN_PROJECT=1 pipenv sync --dev --keep-outdated + rm -f pyproject.toml @touch .venv .PHONY: check-pipenv
@@ -91,6 +92,7 @@ check-tox: .PHONY: clean clean: python3 setup.py clean --all + rm -f pyproject.toml .PHONY: distclean distclean: clean
Signature · 1 line
2.31.1
PATCH v3 15/15 python: Fix broken ReST docstrings
In reply to John SnowThis patch *doesn't* update all of the docstring standards across the QEMU package directory to make our docstring usage consistent. It *doesn't* fix the formatting to make it look pretty or reasonable in generated output. It *does* fix a few small instances where Sphinx would emit a build warning because of malformed ReST -- If we built our Python docs with Sphinx.
---
You'll have to take my word for it for now, or, to test that (ugly though it may be) a theoretical Sphinx build would produce no build errors:
Quoted 2 lines
cd ~/src/qemu/python
sphinx-apidoc --separate --private --no-toc --module-first \
--implicit-namespaces --full --ext-intersphinx --ext-coverage \
--ext-viewcode qemu/ -o docs/
Quoted 3 lines
sed -i '1s|^|import os; import sys;
sys.path.insert(0, os.path.abspath("../"))\n|'docs/conf.pymake -C docs html
rm -rf docs/
I am preparing to add Sphinx, but need to fix these annoyances first so that regressions are easy to spot as fixes are applied across the tree. I plan to use my forthcoming Asynchronous QMP series as a test pilot for documenting our docstring standards. Assuming it goes well, I will update the docstrings elsewhere in this package at that time.
--- python/qemu/machine/__init__.py | 6 +++--- python/qemu/machine/machine.py | 3 ++- python/qemu/qmp/__init__.py | 1 + python/qemu/qmp/qom_common.py | 2 +- python/qemu/utils/accel.py | 2 +- 5 files changed, 8 insertions(+), 6 deletions(-)
python/qemu/machine/__init__.py +3 -3
@@ -4,10 +4,10 @@ This library provides a few high-level classes for driving QEMU from a test suite, not intended for production use. -- QEMUMachine: Configure and Boot a QEMU VM - - QEMUQtestMachine: VM class, with a qtest socket. + | QEMUQtestProtocol: send/receive qtest messages. + | QEMUMachine: Configure and Boot a QEMU VM + | +-- QEMUQtestMachine: VM class, with a qtest socket. -- QEMUQtestProtocol: Connect to, send/receive qtest messages. """ # Copyright (C) 2020-2021 John Snow for Red Hat Inc.
python/qemu/machine/machine.py +2 -1
@@ -545,7 +545,8 @@ def set_qmp_monitor(self, enabled: bool = True) -> None: @param enabled: if False, qmp monitor options will be removed from the base arguments of the resulting QEMU command line. Default is True. - @note: call this function before launch(). + + .. note:: Call this function before launch(). """ self._qmp_set = enabled
python/qemu/qmp/__init__.py +1 -0
@@ -279,6 +279,7 @@ def accept(self, timeout: Optional[float] = 15.0) -> QMPMessage: None). The value passed will set the behavior of the underneath QMP socket as described in [1]. Default value is set to 15.0. + @return QMP greeting dict @raise OSError on socket connection errors @raise QMPConnectError if the greeting is not received
python/qemu/qmp/qom_common.py +1 -1
@@ -156,7 +156,7 @@ def command_runner( """ Run a fully-parsed subcommand, with error-handling for the CLI. - :return: The return code from `.run()`. + :return: The return code from `run()`. """ try: cmd = cls(args)
python/qemu/utils/accel.py +1 -1
@@ -36,7 +36,7 @@ def list_accel(qemu_bin: str) -> List[str]: List accelerators enabled in the QEMU binary. @param qemu_bin (str): path to the QEMU binary. - @raise Exception: if failed to run `qemu -accel help` + @raise Exception: if failed to run ``qemu -accel help`` @return a list of accelerator names. """ if not qemu_bin:
Signature · 1 line
2.31.1
Re: [PATCH v3 15/15] python: Fix broken ReST docstrings
Quoted 30 lines · John Snow
This patch *doesn't* update all of the docstring standards across the QEMU package directory to make our docstring usage consistent. It *doesn't* fix the formatting to make it look pretty or reasonable in generated output. It *does* fix a few small instances where Sphinx would emit a build warning because of malformed ReST -- If we built our Python docs with Sphinx.
Signed-off-by John Snow <[email protected]>---
You'll have to take my word for it for now, or, to test that (ugly though it may be) a theoretical Sphinx build would produce no build errors:
Quoted 2 lines
cd ~/src/qemu/python
sphinx-apidoc --separate --private --no-toc --module-first \--implicit-namespaces --full --ext-intersphinx --ext-coverage \
--ext-viewcode qemu/ -o docs/Quoted 3 lines
sed -i '1s|^|import os; import sys;
sys.path.insert(0, os.path.abspath("../"))\n|'docs/conf.pymake -C docs html
rm -rf docs/I am preparing to add Sphinx, but need to fix these annoyances first so that regressions are easy to spot as fixes are applied across the tree. I plan to use my forthcoming Asynchronous QMP series as a test pilot for documenting our docstring standards. Assuming it goes well, I will update the docstrings elsewhere in this package at that time.
Quoted 9 lines
Signed-off-by John Snow <[email protected]>--- python/qemu/machine/__init__.py | 6 +++--- python/qemu/machine/machine.py | 3 ++- python/qemu/qmp/__init__.py | 1 + python/qemu/qmp/qom_common.py | 2 +- python/qemu/utils/accel.py | 2 +- 5 files changed, 8 insertions(+), 6 deletions(-)
Re: [PATCH v3 15/15] python: Fix broken ReST docstrings
In reply to John SnowQuoted 28 lines · John Snow
This patch *doesn't* update all of the docstring standards across the QEMU package directory to make our docstring usage consistent. It *doesn't* fix the formatting to make it look pretty or reasonable in generated output. It *does* fix a few small instances where Sphinx would emit a build warning because of malformed ReST -- If we built our Python docs with Sphinx.
Signed-off-by John Snow <[email protected]>---
You'll have to take my word for it for now, or, to test that (ugly though it may be) a theoretical Sphinx build would produce no build errors:
Quoted 2 lines
cd ~/src/qemu/python
sphinx-apidoc --separate --private --no-toc --module-first \--implicit-namespaces --full --ext-intersphinx --ext-coverage \
--ext-viewcode qemu/ -o docs/Quoted 3 lines
sed -i '1s|^|import os; import sys;
sys.path.insert(0, os.path.abspath("../"))\n|'docs/conf.pymake -C docs html
rm -rf docs/I am preparing to add Sphinx, but need to fix these annoyances first so that regressions are easy to spot as fixes are applied across the tree. I plan to use my forthcoming Asynchronous QMP series as a test pilot for documenting our docstring standards. Assuming it goes well, I will update the docstrings elsewhere in this package at that time.
Quoted 8 lines
Signed-off-by John Snow <[email protected]>--- python/qemu/machine/__init__.py | 6 +++--- python/qemu/machine/machine.py | 3 ++- python/qemu/qmp/__init__.py | 1 + python/qemu/qmp/qom_common.py | 2 +- python/qemu/utils/accel.py | 2 +- 5 files changed, 8 insertions(+), 6 deletions(-)
Quoted 71 lines
python/qemu/machine/__init__.py +3
-3@@ -4,10 +4,10 @@ This library provides a few high-level classes for driving QEMU from a test suite, not intended for production use. -- QEMUMachine: Configure and Boot a QEMU VM - - QEMUQtestMachine: VM class, with a qtest socket. + | QEMUQtestProtocol: send/receive qtest messages. + | QEMUMachine: Configure and Boot a QEMU VM + | +-- QEMUQtestMachine: VM class, with a qtest socket. -- QEMUQtestProtocol: Connect to, send/receive qtest messages. """ # Copyright (C) 2020-2021 John Snow for Red Hat Inc.python/qemu/machine/machine.py +2
-1@@ -545,7 +545,8 @@ def set_qmp_monitor(self, enabled: bool = True) -> None: @param enabled: if False, qmp monitor options will be removed from the base arguments of the resulting QEMU command line. Default is True. - @note: call this function before launch(). + + .. note:: Call this function before launch(). """ self._qmp_set = enabledpython/qemu/qmp/__init__.py +1
-0@@ -279,6 +279,7 @@ def accept(self, timeout: Optional[float] = 15.0) -> QMPMessage: None). The value passed will set the behavior of the underneath QMP socket as described in [1]. Default value is set to 15.0. + @return QMP greeting dict @raise OSError on socket connection errors @raise QMPConnectError if the greeting is not receivedpython/qemu/qmp/qom_common.py +1
-1@@ -156,7 +156,7 @@ def command_runner( """ Run a fully-parsed subcommand, with error-handling for the CLI. - :return: The return code from `.run()`. + :return: The return code from `run()`. """ try: cmd = cls(args)python/qemu/utils/accel.py +1
-1@@ -36,7 +36,7 @@ def list_accel(qemu_bin: str) -> List[str]: List accelerators enabled in the QEMU binary. @param qemu_bin (str): path to the QEMU binary. - @raise Exception: if failed to run `qemu -accel help` + @raise Exception: if failed to run ``qemu -accel help`` @return a list of accelerator names. """ if not qemu_bin: