Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom*
v3 cover Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom*
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-qom
CI: https://gitlab.com/jsnow/qemu/-/pipelines/313932818
Hello!
------
This series applies the usual linting cleanups to modernize the qom tools and then integrates them into the python packaging hierarchy.
This will help prevent further bitrot of these tools.
I don't expect or need a detailed review of the QOM tools themselves -- these tools are not used during build OR testing, and some are fairly bitrotted in places.
However, some details of how the python packaging system is being utilized here may attract your attention and could be worth a look. (Patches 5-6 and 16-19 are the interesting ones.)
Since these scripts aren't critical, I'm OK with sending a fairly hasty PR to merge these sooner rather than later.
Overview:
---------
Patch 1: Update Pipfile.lock (See the commit as for why ...)
Patches 2-3: Correct some existing typing issues in qemu.qmp
Patch 4: Combine qom-set, qom-get, (etc) into one, newly written script that makes all of the command invocations, help text, etc. consistent. (I ask that review for this patch should be limited to critical
mistakes: I have no interest in developing the QOM tools further.)
Patches 5-6: Integrate the qom tools into the python package.
Patches 7-15: Delinting of the qom_fuse script. Similarly, I am not terribly interested in further improvements here, personally.
Patches 16-19: Integrating qom-fuse into the Python packaging directory; additional care is taken to ensure that "optional" dependencies like fusepy are handled well.
Changelog
---------
V3:
- Technically, I sent two versions of this before, a long time ago. This has been cleaned up and based on the latest origin/master.
John Snow (19):
python/pipenv: Update Pipfile.lock
python/qmp: Fix type of SocketAddrT
python/qmp: add parse_address classmethod
python/qmp: Add qom script rewrites
python/qmp: add qom script entry points
scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/ scripts/qom-fuse: apply isort rules
scripts/qom-fuse: apply flake8 rules
python: Add 'fh' to known-good variable names scripts/qom-fuse: Apply pylint rules
scripts/qom-fuse: Add docstrings
scripts/qom-fuse: Convert to QOMCommand
scripts/qom-fuse: use QOMCommand.qom_list() scripts/qom-fuse: ensure QOMFuse.read always returns bytes scripts/qom-fuse: add static type hints
python: add optional FUSE dependencies
scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py python/qmp: add fuse command to 'qom' tools
python/Pipfile.lock | 97 +++++++++++- python/qemu/qmp/__init__.py | 28 +++- python/qemu/qmp/qom.py | 272 ++++++++++++++++++++++++++++++++++ python/qemu/qmp/qom_common.py | 178 ++++++++++++++++++++++ python/qemu/qmp/qom_fuse.py | 206 +++++++++++++++++++++++++ python/setup.cfg | 33 ++++- scripts/qmp/qmp-shell | 21 +-- scripts/qmp/qom-fuse | 144 +----------------- scripts/qmp/qom-get | 66 +-------- scripts/qmp/qom-list | 63 +------- scripts/qmp/qom-set | 63 +------- scripts/qmp/qom-tree | 74 +-------- 12 files changed, 828 insertions(+), 417 deletions(-) create mode 100644 python/qemu/qmp/qom.py create mode 100644 python/qemu/qmp/qom_common.py create mode 100644 python/qemu/qmp/qom_fuse.py
Signature · 1 line
2.31.1
PATCH v3 1/19 python/pipenv: Update Pipfile.lock
In a previous commit, I added tox to the development requirements of the Python library. I never bothered to add them to the Pipfile, because they aren't needed there. Here, I sync it anyway in its own commit so that when we add new packages later that the diffstats will not confusingly appear to pull in lots of extra packages.
Ideally I could tell Pipenv simply not to install these, but it doesn't seem to support that, exactly. The alternative is removing Tox from the development requires, which I'd rather not do.
The other alternative is re-specifying all of the dependencies of setup.cfg in the Pipfile, which I'd also rather not do.
Picking what feels least-worst here.
--- python/Pipfile.lock | 91 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 88 insertions(+), 3 deletions(-)
python/Pipfile.lock +88 -3
@@ -22,6 +22,13 @@ } }, "develop": { + "appdirs": { + "hashes": [ + "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41", + "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128" + ], + "version": "==1.4.4" + }, "astroid": { "hashes": [ "sha256:4db03ab5fc3340cf619dbc25e42c2cc3755154ce6009469766d7143d1fc2ee4e",
@@ -38,6 +45,20 @@ "markers": "python_version >= '3.6'", "version": "==88.1" }, + "distlib": { + "hashes": [ + "sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa28420463a6f799736", + "sha256:23e223426b28491b1ced97dc3bbe183027419dfc7982b4fa2f05d5f3ff10711c" + ], + "version": "==0.3.2" + }, + "filelock": { + "hashes": [ + "sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59", + "sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836" + ], + "version": "==3.0.12" + }, "flake8": { "hashes": [ "sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b",
@@ -54,6 +75,14 @@ "markers": "python_version < '3.8'", "version": "==4.0.1" }, + "importlib-resources": { + "hashes": [ + "sha256:54161657e8ffc76596c4ede7080ca68cb02962a2e074a2586b695a93a925d36e", + "sha256:e962bff7440364183203d179d7ae9ad90cb1f2b74dcb84300e88ecc42dca3351" + ], + "markers": "python_version < '3.7'", + "version": "==5.1.4" + }, "isort": { "hashes": [ "sha256:0a943902919f65c5684ac4e0154b1ad4fac6dcaa5d9f3426b732f1c8b5419be6",
@@ -132,6 +161,30 @@ ], "version": "==0.4.3" }, + "packaging": { + "hashes": [ + "sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5", + "sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==20.9" + }, + "pluggy": { + "hashes": [ + "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0", + "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==0.13.1" + }, + "py": { + "hashes": [ + "sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3", + "sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==1.10.0" + }, "pycodestyle": { "hashes": [ "sha256:514f76d918fcc0b55c6680472f0a37970994e07bbb80725808c17089be302068",
@@ -156,18 +209,42 @@ "markers": "python_version ~= '3.6'", "version": "==2.8.2" }, + "pyparsing": { + "hashes": [ + "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1", + "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b" + ], + "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==2.4.7" + }, "qemu": { "editable": true, "path": "." }, + "six": { + "hashes": [ + "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926", + "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==1.16.0" + }, "toml": { "hashes": [ "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b", "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f" ], - "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2'", + "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", "version": "==0.10.2" }, + "tox": { + "hashes": [ + "sha256:307a81ddb82bd463971a273f33e9533a24ed22185f27db8ce3386bff27d324e3", + "sha256:b0b5818049a1c1997599d42012a637a33f24c62ab8187223fdd318fa8522637b" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", + "version": "==3.23.1" + }, "typed-ast": { "hashes": [ "sha256:01ae5f73431d21eead5015997ab41afa53aa1fbe252f9da060be5dad2c730ace",
@@ -201,7 +278,7 @@ "sha256:f8afcf15cc511ada719a88e013cec87c11aff7b91f019295eb4530f96fe5ef2f", "sha256:fb1bbeac803adea29cedd70781399c99138358c26d05fcbd23c13016b7f5ec65" ], - "markers": "implementation_name == 'cpython' and python_version < '3.8'", + "markers": "python_version < '3.8' and implementation_name == 'cpython'", "version": "==1.4.3" }, "typing-extensions": {
@@ -213,6 +290,14 @@ "markers": "python_version < '3.8'", "version": "==3.10.0.0" }, + "virtualenv": { + "hashes": [ + "sha256:14fdf849f80dbb29a4eb6caa9875d476ee2a5cf76a5f5415fa2f1606010ab467", + "sha256:2b0126166ea7c9c3661f5b8e06773d28f83322de7a3ff7d06f0aed18c9de6a76" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==20.4.7" + }, "wrapt": { "hashes": [ "sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7"
@@ -224,7 +309,7 @@ "sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76", "sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098" ], - "markers": "python_version >= '3.6'", + "markers": "python_version < '3.10'", "version": "==3.4.1" } }
Signature · 1 line
2.31.1
PATCH v3 2/19 python/qmp: Fix type of SocketAddrT
In reply to John SnowIn porting the qom tools, qmp-shell, etc; it becomes evident that this type is wrong.
This is an integer, not a string. We didn't catch this before because none of QEMUMonitorProtocol's *users* happen to be checked, and the internal logic of this class is otherwise self-consistent. Additionally, mypy was not introspecting into the socket() interface to realize we were passing a bad type for AF_INET. Fixed now.
--- python/qemu/qmp/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
python/qemu/qmp/__init__.py +1 -1
@@ -44,7 +44,7 @@ QMPMessage = Dict[str, Any] QMPReturnValue = Dict[str, Any] -InternetAddrT = Tuple[str, str] +InternetAddrT = Tuple[str, int] UnixAddrT = str SocketAddrT = Union[InternetAddrT, UnixAddrT]
Signature · 1 line
2.31.1
Re: [PATCH v3 02/19] python/qmp: Fix type of SocketAddrT
Quoted 9 lines · John Snow
In porting the qom tools, qmp-shell, etc; it becomes evident that this type is wrong.
This is an integer, not a string. We didn't catch this before because none of QEMUMonitorProtocol's *users* happen to be checked, and the internal logic of this class is otherwise self-consistent. Additionally, mypy was not introspecting into the
socket()interface to realize we were passing a bad type forAF_INET. Fixed now.
Quoted 4 lines
Signed-off-by John Snow <[email protected]>--- python/qemu/qmp/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
PATCH v3 3/19 python/qmp: add parse_address classmethod
In reply to John SnowThis takes the place of qmp-shell's __get_address function. It also allows other utilities to share the same parser and syntax for specifying QMP locations.
--- python/qemu/qmp/__init__.py | 26 ++++++++++++++++++++++++++ scripts/qmp/qmp-shell | 21 ++------------------- 2 files changed, 28 insertions(+), 19 deletions(-)
python/qemu/qmp/__init__.py +26 -0
@@ -92,6 +92,12 @@ def __init__(self, reply: QMPMessage): self.reply = reply +class QMPBadPortError(QMPError): + """ + Unable to parse socket address: Port was non-numerical. + """ + + class QEMUMonitorProtocol: """ Provide an API to connect to QEMU via QEMU Monitor Protocol (QMP) and then
@@ -219,6 +225,26 @@ def __exit__(self, # Implement context manager exit function. self.close() + @classmethod + def parse_address(cls, address: str) -> SocketAddrT: + """ + Parse a string into a QMP address. + + Figure out if the argument is in the port:host form. + If it's not, it's probably a file path. + """ + components = address.split(':') + if len(components) == 2: + try: + port = int(components[1]) + except ValueError: + msg = f"Bad port: '{components[1]}' in '{address}'." + raise QMPBadPortError(msg) from None + return (components[0], port) + + # Treat as filepath. + return address + def connect(self, negotiate: bool = True) -> Optional[QMPMessage]: """ Connect to the QMP Monitor and perform capabilities negotiation.
scripts/qmp/qmp-shell +2 -19
@@ -89,8 +89,6 @@ class QMPCompleter(list): class QMPShellError(Exception): pass -class QMPShellBadPort(QMPShellError): - pass class FuzzyJSON(ast.NodeTransformer): '''This extension of ast.NodeTransformer filters literal "true/false/null"
@@ -109,7 +107,7 @@ class FuzzyJSON(ast.NodeTransformer): # _execute_cmd()). Let's design a better one. class QMPShell(qmp.QEMUMonitorProtocol): def __init__(self, address, pretty=False): - super(QMPShell, self).__init__(self.__get_address(address)) + super(QMPShell, self).__init__(self.parse_address(address)) self._greeting = None self._completer = None self._pretty = pretty
@@ -118,21 +116,6 @@ class QMPShell(qmp.QEMUMonitorProtocol): self._histfile = os.path.join(os.path.expanduser('~'), '.qmp-shell_history') - def __get_address(self, arg): - """ - Figure out if the argument is in the port:host form, if it's not it's - probably a file path. - """ - addr = arg.split(':') - if len(addr) == 2: - try: - port = int(addr[1]) - except ValueError: - raise QMPShellBadPort - return ( addr[0], port ) - # socket path - return arg - def _fill_completion(self): cmds = self.cmd('query-commands') if 'error' in cmds:
@@ -437,7 +420,7 @@ def main(): if qemu is None: fail_cmdline() - except QMPShellBadPort: + except qmp.QMPBadPortError: die('bad port number in command-line') try:
Signature · 1 line
2.31.1
PATCH v3 4/19 python/qmp: Add qom script rewrites
In reply to John SnowInspired by qom-set, qom-get, qom-tree and qom-list; combine all four of those scripts into a single script.
A later addition of qom-fuse as an 'extension' necessitates that some common features are split out and shared between them.
--- python/qemu/qmp/qom.py | 262 ++++++++++++++++++++++++++++++++++ python/qemu/qmp/qom_common.py | 178 +++++++++++++++++++++++ 2 files changed, 440 insertions(+) create mode 100644 python/qemu/qmp/qom.py create mode 100644 python/qemu/qmp/qom_common.py
python/qemu/qmp/qom.py new file +262 -0
@@ -0,0 +1,262 @@ +""" +QEMU Object Model testing tools. + +usage: qom [-h] {set,get,list,tree} ... + +Query and manipulate QOM data + +optional arguments: + -h, --help show this help message and exit + +QOM commands: + {set,get,list,tree} + set Set a QOM property value + get Get a QOM property value + list List QOM properties at a given path + tree Show QOM tree from a given path +""" +## +# Copyright John Snow 2020, for Red Hat, Inc. +# Copyright IBM, Corp. 2011 +# +# Authors: +# John Snow <[email protected]> +# Anthony Liguori <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. +# +# Based on ./scripts/qmp/qom-[set|get|tree|list] +## + +import argparse + +from . import QMPResponseError +from .qom_common import QOMCommand + + +class QOMSet(QOMCommand): + """ + QOM Command - Set a property to a given value. + + usage: qom-set [-h] [--socket SOCKET] <path>.<property> <value> + + Set a QOM property value + + positional arguments: + <path>.<property> QOM path and property, separated by a period '.' + <value> new QOM property value + + optional arguments: + -h, --help show this help message and exit + --socket SOCKET, -s SOCKET + QMP socket path or address (addr:port). May also be + set via QMP_SOCKET environment variable. + """ + name = 'set' + help = 'Set a QOM property value' + + @classmethod + def configure_parser(cls, parser: argparse.ArgumentParser) -> None: + super().configure_parser(parser) + cls.add_path_prop_arg(parser) + parser.add_argument( + 'value', + metavar='<value>', + action='store', + help='new QOM property value' + ) + + def __init__(self, args: argparse.Namespace): + super().__init__(args) + self.path, self.prop = args.path_prop.rsplit('.', 1) + self.value = args.value + + def run(self) -> int: + rsp = self.qmp.command( + 'qom-set', + path=self.path, + property=self.prop, + value=self.value + ) + print(rsp) + return 0 + + +class QOMGet(QOMCommand): + """ + QOM Command - Get a property's current value. + + usage: qom-get [-h] [--socket SOCKET] <path>.<property> + + Get a QOM property value + + positional arguments: + <path>.<property> QOM path and property, separated by a period '.' + + optional arguments: + -h, --help show this help message and exit + --socket SOCKET, -s SOCKET + QMP socket path or address (addr:port). May also be + set via QMP_SOCKET environment variable. + """ + name = 'get' + help = 'Get a QOM property value' + + @classmethod + def configure_parser(cls, parser: argparse.ArgumentParser) -> None: + super().configure_parser(parser) + cls.add_path_prop_arg(parser) + + def __init__(self, args: argparse.Namespace): + super().__init__(args) + try: + tmp = args.path_prop.rsplit('.', 1) + except ValueError as err: + raise ValueError('Invalid format for <path>.<property>') from err + self.path = tmp[0] + self.prop = tmp[1] + + def run(self) -> int: + rsp = self.qmp.command( + 'qom-get', + path=self.path, + property=self.prop + ) + if isinstance(rsp, dict): + for key, value in rsp.items(): + print(f"{key}: {value}") + else: + print(rsp) + return 0 + + +class QOMList(QOMCommand): + """ + QOM Command - List the properties at a given path. + + usage: qom-list [-h] [--socket SOCKET] <path> + + List QOM properties at a given path + + positional arguments: + <path> QOM path + + optional arguments: + -h, --help show this help message and exit + --socket SOCKET, -s SOCKET + QMP socket path or address (addr:port). May also be + set via QMP_SOCKET environment variable. + """ + name = 'list' + help = 'List QOM properties at a given path' + + @classmethod + def configure_parser(cls, parser: argparse.ArgumentParser) -> None: + super().configure_parser(parser) + parser.add_argument( + 'path', + metavar='<path>', + action='store', + help='QOM path', + ) + + def __init__(self, args: argparse.Namespace): + super().__init__(args) + self.path = args.path + + def run(self) -> int: + rsp = self.qom_list(self.path) + for item in rsp: + if item.child: + print(f"{item.name}/") + elif item.link: + print(f"@{item.name}/") + else: + print(item.name) + return 0 + + +class QOMTree(QOMCommand): + """ + QOM Command - Show the full tree below a given path. + + usage: qom-tree [-h] [--socket SOCKET] [<path>] + + Show QOM tree from a given path + + positional arguments: + <path> QOM path + + optional arguments: + -h, --help show this help message and exit + --socket SOCKET, -s SOCKET + QMP socket path or address (addr:port). May also be + set via QMP_SOCKET environment variable. + """ + name = 'tree' + help = 'Show QOM tree from a given path' + + @classmethod + def configure_parser(cls, parser: argparse.ArgumentParser) -> None: + super().configure_parser(parser) + parser.add_argument( + 'path', + metavar='<path>', + action='store', + help='QOM path', + nargs='?', + default='/' + ) + + def __init__(self, args: argparse.Namespace): + super().__init__(args) + self.path = args.path + + def _list_node(self, path: str) -> None: + print(path) + items = self.qom_list(path) + for item in items: + if item.child: + continue + try: + rsp = self.qmp.command('qom-get', path=path, + property=item.name) + print(f" {item.name}: {rsp} ({item.type})") + except QMPResponseError as err: + print(f" {item.name}: <EXCEPTION: {err!s}> ({item.type})") + print('') + for item in items: + if not item.child: + continue + if path == '/': + path = '' + self._list_node(f"{path}/{item.name}") + + def run(self) -> int: + self._list_node(self.path) + return 0 + + +def main() -> int: + """QOM script main entry point.""" + parser = argparse.ArgumentParser( + description='Query and manipulate QOM data' + ) + subparsers = parser.add_subparsers( + title='QOM commands', + dest='command' + ) + + for command in QOMCommand.__subclasses__(): + command.register(subparsers) + + args = parser.parse_args() + + if args.command is None: + parser.error('Command not specified.') + return 1 + + cmd_class = args.cmd_class + assert isinstance(cmd_class, type(QOMCommand)) + return cmd_class.command_runner(args)
python/qemu/qmp/qom_common.py new file +178 -0
178 lines not on this page · raw message
Signature · 1 line
2.31.1
PATCH v3 5/19 python/qmp: add qom script entry points
In reply to John SnowAdd the 'qom', 'qom-set', 'qom-get', 'qom-list', and 'qom-tree' scripts to the qemu.qmp package. When you install this package, these scripts will become available on your command line.
(e.g. when inside of a venv, cd python && pip install . will add 'qom', 'qom-set', etc to your $PATH.)
--- python/setup.cfg | 8 ++++++++ 1 file changed, 8 insertions(+)
python/setup.cfg +8 -0
@@ -37,6 +37,14 @@ devel = pylint >= 2.8.0 tox >= 3.18.0 +[options.entry_points] +console_scripts = + qom = qemu.qmp.qom:main + qom-set = qemu.qmp.qom:QOMSet.entry_point + qom-get = qemu.qmp.qom:QOMGet.entry_point + qom-list = qemu.qmp.qom:QOMList.entry_point + qom-tree = qemu.qmp.qom:QOMTree.entry_point + [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's exclude = __pycache__,
Signature · 1 line
2.31.1
PATCH v3 6/19 scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/
In reply to John SnowRedirect to the new qom scripts. These forwarders can be deleted eventually when there has been more time for the dust on the Python packaging to settle and people understand how to find these commands.
Note: You can run these by setting $PYTHONPATH in your shell and then running "python3 -m qemu.qmp.qom", or you can install the qemu namespace package and use the "qom" or "qom-set" scripts.
I've written how to install the package elsewhere, but for the sake of git-blame, cd to ./python, and then do:
- pip3 install [--user] [-e] .
--user will install to your local user install (will not work inside of
a venv), omitting this flag installs to your system-wide packages (outside of a venv) or to your current virtual environment (inside the venv).
When installing to a venv or to your system-wide packages, "qom" should be in your $PATH already. If you do a user install, you may need to add ~/.local/bin to your $PATH if you haven't already.
-e installs in editable mode: the installed package is effectively just
a symlink to this folder; so changes to your git working tree are reflected in the installed package.
Note: installing these packages to an environment outside a venv can be
dangerous: Many QEMU scripts will begin to prefer the installed version instead of the version directly in the tree. Use with caution. editable mode is recommended when working outside of a venv.
--- scripts/qmp/qom-get | 66 +++------------------------------------ scripts/qmp/qom-list | 63 +++---------------------------------- scripts/qmp/qom-set | 63 +++---------------------------------- scripts/qmp/qom-tree | 74 +++----------------------------------------- 4 files changed, 16 insertions(+), 250 deletions(-)
scripts/qmp/qom-get +4 -62
@@ -1,69 +1,11 @@ #!/usr/bin/env python3 -## -# QEMU Object Model test tools -# -# Copyright IBM, Corp. 2011 -# -# Authors: -# Anthony Liguori <[email protected]> -# -# This work is licensed under the terms of the GNU GPL, version 2 or later. See -# the COPYING file in the top-level directory. -## -import sys import os +import sys sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) -from qemu.qmp import QEMUMonitorProtocol +from qemu.qmp.qom import QOMGet -cmd, args = sys.argv[0], sys.argv[1:] -socket_path = None -path = None -prop = None -def usage(): - return '''environment variables: - QMP_SOCKET=<path | addr:port> -usage: - %s [-h] [-s <QMP socket path | addr:port>] <path>.<property> -''' % cmd - -def usage_error(error_msg = "unspecified error"): - sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg)) - exit(1) - -if len(args) > 0: - if args[0] == "-h": - print(usage()) - exit(0); - elif args[0] == "-s": - try: - socket_path = args[1] - except: - usage_error("missing argument: QMP socket path or address"); - args = args[2:] - -if not socket_path: - if 'QMP_SOCKET' in os.environ: - socket_path = os.environ['QMP_SOCKET'] - else: - usage_error("no QMP socket path or address given"); - -if len(args) > 0: - try: - path, prop = args[0].rsplit('.', 1) - except: - usage_error("invalid format for path/property/value") -else: - usage_error("not enough arguments") - -srv = QEMUMonitorProtocol(socket_path) -srv.connect() - -rsp = srv.command('qom-get', path=path, property=prop) -if type(rsp) == dict: - for i in rsp.keys(): - print('%s: %s' % (i, rsp[i])) -else: - print(rsp) +if __name__ == '__main__': + sys.exit(QOMGet.entry_point())
scripts/qmp/qom-list +4 -59
70 lines not on this page · raw message
scripts/qmp/qom-set +4 -59
70 lines not on this page · raw message
scripts/qmp/qom-tree +4 -70
81 lines not on this page · raw message
Signature · 1 line
2.31.1
PATCH v3 7/19 scripts/qom-fuse: apply isort rules
In reply to John SnowHint: you can use symlinks to create qom_fuse.py in python/qemu/qmp/ and point to scripts/qom-fuse to apply the standard linting rules to this script.
--- scripts/qmp/qom-fuse | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
scripts/qmp/qom-fuse +9 -3
@@ -13,14 +13,20 @@ # the COPYING file in the top-level directory. ## -import fuse, stat -from fuse import FUSE, FuseOSError, Operations -import os, posix, sys from errno import * +import os +import posix +import stat +import sys + +import fuse +from fuse import FUSE, FuseOSError, Operations + sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu.qmp import QEMUMonitorProtocol + fuse.fuse_python_api = (0, 2) class QOMFS(Operations):
Signature · 1 line
2.31.1
PATCH v3 8/19 scripts/qom-fuse: apply flake8 rules
In reply to John Snowflake8 still has one warning because of the sys.path hack, but that will be going away by the end of this patch series.
--- scripts/qmp/qom-fuse | 81 +++++++++++++++++++++++--------------------- 1 file changed, 43 insertions(+), 38 deletions(-)
scripts/qmp/qom-fuse +43 -38
@@ -9,13 +9,12 @@ # Anthony Liguori <[email protected]> # Markus Armbruster <[email protected]> # -# This work is licensed under the terms of the GNU GPL, version 2 or later. See -# the COPYING file in the top-level directory. +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. ## -from errno import * +from errno import ENOENT, EPERM import os -import posix import stat import sys
@@ -29,6 +28,7 @@ from qemu.qmp import QEMUMonitorProtocol fuse.fuse_python_api = (0, 2) + class QOMFS(Operations): def __init__(self, qmp): self.qmp = qmp
@@ -45,7 +45,7 @@ class QOMFS(Operations): def is_object(self, path): try: - items = self.qmp.command('qom-list', path=path) + self.qmp.command('qom-list', path=path) return True except: return False
@@ -85,7 +85,7 @@ class QOMFS(Operations): path = '/' try: data = self.qmp.command('qom-get', path=path, property=prop) - data += '\n' # make values shell friendly + data += '\n' # make values shell friendly except: raise FuseOSError(EPERM)
@@ -104,38 +104,44 @@ class QOMFS(Operations): def getattr(self, path, fh=None): if self.is_link(path): - value = { 'st_mode': 0o755 | stat.S_IFLNK, - 'st_ino': self.get_ino(path), - 'st_dev': 0, - 'st_nlink': 2, - 'st_uid': 1000, - 'st_gid': 1000, - 'st_size': 4096, - 'st_atime': 0, - 'st_mtime': 0, - 'st_ctime': 0 } + value = { + 'st_mode': 0o755 | stat.S_IFLNK, + 'st_ino': self.get_ino(path), + 'st_dev': 0, + 'st_nlink': 2, + 'st_uid': 1000, + 'st_gid': 1000, + 'st_size': 4096, + 'st_atime': 0, + 'st_mtime': 0, + 'st_ctime': 0 + } elif self.is_object(path): - value = { 'st_mode': 0o755 | stat.S_IFDIR, - 'st_ino': self.get_ino(path), - 'st_dev': 0, - 'st_nlink': 2, - 'st_uid': 1000, - 'st_gid': 1000, - 'st_size': 4096, - 'st_atime': 0, - 'st_mtime': 0, - 'st_ctime': 0 } + value = { + 'st_mode': 0o755 | stat.S_IFDIR, + 'st_ino': self.get_ino(path), + 'st_dev': 0, + 'st_nlink': 2, + 'st_uid': 1000, + 'st_gid': 1000, + 'st_size': 4096, + 'st_atime': 0, + 'st_mtime': 0, + 'st_ctime': 0 + } elif self.is_property(path): - value = { 'st_mode': 0o644 | stat.S_IFREG, - 'st_ino': self.get_ino(path), - 'st_dev': 0, - 'st_nlink': 1, - 'st_uid': 1000, - 'st_gid': 1000, - 'st_size': 4096, - 'st_atime': 0, - 'st_mtime': 0, - 'st_ctime': 0 } + value = { + 'st_mode': 0o644 | stat.S_IFREG, + 'st_ino': self.get_ino(path), + 'st_dev': 0, + 'st_nlink': 1, + 'st_uid': 1000, + 'st_gid': 1000, + 'st_size': 4096, + 'st_atime': 0, + 'st_mtime': 0, + 'st_ctime': 0 + } else: raise FuseOSError(ENOENT) return value
@@ -146,8 +152,7 @@ class QOMFS(Operations): for item in self.qmp.command('qom-list', path=path): yield str(item['name']) + if __name__ == '__main__': - import os - fuse = FUSE(QOMFS(QEMUMonitorProtocol(os.environ['QMP_SOCKET'])), sys.argv[1], foreground=True)
Signature · 1 line
2.31.1
Re: [PATCH v3 08/19] scripts/qom-fuse: apply flake8 rules
Quoted 7 lines · John Snow
flake8 still has one warning because of the sys.path hack, but that will be going away by the end of this patch series.
Signed-off-by John Snow <[email protected]>--- scripts/qmp/qom-fuse | 81 +++++++++++++++++++++++--------------------- 1 file changed, 43 insertions(+), 38 deletions(-)
PATCH v3 9/19 python: Add 'fh' to known-good variable names
In reply to John Snowfd and fh are fine: we often use these for "file descriptor" or "file handle" accordingly. It is rarely the case that you need to enforce a more semantically meaningful name beyond "This is the file we are using right now."
While we're here: add comments for all of the non-standard pylint names. (And the underscore.)
--- python/setup.cfg | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
python/setup.cfg +4 -3
@@ -78,9 +78,10 @@ good-names=i, k, ex, Run, - _, - fd, - c, + _, # By convention: Unused variable + fh, # fh = open(...) + fd, # fd = os.open(...) + c, # for c in string: ... [pylint.similarities] # Ignore imports when computing similarities.
Signature · 1 line
2.31.1
PATCH v3 10/19 scripts/qom-fuse: Apply pylint rules
In reply to John Snow- Catch specific exceptions from QMP
- Reraise errors with explicit context
- method parameters should match parent's names
--- scripts/qmp/qom-fuse | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
scripts/qmp/qom-fuse +8 -8
@@ -23,7 +23,7 @@ from fuse import FUSE, FuseOSError, Operations sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) -from qemu.qmp import QEMUMonitorProtocol +from qemu.qmp import QEMUMonitorProtocol, QMPResponseError fuse.fuse_python_api = (0, 2)
@@ -47,7 +47,7 @@ class QOMFS(Operations): try: self.qmp.command('qom-list', path=path) return True - except: + except QMPResponseError: return False def is_property(self, path):
@@ -59,7 +59,7 @@ class QOMFS(Operations): if item['name'] == prop: return True return False - except: + except QMPResponseError: return False def is_link(self, path):
@@ -73,10 +73,10 @@ class QOMFS(Operations): return True return False return False - except: + except QMPResponseError: return False - def read(self, path, length, offset, fh): + def read(self, path, size, offset, fh): if not self.is_property(path): return -ENOENT
@@ -86,13 +86,13 @@ class QOMFS(Operations): try: data = self.qmp.command('qom-get', path=path, property=prop) data += '\n' # make values shell friendly - except: - raise FuseOSError(EPERM) + except QMPResponseError as err: + raise FuseOSError(EPERM) from err if offset > len(data): return '' - return bytes(data[offset:][:length], encoding='utf-8') + return bytes(data[offset:][:size], encoding='utf-8') def readlink(self, path): if not self.is_link(path):
Signature · 1 line
2.31.1
PATCH v3 11/19 scripts/qom-fuse: Add docstrings
In reply to John SnowThe methods inherited from fuse don't need docstrings; that's up to fusepy to handle.
--- scripts/qmp/qom-fuse | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)
scripts/qmp/qom-fuse +19 -2
@@ -1,7 +1,19 @@ #!/usr/bin/env python3 +""" +QEMU Object Model FUSE filesystem tool + +This script offers a simple FUSE filesystem within which the QOM tree +may be browsed, queried and edited using traditional shell tooling. + +This script requires the 'fusepy' python package. + +ENV: + QMP_SOCKET: Path to the QMP server socket + +Usage: + qom-fuse /mount/to/here +""" ## -# QEMU Object Model test tools -# # Copyright IBM, Corp. 2012 # Copyright (C) 2020 Red Hat, Inc. #
@@ -30,6 +42,7 @@ fuse.fuse_python_api = (0, 2) class QOMFS(Operations): + """QOMFS implements fuse.Operations to provide a QOM filesystem.""" def __init__(self, qmp): self.qmp = qmp self.qmp.connect()
@@ -37,6 +50,7 @@ class QOMFS(Operations): self.ino_count = 1 def get_ino(self, path): + """Get an inode number for a given QOM path.""" if path in self.ino_map: return self.ino_map[path] self.ino_map[path] = self.ino_count
@@ -44,6 +58,7 @@ class QOMFS(Operations): return self.ino_map[path] def is_object(self, path): + """Is the given QOM path an object?""" try: self.qmp.command('qom-list', path=path) return True
@@ -51,6 +66,7 @@ class QOMFS(Operations): return False def is_property(self, path): + """Is the given QOM path a property?""" path, prop = path.rsplit('/', 1) if path == '': path = '/'
@@ -63,6 +79,7 @@ class QOMFS(Operations): return False def is_link(self, path): + """Is the given QOM path a link?""" path, prop = path.rsplit('/', 1) if path == '': path = '/'
Signature · 1 line
2.31.1
PATCH v3 12/19 scripts/qom-fuse: Convert to QOMCommand
In reply to John SnowMove qom-fuse onto the QOMCommand base established in python/qemu/qmp/qom_common.py. The interface doesn't change incompatibly, "qom-fuse mountpoint" still works as an invocation, and QMP_SOCKET is still used as the environment variable.
--- scripts/qmp/qom-fuse | 59 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 46 insertions(+), 13 deletions(-)
scripts/qmp/qom-fuse +46 -13
@@ -7,11 +7,19 @@ may be browsed, queried and edited using traditional shell tooling. This script requires the 'fusepy' python package. -ENV: - QMP_SOCKET: Path to the QMP server socket -Usage: - qom-fuse /mount/to/here +usage: qom-fuse [-h] [--socket SOCKET] <mount> + +Mount a QOM tree as a FUSE filesystem + +positional arguments: + <mount> Mount point + +optional arguments: + -h, --help show this help message and exit + --socket SOCKET, -s SOCKET + QMP socket path or address (addr:port). May also be + set via QMP_SOCKET environment variable. """ ## # Copyright IBM, Corp. 2012
@@ -25,30 +33,56 @@ Usage: # See the COPYING file in the top-level directory. ## +import argparse from errno import ENOENT, EPERM import os import stat import sys +from typing import Dict import fuse from fuse import FUSE, FuseOSError, Operations sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) -from qemu.qmp import QEMUMonitorProtocol, QMPResponseError +from qemu.qmp import QMPResponseError +from qemu.qmp.qom_common import QOMCommand fuse.fuse_python_api = (0, 2) -class QOMFS(Operations): - """QOMFS implements fuse.Operations to provide a QOM filesystem.""" - def __init__(self, qmp): - self.qmp = qmp - self.qmp.connect() - self.ino_map = {} +class QOMFuse(QOMCommand, Operations): + """ + QOMFuse implements both fuse.Operations and QOMCommand. + + Operations implements the FS, and QOMCommand implements the CLI command. + """ + name = 'fuse' + help = 'Mount a QOM tree as a FUSE filesystem' + fuse: FUSE + + @classmethod + def configure_parser(cls, parser: argparse.ArgumentParser) -> None: + super().configure_parser(parser) + parser.add_argument( + 'mount', + metavar='<mount>', + action='store', + help="Mount point", + ) + + def __init__(self, args: argparse.Namespace): + super().__init__(args) + self.mount = args.mount + self.ino_map: Dict[str, int] = {} self.ino_count = 1 + def run(self) -> int: + print(f"Mounting QOMFS to '{self.mount}'", file=sys.stderr) + self.fuse = FUSE(self, self.mount, foreground=True) + return 0 + def get_ino(self, path): """Get an inode number for a given QOM path.""" if path in self.ino_map:
@@ -171,5 +205,4 @@ class QOMFS(Operations): if __name__ == '__main__': - fuse = FUSE(QOMFS(QEMUMonitorProtocol(os.environ['QMP_SOCKET'])), - sys.argv[1], foreground=True) + sys.exit(QOMFuse.entry_point())
Signature · 1 line
2.31.1
PATCH v3 13/19 scripts/qom-fuse: use QOMCommand.qom_list()
In reply to John Snowthe qom_list method provides a type-safe object that's easier to type check, so switch to using it.
--- scripts/qmp/qom-fuse | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
scripts/qmp/qom-fuse +8 -10
@@ -94,7 +94,7 @@ class QOMFuse(QOMCommand, Operations): def is_object(self, path): """Is the given QOM path an object?""" try: - self.qmp.command('qom-list', path=path) + self.qom_list(path) return True except QMPResponseError: return False
@@ -105,8 +105,8 @@ class QOMFuse(QOMCommand, Operations): if path == '': path = '/' try: - for item in self.qmp.command('qom-list', path=path): - if item['name'] == prop: + for item in self.qom_list(path): + if item.name == prop: return True return False except QMPResponseError:
@@ -118,11 +118,9 @@ class QOMFuse(QOMCommand, Operations): if path == '': path = '/' try: - for item in self.qmp.command('qom-list', path=path): - if item['name'] == prop: - if item['type'].startswith('link<'): - return True - return False + for item in self.qom_list(path): + if item.name == prop and item.link: + return True return False except QMPResponseError: return False
@@ -200,8 +198,8 @@ class QOMFuse(QOMCommand, Operations): def readdir(self, path, fh): yield '.' yield '..' - for item in self.qmp.command('qom-list', path=path): - yield str(item['name']) + for item in self.qom_list(path): + yield item.name if __name__ == '__main__':
Signature · 1 line
2.31.1
Re: [PATCH v3 13/19] scripts/qom-fuse: use QOMCommand.qom_list()
Quoted 7 lines · John Snow
the
qom_listmethod provides a type-safe object that's easier to type check, so switch to using it.Signed-off-by John Snow <[email protected]>--- scripts/qmp/qom-fuse | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
PATCH v3 14/19 scripts/qom-fuse: ensure QOMFuse.read always returns bytes
In reply to John Snow- Use FuseOSError to signal ENOENT instead of returning it
- Wrap qom-get in str(), as we don't always know its type
- The empty return should be b'', not ''.
--- scripts/qmp/qom-fuse | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
scripts/qmp/qom-fuse +3 -3
@@ -127,19 +127,19 @@ class QOMFuse(QOMCommand, Operations): def read(self, path, size, offset, fh): if not self.is_property(path): - return -ENOENT + raise FuseOSError(ENOENT) path, prop = path.rsplit('/', 1) if path == '': path = '/' try: - data = self.qmp.command('qom-get', path=path, property=prop) + data = str(self.qmp.command('qom-get', path=path, property=prop)) data += '\n' # make values shell friendly except QMPResponseError as err: raise FuseOSError(EPERM) from err if offset > len(data): - return '' + return b'' return bytes(data[offset:][:size], encoding='utf-8')
Signature · 1 line
2.31.1
PATCH v3 15/19 scripts/qom-fuse: add static type hints
In reply to John SnowBecause fusepy does not have type hints, add some targeted warning suppressions.
Namely, we need to allow subclassing something of an unknown type (in qom_fuse.py), and we need to allow missing imports (recorded against fuse itself) because mypy will be unable to import fusepy (even when installed) as it has no types nor type stubs available.
Note: Until now, it was possible to run invocations like 'mypy qemu/' from ./python and have that work. However, these targeted suppressions require that you run 'mypy -p qemu/' instead. The correct, canonical invocation is recorded in ./python/tests/mypy.sh and all of the various CI invocations always use this correct form.
--- python/setup.cfg | 8 ++++++++ scripts/qmp/qom-fuse | 26 +++++++++++++++++--------- 2 files changed, 25 insertions(+), 9 deletions(-)
python/setup.cfg +8 -0
@@ -57,6 +57,14 @@ python_version = 3.6 warn_unused_configs = True namespace_packages = True +[mypy-qemu.qmp.qom_fuse] +# fusepy has no type stubs: +allow_subclassing_any = True + +[mypy-fuse] +# fusepy has no type stubs: +ignore_missing_imports = True + [pylint.messages control] # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this
scripts/qmp/qom-fuse +17 -9
@@ -38,7 +38,14 @@ from errno import ENOENT, EPERM import os import stat import sys -from typing import Dict +from typing import ( + IO, + Dict, + Iterator, + Mapping, + Optional, + Union, +) import fuse from fuse import FUSE, FuseOSError, Operations
@@ -83,7 +90,7 @@ class QOMFuse(QOMCommand, Operations): self.fuse = FUSE(self, self.mount, foreground=True) return 0 - def get_ino(self, path): + def get_ino(self, path: str) -> int: """Get an inode number for a given QOM path.""" if path in self.ino_map: return self.ino_map[path]
@@ -91,7 +98,7 @@ class QOMFuse(QOMCommand, Operations): self.ino_count += 1 return self.ino_map[path] - def is_object(self, path): + def is_object(self, path: str) -> bool: """Is the given QOM path an object?""" try: self.qom_list(path)
@@ -99,7 +106,7 @@ class QOMFuse(QOMCommand, Operations): except QMPResponseError: return False - def is_property(self, path): + def is_property(self, path: str) -> bool: """Is the given QOM path a property?""" path, prop = path.rsplit('/', 1) if path == '':
@@ -112,7 +119,7 @@ class QOMFuse(QOMCommand, Operations): except QMPResponseError: return False - def is_link(self, path): + def is_link(self, path: str) -> bool: """Is the given QOM path a link?""" path, prop = path.rsplit('/', 1) if path == '':
@@ -125,7 +132,7 @@ class QOMFuse(QOMCommand, Operations): except QMPResponseError: return False - def read(self, path, size, offset, fh): + def read(self, path: str, size: int, offset: int, fh: IO[bytes]) -> bytes: if not self.is_property(path): raise FuseOSError(ENOENT)
@@ -143,7 +150,7 @@ class QOMFuse(QOMCommand, Operations): return bytes(data[offset:][:size], encoding='utf-8') - def readlink(self, path): + def readlink(self, path: str) -> Union[bool, str]: if not self.is_link(path): return False path, prop = path.rsplit('/', 1)
@@ -151,7 +158,8 @@ class QOMFuse(QOMCommand, Operations): return prefix + str(self.qmp.command('qom-get', path=path, property=prop)) - def getattr(self, path, fh=None): + def getattr(self, path: str, + fh: Optional[IO[bytes]] = None) -> Mapping[str, object]: if self.is_link(path): value = { 'st_mode': 0o755 | stat.S_IFLNK,
@@ -195,7 +203,7 @@ class QOMFuse(QOMCommand, Operations): raise FuseOSError(ENOENT) return value - def readdir(self, path, fh): + def readdir(self, path: str, fh: IO[bytes]) -> Iterator[str]: yield '.' yield '..' for item in self.qom_list(path):
Signature · 1 line
2.31.1
PATCH v3 16/19 python: add optional FUSE dependencies
In reply to John SnowIn preparation for moving qom-fuse over to the python package, we need some new dependencies to support it.
Add an optional 'fusepy' dependency that users of the package can opt into with e.g. "pip install qemu[fuse]" which installs the requirements necessary to obtain the additional functionality.
Add the same fusepy dependency to the 'devel' extras group -- unfortunately I do not see a way for optional groups to imply other optional groups at present, so the dependency is repeated. The development group needs to include the full set of dependencies for the purpose of static analysis of all features offered by this library.
Lastly, add the [fuse] extras group to tox's configuration as a workaround so that if a stale tox environment is found when running make check-tox, tox will know to rebuild its environments.
--- python/Pipfile.lock | 6 ++++++ python/setup.cfg | 9 ++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
python/Pipfile.lock +6 -0
@@ -67,6 +67,12 @@ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", "version": "==3.9.2" }, + "fusepy": { + "hashes": [ + "sha256:72ff783ec2f43de3ab394e3f7457605bf04c8cf288a2f4068b4cde141d4ee6bd" + ], + "version": "==3.0.1" + }, "importlib-metadata": { "hashes": [ "sha256:8c501196e49fb9df5df43833bdb1e4328f64847763ec8a50703148b73784d581",
python/setup.cfg +8 -1
@@ -32,11 +32,16 @@ packages = devel = avocado-framework >= 87.0 flake8 >= 3.6.0 + fusepy >= 2.0.4 isort >= 5.1.2 mypy >= 0.770 pylint >= 2.8.0 tox >= 3.18.0 +# Provides qom-fuse functionality +fuse = + fusepy >= 2.0.4 + [options.entry_points] console_scripts = qom = qemu.qmp.qom:main
@@ -114,6 +119,8 @@ envlist = py36, py37, py38, py39, py310 [testenv] allowlist_externals = make -deps = .[devel] +deps = + .[devel] + .[fuse] # Workaround to trigger tox venv rebuild commands = make check
Signature · 1 line
2.31.1
PATCH v3 17/19 scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py
In reply to John SnowMove qom-fuse over to the python package now that it passes the linter. Update the import paradigms so that it continues to pass in the context of the Python package.
--- scripts/qmp/qom-fuse => python/qemu/qmp/qom_fuse.py | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) rename scripts/qmp/qom-fuse => python/qemu/qmp/qom_fuse.py (95%) mode change 100755 => 100644
python/qemu/qmp/qom_fuse.py renamed from scripts/qmp/qom-fuse +2 -10
@@ -1,4 +1,3 @@ -#!/usr/bin/env python3 """ QEMU Object Model FUSE filesystem tool
@@ -35,7 +34,6 @@ import argparse from errno import ENOENT, EPERM -import os import stat import sys from typing import (
@@ -50,10 +48,8 @@ import fuse from fuse import FUSE, FuseOSError, Operations - -sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) -from qemu.qmp import QMPResponseError -from qemu.qmp.qom_common import QOMCommand +from . import QMPResponseError +from .qom_common import QOMCommand fuse.fuse_python_api = (0, 2)
@@ -208,7 +204,3 @@ def readdir(self, path: str, fh: IO[bytes]) -> Iterator[str]: yield '..' for item in self.qom_list(path): yield item.name - - -if __name__ == '__main__': - sys.exit(QOMFuse.entry_point())
Signature · 1 line
2.31.1
PATCH v3 18/19 scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py
In reply to John SnowBy leaving the script absent for a commit, git-blame travels to the new file instead of staying on the shim.
--- scripts/qmp/qom-fuse | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100755 scripts/qmp/qom-fuse
scripts/qmp/qom-fuse new file +11 -0
@@ -0,0 +1,11 @@ +#!/usr/bin/env python3 + +import os +import sys + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) +from qemu.qmp.qom_fuse import QOMFuse + + +if __name__ == '__main__': + sys.exit(QOMFuse.entry_point())
Signature · 1 line
2.31.1
PATCH v3 19/19 python/qmp: add fuse command to 'qom' tools
In reply to John SnowThe 'fuse' command will be unavailable if 'fusepy' is not installed. It will simply not load and subsequently be unavailable as a subcommand.
--- python/qemu/qmp/qom.py | 14 ++++++++++++-- python/setup.cfg | 1 + 2 files changed, 13 insertions(+), 2 deletions(-)
python/qemu/qmp/qom.py +12 -2
@@ -1,7 +1,7 @@ """ QEMU Object Model testing tools. -usage: qom [-h] {set,get,list,tree} ... +usage: qom [-h] {set,get,list,tree,fuse} ... Query and manipulate QOM data
@@ -9,11 +9,12 @@ -h, --help show this help message and exit QOM commands: - {set,get,list,tree} + {set,get,list,tree,fuse} set Set a QOM property value get Get a QOM property value list List QOM properties at a given path tree Show QOM tree from a given path + fuse Mount a QOM tree as a FUSE filesystem """ ## # Copyright John Snow 2020, for Red Hat, Inc.
@@ -35,6 +36,15 @@ from .qom_common import QOMCommand +try: + from .qom_fuse import QOMFuse +except ModuleNotFoundError as err: + if err.name != 'fuse': + raise +else: + assert issubclass(QOMFuse, QOMCommand) + + class QOMSet(QOMCommand): """ QOM Command - Set a property to a given value.
python/setup.cfg +1 -0
@@ -49,6 +49,7 @@ console_scripts = qom-get = qemu.qmp.qom:QOMGet.entry_point qom-list = qemu.qmp.qom:QOMList.entry_point qom-tree = qemu.qmp.qom:QOMTree.entry_point + qom-fuse = qemu.qmp.qom_fuse:QOMFuse.entry_point [fuse] [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's
Signature · 1 line
2.31.1
Re: [PATCH v3 19/19] python/qmp: add fuse command to 'qom' tools
Quoted 8 lines · John Snow
The 'fuse' command will be unavailable if 'fusepy' is not installed. It will simply not load and subsequently be unavailable as a subcommand.
Signed-off-by John Snow <[email protected]>--- python/qemu/qmp/qom.py | 14 ++++++++++++-- python/setup.cfg | 1 + 2 files changed, 13 insertions(+), 2 deletions(-)
Quoted 85 lines · John Snow
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-qom
CI: https://gitlab.com/jsnow/qemu/-/pipelines/313932818Hello!
------This series applies the usual linting cleanups to modernize the qom tools and then integrates them into the python packaging hierarchy.
This will help prevent further bitrot of these tools.
I don't expect or need a detailed review of the QOM tools themselves -- these tools are not used during build OR testing, and some are fairly bitrotted in places.
However, some details of how the python packaging system is being utilized here may attract your attention and could be worth a look. (Patches 5-6 and 16-19 are the interesting ones.)
Since these scripts aren't critical, I'm OK with sending a fairly hasty PR to merge these sooner rather than later.
Overview:
---------Patch 1: Update Pipfile.lock (See the commit as for why ...)
Patches 2-3: Correct some existing typing issues in qemu.qmp
Patch 4: Combine qom-set, qom-get, (etc) into one, newly written script that makes all of the command invocations, help text, etc. consistent. (I ask that review for this patch should be limited to critical
mistakes: I have no interest in developing the QOM tools further.)Patches 5-6: Integrate the qom tools into the python package.
Patches 7-15: Delinting of the
qom_fusescript. Similarly, I am not terribly interested in further improvements here, personally.Patches 16-19: Integrating qom-fuse into the Python packaging directory; additional care is taken to ensure that "optional" dependencies like fusepy are handled well.
Changelog
---------V3:
- Technically, I sent two versions of this before, a long time ago. This has been cleaned up and based on the latest origin/master.John Snow (19):
python/pipenv: Update Pipfile.lock
python/qmp: Fix type of SocketAddrT
python/qmp: addparse_addressclassmethod
python/qmp: Add qom script rewrites
python/qmp: add qom script entry points
scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/scripts/qom-fuse: apply isort rules
scripts/qom-fuse: apply flake8 rules
python: Add 'fh' to known-good variable namesscripts/qom-fuse: Apply pylint rules
scripts/qom-fuse: Add docstrings
scripts/qom-fuse: Convert to QOMCommand
scripts/qom-fuse: useQOMCommand.qom_list()scripts/qom-fuse: ensure QOMFuse.read always returns bytesscripts/qom-fuse: add static type hints
python: add optional FUSE dependencies
scripts/qom-fuse: move topython/qemu/qmp/qom_fuse.pyscripts/qom-fuse: add redirection shim topython/qemu/qmp/qom-fuse.pypython/qmp: add fuse command to 'qom' toolspython/Pipfile.lock | 97 +++++++++++- python/qemu/qmp/__init__.py | 28 +++- python/qemu/qmp/qom.py | 272 ++++++++++++++++++++++++++++++++++ python/qemu/qmp/qom_common.py | 178 ++++++++++++++++++++++ python/qemu/qmp/qom_fuse.py | 206 +++++++++++++++++++++++++ python/setup.cfg | 33 ++++- scripts/qmp/qmp-shell | 21 +-- scripts/qmp/qom-fuse | 144 +----------------- scripts/qmp/qom-get | 66 +-------- scripts/qmp/qom-list | 63 +------- scripts/qmp/qom-set | 63 +------- scripts/qmp/qom-tree | 74 +-------- 12 files changed, 828 insertions(+), 417 deletions(-)
Quoted 4 lines
create mode 100644
python/qemu/qmp/qom.pycreate mode 100644python/qemu/qmp/qom_common.pycreate mode 100644python/qemu/qmp/qom_fuse.py
Provisionally staged to my Python branch:
https://gitlab.com/jsnow/qemu/-/commits/python/
(Barring objections, I intend to send a PR for all the ./scripts/qmp/ cleanups at once, next Friday.)
--js