AQMP TUI Draft
v3 cover AQMP TUI Draft
Hello all,
Gitlab: https://gitlab.com/niteesh.gs/qemu/-/commits/aqmp-tui-prototype-v3
CI: https://gitlab.com/niteesh.gs/qemu/-/pipelines/345738265
Major revision since V2:
1) Refined QMP connection manager.
2) Added static typing.
3) Allow highlighting specific messages in history box.
4) Allow copying of QMP message from TUI using keyboard shortcut alt-c.
G S Niteesh Babu (13):
python/aqmp: Fix wait_closed work-around for python 3.6
python: disable pylint errors for aqmp-tui
python: Add dependencies for AQMP TUI python/aqmp-tui: Add AQMP TUI draft
python: add entry point for aqmp-tui python/aqmp-tui: Added type annotations for aqmp-tui
python: add optional pygments dependency python/aqmp-tui: add syntax highlighting python/aqmp-tui: Add QMP connection manager python/aqmp-tui: Add scrolling to history box python/aqmp-tui: Add ability to highlight messages python/aqmp-tui: Add pyperclip dependency python/aqmp-tui: Allow copying message from TUI
python/Pipfile.lock | 42 +++ python/qemu/aqmp/aqmp_tui.py | 519 +++++++++++++++++++++++++++++++++++ python/qemu/aqmp/util.py | 12 +- python/setup.cfg | 34 ++- 4 files changed, 605 insertions(+), 2 deletions(-) create mode 100644 python/qemu/aqmp/aqmp_tui.py
Signature · 1 line
2.17.1
PATCH v3 1/13 python/aqmp: Fix wait_closed work-around for python 3.6
Before this patch the wait_closed work-around for python 3.6 fails during disconnect.
This is a temproray work around for which might be fixed in the future or will be completely removed when the minimum python version is raised to 3.7.
This patch was originally written by John Snow <[email protected]>
--- python/qemu/aqmp/util.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
python/qemu/aqmp/util.py +11 -1
@@ -134,7 +134,17 @@ def is_closing(writer: asyncio.StreamWriter) -> bool: while not transport.is_closing(): await asyncio.sleep(0) - await flush(writer) + + # This is an ugly workaround, but it's the best I can come up with. + sock = transport.get_extra_info('socket') + + if sock is None: + # Our transport doesn't have a socket? ... + # Nothing we can reasonably do. + return + + while sock.fileno() != -1: + await asyncio.sleep(0) def asyncio_run(coro: Coroutine[Any, Any, T], *, debug: bool = False) -> T:
Signature · 1 line
2.17.1
Re: [PATCH v3 01/13] python/aqmp: Fix wait_closed work-around for python 3.6
Quoted 36 lines · G S Niteesh Babu
Before this patch the
wait_closedwork-around for python 3.6 fails during disconnect.
This is a temproray work around for which might be fixed in the future or will be completely removed when the minimum python version is raised to 3.7.This patch was originally written by John Snow <[email protected]>
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/util.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)python/qemu/aqmp/util.py +11
-1@@ -134,7 +134,17 @@ def is_closing(writer: asyncio.StreamWriter) -> bool: while not transport.is_closing(): await asyncio.sleep(0) - await flush(writer) + + # This is an ugly workaround, but it's the best I can come up with. + sock = transport.get_extra_info('socket') + + if sock is None: + # Our transport doesn't have a socket? ... + # Nothing we can reasonably do. + return + + while sock.fileno() != -1: + await asyncio.sleep(0) def asyncio_run(coro: Coroutine[Any, Any, T], *, debug: bool = False) ->
Quoted 5 lines
T:
--
2.17.1
Sorry for the trouble. This is now included in the v3 version of my series and can be dropped. I hope.
Re: [PATCH v3 01/13] python/aqmp: Fix wait_closed work-around for python 3.6
Quoted 40 lines · John Snow
Quoted 35 lines · G S Niteesh Babu
Before this patch the
wait_closedwork-around for python 3.6 fails during disconnect.
This is a temproray work around for which might be fixed in the future or will be completely removed when the minimum python version is raised to 3.7.This patch was originally written by John Snow <[email protected]>
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/util.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)python/qemu/aqmp/util.py +11
-1@@ -134,7 +134,17 @@ def is_closing(writer: asyncio.StreamWriter) -> bool: while not transport.is_closing(): await asyncio.sleep(0) - await flush(writer) + + # This is an ugly workaround, but it's the best I can come up with. + sock = transport.get_extra_info('socket') + + if sock is None: + # Our transport doesn't have a socket? ... + # Nothing we can reasonably do. + return + + while sock.fileno() != -1: + await asyncio.sleep(0)
Quoted 9 lines
Quoted 6 lines
python/qemu/aqmp/util.py +0
-0def asyncio_run(coro: Coroutine[Any, Any, T], *, debug: bool = False) ->T:
--
2.17.1Sorry for the trouble. This is now included in the v3 version of my series and can be dropped. I hope.
Thanks. I'll remove this in the upcoming v4
PATCH v3 2/13 python: disable pylint errors for aqmp-tui
In reply to G S Niteesh BabuDisable missing-docstring and fixme pylint warnings. This is because since the AQMP is just a prototype it is currently not documented properly and lot of todo and fixme's are still in place.
--- python/setup.cfg | 2 ++ 1 file changed, 2 insertions(+)
python/setup.cfg +2 -0
@@ -90,6 +90,8 @@ ignore_missing_imports = True # --disable=W". disable=too-many-function-args, # mypy handles this with less false positives. no-member, # mypy also handles this better. + missing-docstring, # FIXME + fixme, # FIXME [pylint.basic] # Good variable names which should always be accepted, separated by a comma.
Signature · 1 line
2.17.1
Re: [PATCH v3 02/13] python: disable pylint errors for aqmp-tui
Quoted 21 lines · G S Niteesh Babu
Disable missing-docstring and fixme pylint warnings. This is because since the AQMP is just a prototype it is currently not documented properly and lot of todo and fixme's are still in place.
Signed-off-by G S Niteesh Babu <[email protected]>--- python/setup.cfg | 2 ++ 1 file changed, 2 insertions(+)python/setup.cfg +0
-0@@ -90,6 +90,8 @@ ignore_missing_imports = True # --disable=W". disable=too-many-function-args, # mypy handles this with less falsepositives.
no-member, # mypy also handles this better.
python/setup.cfg ignore_missing_imports = True · line 94
Quoted 2 lines
python/setup.cfg +2
-0+ missing-docstring, # FIXME + fixme, # FIXME
Please put some attention into removing the missing-docstring flag. At this point, anything that is "FIXME" should either be fixed or re-worded as a "TODO" and an exemption added to the pylint configuration such that "TODO" is allowed but "FIXME" is not.
Take a look at pylint --generate-rcfile and find this section:
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma. notes=FIXME,
XXX,
TODO
Quoted 7 lines
[pylint.basic]
# Good variable names which should always be accepted, separated by a
comma.
--
2.17.1
PATCH v3 3/13 python: Add dependencies for AQMP TUI
In reply to G S Niteesh BabuAdded dependencies for the upcoming AQMP TUI under the optional 'tui' group.
The same dependencies have also been added under the devel group since no work around has been found for optional groups to imply other optional groups.
--- python/Pipfile.lock | 12 ++++++++++++ python/setup.cfg | 8 ++++++++ 2 files changed, 20 insertions(+)
python/Pipfile.lock +12 -0
@@ -289,6 +289,18 @@ "markers": "python_version < '3.8'", "version": "==3.10.0.0" }, + "urwid": { + "hashes": [ + "sha256:588bee9c1cb208d0906a9f73c613d2bd32c3ed3702012f51efe318a3f2127eae" + ], + "version": "==2.1.2" + }, + "urwid-readline": { + "hashes": [ + "sha256:018020cbc864bb5ed87be17dc26b069eae2755cb29f3a9c569aac3bded1efaf4" + ], + "version": "==0.13" + }, "virtualenv": { "hashes": [ "sha256:14fdf849f80dbb29a4eb6caa9875d476ee2a5cf76a5f5415fa2f1606010ab467",
python/setup.cfg +8 -0
@@ -44,11 +44,18 @@ devel = mypy >= 0.770 pylint >= 2.8.0 tox >= 3.18.0 + urwid >= 2.1.2 + urwid-readline >= 0.13 # Provides qom-fuse functionality fuse = fusepy >= 2.0.4 +# AQMP TUI dependencies +tui = + urwid >= 2.1.2 + urwid-readline >= 0.13 + [options.entry_points] console_scripts = qom = qemu.qmp.qom:main
@@ -133,5 +140,6 @@ allowlist_externals = make deps = .[devel] .[fuse] # Workaround to trigger tox venv rebuild + .[tui] # Workaround to trigger tox venv rebuild commands = make check
Signature · 1 line
2.17.1
Re: [PATCH v3 03/13] python: Add dependencies for AQMP TUI
Quoted 9 lines · G S Niteesh Babu
Added dependencies for the upcoming AQMP TUI under the optional 'tui' group.
The same dependencies have also been added under the devel group since no work around has been found for optional groups to imply other optional groups.
Signed-off-by G S Niteesh Babu <[email protected]>
PATCH v3 4/13 python/aqmp-tui: Add AQMP TUI draft
In reply to G S Niteesh BabuAdded a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.
Also added necessary pylint, mypy configurations
--- python/qemu/aqmp/aqmp_tui.py | 333 +++++++++++++++++++++++++++++++++++ python/setup.cfg | 16 +- 2 files changed, 348 insertions(+), 1 deletion(-) create mode 100644 python/qemu/aqmp/aqmp_tui.py
python/qemu/aqmp/aqmp_tui.py new file +333 -0
@@ -0,0 +1,333 @@ +# Copyright (c) 2021 +# +# Authors: +# Niteesh Babu G S <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import asyncio +import logging +from logging import Handler +import signal + +import urwid +import urwid_readline + +from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .message import DeserializationError, Message, UnexpectedTypeError +from .protocol import ConnectError +from .qmp_client import ExecInterruptedError, QMPClient +from .util import create_task, pretty_traceback + + +UPDATE_MSG = 'UPDATE_MSG' + +# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger() + + +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing when looking + along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at more complex JSON + # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) + + +class App(QMPClient): + def __init__(self, address): + urwid.register_signal(type(self), UPDATE_MSG) + self.window = Window(self) + self.address = address + self.aloop = None + super().__init__() + + def add_to_history(self, msg): + urwid.emit_signal(self, UPDATE_MSG, msg) + + def _cb_outbound(self, msg): + # FIXME: I think the ideal way to omit these messages during in-TUI + # logging will be to add a filter to the logger. We can use regex to + # filter out messages starting with 'Request:' or 'Response:' but I + # think a better approach will be encapsulate the message in an object + # and filter based on the object. Encapsulation of the message will + # also be necessary when we want different formatting of messages + # inside TUI. + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Request: %s', str(msg)) + self.add_to_history('<-- ' + str(msg)) + return msg + + def _cb_inbound(self, msg): + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Response: %s', str(msg)) + self.add_to_history('--> ' + str(msg)) + return msg + + async def wait_for_events(self): + async for event in self.events: + self.handle_event(event) + + async def _send_to_server(self, raw_msg): + # FIXME: Format the raw_msg in history view to one line. It is not + # pleasing to see multiple lines JSON object with an error statement. + try: + msg = Message(bytes(raw_msg, encoding='utf-8')) + # Format multiline json into a single line JSON, since it is more + # pleasing to look along with err message in TUI. + raw_msg = self.format_json(raw_msg) + await self._raw(msg, assign_id='id' not in msg) + except (ValueError, TypeError) as err: + LOGGER.info('Invalid message: %s', str(err)) + self.add_to_history(f'{raw_msg}: {err}') + except (DeserializationError, UnexpectedTypeError) as err: + LOGGER.info('Invalid message: %s', err.error_message) + self.add_to_history(f'{raw_msg}: {err.error_message}') + except ExecInterruptedError: + LOGGER.info('Error server disconnected before reply') + urwid.emit_signal(self, UPDATE_MSG, + '{"error": "Server disconnected before reply"}') + self._set_status("Server disconnected") + except Exception as err: + LOGGER.error('Exception from _send_to_server: %s', str(err)) + raise err + + def cb_send_to_server(self, msg): + create_task(self._send_to_server(msg)) + + def unhandled_input(self, key): + if key == 'esc': + self.kill_app() + + def kill_app(self): + # TODO: Work on the disconnect logic + create_task(self._kill_app()) + + async def _kill_app(self): + # It is ok to call disconnect even in disconnect state + try: + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + except Exception as err: + LOGGER.info('_kill_app: %s', str(err)) + # Let the app crash after providing a proper stack trace + raise err + raise urwid.ExitMainLoop() + + def handle_event(self, event): + # FIXME: Consider all states present in qapi/run-state.json + if event['event'] == 'SHUTDOWN': + self._set_status('Server shutdown') + + def _set_status(self, msg: str) -> None: + self.window.footer.set_text(msg) + + def _get_formatted_address(self) -> str: + addr = f'{self.address}' + if isinstance(self.address, tuple): + host, port = self.address + addr = f'{host}:{port}' + return addr + + async def connect_server(self): + try: + await self.connect(self.address) + addr = self._get_formatted_address() + self._set_status(f'Connected to {addr}') + except ConnectError as err: + LOGGER.info('connect_server: ConnectError %s', str(err)) + self._set_status('Server shutdown') + + def run(self, debug=False): + self.aloop = asyncio.get_event_loop() + self.aloop.set_debug(debug) + + # Gracefully handle SIGTERM and SIGINT signals + cancel_signals = [signal.SIGTERM, signal.SIGINT] + for sig in cancel_signals: + self.aloop.add_signal_handler(sig, self.kill_app) + + event_loop = urwid.AsyncioEventLoop(loop=self.aloop) + main_loop = urwid.MainLoop(urwid.AttrMap(self.window, 'background'), + unhandled_input=self.unhandled_input, + handle_mouse=True, + event_loop=event_loop) + + create_task(self.wait_for_events(), self.aloop) + create_task(self.connect_server(), self.aloop) + try: + main_loop.run() + except Exception as err: + LOGGER.error('%s\n%s\n', str(err), pretty_traceback()) + raise err + + +class StatusBar(urwid.Text): + """ + A simple Text widget that currently only shows connection status. + """ + def __init__(self, text=''): + super().__init__(text, align='right') + + +class Editor(urwid_readline.ReadlineEdit): + """ + Support urwid_readline features along with + history support which lacks in urwid_readline + """ + def __init__(self, master): + super().__init__(caption='> ', multiline=True) + self.master = master + self.history = [] + self.last_index = -1 + self.show_history = False + + def keypress(self, size, key): + # TODO: Add some logic for down key and clean up logic if possible. + # Returning None means the key has been handled by this widget + # which otherwise is propogated to the parent widget to be + # handled + msg = self.get_edit_text() + if key == 'up' and not msg: + # Show the history when 'up arrow' is pressed with no input text. + # NOTE: The show_history logic is necessary because in 'multiline' + # mode (which we use) 'up arrow' is used to move between lines. + self.show_history = True + last_msg = self.history[self.last_index] if self.history else '' + self.set_edit_text(last_msg) + self.edit_pos = len(last_msg) + self.last_index += 1 + elif key == 'up' and self.show_history: + if self.last_index < len(self.history): + self.set_edit_text(self.history[self.last_index]) + self.edit_pos = len(self.history[self.last_index]) + self.last_index += 1 + elif key == 'meta enter': + # When using multiline, enter inserts a new line into the editor + # send the input to the server on alt + enter + self.master.cb_send_to_server(msg) + self.history.insert(0, msg) + self.set_edit_text('') + self.last_index = 0 + self.show_history = False + else: + self.show_history = False + self.last_index = 0 + return super().keypress(size, key) + return None + + +class EditorWidget(urwid.Filler): + """ + Wraps CustomEdit + """ + def __init__(self, master): + super().__init__(Editor(master), valign='top') + + +class HistoryBox(urwid.ListBox): + """ + Shows all the QMP message transmitted/received + """ + def __init__(self, master): + self.master = master + self.history = urwid.SimpleFocusListWalker([]) + super().__init__(self.history) + + def add_to_history(self, history): + self.history.append(urwid.Text(history)) + if self.history: + self.history.set_focus(len(self.history) - 1) + + +class HistoryWindow(urwid.Frame): + """ + Composes the HistoryBox and EditorWidget + """ + def __init__(self, master): + self.master = master + self.editor_widget = EditorWidget(master) + self.editor = urwid.LineBox(self.editor_widget) + self.history = HistoryBox(master) + self.body = urwid.Pile([('weight', 80, self.history), + ('weight', 20, self.editor)]) + super().__init__(self.body) + urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) + + def cb_add_to_history(self, msg): + self.history.add_to_history(msg) + + +class Window(urwid.Frame): + """ + This is going to be the main window that is going to compose other + windows. In this stage it is unnecesssary but will be necessary in + future when we will have multiple windows and want to the switch between + them and display overlays + """ + def __init__(self, master): + self.master = master + footer = StatusBar() + body = HistoryWindow(master) + super().__init__(body, footer=footer) + + +class TUILogHandler(Handler): + def __init__(self, tui): + super().__init__() + self.tui = tui + + def emit(self, record): + level = record.levelname + msg = record.getMessage() + msg = f'[{level}]: {msg}' + self.tui.add_to_history(msg) + + +def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level <CRITICAL|ERROR|WARNING|INFO|DEBUG|>') + parser.add_argument('--asyncio-debug', action='store_true', + help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusable when' + 'logs are logged in the TUI itself.' + 'Use only when logging to a file') + args = parser.parse_args() + + try: + address = QEMUMonitorProtocol.parse_address(args.qmp_server) + except QMPBadPortError as err: + parser.error(err) + + app = App(address) + + if args.log_file: + LOGGER.addHandler(logging.FileHandler(args.log_file)) + else: + LOGGER.addHandler(TUILogHandler(app)) + + log_level = logging.getLevelName(args.log_level) + # getLevelName returns 'Level {log_level}' when a invalid level is passed. + if log_level == f'Level {args.log_level}': + parser.error('Invalid log level') + LOGGER.setLevel(log_level) + + app.run(args.asyncio_debug) + + +if __name__ == '__main__': + main() # type: ignore
python/setup.cfg +15 -1
@@ -81,8 +81,22 @@ namespace_packages = True # fusepy has no type stubs: allow_subclassing_any = True +[mypy-qemu.aqmp.aqmp_tui] +disallow_untyped_defs = False +disallow_incomplete_defs = False +check_untyped_defs = False +# urwid and urwid_readline have no type stubs: +allow_subclassing_any = True + +# The following missing import directives are because these libraries do not +# provide type stubs. Allow them on an as-needed basis for mypy. [mypy-fuse] -# fusepy has no type stubs: +ignore_missing_imports = True + +[mypy-urwid] +ignore_missing_imports = True + +[mypy-urwid_readline] ignore_missing_imports = True [pylint.messages control]
Signature · 1 line
2.17.1
Re: [PATCH v3 04/13] python/aqmp-tui: Add AQMP TUI draft
Quoted 38 lines · G S Niteesh Babu
Added a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.Also added necessary pylint, mypy configurations
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 333 +++++++++++++++++++++++++++++++++++ python/setup.cfg | 16 +- 2 files changed, 348 insertions(+), 1 deletion(-) create mode 100644 python/qemu/aqmp/aqmp_tui.pypython/qemu/aqmp/aqmp_tui.py new file +15
-0@@ -0,0 +1,333 @@ +# Copyright (c) 2021 +# +# Authors: +# Niteesh Babu G S <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import asyncio +import logging +from logging import Handler +import signal + +import urwid
python/qemu/aqmp/aqmp_tui.py line 25
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py new file +10
-0+import urwid_readline + +from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .message import DeserializationError, Message, UnexpectedTypeError +from .protocol import ConnectError +from .qmp_client import ExecInterruptedError, QMPClient +from .util import create_task, pretty_traceback + + +UPDATE_MSG = 'UPDATE_MSG'
This line still feels kind of "naked" on a cold read. It could use a small comment.
python/qemu/aqmp/aqmp_tui.py line 28
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ +# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger()
The comment isn't quite true; this is the root logger -- but you go on to use it to directly log messages. I don't think you should; use a module-level logger.
(1) Create a module-level logger that is named after the current module name (e.g. qemu.aqmp.aqmp_tui) and use that for logging messages relating to this module:
LOGGER = logging.getLogger(__name__)
(2) Where you need to address the root logger, just use root_logger = logging.getLogger() .... I think the main() function is the only place you might need this.
Quoted 11 lines
python/qemu/aqmp/aqmp_tui.py new file +9
-0+ + +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing when looking + along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at more complexJSON
python/qemu/aqmp/aqmp_tui.py line 43
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) +
You can use the JSON module to do this,
https://docs.python.org/3/library/json.html#json.dumps
Message._serialize uses this technique to send JSON messages over the wire that have no newlines:
https://gitlab.com/jsnow/qemu/-/blob/python-async-qmp-aqmp/python/qemu/aqmp/message.py#L132
by not specifying an indent and including separators with no spaces, we can get a JSON representation with all the space squeezed out. You can add spaces and still omit the indent/newlines and so on.
python/qemu/aqmp/aqmp_tui.py line 47
Quoted 5 lines
python/qemu/aqmp/aqmp_tui.py new file +4
-0+ +class App(QMPClient): + def __init__(self, address): + urwid.register_signal(type(self), UPDATE_MSG)
I found some hidden documentation about this -- I was struggling to find it before:
https://github.com/urwid/urwid/blob/master/urwid/signals.py#L62
So the first argument is meant to be "The class of an object sending the signal". Alright. And it looks like we only emit from add_to_history and in _send_to_server, so I guess that checks out!
One thing I am noticing is that these signals are global and not per-instance, so if for some reason we ever create a second App I wonder if the __init__ method here will explode ... Well, it's probably not too important right now.
Quoted 21 lines
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ self.window = Window(self) + self.address = address + self.aloop = None + super().__init__() + + def add_to_history(self, msg): + urwid.emit_signal(self, UPDATE_MSG, msg) + + def _cb_outbound(self, msg): + # FIXME: I think the ideal way to omit these messages duringin-TUI
+ # logging will be to add a filter to the logger. We can use regex
to
+ # filter out messages starting with 'Request:' or 'Response:' but
I
+ # think a better approach will be encapsulate the message in an
object
+ # and filter based on the object. Encapsulation of the message
will
python/qemu/aqmp/aqmp_tui.py line 63
Quoted 2 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ # also be necessary when we want different formatting of messages + # inside TUI.
+ handler = LOGGER.handlers[0]
Quoted 7 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ if not isinstance(handler, TUILogHandler): + LOGGER.debug('Request: %s', str(msg)) + self.add_to_history('<-- ' + str(msg)) + return msg + + def _cb_inbound(self, msg):
python/qemu/aqmp/aqmp_tui.py line 80
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Response: %s', str(msg)) + self.add_to_history('--> ' + str(msg)) + return msg + + async def wait_for_events(self): + async for event in self.events: + self.handle_event(event) +
Can handle_event be moved here so that related functions are somewhat near each other?
Quoted 29 lines
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ async def _send_to_server(self, raw_msg): + # FIXME: Format the raw_msg in history view to one line. It is not + # pleasing to see multiple lines JSON object with an errorstatement.
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ try: + msg = Message(bytes(raw_msg, encoding='utf-8')) + # Format multiline json into a single line JSON, since it ismore
python/qemu/aqmp/aqmp_tui.py new file +13
-0+ # pleasing to look along with err message in TUI. + raw_msg = self.format_json(raw_msg) + await self._raw(msg, assign_id='id' not in msg) + except (ValueError, TypeError) as err: + LOGGER.info('Invalid message: %s', str(err)) + self.add_to_history(f'{raw_msg}: {err}') + except (DeserializationError, UnexpectedTypeError) as err: + LOGGER.info('Invalid message: %s', err.error_message) + self.add_to_history(f'{raw_msg}: {err.error_message}') + except ExecInterruptedError: + LOGGER.info('Error server disconnected before reply') + urwid.emit_signal(self, UPDATE_MSG, + '{"error": "Server disconnected beforereply"}')
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ self._set_status("Server disconnected") + except Exception as err: + LOGGER.error('Exception from _send_to_server: %s', str(err)) + raise err + + def cb_send_to_server(self, msg):
python/qemu/aqmp/aqmp_tui.py line 115
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ create_task(self._send_to_server(msg)) + + def unhandled_input(self, key): + if key == 'esc': + self.kill_app() + + def kill_app(self): + # TODO: Work on the disconnect logic + create_task(self._kill_app()) +
That logic needs to be here -- each commit needs to make sense by itself. If you've improved the connection logic, it needs to be merged into this commit.
Quoted 7 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ async def _kill_app(self): + # It is ok to call disconnect even in disconnect state + try: + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + except Exception as err:
python/qemu/aqmp/aqmp_tui.py line 131
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ LOGGER.info('_kill_app: %s', str(err)) + # Let the app crash after providing a proper stack trace + raise err + raise urwid.ExitMainLoop() + + def handle_event(self, event): + # FIXME: Consider all states present in qapi/run-state.json + if event['event'] == 'SHUTDOWN': + self._set_status('Server shutdown') +
This is an odd function to me because it will *overwrite* the status bar with this event, but surely shortly thereafter we're going to see a disconnection event from the transport itself, so this message will go away almost immediately, won't it? I understand that connection management stuff is improved in later patches, but the initial patch here would be better served not having a partial and incorrect implementation. You can just remove features that you intend to "build out" in later commits. And if we remove this, then we don't need wait_for_events() either, and they can be re-added in a later commit when they're given a more full, proper treatment.
python/qemu/aqmp/aqmp_tui.py line 140
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py new file +9
-0+ def _set_status(self, msg: str) -> None: + self.window.footer.set_text(msg) + + def _get_formatted_address(self) -> str: + addr = f'{self.address}' + if isinstance(self.address, tuple): + host, port = self.address + addr = f'{host}:{port}' + return addr
Don't format addr twice in the TCP case -- make sure it formats it once. It won't matter much for performance in something like Python, but it's misleading when reading it.
+
python/qemu/aqmp/aqmp_tui.py line 149
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py new file +8
-0+ async def connect_server(self): + try: + await self.connect(self.address) + addr = self._get_formatted_address() + self._set_status(f'Connected to {addr}') + except ConnectError as err: + LOGGER.info('connect_server: ConnectError %s', str(err)) + self._set_status('Server shutdown')
A failure to connect doesn't necessary mean the server was shut down.
Quoted 2 lines
+
I suspect this is changed in later commits in this series, but you should probably set the footer to something before we potentially hang on connect() and do a whole lot of nothing. LIke other critiques about unfinished interfaces, once you've implemented them, they should be worked into this patch so that each commit makes sense by itself. Reviewers will have to read these patches and make sense of them. If they have to flip through your entire series to understand what it will eventually look like, it's a lot more work and you're less likely to get good or helpful reviews.
Quoted 164 lines
python/qemu/aqmp/aqmp_tui.py new file +11
-0+ def run(self, debug=False): + self.aloop = asyncio.get_event_loop() + self.aloop.set_debug(debug) + + # Gracefully handle SIGTERM and SIGINT signals + cancel_signals = [signal.SIGTERM, signal.SIGINT] + for sig in cancel_signals: + self.aloop.add_signal_handler(sig, self.kill_app) + + event_loop = urwid.AsyncioEventLoop(loop=self.aloop) + main_loop = urwid.MainLoop(urwid.AttrMap(self.window,'background'),
python/qemu/aqmp/aqmp_tui.py new file +35
-0+ unhandled_input=self.unhandled_input, + handle_mouse=True, + event_loop=event_loop) + + create_task(self.wait_for_events(), self.aloop) + create_task(self.connect_server(), self.aloop) + try: + main_loop.run() + except Exception as err: + LOGGER.error('%s\n%s\n', str(err), pretty_traceback()) + raise err + + +class StatusBar(urwid.Text): + """ + A simple Text widget that currently only shows connection status. + """ + def __init__(self, text=''): + super().__init__(text, align='right') + + +class Editor(urwid_readline.ReadlineEdit): + """ + Support urwid_readline features along with + history support which lacks in urwid_readline + """ + def __init__(self, master): + super().__init__(caption='> ', multiline=True) + self.master = master + self.history = [] + self.last_index = -1 + self.show_history = False + + def keypress(self, size, key): + # TODO: Add some logic for down key and clean up logic ifpossible.
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # Returning None means the key has been handled by this widget + # which otherwise is propogated to the parent widget to be + # handled + msg = self.get_edit_text() + if key == 'up' and not msg: + # Show the history when 'up arrow' is pressed with no inputtext.
+ # NOTE: Theshow_historylogic is necessary because in
'multiline'
+ # mode (which we use) 'up arrow' is used to move between
lines.python/qemu/aqmp/aqmp_tui.py new file +2
-0+ self.show_history = True + last_msg = self.history[self.last_index] if self.history else''
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ self.set_edit_text(last_msg) + self.edit_pos = len(last_msg) + self.last_index += 1 + elif key == 'up' and self.show_history: + if self.last_index < len(self.history): + self.set_edit_text(self.history[self.last_index]) + self.edit_pos = len(self.history[self.last_index]) + self.last_index += 1 + elif key == 'meta enter': + # When using multiline, enter inserts a new line into theeditor
python/qemu/aqmp/aqmp_tui.py new file +49
-0+ # send the input to the server on alt + enter + self.master.cb_send_to_server(msg) + self.history.insert(0, msg) + self.set_edit_text('') + self.last_index = 0 + self.show_history = False + else: + self.show_history = False + self.last_index = 0 + return super().keypress(size, key) + return None + + +class EditorWidget(urwid.Filler): + """ + Wraps CustomEdit + """ + def __init__(self, master): + super().__init__(Editor(master), valign='top') + + +class HistoryBox(urwid.ListBox): + """ + Shows all the QMP message transmitted/received + """ + def __init__(self, master): + self.master = master + self.history = urwid.SimpleFocusListWalker([]) + super().__init__(self.history) + + def add_to_history(self, history): + self.history.append(urwid.Text(history)) + if self.history: + self.history.set_focus(len(self.history) - 1) + + +class HistoryWindow(urwid.Frame): + """ + Composes the HistoryBox and EditorWidget + """ + def __init__(self, master): + self.master = master + self.editor_widget = EditorWidget(master) + self.editor = urwid.LineBox(self.editor_widget) + self.history = HistoryBox(master) + self.body = urwid.Pile([('weight', 80, self.history), + ('weight', 20, self.editor)]) + super().__init__(self.body) + urwid.connect_signal(self.master, UPDATE_MSG,self.cb_add_to_history)
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ + def cb_add_to_history(self, msg): + self.history.add_to_history(msg) + + +class Window(urwid.Frame): + """ + This is going to be the main window that is going to compose other + windows. In this stage it is unnecesssary but will be necessary in + future when we will have multiple windows and want to the switchbetween
python/qemu/aqmp/aqmp_tui.py new file +28
-0+ them and display overlays + """ + def __init__(self, master): + self.master = master + footer = StatusBar() + body = HistoryWindow(master) + super().__init__(body, footer=footer) + + +class TUILogHandler(Handler): + def __init__(self, tui): + super().__init__() + self.tui = tui + + def emit(self, record): + level = record.levelname + msg = record.getMessage() + msg = f'[{level}]: {msg}' + self.tui.add_to_history(msg) + + +def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
python/qemu/aqmp/aqmp_tui.py line 308
Quoted 5 lines
python/qemu/aqmp/aqmp_tui.py new file +5
-0+ parser.add_argument('--asyncio-debug', action='store_true', + help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusable when' + 'logs are logged in the TUI itself.' + 'Use only when logging to a file')
Needs spaces between the lines here so that the output reads correctly.
Quoted 18 lines
python/qemu/aqmp/aqmp_tui.py new file +16
-0+ args = parser.parse_args() + + try: + address = QEMUMonitorProtocol.parse_address(args.qmp_server) + except QMPBadPortError as err: + parser.error(err) + + app = App(address) + + if args.log_file: + LOGGER.addHandler(logging.FileHandler(args.log_file)) + else: + LOGGER.addHandler(TUILogHandler(app)) + + log_level = logging.getLevelName(args.log_level) + # getLevelName returns 'Level {log_level}' when a invalid level ispassed.
python/qemu/aqmp/aqmp_tui.py line 328
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py new file +4
-0+ if log_level == f'Level {args.log_level}': + parser.error('Invalid log level') + LOGGER.setLevel(log_level) +
I think you could do:
LOGGER.setLevel(logging.getLevelName(args.log_level))
And it'll fail if the level was unknown, but work otherwise. This would avoid a kind of precarious string-comparison check, which I worry is fragile.
Quoted 35 lines
python/qemu/aqmp/aqmp_tui.py new file +5
-0+ app.run(args.asyncio_debug) + + +if __name__ == '__main__': + main() # type: ignorepython/setup.cfg +8
-0@@ -81,8 +81,22 @@ namespace_packages = True # fusepy has no type stubs: allow_subclassing_any = True +[mypy-qemu.aqmp.aqmp_tui] +disallow_untyped_defs = False +disallow_incomplete_defs = False +check_untyped_defs = False +# urwid and urwid_readline have no type stubs: +allow_subclassing_any = True + +# The following missing import directives are because these libraries donot
python/setup.cfg +7
-2+# provide type stubs. Allow them on an as-needed basis for mypy. [mypy-fuse] -# fusepy has no type stubs: +ignore_missing_imports = True + +[mypy-urwid] +ignore_missing_imports = True + +[mypy-urwid_readline] ignore_missing_imports = True [pylint.messages control] --
Quoted 3 lines
2.17.1
I have some more thoughts, but need to research them and wanted to give some feedback first so you weren't waiting.
Re: [PATCH v3 04/13] python/aqmp-tui: Add AQMP TUI draft
Quoted 48 lines · John Snow
Quoted 43 lines · G S Niteesh Babu
Added a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.Also added necessary pylint, mypy configurations
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 333 +++++++++++++++++++++++++++++++++++ python/setup.cfg | 16 +- 2 files changed, 348 insertions(+), 1 deletion(-) create mode 100644 python/qemu/aqmp/aqmp_tui.pypython/qemu/aqmp/aqmp_tui.py new file +21
-0@@ -0,0 +1,333 @@ +# Copyright (c) 2021 +# +# Authors: +# Niteesh Babu G S <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import asyncio +import logging +from logging import Handler +import signal + +import urwid +import urwid_readline + +from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .message import DeserializationError, Message, UnexpectedTypeError +from .protocol import ConnectError +from .qmp_client import ExecInterruptedError, QMPClient
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py line 25
Quoted 5 lines
python/qemu/aqmp/aqmp_tui.py new file +4
-0+from .util import create_task, pretty_traceback + + +UPDATE_MSG = 'UPDATE_MSG'This line still feels kind of "naked" on a cold read. It could use a small comment.
Fixed.
Quoted 11 lines
python/qemu/aqmp/aqmp_tui.py line 28
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ +# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger()The comment isn't quite true; this is the root logger -- but you go on to use it to directly log messages. I don't think you should; use a module-level logger.
Quoted 9 lines
(1) Create a module-level logger that is named after the current module name (e.g. qemu.aqmp.aqmp_tui) and use that for logging messages relating to this module:
LOGGER =logging.getLogger(__name__)(2) Where you need to address the root logger, just use
root_logger = logging.getLogger().... I think themain()function is the only place you might need this.
The way of logging in the TUI is changed such that, all logging is done through the root level logger. The handlers and levels are installed to the root level
logger to allow logging from other libraries to be rerouted to the screen or file.
Quoted 25 lines
Quoted 15 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ + +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing whenlooking
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at morecomplex JSON
python/qemu/aqmp/aqmp_tui.py line 43
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) +You can use the JSON module to do this,
https://docs.python.org/3/library/json.html#json.dumps
Quoted 9 lines
Message._serialize uses this technique to send JSON messages over the wire that have no newlines:
https://gitlab.com/jsnow/qemu/-/blob/python-async-qmp-aqmp/python/qemu/aqmp/message.py#L132
by not specifying an indent and including separators with no spaces, we can get a JSON representation with all the space squeezed out. You can add spaces and still omit the indent/newlines and so on.
But will this work for invalid JSON messages? As far as I have tried it doesn't work.
Quoted 57 lines
python/qemu/aqmp/aqmp_tui.py line 47
Quoted 7 lines
python/qemu/aqmp/aqmp_tui.py new file +4
-0+ +class App(QMPClient): + def __init__(self, address): + urwid.register_signal(type(self), UPDATE_MSG)I found some hidden documentation about this -- I was struggling to find it before:
https://github.com/urwid/urwid/blob/master/urwid/signals.py#L62So the first argument is meant to be "The class of an object sending the signal". Alright. And it looks like we only emit from
add_to_historyand in_send_to_server, so I guess that checks out!One thing I am noticing is that these signals are global and not per-instance, so if for some reason we ever create a second App I wonder if the
__init__method here will explode ... Well, it's probably not too important right now.Quoted 21 lines
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ self.window = Window(self) + self.address = address + self.aloop = None + super().__init__() + + def add_to_history(self, msg): + urwid.emit_signal(self, UPDATE_MSG, msg) + + def _cb_outbound(self, msg): + # FIXME: I think the ideal way to omit these messages duringin-TUI
+ # logging will be to add a filter to the logger. We can use
regex to
+ # filter out messages starting with 'Request:' or 'Response:'
but I
+ # think a better approach will be encapsulate the message in an
object
+ # and filter based on the object. Encapsulation of the message
willpython/qemu/aqmp/aqmp_tui.py line 63
Quoted 2 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ # also be necessary when we want different formatting of messages + # inside TUI.+ handler = LOGGER.handlers[0]
Quoted 12 lines
python/qemu/aqmp/aqmp_tui.py new file +12
-0+ if not isinstance(handler, TUILogHandler): + LOGGER.debug('Request: %s', str(msg)) + self.add_to_history('<-- ' + str(msg)) + return msg + + def _cb_inbound(self, msg): + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Response: %s', str(msg)) + self.add_to_history('--> ' + str(msg)) + return msg +
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py line 80
Quoted 5 lines
python/qemu/aqmp/aqmp_tui.py new file +4
-0+ async def wait_for_events(self): + async for event in self.events: + self.handle_event(event) +Can
handle_eventbe moved here so that related functions are somewhat near each other?
Fixed.
Quoted 38 lines
Quoted 38 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ async def _send_to_server(self, raw_msg): + # FIXME: Format the raw_msg in history view to one line. It isnot
+ # pleasing to see multiple lines JSON object with an error
statement.python/qemu/aqmp/aqmp_tui.py new file +3
-0+ try: + msg = Message(bytes(raw_msg, encoding='utf-8')) + # Format multiline json into a single line JSON, since it ismore
python/qemu/aqmp/aqmp_tui.py new file +13
-0+ # pleasing to look along with err message in TUI. + raw_msg = self.format_json(raw_msg) + await self._raw(msg, assign_id='id' not in msg) + except (ValueError, TypeError) as err: + LOGGER.info('Invalid message: %s', str(err)) + self.add_to_history(f'{raw_msg}: {err}') + except (DeserializationError, UnexpectedTypeError) as err: + LOGGER.info('Invalid message: %s', err.error_message) + self.add_to_history(f'{raw_msg}: {err.error_message}') + except ExecInterruptedError: + LOGGER.info('Error server disconnected before reply') + urwid.emit_signal(self, UPDATE_MSG, + '{"error": "Server disconnected beforereply"}')
python/qemu/aqmp/aqmp_tui.py new file +13
-0+ self._set_status("Server disconnected") + except Exception as err: + LOGGER.error('Exception from _send_to_server: %s', str(err)) + raise err + + def cb_send_to_server(self, msg): + create_task(self._send_to_server(msg)) + + def unhandled_input(self, key): + if key == 'esc': + self.kill_app() + + def kill_app(self):
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py line 115
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ # TODO: Work on the disconnect logic + create_task(self._kill_app()) +That logic needs to be here -- each commit needs to make sense by itself. If you've improved the connection logic, it needs to be merged into this commit.
Through this to-do, I meant to add the connection manager. I'll remove this TODO since it's kind of misleading.
Quoted 16 lines
Quoted 16 lines
python/qemu/aqmp/aqmp_tui.py new file +14
-0+ async def _kill_app(self): + # It is ok to call disconnect even in disconnect state + try: + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + except Exception as err: + LOGGER.info('_kill_app: %s', str(err)) + # Let the app crash after providing a proper stack trace + raise err + raise urwid.ExitMainLoop() + + def handle_event(self, event): + # FIXME: Consider all states present in qapi/run-state.json + if event['event'] == 'SHUTDOWN':
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py
Quoted 3 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ self._set_status('Server shutdown') +This is an odd function to me because it will *overwrite* the status bar with this event, but surely shortly thereafter we're going to see a disconnection event from the transport itself, so this message will go away almost immediately, won't it?
It won't be overwritten in this patch. Once the server shutdown/disconnects, the generated SHUTDOWN event is handled to update the status bar and the library internally calls disconnect and that's it. Only in the later patches do we decide whether to reconnect or not that's when there is a chance of overwriting. But in this patch, overwriting won't occur.
Quoted 19 lines
I understand that connection management stuff is improved in later patches, but the initial patch here would be better served not having a partial and incorrect implementation. You can just remove features that you intend to "build out" in later commits. And if we remove this, then we don't need
wait_for_events()either, and they can be re-added in a later commit when they're given a more full, proper treatment.python/qemu/aqmp/aqmp_tui.py line 140
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py new file +9
-0+ def _set_status(self, msg: str) -> None: + self.window.footer.set_text(msg) + + def _get_formatted_address(self) -> str: + addr = f'{self.address}' + if isinstance(self.address, tuple): + host, port = self.address + addr = f'{host}:{port}' + return addr
Quoted 4 lines
Don't format addr twice in the TCP case -- make sure it formats it once. It won't matter much for performance in something like Python, but it's misleading when reading it.
Fixed.
Quoted 12 lines
+
python/qemu/aqmp/aqmp_tui.py line 149
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py new file +8
-0+ async def connect_server(self): + try: + await self.connect(self.address) + addr = self._get_formatted_address() + self._set_status(f'Connected to {addr}') + except ConnectError as err: + LOGGER.info('connect_server: ConnectError %s', str(err)) + self._set_status('Server shutdown')
Quoted 2 lines
A failure to connect doesn't necessary mean the server was shut down.
Fixed.
It now shows [ConnectError: <Short description>] eg: [ConnectError: Failed to establish connection].
Quoted 179 lines
Quoted 4 lines
+
I suspect this is changed in later commits in this series, but you should probably set the footer to something before we potentially hang on
connect()and do a whole lot of nothing. LIke other critiques about unfinished interfaces, once you've implemented them, they should be worked into this patch so that each commit makes sense by itself. Reviewers will have to read these patches and make sense of them. If they have to flip through your entire series to understand what it will eventually look like, it's a lot more work and you're less likely to get good or helpful reviews.Quoted 164 lines
python/qemu/aqmp/aqmp_tui.py new file +11
-0+ def run(self, debug=False): + self.aloop = asyncio.get_event_loop() + self.aloop.set_debug(debug) + + # Gracefully handle SIGTERM and SIGINT signals + cancel_signals = [signal.SIGTERM, signal.SIGINT] + for sig in cancel_signals: + self.aloop.add_signal_handler(sig, self.kill_app) + + event_loop = urwid.AsyncioEventLoop(loop=self.aloop) + main_loop = urwid.MainLoop(urwid.AttrMap(self.window,'background'),
python/qemu/aqmp/aqmp_tui.py new file +35
-0+ unhandled_input=self.unhandled_input, + handle_mouse=True, + event_loop=event_loop) + + create_task(self.wait_for_events(), self.aloop) + create_task(self.connect_server(), self.aloop) + try: + main_loop.run() + except Exception as err: + LOGGER.error('%s\n%s\n', str(err), pretty_traceback()) + raise err + + +class StatusBar(urwid.Text): + """ + A simple Text widget that currently only shows connection status. + """ + def __init__(self, text=''): + super().__init__(text, align='right') + + +class Editor(urwid_readline.ReadlineEdit): + """ + Support urwid_readline features along with + history support which lacks in urwid_readline + """ + def __init__(self, master): + super().__init__(caption='> ', multiline=True) + self.master = master + self.history = [] + self.last_index = -1 + self.show_history = False + + def keypress(self, size, key): + # TODO: Add some logic for down key and clean up logic ifpossible.
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # Returning None means the key has been handled by this widget + # which otherwise is propogated to the parent widget to be + # handled + msg = self.get_edit_text() + if key == 'up' and not msg: + # Show the history when 'up arrow' is pressed with no inputtext.
+ # NOTE: Theshow_historylogic is necessary because in
'multiline'
+ # mode (which we use) 'up arrow' is used to move between
lines.python/qemu/aqmp/aqmp_tui.py new file +2
-0+ self.show_history = True + last_msg = self.history[self.last_index] if self.historyelse ''
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ self.set_edit_text(last_msg) + self.edit_pos = len(last_msg) + self.last_index += 1 + elif key == 'up' and self.show_history: + if self.last_index < len(self.history): + self.set_edit_text(self.history[self.last_index]) + self.edit_pos = len(self.history[self.last_index]) + self.last_index += 1 + elif key == 'meta enter': + # When using multiline, enter inserts a new line into theeditor
python/qemu/aqmp/aqmp_tui.py new file +49
-0+ # send the input to the server on alt + enter + self.master.cb_send_to_server(msg) + self.history.insert(0, msg) + self.set_edit_text('') + self.last_index = 0 + self.show_history = False + else: + self.show_history = False + self.last_index = 0 + return super().keypress(size, key) + return None + + +class EditorWidget(urwid.Filler): + """ + Wraps CustomEdit + """ + def __init__(self, master): + super().__init__(Editor(master), valign='top') + + +class HistoryBox(urwid.ListBox): + """ + Shows all the QMP message transmitted/received + """ + def __init__(self, master): + self.master = master + self.history = urwid.SimpleFocusListWalker([]) + super().__init__(self.history) + + def add_to_history(self, history): + self.history.append(urwid.Text(history)) + if self.history: + self.history.set_focus(len(self.history) - 1) + + +class HistoryWindow(urwid.Frame): + """ + Composes the HistoryBox and EditorWidget + """ + def __init__(self, master): + self.master = master + self.editor_widget = EditorWidget(master) + self.editor = urwid.LineBox(self.editor_widget) + self.history = HistoryBox(master) + self.body = urwid.Pile([('weight', 80, self.history), + ('weight', 20, self.editor)]) + super().__init__(self.body) + urwid.connect_signal(self.master, UPDATE_MSG,self.cb_add_to_history)
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ + def cb_add_to_history(self, msg): + self.history.add_to_history(msg) + + +class Window(urwid.Frame): + """ + This is going to be the main window that is going to compose other + windows. In this stage it is unnecesssary but will be necessary in + future when we will have multiple windows and want to the switchbetween
python/qemu/aqmp/aqmp_tui.py new file +28
-0+ them and display overlays + """ + def __init__(self, master): + self.master = master + footer = StatusBar() + body = HistoryWindow(master) + super().__init__(body, footer=footer) + + +class TUILogHandler(Handler): + def __init__(self, tui): + super().__init__() + self.tui = tui + + def emit(self, record): + level = record.levelname + msg = record.getMessage() + msg = f'[{level}]: {msg}' + self.tui.add_to_history(msg) + + +def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
+ parser.add_argument('--asyncio-debug', action='store_true',
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py line 308
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusablewhen'
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ 'logs are logged in the TUI itself.' + 'Use only when logging to a file')Needs spaces between the lines here so that the output reads correctly.
The output renders properly for me. Can you please post a pic if it doesn't render correctly for you?
Quoted 27 lines
Quoted 20 lines
python/qemu/aqmp/aqmp_tui.py new file +16
-0+ args = parser.parse_args() + + try: + address = QEMUMonitorProtocol.parse_address(args.qmp_server) + except QMPBadPortError as err: + parser.error(err) + + app = App(address) + + if args.log_file: + LOGGER.addHandler(logging.FileHandler(args.log_file)) + else: + LOGGER.addHandler(TUILogHandler(app)) + + log_level = logging.getLevelName(args.log_level) + # getLevelName returns 'Level {log_level}' when a invalid level ispassed.
python/qemu/aqmp/aqmp_tui.py line 328
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py new file +4
-0+ if log_level == f'Level {args.log_level}': + parser.error('Invalid log level') + LOGGER.setLevel(log_level) +I think you could do:
Quoted 6 lines
LOGGER.setLevel(logging.getLevelName(args.log_level))
And it'll fail if the level was unknown, but work otherwise. This would avoid a kind of precarious string-comparison check, which I worry is fragile.
Fixed.
Quoted 35 lines
Quoted 35 lines
python/qemu/aqmp/aqmp_tui.py new file +5
-0+ app.run(args.asyncio_debug) + + +if __name__ == '__main__': + main() # type: ignorepython/setup.cfg +8
-0@@ -81,8 +81,22 @@ namespace_packages = True # fusepy has no type stubs: allow_subclassing_any = True +[mypy-qemu.aqmp.aqmp_tui] +disallow_untyped_defs = False +disallow_incomplete_defs = False +check_untyped_defs = False +# urwid and urwid_readline have no type stubs: +allow_subclassing_any = True + +# The following missing import directives are because these libraries donot
python/setup.cfg +7
-1+# provide type stubs. Allow them on an as-needed basis for mypy. [mypy-fuse] -# fusepy has no type stubs: +ignore_missing_imports = True + +[mypy-urwid] +ignore_missing_imports = True + +[mypy-urwid_readline] ignore_missing_imports = True
Quoted 8 lines
Quoted 5 lines
[pylint.messages control]
--
2.17.1I have some more thoughts, but need to research them and wanted to give some feedback first so you weren't waiting.
Thanks for the feedback.
Re: [PATCH v3 04/13] python/aqmp-tui: Add AQMP TUI draft
Quoted 19 lines · Niteesh G. S.
Quoted 16 lines · John Snow
Quoted 11 lines · G S Niteesh Babu
Added a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.Also added necessary pylint, mypy configurations
Signed-off-by G S Niteesh Babu <[email protected]>
...
Quoted 18 lines
+
Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py line 28
Quoted 3 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger()The comment isn't quite true; this is the root logger -- but you go on to use it to directly log messages. I don't think you should; use a module-level logger.
Quoted 9 lines
(1) Create a module-level logger that is named after the current module name (e.g. qemu.aqmp.aqmp_tui) and use that for logging messages relating to this module:
LOGGER =logging.getLogger(__name__)(2) Where you need to address the root logger, just use
root_logger = logging.getLogger().... I think themain()function is the only place you might need this.
Quoted 6 lines
The way of logging in the TUI is changed such that, all logging is done through the root level logger. The handlers and levels are installed to the root level
logger to allow logging from other libraries to be rerouted to the screen or file.
We talked about this on IRC, so I'll assume our disagreement in vision here is cleared up ... or at least different :)
Quoted 27 lines
+
Quoted 26 lines
Quoted 11 lines
python/qemu/aqmp/aqmp_tui.py new file +5
-0+ +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing whenlooking
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at morecomplex JSON
python/qemu/aqmp/aqmp_tui.py line 43
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) +You can use the JSON module to do this,
https://docs.python.org/3/library/json.html#json.dumpsMessage._serialize uses this technique to send JSON messages over the wire that have no newlines:
https://gitlab.com/jsnow/qemu/-/blob/python-async-qmp-aqmp/python/qemu/aqmp/message.py#L132
Quoted 9 lines
Quoted 4 lines
by not specifying an indent and including separators with no spaces, we can get a JSON representation with all the space squeezed out. You can add spaces and still omit the indent/newlines and so on.
But will this work for invalid JSON messages? As far as I have tried it doesn't work.
Ah, I see ... Nope, that requires a valid JSON message ... Do we *need* to pretty-print invalid JSON?
Quoted 17 lines
+
Quoted 16 lines
Quoted 15 lines
python/qemu/aqmp/aqmp_tui.py new file +7
-0+def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ parser.add_argument('--asyncio-debug', action='store_true', + help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusablewhen'
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ 'logs are logged in the TUI itself.' + 'Use only when logging to a file')
Quoted 5 lines
Quoted 2 lines
Needs spaces between the lines here so that the output reads correctly.
The output renders properly for me. Can you please post a pic if it doesn't render correctly for you?
No screenshot needed, just try running with '--help'. When you concatenate strings like this:
parser.add_argument('--asyncio-debug', action='store_true',
help='Enable debug mode for asyncio loop' 'Generates lot of output, makes TUI unusable when' 'logs are logged in the TUI itself.' 'Use only when logging to a file')
Python is going to do this:
help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusable when' + ...
so you'll get a string like this:
help='Enable debug mode for asyncio loopGenerates lot of output, makes TUI unusable when' + ...
Re: [PATCH v3 04/13] python/aqmp-tui: Add AQMP TUI draft
Quoted 52 lines · John Snow
Quoted 19 lines · Niteesh G. S.
Quoted 16 lines · John Snow
Quoted 11 lines · G S Niteesh Babu
Added a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.Also added necessary pylint, mypy configurations
Signed-off-by G S Niteesh Babu <[email protected]>...
Quoted 18 lines
+
Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py line 28
Quoted 3 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger()The comment isn't quite true; this is the root logger -- but you go on to use it to directly log messages. I don't think you should; use a module-level logger.
Quoted 9 lines
(1) Create a module-level logger that is named after the current module name (e.g. qemu.aqmp.aqmp_tui) and use that for logging messages relating to this module:
LOGGER =logging.getLogger(__name__)(2) Where you need to address the root logger, just use
root_logger = logging.getLogger().... I think themain()function is the only place you might need this.Quoted 6 lines
The way of logging in the TUI is changed such that, all logging is done through the root level logger. The handlers and levels are installed to the root level
logger to allow logging from other libraries to be rerouted to the screen or file.
Quoted 3 lines
We talked about this on IRC, so I'll assume our disagreement in vision here is cleared up ... or at least different :)
Maybe we'll come back to this on v4 :)
Quoted 35 lines
Quoted 34 lines
+
Quoted 30 lines
Quoted 11 lines
python/qemu/aqmp/aqmp_tui.py new file +5
-0+ +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing whenlooking
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at morecomplex JSON
python/qemu/aqmp/aqmp_tui.py line 43
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) +You can use the JSON module to do this,
https://docs.python.org/3/library/json.html#json.dumpsMessage._serialize uses this technique to send JSON messages over the wire that have no newlines:
https://gitlab.com/jsnow/qemu/-/blob/python-async-qmp-aqmp/python/qemu/aqmp/message.py#L132
by not specifying an indent and including separators with no spaces, we can get a JSON representation with all the space squeezed out. You can add spaces and still omit the indent/newlines and so on.
Quoted 7 lines
Quoted 3 lines
But will this work for invalid JSON messages? As far as I have tried it doesn't work.
Ah, I see ... Nope, that requires a valid JSON message ... Do we *need* to pretty-print invalid JSON?
IMHO Yes it looks pretty.
[1, true, 3]: QMP message is not a JSON object.
This one looks much better than the below one
[ 1,
true,
3 ]: QMP message is not a JSON object.
Quoted 34 lines
Quoted 18 lines
+
Quoted 16 lines
Quoted 15 lines
python/qemu/aqmp/aqmp_tui.py new file +7
-0+def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ parser.add_argument('--asyncio-debug', action='store_true', + help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusablewhen'
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ 'logs are logged in the TUI itself.' + 'Use only when logging to a file')Quoted 5 lines
Quoted 2 lines
Needs spaces between the lines here so that the output reads correctly.
The output renders properly for me. Can you please post a pic if it doesn't render correctly for you?
No screenshot needed, just try running with '--help'. When you concatenate strings like this:
parser.add_argument('--asyncio-debug', action='store_true',
help='Enable debug mode for asyncio loop' 'Generates lot of output, makes TUI unusable when' 'logs are logged in the TUI itself.' 'Use only when logging to a file')
Python is going to do this:
Quoted 8 lines
help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusable when' + ...
so you'll get a string like this:
help='Enable debug mode for asyncio loopGenerates lot of output, makes TUI unusable when' + ...
Thanks. Fixed.
Re: [PATCH v3 04/13] python/aqmp-tui: Add AQMP TUI draft
In reply to G S Niteesh BabuQuoted 255 lines · G S Niteesh Babu
Added a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.Also added necessary pylint, mypy configurations
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 333 +++++++++++++++++++++++++++++++++++ python/setup.cfg | 16 +- 2 files changed, 348 insertions(+), 1 deletion(-) create mode 100644 python/qemu/aqmp/aqmp_tui.pypython/qemu/aqmp/aqmp_tui.py new file +37
-0@@ -0,0 +1,333 @@ +# Copyright (c) 2021 +# +# Authors: +# Niteesh Babu G S <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import asyncio +import logging +from logging import Handler +import signal + +import urwid +import urwid_readline + +from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .message import DeserializationError, Message, UnexpectedTypeError +from .protocol import ConnectError +from .qmp_client import ExecInterruptedError, QMPClient +from .util import create_task, pretty_traceback + + +UPDATE_MSG = 'UPDATE_MSG' + +# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger() + + +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing when looking + along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at more complexJSON
python/qemu/aqmp/aqmp_tui.py new file +20
-0+ # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) + + +class App(QMPClient): + def __init__(self, address): + urwid.register_signal(type(self), UPDATE_MSG) + self.window = Window(self) + self.address = address + self.aloop = None + super().__init__() + + def add_to_history(self, msg): + urwid.emit_signal(self, UPDATE_MSG, msg) + + def _cb_outbound(self, msg): + # FIXME: I think the ideal way to omit these messages duringin-TUI
+ # logging will be to add a filter to the logger. We can use regex
to
+ # filter out messages starting with 'Request:' or 'Response:' but
I
+ # think a better approach will be encapsulate the message in an
object
+ # and filter based on the object. Encapsulation of the message
willpython/qemu/aqmp/aqmp_tui.py new file +22
-0+ # also be necessary when we want different formatting of messages + # inside TUI. + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Request: %s', str(msg)) + self.add_to_history('<-- ' + str(msg)) + return msg + + def _cb_inbound(self, msg): + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Response: %s', str(msg)) + self.add_to_history('--> ' + str(msg)) + return msg + + async def wait_for_events(self): + async for event in self.events: + self.handle_event(event) + + async def _send_to_server(self, raw_msg): + # FIXME: Format the raw_msg in history view to one line. It is not + # pleasing to see multiple lines JSON object with an errorstatement.
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ try: + msg = Message(bytes(raw_msg, encoding='utf-8')) + # Format multiline json into a single line JSON, since it ismore
python/qemu/aqmp/aqmp_tui.py new file +13
-0+ # pleasing to look along with err message in TUI. + raw_msg = self.format_json(raw_msg) + await self._raw(msg, assign_id='id' not in msg) + except (ValueError, TypeError) as err: + LOGGER.info('Invalid message: %s', str(err)) + self.add_to_history(f'{raw_msg}: {err}') + except (DeserializationError, UnexpectedTypeError) as err: + LOGGER.info('Invalid message: %s', err.error_message) + self.add_to_history(f'{raw_msg}: {err.error_message}') + except ExecInterruptedError: + LOGGER.info('Error server disconnected before reply') + urwid.emit_signal(self, UPDATE_MSG, + '{"error": "Server disconnected beforereply"}')
python/qemu/aqmp/aqmp_tui.py new file +62
-0+ self._set_status("Server disconnected") + except Exception as err: + LOGGER.error('Exception from _send_to_server: %s', str(err)) + raise err + + def cb_send_to_server(self, msg): + create_task(self._send_to_server(msg)) + + def unhandled_input(self, key): + if key == 'esc': + self.kill_app() + + def kill_app(self): + # TODO: Work on the disconnect logic + create_task(self._kill_app()) + + async def _kill_app(self): + # It is ok to call disconnect even in disconnect state + try: + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + except Exception as err: + LOGGER.info('_kill_app: %s', str(err)) + # Let the app crash after providing a proper stack trace + raise err + raise urwid.ExitMainLoop() + + def handle_event(self, event): + # FIXME: Consider all states present in qapi/run-state.json + if event['event'] == 'SHUTDOWN': + self._set_status('Server shutdown') + + def _set_status(self, msg: str) -> None: + self.window.footer.set_text(msg) + + def _get_formatted_address(self) -> str: + addr = f'{self.address}' + if isinstance(self.address, tuple): + host, port = self.address + addr = f'{host}:{port}' + return addr + + async def connect_server(self): + try: + await self.connect(self.address) + addr = self._get_formatted_address() + self._set_status(f'Connected to {addr}') + except ConnectError as err: + LOGGER.info('connect_server: ConnectError %s', str(err)) + self._set_status('Server shutdown') + + def run(self, debug=False): + self.aloop = asyncio.get_event_loop() + self.aloop.set_debug(debug) + + # Gracefully handle SIGTERM and SIGINT signals + cancel_signals = [signal.SIGTERM, signal.SIGINT] + for sig in cancel_signals: + self.aloop.add_signal_handler(sig, self.kill_app) + + event_loop = urwid.AsyncioEventLoop(loop=self.aloop) + main_loop = urwid.MainLoop(urwid.AttrMap(self.window,'background'),
python/qemu/aqmp/aqmp_tui.py new file +35
-0+ unhandled_input=self.unhandled_input, + handle_mouse=True, + event_loop=event_loop) + + create_task(self.wait_for_events(), self.aloop) + create_task(self.connect_server(), self.aloop) + try: + main_loop.run() + except Exception as err: + LOGGER.error('%s\n%s\n', str(err), pretty_traceback()) + raise err + + +class StatusBar(urwid.Text): + """ + A simple Text widget that currently only shows connection status. + """ + def __init__(self, text=''): + super().__init__(text, align='right') + + +class Editor(urwid_readline.ReadlineEdit): + """ + Support urwid_readline features along with + history support which lacks in urwid_readline + """ + def __init__(self, master): + super().__init__(caption='> ', multiline=True) + self.master = master + self.history = [] + self.last_index = -1 + self.show_history = False + + def keypress(self, size, key): + # TODO: Add some logic for down key and clean up logic ifpossible.
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # Returning None means the key has been handled by this widget + # which otherwise is propogated to the parent widget to be + # handled + msg = self.get_edit_text() + if key == 'up' and not msg: + # Show the history when 'up arrow' is pressed with no inputtext.
+ # NOTE: Theshow_historylogic is necessary because in
'multiline'
+ # mode (which we use) 'up arrow' is used to move between
lines.python/qemu/aqmp/aqmp_tui.py new file +2
-0+ self.show_history = True + last_msg = self.history[self.last_index] if self.history else''
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ self.set_edit_text(last_msg) + self.edit_pos = len(last_msg) + self.last_index += 1 + elif key == 'up' and self.show_history: + if self.last_index < len(self.history): + self.set_edit_text(self.history[self.last_index]) + self.edit_pos = len(self.history[self.last_index]) + self.last_index += 1 + elif key == 'meta enter': + # When using multiline, enter inserts a new line into theeditor
python/qemu/aqmp/aqmp_tui.py line 218
Quoted 2 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ # send the input to the server on alt + enter + self.master.cb_send_to_server(msg)
cb_send_to_server calls create_task(App._send_to_server(msg)) at which point in the coroutine you finally do msg = Message(bytes(raw_msg, encoding='utf-8')). However, you can do your message conversion *first*, here in the Editor.
That way, you can avoid erasing the editor bar when the message is garbled and the user has a chance to fix the corrupted message *first* before we dispatch to the callback-async-machine. That should reduce quite a lot of the error cases in _send_to_server and helps separate out the two kinds of errors we expect to see here:
1. The user typed something that's garbage, and we didn't actually even send it (because we couldn't), and
2. We sent (or tried to send) the message; but a failure occurred.
Quoted 152 lines
python/qemu/aqmp/aqmp_tui.py new file +47
-0+ self.history.insert(0, msg) + self.set_edit_text('') + self.last_index = 0 + self.show_history = False + else: + self.show_history = False + self.last_index = 0 + return super().keypress(size, key) + return None + + +class EditorWidget(urwid.Filler): + """ + Wraps CustomEdit + """ + def __init__(self, master): + super().__init__(Editor(master), valign='top') + + +class HistoryBox(urwid.ListBox): + """ + Shows all the QMP message transmitted/received + """ + def __init__(self, master): + self.master = master + self.history = urwid.SimpleFocusListWalker([]) + super().__init__(self.history) + + def add_to_history(self, history): + self.history.append(urwid.Text(history)) + if self.history: + self.history.set_focus(len(self.history) - 1) + + +class HistoryWindow(urwid.Frame): + """ + Composes the HistoryBox and EditorWidget + """ + def __init__(self, master): + self.master = master + self.editor_widget = EditorWidget(master) + self.editor = urwid.LineBox(self.editor_widget) + self.history = HistoryBox(master) + self.body = urwid.Pile([('weight', 80, self.history), + ('weight', 20, self.editor)]) + super().__init__(self.body) + urwid.connect_signal(self.master, UPDATE_MSG,self.cb_add_to_history)
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ + def cb_add_to_history(self, msg): + self.history.add_to_history(msg) + + +class Window(urwid.Frame): + """ + This is going to be the main window that is going to compose other + windows. In this stage it is unnecesssary but will be necessary in + future when we will have multiple windows and want to the switchbetween
python/qemu/aqmp/aqmp_tui.py new file +28
-0+ them and display overlays + """ + def __init__(self, master): + self.master = master + footer = StatusBar() + body = HistoryWindow(master) + super().__init__(body, footer=footer) + + +class TUILogHandler(Handler): + def __init__(self, tui): + super().__init__() + self.tui = tui + + def emit(self, record): + level = record.levelname + msg = record.getMessage() + msg = f'[{level}]: {msg}' + self.tui.add_to_history(msg) + + +def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
python/qemu/aqmp/aqmp_tui.py new file +21
-0+ parser.add_argument('--asyncio-debug', action='store_true', + help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusable when' + 'logs are logged in the TUI itself.' + 'Use only when logging to a file') + args = parser.parse_args() + + try: + address = QEMUMonitorProtocol.parse_address(args.qmp_server) + except QMPBadPortError as err: + parser.error(err) + + app = App(address) + + if args.log_file: + LOGGER.addHandler(logging.FileHandler(args.log_file)) + else: + LOGGER.addHandler(TUILogHandler(app)) + + log_level = logging.getLevelName(args.log_level) + # getLevelName returns 'Level {log_level}' when a invalid level ispassed.
python/qemu/aqmp/aqmp_tui.py new file +9
-0+ if log_level == f'Level {args.log_level}': + parser.error('Invalid log level') + LOGGER.setLevel(log_level) + + app.run(args.asyncio_debug) + + +if __name__ == '__main__': + main() # type: ignorepython/setup.cfg +8
-0@@ -81,8 +81,22 @@ namespace_packages = True # fusepy has no type stubs: allow_subclassing_any = True +[mypy-qemu.aqmp.aqmp_tui] +disallow_untyped_defs = False +disallow_incomplete_defs = False +check_untyped_defs = False +# urwid and urwid_readline have no type stubs: +allow_subclassing_any = True + +# The following missing import directives are because these libraries donot
python/setup.cfg +7
-2+# provide type stubs. Allow them on an as-needed basis for mypy. [mypy-fuse] -# fusepy has no type stubs: +ignore_missing_imports = True + +[mypy-urwid] +ignore_missing_imports = True + +[mypy-urwid_readline] ignore_missing_imports = True [pylint.messages control] --2.17.1
Re: [PATCH v3 04/13] python/aqmp-tui: Add AQMP TUI draft
Quoted 276 lines · John Snow
Quoted 257 lines · G S Niteesh Babu
Added a draft of AQMP TUI.
Implements the follwing basic features:
1) Command transmission/reception.
2) Shows events asynchronously.
3) Shows server status in the bottom status bar.Also added necessary pylint, mypy configurations
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 333 +++++++++++++++++++++++++++++++++++ python/setup.cfg | 16 +- 2 files changed, 348 insertions(+), 1 deletion(-) create mode 100644 python/qemu/aqmp/aqmp_tui.pypython/qemu/aqmp/aqmp_tui.py new file +34
-0@@ -0,0 +1,333 @@ +# Copyright (c) 2021 +# +# Authors: +# Niteesh Babu G S <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import asyncio +import logging +from logging import Handler +import signal + +import urwid +import urwid_readline + +from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .message import DeserializationError, Message, UnexpectedTypeError +from .protocol import ConnectError +from .qmp_client import ExecInterruptedError, QMPClient +from .util import create_task, pretty_traceback + + +UPDATE_MSG = 'UPDATE_MSG' + +# Using root logger to enable all loggers under qemu and asyncio +LOGGER = logging.getLogger() + + +def format_json(msg): + """ + Formats given multiline JSON message into a single line message. + Converting into single line is more asthetically pleasing whenlooking
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ along with error messages compared to multiline JSON. + """ + # FIXME: Use better formatting mechanism. Might break at morecomplex JSON
python/qemu/aqmp/aqmp_tui.py new file +20
-0+ # data. + msg = msg.replace('\n', '') + words = msg.split(' ') + words = [word for word in words if word != ''] + return ' '.join(words) + + +class App(QMPClient): + def __init__(self, address): + urwid.register_signal(type(self), UPDATE_MSG) + self.window = Window(self) + self.address = address + self.aloop = None + super().__init__() + + def add_to_history(self, msg): + urwid.emit_signal(self, UPDATE_MSG, msg) + + def _cb_outbound(self, msg): + # FIXME: I think the ideal way to omit these messages duringin-TUI
+ # logging will be to add a filter to the logger. We can use
regex to
+ # filter out messages starting with 'Request:' or 'Response:'
but I
+ # think a better approach will be encapsulate the message in an
object
+ # and filter based on the object. Encapsulation of the message
willpython/qemu/aqmp/aqmp_tui.py new file +21
-0+ # also be necessary when we want different formatting of messages + # inside TUI. + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Request: %s', str(msg)) + self.add_to_history('<-- ' + str(msg)) + return msg + + def _cb_inbound(self, msg): + handler = LOGGER.handlers[0] + if not isinstance(handler, TUILogHandler): + LOGGER.debug('Response: %s', str(msg)) + self.add_to_history('--> ' + str(msg)) + return msg + + async def wait_for_events(self): + async for event in self.events: + self.handle_event(event) + + async def _send_to_server(self, raw_msg): + # FIXME: Format the raw_msg in history view to one line. It isnot
+ # pleasing to see multiple lines JSON object with an error
statement.python/qemu/aqmp/aqmp_tui.py new file +3
-0+ try: + msg = Message(bytes(raw_msg, encoding='utf-8')) + # Format multiline json into a single line JSON, since it ismore
python/qemu/aqmp/aqmp_tui.py new file +13
-0+ # pleasing to look along with err message in TUI. + raw_msg = self.format_json(raw_msg) + await self._raw(msg, assign_id='id' not in msg) + except (ValueError, TypeError) as err: + LOGGER.info('Invalid message: %s', str(err)) + self.add_to_history(f'{raw_msg}: {err}') + except (DeserializationError, UnexpectedTypeError) as err: + LOGGER.info('Invalid message: %s', err.error_message) + self.add_to_history(f'{raw_msg}: {err.error_message}') + except ExecInterruptedError: + LOGGER.info('Error server disconnected before reply') + urwid.emit_signal(self, UPDATE_MSG, + '{"error": "Server disconnected beforereply"}')
python/qemu/aqmp/aqmp_tui.py new file +62
-0+ self._set_status("Server disconnected") + except Exception as err: + LOGGER.error('Exception from _send_to_server: %s', str(err)) + raise err + + def cb_send_to_server(self, msg): + create_task(self._send_to_server(msg)) + + def unhandled_input(self, key): + if key == 'esc': + self.kill_app() + + def kill_app(self): + # TODO: Work on the disconnect logic + create_task(self._kill_app()) + + async def _kill_app(self): + # It is ok to call disconnect even in disconnect state + try: + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + except Exception as err: + LOGGER.info('_kill_app: %s', str(err)) + # Let the app crash after providing a proper stack trace + raise err + raise urwid.ExitMainLoop() + + def handle_event(self, event): + # FIXME: Consider all states present in qapi/run-state.json + if event['event'] == 'SHUTDOWN': + self._set_status('Server shutdown') + + def _set_status(self, msg: str) -> None: + self.window.footer.set_text(msg) + + def _get_formatted_address(self) -> str: + addr = f'{self.address}' + if isinstance(self.address, tuple): + host, port = self.address + addr = f'{host}:{port}' + return addr + + async def connect_server(self): + try: + await self.connect(self.address) + addr = self._get_formatted_address() + self._set_status(f'Connected to {addr}') + except ConnectError as err: + LOGGER.info('connect_server: ConnectError %s', str(err)) + self._set_status('Server shutdown') + + def run(self, debug=False): + self.aloop = asyncio.get_event_loop() + self.aloop.set_debug(debug) + + # Gracefully handle SIGTERM and SIGINT signals + cancel_signals = [signal.SIGTERM, signal.SIGINT] + for sig in cancel_signals: + self.aloop.add_signal_handler(sig, self.kill_app) + + event_loop = urwid.AsyncioEventLoop(loop=self.aloop) + main_loop = urwid.MainLoop(urwid.AttrMap(self.window,'background'),
python/qemu/aqmp/aqmp_tui.py new file +35
-0+ unhandled_input=self.unhandled_input, + handle_mouse=True, + event_loop=event_loop) + + create_task(self.wait_for_events(), self.aloop) + create_task(self.connect_server(), self.aloop) + try: + main_loop.run() + except Exception as err: + LOGGER.error('%s\n%s\n', str(err), pretty_traceback()) + raise err + + +class StatusBar(urwid.Text): + """ + A simple Text widget that currently only shows connection status. + """ + def __init__(self, text=''): + super().__init__(text, align='right') + + +class Editor(urwid_readline.ReadlineEdit): + """ + Support urwid_readline features along with + history support which lacks in urwid_readline + """ + def __init__(self, master): + super().__init__(caption='> ', multiline=True) + self.master = master + self.history = [] + self.last_index = -1 + self.show_history = False + + def keypress(self, size, key): + # TODO: Add some logic for down key and clean up logic ifpossible.
python/qemu/aqmp/aqmp_tui.py new file +6
-0+ # Returning None means the key has been handled by this widget + # which otherwise is propogated to the parent widget to be + # handled + msg = self.get_edit_text() + if key == 'up' and not msg: + # Show the history when 'up arrow' is pressed with no inputtext.
+ # NOTE: Theshow_historylogic is necessary because in
'multiline'
+ # mode (which we use) 'up arrow' is used to move between
lines.python/qemu/aqmp/aqmp_tui.py new file +2
-0+ self.show_history = True + last_msg = self.history[self.last_index] if self.historyelse ''
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ self.set_edit_text(last_msg) + self.edit_pos = len(last_msg) + self.last_index += 1 + elif key == 'up' and self.show_history: + if self.last_index < len(self.history): + self.set_edit_text(self.history[self.last_index]) + self.edit_pos = len(self.history[self.last_index]) + self.last_index += 1 + elif key == 'meta enter': + # When using multiline, enter inserts a new line into theeditor
python/qemu/aqmp/aqmp_tui.py line 218
Quoted 2 lines
python/qemu/aqmp/aqmp_tui.py new file +2
-0+ # send the input to the server on alt + enter + self.master.cb_send_to_server(msg)
cb_send_to_servercalls create_task(App._send_to_server(msg)) at which point in the coroutine you finally do msg =Message(bytes(raw_msg, encoding='utf-8')). However, you can do your message conversion *first*, here in the Editor.That way, you can avoid erasing the editor bar when the message is garbled and the user has a chance to fix the corrupted message *first* before we dispatch to the callback-async-machine. That should reduce quite a lot of the error cases in
_send_to_serverand helps separate out the two kinds of errors we expect to see here:
Quoted 4 lines
1. The user typed something that's garbage, and we didn't actually even send it (because we couldn't), and
2. We sent (or tried to send) the message; but a failure occurred.
Thanks. Refactored this.
Quoted 155 lines
Quoted 155 lines
python/qemu/aqmp/aqmp_tui.py new file +47
-0+ self.history.insert(0, msg) + self.set_edit_text('') + self.last_index = 0 + self.show_history = False + else: + self.show_history = False + self.last_index = 0 + return super().keypress(size, key) + return None + + +class EditorWidget(urwid.Filler): + """ + Wraps CustomEdit + """ + def __init__(self, master): + super().__init__(Editor(master), valign='top') + + +class HistoryBox(urwid.ListBox): + """ + Shows all the QMP message transmitted/received + """ + def __init__(self, master): + self.master = master + self.history = urwid.SimpleFocusListWalker([]) + super().__init__(self.history) + + def add_to_history(self, history): + self.history.append(urwid.Text(history)) + if self.history: + self.history.set_focus(len(self.history) - 1) + + +class HistoryWindow(urwid.Frame): + """ + Composes the HistoryBox and EditorWidget + """ + def __init__(self, master): + self.master = master + self.editor_widget = EditorWidget(master) + self.editor = urwid.LineBox(self.editor_widget) + self.history = HistoryBox(master) + self.body = urwid.Pile([('weight', 80, self.history), + ('weight', 20, self.editor)]) + super().__init__(self.body) + urwid.connect_signal(self.master, UPDATE_MSG,self.cb_add_to_history)
python/qemu/aqmp/aqmp_tui.py new file +10
-0+ + def cb_add_to_history(self, msg): + self.history.add_to_history(msg) + + +class Window(urwid.Frame): + """ + This is going to be the main window that is going to compose other + windows. In this stage it is unnecesssary but will be necessary in + future when we will have multiple windows and want to the switchbetween
python/qemu/aqmp/aqmp_tui.py new file +28
-0+ them and display overlays + """ + def __init__(self, master): + self.master = master + footer = StatusBar() + body = HistoryWindow(master) + super().__init__(body, footer=footer) + + +class TUILogHandler(Handler): + def __init__(self, tui): + super().__init__() + self.tui = tui + + def emit(self, record): + level = record.levelname + msg = record.getMessage() + msg = f'[{level}]: {msg}' + self.tui.add_to_history(msg) + + +def main(): + parser = argparse.ArgumentParser(description='AQMP TUI') + parser.add_argument('qmp_server', help='Address of the QMP server' + '< UNIX socket path | TCP addr:port >') + parser.add_argument('--log-file', help='The Log file name') + parser.add_argument('--log-level', default='WARNING', + help='Log level<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
python/qemu/aqmp/aqmp_tui.py new file +3
-0+ parser.add_argument('--asyncio-debug', action='store_true', + help='Enable debug mode for asyncio loop' + 'Generates lot of output, makes TUI unusablewhen'
python/qemu/aqmp/aqmp_tui.py new file +18
-0+ 'logs are logged in the TUI itself.' + 'Use only when logging to a file') + args = parser.parse_args() + + try: + address = QEMUMonitorProtocol.parse_address(args.qmp_server) + except QMPBadPortError as err: + parser.error(err) + + app = App(address) + + if args.log_file: + LOGGER.addHandler(logging.FileHandler(args.log_file)) + else: + LOGGER.addHandler(TUILogHandler(app)) + + log_level = logging.getLevelName(args.log_level) + # getLevelName returns 'Level {log_level}' when a invalid level ispassed.
python/qemu/aqmp/aqmp_tui.py new file +9
-0+ if log_level == f'Level {args.log_level}': + parser.error('Invalid log level') + LOGGER.setLevel(log_level) + + app.run(args.asyncio_debug) + + +if __name__ == '__main__': + main() # type: ignorepython/setup.cfg +8
-0@@ -81,8 +81,22 @@ namespace_packages = True # fusepy has no type stubs: allow_subclassing_any = True +[mypy-qemu.aqmp.aqmp_tui] +disallow_untyped_defs = False +disallow_incomplete_defs = False +check_untyped_defs = False +# urwid and urwid_readline have no type stubs: +allow_subclassing_any = True + +# The following missing import directives are because these libraries donot
python/setup.cfg +7
-2+# provide type stubs. Allow them on an as-needed basis for mypy. [mypy-fuse] -# fusepy has no type stubs: +ignore_missing_imports = True + +[mypy-urwid] +ignore_missing_imports = True + +[mypy-urwid_readline] ignore_missing_imports = True [pylint.messages control] --2.17.1
PATCH v3 5/13 python: add entry point for aqmp-tui
In reply to G S Niteesh BabuAdd an entry point for aqmp-tui. This will allow it to be run from the command line using "aqmp-tui localhost:1234"
More options available in the TUI can be found using "aqmp-tui -h"
--- python/setup.cfg | 1 + 1 file changed, 1 insertion(+)
python/setup.cfg +1 -0
@@ -66,6 +66,7 @@ console_scripts = qom-fuse = qemu.qmp.qom_fuse:QOMFuse.entry_point [fuse] qemu-ga-client = qemu.qmp.qemu_ga_client:main qmp-shell = qemu.qmp.qmp_shell:main + aqmp-tui = qemu.aqmp.aqmp_tui:main [tui] [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's
Signature · 1 line
2.17.1
Re: [PATCH v3 05/13] python: add entry point for aqmp-tui
Quoted 6 lines · G S Niteesh Babu
Add an entry point for aqmp-tui. This will allow it to be run from the command line using "aqmp-tui localhost:1234"
More options available in the TUI can be found using "aqmp-tui -h"Signed-off-by G S Niteesh Babu <[email protected]>
PATCH v3 6/13 python/aqmp-tui: Added type annotations for aqmp-tui
In reply to G S Niteesh BabuThis patch adds type annotations for aqmp-tui using the mypy library.
--- python/qemu/aqmp/aqmp_tui.py | 79 ++++++++++++++++++++---------------- python/setup.cfg | 3 -- 2 files changed, 43 insertions(+), 39 deletions(-)
python/qemu/aqmp/aqmp_tui.py +43 -36
@@ -9,8 +9,15 @@ import argparse import asyncio import logging -from logging import Handler +from logging import Handler, LogRecord import signal +from typing import ( + Any, + List, + Optional, + Tuple, + Union, +) import urwid import urwid_readline
@@ -22,13 +29,13 @@ from .util import create_task, pretty_traceback -UPDATE_MSG = 'UPDATE_MSG' +UPDATE_MSG: str = 'UPDATE_MSG' # Using root logger to enable all loggers under qemu and asyncio LOGGER = logging.getLogger() -def format_json(msg): +def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message. Converting into single line is more asthetically pleasing when looking
@@ -43,17 +50,17 @@ def format_json(msg): class App(QMPClient): - def __init__(self, address): + def __init__(self, address: Union[str, Tuple[str, int]]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self) self.address = address - self.aloop = None + self.aloop: Optional[Any] = None # FIXME: Use more concrete type. super().__init__() - def add_to_history(self, msg): + def add_to_history(self, msg: str) -> None: urwid.emit_signal(self, UPDATE_MSG, msg) - def _cb_outbound(self, msg): + def _cb_outbound(self, msg: Message) -> Message: # FIXME: I think the ideal way to omit these messages during in-TUI # logging will be to add a filter to the logger. We can use regex to # filter out messages starting with 'Request:' or 'Response:' but I
@@ -67,25 +74,25 @@ def _cb_outbound(self, msg): self.add_to_history('<-- ' + str(msg)) return msg - def _cb_inbound(self, msg): + def _cb_inbound(self, msg: Message) -> Message: handler = LOGGER.handlers[0] if not isinstance(handler, TUILogHandler): LOGGER.debug('Response: %s', str(msg)) self.add_to_history('--> ' + str(msg)) return msg - async def wait_for_events(self): + async def wait_for_events(self) -> None: async for event in self.events: self.handle_event(event) - async def _send_to_server(self, raw_msg): + async def _send_to_server(self, raw_msg: str) -> None: # FIXME: Format the raw_msg in history view to one line. It is not # pleasing to see multiple lines JSON object with an error statement. try: msg = Message(bytes(raw_msg, encoding='utf-8')) # Format multiline json into a single line JSON, since it is more # pleasing to look along with err message in TUI. - raw_msg = self.format_json(raw_msg) + raw_msg = format_json(raw_msg) await self._raw(msg, assign_id='id' not in msg) except (ValueError, TypeError) as err: LOGGER.info('Invalid message: %s', str(err))
@@ -102,18 +109,18 @@ def _cb_inbound(self, msg): LOGGER.error('Exception from _send_to_server: %s', str(err)) raise err - def cb_send_to_server(self, msg): + def cb_send_to_server(self, msg: str) -> None: create_task(self._send_to_server(msg)) - def unhandled_input(self, key): + def unhandled_input(self, key: str) -> None: if key == 'esc': self.kill_app() - def kill_app(self): + def kill_app(self) -> None: # TODO: Work on the disconnect logic create_task(self._kill_app()) - async def _kill_app(self): + async def _kill_app(self) -> None: # It is ok to call disconnect even in disconnect state try: await self.disconnect()
@@ -124,7 +131,7 @@ def kill_app(self): raise err raise urwid.ExitMainLoop() - def handle_event(self, event): + def handle_event(self, event: Message) -> None: # FIXME: Consider all states present in qapi/run-state.json if event['event'] == 'SHUTDOWN': self._set_status('Server shutdown')
@@ -139,7 +146,7 @@ def _get_formatted_address(self) -> str: addr = f'{host}:{port}' return addr - async def connect_server(self): + async def connect_server(self) -> None: try: await self.connect(self.address) addr = self._get_formatted_address()
@@ -148,7 +155,7 @@ def _get_formatted_address(self) -> str: LOGGER.info('connect_server: ConnectError %s', str(err)) self._set_status('Server shutdown') - def run(self, debug=False): + def run(self, debug: bool = False) -> None: self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)
@@ -176,7 +183,7 @@ class StatusBar(urwid.Text): """ A simple Text widget that currently only shows connection status. """ - def __init__(self, text=''): + def __init__(self, text: str = ''): super().__init__(text, align='right')
@@ -185,14 +192,14 @@ class Editor(urwid_readline.ReadlineEdit): Support urwid_readline features along with history support which lacks in urwid_readline """ - def __init__(self, master): + def __init__(self, master: App) -> None: super().__init__(caption='> ', multiline=True) self.master = master - self.history = [] - self.last_index = -1 - self.show_history = False + self.history: List[str] = [] + self.last_index: int = -1 + self.show_history: bool = False - def keypress(self, size, key): + def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: # TODO: Add some logic for down key and clean up logic if possible. # Returning None means the key has been handled by this widget # which otherwise is propogated to the parent widget to be
@@ -223,7 +230,7 @@ def keypress(self, size, key): else: self.show_history = False self.last_index = 0 - return super().keypress(size, key) + return super().keypress(size, key) # type: ignore return None
@@ -231,7 +238,7 @@ class EditorWidget(urwid.Filler): """ Wraps CustomEdit """ - def __init__(self, master): + def __init__(self, master: App) -> None: super().__init__(Editor(master), valign='top')
@@ -239,12 +246,12 @@ class HistoryBox(urwid.ListBox): """ Shows all the QMP message transmitted/received """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history): + def add_to_history(self, history: str) -> None: self.history.append(urwid.Text(history)) if self.history: self.history.set_focus(len(self.history) - 1)
@@ -254,7 +261,7 @@ class HistoryWindow(urwid.Frame): """ Composes the HistoryBox and EditorWidget """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master self.editor_widget = EditorWidget(master) self.editor = urwid.LineBox(self.editor_widget)
@@ -264,7 +271,7 @@ def __init__(self, master): super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg): + def cb_add_to_history(self, msg: str) -> None: self.history.add_to_history(msg)
@@ -275,7 +282,7 @@ class Window(urwid.Frame): future when we will have multiple windows and want to the switch between them and display overlays """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master footer = StatusBar() body = HistoryWindow(master)
@@ -283,18 +290,18 @@ def __init__(self, master): class TUILogHandler(Handler): - def __init__(self, tui): + def __init__(self, tui: App) -> None: super().__init__() self.tui = tui - def emit(self, record): + def emit(self, record: LogRecord) -> None: level = record.levelname msg = record.getMessage() msg = f'[{level}]: {msg}' self.tui.add_to_history(msg) -def main(): +def main() -> None: parser = argparse.ArgumentParser(description='AQMP TUI') parser.add_argument('qmp_server', help='Address of the QMP server' '< UNIX socket path | TCP addr:port >')
@@ -311,7 +318,7 @@ def main(): try: address = QEMUMonitorProtocol.parse_address(args.qmp_server) except QMPBadPortError as err: - parser.error(err) + parser.error(str(err)) app = App(address)
@@ -330,4 +337,4 @@ def main(): if __name__ == '__main__': - main() # type: ignore + main()
python/setup.cfg +0 -3
@@ -83,9 +83,6 @@ namespace_packages = True allow_subclassing_any = True [mypy-qemu.aqmp.aqmp_tui] -disallow_untyped_defs = False -disallow_incomplete_defs = False -check_untyped_defs = False # urwid and urwid_readline have no type stubs: allow_subclassing_any = True
Signature · 1 line
2.17.1
Re: [PATCH v3 06/13] python/aqmp-tui: Added type annotations for aqmp-tui
Quoted 4 lines · G S Niteesh Babu
This patch adds type annotations for aqmp-tui using the mypy library.
Awesome, thanks for taking a swing at this. Looks like it wasn't as bad as I was fearing.
Quoted 45 lines
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 79 ++++++++++++++++++++---------------- python/setup.cfg | 3 -- 2 files changed, 43 insertions(+), 39 deletions(-)python/qemu/aqmp/aqmp_tui.py +10
-3@@ -9,8 +9,15 @@ import argparse import asyncio import logging -from logging import Handler +from logging import Handler, LogRecord import signal +from typing import ( + Any, + List, + Optional, + Tuple, + Union, +) import urwid import urwid_readline@@ -22,13 +29,13 @@ from .util import create_task, pretty_traceback -UPDATE_MSG = 'UPDATE_MSG' +UPDATE_MSG: str = 'UPDATE_MSG' # Using root logger to enable all loggers under qemu and asyncio LOGGER = logging.getLogger() -def format_json(msg): +def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message. Converting into single line is more asthetically pleasing when looking@@ -43,17 +50,17 @@ def format_json(msg):
python/qemu/aqmp/aqmp_tui.py def format_json(msg): · line 58
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py +2
-2class App(QMPClient): - def __init__(self, address): + def __init__(self, address: Union[str, Tuple[str, int]]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self) self.address = address - self.aloop = None + self.aloop: Optional[Any] = None # FIXME: Use more concrete type.
I ran into this in util.py; you want Optional[asyncio.AbstractEventLoop] here.
Quoted 127 lines
python/qemu/aqmp/aqmp_tui.py +2
-2super().__init__() - def add_to_history(self, msg): + def add_to_history(self, msg: str) -> None: urwid.emit_signal(self, UPDATE_MSG, msg) - def _cb_outbound(self, msg): + def _cb_outbound(self, msg: Message) -> Message: # FIXME: I think the ideal way to omit these messages duringin-TUI
# logging will be to add a filter to the logger. We can use regex to # filter out messages starting with 'Request:' or 'Response:' but Ipython/qemu/aqmp/aqmp_tui.py +3
-3@@ -67,25 +74,25 @@ def _cb_outbound(self, msg): self.add_to_history('<-- ' + str(msg)) return msg - def _cb_inbound(self, msg): + def _cb_inbound(self, msg: Message) -> Message: handler = LOGGER.handlers[0] if not isinstance(handler, TUILogHandler): LOGGER.debug('Response: %s', str(msg)) self.add_to_history('--> ' + str(msg)) return msg - async def wait_for_events(self): + async def wait_for_events(self) -> None: async for event in self.events: self.handle_event(event) - async def _send_to_server(self, raw_msg): + async def _send_to_server(self, raw_msg: str) -> None: # FIXME: Format the raw_msg in history view to one line. It is not # pleasing to see multiple lines JSON object with an errorstatement.
try:
msg =Message(bytes(raw_msg, encoding='utf-8'))# Format multiline json into a single line JSON, since it is
more# pleasing to look along with err message in TUI.python/qemu/aqmp/aqmp_tui.py +14
-14- raw_msg = self.format_json(raw_msg) + raw_msg = format_json(raw_msg) await self._raw(msg, assign_id='id' not in msg) except (ValueError, TypeError) as err: LOGGER.info('Invalid message: %s', str(err))@@ -102,18 +109,18 @@ def _cb_inbound(self, msg): LOGGER.error('Exception from _send_to_server: %s', str(err)) raise err - def cb_send_to_server(self, msg): + def cb_send_to_server(self, msg: str) -> None: create_task(self._send_to_server(msg)) - def unhandled_input(self, key): + def unhandled_input(self, key: str) -> None: if key == 'esc': self.kill_app() - def kill_app(self): + def kill_app(self) -> None: # TODO: Work on the disconnect logic create_task(self._kill_app()) - async def _kill_app(self): + async def _kill_app(self) -> None: # It is ok to call disconnect even in disconnect state try: await self.disconnect()@@ -124,7 +131,7 @@ def kill_app(self): raise err raise urwid.ExitMainLoop() - def handle_event(self, event): + def handle_event(self, event: Message) -> None: # FIXME: Consider all states present in qapi/run-state.json if event['event'] == 'SHUTDOWN': self._set_status('Server shutdown')@@ -139,7 +146,7 @@ def _get_formatted_address(self) -> str: addr = f'{host}:{port}' return addr - async def connect_server(self): + async def connect_server(self) -> None: try: await self.connect(self.address) addr = self._get_formatted_address()@@ -148,7 +155,7 @@ def _get_formatted_address(self) -> str: LOGGER.info('connect_server: ConnectError %s', str(err)) self._set_status('Server shutdown') - def run(self, debug=False): + def run(self, debug: bool = False) -> None: self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)@@ -176,7 +183,7 @@ class StatusBar(urwid.Text): """ A simple Text widget that currently only shows connection status. """ - def __init__(self, text=''): + def __init__(self, text: str = ''): super().__init__(text, align='right')@@ -185,14 +192,14 @@ class Editor(urwid_readline.ReadlineEdit): Support urwid_readline features along with history support which lacks in urwid_readline """ - def __init__(self, master): + def __init__(self, master: App) -> None: super().__init__(caption='> ', multiline=True) self.master = master - self.history = [] - self.last_index = -1 - self.show_history = False + self.history: List[str] = [] + self.last_index: int = -1 + self.show_history: bool = False - def keypress(self, size, key): + def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: # TODO: Add some logic for down key and clean up logic ifpossible.
# Returning None means the key has been handled by this widget # which otherwise is propogated to the parent widget to be
python/qemu/aqmp/aqmp_tui.py def keypress(self, size, key): · line 234
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py +1
-1@@ -223,7 +230,7 @@ def keypress(self, size, key): else: self.show_history = False self.last_index = 0 - return super().keypress(size, key) + return super().keypress(size, key) # type: ignore
try using cast(Optional[str], super().keypress(size, key)) instead of the
type: ignore. It doesn't make a gigantic difference, really, but it looks "narrower" in scope and will probably confuse *me* less in the future.
Quoted 108 lines
python/qemu/aqmp/aqmp_tui.py +5
-5return None@@ -231,7 +238,7 @@ class EditorWidget(urwid.Filler): """ Wraps CustomEdit """ - def __init__(self, master): + def __init__(self, master: App) -> None: super().__init__(Editor(master), valign='top')@@ -239,12 +246,12 @@ class HistoryBox(urwid.ListBox): """ Shows all the QMP message transmitted/received """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history): + def add_to_history(self, history: str) -> None: self.history.append(urwid.Text(history)) if self.history: self.history.set_focus(len(self.history) - 1)@@ -254,7 +261,7 @@ class HistoryWindow(urwid.Frame): """ Composes the HistoryBox and EditorWidget """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master self.editor_widget = EditorWidget(master) self.editor = urwid.LineBox(self.editor_widget)@@ -264,7 +271,7 @@ def __init__(self, master): super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg): + def cb_add_to_history(self, msg: str) -> None: self.history.add_to_history(msg)@@ -275,7 +282,7 @@ class Window(urwid.Frame): future when we will have multiple windows and want to the switchbetween
them and display overlays
"""python/qemu/aqmp/aqmp_tui.py +6
-6- def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master footer = StatusBar() body = HistoryWindow(master)@@ -283,18 +290,18 @@ def __init__(self, master): class TUILogHandler(Handler): - def __init__(self, tui): + def __init__(self, tui: App) -> None: super().__init__() self.tui = tui - def emit(self, record): + def emit(self, record: LogRecord) -> None: level = record.levelname msg = record.getMessage() msg = f'[{level}]: {msg}' self.tui.add_to_history(msg) -def main(): +def main() -> None: parser = argparse.ArgumentParser(description='AQMP TUI') parser.add_argument('qmp_server', help='Address of the QMP server' '< UNIX socket path | TCP addr:port >')@@ -311,7 +318,7 @@ def main(): try: address = QEMUMonitorProtocol.parse_address(args.qmp_server) except QMPBadPortError as err: - parser.error(err) + parser.error(str(err)) app = App(address)@@ -330,4 +337,4 @@ def main(): if __name__ == '__main__': - main() # type: ignore + main()python/setup.cfg +0
-4@@ -83,9 +83,6 @@ namespace_packages = True allow_subclassing_any = True [mypy-qemu.aqmp.aqmp_tui] -disallow_untyped_defs = False -disallow_incomplete_defs = False -check_untyped_defs = False # urwid and urwid_readline have no type stubs: allow_subclassing_any = True --
Quoted 3 lines
2.17.1
Please squash this into the first patch, but it looks good, thanks!
--js
Re: [PATCH v3 06/13] python/aqmp-tui: Added type annotations for aqmp-tui
Quoted 60 lines · John Snow
Quoted 4 lines · G S Niteesh Babu
This patch adds type annotations for aqmp-tui using the mypy library.
Awesome, thanks for taking a swing at this. Looks like it wasn't as bad as I was fearing.
Quoted 47 lines
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 79 ++++++++++++++++++++---------------- python/setup.cfg | 3 -- 2 files changed, 43 insertions(+), 39 deletions(-)python/qemu/aqmp/aqmp_tui.py +10
-3@@ -9,8 +9,15 @@ import argparse import asyncio import logging -from logging import Handler +from logging import Handler, LogRecord import signal +from typing import ( + Any, + List, + Optional, + Tuple, + Union, +) import urwid import urwid_readline@@ -22,13 +29,13 @@ from .util import create_task, pretty_traceback -UPDATE_MSG = 'UPDATE_MSG' +UPDATE_MSG: str = 'UPDATE_MSG' # Using root logger to enable all loggers under qemu and asyncio LOGGER = logging.getLogger() -def format_json(msg): +def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message. Converting into single line is more asthetically pleasing whenlooking
python/qemu/aqmp/aqmp_tui.py +0
-0@@ -43,17 +50,17 @@ def format_json(msg):
Quoted 14 lines
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py +2
-2class App(QMPClient): - def __init__(self, address): + def __init__(self, address: Union[str, Tuple[str, int]]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self) self.address = address - self.aloop = None + self.aloop: Optional[Any] = None # FIXME: Use more concretetype.
I ran into this in
util.py; you want Optional[asyncio.AbstractEventLoop] here.
Thanks. Fixed.
Quoted 132 lines
Quoted 132 lines
super().__init__()python/qemu/aqmp/aqmp_tui.py +2
-2- def add_to_history(self, msg): + def add_to_history(self, msg: str) -> None: urwid.emit_signal(self, UPDATE_MSG, msg) - def _cb_outbound(self, msg): + def _cb_outbound(self, msg: Message) -> Message: # FIXME: I think the ideal way to omit these messages duringin-TUI
# logging will be to add a filter to the logger. We can use regex to # filter out messages starting with 'Request:' or 'Response:' but Ipython/qemu/aqmp/aqmp_tui.py +3
-3@@ -67,25 +74,25 @@ def _cb_outbound(self, msg): self.add_to_history('<-- ' + str(msg)) return msg - def _cb_inbound(self, msg): + def _cb_inbound(self, msg: Message) -> Message: handler = LOGGER.handlers[0] if not isinstance(handler, TUILogHandler): LOGGER.debug('Response: %s', str(msg)) self.add_to_history('--> ' + str(msg)) return msg - async def wait_for_events(self): + async def wait_for_events(self) -> None: async for event in self.events: self.handle_event(event) - async def _send_to_server(self, raw_msg): + async def _send_to_server(self, raw_msg: str) -> None: # FIXME: Format the raw_msg in history view to one line. It isnot
# pleasing to see multiple lines JSON object with an error statement. try: msg = Message(bytes(raw_msg, encoding='utf-8')) # Format multiline json into a single line JSON, since it is more # pleasing to look along with err message in TUI.python/qemu/aqmp/aqmp_tui.py +14
-14- raw_msg = self.format_json(raw_msg) + raw_msg = format_json(raw_msg) await self._raw(msg, assign_id='id' not in msg) except (ValueError, TypeError) as err: LOGGER.info('Invalid message: %s', str(err))@@ -102,18 +109,18 @@ def _cb_inbound(self, msg): LOGGER.error('Exception from _send_to_server: %s', str(err)) raise err - def cb_send_to_server(self, msg): + def cb_send_to_server(self, msg: str) -> None: create_task(self._send_to_server(msg)) - def unhandled_input(self, key): + def unhandled_input(self, key: str) -> None: if key == 'esc': self.kill_app() - def kill_app(self): + def kill_app(self) -> None: # TODO: Work on the disconnect logic create_task(self._kill_app()) - async def _kill_app(self): + async def _kill_app(self) -> None: # It is ok to call disconnect even in disconnect state try: await self.disconnect()@@ -124,7 +131,7 @@ def kill_app(self): raise err raise urwid.ExitMainLoop() - def handle_event(self, event): + def handle_event(self, event: Message) -> None: # FIXME: Consider all states present in qapi/run-state.json if event['event'] == 'SHUTDOWN': self._set_status('Server shutdown')@@ -139,7 +146,7 @@ def _get_formatted_address(self) -> str: addr = f'{host}:{port}' return addr - async def connect_server(self): + async def connect_server(self) -> None: try: await self.connect(self.address) addr = self._get_formatted_address()@@ -148,7 +155,7 @@ def _get_formatted_address(self) -> str: LOGGER.info('connect_server: ConnectError %s', str(err)) self._set_status('Server shutdown') - def run(self, debug=False): + def run(self, debug: bool = False) -> None: self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)@@ -176,7 +183,7 @@ class StatusBar(urwid.Text): """ A simple Text widget that currently only shows connection status. """ - def __init__(self, text=''): + def __init__(self, text: str = ''): super().__init__(text, align='right')@@ -185,14 +192,14 @@ class Editor(urwid_readline.ReadlineEdit): Support urwid_readline features along with history support which lacks in urwid_readline """ - def __init__(self, master): + def __init__(self, master: App) -> None: super().__init__(caption='> ', multiline=True) self.master = master - self.history = [] - self.last_index = -1 - self.show_history = False + self.history: List[str] = [] + self.last_index: int = -1 + self.show_history: bool = False - def keypress(self, size, key): + def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: # TODO: Add some logic for down key and clean up logic ifpossible.
# Returning None means the key has been handled by this widget # which otherwise is propogated to the parent widget to bepython/qemu/aqmp/aqmp_tui.py +0
-0@@ -223,7 +230,7 @@ def keypress(self, size, key): else: self.show_history = False
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py def keypress(self, size, key): · line 235
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py +1
-1self.last_index = 0 - return super().keypress(size, key) + return super().keypress(size, key) # type: ignoretry using
cast(Optional[str], super().keypress(size, key))instead of the
type: ignore. It doesn't make a gigantic difference, really, but it looks "narrower" in scope and will probably confuse *me* less in the future.
Fixed
Quoted 109 lines
Quoted 109 lines
python/qemu/aqmp/aqmp_tui.py +5
-5return None@@ -231,7 +238,7 @@ class EditorWidget(urwid.Filler): """ Wraps CustomEdit """ - def __init__(self, master): + def __init__(self, master: App) -> None: super().__init__(Editor(master), valign='top')@@ -239,12 +246,12 @@ class HistoryBox(urwid.ListBox): """ Shows all the QMP message transmitted/received """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history): + def add_to_history(self, history: str) -> None: self.history.append(urwid.Text(history)) if self.history: self.history.set_focus(len(self.history) - 1)@@ -254,7 +261,7 @@ class HistoryWindow(urwid.Frame): """ Composes the HistoryBox and EditorWidget """ - def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master self.editor_widget = EditorWidget(master) self.editor = urwid.LineBox(self.editor_widget)@@ -264,7 +271,7 @@ def __init__(self, master): super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg): + def cb_add_to_history(self, msg: str) -> None: self.history.add_to_history(msg)@@ -275,7 +282,7 @@ class Window(urwid.Frame): future when we will have multiple windows and want to the switchbetween
them and display overlays
"""python/qemu/aqmp/aqmp_tui.py +6
-6- def __init__(self, master): + def __init__(self, master: App) -> None: self.master = master footer = StatusBar() body = HistoryWindow(master)@@ -283,18 +290,18 @@ def __init__(self, master): class TUILogHandler(Handler): - def __init__(self, tui): + def __init__(self, tui: App) -> None: super().__init__() self.tui = tui - def emit(self, record): + def emit(self, record: LogRecord) -> None: level = record.levelname msg = record.getMessage() msg = f'[{level}]: {msg}' self.tui.add_to_history(msg) -def main(): +def main() -> None: parser = argparse.ArgumentParser(description='AQMP TUI') parser.add_argument('qmp_server', help='Address of the QMP server' '< UNIX socket path | TCP addr:port >')@@ -311,7 +318,7 @@ def main(): try: address = QEMUMonitorProtocol.parse_address(args.qmp_server) except QMPBadPortError as err: - parser.error(err) + parser.error(str(err)) app = App(address)@@ -330,4 +337,4 @@ def main(): if __name__ == '__main__': - main() # type: ignore + main()python/setup.cfg +0
-3@@ -83,9 +83,6 @@ namespace_packages = True allow_subclassing_any = True [mypy-qemu.aqmp.aqmp_tui] -disallow_untyped_defs = False -disallow_incomplete_defs = False -check_untyped_defs = False # urwid and urwid_readline have no type stubs: allow_subclassing_any = True
Quoted 6 lines
Quoted 4 lines
--
2.17.1Please squash this into the first patch, but it looks good, thanks!
Thanks.
Quoted 3 lines
--js
PATCH v3 7/13 python: add optional pygments dependency
In reply to G S Niteesh BabuAdded pygments as optional dependency for AQMP TUI. This is required for the upcoming syntax highlighting feature in AQMP TUI.
The dependency has also been added in the devel optional group.
Added mypy 'ignore_missing_imports' for pygments since it does not have any type stubs.
--- python/Pipfile.lock | 8 ++++++++ python/setup.cfg | 5 +++++ 2 files changed, 13 insertions(+)
python/Pipfile.lock +8 -0
@@ -200,6 +200,14 @@ ], "version": "==2.0.0" }, + "pygments": { + "hashes": [ + "sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15aef65f0545519f", + "sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd34d272ca383b8e" + ], + "markers": "python_version >= '3.5'", + "version": "==2.9.0" + }, "pylint": { "hashes": [ "sha256:082a6d461b54f90eea49ca90fff4ee8b6e45e8029e5dbd72f6107ef84f3779c0",
python/setup.cfg +5 -0
@@ -46,6 +46,7 @@ devel = tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13 + Pygments >= 2.9.0 # Provides qom-fuse functionality fuse =
@@ -55,6 +56,7 @@ fuse = tui = urwid >= 2.1.2 urwid-readline >= 0.13 + Pygments >= 2.9.0 [options.entry_points] console_scripts =
@@ -97,6 +99,9 @@ ignore_missing_imports = True [mypy-urwid_readline] ignore_missing_imports = True +[mypy-pygments] +ignore_missing_imports = True + [pylint.messages control] # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this
Signature · 1 line
2.17.1
Re: [PATCH v3 07/13] python: add optional pygments dependency
Quoted 10 lines · G S Niteesh Babu
Added pygments as optional dependency for AQMP TUI. This is required for the upcoming syntax highlighting feature in AQMP TUI.
The dependency has also been added in the devel optional group.Added mypy '
ignore_missing_imports' for pygments since it does not have any type stubs.Signed-off-by G S Niteesh Babu <[email protected]>
LGTM
PATCH v3 8/13 python/aqmp-tui: add syntax highlighting
In reply to G S Niteesh BabuAdd syntax highlighting for the incoming and outgoing QMP messages. This is achieved using the pygments module which was added in a previous commit.
The current implementation is a really simple one which doesn't allow for any configuration. In future this has to be improved to allow for easier theme config using an external config of some sort.
--- python/qemu/aqmp/aqmp_tui.py | 52 +++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 12 deletions(-)
python/qemu/aqmp/aqmp_tui.py +40 -12
@@ -19,6 +19,8 @@ Union, ) +from pygments import lexers +from pygments import token as Token import urwid import urwid_readline
@@ -35,6 +37,22 @@ LOGGER = logging.getLogger() +palette = [ + (Token.Punctuation, '', '', '', 'h15,bold', 'g7'), + (Token.Text, '', '', '', '', 'g7'), + (Token.Name.Tag, '', '', '', 'bold,#f88', 'g7'), + (Token.Literal.Number.Integer, '', '', '', '#fa0', 'g7'), + (Token.Literal.String.Double, '', '', '', '#6f6', 'g7'), + (Token.Keyword.Constant, '', '', '', '#6af', 'g7'), + ('DEBUG', '', '', '', '#ddf', 'g7'), + ('INFO', '', '', '', 'g100', 'g7'), + ('WARNING', '', '', '', '#ff6', 'g7'), + ('ERROR', '', '', '', '#a00', 'g7'), + ('CRITICAL', '', '', '', '#a00', 'g7'), + ('background', '', 'black', '', '', 'g7'), +] + + def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message.
@@ -57,17 +75,14 @@ def __init__(self, address: Union[str, Tuple[str, int]]) -> None: self.aloop: Optional[Any] = None # FIXME: Use more concrete type. super().__init__() - def add_to_history(self, msg: str) -> None: - urwid.emit_signal(self, UPDATE_MSG, msg) + def add_to_history(self, msg: str, level: Optional[str] = None) -> None: + urwid.emit_signal(self, UPDATE_MSG, msg, level) def _cb_outbound(self, msg: Message) -> Message: # FIXME: I think the ideal way to omit these messages during in-TUI - # logging will be to add a filter to the logger. We can use regex to - # filter out messages starting with 'Request:' or 'Response:' but I - # think a better approach will be encapsulate the message in an object - # and filter based on the object. Encapsulation of the message will - # also be necessary when we want different formatting of messages - # inside TUI. + # logging will be to add a filter to the logger. We can use + # regex/startswith to filter out messages starting with 'Request:' or + # 'Response:'. If possible please suggest other ideas. handler = LOGGER.handlers[0] if not isinstance(handler, TUILogHandler): LOGGER.debug('Request: %s', str(msg))
@@ -156,6 +171,9 @@ def _get_formatted_address(self) -> str: self._set_status('Server shutdown') def run(self, debug: bool = False) -> None: + screen = urwid.raw_display.Screen() + screen.set_terminal_properties(256) + self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)
@@ -167,6 +185,8 @@ def run(self, debug: bool = False) -> None: event_loop = urwid.AsyncioEventLoop(loop=self.aloop) main_loop = urwid.MainLoop(urwid.AttrMap(self.window, 'background'), unhandled_input=self.unhandled_input, + screen=screen, + palette=palette, handle_mouse=True, event_loop=event_loop)
@@ -251,7 +271,8 @@ def __init__(self, master: App) -> None: self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history: str) -> None: + def add_to_history(self, + history: Union[str, List[Tuple[str, str]]]) -> None: self.history.append(urwid.Text(history)) if self.history: self.history.set_focus(len(self.history) - 1)
@@ -271,8 +292,15 @@ def __init__(self, master: App) -> None: super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg: str) -> None: - self.history.add_to_history(msg) + def cb_add_to_history(self, msg: str, level: Optional[str] = None) -> None: + formatted = [] + if level: + formatted.append((level, msg)) + else: + lexer = lexers.JsonLexer() # pylint: disable=no-member + for token in lexer.get_tokens(msg): + formatted.append(token) + self.history.add_to_history(formatted) class Window(urwid.Frame):
@@ -298,7 +326,7 @@ def emit(self, record: LogRecord) -> None: level = record.levelname msg = record.getMessage() msg = f'[{level}]: {msg}' - self.tui.add_to_history(msg) + self.tui.add_to_history(msg, level) def main() -> None:
Signature · 1 line
2.17.1
Re: [PATCH v3 08/13] python/aqmp-tui: add syntax highlighting
Quoted 76 lines · G S Niteesh Babu
Add syntax highlighting for the incoming and outgoing QMP messages. This is achieved using the pygments module which was added in a previous commit.
The current implementation is a really simple one which doesn't allow for any configuration. In future this has to be improved to allow for easier theme config using an external config of some sort.
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 52 +++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 12 deletions(-)python/qemu/aqmp/aqmp_tui.py +18
-0@@ -19,6 +19,8 @@ Union, ) +from pygments import lexers +from pygments import token as Token import urwid import urwid_readline@@ -35,6 +37,22 @@ LOGGER = logging.getLogger() +palette = [ + (Token.Punctuation, '', '', '', 'h15,bold', 'g7'), + (Token.Text, '', '', '', '', 'g7'), + (Token.Name.Tag, '', '', '', 'bold,#f88', 'g7'), + (Token.Literal.Number.Integer, '', '', '', '#fa0', 'g7'), + (Token.Literal.String.Double, '', '', '', '#6f6', 'g7'), + (Token.Keyword.Constant, '', '', '', '#6af', 'g7'), + ('DEBUG', '', '', '', '#ddf', 'g7'), + ('INFO', '', '', '', 'g100', 'g7'), + ('WARNING', '', '', '', '#ff6', 'g7'), + ('ERROR', '', '', '', '#a00', 'g7'), + ('CRITICAL', '', '', '', '#a00', 'g7'), + ('background', '', 'black', '', '', 'g7'), +] + + def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message.@@ -57,17 +75,14 @@ def __init__(self, address: Union[str, Tuple[str,int]]) -> None:
self.aloop: Optional[Any] = None # FIXME: Use more concrete type. super().__init__()python/qemu/aqmp/aqmp_tui.py +1
-2- def add_to_history(self, msg: str) -> None: - urwid.emit_signal(self, UPDATE_MSG, msg) + def add_to_history(self, msg: str, level: Optional[str] = None) ->None:
+urwid.emit_signal(self, UPDATE_MSG, msg, level)def
_cb_outbound(self, msg: Message)-> Message:
# FIXME: I think the ideal way to omit these messages during
in-TUI
- # logging will be to add a filter to the logger. We can use regex
to
- # filter out messages starting with 'Request:' or 'Response:' but
I
- # think a better approach will be encapsulate the message in an
object
- # and filter based on the object. Encapsulation of the message
willpython/qemu/aqmp/aqmp_tui.py +2
-2- # also be necessary when we want different formatting of messages - # inside TUI. + # logging will be to add a filter to the logger. We can use + # regex/startswith to filter out messages starting with
Quoted 3 lines
'Request:' or
+ # 'Response:'. If possible please suggest other ideas.
We'll want to fix this up earlier in the series -- the finished version of your series shouldn't have FIXME comments, and later patches in the series shouldn't edit comments added earlier. We chatted on IRC about the right strategy for filtering log messages, so I think V4 should have a strategy in place to avoid this comment, right?
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 176
Quoted 10 lines
handler = LOGGER.handlers[0]
if notisinstance(handler, TUILogHandler):
LOGGER.debug('Request: %s', str(msg))python/qemu/aqmp/aqmp_tui.py +2
-0@@ -156,6 +171,9 @@ def _get_formatted_address(self) -> str: self._set_status('Server shutdown') def run(self, debug: bool = False) -> None: + screen = urwid.raw_display.Screen() + screen.set_terminal_properties(256)
I'm not sure if this will work everywhere, but ... it's fine for now, I assume.
Quoted 35 lines
python/qemu/aqmp/aqmp_tui.py +5
-1+ self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)@@ -167,6 +185,8 @@ def run(self, debug: bool = False) -> None: event_loop = urwid.AsyncioEventLoop(loop=self.aloop) main_loop = urwid.MainLoop(urwid.AttrMap(self.window, 'background'), unhandled_input=self.unhandled_input, + screen=screen, + palette=palette, handle_mouse=True, event_loop=event_loop)@@ -251,7 +271,8 @@ def __init__(self, master: App) -> None: self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history: str) -> None: + def add_to_history(self, + history: Union[str, List[Tuple[str, str]]]) ->None:
self.history.append(urwid.Text(history)) if self.history:
self.history.set_focus(len(self.history) - 1)python/qemu/aqmp/aqmp_tui.py +1
-2@@ -271,8 +292,15 @@ def __init__(self, master: App) -> None: super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg: str) -> None: - self.history.add_to_history(msg) + def cb_add_to_history(self, msg: str, level: Optional[str] = None) ->None:
python/qemu/aqmp/aqmp_tui.py def __init__(self, master: App) -> None: · line 303
Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py +8
-0+ formatted = [] + if level: + formatted.append((level, msg)) + else: + lexer = lexers.JsonLexer() # pylint: disable=no-member + for token in lexer.get_tokens(msg): + formatted.append(token) + self.history.add_to_history(formatted)
This looks like it's conflating having a debugging level with receiving a message that's already formatted.
i.e., the code here assumes that if it receives a level, it is ALSO already formatted. That smells like trouble -- but I think there are some changes to the logging and rendering you had planned for V4 anyway, I believe?
Quoted 16 lines
class Window(urwid.Frame):
python/qemu/aqmp/aqmp_tui.py +1
-2@@ -298,7 +326,7 @@ def emit(self, record: LogRecord) -> None: level = record.levelname msg = record.getMessage() msg = f'[{level}]: {msg}' - self.tui.add_to_history(msg) + self.tui.add_to_history(msg, level) def main() -> None: --2.17.1
Re: [PATCH v3 08/13] python/aqmp-tui: add syntax highlighting
Quoted 83 lines · John Snow
Quoted 78 lines · G S Niteesh Babu
Add syntax highlighting for the incoming and outgoing QMP messages. This is achieved using the pygments module which was added in a previous commit.
The current implementation is a really simple one which doesn't allow for any configuration. In future this has to be improved to allow for easier theme config using an external config of some sort.
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 52 +++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 12 deletions(-)python/qemu/aqmp/aqmp_tui.py +18
-0@@ -19,6 +19,8 @@ Union, ) +from pygments import lexers +from pygments import token as Token import urwid import urwid_readline@@ -35,6 +37,22 @@ LOGGER = logging.getLogger() +palette = [ + (Token.Punctuation, '', '', '', 'h15,bold', 'g7'), + (Token.Text, '', '', '', '', 'g7'), + (Token.Name.Tag, '', '', '', 'bold,#f88', 'g7'), + (Token.Literal.Number.Integer, '', '', '', '#fa0', 'g7'), + (Token.Literal.String.Double, '', '', '', '#6f6', 'g7'), + (Token.Keyword.Constant, '', '', '', '#6af', 'g7'), + ('DEBUG', '', '', '', '#ddf', 'g7'), + ('INFO', '', '', '', 'g100', 'g7'), + ('WARNING', '', '', '', '#ff6', 'g7'), + ('ERROR', '', '', '', '#a00', 'g7'), + ('CRITICAL', '', '', '', '#a00', 'g7'), + ('background', '', 'black', '', '', 'g7'), +] + + def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message.@@ -57,17 +75,14 @@ def __init__(self, address: Union[str, Tuple[str,int]]) -> None:
self.aloop: Optional[Any] = None # FIXME: Use more concrete
type.super().__init__()python/qemu/aqmp/aqmp_tui.py +1
-2- def add_to_history(self, msg: str) -> None: - urwid.emit_signal(self, UPDATE_MSG, msg) + def add_to_history(self, msg: str, level: Optional[str] = None) ->None:
+urwid.emit_signal(self, UPDATE_MSG, msg, level)def
_cb_outbound(self, msg: Message)-> Message:
# FIXME: I think the ideal way to omit these messages during
in-TUI
- # logging will be to add a filter to the logger. We can use
regex to
- # filter out messages starting with 'Request:' or 'Response:'
but I
- # think a better approach will be encapsulate the message in an
object
- # and filter based on the object. Encapsulation of the message
willpython/qemu/aqmp/aqmp_tui.py +2
-2- # also be necessary when we want different formatting of messages - # inside TUI. + # logging will be to add a filter to the logger. We can use + # regex/startswith to filter out messages starting with'Request:' or
Quoted 9 lines
Quoted 2 lines
+ # 'Response:'. If possible please suggest other ideas.
We'll want to fix this up earlier in the series -- the finished version of your series shouldn't have FIXME comments, and later patches in the series shouldn't edit comments added earlier. We chatted on IRC about the right strategy for filtering log messages, so I think V4 should have a strategy in place to avoid this comment, right?
Yes, this has been addressed in the v4.
We avoid logging messages when we log the messages to the TUI screen. This is done by checking if TUILogHandler is installed on the logger or not.
Quoted 8 lines
Quoted 8 lines
handler = LOGGER.handlers[0]
if notisinstance(handler, TUILogHandler):
LOGGER.debug('Request: %s', str(msg))python/qemu/aqmp/aqmp_tui.py +0
-0@@ -156,6 +171,9 @@ def _get_formatted_address(self) -> str: self._set_status('Server shutdown')
Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 179
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py +2
-0def run(self, debug: bool = False) -> None: + screen = urwid.raw_display.Screen() + screen.set_terminal_properties(256)I'm not sure if this will work everywhere, but ... it's fine for now, I assume.
I'll add a note here.
Quoted 47 lines
+
Quoted 32 lines
python/qemu/aqmp/aqmp_tui.py +4
-1self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)@@ -167,6 +185,8 @@ def run(self, debug: bool = False) -> None: event_loop = urwid.AsyncioEventLoop(loop=self.aloop) main_loop = urwid.MainLoop(urwid.AttrMap(self.window, 'background'), unhandled_input=self.unhandled_input, + screen=screen, + palette=palette, handle_mouse=True, event_loop=event_loop)@@ -251,7 +271,8 @@ def __init__(self, master: App) -> None: self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history: str) -> None: + def add_to_history(self, + history: Union[str, List[Tuple[str, str]]]) ->None:
self.history.append(urwid.Text(history)) if self.history:
self.history.set_focus(len(self.history) - 1)python/qemu/aqmp/aqmp_tui.py +1
-2@@ -271,8 +292,15 @@ def __init__(self, master: App) -> None: super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg: str) -> None: - self.history.add_to_history(msg) + def cb_add_to_history(self, msg: str, level: Optional[str] = None)python/qemu/aqmp/aqmp_tui.py def __init__(self, master: App) -> None: · line 305
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py +8
-1-> None: + formatted = [] + if level: + formatted.append((level, msg)) + else: + lexer = lexers.JsonLexer() # pylint: disable=no-member + for token in lexer.get_tokens(msg): + formatted.append(token) + self.history.add_to_history(formatted)This looks like it's conflating having a debugging level with receiving a message that's already formatted.
Quoted 5 lines
i.e., the code here assumes that if it receives a level, it is ALSO already formatted. That smells like trouble -- but I think there are some changes to the logging and rendering you had planned for V4 anyway, I believe?
Why do you think it is trouble?
When a level is provided, it is most probably going to be text messages or QMP messages with text. In either case, there is no formatting to be done. In the second case, we don't have a JSON formattor that can format a JSON message along with text without breaking. So I thought it is better to leave the formatting to the user and just use the level to provide syntax highlighting.
One thing I have changed in v4 is the level is prefixed to the messages rather than the user adding it manually.
i.e. in v3 -
self.add_to_history('[ERROR]: This is error', 'ERROR') results
in [ERROR]: This is error
in v4 -
self.add_to_history('This is error', 'ERROR') results in
[ERROR]: This is error
Quoted 14 lines
Quoted 14 lines
python/qemu/aqmp/aqmp_tui.py +1
-2class Window(urwid.Frame):@@ -298,7 +326,7 @@ def emit(self, record: LogRecord) -> None: level = record.levelname msg = record.getMessage() msg = f'[{level}]: {msg}' - self.tui.add_to_history(msg) + self.tui.add_to_history(msg, level) def main() -> None: --2.17.1
Re: [PATCH v3 08/13] python/aqmp-tui: add syntax highlighting
Quoted 171 lines · Niteesh G. S.
Quoted 84 lines · John Snow
Quoted 79 lines · G S Niteesh Babu
Add syntax highlighting for the incoming and outgoing QMP messages. This is achieved using the pygments module which was added in a previous commit.
The current implementation is a really simple one which doesn't allow for any configuration. In future this has to be improved to allow for easier theme config using an external config of some sort.
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 52 +++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 12 deletions(-)python/qemu/aqmp/aqmp_tui.py +18
-0@@ -19,6 +19,8 @@ Union, ) +from pygments import lexers +from pygments import token as Token import urwid import urwid_readline@@ -35,6 +37,22 @@ LOGGER = logging.getLogger() +palette = [ + (Token.Punctuation, '', '', '', 'h15,bold', 'g7'), + (Token.Text, '', '', '', '', 'g7'), + (Token.Name.Tag, '', '', '', 'bold,#f88', 'g7'), + (Token.Literal.Number.Integer, '', '', '', '#fa0', 'g7'), + (Token.Literal.String.Double, '', '', '', '#6f6', 'g7'), + (Token.Keyword.Constant, '', '', '', '#6af', 'g7'), + ('DEBUG', '', '', '', '#ddf', 'g7'), + ('INFO', '', '', '', 'g100', 'g7'), + ('WARNING', '', '', '', '#ff6', 'g7'), + ('ERROR', '', '', '', '#a00', 'g7'), + ('CRITICAL', '', '', '', '#a00', 'g7'), + ('background', '', 'black', '', '', 'g7'), +] + + def format_json(msg: str) -> str: """ Formats given multiline JSON message into a single line message.@@ -57,17 +75,14 @@ def __init__(self, address: Union[str, Tuple[str,int]]) -> None:
self.aloop: Optional[Any] = None # FIXME: Use more concrete
type.super().__init__()python/qemu/aqmp/aqmp_tui.py +1
-2- def add_to_history(self, msg: str) -> None: - urwid.emit_signal(self, UPDATE_MSG, msg) + def add_to_history(self, msg: str, level: Optional[str] = None) ->None:
+urwid.emit_signal(self, UPDATE_MSG, msg, level)def
_cb_outbound(self, msg: Message)-> Message:
# FIXME: I think the ideal way to omit these messages during
in-TUI
- # logging will be to add a filter to the logger. We can use
regex to
- # filter out messages starting with 'Request:' or 'Response:'
but I
- # think a better approach will be encapsulate the message in an
object
- # and filter based on the object. Encapsulation of the message
will
- # also be necessary when we want different formatting of
messagespython/qemu/aqmp/aqmp_tui.py +2
-1- # inside TUI. + # logging will be to add a filter to the logger. We can use + # regex/startswith to filter out messages starting with'Request:' or
Quoted 9 lines
Quoted 2 lines
+ # 'Response:'. If possible please suggest other ideas.
We'll want to fix this up earlier in the series -- the finished version of your series shouldn't have FIXME comments, and later patches in the series shouldn't edit comments added earlier. We chatted on IRC about the right strategy for filtering log messages, so I think V4 should have a strategy in place to avoid this comment, right?
Yes, this has been addressed in the v4.
We avoid logging messages when we log the messages to the TUI screen. This is done by checking if TUILogHandler is installed on the logger or not.Quoted 8 lines
Quoted 8 lines
handler = LOGGER.handlers[0]
if notisinstance(handler, TUILogHandler):
LOGGER.debug('Request: %s', str(msg))python/qemu/aqmp/aqmp_tui.py +0
-0@@ -156,6 +171,9 @@ def _get_formatted_address(self) -> str: self._set_status('Server shutdown')Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 179
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py +2
-0def run(self, debug: bool = False) -> None: + screen = urwid.raw_display.Screen() + screen.set_terminal_properties(256)I'm not sure if this will work everywhere, but ... it's fine for now, I assume.
I'll add a note here.
Quoted 47 lines
+
Quoted 32 lines
python/qemu/aqmp/aqmp_tui.py +4
-1self.aloop = asyncio.get_event_loop() self.aloop.set_debug(debug)@@ -167,6 +185,8 @@ def run(self, debug: bool = False) -> None: event_loop = urwid.AsyncioEventLoop(loop=self.aloop) main_loop = urwid.MainLoop(urwid.AttrMap(self.window, 'background'), unhandled_input=self.unhandled_input, + screen=screen, + palette=palette, handle_mouse=True, event_loop=event_loop)@@ -251,7 +271,8 @@ def __init__(self, master: App) -> None: self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) - def add_to_history(self, history: str) -> None: + def add_to_history(self, + history: Union[str, List[Tuple[str, str]]]) ->None:
self.history.append(urwid.Text(history)) if self.history:
self.history.set_focus(len(self.history) - 1)python/qemu/aqmp/aqmp_tui.py +1
-2@@ -271,8 +292,15 @@ def __init__(self, master: App) -> None: super().__init__(self.body) urwid.connect_signal(self.master, UPDATE_MSG, self.cb_add_to_history) - def cb_add_to_history(self, msg: str) -> None: - self.history.add_to_history(msg) + def cb_add_to_history(self, msg: str, level: Optional[str] = None)python/qemu/aqmp/aqmp_tui.py def __init__(self, master: App) -> None: · line 305
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py +8
-1-> None: + formatted = [] + if level: + formatted.append((level, msg)) + else: + lexer = lexers.JsonLexer() # pylint: disable=no-member + for token in lexer.get_tokens(msg): + formatted.append(token) + self.history.add_to_history(formatted)This looks like it's conflating having a debugging level with receiving a message that's already formatted.
Quoted 5 lines
i.e., the code here assumes that if it receives a level, it is ALSO already formatted. That smells like trouble -- but I think there are some changes to the logging and rendering you had planned for V4 anyway, I believe?
Quoted 9 lines
Why do you think it is trouble?
When a level is provided, it is most probably going to be text messages or QMP messages with text. In either case, there is no formatting to be done. In the second case, we don't have a JSON formattor that can format a JSON message along with text without breaking. So I thought it is better to leave the formatting to the user and just use the level to provide syntax highlighting.
Because it's an indirect type inference -- you are *indirectly* measuring whether or not a message has been formatted yet. Using the presence or absence of 'level' is an indirect way of proving the condition; are there not more direct ways?
Talking on IRC, I see that you have plans for an abstracted Item view that might make this a null point, though -- so I'll drop this line of interrogation for now.
Thanks!
PATCH v3 9/13 python/aqmp-tui: Add QMP connection manager
In reply to G S Niteesh BabuInstead of manually connecting and disconnecting from the server. We now rely on the runstate to manage the QMP connection.
Along with this the ability to reconnect on certain exceptions has also been added.
--- python/qemu/aqmp/aqmp_tui.py | 109 ++++++++++++++++++++++++++++++----- 1 file changed, 94 insertions(+), 15 deletions(-)
python/qemu/aqmp/aqmp_tui.py +94 -15
@@ -25,8 +25,9 @@ import urwid_readline from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .error import ProtocolError from .message import DeserializationError, Message, UnexpectedTypeError -from .protocol import ConnectError +from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient from .util import create_task, pretty_traceback
@@ -67,12 +68,24 @@ def format_json(msg: str) -> str: return ' '.join(words) +def type_name(mtype: Any) -> str: + """ + Returns the type name + """ + return type(mtype).__name__ + + class App(QMPClient): - def __init__(self, address: Union[str, Tuple[str, int]]) -> None: + def __init__(self, address: Union[str, Tuple[str, int]], num_retries: int, + retry_delay: Optional[int]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self) self.address = address self.aloop: Optional[Any] = None # FIXME: Use more concrete type. + self.num_retries = num_retries + self.retry_delay = retry_delay + self.retry: bool = False + self.disconnecting: bool = False super().__init__() def add_to_history(self, msg: str, level: Optional[str] = None) -> None:
@@ -119,7 +132,7 @@ def _cb_inbound(self, msg: Message) -> Message: LOGGER.info('Error server disconnected before reply') urwid.emit_signal(self, UPDATE_MSG, '{"error": "Server disconnected before reply"}') - self._set_status("Server disconnected") + await self.disconnect() except Exception as err: LOGGER.error('Exception from _send_to_server: %s', str(err)) raise err
@@ -136,15 +149,29 @@ def kill_app(self) -> None: create_task(self._kill_app()) async def _kill_app(self) -> None: - # It is ok to call disconnect even in disconnect state + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + raise urwid.ExitMainLoop() + + async def disconnect(self) -> None: + if self.disconnecting: + return try: - await self.disconnect() - LOGGER.debug('Disconnect finished. Exiting app') + self.disconnecting = True + await super().disconnect() + self.retry = True + except EOFError as err: + LOGGER.info('disconnect: %s', type_name(err)) + self.retry = True + except ProtocolError as err: + LOGGER.info('disconnect: %s', type_name(err)) + self.retry = False except Exception as err: - LOGGER.info('_kill_app: %s', str(err)) - # Let the app crash after providing a proper stack trace + LOGGER.error('disconnect: Unhandled exception %s', str(err)) + self.retry = False raise err - raise urwid.ExitMainLoop() + finally: + self.disconnecting = False def handle_event(self, event: Message) -> None: # FIXME: Consider all states present in qapi/run-state.json
@@ -161,14 +188,61 @@ def _get_formatted_address(self) -> str: addr = f'{host}:{port}' return addr - async def connect_server(self) -> None: + async def _retry_connection(self) -> Optional[str]: + current_retries = 0 + err = None + # Increase in power sequence of 2 if no delay is provided + cur_delay = 1 + inc_delay = 2 + if self.retry_delay: + inc_delay = 1 + cur_delay = self.retry_delay + # initial try + await self.connect_server() + while self.retry and current_retries < self.num_retries: + LOGGER.info('Connection Failed, retrying in %d', cur_delay) + status = f'[Retry #{current_retries} ({cur_delay}s)]' + self._set_status(status) + + await asyncio.sleep(cur_delay) + + err = await self.connect_server() + cur_delay *= inc_delay + # Cap delay to 5mins + cur_delay = min(cur_delay, 5 * 60) + current_retries += 1 + # If all retries failed report the last error + LOGGER.info('All retries failed: %s', str(err)) + return type_name(err) + + async def manage_connection(self) -> None: + while True: + if self.runstate == Runstate.IDLE: + LOGGER.info('Trying to reconnect') + err = await self._retry_connection() + # If retry is still true then, we have exhausted all our tries. + if self.retry: + self._set_status(f'Error: {err}') + else: + addr = self._get_formatted_address() + self._set_status(f'[Connected {addr}]') + elif self.runstate == Runstate.DISCONNECTING: + self._set_status('[Disconnected]') + await self.disconnect() + # check if a retry is needed + if self.runstate == Runstate.IDLE: + continue + await self.runstate_changed() + + async def connect_server(self) -> Optional[str]: try: await self.connect(self.address) - addr = self._get_formatted_address() - self._set_status(f'Connected to {addr}') + self.retry = False except ConnectError as err: LOGGER.info('connect_server: ConnectError %s', str(err)) - self._set_status('Server shutdown') + self.retry = True + return type_name(err) + return None def run(self, debug: bool = False) -> None: screen = urwid.raw_display.Screen()
@@ -191,7 +265,7 @@ def run(self, debug: bool = False) -> None: event_loop=event_loop) create_task(self.wait_for_events(), self.aloop) - create_task(self.connect_server(), self.aloop) + create_task(self.manage_connection(), self.aloop) try: main_loop.run() except Exception as err:
@@ -333,6 +407,11 @@ def main() -> None: parser = argparse.ArgumentParser(description='AQMP TUI') parser.add_argument('qmp_server', help='Address of the QMP server' '< UNIX socket path | TCP addr:port >') + parser.add_argument('--num-retries', type=int, default=10, + help='Number of times to reconnect before giving up') + parser.add_argument('--retry-delay', type=int, + help='Time(s) to wait before next retry.' + 'Default action is to increase delay in powers of 2') parser.add_argument('--log-file', help='The Log file name') parser.add_argument('--log-level', default='WARNING', help='Log level <CRITICAL|ERROR|WARNING|INFO|DEBUG|>')
@@ -348,7 +427,7 @@ def main() -> None: except QMPBadPortError as err: parser.error(str(err)) - app = App(address) + app = App(address, args.num_retries, args.retry_delay) if args.log_file: LOGGER.addHandler(logging.FileHandler(args.log_file))
Signature · 1 line
2.17.1
Re: [PATCH v3 09/13] python/aqmp-tui: Add QMP connection manager
Quoted 28 lines · G S Niteesh Babu
Instead of manually connecting and disconnecting from the server. We now rely on the runstate to manage the QMP connection.
Along with this the ability to reconnect on certain exceptions has also been added.
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 109 ++++++++++++++++++++++++++++++----- 1 file changed, 94 insertions(+), 15 deletions(-)python/qemu/aqmp/aqmp_tui.py +2
-1@@ -25,8 +25,9 @@ import urwid_readline from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .error import ProtocolError from .message import DeserializationError, Message, UnexpectedTypeError -from .protocol import ConnectError +from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient from .util import create_task, pretty_traceback
python/qemu/aqmp/aqmp_tui.py def format_json(msg: str) -> str: · line 76
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py +5
-0@@ -67,12 +68,24 @@ def format_json(msg: str) -> str: return ' '.join(words) +def type_name(mtype: Any) -> str: + """ + Returns the type name + """ + return type(mtype).__name__
This is a lot of lines for something that doesn't do very much -- do we really need it?
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py +3
-1+ + class App(QMPClient): - def __init__(self, address: Union[str, Tuple[str, int]]) -> None: + def __init__(self, address: Union[str, Tuple[str, int]], num_retries: int,
python/qemu/aqmp/aqmp_tui.py def format_json(msg: str) -> str: · line 90
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py +5
-0+ retry_delay: Optional[int]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self) self.address = address self.aloop: Optional[Any] = None # FIXME: Use more concrete type. + self.num_retries = num_retries + self.retry_delay = retry_delay + self.retry: bool = False + self.disconnecting: bool = False
Why is this one needed again ? ...
Quoted 72 lines
super().__init__() def add_to_history(self, msg: str, level: Optional[str] = None) ->None:
python/qemu/aqmp/aqmp_tui.py +1
-1@@ -119,7 +132,7 @@ def _cb_inbound(self, msg: Message) -> Message: LOGGER.info('Error server disconnected before reply') urwid.emit_signal(self, UPDATE_MSG, '{"error": "Server disconnected before reply"}') - self._set_status("Server disconnected") + await self.disconnect() except Exception as err: LOGGER.error('Exception from _send_to_server: %s', str(err))raise err
python/qemu/aqmp/aqmp_tui.py +37
-7@@ -136,15 +149,29 @@ def kill_app(self) -> None: create_task(self._kill_app()) async def _kill_app(self) -> None: - # It is ok to call disconnect even in disconnect state + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + raise urwid.ExitMainLoop() + + async def disconnect(self) -> None: + if self.disconnecting: + return try: - await self.disconnect() - LOGGER.debug('Disconnect finished. Exiting app') + self.disconnecting = True + await super().disconnect() + self.retry = True + except EOFError as err: + LOGGER.info('disconnect: %s', type_name(err)) + self.retry = True + except ProtocolError as err: + LOGGER.info('disconnect: %s', type_name(err)) + self.retry = False except Exception as err: - LOGGER.info('_kill_app: %s', str(err)) - # Let the app crash after providing a proper stack trace + LOGGER.error('disconnect: Unhandled exception %s', str(err)) + self.retry = False raise err - raise urwid.ExitMainLoop() + finally: + self.disconnecting = False def handle_event(self, event: Message) -> None: # FIXME: Consider all states present in qapi/run-state.json@@ -161,14 +188,61 @@ def _get_formatted_address(self) -> str: addr = f'{host}:{port}' return addr - async def connect_server(self) -> None: + async def _retry_connection(self) -> Optional[str]: + current_retries = 0 + err = None + # Increase in power sequence of 2 if no delay is provided + cur_delay = 1 + inc_delay = 2 + if self.retry_delay: + inc_delay = 1 + cur_delay = self.retry_delay + # initial try + await self.connect_server() + while self.retry and current_retries < self.num_retries: + LOGGER.info('Connection Failed, retrying in %d', cur_delay) + status = f'[Retry #{current_retries} ({cur_delay}s)]' + self._set_status(status) + + await asyncio.sleep(cur_delay)
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 217
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py +9
-0+ + err = await self.connect_server() + cur_delay *= inc_delay + # Cap delay to 5mins + cur_delay = min(cur_delay, 5 * 60) + current_retries += 1 + # If all retries failed report the last error + LOGGER.info('All retries failed: %s', str(err)) + return type_name(err)
I had suggested something like an exponential backoff, but maybe a constant delay would be a little cleaner to implement for right now without getting too fancy over it. If you go with a simpler retry algorithm, do you think you could clean up the logic in the retry loop here a bit more?
Something like:
for _ in range(num_retries):
try:
whatever_you_have_to_do_to_connect()
return
except ConnectError as err:
LOGGER.info(...etc)
await asyncio.sleep(whatever_the_delay_is)
# ran out of retries here, presumably the connection manager will just go idle until the user interferes some other way.
In particular, I think passing around the name of the exception is a little dubious -- we should be logging with the actual Exception we've received.
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 246
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py +5
-0+ + async def manage_connection(self) -> None: + while True: + if self.runstate == Runstate.IDLE: + LOGGER.info('Trying to reconnect')
But will this be true upon the very first boot? This message might not be right.
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 246
Quoted 2 lines
python/qemu/aqmp/aqmp_tui.py +1
-0+ err = await self._retry_connection()
This seems named oddly too, since it might be the initial attempt and not necessarily a reconnection or a retry.
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 225
Quoted 5 lines
python/qemu/aqmp/aqmp_tui.py +1
-0+ # If retry is still true then, we have exhausted all ourtries.
python/qemu/aqmp/aqmp_tui.py +2
-0+ if self.retry: + self._set_status(f'Error: {err}')
+ else:
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 232
Quoted 7 lines
python/qemu/aqmp/aqmp_tui.py +6
-0+ addr = self._get_formatted_address() + self._set_status(f'[Connected {addr}]') + elif self.runstate == Runstate.DISCONNECTING: + self._set_status('[Disconnected]') + await self.disconnect() + # check if a retry is needed
Is this required? I would have hoped that after calling disconnect that the state would have again changed to IDLE and you wouldn't need this clause here.
Quoted 53 lines
python/qemu/aqmp/aqmp_tui.py +15
-4+ if self.runstate == Runstate.IDLE: + continue + await self.runstate_changed() + + async def connect_server(self) -> Optional[str]: try: await self.connect(self.address) - addr = self._get_formatted_address() - self._set_status(f'Connected to {addr}') + self.retry = False except ConnectError as err: LOGGER.info('connect_server: ConnectError %s', str(err)) - self._set_status('Server shutdown') + self.retry = True + return type_name(err) + return None def run(self, debug: bool = False) -> None: screen = urwid.raw_display.Screen()@@ -191,7 +265,7 @@ def run(self, debug: bool = False) -> None: event_loop=event_loop) create_task(self.wait_for_events(), self.aloop) - create_task(self.connect_server(), self.aloop) + create_task(self.manage_connection(), self.aloop) try: main_loop.run() except Exception as err:@@ -333,6 +407,11 @@ def main() -> None: parser = argparse.ArgumentParser(description='AQMP TUI') parser.add_argument('qmp_server', help='Address of the QMP server' '< UNIX socket path | TCP addr:port >') + parser.add_argument('--num-retries', type=int, default=10, + help='Number of times to reconnect before giving up') + parser.add_argument('--retry-delay', type=int, + help='Time(s) to wait before next retry.' + 'Default action is to increase delay in powers of 2') parser.add_argument('--log-file', help='The Log file name')parser.add_argument('--log-level', default='WARNING',
help='Log level
<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')python/qemu/aqmp/aqmp_tui.py +1
-2@@ -348,7 +427,7 @@ def main() -> None: except QMPBadPortError as err: parser.error(str(err)) - app = App(address) + app = App(address, args.num_retries, args.retry_delay) if args.log_file: LOGGER.addHandler(logging.FileHandler(args.log_file)) --
Quoted 3 lines
2.17.1
Right idea overall - possibly needs some polish and to be integrated with an earlier patch to avoid the intermediate FIXMEs.
Thanks,
--js
Re: [PATCH v3 09/13] python/aqmp-tui: Add QMP connection manager
Quoted 36 lines · John Snow
Quoted 31 lines · G S Niteesh Babu
Instead of manually connecting and disconnecting from the server. We now rely on the runstate to manage the QMP connection.
Along with this the ability to reconnect on certain exceptions has also been added.
Signed-off-by G S Niteesh Babu <[email protected]>--- python/qemu/aqmp/aqmp_tui.py | 109 ++++++++++++++++++++++++++++++----- 1 file changed, 94 insertions(+), 15 deletions(-)python/qemu/aqmp/aqmp_tui.py +2
-1@@ -25,8 +25,9 @@ import urwid_readline from ..qmp import QEMUMonitorProtocol, QMPBadPortError +from .error import ProtocolError from .message import DeserializationError, Message, UnexpectedTypeError -from .protocol import ConnectError +from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient from .util import create_task, pretty_traceback@@ -67,12 +68,24 @@ def format_json(msg: str) -> str: return ' '.join(words)
Quoted 10 lines
python/qemu/aqmp/aqmp_tui.py def format_json(msg: str) -> str: · line 90
Quoted 6 lines
python/qemu/aqmp/aqmp_tui.py +5
-0+def type_name(mtype: Any) -> str: + """ + Returns the type name + """ + return type(mtype).__name__This is a lot of lines for something that doesn't do very much -- do we really need it?
No. This has been removed in v4.
Quoted 13 lines
Quoted 13 lines
python/qemu/aqmp/aqmp_tui.py +4
-1+ + class App(QMPClient): - def __init__(self, address: Union[str, Tuple[str, int]]) -> None: + def __init__(self, address: Union[str, Tuple[str, int]], num_retries: int, + retry_delay: Optional[int]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self) self.address = address self.aloop: Optional[Any] = None # FIXME: Use more concrete
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py def format_json(msg: str) -> str: · line 88
Quoted 6 lines
type.
python/qemu/aqmp/aqmp_tui.py +4
-0+ self.num_retries = num_retries + self.retry_delay = retry_delay + self.retry: bool = False + self.disconnecting: bool = FalseWhy is this one needed again ? ...
A race condition occurs in protocol.py line 597
The reason behind this is there are two disconnect calls initiated. The first one via kill_app
and the second one via manage_connection when the state is set to disconnecting by the first call.
One of the calls set's the state to IDLE(protocol.py:584) after it has finished disconnecting, meanwhile
the second call is somehow in the process of disconnecting and assert the state to be in DISCONNECTING
in protocol.py:597, which it is not since it has been set to IDLE by the first call.
If I don't gaurd against the second call I get the following exception
------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/niteesh/development/qemu/python/.venv/bin/aqmp-tui", line 33,
in <module>
sys.exit(load_entry_point('qemu', 'console_scripts', 'aqmp-tui')()) File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
695, in main
app.run(args.asyncio_debug)
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
444, in run
raise err
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
441, in run
main_loop.run()
File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
line 287, in run
self._run() File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py", line 385, in _run
self.event_loop.run()
File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py", line 1494, in run
reraise(*exc_info)
File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/compat.py", line 58, in reraise
raise value
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
391, in manage_connection
await self.disconnect()
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
312, in disconnect
raise err
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
300, in disconnect
await super().disconnect()
File "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line
302, in disconnect
await self._wait_disconnect()
File "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line
583, in _wait_disconnect
self._cleanup()
File "/home/niteesh/development/qemu/python/qemu/aqmp/qmp_client.py",
line 331, in _cleanup
super()._cleanup()
File "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line
597, in _cleanup
assert self.runstate == Runstate.DISCONNECTINGAssertionError
-------------------------------------------------------------------------------------------
Quoted 80 lines
Quoted 80 lines
super().__init__() def add_to_history(self, msg: str, level: Optional[str] = None) ->None:
python/qemu/aqmp/aqmp_tui.py +1
-1@@ -119,7 +132,7 @@ def _cb_inbound(self, msg: Message) -> Message: LOGGER.info('Error server disconnected before reply') urwid.emit_signal(self, UPDATE_MSG, '{"error": "Server disconnected before reply"}') - self._set_status("Server disconnected") + await self.disconnect() except Exception as err: LOGGER.error('Exception from _send_to_server: %s', str(err))raise err
python/qemu/aqmp/aqmp_tui.py +44
-7@@ -136,15 +149,29 @@ def kill_app(self) -> None: create_task(self._kill_app()) async def _kill_app(self) -> None: - # It is ok to call disconnect even in disconnect state + await self.disconnect() + LOGGER.debug('Disconnect finished. Exiting app') + raise urwid.ExitMainLoop() + + async def disconnect(self) -> None: + if self.disconnecting: + return try: - await self.disconnect() - LOGGER.debug('Disconnect finished. Exiting app') + self.disconnecting = True + await super().disconnect() + self.retry = True + except EOFError as err: + LOGGER.info('disconnect: %s', type_name(err)) + self.retry = True + except ProtocolError as err: + LOGGER.info('disconnect: %s', type_name(err)) + self.retry = False except Exception as err: - LOGGER.info('_kill_app: %s', str(err)) - # Let the app crash after providing a proper stack trace + LOGGER.error('disconnect: Unhandled exception %s', str(err)) + self.retry = False raise err - raise urwid.ExitMainLoop() + finally: + self.disconnecting = False def handle_event(self, event: Message) -> None: # FIXME: Consider all states present in qapi/run-state.json@@ -161,14 +188,61 @@ def _get_formatted_address(self) -> str: addr = f'{host}:{port}' return addr - async def connect_server(self) -> None: + async def _retry_connection(self) -> Optional[str]: + current_retries = 0 + err = None + # Increase in power sequence of 2 if no delay is provided + cur_delay = 1 + inc_delay = 2 + if self.retry_delay: + inc_delay = 1 + cur_delay = self.retry_delay + # initial try + await self.connect_server() + while self.retry and current_retries < self.num_retries: + LOGGER.info('Connection Failed, retrying in %d', cur_delay) + status = f'[Retry #{current_retries} ({cur_delay}s)]' + self._set_status(status) + + await asyncio.sleep(cur_delay) + + err = await self.connect_server() + cur_delay *= inc_delay + # Cap delay to 5mins + cur_delay = min(cur_delay, 5 * 60) + current_retries += 1 + # If all retries failed report the last error
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 246
Quoted 3 lines
python/qemu/aqmp/aqmp_tui.py +2
-0+ LOGGER.info('All retries failed: %s', str(err)) + return type_name(err)I had suggested something like an exponential backoff, but maybe a constant delay would be a little cleaner to implement for right now without getting too fancy over it. If you go with a simpler retry algorithm, do you think you could clean up the logic in the retry loop here a bit more?
Yes, we can. I'll refactor it to constant delay.
Quoted 9 lines
Something like:
for _ in
range(num_retries):
try:
whatever_you_have_to_do_to_connect()
return
except ConnectError as err:
LOGGER.info(...etc)
awaitasyncio.sleep(whatever_the_delay_is)
Quoted 8 lines
# ran out of retries here, presumably the connection manager will just go idle until the user interferes some other way.In particular, I think passing around the name of the exception is a little dubious -- we should be logging with the actual Exception we've received.
This has been fixed in V4. We pass the exception now instead of just passing around the name.
Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 246
Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py +5
-0+ + async def manage_connection(self) -> None: + while True: + if self.runstate == Runstate.IDLE: + LOGGER.info('Trying to reconnect')
Quoted 3 lines
But will this be true upon the very first boot? This message might not be right.
Yes, it also occurs in the first boot. I'll fix this in the V3.
Quoted 8 lines
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 246
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py +1
-0+ err = await self._retry_connection()This seems named oddly too, since it might be the initial attempt and not necessarily a reconnection or a retry.
Will fix that.
Quoted 16 lines
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 225
Quoted 7 lines
python/qemu/aqmp/aqmp_tui.py +1
-0+ # If retry is still true then, we have exhausted all ourtries.
python/qemu/aqmp/aqmp_tui.py +2
-0+ if self.retry: + self._set_status(f'Error: {err}')+ else:
python/qemu/aqmp/aqmp_tui.py def _get_formatted_address(self) -> str: · line 232
Quoted 7 lines
python/qemu/aqmp/aqmp_tui.py +6
-0+ addr = self._get_formatted_address() + self._set_status(f'[Connected {addr}]') + elif self.runstate == Runstate.DISCONNECTING: + self._set_status('[Disconnected]') + await self.disconnect() + # check if a retry is needed
Quoted 4 lines
Is this required? I would have hoped that after calling disconnect that the state would have again changed to IDLE and you wouldn't need this clause here.
After you mentioned it I too felt it was redundant. But on removing it the whole app freezes when trying to exit.
I logged the state after the call to disconnect, instead of being in the IDLE state, it is still in DISCONNECTING state.
I suspect this results in the constant infinite looping which doesn't give other coroutines a chance to run and blocks
the event loop thus resulting in the freezing of the app. But I am not sure why the state isn't changing to IDLE.
Quoted 64 lines
Quoted 55 lines
python/qemu/aqmp/aqmp_tui.py +15
-4+ if self.runstate == Runstate.IDLE: + continue + await self.runstate_changed() + + async def connect_server(self) -> Optional[str]: try: await self.connect(self.address) - addr = self._get_formatted_address() - self._set_status(f'Connected to {addr}') + self.retry = False except ConnectError as err: LOGGER.info('connect_server: ConnectError %s', str(err)) - self._set_status('Server shutdown') + self.retry = True + return type_name(err) + return None def run(self, debug: bool = False) -> None: screen = urwid.raw_display.Screen()@@ -191,7 +265,7 @@ def run(self, debug: bool = False) -> None: event_loop=event_loop) create_task(self.wait_for_events(), self.aloop) - create_task(self.connect_server(), self.aloop) + create_task(self.manage_connection(), self.aloop) try: main_loop.run() except Exception as err:@@ -333,6 +407,11 @@ def main() -> None: parser = argparse.ArgumentParser(description='AQMP TUI') parser.add_argument('qmp_server', help='Address of the QMP server' '< UNIX socket path | TCP addr:port >') + parser.add_argument('--num-retries', type=int, default=10, + help='Number of times to reconnect before giving up') + parser.add_argument('--retry-delay', type=int, + help='Time(s) to wait before next retry.' + 'Default action is to increase delay in powers of 2') parser.add_argument('--log-file', help='The Log file name')parser.add_argument('--log-level', default='WARNING',
help='Log level
<CRITICAL|ERROR|WARNING|INFO|DEBUG|>')python/qemu/aqmp/aqmp_tui.py +1
-2@@ -348,7 +427,7 @@ def main() -> None: except QMPBadPortError as err: parser.error(str(err)) - app = App(address) + app = App(address, args.num_retries, args.retry_delay) if args.log_file: LOGGER.addHandler(logging.FileHandler(args.log_file)) --Quoted 3 lines
2.17.1
Right idea overall - possibly needs some polish and to be integrated with an earlier patch to avoid the intermediate FIXMEs.
Thanks,
--js
Re: [PATCH v3 09/13] python/aqmp-tui: Add QMP connection manager
Quoted 10 lines · Niteesh G. S.
Quoted 6 lines · John Snow
On Fri, Jul 30, 2021 at 4:19 PM G S Niteesh Babu <[email protected]> wrote:
[...]
Quoted 89 lines
Quoted 13 lines
Quoted 13 lines
python/qemu/aqmp/aqmp_tui.py +3
-1+ + class App(QMPClient): - def __init__(self, address: Union[str, Tuple[str, int]]) -> None: + def __init__(self, address: Union[str, Tuple[str, int]],
num_retries: int,
+retry_delay: Optional[int]) -> None:
urwid.register_signal(type(self), UPDATE_MSG)self.window = Window(self)
self.address = address
self.aloop: Optional[Any] = None # FIXME: Use more concreteQuoted 9 lines
python/qemu/aqmp/aqmp_tui.py def format_json(msg: str) -> str: · line 88
Quoted 6 lines
type.
python/qemu/aqmp/aqmp_tui.py +4
-0+ self.num_retries = num_retries + self.retry_delay = retry_delay + self.retry: bool = False + self.disconnecting: bool = FalseWhy is this one needed again ? ...
A race condition occurs in
protocol.pyline 597
The reason behind this is there are two disconnect calls initiated. The first one viakill_app
and the second one viamanage_connectionwhen the state is set to disconnecting by the first call.
One of the calls set's the state to IDLE(protocol.py:584) after it has finished disconnecting, meanwhile
the second call is somehow in the process of disconnecting and assert the state to be in DISCONNECTING
in protocol.py:597, which it is not since it has been set to IDLE by the first call.If I don't gaurd against the second call I get the following exception
------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/niteesh/development/qemu/python/.venv/bin/aqmp-tui", line
33, in <module>sys.exit(load_entry_point('qemu', 'console_scripts', 'aqmp-tui')())File "
/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
695, in main
app.run(args.asyncio_debug)
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
444, in run
raise err
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
441, in runmain_loop.run() File "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py", line 287, in run self._run()File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py", line 385, in_run
self.event_loop.run()
File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py", line 1494, in run
reraise(*exc_info)
File
"/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/compat.py", line 58, in reraise
raise value
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
391, inmanage_connection
awaitself.disconnect()
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
312, in disconnect
raise err
File "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line
300, in disconnect
await super().disconnect()
File "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line
302, in disconnect
awaitself._wait_disconnect()
File "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line
583, in_wait_disconnectself._cleanup() File "/home/niteesh/development/qemu/python/qemu/aqmp/qmp_client.py", line 331, in _cleanup super()._cleanup() File "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 597, in _cleanup assert self.runstate == Runstate.DISCONNECTING
Quoted 4 lines
AssertionError
-------------------------------------------------------------------------------------------
Hm, OK. I'm not sure if this is a bug on my part or not yet, I'll investigate.
Quoted 19 lines
def add_to_history(self, msg: str, level: Optional[str] = None) ->Quoted 9 lines
Quoted 9 lines
None:
python/qemu/aqmp/aqmp_tui.py +1
-1@@ -119,7 +132,7 @@ def _cb_inbound(self, msg: Message) -> Message: LOGGER.info('Error server disconnected before reply') urwid.emit_signal(self, UPDATE_MSG, '{"error": "Server disconnected before reply"}') - self._set_status("Server disconnected") + await self.disconnect() except Exception as err:Quoted 9 lines
python/qemu/aqmp/aqmp_tui.py def kill_app(self) -> None: · line 149
Quoted 4 lines
python/qemu/aqmp/aqmp_tui.py +0
-0LOGGER.error('Exception from _send_to_server: %s', str(err))raise err
python/qemu/aqmp/aqmp_tui.py +0
-0@@ -136,15 +149,29 @@ def kill_app(self) -> None: create_task(self._kill_app())Is this required? I would have hoped that after calling disconnect that the state would have again changed to IDLE and you wouldn't need this clause here.
Quoted 9 lines
After you mentioned it I too felt it was redundant. But on removing it the whole app freezes when trying to exit.
I logged the state after the call to disconnect, instead of being in the IDLE state, it is still in DISCONNECTING state.
I suspect this results in the constant infinite looping which doesn't give other coroutines a chance to run and blocks
the event loop thus resulting in the freezing of the app. But I am not sure why the state isn't changing to IDLE.
Hmm ... That may well be a bug in AQMP then. I will investigate.
Re: [PATCH v3 09/13] python/aqmp-tui: Add QMP connection manager
Quoted 23 lines · John Snow
Quoted 6 lines · Niteesh G. S.
Quoted 3 lines · John Snow
On Fri, Jul 30, 2021 at 4:19 PM G S Niteesh Babu <[email protected]> wrote:
Is this required? I would have hoped that after calling disconnect that
Quoted 12 lines
Quoted 3 lines
the state would have again changed to IDLE and you wouldn't need this clause here.
After you mentioned it I too felt it was redundant. But on removing it the whole app freezes when trying to exit.
I logged the state after the call to disconnect, instead of being in the IDLE state, it is still in DISCONNECTING state.
I suspect this results in the constant infinite looping which doesn't give other coroutines a chance to run and blocks
the event loop thus resulting in the freezing of the app. But I am not sure why the state isn't changing to IDLE.
Quoted 2 lines
Hmm ... That may well be a bug in AQMP then. I will investigate.
No, it's not -- It's just tricky. The "problem" is that the runstate_changed() event only returns once the runstate has changed *after* you await it. So this code is perfectly correct and I am just bad at reading.
--js
PATCH v3 10/13 python/aqmp-tui: Add scrolling to history box
In reply to G S Niteesh BabuAdds scroll support to history box. The list can now be scrolled using arrow keys, page up/down and the mouse.
The current implementation requires the widget to be in focus to enable scrolling. Therefore the user has to click on the widget before scrolling.
--- python/qemu/aqmp/aqmp_tui.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)
python/qemu/aqmp/aqmp_tui.py +13 -0
@@ -351,6 +351,19 @@ def add_to_history(self, if self.history: self.history.set_focus(len(self.history) - 1) + def mouse_event(self, size: Tuple[int, int], _event: str, button: float, + _x: int, _y: int, focus: bool) -> None: + # Scroll only on focus. Therefore it is required to + # click on the widget to enable scrolling. + if not focus: + return + # button == 4 represents scroll up event + if button == 4.0: + super().keypress(size, 'up') + # button == 5 represents scroll down event + elif button == 5.0: + super().keypress(size, 'down') + class HistoryWindow(urwid.Frame): """
Signature · 1 line
2.17.1
PATCH v3 11/13 python/aqmp-tui: Add ability to highlight messages
In reply to G S Niteesh BabuAdds ability to highlight messages in the history box. The messages can be selected using up/down arrow keys.
This can be enhanced in the future to apply specific settings to a particular message.
--- python/qemu/aqmp/aqmp_tui.py | 50 ++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+)
python/qemu/aqmp/aqmp_tui.py +50 -0
@@ -344,6 +344,7 @@ def __init__(self, master: App) -> None: self.master = master self.history = urwid.SimpleFocusListWalker([]) super().__init__(self.history) + self.highlighting = -1 def add_to_history(self, history: Union[str, List[Tuple[str, str]]]) -> None:
@@ -351,8 +352,57 @@ def add_to_history(self, if self.history: self.history.set_focus(len(self.history) - 1) + def _remove_highlighting(self) -> None: + assert self.highlighting != -1 + pos = self.highlighting + widget = self.history[pos] + widget = widget.original_widget + self.history[pos] = widget + + def _update_highlighting(self) -> None: + assert self.highlighting != -1 + pos = self.highlighting + widget = self.history[pos] + self.history[pos] = urwid.LineBox(widget) + + def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: + if key == 'up': + if self.highlighting != -1: + pos = self.highlighting + self._remove_highlighting() + pos = max(pos - 1, 0) + self.highlighting = pos + else: + self.highlighting = len(self.history) - 1 + self._update_highlighting() + self.change_focus(size, self.highlighting) + return None + if key == 'down': + pos = self.highlighting + if pos == -1: + return None + + self._remove_highlighting() + if pos == len(self.history) - 1: + self.highlighting = -1 + else: + self.highlighting = pos + 1 + self._update_highlighting() + self.change_focus(size, self.highlighting) + return None + + # Remove highlighting if someother key is pressed + if self.highlighting != -1: + self._remove_highlighting() + self.highlighting = -1 + return super().keypress(size, key) # type: ignore + def mouse_event(self, size: Tuple[int, int], _event: str, button: float, _x: int, _y: int, focus: bool) -> None: + if self.highlighting != -1: + self._remove_highlighting() + self.highlighting = -1 + # Scroll only on focus. Therefore it is required to # click on the widget to enable scrolling. if not focus:
Signature · 1 line
2.17.1
PATCH v3 12/13 python/aqmp-tui: Add pyperclip dependency
In reply to G S Niteesh BabuThis dependency is required to enable copying from the TUI using special keys to the system clipboard.
pyperclip works out of the box on windows and macos but requires xsel/xclip to be installed on linux machines.
--- python/Pipfile.lock | 22 ++++++++++++++++++++++ python/setup.cfg | 5 +++++ 2 files changed, 27 insertions(+)
python/Pipfile.lock +22 -0
@@ -45,6 +45,14 @@ "index": "pypi", "version": "==87.0" }, + "backports.entry-points-selectable": { + "hashes": [ + "sha256:988468260ec1c196dab6ae1149260e2f5472c9110334e5d51adcb77867361f6a", + "sha256:a6d9a871cde5e15b4c4a53e3d43ba890cc6861ec1332c9c2428c92f977192acc" + ], + "markers": "python_version >= '2.7'", + "version": "==1.1.0" + }, "distlib": { "hashes": [ "sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa28420463a6f799736",
@@ -169,6 +177,14 @@ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", "version": "==20.9" }, + "platformdirs": { + "hashes": [ + "sha256:4666d822218db6a262bdfdc9c39d21f23b4cfdb08af331a81e92751daf6c866c", + "sha256:632daad3ab546bd8e6af0537d09805cec458dce201bccfe23012df73332e181e" + ], + "markers": "python_version >= '3.6'", + "version": "==2.2.0" + }, "pluggy": { "hashes": [ "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
@@ -224,6 +240,12 @@ "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", "version": "==2.4.7" }, + "pyperclip": { + "hashes": [ + "sha256:105254a8b04934f0bc84e9c24eb360a591aaf6535c9def5f29d92af107a9bf57" + ], + "version": "==1.8.2" + }, "qemu": { "editable": true, "path": "."
python/setup.cfg +5 -0
@@ -47,6 +47,7 @@ devel = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + pyperclip >= 1.8.2 # Provides qom-fuse functionality fuse =
@@ -57,6 +58,7 @@ tui = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + pyperclip >= 1.8.2 [options.entry_points] console_scripts =
@@ -102,6 +104,9 @@ ignore_missing_imports = True [mypy-pygments] ignore_missing_imports = True +[mypy-pyperclip] +ignore_missing_imports = True + [pylint.messages control] # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this
Signature · 1 line
2.17.1
PATCH v3 13/13 python/aqmp-tui: Allow copying message from TUI
In reply to G S Niteesh BabuThis commit adds a feature that enables use to copy messages from the TUI after highlighting the message in the history box using up/down arrow keys and pressing alt-c.
--- python/qemu/aqmp/aqmp_tui.py | 9 +++++++++ 1 file changed, 9 insertions(+)
python/qemu/aqmp/aqmp_tui.py +9 -0
@@ -21,6 +21,7 @@ from pygments import lexers from pygments import token as Token +import pyperclip import urwid import urwid_readline
@@ -390,6 +391,14 @@ def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: self._update_highlighting() self.change_focus(size, self.highlighting) return None + if key == 'meta c': + if self.highlighting == -1: + return None + widget = self.history[self.highlighting].original_widget + text = widget.get_text()[0] + LOGGER.info('Text is %s', text) + pyperclip.copy(text) + return None # Remove highlighting if someother key is pressed if self.highlighting != -1:
Signature · 1 line
2.17.1