qemu_iotests: improve debugging options
v6 cover qemu_iotests: improve debugging options
This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.
Patches 1-9 introduce the -gdb option to both python and bash tests, 10-14 extend the already existing -valgrind flag to work also on python tests, and patch 15-16 introduces -p to enable logging to stdout.
In particular, patches 1,6,11 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.
---
v6:
* undo the previous series change "base this serie on the double dash options, so define --gdb instead of -gdb"
* undo Vladimir's suggestion on patch 5 to use @contextmanager, because it produces a pylint warning.
Emanuele Giuseppe Esposito (15):
python: qemu: add timer parameter for qmp.accept socket
python: qemu: pass the wrapper field from QEMUQtestmachine to
QEMUMachine
docs/devel/testing: add debug section to the QEMU iotests chapter
qemu-iotests: add option to attach gdbserver
qemu-iotests: delay QMP socket timers
qemu_iotests: insert gdbserver command line as wrapper for qemu binary
qemu-iotests: add gdbserver option to script tests too docs/devel/testing: add -gdb option to the debugging section of QEMU
iotests
qemu-iotests: extend the check script to prepare supporting valgrind
for python tests
qemu-iotests: extend QMP socket timeout when using valgrind
qemu-iotests: allow valgrind to read/delete the generated log file
qemu-iotests: insert valgrind command line as wrapper for qemu binary docs/devel/testing: add -valgrind option to the debug section of QEMU
iotests
qemu-iotests: add option to show qemu binary logs on stdout docs/devel/testing: add -p option to the debug section of QEMU iotests
John Snow (1):
python: Reduce strictness of pylint's duplicate-code check
docs/devel/testing.rst | 30 +++++++++++++++++++++ python/qemu/machine/machine.py | 16 +++++++---- python/qemu/machine/qtest.py | 9 ++++--- python/setup.cfg | 5 ++++ tests/qemu-iotests/check | 15 ++++++++--- tests/qemu-iotests/common.rc | 8 +++++- tests/qemu-iotests/iotests.py | 49 ++++++++++++++++++++++++++++++++-- tests/qemu-iotests/testenv.py | 23 ++++++++++++++-- 8 files changed, 138 insertions(+), 17 deletions(-)
Signature · 1 line
2.31.1
PATCH v6 1/16 python: qemu: add timer parameter for qmp.accept socket
Also add a new _qmp_timer field to the QEMUMachine class.
Let's change the default socket timeout to None, so that if a subclass needs to add a timer, it can be done by modifying this private field.
At the same time, restore the timer to be 15 seconds in iotests.py, to give an upper bound to the QMP monitor test command execution.
--- python/qemu/machine/machine.py | 7 +++++-- python/qemu/machine/qtest.py | 5 +++-- tests/qemu-iotests/iotests.py | 3 ++- 3 files changed, 10 insertions(+), 5 deletions(-)
python/qemu/machine/machine.py +5 -2
@@ -95,7 +95,8 @@ def __init__(self, socket_scm_helper: Optional[str] = None, sock_dir: Optional[str] = None, drain_console: bool = False, - console_log: Optional[str] = None): + console_log: Optional[str] = None, + qmp_timer: Optional[float] = None): ''' Initialize a QEMUMachine
@@ -109,6 +110,7 @@ def __init__(self, @param sock_dir: where to create socket (defaults to base_temp_dir) @param drain_console: (optional) True to drain console socket to buffer @param console_log: (optional) path to console log file + @param qmp_timer: (optional) default QMP socket timeout @note: Qemu process is not started until launch() is used. ''' # Direct user configuration
@@ -116,6 +118,7 @@ def __init__(self, self._binary = binary self._args = list(args) self._wrapper = wrapper + self._qmp_timer = qmp_timer self._name = name or "qemu-%d" % os.getpid() self._base_temp_dir = base_temp_dir
@@ -333,7 +336,7 @@ def _pre_launch(self) -> None: def _post_launch(self) -> None: if self._qmp_connection: - self._qmp.accept() + self._qmp.accept(self._qmp_timer) def _post_shutdown(self) -> None: """
python/qemu/machine/qtest.py +3 -2
@@ -115,14 +115,15 @@ def __init__(self, name: Optional[str] = None, base_temp_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None, - sock_dir: Optional[str] = None): + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): if name is None: name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = base_temp_dir super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir, socket_scm_helper=socket_scm_helper, - sock_dir=sock_dir) + sock_dir=sock_dir, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock")
tests/qemu-iotests/iotests.py +2 -1
@@ -570,10 +570,11 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) + timer = 15.0 super().__init__(qemu_prog, qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, - sock_dir=sock_dir) + sock_dir=sock_dir, qmp_timer=timer) self._num_drives = 0 def add_object(self, opts):
Signature · 1 line
2.31.1
Re: [PATCH v6 01/16] python: qemu: add timer parameter for qmp.accept socket
Quoted 11 lines · Emanuele Giuseppe Esposito
Also add a new
_qmp_timerfield to the QEMUMachine class.Let's change the default socket timeout to None, so that if a subclass needs to add a timer, it can be done by modifying this private field.
At the same time, restore the timer to be 15 seconds in
iotests.py, to give an upper bound to the QMP monitor test command execution.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>
LGTM, thank you for your patience while I re-settled the Python subcomponent.
and, for patches 1-3; I am happy with these being staged by a block subsystem maintainer:
--js
Quoted 85 lines
--- python/qemu/machine/machine.py | 7 +++++-- python/qemu/machine/qtest.py | 5 +++-- tests/qemu-iotests/iotests.py | 3 ++- 3 files changed, 10 insertions(+), 5 deletions(-)python/qemu/machine/machine.py +5
-2@@ -95,7 +95,8 @@ def __init__(self, socket_scm_helper: Optional[str] = None, sock_dir: Optional[str] = None, drain_console: bool = False, - console_log: Optional[str] = None): + console_log: Optional[str] = None, + qmp_timer: Optional[float] = None): ''' Initialize a QEMUMachine@@ -109,6 +110,7 @@ def __init__(self, @param sock_dir: where to create socket (defaults to base_temp_dir) @param drain_console: (optional) True to drain console socket to buffer @param console_log: (optional) path to console log file + @param qmp_timer: (optional) default QMP socket timeout @note: Qemu process is not started until launch() is used. ''' # Direct user configuration@@ -116,6 +118,7 @@ def __init__(self, self._binary = binary self._args = list(args) self._wrapper = wrapper + self._qmp_timer = qmp_timer self._name = name or "qemu-%d" % os.getpid() self._base_temp_dir = base_temp_dir@@ -333,7 +336,7 @@ def _pre_launch(self) -> None: def _post_launch(self) -> None: if self._qmp_connection: - self._qmp.accept() + self._qmp.accept(self._qmp_timer) def _post_shutdown(self) -> None: """python/qemu/machine/qtest.py +3
-2@@ -115,14 +115,15 @@ def __init__(self, name: Optional[str] = None, base_temp_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None, - sock_dir: Optional[str] = None): + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): if name is None: name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = base_temp_dir super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir, socket_scm_helper=socket_scm_helper, - sock_dir=sock_dir) + sock_dir=sock_dir, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock")tests/qemu-iotests/iotests.py +2
-1@@ -570,10 +570,11 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) + timer = 15.0 super().__init__(qemu_prog, qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, - sock_dir=sock_dir) + sock_dir=sock_dir, qmp_timer=timer) self._num_drives = 0 def add_object(self, opts):
PATCH v6 2/16 python: Reduce strictness of pylint's duplicate-code check
In reply to Emanuele Giuseppe EspositoFrom: John Snow <[email protected]>
Pylint prior to 2.8.3 (We pin at >= 2.8.0) includes function and method signatures as part of its duplicate checking algorithm. This check does not listen to pragmas, so the only way to disable it is to turn it off completely or increase the minimum duplicate lines so that it doesn't trigger for functions with long, multi-line signatures.
When we decide to upgrade to pylint 2.8.3 or greater, we will be able to use 'ignore-signatures = true' to the config instead.
I'd prefer not to keep us on the very bleeding edge of pylint if I can help it -- 2.8.3 came out only three days ago at time of writing.
See: https://github.com/PyCQA/pylint/pull/4474
--- python/setup.cfg | 5 +++++ 1 file changed, 5 insertions(+)
python/setup.cfg +5 -0
@@ -78,6 +78,11 @@ good-names=i, # Ignore imports when computing similarities. ignore-imports=yes +# Minimum lines number of a similarity. +# TODO: Remove after we opt in to Pylint 2.8.3. See commit msg. +min-similarity-lines=6 + + [isort] force_grid_wrap=4 force_sort_within_sections=True
Signature · 1 line
2.31.1
PATCH v6 3/16 python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine
In reply to Emanuele Giuseppe Esposito--- python/qemu/machine/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
python/qemu/machine/qtest.py +3 -1
@@ -112,6 +112,7 @@ class QEMUQtestMachine(QEMUMachine): def __init__(self, binary: str, args: Sequence[str] = (), + wrapper: Sequence[str] = (), name: Optional[str] = None, base_temp_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None,
@@ -121,7 +122,8 @@ def __init__(self, name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = base_temp_dir - super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir, + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None
Signature · 1 line
2.31.1
PATCH v6 4/16 docs/devel/testing: add debug section to the QEMU iotests chapter
In reply to Emanuele Giuseppe EspositoIntroduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches.
--- docs/devel/testing.rst | 8 ++++++++ 1 file changed, 8 insertions(+)
docs/devel/testing.rst +8 -0
@@ -224,6 +224,14 @@ another application on the host may have locked the file, possibly leading to a test failure. If using such devices are explicitly desired, consider adding ``locking=off`` option to disable image locking. +Debugging a test case +----------------------- +The following options to the ``check`` script can be useful when debugging +a failing test: + +* ``-d`` (debug) just increases the logging verbosity, showing + for example the QMP commands and answers. + Test case groups ----------------
Signature · 1 line
2.31.1
PATCH v6 5/16 qemu-iotests: add option to attach gdbserver
In reply to Emanuele Giuseppe EspositoDefine -gdb flag and GDB_OPTIONS environment variable to python tests to attach a gdbserver to each qemu instance. This patch only adds and parses this flag, it does not yet add the implementation for it.
if -gdb is not provided but $GDB_OPTIONS is set, ignore the environment variable.
--- tests/qemu-iotests/check | 6 +++++- tests/qemu-iotests/iotests.py | 5 +++++ tests/qemu-iotests/testenv.py | 17 +++++++++++++++-- 3 files changed, 25 insertions(+), 3 deletions(-)
tests/qemu-iotests/check +5 -1
@@ -36,6 +36,9 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-gdb', action='store_true', + help="start gdbserver with $GDB_OPTIONS options \ + ('localhost:12345' if $GDB_OPTIONS is empty)") p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -114,7 +117,8 @@ if __name__ == '__main__': env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto, aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, - debug=args.debug, valgrind=args.valgrind) + debug=args.debug, valgrind=args.valgrind, + gdb=args.gdb) if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--': if not args.tests:
tests/qemu-iotests/iotests.py +5 -0
@@ -74,6 +74,11 @@ qemu_prog = os.environ.get('QEMU_PROG', 'qemu') qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') +gdb_qemu_env = os.environ.get('GDB_OPTIONS') +qemu_gdb = [] +if gdb_qemu_env: + qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' ') + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.')
tests/qemu-iotests/testenv.py +15 -2
@@ -27,6 +27,7 @@ import glob from typing import List, Dict, Any, Optional, ContextManager +DEF_GDB_OPTIONS = 'localhost:12345' def isxfile(path: str) -> bool: return os.path.isfile(path) and os.access(path, os.X_OK)
@@ -72,7 +73,8 @@ class TestEnv(ContextManager['TestEnv']): 'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', - 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_'] + 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', + 'GDB_OPTIONS'] def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: if self.debug:
@@ -178,7 +180,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, imgopts: Optional[str] = None, misalign: bool = False, debug: bool = False, - valgrind: bool = False) -> None: + valgrind: bool = False, + gdb: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode
@@ -186,6 +189,15 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if gdb: + self.gdb_options = os.getenv('GDB_OPTIONS', DEF_GDB_OPTIONS) + if not self.gdb_options: + # cover the case 'export GDB_OPTIONS=' + self.gdb_options = DEF_GDB_OPTIONS + elif 'GDB_OPTIONS' in os.environ: + # to not propagate it in prepare_subprocess() + del os.environ['GDB_OPTIONS'] + if valgrind: self.valgrind_qemu = 'y'
@@ -285,6 +297,7 @@ def print_env(self) -> None: TEST_DIR -- {TEST_DIR} SOCK_DIR -- {SOCK_DIR} SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} +GDB_OPTIONS -- {GDB_OPTIONS} """ args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.31.1
PATCH v6 6/16 qemu-iotests: delay QMP socket timers
In reply to Emanuele Giuseppe EspositoAttaching gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.
--- tests/qemu-iotests/iotests.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +5 -1
@@ -477,10 +477,14 @@ def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg def __enter__(self): + if qemu_gdb: + return self signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self def __exit__(self, exc_type, value, traceback): + if qemu_gdb: + return False signal.setitimer(signal.ITIMER_REAL, 0) return False def timeout(self, signum, frame):
@@ -575,7 +579,7 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - timer = 15.0 + timer = 15.0 if not qemu_gdb else None super().__init__(qemu_prog, qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper,
Signature · 1 line
2.31.1
Re: [PATCH v6 06/16] qemu-iotests: delay QMP socket timers
Quoted 5 lines · Emanuele Giuseppe Esposito
Attaching gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>
Please, let this patch go without my r-b. I don't like it, I explained my thoughts, and you even used my suggested way in previous version.
What I don't like: good component - Timeout class, which is currently independent of outer code and can be simply moved to another module becomes dependent on global variable, which doesn't relate to the class itself. Neither I like logic of the dependency which just make the whole class do nothing by skipping any action internally. So, IMHO Timeout class becomes worse.
I'm not a maintainer here anyway, so my r-b isn't necessary :)
Quoted 33 lines
--- tests/qemu-iotests/iotests.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)tests/qemu-iotests/iotests.py +5
-1@@ -477,10 +477,14 @@ def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg def __enter__(self): + if qemu_gdb: + return self signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self def __exit__(self, exc_type, value, traceback): + if qemu_gdb: + return False signal.setitimer(signal.ITIMER_REAL, 0) return False def timeout(self, signum, frame):@@ -575,7 +579,7 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - timer = 15.0 + timer = 15.0 if not qemu_gdb else None super().__init__(qemu_prog, qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper,
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH v6 06/16] qemu-iotests: delay QMP socket timers
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
Quoted 3 lines · Emanuele Giuseppe Esposito
Attaching gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>Please, let this patch go without my r-b. I don't like it, I explained my thoughts, and you even used my suggested way in previous version.
What I don't like: good component - Timeout class, which is currently independent of outer code and can be simply moved to another module becomes dependent on global variable, which doesn't relate to the class itself. Neither I like logic of the dependency which just make the whole class do nothing by skipping any action internally. So, IMHO Timeout class becomes worse.
I'm not a maintainer here anyway, so my r-b isn't necessary :)
Oh okay. Sorry I understood from previous versions that you liked it even without your suggested change. Apologies.
Emanuele
Quoted 35 lines
Quoted 33 lines
---
tests/qemu-iotests/iotests.py| 6 +++++-1 file changed, 5 insertions(+), 1 deletion(-)tests/qemu-iotests/iotests.py +0
-0@@ -477,10 +477,14 @@ def __init__(self, seconds, errmsg="Timeout"):self.seconds = seconds
self.errmsg = errmsg
def__enter__(self):
+ ifqemu_gdb:
+ return selfsignal.signal(signal.SIGALRM, self.timeout)signal.setitimer(signal.ITIMER_REAL, self.seconds)return self
def__exit__(self, exc_type, value, traceback):
+ ifqemu_gdb:
+ return Falsesignal.setitimer(signal.ITIMER_REAL, 0)return False
deftimeout(self, signum, frame):tests/qemu-iotests/iotests.py +0
-0@@ -575,7 +579,7 @@ class VM(qtest.QEMUQtestMachine):def
__init__(self, path_suffix=''):
name = "qemu%s-%d" % (path_suffix,os.getpid()) - timer = 15.0
+ timer = 15.0 if notqemu_gdbelse None super().__init__(qemu_prog,qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper,
Re: [PATCH v6 06/16] qemu-iotests: delay QMP socket timers
Quoted 10 lines · Emanuele Giuseppe Esposito
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Quoted 5 lines · Emanuele Giuseppe Esposito
Attaching gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>
Quoted 8 lines
Quoted 5 lines
Please, let this patch go without my r-b. I don't like it, I explained my thoughts, and you even used my suggested way in previous version.
What I don't like: good component - Timeout class, which is currently independent of outer code and can be simply moved to another module becomes dependent on global variable, which doesn't relate to the class itself. Neither I like logic of the dependency which just make the whole class do nothing by skipping any action internally. So, IMHO Timeout class becomes worse.
I'm not a maintainer here anyway, so my r-b isn't necessary :)
Oh okay. Sorry I understood from previous versions that you liked it even without your suggested change. Apologies.
No problem. I really gave an r-b to earlier version, so formally you had it until now :) I thought I was explicit enough in my last comment to this patch in v5. apologies if not(
Signature · 2 lines
Best regards, Vladimir
PATCH v6 7/16 qemu_iotests: insert gdbserver command line as wrapper for qemu binary
In reply to Emanuele Giuseppe Esposito--- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
@@ -580,7 +580,8 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) timer = 15.0 if not qemu_gdb else None - super().__init__(qemu_prog, qemu_opts, name=name, + super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, + name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir, qmp_timer=timer)
Signature · 1 line
2.31.1
PATCH v6 8/16 qemu-iotests: add gdbserver option to script tests too
In reply to Emanuele Giuseppe EspositoThe only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail.
--- tests/qemu-iotests/common.rc | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
tests/qemu-iotests/common.rc +7 -1
@@ -166,8 +166,14 @@ _qemu_wrapper() if [ -n "${QEMU_NEED_PID}" ]; then echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid" fi + + GDB="" + if [ ! -z ${GDB_OPTIONS} ]; then + GDB="gdbserver ${GDB_OPTIONS}" + fi + VALGRIND_QEMU="${VALGRIND_QEMU_VM}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \ - "$QEMU_PROG" $QEMU_OPTIONS "$@" + $GDB "$QEMU_PROG" $QEMU_OPTIONS "$@" ) RETVAL=$? _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL
Signature · 1 line
2.31.1
PATCH v6 9/16 docs/devel/testing: add -gdb option to the debugging section of QEMU iotests
In reply to Emanuele Giuseppe Esposito--- docs/devel/testing.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)
docs/devel/testing.rst +11 -0
@@ -229,6 +229,17 @@ Debugging a test case The following options to the ``check`` script can be useful when debugging a failing test: +* ``-gdb`` wraps every QEMU invocation in a ``gdbserver``, which waits for a + connection from a gdb client. The options given to ``gdbserver`` (e.g. the + address on which to listen for connections) are taken from the ``$GDB_OPTIONS`` + environment variable. By default (if ``$GDB_OPTIONS`` is empty), it listens on + ``localhost:12345``. + It is possible to connect to it for example with + ``gdb -iex "target remote $addr"``, where ``$addr`` is the address + ``gdbserver`` listens on. + If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored, + regardless on whether it is set or not. + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers.
Signature · 1 line
2.31.1
PATCH v6 10/16 qemu-iotests: extend the check script to prepare supporting valgrind for python tests
In reply to Emanuele Giuseppe EspositoCurrently, the check script only parses the option and sets the VALGRIND_QEMU environmental variable to "y".
Add another local python variable that prepares the command line, identical to the one provided in the test scripts.
Because the python script does not know in advance the valgrind PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime.
--- tests/qemu-iotests/check | 7 ++++--- tests/qemu-iotests/iotests.py | 11 +++++++++++ tests/qemu-iotests/testenv.py | 1 + 3 files changed, 16 insertions(+), 3 deletions(-)
tests/qemu-iotests/check +4 -3
@@ -39,6 +39,10 @@ def make_argparser() -> argparse.ArgumentParser: p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_OPTIONS options \ ('localhost:12345' if $GDB_OPTIONS is empty)") + p.add_argument('-valgrind', action='store_true', + help='use valgrind, sets VALGRIND_QEMU environment ' + 'variable') + p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -88,9 +92,6 @@ def make_argparser() -> argparse.ArgumentParser: g_bash.add_argument('-o', dest='imgopts', help='options to pass to qemu-img create/convert, ' 'sets IMGOPTS environment variable') - g_bash.add_argument('-valgrind', action='store_true', - help='use valgrind, sets VALGRIND_QEMU environment ' - 'variable') g_sel = p.add_argument_group('test selecting options', 'The following options specify test set '
tests/qemu-iotests/iotests.py +11 -0
@@ -96,6 +96,17 @@ sys.stderr.write('Please run this test via the "check" script\n') sys.exit(os.EX_USAGE) +qemu_valgrind = [] +if os.environ.get('VALGRIND_QEMU') == "y" and \ + os.environ.get('NO_VALGRIND') != "y": + valgrind_logfile = "--log-file=" + test_dir + # %p allows to put the valgrind process PID, since + # we don't know it a priori (subprocess.Popen is + # not yet invoked) + valgrind_logfile += "/%p.valgrind" + + qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99'] + socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \
tests/qemu-iotests/testenv.py +1 -0
@@ -298,6 +298,7 @@ def print_env(self) -> None: SOCK_DIR -- {SOCK_DIR} SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_OPTIONS -- {GDB_OPTIONS} +VALGRIND_QEMU -- {VALGRIND_QEMU} """ args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.31.1
PATCH v6 11/16 qemu-iotests: extend QMP socket timeout when using valgrind
In reply to Emanuele Giuseppe EspositoAs with gdbserver, valgrind delays the test execution, so the default QMP socket timeout and the generic class Timeout in iotests.py timeouts too soon.
--- tests/qemu-iotests/iotests.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/iotests.py +3 -3
@@ -488,13 +488,13 @@ def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg def __enter__(self): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: return self signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self def __exit__(self, exc_type, value, traceback): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: return False signal.setitimer(signal.ITIMER_REAL, 0) return False
@@ -590,7 +590,7 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - timer = 15.0 if not qemu_gdb else None + timer = 15.0 if not (qemu_gdb or qemu_valgrind) else None super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, name=name, base_temp_dir=test_dir,
Signature · 1 line
2.31.1
PATCH v6 12/16 qemu-iotests: allow valgrind to read/delete the generated log file
In reply to Emanuele Giuseppe EspositoWhen using -valgrind on the script tests, it generates a log file in $TEST_DIR that is either read (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests.
--- tests/qemu-iotests/iotests.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
tests/qemu-iotests/iotests.py +11 -0
@@ -598,6 +598,17 @@ def __init__(self, path_suffix=''): sock_dir=sock_dir, qmp_timer=timer) self._num_drives = 0 + def _post_shutdown(self) -> None: + super()._post_shutdown() + if not qemu_valgrind or not self._popen: + return + valgrind_filename = f"{test_dir}/{self._popen.pid}.valgrind" + if self.exitcode() == 99: + with open(valgrind_filename) as f: + print(f.read()) + else: + os.remove(valgrind_filename) + def add_object(self, opts): self._args.append('-object') self._args.append(opts)
Signature · 1 line
2.31.1
PATCH v6 13/16 qemu-iotests: insert valgrind command line as wrapper for qemu binary
In reply to Emanuele Giuseppe EspositoIf -gdb and -valgrind are both defined, return an error.
--- tests/qemu-iotests/iotests.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +5 -1
@@ -591,7 +591,11 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) timer = 15.0 if not (qemu_gdb or qemu_valgrind) else None - super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, + if qemu_gdb and qemu_valgrind: + sys.stderr.write('Either use gdb or valgrind, not together\n') + sys.exit(1) + wrapper = qemu_gdb if qemu_gdb else qemu_valgrind + super().__init__(qemu_prog, qemu_opts, wrapper=wrapper, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper,
Signature · 1 line
2.31.1
PATCH v6 14/16 docs/devel/testing: add -valgrind option to the debug section of QEMU iotests
In reply to Emanuele Giuseppe Esposito--- docs/devel/testing.rst | 7 +++++++ 1 file changed, 7 insertions(+)
docs/devel/testing.rst +7 -0
@@ -240,6 +240,13 @@ a failing test: If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored, regardless on whether it is set or not. +* ``-valgrind`` attaches a valgrind instance to QEMU. If it detects + warnings, it will print and save the log in + ``$TEST_DIR/<valgrind_pid>.valgrind``. + The final command line will be ``valgrind --log-file=$TEST_DIR/ + <valgrind_pid>.valgrind --error-exitcode=99 $QEMU ...`` + Note: if used together with ``-gdb``, this command will be ignored. + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers.
Signature · 1 line
2.31.1
PATCH v6 15/16 qemu-iotests: add option to show qemu binary logs on stdout
In reply to Emanuele Giuseppe EspositoUsing the flag -p, allow the qemu binary to print to stdout.
Also create the common function _close_qemu_log_file() to avoid accessing machine.py private fields directly and have duplicate code.
--- python/qemu/machine/machine.py | 9 ++++++--- tests/qemu-iotests/check | 4 +++- tests/qemu-iotests/iotests.py | 8 ++++++++ tests/qemu-iotests/testenv.py | 9 +++++++-- 4 files changed, 24 insertions(+), 6 deletions(-)
python/qemu/machine/machine.py +6 -3
@@ -338,6 +338,11 @@ def _post_launch(self) -> None: if self._qmp_connection: self._qmp.accept(self._qmp_timer) + def _close_qemu_log_file(self) -> None: + if self._qemu_log_file is not None: + self._qemu_log_file.close() + self._qemu_log_file = None + def _post_shutdown(self) -> None: """ Called to cleanup the VM instance after the process has exited.
@@ -350,9 +355,7 @@ def _post_shutdown(self) -> None: self._qmp.close() self._qmp_connection = None - if self._qemu_log_file is not None: - self._qemu_log_file.close() - self._qemu_log_file = None + self._close_qemu_log_file() self._load_io_log()
tests/qemu-iotests/check +3 -1
@@ -36,6 +36,8 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-p', dest='print', action='store_true', + help='redirects qemu\'s stdout and stderr to the test output') p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_OPTIONS options \ ('localhost:12345' if $GDB_OPTIONS is empty)")
@@ -119,7 +121,7 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb) + gdb=args.gdb, qprint=args.print) if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--': if not args.tests:
tests/qemu-iotests/iotests.py +8 -0
@@ -79,6 +79,8 @@ if gdb_qemu_env: qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' ') +qemu_print = os.environ.get('PRINT_QEMU', False) + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.')
@@ -613,6 +615,12 @@ def _post_shutdown(self) -> None: else: os.remove(valgrind_filename) + def _pre_launch(self) -> None: + super()._pre_launch() + if qemu_print: + # set QEMU binary output to stdout + self._close_qemu_log_file() + def add_object(self, opts): self._args.append('-object') self._args.append(opts)
tests/qemu-iotests/testenv.py +7 -2
@@ -74,7 +74,7 @@ class TestEnv(ContextManager['TestEnv']): 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', - 'GDB_OPTIONS'] + 'GDB_OPTIONS', 'PRINT_QEMU'] def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: if self.debug:
@@ -181,7 +181,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, misalign: bool = False, debug: bool = False, valgrind: bool = False, - gdb: bool = False) -> None: + gdb: bool = False, + qprint: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode
@@ -189,6 +190,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if qprint: + self.print_qemu = 'y' + if gdb: self.gdb_options = os.getenv('GDB_OPTIONS', DEF_GDB_OPTIONS) if not self.gdb_options:
@@ -299,6 +303,7 @@ def print_env(self) -> None: SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_OPTIONS -- {GDB_OPTIONS} VALGRIND_QEMU -- {VALGRIND_QEMU} +PRINT_QEMU_OUTPUT -- {PRINT_QEMU} """ args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.31.1
PATCH v6 16/16 docs/devel/testing: add -p option to the debug section of QEMU iotests
In reply to Emanuele Giuseppe Esposito--- docs/devel/testing.rst | 4 ++++ 1 file changed, 4 insertions(+)
docs/devel/testing.rst +4 -0
@@ -250,6 +250,10 @@ a failing test: * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers. +* ``-p`` (print) redirect QEMU’s stdout and stderr to the test output, + instead of saving it into a log file in + ``$TEST_DIR/qemu-machine-<random_string>``. + Test case groups ----------------
Signature · 1 line
2.31.1