qemu_iotests: improve debugging options
v3 cover qemu_iotests: improve debugging options
This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.
Patches 1-6 introduce the -gdb option to both python and bash tests, 7-10 extend the already existing -valgrind flag to work also on python tests, and patch 11 introduces -p to enable logging to stdout.
In particular, patches 1,2,4,8 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.
This series is tested on the previous serie "qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.
---
v3:
- Introduce the class field _qmp_timer instead of a function parameter in the _post_launch() function [John]
- style and cleanup fixes in iotests.py [Paolo]
Emanuele Giuseppe Esposito (15):
python: qemu: add timer parameter for qmp.accept socket
python: qemu: pass the wrapper field from QEMUQtestmachine to
QEMUMachine
docs/devel/testing: add debug section to the QEMU iotests chapter
qemu-iotests: add option to attach gdbserver
qemu-iotests: delay QMP socket timers
qemu_iotests: insert gdbserver command line as wrapper for qemu binary
qemu-iotests: add gdbserver option to script tests too docs/devel/testing: add -gdb option to the debugging section of QEMU
iotests
qemu_iotests: extend the check script to support valgrind for python
tests
qemu_iotests: extent QMP socket timeout when using valgrind
qemu_iotests: allow valgrind to read/delete the generated log file
qemu_iotests: insert valgrind command line as wrapper for qemu binary docs/devel/testing: add -valgrind option to the debug section of QEMU
iotests
qemu_iotests: add option to show qemu binary logs on stdout docs/devel/testing: add -p option to the debug section of QEMU iotests
docs/devel/testing.rst | 26 ++++++++++++++++ python/qemu/machine.py | 6 +++- python/qemu/qtest.py | 4 ++- tests/qemu-iotests/check | 14 ++++++--- tests/qemu-iotests/common.rc | 8 ++++- tests/qemu-iotests/iotests.py | 58 +++++++++++++++++++++++++++++++++-- tests/qemu-iotests/testenv.py | 21 +++++++++++-- 7 files changed, 125 insertions(+), 12 deletions(-)
Signature · 1 line
2.30.2
PATCH v3 1/15 python: qemu: add timer parameter for qmp.accept socket
Add a new _qmp_timer field to the QEMUMachine class. The default timer is 15 sec, as per the default in the qmp accept() function.
--- python/qemu/machine.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
python/qemu/machine.py +2 -1
@@ -110,6 +110,7 @@ def __init__(self, self._binary = binary self._args = list(args) self._wrapper = wrapper + self._qmp_timer = 15.0 self._name = name or "qemu-%d" % os.getpid() self._test_dir = test_dir
@@ -323,7 +324,7 @@ def _pre_launch(self) -> None: def _post_launch(self) -> None: if self._qmp_connection: - self._qmp.accept() + self._qmp.accept(self._qmp_timer) def _post_shutdown(self) -> None: """
Signature · 1 line
2.30.2
Re: [PATCH v3 01/15] python: qemu: add timer parameter for qmp.accept socket
Quoted 8 lines · Emanuele Giuseppe Esposito
Add a new
_qmp_timerfield to the QEMUMachine class. The default timer is 15 sec, as per the default in the qmpaccept()function.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- python/qemu/machine.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
(I would have preferred for the commit message to tell me why we want this (I suspect some later patch is going to use it), but oh well.)
Re: [PATCH v3 01/15] python: qemu: add timer parameter for qmp.accept socket
In reply to Emanuele Giuseppe EspositoQuoted 3 lines · Emanuele Giuseppe Esposito
Add a new
_qmp_timerfield to the QEMUMachine class. The default timer is 15 sec, as per the default in the qmpaccept()function.
Fine enough for now.
What's the exact need for this change, exactly? Is it just to prevent any unbounded blocking waits? I assume this came up in development or you'd not have added it.
Quoted 28 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- python/qemu/machine.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)python/qemu/machine.py +2
-1@@ -110,6 +110,7 @@ def __init__(self, self._binary = binary self._args = list(args) self._wrapper = wrapper + self._qmp_timer = 15.0 self._name = name or "qemu-%d" % os.getpid() self._test_dir = test_dir@@ -323,7 +324,7 @@ def _pre_launch(self) -> None: def _post_launch(self) -> None: if self._qmp_connection: - self._qmp.accept() + self._qmp.accept(self._qmp_timer) def _post_shutdown(self) -> None: """
Re: [PATCH v3 01/15] python: qemu: add timer parameter for qmp.accept socket
Quoted 8 lines · John Snow
Quoted 3 lines · Emanuele Giuseppe Esposito
Add a new
_qmp_timerfield to the QEMUMachine class. The default timer is 15 sec, as per the default in the qmpaccept()function.Fine enough for now.
What's the exact need for this change, exactly? Is it just to prevent any unbounded blocking waits? I assume this came up in development or you'd not have added it.
The default was already 15 sec, but since we now want to make the wait unbounded if we use gdbserver or valgrind and we do it by always passing the _qmp_timer to the .accept function, we need to set a default value. Yes, it came up while testing with gdb.
Quoted 29 lines
Reviewed-by John Snow <[email protected]>Quoted 25 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py| 3 ++-1 file changed, 2 insertions(+), 1 deletion(-)python/qemu/machine.py +0
-0@@ -110,6 +110,7 @@ def __init__(self,self._binary = binary
self._args = list(args) self._wrapper = wrapper
+ self._qmp_timer = 15.0
self._name = name or "qemu-%d" %os.getpid()self._test_dir =test_dirpython/qemu/machine.py +0
-0@@ -323,7 +324,7 @@ def _pre_launch(self) -> None:def
_post_launch(self)-> None:
if self._qmp_connection:
-self._qmp.accept()
+self._qmp.accept(self._qmp_timer)
def_post_shutdown(self)-> None:
"""
PATCH v3 2/15 python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine
In reply to Emanuele Giuseppe Esposito--- python/qemu/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
python/qemu/qtest.py +3 -1
@@ -111,6 +111,7 @@ class QEMUQtestMachine(QEMUMachine): def __init__(self, binary: str, args: Sequence[str] = (), + wrapper: Sequence[str] = (), name: Optional[str] = None, test_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None,
@@ -119,7 +120,8 @@ def __init__(self, name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = test_dir - super().__init__(binary, args, name=name, test_dir=test_dir, + super().__init__(binary, args, wrapper=wrapper, name=name, + test_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir) self._qtest: Optional[QEMUQtestProtocol] = None
Signature · 1 line
2.30.2
Re: [PATCH v3 02/15] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine
Quoted 18 lines · Emanuele Giuseppe Esposito
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- python/qemu/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)python/qemu/qtest.py +1
-0@@ -111,6 +111,7 @@ class QEMUQtestMachine(QEMUMachine): def __init__(self, binary: str, args: Sequence[str] = (), + wrapper: Sequence[str] = (), name: Optional[str] = None, test_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None,
python/qemu/qtest.py def __init__(self, · line 127
Quoted 9 lines
python/qemu/qtest.py +2
-1@@ -119,7 +120,8 @@ def __init__(self, name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = test_dir - super().__init__(binary, args, name=name, test_dir=test_dir, + super().__init__(binary, args, wrapper=wrapper, name=name, + test_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir) self._qtest: Optional[QEMUQtestProtocol] = None
Re: [PATCH v3 02/15] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine
In reply to Emanuele Giuseppe EspositoQuoted 1 line · Emanuele Giuseppe Esposito
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>
Assuming it doesn't make the linter explode, which I didn't run:
Quoted 27 lines
--- python/qemu/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)python/qemu/qtest.py +3
-1@@ -111,6 +111,7 @@ class QEMUQtestMachine(QEMUMachine): def __init__(self, binary: str, args: Sequence[str] = (), + wrapper: Sequence[str] = (), name: Optional[str] = None, test_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None,@@ -119,7 +120,8 @@ def __init__(self, name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = test_dir - super().__init__(binary, args, name=name, test_dir=test_dir, + super().__init__(binary, args, wrapper=wrapper, name=name, + test_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir) self._qtest: Optional[QEMUQtestProtocol] = None
PATCH v3 3/15 docs/devel/testing: add debug section to the QEMU iotests chapter
In reply to Emanuele Giuseppe EspositoIntroduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches.
--- docs/devel/testing.rst | 8 ++++++++ 1 file changed, 8 insertions(+)
docs/devel/testing.rst +8 -0
@@ -224,6 +224,14 @@ another application on the host may have locked the file, possibly leading to a test failure. If using such devices are explicitly desired, consider adding ``locking=off`` option to disable image locking. +Debugging a test case +----------------------- +QEMU iotests offers some options to debug a failing test, that can be +given as options to the ``check`` script: + +* ``-d`` (debug) just increases the logging verbosity, showing + for example the QMP commands and answers. + Test case groups ----------------
Signature · 1 line
2.30.2
Re: [PATCH v3 03/15] docs/devel/testing: add debug section to the QEMU iotests chapter
Quoted 13 lines · Emanuele Giuseppe Esposito
Introduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- docs/devel/testing.rst | 8 ++++++++ 1 file changed, 8 insertions(+)docs/devel/testing.rst +0
-0
docs/devel/testing.rst another application on the host may have locked the file, possibly leading to a · line 229
Quoted 6 lines
docs/devel/testing.rst +3
-0@@ -224,6 +224,14 @@ another application on the host may have locked the file, possibly leading to a test failure. If using such devices are explicitly desired, consider adding ``locking=off`` option to disable image locking. +Debugging a test case +----------------------- +QEMU iotests offers some options to debug a failing test, that can be
-,
Max
Quoted 9 lines
docs/devel/testing.rst +5
-0+given as options to the ``check`` script: + +* ``-d`` (debug) just increases the logging verbosity, showing + for example the QMP commands and answers. + Test case groups ----------------
Re: [PATCH v3 03/15] docs/devel/testing: add debug section to the QEMU iotests chapter
Quoted 6 lines · Max Reitz
docs/devel/testing.rst another application on the host may have locked the file, possibly leading to a · line 230
Quoted 4 lines
+-----------------------
docs/devel/testing.rst +2
-0+QEMU iotests offers some options to debug a failing test, that can be +given as options to the ``check`` script:-,
Even better: "The following options to the ``check`` script can be useful when debugging a failing test:".
Paolo
PATCH v3 4/15 qemu-iotests: add option to attach gdbserver
In reply to Emanuele Giuseppe EspositoAdd -gdb flag and GDB_QEMU environmental variable to python tests to attach a gdbserver to each qemu instance.
if -gdb is not provided but $GDB_QEMU is set, ignore the environmental variable.
--- tests/qemu-iotests/check | 6 +++++- tests/qemu-iotests/iotests.py | 4 ++++ tests/qemu-iotests/testenv.py | 15 ++++++++++++--- 3 files changed, 21 insertions(+), 4 deletions(-)
tests/qemu-iotests/check +5 -1
@@ -33,6 +33,9 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-gdb', action='store_true', + help="start gdbserver with $GDB_QEMU options. \ + Default is localhost:12345") p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -112,7 +115,8 @@ if __name__ == '__main__': env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto, aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, - debug=args.debug, valgrind=args.valgrind) + debug=args.debug, valgrind=args.valgrind, + gdb=args.gdb) testfinder = TestFinder(test_dir=env.source_iotests)
tests/qemu-iotests/iotests.py +4 -0
@@ -75,6 +75,10 @@ qemu_prog = os.environ.get('QEMU_PROG', 'qemu') qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') +qemu_gdb = [] +if os.environ.get('GDB_QEMU'): + qemu_gdb = ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ') + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.')
tests/qemu-iotests/testenv.py +12 -3
@@ -72,7 +72,8 @@ class TestEnv(ContextManager['TestEnv']): 'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', - 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_'] + 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', + 'GDB_QEMU'] def get_env(self) -> Dict[str, str]: env = {}
@@ -163,7 +164,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, imgopts: Optional[str] = None, misalign: bool = False, debug: bool = False, - valgrind: bool = False) -> None: + valgrind: bool = False, + gdb: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode
@@ -171,6 +173,11 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if gdb: + self.gdb_qemu = os.environ.get('GDB_QEMU', 'localhost:12345') + elif 'GDB_QEMU' in os.environ: + del os.environ['GDB_QEMU'] + if valgrind: self.valgrind_qemu = 'y'
@@ -268,7 +275,9 @@ def print_env(self) -> None: PLATFORM -- {platform} TEST_DIR -- {TEST_DIR} SOCK_DIR -- {SOCK_DIR} -SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}""" +SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} +GDB_QEMU -- "{GDB_QEMU}" +""" args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.30.2
Re: [PATCH v3 04/15] qemu-iotests: add option to attach gdbserver
Quoted 2 lines · Emanuele Giuseppe Esposito
Add -gdb flag and
GDB_QEMUenvironmental variable to python tests to attach a gdbserver to each qemu instance.
Well, this patch doesn’t do this, but OK.
Out of interest: Why gdbserver and not “just” gdb? On Fedora, those are separate packages, so I don’t have gdbserver installed, that’s why I’m asking.
(I’ve also never used gdbserver before. From what I can tell, it’s basically just a limited version of gdb so it only serves as a server.)
Quoted 2 lines
if -gdb is not provided but $GDB_QEMU is set, ignore the environmental variable.
s/environmental/environment/
Quoted 13 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/check | 6 +++++- tests/qemu-iotests/iotests.py | 4 ++++ tests/qemu-iotests/testenv.py | 15 ++++++++++++--- 3 files changed, 21 insertions(+), 4 deletions(-)tests/qemu-iotests/check +0
-0
tests/qemu-iotests/check def make_argparser() -> argparse.ArgumentParser: · line 38
Quoted 6 lines
tests/qemu-iotests/check +3
-0@@ -33,6 +33,9 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-gdb', action='store_true', + help="start gdbserver with $GDB_QEMU options. \ + Default is localhost:12345")
That makes it sound like this were the default for the -gdb option. Since -gdb is just a switch, it doesn’t have a default (apart from being off by default).
So I’d rephrase this at least to “The default options are 'localhost:12345'”. Or maybe “start gdbserver with $GDB_QEMU options ('localhost:12345' if $GDB_QEMU is empty)”.
Also, $GDB_QEMU as a name is a bit strange, because it does not specify which gdb to use; it just gives the options to use for gdb. $GDB_QEMU_OPTIONS would be more in line with the naming of the rest of the environment variables (or just $GDB_OPTIONS).
Max
Quoted 3 lines
tests/qemu-iotests/check +0
-0p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],
Re: [PATCH v3 04/15] qemu-iotests: add option to attach gdbserver
Quoted 5 lines · Max Reitz
Quoted 2 lines · Emanuele Giuseppe Esposito
Add -gdb flag and
GDB_QEMUenvironmental variable to python tests to attach a gdbserver to each qemu instance.Well, this patch doesn’t do this, but OK.
Maybe "define" rather than "add"? In the sense of defining the "-gdb" option, which is what it actually does.
Quoted 2 lines
Out of interest: Why gdbserver and not “just” gdb? On Fedora, those are separate packages, so I don’t have gdbserver installed, that’s why I’m asking.
As far as I have tried, using only gdb with ./check is very hard to use, because the stdout is filtered out by the script.
So invoking gdb attaches it to QEMU, but it is not possible to start execution (run command) or interact with it, because of the python script filtering. This leaves the test hanging.
gdbserver is just something that a gdb client can attach to (for example, in another console or even in another host) for example with the command
# gdb -iex "target remote localhost:12345" . This provides a nice and separate gdb monitor to the client.
Emanuele
Quoted 39 lines
(I’ve also never used gdbserver before. From what I can tell, it’s basically just a limited version of gdb so it only serves as a server.)
Quoted 2 lines
if -gdb is not provided but $GDB_QEMU is set, ignore the environmental variable.
s/environmental/environment/
Quoted 13 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
tests/qemu-iotests/check | 6 +++++-tests/qemu-iotests/iotests.py| 4 ++++tests/qemu-iotests/testenv.py| 15 ++++++++++++---3 files changed, 21 insertions(+), 4 deletions(-)tests/qemu-iotests/check +0
-0tests/qemu-iotests/check def make_argparser() -> argparse.ArgumentParser: · line 37
Quoted 5 lines
tests/qemu-iotests/check +3
-0@@ -33,6 +33,9 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-gdb', action='store_true', + help="start gdbserver with $GDB_QEMU options. \ + Default is localhost:12345")That makes it sound like this were the default for the
-gdboption. Since-gdbis just a switch, it doesn’t have a default (apart from being off by default).So I’d rephrase this at least to “The default options are 'localhost:12345'”. Or maybe “start gdbserver with $GDB_QEMU options ('localhost:12345' if $GDB_QEMU is empty)”.
Also, $GDB_QEMU as a name is a bit strange, because it does not specify which gdb to use; it just gives the options to use for gdb. $GDB_QEMU_OPTIONS would be more in line with the naming of the rest of the environment variables (or just $GDB_OPTIONS).
Max
Quoted 3 lines
tests/qemu-iotests/check +0
-0p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],
Re: [PATCH v3 04/15] qemu-iotests: add option to attach gdbserver
Quoted 10 lines · Emanuele Giuseppe Esposito
Quoted 5 lines · Max Reitz
Quoted 2 lines · Emanuele Giuseppe Esposito
Add -gdb flag and
GDB_QEMUenvironmental variable to python tests to attach a gdbserver to each qemu instance.Well, this patch doesn’t do this, but OK.
Maybe "define" rather than "add"? In the sense of defining the "-gdb" option, which is what it actually does.
That’s possible, but I think better would be to contrast it with what this patch doesn’t do, but what one could think when reading this description.
I.e. to say “Add/define -gdb flag [...] to each qemu instance. This patch only adds and parses this flag, it does not yet add the implementation for it.”
Quoted 7 lines
Quoted 1 line
Out of interest: Why gdbserver and not “just” gdb? On Fedora, those are separate packages, so I don’t have gdbserver installed, that’s why I’m asking.
As far as I have tried, using only gdb with ./check is very hard to use, because the stdout is filtered out by the script.
So invoking gdb attaches it to QEMU, but it is not possible to start execution (run command) or interact with it, because of the python script filtering. This leaves the test hanging.gdbserver is just something that a gdb client can attach to (for example, in another console or even in another host) for example with the command
# gdb -iex "target remote localhost:12345" . This provides a nice and separate gdb monitor to the client.
All right. I thought gdb could be used as a server, too, but... Looks like it can’t. (Like, I thought, you could do something like “gdb -ex 'listen localhost:12345' $cmd”. But seems like there is no server built into gdb proper.)
Max
Re: [PATCH v3 04/15] qemu-iotests: add option to attach gdbserver
In reply to Emanuele Giuseppe EspositoQuoted 42 lines · Emanuele Giuseppe Esposito
Add -gdb flag and
GDB_QEMUenvironmental variable to python tests to attach a gdbserver to each qemu instance.if -gdb is not provided but $GDB_QEMU is set, ignore the environmental variable.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/check | 6 +++++- tests/qemu-iotests/iotests.py | 4 ++++ tests/qemu-iotests/testenv.py | 15 ++++++++++++--- 3 files changed, 21 insertions(+), 4 deletions(-)tests/qemu-iotests/check +5
-1@@ -33,6 +33,9 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-gdb', action='store_true', + help="start gdbserver with $GDB_QEMU options. \ + Default is localhost:12345") p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],@@ -112,7 +115,8 @@ if __name__ == '__main__': env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto, aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, - debug=args.debug, valgrind=args.valgrind) + debug=args.debug, valgrind=args.valgrind, + gdb=args.gdb) testfinder = TestFinder(test_dir=env.source_iotests)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py line 80
Quoted 6 lines
tests/qemu-iotests/iotests.py +3
-0@@ -75,6 +75,10 @@ qemu_prog = os.environ.get('QEMU_PROG', 'qemu') qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') +qemu_gdb = [] +if os.environ.get('GDB_QEMU'): + qemu_gdb = ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ')
os.environ.get('GDB_QEMU') returns an Option[str], so mypy complains about the .strip() (thus failing iotest 297).
(That can be fixed for example by either using os.environ['GDB_QEMU'] here, like most other places here do, or by something like:
gdb_qemu_env = os.environ.get('GDB_QEMU')
if gdb_qemu_env:
qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' '))
Max
Quoted 52 lines
tests/qemu-iotests/iotests.py +1
-0+ imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.')tests/qemu-iotests/testenv.py +12
-3@@ -72,7 +72,8 @@ class TestEnv(ContextManager['TestEnv']): 'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', - 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_'] + 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', + 'GDB_QEMU'] def get_env(self) -> Dict[str, str]: env = {}@@ -163,7 +164,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, imgopts: Optional[str] = None, misalign: bool = False, debug: bool = False, - valgrind: bool = False) -> None: + valgrind: bool = False, + gdb: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode@@ -171,6 +173,11 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if gdb: + self.gdb_qemu = os.environ.get('GDB_QEMU', 'localhost:12345') + elif 'GDB_QEMU' in os.environ: + del os.environ['GDB_QEMU'] + if valgrind: self.valgrind_qemu = 'y'@@ -268,7 +275,9 @@ def print_env(self) -> None: PLATFORM -- {platform} TEST_DIR -- {TEST_DIR} SOCK_DIR -- {SOCK_DIR} -SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}""" +SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} +GDB_QEMU -- "{GDB_QEMU}" +""" args = collections.defaultdict(str, self.get_env())
PATCH v3 5/15 qemu-iotests: delay QMP socket timers
In reply to Emanuele Giuseppe EspositoAttaching a gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.
--- python/qemu/machine.py | 3 +++ tests/qemu-iotests/iotests.py | 10 +++++++++- 2 files changed, 12 insertions(+), 1 deletion(-)
python/qemu/machine.py +3 -0
@@ -409,6 +409,9 @@ def _launch(self) -> None: stderr=subprocess.STDOUT, shell=False, close_fds=False) + + if 'gdbserver' in self._wrapper: + self._qmp_timer = None self._post_launch() def _early_cleanup(self) -> None:
tests/qemu-iotests/iotests.py +9 -1
@@ -478,7 +478,10 @@ def log(msg: Msg, class Timeout: def __init__(self, seconds, errmsg="Timeout"): - self.seconds = seconds + if qemu_gdb: + self.seconds = 3000 + else: + self.seconds = seconds self.errmsg = errmsg def __enter__(self): signal.signal(signal.SIGALRM, self.timeout)
@@ -684,6 +687,11 @@ def qmp_to_opts(self, obj): output_list += [key + '=' + obj[key]] return ','.join(output_list) + def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: + if qemu_gdb: + wait = 0.0 + return super().get_qmp_events(wait=wait) + def get_qmp_events_filtered(self, wait=60.0): result = [] for ev in self.get_qmp_events(wait=wait):
Signature · 1 line
2.30.2
Re: [PATCH v3 05/15] qemu-iotests: delay QMP socket timers
Quoted 14 lines · Emanuele Giuseppe Esposito
Attaching a gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- python/qemu/machine.py | 3 +++ tests/qemu-iotests/iotests.py | 10 +++++++++- 2 files changed, 12 insertions(+), 1 deletion(-)python/qemu/machine.py +0
-0
python/qemu/machine.py def _launch(self) -> None: · line 414
Quoted 6 lines
python/qemu/machine.py +3
-0@@ -409,6 +409,9 @@ def _launch(self) -> None: stderr=subprocess.STDOUT, shell=False, close_fds=False) + + if 'gdbserver' in self._wrapper: + self._qmp_timer = None
Why doesn’t __init__() evaluate this? This here doesn’t feel like the right place for it. If we want to evaluate it here, self._qmp_timer shouldn’t exist, and instead the timeout should be a _post_launch() parameter. (Which I would have nothing against, by the way.)
Also, mypy complains that this variable is a float, so iotest 297 (which runs mypy) fails.
Quoted 8 lines
python/qemu/machine.py +0
-0self._post_launch() def _early_cleanup(self) -> None:tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py def log(msg: Msg, · line 484
Quoted 8 lines
tests/qemu-iotests/iotests.py +4
-1@@ -478,7 +478,10 @@ def log(msg: Msg, class Timeout: def __init__(self, seconds, errmsg="Timeout"): - self.seconds = seconds + if qemu_gdb: + self.seconds = 3000 + else: + self.seconds = seconds
We might as well completely disable the timeout then, that would be more honest, I think. (I.e. to make __enter__ and __exit__ no-ops.)
tests/qemu-iotests/iotests.py def qmp_to_opts(self, obj): · line 692
Quoted 10 lines
tests/qemu-iotests/iotests.py +3
-0self.errmsg = errmsg def __enter__(self): signal.signal(signal.SIGALRM, self.timeout)@@ -684,6 +687,11 @@ def qmp_to_opts(self, obj): output_list += [key + '=' + obj[key]] return ','.join(output_list) + def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: + if qemu_gdb: + wait = 0.0
First, this is a bool. I can see that qmp.py expects a
Union[bool, float], but machine.py expects just a bool. (Also, mypy complains that this specific wait here is a bool. You can see that by running iotest 297.)
Second, I don’t understand this. If the caller wants to block waiting on an event, then that should have nothing to do with whether we have gdb running or not. As far as I understand, setting wait to 0.0 is the same as wait = False, i.e. we don’t block and just return None immediately if there is no pending event.
Max
Quoted 6 lines
tests/qemu-iotests/iotests.py +2
-0+ return super().get_qmp_events(wait=wait) + def get_qmp_events_filtered(self, wait=60.0): result = [] for ev in self.get_qmp_events(wait=wait):
Re: [PATCH v3 05/15] qemu-iotests: delay QMP socket timers
Quoted 10 lines · Max Reitz
Quoted 9 lines · Emanuele Giuseppe Esposito
Attaching a gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py | 3 +++
tests/qemu-iotests/iotests.py| 10 +++++++++-2 files changed, 12 insertions(+), 1 deletion(-)
Quoted 13 lines
python/qemu/machine.py def _launch(self) -> None: · line 414
Quoted 11 lines
python/qemu/machine.py +3
-0@@ -409,6 +409,9 @@ def _launch(self) -> None: stderr=subprocess.STDOUT, shell=False, close_fds=False) + + if 'gdbserver' in self._wrapper: + self._qmp_timer = NoneWhy doesn’t
__init__()evaluate this? This here doesn’t feel like the right place for it. If we want to evaluate it here, self._qmp_timer shouldn’t exist, and instead the timeout should be a_post_launch()parameter. (Which I would have nothing against, by the way.)
Uhm.. I got another comment in a previous version where for the "event" callbacks it was better a property than passing around a parameter. Which I honestly agree.
What should __init__() do? The check here is to see if the invocation has gdb (and a couple of patches ahead also valgrind), to remove the timer. If I understand what you mean, you want something like
def __init__(self, timer):
But from my understanding, the class hierarchy is:
QEMUMachine: in machine.py QEMUQtestMachine(QEMUMachine): in qtest.py VM(qtest.QEMUQtestMachine): in iotests.py VM() is then invoked in each test.
So this is not easily reachable by check.py, to pass the parameter into __init__
Quoted 2 lines
Also, mypy complains that this variable is a float, so iotest 297 (which runs mypy) fails.
This and all mistakes of test 297 (mypy) is my fault: I did not have pylint-3 installed thus when testing it skipped the 297 test.
Quoted 18 lines
Quoted 8 lines
python/qemu/machine.py +0
-0self._post_launch()def
_early_cleanup(self)-> None:tests/qemu-iotests/iotests.py +0
-0@@ -478,7 +478,10 @@ def log(msg: Msg,Quoted 7 lines
class Timeout:
def__init__(self, seconds, errmsg="Timeout"):
- self.seconds = seconds
+ ifqemu_gdb:
+ self.seconds = 3000
+ else:
+ self.seconds = secondsWe might as well completely disable the timeout then, that would be more honest, I think. (I.e. to make
__enter__and__exit__no-ops.)
Quoted 12 lines
Quoted 9 lines
self.errmsg = errmsg
def__enter__(self):
signal.signal(signal.SIGALRM, self.timeout)tests/qemu-iotests/iotests.py +0
-0@@ -684,6 +687,11 @@ def qmp_to_opts(self, obj):
output_list+= [key + '=' + obj[key]]
return ','.join(output_list)
+ defget_qmp_events(self, wait: bool = False)-> List[QMPMessage]:
+ ifqemu_gdb:
+ wait = 0.0First, this is a bool. I can see that
qmp.pyexpects a
Union[bool, float], butmachine.pyexpects just a bool. (Also, mypy complains that this specificwaithere is abool. You can see that by running iotest 297.)
I think here machine.py should reflect the qmp.py behavior and have an Union[bool, float] too.
Quoted 2 lines
Second, I don’t understand this. If the caller wants to block waiting on an event, then that should have nothing to do with whether we have gdb running or not. As far as I understand, setting wait to 0.0 is the same as wait = False, i.e. we don’t block and just return None immediately if there is no pending event.
You're right, this might not be needed here. The problem I had was that calling gdb and pausing at a breakpoint or something for a while would make the QMP socket timeout, thus aborting the whole test. In order to avoid that, I need to stop or delay timers.
I can't remember why I added this check here. At some point I am sure the test was failing because of socket timeout expiration, but I cannot reproduce the problem when commenting out this check above in get_qmp_events. The other check in patch 3 should be enough.
Emanuele
Quoted 10 lines
Max
Quoted 6 lines
+ return super().get_qmp_events(wait=wait) +
defget_qmp_events_filtered(self, wait=60.0):
result = []
for ev inself.get_qmp_events(wait=wait):
Re: [PATCH v3 05/15] qemu-iotests: delay QMP socket timers
Quoted 27 lines · Emanuele Giuseppe Esposito
Quoted 10 lines · Max Reitz
Quoted 9 lines · Emanuele Giuseppe Esposito
Attaching a gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py | 3 +++
tests/qemu-iotests/iotests.py| 10 +++++++++-2 files changed, 12 insertions(+), 1 deletion(-)Quoted 13 lines
python/qemu/machine.py def _launch(self) -> None: · line 414
Quoted 11 lines
python/qemu/machine.py +3
-0@@ -409,6 +409,9 @@ def _launch(self) -> None: stderr=subprocess.STDOUT, shell=False, close_fds=False) + + if 'gdbserver' in self._wrapper: + self._qmp_timer = NoneWhy doesn’t
__init__()evaluate this? This here doesn’t feel like the right place for it. If we want to evaluate it here, self._qmp_timer shouldn’t exist, and instead the timeout should be a_post_launch()parameter. (Which I would have nothing against, by the way.)
Quoted 1 line
Uhm.. I got another comment in a previous version where for the "event" callbacks it was better a property than passing around a parameter. Which I honestly agree.
I think that comment was in the sense of providing a default value, which can be expressed by having a property that is set in __init__.
I don’t have anything against making this a property, but I also don’t have anything against making it a _post_launch() parameter. I could even live with both, i.e. set _qmp_timer to 15 in __init__, then have a _post_launch parameter, and pass either self._qmp_timer or None if self._wrapper includes 'gdbserver'.
What I do mind is that I don’t understand why the property is modified here. The value of self._qmp_timer is supposed to be 15 by default and None if self._wrapper includes 'gdbserver'. It should thus be changed to None the moment self._wrapper is made to include 'gdbserver'. Because self._wrapper is set only in __init__, this should happen in __init__.
Quoted 3 lines
What should
__init__()do? The check here is to see if the invocation has gdb (and a couple of patches ahead also valgrind), to remove the timer. If I understand what you mean, you want something like
def__init__(self, timer):
Oh, no. We can optionally do that perhaps later, but what I meant is just to put this in __init__() (without adding any parameters to it):
self._qmp_timer = 15.0 if 'gdbserver' not in self._wrapper else None
I think self._qmp_timer should always reflect what timeout we are going to use when a VM is launched. So if the conditions influencing the timeout change, it should be updated immediately to reflect this. The only condition we have right now is the content of self._wrapper, which is only set in __init__, so self._qmp_timer should be set once in __init__ and not changed afterwards.
That sounds academic, but imagine what would happen if we had a set_qmp_timer() method: The timout could be adjusted, but launch() would just ignore it and update the property, even though the conditions influencing the timout didn’t change between set_qmp_timer() and launch().
Or if we had a get_qmp_timer(); a caller would read a timeout of 15.0 before launch(), even though the timeout is going to be None.
Therefore, I think a property should not be updated just before it is read, but instead when any condition that’s supposed to influence its value changes.
I suggested making it a parameter because updating a property when reading it sounds like it should be a parameter instead. I.e., one would say
def __init__():
self._qmp_timeout_default = 15.0
def post_launch(qmp_timeout):
self._qmp.accept(qmp_timeout)
def launch(self):
...
qmp_timeout = None if 'gdbserver' in self._wrapper \
else self._qmp_timout_default
self.post_launch(qmp_timeout)Which is basically the structure your patch has, which gave me the idea.
[...]
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
python/qemu/machine.py +0
-0self._post_launch()def
_early_cleanup(self)-> None:tests/qemu-iotests/iotests.py +0
-0
[...]
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
tests/qemu-iotests/iotests.py +0
-0@@ -684,6 +687,11 @@ def qmp_to_opts(self, obj):
output_list+= [key + '=' + obj[key]]
return ','.join(output_list)
+ defget_qmp_events(self, wait: bool = False)-> List[QMPMessage]:
+ ifqemu_gdb:
+ wait = 0.0
[...]
Quoted 6 lines
Quoted 2 lines
Second, I don’t understand this. If the caller wants to block waiting on an event, then that should have nothing to do with whether we have gdb running or not. As far as I understand, setting wait to 0.0 is the same as wait = False, i.e. we don’t block and just return None immediately if there is no pending event.
You're right, this might not be needed here. The problem I had was that calling gdb and pausing at a breakpoint or something for a while would make the QMP socket timeout, thus aborting the whole test. In order to avoid that, I need to stop or delay timers.
I can't remember why I added this check here. At some point I am sure the test was failing because of socket timeout expiration, but I cannot reproduce the problem when commenting out this check above in
get_qmp_events. The other check in patch 3 should be enough.
Hm, ok. I’d guessed that you intended the wait=0.0 or wait=False to mean that we get an infinite timeout (i.e., no timeout), but that’s exactly why I didn’t get it. wait=0.0 doesn’t give an infinite timeout, but instead basically times out immediately.
Max
Re: [PATCH v3 05/15] qemu-iotests: delay QMP socket timers
Quoted 28 lines · Max Reitz
Quoted 27 lines · Emanuele Giuseppe Esposito
Quoted 23 lines · Max Reitz
Quoted 14 lines · Emanuele Giuseppe Esposito
Attaching a gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py | 3 +++
tests/qemu-iotests/iotests.py| 10 +++++++++-2 files changed, 12 insertions(+), 1 deletion(-)python/qemu/machine.py +0
-0python/qemu/machine.py def _launch(self) -> None: · line 414
Quoted 6 lines
python/qemu/machine.py +3
-0@@ -409,6 +409,9 @@ def _launch(self) -> None: stderr=subprocess.STDOUT, shell=False, close_fds=False) + + if 'gdbserver' in self._wrapper: + self._qmp_timer = NoneWhy doesn’t
__init__()evaluate this? This here doesn’t feel like the right place for it. If we want to evaluate it here, self._qmp_timer shouldn’t exist, and instead the timeout should be a_post_launch()parameter. (Which I would have nothing against, by the way.)
Quoted 4 lines
Quoted 1 line
Uhm.. I got another comment in a previous version where for the "event" callbacks it was better a property than passing around a parameter. Which I honestly agree.
I think that comment was in the sense of providing a default value, which can be expressed by having a property that is set in
__init__.
My comment was along the lines that "_post_launch()" is behaving as an event loop hook and not the sort of thing I want to pass parameters to. It's a private method, so the only possibility for someone passing a parameter to is another class method anyway.
We have a hierarchy of things that depend on the Machine class and I didn't want to start cascading optional parameters into the subclasses.
It was my intent that the information needed to run _post_launch() correctly should be known by the state of the object -- which I think should be true anyway.
Quoted 12 lines
I don’t have anything against making this a property, but I also don’t have anything against making it a
_post_launch()parameter. I could even live with both, i.e. set_qmp_timerto 15 in__init__, then have a_post_launchparameter, and pass either self._qmp_timer or None if self._wrapper includes 'gdbserver'.What I do mind is that I don’t understand why the property is modified here. The value of self._qmp_timer is supposed to be 15 by default and None if self._wrapper includes 'gdbserver'. It should thus be changed to None the moment self._wrapper is made to include 'gdbserver'. Because self._wrapper is set only in
__init__, this should happen in__init__.Quoted 3 lines
What should
__init__()do? The check here is to see if the invocation has gdb (and a couple of patches ahead also valgrind), to remove the timer. If I understand what you mean, you want something like
def__init__(self, timer):Oh, no. We can optionally do that perhaps later, but what I meant is just to put this in
__init__()(without adding any parameters to it):self._qmp_timer = 15.0 if 'gdbserver' not in self._wrapper else None
Quoted 8 lines
I think self._qmp_timer should always reflect what timeout we are going to use when a VM is launched. So if the conditions influencing the timeout change, it should be updated immediately to reflect this. The only condition we have right now is the content of self._wrapper, which is only set in
__init__, so self._qmp_timer should be set once in__init__and not changed afterwards.That sounds academic, but imagine what would happen if we had a
set_qmp_timer()method: The timout could be adjusted, butlaunch()would just ignore it and update the property, even though the conditions influencing the timout didn’t change betweenset_qmp_timer()andlaunch().Or if we had a
get_qmp_timer(); a caller would read a timeout of 15.0 beforelaunch(), even though the timeout is going to be None.Therefore, I think a property should not be updated just before it is read, but instead when any condition that’s supposed to influence its value changes.
I agree with Max's reasoning here.
I am also not a fan of squishing magic into this class; changing class behavior based on introspection of wrapper arguments feels like a layering violation.
Maybe what you want is a subclass or a wrapper class that knows how to run QEMU using gdbserver, and changes some behaviors accordingly?
The factoring of Machine is quite bad already, admittedly, and is in need of a good spit-shine. Too many init parameters, too many state variables, too many methods that got patched in to support one specific use-case at one point or another. At a certain point, I begin to worry about how it's possible to audit how all of these one-off features behave and interact. It's getting complex.
Is it time to dream up a refactoring for how the Machine class behaves?
Quoted 41 lines
I suggested making it a parameter because updating a property when reading it sounds like it should be a parameter instead. I.e., one would say
def
__init__():
self._qmp_timeout_default = 15.0def
post_launch(qmp_timeout):
self._qmp.accept(qmp_timeout)def launch(self):
...
qmp_timeout= None if 'gdbserver' in self._wrapper \
else self._qmp_timout_defaultself.post_launch(qmp_timeout)Which is basically the structure your patch has, which gave me the idea.
[...]
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
python/qemu/machine.py +0
-0self._post_launch()def
_early_cleanup(self)-> None:tests/qemu-iotests/iotests.py +0
-0[...]
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
tests/qemu-iotests/iotests.py +0
-0@@ -684,6 +687,11 @@ def qmp_to_opts(self, obj):
output_list+= [key + '=' + obj[key]]
return ','.join(output_list)
+ defget_qmp_events(self, wait: bool = False)-> List[QMPMessage]:
+ ifqemu_gdb:
+ wait = 0.0[...]
Quoted 3 lines
Quoted 2 lines
Second, I don’t understand this. If the caller wants to block waiting on an event, then that should have nothing to do with whether we have gdb running or not. As far as I understand, setting wait to 0.0 is the same as wait = False, i.e. we don’t block and just return None immediately if there is no pending event.
Quoted 7 lines
Quoted 3 lines
You're right, this might not be needed here. The problem I had was that calling gdb and pausing at a breakpoint or something for a while would make the QMP socket timeout, thus aborting the whole test. In order to avoid that, I need to stop or delay timers.
I can't remember why I added this check here. At some point I am sure the test was failing because of socket timeout expiration, but I cannot reproduce the problem when commenting out this check above in
get_qmp_events. The other check in patch 3 should be enough.Hm, ok. I’d guessed that you intended the wait=0.0 or wait=False to mean that we get an infinite timeout (i.e., no timeout), but that’s exactly why I didn’t get it. wait=0.0 doesn’t give an infinite timeout, but instead basically times out immediately.
Max
Well, I suppose if we don't need it, then that makes things easier too :)
PATCH v3 6/15 qemu_iotests: insert gdbserver command line as wrapper for qemu binary
In reply to Emanuele Giuseppe Esposito--- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
@@ -582,7 +582,8 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - super().__init__(qemu_prog, qemu_opts, name=name, + super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, + name=name, test_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir)
Signature · 1 line
2.30.2
Re: [PATCH v3 06/15] qemu_iotests: insert gdbserver command line as wrapper for qemu binary
Quoted 4 lines · Emanuele Giuseppe Esposito
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
PATCH v3 7/15 qemu-iotests: add gdbserver option to script tests too
In reply to Emanuele Giuseppe EspositoThe only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail.
--- tests/qemu-iotests/common.rc | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
tests/qemu-iotests/common.rc +7 -1
@@ -166,8 +166,14 @@ _qemu_wrapper() if [ -n "${QEMU_NEED_PID}" ]; then echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid" fi + + GDB="${QEMU_PROG}" + if [ ! -z ${GDB_QEMU} ]; then + GDB="gdbserver ${GDB_QEMU} ${GDB}" + fi + VALGRIND_QEMU="${VALGRIND_QEMU_VM}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \ - "$QEMU_PROG" $QEMU_OPTIONS "$@" + $GDB $QEMU_OPTIONS "$@" ) RETVAL=$? _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL
Signature · 1 line
2.30.2
Re: [PATCH v3 07/15] qemu-iotests: add gdbserver option to script tests too
Quoted 16 lines · Emanuele Giuseppe Esposito
The only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/common.rc | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)tests/qemu-iotests/common.rc +0
-0@@ -166,8 +166,14 @@ _qemu_wrapper() if [ -n "${QEMU_NEED_PID}" ]; then echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid"
tests/qemu-iotests/common.rc _qemu_wrapper() · line 176
Quoted 10 lines
tests/qemu-iotests/common.rc +7
-1fi + + GDB="${QEMU_PROG}" + if [ ! -z ${GDB_QEMU} ]; then + GDB="gdbserver ${GDB_QEMU} ${GDB}" + fi + VALGRIND_QEMU="${VALGRIND_QEMU_VM}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \ - "$QEMU_PROG" $QEMU_OPTIONS "$@" + $GDB $QEMU_OPTIONS "$@"
This looks strange, because now there is no qemu here.
Why not just fill $GDB with the GDB options, keeping it empty if no gdb is to be used? Then we’d have
VALGRIND_QEMU=... ... \
$GDB "$QEMU_PROG" $QEMU_OPTIONS "$@"here.
(Also, the indentation changed. I’d keep it aligned to four spaces.)
Max
Quoted 4 lines
tests/qemu-iotests/common.rc +0
-0) RETVAL=$? _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL
PATCH v3 8/15 docs/devel/testing: add -gdb option to the debugging section of QEMU iotests
In reply to Emanuele Giuseppe Esposito--- docs/devel/testing.rst | 7 +++++++ 1 file changed, 7 insertions(+)
docs/devel/testing.rst +7 -0
@@ -229,6 +229,13 @@ Debugging a test case QEMU iotests offers some options to debug a failing test, that can be given as options to the ``check`` script: +* ``-gdb`` wraps ``gdbsever`` to the QEMU binary, + so it is possible to connect to it via gdb. + One way to do so is via ``gdb -iex "target remote $GDB_QEMU"`` + The default address is ``localhost:12345``, and can be changed + by setting the ``$GDB_QEMU`` environmental variable. + The final command line will be ``gdbserver $GDB_QEMU $QEMU ...`` + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers.
Signature · 1 line
2.30.2
Re: [PATCH v3 08/15] docs/devel/testing: add -gdb option to the debugging section of QEMU iotests
Quoted 10 lines · Emanuele Giuseppe Esposito
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- docs/devel/testing.rst | 7 +++++++ 1 file changed, 7 insertions(+)docs/devel/testing.rst +0
-0
docs/devel/testing.rst Debugging a test case · line 236
Quoted 8 lines
docs/devel/testing.rst +5
-0@@ -229,6 +229,13 @@ Debugging a test case QEMU iotests offers some options to debug a failing test, that can be given as options to the ``check`` script: +* ``-gdb`` wraps ``gdbsever`` to the QEMU binary, + so it is possible to connect to it via gdb. + One way to do so is via ``gdb -iex "target remote $GDB_QEMU"`` + The default address is ``localhost:12345``, and can be changed + by setting the ``$GDB_QEMU`` environmental variable.
*environment variable
docs/devel/testing.rst Debugging a test case · line 238
Quoted 2 lines
docs/devel/testing.rst +2
-0+ The final command line will be ``gdbserver $GDB_QEMU $QEMU ...`` +
I think the order in this explanation is ordered not quite right, because it uses $GDB_QEMU before explaining what it is. (Also, I suppose $GDB_QEMU might contain other options than the socket address, so "target remote $GDB_QEMU" does not necessarily work.) I’d reorder/change it to:
``-gdb`` wraps every QEMU invocation in a ``gdbserver``, which waits for a connection from a gdb client. The options given to ``gdbserver`` (e.g. the address on which to listen for connections) are taken from the ``$GDB_QEMU`` environment variable. By default (if ``$GDB_QEMU`` is empty), it listens on ``localhost:12345``. You can connect to it for example with ``gdb -iex "target remote $addr"``, where ``$addr`` is the address ``gdbserver`` listens on.
Max
Quoted 4 lines
docs/devel/testing.rst +0
-0* ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers.
PATCH v3 9/15 qemu_iotests: extend the check script to support valgrind for python tests
In reply to Emanuele Giuseppe EspositoCurrently, the check script only parses the option and sets the VALGRIND_QEMU environmental variable to "y".
Add another local python variable that prepares the command line, identical to the one provided in the test scripts.
Because the python script does not know in advance the valgring PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime.
--- tests/qemu-iotests/check | 7 ++++--- tests/qemu-iotests/iotests.py | 11 +++++++++++ tests/qemu-iotests/testenv.py | 1 + 3 files changed, 16 insertions(+), 3 deletions(-)
tests/qemu-iotests/check +4 -3
@@ -36,6 +36,10 @@ def make_argparser() -> argparse.ArgumentParser: p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_QEMU options. \ Default is localhost:12345") + p.add_argument('-valgrind', action='store_true', + help='use valgrind, sets VALGRIND_QEMU environment ' + 'variable') + p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -86,9 +90,6 @@ def make_argparser() -> argparse.ArgumentParser: g_bash.add_argument('-o', dest='imgopts', help='options to pass to qemu-img create/convert, ' 'sets IMGOPTS environment variable') - g_bash.add_argument('-valgrind', action='store_true', - help='use valgrind, sets VALGRIND_QEMU environment ' - 'variable') g_sel = p.add_argument_group('test selecting options', 'The following options specify test set '
tests/qemu-iotests/iotests.py +11 -0
@@ -96,6 +96,17 @@ sys.stderr.write('Please run this test via the "check" script\n') sys.exit(os.EX_USAGE) +qemu_valgrind = [] +if os.environ.get('VALGRIND_QEMU') == "y" and \ + os.environ.get('NO_VALGRIND') != "y": + valgrind_logfile = "--log-file=" + test_dir.strip() + # %p allows to put the valgrind process PID, since + # we don't know it a priori (subprocess.Peopen is + # not yet invoked) + valgrind_logfile += "/%p.valgrind" + + qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99'] + socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \
tests/qemu-iotests/testenv.py +1 -0
@@ -277,6 +277,7 @@ def print_env(self) -> None: SOCK_DIR -- {SOCK_DIR} SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_QEMU -- "{GDB_QEMU}" +VALGRIND_QEMU -- "{VALGRIND_QEMU}" """ args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.30.2
Re: [PATCH v3 09/15] qemu_iotests: extend the check script to support valgrind for python tests
Quoted 46 lines · Emanuele Giuseppe Esposito
Currently, the check script only parses the option and sets the
VALGRIND_QEMUenvironmental variable to "y".
Add another local python variable that prepares the command line, identical to the one provided in the test scripts.Because the python script does not know in advance the valgring PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/check | 7 ++++--- tests/qemu-iotests/iotests.py | 11 +++++++++++ tests/qemu-iotests/testenv.py | 1 + 3 files changed, 16 insertions(+), 3 deletions(-)tests/qemu-iotests/check +4
-3@@ -36,6 +36,10 @@ def make_argparser() -> argparse.ArgumentParser: p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_QEMU options. \ Default is localhost:12345") + p.add_argument('-valgrind', action='store_true', + help='use valgrind, sets VALGRIND_QEMU environment ' + 'variable') + p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'],@@ -86,9 +90,6 @@ def make_argparser() -> argparse.ArgumentParser: g_bash.add_argument('-o', dest='imgopts', help='options to pass to qemu-img create/convert, ' 'sets IMGOPTS environment variable') - g_bash.add_argument('-valgrind', action='store_true', - help='use valgrind, sets VALGRIND_QEMU environment ' - 'variable') g_sel = p.add_argument_group('test selecting options', 'The following options specify test set 'tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py line 104
Quoted 9 lines
tests/qemu-iotests/iotests.py +6
-0@@ -96,6 +96,17 @@ sys.stderr.write('Please run this test via the "check" script\n') sys.exit(os.EX_USAGE) +qemu_valgrind = [] +if os.environ.get('VALGRIND_QEMU') == "y" and \ + os.environ.get('NO_VALGRIND') != "y": + valgrind_logfile = "--log-file=" + test_dir.strip() + # %p allows to put the valgrind process PID, since + # we don't know it a priori (subprocess.Peopen is
*Popen
Quoted 13 lines
tests/qemu-iotests/iotests.py +5
-0+ # not yet invoked) + valgrind_logfile += "/%p.valgrind" + + qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99'] + socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \tests/qemu-iotests/testenv.py +0
-0
tests/qemu-iotests/testenv.py def print_env(self) -> None: · line 280
Quoted 4 lines
tests/qemu-iotests/testenv.py +1
-0@@ -277,6 +277,7 @@ def print_env(self) -> None: SOCK_DIR -- {SOCK_DIR} SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_QEMU -- "{GDB_QEMU}" +VALGRIND_QEMU -- "{VALGRIND_QEMU}"
I don’t think this should be enclosed in quotes (and neither should GDB_QEMU be). AFAIU, only the *_PROG options are quoted to signify that they will be executed as a single program, whether they include spaces or not. OTOH, GDB_QEMU is a list of options, so spaces act as separators, and VALGRIND_QEMU is just y or not set, so it’s impossible for spaces to be in there.
(Also, I think the “--” should be aligned to all the other “--”s.)
Bikeshedding, though, so:
Quoted 4 lines
tests/qemu-iotests/testenv.py +0
-0""" args = collections.defaultdict(str, self.get_env())
PATCH v3 10/15 qemu_iotests: extent QMP socket timeout when using valgrind
In reply to Emanuele Giuseppe EspositoAs with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon.
--- python/qemu/machine.py | 2 +- tests/qemu-iotests/iotests.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
python/qemu/machine.py +1 -1
@@ -410,7 +410,7 @@ def _launch(self) -> None: shell=False, close_fds=False) - if 'gdbserver' in self._wrapper: + if 'gdbserver' in self._wrapper or 'valgrind' in self._wrapper: self._qmp_timer = None self._post_launch()
tests/qemu-iotests/iotests.py +2 -2
@@ -489,7 +489,7 @@ def log(msg: Msg, class Timeout: def __init__(self, seconds, errmsg="Timeout"): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: self.seconds = 3000 else: self.seconds = seconds
@@ -700,7 +700,7 @@ def qmp_to_opts(self, obj): return ','.join(output_list) def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: - if qemu_gdb: + if qemu_gdb or qemu_valgrind: wait = 0.0 return super().get_qmp_events(wait=wait)
Signature · 1 line
2.30.2
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 2 lines · Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon.
I’m curious: The default timeouts should be long enough for slow systems, too, though (e.g. heavily-loaded CI systems). I would expect that valgrind is used on developer systems where there is more leeway, so the timeouts should still work.
But in practice, thinking about that doesn’t matter. If valgrind leads to a timeout being hit, that wouldn’t be nice. OTOH, if you run valgrind to debug a test/qemu, you don’t particularly care about the timeouts anyway.
So in principle, this patch sounds good to me, it’s just that it’s based on patch 5, which I don’t fully agree with.
Max
Quoted 42 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- python/qemu/machine.py | 2 +- tests/qemu-iotests/iotests.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)python/qemu/machine.py +1
-1@@ -410,7 +410,7 @@ def _launch(self) -> None: shell=False, close_fds=False) - if 'gdbserver' in self._wrapper: + if 'gdbserver' in self._wrapper or 'valgrind' in self._wrapper: self._qmp_timer = None self._post_launch()tests/qemu-iotests/iotests.py +2
-2@@ -489,7 +489,7 @@ def log(msg: Msg, class Timeout: def __init__(self, seconds, errmsg="Timeout"): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: self.seconds = 3000 else: self.seconds = seconds@@ -700,7 +700,7 @@ def qmp_to_opts(self, obj): return ','.join(output_list) def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: - if qemu_gdb: + if qemu_gdb or qemu_valgrind: wait = 0.0 return super().get_qmp_events(wait=wait)
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 5 lines · Max Reitz
Quoted 2 lines · Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon.
I’m curious: The default timeouts should be long enough for slow systems, too, though (e.g. heavily-loaded CI systems). I would expect that valgrind is used on developer systems where there is more leeway, so the timeouts should still work.
As said in patch 5, I will check again which timeout is essential to avoid and which not.
Emanuele
Quoted 45 lines
But in practice, thinking about that doesn’t matter. If valgrind leads to a timeout being hit, that wouldn’t be nice. OTOH, if you run valgrind to debug a test/qemu, you don’t particularly care about the timeouts anyway.
So in principle, this patch sounds good to me, it’s just that it’s based on patch 5, which I don’t fully agree with.
Max
Quoted 37 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py | 2 +-
tests/qemu-iotests/iotests.py| 4 ++--2 files changed, 3 insertions(+), 3 deletions(-)python/qemu/machine.py +1
-1@@ -410,7 +410,7 @@ def _launch(self) -> None: shell=False, close_fds=False) - if 'gdbserver' in self._wrapper: + if 'gdbserver' in self._wrapper or 'valgrind' in self._wrapper:self._qmp_timer = None
self._post_launch()tests/qemu-iotests/iotests.py +0
-0@@ -489,7 +489,7 @@ def log(msg: Msg,class Timeout:
def__init__(self, seconds, errmsg="Timeout"):
- ifqemu_gdb:
+ ifqemu_gdborqemu_valgrind:
self.seconds = 3000
else:
self.seconds = secondstests/qemu-iotests/iotests.py +0
-0@@ -700,7 +700,7 @@ def qmp_to_opts(self, obj): return ','.join(output_list)def
get_qmp_events(self, wait: bool = False)-> List[QMPMessage]:
- ifqemu_gdb:
+ ifqemu_gdborqemu_valgrind:
wait = 0.0
return super().get_qmp_events(wait=wait)
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
In reply to Emanuele Giuseppe EspositoQuoted 14 lines · Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- python/qemu/machine.py | 2 +- tests/qemu-iotests/iotests.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)python/qemu/machine.py +0
-0
python/qemu/machine.py def _launch(self) -> None: · line 413
Quoted 5 lines
python/qemu/machine.py +1
-1@@ -410,7 +410,7 @@ def _launch(self) -> None: shell=False, close_fds=False) - if 'gdbserver' in self._wrapper: + if 'gdbserver' in self._wrapper or 'valgrind' in self._wrapper:
This approaches me suggesting that we just change __init__ to accept a parameter that lets the caller decide what kind of timeout(s) they find acceptable. They know more about what they're trying to run than we do.
Certainly after launch occurs, the user is free to just grab the qmp object and tinker around with the timeouts, but that does not allow us to change the timeout(s) for accept itself.
D'oh.
(Spilled milk: It was probably a mistake to make the default launch behavior here have a timeout of 15 seconds. That logic likely belongs to the iotests implementation. The default here probably ought to indeed be "wait forever".)
In the here and now ... would it be acceptable to change the launch() method to add a timeout parameter? It's still a little awkward, because conceptually it's a timeout for just QMP and not for the actual duration of the entire launch process.
But, I guess, it's *closer* to the truth.
If you wanted to route it that way, I take back what I said about not wanting to pass around variables to event loop hooks.
If we defined the timeout as something that applies exclusively to the launching process, then it'd be appropriate to route that to the launch-related functions ... and subclasses would have to be adjusted to be made aware that they're expected to operate within those parameters, which is good.
Sorry for my waffling back and forth on this. Let me know what the actual requirements are if you figure out which timeouts you need / don't need and I'll give you some review priority.
If you attack this series again, can you please split out the python/* patches into its own little series and CC me?
--js
Quoted 26 lines
python/qemu/machine.py +0
-0self._qmp_timer = None self._post_launch()tests/qemu-iotests/iotests.py +2
-2@@ -489,7 +489,7 @@ def log(msg: Msg, class Timeout: def __init__(self, seconds, errmsg="Timeout"): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: self.seconds = 3000 else: self.seconds = seconds@@ -700,7 +700,7 @@ def qmp_to_opts(self, obj): return ','.join(output_list) def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: - if qemu_gdb: + if qemu_gdb or qemu_valgrind: wait = 0.0 return super().get_qmp_events(wait=wait)
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 30 lines · John Snow
Quoted 14 lines · Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py | 2 +-
tests/qemu-iotests/iotests.py| 4 ++--2 files changed, 3 insertions(+), 3 deletions(-)python/qemu/machine.py +0
-0python/qemu/machine.py def _launch(self) -> None: · line 412
Quoted 4 lines
python/qemu/machine.py +1
-1@@ -410,7 +410,7 @@ def _launch(self) -> None: shell=False, close_fds=False) - if 'gdbserver' in self._wrapper: + if 'gdbserver' in self._wrapper or 'valgrind' in self._wrapper:This approaches me suggesting that we just change
__init__to accept a parameter that lets the caller decide what kind of timeout(s) they find acceptable. They know more about what they're trying to run than we do.Certainly after launch occurs, the user is free to just grab the qmp object and tinker around with the timeouts, but that does not allow us to change the timeout(s) for accept itself.
D'oh.
(Spilled milk: It was probably a mistake to make the default launch behavior here have a timeout of 15 seconds. That logic likely belongs to the iotests implementation. The default here probably ought to indeed be "wait forever".)
In the here and now ... would it be acceptable to change the
launch()method to add a timeout parameter? It's still a little awkward, because conceptually it's a timeout for just QMP and not for the actual duration of the entire launch process.
Quoted 7 lines
But, I guess, it's *closer* to the truth.
If you wanted to route it that way, I take back what I said about not wanting to pass around variables to event loop hooks.
If we defined the timeout as something that applies exclusively to the launching process, then it'd be appropriate to route that to the launch-related functions ... and subclasses would have to be adjusted to be made aware that they're expected to operate within those parameters, which is good.
Sorry for my waffling back and forth on this. Let me know what the actual requirements are if you figure out which timeouts you need / don't need and I'll give you some review priority.
Uhm.. I am getting a little bit confused on what to do too :)
So the current plan I have for _qmp_timer is:
- As Max suggested, move it in __init__ and check there for the wrapper contents. If we need to block forever (gdb, valgrind), we set it to None. Otherwise to 15 seconds. I think setting it always to None is not ideal, because if you are testing something that deadlocks (see my attempts to remove/add locks in QEMU multiqueue) and the socket is set to block forever, you don't know if the test is super slow or it just deadlocked.
Well, one can argue that in both cases this is not the expected behavior, but I think having an upper bound on each QMP command execution would be good.
- pass _qmp_timer directly to self._qmp.accept() in _post launch, leaving _launch() intact. I think this makes sense because as you also mentioned, changing _post_launch() into taking a parameter requires changing also all subclasses and pass values around.
Any opinion on this is very welcome.
Spoiler alert I haven't tested these changes yet, but I am positive that there shouldn't be any problem. (Last famous words)
Emanuele
Quoted 28 lines
If you attack this series again, can you please split out the python/* patches into its own little series and CC me?
--js
Quoted 22 lines
self._qmp_timer = None
self._post_launch()tests/qemu-iotests/iotests.py +0
-0@@ -489,7 +489,7 @@ def log(msg: Msg,class Timeout:
def__init__(self, seconds, errmsg="Timeout"):
- ifqemu_gdb:
+ ifqemu_gdborqemu_valgrind:
self.seconds = 3000
else:
self.seconds = secondstests/qemu-iotests/iotests.py +0
-0@@ -700,7 +700,7 @@ def qmp_to_opts(self, obj): return ','.join(output_list)def
get_qmp_events(self, wait: bool = False)-> List[QMPMessage]:
- ifqemu_gdb:
+ ifqemu_gdborqemu_valgrind:
wait = 0.0
return super().get_qmp_events(wait=wait)
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 35 lines · Emanuele Giuseppe Esposito
Quoted 32 lines · John Snow
Quoted 14 lines · Emanuele Giuseppe Esposito
As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
python/qemu/machine.py | 2 +-
tests/qemu-iotests/iotests.py| 4 ++--2 files changed, 3 insertions(+), 3 deletions(-)python/qemu/machine.py +0
-0python/qemu/machine.py def _launch(self) -> None: · line 412
Quoted 4 lines
python/qemu/machine.py +1
-1@@ -410,7 +410,7 @@ def _launch(self) -> None: shell=False, close_fds=False) - if 'gdbserver' in self._wrapper: + if 'gdbserver' in self._wrapper or 'valgrind' in self._wrapper:This approaches me suggesting that we just change
__init__to accept a parameter that lets the caller decide what kind of timeout(s) they find acceptable. They know more about what they're trying to run than we do.Certainly after launch occurs, the user is free to just grab the qmp object and tinker around with the timeouts, but that does not allow us to change the timeout(s) for accept itself.
D'oh.
(Spilled milk: It was probably a mistake to make the default launch behavior here have a timeout of 15 seconds. That logic likely belongs to the iotests implementation. The default here probably ought to indeed be "wait forever".)
In the here and now ... would it be acceptable to change the
launch()method to add a timeout parameter? It's still a little awkward, because conceptually it's a timeout for just QMP and not for the actual duration of the entire launch process.But, I guess, it's *closer* to the truth.
Quoted 8 lines
Quoted 5 lines
If you wanted to route it that way, I take back what I said about not wanting to pass around variables to event loop hooks.
If we defined the timeout as something that applies exclusively to the launching process, then it'd be appropriate to route that to the launch-related functions ... and subclasses would have to be adjusted to be made aware that they're expected to operate within those parameters, which is good.
Sorry for my waffling back and forth on this. Let me know what the actual requirements are if you figure out which timeouts you need / don't need and I'll give you some review priority.
Uhm.. I am getting a little bit confused on what to do too :)
SORRY, I hit send too quickly and then change my mind. I've handed you a giant bag of my own confusion. Very unfair of me!
Quoted 5 lines
So the current plan I have for
_qmp_timeris:- As Max suggested, move it in
__init__and check there for the wrapper
contents. If we need to block forever (gdb, valgrind), we set it to None. Otherwise to 15 seconds. I think setting it always to None is not ideal, because if you are testing something that deadlocks (see my attempts to remove/add locks in QEMU multiqueue) and the socket is set to block forever, you don't know if the test is super slow or it just deadlocked.
I agree with your concern on rational defaults, let's focus on that briefly:
Let's have QEMUMachine default to *no timeouts* moving forward, and have the timeouts be *opt-in*. This keeps the Machine class somewhat pure and free of opinions. The separation of mechanism and policy.
Next, instead of modifying hundreds of tests to opt-in to the timeout, let's modify the VM class in iotests.py to opt-in to that timeout, restoring the current "safe" behavior of iotests.
The above items can happen in a single commit, preserving behavior in the bisect.
Finally, we can add a non-private property that individual tests can re-override to opt BACK out of the default.
Something as simple as:
vm.qmp_timeout = None
would be just fine.
Quoted 5 lines
Well, one can argue that in both cases this is not the expected behavior, but I think having an upper bound on each QMP command execution would be good.
- pass
_qmp_timerdirectly toself._qmp.accept()in_postlaunch,
leaving_launch()intact. I think this makes sense because as you also mentioned, changing_post_launch()into taking a parameter requires changing also all subclasses and pass values around.
Sounds OK. If we do change the defaults back to "No Timeout" in a way that allows an override by an opinionated class, we'll already have the public property, though, so a parameter might not be needed.
(Yes, this is the THIRD time I've changed my mind in 48 hours.)
Quoted 2 lines
Any opinion on this is very welcome.
Brave words!
My last thought here is that I still don't like the idea of QEMUMachine class changing its timeout behavior based on the introspection of wrapper args.
It feels much more like the case that a caller who is knowingly wrapping it with a program that delays its execution should change its parameters accordingly based on what the caller knows about what they're trying to accomplish.
Does that make the code too messy? I understand you probably want to ensure that adding a GDB wrapper is painless and simple, so it might not be great to always ask a caller to remember to set some timeout value to use it.
I figure that the right place to do this, though, is wherever the boilerplate code gets written that knows how to set up the right gdb args and so on, and not in machine.py. It sounds like iotests.py code to me, maybe in the VM class.
Quoted 5 lines
Spoiler alert I haven't tested these changes yet, but I am positive that there shouldn't be any problem. (Last famous words)
Emanuele
Clear as mud?
--js
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 15 lines
Quoted 6 lines
So the current plan I have for
_qmp_timeris:- As Max suggested, move it in
__init__and check there for the
wrapper contents. If we need to block forever (gdb, valgrind), we set it to None. Otherwise to 15 seconds. I think setting it always to None is not ideal, because if you are testing something that deadlocks (see my attempts to remove/add locks in QEMU multiqueue) and the socket is set to block forever, you don't know if the test is super slow or it just deadlocked.I agree with your concern on rational defaults, let's focus on that briefly:
Let's have QEMUMachine default to *no timeouts* moving forward, and have the timeouts be *opt-in*. This keeps the Machine class somewhat pure and free of opinions. The separation of mechanism and policy.
Next, instead of modifying hundreds of tests to opt-in to the timeout, let's modify the VM class in
iotests.pyto opt-in to that timeout, restoring the current "safe" behavior of iotests.The above items can happen in a single commit, preserving behavior in the bisect.
Quoted 8 lines
Finally, we can add a non-private property that individual tests can re-override to opt BACK out of the default.
Something as simple as:
vm.qmp_timeout = None
would be just fine.
I applied these suggested changes, will send v4 and we'll see what comes out of it.
Quoted 13 lines
Quoted 5 lines
Well, one can argue that in both cases this is not the expected behavior, but I think having an upper bound on each QMP command execution would be good.
- pass
_qmp_timerdirectly toself._qmp.accept()in_postlaunch,
leaving_launch()intact. I think this makes sense because as you also mentioned, changing_post_launch()into taking a parameter requires changing also all subclasses and pass values around.Sounds OK. If we do change the defaults back to "No Timeout" in a way that allows an override by an opinionated class, we'll already have the public property, though, so a parameter might not be needed.
(Yes, this is the THIRD time I've changed my mind in 48 hours.)
Quoted 2 lines
Any opinion on this is very welcome.
Quoted 8 lines
Brave words!
My last thought here is that I still don't like the idea of QEMUMachine class changing its timeout behavior based on the introspection of wrapper args.
It feels much more like the case that a caller who is knowingly wrapping it with a program that delays its execution should change its parameters accordingly based on what the caller knows about what they're trying to accomplish.
Does that make the code too messy? I understand you probably want to ensure that adding a GDB wrapper is painless and simple, so it might not be great to always ask a caller to remember to set some timeout value to use it.
I am not sure I follow you here, where do you want to move this logic? Can you please elaborate more, I did not understand what you mean.
I understand that tweaking the timers in iotests.py with checks like
if not (qemu_gdb or qemu_valgrind):
normal timer
may not be the most beautiful piece of code, but as you said it keeps things as simple as they can.
Quoted 1 line
I figure that the right place to do this, though, is wherever the boilerplate code gets written that knows how to set up the right gdb args and so on, and not in
machine.py. It sounds likeiotests.pycode to me, maybe in the VM class.
After the changes suggested on qmp_timeout, iotests.py already contains the only code to perform the setup right for gdb and valgrind, and machine.py is not touched (except for qmp_timeout). iotests.py will essentially contain a couple of ifs like the one above, changing the timer when gdb and valgring are *not* needed.
Emanuele
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 47 lines · Emanuele Giuseppe Esposito
Quoted 15 lines
Quoted 6 lines
So the current plan I have for
_qmp_timeris:- As Max suggested, move it in
__init__and check there for the
wrapper contents. If we need to block forever (gdb, valgrind), we set it to None. Otherwise to 15 seconds. I think setting it always to None is not ideal, because if you are testing something that deadlocks (see my attempts to remove/add locks in QEMU multiqueue) and the socket is set to block forever, you don't know if the test is super slow or it just deadlocked.I agree with your concern on rational defaults, let's focus on that briefly:
Let's have QEMUMachine default to *no timeouts* moving forward, and have the timeouts be *opt-in*. This keeps the Machine class somewhat pure and free of opinions. The separation of mechanism and policy.
Next, instead of modifying hundreds of tests to opt-in to the timeout, let's modify the VM class in
iotests.pyto opt-in to that timeout, restoring the current "safe" behavior of iotests.The above items can happen in a single commit, preserving behavior in the bisect.
Quoted 8 lines
Finally, we can add a non-private property that individual tests can re-override to opt BACK out of the default.
Something as simple as:
vm.qmp_timeout = None
would be just fine.
I applied these suggested changes, will send v4 and we'll see what comes out of it.
Quoted 13 lines
Quoted 5 lines
Well, one can argue that in both cases this is not the expected behavior, but I think having an upper bound on each QMP command execution would be good.
- pass
_qmp_timerdirectly toself._qmp.accept()in_postlaunch,
leaving_launch()intact. I think this makes sense because as you also mentioned, changing_post_launch()into taking a parameter requires changing also all subclasses and pass values around.Sounds OK. If we do change the defaults back to "No Timeout" in a way that allows an override by an opinionated class, we'll already have the public property, though, so a parameter might not be needed.
(Yes, this is the THIRD time I've changed my mind in 48 hours.)
Quoted 2 lines
Any opinion on this is very welcome.
Quoted 8 lines
Brave words!
My last thought here is that I still don't like the idea of QEMUMachine class changing its timeout behavior based on the introspection of wrapper args.
It feels much more like the case that a caller who is knowingly wrapping it with a program that delays its execution should change its parameters accordingly based on what the caller knows about what they're trying to accomplish.
Does that make the code too messy? I understand you probably want to ensure that adding a GDB wrapper is painless and simple, so it might not be great to always ask a caller to remember to set some timeout value to use it.
Quoted 9 lines
I am not sure I follow you here, where do you want to move this logic? Can you please elaborate more, I did not understand what you mean.
I understand that tweaking the timers in
iotests.pywith checks likeif not (
qemu_gdborqemu_valgrind):
normal timermay not be the most beautiful piece of code, but as you said it keeps things as simple as they can.
What I mean is that of the two patterns:
caller.py:
vm = machine(..., wrapper_args=['gdb', ...])
machine.py:
def __init__(...):
if 'gdb' in wrapper_args:
self.timer = None
vs this one:
caller.py:
vm = machine(..., wrapper_args=['gdb', ...], timer=None)
machine.py:
def __init__(...):
... # No introspection of wrapper_args
I prefer the second. I would assume it's possible to localize the logic that creates a GDB-wrapped machine alongside the knowledge that it needs the timeout turned off *outside* of the machine class.
I could be *very wrong* about that assumption though. The reason I prefer the second pattern is because it avoids having to deal with what happens when a caller specifies both a timeout and a gdb-wrapper. In the second case, the caller explicitly requested the timeout to be None, so anything that occurs afterwards is the fault of the caller, not machine.py.
To me, that's "simpler". (I could be wrong, I don't have a great overall view of your series, just the bits that I have seen that touch machine.py.)
--js
Quoted 6 lines
Quoted 1 line
I figure that the right place to do this, though, is wherever the boilerplate code gets written that knows how to set up the right gdb args and so on, and not in
machine.py. It sounds likeiotests.pycode to me, maybe in the VM class.After the changes suggested on
qmp_timeout,iotests.pyalready contains the only code to perform the setup right for gdb and valgrind, andmachine.pyis not touched (except forqmp_timeout).iotests.pywill essentially contain a couple of ifs like the one above, changing the timer when gdb and valgring are *not* needed.Emanuele
Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind
Quoted 78 lines · John Snow
Quoted 47 lines · Emanuele Giuseppe Esposito
Quoted 15 lines
Quoted 6 lines
So the current plan I have for
_qmp_timeris:- As Max suggested, move it in
__init__and check there for the
wrapper contents. If we need to block forever (gdb, valgrind), we set it to None. Otherwise to 15 seconds. I think setting it always to None is not ideal, because if you are testing something that deadlocks (see my attempts to remove/add locks in QEMU multiqueue) and the socket is set to block forever, you don't know if the test is super slow or it just deadlocked.I agree with your concern on rational defaults, let's focus on that briefly:
Let's have QEMUMachine default to *no timeouts* moving forward, and have the timeouts be *opt-in*. This keeps the Machine class somewhat pure and free of opinions. The separation of mechanism and policy.
Next, instead of modifying hundreds of tests to opt-in to the timeout, let's modify the VM class in
iotests.pyto opt-in to that timeout, restoring the current "safe" behavior of iotests.The above items can happen in a single commit, preserving behavior in the bisect.
Quoted 8 lines
Finally, we can add a non-private property that individual tests can re-override to opt BACK out of the default.
Something as simple as:
vm.qmp_timeout = None
would be just fine.
I applied these suggested changes, will send v4 and we'll see what comes out of it.
Quoted 13 lines
Quoted 5 lines
Well, one can argue that in both cases this is not the expected behavior, but I think having an upper bound on each QMP command execution would be good.
- pass
_qmp_timerdirectly toself._qmp.accept()in_postlaunch,
leaving_launch()intact. I think this makes sense because as you also mentioned, changing_post_launch()into taking a parameter requires changing also all subclasses and pass values around.Sounds OK. If we do change the defaults back to "No Timeout" in a way that allows an override by an opinionated class, we'll already have the public property, though, so a parameter might not be needed.
(Yes, this is the THIRD time I've changed my mind in 48 hours.)
Quoted 2 lines
Any opinion on this is very welcome.
Quoted 8 lines
Brave words!
My last thought here is that I still don't like the idea of QEMUMachine class changing its timeout behavior based on the introspection of wrapper args.
It feels much more like the case that a caller who is knowingly wrapping it with a program that delays its execution should change its parameters accordingly based on what the caller knows about what they're trying to accomplish.
Does that make the code too messy? I understand you probably want to ensure that adding a GDB wrapper is painless and simple, so it might not be great to always ask a caller to remember to set some timeout value to use it.
Quoted 9 lines
I am not sure I follow you here, where do you want to move this logic? Can you please elaborate more, I did not understand what you mean.
I understand that tweaking the timers in
iotests.pywith checks likeif not (
qemu_gdborqemu_valgrind):
normal timermay not be the most beautiful piece of code, but as you said it keeps things as simple as they can.
What I mean is that of the two patterns:
caller.py:
vm =machine(..., wrapper_args=['gdb', ...])
machine.py:
def__init__(...):
if 'gdb' inwrapper_args:
self.timer = Nonevs this one:
caller.py:
vm =machine(..., wrapper_args=['gdb', ...], timer=None)
machine.py:
def__init__(...):
... # No introspection ofwrapper_args
Quoted 5 lines
I prefer the second. I would assume it's possible to localize the logic that creates a GDB-wrapped machine alongside the knowledge that it needs the timeout turned off *outside* of the machine class.
I could be *very wrong* about that assumption though. The reason I prefer the second pattern is because it avoids having to deal with what happens when a caller specifies both a timeout and a gdb-wrapper. In the second case, the caller explicitly requested the timeout to be None, so anything that occurs afterwards is the fault of the caller, not
machine.py.To me, that's "simpler". (I could be wrong, I don't have a great overall view of your series, just the bits that I have seen that touch
machine.py.)
I think this can be done almost effortless. With your suggested changes on qmp_timer, we can have:
machine.py
def __init__(self, ..., wrapper, timer: None)
self._qmp_timer = timer
def _post_launch(self)
self._qmp.accept(self._qmp_timer)
iotests.py
timer = None if qemu_gdb or qemu_valgrind else 15.0
wrapper = qemu_gdb or qemu_valgrind # did not know about this OR trick btw vm = machine(..., wrapper, timer)
Thank you,
Emanuele
Quoted 10 lines
--js
Quoted 6 lines
Quoted 1 line
I figure that the right place to do this, though, is wherever the boilerplate code gets written that knows how to set up the right gdb args and so on, and not in
machine.py. It sounds likeiotests.pycode to me, maybe in the VM class.After the changes suggested on
qmp_timeout,iotests.pyalready contains the only code to perform the setup right for gdb and valgrind, andmachine.pyis not touched (except forqmp_timeout).iotests.pywill essentially contain a couple of ifs like the one above, changing the timer when gdb and valgring are *not* needed.Emanuele
PATCH v3 11/15 qemu_iotests: allow valgrind to read/delete the generated log file
In reply to Emanuele Giuseppe EspositoWhen using -valgrind on the script tests, it generates a log file in $TEST_DIR that is either read (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests.
--- tests/qemu-iotests/iotests.py | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
tests/qemu-iotests/iotests.py +20 -0
@@ -600,6 +600,26 @@ def __init__(self, path_suffix=''): sock_dir=sock_dir) self._num_drives = 0 + def subprocess_check_valgrind(self, valgrind) -> None: + + if not valgrind: + return + + valgrind_filename = test_dir + "/" + str(self._popen.pid) + ".valgrind" + + if self.exitcode() == 99: + with open(valgrind_filename) as f: + content = f.readlines() + for line in content: + print(line, end ="") + print("") + else: + os.remove(valgrind_filename) + + def _post_shutdown(self) -> None: + super()._post_shutdown() + self.subprocess_check_valgrind(qemu_valgrind) + def add_object(self, opts): self._args.append('-object') self._args.append(opts)
Signature · 1 line
2.30.2
Re: [PATCH v3 11/15] qemu_iotests: allow valgrind to read/delete the generated log file
Quoted 15 lines · Emanuele Giuseppe Esposito
When using -valgrind on the script tests, it generates a log file in $TEST_DIR that is either read (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/iotests.py | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)tests/qemu-iotests/iotests.py +0
-0
tests/qemu-iotests/iotests.py def __init__(self, path_suffix=''): · line 603
Quoted 4 lines
tests/qemu-iotests/iotests.py +1
-0@@ -600,6 +600,26 @@ def __init__(self, path_suffix=''): sock_dir=sock_dir) self._num_drives = 0 + def subprocess_check_valgrind(self, valgrind) -> None:
A type annotation would be nice. (I.e. List[str], or we make it a bool and the caller uses bool(qemu_valgrind).)
tests/qemu-iotests/iotests.py def __init__(self, path_suffix=''): · line 604
Quoted 1 line
tests/qemu-iotests/iotests.py +1
-0+
I’d drop this empty line.
tests/qemu-iotests/iotests.py def __init__(self, path_suffix=''): · line 608
Quoted 4 lines
tests/qemu-iotests/iotests.py +4
-0+ if not valgrind: + return + + valgrind_filename = test_dir + "/" + str(self._popen.pid) + ".valgrind"
mypy (iotest 297) complains that _popen is Optional[], so .pid might not exist. Perhaps this should be safeguarded in the "if not valgrind" condition (i.e. "if not valgrind or not self._popen").
Also, pylint complains about the line being too long (79 is the maximum length). Can be fixed with an f-string:
valgrind_filename = f"{test_dir}/{self._popen.pid}.valgrind"
tests/qemu-iotests/iotests.py def __init__(self, path_suffix=''): · line 614
Quoted 6 lines
tests/qemu-iotests/iotests.py +6
-0+ + if self.exitcode() == 99: + with open(valgrind_filename) as f: + content = f.readlines() + for line in content: + print(line, end ="")
'end=""' would be better, I think. (flake8 complains.)
Also, would this be better as:
with open(valgrind_filename) as f:
for line in f.readlines():
print(line, end="")
?
(Or just
with open(valgrind_filename) as f:
print(f.read())
– wouldn’t that work, too?)
Max
Quoted 12 lines
tests/qemu-iotests/iotests.py +8
-0+ print("") + else: + os.remove(valgrind_filename) + + def _post_shutdown(self) -> None: + super()._post_shutdown() + self.subprocess_check_valgrind(qemu_valgrind) + def add_object(self, opts): self._args.append('-object') self._args.append(opts)
PATCH v3 12/15 qemu_iotests: insert valgrind command line as wrapper for qemu binary
In reply to Emanuele Giuseppe EspositoThe priority will be given to gdb command line, meaning if the -gdb parameter and -valgrind are given, gdb will be wrapped around the qemu binary.
--- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
@@ -593,7 +593,8 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, + wrapper = qemu_gdb if qemu_gdb else qemu_valgrind + super().__init__(qemu_prog, qemu_opts, wrapper=wrapper, name=name, test_dir=test_dir, socket_scm_helper=socket_scm_helper,
Signature · 1 line
2.30.2
Re: [PATCH v3 12/15] qemu_iotests: insert valgrind command line as wrapper for qemu binary
Quoted 8 lines · Emanuele Giuseppe Esposito
The priority will be given to gdb command line, meaning if the -gdb parameter and -valgrind are given, gdb will be wrapped around the qemu binary.
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
PATCH v3 13/15 docs/devel/testing: add -valgrind option to the debug section of QEMU iotests
In reply to Emanuele Giuseppe Esposito--- docs/devel/testing.rst | 7 +++++++ 1 file changed, 7 insertions(+)
docs/devel/testing.rst +7 -0
@@ -236,6 +236,13 @@ given as options to the ``check`` script: by setting the ``$GDB_QEMU`` environmental variable. The final command line will be ``gdbserver $GDB_QEMU $QEMU ...`` +* ``-valgrind`` wraps a valgrind instance to QEMU. If it detects + warnings, it will print and save the log in + ``$TEST_DIR/<valgrind_pid>.valgrind``. + The final command line will be ``valgrind --log-file=$TEST_DIR/ + <valgrind_pid>.valgrind --error-exitcode=99 $QEMU ...`` + Note: if used together with ``-gdb``, this command will be ignored. + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers.
Signature · 1 line
2.30.2
Re: [PATCH v3 13/15] docs/devel/testing: add -valgrind option to the debug section of QEMU iotests
Quoted 10 lines · Emanuele Giuseppe Esposito
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- docs/devel/testing.rst | 7 +++++++ 1 file changed, 7 insertions(+)docs/devel/testing.rst +0
-0
docs/devel/testing.rst given as options to the ``check`` script: · line 239
Quoted 4 lines
docs/devel/testing.rst +1
-0@@ -236,6 +236,13 @@ given as options to the ``check`` script: by setting the ``$GDB_QEMU`` environmental variable. The final command line will be ``gdbserver $GDB_QEMU $QEMU ...`` +* ``-valgrind`` wraps a valgrind instance to QEMU. If it detects
Not a native speaker, but I think it should be “attaches ... to QEMU”, “wraps QEMU in a valgrind instances”, or “wraps ... around QEMU".
Apart from that:
Quoted 10 lines
docs/devel/testing.rst +6
-0+ warnings, it will print and save the log in + ``$TEST_DIR/<valgrind_pid>.valgrind``. + The final command line will be ``valgrind --log-file=$TEST_DIR/ + <valgrind_pid>.valgrind --error-exitcode=99 $QEMU ...`` + Note: if used together with ``-gdb``, this command will be ignored. + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers.
PATCH v3 14/15 qemu_iotests: add option to show qemu binary logs on stdout
In reply to Emanuele Giuseppe EspositoUsing the flag -p, allow the qemu binary to print to stdout. This helps especially when doing print-debugging.
--- tests/qemu-iotests/check | 3 ++- tests/qemu-iotests/iotests.py | 9 +++++++++ tests/qemu-iotests/testenv.py | 9 +++++++-- 3 files changed, 18 insertions(+), 3 deletions(-)
tests/qemu-iotests/check +2 -1
@@ -33,6 +33,7 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-p', dest='print', action='store_true', help='shows qemu binary prints to stdout') p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_QEMU options. \ Default is localhost:12345")
@@ -117,7 +118,7 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb) + gdb=args.gdb, qprint=args.print) testfinder = TestFinder(test_dir=env.source_iotests)
tests/qemu-iotests/iotests.py +9 -0
@@ -79,6 +79,8 @@ if os.environ.get('GDB_QEMU'): qemu_gdb = ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ') +qemu_print = os.environ.get('PRINT_QEMU', False) + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.')
@@ -621,6 +623,13 @@ def _post_shutdown(self) -> None: super()._post_shutdown() self.subprocess_check_valgrind(qemu_valgrind) + def _pre_launch(self) -> None: + super()._pre_launch() + if qemu_print and self._qemu_log_file != None: + # set QEMU binary output to stdout + self._qemu_log_file.close() + self._qemu_log_file = None + def add_object(self, opts): self._args.append('-object') self._args.append(opts)
tests/qemu-iotests/testenv.py +7 -2
@@ -73,7 +73,7 @@ class TestEnv(ContextManager['TestEnv']): 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', - 'GDB_QEMU'] + 'GDB_QEMU', 'PRINT_QEMU'] def get_env(self) -> Dict[str, str]: env = {}
@@ -165,7 +165,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, misalign: bool = False, debug: bool = False, valgrind: bool = False, - gdb: bool = False) -> None: + gdb: bool = False, + qprint: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode
@@ -173,6 +174,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if qprint: + self.print_qemu = 'y' + if gdb: self.gdb_qemu = os.environ.get('GDB_QEMU', 'localhost:12345') elif 'GDB_QEMU' in os.environ:
@@ -278,6 +282,7 @@ def print_env(self) -> None: SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_QEMU -- "{GDB_QEMU}" VALGRIND_QEMU -- "{VALGRIND_QEMU}" +PRINT_QEMU -- "{PRINT_QEMU}" """ args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.30.2
Re: [PATCH v3 14/15] qemu_iotests: add option to show qemu binary logs on stdout
Quoted 2 lines · Emanuele Giuseppe Esposito
Using the flag -p, allow the qemu binary to print to stdout. This helps especially when doing print-debugging.
I think this shouldn’t refer to prints but to qemu’s stdout/stderr in general, i.e....
Quoted 12 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- tests/qemu-iotests/check | 3 ++- tests/qemu-iotests/iotests.py | 9 +++++++++ tests/qemu-iotests/testenv.py | 9 +++++++-- 3 files changed, 18 insertions(+), 3 deletions(-)tests/qemu-iotests/check +0
-0
tests/qemu-iotests/check def make_argparser() -> argparse.ArgumentParser: · line 36
Quoted 4 lines
tests/qemu-iotests/check +1
-0@@ -33,6 +33,7 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-p', dest='print', action='store_true', help='shows qemu binary prints to stdout')
I’d prefer for the description to be “redirects qemu's stdout and stderr to the test output”.
Also, this line is too long.
Quoted 26 lines
tests/qemu-iotests/check +1
-1p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_QEMU options. \ Default is localhost:12345")@@ -117,7 +118,7 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb) + gdb=args.gdb, qprint=args.print) testfinder = TestFinder(test_dir=env.source_iotests)tests/qemu-iotests/iotests.py +2
-0@@ -79,6 +79,8 @@ if os.environ.get('GDB_QEMU'): qemu_gdb = ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ') +qemu_print = os.environ.get('PRINT_QEMU', False) + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.')
tests/qemu-iotests/iotests.py def _post_shutdown(self) -> None: · line 628
Quoted 6 lines
tests/qemu-iotests/iotests.py +3
-0@@ -621,6 +623,13 @@ def _post_shutdown(self) -> None: super()._post_shutdown() self.subprocess_check_valgrind(qemu_valgrind) + def _pre_launch(self) -> None: + super()._pre_launch() + if qemu_print and self._qemu_log_file != None:
I think "is not None" is mostly used in Python. (I’m saying this in this weird way because I’m not the one to ask what’s mostly used in Python...)
(That also silences mypy, which otherwise complains and then fails 297.)
tests/qemu-iotests/iotests.py def _post_shutdown(self) -> None: · line 631
Quoted 3 lines
tests/qemu-iotests/iotests.py +3
-0+ # set QEMU binary output to stdout + self._qemu_log_file.close() + self._qemu_log_file = None
I don’t know enough Python to know how this works. I suppose this does access the super class’s member? (I could also imagine it creates a new member, just for this child class, but that then effectively overwrites the super class’s member.)
Considering _qemu_log_file is apparently meant to be a private attribute (from the leading underscore), I think it would be better to have a function provided by the super class for this.
Quoted 38 lines
tests/qemu-iotests/iotests.py +1
-0+ def add_object(self, opts): self._args.append('-object') self._args.append(opts)tests/qemu-iotests/testenv.py +6
-2@@ -73,7 +73,7 @@ class TestEnv(ContextManager['TestEnv']): 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', - 'GDB_QEMU'] + 'GDB_QEMU', 'PRINT_QEMU'] def get_env(self) -> Dict[str, str]: env = {}@@ -165,7 +165,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, misalign: bool = False, debug: bool = False, valgrind: bool = False, - gdb: bool = False) -> None: + gdb: bool = False, + qprint: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode@@ -173,6 +174,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if qprint: + self.print_qemu = 'y' + if gdb: self.gdb_qemu = os.environ.get('GDB_QEMU', 'localhost:12345') elif 'GDB_QEMU' in os.environ:
tests/qemu-iotests/testenv.py def print_env(self) -> None: · line 285
Quoted 4 lines
tests/qemu-iotests/testenv.py +1
-0@@ -278,6 +282,7 @@ def print_env(self) -> None: SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_QEMU -- "{GDB_QEMU}" VALGRIND_QEMU -- "{VALGRIND_QEMU}" +PRINT_QEMU -- "{PRINT_QEMU}"
Again, I personally don’t like the quotes.
(I think “PRINT_QEMU -- y” and “PRINT_QEMU --” look better.)
Another thing: Here in this place, the name of the environment variable is visible. “PRINT_QEMU” doesn’t really have meaning; now, if it weren’t visible, I wouldn’t care, but here it is visible. Perhaps “PRINT_QEMU_OUTPUT”, or "SHOW_QEMU_OUTPUT" would mean more?
Well, it’s all bike-shedding, so with “is not None” and the add_argument line broken so it doesn’t exceed 79 characters:
Quoted 4 lines
tests/qemu-iotests/testenv.py +0
-0""" args = collections.defaultdict(str, self.get_env())
Re: [PATCH v3 14/15] qemu_iotests: add option to show qemu binary logs on stdout
Quoted 60 lines · Max Reitz
Quoted 2 lines · Emanuele Giuseppe Esposito
Using the flag -p, allow the qemu binary to print to stdout. This helps especially when doing print-debugging.
I think this shouldn’t refer to prints but to qemu’s stdout/stderr in general, i.e....
Quoted 12 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
tests/qemu-iotests/check | 3 ++-
tests/qemu-iotests/iotests.py| 9 +++++++++
tests/qemu-iotests/testenv.py| 9 +++++++--3 files changed, 18 insertions(+), 3 deletions(-)tests/qemu-iotests/check +0
-0tests/qemu-iotests/check def make_argparser() -> argparse.ArgumentParser: · line 36
Quoted 4 lines
tests/qemu-iotests/check +1
-0@@ -33,6 +33,7 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-p', dest='print', action='store_true', help='shows qemu binary prints to stdout')I’d prefer for the description to be “redirects qemu's stdout and stderr to the test output”.
Also, this line is too long.
Quoted 23 lines
tests/qemu-iotests/check +1
-1p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_QEMU options. \ Default is localhost:12345")@@ -117,7 +118,7 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb) + gdb=args.gdb, qprint=args.print) testfinder = TestFinder(test_dir=env.source_iotests)tests/qemu-iotests/iotests.py +0
-0@@ -79,6 +79,8 @@if
os.environ.get('GDB_QEMU'):
qemu_gdb= ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ')tests/qemu-iotests/iotests.py +2
-0+qemu_print = os.environ.get('PRINT_QEMU', False) +imgfmt =
os.environ.get('IMGFMT', 'raw')
imgproto =os.environ.get('IMGPROTO', 'file')
output_dir=os.environ.get('OUTPUT_DIR', '.')tests/qemu-iotests/iotests.py def _post_shutdown(self) -> None: · line 627
Quoted 5 lines
tests/qemu-iotests/iotests.py +3
-0@@ -621,6 +623,13 @@ def _post_shutdown(self) -> None: super()._post_shutdown() self.subprocess_check_valgrind(qemu_valgrind) + def _pre_launch(self) -> None: + super()._pre_launch() + if qemu_print and self._qemu_log_file != None:I think "is not None" is mostly used in Python. (I’m saying this in this weird way because I’m not the one to ask what’s mostly used in Python...)
(That also silences mypy, which otherwise complains and then fails 297.)
Quoted 7 lines
tests/qemu-iotests/iotests.py def _post_shutdown(self) -> None: · line 631
Quoted 3 lines
tests/qemu-iotests/iotests.py +3
-0+ # set QEMU binary output to stdout + self._qemu_log_file.close() + self._qemu_log_file = NoneI don’t know enough Python to know how this works. I suppose this does access the super class’s member? (I could also imagine it creates a new member, just for this child class, but that then effectively overwrites the super class’s member.)
Considering
_qemu_log_fileis apparently meant to be a private attribute (from the leading underscore), I think it would be better to have a function provided by the super class for this.
It should access the superclass member, and it's the same that _post_shutdown does. I can make a function for that.
Regarding all other feedback in all patches that I did not answer: agree on all of them, will adjust everything in v4.
Thank you,
Emanuele
Quoted 58 lines
Quoted 40 lines
tests/qemu-iotests/iotests.py +1
-0+def
add_object(self, opts):
self._args.append('-object')
self._args.append(opts)tests/qemu-iotests/testenv.py +1
-1@@ -73,7 +73,7 @@ class TestEnv(ContextManager['TestEnv']): 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', - 'GDB_QEMU'] + 'GDB_QEMU', 'PRINT_QEMU']def
get_env(self)-> Dict[str, str]:
env = {}tests/qemu-iotests/testenv.py +2
-1@@ -165,7 +165,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, misalign: bool = False, debug: bool = False, valgrind: bool = False, - gdb: bool = False) -> None: + gdb: bool = False, + qprint: bool = False) -> None:self.imgfmt = imgfmt
self.imgproto = imgproto
self.aiomode = aiomodetests/qemu-iotests/testenv.py +0
-0@@ -173,6 +174,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,self.misalign = misalign
self.debug = debug
+ if qprint:
+ self.print_qemu = 'y'
+
if gdb:
self.gdb_qemu =os.environ.get('GDB_QEMU', 'localhost:12345')elif 'GDB_QEMU' in os.environ:tests/qemu-iotests/testenv.py +0
-0@@ -278,6 +282,7 @@ def print_env(self) -> None: SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}Quoted 3 lines
GDB_QEMU -- "{
GDB_QEMU}"
VALGRIND_QEMU -- "{VALGRIND_QEMU}"
+PRINT_QEMU -- "{PRINT_QEMU}"Again, I personally don’t like the quotes.
(I think “PRINT_QEMU -- y” and “PRINT_QEMU --” look better.)
Another thing: Here in this place, the name of the environment variable is visible. “PRINT_QEMU” doesn’t really have meaning; now, if it weren’t visible, I wouldn’t care, but here it is visible. Perhaps “PRINT_QEMU_OUTPUT”, or "
SHOW_QEMU_OUTPUT" would mean more?Well, it’s all bike-shedding, so with “is not None” and the
add_argumentline broken so it doesn’t exceed 79 characters:Reviewed-by Max Reitz <[email protected]>Quoted 3 lines
"""
args =collections.defaultdict(str, self.get_env())
Re: [PATCH v3 14/15] qemu_iotests: add option to show qemu binary logs on stdout
Quoted 67 lines · Emanuele Giuseppe Esposito
Quoted 64 lines · Max Reitz
Quoted 2 lines · Emanuele Giuseppe Esposito
Using the flag -p, allow the qemu binary to print to stdout. This helps especially when doing print-debugging.
I think this shouldn’t refer to prints but to qemu’s stdout/stderr in general, i.e....
Quoted 12 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
tests/qemu-iotests/check | 3 ++-
tests/qemu-iotests/iotests.py| 9 +++++++++
tests/qemu-iotests/testenv.py| 9 +++++++--3 files changed, 18 insertions(+), 3 deletions(-)tests/qemu-iotests/check +0
-0tests/qemu-iotests/check def make_argparser() -> argparse.ArgumentParser: · line 36
Quoted 4 lines
tests/qemu-iotests/check +1
-0@@ -33,6 +33,7 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-p', dest='print', action='store_true', help='shows qemu binary prints to stdout')I’d prefer for the description to be “redirects qemu's stdout and stderr to the test output”.
Also, this line is too long.
Quoted 23 lines
tests/qemu-iotests/check +1
-1p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_QEMU options. \ Default is localhost:12345")@@ -117,7 +118,7 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb) + gdb=args.gdb, qprint=args.print) testfinder = TestFinder(test_dir=env.source_iotests)tests/qemu-iotests/iotests.py +0
-0@@ -79,6 +79,8 @@if
os.environ.get('GDB_QEMU'):
qemu_gdb= ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ')tests/qemu-iotests/iotests.py +2
-0+qemu_print = os.environ.get('PRINT_QEMU', False) +imgfmt =
os.environ.get('IMGFMT', 'raw')
imgproto =os.environ.get('IMGPROTO', 'file')
output_dir=os.environ.get('OUTPUT_DIR', '.')tests/qemu-iotests/iotests.py def _post_shutdown(self) -> None: · line 627
Quoted 5 lines
tests/qemu-iotests/iotests.py +3
-0@@ -621,6 +623,13 @@ def _post_shutdown(self) -> None: super()._post_shutdown() self.subprocess_check_valgrind(qemu_valgrind) + def _pre_launch(self) -> None: + super()._pre_launch() + if qemu_print and self._qemu_log_file != None:I think "is not None" is mostly used in Python. (I’m saying this in this weird way because I’m not the one to ask what’s mostly used in Python...)
(That also silences mypy, which otherwise complains and then fails 297.)
Quoted 3 lines
tests/qemu-iotests/iotests.py +3
-0+ # set QEMU binary output to stdout + self._qemu_log_file.close() + self._qemu_log_file = None
Quoted 5 lines
Quoted 3 lines
I don’t know enough Python to know how this works. I suppose this does access the super class’s member? (I could also imagine it creates a new member, just for this child class, but that then effectively overwrites the super class’s member.)
Considering
_qemu_log_fileis apparently meant to be a private attribute (from the leading underscore), I think it would be better to have a function provided by the super class for this.It should access the superclass member, and it's the same that
_post_shutdowndoes. I can make a function for that.
Understood.
Quoted 1 line
Regarding all other feedback in all patches that I did not answer: agree on all of them, will adjust everything in v4.
Thanks, looking forward to it! :)
Max
PATCH v3 15/15 docs/devel/testing: add -p option to the debug section of QEMU iotests
In reply to Emanuele Giuseppe Esposito--- docs/devel/testing.rst | 4 ++++ 1 file changed, 4 insertions(+)
docs/devel/testing.rst +4 -0
@@ -246,6 +246,10 @@ given as options to the ``check`` script: * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers. +* ``-p`` (print) allows QEMU binary stdout to be shown in the + test console, instead of saving it into a log file in + ``$TEST_DIR/qemu-machine-<random_string>``. + Test case groups ----------------
Signature · 1 line
2.30.2
Re: [PATCH v3 15/15] docs/devel/testing: add -p option to the debug section of QEMU iotests
Quoted 10 lines · Emanuele Giuseppe Esposito
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- docs/devel/testing.rst | 4 ++++ 1 file changed, 4 insertions(+)docs/devel/testing.rst +0
-0
docs/devel/testing.rst given as options to the ``check`` script: · line 249
Quoted 4 lines
docs/devel/testing.rst +1
-0@@ -246,6 +246,10 @@ given as options to the ``check`` script: * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers. +* ``-p`` (print) allows QEMU binary stdout to be shown in the
stderr, too.
docs/devel/testing.rst given as options to the ``check`` script: · line 252
Quoted 3 lines
docs/devel/testing.rst +3
-0+ test console, instead of saving it into a log file in + ``$TEST_DIR/qemu-machine-<random_string>``. +
It doesn’t allow this, though, it forces it. That means that tests that use the log will fail (e.g. 245)[1]. That is, I’d drop the “allows” and just state “redirect QEMU’s stdout and stderr to the test output, instead of...”.
[1] I realize that all tests will technically fail with -p, because the reference output differs, but 245 will not just fail because of that difference, but because two of its cases actually really fail. No need to make a note of this, though. It’s just that “allows” sounds a bit like qemu could choose to put some info into the test output and some into the log, when really it’s all or nothing (-p or not -p).
Max
Quoted 4 lines
docs/devel/testing.rst +0
-0Test case groups ----------------