Skip to contentSkip to conversation

qemu_iotests: improve debugging options

RFC PATCH Superseded Emanuele Giuseppe Esposito 4 participants 27 messages Apr 7, 2021 – Apr 9, 2021

v2 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v2 cover qemu_iotests: improve debugging options

This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.

Patches 1-6 introduce the -gdb option to both python and bash tests, 7-10 extend the already existing -valgrind flag to work also on python tests, and patch 11 introduces -p to enable logging to stdout.

In particular, patches 1,2,4,8 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.

This series is tested on the previous serie "qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

---
v2:
- add valgrind and print patches
- better splitup of patches, and clearer commit messages

Emanuele Giuseppe Esposito (11):
python: qemu: add timer parameter for qmp.accept socket
python: qemu: pass the wrapper field from QEMUQtestmachine to
QEMUMachine
qemu-iotests: add option to attach gdbserver
qemu-iotests: delay QMP socket timers
qemu_iotests: insert gdbserver command line as wrapper for qemu binary
qemu-iotests: add gdbserver option to script tests too
qemu_iotests: extend the check script to support valgrind for python
tests
qemu_iotests: extent QMP socket timeout when using valgrind
qemu_iotests: allow valgrint to print/delete the generated log file
qemu_iotests: insert valgrind command line as wrapper for qemu binary
qemu_iotests: add option to show qemu binary logs on stdout

 python/qemu/machine.py        | 12 ++++--
 python/qemu/qtest.py          |  8 ++--
 tests/qemu-iotests/check      |  7 +++-
 tests/qemu-iotests/common.rc  |  8 +++-
 tests/qemu-iotests/iotests.py | 69 ++++++++++++++++++++++++++++++++---
 tests/qemu-iotests/testenv.py | 24 ++++++++++--
 6 files changed, 111 insertions(+), 17 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

RFC PATCH v2 1/11 python: qemu: add timer parameter for qmp.accept socket

Extend the _post_launch function to include the timer as parameter instead of defaulting to 15 sec.

---
 python/qemu/machine.py | 4 ++--
 python/qemu/qtest.py   | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
python/qemu/machine.py +2 -2
index 6e44bda337..c721e07d63 100644
@@ -321,9 +321,9 @@ def _pre_launch(self) -> None:
                 nickname=self._name
             )
 
-    def _post_launch(self) -> None:
+    def _post_launch(self, timer) -> None:
         if self._qmp_connection:
-            self._qmp.accept()
+            self._qmp.accept(timer)
 
     def _post_shutdown(self) -> None:
         """
python/qemu/qtest.py +2 -2
index 39a0cf62fe..0d01715086 100644
@@ -138,9 +138,9 @@ def _pre_launch(self) -> None:
         super()._pre_launch()
         self._qtest = QEMUQtestProtocol(self._qtest_path, server=True)
 
-    def _post_launch(self) -> None:
+    def _post_launch(self, timer) -> None:
         assert self._qtest is not None
-        super()._post_launch()
+        super()._post_launch(timer)
         self._qtest.accept()
 
     def _post_shutdown(self) -> None:
Signature · 1 line
2.30.2
#2 lore raw

Re: [RFC PATCH v2 01/11] python: qemu: add timer parameter for qmp.accept socket

Quoted 32 lines · Emanuele Giuseppe Esposito

Extend the _post_launch function to include the timer as parameter instead of defaulting to 15 sec.

---
  python/qemu/machine.py | 4 ++--
  python/qemu/qtest.py   | 4 ++--
  2 files changed, 4 insertions(+), 4 deletions(-)
python/qemu/machine.py +2 -2
index 6e44bda337..c721e07d63 100644
@@ -321,9 +321,9 @@ def _pre_launch(self) -> None:
                  nickname=self._name
              )
 
-    def _post_launch(self) -> None:
+    def _post_launch(self, timer) -> None:
          if self._qmp_connection:
-            self._qmp.accept()
+            self._qmp.accept(timer)
 
      def _post_shutdown(self) -> None:
          """
python/qemu/qtest.py +0 -0
index 39a0cf62fe..0d01715086 100644
@@ -138,9 +138,9 @@ def _pre_launch(self) -> None:
          super()._pre_launch()

python/qemu/qtest.py def _pre_launch(self) -> None: · line 146

Quoted 10 lines
python/qemu/qtest.py +2 -2
          self._qtest = QEMUQtestProtocol(self._qtest_path, server=True)
 
-    def _post_launch(self) -> None:
+    def _post_launch(self, timer) -> None:
          assert self._qtest is not None
-        super()._post_launch()
+        super()._post_launch(timer)
          self._qtest.accept()
 
      def _post_shutdown(self) -> None:

Are you forgetting to change _launch() to provide some default value for what timer needs to be?

I think for the "event" callbacks here, I'd prefer configuring the behavior as a property instead of passing it around as a parameter.

(Also, we have an awful lot of timeouts now... is it time to think about rewriting this using asyncio so that we can allow the callers to specify their own timeouts in with context blocks? Just a thought for later; we have an awful lot of timeouts scattered throughout machine.py, qmp.py, etc.)

--js

#3 lore raw

Re: [RFC PATCH v2 01/11] python: qemu: add timer parameter for qmp.accept socket

Quoted 17 lines
Quoted 8 lines
python/qemu/qtest.py +1 -1
@@ -138,9 +138,9 @@ def _pre_launch(self) -> None:
           super()._pre_launch()
           self._qtest = QEMUQtestProtocol(self._qtest_path, server=True)
-    def _post_launch(self) -> None:
+    def _post_launch(self, timer) -> None:
Quoted 6 lines

          assert self._qtest is not None
-        super()._post_launch()
+        super()._post_launch(timer)
          self._qtest.accept()
      def _post_shutdown(self) -> None:

Are you forgetting to change _launch() to provide some default value for what timer needs to be?

Quoted 1 line

I think for the "event" callbacks here, I'd prefer configuring the behavior as a property instead of passing it around as a parameter.

I agree, I changed it in a field of the QEMUMachine class called _qmp_timer that defaults to 15 seconds.

Emanuele

#4 lore raw

RFC PATCH v2 2/11 python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

In reply to Emanuele Giuseppe Esposito
---
 python/qemu/machine.py | 2 +-
 python/qemu/qtest.py   | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
python/qemu/machine.py +1 -1
index c721e07d63..18d32ebe45 100644
@@ -109,7 +109,7 @@ def __init__(self,
 
         self._binary = binary
         self._args = list(args)
-        self._wrapper = wrapper
+        self._wrapper = list(wrapper)
 
         self._name = name or "qemu-%d" % os.getpid()
         self._test_dir = test_dir
python/qemu/qtest.py +3 -1
index 0d01715086..4c90daf430 100644
@@ -111,6 +111,7 @@ class QEMUQtestMachine(QEMUMachine):
     def __init__(self,
                  binary: str,
                  args: Sequence[str] = (),
+                 wrapper: Sequence[str] = (),
                  name: Optional[str] = None,
                  test_dir: str = "/var/tmp",
                  socket_scm_helper: Optional[str] = None,
@@ -119,7 +120,8 @@ def __init__(self,
             name = "qemu-%d" % os.getpid()
         if sock_dir is None:
             sock_dir = test_dir
-        super().__init__(binary, args, name=name, test_dir=test_dir,
+        super().__init__(binary, args, wrapper=wrapper, name=name,
+                         test_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
                          sock_dir=sock_dir)
         self._qtest: Optional[QEMUQtestProtocol] = None
Signature · 1 line
2.30.2
#5 lore raw

Re: [RFC PATCH v2 02/11] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

Quoted 11 lines · Emanuele Giuseppe Esposito
---
  python/qemu/machine.py | 2 +-
  python/qemu/qtest.py   | 4 +++-
  2 files changed, 4 insertions(+), 2 deletions(-)
python/qemu/machine.py +0 -0
index c721e07d63..18d32ebe45 100644

python/qemu/machine.py def __init__(self, · line 113

Quoted 6 lines
python/qemu/machine.py +1 -1
@@ -109,7 +109,7 @@ def __init__(self,
 
          self._binary = binary
          self._args = list(args)
-        self._wrapper = wrapper
+        self._wrapper = list(wrapper)
 

Unrelated change?

(I'm assuming you want to copy the user's input to explicitly avoid sharing state. Commit message blurb for this would be good.)

Quoted 16 lines
python/qemu/machine.py +0 -0
          self._name = name or "qemu-%d" % os.getpid()
          self._test_dir = test_dir
python/qemu/qtest.py +1 -0
index 0d01715086..4c90daf430 100644
@@ -111,6 +111,7 @@ class QEMUQtestMachine(QEMUMachine):
      def __init__(self,
                   binary: str,
                   args: Sequence[str] = (),
+                 wrapper: Sequence[str] = (),
                   name: Optional[str] = None,
                   test_dir: str = "/var/tmp",
                   socket_scm_helper: Optional[str] = None,

python/qemu/qtest.py def __init__(self, · line 127

Quoted 9 lines
python/qemu/qtest.py +2 -1
@@ -119,7 +120,8 @@ def __init__(self,
              name = "qemu-%d" % os.getpid()
          if sock_dir is None:
              sock_dir = test_dir
-        super().__init__(binary, args, name=name, test_dir=test_dir,
+        super().__init__(binary, args, wrapper=wrapper, name=name,
+                         test_dir=test_dir,
                           socket_scm_helper=socket_scm_helper,
                           sock_dir=sock_dir)
          self._qtest: Optional[QEMUQtestProtocol] = None

ACK

#6 lore raw

Re: [RFC PATCH v2 02/11] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

Quoted 14 lines
Quoted 11 lines
python/qemu/machine.py +0 -0
index c721e07d63..18d32ebe45 100644
@@ -109,7 +109,7 @@ def __init__(self,

          self._binary = binary
          self._args = list(args)
-        self._wrapper = wrapper
+        self._wrapper = list(wrapper)

Unrelated change?

Quoted 1 line

(I'm assuming you want to copy the user's input to explicitly avoid sharing state. Commit message blurb for this would be good.)

Yes, unrelated change. I do not see the benefit of copying the user state. I will drop it.

Quoted 3 lines
Quoted 1 line

ACK

(Apologies for the ignorance, is this an Acked-by?)

Emanuele

#7 lore raw

Re: [RFC PATCH v2 02/11] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

Quoted 20 lines · Emanuele Giuseppe Esposito
Quoted 14 lines
Quoted 11 lines
python/qemu/machine.py +0 -0
index c721e07d63..18d32ebe45 100644
@@ -109,7 +109,7 @@ def __init__(self,

          self._binary = binary
          self._args = list(args)
-        self._wrapper = wrapper
+        self._wrapper = list(wrapper)

Unrelated change?

Quoted 1 line

(I'm assuming you want to copy the user's input to explicitly avoid sharing state. Commit message blurb for this would be good.)

Yes, unrelated change. I do not see the benefit of copying the user state. I will drop it.

Quoted 2 lines
Quoted 1 line
Quoted 6 lines
Quoted 1 line

ACK

(Apologies for the ignorance, is this an Acked-by?)

Emanuele

Ah, yes. I just mean to say: "Minor style stuff, but looks good to me and I'll [likely] approve the non-RFC versions of these."

(Pending tests passing via flake8/pylint/mypy. Sorry those aren't formalized in the tree yet.)

--js

#8 lore raw

RFC PATCH v2 3/11 qemu-iotests: add option to attach gdbserver

In reply to Emanuele Giuseppe Esposito

Add -gdb flag and GDB_QEMU environmental variable to python tests to attach a gdbserver to each qemu instance.

if -gdb is not provided but $GDB_QEMU is set, ignore the environmental variable.

---
 tests/qemu-iotests/check      |  6 +++++-
 tests/qemu-iotests/iotests.py |  4 ++++
 tests/qemu-iotests/testenv.py | 18 +++++++++++++++---
 3 files changed, 24 insertions(+), 4 deletions(-)
tests/qemu-iotests/check +5 -1
index d1c87ceaf1..6186495eee 100755
@@ -33,6 +33,9 @@ def make_argparser() -> argparse.ArgumentParser:
                    help='pretty print output for make check')
 
     p.add_argument('-d', dest='debug', action='store_true', help='debug')
+    p.add_argument('-gdb', action='store_true',
+                   help="start gdbserver with $GDB_QEMU options. \
+                         Default is localhost:12345")
     p.add_argument('-misalign', action='store_true',
                    help='misalign memory allocations')
     p.add_argument('--color', choices=['on', 'off', 'auto'],
@@ -112,7 +115,8 @@ if __name__ == '__main__':
     env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto,
                   aiomode=args.aiomode, cachemode=args.cachemode,
                   imgopts=args.imgopts, misalign=args.misalign,
-                  debug=args.debug, valgrind=args.valgrind)
+                  debug=args.debug, valgrind=args.valgrind,
+                  gdb=args.gdb)
 
     testfinder = TestFinder(test_dir=env.source_iotests)
 
tests/qemu-iotests/iotests.py +4 -0
index 90d0b62523..05d0dc0751 100644
@@ -75,6 +75,10 @@
 qemu_prog = os.environ.get('QEMU_PROG', 'qemu')
 qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ')
 
+qemu_gdb = []
+if os.environ.get('GDB_QEMU'):
+    qemu_gdb = ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ')
+
 imgfmt = os.environ.get('IMGFMT', 'raw')
 imgproto = os.environ.get('IMGPROTO', 'file')
 output_dir = os.environ.get('OUTPUT_DIR', '.')
tests/qemu-iotests/testenv.py +15 -3
index 1fbec854c1..669eb6b925 100644
@@ -72,7 +72,8 @@ class TestEnv(ContextManager['TestEnv']):
                      'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO',
                      'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU',
                      'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX',
-                     'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_']
+                     'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_',
+                     'GDB_QEMU']
 
     def get_env(self) -> Dict[str, str]:
         env = {}
@@ -163,7 +164,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
                  imgopts: Optional[str] = None,
                  misalign: bool = False,
                  debug: bool = False,
-                 valgrind: bool = False) -> None:
+                 valgrind: bool = False,
+                 gdb: bool = False) -> None:
         self.imgfmt = imgfmt
         self.imgproto = imgproto
         self.aiomode = aiomode
@@ -171,6 +173,14 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
         self.misalign = misalign
         self.debug = debug
 
+        self.gdb_qemu = os.getenv('GDB_QEMU')
+
+        if gdb and not self.gdb_qemu:
+            self.gdb_qemu = 'localhost:12345'
+        elif self.gdb_qemu and not gdb:
+            del self.gdb_qemu
+            del os.environ['GDB_QEMU']
+
         if valgrind:
             self.valgrind_qemu = 'y'
 
@@ -268,7 +278,9 @@ def print_env(self) -> None:
 PLATFORM      -- {platform}
 TEST_DIR      -- {TEST_DIR}
 SOCK_DIR      -- {SOCK_DIR}
-SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}"""
+SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
+GDB_QEMU      -- "{GDB_QEMU}"
+"""
 
         args = collections.defaultdict(str, self.get_env())
 
Signature · 1 line
2.30.2
#9 lore raw

Re: [RFC PATCH v2 03/11] qemu-iotests: add option to attach gdbserver

tests/qemu-iotests/testenv.py def __init__(self, imgfmt: str, imgproto: str, aiomode: str, · line 182

Quoted 7 lines · Emanuele Giuseppe Esposito
tests/qemu-iotests/testenv.py +7 -0
+        self.gdb_qemu = os.getenv('GDB_QEMU')
+
+        if gdb and not self.gdb_qemu:
+            self.gdb_qemu = 'localhost:12345'
+        elif self.gdb_qemu and not gdb:
+            del self.gdb_qemu
+            del os.environ['GDB_QEMU']

Alternatively:

if gdb:
self.gdb_qemu = os.environ.get('GDB_QEMU', 'localhost:12345')
elif 'GDB_QEMU' in os.environ:
del os.environ['GDB_QEMU']

Quoted 1 line

+GDB_QEMU -- "{GDB_QEMU}"

Perhaps only include this if gdbserver is actually in use? (Or not at all, since gdbserver anyway prints the port).

Paolo

#10 lore raw

Re: [RFC PATCH v2 03/11] qemu-iotests: add option to attach gdbserver

Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Emanuele Giuseppe Esposito

+        self.gdb_qemu = os.getenv('GDB_QEMU')
+
+        if gdb and not self.gdb_qemu:
+            self.gdb_qemu = 'localhost:12345'
+        elif self.gdb_qemu and not gdb:
+            del self.gdb_qemu
+            del os.environ['GDB_QEMU']

Quoted 6 lines

Alternatively:

    if gdb:
        self.gdb_qemu = os.environ.get('GDB_QEMU', 'localhost:12345') elif 'GDB_QEMU' in os.environ:
        del os.environ['GDB_QEMU']

makes sense, thanks.

Quoted 4 lines
Quoted 2 lines

+GDB_QEMU      -- "{GDB_QEMU}"

Perhaps only include this if gdbserver is actually in use?  (Or not at all, since gdbserver anyway prints the port).

You forgot that by default all logs go to a log file :) so unless you find the corresponding log, it is not easy to find the GDB port.

Thank you,
Emanuele

Quoted 3 lines

Paolo

#11 lore raw

RFC PATCH v2 4/11 qemu-iotests: delay QMP socket timers

In reply to Emanuele Giuseppe Esposito

Attaching a gdbserver implies that the qmp socket
should wait indefinitely for an answer from QEMU.

---
 python/qemu/machine.py        |  4 +++-
 tests/qemu-iotests/iotests.py | 21 +++++++++++++++++----
 2 files changed, 20 insertions(+), 5 deletions(-)
python/qemu/machine.py +3 -1
index 18d32ebe45..284b73385f 100644
@@ -408,7 +408,9 @@ def _launch(self) -> None:
                                        stderr=subprocess.STDOUT,
                                        shell=False,
                                        close_fds=False)
-        self._post_launch()
+
+        timer = None if 'gdbserver' in self._wrapper else 15.0
+        self._post_launch(timer)
 
     def _early_cleanup(self) -> None:
         """
tests/qemu-iotests/iotests.py +17 -4
index 05d0dc0751..17f07710db 100644
@@ -686,7 +686,10 @@ def qmp_to_opts(self, obj):
 
     def get_qmp_events_filtered(self, wait=60.0):
         result = []
-        for ev in self.get_qmp_events(wait=wait):
+        qmp_wait = wait
+        if qemu_gdb:
+            qmp_wait = 0.0
+        for ev in self.get_qmp_events(wait=qmp_wait):
             result.append(filter_qmp_event(ev))
         return result
 
@@ -987,13 +990,17 @@ def cancel_and_wait(self, drive='drive0', force=False,
         result = self.vm.qmp('block-job-cancel', device=drive, force=force)
         self.assert_qmp(result, 'return', {})
 
+        qmp_wait = wait
+        if qemu_gdb:
+            qmp_wait = 0.0
+
         if resume:
             self.vm.resume_drive(drive)
 
         cancelled = False
         result = None
         while not cancelled:
-            for event in self.vm.get_qmp_events(wait=wait):
+            for event in self.vm.get_qmp_events(wait=qmp_wait):
                 if event['event'] == 'BLOCK_JOB_COMPLETED' or \
                    event['event'] == 'BLOCK_JOB_CANCELLED':
                     self.assert_qmp(event, 'data/device', drive)
@@ -1009,8 +1016,11 @@ def cancel_and_wait(self, drive='drive0', force=False,
     def wait_until_completed(self, drive='drive0', check_offset=True,
                              wait=60.0, error=None):
         '''Wait for a block job to finish, returning the event'''
+        qmp_wait = wait
+        if qemu_gdb:
+            qmp_wait = 0.0
         while True:
-            for event in self.vm.get_qmp_events(wait=wait):
+            for event in self.vm.get_qmp_events(wait=qmp_wait):
                 if event['event'] == 'BLOCK_JOB_COMPLETED':
                     self.assert_qmp(event, 'data/device', drive)
                     if error is None:
@@ -1054,7 +1064,10 @@ def complete_and_wait(self, drive='drive0', wait_ready=True,
         self.assertTrue(event['data']['type'] in ['mirror', 'commit'])
 
     def pause_wait(self, job_id='job0'):
-        with Timeout(3, "Timeout waiting for job to pause"):
+        def_timeout = 3
+        if qemu_gdb:
+            def_timeout = 3000
+        with Timeout(def_timeout, "Timeout waiting for job to pause"):
             while True:
                 result = self.vm.qmp('query-block-jobs')
                 found = False
Signature · 1 line
2.30.2
#12 lore raw

Re: [RFC PATCH v2 04/11] qemu-iotests: delay QMP socket timers

tests/qemu-iotests/iotests.py def qmp_to_opts(self, obj): · line 694

Quoted 9 lines · Emanuele Giuseppe Esposito
      def get_qmp_events_filtered(self, wait=60.0):
          result = []
tests/qemu-iotests/iotests.py +4 -1
-        for ev in self.get_qmp_events(wait=wait):
+        qmp_wait = wait
+        if qemu_gdb:
+            qmp_wait = 0.0
+        for ev in self.get_qmp_events(wait=qmp_wait):
              result.append(filter_qmp_event(ev))
          return result

Should this be handled in get_qmp_events instead, since you're basically changing all the callers?

Paolo

#13 lore raw

Re: [RFC PATCH v2 04/11] qemu-iotests: delay QMP socket timers

Quoted 11 lines · Paolo Bonzini
Quoted 9 lines · Emanuele Giuseppe Esposito

      def get_qmp_events_filtered(self, wait=60.0):
          result = []
-        for ev in self.get_qmp_events(wait=wait):
+        qmp_wait = wait
+        if qemu_gdb:
+            qmp_wait = 0.0
+        for ev in self.get_qmp_events(wait=qmp_wait):
              result.append(filter_qmp_event(ev)) return result

Quoted 1 line

Should this be handled in get_qmp_events instead, since you're basically changing all the callers?

get_qmp_events is in python/machine.py, which as I understand might be used also by some other scripts, so I want to keep the changes there to the minimum. Also, machine.py has no access to qemu_gdb or qemu_valgrind, so passing a boolean or something to delay the timer would still require to add a similar check in all sections.

Or do you have a cleaner way to do this?

Emanuele

#14 lore raw

Re: [RFC PATCH v2 04/11] qemu-iotests: delay QMP socket timers

Il gio 8 apr 2021, 18:06 Emanuele Giuseppe Esposito <[email protected]> ha scritto:

Quoted 17 lines · Paolo Bonzini
Quoted 11 lines · Emanuele Giuseppe Esposito

tests/qemu-iotests/iotests.py def qmp_to_opts(self, obj): · line 694

Quoted 9 lines
      def get_qmp_events_filtered(self, wait=60.0):
          result = []
tests/qemu-iotests/iotests.py +4 -1
-        for ev in self.get_qmp_events(wait=wait):
+        qmp_wait = wait
+        if qemu_gdb:
+            qmp_wait = 0.0
+        for ev in self.get_qmp_events(wait=qmp_wait):
              result.append(filter_qmp_event(ev))
          return result
Quoted 2 lines

Should this be handled in get_qmp_events instead, since you're basically changing all the callers?

Quoted 8 lines

get_qmp_events is in python/machine.py, which as I understand might be used also by some other scripts, so I want to keep the changes there to the minimum. Also, machine.py has no access to qemu_gdb or qemu_valgrind, so passing a boolean or something to delay the timer would still require to add a similar check in all sections.

Or do you have a cleaner way to do this?

Maybe a subclass IotestsMachine?

Paolo

Quoted 3 lines

Emanuele

#15 lore raw

Re: [RFC PATCH v2 04/11] qemu-iotests: delay QMP socket timers

Quoted 28 lines · Paolo Bonzini

Il gio 8 apr 2021, 18:06 Emanuele Giuseppe Esposito <[email protected] <mailto:[email protected]>> ha scritto:

On 08/04/2021 17:40, Paolo Bonzini wrote:
> On 07/04/21 15:50, Emanuele Giuseppe Esposito wrote:
>>       def get_qmp_events_filtered(self, wait=60.0):
>>           result = []
>> -        for ev in self.get_qmp_events(wait=wait):
>> +        qmp_wait = wait
>> +        if qemu_gdb:
>> +            qmp_wait = 0.0
>> +        for ev in self.get_qmp_events(wait=qmp_wait):
>>               result.append(filter_qmp_event(ev)) >>           return result
>
> Should this be handled in get_qmp_events instead, since you're
basically
> changing all the callers?

get_qmp_events is in python/machine.py, which as I understand might be used also by some other scripts, so I want to keep the changes there to the minimum. Also, machine.py has no access to qemu_gdb or qemu_valgrind, so passing a boolean or something to delay the timer would still require to add a similar check in all sections.

Quoted 5 lines

Or do you have a cleaner way to do this?

Maybe a subclass IotestsMachine?

I actually figured that I could override get_qmp_events and put the check there. Something like (simplified):

class VM(qtest.QEMUQtestMachine):

	...

	def get_qmp_events(self, wait)
		if qemu_gdb or qemu_valgrind:
			wait = 0.0
		return super().get_qmp_events(wait)

Emanuele

#16 lore raw

RFC PATCH v2 5/11 qemu_iotests: insert gdbserver command line as wrapper for qemu binary

In reply to Emanuele Giuseppe Esposito
---
 tests/qemu-iotests/iotests.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
index 17f07710db..8f6bb20af5 100644
@@ -579,7 +579,8 @@ class VM(qtest.QEMUQtestMachine):
 
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
-        super().__init__(qemu_prog, qemu_opts, name=name,
+        super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb,
+                         name=name,
                          test_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
                          sock_dir=sock_dir)
Signature · 1 line
2.30.2
#17 lore raw

RFC PATCH v2 6/11 qemu-iotests: add gdbserver option to script tests too

In reply to Emanuele Giuseppe Esposito

The only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail.

---
 tests/qemu-iotests/common.rc | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tests/qemu-iotests/common.rc +7 -1
index 65cdba5723..53a3310fee 100644
@@ -166,8 +166,14 @@ _qemu_wrapper()
         if [ -n "${QEMU_NEED_PID}" ]; then
             echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid"
         fi
+
+        GDB="${QEMU_PROG}"
+        if [ ! -z ${GDB_QEMU} ]; then
+            GDB="gdbserver ${GDB_QEMU} ${GDB}"
+        fi
+
         VALGRIND_QEMU="${VALGRIND_QEMU_VM}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \
-            "$QEMU_PROG" $QEMU_OPTIONS "$@"
+           $GDB $QEMU_OPTIONS "$@"
     )
     RETVAL=$?
     _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL
Signature · 1 line
2.30.2
#18 lore raw

RFC PATCH v2 7/11 qemu_iotests: extend the check script to support valgrind for python tests

In reply to Emanuele Giuseppe Esposito

Currently, the check script only parses the option and sets the VALGRIND_QEMU environmental variable to "y".
Add another iotests python variable that prepares the command line, identical to the one provided in the test scripts.

Because the python script does not know in advance the valgrind PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime.

---
 tests/qemu-iotests/iotests.py | 11 +++++++++++
 tests/qemu-iotests/testenv.py |  1 +
 2 files changed, 12 insertions(+)
tests/qemu-iotests/iotests.py +11 -0
index 8f6bb20af5..7c28f0cb74 100644
@@ -96,6 +96,17 @@
     sys.stderr.write('Please run this test via the "check" script\n')
     sys.exit(os.EX_USAGE)
 
+qemu_valgrind = []
+if os.environ.get('VALGRIND_QEMU') == "y" and \
+    os.environ.get('NO_VALGRIND') != "y":
+    valgrind_logfile = "--log-file=" + test_dir.strip()
+    # %p allows to put the valgrind process PID, since
+    # we don't know it a priori (subprocess.Peopen is
+    # not yet invoked)
+    valgrind_logfile += "/%p.valgrind"
+
+    qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99']
+
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
tests/qemu-iotests/testenv.py +1 -0
index 669eb6b925..86798bf752 100644
@@ -280,6 +280,7 @@ def print_env(self) -> None:
 SOCK_DIR      -- {SOCK_DIR}
 SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
 GDB_QEMU      -- "{GDB_QEMU}"
+VALGRIND_QEMU     -- "{VALGRIND_QEMU}"
 """
 
         args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.30.2
#19 lore raw

RFC PATCH v2 8/11 qemu_iotests: extent QMP socket timeout when using valgrind

In reply to Emanuele Giuseppe Esposito

As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout expires too soon.

---
 python/qemu/machine.py        | 4 +++-
 tests/qemu-iotests/iotests.py | 8 ++++----
 2 files changed, 7 insertions(+), 5 deletions(-)
python/qemu/machine.py +3 -1
index 284b73385f..4b6eb39856 100644
@@ -409,7 +409,9 @@ def _launch(self) -> None:
                                        shell=False,
                                        close_fds=False)
 
-        timer = None if 'gdbserver' in self._wrapper else 15.0
+        delay_timer = 'gdbserver' in self._wrapper
+        delay_timer |= 'valgrind' in self._wrapper
+        timer = None if delay_timer else 15.0
         self._post_launch(timer)
 
     def _early_cleanup(self) -> None:
tests/qemu-iotests/iotests.py +4 -4
index 7c28f0cb74..56733954b2 100644
@@ -699,7 +699,7 @@ def qmp_to_opts(self, obj):
     def get_qmp_events_filtered(self, wait=60.0):
         result = []
         qmp_wait = wait
-        if qemu_gdb:
+        if qemu_gdb or qemu_valgrind:
             qmp_wait = 0.0
         for ev in self.get_qmp_events(wait=qmp_wait):
             result.append(filter_qmp_event(ev))
@@ -1003,7 +1003,7 @@ def cancel_and_wait(self, drive='drive0', force=False,
         self.assert_qmp(result, 'return', {})
 
         qmp_wait = wait
-        if qemu_gdb:
+        if qemu_gdb or qemu_valgrind:
             qmp_wait = 0.0
 
         if resume:
@@ -1029,7 +1029,7 @@ def wait_until_completed(self, drive='drive0', check_offset=True,
                              wait=60.0, error=None):
         '''Wait for a block job to finish, returning the event'''
         qmp_wait = wait
-        if qemu_gdb:
+        if qemu_gdb or qemu_valgrind:
             qmp_wait = 0.0
         while True:
             for event in self.vm.get_qmp_events(wait=qmp_wait):
@@ -1077,7 +1077,7 @@ def complete_and_wait(self, drive='drive0', wait_ready=True,
 
     def pause_wait(self, job_id='job0'):
         def_timeout = 3
-        if qemu_gdb:
+        if qemu_gdb or qemu_valgrind:
             def_timeout = 3000
         with Timeout(def_timeout, "Timeout waiting for job to pause"):
             while True:
Signature · 1 line
2.30.2
#20 lore raw

RFC PATCH v2 9/11 qemu_iotests: allow valgrint to print/delete the generated log file

In reply to Emanuele Giuseppe Esposito

When using valgrind on the test scripts, it generates a log file in $TEST_DIR that is either print (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests.

---
 tests/qemu-iotests/iotests.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
tests/qemu-iotests/iotests.py +20 -0
index 56733954b2..b6166b6f7b 100644
@@ -597,6 +597,26 @@ def __init__(self, path_suffix=''):
                          sock_dir=sock_dir)
         self._num_drives = 0
 
+    def subprocess_check_valgrind(self, valgrind) -> None:
+
+        if not valgrind:
+            return
+
+        valgrind_filename =  test_dir + "/" + str(self._popen.pid) + ".valgrind"
+
+        if self.exitcode() == 99:
+            with open(valgrind_filename) as f:
+                content = f.readlines()
+            for line in content:
+                print(line, end ="")
+            print("")
+        else:
+            os.remove(valgrind_filename)
+
+    def _post_shutdown(self) -> None:
+        super()._post_shutdown()
+        self.subprocess_check_valgrind(qemu_valgrind)
+
     def add_object(self, opts):
         self._args.append('-object')
         self._args.append(opts)
Signature · 1 line
2.30.2
#21 lore raw

RFC PATCH v2 10/11 qemu_iotests: insert valgrind command line as wrapper for qemu binary

In reply to Emanuele Giuseppe Esposito

The priority will be given to gdb command line, meaning if -gdb and -valgrind parameters are given, only gdb will be wrapped around the qemu binary.

---
 tests/qemu-iotests/iotests.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +2 -1
index b6166b6f7b..b23bfdfdff 100644
@@ -590,7 +590,8 @@ class VM(qtest.QEMUQtestMachine):
 
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
-        super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb,
+        wrapper = qemu_gdb if qemu_gdb else qemu_valgrind
+        super().__init__(qemu_prog, qemu_opts, wrapper=wrapper,
                          name=name,
                          test_dir=test_dir,
                          socket_scm_helper=socket_scm_helper,
Signature · 1 line
2.30.2
#22 lore raw

RFC PATCH v2 11/11 qemu_iotests: add option to show qemu binary logs on stdout

In reply to Emanuele Giuseppe Esposito

Using the flag -p, allow the qemu binary to print to stdout. This is helpful especially for print-debugging.

---
 tests/qemu-iotests/check      | 3 ++-
 tests/qemu-iotests/iotests.py | 9 +++++++++
 tests/qemu-iotests/testenv.py | 9 +++++++--
 3 files changed, 18 insertions(+), 3 deletions(-)
tests/qemu-iotests/check +2 -1
index 6186495eee..6c469c14ff 100755
@@ -33,6 +33,7 @@ def make_argparser() -> argparse.ArgumentParser:
                    help='pretty print output for make check')
 
     p.add_argument('-d', dest='debug', action='store_true', help='debug')
+    p.add_argument('-p', dest='print', action='store_true', help='enable prints')
     p.add_argument('-gdb', action='store_true',
                    help="start gdbserver with $GDB_QEMU options. \
                          Default is localhost:12345")
@@ -116,7 +117,7 @@ if __name__ == '__main__':
                   aiomode=args.aiomode, cachemode=args.cachemode,
                   imgopts=args.imgopts, misalign=args.misalign,
                   debug=args.debug, valgrind=args.valgrind,
-                  gdb=args.gdb)
+                  gdb=args.gdb, qprint=args.print)
 
     testfinder = TestFinder(test_dir=env.source_iotests)
 
tests/qemu-iotests/iotests.py +9 -0
index b23bfdfdff..bb29074b63 100644
@@ -79,6 +79,8 @@
 if os.environ.get('GDB_QEMU'):
     qemu_gdb = ['gdbserver'] + os.environ.get('GDB_QEMU').strip().split(' ')
 
+qemu_print = os.environ.get('PRINT_QEMU', False)
+
 imgfmt = os.environ.get('IMGFMT', 'raw')
 imgproto = os.environ.get('IMGPROTO', 'file')
 output_dir = os.environ.get('OUTPUT_DIR', '.')
@@ -618,6 +620,13 @@ def _post_shutdown(self) -> None:
         super()._post_shutdown()
         self.subprocess_check_valgrind(qemu_valgrind)
 
+    def _pre_launch(self) -> None:
+        super()._pre_launch()
+        if qemu_print and self._qemu_log_file != None:
+            # set QEMU binary output to stdout
+            self._qemu_log_file.close()
+            self._qemu_log_file = None
+
     def add_object(self, opts):
         self._args.append('-object')
         self._args.append(opts)
tests/qemu-iotests/testenv.py +7 -2
index 86798bf752..9192d7154b 100644
@@ -73,7 +73,7 @@ class TestEnv(ContextManager['TestEnv']):
                      'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU',
                      'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX',
                      'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_',
-                     'GDB_QEMU']
+                     'GDB_QEMU', 'PRINT_QEMU']
 
     def get_env(self) -> Dict[str, str]:
         env = {}
@@ -165,7 +165,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
                  misalign: bool = False,
                  debug: bool = False,
                  valgrind: bool = False,
-                 gdb: bool = False) -> None:
+                 gdb: bool = False,
+                 qprint: bool = False) -> None:
         self.imgfmt = imgfmt
         self.imgproto = imgproto
         self.aiomode = aiomode
@@ -173,6 +174,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str,
         self.misalign = misalign
         self.debug = debug
 
+        if qprint:
+            self.print_qemu = 'y'
+
         self.gdb_qemu = os.getenv('GDB_QEMU')
 
         if gdb and not self.gdb_qemu:
@@ -281,6 +285,7 @@ def print_env(self) -> None:
 SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}
 GDB_QEMU      -- "{GDB_QEMU}"
 VALGRIND_QEMU     -- "{VALGRIND_QEMU}"
+PRINT_QEMU    --  "{PRINT_QEMU}"
 """
 
         args = collections.defaultdict(str, self.get_env())
Signature · 1 line
2.30.2
#23 lore raw
In reply to Emanuele Giuseppe Esposito
Quoted 13 lines · Emanuele Giuseppe Esposito

This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.

Patches 1-6 introduce the -gdb option to both python and bash tests, 7-10 extend the already existing -valgrind flag to work also on python tests, and patch 11 introduces -p to enable logging to stdout.

In particular, patches 1,2,4,8 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.

Quoted 5 lines

This series is tested on the previous serie "qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

How discoverable are these goodies for developers with only superficial knowledge of iotests?

#24 lore raw
Quoted 15 lines · Markus Armbruster
Quoted 13 lines · Emanuele Giuseppe Esposito

This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.

Patches 1-6 introduce the -gdb option to both python and bash tests, 7-10 extend the already existing -valgrind flag to work also on python tests, and patch 11 introduces -p to enable logging to stdout.

In particular, patches 1,2,4,8 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.

Quoted 9 lines
Quoted 5 lines

This series is tested on the previous serie "qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

How discoverable are these goodies for developers with only superficial knowledge of iotests?

Not really sure what you mean, but

./check --help now shows:

Review

Quoted 2 lines
(quoted diff) +0 -2
-p         enable prints
-gdb       start gdbserver with $GDB_QEMU options. Default is localhost:12345

Which I guess should be clear enough? Btw two-three weeks ago I didn't know anything about these tests either.

I agree I can make -p more clear, saying "enable qemu binary prints to stdout", and move -valgrind to the "optional arguments" instead of "bash-only"

Emanuele

#25 lore raw
Quoted 36 lines · Emanuele Giuseppe Esposito
Quoted 15 lines · Markus Armbruster
Quoted 13 lines · Emanuele Giuseppe Esposito

This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.

Patches 1-6 introduce the -gdb option to both python and bash tests, 7-10 extend the already existing -valgrind flag to work also on python tests, and patch 11 introduces -p to enable logging to stdout.

In particular, patches 1,2,4,8 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.

Quoted 9 lines
Quoted 5 lines

This series is tested on the previous serie "qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

How discoverable are these goodies for developers with only superficial knowledge of iotests?

Not really sure what you mean, but

./check --help now shows:

Review

Quoted 2 lines
(quoted diff) +0 -2
-p         enable prints
-gdb       start gdbserver with $GDB_QEMU options. Default is localhost:12345

Which I guess should be clear enough? Btw two-three weeks ago I didn't know anything about these tests either.

Quoted 3 lines

I agree I can make -p more clear, saying "enable qemu binary prints to stdout", and move -valgrind to the "optional arguments" instead of "bash-only"

Yes, please (this is not a demand).

docs/devel/testing.rst section "QEMU iotests" is the place to explain things in more detail than --help, if that's useful. Right now it simply points to check -h.

#26 lore raw
Quoted 42 lines · Markus Armbruster
Quoted 36 lines · Emanuele Giuseppe Esposito
Quoted 15 lines · Markus Armbruster
Quoted 13 lines · Emanuele Giuseppe Esposito

This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests.
It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file.

Patches 1-6 introduce the -gdb option to both python and bash tests, 7-10 extend the already existing -valgrind flag to work also on python tests, and patch 11 introduces -p to enable logging to stdout.

In particular, patches 1,2,4,8 focus on extending the QMP socket timers when using gdb/valgrind, otherwise the python tests will fail due to delays in the QMP responses.

Quoted 9 lines
Quoted 5 lines

This series is tested on the previous serie "qemu-iotests: quality of life improvements"
but independent from it, so it can be applied separately.

How discoverable are these goodies for developers with only superficial knowledge of iotests?

Not really sure what you mean, but

./check --help now shows:

Review

Quoted 2 lines
(quoted diff) +0 -2
-p         enable prints
-gdb       start gdbserver with $GDB_QEMU options. Default is localhost:12345

Which I guess should be clear enough? Btw two-three weeks ago I didn't know anything about these tests either.

Quoted 3 lines

I agree I can make -p more clear, saying "enable qemu binary prints to stdout", and move -valgrind to the "optional arguments" instead of "bash-only"

Quoted 6 lines

Yes, please (this is not a demand).

docs/devel/testing.rst section "QEMU iotests" is the place to explain things in more detail than --help, if that's useful. Right now it simply points to check -h.

Ok, I will add a new section in testing.rst explaining the new flags.

Thank you,
Emanuele