Skip to contentSkip to conversation

Misc patches for iotests, qom, crypt & io

GIT PULL Applied Daniel P. Berrangé 2 participants 31 messages May 21 – May 24

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

The following changes since commit 3f129ea545f16e82b5e43fcc3866b134b7cc4dfc:

Merge tag 'pull-vfio-20260520' of https://github.com/legoater/qemu into staging (2026-05-20 16:53:28 -0400)

are available in the Git repository at:

https://gitlab.com/berrange/qemu tags/misc-next-pull-request

for you to fetch changes up to 7b0376d24ce34893a9512587d1d82a6e07680e6d:

qom: drop user_creatable_add_type method (2026-05-21 12:39:54 +0100)

----------------------------------------------------------------
Misc patches for iotests, qom, crypt & io

* Fix client side anoymous TLS credentials
* Fix return value semantics for qio_channel_flush
* Add ID validation of internal QOM constructor
* Fix ability to create internal QOM objects without a parent
* Merge user creatable object constructor into main QOM file
* Print reason for skipping I/O tests
* Remove redundant meson suits for I/O tests
* Add optional suites for many block drivers I/O tests
* Run I/O tests for 10 block drivers in GitLab CI
* Fix sudo check for LUKS I/O test
* Mark I/O test 151, 181, 185, 308 as flaky

----------------------------------------------------------------

Daniel P. Berrangé (27):
crypto: fix client side anonymous TLS credentials
gitlab: ensure all meson jobs capture build/meson-logs by default
iotests: print reason when I/O test is skipped in TAP mode
iotests: remove redundant meson suite for iotests
iotests: ensure all qcow2 I/O tests are able to be run via make scripts/mtest2make: ensure output has stable sorting scripts/mtest2make: support optional tests grouping
iotests: add a meson suite / make target per block I/O tests format docs/devel/testing: expand documentation for 'make check-block'
iotests: add nbd and luks to the I/O test suites
iotests: use 'driver' as collective term for either format or protocol
iotests: validate dmsetup result in test 128
iotests: fix check for sudo access in LUKS I/O test
iotests: mark 151, 181, 185 & 308 as flaky tests
gitlab: add jobs for thorough block tests
gitlab: remove I/O tests from build-tcg-disabled job
qom: add trace events for object/property lifecycle hw/vfio-user: use a valid object ID for iothread
qom: validate ID format when creating objects
qom: make errp last param in methods taking va_list
qom: shorten name of object_set_properties_from_keyval
qom: have object_set_props_keyval return bool
qom: move object_set_prop_keyval into object.c
qom: add object_new_with_props_from_qdict
qom: fix ability to create objects without a parent
qom: allow object_new_with_prop* to trigger module loading
qom: drop user_creatable_add_type method

John Snow (1):
python: bump qemu.qmp to v0.0.6

Tejus GK (1):
io: invert the return semantics of qio_channel_flush

 .gitlab-ci.d/buildtest-template.yml           |  30 ++-
 .gitlab-ci.d/buildtest.yml                    |  27 +-
 authz/listfile.c                              |   4 +-
 crypto/tlscredsanon.c                         |   2 +
 docs/devel/testing/main.rst                   |  26 +-
 hw/vfio-user/proxy.c                          |   2 +-
 include/io/channel-socket.h                   |   6 +-
 include/io/channel.h                          |   4 +-
 include/qom/object.h                          | 121 +++++++--
 include/qom/object_interfaces.h               |  18 --
 io/channel-socket.c                           |  16 +-
 python/scripts/vendor.py                      |   4 +-
 ...ny.whl => qemu_qmp-0.0.6-py3-none-any.whl} | Bin 72263 -> 72379 bytes
 pythondeps.toml                               |   2 +-
 qom/object.c                                  | 230 ++++++++++++++++--
 qom/object_interfaces.c                       | 101 +-------
 qom/trace-events                              |  12 +-
 scripts/mtest2make.py                         |  30 ++-
 system/qdev-monitor.c                         |   4 +-
 system/vl.c                                   |   7 +-
 tests/Makefile.include                        |   3 +-
 tests/qemu-iotests/128                        |   6 +
 tests/qemu-iotests/149                        |  13 +-
 tests/qemu-iotests/151                        |   1 +
 tests/qemu-iotests/181                        |   2 +
 tests/qemu-iotests/185                        |   1 +
 tests/qemu-iotests/308                        |   2 +
 tests/qemu-iotests/common.rc                  |  16 ++
 tests/qemu-iotests/iotests.py                 |  16 ++
 tests/qemu-iotests/meson.build                |  65 ++++-
 tests/qemu-iotests/testrunner.py              |   2 +-
 tests/unit/check-qom-proplist.c               |  97 ++++++--
 tests/unit/test-crypto-tlssession.c           | 120 ++++++++-
 33 files changed, 734 insertions(+), 256 deletions(-)
 rename python/wheels/{qemu_qmp-0.0.5-py3-none-any.whl => qemu_qmp-0.0.6-py3-none-any.whl} (64%)
Signature · 1 line
2.54.0
#1 lore raw

[PULL 04/29] gitlab: ensure all meson jobs capture build/meson-logs by default

The build-without-defaults and build-tci jobs do not capture any artifacts, despite running 'make check'. This has proved a repeated bug with CI jobs, so introduce a new '.meson_job_template' rule which always captures 'build/meson-logs'. Jobs can still provide their own 'artifacts:' config which will override this default behaviour.

---
 .gitlab-ci.d/buildtest-template.yml | 30 ++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)
.gitlab-ci.d/buildtest-template.yml +19 -11
index d866cb12bb..005058625e 100644
@@ -1,5 +1,20 @@
-.native_build_job_template:
+
+# Any job running meson should capture meson logs
+# by default. Some jobs might override the artifacts
+# to capture further files
+.meson_job_template:
   extends: .base_job_template
+  artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+    when: always
+    expire_in: 7 days
+    paths:
+      - build/meson-logs
+    reports:
+      junit: build/meson-logs/*.junit.xml
+
+.native_build_job_template:
+  extends: .meson_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
   cache:
@@ -60,7 +75,7 @@
       - build/**/*.c.o.d
 
 .common_test_job_template:
-  extends: .base_job_template
+  extends: .meson_job_template
   stage: test
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
   script:
@@ -87,14 +102,7 @@
     # Prevent logs from the build job that run earlier
     # from being duplicated in the test job artifacts
     - rm -f build/meson-logs/*
-  artifacts:
-    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
-    when: always
-    expire_in: 7 days
-    paths:
-      - build/meson-logs
-    reports:
-      junit: build/meson-logs/*.junit.xml
+
 
 .functional_test_job_template:
   extends: .common_test_job_template
@@ -125,7 +133,7 @@
     QEMU_JOB_FUNCTIONAL: 1
 
 .wasm_build_job_template:
-  extends: .base_job_template
+  extends: .meson_job_template
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:$QEMU_CI_CONTAINER_TAG
   before_script:
Signature · 1 line
2.54.0
#2 lore raw

[PULL 07/29] iotests: ensure all qcow2 I/O tests are able to be run via make

In reply to Daniel P. Berrangé

For block formats marked as 'quick', only tests in the 'auto' group are added to the meson test suite.

The result of this is that qcow2 tests not in the 'auto' group cannot be run at all, even if passing SPEED=slow or SPEED=thorough.

To fix this we need todo two passes over the I/O test list. First add all tests from 'auto' group into the 'block' suite, so they are run by default. Then on the second pass add any tests which were not in 'auto' into the 'block-slow' suite, so they get run when SPEED=slow or SPEED=thorough.

---
 tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 5 deletions(-)
tests/qemu-iotests/meson.build +43 -5
index bf588cc2c9..1a24d801a3 100644
@@ -30,16 +30,48 @@ endforeach
 qemu_iotests_check_cmd = files('check')
 
 foreach format, speed: qemu_iotests_formats
+  # Formats tagged 'quick' get the subset of tests in the 'auto'
+  # group, run by default with 'make check' / 'make check-block'
+  seen = []
   if speed == 'quick'
-    suites = 'block'
+    args = ['-tap', '-' + format, '-g', 'auto']
+    suites = ['block']
+
+    rc = run_command(
+      [python, qemu_iotests_check_cmd] + args + ['-n'],
+      check: true,
+    )
+
+    foreach item: rc.stdout().strip().split()
+      seen += item
+      args = [qemu_iotests_check_cmd,
+              '-tap', '-' + format, item,
+              '--source-dir', meson.current_source_dir(),
+              '--build-dir', meson.current_build_dir()]
+      # Some individual tests take as long as 45 seconds
+      # Bump the timeout to 3 minutes for some headroom
+      # on slow machines to minimize spurious failures
+      test('io-' + format + '-' + item,
+           python,
+           args: args,
+           depends: qemu_iotests_binaries,
+           env: qemu_iotests_env,
+           protocol: 'tap',
+           timeout: 180,
+           suite: suites)
+    endforeach
+  endif
+
+  suites = []
+  # Any format tagged quick or slow also gets added to slow
+  # otherwise its tagged thorough
+  if speed != 'thorough'
+    suites += ['block-slow']
   else
-    suites = ['block-' + speed]
+    suites += ['block-thorough']
   endif
 
   args = ['-tap', '-' + format]
-  if speed == 'quick'
-      args += ['-g', 'auto']
-  endif
 
   rc = run_command(
       [python, qemu_iotests_check_cmd] + args + ['-n'],
@@ -47,6 +79,12 @@ foreach format, speed: qemu_iotests_formats
   )
 
   foreach item: rc.stdout().strip().split()
+      # Skip any tests already added from the 'auto' group
+      # as they're run in the 'quick' suite already
+      if item in seen
+          continue
+      endif
+
       args = [qemu_iotests_check_cmd,
               '-tap', '-' + format, item,
               '--source-dir', meson.current_source_dir(),
Signature · 1 line
2.54.0
#3 lore raw

[PULL 03/29] python: bump qemu.qmp to v0.0.6

In reply to Daniel P. Berrangé

From: John Snow <[email protected]>

This release removes some deprecated warnings for our use of the old sendmsg API on older python versions.

---
 python/scripts/vendor.py                      |   4 ++--
 ...ny.whl => qemu_qmp-0.0.6-py3-none-any.whl} | Bin 72263 -> 72379 bytes
 pythondeps.toml                               |   2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
 rename python/wheels/{qemu_qmp-0.0.5-py3-none-any.whl => qemu_qmp-0.0.6-py3-none-any.whl} (64%)
python/scripts/vendor.py +2 -2
index 7c61afd37b..1bb59d863a 100755
@@ -43,8 +43,8 @@ def main() -> int:
     packages = {
         "meson==1.11.1":
         "9b3a023657e393dbc5335b95c561337d49b7a458f5541e47ec44f2cc566e0d80",
-        "qemu.qmp==0.0.5":
-        "e05782d6df5844b34e0d2f7c68693525da074deef7b641c1401dda6e4e3d6303",
+        "qemu.qmp==0.0.6":
+        "5d7c5af0e9de427696e3bf72e333965c3a697929f77f6b7ddc30c989fc7b539b",
         "pycotap==1.3.1":
         "1c3a25b3ff89e48f4e00f1f71dbbc1642b4f65c65d416524d07e73492fff25ea",
     }
python/wheels/qemu_qmp-0.0.6-py3-none-any.whl renamed from python/wheels/qemu_qmp-0.0.5-py3-none-any.whl binary +0 -0
similarity index 64%
rename from python/wheels/qemu_qmp-0.0.5-py3-none-any.whl
rename to python/wheels/qemu_qmp-0.0.6-py3-none-any.whl
index 6372b7544ffb015a5c77d9686e921b65c848d433..5754130b408e2f13081b368a5506101a6ac490f2 100644
GIT binary patch
pythondeps.toml +1 -1
index 47d4caefa9..bef88d49a6 100644
@@ -36,7 +36,7 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
 # only include dependencies that can be guaranteed via configure from
 # system packages, or python packages we vendor.
 [tooling]
-"qemu.qmp" = { accepted = ">=0.0.5", installed = "0.0.5" }
+"qemu.qmp" = { accepted = ">=0.0.5", installed = "0.0.6" }
 "qemu" = { path = "python/" }
 # NB: The following dependencies should be a little bit more modern than
 # the versions listed here, but we are still using Debian 11 for several
Signature · 1 line
2.54.0
#4 lore raw

[PULL 02/29] crypto: fix client side anonymous TLS credentials

In reply to Daniel P. Berrangé

The previous refactoring of credential creation failed to allocate storage fo the anonymous TLS credentials on the client endpoint.

Fixes 70f9fd8dbf7233bee497055a9b7825e3729ce853
---
 crypto/tlscredsanon.c               |   2 +
 tests/unit/test-crypto-tlssession.c | 120 +++++++++++++++++++++++++++-
 2 files changed, 121 insertions(+), 1 deletion(-)
crypto/tlscredsanon.c +2 -0
index 1551382e1f..190c9833a7 100644
@@ -73,6 +73,8 @@ qcrypto_tls_creds_anon_load(QCryptoTLSCredsAnon *creds,
                                              box->dh_params);
         }
     } else {
+        box = qcrypto_tls_creds_box_new_client(GNUTLS_CRD_ANON);
+
         ret = gnutls_anon_allocate_client_credentials(&box->data.anonclient);
         if (ret < 0) {
             error_setg(errp, "Cannot allocate credentials: %s",
tests/unit/test-crypto-tlssession.c +119 -1

140 lines not on this page · raw message

Signature · 1 line
2.54.0
#5 lore raw

[PULL 01/29] io: invert the return semantics of qio_channel_flush

In reply to Daniel P. Berrangé

From: Tejus GK <[email protected]>

With the kernel's zerocopy notification mechanism, the caller can determine whether
* All syscalls successfully used zero copy
* At least one syscall failed to use zero copy

But, as of now QEMU's IO channel flush function semantics are like
* 1 => all syscalls failed to use zero copy
* 0 => at least one syscall successfully used zero copy

This is not aligned with what the kernel reports, and ends up reporting false negatives for cases like when there's just a single successful zerocopy amongst a collection of deferred zero-copies during a flush.

Fix this by inverting the return semantics of the IO flush function.

---
 include/io/channel-socket.h |  6 +-----
 include/io/channel.h        |  4 ++--
 io/channel-socket.c         | 16 ++++++++--------
 3 files changed, 11 insertions(+), 15 deletions(-)
include/io/channel-socket.h +1 -5
index a1ef3136ea..b07cd61477 100644
@@ -50,11 +50,7 @@ struct QIOChannelSocket {
     ssize_t zero_copy_queued;
     ssize_t zero_copy_sent;
     bool blocking;
-    /**
-     * This flag indicates whether any new data was successfully sent with
-     * zerocopy since the last qio_channel_socket_flush() call.
-     */
-    bool new_zero_copy_sent_success;
+    bool zero_copy_fallback;
 };
 
 
include/io/channel.h +2 -2
index 287d10cd6f..98485c9280 100644
@@ -1147,8 +1147,8 @@ int coroutine_mixed_fn qio_channel_writev_full_all(QIOChannel *ioc,
  * If not implemented, acts as a no-op, and returns 0.
  *
  * Returns -1 if any error is found,
- *          1 if every send failed to use zero copy.
- *          0 otherwise.
+ *          1 if at least one send failed to use zero copy.
+ *          0 if every send successfully used zero copy.
  */
 
 int qio_channel_flush(QIOChannel *ioc,
io/channel-socket.c +8 -8
index 3053b35ad8..ea2ec84108 100644
@@ -72,7 +72,7 @@ qio_channel_socket_new(void)
     sioc->zero_copy_queued = 0;
     sioc->zero_copy_sent = 0;
     sioc->blocking = false;
-    sioc->new_zero_copy_sent_success = false;
+    sioc->zero_copy_fallback = false;
 
     ioc = QIO_CHANNEL(sioc);
     qio_channel_set_feature(ioc, QIO_CHANNEL_FEATURE_SHUTDOWN);
@@ -880,9 +880,9 @@ static int qio_channel_socket_flush_internal(QIOChannel *ioc,
         /* No errors, count successfully finished sendmsg()*/
         sioc->zero_copy_sent += serr->ee_data - serr->ee_info + 1;
 
-        /* If any sendmsg() succeeded using zero copy, mark zerocopy success */
-        if (serr->ee_code != SO_EE_CODE_ZEROCOPY_COPIED) {
-            sioc->new_zero_copy_sent_success = true;
+        if (serr->ee_code == SO_EE_CODE_ZEROCOPY_COPIED) {
+            /* If any sendmsg() fell back to a copy, mark fallback as true */
+            sioc->zero_copy_fallback = true;
         }
     }
 
@@ -900,12 +900,12 @@ static int qio_channel_socket_flush(QIOChannel *ioc,
         return ret;
     }
 
-    if (sioc->new_zero_copy_sent_success) {
-        sioc->new_zero_copy_sent_success = false;
-        return 0;
+    if (sioc->zero_copy_fallback) {
+        sioc->zero_copy_fallback = false;
+        return 1;
     }
 
-    return 1;
+    return 0;
 }
 
 #endif /* QEMU_MSG_ZEROCOPY */
Signature · 1 line
2.54.0
#6 lore raw

[PULL 06/29] iotests: remove redundant meson suite for iotests

In reply to Daniel P. Berrangé

If a test is in the 'block-slow' or 'block-thorough' suite, there is no need to also add it to the 'slow' or 'thorough' suites.

---
 tests/qemu-iotests/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/meson.build +1 -1
index d7bae71ced..bf588cc2c9 100644
@@ -33,7 +33,7 @@ foreach format, speed: qemu_iotests_formats
   if speed == 'quick'
     suites = 'block'
   else
-    suites = ['block-' + speed, speed]
+    suites = ['block-' + speed]
   endif
 
   args = ['-tap', '-' + format]
Signature · 1 line
2.54.0
#7 lore raw

[PULL 15/29] iotests: fix check for sudo access in LUKS I/O test

In reply to Daniel P. Berrangé

The test did not cope with the possibility that 'sudo' was not installed at all, merely that it was not configured. This broke tests in any CI env which lacks 'sudo'.

---
 tests/qemu-iotests/149 | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
tests/qemu-iotests/149 +8 -5
index c13343d7ef..6dff39a28a 100755
@@ -95,11 +95,14 @@ def verify_passwordless_sudo():
 
     args = ["sudo", "-n", "/bin/true"]
 
-    proc = subprocess.Popen(args,
-                            stdin=subprocess.PIPE,
-                            stdout=subprocess.PIPE,
-                            stderr=subprocess.STDOUT,
-                            universal_newlines=True)
+    try:
+        proc = subprocess.Popen(args,
+                                stdin=subprocess.PIPE,
+                                stdout=subprocess.PIPE,
+                                stderr=subprocess.STDOUT,
+                                universal_newlines=True)
+    except FileNotFoundError as e:
+        iotests.notrun('requires sudo binary: %s' % e)
 
     msg = proc.communicate()[0]
 
Signature · 1 line
2.54.0
#8 lore raw

[PULL 05/29] iotests: print reason when I/O test is skipped in TAP mode

In reply to Daniel P. Berrangé

The TAP output on a skipped test:

ok raw 181 # SKIP

is not informative.

The test program included a reason, and that should be displayed in TAP mode (it is already shown in non-TAP mode):

ok raw 181 # SKIP Postcopy is not supported

---
 tests/qemu-iotests/testrunner.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/testrunner.py +1 -1
index e2a3658994..dbe2dddc32 100644
@@ -174,7 +174,7 @@ def test_print_one_line(self, test: str,
             elif status == 'fail':
                 print(f'not ok {self.env.imgfmt} {test}')
             elif status == 'not run':
-                print(f'ok {self.env.imgfmt} {test} # SKIP')
+                print(f'ok {self.env.imgfmt} {test} # SKIP {description}')
             return
 
         if lasttime:
Signature · 1 line
2.54.0
#9 lore raw

[PULL 13/29] iotests: use 'driver' as collective term for either format or protocol

In reply to Daniel P. Berrangé

The I/O tests integration previously exclusively tested block formats and now also covers the NBD protocol. Replace references to 'format' with 'driver', as a generic term to collectively apply to either a format or protocol.

---
 docs/devel/testing/main.rst    | 13 +++++++------
 tests/Makefile.include         |  4 ++--
 tests/qemu-iotests/meson.build | 24 ++++++++++++------------
 3 files changed, 21 insertions(+), 20 deletions(-)
docs/devel/testing/main.rst +7 -6
index 843592e3e0..c0321d1d25 100644
@@ -237,7 +237,8 @@ check-block
 ~~~~~~~~~~~
 
 There are a variety of ways to exercise the block layer I/O tests
-via make targets.
+via make targets for a selection of formats / protocols (collectively
+referred to as ``drivers`` below).
 
 A default ``make check`` or ``make check-block`` command will exercise
 the ``qcow2`` format, using the tests tagged into the ``auto`` group
@@ -246,14 +247,14 @@ only.
 These targets accept the ``SPEED`` variable to augment the set of tests
 to run. A slightly more comprehensive test plan can be run by defining
 ``SPEED=slow``, which enables all tests for the ``qcow2`` and ``raw``
-formats. The most comprehensive test plan can be run by defining
-``SPEED=thorough``, which enables all available tests for the formats
+drivers. The most comprehensive test plan can be run by defining
+``SPEED=thorough``, which enables all available tests for the drivers
 ``luks``, ``nbd``, ``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``,
 ``vhdx``, ``vmdk``, and ``vpc``.
 
-Each of formats also has its own dedicated make target, named
-``make check-block-$FORMAT`` which will run all available tests for
-the designated format and does not require the ``SPEED`` variable
+Each of drivers also has its own dedicated make target, named
+``make check-block-$DRIVER`` which will run all available tests for
+the designated driver and does not require the ``SPEED`` variable
 to be set.
 
 See the "QEMU iotests" section below for more information on the
tests/Makefile.include +2 -2
index a820980bbe..a063a1d60f 100644
@@ -14,8 +14,8 @@ check-help:
 	@echo " $(MAKE) check-unit                    Run qobject tests"
 	@echo " $(MAKE) check-qapi-schema             Run QAPI schema tests"
 	@echo " $(MAKE) check-tracetool               Run tracetool generator tests"
-	@echo " $(MAKE) check-block                   Run block tests (all formats)"
-	@echo " $(MAKE) check-block-FORMAT            Run block tests (only for FORMAT)"
+	@echo " $(MAKE) check-block                   Run block tests (all formats/protocols)"
+	@echo " $(MAKE) check-block-DRIVER            Run block tests (only for format/protocol 'DRIVER')"
 ifneq ($(filter $(all-check-targets), check-softfloat),)
 	@echo " $(MAKE) check-softfloat               Run FPU emulation tests"
 	@echo " $(MAKE) check-tcg                     Run TCG tests"
tests/qemu-iotests/meson.build +12 -12
index 744d0b6e88..bc6132a0f7 100644
@@ -12,7 +12,7 @@ qemu_iotests_binaries = [qemu_img, qemu_io, qemu_nbd, qsd]
 qemu_iotests_env = {'PYTHON': python.full_path()}
 # If altering this definition, also update docs/devel/testing/main.rst
 # section on 'check-block' targets to reflect the changes
-qemu_iotests_formats = {
+qemu_iotests_drivers = {
   'qcow2': 'quick',
   'raw': 'slow',
   'luks': 'thorough',
@@ -33,12 +33,12 @@ endforeach
 
 qemu_iotests_check_cmd = files('check')
 
-foreach format, speed: qemu_iotests_formats
-  # Formats tagged 'quick' get the subset of tests in the 'auto'
+foreach driver, speed: qemu_iotests_drivers
+  # Drivers tagged 'quick' get the subset of tests in the 'auto'
   # group, run by default with 'make check' / 'make check-block'
   seen = []
   if speed == 'quick'
-    args = ['-tap', '-' + format, '-g', 'auto']
+    args = ['-tap', '-' + driver, '-g', 'auto']
     suites = ['block']
 
     rc = run_command(
@@ -49,13 +49,13 @@ foreach format, speed: qemu_iotests_formats
     foreach item: rc.stdout().strip().split()
       seen += item
       args = [qemu_iotests_check_cmd,
-              '-tap', '-' + format, item,
+              '-tap', '-' + driver, item,
               '--source-dir', meson.current_source_dir(),
               '--build-dir', meson.current_build_dir()]
       # Some individual tests take as long as 45 seconds
       # Bump the timeout to 3 minutes for some headroom
       # on slow machines to minimize spurious failures
-      test('io-' + format + '-' + item,
+      test('io-' + driver + '-' + item,
            python,
            args: args,
            depends: qemu_iotests_binaries,
@@ -66,9 +66,9 @@ foreach format, speed: qemu_iotests_formats
     endforeach
   endif
 
-  # Every format gets put in the format specific suite
-  suites = ['block-' + format + '-optional']
-  # Any format tagged quick or slow also gets added to slow
+  # Every driver gets put in the driver specific suite
+  suites = ['block-' + driver + '-optional']
+  # Any driver tagged quick or slow also gets added to slow
   # otherwise its tagged thorough
   if speed != 'thorough'
     suites += ['block-slow']
@@ -76,7 +76,7 @@ foreach format, speed: qemu_iotests_formats
     suites += ['block-thorough']
   endif
 
-  args = ['-tap', '-' + format]
+  args = ['-tap', '-' + driver]
 
   rc = run_command(
       [python, qemu_iotests_check_cmd] + args + ['-n'],
@@ -91,13 +91,13 @@ foreach format, speed: qemu_iotests_formats
       endif
 
       args = [qemu_iotests_check_cmd,
-              '-tap', '-' + format, item,
+              '-tap', '-' + driver, item,
               '--source-dir', meson.current_source_dir(),
               '--build-dir', meson.current_build_dir()]
       # Some individual tests take as long as 45 seconds
       # Bump the timeout to 3 minutes for some headroom
       # on slow machines to minimize spurious failures
-      test('io-' + format + '-' + item,
+      test('io-' + driver + '-' + item,
            python,
            args: args,
            depends: qemu_iotests_binaries,
Signature · 1 line
2.54.0
#10 lore raw

[PULL 09/29] scripts/mtest2make: support optional tests grouping

In reply to Daniel P. Berrangé

Currently tests can be classified into three speed groups depending on whether the meson suite name ends in '-slow' or '-thorough' or neither.

This gets turned into make targets that match the name of the meson suite, with the speed suffix stripped. e.g.

* suite=block -> 'make check-block'
* suite=block-slow -> 'make check-block SPEED=slow'
* suite=block-thorough -> 'make check-block SPEED=thorough'

The set of tests under the "thorough" speed, however, can get rather large and it would be useful to have a way to expose further make targets for directly running a particular subset of tests.

This needs a way to run a target without requiring the SPEED variable, while also not having them enabled by default as if they were 'quick' tests.

This modifies mtest2make.py to support this idea by allowing for a new suffix '-optional' on a suite. When this is present, a correspondingly named make target will be created without the '-optional' suffix which will never be run automatically.

This is intended to be combined with use of other suites. For example, a single NBD test might be added to two suites, 'block-thorough' and 'block-nbd-optional'.

This would allow running it as part of all the block tests with 'make check-block SPEED=thorough', and as part of a standalone target 'make check-block-nbd'.

---
 scripts/mtest2make.py | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)
scripts/mtest2make.py +15 -7
index 915f02d600..383ea68b16 100644
@@ -22,7 +22,7 @@ def names(self, base):
 print(r'''
 SPEED = quick
 
-.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
+.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
 .speed.slow = $(sort $(filter-out %-thorough, $1))
 .speed.thorough = $(sort $1)
 
@@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
             s = s[:-9]
             suites[s].speeds.add('thorough')
 
+def target_name(suite):
+    if suite.endswith('-optional'):
+        return suite[0:-9]
+    return suite
+
 def emit_prolog(suites, prefix):
-    all_targets = ' '.join((f'{prefix}-{k}'
+    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
                             for k in sorted(suites.keys())))
-    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
+    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
                         for k in sorted(suites.keys())))
     print()
     print(f'all-{prefix}-targets = {all_targets}')
@@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
     print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
 
 def emit_suite(name, suite, prefix):
+    tgtname = target_name(name)
     deps = ' '.join(sorted(suite.deps))
     print()
-    print(f'.{prefix}-{name}.deps = {deps}')
-    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
+    print(f'.{prefix}-{tgtname}.deps = {deps}')
+    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
 
     names = ' '.join(sorted(suite.names(name)))
-    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
-    if not name.endswith('-slow') and not name.endswith('-thorough'):
+    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
+    if not name.endswith('-slow') and \
+       not name.endswith('-thorough') and \
+       not name.endswith('-optional'):
         targets += f' {prefix} {prefix}-report.junit.xml'
     print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
     # for the "base" suite possibly add FOO-slow and FOO-thorough
Signature · 1 line
2.54.0
#11 lore raw

[PULL 24/29] qom: have object_set_props_keyval return bool

In reply to Daniel P. Berrangé

This matches the convention established by the object_set_props and object_set_propv methods.

---
 include/qom/object.h    |  4 +++-
 qom/object_interfaces.c | 13 +++++++++----
 2 files changed, 12 insertions(+), 5 deletions(-)
include/qom/object.h +3 -1
index 83b0cd01b5..822f56c87e 100644
@@ -924,8 +924,10 @@ bool type_print_class_properties(const char *type);
  *
  * For each key in the dictionary, parse the value string if needed,
  * then set the corresponding property in @obj.
+ *
+ * Returns: %true on success, %false on error.
  */
-void object_set_props_from_keyval(Object *obj, const QDict *qdict,
+bool object_set_props_from_keyval(Object *obj, const QDict *qdict,
                                   bool from_json, Error **errp);
 
 /**
qom/object_interfaces.c +9 -4
index 4377d65b76..50736b80c8 100644
@@ -44,13 +44,14 @@ bool user_creatable_can_be_deleted(UserCreatable *uc)
     }
 }
 
-static void object_set_props_from_qdict(Object *obj, const QDict *qdict,
+static bool object_set_props_from_qdict(Object *obj, const QDict *qdict,
                                         Visitor *v, Error **errp)
 {
+    ERRP_GUARD();
     const QDictEntry *e;
 
     if (!visit_start_struct(v, NULL, NULL, 0, errp)) {
-        return;
+        return false;
     }
     for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
         if (!object_property_set(obj, e->key, v, errp)) {
@@ -60,19 +61,23 @@ static void object_set_props_from_qdict(Object *obj, const QDict *qdict,
     visit_check_struct(v, errp);
 out:
     visit_end_struct(v, NULL);
+
+    return *errp == NULL;
 }
 
-void object_set_props_from_keyval(Object *obj, const QDict *qdict,
+bool object_set_props_from_keyval(Object *obj, const QDict *qdict,
                                   bool from_json, Error **errp)
 {
+    bool ret;
     Visitor *v;
     if (from_json) {
         v = qobject_input_visitor_new(QOBJECT(qdict));
     } else {
         v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
     }
-    object_set_props_from_qdict(obj, qdict, v, errp);
+    ret = object_set_props_from_qdict(obj, qdict, v, errp);
     visit_free(v);
+    return ret;
 }
 
 Object *user_creatable_add_type(const char *type, const char *id,
Signature · 1 line
2.54.0
#12 lore raw

[PULL 28/29] qom: allow object_new_with_prop* to trigger module loading

In reply to Daniel P. Berrangé

The object_new_with_prop* methods will shortly be replacing the user_creatable_add_type method. In order to do that, the object_new_with_prop* methods must allow module loading to be triggered for any types.

---
 qom/object.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
qom/object.c +1 -1
index 5a7ac457c1..0ac201de4c 100644
@@ -774,7 +774,7 @@ object_new_with_props_helper(const char *typename,
         return NULL;
     }
 
-    klass = object_class_by_name(typename);
+    klass = module_object_class_by_name(typename);
     if (!klass) {
         error_setg(errp, "invalid object type: %s", typename);
         return NULL;
Signature · 1 line
2.54.0
#13 lore raw

[PULL 25/29] qom: move object_set_prop_keyval into object.c

In reply to Daniel P. Berrangé

This matches the location of the object_set_props and object_set_propv methods, since this method is not inherently tied to the user creatable interface. As part of this, object_set_props_from_qdict is also exposed as a public API since it is still called from object_interfaces.c.

---
 include/qom/object.h    | 47 +++++++++++++++++++++++++++--------------
 qom/object.c            | 36 +++++++++++++++++++++++++++++++
 qom/object_interfaces.c | 36 -------------------------------
 3 files changed, 67 insertions(+), 52 deletions(-)
include/qom/object.h +31 -16
index 822f56c87e..be65acff93 100644
@@ -748,6 +748,37 @@ bool object_set_props(Object *obj, Error **errp, ...) G_GNUC_NULL_TERMINATED;
  */
 bool object_set_propv(Object *obj, va_list vargs, Error **errp);
 
+/**
+ * object_set_props_from_qdict:
+ * @obj: a QOM object
+ * @qdict: a dictionary with the properties to be set
+ * @v: a visitor to iterate over @dict
+ * @errp: pointer to error object
+ *
+ * For each key in the dictionary, set the corresponding
+ * property in @obj.
+ *
+ * Returns: %true on success, %false on error.
+ */
+bool object_set_props_from_qdict(Object *obj, const QDict *qdict,
+                                 Visitor *v, Error **errp);
+
+/**
+ * object_set_props_from_keyval:
+ * @obj: a QOM object
+ * @qdict: a dictionary with the properties to be set
+ * @from_json: true if leaf values of @qdict are typed, false if they
+ * are strings
+ * @errp: pointer to error object
+ *
+ * For each key in the dictionary, parse the value string if needed,
+ * then set the corresponding property in @obj.
+ *
+ * Returns: %true on success, %false on error.
+ */
+bool object_set_props_from_keyval(Object *obj, const QDict *qdict,
+                                  bool from_json, Error **errp);
+
 /**
  * object_initialize:
  * @obj: A pointer to the memory to be used for the object.
@@ -914,22 +945,6 @@ type_init(do_qemu_init_ ## type_array)
  */
 bool type_print_class_properties(const char *type);
 
-/**
- * object_set_props_from_keyval:
- * @obj: a QOM object
- * @qdict: a dictionary with the properties to be set
- * @from_json: true if leaf values of @qdict are typed, false if they
- * are strings
- * @errp: pointer to error object
- *
- * For each key in the dictionary, parse the value string if needed,
- * then set the corresponding property in @obj.
- *
- * Returns: %true on success, %false on error.
- */
-bool object_set_props_from_keyval(Object *obj, const QDict *qdict,
-                                  bool from_json, Error **errp);
-
 /**
  * object_class_dynamic_cast_assert:
  * @klass: The #ObjectClass to attempt to cast.
qom/object.c +36 -0
index ecae322d2a..3f2569d654 100644
@@ -23,6 +23,7 @@
 #include "qapi/qobject-input-visitor.h"
 #include "qapi/forward-visitor.h"
 #include "qapi/qapi-builtin-visit.h"
+#include "qobject/qdict.h"
 #include "qobject/qjson.h"
 #include "qemu/id.h"
 #include "qapi/qmp/qerror.h"
@@ -838,6 +839,41 @@ bool object_set_propv(Object *obj,
     return true;
 }
 
+bool object_set_props_from_qdict(Object *obj, const QDict *qdict,
+                                 Visitor *v, Error **errp)
+{
+    ERRP_GUARD();
+    const QDictEntry *e;
+
+    if (!visit_start_struct(v, NULL, NULL, 0, errp)) {
+        return false;
+    }
+    for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+        if (!object_property_set(obj, e->key, v, errp)) {
+            goto out;
+        }
+    }
+    visit_check_struct(v, errp);
+out:
+    visit_end_struct(v, NULL);
+
+    return *errp == NULL;
+}
+
+bool object_set_props_from_keyval(Object *obj, const QDict *qdict,
+                                  bool from_json, Error **errp)
+{
+    bool ret;
+    Visitor *v;
+    if (from_json) {
+        v = qobject_input_visitor_new(QOBJECT(qdict));
+    } else {
+        v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
+    }
+    ret = object_set_props_from_qdict(obj, qdict, v, errp);
+    visit_free(v);
+    return ret;
+}
 
 Object *object_dynamic_cast(Object *obj, const char *typename)
 {
qom/object_interfaces.c +0 -36
index 50736b80c8..e0a3cd8d0f 100644
@@ -44,42 +44,6 @@ bool user_creatable_can_be_deleted(UserCreatable *uc)
     }
 }
 
-static bool object_set_props_from_qdict(Object *obj, const QDict *qdict,
-                                        Visitor *v, Error **errp)
-{
-    ERRP_GUARD();
-    const QDictEntry *e;
-
-    if (!visit_start_struct(v, NULL, NULL, 0, errp)) {
-        return false;
-    }
-    for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
-        if (!object_property_set(obj, e->key, v, errp)) {
-            goto out;
-        }
-    }
-    visit_check_struct(v, errp);
-out:
-    visit_end_struct(v, NULL);
-
-    return *errp == NULL;
-}
-
-bool object_set_props_from_keyval(Object *obj, const QDict *qdict,
-                                  bool from_json, Error **errp)
-{
-    bool ret;
-    Visitor *v;
-    if (from_json) {
-        v = qobject_input_visitor_new(QOBJECT(qdict));
-    } else {
-        v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
-    }
-    ret = object_set_props_from_qdict(obj, qdict, v, errp);
-    visit_free(v);
-    return ret;
-}
-
 Object *user_creatable_add_type(const char *type, const char *id,
                                 const QDict *qdict,
                                 Visitor *v, Error **errp)
Signature · 1 line
2.54.0
#14 lore raw

[PULL 11/29] docs/devel/testing: expand documentation for 'make check-block'

In reply to Daniel P. Berrangé

Explain in greater detail what 'check-block' will run for each format, and also document the new format specific targets.

---
 docs/devel/testing/main.rst    | 25 ++++++++++++++++++++++---
 tests/qemu-iotests/meson.build |  2 ++
 2 files changed, 24 insertions(+), 3 deletions(-)
docs/devel/testing/main.rst +22 -3
index b01a374865..f691c624c1 100644
@@ -236,9 +236,28 @@ same commit that alters the generator code.
 check-block
 ~~~~~~~~~~~
 
-``make check-block`` runs a subset of the block layer iotests (the tests that
-are in the "auto" group).
-See the "QEMU iotests" section below for more information.
+There are a variety of ways to exercise the block layer I/O tests
+via make targets.
+
+A default ``make check`` or ``make check-block`` command will exercise
+the ``qcow2`` format, using the tests tagged into the ``auto`` group
+only.
+
+These targets accept the ``SPEED`` variable to augment the set of tests
+to run. A slightly more comprehensive test plan can be run by defining
+``SPEED=slow``, which enables all tests for the ``qcow2`` and ``raw``
+formats. The most comprehensive test plan can be run by defining
+``SPEED=thorough``, which enables all available tests for the formats
+``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
+``vmdk``, and ``vpc``.
+
+Each of formats also has its own dedicated make target, named
+``make check-block-$FORMAT`` which will run all available tests for
+the designated format and does not require the ``SPEED`` variable
+to be set.
+
+See the "QEMU iotests" section below for more information on the
+block I/O test framework that is leveraged by these ``make`` targets.
 
 .. _qemu-iotests:
 
tests/qemu-iotests/meson.build +2 -0
index 16a5e39476..66b09d6b97 100644
@@ -10,6 +10,8 @@ endif
 
 qemu_iotests_binaries = [qemu_img, qemu_io, qemu_nbd, qsd]
 qemu_iotests_env = {'PYTHON': python.full_path()}
+# If altering this definition, also update docs/devel/testing/main.rst
+# section on 'check-block' targets to reflect the changes
 qemu_iotests_formats = {
   'qcow2': 'quick',
   'raw': 'slow',
Signature · 1 line
2.54.0
#15 lore raw

[PULL 10/29] iotests: add a meson suite / make target per block I/O tests format

In reply to Daniel P. Berrangé

Currently each block format is classified as either 'quick', 'slow' or 'thorough' and this controls whether its I/O tests are added to the meson suites 'block-quick', 'block-slow' or 'block-thorough'.

This suites are exposed the 'check-block' make target, accepting the optional SPEED variable.

As we add more formats to the 'thorough' group, however, it becomes increasingly large and time consuming to run. What is needed is a make target that can exercise all tests for an individual format, regardless of speed classification.

This makes use of the previous enhancement to mtest2make.py to introduce new meson suites 'block-$FORMAT-optional', which translate to new top level make targets 'check-block-$FORMAT'. These new targets always run all tests and as such do not need the SPEED variable to be set, but are not triggered by 'make check' or 'make check-block'.

---
 tests/Makefile.include         | 3 ++-
 tests/qemu-iotests/meson.build | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
tests/Makefile.include +2 -1
index f257288194..a820980bbe 100644
@@ -14,7 +14,8 @@ check-help:
 	@echo " $(MAKE) check-unit                    Run qobject tests"
 	@echo " $(MAKE) check-qapi-schema             Run QAPI schema tests"
 	@echo " $(MAKE) check-tracetool               Run tracetool generator tests"
-	@echo " $(MAKE) check-block                   Run block tests"
+	@echo " $(MAKE) check-block                   Run block tests (all formats)"
+	@echo " $(MAKE) check-block-FORMAT            Run block tests (only for FORMAT)"
 ifneq ($(filter $(all-check-targets), check-softfloat),)
 	@echo " $(MAKE) check-softfloat               Run FPU emulation tests"
 	@echo " $(MAKE) check-tcg                     Run TCG tests"
tests/qemu-iotests/meson.build +2 -1
index 1a24d801a3..16a5e39476 100644
@@ -62,7 +62,8 @@ foreach format, speed: qemu_iotests_formats
     endforeach
   endif
 
-  suites = []
+  # Every format gets put in the format specific suite
+  suites = ['block-' + format + '-optional']
   # Any format tagged quick or slow also gets added to slow
   # otherwise its tagged thorough
   if speed != 'thorough'
Signature · 1 line
2.54.0
#16 lore raw

[PULL 14/29] iotests: validate dmsetup result in test 128

In reply to Daniel P. Berrangé

The I/O test 128 uses 'dmsetup create' to create a device, optionally using sudo to elevate privileges.

This dmsetup command works in GitLab CI, however, the test then fails with a missing device name:

1..1

  # running raw 128
  not ok raw 128
  ----------------------------------- stderr -----------------------------------
  --- /builds/berrange/qemu/tests/qemu-iotests/128.out
  +++ /builds/berrange/qemu/build/scratch/raw-file-128/128.out.bad
  @@ -1,5 +1,5 @@
   QA output created by 128

== reading from error device ==
-read failed: Input/output error
+qemu-io: can't open device /dev/mapper/eiodev16546: Could not open '/dev/mapper/eiodev16546': No such file or directory
*** done

(test program exited with status code 1)

It is believed that this is due to the build env using a manually populated /dev, such that the device mapper node won't ever appear. It is not a race, since a test adding a sleep did not result in the device appearing.

---
 tests/qemu-iotests/128 | 6 ++++++
 1 file changed, 6 insertions(+)
tests/qemu-iotests/128 +6 -0
index d0e00d24b1..d75b1a451a 100755
@@ -42,6 +42,12 @@ _setup_eiodev()
 		echo "0 $((1024 * 1024 * 1024 / 512)) error" | \
 			$cmd dmsetup create "$devname" 2>/dev/null
 		if [ "$?" -eq 0 ]; then
+			DEV="/dev/mapper/$devname"
+			if ! -e $DEV
+			then
+				_notrun "Device $DEV not appearing"
+			fi
+
 			sudo="$cmd"
 			return
 		fi
Signature · 1 line
2.54.0
#17 lore raw

[PULL 18/29] gitlab: remove I/O tests from build-tcg-disabled job

In reply to Daniel P. Berrangé

Now that we have dedicated CI jobs for running I/O tests on each supported format/protocol, we no longer need to special case a run of a hand picked set of tests in the build-tcg-disabled job.

---
 .gitlab-ci.d/buildtest.yml | 9 ---------
 1 file changed, 9 deletions(-)
.gitlab-ci.d/buildtest.yml +0 -9
index cf0281ecd6..d054349030 100644
@@ -366,15 +366,6 @@ build-tcg-disabled:
     - make -j"$JOBS"
     - make check-unit
     - make check-qapi-schema
-    - ./run tests/qemu-iotests/check -raw 001 002 003 004 005 008 009
-            010 011 012 021 025 032 033 048 052 063 077 086 101 104 106
-            113 148 150 151 152 157 159 160 163 170 171 184 192 194 208
-            221 226 227 236 253 277 image-fleecing
-    - ./run tests/qemu-iotests/check -qcow2 028 051 056 057 058 065 068
-            082 085 091 095 096 102 122 124 132 139 142 144 145 151 152
-            155 157 165 194 196 200 202 208 209 216 218 227 234 246 247
-            248 250 254 255 257 258 260 261 262 263 264 270 272 273 277
-            279 image-fleecing
     - make distclean
 
 build-user:
Signature · 1 line
2.54.0
#18 lore raw

[PULL 23/29] qom: shorten name of object_set_properties_from_keyval

In reply to Daniel P. Berrangé

This matches the convention established by the object_set_props and object_set_propv methods.

---
 include/qom/object.h    |  6 +++---
 qom/object_interfaces.c | 12 ++++++------
 system/qdev-monitor.c   |  4 ++--
 system/vl.c             |  7 ++++---
 4 files changed, 15 insertions(+), 14 deletions(-)
include/qom/object.h +3 -3
index a905a9e42f..83b0cd01b5 100644
@@ -915,7 +915,7 @@ type_init(do_qemu_init_ ## type_array)
 bool type_print_class_properties(const char *type);
 
 /**
- * object_set_properties_from_keyval:
+ * object_set_props_from_keyval:
  * @obj: a QOM object
  * @qdict: a dictionary with the properties to be set
  * @from_json: true if leaf values of @qdict are typed, false if they
@@ -925,8 +925,8 @@ bool type_print_class_properties(const char *type);
  * For each key in the dictionary, parse the value string if needed,
  * then set the corresponding property in @obj.
  */
-void object_set_properties_from_keyval(Object *obj, const QDict *qdict,
-                                       bool from_json, Error **errp);
+void object_set_props_from_keyval(Object *obj, const QDict *qdict,
+                                  bool from_json, Error **errp);
 
 /**
  * object_class_dynamic_cast_assert:
qom/object_interfaces.c +6 -6
index 415cbee8c5..4377d65b76 100644
@@ -44,8 +44,8 @@ bool user_creatable_can_be_deleted(UserCreatable *uc)
     }
 }
 
-static void object_set_properties_from_qdict(Object *obj, const QDict *qdict,
-                                             Visitor *v, Error **errp)
+static void object_set_props_from_qdict(Object *obj, const QDict *qdict,
+                                        Visitor *v, Error **errp)
 {
     const QDictEntry *e;
 
@@ -62,8 +62,8 @@ out:
     visit_end_struct(v, NULL);
 }
 
-void object_set_properties_from_keyval(Object *obj, const QDict *qdict,
-                                       bool from_json, Error **errp)
+void object_set_props_from_keyval(Object *obj, const QDict *qdict,
+                                  bool from_json, Error **errp)
 {
     Visitor *v;
     if (from_json) {
@@ -71,7 +71,7 @@ void object_set_properties_from_keyval(Object *obj, const QDict *qdict,
     } else {
         v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
     }
-    object_set_properties_from_qdict(obj, qdict, v, errp);
+    object_set_props_from_qdict(obj, qdict, v, errp);
     visit_free(v);
 }
 
@@ -110,7 +110,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
 
     assert(qdict);
     obj = object_new_with_class(klass);
-    object_set_properties_from_qdict(obj, qdict, v, &local_err);
+    object_set_props_from_qdict(obj, qdict, v, &local_err);
     if (local_err) {
         goto out;
     }
system/qdev-monitor.c +2 -2
index e5b55e3004..dfc95a08c1 100644
@@ -730,8 +730,8 @@ DeviceState *qdev_device_add_from_qdict(const QDict *opts,
     qdict_del(properties, "bus");
     qdict_del(properties, "id");
 
-    object_set_properties_from_keyval(&dev->parent_obj, properties, from_json,
-                                      errp);
+    object_set_props_from_keyval(&dev->parent_obj, properties, from_json,
+                                 errp);
     qobject_unref(properties);
     if (*errp) {
         goto err_del_dev;
system/vl.c +4 -3
index e690aa3ed8..da36b2c6e1 100644
@@ -2010,7 +2010,8 @@ static bool object_create_early(const char *type)
 
 static void qemu_apply_machine_options(QDict *qdict)
 {
-    object_set_properties_from_keyval(OBJECT(current_machine), qdict, false, &error_fatal);
+    object_set_props_from_keyval(OBJECT(current_machine), qdict,
+                                 false, &error_fatal);
 
     if (semihosting_enabled(false) && !semihosting_get_argc()) {
         /* fall back to the -kernel/-append */
@@ -2225,8 +2226,8 @@ static void qemu_create_machine(QDict *qdict)
             keyval_parse(machine_class->default_machine_opts, NULL, NULL,
                          &error_abort);
         qemu_apply_legacy_machine_options(default_opts);
-        object_set_properties_from_keyval(OBJECT(current_machine), default_opts,
-                                          false, &error_abort);
+        object_set_props_from_keyval(OBJECT(current_machine), default_opts,
+                                     false, &error_abort);
         qobject_unref(default_opts);
     }
 }
Signature · 1 line
2.54.0
#19 lore raw

[PULL 20/29] hw/vfio-user: use a valid object ID for iothread

In reply to Daniel P. Berrangé

The objectg_new_with_propv/props methods failed to validate that the QOM "id" was well formed. This allowed the vfio-user code to use an invalid ID of "VFIO user" (space is not permitted) in its internal code.

---
 hw/vfio-user/proxy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/vfio-user/proxy.c +1 -1
index 8b7cc36231..e02a45e997 100644
@@ -933,7 +933,7 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp)
     qemu_cond_init(&proxy->close_cv);
 
     if (vfio_user_iothread == NULL) {
-        vfio_user_iothread = iothread_create("VFIO user", errp);
+        vfio_user_iothread = iothread_create("vfio-user", errp);
     }
 
     proxy->ctx = iothread_get_aio_context(vfio_user_iothread);
Signature · 1 line
2.54.0
#20 lore raw

[PULL 17/29] gitlab: add jobs for thorough block tests

In reply to Daniel P. Berrangé

CI is only exercising the qcow2 'auto' tests currently. As a result we get no exposure of changes which cause regressions in other block format drivers.

This adds new CI jobs for each block format, that will run the target 'make check-block-$FORMAT'. The jobs are separate so that we have the ability to make each format gating or not, depending on their level of reliability.

The 'centos' image is used to run the I/O tests since several tests have an implicit dependency on x86_64-softmmu, and thus break with other architecture targets. The 'centos' build job is the only one that creates the x86_64-softmmu target in CI. Ideally this target portability in I/O tests would be fixed to avoid this limitation.

---
 .gitlab-ci.d/buildtest.yml | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
.gitlab-ci.d/buildtest.yml +18 -0
index 4b1949a3a5..cf0281ecd6 100644
@@ -174,6 +174,24 @@ build-system-centos:
       x86_64-softmmu rx-softmmu sh4-softmmu
     MAKE_CHECK_ARGS: check-build
 
+
+# NB: block-XXX jobs use 'centos' since that is the build
+# job that provides the x86_64-softmmu.  Some I/O tests
+# are currently buggy and blindly assume characteristics
+# of x86 (such as PCIe) causing failures with other arches
+
+block:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: "check-block-$FORMAT"
+  parallel:
+    matrix:
+      - FORMAT: [luks, nbd, parallels, qcow2, qed, raw, vdi, vhdx, vmdk, vpc]
+
 # Previous QEMU release. Used for cross-version migration tests.
 build-previous-qemu:
   extends: .native_build_job_template
Signature · 1 line
2.54.0
#21 lore raw

[PULL 08/29] scripts/mtest2make: ensure output has stable sorting

In reply to Daniel P. Berrangé

When debugging mtest2make.py changes it is important to be able to compare the old and new output. This requires that any lists in the output have stable sort ordering.

---
 scripts/mtest2make.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
scripts/mtest2make.py +6 -4
index 4b252defc3..915f02d600 100644
@@ -67,8 +67,10 @@ def process_tests(test, targets, suites):
             suites[s].speeds.add('thorough')
 
 def emit_prolog(suites, prefix):
-    all_targets = ' '.join((f'{prefix}-{k}' for k in suites.keys()))
-    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml' for k in suites.keys()))
+    all_targets = ' '.join((f'{prefix}-{k}'
+                            for k in sorted(suites.keys())))
+    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
+                        for k in sorted(suites.keys())))
     print()
     print(f'all-{prefix}-targets = {all_targets}')
     print(f'all-{prefix}-xml = {all_xml}')
@@ -81,12 +83,12 @@ def emit_prolog(suites, prefix):
     print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
 
 def emit_suite(name, suite, prefix):
-    deps = ' '.join(suite.deps)
+    deps = ' '.join(sorted(suite.deps))
     print()
     print(f'.{prefix}-{name}.deps = {deps}')
     print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
 
-    names = ' '.join(suite.names(name))
+    names = ' '.join(sorted(suite.names(name)))
     targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
     if not name.endswith('-slow') and not name.endswith('-thorough'):
         targets += f' {prefix} {prefix}-report.junit.xml'
Signature · 1 line
2.54.0
#22 lore raw

[PULL 12/29] iotests: add nbd and luks to the I/O test suites

In reply to Daniel P. Berrangé

This introduces new suites for running I/O tests on NBD and LUKS drivers, giving new make targets

* make check-block-luks
* make check-block-nbd

as well as adding their tests to 'make check-block SPEED=thorough'

---
 docs/devel/testing/main.rst    | 4 ++--
 tests/qemu-iotests/meson.build | 4 +++-
 2 files changed, 5 insertions(+), 3 deletions(-)
docs/devel/testing/main.rst +2 -2
index f691c624c1..843592e3e0 100644
@@ -248,8 +248,8 @@ to run. A slightly more comprehensive test plan can be run by defining
 ``SPEED=slow``, which enables all tests for the ``qcow2`` and ``raw``
 formats. The most comprehensive test plan can be run by defining
 ``SPEED=thorough``, which enables all available tests for the formats
-``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
-``vmdk``, and ``vpc``.
+``luks``, ``nbd``, ``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``,
+``vhdx``, ``vmdk``, and ``vpc``.
 
 Each of formats also has its own dedicated make target, named
 ``make check-block-$FORMAT`` which will run all available tests for
tests/qemu-iotests/meson.build +3 -1
index 66b09d6b97..744d0b6e88 100644
@@ -15,12 +15,14 @@ qemu_iotests_env = {'PYTHON': python.full_path()}
 qemu_iotests_formats = {
   'qcow2': 'quick',
   'raw': 'slow',
+  'luks': 'thorough',
+  'nbd': 'thorough',
   'parallels': 'thorough',
   'qed': 'thorough',
   'vdi': 'thorough',
   'vhdx': 'thorough',
   'vmdk': 'thorough',
-  'vpc': 'thorough'
+  'vpc': 'thorough',
 }
 
 foreach k, v : emulators
Signature · 1 line
2.54.0
#23 lore raw

[PULL 16/29] iotests: mark 151, 181, 185 & 308 as flaky tests

In reply to Daniel P. Berrangé

Introduce a "_flaky_test" function for shell based I/O tests which accepts a GitLab issue URL, and causes the I/O test to be skipped unless the $QEMU_TEST_FLAKY_TESTS environment variable is set.

The equivalent "skip_flaky" test decorator is added for python based I/O tests with the same behaviour.

This is used by:

* Test 151 which fails in QEMU private AWS runners due to failure to make progress in time
* Test 181 which fails with a non-responsive QEMU on AWS runners with the QED format
* Test 185 which fails in GitLab shared runners
* Test 308 which fails to see disk usage increase after fallocate

---
 tests/qemu-iotests/151        |  1 +
 tests/qemu-iotests/181        |  2 ++
 tests/qemu-iotests/185        |  1 +
 tests/qemu-iotests/308        |  2 ++
 tests/qemu-iotests/common.rc  | 16 ++++++++++++++++
 tests/qemu-iotests/iotests.py | 16 ++++++++++++++++
 6 files changed, 38 insertions(+)
tests/qemu-iotests/151 +1 -0
index 9b9c815db5..647194094a 100755
@@ -308,6 +308,7 @@ class TestThrottledWithNbdExportBase(iotests.QMPTestCase):
 class TestLowThrottledWithNbdExport(TestThrottledWithNbdExportBase):
     iops = 16
 
+    @iotests.skip_flaky("https://gitlab.com/qemu-project/qemu/-/work_items/3513")
     def testUnderLoad(self):
         '''
         Throttle the source node, then issue a whole bunch of external requests
tests/qemu-iotests/181 +2 -0
index dc90a10757..b904e8ce01 100755
@@ -48,6 +48,8 @@ _unsupported_fmt qcow vdi vhdx vmdk vpc vvfat parallels
 _supported_proto generic
 _supported_os Linux
 
+_flaky_test https://gitlab.com/qemu-project/qemu/-/work_items/3515
+
 size=64M
 _make_test_img $size
 
tests/qemu-iotests/185 +1 -0
index 17489fb91c..a62ae8d329 100755
@@ -50,6 +50,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
+_flaky_test https://gitlab.com/qemu-project/qemu/-/issues/3270
 
 size=$((64 * 1048576))
 TEST_IMG="${TEST_IMG}.base" _make_test_img $size
tests/qemu-iotests/308 +2 -0
index f4a06a522e..a2290825f4 100755
@@ -53,6 +53,8 @@ _supported_proto file # We create the FUSE export manually
 _supported_os Linux # We need /dev/urandom
 _require_disk_usage
 
+_flaky_test https://gitlab.com/qemu-project/qemu/-/work_items/3514
+
 # $1: Export ID
 # $2: Options (beyond the node-name and ID)
 # $3: Expected return value (defaults to 'return')
tests/qemu-iotests/common.rc +16 -0
index 731e4b2b99..298bc483e0 100644
@@ -1088,5 +1088,21 @@ _qcow2_dump_header()
     fi
 }
 
+# This must be referenced after any _require_  lines, so that
+# test filtering happens first
+_flaky_test()
+{
+    if test -z "$1"
+    then
+	echo "A GitLab issue URL must be provided for a flaky test"
+	exit 1
+    fi
+
+    if test -z "$QEMU_TEST_FLAKY_TESTS"
+    then
+	_notrun "Test is flaky (see $1) and \$QEMU_TEST_FLAKY_TESTS is not set"
+    fi
+}
+
 # make sure this script returns success
 true
tests/qemu-iotests/iotests.py +16 -0
index 05274772ce..072be80e07 100644
@@ -1603,6 +1603,22 @@ def func_wrapper(*args, **kwargs):
             return func(*args, **kwargs)
     return func_wrapper
 
+def skip_flaky(bugurl):
+    '''Skip Test Decorator
+       Always skips test due to unreliable design.
+       Requires a bug report URL for historical record.'''
+    def skip_test_decorator(func):
+        def func_wrapper(*args, **kwargs):
+            if os.environ.get("QEMU_TEST_FLAKY_TESTS", None) is None:
+                case_notrun(
+                    ('{}: test is flaky (see {}) and $QEMU_TEST_FLAKY_TESTS ' +
+                     'is not set').format(args[0], bugurl))
+                return None
+            else:
+                return func(*args, **kwargs)
+        return func_wrapper
+    return skip_test_decorator
+
 # We need to filter out the time taken from the output so that
 # qemu-iotest can reliably diff the results against master output,
 # and hide skipped tests from the reference output.
Signature · 1 line
2.54.0
#24 lore raw

[PULL 27/29] qom: fix ability to create objects without a parent

In reply to Daniel P. Berrangé

object_new_with_propv allowed id/parent to be optional, in which case the caller was expected to own the returned object. Unfortunately a trailing object_unref() meant that the returned object was already freed.

It is confusing to have a single method with two different ownership scenarios for the returned object.

Make id/parent mandatory in object_new_with_propv once more, and add a new object_new_with_propv_parentless that does not accept id/parent at all and lets the caller own the returned reference.

The helper method has abstracted the way properties are represented and set in order to facilitate the subsequent commit.

Unit tests are added to address the root cause that allowed the bug to slip through in commit 6134d752.

Fixes 6134d7522e5 ("qom: don't require user creatable objects to be registered")
---
 include/qom/object.h            | 47 ++++++++++++++++++
 qom/object.c                    | 74 +++++++++++++++++++++++----
 tests/unit/check-qom-proplist.c | 88 ++++++++++++++++++++++++++++-----
 3 files changed, 187 insertions(+), 22 deletions(-)
include/qom/object.h +47 -0
index fb1cd92d97..11f55613fc 100644
@@ -719,6 +719,53 @@ Object *object_new_with_props_from_qdict(const char *typename,
                                          Visitor *v,
                                          Error **errp);
 
+/**
+ * object_new_with_props_parentless:
+ * @typename:  The name of the type of the object to instantiate.
+ * @errp: pointer to error object
+ * @...: list of property names and values
+ *
+ * Behaviour as object_new_with_props(), except the object
+ * will not be added to any parent and thus the caller will
+ * own the returned instance. The caller must call
+ * object_unref when it is no longer required.
+ */
+Object *object_new_with_props_parentless(const char *typename,
+                                         Error **errp,
+                                         ...) G_GNUC_NULL_TERMINATED;
+
+/**
+ * object_new_with_propv_parentless:
+ * @typename:  The name of the type of the object to instantiate.
+ * @vargs: list of property names and values
+ * @errp: pointer to error object
+ *
+ * Behaviour as object_new_with_propv(), except the object
+ * will not be added to any parent and thus the caller will
+ * own the returned instance. The caller must call
+ * object_unref when it is no longer required.
+ */
+Object *object_new_with_propv_parentless(const char *typename,
+                                         va_list vargs,
+                                         Error **errp);
+
+/**
+ * object_new_with_props_from_qdict_parentless:
+ * @typename:  The name of the type of the object to instantiate.
+ * @props: dictionary of property names and values
+ * @v: visitor to iterate over @props
+ * @errp: pointer to error object
+ *
+ * Behaviour as object_new_with_props_from_qdict(), except the
+ * object will not be added to any parent and thus the caller
+ * will own the returned instance. The caller must call
+ * object_unref when it is no longer required.
+ */
+Object *object_new_with_props_from_qdict_parentless(const char *typename,
+                                                    const QDict *props,
+                                                    Visitor *v,
+                                                    Error **errp);
+
 /**
  * object_set_props:
  * @obj: the object instance to set properties on
qom/object.c +65 -9

120 lines not on this page · raw message

tests/unit/check-qom-proplist.c +75 -13

130 lines not on this page · raw message

Signature · 1 line
2.54.0
#25 lore raw

[PULL 21/29] qom: validate ID format when creating objects

In reply to Daniel P. Berrangé

The object_new_with_props/propv methods failed to validate the ID string format, thus diverging from user_creatable_add_type.

---
 qom/object.c | 9 +++++++++
 1 file changed, 9 insertions(+)
qom/object.c +9 -0
index 4a15151c3b..08fc840f39 100644
@@ -24,6 +24,8 @@
 #include "qapi/forward-visitor.h"
 #include "qapi/qapi-builtin-visit.h"
 #include "qobject/qjson.h"
+#include "qemu/id.h"
+#include "qapi/qmp/qerror.h"
 #include "trace.h"
 
 /* TODO: replace QObject with a simpler visitor to avoid a dependency
@@ -755,6 +757,13 @@ Object *object_new_with_propv(const char *typename,
     ObjectClass *klass;
     UserCreatable *uc;
 
+    if (id != NULL && !id_wellformed(id)) {
+        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "id", "an identifier");
+        error_append_hint(errp, "Identifiers consist of letters, digits, "
+                          "'-', '.', '_', starting with a letter.\n");
+        return NULL;
+    }
+
     klass = object_class_by_name(typename);
     if (!klass) {
         error_setg(errp, "invalid object type: %s", typename);
Signature · 1 line
2.54.0
#26 lore raw

[PULL 26/29] qom: add object_new_with_props_from_qdict

In reply to Daniel P. Berrangé

This will be used to replace user_creatable_add_type with an impl that shares most code with object_new_with_props, and is not tied to the user creatable interface.

---
 include/qom/object.h | 19 +++++++++++
 qom/object.c         | 75 ++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 88 insertions(+), 6 deletions(-)
include/qom/object.h +19 -0
index be65acff93..fb1cd92d97 100644
@@ -700,6 +700,25 @@ Object *object_new_with_propv(const char *typename,
                               va_list vargs,
                               Error **errp);
 
+/**
+ * object_new_with_props_from_qdict:
+ * @typename:  The name of the type of the object to instantiate.
+ * @parent: the parent object
+ * @id: The unique ID of the object
+ * @props: dictionary of property names and values
+ * @v: visitor to iterate over @props
+ * @errp: pointer to error object
+ *
+ * A variant of object_new_with_props() which accepts the
+ * properties in a QDict.
+ */
+Object *object_new_with_props_from_qdict(const char *typename,
+                                         Object *parent,
+                                         const char *id,
+                                         const QDict *props,
+                                         Visitor *v,
+                                         Error **errp);
+
 /**
  * object_set_props:
  * @obj: the object instance to set properties on
qom/object.c +69 -6

93 lines not on this page · raw message

Signature · 1 line
2.54.0
#27 lore raw

[PULL 29/29] qom: drop user_creatable_add_type method

In reply to Daniel P. Berrangé

This can be replaced by object_new_with_props_from_qdict, which does functionally the same job, but the caller does not own the returned reference, instead the parent object owns it.

In one case we can use object_new_with_props_from_qdict_owned instead since the object is not intended to have any parent.

---
 authz/listfile.c                |  4 +-
 include/qom/object_interfaces.h | 18 ---------
 qom/object_interfaces.c         | 70 ++-------------------------------
 tests/unit/check-qom-proplist.c |  5 +--
 4 files changed, 7 insertions(+), 90 deletions(-)
authz/listfile.c +2 -2
index 13741d5a72..23655f8663 100644
@@ -79,8 +79,8 @@ qauthz_list_file_load(QAuthZListFile *fauthz, Error **errp)
 
     v = qobject_input_visitor_new(obj);
 
-    ret = (QAuthZ *)user_creatable_add_type(TYPE_QAUTHZ_LIST,
-                                            NULL, pdict, v, errp);
+    ret = QAUTHZ(object_new_with_props_from_qdict_parentless(
+                     TYPE_QAUTHZ_LIST, pdict, v, errp));
 
  cleanup:
     visit_free(v);
include/qom/object_interfaces.h +0 -18
index 02b11a7ef0..e2b8615617 100644
@@ -69,24 +69,6 @@ bool user_creatable_complete(UserCreatable *uc, Error **errp);
  */
 bool user_creatable_can_be_deleted(UserCreatable *uc);
 
-/**
- * user_creatable_add_type:
- * @type: the object type name
- * @id: the unique ID for the object
- * @qdict: the object properties
- * @v: the visitor
- * @errp: if an error occurs, a pointer to an area to store the error
- *
- * Create an instance of the user creatable object @type, placing
- * it in the object composition tree with name @id, initializing
- * it with properties from @qdict
- *
- * Returns: the newly created object or NULL on error
- */
-Object *user_creatable_add_type(const char *type, const char *id,
-                                const QDict *qdict,
-                                Visitor *v, Error **errp);
-
 /**
  * user_creatable_add_qapi:
  * @options: the object definition
qom/object_interfaces.c +3 -67

87 lines not on this page · raw message

tests/unit/check-qom-proplist.c +2 -3
index 954c898ce1..89de92b7d9 100644
@@ -461,10 +461,9 @@ static void test_dummy_createlist_parentless(void)
 static bool test_create_obj(QDict *qdict, Error **errp)
 {
     Visitor *v = qobject_input_visitor_new_keyval(QOBJECT(qdict));
-    Object *obj = user_creatable_add_type(TYPE_DUMMY, "dev0", qdict, v, errp);
-
+    Object *obj = object_new_with_props_from_qdict(
+        TYPE_DUMMY, object_get_objects_root(), "dev0", qdict, v, errp);
     visit_free(v);
-    object_unref(obj);
     return !!obj;
 }
 
Signature · 1 line
2.54.0
#28 lore raw

[PULL 22/29] qom: make errp last param in methods taking va_list

In reply to Daniel P. Berrangé

object_new_with_props can't put 'errp' last due to the use of variadic arguments. That constraint does not apply to the use of va_list with object_new_with_propv, so follow normal practice with 'errp' placement.

The same rationale applies to object_set_propv.

---
 include/qom/object.h            | 10 +++++-----
 qom/object.c                    | 20 ++++++++++----------
 tests/unit/check-qom-proplist.c |  4 ++--
 3 files changed, 17 insertions(+), 17 deletions(-)
include/qom/object.h +5 -5
index 2a2829343d..a905a9e42f 100644
@@ -689,16 +689,16 @@ Object *object_new_with_props(const char *typename,
  * @typename:  The name of the type of the object to instantiate.
  * @parent: the parent object
  * @id: The unique ID of the object
- * @errp: pointer to error object
  * @vargs: list of property names and values
+ * @errp: pointer to error object
  *
  * See object_new_with_props() for documentation.
  */
 Object *object_new_with_propv(const char *typename,
                               Object *parent,
                               const char *id,
-                              Error **errp,
-                              va_list vargs);
+                              va_list vargs,
+                              Error **errp);
 
 /**
  * object_set_props:
@@ -739,14 +739,14 @@ bool object_set_props(Object *obj, Error **errp, ...) G_GNUC_NULL_TERMINATED;
 /**
  * object_set_propv:
  * @obj: the object instance to set properties on
- * @errp: pointer to error object
  * @vargs: list of property names and values
+ * @errp: pointer to error object
  *
  * See object_set_props() for documentation.
  *
  * Returns: %true on success, %false on error.
  */
-bool object_set_propv(Object *obj, Error **errp, va_list vargs);
+bool object_set_propv(Object *obj, va_list vargs, Error **errp);
 
 /**
  * object_initialize:
qom/object.c +10 -10
index 08fc840f39..ecae322d2a 100644
@@ -541,7 +541,7 @@ bool object_initialize_child_with_propsv(Object *parentobj,
     object_initialize(childobj, size, type);
     obj = OBJECT(childobj);
 
-    if (!object_set_propv(obj, errp, vargs)) {
+    if (!object_set_propv(obj, vargs, errp)) {
         goto out;
     }
 
@@ -740,7 +740,7 @@ Object *object_new_with_props(const char *typename,
     Object *obj;
 
     va_start(vargs, errp);
-    obj = object_new_with_propv(typename, parent, id, errp, vargs);
+    obj = object_new_with_propv(typename, parent, id, vargs, errp);
     va_end(vargs);
 
     return obj;
@@ -750,8 +750,8 @@ Object *object_new_with_props(const char *typename,
 Object *object_new_with_propv(const char *typename,
                               Object *parent,
                               const char *id,
-                              Error **errp,
-                              va_list vargs)
+                              va_list vargs,
+                              Error **errp)
 {
     Object *obj;
     ObjectClass *klass;
@@ -776,7 +776,7 @@ Object *object_new_with_propv(const char *typename,
     }
     obj = object_new_with_type(klass->type);
 
-    if (!object_set_propv(obj, errp, vargs)) {
+    if (!object_set_propv(obj, vargs, errp)) {
         goto error;
     }
 
@@ -804,14 +804,14 @@ Object *object_new_with_propv(const char *typename,
 
 
 bool object_set_props(Object *obj,
-                     Error **errp,
-                     ...)
+                      Error **errp,
+                      ...)
 {
     va_list vargs;
     bool ret;
 
     va_start(vargs, errp);
-    ret = object_set_propv(obj, errp, vargs);
+    ret = object_set_propv(obj, vargs, errp);
     va_end(vargs);
 
     return ret;
@@ -819,8 +819,8 @@ bool object_set_props(Object *obj,
 
 
 bool object_set_propv(Object *obj,
-                     Error **errp,
-                     va_list vargs)
+                      va_list vargs,
+                      Error **errp)
 {
     const char *propname;
 
tests/unit/check-qom-proplist.c +2 -2
index ee3c6fb32b..7f31735459 100644
@@ -373,8 +373,8 @@ static Object *new_helper(Error **errp,
     obj = object_new_with_propv(TYPE_DUMMY,
                                 parent,
                                 "dummy0",
-                                errp,
-                                vargs);
+                                vargs,
+                                errp);
     va_end(vargs);
     return obj;
 }
Signature · 1 line
2.54.0
#29 lore raw

[PULL 19/29] qom: add trace events for object/property lifecycle

In reply to Daniel P. Berrangé

This adds tracing around object allocation & finalization, the addition & deletion of properties, and the addition & deletion of children.

---
 qom/object.c     | 34 +++++++++++++++++++++++++++-------
 qom/trace-events | 12 ++++++++++--
 2 files changed, 37 insertions(+), 9 deletions(-)
qom/object.c +27 -7
index dfdc5c8e42..4a15151c3b 100644
@@ -594,6 +594,8 @@ static void object_property_del_all(Object *obj)
         object_property_iter_init(&iter, obj);
         while ((prop = object_property_iter_next(&iter)) != NULL) {
             if (g_hash_table_add(done, prop)) {
+                trace_object_property_del(obj, obj->class->type->name,
+                                          prop->name, prop->opaque);
                 if (prop->release) {
                     prop->release(obj, prop->name, prop->opaque);
                     released = true;
@@ -612,10 +614,14 @@ static void object_property_del_child(Object *obj, Object *child)
     GHashTableIter iter;
     gpointer key, value;
 
+    trace_object_property_del_child(obj, obj->class->type->name,
+                                    child, child->class->type->name);
     g_hash_table_iter_init(&iter, obj->properties);
     while (g_hash_table_iter_next(&iter, &key, &value)) {
         prop = value;
         if (object_property_is_child(prop) && prop->opaque == child) {
+            trace_object_property_del(obj, obj->class->type->name,
+                                      prop->name, prop->opaque);
             if (prop->release) {
                 prop->release(obj, prop->name, prop->opaque);
                 prop->release = NULL;
@@ -655,7 +661,7 @@ static void object_finalize(void *data)
 {
     Object *obj = data;
     TypeImpl *ti = obj->class->type;
-
+    trace_object_finalize(obj, obj->class->type->name);
     object_property_del_all(obj);
     object_deinit(obj, ti);
 
@@ -705,6 +711,7 @@ static Object *object_new_with_type(Type type)
     object_initialize_with_type(obj, size, type);
     obj->free = obj_free;
 
+    trace_object_new(obj, obj->class->type->name);
     return obj;
 }
 
@@ -835,8 +842,9 @@ Object *object_dynamic_cast(Object *obj, const char *typename)
 Object *object_dynamic_cast_assert(Object *obj, const char *typename,
                                    const char *file, int line, const char *func)
 {
-    trace_object_dynamic_cast_assert(obj ? obj->class->type->name : "(null)",
-                                     typename, file, line, func);
+    trace_object_dynamic_cast_assert(
+        obj, obj ? obj->class->type->name : "(null)",
+        typename, file, line, func);
 
 #ifdef CONFIG_QOM_CAST_DEBUG
     int i;
@@ -926,8 +934,9 @@ ObjectClass *object_class_dynamic_cast_assert(ObjectClass *class,
 {
     ObjectClass *ret;
 
-    trace_object_class_dynamic_cast_assert(class ? class->type->name : "(null)",
-                                           typename, file, line, func);
+    trace_object_class_dynamic_cast_assert(
+        class ? class->type->name : "(null)",
+        typename, file, line, func);
 
 #ifdef CONFIG_QOM_CAST_DEBUG
     int i;
@@ -1211,6 +1220,8 @@ object_property_try_add(Object *obj, const char *name, const char *type,
     prop->release = release;
     prop->opaque = opaque;
 
+    trace_object_property_add(obj, obj->class->type->name,
+                              prop->name, prop->opaque);
     g_hash_table_insert(obj->properties, prop->name, prop);
     return prop;
 }
@@ -1249,6 +1260,8 @@ object_class_property_add(ObjectClass *klass,
     prop->release = release;
     prop->opaque = opaque;
 
+    trace_object_class_property_add(klass->type->name, prop->name,
+                                    prop->opaque);
     g_hash_table_insert(klass->properties, prop->name, prop);
 
     return prop;
@@ -1337,6 +1350,8 @@ void object_property_del(Object *obj, const char *name)
 {
     ObjectProperty *prop = g_hash_table_lookup(obj->properties, name);
 
+    trace_object_property_del(obj, obj->class->type->name, prop->name,
+                              prop->opaque);
     if (prop->release) {
         prop->release(obj, name, prop->opaque);
     }
@@ -1625,8 +1640,11 @@ int object_property_get_enum(Object *obj, const char *name,
 bool object_property_parse(Object *obj, const char *name,
                            const char *string, Error **errp)
 {
-    Visitor *v = string_input_visitor_new(string);
-    bool ok = object_property_set(obj, name, v, errp);
+    Visitor *v;
+    bool ok;
+    trace_object_property_parse(obj, obj->class->type->name, name, string);
+    v = string_input_visitor_new(string);
+    ok = object_property_set(obj, name, v, errp);
 
     visit_free(v);
     return ok;
@@ -1757,6 +1775,8 @@ object_property_try_add_child(Object *obj, const char *name,
     g_autofree char *type = NULL;
     ObjectProperty *op;
 
+    trace_object_property_add_child(obj, obj->class->type->name, name,
+                                    child, child->class->type->name);
     assert(!child->parent);
 
     type = g_strdup_printf("child<%s>", object_get_typename(child));
qom/trace-events +10 -2
index b2e9f4a712..44c63e72af 100644
@@ -1,5 +1,13 @@
 # See docs/devel/tracing.rst for syntax documentation.
 
 # object.c
-object_dynamic_cast_assert(const char *type, const char *target, const char *file, int line, const char *func) "%s->%s (%s:%d:%s)"
-object_class_dynamic_cast_assert(const char *type, const char *target, const char *file, int line, const char *func) "%s->%s (%s:%d:%s)"
+object_dynamic_cast_assert(void *obj, const char *type, const char *target, const char *file, int line, const char *func) "obj=%p type=%s->%s (%s:%d:%s)"
+object_finalize(void *obj, const char *type) "obj=%p type=%s"
+object_new(void *obj, const char *type) "obj=%p type=%s"
+object_property_add(void *obj, const char *type, const char *name, void *value) "obj=%p type=%s name=%s value=%p"
+object_property_add_child(void *obj, const char *type, const char *name, void *child, const char *childtype) "obj=%p type=%s name=%s child=%p child-type=%s"
+object_property_del(void *obj, const char *type, const char *name, void *value) "obj=%p type=%s name=%s value=%p"
+object_property_del_child(void *obj, const char *type, void *child, const char *childtype) "obj=%p type=%s child=%p child-type=%s"
+object_property_parse(void *obj, const char *type, const char *name, const char *value) "obj=%p type=%s prop=%s value=%s"
+object_class_dynamic_cast_assert(const char *type, const char *target, const char *file, int line, const char *func) "type=%s->%s (%s:%d:%s)"
+object_class_property_add(const char *type, const char *name, void *value) "type=%s name=%s value=%p"
Signature · 1 line
2.54.0