Skip to contentSkip to conversation

Deprecate implicit filters

PATCH In review Vladimir Sementsov-Ogievskiy 4 participants 10 messages Nov 8, 2019

lore
Share & tools
lore
#0 lore raw

v2 cover Deprecate implicit filters

v2:
Don't deprecate drive-backup, it is unrelated thing and will be resent in separate.
Don't deprecate drive-mirror. Instead add filter-node-name to drive-mirror to behave like blockdev-mirror
Fix all broken iotests.

Vladimir Sementsov-Ogievskiy (2):
qapi: add filter-node-name option to drive-mirror
qapi: deprecate implicit filters

 qemu-deprecated.texi       |  6 ++++++
 qapi/block-core.json       | 14 ++++++++++++--
 include/block/block_int.h  | 10 +++++++++-
 blockdev.c                 | 12 +++++++++++-
 tests/qemu-iotests/094     |  1 +
 tests/qemu-iotests/095     |  6 ++++--
 tests/qemu-iotests/109     |  1 +
 tests/qemu-iotests/127     |  1 +
 tests/qemu-iotests/141     |  5 ++++-
 tests/qemu-iotests/144     |  3 ++-
 tests/qemu-iotests/156     |  1 +
 tests/qemu-iotests/161     |  7 +++++++
 tests/qemu-iotests/161.out |  1 +
 tests/qemu-iotests/185     |  3 +++
 tests/qemu-iotests/191     |  2 ++
 tests/qemu-iotests/229     |  1 +
 tests/qemu-iotests/247     |  8 +++++---
 tests/qemu-iotests/249     |  5 +++--
 tests/qemu-iotests/249.out |  2 +-
 19 files changed, 75 insertions(+), 14 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH v2 1/2 qapi: add filter-node-name option to drive-mirror

To correspond to blockdev-mirror command and make it possible to deprecate implicit filters in the next commit.

---
 qapi/block-core.json | 7 +++++++
 blockdev.c           | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)
qapi/block-core.json +7 -0
index aa97ee2641..93530d3a13 100644
@@ -1992,6 +1992,12 @@
 # @on-target-error: the action to take on an error on the target,
 #                   default 'report' (no limitations, since this applies to
 #                   a different block device than @device).
+#
+# @filter-node-name: the node name that should be assigned to the
+#                    filter driver that the mirror job inserts into the graph
+#                    above @device. If this option is not given, a node name is
+#                    autogenerated. (Since: 4.2)
+#
 # @unmap: Whether to try to unmap target sectors where source has
 #         only zero. If true, and target unallocated sectors will read as zero,
 #         target image sectors will be unmapped; otherwise, zeroes will be
@@ -2022,6 +2028,7 @@
             '*speed': 'int', '*granularity': 'uint32',
             '*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
             '*on-target-error': 'BlockdevOnError',
+            '*filter-node-name': 'str',
             '*unmap': 'bool', '*copy-mode': 'MirrorCopyMode',
             '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
 
blockdev.c +1 -1
index 8e029e9c01..2ca614c77f 100644
@@ -4008,7 +4008,7 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
                            arg->has_on_source_error, arg->on_source_error,
                            arg->has_on_target_error, arg->on_target_error,
                            arg->has_unmap, arg->unmap,
-                           false, NULL,
+                           arg->has_filter_node_name, arg->filter_node_name,
                            arg->has_copy_mode, arg->copy_mode,
                            arg->has_auto_finalize, arg->auto_finalize,
                            arg->has_auto_dismiss, arg->auto_dismiss,
Signature · 1 line
2.21.0
#2 lore raw

Re: [PATCH v2 1/2] qapi: add filter-node-name option to drive-mirror

Quoted 14 lines · Vladimir Sementsov-Ogievskiy

To correspond to blockdev-mirror command and make it possible to deprecate implicit filters in the next commit.

---
 qapi/block-core.json | 7 +++++++
 blockdev.c           | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)
qapi/block-core.json +0 -0
index aa97ee2641..93530d3a13 100644

qapi/block-core.json line 2000

Quoted 9 lines
qapi/block-core.json +6 -0
@@ -1992,6 +1992,12 @@
 # @on-target-error: the action to take on an error on the target,
 #                   default 'report' (no limitations, since this applies to
 #                   a different block device than @device).
+#
+# @filter-node-name: the node name that should be assigned to the
+#                    filter driver that the mirror job inserts into the graph
+#                    above @device. If this option is not given, a node name is
+#                    autogenerated. (Since: 4.2)
+#

Speaking for libvirt and based on what I've responded to patch 2 there's no value in adding this parameter for libvirt's use.

This also kind of means that drive-mirror can be deprecated together with deprecating -drive.

#3 lore raw

PATCH v2 2/2 qapi: deprecate implicit filters

In reply to Vladimir Sementsov-Ogievskiy

To get rid of implicit filters related workarounds in future let's deprecate them now.

Deprecation warning breaks some bash iotests output, so fix it here
too: in most of cases just add filter-node-name in test.

In 161 add FIXME and deprecation warning into 161.out.

In 249, the test case is changed, as we don't need to test "default" filter node name anymore.

---
 qemu-deprecated.texi       |  6 ++++++
 qapi/block-core.json       |  9 ++++++---
 include/block/block_int.h  | 10 +++++++++-
 blockdev.c                 | 10 ++++++++++
 tests/qemu-iotests/094     |  1 +
 tests/qemu-iotests/095     |  6 ++++--
 tests/qemu-iotests/109     |  1 +
 tests/qemu-iotests/127     |  1 +
 tests/qemu-iotests/141     |  5 ++++-
 tests/qemu-iotests/144     |  3 ++-
 tests/qemu-iotests/156     |  1 +
 tests/qemu-iotests/161     |  7 +++++++
 tests/qemu-iotests/161.out |  1 +
 tests/qemu-iotests/185     |  3 +++
 tests/qemu-iotests/191     |  2 ++
 tests/qemu-iotests/229     |  1 +
 tests/qemu-iotests/247     |  8 +++++---
 tests/qemu-iotests/249     |  5 +++--
 tests/qemu-iotests/249.out |  2 +-
 19 files changed, 68 insertions(+), 14 deletions(-)
qemu-deprecated.texi +6 -0
index 296bfc93a3..c969faf55a 100644
@@ -204,6 +204,12 @@ and accurate ``query-qmp-schema'' command.
 Character devices creating sockets in client mode should not specify
 the 'wait' field, which is only applicable to sockets in server mode
 
+@subsection implicit filters in mirror and commit (since 4.2)
+
+Mirror and commit jobs insert filters, which becomes implicit if user
+omitted @option(filter-node-name) parameter. So omitting it is deprecated
+in ``blockdev-mirror'', ``drive-mirror'' and ``block-commit'', set it always.
+
 @section Human Monitor Protocol (HMP) commands
 
 @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove' (since 3.1)
qapi/block-core.json +6 -3
index 93530d3a13..37caed775f 100644
@@ -1659,7 +1659,8 @@
 # @filter-node-name: the node name that should be assigned to the
 #                    filter driver that the commit job inserts into the graph
 #                    above @top. If this option is not given, a node name is
-#                    autogenerated. (Since: 2.9)
+#                    autogenerated. Omitting this option is deprecated, it will
+#                    be required in future. (Since: 2.9)
 #
 # @auto-finalize: When false, this job will wait in a PENDING state after it has
 #                 finished its work, waiting for @block-job-finalize before
@@ -1996,7 +1997,8 @@
 # @filter-node-name: the node name that should be assigned to the
 #                    filter driver that the mirror job inserts into the graph
 #                    above @device. If this option is not given, a node name is
-#                    autogenerated. (Since: 4.2)
+#                    autogenerated. Omitting this option is deprecated, it will
+#                    be required in future. (Since: 4.2)
 #
 # @unmap: Whether to try to unmap target sectors where source has
 #         only zero. If true, and target unallocated sectors will read as zero,
@@ -2311,7 +2313,8 @@
 # @filter-node-name: the node name that should be assigned to the
 #                    filter driver that the mirror job inserts into the graph
 #                    above @device. If this option is not given, a node name is
-#                    autogenerated. (Since: 2.9)
+#                    autogenerated. Omitting this option is deprecated, it will
+#                    be required in future. (Since: 2.9)
 #
 # @copy-mode: when to copy data to the destination; defaults to 'background'
 #             (Since: 3.0)
include/block/block_int.h +9 -1
index dd033d0b37..48ff3af48d 100644
@@ -793,7 +793,15 @@ struct BlockDriverState {
     bool sg;        /* if true, the device is a /dev/sg* */
     bool probed;    /* if true, format was probed rather than specified */
     bool force_share; /* if true, always allow all shared permissions */
-    bool implicit;  /* if true, this filter node was automatically inserted */
+
+    /*
+     * @implicit field is deprecated, don't set it to true for new filters.
+     * If true, this filter node was automatically inserted and user don't
+     * know about it and unprepared for any effects of it. So, implicit
+     * filters are workarounded and skipped in many places of the block
+     * layer code.
+     */
+    bool implicit;
 
     BlockDriver *drv; /* NULL means no media */
     void *opaque;
blockdev.c +10 -0
index 2ca614c77f..8c3a409c94 100644
@@ -3333,6 +3333,11 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device,
     BlockdevOnError on_error = BLOCKDEV_ON_ERROR_REPORT;
     int job_flags = JOB_DEFAULT;
 
+    if (!has_filter_node_name) {
+        warn_report("Omitting filter-node-name parameter is deprecated, it "
+                    "will be required in future");
+    }
+
     if (!has_speed) {
         speed = 0;
     }
@@ -3780,6 +3785,11 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
 {
     int job_flags = JOB_DEFAULT;
 
+    if (!has_filter_node_name) {
+        warn_report("Omitting filter-node-name parameter is deprecated, it "
+                    "will be required in future");
+    }
+
     if (!has_speed) {
         speed = 0;
     }
tests/qemu-iotests/094 +1 -0
index 9343e09492..07017d5821 100755
@@ -61,6 +61,7 @@ _send_qemu_cmd $QEMU_HANDLE \
 _send_qemu_cmd $QEMU_HANDLE  \
     "{'execute': 'drive-mirror',
       'arguments': {'device': 'src',
+                    'filter-node-name': 'filter0',
                     'target': '$TEST_IMG',
                     'format': 'nbd',
                     'sync':'full',
tests/qemu-iotests/095 +4 -2
index 58fe174b5e..3dc206ba9d 100755
@@ -69,8 +69,10 @@ h=$QEMU_HANDLE
 _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" "return"
 
 _send_qemu_cmd $h "{ 'execute': 'block-commit',
-                                 'arguments': { 'device': 'test',
-                                 'top': '"${TEST_IMG}.snp1"' } }" '"status": "null"'
+                     'arguments': { 'device': 'test',
+                                    'filter-node-name': 'filter0',
+                                    'top': '"${TEST_IMG}.snp1"' } }" \
+                  '"status": "null"'
 
 _cleanup_qemu
 
tests/qemu-iotests/109 +1 -0
index 9897ceb6cd..aa7da57f8c 100755
@@ -57,6 +57,7 @@ run_qemu()
 
     _send_qemu_cmd $QEMU_HANDLE \
         "{'execute':'drive-mirror', 'arguments':{
+            'filter-node-name': 'filter0',
             'device': 'src', 'target': '$raw_img', $qmp_format
             'mode': 'existing', 'sync': 'full'}}" \
         "return"
tests/qemu-iotests/127 +1 -0
index b64926ab31..b2ba9c66cb 100755
@@ -70,6 +70,7 @@ _send_qemu_cmd $QEMU_HANDLE \
     "{ 'execute': 'drive-mirror',
        'arguments': {
            'job-id': 'mirror',
+           'filter-node-name': 'filter0',
            'device': 'source',
            'target': '$TEST_IMG.overlay1',
            'mode':   'existing',
tests/qemu-iotests/141 +4 -1
index 8c2ae79f2b..178cee460d 100755
@@ -123,6 +123,7 @@ echo
 test_blockjob \
     "{'execute': 'drive-mirror',
       'arguments': {'job-id': 'job0',
+                    'filter-node-name': 'filter0',
                     'device': 'drv0',
                     'target': '$TEST_DIR/o.$IMGFMT',
                     'format': '$IMGFMT',
@@ -140,7 +141,8 @@ echo
 
 test_blockjob \
     "{'execute': 'block-commit',
-      'arguments': {'job-id': 'job0', 'device': 'drv0'}}" \
+      'arguments': {'job-id': 'job0', 'filter-node-name': 'filter0',
+                    'device': 'drv0'}}" \
     'BLOCK_JOB_READY' \
     '"status": "null"'
 
@@ -157,6 +159,7 @@ $QEMU_IO -c 'write 0 1M' "$TEST_DIR/m.$IMGFMT" | _filter_qemu_io
 test_blockjob \
     "{'execute': 'block-commit',
       'arguments': {'job-id': 'job0',
+                    'filter-node-name': 'filter0',
                     'device': 'drv0',
                     'top':    '$TEST_DIR/m.$IMGFMT',
                     'speed':  1}}" \
tests/qemu-iotests/144 +2 -1
index 011ed4f2bc..de60b0ccba 100755
@@ -83,7 +83,8 @@ echo
 # Block commit on active layer, push the new overlay into base
 _send_qemu_cmd $h "{ 'execute': 'block-commit',
                                 'arguments': {
-                                                 'device': 'virtio0'
+                                                 'device': 'virtio0',
+                                                 'filter-node-name': 'filter0'
                                               }
                     }" "READY"
 
tests/qemu-iotests/156 +1 -0
index 2ffa3ca942..2dc50444db 100755
@@ -87,6 +87,7 @@ TEST_IMG="$TEST_IMG.target.overlay" _make_test_img -u -b "$TEST_IMG.target" 1M
 _send_qemu_cmd $QEMU_HANDLE \
     "{ 'execute': 'drive-mirror',
        'arguments': { 'device': 'source',
+                      'filter-node-name': 'filter0',
                       'target': '$TEST_IMG.target.overlay',
                       'mode': 'existing',
                       'sync': 'top' } }" \
tests/qemu-iotests/161 +7 -0
index 456a4bd8c4..9cf3e96c74 100755
@@ -120,6 +120,13 @@ _send_qemu_cmd $QEMU_HANDLE \
                       'top': '${TEST_IMG}.int' } }" \
     'return'
 
+# FIXME
+# Need to use filter-node-name in previous command, as it's omitting is
+# deprecated, as described by warning in test output. But if just put
+# filter-node-name into previous command the following reopen is failed with
+# messasge "Cannot change the option 'backing.detect-zeroes". It's a bug and
+# shoud be fixed.
+
 # Wait for block-commit to finish
 sleep 0.5
 
tests/qemu-iotests/161.out +1 -0
index 39951993ee..497f98f3c3 100644
@@ -27,6 +27,7 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=1048576
 Formatting 'TEST_DIR/t.IMGFMT.int', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.int
 {"return": {}}
+warning: Omitting filter-node-name parameter is deprecated, it will be required in future
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "none0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "none0"}}
 {"return": {}}
tests/qemu-iotests/185 +3 -0
index 454ff600cc..65ef64f0c0 100755
@@ -107,6 +107,7 @@ echo
 _send_qemu_cmd $h \
     "{ 'execute': 'block-commit',
        'arguments': { 'device': 'disk',
+                      'filter-node-name': 'filter0',
                       'base':'$TEST_IMG.base',
                       'top': '$TEST_IMG.mid',
                       'speed': 65536 } }" \
@@ -132,6 +133,7 @@ _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 _send_qemu_cmd $h \
     "{ 'execute': 'block-commit',
        'arguments': { 'device': 'disk',
+                      'filter-node-name': 'filter0',
                       'base':'$TEST_IMG.base',
                       'speed': 65536 } }" \
     "return"
@@ -154,6 +156,7 @@ _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" 'return'
 _send_qemu_cmd $h \
     "{ 'execute': 'drive-mirror',
        'arguments': { 'device': 'disk',
+                      'filter-node-name': 'filter0',
                       'target': '$TEST_IMG.copy',
                       'format': '$IMGFMT',
                       'sync': 'full',
tests/qemu-iotests/191 +2 -0
index 528022e8d8..79d572f6d5 100755
@@ -76,6 +76,7 @@ echo
 _send_qemu_cmd $h \
     "{ 'execute': 'block-commit',
        'arguments': { 'job-id': 'commit0',
+                      'filter-node-name': 'filter0',
                       'device': 'top',
                       'base':'$TEST_IMG.base',
                       'top': '$TEST_IMG.mid' } }" \
@@ -124,6 +125,7 @@ echo
 _send_qemu_cmd $h \
     "{ 'execute': 'block-commit',
        'arguments': { 'job-id': 'commit0',
+                      'filter-node-name': 'filter0',
                       'device': 'top',
                       'base':'$TEST_IMG.base',
                       'top': '$TEST_IMG.mid' } }" \
tests/qemu-iotests/229 +1 -0
index e18a464fe0..08a7887e90 100755
@@ -69,6 +69,7 @@ echo
 _send_qemu_cmd $QEMU_HANDLE \
     "{'execute': 'drive-mirror',
                  'arguments': {'device': 'testdisk',
+                               'filter-node-name': 'filter0',
                                'format': '$IMGFMT',
                                'target': '$DEST_IMG',
                                'sync':   'full',
tests/qemu-iotests/247 +5 -3
index c853b73819..394f4e221c 100755
@@ -53,9 +53,11 @@ TEST_IMG="$TEST_IMG.3" _make_test_img $size
 TEST_IMG="$TEST_IMG.4" _make_test_img $size
 
 (cat <<EOF
-{"execute":"qmp_capabilities"}
-{"execute":"block-commit",
- "arguments":{"device":"format-4", "top-node": "format-2", "base-node":"format-0", "job-id":"job0"}}
+{"execute": "qmp_capabilities"}
+{"execute": "block-commit",
+ "arguments": {"device": "format-4", "filter-node-name": "filter0",
+               "top-node": "format-2", "base-node":"format-0",
+               "job-id":"job0"}}
 EOF
 if [ "${VALGRIND_QEMU}" == "y" ]; then
     sleep 10
tests/qemu-iotests/249 +3 -2
index e4650ecf6b..be84b9f961 100755
@@ -88,11 +88,12 @@ _send_qemu_cmd $QEMU_HANDLE \
     'return'
 
 echo
-echo '=== Run block-commit on base using the default filter node name'
+echo '=== Run block-commit on base using the valid filter node name'
 echo
 _send_qemu_cmd $QEMU_HANDLE \
     "{ 'execute': 'block-commit',
-       'arguments': {'job-id': 'job0', 'device': 'none1', 'top-node': 'int'}}" \
+       'arguments': {'job-id': 'job0', 'filter-node-name': 'filter0',
+                     'device': 'none1', 'top-node': 'int'}}" \
     'return'
 
 # Wait for block-commit to finish
tests/qemu-iotests/249.out +1 -1
index 1c93164e84..89092b75d7 100644
@@ -18,7 +18,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.
 
 {"return": "Block node is read-onlyrn"}
 
-=== Run block-commit on base using the default filter node name
+=== Run block-commit on base using the valid filter node name
 
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
Signature · 1 line
2.21.0
#4 lore raw

Re: [PATCH v2 2/2] qapi: deprecate implicit filters

Quoted 56 lines · Vladimir Sementsov-Ogievskiy

To get rid of implicit filters related workarounds in future let's deprecate them now.

Deprecation warning breaks some bash iotests output, so fix it here
too: in most of cases just add filter-node-name in test.

In 161 add FIXME and deprecation warning into 161.out.

In 249, the test case is changed, as we don't need to test "default" filter node name anymore.

---
 qemu-deprecated.texi       |  6 ++++++
 qapi/block-core.json       |  9 ++++++---
 include/block/block_int.h  | 10 +++++++++-
 blockdev.c                 | 10 ++++++++++
 tests/qemu-iotests/094     |  1 +
 tests/qemu-iotests/095     |  6 ++++--
 tests/qemu-iotests/109     |  1 +
 tests/qemu-iotests/127     |  1 +
 tests/qemu-iotests/141     |  5 ++++-
 tests/qemu-iotests/144     |  3 ++-
 tests/qemu-iotests/156     |  1 +
 tests/qemu-iotests/161     |  7 +++++++
 tests/qemu-iotests/161.out |  1 +
 tests/qemu-iotests/185     |  3 +++
 tests/qemu-iotests/191     |  2 ++
 tests/qemu-iotests/229     |  1 +
 tests/qemu-iotests/247     |  8 +++++---
 tests/qemu-iotests/249     |  5 +++--
 tests/qemu-iotests/249.out |  2 +-
 19 files changed, 68 insertions(+), 14 deletions(-)
qemu-deprecated.texi +6 -0
index 296bfc93a3..c969faf55a 100644
@@ -204,6 +204,12 @@ and accurate ``query-qmp-schema'' command.
 Character devices creating sockets in client mode should not specify
 the 'wait' field, which is only applicable to sockets in server mode
 
+@subsection implicit filters in mirror and commit (since 4.2)
+
+Mirror and commit jobs insert filters, which becomes implicit if user
+omitted @option(filter-node-name) parameter. So omitting it is deprecated
+in ``blockdev-mirror'', ``drive-mirror'' and ``block-commit'', set it always.
+
 @section Human Monitor Protocol (HMP) commands
 
 @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove' (since 3.1)
qapi/block-core.json +0 -0
index 93530d3a13..37caed775f 100644

qapi/block-core.json line 1666

Quoted 9 lines
qapi/block-core.json +2 -1
@@ -1659,7 +1659,8 @@
 # @filter-node-name: the node name that should be assigned to the
 #                    filter driver that the commit job inserts into the graph
 #                    above @top. If this option is not given, a node name is
-#                    autogenerated. (Since: 2.9)
+#                    autogenerated. Omitting this option is deprecated, it will
+#                    be required in future. (Since: 2.9)
 #
 # @auto-finalize: When false, this job will wait in a PENDING state after it has
 #                 finished its work, waiting for @block-job-finalize before

Note that 'block-commit' and 'drive-mirror' commands are used by libvirt in the pre-blockdev era. In those instances we gather statistics of block devices by nesting in the output of query-blockstats and query-block rather than selecting the appropriate info by any other means (e.g. by node name).

This means that the output MUST stay consistend when block jobs are used and the hack this patch is deprcating will break those.

Note that in libvirt we don't plan to invest time to add workarounds for non-blockdev cases since blockdev by itself is complex enough and I'd strongly prefer not having a third code path to care about.

Given that -blockdev can't be used in all cases (e.g. for sd-cards) which also blocks deprecation of -drive I don't think that hiding of implicit filter nodes can be deprecated until -drive is deprecated.

#5 lore raw

Re: [PATCH v2 2/2] qapi: deprecate implicit filters

Quoted 76 lines · Peter Krempa
Quoted 56 lines · Vladimir Sementsov-Ogievskiy

To get rid of implicit filters related workarounds in future let's deprecate them now.

Deprecation warning breaks some bash iotests output, so fix it here
too: in most of cases just add filter-node-name in test.

In 161 add FIXME and deprecation warning into 161.out.

In 249, the test case is changed, as we don't need to test "default" filter node name anymore.

---
  qemu-deprecated.texi       |  6 ++++++
  qapi/block-core.json       |  9 ++++++---
  include/block/block_int.h  | 10 +++++++++-
  blockdev.c                 | 10 ++++++++++
  tests/qemu-iotests/094     |  1 +
  tests/qemu-iotests/095     |  6 ++++--
  tests/qemu-iotests/109     |  1 +
  tests/qemu-iotests/127     |  1 +
  tests/qemu-iotests/141     |  5 ++++-
  tests/qemu-iotests/144     |  3 ++-
  tests/qemu-iotests/156     |  1 +
  tests/qemu-iotests/161     |  7 +++++++
  tests/qemu-iotests/161.out |  1 +
  tests/qemu-iotests/185     |  3 +++
  tests/qemu-iotests/191     |  2 ++
  tests/qemu-iotests/229     |  1 +
  tests/qemu-iotests/247     |  8 +++++---
  tests/qemu-iotests/249     |  5 +++--
  tests/qemu-iotests/249.out |  2 +-
  19 files changed, 68 insertions(+), 14 deletions(-)
qemu-deprecated.texi +6 -0
index 296bfc93a3..c969faf55a 100644
@@ -204,6 +204,12 @@ and accurate ``query-qmp-schema'' command.
  Character devices creating sockets in client mode should not specify
  the 'wait' field, which is only applicable to sockets in server mode
 
+@subsection implicit filters in mirror and commit (since 4.2)
+
+Mirror and commit jobs insert filters, which becomes implicit if user
+omitted @option(filter-node-name) parameter. So omitting it is deprecated
+in ``blockdev-mirror'', ``drive-mirror'' and ``block-commit'', set it always.
+
  @section Human Monitor Protocol (HMP) commands
 
  @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove' (since 3.1)
qapi/block-core.json +0 -0
index 93530d3a13..37caed775f 100644

qapi/block-core.json line 1666

Quoted 9 lines
qapi/block-core.json +2 -1
@@ -1659,7 +1659,8 @@
  # @filter-node-name: the node name that should be assigned to the
  #                    filter driver that the commit job inserts into the graph
  #                    above @top. If this option is not given, a node name is
-#                    autogenerated. (Since: 2.9)
+#                    autogenerated. Omitting this option is deprecated, it will
+#                    be required in future. (Since: 2.9)
  #
  # @auto-finalize: When false, this job will wait in a PENDING state after it has
  #                 finished its work, waiting for @block-job-finalize before

Note that 'block-commit' and 'drive-mirror' commands are used by libvirt in the pre-blockdev era. In those instances we gather statistics of block devices by nesting in the output of query-blockstats and query-block rather than selecting the appropriate info by any other means (e.g. by node name).

This means that the output MUST stay consistend when block jobs are used and the hack this patch is deprcating will break those.

Quoted 8 lines

Note that in libvirt we don't plan to invest time to add workarounds for non-blockdev cases since blockdev by itself is complex enough and I'd strongly prefer not having a third code path to care about.

Given that -blockdev can't be used in all cases (e.g. for sd-cards) which also blocks deprecation of -drive I don't think that hiding of implicit filter nodes can be deprecated until -drive is deprecated.

OK, so, we can't deprecate anything around it now.

What is the problem with sd-cards?

Signature · 2 lines
Best regards,
Vladimir
#6 lore raw

Re: [PATCH v2 2/2] qapi: deprecate implicit filters

Quoted 2 lines · Vladimir Sementsov-Ogievskiy
Quoted 1 line · Peter Krempa

On Fri, Nov 08, 2019 at 13:16:55 +0300, Vladimir Sementsov-Ogievskiy wrote:

[...]

Quoted 19 lines
Quoted 9 lines

Note that 'block-commit' and 'drive-mirror' commands are used by libvirt in the pre-blockdev era. In those instances we gather statistics of block devices by nesting in the output of query-blockstats and query-block rather than selecting the appropriate info by any other means (e.g. by node name).

This means that the output MUST stay consistend when block jobs are used and the hack this patch is deprcating will break those.

Quoted 8 lines

Note that in libvirt we don't plan to invest time to add workarounds for non-blockdev cases since blockdev by itself is complex enough and I'd strongly prefer not having a third code path to care about.

Given that -blockdev can't be used in all cases (e.g. for sd-cards) which also blocks deprecation of -drive I don't think that hiding of implicit filter nodes can be deprecated until -drive is deprecated.

Quoted 3 lines

OK, so, we can't deprecate anything around it now.

What is the problem with sd-cards?

So the problem was that it was impossible to instantiate it via -device, but looking at the qemu code base this doesn't seem to be true any more.

I'll have a look whether we can rework the instantiation of sd card frontends in libvirt somehow or whether it actually ever worked. Unfortunately the documentation seems to be rather sparse.

#7 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

SIGN pc-bios/optionrom/pvh.bin
GEN docs/interop/qemu-ga-ref.7
/tmp/qemu-test/src/qemu-deprecated.texi:210: @option expected braces
make: *** [Makefile:994: qemu-doc.txt] Error 1
make: *** Waiting for unfinished jobs.... /tmp/qemu-test/src/qemu-deprecated.texi:210: @option expected braces
make: *** [Makefile:987: qemu-doc.html] Error 1
Traceback (most recent call last):
File "./tests/docker/docker.py", line 662, in <module>

    sys.exit(main())

---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=3d404482279f4fe68fcb2c9971f2480a', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-mlpokwtg/src/docker-src.2019-11-08-06.56.33.23805:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=3d404482279f4fe68fcb2c9971f2480a
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-mlpokwtg/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real 4m11.170s
user 0m4.404s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#8 lore raw

Something strange, I don't think it related to patchset.

Quoted 45 lines · 08.11.2019 15:00, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

SIGN pc-bios/optionrom/pvh.bin
GEN docs/interop/qemu-ga-ref.7
/tmp/qemu-test/src/qemu-deprecated.texi:210: @option expected braces
make: *** [Makefile:994: qemu-doc.txt] Error 1
make: *** Waiting for unfinished jobs.... /tmp/qemu-test/src/qemu-deprecated.texi:210: @option expected braces
make: *** [Makefile:987: qemu-doc.html] Error 1
Traceback (most recent call last):

   File "./tests/docker/docker.py", line 662, in <module>
     sys.exit(main())

---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=3d404482279f4fe68fcb2c9971f2480a', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-mlpokwtg/src/docker-src.2019-11-08-06.56.33.23805:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=3d404482279f4fe68fcb2c9971f2480a
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-mlpokwtg/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real 4m11.170s
user 0m4.404s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Signature · 2 lines
Best regards,
Vladimir
#9 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Quoted 6 lines · Vladimir Sementsov-Ogievskiy

v2:
Don't deprecate drive-backup, it is unrelated thing and will be resent in separate.
Don't deprecate drive-mirror. Instead add filter-node-name to drive-mirror to behave like blockdev-mirror
Fix all broken iotests.

I did a quick overview of these patches (I don't know the area well to do a full review) and it looks fine to me, other than that FIXME you added, which (at least looking at the explanation) I think should be investigated, as it might point to a deeper problem somewhere.

Also *I think* that I would merge these two patches together, but this is only my personal taste.

Best regards,
Maxim Levitsky

Quoted 26 lines

Vladimir Sementsov-Ogievskiy (2):
qapi: add filter-node-name option to drive-mirror
qapi: deprecate implicit filters

 qemu-deprecated.texi       |  6 ++++++
 qapi/block-core.json       | 14 ++++++++++++--
 include/block/block_int.h  | 10 +++++++++-
 blockdev.c                 | 12 +++++++++++-
 tests/qemu-iotests/094     |  1 +
 tests/qemu-iotests/095     |  6 ++++--
 tests/qemu-iotests/109     |  1 +
 tests/qemu-iotests/127     |  1 +
 tests/qemu-iotests/141     |  5 ++++-
 tests/qemu-iotests/144     |  3 ++-
 tests/qemu-iotests/156     |  1 +
 tests/qemu-iotests/161     |  7 +++++++
 tests/qemu-iotests/161.out |  1 +
 tests/qemu-iotests/185     |  3 +++
 tests/qemu-iotests/191     |  2 ++
 tests/qemu-iotests/229     |  1 +
 tests/qemu-iotests/247     |  8 +++++---
 tests/qemu-iotests/249     |  5 +++--
 tests/qemu-iotests/249.out |  2 +-
 19 files changed, 75 insertions(+), 14 deletions(-)