Skip to contentSkip to conversation

block: Fix some filename generation issues

PATCH Superseded Max Reitz 3 participants 50 messages Nov 20, 2017 – Feb 2, 2018 Block layer

v7 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Max Reitz

author
#0 lore raw

v7 cover block: Fix some filename generation issues

I'm sparing myself writing this cover letter again, and I'll just give you a link to the previous version:

http://lists.nongnu.org/archive/html/qemu-block/2017-09/msg01030.html

The only difference is that I dropped patch 16 which added a QAPI @base-directory option for any node that could be used to override the base directory to be used for resolving relative filenames. (Because Berto and Kevin convinced me it's not that useful now -- and the patch is pretty stand-alone, so we can always add it later if needed.)

In turn, I had to add patch 4 because test 191 was broken by this series.

v7: Rebased on block-next, and:
- Patch 4: Added (iotest 191 tries to both override a node's backing file and use filenames for node identification at the same time -- not a very good idea, but with a simple modification we can still get away with it)
- Patch 5: Changes 191's output now
- Patch 10 (was 9): Shorten the code [Berto]
- Old patch 16: Dropped [Berto/Kevin]
- Patch 17: Remove test case for @base-directory
- Patch 18:
- Never use inline definition of the significant runtime options array

    [Kevin]

- Add encrypt.key-secret options for qcow/qcow2
- Add transfer limit options for blkdebug
- s/uid/user/ and s/gid/group/ for NFS
- Add offset/size for raw
- Patch 19: Sprinkle a couple of notes in about how we actually don't want this new function [Kevin, kind of]
- Patch 20: Changes 191's output now (and also, there is a rebase conflict for 110 due to patch 16 being removed and thus patch 17 having been modified)

git-backport-diff against v6:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/25:[----] [--] 'block/mirror: Small absolute-paths simplification' 002/25:[----] [--] 'block: Use children list in bdrv_refresh_filename' 003/25:[----] [--] 'block: Add BDS.backing_overridden' 004/25:[down] 'iotests: Drop explicit base blockdev in 191' 005/25:[0024] [FC] 'block: Respect backing bs in bdrv_refresh_filename' 006/25:[----] [--] 'block: Make path_combine() return the path' 007/25:[----] [-C] 'block: bdrv_get_full_backing_filename_from_...'s ret. val.' 008/25:[----] [--] 'block: bdrv_get_full_backing_filename's ret. val.' 009/25:[----] [--] 'block: Add bdrv_make_absolute_filename()' 010/25:[0014] [FC] 'block: Fix bdrv_find_backing_image()' 011/25:[----] [--] 'block: Add bdrv_dirname()'
012/25:[----] [--] 'blkverify: Make bdrv_dirname() return NULL' 013/25:[----] [--] 'quorum: Make bdrv_dirname() return NULL' 014/25:[----] [--] 'block/nbd: Make bdrv_dirname() return NULL' 015/25:[----] [--] 'block/nfs: Implement bdrv_dirname()' 016/25:[----] [--] 'block: Use bdrv_dirname() for relative filenames' 017/25:[0027] [FC] 'iotests: Add quorum case to test 110' 018/25:[0188] [FC] 'block: Add sgfnt_runtime_opts to BlockDriver' 019/25:[0044] [FC] 'block: Add BlockDriver.bdrv_gather_child_options' 020/25:[0026] [FC] 'block: Generically refresh runtime options' 021/25:[----] [-C] 'block: Purify .bdrv_refresh_filename()' 022/25:[----] [--] 'block: Do not copy exact_filename from format file' 023/25:[----] [-C] 'block: Fix FIXME from "Add BDS.backing_overridden"' 024/25:[----] [--] 'block/curl: Implement bdrv_refresh_filename()' 025/25:[----] [--] 'block/null: Generate filename even with latency-ns'

Max Reitz (25):
block/mirror: Small absolute-paths simplification
block: Use children list in bdrv_refresh_filename
block: Add BDS.backing_overridden
iotests: Drop explicit base blockdev in 191
block: Respect backing bs in bdrv_refresh_filename
block: Make path_combine() return the path
block: bdrv_get_full_backing_filename_from_...'s ret. val.
block: bdrv_get_full_backing_filename's ret. val.
block: Add bdrv_make_absolute_filename()
block: Fix bdrv_find_backing_image()
block: Add bdrv_dirname()
blkverify: Make bdrv_dirname() return NULL
quorum: Make bdrv_dirname() return NULL block/nbd: Make bdrv_dirname() return NULL block/nfs: Implement bdrv_dirname()
block: Use bdrv_dirname() for relative filenames
iotests: Add quorum case to test 110
block: Add sgfnt_runtime_opts to BlockDriver
block: Add BlockDriver.bdrv_gather_child_options
block: Generically refresh runtime options
block: Purify .bdrv_refresh_filename()
block: Do not copy exact_filename from format file
block: Fix FIXME from "Add BDS.backing_overridden" block/curl: Implement bdrv_refresh_filename() block/null: Generate filename even with latency-ns

 include/block/block.h         |  15 +-
 include/block/block_int.h     |  38 +++-
 block.c                       | 507 ++++++++++++++++++++++++++++--------------
 block/blkdebug.c              |  69 +++---
 block/blkverify.c             |  29 +--
 block/commit.c                |   3 +-
 block/crypto.c                |   8 +
 block/curl.c                  |  39 ++++
 block/gluster.c               |  19 ++
 block/iscsi.c                 |  18 ++
 block/mirror.c                |  19 +-
 block/nbd.c                   |  46 ++--
 block/nfs.c                   |  53 ++---
 block/null.c                  |  33 ++-
 block/qapi.c                  |  12 +-
 block/qcow.c                  |   7 +
 block/qcow2.c                 |   7 +
 block/quorum.c                |  69 ++++--
 block/raw-format.c            |  10 +-
 block/rbd.c                   |  14 ++
 block/replication.c           |   8 +
 block/sheepdog.c              |  12 +
 block/ssh.c                   |  12 +
 block/throttle.c              |   7 +
 block/vmdk.c                  |  25 ++-
 block/vpc.c                   |   7 +
 block/vvfat.c                 |  12 +
 block/vxhs.c                  |  11 +
 blockdev.c                    |  16 ++
 tests/qemu-iotests/051.out    |   8 +-
 tests/qemu-iotests/051.pc.out |   8 +-
 tests/qemu-iotests/110        |  29 ++-
 tests/qemu-iotests/110.out    |   9 +-
 tests/qemu-iotests/191        |   3 +-
 tests/qemu-iotests/191.out    |  94 ++++----
 35 files changed, 877 insertions(+), 399 deletions(-)
Signature · 1 line
2.13.6
author
#1 lore raw

PATCH v7 1/25 block/mirror: Small absolute-paths simplification

When invoking drive-mirror in absolute-paths mode, the target's backing BDS is assigned to it in mirror_exit(). The current logic only does so if the target does not have that backing BDS already; but it actually cannot have a backing BDS at all (the BDS is opened with O_NO_BACKING in qmp_drive_mirror()), so just assert that and assign the new backing BDS unconditionally.

---
 block/mirror.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
block/mirror.c +6 -6
index 307b6391a8..f995924032 100644
@@ -523,12 +523,12 @@ static void mirror_exit(BlockJob *job, void *opaque)
                             &error_abort);
     if (s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) {
         BlockDriverState *backing = s->is_none_mode ? src : s->base;
-        if (backing_bs(target_bs) != backing) {
-            bdrv_set_backing_hd(target_bs, backing, &local_err);
-            if (local_err) {
-                error_report_err(local_err);
-                data->ret = -EPERM;
-            }
+
+        assert(!target_bs->backing);
+        bdrv_set_backing_hd(target_bs, backing, &local_err);
+        if (local_err) {
+            error_report_err(local_err);
+            data->ret = -EPERM;
         }
     }
 
Signature · 1 line
2.13.6
author
#2 lore raw

PATCH v7 2/25 block: Use children list in bdrv_refresh_filename

In reply to Max Reitz

bdrv_refresh_filename() should invoke itself recursively on all children, not just on file.

With that change, we can remove the manual invocations in blkverify, quorum, commit, and mirror.

---
 block.c           | 9 +++++----
 block/blkverify.c | 3 ---
 block/commit.c    | 1 -
 block/mirror.c    | 1 -
 block/quorum.c    | 1 -
 5 files changed, 5 insertions(+), 10 deletions(-)
block.c +5 -4
index 6c8ef98dfa..4aebf590d8 100644
@@ -4965,16 +4965,17 @@ static bool append_open_options(QDict *d, BlockDriverState *bs)
 void bdrv_refresh_filename(BlockDriverState *bs)
 {
     BlockDriver *drv = bs->drv;
+    BdrvChild *child;
     QDict *opts;
 
     if (!drv) {
         return;
     }
 
-    /* This BDS's file name will most probably depend on its file's name, so
-     * refresh that first */
-    if (bs->file) {
-        bdrv_refresh_filename(bs->file->bs);
+    /* This BDS's file name may depend on any of its children's file names, so
+     * refresh those first */
+    QLIST_FOREACH(child, &bs->children, next) {
+        bdrv_refresh_filename(child->bs);
     }
 
     if (drv->bdrv_refresh_filename) {
block/blkverify.c +0 -3
index 06369f9eac..b2ed8cd70d 100644
@@ -281,9 +281,6 @@ static void blkverify_refresh_filename(BlockDriverState *bs, QDict *options)
 {
     BDRVBlkverifyState *s = bs->opaque;
 
-    /* bs->file->bs has already been refreshed */
-    bdrv_refresh_filename(s->test_file->bs);
-
     if (bs->file->bs->full_open_options
         && s->test_file->bs->full_open_options)
     {
block/commit.c +0 -1
index 5036eec434..cb529cc2aa 100644
@@ -241,7 +241,6 @@ static int coroutine_fn bdrv_commit_top_preadv(BlockDriverState *bs,
 
 static void bdrv_commit_top_refresh_filename(BlockDriverState *bs, QDict *opts)
 {
-    bdrv_refresh_filename(bs->backing->bs);
     pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
             bs->backing->bs->filename);
 }
block/mirror.c +0 -1
index f995924032..f059981f19 100644
@@ -1061,7 +1061,6 @@ static void bdrv_mirror_top_refresh_filename(BlockDriverState *bs, QDict *opts)
          * bdrv_set_backing_hd */
         return;
     }
-    bdrv_refresh_filename(bs->backing->bs);
     pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
             bs->backing->bs->filename);
 }
block/quorum.c +0 -1
index 272f9a5b77..2f1a628449 100644
@@ -1074,7 +1074,6 @@ static void quorum_refresh_filename(BlockDriverState *bs, QDict *options)
     int i;
 
     for (i = 0; i < s->num_children; i++) {
-        bdrv_refresh_filename(s->children[i]->bs);
         if (!s->children[i]->bs->full_open_options) {
             return;
         }
Signature · 1 line
2.13.6
author
#3 lore raw

PATCH v7 3/25 block: Add BDS.backing_overridden

In reply to Max Reitz

If the backing file is overridden, this most probably does change the guest-visible data of a BDS. Therefore, we will need to consider this in bdrv_refresh_filename().

Adding a new field to the BDS is not nice, but it is very simple and exactly keeps track of whether the backing file has been overridden.

This commit adds a FIXME which will be remedied by a follow-up commit. Until then, the respective piece of code will not result in any behavior that is worse than what we currently have.

---
 include/block/block_int.h |  1 +
 block.c                   | 13 +++++++++++++
 block/mirror.c            |  4 ++++
 blockdev.c                | 16 ++++++++++++++++
 4 files changed, 34 insertions(+)
include/block/block_int.h +1 -0
index a5482775ec..5e9734d8b5 100644
@@ -619,6 +619,7 @@ struct BlockDriverState {
     char backing_file[PATH_MAX]; /* if non zero, the image is a diff of
                                     this file image */
     char backing_format[16]; /* if non-zero and backing_file exists */
+    bool backing_overridden; /* backing file has been specified by the user */
 
     QDict *full_open_options;
     char exact_filename[PATH_MAX];
block.c +13 -0
index 4aebf590d8..eb67dfbcc0 100644
@@ -2233,6 +2233,11 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
     reference = qdict_get_try_str(parent_options, bdref_key);
     if (reference || qdict_haskey(options, "file.filename")) {
         backing_filename[0] = '\0';
+
+        /* FIXME: Should also be set to true if @options contains other runtime
+         *        options which control the data that is read from the backing
+         *        BDS */
+        bs->backing_overridden = true;
     } else if (bs->backing_file[0] == '\0' && qdict_size(options) == 0) {
         QDECREF(options);
         goto free_exit;
@@ -2434,6 +2439,9 @@ static BlockDriverState *bdrv_append_temp_snapshot(BlockDriverState *bs,
         goto out;
     }
 
+    bs_snapshot->backing_overridden = true;
+    bdrv_refresh_filename(bs_snapshot);
+
 out:
     QDECREF(snapshot_options);
     g_free(tmp_filename);
@@ -2564,6 +2572,7 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,
     backing = qdict_get_try_str(options, "backing");
     if (backing && *backing == '\0') {
         flags |= BDRV_O_NO_BACKING;
+        bs->backing_overridden = true;
         qdict_del(options, "backing");
     }
 
@@ -4976,6 +4985,10 @@ void bdrv_refresh_filename(BlockDriverState *bs)
      * refresh those first */
     QLIST_FOREACH(child, &bs->children, next) {
         bdrv_refresh_filename(child->bs);
+
+        if (child->role == &child_backing && child->bs->backing_overridden) {
+            bs->backing_overridden = true;
+        }
     }
 
     if (drv->bdrv_refresh_filename) {
block/mirror.c +4 -0
index f059981f19..d6e487fb2e 100644
@@ -530,6 +530,10 @@ static void mirror_exit(BlockJob *job, void *opaque)
             error_report_err(local_err);
             data->ret = -EPERM;
         }
+
+        /* The target image's file already has been created with the backing
+         * file we just set, so there is no need to set backing_overridden or
+         * call bdrv_refresh_filename(). */
     }
 
     if (s->to_replace) {
blockdev.c +16 -0
index 56a6b24a0b..b0393e1786 100644
@@ -1781,6 +1781,8 @@ static void external_snapshot_commit(BlkActionState *common)
 {
     ExternalSnapshotState *state =
                              DO_UPCAST(ExternalSnapshotState, common, common);
+    TransactionAction *action = common->action;
+    bool image_was_existing = false;
 
     /* We don't need (or want) to use the transactional
      * bdrv_reopen_multiple() across all the entries at once, because we
@@ -1789,6 +1791,20 @@ static void external_snapshot_commit(BlkActionState *common)
         bdrv_reopen(state->old_bs, state->old_bs->open_flags & ~BDRV_O_RDWR,
                     NULL);
     }
+
+    if (action->type == TRANSACTION_ACTION_KIND_BLOCKDEV_SNAPSHOT_SYNC) {
+        BlockdevSnapshotSync *s = action->u.blockdev_snapshot_sync.data;
+        if (s->has_mode && s->mode == NEW_IMAGE_MODE_EXISTING) {
+            image_was_existing = true;
+        }
+    } else {
+        image_was_existing = true;
+    }
+
+    if (image_was_existing) {
+        state->new_bs->backing_overridden = true;
+        bdrv_refresh_filename(state->new_bs);
+    }
 }
 
 static void external_snapshot_abort(BlkActionState *common)
Signature · 1 line
2.13.6
Reviewed-by on v7 3/25
#4 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 03/25] block: Add BDS.backing_overridden

Quoted 7 lines · Max Reitz

If the backing file is overridden, this most probably does change the guest-visible data of a BDS. Therefore, we will need to consider this in bdrv_refresh_filename().

Adding a new field to the BDS is not nice, but it is very simple and exactly keeps track of whether the backing file has been overridden.

Quoted 6 lines

This commit adds a FIXME which will be remedied by a follow-up commit. Until then, the respective piece of code will not result in any behavior that is worse than what we currently have.

Berto

author
#5 lore raw

PATCH v7 4/25 iotests: Drop explicit base blockdev in 191

In reply to Max Reitz

Overriding the backing image should result in a json:{} pseudo-filename. Then, you can no longer use the commit block job with filename parameters. Therefore, do not explicitly add the base and override the middle image in iotest 191, since we do not need to anyway. This will allow us to continue to use the middle image's filename to identify it.

In the long run, we want block-commit to accept node names for base and top (just like block-stream does).

---
 tests/qemu-iotests/191     |  3 +-
 tests/qemu-iotests/191.out | 70 +++++++++++++++++++++++-----------------------
 2 files changed, 36 insertions(+), 37 deletions(-)
tests/qemu-iotests/191 +1 -2
index ad785e10b1..92805e9ca2 100755
@@ -67,8 +67,7 @@ qemu_comm_method="qmp"
 qmp_pretty="y"
 
 _launch_qemu \
-    -blockdev "driver=${IMGFMT},file.driver=file,file.filename=${TEST_IMG}.base,node-name=base" \
-    -blockdev "driver=${IMGFMT},file.driver=file,file.filename=${TEST_IMG}.mid,node-name=mid,backing=base" \
+    -blockdev "driver=${IMGFMT},file.driver=file,file.filename=${TEST_IMG}.mid,node-name=mid" \
     -blockdev "driver=${IMGFMT},file.driver=file,file.filename=${TEST_IMG},node-name=top,backing=mid" \
     -blockdev "driver=${IMGFMT},file.driver=file,file.filename=${TEST_IMG}.ovl2,node-name=top2,backing=mid"
 h=$QEMU_HANDLE
tests/qemu-iotests/191.out +35 -35

152 lines not on this page · raw message

Signature · 1 line
2.13.6
Reviewed-by on v7 4/25
#6 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 04/25] iotests: Drop explicit base blockdev in 191

Quoted 10 lines · Max Reitz

Overriding the backing image should result in a json:{} pseudo-filename. Then, you can no longer use the commit block job with filename parameters. Therefore, do not explicitly add the base and override the middle image in iotest 191, since we do not need to anyway. This will allow us to continue to use the middle image's filename to identify it.

In the long run, we want block-commit to accept node names for base and top (just like block-stream does).

Berto

author
#7 lore raw

PATCH v7 5/25 block: Respect backing bs in bdrv_refresh_filename

In reply to Max Reitz

Basically, bdrv_refresh_filename() should respect all children of a BlockDriverState. However, generally those children are driver-specific, so this function cannot handle the general case. On the other hand, there are only few drivers which use other children than @file and @backing (that being vmdk, quorum, and blkverify).

Most block drivers only use @file and/or @backing (if they use any children at all). Both can be implemented directly in bdrv_refresh_filename.

The user overriding the file's filename is already handled, however, the user overriding the backing file is not. If this is done, opening the BDS with the plain filename of its file will not be correct, so we may not set bs->exact_filename in that case.

iotests 051 and 191 contain test cases for overwriting the backing file, and so their output changes with this patch applied (which I consider a good thing). Note that in the case of 191, the implicitly opened (non-overridden) base file is included in the json:{} filename as well -- this will be remedied by a later patch.

---
 block.c                       | 12 +++++++++++-
 tests/qemu-iotests/051.out    |  8 ++++----
 tests/qemu-iotests/051.pc.out |  8 ++++----
 tests/qemu-iotests/191.out    | 24 ++++++++++++------------
 4 files changed, 31 insertions(+), 21 deletions(-)
block.c +11 -1
index eb67dfbcc0..2dc06cb9cb 100644
@@ -5016,6 +5016,7 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
         opts = qdict_new();
         has_open_options = append_open_options(opts, bs);
+        has_open_options |= bs->backing_overridden;
 
         /* If no specific options have been given for this BDS, the filename of
          * the underlying file should suffice for this one as well */
@@ -5027,11 +5028,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
          * file BDS. The full options QDict of that file BDS should somehow
          * contain a representation of the filename, therefore the following
          * suffices without querying the (exact_)filename of this BDS. */
-        if (bs->file->bs->full_open_options) {
+        if (bs->file->bs->full_open_options &&
+            (!bs->backing || bs->backing->bs->full_open_options))
+        {
             qdict_put_str(opts, "driver", drv->format_name);
             QINCREF(bs->file->bs->full_open_options);
             qdict_put(opts, "file", bs->file->bs->full_open_options);
 
+            if (bs->backing) {
+                QINCREF(bs->backing->bs->full_open_options);
+                qdict_put(opts, "backing", bs->backing->bs->full_open_options);
+            } else if (bs->backing_overridden && !bs->backing) {
+                qdict_put(opts, "backing", qstring_new());
+            }
+
             bs->full_open_options = opts;
         } else {
             QDECREF(opts);
tests/qemu-iotests/051.out +4 -4
index e3c6eaba57..50d5cd07c8 100644
@@ -59,7 +59,7 @@ QEMU X.Y.Z monitor - type 'help' for more information
 Testing: -drive file=TEST_DIR/t.qcow2,driver=qcow2,backing.file.filename=TEST_DIR/t.qcow2.orig,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.orig"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writeback
     Backing file:     TEST_DIR/t.qcow2.orig (chain depth: 1)
@@ -148,7 +148,7 @@ QEMU_PROG: -drive driver=null-co,cache=invalid_value: invalid cache option
 Testing: -drive file=TEST_DIR/t.qcow2,cache=writeback,backing.file.filename=TEST_DIR/t.qcow2.base,backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writeback
     Backing file:     TEST_DIR/t.qcow2.base (chain depth: 1)
@@ -168,7 +168,7 @@ backing-file: TEST_DIR/t.qcow2.base (file, read-only)
 Testing: -drive file=TEST_DIR/t.qcow2,cache=writethrough,backing.file.filename=TEST_DIR/t.qcow2.base,backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writethrough
     Backing file:     TEST_DIR/t.qcow2.base (chain depth: 1)
@@ -188,7 +188,7 @@ backing-file: TEST_DIR/t.qcow2.base (file, read-only)
 Testing: -drive file=TEST_DIR/t.qcow2,cache=unsafe,backing.file.filename=TEST_DIR/t.qcow2.base,backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writeback, ignore flushes
     Backing file:     TEST_DIR/t.qcow2.base (chain depth: 1)
tests/qemu-iotests/051.pc.out +4 -4
index f2c5622cee..406f83780b 100644
@@ -59,7 +59,7 @@ QEMU X.Y.Z monitor - type 'help' for more information
 Testing: -drive file=TEST_DIR/t.qcow2,driver=qcow2,backing.file.filename=TEST_DIR/t.qcow2.orig,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.orig"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writeback
     Backing file:     TEST_DIR/t.qcow2.orig (chain depth: 1)
@@ -240,7 +240,7 @@ QEMU_PROG: -drive driver=null-co,cache=invalid_value: invalid cache option
 Testing: -drive file=TEST_DIR/t.qcow2,cache=writeback,backing.file.filename=TEST_DIR/t.qcow2.base,backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writeback
     Backing file:     TEST_DIR/t.qcow2.base (chain depth: 1)
@@ -260,7 +260,7 @@ backing-file: TEST_DIR/t.qcow2.base (file, read-only)
 Testing: -drive file=TEST_DIR/t.qcow2,cache=writethrough,backing.file.filename=TEST_DIR/t.qcow2.base,backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writethrough
     Backing file:     TEST_DIR/t.qcow2.base (chain depth: 1)
@@ -280,7 +280,7 @@ backing-file: TEST_DIR/t.qcow2.base (file, read-only)
 Testing: -drive file=TEST_DIR/t.qcow2,cache=unsafe,backing.file.filename=TEST_DIR/t.qcow2.base,backing.cache.no-flush=on,backing.node-name=backing,backing.file.node-name=backing-file,file.node-name=file,if=none,id=drive0 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}} (qcow2)
     Removable device: not locked, tray closed
     Cache mode:       writeback, ignore flushes
     Backing file:     TEST_DIR/t.qcow2.base (chain depth: 1)
tests/qemu-iotests/191.out +12 -12
index 26ff0a315d..c1ce99985e 100644
@@ -61,7 +61,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "TEST_DIR/t.qcow2.ovl2",
+                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -95,7 +95,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "TEST_DIR/t.qcow2.ovl2",
+            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
             "encryption_key_missing": false
         },
         {
@@ -150,7 +150,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "TEST_DIR/t.qcow2",
+                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -184,7 +184,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "TEST_DIR/t.qcow2",
+            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}",
             "encryption_key_missing": false
         },
         {
@@ -464,7 +464,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "TEST_DIR/t.qcow2.ovl2",
+                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -498,7 +498,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "TEST_DIR/t.qcow2.ovl2",
+            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
             "encryption_key_missing": false
         },
         {
@@ -554,7 +554,7 @@ wrote 65536/65536 bytes at offset 1048576
                     },
                     "backing-filename-format": "qcow2",
                     "virtual-size": 67108864,
-                    "filename": "TEST_DIR/t.qcow2.ovl2",
+                    "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                     "cluster-size": 65536,
                     "format": "qcow2",
                     "actual-size": SIZE,
@@ -573,7 +573,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "TEST_DIR/t.qcow2.ovl3",
+                "filename": "json:{"backing": {"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl3"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -586,8 +586,8 @@ wrote 65536/65536 bytes at offset 1048576
                         "corrupt": false
                     }
                 },
-                "full-backing-filename": "TEST_DIR/t.qcow2.ovl2",
-                "backing-filename": "TEST_DIR/t.qcow2.ovl2",
+                "full-backing-filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+                "backing-filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                 "dirty-flag": false
             },
             "iops_wr": 0,
@@ -598,7 +598,7 @@ wrote 65536/65536 bytes at offset 1048576
             "iops": 0,
             "bps_wr": 0,
             "write_threshold": 0,
-            "backing_file": "TEST_DIR/t.qcow2.ovl2",
+            "backing_file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
             "encrypted": false,
             "bps": 0,
             "bps_rd": 0,
@@ -607,7 +607,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "TEST_DIR/t.qcow2.ovl3",
+            "file": "json:{"backing": {"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl3"}}",
             "encryption_key_missing": false
         },
         {
Signature · 1 line
2.13.6
#8 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 05/25] block: Respect backing bs in bdrv_refresh_filename

Quoted 9 lines · Max Reitz
block.c +1 -0
@@ -5016,6 +5016,7 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
         opts = qdict_new();
         has_open_options = append_open_options(opts, bs);
+        has_open_options |= bs->backing_overridden;
 
         /* If no specific options have been given for this BDS, the filename of
          * the underlying file should suffice for this one as well */

block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 5033

Quoted 7 lines
block.c +3 -1
@@ -5027,11 +5028,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
          * file BDS. The full options QDict of that file BDS should somehow
          * contain a representation of the filename, therefore the following
          * suffices without querying the (exact_)filename of this BDS. */
-        if (bs->file->bs->full_open_options) {
+        if (bs->file->bs->full_open_options &&
+            (!bs->backing || bs->backing->bs->full_open_options))
+        {

Does this mean that both file. and backing. have to be overriden?

Shouldn't that be a || instead of a && ??

block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 5043

Quoted 10 lines
block.c +6 -0
             qdict_put_str(opts, "driver", drv->format_name);
             QINCREF(bs->file->bs->full_open_options);
             qdict_put(opts, "file", bs->file->bs->full_open_options);
 
+            if (bs->backing) {
+                QINCREF(bs->backing->bs->full_open_options);
+                qdict_put(opts, "backing", bs->backing->bs->full_open_options);
+            } else if (bs->backing_overridden && !bs->backing) {
+                qdict_put(opts, "backing", qstring_new());
+            }

You don't need the !bs->backing in the second if, it's implied from the previous one.

Berto

author
#9 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 05/25] block: Respect backing bs in bdrv_refresh_filename

Quoted 7 lines · Alberto Garcia
Quoted 6 lines · Max Reitz
block.c +1 -0
@@ -5016,6 +5016,7 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
         opts = qdict_new();
         has_open_options = append_open_options(opts, bs);
+        has_open_options |= bs->backing_overridden;
 
Quoted 12 lines

block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 5033

Quoted 10 lines
block.c +3 -1
         /* If no specific options have been given for this BDS, the filename of
          * the underlying file should suffice for this one as well */
@@ -5027,11 +5028,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
          * file BDS. The full options QDict of that file BDS should somehow
          * contain a representation of the filename, therefore the following
          * suffices without querying the (exact_)filename of this BDS. */
-        if (bs->file->bs->full_open_options) {
+        if (bs->file->bs->full_open_options &&
+            (!bs->backing || bs->backing->bs->full_open_options))
+        {

Does this mean that both file. and backing. have to be overriden?

The bs->file->bs->full_open_options is not a check whether anything has been overridden but just whether we have a QDict of runtime options for file (and we should always have one). After this series, that check therefore disappears.

Quoted 1 line

Shouldn't that be a || instead of a && ??

The block under this if condition automatically constructs bs->full_open_options -- but it can only do so if all of the relevant children's options are known (because their options need to be put into bs->full_open_options).

Previously, that was only the file child. This patch adds support for the backing child: Whenever there is a backing child node, we need to include its options under the "backing" key.

So that's where the condition comes from: For all children that this node has, we need their full_open_options or we cannot construct this node's full_open_options.

We do have bs->file (because that is the condition for the block this condition is in), so we need bs->file->bs->full_open_options. And if we have bs->backing, we also need bs->backing->bs->full_open_options.

Quoted 11 lines

block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 5043

Quoted 10 lines
block.c +6 -0
             qdict_put_str(opts, "driver", drv->format_name);
             QINCREF(bs->file->bs->full_open_options);
             qdict_put(opts, "file", bs->file->bs->full_open_options);
 
+            if (bs->backing) {
+                QINCREF(bs->backing->bs->full_open_options);
+                qdict_put(opts, "backing", bs->backing->bs->full_open_options);
+            } else if (bs->backing_overridden && !bs->backing) {
+                qdict_put(opts, "backing", qstring_new());
+            }
Quoted 2 lines

You don't need the !bs->backing in the second if, it's implied from the previous one.

Maybe that was intentional (to be more explicit, and because after this series only this branch remains), maybe it wasn't. I don't know anymore, so I'll just drop it (even though I'll have to re-add it later, because as I said, only this else if branch will remain).

Max

Reviewed-by on v7 5/25
#10 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 05/25] block: Respect backing bs in bdrv_refresh_filename

In reply to Max Reitz
Quoted 15 lines · Max Reitz

Basically, bdrv_refresh_filename() should respect all children of a BlockDriverState. However, generally those children are driver-specific, so this function cannot handle the general case. On the other hand, there are only few drivers which use other children than @file and @backing (that being vmdk, quorum, and blkverify).

Most block drivers only use @file and/or @backing (if they use any children at all). Both can be implemented directly in bdrv_refresh_filename.

The user overriding the file's filename is already handled, however, the user overriding the backing file is not. If this is done, opening the BDS with the plain filename of its file will not be correct, so we may not set bs->exact_filename in that case.

Quoted 7 lines

iotests 051 and 191 contain test cases for overwriting the backing file, and so their output changes with this patch applied (which I consider a good thing). Note that in the case of 191, the implicitly opened (non-overridden) base file is included in the json:{} filename as well -- this will be remedied by a later patch.

Berto

author
#11 lore raw

PATCH v7 6/25 block: Make path_combine() return the path

In reply to Max Reitz

Besides being safe for arbitrary path lengths, after some follow-up patches all callers will want a freshly allocated buffer anyway.

In the meantime, path_combine_deprecated() is added which has the same interface as path_combine() had before this patch. All callers to that function will be converted in follow-up patches.

---
 include/block/block.h |  4 +--
 block.c               | 85 ++++++++++++++++++++++++++++-----------------------
 block/vmdk.c          |  3 +-
 3 files changed, 49 insertions(+), 43 deletions(-)
include/block/block.h +1 -3
index c05cac57e5..4584893c78 100644
@@ -490,9 +490,7 @@ void bdrv_get_full_backing_filename_from_filename(const char *backed,
 
 int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
-void path_combine(char *dest, int dest_size,
-                  const char *base_path,
-                  const char *filename);
+char *path_combine(const char *base_path, const char *filename);
 
 int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
 int bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
block.c +47 -38

123 lines not on this page · raw message

block/vmdk.c +1 -2
index c665bcc977..5397a4345e 100644
@@ -857,8 +857,7 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             return -EINVAL;
         }
 
-        extent_path = g_malloc0(PATH_MAX);
-        path_combine(extent_path, PATH_MAX, desc_file_path, fname);
+        extent_path = path_combine(desc_file_path, fname);
 
         ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
         assert(ret < 32);
Signature · 1 line
2.13.6
author
#12 lore raw

PATCH v7 7/25 block: bdrv_get_full_backing_filename_from_...'s ret. val.

In reply to Max Reitz

Make bdrv_get_full_backing_filename_from_filename() return an allocated string instead of placing the result in a caller-provided buffer.

---
 include/block/block.h |  7 +++----
 block.c               | 51 +++++++++++++++++++++++++++++++++++----------------
 block/vmdk.c          |  9 ++++-----
 3 files changed, 42 insertions(+), 25 deletions(-)
include/block/block.h +3 -4
index 4584893c78..46c6190414 100644
@@ -483,10 +483,9 @@ void bdrv_get_backing_filename(BlockDriverState *bs,
                                char *filename, int filename_size);
 void bdrv_get_full_backing_filename(BlockDriverState *bs,
                                     char *dest, size_t sz, Error **errp);
-void bdrv_get_full_backing_filename_from_filename(const char *backed,
-                                                  const char *backing,
-                                                  char *dest, size_t sz,
-                                                  Error **errp);
+char *bdrv_get_full_backing_filename_from_filename(const char *backed,
+                                                   const char *backing,
+                                                   Error **errp);
 
 int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
block.c +35 -16

81 lines not on this page · raw message

block/vmdk.c +4 -5
index 5397a4345e..c1dbcfe133 100644
@@ -2001,16 +2001,15 @@ static int vmdk_create(const char *filename, QemuOpts *opts, Error **errp)
     }
     if (backing_file) {
         BlockBackend *blk;
-        char *full_backing = g_new0(char, PATH_MAX);
-        bdrv_get_full_backing_filename_from_filename(filename, backing_file,
-                                                     full_backing, PATH_MAX,
-                                                     &local_err);
+        char *full_backing =
+            bdrv_get_full_backing_filename_from_filename(filename, backing_file,
+                                                         &local_err);
         if (local_err) {
-            g_free(full_backing);
             error_propagate(errp, local_err);
             ret = -ENOENT;
             goto exit;
         }
+        assert(full_backing);
 
         blk = blk_new_open(full_backing, NULL, NULL,
                            BDRV_O_NO_BACKING, errp);
Signature · 1 line
2.13.6
author
#13 lore raw

PATCH v7 8/25 block: bdrv_get_full_backing_filename's ret. val.

In reply to Max Reitz

Make bdrv_get_full_backing_filename() return an allocated string instead of placing the result in a caller-provided buffer.

---
 include/block/block.h |  3 +--
 block.c               | 47 +++++++++++++++++------------------------------
 block/qapi.c          | 12 ++----------
 3 files changed, 20 insertions(+), 42 deletions(-)
include/block/block.h +1 -2
index 46c6190414..3e2ea6c879 100644
@@ -481,8 +481,7 @@ void bdrv_round_to_clusters(BlockDriverState *bs,
 const char *bdrv_get_encrypted_filename(BlockDriverState *bs);
 void bdrv_get_backing_filename(BlockDriverState *bs,
                                char *filename, int filename_size);
-void bdrv_get_full_backing_filename(BlockDriverState *bs,
-                                    char *dest, size_t sz, Error **errp);
+char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp);
 char *bdrv_get_full_backing_filename_from_filename(const char *backed,
                                                    const char *backing,
                                                    Error **errp);
block.c +17 -30

100 lines not on this page · raw message

block/qapi.c +2 -10
index fc10f0a565..c7da9bd312 100644
@@ -282,18 +282,10 @@ void bdrv_query_image_info(BlockDriverState *bs,
 
     backing_filename = bs->backing_file;
     if (backing_filename[0] != '\0') {
-        char *backing_filename2 = g_malloc0(PATH_MAX);
+        char *backing_filename2;
         info->backing_filename = g_strdup(backing_filename);
         info->has_backing_filename = true;
-        bdrv_get_full_backing_filename(bs, backing_filename2, PATH_MAX, &err);
-        if (err) {
-            /* Can't reconstruct the full backing filename, so we must omit
-             * this field and apply a Best Effort to this query. */
-            g_free(backing_filename2);
-            backing_filename2 = NULL;
-            error_free(err);
-            err = NULL;
-        }
+        backing_filename2 = bdrv_get_full_backing_filename(bs, NULL);
 
         /* Always report the full_backing_filename if present, even if it's the
          * same as backing_filename. That they are same is useful info. */
Signature · 1 line
2.13.6
author
#14 lore raw

PATCH v7 9/25 block: Add bdrv_make_absolute_filename()

In reply to Max Reitz

This is a general function for making a filename that is relative to a certain BDS absolute.

It calls bdrv_get_full_backing_filename_from_filename() for now, but that will be changed in a follow-up patch.

---
 block.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)
block.c +16 -5
index 804adceafa..a11cebf0cb 100644
@@ -313,13 +313,24 @@ char *bdrv_get_full_backing_filename_from_filename(const char *backed,
     }
 }
 
-char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
+/* If @filename is empty or NULL, this function returns NULL without
+ * setting @errp.  In all other cases, NULL will only be returned with
+ * @errp set.
+ */
+static char *bdrv_make_absolute_filename(BlockDriverState *relative_to,
+                                         const char *filename, Error **errp)
 {
-    char *backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
+    char *bs_filename = relative_to->exact_filename[0]
+                        ? relative_to->exact_filename
+                        : relative_to->filename;
+
+    return bdrv_get_full_backing_filename_from_filename(bs_filename,
+                                                        filename ?: "", errp);
+}
 
-    return bdrv_get_full_backing_filename_from_filename(backed,
-                                                        bs->backing_file,
-                                                        errp);
+char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
+{
+    return bdrv_make_absolute_filename(bs, bs->backing_file, errp);
 }
 
 void bdrv_register(BlockDriver *bdrv)
Signature · 1 line
2.13.6
author
#15 lore raw

PATCH v7 10/25 block: Fix bdrv_find_backing_image()

In reply to Max Reitz

bdrv_find_backing_image() should use bdrv_get_full_backing_filename() or bdrv_make_absolute_filename() instead of trying to do what those functions do by itself.

path_combine_deprecated() can now be dropped, so let's do that.

---
 block.c | 30 ++++++++++--------------------
 1 file changed, 10 insertions(+), 20 deletions(-)
block.c +10 -20
index a11cebf0cb..ecea78dae0 100644
@@ -196,15 +196,6 @@ char *path_combine(const char *base_path, const char *filename)
     return result;
 }
 
-static void path_combine_deprecated(char *dest, int dest_size,
-                                    const char *base_path,
-                                    const char *filename)
-{
-    char *combined = path_combine(base_path, filename);
-    pstrcpy(dest, dest_size, combined);
-    g_free(combined);
-}
-
 /*
  * Helper function for bdrv_parse_filename() implementations to remove optional
  * protocol prefixes (especially "file:") from a filename and for putting the
@@ -4133,7 +4124,6 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
 
     filename_full     = g_malloc(PATH_MAX);
     backing_file_full = g_malloc(PATH_MAX);
-    filename_tmp      = g_malloc(PATH_MAX);
 
     is_protocol = path_has_protocol(backing_file);
 
@@ -4162,22 +4152,23 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
         } else {
             /* If not an absolute filename path, make it relative to the current
              * image's filename path */
-            path_combine_deprecated(filename_tmp, PATH_MAX, curr_bs->filename,
-                                    backing_file);
-
-            /* We are going to compare absolute pathnames */
-            if (!realpath(filename_tmp, filename_full)) {
+            filename_tmp = bdrv_make_absolute_filename(curr_bs, backing_file,
+                                                       NULL);
+            /* We are going to compare canonicalized absolute pathnames */
+            if (!filename_tmp || !realpath(filename_tmp, filename_full)) {
+                g_free(filename_tmp);
                 continue;
             }
+            g_free(filename_tmp);
 
             /* We need to make sure the backing filename we are comparing against
              * is relative to the current image filename (or absolute) */
-            path_combine_deprecated(filename_tmp, PATH_MAX, curr_bs->filename,
-                                    curr_bs->backing_file);
-
-            if (!realpath(filename_tmp, backing_file_full)) {
+            filename_tmp = bdrv_get_full_backing_filename(curr_bs, NULL);
+            if (!filename_tmp || !realpath(filename_tmp, backing_file_full)) {
+                g_free(filename_tmp);
                 continue;
             }
+            g_free(filename_tmp);
 
             if (strcmp(backing_file_full, filename_full) == 0) {
                 retval = curr_bs->backing->bs;
@@ -4188,7 +4179,6 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
 
     g_free(filename_full);
     g_free(backing_file_full);
-    g_free(filename_tmp);
     return retval;
 }
 
Signature · 1 line
2.13.6
Reviewed-by on v7 10/25
#16 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 10/25] block: Fix bdrv_find_backing_image()

Quoted 7 lines · Max Reitz

bdrv_find_backing_image() should use bdrv_get_full_backing_filename() or bdrv_make_absolute_filename() instead of trying to do what those functions do by itself.

path_combine_deprecated() can now be dropped, so let's do that.

Berto

author
#17 lore raw

PATCH v7 11/25 block: Add bdrv_dirname()

In reply to Max Reitz

This function may be implemented by block drivers to derive a directory name from a BDS. Concatenating this g_free()-able string with a relative filename must result in a valid (not necessarily existing) filename, so this is a function that should generally be not implemented by format drivers, because this is protocol-specific.

If a BDS's driver does not implement this function, bdrv_dirname() will fall through to the BDS's file if it exists. If it does not, the exact_filename field will be used to generate a directory name.

---
 include/block/block.h     |  1 +
 include/block/block_int.h |  7 +++++++
 block.c                   | 26 ++++++++++++++++++++++++++
 3 files changed, 34 insertions(+)
include/block/block.h +1 -0
index 3e2ea6c879..a053a27148 100644
@@ -485,6 +485,7 @@ char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp);
 char *bdrv_get_full_backing_filename_from_filename(const char *backed,
                                                    const char *backing,
                                                    Error **errp);
+char *bdrv_dirname(BlockDriverState *bs, Error **errp);
 
 int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
include/block/block_int.h +7 -0
index 5e9734d8b5..c07882962d 100644
@@ -135,6 +135,13 @@ struct BlockDriver {
 
     void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options);
 
+    /*
+     * Returns an allocated string which is the directory name of this BDS: It
+     * will be used to make relative filenames absolute by prepending this
+     * function's return value to them.
+     */
+    char *(*bdrv_dirname)(BlockDriverState *bs, Error **errp);
+
     /* aio */
     BlockAIOCB *(*bdrv_aio_readv)(BlockDriverState *bs,
         int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
block.c +26 -0
index ecea78dae0..20e45c85c6 100644
@@ -5098,6 +5098,32 @@ void bdrv_refresh_filename(BlockDriverState *bs)
     }
 }
 
+char *bdrv_dirname(BlockDriverState *bs, Error **errp)
+{
+    BlockDriver *drv = bs->drv;
+
+    if (!drv) {
+        error_setg(errp, "Node '%s' is ejected", bs->node_name);
+        return NULL;
+    }
+
+    if (drv->bdrv_dirname) {
+        return drv->bdrv_dirname(bs, errp);
+    }
+
+    if (bs->file) {
+        return bdrv_dirname(bs->file->bs, errp);
+    }
+
+    if (bs->exact_filename[0] != '\0') {
+        return path_combine(bs->exact_filename, "");
+    }
+
+    error_setg(errp, "Cannot generate a base directory for %s nodes",
+               drv->format_name);
+    return NULL;
+}
+
 /*
  * Hot add/remove a BDS's child. So the user can take a child offline when
  * it is broken and take a new child online
Signature · 1 line
2.13.6
author
#18 lore raw

PATCH v7 12/25 blkverify: Make bdrv_dirname() return NULL

In reply to Max Reitz

blkverify's BDSs have a file BDS, but we do not want this to be preferred over the raw node. There is no way to decide between the two (and not really a reason to, either), so just return NULL in blkverify's implementation of bdrv_dirname().

---
 block/blkverify.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
block/blkverify.c +10 -0
index b2ed8cd70d..d5233eeaf9 100644
@@ -309,6 +309,15 @@ static void blkverify_refresh_filename(BlockDriverState *bs, QDict *options)
     }
 }
 
+static char *blkverify_dirname(BlockDriverState *bs, Error **errp)
+{
+    /* In general, there are two BDSs with different dirnames below this one;
+     * so there is no unique dirname we could return (unless both are equal by
+     * chance). Therefore, to be consistent, just always return NULL. */
+    error_setg(errp, "Cannot generate a base directory for blkverify nodes");
+    return NULL;
+}
+
 static BlockDriver bdrv_blkverify = {
     .format_name                      = "blkverify",
     .protocol_name                    = "blkverify",
@@ -320,6 +329,7 @@ static BlockDriver bdrv_blkverify = {
     .bdrv_child_perm                  = bdrv_filter_default_perms,
     .bdrv_getlength                   = blkverify_getlength,
     .bdrv_refresh_filename            = blkverify_refresh_filename,
+    .bdrv_dirname                     = blkverify_dirname,
 
     .bdrv_co_preadv                   = blkverify_co_preadv,
     .bdrv_co_pwritev                  = blkverify_co_pwritev,
Signature · 1 line
2.13.6
author
#19 lore raw

PATCH v7 13/25 quorum: Make bdrv_dirname() return NULL

In reply to Max Reitz

While the common implementation for bdrv_dirname() should return NULL for quorum BDSs already (because they do not have a file node and their exact_filename field should be empty), there is no reason not to make that explicit.

---
 block/quorum.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
block/quorum.c +11 -0
index 2f1a628449..e5a844335e 100644
@@ -1095,6 +1095,16 @@ static void quorum_refresh_filename(BlockDriverState *bs, QDict *options)
     bs->full_open_options = opts;
 }
 
+static char *quorum_dirname(BlockDriverState *bs, Error **errp)
+{
+    /* In general, there are multiple BDSs with different dirnames below this
+     * one; so there is no unique dirname we could return (unless all are equal
+     * by chance, or there is only one). Therefore, to be consistent, just
+     * always return NULL. */
+    error_setg(errp, "Cannot generate a base directory for quorum nodes");
+    return NULL;
+}
+
 static BlockDriver bdrv_quorum = {
     .format_name                        = "quorum",
     .protocol_name                      = "quorum",
@@ -1104,6 +1114,7 @@ static BlockDriver bdrv_quorum = {
     .bdrv_file_open                     = quorum_open,
     .bdrv_close                         = quorum_close,
     .bdrv_refresh_filename              = quorum_refresh_filename,
+    .bdrv_dirname                       = quorum_dirname,
 
     .bdrv_co_flush_to_disk              = quorum_co_flush,
 
Signature · 1 line
2.13.6
author
#20 lore raw

PATCH v7 14/25 block/nbd: Make bdrv_dirname() return NULL

In reply to Max Reitz

The generic bdrv_dirname() implementation would be able to generate some form of directory name for many NBD nodes, but it would be always wrong. Therefore, we have to explicitly make it an error (until NBD has some form of specification for export paths, if it ever will).

---
 block/nbd.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
block/nbd.c +13 -0
index a50d24b50a..ed921fa333 100644
@@ -565,6 +565,16 @@ static void nbd_refresh_filename(BlockDriverState *bs, QDict *options)
     bs->full_open_options = opts;
 }
 
+static char *nbd_dirname(BlockDriverState *bs, Error **errp)
+{
+    /* The generic bdrv_dirname() implementation is able to work out some
+     * directory name for NBD nodes, but that would be wrong. So far there is no
+     * specification for how "export paths" would work, so NBD does not have
+     * directory names. */
+    error_setg(errp, "Cannot generate a base directory for NBD nodes");
+    return NULL;
+}
+
 static BlockDriver bdrv_nbd = {
     .format_name                = "nbd",
     .protocol_name              = "nbd",
@@ -582,6 +592,7 @@ static BlockDriver bdrv_nbd = {
     .bdrv_detach_aio_context    = nbd_detach_aio_context,
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
+    .bdrv_dirname               = nbd_dirname,
 };
 
 static BlockDriver bdrv_nbd_tcp = {
@@ -601,6 +612,7 @@ static BlockDriver bdrv_nbd_tcp = {
     .bdrv_detach_aio_context    = nbd_detach_aio_context,
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
+    .bdrv_dirname               = nbd_dirname,
 };
 
 static BlockDriver bdrv_nbd_unix = {
@@ -620,6 +632,7 @@ static BlockDriver bdrv_nbd_unix = {
     .bdrv_detach_aio_context    = nbd_detach_aio_context,
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
+    .bdrv_dirname               = nbd_dirname,
 };
 
 static void bdrv_nbd_init(void)
Signature · 1 line
2.13.6
Reviewed-by on v7 14/25
#21 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 14/25] block/nbd: Make bdrv_dirname() return NULL

Quoted 6 lines · Max Reitz

The generic bdrv_dirname() implementation would be able to generate some form of directory name for many NBD nodes, but it would be always wrong. Therefore, we have to explicitly make it an error (until NBD has some form of specification for export paths, if it ever will).

Berto

author
#22 lore raw

PATCH v7 15/25 block/nfs: Implement bdrv_dirname()

In reply to Max Reitz

While the basic idea is obvious and could be handled by the default bdrv_dirname() implementation, we cannot generate a directory name if the gid or uid are set, so we have to explicitly return NULL in those cases.

---
 block/nfs.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
block/nfs.c +14 -0
index 337fcd9c84..0152a9bd32 100644
@@ -875,6 +875,19 @@ static void nfs_refresh_filename(BlockDriverState *bs, QDict *options)
     bs->full_open_options = opts;
 }
 
+static char *nfs_dirname(BlockDriverState *bs, Error **errp)
+{
+    NFSClient *client = bs->opaque;
+
+    if (client->uid || client->gid) {
+        error_setg(errp, "Cannot generate a base directory for NBD node '%s'",
+                   bs->filename);
+        return NULL;
+    }
+
+    return g_strdup_printf("nfs://%s%s/", client->server->host, client->path);
+}
+
 #ifdef LIBNFS_FEATURE_PAGECACHE
 static void nfs_invalidate_cache(BlockDriverState *bs,
                                  Error **errp)
@@ -908,6 +921,7 @@ static BlockDriver bdrv_nfs = {
     .bdrv_detach_aio_context        = nfs_detach_aio_context,
     .bdrv_attach_aio_context        = nfs_attach_aio_context,
     .bdrv_refresh_filename          = nfs_refresh_filename,
+    .bdrv_dirname                   = nfs_dirname,
 
 #ifdef LIBNFS_FEATURE_PAGECACHE
     .bdrv_invalidate_cache          = nfs_invalidate_cache,
Signature · 1 line
2.13.6
author
#23 lore raw

PATCH v7 16/25 block: Use bdrv_dirname() for relative filenames

In reply to Max Reitz

bdrv_get_full_backing_filename_from_filename() breaks down when it comes to JSON filenames. Using bdrv_dirname() as the basis is better because since we have BDS, we can descend through the BDS tree to the protocol layer, which gives us a greater probability of finding a non-JSON name; also, bdrv_dirname() is more correct as it allows block drivers to override the generation of that directory name in a protocol-specific way.

We still need to keep bdrv_get_full_backing_filename_from_filename(), though, because it has valid callers which need it during image creation when no BDS is available yet.

This makes a test case in qemu-iotest 110, which was supposed to fail, work. That is actually good, but we need to change the reference output (and the comment in 110) accordingly.

---
 block.c                    | 20 +++++++++++++++-----
 tests/qemu-iotests/110     |  3 ++-
 tests/qemu-iotests/110.out |  2 +-
 3 files changed, 18 insertions(+), 7 deletions(-)
block.c +15 -5
index 20e45c85c6..2e59e77f9d 100644
@@ -311,12 +311,22 @@ char *bdrv_get_full_backing_filename_from_filename(const char *backed,
 static char *bdrv_make_absolute_filename(BlockDriverState *relative_to,
                                          const char *filename, Error **errp)
 {
-    char *bs_filename = relative_to->exact_filename[0]
-                        ? relative_to->exact_filename
-                        : relative_to->filename;
+    char *dir, *full_name;
 
-    return bdrv_get_full_backing_filename_from_filename(bs_filename,
-                                                        filename ?: "", errp);
+    if (!filename || filename[0] == '\0') {
+        return NULL;
+    } else if (path_has_protocol(filename) || path_is_absolute(filename)) {
+        return g_strdup(filename);
+    }
+
+    dir = bdrv_dirname(relative_to, errp);
+    if (!dir) {
+        return NULL;
+    }
+
+    full_name = g_strconcat(dir, filename, NULL);
+    g_free(dir);
+    return full_name;
 }
 
 char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
tests/qemu-iotests/110 +2 -1
index 9de7369f3a..ba1b3c6c7d 100755
@@ -61,7 +61,8 @@ echo '=== Non-reconstructable filename ==='
 echo
 
 # Across blkdebug without a config file, you cannot reconstruct filenames, so
-# qemu is incapable of knowing the directory of the top image
+# qemu is incapable of knowing the directory of the top image from the filename
+# alone. However, using bdrv_dirname(), it should still work.
 TEST_IMG="json:{
     'driver': '$IMGFMT',
     'file': {
tests/qemu-iotests/110.out +1 -1
index b3584ff87f..5370bc1d26 100644
@@ -14,7 +14,7 @@ backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
 image: json:{"driver": "IMGFMT", "file": {"set-state.0.event": "read_aio", "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "set-state.0.new_state": 42}}
 file format: IMGFMT
 virtual size: 64M (67108864 bytes)
-backing file: t.IMGFMT.base (cannot determine actual path)
+backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
 
 === Backing name is always relative to the backed image ===
 
Signature · 1 line
2.13.6
author
#24 lore raw

PATCH v7 17/25 iotests: Add quorum case to test 110

In reply to Max Reitz

Test 110 tests relative backing filenames for complex BDS trees. Now that the originally supposedly failing test passes, let us add a new failing test: Quorum can never work automatically (without detecting whether all child nodes have the same base directory, but that would be rather inconsistent behavior).

---
 tests/qemu-iotests/110     | 26 ++++++++++++++++++++++++++
 tests/qemu-iotests/110.out |  7 +++++++
 2 files changed, 33 insertions(+)
tests/qemu-iotests/110 +26 -0
index ba1b3c6c7d..08976c8a61 100755
@@ -30,6 +30,7 @@ status=1	# failure is the default!
 _cleanup()
 {
 	_cleanup_test_img
+        rm -f "$TEST_IMG.copy"
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15
 
@@ -87,6 +88,31 @@ echo
 # omit the image size; it should work anyway
 _make_test_img -b "$TEST_IMG_REL.base"
 
+echo
+echo '=== Nodes without a common directory ==='
+echo
+
+cp "$TEST_IMG" "$TEST_IMG.copy"
+
+# Should inform us that the actual path of the backing file cannot be determined
+TEST_IMG="json:{
+    'driver': '$IMGFMT',
+    'file': {
+        'driver': 'quorum',
+        'vote-threshold': 1,
+        'children': [
+            {
+                'driver': 'file',
+                'filename': '$TEST_IMG'
+            },
+            {
+                'driver': 'file',
+                'filename': '$TEST_IMG.copy'
+            }
+        ]
+    }
+}" _img_info | _filter_img_info
+
 
 # success, all done
 echo '*** done'
tests/qemu-iotests/110.out +7 -0
index 5370bc1d26..1d0b2475cc 100644
@@ -19,4 +19,11 @@ backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
 === Backing name is always relative to the backed image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=t.IMGFMT.base
+
+=== Nodes without a common directory ===
+
+image: json:{"driver": "IMGFMT", "file": {"children": [{"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.copy"}], "driver": "quorum", "blkverify": false, "rewrite-corrupted": false, "vote-threshold": 1}}
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+backing file: t.IMGFMT.base (cannot determine actual path)
 *** done
Signature · 1 line
2.13.6
Reviewed-by on v7 17/25
#25 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 17/25] iotests: Add quorum case to test 110

Quoted 7 lines · Max Reitz

Test 110 tests relative backing filenames for complex BDS trees. Now that the originally supposedly failing test passes, let us add a new failing test: Quorum can never work automatically (without detecting whether all child nodes have the same base directory, but that would be rather inconsistent behavior).

Berto

author
#26 lore raw

PATCH v7 18/25 block: Add sgfnt_runtime_opts to BlockDriver

In reply to Max Reitz

This new field can be set by block drivers to list the runtime options they accept that may influence the contents of the respective BDS. As of a follow-up patch, this list will be used by the common bdrv_refresh_filename() implementation to decide which options to put into BDS.full_open_options (and consequently whether a JSON filename has to be created), thus freeing the drivers of having to implement that logic themselves.

Additionally, this patch adds the field to all of the block drivers that need it and sets it accordingly.

---
 include/block/block_int.h |  7 +++++++
 block/blkdebug.c          | 17 +++++++++++++++++
 block/crypto.c            |  8 ++++++++
 block/curl.c              | 21 +++++++++++++++++++++
 block/gluster.c           | 19 +++++++++++++++++++
 block/iscsi.c             | 18 ++++++++++++++++++
 block/nbd.c               | 14 ++++++++++++++
 block/nfs.c               | 11 +++++++++++
 block/null.c              |  9 +++++++++
 block/qcow.c              |  7 +++++++
 block/qcow2.c             |  7 +++++++
 block/quorum.c            | 11 +++++++++++
 block/raw-format.c        | 10 +++++++++-
 block/rbd.c               | 14 ++++++++++++++
 block/replication.c       |  8 ++++++++
 block/sheepdog.c          | 12 ++++++++++++
 block/ssh.c               | 12 ++++++++++++
 block/throttle.c          |  7 +++++++
 block/vpc.c               |  7 +++++++
 block/vvfat.c             | 12 ++++++++++++
 block/vxhs.c              | 11 +++++++++++
 21 files changed, 241 insertions(+), 1 deletion(-)
include/block/block_int.h +7 -0
index c07882962d..d99ded8891 100644
@@ -454,6 +454,13 @@ struct BlockDriver {
                                                 Error **errp);
 
     QLIST_ENTRY(BlockDriver) list;
+
+    /* Pointer to a NULL-terminated array of names of significant options that
+     * can be specified for bdrv_open(). A significant option is one that
+     * changes the data of a BDS.
+     * If this pointer is NULL, the array is considered empty.
+     * "filename" and "driver" are always considered significant. */
+    const char *const *sgfnt_runtime_opts;
 };
 
 typedef struct BlockLimits {
block/blkdebug.c +17 -0
index e21669979d..f93139da58 100644
@@ -886,6 +886,21 @@ static int blkdebug_reopen_prepare(BDRVReopenState *reopen_state,
     return 0;
 }
 
+static const char *const blkdebug_sgfnt_runtime_opts[] = {
+    "config",
+    "inject-error.",
+    "set-state.",
+    "suspend.",
+    "align",
+    "max-transfer",
+    "opt-write-zero",
+    "max-write-zero",
+    "opt-discard",
+    "max-discard",
+
+    NULL
+};
+
 static BlockDriver bdrv_blkdebug = {
     .format_name            = "blkdebug",
     .protocol_name          = "blkdebug",
@@ -915,6 +930,8 @@ static BlockDriver bdrv_blkdebug = {
                                 = blkdebug_debug_remove_breakpoint,
     .bdrv_debug_resume          = blkdebug_debug_resume,
     .bdrv_debug_is_suspended    = blkdebug_debug_is_suspended,
+
+    .sgfnt_runtime_opts         = blkdebug_sgfnt_runtime_opts,
 };
 
 static void bdrv_blkdebug_init(void)
block/crypto.c +8 -0
index 60ddf8623e..492e64e922 100644
@@ -609,6 +609,12 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs)
     return spec_info;
 }
 
+static const char *const block_crypto_sgfnt_runtime_opts[] = {
+    BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET,
+
+    NULL
+};
+
 BlockDriver bdrv_crypto_luks = {
     .format_name        = "luks",
     .instance_size      = sizeof(BlockCrypto),
@@ -626,6 +632,8 @@ BlockDriver bdrv_crypto_luks = {
     .bdrv_getlength     = block_crypto_getlength,
     .bdrv_get_info      = block_crypto_get_info_luks,
     .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
+
+    .sgfnt_runtime_opts = block_crypto_sgfnt_runtime_opts,
 };
 
 static void block_crypto_init(void)
block/curl.c +21 -0
index 2a244e2439..11318a9a29 100644
@@ -957,6 +957,19 @@ static int64_t curl_getlength(BlockDriverState *bs)
     return s->len;
 }
 
+static const char *const curl_sgfnt_runtime_opts[] = {
+    CURL_BLOCK_OPT_URL,
+    CURL_BLOCK_OPT_SSLVERIFY,
+    CURL_BLOCK_OPT_COOKIE,
+    CURL_BLOCK_OPT_COOKIE_SECRET,
+    CURL_BLOCK_OPT_USERNAME,
+    CURL_BLOCK_OPT_PASSWORD_SECRET,
+    CURL_BLOCK_OPT_PROXY_USERNAME,
+    CURL_BLOCK_OPT_PROXY_PASSWORD_SECRET,
+
+    NULL
+};
+
 static BlockDriver bdrv_http = {
     .format_name                = "http",
     .protocol_name              = "http",
@@ -971,6 +984,8 @@ static BlockDriver bdrv_http = {
 
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
+
+    .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_https = {
@@ -987,6 +1002,8 @@ static BlockDriver bdrv_https = {
 
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
+
+    .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_ftp = {
@@ -1003,6 +1020,8 @@ static BlockDriver bdrv_ftp = {
 
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
+
+    .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_ftps = {
@@ -1019,6 +1038,8 @@ static BlockDriver bdrv_ftps = {
 
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
+
+    .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
 static void curl_block_init(void)
block/gluster.c +19 -0
index 0f4265a3a4..e47bda97df 100644
@@ -1414,6 +1414,21 @@ static int64_t coroutine_fn qemu_gluster_co_get_block_status(
 }
 
 
+static const char *const gluster_sgfnt_open_opts[] = {
+    GLUSTER_OPT_VOLUME,
+    GLUSTER_OPT_PATH,
+    GLUSTER_OPT_TYPE,
+    GLUSTER_OPT_SERVER_PATTERN,
+    GLUSTER_OPT_HOST,
+    GLUSTER_OPT_PORT,
+    GLUSTER_OPT_TO,
+    GLUSTER_OPT_IPV4,
+    GLUSTER_OPT_IPV6,
+    GLUSTER_OPT_SOCKET,
+
+    NULL
+};
+
 static BlockDriver bdrv_gluster = {
     .format_name                  = "gluster",
     .protocol_name                = "gluster",
@@ -1440,6 +1455,7 @@ static BlockDriver bdrv_gluster = {
 #endif
     .bdrv_co_get_block_status     = qemu_gluster_co_get_block_status,
     .create_opts                  = &qemu_gluster_create_opts,
+    .sgfnt_runtime_opts           = gluster_sgfnt_open_opts,
 };
 
 static BlockDriver bdrv_gluster_tcp = {
@@ -1468,6 +1484,7 @@ static BlockDriver bdrv_gluster_tcp = {
 #endif
     .bdrv_co_get_block_status     = qemu_gluster_co_get_block_status,
     .create_opts                  = &qemu_gluster_create_opts,
+    .sgfnt_runtime_opts           = gluster_sgfnt_open_opts,
 };
 
 static BlockDriver bdrv_gluster_unix = {
@@ -1496,6 +1513,7 @@ static BlockDriver bdrv_gluster_unix = {
 #endif
     .bdrv_co_get_block_status     = qemu_gluster_co_get_block_status,
     .create_opts                  = &qemu_gluster_create_opts,
+    .sgfnt_runtime_opts           = gluster_sgfnt_open_opts,
 };
 
 /* rdma is deprecated (actually never supported for volfile fetch).
@@ -1530,6 +1548,7 @@ static BlockDriver bdrv_gluster_rdma = {
 #endif
     .bdrv_co_get_block_status     = qemu_gluster_co_get_block_status,
     .create_opts                  = &qemu_gluster_create_opts,
+    .sgfnt_runtime_opts           = gluster_sgfnt_open_opts,
 };
 
 static void bdrv_gluster_init(void)
block/iscsi.c +18 -0
index 4683f3b244..5da74e3ccb 100644
@@ -2161,6 +2161,20 @@ static QemuOptsList iscsi_create_opts = {
     }
 };
 
+static const char *const iscsi_sgfnt_runtime_opts[] = {
+    "transport",
+    "portal",
+    "target",
+    "user",
+    "password",
+    "password-secret",
+    "lun",
+    "initiator-name",
+    "header-digest",
+
+    NULL
+};
+
 static BlockDriver bdrv_iscsi = {
     .format_name     = "iscsi",
     .protocol_name   = "iscsi",
@@ -2193,6 +2207,8 @@ static BlockDriver bdrv_iscsi = {
 
     .bdrv_detach_aio_context = iscsi_detach_aio_context,
     .bdrv_attach_aio_context = iscsi_attach_aio_context,
+
+    .sgfnt_runtime_opts = iscsi_sgfnt_runtime_opts,
 };
 
 #if LIBISCSI_API_VERSION >= (20160603)
@@ -2228,6 +2244,8 @@ static BlockDriver bdrv_iser = {
 
     .bdrv_detach_aio_context = iscsi_detach_aio_context,
     .bdrv_attach_aio_context = iscsi_attach_aio_context,
+
+    .sgfnt_runtime_opts = iscsi_sgfnt_runtime_opts,
 };
 #endif
 
block/nbd.c +14 -0
index ed921fa333..0a36394b73 100644
@@ -575,6 +575,17 @@ static char *nbd_dirname(BlockDriverState *bs, Error **errp)
     return NULL;
 }
 
+static const char *const nbd_sgfnt_runtime_opts[] = {
+    "path",
+    "host",
+    "port",
+    "export",
+    "tls-creds",
+    "server.",
+
+    NULL
+};
+
 static BlockDriver bdrv_nbd = {
     .format_name                = "nbd",
     .protocol_name              = "nbd",
@@ -593,6 +604,7 @@ static BlockDriver bdrv_nbd = {
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
     .bdrv_dirname               = nbd_dirname,
+    .sgfnt_runtime_opts         = nbd_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_nbd_tcp = {
@@ -613,6 +625,7 @@ static BlockDriver bdrv_nbd_tcp = {
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
     .bdrv_dirname               = nbd_dirname,
+    .sgfnt_runtime_opts         = nbd_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_nbd_unix = {
@@ -633,6 +646,7 @@ static BlockDriver bdrv_nbd_unix = {
     .bdrv_attach_aio_context    = nbd_attach_aio_context,
     .bdrv_refresh_filename      = nbd_refresh_filename,
     .bdrv_dirname               = nbd_dirname,
+    .sgfnt_runtime_opts         = nbd_sgfnt_runtime_opts,
 };
 
 static void bdrv_nbd_init(void)
block/nfs.c +11 -0
index 0152a9bd32..445a8d9880 100644
@@ -897,6 +897,15 @@ static void nfs_invalidate_cache(BlockDriverState *bs,
 }
 #endif
 
+static const char *nfs_sgfnt_runtime_opts[] = {
+    "path",
+    "user",
+    "group",
+    "server.",
+
+    NULL
+};
+
 static BlockDriver bdrv_nfs = {
     .format_name                    = "nfs",
     .protocol_name                  = "nfs",
@@ -923,6 +932,8 @@ static BlockDriver bdrv_nfs = {
     .bdrv_refresh_filename          = nfs_refresh_filename,
     .bdrv_dirname                   = nfs_dirname,
 
+    .sgfnt_runtime_opts             = nfs_sgfnt_runtime_opts,
+
 #ifdef LIBNFS_FEATURE_PAGECACHE
     .bdrv_invalidate_cache          = nfs_invalidate_cache,
 #endif
block/null.c +9 -0
index dd9c13f9ba..59673d2449 100644
@@ -255,6 +255,13 @@ static void null_refresh_filename(BlockDriverState *bs, QDict *opts)
     bs->full_open_options = opts;
 }
 
+static const char *const null_sgfnt_runtime_opts[] = {
+    BLOCK_OPT_SIZE,
+    NULL_OPT_ZEROES,
+
+    NULL
+};
+
 static BlockDriver bdrv_null_co = {
     .format_name            = "null-co",
     .protocol_name          = "null-co",
@@ -273,6 +280,7 @@ static BlockDriver bdrv_null_co = {
     .bdrv_co_get_block_status   = null_co_get_block_status,
 
     .bdrv_refresh_filename  = null_refresh_filename,
+    .sgfnt_runtime_opts     = null_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_null_aio = {
@@ -293,6 +301,7 @@ static BlockDriver bdrv_null_aio = {
     .bdrv_co_get_block_status   = null_co_get_block_status,
 
     .bdrv_refresh_filename  = null_refresh_filename,
+    .sgfnt_runtime_opts     = null_sgfnt_runtime_opts,
 };
 
 static void bdrv_null_init(void)
block/qcow.c +7 -0
index 9569deeaf0..2bfe506ec4 100644
@@ -1097,6 +1097,12 @@ static QemuOptsList qcow_create_opts = {
     }
 };
 
+static const char *const qcow_sgfnt_runtime_opts[] = {
+    "encrypt." BLOCK_CRYPTO_OPT_QCOW_KEY_SECRET,
+
+    NULL
+};
+
 static BlockDriver bdrv_qcow = {
     .format_name	= "qcow",
     .instance_size	= sizeof(BDRVQcowState),
@@ -1118,6 +1124,7 @@ static BlockDriver bdrv_qcow = {
     .bdrv_get_info          = qcow_get_info,
 
     .create_opts            = &qcow_create_opts,
+    .sgfnt_runtime_opts     = qcow_sgfnt_runtime_opts,
 };
 
 static void bdrv_qcow_init(void)
block/qcow2.c +7 -0
index 1914a940e5..2c50eb9245 100644
@@ -4342,6 +4342,12 @@ static QemuOptsList qcow2_create_opts = {
     }
 };
 
+static const char *const qcow2_sgfnt_runtime_opts[] = {
+    "encrypt." BLOCK_CRYPTO_OPT_QCOW_KEY_SECRET,
+
+    NULL
+};
+
 BlockDriver bdrv_qcow2 = {
     .format_name        = "qcow2",
     .instance_size      = sizeof(BDRVQcow2State),
@@ -4387,6 +4393,7 @@ BlockDriver bdrv_qcow2 = {
     .bdrv_inactivate            = qcow2_inactivate,
 
     .create_opts         = &qcow2_create_opts,
+    .sgfnt_runtime_opts  = qcow2_sgfnt_runtime_opts,
     .bdrv_check          = qcow2_check,
     .bdrv_amend_options  = qcow2_amend_options,
 
block/quorum.c +11 -0
index e5a844335e..4b38201aa2 100644
@@ -1105,6 +1105,15 @@ static char *quorum_dirname(BlockDriverState *bs, Error **errp)
     return NULL;
 }
 
+static const char *const quorum_sgfnt_runtime_opts[] = {
+    QUORUM_OPT_VOTE_THRESHOLD,
+    QUORUM_OPT_BLKVERIFY,
+    QUORUM_OPT_REWRITE,
+    QUORUM_OPT_READ_PATTERN,
+
+    NULL
+};
+
 static BlockDriver bdrv_quorum = {
     .format_name                        = "quorum",
     .protocol_name                      = "quorum",
@@ -1130,6 +1139,8 @@ static BlockDriver bdrv_quorum = {
 
     .is_filter                          = true,
     .bdrv_recurse_is_first_non_filter   = quorum_recurse_is_first_non_filter,
+
+    .sgfnt_runtime_opts                 = quorum_sgfnt_runtime_opts,
 };
 
 static void bdrv_quorum_init(void)
block/raw-format.c +9 -1
index ab552c0954..f09b707b0d 100644
@@ -481,6 +481,13 @@ static int raw_probe_geometry(BlockDriverState *bs, HDGeometry *geo)
     return bdrv_probe_geometry(bs->file->bs, geo);
 }
 
+static const char *const raw_sgfnt_runtime_opts[] = {
+    "offset",
+    "size",
+
+    NULL
+};
+
 BlockDriver bdrv_raw = {
     .format_name          = "raw",
     .instance_size        = sizeof(BDRVRawState),
@@ -509,7 +516,8 @@ BlockDriver bdrv_raw = {
     .bdrv_lock_medium     = &raw_lock_medium,
     .bdrv_co_ioctl        = &raw_co_ioctl,
     .create_opts          = &raw_create_opts,
-    .bdrv_has_zero_init   = &raw_has_zero_init
+    .bdrv_has_zero_init   = &raw_has_zero_init,
+    .sgfnt_runtime_opts   = raw_sgfnt_runtime_opts,
 };
 
 static void bdrv_raw_init(void)
block/rbd.c +14 -0
index a76a5e8755..389f006bea 100644
@@ -1122,6 +1122,18 @@ static QemuOptsList qemu_rbd_create_opts = {
     }
 };
 
+static const char *const qemu_rbd_sgfnt_runtime_opts[] = {
+    "pool",
+    "image",
+    "conf",
+    "snapshot",
+    "user",
+    "server.",
+    "password-secret",
+
+    NULL
+};
+
 static BlockDriver bdrv_rbd = {
     .format_name            = "rbd",
     .instance_size          = sizeof(BDRVRBDState),
@@ -1157,6 +1169,8 @@ static BlockDriver bdrv_rbd = {
 #ifdef LIBRBD_SUPPORTS_INVALIDATE
     .bdrv_invalidate_cache  = qemu_rbd_invalidate_cache,
 #endif
+
+    .sgfnt_runtime_opts     = qemu_rbd_sgfnt_runtime_opts,
 };
 
 static void bdrv_rbd_init(void)
block/replication.c +8 -0
index e41e293d2b..c2b9442a5f 100644
@@ -695,6 +695,13 @@ static void replication_stop(ReplicationState *rs, bool failover, Error **errp)
     aio_context_release(aio_context);
 }
 
+static const char *const replication_sgfnt_runtime_opts[] = {
+    REPLICATION_MODE,
+    REPLICATION_TOP_ID,
+
+    NULL
+};
+
 BlockDriver bdrv_replication = {
     .format_name                = "replication",
     .protocol_name              = "replication",
@@ -712,6 +719,7 @@ BlockDriver bdrv_replication = {
     .bdrv_recurse_is_first_non_filter = replication_recurse_is_first_non_filter,
 
     .has_variable_length        = true,
+    .sgfnt_runtime_opts         = replication_sgfnt_runtime_opts,
 };
 
 static void bdrv_replication_init(void)
block/sheepdog.c +12 -0
index 696a71442a..1b788534b6 100644
@@ -3077,6 +3077,15 @@ static QemuOptsList sd_create_opts = {
     }
 };
 
+static const char *const sd_sgfnt_runtime_opts[] = {
+    "vdi",
+    "snap-id",
+    "tag",
+    "server.",
+
+    NULL
+};
+
 static BlockDriver bdrv_sheepdog = {
     .format_name    = "sheepdog",
     .protocol_name  = "sheepdog",
@@ -3111,6 +3120,7 @@ static BlockDriver bdrv_sheepdog = {
     .bdrv_attach_aio_context = sd_attach_aio_context,
 
     .create_opts    = &sd_create_opts,
+    .sgfnt_runtime_opts = sd_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_sheepdog_tcp = {
@@ -3147,6 +3157,7 @@ static BlockDriver bdrv_sheepdog_tcp = {
     .bdrv_attach_aio_context = sd_attach_aio_context,
 
     .create_opts    = &sd_create_opts,
+    .sgfnt_runtime_opts = sd_sgfnt_runtime_opts,
 };
 
 static BlockDriver bdrv_sheepdog_unix = {
@@ -3183,6 +3194,7 @@ static BlockDriver bdrv_sheepdog_unix = {
     .bdrv_attach_aio_context = sd_attach_aio_context,
 
     .create_opts    = &sd_create_opts,
+    .sgfnt_runtime_opts = sd_sgfnt_runtime_opts,
 };
 
 static void bdrv_sheepdog_init(void)
block/ssh.c +12 -0
index b049a16eb9..0015c06a3b 100644
@@ -1195,6 +1195,17 @@ static int64_t ssh_getlength(BlockDriverState *bs)
     return length;
 }
 
+static const char *const ssh_sgfnt_runtime_opts[] = {
+    "host",
+    "port",
+    "path",
+    "user",
+    "host_key_check",
+    "server.",
+
+    NULL
+};
+
 static BlockDriver bdrv_ssh = {
     .format_name                  = "ssh",
     .protocol_name                = "ssh",
@@ -1209,6 +1220,7 @@ static BlockDriver bdrv_ssh = {
     .bdrv_getlength               = ssh_getlength,
     .bdrv_co_flush_to_disk        = ssh_co_flush,
     .create_opts                  = &ssh_create_opts,
+    .sgfnt_runtime_opts           = ssh_sgfnt_runtime_opts,
 };
 
 static void bdrv_ssh_init(void)
block/throttle.c +7 -0
index 833175ac77..efdc57616b 100644
@@ -212,6 +212,12 @@ static void coroutine_fn throttle_co_drain_end(BlockDriverState *bs)
     atomic_dec(&tgm->io_limits_disabled);
 }
 
+static const char *const throttle_sgfnt_runtime_opts[] = {
+    QEMU_OPT_THROTTLE_GROUP_NAME,
+
+    NULL
+};
+
 static BlockDriver bdrv_throttle = {
     .format_name                        =   "throttle",
     .protocol_name                      =   "throttle",
@@ -245,6 +251,7 @@ static BlockDriver bdrv_throttle = {
     .bdrv_co_drain_end                  =   throttle_co_drain_end,
 
     .is_filter                          =   true,
+    .sgfnt_runtime_opts                 =   throttle_sgfnt_runtime_opts,
 };
 
 static void bdrv_throttle_init(void)
block/vpc.c +7 -0
index 1576d7b595..a0a30b6e33 100644
@@ -1084,6 +1084,12 @@ static QemuOptsList vpc_create_opts = {
     }
 };
 
+static const char *const vpc_sgfnt_runtime_opts[] = {
+    VPC_OPT_SIZE_CALC,
+
+    NULL
+};
+
 static BlockDriver bdrv_vpc = {
     .format_name    = "vpc",
     .instance_size  = sizeof(BDRVVPCState),
@@ -1103,6 +1109,7 @@ static BlockDriver bdrv_vpc = {
 
     .create_opts            = &vpc_create_opts,
     .bdrv_has_zero_init     = vpc_has_zero_init,
+    .sgfnt_runtime_opts     = vpc_sgfnt_runtime_opts,
 };
 
 static void bdrv_vpc_init(void)
block/vvfat.c +12 -0
index a690595f2c..76a91f97cb 100644
@@ -3242,6 +3242,16 @@ static void vvfat_close(BlockDriverState *bs)
     }
 }
 
+static const char *const vvfat_sgfnt_runtime_opts[] = {
+    "dir",
+    "fat-type",
+    "floppy",
+    "label",
+    "rw",
+
+    NULL
+};
+
 static BlockDriver bdrv_vvfat = {
     .format_name            = "vvfat",
     .protocol_name          = "fat",
@@ -3256,6 +3266,8 @@ static BlockDriver bdrv_vvfat = {
     .bdrv_co_preadv         = vvfat_co_preadv,
     .bdrv_co_pwritev        = vvfat_co_pwritev,
     .bdrv_co_get_block_status = vvfat_co_get_block_status,
+
+    .sgfnt_runtime_opts     = vvfat_sgfnt_runtime_opts,
 };
 
 static void bdrv_vvfat_init(void)
block/vxhs.c +11 -0
index 75cc6c8672..4d6dc5fe4c 100644
@@ -555,6 +555,16 @@ static int64_t vxhs_getlength(BlockDriverState *bs)
     return vdisk_size;
 }
 
+static const char *const vxhs_sgfnt_runtime_opts[] = {
+    VXHS_OPT_VDISK_ID,
+    "tls-creds",
+    VXHS_OPT_HOST,
+    VXHS_OPT_PORT,
+    VXHS_OPT_SERVER".",
+
+    NULL
+};
+
 static BlockDriver bdrv_vxhs = {
     .format_name                  = "vxhs",
     .protocol_name                = "vxhs",
@@ -565,6 +575,7 @@ static BlockDriver bdrv_vxhs = {
     .bdrv_getlength               = vxhs_getlength,
     .bdrv_aio_readv               = vxhs_aio_readv,
     .bdrv_aio_writev              = vxhs_aio_writev,
+    .sgfnt_runtime_opts           = vxhs_sgfnt_runtime_opts,
 };
 
 static void bdrv_vxhs_init(void)
Signature · 1 line
2.13.6
author
#27 lore raw

PATCH v7 19/25 block: Add BlockDriver.bdrv_gather_child_options

In reply to Max Reitz

Some follow-up patches will rework the way bs->full_open_options is refreshed in bdrv_refresh_filename(). The new implementation will remove the need for the block drivers' bdrv_refresh_filename() implementations to set bs->full_open_options; instead, it will be generic and use static information from each block driver.

However, by implementing bdrv_gather_child_options(), block drivers will still be able to override the way the full_open_options of their children are incorporated into their own.

We need to implement this function for VMDK because we have to prevent the generic implementation from gathering the options of all children:
It is not possible to specify options for the extents through the runtime options.

For quorum, the child names that would be used by the generic implementation and the ones that we actually (currently) want to use differ. See quorum_gather_child_options() for more information.

Note that both of these are cases which are not ideal: In case of VMDK it would probably be nice to be able to specify options for all extents. In case of quorum, the current runtime option structure is simply broken and needs to be fixed (but that is left for another patch).

---
 include/block/block_int.h | 17 +++++++++++++++++
 block/quorum.c            | 38 ++++++++++++++++++++++++++++++++++++++
 block/vmdk.c              | 13 +++++++++++++
 3 files changed, 68 insertions(+)
include/block/block_int.h +17 -0
index d99ded8891..f81516b615 100644
@@ -136,6 +136,23 @@ struct BlockDriver {
     void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options);
 
     /*
+     * Gathers the open options for all children into @target.
+     * A simple format driver (without backing file support) might
+     * implement this function like this:
+     *
+     *     QINCREF(bs->file->bs->full_open_options);
+     *     qdict_put(target, "file", bs->file->bs->full_open_options);
+     *
+     * If not specified, the generic implementation will simply put
+     * all children's options under their respective name.
+     *
+     * Note that ideally this function would not be needed.  Every
+     * block driver which implements it is probably doing something
+     * shady regarding its runtime option structure.
+     */
+    void (*bdrv_gather_child_options)(BlockDriverState *bs, QDict *target);
+
+    /*
      * Returns an allocated string which is the directory name of this BDS: It
      * will be used to make relative filenames absolute by prepending this
      * function's return value to them.
block/quorum.c +38 -0
index 4b38201aa2..9d680addc6 100644
@@ -1095,6 +1095,43 @@ static void quorum_refresh_filename(BlockDriverState *bs, QDict *options)
     bs->full_open_options = opts;
 }
 
+static void quorum_gather_child_options(BlockDriverState *bs, QDict *target)
+{
+    BDRVQuorumState *s = bs->opaque;
+    QList *children_list;
+    int i;
+
+    /* The generic implementation for gathering child options in
+     * bdrv_refresh_filename() would use the names of the children
+     * as specified for bdrv_open_child() or bdrv_attach_child(),
+     * which is "children.%u" with %u being a value
+     * (s->next_child_index) that is incremented each time a new child
+     * is added (and never decremented).  Since children can be
+     * deleted at runtime, there may be gaps in that enumeration.
+     * When creating a new quorum BDS and specifying the children for
+     * it through runtime options, the enumeration used there may not
+     * have any gaps, though.
+     *
+     * Therefore, we have to create a new gap-less enumeration here
+     * (which we can achieve by simply putting all of the children's
+     * full_open_options into a QList).
+     *
+     * XXX: Note that there are issues with the current child option
+     *      structure quorum uses (such as the fact that children do
+     *      not really have unique permanent names).  Therefore, this
+     *      is going to have to change in the future and ideally we
+     *      want quorum to be covered by the generic implementation.
+     */
+
+    children_list = qlist_new();
+    qdict_put(target, "children", children_list);
+
+    for (i = 0; i < s->num_children; i++) {
+        QINCREF(s->children[i]->bs->full_open_options);
+        qlist_append(children_list, s->children[i]->bs->full_open_options);
+    }
+}
+
 static char *quorum_dirname(BlockDriverState *bs, Error **errp)
 {
     /* In general, there are multiple BDSs with different dirnames below this
@@ -1123,6 +1160,7 @@ static BlockDriver bdrv_quorum = {
     .bdrv_file_open                     = quorum_open,
     .bdrv_close                         = quorum_close,
     .bdrv_refresh_filename              = quorum_refresh_filename,
+    .bdrv_gather_child_options          = quorum_gather_child_options,
     .bdrv_dirname                       = quorum_dirname,
 
     .bdrv_co_flush_to_disk              = quorum_co_flush,
block/vmdk.c +13 -0
index c1dbcfe133..dffab90a8b 100644
@@ -2327,6 +2327,18 @@ static int vmdk_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
     return 0;
 }
 
+static void vmdk_gather_child_options(BlockDriverState *bs, QDict *target)
+{
+    /* No children but file and backing can be explicitly specified (TODO) */
+    QINCREF(bs->file->bs->full_open_options);
+    qdict_put(target, "file", bs->file->bs->full_open_options);
+
+    if (bs->backing && bs->backing_overridden) {
+        QINCREF(bs->backing->bs->full_open_options);
+        qdict_put(target, "backing", bs->backing->bs->full_open_options);
+    }
+}
+
 static QemuOptsList vmdk_create_opts = {
     .name = "vmdk-create-opts",
     .head = QTAILQ_HEAD_INITIALIZER(vmdk_create_opts.head),
@@ -2397,6 +2409,7 @@ static BlockDriver bdrv_vmdk = {
     .bdrv_get_specific_info       = vmdk_get_specific_info,
     .bdrv_refresh_limits          = vmdk_refresh_limits,
     .bdrv_get_info                = vmdk_get_info,
+    .bdrv_gather_child_options    = vmdk_gather_child_options,
 
     .supports_backing             = true,
     .create_opts                  = &vmdk_create_opts,
Signature · 1 line
2.13.6
Reviewed-by on v7 19/25
#28 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 19/25] block: Add BlockDriver.bdrv_gather_child_options

Quoted 19 lines · Max Reitz

Some follow-up patches will rework the way bs->full_open_options is refreshed in bdrv_refresh_filename(). The new implementation will remove the need for the block drivers' bdrv_refresh_filename() implementations to set bs->full_open_options; instead, it will be generic and use static information from each block driver.

However, by implementing bdrv_gather_child_options(), block drivers will still be able to override the way the full_open_options of their children are incorporated into their own.

We need to implement this function for VMDK because we have to prevent the generic implementation from gathering the options of all children:
It is not possible to specify options for the extents through the runtime options.

For quorum, the child names that would be used by the generic implementation and the ones that we actually (currently) want to use differ. See quorum_gather_child_options() for more information.

Quoted 6 lines

Note that both of these are cases which are not ideal: In case of VMDK it would probably be nice to be able to specify options for all extents. In case of quorum, the current runtime option structure is simply broken and needs to be fixed (but that is left for another patch).

Berto

author
#29 lore raw

PATCH v7 20/25 block: Generically refresh runtime options

In reply to Max Reitz

Instead of having every block driver which implements bdrv_refresh_filename() copy all of the significant runtime options over to bs->full_open_options, implement this process generically in bdrv_refresh_filename().

This patch only adds this new generic implementation, it does not remove the old functionality. This is done in a follow-up patch.

With this patch, some superfluous information (that should never have been there) may be removed from some JSON filenames, as can be seen in the change to iotest 110's reference output. In case of 191, backing nodes that have not been overridden are now removed from the filename.

---
 block.c                    | 116 ++++++++++++++++++++++++++++++++++++++++++++-
 tests/qemu-iotests/110.out |   2 +-
 tests/qemu-iotests/191.out |  24 +++++-----
 3 files changed, 128 insertions(+), 14 deletions(-)
block.c +115 -1
index 2e59e77f9d..64ef30bbac 100644
@@ -4944,6 +4944,92 @@ out:
     return to_replace_bs;
 }
 
+/**
+ * Iterates through the list of runtime option keys that are said to be
+ * "significant" for a BDS. An option is called "significant" if it changes a
+ * BDS's data. For example, the null block driver's "size" and "read-zeroes"
+ * options are significant, but its "latency-ns" option is not.
+ *
+ * If a key returned by this function ends with a dot, all options starting with
+ * that prefix are significant.
+ */
+static const char *const *significant_options(BlockDriverState *bs,
+                                              const char *const *curopt)
+{
+    static const char *const global_options[] = {
+        "driver", "filename", "base-directory", NULL
+    };
+
+    if (!curopt) {
+        return &global_options[0];
+    }
+
+    curopt++;
+    if (curopt == &global_options[ARRAY_SIZE(global_options) - 1] && bs->drv) {
+        curopt = bs->drv->sgfnt_runtime_opts;
+    }
+
+    return (curopt && *curopt) ? curopt : NULL;
+}
+
+/**
+ * Copies all significant runtime options from bs->options to the given QDict.
+ * The set of significant option keys is determined by invoking
+ * significant_options().
+ *
+ * Returns true iff any significant option was present in bs->options (and thus
+ * copied to the target QDict) with the exception of "filename" and "driver".
+ * The caller is expected to use this value to decide whether the existence of
+ * significant options prevents the generation of a plain filename.
+ */
+static bool append_significant_runtime_options(QDict *d, BlockDriverState *bs)
+{
+    bool found_any = false;
+    const char *const *option_name = NULL;
+
+    if (!bs->drv) {
+        return false;
+    }
+
+    while ((option_name = significant_options(bs, option_name))) {
+        bool option_given = false;
+
+        assert(strlen(*option_name) > 0);
+        if ((*option_name)[strlen(*option_name) - 1] != '.') {
+            QObject *entry = qdict_get(bs->options, *option_name);
+            if (!entry) {
+                continue;
+            }
+
+            qobject_incref(entry);
+            qdict_put_obj(d, *option_name, entry);
+            option_given = true;
+        } else {
+            const QDictEntry *entry;
+            for (entry = qdict_first(bs->options); entry;
+                 entry = qdict_next(bs->options, entry))
+            {
+                if (strstart(qdict_entry_key(entry), *option_name, NULL)) {
+                    qobject_incref(qdict_entry_value(entry));
+                    qdict_put_obj(d, qdict_entry_key(entry),
+                                  qdict_entry_value(entry));
+                    option_given = true;
+                }
+            }
+        }
+
+        /* While "driver" and "filename" need to be included in a JSON filename,
+         * their existence does not prohibit generation of a plain filename. */
+        if (!found_any && option_given &&
+            strcmp(*option_name, "driver") && strcmp(*option_name, "filename"))
+        {
+            found_any = true;
+        }
+    }
+
+    return found_any;
+}
+
 static bool append_open_options(QDict *d, BlockDriverState *bs)
 {
     const QDictEntry *entry;
@@ -5098,9 +5184,37 @@ void bdrv_refresh_filename(BlockDriverState *bs)
         bs->full_open_options = opts;
     }
 
+    /* Gather the options QDict */
+    opts = qdict_new();
+    append_significant_runtime_options(opts, bs);
+
+    if (drv->bdrv_gather_child_options) {
+        /* Some block drivers may not want to present all of their children's
+         * options, or name them differently from BdrvChild.name */
+        drv->bdrv_gather_child_options(bs, opts);
+    } else {
+        QLIST_FOREACH(child, &bs->children, next) {
+            if (child->role == &child_backing && !bs->backing_overridden) {
+                /* We can skip the backing BDS if it has not been overridden */
+                continue;
+            }
+
+            QINCREF(child->bs->full_open_options);
+            qdict_put(opts, child->name, child->bs->full_open_options);
+        }
+
+        if (bs->backing_overridden && !bs->backing) {
+            /* Force no backing file */
+            qdict_put(opts, "backing", qstring_new());
+        }
+    }
+
+    QDECREF(bs->full_open_options);
+    bs->full_open_options = opts;
+
     if (bs->exact_filename[0]) {
         pstrcpy(bs->filename, sizeof(bs->filename), bs->exact_filename);
-    } else if (bs->full_open_options) {
+    } else {
         QString *json = qobject_to_json(QOBJECT(bs->full_open_options));
         snprintf(bs->filename, sizeof(bs->filename), "json:%s",
                  qstring_get_str(json));
tests/qemu-iotests/110.out +1 -1
index 1d0b2475cc..46e6a60510 100644
@@ -22,7 +22,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=t.IMGFMT.b
 
 === Nodes without a common directory ===
 
-image: json:{"driver": "IMGFMT", "file": {"children": [{"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.copy"}], "driver": "quorum", "blkverify": false, "rewrite-corrupted": false, "vote-threshold": 1}}
+image: json:{"driver": "IMGFMT", "file": {"children": [{"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.copy"}], "driver": "quorum", "vote-threshold": 1}}
 file format: IMGFMT
 virtual size: 64M (67108864 bytes)
 backing file: t.IMGFMT.base (cannot determine actual path)
tests/qemu-iotests/191.out +12 -12
index c1ce99985e..7e42b45ddb 100644
@@ -61,7 +61,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+                "filename": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -95,7 +95,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+            "file": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
             "encryption_key_missing": false
         },
         {
@@ -150,7 +150,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}",
+                "filename": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -184,7 +184,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}",
+            "file": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}",
             "encryption_key_missing": false
         },
         {
@@ -464,7 +464,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+                "filename": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -498,7 +498,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+            "file": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
             "encryption_key_missing": false
         },
         {
@@ -554,7 +554,7 @@ wrote 65536/65536 bytes at offset 1048576
                     },
                     "backing-filename-format": "qcow2",
                     "virtual-size": 67108864,
-                    "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+                    "filename": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                     "cluster-size": 65536,
                     "format": "qcow2",
                     "actual-size": SIZE,
@@ -573,7 +573,7 @@ wrote 65536/65536 bytes at offset 1048576
                 },
                 "backing-filename-format": "qcow2",
                 "virtual-size": 67108864,
-                "filename": "json:{"backing": {"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl3"}}",
+                "filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl3"}}",
                 "cluster-size": 65536,
                 "format": "qcow2",
                 "actual-size": SIZE,
@@ -586,8 +586,8 @@ wrote 65536/65536 bytes at offset 1048576
                         "corrupt": false
                     }
                 },
-                "full-backing-filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
-                "backing-filename": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+                "full-backing-filename": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+                "backing-filename": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
                 "dirty-flag": false
             },
             "iops_wr": 0,
@@ -598,7 +598,7 @@ wrote 65536/65536 bytes at offset 1048576
             "iops": 0,
             "bps_wr": 0,
             "write_threshold": 0,
-            "backing_file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
+            "backing_file": "json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}",
             "encrypted": false,
             "bps": 0,
             "bps_rd": 0,
@@ -607,7 +607,7 @@ wrote 65536/65536 bytes at offset 1048576
                 "direct": false,
                 "writeback": true
             },
-            "file": "json:{"backing": {"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.base"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl3"}}",
+            "file": "json:{"backing": {"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.mid"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2.ovl3"}}",
             "encryption_key_missing": false
         },
         {
Signature · 1 line
2.13.6
Reviewed-by on v7 20/25
#30 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 20/25] block: Generically refresh runtime options

Quoted 16 lines · Max Reitz

Instead of having every block driver which implements bdrv_refresh_filename() copy all of the significant runtime options over to bs->full_open_options, implement this process generically in bdrv_refresh_filename().

This patch only adds this new generic implementation, it does not remove the old functionality. This is done in a follow-up patch.

With this patch, some superfluous information (that should never have been there) may be removed from some JSON filenames, as can be seen in the change to iotest 110's reference output. In case of 191, backing nodes that have not been overridden are now removed from the filename.

block.c out: · line 4961

Quoted 6 lines
block.c +6 -0
+static const char *const *significant_options(BlockDriverState *bs,
+                                              const char *const *curopt)
+{
+    static const char *const global_options[] = {
+        "driver", "filename", "base-directory", NULL
+    };

You forgot to remove "base-directory" here from v6, but else the patch looks good.

Berto

author
#31 lore raw

PATCH v7 21/25 block: Purify .bdrv_refresh_filename()

In reply to Max Reitz

Currently, BlockDriver.bdrv_refresh_filename() is supposed to both refresh the filename (BDS.exact_filename) and set BDS.full_open_options. Now that we have generic code in the central bdrv_refresh_filename() for creating BDS.full_open_options, we can drop the latter part from all BlockDriver.bdrv_refresh_filename() implementations.

This also means that we can drop all of the existing default code for this from the global bdrv_refresh_filename() itself.

Furthermore, we now have to call BlockDriver.bdrv_refresh_filename() after having set BDS.full_open_options, because the block driver's implementation should now be allowed to depend on BDS.full_open_options being set correctly.

Finally, with this patch we can drop the @options parameter from BlockDriver.bdrv_refresh_filename(); also, add a comment on this function's purpose in block/block_int.h while touching its interface.

---
 include/block/block_int.h |   6 +-
 block.c                   | 145 +++++++---------------------------------------
 block/blkdebug.c          |  52 ++++++-----------
 block/blkverify.c         |  16 +----
 block/commit.c            |   2 +-
 block/mirror.c            |   2 +-
 block/nbd.c               |  23 +-------
 block/nfs.c               |  36 +-----------
 block/null.c              |  23 +++++---
 block/quorum.c            |  30 ----------
 10 files changed, 63 insertions(+), 272 deletions(-)
include/block/block_int.h +5 -1
index f81516b615..b67f3af599 100644
@@ -133,7 +133,11 @@ struct BlockDriver {
     int (*bdrv_create)(const char *filename, QemuOpts *opts, Error **errp);
     int (*bdrv_make_empty)(BlockDriverState *bs);
 
-    void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options);
+    /*
+     * Refreshes the bs->exact_filename field. If that is impossible,
+     * bs->exact_filename has to be left empty.
+     */
+    void (*bdrv_refresh_filename)(BlockDriverState *bs);
 
     /*
      * Gathers the open options for all children into @target.
block.c +22 -123

171 lines not on this page · raw message

block/blkdebug.c +17 -35

69 lines not on this page · raw message

block/blkverify.c +1 -15
index d5233eeaf9..7da2e08a06 100644
@@ -277,24 +277,10 @@ static bool blkverify_recurse_is_first_non_filter(BlockDriverState *bs,
     return bdrv_recurse_is_first_non_filter(s->test_file->bs, candidate);
 }
 
-static void blkverify_refresh_filename(BlockDriverState *bs, QDict *options)
+static void blkverify_refresh_filename(BlockDriverState *bs)
 {
     BDRVBlkverifyState *s = bs->opaque;
 
-    if (bs->file->bs->full_open_options
-        && s->test_file->bs->full_open_options)
-    {
-        QDict *opts = qdict_new();
-        qdict_put_str(opts, "driver", "blkverify");
-
-        QINCREF(bs->file->bs->full_open_options);
-        qdict_put(opts, "raw", bs->file->bs->full_open_options);
-        QINCREF(s->test_file->bs->full_open_options);
-        qdict_put(opts, "test", s->test_file->bs->full_open_options);
-
-        bs->full_open_options = opts;
-    }
-
     if (bs->file->bs->exact_filename[0]
         && s->test_file->bs->exact_filename[0])
     {
block/commit.c +1 -1
index cb529cc2aa..c0d1d749e5 100644
@@ -239,7 +239,7 @@ static int coroutine_fn bdrv_commit_top_preadv(BlockDriverState *bs,
     return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);
 }
 
-static void bdrv_commit_top_refresh_filename(BlockDriverState *bs, QDict *opts)
+static void bdrv_commit_top_refresh_filename(BlockDriverState *bs)
 {
     pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
             bs->backing->bs->filename);
block/mirror.c +1 -1
index d6e487fb2e..718c492ded 100644
@@ -1058,7 +1058,7 @@ static int coroutine_fn bdrv_mirror_top_pdiscard(BlockDriverState *bs,
     return bdrv_co_pdiscard(bs->backing->bs, offset, bytes);
 }
 
-static void bdrv_mirror_top_refresh_filename(BlockDriverState *bs, QDict *opts)
+static void bdrv_mirror_top_refresh_filename(BlockDriverState *bs)
 {
     if (bs->backing == NULL) {
         /* we can be here after failed bdrv_attach_child in
block/nbd.c +1 -22
index 0a36394b73..9ce9fad7ea 100644
@@ -514,12 +514,9 @@ static void nbd_attach_aio_context(BlockDriverState *bs,
     nbd_client_attach_aio_context(bs, new_context);
 }
 
-static void nbd_refresh_filename(BlockDriverState *bs, QDict *options)
+static void nbd_refresh_filename(BlockDriverState *bs)
 {
     BDRVNBDState *s = bs->opaque;
-    QDict *opts = qdict_new();
-    QObject *saddr_qdict;
-    Visitor *ov;
     const char *host = NULL, *port = NULL, *path = NULL;
 
     if (s->saddr->type == SOCKET_ADDRESS_TYPE_INET) {
@@ -532,8 +529,6 @@ static void nbd_refresh_filename(BlockDriverState *bs, QDict *options)
         path = s->saddr->u.q_unix.path;
     } /* else can't represent as pseudo-filename */
 
-    qdict_put_str(opts, "driver", "nbd");
-
     if (path && s->export) {
         snprintf(bs->exact_filename, sizeof(bs->exact_filename),
                  "nbd+unix:///%s?socket=%s", s->export, path);
@@ -547,22 +542,6 @@ static void nbd_refresh_filename(BlockDriverState *bs, QDict *options)
         snprintf(bs->exact_filename, sizeof(bs->exact_filename),
                  "nbd://%s:%s", host, port);
     }
-
-    ov = qobject_output_visitor_new(&saddr_qdict);
-    visit_type_SocketAddress(ov, NULL, &s->saddr, &error_abort);
-    visit_complete(ov, &saddr_qdict);
-    visit_free(ov);
-    qdict_put_obj(opts, "server", saddr_qdict);
-
-    if (s->export) {
-        qdict_put_str(opts, "export", s->export);
-    }
-    if (s->tlscredsid) {
-        qdict_put_str(opts, "tls-creds", s->tlscredsid);
-    }
-
-    qdict_flatten(opts);
-    bs->full_open_options = opts;
 }
 
 static char *nbd_dirname(BlockDriverState *bs, Error **errp)
block/nfs.c +1 -35
index 445a8d9880..250e01167d 100644
@@ -819,14 +819,9 @@ static int nfs_reopen_prepare(BDRVReopenState *state,
     return 0;
 }
 
-static void nfs_refresh_filename(BlockDriverState *bs, QDict *options)
+static void nfs_refresh_filename(BlockDriverState *bs)
 {
     NFSClient *client = bs->opaque;
-    QDict *opts = qdict_new();
-    QObject *server_qdict;
-    Visitor *ov;
-
-    qdict_put_str(opts, "driver", "nfs");
 
     if (client->uid && !client->gid) {
         snprintf(bs->exact_filename, sizeof(bs->exact_filename),
@@ -844,35 +839,6 @@ static void nfs_refresh_filename(BlockDriverState *bs, QDict *options)
         snprintf(bs->exact_filename, sizeof(bs->exact_filename),
                  "nfs://%s%s", client->server->host, client->path);
     }
-
-    ov = qobject_output_visitor_new(&server_qdict);
-    visit_type_NFSServer(ov, NULL, &client->server, &error_abort);
-    visit_complete(ov, &server_qdict);
-    qdict_put_obj(opts, "server", server_qdict);
-    qdict_put_str(opts, "path", client->path);
-
-    if (client->uid) {
-        qdict_put_int(opts, "user", client->uid);
-    }
-    if (client->gid) {
-        qdict_put_int(opts, "group", client->gid);
-    }
-    if (client->tcp_syncnt) {
-        qdict_put_int(opts, "tcp-syn-cnt", client->tcp_syncnt);
-    }
-    if (client->readahead) {
-        qdict_put_int(opts, "readahead-size", client->readahead);
-    }
-    if (client->pagecache) {
-        qdict_put_int(opts, "page-cache-size", client->pagecache);
-    }
-    if (client->debug) {
-        qdict_put_int(opts, "debug", client->debug);
-    }
-
-    visit_free(ov);
-    qdict_flatten(opts);
-    bs->full_open_options = opts;
 }
 
 static char *nfs_dirname(BlockDriverState *bs, Error **errp)
block/null.c +14 -9
index 59673d2449..65f5d681c0 100644
@@ -241,18 +241,23 @@ static int64_t coroutine_fn null_co_get_block_status(BlockDriverState *bs,
     }
 }
 
-static void null_refresh_filename(BlockDriverState *bs, QDict *opts)
+static void null_refresh_filename(BlockDriverState *bs)
 {
-    QINCREF(opts);
-    qdict_del(opts, "filename");
-
-    if (!qdict_size(opts)) {
-        snprintf(bs->exact_filename, sizeof(bs->exact_filename), "%s://",
-                 bs->drv->format_name);
+    const QDictEntry *e;
+
+    for (e = qdict_first(bs->full_open_options); e;
+         e = qdict_next(bs->full_open_options, e))
+    {
+        /* These options can be ignored */
+        if (strcmp(qdict_entry_key(e), "filename") &&
+            strcmp(qdict_entry_key(e), "driver"))
+        {
+            return;
+        }
     }
 
-    qdict_put_str(opts, "driver", bs->drv->format_name);
-    bs->full_open_options = opts;
+    snprintf(bs->exact_filename, sizeof(bs->exact_filename), "%s://",
+             bs->drv->format_name);
 }
 
 static const char *const null_sgfnt_runtime_opts[] = {
block/quorum.c +0 -30
index 9d680addc6..7e8cccd60b 100644
@@ -1066,35 +1066,6 @@ static void quorum_del_child(BlockDriverState *bs, BdrvChild *child,
     bdrv_drained_end(bs);
 }
 
-static void quorum_refresh_filename(BlockDriverState *bs, QDict *options)
-{
-    BDRVQuorumState *s = bs->opaque;
-    QDict *opts;
-    QList *children;
-    int i;
-
-    for (i = 0; i < s->num_children; i++) {
-        if (!s->children[i]->bs->full_open_options) {
-            return;
-        }
-    }
-
-    children = qlist_new();
-    for (i = 0; i < s->num_children; i++) {
-        QINCREF(s->children[i]->bs->full_open_options);
-        qlist_append(children, s->children[i]->bs->full_open_options);
-    }
-
-    opts = qdict_new();
-    qdict_put_str(opts, "driver", "quorum");
-    qdict_put_int(opts, QUORUM_OPT_VOTE_THRESHOLD, s->threshold);
-    qdict_put_bool(opts, QUORUM_OPT_BLKVERIFY, s->is_blkverify);
-    qdict_put_bool(opts, QUORUM_OPT_REWRITE, s->rewrite_corrupted);
-    qdict_put(opts, "children", children);
-
-    bs->full_open_options = opts;
-}
-
 static void quorum_gather_child_options(BlockDriverState *bs, QDict *target)
 {
     BDRVQuorumState *s = bs->opaque;
@@ -1159,7 +1130,6 @@ static BlockDriver bdrv_quorum = {
 
     .bdrv_file_open                     = quorum_open,
     .bdrv_close                         = quorum_close,
-    .bdrv_refresh_filename              = quorum_refresh_filename,
     .bdrv_gather_child_options          = quorum_gather_child_options,
     .bdrv_dirname                       = quorum_dirname,
 
Signature · 1 line
2.13.6
#32 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 21/25] block: Purify .bdrv_refresh_filename()

Quoted 27 lines · Max Reitz
block/blkdebug.c +3 -17
-static void blkdebug_refresh_filename(BlockDriverState *bs, QDict *options)
+static void blkdebug_refresh_filename(BlockDriverState *bs)
 {
     BDRVBlkdebugState *s = bs->opaque;
-    QDict *opts;
     const QDictEntry *e;
-    bool force_json = false;
-
-    for (e = qdict_first(options); e; e = qdict_next(options, e)) {
-        if (strcmp(qdict_entry_key(e), "config") &&
-            strcmp(qdict_entry_key(e), "x-image"))
-        {
-            force_json = true;
-            break;
-        }
-    }
+    int ret;
 
-    if (force_json && !bs->file->bs->full_open_options) {
-        /* The config file cannot be recreated, so creating a plain filename
-         * is impossible */
+    if (!bs->file->bs->exact_filename[0]) {
         return;
     }
 
-    if (!force_json && bs->file->bs->exact_filename[0]) {
-        int ret = snprintf(bs->exact_filename, sizeof(bs->exact_filename),

block/blkdebug.c static int64_t blkdebug_getlength(BlockDriverState *bs) · line 825

Quoted 10 lines
block/blkdebug.c +5 -5
-                           "blkdebug:%s:%s", s->config_file ?: "",
-                           bs->file->bs->exact_filename);
-        if (ret >= sizeof(bs->exact_filename)) {
-            /* An overflow makes the filename unusable, so do not report any */
-            bs->exact_filename[0] = 0;
+    for (e = qdict_first(bs->full_open_options); e;
+         e = qdict_next(bs->full_open_options, e))
+    {
+        if (strcmp(qdict_entry_key(e), "config") &&
+            strcmp(qdict_entry_key(e), "image") &&

Shouldn't this be "x-image" ?

Berto

author
#33 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 21/25] block: Purify .bdrv_refresh_filename()

Quoted 26 lines · Alberto Garcia
Quoted 25 lines · Max Reitz
block/blkdebug.c +3 -15
-static void blkdebug_refresh_filename(BlockDriverState *bs, QDict *options)
+static void blkdebug_refresh_filename(BlockDriverState *bs)
 {
     BDRVBlkdebugState *s = bs->opaque;
-    QDict *opts;
     const QDictEntry *e;
-    bool force_json = false;
-
-    for (e = qdict_first(options); e; e = qdict_next(options, e)) {
-        if (strcmp(qdict_entry_key(e), "config") &&
-            strcmp(qdict_entry_key(e), "x-image"))
-        {
-            force_json = true;
-            break;
-        }
-    }
+    int ret;
 
-    if (force_json && !bs->file->bs->full_open_options) {
-        /* The config file cannot be recreated, so creating a plain filename
-         * is impossible */
+    if (!bs->file->bs->exact_filename[0]) {
         return;
     }
Quoted 14 lines

block/blkdebug.c static int64_t blkdebug_getlength(BlockDriverState *bs) · line 825

Quoted 12 lines
block/blkdebug.c +5 -7
-    if (!force_json && bs->file->bs->exact_filename[0]) {
-        int ret = snprintf(bs->exact_filename, sizeof(bs->exact_filename),
-                           "blkdebug:%s:%s", s->config_file ?: "",
-                           bs->file->bs->exact_filename);
-        if (ret >= sizeof(bs->exact_filename)) {
-            /* An overflow makes the filename unusable, so do not report any */
-            bs->exact_filename[0] = 0;
+    for (e = qdict_first(bs->full_open_options); e;
+         e = qdict_next(bs->full_open_options, e))
+    {
+        if (strcmp(qdict_entry_key(e), "config") &&
+            strcmp(qdict_entry_key(e), "image") &&

Shouldn't this be "x-image" ?

Er, yes. It should.

Max

author
#34 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 21/25] block: Purify .bdrv_refresh_filename()

Quoted 42 lines · Max Reitz
Quoted 26 lines · Alberto Garcia
Quoted 25 lines · Max Reitz
block/blkdebug.c +3 -15
-static void blkdebug_refresh_filename(BlockDriverState *bs, QDict *options)
+static void blkdebug_refresh_filename(BlockDriverState *bs)
 {
     BDRVBlkdebugState *s = bs->opaque;
-    QDict *opts;
     const QDictEntry *e;
-    bool force_json = false;
-
-    for (e = qdict_first(options); e; e = qdict_next(options, e)) {
-        if (strcmp(qdict_entry_key(e), "config") &&
-            strcmp(qdict_entry_key(e), "x-image"))
-        {
-            force_json = true;
-            break;
-        }
-    }
+    int ret;
 
-    if (force_json && !bs->file->bs->full_open_options) {
-        /* The config file cannot be recreated, so creating a plain filename
-         * is impossible */
+    if (!bs->file->bs->exact_filename[0]) {
         return;
     }
Quoted 14 lines

block/blkdebug.c static int64_t blkdebug_getlength(BlockDriverState *bs) · line 825

Quoted 12 lines
block/blkdebug.c +5 -7
-    if (!force_json && bs->file->bs->exact_filename[0]) {
-        int ret = snprintf(bs->exact_filename, sizeof(bs->exact_filename),
-                           "blkdebug:%s:%s", s->config_file ?: "",
-                           bs->file->bs->exact_filename);
-        if (ret >= sizeof(bs->exact_filename)) {
-            /* An overflow makes the filename unusable, so do not report any */
-            bs->exact_filename[0] = 0;
+    for (e = qdict_first(bs->full_open_options); e;
+         e = qdict_next(bs->full_open_options, e))
+    {
+        if (strcmp(qdict_entry_key(e), "config") &&
+            strcmp(qdict_entry_key(e), "image") &&

Shouldn't this be "x-image" ?

Quoted 1 line

Er, yes. It should.

Actually, it should be both. That's because the child is attached as "image" and not "x-image", so when the child options are gathered, they are put under "image".

And since the child is attached using bdrv_open_child(), you have to specify all child options in an "image" sub qdict, too (as can be seen in iotest 099), so this is indeed correct. (Btw, note that the old code already put these options under "image".)

(So with "x-image" instead of "image", iotest 162 fails.)

Of course, x-image can be specified, too (although I wouldn't really mind breaking that for users...), so we have to ignore that, still.

Before this patch, we could ignore "image" because we iterated over the options before they were newly generated. Now they are generated automatically before this function is called, so there may be an "image" key now.

x-image

author
#35 lore raw

PATCH v7 22/25 block: Do not copy exact_filename from format file

In reply to Max Reitz

If the a format BDS's file BDS is in turn a format BDS, we cannot simply use the same filename, because when opening a BDS tree based on a filename alone, qemu will create only one format node on top of one protocol node (disregarding a potential backing file).

---
 block.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
block.c +14 -3
index 275cae460a..526ce43e9b 100644
@@ -5104,9 +5104,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
         bs->exact_filename[0] = '\0';
 
-        /* If no specific options have been given for this BDS, the filename of
-         * the underlying file should suffice for this one as well */
-        if (bs->file->bs->exact_filename[0] && !generate_json_filename) {
+        /* We can use the underlying file's filename if:
+         * - it has a filename,
+         * - the file is a protocol BDS, and
+         * - opening that file (as this BDS's format) will automatically create
+         *   the BDS tree we have right now, that is:
+         *   - the user did not significantly change this BDS's behavior with
+         *     some explicit options
+         *   - no non-file child of this BDS has been overridden by the user
+         *   Both of these conditions are represented by generate_json_filename.
+         */
+        if (bs->file->bs->exact_filename[0] &&
+            bs->file->bs->drv->bdrv_file_open &&
+            !generate_json_filename)
+        {
             strcpy(bs->exact_filename, bs->file->bs->exact_filename);
         }
     }
Signature · 1 line
2.13.6
Reviewed-by on v7 22/25
#36 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 22/25] block: Do not copy exact_filename from format file

Quoted 6 lines · Max Reitz

If the a format BDS's file BDS is in turn a format BDS, we cannot simply use the same filename, because when opening a BDS tree based on a filename alone, qemu will create only one format node on top of one protocol node (disregarding a potential backing file).

Berto

author
#37 lore raw

PATCH v7 23/25 block: Fix FIXME from "Add BDS.backing_overridden"

In reply to Max Reitz

Said commit introduced a FIXME stating that bdrv_open_backing_file() should set bs->backing_overridden to true not only if the file.filename option was set, but if the "options" QDict contained any option that is significant for any node in the BDS tree emerging from the backing BDS. This behavior is implemented by this patch.

---
 block.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 75 insertions(+), 6 deletions(-)
block.c +75 -6
index 526ce43e9b..40007544e5 100644
@@ -73,6 +73,9 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,
                                            const BdrvChildRole *child_role,
                                            Error **errp);
 
+static bool has_significant_runtime_options(BlockDriverState *bs,
+                                            const QDict *d);
+
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
 
@@ -2214,6 +2217,42 @@ out:
     bdrv_refresh_limits(bs, NULL);
 }
 
+/**
+ * Checks whether @options contains any significant option for any of the nodes
+ * in the BDS tree emerging from @bs.
+ */
+static bool is_significant_option_tree(BlockDriverState *bs, QDict *options)
+{
+    BdrvChild *child;
+
+    if (!qdict_size(options)) {
+        /* No need to recurse */
+        return false;
+    }
+
+    if (has_significant_runtime_options(bs, options)) {
+        return true;
+    }
+
+    QLIST_FOREACH(child, &bs->children, next) {
+        QDict *child_options;
+        char *option_prefix;
+
+        option_prefix = g_strdup_printf("%s.", child->name);
+        qdict_extract_subqdict(options, &child_options, option_prefix);
+        g_free(option_prefix);
+
+        if (is_significant_option_tree(child->bs, child_options)) {
+            QDECREF(child_options);
+            return true;
+        }
+
+        QDECREF(child_options);
+    }
+
+    return false;
+}
+
 /*
  * Opens the backing file for a BlockDriverState if not yet open
  *
@@ -2232,7 +2271,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
     const char *reference = NULL;
     int ret = 0;
     BlockDriverState *backing_hd;
-    QDict *options;
+    QDict *options, *cloned_options = NULL;
     QDict *tmp_parent_options = NULL;
     Error *local_err = NULL;
 
@@ -2262,11 +2301,6 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
     reference = qdict_get_try_str(parent_options, bdref_key);
     if (reference || qdict_haskey(options, "file.filename")) {
         /* keep backing_filename NULL */
-
-        /* FIXME: Should also be set to true if @options contains other runtime
-         *        options which control the data that is read from the backing
-         *        BDS */
-        bs->backing_overridden = true;
     } else if (bs->backing_file[0] == '\0' && qdict_size(options) == 0) {
         QDECREF(options);
         goto free_exit;
@@ -2287,6 +2321,8 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
         goto free_exit;
     }
 
+    cloned_options = qdict_clone_shallow(options);
+
     if (!reference &&
         bs->backing_format[0] != '\0' && !qdict_haskey(options, "driver")) {
         qdict_put_str(options, "driver", bs->backing_format);
@@ -2302,6 +2338,10 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
     }
     bdrv_set_aio_context(backing_hd, bdrv_get_aio_context(bs));
 
+    if (reference || is_significant_option_tree(backing_hd, cloned_options)) {
+        bs->backing_overridden = true;
+    }
+
     /* Hook up the backing file link; drop our reference, bs owns the
      * backing_hd reference now */
     bdrv_set_backing_hd(bs, backing_hd, &local_err);
@@ -2317,6 +2357,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
 free_exit:
     g_free(backing_filename);
     QDECREF(tmp_parent_options);
+    QDECREF(cloned_options);
     return ret;
 }
 
@@ -4973,6 +5014,34 @@ static const char *const *significant_options(BlockDriverState *bs,
 }
 
 /**
+ * Returns true if @d contains any options the block driver of @bs considers to
+ * be significant runtime options.
+ */
+static bool has_significant_runtime_options(BlockDriverState *bs,
+                                            const QDict *d)
+{
+    const char *const *option_name = NULL;
+
+    while ((option_name = significant_options(bs, option_name))) {
+        assert(strlen(*option_name) > 0);
+        if ((*option_name)[strlen(*option_name) - 1] != '.') {
+            if (qdict_haskey(d, *option_name)) {
+                return true;
+            }
+        } else {
+            const QDictEntry *entry;
+            for (entry = qdict_first(d); entry; entry = qdict_next(d, entry)) {
+                if (strstart(qdict_entry_key(entry), *option_name, NULL)) {
+                    return true;
+                }
+            }
+        }
+    }
+
+    return false;
+}
+
+/**
  * Copies all significant runtime options from bs->options to the given QDict.
  * The set of significant option keys is determined by invoking
  * significant_options().
Signature · 1 line
2.13.6
Reviewed-by on v7 23/25
#38 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 23/25] block: Fix FIXME from "Add BDS.backing_overridden"

Quoted 7 lines · Max Reitz

Said commit introduced a FIXME stating that bdrv_open_backing_file() should set bs->backing_overridden to true not only if the file.filename option was set, but if the "options" QDict contained any option that is significant for any node in the BDS tree emerging from the backing BDS. This behavior is implemented by this patch.

Berto

author
#39 lore raw

PATCH v7 24/25 block/curl: Implement bdrv_refresh_filename()

In reply to Max Reitz
---
 block/curl.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
block/curl.c +18 -0
index 11318a9a29..fe57223fda 100644
@@ -957,6 +957,20 @@ static int64_t curl_getlength(BlockDriverState *bs)
     return s->len;
 }
 
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {
+        return;
+    }
+
+    pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), s->url);
+}
+
+
 static const char *const curl_sgfnt_runtime_opts[] = {
     CURL_BLOCK_OPT_URL,
     CURL_BLOCK_OPT_SSLVERIFY,
@@ -985,6 +999,7 @@ static BlockDriver bdrv_http = {
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
 
+    .bdrv_refresh_filename      = curl_refresh_filename,
     .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
@@ -1003,6 +1018,7 @@ static BlockDriver bdrv_https = {
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
 
+    .bdrv_refresh_filename      = curl_refresh_filename,
     .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
@@ -1021,6 +1037,7 @@ static BlockDriver bdrv_ftp = {
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
 
+    .bdrv_refresh_filename      = curl_refresh_filename,
     .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
@@ -1039,6 +1056,7 @@ static BlockDriver bdrv_ftps = {
     .bdrv_detach_aio_context    = curl_detach_aio_context,
     .bdrv_attach_aio_context    = curl_attach_aio_context,
 
+    .bdrv_refresh_filename      = curl_refresh_filename,
     .sgfnt_runtime_opts         = curl_sgfnt_runtime_opts,
 };
 
Signature · 1 line
2.13.6
#40 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

Quoted 10 lines · Max Reitz
---
 block/curl.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
block/curl.c +0 -0
index 11318a9a29..fe57223fda 100644

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 10 lines
block/curl.c +7 -0
@@ -957,6 +957,20 @@ static int64_t curl_getlength(BlockDriverState *bs)
     return s->len;
 }
 
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Berto

author
#41 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

Quoted 14 lines · Alberto Garcia
Quoted 13 lines · Max Reitz
---
 block/curl.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
block/curl.c +0 -0
index 11318a9a29..fe57223fda 100644
@@ -957,6 +957,20 @@ static int64_t curl_getlength(BlockDriverState *bs)
     return s->len;
 }
 
Quoted 9 lines

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 7 lines
block/curl.c +7 -0
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Yes, exactly. If it's false, you'd need to override it (and you can't do that through a plain filename).

Thanks for reviewing, again. :-)

Max

#42 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

Quoted 10 lines · Max Reitz
Quoted 9 lines

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 7 lines
block/curl.c +7 -0
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Quoted 2 lines

Yes, exactly. If it's false, you'd need to override it (and you can't do that through a plain filename).

I think this is not the only case in this series, but I'm not very comfortable with the idea that this condition and the default value of the setting are implicity dependent on each other. If you change one and forget to change the other things will break.

I understand that the default value is never supposed to change so in practice I don't see this breaking, but is it perhaps worth adding tests for all these cases?

Berto

author
#43 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

Quoted 11 lines · Alberto Garcia
Quoted 10 lines · Max Reitz
Quoted 9 lines

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 7 lines
block/curl.c +7 -0
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Quoted 7 lines
Quoted 2 lines

Yes, exactly. If it's false, you'd need to override it (and you can't do that through a plain filename).

I think this is not the only case in this series, but I'm not very comfortable with the idea that this condition and the default value of the setting are implicity dependent on each other. If you change one and forget to change the other things will break.

Well, yes, but...

Quoted 2 lines

I understand that the default value is never supposed to change so in practice I don't see this breaking,

Yes.

Quoted 2 lines

but is it perhaps worth adding tests
for all these cases?

In theory, sure. In practice, adding a curl test case seems hard.

Well, I could connect to, I don't know, qemu.org or something and then just test things there (with the index used as a raw image), but if I add one test for the curl protocol, nobody is ever going to run them... And in theory, that's not how a curl test should work. In theory, you'd expect the user to set up a localhost server with some known root directory and then _make_test_img etc. would set up images there. But that way, really nobody is ever going to run them.

And just adding a test for all protocols that then accesses qemu.org feels somehow wrong, too... Maybe if I add it to a network group, that wouldn't feel so bad.

Also, adding macros for the default values could help, I think.

Max

#44 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

Quoted 32 lines · Max Reitz
Quoted 11 lines · Alberto Garcia
Quoted 10 lines · Max Reitz
Quoted 9 lines

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 7 lines
block/curl.c +7 -0
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Quoted 7 lines
Quoted 2 lines

Yes, exactly. If it's false, you'd need to override it (and you can't do that through a plain filename).

I think this is not the only case in this series, but I'm not very comfortable with the idea that this condition and the default value of the setting are implicity dependent on each other. If you change one and forget to change the other things will break.

Well, yes, but...

Quoted 2 lines

I understand that the default value is never supposed to change so in practice I don't see this breaking,

Yes.

Quoted 2 lines

but is it perhaps worth adding tests
for all these cases?

In theory, sure. In practice, adding a curl test case seems hard.

Quoted 7 lines

Well, I could connect to, I don't know, qemu.org or something and then just test things there (with the index used as a raw image), but if I add one test for the curl protocol, nobody is ever going to run them... And in theory, that's not how a curl test should work. In theory, you'd expect the user to set up a localhost server with some known root directory and then _make_test_img etc. would set up images there. But that way, really nobody is ever going to run them.

For qemu I/O tests I would suggest just having the iotest spin up a simple http server in Python - there's standard APIs in python that make this pretty trivial

http://www.pythonforbeginners.com/modules-in-python/how-to-use-simplehttpserver/

That way curl could be reliably tested without any special setup by the developer

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#45 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

Quoted 41 lines · Daniel P. Berrange
Quoted 32 lines · Max Reitz
Quoted 11 lines · Alberto Garcia
Quoted 10 lines · Max Reitz
Quoted 9 lines

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 7 lines
block/curl.c +7 -0
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Quoted 7 lines
Quoted 2 lines

Yes, exactly. If it's false, you'd need to override it (and you can't do that through a plain filename).

I think this is not the only case in this series, but I'm not very comfortable with the idea that this condition and the default value of the setting are implicity dependent on each other. If you change one and forget to change the other things will break.

Well, yes, but...

Quoted 2 lines

I understand that the default value is never supposed to change so in practice I don't see this breaking,

Yes.

Quoted 2 lines

but is it perhaps worth adding tests
for all these cases?

In theory, sure. In practice, adding a curl test case seems hard.

Quoted 7 lines

Well, I could connect to, I don't know, qemu.org or something and then just test things there (with the index used as a raw image), but if I add one test for the curl protocol, nobody is ever going to run them... And in theory, that's not how a curl test should work. In theory, you'd expect the user to set up a localhost server with some known root directory and then _make_test_img etc. would set up images there. But that way, really nobody is ever going to run them.

Quoted 8 lines

For qemu I/O tests I would suggest just having the iotest spin up a simple http server in Python - there's standard APIs in python that make this pretty trivial

http://www.pythonforbeginners.com/modules-in-python/how-to-use-simplehttpserver/

That way curl could be reliably tested without any special setup by the developer

Good idea, thanks. It might be an issue if we ever want to test HTTPS (putting a self-signed certificate into the iotests directory seems a bit over the top), but since the sslverify option works just fine with HTTP itself, that's not an issue for this case.

Now I just have to discuss with myself whether I want to add curl testing infrastructure for testing whether the defaults are still the defaults or whether adding default value macros is enough...

Max

#46 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

In reply to Max Reitz
Quoted 20 lines · Max Reitz
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 966

Quoted 7 lines
block/curl.c +7 -0
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {

Is !s->sslverify negative because that setting is true by default?

Quoted 7 lines
Quoted 2 lines

Yes, exactly. If it's false, you'd need to override it (and you can't do that through a plain filename).

I think this is not the only case in this series, but I'm not very comfortable with the idea that this condition and the default value of the setting are implicity dependent on each other. If you change one and forget to change the other things will break.

Well, yes, but...

Quoted 8 lines
Quoted 2 lines

I understand that the default value is never supposed to change so in practice I don't see this breaking,

Yes.

Quoted 1 line

but is it perhaps worth adding tests for all these cases?

In theory, sure. In practice, adding a curl test case seems hard.

Indeed, I though it would perhaps be possible to create a curl BDS to this without having to perform an actual connection, but never mind if it's not possible / too complicated.

Quoted 1 line

Also, adding macros for the default values could help, I think.

Yep.

Berto

Reviewed-by on v7 24/25
#47 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

In reply to Max Reitz
Quoted 15 lines · Max Reitz
---
 block/curl.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
block/curl.c +2 -0
index 11318a9a29..fe57223fda 100644
@@ -957,6 +957,20 @@ static int64_t curl_getlength(BlockDriverState *bs)
     return s->len;
 }
 
+static void curl_refresh_filename(BlockDriverState *bs)
+{

block/curl.c static int64_t curl_getlength(BlockDriverState *bs) · line 971

Quoted 10 lines
block/curl.c +10 -0
+    BDRVCURLState *s = bs->opaque;
+
+    if (!s->sslverify || s->cookie ||
+        s->username || s->password || s->proxyusername || s->proxypassword)
+    {
+        return;
+    }
+
+    pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), s->url);
+}

Berto

author
#48 lore raw

PATCH v7 25/25 block/null: Generate filename even with latency-ns

In reply to Max Reitz

While we cannot represent the latency-ns option in a filename, it is not a significant option so not being able to should not stop us from generating a filename nonetheless.

---
 block/null.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
block/null.c +2 -1
index 65f5d681c0..ca0aa3d528 100644
@@ -250,7 +250,8 @@ static void null_refresh_filename(BlockDriverState *bs)
     {
         /* These options can be ignored */
         if (strcmp(qdict_entry_key(e), "filename") &&
-            strcmp(qdict_entry_key(e), "driver"))
+            strcmp(qdict_entry_key(e), "driver") &&
+            strcmp(qdict_entry_key(e), NULL_OPT_LATENCY))
         {
             return;
         }
Signature · 1 line
2.13.6
Reviewed-by on v7 25/25
#49 lore raw

Re: [Qemu-devel] [PATCH v7 for-2.12 25/25] block/null: Generate filename even with latency-ns

Quoted 5 lines · Max Reitz

While we cannot represent the latency-ns option in a filename, it is not a significant option so not being able to should not stop us from generating a filename nonetheless.

Berto