Skip to contentSkip to conversation

block devices record/replay update

PATCH Superseded Pavel Dovgalyuk 3 participants 11 messages Jan 31, 2017 – Feb 27, 2017 Block layer

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover block devices record/replay update

This set of patches includes several fixes for preserving the state of the block device images while recording and replaying virtual machine execution.

blkreplay driver now creates temporary overlay for underlaying devices This patch implicitly enables '-snapshot' behavior in record/replay mode. blkreplay layer creates temporary overlays on top of underlaying disk images. It is needed, because creating an overlay over blkreplay with explicit '-snapshot' option breaks the determinism.

---

Pavel Dovgalyuk (3):
block: implement bdrv_snapshot_goto for blkreplay
blkreplay: create temporary overlay for underlaying devices
replay: disable default snapshot for record/replay

 block/blkreplay.c |   84 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 stubs/replay.c    |    1 +
 vl.c              |   10 +++++-
 3 files changed, 93 insertions(+), 2 deletions(-)
Signature · 1 line
Pavel Dovgalyuk
#1 lore raw

PATCH 1/3 block: implement bdrv_snapshot_goto for blkreplay

This patch enables making snapshots with blkreplay used in block devices.

---
 block/blkreplay.c |    8 ++++++++
 1 file changed, 8 insertions(+)
block/blkreplay.c +8 -0
index a741654..8a03d62 100644
@@ -130,6 +130,12 @@ static int coroutine_fn blkreplay_co_flush(BlockDriverState *bs)
     return ret;
 }
 
+static int blkreplay_snapshot_goto(BlockDriverState *bs,
+                                   const char *snapshot_id)
+{
+    return bdrv_snapshot_goto(bs->file->bs, snapshot_id);
+}
+
 static BlockDriver bdrv_blkreplay = {
     .format_name            = "blkreplay",
     .protocol_name          = "blkreplay",
@@ -145,6 +151,8 @@ static BlockDriver bdrv_blkreplay = {
     .bdrv_co_pwrite_zeroes  = blkreplay_co_pwrite_zeroes,
     .bdrv_co_pdiscard       = blkreplay_co_pdiscard,
     .bdrv_co_flush          = blkreplay_co_flush,
+
+    .bdrv_snapshot_goto     = blkreplay_snapshot_goto,
 };
 
 static void bdrv_blkreplay_init(void)
#5 lore raw

PATCH 2/3 blkreplay: create temporary overlay for underlaying devices

In reply to Pavel Dovgalyuk

This patch allows using '-snapshot' behavior in record/replay mode. blkreplay layer creates temporary overlays on top of underlaying disk images. It is needed, because creating an overlay over blkreplay breaks the determinism.

---
 block/blkreplay.c |   76 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 stubs/replay.c    |    1 +
 vl.c              |    2 +
 3 files changed, 78 insertions(+), 1 deletion(-)
block/blkreplay.c +76 -0
index 8a03d62..172642f 100644
@@ -14,12 +14,76 @@
 #include "block/block_int.h"
 #include "sysemu/replay.h"
 #include "qapi/error.h"
+#include "qapi/qmp/qstring.h"
 
 typedef struct Request {
     Coroutine *co;
     QEMUBH *bh;
 } Request;
 
+static BlockDriverState *blkreplay_append_snapshot(BlockDriverState *bs,
+                                                   Error **errp)
+{
+    int ret;
+    BlockDriverState *bs_snapshot;
+    int64_t total_size;
+    QemuOpts *opts = NULL;
+    char tmp_filename[PATH_MAX + 1];
+    QDict *snapshot_options = qdict_new();
+
+    /* Prepare options QDict for the overlay file */
+    qdict_put(snapshot_options, "file.driver",
+              qstring_from_str("file"));
+    qdict_put(snapshot_options, "driver",
+              qstring_from_str("qcow2"));
+
+    /* Create temporary file */
+    ret = get_tmp_filename(tmp_filename, PATH_MAX + 1);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not get temporary filename");
+        goto out;
+    }
+    qdict_put(snapshot_options, "file.filename",
+              qstring_from_str(tmp_filename));
+
+    /* Get the required size from the image */
+    total_size = bdrv_getlength(bs);
+    if (total_size < 0) {
+        error_setg_errno(errp, -total_size, "Could not get image size");
+        goto out;
+    }
+
+    opts = qemu_opts_create(bdrv_qcow2.create_opts, NULL, 0, &error_abort);
+    qemu_opt_set_number(opts, BLOCK_OPT_SIZE, total_size, &error_abort);
+    ret = bdrv_create(&bdrv_qcow2, tmp_filename, opts, errp);
+    qemu_opts_del(opts);
+    if (ret < 0) {
+        error_prepend(errp, "Could not create temporary overlay '%s': ",
+                      tmp_filename);
+        goto out;
+    }
+
+    bs_snapshot = bdrv_open(NULL, NULL, snapshot_options,
+                            BDRV_O_RDWR | BDRV_O_TEMPORARY, errp);
+    snapshot_options = NULL;
+    if (!bs_snapshot) {
+        ret = -EINVAL;
+        goto out;
+    }
+
+    /* bdrv_append() consumes a strong reference to bs_snapshot (i.e. it will
+     * call bdrv_unref() on it), so in order to be able to return one, we have
+     * to increase bs_snapshot's refcount here */
+    bdrv_ref(bs_snapshot);
+    bdrv_append(bs_snapshot, bs);
+
+    return bs_snapshot;
+
+out:
+    QDECREF(snapshot_options);
+    return NULL;
+}
+
 static int blkreplay_open(BlockDriverState *bs, QDict *options, int flags,
                           Error **errp)
 {
@@ -35,6 +99,14 @@ static int blkreplay_open(BlockDriverState *bs, QDict *options, int flags,
         goto fail;
     }
 
+    /* Add temporary snapshot to preserve the image */
+    if (!replay_snapshot
+        && !blkreplay_append_snapshot(bs->file->bs, &local_err)) {
+        ret = -EINVAL;
+        error_propagate(errp, local_err);
+        goto fail;
+    }
+
     ret = 0;
 fail:
     if (ret < 0) {
@@ -45,6 +117,10 @@ fail:
 
 static void blkreplay_close(BlockDriverState *bs)
 {
+    if (!replay_snapshot) {
+        /* Unref created snapshot file */
+        bdrv_unref(bs->file->bs);
+    }
 }
 
 static int64_t blkreplay_getlength(BlockDriverState *bs)
stubs/replay.c +1 -0
index 9c8aa48..9991ee5 100644
@@ -3,6 +3,7 @@
 #include "sysemu/sysemu.h"
 
 ReplayMode replay_mode;
+char *replay_snapshot;
 
 int64_t replay_save_clock(unsigned int kind, int64_t clock)
 {
vl.c +1 -1
index 0b72b12..c6fc5c9 100644
@@ -4414,7 +4414,7 @@ int main(int argc, char **argv, char **envp)
     }
 
     /* open the virtual block devices */
-    if (snapshot || replay_mode != REPLAY_MODE_NONE) {
+    if (snapshot) {
         qemu_opts_foreach(qemu_find_opts("drive"), drive_enable_snapshot,
                           NULL, NULL);
     }
#7 lore raw

PATCH 3/3 replay: disable default snapshot for record/replay

In reply to Pavel Dovgalyuk

This patch disables setting '-snapshot' option on by default in record/replay mode. This is needed for creating vmstates in record and replay modes.

---
 vl.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
vl.c +7 -1
index c6fc5c9..671fc04 100644
@@ -3129,7 +3129,13 @@ int main(int argc, char **argv, char **envp)
                 drive_add(IF_PFLASH, -1, optarg, PFLASH_OPTS);
                 break;
             case QEMU_OPTION_snapshot:
-                snapshot = 1;
+                {
+                    Error *blocker = NULL;
+                    snapshot = 1;
+                    error_setg(&blocker, QERR_REPLAY_NOT_SUPPORTED,
+                               "-snapshot");
+                    replay_add_blocker(blocker);
+                }
                 break;
             case QEMU_OPTION_hdachs:
                 {
#9 lore raw
In reply to Pavel Dovgalyuk

Ping?

Pavel Dovgalyuk

Quoted 33 lines

-----Original Message-----
From: Pavel Dovgalyuk [mailto:[email protected]]
Sent: Tuesday, January 31, 2017 2:57 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 0/3] block devices record/replay update

This set of patches includes several fixes for preserving the state of the block device images while recording and replaying virtual machine execution.

blkreplay driver now creates temporary overlay for underlaying devices This patch implicitly enables '-snapshot' behavior in record/replay mode. blkreplay layer creates temporary overlays on top of underlaying disk images. It is needed, because creating an overlay over blkreplay with explicit '-snapshot' option breaks the determinism.

---

Pavel Dovgalyuk (3):
block: implement bdrv_snapshot_goto for blkreplay
blkreplay: create temporary overlay for underlaying devices
replay: disable default snapshot for record/replay

 block/blkreplay.c |   84 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 stubs/replay.c    |    1 +
 vl.c              |   10 +++++-
 3 files changed, 93 insertions(+), 2 deletions(-)

--
Pavel Dovgalyuk

#10 lore raw

Destination host unreachable.

Ping again.

Pavel Dovgalyuk

Quoted 44 lines

-----Original Message-----
From: Pavel Dovgalyuk [mailto:[email protected]]
Sent: Monday, February 13, 2017 8:05 AM
To: 'Pavel Dovgalyuk'; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH 0/3] block devices record/replay update

Ping?

Pavel Dovgalyuk

Quoted 33 lines

-----Original Message-----
From: Pavel Dovgalyuk [mailto:[email protected]]
Sent: Tuesday, January 31, 2017 2:57 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 0/3] block devices record/replay update

This set of patches includes several fixes for preserving the state of the block device images while recording and replaying virtual machine execution.

blkreplay driver now creates temporary overlay for underlaying devices This patch implicitly enables '-snapshot' behavior in record/replay mode. blkreplay layer creates temporary overlays on top of underlaying disk images. It is needed, because creating an overlay over blkreplay with explicit '-snapshot' option breaks the determinism.

---

Pavel Dovgalyuk (3):
block: implement bdrv_snapshot_goto for blkreplay
blkreplay: create temporary overlay for underlaying devices
replay: disable default snapshot for record/replay

 block/blkreplay.c |   84 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 stubs/replay.c    |    1 +
 vl.c              |   10 +++++-
 3 files changed, 93 insertions(+), 2 deletions(-)

--
Pavel Dovgalyuk

#2 lore raw

Re: [Qemu-devel] [PATCH 1/3] block: implement bdrv_snapshot_goto for blkreplay

In reply to Pavel Dovgalyuk

Am 31.01.2017 um 12:57 hat Pavel Dovgalyuk geschrieben:

Quoted 4 lines

This patch enables making snapshots with blkreplay used in block devices.

Specifically, I think it avoids the blkreplay_open/close sequence. Is this what is needed to make it work?

We should probably mention in the commit message the exact reason why implementing .bdrv_snapshot_goto, but not the other snapshot related callbacks, fixes things. If you confirm my assumption, I can add that while applying.

Kevin

#6 lore raw

Re: [Qemu-devel] [PATCH 2/3] blkreplay: create temporary overlay for underlaying devices

In reply to Pavel Dovgalyuk

Am 31.01.2017 um 12:57 hat Pavel Dovgalyuk geschrieben:

Quoted 6 lines

This patch allows using '-snapshot' behavior in record/replay mode. blkreplay layer creates temporary overlays on top of underlaying disk images. It is needed, because creating an overlay over blkreplay breaks the determinism.

Is replacing the '-snapshot' behaviour (which was automatically enabled before this patch) with custom code what this patch really does? In other words, does it fix that the old behaviour didn't work correctly rather than adding a new feature? If so, the commit message is prone to misunderstanding, I think.

Quoted 20 lines
block/blkreplay.c +6 -0
index 8a03d62..172642f 100644
@@ -14,12 +14,76 @@
 #include "block/block_int.h"
 #include "sysemu/replay.h"
 #include "qapi/error.h"
+#include "qapi/qmp/qstring.h"
 
 typedef struct Request {
     Coroutine *co;
     QEMUBH *bh;
 } Request;
 
+static BlockDriverState *blkreplay_append_snapshot(BlockDriverState *bs,
+                                                   Error **errp)
+{
+    int ret;
+    BlockDriverState *bs_snapshot;

block/blkreplay.c line 38

Quoted 10 lines
block/blkreplay.c +10 -0
+    int64_t total_size;
+    QemuOpts *opts = NULL;
+    char tmp_filename[PATH_MAX + 1];
+    QDict *snapshot_options = qdict_new();
+
+    /* Prepare options QDict for the overlay file */
+    qdict_put(snapshot_options, "file.driver",
+              qstring_from_str("file"));
+    qdict_put(snapshot_options, "driver",
+              qstring_from_str("qcow2"));

Both of these statements fit each in a single line.

Quoted 21 lines
block/blkreplay.c +21 -0
+    /* Create temporary file */
+    ret = get_tmp_filename(tmp_filename, PATH_MAX + 1);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not get temporary filename");
+        goto out;
+    }
+    qdict_put(snapshot_options, "file.filename",
+              qstring_from_str(tmp_filename));
+
+    /* Get the required size from the image */
+    total_size = bdrv_getlength(bs);
+    if (total_size < 0) {
+        error_setg_errno(errp, -total_size, "Could not get image size");
+        goto out;
+    }
+
+    opts = qemu_opts_create(bdrv_qcow2.create_opts, NULL, 0, &error_abort);
+    qemu_opt_set_number(opts, BLOCK_OPT_SIZE, total_size, &error_abort);
+    ret = bdrv_create(&bdrv_qcow2, tmp_filename, opts, errp);
+    qemu_opts_del(opts);
+    if (ret < 0) {

block/blkreplay.c line 70

Quoted 10 lines
block/blkreplay.c +10 -0
+        error_prepend(errp, "Could not create temporary overlay '%s': ",
+                      tmp_filename);
+        goto out;
+    }
+
+    bs_snapshot = bdrv_open(NULL, NULL, snapshot_options,
+                            BDRV_O_RDWR | BDRV_O_TEMPORARY, errp);
+    snapshot_options = NULL;
+    if (!bs_snapshot) {
+        ret = -EINVAL;

The value of ret is unused, so why set it here?

block/blkreplay.c line 78

Quoted 8 lines
block/blkreplay.c +8 -0
+        goto out;
+    }
+
+    /* bdrv_append() consumes a strong reference to bs_snapshot (i.e. it will
+     * call bdrv_unref() on it), so in order to be able to return one, we have
+     * to increase bs_snapshot's refcount here */
+    bdrv_ref(bs_snapshot);
+    bdrv_append(bs_snapshot, bs);

Actually, your case is exactly what bdrv_append() is designed for: You don't need to return a strong reference here, the caller just checks for NULL and that's it. So:

1. bdrv_open() return a strong reference (refcount = 1)
2. You do bdrv_ref() (refcount = 2)
3. bdrv_append() takes a reference and uses it for bs->file
...
4. bdrv_close() calls blkreplay_close(), which does a bdrv_unref() (refcount = 1)
5. bdrv_close() unrefs all child nodes (refcount = 0 --> delete)

You can simplify this by just not doing the extra bdrv_ref/unref (i.e. remove steps 2 and 4). The correct reference counting is already taken care of by bdrv_append() and bdrv_close().

block/blkreplay.c line 85

Quoted 6 lines
block/blkreplay.c +6 -0
+    return bs_snapshot;
+
+out:
+    QDECREF(snapshot_options);
+    return NULL;
+}

Kevin

#8 lore raw

Re: [Qemu-devel] [PATCH 3/3] replay: disable default snapshot for record/replay

In reply to Pavel Dovgalyuk

Am 31.01.2017 um 12:57 hat Pavel Dovgalyuk geschrieben:

Quoted 5 lines

This patch disables setting '-snapshot' option on by default in record/replay mode. This is needed for creating vmstates in record and replay modes.

Looks good the me. (Though obviously, record/replay mode really comes with a lot more restrictions for block devices that aren't checked yet. But it's a start.)

Kevin

#3 lore raw

Re: [Qemu-devel] [PATCH 1/3] block: implement bdrv_snapshot_goto for blkreplay

In reply to Kevin Wolf
Quoted 9 lines

From: Kevin Wolf [mailto:[email protected]]
Am 31.01.2017 um 12:57 hat Pavel Dovgalyuk geschrieben:

Quoted 4 lines

This patch enables making snapshots with blkreplay used in block devices.

Specifically, I think it avoids the blkreplay_open/close sequence. Is this what is needed to make it work?

Then I'll need to implement bdrv_open, because there is only bdrv_file_open for blkreplay now.

Which way is better?

Quoted 4 lines

We should probably mention in the commit message the exact reason why implementing .bdrv_snapshot_goto, but not the other snapshot related callbacks, fixes things. If you confirm my assumption, I can add that while applying.

Pavel Dovgalyuk

#4 lore raw

Re: [Qemu-devel] [PATCH 1/3] block: implement bdrv_snapshot_goto for blkreplay

Am 27.02.2017 um 13:07 hat Pavel Dovgalyuk geschrieben:

Quoted 7 lines
Quoted 7 lines

From: Kevin Wolf [mailto:[email protected]]
Am 31.01.2017 um 12:57 hat Pavel Dovgalyuk geschrieben:

Quoted 4 lines

This patch enables making snapshots with blkreplay used in block devices.

Quoted 7 lines
Quoted 2 lines

Specifically, I think it avoids the blkreplay_open/close sequence. Is this what is needed to make it work?

Then I'll need to implement bdrv_open, because there is only bdrv_file_open for blkreplay now.

Which way is better?

I was just checking whether I understood the reason for this correctly.

If I did, then I think your solution is fine and we should just make the commit message a bit more explicit.

Kevin

Quoted 7 lines
Quoted 4 lines

We should probably mention in the commit message the exact reason why implementing .bdrv_snapshot_goto, but not the other snapshot related callbacks, fixes things. If you confirm my assumption, I can add that while applying.

Pavel Dovgalyuk