Skip to contentSkip to conversation

migration: Support gtree migration

PATCH Superseded Eric Auger 5 participants 7 messages Oct 10, 2019 – Oct 11, 2019 1 message not in the archive

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Eric Auger

author
#0 lore raw

PATCH v4 1/1 migration: Support gtree migration

Introduce support for GTree migration. A custom save/restore is implemented. Each item is made of a key and a data.

If the key is a pointer to an object, 2 VMSDs are passed into the GTree VMStateField.

When putting the items, the tree is traversed in sorted order by g_tree_foreach.

On the get() path, gtrees must be allocated using the proper key compare, key destroy and value destroy. This must be handled beforehand, for example in a pre_load method.

Tests are added to test save/dump of structs containing gtrees including the virtio-iommu domain/mappings scenario.

---

v3 -> v4:
- properly initialize capsule.vmdesc in put_gtree()
- use uintptr_t
- add trace points
- I did not use unions in VMStateField as suggested by Peter as this may complexify the struct. This can be added later on though.

v2 -> v3:
- do not include glib.h anymore
- introduce VMSTATE_GTREE_DIRECT_KEY_V
- use pre_load to allocate the tree and remove data pointer
- dump the number of nnodes and add checks on get path

v1 -> v2:
- fix compilation issues reported by robots
- fixed init of VMSD array
- direct key now dumped as a 32b
- removed useless cast from/to pointer

---
 include/migration/vmstate.h |  40 ++++
 migration/trace-events      |   5 +
 migration/vmstate-types.c   | 148 +++++++++++++
 tests/test-vmstate.c        | 421 ++++++++++++++++++++++++++++++++++++
 4 files changed, 614 insertions(+)
include/migration/vmstate.h +40 -0
index 1fbfd099dd..78caff55ef 100644
@@ -224,6 +224,7 @@ extern const VMStateInfo vmstate_info_unused_buffer;
 extern const VMStateInfo vmstate_info_tmp;
 extern const VMStateInfo vmstate_info_bitmap;
 extern const VMStateInfo vmstate_info_qtailq;
+extern const VMStateInfo vmstate_info_gtree;
 
 #define type_check_2darray(t1,t2,n,m) ((t1(*)[n][m])0 - (t2*)0)
 /*
@@ -754,6 +755,45 @@ extern const VMStateInfo vmstate_info_qtailq;
     .start        = offsetof(_type, _next),                              \
 }
 
+/*
+ * For migrating a GTree whose key is a pointer to _key_type and the
+ * value, a pointer to _val_type
+ * The target tree must have been properly initialized
+ * _vmsd: Start address of the 2 element array containing the key vmsd
+ *        and the data vmsd
+ * _key_type: type of the key
+ * _val_type: type of the value
+ */
+#define VMSTATE_GTREE_V(_field, _state, _version, _vmsd,                       \
+                        _key_type, _val_type)                                  \
+{                                                                              \
+    .name         = (stringify(_field)),                                       \
+    .version_id   = (_version),                                                \
+    .vmsd         = (_vmsd),                                                   \
+    .info         = &vmstate_info_gtree,                                       \
+    .start        = sizeof(_key_type),                                         \
+    .size         = sizeof(_val_type),                                         \
+    .offset       = offsetof(_state, _field),                                  \
+}
+
+/*
+ * For migrating a GTree with direct key and the value a pointer
+ * to _val_type
+ * The target tree must have been properly initialized
+ * _vmsd: data vmsd
+ * _val_type: type of the value
+ */
+#define VMSTATE_GTREE_DIRECT_KEY_V(_field, _state, _version, _vmsd, _val_type) \
+{                                                                              \
+    .name         = (stringify(_field)),                                       \
+    .version_id   = (_version),                                                \
+    .vmsd         = (_vmsd),                                                   \
+    .info         = &vmstate_info_gtree,                                       \
+    .start        = 0,                                                         \
+    .size         = sizeof(_val_type),                                         \
+    .offset       = offsetof(_state, _field),                                  \
+}
+
 /* _f : field name
    _f_n : num of elements field_name
    _n : num of elements
migration/trace-events +5 -0
index 858d415d56..6dee7b5389 100644
@@ -71,6 +71,11 @@ get_qtailq_end(const char *name, const char *reason, int val) "%s %s/%d"
 put_qtailq(const char *name, int version_id) "%s v%d"
 put_qtailq_end(const char *name, const char *reason) "%s %s"
 
+get_gtree(const char *field_name, const char *key_vmsd_name, const char *val_vmsd_name, uint32_t nnodes) "%s(%s/%s) nnodes=%d"
+get_gtree_end(const char *field_name, const char *key_vmsd_name, const char *val_vmsd_name, int ret) "%s(%s/%s) %d"
+put_gtree(const char *field_name, const char *key_vmsd_name, const char *val_vmsd_name, uint32_t nnodes) "%s(%s/%s) nnodes=%d"
+put_gtree_end(const char *field_name, const char *key_vmsd_name, const char *val_vmsd_name, int ret) "%s(%s/%s) %d"
+
 # qemu-file.c
 qemu_file_fclose(void) ""
 
migration/vmstate-types.c +148 -0

151 lines not on this page · raw message

tests/test-vmstate.c +421 -0

433 lines not on this page · raw message

Signature · 1 line
2.20.1
#1 lore raw
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      block/nbd.o
  CC      block/sheepdog.o
  CC      block/accounting.o

make: *** [Makefile:994: docs/interop/index.html] Error 2
make: *** Deleting file 'docs/interop/index.html'
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=b0665a6b133e47c781238452d53aabf1', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-f9u21j2v/src/docker-src.2019-10-10-19.48.45.2708:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=b0665a6b133e47c781238452d53aabf1
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-f9u21j2v/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real 2m25.181s
user 0m7.455s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

author
#2 lore raw

Hi,

Quoted 17 lines · On 10/11/19 1:51 AM, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

Quoted 4 lines
  CC      block/nbd.o
  CC      block/sheepdog.o
  CC      block/accounting.o

make: *** [Makefile:994: docs/interop/index.html] Error 2

This failure looks unrelated to this series; and the above script succeeds on my side.

Best Regards

Eric

Quoted 21 lines

make: *** Deleting file 'docs/interop/index.html'
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=b0665a6b133e47c781238452d53aabf1', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-f9u21j2v/src/docker-src.2019-10-10-19.48.45.2708:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=b0665a6b133e47c781238452d53aabf1
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-f9u21j2v/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real 2m25.181s
user 0m7.455s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Sphinx UnpicklingError (was Re: [Patchew-devel] Fwd: Re: [PATCH v4] migration: Support gtree migration)

Peter, this UnpicklingError from Sphinx seems to be caused by building manuals in parallel. Can you take a look?

Thanks,

Paolo

Quoted 71 lines · Auger Eric

Hi,

This failure looks unrelated to this series; and the above script succeeds on my side.

Best Regards

Eric

-------- Forwarded Message --------
Subject: Re: [PATCH v4] migration: Support gtree migration
Date: Fri, 11 Oct 2019 07:19:29 +0200
From: Auger Eric <[email protected]>
To: [email protected], [email protected]
CC: [email protected], [email protected], [email protected], [email protected]

Hi,

Quoted 38 lines · On 10/11/19 1:51 AM, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      block/nbd.o
  CC      block/sheepdog.o
  CC      block/accounting.o

make: *** [Makefile:994: docs/interop/index.html] Error 2

make: *** Deleting file 'docs/interop/index.html'
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=b0665a6b133e47c781238452d53aabf1', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-f9u21j2v/src/docker-src.2019-10-10-19.48.45.2708:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=b0665a6b133e47c781238452d53aabf1
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-f9u21j2v/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real 2m25.181s
user 0m7.455s

Quoted 6 lines

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

_______________________________________________
Patchew-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/patchew-devel

Reviewed-by on v4 1/1
#5 lore raw
In reply to Eric Auger
Quoted 9 lines · Eric Auger

Introduce support for GTree migration. A custom save/restore is implemented. Each item is made of a key and a data.

If the key is a pointer to an object, 2 VMSDs are passed into the GTree VMStateField.

When putting the items, the tree is traversed in sorted order by g_tree_foreach.

Quoted 8 lines

On the get() path, gtrees must be allocated using the proper key compare, key destroy and value destroy. This must be handled beforehand, for example in a pre_load method.

Tests are added to test save/dump of structs containing gtrees including the virtio-iommu domain/mappings scenario.

Signature · 1 line
Peter Xu
#6 lore raw
In reply to Eric Auger
Quoted 9 lines · Eric Auger

Introduce support for GTree migration. A custom save/restore is implemented. Each item is made of a key and a data.

If the key is a pointer to an object, 2 VMSDs are passed into the GTree VMStateField.

When putting the items, the tree is traversed in sorted order by g_tree_foreach.

Quoted 8 lines

On the get() path, gtrees must be allocated using the proper key compare, key destroy and value destroy. This must be handled beforehand, for example in a pre_load method.

Tests are added to test save/dump of structs containing gtrees including the virtio-iommu domain/mappings scenario.

I found a bug, you have to respin, go to BUG
(here was a reviewed-by)

But, ....

I didn't noticed on the 1st look

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 738

Quoted 4 lines
migration/vmstate-types.c +4 -0
+    const VMStateDescription *key_vmsd = direct_key ? NULL : &field->vmsd[0];
+    const VMStateDescription *val_vmsd = direct_key ? &field->vmsd[0] :
+                                                      &field->vmsd[1];
+    const char *key_vmsd_name = direct_key ? "direct" : key_vmsd->name;

This is ugly as hell.
We are using a pointer to pass to mean an array, and abuse it.

But once told that, it is not trivial to do this in a proper way.

On the other hand, if you have to respin for any other reason, please consider changing the meaning of vmsd[0] and [1].

vmsd[0] -> val_t
vmsd[1] -> key_t

if (vmsd[1] == NULL)
direct_key = true;

    const VMStateDescription *val_vmsd = &field->vmsd[0];
    const VMStateDescription *key_vmsd = &field->vmsd[1]
    const char *key_vmsd_name = key_vmsd ? "direct" : key_vmsd->name;

Same for get_gtree().

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 806

Quoted 2 lines
migration/vmstate-types.c +2 -0
+        if (direct_key) {
+            key = (void *)(uintptr_t)qemu_get_be64(f);

no g_malloc().

Quoted 6 lines
migration/vmstate-types.c +6 -0
+        } else {
+            key = g_malloc0(key_size);
+            ret = vmstate_load_state(f, key_vmsd, key, version_id);
+            if (ret) {
+                error_report("%s : failed to load %s (%d)",
+                             field->name, key_vmsd->name, ret);

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 822

Quoted 10 lines
migration/vmstate-types.c +10 -0
+                g_free(key);
+                return ret;
+            }
+        }
+        val = g_malloc0(val_size);
+        ret = vmstate_load_state(f, val_vmsd, val, version_id);
+        if (ret) {
+            error_report("%s : failed to load %s (%d)",
+                         field->name, val_vmsd->name, ret);
+            g_free(key);
BUG: Allways free.  This need to be protected by a direct_key(), no?

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 830

Quoted 8 lines
migration/vmstate-types.c +8 -0
+            g_free(val);
+            return ret;
+        }
+        g_tree_insert(tree, key, val);
+    }
+    if (count != nnodes) {
+        error_report("%s inconsistent stream when loading the gtree",
+                     field->name);

BUG2: we need to return an error here, right?

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 835

Quoted 5 lines
migration/vmstate-types.c +5 -0
+    }
+    trace_get_gtree_end(field->name, key_vmsd_name, val_vmsd->name, ret);
+    return ret;
+}
+

Later, Juan.

author
#7 lore raw

Hi Juan,

Quoted 26 lines · Juan Quintela
Quoted 9 lines · Eric Auger

Introduce support for GTree migration. A custom save/restore is implemented. Each item is made of a key and a data.

If the key is a pointer to an object, 2 VMSDs are passed into the GTree VMStateField.

When putting the items, the tree is traversed in sorted order by g_tree_foreach.

Quoted 8 lines

On the get() path, gtrees must be allocated using the proper key compare, key destroy and value destroy. This must be handled beforehand, for example in a pre_load method.

Tests are added to test save/dump of structs containing gtrees including the virtio-iommu domain/mappings scenario.

I found a bug, you have to respin, go to BUG
(here was a reviewed-by)

But, ....

I didn't noticed on the 1st look

Quoted 7 lines

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 738

Quoted 4 lines
migration/vmstate-types.c +4 -0
+    const VMStateDescription *key_vmsd = direct_key ? NULL : &field->vmsd[0];
+    const VMStateDescription *val_vmsd = direct_key ? &field->vmsd[0] :
+                                                      &field->vmsd[1];
+    const char *key_vmsd_name = direct_key ? "direct" : key_vmsd->name;

This is ugly as hell.
We are using a pointer to pass to mean an array, and abuse it.

I agree. My first attempt was using subsections to pass the second vmsd but it was similarly ugly.

Quoted 8 lines

But once told that, it is not trivial to do this in a proper way.

On the other hand, if you have to respin for any other reason, please consider changing the meaning of vmsd[0] and [1].

vmsd[0] -> val_t
vmsd[1] -> key_t

OK. I chose that order since usually the pair is expressed as key/value and I found it more logical from the qemu user perspective. But I have no strong preference.

Quoted 9 lines

if (vmsd[1] == NULL)
direct_key = true;

    const VMStateDescription *val_vmsd = &field->vmsd[0];
    const VMStateDescription *key_vmsd = &field->vmsd[1]
    const char *key_vmsd_name = key_vmsd ? "direct" : key_vmsd->name;

Same for get_gtree().

OK

Quoted 24 lines

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 806

Quoted 4 lines
migration/vmstate-types.c +2 -0
+        if (direct_key) {
+            key = (void *)(uintptr_t)qemu_get_be64(f);

no g_malloc().

Quoted 6 lines
migration/vmstate-types.c +6 -0
+        } else {
+            key = g_malloc0(key_size);
+            ret = vmstate_load_state(f, key_vmsd, key, version_id);
+            if (ret) {
+                error_report("%s : failed to load %s (%d)",
+                             field->name, key_vmsd->name, ret);

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 822

Quoted 10 lines
migration/vmstate-types.c +10 -0
+                g_free(key);
+                return ret;
+            }
+        }
+        val = g_malloc0(val_size);
+        ret = vmstate_load_state(f, val_vmsd, val, version_id);
+        if (ret) {
+            error_report("%s : failed to load %s (%d)",
+                         field->name, val_vmsd->name, ret);
+            g_free(key);
Quoted 1 line
BUG: Allways free.  This need to be protected by a direct_key(), no?

ouch yes

Quoted 11 lines

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 830

Quoted 9 lines
migration/vmstate-types.c +8 -0
+            g_free(val);
+            return ret;
+        }
+        g_tree_insert(tree, key, val);
+    }
+    if (count != nnodes) {
+        error_report("%s inconsistent stream when loading the gtree",
+                     field->name);

BUG2: we need to return an error here, right?

yep

Quoted 9 lines

migration/vmstate-types.c const VMStateInfo vmstate_info_qtailq = { · line 835

Quoted 6 lines
migration/vmstate-types.c +5 -0
+    }
+    trace_get_gtree_end(field->name, key_vmsd_name, val_vmsd->name, ret);
+    return ret;
+}
+

Later, Juan.

Thanks for the review

Eric