Skip to contentSkip to conversation

migration: Add migrate-set-bitmap-node-mapping

RFC Superseded Max Reitz 3 participants 6 messages May 13, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Max Reitz

author
#0 lore raw

RFC 1/1 migration: Add migrate-set-bitmap-node-mapping

This command allows mapping block node names to aliases for the purpose of block dirty bitmap migration.

This way, management tools can use different node names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (on the source, the destination, or even both with arbitrary aliases in the migration stream).

---
Branch: https://github.com/XanClic/qemu.git migration-bitmap-mapping-rfc
Branch: https://git.xanclic.moe/XanClic/qemu.git migration-bitmap-mapping-rfc

Vladimir has proposed something like this in April:
https://lists.nongnu.org/archive/html/qemu-block/2020-04/msg00171.html

Now I’ve been asked by my manager to look at this, so I decided to just write a patch to see how it’d play out.

This is an RFC, because I’d like to tack on tests to the final version, but I’m not sure whether I can come up with something before the end of the week (and I’ll be on PTO for the next two weeks).

Also, I don’t know whether migration/block-dirty-bitmap.c is the best place to put qmp_migrate_set_bitmap_mapping(), but it appears we already have some QMP handlers in migration/, so I suppose it isn’t too bad.

---
 qapi/migration.json            | 36 ++++++++++++++++++++
 migration/block-dirty-bitmap.c | 60 ++++++++++++++++++++++++++++++++--
 2 files changed, 94 insertions(+), 2 deletions(-)
qapi/migration.json +36 -0
index d5000558c6..94aec8d194 100644
@@ -1621,3 +1621,39 @@
 ##
 { 'event': 'UNPLUG_PRIMARY',
   'data': { 'device-id': 'str' } }
+
+##
+# @MigrationBlockNodeMapping:
+#
+# Maps a block node name to an alias for migration.
+#
+# @node-name: A block node name.
+#
+# @alias: An alias name for migration (for example the node name on
+#         the opposite site).
+#
+# Since: 5.1
+##
+{ 'struct': 'MigrationBitmapMapping',
+  'data': {
+      'node-name': 'str',
+      'alias': 'str'
+  } }
+
+##
+# @migrate-set-bitmap-node-mapping:
+#
+# Maps block node names to arbitrary aliases for the purpose of dirty
+# bitmap migration.  Such aliases may for example be the corresponding
+# node names on the opposite site.
+#
+# By default, every node name is mapped to itself.
+#
+# @mapping: The mapping; must be one-to-one, but not necessarily
+#           complete.  Any mapping not given will be reset to the
+#           default (i.e. the identity mapping).
+#
+# Since: 5.1
+##
+{ 'command': 'migrate-set-bitmap-node-mapping',
+  'data': { 'mapping': ['MigrationBlockNodeMapping'] } }
migration/block-dirty-bitmap.c +58 -2

104 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#1 lore raw
Quoted 10 lines · Max Reitz

This command allows mapping block node names to aliases for the purpose of block dirty bitmap migration.

This way, management tools can use different node names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (on the source, the destination, or even both with arbitrary aliases in the migration stream).

Sorry, let me try again, I changed the name of the command and the type just before sending and that was a stupid idea, as it turns out.

Max

#2 lore raw
In reply to Max Reitz
Quoted 20 lines · Max Reitz

This command allows mapping block node names to aliases for the purpose of block dirty bitmap migration.

This way, management tools can use different node names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (on the source, the destination, or even both with arbitrary aliases in the migration stream).

---
Branch: https://github.com/XanClic/qemu.git migration-bitmap-mapping-rfc
Branch: https://git.xanclic.moe/XanClic/qemu.git migration-bitmap-mapping-rfc

Vladimir has proposed something like this in April:
https://lists.nongnu.org/archive/html/qemu-block/2020-04/msg00171.html

Now I’ve been asked by my manager to look at this, so I decided to just write a patch to see how it’d play out.

Quoted 7 lines

This is an RFC, because I’d like to tack on tests to the final version, but I’m not sure whether I can come up with something before the end of the week (and I’ll be on PTO for the next two weeks).

Also, I don’t know whether migration/block-dirty-bitmap.c is the best place to put qmp_migrate_set_bitmap_mapping(), but it appears we already have some QMP handlers in migration/, so I suppose it isn’t too bad.

My 'qemu-img bitmap' series has block/monitor/bitmap-qmp-cmds.c - perhaps that's a decent location? Or maybe another new file in that directory, if putting it directly in an existing file would drag in too much into qemu-img?

Quoted 4 lines
---
  qapi/migration.json            | 36 ++++++++++++++++++++
  migration/block-dirty-bitmap.c | 60 ++++++++++++++++++++++++++++++++--
  2 files changed, 94 insertions(+), 2 deletions(-)

But I see you've already posted a v2, so I'll stop reviewing here and switch to that thread.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#3 lore raw
In reply to Max Reitz
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

/tmp/qemu-test/src/qapi/migration.json:1637: documentation comment is for 'MigrationBlockNodeMapping'
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
make: *** [qapi-gen-timestamp] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=4aeb5780383b41b3936ebd2876745e60', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-txmz1ch1/src/docker-src.2020-05-13-15.32.06.27394:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=4aeb5780383b41b3936ebd2876745e60
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-txmz1ch1/src'
make: *** [docker-run-test-quick@centos7] Error 2

real 2m2.691s
user 0m7.876s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#4 lore raw
In reply to Max Reitz
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

  GEN     hw/intc/trace.h
  GEN     hw/isa/trace.h
  GEN     hw/mem/trace.h

make: *** [Makefile:666: qapi-gen-timestamp] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=779d0814f6dd4e8c93f07f5faeea0d47', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-vpaza0yt/src/docker-src.2020-05-13-15.34.45.31590:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=779d0814f6dd4e8c93f07f5faeea0d47
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-vpaza0yt/src'
make: *** [docker-run-test-debug@fedora] Error 2

real 3m8.744s
user 0m8.054s

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#5 lore raw
In reply to Max Reitz
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  GEN     hw/dma/trace.h
  GEN     hw/hppa/trace.h
  GEN     hw/i2c/trace.h

make: *** [Makefile:666: qapi-gen-timestamp] Error 1
make: *** Waiting for unfinished jobs....
AR /tmp/qemu-test/build/slirp/libslirp.a
make[1]: Leaving directory '/tmp/qemu-test/src/slirp'
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=f9fde3d4c1a0419085775fd925ba23aa', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-s3k90id6/src/docker-src.2020-05-13-15.38.50.6519:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=f9fde3d4c1a0419085775fd925ba23aa
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-s3k90id6/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real 1m28.202s
user 0m8.010s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]