Skip to contentSkip to conversation

migration: Multi-thread compression with zstd method

RFC PATCH In review Zeyu Jin 2 participants 10 messages Nov 9, 2020 – Nov 16, 2020

lore
Share & tools
lore

Zeyu Jin

author
#0 lore raw

cover migration: Multi-thread compression with zstd method

Currently we have both multi-thread compression and multifd to optimize live migration in Qemu. Mulit-thread compression deals with the situation where network bandwith is limited but cpu resource adequate. Multifd instead aims to take full advantage of network bandwith. Moreover it supports both zlib and zstd compression on each channel.

In this patch series, we did some code refactoring on multi-thread compression live migration and bring zstd compression method support for it.

Below is the test result of multi-thread compression live migration with different compress methods. Test result shows that zstd outperforms zlib by about 70%.

Migration Configuration:
Guest 8U 32G
compress-threads 8
decompress-threads 2
compress-level 1
bandwidth-limit 100Mbps

Test Result:

 +---------------------+--------------+-------------+
 |  compress method    |   zlib       |    zstd     |
 +---------------------+--------------+-------------+
 |  total time (ms)    |   75256      |    44187    |
 +---------------------+--------------+-------------+
 |  downtime(ms)       |   128        |    81       |
 +---------------------+--------------+-------------+
 |  transferred ram(kB)|   1576866    |    736117   |
 +---------------------+--------------+-------------+
 |  throughput(mbps)   |   172.06     |    137.16   |
 +---------------------+--------------+-------------+
 |  total ram(kB)      |   33685952   |    33685952 |
 +---------------------+--------------+-------------+

Zeyu Jin (6):
migration: Add multi-thread compress method
migration: Refactoring multi-thread compress migration
migration: Add multi-thread compress ops
migration: Add zstd support in multi-thread compression
migration: Add compress_level sanity check
doc: Update multi-thread compression doc

 docs/multi-thread-compression.txt |  31 ++-
 hw/core/qdev-properties-system.c  |  11 +
 include/hw/qdev-properties.h      |   4 +
 migration/migration.c             |  56 ++++-
 migration/migration.h             |   1 +
 migration/qemu-file.c             |  62 +----
 migration/qemu-file.h             |   4 +-
 migration/ram.c                   | 381 +++++++++++++++++++++++++-----
 monitor/hmp-cmds.c                |  12 +
 qapi/migration.json               |  26 +-
 10 files changed, 465 insertions(+), 123 deletions(-)
Signature · 1 line
2.23.0
author
#1 lore raw

RFC PATCH 1/6 migration: Add multi-thread compress method

A multi-thread compress method parameter is added to hold the method we are going to use. By default the 'zlib' method is used to maintain the compatibility as before.

---
 hw/core/qdev-properties-system.c | 11 +++++++++++
 include/hw/qdev-properties.h     |  4 ++++
 migration/migration.c            | 15 +++++++++++++++
 monitor/hmp-cmds.c               | 12 ++++++++++++
 qapi/migration.json              | 26 +++++++++++++++++++++++++-
 5 files changed, 67 insertions(+), 1 deletion(-)
hw/core/qdev-properties-system.c +11 -0
index b81a4e8d14..d757b2cd70 100644
@@ -663,6 +663,17 @@ const PropertyInfo qdev_prop_multifd_compression = {
     .set_default_value = qdev_propinfo_set_default_value_enum,
 };
 
+/* --- CompressMethod --- */
+const PropertyInfo qdev_prop_compress_method = {
+    .name = "CompressMethod",
+    .description = "multi-thread compression method, "
+                   "zlib",
+    .enum_table = &CompressMethod_lookup,
+    .get = qdev_propinfo_get_enum,
+    .set = qdev_propinfo_set_enum,
+    .set_default_value = qdev_propinfo_set_default_value_enum,
+};
+
 /* --- Reserved Region --- */
 
 /*
include/hw/qdev-properties.h +4 -0
index 4437450065..4a943f7e80 100644
@@ -23,6 +23,7 @@ extern const PropertyInfo qdev_prop_macaddr;
 extern const PropertyInfo qdev_prop_reserved_region;
 extern const PropertyInfo qdev_prop_on_off_auto;
 extern const PropertyInfo qdev_prop_multifd_compression;
+extern const PropertyInfo qdev_prop_compress_method;
 extern const PropertyInfo qdev_prop_losttickpolicy;
 extern const PropertyInfo qdev_prop_blockdev_on_error;
 extern const PropertyInfo qdev_prop_bios_chs_trans;
@@ -193,6 +194,9 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
 #define DEFINE_PROP_MULTIFD_COMPRESSION(_n, _s, _f, _d) \
     DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_multifd_compression, \
                        MultiFDCompression)
+#define DEFINE_PROP_COMPRESS_METHOD(_n, _s, _f, _d) \
+    DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_compress_method, \
+                       CompressMethod)
 #define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \
     DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_losttickpolicy, \
                         LostTickPolicy)
migration/migration.c +15 -0
index 3263aa55a9..d0da95fc0d 100644
@@ -83,6 +83,7 @@
 #define DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT 2
 /*0: means nocompress, 1: best speed, ... 9: best compress ratio */
 #define DEFAULT_MIGRATE_COMPRESS_LEVEL 1
+#define DEFAULT_MIGRATE_COMPRESS_METHOD COMPRESS_METHOD_ZLIB
 /* Define default autoconverge cpu throttle migration parameters */
 #define DEFAULT_MIGRATE_THROTTLE_TRIGGER_THRESHOLD 50
 #define DEFAULT_MIGRATE_CPU_THROTTLE_INITIAL 20
@@ -843,6 +844,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp)
     params->compress_wait_thread = s->parameters.compress_wait_thread;
     params->has_decompress_threads = true;
     params->decompress_threads = s->parameters.decompress_threads;
+    params->has_compress_method = true;
+    params->compress_method = s->parameters.compress_method;
     params->has_throttle_trigger_threshold = true;
     params->throttle_trigger_threshold = s->parameters.throttle_trigger_threshold;
     params->has_cpu_throttle_initial = true;
@@ -1407,6 +1410,10 @@ static void migrate_params_test_apply(MigrateSetParameters *params,
         dest->decompress_threads = params->decompress_threads;
     }
 
+    if (params->has_compress_method) {
+        dest->compress_method = params->compress_method;
+    }
+
     if (params->has_throttle_trigger_threshold) {
         dest->throttle_trigger_threshold = params->throttle_trigger_threshold;
     }
@@ -1504,6 +1511,10 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp)
         s->parameters.decompress_threads = params->decompress_threads;
     }
 
+    if (params->has_compress_method) {
+        s->parameters.compress_method = params->compress_method;
+    }
+
     if (params->has_throttle_trigger_threshold) {
         s->parameters.throttle_trigger_threshold = params->throttle_trigger_threshold;
     }
@@ -3715,6 +3726,9 @@ static Property migration_properties[] = {
     DEFINE_PROP_UINT8("x-decompress-threads", MigrationState,
                       parameters.decompress_threads,
                       DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT),
+    DEFINE_PROP_COMPRESS_METHOD("compress-method", MigrationState,
+                      parameters.compress_method,
+                      DEFAULT_MIGRATE_COMPRESS_METHOD),
     DEFINE_PROP_UINT8("x-throttle-trigger-threshold", MigrationState,
                       parameters.throttle_trigger_threshold,
                       DEFAULT_MIGRATE_THROTTLE_TRIGGER_THRESHOLD),
@@ -3829,6 +3843,7 @@ static void migration_instance_init(Object *obj)
     params->has_compress_level = true;
     params->has_compress_threads = true;
     params->has_decompress_threads = true;
+    params->has_compress_method = true;
     params->has_throttle_trigger_threshold = true;
     params->has_cpu_throttle_initial = true;
     params->has_cpu_throttle_increment = true;
monitor/hmp-cmds.c +12 -0
index 56e9bad33d..e771c36e1b 100644
@@ -419,6 +419,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict)
             MigrationParameter_str(MIGRATION_PARAMETER_DECOMPRESS_THREADS),
             params->decompress_threads);
         assert(params->has_throttle_trigger_threshold);
+        monitor_printf(mon, "%s: %s\n",
+            MigrationParameter_str(MIGRATION_PARAMETER_COMPRESS_METHOD),
+            CompressMethod_str(params->compress_method));
         monitor_printf(mon, "%s: %u\n",
             MigrationParameter_str(MIGRATION_PARAMETER_THROTTLE_TRIGGER_THRESHOLD),
             params->throttle_trigger_threshold);
@@ -1281,6 +1284,7 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
     MigrateSetParameters *p = g_new0(MigrateSetParameters, 1);
     uint64_t valuebw = 0;
     uint64_t cache_size;
+    CompressMethod compress_method;
     Error *err = NULL;
     int val, ret;
 
@@ -1306,6 +1310,14 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
         p->has_decompress_threads = true;
         visit_type_int(v, param, &p->decompress_threads, &err);
         break;
+    case MIGRATION_PARAMETER_COMPRESS_METHOD:
+        p->has_compress_method = true;
+        visit_type_CompressMethod(v, param, &compress_method, &err);
+        if (err) {
+            break;
+        }
+        p->compress_method = compress_method;
+        break;
     case MIGRATION_PARAMETER_THROTTLE_TRIGGER_THRESHOLD:
         p->has_throttle_trigger_threshold = true;
         visit_type_int(v, param, &p->throttle_trigger_threshold, &err);
qapi/migration.json +25 -1
index 3c75820527..d262683a38 100644
@@ -525,6 +525,19 @@
   'data': [ 'none', 'zlib',
             { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
 
+##
+# @CompressMethod:
+#
+# An enumeration of multi-thread compression methods.
+#
+# @zlib: use zlib compression method.
+#
+# Since: 5.0
+#
+##
+{ 'enum': 'CompressMethod',
+  'data': [ 'zlib' ] }
+
 ##
 # @BitmapMigrationBitmapAlias:
 #
@@ -599,6 +612,9 @@
 #                      compression, so set the decompress-threads to the number about 1/4
 #                      of compress-threads is adequate.
 #
+# @compress-method: Set compression method to use in multi-thread compression.
+#                   Defaults to none. (Since 5.0)
+#
 # @throttle-trigger-threshold: The ratio of bytes_dirty_period and bytes_xfer_period
 #                              to trigger throttling. It is expressed as percentage.
 #                              The default value is 50. (Since 5.0)
@@ -722,7 +738,7 @@
   'data': ['announce-initial', 'announce-max',
            'announce-rounds', 'announce-step',
            'compress-level', 'compress-threads', 'decompress-threads',
-           'compress-wait-thread', 'throttle-trigger-threshold',
+           'compress-wait-thread', 'compress-method', 'throttle-trigger-threshold',
            'cpu-throttle-initial', 'cpu-throttle-increment',
            'cpu-throttle-tailslow',
            'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth',
@@ -759,6 +775,9 @@
 #
 # @decompress-threads: decompression thread count
 #
+# @compress-method: Which multi-thread compression method to use.
+#                   Defaults to none. (Since 5.0)
+#
 # @throttle-trigger-threshold: The ratio of bytes_dirty_period and bytes_xfer_period
 #                              to trigger throttling. It is expressed as percentage.
 #                              The default value is 50. (Since 5.0)
@@ -889,6 +908,7 @@
             '*compress-threads': 'int',
             '*compress-wait-thread': 'bool',
             '*decompress-threads': 'int',
+            '*compress-method': 'CompressMethod',
             '*throttle-trigger-threshold': 'int',
             '*cpu-throttle-initial': 'int',
             '*cpu-throttle-increment': 'int',
@@ -953,6 +973,9 @@
 #
 # @decompress-threads: decompression thread count
 #
+# @compress-method: Which multi-thread compression method to use.
+#                   Defaults to none. (Since 5.0)
+#
 # @throttle-trigger-threshold: The ratio of bytes_dirty_period and bytes_xfer_period
 #                              to trigger throttling. It is expressed as percentage.
 #                              The default value is 50. (Since 5.0)
@@ -1083,6 +1106,7 @@
             '*compress-threads': 'uint8',
             '*compress-wait-thread': 'bool',
             '*decompress-threads': 'uint8',
+            '*compress-method': 'CompressMethod',
             '*throttle-trigger-threshold': 'uint8',
             '*cpu-throttle-initial': 'uint8',
             '*cpu-throttle-increment': 'uint8',
Signature · 1 line
2.23.0
#2 lore raw

Re: [RFC PATCH 1/6] migration: Add multi-thread compress method

Quoted 7 lines · Zeyu Jin

A multi-thread compress method parameter is added to hold the method we are going to use. By default the 'zlib' method is used to maintain the compatibility as before.

---

Focusing on just the UI:

qapi/migration.json line 535

Quoted 13 lines

+++ b/qapi/migration.json

qapi/migration.json +8 -0
@@ -525,6 +525,19 @@
   'data': [ 'none', 'zlib',
             { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
 
+##
+# @CompressMethod:
+#
+# An enumeration of multi-thread compression methods.
+#
+# @zlib: use zlib compression method.
+#
+# Since: 5.0

This will have to be Since: 6.0, as it has missed feature freeze for 5.2.

Quoted 9 lines
qapi/migration.json +5 -0
+#
+##
+{ 'enum': 'CompressMethod',
+  'data': [ 'zlib' ] }
+
 ##
 # @BitmapMigrationBitmapAlias:
 #

qapi/migration.json line 616

Quoted 5 lines
qapi/migration.json +2 -0
@@ -599,6 +612,9 @@
 #                      compression, so set the decompress-threads to the number about 1/4
 #                      of compress-threads is adequate.
 #
+# @compress-method: Set compression method to use in multi-thread compression.
+#                   Defaults to none. (Since 5.0)

Looks like it will be a recurring issue in the series, so I'll quit pointing it out.

CompressMethod above does not have a 'none' option, so how can it default to none? Did you forget an enum option?

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#3 lore raw

Re: [RFC PATCH 1/6] migration: Add multi-thread compress method

Quoted 16 lines · Eric Blake
Quoted 7 lines · Zeyu Jin

A multi-thread compress method parameter is added to hold the method we are going to use. By default the 'zlib' method is used to maintain the compatibility as before.

---

Focusing on just the UI:

Quoted 5 lines

+++ b/qapi/migration.json

qapi/migration.json +0 -0
@@ -525,6 +525,19 @@
   'data': [ 'none', 'zlib',
             { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
 
Quoted 11 lines

qapi/migration.json line 535

Quoted 8 lines
qapi/migration.json +8 -0
+##
+# @CompressMethod:
+#
+# An enumeration of multi-thread compression methods.
+#
+# @zlib: use zlib compression method.
+#
+# Since: 5.0

This will have to be Since: 6.0, as it has missed feature freeze for 5.2.

Thank you for noticing. I will fix that.

Quoted 18 lines
Quoted 9 lines
qapi/migration.json +5 -0
+#
+##
+{ 'enum': 'CompressMethod',
+  'data': [ 'zlib' ] }
+
 ##
 # @BitmapMigrationBitmapAlias:
 #

qapi/migration.json line 616

Quoted 5 lines
qapi/migration.json +2 -0
@@ -599,6 +612,9 @@
 #                      compression, so set the decompress-threads to the number about 1/4
 #                      of compress-threads is adequate.
 #
+# @compress-method: Set compression method to use in multi-thread compression.
+#                   Defaults to none. (Since 5.0)

Looks like it will be a recurring issue in the series, so I'll quit pointing it out.

Quoted 3 lines

CompressMethod above does not have a 'none' option, so how can it default to none? Did you forget an enum option?

Actually this is a comment mistake. The default option is expected to be 'zlib'. I will fix the comment.

author
#4 lore raw

RFC PATCH 2/6 migration: Refactoring multi-thread compress migration

In reply to Zeyu Jin

Code refactor for the compression procedure which includes:

1. Move qemu_compress_data and qemu_put_compression_data from qemu-file.c to ram.c, for the reason that most part of the code logical has nothing to do with qemu-file. Besides, the decompression code is located at ram.c only.

2. Simplify the function input arguments for compression and decompression. Wrap the input into the param structure which already exists. This change also makes the function much more flexible for other compression methods.

---
 migration/qemu-file.c | 62 ++++++-------------------------
 migration/qemu-file.h |  4 +-
 migration/ram.c       | 86 ++++++++++++++++++++++++++++++-------------
 3 files changed, 75 insertions(+), 77 deletions(-)
migration/qemu-file.c +12 -50
index be21518c57..1efb667aa1 100644
@@ -737,56 +737,6 @@ uint64_t qemu_get_be64(QEMUFile *f)
     return v;
 }
 
-/* return the size after compression, or negative value on error */
-static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len,
-                              const uint8_t *source, size_t source_len)
-{
-    int err;
-
-    err = deflateReset(stream);
-    if (err != Z_OK) {
-        return -1;
-    }
-
-    stream->avail_in = source_len;
-    stream->next_in = (uint8_t *)source;
-    stream->avail_out = dest_len;
-    stream->next_out = dest;
-
-    err = deflate(stream, Z_FINISH);
-    if (err != Z_STREAM_END) {
-        return -1;
-    }
-
-    return stream->next_out - dest;
-}
-
-/* Compress size bytes of data start at p and store the compressed
- * data to the buffer of f.
- *
- * Since the file is dummy file with empty_ops, return -1 if f has no space to
- * save the compressed data.
- */
-ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream,
-                                  const uint8_t *p, size_t size)
-{
-    ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t);
-
-    if (blen < compressBound(size)) {
-        return -1;
-    }
-
-    blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t),
-                              blen, p, size);
-    if (blen < 0) {
-        return -1;
-    }
-
-    qemu_put_be32(f, blen);
-    add_buf_to_iovec(f, blen);
-    return blen + sizeof(int32_t);
-}
-
 /* Put the data in the buffer of f_src to the buffer of f_des, and
  * then reset the buf_index of f_src to 0.
  */
@@ -846,3 +796,15 @@ void qemu_file_set_blocking(QEMUFile *f, bool block)
         f->ops->set_blocking(f->opaque, block, NULL);
     }
 }
+
+ssize_t qemu_put_compress_start(QEMUFile *f, uint8_t **dest_ptr)
+{
+    *dest_ptr = f->buf + f->buf_index + sizeof(int32_t);
+    return IO_BUF_SIZE - f->buf_index - sizeof(int32_t);
+}
+
+void qemu_put_compress_end(QEMUFile *f, unsigned int v)
+{
+    qemu_put_be32(f, v);
+    add_buf_to_iovec(f, v);
+}
migration/qemu-file.h +2 -2
index a9b6d6ccb7..1ac1566460 100644
@@ -138,8 +138,6 @@ bool qemu_file_is_writable(QEMUFile *f);
 
 size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t offset);
 size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size);
-ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream,
-                                  const uint8_t *p, size_t size);
 int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src);
 
 /*
@@ -166,6 +164,8 @@ void ram_control_before_iterate(QEMUFile *f, uint64_t flags);
 void ram_control_after_iterate(QEMUFile *f, uint64_t flags);
 void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data);
 
+ssize_t qemu_put_compress_start(QEMUFile *f, uint8_t **dest_ptr);
+void qemu_put_compress_end(QEMUFile *f, unsigned int v);
 /* Whenever this is found in the data stream, the flags
  * will be passed to ram_control_load_hook in the incoming-migration
  * side. This lets before_ram_iterate/after_ram_iterate add
migration/ram.c +61 -25

157 lines not on this page · raw message

Signature · 1 line
2.23.0
author
#5 lore raw

RFC PATCH 3/6 migration: Add multi-thread compress ops

In reply to Zeyu Jin

Add the MigrationCompressOps and MigrationDecompressOps structures to make the compression method configurable for multi-thread compression migration.

---
 migration/migration.c |   9 ++
 migration/migration.h |   1 +
 migration/ram.c       | 273 +++++++++++++++++++++++++++++-------------
 3 files changed, 203 insertions(+), 80 deletions(-)
migration/migration.c +9 -0
index d0da95fc0d..2c68012029 100644
@@ -2356,6 +2356,15 @@ int migrate_decompress_threads(void)
     return s->parameters.decompress_threads;
 }
 
+CompressMethod migrate_compress_method(void)
+{
+    MigrationState *s;
+
+    s = migrate_get_current();
+
+    return s->parameters.compress_method;
+}
+
 bool migrate_dirty_bitmaps(void)
 {
     MigrationState *s;
migration/migration.h +1 -0
index d096b77f74..e22b2ef840 100644
@@ -339,6 +339,7 @@ int migrate_compress_level(void);
 int migrate_compress_threads(void);
 int migrate_compress_wait_thread(void);
 int migrate_decompress_threads(void);
+CompressMethod migrate_compress_method(void);
 bool migrate_use_events(void);
 bool migrate_postcopy_blocktime(void);
 
migration/ram.c +193 -80

382 lines not on this page · raw message

Signature · 1 line
2.23.0
author
#6 lore raw

RFC PATCH 4/6 migration: Add zstd support in multi-thread compression

In reply to Zeyu Jin

This patch enables zstd option in multi-thread compression.

---
 hw/core/qdev-properties-system.c |   2 +-
 migration/ram.c                  | 128 ++++++++++++++++++++++++++++++-
 qapi/migration.json              |   2 +-
 3 files changed, 129 insertions(+), 3 deletions(-)
hw/core/qdev-properties-system.c +1 -1
index d757b2cd70..54760d94b3 100644
@@ -667,7 +667,7 @@ const PropertyInfo qdev_prop_multifd_compression = {
 const PropertyInfo qdev_prop_compress_method = {
     .name = "CompressMethod",
     .description = "multi-thread compression method, "
-                   "zlib",
+                   "zlib/zstd",
     .enum_table = &CompressMethod_lookup,
     .get = qdev_propinfo_get_enum,
     .set = qdev_propinfo_set_enum,
migration/ram.c +127 -1

166 lines not on this page · raw message

qapi/migration.json +1 -1
index d262683a38..ac6d06c683 100644
@@ -536,7 +536,7 @@
 #
 ##
 { 'enum': 'CompressMethod',
-  'data': [ 'zlib' ] }
+  'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
 
 ##
 # @BitmapMigrationBitmapAlias:
Signature · 1 line
2.23.0
author
#7 lore raw

RFC PATCH 5/6 migration: Add compress_level sanity check

In reply to Zeyu Jin

Zlib compression has level from 1 to 9. However Zstd compression has level from 1 to 22 (level >= 20 not recommanded). Let's do sanity check here to make sure a vaild compress_level is given by user.

---
 migration/migration.c | 32 ++++++++++++++++++++++++++++----
 1 file changed, 28 insertions(+), 4 deletions(-)
migration/migration.c +28 -4
index 2c68012029..6fc0c3b532 100644
@@ -1235,16 +1235,40 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params,
     }
 }
 
+static bool compress_level_check(MigrationParameters *params, Error **errp)
+{
+    switch (params->compress_method) {
+    case COMPRESS_METHOD_ZLIB:
+        if (params->compress_level > 9 || params->compress_level < 1) {
+            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "compress_level",
+                           "a value in the range of 0 to 9 for Zlib method");
+            return false;
+        }
+        break;
+#ifdef CONFIG_ZSTD
+    case COMPRESS_METHOD_ZSTD:
+        if (params->compress_level > 19 || params->compress_level < 1) {
+            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "compress_level",
+                        "a value in the range of 1 to 19 for Zstd method");
+            return false;
+        }
+        break;
+#endif
+    default:
+        error_setg(errp, "Checking compress_level failed for unknown reason");
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Check whether the parameters are valid. Error will be put into errp
  * (if provided). Return true if valid, otherwise false.
  */
 static bool migrate_params_check(MigrationParameters *params, Error **errp)
 {
-    if (params->has_compress_level &&
-        (params->compress_level > 9)) {
-        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "compress_level",
-                   "is invalid, it should be in the range of 0 to 9");
+    if (params->has_compress_level && !compress_level_check(params, errp)) {
         return false;
     }
 
Signature · 1 line
2.23.0
author
#8 lore raw

RFC PATCH 6/6 doc: Update multi-thread compression doc

In reply to Zeyu Jin

Modify the doc to fit the previous changes.

---
 docs/multi-thread-compression.txt | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)
docs/multi-thread-compression.txt +18 -13
index bb88c6bdf1..d429963cb0 100644
@@ -33,14 +33,15 @@ thread compression can be used to accelerate the compression process.
 
 The decompression speed of Zlib is at least 4 times as quick as
 compression, if the source and destination CPU have equal speed,
-keeping the compression thread count 4 times the decompression
-thread count can avoid resource waste.
+and you choose Zlib as compression method, keeping the compression
+thread count 4 times the decompression thread count can avoid resource waste.
 
 Compression level can be used to control the compression speed and the
-compression ratio. High compression ratio will take more time, level 0
-stands for no compression, level 1 stands for the best compression
-speed, and level 9 stands for the best compression ratio. Users can
-select a level number between 0 and 9.
+compression ratio. High compression ratio will take more time,
+level 1 stands for the best compression speed, and higher level means higher
+compression ration. For Zlib, users can select a level number between 0 and 9,
+where level 0 stands for no compression. For Zstd, users can select a
+level number between 1 and 22.
 
 
 When to use the multiple thread compression in live migration
@@ -116,16 +117,19 @@ to support the multiple thread compression migration:
 2. Activate compression on the source:
     {qemu} migrate_set_capability compress on
 
-3. Set the compression thread count on source:
+3. Set the compression method:
+    {qemu} migrate_set_parameter compress_method zstd
+
+4. Set the compression thread count on source:
     {qemu} migrate_set_parameter compress_threads 12
 
-4. Set the compression level on the source:
+5. Set the compression level on the source:
     {qemu} migrate_set_parameter compress_level 1
 
-5. Set the decompression thread count on destination:
+6. Set the decompression thread count on destination:
     {qemu} migrate_set_parameter decompress_threads 3
 
-6. Start outgoing migration:
+7. Start outgoing migration:
     {qemu} migrate -d tcp:destination.host:4444
     {qemu} info migrate
     Capabilities: ... compress: on
@@ -136,6 +140,7 @@ The following are the default settings:
     compress_threads: 8
     decompress_threads: 2
     compress_level: 1 (which means best speed)
+    compress_method: zlib
 
 So, only the first two steps are required to use the multiple
 thread compression in migration. You can do more if the default
@@ -143,7 +148,7 @@ settings are not appropriate.
 
 TODO
 ====
-Some faster (de)compression method such as LZ4 and Quicklz can help
-to reduce the CPU consumption when doing (de)compression. If using
-these faster (de)compression method, less (de)compression threads
+Comparing to Zlib, Some faster (de)compression method such as LZ4
+and Quicklz can help to reduce the CPU consumption when doing (de)compression.
+If using these faster (de)compression method, less (de)compression threads
 are needed when doing the migration.
Signature · 1 line
2.23.0
author
#9 lore raw
In reply to Zeyu Jin
Quoted 43 lines · Zeyu Jin

Currently we have both multi-thread compression and multifd to optimize live migration in Qemu. Mulit-thread compression deals with the situation where network bandwith is limited but cpu resource adequate. Multifd instead aims to take full advantage of network bandwith. Moreover it supports both zlib and zstd compression on each channel.

In this patch series, we did some code refactoring on multi-thread compression live migration and bring zstd compression method support for it.

Below is the test result of multi-thread compression live migration with different compress methods. Test result shows that zstd outperforms zlib by about 70%.

Migration Configuration:
Guest 8U 32G
compress-threads 8
decompress-threads 2
compress-level 1
bandwidth-limit 100Mbps

Test Result:

 +---------------------+--------------+-------------+
 |  compress method    |   zlib       |    zstd     |
 +---------------------+--------------+-------------+
 |  total time (ms)    |   75256      |    44187    |
 +---------------------+--------------+-------------+
 |  downtime(ms)       |   128        |    81       |
 +---------------------+--------------+-------------+
 |  transferred ram(kB)|   1576866    |    736117   |
 +---------------------+--------------+-------------+
 |  throughput(mbps)   |   172.06     |    137.16   |
 +---------------------+--------------+-------------+
 |  total ram(kB)      |   33685952   |    33685952 |
 +---------------------+--------------+-------------+

Zeyu Jin (6):
migration: Add multi-thread compress method
migration: Refactoring multi-thread compress migration
migration: Add multi-thread compress ops
migration: Add zstd support in multi-thread compression
migration: Add compress_level sanity check
doc: Update multi-thread compression doc

Quoted 12 lines
 docs/multi-thread-compression.txt |  31 ++-
 hw/core/qdev-properties-system.c  |  11 +
 include/hw/qdev-properties.h      |   4 +
 migration/migration.c             |  56 ++++-
 migration/migration.h             |   1 +
 migration/qemu-file.c             |  62 +----
 migration/qemu-file.h             |   4 +-
 migration/ram.c                   | 381 +++++++++++++++++++++++++-----
 monitor/hmp-cmds.c                |  12 +
 qapi/migration.json               |  26 +-
 10 files changed, 465 insertions(+), 123 deletions(-)

Ping