Implement QATzip compression method
v5 cover Implement QATzip compression method
v5:
- Rebase changes on top of 59084feb256c617063e0dbe7e64821ae8852d7cf
- Add documentations about migration with qatzip accerlation
- Remove multifd-qatzip-sw-fallback option
v4:
- Rebase changes on top of 1a2d52c7fcaeaaf4f2fe8d4d5183dccaeab67768
- Move the IOV initialization to qatzip implementation
- Only use qatzip to compress normal pages
v3:
- Rebase changes on top of master
- Merge two patches per Fabiano Rosas's comment
- Add versions into comments and documentations
v2:
- Rebase changes on top of recent multifd code changes.
- Use QATzip API 'qzMalloc' and 'qzFree' to allocate QAT buffers.
- Remove parameter tuning and use QATzip's defaults for better performance.
- Add parameter to enable QAT software fallback.
v1:
https://lists.nongnu.org/archive/html/qemu-devel/2023-12/msg03761.html
* Performance
We present updated performance results. For circumstantial reasons, v1 presented performance on a low-bandwidth (1Gbps) network.
Here, we present updated results with a similar setup as before but with two main differences:
1. Our machines have a ~50Gbps connection, tested using 'iperf3'.
2. We had a bug in our memory allocation causing us to only use ~1/2 of the VM's RAM. Now we properly allocate and fill nearly all of the VM's RAM.
Thus, the test setup is as follows:
We perform multifd live migration over TCP using a VM with 64GB memory. We prepare the machine's memory by powering it on, allocating a large amount of memory (60GB) as a single buffer, and filling the buffer with the repeated contents of the Silesia corpus[0]. This is in lieu of a more realistic memory snapshot, which proved troublesome to acquire.
We analyze CPU usage by averaging the output of 'top' every second during migration. This is admittedly imprecise, but we feel that it accurately portrays the different degrees of CPU usage of varying compression methods.
We present the latency, throughput, and CPU usage results for all of the compression methods, with varying numbers of multifd threads (4, 8, and 16).
[0] The Silesia corpus can be accessed here:
https://sun.aei.polsl.pl//~sdeor/index.php?page=silesia
** Results
4 multifd threads:
|---------------|---------------|----------------|---------|---------|
|method |time(sec) |throughput(mbps)|send cpu%|recv cpu%|
|---------------|---------------|----------------|---------|---------|
|qatzip | 23.13 | 8749.94 |117.50 |186.49 |
|---------------|---------------|----------------|---------|---------|
|zlib |254.35 | 771.87 |388.20 |144.40 |
|---------------|---------------|----------------|---------|---------|
|zstd | 54.52 | 3442.59 |414.59 |149.77 |
|---------------|---------------|----------------|---------|---------|
|none | 12.45 |43739.60 |159.71 |204.96 |
|---------------|---------------|----------------|---------|---------|8 multifd threads:
|---------------|---------------|----------------|---------|---------|
|method |time(sec) |throughput(mbps)|send cpu%|recv cpu%|
|---------------|---------------|----------------|---------|---------|
|qatzip | 16.91 |12306.52 |186.37 |391.84 |
|---------------|---------------|----------------|---------|---------|
|zlib |130.11 | 1508.89 |753.86 |289.35 |
|---------------|---------------|----------------|---------|---------|
|zstd | 27.57 | 6823.23 |786.83 |303.80 |
|---------------|---------------|----------------|---------|---------|
|none | 11.82 |46072.63 |163.74 |238.56 |
|---------------|---------------|----------------|---------|---------|16 multifd threads:
|---------------|---------------|----------------|---------|---------|
|method |time(sec) |throughput(mbps)|send cpu%|recv cpu%|
|---------------|---------------|----------------|---------|---------|
|qatzip |18.64 |11044.52 | 573.61 |437.65 |
|---------------|---------------|----------------|---------|---------|
|zlib |66.43 | 2955.79 |1469.68 |567.47 |
|---------------|---------------|----------------|---------|---------|
|zstd |14.17 |13290.66 |1504.08 |615.33 |
|---------------|---------------|----------------|---------|---------|
|none |16.82 |32363.26 | 180.74 |217.17 |
|---------------|---------------|----------------|---------|---------|** Observations
- In general, not using compression outperforms using compression in a non-network-bound environment.
- 'qatzip' outperforms other compression workers with 4 and 8 workers, achieving a ~91% latency reduction over 'zlib' with 4 workers, and a
~58% latency reduction over 'zstd' with 4 workers.
- 'qatzip' maintains comparable performance with 'zstd' at 16 workers, showing a ~32% increase in latency. This performance difference
becomes more noticeable with more workers, as CPU compression is highly parallelizable.
- 'qatzip' compression uses considerably less CPU than other compression methods. At 8 workers, 'qatzip' demonstrates a ~75% reduction in
compression CPU usage compared to 'zstd' and 'zlib'.
- 'qatzip' decompression CPU usage is less impressive, and is even slightly worse than 'zstd' and 'zlib' CPU usage at 4 and 16 workers.
Bryan Zhang (4):
meson: Introduce 'qatzip' feature to the build system
migration: Add migration parameters for QATzip
migration: Introduce 'qatzip' compression method tests/migration: Add integration test for 'qatzip' compression method
Yuan Liu (1):
docs/migration: add qatzip compression feature
docs/devel/migration/features.rst | 1 + docs/devel/migration/qatzip-compression.rst | 251 ++++++++++++ hw/core/qdev-properties-system.c | 6 +- meson.build | 10 + meson_options.txt | 2 + migration/meson.build | 1 + migration/migration-hmp-cmds.c | 4 + migration/multifd-qatzip.c | 403 ++++++++++++++++++++ migration/multifd.h | 5 +- migration/options.c | 34 ++ migration/options.h | 1 + qapi/migration.json | 21 + scripts/meson-buildoptions.sh | 3 + tests/qtest/meson.build | 4 + tests/qtest/migration-test.c | 35 ++ 15 files changed, 778 insertions(+), 3 deletions(-) create mode 100644 docs/devel/migration/qatzip-compression.rst create mode 100644 migration/multifd-qatzip.c
Signature · 1 line
Yichen Wang
PATCH v5 1/5 docs/migration: add qatzip compression feature
From: Yuan Liu <[email protected]>
add Intel QATzip compression method introduction
--- docs/devel/migration/features.rst | 1 + docs/devel/migration/qatzip-compression.rst | 251 ++++++++++++++++++++ 2 files changed, 252 insertions(+) create mode 100644 docs/devel/migration/qatzip-compression.rst
docs/devel/migration/features.rst +1 -0
@@ -14,3 +14,4 @@ Migration has plenty of features to support different use cases. CPR qpl-compression uadk-compression + qatzip-compression
docs/devel/migration/qatzip-compression.rst new file +251 -0
251 lines not on this page · raw message
Signature · 1 line
Yichen Wang
PATCH v5 2/5 meson: Introduce 'qatzip' feature to the build system
In reply to Yichen WangFrom: Bryan Zhang <[email protected]>
Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled.
--- meson.build | 10 ++++++++++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 15 insertions(+)
meson.build +10 -0
@@ -1244,6 +1244,14 @@ if not get_option('uadk').auto() or have_system uadk = declare_dependency(dependencies: [libwd, libwd_comp]) endif endif + +qatzip = not_found +if get_option('qatzip').enabled() + qatzip = dependency('qatzip', version: '>=1.1.2', + required: get_option('qatzip'), + method: 'pkg-config') +endif + virgl = not_found have_vhost_user_gpu = have_tools and host_os == 'linux' and pixman.found()
@@ -2378,6 +2386,7 @@ config_host_data.set('CONFIG_STATX_MNT_ID', has_statx_mnt_id) config_host_data.set('CONFIG_ZSTD', zstd.found()) config_host_data.set('CONFIG_QPL', qpl.found()) config_host_data.set('CONFIG_UADK', uadk.found()) +config_host_data.set('CONFIG_QATZIP', qatzip.found()) config_host_data.set('CONFIG_FUSE', fuse.found()) config_host_data.set('CONFIG_FUSE_LSEEK', fuse_lseek.found()) config_host_data.set('CONFIG_SPICE_PROTOCOL', spice_protocol.found())
@@ -4484,6 +4493,7 @@ summary_info += {'lzfse support': liblzfse} summary_info += {'zstd support': zstd} summary_info += {'Query Processing Library support': qpl} summary_info += {'UADK Library support': uadk} +summary_info += {'qatzip support': qatzip} summary_info += {'NUMA host support': numa} summary_info += {'capstone': capstone} summary_info += {'libpmem support': libpmem}
meson_options.txt +2 -0
@@ -261,6 +261,8 @@ option('qpl', type : 'feature', value : 'auto', description: 'Query Processing Library support') option('uadk', type : 'feature', value : 'auto', description: 'UADK Library support') +option('qatzip', type: 'feature', value: 'disabled', + description: 'QATzip compression support') option('fuse', type: 'feature', value: 'auto', description: 'FUSE block device export') option('fuse_lseek', type : 'feature', value : 'auto',
scripts/meson-buildoptions.sh +3 -0
@@ -163,6 +163,7 @@ meson_options_help() { printf "%s\n" ' pixman pixman support' printf "%s\n" ' plugins TCG plugins via shared library loading' printf "%s\n" ' png PNG support with libpng' + printf "%s\n" ' qatzip QATzip compression support' printf "%s\n" ' qcow1 qcow1 image format support' printf "%s\n" ' qed qed image format support' printf "%s\n" ' qga-vss build QGA VSS support (broken with MinGW)'
@@ -427,6 +428,8 @@ _meson_option_parse() { --enable-png) printf "%s" -Dpng=enabled ;; --disable-png) printf "%s" -Dpng=disabled ;; --prefix=*) quote_sh "-Dprefix=$2" ;; + --enable-qatzip) printf "%s" -Dqatzip=enabled ;; + --disable-qatzip) printf "%s" -Dqatzip=disabled ;; --enable-qcow1) printf "%s" -Dqcow1=enabled ;; --disable-qcow1) printf "%s" -Dqcow1=disabled ;; --enable-qed) printf "%s" -Dqed=enabled ;;
Signature · 1 line
Yichen Wang
PATCH v5 3/5 migration: Add migration parameters for QATzip
In reply to Yichen WangFrom: Bryan Zhang <[email protected]>
Adds support for migration parameters to control QATzip compression level and to enable/disable software fallback when QAT hardware is unavailable. This is a preparatory commit for a subsequent commit that will actually use QATzip compression.
--- migration/migration-hmp-cmds.c | 4 ++++ migration/options.c | 34 ++++++++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 18 ++++++++++++++++++ 4 files changed, 57 insertions(+)
migration/migration-hmp-cmds.c +4 -0
@@ -576,6 +576,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_multifd_zlib_level = true;visit_type_uint8 (v, param, &p->multifd_zlib_level, &err);break ; +case MIGRATION_PARAMETER_MULTIFD_QATZIP_LEVEL : + p->has_multifd_qatzip_level = true; +visit_type_uint8 (v, param, &p->multifd_qatzip_level, &err); +break ;case MIGRATION_PARAMETER_MULTIFD_ZSTD_LEVEL : p->has_multifd_zstd_level = true;visit_type_uint8 (v, param, &p->multifd_zstd_level, &err);
migration/options.c +34 -0
@@ -55,6 +55,13 @@#define DEFAULT_MIGRATE_MULTIFD_COMPRESSION MULTIFD_COMPRESSION_NONE/* 0: means nocompress, 1: best speed, ... 9: best compress ratio */ #define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1 +/* +* 1: best speed, ... 9: best compress ratio +* There is some nuance here. Refer to QATzip documentation to understand +* the mapping of QATzip levels to standard deflate levels. +*/ +#define DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL 1 +/* 0: means nocompress, 1: best speed, ... 20: best compress ratio */ #define DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL 1
@@ -123,6 +130,9 @@ Property migration_properties[] = {DEFINE_PROP_UINT8 ("multifd-zlib-level" , MigrationState, parameters.multifd_zlib_level,DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL ), +DEFINE_PROP_UINT8 ("multifd-qatzip-level" , MigrationState, + parameters.multifd_qatzip_level, +DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL ),DEFINE_PROP_UINT8 ("multifd-zstd-level" , MigrationState, parameters.multifd_zstd_level, DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL),
@@ -787,6 +797,13 @@ int migrate_multifd_zlib_level(void)return s->parameters.multifd_zlib_level; } +int migrate_multifd_qatzip_level (void ) +{ +MigrationState *s =migrate_get_current (); + +return s->parameters.multifd_qatzip_level; +} +int migrate_multifd_zstd_level (void ) {MigrationState *s =migrate_get_current ();
@@ -892,6 +909,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->multifd_compression = s->parameters.multifd_compression; params->has_multifd_zlib_level = true; params->multifd_zlib_level = s->parameters.multifd_zlib_level; + params->has_multifd_qatzip_level = true; + params->multifd_qatzip_level = s->parameters.multifd_qatzip_level; params->has_multifd_zstd_level = true; params->multifd_zstd_level = s->parameters.multifd_zstd_level; params->has_xbzrle_cache_size = true;
@@ -946,6 +965,7 @@ void migrate_params_init(MigrationParameters *params) params->has_multifd_channels = true; params->has_multifd_compression = true; params->has_multifd_zlib_level = true; + params->has_multifd_qatzip_level = true; params->has_multifd_zstd_level = true; params->has_xbzrle_cache_size = true; params->has_max_postcopy_bandwidth = true;
@@ -1038,6 +1058,14 @@ bool migrate_params_check(MigrationParameters *params, Error **errp)return false; } +if (params->has_multifd_qatzip_level && + ((params->multifd_qatzip_level >9 ) || + (params->multifd_qatzip_level <1 ))) { +error_setg (errp,QERR_INVALID_PARAMETER_VALUE ,"multifd_qatzip_level" , +"a value between 1 and 9" ); +return false; + } +if (params->has_multifd_zstd_level && (params->multifd_zstd_level >20 )) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zstd_level",
@@ -1195,6 +1223,9 @@ static void migrate_params_test_apply(MigrateSetParameters *params,if (params->has_multifd_compression) { dest->multifd_compression = params->multifd_compression; } +if (params->has_multifd_qatzip_level) { + dest->multifd_qatzip_level = params->multifd_qatzip_level; + }if (params->has_multifd_zlib_level) { dest->multifd_zlib_level = params->multifd_zlib_level; }
@@ -1315,6 +1346,9 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp)if (params->has_multifd_compression) { s->parameters.multifd_compression = params->multifd_compression; } +if (params->has_multifd_qatzip_level) { + s->parameters.multifd_qatzip_level = params->multifd_qatzip_level; + }if (params->has_multifd_zlib_level) { s->parameters.multifd_zlib_level = params->multifd_zlib_level; }
migration/options.h +1 -0
@@ -78,6 +78,7 @@ uint64_t migrate_max_postcopy_bandwidth(void);int migrate_multifd_channels (void );MultiFDCompression migrate_multifd_compression (void );int migrate_multifd_zlib_level (void ); +int migrate_multifd_qatzip_level (void );int migrate_multifd_zstd_level (void );uint8_t migrate_throttle_trigger_threshold (void );const char *migrate_tls_authz (void );
qapi/migration.json +18 -0
@@ -789,6 +789,11 @@ # speed, and 9 means best compression ratio which will consume # more CPU. Defaults to 1. (Since 5.0) # +# @multifd-qatzip-level: Set the compression level to be used in live +# migration. The level is an integer between 1 and 9, where 1 means +# the best compression speed, and 9 means the best compression +# ratio which will consume more CPU. Defaults to 1. (Since 9.1) +# # @multifd-zstd-level: Set the compression level to be used in live # migration, the compression level is an integer between 0 and 20, # where 0 means no compression, 1 means the best compression
@@ -849,6 +854,7 @@ 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level', 'multifd-zstd-level', + 'multifd-qatzip-level', 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable'] }, 'vcpu-dirty-limit',
@@ -964,6 +970,11 @@ # speed, and 9 means best compression ratio which will consume # more CPU. Defaults to 1. (Since 5.0) # +# @multifd-qatzip-level: Set the compression level to be used in live +# migration. The level is an integer between 1 and 9, where 1 means +# the best compression speed, and 9 means the best compression +# ratio which will consume more CPU. Defaults to 1. (Since 9.1) +# # @multifd-zstd-level: Set the compression level to be used in live # migration, the compression level is an integer between 0 and 20, # where 0 means no compression, 1 means the best compression
@@ -1037,6 +1048,7 @@ '*max-cpu-throttle': 'uint8', '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', + '*multifd-qatzip-level': 'uint8', '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64',
@@ -1168,6 +1180,11 @@ # speed, and 9 means best compression ratio which will consume # more CPU. Defaults to 1. (Since 5.0) # +# @multifd-qatzip-level: Set the compression level to be used in live +# migration. The level is an integer between 1 and 9, where 1 means +# the best compression speed, and 9 means the best compression +# ratio which will consume more CPU. Defaults to 1. (Since 9.1) +# # @multifd-zstd-level: Set the compression level to be used in live # migration, the compression level is an integer between 0 and 20, # where 0 means no compression, 1 means the best compression
@@ -1238,6 +1255,7 @@ '*max-cpu-throttle': 'uint8', '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', + '*multifd-qatzip-level': 'uint8', '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64',
Signature · 1 line
Yichen Wang
PATCH v5 4/5 migration: Introduce 'qatzip' compression method
In reply to Yichen WangFrom: Bryan Zhang <[email protected]>
Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.
--- hw/core/qdev-properties-system.c | 6 +- migration/meson.build | 1 + migration/multifd-qatzip.c | 403 +++++++++++++++++++++++++++++++ migration/multifd.h | 5 +- qapi/migration.json | 3 + tests/qtest/meson.build | 4 + 6 files changed, 419 insertions(+), 3 deletions(-) create mode 100644 migration/multifd-qatzip.c
hw/core/qdev-properties-system.c +5 -1
@@ -659,7 +659,11 @@ const PropertyInfo qdev_prop_fdc_drive_type = {const PropertyInfo qdev_prop_multifd_compression = { .name ="MultiFDCompression" , .description ="multifd_compression values, " -"none/zlib/zstd/qpl/uadk" , +"none/zlib/zstd/qpl/uadk" +#ifdef CONFIG_QATZIP +"/qatzip" +#endif + , .enum_table = &MultiFDCompression_lookup, .get = qdev_propinfo_get_enum, .set = qdev_propinfo_set_enum,
migration/meson.build +1 -0
@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c')) system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) system_ss.add(when: qpl, if_true: files('multifd-qpl.c')) system_ss.add(when: uadk, if_true: files('multifd-uadk.c')) +system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c')) specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c',
migration/multifd-qatzip.c new file +403 -0
403 lines not on this page · raw message
migration/multifd.h +3 -2
@@ -34,14 +34,15 @@ MultiFDRecvData *multifd_get_recv_data(void);/* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) -/* We reserve 4 bits for compression methods */ -#define MULTIFD_FLAG_COMPRESSION_MASK (0xf << 1) +/* We reserve 5 bits for compression methods */ +#define MULTIFD_FLAG_COMPRESSION_MASK (0x1f << 1)/* we need to be compatible. Before compression value was 0 */ #define MULTIFD_FLAG_NOCOMP (0 << 1)#define MULTIFD_FLAG_ZLIB (1 << 1)#define MULTIFD_FLAG_ZSTD (2 << 1)#define MULTIFD_FLAG_QPL (4 << 1)#define MULTIFD_FLAG_UADK (8 << 1) +#define MULTIFD_FLAG_QATZIP (16 << 1)/* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024)
qapi/migration.json +3 -0
@@ -558,6 +558,8 @@ # # @zstd: use zstd compression method. # +# @qatzip: use qatzip compression method. (Since 9.1) +# # @qpl: use qpl compression method. Query Processing Library(qpl) is # based on the deflate compression algorithm and use the Intel # In-Memory Analytics Accelerator(IAA) accelerated compression
@@ -570,6 +572,7 @@ { 'enum': 'MultiFDCompression', 'data': [ 'none', 'zlib', { 'name': 'zstd', 'if': 'CONFIG_ZSTD' }, + { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'}, { 'name': 'qpl', 'if': 'CONFIG_QPL' }, { 'name': 'uadk', 'if': 'CONFIG_UADK' } ] }
tests/qtest/meson.build +4 -0
@@ -327,6 +327,10 @@ if gnutls.found() endif endif +if qatzip.found() + migration_files += [qatzip] +endif + qtests = { 'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'], 'cdrom-test': files('boot-sector.c'),
Signature · 1 line
Yichen Wang
Re: [PATCH v5 4/5] migration: Introduce 'qatzip' compression method
Quoted 24 lines · Yichen Wang
From: Bryan Zhang <[email protected]>
Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.
Signed-off-by Bryan Zhang <[email protected]>Signed-off-by Hao Xiang <[email protected]>Signed-off-by Yichen Wang <[email protected]>--- hw/core/qdev-properties-system.c | 6 +- migration/meson.build | 1 + migration/multifd-qatzip.c | 403 +++++++++++++++++++++++++++++++ migration/multifd.h | 5 +- qapi/migration.json | 3 + tests/qtest/meson.build | 4 + 6 files changed, 419 insertions(+), 3 deletions(-) create mode 100644 migration/multifd-qatzip.chw/core/qdev-properties-system.c +0
-0
hw/core/qdev-properties-system.c const PropertyInfo qdev_prop_fdc_drive_type = { · line 665
Quoted 8 lines
hw/core/qdev-properties-system.c +4
-1@@ -659,7 +659,11 @@ const PropertyInfo qdev_prop_fdc_drive_type = {const PropertyInfo qdev_prop_multifd_compression = { .name ="MultiFDCompression" , .description ="multifd_compression values, " -"none/zlib/zstd/qpl/uadk" , +"none/zlib/zstd/qpl/uadk" +#ifdef CONFIG_QATZIP +"/qatzip" +#endif
It seems the other accelerators don't need the ifdef. What's different here?
Quoted 77 lines
hw/core/qdev-properties-system.c +1
-0+ , .enum_table = &MultiFDCompression_lookup, .get = qdev_propinfo_get_enum, .set = qdev_propinfo_set_enum,migration/meson.build +1
-0@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c')) system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) system_ss.add(when: qpl, if_true: files('multifd-qpl.c')) system_ss.add(when: uadk, if_true: files('multifd-uadk.c')) +system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c')) specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c',migration/multifd-qatzip.c new file +55
-0@@ -0,0 +1,403 @@ +/* +* Multifd QATzip compression implementation +* +* Copyright (c) Bytedance +* +* Authors: +* Bryan Zhang <[email protected]> +* Hao Xiang <[email protected]> +* Yichen Wang <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#include "qemu/osdep.h" +#include "exec/ramblock.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qapi/qapi-types-migration.h" +#include "options.h" +#include "multifd.h" +#include <qatzip.h> + +typedef struct { +/* +* Unique session for use with QATzip API +*/ +QzSession_T sess; + +/* +* For compression: Buffer for pages to compress +* For decompression: Buffer for data to decompress +*/ +uint8_t *in_buf; +uint32_t in_len; + +/* +* For compression: Output buffer of compressed data +* For decompression: Output buffer of decompressed data +*/ +uint8_t *out_buf; +uint32_t out_len; +}QatzipData ; + +/** +* qatzip_send_setup: Set up QATzip session and private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_send_setup (MultiFDSendParams *p,Error **errp) +{ +QatzipData *q; +QzSessionParamsDeflate_T params;
migration/multifd-qatzip.c line 65
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+const char *err_msg; +int ret; + + q =g_new0 (QatzipData,1 ); + p->compress_data = q; +/* We need one extra place for the packet header */ + p->iov = g_new0(struct iovec, 2); + +/* Prefer without sw_fallback because of bad performance with sw_fallback. +* Warn if sw_fallback needs to be used. */
Please run scripts/checkpatch.pl on your series. This style of comments should have been flagged as non-conformant with our guidelines.
migration/multifd-qatzip.c line 69
Quoted 4 lines
migration/multifd-qatzip.c new file +4
-0+ ret =qzInit (&q->sess, false); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ +warn_report ("Initilizing QAT with sw_fallback..." );
This will warn for each multifd channel, maybe use warn_report_once instead. Also s/Initilizing/Initializing/ and let's spell out "software fallback".
Quoted 71 lines
migration/multifd-qatzip.c new file +71
-0+ ret =qzInit (&q->sess, true); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ + err_msg ="qzInit failed" ; + goto err_free_q; + } + } + + ret =qzGetDefaultsDeflate (¶ms); +if (ret !=QZ_OK ) { + err_msg ="qzGetDefaultsDeflate failed" ; + goto err_close; + } + +/* Make sure to use configured QATzip compression level. */ + params.common_params.comp_lvl =migrate_multifd_qatzip_level (); + + ret =qzSetupSessionDeflate (&q->sess, ¶ms); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { + err_msg ="qzSetupSessionDeflate failed" ; + goto err_close; + } + +if (MULTIFD_PACKET_SIZE >UINT32_MAX ) { + err_msg ="packet size too large for QAT" ; + goto err_close; + } + + q->in_len =MULTIFD_PACKET_SIZE ; + q->in_buf =qzMalloc (q->in_len,0 ,PINNED_MEM ); +if (!q->in_buf) { + err_msg ="qzMalloc failed" ; + goto err_close; + } + + q->out_len =qzMaxCompressedLength (MULTIFD_PACKET_SIZE , &q->sess); + q->out_buf =qzMalloc (q->out_len,0 ,PINNED_MEM ); +if (!q->out_buf) { + err_msg ="qzMalloc failed" ; + goto err_free_inbuf; + } + +return 0 ; + +err_free_inbuf: +qzFree (q->in_buf); +err_close: +qzClose (&q->sess); +err_free_q: +g_free (q); +g_free (p->iov); + p->iov =NULL ; + p->compress_data =NULL ; +error_setg (errp,"multifd %u: %s" , p->id, err_msg); +return -1 ; +} + +/** +* qatzip_send_cleanup: Tear down QATzip session and release private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return None +*/ +static void qatzip_send_cleanup (MultiFDSendParams *p,Error **errp) +{ +QatzipData *q = p->compress_data; +const char *err_msg; +int ret; + + ret =qzTeardownSession (&q->sess);
migration/multifd-qatzip.c line 150
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+if (ret !=QZ_OK ) { + err_msg ="qzTeardownSession failed" ; + goto err; + } + + ret =qzClose (&q->sess); +if (ret !=QZ_OK ) { + err_msg ="qzClose failed" ; + goto err; + }
Can qzClose() be called twice on the same session pointer? It's possible that we have already failed at multifd_send_setup() and still reach here.
And what about qzTeardownSession()? Can it cope with an already closed session?
And what about the sessions that never got created because we might have exited early at the ops->send_setup() loop?
migration/multifd-qatzip.c line 155
Quoted 5 lines
migration/multifd-qatzip.c new file +5
-0+ +qzFree (q->in_buf); + q->in_buf =NULL ; +qzFree (q->out_buf); + q->out_buf =NULL ;
These will double free here if send_setup has already freed.
Quoted 24 lines
migration/multifd-qatzip.c new file +24
-0+g_free (p->iov); + p->iov =NULL ; +g_free (p->compress_data); + p->compress_data =NULL ; +return ; + +err: +error_setg (errp,"multifd %u: %s" , p->id, err_msg); +} + +/** +* qatzip_send_prepare: Compress pages and update IO channel info. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_send_prepare (MultiFDSendParams *p,Error **errp) +{ +MultiFDPages_t *pages = p->pages; +QatzipData *q = p->compress_data; +int ret; +unsigned int in_len, out_len; +
migration/multifd-qatzip.c line 189
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+if (!multifd_send_prepare_common (p)) { + goto out; + } + +/* Unlike other multifd compression implementations, we use a +* non-streaming API and place all the data into one buffer, rather than +* sending each page to the compression API at a time. */ +for (int i =0 ; i < pages->normal_num; i++) { + memcpy(q->in_buf + (i * p->page_size), + p->pages->block->host + pages->offset[i],
pages->block->host
Quoted 55 lines
migration/multifd-qatzip.c new file +55
-0+ p->page_size); + } + + in_len = pages->normal_num * p->page_size; +if (in_len > q->in_len) { +error_setg (errp,"multifd %u: unexpectedly large input" , p->id); +return -1 ; + } + out_len = q->out_len; + +/* +* Unlike other multifd compression implementations, we use a non-streaming +* API and place all the data into one buffer, rather than sending each page +* to the compression API at a time. Based on initial benchmarks, the +* non-streaming API outperforms the streaming API. Plus, the logic in QEMU +* is friendly to using the non-streaming API anyway. If either of these +* statements becomes no longer true, we can revisit adding a streaming +* implementation. +*/ + ret =qzCompress (&q->sess, q->in_buf, &in_len, q->out_buf, &out_len,1 ); +if (ret !=QZ_OK ) { +error_setg (errp,"multifd %u: QATzip returned %d instead of QZ_OK" , + p->id, ret); +return -1 ; + } +if (in_len != pages->normal_num * p->page_size) { +error_setg (errp,"multifd %u: QATzip failed to compress all input" , + p->id); +return -1 ; + } + + p->iov[p->iovs_num].iov_base = q->out_buf; + p->iov[p->iovs_num].iov_len = out_len; + p->iovs_num++; + p->next_packet_size = out_len; + +out: + p->flags |=MULTIFD_FLAG_QATZIP ; +multifd_send_fill_packet (p); +return 0 ; +} + +/** +* qatzip_recv_setup: Set up QATzip session and allocate private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_recv_setup (MultiFDRecvParams *p,Error **errp) +{ +QatzipData *q; +QzSessionParamsDeflate_T params; +const char *err_msg; +int ret;
migration/multifd-qatzip.c line 254
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+ + q =g_new0 (QatzipData,1 ); + p->compress_data = q; + +/* Prefer without sw_fallback because of bad performance with sw_fallback. +* Warn if sw_fallback needs to be used. */ + ret =qzInit (&q->sess, false); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ +warn_report ("Initilizing QAT with sw_fallback..." );
Same here. Also please add a hint that this is recv and the other one is send. It helps with debug.
Quoted 13 lines
migration/multifd-qatzip.c new file +13
-0+ ret =qzInit (&q->sess, true); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ + err_msg ="qzInit failed" ; + goto err_free_q; + } + } + + ret =qzGetDefaultsDeflate (¶ms); +if (ret !=QZ_OK ) { + err_msg ="qzGetDefaultsDeflate failed" ; + goto err_close; + }
migration/multifd-qatzip.c line 277
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+ + ret =qzSetupSessionDeflate (&q->sess, ¶ms); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { + err_msg ="qzSetupSessionDeflate failed" ; + goto err_close; + } + +/* +* Mimic multifd-zlib, which reserves extra space for the +* incoming packet.
I'd put the actual rationale here. It will also help in the future to spot that this implementation doesn't send uncompressed pages in case the compression got too big.
Quoted 17 lines
migration/multifd-qatzip.c new file +17
-0+*/ + q->in_len =MULTIFD_PACKET_SIZE *2 ; +/* PINNED_MEM is an enum from qatzip headers, which means to use +* kzalloc_node() to allocate memory for QAT DMA purposes. */ + q->in_buf =qzMalloc (q->in_len,0 ,PINNED_MEM ); +if (!q->in_buf) { + err_msg ="qzMalloc failed" ; + goto err_close; + } + + q->out_len =MULTIFD_PACKET_SIZE ; + q->out_buf =qzMalloc (q->out_len,0 ,PINNED_MEM ); +if (!q->out_buf) { + err_msg ="qzMalloc failed" ; + goto err_free_inbuf; + } +
migration/multifd-qatzip.c line 304
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+return 0 ; + +err_free_inbuf: +qzFree (q->in_buf); +err_close: +qzClose (&q->sess); +err_free_q: +g_free (q); + p->compress_data =NULL ; +error_setg (errp,"multifd %u: %s" , p->id, err_msg);
Or maybe put the recv/send information on this string.
Quoted 157 lines
migration/multifd-qatzip.c new file +99
-0+return -1 ; +} + +/** +* qatzip_recv_cleanup: Tear down QATzip session and release private buffers. +* +* @param p Multifd channel params +* @return None +*/ +static void qatzip_recv_cleanup (MultiFDRecvParams *p) +{ +QatzipData *q = p->compress_data; + +/* Ignoring return values here due to function signature. */ +qzTeardownSession (&q->sess); +qzClose (&q->sess); +qzFree (q->in_buf); +qzFree (q->out_buf); +g_free (p->compress_data); +} + + +/** +* qatzip_recv: Decompress pages and copy them to the appropriate +* locations. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_recv (MultiFDRecvParams *p,Error **errp) +{ +QatzipData *q = p->compress_data; +int ret; +unsigned int in_len, out_len; +uint32_t in_size = p->next_packet_size; +uint32_t expected_size = p->normal_num * p->page_size; +uint32_t flags = p->flags &MULTIFD_FLAG_COMPRESSION_MASK ; + +if (in_size > q->in_len) { +error_setg (errp,"multifd %u: received unexpectedly large packet" , + p->id); +return -1 ; + } + +if (flags !=MULTIFD_FLAG_QATZIP ) { +error_setg (errp,"multifd %u: flags received %x flags expected %x" , + p->id, flags,MULTIFD_FLAG_QATZIP ); +return -1 ; + } + +multifd_recv_zero_page_process (p); +if (!p->normal_num) { +assert (in_size ==0 ); +return 0 ; + } + + ret =qio_channel_read_all (p->c, (void *)q->in_buf, in_size, errp); +if (ret !=0 ) { +return ret; + } + + in_len = in_size; + out_len = q->out_len; + ret =qzDecompress (&q->sess, q->in_buf, &in_len, q->out_buf, &out_len); +if (ret !=QZ_OK ) { +error_setg (errp,"multifd %u: qzDecompress failed" , p->id); +return -1 ; + } +if (out_len != expected_size) { +error_setg (errp,"multifd %u: packet size received %u size expected %u" , + p->id, out_len, expected_size); +return -1 ; + } + +/* Copy each page to its appropriate location. */ +for (int i =0 ; i < p->normal_num; i++) { +memcpy (p->host + p->normal[i], + q->out_buf + p->page_size * i, + p->page_size); + } +return 0 ; +} + +static MultiFDMethods multifd_qatzip_ops = { + .send_setup = qatzip_send_setup, + .send_cleanup = qatzip_send_cleanup, + .send_prepare = qatzip_send_prepare, + .recv_setup = qatzip_recv_setup, + .recv_cleanup = qatzip_recv_cleanup, + .recv = qatzip_recv +}; + +static void multifd_qatzip_register (void ) +{ +multifd_register_ops (MULTIFD_COMPRESSION_QATZIP , &multifd_qatzip_ops); +} + +migration_init (multifd_qatzip_register);migration/multifd.h +3
-2@@ -34,14 +34,15 @@ MultiFDRecvData *multifd_get_recv_data(void);/* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) -/* We reserve 4 bits for compression methods */ -#define MULTIFD_FLAG_COMPRESSION_MASK (0xf << 1) +/* We reserve 5 bits for compression methods */ +#define MULTIFD_FLAG_COMPRESSION_MASK (0x1f << 1)/* we need to be compatible. Before compression value was 0 */ #define MULTIFD_FLAG_NOCOMP (0 << 1)#define MULTIFD_FLAG_ZLIB (1 << 1)#define MULTIFD_FLAG_ZSTD (2 << 1)#define MULTIFD_FLAG_QPL (4 << 1)#define MULTIFD_FLAG_UADK (8 << 1) +#define MULTIFD_FLAG_QATZIP (16 << 1)/* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024)qapi/migration.json +3
-0@@ -558,6 +558,8 @@ # # @zstd: use zstd compression method. # +# @qatzip: use qatzip compression method. (Since 9.1) +# # @qpl: use qpl compression method. Query Processing Library(qpl) is # based on the deflate compression algorithm and use the Intel # In-Memory Analytics Accelerator(IAA) accelerated compression@@ -570,6 +572,7 @@ { 'enum': 'MultiFDCompression', 'data': [ 'none', 'zlib', { 'name': 'zstd', 'if': 'CONFIG_ZSTD' }, + { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'}, { 'name': 'qpl', 'if': 'CONFIG_QPL' }, { 'name': 'uadk', 'if': 'CONFIG_UADK' } ] }tests/qtest/meson.build +4
-0@@ -327,6 +327,10 @@ if gnutls.found() endif endif +if qatzip.found() + migration_files += [qatzip] +endif + qtests = { 'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'], 'cdrom-test': files('boot-sector.c'),
Re: [External] Re: [PATCH v5 4/5] migration: Introduce 'qatzip' compression method
Quoted 22 lines · Fabiano Rosas
Quoted 19 lines · Yichen Wang
From: Bryan Zhang <[email protected]>
Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.
Signed-off-by Bryan Zhang <[email protected]>Signed-off-by Hao Xiang <[email protected]>Signed-off-by Yichen Wang <[email protected]>--- hw/core/qdev-properties-system.c | 6 +- migration/meson.build | 1 + migration/multifd-qatzip.c | 403 +++++++++++++++++++++++++++++++ migration/multifd.h | 5 +- qapi/migration.json | 3 + tests/qtest/meson.build | 4 + 6 files changed, 419 insertions(+), 3 deletions(-) create mode 100644 migration/multifd-qatzip.c
Quoted 16 lines
hw/core/qdev-properties-system.c const PropertyInfo qdev_prop_fdc_drive_type = { · line 665
Quoted 13 lines
hw/core/qdev-properties-system.c +4
-1@@ -659,7 +659,11 @@ const PropertyInfo qdev_prop_fdc_drive_type = {const PropertyInfo qdev_prop_multifd_compression = { .name ="MultiFDCompression" , .description ="multifd_compression values, " -"none/zlib/zstd/qpl/uadk" , +"none/zlib/zstd/qpl/uadk" +#ifdef CONFIG_QATZIP +"/qatzip" +#endifIt seems the other accelerators don't need the ifdef. What's different here?
Just changed and align to other methods. Will fix in next version.
Quoted 83 lines
Quoted 83 lines
hw/core/qdev-properties-system.c +1
-0+ , .enum_table = &MultiFDCompression_lookup, .get = qdev_propinfo_get_enum, .set = qdev_propinfo_set_enum,migration/meson.build +1
-0@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c')) system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) system_ss.add(when: qpl, if_true: files('multifd-qpl.c')) system_ss.add(when: uadk, if_true: files('multifd-uadk.c')) +system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c')) specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c',migration/multifd-qatzip.c new file +60
-0@@ -0,0 +1,403 @@ +/* +* Multifd QATzip compression implementation +* +* Copyright (c) Bytedance +* +* Authors: +* Bryan Zhang <[email protected]> +* Hao Xiang <[email protected]> +* Yichen Wang <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#include "qemu/osdep.h" +#include "exec/ramblock.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qapi/qapi-types-migration.h" +#include "options.h" +#include "multifd.h" +#include <qatzip.h> + +typedef struct { +/* +* Unique session for use with QATzip API +*/ +QzSession_T sess; + +/* +* For compression: Buffer for pages to compress +* For decompression: Buffer for data to decompress +*/ +uint8_t *in_buf; +uint32_t in_len; + +/* +* For compression: Output buffer of compressed data +* For decompression: Output buffer of decompressed data +*/ +uint8_t *out_buf; +uint32_t out_len; +}QatzipData ; + +/** +* qatzip_send_setup: Set up QATzip session and private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_send_setup (MultiFDSendParams *p,Error **errp) +{ +QatzipData *q; +QzSessionParamsDeflate_T params; +const char *err_msg; +int ret; + + q =g_new0 (QatzipData,1 ); + p->compress_data = q;
Quoted 8 lines
migration/multifd-qatzip.c line 65
Quoted 5 lines
migration/multifd-qatzip.c new file +5
-0+/* We need one extra place for the packet header */ + p->iov = g_new0(struct iovec, 2); + +/* Prefer without sw_fallback because of bad performance with sw_fallback. +* Warn if sw_fallback needs to be used. */ Please run
scripts/checkpatch.plon your series. This style of comments should have been flagged as non-conformant with our guidelines.
Sorry for that. Will fix in next version.
Quoted 10 lines
migration/multifd-qatzip.c line 69
Quoted 5 lines
migration/multifd-qatzip.c new file +4
-0+ ret =qzInit (&q->sess, false); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ +warn_report ("Initilizing QAT with sw_fallback..." );This will warn for each multifd channel, maybe use
warn_report_onceinstead. Also s/Initilizing/Initializing/ and let's spell out "software fallback".
Will fix in next version.
Quoted 86 lines
Quoted 71 lines
migration/multifd-qatzip.c new file +71
-0+ ret =qzInit (&q->sess, true); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ + err_msg ="qzInit failed" ; + goto err_free_q; + } + } + + ret =qzGetDefaultsDeflate (¶ms); +if (ret !=QZ_OK ) { + err_msg ="qzGetDefaultsDeflate failed" ; + goto err_close; + } + +/* Make sure to use configured QATzip compression level. */ + params.common_params.comp_lvl =migrate_multifd_qatzip_level (); + + ret =qzSetupSessionDeflate (&q->sess, ¶ms); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { + err_msg ="qzSetupSessionDeflate failed" ; + goto err_close; + } + +if (MULTIFD_PACKET_SIZE >UINT32_MAX ) { + err_msg ="packet size too large for QAT" ; + goto err_close; + } + + q->in_len =MULTIFD_PACKET_SIZE ; + q->in_buf =qzMalloc (q->in_len,0 ,PINNED_MEM ); +if (!q->in_buf) { + err_msg ="qzMalloc failed" ; + goto err_close; + } + + q->out_len =qzMaxCompressedLength (MULTIFD_PACKET_SIZE , &q->sess); + q->out_buf =qzMalloc (q->out_len,0 ,PINNED_MEM ); +if (!q->out_buf) { + err_msg ="qzMalloc failed" ; + goto err_free_inbuf; + } + +return 0 ; + +err_free_inbuf: +qzFree (q->in_buf); +err_close: +qzClose (&q->sess); +err_free_q: +g_free (q); +g_free (p->iov); + p->iov =NULL ; + p->compress_data =NULL ; +error_setg (errp,"multifd %u: %s" , p->id, err_msg); +return -1 ; +} + +/** +* qatzip_send_cleanup: Tear down QATzip session and release private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return None +*/ +static void qatzip_send_cleanup (MultiFDSendParams *p,Error **errp) +{ +QatzipData *q = p->compress_data; +const char *err_msg; +int ret; + + ret =qzTeardownSession (&q->sess);migration/multifd-qatzip.c line 150
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+if (ret !=QZ_OK ) { + err_msg ="qzTeardownSession failed" ; + goto err; + } + + ret =qzClose (&q->sess); +if (ret !=QZ_OK ) { + err_msg ="qzClose failed" ; + goto err; + }Can
qzClose()be called twice on the same session pointer? It's possible that we have already failed atmultifd_send_setup()and still reach here.
Quoted 6 lines
And what about
qzTeardownSession()? Can it cope with an already closed session?And what about the sessions that never got created because we might have exited early at the
ops->send_setup()loop?
qzTeardownSession() and qzClose() are safe to call on NULL pointers. But thanks to your comments which corrected my understanding. These patch was wrote under the impression that when setup() failed, cleanup() won't be fired. After learning in gdb, apparently I was wrong. The cleanup() will be called from another thread, which will be called regardless if setup() returns zero or non-zero. I will rewrite the setup()/cleanup() logics in my next patchset.
Quoted 36 lines
migration/multifd-qatzip.c line 155
Quoted 5 lines
migration/multifd-qatzip.c new file +5
-0+ +qzFree (q->in_buf); + q->in_buf =NULL ; +qzFree (q->out_buf); + q->out_buf =NULL ;These will double free here if
send_setuphas already freed.Quoted 28 lines
migration/multifd-qatzip.c new file +28
-0+g_free (p->iov); + p->iov =NULL ; +g_free (p->compress_data); + p->compress_data =NULL ; +return ; + +err: +error_setg (errp,"multifd %u: %s" , p->id, err_msg); +} + +/** +* qatzip_send_prepare: Compress pages and update IO channel info. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_send_prepare (MultiFDSendParams *p,Error **errp) +{ +MultiFDPages_t *pages = p->pages; +QatzipData *q = p->compress_data; +int ret; +unsigned int in_len, out_len; + +if (!multifd_send_prepare_common (p)) { + goto out; + } +
Quoted 9 lines
migration/multifd-qatzip.c line 189
Quoted 6 lines
migration/multifd-qatzip.c new file +6
-0+/* Unlike other multifd compression implementations, we use a +* non-streaming API and place all the data into one buffer, rather than +* sending each page to the compression API at a time. */ +for (int i =0 ; i < pages->normal_num; i++) { + memcpy(q->in_buf + (i * p->page_size), + p->pages->block->host + pages->offset[i],
pages->block->host
I am not sure if I understand your comment here?
Quoted 60 lines
Quoted 60 lines
migration/multifd-qatzip.c new file +60
-0+ p->page_size); + } + + in_len = pages->normal_num * p->page_size; +if (in_len > q->in_len) { +error_setg (errp,"multifd %u: unexpectedly large input" , p->id); +return -1 ; + } + out_len = q->out_len; + +/* +* Unlike other multifd compression implementations, we use a non-streaming +* API and place all the data into one buffer, rather than sending each page +* to the compression API at a time. Based on initial benchmarks, the +* non-streaming API outperforms the streaming API. Plus, the logic in QEMU +* is friendly to using the non-streaming API anyway. If either of these +* statements becomes no longer true, we can revisit adding a streaming +* implementation. +*/ + ret =qzCompress (&q->sess, q->in_buf, &in_len, q->out_buf, &out_len,1 ); +if (ret !=QZ_OK ) { +error_setg (errp,"multifd %u: QATzip returned %d instead of QZ_OK" , + p->id, ret); +return -1 ; + } +if (in_len != pages->normal_num * p->page_size) { +error_setg (errp,"multifd %u: QATzip failed to compress all input" , + p->id); +return -1 ; + } + + p->iov[p->iovs_num].iov_base = q->out_buf; + p->iov[p->iovs_num].iov_len = out_len; + p->iovs_num++; + p->next_packet_size = out_len; + +out: + p->flags |=MULTIFD_FLAG_QATZIP ; +multifd_send_fill_packet (p); +return 0 ; +} + +/** +* qatzip_recv_setup: Set up QATzip session and allocate private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_recv_setup (MultiFDRecvParams *p,Error **errp) +{ +QatzipData *q; +QzSessionParamsDeflate_T params; +const char *err_msg; +int ret; + + q =g_new0 (QatzipData,1 ); + p->compress_data = q; + +/* Prefer without sw_fallback because of bad performance with sw_fallback.
Quoted 8 lines
migration/multifd-qatzip.c line 69
Quoted 5 lines
migration/multifd-qatzip.c new file +5
-0+* Warn if sw_fallback needs to be used. */ + ret =qzInit (&q->sess, false); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ +warn_report ("Initilizing QAT with sw_fallback..." );Same here. Also please add a hint that this is recv and the other one is send. It helps with debug.
Will fix in next version.
Quoted 20 lines
Quoted 20 lines
migration/multifd-qatzip.c new file +19
-0+ ret =qzInit (&q->sess, true); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ + err_msg ="qzInit failed" ; + goto err_free_q; + } + } + + ret =qzGetDefaultsDeflate (¶ms); +if (ret !=QZ_OK ) { + err_msg ="qzGetDefaultsDeflate failed" ; + goto err_close; + } + + ret =qzSetupSessionDeflate (&q->sess, ¶ms); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { + err_msg ="qzSetupSessionDeflate failed" ; + goto err_close; + }
Quoted 8 lines
migration/multifd-qatzip.c line 277
Quoted 4 lines
migration/multifd-qatzip.c new file +4
-0+ +/* +* Mimic multifd-zlib, which reserves extra space for the +* incoming packet. I'd put the actual rationale here. It will also help in the future to spot that this implementation doesn't send uncompressed pages in case the compression got too big.
Will fix in next version.
Quoted 22 lines
Quoted 22 lines
migration/multifd-qatzip.c new file +21
-0+*/ + q->in_len =MULTIFD_PACKET_SIZE *2 ; +/* PINNED_MEM is an enum from qatzip headers, which means to use +* kzalloc_node() to allocate memory for QAT DMA purposes. */ + q->in_buf =qzMalloc (q->in_len,0 ,PINNED_MEM ); +if (!q->in_buf) { + err_msg ="qzMalloc failed" ; + goto err_close; + } + + q->out_len =MULTIFD_PACKET_SIZE ; + q->out_buf =qzMalloc (q->out_len,0 ,PINNED_MEM ); +if (!q->out_buf) { + err_msg ="qzMalloc failed" ; + goto err_free_inbuf; + } + +return 0 ; + +err_free_inbuf: +qzFree (q->in_buf);
Quoted 9 lines
migration/multifd-qatzip.c line 123
Quoted 6 lines
migration/multifd-qatzip.c new file +6
-0+err_close: +qzClose (&q->sess); +err_free_q: +g_free (q); + p->compress_data =NULL ; +error_setg (errp,"multifd %u: %s" , p->id, err_msg);Or maybe put the recv/send information on this string.
Will fix in next version.
Quoted 157 lines
Quoted 157 lines
migration/multifd-qatzip.c new file +99
-0+return -1 ; +} + +/** +* qatzip_recv_cleanup: Tear down QATzip session and release private buffers. +* +* @param p Multifd channel params +* @return None +*/ +static void qatzip_recv_cleanup (MultiFDRecvParams *p) +{ +QatzipData *q = p->compress_data; + +/* Ignoring return values here due to function signature. */ +qzTeardownSession (&q->sess); +qzClose (&q->sess); +qzFree (q->in_buf); +qzFree (q->out_buf); +g_free (p->compress_data); +} + + +/** +* qatzip_recv: Decompress pages and copy them to the appropriate +* locations. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_recv (MultiFDRecvParams *p,Error **errp) +{ +QatzipData *q = p->compress_data; +int ret; +unsigned int in_len, out_len; +uint32_t in_size = p->next_packet_size; +uint32_t expected_size = p->normal_num * p->page_size; +uint32_t flags = p->flags &MULTIFD_FLAG_COMPRESSION_MASK ; + +if (in_size > q->in_len) { +error_setg (errp,"multifd %u: received unexpectedly large packet" , + p->id); +return -1 ; + } + +if (flags !=MULTIFD_FLAG_QATZIP ) { +error_setg (errp,"multifd %u: flags received %x flags expected %x" , + p->id, flags,MULTIFD_FLAG_QATZIP ); +return -1 ; + } + +multifd_recv_zero_page_process (p); +if (!p->normal_num) { +assert (in_size ==0 ); +return 0 ; + } + + ret =qio_channel_read_all (p->c, (void *)q->in_buf, in_size, errp); +if (ret !=0 ) { +return ret; + } + + in_len = in_size; + out_len = q->out_len; + ret =qzDecompress (&q->sess, q->in_buf, &in_len, q->out_buf, &out_len); +if (ret !=QZ_OK ) { +error_setg (errp,"multifd %u: qzDecompress failed" , p->id); +return -1 ; + } +if (out_len != expected_size) { +error_setg (errp,"multifd %u: packet size received %u size expected %u" , + p->id, out_len, expected_size); +return -1 ; + } + +/* Copy each page to its appropriate location. */ +for (int i =0 ; i < p->normal_num; i++) { +memcpy (p->host + p->normal[i], + q->out_buf + p->page_size * i, + p->page_size); + } +return 0 ; +} + +static MultiFDMethods multifd_qatzip_ops = { + .send_setup = qatzip_send_setup, + .send_cleanup = qatzip_send_cleanup, + .send_prepare = qatzip_send_prepare, + .recv_setup = qatzip_recv_setup, + .recv_cleanup = qatzip_recv_cleanup, + .recv = qatzip_recv +}; + +static void multifd_qatzip_register (void ) +{ +multifd_register_ops (MULTIFD_COMPRESSION_QATZIP , &multifd_qatzip_ops); +} + +migration_init (multifd_qatzip_register);migration/multifd.h +3
-2@@ -34,14 +34,15 @@ MultiFDRecvData *multifd_get_recv_data(void);/* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) -/* We reserve 4 bits for compression methods */ -#define MULTIFD_FLAG_COMPRESSION_MASK (0xf << 1) +/* We reserve 5 bits for compression methods */ +#define MULTIFD_FLAG_COMPRESSION_MASK (0x1f << 1)/* we need to be compatible. Before compression value was 0 */ #define MULTIFD_FLAG_NOCOMP (0 << 1)#define MULTIFD_FLAG_ZLIB (1 << 1)#define MULTIFD_FLAG_ZSTD (2 << 1)#define MULTIFD_FLAG_QPL (4 << 1)#define MULTIFD_FLAG_UADK (8 << 1) +#define MULTIFD_FLAG_QATZIP (16 << 1)/* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024)qapi/migration.json +3
-0@@ -558,6 +558,8 @@ # # @zstd: use zstd compression method. # +# @qatzip: use qatzip compression method. (Since 9.1) +# # @qpl: use qpl compression method. Query Processing Library(qpl) is # based on the deflate compression algorithm and use the Intel # In-Memory Analytics Accelerator(IAA) accelerated compression@@ -570,6 +572,7 @@ { 'enum': 'MultiFDCompression', 'data': [ 'none', 'zlib', { 'name': 'zstd', 'if': 'CONFIG_ZSTD' }, + { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'}, { 'name': 'qpl', 'if': 'CONFIG_QPL' }, { 'name': 'uadk', 'if': 'CONFIG_UADK' } ] }tests/qtest/meson.build +4
-0@@ -327,6 +327,10 @@ if gnutls.found() endif endif +if qatzip.found() + migration_files += [qatzip] +endif + qtests = { 'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'], 'cdrom-test': files('boot-sector.c'),
Re: [External] Re: [PATCH v5 4/5] migration: Introduce 'qatzip' compression method
Quoted 296 lines · Yichen Wang
Quoted 22 lines · Fabiano Rosas
Quoted 19 lines · Yichen Wang
From: Bryan Zhang <[email protected]>
Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.
Signed-off-by Bryan Zhang <[email protected]>Signed-off-by Hao Xiang <[email protected]>Signed-off-by Yichen Wang <[email protected]>--- hw/core/qdev-properties-system.c | 6 +- migration/meson.build | 1 + migration/multifd-qatzip.c | 403 +++++++++++++++++++++++++++++++ migration/multifd.h | 5 +- qapi/migration.json | 3 + tests/qtest/meson.build | 4 + 6 files changed, 419 insertions(+), 3 deletions(-) create mode 100644 migration/multifd-qatzip.cQuoted 16 lines
hw/core/qdev-properties-system.c const PropertyInfo qdev_prop_fdc_drive_type = { · line 665
Quoted 13 lines
hw/core/qdev-properties-system.c +4
-1@@ -659,7 +659,11 @@ const PropertyInfo qdev_prop_fdc_drive_type = {const PropertyInfo qdev_prop_multifd_compression = { .name ="MultiFDCompression" , .description ="multifd_compression values, " -"none/zlib/zstd/qpl/uadk" , +"none/zlib/zstd/qpl/uadk" +#ifdef CONFIG_QATZIP +"/qatzip" +#endifIt seems the other accelerators don't need the ifdef. What's different here?
Just changed and align to other methods. Will fix in next version.
Quoted 83 lines
Quoted 83 lines
hw/core/qdev-properties-system.c +1
-0+ , .enum_table = &MultiFDCompression_lookup, .get = qdev_propinfo_get_enum, .set = qdev_propinfo_set_enum,migration/meson.build +1
-0@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c')) system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) system_ss.add(when: qpl, if_true: files('multifd-qpl.c')) system_ss.add(when: uadk, if_true: files('multifd-uadk.c')) +system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c')) specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c',migration/multifd-qatzip.c new file +60
-0@@ -0,0 +1,403 @@ +/* +* Multifd QATzip compression implementation +* +* Copyright (c) Bytedance +* +* Authors: +* Bryan Zhang <[email protected]> +* Hao Xiang <[email protected]> +* Yichen Wang <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ + +#include "qemu/osdep.h" +#include "exec/ramblock.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qapi/qapi-types-migration.h" +#include "options.h" +#include "multifd.h" +#include <qatzip.h> + +typedef struct { +/* +* Unique session for use with QATzip API +*/ +QzSession_T sess; + +/* +* For compression: Buffer for pages to compress +* For decompression: Buffer for data to decompress +*/ +uint8_t *in_buf; +uint32_t in_len; + +/* +* For compression: Output buffer of compressed data +* For decompression: Output buffer of decompressed data +*/ +uint8_t *out_buf; +uint32_t out_len; +}QatzipData ; + +/** +* qatzip_send_setup: Set up QATzip session and private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_send_setup (MultiFDSendParams *p,Error **errp) +{ +QatzipData *q; +QzSessionParamsDeflate_T params; +const char *err_msg; +int ret; + + q =g_new0 (QatzipData,1 ); + p->compress_data = q;Quoted 8 lines
migration/multifd-qatzip.c line 65
Quoted 5 lines
migration/multifd-qatzip.c new file +5
-0+/* We need one extra place for the packet header */ + p->iov = g_new0(struct iovec, 2); + +/* Prefer without sw_fallback because of bad performance with sw_fallback. +* Warn if sw_fallback needs to be used. */ Please run
scripts/checkpatch.plon your series. This style of comments should have been flagged as non-conformant with our guidelines.Sorry for that. Will fix in next version.
Quoted 10 lines
migration/multifd-qatzip.c line 69
Quoted 5 lines
migration/multifd-qatzip.c new file +4
-0+ ret =qzInit (&q->sess, false); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ +warn_report ("Initilizing QAT with sw_fallback..." );This will warn for each multifd channel, maybe use
warn_report_onceinstead. Also s/Initilizing/Initializing/ and let's spell out "software fallback".Will fix in next version.
Quoted 86 lines
Quoted 71 lines
migration/multifd-qatzip.c new file +71
-0+ ret =qzInit (&q->sess, true); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { +/* Warn, and try with sw_fallback. */ + err_msg ="qzInit failed" ; + goto err_free_q; + } + } + + ret =qzGetDefaultsDeflate (¶ms); +if (ret !=QZ_OK ) { + err_msg ="qzGetDefaultsDeflate failed" ; + goto err_close; + } + +/* Make sure to use configured QATzip compression level. */ + params.common_params.comp_lvl =migrate_multifd_qatzip_level (); + + ret =qzSetupSessionDeflate (&q->sess, ¶ms); +if (ret !=QZ_OK && ret !=QZ_DUPLICATE ) { + err_msg ="qzSetupSessionDeflate failed" ; + goto err_close; + } + +if (MULTIFD_PACKET_SIZE >UINT32_MAX ) { + err_msg ="packet size too large for QAT" ; + goto err_close; + } + + q->in_len =MULTIFD_PACKET_SIZE ; + q->in_buf =qzMalloc (q->in_len,0 ,PINNED_MEM ); +if (!q->in_buf) { + err_msg ="qzMalloc failed" ; + goto err_close; + } + + q->out_len =qzMaxCompressedLength (MULTIFD_PACKET_SIZE , &q->sess); + q->out_buf =qzMalloc (q->out_len,0 ,PINNED_MEM ); +if (!q->out_buf) { + err_msg ="qzMalloc failed" ; + goto err_free_inbuf; + } + +return 0 ; + +err_free_inbuf: +qzFree (q->in_buf); +err_close: +qzClose (&q->sess); +err_free_q: +g_free (q); +g_free (p->iov); + p->iov =NULL ; + p->compress_data =NULL ; +error_setg (errp,"multifd %u: %s" , p->id, err_msg); +return -1 ; +} + +/** +* qatzip_send_cleanup: Tear down QATzip session and release private buffers. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return None +*/ +static void qatzip_send_cleanup (MultiFDSendParams *p,Error **errp) +{ +QatzipData *q = p->compress_data; +const char *err_msg; +int ret; + + ret =qzTeardownSession (&q->sess);migration/multifd-qatzip.c line 150
Quoted 10 lines
migration/multifd-qatzip.c new file +10
-0+if (ret !=QZ_OK ) { + err_msg ="qzTeardownSession failed" ; + goto err; + } + + ret =qzClose (&q->sess); +if (ret !=QZ_OK ) { + err_msg ="qzClose failed" ; + goto err; + }Can
qzClose()be called twice on the same session pointer? It's possible that we have already failed atmultifd_send_setup()and still reach here.Quoted 6 lines
And what about
qzTeardownSession()? Can it cope with an already closed session?And what about the sessions that never got created because we might have exited early at the
ops->send_setup()loop?
qzTeardownSession()andqzClose()are safe to call on NULL pointers. But thanks to your comments which corrected my understanding. These patch was wrote under the impression that whensetup()failed,cleanup()won't be fired. After learning in gdb, apparently I was wrong. Thecleanup()will be called from another thread, which will be called regardless ifsetup()returns zero or non-zero. I will rewrite the setup()/cleanup() logics in my next patchset.Quoted 36 lines
migration/multifd-qatzip.c line 155
Quoted 5 lines
migration/multifd-qatzip.c new file +5
-0+ +qzFree (q->in_buf); + q->in_buf =NULL ; +qzFree (q->out_buf); + q->out_buf =NULL ;These will double free here if
send_setuphas already freed.Quoted 28 lines
migration/multifd-qatzip.c new file +28
-0+g_free (p->iov); + p->iov =NULL ; +g_free (p->compress_data); + p->compress_data =NULL ; +return ; + +err: +error_setg (errp,"multifd %u: %s" , p->id, err_msg); +} + +/** +* qatzip_send_prepare: Compress pages and update IO channel info. +* +* @param p Multifd channel params +* @param errp Pointer to error, which will be set in case of error +* @return 0 on success, -1 on error (and *errp will be set) +*/ +static int qatzip_send_prepare (MultiFDSendParams *p,Error **errp) +{ +MultiFDPages_t *pages = p->pages; +QatzipData *q = p->compress_data; +int ret; +unsigned int in_len, out_len; + +if (!multifd_send_prepare_common (p)) { + goto out; + } +Quoted 9 lines
migration/multifd-qatzip.c line 189
Quoted 6 lines
migration/multifd-qatzip.c new file +6
-0+/* Unlike other multifd compression implementations, we use a +* non-streaming API and place all the data into one buffer, rather than +* sending each page to the compression API at a time. */ +for (int i =0 ; i < pages->normal_num; i++) { + memcpy(q->in_buf + (i * p->page_size), + p->pages->block->host + pages->offset[i],
pages->block->host
Quoted 1 line
I am not sure if I understand your comment here?
You're holding the pointer to p->pages in the local pages variable. I'm suggesting you use it instead of p->pages here. In another series[1], we're looking into changing p->pages into something else and having to change it all over the code gets bothersome.
PATCH v5 5/5 tests/migration: Add integration test for 'qatzip' compression method
In reply to Yichen WangFrom: Bryan Zhang <[email protected]>
Adds an integration test for 'qatzip'.
--- tests/qtest/migration-test.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
tests/qtest/migration-test.c +35 -0
@@ -32,6 +32,10 @@# endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ +#ifdef CONFIG_QATZIP +#include <qatzip.h> +#endif /* CONFIG_QATZIP */ +/* For dirty ring test; so far only x86_64 is supported */ #if defined(__linux__) && defined(HOST_X86_64 )#include "linux/kvm.h"
@@ -2992,6 +2996,22 @@ test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from, }#endif /* CONFIG_ZSTD */ +#ifdef CONFIG_QATZIP +static void * +test_migrate_precopy_tcp_multifd_qatzip_start (QTestState *from, +QTestState *to) +{ +migrate_set_parameter_int (from,"multifd-qatzip-level" ,2 ); +migrate_set_parameter_int (to,"multifd-qatzip-level" ,2 ); + +/* SW fallback is disabled by default, so enable it for testing. */ +migrate_set_parameter_bool (from,"multifd-qatzip-sw-fallback" , true); +migrate_set_parameter_bool (to,"multifd-qatzip-sw-fallback" , true); + +return test_migrate_precopy_tcp_multifd_start_common (from, to,"qatzip" ); +} +#endif +#ifdef CONFIG_QPL static void * test_migrate_precopy_tcp_multifd_qpl_start(QTestState *from,
@@ -3089,6 +3109,17 @@ static void test_multifd_tcp_zstd(void) }#endif +#ifdef CONFIG_QATZIP +static void test_multifd_tcp_qatzip (void ) +{ +MigrateCommon args = { + .listen_uri ="defer" , + .start_hook = test_migrate_precopy_tcp_multifd_qatzip_start, + }; +test_precopy_common (&args); +} +#endif +#ifdef CONFIG_QPL static void test_multifd_tcp_qpl (void ) {
@@ -3992,6 +4023,10 @@ int main(int argc, char **argv)migration_test_add ("/migration/multifd/tcp/plain/zstd" , test_multifd_tcp_zstd);#endif +#ifdef CONFIG_QATZIP +migration_test_add ("/migration/multifd/tcp/plain/qatzip" , + test_multifd_tcp_qatzip); +#endif #ifdef CONFIG_QPL migration_test_add ("/migration/multifd/tcp/plain/qpl" , test_multifd_tcp_qpl);
Signature · 1 line
Yichen Wang
Re: [PATCH v5 5/5] tests/migration: Add integration test for 'qatzip' compression method
Quoted 33 lines · Yichen Wang
From: Bryan Zhang <[email protected]>
Adds an integration test for 'qatzip'.
Signed-off-by Bryan Zhang <[email protected]>Signed-off-by Hao Xiang <[email protected]>Signed-off-by Yichen Wang <[email protected]>Reviewed-by Fabiano Rosas <[email protected]>Signed-off-by Yichen Wang <[email protected]>--- tests/qtest/migration-test.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)tests/qtest/migration-test.c +5
-0@@ -32,6 +32,10 @@# endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ +#ifdef CONFIG_QATZIP +#include <qatzip.h> +#endif /* CONFIG_QATZIP */ +/* For dirty ring test; so far only x86_64 is supported */ #if defined(__linux__) && defined(HOST_X86_64 )#include "linux/kvm.h" @@ -2992,6 +2996,22 @@ test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from, }#endif /* CONFIG_ZSTD */ +#ifdef CONFIG_QATZIP
tests/qtest/migration-test.c test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from, · line 3009
Quoted 10 lines
tests/qtest/migration-test.c +10
-0+static void * +test_migrate_precopy_tcp_multifd_qatzip_start (QTestState *from, +QTestState *to) +{ +migrate_set_parameter_int (from,"multifd-qatzip-level" ,2 ); +migrate_set_parameter_int (to,"multifd-qatzip-level" ,2 ); + +/* SW fallback is disabled by default, so enable it for testing. */ +migrate_set_parameter_bool (from,"multifd-qatzip-sw-fallback" , true); +migrate_set_parameter_bool (to,"multifd-qatzip-sw-fallback" , true);
Shouldn't this already crash when without the parameter?
Quoted 40 lines
tests/qtest/migration-test.c +20
-0+ +return test_migrate_precopy_tcp_multifd_start_common (from, to,"qatzip" ); +} +#endif +#ifdef CONFIG_QPL static void * test_migrate_precopy_tcp_multifd_qpl_start(QTestState *from,@@ -3089,6 +3109,17 @@ static void test_multifd_tcp_zstd(void) }#endif +#ifdef CONFIG_QATZIP +static void test_multifd_tcp_qatzip (void ) +{ +MigrateCommon args = { + .listen_uri ="defer" , + .start_hook = test_migrate_precopy_tcp_multifd_qatzip_start, + }; +test_precopy_common (&args); +} +#endif +#ifdef CONFIG_QPL static void test_multifd_tcp_qpl (void ) {@@ -3992,6 +4023,10 @@ int main(int argc, char **argv)migration_test_add ("/migration/multifd/tcp/plain/zstd" , test_multifd_tcp_zstd);#endif +#ifdef CONFIG_QATZIP +migration_test_add ("/migration/multifd/tcp/plain/qatzip" , + test_multifd_tcp_qatzip); +#endif #ifdef CONFIG_QPL migration_test_add ("/migration/multifd/tcp/plain/qpl" , test_multifd_tcp_qpl);Signature · 1 line
Yichen Wang
Signature · 1 line
Peter Xu
Re: [External] Re: [PATCH v5 5/5] tests/migration: Add integration test for 'qatzip' compression method
Quoted 34 lines · Peter Xu
Quoted 32 lines · Yichen Wang
From: Bryan Zhang <[email protected]>
Adds an integration test for 'qatzip'.
Signed-off-by Bryan Zhang <[email protected]>Signed-off-by Hao Xiang <[email protected]>Signed-off-by Yichen Wang <[email protected]>Reviewed-by Fabiano Rosas <[email protected]>Signed-off-by Yichen Wang <[email protected]>--- tests/qtest/migration-test.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)tests/qtest/migration-test.c +4
-0@@ -32,6 +32,10 @@# endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ +#ifdef CONFIG_QATZIP +#include <qatzip.h> +#endif /* CONFIG_QATZIP */ +/* For dirty ring test; so far only x86_64 is supported */ #if defined(__linux__) && defined(HOST_X86_64 )#include "linux/kvm.h" @@ -2992,6 +2996,22 @@ test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from, }#endif /* CONFIG_ZSTD */
Quoted 13 lines
tests/qtest/migration-test.c test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from, · line 3009
Quoted 11 lines
tests/qtest/migration-test.c +11
-0+#ifdef CONFIG_QATZIP +static void * +test_migrate_precopy_tcp_multifd_qatzip_start (QTestState *from, +QTestState *to) +{ +migrate_set_parameter_int (from,"multifd-qatzip-level" ,2 ); +migrate_set_parameter_int (to,"multifd-qatzip-level" ,2 ); + +/* SW fallback is disabled by default, so enable it for testing. */ +migrate_set_parameter_bool (from,"multifd-qatzip-sw-fallback" , true); +migrate_set_parameter_bool (to,"multifd-qatzip-sw-fallback" , true);Shouldn't this already crash when without the parameter?
Ah, my bad. I tested the features manually with two machines, and didn't run this. I will fix it in my next version.
Quoted 45 lines
Quoted 39 lines
tests/qtest/migration-test.c +20
-1+ +return test_migrate_precopy_tcp_multifd_start_common (from, to,"qatzip" ); +} +#endif +#ifdef CONFIG_QPL static void * test_migrate_precopy_tcp_multifd_qpl_start(QTestState *from,@@ -3089,6 +3109,17 @@ static void test_multifd_tcp_zstd(void) }#endif +#ifdef CONFIG_QATZIP +static void test_multifd_tcp_qatzip (void ) +{ +MigrateCommon args = { + .listen_uri ="defer" , + .start_hook = test_migrate_precopy_tcp_multifd_qatzip_start, + }; +test_precopy_common (&args); +} +#endif +#ifdef CONFIG_QPL static void test_multifd_tcp_qpl (void ) {@@ -3992,6 +4023,10 @@ int main(int argc, char **argv)migration_test_add ("/migration/multifd/tcp/plain/zstd" , test_multifd_tcp_zstd);#endif +#ifdef CONFIG_QATZIP +migration_test_add ("/migration/multifd/tcp/plain/qatzip" , + test_multifd_tcp_qatzip); +#endif #ifdef CONFIG_QPL migration_test_add ("/migration/multifd/tcp/plain/qpl" , test_multifd_tcp_qpl); --Quoted 2 lines
Yichen Wang
--
Peter Xu
Quoted 4 lines · Yichen Wang
v5:
- Rebase changes on top of59084feb256c617063e0dbe7e64821ae8852d7cf
- Add documentations about migration with qatzip accerlation
- Remove multifd-qatzip-sw-fallback option
I think Yuan provided quite a few meaningful comments, did you address all of them?
You didn't reply in the previous version, and you didn't add anything in the changelog. I suggest you at least do one of them in the future so that reviewers can understand what happen.
Thanks,
Signature · 1 line
Peter Xu
Quoted 9 lines · Peter Xu
Quoted 4 lines · Yichen Wang
v5:
- Rebase changes on top of59084feb256c617063e0dbe7e64821ae8852d7cf
- Add documentations about migration with qatzip accerlation
- Remove multifd-qatzip-sw-fallback optionI think Yuan provided quite a few meaningful comments, did you address all of them?
Yes. I do.
Quoted 4 lines
You didn't reply in the previous version, and you didn't add anything in the changelog. I suggest you at least do one of them in the future so that reviewers can understand what happen.
They are all very good comments, and instead of replying I just fix them all and include it in my next patch. In my changelog I do include all the changes and comments we discussed in v4. Sorry I am new to the community, so I will reply "fixed" in the previous email before pushing the next version. Thanks a lot, and sorry for that.
Quoted 6 lines
Thanks,
--
Peter Xu
Quoted 12 lines · Yichen Wang
Quoted 9 lines · Peter Xu
Quoted 4 lines · Yichen Wang
v5:
- Rebase changes on top of59084feb256c617063e0dbe7e64821ae8852d7cf
- Add documentations about migration with qatzip accerlation
- Remove multifd-qatzip-sw-fallback optionI think Yuan provided quite a few meaningful comments, did you address all of them?
Yes. I do.
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
You didn't reply in the previous version, and you didn't add anything in the changelog. I suggest you at least do one of them in the future so that reviewers can understand what happen.
They are all very good comments, and instead of replying I just fix them all and include it in my next patch. In my changelog I do include all the changes and comments we discussed in v4. Sorry I am new to the community, so I will reply "fixed" in the previous email before pushing the next version. Thanks a lot, and sorry for that.
That's all fine! You can definitely mention them too here in the changelog if you think that's easier.
One last nitpick is in the major patch you duplicated part of the comment when I was requesting a movement (the part explaining why you used a buffer rather than submit compression for each page without memcpy), I suggest you can simply move that whole comment above, rather than copying.
I don't have any further questions on this series.
Thanks,
Signature · 1 line
Peter Xu