Skip to contentSkip to conversation

Implement QATzip compression method

PATCH Superseded Yichen Wang 4 participants 13 messages Jul 15, 2024 – Jul 19, 2024 Arm/Amlogic Meson SoC support

v6 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v6 cover Implement QATzip compression method

v6:
- Rebase changes on top of 4ea7e9cd882f1574c129d67431784fecc426d23b
- Address comments, typos, and styling issues
- Re-factor QAT setup()/cleanup() logic

v5:
- Rebase changes on top of 59084feb256c617063e0dbe7e64821ae8852d7cf
- Add documentations about migration with qatzip accerlation
- Remove multifd-qatzip-sw-fallback option

v4:
- Rebase changes on top of 1a2d52c7fcaeaaf4f2fe8d4d5183dccaeab67768
- Move the IOV initialization to qatzip implementation
- Only use qatzip to compress normal pages

v3:
- Rebase changes on top of master
- Merge two patches per Fabiano Rosas's comment
- Add versions into comments and documentations

v2:
- Rebase changes on top of recent multifd code changes.
- Use QATzip API 'qzMalloc' and 'qzFree' to allocate QAT buffers.
- Remove parameter tuning and use QATzip's defaults for better performance.
- Add parameter to enable QAT software fallback.

v1:
https://lists.nongnu.org/archive/html/qemu-devel/2023-12/msg03761.html

* Performance

We present updated performance results. For circumstantial reasons, v1 presented performance on a low-bandwidth (1Gbps) network.

Here, we present updated results with a similar setup as before but with two main differences:

1. Our machines have a ~50Gbps connection, tested using 'iperf3'.
2. We had a bug in our memory allocation causing us to only use ~1/2 of the VM's RAM. Now we properly allocate and fill nearly all of the VM's RAM.

Thus, the test setup is as follows:

We perform multifd live migration over TCP using a VM with 64GB memory. We prepare the machine's memory by powering it on, allocating a large amount of memory (60GB) as a single buffer, and filling the buffer with the repeated contents of the Silesia corpus[0]. This is in lieu of a more realistic memory snapshot, which proved troublesome to acquire.

We analyze CPU usage by averaging the output of 'top' every second during migration. This is admittedly imprecise, but we feel that it accurately portrays the different degrees of CPU usage of varying compression methods.

We present the latency, throughput, and CPU usage results for all of the compression methods, with varying numbers of multifd threads (4, 8, and 16).

[0] The Silesia corpus can be accessed here:
https://sun.aei.polsl.pl//~sdeor/index.php?page=silesia

** Results

4 multifd threads:

    |---------------|---------------|----------------|---------|---------|
    |method         |time(sec)      |throughput(mbps)|send cpu%|recv cpu%|
    |---------------|---------------|----------------|---------|---------|
    |qatzip         | 23.13         | 8749.94        |117.50   |186.49   |
    |---------------|---------------|----------------|---------|---------|
    |zlib           |254.35         |  771.87        |388.20   |144.40   |
    |---------------|---------------|----------------|---------|---------|
    |zstd           | 54.52         | 3442.59        |414.59   |149.77   |
    |---------------|---------------|----------------|---------|---------|
    |none           | 12.45         |43739.60        |159.71   |204.96   |
    |---------------|---------------|----------------|---------|---------|

8 multifd threads:

    |---------------|---------------|----------------|---------|---------|
    |method         |time(sec)      |throughput(mbps)|send cpu%|recv cpu%|
    |---------------|---------------|----------------|---------|---------|
    |qatzip         | 16.91         |12306.52        |186.37   |391.84   |
    |---------------|---------------|----------------|---------|---------|
    |zlib           |130.11         | 1508.89        |753.86   |289.35   |
    |---------------|---------------|----------------|---------|---------|
    |zstd           | 27.57         | 6823.23        |786.83   |303.80   |
    |---------------|---------------|----------------|---------|---------|
    |none           | 11.82         |46072.63        |163.74   |238.56   |
    |---------------|---------------|----------------|---------|---------|

16 multifd threads:

    |---------------|---------------|----------------|---------|---------|
    |method         |time(sec)      |throughput(mbps)|send cpu%|recv cpu%|
    |---------------|---------------|----------------|---------|---------|
    |qatzip         |18.64          |11044.52        | 573.61  |437.65   |
    |---------------|---------------|----------------|---------|---------|
    |zlib           |66.43          | 2955.79        |1469.68  |567.47   |
    |---------------|---------------|----------------|---------|---------|
    |zstd           |14.17          |13290.66        |1504.08  |615.33   |
    |---------------|---------------|----------------|---------|---------|
    |none           |16.82          |32363.26        | 180.74  |217.17   |
    |---------------|---------------|----------------|---------|---------|

** Observations

- In general, not using compression outperforms using compression in a non-network-bound environment.
- 'qatzip' outperforms other compression workers with 4 and 8 workers, achieving a ~91% latency reduction over 'zlib' with 4 workers, and a
~58% latency reduction over 'zstd' with 4 workers.
- 'qatzip' maintains comparable performance with 'zstd' at 16 workers, showing a ~32% increase in latency. This performance difference
becomes more noticeable with more workers, as CPU compression is highly parallelizable.
- 'qatzip' compression uses considerably less CPU than other compression methods. At 8 workers, 'qatzip' demonstrates a ~75% reduction in
compression CPU usage compared to 'zstd' and 'zlib'.
- 'qatzip' decompression CPU usage is less impressive, and is even slightly worse than 'zstd' and 'zlib' CPU usage at 4 and 16 workers.

Bryan Zhang (4):
meson: Introduce 'qatzip' feature to the build system
migration: Add migration parameters for QATzip
migration: Introduce 'qatzip' compression method tests/migration: Add integration test for 'qatzip' compression method

Yuan Liu (1):
docs/migration: add qatzip compression feature

 docs/devel/migration/features.rst           |   1 +
 docs/devel/migration/qatzip-compression.rst | 251 +++++++++++++
 hw/core/qdev-properties-system.c            |   2 +-
 meson.build                                 |  10 +
 meson_options.txt                           |   2 +
 migration/meson.build                       |   1 +
 migration/migration-hmp-cmds.c              |   4 +
 migration/multifd-qatzip.c                  | 382 ++++++++++++++++++++
 migration/multifd.h                         |   5 +-
 migration/options.c                         |  34 ++
 migration/options.h                         |   1 +
 qapi/migration.json                         |  21 ++
 scripts/meson-buildoptions.sh               |   3 +
 tests/qtest/meson.build                     |   4 +
 tests/qtest/migration-test.c                |  31 ++
 15 files changed, 749 insertions(+), 3 deletions(-)
 create mode 100644 docs/devel/migration/qatzip-compression.rst
 create mode 100644 migration/multifd-qatzip.c
Signature · 1 line
Yichen Wang
#1 lore raw

PATCH v6 1/5 docs/migration: add qatzip compression feature

From: Yuan Liu <[email protected]>

add Intel QATzip compression method introduction

---
 docs/devel/migration/features.rst           |   1 +
 docs/devel/migration/qatzip-compression.rst | 251 ++++++++++++++++++++
 2 files changed, 252 insertions(+)
 create mode 100644 docs/devel/migration/qatzip-compression.rst
docs/devel/migration/features.rst +1 -0
index 58f8fd9e16..8f431d52f9 100644
@@ -14,3 +14,4 @@ Migration has plenty of features to support different use cases.
    CPR
    qpl-compression
    uadk-compression
+   qatzip-compression
docs/devel/migration/qatzip-compression.rst new file +251 -0

251 lines not on this page · raw message

Signature · 1 line
Yichen Wang
#2 lore raw

RE: [PATCH v6 1/5] docs/migration: add qatzip compression feature

Quoted 52 lines

-----Original Message-----
From: Yichen Wang <[email protected]>
Sent: Tuesday, July 16, 2024 6:13 AM
To: Peter Xu <[email protected]>; Fabiano Rosas <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Marc-André Lureau <[email protected]>; Thomas Huth <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Laurent Vivier <[email protected]>; qemu- [email protected]
Cc: Hao Xiang <[email protected]>; Liu, Yuan1 <[email protected]>; Zou, Nanhai <[email protected]>; Ho-Ren (Jack) Chuang <[email protected]>; Wang, Yichen <[email protected]>
Subject: [PATCH v6 1/5] docs/migration: add qatzip compression feature

From: Yuan Liu <[email protected]>

add Intel QATzip compression method introduction

---
 docs/devel/migration/features.rst           |   1 +
 docs/devel/migration/qatzip-compression.rst | 251 ++++++++++++++++++++
 2 files changed, 252 insertions(+)
 create mode 100644 docs/devel/migration/qatzip-compression.rst
docs/devel/migration/features.rst +0 -0
index 58f8fd9e16..8f431d52f9 100644
@@ -14,3 +14,4 @@ Migration has plenty of features to support different

use cases.
CPR
qpl-compression
uadk-compression

docs/devel/migration/features.rst +1 -0
+   qatzip-compression
docs/devel/migration/qatzip-compression.rst new file +4 -0
new file mode 100644
index 0000000000..72fa3e2826
@@ -0,0 +1,251 @@
+==================
+QATzip Compression
+==================
+In scenarios with limited network bandwidth, the ``QATzip`` solution can

help

docs/devel/migration/qatzip-compression.rst line 6

Quoted 2 lines
docs/devel/migration/qatzip-compression.rst new file +2 -0
+users save a lot of host CPU resources by accelerating compression and
+decompression through the Intel QuickAssist Technology(``QAT``) hardware.

Hi Yichen

Thanks for adding the part of Performance Testing with QATzip, I wonder if we can remove Performance Testing with QATzip part and directly add the following content.

Here, we use a typical example of limited bandwidth to illustrate the advantages of QATzip. If the user is interested in qatzip, he still needs to verify the performance by himself.

+The following test was conducted using 8 multifd channels and 10Gbps network +bandwidth. The results show that, compared to zstd, ``QATzip`` significantly +saves CPU resources on the sender and reduces migration time. Compared to the +uncompressed solution, ``QATzip`` greatly improves the dirty page processing +capability, indicated by the Pages per Second metric, and also reduces the +total migration time.
+
+::
+

+   VM Configuration: 16 vCPU and 64G memory
+   VM Workload: all vCPUs are idle and 54G memory is filled with Silesia data.
+   QAT Devices: 4
+   |-----------|--------|---------|----------|----------|------|------|
+   |8 Channels |Total   |down     |throughput|pages per | send | recv |
+   |           |time(ms)|time(ms) |(mbps)    |second    | cpu %| cpu% |
+   |-----------|--------|---------|----------|----------|------|------|
+   |qatzip     |   16630|       28|     10467|   2940235|   160|   360|
+   |-----------|--------|---------|----------|----------|------|------|
+   |zstd       |   20165|       24|      8579|   2391465|   810|   340|
+   |-----------|--------|---------|----------|----------|------|------|
+   |none       |   46063|       40|     10848|    330240|    45|    85|
+   |-----------|--------|---------|----------|----------|------|------|
Quoted 278 lines

+``QATzip`` is a user space library which builds on top of the Intel QuickAssist
+Technology user space library, to provide extended accelerated compression and

docs/devel/migration/qatzip-compression.rst new file +39 -0
+decompression services.
+
+For more ``QATzip`` introduction, please refer to `QATzip Introduction
+<https://github.com/intel/QATzip?tab=readme-ov-file#introductionl>`_
+
+QATzip Compression Framework
+============================
+
+::
+
+  +----------------+
+  | MultiFd Thread |
+  +-------+--------+
+          |
+          | compress/decompress
+  +-------+--------+
+  | QATzip library |
+  +-------+--------+
+          |
+  +-------+--------+
+  |  QAT library   |
+  +-------+--------+
+          |         user space
+  --------+---------------------
+          |         kernel space
+   +------+-------+
+   |  QAT  Driver |
+   +------+-------+
+          |
+   +------+-------+
+   | QAT Devices  |
+   +--------------+
+
+
+QATzip Installation
+-------------------
+
+The ``QATzip`` installation package has been integrated into some Linux
+distributions and can be installed directly. For example, the Ubuntu

Server
+24.04 LTS system can be installed using below command

docs/devel/migration/qatzip-compression.rst new file +16 -0
+
+.. code-block:: shell
+
+   #apt search qatzip
+   libqatzip-dev/noble 1.2.0-0ubuntu3 amd64
+     Intel QuickAssist user space library development files
+
+   libqatzip3/noble 1.2.0-0ubuntu3 amd64
+     Intel QuickAssist user space library
+
+   qatzip/noble,now 1.2.0-0ubuntu3 amd64 [installed]
+     Compression user-space tool for Intel QuickAssist Technology
+
+   #sudo apt install libqatzip-dev libqatzip3 qatzip
+
+If your system does not support the ``QATzip`` installation package, you

can
+use the source code to build and install, please refer to QATzip source code installation +<https://github.com/intel/QATzip?tab=readme-ov-file#build-intel- quickassist-technology-driver>_

docs/devel/migration/qatzip-compression.rst new file +6 -0
+
+QAT Hardware Deployment
+-----------------------
+
+``QAT`` supports physical functions(PFs) and virtual functions(VFs) for
+deployment, and users can configure ``QAT`` resources for migration

according
+to actual needs. For more details about ``QAT`` deployment, please refer to
+Intel QuickAssist Technology Documentation +<https://intel.github.io/quickassist/index.html>_

docs/devel/migration/qatzip-compression.rst new file +2 -0
+
+For more ``QAT`` hardware introduction, please refer to `intel-quick-

assist-technology-overview +<https://www.intel.com/content/www/us/en/architecture-and- technology/intel-quick-assist-technology-overview.html>`_

docs/devel/migration/qatzip-compression.rst new file +9 -0
+
+How To Use QATzip Compression
+=============================
+
+1 - Install ``QATzip`` library
+
+2 - Build ``QEMU`` with ``--enable-qatzip`` parameter
+
+  E.g. configure --target-list=x86_64-softmmu --enable-kvm ``--enable-

qatzip``

docs/devel/migration/qatzip-compression.rst new file +4 -0
+
+3 - Set ``migrate_set_parameter multifd-compression qatzip``
+
+4 - Set ``migrate_set_parameter multifd-qatzip-level comp_level``, the

default

docs/devel/migration/qatzip-compression.rst new file +11 -0
+comp_level value is 1, and it supports levels from 1 to 9
+
+
+Performance Testing with QATzip
+===============================
+
+Testing environment is being set as below:
+
+VM configuration:16 vCPU, 64G memory;
+
+VM Workload: all vCPUs are idle and 54G memory is filled with Silesia

data;

docs/devel/migration/qatzip-compression.rst new file +98 -0
+
+QAT Devices: 4;
+
+Sender migration parameters:
+
+.. code-block:: shell
+
+    migrate_set_capability multifd on
+    migrate_set_parameter multifd-channels 2/4/8
+    migrate_set_parameter max-bandwidth 1G/10G
+    migrate_set_parameter multifd-compression qatzip/zstd
+
+Receiver migration parameters:
+
+.. code-block:: shell
+
+    migrate_set_capability multifd on
+    migrate_set_parameter multifd-channels 2
+    migrate_set_parameter multifd-compression qatzip/zstd
+
+max-bandwidth: 1 GBps (Gbytes/sec)
+
+.. code-block:: text
+
+    |-----------|--------|---------|----------|------|------|
+    |2 Channels |Total   |down     |throughput| send | recv |
+    |           |time(ms)|time(ms) |(mbps)    | cpu %| cpu% |
+    |-----------|--------|---------|----------|------|------|
+    |qatzip     |   21607|       77|      8051|    88|   125|
+    |-----------|--------|---------|----------|------|------|
+    |zstd       |   78351|       96|      2199|   204|    80|
+    |-----------|--------|---------|----------|------|------|
+
+    |-----------|--------|---------|----------|------|------|
+    |4 Channels |Total   |down     |throughput| send | recv |
+    |           |time(ms)|time(ms) |(mbps)    | cpu %| cpu% |
+    |-----------|--------|---------|----------|------|------|
+    |qatzip     |   20336|       25|      8557|   110|   190|
+    |-----------|--------|---------|----------|------|------|
+    |zstd       |   39324|       31|      4389|   406|   160|
+    |-----------|--------|---------|----------|------|------|
+
+    |-----------|--------|---------|----------|------|------|
+    |8 Channels |Total   |down     |throughput| send | recv |
+    |           |time(ms)|time(ms) |(mbps)    | cpu %| cpu% |
+    |-----------|--------|---------|----------|------|------|
+    |qatzip     |   20208|       22|      8613|   125|   300|
+    |-----------|--------|---------|----------|------|------|
+    |zstd       |   20515|       22|      8438|   800|   340|
+    |-----------|--------|---------|----------|------|------|
+
+max-bandwidth: 10 GBps (Gbytes/sec)
+
+.. code-block:: text
+
+    |-----------|--------|---------|----------|------|------|
+    |2 Channels |Total   |down     |throughput| send | recv |
+    |           |time(ms)|time(ms) |(mbps)    | cpu %| cpu% |
+    |-----------|--------|---------|----------|------|------|
+    |qatzip     |   22450|       77|      7748|    80|   125|
+    |-----------|--------|---------|----------|------|------|
+    |zstd       |   78339|       76|      2199|   204|    80|
+    |-----------|--------|---------|----------|------|------|
+
+    |-----------|--------|---------|----------|------|------|
+    |4 Channels |Total   |down     |throughput| send | recv |
+    |           |time(ms)|time(ms) |(mbps)    | cpu %| cpu% |
+    |-----------|--------|---------|----------|------|------|
+    |qatzip     |   13017|       24|     13401|   180|   285|
+    |-----------|--------|---------|----------|------|------|
+    |zstd       |   39466|       21|      4373|   406|   160|
+    |-----------|--------|---------|----------|------|------|
+
+    |-----------|--------|---------|----------|------|------|
+    |8 Channels |Total   |down     |throughput| send | recv |
+    |           |time(ms)|time(ms) |(mbps)    | cpu %| cpu% |
+    |-----------|--------|---------|----------|------|------|
+    |qatzip     |   10255|       22|     17037|   280|   590|
+    |-----------|--------|---------|----------|------|------|
+    |zstd       |   20126|       77|      8595|   810|   340|
+    |-----------|--------|---------|----------|------|------|
+
+max-bandwidth: 1.25 GBps (Gbytes/sec)
+
+.. code-block:: text
+
+    |-----------|--------|---------|----------|----------|------|------|
+    |8 Channels |Total   |down     |throughput|pages per | send | recv |
+    |           |time(ms)|time(ms) |(mbps)    |second    | cpu %| cpu% |
+    |-----------|--------|---------|----------|----------|------|------|
+    |qatzip     |   16630|       28|     10467|   2940235|   160|   360|
+    |-----------|--------|---------|----------|----------|------|------|
+    |zstd       |   20165|       24|      8579|   2391465|   810|   340|
+    |-----------|--------|---------|----------|----------|------|------|
+    |none       |   46063|       40|     10848|    330240|    45|    85|
+    |-----------|--------|---------|----------|----------|------|------|
+
+If the user has enabled compression in live migration, using QAT can save

the

docs/devel/migration/qatzip-compression.rst new file +4 -0
+host CPU resources.
+
+When compression is enabled, the bottleneck of migration is usually the
+compression throughput on the sender side, since CPU decompression

throughput

docs/devel/migration/qatzip-compression.rst new file +2 -0
+is higher than compression, some reference data
+https://github.com/inikep/lzbench, so more CPU resources need to be

allocated

docs/devel/migration/qatzip-compression.rst new file +5 -0
+to the sender side.
+
+Summary:
+
+1. In the 1GBps case, QAT only uses 88% CPU utilization to reach 1GBps,

but

docs/devel/migration/qatzip-compression.rst new file +3 -0
+   ZSTD needs 800%.
+
+2. In the 10Gbps case, QAT uses 180% CPU utilization to reach 10GBps. but

ZSTD

docs/devel/migration/qatzip-compression.rst new file +3 -0
+   still cannot reach 10Gbps even if it uses 810%.
+
+3. The QAT decompression CPU utilization is higher than compression and

ZSTD,

docs/devel/migration/qatzip-compression.rst new file +3 -0
+   because:
+
+   a. When using QAT compression, the data needs to be copied to the QAT

memory

docs/devel/migration/qatzip-compression.rst new file +2 -0
+   (for DMA operations), and the same for decompression. However,
+   do_user_addr_fault will be triggered during decompression because the

QAT
+ decompressed data is copied to the VM address space for the first
time, in
+ addition, both compression and decompression are processed by QAT and
do not
+ consume CPU resources, so the CPU utilization of the receiver is
slightly

docs/devel/migration/qatzip-compression.rst new file +3 -0
+   higher than the sender.
+
+   b. Since zstd decompression decompresses data directly into the VM

address
+ space, there is one less memory copy than QAT, so the CPU utilization
on the

docs/devel/migration/qatzip-compression.rst new file +2 -0
+   receiver is better than QAT. For the 1GBps case, the receiver CPU
+   utilization is 125%, and the memory copy occupies ~80% of CPU

utilization.

docs/devel/migration/qatzip-compression.rst new file +4 -0
+
+How To Choose Between QATzip and QPL
+====================================
+Starting from Intel 4th Gen Intel Xeon Scalable processors, codenamed

Sapphire
+Rapids processor(``SPR``), it supports multiple build-in accelerators including
+``QAT`` and ``IAA``, the former can accelerate ``QATzip``, and the latter is

docs/devel/migration/qatzip-compression.rst new file +5 -0
+used to accelerate ``QPL``.
+
+Here are some suggestions:
+
+1 - If your live migration scenario is limited network bandwidth and

``QAT``

docs/devel/migration/qatzip-compression.rst new file +4 -0
+hardware resources exceed ``IAA``, then use the ``QATzip`` method, which
+can save a lot of host CPU resources for compression.
+
+2 - If your system cannot support shared virtual memory(SVM) technology,

please

docs/devel/migration/qatzip-compression.rst new file +4 -1
+use ``QATzip`` method because ``QPL`` performance is not good without SVM
+support.
+
+3 - For other scenarios, please use the ``QPL`` method first.
--

Yichen Wang

#3 lore raw

Re: [PATCH v6 1/5] docs/migration: add qatzip compression feature

Quoted 77 lines · On Tue, Jul 16, 2024 at 02:34:07AM +0000, Liu, Yuan1 wrote:
Quoted 52 lines

-----Original Message-----
From: Yichen Wang <[email protected]>
Sent: Tuesday, July 16, 2024 6:13 AM
To: Peter Xu <[email protected]>; Fabiano Rosas <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Marc-André Lureau <[email protected]>; Thomas Huth <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Laurent Vivier <[email protected]>; qemu- [email protected]
Cc: Hao Xiang <[email protected]>; Liu, Yuan1 <[email protected]>; Zou, Nanhai <[email protected]>; Ho-Ren (Jack) Chuang <[email protected]>; Wang, Yichen <[email protected]>
Subject: [PATCH v6 1/5] docs/migration: add qatzip compression feature

From: Yuan Liu <[email protected]>

add Intel QATzip compression method introduction

---
 docs/devel/migration/features.rst           |   1 +
 docs/devel/migration/qatzip-compression.rst | 251 ++++++++++++++++++++
 2 files changed, 252 insertions(+)
 create mode 100644 docs/devel/migration/qatzip-compression.rst
docs/devel/migration/features.rst +0 -0
index 58f8fd9e16..8f431d52f9 100644
@@ -14,3 +14,4 @@ Migration has plenty of features to support different

use cases.
CPR
qpl-compression
uadk-compression

docs/devel/migration/features.rst +1 -0
+   qatzip-compression
docs/devel/migration/qatzip-compression.rst new file +4 -0
new file mode 100644
index 0000000000..72fa3e2826
@@ -0,0 +1,251 @@
+==================
+QATzip Compression
+==================
+In scenarios with limited network bandwidth, the ``QATzip`` solution can

help

docs/devel/migration/qatzip-compression.rst line 6

Quoted 2 lines
docs/devel/migration/qatzip-compression.rst new file +2 -0
+users save a lot of host CPU resources by accelerating compression and
+decompression through the Intel QuickAssist Technology(``QAT``) hardware.

Hi Yichen

Thanks for adding the part of Performance Testing with QATzip, I wonder if we can remove Performance Testing with QATzip part and directly add the following content.

Here, we use a typical example of limited bandwidth to illustrate the advantages of QATzip. If the user is interested in qatzip, he still needs to verify the performance by himself.

docs/devel/migration/qatzip-compression.rst new file +12 -0
+The following test was conducted using 8 multifd channels and 10Gbps network
+bandwidth. The results show that, compared to zstd, ``QATzip`` significantly
+saves CPU resources on the sender and reduces migration time. Compared to the
+uncompressed solution, ``QATzip`` greatly improves the dirty page processing
+capability, indicated by the Pages per Second metric, and also reduces the
+total migration time.
+
+::
+
+   VM Configuration: 16 vCPU and 64G memory
+   VM Workload: all vCPUs are idle and 54G memory is filled with Silesia data.
+   QAT Devices: 4

docs/devel/migration/qatzip-compression.rst line 199

Quoted 10 lines
docs/devel/migration/qatzip-compression.rst new file +10 -0
+   |-----------|--------|---------|----------|----------|------|------|
+   |8 Channels |Total   |down     |throughput|pages per | send | recv |
+   |           |time(ms)|time(ms) |(mbps)    |second    | cpu %| cpu% |
+   |-----------|--------|---------|----------|----------|------|------|
+   |qatzip     |   16630|       28|     10467|   2940235|   160|   360|
+   |-----------|--------|---------|----------|----------|------|------|
+   |zstd       |   20165|       24|      8579|   2391465|   810|   340|
+   |-----------|--------|---------|----------|----------|------|------|
+   |none       |   46063|       40|     10848|    330240|    45|    85|
+   |-----------|--------|---------|----------|----------|------|------|

Yes this looks much simpler and better. The 10GBps test isn't that useful at least, especially with nocomp numbers absent. I didn't say when looking previously, but it'll be better to clarify the numbers.

Yuan, thanks so much for reviewing all the relevant patches. It's very helpful to us.

Signature · 1 line
Peter Xu
#4 lore raw

PATCH v6 2/5 meson: Introduce 'qatzip' feature to the build system

In reply to Yichen Wang

From: Bryan Zhang <[email protected]>

Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled.

---
 meson.build                   | 10 ++++++++++
 meson_options.txt             |  2 ++
 scripts/meson-buildoptions.sh |  3 +++
 3 files changed, 15 insertions(+)
meson.build +10 -0
index 6a93da48e1..ea977c6cbf 100644
@@ -1244,6 +1244,14 @@ if not get_option('uadk').auto() or have_system
      uadk = declare_dependency(dependencies: [libwd, libwd_comp])
   endif
 endif
+
+qatzip = not_found
+if get_option('qatzip').enabled()
+  qatzip = dependency('qatzip', version: '>=1.1.2',
+                      required: get_option('qatzip'),
+                      method: 'pkg-config')
+endif
+
 virgl = not_found
 
 have_vhost_user_gpu = have_tools and host_os == 'linux' and pixman.found()
@@ -2378,6 +2386,7 @@ config_host_data.set('CONFIG_STATX_MNT_ID', has_statx_mnt_id)
 config_host_data.set('CONFIG_ZSTD', zstd.found())
 config_host_data.set('CONFIG_QPL', qpl.found())
 config_host_data.set('CONFIG_UADK', uadk.found())
+config_host_data.set('CONFIG_QATZIP', qatzip.found())
 config_host_data.set('CONFIG_FUSE', fuse.found())
 config_host_data.set('CONFIG_FUSE_LSEEK', fuse_lseek.found())
 config_host_data.set('CONFIG_SPICE_PROTOCOL', spice_protocol.found())
@@ -4484,6 +4493,7 @@ summary_info += {'lzfse support':     liblzfse}
 summary_info += {'zstd support':      zstd}
 summary_info += {'Query Processing Library support': qpl}
 summary_info += {'UADK Library support': uadk}
+summary_info += {'qatzip support':    qatzip}
 summary_info += {'NUMA host support': numa}
 summary_info += {'capstone':          capstone}
 summary_info += {'libpmem support':   libpmem}
meson_options.txt +2 -0
index 0269fa0f16..35a69f6697 100644
@@ -261,6 +261,8 @@ option('qpl', type : 'feature', value : 'auto',
        description: 'Query Processing Library support')
 option('uadk', type : 'feature', value : 'auto',
        description: 'UADK Library support')
+option('qatzip', type: 'feature', value: 'disabled',
+       description: 'QATzip compression support')
 option('fuse', type: 'feature', value: 'auto',
        description: 'FUSE block device export')
 option('fuse_lseek', type : 'feature', value : 'auto',
scripts/meson-buildoptions.sh +3 -0
index cfadb5ea86..1ce467e9cc 100644
@@ -163,6 +163,7 @@ meson_options_help() {
   printf "%s\n" '  pixman          pixman support'
   printf "%s\n" '  plugins         TCG plugins via shared library loading'
   printf "%s\n" '  png             PNG support with libpng'
+  printf "%s\n" '  qatzip          QATzip compression support'
   printf "%s\n" '  qcow1           qcow1 image format support'
   printf "%s\n" '  qed             qed image format support'
   printf "%s\n" '  qga-vss         build QGA VSS support (broken with MinGW)'
@@ -427,6 +428,8 @@ _meson_option_parse() {
     --enable-png) printf "%s" -Dpng=enabled ;;
     --disable-png) printf "%s" -Dpng=disabled ;;
     --prefix=*) quote_sh "-Dprefix=$2" ;;
+    --enable-qatzip) printf "%s" -Dqatzip=enabled ;;
+    --disable-qatzip) printf "%s" -Dqatzip=disabled ;;
     --enable-qcow1) printf "%s" -Dqcow1=enabled ;;
     --disable-qcow1) printf "%s" -Dqcow1=disabled ;;
     --enable-qed) printf "%s" -Dqed=enabled ;;
Signature · 1 line
Yichen Wang
#5 lore raw

RE: [PATCH v6 2/5] meson: Introduce 'qatzip' feature to the build system

Quoted 37 lines

-----Original Message-----
From: Yichen Wang <[email protected]>
Sent: Tuesday, July 16, 2024 6:13 AM
To: Peter Xu <[email protected]>; Fabiano Rosas <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Marc-André Lureau <[email protected]>; Thomas Huth <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Laurent Vivier <[email protected]>; qemu- [email protected]
Cc: Hao Xiang <[email protected]>; Liu, Yuan1 <[email protected]>; Zou, Nanhai <[email protected]>; Ho-Ren (Jack) Chuang <[email protected]>; Wang, Yichen <[email protected]>; Bryan Zhang <[email protected]>
Subject: [PATCH v6 2/5] meson: Introduce 'qatzip' feature to the build system

From: Bryan Zhang <[email protected]>

Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled.

---
 meson.build                   | 10 ++++++++++
 meson_options.txt             |  2 ++
 scripts/meson-buildoptions.sh |  3 +++
 3 files changed, 15 insertions(+)
meson.build +0 -0
index 6a93da48e1..ea977c6cbf 100644
@@ -1244,6 +1244,14 @@ if not get_option('uadk').auto() or have_system
      uadk = declare_dependency(dependencies: [libwd, libwd_comp])

meson.build if not get_option('uadk').auto() or have_system · line 1254

Quoted 10 lines
meson.build +8 -0
   endif
 endif
+
+qatzip = not_found
+if get_option('qatzip').enabled()
+  qatzip = dependency('qatzip', version: '>=1.1.2',
+                      required: get_option('qatzip'),
+                      method: 'pkg-config')
+endif
+

How about changing to the following code
if not get_option('qatzip').auto() or have_system
qatzip = dependency('qatzip', version: '>=1.1.2',
required: get_option('qatzip'),
method: 'pkg-config')
endif

This means that on all Qemu emulation targets, always use --enable-qatzip and --disable-qatzip to determine whether qatzip is required.

I think your previous code can remove if get_option('qatzip').enabled() And it also can work

Quoted 26 lines
meson.build +0 -0
 virgl = not_found
 
 have_vhost_user_gpu = have_tools and host_os == 'linux' and

pixman.found()

meson.build +1 -0
@@ -2378,6 +2386,7 @@ config_host_data.set('CONFIG_STATX_MNT_ID',
 has_statx_mnt_id)
 config_host_data.set('CONFIG_ZSTD', zstd.found())
 config_host_data.set('CONFIG_QPL', qpl.found())
 config_host_data.set('CONFIG_UADK', uadk.found())
+config_host_data.set('CONFIG_QATZIP', qatzip.found())
 config_host_data.set('CONFIG_FUSE', fuse.found())
 config_host_data.set('CONFIG_FUSE_LSEEK', fuse_lseek.found())

config_host_data.set('CONFIG_SPICE_PROTOCOL', spice_protocol.found())

meson.build +1 -0
@@ -4484,6 +4493,7 @@ summary_info += {'lzfse support':     liblzfse}
 summary_info += {'zstd support':      zstd}
 summary_info += {'Query Processing Library support': qpl}
 summary_info += {'UADK Library support': uadk}
+summary_info += {'qatzip support':    qatzip}
 summary_info += {'NUMA host support': numa}
 summary_info += {'capstone':          capstone}
 summary_info += {'libpmem support':   libpmem}
meson_options.txt +0 -0
index 0269fa0f16..35a69f6697 100644

meson_options.txt option('qpl', type : 'feature', value : 'auto', · line 264

Quoted 4 lines
meson_options.txt +1 -0
@@ -261,6 +261,8 @@ option('qpl', type : 'feature', value : 'auto',
        description: 'Query Processing Library support')
 option('uadk', type : 'feature', value : 'auto',
        description: 'UADK Library support')
+option('qatzip', type: 'feature', value: 'disabled',

If you agree with the above changes, set the qatzip value to auto option('qatzip', type: 'feature', value: 'auto'

Quoted 29 lines
meson_options.txt +1 -0
+       description: 'QATzip compression support')
 option('fuse', type: 'feature', value: 'auto',
        description: 'FUSE block device export')
 option('fuse_lseek', type : 'feature', value : 'auto',
scripts/meson-buildoptions.sh +0 -0
index cfadb5ea86..1ce467e9cc 100644
@@ -163,6 +163,7 @@ meson_options_help() {
   printf "%s\n" '  pixman          pixman support'
   printf "%s\n" '  plugins         TCG plugins via shared library

loading'

   printf "%s\n" '  png             PNG support with libpng'
+  printf "%s\n" '  qatzip          QATzip compression support'
   printf "%s\n" '  qcow1           qcow1 image format support'
   printf "%s\n" '  qed             qed image format support'
   printf "%s\n" '  qga-vss         build QGA VSS support (broken with

MinGW)'

scripts/meson-buildoptions.sh +2 -1
@@ -427,6 +428,8 @@ _meson_option_parse() {
     --enable-png) printf "%s" -Dpng=enabled ;;
     --disable-png) printf "%s" -Dpng=disabled ;;
     --prefix=*) quote_sh "-Dprefix=$2" ;;
+    --enable-qatzip) printf "%s" -Dqatzip=enabled ;;
+    --disable-qatzip) printf "%s" -Dqatzip=disabled ;;
     --enable-qcow1) printf "%s" -Dqcow1=enabled ;;
     --disable-qcow1) printf "%s" -Dqcow1=disabled ;;
     --enable-qed) printf "%s" -Dqed=enabled ;;
--

Yichen Wang

#6 lore raw

PATCH v6 3/5 migration: Add migration parameters for QATzip

In reply to Yichen Wang

From: Bryan Zhang <[email protected]>

Adds support for migration parameters to control QATzip compression level and to enable/disable software fallback when QAT hardware is unavailable. This is a preparatory commit for a subsequent commit that will actually use QATzip compression.

---
 migration/migration-hmp-cmds.c |  4 ++++
 migration/options.c            | 34 ++++++++++++++++++++++++++++++++++
 migration/options.h            |  1 +
 qapi/migration.json            | 18 ++++++++++++++++++
 4 files changed, 57 insertions(+)
migration/migration-hmp-cmds.c +4 -0
index 7d608d26e1..28165cfc9e 100644
@@ -576,6 +576,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
         p->has_multifd_zlib_level = true;
         visit_type_uint8(v, param, &p->multifd_zlib_level, &err);
         break;
+    case MIGRATION_PARAMETER_MULTIFD_QATZIP_LEVEL:
+        p->has_multifd_qatzip_level = true;
+        visit_type_uint8(v, param, &p->multifd_qatzip_level, &err);
+        break;
     case MIGRATION_PARAMETER_MULTIFD_ZSTD_LEVEL:
         p->has_multifd_zstd_level = true;
         visit_type_uint8(v, param, &p->multifd_zstd_level, &err);
migration/options.c +34 -0
index 645f55003d..147cd2b8fd 100644
@@ -55,6 +55,13 @@
 #define DEFAULT_MIGRATE_MULTIFD_COMPRESSION MULTIFD_COMPRESSION_NONE
 /* 0: means nocompress, 1: best speed, ... 9: best compress ratio */
 #define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1
+/*
+ * 1: best speed, ... 9: best compress ratio
+ * There is some nuance here. Refer to QATzip documentation to understand
+ * the mapping of QATzip levels to standard deflate levels.
+ */
+#define DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL 1
+
 /* 0: means nocompress, 1: best speed, ... 20: best compress ratio */
 #define DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL 1
 
@@ -123,6 +130,9 @@ Property migration_properties[] = {
     DEFINE_PROP_UINT8("multifd-zlib-level", MigrationState,
                       parameters.multifd_zlib_level,
                       DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL),
+    DEFINE_PROP_UINT8("multifd-qatzip-level", MigrationState,
+                      parameters.multifd_qatzip_level,
+                      DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL),
     DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState,
                       parameters.multifd_zstd_level,
                       DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL),
@@ -787,6 +797,13 @@ int migrate_multifd_zlib_level(void)
     return s->parameters.multifd_zlib_level;
 }
 
+int migrate_multifd_qatzip_level(void)
+{
+    MigrationState *s = migrate_get_current();
+
+    return s->parameters.multifd_qatzip_level;
+}
+
 int migrate_multifd_zstd_level(void)
 {
     MigrationState *s = migrate_get_current();
@@ -892,6 +909,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp)
     params->multifd_compression = s->parameters.multifd_compression;
     params->has_multifd_zlib_level = true;
     params->multifd_zlib_level = s->parameters.multifd_zlib_level;
+    params->has_multifd_qatzip_level = true;
+    params->multifd_qatzip_level = s->parameters.multifd_qatzip_level;
     params->has_multifd_zstd_level = true;
     params->multifd_zstd_level = s->parameters.multifd_zstd_level;
     params->has_xbzrle_cache_size = true;
@@ -946,6 +965,7 @@ void migrate_params_init(MigrationParameters *params)
     params->has_multifd_channels = true;
     params->has_multifd_compression = true;
     params->has_multifd_zlib_level = true;
+    params->has_multifd_qatzip_level = true;
     params->has_multifd_zstd_level = true;
     params->has_xbzrle_cache_size = true;
     params->has_max_postcopy_bandwidth = true;
@@ -1038,6 +1058,14 @@ bool migrate_params_check(MigrationParameters *params, Error **errp)
         return false;
     }
 
+    if (params->has_multifd_qatzip_level &&
+        ((params->multifd_qatzip_level > 9) ||
+        (params->multifd_qatzip_level < 1))) {
+        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_qatzip_level",
+                   "a value between 1 and 9");
+        return false;
+    }
+
     if (params->has_multifd_zstd_level &&
         (params->multifd_zstd_level > 20)) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zstd_level",
@@ -1195,6 +1223,9 @@ static void migrate_params_test_apply(MigrateSetParameters *params,
     if (params->has_multifd_compression) {
         dest->multifd_compression = params->multifd_compression;
     }
+    if (params->has_multifd_qatzip_level) {
+        dest->multifd_qatzip_level = params->multifd_qatzip_level;
+    }
     if (params->has_multifd_zlib_level) {
         dest->multifd_zlib_level = params->multifd_zlib_level;
     }
@@ -1315,6 +1346,9 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp)
     if (params->has_multifd_compression) {
         s->parameters.multifd_compression = params->multifd_compression;
     }
+    if (params->has_multifd_qatzip_level) {
+        s->parameters.multifd_qatzip_level = params->multifd_qatzip_level;
+    }
     if (params->has_multifd_zlib_level) {
         s->parameters.multifd_zlib_level = params->multifd_zlib_level;
     }
migration/options.h +1 -0
index a2397026db..a0bd6edc06 100644
@@ -78,6 +78,7 @@ uint64_t migrate_max_postcopy_bandwidth(void);
 int migrate_multifd_channels(void);
 MultiFDCompression migrate_multifd_compression(void);
 int migrate_multifd_zlib_level(void);
+int migrate_multifd_qatzip_level(void);
 int migrate_multifd_zstd_level(void);
 uint8_t migrate_throttle_trigger_threshold(void);
 const char *migrate_tls_authz(void);
qapi/migration.json +18 -0
index 1234bef888..cd08f2f710 100644
@@ -789,6 +789,11 @@
 #     speed, and 9 means best compression ratio which will consume
 #     more CPU. Defaults to 1.  (Since 5.0)
 #
+# @multifd-qatzip-level: Set the compression level to be used in live
+#     migration. The level is an integer between 1 and 9, where 1 means
+#     the best compression speed, and 9 means the best compression
+#     ratio which will consume more CPU. Defaults to 1. (Since 9.1)
+#
 # @multifd-zstd-level: Set the compression level to be used in live
 #     migration, the compression level is an integer between 0 and 20,
 #     where 0 means no compression, 1 means the best compression
@@ -849,6 +854,7 @@
            'xbzrle-cache-size', 'max-postcopy-bandwidth',
            'max-cpu-throttle', 'multifd-compression',
            'multifd-zlib-level', 'multifd-zstd-level',
+           'multifd-qatzip-level',
            'block-bitmap-mapping',
            { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable'] },
            'vcpu-dirty-limit',
@@ -964,6 +970,11 @@
 #     speed, and 9 means best compression ratio which will consume
 #     more CPU. Defaults to 1.  (Since 5.0)
 #
+# @multifd-qatzip-level: Set the compression level to be used in live
+#     migration. The level is an integer between 1 and 9, where 1 means
+#     the best compression speed, and 9 means the best compression
+#     ratio which will consume more CPU. Defaults to 1. (Since 9.1)
+#
 # @multifd-zstd-level: Set the compression level to be used in live
 #     migration, the compression level is an integer between 0 and 20,
 #     where 0 means no compression, 1 means the best compression
@@ -1037,6 +1048,7 @@
             '*max-cpu-throttle': 'uint8',
             '*multifd-compression': 'MultiFDCompression',
             '*multifd-zlib-level': 'uint8',
+            '*multifd-qatzip-level': 'uint8',
             '*multifd-zstd-level': 'uint8',
             '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ],
             '*x-vcpu-dirty-limit-period': { 'type': 'uint64',
@@ -1168,6 +1180,11 @@
 #     speed, and 9 means best compression ratio which will consume
 #     more CPU. Defaults to 1.  (Since 5.0)
 #
+# @multifd-qatzip-level: Set the compression level to be used in live
+#     migration. The level is an integer between 1 and 9, where 1 means
+#     the best compression speed, and 9 means the best compression
+#     ratio which will consume more CPU. Defaults to 1. (Since 9.1)
+#
 # @multifd-zstd-level: Set the compression level to be used in live
 #     migration, the compression level is an integer between 0 and 20,
 #     where 0 means no compression, 1 means the best compression
@@ -1238,6 +1255,7 @@
             '*max-cpu-throttle': 'uint8',
             '*multifd-compression': 'MultiFDCompression',
             '*multifd-zlib-level': 'uint8',
+            '*multifd-qatzip-level': 'uint8',
             '*multifd-zstd-level': 'uint8',
             '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ],
             '*x-vcpu-dirty-limit-period': { 'type': 'uint64',
Signature · 1 line
Yichen Wang
Acked-by on v6 3/5
#7 lore raw

Re: [PATCH v6 3/5] migration: Add migration parameters for QATzip

Quoted 6 lines · Yichen Wang

From: Bryan Zhang <[email protected]>

Adds support for migration parameters to control QATzip compression level and to enable/disable software fallback when QAT hardware is unavailable. This is a preparatory commit for a subsequent commit that will actually use QATzip compression.

Not sure splitting these two makes sense, but we're at v6, so let's not rock the boat.

Quoted 3 lines

QAPI schema

#8 lore raw

PATCH v6 4/5 migration: Introduce 'qatzip' compression method

In reply to Yichen Wang

From: Bryan Zhang <[email protected]>

Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.

---
 hw/core/qdev-properties-system.c |   2 +-
 migration/meson.build            |   1 +
 migration/multifd-qatzip.c       | 382 +++++++++++++++++++++++++++++++
 migration/multifd.h              |   5 +-
 qapi/migration.json              |   3 +
 tests/qtest/meson.build          |   4 +
 6 files changed, 394 insertions(+), 3 deletions(-)
 create mode 100644 migration/multifd-qatzip.c
hw/core/qdev-properties-system.c +1 -1
index f13350b4fb..a56fbf728d 100644
@@ -659,7 +659,7 @@ const PropertyInfo qdev_prop_fdc_drive_type = {
 const PropertyInfo qdev_prop_multifd_compression = {
     .name = "MultiFDCompression",
     .description = "multifd_compression values, "
-                   "none/zlib/zstd/qpl/uadk",
+                   "none/zlib/zstd/qpl/uadk/qatzip",
     .enum_table = &MultiFDCompression_lookup,
     .get = qdev_propinfo_get_enum,
     .set = qdev_propinfo_set_enum,
migration/meson.build +1 -0
index 5ce2acb41e..c9454c26ae 100644
@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c'))
 system_ss.add(when: zstd, if_true: files('multifd-zstd.c'))
 system_ss.add(when: qpl, if_true: files('multifd-qpl.c'))
 system_ss.add(when: uadk, if_true: files('multifd-uadk.c'))
+system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c'))
 
 specific_ss.add(when: 'CONFIG_SYSTEM_ONLY',
                 if_true: files('ram.c',
migration/multifd-qatzip.c new file +382 -0

382 lines not on this page · raw message

migration/multifd.h +3 -2
index 0ecd6f47d7..adceb65050 100644
@@ -34,14 +34,15 @@ MultiFDRecvData *multifd_get_recv_data(void);
 /* Multifd Compression flags */
 #define MULTIFD_FLAG_SYNC (1 << 0)
 
-/* We reserve 4 bits for compression methods */
-#define MULTIFD_FLAG_COMPRESSION_MASK (0xf << 1)
+/* We reserve 5 bits for compression methods */
+#define MULTIFD_FLAG_COMPRESSION_MASK (0x1f << 1)
 /* we need to be compatible. Before compression value was 0 */
 #define MULTIFD_FLAG_NOCOMP (0 << 1)
 #define MULTIFD_FLAG_ZLIB (1 << 1)
 #define MULTIFD_FLAG_ZSTD (2 << 1)
 #define MULTIFD_FLAG_QPL (4 << 1)
 #define MULTIFD_FLAG_UADK (8 << 1)
+#define MULTIFD_FLAG_QATZIP (16 << 1)
 
 /* This value needs to be a multiple of qemu_target_page_size() */
 #define MULTIFD_PACKET_SIZE (512 * 1024)
qapi/migration.json +3 -0
index cd08f2f710..42b5363449 100644
@@ -558,6 +558,8 @@
 #
 # @zstd: use zstd compression method.
 #
+# @qatzip: use qatzip compression method. (Since 9.1)
+#
 # @qpl: use qpl compression method.  Query Processing Library(qpl) is
 #       based on the deflate compression algorithm and use the Intel
 #       In-Memory Analytics Accelerator(IAA) accelerated compression
@@ -570,6 +572,7 @@
 { 'enum': 'MultiFDCompression',
   'data': [ 'none', 'zlib',
             { 'name': 'zstd', 'if': 'CONFIG_ZSTD' },
+            { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'},
             { 'name': 'qpl', 'if': 'CONFIG_QPL' },
             { 'name': 'uadk', 'if': 'CONFIG_UADK' } ] }
 
tests/qtest/meson.build +4 -0
index 6508bfb1a2..3068d73e08 100644
@@ -327,6 +327,10 @@ if gnutls.found()
   endif
 endif
 
+if qatzip.found()
+  migration_files += [qatzip]
+endif
+
 qtests = {
   'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'],
   'cdrom-test': files('boot-sector.c'),
Signature · 1 line
Yichen Wang
#9 lore raw

RE: [PATCH v6 4/5] migration: Introduce 'qatzip' compression method

Quoted 137 lines

-----Original Message-----
From: Yichen Wang <[email protected]>
Sent: Tuesday, July 16, 2024 6:13 AM
To: Peter Xu <[email protected]>; Fabiano Rosas <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Marc-André Lureau <[email protected]>; Thomas Huth <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Laurent Vivier <[email protected]>; qemu- [email protected]
Cc: Hao Xiang <[email protected]>; Liu, Yuan1 <[email protected]>; Zou, Nanhai <[email protected]>; Ho-Ren (Jack) Chuang <[email protected]>; Wang, Yichen <[email protected]>; Bryan Zhang <[email protected]>
Subject: [PATCH v6 4/5] migration: Introduce 'qatzip' compression method

From: Bryan Zhang <[email protected]>

Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.

---
 hw/core/qdev-properties-system.c |   2 +-
 migration/meson.build            |   1 +
 migration/multifd-qatzip.c       | 382 +++++++++++++++++++++++++++++++
 migration/multifd.h              |   5 +-
 qapi/migration.json              |   3 +
 tests/qtest/meson.build          |   4 +
 6 files changed, 394 insertions(+), 3 deletions(-)
 create mode 100644 migration/multifd-qatzip.c
hw/core/qdev-properties- renamed from hw/core/qdev-properties-system.c +0 -0

system.c
index f13350b4fb..a56fbf728d 100644

hw/core/qdev-properties-system.c +1 -1
@@ -659,7 +659,7 @@ const PropertyInfo qdev_prop_fdc_drive_type = {
 const PropertyInfo qdev_prop_multifd_compression = {
     .name = "MultiFDCompression",
     .description = "multifd_compression values, "
-                   "none/zlib/zstd/qpl/uadk",
+                   "none/zlib/zstd/qpl/uadk/qatzip",
     .enum_table = &MultiFDCompression_lookup,
     .get = qdev_propinfo_get_enum,
     .set = qdev_propinfo_set_enum,
migration/meson.build +1 -0
index 5ce2acb41e..c9454c26ae 100644
@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c'))
 system_ss.add(when: zstd, if_true: files('multifd-zstd.c'))
 system_ss.add(when: qpl, if_true: files('multifd-qpl.c'))
 system_ss.add(when: uadk, if_true: files('multifd-uadk.c'))
+system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c'))
 
 specific_ss.add(when: 'CONFIG_SYSTEM_ONLY',
                 if_true: files('ram.c',
migration/multifd-qatzip.c new file +11 -0
new file mode 100644
index 0000000000..b74cda503a
@@ -0,0 +1,382 @@
+/*
+ * Multifd QATzip compression implementation
+ *
+ * Copyright (c) Bytedance
+ *
+ * Authors:
+ *  Bryan Zhang <[email protected]>
+ *  Hao Xiang <[email protected]>
+ *  Yichen Wang <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or

later.
+ * See the COPYING file in the top-level directory.
+ */

migration/multifd-qatzip.c new file +52 -0
+
+#include "qemu/osdep.h"
+#include "exec/ramblock.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "qapi/qapi-types-migration.h"
+#include "options.h"
+#include "multifd.h"
+#include <qatzip.h>
+
+typedef struct {
+    /*
+     * Unique session for use with QATzip API
+     */
+    QzSession_T sess;
+
+    /*
+     * For compression: Buffer for pages to compress
+     * For decompression: Buffer for data to decompress
+     */
+    uint8_t *in_buf;
+    uint32_t in_len;
+
+    /*
+     * For compression: Output buffer of compressed data
+     * For decompression: Output buffer of decompressed data
+     */
+    uint8_t *out_buf;
+    uint32_t out_len;
+} QatzipData;
+
+/**
+ * qatzip_send_setup: Set up QATzip session and private buffers.
+ *
+ * @param p    Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return     0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_send_setup(MultiFDSendParams *p, Error **errp)
+{
+    QatzipData *q;
+    QzSessionParamsDeflate_T params;
+    const char *err_msg;
+    int ret;
+
+    q = g_new0(QatzipData, 1);
+    p->compress_data = q;
+    /* We need one extra place for the packet header */
+    p->iov = g_new0(struct iovec, 2);
+
+    /*
+     * Prefer without sw_fallback because of bad performance with

sw_fallback.

migration/multifd-qatzip.c new file +3 -0
+     * Warn if sw_fallback needs to be used.
+     */
+    ret = qzInit(&q->sess, false);

migration/multifd-qatzip.c line 78

Quoted 10 lines
migration/multifd-qatzip.c new file +10 -0
+    if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+        /* Warn, and try with sw_fallback. */
+        warn_report_once(
+            "[Sender] Initializing QAT with software fallback...");
+        ret = qzInit(&q->sess, true);
+        if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+            err_msg = "qzInit failed";
+            goto err;
+        }
+    }

I have some concerns here
1. if the QAT hardware and driver do not exist, the qzMalloc(q->in_len, 0, PINNED_MEM) will fail, since the PINNED_MEM can't be allocated. So if using the software path, COMMON_MEM should be used for qzMalloc.

2. And if the QAT software path is enabled, there will be a lot of log printing, which is a very bad experience, the error log is below, and it seems that it cannot be disabled.
Error no hardware, switch to SW if permitted status = -6 g_process.qz_init_status = QZ_NO_HW

I think when the user chooses qatzip, his environment should support QAT hardware, so we can always use qzInit(&q->sess, true) and PINNED_MEM to initialization.

However, we may still need to consider the Qemu CI validation that qatzip + no QAT hardware case. Maybe we can complete the migration without compression after QAT initialization fails including qzInit and qzMalloc with PINNED_MEM.

Regarding your concern about performance degradation after fallback to the software path, qatzip will print relevant logs in standard error, maybe this is enough to notify the user.

Regarding the above concerns, you can uninstall the QAT drivers (qat_4xxx,usdm_drv,intel_qat) and run "make check", the qatzip will fail.

Quoted 11 lines
migration/multifd-qatzip.c new file +11 -0
+    ret = qzGetDefaultsDeflate(&params);
+    if (ret != QZ_OK) {
+        err_msg = "qzGetDefaultsDeflate failed";
+        goto err;
+    }
+
+    /* Make sure to use configured QATzip compression level. */
+    params.common_params.comp_lvl = migrate_multifd_qatzip_level();
+    ret = qzSetupSessionDeflate(&q->sess, &params);
+    if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+        err_msg = "qzSetupSessionDeflate failed";

migration/multifd-qatzip.c line 100

Quoted 10 lines
migration/multifd-qatzip.c new file +10 -0
+        goto err;
+    }
+
+    if (MULTIFD_PACKET_SIZE > UINT32_MAX) {
+        err_msg = "packet size too large for QAT";
+        goto err;
+    }
+
+    q->in_len = MULTIFD_PACKET_SIZE;
+    q->in_buf = qzMalloc(q->in_len, 0, PINNED_MEM);

Peter previously asked about the memory size used QAT here, we need to check the QAT memory footprint and update the doc patch.

We need to let users know that the system needs to reserve some memory when using qatzip.

Quoted 27 lines
migration/multifd-qatzip.c new file +21 -0
+    if (!q->in_buf) {
+        err_msg = "qzMalloc failed";
+        goto err;
+    }
+
+    q->out_len = qzMaxCompressedLength(MULTIFD_PACKET_SIZE, &q->sess);
+    q->out_buf = qzMalloc(q->out_len, 0, PINNED_MEM);
+    if (!q->out_buf) {
+        err_msg = "qzMalloc failed";
+        goto err;
+    }
+
+    return 0;
+
+err:
+    error_setg(errp, "multifd %u: [sender] %s", p->id, err_msg);
+    return -1;
+}
+
+/**
+ * qatzip_send_cleanup: Tear down QATzip session and release private

buffers.
+ *
+ * @param p Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return None
+ */

migration/multifd-qatzip.c line 130

Quoted 4 lines
migration/multifd-qatzip.c new file +4 -0
+static void qatzip_send_cleanup(MultiFDSendParams *p, Error **errp)
+{
+    QatzipData *q = p->compress_data;
+

Add NULL pointer check here, when the number of channels is more than 1 And if the first channel's qatzip_send_setup returns an error, the Qemu will crash here, since it will be called multiple times according to the number of channels, but only the first channel invokes qatzip_send_setup.

if (!q) {
    return;
}
Quoted 171 lines
migration/multifd-qatzip.c new file +35 -0
+    if (q->in_buf) {
+        qzFree(q->in_buf);
+    }
+    if (q->out_buf) {
+        qzFree(q->out_buf);
+    }
+    (void)qzTeardownSession(&q->sess);
+    (void)qzClose(&q->sess);
+
+    g_free(p->iov);
+    p->iov = NULL;
+    g_free(q);
+    p->compress_data = NULL;
+}
+
+/**
+ * qatzip_send_prepare: Compress pages and update IO channel info.
+ *
+ * @param p    Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return     0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_send_prepare(MultiFDSendParams *p, Error **errp)
+{
+    MultiFDPages_t *pages = p->pages;
+    QatzipData *q = p->compress_data;
+    int ret;
+    unsigned int in_len, out_len;
+
+    if (!multifd_send_prepare_common(p)) {
+        goto out;
+    }
+
+    /*
+     * Unlike other multifd compression implementations, we use a non-

streaming
+ * API and place all the data into one buffer, rather than sending
each
+ * page to the compression API at a time. Based on initial
benchmarks, the
+ * non-streaming API outperforms the streaming API. Plus, the logic
in QEMU
+ * is friendly to using the non-streaming API anyway. If either of
these
+ * statements becomes no longer true, we can revisit adding a
streaming

migration/multifd-qatzip.c new file +16 -0
+     * implementation.
+     */
+    for (int i = 0; i < pages->normal_num; i++) {
+        memcpy(q->in_buf + (i * p->page_size),
+               pages->block->host + pages->offset[i],
+               p->page_size);
+    }
+
+    in_len = pages->normal_num * p->page_size;
+    if (in_len > q->in_len) {
+        error_setg(errp, "multifd %u: unexpectedly large input", p->id);
+        return -1;
+    }
+    out_len = q->out_len;
+
+    ret = qzCompress(&q->sess, q->in_buf, &in_len, q->out_buf, &out_len,

1);

migration/multifd-qatzip.c new file +2 -0
+    if (ret != QZ_OK) {
+        error_setg(errp, "multifd %u: QATzip returned %d instead of

QZ_OK",

migration/multifd-qatzip.c new file +5 -0
+                   p->id, ret);
+        return -1;
+    }
+    if (in_len != pages->normal_num * p->page_size) {
+        error_setg(errp, "multifd %u: QATzip failed to compress all

input",

migration/multifd-qatzip.c new file +34 -0
+                   p->id);
+        return -1;
+    }
+
+    p->iov[p->iovs_num].iov_base = q->out_buf;
+    p->iov[p->iovs_num].iov_len = out_len;
+    p->iovs_num++;
+    p->next_packet_size = out_len;
+
+out:
+    p->flags |= MULTIFD_FLAG_QATZIP;
+    multifd_send_fill_packet(p);
+    return 0;
+}
+
+/**
+ * qatzip_recv_setup: Set up QATzip session and allocate private buffers.
+ *
+ * @param p    Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return     0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_recv_setup(MultiFDRecvParams *p, Error **errp)
+{
+    QatzipData *q;
+    QzSessionParamsDeflate_T params;
+    const char *err_msg;
+    int ret;
+
+    q = g_new0(QatzipData, 1);
+    p->compress_data = q;
+
+    /*
+     * Prefer without sw_fallback because of bad performance with

sw_fallback.

migration/multifd-qatzip.c new file +28 -0
+     * Warn if sw_fallback needs to be used.
+     */
+    ret = qzInit(&q->sess, false);
+    if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+        /* Warn, and try with sw_fallback. */
+        warn_report_once(
+            "[Receiver] Initializing QAT with software fallback...");
+        ret = qzInit(&q->sess, true);
+        if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+            err_msg = "qzInit failed";
+            goto err;
+        }
+    }
+
+    ret = qzGetDefaultsDeflate(&params);
+    if (ret != QZ_OK) {
+        err_msg = "qzGetDefaultsDeflate failed";
+        goto err;
+    }
+
+    ret = qzSetupSessionDeflate(&q->sess, &params);
+    if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+        err_msg = "qzSetupSessionDeflate failed";
+        goto err;
+    }
+
+    /*
+     * Reserve extra spaces for the incoming packets. Current

implementation
+ * doesn't send uncompressed pages in case the compression gets too
big.

migration/multifd-qatzip.c new file +28 -0
+     */
+    q->in_len = MULTIFD_PACKET_SIZE * 2;
+    /*
+     * PINNED_MEM is an enum from qatzip headers, which means to use
+     * kzalloc_node() to allocate memory for QAT DMA purposes.
+     */
+    q->in_buf = qzMalloc(q->in_len, 0, PINNED_MEM);
+    if (!q->in_buf) {
+        err_msg = "qzMalloc failed";
+        goto err;
+    }
+
+    q->out_len = MULTIFD_PACKET_SIZE;
+    q->out_buf = qzMalloc(q->out_len, 0, PINNED_MEM);
+    if (!q->out_buf) {
+        err_msg = "qzMalloc failed";
+        goto err;
+    }
+
+    return 0;
+
+err:
+    error_setg(errp, "multifd %u: [receiver] %s", p->id, err_msg);
+    return -1;
+}
+
+/**
+ * qatzip_recv_cleanup: Tear down QATzip session and release private

buffers.
+ *
+ * @param p Multifd channel params
+ * @return None
+ */

migration/multifd-qatzip.c line 291

Quoted 3 lines
migration/multifd-qatzip.c new file +3 -0
+static void qatzip_recv_cleanup(MultiFDRecvParams *p)
+{
+    QatzipData *q = p->compress_data;

ditto

Quoted 143 lines
migration/multifd-qatzip.c new file +32 -0
+    if (q->in_buf) {
+        qzFree(q->in_buf);
+    }
+    if (q->out_buf) {
+        qzFree(q->out_buf);
+    }
+    (void)qzTeardownSession(&q->sess);
+    (void)qzClose(&q->sess);
+    g_free(q);
+    p->compress_data = NULL;
+}
+
+
+/**
+ * qatzip_recv: Decompress pages and copy them to the appropriate
+ * locations.
+ *
+ * @param p    Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return     0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_recv(MultiFDRecvParams *p, Error **errp)
+{
+    QatzipData *q = p->compress_data;
+    int ret;
+    unsigned int in_len, out_len;
+    uint32_t in_size = p->next_packet_size;
+    uint32_t expected_size = p->normal_num * p->page_size;
+    uint32_t flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK;
+
+    if (in_size > q->in_len) {
+        error_setg(errp, "multifd %u: received unexpectedly large

packet",

migration/multifd-qatzip.c new file +6 -0
+                   p->id);
+        return -1;
+    }
+
+    if (flags != MULTIFD_FLAG_QATZIP) {
+        error_setg(errp, "multifd %u: flags received %x flags

expected %x",

migration/multifd-qatzip.c new file +18 -0
+                   p->id, flags, MULTIFD_FLAG_QATZIP);
+        return -1;
+    }
+
+    multifd_recv_zero_page_process(p);
+    if (!p->normal_num) {
+        assert(in_size == 0);
+        return 0;
+    }
+
+    ret = qio_channel_read_all(p->c, (void *)q->in_buf, in_size, errp);
+    if (ret != 0) {
+        return ret;
+    }
+
+    in_len = in_size;
+    out_len = q->out_len;
+    ret = qzDecompress(&q->sess, q->in_buf, &in_len, q->out_buf,

&out_len);

migration/multifd-qatzip.c new file +6 -0
+    if (ret != QZ_OK) {
+        error_setg(errp, "multifd %u: qzDecompress failed", p->id);
+        return -1;
+    }
+    if (out_len != expected_size) {
+        error_setg(errp, "multifd %u: packet size received %u size

expected %u",

migration/multifd-qatzip.c new file +25 -0
+                   p->id, out_len, expected_size);
+        return -1;
+    }
+
+    /* Copy each page to its appropriate location. */
+    for (int i = 0; i < p->normal_num; i++) {
+        memcpy(p->host + p->normal[i],
+               q->out_buf + p->page_size * i,
+               p->page_size);
+    }
+    return 0;
+}
+
+static MultiFDMethods multifd_qatzip_ops = {
+    .send_setup = qatzip_send_setup,
+    .send_cleanup = qatzip_send_cleanup,
+    .send_prepare = qatzip_send_prepare,
+    .recv_setup = qatzip_recv_setup,
+    .recv_cleanup = qatzip_recv_cleanup,
+    .recv = qatzip_recv
+};
+
+static void multifd_qatzip_register(void)
+{
+    multifd_register_ops(MULTIFD_COMPRESSION_QATZIP,

&multifd_qatzip_ops);

migration/multifd-qatzip.c new file +3 -0
+}
+
+migration_init(multifd_qatzip_register);
migration/multifd.h +3 -2
index 0ecd6f47d7..adceb65050 100644
@@ -34,14 +34,15 @@ MultiFDRecvData *multifd_get_recv_data(void);
 /* Multifd Compression flags */
 #define MULTIFD_FLAG_SYNC (1 << 0)
 
-/* We reserve 4 bits for compression methods */
-#define MULTIFD_FLAG_COMPRESSION_MASK (0xf << 1)
+/* We reserve 5 bits for compression methods */
+#define MULTIFD_FLAG_COMPRESSION_MASK (0x1f << 1)
 /* we need to be compatible. Before compression value was 0 */
 #define MULTIFD_FLAG_NOCOMP (0 << 1)
 #define MULTIFD_FLAG_ZLIB (1 << 1)
 #define MULTIFD_FLAG_ZSTD (2 << 1)
 #define MULTIFD_FLAG_QPL (4 << 1)
 #define MULTIFD_FLAG_UADK (8 << 1)
+#define MULTIFD_FLAG_QATZIP (16 << 1)
 
 /* This value needs to be a multiple of qemu_target_page_size() */
 #define MULTIFD_PACKET_SIZE (512 * 1024)
qapi/migration.json +3 -0
index cd08f2f710..42b5363449 100644
@@ -558,6 +558,8 @@
 #
 # @zstd: use zstd compression method.
 #
+# @qatzip: use qatzip compression method. (Since 9.1)
+#
 # @qpl: use qpl compression method.  Query Processing Library(qpl) is
 #       based on the deflate compression algorithm and use the Intel
 #       In-Memory Analytics Accelerator(IAA) accelerated compression
@@ -570,6 +572,7 @@
 { 'enum': 'MultiFDCompression',
   'data': [ 'none', 'zlib',
             { 'name': 'zstd', 'if': 'CONFIG_ZSTD' },
+            { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'},
             { 'name': 'qpl', 'if': 'CONFIG_QPL' },
             { 'name': 'uadk', 'if': 'CONFIG_UADK' } ] }
 
tests/qtest/meson.build +0 -0
index 6508bfb1a2..3068d73e08 100644

tests/qtest/meson.build if gnutls.found() · line 333

Quoted 7 lines
tests/qtest/meson.build +4 -0
@@ -327,6 +327,10 @@ if gnutls.found()
   endif
 endif
 
+if qatzip.found()
+  migration_files += [qatzip]
+endif
+

I think this part can be removed, it seems no need for qatzip dependency in migration tests

Quoted 5 lines
tests/qtest/meson.build +0 -1
 qtests = {
   'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'],
   'cdrom-test': files('boot-sector.c'),
--

Yichen Wang

Acked-by on v6 4/5
#10 lore raw

Re: [PATCH v6 4/5] migration: Introduce 'qatzip' compression method

In reply to Yichen Wang
Quoted 9 lines · Yichen Wang

From: Bryan Zhang <[email protected]>

Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression.

QAPI schema

#11 lore raw

PATCH v6 5/5 tests/migration: Add integration test for 'qatzip' compression method

In reply to Yichen Wang

From: Bryan Zhang <[email protected]>

Adds an integration test for 'qatzip'.

---
 tests/qtest/migration-test.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
tests/qtest/migration-test.c +31 -0
index 70b606b888..98f60d58a7 100644
@@ -32,6 +32,10 @@
 # endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
+#ifdef CONFIG_QATZIP
+#include <qatzip.h>
+#endif /* CONFIG_QATZIP */
+
 /* For dirty ring test; so far only x86_64 is supported */
 #if defined(__linux__) && defined(HOST_X86_64)
 #include "linux/kvm.h"
@@ -2992,6 +2996,18 @@ test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from,
 }
 #endif /* CONFIG_ZSTD */
 
+#ifdef CONFIG_QATZIP
+static void *
+test_migrate_precopy_tcp_multifd_qatzip_start(QTestState *from,
+                                              QTestState *to)
+{
+    migrate_set_parameter_int(from, "multifd-qatzip-level", 2);
+    migrate_set_parameter_int(to, "multifd-qatzip-level", 2);
+
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "qatzip");
+}
+#endif
+
 #ifdef CONFIG_QPL
 static void *
 test_migrate_precopy_tcp_multifd_qpl_start(QTestState *from,
@@ -3089,6 +3105,17 @@ static void test_multifd_tcp_zstd(void)
 }
 #endif
 
+#ifdef CONFIG_QATZIP
+static void test_multifd_tcp_qatzip(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_precopy_tcp_multifd_qatzip_start,
+    };
+    test_precopy_common(&args);
+}
+#endif
+
 #ifdef CONFIG_QPL
 static void test_multifd_tcp_qpl(void)
 {
@@ -3992,6 +4019,10 @@ int main(int argc, char **argv)
     migration_test_add("/migration/multifd/tcp/plain/zstd",
                        test_multifd_tcp_zstd);
 #endif
+#ifdef CONFIG_QATZIP
+    migration_test_add("/migration/multifd/tcp/plain/qatzip",
+                test_multifd_tcp_qatzip);
+#endif
 #ifdef CONFIG_QPL
     migration_test_add("/migration/multifd/tcp/plain/qpl",
                        test_multifd_tcp_qpl);
Signature · 1 line
Yichen Wang
#12 lore raw

RE: [PATCH v6 5/5] tests/migration: Add integration test for 'qatzip' compression method

Quoted 34 lines

-----Original Message-----
From: Yichen Wang <[email protected]>
Sent: Tuesday, July 16, 2024 6:13 AM
To: Peter Xu <[email protected]>; Fabiano Rosas <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Marc-André Lureau <[email protected]>; Thomas Huth <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Laurent Vivier <[email protected]>; qemu- [email protected]
Cc: Hao Xiang <[email protected]>; Liu, Yuan1 <[email protected]>; Zou, Nanhai <[email protected]>; Ho-Ren (Jack) Chuang <[email protected]>; Wang, Yichen <[email protected]>; Bryan Zhang <[email protected]>
Subject: [PATCH v6 5/5] tests/migration: Add integration test for 'qatzip' compression method

From: Bryan Zhang <[email protected]>

Adds an integration test for 'qatzip'.

---
 tests/qtest/migration-test.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
tests/qtest/migration-test.c +0 -0
index 70b606b888..98f60d58a7 100644

tests/qtest/migration-test.c line 37

Quoted 6 lines
tests/qtest/migration-test.c +3 -0
@@ -32,6 +32,10 @@
 # endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
+#ifdef CONFIG_QATZIP
+#include <qatzip.h>
+#endif /* CONFIG_QATZIP */

It looks like <qatzip.h> file is not needed, maybe it can be removed.

Quoted 55 lines
tests/qtest/migration-test.c +12 -0
 /* For dirty ring test; so far only x86_64 is supported */
 #if defined(__linux__) && defined(HOST_X86_64)
 #include "linux/kvm.h"
@@ -2992,6 +2996,18 @@
 test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from,
 }
 #endif /* CONFIG_ZSTD */
 
+#ifdef CONFIG_QATZIP
+static void *
+test_migrate_precopy_tcp_multifd_qatzip_start(QTestState *from,
+                                              QTestState *to)
+{
+    migrate_set_parameter_int(from, "multifd-qatzip-level", 2);
+    migrate_set_parameter_int(to, "multifd-qatzip-level", 2);
+
+    return test_migrate_precopy_tcp_multifd_start_common(from, to,
 "qatzip");
+}
+#endif
+
 #ifdef CONFIG_QPL

static void * test_migrate_precopy_tcp_multifd_qpl_start(QTestState *from,

tests/qtest/migration-test.c +15 -1
@@ -3089,6 +3105,17 @@ static void test_multifd_tcp_zstd(void)
 }
 #endif
 
+#ifdef CONFIG_QATZIP
+static void test_multifd_tcp_qatzip(void)
+{
+    MigrateCommon args = {
+        .listen_uri = "defer",
+        .start_hook = test_migrate_precopy_tcp_multifd_qatzip_start,
+    };
+    test_precopy_common(&args);
+}
+#endif
+
 #ifdef CONFIG_QPL
 static void test_multifd_tcp_qpl(void)
 {
@@ -3992,6 +4019,10 @@ int main(int argc, char **argv)
     migration_test_add("/migration/multifd/tcp/plain/zstd",
                        test_multifd_tcp_zstd);
 #endif
+#ifdef CONFIG_QATZIP
+    migration_test_add("/migration/multifd/tcp/plain/qatzip",
+                test_multifd_tcp_qatzip);
+#endif
 #ifdef CONFIG_QPL
     migration_test_add("/migration/multifd/tcp/plain/qpl",
                        test_multifd_tcp_qpl);
--

Yichen Wang