Skip to content

implement zstd cluster compression method

Denis Plotnikov qemu-devel 24 versions – Applied

Status from this message: Thanks, applied to my block branch:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v24 Latest4 patches 1 replyApplied
  2. v234 patches 4 repliesApplied
  3. v224 patches 16 repliesSuperseded
Earlier versions · 21
VersionPostedPatchesRepliesStatus
v214 patches0 repliesSuperseded
v204 patches14 repliesSuperseded
v194 patches5 repliesSuperseded
v184 patches4 repliesSuperseded
v174 patches0 repliesSuperseded
v164 patches3 repliesSuperseded
v154 patches2 repliesSuperseded
v144 patches8 repliesSuperseded
v134 patches2 repliesSuperseded
v124 patches0 repliesSuperseded
v114 patches7 repliesSuperseded
v104 patches0 repliesSuperseded
v94 patches6 repliesSuperseded
v84 patches9 repliesSuperseded
v74 patches0 repliesSuperseded
v64 patches4 repliesSuperseded
v55 patches8 repliesSuperseded
v45 patches8 repliesSuperseded
v34 patches0 repliesSuperseded
v24 patches5 repliesSuperseded
v18 patches26 repliesSuperseded

Changes in v24 from the cover letter

01: add "compression_type" to the tests output [Max]
hopefully, I've found them all.

Patches in v24 Applied

#Subject and filesSizeRepliesApplied
1/4qcow2: introduce compression type featureblock/qcow2.c block/qcow2.h include/block/block_int.h +20 more+365 -1500 replies
2/4qcow2: rework the cluster compression routineblock/qcow2-threads.c+60 -110 replies
3/4qcow2: add zstd cluster compressionblock/qcow2-threads.c block/qcow2.c configure +2 more+180 -20 replies
4/4iotests: 287: add qcow2 compression type testtests/qemu-iotests/287 tests/qemu-iotests/287.out tests/qemu-iotests/group+220 -00 replies

Reviews trailers per patch

All 4 patches have a Reviewed-by or Acked-by.

PatchVladimir Sementsov-OgievskiyEric BlakeAlberto GarciaMax ReitzMarkus Armbruster
1/4R-b R-b R-b A-b
2/4R-b R-b R-b
3/4A-b
4/4R-b T-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.

Read the v24 thread

Cover letter [PATCH v24 0/4] implement zstd cluster compression method

v24:
01: add "compression_type" to the tests output [Max]
hopefully, I've found them all.

v23:
Undecided: whether to add zstd(zlib) compression
details to the qcow2 spec
03: tighten assertion on zstd decompression [Eric] 04: use _rm_test_img appropriately [Max]

v22:
03: remove assignemnt in if condition

v21:
03:
* remove the loop on compression [Max]
* use designated initializers [Max]
04:
* don't erase user's options [Max]
* use _rm_test_img [Max]
* add unsupported qcow2 options [Max]

v20:
04: fix a number of flaws [Vladimir]
* don't use $RAND_FILE passing to qemu-io,
so check $TEST_DIR is redundant
* re-arrage $RAND_FILE writing
* fix a typo

v19:
04: fix a number of flaws [Eric]
* remove rudundant test case descriptions
* fix stdout redirect
* don't use (())
* use peek_file_be instead of od
* check $TEST_DIR for spaces and other before using
* use $RAND_FILE safer

v18:
* 04: add quotes to all file name variables [Vladimir]
* 04: add Vladimir's comment according to "qemu-io write -s" option issue.

v17:
* 03: remove incorrect comment in zstd decompress [Vladimir]
* 03: remove "paraniod" and rewrite the comment on decompress [Vladimir]
* 03: fix dead assignment [Vladimir]
* 04: add and remove quotes [Vladimir]
* 04: replace long offset form with the short one [Vladimir]

v16:
* 03: ssize_t for ret, size_t for zstd_ret [Vladimir]
* 04: small fixes according to the comments [Vladimir]

v15:
* 01: aiming qemu 5.1 [Eric]
* 03: change zstd_res definition place [Vladimir]
* 04: add two new test cases [Eric]
1. test adjacent cluster compression with zstd
2. test incompressible cluster processing
* 03, 04: many rewording and gramma fixing [Eric]

v14:
* fix bug on compression - looping until compress == 0 [Me]
* apply reworked Vladimir's suggestions:
1. not mixing ssize_t with size_t
2. safe check for ENOMEM in compression part - avoid overflow
3. tolerate sanity check allow zstd to make progress only on one of the buffers
v13:
* 03: add progress sanity check to decompression loop [Vladimir] 03: add successful decompression check [Me]

v12:
* 03: again, rework compression and decompression loops
to make them more correct [Vladimir]
03: move assert in compression to more appropriate place
[Vladimir]
v11:
* 03: the loops don't need "do{}while" form anymore and
the they were buggy (missed "do" in the beginning)
replace them with usual "while(){}" loops [Vladimir]
v10:
* 03: fix zstd (de)compressed loops for multi-frame
cases [Vladimir]
v9:
* 01: fix error checking and reporting in qcow2_amend compression type part [Vladimir]
* 03: replace asserts with -EIO in qcow2_zstd_decompression [Vladimir, Alberto]
* 03: reword/amend/add comments, fix typos [Vladimir]

v8:
* 03: switch zstd API from simple to stream [Eric] No need to state a special cluster layout for zstd compressed clusters.
v7:
* use qapi_enum_parse instead of the open-coding [Eric]
* fix wording, typos and spelling [Eric]

v6:
* "block/qcow2-threads: fix qcow2_decompress" is removed from the series since it has been accepted by Max already
* add compile time checking for Qcow2Header to be a multiple of 8 [Max, Alberto]
* report error on qcow2 amending when the compression type is actually chnged [Max]
* remove the extra space and the extra new line [Max]
* re-arrange acks and signed-off-s [Vladimir]

v5:
* replace -ENOTSUP with abort in qcow2_co_decompress [Vladimir]
* set cluster size for all test cases in the beginning of the 287 test

v4:
* the series is rebased on top of 01 "block/qcow2-threads: fix qcow2_decompress"
* 01 is just a no-change resend to avoid extra dependencies. Still, it may be merged in separate

v3:
* remove redundant max compression type value check [Vladimir, Eric] (the switch below checks everything)
* prevent compression type changing on "qemu-img amend" [Vladimir]
* remove zstd config setting, since it has been added already by "migration" patches [Vladimir]
* change the compression type error message [Vladimir]
* fix alignment and 80-chars exceeding [Vladimir]

v2:
* rework compression type setting [Vladimir]
* squash iotest changes to the compression type introduction patch [Vladimir, Eric]
* fix zstd availability checking in zstd iotest [Vladimir]
* remove unnecessry casting [Eric]
* remove rudundant checks [Eric]
* fix compressed cluster layout in qcow2 spec [Vladimir]
* fix wording [Eric, Vladimir]
* fix compression type filtering in iotests [Eric]

v1:
the initial series

Denis Plotnikov (4):
qcow2: introduce compression type feature
qcow2: rework the cluster compression routine
qcow2: add zstd cluster compression
iotests: 287: add qcow2 compression type test

 docs/interop/qcow2.txt           |   1 +
 configure                        |   2 +-
 qapi/block-core.json             |  23 ++-
 block/qcow2.h                    |  20 ++-
 include/block/block_int.h        |   1 +
 block/qcow2-threads.c            | 240 +++++++++++++++++++++++++++++--
 block/qcow2.c                    | 120 ++++++++++++++++
 tests/qemu-iotests/031.out       |  14 +-
 tests/qemu-iotests/036.out       |   4 +-
 tests/qemu-iotests/049.out       | 102 ++++++-------
 tests/qemu-iotests/060.out       |   1 +
 tests/qemu-iotests/061.out       |  34 +++--
 tests/qemu-iotests/065           |  28 ++--
 tests/qemu-iotests/080           |   2 +-
 tests/qemu-iotests/082.out       |  48 ++++++-
 tests/qemu-iotests/085.out       |  38 ++---
 tests/qemu-iotests/144.out       |   4 +-
 tests/qemu-iotests/182.out       |   2 +-
 tests/qemu-iotests/185.out       |   8 +-
 tests/qemu-iotests/198.out       |   2 +
 tests/qemu-iotests/206.out       |   5 +
 tests/qemu-iotests/242.out       |   5 +
 tests/qemu-iotests/255.out       |   8 +-
 tests/qemu-iotests/274.out       |  49 ++++---
 tests/qemu-iotests/280.out       |   2 +-
 tests/qemu-iotests/287           | 152 ++++++++++++++++++++
 tests/qemu-iotests/287.out       |  67 +++++++++
 tests/qemu-iotests/common.filter |   3 +-
 tests/qemu-iotests/group         |   1 +
 29 files changed, 824 insertions(+), 162 deletions(-)
 create mode 100755 tests/qemu-iotests/287
 create mode 100644 tests/qemu-iotests/287.out
Signature · 1 line
2.17.0