Skip to content

Allow configuring the qcow2 L2 cache entry size

Alberto Garcia qemu-devel 4 versions – Applied

Status from this message: Thanks, applied to my block branch

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest39 patches 13 repliesApplied
  2. v339 patches 68 repliesSuperseded
  3. v232 patches 49 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v131 patches3 repliesSuperseded

Changes in v4 from the cover letter

- Rebased on top of f24ee107a07f093bd7ed475dd48d7ba57ea3d8fe
- Patch 17: Fix typo in comment
- Patch 24: Rename discard_single_l2() to discard_in_l2_slice() [Max]
- Patch 25: Rename zero_single_l2() to zero_in_l2_slice() [Max]
- Patch 27: Add comment clarifying the l2_index variable [Max]
- Patch 30: Remove unneeded void * casts [Eric, Max]
- Patch 31: Use offset_to_l2_slice_index() instead of doing it manually [Max]

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/39qcow2: Fix documentation of get_cluster_table()block/qcow2-cluster.c+1 -20 replies
2/39qcow2: Add table size field to Qcow2Cacheblock/qcow2-cache.c+14 -150 replies
3/39qcow2: Remove BDS parameter from qcow2_cache_get_table_addr()block/qcow2-cache.c+6 -70 replies
4/39qcow2: Remove BDS parameter from qcow2_cache_get_table_idx()block/qcow2-cache.c+4 -50 replies
5/39qcow2: Remove BDS parameter from qcow2_cache_table_release()block/qcow2-cache.c+4 -50 replies
6/39qcow2: Remove BDS parameter from qcow2_cache_entry_mark_dirty()block/qcow2-cache.c block/qcow2-cluster.c block/qcow2-refcount.c +1 more+14 -180 replies
7/39qcow2: Remove BDS parameter from qcow2_cache_put()block/qcow2-cache.c block/qcow2-cluster.c block/qcow2-refcount.c +1 more+31 -310 replies
8/39qcow2: Remove BDS parameter from qcow2_cache_destroy()block/qcow2-cache.c block/qcow2.c block/qcow2.h+10 -100 replies
9/39qcow2: Remove BDS parameter from qcow2_cache_clean_unused()block/qcow2-cache.c block/qcow2.c block/qcow2.h+4 -40 replies
10/39qcow2: Remove BDS parameter from qcow2_cache_discard()block/qcow2-cache.c block/qcow2-refcount.c block/qcow2.h+5 -50 replies
11/39qcow2: Remove BDS parameter from qcow2_cache_is_table_offset()block/qcow2-cache.c block/qcow2-refcount.c block/qcow2.h+5 -70 replies
12/39qcow2: Add offset_to_l1_index()block/qcow2-cluster.c block/qcow2.h+7 -20 replies
13/39qcow2: Add l2_slice_size field to BDRVQcow2Stateblock/qcow2.c block/qcow2.h+4 -00 replies
14/39qcow2: Add offset_to_l2_slice_index()block/qcow2.h+5 -00 replies
15/39qcow2: Update l2_load() to support L2 slicesblock/qcow2-cluster.c+17 -110 replies
16/39qcow2: Prepare l2_allocate() for adding L2 slice supportblock/qcow2-cluster.c+30 -250 replies
17/39qcow2: Update l2_allocate() to support L2 slicesblock/qcow2-cluster.c+34 -221 reply
18/39qcow2: Refactor get_cluster_table()block/qcow2-cluster.c+7 -140 replies
19/39qcow2: Update get_cluster_table() to support L2 slicesblock/qcow2-cluster.c+8 -80 replies
20/39qcow2: Update qcow2_get_cluster_offset() to support L2 slicesblock/qcow2-cluster.c+15 -150 replies
21/39qcow2: Update qcow2_alloc_cluster_link_l2() to support L2 slicesblock/qcow2-cluster.c+8 -80 replies
22/39qcow2: Update handle_copied() to support L2 slicesblock/qcow2-cluster.c+8 -80 replies
23/39qcow2: Update handle_alloc() to support L2 slicesblock/qcow2-cluster.c+9 -90 replies
24/39qcow2: Update discard_single_l2() to support L2 slicesblock/qcow2-cluster.c+16 -161 reply
25/39qcow2: Update zero_single_l2() to support L2 slicesblock/qcow2-cluster.c+14 -141 reply
26/39qcow2: Prepare qcow2_update_snapshot_refcount() for adding L2 slice supportblock/qcow2-refcount.c+73 -670 replies
27/39qcow2: Update qcow2_update_snapshot_refcount() to support L2 slicesblock/qcow2-refcount.c+18 -142 replies
28/39qcow2: Read refcount before L2 table in expand_zero_clusters_in_l1()block/qcow2-cluster.c+6 -60 replies
29/39qcow2: Prepare expand_zero_clusters_in_l1() for adding L2 slice supportblock/qcow2-cluster.c+96 -910 replies
30/39qcow2: Update expand_zero_clusters_in_l1() to support L2 slicesblock/qcow2-cluster.c+28 -230 replies
31/39qcow2: Update qcow2_truncate() to support L2 slicesblock/qcow2.c+3 -32 replies
32/39qcow2: Rename l2_table in qcow2_alloc_compressed_cluster_offset()block/qcow2-cluster.c+8 -80 replies
33/39qcow2: Rename l2_table in count_contiguous_clusters()block/qcow2-cluster.c+4 -40 replies
34/39qcow2: Rename l2_table in count_contiguous_clusters_unallocated()block/qcow2-cluster.c+3 -31 reply
35/39qcow2: Rename l2_table in count_cow_clusters()block/qcow2-cluster.c+2 -20 replies
36/39qcow2: Allow configuring the L2 slice sizeblock/qcow2-cache.c block/qcow2.c block/qcow2.h +1 more+45 -110 replies
37/39iotests: Test valid values of l2-cache-entry-sizetests/qemu-iotests/103 tests/qemu-iotests/103.out+20 -02 replies
38/39iotests: Test downgrading an image using a small L2 slice sizetests/qemu-iotests/061 tests/qemu-iotests/061.out+77 -00 replies
39/39iotests: Add l2-cache-entry-size to iotest 137tests/qemu-iotests/137 tests/qemu-iotests/137.out+7 -00 replies

Reviews trailers per patch

All 39 patches have a Reviewed-by or Acked-by.

PatchEric BlakeMax Reitz
1/39R-b R-b
2/39R-b R-b
3/39R-b R-b
4/39R-b R-b
5/39R-b R-b
6/39R-b R-b
7/39R-b R-b
8/39R-b R-b
9/39R-b R-b
10/39R-b R-b
11/39R-b R-b
12/39R-b R-b
13/39R-b R-b
14/39R-b R-b
15/39R-b R-b
16/39R-b R-b
17/39R-b R-b
18/39R-b R-b
19/39R-b R-b
20/39R-b R-b
21/39R-b R-b
22/39R-b R-b
23/39R-b R-b
24/39R-b
25/39R-b
26/39R-b R-b
27/39R-b R-b
28/39R-b R-b
29/39R-b R-b
30/39R-b R-b
31/39R-b R-b
32/39R-b R-b
33/39R-b R-b
34/39R-b R-b
35/39R-b R-b
36/39R-b R-b
37/39R-b R-b
38/39R-b R-b
39/39R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [Qemu-devel] [PATCH v4 00/39] Allow configuring the qcow2 L2 cache entry size

this is the new revision of the patch series to allow configuring the entry size of the qcow2 L2 cache. Follow this link for the full description from the first version:

https://lists.gnu.org/archive/html/qemu-block/2017-10/msg00458.html

And here are some numbers showing the performance improvements:

https://lists.gnu.org/archive/html/qemu-block/2017-12/msg00507.html

Regards,

Berto

Changes:

v4:
- Rebased on top of f24ee107a07f093bd7ed475dd48d7ba57ea3d8fe
- Patch 17: Fix typo in comment
- Patch 24: Rename discard_single_l2() to discard_in_l2_slice() [Max]
- Patch 25: Rename zero_single_l2() to zero_in_l2_slice() [Max]
- Patch 27: Add comment clarifying the l2_index variable [Max]
- Patch 30: Remove unneeded void * casts [Eric, Max]
- Patch 31: Use offset_to_l2_slice_index() instead of doing it manually [Max]

v3: https://lists.gnu.org/archive/html/qemu-block/2018-01/msg00635.html
- Rebased on top of 1867d97b372d452184c65e8fcc273cfc45937541
- Patch 2: Use QEMU_IS_ALIGNED() instead of doing the bit operations manually [Eric]
- Patch 13: Clarify that l2_slice_size means size in entries, not bytes [Eric].
- Patch 16-17: The previous code has been splitted into patches 16 and 17 for clarity.
- Patch 17: Use slice_size2 for storing the size of the slice in bytes (keeping the existing convention used in l1_size2 or refcount_table_size2)
- Patch 17: Flush the cache only once [Anton, Eric]
- Patch 17: Call l2_load() after qcow2_free_clusters() [Anton]
- Patch 18 [new]: Refactor the l2_load() call in get_cluster_table()

           [Anton]

- Patch 19: Fix typo [Eric]
- Patch 26-27: The previous code has been splitted into patches 26 and 27 for clarity.
- Patch 27: Use slice_size2 for storing the size of the slice in bytes
- Patch 28-30: The previous code has been splitted into patches 28, 29 and 30 for clarity.
- Patch 30: Use slice_size2 for storing the size of the slice in bytes
- Patch 30: Replace bdrv_read/write() with bdrv_pread/pwrite() [Eric]
- Patch 30: Initialize l2_dirty inside the inner loop [Anton]
- Patch 36: Fix maybe-uninitialized warning of l2_cache_entry_size

            [Anton]

- Patch 36: Make MIN_L2_CACHE_SIZE the minimum number of cache entries, not clusters [Anton]
- Patch 36: Add l2-cache-entry-size to BlockdevOptionsQcow2 [Eric]
- Patch 37-39: New iotests.

v2: https://lists.gnu.org/archive/html/qemu-block/2017-12/msg00507.html
- Rebased after the v2.11.0 release.
- Patch 2: Adjust the unaligned access check introduce by Max in 4efb1f7c612
- Patch 18: Prevent overflow when computing bytes_available in qcow2_get_cluster_offset()
- Patch 31: Fix typo in error message in read_cache_sizes()
- Patch 32 [new]: Add test for l2-cache-entry-size'

v1: https://lists.gnu.org/archive/html/qemu-block/2017-10/msg00458.html
- Initial version

Here's the ouput of git backport-diff against v3:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/39:[----] [--] 'qcow2: Fix documentation of get_cluster_table()' 002/39:[----] [--] 'qcow2: Add table size field to Qcow2Cache'
003/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_get_table_addr()' 004/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_get_table_idx()' 005/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_table_release()' 006/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_entry_mark_dirty()' 007/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_put()' 008/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_destroy()' 009/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_clean_unused()' 010/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_discard()' 011/39:[----] [--] 'qcow2: Remove BDS parameter from qcow2_cache_is_table_offset()' 012/39:[----] [--] 'qcow2: Add offset_to_l1_index()'
013/39:[----] [--] 'qcow2: Add l2_slice_size field to BDRVQcow2State' 014/39:[----] [--] 'qcow2: Add offset_to_l2_slice_index()'
015/39:[----] [--] 'qcow2: Update l2_load() to support L2 slices'
016/39:[----] [--] 'qcow2: Prepare l2_allocate() for adding L2 slice support' 017/39:[0002] [FC] 'qcow2: Update l2_allocate() to support L2 slices' 018/39:[----] [--] 'qcow2: Refactor get_cluster_table()'
019/39:[----] [--] 'qcow2: Update get_cluster_table() to support L2 slices' 020/39:[----] [--] 'qcow2: Update qcow2_get_cluster_offset() to support L2 slices' 021/39:[----] [--] 'qcow2: Update qcow2_alloc_cluster_link_l2() to support L2 slices' 022/39:[----] [--] 'qcow2: Update handle_copied() to support L2 slices' 023/39:[----] [--] 'qcow2: Update handle_alloc() to support L2 slices' 024/39:[0012] [FC] 'qcow2: Update discard_single_l2() to support L2 slices' 025/39:[0008] [FC] 'qcow2: Update zero_single_l2() to support L2 slices' 026/39:[----] [--] 'qcow2: Prepare qcow2_update_snapshot_refcount() for adding L2 slice support' 027/39:[0001] [FC] 'qcow2: Update qcow2_update_snapshot_refcount() to support L2 slices' 028/39:[----] [--] 'qcow2: Read refcount before L2 table in expand_zero_clusters_in_l1()' 029/39:[----] [--] 'qcow2: Prepare expand_zero_clusters_in_l1() for adding L2 slice support' 030/39:[0005] [FC] 'qcow2: Update expand_zero_clusters_in_l1() to support L2 slices' 031/39:[0003] [FC] 'qcow2: Update qcow2_truncate() to support L2 slices' 032/39:[----] [--] 'qcow2: Rename l2_table in qcow2_alloc_compressed_cluster_offset()' 033/39:[----] [--] 'qcow2: Rename l2_table in count_contiguous_clusters()' 034/39:[----] [--] 'qcow2: Rename l2_table in count_contiguous_clusters_unallocated()' 035/39:[----] [--] 'qcow2: Rename l2_table in count_cow_clusters()' 036/39:[----] [--] 'qcow2: Allow configuring the L2 slice size'
037/39:[----] [--] 'iotests: Test valid values of l2-cache-entry-size' 038/39:[----] [--] 'iotests: Test downgrading an image using a small L2 slice size' 039/39:[----] [--] 'iotests: Add l2-cache-entry-size to iotest 137'

Alberto Garcia (39):
qcow2: Fix documentation of get_cluster_table()
qcow2: Add table size field to Qcow2Cache
qcow2: Remove BDS parameter from qcow2_cache_get_table_addr()
qcow2: Remove BDS parameter from qcow2_cache_get_table_idx()
qcow2: Remove BDS parameter from qcow2_cache_table_release()
qcow2: Remove BDS parameter from qcow2_cache_entry_mark_dirty()
qcow2: Remove BDS parameter from qcow2_cache_put()
qcow2: Remove BDS parameter from qcow2_cache_destroy()
qcow2: Remove BDS parameter from qcow2_cache_clean_unused()
qcow2: Remove BDS parameter from qcow2_cache_discard()
qcow2: Remove BDS parameter from qcow2_cache_is_table_offset()
qcow2: Add offset_to_l1_index()
qcow2: Add l2_slice_size field to BDRVQcow2State
qcow2: Add offset_to_l2_slice_index()
qcow2: Update l2_load() to support L2 slices
qcow2: Prepare l2_allocate() for adding L2 slice support
qcow2: Update l2_allocate() to support L2 slices
qcow2: Refactor get_cluster_table()
qcow2: Update get_cluster_table() to support L2 slices
qcow2: Update qcow2_get_cluster_offset() to support L2 slices
qcow2: Update qcow2_alloc_cluster_link_l2() to support L2 slices
qcow2: Update handle_copied() to support L2 slices
qcow2: Update handle_alloc() to support L2 slices
qcow2: Update discard_single_l2() to support L2 slices
qcow2: Update zero_single_l2() to support L2 slices
qcow2: Prepare qcow2_update_snapshot_refcount() for adding L2 slice
support
qcow2: Update qcow2_update_snapshot_refcount() to support L2 slices
qcow2: Read refcount before L2 table in expand_zero_clusters_in_l1()
qcow2: Prepare expand_zero_clusters_in_l1() for adding L2 slice
support
qcow2: Update expand_zero_clusters_in_l1() to support L2 slices
qcow2: Update qcow2_truncate() to support L2 slices
qcow2: Rename l2_table in qcow2_alloc_compressed_cluster_offset()
qcow2: Rename l2_table in count_contiguous_clusters()
qcow2: Rename l2_table in count_contiguous_clusters_unallocated()
qcow2: Rename l2_table in count_cow_clusters()
qcow2: Allow configuring the L2 slice size
iotests: Test valid values of l2-cache-entry-size
iotests: Test downgrading an image using a small L2 slice size
iotests: Add l2-cache-entry-size to iotest 137

 block/qcow2-cache.c        |  80 +++----
 block/qcow2-cluster.c      | 511 ++++++++++++++++++++++++---------------------
 block/qcow2-refcount.c     | 200 +++++++++---------
 block/qcow2.c              |  61 ++++--
 block/qcow2.h              |  31 ++-
 qapi/block-core.json       |   6 +
 tests/qemu-iotests/061     |  16 ++
 tests/qemu-iotests/061.out |  61 ++++++
 tests/qemu-iotests/103     |  17 ++
 tests/qemu-iotests/103.out |   3 +
 tests/qemu-iotests/137     |   5 +
 tests/qemu-iotests/137.out |   2 +
 12 files changed, 585 insertions(+), 408 deletions(-)
Signature · 1 line
2.11.0