spec: add qcow2 bitmaps extension specification
PATCH v7 1/1 spec: add qcow2 bitmaps extension specification
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.
- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword bitmap_directory_size description
- bitmap type: make 0 reserved
- extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeor extra_data_size should be considered as an error.
- swap name and extra_data to give good alignment to extra_data.
v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- move name_size field closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate section
docs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)
docs/specs/qcow2.txt +171 -1
@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clear features if it clears the respective bits from this field first. - Bits 0-63: Reserved (set to 0) + Bit 0: Bitmaps extension bit + This bit indicates consistency for the bitmaps + extension data. + + It is an error if this bit is set without the + bitmaps extension present. + + If the bitmaps extension is present but this + bit is unset, the bitmaps extension data is + inconsistent. + + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width
@@ -123,6 +134,7 @@ be stored. Each extension has a structure like the following: 0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table + 0x23852875 - Bitmaps extension other - Unknown header extension, can be safely ignored
@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension == + +The bitmaps extension is an optional header extension. It provides the ability +to store bitmaps related to a virtual disk. For now, there is only one bitmap +type: the dirty tracking bitmap, which tracks virtual disk changes from some +point in time. + +The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_features above. + +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmap directory + starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining a reference count
@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entry size to the next multiple of 8. + + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored. + +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap. + + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of the virtual + disk by any application that would write to this qcow2 + file (including writes, snapshot switching, etc.). The + type of this bitmap must be 'dirty tracking bitmap'. + + Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support granularity_bits + greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of the image + accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support values greater than + 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap. + + variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table. + +Each bitmap table has a variable size (stored in the bitmap directory Entry) +and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero. + If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see +above). + +Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the last +cluster of the bitmap data contains some unused tail bits. These bits must be +zero.
Signature · 1 line
2.1.4
Quoted 243 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +161
-1@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clear features if it clears the respective bits from this field first. - Bits 0-63: Reserved (set to 0) + Bit 0: Bitmaps extension bit + This bit indicates consistency for the bitmaps + extension data. + + It is an error if this bit is set without the + bitmaps extension present. + + If the bitmaps extension is present but this + bit is unset, the bitmaps extension data is + inconsistent. + + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width@@ -123,6 +134,7 @@ be stored. Each extension has a structure like the following: 0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table + 0x23852875 - Bitmaps extension other - Unknown header extension, can be safely ignored@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension == + +The bitmaps extension is an optional header extension. It provides the ability +to store bitmaps related to a virtual disk. For now, there is only one bitmap +type: the dirty tracking bitmap, which tracks virtual disk changes from some +point in time. + +The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_features above. + +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmap directory + starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining a reference count@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entry size to the next multiple of 8. + + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored. + +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap. + + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of the virtual + disk by any application that would write to this qcow2 + file (including writes, snapshot switching, etc.). The + type of this bitmap must be 'dirty tracking bitmap'. + + Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support granularity_bits + greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of the image + accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support values greater than + 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap. + + variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table. + +Each bitmap table has a variable size (stored in the bitmap directory Entry) +and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero. + If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see +above). +
docs/specs/qcow2.txt Snapshot table entry: · line 532
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the last +cluster of the bitmap data contains some unused tail bits. These bits must be +zero.
s/multiply/multiple/.
Looks good otherwise, pending the results of our discussion about what to do about the extra_data_size / type-specific data fields.
Thanks!
Signature · 1 line
—js
Quoted 257 lines · John Snow
Quoted 243 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +161
-1@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clear features if it clears the respective bits from this field first. - Bits 0-63: Reserved (set to 0) + Bit 0: Bitmaps extension bit + This bit indicates consistency for the bitmaps + extension data. + + It is an error if this bit is set without the + bitmaps extension present. + + If the bitmaps extension is present but this + bit is unset, the bitmaps extension data is + inconsistent. + + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width@@ -123,6 +134,7 @@ be stored. Each extension has a structure like the following: 0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table + 0x23852875 - Bitmaps extension other - Unknown header extension, can be safely ignored@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension == + +The bitmaps extension is an optional header extension. It provides the ability +to store bitmaps related to a virtual disk. For now, there is only one bitmap +type: the dirty tracking bitmap, which tracks virtual disk changes from some +point in time. + +The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_features above. + +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmap directory + starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining a reference count@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entry size to the next multiple of 8. + + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored. + +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap. + + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of the virtual + disk by any application that would write to this qcow2 + file (including writes, snapshot switching, etc.). The + type of this bitmap must be 'dirty tracking bitmap'. + + Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support granularity_bits + greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of the image + accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support values greater than + 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap. + + variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table. + +Each bitmap table has a variable size (stored in the bitmap directory Entry) +and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero. + If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see +above). +docs/specs/qcow2.txt Snapshot table entry: · line 532
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the last +cluster of the bitmap data contains some unused tail bits. These bits must be +zero.s/multiply/multiple/.
Quoted 5 lines
Looks good otherwise, pending the results of our discussion about what to do about the
extra_data_size/ type-specific data fields.Thanks!
should we re-submit? Is this version good enough to be accepted? :)
Den
Quoted 306 lines · Denis V. Lunev
Quoted 305 lines · John Snow
Quoted 296 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-0@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clearfeatures if it
clears the respective bits from this field first.
- Bits 0-63: Reserved (set to 0)docs/specs/qcow2.txt +2
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency forthe bitmaps
docs/specs/qcow2.txt +3
-0+ extension data. + + It is an error if this bit is setwithout the
docs/specs/qcow2.txt +3
-0+ bitmaps extension present. + + If the bitmaps extension is presentbut this
+ bit is unset, the bitmaps extension
data isdocs/specs/qcow2.txt +3
-0+ inconsistent. + + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count blockentry (width
docs/specs/qcow2.txt +0
-0@@ -123,6 +134,7 @@ be stored. Each extension has a structure likethe following:
0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table+
0x23852875- Bitmaps extension
other - Unknown header extension, can
be safely
ignored
@@ -166,6 +178,34 @@ the header extension data. Each entry look
like this:
terminated if it has full length)
+== Bitmaps extension ==docs/specs/qcow2.txt +2
-0+ +The bitmaps extension is an optional header extension. It providesthe ability
+to store bitmaps related to a virtual disk. For now, there is only one bitmap
+type: the dirty tracking bitmap, which tracks virtual disk changes from somedocs/specs/qcow2.txt +4
-0+point in time. + +The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_featuresabove.
docs/specs/qcow2.txt +5
-0+ +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image.Must be
docs/specs/qcow2.txt +3
-0+ greater than or equal to 1. + + Note: Qemu currently only supports up to 65535bitmaps per
docs/specs/qcow2.txt +4
-0+ image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is thecumulative
docs/specs/qcow2.txt +4
-0+ size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmapdirectory
docs/specs/qcow2.txt +3
-0+ starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining areference count
docs/specs/qcow2.txt +0
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entrysize to the
next multiple of 8.docs/specs/qcow2.txt +5
-0+ + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability tostore bitmaps
+related a virtual disk. This section describes how these bitmaps are stored.docs/specs/qcow2.txt +6
-0+ +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directoryentry. The
+bitmap directory is a contiguous area in the image file, whose starting offset
+and length are given by the header extension fieldsbitmap_directory_offsetand
+bitmap_directory_size. The entries of the bitmap directory have variable
+length, depending on the length of the bitmap name and extra data. Thesedocs/specs/qcow2.txt +6
-0+entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmaptable
+ (described below) for the bitmap starts. Must be
aligned todocs/specs/qcow2.txt +4
-0+ a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of thebitmap.
docs/specs/qcow2.txt +9
-0+ + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of thevirtual
+ disk by any application that would write to
this qcow2
+ file (including writes, snapshot switching,
etc.). The
+ type of this bitmap must be 'dirty tracking
bitmap'.docs/specs/qcow2.txt +14
-0+ + Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support
granularity_bitsdocs/specs/qcow2.txt +6
-0+ greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of theimage
docs/specs/qcow2.txt +6
-0+ accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support valuesgreater than
docs/specs/qcow2.txt +6
-0+ 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined,
extra_data_sizeisdocs/specs/qcow2.txt +5
-0+ reserved and must be zero. + + variable: Type-specific extra data for the bitmap. + + variable: The name of the bitmap (not null terminated).Must be
+ unique among all bitmap names within the bitmaps
extension.docs/specs/qcow2.txt +2
-0+ + variable: Padding to round up the bitmap directory entrysize to the
docs/specs/qcow2.txt +6
-0+ next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for themapping of
+bitmap data to host clusters. This structure is called the bitmap table.docs/specs/qcow2.txt +2
-0+ +Each bitmap table has a variable size (stored in the bitmapdirectory Entry)
+and may use multiple clusters, however, it must be contiguous in the imagedocs/specs/qcow2.txt +5
-0+file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 arenon-zero.
docs/specs/qcow2.txt +7
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must bealigned to
+ a cluster boundary. If the offset is 0, the
cluster is
+ unallocated; in that case, bit 0 determines how
thisdocs/specs/qcow2.txt +7
-0+ cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters,described by the
+bitmap table. Given an offset (in bytes) into the bitmap data, the offset intodocs/specs/qcow2.txt +7
-0+the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry arezero (see
docs/specs/qcow2.txt +3
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap'sgranularity, the
docs/specs/qcow2.txt +7
-0+bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster sizeQuoted 5 lines
then the last
+cluster of the bitmap data contains some unused tail bits. These bits must be
+zero.s/multiply/multiple/.
Quoted 8 lines
Quoted 5 lines
Looks good otherwise, pending the results of our discussion about what to do about the
extra_data_size/ type-specific data fields.Thanks!
should we re-submit? Is this version good enough to be accepted? :)
Den
For my money, I'm happy -- though depending on how Vladimir wishes to use his type-specific data field, it may require some clarification. (I had a question over whether or not we needed versioning for this field if the plan was to add type-specific data to a type later, or if the extra data was always defined directly alongside a new type.)
The 2.6 window isn't close to over yet, though, so I would personally be OK with follow-up patches.
Max, Eric, any further input?
--js
Quoted 21 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
docs/specs/qcow2.txt +1
-0@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension ==
docs/specs/qcow2.txt the header extension data. Each entry look like this: · line 192
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers.
Only 4 bytes - if we ever raise our 64k entry restriction (nb_bitmaps), could we run into an image that has so many directory entries as to make the directory itself spill past 4G? But I don't think it is likely, so I can live with your choice.
Quoted 16 lines
docs/specs/qcow2.txt +16
-0+ +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored. + +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers.
docs/specs/qcow2.txt Snapshot table entry: · line 429
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap.
Should this be the size in bytes, instead of the number of entries? But at least the entries are fixed width of 8 bytes each, so this lets you get a bitmap table up to 32G bytes rather than just 4G in size. (Let's see here - if we have 32G bytes in the bitmap table, that means 4G clusters occupied by the bitmap itself; in the worst case of 512-byte clusters and granularity 0, that is a maximum bitmap size of 2T describing 16T of virtual guest image; with larger cluster size and/or larger granularity, we cover a lot more virtual guest space with less bitmap size; so I guess we aren't too worried about running out of space?).
docs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 7 lines
docs/specs/qcow2.txt +7
-0+ 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap.
I'd write this as:
variable: extra_data
Type-specific extra data for the bitmap, occupying extra_data_size bytes.
docs/specs/qcow2.txt Snapshot table entry: · line 482
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+ + variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8.
Should we require the padding to be all NUL bytes? (We aren't consistent on whether we require that for other locations of padding in the spec, so that could be a followup patch).
docs/specs/qcow2.txt Snapshot table entry: · line 488
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+ +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table.
Possible wording tweak:
Bitmaps are stored using a one-level structure (as opposed to the two-level structures for refcounts and guest cluster mapping), and are used for the mapping of bitmap data to host clusters
Review
Quoted 2 lines
(quoted diff) +2
-0+ +Each bitmap table has a variable size (stored in the bitmap directory Entry)
Does 'Entry' still need to be capitalized?
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero.
docs/specs/qcow2.txt Snapshot table entry: · line 506
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from.
Possible wording tweak:
s/when read from/during reads/.
Quoted 14 lines
docs/specs/qcow2.txt +14
-0+ + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see
docs/specs/qcow2.txt Snapshot table entry: · line 530
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the last
s/multiply of cluster size/multiple of the cluster size,/
Quoted 3 lines
docs/specs/qcow2.txt +2
-0+cluster of the bitmap data contains some unused tail bits. These bits must be +zero.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Quoted 22 lines · Eric Blake
Quoted 20 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
docs/specs/qcow2.txt +1
-0@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension ==
Quoted 16 lines
docs/specs/qcow2.txt the header extension data. Each entry look like this: · line 202
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers.Only 4 bytes - if we ever raise our 64k entry restriction (
nb_bitmaps), could we run into an image that has so many directory entries as to make the directory itself spill past 4G? But I don't think it is likely, so I can live with your choice.
"We'll never need this!"
I hope someone in 2082 is reading this right now and is quite angry.
(But really, I can't foresee needing this many per each drive -- and if we do, we have external storage mechanisms in development to handle such wild cases.)
Quoted 20 lines
Quoted 20 lines
docs/specs/qcow2.txt +20
-0+ +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored. + +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset
Quoted 8 lines
docs/specs/qcow2.txt Snapshot table entry: · line 429
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+ Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap.Should this be the size in bytes, instead of the number of entries? But
For what benefit? We can calculate either from the other, and this gives us a better resolution.
Quoted 9 lines
at least the entries are fixed width of 8 bytes each, so this lets you get a bitmap table up to 32G bytes rather than just 4G in size. (Let's see here - if we have 32G bytes in the bitmap table, that means 4G clusters occupied by the bitmap itself; in the worst case of 512-byte clusters and granularity 0, that is a maximum bitmap size of 2T describing 16T of virtual guest image; with larger cluster size and/or larger granularity, we cover a lot more virtual guest space with less bitmap size; so I guess we aren't too worried about running out of space?).
Yes, worst case of g=0 and cluster size of 512 bytes, we can get 2T bitmaps describing 16T of virtual data.
"default case" of 64K clusters and 64K granularity: 256TiB bitmaps describing ... let's see ... if my math is right, 128EiB?
We're probably fine :)
(Cue future space-person from 2159 wondering how I could have ever been so naive. Sorry, future space-person!)
Quoted 88 lines
docs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 7 lines
docs/specs/qcow2.txt +7
-0+ 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap.I'd write this as:
variable:extra_data
Type-specific extra data for the bitmap, occupyingextra_data_sizebytes.docs/specs/qcow2.txt Snapshot table entry: · line 482
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+ + variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8.Should we require the padding to be all NUL bytes? (We aren't consistent on whether we require that for other locations of padding in the spec, so that could be a followup patch).
docs/specs/qcow2.txt Snapshot table entry: · line 488
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+ +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table.Possible wording tweak:
Bitmaps are stored using a one-level structure (as opposed to the two-level structures for refcounts and guest cluster mapping), and are used for the mapping of bitmap data to host clustersReview
Quoted 2 lines
(quoted diff) +2
-0+ +Each bitmap table has a variable size (stored in the bitmap directory Entry)Does 'Entry' still need to be capitalized?
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero.docs/specs/qcow2.txt Snapshot table entry: · line 506
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from.Possible wording tweak:
s/when read from/during reads/.Quoted 14 lines
docs/specs/qcow2.txt +14
-0+ + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (seedocs/specs/qcow2.txt Snapshot table entry: · line 530
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the lasts/multiply of cluster size/multiple of the cluster size,/
Quoted 4 lines
docs/specs/qcow2.txt Snapshot table entry: · line 532
Quoted 3 lines
docs/specs/qcow2.txt +2
-0+cluster of the bitmap data contains some unused tail bits. These bits must be +zero.
Thanks!
--js
Quoted 37 lines · John Snow
Quoted 19 lines · Eric Blake
Quoted 18 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
docs/specs/qcow2.txt +0
-0@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length)Quoted 16 lines
docs/specs/qcow2.txt the header extension data. Each entry look like this: · line 202
Quoted 11 lines
docs/specs/qcow2.txt +11
-0+== Bitmaps extension == + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers.Only 4 bytes - if we ever raise our 64k entry restriction (
nb_bitmaps), could we run into an image that has so many directory entries as to make the directory itself spill past 4G? But I don't think it is likely, so I can live with your choice.
Quoted 7 lines
"We'll never need this!"
I hope someone in 2082 is reading this right now and is quite angry.
(But really, I can't foresee needing this many per each drive -- and if we do, we have external storage mechanisms in development to handle such wild cases.)
Let's change it to 8 bytes, and add 4bytes reserved field to keep 8b alignment.
Quoted 43 lines
Quoted 20 lines
Quoted 20 lines
docs/specs/qcow2.txt +19
-0+ +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored. + +Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: +Quoted 8 lines
docs/specs/qcow2.txt Snapshot table entry: · line 429
Quoted 7 lines
docs/specs/qcow2.txt +7
-0+ Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap.Should this be the size in bytes, instead of the number of entries? But
For what benefit? We can calculate either from the other, and this gives us a better resolution.
Quoted 9 lines
at least the entries are fixed width of 8 bytes each, so this lets you get a bitmap table up to 32G bytes rather than just 4G in size. (Let's see here - if we have 32G bytes in the bitmap table, that means 4G clusters occupied by the bitmap itself; in the worst case of 512-byte clusters and granularity 0, that is a maximum bitmap size of 2T describing 16T of virtual guest image; with larger cluster size and/or larger granularity, we cover a lot more virtual guest space with less bitmap size; so I guess we aren't too worried about running out of space?).
Yes, worst case of g=0 and cluster size of 512 bytes, we can get 2T bitmaps describing 16T of virtual data.
Quoted 7 lines
"default case" of 64K clusters and 64K granularity: 256TiB bitmaps describing ... let's see ... if my math is right, 128EiB?
We're probably fine :)
(Cue future space-person from 2159 wondering how I could have ever been so naive. Sorry, future space-person!)
it is just like l1_size from qcow2 header.
36 - 39: l1_size
Number of entries in the active L1 table
Quoted 12 lines
Quoted 12 lines
docs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 8 lines
docs/specs/qcow2.txt +7
-0+ 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap.I'd write this as:
variable:extra_data
Type-specific extra data for the bitmap, occupyingextra_data_sizebytes.
Ok.. then, similar should be done for the name.
About extra data and our discussion of versioning it. For now, we don't have concrete cases and requirements for this data, we are only foresee, that it may be needed in future, and waste time in assumptions about what kind of data will it be..
Will it be really type specific, i.e. bitmap type strictly define its structure? Or it will contain self-defined fields, some of them will be optional, some of them will depend on bitmap type or something like this? We have no concrete cases and strategy of using this data... We need not versioning for now, as we have no versions).
The only question, that should be answered, what should do the software, if it find extra_data of unknown format. (currently any extra data is unknown).
variants:
1) abort (or, more precisely: "image file must not be written") (like incompatible feature)
2) delete extra data (like autoclear feature)
3) ignore (like compatible feature)
We can use two bits of flags field to specify the behavior.
===========================================
Or, the other way, we should fix extra data format like this:
Extra data contains several fields of the following format:
4b: field magic
1b: flags: specifies abort/delete/ignore behavior if the field is unknown 1b: field name size - for error message if the field is unknown 2b: field size
var: field name
var: field data
var: padding
- where I've seen it...
=================
And the last option:
We can select abort behavior for now for any extra data, i.e. just remove "variable extra_data" and make field extra_data_size 'reserved must be zero'.
================
So, lets finally choose something and go on.
Note: for ways 1 and 3, we should in future define extra data in the way that gives a software clear method to understand, is this extra data known or unknown for it. I.e. it should have some magic number, defining format, or version, or structure like in way 2.
Quoted 10 lines
Quoted 10 lines
docs/specs/qcow2.txt Snapshot table entry: · line 482
Quoted 7 lines
docs/specs/qcow2.txt +6
-0+ + variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8.Should we require the padding to be all NUL bytes? (We aren't consistent on whether we require that for other locations of padding in the spec, so that could be a followup patch).
Let's require this, I do not see any shortcomings of this, only benefits.
Quoted 11 lines
Quoted 11 lines
docs/specs/qcow2.txt Snapshot table entry: · line 488
Quoted 7 lines
docs/specs/qcow2.txt +6
-0+ +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table.Possible wording tweak:
Bitmaps are stored using a one-level structure (as opposed to the two-level structures for refcounts and guest cluster mapping), and are used for the mapping of bitmap data to host clusters
No, this is wrong.. Who are used? Bitmaps are stored and are used? No, not bitmaps, structures are used.
Quoted 4 lines
Quoted 4 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ +Each bitmap table has a variable size (stored in the bitmap directory Entry)Does 'Entry' still need to be capitalized?
It's a mistake, thanks.
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
docs/specs/qcow2.txt +10
-0+and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero. + If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. +
Quoted 8 lines
Quoted 8 lines
docs/specs/qcow2.txt Snapshot table entry: · line 506
Quoted 6 lines
docs/specs/qcow2.txt +6
-0+ 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from.Possible wording tweak:
s/when read from/during reads/.
ok
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
docs/specs/qcow2.txt +17
-0+ + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see +above). + +Given an offset byte_nr into the virtual disk and the bitmap's granularity, the
Quoted 8 lines
Quoted 8 lines
docs/specs/qcow2.txt Snapshot table entry: · line 530
Quoted 7 lines
docs/specs/qcow2.txt +7
-0+bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the lasts/multiply of cluster size/multiple of the cluster size,/
ok, thanks
Quoted 7 lines
Quoted 4 lines
docs/specs/qcow2.txt Snapshot table entry: · line 532
Quoted 4 lines
docs/specs/qcow2.txt +2
-0+cluster of the bitmap data contains some unused tail bits. These bits must be +zero.Thanks!
--js
Please, let's decide finally about extra data, than I'll reroll it and, I hope, it will be committed, to make it possible to continue work on persistence series. About extra data, I'm ready to accept any variant, strictly defining, what software should do with unknown extra data.
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Quoted 305 lines · Vladimir Sementsov-Ogievskiy
Quoted 39 lines · John Snow
Quoted 28 lines · Eric Blake
Quoted 27 lines · Vladimir Sementsov-Ogievskiy
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
(quoted diff) +0
-0@@ -166,6 +178,34 @@ the header extension data. Each entry look likethis:
terminated if it has full length)
+== Bitmaps extension ==(quoted diff) +2
-0+ 0 - 3: nb_bitmaps + The number of bitmaps contained in the image.Must be
docs/specs/qcow2.txt +3
-0+ greater than or equal to 1. + + Note: Qemu currently only supports up to 65535bitmaps per
(quoted diff) +2
-0+ image. +Quoted 9 lines
Quoted 4 lines
(quoted diff) +2
-0+ 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is thecumulative
+ size of all (nb_bitmaps) bitmap headers.Only 4 bytes - if we ever raise our 64k entry restriction (
nb_bitmaps), could we run into an image that has so many directory entries as to make the directory itself spill past 4G? But I don't think it is likely, so I can live with your choice.Quoted 7 lines
"We'll never need this!"
I hope someone in 2082 is reading this right now and is quite angry.
(But really, I can't foresee needing this many per each drive -- and if we do, we have external storage mechanisms in development to handle such wild cases.)
Let's change it to 8 bytes, and add 4bytes reserved field to keep 8b alignment.
Quoted 55 lines
Quoted 31 lines
Quoted 31 lines
docs/specs/qcow2.txt +4
-0+ +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability tostore bitmaps
+related a virtual disk. This section describes how these bitmaps are stored.(quoted diff) +2
-0+ +Note: all bitmaps are related to the virtual disk stored in thisimage.
docs/specs/qcow2.txt +4
-0+ +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directoryentry. The
+bitmap directory is a contiguous area in the image file, whose starting offset
+and length are given by the header extension fieldsbitmap_directory_offsetand
+bitmap_directory_size. The entries of the bitmap directory have variable
+length, depending on the length of the bitmap name and extra data. Thesedocs/specs/qcow2.txt +6
-0+entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmaptable
Quoted 8 lines
Quoted 7 lines
+ (described below) for the bitmap starts. Must
be aligned todocs/specs/qcow2.txt +4
-0+ a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of thebitmap.
Should this be the size in bytes, instead of the number of entries? But
For what benefit? We can calculate either from the other, and this gives us a better resolution.
Quoted 10 lines
at least the entries are fixed width of 8 bytes each, so this lets you get a bitmap table up to 32G bytes rather than just 4G in size. (Let's see here - if we have 32G bytes in the bitmap table, that means 4G clusters occupied by the bitmap itself; in the worst case of 512-byte clusters and granularity 0, that is a maximum bitmap size of 2T describing 16T of virtual guest image; with larger cluster size and/or larger granularity, we cover a lot more virtual guest space with less bitmap size; so I guess we aren't too worried about running out of space?).
Yes, worst case of g=0 and cluster size of 512 bytes, we can get 2T bitmaps describing 16T of virtual data.
Quoted 7 lines
"default case" of 64K clusters and 64K granularity: 256TiB bitmaps describing ... let's see ... if my math is right, 128EiB?
We're probably fine :)
(Cue future space-person from 2159 wondering how I could have ever been so naive. Sorry, future space-person!)
it is just like
l1_sizefrom qcow2 header.36 - 39:
l1_size
Number of entries in the active L1 tableQuoted 13 lines
Quoted 13 lines
docs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 9 lines
docs/specs/qcow2.txt +4
-0+ 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined,
extra_data_sizeisdocs/specs/qcow2.txt +3
-0+ reserved and must be zero. + + variable: Type-specific extra data for the bitmap.I'd write this as:
variable:extra_data
Type-specific extra data for the bitmap, occupyingextra_data_sizebytes.Ok.. then, similar should be done for the name.
About extra data and our discussion of versioning it. For now, we don't have concrete cases and requirements for this data, we are only foresee, that it may be needed in future, and waste time in assumptions about what kind of data will it be..
Will it be really type specific, i.e. bitmap type strictly define its structure? Or it will contain self-defined fields, some of them will be optional, some of them will depend on bitmap type or something like this? We have no concrete cases and strategy of using this data... We need not versioning for now, as we have no versions).
The only question, that should be answered, what should do the software, if it find
extra_dataof unknown format. (currently any extra data is unknown).variants:
1) abort (or, more precisely: "image file must not be written") (like incompatible feature)
2) delete extra data (like autoclear feature)
3) ignore (like compatible feature)We can use two bits of flags field to specify the behavior.
===========================================
Or, the other way, we should fix extra data format like this:
Extra data contains several fields of the following format:
4b: field magic
1b: flags: specifies abort/delete/ignore behavior if the field is unknown 1b: field name size - for error message if the field is unknown 2b: field size
var: field name
var: field data
var: padding- where I've seen it...
=================
And the last option:
We can select abort behavior for now for any extra data, i.e. just remove "variable
extra_data" and make fieldextra_data_size'reserved must be zero'.================
So, lets finally choose something and go on.
Note: for ways 1 and 3, we should in future define extra data in the way that gives a software clear method to understand, is this extra data known or unknown for it. I.e. it should have some magic number, defining format, or version, or structure like in way 2.
Quoted 13 lines
Quoted 13 lines
Quoted 10 lines
(quoted diff) +2
-0+ + variable: The name of the bitmap (not null terminated).Must be
+ unique among all bitmap names within the
bitmaps extension.(quoted diff) +2
-0+ + variable: Padding to round up the bitmap directory entrysize to the
+ next multiple of 8.Should we require the padding to be all NUL bytes? (We aren't consistent on whether we require that for other locations of padding in the spec, so that could be a followup patch).
Let's require this, I do not see any shortcomings of this, only benefits.
Quoted 14 lines
Quoted 14 lines
Quoted 10 lines
docs/specs/qcow2.txt +4
-0+ +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed totwo-level
+structure like for refcounts and guest clusters mapping) for the mapping of
+bitmap data to host clusters. This structure is called the bitmap table.Possible wording tweak:
Bitmaps are stored using a one-level structure (as opposed to the two-level structures for refcounts and guest cluster mapping), and are used for the mapping of bitmap data to host clustersNo, this is wrong.. Who are used? Bitmaps are stored and are used? No, not bitmaps, structures are used.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
(quoted diff) +2
-0+ +Each bitmap table has a variable size (stored in the bitmapdirectory Entry)
Does 'Entry' still need to be capitalized?
It's a mistake, thanks.
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
+and may use multiple clusters, however, it must be contiguous in the image
docs/specs/qcow2.txt +5
-0+file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 arenon-zero.
docs/specs/qcow2.txt +7
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must beQuoted 8 lines
Quoted 8 lines
Quoted 6 lines
aligned to
+ a cluster boundary. If the offset is 0, the
cluster is
+ unallocated; in that case, bit 0 determines how
this
+ cluster should be treated when read from.Possible wording tweak:
s/when read from/during reads/.ok
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
docs/specs/qcow2.txt +6
-0+ + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters,described by the
+bitmap table. Given an offset (in bytes) into the bitmap data, the offset intodocs/specs/qcow2.txt +7
-0+the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry arezero (see
docs/specs/qcow2.txt +3
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap'sgranularity, the
+bit offset into the bitmap can be calculated like this:Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
docs/specs/qcow2.txt +6
-0+ + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster sizethen the last
s/multiply of cluster size/multiple of the cluster size,/
ok, thanks
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
+cluster of the bitmap data contains some unused tail bits. These bits must be
+zero.Thanks!
Quoted 8 lines
Quoted 1 line
--js
Please, let's decide finally about extra data, than I'll reroll it and, I hope, it will be committed, to make it possible to continue work on persistence series. About extra data, I'm ready to accept any variant, strictly defining, what software should do with unknown extra data.
I discussed this with Eric Blake on IRC briefly, and I mentioned I was concerned that we didn't specify a format at all for the extra data. Eric felt that it was not unusual to leave a space for future expansion and that as we haven't used it yet, we don't need to solidify it.
He also felt it would be unusual to stipulate the format of data that we don't even intend to use yet.
In short, I'm being too proactive.
A commit message mention that, should anyone wish to expand the type-specific data in the future that adding a 2-byte version as the first field in extra data would probably be sufficient, and we can worry about the spec wording later. It is fine to assume for now that if extra_data_size is 0 that the version/format of the data is "v0" and that does not limit our future expansion.
Sound good? Sorry for the red tape.
--John Snow
Quoted 15 lines · John Snow
Quoted 6 lines
Please, let's decide finally about extra data, than I'll reroll it and, I hope, it will be committed, to make it possible to continue work on persistence series. About extra data, I'm ready to accept any variant, strictly defining, what software should do with unknown extra data.
I discussed this with Eric Blake on IRC briefly, and I mentioned I was concerned that we didn't specify a format at all for the extra data. Eric felt that it was not unusual to leave a space for future expansion and that as we haven't used it yet, we don't need to solidify it.
He also felt it would be unusual to stipulate the format of data that we don't even intend to use yet.
Quoted 8 lines
In short, I'm being too proactive.
A commit message mention that, should anyone wish to expand the type-specific data in the future that adding a 2-byte version as the first field in extra data would probably be sufficient, and we can worry about the spec wording later. It is fine to assume for now that if
extra_data_sizeis 0 that the version/format of the data is "v0" and that does not limit our future expansion.
Or put another way:
I'm just fine if our initial implementation provides sufficient information for us to completely parse the file even when the file is generated by a newer qemu (we have a length, so we know how far to skip to find the next entry), while at the same time throwing up our hands if the length is non-zero (we won't read the bitmap at all, because we don't know if the non-zero extra_data contains instructions that would change how to interpret the data) or even prevent writes (if the bitmap entry is marked automatic, we must refuse any write that would requiring an update to the bitmap because we don't know how to write to a bitmap while correctly preserving semantics of those extra_data bytes). We have enough room for future extension, and that's good enough for now; the commit message can document that we thought about the future, without having to actually nail down in the spec what the future will actually do.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Quoted 2 lines · Eric Blake
preserving semantics of those
extra_databytes). We have enough room for future extension, and that's good e
Ok, so, what should go to the spec? Current wording is ok? Just delete "Type-specific":
+
+ 20 - 23: extra_data_size
+ Size of type-specific extra data.
+
+ For now, as no extra data is defined, extra_data_size is
+ reserved and must be zero.
+
+ variable: Extra data for the bitmap.
+
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Am 19.01.2016 um 09:57 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 7 lines · Eric Blake
Quoted 2 lines
preserving semantics of those
extra_databytes). We have enough room for future extension, and that's good eOk, so, what should go to the spec? Current wording is ok? Just delete "Type-specific":
Review
Quoted 8 lines
(quoted diff) +8
-0+ + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Extra data for the bitmap.
Please be explicit that if extra_data_size is non-zero, the bitmap must not be used (i.e. specify the incompatible-feature-bit-like behaviour).
Kevin
Quoted 7 lines · Kevin Wolf
Am 19.01.2016 um 09:57 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 6 lines · Eric Blake
Quoted 2 lines
preserving semantics of those
extra_databytes). We have enough room for future extension, and that's good eOk, so, what should go to the spec? Current wording is ok? Just delete "Type-specific":
Quoted 10 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Extra data for the bitmap.Please be explicit that if
extra_data_sizeis non-zero, the bitmap must not be used (i.e. specify the incompatible-feature-bit-like behaviour).
It is not enough. If there are some unknown extra data, then just ignoring this bitmap may lead to its inconsistency. So, if it is non-zero, the whole image should not be written. (real incompatible-feature-bit behavior).
Quoted 2 lines
Kevin
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Am 25.01.2016 um 11:15 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 19 lines · Kevin Wolf
Quoted 7 lines
Am 19.01.2016 um 09:57 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 6 lines · Eric Blake
Quoted 2 lines
preserving semantics of those
extra_databytes). We have enough room for future extension, and that's good eOk, so, what should go to the spec? Current wording is ok? Just delete "Type-specific":
Quoted 10 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Extra data for the bitmap.Please be explicit that if
extra_data_sizeis non-zero, the bitmap must not be used (i.e. specify the incompatible-feature-bit-like behaviour).
Quoted 4 lines
It is not enough. If there are some unknown extra data, then just ignoring this bitmap may lead to its inconsistency. So, if it is non-zero, the whole image should not be written. (real incompatible-feature-bit behavior).
Don't we generally ignore all bitmaps until a user actively tries to make use of it? Of course, with the 'auto' flag set, just ignoring the bitmap isn't possible, but I think in all other cases it should be.
If we ever add another type of bitmaps that doesn't have the 'auto' flag set, but is still automatically used, so that the image as a whole must become read-only without the bitmap, we can still add a normal incompatible feature flag. But I think it's more likely that we add extra_data that doesn't prevent use of the image, so we should have a way to express that.
Kevin
Quoted 27 lines · Kevin Wolf
Am 25.01.2016 um 11:15 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 18 lines · Kevin Wolf
Quoted 7 lines
Am 19.01.2016 um 09:57 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 6 lines · Eric Blake
Quoted 2 lines
preserving semantics of those
extra_databytes). We have enough room for future extension, and that's good eOk, so, what should go to the spec? Current wording is ok? Just delete "Type-specific":
Quoted 10 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Extra data for the bitmap.Please be explicit that if
extra_data_sizeis non-zero, the bitmap must not be used (i.e. specify the incompatible-feature-bit-like behaviour).Quoted 4 lines
It is not enough. If there are some unknown extra data, then just ignoring this bitmap may lead to its inconsistency. So, if it is non-zero, the whole image should not be written. (real incompatible-feature-bit behavior).
Don't we generally ignore all bitmaps until a user actively tries to make use of it? Of course, with the 'auto' flag set, just ignoring the bitmap isn't possible, but I think in all other cases it should be.
Quoted 8 lines
If we ever add another type of bitmaps that doesn't have the 'auto' flag set, but is still automatically used, so that the image as a whole must become read-only without the bitmap, we can still add a normal incompatible feature flag. But I think it's more likely that we add
extra_datathat doesn't prevent use of the image, so we should have a way to express that.Kevin
let's add one flag:
2: extra_data_compatible
This flag is meaningful when extra data is unknown for the software. If it is set, the bitmap may be used as usual, extra data must
be left as is.
If it is unset, the bitmap must not be used, but left as is with
extra data.
Ok? And, if there will be incompatible features we will use normal incompatible feature flag.
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Am 18.01.2016 um 22:16 hat Eric Blake geschrieben:
Quoted 28 lines · John Snow
Quoted 16 lines
Quoted 7 lines
Please, let's decide finally about extra data, than I'll reroll it and, I hope, it will be committed, to make it possible to continue work on persistence series. About extra data, I'm ready to accept any variant, strictly defining, what software should do with unknown extra data.
I discussed this with Eric Blake on IRC briefly, and I mentioned I was concerned that we didn't specify a format at all for the extra data. Eric felt that it was not unusual to leave a space for future expansion and that as we haven't used it yet, we don't need to solidify it.
He also felt it would be unusual to stipulate the format of data that we don't even intend to use yet.
Quoted 8 lines
In short, I'm being too proactive.
A commit message mention that, should anyone wish to expand the type-specific data in the future that adding a 2-byte version as the first field in extra data would probably be sufficient, and we can worry about the spec wording later. It is fine to assume for now that if
extra_data_sizeis 0 that the version/format of the data is "v0" and that does not limit our future expansion.Or put another way:
Quoted 10 lines
I'm just fine if our initial implementation provides sufficient information for us to completely parse the file even when the file is generated by a newer qemu (we have a length, so we know how far to skip to find the next entry), while at the same time throwing up our hands if the length is non-zero (we won't read the bitmap at all, because we don't know if the non-zero
extra_datacontains instructions that would change how to interpret the data) or even prevent writes (if the bitmap entry is marked automatic, we must refuse any write that would requiring an update to the bitmap because we don't know how to write to a bitmap while correctly preserving semantics of thoseextra_databytes).
Can we assume that the extra_data doesn't contain references to clusters? Otherwise we need to forbid 'qemu-img check -r leaks' when there is unknown extra_data.
FWIW, this assumption is already made for snapshots, so it seems okay to make it here as well. But we could be explicit about it.
Kevin
Quoted 37 lines · Kevin Wolf
Am 18.01.2016 um 22:16 hat Eric Blake geschrieben:
Quoted 26 lines · John Snow
Quoted 15 lines
Quoted 7 lines
Please, let's decide finally about extra data, than I'll reroll it and, I hope, it will be committed, to make it possible to continue work on persistence series. About extra data, I'm ready to accept any variant, strictly defining, what software should do with unknown extra data.
I discussed this with Eric Blake on IRC briefly, and I mentioned I was concerned that we didn't specify a format at all for the extra data. Eric felt that it was not unusual to leave a space for future expansion and that as we haven't used it yet, we don't need to solidify it.
He also felt it would be unusual to stipulate the format of data that we don't even intend to use yet.
Quoted 8 lines
In short, I'm being too proactive.
A commit message mention that, should anyone wish to expand the type-specific data in the future that adding a 2-byte version as the first field in extra data would probably be sufficient, and we can worry about the spec wording later. It is fine to assume for now that if
extra_data_sizeis 0 that the version/format of the data is "v0" and that does not limit our future expansion.Or put another way:
Quoted 10 lines
I'm just fine if our initial implementation provides sufficient information for us to completely parse the file even when the file is generated by a newer qemu (we have a length, so we know how far to skip to find the next entry), while at the same time throwing up our hands if the length is non-zero (we won't read the bitmap at all, because we don't know if the non-zero
extra_datacontains instructions that would change how to interpret the data) or even prevent writes (if the bitmap entry is marked automatic, we must refuse any write that would requiring an update to the bitmap because we don't know how to write to a bitmap while correctly preserving semantics of thoseextra_databytes).
Quoted 6 lines
Can we assume that the
extra_datadoesn't contain references to clusters? Otherwise we need to forbid 'qemu-img check -r leaks' when there is unknownextra_data.FWIW, this assumption is already made for snapshots, so it seems okay to make it here as well. But we could be explicit about it.
ok, will do.
Quoted 2 lines
Kevin
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Am 11.01.2016 um 14:05 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 60 lines
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-1@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clear features if it clears the respective bits from this field first. - Bits 0-63: Reserved (set to 0)
docs/specs/qcow2.txt in the description of a field. · line 115
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency for the bitmaps + extension data. + + It is an error if this bit is set without the + bitmaps extension present. + + If the bitmaps extension is present but this + bit is unset, the bitmaps extension data is + inconsistent.
It may as well be consistent, but we don't know.
Perhaps something like "must be considered inconsistent" or "is potentially inconsistent".
Quoted 14 lines
docs/specs/qcow2.txt +3
-0+ + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width@@ -123,6 +134,7 @@ be stored. Each extension has a structure like the following: 0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table + 0x23852875 - Bitmaps extension other - Unknown header extension, can be safely ignored
docs/specs/qcow2.txt the header extension data. Each entry look like this: · line 186
Quoted 9 lines
docs/specs/qcow2.txt +6
-0@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension == + +The bitmaps extension is an optional header extension. It provides the ability +to store bitmaps related to a virtual disk. For now, there is only one bitmap +type: the dirty tracking bitmap, which tracks virtual disk changes from some +point in time.
I have one major problem with this patch, and it starts here.
The spec talks about dirty tracking bitmaps all the way, but it never really defines what a dirty tracking bitmap even contains. It has a few hints here and there, but they aren't consistent.
Here's the first hint: They track "virtual disk changes", which implies they track guest clusters rather than host clusters.
Quoted 25 lines
docs/specs/qcow2.txt +21
-0+The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_features above. + +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmap directory + starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining a reference count
docs/specs/qcow2.txt Snapshot table entry: · line 408
Quoted 9 lines
docs/specs/qcow2.txt +6
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entry size to the next multiple of 8. + + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored.
s/related/related to/
Quoted 22 lines
docs/specs/qcow2.txt +22
-0+Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap. + + 12 - 15: flags
docs/specs/qcow2.txt Snapshot table entry: · line 441
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of the virtual + disk by any application that would write to this qcow2 + file (including writes, snapshot switching, etc.). The + type of this bitmap must be 'dirty tracking bitmap'.
This suggests that we can represent snapshot switching in a dirty tracking bitmap. Which is almost impossible if we track guest clusters, except if loading a snapshot should mean that all bits in the bitmap are set. However, that feels a bit useless and dirty tracking across snapshots doesn't seem to make a whole lot of sense anyway.
Maybe what would make more sense is that a bitmap is tied to a specific snapshot or bitmaps are included in a snapshot, so that you can revert to the dirty status as it was when the snapshot was taken.
Or, if you really meant, that the tracking words on a host cluster level, what clusters would be included in the bitmap? Would only the virtual disk be included? VM state? Any metadata?
It seems we definitely need a new section on dirty tracking bitmaps that describes what the bitmap actually means and how it's supposed to work with snapshots. I guess we could also talk about how it works with other changes to the image like resizing.
Quoted 24 lines
docs/specs/qcow2.txt +24
-0+ Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support granularity_bits + greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of the image + accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. +
docs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ Note: Qemu currently doesn't support values greater than + 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap.
We talked about this in the other subthread.
Quoted 45 lines
docs/specs/qcow2.txt +45
-0+ variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table. + +Each bitmap table has a variable size (stored in the bitmap directory Entry) +and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero. + If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see +above). +
docs/specs/qcow2.txt Snapshot table entry: · line 532
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the last +cluster of the bitmap data contains some unused tail bits. These bits must be +zero.
In which order are the bits stored in the bitmap?
Kevin
Quoted 186 lines · Kevin Wolf
Am 11.01.2016 um 14:05 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 60 lines
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-1@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clear features if it clears the respective bits from this field first. - Bits 0-63: Reserved (set to 0)docs/specs/qcow2.txt in the description of a field. · line 115
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency for the bitmaps + extension data. + + It is an error if this bit is set without the + bitmaps extension present. + + If the bitmaps extension is present but this + bit is unset, the bitmaps extension data is + inconsistent.It may as well be consistent, but we don't know.
Perhaps something like "must be considered inconsistent" or "is potentially inconsistent".
Quoted 14 lines
docs/specs/qcow2.txt +3
-0+ + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width@@ -123,6 +134,7 @@ be stored. Each extension has a structure like the following: 0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table + 0x23852875 - Bitmaps extension other - Unknown header extension, can be safely ignoreddocs/specs/qcow2.txt the header extension data. Each entry look like this: · line 186
Quoted 9 lines
docs/specs/qcow2.txt +6
-0@@ -166,6 +178,34 @@ the header extension data. Each entry look like this: terminated if it has full length) +== Bitmaps extension == + +The bitmaps extension is an optional header extension. It provides the ability +to store bitmaps related to a virtual disk. For now, there is only one bitmap +type: the dirty tracking bitmap, which tracks virtual disk changes from some +point in time.I have one major problem with this patch, and it starts here.
The spec talks about dirty tracking bitmaps all the way, but it never really defines what a dirty tracking bitmap even contains. It has a few hints here and there, but they aren't consistent.
Here's the first hint: They track "virtual disk changes", which implies they track guest clusters rather than host clusters.
Quoted 25 lines
docs/specs/qcow2.txt +21
-0+The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_features above. + +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image. Must be + greater than or equal to 1. + + Note: Qemu currently only supports up to 65535 bitmaps per + image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is the cumulative + size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmap directory + starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining a reference countdocs/specs/qcow2.txt Snapshot table entry: · line 408
Quoted 9 lines
docs/specs/qcow2.txt +6
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entry size to the next multiple of 8. + + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability to store bitmaps +related a virtual disk. This section describes how these bitmaps are stored.s/related/related to/
Quoted 22 lines
docs/specs/qcow2.txt +22
-0+Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directory entry. The +bitmap directory is a contiguous area in the image file, whose starting offset +and length are given by the header extension fields bitmap_directory_offset and +bitmap_directory_size. The entries of the bitmap directory have variable +length, depending on the length of the bitmap name and extra data. These +entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmap table + (described below) for the bitmap starts. Must be aligned to + a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of the bitmap. + + 12 - 15: flagsdocs/specs/qcow2.txt Snapshot table entry: · line 441
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of the virtual + disk by any application that would write to this qcow2 + file (including writes, snapshot switching, etc.). The + type of this bitmap must be 'dirty tracking bitmap'.This suggests that we can represent snapshot switching in a dirty tracking bitmap. Which is almost impossible if we track guest clusters, except if loading a snapshot should mean that all bits in the bitmap are set. However, that feels a bit useless and dirty tracking across snapshots doesn't seem to make a whole lot of sense anyway.
Maybe what would make more sense is that a bitmap is tied to a specific snapshot or bitmaps are included in a snapshot, so that you can revert to the dirty status as it was when the snapshot was taken.
Quoted 8 lines
Or, if you really meant, that the tracking words on a host cluster level, what clusters would be included in the bitmap? Would only the virtual disk be included? VM state? Any metadata?
It seems we definitely need a new section on dirty tracking bitmaps that describes what the bitmap actually means and how it's supposed to work with snapshots. I guess we could also talk about how it works with other changes to the image like resizing.
Bitmaps track guest clusters.
Backup is not related to snapshots, so, I think set all bits in the dirty bitmap on snapshot switch is ok. Of course we can't just ignore snapshot switch, as it changes how user (and backup) sees the virtual disk. Deleting the bitmap on snapshot switch is not good option too I think.
Bitmap size is defined to be equal to virtual disk size. So on resize, the one who resize must resize the bitmap too, to maintain accordance of the image file and the spec.
I'll think about such section, ok.
Quoted 85 lines
Quoted 25 lines
docs/specs/qcow2.txt +24
-0+ Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support granularity_bits + greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of the image + accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. +docs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 10 lines
docs/specs/qcow2.txt +10
-0+ Note: Qemu currently doesn't support values greater than + 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined, extra_data_size is + reserved and must be zero. + + variable: Type-specific extra data for the bitmap.We talked about this in the other subthread.
Quoted 48 lines
docs/specs/qcow2.txt +48
-0+ variable: The name of the bitmap (not null terminated). Must be + unique among all bitmap names within the bitmaps extension. + + variable: Padding to round up the bitmap directory entry size to the + next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for the mapping of +bitmap data to host clusters. This structure is called the bitmap table. + +Each bitmap table has a variable size (stored in the bitmap directory Entry) +and may use multiple clusters, however, it must be contiguous in the image +file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 are non-zero. + If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be aligned to + a cluster boundary. If the offset is 0, the cluster is + unallocated; in that case, bit 0 determines how this + cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters, described by the +bitmap table. Given an offset (in bytes) into the bitmap data, the offset into +the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry are zero (see +above). + +Given an offset byte_nr into the virtual disk and the bitmap's granularity, the +bit offset into the bitmap can be calculated like this: +
Quoted 8 lines
docs/specs/qcow2.txt Snapshot table entry: · line 532
Quoted 7 lines
docs/specs/qcow2.txt +7
-0+ bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster size then the last +cluster of the bitmap data contains some unused tail bits. These bits must be +zero.In which order are the bits stored in the bitmap?
What do you mean?
Quoted 2 lines
Kevin
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Quoted 351 lines · Vladimir Sementsov-Ogievskiy
Quoted 213 lines · Kevin Wolf
Am 11.01.2016 um 14:05 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 71 lines
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-0@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clearfeatures if it
clears the respective bits from this field first.
- Bits 0-63: Reserved (set to 0)docs/specs/qcow2.txt +2
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency forthe bitmaps
docs/specs/qcow2.txt +3
-0+ extension data. + + It is an error if this bit is setwithout the
docs/specs/qcow2.txt +3
-0+ bitmaps extension present. + + If the bitmaps extension is presentQuoted 4 lines
but this
+ bit is unset, the bitmaps extension
data is
+ inconsistent.It may as well be consistent, but we don't know.
Perhaps something like "must be considered inconsistent" or "is potentially inconsistent".
Quoted 20 lines
docs/specs/qcow2.txt +2
-0+ + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count blockentry (width
docs/specs/qcow2.txt +0
-0@@ -123,6 +134,7 @@ be stored. Each extension has a structure likethe following:
0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table+
0x23852875- Bitmaps extension
other - Unknown header extension, can
be safely
ignored
@@ -166,6 +178,34 @@ the header extension data. Each entry look
like this:
terminated if it has full length)
+== Bitmaps extension ==docs/specs/qcow2.txt +2
-0+ +The bitmaps extension is an optional header extension. It providesQuoted 6 lines
the ability
+to store bitmaps related to a virtual disk. For now, there is only one bitmap
+type: the dirty tracking bitmap, which tracks virtual disk changes from some
+point in time.I have one major problem with this patch, and it starts here.
The spec talks about dirty tracking bitmaps all the way, but it never really defines what a dirty tracking bitmap even contains. It has a few hints here and there, but they aren't consistent.
Here's the first hint: They track "virtual disk changes", which implies they track guest clusters rather than host clusters.
Quoted 38 lines
docs/specs/qcow2.txt +2
-0+The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_featuresabove.
docs/specs/qcow2.txt +5
-0+ +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image.Must be
docs/specs/qcow2.txt +3
-0+ greater than or equal to 1. + + Note: Qemu currently only supports up to 65535bitmaps per
docs/specs/qcow2.txt +4
-0+ image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is thecumulative
docs/specs/qcow2.txt +4
-0+ size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmapdirectory
docs/specs/qcow2.txt +3
-0+ starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining areference count
docs/specs/qcow2.txt +0
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entrysize to the
next multiple of 8.docs/specs/qcow2.txt +5
-0+ + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability toQuoted 3 lines
store bitmaps
+related a virtual disk. This section describes how these bitmaps are stored.s/related/related to/
Quoted 37 lines
docs/specs/qcow2.txt +5
-0+Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directoryentry. The
+bitmap directory is a contiguous area in the image file, whose starting offset
+and length are given by the header extension fieldsbitmap_directory_offsetand
+bitmap_directory_size. The entries of the bitmap directory have variable
+length, depending on the length of the bitmap name and extra data. Thesedocs/specs/qcow2.txt +6
-0+entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmaptable
+ (described below) for the bitmap starts. Must be
aligned todocs/specs/qcow2.txt +4
-0+ a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of thebitmap.
docs/specs/qcow2.txt +9
-0+ + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of theQuoted 7 lines
virtual
+ disk by any application that would write to
this qcow2
+ file (including writes, snapshot switching,
etc.). The
+ type of this bitmap must be 'dirty tracking
bitmap'.This suggests that we can represent snapshot switching in a dirty tracking bitmap. Which is almost impossible if we track guest clusters, except if loading a snapshot should mean that all bits in the bitmap are set. However, that feels a bit useless and dirty tracking across snapshots doesn't seem to make a whole lot of sense anyway.
Maybe what would make more sense is that a bitmap is tied to a specific snapshot or bitmaps are included in a snapshot, so that you can revert to the dirty status as it was when the snapshot was taken.
Quoted 8 lines
Or, if you really meant, that the tracking words on a host cluster level, what clusters would be included in the bitmap? Would only the virtual disk be included? VM state? Any metadata?
It seems we definitely need a new section on dirty tracking bitmaps that describes what the bitmap actually means and how it's supposed to work with snapshots. I guess we could also talk about how it works with other changes to the image like resizing.
Bitmaps track guest clusters.
Backup is not related to snapshots, so, I think set all bits in the dirty bitmap on snapshot switch is ok. Of course we can't just ignore snapshot switch, as it changes how user (and backup) sees the virtual disk. Deleting the bitmap on snapshot switch is not good option too I think.Bitmap size is defined to be equal to virtual disk size. So on resize, the one who resize must resize the bitmap too, to maintain accordance of the image file and the spec.
I'll think about such section, ok.
Quoted 106 lines
Quoted 36 lines
docs/specs/qcow2.txt +13
-0+ Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support
granularity_bitsdocs/specs/qcow2.txt +6
-0+ greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of theimage
docs/specs/qcow2.txt +6
-0+ accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support valuesgreater than
docs/specs/qcow2.txt +6
-0+ 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined,
extra_data_sizeisdocs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 3 lines
docs/specs/qcow2.txt +3
-0+ reserved and must be zero. + + variable: Type-specific extra data for the bitmap.We talked about this in the other subthread.
Quoted 65 lines
+ variable: The name of the bitmap (not null terminated).
Must be
+ unique among all bitmap names within the bitmaps
extension.docs/specs/qcow2.txt +2
-0+ + variable: Padding to round up the bitmap directory entrysize to the
docs/specs/qcow2.txt +6
-0+ next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for themapping of
+bitmap data to host clusters. This structure is called the bitmap table.docs/specs/qcow2.txt +2
-0+ +Each bitmap table has a variable size (stored in the bitmapdirectory Entry)
+and may use multiple clusters, however, it must be contiguous in the imagedocs/specs/qcow2.txt +5
-0+file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 arenon-zero.
docs/specs/qcow2.txt +7
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must bealigned to
+ a cluster boundary. If the offset is 0, the
cluster is
+ unallocated; in that case, bit 0 determines how
thisdocs/specs/qcow2.txt +7
-0+ cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters,described by the
+bitmap table. Given an offset (in bytes) into the bitmap data, the offset intodocs/specs/qcow2.txt +7
-0+the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry arezero (see
docs/specs/qcow2.txt +3
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap'sgranularity, the
docs/specs/qcow2.txt +4
-0+bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 +Quoted 8 lines
Quoted 7 lines
docs/specs/qcow2.txt +3
-0+ (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster sizethen the last
+cluster of the bitmap data contains some unused tail bits. These bits must be
+zero.In which order are the bits stored in the bitmap?
Quoted 1 line
What do you mean?
He means BE or LE. You can intuit it from the formula, but it's not explicitly stated, I think.
Quoted 5 lines
Quoted 3 lines
Kevin
Signature · 1 line
—js
Quoted 350 lines · John Snow
Quoted 348 lines · Vladimir Sementsov-Ogievskiy
Quoted 213 lines · Kevin Wolf
Am 11.01.2016 um 14:05 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 71 lines
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-0@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clearfeatures if it
clears the respective bits from this field first.
- Bits 0-63: Reserved (set to 0)docs/specs/qcow2.txt +2
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency forthe bitmaps
docs/specs/qcow2.txt +3
-0+ extension data. + + It is an error if this bit is setwithout the
docs/specs/qcow2.txt +3
-0+ bitmaps extension present. + + If the bitmaps extension is presentQuoted 4 lines
but this
+ bit is unset, the bitmaps extension
data is
+ inconsistent.It may as well be consistent, but we don't know.
Perhaps something like "must be considered inconsistent" or "is potentially inconsistent".
Quoted 20 lines
docs/specs/qcow2.txt +2
-0+ + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count blockentry (width
docs/specs/qcow2.txt +0
-0@@ -123,6 +134,7 @@ be stored. Each extension has a structure likethe following:
0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table+
0x23852875- Bitmaps extension
other - Unknown header extension, can
be safely
ignored
@@ -166,6 +178,34 @@ the header extension data. Each entry look
like this:
terminated if it has full length)
+== Bitmaps extension ==docs/specs/qcow2.txt +2
-0+ +The bitmaps extension is an optional header extension. It providesQuoted 6 lines
the ability
+to store bitmaps related to a virtual disk. For now, there is only one bitmap
+type: the dirty tracking bitmap, which tracks virtual disk changes from some
+point in time.I have one major problem with this patch, and it starts here.
The spec talks about dirty tracking bitmaps all the way, but it never really defines what a dirty tracking bitmap even contains. It has a few hints here and there, but they aren't consistent.
Here's the first hint: They track "virtual disk changes", which implies they track guest clusters rather than host clusters.
Quoted 38 lines
docs/specs/qcow2.txt +2
-0+The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_featuresabove.
docs/specs/qcow2.txt +5
-0+ +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image.Must be
docs/specs/qcow2.txt +3
-0+ greater than or equal to 1. + + Note: Qemu currently only supports up to 65535bitmaps per
docs/specs/qcow2.txt +4
-0+ image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is thecumulative
docs/specs/qcow2.txt +4
-0+ size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmapdirectory
docs/specs/qcow2.txt +3
-0+ starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining areference count
docs/specs/qcow2.txt +0
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entrysize to the
next multiple of 8.docs/specs/qcow2.txt +5
-0+ + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability toQuoted 3 lines
store bitmaps
+related a virtual disk. This section describes how these bitmaps are stored.s/related/related to/
Quoted 37 lines
docs/specs/qcow2.txt +5
-0+Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directoryentry. The
+bitmap directory is a contiguous area in the image file, whose starting offset
+and length are given by the header extension fieldsbitmap_directory_offsetand
+bitmap_directory_size. The entries of the bitmap directory have variable
+length, depending on the length of the bitmap name and extra data. Thesedocs/specs/qcow2.txt +6
-0+entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmaptable
+ (described below) for the bitmap starts. Must be
aligned todocs/specs/qcow2.txt +4
-0+ a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of thebitmap.
docs/specs/qcow2.txt +9
-0+ + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of theQuoted 7 lines
virtual
+ disk by any application that would write to
this qcow2
+ file (including writes, snapshot switching,
etc.). The
+ type of this bitmap must be 'dirty tracking
bitmap'.This suggests that we can represent snapshot switching in a dirty tracking bitmap. Which is almost impossible if we track guest clusters, except if loading a snapshot should mean that all bits in the bitmap are set. However, that feels a bit useless and dirty tracking across snapshots doesn't seem to make a whole lot of sense anyway.
Maybe what would make more sense is that a bitmap is tied to a specific snapshot or bitmaps are included in a snapshot, so that you can revert to the dirty status as it was when the snapshot was taken.
Quoted 8 lines
Or, if you really meant, that the tracking words on a host cluster level, what clusters would be included in the bitmap? Would only the virtual disk be included? VM state? Any metadata?
It seems we definitely need a new section on dirty tracking bitmaps that describes what the bitmap actually means and how it's supposed to work with snapshots. I guess we could also talk about how it works with other changes to the image like resizing.
Bitmaps track guest clusters.
Backup is not related to snapshots, so, I think set all bits in the dirty bitmap on snapshot switch is ok. Of course we can't just ignore snapshot switch, as it changes how user (and backup) sees the virtual disk. Deleting the bitmap on snapshot switch is not good option too I think.Bitmap size is defined to be equal to virtual disk size. So on resize, the one who resize must resize the bitmap too, to maintain accordance of the image file and the spec.
I'll think about such section, ok.
Quoted 113 lines
Quoted 35 lines
docs/specs/qcow2.txt +13
-0+ Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support
granularity_bitsdocs/specs/qcow2.txt +6
-0+ greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of theimage
docs/specs/qcow2.txt +6
-0+ accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support valuesgreater than
docs/specs/qcow2.txt +6
-0+ 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined,
extra_data_sizeisdocs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 3 lines
docs/specs/qcow2.txt +3
-0+ reserved and must be zero. + + variable: Type-specific extra data for the bitmap.We talked about this in the other subthread.
Quoted 68 lines
+ variable: The name of the bitmap (not null terminated).
Must be
+ unique among all bitmap names within the bitmaps
extension.docs/specs/qcow2.txt +2
-0+ + variable: Padding to round up the bitmap directory entrysize to the
docs/specs/qcow2.txt +6
-0+ next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for themapping of
+bitmap data to host clusters. This structure is called the bitmap table.docs/specs/qcow2.txt +2
-0+ +Each bitmap table has a variable size (stored in the bitmapdirectory Entry)
+and may use multiple clusters, however, it must be contiguous in the imagedocs/specs/qcow2.txt +5
-0+file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 arenon-zero.
docs/specs/qcow2.txt +7
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must bealigned to
+ a cluster boundary. If the offset is 0, the
cluster is
+ unallocated; in that case, bit 0 determines how
thisdocs/specs/qcow2.txt +7
-0+ cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters,described by the
+bitmap table. Given an offset (in bytes) into the bitmap data, the offset intodocs/specs/qcow2.txt +7
-0+the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry arezero (see
docs/specs/qcow2.txt +3
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap'sgranularity, the
docs/specs/qcow2.txt +7
-0+bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster sizeQuoted 4 lines
then the last
+cluster of the bitmap data contains some unused tail bits. These bits must be
+zero.In which order are the bits stored in the bitmap?
Quoted 3 lines
Quoted 1 line
What do you mean?
He means BE or LE. You can intuit it from the formula, but it's not explicitly stated, I think
byte ordering? But it make sense when we deal with integers, which occupies more then 1 byte. Here is plain byte sequence. Bitmap. Like a string.. What should I write here additionally?
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
Kevin
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
Am 21.01.2016 um 09:22 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 351 lines · John Snow
Quoted 350 lines · Vladimir Sementsov-Ogievskiy
Quoted 348 lines · Kevin Wolf
Quoted 213 lines
Am 11.01.2016 um 14:05 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 71 lines
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-0@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clearfeatures if it
clears the respective bits from this field first.
- Bits 0-63: Reserved (set to 0)docs/specs/qcow2.txt +2
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency forthe bitmaps
docs/specs/qcow2.txt +3
-0+ extension data. + + It is an error if this bit is setwithout the
docs/specs/qcow2.txt +3
-0+ bitmaps extension present. + + If the bitmaps extension is presentQuoted 4 lines
but this
+ bit is unset, the bitmaps extension
data is
+ inconsistent.It may as well be consistent, but we don't know.
Perhaps something like "must be considered inconsistent" or "is potentially inconsistent".
Quoted 20 lines
docs/specs/qcow2.txt +2
-0+ + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count blockentry (width
docs/specs/qcow2.txt +0
-0@@ -123,6 +134,7 @@ be stored. Each extension has a structure likethe following:
0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table+
0x23852875- Bitmaps extension
other - Unknown header extension, can
be safely
ignored
@@ -166,6 +178,34 @@ the header extension data. Each entry look
like this:
terminated if it has full length)
+== Bitmaps extension ==docs/specs/qcow2.txt +2
-0+ +The bitmaps extension is an optional header extension. It providesQuoted 6 lines
the ability
+to store bitmaps related to a virtual disk. For now, there is only one bitmap
+type: the dirty tracking bitmap, which tracks virtual disk changes from some
+point in time.I have one major problem with this patch, and it starts here.
The spec talks about dirty tracking bitmaps all the way, but it never really defines what a dirty tracking bitmap even contains. It has a few hints here and there, but they aren't consistent.
Here's the first hint: They track "virtual disk changes", which implies they track guest clusters rather than host clusters.
Quoted 38 lines
docs/specs/qcow2.txt +2
-0+The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_featuresabove.
docs/specs/qcow2.txt +5
-0+ +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image.Must be
docs/specs/qcow2.txt +3
-0+ greater than or equal to 1. + + Note: Qemu currently only supports up to 65535bitmaps per
docs/specs/qcow2.txt +4
-0+ image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is thecumulative
docs/specs/qcow2.txt +4
-0+ size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmapdirectory
docs/specs/qcow2.txt +3
-0+ starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining areference count
docs/specs/qcow2.txt +0
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entrysize to the
next multiple of 8.docs/specs/qcow2.txt +5
-0+ + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability toQuoted 3 lines
store bitmaps
+related a virtual disk. This section describes how these bitmaps are stored.s/related/related to/
Quoted 37 lines
docs/specs/qcow2.txt +5
-0+Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directoryentry. The
+bitmap directory is a contiguous area in the image file, whose starting offset
+and length are given by the header extension fieldsbitmap_directory_offsetand
+bitmap_directory_size. The entries of the bitmap directory have variable
+length, depending on the length of the bitmap name and extra data. Thesedocs/specs/qcow2.txt +6
-0+entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmaptable
+ (described below) for the bitmap starts. Must be
aligned todocs/specs/qcow2.txt +4
-0+ a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of thebitmap.
docs/specs/qcow2.txt +9
-0+ + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of theQuoted 7 lines
virtual
+ disk by any application that would write to
this qcow2
+ file (including writes, snapshot switching,
etc.). The
+ type of this bitmap must be 'dirty tracking
bitmap'.This suggests that we can represent snapshot switching in a dirty tracking bitmap. Which is almost impossible if we track guest clusters, except if loading a snapshot should mean that all bits in the bitmap are set. However, that feels a bit useless and dirty tracking across snapshots doesn't seem to make a whole lot of sense anyway.
Maybe what would make more sense is that a bitmap is tied to a specific snapshot or bitmaps are included in a snapshot, so that you can revert to the dirty status as it was when the snapshot was taken.
Quoted 8 lines
Or, if you really meant, that the tracking words on a host cluster level, what clusters would be included in the bitmap? Would only the virtual disk be included? VM state? Any metadata?
It seems we definitely need a new section on dirty tracking bitmaps that describes what the bitmap actually means and how it's supposed to work with snapshots. I guess we could also talk about how it works with other changes to the image like resizing.
Bitmaps track guest clusters.
Backup is not related to snapshots, so, I think set all bits in the dirty bitmap on snapshot switch is ok. Of course we can't just ignore snapshot switch, as it changes how user (and backup) sees the virtual disk. Deleting the bitmap on snapshot switch is not good option too I think.Bitmap size is defined to be equal to virtual disk size. So on resize, the one who resize must resize the bitmap too, to maintain accordance of the image file and the spec.
I'll think about such section, ok.
Quoted 113 lines
Quoted 35 lines
docs/specs/qcow2.txt +13
-0+ Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support
granularity_bitsdocs/specs/qcow2.txt +6
-0+ greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of theimage
docs/specs/qcow2.txt +6
-0+ accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support valuesgreater than
docs/specs/qcow2.txt +6
-0+ 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined,
extra_data_sizeisdocs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 3 lines
docs/specs/qcow2.txt +3
-0+ reserved and must be zero. + + variable: Type-specific extra data for the bitmap.We talked about this in the other subthread.
Quoted 68 lines
+ variable: The name of the bitmap (not null terminated).
Must be
+ unique among all bitmap names within the bitmaps
extension.docs/specs/qcow2.txt +2
-0+ + variable: Padding to round up the bitmap directory entrysize to the
docs/specs/qcow2.txt +6
-0+ next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for themapping of
+bitmap data to host clusters. This structure is called the bitmap table.docs/specs/qcow2.txt +2
-0+ +Each bitmap table has a variable size (stored in the bitmapdirectory Entry)
+and may use multiple clusters, however, it must be contiguous in the imagedocs/specs/qcow2.txt +5
-0+file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 arenon-zero.
docs/specs/qcow2.txt +7
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must bealigned to
+ a cluster boundary. If the offset is 0, the
cluster is
+ unallocated; in that case, bit 0 determines how
thisdocs/specs/qcow2.txt +7
-0+ cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters,described by the
+bitmap table. Given an offset (in bytes) into the bitmap data, the offset intodocs/specs/qcow2.txt +7
-0+the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry arezero (see
docs/specs/qcow2.txt +3
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap'sgranularity, the
docs/specs/qcow2.txt +7
-0+bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster sizeQuoted 4 lines
then the last
+cluster of the bitmap data contains some unused tail bits. These bits must be
+zero.In which order are the bits stored in the bitmap?
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
What do you mean?
He means BE or LE. You can intuit it from the formula, but it's not explicitly stated, I think
byte ordering? But it make sense when we deal with integers, which occupies more then 1 byte. Here is plain byte sequence. Bitmap. Like a string.. What should I write here additionally?
I meant sub-byte ordering. Whether bit 0 is the LSB or the MSB. I assume you intend to use the LSB for bit 0, but it's not written anywhere.
Actually, that's an assumption that is made for bitmaps in the whole spec (including e.g. feature flags), even though it is only explicitly spelt out for the refcount blocks. I guess we should move it to the 'General' section.
Kevin
Quoted 357 lines · Kevin Wolf
Am 21.01.2016 um 09:22 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 350 lines · John Snow
Quoted 349 lines · Vladimir Sementsov-Ogievskiy
Quoted 348 lines · Kevin Wolf
Quoted 213 lines
Am 11.01.2016 um 14:05 hat Vladimir Sementsov-Ogievskiy geschrieben:
Quoted 71 lines
The new feature for qcow2: storing bitmaps.
This patch adds new header extension to qcow2 - Bitmaps Extension. It provides an ability to store virtual disk related bitmaps in a qcow2 image. For now there is only one type of such bitmaps: Dirty Tracking Bitmap, which just tracks virtual disk changes from some moment.
Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file, should be stored in this qcow2 file. The size of each bitmap (considering its granularity) is equal to virtual disk size.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
v7:
- Rewordings, grammar.
Max, Eric, John, thank you very much.- add last paragraph: remaining bits in bitmap data clusters must be zero.
- s/Bitmap Directory/bitmap directory/ and other names like this at the request of Max.
v6:
- reword
bitmap_directory_sizedescription
- bitmap type: make 0 reserved
-extra_data_size: resize to 4bytes
Also, I've marked this field as "must be zero". We can always change it, if we decide allowing managing app to specify any extra data, by defining some magic value as a top of user extra data.. So, for now non zeorextra_data_sizeshould be considered as an error.
- swap name andextra_datato give good alignment toextra_data.v5:
- 'Dirty bitmaps' renamed to 'Bitmaps', as we may have several types of bitmaps.
- rewordings
- move upper bounds to "Notes about Qemu limits"
- s/should/must somewhere. (but not everywhere)
- movename_sizefield closer to name itself in bitmap header
- add extra data area to bitmap header
- move bitmap data description to separate sectiondocs/specs/qcow2.txt | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 171 insertions(+), 1 deletion(-)docs/specs/qcow2.txt +0
-0@@ -103,7 +103,18 @@ in the description of a field. write to an image with unknown auto-clearfeatures if it
clears the respective bits from this field first.
- Bits 0-63: Reserved (set to 0)docs/specs/qcow2.txt +2
-0+ Bit 0: Bitmaps extension bit + This bit indicates consistency forthe bitmaps
docs/specs/qcow2.txt +3
-0+ extension data. + + It is an error if this bit is setwithout the
docs/specs/qcow2.txt +3
-0+ bitmaps extension present. + + If the bitmaps extension is presentQuoted 4 lines
but this
+ bit is unset, the bitmaps extension
data is
+ inconsistent.It may as well be consistent, but we don't know.
Perhaps something like "must be considered inconsistent" or "is potentially inconsistent".
Quoted 20 lines
docs/specs/qcow2.txt +2
-0+ + Bits 1-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count blockentry (width
docs/specs/qcow2.txt +0
-0@@ -123,6 +134,7 @@ be stored. Each extension has a structure likethe following:
0x00000000 - End of the header extension area 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table+
0x23852875- Bitmaps extension
other - Unknown header extension, can
be safely
ignored
@@ -166,6 +178,34 @@ the header extension data. Each entry look
like this:
terminated if it has full length)
+== Bitmaps extension ==docs/specs/qcow2.txt +2
-0+ +The bitmaps extension is an optional header extension. It providesQuoted 6 lines
the ability
+to store bitmaps related to a virtual disk. For now, there is only one bitmap
+type: the dirty tracking bitmap, which tracks virtual disk changes from some
+point in time.I have one major problem with this patch, and it starts here.
The spec talks about dirty tracking bitmaps all the way, but it never really defines what a dirty tracking bitmap even contains. It has a few hints here and there, but they aren't consistent.
Here's the first hint: They track "virtual disk changes", which implies they track guest clusters rather than host clusters.
Quoted 38 lines
docs/specs/qcow2.txt +2
-0+The data of the extension should be considered consistent only if the +corresponding auto-clear feature bit is set, see autoclear_featuresabove.
docs/specs/qcow2.txt +5
-0+ +The fields of the bitmaps extension are: + + 0 - 3: nb_bitmaps + The number of bitmaps contained in the image.Must be
docs/specs/qcow2.txt +3
-0+ greater than or equal to 1. + + Note: Qemu currently only supports up to 65535bitmaps per
docs/specs/qcow2.txt +4
-0+ image. + + 4 - 7: bitmap_directory_size + Size of the bitmap directory in bytes. It is thecumulative
docs/specs/qcow2.txt +4
-0+ size of all (nb_bitmaps) bitmap headers. + + 8 - 15: bitmap_directory_offset + Offset into the image file at which the bitmapdirectory
docs/specs/qcow2.txt +3
-0+ starts. Must be aligned to a cluster boundary. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining areference count
docs/specs/qcow2.txt +0
-0@@ -360,3 +400,133 @@ Snapshot table entry: variable: Padding to round up the snapshot table entrysize to the
next multiple of 8.docs/specs/qcow2.txt +5
-0+ + +== Bitmaps == + +As mentioned above, the bitmaps extension provides the ability toQuoted 3 lines
store bitmaps
+related a virtual disk. This section describes how these bitmaps are stored.s/related/related to/
Quoted 37 lines
docs/specs/qcow2.txt +5
-0+Note: all bitmaps are related to the virtual disk stored in this image. + +=== Bitmap directory === + +Each bitmap saved in the image is described in a bitmap directoryentry. The
+bitmap directory is a contiguous area in the image file, whose starting offset
+and length are given by the header extension fieldsbitmap_directory_offsetand
+bitmap_directory_size. The entries of the bitmap directory have variable
+length, depending on the length of the bitmap name and extra data. Thesedocs/specs/qcow2.txt +6
-0+entries are also called bitmap headers. + +Structure of a bitmap directory entry: + + Byte 0 - 7: bitmap_table_offset + Offset into the image file at which the bitmaptable
+ (described below) for the bitmap starts. Must be
aligned todocs/specs/qcow2.txt +4
-0+ a cluster boundary. + + 8 - 11: bitmap_table_size + Number of entries in the bitmap table of thebitmap.
docs/specs/qcow2.txt +9
-0+ + 12 - 15: flags + Bit + 0: in_use + The bitmap was not saved correctly and may be + inconsistent. + + 1: auto + The bitmap must reflect all changes of theQuoted 7 lines
virtual
+ disk by any application that would write to
this qcow2
+ file (including writes, snapshot switching,
etc.). The
+ type of this bitmap must be 'dirty tracking
bitmap'.This suggests that we can represent snapshot switching in a dirty tracking bitmap. Which is almost impossible if we track guest clusters, except if loading a snapshot should mean that all bits in the bitmap are set. However, that feels a bit useless and dirty tracking across snapshots doesn't seem to make a whole lot of sense anyway.
Maybe what would make more sense is that a bitmap is tied to a specific snapshot or bitmaps are included in a snapshot, so that you can revert to the dirty status as it was when the snapshot was taken.
Quoted 8 lines
Or, if you really meant, that the tracking words on a host cluster level, what clusters would be included in the bitmap? Would only the virtual disk be included? VM state? Any metadata?
It seems we definitely need a new section on dirty tracking bitmaps that describes what the bitmap actually means and how it's supposed to work with snapshots. I guess we could also talk about how it works with other changes to the image like resizing.
Bitmaps track guest clusters.
Backup is not related to snapshots, so, I think set all bits in the dirty bitmap on snapshot switch is ok. Of course we can't just ignore snapshot switch, as it changes how user (and backup) sees the virtual disk. Deleting the bitmap on snapshot switch is not good option too I think.Bitmap size is defined to be equal to virtual disk size. So on resize, the one who resize must resize the bitmap too, to maintain accordance of the image file and the spec.
I'll think about such section, ok.
Quoted 113 lines
Quoted 35 lines
docs/specs/qcow2.txt +13
-0+ Bits 2 - 31 are reserved and must be 0. + + 16: type + This field describes the sort of the bitmap. + Values: + 1: Dirty tracking bitmap + + Values 0, 2 - 255 are reserved. + + 17: granularity_bits + Granularity bits. Valid values: 0 - 63. + + Note: Qemu currently doesn't support
granularity_bitsdocs/specs/qcow2.txt +6
-0+ greater than 31. + + Granularity is calculated as + granularity = 1 << granularity_bits + + A bitmap's granularity is how many bytes of theimage
docs/specs/qcow2.txt +6
-0+ accounts for one bit of the bitmap. + + 18 - 19: name_size + Size of the bitmap name. Must be non-zero. + + Note: Qemu currently doesn't support valuesgreater than
docs/specs/qcow2.txt +6
-0+ 1023. + + 20 - 23: extra_data_size + Size of type-specific extra data. + + For now, as no extra data is defined,
extra_data_sizeisdocs/specs/qcow2.txt Snapshot table entry: · line 476
Quoted 3 lines
docs/specs/qcow2.txt +3
-0+ reserved and must be zero. + + variable: Type-specific extra data for the bitmap.We talked about this in the other subthread.
Quoted 68 lines
+ variable: The name of the bitmap (not null terminated).
Must be
+ unique among all bitmap names within the bitmaps
extension.docs/specs/qcow2.txt +2
-0+ + variable: Padding to round up the bitmap directory entrysize to the
docs/specs/qcow2.txt +6
-0+ next multiple of 8. + +=== Bitmap table === + +Bitmaps are stored using a one-level structure (as opposed to two-level +structure like for refcounts and guest clusters mapping) for themapping of
+bitmap data to host clusters. This structure is called the bitmap table.docs/specs/qcow2.txt +2
-0+ +Each bitmap table has a variable size (stored in the bitmapdirectory Entry)
+and may use multiple clusters, however, it must be contiguous in the imagedocs/specs/qcow2.txt +5
-0+file. + +Structure of a bitmap table entry: + + Bit 0: Reserved and must be zero if bits 9 - 55 arenon-zero.
docs/specs/qcow2.txt +7
-0+ If bits 9 - 55 are zero: + 0: Cluster should be read as all zeros. + 1: Cluster should be read as all ones. + + 1 - 8: Reserved and must be zero. + + 9 - 55: Bits 9 - 55 of the host cluster offset. Must bealigned to
+ a cluster boundary. If the offset is 0, the
cluster is
+ unallocated; in that case, bit 0 determines how
thisdocs/specs/qcow2.txt +7
-0+ cluster should be treated when read from. + + 56 - 63: Reserved and must be zero. + +=== Bitmap data === + +As noted above, bitmap data is stored in separate clusters,described by the
+bitmap table. Given an offset (in bytes) into the bitmap data, the offset intodocs/specs/qcow2.txt +7
-0+the image file can be obtained as follows: + + image_offset = + bitmap_table[bitmap_data_offset / cluster_size] + + (bitmap_data_offset % cluster_size) + +This offset is not defined if bits 9 - 55 of bitmap table entry arezero (see
docs/specs/qcow2.txt +3
-0+above). + +Given an offset byte_nr into the virtual disk and the bitmap'sgranularity, the
docs/specs/qcow2.txt +7
-0+bit offset into the bitmap can be calculated like this: + + bit_offset = + image_offset(byte_nr / granularity / 8) * 8 + + (byte_nr / granularity) % 8 + +If the size of the bitmap data is not a multiply of cluster sizeQuoted 4 lines
then the last
+cluster of the bitmap data contains some unused tail bits. These bits must be
+zero.In which order are the bits stored in the bitmap?
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
What do you mean?
He means BE or LE. You can intuit it from the formula, but it's not explicitly stated, I think
byte ordering? But it make sense when we deal with integers, which occupies more then 1 byte. Here is plain byte sequence. Bitmap. Like a string.. What should I write here additionally?
Quoted 7 lines
I meant sub-byte ordering. Whether bit 0 is the LSB or the MSB. I assume you intend to use the LSB for bit 0, but it's not written anywhere.
Actually, that's an assumption that is made for bitmaps in the whole spec (including e.g. feature flags), even though it is only explicitly spelt out for the refcount blocks. I guess we should move it to the 'General' section.
Understand now. Then it should be separate patch.
Quoted 2 lines
Kevin
Signature · 3 lines
Best regards, Vladimir * now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.