Skip to contentSkip to conversation

drop unallocated_blocks_are_zero

PATCH Superseded Vladimir Sementsov-Ogievskiy 2 participants 19 messages May 7, 2020 – May 28, 2020 Block layer

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover drop unallocated_blocks_are_zero

Hi all!

v2 (by Eric's review):

01: moved to the start of the series, add Eric's r-b 02: new
03-04: improve commit message
05: add Eric's r-b
06-08: improve commit message a bit, add Eric's r-b 09: typos and wording, rebase on 02

This is first step to block-status refactoring, and solves most simple problem mentioned in my investigation of block-status described in the thread "backing chain & block status & filters":
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg04706.html

unallocated_blocks_are_zero doesn't simplify all the logic about block-status, and happily it's not needed, as shown by the following patches. So, let's get rid of it.

Vladimir Sementsov-Ogievskiy (9):
qemu-img: convert: don't use unallocated_blocks_are_zero
block: inline bdrv_unallocated_blocks_are_zero() block/vdi: return ZERO block-status when appropriate block/vpc: return ZERO block-status when appropriate block/crypto: drop unallocated_blocks_are_zero block/iscsi: drop unallocated_blocks_are_zero block/file-posix: drop unallocated_blocks_are_zero block/vhdx: drop unallocated_blocks_are_zero
block: drop unallocated_blocks_are_zero

 include/block/block.h     |  6 ------
 include/block/block_int.h | 12 +++++++++++-
 block.c                   | 15 ---------------
 block/crypto.c            |  1 -
 block/file-posix.c        |  3 ---
 block/io.c                |  8 ++++----
 block/iscsi.c             |  1 -
 block/qcow2.c             |  1 -
 block/qed.c               |  1 -
 block/vdi.c               |  3 +--
 block/vhdx.c              |  3 ---
 block/vpc.c               |  3 +--
 qemu-img.c                |  4 +---
 13 files changed, 18 insertions(+), 43 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH v2 1/9 qemu-img: convert: don't use unallocated_blocks_are_zero

qemu-img convert wants to distinguish ZERO which comes from short backing files. unallocated_blocks_are_zero field of bdi is unrelated:
space after EOF is always considered to be zero anyway. So, just make post_backing_zero true in case of short backing file.

---
 qemu-img.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
qemu-img.c +1 -3
index 6a4327aaba..4fe751878b 100644
@@ -1632,7 +1632,6 @@ typedef struct ImgConvertState {
     BlockBackend *target;
     bool has_zero_init;
     bool compressed;
-    bool unallocated_blocks_are_zero;
     bool target_is_new;
     bool target_has_backing;
     int64_t target_backing_sectors; /* negative if unknown */
@@ -1677,7 +1676,7 @@ static int convert_iteration_sectors(ImgConvertState *s, int64_t sector_num)
 
     if (s->target_backing_sectors >= 0) {
         if (sector_num >= s->target_backing_sectors) {
-            post_backing_zero = s->unallocated_blocks_are_zero;
+            post_backing_zero = true;
         } else if (sector_num + n > s->target_backing_sectors) {
             /* Split requests around target_backing_sectors (because
              * starting from there, zeros are handled differently) */
@@ -2580,7 +2579,6 @@ static int img_convert(int argc, char **argv)
     } else {
         s.compressed = s.compressed || bdi.needs_compressed_writes;
         s.cluster_sectors = bdi.cluster_size / BDRV_SECTOR_SIZE;
-        s.unallocated_blocks_are_zero = bdi.unallocated_blocks_are_zero;
     }
 
     ret = convert_do_copy(&s);
Signature · 1 line
2.21.0
#2 lore raw

PATCH v2 2/9 block: inline bdrv_unallocated_blocks_are_zero()

In reply to Vladimir Sementsov-Ogievskiy

The function has the only user: bdrv_co_block_status(). Inline it to simplify reviewing of the following patches, which will finally drop unallocated_blocks_are_zero field too.

---
 include/block/block.h |  1 -
 block.c               | 15 ---------------
 block/io.c            | 11 ++++++++---
 3 files changed, 8 insertions(+), 19 deletions(-)
include/block/block.h +0 -1
index 8b62429aa4..931003a476 100644
@@ -431,7 +431,6 @@ int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
 int bdrv_has_zero_init_1(BlockDriverState *bs);
 int bdrv_has_zero_init(BlockDriverState *bs);
 int bdrv_has_zero_init_truncate(BlockDriverState *bs);
-bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs);
 bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
 int bdrv_block_status(BlockDriverState *bs, int64_t offset,
                       int64_t bytes, int64_t *pnum, int64_t *map,
block.c +0 -15
index cf5c19b1db..0283fdecbc 100644
@@ -5305,21 +5305,6 @@ int bdrv_has_zero_init_truncate(BlockDriverState *bs)
     return 0;
 }
 
-bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs)
-{
-    BlockDriverInfo bdi;
-
-    if (bs->backing) {
-        return false;
-    }
-
-    if (bdrv_get_info(bs, &bdi) == 0) {
-        return bdi.unallocated_blocks_are_zero;
-    }
-
-    return false;
-}
-
 bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs)
 {
     if (!(bs->open_flags & BDRV_O_UNMAP)) {
block/io.c +8 -3
index a4f9714230..00e7371d50 100644
@@ -2386,15 +2386,20 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
     if (ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ZERO)) {
         ret |= BDRV_BLOCK_ALLOCATED;
     } else if (want_zero) {
-        if (bdrv_unallocated_blocks_are_zero(bs)) {
-            ret |= BDRV_BLOCK_ZERO;
-        } else if (bs->backing) {
+        if (bs->backing) {
             BlockDriverState *bs2 = bs->backing->bs;
             int64_t size2 = bdrv_getlength(bs2);
 
             if (size2 >= 0 && offset >= size2) {
                 ret |= BDRV_BLOCK_ZERO;
             }
+        } else {
+            BlockDriverInfo bdi;
+            int ret2 = bdrv_get_info(bs, &bdi);
+
+            if (ret2 == 0 && bdi.unallocated_blocks_are_zero) {
+                ret |= BDRV_BLOCK_ZERO;
+            }
         }
     }
 
Signature · 1 line
2.21.0
Reviewed-by on v2 2/9
#3 lore raw

Re: [PATCH v2 2/9] block: inline bdrv_unallocated_blocks_are_zero()

Quoted 1 line · Vladimir Sementsov-Ogievskiy

The function has the only user: bdrv_co_block_status(). Inline it to

s/the only/only one/

Quoted 21 lines

simplify reviewing of the following patches, which will finally drop unallocated_blocks_are_zero field too.

---
  include/block/block.h |  1 -
  block.c               | 15 ---------------
  block/io.c            | 11 ++++++++---
  3 files changed, 8 insertions(+), 19 deletions(-)

+++ b/block/io.c

block/io.c +1 -3
@@ -2386,15 +2386,20 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
      if (ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ZERO)) {
          ret |= BDRV_BLOCK_ALLOCATED;
      } else if (want_zero) {
-        if (bdrv_unallocated_blocks_are_zero(bs)) {
-            ret |= BDRV_BLOCK_ZERO;
-        } else if (bs->backing) {
+        if (bs->backing) {
              BlockDriverState *bs2 = bs->backing->bs;

block/io.c static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs, · line 2400

Quoted 10 lines
block/io.c +5 -0
              int64_t size2 = bdrv_getlength(bs2);
 
              if (size2 >= 0 && offset >= size2) {
                  ret |= BDRV_BLOCK_ZERO;
              }
+        } else {
+            BlockDriverInfo bdi;
+            int ret2 = bdrv_get_info(bs, &bdi);
+
+            if (ret2 == 0 && bdi.unallocated_blocks_are_zero) {

Could perhaps condense to:

else {
    BlockDriverInfo bdi;

    if (bdrv_get_info(bs, &bd) == 0 &&
        bdi.unallocated_blocks_are_zero) {

but that's cosmetic.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#4 lore raw

Re: [PATCH v2 2/9] block: inline bdrv_unallocated_blocks_are_zero()

Quoted 38 lines · Eric Blake
Quoted 1 line · Vladimir Sementsov-Ogievskiy

The function has the only user: bdrv_co_block_status(). Inline it to

s/the only/only one/

Quoted 20 lines

simplify reviewing of the following patches, which will finally drop unallocated_blocks_are_zero field too.

---
  include/block/block.h |  1 -
  block.c               | 15 ---------------
  block/io.c            | 11 ++++++++---

  3 files changed, 8 insertions(+), 19 deletions(-)

+++ b/block/io.c

block/io.c +1 -3
@@ -2386,15 +2386,20 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
       if (ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ZERO)) {
           ret |= BDRV_BLOCK_ALLOCATED;
       } else if (want_zero) {
-        if (bdrv_unallocated_blocks_are_zero(bs)) {
-            ret |= BDRV_BLOCK_ZERO;
-        } else if (bs->backing) {
+        if (bs->backing) {

block/io.c static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs, · line 2399

Quoted 10 lines
block/io.c +5 -0
               BlockDriverState *bs2 = bs->backing->bs;
               int64_t size2 = bdrv_getlength(bs2);
               if (size2 >= 0 && offset >= size2) {
                   ret |= BDRV_BLOCK_ZERO;
               }
+        } else {
+            BlockDriverInfo bdi;
+            int ret2 = bdrv_get_info(bs, &bdi);
+
+            if (ret2 == 0 && bdi.unallocated_blocks_are_zero) {

Could perhaps condense to:

Quoted 7 lines

else {
    BlockDriverInfo bdi;

    if (bdrv_get_info(bs, &bd) == 0 &&
        bdi.unallocated_blocks_are_zero) {

but that's cosmetic.

I'll keep it as is, as it is to be removed in 09 anyway.

Quoted 3 lines
Signature · 2 lines
Best regards,
Vladimir
#5 lore raw

PATCH v2 3/9 block/vdi: return ZERO block-status when appropriate

In reply to Vladimir Sementsov-Ogievskiy

In case of !VDI_IS_ALLOCATED[], we do zero out the corresponding chunk of qiov. So, this should be reported as ZERO.

Note that this changes visible output of "qemu-img map --output=json" and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zero. For qemu-io it's less
obvious: what was unallocated now is allocated.

There is an inconsistency in understanding of unallocated regions in
Qemu: backing-supporting format-drivers return 0 block-status to report go-to-backing logic for this area. Some protocol-drivers (iscsi) return 0 to report fs-unallocated-non-zero status (i.e., don't occupy space on disk, read result is undefined).

BDRV_BLOCK_ALLOCATED is defined as something more close to go-to-backing logic. Still it is calculated as ZERO | DATA, so 0 from iscsi is treated as unallocated. It doesn't influence backing-chain behavior, as iscsi can't have backing file. But it does influence "qemu-io -c map".

We should solve this inconsistency at some future point. Now, let's just make backing-not-supporting format drivers (vdi at this patch and vpc with the following) to behave more like backing-supporting drivers and not report 0 block-status. More over, returning ZERO status is absolutely valid thing, and again, corresponds to how the other format-drivers (backing-supporting) work.

After block-status update, it never reports 0, so setting unallocated_blocks_are_zero doesn't make sense (as the only user of it is bdrv_co_block_status and it checks unallocated_blocks_are_zero only for unallocated areas). Drop it.

---
 block/vdi.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
block/vdi.c +1 -2
index 0c7835ae70..83471528d2 100644
@@ -334,7 +334,6 @@ static int vdi_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
     logout("\n");
     bdi->cluster_size = s->block_size;
     bdi->vm_state_offset = 0;
-    bdi->unallocated_blocks_are_zero = true;
     return 0;
 }
 
@@ -536,7 +535,7 @@ static int coroutine_fn vdi_co_block_status(BlockDriverState *bs,
     *pnum = MIN(s->block_size - index_in_block, bytes);
     result = VDI_IS_ALLOCATED(bmap_entry);
     if (!result) {
-        return 0;
+        return BDRV_BLOCK_ZERO;
     }
 
     *map = s->header.offset_data + (uint64_t)bmap_entry * s->block_size +
Signature · 1 line
2.21.0
Reviewed-by on v2 3/9
#6 lore raw

Re: [PATCH v2 3/9] block/vdi: return ZERO block-status when appropriate

Quoted 32 lines · Vladimir Sementsov-Ogievskiy

In case of !VDI_IS_ALLOCATED[], we do zero out the corresponding chunk of qiov. So, this should be reported as ZERO.

Note that this changes visible output of "qemu-img map --output=json" and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zero. For qemu-io it's less
obvious: what was unallocated now is allocated.

There is an inconsistency in understanding of unallocated regions in
Qemu: backing-supporting format-drivers return 0 block-status to report go-to-backing logic for this area. Some protocol-drivers (iscsi) return 0 to report fs-unallocated-non-zero status (i.e., don't occupy space on disk, read result is undefined).

BDRV_BLOCK_ALLOCATED is defined as something more close to go-to-backing logic. Still it is calculated as ZERO | DATA, so 0 from iscsi is treated as unallocated. It doesn't influence backing-chain behavior, as iscsi can't have backing file. But it does influence "qemu-io -c map".

We should solve this inconsistency at some future point. Now, let's just make backing-not-supporting format drivers (vdi at this patch and vpc with the following) to behave more like backing-supporting drivers and not report 0 block-status. More over, returning ZERO status is absolutely valid thing, and again, corresponds to how the other format-drivers (backing-supporting) work.

After block-status update, it never reports 0, so setting unallocated_blocks_are_zero doesn't make sense (as the only user of it is bdrv_co_block_status and it checks unallocated_blocks_are_zero only for unallocated areas). Drop it.

Quoted 5 lines
---
  block/vdi.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)

Yes, much better commit message, showing why the code change is correct.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#7 lore raw

PATCH v2 4/9 block/vpc: return ZERO block-status when appropriate

In reply to Vladimir Sementsov-Ogievskiy

In case when get_image_offset() returns -1, we do zero out the corresponding chunk of qiov. So, this should be reported as ZERO.

Note that this changes visible output of "qemu-img map --output=json" and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zero. For qemu-io it's less
obvious: what was unallocated now is allocated.

There is an inconsistency in understanding of unallocated regions in
Qemu: backing-supporting format-drivers return 0 block-status to report go-to-backing logic for this area. Some protocol-drivers (iscsi) return 0 to report fs-unallocated-non-zero status (i.e., don't occupy space on disk, read result is undefined).

BDRV_BLOCK_ALLOCATED is defined as something more close to go-to-backing logic. Still it is calculated as ZERO | DATA, so 0 from iscsi is treated as unallocated. It doesn't influence backing-chain behavior, as iscsi can't have backing file. But it does influence "qemu-io -c map".

We should solve this inconsistency at some future point. Now, let's just make backing-not-supporting format drivers (vdi in the previous patch and vpc now) to behave more like backing-supporting drivers and not report 0 block-status. More over, returning ZERO status is absolutely valid thing, and again, corresponds to how the other format-drivers (backing-supporting) work.

After block-status update, it never reports 0, so setting unallocated_blocks_are_zero doesn't make sense (as the only user of it is bdrv_co_block_status and it checks unallocated_blocks_are_zero only for unallocated areas). Drop it.

---
 block/vpc.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
block/vpc.c +1 -2
index 2d1eade146..555f9d8587 100644
@@ -606,7 +606,6 @@ static int vpc_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
         bdi->cluster_size = s->block_size;
     }
 
-    bdi->unallocated_blocks_are_zero = true;
     return 0;
 }
 
@@ -745,7 +744,7 @@ static int coroutine_fn vpc_co_block_status(BlockDriverState *bs,
     image_offset = get_image_offset(bs, offset, false, NULL);
     allocated = (image_offset != -1);
     *pnum = 0;
-    ret = 0;
+    ret = BDRV_BLOCK_ZERO;
 
     do {
         /* All sectors in a block are contiguous (without using the bitmap) */
Signature · 1 line
2.21.0
Reviewed-by on v2 4/9
#8 lore raw

Re: [PATCH v2 4/9] block/vpc: return ZERO block-status when appropriate

Quoted 32 lines · Vladimir Sementsov-Ogievskiy

In case when get_image_offset() returns -1, we do zero out the corresponding chunk of qiov. So, this should be reported as ZERO.

Note that this changes visible output of "qemu-img map --output=json" and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zero. For qemu-io it's less
obvious: what was unallocated now is allocated.

There is an inconsistency in understanding of unallocated regions in
Qemu: backing-supporting format-drivers return 0 block-status to report go-to-backing logic for this area. Some protocol-drivers (iscsi) return 0 to report fs-unallocated-non-zero status (i.e., don't occupy space on disk, read result is undefined).

BDRV_BLOCK_ALLOCATED is defined as something more close to go-to-backing logic. Still it is calculated as ZERO | DATA, so 0 from iscsi is treated as unallocated. It doesn't influence backing-chain behavior, as iscsi can't have backing file. But it does influence "qemu-io -c map".

We should solve this inconsistency at some future point. Now, let's just make backing-not-supporting format drivers (vdi in the previous patch and vpc now) to behave more like backing-supporting drivers and not report 0 block-status. More over, returning ZERO status is absolutely valid thing, and again, corresponds to how the other format-drivers (backing-supporting) work.

After block-status update, it never reports 0, so setting unallocated_blocks_are_zero doesn't make sense (as the only user of it is bdrv_co_block_status and it checks unallocated_blocks_are_zero only for unallocated areas). Drop it.

Quoted 4 lines
---
  block/vpc.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#9 lore raw

PATCH v2 5/9 block/crypto: drop unallocated_blocks_are_zero

In reply to Vladimir Sementsov-Ogievskiy

It's false by default, no needs to set it. We are going to drop this variable at all, so drop it now here, it doesn't hurt.

---
 block/crypto.c | 1 -
 1 file changed, 1 deletion(-)
block/crypto.c +0 -1
index e02f343590..7685e61844 100644
@@ -694,7 +694,6 @@ static int block_crypto_get_info_luks(BlockDriverState *bs,
         return ret;
     }
 
-    bdi->unallocated_blocks_are_zero = false;
     bdi->cluster_size = subbdi.cluster_size;
 
     return 0;
Signature · 1 line
2.21.0
#10 lore raw

PATCH v2 6/9 block/iscsi: drop unallocated_blocks_are_zero

In reply to Vladimir Sementsov-Ogievskiy

We set bdi->unallocated_blocks_are_zero = iscsilun->lbprz, but iscsi_co_block_status doesn't return 0 in case of iscsilun->lbprz, it returns ZERO when appropriate. So actually unallocated_blocks_are_zero is useless (it doesn't affect the only user of the field:
bdrv_co_block_status()). Drop it now.

---
 block/iscsi.c | 1 -
 1 file changed, 1 deletion(-)
block/iscsi.c +0 -1
index a8b76979d8..767e3e75fd 100644
@@ -2163,7 +2163,6 @@ static int coroutine_fn iscsi_co_truncate(BlockDriverState *bs, int64_t offset,
 static int iscsi_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
 {
     IscsiLun *iscsilun = bs->opaque;
-    bdi->unallocated_blocks_are_zero = iscsilun->lbprz;
     bdi->cluster_size = iscsilun->cluster_size;
     return 0;
 }
Signature · 1 line
2.21.0
#11 lore raw

PATCH v2 7/9 block/file-posix: drop unallocated_blocks_are_zero

In reply to Vladimir Sementsov-Ogievskiy

raw_co_block_status() in block/file-posix.c never returns 0, so unallocated_blocks_are_zero is useless (it doesn't affect the only user of the field: bdrv_co_block_status()). Drop it.

---
 block/file-posix.c | 3 ---
 1 file changed, 3 deletions(-)
block/file-posix.c +0 -3
index 05e094be29..5c01735108 100644
@@ -2878,9 +2878,6 @@ static int coroutine_fn raw_co_pwrite_zeroes(
 
 static int raw_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
 {
-    BDRVRawState *s = bs->opaque;
-
-    bdi->unallocated_blocks_are_zero = s->discard_zeroes;
     return 0;
 }
 
Signature · 1 line
2.21.0
#12 lore raw

PATCH v2 8/9 block/vhdx: drop unallocated_blocks_are_zero

In reply to Vladimir Sementsov-Ogievskiy

vhdx doesn't have .bdrv_co_block_status handler, so DATA|ALLOCATED is always assumed for it in bdrv_co_block_status(). unallocated_blocks_are_zero is useless (it doesn't affect the only user of the field: bdrv_co_block_status()), drop it.

---
 block/vhdx.c | 3 ---
 1 file changed, 3 deletions(-)
block/vhdx.c +0 -3
index aedd782604..45963a3166 100644
@@ -1164,9 +1164,6 @@ static int vhdx_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
 
     bdi->cluster_size = s->block_size;
 
-    bdi->unallocated_blocks_are_zero =
-        (s->params.data_bits & VHDX_PARAMS_HAS_PARENT) == 0;
-
     return 0;
 }
 
Signature · 1 line
2.21.0
#13 lore raw

PATCH v2 9/9 block: drop unallocated_blocks_are_zero

In reply to Vladimir Sementsov-Ogievskiy

Currently this field only set by qed and qcow2. But in fact, all backing-supporting formats (parallels, qcow, qcow2, qed, vmdk) share these semantics: on unallocated blocks, if there is no backing file they just memset the buffer with zeroes.

So, document this behavior for .supports_backing and drop .unallocated_blocks_are_zero

---
 include/block/block.h     |  5 -----
 include/block/block_int.h | 12 +++++++++++-
 block/io.c                |  9 ++-------
 block/qcow2.c             |  1 -
 block/qed.c               |  1 -
 5 files changed, 13 insertions(+), 15 deletions(-)
include/block/block.h +0 -5
index 931003a476..db1cb503ec 100644
@@ -21,11 +21,6 @@ typedef struct BlockDriverInfo {
     /* offset at which the VM state can be saved (0 if not possible) */
     int64_t vm_state_offset;
     bool is_dirty;
-    /*
-     * True if unallocated blocks read back as zeroes. This is equivalent
-     * to the LBPRZ flag in the SCSI logical block provisioning page.
-     */
-    bool unallocated_blocks_are_zero;
     /*
      * True if this block driver only supports compressed writes
      */
include/block/block_int.h +11 -1
index 92335f33c7..8fac6c3ce2 100644
@@ -115,7 +115,17 @@ struct BlockDriver {
      */
     bool bdrv_needs_filename;
 
-    /* Set if a driver can support backing files */
+    /*
+     * Set if a driver can support backing files. This also implies the
+     * following semantics:
+     *
+     *  - Return status 0 of .bdrv_co_block_status means that corresponding
+     *    blocks are not allocated in this layer of backing-chain
+     *  - For such (unallocated) blocks, read will:
+     *    - fill buffer with zeros if there is no backing file
+     *    - read from the backing file otherwise, where the block layer
+     *      takes care of reading zeros beyond EOF if backing file is short
+     */
     bool supports_backing;
 
     /* For handling image reopen for split or non-split files */
block/io.c +2 -7
index 00e7371d50..484adec5a1 100644
@@ -2385,7 +2385,7 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
 
     if (ret & (BDRV_BLOCK_DATA | BDRV_BLOCK_ZERO)) {
         ret |= BDRV_BLOCK_ALLOCATED;
-    } else if (want_zero) {
+    } else if (want_zero && bs->drv->supports_backing) {
         if (bs->backing) {
             BlockDriverState *bs2 = bs->backing->bs;
             int64_t size2 = bdrv_getlength(bs2);
@@ -2394,12 +2394,7 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
                 ret |= BDRV_BLOCK_ZERO;
             }
         } else {
-            BlockDriverInfo bdi;
-            int ret2 = bdrv_get_info(bs, &bdi);
-
-            if (ret2 == 0 && bdi.unallocated_blocks_are_zero) {
-                ret |= BDRV_BLOCK_ZERO;
-            }
+            ret |= BDRV_BLOCK_ZERO;
         }
     }
 
block/qcow2.c +0 -1
index 2ba0b17c39..dc3c0aac2b 100644
@@ -4858,7 +4858,6 @@ err:
 static int qcow2_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
 {
     BDRVQcow2State *s = bs->opaque;
-    bdi->unallocated_blocks_are_zero = true;
     bdi->cluster_size = s->cluster_size;
     bdi->vm_state_offset = qcow2_vm_state_offset(s);
     return 0;
block/qed.c +0 -1
index b0fdb8f565..fb7833dc8b 100644
@@ -1514,7 +1514,6 @@ static int bdrv_qed_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)
     memset(bdi, 0, sizeof(*bdi));
     bdi->cluster_size = s->header.cluster_size;
     bdi->is_dirty = s->header.features & QED_F_NEED_CHECK;
-    bdi->unallocated_blocks_are_zero = true;
     return 0;
 }
 
Signature · 1 line
2.21.0
Reviewed-by on v2 9/9
#14 lore raw

Re: [PATCH v2 9/9] block: drop unallocated_blocks_are_zero

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Currently this field only set by qed and qcow2. But in fact, all backing-supporting formats (parallels, qcow, qcow2, qed, vmdk) share these semantics: on unallocated blocks, if there is no backing file they just memset the buffer with zeroes.

So, document this behavior for .supports_backing and drop .unallocated_blocks_are_zero

Quoted 8 lines
---
  include/block/block.h     |  5 -----
  include/block/block_int.h | 12 +++++++++++-
  block/io.c                |  9 ++-------
  block/qcow2.c             |  1 -
  block/qed.c               |  1 -
  5 files changed, 13 insertions(+), 15 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#15 lore raw

[PATCH v2 10/9] qed: Simplify backing reads

In reply to Vladimir Sementsov-Ogievskiy

The other four drivers that support backing files (qcow, qcow2, parallels, vmdk) all rely on the block layer to populate zeroes when reading beyond EOF of a short backing file. We can simplify the qed code by doing likewise.

---

I noticed this during my audit that v1 of Vladimir's series was correct.

No change in iotests results (test 274 is currently failing for qed, but for other reasons:
+Traceback (most recent call last):
+ File "274", line 24, in <module>
+ iotests.verify_image_format(supported_fmts=['qcow2'])
+AttributeError: module 'iotests' has no attribute 'verify_image_format' )

 block/qed.h |  1 -
 block/qed.c | 64 +++++------------------------------------------------
 2 files changed, 6 insertions(+), 59 deletions(-)
block/qed.h +0 -1
index 42c115d8220c..3d12bf78d412 100644
@@ -140,7 +140,6 @@ typedef struct QEDAIOCB {
 
     /* Current cluster scatter-gather list */
     QEMUIOVector cur_qiov;
-    QEMUIOVector *backing_qiov;
     uint64_t cur_pos;               /* position on block device, in bytes */
     uint64_t cur_cluster;           /* cluster offset in image file */
     unsigned int cur_nclusters;     /* number of clusters being accessed */
block/qed.c +6 -58

102 lines not on this page · raw message

Signature · 1 line
2.26.2
#16 lore raw

Re: [PATCH v2 10/9] qed: Simplify backing reads

Quoted 16 lines · Eric Blake

The other four drivers that support backing files (qcow, qcow2, parallels, vmdk) all rely on the block layer to populate zeroes when reading beyond EOF of a short backing file. We can simplify the qed code by doing likewise.

---

I noticed this during my audit that v1 of Vladimir's series was correct.

No change in iotests results (test 274 is currently failing for qed, but for other reasons:

(quoted diff) +4 -0
+Traceback (most recent call last):
+  File "274", line 24, in <module>
+    iotests.verify_image_format(supported_fmts=['qcow2'])
+AttributeError: module 'iotests' has no attribute 'verify_image_format'
Quoted 1 line

)

That iotest failure was due to a stale branch on my end; after updating to latest git master plus Kevin's latest 'block' branch, 274 is now skipped on qed.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author Reviewed-by
#17 lore raw

Re: [PATCH v2 10/9] qed: Simplify backing reads

In reply to Eric Blake
Quoted 6 lines · Eric Blake

The other four drivers that support backing files (qcow, qcow2, parallels, vmdk) all rely on the block layer to populate zeroes when reading beyond EOF of a short backing file. We can simplify the qed code by doing likewise.

Signature · 2 lines
Best regards,
Vladimir
#18 lore raw
In reply to Vladimir Sementsov-Ogievskiy

Ping

The whole series reviewed by Eric, with only one grammar fix needed in 02 commit message (and possible drop of ret2, up to maintainer).

Quoted 48 lines · Vladimir Sementsov-Ogievskiy

Hi all!

v2 (by Eric's review):

01: moved to the start of the series, add Eric's r-b 02: new
03-04: improve commit message
05: add Eric's r-b
06-08: improve commit message a bit, add Eric's r-b 09: typos and wording, rebase on 02

This is first step to block-status refactoring, and solves most simple problem mentioned in my investigation of block-status described in the thread "backing chain & block status & filters":

   https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg04706.html

unallocated_blocks_are_zero doesn't simplify all the logic about block-status, and happily it's not needed, as shown by the following patches. So, let's get rid of it.

Vladimir Sementsov-Ogievskiy (9):
qemu-img: convert: don't use unallocated_blocks_are_zero
block: inline bdrv_unallocated_blocks_are_zero() block/vdi: return ZERO block-status when appropriate block/vpc: return ZERO block-status when appropriate block/crypto: drop unallocated_blocks_are_zero block/iscsi: drop unallocated_blocks_are_zero block/file-posix: drop unallocated_blocks_are_zero block/vhdx: drop unallocated_blocks_are_zero
block: drop unallocated_blocks_are_zero

  include/block/block.h     |  6 ------
  include/block/block_int.h | 12 +++++++++++-
  block.c                   | 15 ---------------
  block/crypto.c            |  1 -
  block/file-posix.c        |  3 ---
  block/io.c                |  8 ++++----
  block/iscsi.c             |  1 -
  block/qcow2.c             |  1 -
  block/qed.c               |  1 -
  block/vdi.c               |  3 +--
  block/vhdx.c              |  3 ---
  block/vpc.c               |  3 +--
  qemu-img.c                |  4 +---
  13 files changed, 18 insertions(+), 43 deletions(-)
Signature · 2 lines
Best regards,
Vladimir