Skip to contentSkip to conversation

block: logical block provisioning enhancements

PATCH Superseded Peter Lieven 3 participants 42 messages Sep 17, 2013 – Sep 19, 2013 Block layer

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover block: logical block provisioning enhancements

this patch adds the ability for targets to stay sparse during block migration (if the zero_blocks capability is set) and qemu-img convert even if the target does not have has_zero_init = 1.

the series was especially developed for iSCSI, but it should also work with other drivers with little or no adjustments. these adjustments should be limited to providing block provisioning information through get_block_info and/or honouring BDRV_REQ_MAY_UNMAP on writing zeroes.

there are also 4 patches which are fixes/enhancements for the get_block_status API discovered during development of this series.

v1->v2:
- moved block max_discard and max_write_zeroes to BlockDriverState
- added discard_alignment and write_zeroes_alignment to BlockDriverState
- added bdrv_has_discard_zeroes() and bdrv_has_discard_write_zeroes()
- added logic to bdrv_co_discard and bdrv_co_do_write_zeroes to honour limit and alignment info.
- added support for -S 0 in qemu-img convert.

Peter Lieven (20):
block: make BdrvRequestFlags public
block: add flags to bdrv_*_write_zeroes
block: introduce BDRV_REQ_MAY_UNMAP request flag
block: introduce bdrv_has_discard_zeroes and
bdrv_has_discard_write_zeroes
block/raw: add bdrv_has_discard_zeroes and
bdrv_has_discard_write_zeroes
block: add discard and write_zeroes limits and alignment to
BlockDriverState
block: honour alignment and limit in bdrv_co_do_write_zeroes
block: honour alignment and limit in bdrv_co_discard
iscsi: simplify iscsi_co_discard
iscsi: set limits in BlockDriverState
iscsi: add bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes
iscsi: add bdrv_co_write_zeroes
block: introduce bdrv_zeroize
block/get_block_status: set *pnum = 0 on error block/get_block_status: avoid segfault if there is no backing_hd block/get_block_status: avoid redundant callouts on raw devices block/get_block_status: fix BDRV_BLOCK_ZERO for unallocated blocks
qemu-img: add support for fully allocated images
qemu-img: conditionally zero out target on convert block/raw: copy block limits and alignment information on raw_open

 block-migration.c         |    3 +-
 block.c                   |  187 ++++++++++++++++++++++++++++++++++++---------
 block/backup.c            |    3 +-
 block/iscsi.c             |  148 +++++++++++++++++++++++++----------
 block/qcow2.c             |    2 +-
 block/qed.c               |    3 +-
 block/raw_bsd.c           |   65 ++++++++++------
 block/vmdk.c              |    3 +-
 include/block/block.h     |   19 ++++-
 include/block/block_int.h |   27 ++++++-
 qemu-img.c                |   18 ++++-
 qemu-io-cmds.c            |    2 +-
 12 files changed, 368 insertions(+), 112 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v2 1/20 block: make BdrvRequestFlags public

---
 block.c               |    5 -----
 include/block/block.h |    5 +++++
 2 files changed, 5 insertions(+), 5 deletions(-)
block.c +0 -5
index a325efc..878f365 100644
@@ -51,11 +51,6 @@
 
 #define NOT_DONE 0x7fffffff /* used while emulated sync operation in progress */
 
-typedef enum {
-    BDRV_REQ_COPY_ON_READ = 0x1,
-    BDRV_REQ_ZERO_WRITE   = 0x2,
-} BdrvRequestFlags;
-
 static void bdrv_dev_change_media_cb(BlockDriverState *bs, bool load);
 static BlockDriverAIOCB *bdrv_aio_readv_em(BlockDriverState *bs,
         int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
include/block/block.h +5 -0
index 728ec1a..3249261 100644
@@ -62,6 +62,11 @@ typedef struct BlockDevOps {
     void (*resize_cb)(void *opaque);
 } BlockDevOps;
 
+typedef enum {
+    BDRV_REQ_COPY_ON_READ = 0x1,
+    BDRV_REQ_ZERO_WRITE   = 0x2,
+} BdrvRequestFlags;
+
 #define BDRV_O_RDWR        0x0002
 #define BDRV_O_SNAPSHOT    0x0008 /* open the file read only and save writes in a snapshot */
 #define BDRV_O_NOCACHE     0x0020 /* do not use the host page cache */
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v2 2/20 block: add flags to bdrv_*_write_zeroes

In reply to Peter Lieven
---
 block-migration.c         |    2 +-
 block.c                   |   20 +++++++++++---------
 block/backup.c            |    3 ++-
 block/qcow2.c             |    2 +-
 block/qed.c               |    3 ++-
 block/raw_bsd.c           |    5 +++--
 block/vmdk.c              |    3 ++-
 include/block/block.h     |    4 ++--
 include/block/block_int.h |    2 +-
 qemu-io-cmds.c            |    2 +-
 10 files changed, 26 insertions(+), 20 deletions(-)
block-migration.c +1 -1
index daf9ec1..713a8e3 100644
@@ -780,7 +780,7 @@ static int block_load(QEMUFile *f, void *opaque, int version_id)
             }
 
             if (flags & BLK_MIG_FLAG_ZERO_BLOCK) {
-                ret = bdrv_write_zeroes(bs, addr, nr_sectors);
+                ret = bdrv_write_zeroes(bs, addr, nr_sectors, 0);
             } else {
                 buf = g_malloc(BLOCK_SIZE);
                 qemu_get_buffer(f, buf, BLOCK_SIZE);
block.c +11 -9
index 878f365..e5918a7 100644
@@ -79,7 +79,7 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
                                                bool is_write);
 static void coroutine_fn bdrv_co_do_rw(void *opaque);
 static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
-    int64_t sector_num, int nb_sectors);
+    int64_t sector_num, int nb_sectors, BdrvRequestFlags flags);
 
 static QTAILQ_HEAD(, BlockDriverState) bdrv_states =
     QTAILQ_HEAD_INITIALIZER(bdrv_states);
@@ -2335,10 +2335,11 @@ int bdrv_writev(BlockDriverState *bs, int64_t sector_num, QEMUIOVector *qiov)
     return bdrv_rwv_co(bs, sector_num, qiov, true, 0);
 }
 
-int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num, int nb_sectors)
+int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num,
+                      int nb_sectors, BdrvRequestFlags flags)
 {
     return bdrv_rw_co(bs, sector_num, NULL, nb_sectors, true,
-                      BDRV_REQ_ZERO_WRITE);
+                      BDRV_REQ_ZERO_WRITE | flags);
 }
 
 int bdrv_pread(BlockDriverState *bs, int64_t offset,
@@ -2520,7 +2521,7 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BlockDriverState *bs,
     if (drv->bdrv_co_write_zeroes &&
         buffer_is_zero(bounce_buffer, iov.iov_len)) {
         ret = bdrv_co_do_write_zeroes(bs, cluster_sector_num,
-                                      cluster_nb_sectors);
+                                      cluster_nb_sectors, 0);
     } else {
         /* This does not change the data on the disk, it is not necessary
          * to flush even in cache=writethrough mode.
@@ -2654,7 +2655,7 @@ int coroutine_fn bdrv_co_copy_on_readv(BlockDriverState *bs,
 }
 
 static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
-    int64_t sector_num, int nb_sectors)
+    int64_t sector_num, int nb_sectors, BdrvRequestFlags flags)
 {
     BlockDriver *drv = bs->drv;
     QEMUIOVector qiov;
@@ -2666,7 +2667,7 @@ static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
 
     /* First try the efficient write zeroes operation */
     if (drv->bdrv_co_write_zeroes) {
-        ret = drv->bdrv_co_write_zeroes(bs, sector_num, nb_sectors);
+        ret = drv->bdrv_co_write_zeroes(bs, sector_num, nb_sectors, flags);
         if (ret != -ENOTSUP) {
             return ret;
         }
@@ -2721,7 +2722,7 @@ static int coroutine_fn bdrv_co_do_writev(BlockDriverState *bs,
     if (ret < 0) {
         /* Do nothing, write notifier decided to fail this request */
     } else if (flags & BDRV_REQ_ZERO_WRITE) {
-        ret = bdrv_co_do_write_zeroes(bs, sector_num, nb_sectors);
+        ret = bdrv_co_do_write_zeroes(bs, sector_num, nb_sectors, flags);
     } else {
         ret = drv->bdrv_co_writev(bs, sector_num, nb_sectors, qiov);
     }
@@ -2755,12 +2756,13 @@ int coroutine_fn bdrv_co_writev(BlockDriverState *bs, int64_t sector_num,
 }
 
 int coroutine_fn bdrv_co_write_zeroes(BlockDriverState *bs,
-                                      int64_t sector_num, int nb_sectors)
+                                      int64_t sector_num, int nb_sectors,
+                                      BdrvRequestFlags flags)
 {
     trace_bdrv_co_write_zeroes(bs, sector_num, nb_sectors);
 
     return bdrv_co_do_writev(bs, sector_num, nb_sectors, NULL,
-                             BDRV_REQ_ZERO_WRITE);
+                             BDRV_REQ_ZERO_WRITE | flags);
 }
 
 /**
block/backup.c +2 -1
index 04c4b5c..5f6a642 100644
@@ -138,7 +138,8 @@ static int coroutine_fn backup_do_cow(BlockDriverState *bs,
 
         if (buffer_is_zero(iov.iov_base, iov.iov_len)) {
             ret = bdrv_co_write_zeroes(job->target,
-                                       start * BACKUP_SECTORS_PER_CLUSTER, n);
+                                       start * BACKUP_SECTORS_PER_CLUSTER,
+                                       n, 0);
         } else {
             ret = bdrv_co_writev(job->target,
                                  start * BACKUP_SECTORS_PER_CLUSTER, n,
block/qcow2.c +1 -1
index 578792f..2cb59a2 100644
@@ -1555,7 +1555,7 @@ static int qcow2_make_empty(BlockDriverState *bs)
 }
 
 static coroutine_fn int qcow2_co_write_zeroes(BlockDriverState *bs,
-    int64_t sector_num, int nb_sectors)
+    int64_t sector_num, int nb_sectors, BdrvRequestFlags flags)
 {
     int ret;
     BDRVQcowState *s = bs->opaque;
block/qed.c +2 -1
index 49b3a37..d6e42f3 100644
@@ -1389,7 +1389,8 @@ static void coroutine_fn qed_co_write_zeroes_cb(void *opaque, int ret)
 
 static int coroutine_fn bdrv_qed_co_write_zeroes(BlockDriverState *bs,
                                                  int64_t sector_num,
-                                                 int nb_sectors)
+                                                 int nb_sectors,
+                                                 BdrvRequestFlags flags)
 {
     BlockDriverAIOCB *blockacb;
     BDRVQEDState *s = bs->opaque;
block/raw_bsd.c +3 -2
index a9060ca..bd4811b 100644
@@ -66,9 +66,10 @@ static int64_t coroutine_fn raw_co_get_block_status(BlockDriverState *bs,
 }
 
 static int coroutine_fn raw_co_write_zeroes(BlockDriverState *bs,
-                                            int64_t sector_num, int nb_sectors)
+                                            int64_t sector_num, int nb_sectors,
+                                            BdrvRequestFlags flags)
 {
-    return bdrv_co_write_zeroes(bs->file, sector_num, nb_sectors);
+    return bdrv_co_write_zeroes(bs->file, sector_num, nb_sectors, flags);
 }
 
 static int coroutine_fn raw_co_discard(BlockDriverState *bs,
block/vmdk.c +2 -1
index fb5b529..914a9b7 100644
@@ -1386,7 +1386,8 @@ static coroutine_fn int vmdk_co_write(BlockDriverState *bs, int64_t sector_num,
 
 static int coroutine_fn vmdk_co_write_zeroes(BlockDriverState *bs,
                                              int64_t sector_num,
-                                             int nb_sectors)
+                                             int nb_sectors,
+                                             BdrvRequestFlags flags)
 {
     int ret;
     BDRVVmdkState *s = bs->opaque;
include/block/block.h +2 -2
index 3249261..ddb6870 100644
@@ -187,7 +187,7 @@ int bdrv_read_unthrottled(BlockDriverState *bs, int64_t sector_num,
 int bdrv_write(BlockDriverState *bs, int64_t sector_num,
                const uint8_t *buf, int nb_sectors);
 int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num,
-               int nb_sectors);
+               int nb_sectors, BdrvRequestFlags flags);
 int bdrv_writev(BlockDriverState *bs, int64_t sector_num, QEMUIOVector *qiov);
 int bdrv_pread(BlockDriverState *bs, int64_t offset,
                void *buf, int count);
@@ -209,7 +209,7 @@ int coroutine_fn bdrv_co_writev(BlockDriverState *bs, int64_t sector_num,
  * because it may allocate memory for the entire region.
  */
 int coroutine_fn bdrv_co_write_zeroes(BlockDriverState *bs, int64_t sector_num,
-    int nb_sectors);
+    int nb_sectors, BdrvRequestFlags flags);
 BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
     const char *backing_file);
 int bdrv_get_backing_file_depth(BlockDriverState *bs);
include/block/block_int.h +1 -1
index 7c35198..26bad36 100644
@@ -115,7 +115,7 @@ struct BlockDriver {
      * instead.
      */
     int coroutine_fn (*bdrv_co_write_zeroes)(BlockDriverState *bs,
-        int64_t sector_num, int nb_sectors);
+        int64_t sector_num, int nb_sectors, BdrvRequestFlags flags);
     int coroutine_fn (*bdrv_co_discard)(BlockDriverState *bs,
         int64_t sector_num, int nb_sectors);
     int64_t coroutine_fn (*bdrv_co_get_block_status)(BlockDriverState *bs,
qemu-io-cmds.c +1 -1
index 8565d49..66c0b9e 100644
@@ -441,7 +441,7 @@ static void coroutine_fn co_write_zeroes_entry(void *opaque)
     CoWriteZeroes *data = opaque;
 
     data->ret = bdrv_co_write_zeroes(data->bs, data->offset / BDRV_SECTOR_SIZE,
-                                     data->count / BDRV_SECTOR_SIZE);
+                                     data->count / BDRV_SECTOR_SIZE, 0);
     data->done = true;
     if (data->ret < 0) {
         *data->total = data->ret;
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH v2 3/20 block: introduce BDRV_REQ_MAY_UNMAP request flag

In reply to Peter Lieven
---
 block-migration.c     |    3 ++-
 block.c               |    4 ++++
 block/backup.c        |    2 +-
 include/block/block.h |    7 +++++++
 4 files changed, 14 insertions(+), 2 deletions(-)
block-migration.c +2 -1
index 713a8e3..fc4ef93 100644
@@ -780,7 +780,8 @@ static int block_load(QEMUFile *f, void *opaque, int version_id)
             }
 
             if (flags & BLK_MIG_FLAG_ZERO_BLOCK) {
-                ret = bdrv_write_zeroes(bs, addr, nr_sectors, 0);
+                ret = bdrv_write_zeroes(bs, addr, nr_sectors,
+                                        BDRV_REQ_MAY_UNMAP);
             } else {
                 buf = g_malloc(BLOCK_SIZE);
                 qemu_get_buffer(f, buf, BLOCK_SIZE);
block.c +4 -0
index e5918a7..6f498fc 100644
@@ -2761,6 +2761,10 @@ int coroutine_fn bdrv_co_write_zeroes(BlockDriverState *bs,
 {
     trace_bdrv_co_write_zeroes(bs, sector_num, nb_sectors);
 
+    if (!(bs->open_flags & BDRV_O_UNMAP)) {
+        flags &= ~BDRV_REQ_MAY_UNMAP;
+    }
+
     return bdrv_co_do_writev(bs, sector_num, nb_sectors, NULL,
                              BDRV_REQ_ZERO_WRITE | flags);
 }
block/backup.c +1 -1
index 5f6a642..1ab080d 100644
@@ -139,7 +139,7 @@ static int coroutine_fn backup_do_cow(BlockDriverState *bs,
         if (buffer_is_zero(iov.iov_base, iov.iov_len)) {
             ret = bdrv_co_write_zeroes(job->target,
                                        start * BACKUP_SECTORS_PER_CLUSTER,
-                                       n, 0);
+                                       n, BDRV_REQ_MAY_UNMAP);
         } else {
             ret = bdrv_co_writev(job->target,
                                  start * BACKUP_SECTORS_PER_CLUSTER, n,
include/block/block.h +7 -0
index ddb6870..0328234 100644
@@ -65,6 +65,13 @@ typedef struct BlockDevOps {
 typedef enum {
     BDRV_REQ_COPY_ON_READ = 0x1,
     BDRV_REQ_ZERO_WRITE   = 0x2,
+    /* the BDRV_REQ_MAY_UNMAP flag is used to indicate that block driver
+     * is allowed to optimize a write zeroes request by unmapping (discarding)
+     * blocks if it is guaranteed that the result will read back as
+     * zeroes. the flag is only passed to the driver if the block device is
+     * opened with BDRV_O_UNMAP.
+     */
+    BDRV_REQ_MAY_UNMAP    = 0x4,
 } BdrvRequestFlags;
 
 #define BDRV_O_RDWR        0x0002
Signature · 1 line
1.7.9.5
Reviewed-by on v2 3/20
#4 lore raw

Re: [Qemu-devel] [PATCHv2 03/20] block: introduce BDRV_REQ_MAY_UNMAP request flag

Quoted 11 lines · Peter Lieven
---
 block-migration.c     |    3 ++-
 block.c               |    4 ++++
 block/backup.c        |    2 +-
 include/block/block.h |    7 +++++++
 4 files changed, 14 insertions(+), 2 deletions(-)

+++ b/include/block/block.h

include/block/block.h typedef struct BlockDevOps { · line 68

Quoted 4 lines
include/block/block.h +1 -0
@@ -65,6 +65,13 @@ typedef struct BlockDevOps {
 typedef enum {
     BDRV_REQ_COPY_ON_READ = 0x1,
     BDRV_REQ_ZERO_WRITE   = 0x2,
+    /* the BDRV_REQ_MAY_UNMAP flag is used to indicate that block driver

Minor grammar improvements:

s/the/The/
s/that/that the/

include/block/block.h typedef struct BlockDevOps { · line 71

Quoted 3 lines
include/block/block.h +3 -0
+     * is allowed to optimize a write zeroes request by unmapping (discarding)
+     * blocks if it is guaranteed that the result will read back as
+     * zeroes. the flag is only passed to the driver if the block device is

s/the flag/The flag/

include/block/block.h typedef struct BlockDevOps { · line 74

Quoted 3 lines
include/block/block.h +3 -0
+     * opened with BDRV_O_UNMAP.
+     */
+    BDRV_REQ_MAY_UNMAP    = 0x4,

As those are trivial, feel free to add:

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#5 lore raw

PATCH v2 4/20 block: introduce bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes

In reply to Peter Lieven
---
 block.c                   |   29 +++++++++++++++++++++++++++++
 include/block/block.h     |    2 ++
 include/block/block_int.h |   13 +++++++++++++
 3 files changed, 44 insertions(+)
block.c +29 -0
index 6f498fc..177720e 100644
@@ -3045,6 +3045,35 @@ int bdrv_has_zero_init(BlockDriverState *bs)
     return 0;
 }
 
+int bdrv_has_discard_zeroes(BlockDriverState *bs)
+{
+    assert(bs->drv);
+
+    if (bs->backing_hd) {
+        return 0;
+    }
+    if (bs->drv->bdrv_has_discard_zeroes) {
+        return bs->drv->bdrv_has_discard_zeroes(bs);
+    }
+
+    return 0;
+}
+
+int bdrv_has_discard_write_zeroes(BlockDriverState *bs)
+{
+    assert(bs->drv);
+
+    if (bs->backing_hd || !(bs->open_flags & BDRV_O_UNMAP)) {
+        return 0;
+    }
+
+    if (bs->drv->bdrv_has_discard_write_zeroes) {
+        return bs->drv->bdrv_has_discard_write_zeroes(bs);
+    }
+
+    return 0;
+}
+
 typedef struct BdrvCoGetBlockStatusData {
     BlockDriverState *bs;
     BlockDriverState *base;
include/block/block.h +2 -0
index 0328234..8aa325a 100644
@@ -307,6 +307,8 @@ int bdrv_discard(BlockDriverState *bs, int64_t sector_num, int nb_sectors);
 int bdrv_co_discard(BlockDriverState *bs, int64_t sector_num, int nb_sectors);
 int bdrv_has_zero_init_1(BlockDriverState *bs);
 int bdrv_has_zero_init(BlockDriverState *bs);
+int bdrv_has_discard_zeroes(BlockDriverState *bs);
+int bdrv_has_discard_write_zeroes(BlockDriverState *bs);
 int64_t bdrv_get_block_status(BlockDriverState *bs, int64_t sector_num,
                               int nb_sectors, int *pnum);
 int bdrv_is_allocated(BlockDriverState *bs, int64_t sector_num, int nb_sectors,
include/block/block_int.h +13 -0
index 26bad36..85c3474 100644
@@ -202,6 +202,19 @@ struct BlockDriver {
      */
     int (*bdrv_has_zero_init)(BlockDriverState *bs);
 
+    /*
+     * Returns 1 if discarded blocks read back as zeroes, 0 otherwise.
+     */
+    int (*bdrv_has_discard_zeroes)(BlockDriverState *bs);
+
+    /*
+     * Returns 1 if the driver can optimize writing zeroes by discarding
+     * sectors. It is additionally required that the block device is
+     * opened with BDRV_O_UNMAP and the that write zeroes request carries
+     * the BDRV_REQ_MAY_UNMAP flag for this to work.
+     */
+    int (*bdrv_has_discard_write_zeroes)(BlockDriverState *bs);
+
     QLIST_ENTRY(BlockDriver) list;
 };
 
Signature · 1 line
1.7.9.5
Reviewed-by on v2 4/20
#6 lore raw

Re: [Qemu-devel] [PATCHv2 04/20] block: introduce bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes

Quoted 6 lines · Peter Lieven
---
 block.c                   |   29 +++++++++++++++++++++++++++++
 include/block/block.h     |    2 ++
 include/block/block_int.h |   13 +++++++++++++
 3 files changed, 44 insertions(+)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#7 lore raw

PATCH v2 5/20 block/raw: add bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes

In reply to Peter Lieven
---
 block/raw_bsd.c |   56 +++++++++++++++++++++++++++++++++----------------------
 1 file changed, 34 insertions(+), 22 deletions(-)
block/raw_bsd.c +34 -22
index bd4811b..0bfa5fc 100644
@@ -131,6 +131,16 @@ static int raw_has_zero_init(BlockDriverState *bs)
     return bdrv_has_zero_init(bs->file);
 }
 
+static int raw_has_discard_zeroes(BlockDriverState *bs)
+{
+    return bdrv_has_discard_zeroes(bs->file);
+}
+
+static int raw_has_discard_write_zeroes(BlockDriverState *bs)
+{
+    return bdrv_has_discard_write_zeroes(bs->file);
+}
+
 static int raw_create(const char *filename, QEMUOptionParameter *options)
 {
     return bdrv_create_file(filename, options);
@@ -155,28 +165,30 @@ static int raw_probe(const uint8_t *buf, int buf_size, const char *filename)
 }
 
 static BlockDriver bdrv_raw = {
-    .format_name          = "raw",
-    .bdrv_probe           = &raw_probe,
-    .bdrv_reopen_prepare  = &raw_reopen_prepare,
-    .bdrv_open            = &raw_open,
-    .bdrv_close           = &raw_close,
-    .bdrv_create          = &raw_create,
-    .bdrv_co_readv        = &raw_co_readv,
-    .bdrv_co_writev       = &raw_co_writev,
-    .bdrv_co_write_zeroes = &raw_co_write_zeroes,
-    .bdrv_co_discard      = &raw_co_discard,
-    .bdrv_co_get_block_status = &raw_co_get_block_status,
-    .bdrv_truncate        = &raw_truncate,
-    .bdrv_getlength       = &raw_getlength,
-    .bdrv_get_info        = &raw_get_info,
-    .bdrv_is_inserted     = &raw_is_inserted,
-    .bdrv_media_changed   = &raw_media_changed,
-    .bdrv_eject           = &raw_eject,
-    .bdrv_lock_medium     = &raw_lock_medium,
-    .bdrv_ioctl           = &raw_ioctl,
-    .bdrv_aio_ioctl       = &raw_aio_ioctl,
-    .create_options       = &raw_create_options[0],
-    .bdrv_has_zero_init   = &raw_has_zero_init
+    .format_name                   = "raw",
+    .bdrv_probe                    = &raw_probe,
+    .bdrv_reopen_prepare           = &raw_reopen_prepare,
+    .bdrv_open                     = &raw_open,
+    .bdrv_close                    = &raw_close,
+    .bdrv_create                   = &raw_create,
+    .bdrv_co_readv                 = &raw_co_readv,
+    .bdrv_co_writev                = &raw_co_writev,
+    .bdrv_co_write_zeroes          = &raw_co_write_zeroes,
+    .bdrv_co_discard               = &raw_co_discard,
+    .bdrv_co_get_block_status      = &raw_co_get_block_status,
+    .bdrv_truncate                 = &raw_truncate,
+    .bdrv_getlength                = &raw_getlength,
+    .bdrv_get_info                 = &raw_get_info,
+    .bdrv_is_inserted              = &raw_is_inserted,
+    .bdrv_media_changed            = &raw_media_changed,
+    .bdrv_eject                    = &raw_eject,
+    .bdrv_lock_medium              = &raw_lock_medium,
+    .bdrv_ioctl                    = &raw_ioctl,
+    .bdrv_aio_ioctl                = &raw_aio_ioctl,
+    .create_options                = &raw_create_options[0],
+    .bdrv_has_zero_init            = &raw_has_zero_init,
+    .bdrv_has_discard_zeroes       = &raw_has_discard_zeroes,
+    .bdrv_has_discard_write_zeroes = &raw_has_discard_write_zeroes,
 };
 
 static void bdrv_raw_init(void)
Signature · 1 line
1.7.9.5
Reviewed-by on v2 5/20
#8 lore raw

Re: [Qemu-devel] [PATCHv2 05/20] block/raw: add bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes

Quoted 4 lines · Peter Lieven
---
 block/raw_bsd.c |   56 +++++++++++++++++++++++++++++++++----------------------
 1 file changed, 34 insertions(+), 22 deletions(-)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#9 lore raw

PATCH v2 6/20 block: add discard and write_zeroes limits and alignment to BlockDriverState

In reply to Peter Lieven
---
 include/block/block_int.h |   12 ++++++++++++
 1 file changed, 12 insertions(+)
include/block/block_int.h +12 -0
index 85c3474..692b9ed 100644
@@ -280,6 +280,18 @@ struct BlockDriverState {
     /* the memory alignment required for the buffers handled by this driver */
     int buffer_alignment;
 
+    /* maximum number of sectors that can be discarded at once */
+    int max_discard;
+
+    /* optimal alignment for discard requests in sectors */
+    int64_t discard_alignment;
+
+    /* maximum number of sectors that can zeroized at once */
+    int max_write_zeroes;
+
+    /* optimal alignment for write zeroes requests in sectors */
+    int64_t write_zeroes_alignment;
+
     /* do we need to tell the quest if we have a volatile write cache? */
     int enable_write_cache;
 
Signature · 1 line
1.7.9.5
Reviewed-by on v2 6/20
#10 lore raw

Re: [Qemu-devel] [PATCHv2 06/20] block: add discard and write_zeroes limits and alignment to BlockDriverState

Quoted 4 lines · Peter Lieven
---
 include/block/block_int.h |   12 ++++++++++++
 1 file changed, 12 insertions(+)

include/block/block_int.h struct BlockDriverState { · line 296

Quoted 6 lines
include/block/block_int.h +4 -0
+
+    /* optimal alignment for write zeroes requests in sectors */
+    int64_t write_zeroes_alignment;
+
     /* do we need to tell the quest if we have a volatile write cache? */
     int enable_write_cache;

Hmm, I just pointed out to Paolo that this ought to be bool. But as it is in the context and not your actual patch, it has no bearing on this series, and even if that gets changed first, git's pretty good about context-only conflict resolution.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#11 lore raw

Re: [Qemu-devel] [PATCHv2 06/20] block: add discard and write_zeroes limits and alignment to BlockDriverState

Am 19.09.2013 20:53, schrieb Eric Blake:

Quoted 11 lines · Peter Lieven
Quoted 4 lines
---
 include/block/block_int.h |   12 ++++++++++++
 1 file changed, 12 insertions(+)

include/block/block_int.h struct BlockDriverState { · line 294

Quoted 4 lines
include/block/block_int.h +4 -0
+
+    /* optimal alignment for write zeroes requests in sectors */
+    int64_t write_zeroes_alignment;
+

Paolo voted for a BlockLimits struct to be nested into the BlockDriverState. It would make it easier if the settings have to be copied (like in raw_open). There might be more limits in the future so it might be good that its not necessary to change the code everywhere.

Quoted 7 lines
Quoted 2 lines
     /* do we need to tell the quest if we have a volatile write cache? */
     int enable_write_cache;

Hmm, I just pointed out to Paolo that this ought to be bool. But as it is in the context and not your actual patch, it has no bearing on this series, and even if that gets changed first, git's pretty good about context-only conflict resolution.

If I put a v3 which is likely I can put a patch for that in as well.

Peter

#12 lore raw

PATCH v2 7/20 block: honour alignment and limit in bdrv_co_do_write_zeroes

In reply to Peter Lieven
---
 block.c |   57 +++++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 41 insertions(+), 16 deletions(-)
block.c +41 -16
index 177720e..74ec342 100644
@@ -2660,28 +2660,53 @@ static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
     BlockDriver *drv = bs->drv;
     QEMUIOVector qiov;
     struct iovec iov;
-    int ret;
+    int ret = 0;
 
-    /* TODO Emulate only part of misaligned requests instead of letting block
-     * drivers return -ENOTSUP and emulate everything */
+    /* if no limit is specified in the BlockDriverState use a default
+     * of 32768 512-byte sectors (16 MiB) per request.
+     */
+    int max_write_zeroes = bs->max_write_zeroes ? bs->max_write_zeroes : 32768;
 
-    /* First try the efficient write zeroes operation */
-    if (drv->bdrv_co_write_zeroes) {
-        ret = drv->bdrv_co_write_zeroes(bs, sector_num, nb_sectors, flags);
-        if (ret != -ENOTSUP) {
-            return ret;
+    while (nb_sectors > 0 && !ret) {
+        int num = nb_sectors;
+
+        /* align request */
+        if (bs->write_zeroes_alignment &&
+            num >= bs->write_zeroes_alignment &&
+            sector_num % bs->write_zeroes_alignment) {
+            if (num > bs->write_zeroes_alignment) {
+                num = bs->write_zeroes_alignment;
+            }
+            num -= sector_num % bs->write_zeroes_alignment;
         }
-    }
 
-    /* Fall back to bounce buffer if write zeroes is unsupported */
-    iov.iov_len  = nb_sectors * BDRV_SECTOR_SIZE;
-    iov.iov_base = qemu_blockalign(bs, iov.iov_len);
-    memset(iov.iov_base, 0, iov.iov_len);
-    qemu_iovec_init_external(&qiov, &iov, 1);
+        /* limit request size */
+        if (num > max_write_zeroes) {
+            num = max_write_zeroes;
+        }
 
-    ret = drv->bdrv_co_writev(bs, sector_num, nb_sectors, &qiov);
+        ret = -ENOTSUP;
+        /* First try the efficient write zeroes operation */
+        if (drv->bdrv_co_write_zeroes) {
+            ret = drv->bdrv_co_write_zeroes(bs, sector_num, num, flags);
+        }
+
+        if (ret == -ENOTSUP) {
+            /* Fall back to bounce buffer if write zeroes is unsupported */
+            iov.iov_len  = nb_sectors * BDRV_SECTOR_SIZE;
+            iov.iov_base = qemu_blockalign(bs, iov.iov_len);
+            memset(iov.iov_base, 0, iov.iov_len);
+            qemu_iovec_init_external(&qiov, &iov, 1);
+
+            ret = drv->bdrv_co_writev(bs, sector_num, num, &qiov);
+
+            qemu_vfree(iov.iov_base);
+        }
+
+        sector_num += num;
+        nb_sectors -= num;
+    }
 
-    qemu_vfree(iov.iov_base);
     return ret;
 }
 
Signature · 1 line
1.7.9.5
#13 lore raw

Re: [Qemu-devel] [PATCHv2 07/20] block: honour alignment and limit in bdrv_co_do_write_zeroes

Quoted 12 lines · Peter Lieven
---
 block.c |   57 +++++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 41 insertions(+), 16 deletions(-)
block.c +0 -0
index 177720e..74ec342 100644
@@ -2660,28 +2660,53 @@ static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
     BlockDriver *drv = bs->drv;
     QEMUIOVector qiov;

block.c static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs, · line 2668

Quoted 10 lines
block.c +5 -3
     struct iovec iov;
-    int ret;
+    int ret = 0;
 
-    /* TODO Emulate only part of misaligned requests instead of letting block
-     * drivers return -ENOTSUP and emulate everything */
+    /* if no limit is specified in the BlockDriverState use a default
+     * of 32768 512-byte sectors (16 MiB) per request.
+     */
+    int max_write_zeroes = bs->max_write_zeroes ? bs->max_write_zeroes : 32768;

Worth having a named constant instead of a magic number?

block.c static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs, · line 2698

Quoted 8 lines
block.c +7 -0
+    while (nb_sectors > 0 && !ret) {
 
+
+        if (ret == -ENOTSUP) {
+            /* Fall back to bounce buffer if write zeroes is unsupported */
+            iov.iov_len  = nb_sectors * BDRV_SECTOR_SIZE;
+            iov.iov_base = qemu_blockalign(bs, iov.iov_len);
+            memset(iov.iov_base, 0, iov.iov_len);

This allocates, clears, and frees iov.iov_len bytes of 0 every iteration through the loop. Can you hoist that so you only allocate the bounce buffer once, and then clean it up after the loop completes?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#14 lore raw

Re: [Qemu-devel] [PATCHv2 07/20] block: honour alignment and limit in bdrv_co_do_write_zeroes

Am 19.09.2013 um 22:07 schrieb Eric Blake <[email protected]>:

Quoted 17 lines · Peter Lieven
Quoted 16 lines

---
block.c | 57 +++++++++++++++++++++++++++++++++++++++++----------------

1 file changed, 41 insertions(+), 16 deletions(-)
block.c +1 -1
index 177720e..74ec342 100644
@@ -2660,28 +2660,53 @@ static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
    BlockDriver *drv = bs->drv;
    QEMUIOVector qiov;
    struct iovec iov;
-    int ret;
+    int ret = 0;
 
Quoted 8 lines

block.c static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs, · line 2668

Quoted 6 lines
block.c +4 -2
-    /* TODO Emulate only part of misaligned requests instead of letting block
-     * drivers return -ENOTSUP and emulate everything */
+    /* if no limit is specified in the BlockDriverState use a default
+     * of 32768 512-byte sectors (16 MiB) per request.
+     */
+    int max_write_zeroes = bs->max_write_zeroes ? bs->max_write_zeroes : 32768;

Worth having a named constant instead of a magic number?

Its the only place where it is used. I can do that, but I would keep it in front of the function rather than at the top of block.h or even in block_int.h

Quoted 10 lines

block.c static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs, · line 2698

Quoted 9 lines
block.c +7 -0
+    while (nb_sectors > 0 && !ret) {
 
+
+        if (ret == -ENOTSUP) {
+            /* Fall back to bounce buffer if write zeroes is unsupported */
+            iov.iov_len  = nb_sectors * BDRV_SECTOR_SIZE;
+            iov.iov_base = qemu_blockalign(bs, iov.iov_len);
+            memset(iov.iov_base, 0, iov.iov_len);
Quoted 3 lines

This allocates, clears, and frees iov.iov_len bytes of 0 every iteration through the loop. Can you hoist that so you only allocate the bounce buffer once, and then clean it up after the loop completes?

I can do that, but in this case, I have to allocate max_write_zeroes * BDRV_SECTOR_SIZE bytes because the iov_len might not be maximal in the first iteration due to the alignment logic.

Peter

#15 lore raw

PATCH v2 8/20 block: honour alignment and limit in bdrv_co_discard

In reply to Peter Lieven
---
 block.c |   34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)
block.c +33 -1
index 74ec342..ecc5be4 100644
@@ -4181,7 +4181,39 @@ int coroutine_fn bdrv_co_discard(BlockDriverState *bs, int64_t sector_num,
     }
 
     if (bs->drv->bdrv_co_discard) {
-        return bs->drv->bdrv_co_discard(bs, sector_num, nb_sectors);
+        /* if no limit is specified in the BlockDriverState use a default
+         * of 32768 512-byte sectors (16 MiB) per request.
+         */
+        int max_discard = bs->max_discard ? bs->max_discard : 32768;
+
+        while (nb_sectors > 0) {
+            int ret;
+            int num = nb_sectors;
+
+            /* align request */
+            if (bs->discard_alignment &&
+                num >= bs->discard_alignment &&
+                sector_num % bs->discard_alignment) {
+                if (num > bs->discard_alignment) {
+                    num = bs->discard_alignment;
+                }
+                num -= sector_num % bs->discard_alignment;
+            }
+
+            /* limit request size */
+            if (num > max_discard) {
+                num = max_discard;
+            }
+
+            ret = bs->drv->bdrv_co_discard(bs, sector_num, num);
+            if (ret) {
+                return ret;
+            }
+
+            sector_num += num;
+            nb_sectors -= num;
+        }
+        return 0;
     } else if (bs->drv->bdrv_aio_discard) {
         BlockDriverAIOCB *acb;
         CoroutineIOCompletion co = {
Signature · 1 line
1.7.9.5
#16 lore raw

Re: [Qemu-devel] [PATCHv2 08/20] block: honour alignment and limit in bdrv_co_discard

Quoted 10 lines · Peter Lieven
---
 block.c |   34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)
block.c +0 -0
index 74ec342..ecc5be4 100644

block.c int coroutine_fn bdrv_co_discard(BlockDriverState *bs, int64_t sector_num, · line 4187

Quoted 8 lines
block.c +4 -1
@@ -4181,7 +4181,39 @@ int coroutine_fn bdrv_co_discard(BlockDriverState *bs, int64_t sector_num,
     }
 
     if (bs->drv->bdrv_co_discard) {
-        return bs->drv->bdrv_co_discard(bs, sector_num, nb_sectors);
+        /* if no limit is specified in the BlockDriverState use a default
+         * of 32768 512-byte sectors (16 MiB) per request.
+         */
+        int max_discard = bs->max_discard ? bs->max_discard : 32768;

Worth having a named constant instead of a magic number?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#17 lore raw

PATCH v2 9/20 iscsi: simplify iscsi_co_discard

In reply to Peter Lieven

now that bdrv_co_discard can handle limits we do not need the request split logic here anymore.

---
 block/iscsi.c |   67 +++++++++++++++++++++------------------------------------
 1 file changed, 25 insertions(+), 42 deletions(-)
block/iscsi.c +25 -42
index 68f99d3..aabcddb 100644
@@ -87,7 +87,6 @@ typedef struct IscsiAIOCB {
 #define NOP_INTERVAL 5000
 #define MAX_NOP_FAILURES 3
 #define ISCSI_CMD_RETRIES 5
-#define ISCSI_MAX_UNMAP 131072
 
 static void
 iscsi_bh_cb(void *p)
@@ -908,8 +907,6 @@ coroutine_fn iscsi_co_discard(BlockDriverState *bs, int64_t sector_num,
     IscsiLun *iscsilun = bs->opaque;
     struct IscsiTask iTask;
     struct unmap_list list;
-    uint32_t nb_blocks;
-    uint32_t max_unmap;
 
     if (!is_request_lun_aligned(sector_num, nb_sectors, iscsilun)) {
         return -EINVAL;
@@ -921,52 +918,38 @@ coroutine_fn iscsi_co_discard(BlockDriverState *bs, int64_t sector_num,
     }
 
     list.lba = sector_qemu2lun(sector_num, iscsilun);
-    nb_blocks = sector_qemu2lun(nb_sectors, iscsilun);
+    list.num = sector_qemu2lun(nb_sectors, iscsilun);
 
-    max_unmap = iscsilun->bl.max_unmap;
-    if (max_unmap == 0xffffffff) {
-        max_unmap = ISCSI_MAX_UNMAP;
-    }
-
-    while (nb_blocks > 0) {
-        iscsi_co_init_iscsitask(iscsilun, &iTask);
-        list.num = nb_blocks;
-        if (list.num > max_unmap) {
-            list.num = max_unmap;
-        }
+    iscsi_co_init_iscsitask(iscsilun, &iTask);
 retry:
-        if (iscsi_unmap_task(iscsilun->iscsi, iscsilun->lun, 0, 0, &list, 1,
-                         iscsi_co_generic_cb, &iTask) == NULL) {
-            return -EIO;
-        }
-
-        while (!iTask.complete) {
-            iscsi_set_events(iscsilun);
-            qemu_coroutine_yield();
-        }
+    if (iscsi_unmap_task(iscsilun->iscsi, iscsilun->lun, 0, 0, &list, 1,
+                     iscsi_co_generic_cb, &iTask) == NULL) {
+        return -EIO;
+    }
 
-        if (iTask.task != NULL) {
-            scsi_free_scsi_task(iTask.task);
-            iTask.task = NULL;
-        }
+    while (!iTask.complete) {
+        iscsi_set_events(iscsilun);
+        qemu_coroutine_yield();
+    }
 
-        if (iTask.do_retry) {
-            goto retry;
-        }
+    if (iTask.task != NULL) {
+        scsi_free_scsi_task(iTask.task);
+        iTask.task = NULL;
+    }
 
-        if (iTask.status == SCSI_STATUS_CHECK_CONDITION) {
-            /* the target might fail with a check condition if it
-               is not happy with the alignment of the UNMAP request
-               we silently fail in this case */
-            return 0;
-        }
+    if (iTask.do_retry) {
+        goto retry;
+    }
 
-        if (iTask.status != SCSI_STATUS_GOOD) {
-            return -EIO;
-        }
+    if (iTask.status == SCSI_STATUS_CHECK_CONDITION) {
+        /* the target might fail with a check condition if it
+           is not happy with the alignment of the UNMAP request
+           we silently fail in this case */
+        return 0;
+    }
 
-        list.lba += list.num;
-        nb_blocks -= list.num;
+    if (iTask.status != SCSI_STATUS_GOOD) {
+        return -EIO;
     }
 
     return 0;
Signature · 1 line
1.7.9.5
Reviewed-by on v2 9/20
#18 lore raw

Re: [Qemu-devel] [PATCHv2 09/20] iscsi: simplify iscsi_co_discard

Quoted 8 lines · Peter Lieven

now that bdrv_co_discard can handle limits we do not need the request split logic here anymore.

---
 block/iscsi.c |   67 +++++++++++++++++++++------------------------------------
 1 file changed, 25 insertions(+), 42 deletions(-)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#19 lore raw

PATCH v2 10/20 iscsi: set limits in BlockDriverState

In reply to Peter Lieven
---
 block/iscsi.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
block/iscsi.c +10 -0
index aabcddb..21b1ecf 100644
@@ -1362,6 +1362,16 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags)
                sizeof(struct scsi_inquiry_block_limits));
         scsi_free_scsi_task(task);
         task = NULL;
+        
+        if (iscsilun->bl.max_unmap < 0xffffffff) {
+            bs->max_discard = sector_lun2qemu(iscsilun->bl.max_unmap, iscsilun);
+        }
+        bs->discard_alignment = sector_lun2qemu(iscsilun->bl.opt_unmap_gran, iscsilun);
+        
+        if (iscsilun->bl.max_ws_len < 0xffffffff) {
+            bs->max_write_zeroes = sector_lun2qemu(iscsilun->bl.max_ws_len, iscsilun);
+        }
+        bs->write_zeroes_alignment = sector_lun2qemu(iscsilun->bl.opt_unmap_gran, iscsilun);
     }
 
 #if defined(LIBISCSI_FEATURE_NOP_COUNTER)
Signature · 1 line
1.7.9.5
Reviewed-by on v2 10/20
#20 lore raw

Re: [Qemu-devel] [PATCHv2 10/20] iscsi: set limits in BlockDriverState

Quoted 4 lines · Peter Lieven
---
 block/iscsi.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
Quoted 23 lines
block/iscsi.c +10 -0
index aabcddb..21b1ecf 100644
@@ -1362,6 +1362,16 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags)
                sizeof(struct scsi_inquiry_block_limits));
         scsi_free_scsi_task(task);
         task = NULL;
+        
+        if (iscsilun->bl.max_unmap < 0xffffffff) {
+            bs->max_discard = sector_lun2qemu(iscsilun->bl.max_unmap, iscsilun);
+        }
+        bs->discard_alignment = sector_lun2qemu(iscsilun->bl.opt_unmap_gran, iscsilun);
+        
+        if (iscsilun->bl.max_ws_len < 0xffffffff) {
+            bs->max_write_zeroes = sector_lun2qemu(iscsilun->bl.max_ws_len, iscsilun);
+        }
+        bs->write_zeroes_alignment = sector_lun2qemu(iscsilun->bl.opt_unmap_gran, iscsilun);
     }
 
 #if defined(LIBISCSI_FEATURE_NOP_COUNTER)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#21 lore raw

PATCH v2 11/20 iscsi: add bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes

In reply to Peter Lieven
---
 block/iscsi.c |   16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
block/iscsi.c +15 -1
index 21b1ecf..46c7c8d 100644
@@ -1440,6 +1440,18 @@ static int iscsi_has_zero_init(BlockDriverState *bs)
     return 0;
 }
 
+static int iscsi_has_discard_zeroes(BlockDriverState *bs)
+{
+    IscsiLun *iscsilun = bs->opaque;
+    return iscsilun->lbprz;
+}
+
+static int iscsi_has_discard_write_zeroes(BlockDriverState *bs)
+{
+    IscsiLun *iscsilun = bs->opaque;
+    return iscsilun->lbprz && iscsilun->lbp.lbpws;
+}
+
 static int iscsi_create(const char *filename, QEMUOptionParameter *options)
 {
     int ret = 0;
@@ -1522,7 +1534,9 @@ static BlockDriver bdrv_iscsi = {
     .bdrv_aio_writev = iscsi_aio_writev,
     .bdrv_aio_flush  = iscsi_aio_flush,
 
-    .bdrv_has_zero_init = iscsi_has_zero_init,
+    .bdrv_has_zero_init            = iscsi_has_zero_init,
+    .bdrv_has_discard_zeroes       = iscsi_has_discard_zeroes ,
+    .bdrv_has_discard_write_zeroes = iscsi_has_discard_write_zeroes,
 
 #ifdef __linux__
     .bdrv_ioctl       = iscsi_ioctl,
Signature · 1 line
1.7.9.5
Reviewed-by on v2 11/20
#22 lore raw

Re: [Qemu-devel] [PATCHv2 11/20] iscsi: add bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes

block/iscsi.c static BlockDriver bdrv_iscsi = { · line 1538

Quoted 13 lines · Peter Lieven
---
 block/iscsi.c |   16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
block/iscsi.c +2 -1
@@ -1522,7 +1534,9 @@ static BlockDriver bdrv_iscsi = {
     .bdrv_aio_writev = iscsi_aio_writev,
     .bdrv_aio_flush  = iscsi_aio_flush,
 
-    .bdrv_has_zero_init = iscsi_has_zero_init,
+    .bdrv_has_zero_init            = iscsi_has_zero_init,
+    .bdrv_has_discard_zeroes       = iscsi_has_discard_zeroes ,

Spurious space before ','. As that's trivial to fix, feel free to add:

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#23 lore raw

PATCH v2 12/20 iscsi: add bdrv_co_write_zeroes

In reply to Peter Lieven
---
 block/iscsi.c |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
block/iscsi.c +59 -0
index 46c7c8d..4df33fc 100644
@@ -56,6 +56,7 @@ typedef struct IscsiLun {
     uint8_t lbprz;
     struct scsi_inquiry_logical_block_provisioning lbp;
     struct scsi_inquiry_block_limits bl;
+    unsigned char *zeroblock;
 } IscsiLun;
 
 typedef struct IscsiTask {
@@ -955,6 +956,62 @@ retry:
     return 0;
 }
 
+
+static int
+coroutine_fn iscsi_co_write_zeroes(BlockDriverState *bs, int64_t sector_num,
+                                   int nb_sectors, BdrvRequestFlags flags)
+{
+    IscsiLun *iscsilun = bs->opaque;
+    struct IscsiTask iTask;
+    uint64_t lba;
+    uint32_t nb_blocks;
+
+    if (!is_request_lun_aligned(sector_num, nb_sectors, iscsilun)) {
+        return -EINVAL;
+    }
+
+    if (!iscsilun->lbp.lbpws) {
+        /* WRITE SAME is not supported by the target */
+        return -ENOTSUP;
+    }
+
+    lba = sector_qemu2lun(sector_num, iscsilun);
+    nb_blocks = sector_qemu2lun(nb_sectors, iscsilun);
+
+    if (iscsilun->zeroblock == NULL) {
+        iscsilun->zeroblock = g_malloc0(iscsilun->block_size);
+    }
+
+    iscsi_co_init_iscsitask(iscsilun, &iTask);
+retry:
+    if (iscsi_writesame16_task(iscsilun->iscsi, iscsilun->lun, lba,
+                               iscsilun->zeroblock, iscsilun->block_size,
+                               nb_blocks, 0, !!(flags & BDRV_REQ_MAY_UNMAP),
+                               0, 0, iscsi_co_generic_cb, &iTask) == NULL) {
+        return -EIO;
+    }
+
+    while (!iTask.complete) {
+        iscsi_set_events(iscsilun);
+        qemu_coroutine_yield();
+    }
+
+    if (iTask.task != NULL) {
+        scsi_free_scsi_task(iTask.task);
+        iTask.task = NULL;
+    }
+
+    if (iTask.do_retry) {
+        goto retry;
+    }
+
+    if (iTask.status != SCSI_STATUS_GOOD) {
+        return -EIO;
+    }
+
+    return 0;
+}
+
 static int parse_chap(struct iscsi_context *iscsi, const char *target)
 {
     QemuOptsList *list;
@@ -1412,6 +1469,7 @@ static void iscsi_close(BlockDriverState *bs)
     }
     qemu_aio_set_fd_handler(iscsi_get_fd(iscsi), NULL, NULL, NULL);
     iscsi_destroy_context(iscsi);
+    g_free(iscsilun->zeroblock);
     memset(iscsilun, 0, sizeof(IscsiLun));
 }
 
@@ -1529,6 +1587,7 @@ static BlockDriver bdrv_iscsi = {
 
     .bdrv_co_get_block_status = iscsi_co_get_block_status,
     .bdrv_co_discard      = iscsi_co_discard,
+    .bdrv_co_write_zeroes = iscsi_co_write_zeroes,
 
     .bdrv_aio_readv  = iscsi_aio_readv,
     .bdrv_aio_writev = iscsi_aio_writev,
Signature · 1 line
1.7.9.5
Reviewed-by on v2 12/20
#24 lore raw

Re: [Qemu-devel] [PATCHv2 12/20] iscsi: add bdrv_co_write_zeroes

Quoted 5 lines · Peter Lieven
---
 block/iscsi.c |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#25 lore raw

PATCH v2 13/20 block: introduce bdrv_zeroize

In reply to Peter Lieven

this patch adds a call to completely zero out a block device. the operation is sped up by checking the block status and only writing zeroes to the device if they currently do not return zeroes. optionally the zero writing can be sped up by setting the flag BDRV_REQ_MAY_UNMAP to emulate the zero write by unmapping if the driver supports it.

---
 block.c               |   30 ++++++++++++++++++++++++++++++
 include/block/block.h |    1 +
 2 files changed, 31 insertions(+)
block.c +30 -0
index ecc5be4..88b137c 100644
@@ -2342,6 +2342,36 @@ int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num,
                       BDRV_REQ_ZERO_WRITE | flags);
 }
 
+int bdrv_zeroize(BlockDriverState *bs, BdrvRequestFlags flags)
+{
+    int64_t target_size = bdrv_getlength(bs) / BDRV_SECTOR_SIZE;
+    int64_t ret, nb_sectors, sector_num = 0;
+    int n;
+    /* split the write requests into 1MB chunks if the driver
+     * does not return a maximal size via bdi */
+    for (;;) {
+        nb_sectors = target_size - sector_num;
+        if (nb_sectors <= 0) {
+            return 0;
+        }
+        if (nb_sectors > INT_MAX) {
+            nb_sectors = INT_MAX;
+        }
+        ret = bdrv_get_block_status(bs, sector_num, nb_sectors, &n);
+        if (ret & BDRV_BLOCK_ZERO) {
+            sector_num += n;
+            continue;
+        }
+        ret = bdrv_write_zeroes(bs, sector_num, n, flags & BDRV_REQ_MAY_UNMAP);
+        if (ret < 0) {
+            error_report("error writing zeroes at sector %" PRId64 ": %s",
+                         sector_num, strerror(-ret));
+            return -EIO;
+        }
+        sector_num += n;
+    }
+}
+
 int bdrv_pread(BlockDriverState *bs, int64_t offset,
                void *buf, int count1)
 {
include/block/block.h +1 -0
index 8aa325a..a45166e 100644
@@ -195,6 +195,7 @@ int bdrv_write(BlockDriverState *bs, int64_t sector_num,
                const uint8_t *buf, int nb_sectors);
 int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num,
                int nb_sectors, BdrvRequestFlags flags);
+int bdrv_zeroize(BlockDriverState *bs, BdrvRequestFlags flags);
 int bdrv_writev(BlockDriverState *bs, int64_t sector_num, QEMUIOVector *qiov);
 int bdrv_pread(BlockDriverState *bs, int64_t offset,
                void *buf, int count);
Signature · 1 line
1.7.9.5
#26 lore raw

Re: [Qemu-devel] [PATCHv2 13/20] block: introduce bdrv_zeroize

Quoted 18 lines · Peter Lieven

this patch adds a call to completely zero out a block device. the operation is sped up by checking the block status and only writing zeroes to the device if they currently do not return zeroes. optionally the zero writing can be sped up by setting the flag BDRV_REQ_MAY_UNMAP to emulate the zero write by unmapping if the driver supports it.

---
 block.c               |   30 ++++++++++++++++++++++++++++++
 include/block/block.h |    1 +
 2 files changed, 31 insertions(+)
block.c +0 -0
index ecc5be4..88b137c 100644

block.c int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num, · line 2345

Quoted 4 lines
block.c +1 -0
@@ -2342,6 +2342,36 @@ int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num,
                       BDRV_REQ_ZERO_WRITE | flags);
 }
 
+int bdrv_zeroize(BlockDriverState *bs, BdrvRequestFlags flags)

Please add documentation in the code base about what this function does, and what return values mean. (Bad practice in the past doesn't excuse new patches from being more maintainer-friendly)

Quoted 10 lines
block.c +10 -0
+{
+    int64_t target_size = bdrv_getlength(bs) / BDRV_SECTOR_SIZE;
+    int64_t ret, nb_sectors, sector_num = 0;
+    int n;
+    /* split the write requests into 1MB chunks if the driver
+     * does not return a maximal size via bdi */
+    for (;;) {
+        nb_sectors = target_size - sector_num;
+        if (nb_sectors <= 0) {
+            return 0;

block.c int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num, · line 2365

Quoted 10 lines
block.c +10 -0
+        }
+        if (nb_sectors > INT_MAX) {
+            nb_sectors = INT_MAX;
+        }
+        ret = bdrv_get_block_status(bs, sector_num, nb_sectors, &n);
+        if (ret & BDRV_BLOCK_ZERO) {
+            sector_num += n;
+            continue;
+        }
+        ret = bdrv_write_zeroes(bs, sector_num, n, flags & BDRV_REQ_MAY_UNMAP);

Is this intentionally throwing away all other flags except BDRV_REQ_MAY_UNMAP?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#27 lore raw

Re: [Qemu-devel] [PATCHv2 13/20] block: introduce bdrv_zeroize

Am 19.09.2013 um 22:26 schrieb Eric Blake <[email protected]>:

Quoted 22 lines · Peter Lieven
Quoted 21 lines

this patch adds a call to completely zero out a block device. the operation is sped up by checking the block status and only writing zeroes to the device if they currently do not return zeroes. optionally the zero writing can be sped up by setting the flag BDRV_REQ_MAY_UNMAP to emulate the zero write by unmapping if the driver supports it.

---
block.c | 30 ++++++++++++++++++++++++++++++
include/block/block.h | 1 +

2 files changed, 31 insertions(+)
block.c +0 -0
index ecc5be4..88b137c 100644
@@ -2342,6 +2342,36 @@ int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num,
                      BDRV_REQ_ZERO_WRITE | flags);
 }
 
Quoted 5 lines

block.c int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num, · line 2345

Quoted 1 line
block.c +1 -0
+int bdrv_zeroize(BlockDriverState *bs, BdrvRequestFlags flags)

Please add documentation in the code base about what this function does, and what return values mean. (Bad practice in the past doesn't excuse new patches from being more maintainer-friendly)

ok ;-)

Quoted 22 lines
Quoted 11 lines
block.c +10 -0
+{
+    int64_t target_size = bdrv_getlength(bs) / BDRV_SECTOR_SIZE;
+    int64_t ret, nb_sectors, sector_num = 0;
+    int n;
+    /* split the write requests into 1MB chunks if the driver
+     * does not return a maximal size via bdi */
+    for (;;) {
+        nb_sectors = target_size - sector_num;
+        if (nb_sectors <= 0) {
+            return 0;

block.c int bdrv_write_zeroes(BlockDriverState *bs, int64_t sector_num, · line 2365

Quoted 10 lines
block.c +10 -0
+        }
+        if (nb_sectors > INT_MAX) {
+            nb_sectors = INT_MAX;
+        }
+        ret = bdrv_get_block_status(bs, sector_num, nb_sectors, &n);
+        if (ret & BDRV_BLOCK_ZERO) {
+            sector_num += n;
+            continue;
+        }
+        ret = bdrv_write_zeroes(bs, sector_num, n, flags & BDRV_REQ_MAY_UNMAP);
Quoted 2 lines

Is this intentionally throwing away all other flags except BDRV_REQ_MAY_UNMAP?

This is the only option that bdrv_write_zeroes currently expects, but as this might change some day I don't mind to pass all flags.

Peter

#28 lore raw

PATCH v2 14/20 block/get_block_status: set *pnum = 0 on error

In reply to Peter Lieven

if the call is invoked through bdrv_is_allocated the caller might expect *pnum = 0 on error. however, a new implementation of bdrv_get_block_status might only return a negative exit value on error while keeping *pnum untouched.

---
 block.c |    1 +
 1 file changed, 1 insertion(+)
block.c +1 -0
index 88b137c..805ee26 100644
@@ -3188,6 +3188,7 @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs,
 
     ret = bs->drv->bdrv_co_get_block_status(bs, sector_num, nb_sectors, pnum);
     if (ret < 0) {
+        *pnum = 0;
         return ret;
     }
 
Signature · 1 line
1.7.9.5
#29 lore raw

PATCH v2 15/20 block/get_block_status: avoid segfault if there is no backing_hd

In reply to Peter Lieven
---
 block.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block.c +1 -1
index 805ee26..461a1d7 100644
@@ -3195,7 +3195,7 @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs,
     if (!(ret & BDRV_BLOCK_DATA)) {
         if (bdrv_has_zero_init(bs)) {
             ret |= BDRV_BLOCK_ZERO;
-        } else {
+        } else if (bs->backing_hd) {
             BlockDriverState *bs2 = bs->backing_hd;
             int64_t length2 = bdrv_getlength(bs2);
             if (length2 >= 0 && sector_num >= (length2 >> BDRV_SECTOR_BITS)) {
Signature · 1 line
1.7.9.5
#30 lore raw

PATCH v2 16/20 block/get_block_status: avoid redundant callouts on raw devices

In reply to Peter Lieven

if a raw device like an iscsi target or host device is used the current implementation makes a second call out to get the block status of bs->file. however, the raw driver already has called bdrv_get_block_status on bs->file.

---
 block.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
block.c +2 -1
index 461a1d7..64fca1b 100644
@@ -3206,7 +3206,8 @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs,
 
     if (bs->file &&
         (ret & BDRV_BLOCK_DATA) && !(ret & BDRV_BLOCK_ZERO) &&
-        (ret & BDRV_BLOCK_OFFSET_VALID)) {
+        (ret & BDRV_BLOCK_OFFSET_VALID) &&
+        strcmp(bs->drv->format_name, "raw")) {
         ret2 = bdrv_co_get_block_status(bs->file, ret >> BDRV_SECTOR_BITS,
                                         *pnum, pnum);
         if (ret2 >= 0) {
Signature · 1 line
1.7.9.5
#31 lore raw

PATCH v2 17/20 block/get_block_status: fix BDRV_BLOCK_ZERO for unallocated blocks

In reply to Peter Lieven

this patch does 2 things:
a) only do additional call outs if BDRV_BLOCK_ZERO is not already set.
b) use the newly introduced bdrv_has_discard_zeroes() to return the zero state of an unallocated block. the used callout to bdrv_has_zero_init() is only valid right after bdrv_create.

---
 block.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
block.c +2 -2
index 64fca1b..7d7ee0f 100644
@@ -3192,8 +3192,8 @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs,
         return ret;
     }
 
-    if (!(ret & BDRV_BLOCK_DATA)) {
-        if (bdrv_has_zero_init(bs)) {
+    if (!(ret & BDRV_BLOCK_DATA) && !(ret & BDRV_BLOCK_ZERO)) { 
+        if (bdrv_has_discard_zeroes(bs)) {
             ret |= BDRV_BLOCK_ZERO;
         } else if (bs->backing_hd) {
             BlockDriverState *bs2 = bs->backing_hd;
Signature · 1 line
1.7.9.5
Reviewed-by on v2 17/20
#32 lore raw

Re: [Qemu-devel] [PATCHv2 17/20] block/get_block_status: fix BDRV_BLOCK_ZERO for unallocated blocks

Quoted 10 lines · Peter Lieven

this patch does 2 things:
a) only do additional call outs if BDRV_BLOCK_ZERO is not already set.
b) use the newly introduced bdrv_has_discard_zeroes() to return the zero state of an unallocated block. the used callout to bdrv_has_zero_init() is only valid right after bdrv_create.

---
 block.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 17 lines
block.c +2 -2
index 64fca1b..7d7ee0f 100644
@@ -3192,8 +3192,8 @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs,
         return ret;
     }
 
-    if (!(ret & BDRV_BLOCK_DATA)) {
-        if (bdrv_has_zero_init(bs)) {
+    if (!(ret & BDRV_BLOCK_DATA) && !(ret & BDRV_BLOCK_ZERO)) { 
+        if (bdrv_has_discard_zeroes(bs)) {
             ret |= BDRV_BLOCK_ZERO;
         } else if (bs->backing_hd) {
             BlockDriverState *bs2 = bs->backing_hd;
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#33 lore raw

PATCH v2 18/20 qemu-img: add support for fully allocated images

In reply to Peter Lieven
---
 qemu-img.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
qemu-img.c +5 -3
index 3e5e388..7600b58 100644
@@ -100,8 +100,10 @@ static void help(void)
            "  '-h' with or without a command shows this help and lists the supported formats\n"
            "  '-p' show progress of command (only certain commands)\n"
            "  '-q' use Quiet mode - do not print any output (except errors)\n"
-           "  '-S' indicates the consecutive number of bytes that must contain only zeros\n"
-           "       for qemu-img to create a sparse image during conversion\n"
+           "  '-S' indicates the consecutive number of bytes (defaults to 4096) that must\n"
+           "       contain only zeros for qemu-img to create a sparse image during\n"
+           "       conversion. if the number of bytes is 0 sparse files are disabled and\n"
+           "       images will always be fully allocated\n"
            "  '--output' takes the format in which the output must be done (human or json)\n"
            "  '-n' skips the target volume creation (useful if the volume is created\n"
            "       prior to running qemu-img)\n"
@@ -1468,7 +1470,7 @@ static int img_convert(int argc, char **argv)
         /* signal EOF to align */
         bdrv_write_compressed(out_bs, 0, NULL, 0);
     } else {
-        int has_zero_init = bdrv_has_zero_init(out_bs);
+        int has_zero_init = min_sparse ? bdrv_has_zero_init(out_bs) : 0;
 
         sector_num = 0; // total number of sectors converted so far
         nb_sectors = total_sectors - sector_num;
Signature · 1 line
1.7.9.5
#34 lore raw

Re: [Qemu-devel] [PATCHv2 18/20] qemu-img: add support for fully allocated images

Quoted 10 lines · Peter Lieven
---
 qemu-img.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
qemu-img.c +0 -0
index 3e5e388..7600b58 100644

qemu-img.c static void help(void) · line 105

Quoted 8 lines
qemu-img.c +3 -2
@@ -100,8 +100,10 @@ static void help(void)
            "  '-h' with or without a command shows this help and lists the supported formats\n"
            "  '-p' show progress of command (only certain commands)\n"
            "  '-q' use Quiet mode - do not print any output (except errors)\n"
-           "  '-S' indicates the consecutive number of bytes that must contain only zeros\n"
-           "       for qemu-img to create a sparse image during conversion\n"
+           "  '-S' indicates the consecutive number of bytes (defaults to 4096) that must\n"
+           "       contain only zeros for qemu-img to create a sparse image during\n"
+           "       conversion. if the number of bytes is 0 sparse files are disabled and\n"

s/if/If/
s/0/0,/

qemu-img.c static void help(void) · line 106

Quoted 1 line
qemu-img.c +1 -0
+           "       images will always be fully allocated\n"

IIRC correctly, on NTFS file system, the minimum hole size for a sparse file is 64k. Can you please make this option accept a suffixed number (-S 64k) instead of forcing me to spell it out (-S 65536)?

qemu-img.c static int img_convert(int argc, char **argv) · line 1473

Quoted 9 lines
qemu-img.c +1 -1
            "  '--output' takes the format in which the output must be done (human or json)\n"
            "  '-n' skips the target volume creation (useful if the volume is created\n"
            "       prior to running qemu-img)\n"
@@ -1468,7 +1470,7 @@ static int img_convert(int argc, char **argv)
         /* signal EOF to align */
         bdrv_write_compressed(out_bs, 0, NULL, 0);
     } else {
-        int has_zero_init = bdrv_has_zero_init(out_bs);
+        int has_zero_init = min_sparse ? bdrv_has_zero_init(out_bs) : 0;

Is this another function that should be returning bool instead of int? But that's a question for a separate patch.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#35 lore raw

Re: [Qemu-devel] [PATCHv2 18/20] qemu-img: add support for fully allocated images

Am 19.09.2013 um 22:36 schrieb Eric Blake <[email protected]>:

Quoted 23 lines · Peter Lieven
Quoted 10 lines

---
qemu-img.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
qemu-img.c +0 -0
index 3e5e388..7600b58 100644

qemu-img.c static void help(void) · line 105

Quoted 8 lines
qemu-img.c +3 -2
@@ -100,8 +100,10 @@ static void help(void)
           "  '-h' with or without a command shows this help and lists the supported formats\n"
           "  '-p' show progress of command (only certain commands)\n"
           "  '-q' use Quiet mode - do not print any output (except errors)\n"
-           "  '-S' indicates the consecutive number of bytes that must contain only zeros\n"
-           "       for qemu-img to create a sparse image during conversion\n"
+           "  '-S' indicates the consecutive number of bytes (defaults to 4096) that must\n"
+           "       contain only zeros for qemu-img to create a sparse image during\n"
+           "       conversion. if the number of bytes is 0 sparse files are disabled and\n"

s/if/If/
s/0/0,/

Quoted 5 lines

qemu-img.c static void help(void) · line 106

Quoted 1 line
qemu-img.c +1 -0
+           "       images will always be fully allocated\n"

IIRC correctly, on NTFS file system, the minimum hole size for a sparse file is 64k. Can you please make this option accept a suffixed number (-S 64k) instead of forcing me to spell it out (-S 65536)?

I can do that, of course. But the option is old it was not introduced by me ;-) I only added the handling for -S 0 ;-)

Quoted 13 lines

qemu-img.c static int img_convert(int argc, char **argv) · line 1473

Quoted 10 lines
qemu-img.c +1 -1
           "  '--output' takes the format in which the output must be done (human or json)\n"
           "  '-n' skips the target volume creation (useful if the volume is created\n"
           "       prior to running qemu-img)\n"
@@ -1468,7 +1470,7 @@ static int img_convert(int argc, char **argv)
        /* signal EOF to align */
        bdrv_write_compressed(out_bs, 0, NULL, 0);
    } else {
-        int has_zero_init = bdrv_has_zero_init(out_bs);
+        int has_zero_init = min_sparse ? bdrv_has_zero_init(out_bs) : 0;

Is this another function that should be returning bool instead of int? But that's a question for a separate patch.

If the answer is yes, bdrv_has_discard_zeroes and bdrv_has_discard_write_zeroes should return bool as well.

Peter

#36 lore raw

PATCH v2 19/20 qemu-img: conditionally zero out target on convert

In reply to Peter Lieven

if the target has_zero_init = 0, but supports efficiently writing zeroes by unmapping we call bdrv_zeroize to avoid fully allocating the target. this currently is designed especially for iscsi.

---
 qemu-img.c |   10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
qemu-img.c +9 -1
index 7600b58..d6af941 100644
@@ -1356,7 +1356,7 @@ static int img_convert(int argc, char **argv)
         }
     }
 
-    flags = BDRV_O_RDWR;
+    flags = min_sparse ? (BDRV_O_RDWR | BDRV_O_UNMAP) : BDRV_O_RDWR;
     ret = bdrv_parse_cache_flags(cache, &flags);
     if (ret < 0) {
         error_report("Invalid cache option: %s", cache);
@@ -1472,6 +1472,14 @@ static int img_convert(int argc, char **argv)
     } else {
         int has_zero_init = min_sparse ? bdrv_has_zero_init(out_bs) : 0;
 
+        if (!has_zero_init && bdrv_has_discard_write_zeroes(out_bs)) {
+            ret = bdrv_zeroize(out_bs, BDRV_REQ_MAY_UNMAP);
+            if (ret < 0) {
+                goto out;
+            }
+            has_zero_init = 1;
+        }
+
         sector_num = 0; // total number of sectors converted so far
         nb_sectors = total_sectors - sector_num;
         if (nb_sectors != 0) {
Signature · 1 line
1.7.9.5
Reviewed-by on v2 19/20
#37 lore raw

Re: [Qemu-devel] [PATCHv2 19/20] qemu-img: conditionally zero out target on convert

Quoted 3 lines · Peter Lieven

if the target has_zero_init = 0, but supports efficiently writing zeroes by unmapping we call bdrv_zeroize to avoid fully allocating the target. this currently

You don't like the shift key, do you? :)

s/this/This/ if you care about grammar in commit messages (I care about it in user-visible strings, but am more than willing to let it slide in commit messages)

Quoted 6 lines

is designed especially for iscsi.

---
 qemu-img.c |   10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#38 lore raw

PATCH v2 20/20 block/raw: copy block limits and alignment information on raw_open

In reply to Peter Lieven
---
 block/raw_bsd.c |    4 ++++
 1 file changed, 4 insertions(+)
block/raw_bsd.c +4 -0
index 0bfa5fc..dfdb375 100644
@@ -149,6 +149,10 @@ static int raw_create(const char *filename, QEMUOptionParameter *options)
 static int raw_open(BlockDriverState *bs, QDict *options, int flags)
 {
     bs->sg = bs->file->sg;
+    bs->max_discard = bs->file->max_discard;
+    bs->discard_alignment = bs->file->discard_alignment;
+    bs->max_write_zeroes = bs->file->max_write_zeroes;
+    bs->write_zeroes_alignment = bs->file->write_zeroes_alignment;
     return 0;
 }
 
Signature · 1 line
1.7.9.5
#39 lore raw

Re: [Qemu-devel] [PATCHv2 20/20] block/raw: copy block limits and alignment information on raw_open

Il 17/09/2013 15:48, Peter Lieven ha scritto:

Quoted 10 lines
---
 block/raw_bsd.c |    4 ++++
 1 file changed, 4 insertions(+)
block/raw_bsd.c +0 -0
index 0bfa5fc..dfdb375 100644

block/raw_bsd.c static int raw_create(const char *filename, QEMUOptionParameter *options) · line 156

Quoted 8 lines
block/raw_bsd.c +4 -0
@@ -149,6 +149,10 @@ static int raw_create(const char *filename, QEMUOptionParameter *options)
 static int raw_open(BlockDriverState *bs, QDict *options, int flags)
 {
     bs->sg = bs->file->sg;
+    bs->max_discard = bs->file->max_discard;
+    bs->discard_alignment = bs->file->discard_alignment;
+    bs->max_write_zeroes = bs->file->max_write_zeroes;
+    bs->write_zeroes_alignment = bs->file->write_zeroes_alignment;
     return 0;

What about introducing a BlockLimits struct?

Paolo

Quoted 3 lines
block/raw_bsd.c +0 -0
 }
 
#40 lore raw

Re: [Qemu-devel] [PATCHv2 20/20] block/raw: copy block limits and alignment information on raw_open

Quoted 6 lines · Paolo Bonzini

Il 17/09/2013 15:48, Peter Lieven ha scritto:

Quoted 5 lines
---
  block/raw_bsd.c |    4 ++++
  1 file changed, 4 insertions(+)
Quoted 14 lines

block/raw_bsd.c static int raw_create(const char *filename, QEMUOptionParameter *options) · line 156

Quoted 13 lines
block/raw_bsd.c +4 -0
index 0bfa5fc..dfdb375 100644
@@ -149,6 +149,10 @@ static int raw_create(const char *filename, QEMUOptionParameter *options)
  static int raw_open(BlockDriverState *bs, QDict *options, int flags)
  {
      bs->sg = bs->file->sg;
+    bs->max_discard = bs->file->max_discard;
+    bs->discard_alignment = bs->file->discard_alignment;
+    bs->max_write_zeroes = bs->file->max_write_zeroes;
+    bs->write_zeroes_alignment = bs->file->write_zeroes_alignment;
      return 0;

What about introducing a BlockLimits struct?

Good idea.

Do you like the rest of the series?

Peter

#41 lore raw

Re: [Qemu-devel] [PATCHv2 20/20] block/raw: copy block limits and alignment information on raw_open

Il 19/09/2013 09:29, Peter Lieven ha scritto:

Quoted 22 lines · Paolo Bonzini
Quoted 6 lines

Il 17/09/2013 15:48, Peter Lieven ha scritto:

Quoted 5 lines
---
  block/raw_bsd.c |    4 ++++
  1 file changed, 4 insertions(+)
Quoted 15 lines
Quoted 5 lines
block/raw_bsd.c +0 -0
index 0bfa5fc..dfdb375 100644

block/raw_bsd.c static int raw_create(const char *filename, · line 157

Quoted 9 lines
block/raw_bsd.c +4 -0
@@ -149,6 +149,10 @@ static int raw_create(const char *filename,
 QEMUOptionParameter *options)
  static int raw_open(BlockDriverState *bs, QDict *options, int flags)
  {
      bs->sg = bs->file->sg;
+    bs->max_discard = bs->file->max_discard;
+    bs->discard_alignment = bs->file->discard_alignment;
+    bs->max_write_zeroes = bs->file->max_write_zeroes;
+    bs->write_zeroes_alignment = bs->file->write_zeroes_alignment;
      return 0;

What about introducing a BlockLimits struct?

Quoted 3 lines

Good idea.

Do you like the rest of the series?

Yes, I do.

Paolo