Skip to contentSkip to conversation

qcow2, raw: add preallocation=full and preallocation=falloc

RFC PATCH Superseded Hu Tao 3 participants 14 messages Jul 2, 2014 – Jul 3, 2014 Block layer

v11 of 15 Superseded Latest is v15 →

lore
Share & tools
lore

Hu Tao

author
#0 lore raw

v11 cover qcow2, raw: add preallocation=full and preallocation=falloc

Kevin,

This is v11 series for you to check how the series depends on Max's minimal_blob_size(). I'm aware that you rejected the calculation of metadata size, but posting this series so that you can comment on it.

This series depends on patches 1-3 of Max's series 'qemu-img: Implement commit like QMP'.

Option preallocation=full preallocates disk space for image by writing zeros to disk, this ensures disk space in any cases.

Option preallocation=falloc preallocates disk space by calling posix_fallocate(). This is faster than preallocation=full.

The series is also at https://github.com/taohu/qemu/commits/preallocation-v11
for you to check out.

changes to v10:

- PreallocMode is moved from file qapi-schema.json to qapi/block-core.json
- introdues preallocation=falloc, no changes to preallocation=metadata
- using minimal_blob_size() to calculate metadata size for qcow2
- indentation fix in file blockdev.c

Hu Tao (6):
block: round up file size to nearest sector
raw, qcow2: don't convert file size to sector size rename parse_enum_option to qapi_enum_parse and make it public
qapi: introduce PreallocMode and a new PreallocMode full.
raw-posix: Add falloc and full preallocation option
qcow2: Add falloc and full preallocation option

 block/qcow2.c              | 54 ++++++++++++++++++++++++++++---------
 block/raw-posix.c          | 66 +++++++++++++++++++++++++++++++++++++++-------
 block/raw-win32.c          |  6 ++---
 blockdev.c                 | 30 +++++----------------
 include/qapi/util.h        | 17 ++++++++++++
 qapi/Makefile.objs         |  1 +
 qapi/block-core.json       | 17 ++++++++++++
 qapi/qapi-util.c           | 32 ++++++++++++++++++++++
 tests/qemu-iotests/082.out | 54 ++++++++++++++++++-------------------
 tests/qemu-iotests/096     | 64 ++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/096.out | 14 ++++++++++
 tests/qemu-iotests/group   |  1 +
 12 files changed, 280 insertions(+), 76 deletions(-)
 create mode 100644 include/qapi/util.h
 create mode 100644 qapi/qapi-util.c
 create mode 100755 tests/qemu-iotests/096
 create mode 100644 tests/qemu-iotests/096.out
Signature · 1 line
1.9.3
author
#1 lore raw

RFC PATCH v11 1/6 block: round up file size to nearest sector

---
 block/qcow2.c              |  3 ++-
 block/raw-posix.c          |  4 +--
 block/raw-win32.c          |  4 +--
 tests/qemu-iotests/096     | 64 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/096.out | 14 ++++++++++
 tests/qemu-iotests/group   |  1 +
 6 files changed, 85 insertions(+), 5 deletions(-)
 create mode 100755 tests/qemu-iotests/096
 create mode 100644 tests/qemu-iotests/096.out
block/qcow2.c +2 -1
index 0aba8dd..0dfbb9a 100644
@@ -1777,7 +1777,8 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
     int ret;
 
     /* Read out options */
-    sectors = qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0) / 512;
+    sectors = DIV_ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+                           BDRV_SECTOR_SIZE);
     backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
     backing_fmt = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FMT);
     if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) {
block/raw-posix.c +2 -2
index dacf4fb..b861bd9 100644
@@ -1282,8 +1282,8 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
     strstart(filename, "file:", &filename);
 
     /* Read out options */
-    total_size =
-        qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0) / BDRV_SECTOR_SIZE;
+    total_size = DIV_ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+                              BDRV_SECTOR_SIZE);
 
     fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
                    0644);
block/raw-win32.c +2 -2
index 902eab6..1e1880d 100644
@@ -511,8 +511,8 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
     strstart(filename, "file:", &filename);
 
     /* Read out options */
-    total_size =
-        qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0) / 512;
+    total_size = DIV_ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+                              BDRV_SECTOR_SIZE);
 
     fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
                    0644);
tests/qemu-iotests/096 new file +64 -0

64 lines not on this page · raw message

tests/qemu-iotests/096.out new file +14 -0
new file mode 100644
index 0000000..5f6c262
@@ -0,0 +1,14 @@
+QA output created by 096
+=== Check qemu-img info output ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1024 
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 1.0K (1024 bytes)
+cluster_size: 65536
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1234 
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 1.5K (1536 bytes)
+cluster_size: 65536
+***done
tests/qemu-iotests/group +1 -0
index e3dc4e8..65d04ee 100644
@@ -100,3 +100,4 @@
 091 rw auto
 092 rw auto quick
 095 rw auto
+096 rw auto
Signature · 1 line
1.9.3
Reviewed-by on v11 1/6
#2 lore raw

Re: [Qemu-devel] [PATCH for-2.1? RFC v11 1/6] block: round up file size to nearest sector

Quoted 12 lines · Hu Tao
---
 block/qcow2.c              |  3 ++-
 block/raw-posix.c          |  4 +--
 block/raw-win32.c          |  4 +--
 tests/qemu-iotests/096     | 64 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/096.out | 14 ++++++++++
 tests/qemu-iotests/group   |  1 +
 6 files changed, 85 insertions(+), 5 deletions(-)
 create mode 100755 tests/qemu-iotests/096
 create mode 100644 tests/qemu-iotests/096.out

I think this patch qualifies as a bug fix; would you like to separate it out, remove the RFC, and tag it for 2.1, so that it isn't held up by the rest of the series?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
author
#3 lore raw

RFC PATCH v11 2/6 raw, qcow2: don't convert file size to sector size

In reply to Hu Tao

and avoid converting it back later.

---
 block/qcow2.c     | 10 +++++-----
 block/raw-posix.c |  6 +++---
 block/raw-win32.c |  6 +++---
 3 files changed, 11 insertions(+), 11 deletions(-)
block/qcow2.c +5 -5
index 0dfbb9a..05df4b9 100644
@@ -1715,7 +1715,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
     }
 
     /* Okay, now that we have a valid image, let's give it the right size */
-    ret = bdrv_truncate(bs, total_size * BDRV_SECTOR_SIZE);
+    ret = bdrv_truncate(bs, total_size);
     if (ret < 0) {
         error_setg_errno(errp, -ret, "Could not resize image");
         goto out;
@@ -1768,7 +1768,7 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
     char *backing_file = NULL;
     char *backing_fmt = NULL;
     char *buf = NULL;
-    uint64_t sectors = 0;
+    uint64_t size = 0;
     int flags = 0;
     size_t cluster_size = DEFAULT_CLUSTER_SIZE;
     int prealloc = 0;
@@ -1777,8 +1777,8 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
     int ret;
 
     /* Read out options */
-    sectors = DIV_ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-                           BDRV_SECTOR_SIZE);
+    size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+                    BDRV_SECTOR_SIZE);
     backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
     backing_fmt = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FMT);
     if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) {
@@ -1828,7 +1828,7 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
         goto finish;
     }
 
-    ret = qcow2_create2(filename, sectors, backing_file, backing_fmt, flags,
+    ret = qcow2_create2(filename, size, backing_file, backing_fmt, flags,
                         cluster_size, prealloc, opts, version, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
block/raw-posix.c +3 -3
index b861bd9..58c51d8 100644
@@ -1282,8 +1282,8 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
     strstart(filename, "file:", &filename);
 
     /* Read out options */
-    total_size = DIV_ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-                              BDRV_SECTOR_SIZE);
+    total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+                          BDRV_SECTOR_SIZE);
 
     fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
                    0644);
@@ -1291,7 +1291,7 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
         result = -errno;
         error_setg_errno(errp, -result, "Could not create file");
     } else {
-        if (ftruncate(fd, total_size * BDRV_SECTOR_SIZE) != 0) {
+        if (ftruncate(fd, total_size) != 0) {
             result = -errno;
             error_setg_errno(errp, -result, "Could not resize file");
         }
block/raw-win32.c +3 -3
index 1e1880d..9bf8225 100644
@@ -511,8 +511,8 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
     strstart(filename, "file:", &filename);
 
     /* Read out options */
-    total_size = DIV_ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-                              BDRV_SECTOR_SIZE);
+    total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+                          BDRV_SECTOR_SIZE);
 
     fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
                    0644);
@@ -521,7 +521,7 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
         return -EIO;
     }
     set_sparse(fd);
-    ftruncate(fd, total_size * 512);
+    ftruncate(fd, total_size);
     qemu_close(fd);
     return 0;
 }
Signature · 1 line
1.9.3
Reviewed-by on v11 2/6
#4 lore raw

Re: [Qemu-devel] [PATCH RFC v11 2/6] raw, qcow2: don't convert file size to sector size

Quoted 9 lines · Hu Tao

and avoid converting it back later.

---
 block/qcow2.c     | 10 +++++-----
 block/raw-posix.c |  6 +++---
 block/raw-win32.c |  6 +++---
 3 files changed, 11 insertions(+), 11 deletions(-)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
author
#5 lore raw

RFC PATCH v11 3/6 rename parse_enum_option to qapi_enum_parse and make it public

In reply to Hu Tao
---
 blockdev.c          | 30 ++++++------------------------
 include/qapi/util.h | 17 +++++++++++++++++
 qapi/Makefile.objs  |  1 +
 qapi/qapi-util.c    | 32 ++++++++++++++++++++++++++++++++
 4 files changed, 56 insertions(+), 24 deletions(-)
 create mode 100644 include/qapi/util.h
 create mode 100644 qapi/qapi-util.c
blockdev.c +6 -24
index 69b7c2a..0967b93 100644
@@ -39,6 +39,7 @@
 #include "qapi/qmp/types.h"
 #include "qapi-visit.h"
 #include "qapi/qmp-output-visitor.h"
+#include "qapi/util.h"
 #include "sysemu/sysemu.h"
 #include "block/block_int.h"
 #include "qmp-commands.h"
@@ -274,25 +275,6 @@ static int parse_block_error_action(const char *buf, bool is_read, Error **errp)
     }
 }
 
-static inline int parse_enum_option(const char *lookup[], const char *buf,
-                                    int max, int def, Error **errp)
-{
-    int i;
-
-    if (!buf) {
-        return def;
-    }
-
-    for (i = 0; i < max; i++) {
-        if (!strcmp(buf, lookup[i])) {
-            return i;
-        }
-    }
-
-    error_setg(errp, "invalid parameter value: %s", buf);
-    return def;
-}
-
 static bool check_throttle_config(ThrottleConfig *cfg, Error **errp)
 {
     if (throttle_conflicting(cfg)) {
@@ -456,11 +438,11 @@ static DriveInfo *blockdev_init(const char *file, QDict *bs_opts,
     }
 
     detect_zeroes =
-        parse_enum_option(BlockdevDetectZeroesOptions_lookup,
-                          qemu_opt_get(opts, "detect-zeroes"),
-                          BLOCKDEV_DETECT_ZEROES_OPTIONS_MAX,
-                          BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF,
-                          &error);
+        qapi_enum_parse(BlockdevDetectZeroesOptions_lookup,
+                        qemu_opt_get(opts, "detect-zeroes"),
+                        BLOCKDEV_DETECT_ZEROES_OPTIONS_MAX,
+                        BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF,
+                        &error);
     if (error) {
         error_propagate(errp, error);
         goto early_err;
include/qapi/util.h new file +17 -0
new file mode 100644
index 0000000..de9238b
@@ -0,0 +1,17 @@
+/*
+ * QAPI util functions
+ *
+ * Copyright Fujitsu, Inc. 2014
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QAPI_UTIL_H
+#define QAPI_UTIL_H
+
+int qapi_enum_parse(const char *lookup[], const char *buf,
+                    int max, int def, Error **errp);
+
+#endif
qapi/Makefile.objs +1 -0
index d14b769..ffd88a6 100644
@@ -4,3 +4,4 @@ util-obj-y += string-input-visitor.o string-output-visitor.o
 
 util-obj-y += opts-visitor.o
 util-obj-y += qmp-event.o
+util-obj-y += qapi-util.o
qapi/qapi-util.c new file +32 -0
new file mode 100644
index 0000000..3e4e07c
@@ -0,0 +1,32 @@
+/*
+ * QAPI util functions
+ *
+ * Copyright Fujitsu, Inc. 2014
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "qapi/util.h"
+
+int qapi_enum_parse(const char *lookup[], const char *buf,
+                    int max, int def, Error **errp)
+{
+    int i;
+
+    if (!buf) {
+        return def;
+    }
+
+    for (i = 0; i < max; i++) {
+        if (!strcmp(buf, lookup[i])) {
+            return i;
+        }
+    }
+
+    error_setg(errp, "invalid parameter value: %s", buf);
+    return def;
+}
Signature · 1 line
1.9.3
Reviewed-by on v11 3/6
#6 lore raw

Re: [Qemu-devel] [PATCH RFC v11 3/6] rename parse_enum_option to qapi_enum_parse and make it public

Quoted 11 lines · Hu Tao
---
 blockdev.c          | 30 ++++++------------------------
 include/qapi/util.h | 17 +++++++++++++++++
 qapi/Makefile.objs  |  1 +
 qapi/qapi-util.c    | 32 ++++++++++++++++++++++++++++++++
 4 files changed, 56 insertions(+), 24 deletions(-)
 create mode 100644 include/qapi/util.h
 create mode 100644 qapi/qapi-util.c
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
author
#7 lore raw

RFC PATCH v11 4/6 qapi: introduce PreallocMode and a new PreallocMode full.

In reply to Hu Tao

This patch prepares for the subsequent patches.

---
 block/qcow2.c        | 16 ++++++++--------
 qapi/block-core.json | 17 +++++++++++++++++
 2 files changed, 25 insertions(+), 8 deletions(-)
block/qcow2.c +8 -8
index 05df4b9..cfba93b 100644
@@ -30,6 +30,7 @@
 #include "qemu/error-report.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qbool.h"
+#include "qapi/util.h"
 #include "trace.h"
 #include "qemu/option_int.h"
 
@@ -1594,7 +1595,7 @@ static int preallocate(BlockDriverState *bs)
 
 static int qcow2_create2(const char *filename, int64_t total_size,
                          const char *backing_file, const char *backing_format,
-                         int flags, size_t cluster_size, int prealloc,
+                         int flags, size_t cluster_size, PreallocMode prealloc,
                          QemuOpts *opts, int version,
                          Error **errp)
 {
@@ -1771,7 +1772,7 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
     uint64_t size = 0;
     int flags = 0;
     size_t cluster_size = DEFAULT_CLUSTER_SIZE;
-    int prealloc = 0;
+    PreallocMode prealloc = PREALLOC_MODE_OFF;
     int version = 3;
     Error *local_err = NULL;
     int ret;
@@ -1787,12 +1788,11 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
     cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE,
                                          DEFAULT_CLUSTER_SIZE);
     buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
-    if (!buf || !strcmp(buf, "off")) {
-        prealloc = 0;
-    } else if (!strcmp(buf, "metadata")) {
-        prealloc = 1;
-    } else {
-        error_setg(errp, "Invalid preallocation mode: '%s'", buf);
+    prealloc = qapi_enum_parse(PreallocMode_lookup, buf,
+                               PREALLOC_MODE_MAX, PREALLOC_MODE_OFF,
+                               &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
         ret = -EINVAL;
         goto finish;
     }
qapi/block-core.json +17 -0
index faf394c..ca8a938 100644
@@ -1585,3 +1585,20 @@
             'len'   : 'int',
             'offset': 'int',
             'speed' : 'int' } }
+
+# @PreallocMode
+#
+# Preallocation mode of QEMU image file
+#
+# @off: no preallocation
+# @metadata: preallocate only for metadata
+# @falloc: like @full preallocation but allocate disk space by
+#          posix_fallocate() rather than writing zeros.
+# @full: preallocate all data by writing zeros to device to ensure disk
+#        space is really available. @full preallocation also sets up
+#        metadata correctly.
+#
+# Since 2.2
+##
+{ 'enum': 'PreallocMode',
+  'data': [ 'off', 'metadata', 'falloc', 'full' ] }
Signature · 1 line
1.9.3
Reviewed-by on v11 4/6
#8 lore raw

Re: [Qemu-devel] [PATCH RFC v11 4/6] qapi: introduce PreallocMode and a new PreallocMode full.

Quoted 9 lines · Hu Tao

This patch prepares for the subsequent patches.

---
  block/qcow2.c        | 16 ++++++++--------
  qapi/block-core.json | 17 +++++++++++++++++
  2 files changed, 25 insertions(+), 8 deletions(-)
author
#9 lore raw

RFC PATCH v11 5/6 raw-posix: Add falloc and full preallocation option

In reply to Hu Tao

This patch adds a new option preallocation for raw format, and implements falloc and full preallocation.

---
 block/raw-posix.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 55 insertions(+), 7 deletions(-)
block/raw-posix.c +55 -7
index 58c51d8..5dcd465 100644
@@ -30,6 +30,7 @@
 #include "block/thread-pool.h"
 #include "qemu/iov.h"
 #include "raw-aio.h"
+#include "qapi/util.h"
 
 #if defined(__APPLE__) && (__MACH__)
 #include <paths.h>
@@ -1278,28 +1279,70 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
     int fd;
     int result = 0;
     int64_t total_size = 0;
+    PreallocMode prealloc = PREALLOC_MODE_OFF;
+    char *buf = NULL;
+    Error *local_err = NULL;
 
     strstart(filename, "file:", &filename);
 
     /* Read out options */
     total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
                           BDRV_SECTOR_SIZE);
+    buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
+    prealloc = qapi_enum_parse(PreallocMode_lookup, buf,
+                               PREALLOC_MODE_MAX, PREALLOC_MODE_OFF,
+                               &local_err);
+    g_free(buf);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        result = -EINVAL;
+        goto out;
+    }
 
     fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
                    0644);
     if (fd < 0) {
         result = -errno;
         error_setg_errno(errp, -result, "Could not create file");
-    } else {
-        if (ftruncate(fd, total_size) != 0) {
-            result = -errno;
-            error_setg_errno(errp, -result, "Could not resize file");
+        goto out;
+    }
+    if (ftruncate(fd, total_size) != 0) {
+        result = -errno;
+        error_setg_errno(errp, -result, "Could not resize file");
+        goto out_close;
+    }
+    if (prealloc == PREALLOC_MODE_FALLOC) {
+        /* posix_fallocate() doesn't set errno. */
+        result = -posix_fallocate(fd, 0, total_size);
+        if (result != 0) {
+            error_setg_errno(errp, -result,
+                             "Could not preallocate data for the new file");
         }
-        if (qemu_close(fd) != 0) {
-            result = -errno;
-            error_setg_errno(errp, -result, "Could not close the new file");
+    } else if (prealloc == PREALLOC_MODE_FULL) {
+        buf = g_malloc0(65536);
+        int64_t num = 0, left = total_size;
+
+        while (left > 0) {
+            num = MIN(left, 65536);
+            result = write(fd, buf, num);
+            if (result < 0) {
+                result = -errno;
+                error_setg_errno(errp, -result,
+                                 "Could not write to the new file");
+                g_free(buf);
+                goto out_close;
+            }
+            left -= num;
         }
+        fsync(fd);
+        g_free(buf);
+    }
+out_close:
+    if (qemu_close(fd) != 0 && result == 0) {
+        result = -errno;
+        error_setg_errno(errp, -result, "Could not close the new file");
     }
+out:
     return result;
 }
 
@@ -1477,6 +1520,11 @@ static QemuOptsList raw_create_opts = {
             .type = QEMU_OPT_SIZE,
             .help = "Virtual disk size"
         },
+        {
+            .name = BLOCK_OPT_PREALLOC,
+            .type = QEMU_OPT_STRING,
+            .help = "Preallocation mode (allowed values: off, falloc, full)"
+        },
         { /* end of list */ }
     }
 };
Signature · 1 line
1.9.3
Reviewed-by on v11 5/6
#10 lore raw

Re: [Qemu-devel] [PATCH RFC v11 5/6] raw-posix: Add falloc and full preallocation option

Quoted 21 lines · Hu Tao

This patch adds a new option preallocation for raw format, and implements falloc and full preallocation.

---
  block/raw-posix.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++-------
  1 file changed, 55 insertions(+), 7 deletions(-)
block/raw-posix.c +1 -0
index 58c51d8..5dcd465 100644
@@ -30,6 +30,7 @@
  #include "block/thread-pool.h"
  #include "qemu/iov.h"
  #include "raw-aio.h"
+#include "qapi/util.h"
 
  #if defined(__APPLE__) && (__MACH__)
  #include <paths.h>

block/raw-posix.c static int raw_create(const char *filename, QemuOpts *opts, Error **errp) · line 1283

Quoted 5 lines
block/raw-posix.c +2 -0
@@ -1278,28 +1279,70 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
      int fd;
      int result = 0;
      int64_t total_size = 0;
+    PreallocMode prealloc = PREALLOC_MODE_OFF;
+    char *buf = NULL;

It's not really necessary to initialize these two here, but it won't hurt either, of course (same thing later with num, which could have even been declared inside of the while loop).

author
#11 lore raw

RFC PATCH v11 6/6 qcow2: Add falloc and full preallocation option

In reply to Hu Tao

This adds preallocation=falloc and preallocation=full mode to qcow2 image creation.

preallocation=full allocates disk space by writing zeros to disk to ensure disk space in any cases.

preallocation=falloc likes preallocation=full, but allocates disk space by posix_fallocate().

---
 block/qcow2.c              | 29 ++++++++++++++++++++++++-
 tests/qemu-iotests/082.out | 54 +++++++++++++++++++++++-----------------------
 2 files changed, 55 insertions(+), 28 deletions(-)
block/qcow2.c +28 -1
index cfba93b..f2df8cb 100644
@@ -1593,6 +1593,9 @@ static int preallocate(BlockDriverState *bs)
     return 0;
 }
 
+static uint64_t minimal_blob_size(uint64_t ts, int cb, int spcb,
+                                  uint64_t overhead);
+
 static int qcow2_create2(const char *filename, int64_t total_size,
                          const char *backing_file, const char *backing_format,
                          int flags, size_t cluster_size, PreallocMode prealloc,
@@ -1628,6 +1631,29 @@ static int qcow2_create2(const char *filename, int64_t total_size,
     Error *local_err = NULL;
     int ret;
 
+    if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
+        int64_t meta_size = 0;
+        unsigned nl2e;
+
+        total_size = align_offset(total_size, cluster_size);
+
+        /* total size of L2 tables */
+        nl2e = total_size / cluster_size;
+        nl2e = align_offset(nl2e, cluster_size / sizeof(uint64_t));
+        uint64_t l2_clusters = nl2e * sizeof(uint64_t) >> cluster_bits;
+
+        meta_size =
+            (1 +
+             minimal_blob_size(total_size / BDRV_SECTOR_SIZE,
+                               cluster_bits, cluster_bits - BDRV_SECTOR_BITS,
+                               1 + l2_clusters +
+                               (total_size >> cluster_bits)) +
+             l2_clusters) << cluster_bits;
+
+        qemu_opt_set_number(opts, BLOCK_OPT_SIZE, total_size + meta_size);
+        qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_lookup[prealloc]);
+    }
+
     ret = bdrv_create_file(filename, opts, &local_err);
     if (ret < 0) {
         error_propagate(errp, local_err);
@@ -2760,7 +2786,8 @@ static QemuOptsList qcow2_create_opts = {
         {
             .name = BLOCK_OPT_PREALLOC,
             .type = QEMU_OPT_STRING,
-            .help = "Preallocation mode (allowed values: off, metadata)"
+            .help = "Preallocation mode (allowed values: off, metadata, "
+                    "falloc, full)"
         },
         {
             .name = BLOCK_OPT_LAZY_REFCOUNTS,
tests/qemu-iotests/082.out +27 -27

216 lines not on this page · raw message

Signature · 1 line
1.9.3
Reviewed-by on v11 6/6
#12 lore raw

Re: [Qemu-devel] [PATCH RFC v11 6/6] qcow2: Add falloc and full preallocation option

Quoted 31 lines · Hu Tao

This adds preallocation=falloc and preallocation=full mode to qcow2 image creation.

preallocation=full allocates disk space by writing zeros to disk to ensure disk space in any cases.

preallocation=falloc likes preallocation=full, but allocates disk space by posix_fallocate().

---
  block/qcow2.c              | 29 ++++++++++++++++++++++++-
  tests/qemu-iotests/082.out | 54 +++++++++++++++++++++++-----------------------
  2 files changed, 55 insertions(+), 28 deletions(-)
block/qcow2.c +3 -0
index cfba93b..f2df8cb 100644
@@ -1593,6 +1593,9 @@ static int preallocate(BlockDriverState *bs)
      return 0;
  }
 
+static uint64_t minimal_blob_size(uint64_t ts, int cb, int spcb,
+                                  uint64_t overhead);
+
  static int qcow2_create2(const char *filename, int64_t total_size,
                           const char *backing_file, const char *backing_format,
                           int flags, size_t cluster_size, PreallocMode prealloc,
@@ -1628,6 +1631,29 @@ static int qcow2_create2(const char *filename, int64_t total_size,
      Error *local_err = NULL;

block/qcow2.c static int qcow2_create2(const char *filename, int64_t total_size, · line 1641

Quoted 10 lines
block/qcow2.c +8 -0
      int ret;
 
+    if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
+        int64_t meta_size = 0;
+        unsigned nl2e;
+
+        total_size = align_offset(total_size, cluster_size);
+
+        /* total size of L2 tables */
+        nl2e = total_size / cluster_size;

This might overflow, better use a uint64_t for nl2e. Also: You're using ">> cluster_bits" below to divide by the cluster size; I'd prefer using the same method in both places (either ">> cluster_bits" or "/ cluster_size").

Quoted 8 lines
block/qcow2.c +8 -0
+        nl2e = align_offset(nl2e, cluster_size / sizeof(uint64_t));
+        uint64_t l2_clusters = nl2e * sizeof(uint64_t) >> cluster_bits;
+
+        meta_size =
+            (1 +
+             minimal_blob_size(total_size / BDRV_SECTOR_SIZE,
+                               cluster_bits, cluster_bits - BDRV_SECTOR_BITS,
+                               1 + l2_clusters +

block/qcow2.c static int qcow2_create2(const char *filename, int64_t total_size, · line 1659

Quoted 10 lines
block/qcow2.c +7 -0
+                               (total_size >> cluster_bits)) +
+             l2_clusters) << cluster_bits;
+
+        qemu_opt_set_number(opts, BLOCK_OPT_SIZE, total_size + meta_size);
+        qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_lookup[prealloc]);
+    }
+
      ret = bdrv_create_file(filename, opts, &local_err);
      if (ret < 0) {
          error_propagate(errp, local_err);

Maybe I'd additionally change the "if (prealloc)" later in qcow2_create2() to "if (prealloc != PREALLOC_MODE_NONE)" to make it more clear that there is more than just one method of preallocation now.

Maybe we also want to modify preallocate() later on to allocate all metadata structures in a single block (because, who knows, we might want a linear layout for the data clusters for some strange reason), but currently there is no need for that.

#13 lore raw

Re: [Qemu-devel] [PATCH RFC v11 6/6] qcow2: Add falloc and full preallocation option

In reply to Hu Tao
Quoted 6 lines · Hu Tao

This adds preallocation=falloc and preallocation=full mode to qcow2 image creation.

preallocation=full allocates disk space by writing zeros to disk to ensure disk space in any cases.

Quoted 8 lines

preallocation=falloc likes preallocation=full, but allocates disk space by posix_fallocate().

---
  block/qcow2.c              | 29 ++++++++++++++++++++++++-
  tests/qemu-iotests/082.out | 54 +++++++++++++++++++++++-----------------------
  2 files changed, 55 insertions(+), 28 deletions(-)

Ah, I missed something: You'll need to fix the output of test 049 as well ("Invalid preallocation mode: '1234'" -> "invalid parameter value: 1234").

Max