Skip to contentSkip to conversation

Exposing backing-chain allocation over NBD

PATCH Superseded Eric Blake 2 participants 16 messages Sep 30, 2020 – Oct 8, 2020

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Eric Blake

author
#0 lore raw

v2 cover Exposing backing-chain allocation over NBD

v1 was here: https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg09623.htm
l

Based-on: <[email protected]> (block/export: Add infrastructure and QAPI for block exports)

Also available at:
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/nbd-alloc-depth-v2

Since then:
- rebase on Kevin's work
- add new patch to fix qemu-nbd SIGINT (conflicts with Stefan's work, we can either rebase his on mine or drop mine if his goes in first)
- split out fix for handling NUL bytes from client [Vladimir]
- further cleanups of query parsing [Vladimir]
- more documentation of how we could also expose actual depth in remaining bits of the context reply [Rich]

001/5:[down] 'qemu-nbd: Honor SIGINT and SIGHUP' 002/5:[down] 'nbd/server: Reject embedded NUL in NBD strings' 003/5:[0139] [FC] 'nbd: Simplify meta-context parsing' 004/5:[0035] [FC] 'nbd: Add new qemu:allocation-depth metacontext' 005/5:[0038] [FC] 'nbd: Add 'qemu-nbd -A' to expose allocation depth'

Eric Blake (5):
qemu-nbd: Honor SIGINT and SIGHUP nbd/server: Reject embedded NUL in NBD strings
nbd: Simplify meta-context parsing
nbd: Add new qemu:allocation-depth metacontext
nbd: Add 'qemu-nbd -A' to expose allocation depth

 docs/interop/nbd.txt       |  22 ++-
 docs/tools/qemu-nbd.rst    |   6 +
 qapi/block-core.json       |   7 +-
 qapi/block-export.json     |   6 +-
 include/block/nbd.h        |   8 +-
 blockdev-nbd.c             |   2 +
 nbd/server.c               | 324 +++++++++++++++++++++----------------
 qemu-nbd.c                 |  20 ++-
 tests/qemu-iotests/309     |  73 +++++++++
 tests/qemu-iotests/309.out |  22 +++
 tests/qemu-iotests/group   |   1 +
 11 files changed, 342 insertions(+), 149 deletions(-)
 create mode 100755 tests/qemu-iotests/309
 create mode 100644 tests/qemu-iotests/309.out
Signature · 1 line
2.28.0
author
#1 lore raw

PATCH v2 1/5 qemu-nbd: Honor SIGINT and SIGHUP

Honoring just SIGTERM on Linux is too weak; we also want to handle other common signals, and do so even on BSD. Why? Because at least 'qemu-nbd -B bitmap' needs a chance to clean up the in-use bit on bitmaps when the server is shut down via a signal.

See also: http://bugzilla.redhat.com/1883608

---
 qemu-nbd.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
qemu-nbd.c +3 -3
index bacb69b0898b..e7520261134f 100644
@@ -581,7 +581,7 @@ int main(int argc, char **argv)
     const char *pid_file_name = NULL;
     BlockExportOptions *export_opts;
 
-#if HAVE_NBD_DEVICE
+#ifdef CONFIG_POSIX
     /* The client thread uses SIGTERM to interrupt the server.  A signal
      * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
      */
@@ -589,9 +589,9 @@ int main(int argc, char **argv)
     memset(&sa_sigterm, 0, sizeof(sa_sigterm));
     sa_sigterm.sa_handler = termsig_handler;
     sigaction(SIGTERM, &sa_sigterm, NULL);
-#endif /* HAVE_NBD_DEVICE */
+    sigaction(SIGINT, &sa_sigterm, NULL);
+    sigaction(SIGHUP, &sa_sigterm, NULL);
 
-#ifdef CONFIG_POSIX
     signal(SIGPIPE, SIG_IGN);
 #endif
 
Signature · 1 line
2.28.0
Reviewed-by on v2 1/5
#2 lore raw

Re: [PATCH v2 1/5] qemu-nbd: Honor SIGINT and SIGHUP

Quoted 4 lines · Eric Blake

Honoring just SIGTERM on Linux is too weak; we also want to handle other common signals, and do so even on BSD. Why? Because at least 'qemu-nbd -B bitmap' needs a chance to clean up the in-use bit on bitmaps when the server is shut down via a signal.

Probably not bad to update a comment [*] if you have a good wording in mind.

Quoted 4 lines

See also: http://bugzilla.redhat.com/1883608

Quoted 9 lines
---
  qemu-nbd.c | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
qemu-nbd.c +0 -0
index bacb69b0898b..e7520261134f 100644

qemu-nbd.c int main(int argc, char **argv) · line 586

Quoted 7 lines
qemu-nbd.c +1 -1
@@ -581,7 +581,7 @@ int main(int argc, char **argv)
      const char *pid_file_name = NULL;
      BlockExportOptions *export_opts;
 
-#if HAVE_NBD_DEVICE
+#ifdef CONFIG_POSIX
      /* The client thread uses SIGTERM to interrupt the server.  A signal
       * handler ensures that "qemu-nbd -v -c" exits with a nice status code.

[*]

Quoted 13 lines
qemu-nbd.c +2 -2
       */
@@ -589,9 +589,9 @@ int main(int argc, char **argv)
      memset(&sa_sigterm, 0, sizeof(sa_sigterm));
      sa_sigterm.sa_handler = termsig_handler;
      sigaction(SIGTERM, &sa_sigterm, NULL);
-#endif /* HAVE_NBD_DEVICE */
+    sigaction(SIGINT, &sa_sigterm, NULL);
+    sigaction(SIGHUP, &sa_sigterm, NULL);
 
-#ifdef CONFIG_POSIX
      signal(SIGPIPE, SIG_IGN);
  #endif
 
Signature · 2 lines
Best regards,
Vladimir
author
#3 lore raw

Re: [PATCH v2 1/5] qemu-nbd: Honor SIGINT and SIGHUP

Quoted 28 lines · Vladimir Sementsov-Ogievskiy
Quoted 4 lines · Eric Blake

Honoring just SIGTERM on Linux is too weak; we also want to handle other common signals, and do so even on BSD.  Why?  Because at least 'qemu-nbd -B bitmap' needs a chance to clean up the in-use bit on bitmaps when the server is shut down via a signal.

Probably not bad to update a comment [*] if you have a good wording in mind.

Quoted 4 lines

See also: http://bugzilla.redhat.com/1883608

Quoted 12 lines

---
  qemu-nbd.c | 6 +++---

  1 file changed, 3 insertions(+), 3 deletions(-)
qemu-nbd.c +0 -0
index bacb69b0898b..e7520261134f 100644
@@ -581,7 +581,7 @@ int main(int argc, char **argv)
       const char *pid_file_name = NULL;
       BlockExportOptions *export_opts;
 
Quoted 9 lines
Quoted 6 lines
qemu-nbd.c +1 -1
-#if HAVE_NBD_DEVICE
+#ifdef CONFIG_POSIX
       /* The client thread uses SIGTERM to interrupt the server.  A

signal
       * handler ensures that "qemu-nbd -v -c" exits with a nice status code.

[*]

Sure, I went with:

qemu-nbd.c +3 -2
index e7520261134f..c731dda04ec0 100644
@@ -582,8 +582,9 @@ int main(int argc, char **argv)
     BlockExportOptions *export_opts;
 
 #ifdef CONFIG_POSIX
-    /* The client thread uses SIGTERM to interrupt the server.  A signal
-     * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
+    /*
+     * Exit gracefully on various signals, which includes SIGTERM used
+     * by 'qemu-nbd -v -c'.
      */
     struct sigaction sa_sigterm;
     memset(&sa_sigterm, 0, sizeof(sa_sigterm));
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#4 lore raw

Re: [PATCH v2 1/5] qemu-nbd: Honor SIGINT and SIGHUP

Quoted 48 lines · Eric Blake
Quoted 28 lines · Vladimir Sementsov-Ogievskiy
Quoted 4 lines · Eric Blake

Honoring just SIGTERM on Linux is too weak; we also want to handle other common signals, and do so even on BSD.  Why?  Because at least 'qemu-nbd -B bitmap' needs a chance to clean up the in-use bit on bitmaps when the server is shut down via a signal.

Probably not bad to update a comment [*] if you have a good wording in mind.

Quoted 4 lines

See also: http://bugzilla.redhat.com/1883608

Quoted 12 lines
---
   qemu-nbd.c | 6 +++---
   1 file changed, 3 insertions(+), 3 deletions(-)
qemu-nbd.c +0 -0
index bacb69b0898b..e7520261134f 100644
@@ -581,7 +581,7 @@ int main(int argc, char **argv)
       const char *pid_file_name = NULL;
       BlockExportOptions *export_opts;
 
Quoted 9 lines
Quoted 6 lines
qemu-nbd.c +1 -1
-#if HAVE_NBD_DEVICE
+#ifdef CONFIG_POSIX
       /* The client thread uses SIGTERM to interrupt the server.  A

signal
       * handler ensures that "qemu-nbd -v -c" exits with a nice
status code.

[*]

Sure, I went with:

qemu-nbd.c +0 -0
index e7520261134f..c731dda04ec0 100644
@@ -582,8 +582,9 @@ int main(int argc, char **argv)
      BlockExportOptions *export_opts;

qemu-nbd.c int main(int argc, char **argv) · line 590

Quoted 10 lines
qemu-nbd.c +3 -2
 
  #ifdef CONFIG_POSIX
-    /* The client thread uses SIGTERM to interrupt the server.  A signal
-     * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
+    /*
+     * Exit gracefully on various signals, which includes SIGTERM used
+     * by 'qemu-nbd -v -c'.
       */
      struct sigaction sa_sigterm;
      memset(&sa_sigterm, 0, sizeof(sa_sigterm));

Thanks!

Signature · 2 lines
Best regards,
Vladimir
author
#5 lore raw

PATCH v2 2/5 nbd/server: Reject embedded NUL in NBD strings

In reply to Eric Blake

The NBD spec is clear that any string sent from the client must not contain embedded NUL characters. If the client passes "a\0", we should reject that option request rather than act on "a".

Testing this is not possible with a compliant client, but I was able to use gdb to coerce libnbd into temporarily behaving as such a client.

---
 nbd/server.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)
nbd/server.c +20 -10
index f74766add7b7..809f88ce6607 100644
@@ -301,10 +301,11 @@ nbd_opt_invalid(NBDClient *client, Error **errp, const char *fmt, ...)
 }
 
 /* Read size bytes from the unparsed payload of the current option.
+ * If @check_nul, require that no NUL bytes appear in buffer.
  * Return -errno on I/O error, 0 if option was completely handled by
  * sending a reply about inconsistent lengths, or 1 on success. */
 static int nbd_opt_read(NBDClient *client, void *buffer, size_t size,
-                        Error **errp)
+                        bool check_nul, Error **errp)
 {
     if (size > client->optlen) {
         return nbd_opt_invalid(client, errp,
@@ -312,7 +313,16 @@ static int nbd_opt_read(NBDClient *client, void *buffer, size_t size,
                                nbd_opt_lookup(client->opt));
     }
     client->optlen -= size;
-    return qio_channel_read_all(client->ioc, buffer, size, errp) < 0 ? -EIO : 1;
+    if (qio_channel_read_all(client->ioc, buffer, size, errp) < 0) {
+        return -EIO;
+    }
+
+    if (check_nul && strnlen(buffer, size) != size) {
+        return nbd_opt_invalid(client, errp,
+                               "Unexpected embedded NUL in option %s",
+                               nbd_opt_lookup(client->opt));
+    }
+    return 1;
 }
 
 /* Drop size bytes from the unparsed payload of the current option.
@@ -349,7 +359,7 @@ static int nbd_opt_read_name(NBDClient *client, char **name, uint32_t *length,
     g_autofree char *local_name = NULL;
 
     *name = NULL;
-    ret = nbd_opt_read(client, &len, sizeof(len), errp);
+    ret = nbd_opt_read(client, &len, sizeof(len), false, errp);
     if (ret <= 0) {
         return ret;
     }
@@ -361,7 +371,7 @@ static int nbd_opt_read_name(NBDClient *client, char **name, uint32_t *length,
     }
 
     local_name = g_malloc(len + 1);
-    ret = nbd_opt_read(client, local_name, len, errp);
+    ret = nbd_opt_read(client, local_name, len, true, errp);
     if (ret <= 0) {
         return ret;
     }
@@ -576,14 +586,14 @@ static int nbd_negotiate_handle_info(NBDClient *client, Error **errp)
     }
     trace_nbd_negotiate_handle_export_name_request(name);
 
-    rc = nbd_opt_read(client, &requests, sizeof(requests), errp);
+    rc = nbd_opt_read(client, &requests, sizeof(requests), false, errp);
     if (rc <= 0) {
         return rc;
     }
     requests = be16_to_cpu(requests);
     trace_nbd_negotiate_handle_info_requests(requests);
     while (requests--) {
-        rc = nbd_opt_read(client, &request, sizeof(request), errp);
+        rc = nbd_opt_read(client, &request, sizeof(request), false, errp);
         if (rc <= 0) {
             return rc;
         }
@@ -806,7 +816,7 @@ static int nbd_meta_pattern(NBDClient *client, const char *pattern, bool *match,
     assert(len);
 
     query = g_malloc(len);
-    ret = nbd_opt_read(client, query, len, errp);
+    ret = nbd_opt_read(client, query, len, true, errp);
     if (ret <= 0) {
         g_free(query);
         return ret;
@@ -943,7 +953,7 @@ static int nbd_negotiate_meta_query(NBDClient *client,
     char ns[5];
     uint32_t len;
 
-    ret = nbd_opt_read(client, &len, sizeof(len), errp);
+    ret = nbd_opt_read(client, &len, sizeof(len), false, errp);
     if (ret <= 0) {
         return ret;
     }
@@ -959,7 +969,7 @@ static int nbd_negotiate_meta_query(NBDClient *client,
     }
 
     len -= ns_len;
-    ret = nbd_opt_read(client, ns, ns_len, errp);
+    ret = nbd_opt_read(client, ns, ns_len, true, errp);
     if (ret <= 0) {
         return ret;
     }
@@ -1016,7 +1026,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
                             "export '%s' not present", sane_name);
     }
 
-    ret = nbd_opt_read(client, &nb_queries, sizeof(nb_queries), errp);
+    ret = nbd_opt_read(client, &nb_queries, sizeof(nb_queries), false, errp);
     if (ret <= 0) {
         return ret;
     }
Signature · 1 line
2.28.0
Reviewed-by on v2 2/5
#6 lore raw

Re: [PATCH v2 2/5] nbd/server: Reject embedded NUL in NBD strings

Quoted 9 lines · Eric Blake

The NBD spec is clear that any string sent from the client must not contain embedded NUL characters. If the client passes "a\0", we should reject that option request rather than act on "a".

Testing this is not possible with a compliant client, but I was able to use gdb to coerce libnbd into temporarily behaving as such a client.

Signature · 2 lines
Best regards,
Vladimir
author
#7 lore raw

PATCH v2 3/5 nbd: Simplify meta-context parsing

In reply to Eric Blake

We had a premature optimization of trying to read as little from the wire as possible while handling NBD_OPT_SET_META_CONTEXT in phases. But in reality, we HAVE to read the entire string from the client before we can get to the next command, and it is easier to just read it all at once than it is to read it in pieces. And once we do that, several functions end up no longer performing I/O, so they can drop length and errp parameters, and just return a bool instead of modifying through a pointer.

Our iotests still pass; I also checked that libnbd's testsuite (which covers more corner cases of odd meta context requests) still passes.

---
 nbd/server.c | 193 +++++++++++++++++++--------------------------------
 1 file changed, 70 insertions(+), 123 deletions(-)
nbd/server.c +70 -123
index 809f88ce6607..7271a09b5c2b 100644
@@ -1,5 +1,5 @@
 /*
- *  Copyright (C) 2016-2018 Red Hat, Inc.
+ *  Copyright (C) 2016-2020 Red Hat, Inc.
  *  Copyright (C) 2005  Anthony Liguori <[email protected]>
  *
  *  Network Block Device Server Side
@@ -797,135 +797,95 @@ static int nbd_negotiate_send_meta_context(NBDClient *client,
     return qio_channel_writev_all(client->ioc, iov, 2, errp) < 0 ? -EIO : 0;
 }
 
-/* Read strlen(@pattern) bytes, and set @match to true if they match @pattern.
- * @match is never set to false.
- *
- * Return -errno on I/O error, 0 if option was completely handled by
- * sending a reply about inconsistent lengths, or 1 on success.
- *
- * Note: return code = 1 doesn't mean that we've read exactly @pattern.
- * It only means that there are no errors.
+/*
+ * Return true if @query matches @pattern, or if @query is empty when
+ * the @client is performing _LIST_.
  */
-static int nbd_meta_pattern(NBDClient *client, const char *pattern, bool *match,
-                            Error **errp)
+static bool nbd_meta_empty_or_pattern(NBDClient *client, const char *pattern,
+                                      const char *query)
 {
-    int ret;
-    char *query;
-    size_t len = strlen(pattern);
-
-    assert(len);
-
-    query = g_malloc(len);
-    ret = nbd_opt_read(client, query, len, true, errp);
-    if (ret <= 0) {
-        g_free(query);
-        return ret;
+    if (!*query) {
+        trace_nbd_negotiate_meta_query_parse("empty");
+        return client->opt == NBD_OPT_LIST_META_CONTEXT;
     }
-
-    if (strncmp(query, pattern, len) == 0) {
+    if (strcmp(query, pattern) == 0) {
         trace_nbd_negotiate_meta_query_parse(pattern);
-        *match = true;
-    } else {
-        trace_nbd_negotiate_meta_query_skip("pattern not matched");
+        return true;
     }
-    g_free(query);
-
-    return 1;
+    trace_nbd_negotiate_meta_query_skip("pattern not matched");
+    return false;
 }
 
 /*
- * Read @len bytes, and set @match to true if they match @pattern, or if @len
- * is 0 and the client is performing _LIST_. @match is never set to false.
- *
- * Return -errno on I/O error, 0 if option was completely handled by
- * sending a reply about inconsistent lengths, or 1 on success.
- *
- * Note: return code = 1 doesn't mean that we've read exactly @pattern.
- * It only means that there are no errors.
+ * Return true and adjust @str in place if it begins with @prefix.
  */
-static int nbd_meta_empty_or_pattern(NBDClient *client, const char *pattern,
-                                     uint32_t len, bool *match, Error **errp)
+static bool nbd_strshift(const char **str, const char *prefix)
 {
-    if (len == 0) {
-        if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
-            *match = true;
-        }
-        trace_nbd_negotiate_meta_query_parse("empty");
-        return 1;
-    }
+    size_t len = strlen(prefix);
 
-    if (len != strlen(pattern)) {
-        trace_nbd_negotiate_meta_query_skip("different lengths");
-        return nbd_opt_skip(client, len, errp);
+    if (strncmp(*str, prefix, len) == 0) {
+        *str += len;
+        return true;
     }
-
-    return nbd_meta_pattern(client, pattern, match, errp);
+    return false;
 }
 
 /* nbd_meta_base_query
  *
  * Handle queries to 'base' namespace. For now, only the base:allocation
- * context is available.  'len' is the amount of text remaining to be read from
- * the current name, after the 'base:' portion has been stripped.
- *
- * Return -errno on I/O error, 0 if option was completely handled by
- * sending a reply about inconsistent lengths, or 1 on success.
+ * context is available.  Return true if @query has been handled.
  */
-static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta,
-                               uint32_t len, Error **errp)
+static bool nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta,
+                                const char *query)
 {
-    return nbd_meta_empty_or_pattern(client, "allocation", len,
-                                     &meta->base_allocation, errp);
+    if (!nbd_strshift(&query, "base:")) {
+        return false;
+    }
+    trace_nbd_negotiate_meta_query_parse("base:");
+
+    if (nbd_meta_empty_or_pattern(client, "allocation", query)) {
+        meta->base_allocation = true;
+    }
+    return true;
 }
 
-/* nbd_meta_bitmap_query
+/* nbd_meta_qemu_query
  *
- * Handle query to 'qemu:' namespace.
- * @len is the amount of text remaining to be read from the current name, after
- * the 'qemu:' portion has been stripped.
- *
- * Return -errno on I/O error, 0 if option was completely handled by
- * sending a reply about inconsistent lengths, or 1 on success. */
-static int nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
-                               uint32_t len, Error **errp)
+ * Handle queries to 'qemu' namespace. For now, only the qemu:dirty-bitmap:
+ * context is available.  Return true if @query has been handled.
+ */
+static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
+                                const char *query)
 {
-    bool dirty_bitmap = false;
-    size_t dirty_bitmap_len = strlen("dirty-bitmap:");
-    int ret;
-
-    if (!meta->exp->export_bitmap) {
-        trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
-        return nbd_opt_skip(client, len, errp);
+    if (!nbd_strshift(&query, "qemu:")) {
+        return false;
     }
+    trace_nbd_negotiate_meta_query_parse("qemu:");
 
-    if (len == 0) {
+    if (!*query) {
         if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+            meta->bitmap = !!meta->exp->export_bitmap;
+        }
+        trace_nbd_negotiate_meta_query_parse("empty");
+        return true;
+    }
+
+    if (nbd_strshift(&query, "dirty-bitmap:")) {
+        trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
+        if (!meta->exp->export_bitmap) {
+            trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
+            return true;
+        }
+        if (nbd_meta_empty_or_pattern(client,
+                                      meta->exp->export_bitmap_context +
+                                      strlen("qemu:dirty-bitmap:"), query)) {
             meta->bitmap = true;
         }
-        trace_nbd_negotiate_meta_query_parse("empty");
-        return 1;
+        return true;
     }
 
-    if (len < dirty_bitmap_len) {
-        trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:");
-        return nbd_opt_skip(client, len, errp);
-    }
-
-    len -= dirty_bitmap_len;
-    ret = nbd_meta_pattern(client, "dirty-bitmap:", &dirty_bitmap, errp);
-    if (ret <= 0) {
-        return ret;
-    }
-    if (!dirty_bitmap) {
-        trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:");
-        return nbd_opt_skip(client, len, errp);
-    }
-
-    trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
-
-    return nbd_meta_empty_or_pattern(
-            client, meta->exp->export_bitmap_context +
-            strlen("qemu:dirty_bitmap:"), len, &meta->bitmap, errp);
+    trace_nbd_negotiate_meta_query_skip("not dirty-bitmap");
+    return true;
 }
 
 /* nbd_negotiate_meta_query
@@ -935,22 +895,13 @@ static int nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
  *
  * The only supported namespaces are 'base' and 'qemu'.
  *
- * The function aims not wasting time and memory to read long unknown namespace
- * names.
- *
  * Return -errno on I/O error, 0 if option was completely handled by
  * sending a reply about inconsistent lengths, or 1 on success. */
 static int nbd_negotiate_meta_query(NBDClient *client,
                                     NBDExportMetaContexts *meta, Error **errp)
 {
-    /*
-     * Both 'qemu' and 'base' namespaces have length = 5 including a
-     * colon. If another length namespace is later introduced, this
-     * should certainly be refactored.
-     */
     int ret;
-    size_t ns_len = 5;
-    char ns[5];
+    g_autofree char *query = NULL;
     uint32_t len;
 
     ret = nbd_opt_read(client, &len, sizeof(len), false, errp);
@@ -963,27 +914,23 @@ static int nbd_negotiate_meta_query(NBDClient *client,
         trace_nbd_negotiate_meta_query_skip("length too long");
         return nbd_opt_skip(client, len, errp);
     }
-    if (len < ns_len) {
-        trace_nbd_negotiate_meta_query_skip("length too short");
-        return nbd_opt_skip(client, len, errp);
-    }
 
-    len -= ns_len;
-    ret = nbd_opt_read(client, ns, ns_len, true, errp);
+    query = g_malloc(len + 1);
+    ret = nbd_opt_read(client, query, len, true, errp);
     if (ret <= 0) {
         return ret;
     }
+    query[len] = '\0';
 
-    if (!strncmp(ns, "base:", ns_len)) {
-        trace_nbd_negotiate_meta_query_parse("base:");
-        return nbd_meta_base_query(client, meta, len, errp);
-    } else if (!strncmp(ns, "qemu:", ns_len)) {
-        trace_nbd_negotiate_meta_query_parse("qemu:");
-        return nbd_meta_qemu_query(client, meta, len, errp);
+    if (nbd_meta_base_query(client, meta, query)) {
+        return 1;
+    }
+    if (nbd_meta_qemu_query(client, meta, query)) {
+        return 1;
     }
 
     trace_nbd_negotiate_meta_query_skip("unknown namespace");
-    return nbd_opt_skip(client, len, errp);
+    return 1;
 }
 
 /* nbd_negotiate_meta_queries
Signature · 1 line
2.28.0
Reviewed-by on v2 3/5
#8 lore raw

Re: [PATCH v2 3/5] nbd: Simplify meta-context parsing

Quoted 9 lines · Eric Blake

We had a premature optimization of trying to read as little from the wire as possible while handling NBD_OPT_SET_META_CONTEXT in phases. But in reality, we HAVE to read the entire string from the client before we can get to the next command, and it is easier to just read it all at once than it is to read it in pieces. And once we do that, several functions end up no longer performing I/O, so they can drop length and errp parameters, and just return a bool instead of modifying through a pointer.

Quoted 3 lines

Our iotests still pass; I also checked that libnbd's testsuite (which covers more corner cases of odd meta context requests) still passes.

Also, do not advertise bitmaps meta context when bitmap export is not set.

Quoted 1 line
Signature · 2 lines
Best regards,
Vladimir
author
#9 lore raw

Re: [PATCH v2 3/5] nbd: Simplify meta-context parsing

Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Eric Blake

We had a premature optimization of trying to read as little from the wire as possible while handling NBD_OPT_SET_META_CONTEXT in phases. But in reality, we HAVE to read the entire string from the client before we can get to the next command, and it is easier to just read it all at once than it is to read it in pieces.  And once we do that, several functions end up no longer performing I/O, so they can drop length and errp parameters, and just return a bool instead of modifying through a pointer.

Quoted 5 lines
Quoted 3 lines

Our iotests still pass; I also checked that libnbd's testsuite (which covers more corner cases of odd meta context requests) still passes.

Also, do not advertise bitmaps meta context when bitmap export is not set.

That was already there, although seeing the logic change is tricky and the trace messages change:

nbd/server.c static int nbd_negotiate_send_meta_context(NBDClient *client, · line 898

Quoted 10 lines
nbd/server.c +2 -7
+static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
+                                const char *query)
 {
-    bool dirty_bitmap = false;
-    size_t dirty_bitmap_len = strlen("dirty-bitmap:");
-    int ret;
-
-    if (!meta->exp->export_bitmap) {
-        trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
-        return nbd_opt_skip(client, len, errp);

Old code returned early if there was no bitmap export set

nbd/server.c static int nbd_negotiate_send_meta_context(NBDClient *client, · line 867

Quoted 9 lines
nbd/server.c +5 -1
+    if (!nbd_strshift(&query, "qemu:")) {
+        return false;
     }
+    trace_nbd_negotiate_meta_query_parse("qemu:");
 
-    if (len == 0) {
+    if (!*query) {
         if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+            meta->bitmap = !!meta->exp->export_bitmap;

while the new code has to handle it specifically. I'll tweak the commit message to mention the change in trace messages, even when the end behavior is the same.

Quoted 5 lines
Quoted 2 lines
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#10 lore raw

PATCH v2 4/5 nbd: Add new qemu:allocation-depth metacontext

In reply to Eric Blake

'qemu-img map' provides a way to determine which extents of an image come from the top layer vs. inherited from a backing chain. This is useful information worth exposing over NBD. There is a proposal to add a QMP command block-dirty-bitmap-populate which can create a dirty bitmap that reflects allocation information, at which point qemu:dirty-bitmap:NAME can expose that information via the creation of a temporary bitmap, but we can shorten the effort by adding a new qemu:allocation-depth context that does the same thing without an intermediate bitmap (this patch does not eliminate the need for that proposal, as it will have other uses as well).

For this patch, I just encoded a tri-state value (unallocated, from this layer, from any of the backing layers); an obvious extension would be to provide the actual depth in bits 31-4 while keeping bits 1-0 as a tri-state (leaving bits 3-2 unused, for ease of reading depth from a hex number). But this extension would require bdrv_is_allocated_above to return a depth number.

Note that this patch does not actually enable any way to request a server to enable this context; that will come in the next patch.

---
 docs/interop/nbd.txt |  22 +++++++--
 include/block/nbd.h  |   8 +++-
 nbd/server.c         | 105 ++++++++++++++++++++++++++++++++++++++++---
 3 files changed, 125 insertions(+), 10 deletions(-)
docs/interop/nbd.txt +18 -4
index f3b3cacc9621..56efec7aee12 100644
@@ -17,9 +17,9 @@ namespace "qemu".
 
 == "qemu" namespace ==
 
-The "qemu" namespace currently contains only one type of context,
-related to exposing the contents of a dirty bitmap alongside the
-associated disk contents.  That context has the following form:
+The "qemu" namespace currently contains two types of context.  The
+first is related to exposing the contents of a dirty bitmap alongside
+the associated disk contents.  That context has the following form:
 
     qemu:dirty-bitmap:<dirty-bitmap-export-name>
 
@@ -28,8 +28,21 @@ in reply for NBD_CMD_BLOCK_STATUS:
 
     bit 0: NBD_STATE_DIRTY, means that the extent is "dirty"
 
+The second is related to exposing the source of various extents within
+the image, with a single context named:
+
+    qemu:allocation-depth
+
+In the allocation depth context, bits 0 and 1 form a tri-state value:
+
+    bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, means the extent is unallocated
+    bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in this image
+    bit 1 set: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
+               backing layer
+
 For NBD_OPT_LIST_META_CONTEXT the following queries are supported
-in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>":
+in addition to the specific "qemu:allocation-depth" and
+"qemu:dirty-bitmap:<dirty-bitmap-export-name>":
 
 * "qemu:" - returns list of all available metadata contexts in the
             namespace.
@@ -55,3 +68,4 @@ the operation of that feature.
 NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
 * 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports,
 NBD_CMD_FLAG_FAST_ZERO
+* 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth"
include/block/nbd.h +7 -1
index 3dd9a04546ec..06208bc25027 100644
@@ -1,5 +1,5 @@
 /*
- *  Copyright (C) 2016-2019 Red Hat, Inc.
+ *  Copyright (C) 2016-2020 Red Hat, Inc.
  *  Copyright (C) 2005  Anthony Liguori <[email protected]>
  *
  *  Network Block Device
@@ -259,6 +259,12 @@ enum {
 /* Extent flags for qemu:dirty-bitmap in NBD_REPLY_TYPE_BLOCK_STATUS */
 #define NBD_STATE_DIRTY (1 << 0)
 
+/* Extent flags for qemu:allocation-depth in NBD_REPLY_TYPE_BLOCK_STATUS */
+#define NBD_STATE_DEPTH_UNALLOC (0 << 0)
+#define NBD_STATE_DEPTH_LOCAL   (1 << 0)
+#define NBD_STATE_DEPTH_BACKING (2 << 0)
+#define NBD_STATE_DEPTH_MASK    (0x3)
+
 static inline bool nbd_reply_type_is_error(int type)
 {
     return type & (1 << 15);
nbd/server.c +100 -5

195 lines not on this page · raw message

Signature · 1 line
2.28.0
#11 lore raw

Re: [PATCH v2 4/5] nbd: Add new qemu:allocation-depth metacontext

Quoted 48 lines · Eric Blake

'qemu-img map' provides a way to determine which extents of an image come from the top layer vs. inherited from a backing chain. This is useful information worth exposing over NBD. There is a proposal to add a QMP command block-dirty-bitmap-populate which can create a dirty bitmap that reflects allocation information, at which point qemu:dirty-bitmap:NAME can expose that information via the creation of a temporary bitmap, but we can shorten the effort by adding a new qemu:allocation-depth context that does the same thing without an intermediate bitmap (this patch does not eliminate the need for that proposal, as it will have other uses as well).

For this patch, I just encoded a tri-state value (unallocated, from this layer, from any of the backing layers); an obvious extension would be to provide the actual depth in bits 31-4 while keeping bits 1-0 as a tri-state (leaving bits 3-2 unused, for ease of reading depth from a hex number). But this extension would require bdrv_is_allocated_above to return a depth number.

Note that this patch does not actually enable any way to request a server to enable this context; that will come in the next patch.

---
  docs/interop/nbd.txt |  22 +++++++--
  include/block/nbd.h  |   8 +++-
  nbd/server.c         | 105 ++++++++++++++++++++++++++++++++++++++++---
  3 files changed, 125 insertions(+), 10 deletions(-)
docs/interop/nbd.txt +3 -3
index f3b3cacc9621..56efec7aee12 100644
@@ -17,9 +17,9 @@ namespace "qemu".
 
  == "qemu" namespace ==
 
-The "qemu" namespace currently contains only one type of context,
-related to exposing the contents of a dirty bitmap alongside the
-associated disk contents.  That context has the following form:
+The "qemu" namespace currently contains two types of context.  The
+first is related to exposing the contents of a dirty bitmap alongside
+the associated disk contents.  That context has the following form:
 
      qemu:dirty-bitmap:<dirty-bitmap-export-name>
 
@@ -28,8 +28,21 @@ in reply for NBD_CMD_BLOCK_STATUS:
 
      bit 0: NBD_STATE_DIRTY, means that the extent is "dirty"

docs/interop/nbd.txt in reply for NBD_CMD_BLOCK_STATUS: · line 39

Quoted 10 lines
docs/interop/nbd.txt +9 -0
 
+The second is related to exposing the source of various extents within
+the image, with a single context named:
+
+    qemu:allocation-depth
+
+In the allocation depth context, bits 0 and 1 form a tri-state value:
+
+    bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, means the extent is unallocated
+    bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in this image

Maybe, s/this/the top level/, as, what is "this" for NBD client?

Quoted 82 lines
docs/interop/nbd.txt +6 -1
+    bit 1 set: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
+               backing layer
+
  For NBD_OPT_LIST_META_CONTEXT the following queries are supported
-in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>":
+in addition to the specific "qemu:allocation-depth" and
+"qemu:dirty-bitmap:<dirty-bitmap-export-name>":
 
  * "qemu:" - returns list of all available metadata contexts in the
              namespace.
@@ -55,3 +68,4 @@ the operation of that feature.
  NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
  * 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports,
  NBD_CMD_FLAG_FAST_ZERO
+* 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth"
include/block/nbd.h +7 -1
index 3dd9a04546ec..06208bc25027 100644
@@ -1,5 +1,5 @@
  /*
- *  Copyright (C) 2016-2019 Red Hat, Inc.
+ *  Copyright (C) 2016-2020 Red Hat, Inc.
   *  Copyright (C) 2005  Anthony Liguori <[email protected]>
   *
   *  Network Block Device
@@ -259,6 +259,12 @@ enum {
  /* Extent flags for qemu:dirty-bitmap in NBD_REPLY_TYPE_BLOCK_STATUS */
  #define NBD_STATE_DIRTY (1 << 0)
 
+/* Extent flags for qemu:allocation-depth in NBD_REPLY_TYPE_BLOCK_STATUS */
+#define NBD_STATE_DEPTH_UNALLOC (0 << 0)
+#define NBD_STATE_DEPTH_LOCAL   (1 << 0)
+#define NBD_STATE_DEPTH_BACKING (2 << 0)
+#define NBD_STATE_DEPTH_MASK    (0x3)
+
  static inline bool nbd_reply_type_is_error(int type)
  {
      return type & (1 << 15);
nbd/server.c +5 -2
index 7271a09b5c2b..830b21000be3 100644
@@ -27,7 +27,8 @@
  #include "qemu/units.h"
 
  #define NBD_META_ID_BASE_ALLOCATION 0
-#define NBD_META_ID_DIRTY_BITMAP 1
+#define NBD_META_ID_ALLOCATION_DEPTH 1
+#define NBD_META_ID_DIRTY_BITMAP 2
 
  /*
   * NBD_MAX_BLOCK_STATUS_EXTENTS: 1 MiB of extents data. An empirical
@@ -94,6 +95,7 @@ struct NBDExport {
      BlockBackend *eject_notifier_blk;
      Notifier eject_notifier;
 
+    bool alloc_context;
      BdrvDirtyBitmap *export_bitmap;
      char *export_bitmap_context;
  };
@@ -108,6 +110,7 @@ typedef struct NBDExportMetaContexts {
      bool valid; /* means that negotiation of the option finished without
                     errors */
      bool base_allocation; /* export base:allocation context (block status) */
+    bool allocation_depth; /* export qemu:allocation-depth */
      bool bitmap; /* export qemu:dirty-bitmap:<export bitmap name> */
  } NBDExportMetaContexts;
 
@@ -806,7 +809,7 @@ static bool nbd_meta_empty_or_pattern(NBDClient *client, const char *pattern,
  {
      if (!*query) {
          trace_nbd_negotiate_meta_query_parse("empty");
-        return client->opt == NBD_OPT_LIST_META_CONTEXT;
+        return !*pattern || client->opt == NBD_OPT_LIST_META_CONTEXT;
      }
      if (strcmp(query, pattern) == 0) {
          trace_nbd_negotiate_meta_query_parse(pattern);
@@ -864,12 +867,21 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
 
      if (!*query) {
          if (client->opt == NBD_OPT_LIST_META_CONTEXT) {

nbd/server.c static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, · line 879

Quoted 10 lines
nbd/server.c +4 -0
+            meta->allocation_depth = meta->exp->alloc_context;
              meta->bitmap = !!meta->exp->export_bitmap;
          }
          trace_nbd_negotiate_meta_query_parse("empty");
          return true;
      }
 
+    if (nbd_strshift(&query, "allocation-depth")) {
+        trace_nbd_negotiate_meta_query_parse("allocation-depth");
+        if (nbd_meta_empty_or_pattern(client, "", query)) {

How much it differs from "if (!*query) {", I don't see...

nbd/server.c static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, · line 883

Quoted 4 lines
nbd/server.c +4 -0
+            meta->allocation_depth = meta->exp->alloc_context;
+        }
+        return true;
+    }

why not just

 if (!strcmp(query, "allocation-depth")) {
    meta->allocation_depth = meta->exp->alloc_context;
    return true;

}

?

With your code you also parse something like "allocation-depth-extended".. Is it on purpose?

nbd/server.c static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, · line 887

Quoted 4 lines
nbd/server.c +1 -0
+
      if (nbd_strshift(&query, "dirty-bitmap:")) {
          trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
          if (!meta->exp->export_bitmap) {

Also, "trace_nbd_negotiate_meta_query_skip("not dirty-bitmap"); " at function end should now be something like trace_nbd_negotiate_meta_query_skip("unknown context in qemu: namespace");

Quoted 78 lines
nbd/server.c +53 -0
@@ -984,6 +996,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
      if (client->opt == NBD_OPT_LIST_META_CONTEXT && !nb_queries) {
          /* enable all known contexts */
          meta->base_allocation = true;
+        meta->allocation_depth = meta->exp->alloc_context;
          meta->bitmap = !!meta->exp->export_bitmap;
      } else {
          for (i = 0; i < nb_queries; ++i) {
@@ -1003,6 +1016,15 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
          }
      }
 
+    if (meta->allocation_depth) {
+        ret = nbd_negotiate_send_meta_context(client, "qemu:allocation-depth",
+                                              NBD_META_ID_ALLOCATION_DEPTH,
+                                              errp);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
      if (meta->bitmap) {
          ret = nbd_negotiate_send_meta_context(client,
                                                meta->exp->export_bitmap_context,
@@ -1961,6 +1983,40 @@ static int blockstatus_to_extents(BlockDriverState *bs, uint64_t offset,
      return 0;
  }
 
+static int blockalloc_to_extents(BlockDriverState *bs, uint64_t offset,
+                                 uint64_t bytes, NBDExtentArray *ea)
+{
+    while (bytes) {
+        uint32_t flags;
+        int64_t num;
+        int ret = bdrv_is_allocated(bs, offset, bytes, &num);
+
+        if (ret < 0) {
+            return ret;
+        }
+
+        if (ret == 1) {
+            flags = NBD_STATE_DEPTH_LOCAL;
+        } else {
+            ret = bdrv_is_allocated_above(bs, NULL, false, offset, num,
+                                          &num);
+            if (ret < 0) {
+                return ret;
+            }
+            flags = ret ? NBD_STATE_DEPTH_BACKING : NBD_STATE_DEPTH_UNALLOC;
+        }
+
+        if (nbd_extent_array_add(ea, num, flags) < 0) {
+            return 0;
+        }
+
+        offset += num;
+        bytes -= num;
+    }
+
+    return 0;
+}
+
  /*
   * nbd_co_send_extents
   *
@@ -2009,6 +2065,26 @@ static int nbd_co_send_block_status(NBDClient *client, uint64_t handle,
      return nbd_co_send_extents(client, handle, ea, last, context_id, errp);
  }
 
+/* Get allocation depth from the exported device and send it to the client */
+static int nbd_co_send_block_alloc(NBDClient *client, uint64_t handle,
+                                   BlockDriverState *bs, uint64_t offset,
+                                   uint32_t length, bool dont_fragment,
+                                   bool last, uint32_t context_id,
+                                   Error **errp)
+{
+    int ret;
+    unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BLOCK_STATUS_EXTENTS;

nbd/server.c static int nbd_co_send_block_status(NBDClient *client, uint64_t handle, · line 2086

Quoted 10 lines
nbd/server.c +10 -0
+    g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents);
+
+    ret = blockalloc_to_extents(bs, offset, length, ea);
+    if (ret < 0) {
+        return nbd_co_send_structured_error(
+                client, handle, -ret, "can't get block status", errp);
+    }
+
+    return nbd_co_send_extents(client, handle, ea, last, context_id, errp);
+}

The function is a copy of nbd_co_send_block_status()..

Probably, we can reuse nbd_co_send_block_status(), and just call blockstatus_to_extents() or blockalloc_to_extents() depending on context_id parameter.

Still, I'm OK with it as is.

Quoted 62 lines
nbd/server.c +23 -3
+
  /* Populate @ea from a dirty bitmap. */
  static void bitmap_to_extents(BdrvDirtyBitmap *bitmap,
                                uint64_t offset, uint64_t length,
@@ -2335,16 +2411,20 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
          }
          if (client->export_meta.valid &&
              (client->export_meta.base_allocation ||
+             client->export_meta.allocation_depth ||
               client->export_meta.bitmap))
          {
              bool dont_fragment = request->flags & NBD_CMD_FLAG_REQ_ONE;
+            int contexts_remaining = client->export_meta.base_allocation +
+                client->export_meta.allocation_depth +
+                client->export_meta.bitmap;
 
              if (client->export_meta.base_allocation) {
                  ret = nbd_co_send_block_status(client, request->handle,
                                                 blk_bs(exp->common.blk),
                                                 request->from,
                                                 request->len, dont_fragment,
-                                               !client->export_meta.bitmap,
+                                               !--contexts_remaining,
                                                 NBD_META_ID_BASE_ALLOCATION,
                                                 errp);
                  if (ret < 0) {
@@ -2352,17 +2432,32 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
                  }
              }
 
+            if (client->export_meta.allocation_depth) {
+                ret = nbd_co_send_block_alloc(client, request->handle,
+                                              blk_bs(exp->common.blk),
+                                              request->from, request->len,
+                                              dont_fragment,
+                                              !--contexts_remaining,
+                                              NBD_META_ID_ALLOCATION_DEPTH,
+                                              errp);
+                if (ret < 0) {
+                    return ret;
+                }
+            }
+
              if (client->export_meta.bitmap) {
                  ret = nbd_co_send_bitmap(client, request->handle,
                                           client->exp->export_bitmap,
                                           request->from, request->len,
-                                         dont_fragment,
-                                         true, NBD_META_ID_DIRTY_BITMAP, errp);
+                                         dont_fragment, !--contexts_remaining,
+                                         NBD_META_ID_DIRTY_BITMAP, errp);
                  if (ret < 0) {
                      return ret;
                  }
              }
 
+            assert(!contexts_remaining);
+
              return 0;
          } else {
              return nbd_send_generic_reply(client, request->handle, -EINVAL,
Signature · 2 lines
Best regards,
Vladimir
author
#12 lore raw

Re: [PATCH v2 4/5] nbd: Add new qemu:allocation-depth metacontext

Quoted 23 lines · Vladimir Sementsov-Ogievskiy
Quoted 21 lines · Eric Blake

'qemu-img map' provides a way to determine which extents of an image come from the top layer vs. inherited from a backing chain.  This is useful information worth exposing over NBD.  There is a proposal to add a QMP command block-dirty-bitmap-populate which can create a dirty bitmap that reflects allocation information, at which point qemu:dirty-bitmap:NAME can expose that information via the creation of a temporary bitmap, but we can shorten the effort by adding a new qemu:allocation-depth context that does the same thing without an intermediate bitmap (this patch does not eliminate the need for that proposal, as it will have other uses as well).

For this patch, I just encoded a tri-state value (unallocated, from this layer, from any of the backing layers); an obvious extension would be to provide the actual depth in bits 31-4 while keeping bits 1-0 as a tri-state (leaving bits 3-2 unused, for ease of reading depth from a hex number).  But this extension would require bdrv_is_allocated_above to return a depth number.

Note that this patch does not actually enable any way to request a server to enable this context; that will come in the next patch.

Quoted 8 lines
Quoted 6 lines
docs/interop/nbd.txt +3 -0
+In the allocation depth context, bits 0 and 1 form a tri-state value:
+
+    bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, means the extent is

unallocated
+    bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in this image

Maybe, s/this/the top level/, as, what is "this" for NBD client?

Sure.

Quoted 4 lines

nbd/server.c static bool nbd_meta_qemu_query(NBDClient · line 869

Quoted 4 lines
nbd/server.c +0 -0
@@ -864,12 +867,21 @@ static bool nbd_meta_qemu_query(NBDClient
 *client, NBDExportMetaContexts *meta,
 
       if (!*query) {

We get here for "qemu:".

Quoted 9 lines

nbd/server.c static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, · line 877

Quoted 9 lines
nbd/server.c +2 -0
           if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+            meta->allocation_depth = meta->exp->alloc_context;
               meta->bitmap = !!meta->exp->export_bitmap;
           }
           trace_nbd_negotiate_meta_query_parse("empty");
           return true;
       }
 
+    if (nbd_strshift(&query, "allocation-depth")) {

We get here for "qemu:allocation-depth", and as you pointed out, "qemu:allocation-depth-extended".

Quoted 4 lines

nbd/server.c static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, · line 879

Quoted 2 lines
nbd/server.c +2 -0
+        trace_nbd_negotiate_meta_query_parse("allocation-depth");
+        if (nbd_meta_empty_or_pattern(client, "", query)) {

How much it differs from "if (!*query) {", I don't see...

The nbd_meta_empty_or_pattern returns false for "qemu:allocation-depth-extended"; it only returns true for "qemu:allocation-depth". But, as you pointed out,

Quoted 6 lines

nbd/server.c static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, · line 883

Quoted 5 lines
nbd/server.c +4 -0
+            meta->allocation_depth = meta->exp->alloc_context;
+        }
+        return true;
+    }
Quoted 8 lines

why not just

 if (!strcmp(query, "allocation-depth")) { meta->allocation_depth = meta->exp->alloc_context; return true;
 }

?

that does seem shorter.

Quoted 3 lines

With your code you also parse something like
"allocation-depth-extended".. Is it on purpose?

The string is parsed, but does not affect meta->XXX, which means nothing gets advertised to the client. The trace messages might differ, but behavior is correct.

Quoted 11 lines
Quoted 5 lines
nbd/server.c +1 -0
+
       if (nbd_strshift(&query, "dirty-bitmap:")) {
           trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");

          if (!meta->exp->export_bitmap) {

Also, "trace_nbd_negotiate_meta_query_skip("not dirty-bitmap"); " at function end should
now be something like trace_nbd_negotiate_meta_query_skip("unknown context in qemu: namespace");

Good idea.

Quoted 27 lines
Quoted 21 lines

+/* Get allocation depth from the exported device and send it to the client */
+static int nbd_co_send_block_alloc(NBDClient *client, uint64_t handle, +                                   BlockDriverState *bs, uint64_t offset,
+                                   uint32_t length, bool dont_fragment, +                                   bool last, uint32_t context_id, +                                   Error **errp)
+{
+    int ret;
+    unsigned int nb_extents = dont_fragment ? 1 :
NBD_MAX_BLOCK_STATUS_EXTENTS;
+    g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents);

nbd/server.c +8 -0
+
+    ret = blockalloc_to_extents(bs, offset, length, ea);
+    if (ret < 0) {
+        return nbd_co_send_structured_error(
+                client, handle, -ret, "can't get block status", errp);
+    }
+
+    return nbd_co_send_extents(client, handle, ea, last, context_id,
Quoted 2 lines

errp);
+}

The function is a copy of nbd_co_send_block_status()..

Quoted 3 lines

Probably, we can reuse nbd_co_send_block_status(), and just call blockstatus_to_extents()
or blockalloc_to_extents() depending on context_id parameter.

Nice idea to reduce the duplication.

Quoted 3 lines

Still, I'm OK with it as is.

So is that Reviewed-by:, or do I need to post v3 with my tweaks in response to your comments?

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#13 lore raw

Re: [PATCH v2 4/5] nbd: Add new qemu:allocation-depth metacontext

Quoted 132 lines · Eric Blake
Quoted 23 lines · Vladimir Sementsov-Ogievskiy
Quoted 21 lines · Eric Blake

'qemu-img map' provides a way to determine which extents of an image come from the top layer vs. inherited from a backing chain.  This is useful information worth exposing over NBD.  There is a proposal to add a QMP command block-dirty-bitmap-populate which can create a dirty bitmap that reflects allocation information, at which point qemu:dirty-bitmap:NAME can expose that information via the creation of a temporary bitmap, but we can shorten the effort by adding a new qemu:allocation-depth context that does the same thing without an intermediate bitmap (this patch does not eliminate the need for that proposal, as it will have other uses as well).

For this patch, I just encoded a tri-state value (unallocated, from this layer, from any of the backing layers); an obvious extension would be to provide the actual depth in bits 31-4 while keeping bits 1-0 as a tri-state (leaving bits 3-2 unused, for ease of reading depth from a hex number).  But this extension would require bdrv_is_allocated_above to return a depth number.

Note that this patch does not actually enable any way to request a server to enable this context; that will come in the next patch.

Quoted 8 lines
Quoted 6 lines
(quoted diff) +3 -0
+In the allocation depth context, bits 0 and 1 form a tri-state value:
+
+    bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, means the extent is

unallocated
+    bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in this image

Maybe, s/this/the top level/, as, what is "this" for NBD client?

Sure.

Quoted 4 lines

Review static bool nbd_meta_qemu_query(NBDClient · line 869

Quoted 4 lines
(quoted diff) +0 -0
@@ -864,12 +867,21 @@ static bool nbd_meta_qemu_query(NBDClient
 *client, NBDExportMetaContexts *meta,
 
       if (!*query) {

We get here for "qemu:".

Quoted 9 lines

Review static bool nbd_meta_qemu_query(NBDClient · line 878

Quoted 9 lines
(quoted diff) +2 -0
           if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+            meta->allocation_depth = meta->exp->alloc_context;
               meta->bitmap = !!meta->exp->export_bitmap;
           }
           trace_nbd_negotiate_meta_query_parse("empty");
           return true;
       }
 
+    if (nbd_strshift(&query, "allocation-depth")) {

We get here for "qemu:allocation-depth", and as you pointed out, "qemu:allocation-depth-extended".

Quoted 4 lines

Review static bool nbd_meta_qemu_query(NBDClient · line 880

Quoted 2 lines
(quoted diff) +2 -0
+        trace_nbd_negotiate_meta_query_parse("allocation-depth");
+        if (nbd_meta_empty_or_pattern(client, "", query)) {

How much it differs from "if (!*query) {", I don't see...

The nbd_meta_empty_or_pattern returns false for "qemu:allocation-depth-extended"; it only returns true for "qemu:allocation-depth". But, as you pointed out,

Quoted 6 lines

Review static bool nbd_meta_qemu_query(NBDClient · line 884

Quoted 5 lines
(quoted diff) +4 -0
+            meta->allocation_depth = meta->exp->alloc_context;
+        }
+        return true;
+    }
Quoted 8 lines

why not just

 if (!strcmp(query, "allocation-depth")) { meta->allocation_depth = meta->exp->alloc_context; return true;
 }

?

that does seem shorter.

Quoted 3 lines

With your code you also parse something like
"allocation-depth-extended".. Is it on purpose?

The string is parsed, but does not affect meta->XXX, which means nothing gets advertised to the client. The trace messages might differ, but behavior is correct.

Quoted 11 lines
Quoted 5 lines
(quoted diff) +1 -0
+
       if (nbd_strshift(&query, "dirty-bitmap:")) {
           trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");

          if (!meta->exp->export_bitmap) {

Also, "trace_nbd_negotiate_meta_query_skip("not dirty-bitmap"); " at function end should
now be something like trace_nbd_negotiate_meta_query_skip("unknown context in qemu: namespace");

Good idea.

Quoted 27 lines
Quoted 21 lines

+/* Get allocation depth from the exported device and send it to the client */
+static int nbd_co_send_block_alloc(NBDClient *client, uint64_t handle, +                                   BlockDriverState *bs, uint64_t offset,
+                                   uint32_t length, bool dont_fragment, +                                   bool last, uint32_t context_id, +                                   Error **errp)
+{
+    int ret;
+    unsigned int nb_extents = dont_fragment ? 1 :
NBD_MAX_BLOCK_STATUS_EXTENTS;
+    g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents);

(quoted diff) +8 -0
+
+    ret = blockalloc_to_extents(bs, offset, length, ea);
+    if (ret < 0) {
+        return nbd_co_send_structured_error(
+                client, handle, -ret, "can't get block status", errp);
+    }
+
+    return nbd_co_send_extents(client, handle, ea, last, context_id,
Quoted 2 lines

errp);
+}

The function is a copy of nbd_co_send_block_status()..

Quoted 9 lines
Quoted 3 lines

Probably, we can reuse nbd_co_send_block_status(), and just call blockstatus_to_extents()
or blockalloc_to_extents() depending on context_id parameter.

Nice idea to reduce the duplication.

Quoted 3 lines

Still, I'm OK with it as is.

it was only about the duplicated function

Quoted 4 lines

So is that Reviewed-by:, or do I need to post v3 with my tweaks in response to your comments?

Honestly it wasn't.. So, I found some things to discuss, and it's possible that I've looked through with less attention. Hm. So, with v3 or without, I've have to lookthrough it again..

My suggested diff:

docs/interop/nbd.txt +2 -1
index 56efec7aee..a0d91ff08b 100644
@@ -36,7 +36,8 @@ the image, with a single context named:
 In the allocation depth context, bits 0 and 1 form a tri-state value:
 
     bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, means the extent is unallocated
-    bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in this image
+    bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the top
+               level image
     bit 1 set: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
                backing layer
 
include/block/nbd.h +1 -1
index 06208bc250..2d7165960d 100644
@@ -262,7 +262,7 @@ enum {
 /* Extent flags for qemu:allocation-depth in NBD_REPLY_TYPE_BLOCK_STATUS */
 #define NBD_STATE_DEPTH_UNALLOC (0 << 0)
 #define NBD_STATE_DEPTH_LOCAL   (1 << 0)
-#define NBD_STATE_DEPTH_BACKING (2 << 0)
+#define NBD_STATE_DEPTH_BACKING (1 << 1)
 #define NBD_STATE_DEPTH_MASK    (0x3)
 
 static inline bool nbd_reply_type_is_error(int type)
nbd/server.c +21 -34

93 lines not on this page · raw message

with it:

Signature · 2 lines
Best regards,
Vladimir
author
#14 lore raw

PATCH v2 5/5 nbd: Add 'qemu-nbd -A' to expose allocation depth

In reply to Eric Blake

Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add.

qemu as client can be hacked into viewing this new context by using the now-misnamed x-dirty-bitmap option when creating an NBD blockdev (even though our x- naming means we could rename it, I did not think it worth breaking back-compat of tools that have been using it while waiting for a better solution). It is worth noting the decoding of how such context information will appear in 'qemu-img map --output=json':

NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true

libnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;)

---
 docs/tools/qemu-nbd.rst    |  6 ++++
 qapi/block-core.json       |  7 ++--
 qapi/block-export.json     |  6 +++-
 blockdev-nbd.c             |  2 ++
 nbd/server.c               |  2 ++
 qemu-nbd.c                 | 14 ++++++--
 tests/qemu-iotests/309     | 73 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/309.out | 22 ++++++++++++
 tests/qemu-iotests/group   |  1 +
 9 files changed, 127 insertions(+), 6 deletions(-)
 create mode 100755 tests/qemu-iotests/309
 create mode 100644 tests/qemu-iotests/309.out
docs/tools/qemu-nbd.rst +6 -0
index 667861cb22e9..0e545a97cfa3 100644
@@ -72,6 +72,12 @@ driver options if ``--image-opts`` is specified.
 
   Export the disk as read-only.
 
+.. option:: -A, --allocation-depth
+
+  Expose allocation depth information via the
+  ``qemu:allocation-depth`` context accessible through
+  NBD_OPT_SET_META_CONTEXT.
+
 .. option:: -B, --bitmap=NAME
 
   If *filename* has a qcow2 persistent bitmap *NAME*, expose
qapi/block-core.json +5 -2
index d620bd1302b2..0379eb992db8 100644
@@ -3874,9 +3874,12 @@
 #
 # @tls-creds: TLS credentials ID
 #
-# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of
+# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" or
+#                  "qemu:allocation-depth" to query in place of the
 #                  traditional "base:allocation" block status (see
-#                  NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0)
+#                  NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and
+#                  yes, naming this option x-context would have made
+#                  more sense) (since 3.0)
 #
 # @reconnect-delay: On an unexpected disconnect, the nbd client tries to
 #                   connect again until succeeding or encountering a serious
qapi/block-export.json +5 -1
index 2291d6cb0cbc..45e43984b11d 100644
@@ -78,11 +78,15 @@
 #          NBD client can use NBD_OPT_SET_META_CONTEXT with
 #          "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0)
 #
+# @alloc: Also export the allocation map for @device, so the NBD client
+#         can use NBD_OPT_SET_META_CONTEXT with "qemu:allocation-depth"
+#         to inspect allocation details. (since 5.2)
+#
 # Since: 5.0
 ##
 { 'struct': 'BlockExportOptionsNbd',
   'data': { '*name': 'str', '*description': 'str',
-            '*bitmap': 'str' } }
+            '*bitmap': 'str', '*alloc': 'bool' } }
 
 ##
 # @NbdServerAddOptions:
blockdev-nbd.c +2 -0
index 8174023e5c47..f9012f93e2bb 100644
@@ -212,6 +212,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
             .description        = g_strdup(arg->description),
             .has_bitmap         = arg->has_bitmap,
             .bitmap             = g_strdup(arg->bitmap),
+            .has_alloc          = arg->alloc,
+            .alloc              = arg->alloc,
         },
     };
 
nbd/server.c +2 -0
index 830b21000be3..11cdc2eab0b3 100644
@@ -1598,6 +1598,8 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,
         assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
     }
 
+    exp->alloc_context = arg->alloc;
+
     blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
 
     QTAILQ_INSERT_TAIL(&exports, exp, next);
qemu-nbd.c +11 -3
index e7520261134f..3a92e00464de 100644
@@ -99,6 +99,7 @@ static void usage(const char *name)
 "\n"
 "Exposing part of the image:\n"
 "  -o, --offset=OFFSET       offset into the image\n"
+"  -A, --allocation-depth    expose the allocation depth\n"
 "  -B, --bitmap=NAME         expose a persistent dirty bitmap\n"
 "\n"
 "General purpose options:\n"
@@ -519,7 +520,7 @@ int main(int argc, char **argv)
     char *device = NULL;
     QemuOpts *sn_opts = NULL;
     const char *sn_id_or_name = NULL;
-    const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L";
+    const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:AB:L";
     struct option lopt[] = {
         { "help", no_argument, NULL, 'h' },
         { "version", no_argument, NULL, 'V' },
@@ -528,6 +529,7 @@ int main(int argc, char **argv)
         { "socket", required_argument, NULL, 'k' },
         { "offset", required_argument, NULL, 'o' },
         { "read-only", no_argument, NULL, 'r' },
+        { "allocation-depth", no_argument, NULL, 'A' },
         { "bitmap", required_argument, NULL, 'B' },
         { "connect", required_argument, NULL, 'c' },
         { "disconnect", no_argument, NULL, 'd' },
@@ -569,6 +571,7 @@ int main(int argc, char **argv)
     QDict *options = NULL;
     const char *export_name = NULL; /* defaults to "" later for server mode */
     const char *export_description = NULL;
+    bool alloc_depth = false;
     const char *bitmap = NULL;
     const char *tlscredsid = NULL;
     bool imageOpts = false;
@@ -693,6 +696,9 @@ int main(int argc, char **argv)
             readonly = true;
             flags &= ~BDRV_O_RDWR;
             break;
+        case 'A':
+            alloc_depth = true;
+            break;
         case 'B':
             bitmap = optarg;
             break;
@@ -790,8 +796,8 @@ int main(int argc, char **argv)
             exit(EXIT_FAILURE);
         }
         if (export_name || export_description || dev_offset ||
-            device || disconnect || fmt || sn_id_or_name || bitmap ||
-            seen_aio || seen_discard || seen_cache) {
+            device || disconnect || fmt || sn_id_or_name || alloc_depth ||
+            bitmap || seen_aio || seen_discard || seen_cache) {
             error_report("List mode is incompatible with per-device settings");
             exit(EXIT_FAILURE);
         }
@@ -1077,6 +1083,8 @@ int main(int argc, char **argv)
             .description        = g_strdup(export_description),
             .has_bitmap         = !!bitmap,
             .bitmap             = g_strdup(bitmap),
+            .has_alloc          = alloc_depth,
+            .alloc              = alloc_depth,
         },
     };
     blk_exp_add(export_opts, &error_fatal);
tests/qemu-iotests/309 new file +73 -0

73 lines not on this page · raw message

tests/qemu-iotests/309.out new file +22 -0
new file mode 100644
index 000000000000..db75bb6b0df9
@@ -0,0 +1,22 @@
+QA output created by 309
+
+=== Initial image setup ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=4194304
+wrote 2097152/2097152 bytes at offset 0
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+wrote 2097152/2097152 bytes at offset 1048576
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+=== Check allocation over NBD ===
+
+[{ "start": 0, "length": 1048576, "depth": 1, "zero": false, "data": true, "offset": 327680},
+{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": 327680},
+{ "start": 3145728, "length": 1048576, "depth": 1, "zero": true, "data": false}]
+[{ "start": 0, "length": 3145728, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 3145728, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": true, "offset": OFFSET},
+{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": false},
+{ "start": 3145728, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
+*** done
tests/qemu-iotests/group +1 -0
index 9e4f7c01530d..a567fa97d7e5 100644
@@ -315,3 +315,4 @@
 304 rw quick
 305 rw quick
 307 rw quick export
+309 rw auto quick
Signature · 1 line
2.28.0
Reviewed-by on v2 5/5
#15 lore raw

Re: [PATCH v2 5/5] nbd: Add 'qemu-nbd -A' to expose allocation depth

Quoted 54 lines · Eric Blake

Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add.

qemu as client can be hacked into viewing this new context by using the now-misnamed x-dirty-bitmap option when creating an NBD blockdev (even though our x- naming means we could rename it, I did not think it worth breaking back-compat of tools that have been using it while waiting for a better solution). It is worth noting the decoding of how such context information will appear in 'qemu-img map --output=json':

NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true

libnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;)

---
  docs/tools/qemu-nbd.rst    |  6 ++++
  qapi/block-core.json       |  7 ++--
  qapi/block-export.json     |  6 +++-
  blockdev-nbd.c             |  2 ++
  nbd/server.c               |  2 ++
  qemu-nbd.c                 | 14 ++++++--
  tests/qemu-iotests/309     | 73 ++++++++++++++++++++++++++++++++++++++
  tests/qemu-iotests/309.out | 22 ++++++++++++
  tests/qemu-iotests/group   |  1 +
  9 files changed, 127 insertions(+), 6 deletions(-)

create mode 100755 tests/qemu-iotests/309
create mode 100644 tests/qemu-iotests/309.out

docs/tools/qemu-nbd.rst +6 -0
index 667861cb22e9..0e545a97cfa3 100644
@@ -72,6 +72,12 @@ driver options if ``--image-opts`` is specified.
 
    Export the disk as read-only.
 
+.. option:: -A, --allocation-depth
+
+  Expose allocation depth information via the
+  ``qemu:allocation-depth`` context accessible through
+  NBD_OPT_SET_META_CONTEXT.
+
  .. option:: -B, --bitmap=NAME
 
    If *filename* has a qcow2 persistent bitmap *NAME*, expose
qapi/block-core.json +0 -0
index d620bd1302b2..0379eb992db8 100644

[..]

qapi/block-core.json

Quoted 5 lines
qapi/block-core.json +5 -0
+echo
+echo "=== Check allocation over NBD ==="
+echo
+
+# Normal -f raw NBD block status loses access to allocation information

^this comment is not for the next line, but for further lines ...

qapi/block-core.json

Quoted 3 lines

+$QEMU_IMG map --output=json -f qcow2 "$TEST_IMG"

qapi/block-core.json +2 -0
+IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket"
+nbd_server_start_unix_socket -r -f qcow2 -A "$TEST_IMG"

... should it be here?

qapi/block-core.json

Quoted 12 lines

+$QEMU_IMG map --output=json --image-opts \

qapi/block-core.json +11 -0
+    "$IMG" | _filter_qemu_img_map
+# But since we used -A, and use x-dirty-bitmap as a hack for reading bitmaps,
+# we can reconstruct it, by abusing block status to report:
+#    NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
+#    NBD_STATE_DEPTH_LOCAL   => "zero":false, "data":false
+#    NBD_STATE_DEPTH_BACKING => "zero":true,  "data":true
+$QEMU_IMG map --output=json --image-opts \
+    "$IMG,x-dirty-bitmap=qemu:allocation-depth" | _filter_qemu_img_map
+
+# success, all done
+echo '*** done'
Signature · 2 lines
Best regards,
Vladimir