Skip to contentSkip to conversation

block/export: Add infrastructure and QAPI for block exports

RFC PATCH In review Kevin Wolf 5 participants 90 messages Aug 13, 2020 – Aug 20, 2020

Subsystems (2)
lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

cover block/export: Add infrastructure and QAPI for block exports

We are planning to add more block export types than just NBD in the near future (e.g. vhost-user-blk and FUSE). This series lays the ground for this with some generic block export infrastructure and QAPI interfaces that will allow managing all of them (for now add/remove/query).

As a side effect, qemu-storage-daemon can now map --export directly to the block-export-add QMP command, similar to other command line options. The built-in NBD servers also gains new options that bring it at least a little closer to feature parity with qemu-nbd.

This series is still RFC because the patches aren't very polished (though they also shouldn't be too bad), it's not completely clear if more features should be moved from NBD to the BlockExport layer in this series (or maybe even less, and the rest dealt with in a separate series) and qemu-iotests cases for the new interfaces are still missing.

Kevin Wolf (22):
nbd: Remove unused nbd_export_get_blockdev()
qapi: Create block-export module
qapi: Rename BlockExport to BlockExportOptions block/export: Add BlockExport infrastructure and block-export-add
qemu-storage-daemon: Use qmp_block_export_add()
qemu-nbd: Use raw block driver for --offset block/export: Remove magic from block-export-add
nbd: Add max-connections to nbd-server-start
nbd: Add writethrough to block-export-add
nbd: Remove NBDExport.close callback
qemu-nbd: Use blk_exp_add() to create the export nbd/server: Simplify export shutdown
block/export: Move refcount from NBDExport to BlockExport block/export: Move AioContext from NBDExport to BlockExport block/export: Move device to BlockExportOptions block/export: Allocate BlockExport in blk_exp_add() block/export: Add blk_exp_close_all(_type) block/export: Add 'id' option to block-export-add block/export: Move strong user reference to block_exports block/export: Add block-export-del
block/export: Move blk to BlockExport
block/export: Add query-block-exports

 qapi/block-core.json                 | 166 ---------------
 qapi/block-export.json               | 261 +++++++++++++++++++++++
 qapi/qapi-schema.json                |   1 +
 include/block/export.h               |  73 +++++++
 include/block/nbd.h                  |  25 +--
 block.c                              |   2 +-
 block/export/export.c                | 297 +++++++++++++++++++++++++++
 block/monitor/block-hmp-cmds.c       |  11 +-
 blockdev-nbd.c                       | 124 +++++------
 nbd/server.c                         | 243 +++++++++-------------
 qemu-nbd.c                           |  67 +++---
 qemu-storage-daemon.c                |  25 +--
 Makefile.objs                        |   6 +-
 block/Makefile.objs                  |   2 +
 block/export/Makefile.objs           |   1 +
 qapi/Makefile.objs                   |   5 +-
 storage-daemon/qapi/qapi-schema.json |   1 +
 17 files changed, 857 insertions(+), 453 deletions(-)
 create mode 100644 qapi/block-export.json
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs
Signature · 1 line
2.25.4
author
#1 lore raw

RFC PATCH 1/22 nbd: Remove unused nbd_export_get_blockdev()

---
 include/block/nbd.h | 2 --
 nbd/server.c        | 5 -----
 2 files changed, 7 deletions(-)
include/block/nbd.h +0 -2
index 9bc3bfaeec..0451683d03 100644
@@ -338,8 +338,6 @@ void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
 void nbd_export_get(NBDExport *exp);
 void nbd_export_put(NBDExport *exp);
 
-BlockBackend *nbd_export_get_blockdev(NBDExport *exp);
-
 AioContext *nbd_export_aio_context(NBDExport *exp);
 NBDExport *nbd_export_find(const char *name);
 void nbd_export_close_all(void);
nbd/server.c +0 -5
index 4752a6c8bc..bee2ef8bd1 100644
@@ -1731,11 +1731,6 @@ void nbd_export_put(NBDExport *exp)
     }
 }
 
-BlockBackend *nbd_export_get_blockdev(NBDExport *exp)
-{
-    return exp->blk;
-}
-
 void nbd_export_close_all(void)
 {
     NBDExport *exp, *next;
Signature · 1 line
2.25.4
author
#4 lore raw

RFC PATCH 2/22 qapi: Create block-export module

In reply to Kevin Wolf

Move all block export related types and commands from block-core to the new QAPI module block-export.

---
 qapi/block-core.json                 | 166 --------------------------
 qapi/block-export.json               | 172 +++++++++++++++++++++++++++
 qapi/qapi-schema.json                |   1 +
 include/block/nbd.h                  |   2 +-
 block/monitor/block-hmp-cmds.c       |   1 +
 blockdev-nbd.c                       |   2 +-
 qemu-storage-daemon.c                |   2 +-
 qapi/Makefile.objs                   |   5 +-
 storage-daemon/qapi/qapi-schema.json |   1 +
 9 files changed, 181 insertions(+), 171 deletions(-)
 create mode 100644 qapi/block-export.json
qapi/block-core.json +0 -166
index ab7bf3c612..5c491d4cbd 100644
@@ -5176,172 +5176,6 @@
              'iothread': 'StrOrNull',
              '*force': 'bool' } }
 
-##
-# @NbdServerOptions:
-#
-# @addr: Address on which to listen.
-# @tls-creds: ID of the TLS credentials object (since 2.6).
-# @tls-authz: ID of the QAuthZ authorization object used to validate
-#             the client's x509 distinguished name. This object is
-#             is only resolved at time of use, so can be deleted and
-#             recreated on the fly while the NBD server is active.
-#             If missing, it will default to denying access (since 4.0).
-#
-# Keep this type consistent with the nbd-server-start arguments. The only
-# intended difference is using SocketAddress instead of SocketAddressLegacy.
-#
-# Since: 4.2
-##
-{ 'struct': 'NbdServerOptions',
-  'data': { 'addr': 'SocketAddress',
-            '*tls-creds': 'str',
-            '*tls-authz': 'str'} }
-
-##
-# @nbd-server-start:
-#
-# Start an NBD server listening on the given host and port.  Block
-# devices can then be exported using @nbd-server-add.  The NBD
-# server will present them as named exports; for example, another
-# QEMU instance could refer to them as "nbd:HOST:PORT:exportname=NAME".
-#
-# @addr: Address on which to listen.
-# @tls-creds: ID of the TLS credentials object (since 2.6).
-# @tls-authz: ID of the QAuthZ authorization object used to validate
-#             the client's x509 distinguished name. This object is
-#             is only resolved at time of use, so can be deleted and
-#             recreated on the fly while the NBD server is active.
-#             If missing, it will default to denying access (since 4.0).
-#
-# Returns: error if the server is already running.
-#
-# Keep this type consistent with the NbdServerOptions type. The only intended
-# difference is using SocketAddressLegacy instead of SocketAddress.
-#
-# Since: 1.3.0
-##
-{ 'command': 'nbd-server-start',
-  'data': { 'addr': 'SocketAddressLegacy',
-            '*tls-creds': 'str',
-            '*tls-authz': 'str'} }
-
-##
-# @BlockExportNbd:
-#
-# An NBD block export.
-#
-# @device: The device name or node name of the node to be exported
-#
-# @name: Export name. If unspecified, the @device parameter is used as the
-#        export name. (Since 2.12)
-#
-# @description: Free-form description of the export, up to 4096 bytes.
-#               (Since 5.0)
-#
-# @writable: Whether clients should be able to write to the device via the
-#            NBD connection (default false).
-#
-# @bitmap: Also export the dirty bitmap reachable from @device, so the
-#          NBD client can use NBD_OPT_SET_META_CONTEXT with
-#          "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0)
-#
-# Since: 5.0
-##
-{ 'struct': 'BlockExportNbd',
-  'data': {'device': 'str', '*name': 'str', '*description': 'str',
-           '*writable': 'bool', '*bitmap': 'str' } }
-
-##
-# @nbd-server-add:
-#
-# Export a block node to QEMU's embedded NBD server.
-#
-# Returns: error if the server is not running, or export with the same name
-#          already exists.
-#
-# Since: 1.3.0
-##
-{ 'command': 'nbd-server-add',
-  'data': 'BlockExportNbd', 'boxed': true }
-
-##
-# @NbdServerRemoveMode:
-#
-# Mode for removing an NBD export.
-#
-# @safe: Remove export if there are no existing connections, fail otherwise.
-#
-# @hard: Drop all connections immediately and remove export.
-#
-# Potential additional modes to be added in the future:
-#
-# hide: Just hide export from new clients, leave existing connections as is.
-# Remove export after all clients are disconnected.
-#
-# soft: Hide export from new clients, answer with ESHUTDOWN for all further
-# requests from existing clients.
-#
-# Since: 2.12
-##
-{'enum': 'NbdServerRemoveMode', 'data': ['safe', 'hard']}
-
-##
-# @nbd-server-remove:
-#
-# Remove NBD export by name.
-#
-# @name: Export name.
-#
-# @mode: Mode of command operation. See @NbdServerRemoveMode description.
-#        Default is 'safe'.
-#
-# Returns: error if
-#            - the server is not running
-#            - export is not found
-#            - mode is 'safe' and there are existing connections
-#
-# Since: 2.12
-##
-{ 'command': 'nbd-server-remove',
-  'data': {'name': 'str', '*mode': 'NbdServerRemoveMode'} }
-
-##
-# @nbd-server-stop:
-#
-# Stop QEMU's embedded NBD server, and unregister all devices previously
-# added via @nbd-server-add.
-#
-# Since: 1.3.0
-##
-{ 'command': 'nbd-server-stop' }
-
-##
-# @BlockExportType:
-#
-# An enumeration of block export types
-#
-# @nbd: NBD export
-#
-# Since: 4.2
-##
-{ 'enum': 'BlockExportType',
-  'data': [ 'nbd' ] }
-
-##
-# @BlockExport:
-#
-# Describes a block export, i.e. how single node should be exported on an
-# external interface.
-#
-# Since: 4.2
-##
-{ 'union': 'BlockExport',
-  'base': { 'type': 'BlockExportType' },
-  'discriminator': 'type',
-  'data': {
-      'nbd': 'BlockExportNbd'
-   } }
-
 ##
 # @QuorumOpType:
 #
qapi/block-export.json new file +172 -0

172 lines not on this page · raw message

qapi/qapi-schema.json +1 -0
index 43b0ba0dea..2b37f6eb88 100644
@@ -63,6 +63,7 @@
 { 'include': 'run-state.json' }
 { 'include': 'crypto.json' }
 { 'include': 'block.json' }
+{ 'include': 'block-export.json' }
 { 'include': 'char.json' }
 { 'include': 'dump.json' }
 { 'include': 'job.json' }
include/block/nbd.h +1 -1
index 0451683d03..262f6da2ce 100644
@@ -20,7 +20,7 @@
 #ifndef NBD_H
 #define NBD_H
 
-#include "qapi/qapi-types-block.h"
+#include "qapi/qapi-types-block-export.h"
 #include "io/channel-socket.h"
 #include "crypto/tlscreds.h"
 #include "qapi/error.h"
block/monitor/block-hmp-cmds.c +1 -0
index 4c8c375172..fb9d87ee89 100644
@@ -40,6 +40,7 @@
 #include "sysemu/block-backend.h"
 #include "sysemu/blockdev.h"
 #include "qapi/qapi-commands-block.h"
+#include "qapi/qapi-commands-block-export.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/error.h"
 #include "qapi/qmp/qerror.h"
blockdev-nbd.c +1 -1
index 1a95d89f00..0f6b80c58f 100644
@@ -14,7 +14,7 @@
 #include "sysemu/block-backend.h"
 #include "hw/block/block.h"
 #include "qapi/error.h"
-#include "qapi/qapi-commands-block.h"
+#include "qapi/qapi-commands-block-export.h"
 #include "block/nbd.h"
 #include "io/channel-socket.h"
 #include "io/net-listener.h"
qemu-storage-daemon.c +1 -1
index 7e9b0e0d3f..ed9d2afcf3 100644
@@ -35,8 +35,8 @@
 #include "monitor/monitor-internal.h"
 
 #include "qapi/error.h"
-#include "qapi/qapi-visit-block.h"
 #include "qapi/qapi-visit-block-core.h"
+#include "qapi/qapi-visit-block-export.h"
 #include "qapi/qapi-visit-control.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qstring.h"
qapi/Makefile.objs +3 -2
index 4673ab7490..c5093b4d61 100644
@@ -5,7 +5,8 @@ util-obj-y += opts-visitor.o qapi-clone-visitor.o
 util-obj-y += qmp-event.o
 util-obj-y += qapi-util.o
 
-QAPI_COMMON_MODULES = audio authz block-core block char common control crypto
+QAPI_COMMON_MODULES = audio authz block-core block-export block char common
+QAPI_COMMON_MODULES += control crypto
 QAPI_COMMON_MODULES += dump error introspect job machine migration misc
 QAPI_COMMON_MODULES += net pragma qdev qom rdma rocker run-state sockets tpm
 QAPI_COMMON_MODULES += trace transaction ui
@@ -32,7 +33,7 @@ obj-y += $(QAPI_TARGET_MODULES:%=qapi-commands-%.o)
 obj-y += qapi-commands.o
 obj-y += qapi-init-commands.o
 
-QAPI_MODULES_STORAGE_DAEMON = block-core char common control crypto
+QAPI_MODULES_STORAGE_DAEMON = block-core block-export char common control crypto
 QAPI_MODULES_STORAGE_DAEMON += introspect job qom sockets pragma transaction
 
 storage-daemon-obj-y += $(QAPI_MODULES_STORAGE_DAEMON:%=qapi-commands-%.o)
storage-daemon/qapi/qapi-schema.json +1 -0
index 14f4f8fe61..9c7133724a 100644
@@ -15,6 +15,7 @@
 { 'include': '../../qapi/pragma.json' }
 
 { 'include': '../../qapi/block-core.json' }
+{ 'include': '../../qapi/block-export.json' }
 { 'include': '../../qapi/char.json' }
 { 'include': '../../qapi/common.json' }
 { 'include': '../../qapi/control.json' }
Signature · 1 line
2.25.4
Reviewed-by on v1 2/22
#5 lore raw

Re: [RFC PATCH 02/22] qapi: Create block-export module

Quoted 16 lines · Kevin Wolf

Move all block export related types and commands from block-core to the new QAPI module block-export.

---
 qapi/block-core.json                 | 166 --------------------------
 qapi/block-export.json               | 172 +++++++++++++++++++++++++++
 qapi/qapi-schema.json                |   1 +
 include/block/nbd.h                  |   2 +-
 block/monitor/block-hmp-cmds.c       |   1 +
 blockdev-nbd.c                       |   2 +-
 qemu-storage-daemon.c                |   2 +-
 qapi/Makefile.objs                   |   5 +-
 storage-daemon/qapi/qapi-schema.json |   1 +
 9 files changed, 181 insertions(+), 171 deletions(-)
 create mode 100644 qapi/block-export.json

[...]

qapi/block-export.json

Quoted 6 lines
qapi/block-export.json new file +0 -0
new file mode 100644
index 0000000000..62f4938e83
@@ -0,0 +1,172 @@

There should probably be a filetype modeline here (not trying to get into any hornets’ nest here, but... :)).

With that:

Reviewed-by on v1 2/22
#6 lore raw

Re: [RFC PATCH 02/22] qapi: Create block-export module

In reply to Kevin Wolf
Quoted 17 lines · Kevin Wolf

Move all block export related types and commands from block-core to the new QAPI module block-export.

---

+++ b/qapi/block-export.json

qapi/block-export.json new file +7 -0
@@ -0,0 +1,172 @@
+##
+# == Block device exports
+##
+
+{ 'include': 'sockets.json' }
+
 
+++ b/qapi/Makefile.objs

qapi/block-export.json util-obj-y += opts-visitor.o qapi-clone-visitor.o · line 9

Quoted 6 lines
qapi/block-export.json new file +2 -1
@@ -5,7 +5,8 @@ util-obj-y += opts-visitor.o qapi-clone-visitor.o
  util-obj-y += qmp-event.o
  util-obj-y += qapi-util.o
 
-QAPI_COMMON_MODULES = audio authz block-core block char common control crypto
+QAPI_COMMON_MODULES = audio authz block-core block-export block char common
+QAPI_COMMON_MODULES += control crypto

I don't know what Paolo's Meson conversion will affect here, but even if it changes how this hunk looks, the intent is obvious.

Looks like a straightforward refactoring. I agree with Max's comment on the missing modeline, but that doesn't stop me from:

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#7 lore raw

RFC PATCH 3/22 qapi: Rename BlockExport to BlockExportOptions

In reply to Kevin Wolf

The name BlockExport will be used for the struct containing the runtime state of block exports, so change the name of export creation options.

---
 qapi/block-export.json         | 12 ++++++------
 block/monitor/block-hmp-cmds.c |  6 +++---
 blockdev-nbd.c                 |  2 +-
 qemu-storage-daemon.c          |  8 ++++----
 4 files changed, 14 insertions(+), 14 deletions(-)
qapi/block-export.json +6 -6
index 62f4938e83..9332076a05 100644
@@ -54,7 +54,7 @@
             '*tls-authz': 'str'} }
 
 ##
-# @BlockExportNbd:
+# @BlockExportOptionsNbd:
 #
 # An NBD block export.
 #
@@ -75,7 +75,7 @@
 #
 # Since: 5.0
 ##
-{ 'struct': 'BlockExportNbd',
+{ 'struct': 'BlockExportOptionsNbd',
   'data': {'device': 'str', '*name': 'str', '*description': 'str',
            '*writable': 'bool', '*bitmap': 'str' } }
 
@@ -90,7 +90,7 @@
 # Since: 1.3.0
 ##
 { 'command': 'nbd-server-add',
-  'data': 'BlockExportNbd', 'boxed': true }
+  'data': 'BlockExportOptionsNbd', 'boxed': true }
 
 ##
 # @NbdServerRemoveMode:
@@ -156,17 +156,17 @@
   'data': [ 'nbd' ] }
 
 ##
-# @BlockExport:
+# @BlockExportOptions:
 #
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
 # Since: 4.2
 ##
-{ 'union': 'BlockExport',
+{ 'union': 'BlockExportOptions',
   'base': { 'type': 'BlockExportType' },
   'discriminator': 'type',
   'data': {
-      'nbd': 'BlockExportNbd'
+      'nbd': 'BlockExportOptionsNbd'
    } }
 
block/monitor/block-hmp-cmds.c +3 -3
index fb9d87ee89..56bc83ac97 100644
@@ -398,7 +398,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
     Error *local_err = NULL;
     BlockInfoList *block_list, *info;
     SocketAddress *addr;
-    BlockExportNbd export;
+    BlockExportOptionsNbd export;
 
     if (writable && !all) {
         error_setg(&local_err, "-w only valid together with -a");
@@ -431,7 +431,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
             continue;
         }
 
-        export = (BlockExportNbd) {
+        export = (BlockExportOptionsNbd) {
             .device         = info->value->device,
             .has_writable   = true,
             .writable       = writable,
@@ -458,7 +458,7 @@ void hmp_nbd_server_add(Monitor *mon, const QDict *qdict)
     bool writable = qdict_get_try_bool(qdict, "writable", false);
     Error *local_err = NULL;
 
-    BlockExportNbd export = {
+    BlockExportOptionsNbd export = {
         .device         = (char *) device,
         .has_name       = !!name,
         .name           = (char *) name,
blockdev-nbd.c +1 -1
index 0f6b80c58f..98ee1b6170 100644
@@ -148,7 +148,7 @@ void qmp_nbd_server_start(SocketAddressLegacy *addr,
     qapi_free_SocketAddress(addr_flat);
 }
 
-void qmp_nbd_server_add(BlockExportNbd *arg, Error **errp)
+void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 {
     BlockDriverState *bs = NULL;
     BlockBackend *on_eject_blk;
qemu-storage-daemon.c +4 -4
index ed9d2afcf3..ed26097254 100644
@@ -150,7 +150,7 @@ static void init_qmp_commands(void)
                          qmp_marshal_qmp_capabilities, QCO_ALLOW_PRECONFIG);
 }
 
-static void init_export(BlockExport *export, Error **errp)
+static void init_export(BlockExportOptions *export, Error **errp)
 {
     switch (export->type) {
     case BLOCK_EXPORT_TYPE_NBD:
@@ -235,14 +235,14 @@ static void process_options(int argc, char *argv[])
         case OPTION_EXPORT:
             {
                 Visitor *v;
-                BlockExport *export;
+                BlockExportOptions *export;
 
                 v = qobject_input_visitor_new_str(optarg, "type", &error_fatal);
-                visit_type_BlockExport(v, NULL, &export, &error_fatal);
+                visit_type_BlockExportOptions(v, NULL, &export, &error_fatal);
                 visit_free(v);
 
                 init_export(export, &error_fatal);
-                qapi_free_BlockExport(export);
+                qapi_free_BlockExportOptions(export);
                 break;
             }
         case OPTION_MONITOR:
Signature · 1 line
2.25.4
Reviewed-by on v1 3/22
#8 lore raw

Re: [RFC PATCH 03/22] qapi: Rename BlockExport to BlockExportOptions

Quoted 10 lines · Kevin Wolf

The name BlockExport will be used for the struct containing the runtime state of block exports, so change the name of export creation options.

---
 qapi/block-export.json         | 12 ++++++------
 block/monitor/block-hmp-cmds.c |  6 +++---
 blockdev-nbd.c                 |  2 +-
 qemu-storage-daemon.c          |  8 ++++----
 4 files changed, 14 insertions(+), 14 deletions(-)
Reviewed-by on v1 3/22
#9 lore raw

Re: [RFC PATCH 03/22] qapi: Rename BlockExport to BlockExportOptions

In reply to Kevin Wolf
Quoted 10 lines · Kevin Wolf

The name BlockExport will be used for the struct containing the runtime state of block exports, so change the name of export creation options.

---
  qapi/block-export.json         | 12 ++++++------
  block/monitor/block-hmp-cmds.c |  6 +++---
  blockdev-nbd.c                 |  2 +-
  qemu-storage-daemon.c          |  8 ++++----
  4 files changed, 14 insertions(+), 14 deletions(-)

Mechanical, and doesn't impact introspection.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#10 lore raw

RFC PATCH 4/22 block/export: Add BlockExport infrastructure and block-export-add

In reply to Kevin Wolf

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

---
 qapi/block-export.json     |  9 ++++++
 include/block/export.h     | 32 +++++++++++++++++++++
 include/block/nbd.h        |  3 +-
 block/export/export.c      | 57 ++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c             | 19 ++++++++-----
 nbd/server.c               | 15 +++++++++-
 Makefile.objs              |  6 ++--
 block/Makefile.objs        |  2 ++
 block/export/Makefile.objs |  1 +
 9 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs
qapi/block-export.json +9 -0
index 9332076a05..40369814b4 100644
@@ -170,3 +170,12 @@
       'nbd': 'BlockExportOptionsNbd'
    } }
 
+##
+# @block-export-add:
+#
+# Creates a new block export.
+#
+# Since: 5.2
+##
+{ 'command': 'block-export-add',
+  'data': 'BlockExportOptions', 'boxed': true }
include/block/export.h new file +32 -0
new file mode 100644
index 0000000000..b1d7325403
@@ -0,0 +1,32 @@
+/*
+ * Declarations for block exports
+ *
+ * Copyright (c) 2012, 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Paolo Bonzini <[email protected]>
+ * Kevin Wolf <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#ifndef BLOCK_EXPORT_H
+#define BLOCK_EXPORT_H
+
+#include "qapi/qapi-types-block-export.h"
+
+typedef struct BlockExport BlockExport;
+
+typedef struct BlockExportDriver {
+    BlockExportType type;
+    BlockExport *(*create)(BlockExportOptions *, Error **);
+} BlockExportDriver;
+
+struct BlockExport {
+    const BlockExportDriver *drv;
+};
+
+extern const BlockExportDriver blk_exp_nbd;
+
+#endif
include/block/nbd.h +2 -1
index 262f6da2ce..c8c5cb6b61 100644
@@ -20,7 +20,7 @@
 #ifndef NBD_H
 #define NBD_H
 
-#include "qapi/qapi-types-block-export.h"
+#include "block/export.h"
 #include "io/channel-socket.h"
 #include "crypto/tlscreds.h"
 #include "qapi/error.h"
@@ -328,6 +328,7 @@ int nbd_errno_to_system_errno(int err);
 typedef struct NBDExport NBDExport;
 typedef struct NBDClient NBDClient;
 
+BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp);
 NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
                           uint64_t size, const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
block/export/export.c new file +57 -0

57 lines not on this page · raw message

blockdev-nbd.c +12 -7
index 98ee1b6170..a1dc11bdd7 100644
@@ -148,17 +148,20 @@ void qmp_nbd_server_start(SocketAddressLegacy *addr,
     qapi_free_SocketAddress(addr_flat);
 }
 
-void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
+BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 {
+    BlockExportOptionsNbd *arg = &exp_args->u.nbd;
     BlockDriverState *bs = NULL;
     BlockBackend *on_eject_blk;
-    NBDExport *exp;
+    NBDExport *exp = NULL;
     int64_t len;
     AioContext *aio_context;
 
+    assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
+
     if (!nbd_server) {
         error_setg(errp, "NBD server not running");
-        return;
+        return NULL;
     }
 
     if (!arg->has_name) {
@@ -167,24 +170,24 @@ void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 
     if (strlen(arg->name) > NBD_MAX_STRING_SIZE) {
         error_setg(errp, "export name '%s' too long", arg->name);
-        return;
+        return NULL;
     }
 
     if (arg->description && strlen(arg->description) > NBD_MAX_STRING_SIZE) {
         error_setg(errp, "description '%s' too long", arg->description);
-        return;
+        return NULL;
     }
 
     if (nbd_export_find(arg->name)) {
         error_setg(errp, "NBD server already has export named '%s'", arg->name);
-        return;
+        return NULL;
     }
 
     on_eject_blk = blk_by_name(arg->device);
 
     bs = bdrv_lookup_bs(arg->device, arg->device, errp);
     if (!bs) {
-        return;
+        return NULL;
     }
 
     aio_context = bdrv_get_aio_context(bs);
@@ -217,6 +220,8 @@ void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 
  out:
     aio_context_release(aio_context);
+    /* TODO Remove the cast: Move to server.c which can access fields of exp */
+    return (BlockExport*) exp;
 }
 
 void qmp_nbd_server_remove(const char *name,
nbd/server.c +14 -1
index bee2ef8bd1..774325dbe5 100644
@@ -18,6 +18,8 @@
  */
 
 #include "qemu/osdep.h"
+
+#include "block/export.h"
 #include "qapi/error.h"
 #include "qemu/queue.h"
 #include "trace.h"
@@ -80,6 +82,7 @@ struct NBDRequestData {
 };
 
 struct NBDExport {
+    BlockExport common;
     int refcount;
     void (*close)(NBDExport *exp);
 
@@ -1512,10 +1515,15 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
 {
     AioContext *ctx;
     BlockBackend *blk;
-    NBDExport *exp = g_new0(NBDExport, 1);
+    NBDExport *exp;
     uint64_t perm;
     int ret;
 
+    exp = g_new0(NBDExport, 1);
+    exp->common = (BlockExport) {
+        .drv = &blk_exp_nbd,
+    };
+
     /*
      * NBD exports are used for non-shared storage migration.  Make sure
      * that BDRV_O_INACTIVE is cleared and the image is ready for write
@@ -1731,6 +1739,11 @@ void nbd_export_put(NBDExport *exp)
     }
 }
 
+const BlockExportDriver blk_exp_nbd = {
+    .type               = BLOCK_EXPORT_TYPE_NBD,
+    .create             = nbd_export_create,
+};
+
 void nbd_export_close_all(void)
 {
     NBDExport *exp, *next;
Makefile.objs +3 -3
index d22b3b45d7..9b864ca046 100644
@@ -14,7 +14,7 @@ chardev-obj-y = chardev/
 authz-obj-y = authz/
 
 block-obj-y = block/ nbd/ scsi/
-block-obj-y += block.o blockjob.o job.o
+block-obj-y += block.o blockjob.o job.o blockdev-nbd.o
 block-obj-y += qemu-io-cmds.o
 block-obj-$(CONFIG_REPLICATION) += replication.o
 
@@ -31,7 +31,7 @@ endif # CONFIG_SOFTMMU or CONFIG_TOOLS
 # used for system emulation, too, but specified separately there)
 
 storage-daemon-obj-y = block/ monitor/ qapi/ qom/ storage-daemon/
-storage-daemon-obj-y += blockdev.o blockdev-nbd.o iothread.o job-qmp.o
+storage-daemon-obj-y += blockdev.o iothread.o job-qmp.o
 storage-daemon-obj-$(CONFIG_WIN32) += os-win32.o
 storage-daemon-obj-$(CONFIG_POSIX) += os-posix.o
 
@@ -41,7 +41,7 @@ storage-daemon-obj-$(CONFIG_POSIX) += os-posix.o
 # single QEMU executable should support all CPUs and machines.
 
 ifeq ($(CONFIG_SOFTMMU),y)
-common-obj-y = blockdev.o blockdev-nbd.o block/
+common-obj-y = blockdev.o block/
 common-obj-y += bootdevice.o iothread.o
 common-obj-y += dump/
 common-obj-y += job-qmp.o
block/Makefile.objs +2 -0
index 19c6f371c9..55b45c2f7d 100644
@@ -44,6 +44,8 @@ block-obj-y += crypto.o
 block-obj-y += aio_task.o
 block-obj-y += backup-top.o
 block-obj-y += filter-compress.o
+
+block-obj-y += export/
 common-obj-y += monitor/
 block-obj-y += monitor/
 
block/export/Makefile.objs new file +1 -0
new file mode 100644
index 0000000000..0c170ee6f1
@@ -0,0 +1 @@
+block-obj-y += export.o
Signature · 1 line
2.25.4
#11 lore raw

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

Quoted 10 lines · Kevin Wolf

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

Quoted 14 lines
---
 qapi/block-export.json     |  9 ++++++
 include/block/export.h     | 32 +++++++++++++++++++++
 include/block/nbd.h        |  3 +-
 block/export/export.c      | 57 ++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c             | 19 ++++++++-----
 nbd/server.c               | 15 +++++++++-
 Makefile.objs              |  6 ++--
 block/Makefile.objs        |  2 ++
 block/export/Makefile.objs |  1 +
 9 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs

Nothing of too great importance below. But it’s an RFC, so comments I will give.

Quoted 17 lines
block/export/export.c new file +11 -0
new file mode 100644
index 0000000000..3d0dacb3f2
@@ -0,0 +1,57 @@
+/*
+ * Common block export infrastructure
+ *
+ * Copyright (c) 2012, 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Paolo Bonzini <[email protected]>
+ * Kevin Wolf <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.

block/export/export.c line 21

Quoted 10 lines
block/export/export.c new file +10 -0
+ */
+
+#include "qemu/osdep.h"
+
+#include "block/export.h"
+#include "block/nbd.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-block-export.h"
+
+static const BlockExportDriver* blk_exp_drivers[] = {
                                 ^^

Sternenplatzierung *hust*

block/export/export.c line 23

Quoted 2 lines
block/export/export.c new file +2 -0
+    &blk_exp_nbd,
+};

Not sure whether I like this better than the block driver way of registering block drivers with a constructor. It requires writing less code, at the expense of making the variable global. So I think there’s no good reason to prefer the block driver approach.

Maybe my hesitance comes from the variable being declared (as extern) in a header file (block/export.h). I think I would prefer it if we put that external reference only here in this file. Would that work, or do you have other plans that require blk_exp_nbd to be visible outside of nbd/server.c and this file here?

block/export/export.c line 33

Quoted 9 lines
block/export/export.c new file +9 -0
+static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
+{
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(blk_exp_drivers); i++) {
+        if (blk_exp_drivers[i]->type == type) {
+            return blk_exp_drivers[i];
+        }
+    }

How bad would it be to define blk_exp_drivers as blk_exp_drivers[BLOCK_EXPORT_TYPE__MAX] and use the BlockExportType as the driver index so we don’t have to loop here?

Not that it matters performance-wise. Just something I wondered.

block/export/export.c line 33

Quoted 1 line
block/export/export.c new file +1 -0
+    return NULL;

Why not e.g. g_assert_not_reached()?

(If the BlockExportType were used as the index, I’d assert that type < ARRAY_SIZE(blk_exp_drivers) && blk_exp_drivers[type] != NULL. I don’t think there’s a reason for graceful handling.)

Quoted 1 line
block/export/export.c new file +1 -0
+}

[...]

block/export/export.c line 57

Quoted 8 lines
block/export/export.c new file +8 -0
+void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
+{
+    BlockExportOptions export = {
+        .type = BLOCK_EXPORT_TYPE_NBD,
+        .u.nbd = *arg,
+    };
+    qmp_block_export_add(&export, errp);
+}

Hm. I think I’d’ve kept this in blockdev-nbd.c, actually, but, well. It’ll stay a one-off.

Quoted 4 lines
blockdev-nbd.c +0 -0
index 98ee1b6170..a1dc11bdd7 100644

[...]

blockdev-nbd.c void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp) · line 224

Quoted 6 lines
blockdev-nbd.c +2 -0
@@ -217,6 +220,8 @@ void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 
  out:
     aio_context_release(aio_context);
+    /* TODO Remove the cast: Move to server.c which can access fields of exp */
+    return (BlockExport*) exp;

*hust*

(But if it’s moved soon anyway so we can use &exp->common, then whatever.)

Max

author
#12 lore raw

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

Am 17.08.2020 um 12:03 hat Max Reitz geschrieben:

Quoted 59 lines · Kevin Wolf
Quoted 10 lines

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

Quoted 14 lines
---
 qapi/block-export.json     |  9 ++++++
 include/block/export.h     | 32 +++++++++++++++++++++
 include/block/nbd.h        |  3 +-
 block/export/export.c      | 57 ++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c             | 19 ++++++++-----
 nbd/server.c               | 15 +++++++++-
 Makefile.objs              |  6 ++--
 block/Makefile.objs        |  2 ++
 block/export/Makefile.objs |  1 +
 9 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs

Nothing of too great importance below. But it’s an RFC, so comments I will give.

Quoted 17 lines
block/export/export.c new file +11 -0
new file mode 100644
index 0000000000..3d0dacb3f2
@@ -0,0 +1,57 @@
+/*
+ * Common block export infrastructure
+ *
+ * Copyright (c) 2012, 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Paolo Bonzini <[email protected]>
+ * Kevin Wolf <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.

block/export/export.c line 21

Quoted 10 lines
block/export/export.c new file +10 -0
+ */
+
+#include "qemu/osdep.h"
+
+#include "block/export.h"
+#include "block/nbd.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-block-export.h"
+
+static const BlockExportDriver* blk_exp_drivers[] = {
                                 ^^

Sternenplatzierung *hust*

Quoted 7 lines

block/export/export.c line 23

Quoted 2 lines
block/export/export.c new file +2 -0
+    &blk_exp_nbd,
+};

Not sure whether I like this better than the block driver way of registering block drivers with a constructor. It requires writing less code, at the expense of making the variable global. So I think there’s no good reason to prefer the block driver approach.

I guess I can see one reason why we may want to switch to the registration style eventually: If we we want to make export drivers optional modules which may or may not be present.

Quoted 5 lines

Maybe my hesitance comes from the variable being declared (as extern) in a header file (block/export.h). I think I would prefer it if we put that external reference only here in this file. Would that work, or do you have other plans that require blk_exp_nbd to be visible outside of nbd/server.c and this file here?

Hm, do we have precedence for "public, but not really" variables? Normally I expect public symbols to be declared in a header file.

Quoted 14 lines

block/export/export.c line 33

Quoted 9 lines
block/export/export.c new file +9 -0
+static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
+{
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(blk_exp_drivers); i++) {
+        if (blk_exp_drivers[i]->type == type) {
+            return blk_exp_drivers[i];
+        }
+    }

How bad would it be to define blk_exp_drivers as blk_exp_drivers[BLOCK_EXPORT_TYPE__MAX] and use the BlockExportType as the driver index so we don’t have to loop here?

Quoted 1 line

Not that it matters performance-wise. Just something I wondered.

Might be nicer indeed. It would be incompatible with a registration model, though, so if we're not sure yet what we want to have in the long term, maybe the more neutral way is to leave it as it is.

Quoted 7 lines

block/export/export.c line 33

Quoted 1 line
block/export/export.c new file +1 -0
+    return NULL;

Why not e.g. g_assert_not_reached()?

(If the BlockExportType were used as the index, I’d assert that type < ARRAY_SIZE(blk_exp_drivers) && blk_exp_drivers[type] != NULL. I don’t think there’s a reason for graceful handling.)

Same thing actually. This works as long as all drivers are always present.

Now I understand that the current state is somewhat inconsistent in that it uses a simple array of things that are always present, but has functions that work as if it were dynamic. I don't mind this inconsistency very much, but if you do, I guess I could implement a registration type thing right away.

Kevin

Reviewed-by
#13 lore raw

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

Quoted 63 lines · Kevin Wolf

Am 17.08.2020 um 12:03 hat Max Reitz geschrieben:

Quoted 62 lines · Kevin Wolf
Quoted 10 lines

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

Quoted 14 lines
---
 qapi/block-export.json     |  9 ++++++
 include/block/export.h     | 32 +++++++++++++++++++++
 include/block/nbd.h        |  3 +-
 block/export/export.c      | 57 ++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c             | 19 ++++++++-----
 nbd/server.c               | 15 +++++++++-
 Makefile.objs              |  6 ++--
 block/Makefile.objs        |  2 ++
 block/export/Makefile.objs |  1 +
 9 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs

Nothing of too great importance below. But it’s an RFC, so comments I will give.

Quoted 17 lines
block/export/export.c new file +11 -0
new file mode 100644
index 0000000000..3d0dacb3f2
@@ -0,0 +1,57 @@
+/*
+ * Common block export infrastructure
+ *
+ * Copyright (c) 2012, 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Paolo Bonzini <[email protected]>
+ * Kevin Wolf <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.

block/export/export.c line 21

Quoted 10 lines
block/export/export.c new file +10 -0
+ */
+
+#include "qemu/osdep.h"
+
+#include "block/export.h"
+#include "block/nbd.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-block-export.h"
+
+static const BlockExportDriver* blk_exp_drivers[] = {
                                 ^^

Sternenplatzierung *hust*

Quoted 2 lines
block/export/export.c new file +2 -0
+    &blk_exp_nbd,
+};
Quoted 8 lines
Quoted 4 lines

Not sure whether I like this better than the block driver way of registering block drivers with a constructor. It requires writing less code, at the expense of making the variable global. So I think there’s no good reason to prefer the block driver approach.

I guess I can see one reason why we may want to switch to the registration style eventually: If we we want to make export drivers optional modules which may or may not be present.

Good point.

Quoted 8 lines
Quoted 5 lines

Maybe my hesitance comes from the variable being declared (as extern) in a header file (block/export.h). I think I would prefer it if we put that external reference only here in this file. Would that work, or do you have other plans that require blk_exp_nbd to be visible outside of nbd/server.c and this file here?

Hm, do we have precedence for "public, but not really" variables? Normally I expect public symbols to be declared in a header file.

Hm, yes.

tl;dr: I was wrong about a local external reference being nicer. But I believe there is a difference between externally-facing header files (e.g. block.h) and internal header files (e.g. block_int.h). I don’t know which of those block/export.h is supposed to be.

(And of course it doesn’t even matter at all, really.)

non-tl;dr:

We have a similar case for bdrv_{file,raw,qcow2}, but those are at least in a *_int.h. I can’t say I like that style.

OK, let me try to figure out what my problem with this is.

I think if a module (in this case the NBD export code) exports something, it should be available in the respective header (i.e., some NBD header), not in some other header. A module’s header should present what it exports to the rest of the code. The export code probably doesn’t want to export the NBD driver object, it wants to import it, actually. So if it should be in a header file, it should be in an NBD header.

Now none of our block drivers has a header file for exporting symbols to the rest of the block code, which is why their symbols have been put into block_int.h. I think that’s cutting corners, but can be defended by saying that block_int.h is not for exporting anything, but just collects stuff internal to the block layer, so it kind of fits there.

(Still, technically, I believe bdrv_{file,raw,qcow2} should be exported by each respective block driver in a driver-specific header file. If that isn’t the case, it doesn’t really matter to me whether it’s put into a dedicated header file to collect internal stuff (block_int.h) or just imported locally (with an external declaration) where it’s used. Probably the dedicated header file is cleaner after all, right.)

Maybe block/export.h is the same in that it’s just supposed to collect symbols used internally by the export code, then it isn’t wrong to put it there. But if it’s a header file that may be used by non-export code to use export functionality, then it would be wrong.

But whatever.

Now I have sorted out my feelings, and they don’t give any result at all, but it was kind of therapeutic for me.

Quoted 14 lines
Quoted 14 lines

block/export/export.c line 33

Quoted 9 lines
block/export/export.c new file +9 -0
+static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
+{
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(blk_exp_drivers); i++) {
+        if (blk_exp_drivers[i]->type == type) {
+            return blk_exp_drivers[i];
+        }
+    }

How bad would it be to define blk_exp_drivers as blk_exp_drivers[BLOCK_EXPORT_TYPE__MAX] and use the BlockExportType as the driver index so we don’t have to loop here?

Quoted 5 lines
Quoted 1 line

Not that it matters performance-wise. Just something I wondered.

Might be nicer indeed. It would be incompatible with a registration model, though, so if we're not sure yet what we want to have in the long term, maybe the more neutral way is to leave it as it is.

Yes, true.

Quoted 8 lines
Quoted 7 lines

block/export/export.c line 33

Quoted 1 line
block/export/export.c new file +1 -0
+    return NULL;

Why not e.g. g_assert_not_reached()?

(If the BlockExportType were used as the index, I’d assert that type < ARRAY_SIZE(blk_exp_drivers) && blk_exp_drivers[type] != NULL. I don’t think there’s a reason for graceful handling.)

Quoted 8 lines

Same thing actually. This works as long as all drivers are always present.

Now I understand that the current state is somewhat inconsistent in that it uses a simple array of things that are always present, but has functions that work as if it were dynamic. I don't mind this inconsistency very much, but if you do, I guess I could implement a registration type thing right away.

Sounds all reasonable. Thus, I’d leave it like you did it and care about a registration model if/when we need it.

author
#14 lore raw

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

Am 17.08.2020 um 15:19 hat Max Reitz geschrieben:

Quoted 127 lines · Kevin Wolf
Quoted 63 lines

Am 17.08.2020 um 12:03 hat Max Reitz geschrieben:

Quoted 62 lines · Kevin Wolf
Quoted 10 lines

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

Quoted 14 lines
---
 qapi/block-export.json     |  9 ++++++
 include/block/export.h     | 32 +++++++++++++++++++++
 include/block/nbd.h        |  3 +-
 block/export/export.c      | 57 ++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c             | 19 ++++++++-----
 nbd/server.c               | 15 +++++++++-
 Makefile.objs              |  6 ++--
 block/Makefile.objs        |  2 ++
 block/export/Makefile.objs |  1 +
 9 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs

Nothing of too great importance below. But it’s an RFC, so comments I will give.

Quoted 17 lines
block/export/export.c new file +11 -0
new file mode 100644
index 0000000000..3d0dacb3f2
@@ -0,0 +1,57 @@
+/*
+ * Common block export infrastructure
+ *
+ * Copyright (c) 2012, 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Paolo Bonzini <[email protected]>
+ * Kevin Wolf <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.

block/export/export.c line 21

Quoted 10 lines
block/export/export.c new file +10 -0
+ */
+
+#include "qemu/osdep.h"
+
+#include "block/export.h"
+#include "block/nbd.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-block-export.h"
+
+static const BlockExportDriver* blk_exp_drivers[] = {
                                 ^^

Sternenplatzierung *hust*

Quoted 2 lines
block/export/export.c new file +2 -0
+    &blk_exp_nbd,
+};
Quoted 8 lines
Quoted 4 lines

Not sure whether I like this better than the block driver way of registering block drivers with a constructor. It requires writing less code, at the expense of making the variable global. So I think there’s no good reason to prefer the block driver approach.

I guess I can see one reason why we may want to switch to the registration style eventually: If we we want to make export drivers optional modules which may or may not be present.

Good point.

Quoted 8 lines
Quoted 5 lines

Maybe my hesitance comes from the variable being declared (as extern) in a header file (block/export.h). I think I would prefer it if we put that external reference only here in this file. Would that work, or do you have other plans that require blk_exp_nbd to be visible outside of nbd/server.c and this file here?

Hm, do we have precedence for "public, but not really" variables? Normally I expect public symbols to be declared in a header file.

Hm, yes.

tl;dr: I was wrong about a local external reference being nicer. But I believe there is a difference between externally-facing header files (e.g. block.h) and internal header files (e.g. block_int.h). I don’t know which of those block/export.h is supposed to be.

(And of course it doesn’t even matter at all, really.)

non-tl;dr:

We have a similar case for bdrv_{file,raw,qcow2}, but those are at least in a *_int.h. I can’t say I like that style.

OK, let me try to figure out what my problem with this is.

I think if a module (in this case the NBD export code) exports something, it should be available in the respective header (i.e., some NBD header), not in some other header. A module’s header should present what it exports to the rest of the code. The export code probably doesn’t want to export the NBD driver object, it wants to import it, actually. So if it should be in a header file, it should be in an NBD header.

Now none of our block drivers has a header file for exporting symbols to the rest of the block code, which is why their symbols have been put into block_int.h. I think that’s cutting corners, but can be defended by saying that block_int.h is not for exporting anything, but just collects stuff internal to the block layer, so it kind of fits there.

(Still, technically, I believe bdrv_{file,raw,qcow2} should be exported by each respective block driver in a driver-specific header file. If that isn’t the case, it doesn’t really matter to me whether it’s put into a dedicated header file to collect internal stuff (block_int.h) or just imported locally (with an external declaration) where it’s used. Probably the dedicated header file is cleaner after all, right.)

Maybe block/export.h is the same in that it’s just supposed to collect symbols used internally by the export code, then it isn’t wrong to put it there. But if it’s a header file that may be used by non-export code to use export functionality, then it would be wrong.

Quoted 4 lines

But whatever.

Now I have sorted out my feelings, and they don’t give any result at all, but it was kind of therapeutic for me.

Actually, there could be a conclusion: The declaration shouldn't be in include/block/export.h, but in include/block/nbd.h. We already include both headers in block/export/export.c because of qmp_nbd_*().

Of course, you already requests that I leave the other NBD-related stuff in blockdev-nbd.c rather than moving it there, so the use of blk_exp_nbd would be the only reason that remains for export.c to include nbd.h.

But it might still be better than having it in export.h.

Kevin

#15 lore raw

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

Quoted 128 lines · Kevin Wolf

Am 17.08.2020 um 15:19 hat Max Reitz geschrieben:

Quoted 127 lines · Kevin Wolf
Quoted 63 lines

Am 17.08.2020 um 12:03 hat Max Reitz geschrieben:

Quoted 62 lines · Kevin Wolf
Quoted 10 lines

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

Quoted 14 lines
---
 qapi/block-export.json     |  9 ++++++
 include/block/export.h     | 32 +++++++++++++++++++++
 include/block/nbd.h        |  3 +-
 block/export/export.c      | 57 ++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c             | 19 ++++++++-----
 nbd/server.c               | 15 +++++++++-
 Makefile.objs              |  6 ++--
 block/Makefile.objs        |  2 ++
 block/export/Makefile.objs |  1 +
 9 files changed, 132 insertions(+), 12 deletions(-)
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/Makefile.objs

Nothing of too great importance below. But it’s an RFC, so comments I will give.

Quoted 17 lines
block/export/export.c new file +11 -0
new file mode 100644
index 0000000000..3d0dacb3f2
@@ -0,0 +1,57 @@
+/*
+ * Common block export infrastructure
+ *
+ * Copyright (c) 2012, 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * Paolo Bonzini <[email protected]>
+ * Kevin Wolf <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.

block/export/export.c line 21

Quoted 10 lines
block/export/export.c new file +10 -0
+ */
+
+#include "qemu/osdep.h"
+
+#include "block/export.h"
+#include "block/nbd.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-block-export.h"
+
+static const BlockExportDriver* blk_exp_drivers[] = {
                                 ^^

Sternenplatzierung *hust*

Quoted 2 lines
block/export/export.c new file +2 -0
+    &blk_exp_nbd,
+};
Quoted 8 lines
Quoted 4 lines

Not sure whether I like this better than the block driver way of registering block drivers with a constructor. It requires writing less code, at the expense of making the variable global. So I think there’s no good reason to prefer the block driver approach.

I guess I can see one reason why we may want to switch to the registration style eventually: If we we want to make export drivers optional modules which may or may not be present.

Good point.

Quoted 8 lines
Quoted 5 lines

Maybe my hesitance comes from the variable being declared (as extern) in a header file (block/export.h). I think I would prefer it if we put that external reference only here in this file. Would that work, or do you have other plans that require blk_exp_nbd to be visible outside of nbd/server.c and this file here?

Hm, do we have precedence for "public, but not really" variables? Normally I expect public symbols to be declared in a header file.

Hm, yes.

tl;dr: I was wrong about a local external reference being nicer. But I believe there is a difference between externally-facing header files (e.g. block.h) and internal header files (e.g. block_int.h). I don’t know which of those block/export.h is supposed to be.

(And of course it doesn’t even matter at all, really.)

non-tl;dr:

We have a similar case for bdrv_{file,raw,qcow2}, but those are at least in a *_int.h. I can’t say I like that style.

OK, let me try to figure out what my problem with this is.

I think if a module (in this case the NBD export code) exports something, it should be available in the respective header (i.e., some NBD header), not in some other header. A module’s header should present what it exports to the rest of the code. The export code probably doesn’t want to export the NBD driver object, it wants to import it, actually. So if it should be in a header file, it should be in an NBD header.

Now none of our block drivers has a header file for exporting symbols to the rest of the block code, which is why their symbols have been put into block_int.h. I think that’s cutting corners, but can be defended by saying that block_int.h is not for exporting anything, but just collects stuff internal to the block layer, so it kind of fits there.

(Still, technically, I believe bdrv_{file,raw,qcow2} should be exported by each respective block driver in a driver-specific header file. If that isn’t the case, it doesn’t really matter to me whether it’s put into a dedicated header file to collect internal stuff (block_int.h) or just imported locally (with an external declaration) where it’s used. Probably the dedicated header file is cleaner after all, right.)

Maybe block/export.h is the same in that it’s just supposed to collect symbols used internally by the export code, then it isn’t wrong to put it there. But if it’s a header file that may be used by non-export code to use export functionality, then it would be wrong.

Quoted 8 lines
Quoted 4 lines

But whatever.

Now I have sorted out my feelings, and they don’t give any result at all, but it was kind of therapeutic for me.

Actually, there could be a conclusion: The declaration shouldn't be in include/block/export.h, but in include/block/nbd.h. We already include both headers in block/export/export.c because of qmp_nbd_*().

Sounds good.

Quoted 3 lines

Of course, you already requests that I leave the other NBD-related stuff in blockdev-nbd.c rather than moving it there, so the use of blk_exp_nbd would be the only reason that remains for export.c to include nbd.h.

Aww. That’s too bad.

Quoted 1 line

But it might still be better than having it in export.h.

I’ll take the easy way out and leave it to you.

Max

#16 lore raw

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

In reply to Kevin Wolf
Quoted 11 lines · Kevin Wolf

We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types.

This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given BlockExportOptions.

qmp_nbd_server_add() becomes a wrapper around qmp_block_export_add().

---

Seeing if I can spot anything beyond Max's fine points:

Quoted 4 lines

+++ b/qapi/block-export.json

qapi/block-export.json +0 -0
@@ -170,3 +170,12 @@
        'nbd': 'BlockExportOptionsNbd'
     } }

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +9 -0
 
+##
+# @block-export-add:
+#
+# Creates a new block export.
+#
+# Since: 5.2
+##
+{ 'command': 'block-export-add',
+  'data': 'BlockExportOptions', 'boxed': true }

So if I read patch 3 correctly, the difference between nbd-server-add and block-export-add is that the latter includes a "type":"nbd" member. (If we ever play with allowing a default discriminator value in QAPI, we could declare the default for "type" to be NBD, and then the two would be identical - except that since you are adding a new command designed to extend to more than just nbd, I don't think keeping nbd as default makes sense)

block/export/export.c line 57

Quoted 10 lines

+++ b/block/export/export.c

block/export/export.c new file +8 -0
+void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
+{
+    BlockExportOptions export = {
+        .type = BLOCK_EXPORT_TYPE_NBD,
+        .u.nbd = *arg,
+    };
+    qmp_block_export_add(&export, errp);
+}

And indeed, this matches that analysis.

blockdev-nbd.c void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp) · line 224

Quoted 6 lines
blockdev-nbd.c +2 -0
@@ -217,6 +220,8 @@ void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 
   out:
      aio_context_release(aio_context);
+    /* TODO Remove the cast: Move to server.c which can access fields of exp */
+    return (BlockExport*) exp;

Should this use container_of()? Ah, the TODO says you want to, but can't because exp is an opaque type in this file...

Quoted 17 lines
blockdev-nbd.c +0 -0
  }
 
  void qmp_nbd_server_remove(const char *name,
nbd/server.c +2 -0
index bee2ef8bd1..774325dbe5 100644
@@ -18,6 +18,8 @@
   */
 
  #include "qemu/osdep.h"
+
+#include "block/export.h"
  #include "qapi/error.h"
  #include "qemu/queue.h"
  #include "trace.h"

nbd/server.c struct NBDRequestData { · line 85

Quoted 4 lines
nbd/server.c +1 -0
@@ -80,6 +82,7 @@ struct NBDRequestData {
  };
 
  struct NBDExport {
+    BlockExport common;

...but at least the cast is accurate.

Overall, the idea looks sane. I'm happy if you want to move blockdev-nbd.c out of the top level.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#17 lore raw

RFC PATCH 5/22 qemu-storage-daemon: Use qmp_block_export_add()

In reply to Kevin Wolf

No reason to duplicate the functionality locally, we can now just reuse the QMP command block-export-add for --export.

---
 qemu-storage-daemon.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)
qemu-storage-daemon.c +1 -12
index ed26097254..b6f678d3ab 100644
@@ -150,17 +150,6 @@ static void init_qmp_commands(void)
                          qmp_marshal_qmp_capabilities, QCO_ALLOW_PRECONFIG);
 }
 
-static void init_export(BlockExportOptions *export, Error **errp)
-{
-    switch (export->type) {
-    case BLOCK_EXPORT_TYPE_NBD:
-        qmp_nbd_server_add(&export->u.nbd, errp);
-        break;
-    default:
-        g_assert_not_reached();
-    }
-}
-
 static void process_options(int argc, char *argv[])
 {
     int c;
@@ -241,7 +230,7 @@ static void process_options(int argc, char *argv[])
                 visit_type_BlockExportOptions(v, NULL, &export, &error_fatal);
                 visit_free(v);
 
-                init_export(export, &error_fatal);
+                qmp_block_export_add(export, &error_fatal);
                 qapi_free_BlockExportOptions(export);
                 break;
             }
Signature · 1 line
2.25.4
Reviewed-by on v1 5/22
#18 lore raw

Re: [RFC PATCH 05/22] qemu-storage-daemon: Use qmp_block_export_add()

Quoted 7 lines · Kevin Wolf

No reason to duplicate the functionality locally, we can now just reuse the QMP command block-export-add for --export.

---
 qemu-storage-daemon.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)
Reviewed-by on v1 5/22
#19 lore raw

Re: [RFC PATCH 05/22] qemu-storage-daemon: Use qmp_block_export_add()

In reply to Kevin Wolf
Quoted 8 lines · Kevin Wolf

No reason to duplicate the functionality locally, we can now just reuse the QMP command block-export-add for --export.

---
  qemu-storage-daemon.c | 13 +------------
  1 file changed, 1 insertion(+), 12 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#20 lore raw

RFC PATCH 6/22 qemu-nbd: Use raw block driver for --offset

In reply to Kevin Wolf

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command offers, but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

---
 include/block/nbd.h |  4 ++--
 blockdev-nbd.c      |  9 +--------
 nbd/server.c        | 34 +++++++++++++++++-----------------
 qemu-nbd.c          | 27 ++++++++++++---------------
 4 files changed, 32 insertions(+), 42 deletions(-)
include/block/nbd.h +2 -2
index c8c5cb6b61..3846d2bac8 100644
@@ -329,8 +329,8 @@ typedef struct NBDExport NBDExport;
 typedef struct NBDClient NBDClient;
 
 BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp);
-NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
-                          uint64_t size, const char *name, const char *desc,
+NBDExport *nbd_export_new(BlockDriverState *bs,
+                          const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
                           void (*close)(NBDExport *), bool writethrough,
                           BlockBackend *on_eject_blk, Error **errp);
blockdev-nbd.c +1 -8
index a1dc11bdd7..16cda3b052 100644
@@ -154,7 +154,6 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
     BlockDriverState *bs = NULL;
     BlockBackend *on_eject_blk;
     NBDExport *exp = NULL;
-    int64_t len;
     AioContext *aio_context;
 
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
@@ -192,12 +191,6 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     aio_context = bdrv_get_aio_context(bs);
     aio_context_acquire(aio_context);
-    len = bdrv_getlength(bs);
-    if (len < 0) {
-        error_setg_errno(errp, -len,
-                         "Failed to determine the NBD export's length");
-        goto out;
-    }
 
     if (!arg->has_writable) {
         arg->writable = false;
@@ -206,7 +199,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
         arg->writable = false;
     }
 
-    exp = nbd_export_new(bs, 0, len, arg->name, arg->description, arg->bitmap,
+    exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
                          !arg->writable, !arg->writable,
                          NULL, false, on_eject_blk, errp);
     if (!exp) {
nbd/server.c +17 -17
index 774325dbe5..92360d1f08 100644
@@ -89,7 +89,6 @@ struct NBDExport {
     BlockBackend *blk;
     char *name;
     char *description;
-    uint64_t dev_offset;
     uint64_t size;
     uint16_t nbdflags;
     QTAILQ_HEAD(, NBDClient) clients;
@@ -1507,8 +1506,8 @@ static void nbd_eject_notifier(Notifier *n, void *data)
     aio_context_release(aio_context);
 }
 
-NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
-                          uint64_t size, const char *name, const char *desc,
+NBDExport *nbd_export_new(BlockDriverState *bs,
+                          const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
                           void (*close)(NBDExport *), bool writethrough,
                           BlockBackend *on_eject_blk, Error **errp)
@@ -1516,9 +1515,17 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
     AioContext *ctx;
     BlockBackend *blk;
     NBDExport *exp;
+    int64_t size;
     uint64_t perm;
     int ret;
 
+    size = bdrv_getlength(bs);
+    if (size < 0) {
+        error_setg_errno(errp, -size,
+                         "Failed to determine the NBD export's length");
+        return NULL;
+    }
+
     exp = g_new0(NBDExport, 1);
     exp->common = (BlockExport) {
         .drv = &blk_exp_nbd,
@@ -1553,8 +1560,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
     exp->refcount = 1;
     QTAILQ_INIT(&exp->clients);
     exp->blk = blk;
-    assert(dev_offset <= INT64_MAX);
-    exp->dev_offset = dev_offset;
     exp->name = g_strdup(name);
     assert(!desc || strlen(desc) <= NBD_MAX_STRING_SIZE);
     exp->description = g_strdup(desc);
@@ -1569,7 +1574,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
         exp->nbdflags |= (NBD_FLAG_SEND_TRIM | NBD_FLAG_SEND_WRITE_ZEROES |
                           NBD_FLAG_SEND_FAST_ZERO);
     }
-    assert(size <= INT64_MAX - dev_offset);
+    assert(size <= INT64_MAX);
     exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
 
     if (bitmap) {
@@ -1928,8 +1933,7 @@ static int coroutine_fn nbd_co_send_sparse_read(NBDClient *client,
             stl_be_p(&chunk.length, pnum);
             ret = nbd_co_send_iov(client, iov, 1, errp);
         } else {
-            ret = blk_pread(exp->blk, offset + progress + exp->dev_offset,
-                            data + progress, pnum);
+            ret = blk_pread(exp->blk, offset + progress, data + progress, pnum);
             if (ret < 0) {
                 error_setg_errno(errp, -ret, "reading from file failed");
                 break;
@@ -2303,8 +2307,7 @@ static coroutine_fn int nbd_do_cmd_read(NBDClient *client, NBDRequest *request,
                                        data, request->len, errp);
     }
 
-    ret = blk_pread(exp->blk, request->from + exp->dev_offset, data,
-                    request->len);
+    ret = blk_pread(exp->blk, request->from, data, request->len);
     if (ret < 0) {
         return nbd_send_generic_reply(client, request->handle, ret,
                                       "reading from file failed", errp);
@@ -2339,7 +2342,7 @@ static coroutine_fn int nbd_do_cmd_cache(NBDClient *client, NBDRequest *request,
 
     assert(request->type == NBD_CMD_CACHE);
 
-    ret = blk_co_preadv(exp->blk, request->from + exp->dev_offset, request->len,
+    ret = blk_co_preadv(exp->blk, request->from, request->len,
                         NULL, BDRV_REQ_COPY_ON_READ | BDRV_REQ_PREFETCH);
 
     return nbd_send_generic_reply(client, request->handle, ret,
@@ -2370,8 +2373,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
         if (request->flags & NBD_CMD_FLAG_FUA) {
             flags |= BDRV_REQ_FUA;
         }
-        ret = blk_pwrite(exp->blk, request->from + exp->dev_offset,
-                         data, request->len, flags);
+        ret = blk_pwrite(exp->blk, request->from, data, request->len, flags);
         return nbd_send_generic_reply(client, request->handle, ret,
                                       "writing to file failed", errp);
 
@@ -2386,8 +2388,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
         if (request->flags & NBD_CMD_FLAG_FAST_ZERO) {
             flags |= BDRV_REQ_NO_FALLBACK;
         }
-        ret = blk_pwrite_zeroes(exp->blk, request->from + exp->dev_offset,
-                                request->len, flags);
+        ret = blk_pwrite_zeroes(exp->blk, request->from, request->len, flags);
         return nbd_send_generic_reply(client, request->handle, ret,
                                       "writing to file failed", errp);
 
@@ -2401,8 +2402,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
                                       "flush failed", errp);
 
     case NBD_CMD_TRIM:
-        ret = blk_co_pdiscard(exp->blk, request->from + exp->dev_offset,
-                              request->len);
+        ret = blk_co_pdiscard(exp->blk, request->from, request->len);
         if (ret == 0 && request->flags & NBD_CMD_FLAG_FUA) {
             ret = blk_co_flush(exp->blk);
         }
qemu-nbd.c +12 -15
index d2657b8db5..818c3f5d46 100644
@@ -523,7 +523,6 @@ int main(int argc, char **argv)
     const char *port = NULL;
     char *sockpath = NULL;
     char *device = NULL;
-    int64_t fd_size;
     QemuOpts *sn_opts = NULL;
     const char *sn_id_or_name = NULL;
     const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L";
@@ -1028,6 +1027,17 @@ int main(int argc, char **argv)
     }
     bs = blk_bs(blk);
 
+    if (dev_offset) {
+        QDict *raw_opts = qdict_new();
+        qdict_put_str(raw_opts, "driver", "raw");
+        qdict_put_str(raw_opts, "file", bs->node_name);
+        qdict_put_int(raw_opts, "offset", dev_offset);
+        bs = bdrv_open(NULL, NULL, raw_opts, flags, &error_fatal);
+        blk_remove_bs(blk);
+        blk_insert_bs(blk, bs, &error_fatal);
+        bdrv_unref(bs);
+    }
+
     blk_set_enable_write_cache(blk, !writethrough);
 
     if (sn_opts) {
@@ -1045,21 +1055,8 @@ int main(int argc, char **argv)
     }
 
     bs->detect_zeroes = detect_zeroes;
-    fd_size = blk_getlength(blk);
-    if (fd_size < 0) {
-        error_report("Failed to determine the image length: %s",
-                     strerror(-fd_size));
-        exit(EXIT_FAILURE);
-    }
-
-    if (dev_offset >= fd_size) {
-        error_report("Offset (%" PRIu64 ") has to be smaller than the image "
-                     "size (%" PRId64 ")", dev_offset, fd_size);
-        exit(EXIT_FAILURE);
-    }
-    fd_size -= dev_offset;
 
-    export = nbd_export_new(bs, dev_offset, fd_size, export_name,
+    export = nbd_export_new(bs, export_name,
                             export_description, bitmap, readonly, shared > 1,
                             nbd_export_closed, writethrough, NULL,
                             &error_fatal);
Signature · 1 line
2.25.4
Reviewed-by on v1 6/22
#21 lore raw

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

Quoted 10 lines · Kevin Wolf

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command offers, but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

Quoted 7 lines
---
 include/block/nbd.h |  4 ++--
 blockdev-nbd.c      |  9 +--------
 nbd/server.c        | 34 +++++++++++++++++-----------------
 qemu-nbd.c          | 27 ++++++++++++---------------
 4 files changed, 32 insertions(+), 42 deletions(-)
#22 lore raw

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

In reply to Kevin Wolf
Quoted 17 lines · Kevin Wolf

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command offers, but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

---
 include/block/nbd.h |  4 ++--
 blockdev-nbd.c      |  9 +--------
 nbd/server.c        | 34 +++++++++++++++++-----------------
 qemu-nbd.c          | 27 ++++++++++++---------------
 4 files changed, 32 insertions(+), 42 deletions(-)

[...]

Quoted 4 lines
nbd/server.c +0 -0
index 774325dbe5..92360d1f08 100644

[...]

nbd/server.c NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset, · line 1577

Quoted 6 lines
nbd/server.c +1 -1
@@ -1569,7 +1574,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
         exp->nbdflags |= (NBD_FLAG_SEND_TRIM | NBD_FLAG_SEND_WRITE_ZEROES |
                           NBD_FLAG_SEND_FAST_ZERO);
     }
-    assert(size <= INT64_MAX - dev_offset);
+    assert(size <= INT64_MAX);

Forgot to note: I think we can drop this assertion altogether now.

Max

#23 lore raw

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

In reply to Kevin Wolf
Quoted 10 lines · Kevin Wolf

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command offers, but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

Just to make sure I understand this correctly -

qemu-nbd can work with:

    $ qemu-nbd 'json:{"driver": "file", "filename": "test.raw"}'

And:

    $ qemu-nbd 'json:{"driver": "raw", "file": {"driver": "file",
"filename": "test.raw"}}'

I assumed that we always create the raw node?

oVirt always uses the second form to make it easier to support offset, size, and backing.
https://github.com/oVirt/ovirt-imageio/blob/2021164d064227d7c5e03c8da087adc66e3a577e/daemon/ovirt_imageio/_internal/qemu_nbd.py#L104

This also seems to be the way libvirt builds the nodes using -blockdev.

Do we have a way to visualize the internal node graph used by qemu-nbd/qemu-img?

Nir

Quoted 256 lines
---
 include/block/nbd.h |  4 ++--
 blockdev-nbd.c      |  9 +--------
 nbd/server.c        | 34 +++++++++++++++++-----------------
 qemu-nbd.c          | 27 ++++++++++++---------------
 4 files changed, 32 insertions(+), 42 deletions(-)
include/block/nbd.h +2 -2
index c8c5cb6b61..3846d2bac8 100644
@@ -329,8 +329,8 @@ typedef struct NBDExport NBDExport;
 typedef struct NBDClient NBDClient;
 
 BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error
 **errp);
-NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
-                          uint64_t size, const char *name, const char
 *desc,
+NBDExport *nbd_export_new(BlockDriverState *bs,
+                          const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,

void (*close)(NBDExport *), bool writethrough, BlockBackend *on_eject_blk, Error **errp);

blockdev-nbd.c +0 -1
index a1dc11bdd7..16cda3b052 100644
@@ -154,7 +154,6 @@ BlockExport *nbd_export_create(BlockExportOptions
 *exp_args, Error **errp)
     BlockDriverState *bs = NULL;
     BlockBackend *on_eject_blk;
     NBDExport *exp = NULL;
-    int64_t len;
     AioContext *aio_context;
 
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
blockdev-nbd.c +0 -6
@@ -192,12 +191,6 @@ BlockExport *nbd_export_create(BlockExportOptions
 *exp_args, Error **errp)
 
     aio_context = bdrv_get_aio_context(bs);
     aio_context_acquire(aio_context);
-    len = bdrv_getlength(bs);
-    if (len < 0) {
-        error_setg_errno(errp, -len,
-                         "Failed to determine the NBD export's length");
-        goto out;
-    }
 
     if (!arg->has_writable) {
         arg->writable = false;
blockdev-nbd.c +1 -1
@@ -206,7 +199,7 @@ BlockExport *nbd_export_create(BlockExportOptions
 *exp_args, Error **errp)
         arg->writable = false;
     }
 
-    exp = nbd_export_new(bs, 0, len, arg->name, arg->description,
 arg->bitmap,
+    exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
                          !arg->writable, !arg->writable,
                          NULL, false, on_eject_blk, errp);
     if (!exp) {
nbd/server.c +2 -3
index 774325dbe5..92360d1f08 100644
@@ -89,7 +89,6 @@ struct NBDExport {
     BlockBackend *blk;
     char *name;
     char *description;
-    uint64_t dev_offset;
     uint64_t size;
     uint16_t nbdflags;
     QTAILQ_HEAD(, NBDClient) clients;
@@ -1507,8 +1506,8 @@ static void nbd_eject_notifier(Notifier *n, void
 *data)
     aio_context_release(aio_context);
 }
 
-NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
-                          uint64_t size, const char *name, const char
 *desc,
+NBDExport *nbd_export_new(BlockDriverState *bs,
+                          const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,

void (*close)(NBDExport *), bool writethrough, BlockBackend *on_eject_blk, Error **errp)

nbd/server.c +8 -0
@@ -1516,9 +1515,17 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
 uint64_t dev_offset,
     AioContext *ctx;
     BlockBackend *blk;
     NBDExport *exp;
+    int64_t size;
     uint64_t perm;
     int ret;
 
+    size = bdrv_getlength(bs);
+    if (size < 0) {
+        error_setg_errno(errp, -size,
+                         "Failed to determine the NBD export's length");
+        return NULL;
+    }
+
     exp = g_new0(NBDExport, 1);
     exp->common = (BlockExport) {
         .drv = &blk_exp_nbd,
nbd/server.c +0 -2
@@ -1553,8 +1560,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
 uint64_t dev_offset,
     exp->refcount = 1;
     QTAILQ_INIT(&exp->clients);
     exp->blk = blk;
-    assert(dev_offset <= INT64_MAX);
-    exp->dev_offset = dev_offset;
     exp->name = g_strdup(name);
     assert(!desc || strlen(desc) <= NBD_MAX_STRING_SIZE);
     exp->description = g_strdup(desc);
nbd/server.c +0 -0
@@ -1569,7 +1574,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
 uint64_t dev_offset,
         exp->nbdflags |= (NBD_FLAG_SEND_TRIM | NBD_FLAG_SEND_WRITE_ZEROES

|

                           NBD_FLAG_SEND_FAST_ZERO);
     }
nbd/server.c +2 -3
-    assert(size <= INT64_MAX - dev_offset);
+    assert(size <= INT64_MAX);
     exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
 
     if (bitmap) {
@@ -1928,8 +1933,7 @@ static int coroutine_fn
 nbd_co_send_sparse_read(NBDClient *client,
             stl_be_p(&chunk.length, pnum);
             ret = nbd_co_send_iov(client, iov, 1, errp);
         } else {
-            ret = blk_pread(exp->blk, offset + progress + exp->dev_offset,
-                            data + progress, pnum);
+            ret = blk_pread(exp->blk, offset + progress, data + progress,
 pnum);
             if (ret < 0) {
                 error_setg_errno(errp, -ret, "reading from file failed");
                 break;
nbd/server.c +1 -2
@@ -2303,8 +2307,7 @@ static coroutine_fn int nbd_do_cmd_read(NBDClient
 *client, NBDRequest *request,
                                        data, request->len, errp);
     }
 
-    ret = blk_pread(exp->blk, request->from + exp->dev_offset, data,
-                    request->len);
+    ret = blk_pread(exp->blk, request->from, data, request->len);
     if (ret < 0) {
         return nbd_send_generic_reply(client, request->handle, ret,
                                       "reading from file failed", errp);
nbd/server.c +1 -1
@@ -2339,7 +2342,7 @@ static coroutine_fn int nbd_do_cmd_cache(NBDClient
 *client, NBDRequest *request,
 
     assert(request->type == NBD_CMD_CACHE);
 
-    ret = blk_co_preadv(exp->blk, request->from + exp->dev_offset,
 request->len,
+    ret = blk_co_preadv(exp->blk, request->from, request->len,
                         NULL, BDRV_REQ_COPY_ON_READ | BDRV_REQ_PREFETCH);
     return nbd_send_generic_reply(client, request->handle, ret,
nbd/server.c +1 -2
@@ -2370,8 +2373,7 @@ static coroutine_fn int nbd_handle_request(NBDClient
 *client,
         if (request->flags & NBD_CMD_FLAG_FUA) {
             flags |= BDRV_REQ_FUA;
         }
-        ret = blk_pwrite(exp->blk, request->from + exp->dev_offset,
-                         data, request->len, flags);
+        ret = blk_pwrite(exp->blk, request->from, data, request->len,
 flags);
         return nbd_send_generic_reply(client, request->handle, ret,
                                       "writing to file failed", errp);
nbd/server.c +1 -2
@@ -2386,8 +2388,7 @@ static coroutine_fn int nbd_handle_request(NBDClient
 *client,
         if (request->flags & NBD_CMD_FLAG_FAST_ZERO) {
             flags |= BDRV_REQ_NO_FALLBACK;
         }
-        ret = blk_pwrite_zeroes(exp->blk, request->from + exp->dev_offset,
-                                request->len, flags);
+        ret = blk_pwrite_zeroes(exp->blk, request->from, request->len,
 flags);
         return nbd_send_generic_reply(client, request->handle, ret,
                                       "writing to file failed", errp);
nbd/server.c +1 -2
@@ -2401,8 +2402,7 @@ static coroutine_fn int nbd_handle_request(NBDClient
 *client,
                                       "flush failed", errp);
 
     case NBD_CMD_TRIM:
-        ret = blk_co_pdiscard(exp->blk, request->from + exp->dev_offset,
-                              request->len);
+        ret = blk_co_pdiscard(exp->blk, request->from, request->len);
         if (ret == 0 && request->flags & NBD_CMD_FLAG_FUA) {
             ret = blk_co_flush(exp->blk);
         }
qemu-nbd.c +11 -10
index d2657b8db5..818c3f5d46 100644
@@ -523,7 +523,6 @@ int main(int argc, char **argv)
     const char *port = NULL;
     char *sockpath = NULL;
     char *device = NULL;
-    int64_t fd_size;
     QemuOpts *sn_opts = NULL;
     const char *sn_id_or_name = NULL;
     const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L";
@@ -1028,6 +1027,17 @@ int main(int argc, char **argv)
     }
     bs = blk_bs(blk);
 
+    if (dev_offset) {
+        QDict *raw_opts = qdict_new();
+        qdict_put_str(raw_opts, "driver", "raw");
+        qdict_put_str(raw_opts, "file", bs->node_name);
+        qdict_put_int(raw_opts, "offset", dev_offset);
+        bs = bdrv_open(NULL, NULL, raw_opts, flags, &error_fatal);
+        blk_remove_bs(blk);
+        blk_insert_bs(blk, bs, &error_fatal);
+        bdrv_unref(bs);
+    }
+
     blk_set_enable_write_cache(blk, !writethrough);
 
     if (sn_opts) {
@@ -1045,21 +1055,8 @@ int main(int argc, char **argv)
     }
 
     bs->detect_zeroes = detect_zeroes;
-    fd_size = blk_getlength(blk);
-    if (fd_size < 0) {
-        error_report("Failed to determine the image length: %s",
-                     strerror(-fd_size));
-        exit(EXIT_FAILURE);
-    }
-
-    if (dev_offset >= fd_size) {
-        error_report("Offset (%" PRIu64 ") has to be smaller than the

image "

qemu-nbd.c +1 -5
-                     "size (%" PRId64 ")", dev_offset, fd_size);
-        exit(EXIT_FAILURE);
-    }
-    fd_size -= dev_offset;
 
-    export = nbd_export_new(bs, dev_offset, fd_size, export_name,
+    export = nbd_export_new(bs, export_name,
                             export_description, bitmap, readonly, shared
Quoted 1 line

1,

nbd_export_closed, writethrough, NULL, &error_fatal);
--
2.25.4

author
#24 lore raw

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

Am 17.08.2020 um 19:19 hat Nir Soffer geschrieben:

Quoted 17 lines · Kevin Wolf
Quoted 11 lines

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command offers, but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

Just to make sure I understand this correctly -

qemu-nbd can work with:

Quoted 8 lines
    $ qemu-nbd 'json:{"driver": "file", "filename": "test.raw"}'

And:

    $ qemu-nbd 'json:{"driver": "raw", "file": {"driver": "file",
"filename": "test.raw"}}'

I assumed that we always create the raw node?

No, the first form creates only the 'file' node without a 'raw' node on top. For all practical matters, this should be the same in qemu-img or qemu-nbd. For actually running VMs, omitting the 'raw' node where it's not needed can improve performance a little.

What is true is that if you use a filename without specifying the driver (i.e. you rely on format probing), you'll get a 'raw' node on top of the 'file' node.

Quoted 5 lines

oVirt always uses the second form to make it easier to support offset, size, and backing.
https://github.com/oVirt/ovirt-imageio/blob/2021164d064227d7c5e03c8da087adc66e3a577e/daemon/ovirt_imageio/_internal/qemu_nbd.py#L104

This also seems to be the way libvirt builds the nodes using -blockdev.

libvirt actually has a BZ to avoid the 'raw' node for performance when it's not needed.

Quoted 2 lines

Do we have a way to visualize the internal node graph used by qemu-nbd/qemu-img?

No, but as long as you explicitly specify the driver, you get exactly what you specified.

For exploring what happens, you can pass the same json: filename to QEMU (maybe with -hda) and then use the monitor to inspect the state.

Kevin

#25 lore raw

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

Quoted 26 lines · Kevin Wolf

Am 17.08.2020 um 19:19 hat Nir Soffer geschrieben:

Quoted 8 lines · Kevin Wolf
Quoted 7 lines

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command

offers,

Quoted 16 lines
Quoted 4 lines

but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

Just to make sure I understand this correctly -

qemu-nbd can work with:

    $ qemu-nbd 'json:{"driver": "file", "filename": "test.raw"}'

And:

    $ qemu-nbd 'json:{"driver": "raw", "file": {"driver": "file",
"filename": "test.raw"}}'
Quoted 7 lines
Quoted 1 line

I assumed that we always create the raw node?

No, the first form creates only the 'file' node without a 'raw' node on top. For all practical matters, this should be the same in qemu-img or qemu-nbd. For actually running VMs, omitting the 'raw' node where it's not needed can improve performance a little.

We did not check if we have different performance with the extra raw node. Since in our use case (copying images) small reads/writes are unlikely, I don't
think it will make a difference.

What is true is that if you use a filename without specifying the driver

Quoted 10 lines

(i.e. you rely on format probing), you'll get a 'raw' node on top of the 'file' node.

Quoted 3 lines

oVirt always uses the second form to make it easier to support offset, size, and backing.

https://github.com/oVirt/ovirt-imageio/blob/2021164d064227d7c5e03c8da087adc66e3a577e/daemon/ovirt_imageio/_internal/qemu_nbd.py#L104

Quoted 2 lines

This also seems to be the way libvirt builds the nodes using -blockdev.

Quoted 9 lines

libvirt actually has a BZ to avoid the 'raw' node for performance when it's not needed.

Quoted 2 lines

Do we have a way to visualize the internal node graph used by qemu-nbd/qemu-img?

No, but as long as you explicitly specify the driver, you get exactly what you specified.

So this is not really needed then.

Quoted 6 lines

For exploring what happens, you can pass the same json: filename to QEMU (maybe with -hda) and then use the monitor to inspect the state.

Kevin

Reviewed-by on v1 6/22
#26 lore raw

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

In reply to Kevin Wolf
Quoted 9 lines · Kevin Wolf

Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job.

This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP command offers, but in fact it also eliminates a potential source for bugs in the NBD code which previously had to add the offset manually in all relevant places.

Yay! This patch alone is worth having, regardless of the fate of the rest of the series: no change in end-user functionality, but by making qemu-nbd turn it into proper syntactic sugar, we've reduced the maintenance burden of duplicated code.

Quoted 12 lines
---
  include/block/nbd.h |  4 ++--
  blockdev-nbd.c      |  9 +--------
  nbd/server.c        | 34 +++++++++++++++++-----------------
  qemu-nbd.c          | 27 ++++++++++++---------------
  4 files changed, 32 insertions(+), 42 deletions(-)

+++ b/nbd/server.c

nbd/server.c struct NBDExport { · line 92

Quoted 5 lines
nbd/server.c +0 -1
@@ -89,7 +89,6 @@ struct NBDExport {
      BlockBackend *blk;
      char *name;
      char *description;
-    uint64_t dev_offset;
      uint64_t size;

I'm trying to figure out if we can also drop 'size' here. If we do, the consequence would be that an NBD client could ask for beyond-EOF I/O, and presumably the block layer would reject that gracefully (although not necessarily with the same errno as NBD currently reports). I'm fine leaving it alone in this patch, though.

nbd/server.c NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset, · line 1577

Quoted 6 lines
nbd/server.c +1 -1
@@ -1569,7 +1574,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
          exp->nbdflags |= (NBD_FLAG_SEND_TRIM | NBD_FLAG_SEND_WRITE_ZEROES |
                            NBD_FLAG_SEND_FAST_ZERO);
      }
-    assert(size <= INT64_MAX - dev_offset);
+    assert(size <= INT64_MAX);

As Max caught, this is now dead code.

Quoted 11 lines
nbd/server.c +1 -2
@@ -2386,8 +2388,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
          if (request->flags & NBD_CMD_FLAG_FAST_ZERO) {
              flags |= BDRV_REQ_NO_FALLBACK;
          }
-        ret = blk_pwrite_zeroes(exp->blk, request->from + exp->dev_offset,
-                                request->len, flags);
+        ret = blk_pwrite_zeroes(exp->blk, request->from, request->len, flags);
          return nbd_send_generic_reply(client, request->handle, ret,
                                        "writing to file failed", errp);
 

nbd/server.c static coroutine_fn int nbd_handle_request(NBDClient *client, · line 2405

Quoted 6 lines
nbd/server.c +1 -2
@@ -2401,8 +2402,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,
                                        "flush failed", errp);
 
      case NBD_CMD_TRIM:
-        ret = blk_co_pdiscard(exp->blk, request->from + exp->dev_offset,
-                              request->len);
+        ret = blk_co_pdiscard(exp->blk, request->from, request->len);

Merge conflicts with 890cbccb08; should be obvious enough to resolve, though.

Quoted 10 lines
qemu-nbd.c +1 -1
+++ b/qemu-nbd.c
@@ -523,7 +523,6 @@ int main(int argc, char **argv)
      const char *port = NULL;
      char *sockpath = NULL;
      char *device = NULL;
-    int64_t fd_size;
      QemuOpts *sn_opts = NULL;
      const char *sn_id_or_name = NULL;
      const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L";

qemu-nbd.c int main(int argc, char **argv) · line 1034

Quoted 8 lines
qemu-nbd.c +5 -0
@@ -1028,6 +1027,17 @@ int main(int argc, char **argv)
      }
      bs = blk_bs(blk);
 
+    if (dev_offset) {
+        QDict *raw_opts = qdict_new();
+        qdict_put_str(raw_opts, "driver", "raw");
+        qdict_put_str(raw_opts, "file", bs->node_name);
+        qdict_put_int(raw_opts, "offset", dev_offset);

Huh. When 0bc16997f5 got rid of the --partition option, it also got rid of the only way that the NBD driver could clamp down requests to a range smaller than the end of the file. Now that you are adding a raw driver in the mix, that ability to clamp the end of the range (aka a --size option, in addition to an --offset option) may be worth reinstating. But that can be done as a separate patch, if at all (and whether qemu-nbd should do it, or qemu-storage-daemon, or whether we just point people at 'nbdkit --filter=partition', is part of that discussion). But for this patch, it looks like you are making a straight-across conversion.

qemu-nbd.c int main(int argc, char **argv) · line 1040

Quoted 6 lines
qemu-nbd.c +6 -0
+        bs = bdrv_open(NULL, NULL, raw_opts, flags, &error_fatal);
+        blk_remove_bs(blk);
+        blk_insert_bs(blk, bs, &error_fatal);
+        bdrv_unref(bs);
+    }
+

Slick.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#27 lore raw

RFC PATCH 7/22 block/export: Remove magic from block-export-add

In reply to Kevin Wolf

nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports:

1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context of block-export-add.

2. When using a BlockBackend name, unplugging the device from the guest will automatically stop the NBD server, too. This may sometimes be what you want, but it could also be very surprising. Let's keep things explicit with block-export-add. If the user wants to stop the export, they should tell us so.

Move these things into the nbd-server-add QMP command handler so that they apply only there.

---
 include/block/nbd.h   |  3 ++-
 block/export/export.c | 44 ++++++++++++++++++++++++++++++++++++++-----
 blockdev-nbd.c        | 10 ++++------
 nbd/server.c          | 19 ++++++++++++-------
 qemu-nbd.c            |  3 +--
 5 files changed, 58 insertions(+), 21 deletions(-)
include/block/nbd.h +2 -1
index 3846d2bac8..ffca3be78f 100644
@@ -333,7 +333,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
                           const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
                           void (*close)(NBDExport *), bool writethrough,
-                          BlockBackend *on_eject_blk, Error **errp);
+                          Error **errp);
+void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
 void nbd_export_close(NBDExport *exp);
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
 void nbd_export_get(NBDExport *exp);
block/export/export.c +39 -5

69 lines not on this page · raw message

blockdev-nbd.c +4 -6
index 16cda3b052..019c37c0bc 100644
@@ -152,7 +152,6 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 {
     BlockExportOptionsNbd *arg = &exp_args->u.nbd;
     BlockDriverState *bs = NULL;
-    BlockBackend *on_eject_blk;
     NBDExport *exp = NULL;
     AioContext *aio_context;
 
@@ -182,8 +181,6 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
         return NULL;
     }
 
-    on_eject_blk = blk_by_name(arg->device);
-
     bs = bdrv_lookup_bs(arg->device, arg->device, errp);
     if (!bs) {
         return NULL;
@@ -195,13 +192,14 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
     if (!arg->has_writable) {
         arg->writable = false;
     }
-    if (bdrv_is_read_only(bs)) {
-        arg->writable = false;
+    if (bdrv_is_read_only(bs) && arg->writable) {
+        error_setg(errp, "Cannot export read-only node as writable");
+        goto out;
     }
 
     exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
                          !arg->writable, !arg->writable,
-                         NULL, false, on_eject_blk, errp);
+                         NULL, false, errp);
     if (!exp) {
         goto out;
     }
nbd/server.c +12 -7
index 92360d1f08..0b84fd30e2 100644
@@ -1506,11 +1506,22 @@ static void nbd_eject_notifier(Notifier *n, void *data)
     aio_context_release(aio_context);
 }
 
+void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk)
+{
+    NBDExport *nbd_exp = container_of(exp, NBDExport, common);
+    assert(exp->drv == &blk_exp_nbd);
+
+    blk_ref(blk);
+    nbd_exp->eject_notifier_blk = blk;
+    nbd_exp->eject_notifier.notify = nbd_eject_notifier;
+    blk_add_remove_bs_notifier(blk, &nbd_exp->eject_notifier);
+}
+
 NBDExport *nbd_export_new(BlockDriverState *bs,
                           const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
                           void (*close)(NBDExport *), bool writethrough,
-                          BlockBackend *on_eject_blk, Error **errp)
+                          Error **errp)
 {
     AioContext *ctx;
     BlockBackend *blk;
@@ -1618,12 +1629,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
     exp->ctx = ctx;
     blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
 
-    if (on_eject_blk) {
-        blk_ref(on_eject_blk);
-        exp->eject_notifier_blk = on_eject_blk;
-        exp->eject_notifier.notify = nbd_eject_notifier;
-        blk_add_remove_bs_notifier(on_eject_blk, &exp->eject_notifier);
-    }
     QTAILQ_INSERT_TAIL(&exports, exp, next);
     nbd_export_get(exp);
     return exp;
qemu-nbd.c +1 -2
index 818c3f5d46..e348d5d6d8 100644
@@ -1058,8 +1058,7 @@ int main(int argc, char **argv)
 
     export = nbd_export_new(bs, export_name,
                             export_description, bitmap, readonly, shared > 1,
-                            nbd_export_closed, writethrough, NULL,
-                            &error_fatal);
+                            nbd_export_closed, writethrough, &error_fatal);
 
     if (device) {
 #if HAVE_NBD_DEVICE
Signature · 1 line
2.25.4
#28 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

Quoted 25 lines · Kevin Wolf

nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports:

1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context of block-export-add.

2. When using a BlockBackend name, unplugging the device from the guest will automatically stop the NBD server, too. This may sometimes be what you want, but it could also be very surprising. Let's keep things explicit with block-export-add. If the user wants to stop the export, they should tell us so.

Move these things into the nbd-server-add QMP command handler so that they apply only there.

---
 include/block/nbd.h   |  3 ++-
 block/export/export.c | 44 ++++++++++++++++++++++++++++++++++++++-----
 blockdev-nbd.c        | 10 ++++------
 nbd/server.c          | 19 ++++++++++++-------
 qemu-nbd.c            |  3 +--
 5 files changed, 58 insertions(+), 21 deletions(-)

[...]

Quoted 4 lines
block/export/export.c +0 -0
index 3d0dacb3f2..2d5f92861c 100644

[...]

Quoted 14 lines
block/export/export.c +2 -2
@@ -34,24 +36,56 @@ static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
     return NULL;
 }
 
-void qmp_block_export_add(BlockExportOptions *export, Error **errp)
+static BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
 {
     const BlockExportDriver *drv;
 
     drv = blk_exp_find_driver(export->type);
     if (!drv) {
         error_setg(errp, "No driver found for the requested export type");
-        return;
+        return NULL;

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 55

Quoted 10 lines
block/export/export.c +6 -1
     }
 
-    drv->create(export, errp);
+    return drv->create(export, errp);
+}
+
+void qmp_block_export_add(BlockExportOptions *export, Error **errp)
+{
+    blk_exp_add(export, errp);
 }

Interesting. I would have added it this way from the start then (with a note that we’ll need it later).

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 65

Quoted 10 lines
block/export/export.c +5 -1
 void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 {
-    BlockExportOptions export = {
+    BlockExport *export;
+    BlockDriverState *bs;
+    BlockBackend *on_eject_blk;
+
+    BlockExportOptions export_opts = {
         .type = BLOCK_EXPORT_TYPE_NBD,
         .u.nbd = *arg,

This copies *arg’s contents...

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 75

Quoted 11 lines
block/export/export.c +9 -1
     };
-    qmp_block_export_add(&export, errp);
+
+    /*
+     * nbd-server-add doesn't complain when a read-only device should be
+     * exported as writable, but simply downgrades it. This is an error with
+     * block-export-add.
+     */
+    bs = bdrv_lookup_bs(arg->device, arg->device, NULL);
+    if (bs && bdrv_is_read_only(bs)) {
+        arg->writable = false;

...and here you only modify the original *arg, but not export_opts.u.nbd. So I don’t think this will have any effect.

Quoted 6 lines
block/export/export.c +6 -0
+    }
+
+    export = blk_exp_add(&export_opts, errp);
+    if (!export) {
+        return;
+    }

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 91

Quoted 10 lines
block/export/export.c +9 -0
+
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }
 }

The longer it gets, the more I think maybe it should be in some NBD file like blockdev-nbd.c after all.

[...]

nbd/server.c static void nbd_eject_notifier(Notifier *n, void *data) · line 1513

Quoted 13 lines
nbd/server.c +5 -0
index 92360d1f08..0b84fd30e2 100644
@@ -1506,11 +1506,22 @@ static void nbd_eject_notifier(Notifier *n, void *data)
     aio_context_release(aio_context);
 }
 
+void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk)
+{
+    NBDExport *nbd_exp = container_of(exp, NBDExport, common);
+    assert(exp->drv == &blk_exp_nbd);
+

I think asserting that the nbd_exp->eject_notifier is unused so far would make sense (e.g. just checking that eject_notifier_blk is NULL).

Max

Quoted 6 lines
nbd/server.c +6 -0
+    blk_ref(blk);
+    nbd_exp->eject_notifier_blk = blk;
+    nbd_exp->eject_notifier.notify = nbd_eject_notifier;
+    blk_add_remove_bs_notifier(blk, &nbd_exp->eject_notifier);
+}
+
author
#29 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

Am 17.08.2020 um 13:41 hat Max Reitz geschrieben:

Quoted 89 lines · Kevin Wolf
Quoted 25 lines

nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports:

1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context of block-export-add.

2. When using a BlockBackend name, unplugging the device from the guest will automatically stop the NBD server, too. This may sometimes be what you want, but it could also be very surprising. Let's keep things explicit with block-export-add. If the user wants to stop the export, they should tell us so.

Move these things into the nbd-server-add QMP command handler so that they apply only there.

---
 include/block/nbd.h   |  3 ++-
 block/export/export.c | 44 ++++++++++++++++++++++++++++++++++++++-----
 blockdev-nbd.c        | 10 ++++------
 nbd/server.c          | 19 ++++++++++++-------
 qemu-nbd.c            |  3 +--
 5 files changed, 58 insertions(+), 21 deletions(-)

[...]

Quoted 4 lines
block/export/export.c +0 -0
index 3d0dacb3f2..2d5f92861c 100644

[...]

Quoted 14 lines
block/export/export.c +2 -2
@@ -34,24 +36,56 @@ static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
     return NULL;
 }
 
-void qmp_block_export_add(BlockExportOptions *export, Error **errp)
+static BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
 {
     const BlockExportDriver *drv;
 
     drv = blk_exp_find_driver(export->type);
     if (!drv) {
         error_setg(errp, "No driver found for the requested export type");
-        return;
+        return NULL;

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 55

Quoted 10 lines
block/export/export.c +6 -1
     }
 
-    drv->create(export, errp);
+    return drv->create(export, errp);
+}
+
+void qmp_block_export_add(BlockExportOptions *export, Error **errp)
+{
+    blk_exp_add(export, errp);
 }

Interesting. I would have added it this way from the start then (with a note that we’ll need it later).

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 65

Quoted 10 lines
block/export/export.c +5 -1
 void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
 {
-    BlockExportOptions export = {
+    BlockExport *export;
+    BlockDriverState *bs;
+    BlockBackend *on_eject_blk;
+
+    BlockExportOptions export_opts = {
         .type = BLOCK_EXPORT_TYPE_NBD,
         .u.nbd = *arg,

This copies *arg’s contents...

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 75

Quoted 11 lines
block/export/export.c +9 -1
     };
-    qmp_block_export_add(&export, errp);
+
+    /*
+     * nbd-server-add doesn't complain when a read-only device should be
+     * exported as writable, but simply downgrades it. This is an error with
+     * block-export-add.
+     */
+    bs = bdrv_lookup_bs(arg->device, arg->device, NULL);
+    if (bs && bdrv_is_read_only(bs)) {
+        arg->writable = false;
Quoted 2 lines

...and here you only modify the original *arg, but not export_opts.u.nbd. So I don’t think this will have any effect.

I thought I had tested this... Well, good catch, thanks.

Quoted 11 lines
Quoted 11 lines
block/export/export.c +11 -0
+    }
+
+    export = blk_exp_add(&export_opts, errp);
+    if (!export) {
+        return;
+    }
+
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
Quoted 8 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 91

Quoted 5 lines
block/export/export.c +4 -0
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }
 }

The longer it gets, the more I think maybe it should be in some NBD file like blockdev-nbd.c after all.

Fair enough. Though I think blockdev-nbd.c in the root directory is something that shouldn't even exist.

But I guess I can just leave the functions where they are and we can move the file another day.

Quoted 10 lines

[...]

Quoted 8 lines
nbd/server.c +0 -0
index 92360d1f08..0b84fd30e2 100644
@@ -1506,11 +1506,22 @@ static void nbd_eject_notifier(Notifier *n, void *data)
     aio_context_release(aio_context);
 }
 
Quoted 8 lines

nbd/server.c static void nbd_eject_notifier(Notifier *n, void *data) · line 1513

Quoted 5 lines
nbd/server.c +5 -0
+void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk)
+{
+    NBDExport *nbd_exp = container_of(exp, NBDExport, common);
+    assert(exp->drv == &blk_exp_nbd);
+

I think asserting that the nbd_exp->eject_notifier is unused so far would make sense (e.g. just checking that eject_notifier_blk is NULL).

Makes sense.

Kevin

#30 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

Quoted 27 lines · Kevin Wolf

Am 17.08.2020 um 13:41 hat Max Reitz geschrieben:

Quoted 26 lines · Kevin Wolf
Quoted 25 lines

nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports:

1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context of block-export-add.

2. When using a BlockBackend name, unplugging the device from the guest will automatically stop the NBD server, too. This may sometimes be what you want, but it could also be very surprising. Let's keep things explicit with block-export-add. If the user wants to stop the export, they should tell us so.

Move these things into the nbd-server-add QMP command handler so that they apply only there.

---
 include/block/nbd.h   |  3 ++-
 block/export/export.c | 44 ++++++++++++++++++++++++++++++++++++++-----
 blockdev-nbd.c        | 10 ++++------
 nbd/server.c          | 19 ++++++++++++-------
 qemu-nbd.c            |  3 +--
 5 files changed, 58 insertions(+), 21 deletions(-)

[...]

Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
block/export/export.c +11 -0
+    }
+
+    export = blk_exp_add(&export_opts, errp);
+    if (!export) {
+        return;
+    }
+
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
Quoted 8 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 91

Quoted 5 lines
block/export/export.c +4 -0
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }
 }

The longer it gets, the more I think maybe it should be in some NBD file like blockdev-nbd.c after all.

Quoted 2 lines

Fair enough. Though I think blockdev-nbd.c in the root directory is something that shouldn't even exist.

Absolutely. But unless you (or someone™ else) doesn’t do anything about it, we may as well continue to (ab)use it. O:)

Max

#31 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

In reply to Kevin Wolf
cc Peter Krempa
Quoted 7 lines · Kevin Wolf

nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports:

1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context of block-export-add.

I'd be happy for this to be an error even with nbd-export-add; I don't think it would harm any of libvirt's existing usage (either for storage migration, or for incremental backups).

Side note: In the past, I had a proposal to enhance the NBD Protocol to allow a client to advertise to the server its intent on being a read-only or read-write client. Not relevant to this patch, but this part of the commit message reminds me that I should revisit that topic (Rich and I recently hit another case in nbdkit where such an extension would be nice, when it comes to using NBD's multi-conn for better performance on a read-only connection, but only if the server knows the client intends to be read-only)

Quoted 6 lines

2. When using a BlockBackend name, unplugging the device from the guest will automatically stop the NBD server, too. This may sometimes be what you want, but it could also be very surprising. Let's keep things explicit with block-export-add. If the user wants to stop the export, they should tell us so.

Here, keeping the nbd command different from the block-export command seems tolerable. On the other hand, I wonder if Peter needs to change anything in libvirt's incremental backup code to handle this sudden disappearance of an NBD device during a disk hot-unplug (that is, either the presence of an ongoing pull-mode backup should block disk unplug, or libvirt needs a way to guarantee that an ongoing backup NBD device remains in spite of subsequent disk actions on the guest). Depending on libvirt's needs, we may want to revisit the nbd command to have the same policy as block-export-add, plus an introspectible feature notation.

Quoted 20 lines

Move these things into the nbd-server-add QMP command handler so that they apply only there.

---
  include/block/nbd.h   |  3 ++-
block/export/export.c +7 -1
+void qmp_block_export_add(BlockExportOptions *export, Error **errp)
+{
+    blk_exp_add(export, errp);
  }
 
  void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
  {
-    BlockExportOptions export = {
+    BlockExport *export;
+    BlockDriverState *bs;
+    BlockBackend *on_eject_blk;
+

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 71

Quoted 10 lines
block/export/export.c +6 -1
+    BlockExportOptions export_opts = {
          .type = BLOCK_EXPORT_TYPE_NBD,
          .u.nbd = *arg,
      };
-    qmp_block_export_add(&export, errp);
+
+    /*
+     * nbd-server-add doesn't complain when a read-only device should be
+     * exported as writable, but simply downgrades it. This is an error with
+     * block-export-add.

I'd be happy with either marking this deprecated now (and fixing it in two releases), or declaring it a bug in nbd-server-add now (and fixing it outright).

Quoted 9 lines
block/export/export.c +9 -0
+     */
+    bs = bdrv_lookup_bs(arg->device, arg->device, NULL);
+    if (bs && bdrv_is_read_only(bs)) {
+        arg->writable = false;
+    }
+
+    export = blk_exp_add(&export_opts, errp);
+    if (!export) {
+        return;

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 90

Quoted 10 lines
block/export/export.c +10 -0
+    }
+
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }

Wait - is the magic export removal tied only to exporting a drive name, and not a node name? So as long as libvirt is using only node names whwen adding exports, a drive being unplugged won't interfere?

Overall, the change makes sense to me, although I'd love to see if we could go further on the writable vs. read-only issue.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#32 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

Am 19.08.2020 um 21:50 hat Eric Blake geschrieben:

Quoted 30 lines
cc Peter Krempa
Quoted 7 lines · Kevin Wolf

nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports:

1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context of block-export-add.

I'd be happy for this to be an error even with nbd-export-add; I don't think it would harm any of libvirt's existing usage (either for storage migration, or for incremental backups).

Side note: In the past, I had a proposal to enhance the NBD Protocol to allow a client to advertise to the server its intent on being a read-only or read-write client. Not relevant to this patch, but this part of the commit message reminds me that I should revisit that topic (Rich and I recently hit another case in nbdkit where such an extension would be nice, when it comes to using NBD's multi-conn for better performance on a read-only connection, but only if the server knows the client intends to be read-only)

Quoted 6 lines

2. When using a BlockBackend name, unplugging the device from the guest will automatically stop the NBD server, too. This may sometimes be what you want, but it could also be very surprising. Let's keep things explicit with block-export-add. If the user wants to stop the export, they should tell us so.

Quoted 9 lines

Here, keeping the nbd command different from the block-export command seems tolerable. On the other hand, I wonder if Peter needs to change anything in libvirt's incremental backup code to handle this sudden disappearance of an NBD device during a disk hot-unplug (that is, either the presence of an ongoing pull-mode backup should block disk unplug, or libvirt needs a way to guarantee that an ongoing backup NBD device remains in spite of subsequent disk actions on the guest). Depending on libvirt's needs, we may want to revisit the nbd command to have the same policy as block-export-add, plus an introspectible feature notation.

As long as we can keep the compatibility code local to qmp_nbd_*(), I don't think it's too bad. In particular because it's already written.

Instead of adjusting libvirt to changes in the nbd-* commands, I'd rather have it change over to block-export-*. I would like to see the nbd-server-add/remove commands deprecated soon after we have the replacements.

Quoted 25 lines
Quoted 25 lines

Move these things into the nbd-server-add QMP command handler so that they apply only there.

---
  include/block/nbd.h   |  3 ++-
block/export/export.c +9 -2
+void qmp_block_export_add(BlockExportOptions *export, Error **errp)
+{
+    blk_exp_add(export, errp);
  }
  void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
  {
-    BlockExportOptions export = {
+    BlockExport *export;
+    BlockDriverState *bs;
+    BlockBackend *on_eject_blk;
+
+    BlockExportOptions export_opts = {
          .type = BLOCK_EXPORT_TYPE_NBD,
          .u.nbd = *arg,
      };
-    qmp_block_export_add(&export, errp);
+
Quoted 8 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 71

Quoted 4 lines
block/export/export.c +4 -0
+    /*
+     * nbd-server-add doesn't complain when a read-only device should be
+     * exported as writable, but simply downgrades it. This is an error with
+     * block-export-add.

I'd be happy with either marking this deprecated now (and fixing it in two releases), or declaring it a bug in nbd-server-add now (and fixing it outright).

How about deprecating nbd-server-add completely?

Quoted 15 lines
Quoted 15 lines
block/export/export.c +15 -0
+     */
+    bs = bdrv_lookup_bs(arg->device, arg->device, NULL);
+    if (bs && bdrv_is_read_only(bs)) {
+        arg->writable = false;
+    }
+
+    export = blk_exp_add(&export_opts, errp);
+    if (!export) {
+        return;
+    }
+
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
Quoted 8 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 90

Quoted 4 lines
block/export/export.c +4 -0
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }

Wait - is the magic export removal tied only to exporting a drive name, and not a node name? So as long as libvirt is using only node names whwen adding exports, a drive being unplugged won't interfere?

Yes, seems so. It's the existing behaviour, I'm only moving the code around.

Quoted 2 lines

Overall, the change makes sense to me, although I'd love to see if we could go further on the writable vs. read-only issue.

If nbd-server-add will be going away relatively soon, it's probably not worth the trouble. But if you have reasons to keep it, maybe we should consider it.

Kevin

#33 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

Quoted 7 lines · Kevin Wolf

As long as we can keep the compatibility code local to qmp_nbd_*(), I don't think it's too bad. In particular because it's already written.

Instead of adjusting libvirt to changes in the nbd-* commands, I'd rather have it change over to block-export-*. I would like to see the nbd-server-add/remove commands deprecated soon after we have the replacements.

Makes sense to me. So deprecate nbd-server-add in 5.2, and require block-export in 6.1.

Quoted 10 lines
Quoted 8 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 71

Quoted 4 lines
block/export/export.c +4 -0
+    /*
+     * nbd-server-add doesn't complain when a read-only device should be
+     * exported as writable, but simply downgrades it. This is an error with
+     * block-export-add.

I'd be happy with either marking this deprecated now (and fixing it in two releases), or declaring it a bug in nbd-server-add now (and fixing it outright).

How about deprecating nbd-server-add completely?

Works for me. Keeping the warts backwards-compatible in nbd-server-add is more palatable if we know we are going to drop it wholesale down the road.

Quoted 16 lines
Quoted 9 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 90

Quoted 8 lines
block/export/export.c +8 -0
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }
Quoted 3 lines

Wait - is the magic export removal tied only to exporting a drive name, and not a node name? So as long as libvirt is using only node names whwen adding exports, a drive being unplugged won't interfere?

Yes, seems so. It's the existing behaviour, I'm only moving the code around.

Quoted 6 lines
Quoted 2 lines

Overall, the change makes sense to me, although I'd love to see if we could go further on the writable vs. read-only issue.

If nbd-server-add will be going away relatively soon, it's probably not worth the trouble. But if you have reasons to keep it, maybe we should consider it.

No, I'm fine with the idea of getting rid of nbd-server-add, at which point changing it before removal is pointless.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#34 lore raw

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

Quoted 47 lines · Eric Blake
Quoted 7 lines · Kevin Wolf

As long as we can keep the compatibility code local to qmp_nbd_*(), I don't think it's too bad. In particular because it's already written.

Instead of adjusting libvirt to changes in the nbd-* commands, I'd rather have it change over to block-export-*. I would like to see the nbd-server-add/remove commands deprecated soon after we have the replacements.

Makes sense to me. So deprecate nbd-server-add in 5.2, and require block-export in 6.1.

Quoted 10 lines
Quoted 8 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 71

Quoted 4 lines
block/export/export.c +4 -0
+    /*
+     * nbd-server-add doesn't complain when a read-only device should be
+     * exported as writable, but simply downgrades it. This is an error with
+     * block-export-add.

I'd be happy with either marking this deprecated now (and fixing it in two releases), or declaring it a bug in nbd-server-add now (and fixing it outright).

How about deprecating nbd-server-add completely?

Works for me. Keeping the warts backwards-compatible in nbd-server-add is more palatable if we know we are going to drop it wholesale down the road.

Quoted 19 lines
Quoted 9 lines

block/export/export.c static const BlockExportDriver *blk_exp_find_driver(BlockExportType type) · line 90

Quoted 8 lines
block/export/export.c +8 -0
+    /*
+     * nbd-server-add removes the export when the named BlockBackend used for
+     * @device goes away.
+     */
+    on_eject_blk = blk_by_name(arg->device);
+    if (on_eject_blk) {
+        nbd_export_set_on_eject_blk(export, on_eject_blk);
+    }
Quoted 3 lines

Wait - is the magic export removal tied only to exporting a drive name, and not a node name? So as long as libvirt is using only node names whwen adding exports, a drive being unplugged won't interfere?

Yes, seems so. It's the existing behaviour, I'm only moving the code around.

Quoted 2 lines

Overall, the change makes sense to me, although I'd love to see if we could go further on the writable vs. read-only issue.

Quoted 6 lines
Quoted 3 lines

If nbd-server-add will be going away relatively soon, it's probably not worth the trouble. But if you have reasons to keep it, maybe we should consider it.

No, I'm fine with the idea of getting rid of nbd-server-add, at which point changing it before removal is pointless.

I agree that this is a better approach. While it's technically possible to retrofit old commands since QMP schema introspection allows granilar detection of what's happening in this regard using a new command is IMO better. Specifically for APPS which might not use QMP introspection to the extent libvirt does.

author
#35 lore raw

RFC PATCH 8/22 nbd: Add max-connections to nbd-server-start

In reply to Kevin Wolf

This is a QMP equivalent of qemu-nbd's --share option, limiting the maximum number of clients that can attach at the same time.

---
 qapi/block-export.json         | 10 ++++++++--
 include/block/nbd.h            |  3 ++-
 block/monitor/block-hmp-cmds.c |  2 +-
 blockdev-nbd.c                 | 33 ++++++++++++++++++++++++++-------
 qemu-storage-daemon.c          |  4 ++--
 5 files changed, 39 insertions(+), 13 deletions(-)
qapi/block-export.json +8 -2
index 40369814b4..1fdc55c53a 100644
@@ -14,6 +14,8 @@
 #             is only resolved at time of use, so can be deleted and
 #             recreated on the fly while the NBD server is active.
 #             If missing, it will default to denying access (since 4.0).
+# @max-connections: The maximum number of connections to allow at the same
+#                   time, 0 for unlimited. (since 5.2; default: 0)
 #
 # Keep this type consistent with the nbd-server-start arguments. The only
 # intended difference is using SocketAddress instead of SocketAddressLegacy.
@@ -23,7 +25,8 @@
 { 'struct': 'NbdServerOptions',
   'data': { 'addr': 'SocketAddress',
             '*tls-creds': 'str',
-            '*tls-authz': 'str'} }
+            '*tls-authz': 'str',
+            '*max-connections': 'uint32' } }
 
 ##
 # @nbd-server-start:
@@ -40,6 +43,8 @@
 #             is only resolved at time of use, so can be deleted and
 #             recreated on the fly while the NBD server is active.
 #             If missing, it will default to denying access (since 4.0).
+# @max-connections: The maximum number of connections to allow at the same
+#                   time, 0 for unlimited. (since 5.2; default: 0)
 #
 # Returns: error if the server is already running.
 #
@@ -51,7 +56,8 @@
 { 'command': 'nbd-server-start',
   'data': { 'addr': 'SocketAddressLegacy',
             '*tls-creds': 'str',
-            '*tls-authz': 'str'} }
+            '*tls-authz': 'str',
+            '*max-connections': 'uint32' } }
 
 ##
 # @BlockExportOptionsNbd:
include/block/nbd.h +2 -1
index ffca3be78f..6fc1f05ef4 100644
@@ -352,7 +352,8 @@ void nbd_client_get(NBDClient *client);
 void nbd_client_put(NBDClient *client);
 
 void nbd_server_start(SocketAddress *addr, const char *tls_creds,
-                      const char *tls_authz, Error **errp);
+                      const char *tls_authz, uint32_t max_connections,
+                      Error **errp);
 void nbd_server_start_options(NbdServerOptions *arg, Error **errp);
 
 /* nbd_read
block/monitor/block-hmp-cmds.c +1 -1
index 56bc83ac97..a651954e16 100644
@@ -411,7 +411,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
         goto exit;
     }
 
-    nbd_server_start(addr, NULL, NULL, &local_err);
+    nbd_server_start(addr, NULL, NULL, 0, &local_err);
     qapi_free_SocketAddress(addr);
     if (local_err != NULL) {
         goto exit;
blockdev-nbd.c +26 -7
index 019c37c0bc..28159a92b2 100644
@@ -23,23 +23,41 @@ typedef struct NBDServerData {
     QIONetListener *listener;
     QCryptoTLSCreds *tlscreds;
     char *tlsauthz;
+    uint32_t max_connections;
+    uint32_t connections;
 } NBDServerData;
 
 static NBDServerData *nbd_server;
 
+static void nbd_update_server_watch(NBDServerData *s);
+
 static void nbd_blockdev_client_closed(NBDClient *client, bool ignored)
 {
     nbd_client_put(client);
+    assert(nbd_server->connections > 0);
+    nbd_server->connections--;
+    nbd_update_server_watch(nbd_server);
 }
 
 static void nbd_accept(QIONetListener *listener, QIOChannelSocket *cioc,
                        gpointer opaque)
 {
+    nbd_server->connections++;
+    nbd_update_server_watch(nbd_server);
+
     qio_channel_set_name(QIO_CHANNEL(cioc), "nbd-server");
     nbd_client_new(cioc, nbd_server->tlscreds, nbd_server->tlsauthz,
                    nbd_blockdev_client_closed);
 }
 
+static void nbd_update_server_watch(NBDServerData *s)
+{
+    if (!s->max_connections || s->connections < s->max_connections) {
+        qio_net_listener_set_client_func(s->listener, nbd_accept, NULL, NULL);
+    } else {
+        qio_net_listener_set_client_func(s->listener, NULL, NULL, NULL);
+    }
+}
 
 static void nbd_server_free(NBDServerData *server)
 {
@@ -88,7 +106,8 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
 
 
 void nbd_server_start(SocketAddress *addr, const char *tls_creds,
-                      const char *tls_authz, Error **errp)
+                      const char *tls_authz, uint32_t max_connections,
+                      Error **errp)
 {
     if (nbd_server) {
         error_setg(errp, "NBD server already running");
@@ -96,6 +115,7 @@ void nbd_server_start(SocketAddress *addr, const char *tls_creds,
     }
 
     nbd_server = g_new0(NBDServerData, 1);
+    nbd_server->max_connections = max_connections;
     nbd_server->listener = qio_net_listener_new();
 
     qio_net_listener_set_name(nbd_server->listener,
@@ -120,10 +140,7 @@ void nbd_server_start(SocketAddress *addr, const char *tls_creds,
 
     nbd_server->tlsauthz = g_strdup(tls_authz);
 
-    qio_net_listener_set_client_func(nbd_server->listener,
-                                     nbd_accept,
-                                     NULL,
-                                     NULL);
+    nbd_update_server_watch(nbd_server);
 
     return;
 
@@ -134,17 +151,19 @@ void nbd_server_start(SocketAddress *addr, const char *tls_creds,
 
 void nbd_server_start_options(NbdServerOptions *arg, Error **errp)
 {
-    nbd_server_start(arg->addr, arg->tls_creds, arg->tls_authz, errp);
+    nbd_server_start(arg->addr, arg->tls_creds, arg->tls_authz,
+                     arg->max_connections, errp);
 }
 
 void qmp_nbd_server_start(SocketAddressLegacy *addr,
                           bool has_tls_creds, const char *tls_creds,
                           bool has_tls_authz, const char *tls_authz,
+                          bool has_max_connections, uint32_t max_connections,
                           Error **errp)
 {
     SocketAddress *addr_flat = socket_address_flatten(addr);
 
-    nbd_server_start(addr_flat, tls_creds, tls_authz, errp);
+    nbd_server_start(addr_flat, tls_creds, tls_authz, max_connections, errp);
     qapi_free_SocketAddress(addr_flat);
 }
 
qemu-storage-daemon.c +2 -2
index b6f678d3ab..0fcab6ed2d 100644
@@ -101,9 +101,9 @@ static void help(void)
 "                         configure a QMP monitor\n"
 "\n"
 "  --nbd-server addr.type=inet,addr.host=<host>,addr.port=<port>\n"
-"               [,tls-creds=<id>][,tls-authz=<id>]\n"
+"               [,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>]\n"
 "  --nbd-server addr.type=unix,addr.path=<path>\n"
-"               [,tls-creds=<id>][,tls-authz=<id>]\n"
+"               [,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>]\n"
 "                         start an NBD server for exporting block nodes\n"
 "\n"
 "  --object help          list object types that can be added\n"
Signature · 1 line
2.25.4
Reviewed-by on v1 8/22
#36 lore raw

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

Quoted 1 line · Kevin Wolf

This is a QMP equivalent of qemu-nbd's --share option, limiting the

*--shared

Quoted 10 lines

maximum number of clients that can attach at the same time.

---
 qapi/block-export.json         | 10 ++++++++--
 include/block/nbd.h            |  3 ++-
 block/monitor/block-hmp-cmds.c |  2 +-
 blockdev-nbd.c                 | 33 ++++++++++++++++++++++++++-------
 qemu-storage-daemon.c          |  4 ++--
 5 files changed, 39 insertions(+), 13 deletions(-)

I suppose this is part of this series so that patch 11 can happen?

author
#37 lore raw

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

Am 17.08.2020 um 14:37 hat Max Reitz geschrieben:

Quoted 16 lines · Kevin Wolf
Quoted 1 line

This is a QMP equivalent of qemu-nbd's --share option, limiting the

*--shared

Quoted 10 lines

maximum number of clients that can attach at the same time.

---
 qapi/block-export.json         | 10 ++++++++--
 include/block/nbd.h            |  3 ++-
 block/monitor/block-hmp-cmds.c |  2 +-
 blockdev-nbd.c                 | 33 ++++++++++++++++++++++++++-------
 qemu-storage-daemon.c          |  4 ++--
 5 files changed, 39 insertions(+), 13 deletions(-)
Quoted 1 line

I suppose this is part of this series so that patch 11 can happen?

More like because initially I thought it would be needed for patch 11, and when I realised that server != export and it's not really needed, I still didn't want to throw the patch away...

I could make it a patch separate from this series if that's helpful.

Kevin

Reviewed-by on v1 8/22
#38 lore raw

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

In reply to Kevin Wolf
Quoted 17 lines · Kevin Wolf

This is a QMP equivalent of qemu-nbd's --share option, limiting the maximum number of clients that can attach at the same time.

---
  qapi/block-export.json         | 10 ++++++++--
  include/block/nbd.h            |  3 ++-
  block/monitor/block-hmp-cmds.c |  2 +-
  blockdev-nbd.c                 | 33 ++++++++++++++++++++++++++-------
  qemu-storage-daemon.c          |  4 ++--
  5 files changed, 39 insertions(+), 13 deletions(-)
qapi/block-export.json +0 -0
index 40369814b4..1fdc55c53a 100644

qapi/block-export.json line 18

Quoted 5 lines
qapi/block-export.json +2 -0
@@ -14,6 +14,8 @@
  #             is only resolved at time of use, so can be deleted and
  #             recreated on the fly while the NBD server is active.
  #             If missing, it will default to denying access (since 4.0).
+# @max-connections: The maximum number of connections to allow at the same
+#                   time, 0 for unlimited. (since 5.2; default: 0)

Nice way to add feature parity.

Limiting the number of connections (particularly for a writable client, where we cannot guarantee cache consistency between the connections), seems like a worthwhile feature to have; I've always found it odd that qemu-nbd and QMP nbd-server-add defaulted to different limits (1 vs. unlimited). For reference, nbdkit defaults to unlimited, and I'm happy if qemu-storage-daemon does likewise; but changing qemu-nbd's default of 1 would be backwards incompatible and may cause surprises (there's always 'qemu-nbd -e' when needed). I also wonder if we should change 'qemu-nbd -e 0' to mean unlimited rather than an error (right now, qemu-iotests/common.rc uses -e 42 for all nbd-based tests for a saner limit than just 1, but it smells of being arbitrary compared to unlimited).

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#39 lore raw

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

Am 19.08.2020 um 22:00 hat Eric Blake geschrieben:

Quoted 26 lines · Kevin Wolf
Quoted 17 lines

This is a QMP equivalent of qemu-nbd's --share option, limiting the maximum number of clients that can attach at the same time.

---
  qapi/block-export.json         | 10 ++++++++--
  include/block/nbd.h            |  3 ++-
  block/monitor/block-hmp-cmds.c |  2 +-
  blockdev-nbd.c                 | 33 ++++++++++++++++++++++++++-------
  qemu-storage-daemon.c          |  4 ++--
  5 files changed, 39 insertions(+), 13 deletions(-)
qapi/block-export.json +0 -0
index 40369814b4..1fdc55c53a 100644

qapi/block-export.json line 18

Quoted 5 lines
qapi/block-export.json +2 -0
@@ -14,6 +14,8 @@
  #             is only resolved at time of use, so can be deleted and
  #             recreated on the fly while the NBD server is active.
  #             If missing, it will default to denying access (since 4.0).
+# @max-connections: The maximum number of connections to allow at the same
+#                   time, 0 for unlimited. (since 5.2; default: 0)

Nice way to add feature parity.

Quoted 11 lines

Limiting the number of connections (particularly for a writable client, where we cannot guarantee cache consistency between the connections), seems like a worthwhile feature to have; I've always found it odd that qemu-nbd and QMP nbd-server-add defaulted to different limits (1 vs. unlimited). For reference, nbdkit defaults to unlimited, and I'm happy if qemu-storage-daemon does likewise; but changing qemu-nbd's default of 1 would be backwards incompatible and may cause surprises (there's always 'qemu-nbd -e' when needed). I also wonder if we should change 'qemu-nbd -e 0' to mean unlimited rather than an error (right now, qemu-iotests/common.rc uses -e 42 for all nbd-based tests for a saner limit than just 1, but it smells of being arbitrary compared to unlimited).

I think eventually the actual NBD server implementation in qemu-nbd should go away and it should just reuse the QMP one. Changing the default would remove one more difference between them, which can only be helpful in this process. (Though of course having a different default is still simple enough for a simple wrapper.)

Kevin

author
#40 lore raw

RFC PATCH 9/22 nbd: Add writethrough to block-export-add

In reply to Kevin Wolf

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
 qapi/block-export.json | 7 ++++++-
 blockdev-nbd.c         | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
qapi/block-export.json +6 -1
index 1fdc55c53a..4ce163411f 100644
@@ -167,10 +167,15 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)
+#
 # Since: 4.2
 ##
 { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
       'nbd': 'BlockExportOptionsNbd'
blockdev-nbd.c +1 -1
index 28159a92b2..17417c1b6b 100644
@@ -218,7 +218,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
                          !arg->writable, !arg->writable,
-                         NULL, false, errp);
+                         NULL, exp_args->writethrough, errp);
     if (!exp) {
         goto out;
     }
Signature · 1 line
2.25.4
#41 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

Quoted 21 lines · Kevin Wolf

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
 qapi/block-export.json | 7 ++++++-
 blockdev-nbd.c         | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
qapi/block-export.json +3 -0
index 1fdc55c53a..4ce163411f 100644
@@ -167,10 +167,15 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +3 -1
+#
 # Since: 4.2
 ##
 { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
       'nbd': 'BlockExportOptionsNbd'

Hm. I find it weird to have @writethrough in the base but @device in the specialized class.

I think everything that will be common to all block exports should be in the base, and that probably includes a node-name. I’m aware that will make things more tedious in the code, but perhaps it would be a nicer interface in the end. Or is the real problem that that would create problems in the storage daemon’s command line interface, because then the specialized (legacy) NBD interface would no longer be compatible with the new generalized block export interface?

Anyway, @writable might be a similar story. A @read-only may make sense in general, I think.

Basically, I think that the export code should be separate from the code setting up the BlockBackend that should be exported, so all options regarding that BB should be common; and those options are @node-name, @writethrough, and @read-only. (And perhaps other things like @resizable, too, even though that isn’t something to consider for NBD.)

Max

author
#42 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

Am 17.08.2020 um 14:56 hat Max Reitz geschrieben:

Quoted 36 lines · Kevin Wolf
Quoted 21 lines

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
 qapi/block-export.json | 7 ++++++-
 blockdev-nbd.c         | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
qapi/block-export.json +3 -0
index 1fdc55c53a..4ce163411f 100644
@@ -167,10 +167,15 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +3 -1
+#
 # Since: 4.2
 ##
 { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
       'nbd': 'BlockExportOptionsNbd'

Hm. I find it weird to have @writethrough in the base but @device in the specialized class.

Quoted 7 lines

I think everything that will be common to all block exports should be in the base, and that probably includes a node-name. I’m aware that will make things more tedious in the code, but perhaps it would be a nicer interface in the end. Or is the real problem that that would create problems in the storage daemon’s command line interface, because then the specialized (legacy) NBD interface would no longer be compatible with the new generalized block export interface?

Indeed. I think patch 15 has what you're looking for.

Quoted 2 lines

Anyway, @writable might be a similar story. A @read-only may make sense in general, I think.

Pulling @writable up is easier than a @read-only, but that's a naming detail.

In general I agree, but this part isn't addressed in this series yet. Part of the reason why this is an RFC was to find out if I should include things like this or if we'll do it when we add another export type or common functionality that needs the same option.

Quoted 5 lines

Basically, I think that the export code should be separate from the code setting up the BlockBackend that should be exported, so all options regarding that BB should be common; and those options are @node-name, @writethrough, and @read-only. (And perhaps other things like @resizable, too, even though that isn’t something to consider for NBD.)

Do you mean that the BlockBackend should already be created by the generic block export layer?

Kevin

#43 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

Quoted 45 lines · Kevin Wolf

Am 17.08.2020 um 14:56 hat Max Reitz geschrieben:

Quoted 36 lines · Kevin Wolf
Quoted 21 lines

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
 qapi/block-export.json | 7 ++++++-
 blockdev-nbd.c         | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
qapi/block-export.json +3 -0
index 1fdc55c53a..4ce163411f 100644
@@ -167,10 +167,15 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +3 -1
+#
 # Since: 4.2
 ##
 { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
       'nbd': 'BlockExportOptionsNbd'

Hm. I find it weird to have @writethrough in the base but @device in the specialized class.

Quoted 7 lines

I think everything that will be common to all block exports should be in the base, and that probably includes a node-name. I’m aware that will make things more tedious in the code, but perhaps it would be a nicer interface in the end. Or is the real problem that that would create problems in the storage daemon’s command line interface, because then the specialized (legacy) NBD interface would no longer be compatible with the new generalized block export interface?

Quoted 1 line

Indeed. I think patch 15 has what you're looking for.

Great. :)

Discussions where both participants have the same opinion from the start are the best ones.

Quoted 5 lines
Quoted 2 lines

Anyway, @writable might be a similar story. A @read-only may make sense in general, I think.

Pulling @writable up is easier than a @read-only, but that's a naming detail.

Sure.

Quoted 4 lines

In general I agree, but this part isn't addressed in this series yet. Part of the reason why this is an RFC was to find out if I should include things like this or if we'll do it when we add another export type or common functionality that needs the same option.

Sure, sure.

Meta: I personally don’t like RFC patches very much. RFC to me means everything is fair game, and reviewers should be free to let their thoughts wander and come up with perhaps wild ideas, just trying to gauge what everyone thinks.

When I’m the submitter, I tend to get defensive then, because I’ve invested time in writing the code already, so I tend to be biased against fundamental changes. (Horrible personal trait. I’m working on it.)

As a reviewer, the code and thus some fleshed out design is there already, so it’s difficult to break free from that and find completely different solutions to the original problem.
(I kind of ventured in that direction for this patch, and it seems like you immediately noticed that my response was different from usual and pointed out the RFC status, perhaps to make me feel more comfortable in questioning the fundamental design more. Which I noticed, hence this wall of text.)

Perhaps I’m wrong. Perhaps it’s just myself (the points I’ve just listed are definitely my own personal weaknesses), but I can’t help but project and assume that others may feel similar, at least in part. So I feel like RFCs that consist of patches tend to at least lock me in to the solution that’s present. I find them difficult to handle, both as a submitter and as a reviewer.

All in all, that means on either side I tend to handle patch RFCs as “Standard series, just tests missing”. Not sure if that’s ideal. Or maybe that’s exactly what patch RFCs are?

(Of course, it can and should be argued that even for standard series, I shouldn’t be afraid of questioning the fundamental design still. But that’s hard...)

But, well. The alternative is writing pure design RFCs, and then you tend to get weeks of slow discussion, drawing everything out. Which isn’t ideal either. Or is that just a baseless prejudice I have?

Quoted 8 lines
Quoted 5 lines

Basically, I think that the export code should be separate from the code setting up the BlockBackend that should be exported, so all options regarding that BB should be common; and those options are @node-name, @writethrough, and @read-only. (And perhaps other things like @resizable, too, even though that isn’t something to consider for NBD.)

Do you mean that the BlockBackend should already be created by the generic block export layer?

It would certainly be nice, if it were feasible, don’t you think?

We don’t have to bend backwards for it, but maybe it would force us to bring the natural separation of block device and export parameters to the interface.

Max

author
#44 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

Am 17.08.2020 um 15:51 hat Max Reitz geschrieben:

Quoted 46 lines · Kevin Wolf
Quoted 45 lines

Am 17.08.2020 um 14:56 hat Max Reitz geschrieben:

Quoted 43 lines · Kevin Wolf
Quoted 21 lines

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
 qapi/block-export.json | 7 ++++++-
 blockdev-nbd.c         | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
qapi/block-export.json +3 -0
index 1fdc55c53a..4ce163411f 100644
@@ -167,10 +167,15 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +3 -1
+#
 # Since: 4.2
 ##
 { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
       'nbd': 'BlockExportOptionsNbd'

Hm. I find it weird to have @writethrough in the base but @device in the specialized class.

I think everything that will be common to all block exports should be in the base, and that probably includes a node-name. I’m aware that will make things more tedious in the code, but perhaps it would be a nicer interface in the end. Or is the real problem that that would create problems in the storage daemon’s command line interface, because then the specialized (legacy) NBD interface would no longer be compatible with the new generalized block export interface?

Quoted 6 lines
Quoted 1 line

Indeed. I think patch 15 has what you're looking for.

Great. :)

Discussions where both participants have the same opinion from the start are the best ones.

Makes things a lot easier.

Maybe I should try to move patch 15 earlier. The series is mostly just in the order that I wrote things, but there were also a few nasty dependencies in the part the generalises things from NBD to BlockExport. So I'm not sure if this is a patch that can be moved.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Anyway, @writable might be a similar story. A @read-only may make sense in general, I think.

Pulling @writable up is easier than a @read-only, but that's a naming detail.

Quoted 8 lines

Sure.

Quoted 4 lines

In general I agree, but this part isn't addressed in this series yet. Part of the reason why this is an RFC was to find out if I should include things like this or if we'll do it when we add another export type or common functionality that needs the same option.

Sure, sure.

So should I or not? :-)

Quoted 9 lines

Meta: I personally don’t like RFC patches very much. RFC to me means everything is fair game, and reviewers should be free to let their thoughts wander and come up with perhaps wild ideas, just trying to gauge what everyone thinks.

When I’m the submitter, I tend to get defensive then, because I’ve invested time in writing the code already, so I tend to be biased against fundamental changes. (Horrible personal trait. I’m working on it.)

This makes sense. Nobody likes having to rewrite their RFC series.

But there is one thing I dread even more: Polishing the RFC series for another week until I can send it out as non-RFC and _then_ having to rewrite it.

Quoted 8 lines

As a reviewer, the code and thus some fleshed out design is there already, so it’s difficult to break free from that and find completely different solutions to the original problem.
(I kind of ventured in that direction for this patch, and it seems like you immediately noticed that my response was different from usual and pointed out the RFC status, perhaps to make me feel more comfortable in questioning the fundamental design more. Which I noticed, hence this wall of text.)

Basically just telling you that I was already interested in your input for this point specifically when I sent the series.

Quoted 7 lines

Perhaps I’m wrong. Perhaps it’s just myself (the points I’ve just listed are definitely my own personal weaknesses), but I can’t help but project and assume that others may feel similar, at least in part. So I feel like RFCs that consist of patches tend to at least lock me in to the solution that’s present. I find them difficult to handle, both as a submitter and as a reviewer.

Quoted 7 lines

All in all, that means on either side I tend to handle patch RFCs as “Standard series, just tests missing”. Not sure if that’s ideal. Or maybe that’s exactly what patch RFCs are?

(Of course, it can and should be argued that even for standard series, I shouldn’t be afraid of questioning the fundamental design still. But that’s hard...)

I usually send RFC patches when I know that I wouldn't consider them mergable yet, but I don't want to invest the time to polish them before I know that other people agree with the approach and the time won't be wasted.

Quoted 3 lines

But, well. The alternative is writing pure design RFCs, and then you tend to get weeks of slow discussion, drawing everything out. Which isn’t ideal either. Or is that just a baseless prejudice I have?

In many cases (and I think this is one of them in large parts), I only really learn what the series will look like when I write it.

I could have sent a design RFC for the QAPI part, but I didn't expect this to be contentious because it's just the normal add/del/query thing that exists for pretty much everything else, too.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Basically, I think that the export code should be separate from the code setting up the BlockBackend that should be exported, so all options regarding that BB should be common; and those options are @node-name, @writethrough, and @read-only. (And perhaps other things like @resizable, too, even though that isn’t something to consider for NBD.)

Quoted 8 lines
Quoted 2 lines

Do you mean that the BlockBackend should already be created by the generic block export layer?

It would certainly be nice, if it were feasible, don’t you think?

We don’t have to bend backwards for it, but maybe it would force us to bring the natural separation of block device and export parameters to the interface.

I can try. I seem to remember that you had a reason not to do this the last time we discussed generalised exports, but I'm not sure what it was.

The obvious one could be that the block export layer doesn't know which permissions are needed. But it can always start with minimal permissions and let the driver do a blk_set_perm() if it needs more.

Kevin

#45 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

Quoted 69 lines · Kevin Wolf

Am 17.08.2020 um 15:51 hat Max Reitz geschrieben:

Quoted 46 lines · Kevin Wolf
Quoted 45 lines

Am 17.08.2020 um 14:56 hat Max Reitz geschrieben:

Quoted 43 lines · Kevin Wolf
Quoted 21 lines

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
 qapi/block-export.json | 7 ++++++-
 blockdev-nbd.c         | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
qapi/block-export.json +3 -0
index 1fdc55c53a..4ce163411f 100644
@@ -167,10 +167,15 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +3 -1
+#
 # Since: 4.2
 ##
 { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
       'nbd': 'BlockExportOptionsNbd'

Hm. I find it weird to have @writethrough in the base but @device in the specialized class.

I think everything that will be common to all block exports should be in the base, and that probably includes a node-name. I’m aware that will make things more tedious in the code, but perhaps it would be a nicer interface in the end. Or is the real problem that that would create problems in the storage daemon’s command line interface, because then the specialized (legacy) NBD interface would no longer be compatible with the new generalized block export interface?

Quoted 6 lines
Quoted 1 line

Indeed. I think patch 15 has what you're looking for.

Great. :)

Discussions where both participants have the same opinion from the start are the best ones.

Makes things a lot easier.

Maybe I should try to move patch 15 earlier. The series is mostly just in the order that I wrote things, but there were also a few nasty dependencies in the part the generalises things from NBD to BlockExport. So I'm not sure if this is a patch that can be moved.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Anyway, @writable might be a similar story. A @read-only may make sense in general, I think.

Pulling @writable up is easier than a @read-only, but that's a naming detail.

Sure.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

In general I agree, but this part isn't addressed in this series yet. Part of the reason why this is an RFC was to find out if I should include things like this or if we'll do it when we add another export type or common functionality that needs the same option.

Sure, sure.

So should I or not? :-)

Can we delay it until after this series? I.e., as long as it retains the name “writable”, would pulling it up into BlockExportOptions a compatible change?

If so, then I suppose we could do it afterwards. But I think it does make the most sense to “just” do it as part of this series.

Quoted 10 lines
Quoted 9 lines

Meta: I personally don’t like RFC patches very much. RFC to me means everything is fair game, and reviewers should be free to let their thoughts wander and come up with perhaps wild ideas, just trying to gauge what everyone thinks.

When I’m the submitter, I tend to get defensive then, because I’ve invested time in writing the code already, so I tend to be biased against fundamental changes. (Horrible personal trait. I’m working on it.)

Quoted 5 lines

This makes sense. Nobody likes having to rewrite their RFC series.

But there is one thing I dread even more: Polishing the RFC series for another week until I can send it out as non-RFC and _then_ having to rewrite it.

Yes. Especially bad with tests.

Quoted 11 lines
Quoted 8 lines

As a reviewer, the code and thus some fleshed out design is there already, so it’s difficult to break free from that and find completely different solutions to the original problem.
(I kind of ventured in that direction for this patch, and it seems like you immediately noticed that my response was different from usual and pointed out the RFC status, perhaps to make me feel more comfortable in questioning the fundamental design more. Which I noticed, hence this wall of text.)

Basically just telling you that I was already interested in your input for this point specifically when I sent the series.

OK :)

Quoted 20 lines
Quoted 7 lines

Perhaps I’m wrong. Perhaps it’s just myself (the points I’ve just listed are definitely my own personal weaknesses), but I can’t help but project and assume that others may feel similar, at least in part. So I feel like RFCs that consist of patches tend to at least lock me in to the solution that’s present. I find them difficult to handle, both as a submitter and as a reviewer.

Quoted 7 lines

All in all, that means on either side I tend to handle patch RFCs as “Standard series, just tests missing”. Not sure if that’s ideal. Or maybe that’s exactly what patch RFCs are?

(Of course, it can and should be argued that even for standard series, I shouldn’t be afraid of questioning the fundamental design still. But that’s hard...)

I usually send RFC patches when I know that I wouldn't consider them mergable yet, but I don't want to invest the time to polish them before I know that other people agree with the approach and the time won't be wasted.

Quoted 6 lines
Quoted 3 lines

But, well. The alternative is writing pure design RFCs, and then you tend to get weeks of slow discussion, drawing everything out. Which isn’t ideal either. Or is that just a baseless prejudice I have?

In many cases (and I think this is one of them in large parts), I only really learn what the series will look like when I write it.

That’s true. With a pure design RFC, it’s often difficult to know even the scope of the design until you’ve begun to write code. So there’s a danger of just writing a bunch of uncontroversial basic design stuff because one has no idea of what may actually become problematic and questionable. :/

Quoted 3 lines

I could have sent a design RFC for the QAPI part, but I didn't expect this to be contentious because it's just the normal add/del/query thing that exists for pretty much everything else, too.

Yeah, the functions themselves are clear.

Hm. Perhaps software engineering just is actually difficult, and there’s no way around it.

Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Basically, I think that the export code should be separate from the code setting up the BlockBackend that should be exported, so all options regarding that BB should be common; and those options are @node-name, @writethrough, and @read-only. (And perhaps other things like @resizable, too, even though that isn’t something to consider for NBD.)

Quoted 8 lines
Quoted 2 lines

Do you mean that the BlockBackend should already be created by the generic block export layer?

It would certainly be nice, if it were feasible, don’t you think?

We don’t have to bend backwards for it, but maybe it would force us to bring the natural separation of block device and export parameters to the interface.

Quoted 7 lines

I can try. I seem to remember that you had a reason not to do this the last time we discussed generalised exports, but I'm not sure what it was.

The obvious one could be that the block export layer doesn't know which permissions are needed. But it can always start with minimal permissions and let the driver do a blk_set_perm() if it needs more.

Trying sounds good. Since there shouldn’t be consequences for the QMP interface, we™ can always try again later (i.e., when adding more export types).

Max

#46 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

In reply to Max Reitz
Quoted 38 lines · Max Reitz
Quoted 15 lines · Kevin Wolf

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---

+++ b/qapi/block-export.json

qapi/block-export.json +3 -0
@@ -167,10 +167,15 @@
  # Describes a block export, i.e. how single node should be exported on an
  # external interface.
  #
+# @writethrough: If true, caches are flushed after every write request to the
+#                export before completion is signalled. (since: 5.2;
+#                default: false)

qapi/block-export.json line 181

Quoted 10 lines
qapi/block-export.json +3 -1
+#
  # Since: 4.2
  ##
  { 'union': 'BlockExportOptions',
-  'base': { 'type': 'BlockExportType' },
+  'base': { 'type': 'BlockExportType',
+            '*writethrough': 'bool' },
    'discriminator': 'type',
    'data': {
        'nbd': 'BlockExportOptionsNbd'

Hm. I find it weird to have @writethrough in the base but @device in the specialized class.

I think everything that will be common to all block exports should be in the base, and that probably includes a node-name. I’m aware that will make things more tedious in the code, but perhaps it would be a nicer interface in the end. Or is the real problem that that would create problems in the storage daemon’s command line interface, because then the specialized (legacy) NBD interface would no longer be compatible with the new generalized block export interface?

Quoted 2 lines

Anyway, @writable might be a similar story. A @read-only may make sense in general, I think.

And maybe even auto-read-only. As for @writable vs. @read-only, that's a choice of spelling, but we don't want both; there's also the question of which default is saner (having to opt-in to allowing writes is more verbose than defaulting to allowing writes when possible, but arguably saner as it is less risk of unintended writes when you forgot to specify the option; @auto-read-only can help in choosing nicer defaults).

Quoted 6 lines

Basically, I think that the export code should be separate from the code setting up the BlockBackend that should be exported, so all options regarding that BB should be common; and those options are @node-name, @writethrough, and @read-only. (And perhaps other things like @resizable, too, even though that isn’t something to consider for NBD.)

NBD isn't resizable yet, but extending the protocol to let it become so is on my TODO list.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#47 lore raw

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

In reply to Kevin Wolf
Quoted 13 lines · Kevin Wolf

qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add.

---
  qapi/block-export.json | 7 ++++++-
  blockdev-nbd.c         | 2 +-
  2 files changed, 7 insertions(+), 2 deletions(-)

+++ b/blockdev-nbd.c

blockdev-nbd.c BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 221

Quoted 5 lines
blockdev-nbd.c +1 -1
@@ -218,7 +218,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
      exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
                           !arg->writable, !arg->writable,
-                         NULL, false, errp);
+                         NULL, exp_args->writethrough, errp);

It would someday be nice to get rid of exp_args->has_writethrough in QAPI generated code, but that's independent of this series. Meanwhile, you are safe in relying on writethrough being false when has_writethrough is false.

This change makes sense to me interface-wise. QAPI-wise, should we try harder to make writethrough settable only for writable exports (and an error for read-only exports)? I'm not sure what QAPI contortions would have to look like to make that enforced by the QMP parser; but it's also not the end of the world if we just always permit the optional field and then apply a post-parse semantic check.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#48 lore raw

RFC PATCH 10/22 nbd: Remove NBDExport.close callback

In reply to Kevin Wolf

The export close callback is unused by the built-in NBD server. qemu-nbd uses it only during shutdown to wait for the unrefed export to actually go away. It can just use nbd_export_close_all() instead and do without the callback.

This removes the close callback from nbd_export_new() and makes both callers of it more similar.

---
 include/block/nbd.h |  3 +--
 blockdev-nbd.c      |  2 +-
 nbd/server.c        |  9 +--------
 qemu-nbd.c          | 14 ++++----------
 4 files changed, 7 insertions(+), 21 deletions(-)
include/block/nbd.h +1 -2
index 6fc1f05ef4..50e1a46075 100644
@@ -332,8 +332,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp);
 NBDExport *nbd_export_new(BlockDriverState *bs,
                           const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
-                          void (*close)(NBDExport *), bool writethrough,
-                          Error **errp);
+                          bool writethrough, Error **errp);
 void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
 void nbd_export_close(NBDExport *exp);
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
blockdev-nbd.c +1 -1
index 17417c1b6b..d5b084acc2 100644
@@ -218,7 +218,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
                          !arg->writable, !arg->writable,
-                         NULL, exp_args->writethrough, errp);
+                         exp_args->writethrough, errp);
     if (!exp) {
         goto out;
     }
nbd/server.c +1 -8
index 0b84fd30e2..eadc5b9804 100644
@@ -84,7 +84,6 @@ struct NBDRequestData {
 struct NBDExport {
     BlockExport common;
     int refcount;
-    void (*close)(NBDExport *exp);
 
     BlockBackend *blk;
     char *name;
@@ -1520,8 +1519,7 @@ void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk)
 NBDExport *nbd_export_new(BlockDriverState *bs,
                           const char *name, const char *desc,
                           const char *bitmap, bool readonly, bool shared,
-                          void (*close)(NBDExport *), bool writethrough,
-                          Error **errp)
+                          bool writethrough, Error **errp)
 {
     AioContext *ctx;
     BlockBackend *blk;
@@ -1625,7 +1623,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
         assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
     }
 
-    exp->close = close;
     exp->ctx = ctx;
     blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
 
@@ -1723,10 +1720,6 @@ void nbd_export_put(NBDExport *exp)
         assert(exp->name == NULL);
         assert(exp->description == NULL);
 
-        if (exp->close) {
-            exp->close(exp);
-        }
-
         if (exp->blk) {
             if (exp->eject_notifier_blk) {
                 notifier_remove(&exp->eject_notifier);
qemu-nbd.c +4 -10
index e348d5d6d8..48aa8a9d46 100644
@@ -70,7 +70,7 @@ static int verbose;
 static char *srcpath;
 static SocketAddress *saddr;
 static int persistent = 0;
-static enum { RUNNING, TERMINATE, TERMINATING, TERMINATED } state;
+static enum { RUNNING, TERMINATE, TERMINATED } state;
 static int shared = 1;
 static int nb_fds;
 static QIONetListener *server;
@@ -331,12 +331,6 @@ static int nbd_can_accept(void)
     return state == RUNNING && nb_fds < shared;
 }
 
-static void nbd_export_closed(NBDExport *export)
-{
-    assert(state == TERMINATING);
-    state = TERMINATED;
-}
-
 static void nbd_update_server_watch(void);
 
 static void nbd_client_closed(NBDClient *client, bool negotiated)
@@ -1058,7 +1052,7 @@ int main(int argc, char **argv)
 
     export = nbd_export_new(bs, export_name,
                             export_description, bitmap, readonly, shared > 1,
-                            nbd_export_closed, writethrough, &error_fatal);
+                            writethrough, &error_fatal);
 
     if (device) {
 #if HAVE_NBD_DEVICE
@@ -1098,10 +1092,10 @@ int main(int argc, char **argv)
     do {
         main_loop_wait(false);
         if (state == TERMINATE) {
-            state = TERMINATING;
-            nbd_export_close(export);
             nbd_export_put(export);
+            nbd_export_close_all();
             export = NULL;
+            state = TERMINATED;
         }
     } while (state != TERMINATED);
 
Signature · 1 line
2.25.4
Reviewed-by on v1 10/22
#49 lore raw

Re: [RFC PATCH 10/22] nbd: Remove NBDExport.close callback

Quoted 8 lines · Kevin Wolf

The export close callback is unused by the built-in NBD server. qemu-nbd uses it only during shutdown to wait for the unrefed export to actually go away. It can just use nbd_export_close_all() instead and do without the callback.

This removes the close callback from nbd_export_new() and makes both callers of it more similar.

Quoted 7 lines
---
 include/block/nbd.h |  3 +--
 blockdev-nbd.c      |  2 +-
 nbd/server.c        |  9 +--------
 qemu-nbd.c          | 14 ++++----------
 4 files changed, 7 insertions(+), 21 deletions(-)
Reviewed-by on v1 10/22
#50 lore raw

Re: [RFC PATCH 10/22] nbd: Remove NBDExport.close callback

In reply to Kevin Wolf
Quoted 10 lines · Kevin Wolf

The export close callback is unused by the built-in NBD server. qemu-nbd uses it only during shutdown to wait for the unrefed export to actually go away. It can just use nbd_export_close_all() instead and do without the callback.

This removes the close callback from nbd_export_new() and makes both callers of it more similar.

---

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#51 lore raw

RFC PATCH 11/22 qemu-nbd: Use blk_exp_add() to create the export

In reply to Kevin Wolf

With this change, NBD exports are only created through the BlockExport interface any more. This allows us finally to move things from the NBD layer to the BlockExport layer if they make sense for other export types, too.

blk_exp_add() returns only a weak reference, so the explicit nbd_export_put() goes away.

---
 include/block/export.h |  2 ++
 include/block/nbd.h    |  1 +
 block/export/export.c  |  2 +-
 blockdev-nbd.c         |  8 +++++++-
 qemu-nbd.c             | 28 ++++++++++++++++++++++------
 5 files changed, 33 insertions(+), 8 deletions(-)
include/block/export.h +2 -0
index b1d7325403..5424bdc85d 100644
@@ -29,4 +29,6 @@ struct BlockExport {
 
 extern const BlockExportDriver blk_exp_nbd;
 
+BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp);
+
 #endif
include/block/nbd.h +1 -0
index 50e1a46075..23030db3f1 100644
@@ -350,6 +350,7 @@ void nbd_client_new(QIOChannelSocket *sioc,
 void nbd_client_get(NBDClient *client);
 void nbd_client_put(NBDClient *client);
 
+void nbd_server_is_qemu_nbd(bool value);
 void nbd_server_start(SocketAddress *addr, const char *tls_creds,
                       const char *tls_authz, uint32_t max_connections,
                       Error **errp);
block/export/export.c +1 -1
index 2d5f92861c..12672228c7 100644
@@ -36,7 +36,7 @@ static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
     return NULL;
 }
 
-static BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
+BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
 {
     const BlockExportDriver *drv;
 
blockdev-nbd.c +7 -1
index d5b084acc2..8dd127af52 100644
@@ -28,9 +28,15 @@ typedef struct NBDServerData {
 } NBDServerData;
 
 static NBDServerData *nbd_server;
+static bool is_qemu_nbd;
 
 static void nbd_update_server_watch(NBDServerData *s);
 
+void nbd_server_is_qemu_nbd(bool value)
+{
+    is_qemu_nbd = value;
+}
+
 static void nbd_blockdev_client_closed(NBDClient *client, bool ignored)
 {
     nbd_client_put(client);
@@ -176,7 +182,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
 
-    if (!nbd_server) {
+    if (!nbd_server && !is_qemu_nbd) {
         error_setg(errp, "NBD server not running");
         return NULL;
     }
qemu-nbd.c +22 -6
index 48aa8a9d46..d967b8fcb9 100644
@@ -65,7 +65,6 @@
 
 #define MBR_SIZE 512
 
-static NBDExport *export;
 static int verbose;
 static char *srcpath;
 static SocketAddress *saddr;
@@ -579,6 +578,7 @@ int main(int argc, char **argv)
     int old_stderr = -1;
     unsigned socket_activation;
     const char *pid_file_name = NULL;
+    BlockExportOptions *export_opts;
 
     /* The client thread uses SIGTERM to interrupt the server.  A signal
      * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
@@ -1050,9 +1050,27 @@ int main(int argc, char **argv)
 
     bs->detect_zeroes = detect_zeroes;
 
-    export = nbd_export_new(bs, export_name,
-                            export_description, bitmap, readonly, shared > 1,
-                            writethrough, &error_fatal);
+    nbd_server_is_qemu_nbd(true);
+
+    export_opts = g_new(BlockExportOptions, 1);
+    *export_opts = (BlockExportOptions) {
+        .type               = BLOCK_EXPORT_TYPE_NBD,
+        .has_writethrough   = true,
+        .writethrough       = writethrough,
+        .u.nbd = {
+            .device             = g_strdup(bdrv_get_node_name(bs)),
+            .has_name           = true,
+            .name               = g_strdup(export_name),
+            .has_description    = !!export_description,
+            .description        = g_strdup(export_description),
+            .has_writable       = true,
+            .writable           = !readonly,
+            .has_bitmap         = !!bitmap,
+            .bitmap             = g_strdup(bitmap),
+        },
+    };
+    blk_exp_add(export_opts, &error_fatal);
+    qapi_free_BlockExportOptions(export_opts);
 
     if (device) {
 #if HAVE_NBD_DEVICE
@@ -1092,9 +1110,7 @@ int main(int argc, char **argv)
     do {
         main_loop_wait(false);
         if (state == TERMINATE) {
-            nbd_export_put(export);
             nbd_export_close_all();
-            export = NULL;
             state = TERMINATED;
         }
     } while (state != TERMINATED);
Signature · 1 line
2.25.4
#52 lore raw

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

Quoted 4 lines · Kevin Wolf

With this change, NBD exports are only created through the BlockExport interface any more. This allows us finally to move things from the NBD layer to the BlockExport layer if they make sense for other export types, too.

I see.

Quoted 11 lines

blk_exp_add() returns only a weak reference, so the explicit nbd_export_put() goes away.

---
 include/block/export.h |  2 ++
 include/block/nbd.h    |  1 +
 block/export/export.c  |  2 +-
 blockdev-nbd.c         |  8 +++++++-
 qemu-nbd.c             | 28 ++++++++++++++++++++++------
 5 files changed, 33 insertions(+), 8 deletions(-)

[...]

Quoted 4 lines
blockdev-nbd.c +0 -0
index d5b084acc2..8dd127af52 100644

[...]

blockdev-nbd.c BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 185

Quoted 6 lines
blockdev-nbd.c +1 -1
@@ -176,7 +182,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
 
-    if (!nbd_server) {
+    if (!nbd_server && !is_qemu_nbd) {

(This begs the question of how difficult it would be to let qemu-nbd use QMP’s nbd-server-start, but I will not ask it, for I fear the answer.)

Quoted 7 lines
blockdev-nbd.c +0 -0
         error_setg(errp, "NBD server not running");
         return NULL;
     }
qemu-nbd.c +0 -0
index 48aa8a9d46..d967b8fcb9 100644

[...]

Quoted 17 lines
qemu-nbd.c +10 -3
@@ -1050,9 +1050,27 @@ int main(int argc, char **argv)
 
     bs->detect_zeroes = detect_zeroes;
 
-    export = nbd_export_new(bs, export_name,
-                            export_description, bitmap, readonly, shared > 1,
-                            writethrough, &error_fatal);
+    nbd_server_is_qemu_nbd(true);
+
+    export_opts = g_new(BlockExportOptions, 1);
+    *export_opts = (BlockExportOptions) {
+        .type               = BLOCK_EXPORT_TYPE_NBD,
+        .has_writethrough   = true,
+        .writethrough       = writethrough,
+        .u.nbd = {
+            .device             = g_strdup(bdrv_get_node_name(bs)),
+            .has_name           = true,

qemu-nbd.c int main(int argc, char **argv) · line 1072

Quoted 10 lines
qemu-nbd.c +10 -0
+            .name               = g_strdup(export_name),
+            .has_description    = !!export_description,
+            .description        = g_strdup(export_description),
+            .has_writable       = true,
+            .writable           = !readonly,
+            .has_bitmap         = !!bitmap,
+            .bitmap             = g_strdup(bitmap),
+        },
+    };
+    blk_exp_add(export_opts, &error_fatal);

Why not use the already-global qmp_block_export_add(), if we don’t need the return value here? (Will we require it at some point?)

Max

Quoted 4 lines
qemu-nbd.c +1 -0
+    qapi_free_BlockExportOptions(export_opts);
 
     if (device) {
 #if HAVE_NBD_DEVICE
Reviewed-by
#53 lore raw

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

Quoted 71 lines · Max Reitz
Quoted 4 lines · Kevin Wolf

With this change, NBD exports are only created through the BlockExport interface any more. This allows us finally to move things from the NBD layer to the BlockExport layer if they make sense for other export types, too.

I see.

Quoted 11 lines

blk_exp_add() returns only a weak reference, so the explicit nbd_export_put() goes away.

---
 include/block/export.h |  2 ++
 include/block/nbd.h    |  1 +
 block/export/export.c  |  2 +-
 blockdev-nbd.c         |  8 +++++++-
 qemu-nbd.c             | 28 ++++++++++++++++++++++------
 5 files changed, 33 insertions(+), 8 deletions(-)

[...]

Quoted 4 lines
blockdev-nbd.c +0 -0
index d5b084acc2..8dd127af52 100644

[...]

blockdev-nbd.c BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 185

Quoted 6 lines
blockdev-nbd.c +1 -1
@@ -176,7 +182,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
 
-    if (!nbd_server) {
+    if (!nbd_server && !is_qemu_nbd) {

(This begs the question of how difficult it would be to let qemu-nbd use QMP’s nbd-server-start, but I will not ask it, for I fear the answer.)

Quoted 7 lines
blockdev-nbd.c +0 -0
         error_setg(errp, "NBD server not running");
         return NULL;
     }
qemu-nbd.c +0 -0
index 48aa8a9d46..d967b8fcb9 100644

[...]

Quoted 22 lines
qemu-nbd.c +15 -3
@@ -1050,9 +1050,27 @@ int main(int argc, char **argv)
 
     bs->detect_zeroes = detect_zeroes;
 
-    export = nbd_export_new(bs, export_name,
-                            export_description, bitmap, readonly, shared > 1,
-                            writethrough, &error_fatal);
+    nbd_server_is_qemu_nbd(true);
+
+    export_opts = g_new(BlockExportOptions, 1);
+    *export_opts = (BlockExportOptions) {
+        .type               = BLOCK_EXPORT_TYPE_NBD,
+        .has_writethrough   = true,
+        .writethrough       = writethrough,
+        .u.nbd = {
+            .device             = g_strdup(bdrv_get_node_name(bs)),
+            .has_name           = true,
+            .name               = g_strdup(export_name),
+            .has_description    = !!export_description,
+            .description        = g_strdup(export_description),
+            .has_writable       = true,
+            .writable           = !readonly,
Quoted 8 lines

qemu-nbd.c int main(int argc, char **argv) · line 1072

Quoted 5 lines
qemu-nbd.c +5 -0
+            .has_bitmap         = !!bitmap,
+            .bitmap             = g_strdup(bitmap),
+        },
+    };
+    blk_exp_add(export_opts, &error_fatal);

Why not use the already-global qmp_block_export_add(), if we don’t need the return value here? (Will we require it at some point?)

In the context of patch 13, which adds more blk_exp_* functions, it makes sense to make blk_exp_add() global, and then to use it here. So:

author
#54 lore raw

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

In reply to Max Reitz

Am 17.08.2020 um 16:27 hat Max Reitz geschrieben:

Quoted 31 lines · Kevin Wolf
Quoted 4 lines

With this change, NBD exports are only created through the BlockExport interface any more. This allows us finally to move things from the NBD layer to the BlockExport layer if they make sense for other export types, too.

I see.

Quoted 11 lines

blk_exp_add() returns only a weak reference, so the explicit nbd_export_put() goes away.

---
 include/block/export.h |  2 ++
 include/block/nbd.h    |  1 +
 block/export/export.c  |  2 +-
 blockdev-nbd.c         |  8 +++++++-
 qemu-nbd.c             | 28 ++++++++++++++++++++++------
 5 files changed, 33 insertions(+), 8 deletions(-)

[...]

Quoted 4 lines
blockdev-nbd.c +0 -0
index d5b084acc2..8dd127af52 100644

[...]

Quoted 2 lines
blockdev-nbd.c +0 -0
@@ -176,7 +182,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
Quoted 7 lines

blockdev-nbd.c BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 185

Quoted 4 lines
blockdev-nbd.c +1 -1
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
 
-    if (!nbd_server) {
+    if (!nbd_server && !is_qemu_nbd) {

(This begs the question of how difficult it would be to let qemu-nbd use QMP’s nbd-server-start, but I will not ask it, for I fear the answer.)

(It would probably include something along the lines of "patches welcome". (I initially wanted to do this, but came to the conclusion that it wasn't for this series when I noticed the socket activation code and discussed with danpb on IRC how to integrate it in SocketAddress.))

Quoted 32 lines
Quoted 7 lines
blockdev-nbd.c +0 -0
         error_setg(errp, "NBD server not running");
         return NULL;
     }
qemu-nbd.c +0 -0
index 48aa8a9d46..d967b8fcb9 100644

[...]

Quoted 22 lines
qemu-nbd.c +15 -3
@@ -1050,9 +1050,27 @@ int main(int argc, char **argv)
 
     bs->detect_zeroes = detect_zeroes;
 
-    export = nbd_export_new(bs, export_name,
-                            export_description, bitmap, readonly, shared > 1,
-                            writethrough, &error_fatal);
+    nbd_server_is_qemu_nbd(true);
+
+    export_opts = g_new(BlockExportOptions, 1);
+    *export_opts = (BlockExportOptions) {
+        .type               = BLOCK_EXPORT_TYPE_NBD,
+        .has_writethrough   = true,
+        .writethrough       = writethrough,
+        .u.nbd = {
+            .device             = g_strdup(bdrv_get_node_name(bs)),
+            .has_name           = true,
+            .name               = g_strdup(export_name),
+            .has_description    = !!export_description,
+            .description        = g_strdup(export_description),
+            .has_writable       = true,
+            .writable           = !readonly,
Quoted 8 lines

qemu-nbd.c int main(int argc, char **argv) · line 1072

Quoted 5 lines
qemu-nbd.c +5 -0
+            .has_bitmap         = !!bitmap,
+            .bitmap             = g_strdup(bitmap),
+        },
+    };
+    blk_exp_add(export_opts, &error_fatal);

Why not use the already-global qmp_block_export_add(), if we don’t need the return value here? (Will we require it at some point?)

I can do that. I needed the return value originally, but after some reshuffling of the series it magically disappeared.

Kevin

#55 lore raw

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

In reply to Kevin Wolf
Quoted 2 lines · Kevin Wolf

With this change, NBD exports are only created through the BlockExport interface any more. This allows us finally to move things from the NBD

s/are only/are now only/; s/any more //

Quoted 5 lines

layer to the BlockExport layer if they make sense for other export types, too.

blk_exp_add() returns only a weak reference, so the explicit nbd_export_put() goes away.

Feel free to rename get/put to ref/unref in your series if that makes life any easier.

qemu-nbd.c int main(int argc, char **argv) · line 1053

Quoted 12 lines

---

qemu-nbd.c +1 -3
@@ -1050,9 +1050,27 @@ int main(int argc, char **argv)
 
      bs->detect_zeroes = detect_zeroes;
 
-    export = nbd_export_new(bs, export_name,
-                            export_description, bitmap, readonly, shared > 1,
-                            writethrough, &error_fatal);
+    nbd_server_is_qemu_nbd(true);

Feels a bit like a backdoor hack, but gets the job done (and as you said, we had quite an IRC conversation about what it would take to get socket activation working, so leaving that in qemu-nbd for now is reasonable for the first patch series).

Quoted 10 lines
qemu-nbd.c +10 -0
+
+    export_opts = g_new(BlockExportOptions, 1);
+    *export_opts = (BlockExportOptions) {
+        .type               = BLOCK_EXPORT_TYPE_NBD,
+        .has_writethrough   = true,
+        .writethrough       = writethrough,
+        .u.nbd = {
+            .device             = g_strdup(bdrv_get_node_name(bs)),
+            .has_name           = true,
+            .name               = g_strdup(export_name),

qemu-nbd.c int main(int argc, char **argv) · line 1073

Quoted 10 lines
qemu-nbd.c +10 -0
+            .has_description    = !!export_description,
+            .description        = g_strdup(export_description),
+            .has_writable       = true,
+            .writable           = !readonly,
+            .has_bitmap         = !!bitmap,
+            .bitmap             = g_strdup(bitmap),
+        },
+    };
+    blk_exp_add(export_opts, &error_fatal);
+    qapi_free_BlockExportOptions(export_opts);

Looks sane.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#56 lore raw

RFC PATCH 12/22 nbd/server: Simplify export shutdown

In reply to Kevin Wolf

Closing export is somewhat convoluted because nbd_export_close() and nbd_export_put() call each other and the ways they actually end up being nested is not necessarily obvious.

However, it is not really necessary to call nbd_export_close() from nbd_export_put() when putting the last reference because it only does three things:

1. Close all clients. We're going to refcount 0 and all clients hold a reference, so we know there is no active client any more.

2. Close the user reference (represented by exp->name being non-NULL). The same argument applies: If the export were still named, we would still have a reference.

3. Freeing exp->description. This is really cleanup work to be done when the export is finally freed. There is no reason to already clear it while clients are still in the process of shutting down.

So after moving the cleanup of exp->description, the code can be simplified so that only nbd_export_close() calls nbd_export_put(), but never the other way around.

---
 nbd/server.c | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)
nbd/server.c +4 -13
index eadc5b9804..4c594e6558 100644
@@ -1678,8 +1678,6 @@ void nbd_export_close(NBDExport *exp)
         QTAILQ_REMOVE(&exports, exp, next);
         QTAILQ_INSERT_TAIL(&closed_exports, exp, next);
     }
-    g_free(exp->description);
-    exp->description = NULL;
     nbd_export_put(exp);
 }
 
@@ -1706,19 +1704,12 @@ void nbd_export_get(NBDExport *exp)
 void nbd_export_put(NBDExport *exp)
 {
     assert(exp->refcount > 0);
-    if (exp->refcount == 1) {
-        nbd_export_close(exp);
-    }
-
-    /* nbd_export_close() may theoretically reduce refcount to 0. It may happen
-     * if someone calls nbd_export_put() on named export not through
-     * nbd_export_set_name() when refcount is 1. So, let's assert that
-     * it is > 0.
-     */
-    assert(exp->refcount > 0);
     if (--exp->refcount == 0) {
         assert(exp->name == NULL);
-        assert(exp->description == NULL);
+        assert(QTAILQ_EMPTY(&exp->clients));
+
+        g_free(exp->description);
+        exp->description = NULL;
 
         if (exp->blk) {
             if (exp->eject_notifier_blk) {
Signature · 1 line
2.25.4
Reviewed-by on v1 12/22
#57 lore raw

Re: [RFC PATCH 12/22] nbd/server: Simplify export shutdown

Quoted 11 lines · Kevin Wolf

Closing export is somewhat convoluted because nbd_export_close() and nbd_export_put() call each other and the ways they actually end up being nested is not necessarily obvious.

However, it is not really necessary to call nbd_export_close() from nbd_export_put() when putting the last reference because it only does three things:

1. Close all clients. We're going to refcount 0 and all clients hold a reference, so we know there is no active client any more.

Quoted 7 lines

2. Close the user reference (represented by exp->name being non-NULL). The same argument applies: If the export were still named, we would still have a reference.

3. Freeing exp->description. This is really cleanup work to be done when the export is finally freed. There is no reason to already clear it while clients are still in the process of shutting down.

Convincing.

Quoted 8 lines

So after moving the cleanup of exp->description, the code can be simplified so that only nbd_export_close() calls nbd_export_put(), but never the other way around.

---
 nbd/server.c | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)
Reviewed-by on v1 12/22
#58 lore raw

Re: [RFC PATCH 12/22] nbd/server: Simplify export shutdown

In reply to Kevin Wolf
Quoted 3 lines · Kevin Wolf

Closing export is somewhat convoluted because nbd_export_close() and nbd_export_put() call each other and the ways they actually end up being nested is not necessarily obvious.

You are in a maze of twisty little passages, all alike.

Yes, I've always hated that part of the code; thanks for tackling a cleanup.

Quoted 16 lines

However, it is not really necessary to call nbd_export_close() from nbd_export_put() when putting the last reference because it only does three things:

1. Close all clients. We're going to refcount 0 and all clients hold a reference, so we know there is no active client any more.

2. Close the user reference (represented by exp->name being non-NULL). The same argument applies: If the export were still named, we would still have a reference.

3. Freeing exp->description. This is really cleanup work to be done when the export is finally freed. There is no reason to already clear it while clients are still in the process of shutting down.

Quoted 9 lines

So after moving the cleanup of exp->description, the code can be simplified so that only nbd_export_close() calls nbd_export_put(), but never the other way around.

---
  nbd/server.c | 17 ++++-------------
  1 file changed, 4 insertions(+), 13 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#59 lore raw

RFC PATCH 13/22 block/export: Move refcount from NBDExport to BlockExport

In reply to Kevin Wolf

Having a refcount makes sense for all types of block exports. It is also a prerequisite for keeping a list of all exports at the BlockExport level.

---
 include/block/export.h | 10 ++++++
 include/block/nbd.h    |  2 --
 block/export/export.c  | 14 ++++++++
 blockdev-nbd.c         |  2 +-
 nbd/server.c           | 72 +++++++++++++++++++-----------------------
 5 files changed, 58 insertions(+), 42 deletions(-)
include/block/export.h +10 -0
index 5424bdc85d..f44290a4a2 100644
@@ -21,14 +21,24 @@ typedef struct BlockExport BlockExport;
 typedef struct BlockExportDriver {
     BlockExportType type;
     BlockExport *(*create)(BlockExportOptions *, Error **);
+    void (*delete)(BlockExport *);
 } BlockExportDriver;
 
 struct BlockExport {
     const BlockExportDriver *drv;
+
+    /*
+     * Reference count for this block export. This includes strong references
+     * both from the owner (qemu-nbd or the monitor) and clients connected to
+     * the export.
+     */
+    int refcount;
 };
 
 extern const BlockExportDriver blk_exp_nbd;
 
 BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp);
+void blk_exp_ref(BlockExport *exp);
+void blk_exp_unref(BlockExport *exp);
 
 #endif
include/block/nbd.h +0 -2
index 23030db3f1..af8509ab70 100644
@@ -336,8 +336,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
 void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
 void nbd_export_close(NBDExport *exp);
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
-void nbd_export_get(NBDExport *exp);
-void nbd_export_put(NBDExport *exp);
 
 AioContext *nbd_export_aio_context(NBDExport *exp);
 NBDExport *nbd_export_find(const char *name);
block/export/export.c +14 -0
index 12672228c7..1d5de564c7 100644
@@ -49,6 +49,20 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
     return drv->create(export, errp);
 }
 
+void blk_exp_ref(BlockExport *exp)
+{
+    assert(exp->refcount > 0);
+    exp->refcount++;
+}
+
+void blk_exp_unref(BlockExport *exp)
+{
+    assert(exp->refcount > 0);
+    if (--exp->refcount == 0) {
+        exp->drv->delete(exp);
+    }
+}
+
 void qmp_block_export_add(BlockExportOptions *export, Error **errp)
 {
     blk_exp_add(export, errp);
blockdev-nbd.c +1 -1
index 8dd127af52..a8b7b785e7 100644
@@ -232,7 +232,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
     /* The list of named exports has a strong reference to this export now and
      * our only way of accessing it is through nbd_export_find(), so we can drop
      * the strong reference that is @exp. */
-    nbd_export_put(exp);
+    blk_exp_unref((BlockExport*) exp);
 
  out:
     aio_context_release(aio_context);
nbd/server.c +33 -39

150 lines not on this page · raw message

Signature · 1 line
2.25.4
Reviewed-by on v1 13/22
#60 lore raw

Re: [RFC PATCH 13/22] block/export: Move refcount from NBDExport to BlockExport

Quoted 12 lines · Kevin Wolf

Having a refcount makes sense for all types of block exports. It is also a prerequisite for keeping a list of all exports at the BlockExport level.

---
 include/block/export.h | 10 ++++++
 include/block/nbd.h    |  2 --
 block/export/export.c  | 14 ++++++++
 blockdev-nbd.c         |  2 +-
 nbd/server.c           | 72 +++++++++++++++++++-----------------------
 5 files changed, 58 insertions(+), 42 deletions(-)

[...]

blockdev-nbd.c BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 235

Quoted 10 lines
blockdev-nbd.c +1 -1
index 8dd127af52..a8b7b785e7 100644
@@ -232,7 +232,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
     /* The list of named exports has a strong reference to this export now and
      * our only way of accessing it is through nbd_export_find(), so we can drop
      * the strong reference that is @exp. */
-    nbd_export_put(exp);
+    blk_exp_unref((BlockExport*) exp);

:/

Less so because of the asterisk, but more so because of “another instance of a cast because we can’t access a BlockExport’s fields.

Quoted 6 lines
blockdev-nbd.c +0 -0
  out:
     aio_context_release(aio_context);
nbd/server.c +0 -0
index 4c594e6558..2bf30bb731 100644

[...]

nbd/server.c NBDExport *nbd_export_new(BlockDriverState *bs, · line 1541

Quoted 8 lines
nbd/server.c +2 -1
@@ -1537,7 +1536,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
 
     exp = g_new0(NBDExport, 1);
     exp->common = (BlockExport) {
-        .drv = &blk_exp_nbd,
+        .drv        = &blk_exp_nbd,
+        .refcount   = 1,
     };

This makes me wish... Ah, for patch 16, I see. :)

#61 lore raw

Re: [RFC PATCH 13/22] block/export: Move refcount from NBDExport to BlockExport

In reply to Kevin Wolf
Quoted 12 lines · Kevin Wolf

Having a refcount makes sense for all types of block exports. It is also a prerequisite for keeping a list of all exports at the BlockExport level.

---

+++ b/include/block/export.h

include/block/export.h +0 -0
@@ -21,14 +21,24 @@ typedef struct BlockExport BlockExport;
  typedef struct BlockExportDriver {
      BlockExportType type;
      BlockExport *(*create)(BlockExportOptions *, Error **);

include/block/export.h typedef struct BlockExport BlockExport; · line 33

Quoted 10 lines
include/block/export.h +6 -0
+    void (*delete)(BlockExport *);
  } BlockExportDriver;
 
  struct BlockExport {
      const BlockExportDriver *drv;
+
+    /*
+     * Reference count for this block export. This includes strong references
+     * both from the owner (qemu-nbd or the monitor) and clients connected to
+     * the export.

I guess 'the monitor' includes qemu-storage-daemon.

include/block/export.h typedef struct BlockExport BlockExport; · line 42

Quoted 9 lines
include/block/export.h +4 -0
+     */
+    int refcount;
  };
 
  extern const BlockExportDriver blk_exp_nbd;
 
  BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp);
+void blk_exp_ref(BlockExport *exp);
+void blk_exp_unref(BlockExport *exp);

Yay, I think this naming is more consistent with the rest of qemu...

include/block/nbd.h NBDExport *nbd_export_new(BlockDriverState *bs, · line 340

Quoted 12 lines
(quoted diff) +0 -0
  #endif
include/block/nbd.h +0 -2
index 23030db3f1..af8509ab70 100644
@@ -336,8 +336,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
  void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
  void nbd_export_close(NBDExport *exp);
  void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
-void nbd_export_get(NBDExport *exp);
-void nbd_export_put(NBDExport *exp);

...as opposed to this which is common in kernel but less so in this project. No hard feelings from me :)

include/block/nbd.h BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 235

Quoted 7 lines
include/block/nbd.h +2 -1
+++ b/blockdev-nbd.c
@@ -232,7 +232,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
      /* The list of named exports has a strong reference to this export now and
       * our only way of accessing it is through nbd_export_find(), so we can drop
       * the strong reference that is @exp. */
-    nbd_export_put(exp);
+    blk_exp_unref((BlockExport*) exp);

Even a helper function that converts NBDBlockExport* to BlockExport* rather than a cast might be nicer, but then again, I see from Max's review that this may be a temporary state of things.

(The QAPI contains such type-safe container casts, such as qapi_DriveBackup_base(), if that helps...)

include/block/nbd.h NBDExport *nbd_export_new(BlockDriverState *bs, · line 1540

Quoted 7 lines
include/block/nbd.h +2 -1
@@ -1537,7 +1536,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
 
      exp = g_new0(NBDExport, 1);
      exp->common = (BlockExport) {
-        .drv = &blk_exp_nbd,
+        .drv        = &blk_exp_nbd,
+        .refcount   = 1,

I'm not sure whether trying to align the '=' is good, because the moment you add a longer field name, every other line has to be touched. I'm fine with just one space on both side of =, even if it is more ragged to read. But you're the author, so you get to pick.

include/block/nbd.h NBDExport *nbd_export_new(BlockDriverState *bs, · line 1630

Quoted 8 lines
include/block/nbd.h +2 -1
@@ -1626,8 +1625,9 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
      exp->ctx = ctx;
      blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
 
+    blk_exp_ref(&exp->common);
      QTAILQ_INSERT_TAIL(&exports, exp, next);
-    nbd_export_get(exp);
+

Is there any consequence to this changed ordering in grabbing the reference vs. updating the list?

Overall looks nice.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#62 lore raw

Re: [RFC PATCH 13/22] block/export: Move refcount from NBDExport to BlockExport

Am 19.08.2020 um 22:58 hat Eric Blake geschrieben:

Quoted 8 lines · Kevin Wolf
Quoted 6 lines

Having a refcount makes sense for all types of block exports. It is also a prerequisite for keeping a list of all exports at the BlockExport level.

---

Quoted 16 lines
Quoted 4 lines

+++ b/include/block/export.h

include/block/export.h +0 -0
@@ -21,14 +21,24 @@ typedef struct BlockExport BlockExport;
  typedef struct BlockExportDriver {
      BlockExportType type;

include/block/export.h typedef struct BlockExport BlockExport; · line 32

Quoted 10 lines
include/block/export.h +6 -0
      BlockExport *(*create)(BlockExportOptions *, Error **);
+    void (*delete)(BlockExport *);
  } BlockExportDriver;
  struct BlockExport {
      const BlockExportDriver *drv;
+
+    /*
+     * Reference count for this block export. This includes strong references
+     * both from the owner (qemu-nbd or the monitor) and clients connected to
+     * the export.

I guess 'the monitor' includes qemu-storage-daemon.

Yes, qemu-storage-daemon has a QMP monitor, so I would count it there. Even the --export command line option only calls the QMP command internally.

Quoted 25 lines

include/block/export.h typedef struct BlockExport BlockExport; · line 42

Quoted 7 lines
include/block/export.h +4 -0
+     */
+    int refcount;
  };
  extern const BlockExportDriver blk_exp_nbd;
  BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp);
+void blk_exp_ref(BlockExport *exp);
+void blk_exp_unref(BlockExport *exp);

Yay, I think this naming is more consistent with the rest of qemu...

include/block/nbd.h NBDExport *nbd_export_new(BlockDriverState *bs, · line 340

Quoted 11 lines
(quoted diff) +0 -0
  #endif
include/block/nbd.h +0 -2
index 23030db3f1..af8509ab70 100644
@@ -336,8 +336,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
  void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
  void nbd_export_close(NBDExport *exp);
  void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
-void nbd_export_get(NBDExport *exp);
-void nbd_export_put(NBDExport *exp);

...as opposed to this which is common in kernel but less so in this project. No hard feelings from me :)

Quoted 13 lines

include/block/nbd.h BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp) · line 235

Quoted 7 lines
include/block/nbd.h +2 -1
+++ b/blockdev-nbd.c
@@ -232,7 +232,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
      /* The list of named exports has a strong reference to this export now and
       * our only way of accessing it is through nbd_export_find(), so we can drop
       * the strong reference that is @exp. */
-    nbd_export_put(exp);
+    blk_exp_unref((BlockExport*) exp);

Even a helper function that converts NBDBlockExport* to BlockExport* rather than a cast might be nicer, but then again, I see from Max's review that this may be a temporary state of things.
(The QAPI contains such type-safe container casts, such as qapi_DriveBackup_base(), if that helps...)

Yes, this goes away before the end of the series.

Quoted 11 lines

include/block/nbd.h NBDExport *nbd_export_new(BlockDriverState *bs, · line 1539

Quoted 6 lines
include/block/nbd.h +2 -1
@@ -1537,7 +1536,8 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
      exp = g_new0(NBDExport, 1);
      exp->common = (BlockExport) {
-        .drv = &blk_exp_nbd,
+        .drv        = &blk_exp_nbd,
+        .refcount   = 1,

I'm not sure whether trying to align the '=' is good, because the moment you add a longer field name, every other line has to be touched. I'm fine with just one space on both side of =, even if it is more ragged to read. But you're the author, so you get to pick.

I generally prefer aligned '=' because the code is read much more often than it is written or modified, so being friendly for readers is important.

Quoted 10 lines

include/block/nbd.h NBDExport *nbd_export_new(BlockDriverState *bs, · line 1629

Quoted 7 lines
include/block/nbd.h +2 -1
@@ -1626,8 +1625,9 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
      exp->ctx = ctx;
      blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
+    blk_exp_ref(&exp->common);
      QTAILQ_INSERT_TAIL(&exports, exp, next);
-    nbd_export_get(exp);
+

Is there any consequence to this changed ordering in grabbing the reference vs. updating the list?

No intended consequences, but if Max is right that the code (before and after this series) lacks some locking, it might make a theoretical difference. If it does, the new code is safer than the old one. If it doesn't, it's just more consistent with the order I'm used to see in other places: First take the reference, then use it.

Kevin

author
#63 lore raw

RFC PATCH 14/22 block/export: Move AioContext from NBDExport to BlockExport

In reply to Kevin Wolf
---
 include/block/export.h |  6 ++++++
 nbd/server.c           | 26 +++++++++++++-------------
 2 files changed, 19 insertions(+), 13 deletions(-)
include/block/export.h +6 -0
index f44290a4a2..5459f79469 100644
@@ -33,6 +33,12 @@ struct BlockExport {
      * the export.
      */
     int refcount;
+
+    /*
+     * The AioContex whose lock needs to be held while calling
+     * BlockExportDriver callbacks.
+     */
+    AioContext *ctx;
 };
 
 extern const BlockExportDriver blk_exp_nbd;
nbd/server.c +13 -13
index 2bf30bb731..b735a68429 100644
@@ -92,8 +92,6 @@ struct NBDExport {
     QTAILQ_HEAD(, NBDClient) clients;
     QTAILQ_ENTRY(NBDExport) next;
 
-    AioContext *ctx;
-
     BlockBackend *eject_notifier_blk;
     Notifier eject_notifier;
 
@@ -1333,8 +1331,8 @@ static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)
     }
 
     /* Attach the channel to the same AioContext as the export */
-    if (client->exp && client->exp->ctx) {
-        qio_channel_attach_aio_context(client->ioc, client->exp->ctx);
+    if (client->exp && client->exp->common.ctx) {
+        qio_channel_attach_aio_context(client->ioc, client->exp->common.ctx);
     }
 
     assert(!client->optlen);
@@ -1466,7 +1464,7 @@ static void blk_aio_attached(AioContext *ctx, void *opaque)
 
     trace_nbd_blk_aio_attached(exp->name, ctx);
 
-    exp->ctx = ctx;
+    exp->common.ctx = ctx;
 
     QTAILQ_FOREACH(client, &exp->clients, next) {
         qio_channel_attach_aio_context(client->ioc, ctx);
@@ -1484,13 +1482,13 @@ static void blk_aio_detach(void *opaque)
     NBDExport *exp = opaque;
     NBDClient *client;
 
-    trace_nbd_blk_aio_detach(exp->name, exp->ctx);
+    trace_nbd_blk_aio_detach(exp->name, exp->common.ctx);
 
     QTAILQ_FOREACH(client, &exp->clients, next) {
         qio_channel_detach_aio_context(client->ioc);
     }
 
-    exp->ctx = NULL;
+    exp->common.ctx = NULL;
 }
 
 static void nbd_eject_notifier(Notifier *n, void *data)
@@ -1498,7 +1496,7 @@ static void nbd_eject_notifier(Notifier *n, void *data)
     NBDExport *exp = container_of(n, NBDExport, eject_notifier);
     AioContext *aio_context;
 
-    aio_context = exp->ctx;
+    aio_context = exp->common.ctx;
     aio_context_acquire(aio_context);
     nbd_export_close(exp);
     aio_context_release(aio_context);
@@ -1534,10 +1532,13 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
         return NULL;
     }
 
+    ctx = bdrv_get_aio_context(bs);
+
     exp = g_new0(NBDExport, 1);
     exp->common = (BlockExport) {
         .drv        = &blk_exp_nbd,
         .refcount   = 1,
+        .ctx        = ctx,
     };
 
     /*
@@ -1547,7 +1548,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
      * ctx was acquired in the caller.
      */
     assert(name && strlen(name) <= NBD_MAX_STRING_SIZE);
-    ctx = bdrv_get_aio_context(bs);
+
     bdrv_invalidate_cache(bs, NULL);
 
     /* Don't allow resize while the NBD server is running, otherwise we don't
@@ -1622,7 +1623,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
         assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
     }
 
-    exp->ctx = ctx;
     blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
 
     blk_exp_ref(&exp->common);
@@ -1653,7 +1653,7 @@ NBDExport *nbd_export_find(const char *name)
 AioContext *
 nbd_export_aio_context(NBDExport *exp)
 {
-    return exp->ctx;
+    return exp->common.ctx;
 }
 
 void nbd_export_close(NBDExport *exp)
@@ -1738,7 +1738,7 @@ void nbd_export_close_all(void)
     AioContext *aio_context;
 
     QTAILQ_FOREACH_SAFE(exp, &exports, next, next) {
-        aio_context = exp->ctx;
+        aio_context = exp->common.ctx;
         aio_context_acquire(aio_context);
         nbd_export_close(exp);
         aio_context_release(aio_context);
@@ -2519,7 +2519,7 @@ static void nbd_client_receive_next_request(NBDClient *client)
     if (!client->recv_coroutine && client->nb_requests < MAX_NBD_REQUESTS) {
         nbd_client_get(client);
         client->recv_coroutine = qemu_coroutine_create(nbd_trip, client);
-        aio_co_schedule(client->exp->ctx, client->recv_coroutine);
+        aio_co_schedule(client->exp->common.ctx, client->recv_coroutine);
     }
 }
 
Signature · 1 line
2.25.4
Reviewed-by on v1 14/22
#64 lore raw

Re: [RFC PATCH 14/22] block/export: Move AioContext from NBDExport to BlockExport

Quoted 5 lines · Kevin Wolf
---
 include/block/export.h |  6 ++++++
 nbd/server.c           | 26 +++++++++++++-------------
 2 files changed, 19 insertions(+), 13 deletions(-)

include/block/export.h struct BlockExport { · line 38

Quoted 11 lines
include/block/export.h +3 -0
index f44290a4a2..5459f79469 100644
@@ -33,6 +33,12 @@ struct BlockExport {
      * the export.
      */
     int refcount;
+
+    /*
+     * The AioContex whose lock needs to be held while calling

*AioContext

include/block/export.h struct BlockExport { · line 39

Quoted 1 line
include/block/export.h +1 -0
+     * BlockExportDriver callbacks.

Hm. But other blk_exp_* functions (i.e. the refcount manipulation functions) are fair game?

Quoted 9 lines
include/block/export.h +2 -0
+     */
+    AioContext *ctx;
 };
 
 extern const BlockExportDriver blk_exp_nbd;
nbd/server.c +0 -0
index 2bf30bb731..b735a68429 100644

[...]

nbd/server.c static void blk_aio_attached(AioContext *ctx, void *opaque) · line 1467

Quoted 6 lines
nbd/server.c +1 -1
@@ -1466,7 +1464,7 @@ static void blk_aio_attached(AioContext *ctx, void *opaque)
 
     trace_nbd_blk_aio_attached(exp->name, ctx);
 
-    exp->ctx = ctx;
+    exp->common.ctx = ctx;

(Not sure if Ḯ’m missing anything to that regard), but perhaps after patch 21 we can move this part to the common block export code, and maybe make it call a BlockExportDriver callback (that handles the rest of this function).

Quoted 5 lines
nbd/server.c +0 -0
     QTAILQ_FOREACH(client, &exp->clients, next) {
         qio_channel_attach_aio_context(client->ioc, ctx);
@@ -1484,13 +1482,13 @@ static void blk_aio_detach(void *opaque)
     NBDExport *exp = opaque;
     NBDClient *client;

nbd/server.c static void blk_aio_detach(void *opaque) · line 1491

Quoted 10 lines
nbd/server.c +2 -2
 
-    trace_nbd_blk_aio_detach(exp->name, exp->ctx);
+    trace_nbd_blk_aio_detach(exp->name, exp->common.ctx);
 
     QTAILQ_FOREACH(client, &exp->clients, next) {
         qio_channel_detach_aio_context(client->ioc);
     }
 
-    exp->ctx = NULL;
+    exp->common.ctx = NULL;

(same here)

Max

author
#65 lore raw

Re: [RFC PATCH 14/22] block/export: Move AioContext from NBDExport to BlockExport

Am 17.08.2020 um 16:56 hat Max Reitz geschrieben:

Quoted 23 lines · Kevin Wolf
Quoted 5 lines
---
 include/block/export.h |  6 ++++++
 nbd/server.c           | 26 +++++++++++++-------------
 2 files changed, 19 insertions(+), 13 deletions(-)

include/block/export.h struct BlockExport { · line 38

Quoted 11 lines
include/block/export.h +3 -0
index f44290a4a2..5459f79469 100644
@@ -33,6 +33,12 @@ struct BlockExport {
      * the export.
      */
     int refcount;
+
+    /*
+     * The AioContex whose lock needs to be held while calling

*AioContext

Quoted 4 lines

include/block/export.h struct BlockExport { · line 39

Quoted 1 line
include/block/export.h +1 -0
+     * BlockExportDriver callbacks.

Hm. But other blk_exp_* functions (i.e. the refcount manipulation functions) are fair game?

Hmm... The assumption was the ref/unref are only called from the main thread, but maybe that's not true? So maybe blk_exp_*() shouldn't lock the AioContext internally, but require that the lock is already held, so that they can be called both from within the AioContext (where we don't want to lock a second tim) and from the main context.

I also guess we need a separate mutex to protect the exports list if unref can be called from different threads.

And probably the existing NBD server code has already the same problems with respect to different AioContexts.

Quoted 16 lines
Quoted 9 lines
include/block/export.h +2 -0
+     */
+    AioContext *ctx;
 };
 
 extern const BlockExportDriver blk_exp_nbd;
nbd/server.c +0 -0
index 2bf30bb731..b735a68429 100644

[...]

Quoted 4 lines
nbd/server.c +0 -0
@@ -1466,7 +1464,7 @@ static void blk_aio_attached(AioContext *ctx, void *opaque)
 
     trace_nbd_blk_aio_attached(exp->name, ctx);
 
Quoted 7 lines

nbd/server.c static void blk_aio_attached(AioContext *ctx, void *opaque) · line 1467

Quoted 2 lines
nbd/server.c +1 -1
-    exp->ctx = ctx;
+    exp->common.ctx = ctx;

(Not sure if Ḯ’m missing anything to that regard), but perhaps after patch 21 we can move this part to the common block export code, and maybe make it call a BlockExportDriver callback (that handles the rest of this function).

Could probably be done. Not every export driver may support switching AioContexts, but we can make it conditional on having the callback.

So do I understand right from your comments to the series in general that you would prefer to make this series more complete, even if that means that it becomes quite a bit longer?

Kevin

#66 lore raw

Re: [RFC PATCH 14/22] block/export: Move AioContext from NBDExport to BlockExport

Quoted 65 lines · Kevin Wolf

Am 17.08.2020 um 16:56 hat Max Reitz geschrieben:

Quoted 23 lines · Kevin Wolf
Quoted 5 lines
---
 include/block/export.h |  6 ++++++
 nbd/server.c           | 26 +++++++++++++-------------
 2 files changed, 19 insertions(+), 13 deletions(-)

include/block/export.h struct BlockExport { · line 38

Quoted 11 lines
include/block/export.h +3 -0
index f44290a4a2..5459f79469 100644
@@ -33,6 +33,12 @@ struct BlockExport {
      * the export.
      */
     int refcount;
+
+    /*
+     * The AioContex whose lock needs to be held while calling

*AioContext

Quoted 4 lines

include/block/export.h struct BlockExport { · line 39

Quoted 1 line
include/block/export.h +1 -0
+     * BlockExportDriver callbacks.

Hm. But other blk_exp_* functions (i.e. the refcount manipulation functions) are fair game?

Hmm... The assumption was the ref/unref are only called from the main thread, but maybe that's not true? So maybe blk_exp_*() shouldn't lock the AioContext internally, but require that the lock is already held, so that they can be called both from within the AioContext (where we don't want to lock a second tim) and from the main context.

I also guess we need a separate mutex to protect the exports list if unref can be called from different threads.

And probably the existing NBD server code has already the same problems with respect to different AioContexts.

Quoted 16 lines
Quoted 9 lines
include/block/export.h +2 -0
+     */
+    AioContext *ctx;
 };
 
 extern const BlockExportDriver blk_exp_nbd;
nbd/server.c +0 -0
index 2bf30bb731..b735a68429 100644

[...]

Quoted 4 lines
nbd/server.c +0 -0
@@ -1466,7 +1464,7 @@ static void blk_aio_attached(AioContext *ctx, void *opaque)
 
     trace_nbd_blk_aio_attached(exp->name, ctx);
 
Quoted 7 lines

nbd/server.c static void blk_aio_attached(AioContext *ctx, void *opaque) · line 1467

Quoted 2 lines
nbd/server.c +1 -1
-    exp->ctx = ctx;
+    exp->common.ctx = ctx;

(Not sure if Ḯ’m missing anything to that regard), but perhaps after patch 21 we can move this part to the common block export code, and maybe make it call a BlockExportDriver callback (that handles the rest of this function).

Quoted 2 lines

Could probably be done. Not every export driver may support switching AioContexts, but we can make it conditional on having the callback.

Good point.

Quoted 3 lines

So do I understand right from your comments to the series in general that you would prefer to make this series more complete, even if that means that it becomes quite a bit longer?

I’m not necessarily asking for this now, I’m mostly asking whether you have the same idea as me on things like this. I don’t mind too much leaving this in an unfinished state as long as we both agree that it’s kind of unfinished.

Sorry if this is a bit frustrating to you because you wrote in the cover letter that indeed you are unsure about how complete you want to do this. The problem is that I don’t know exactly what things you’re referring to, so I just point out everything that stands out to me. If you’re aware of those things, and we can work on them later, then that’s OK.

OTOH... Yes, from a design standpoint, I think it makes sense to pull out as much specialized code as possible from NBD into the generalized block export code. But I say that as a reviewer. You would have to do that, so I want to leave it to you how much work you think is reasonable to put into that. Leaving a couple of rough edges here and there shouldn’t be a problem. (Or maybe leaving something to me for when I add fuse export code.)

Max

author
#67 lore raw

RFC PATCH 15/22 block/export: Move device to BlockExportOptions

In reply to Kevin Wolf

Every block export needs a block node to export, so move the 'device' option from BlockExportOptionsNbd to BlockExportOptions.

To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds 'device' back because nbd-server-add doesn't use the BlockExportOptions base type.

---
 qapi/block-export.json         | 27 +++++++++++++++++++++------
 block/export/export.c          | 26 ++++++++++++++++++++------
 block/monitor/block-hmp-cmds.c |  6 +++---
 blockdev-nbd.c                 |  4 ++--
 qemu-nbd.c                     |  2 +-
 5 files changed, 47 insertions(+), 18 deletions(-)
qapi/block-export.json +21 -6
index 4ce163411f..d68f3bf87e 100644
@@ -62,9 +62,8 @@
 ##
 # @BlockExportOptionsNbd:
 #
-# An NBD block export.
-#
-# @device: The device name or node name of the node to be exported
+# An NBD block export (options shared between nbd-server-add and the NBD branch
+# of block-export-add).
 #
 # @name: Export name. If unspecified, the @device parameter is used as the
 #        export name. (Since 2.12)
@@ -82,8 +81,21 @@
 # Since: 5.0
 ##
 { 'struct': 'BlockExportOptionsNbd',
-  'data': {'device': 'str', '*name': 'str', '*description': 'str',
-           '*writable': 'bool', '*bitmap': 'str' } }
+  'data': { '*name': 'str', '*description': 'str',
+            '*writable': 'bool', '*bitmap': 'str' } }
+
+##
+# @NbdServerAddOptions:
+#
+# An NBD block export.
+#
+# @device: The device name or node name of the node to be exported
+#
+# Since: 5.0
+##
+{ 'struct': 'NbdServerAddOptions',
+  'base': 'BlockExportOptionsNbd',
+  'data': { 'device': 'str' } }
 
 ##
 # @nbd-server-add:
@@ -96,7 +108,7 @@
 # Since: 1.3.0
 ##
 { 'command': 'nbd-server-add',
-  'data': 'BlockExportOptionsNbd', 'boxed': true }
+  'data': 'NbdServerAddOptions', 'boxed': true }
 
 ##
 # @NbdServerRemoveMode:
@@ -167,6 +179,8 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @device: The device name or node name of the node to be exported
+#
 # @writethrough: If true, caches are flushed after every write request to the
 #                export before completion is signalled. (since: 5.2;
 #                default: false)
@@ -175,6 +189,7 @@
 ##
 { 'union': 'BlockExportOptions',
   'base': { 'type': 'BlockExportType',
+            'device': 'str',
             '*writethrough': 'bool' },
   'discriminator': 'type',
   'data': {
block/export/export.c +20 -6
index 1d5de564c7..ef86bf892b 100644
@@ -68,15 +68,26 @@ void qmp_block_export_add(BlockExportOptions *export, Error **errp)
     blk_exp_add(export, errp);
 }
 
-void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
+void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
 {
     BlockExport *export;
     BlockDriverState *bs;
     BlockBackend *on_eject_blk;
 
-    BlockExportOptions export_opts = {
-        .type = BLOCK_EXPORT_TYPE_NBD,
-        .u.nbd = *arg,
+    BlockExportOptions *export_opts = g_new(BlockExportOptions, 1);
+    *export_opts = (BlockExportOptions) {
+        .type                   = BLOCK_EXPORT_TYPE_NBD,
+        .device                 = g_strdup(arg->device),
+        .u.nbd = {
+            .has_name           = arg->has_name,
+            .name               = g_strdup(arg->name),
+            .has_description    = arg->has_description,
+            .description        = g_strdup(arg->description),
+            .has_writable       = arg->has_writable,
+            .writable           = arg->writable,
+            .has_bitmap         = arg->has_bitmap,
+            .bitmap             = g_strdup(arg->bitmap),
+        },
     };
 
     /*
@@ -89,9 +100,9 @@ void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
         arg->writable = false;
     }
 
-    export = blk_exp_add(&export_opts, errp);
+    export = blk_exp_add(export_opts, errp);
     if (!export) {
-        return;
+        goto fail;
     }
 
     /*
@@ -102,4 +113,7 @@ void qmp_nbd_server_add(BlockExportOptionsNbd *arg, Error **errp)
     if (on_eject_blk) {
         nbd_export_set_on_eject_blk(export, on_eject_blk);
     }
+
+fail:
+    qapi_free_BlockExportOptions(export_opts);
 }
block/monitor/block-hmp-cmds.c +3 -3
index a651954e16..6c823234a9 100644
@@ -398,7 +398,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
     Error *local_err = NULL;
     BlockInfoList *block_list, *info;
     SocketAddress *addr;
-    BlockExportOptionsNbd export;
+    NbdServerAddOptions export;
 
     if (writable && !all) {
         error_setg(&local_err, "-w only valid together with -a");
@@ -431,7 +431,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
             continue;
         }
 
-        export = (BlockExportOptionsNbd) {
+        export = (NbdServerAddOptions) {
             .device         = info->value->device,
             .has_writable   = true,
             .writable       = writable,
@@ -458,7 +458,7 @@ void hmp_nbd_server_add(Monitor *mon, const QDict *qdict)
     bool writable = qdict_get_try_bool(qdict, "writable", false);
     Error *local_err = NULL;
 
-    BlockExportOptionsNbd export = {
+    NbdServerAddOptions export = {
         .device         = (char *) device,
         .has_name       = !!name,
         .name           = (char *) name,
blockdev-nbd.c +2 -2
index a8b7b785e7..5e97975c80 100644
@@ -188,7 +188,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
     }
 
     if (!arg->has_name) {
-        arg->name = arg->device;
+        arg->name = exp_args->device;
     }
 
     if (strlen(arg->name) > NBD_MAX_STRING_SIZE) {
@@ -206,7 +206,7 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
         return NULL;
     }
 
-    bs = bdrv_lookup_bs(arg->device, arg->device, errp);
+    bs = bdrv_lookup_bs(exp_args->device, exp_args->device, errp);
     if (!bs) {
         return NULL;
     }
qemu-nbd.c +1 -1
index d967b8fcb9..f31868708c 100644
@@ -1055,10 +1055,10 @@ int main(int argc, char **argv)
     export_opts = g_new(BlockExportOptions, 1);
     *export_opts = (BlockExportOptions) {
         .type               = BLOCK_EXPORT_TYPE_NBD,
+        .device             = g_strdup(bdrv_get_node_name(bs)),
         .has_writethrough   = true,
         .writethrough       = writethrough,
         .u.nbd = {
-            .device             = g_strdup(bdrv_get_node_name(bs)),
             .has_name           = true,
             .name               = g_strdup(export_name),
             .has_description    = !!export_description,
Signature · 1 line
2.25.4
#68 lore raw

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

Quoted 7 lines · Kevin Wolf

Every block export needs a block node to export, so move the 'device' option from BlockExportOptionsNbd to BlockExportOptions.

To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds 'device' back because nbd-server-add doesn't use the BlockExportOptions base type.

Quoted 8 lines
---
 qapi/block-export.json         | 27 +++++++++++++++++++++------
 block/export/export.c          | 26 ++++++++++++++++++++------
 block/monitor/block-hmp-cmds.c |  6 +++---
 blockdev-nbd.c                 |  4 ++--
 qemu-nbd.c                     |  2 +-
 5 files changed, 47 insertions(+), 18 deletions(-)

(Code diff looks good, just a question on the interface:)

Quoted 4 lines
qapi/block-export.json +0 -0
index 4ce163411f..d68f3bf87e 100644

[...]

qapi/block-export.json line 183

Quoted 6 lines
qapi/block-export.json +2 -0
@@ -167,6 +179,8 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @device: The device name or node name of the node to be exported
+#

Wouldn’t it be better to restrict ourselves to a node name here? (Bluntly ignoring the fact that doing so would make this patch an incompatible change, which would require some reordering in this series, unless we decide to just ignore that intra-series incompatibility.)

OTOH... What does “better” mean. It won’t hurt anyone to keep this as @device. It’s just that I feel like if we had no legacy burden and did this all from scratch, we would just make it @node-name.

Did you deliberately decide against @node-name?

Max

author
#69 lore raw

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

Am 17.08.2020 um 17:13 hat Max Reitz geschrieben:

Quoted 26 lines · Kevin Wolf
Quoted 7 lines

Every block export needs a block node to export, so move the 'device' option from BlockExportOptionsNbd to BlockExportOptions.

To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds 'device' back because nbd-server-add doesn't use the BlockExportOptions base type.

Quoted 8 lines
---
 qapi/block-export.json         | 27 +++++++++++++++++++++------
 block/export/export.c          | 26 ++++++++++++++++++++------
 block/monitor/block-hmp-cmds.c |  6 +++---
 blockdev-nbd.c                 |  4 ++--
 qemu-nbd.c                     |  2 +-
 5 files changed, 47 insertions(+), 18 deletions(-)

(Code diff looks good, just a question on the interface:)

Quoted 4 lines
qapi/block-export.json +0 -0
index 4ce163411f..d68f3bf87e 100644

[...]

Quoted 11 lines

qapi/block-export.json line 183

Quoted 6 lines
qapi/block-export.json +2 -0
@@ -167,6 +179,8 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @device: The device name or node name of the node to be exported
+#

Wouldn’t it be better to restrict ourselves to a node name here? (Bluntly ignoring the fact that doing so would make this patch an incompatible change, which would require some reordering in this series, unless we decide to just ignore that intra-series incompatibility.)

We already have intra-series incompatibility, so I wouldn't mind that.

Quoted 5 lines

OTOH... What does “better” mean. It won’t hurt anyone to keep this as @device. It’s just that I feel like if we had no legacy burden and did this all from scratch, we would just make it @node-name.

Did you deliberately decide against @node-name?

At first I thought I could still share code between nbd-server-add and block-export-add, but that's not the case. Then I guess the only other reason I have is consistency with other QMP commands. I won't pretend it's a strong one.

Kevin

#70 lore raw

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

Quoted 45 lines · Kevin Wolf

Am 17.08.2020 um 17:13 hat Max Reitz geschrieben:

Quoted 26 lines · Kevin Wolf
Quoted 7 lines

Every block export needs a block node to export, so move the 'device' option from BlockExportOptionsNbd to BlockExportOptions.

To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds 'device' back because nbd-server-add doesn't use the BlockExportOptions base type.

Quoted 8 lines
---
 qapi/block-export.json         | 27 +++++++++++++++++++++------
 block/export/export.c          | 26 ++++++++++++++++++++------
 block/monitor/block-hmp-cmds.c |  6 +++---
 blockdev-nbd.c                 |  4 ++--
 qemu-nbd.c                     |  2 +-
 5 files changed, 47 insertions(+), 18 deletions(-)

(Code diff looks good, just a question on the interface:)

Quoted 4 lines
qapi/block-export.json +0 -0
index 4ce163411f..d68f3bf87e 100644

[...]

Quoted 11 lines

qapi/block-export.json line 183

Quoted 6 lines
qapi/block-export.json +2 -0
@@ -167,6 +179,8 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @device: The device name or node name of the node to be exported
+#

Wouldn’t it be better to restrict ourselves to a node name here? (Bluntly ignoring the fact that doing so would make this patch an incompatible change, which would require some reordering in this series, unless we decide to just ignore that intra-series incompatibility.)

We already have intra-series incompatibility, so I wouldn't mind that.

Quoted 4 lines

OTOH... What does “better” mean. It won’t hurt anyone to keep this as @device. It’s just that I feel like if we had no legacy burden and did this all from scratch, we would just make it @node-name.

Quoted 6 lines
Quoted 1 line

Did you deliberately decide against @node-name?

At first I thought I could still share code between nbd-server-add and block-export-add, but that's not the case. Then I guess the only other reason I have is consistency with other QMP commands. I won't pretend it's a strong one.

If “using @node-name would be more natural” doesn’t resonate with you, then I suppose we should just leave it as @device. It isn’t harder to handle for qemu, and maybe it makes transitioning easier for some users (although I can’t quite imagine how).

Max

#71 lore raw

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

In reply to Kevin Wolf
Quoted 30 lines · Kevin Wolf

Every block export needs a block node to export, so move the 'device' option from BlockExportOptionsNbd to BlockExportOptions.

To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds 'device' back because nbd-server-add doesn't use the BlockExportOptions base type.

---

+++ b/qapi/block-export.json

qapi/block-export.json +3 -5
@@ -62,9 +62,8 @@
  ##
  # @BlockExportOptionsNbd:
  #
-# An NBD block export.
-#
-# @device: The device name or node name of the node to be exported
+# An NBD block export (options shared between nbd-server-add and the NBD branch
+# of block-export-add).
  #
  # @name: Export name. If unspecified, the @device parameter is used as the
  #        export name. (Since 2.12)
@@ -82,8 +81,21 @@
  # Since: 5.0
  ##
  { 'struct': 'BlockExportOptionsNbd',
-  'data': {'device': 'str', '*name': 'str', '*description': 'str',
-           '*writable': 'bool', '*bitmap': 'str' } }
+  'data': { '*name': 'str', '*description': 'str',

qapi/block-export.json line 94

Quoted 10 lines
qapi/block-export.json +10 -0
+            '*writable': 'bool', '*bitmap': 'str' } }
+
+##
+# @NbdServerAddOptions:
+#
+# An NBD block export.
+#
+# @device: The device name or node name of the node to be exported
+#
+# Since: 5.0

5.2, now?

qapi/block-export.json line 98

Quoted 4 lines
qapi/block-export.json +4 -0
+##
+{ 'struct': 'NbdServerAddOptions',
+  'base': 'BlockExportOptionsNbd',
+  'data': { 'device': 'str' } }

I understand why nbd sticks with device that can name device or node, but...

Quoted 22 lines
qapi/block-export.json +3 -1
  ##
  # @nbd-server-add:
@@ -96,7 +108,7 @@
  # Since: 1.3.0
  ##
  { 'command': 'nbd-server-add',
-  'data': 'BlockExportOptionsNbd', 'boxed': true }
+  'data': 'NbdServerAddOptions', 'boxed': true }
 
  ##
  # @NbdServerRemoveMode:
@@ -167,6 +179,8 @@
  # Describes a block export, i.e. how single node should be exported on an
  # external interface.
  #
+# @device: The device name or node name of the node to be exported
+#
  # @writethrough: If true, caches are flushed after every write request to the
  #                export before completion is signalled. (since: 5.2;
  #                default: false)

qapi/block-export.json line 192

Quoted 4 lines
qapi/block-export.json +1 -0
@@ -175,6 +189,7 @@
  ##
  { 'union': 'BlockExportOptions',
    'base': { 'type': 'BlockExportType',
+            'device': 'str',

for block export, I'd prefer that we mandate node name only, and naming it @node-name (rather than @device) feels nicer, for something that only new code will be using (that is, we should be encouraging the use of node names, especially now that libvirt has finally made that leap).

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
author
#72 lore raw

RFC PATCH 16/22 block/export: Allocate BlockExport in blk_exp_add()

In reply to Kevin Wolf

Instead of letting the driver allocate and return the BlockExport object, allocate it already in blk_exp_add() and pass it. This allows us to initialise the generic part before calling into the driver so that the driver can just use these values instead of having to parse the options a second time.

For symmetry, move freeing the BlockExport to blk_exp_unref().

---
 include/block/export.h |  3 ++-
 include/block/nbd.h    | 11 ++++++-----
 block/export/export.c  | 17 ++++++++++++++++-
 blockdev-nbd.c         | 24 +++++++++++++-----------
 nbd/server.c           | 30 +++++++++++++-----------------
 5 files changed, 50 insertions(+), 35 deletions(-)
include/block/export.h +2 -1
index 5459f79469..597fc58245 100644
@@ -20,7 +20,8 @@ typedef struct BlockExport BlockExport;
 
 typedef struct BlockExportDriver {
     BlockExportType type;
-    BlockExport *(*create)(BlockExportOptions *, Error **);
+    size_t instance_size;
+    int (*create)(BlockExport *, BlockExportOptions *, Error **);
     void (*delete)(BlockExport *);
 } BlockExportDriver;
 
include/block/nbd.h +6 -5
index af8509ab70..602536feb2 100644
@@ -328,11 +328,12 @@ int nbd_errno_to_system_errno(int err);
 typedef struct NBDExport NBDExport;
 typedef struct NBDClient NBDClient;
 
-BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp);
-NBDExport *nbd_export_new(BlockDriverState *bs,
-                          const char *name, const char *desc,
-                          const char *bitmap, bool readonly, bool shared,
-                          bool writethrough, Error **errp);
+int nbd_export_create(BlockExport *exp, BlockExportOptions *exp_args,
+                      Error **errp);
+int nbd_export_new(BlockExport *blk_exp, BlockDriverState *bs,
+                   const char *name, const char *desc,
+                   const char *bitmap, bool readonly, bool shared,
+                   bool writethrough, Error **errp);
 void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
 void nbd_export_close(NBDExport *exp);
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
block/export/export.c +16 -1
index ef86bf892b..9de108cbc1 100644
@@ -39,6 +39,8 @@ static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
 BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
 {
     const BlockExportDriver *drv;
+    BlockExport *exp;
+    int ret;
 
     drv = blk_exp_find_driver(export->type);
     if (!drv) {
@@ -46,7 +48,19 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
         return NULL;
     }
 
-    return drv->create(export, errp);
+    exp = g_malloc0(drv->instance_size);
+    *exp = (BlockExport) {
+        .drv        = &blk_exp_nbd,
+        .refcount   = 1,
+    };
+
+    ret = drv->create(exp, export, errp);
+    if (ret < 0) {
+        g_free(exp);
+        return NULL;
+    }
+
+    return exp;
 }
 
 void blk_exp_ref(BlockExport *exp)
@@ -60,6 +74,7 @@ void blk_exp_unref(BlockExport *exp)
     assert(exp->refcount > 0);
     if (--exp->refcount == 0) {
         exp->drv->delete(exp);
+        g_free(exp);
     }
 }
 
blockdev-nbd.c +13 -11
index 5e97975c80..f97deba424 100644
@@ -173,18 +173,19 @@ void qmp_nbd_server_start(SocketAddressLegacy *addr,
     qapi_free_SocketAddress(addr_flat);
 }
 
-BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
+int nbd_export_create(BlockExport *exp, BlockExportOptions *exp_args,
+                      Error **errp)
 {
     BlockExportOptionsNbd *arg = &exp_args->u.nbd;
     BlockDriverState *bs = NULL;
-    NBDExport *exp = NULL;
     AioContext *aio_context;
+    int ret;
 
     assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
 
     if (!nbd_server && !is_qemu_nbd) {
         error_setg(errp, "NBD server not running");
-        return NULL;
+        return -EINVAL;
     }
 
     if (!arg->has_name) {
@@ -193,22 +194,22 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
 
     if (strlen(arg->name) > NBD_MAX_STRING_SIZE) {
         error_setg(errp, "export name '%s' too long", arg->name);
-        return NULL;
+        return -EINVAL;
     }
 
     if (arg->description && strlen(arg->description) > NBD_MAX_STRING_SIZE) {
         error_setg(errp, "description '%s' too long", arg->description);
-        return NULL;
+        return -EINVAL;
     }
 
     if (nbd_export_find(arg->name)) {
         error_setg(errp, "NBD server already has export named '%s'", arg->name);
-        return NULL;
+        return -EEXIST;
     }
 
     bs = bdrv_lookup_bs(exp_args->device, exp_args->device, errp);
     if (!bs) {
-        return NULL;
+        return -ENOENT;
     }
 
     aio_context = bdrv_get_aio_context(bs);
@@ -218,14 +219,15 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
         arg->writable = false;
     }
     if (bdrv_is_read_only(bs) && arg->writable) {
+        ret = -EINVAL;
         error_setg(errp, "Cannot export read-only node as writable");
         goto out;
     }
 
-    exp = nbd_export_new(bs, arg->name, arg->description, arg->bitmap,
+    ret = nbd_export_new(exp, bs, arg->name, arg->description, arg->bitmap,
                          !arg->writable, !arg->writable,
                          exp_args->writethrough, errp);
-    if (!exp) {
+    if (ret < 0) {
         goto out;
     }
 
@@ -234,10 +236,10 @@ BlockExport *nbd_export_create(BlockExportOptions *exp_args, Error **errp)
      * the strong reference that is @exp. */
     blk_exp_unref((BlockExport*) exp);
 
+    ret = 0;
  out:
     aio_context_release(aio_context);
-    /* TODO Remove the cast: Move to server.c which can access fields of exp */
-    return (BlockExport*) exp;
+    return ret;
 }
 
 void qmp_nbd_server_remove(const char *name,
nbd/server.c +13 -17
index b735a68429..6a58297557 100644
@@ -1513,14 +1513,14 @@ void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk)
     blk_add_remove_bs_notifier(blk, &nbd_exp->eject_notifier);
 }
 
-NBDExport *nbd_export_new(BlockDriverState *bs,
-                          const char *name, const char *desc,
-                          const char *bitmap, bool readonly, bool shared,
-                          bool writethrough, Error **errp)
+int nbd_export_new(BlockExport *blk_exp, BlockDriverState *bs,
+                   const char *name, const char *desc,
+                   const char *bitmap, bool readonly, bool shared,
+                   bool writethrough, Error **errp)
 {
+    NBDExport *exp = container_of(blk_exp, NBDExport, common);
     AioContext *ctx;
     BlockBackend *blk;
-    NBDExport *exp;
     int64_t size;
     uint64_t perm;
     int ret;
@@ -1529,17 +1529,11 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
     if (size < 0) {
         error_setg_errno(errp, -size,
                          "Failed to determine the NBD export's length");
-        return NULL;
+        return size;
     }
 
     ctx = bdrv_get_aio_context(bs);
-
-    exp = g_new0(NBDExport, 1);
-    exp->common = (BlockExport) {
-        .drv        = &blk_exp_nbd,
-        .refcount   = 1,
-        .ctx        = ctx,
-    };
+    blk_exp->ctx = ctx;
 
     /*
      * NBD exports are used for non-shared storage migration.  Make sure
@@ -1599,16 +1593,19 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
         }
 
         if (bm == NULL) {
+            ret = -ENOENT;
             error_setg(errp, "Bitmap '%s' is not found", bitmap);
             goto fail;
         }
 
         if (bdrv_dirty_bitmap_check(bm, BDRV_BITMAP_ALLOW_RO, errp)) {
+            ret = -EINVAL;
             goto fail;
         }
 
         if (readonly && bdrv_is_writable(bs) &&
             bdrv_dirty_bitmap_enabled(bm)) {
+            ret = -EINVAL;
             error_setg(errp,
                        "Enabled bitmap '%s' incompatible with readonly export",
                        bitmap);
@@ -1628,14 +1625,13 @@ NBDExport *nbd_export_new(BlockDriverState *bs,
     blk_exp_ref(&exp->common);
     QTAILQ_INSERT_TAIL(&exports, exp, next);
 
-    return exp;
+    return 0;
 
 fail:
     blk_unref(blk);
     g_free(exp->name);
     g_free(exp->description);
-    g_free(exp);
-    return NULL;
+    return ret;
 }
 
 NBDExport *nbd_export_find(const char *name)
@@ -1722,12 +1718,12 @@ static void nbd_export_delete(BlockExport *blk_exp)
     }
 
     QTAILQ_REMOVE(&closed_exports, exp, next);
-    g_free(exp);
     aio_wait_kick();
 }
 
 const BlockExportDriver blk_exp_nbd = {
     .type               = BLOCK_EXPORT_TYPE_NBD,
+    .instance_size      = sizeof(NBDExport),
     .create             = nbd_export_create,
     .delete             = nbd_export_delete,
 };
Signature · 1 line
2.25.4
Reviewed-by on v1 16/22
#73 lore raw

Re: [RFC PATCH 16/22] block/export: Allocate BlockExport in blk_exp_add()

Quoted 16 lines · Kevin Wolf

Instead of letting the driver allocate and return the BlockExport object, allocate it already in blk_exp_add() and pass it. This allows us to initialise the generic part before calling into the driver so that the driver can just use these values instead of having to parse the options a second time.

For symmetry, move freeing the BlockExport to blk_exp_unref().

---
 include/block/export.h |  3 ++-
 include/block/nbd.h    | 11 ++++++-----
 block/export/export.c  | 17 ++++++++++++++++-
 blockdev-nbd.c         | 24 +++++++++++++-----------
 nbd/server.c           | 30 +++++++++++++-----------------
 5 files changed, 50 insertions(+), 35 deletions(-)

[...]

Quoted 4 lines
block/export/export.c +0 -0
index ef86bf892b..9de108cbc1 100644

[...]

block/export/export.c BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp) · line 49

Quoted 5 lines
block/export/export.c +0 -1
@@ -46,7 +48,19 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
         return NULL;
     }
 
-    return drv->create(export, errp);

assert(drv->instance_size >= sizeof(BlockExport)) might be nice before dereferencing *exp.

With that:

Quoted 5 lines
block/export/export.c +5 -0
+    exp = g_malloc0(drv->instance_size);
+    *exp = (BlockExport) {
+        .drv        = &blk_exp_nbd,
+        .refcount   = 1,
+    };
author
#74 lore raw

RFC PATCH 17/22 block/export: Add blk_exp_close_all(_type)

In reply to Kevin Wolf

This adds a function to shut down all block exports, and another one to shut down the block exports of a single type. The latter is used for now when stopping the NBD server. As soon as we implement support for multiple NBD servers, we'll need a per-server list of exports and it will be replaced by a function using that.

As a side effect, the BlockExport layer has a list tracking all existing exports now. closed_exports loses its only user and can go away.

---
 include/block/export.h |  8 +++++++
 include/block/nbd.h    |  2 --
 block.c                |  2 +-
 block/export/export.c  | 52 ++++++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c         |  2 +-
 nbd/server.c           | 34 ++++-----------------------
 qemu-nbd.c             |  2 +-
 7 files changed, 68 insertions(+), 34 deletions(-)
include/block/export.h +8 -0
index 597fc58245..1698b68f09 100644
@@ -15,6 +15,7 @@
 #define BLOCK_EXPORT_H
 
 #include "qapi/qapi-types-block-export.h"
+#include "qemu/queue.h"
 
 typedef struct BlockExport BlockExport;
 
@@ -23,6 +24,7 @@ typedef struct BlockExportDriver {
     size_t instance_size;
     int (*create)(BlockExport *, BlockExportOptions *, Error **);
     void (*delete)(BlockExport *);
+    void (*request_shutdown)(BlockExport *);
 } BlockExportDriver;
 
 struct BlockExport {
@@ -40,6 +42,9 @@ struct BlockExport {
      * BlockExportDriver callbacks.
      */
     AioContext *ctx;
+
+    /* List entry for block_exports */
+    QLIST_ENTRY(BlockExport) next;
 };
 
 extern const BlockExportDriver blk_exp_nbd;
@@ -47,5 +52,8 @@ extern const BlockExportDriver blk_exp_nbd;
 BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp);
 void blk_exp_ref(BlockExport *exp);
 void blk_exp_unref(BlockExport *exp);
+void blk_exp_request_shutdown(BlockExport *exp);
+void blk_exp_close_all(void);
+void blk_exp_close_all_type(BlockExportType type);
 
 #endif
include/block/nbd.h +0 -2
index 602536feb2..91a9d4f96d 100644
@@ -335,12 +335,10 @@ int nbd_export_new(BlockExport *blk_exp, BlockDriverState *bs,
                    const char *bitmap, bool readonly, bool shared,
                    bool writethrough, Error **errp);
 void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
-void nbd_export_close(NBDExport *exp);
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
 
 AioContext *nbd_export_aio_context(NBDExport *exp);
 NBDExport *nbd_export_find(const char *name);
-void nbd_export_close_all(void);
 
 void nbd_client_new(QIOChannelSocket *sioc,
                     QCryptoTLSCreds *tlscreds,
block.c +1 -1
index d9ac0e07eb..357c72846e 100644
@@ -4424,7 +4424,7 @@ static void bdrv_close(BlockDriverState *bs)
 void bdrv_close_all(void)
 {
     assert(job_next(NULL) == NULL);
-    nbd_export_close_all();
+    blk_exp_close_all();
 
     /* Drop references from requests still in flight, such as canceled block
      * jobs whose AIO context has not been polled yet */
block/export/export.c +52 -0

75 lines not on this page · raw message

blockdev-nbd.c +1 -1
index f97deba424..1c7aa874ee 100644
@@ -277,7 +277,7 @@ void qmp_nbd_server_stop(Error **errp)
         return;
     }
 
-    nbd_export_close_all();
+    blk_exp_close_all_type(BLOCK_EXPORT_TYPE_NBD);
 
     nbd_server_free(nbd_server);
     nbd_server = NULL;
nbd/server.c +5 -29
index 6a58297557..7e2976b81d 100644
@@ -100,8 +100,6 @@ struct NBDExport {
 };
 
 static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
-static QTAILQ_HEAD(, NBDExport) closed_exports =
-        QTAILQ_HEAD_INITIALIZER(closed_exports);
 
 /* NBDExportMetaContexts represents a list of contexts to be exported,
  * as selected by NBD_OPT_SET_META_CONTEXT. Also used for
@@ -1494,12 +1492,8 @@ static void blk_aio_detach(void *opaque)
 static void nbd_eject_notifier(Notifier *n, void *data)
 {
     NBDExport *exp = container_of(n, NBDExport, eject_notifier);
-    AioContext *aio_context;
 
-    aio_context = exp->common.ctx;
-    aio_context_acquire(aio_context);
-    nbd_export_close(exp);
-    aio_context_release(aio_context);
+    blk_exp_request_shutdown(&exp->common);
 }
 
 void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk)
@@ -1652,8 +1646,9 @@ nbd_export_aio_context(NBDExport *exp)
     return exp->common.ctx;
 }
 
-void nbd_export_close(NBDExport *exp)
+static void nbd_export_request_shutdown(BlockExport *blk_exp)
 {
+    NBDExport *exp = container_of(blk_exp, NBDExport, common);
     NBDClient *client, *next;
 
     blk_exp_ref(&exp->common);
@@ -1672,7 +1667,6 @@ void nbd_export_close(NBDExport *exp)
         g_free(exp->name);
         exp->name = NULL;
         QTAILQ_REMOVE(&exports, exp, next);
-        QTAILQ_INSERT_TAIL(&closed_exports, exp, next);
     }
     blk_exp_unref(&exp->common);
 }
@@ -1681,7 +1675,7 @@ void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp)
 {
     ERRP_GUARD();
     if (mode == NBD_SERVER_REMOVE_MODE_HARD || QTAILQ_EMPTY(&exp->clients)) {
-        nbd_export_close(exp);
+        nbd_export_request_shutdown(&exp->common);
         return;
     }
 
@@ -1716,9 +1710,6 @@ static void nbd_export_delete(BlockExport *blk_exp)
         bdrv_dirty_bitmap_set_busy(exp->export_bitmap, false);
         g_free(exp->export_bitmap_context);
     }
-
-    QTAILQ_REMOVE(&closed_exports, exp, next);
-    aio_wait_kick();
 }
 
 const BlockExportDriver blk_exp_nbd = {
@@ -1726,24 +1717,9 @@ const BlockExportDriver blk_exp_nbd = {
     .instance_size      = sizeof(NBDExport),
     .create             = nbd_export_create,
     .delete             = nbd_export_delete,
+    .request_shutdown   = nbd_export_request_shutdown,
 };
 
-void nbd_export_close_all(void)
-{
-    NBDExport *exp, *next;
-    AioContext *aio_context;
-
-    QTAILQ_FOREACH_SAFE(exp, &exports, next, next) {
-        aio_context = exp->common.ctx;
-        aio_context_acquire(aio_context);
-        nbd_export_close(exp);
-        aio_context_release(aio_context);
-    }
-
-    AIO_WAIT_WHILE(NULL, !(QTAILQ_EMPTY(&exports) &&
-                           QTAILQ_EMPTY(&closed_exports)));
-}
-
 static int coroutine_fn nbd_co_send_iov(NBDClient *client, struct iovec *iov,
                                         unsigned niov, Error **errp)
 {
qemu-nbd.c +1 -1
index f31868708c..939a08902a 100644
@@ -1110,7 +1110,7 @@ int main(int argc, char **argv)
     do {
         main_loop_wait(false);
         if (state == TERMINATE) {
-            nbd_export_close_all();
+            blk_exp_close_all();
             state = TERMINATED;
         }
     } while (state != TERMINATED);
Signature · 1 line
2.25.4
Reviewed-by on v1 17/22
#75 lore raw

Re: [RFC PATCH 17/22] block/export: Add blk_exp_close_all(_type)

Quoted 10 lines · Kevin Wolf

This adds a function to shut down all block exports, and another one to shut down the block exports of a single type. The latter is used for now when stopping the NBD server. As soon as we implement support for multiple NBD servers, we'll need a per-server list of exports and it will be replaced by a function using that.

As a side effect, the BlockExport layer has a list tracking all existing exports now. closed_exports loses its only user and can go away.

Quoted 9 lines
---
 include/block/export.h |  8 +++++++
 include/block/nbd.h    |  2 --
 block.c                |  2 +-
 block/export/export.c  | 52 ++++++++++++++++++++++++++++++++++++++++++
 blockdev-nbd.c         |  2 +-
 nbd/server.c           | 34 ++++-----------------------
 qemu-nbd.c             |  2 +-
 7 files changed, 68 insertions(+), 34 deletions(-)
Quoted 4 lines
block/export/export.c +0 -0
index 9de108cbc1..675db9a8b9 100644

[...]

Quoted 8 lines
block/export/export.c +8 -0
+/* type == BLOCK_EXPORT_TYPE__MAX for all types */
+void blk_exp_close_all_type(BlockExportType type)
+{
+    BlockExport *exp, *next;
+
+    QLIST_FOREACH_SAFE(exp, &block_exports, next, next) {
+        if (type != BLOCK_EXPORT_TYPE__MAX && exp->drv->type != type) {
+            continue;

block/export/export.c void blk_exp_unref(BlockExport *exp) · line 130

Quoted 10 lines
block/export/export.c +10 -0
+        }
+        blk_exp_request_shutdown(exp);
+    }
+
+    AIO_WAIT_WHILE(NULL, blk_exp_has_type(type));
+}
+
+void blk_exp_close_all(void)
+{
+    blk_exp_close_all_type(BLOCK_EXPORT_TYPE__MAX);

What’s interesting about this is that I saw from the header file that you added both this and the type-specific function and wondered “Why not just pass __MAX to close_all_type() to close all?” And then I thought “Because that would be stupid as an external interface”.

So I see you had the same thinking.

author
#76 lore raw

RFC PATCH 18/22 block/export: Add 'id' option to block-export-add

In reply to Kevin Wolf

We'll need an id to identify block exports in monitor commands. This adds one.

Note that this is different from the 'name' option in the NBD server, which is the externally visible export name. While block export ids need to be unique in the whole process, export names must be unique only for the same server. Different export types or (potentially in the future) multiple NBD servers can have the same export name externally, but still need different block export ids internally.

---
 qapi/block-export.json |  3 +++
 include/block/export.h |  3 +++
 block/export/export.c  | 27 +++++++++++++++++++++++++++
 qemu-nbd.c             |  1 +
 4 files changed, 34 insertions(+)
qapi/block-export.json +3 -0
index d68f3bf87e..0d0db9ca1b 100644
@@ -179,6 +179,8 @@
 # Describes a block export, i.e. how single node should be exported on an
 # external interface.
 #
+# @id: A unique identifier for the block export (across all export types)
+#
 # @device: The device name or node name of the node to be exported
 #
 # @writethrough: If true, caches are flushed after every write request to the
@@ -189,6 +191,7 @@
 ##
 { 'union': 'BlockExportOptions',
   'base': { 'type': 'BlockExportType',
+            'id': 'str',
             'device': 'str',
             '*writethrough': 'bool' },
   'discriminator': 'type',
include/block/export.h +3 -0
index 1698b68f09..43229857b0 100644
@@ -30,6 +30,9 @@ typedef struct BlockExportDriver {
 struct BlockExport {
     const BlockExportDriver *drv;
 
+    /* Unique identifier for the export */
+    char *id;
+
     /*
      * Reference count for this block export. This includes strong references
      * both from the owner (qemu-nbd or the monitor) and clients connected to
block/export/export.c +27 -0
index 675db9a8b9..72f1fab975 100644
@@ -19,6 +19,7 @@
 #include "block/nbd.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-block-export.h"
+#include "qemu/id.h"
 
 static const BlockExportDriver* blk_exp_drivers[] = {
     &blk_exp_nbd,
@@ -27,6 +28,19 @@ static const BlockExportDriver* blk_exp_drivers[] = {
 static QLIST_HEAD(, BlockExport) block_exports =
     QLIST_HEAD_INITIALIZER(block_exports);
 
+static BlockExport *blk_exp_find(const char *id)
+{
+    BlockExport *exp;
+
+    QLIST_FOREACH(exp, &block_exports, next) {
+        if (strcmp(id, exp->id) == 0) {
+            return exp;
+        }
+    }
+
+    return NULL;
+}
+
 static const BlockExportDriver *blk_exp_find_driver(BlockExportType type)
 {
     int i;
@@ -45,6 +59,15 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
     BlockExport *exp;
     int ret;
 
+    if (!id_wellformed(export->id)) {
+        error_setg(errp, "Invalid block export id");
+        return NULL;
+    }
+    if (blk_exp_find(export->id)) {
+        error_setg(errp, "Block export id '%s' is already in use", export->id);
+        return NULL;
+    }
+
     drv = blk_exp_find_driver(export->type);
     if (!drv) {
         error_setg(errp, "No driver found for the requested export type");
@@ -55,10 +78,12 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
     *exp = (BlockExport) {
         .drv        = &blk_exp_nbd,
         .refcount   = 1,
+        .id         = g_strdup(export->id),
     };
 
     ret = drv->create(exp, export, errp);
     if (ret < 0) {
+        g_free(exp->id);
         g_free(exp);
         return NULL;
     }
@@ -79,6 +104,7 @@ void blk_exp_unref(BlockExport *exp)
     if (--exp->refcount == 0) {
         QLIST_REMOVE(exp, next);
         exp->drv->delete(exp);
+        g_free(exp->id);
         g_free(exp);
         aio_wait_kick();
     }
@@ -144,6 +170,7 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
     BlockExportOptions *export_opts = g_new(BlockExportOptions, 1);
     *export_opts = (BlockExportOptions) {
         .type                   = BLOCK_EXPORT_TYPE_NBD,
+        .id                     = g_strdup(arg->name ?: arg->device),
         .device                 = g_strdup(arg->device),
         .u.nbd = {
             .has_name           = arg->has_name,
qemu-nbd.c +1 -0
index 939a08902a..c6fc0581c1 100644
@@ -1055,6 +1055,7 @@ int main(int argc, char **argv)
     export_opts = g_new(BlockExportOptions, 1);
     *export_opts = (BlockExportOptions) {
         .type               = BLOCK_EXPORT_TYPE_NBD,
+        .id                 = g_strdup("qemu-nbd-export"),
         .device             = g_strdup(bdrv_get_node_name(bs)),
         .has_writethrough   = true,
         .writethrough       = writethrough,
Signature · 1 line
2.25.4
#77 lore raw

Re: [RFC PATCH 18/22] block/export: Add 'id' option to block-export-add

Quoted 10 lines · Kevin Wolf

We'll need an id to identify block exports in monitor commands. This adds one.

Note that this is different from the 'name' option in the NBD server, which is the externally visible export name. While block export ids need to be unique in the whole process, export names must be unique only for the same server. Different export types or (potentially in the future) multiple NBD servers can have the same export name externally, but still need different block export ids internally.

Quoted 7 lines
---
 qapi/block-export.json |  3 +++
 include/block/export.h |  3 +++
 block/export/export.c  | 27 +++++++++++++++++++++++++++
 qemu-nbd.c             |  1 +
 4 files changed, 34 insertions(+)

Looks good, just one thing:

Quoted 4 lines
block/export/export.c +0 -0
index 675db9a8b9..72f1fab975 100644

[...]

block/export/export.c void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp) · line 173

Quoted 5 lines
block/export/export.c +1 -0
@@ -144,6 +170,7 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)
     BlockExportOptions *export_opts = g_new(BlockExportOptions, 1);
     *export_opts = (BlockExportOptions) {
         .type                   = BLOCK_EXPORT_TYPE_NBD,
+        .id                     = g_strdup(arg->name ?: arg->device),

Maybe this behavior should be documented for nbd-server-add?

Quoted 3 lines
block/export/export.c +0 -0
         .device                 = g_strdup(arg->device),
         .u.nbd = {
             .has_name           = arg->has_name,
author
#78 lore raw

RFC PATCH 19/22 block/export: Move strong user reference to block_exports

In reply to Kevin Wolf

The reference owned by the user/monitor that is created when adding the export and dropped when removing it was tied to the 'exports' list in nbd/server.c. Every block export will have a user reference, so move it to the block export level and tie it to the 'block_exports' list in block/export/export.c instead. This is necessary for introducing a QMP command for removing exports.

Note that exports are present in block_exports even after the user has requested shutdown. This is different from NBD's exports where exports are immediately removed on a shutdown request, even if they are still in the process of shutting down. In order to avoid that the user still interacts with an export that is shutting down (and possibly removes it a second time), we need to remember if the user actually still owns it.

---
 include/block/export.h | 8 ++++++++
 block/export/export.c  | 4 ++++
 blockdev-nbd.c         | 5 -----
 nbd/server.c           | 2 --
 4 files changed, 12 insertions(+), 7 deletions(-)
include/block/export.h +8 -0
index 43229857b0..83f554b745 100644
@@ -40,6 +40,14 @@ struct BlockExport {
      */
     int refcount;
 
+    /*
+     * True if one of the references in refcount belongs to the user. After the
+     * user has dropped their reference, they may not e.g. remove the same
+     * export a second time (which would decrease the refcount without having
+     * it incremented first).
+     */
+    bool user_owned;
+
     /*
      * The AioContex whose lock needs to be held while calling
      * BlockExportDriver callbacks.
block/export/export.c +4 -0
index 72f1fab975..f94a81258a 100644
@@ -78,6 +78,7 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
     *exp = (BlockExport) {
         .drv        = &blk_exp_nbd,
         .refcount   = 1,
+        .user_owned = true,
         .id         = g_strdup(export->id),
     };
 
@@ -117,6 +118,9 @@ void blk_exp_request_shutdown(BlockExport *exp)
     aio_context_acquire(aio_context);
     exp->drv->request_shutdown(exp);
     aio_context_release(aio_context);
+
+    exp->user_owned = false;
+    blk_exp_unref(exp);
 }
 
 static bool blk_exp_has_type(BlockExportType type)
blockdev-nbd.c +0 -5
index 1c7aa874ee..40013b7d64 100644
@@ -231,11 +231,6 @@ int nbd_export_create(BlockExport *exp, BlockExportOptions *exp_args,
         goto out;
     }
 
-    /* The list of named exports has a strong reference to this export now and
-     * our only way of accessing it is through nbd_export_find(), so we can drop
-     * the strong reference that is @exp. */
-    blk_exp_unref((BlockExport*) exp);
-
     ret = 0;
  out:
     aio_context_release(aio_context);
nbd/server.c +0 -2
index 7e2976b81d..e3ac7f548b 100644
@@ -1616,7 +1616,6 @@ int nbd_export_new(BlockExport *blk_exp, BlockDriverState *bs,
 
     blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
 
-    blk_exp_ref(&exp->common);
     QTAILQ_INSERT_TAIL(&exports, exp, next);
 
     return 0;
@@ -1663,7 +1662,6 @@ static void nbd_export_request_shutdown(BlockExport *blk_exp)
         client_close(client, true);
     }
     if (exp->name) {
-        blk_exp_unref(&exp->common);
         g_free(exp->name);
         exp->name = NULL;
         QTAILQ_REMOVE(&exports, exp, next);
Signature · 1 line
2.25.4
Reviewed-by on v1 19/22
#79 lore raw

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

Quoted 14 lines · Kevin Wolf

The reference owned by the user/monitor that is created when adding the export and dropped when removing it was tied to the 'exports' list in nbd/server.c. Every block export will have a user reference, so move it to the block export level and tie it to the 'block_exports' list in block/export/export.c instead. This is necessary for introducing a QMP command for removing exports.

Note that exports are present in block_exports even after the user has requested shutdown. This is different from NBD's exports where exports are immediately removed on a shutdown request, even if they are still in the process of shutting down. In order to avoid that the user still interacts with an export that is shutting down (and possibly removes it a second time), we need to remember if the user actually still owns it.

Quoted 7 lines
---
 include/block/export.h | 8 ++++++++
 block/export/export.c  | 4 ++++
 blockdev-nbd.c         | 5 -----
 nbd/server.c           | 2 --
 4 files changed, 12 insertions(+), 7 deletions(-)
#80 lore raw

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

In reply to Kevin Wolf
Quoted 14 lines · Kevin Wolf

The reference owned by the user/monitor that is created when adding the export and dropped when removing it was tied to the 'exports' list in nbd/server.c. Every block export will have a user reference, so move it to the block export level and tie it to the 'block_exports' list in block/export/export.c instead. This is necessary for introducing a QMP command for removing exports.

Note that exports are present in block_exports even after the user has requested shutdown. This is different from NBD's exports where exports are immediately removed on a shutdown request, even if they are still in the process of shutting down. In order to avoid that the user still interacts with an export that is shutting down (and possibly removes it a second time), we need to remember if the user actually still owns it.

Quoted 7 lines
---
 include/block/export.h | 8 ++++++++
 block/export/export.c  | 4 ++++
 blockdev-nbd.c         | 5 -----
 nbd/server.c           | 2 --
 4 files changed, 12 insertions(+), 7 deletions(-)

With this patch, there’s an abort in iotest 281. Perhaps because blk_exp_unref() is now done by blk_exp_request_shutdown() outside of where the AIO context is locked?

Max

author
#81 lore raw

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

Am 19.08.2020 um 13:56 hat Max Reitz geschrieben:

Quoted 23 lines · Kevin Wolf
Quoted 14 lines

The reference owned by the user/monitor that is created when adding the export and dropped when removing it was tied to the 'exports' list in nbd/server.c. Every block export will have a user reference, so move it to the block export level and tie it to the 'block_exports' list in block/export/export.c instead. This is necessary for introducing a QMP command for removing exports.

Note that exports are present in block_exports even after the user has requested shutdown. This is different from NBD's exports where exports are immediately removed on a shutdown request, even if they are still in the process of shutting down. In order to avoid that the user still interacts with an export that is shutting down (and possibly removes it a second time), we need to remember if the user actually still owns it.

Quoted 7 lines
---
 include/block/export.h | 8 ++++++++
 block/export/export.c  | 4 ++++
 blockdev-nbd.c         | 5 -----
 nbd/server.c           | 2 --
 4 files changed, 12 insertions(+), 7 deletions(-)
Quoted 3 lines

With this patch, there’s an abort in iotest 281. Perhaps because blk_exp_unref() is now done by blk_exp_request_shutdown() outside of where the AIO context is locked?

I have two fixes locally that were related to failing qemu-iotests. I guess the first one might be for what you're seeing?

Kevin

block/export/export.c +5 -0
index 71d17bd440..d021b98b74 100644
@@ -105,9 +105,14 @@ void blk_exp_unref(BlockExport *exp)
 {
     assert(exp->refcount > 0);
     if (--exp->refcount == 0) {
+        AioContext *aio_context = exp->ctx;
+
+        aio_context_acquire(aio_context);
         QLIST_REMOVE(exp, next);
         exp->drv->delete(exp);
         blk_unref(exp->blk);
+        aio_context_release(aio_context);
+
         g_free(exp->id);
         g_free(exp);
         aio_wait_kick();
tests/qemu-iotests/223.out +2 -2
index e1eaaedb55..31ce9e6fe0 100644
@@ -45,7 +45,7 @@ exports available: 0
 {"execute":"nbd-server-add", "arguments":{"device":"nosuch"}}
 {"error": {"class": "GenericError", "desc": "Cannot find device=nosuch nor node_name=nosuch"}}
 {"execute":"nbd-server-add", "arguments":{"device":"n"}}
-{"error": {"class": "GenericError", "desc": "NBD server already has export named 'n'"}}
+{"error": {"class": "GenericError", "desc": "Block export id 'n' is already in use"}}
 {"execute":"nbd-server-add", "arguments":{"device":"n", "name":"n2", "bitmap":"b2"}}
 {"error": {"class": "GenericError", "desc": "Enabled bitmap 'b2' incompatible with readonly export"}}
 {"execute":"nbd-server-add", "arguments":{"device":"n", "name":"n2", "bitmap":"b3"}}
@@ -126,7 +126,7 @@ exports available: 0
 {"execute":"nbd-server-add", "arguments":{"device":"nosuch"}}
 {"error": {"class": "GenericError", "desc": "Cannot find device=nosuch nor node_name=nosuch"}}
 {"execute":"nbd-server-add", "arguments":{"device":"n"}}
-{"error": {"class": "GenericError", "desc": "NBD server already has export named 'n'"}}
+{"error": {"class": "GenericError", "desc": "Block export id 'n' is already in use"}}
 {"execute":"nbd-server-add", "arguments":{"device":"n", "name":"n2", "bitmap":"b2"}}
 {"error": {"class": "GenericError", "desc": "Enabled bitmap 'b2' incompatible with readonly export"}}
 {"execute":"nbd-server-add", "arguments":{"device":"n", "name":"n2", "bitmap":"b3"}}
#82 lore raw

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

Quoted 38 lines · Kevin Wolf

Am 19.08.2020 um 13:56 hat Max Reitz geschrieben:

Quoted 23 lines · Kevin Wolf
Quoted 14 lines

The reference owned by the user/monitor that is created when adding the export and dropped when removing it was tied to the 'exports' list in nbd/server.c. Every block export will have a user reference, so move it to the block export level and tie it to the 'block_exports' list in block/export/export.c instead. This is necessary for introducing a QMP command for removing exports.

Note that exports are present in block_exports even after the user has requested shutdown. This is different from NBD's exports where exports are immediately removed on a shutdown request, even if they are still in the process of shutting down. In order to avoid that the user still interacts with an export that is shutting down (and possibly removes it a second time), we need to remember if the user actually still owns it.

Quoted 7 lines
---
 include/block/export.h | 8 ++++++++
 block/export/export.c  | 4 ++++
 blockdev-nbd.c         | 5 -----
 nbd/server.c           | 2 --
 4 files changed, 12 insertions(+), 7 deletions(-)
Quoted 3 lines

With this patch, there’s an abort in iotest 281. Perhaps because blk_exp_unref() is now done by blk_exp_request_shutdown() outside of where the AIO context is locked?

I have two fixes locally that were related to failing qemu-iotests. I guess the first one might be for what you're seeing?

Kevin

block/export/export.c +0 -0
index 71d17bd440..d021b98b74 100644

block/export/export.c void blk_exp_unref(BlockExport *exp) · line 107

Quoted 3 lines
block/export/export.c +0 -0
@@ -105,9 +105,14 @@ void blk_exp_unref(BlockExport *exp)
 {
     assert(exp->refcount > 0);
     if (--exp->refcount == 0) {

If this is done without locking the context, should this be an atomic operation?

block/export/export.c void blk_exp_unref(BlockExport *exp) · line 115

Quoted 8 lines
block/export/export.c +5 -0
+        AioContext *aio_context = exp->ctx;
+
+        aio_context_acquire(aio_context);
         QLIST_REMOVE(exp, next);
         exp->drv->delete(exp);
         blk_unref(exp->blk);
+        aio_context_release(aio_context);
+

But for the crash I was seeing, this should be sufficient, yes.

Max

author
#83 lore raw

RFC PATCH 20/22 block/export: Add block-export-del

In reply to Kevin Wolf

Implement a new QMP command block-export-del and make nbd-server-remove a wrapper around it.

---
 qapi/block-export.json         | 30 +++++++++++++++----
 include/block/nbd.h            |  1 -
 block/export/export.c          | 54 ++++++++++++++++++++++++++++++++++
 block/monitor/block-hmp-cmds.c |  2 +-
 blockdev-nbd.c                 | 28 ------------------
 nbd/server.c                   | 14 ---------
 6 files changed, 79 insertions(+), 50 deletions(-)
qapi/block-export.json +24 -6
index 0d0db9ca1b..a067de2ba3 100644
@@ -111,9 +111,9 @@
   'data': 'NbdServerAddOptions', 'boxed': true }
 
 ##
-# @NbdServerRemoveMode:
+# @BlockExportRemoveMode:
 #
-# Mode for removing an NBD export.
+# Mode for removing a block export.
 #
 # @safe: Remove export if there are no existing connections, fail otherwise.
 #
@@ -129,16 +129,16 @@
 #
 # Since: 2.12
 ##
-{'enum': 'NbdServerRemoveMode', 'data': ['safe', 'hard']}
+{'enum': 'BlockExportRemoveMode', 'data': ['safe', 'hard']}
 
 ##
 # @nbd-server-remove:
 #
 # Remove NBD export by name.
 #
-# @name: Export name.
+# @name: Block export id.
 #
-# @mode: Mode of command operation. See @NbdServerRemoveMode description.
+# @mode: Mode of command operation. See @BlockExportRemoveMode description.
 #        Default is 'safe'.
 #
 # Returns: error if
@@ -149,7 +149,7 @@
 # Since: 2.12
 ##
 { 'command': 'nbd-server-remove',
-  'data': {'name': 'str', '*mode': 'NbdServerRemoveMode'} }
+  'data': {'name': 'str', '*mode': 'BlockExportRemoveMode'} }
 
 ##
 # @nbd-server-stop:
@@ -208,3 +208,21 @@
 ##
 { 'command': 'block-export-add',
   'data': 'BlockExportOptions', 'boxed': true }
+
+##
+# @block-export-del:
+#
+# Remove a block export.
+#
+# @id: Block export id.
+#
+# @mode: Mode of command operation. See @BlockExportRemoveMode description.
+#        Default is 'safe'.
+#
+# Returns: Error if the export is not found or @mode is 'safe' and the export
+#          is still in use (e.g. by existing client connections)
+#
+# Since: 5.2
+##
+{ 'command': 'block-export-del',
+  'data': { 'id': 'str', '*mode': 'BlockExportRemoveMode' } }
include/block/nbd.h +0 -1
index 91a9d4f96d..7982a63f96 100644
@@ -335,7 +335,6 @@ int nbd_export_new(BlockExport *blk_exp, BlockDriverState *bs,
                    const char *bitmap, bool readonly, bool shared,
                    bool writethrough, Error **errp);
 void nbd_export_set_on_eject_blk(BlockExport *exp, BlockBackend *blk);
-void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
 
 AioContext *nbd_export_aio_context(NBDExport *exp);
 NBDExport *nbd_export_find(const char *name);
block/export/export.c +54 -0

63 lines not on this page · raw message

block/monitor/block-hmp-cmds.c +1 -1
index 6c823234a9..10165252cf 100644
@@ -477,7 +477,7 @@ void hmp_nbd_server_remove(Monitor *mon, const QDict *qdict)
     Error *err = NULL;
 
     /* Rely on NBD_SERVER_REMOVE_MODE_SAFE being the default */
-    qmp_nbd_server_remove(name, force, NBD_SERVER_REMOVE_MODE_HARD, &err);
+    qmp_nbd_server_remove(name, force, BLOCK_EXPORT_REMOVE_MODE_HARD, &err);
     hmp_handle_error(mon, err);
 }
 
blockdev-nbd.c +0 -28
index 40013b7d64..5ec21045a8 100644
@@ -237,34 +237,6 @@ int nbd_export_create(BlockExport *exp, BlockExportOptions *exp_args,
     return ret;
 }
 
-void qmp_nbd_server_remove(const char *name,
-                           bool has_mode, NbdServerRemoveMode mode,
-                           Error **errp)
-{
-    NBDExport *exp;
-    AioContext *aio_context;
-
-    if (!nbd_server) {
-        error_setg(errp, "NBD server not running");
-        return;
-    }
-
-    exp = nbd_export_find(name);
-    if (exp == NULL) {
-        error_setg(errp, "Export '%s' is not found", name);
-        return;
-    }
-
-    if (!has_mode) {
-        mode = NBD_SERVER_REMOVE_MODE_SAFE;
-    }
-
-    aio_context = nbd_export_aio_context(exp);
-    aio_context_acquire(aio_context);
-    nbd_export_remove(exp, mode, errp);
-    aio_context_release(aio_context);
-}
-
 void qmp_nbd_server_stop(Error **errp)
 {
     if (!nbd_server) {
nbd/server.c +0 -14
index e3ac7f548b..899d00782f 100644
@@ -1669,20 +1669,6 @@ static void nbd_export_request_shutdown(BlockExport *blk_exp)
     blk_exp_unref(&exp->common);
 }
 
-void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp)
-{
-    ERRP_GUARD();
-    if (mode == NBD_SERVER_REMOVE_MODE_HARD || QTAILQ_EMPTY(&exp->clients)) {
-        nbd_export_request_shutdown(&exp->common);
-        return;
-    }
-
-    assert(mode == NBD_SERVER_REMOVE_MODE_SAFE);
-
-    error_setg(errp, "export '%s' still in use", exp->name);
-    error_append_hint(errp, "Use mode='hard' to force client disconnect\n");
-}
-
 static void nbd_export_delete(BlockExport *blk_exp)
 {
     NBDExport *exp = container_of(blk_exp, NBDExport, common);
Signature · 1 line
2.25.4
Reviewed-by on v1 20/22
#84 lore raw

Re: [RFC PATCH 20/22] block/export: Add block-export-del

Quoted 12 lines · Kevin Wolf

Implement a new QMP command block-export-del and make nbd-server-remove a wrapper around it.

---
 qapi/block-export.json         | 30 +++++++++++++++----
 include/block/nbd.h            |  1 -
 block/export/export.c          | 54 ++++++++++++++++++++++++++++++++++
 block/monitor/block-hmp-cmds.c |  2 +-
 blockdev-nbd.c                 | 28 ------------------
 nbd/server.c                   | 14 ---------
 6 files changed, 79 insertions(+), 50 deletions(-)

[...]

block/monitor/block-hmp-cmds.c void hmp_nbd_server_remove(Monitor *mon, const QDict *qdict) · line 479

Quoted 8 lines
block/monitor/block-hmp-cmds.c +0 -0
index 6c823234a9..10165252cf 100644
@@ -477,7 +477,7 @@ void hmp_nbd_server_remove(Monitor *mon, const QDict *qdict)
     Error *err = NULL;
 
     /* Rely on NBD_SERVER_REMOVE_MODE_SAFE being the default */

This comment needs adjustment, too.

With that done:

Quoted 4 lines
block/monitor/block-hmp-cmds.c +1 -1
-    qmp_nbd_server_remove(name, force, NBD_SERVER_REMOVE_MODE_HARD, &err);
+    qmp_nbd_server_remove(name, force, BLOCK_EXPORT_REMOVE_MODE_HARD, &err);
     hmp_handle_error(mon, err);
 }
author
#85 lore raw

RFC PATCH 21/22 block/export: Move blk to BlockExport

In reply to Kevin Wolf

Every block export has a BlockBackend representing the disk that is exported. It should live in BlockExport therefore.

---
 include/block/export.h |  3 +++
 block/export/export.c  |  3 +++
 nbd/server.c           | 44 ++++++++++++++++++++++--------------------
 3 files changed, 29 insertions(+), 21 deletions(-)
include/block/export.h +3 -0
index 83f554b745..53b4163a3b 100644
@@ -54,6 +54,9 @@ struct BlockExport {
      */
     AioContext *ctx;
 
+    /* The block device to export */
+    BlockBackend *blk;
+
     /* List entry for block_exports */
     QLIST_ENTRY(BlockExport) next;
 };
block/export/export.c +3 -0
index 1255f3fc80..3cd448ba72 100644
@@ -89,6 +89,8 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
         return NULL;
     }
 
+    assert(exp->blk != NULL);
+
     QLIST_INSERT_HEAD(&block_exports, exp, next);
     return exp;
 }
@@ -105,6 +107,7 @@ void blk_exp_unref(BlockExport *exp)
     if (--exp->refcount == 0) {
         QLIST_REMOVE(exp, next);
         exp->drv->delete(exp);
+        blk_unref(exp->blk);
         g_free(exp->id);
         g_free(exp);
         aio_wait_kick();
nbd/server.c +23 -21

150 lines not on this page · raw message

Signature · 1 line
2.25.4
Reviewed-by on v1 21/22
#86 lore raw

Re: [RFC PATCH 21/22] block/export: Move blk to BlockExport

Quoted 9 lines · Kevin Wolf

Every block export has a BlockBackend representing the disk that is exported. It should live in BlockExport therefore.

---
 include/block/export.h |  3 +++
 block/export/export.c  |  3 +++
 nbd/server.c           | 44 ++++++++++++++++++++++--------------------
 3 files changed, 29 insertions(+), 21 deletions(-)
author
#87 lore raw

RFC PATCH 22/22 block/export: Add query-block-exports

In reply to Kevin Wolf

This adds a simple QMP command to query the list of block exports.

---
 qapi/block-export.json | 33 +++++++++++++++++++++++++++++++++
 block/export/export.c  | 23 +++++++++++++++++++++++
 2 files changed, 56 insertions(+)
qapi/block-export.json +33 -0
index a067de2ba3..0b184bbd7c 100644
@@ -226,3 +226,36 @@
 ##
 { 'command': 'block-export-del',
   'data': { 'id': 'str', '*mode': 'BlockExportRemoveMode' } }
+
+##
+# @BlockExportInfo:
+#
+# Information about a single block export.
+#
+# @id: The unique identifier for the block export
+#
+# @type: This field is returned only for compatibility reasons, it should
+#        not be used (always returns 'unknown')
+#
+# @node-name: The node name of the block node that is exported
+#
+# @shutting-down: True if the export is shutting down (e.g. after a
+#                 block-export-del command, but before the shutdown has
+#                 completed)
+#
+# Since:  5.2
+##
+{ 'struct': 'BlockExportInfo',
+  'data': { 'id': 'str',
+            'type': 'BlockExportType',
+            'node-name': 'str',
+            'shutting-down': 'bool' } }
+
+##
+# @query-block-exports:
+#
+# Returns: A list of BlockExportInfo describing all block exports
+#
+# Since: 5.2
+##
+{ 'command': 'query-block-exports', 'returns': ['BlockExportInfo'] }
block/export/export.c +23 -0
index 3cd448ba72..71d17bd440 100644
@@ -272,3 +272,26 @@ void qmp_nbd_server_remove(const char *name,
 
     qmp_block_export_del(name, has_mode, mode, errp);
 }
+
+BlockExportInfoList *qmp_query_block_exports(Error **errp)
+{
+    BlockExportInfoList *head = NULL, **p_next = &head;
+    BlockExport *exp;
+
+    QLIST_FOREACH(exp, &block_exports, next) {
+        BlockExportInfoList *entry = g_new0(BlockExportInfoList, 1);
+        BlockExportInfo *info = g_new(BlockExportInfo, 1);
+        *info = (BlockExportInfo) {
+            .id             = g_strdup(exp->id),
+            .type           = exp->drv->type,
+            .node_name      = g_strdup(bdrv_get_node_name(blk_bs(exp->blk))),
+            .shutting_down  = !exp->user_owned,
+        };
+
+        entry->value = info;
+        *p_next = entry;
+        p_next = &entry->next;
+    }
+
+    return head;
+}
Signature · 1 line
2.25.4
#88 lore raw

Re: [RFC PATCH 22/22] block/export: Add query-block-exports

Quoted 16 lines · Kevin Wolf

This adds a simple QMP command to query the list of block exports.

---
 qapi/block-export.json | 33 +++++++++++++++++++++++++++++++++
 block/export/export.c  | 23 +++++++++++++++++++++++
 2 files changed, 56 insertions(+)
qapi/block-export.json +0 -0
index a067de2ba3..0b184bbd7c 100644
@@ -226,3 +226,36 @@
 ##
 { 'command': 'block-export-del',
   'data': { 'id': 'str', '*mode': 'BlockExportRemoveMode' } }

qapi/block-export.json line 238

Quoted 10 lines
qapi/block-export.json +10 -0
+
+##
+# @BlockExportInfo:
+#
+# Information about a single block export.
+#
+# @id: The unique identifier for the block export
+#
+# @type: This field is returned only for compatibility reasons, it should
+#        not be used (always returns 'unknown')

Äh?

I don’t understand. It looks like it definitely doesn’t always return “unknown”. Also, the “compatibility reasons” aren’t really immediately clear to me... :?

Max

author
#89 lore raw

Re: [RFC PATCH 22/22] block/export: Add query-block-exports

Am 19.08.2020 um 13:04 hat Max Reitz geschrieben:

Quoted 30 lines · Kevin Wolf
Quoted 16 lines

This adds a simple QMP command to query the list of block exports.

---
 qapi/block-export.json | 33 +++++++++++++++++++++++++++++++++
 block/export/export.c  | 23 +++++++++++++++++++++++
 2 files changed, 56 insertions(+)
qapi/block-export.json +0 -0
index a067de2ba3..0b184bbd7c 100644
@@ -226,3 +226,36 @@
 ##
 { 'command': 'block-export-del',
   'data': { 'id': 'str', '*mode': 'BlockExportRemoveMode' } }

qapi/block-export.json line 238

Quoted 10 lines
qapi/block-export.json +10 -0
+
+##
+# @BlockExportInfo:
+#
+# Information about a single block export.
+#
+# @id: The unique identifier for the block export
+#
+# @type: This field is returned only for compatibility reasons, it should
+#        not be used (always returns 'unknown')

Äh?

Quoted 3 lines

I don’t understand. It looks like it definitely doesn’t always return “unknown”. Also, the “compatibility reasons” aren’t really immediately clear to me... :?

Oops, this seems to be copied from BlockInfo and I forgot to change it...

Kevin