Enable passing pre-opened chardev socket FDs

The fd_is_socket() helper method is useful in a few places, so put it in the common sockets code.

Signed-off-by Daniel P. Berrange <[email protected]>

---
  include/qemu/sockets.h |  1 +
  io/channel-util.c      | 13 -------------
  util/qemu-sockets.c    | 13 +++++++++++++
  3 files changed, 14 insertions(+), 13 deletions(-)

+++ b/util/qemu-sockets.c

util/qemu-sockets.c +0 -0

@@ -91,6 +91,19 @@ NetworkAddressFamily inet_netfamily(int family)
      return NETWORK_ADDRESS_FAMILY_UNKNOWN;
  }

Quoted 9 lines · Daniel P. Berrange

The fd_is_socket() helper method is useful in a few places, so put it in the common sockets code.

Signed-off-by Daniel P. Berrange <[email protected]>

---
  include/qemu/sockets.h |  1 +
  io/channel-util.c      | 13 -------------
  util/qemu-sockets.c    | 13 +++++++++++++
  3 files changed, 14 insertions(+), 13 deletions(-)

Reviewed-by Eric Blake <[email protected]>

But while you are touching this...

Quoted 12 lines

+++ b/util/qemu-sockets.c

util/qemu-sockets.c +7 -0

@@ -91,6 +91,19 @@ NetworkAddressFamily inet_netfamily(int family)
      return NETWORK_ADDRESS_FAMILY_UNKNOWN;
  }
  +bool fd_is_socket(int fd)
+{
+    int optval;
+    socklen_t optlen;
+    optlen = sizeof(optval);
+    return qemu_getsockopt(fd,
+                           SOL_SOCKET,
+                           SO_TYPE,

util/qemu-sockets.c NetworkAddressFamily inet_netfamily(int family) · line 102

Quoted 1 line

util/qemu-sockets.c +1 -0

+                           (char *)&optval,

This cast is pointless (although you are just moving it from the old code). qemu_getsockopt() already takes care of casting for mingw (where the signature is not POSIX-compliant), and on all other platforms, the argument is already prototyped as void*; and since void* accepts anything, you don't have to go through an intermediate char*.

Quoted 27 lines · Eric Blake

Quoted 9 lines · Daniel P. Berrange

The fd_is_socket() helper method is useful in a few places, so put it in the common sockets code.

Signed-off-by Daniel P. Berrange <[email protected]>

---
  include/qemu/sockets.h |  1 +
  io/channel-util.c      | 13 -------------
  util/qemu-sockets.c    | 13 +++++++++++++
  3 files changed, 14 insertions(+), 13 deletions(-)

Reviewed-by Eric Blake <[email protected]>

But while you are touching this...

Quoted 12 lines

+++ b/util/qemu-sockets.c

util/qemu-sockets.c +7 -0

@@ -91,6 +91,19 @@ NetworkAddressFamily inet_netfamily(int family)
      return NETWORK_ADDRESS_FAMILY_UNKNOWN;
  }
  +bool fd_is_socket(int fd)
+{
+    int optval;
+    socklen_t optlen;
+    optlen = sizeof(optval);
+    return qemu_getsockopt(fd,
+                           SOL_SOCKET,
+                           SO_TYPE,

Quoted 8 lines

util/qemu-sockets.c NetworkAddressFamily inet_netfamily(int family) · line 102

Quoted 1 line

util/qemu-sockets.c +1 -0

+                           (char *)&optval,

This cast is pointless (although you are just moving it from the old code). qemu_getsockopt() already takes care of casting for mingw (where the signature is not POSIX-compliant), and on all other platforms, the argument is already prototyped as void*; and since void* accepts anything, you don't have to go through an intermediate char*.

If we're tweaking, then I'd like it tweaked so:

    bool fd_is_socket(int fd)
    {
        int optval;
        socklen_t optlen = sizeof(optval);

        return !qemu_getsockopt(fd, SOL_SOCKET, SO_TYPE, &optval, &optlen);
    }

With or without such tweaks:

Reviewed-by Markus Armbruster <[email protected]>

When starting QEMU management apps will usually setup a monitor socket, and then open it immediately after startup. If not using QEMU's own -daemonize arg, this process can be troublesome to handle correctly. The mgmt app will need to repeatedly call connect() until it succeeds, because it does not know when QEMU has created the listener socket. If can't retry connect() forever though, because an error might have caused QEMU to exit before it even creates the monitor.

The obvious way to fix this kind of problem is to just pass in a pre-opened socket file descriptor for the QEMU monitor to listen on. The management app can now immediately call connect() just once. If connect() fails it knows that QEMU has exited with an error.

Signed-off-by Daniel P. Berrange <[email protected]>

---
  chardev/char-socket.c |  32 ++++++++--
  chardev/char.c        |   3 +
  tests/test-char.c     | 173 +++++++++++++++++++++++++++++++++++++++++++++++++-
  util/qemu-sockets.c   |  42 +++++++++++-
  4 files changed, 238 insertions(+), 12 deletions(-)

When starting QEMU management apps will usually setup a monitor socket, and then open it immediately after startup. If not using QEMU's own -daemonize arg, this process can be troublesome to handle correctly. The mgmt app will need to repeatedly call connect() until it succeeds, because it does not know when QEMU has created the listener socket. If can't retry connect() forever though, because an error might have caused QEMU to exit before it even creates the monitor.

The obvious way to fix this kind of problem is to just pass in a pre-opened socket file descriptor for the QEMU monitor to listen on. The management app can now immediately call connect() just once. If connect() fails it knows that QEMU has exited with an error.

The SocketAddress(Legacy) structs allow for FD passing via the monitor, using the 'getfd' command, but only when using QMP JSON syntax. The HMP & CLI syntax for chardevs, however, has no way to initialize the SocketAddress(Legacy) 'fd' variant. So this patch wires up the 'fd' parameter to refer to a refer to a passed in file descriptor. It can refer either to a named FD passed via the 'getfd' command, as already supported with QMP, or if numeric it refers to an FD passed to QEMU at startup. This allows both HMP usage:

(pass any FD number with SCM_RIGHTS)
getfd myfd
chardev-add socket,fd=myfd

And also CLI usage

(leak FD 3 from parent by clearing O_CLOEXEC)

-chardev socket,fd=3,id=mon
-mon chardev=mon,mode=control

Note that we do not wire this up in the legacy chardev syntax, so you cannot use FD passing with '-qmp', you must use the modern '-mon' + '-chardev' pair

An illustrative example of usage is:

#!/usr/bin/perl

use IO::Socket::UNIX;
use Fcntl;

unlink "/tmp/qmp";
my $srv = IO::Socket::UNIX->new(

    Type => SOCK_STREAM(),
    Local => "/tmp/qmp",
    Listen => 1,

);

  my $flags = fcntl $srv, F_GETFD, 0;
  fcntl $srv, F_SETFD, $flags & ~FD_CLOEXEC;

  my $fd = $srv->fileno();

  exec "qemu-system-x86_64", \
      "-chardev", "socket,fd=$fd,server,nowait,id=mon", \
      "-mon", "chardev=mon,mode=control";

Signed-off-by Daniel P. Berrange <[email protected]>

---
 chardev/char-socket.c |  32 ++++++++--
 chardev/char.c        |   3 +
 tests/test-char.c     | 173 +++++++++++++++++++++++++++++++++++++++++++++++++-
 util/qemu-sockets.c   |  42 +++++++++++-
 4 files changed, 238 insertions(+), 12 deletions(-)

chardev/char-socket.c +12 -3

index 6013972f72..e162db7542 100644
@@ -28,6 +28,7 @@
 #include "qemu/error-report.h"
 #include "qapi/error.h"
 #include "qapi/clone-visitor.h"
+#include "qemu/cutils.h"
 
 #include "chardev/char-io.h"
 

@@ -983,25 +984,36 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
     const char *path = qemu_opt_get(opts, "path");
     const char *host = qemu_opt_get(opts, "host");
     const char *port = qemu_opt_get(opts, "port");
+    const char *fd = qemu_opt_get(opts, "fd");
     const char *tls_creds = qemu_opt_get(opts, "tls-creds");
     SocketAddressLegacy *addr;
     ChardevSocket *sock;
 
+    if ((!!path + !!fd + !!host) != 1) {
+        error_setg(errp,
+                   "Exactly one of 'path', 'fd' or 'host' required");
+        return;
+    }
+
     backend->type = CHARDEV_BACKEND_KIND_SOCKET;
-    if (!path) {
-        if (!host) {
-            error_setg(errp, "chardev: socket: no host given");
+    if (path) {
+        if (tls_creds) {
+            error_setg(errp, "TLS can only be used over TCP socket");
             return;
         }
+    } else if (host) {
         if (!port) {
             error_setg(errp, "chardev: socket: no port given");

chardev/char-socket.c +7 -1

@@ -1027,7 +1039,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
         addr->type = SOCKET_ADDRESS_LEGACY_KIND_UNIX;
         q_unix = addr->u.q_unix.data = g_new0(UnixSocketAddress, 1);
         q_unix->path = g_strdup(path);
-    } else {
+    } else if (host) {
         addr->type = SOCKET_ADDRESS_LEGACY_KIND_INET;
         addr->u.inet.data = g_new(InetSocketAddress, 1);
         *addr->u.inet.data = (InetSocketAddress) {

@@ -1040,6 +1052,12 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
             .has_ipv6 = qemu_opt_get(opts, "ipv6"),
             .ipv6 = qemu_opt_get_bool(opts, "ipv6", 0),
         };
+    } else if (fd) {
+        addr->type = SOCKET_ADDRESS_LEGACY_KIND_FD;
+        addr->u.fd.data = g_new(String, 1);
+        addr->u.fd.data->str = g_strdup(fd);
+    } else {
+        g_assert_not_reached();
     }
     sock->addr = addr;
 }

chardev/char.c +3 -0

index 2ae4f465ec..9d674855ae 100644
@@ -798,6 +798,9 @@ QemuOptsList qemu_chardev_opts = {
         },{
             .name = "port",
             .type = QEMU_OPT_STRING,
+        },{
+            .name = "fd",
+            .type = QEMU_OPT_STRING,
         },{
             .name = "localaddr",
             .type = QEMU_OPT_STRING,

tests/test-char.c +34 -2

index 7ac25ff73f..e72a20f0c2 100644
@@ -9,6 +9,7 @@
 #include "qapi/error.h"
 #include "qom/qom-qobject.h"
 #include "qmp-commands.h"
+#include "monitor/monitor.h"
 
 static bool quit;
 

@@ -284,9 +285,8 @@ static int socket_can_read_hello(void *opaque)
     return 10;
 }
 
-static void char_socket_test(void)
+static void char_socket_test_common(Chardev *chr)
 {
-    Chardev *chr = qemu_chr_new("server", "tcp:127.0.0.1:0,server,nowait");
     Chardev *chr_client;
     QObject *addr;
     QDict *qdict;

@@ -341,6 +341,169 @@ static void char_socket_test(void)
     object_unparent(OBJECT(chr));
 }
 
+
+static void char_socket_basic_test(void)
+{
+    Chardev *chr = qemu_chr_new("server", "tcp:127.0.0.1:0,server,nowait");
+
+    char_socket_test_common(chr);
+}
+
+
+static int char_socket_listener(void)
+{
+    SocketAddress *addr = g_new0(SocketAddress, 1);
+    int srv;
+
+    addr->type = SOCKET_ADDRESS_TYPE_INET;
+    addr->u.inet.host = g_strdup("127.0.0.1");
+    addr->u.inet.port = g_strdup("0");
+
+    srv = socket_listen(addr, &error_abort);
+    g_assert(srv >= 0);
+
+    qapi_free_SocketAddress(addr);
+    return srv;
+}
+
+
+/* If a monitor is not active (ie cur_mon == NULL), then
+ * we should be able to use fd=<NUMBER> syntax
+ */
+static void char_socket_fdpass_cli_test(void)
+{
+    Chardev *chr;

tests/test-char.c +9 -0

+
+    chr = qemu_chr_new_from_opts(opts, &error_abort);
+
+    qemu_opts_del(opts);
+
+    char_socket_test_common(chr);
+}
+
+

tests/test-char.c +12 -0

+
+/* Syms in libqemustub.a are discarded at .o file granularity.
+ * To replace monitor_get_fd() we must ensure everything in
+ * stubs/monitor.c is defined, to make sure monitor.o is discarded
+ * otherwise we get duplicate syms at link time.
+ */
+Monitor *cur_mon = NULL;
+void monitor_init(Chardev *chr, int flags) {}
+
+/* If a monitor is active (ie cur_mon != NULL), then
+ * we should be able to use fd=<NAME> syntax
+ */

tests/test-char.c +24 -0

+    opts = qemu_opts_parse_noisily(qemu_find_opts("chardev"),
+                                   optstr, true);
+    g_assert_nonnull(opts);
+
+    chr = qemu_chr_new_from_opts(opts, &error_abort);
+
+    qemu_opts_del(opts);
+
+    char_socket_test_common(chr);
+    mon_fd = -1;
+    g_free(cur_mon);
+    cur_mon = NULL;
+}
+
+
+/* If a monitor is active (ie cur_mon != NULL), then
+ * we should not allow using fd=<NUMBER> syntax
+ */
+static void char_socket_fdpass_nocli_test(void)
+{
+    Chardev *chr;
+    char *optstr;
+    QemuOpts *opts;
+    int fd;

tests/test-char.c +19 -0

+    g_free(cur_mon);
+    cur_mon = NULL;
+}
+
+
+/* If a monitor is not active (ie cur_mon == NULL), then
+ * we should not allow using fd=<NAME> syntax
+ */
+static void char_socket_fdpass_nomon_test(void)
+{
+    Chardev *chr;
+    const char *optstr;
+    QemuOpts *opts;
+    Error *local_err = NULL;
+
+    g_assert_null(cur_mon);
+    optstr = "socket,id=cdev,fd=myfd,server,nowait";
+
+    opts = qemu_opts_parse_noisily(qemu_find_opts("chardev"),

tests/test-char.c +8 -1

+}
+
+
 #ifndef _WIN32
 static void char_pipe_test(void)
 {

@@ -757,7 +920,11 @@ int main(int argc, char **argv)
 #ifndef _WIN32
     g_test_add_func("/char/file-fifo", char_file_fifo_test);
 #endif
-    g_test_add_func("/char/socket", char_socket_test);
+    g_test_add_func("/char/socket/basic", char_socket_basic_test);
+    g_test_add_func("/char/socket/fdpass/cli", char_socket_fdpass_cli_test);
+    g_test_add_func("/char/socket/fdpass/mon", char_socket_fdpass_mon_test);
+    g_test_add_func("/char/socket/fdpass/nocli", char_socket_fdpass_nocli_test);
+    g_test_add_func("/char/socket/fdpass/nomon", char_socket_fdpass_nomon_test);
     g_test_add_func("/char/udp", char_udp_test);
 #ifdef HAVE_CHARDEV_SERIAL
     g_test_add_func("/char/serial", char_serial_test);

util/qemu-sockets.c +0 -0

index 1d23f0b742..9400f9a940 100644
@@ -1046,7 +1046,26 @@ int socket_connect(SocketAddress *addr, Error **errp)
         break;

util/qemu-sockets.c +13 -1

+            close(fd);
+            return -1;
+        }
         break;
 
     case SOCKET_ADDRESS_TYPE_VSOCK:

@@ -1073,7 +1092,26 @@ int socket_listen(SocketAddress *addr, Error **errp)
         break;
 
     case SOCKET_ADDRESS_TYPE_FD:
-        fd = monitor_get_fd(cur_mon, addr->u.fd.str, errp);
+        if (cur_mon) {
+            fd = monitor_get_fd(cur_mon, addr->u.fd.str, errp);
+            if (fd < 0) {
+                return -1;
+            }
+        } else {
+            unsigned long i;
+            if (qemu_strtoul(addr->u.fd.str, NULL, 10, &i) < 0) {
+                error_setg_errno(errp, errno,
+                                 "Unable to parse FD number %s",

Quoted 12 lines · Daniel P. Berrange

tests/test-char.c +12 -0

+
+/* Syms in libqemustub.a are discarded at .o file granularity.
+ * To replace monitor_get_fd() we must ensure everything in
+ * stubs/monitor.c is defined, to make sure monitor.o is discarded
+ * otherwise we get duplicate syms at link time.
+ */
+Monitor *cur_mon = NULL;
+void monitor_init(Chardev *chr, int flags) {}
+
+/* If a monitor is active (ie cur_mon != NULL), then
+ * we should be able to use fd=<NAME> syntax
+ */

tests/test-char.c static void char_socket_test(void) · line 429

Quoted 10 lines

tests/test-char.c +10 -0

+static void char_socket_fdpass_mon_test(void)
+{
+    Chardev *chr;
+    const char *optstr;
+    QemuOpts *opts;
+    int fd;
+
+    fd = char_socket_listener();
+    mon_fd = fd;
+    cur_mon = g_malloc(1); /* Pretend we have a mon available */

Feels unnecessarily dirty. Suggest to define cur_mon like this:

static Monitor dummy_mon;
Monitor *cur_mon = &dummy_mon; /* Pretend we have a mon available */

Or in case cur_mon must remain null outside this function, set it like this:

Monitor dummy_mon = {0};
cur_mon = &dummy_mon; /* Pretend we have a mon available */

More of the same below.

Quoted 6 lines

util/qemu-sockets.c +0 -0

index 1d23f0b742..9400f9a940 100644
@@ -1046,7 +1046,26 @@ int socket_connect(SocketAddress *addr, Error **errp)
         break;

util/qemu-sockets.c int socket_connect(SocketAddress *addr, Error **errp) · line 1055

Quoted 10 lines

util/qemu-sockets.c +7 -1

 
     case SOCKET_ADDRESS_TYPE_FD:
-        fd = monitor_get_fd(cur_mon, addr->u.fd.str, errp);
+        if (cur_mon) {
+            fd = monitor_get_fd(cur_mon, addr->u.fd.str, errp);
+            if (fd < 0) {
+                return -1;
+            }
+        } else {
+            unsigned long i;

Naming a long @i is bad taste. Let's rename to @ul.

util/qemu-sockets.c int socket_connect(SocketAddress *addr, Error **errp) · line 1062

Quoted 7 lines

util/qemu-sockets.c +7 -0

+            if (qemu_strtoul(addr->u.fd.str, NULL, 10, &i) < 0) {
+                error_setg_errno(errp, errno,
+                                 "Unable to parse FD number %s",
+                                 addr->u.fd.str);
+                return -1;
+            }
+            fd = i;

Truncates silently. Shouldn't you check for range?

If the parent process screws up passing the file descriptor, fd can hijack some random internal file. I'd ask you to catch that if I had any idea how to do that easily.

Outside monitor context, you can now use numeric fds, and only numeric fds. Makes sense, because named fds are associated with a monitor. Note that before the patch, we crashed in monitor_get_fd() dereferencing cur_mon.

Quoted 33 lines · Markus Armbruster

Quoted 12 lines · Daniel P. Berrange

tests/test-char.c +12 -0

+
+/* Syms in libqemustub.a are discarded at .o file granularity.
+ * To replace monitor_get_fd() we must ensure everything in
+ * stubs/monitor.c is defined, to make sure monitor.o is discarded
+ * otherwise we get duplicate syms at link time.
+ */
+Monitor *cur_mon = NULL;
+void monitor_init(Chardev *chr, int flags) {}
+
+/* If a monitor is active (ie cur_mon != NULL), then
+ * we should be able to use fd=<NAME> syntax
+ */

tests/test-char.c static void char_socket_test(void) · line 429

Quoted 10 lines

tests/test-char.c +10 -0

+static void char_socket_fdpass_mon_test(void)
+{
+    Chardev *chr;
+    const char *optstr;
+    QemuOpts *opts;
+    int fd;
+
+    fd = char_socket_listener();
+    mon_fd = fd;
+    cur_mon = g_malloc(1); /* Pretend we have a mon available */

Feels unnecessarily dirty. Suggest to define cur_mon like this:

static Monitor dummy_mon;
Monitor *cur_mon = &dummy_mon; /* Pretend we have a mon available */

Or in case cur_mon must remain null outside this function, set it like this:

Quoted 4 lines

Monitor dummy_mon = {0};
cur_mon = &dummy_mon; /* Pretend we have a mon available */

More of the same below.

FYI, I didn't do that because 'struct Monitor' is defined inside monitor.c, not exposed in header files. I felt it would be worse

to pollute the header file with what's supposed to be a private struct definition, just for sake of tests, particularly since we don't actually need any of the Monitor object contents.

We could create a monitor-internal.h for the "struct Monitor" definition, if you feel strongly we should take this approach in the tests instead of my hack here ?

Quoted 27 lines

Quoted 6 lines

util/qemu-sockets.c +0 -0

index 1d23f0b742..9400f9a940 100644
@@ -1046,7 +1046,26 @@ int socket_connect(SocketAddress *addr, Error **errp)
         break;

util/qemu-sockets.c int socket_connect(SocketAddress *addr, Error **errp) · line 1055

Quoted 10 lines

util/qemu-sockets.c +7 -1

 
     case SOCKET_ADDRESS_TYPE_FD:
-        fd = monitor_get_fd(cur_mon, addr->u.fd.str, errp);
+        if (cur_mon) {
+            fd = monitor_get_fd(cur_mon, addr->u.fd.str, errp);
+            if (fd < 0) {
+                return -1;
+            }
+        } else {
+            unsigned long i;

Naming a long @i is bad taste. Let's rename to @ul.

Quoted 7 lines

util/qemu-sockets.c +7 -0

+            if (qemu_strtoul(addr->u.fd.str, NULL, 10, &i) < 0) {
+                error_setg_errno(errp, errno,
+                                 "Unable to parse FD number %s",
+                                 addr->u.fd.str);
+                return -1;
+            }
+            fd = i;

Quoted 5 lines

Truncates silently. Shouldn't you check for range?

If the parent process screws up passing the file descriptor, fd can hijack some random internal file. I'd ask you to catch that if I had any idea how to do that easily.

I guess it is just a matter of defining yet another qemu_strtoNN variant that takes an "int" parameter instead of "long", and does range checking.

Quoted 4 lines

Outside monitor context, you can now use numeric fds, and only numeric fds. Makes sense, because named fds are associated with a monitor. Note that before the patch, we crashed in monitor_get_fd() dereferencing cur_mon.

Yeah, that is fun, but I don't think there's any code path that could trigger it.