Skip to contentSkip to conversation

Introduce I/O channels framework

PATCH In review Daniel P. Berrange 2 participants 12 messages Nov 18, 2015 – Dec 11, 2015 Kbuild

lore
Share & tools
lore
#0 lore raw

v3 cover Introduce I/O channels framework

This series of patches is a followup submission for review of another chunk of my large series supporting TLS across chardevs, VNC and migration, previously shown here:

RFC: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html
v1: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04853.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03439.html

In this series, I have provided only the general I/O channels framework, which will ultimately be used by VNC, chardev and migration code, which currently work as follows:

- VNC: uses raw sockets APIs directly, layering in TLS, SASL and websockets where needed. This has resulted in what should be fairly generic code for TLS and websockets being entangled with the rest of the VNC server code.

- Chardevs: uses GLib's GIOChannel framework. This only provides implementations for sockets and files. Its API lacks support for using struct iovec for read/writes, file descriptor passing, misc sockets ioctls/fcntls. While you can work around these problems by accessing the underling file descriptor directly, this breaks the encapsulation, requiring callers to know about specific implementations. It also does not have integration with QEMU Error reporting framework. So while the GIOChannel is extensible, extending it would result in throwing away pretty much the entire of the existing implementations

- Migration: uses QEMUFile framework. The provides an abstract interface for I/O channels used during migration. It has impls for sockets, files, memory buffers & commands. While it has struct iovec support for writes, it does not have the same for reads. It also doesn't support file descriptor passing or control of the sockets ioctls/fcntls. It also does not have any explicit event loop integration, expecting the callers to directly access the underling file desctriptor and setup events on that. This limits its utility in cases where you need channels which are not backed by file descriptors. It has no integration with QEMU Error object for error reporting, has fixed semantics wrt writes ie must always do a full write, no partial writes, and most strangely forces either input or output mode, but refuses to allow both, so no bi-directional channels!

Out of all of these, the migration code is probably closest to what is needed, but is still a good way off from being a generic framework that be can reused outside of the migration code.

There is also the GLib GIO library which provides a generic framework, but we can't depend on that due to the minimum GLib requirement. It also has various missing pieces such as file descriptor passing, and no support for struct iovec either.

Hence, this series was born, which tries to take the best of the designs for the GIOChannel, QIO and QEMUFile code to provide QIOChannel. Right from the start this new framework is explicitly isolated from any other QEMU subsystem, so its implementation will not get mixed up with specific use cases.

The QIOChannel abstract base class defines the overall API contract

- qio_channel_{write,writev,write_full} for writing data. The underling impl always uses struct iovec, but non-iov APIs are provided as simple wrappers for convenience

- qio_channel_{read,readv,read_full} for reading data. The underling impl always uses struct iovec, but non-iov APIs are provided as simple wrappers for convenience

- qio_channel_has_feature to determine which optional features the channel supports - eg file descriptor passing, nagle, etc

- qio_channel_set_{blocking,delay,cork} for various fcntl and ioctl controls

- qio_channel_{close,shutdown} for closing the I/O stream or individual channels

- qio_channel_seek for random access channels

- qio_channel_{add,create}_watch for integration into the main loop for event notifications

- qio_channel_wait for blocking of execution pending an event notification

- qio_channel_yield for switching coroutine until an event notification

All the APIs support Error ** object arguments where needed. The object is built using QOM, in order to get reference counting and sub-classing with type checking. They are *not* user creatable/visible objects though - these are internal infrastructure, so we will be free to adapt APIs/objects at will over time.

In this series there are a variety of implementations. Some of them provide an actual base layer data transport, while others provide a data translation layer:

In this series there are a variety of implementations. Some of them provide an actual base layer data transport, while others provide a data translation layer:

- QIOChannelSocket - for socket based I/O, IPv4, IPV6 & UNIX, both stream and datagram.
- QIOChannelFile - any non-socket file, plain file, char dev, fifo, pipe, etc
- QIOChannelTLS - data translation layer to apply TLS protocol
- QIOChannelWebsock - data translation layer to apply the websockets server protocol
- QIOChannelCommand - spawn & communicate with a command via pipes
- QIOChannelBuffer - a simple in-memory buffer

These 6 implementations are sufficient to support the current needs of the VNC server, chardev network backends, and all the various migration protocols, except RDMA (which is part of the larger RFC series I've posted previously).

The sockets implementation in particular solves a long standing limitation of the qemu-sockets.c API by providing a truely asynchronous API for establishing listener sockets / connections. The current code is only async for the socket establishment, but still blocks the caller for DNS resolution.

I have not attempted to make the socket implementation support multiple listener sockets in a single object. I looked at this but it would significantly complicate the QIOChannelSocket implementation. I intend to suggest an alternative approach for that problem at a later date, whereby we define a QIONetworkService object, which provides the infrastructure for managing multiple QIOChannelSocket listeners, and clients, which was resuable for any QEMU network service.

There are unit tests for the socket, file, tls, command and buffer channel implementations. I didn't create unit test for the websock impl, as it needs a websock client to usefully test it which doesn't exist in QEMU yet.

This series merely introduces all the new framework. I will be sending 2 further series immediately after this, one which converts the chardev backends, and one which converts the VNC backend. Assuming review is acceptable, I would expect to send a PULL request for this series myself, and then send the other chardev & VNC patches via their respective maintainers, hence why I split the series.

Finally I've listed myself as maintainer for the new io/ and include/io/ directory prefixes.

Changed in v3:

- Resolve merge conflicts with Eric's QAPI code which caused rename of SocketAddress struct fields.
- Added further unit test coverage of QIOChanneBuffer class
- Fixed misc memory leaks in code and unit tests

Changed in v2:

- Remove feature checks for TCP_NODELAY and TCP_CORK and treat them as hints which cannot fail (Paolo)
- Push feature checking for FD passing up into QIOChannel base class (Paolo)
- Use more constants in websockets code instead of magic values (David Gilbert)

Daniel P. Berrange (9):
io: add abstract QIOChannel classes
io: add helper module for creating watches on FDs
io: add QIOTask class for async operations
io: add QIOChannelSocket class
io: add QIOChannelFile class
io: add QIOChannelTLS class
io: add QIOChannelWebsock class
io: add QIOChannelCommand class
io: add QIOChannelBuffer class

 MAINTAINERS                     |   7 +
 Makefile                        |   2 +
 Makefile.objs                   |   5 +
 Makefile.target                 |   2 +
 configure                       |  11 +
 include/io/channel-buffer.h     |  60 +++
 include/io/channel-command.h    |  91 ++++
 include/io/channel-file.h       |  93 ++++
 include/io/channel-socket.h     | 244 ++++++++++
 include/io/channel-tls.h        | 142 ++++++
 include/io/channel-watch.h      |  72 +++
 include/io/channel-websock.h    | 108 +++++
 include/io/channel.h            | 503 +++++++++++++++++++++
 include/io/task.h               | 256 +++++++++++
 include/qemu/sockets.h          |  19 +
 io/Makefile.objs                |   9 +
 io/channel-buffer.c             | 261 +++++++++++
 io/channel-command.c            | 369 +++++++++++++++
 io/channel-file.c               | 237 ++++++++++
 io/channel-socket.c             | 753 +++++++++++++++++++++++++++++++
 io/channel-tls.c                | 405 +++++++++++++++++
 io/channel-watch.c              | 200 +++++++++
 io/channel-websock.c            | 975 ++++++++++++++++++++++++++++++++++++++++
 io/channel.c                    | 291 ++++++++++++
 io/task.c                       | 159 +++++++
 scripts/create_config           |   9 +
 tests/.gitignore                |   8 +
 tests/Makefile                  |  19 +
 tests/io-channel-helpers.c      | 246 ++++++++++
 tests/io-channel-helpers.h      |  42 ++
 tests/test-io-channel-buffer.c  |  50 +++
 tests/test-io-channel-command.c | 129 ++++++
 tests/test-io-channel-file.c    | 100 +++++
 tests/test-io-channel-socket.c  | 399 ++++++++++++++++
 tests/test-io-channel-tls.c     | 342 ++++++++++++++
 tests/test-io-task.c            | 276 ++++++++++++
 trace-events                    |  56 +++
 util/qemu-sockets.c             |   2 +-
 38 files changed, 6951 insertions(+), 1 deletion(-)
 create mode 100644 include/io/channel-buffer.h
 create mode 100644 include/io/channel-command.h
 create mode 100644 include/io/channel-file.h
 create mode 100644 include/io/channel-socket.h
 create mode 100644 include/io/channel-tls.h
 create mode 100644 include/io/channel-watch.h
 create mode 100644 include/io/channel-websock.h
 create mode 100644 include/io/channel.h
 create mode 100644 include/io/task.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/channel-buffer.c
 create mode 100644 io/channel-command.c
 create mode 100644 io/channel-file.c
 create mode 100644 io/channel-socket.c
 create mode 100644 io/channel-tls.c
 create mode 100644 io/channel-watch.c
 create mode 100644 io/channel-websock.c
 create mode 100644 io/channel.c
 create mode 100644 io/task.c
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-buffer.c
 create mode 100644 tests/test-io-channel-command.c
 create mode 100644 tests/test-io-channel-file.c
 create mode 100644 tests/test-io-channel-socket.c
 create mode 100644 tests/test-io-channel-tls.c
 create mode 100644 tests/test-io-task.c
Signature · 1 line
2.5.0
#1 lore raw

PATCH v3 1/9 io: add abstract QIOChannel classes

Start the new generic I/O channel framework by defining a QIOChannel abstract base class. This is designed to feel similar to GLib's GIOChannel, but with the addition of support for using iovecs, qemu error reporting, file descriptor passing, coroutine integration and use of the QOM framework for easier sub-classing.

The intention is that anywhere in QEMU that almost anywhere that deals with sockets will use this new I/O infrastructure, so that it becomes trivial to then layer in support for TLS encryption. This will at least include the VNC server, char device backend and migration code.

---
 MAINTAINERS          |   7 +
 Makefile             |   2 +
 Makefile.objs        |   5 +
 Makefile.target      |   2 +
 include/io/channel.h | 503 +++++++++++++++++++++++++++++++++++++++++++++++++++
 io/Makefile.objs     |   1 +
 io/channel.c         | 291 +++++++++++++++++++++++++++++
 7 files changed, 811 insertions(+)
 create mode 100644 include/io/channel.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/channel.c
MAINTAINERS +7 -0
index 9e1fa72..2f9d686 100644
@@ -1231,6 +1231,13 @@ S: Odd fixes
 F: util/buffer.c
 F: include/qemu/buffer.h
 
+I/O Channels
+M: Daniel P. Berrange <[email protected]>
+S: Maintained
+F: io/
+F: include/io/
+F: tests/test-io-*
+
 Usermode Emulation
 ------------------
 Overall
Makefile +2 -0
index c7fa427..b0049ea 100644
@@ -159,6 +159,7 @@ dummy := $(call unnest-vars,, \
                 crypto-obj-y \
                 crypto-aes-obj-y \
                 qom-obj-y \
+                io-obj-y \
                 common-obj-y \
                 common-obj-m)
 
@@ -178,6 +179,7 @@ SOFTMMU_SUBDIR_RULES=$(filter %-softmmu,$(SUBDIR_RULES))
 
 $(SOFTMMU_SUBDIR_RULES): $(block-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(crypto-obj-y)
+$(SOFTMMU_SUBDIR_RULES): $(io-obj-y)
 $(SOFTMMU_SUBDIR_RULES): config-all-devices.mak
 
 subdir-%:
Makefile.objs +5 -0
index 77be052..dac2c02 100644
@@ -28,6 +28,11 @@ crypto-aes-obj-y = crypto/
 
 qom-obj-y = qom/
 
+#######################################################################
+# io-obj-y is code used by both qemu system emulation and qemu-img
+
+io-obj-y = io/
+
 ######################################################################
 # Target independent part of system emulation. The long term path is to
 # suppress *all* target specific code in case of system emulation, i.e. a
Makefile.target +2 -0
index 962d004..34ddb7e 100644
@@ -176,6 +176,7 @@ dummy := $(call unnest-vars,.., \
                crypto-obj-y \
                crypto-aes-obj-y \
                qom-obj-y \
+               io-obj-y \
                common-obj-y \
                common-obj-m)
 target-obj-y := $(target-obj-y-save)
@@ -185,6 +186,7 @@ all-obj-y += $(qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
+all-obj-$(CONFIG_SOFTMMU) += $(io-obj-y)
 
 $(QEMU_PROG_BUILD): config-devices.mak
 
include/io/channel.h new file +503 -0

503 lines not on this page · raw message

io/Makefile.objs new file +1 -0
new file mode 100644
index 0000000..a6ed361
@@ -0,0 +1 @@
+io-obj-y = channel.o
io/channel.c new file +291 -0

291 lines not on this page · raw message

Signature · 1 line
2.5.0
#2 lore raw

PATCH v3 2/9 io: add helper module for creating watches on FDs

In reply to Daniel P. Berrange

A number of the channel implementations will require the ability to create watches on file descriptors / sockets. To avoid duplicating this code in each channel, provide a helper API for dealing with file descriptor watches.

There are two watch implementations provided. The first is useful for bi-directional file descriptors such as sockets, regular files, character devices, etc. The second works with a pair of unidirectional file descriptors such as pipes.

---
 include/io/channel-watch.h |  72 ++++++++++++++++
 io/Makefile.objs           |   1 +
 io/channel-watch.c         | 200 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 273 insertions(+)
 create mode 100644 include/io/channel-watch.h
 create mode 100644 io/channel-watch.c
include/io/channel-watch.h new file +72 -0
new file mode 100644
index 0000000..656358a
@@ -0,0 +1,72 @@
+/*
+ * QEMU I/O channels watch helper APIs
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_WATCH_H__
+#define QIO_CHANNEL_WATCH_H__
+
+#include "io/channel.h"
+
+/*
+ * This module provides helper functions that will be needed by
+ * the various QIOChannel implementations, for creating watches
+ * on file descriptors / sockets
+ */
+
+/**
+ * qio_channel_create_fd_watch:
+ * @ioc: the channel object
+ * @fd: the file descriptor
+ * @condition: the I/O condition
+ *
+ * Create a new main loop source that is able to
+ * monitor the file descriptor @fd for the
+ * I/O conditions in @condition. This is able
+ * monitor block devices, character devices,
+ * sockets, pipes but not plain files.
+ *
+ * Returns: the new main loop source
+ */
+GSource *qio_channel_create_fd_watch(QIOChannel *ioc,
+                                     int fd,
+                                     GIOCondition condition);
+
+/**
+ * qio_channel_create_fd_pair_watch:
+ * @ioc: the channel object
+ * @fdread: the file descriptor for reading
+ * @fdwrite: the file descriptor for writing
+ * @condition: the I/O condition
+ *
+ * Create a new main loop source that is able to
+ * monitor the pair of file descriptors @fdread
+ * and @fdwrite for the I/O conditions in @condition.
+ * This is intended for monitoring unidirectional
+ * file descriptors such as pipes, where a pair
+ * of descriptors is required for bidirectional
+ * I/O
+ *
+ * Returns: the new main loop source
+ */
+GSource *qio_channel_create_fd_pair_watch(QIOChannel *ioc,
+                                          int fdread,
+                                          int fdwrite,
+                                          GIOCondition condition);
+
+#endif /* QIO_CHANNEL_WATCH_H__ */
io/Makefile.objs +1 -0
index a6ed361..b02ea90 100644
@@ -1 +1,2 @@
 io-obj-y = channel.o
+io-obj-y += channel-watch.o
io/channel-watch.c new file +200 -0

200 lines not on this page · raw message

Signature · 1 line
2.5.0
#3 lore raw

PATCH v3 3/9 io: add QIOTask class for async operations

In reply to Daniel P. Berrange

A number of I/O operations need to be performed asynchronously to avoid blocking the main loop. The caller of such APIs need to provide a callback to be invoked on completion/error and need access to the error, if any. The small QIOTask provides a simple framework for dealing with such probes. The API docs inline provide an outline of how this is to be used.

Some functions don't have the ability to run asynchronously (eg getaddrinfo always blocks), so to facilitate their use, the task class provides a mechanism to run a blocking function in a thread, while triggering the completion callback in the main event loop thread. This easily allows any synchronous function to be made asynchronous, albeit at the cost of spawning a thread.

In this series, the QIOTask class will be used for things like the TLS handshake, the websockets handshake and TCP connect() progress.

The concept of QIOTask is inspired by the GAsyncResult interface / GTask class in the GIO libraries. The min version requirements on glib don't allow those to be used from QEMU, so QIOTask provides a facsimilie which can be easily switched to GTask in the future if the min version is increased.

---
 include/io/task.h    | 256 +++++++++++++++++++++++++++++++++++++++++++++++
 io/Makefile.objs     |   1 +
 io/task.c            | 159 +++++++++++++++++++++++++++++
 tests/.gitignore     |   1 +
 tests/Makefile       |   3 +
 tests/test-io-task.c | 276 +++++++++++++++++++++++++++++++++++++++++++++++++++
 trace-events         |   9 ++
 7 files changed, 705 insertions(+)
 create mode 100644 include/io/task.h
 create mode 100644 io/task.c
 create mode 100644 tests/test-io-task.c
include/io/task.h new file +256 -0
new file mode 100644
index 0000000..2418714
@@ -0,0 +1,256 @@
+/*
+ * QEMU I/O task
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_TASK_H__
+#define QIO_TASK_H__
+
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+
+typedef struct QIOTask QIOTask;
+
+typedef void (*QIOTaskFunc)(Object *source,
+                            Error *err,
+                            gpointer opaque);
+
+typedef int (*QIOTaskWorker)(QIOTask *task,
+                             Error **errp,
+                             gpointer opaque);
+
+/**
+ * QIOTask:
+ *
+ * The QIOTask object provides a simple mechanism for reporting
+ * success / failure of long running background operations.
+ *
+ * A object on which the operation is to be performed could have
+ * a public API which accepts a task callback:
+ *
+ * <example>
+ *   <title>Task callback function signature</title>
+ *   <programlisting>
+ *  void myobject_operation(QMyObject *obj,
+ *                          QIOTaskFunc *func,
+ *                          gpointer opaque,
+ *                          GDestroyNotify *notify);
+ *   </programlisting>
+ * </example>
+ *
+ * The 'func' parameter is the callback to be invoked, and 'opaque'
+ * is data to pass to it. The optional 'notify' function is used
+ * to free 'opaque' when no longer needed.
+ *
+ * Now, lets say the implementation of this method wants to set
+ * a timer to run once a second checking for completion of some
+ * activity. It would do something like
+ *
+ * <example>
+ *   <title>Task callback function implementation</title>
+ *   <programlisting>
+ *    void myobject_operation(QMyObject *obj,
+ *                            QIOTaskFunc *func,
+ *                            gpointer opaque,
+ *                            GDestroyNotify *notify)
+ *    {
+ *      QIOTask *task;
+ *
+ *      task = qio_task_new(OBJECT(obj), func, opaque, notify);
+ *
+ *      g_timeout_add_full(G_PRIORITY_DEFAULT,
+ *                         1000,
+ *                         myobject_operation_timer,
+ *                         task,
+ *                         NULL);
+ *    }
+ *   </programlisting>
+ * </example>
+ *
+ * It could equally have setup a watch on a file descriptor or
+ * created a background thread, or something else entirely.
+ * Notice that the source object is passed to the task, and
+ * QIOTask will hold a reference on that. This ensure that
+ * the QMyObject instance cannot be garbage collected while
+ * the async task is still in progress.
+ *
+ * In this case, myobject_operation_timer will fire after
+ * 3 secs and do
+ *
+ * <example>
+ *   <title>Task timer function</title>
+ *   <programlisting>
+ *   gboolean myobject_operation_timer(gpointer opaque)
+ *   {
+ *      QIOTask *task = QIO_TASK(opaque);
+ *      Error *err;*
+ *
+ *      ...check something important...
+ *       if (err) {
+ *           qio_task_abort(task, err);
+ *           error_free(task);
+ *           return FALSE;
+ *       } else if (...work is completed ...) {
+ *           qio_task_complete(task);
+ *           return FALSE;
+ *       }
+ *       ...carry on polling ...
+ *       return TRUE;
+ *   }
+ *   </programlisting>
+ * </example>
+ *
+ * Once this function returns false, object_unref will be called
+ * automatically on the task causing it to be released and the
+ * ref on QMyObject dropped too.
+ *
+ * The QIOTask module can also be used to perform operations
+ * in a background thread context, while still reporting the
+ * results in the main event thread. This allows code which
+ * cannot easily be rewritten to be asychronous (such as DNS
+ * lookups) to be easily run non-blocking. Reporting the
+ * results in the main thread context means that the caller
+ * typically does not need to be concerned about thread
+ * safety wrt the QEMU global mutex.
+ *
+ * For example, the socket_listen() method will block the caller
+ * while DNS lookups take place if given a name, instead of IP
+ * address. The C library often do not provide a practical async
+ * DNS API, so the to get non-blocking DNS lookups in a portable
+ * manner requires use of a thread. So achieve a non-blocking
+ * socket listen using QIOTask would require:
+ *
+ * <example>
+ *    static int myobject_listen_worker(QIOTask *task,
+ *                                      Error **errp,
+ *                                      gpointer opaque)
+ *    {
+ *       QMyObject obj = QMY_OBJECT(qio_task_get_source(task));
+ *       SocketAddress *addr = opaque;
+ *
+ *       obj->fd = socket_listen(addr, errp);
+ *       if (obj->fd < 0) {
+ *          return -1;
+ *       }
+ *       return 0;
+ *    }
+ *
+ *    void myobject_listen_async(QMyObject *obj,
+ *                               SocketAddress *addr,
+ *                               QIOTaskFunc *func,
+ *                               gpointer opaque,
+ *                               GDestroyNotify *notify)
+ *    {
+ *      QIOTask *task;
+ *      SocketAddress *addrCopy;
+ *
+ *      qapi_copy_SocketAddress(&addrCopy, addr);
+ *      task = qio_task_new(OBJECT(obj), func, opaque, notify);
+ *
+ *      qio_task_run_in_thread(task, myobject_listen_worker,
+ *                             addrCopy,
+ *                             qapi_free_SocketAddress);
+ *    }
+ * </example>
+ *
+ * NB, The 'func' callback passed into myobject_listen_async
+ * will be invoked from the main event thread, despite the
+ * actual operation being performed in a different thread.
+ */
+
+/**
+ * qio_task_new:
+ * @source: the object on which the operation is invoked
+ * @func: the callback to invoke when the task completes
+ * @opaque: opaque data to pass to @func when invoked
+ * @destroy: optional callback to free @opaque
+ *
+ * Creates a new task struct to track completion of a
+ * background operation running on the object @source.
+ * When the operation completes or fails, the callback
+ * @func will be invoked. The callback can access the
+ * 'err' attribute in the task object to determine if
+ * the operation was successful or not.
+ *
+ * The returned task will be released when one of
+ * qio_task_abort() or qio_task_complete() are invoked.
+ *
+ * Returns: the task struct
+ */
+QIOTask *qio_task_new(Object *source,
+                      QIOTaskFunc func,
+                      gpointer opaque,
+                      GDestroyNotify destroy);
+
+/**
+ * qio_task_run_in_thread:
+ * @task: the task struct
+ * @worker: the function to invoke in a thread
+ * @opaque: opaque data to pass to @worker
+ * @destroy: function to free @opaque
+ *
+ * Run a task in a background thread. If @worker
+ * returns 0 it will call qio_task_complete() in
+ * the main event thread context. If @worker
+ * returns -1 it will call qio_task_abort() in
+ * the main event thread context.
+ */
+void qio_task_run_in_thread(QIOTask *task,
+                            QIOTaskWorker worker,
+                            gpointer opaque,
+                            GDestroyNotify destroy);
+
+/**
+ * qio_task_complete:
+ * @task: the task struct
+ *
+ * Mark the operation as succesfully completed
+ * and free the memory for @task.
+ */
+void qio_task_complete(QIOTask *task);
+
+/**
+ * qio_task_abort:
+ * @task: the task struct
+ * @err: the error to record for the operation
+ *
+ * Mark the operation as failed, with @err providing
+ * details about the failure. The @err may be freed
+ * afer the function returns, as the notification
+ * callback is invoked synchronously. The @task will
+ * be freed when this call completes.
+ */
+void qio_task_abort(QIOTask *task,
+                    Error *err);
+
+
+/**
+ * qio_task_get_source:
+ * @task: the task struct
+ *
+ * Get the source object associated with the background
+ * task. This returns a new reference to the object,
+ * which the caller must released with object_unref()
+ * when no longer required.
+ *
+ * Returns: the source object
+ */
+Object *qio_task_get_source(QIOTask *task);
+
+#endif /* QIO_TASK_H__ */
io/Makefile.objs +1 -0
index b02ea90..503b95c 100644
@@ -1,2 +1,3 @@
 io-obj-y = channel.o
 io-obj-y += channel-watch.o
+io-obj-y += task.o
io/task.c new file +159 -0

159 lines not on this page · raw message

tests/.gitignore +1 -0
index 1e55722..eec12cc 100644
@@ -24,6 +24,7 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-task
 test-mul64
 test-opts-visitor
 test-qapi-event.[ch]
tests/Makefile +3 -0
index 90c4141..96ce8a5 100644
@@ -84,6 +84,7 @@ check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlscredsx509$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
 check-unit-$(CONFIG_LINUX) += tests/test-qga$(EXESUF)
 check-unit-y += tests/test-timed-average$(EXESUF)
+check-unit-y += tests/test-io-task$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -380,6 +381,7 @@ test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
 	$(test-qom-obj-y)
 test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
 test-block-obj-y = $(block-obj-y) $(test-crypto-obj-y)
+test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
 
 tests/check-qint$(EXESUF): tests/check-qint.o $(test-util-obj-y)
 tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y)
@@ -469,6 +471,7 @@ tests/test-crypto-tlscredsx509$(EXESUF): tests/test-crypto-tlscredsx509.o \
 tests/test-crypto-tlssession.o-cflags := $(TASN1_CFLAGS)
 tests/test-crypto-tlssession$(EXESUF): tests/test-crypto-tlssession.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o $(test-crypto-obj-y)
+tests/test-io-task$(EXESUF): tests/test-io-task.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/test-io-task.c new file +276 -0

276 lines not on this page · raw message

trace-events +9 -0
index 0b0ff02..31cf2e4 100644
@@ -1792,3 +1792,12 @@ qcrypto_tls_session_new(void *session, void *creds, const char *hostname, const
 
 # net/vhost-user.c
 vhost_user_event(const char *chr, int event) "chr: %s got event: %d"
+
+# io/task.c
+qio_task_new(void *task, void *source, void *func, void *opaque) "Task new task=%p source=%p func=%p opaque=%p"
+qio_task_complete(void *task) "Task complete task=%p"
+qio_task_abort(void *task) "Task abort task=%p"
+qio_task_thread_start(void *task, void *worker, void *opaque) "Task thread start task=%p worker=%p opaque=%p"
+qio_task_thread_run(void *task) "Task thread run task=%p"
+qio_task_thread_exit(void *task) "Task thread exit task=%p"
+qio_task_thread_result(void *task) "Task thread result task=%p"
Signature · 1 line
2.5.0
#4 lore raw

PATCH v3 4/9 io: add QIOChannelSocket class

In reply to Daniel P. Berrange

Implement a QIOChannel subclass that supports sockets I/O. The implementation is able to manage a single socket file descriptor, whether a TCP/UNIX listener, TCP/UNIX connection, or a UDP datagram. It provides APIs which can listen and connect either asynchronously or synchronously. Since there is no asynchronous DNS lookup API available, it uses the QIOTask helper for spawning a background thread to ensure non-blocking operation.

---
 configure                      |  11 +
 include/io/channel-socket.h    | 244 +++++++++++++
 include/qemu/sockets.h         |  19 ++
 io/Makefile.objs               |   1 +
 io/channel-socket.c            | 753 +++++++++++++++++++++++++++++++++++++++++
 scripts/create_config          |   9 +
 tests/.gitignore               |   1 +
 tests/Makefile                 |   3 +
 tests/io-channel-helpers.c     | 246 ++++++++++++++
 tests/io-channel-helpers.h     |  42 +++
 tests/test-io-channel-socket.c | 399 ++++++++++++++++++++++
 trace-events                   |  19 ++
 util/qemu-sockets.c            |   2 +-
 13 files changed, 1748 insertions(+), 1 deletion(-)
 create mode 100644 include/io/channel-socket.h
 create mode 100644 io/channel-socket.c
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-socket.c
configure +11 -0
index 0a4c78a..02d77f1 100755
@@ -2395,6 +2395,14 @@ fi
 
 
 ##########################################
+# getifaddrs (for tests/test-io-channel-socket )
+
+have_ifaddrs_h=yes
+if ! check_include "ifaddrs.h" ; then
+  have_ifaddrs_h=no
+fi
+
+##########################################
 # VTE probe
 
 if test "$vte" != "no"; then
@@ -5105,6 +5113,9 @@ fi
 if test "$tasn1" = "yes" ; then
   echo "CONFIG_TASN1=y" >> $config_host_mak
 fi
+if test "$have_ifaddrs_h" = "yes" ; then
+    echo "HAVE_IFADDRS_H=y" >> $config_host_mak
+fi
 if test "$vte" = "yes" ; then
   echo "CONFIG_VTE=y" >> $config_host_mak
   echo "VTE_CFLAGS=$vte_cflags" >> $config_host_mak
include/io/channel-socket.h new file +244 -0

244 lines not on this page · raw message

include/qemu/sockets.h +19 -0
index 5a183c5..74c692d 100644
@@ -89,6 +89,25 @@ int parse_host_port(struct sockaddr_in *saddr, const char *str);
 int socket_init(void);
 
 /**
+ * socket_sockaddr_to_address:
+ * @sa: socket address struct
+ * @salen: size of @sa struct
+ * @errp: pointer to uninitialized error object
+ *
+ * Get the string representation of the socket
+ * address. A pointer to the allocated address information
+ * struct will be returned, which the caller is required to
+ * release with a call qapi_free_SocketAddress when no
+ * longer required.
+ *
+ * Returns: the socket address struct, or NULL on error
+ */
+SocketAddress *
+socket_sockaddr_to_address(struct sockaddr_storage *sa,
+                           socklen_t salen,
+                           Error **errp);
+
+/**
  * socket_local_address:
  * @fd: the socket file handle
  * @errp: pointer to uninitialized error object
io/Makefile.objs +1 -0
index 503b95c..e9d77aa 100644
@@ -1,3 +1,4 @@
 io-obj-y = channel.o
+io-obj-y += channel-socket.o
 io-obj-y += channel-watch.o
 io-obj-y += task.o
io/channel-socket.c new file +753 -0

753 lines not on this page · raw message

scripts/create_config +9 -0
index 546f889..9cb176f 100755
@@ -61,6 +61,15 @@ case $line in
     value=${line#*=}
     echo "#define $name $value"
     ;;
+ HAVE_*=y) # configuration
+    name=${line%=*}
+    echo "#define $name 1"
+    ;;
+ HAVE_*=*) # configuration
+    name=${line%=*}
+    value=${line#*=}
+    echo "#define $name $value"
+    ;;
  ARCH=*) # configuration
     arch=${line#*=}
     arch_name=`echo $arch | LC_ALL=C tr '[a-z]' '[A-Z]'`
tests/.gitignore +1 -0
index eec12cc..6164cfa 100644
@@ -24,6 +24,7 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-socket
 test-io-task
 test-mul64
 test-opts-visitor
tests/Makefile +3 -0
index 96ce8a5..ed06960 100644
@@ -85,6 +85,7 @@ check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
 check-unit-$(CONFIG_LINUX) += tests/test-qga$(EXESUF)
 check-unit-y += tests/test-timed-average$(EXESUF)
 check-unit-y += tests/test-io-task$(EXESUF)
+check-unit-y += tests/test-io-channel-socket$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -472,6 +473,8 @@ tests/test-crypto-tlssession.o-cflags := $(TASN1_CFLAGS)
 tests/test-crypto-tlssession$(EXESUF): tests/test-crypto-tlssession.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o $(test-crypto-obj-y)
 tests/test-io-task$(EXESUF): tests/test-io-task.o $(test-io-obj-y)
+tests/test-io-channel-socket$(EXESUF): tests/test-io-channel-socket.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/io-channel-helpers.c new file +246 -0

246 lines not on this page · raw message

tests/io-channel-helpers.h new file +42 -0

42 lines not on this page · raw message

tests/test-io-channel-socket.c new file +399 -0

399 lines not on this page · raw message

trace-events +19 -0
index 31cf2e4..6c159b3 100644
@@ -1801,3 +1801,22 @@ qio_task_thread_start(void *task, void *worker, void *opaque) "Task thread start
 qio_task_thread_run(void *task) "Task thread run task=%p"
 qio_task_thread_exit(void *task) "Task thread exit task=%p"
 qio_task_thread_result(void *task) "Task thread result task=%p"
+
+# io/channel-socket.c
+qio_channel_socket_new(void *ioc) "Socket new ioc=%p"
+qio_channel_socket_new_fd(void *ioc, int fd) "Socket new ioc=%p fd=%d"
+qio_channel_socket_connect_sync(void *ioc, void *addr) "Socket connect sync ioc=%p addr=%p"
+qio_channel_socket_connect_async(void *ioc, void *addr) "Socket connect async ioc=%p addr=%p"
+qio_channel_socket_connect_fail(void *ioc) "Socket connect fail ioc=%p"
+qio_channel_socket_connect_complete(void *ioc, int fd) "Socket connect complete ioc=%p fd=%d"
+qio_channel_socket_listen_sync(void *ioc, void *addr) "Socket listen sync ioc=%p addr=%p"
+qio_channel_socket_listen_async(void *ioc, void *addr) "Socket listen async ioc=%p addr=%p"
+qio_channel_socket_listen_fail(void *ioc) "Socket listen fail ioc=%p"
+qio_channel_socket_listen_complete(void *ioc, int fd) "Socket listen complete ioc=%p fd=%d"
+qio_channel_socket_dgram_sync(void *ioc, void *localAddr, void *remoteAddr) "Socket dgram sync ioc=%p localAddr=%p remoteAddr=%p"
+qio_channel_socket_dgram_async(void *ioc, void *localAddr, void *remoteAddr) "Socket dgram async ioc=%p localAddr=%p remoteAddr=%p"
+qio_channel_socket_dgram_fail(void *ioc) "Socket dgram fail ioc=%p"
+qio_channel_socket_dgram_complete(void *ioc, int fd) "Socket dgram complete ioc=%p fd=%d"
+qio_channel_socket_accept(void *ioc) "Socket accept start ioc=%p"
+qio_channel_socket_accept_fail(void *ioc) "Socket accept fail ioc=%p"
+qio_channel_socket_accept_complete(void *ioc, void *cioc, int fd) "Socket accept complete ioc=%p cioc=%p fd=%d"
util/qemu-sockets.c +1 -1
index 5a31d16..922efb3 100644
@@ -1086,7 +1086,7 @@ socket_sockaddr_to_address_unix(struct sockaddr_storage *sa,
 }
 #endif /* WIN32 */
 
-static SocketAddress *
+SocketAddress *
 socket_sockaddr_to_address(struct sockaddr_storage *sa,
                            socklen_t salen,
                            Error **errp)
Signature · 1 line
2.5.0
#5 lore raw

PATCH v3 5/9 io: add QIOChannelFile class

In reply to Daniel P. Berrange

Add a QIOChannel subclass that is capable of operating on things that are files, such as plain files, pipes, character/block devices, but notably not sockets.

---
 include/io/channel-file.h    |  93 +++++++++++++++++
 io/Makefile.objs             |   1 +
 io/channel-file.c            | 237 +++++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore             |   2 +
 tests/Makefile               |   3 +
 tests/test-io-channel-file.c | 100 ++++++++++++++++++
 trace-events                 |   4 +
 7 files changed, 440 insertions(+)
 create mode 100644 include/io/channel-file.h
 create mode 100644 io/channel-file.c
 create mode 100644 tests/test-io-channel-file.c
include/io/channel-file.h new file +93 -0
new file mode 100644
index 0000000..308e6d4
@@ -0,0 +1,93 @@
+/*
+ * QEMU I/O channels files driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_FILE_H__
+#define QIO_CHANNEL_FILE_H__
+
+#include "io/channel.h"
+
+#define TYPE_QIO_CHANNEL_FILE "qio-channel-file"
+#define QIO_CHANNEL_FILE(obj)                                     \
+    OBJECT_CHECK(QIOChannelFile, (obj), TYPE_QIO_CHANNEL_FILE)
+
+typedef struct QIOChannelFile QIOChannelFile;
+
+/**
+ * QIOChannelFile:
+ *
+ * The QIOChannelFile object provides a channel implementation
+ * that is able to perform I/O on block devices, character
+ * devices, FIFOs, pipes and plain files. While it is technically
+ * able to work on sockets too on the UNIX platform, this is not
+ * portable to Windows and lacks some extra sockets specific
+ * functionality. So the QIOChannelSocket object is recommended
+ * for that use case.
+ *
+ */
+
+struct QIOChannelFile {
+    QIOChannel parent;
+    int fd;
+};
+
+
+/**
+ * qio_channel_file_new_fd:
+ * @fd: the file descriptor
+ *
+ * Create a new IO channel object for a file represented
+ * by the @fd parameter. @fd can be associated with a
+ * block device, character device, fifo, pipe, or a
+ * regular file. For sockets, the QIOChannelSocket class
+ * should be used instead, as this provides greater
+ * functionality and cross platform portability.
+ *
+ * The channel will own the passed in file descriptor
+ * and will take responsibility for closing it, so the
+ * caller must not close it. If appropriate the caller
+ * should dup() its FD before opening the channel.
+ *
+ * Returns: the new channel object
+ */
+QIOChannelFile *
+qio_channel_file_new_fd(int fd);
+
+/**
+ * qio_channel_file_new_path:
+ * @fd: the file descriptor
+ * @flags: the open flags (O_RDONLY|O_WRONLY|O_RDWR, etc)
+ * @mode: the file creation mode if O_WRONLY is set in @flags
+ * @errp: pointer to initialized error object
+ *
+ * Create a new IO channel object for a file represented
+ * by the @path parameter. @path can point to any
+ * type of file on which sequential I/O can be
+ * performed, whether it be a plain file, character
+ * device or block device.
+ *
+ * Returns: the new channel object
+ */
+QIOChannelFile *
+qio_channel_file_new_path(const char *path,
+                          int flags,
+                          mode_t mode,
+                          Error **errp);
+
+#endif /* QIO_CHANNEL_FILE_H__ */
io/Makefile.objs +1 -0
index e9d77aa..3d2f232 100644
@@ -1,4 +1,5 @@
 io-obj-y = channel.o
+io-obj-y += channel-file.o
 io-obj-y += channel-socket.o
 io-obj-y += channel-watch.o
 io-obj-y += task.o
io/channel-file.c new file +237 -0

237 lines not on this page · raw message

tests/.gitignore +2 -0
index 6164cfa..6160003 100644
@@ -24,6 +24,8 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-file
+test-io-channel-file.txt
 test-io-channel-socket
 test-io-task
 test-mul64
tests/Makefile +3 -0
index ed06960..e2b6059 100644
@@ -86,6 +86,7 @@ check-unit-$(CONFIG_LINUX) += tests/test-qga$(EXESUF)
 check-unit-y += tests/test-timed-average$(EXESUF)
 check-unit-y += tests/test-io-task$(EXESUF)
 check-unit-y += tests/test-io-channel-socket$(EXESUF)
+check-unit-y += tests/test-io-channel-file$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -475,6 +476,8 @@ tests/test-crypto-tlssession$(EXESUF): tests/test-crypto-tlssession.o \
 tests/test-io-task$(EXESUF): tests/test-io-task.o $(test-io-obj-y)
 tests/test-io-channel-socket$(EXESUF): tests/test-io-channel-socket.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-file$(EXESUF): tests/test-io-channel-file.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/test-io-channel-file.c new file +100 -0

100 lines not on this page · raw message

trace-events +4 -0
index 6c159b3..5a45c52 100644
@@ -1820,3 +1820,7 @@ qio_channel_socket_dgram_complete(void *ioc, int fd) "Socket dgram complete ioc=
 qio_channel_socket_accept(void *ioc) "Socket accept start ioc=%p"
 qio_channel_socket_accept_fail(void *ioc) "Socket accept fail ioc=%p"
 qio_channel_socket_accept_complete(void *ioc, void *cioc, int fd) "Socket accept complete ioc=%p cioc=%p fd=%d"
+
+# io/channel-file.c
+qio_channel_file_new_fd(void *ioc, int fd) "File new fd ioc=%p fd=%d"
+qio_channel_file_new_path(void *ioc, const char *path, int flags, int mode, int fd) "File new fd ioc=%p path=%s flags=%d mode=%d fd=%d"
Signature · 1 line
2.5.0
#6 lore raw

PATCH v3 6/9 io: add QIOChannelTLS class

In reply to Daniel P. Berrange

Add a QIOChannel subclass that can run the TLS protocol over the top of another QIOChannel instance. The object provides a simplified API to perform the handshake when starting the TLS session. The layering of TLS over the underlying channel does not have to be setup immediately. It is possible to take an existing QIOChannel that has done some handshake and then swap in the QIOChannelTLS layer. This allows for use with protocols which start TLS right away, and those which start plain text and then negotiate TLS.

---
 include/io/channel-tls.h    | 142 ++++++++++++++++
 io/Makefile.objs            |   1 +
 io/channel-tls.c            | 405 ++++++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore            |   1 +
 tests/Makefile              |   4 +
 tests/test-io-channel-tls.c | 342 +++++++++++++++++++++++++++++++++++++
 trace-events                |  10 ++
 7 files changed, 905 insertions(+)
 create mode 100644 include/io/channel-tls.h
 create mode 100644 io/channel-tls.c
 create mode 100644 tests/test-io-channel-tls.c
include/io/channel-tls.h new file +142 -0
new file mode 100644
index 0000000..0298b17
@@ -0,0 +1,142 @@
+/*
+ * QEMU I/O channels TLS driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_TLS_H__
+#define QIO_CHANNEL_TLS_H__
+
+#include "io/channel.h"
+#include "io/task.h"
+#include "crypto/tlssession.h"
+
+#define TYPE_QIO_CHANNEL_TLS "qio-channel-tls"
+#define QIO_CHANNEL_TLS(obj)                                     \
+    OBJECT_CHECK(QIOChannelTLS, (obj), TYPE_QIO_CHANNEL_TLS)
+
+typedef struct QIOChannelTLS QIOChannelTLS;
+
+/**
+ * QIOChannelTLS
+ *
+ * The QIOChannelTLS class provides a channel wrapper which
+ * can transparently run the TLS encryption protocol. It is
+ * usually used over a TCP socket, but there is actually no
+ * technical restriction on which type of master channel is
+ * used as the transport.
+ *
+ * This channel object is capable of running as either a
+ * TLS server or TLS client.
+ */
+
+struct QIOChannelTLS {
+    QIOChannel parent;
+    QIOChannel *master;
+    QCryptoTLSSession *session;
+};
+
+/**
+ * qio_channel_tls_new_server:
+ * @master: the underlying channel object
+ * @creds: the credentials to use for TLS handshake
+ * @aclname: the access control list for validating clients
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a new TLS channel that runs the server side of
+ * a TLS session. The TLS session handshake will use the
+ * credentials provided in @creds. If the @aclname parameter
+ * is non-NULL, then the client will have to provide
+ * credentials (ie a x509 client certificate) which will
+ * then be validated against the ACL.
+ *
+ * After creating the channel, it is mandatory to call
+ * the qio_channel_tls_handshake() method before attempting
+ * todo any I/O on the channel.
+ *
+ * Once the handshake has completed, all I/O should be done
+ * via the new TLS channel object and not the original
+ * master channel
+ *
+ * Returns: the new TLS channel object, or NULL
+ */
+QIOChannelTLS *
+qio_channel_tls_new_server(QIOChannel *master,
+                           QCryptoTLSCreds *creds,
+                           const char *aclname,
+                           Error **errp);
+
+/**
+ * qio_channel_tls_new_client:
+ * @master: the underlying channel object
+ * @creds: the credentials to use for TLS handshake
+ * @hostname: the user specified server hostname
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a new TLS channel that runs the client side of
+ * a TLS session. The TLS session handshake will use the
+ * credentials provided in @creds. The @hostname parameter
+ * should provide the user specified hostname of the server
+ * and will be validated against the server's credentials
+ * (ie CommonName of the x509 certificate)
+ *
+ * After creating the channel, it is mandatory to call
+ * the qio_channel_tls_handshake() method before attempting
+ * todo any I/O on the channel.
+ *
+ * Once the handshake has completed, all I/O should be done
+ * via the new TLS channel object and not the original
+ * master channel
+ *
+ * Returns: the new TLS channel object, or NULL
+ */
+QIOChannelTLS *
+qio_channel_tls_new_client(QIOChannel *master,
+                           QCryptoTLSCreds *creds,
+                           const char *hostname,
+                           Error **errp);
+
+/**
+ * qio_channel_tls_handshake:
+ * @ioc: the TLS channel object
+ * @func: the callback to invoke when completed
+ * @opaque: opaque data to pass to @func
+ * @destroy: optional callback to free @opaque
+ *
+ * Perform the TLS session handshake. This method
+ * will return immediately and the handshake will
+ * continue in the background, provided the main
+ * loop is running. When the handshake is complete,
+ * or fails, the @func callback will be invoked.
+ */
+void qio_channel_tls_handshake(QIOChannelTLS *ioc,
+                               QIOTaskFunc func,
+                               gpointer opaque,
+                               GDestroyNotify destroy);
+
+/**
+ * qio_channel_tls_get_session:
+ * @ioc: the TLS channel object
+ *
+ * Get the TLS session used by the channel.
+ *
+ * Returns: the TLS session
+ */
+QCryptoTLSSession *
+qio_channel_tls_get_session(QIOChannelTLS *ioc);
+
+#endif /* QIO_CHANNEL_TLS_H__ */
io/Makefile.objs +1 -0
index 3d2f232..a48011b 100644
@@ -1,5 +1,6 @@
 io-obj-y = channel.o
 io-obj-y += channel-file.o
 io-obj-y += channel-socket.o
+io-obj-y += channel-tls.o
 io-obj-y += channel-watch.o
 io-obj-y += task.o
io/channel-tls.c new file +405 -0

405 lines not on this page · raw message

tests/.gitignore +1 -0
index 6160003..810b4f0 100644
@@ -27,6 +27,7 @@ test-iov
 test-io-channel-file
 test-io-channel-file.txt
 test-io-channel-socket
+test-io-channel-tls
 test-io-task
 test-mul64
 test-opts-visitor
tests/Makefile +4 -0
index e2b6059..2919914 100644
@@ -87,6 +87,7 @@ check-unit-y += tests/test-timed-average$(EXESUF)
 check-unit-y += tests/test-io-task$(EXESUF)
 check-unit-y += tests/test-io-channel-socket$(EXESUF)
 check-unit-y += tests/test-io-channel-file$(EXESUF)
+check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -478,6 +479,9 @@ tests/test-io-channel-socket$(EXESUF): tests/test-io-channel-socket.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
 tests/test-io-channel-file$(EXESUF): tests/test-io-channel-file.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-tls$(EXESUF): tests/test-io-channel-tls.o \
+	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o \
+	tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/test-io-channel-tls.c new file +342 -0

342 lines not on this page · raw message

trace-events +10 -0
index 5a45c52..219d1f8 100644
@@ -1824,3 +1824,13 @@ qio_channel_socket_accept_complete(void *ioc, void *cioc, int fd) "Socket accept
 # io/channel-file.c
 qio_channel_file_new_fd(void *ioc, int fd) "File new fd ioc=%p fd=%d"
 qio_channel_file_new_path(void *ioc, const char *path, int flags, int mode, int fd) "File new fd ioc=%p path=%s flags=%d mode=%d fd=%d"
+
+# io/channel-tls.c
+qio_channel_tls_new_client(void *ioc, void *master, void *creds, const char *hostname) "TLS new client ioc=%p master=%p creds=%p hostname=%s"
+qio_channel_tls_new_server(void *ioc, void *master, void *creds, const char *aclname) "TLS new client ioc=%p master=%p creds=%p acltname=%s"
+qio_channel_tls_handshake_start(void *ioc) "TLS handshake start ioc=%p"
+qio_channel_tls_handshake_pending(void *ioc, int status) "TLS handshake pending ioc=%p status=%d"
+qio_channel_tls_handshake_fail(void *ioc) "TLS handshake fail ioc=%p"
+qio_channel_tls_handshake_complete(void *ioc) "TLS handshake complete ioc=%p"
+qio_channel_tls_credentials_allow(void *ioc) "TLS credentials allow ioc=%p"
+qio_channel_tls_credentials_deny(void *ioc) "TLS credentials deny ioc=%p"
Signature · 1 line
2.5.0
#7 lore raw

PATCH v3 7/9 io: add QIOChannelWebsock class

In reply to Daniel P. Berrange

Add a QIOChannel subclass that can run the websocket protocol over the top of another QIOChannel instance. This initial implementation is only capable of acting as a websockets server. There is no support for acting as a websockets client yet.

---
 include/io/channel-websock.h | 108 +++++
 io/Makefile.objs             |   1 +
 io/channel-websock.c         | 975 +++++++++++++++++++++++++++++++++++++++++++
 trace-events                 |   8 +
 4 files changed, 1092 insertions(+)
 create mode 100644 include/io/channel-websock.h
 create mode 100644 io/channel-websock.c
include/io/channel-websock.h new file +108 -0
new file mode 100644
index 0000000..0dc21cc
@@ -0,0 +1,108 @@
+/*
+ * QEMU I/O channels driver websockets
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_WEBSOCK_H__
+#define QIO_CHANNEL_WEBSOCK_H__
+
+#include "io/channel.h"
+#include "qemu/buffer.h"
+#include "io/task.h"
+
+#define TYPE_QIO_CHANNEL_WEBSOCK "qio-channel-websock"
+#define QIO_CHANNEL_WEBSOCK(obj)                                     \
+    OBJECT_CHECK(QIOChannelWebsock, (obj), TYPE_QIO_CHANNEL_WEBSOCK)
+
+typedef struct QIOChannelWebsock QIOChannelWebsock;
+typedef union QIOChannelWebsockMask QIOChannelWebsockMask;
+
+union QIOChannelWebsockMask {
+    char c[4];
+    uint32_t u;
+};
+
+/**
+ * QIOChannelWebsock
+ *
+ * The QIOChannelWebsock class provides a channel wrapper which
+ * can transparently run the HTTP websockets protocol. This is
+ * usually used over a TCP socket, but there is actually no
+ * technical restriction on which type of master channel is
+ * used as the transport.
+ *
+ * This channel object is currently only capable of running as
+ * a websocket server and is a pretty crude implementation
+ * of it, not supporting the full websockets protocol feature
+ * set. It is sufficient to use with a simple websockets
+ * client for encapsulating VNC for noVNC in-browser client.
+ */
+
+struct QIOChannelWebsock {
+    QIOChannel parent;
+    QIOChannel *master;
+    Buffer encinput;
+    Buffer encoutput;
+    Buffer rawinput;
+    Buffer rawoutput;
+    size_t payload_remain;
+    QIOChannelWebsockMask mask;
+    guint io_tag;
+    Error *io_err;
+    gboolean io_eof;
+};
+
+/**
+ * qio_channel_websock_new_server:
+ * @master: the underlying channel object
+ *
+ * Create a new websockets channel that runs the server
+ * side of the protocol.
+ *
+ * After creating the channel, it is mandatory to call
+ * the qio_channel_websock_handshake() method before attempting
+ * todo any I/O on the channel.
+ *
+ * Once the handshake has completed, all I/O should be done
+ * via the new websocket channel object and not the original
+ * master channel
+ *
+ * Returns: the new websockets channel object
+ */
+QIOChannelWebsock *
+qio_channel_websock_new_server(QIOChannel *master);
+
+/**
+ * qio_channel_websock_handshake:
+ * @ioc: the websocket channel object
+ * @func: the callback to invoke when completed
+ * @opaque: opaque data to pass to @func
+ * @destroy: optional callback to free @opaque
+ *
+ * Perform the websocket handshake. This method
+ * will return immediately and the handshake will
+ * continue in the background, provided the main
+ * loop is running. When the handshake is complete,
+ * or fails, the @func callback will be invoked.
+ */
+void qio_channel_websock_handshake(QIOChannelWebsock *ioc,
+                                   QIOTaskFunc func,
+                                   gpointer opaque,
+                                   GDestroyNotify destroy);
+
+#endif /* QIO_CHANNEL_WEBSOCK_H__ */
io/Makefile.objs +1 -0
index a48011b..e3771b1 100644
@@ -3,4 +3,5 @@ io-obj-y += channel-file.o
 io-obj-y += channel-socket.o
 io-obj-y += channel-tls.o
 io-obj-y += channel-watch.o
+io-obj-y += channel-websock.o
 io-obj-y += task.o
io/channel-websock.c new file +975 -0

975 lines not on this page · raw message

trace-events +8 -0
index 219d1f8..01a3602 100644
@@ -1834,3 +1834,11 @@ qio_channel_tls_handshake_fail(void *ioc) "TLS handshake fail ioc=%p"
 qio_channel_tls_handshake_complete(void *ioc) "TLS handshake complete ioc=%p"
 qio_channel_tls_credentials_allow(void *ioc) "TLS credentials allow ioc=%p"
 qio_channel_tls_credentials_deny(void *ioc) "TLS credentials deny ioc=%p"
+
+# io/channel-websock.c
+qio_channel_websock_new_server(void *ioc, void *master) "Websock new client ioc=%p master=%p"
+qio_channel_websock_handshake_start(void *ioc) "Websock handshake start ioc=%p"
+qio_channel_websock_handshake_pending(void *ioc, int status) "Websock handshake pending ioc=%p status=%d"
+qio_channel_websock_handshake_reply(void *ioc) "Websock handshake reply ioc=%p"
+qio_channel_websock_handshake_fail(void *ioc) "Websock handshake fail ioc=%p"
+qio_channel_websock_handshake_complete(void *ioc) "Websock handshake complete ioc=%p"
Signature · 1 line
2.5.0
#8 lore raw

PATCH v3 8/9 io: add QIOChannelCommand class

In reply to Daniel P. Berrange

Add a QIOChannel subclass that is capable of performing I/O to/from a separate process, via a pair of pipes. The command can be used for unidirectional or bi-directional I/O.

---
 include/io/channel-command.h    |  91 ++++++++++
 io/Makefile.objs                |   1 +
 io/channel-command.c            | 369 ++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore                |   2 +
 tests/Makefile                  |   3 +
 tests/test-io-channel-command.c | 129 ++++++++++++++
 trace-events                    |   6 +
 7 files changed, 601 insertions(+)
 create mode 100644 include/io/channel-command.h
 create mode 100644 io/channel-command.c
 create mode 100644 tests/test-io-channel-command.c
include/io/channel-command.h new file +91 -0
new file mode 100644
index 0000000..bd3c599
@@ -0,0 +1,91 @@
+/*
+ * QEMU I/O channels external command driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_COMMAND_H__
+#define QIO_CHANNEL_COMMAND_H__
+
+#include "io/channel.h"
+
+#define TYPE_QIO_CHANNEL_COMMAND "qio-channel-command"
+#define QIO_CHANNEL_COMMAND(obj)                                     \
+    OBJECT_CHECK(QIOChannelCommand, (obj), TYPE_QIO_CHANNEL_COMMAND)
+
+typedef struct QIOChannelCommand QIOChannelCommand;
+
+
+/**
+ * QIOChannelCommand:
+ *
+ * The QIOChannelCommand class provides a channel implementation
+ * that can transport data with an externally running command
+ * via its stdio streams.
+ */
+
+struct QIOChannelCommand {
+    QIOChannel parent;
+    int writefd;
+    int readfd;
+    pid_t pid;
+};
+
+
+/**
+ * qio_channel_command_new_pid:
+ * @writefd: the FD connected to the command's stdin
+ * @readfd: the FD connected to the command's stdout
+ * @pid: the PID of the running child command
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a channel for performing I/O with the
+ * previously spawned command identified by @pid.
+ * The two file descriptors provide the connection
+ * to command's stdio streams, either one or which
+ * may be -1 to indicate that stream is not open.
+ *
+ * The channel will take ownership of the process
+ * @pid and will kill it when closing the channel.
+ * Similarly it will take responsibility for
+ * closing the file descriptors @writefd and @readfd.
+ *
+ * Returns: the command channel object, or NULL on error
+ */
+QIOChannelCommand *
+qio_channel_command_new_pid(int writefd,
+                            int readfd,
+                            pid_t pid);
+
+/**
+ * qio_channel_command_new_spawn:
+ * @argv: the NULL terminated list of command arguments
+ * @flags: the I/O mode, one of O_RDONLY, O_WRONLY, O_RDWR
+ * @errp: pointer to an uninitialized error object
+ *
+ * Create a channel for performing I/O with the
+ * command to be spawned with arguments @argv.
+ *
+ * Returns: the command channel object, or NULL on error
+ */
+QIOChannelCommand *
+qio_channel_command_new_spawn(const char *const argv[],
+                              int flags,
+                              Error **errp);
+
+
+#endif /* QIO_CHANNEL_COMMAND_H__ */
io/Makefile.objs +1 -0
index e3771b1..1a58ccb 100644
@@ -1,4 +1,5 @@
 io-obj-y = channel.o
+io-obj-y += channel-command.o
 io-obj-y += channel-file.o
 io-obj-y += channel-socket.o
 io-obj-y += channel-tls.o
io/channel-command.c new file +369 -0

369 lines not on this page · raw message

tests/.gitignore +2 -0
index 810b4f0..cc9aeec 100644
@@ -24,6 +24,8 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-command
+test-io-channel-command.fifo
 test-io-channel-file
 test-io-channel-file.txt
 test-io-channel-socket
tests/Makefile +3 -0
index 2919914..a0faebf 100644
@@ -88,6 +88,7 @@ check-unit-y += tests/test-io-task$(EXESUF)
 check-unit-y += tests/test-io-channel-socket$(EXESUF)
 check-unit-y += tests/test-io-channel-file$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
+check-unit-y += tests/test-io-channel-command$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -482,6 +483,8 @@ tests/test-io-channel-file$(EXESUF): tests/test-io-channel-file.o \
 tests/test-io-channel-tls$(EXESUF): tests/test-io-channel-tls.o \
 	tests/crypto-tls-x509-helpers.o tests/pkix_asn1_tab.o \
 	tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-command$(EXESUF): tests/test-io-channel-command.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/test-io-channel-command.c new file +129 -0

129 lines not on this page · raw message

trace-events +6 -0
index 01a3602..4933aa6 100644
@@ -1842,3 +1842,9 @@ qio_channel_websock_handshake_pending(void *ioc, int status) "Websock handshake
 qio_channel_websock_handshake_reply(void *ioc) "Websock handshake reply ioc=%p"
 qio_channel_websock_handshake_fail(void *ioc) "Websock handshake fail ioc=%p"
 qio_channel_websock_handshake_complete(void *ioc) "Websock handshake complete ioc=%p"
+
+# io/channel-command.c
+qio_channel_command_new_pid(void *ioc, int writefd, int readfd, int pid) "Command new pid ioc=%p writefd=%d readfd=%d pid=%d"
+qio_channel_command_new_spawn(void *ioc, const char *binary, int flags) "Command new spawn ioc=%p binary=%s flags=%d"
+qio_channel_command_abort(void *ioc, int pid) "Command abort ioc=%p pid=%d"
+qio_channel_command_wait(void *ioc, int pid, int ret, int status) "Command abort ioc=%p pid=%d ret=%d status=%d"
Signature · 1 line
2.5.0
#9 lore raw

PATCH v3 9/9 io: add QIOChannelBuffer class

In reply to Daniel P. Berrange

Add a QIOChannel subclass that is capable of performing I/O to/from a memory buffer. This implementation does not attempt to support concurrent readers & writers. It is designed for serialized access where by a single thread at a time may write data, seek and then read data back out.

---
 include/io/channel-buffer.h    |  60 ++++++++++
 io/Makefile.objs               |   1 +
 io/channel-buffer.c            | 261 +++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore               |   1 +
 tests/Makefile                 |   3 +
 tests/test-io-channel-buffer.c |  50 ++++++++
 6 files changed, 376 insertions(+)
 create mode 100644 include/io/channel-buffer.h
 create mode 100644 io/channel-buffer.c
 create mode 100644 tests/test-io-channel-buffer.c
include/io/channel-buffer.h new file +60 -0
new file mode 100644
index 0000000..91a52b3
@@ -0,0 +1,60 @@
+/*
+ * QEMU I/O channels memory buffer driver
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QIO_CHANNEL_BUFFER_H__
+#define QIO_CHANNEL_BUFFER_H__
+
+#include "io/channel.h"
+
+#define TYPE_QIO_CHANNEL_BUFFER "qio-channel-buffer"
+#define QIO_CHANNEL_BUFFER(obj)                                     \
+    OBJECT_CHECK(QIOChannelBuffer, (obj), TYPE_QIO_CHANNEL_BUFFER)
+
+typedef struct QIOChannelBuffer QIOChannelBuffer;
+
+/**
+ * QIOChannelBuffer:
+ *
+ * The QIOChannelBuffer object provides a channel implementation
+ * that is able to perform I/O to/from a memory buffer.
+ *
+ */
+
+struct QIOChannelBuffer {
+    QIOChannel parent;
+    size_t capacity; /* Total allocated memory */
+    size_t usage;    /* Current size of data */
+    size_t offset;   /* Offset for future I/O ops */
+    char *data;
+};
+
+
+/**
+ * qio_channel_buffer_new:
+ * @capacity: the initial buffer capacity to allocate
+ *
+ * Allocate a new buffer which is initially empty
+ *
+ * Returns: the new channel object
+ */
+QIOChannelBuffer *
+qio_channel_buffer_new(size_t capacity);
+
+#endif /* QIO_CHANNEL_BUFFER_H__ */
io/Makefile.objs +1 -0
index 1a58ccb..0e3de31 100644
@@ -1,4 +1,5 @@
 io-obj-y = channel.o
+io-obj-y += channel-buffer.o
 io-obj-y += channel-command.o
 io-obj-y += channel-file.o
 io-obj-y += channel-socket.o
io/channel-buffer.c new file +261 -0

261 lines not on this page · raw message

tests/.gitignore +1 -0
index cc9aeec..77aaba6 100644
@@ -24,6 +24,7 @@ test-cutils
 test-hbitmap
 test-int128
 test-iov
+test-io-channel-buffer
 test-io-channel-command
 test-io-channel-command.fifo
 test-io-channel-file
tests/Makefile +3 -0
index a0faebf..94bf1eb 100644
@@ -89,6 +89,7 @@ check-unit-y += tests/test-io-channel-socket$(EXESUF)
 check-unit-y += tests/test-io-channel-file$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
 check-unit-y += tests/test-io-channel-command$(EXESUF)
+check-unit-y += tests/test-io-channel-buffer$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -485,6 +486,8 @@ tests/test-io-channel-tls$(EXESUF): tests/test-io-channel-tls.o \
 	tests/io-channel-helpers.o $(test-io-obj-y)
 tests/test-io-channel-command$(EXESUF): tests/test-io-channel-command.o \
         tests/io-channel-helpers.o $(test-io-obj-y)
+tests/test-io-channel-buffer$(EXESUF): tests/test-io-channel-buffer.o \
+        tests/io-channel-helpers.o $(test-io-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/test-io-channel-buffer.c new file +50 -0

50 lines not on this page · raw message

Signature · 1 line
2.5.0
#10 lore raw
In reply to Daniel P. Berrange

Ping, unless anyone wants to review this further I'll send a pull request for it once 2.6 opens up, so I can focus on getting the other dependant patch series reviewed & merged in 2.6 too.

Regards,
Daniel

Quoted 240 lines · Daniel P. Berrange

This series of patches is a followup submission for review of another chunk of my large series supporting TLS across chardevs, VNC and migration, previously shown here:

RFC: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html
v1: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04853.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03439.html

In this series, I have provided only the general I/O channels framework, which will ultimately be used by VNC, chardev and migration code, which currently work as follows:

- VNC: uses raw sockets APIs directly, layering in TLS, SASL and websockets where needed. This has resulted in what should be fairly generic code for TLS and websockets being entangled with the rest of the VNC server code.

- Chardevs: uses GLib's GIOChannel framework. This only provides implementations for sockets and files. Its API lacks support for using struct iovec for read/writes, file descriptor passing, misc sockets ioctls/fcntls. While you can work around these problems by accessing the underling file descriptor directly, this breaks the encapsulation, requiring callers to know about specific implementations. It also does not have integration with QEMU Error reporting framework. So while the GIOChannel is extensible, extending it would result in throwing away pretty much the entire of the existing implementations

- Migration: uses QEMUFile framework. The provides an abstract interface for I/O channels used during migration. It has impls for sockets, files, memory buffers & commands. While it has struct iovec support for writes, it does not have the same for reads. It also doesn't support file descriptor passing or control of the sockets ioctls/fcntls. It also does not have any explicit event loop integration, expecting the callers to directly access the underling file desctriptor and setup events on that. This limits its utility in cases where you need channels which are not backed by file descriptors. It has no integration with QEMU Error object for error reporting, has fixed semantics wrt writes ie must always do a full write, no partial writes, and most strangely forces either input or output mode, but refuses to allow both, so no bi-directional channels!

Out of all of these, the migration code is probably closest to what is needed, but is still a good way off from being a generic framework that be can reused outside of the migration code.

There is also the GLib GIO library which provides a generic framework, but we can't depend on that due to the minimum GLib requirement. It also has various missing pieces such as file descriptor passing, and no support for struct iovec either.

Hence, this series was born, which tries to take the best of the designs for the GIOChannel, QIO and QEMUFile code to provide QIOChannel. Right from the start this new framework is explicitly isolated from any other QEMU subsystem, so its implementation will not get mixed up with specific use cases.

The QIOChannel abstract base class defines the overall API contract

- qio_channel_{write,writev,write_full} for writing data. The underling impl always uses struct iovec, but non-iov APIs are provided as simple wrappers for convenience

- qio_channel_{read,readv,read_full} for reading data. The underling impl always uses struct iovec, but non-iov APIs are provided as simple wrappers for convenience

- qio_channel_has_feature to determine which optional features the channel supports - eg file descriptor passing, nagle, etc

- qio_channel_set_{blocking,delay,cork} for various fcntl and ioctl controls

- qio_channel_{close,shutdown} for closing the I/O stream or individual channels

- qio_channel_seek for random access channels

- qio_channel_{add,create}_watch for integration into the main loop for event notifications

- qio_channel_wait for blocking of execution pending an event notification

- qio_channel_yield for switching coroutine until an event notification

All the APIs support Error ** object arguments where needed. The object is built using QOM, in order to get reference counting and sub-classing with type checking. They are *not* user creatable/visible objects though - these are internal infrastructure, so we will be free to adapt APIs/objects at will over time.

In this series there are a variety of implementations. Some of them provide an actual base layer data transport, while others provide a data translation layer:

In this series there are a variety of implementations. Some of them provide an actual base layer data transport, while others provide a data translation layer:

- QIOChannelSocket - for socket based I/O, IPv4, IPV6 & UNIX, both stream and datagram.
- QIOChannelFile - any non-socket file, plain file, char dev, fifo, pipe, etc
- QIOChannelTLS - data translation layer to apply TLS protocol
- QIOChannelWebsock - data translation layer to apply the websockets server protocol
- QIOChannelCommand - spawn & communicate with a command via pipes
- QIOChannelBuffer - a simple in-memory buffer

These 6 implementations are sufficient to support the current needs of the VNC server, chardev network backends, and all the various migration protocols, except RDMA (which is part of the larger RFC series I've posted previously).

The sockets implementation in particular solves a long standing limitation of the qemu-sockets.c API by providing a truely asynchronous API for establishing listener sockets / connections. The current code is only async for the socket establishment, but still blocks the caller for DNS resolution.

I have not attempted to make the socket implementation support multiple listener sockets in a single object. I looked at this but it would significantly complicate the QIOChannelSocket implementation. I intend to suggest an alternative approach for that problem at a later date, whereby we define a QIONetworkService object, which provides the infrastructure for managing multiple QIOChannelSocket listeners, and clients, which was resuable for any QEMU network service.

There are unit tests for the socket, file, tls, command and buffer channel implementations. I didn't create unit test for the websock impl, as it needs a websock client to usefully test it which doesn't exist in QEMU yet.

This series merely introduces all the new framework. I will be sending 2 further series immediately after this, one which converts the chardev backends, and one which converts the VNC backend. Assuming review is acceptable, I would expect to send a PULL request for this series myself, and then send the other chardev & VNC patches via their respective maintainers, hence why I split the series.

Finally I've listed myself as maintainer for the new io/ and include/io/ directory prefixes.

Changed in v3:

- Resolve merge conflicts with Eric's QAPI code which caused rename of SocketAddress struct fields.
- Added further unit test coverage of QIOChanneBuffer class
- Fixed misc memory leaks in code and unit tests

Changed in v2:

- Remove feature checks for TCP_NODELAY and TCP_CORK and treat them as hints which cannot fail (Paolo)
- Push feature checking for FD passing up into QIOChannel base class (Paolo)
- Use more constants in websockets code instead of magic values (David Gilbert)

Daniel P. Berrange (9):
io: add abstract QIOChannel classes
io: add helper module for creating watches on FDs
io: add QIOTask class for async operations
io: add QIOChannelSocket class
io: add QIOChannelFile class
io: add QIOChannelTLS class
io: add QIOChannelWebsock class
io: add QIOChannelCommand class
io: add QIOChannelBuffer class

 MAINTAINERS                     |   7 +
 Makefile                        |   2 +
 Makefile.objs                   |   5 +
 Makefile.target                 |   2 +
 configure                       |  11 +
 include/io/channel-buffer.h     |  60 +++
 include/io/channel-command.h    |  91 ++++
 include/io/channel-file.h       |  93 ++++
 include/io/channel-socket.h     | 244 ++++++++++
 include/io/channel-tls.h        | 142 ++++++
 include/io/channel-watch.h      |  72 +++
 include/io/channel-websock.h    | 108 +++++
 include/io/channel.h            | 503 +++++++++++++++++++++
 include/io/task.h               | 256 +++++++++++
 include/qemu/sockets.h          |  19 +
 io/Makefile.objs                |   9 +
 io/channel-buffer.c             | 261 +++++++++++
 io/channel-command.c            | 369 +++++++++++++++
 io/channel-file.c               | 237 ++++++++++
 io/channel-socket.c             | 753 +++++++++++++++++++++++++++++++
 io/channel-tls.c                | 405 +++++++++++++++++
 io/channel-watch.c              | 200 +++++++++
 io/channel-websock.c            | 975 ++++++++++++++++++++++++++++++++++++++++
 io/channel.c                    | 291 ++++++++++++
 io/task.c                       | 159 +++++++
 scripts/create_config           |   9 +
 tests/.gitignore                |   8 +
 tests/Makefile                  |  19 +
 tests/io-channel-helpers.c      | 246 ++++++++++
 tests/io-channel-helpers.h      |  42 ++
 tests/test-io-channel-buffer.c  |  50 +++
 tests/test-io-channel-command.c | 129 ++++++
 tests/test-io-channel-file.c    | 100 +++++
 tests/test-io-channel-socket.c  | 399 ++++++++++++++++
 tests/test-io-channel-tls.c     | 342 ++++++++++++++
 tests/test-io-task.c            | 276 ++++++++++++
 trace-events                    |  56 +++
 util/qemu-sockets.c             |   2 +-
 38 files changed, 6951 insertions(+), 1 deletion(-)
 create mode 100644 include/io/channel-buffer.h
 create mode 100644 include/io/channel-command.h
 create mode 100644 include/io/channel-file.h
 create mode 100644 include/io/channel-socket.h
 create mode 100644 include/io/channel-tls.h
 create mode 100644 include/io/channel-watch.h
 create mode 100644 include/io/channel-websock.h
 create mode 100644 include/io/channel.h
 create mode 100644 include/io/task.h
 create mode 100644 io/Makefile.objs
 create mode 100644 io/channel-buffer.c
 create mode 100644 io/channel-command.c
 create mode 100644 io/channel-file.c
 create mode 100644 io/channel-socket.c
 create mode 100644 io/channel-tls.c
 create mode 100644 io/channel-watch.c
 create mode 100644 io/channel-websock.c
 create mode 100644 io/channel.c
 create mode 100644 io/task.c
Quoted 12 lines
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-buffer.c
 create mode 100644 tests/test-io-channel-command.c
 create mode 100644 tests/test-io-channel-file.c
 create mode 100644 tests/test-io-channel-socket.c
 create mode 100644 tests/test-io-channel-tls.c
 create mode 100644 tests/test-io-task.c
Signature · 1 line
2.5.0

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#11 lore raw
Quoted 3 lines · Daniel P. Berrange

Ping, unless anyone wants to review this further I'll send a pull request for it once 2.6 opens up, so I can focus on getting the other dependant patch series reviewed & merged in 2.6 too.

Yes, please do.

Paolo