Skip to content

Generic TLS support across VNC/chardev/migration

Daniel P. Berrange qemu-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest46 patches 10 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/46sockets: add helpers for creating SocketAddress from a socketinclude/qemu/sockets.h util/qemu-sockets.c+140 -00 replies
2/46sockets: move qapi_copy_SocketAddress into qemu-sockets.cinclude/qemu/sockets.h qemu-char.c util/qemu-sockets.c+34 -250 replies
3/46sockets: allow port to be NULL when listening on IP addressutil/qemu-sockets.c+13 -50 replies
4/46osdep: add qemu_fork() wrapper for safely handling signalsinclude/qemu/osdep.h util/oslib-posix.c util/oslib-win32.c+96 -00 replies
5/46coroutine: move into libqemuutil.a libraryMakefile.objs block.c block/qcow2.h +31 more+38 -390 replies
6/46io: add abstract QIOChannel classesMAINTAINERS Makefile Makefile.objs +4 more+620 -00 replies
7/46io: add helper module for creating watches on FDsinclude/io/channel-watch.h io/Makefile.objs io/channel-watch.c+273 -00 replies
8/46io: pull Buffer code out of VNC moduleinclude/io/buffer.h io/Makefile.objs io/buffer.c +9 more+277 -1490 replies
9/46io: add QIOTask class for async operationsinclude/io/task.h io/Makefile.objs io/task.c +3 more+687 -00 replies
10/46io: add QIOChannelSocket classconfigure include/io/channel-socket.h io/Makefile.objs +7 more+1641 -00 replies
11/46io: add QIOChannelFile classinclude/io/channel-file.h io/Makefile.objs io/channel-file.c +5 more+425 -00 replies
12/46io: add QIOChannelTLS classinclude/io/channel-tls.h io/Makefile.objs io/channel-tls.c +3 more+864 -04 replies
13/46io: add QIOChannelWebsock classinclude/io/channel-websock.h io/Makefile.objs io/channel-websock.c+1074 -02 replies
14/46io: add QIOChannelCommand classinclude/io/channel-command.h io/Makefile.objs io/channel-command.c +3 more+588 -00 replies
15/46ui: convert VNC startup code to use SocketAddressui/vnc.c+91 -700 replies
16/46ui: convert VNC server to use QIOChannelSocketui/vnc-auth-sasl.c ui/vnc-auth-vencrypt.c ui/vnc-jobs.c +4 more+426 -3100 replies
17/46ui: convert VNC server to use QIOChannelTLSui/vnc-auth-sasl.c ui/vnc-auth-vencrypt.c ui/vnc-ws.c +2 more+88 -2170 replies
18/46ui: convert VNC server to use QIOChannelWebsockui/vnc-ws.c ui/vnc-ws.h ui/vnc.c +1 more+30 -3900 replies
19/46char: remove fixed length filename allocationqemu-char.c+42 -420 replies
20/46char: convert from GIOChannel to QIOChannelqemu-char.c+247 -3990 replies
21/46char: don't assume telnet initialization will not blockqemu-char.c+67 -180 replies
22/46char: introduce support for TLS encrypted TCP chardev backendqapi-schema.json qemu-char.c qemu-options.hx+138 -150 replies
23/46nbd: convert to use the QAPI SocketAddress objectblock/nbd.c+34 -320 replies
24/46qemu-nbd: convert to use the QAPI SocketAddress objectqemu-nbd.c+38 -640 replies
25/46sockets: remove use of QemuOpts from header fileinclude/qemu/sockets.h util/qemu-sockets.c+11 -200 replies
26/46sockets: remove use of QemuOpts from socket_listenutil/qemu-sockets.c+53 -530 replies
27/46sockets: remove use of QemuOpts from socket_connectutil/qemu-sockets.c+32 -500 replies
28/46sockets: remove use of QemuOpts from socket_dgraminclude/qemu/sockets.h util/qemu-sockets.c+21 -690 replies
29/46migration: remove use of qemu_bufopen from vmstate teststests/Makefile tests/test-vmstate.c+14 -322 replies
30/46migration: remove memory buffer based QEMUFile backendinclude/migration/qemu-file.h migration/Makefile.objs migration/qemu-file-buf.c+1 -4780 replies
31/46migration: move definition of struct QEMUFile back into qemu-file.cmigration/qemu-file-internal.h migration/qemu-file-unix.c migration/qemu-file.c+22 -550 replies
32/46migration: split migration hooks out of QEMUFileOpsinclude/migration/qemu-file.h migration/qemu-file.c migration/rdma.c+30 -110 replies
33/46migration: ensure qemu_fflush() always writes full data amountinclude/migration/qemu-file.h migration/qemu-file.c+16 -60 replies
34/46migration: introduce qemu_fset_blocking function on QEMUFileinclude/migration/qemu-file.h migration/migration.c migration/qemu-file.c+22 -30 replies
35/46migration: force QEMUFile to blocking mode for outgoing migrationmigration/migration.c+1 -00 replies
36/46migration: introduce a new QEMUFile impl based on QIOChannelinclude/migration/qemu-file.h migration/Makefile.objs migration/qemu-file-channel.c+205 -00 replies
37/46migration: convert unix socket protocol to use QIOChannelmigration/Makefile.objs migration/migration.c migration/unix.c+72 -390 replies
38/46migration: convert tcp socket protocol to use QIOChannelmigration/tcp.c+82 -370 replies
39/46migration: convert fd socket protocol to use QIOChannelmigration/Makefile.objs migration/fd.c migration/migration.c+39 -260 replies
40/46migration: convert exec socket protocol to use QIOChannelmigration/Makefile.objs migration/exec.c migration/migration.c+31 -240 replies
41/46migration: convert RDMA to use QIOChannel interfacemigration/rdma.c+156 -980 replies
42/46migration: convert savevm to use QIOChannel for writing to filesmigration/savevm.c tests/Makefile tests/test-vmstate.c+17 -60 replies
43/46migration: delete QEMUFile sockets implementationinclude/migration/qemu-file.h migration/Makefile.objs migration/qemu-file-unix.c+1 -2400 replies
44/46migration: delete QEMUFile stdio implementationinclude/migration/qemu-file.h migration/Makefile.objs migration/qemu-file-stdio.c+1 -1970 replies
45/46migration: support TLS encryption with TCP migration backendmigration/tcp.c qemu-options.hx+264 -162 replies
46/46migration: remove support for non-iovec based write handlersinclude/migration/qemu-file.h migration/qemu-file.c migration/savevm.c+8 -450 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [Qemu-devel] [PATCH FYI 00/46] Generic TLS support across VNC/chardev/migration

Earlier in the year I posted an RFC series providing impl of generic TLS support for VNC and chardevs.

https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg02038.html

Since that time the initial QOM enhancements and crypto API consolidation patches from that RFC have been merged. I currently have a set of patches extracting the VNC server TLS support into a generic crypto API for TLS that is under active review, hopefully getting close to merge (v6):

https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00437.html

What I'm posting here is the 46 remaining patches I have that introduce TLS support across chardevs and migration code, and in the process standardize I/O channel handling across VNC, chardevs and migration. I'm doing this so people can see the use context of the TLS patches I have on review, as well as show the kind of changes I'm making to the migration, chardev & sockets code to facilitate conversations with other devs working on the same bits of code. As compared to the original RFC patches, this series is a more complete job - there is much more unit testing added and win32 compatibility verified, and many todos/hacks cleaned up.

I still need to figure out NBD integration, which is harder than I first thought, due to the need for me to understand the special needs of the block AIO framework, which doesn't seem to use the normal APIs for registering I/O callbacks with the main event loop.

I also realize that the ACL code needs to be QOM'ified to make it possible to create/delete ACLs more generically and share then across services.

I consider pretty much all of these patches as finished and ready for review, but this series is a bit large to be reviewed in one go, so I'm going to continue drip-feed it in reasonable sized chunks.

Some of these 46 patches are fairly generic and can be easily posted in isolation for quicker merge, so I will be focusing on identifying such patches and posting them individually to appropriate maintainers.

This series needs to be applied on top of the patch series here

https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00437.html

For convience I have pushed this entire branch to github

https://github.com/berrange/qemu/commits/qemu-io-channel-18

Daniel P. Berrange (46):
sockets: add helpers for creating SocketAddress from a socket
sockets: move qapi_copy_SocketAddress into qemu-sockets.c
sockets: allow port to be NULL when listening on IP address
osdep: add qemu_fork() wrapper for safely handling signals
coroutine: move into libqemuutil.a library
io: add abstract QIOChannel classes
io: add helper module for creating watches on FDs
io: pull Buffer code out of VNC module
io: add QIOTask class for async operations
io: add QIOChannelSocket class
io: add QIOChannelFile class
io: add QIOChannelTLS class
io: add QIOChannelWebsock class
io: add QIOChannelCommand class
ui: convert VNC startup code to use SocketAddress
ui: convert VNC server to use QIOChannelSocket
ui: convert VNC server to use QIOChannelTLS
ui: convert VNC server to use QIOChannelWebsock
char: remove fixed length filename allocation
char: convert from GIOChannel to QIOChannel
char: don't assume telnet initialization will not block
char: introduce support for TLS encrypted TCP chardev backend
nbd: convert to use the QAPI SocketAddress object
qemu-nbd: convert to use the QAPI SocketAddress object
sockets: remove use of QemuOpts from header file
sockets: remove use of QemuOpts from socket_listen
sockets: remove use of QemuOpts from socket_connect
sockets: remove use of QemuOpts from socket_dgram
migration: remove use of qemu_bufopen from vmstate tests
migration: remove memory buffer based QEMUFile backend
migration: move definition of struct QEMUFile back into qemu-file.c
migration: split migration hooks out of QEMUFileOps
migration: ensure qemu_fflush() always writes full data amount
migration: introduce qemu_fset_blocking function on QEMUFile
migration: force QEMUFile to blocking mode for outgoing migration
migration: introduce a new QEMUFile impl based on QIOChannel
migration: convert unix socket protocol to use QIOChannel
migration: convert tcp socket protocol to use QIOChannel
migration: convert fd socket protocol to use QIOChannel
migration: convert exec socket protocol to use QIOChannel
migration: convert RDMA to use QIOChannel interface
migration: convert savevm to use QIOChannel for writing to files
migration: delete QEMUFile sockets implementation
migration: delete QEMUFile stdio implementation
migration: support TLS encryption with TCP migration backend
migration: remove support for non-iovec based write handlers

 MAINTAINERS                             |   7 +
 Makefile                                |   2 +
 Makefile.objs                           |  12 +-
 Makefile.target                         |   2 +
 block.c                                 |   2 +-
 block/nbd.c                             |  66 +--
 block/qcow2.h                           |   2 +-
 block/vdi.c                             |   2 +-
 block/write-threshold.c                 |   2 +-
 blockjob.c                              |   2 +-
 configure                               |  11 +
 coroutine-gthread.c                     |   2 +-
 coroutine-sigaltstack.c                 |   2 +-
 coroutine-ucontext.c                    |   2 +-
 coroutine-win32.c                       |   2 +-
 hw/9pfs/codir.c                         |   2 +-
 hw/9pfs/cofile.c                        |   2 +-
 hw/9pfs/cofs.c                          |   2 +-
 hw/9pfs/coxattr.c                       |   2 +-
 hw/9pfs/virtio-9p-coth.c                |   2 +-
 hw/9pfs/virtio-9p-coth.h                |   2 +-
 hw/9pfs/virtio-9p.h                     |   2 +-
 include/block/block.h                   |   2 +-
 include/block/block_int.h               |   2 +-
 include/io/buffer.h                     | 118 ++++
 include/io/channel-command.h            |  91 +++
 include/io/channel-file.h               |  93 +++
 include/io/channel-socket.h             | 297 ++++++++++
 include/io/channel-tls.h                | 142 +++++
 include/io/channel-watch.h              |  72 +++
 include/io/channel-websock.h            | 108 ++++
 include/io/channel.h                    | 374 +++++++++++++
 include/io/task.h                       | 256 +++++++++
 include/migration/qemu-file.h           |  48 +-
 include/{block => qemu}/coroutine.h     |   0
 include/{block => qemu}/coroutine_int.h |   2 +-
 include/qemu/osdep.h                    |  16 +
 include/qemu/sockets.h                  |  44 +-
 io/Makefile.objs                        |   9 +
 io/buffer.c                             |  65 +++
 io/channel-command.c                    | 370 ++++++++++++
 io/channel-file.c                       | 209 +++++++
 io/channel-socket.c                     | 718 ++++++++++++++++++++++++
 io/channel-tls.c                        | 381 +++++++++++++
 io/channel-watch.c                      | 200 +++++++
 io/channel-websock.c                    | 965 ++++++++++++++++++++++++++++++++
 io/channel.c                            | 229 ++++++++
 io/task.c                               | 150 +++++
 migration/Makefile.objs                 |   6 +-
 migration/exec.c                        |  48 +-
 migration/fd.c                          |  57 +-
 migration/migration.c                   |   9 +-
 migration/qemu-file-buf.c               | 462 ---------------
 migration/qemu-file-channel.c           | 200 +++++++
 migration/qemu-file-internal.h          |  53 --
 migration/qemu-file-stdio.c             | 194 -------
 migration/qemu-file-unix.c              | 238 --------
 migration/qemu-file.c                   | 113 ++--
 migration/rdma.c                        | 252 ++++++---
 migration/savevm.c                      |  16 +-
 migration/tcp.c                         | 372 ++++++++++--
 migration/unix.c                        | 103 ++--
 nbd.c                                   |   2 +-
 qapi-schema.json                        |   2 +
 qemu-char.c                             | 938 +++++++++++++++----------------
 qemu-coroutine-io.c                     |   2 +-
 qemu-coroutine-lock.c                   |   4 +-
 qemu-coroutine-sleep.c                  |   2 +-
 qemu-coroutine.c                        |   4 +-
 qemu-nbd.c                              | 102 ++--
 qemu-options.hx                         |  16 +-
 scripts/create_config                   |   9 +
 tests/.gitignore                        |   7 +
 tests/Makefile                          |  22 +-
 tests/io-channel-helpers.c              | 247 ++++++++
 tests/io-channel-helpers.h              |  33 ++
 tests/test-coroutine.c                  |   4 +-
 tests/test-io-channel-command.c         | 121 ++++
 tests/test-io-channel-file.c            |  89 +++
 tests/test-io-channel-socket.c          | 349 ++++++++++++
 tests/test-io-channel-tls.c             | 335 +++++++++++
 tests/test-io-task.c                    | 276 +++++++++
 tests/test-vmstate.c                    |  57 +-
 thread-pool.c                           |   2 +-
 ui/vnc-auth-sasl.c                      |  87 ++-
 ui/vnc-auth-vencrypt.c                  |  93 ++-
 ui/vnc-enc-tight.c                      |  38 +-
 ui/vnc-enc-zlib.c                       |   6 +-
 ui/vnc-enc-zrle.c                       |  18 +-
 ui/vnc-jobs.c                           |  25 +-
 ui/vnc-ws.c                             | 400 +++----------
 ui/vnc-ws.h                             |  71 +--
 ui/vnc.c                                | 799 ++++++++++++--------------
 ui/vnc.h                                |  77 +--
 util/oslib-posix.c                      |  71 +++
 util/oslib-win32.c                      |   9 +
 util/qemu-sockets.c                     | 427 ++++++++------
 97 files changed, 8875 insertions(+), 3085 deletions(-)
 create mode 100644 include/io/buffer.h
 create mode 100644 include/io/channel-command.h
 create mode 100644 include/io/channel-file.h
 create mode 100644 include/io/channel-socket.h
 create mode 100644 include/io/channel-tls.h
 create mode 100644 include/io/channel-watch.h
 create mode 100644 include/io/channel-websock.h
 create mode 100644 include/io/channel.h
 create mode 100644 include/io/task.h
 rename include/{block => qemu}/coroutine.h (100%)
 rename include/{block => qemu}/coroutine_int.h (98%)
 create mode 100644 io/Makefile.objs
 create mode 100644 io/buffer.c
 create mode 100644 io/channel-command.c
 create mode 100644 io/channel-file.c
 create mode 100644 io/channel-socket.c
 create mode 100644 io/channel-tls.c
 create mode 100644 io/channel-watch.c
 create mode 100644 io/channel-websock.c
 create mode 100644 io/channel.c
 create mode 100644 io/task.c
 delete mode 100644 migration/qemu-file-buf.c
 create mode 100644 migration/qemu-file-channel.c
 delete mode 100644 migration/qemu-file-internal.h
 delete mode 100644 migration/qemu-file-stdio.c
 delete mode 100644 migration/qemu-file-unix.c
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-command.c
 create mode 100644 tests/test-io-channel-file.c
 create mode 100644 tests/test-io-channel-socket.c
 create mode 100644 tests/test-io-channel-tls.c
 create mode 100644 tests/test-io-task.c
Signature · 1 line
2.4.3