RFC Generic support for TLS protocol & I/O channels
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest34 patches 36 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/34 | ui: remove check for failure of qemu_acl_init()ui/ | +0 | 1 reply | |
| 2/34 | qom: document user creatable object types in help textqemu-options.hx | +54 | 0 replies | |
| 3/34 | qom: create objects in two phasesvl.c | +28 | 0 replies | |
| 4/34 | qom: add object_new_propv / object_new_proplist constructorsinclude/ qom/ tests/ | +123 | 3 replies | |
| 5/34 | qom: make enum string tables const-correctinclude/ include/ include/ +7 more | +16 | 1 reply | |
| 6/34 | qom: add a object_property_add_enum helper methodinclude/ qom/ | +74 | 5 replies | |
| 7/34 | qom: don't pass string table to object_get_enum methodbackends/ include/ numa.c +1 more | +33 | 1 reply | |
| 8/34 | crypto: introduce new module for computing hash digestsMakefile.objs configure crypto/ +8 more | +751 | 3 replies | |
| 9/34 | crypto: move built-in AES implementation into crypto/block/ block/ block/ +8 more | +8 | 0 replies | |
| 10/34 | crypto: move built-in D3DES implementation into crypto/crypto/ crypto/ include/ +2 more | +4 | 0 replies | |
| 11/34 | crypto: introduce generic cipher API & built-in implementationcrypto/ crypto/ crypto/ +4 more | +913 | 0 replies | |
| 12/34 | crypto: add a gcrypt cipher implementationconfigure crypto/ crypto/ +1 more | +327 | 0 replies | |
| 13/34 | crypto: add a nettle cipher implementationconfigure crypto/ crypto/ | +262 | 0 replies | |
| 14/34 | crypto: introduce new module for handling TLS credentialscrypto/ crypto/ crypto/ +3 more | +741 | 0 replies | |
| 15/34 | crypto: add sanity checking of TLS credentialsconfigure crypto/ include/ +6 more | +3007 | 0 replies | |
| 16/34 | crypto: introduce new module for handling TLS sessionscrypto/ crypto/ include/ +3 more | +1407 | 0 replies | |
| 17/34 | block: convert quorum blockdrv to use crypto APIsblock/ block/ configure | +20 | 0 replies | |
| 18/34 | ui: convert VNC websockets to use crypto APIsconfigure ui/ ui/ +3 more | +26 | 0 replies | |
| 19/34 | block: convert qcow/qcow2 to use generic cipher APIblock/ block/ block/ +1 more | +162 | 0 replies | |
| 20/34 | ui: convert VNC to use generic cipher APIui/ | +41 | 0 replies | |
| 21/34 | io: add abstract QIOChannel classesMakefile.objs include/ io/ +1 more | +568 | 0 replies | |
| 22/34 | io: add helper module for creating watches on UNIX FDsinclude/ io/ io/ | +151 | 0 replies | |
| 23/34 | io: add QIOChannelSocket classinclude/ io/ io/ | +741 | 5 replies | |
| 24/34 | io: add QIOChannelFile classinclude/ io/ io/ | +266 | 0 replies | |
| 25/34 | io: add QIOTask class for async operationsinclude/ io/ io/ | +253 | 6 replies | |
| 26/34 | io: add QIOChannelTLS classinclude/ io/ io/ | +536 | 0 replies | |
| 27/34 | io: pull Buffer code out of VNC moduleinclude/ io/ io/ +9 more | +276 | 0 replies | |
| 28/34 | io: add QIOChannelWebsock classinclude/ io/ io/ | +1085 | 0 replies | |
| 29/34 | ui: convert VNC server to use QEMUIOChannelSocket classesui/ ui/ ui/ +5 more | +422 | 0 replies | |
| 30/34 | ui: convert VNC server to use QIOChannelTLSconfigure qemu-options.hx ui/ +7 more | +279 | 0 replies | |
| 31/34 | ui: convert VNC server to use QIOChannelWebsockui/ ui/ ui/ +1 more | +29 | 0 replies | |
| 32/34 | char: convert from GIOChannel to QIOChannelqemu-char.c | +232 | 0 replies | |
| 33/34 | char: don't assume telnet initialization will not blockqemu-char.c | +67 | 0 replies | |
| 34/34 | char: introduce support for TLS encrypted TCP chardev backendqapi-schema.json qemu-char.c qemu-options.hx | +161 | 10 replies |
Reviews trailers per patch
3 of 34 patches have a Reviewed-by or Acked-by.
| Patch | Paolo Bonzini | Eric Blake |
|---|---|---|
| 1/34 | R-b | |
| 2/34 | ||
| 3/34 | ||
| 4/34 | ||
| 5/34 | R-b | |
| 6/34 | R-b | |
| 7/34 | ||
| 8/34 | ||
| 9/34 | ||
| 10/34 | ||
| 11/34 | ||
| 12/34 | ||
| 13/34 | ||
| 14/34 | ||
| 15/34 | ||
| 16/34 | ||
| 17/34 | ||
| 18/34 | ||
| 19/34 | ||
| 20/34 | ||
| 21/34 | ||
| 22/34 | ||
| 23/34 | ||
| 24/34 | ||
| 25/34 | ||
| 26/34 | ||
| 27/34 | ||
| 28/34 | ||
| 29/34 | ||
| 30/34 | ||
| 31/34 | ||
| 32/34 | ||
| 33/34 | ||
| 34/34 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [Qemu-devel] [PATCH v1 RFC 00/34] Generic support for TLS protocol & I/O channels
The following series is a work-in-progress of my effort to bring TLS encryption support to all the QEMU socket based backends. The current patches just illustrate changes to the chardev backends, but future patches will cover NBD client/server and the migration client/server code. The original discussion was here:
https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg00529.html
The bulk of the interesting work here has focused on pulling our the TLS code from the VNC server and turning it into a self-contained, reusable API. This has been accompanied with a general consolidation of all cryptographic related code. So we have one place in the code base for dealing with cryptographic hash functions (md5, sha, etc), ciphers (DES, AES, etc), and protocols (TLS). The benefit of this is that the rest of the code base doesn't have to be littered with #ifdef HAVE_GNUTLS conditionals - all the integration code for gnutls is in one place.
Building on that, the next step has been to define a generic I/O channels API (inspired by GIOChannel, but improved to better suit QEMU's requirements). This provides a higher level API for dealing with POSIX sockets, and running protocols such as TLS, SASL, and WebSockets over them. This again allowed the VNC code to be further untangled, so it doesn't have to directly know about TLS or WebSockets for the most part. This will allow us to add WebSockets support to the chardev backends too at some point.
The patches series is sequenced into a number of logical groupings, with a view to allowing the patches to be incrementally merged, rather than having to take the entire series in one go.
- QOM - a handful of minor enhancements & fixes to QOM, in particular some work to make handling of enum properties clearer, and to make instantiation of objects with properties simpler.
- Crypto - introduce crypto/ and include/crypto/ directories that contain APIs for hash functions, cipher functions and the TLS credential (x509 cert) handling and TLS protocol itself. This will be further enhanced down the line as & when I get time to integrate support for LUKS.
- Hash conversion - the quorum blockdrv and VNC server are converted to use the new crypto hash functions. This removes some of the #ifdef CONFIG_* conditionals from their codepaths
- Cipher conversion - the qcow(2) blockdrv and VNC server are converted to use the new crypto cipher functions. This removes their direct dependency on QEMU's in-tree AES and DES impls. If QEMU is linked to GNUTLS, this will now transparently use either libgcrypt or nettle for the AES & DES algorithms. These impls are more actively maintained than QEMU's built-in impls and also benefit from FIPS certification in some distros. The built-in impls of course remain for those building without GNUTLS.
- I/O Channels - introduce io/ and include/io/ directories that contain APIs for dealing with arbitrary bidirectional I/O channels. These can be POSIX sockets, fifos, pipes, or higher level layers such as TLS or WebSockets. Having a common API for all these concepts greatly facilitates the integration of TLS/WebSockets into the various areas of code in QEMU that are traditionally hardcoded to directly use the POSIX sockets API.
- VNC conversion - convert the VNC server to use the new I/O channels APIs. This enables nearly all the remaining #ifdef CONFIG_* conditionals to be removed from the VNC codebase, greatly clarifying its code.
- Chardev conversion - convert the chardev backend to use the new I/O channels APIs instead of GLib's GIOChannel. This also includes support for enabling TLS on the TCP chardev backend, nicely illustrating how the I/O channels APIs simplify support for such protocols.
I've attempted to get fairly complete API documentation coverage for all the new code files I've created here, with example code usage too. In addition, the crypto modules gain a number of new test suites to validate correctness of the implementations, since this is critical code to get right.
What I see still to be done (high == merge block, medium == nice to have for merge, low == do it later):
- The code is not fully ported to the Windows platform yet. In particular the QIOChannelSocket and QIOChannelFile classes are almost certainly broken & if they compile it is just luck. High priority.
- Need to validate the reference counting / lifecycle of the QIOChannel objects in chardev & vnc servers to ensure I've not introduced any race conditions in their usage vs client disconnects. High priority.
- Unit test coverage of all the QIOChannel subclasses to validate their correct operation. Medium priority
- A QIOChannelMemory implementation that provides a memory buffer backed I/O channel. Mostly this will be used for the unit test suite, but might find other uses at some point. Medium priority.
- APIs for establishing socket connections. Currently the QIOChannelSocket class is instantiated from a pre-connected socket file descriptor. It is desirable to have a constructor that just accepts a hostname/service/family and then performs the name resolution & connection code. This will make the API more consistent to use. Medium priority.
- A QIOChannelSASL implementation that provides integration for the SASL authentication protocol. This will allow the last custom I/O layer to be removed from the VNC server code. Low priority, since we don't immediately want/need SASL support in chardev/migraton/nbd code.
- A QIOChannelTelnet implementation that runs the telnet protocol, to replace the hacky telnet support that is hardwired into the chardev backends. Low priority.
- The crypto API could usefully gain a cipher backend that uses the Linux kernel crypto API as an alternative to nettle or libgcrypt. Low priority, nice to have.
While I will probably start work on it, I'm not intending to submit the update of the NBD/migration code, until this series has been positively reviewed and looks like it is close to accepted for merge, as there are already enough patches in this series as it is :-) The aim though is to convert the NBD code to use QIOChannel instead of direct sockets usage & add the TLS protocol extension previously discussed with the NBD spec maintainer. The migration code will either be adapted to use QIOChannel, or the QEMUFile code will be adapted to use QIOChannel. Undecided which is the best approach there at this time. Probably depends whether we can do a QIOChannelRDMA impl that has performance on parity with what exists today.
For those interesting in testing I have made the series available on github too
https://github.com/berrange/qemu/tree/qemu-io-channel-7
The diffstat may look alarming but a good portion is in the test suite and there's some quite verbose comments inline too which bulk it up:
Daniel P. Berrange (34):
ui: remove check for failure of qemu_acl_init()
qom: document user creatable object types in help text
qom: create objects in two phases
qom: add object_new_propv / object_new_proplist constructors
qom: make enum string tables const-correct
qom: add a object_property_add_enum helper method
qom: don't pass string table to object_get_enum method
crypto: introduce new module for computing hash digests
crypto: move built-in AES implementation into crypto/
crypto: move built-in D3DES implementation into crypto/
crypto: introduce generic cipher API & built-in implementation
crypto: add a gcrypt cipher implementation
crypto: add a nettle cipher implementation
crypto: introduce new module for handling TLS credentials
crypto: add sanity checking of TLS credentials
crypto: introduce new module for handling TLS sessions
block: convert quorum blockdrv to use crypto APIs
ui: convert VNC websockets to use crypto APIs
block: convert qcow/qcow2 to use generic cipher API
ui: convert VNC to use generic cipher API
io: add abstract QIOChannel classes
io: add helper module for creating watches on UNIX FDs
io: add QIOChannelSocket class
io: add QIOChannelFile class
io: add QIOTask class for async operations
io: add QIOChannelTLS class
io: pull Buffer code out of VNC module
io: add QIOChannelWebsock class
ui: convert VNC server to use QEMUIOChannelSocket classes
ui: convert VNC server to use QIOChannelTLS
ui: convert VNC server to use QIOChannelWebsock
char: convert from GIOChannel to QIOChannel
char: don't assume telnet initialization will not block
char: introduce support for TLS encrypted TCP chardev backend
Makefile.objs | 1 +
backends/hostmem.c | 22 +-
block/Makefile.objs | 2 +-
block/qcow.c | 100 ++-
block/qcow2-cluster.c | 46 +-
block/qcow2.c | 95 +--
block/qcow2.h | 13 +-
block/quorum.c | 38 +-
configure | 213 ++++---
crypto/Makefile.objs | 7 +
{util => crypto}/aes.c | 2 +-
crypto/cipher-builtin.c | 391 ++++++++++++
crypto/cipher-gcrypt.c | 204 ++++++
crypto/cipher-nettle.c | 226 +++++++
crypto/cipher.c | 31 +
ui/d3des.c => crypto/desrfb.c | 2 +-
crypto/hash.c | 202 ++++++
crypto/init.c | 160 +++++
crypto/tlscreds.c | 1093 ++++++++++++++++++++++++++++++++
crypto/tlssession.c | 546 ++++++++++++++++
include/{qemu => crypto}/aes.h | 0
include/crypto/cipher.h | 205 ++++++
ui/d3des.h => include/crypto/desrfb.h | 0
include/crypto/hash.h | 189 ++++++
include/crypto/init.h | 29 +
include/crypto/tlscreds.h | 135 ++++
include/crypto/tlssession.h | 313 ++++++++++
include/hw/qdev-core.h | 2 +-
include/io/buffer.h | 118 ++++
include/io/channel-file.h | 67 ++
include/io/channel-socket.h | 168 +++++
include/io/channel-tls.h | 142 +++++
include/io/channel-unix.h | 50 ++
include/io/channel-websock.h | 108 ++++
include/io/channel.h | 388 ++++++++++++
include/io/task.h | 168 +++++
include/qapi/util.h | 2 +-
include/qapi/visitor-impl.h | 6 +-
include/qapi/visitor.h | 2 +-
include/qom/object.h | 78 ++-
io/Makefile.objs | 8 +
io/buffer.c | 65 ++
io/channel-file.c | 198 ++++++
io/channel-socket.c | 572 +++++++++++++++++
io/channel-tls.c | 393 ++++++++++++
io/channel-unix.c | 100 +++
io/channel-websock.c | 976 +++++++++++++++++++++++++++++
io/channel.c | 178 ++++++
io/task.c | 84 +++
numa.c | 1 -
qapi-schema.json | 2 +
qapi/qapi-dealloc-visitor.c | 3 +-
qapi/qapi-util.c | 2 +-
qapi/qapi-visit-core.c | 6 +-
qemu-char.c | 798 ++++++++++++------------
qemu-options.hx | 137 +++-
qom/object.c | 141 ++++-
scripts/qapi-types.py | 4 +-
target-arm/crypto_helper.c | 2 +-
target-i386/fpu_helper.c | 1 -
target-i386/ops_sse.h | 2 +-
target-ppc/int_helper.c | 2 +-
tests/.gitignore | 9 +
tests/Makefile | 16 +-
tests/crypto-tls-helpers.c | 485 +++++++++++++++
tests/crypto-tls-helpers.h | 133 ++++
tests/pkix_asn1_tab.c | 1103 +++++++++++++++++++++++++++++++++
tests/test-crypto-cipher.c | 290 +++++++++
tests/test-crypto-hash.c | 209 +++++++
tests/test-crypto-tlscreds.c | 727 ++++++++++++++++++++++
tests/test-crypto-tlssession.c | 540 ++++++++++++++++
ui/Makefile.objs | 6 +-
ui/vnc-auth-sasl.c | 81 +--
ui/vnc-auth-vencrypt.c | 90 ++-
ui/vnc-enc-tight.c | 38 +-
ui/vnc-enc-zlib.c | 6 +-
ui/vnc-enc-zrle.c | 18 +-
ui/vnc-jobs.c | 25 +-
ui/vnc-tls.c | 24 +-
ui/vnc-tls.h | 69 ---
ui/vnc-ws.c | 393 ++----------
ui/vnc-ws.h | 75 +--
ui/vnc.c | 997 ++++++++++++++---------------
ui/vnc.h | 104 ++--
util/Makefile.objs | 2 +-
vl.c | 37 +-
86 files changed, 12868 insertions(+), 1848 deletions(-)
create mode 100644 crypto/Makefile.objs
rename {util => crypto}/aes.c (99%)
create mode 100644 crypto/cipher-builtin.c
create mode 100644 crypto/cipher-gcrypt.c
create mode 100644 crypto/cipher-nettle.c
create mode 100644 crypto/cipher.c
rename ui/d3des.c => crypto/desrfb.c (99%)
create mode 100644 crypto/hash.c
create mode 100644 crypto/init.c
create mode 100644 crypto/tlscreds.c
create mode 100644 crypto/tlssession.c
rename include/{qemu => crypto}/aes.h (100%)
create mode 100644 include/crypto/cipher.h
rename ui/d3des.h => include/crypto/desrfb.h (100%)
create mode 100644 include/crypto/hash.h
create mode 100644 include/crypto/init.h
create mode 100644 include/crypto/tlscreds.h
create mode 100644 include/crypto/tlssession.h
create mode 100644 include/io/buffer.h
create mode 100644 include/io/channel-file.h
create mode 100644 include/io/channel-socket.h
create mode 100644 include/io/channel-tls.h
create mode 100644 include/io/channel-unix.h
create mode 100644 include/io/channel-websock.h
create mode 100644 include/io/channel.h
create mode 100644 include/io/task.h
create mode 100644 io/Makefile.objs
create mode 100644 io/buffer.c
create mode 100644 io/channel-file.c
create mode 100644 io/channel-socket.c
create mode 100644 io/channel-tls.c
create mode 100644 io/channel-unix.c
create mode 100644 io/channel-websock.c
create mode 100644 io/channel.c
create mode 100644 io/task.c
create mode 100644 tests/crypto-tls-helpers.c
create mode 100644 tests/crypto-tls-helpers.h
create mode 100644 tests/pkix_asn1_tab.c
create mode 100644 tests/test-crypto-cipher.c
create mode 100644 tests/test-crypto-hash.c
create mode 100644 tests/test-crypto-tlscreds.c
create mode 100644 tests/test-crypto-tlssession.c
delete mode 100644 ui/vnc-tls.hSignature · 1 line
2.1.0