Support for full disk encryption

This Work-In-Progress series introduces a framework for full disk encryption, and implements the LUKS encryption format using it. It provides a new block driver that can use this directly, but also aims to integrate it natively into the qcow2 file format (more on that later).

For convenience, this series includes of the patches I've previously sent wrt introducing a secure framework for passing secrets (passwords, keys, etc) to QEMU:

https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04365.html

Patches 1-> 15 are the basic secret handling related patches

Patches 16-25 are preparatory work in the crypto codebase. This is some bug fixing, some QAPI-ification, and some new support modules for generating initialization vectors, the PBKDF2 algorithm and LUKS anti-forensic split algorithm.

Patches 26 & 27 provide the full disk encryption framework and LUKS implementation respectively.

Finally, patch 28 provides a general purpose encrypted block driver, that can be used in any place you'd currently use dm-crypt with QEMU), and 29/30 start to extend QCow2 to support LUKS as an embedded encryption method. Patch 28 quickly illustrates the qemu-img commands for dealing with encrypted volumes.

The block/fde.c driver is a fairly clean example of how to make use of the full disk encryption framework to implement a block driver. So a good place to look. The crypto/block-luks.c file is the actual LUKS implementation. Its creation code will generate volumes with pretty much the same parameters as the reference cryptsetup impl will.

The QCow2 support is non-functional, because I hit a small problem. The QCow2 header and all header extensions appear to need to fit into the first cluster. The LUKS header is typically set to be 2 MB in size, which is clearly not going to fit. The actual QCow2 header appears to include offsets for all the various data tables, so it /appears/ that the first cluster limit for headers is just an implementation limit, not a specification limit. Assuming that's correct, I'd aim to extend the permitted size of the header to be much larger.

The main limitation of the LUKS impl is that it currently lacks support for the XTS cipher mode, so only CBC can be used. This is because I'm yet to find any attractive impl of XTS that QEMU can reuse, as either gcrypt/nettle provide it yet. TBD...

Daniel P. Berrange (30):
crypto: add QCryptoSecret object class for password/key handling
crypto: add support for loading encrypted x509 keys
qcow: add a 'keyid' parameter to qcow options
qcow2: add a 'keyid' parameter to qcow2 options
qom: add user_creatable_add & user_creatable_del methods
qemu-img: add support for --object command line arg
qemu-nbd: add support for --object command line arg
qemu-io: add support for --object command line arg
qemu-io: allow specifying image as a set of options args
qemu-nbd: allow specifying image as a set of options args
qemu-img: allow specifying image as a set of options args
block: rip out all traces of password prompting
block: remove all encryption handling APIs
block: remove support for writing to qcow/qcow2 encrypted images
qcow2: make qcow2_encrypt_sectors encrypt in place
crypto: add ability to query the cipher key, block & IV lens
crypto: add method for querying hash digest size
crypto: move QCryptoHashAlgorithm enum definition into QAPI
crypto: move QCryptoCipherAlgorithm/Mode enum definitions into QAPI
crypto: ensure qapi/crypto.json is listed in qapi-modules
crypto: add cryptographic random byte source
crypto: add support for PBKDF2 algorithm
crypto: add support for generating initialization vectors
crypto: add support for anti-forensic split algorithm
crypto: fix transposed arguments in cipher error message
crypto: add block encryption framework
crypto: implement the LUKS block encryption format
block: add generic full disk encryption driver
qcow2: convert QCow2 to use QCryptoBlock for encryption
qcow2: add LUKS full disk encryption support

 Makefile                        |    3 +-
 block.c                         |   88 +---
 block/Makefile.objs             |    2 +
 block/fde.c                     |  522 +++++++++++++++++++
 block/qapi.c                    |    2 +-
 block/qcow.c                    |  122 +++--
 block/qcow2-cluster.c           |   55 +-
 block/qcow2.c                   |  403 ++++++++++++---
 block/qcow2.h                   |   18 +-
 blockdev.c                      |   40 +-
 crypto/Makefile.objs            |   11 +
 crypto/afsplit.c                |  194 +++++++
 crypto/block-luks.c             | 1056 +++++++++++++++++++++++++++++++++++++++
 crypto/block-luks.h             |   28 ++
 crypto/block-qcowaes.c          |  150 ++++++
 crypto/block-qcowaes.h          |   28 ++
 crypto/block.c                  |  253 ++++++++++
 crypto/blockpriv.h              |   87 ++++
 crypto/cipher.c                 |   54 +-
 crypto/hash.c                   |   17 +-
 crypto/ivgen-essiv.c            |  116 +++++
 crypto/ivgen-essiv.h            |   28 ++
 crypto/ivgen-plain.c            |   60 +++
 crypto/ivgen-plain.h            |   28 ++
 crypto/ivgen-plain64.c          |   60 +++
 crypto/ivgen-plain64.h          |   28 ++
 crypto/ivgen.c                  |  117 +++++
 crypto/ivgenpriv.h              |   47 ++
 crypto/pbkdf-gcrypt.c           |   64 +++
 crypto/pbkdf-nettle.c           |   63 +++
 crypto/pbkdf-stub.c             |   39 ++
 crypto/pbkdf.c                  |   76 +++
 crypto/random.c                 |   50 ++
 crypto/secret.c                 |  567 +++++++++++++++++++++
 crypto/tlscredsx509.c           |   47 ++
 docs/specs/qcow2.txt            |   53 ++
 hmp.c                           |   42 +-
 hw/usb/dev-storage.c            |   34 --
 include/block/block.h           |    5 +-
 include/crypto/afsplit.h        |  133 +++++
 include/crypto/block.h          |  199 ++++++++
 include/crypto/cipher.h         |   54 +-
 include/crypto/hash.h           |   20 +-
 include/crypto/ivgen.h          |  167 +++++++
 include/crypto/pbkdf.h          |  152 ++++++
 include/crypto/random.h         |   43 ++
 include/crypto/secret.h         |  148 ++++++
 include/crypto/tlscredsx509.h   |    1 +
 include/monitor/monitor.h       |   10 -
 include/qemu/option.h           |    1 +
 include/qemu/osdep.h            |    2 -
 include/qom/object_interfaces.h |   31 ++
 monitor.c                       |   69 ---
 qapi/block-core.json            |   23 +-
 qapi/crypto.json                |  173 +++++++
 qemu-img-cmds.hx                |   44 +-
 qemu-img.c                      |  788 +++++++++++++++++++++++++----
 qemu-img.texi                   |    8 +
 qemu-io.c                       |  145 +++++-
 qemu-nbd.c                      |  142 +++++-
 qemu-nbd.texi                   |    7 +
 qemu-options.hx                 |   86 +++-
 qmp.c                           |   84 +---
 qom/object_interfaces.c         |   76 +++
 tests/.gitignore                |    4 +
 tests/Makefile                  |    8 +
 tests/qemu-iotests/087          |   20 +
 tests/qemu-iotests/087.out      |   26 +-
 tests/qemu-iotests/134          |   17 +-
 tests/qemu-iotests/134.out      |   44 +-
 tests/qemu-iotests/common.rc    |    4 +-
 tests/test-crypto-afsplit.c     |  176 +++++++
 tests/test-crypto-cipher.c      |   10 +
 tests/test-crypto-hash.c        |    5 +
 tests/test-crypto-ivgen.c       |  168 +++++++
 tests/test-crypto-pbkdf.c       |  378 ++++++++++++++
 tests/test-crypto-secret.c      |  446 +++++++++++++++++
 util/oslib-posix.c              |   66 ---
 util/oslib-win32.c              |   24 -
 util/qemu-option.c              |    6 +
 vl.c                            |    8 +-
 81 files changed, 7829 insertions(+), 844 deletions(-)
 create mode 100644 block/fde.c
 create mode 100644 crypto/afsplit.c
 create mode 100644 crypto/block-luks.c
 create mode 100644 crypto/block-luks.h
 create mode 100644 crypto/block-qcowaes.c
 create mode 100644 crypto/block-qcowaes.h
 create mode 100644 crypto/block.c
 create mode 100644 crypto/blockpriv.h
 create mode 100644 crypto/ivgen-essiv.c
 create mode 100644 crypto/ivgen-essiv.h
 create mode 100644 crypto/ivgen-plain.c
 create mode 100644 crypto/ivgen-plain.h
 create mode 100644 crypto/ivgen-plain64.c
 create mode 100644 crypto/ivgen-plain64.h
 create mode 100644 crypto/ivgen.c
 create mode 100644 crypto/ivgenpriv.h
 create mode 100644 crypto/pbkdf-gcrypt.c
 create mode 100644 crypto/pbkdf-nettle.c
 create mode 100644 crypto/pbkdf-stub.c
 create mode 100644 crypto/pbkdf.c
 create mode 100644 crypto/random.c
 create mode 100644 crypto/secret.c
 create mode 100644 include/crypto/afsplit.h
 create mode 100644 include/crypto/block.h
 create mode 100644 include/crypto/ivgen.h
 create mode 100644 include/crypto/pbkdf.h
 create mode 100644 include/crypto/random.h
 create mode 100644 include/crypto/secret.h
 create mode 100644 tests/test-crypto-afsplit.c
 create mode 100644 tests/test-crypto-ivgen.c
 create mode 100644 tests/test-crypto-pbkdf.c
 create mode 100644 tests/test-crypto-secret.c

2.5.0