Support LUKS encryption in block devices
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest26 patches 31 repliesIn review
- v326 patches 0 repliesSuperseded
- v217 patches 51 repliesSuperseded
Earlier versions · 1
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v1 | 15 patches | 16 replies | Superseded |
Changes since v2 from the cover letter
- Added support for XTS cipher usage mode
- Added support for serpent, twofish & cast5 ciphers
- Use qemu/osdep.h where applicable (Eric)
- Use MAX macro where applicable (Eric)
- Use bool instead of gboolean (Eric)
- Use bytes instead of sectors in API to future proof for 4k sector sizes (Eric)
- Add asserts for LUKS structs matching spec size (Eric)
- Switch to XTS+plain64 instead of CBC+essiv for new LUKS volumes
- Add qemu I/O test 145 for dm-crypt interoperability
- Rearrange vol creation so that we treat size as the logical guest volume size, instead of physical container size (Fam).
- Push malloc of bounce buffer out of loop in LUKS block driver (Eric)
- Many misc typos/docs fixes (Fam, Eric)
Patches in v4 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/26 | crypto: add cryptographic random byte sourceMakefile.objs crypto/ crypto/ +3 more | +156 | 0 replies | |
| 2/26 | crypto: add support for PBKDF2 algorithmcrypto/ crypto/ crypto/ +6 more | +788 | 2 replies | |
| 3/26 | crypto: add support for generating initialization vectorscrypto/ crypto/ crypto/ +11 more | +869 | 3 replies | |
| 4/26 | crypto: add support for anti-forensic split algorithmcrypto/ crypto/ include/ +3 more | +487 | 4 replies | |
| 5/26 | crypto: skip testing of unsupported cipher algorithmstests/ | +3 | 2 replies | |
| 6/26 | crypto: add support for the cast5-128 cipher algorithmcrypto/ crypto/ crypto/ +2 more | +58 | 2 replies | |
| 7/26 | crypto: add support for the serpent cipher algorithmcrypto/ crypto/ crypto/ +2 more | +99 | 2 replies | |
| 8/26 | crypto: add support for the twofish cipher algorithmcrypto/ crypto/ crypto/ +2 more | +82 | 2 replies | |
| 9/26 | crypto: import an implementation of the XTS cipher modecrypto/ crypto/ include/ +3 more | +769 | 2 replies | |
| 10/26 | crypto: refactor code for dealing with AES ciphercrypto/ crypto/ | +109 | 1 reply | |
| 11/26 | crypto: wire up XTS mode for cipher APIscrypto/ crypto/ crypto/ +3 more | +405 | 1 reply | |
| 12/26 | crypto: add block encryption frameworkcrypto/ crypto/ crypto/ +7 more | +1094 | 1 reply | |
| 13/26 | crypto: implement the LUKS block encryption formatcrypto/ crypto/ crypto/ +3 more | +1486 | 2 replies | |
| 14/26 | block: add flag to indicate that no I/O will be performedblock.c block/ include/ +1 more | +28 | 0 replies | |
| 15/26 | qemu-img/qemu-io: don't prompt for passwords if not requiredqemu-img.c qemu-io.c | +3 | 0 replies | |
| 16/26 | tests: redirect stderr to stdout for ioteststests/ | +5 | 1 reply | |
| 17/26 | tests: refactor python I/O tests helper main methodtests/ | +12 | 1 reply | |
| 18/26 | tests: add output filter to python I/O tests helpertests/ | +24 | 2 replies | |
| 19/26 | block: add generic full disk encryption driverblock/ block/ qapi/ +4 more | +3382 | 2 replies | |
| 20/26 | qcow2: make qcow2_encrypt_sectors encrypt in placeblock/ block/ block/ | +8 | 0 replies | |
| 21/26 | qcow2: convert QCow2 to use QCryptoBlock for encryptionblock/ block/ block/ +11 more | +788 | 0 replies | |
| 22/26 | qcow: make encrypt_sectors encrypt in placeblock/ | +10 | 0 replies | |
| 23/26 | qcow: convert QCow to use QCryptoBlock for encryptionblock/ qapi/ | +96 | 0 replies | |
| 24/26 | block: rip out all traces of password promptinghmp.c hw/ include/ +8 more | +3 | 1 reply | |
| 25/26 | block: remove all encryption handling APIsblock.c block/ block/ +5 more | +5 | 0 replies | |
| 26/26 | block: remove support for legecy AES qcow/qcow2 encryptionblock.c block/ block/ +4 more | +23 | 0 replies |
Reviews trailers per patch
20 of 26 patches have a Reviewed-by or Acked-by.
| Patch | Fam Zheng | Eric Blake |
|---|---|---|
| 1/26 | R-b | R-b |
| 2/26 | R-b | R-b |
| 3/26 | R-b | R-b |
| 4/26 | R-b | R-b |
| 5/26 | R-b | R-b |
| 6/26 | R-b | R-b |
| 7/26 | R-b | R-b |
| 8/26 | R-b | R-b |
| 9/26 | R-b | |
| 10/26 | R-b | |
| 11/26 | R-b | |
| 12/26 | R-b | |
| 13/26 | R-b | |
| 14/26 | R-b | |
| 15/26 | R-b | |
| 16/26 | R-b | |
| 17/26 | R-b | |
| 18/26 | R-b | |
| 19/26 | R-b | |
| 20/26 | R-b | R-b |
| 21/26 | ||
| 22/26 | ||
| 23/26 | ||
| 24/26 | ||
| 25/26 | ||
| 26/26 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [Qemu-devel] [PATCH v4 00/26] Support LUKS encryption in block devices
This series was previously submitted here:
v1: https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg04748.html
v2: https://lists.gnu.org/archive/html/qemu-block/2016-01/msg00534.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg03176.html
This patch series applies as is to master, since all the dependent block tools code is now merged.
I'd like to try to get the crypto patches + the generic luks block driver for QEMU 2.6, but leave the qcow2 enhancements for 2.7
As can be guessed from the subject, the primary goal of this patch series is to support LUKS encryption in the QEMU block layer. QEMU has increasingly been adding native clients for network block protocols (RBD, gluster, NFS, iSCSI, etc) and apps like OpenStack are embracing them as it is much easier to deal with this from a management POV than to deal with the kernel block layer & userspace tools. Unfortunately when using QEMU native clients, apps are locked out of using dm-crypt and LUKS which is undesirable.
This series introduces two new features to the block layer. First there is a general purpose 'luks' format driver which can be layered over any other existing block driver. eg it can be layed above RBD, iSCSI, etc. Second the qcow2 file format is extended so that its embedded encryption can be replaced with the LUKS data format. While you could just layer the general purpose luks driver over qcow2, this is slightly less desirable, as it removes the ability to reliably auto-detect that LUKS is used by QEMU, as opposed to used by the guest OS. Having use of LUKS encoded in the qcow2 header addresses this.
The code is designed such that there is a strict separation between the full disk encryption format and the block I/O layer. Thus there is an generic API for dealing with full disk encryption added to the crypto/ subsystem. The block layer merely calls this FDE API when required, which serves to minimize the code present in the already complex block layer.
The big change in this posting is the addition of an integration test in tests/qemu-iotests/145. This uses 'sudo cryptsetup' to create LUKS volumes on the host kernel and perform I/O to/from them. This is validated against QEMU's implementation using qemu-io. Conversely we also create LUKS volumes wth qemu-img and then validate that the kernel can use them. This proves good interoperability between the two implementations across a wide range of crypto algorithms.
The big thing that I have *not* addressed is Fam's idea about modifying the generic block layer so that it can automatically instantitate a LUKS block driver above the qcow2 driver, if qcow2 indicates that it needs encryption support. I'm still not 100% sure this is the right way to go, but I've not had time to investigate, since the focus since last posting has been on dm-crypt interoperability for the generic 'luks' block driver.
The first 11 patches add some supporting APIs and algorithms to the crypto subsystem.
The 12-13 patches introduce the general full disk encryption API to the crypto subsystem and LUKS implementation
Patches 14-18 do a little preparation to the block code for future patches.
Patch 19 addes the new 'luks' block driver format along with a test for interoperability with dm-crypt/cryptsetup
Patches 20-23 convert qcow & qcow2 to the new FDE APIs, enabling LUKS in qcow2 (but not qcow) at the same time.
Patches 24-25 clean up the horrible password handling cruft in the block layer and monitor.
Patch 26 blocks use of the legacy qcow[2] encryption from the system emulators.
Chagnes since v3:
- Rebased to resolve conflicts with recently merged code
- Add qemu/osdep.h include to all new files
- Fix unit test for new XTS default mode
Changes since v2:
- Added support for XTS cipher usage mode
- Added support for serpent, twofish & cast5 ciphers
- Use qemu/osdep.h where applicable (Eric)
- Use MAX macro where applicable (Eric)
- Use bool instead of gboolean (Eric)
- Use bytes instead of sectors in API to future proof for 4k sector sizes (Eric)
- Add asserts for LUKS structs matching spec size (Eric)
- Switch to XTS+plain64 instead of CBC+essiv for new LUKS volumes
- Add qemu I/O test 145 for dm-crypt interoperability
- Rearrange vol creation so that we treat size as the logical guest volume size, instead of physical container size (Fam).
- Push malloc of bounce buffer out of loop in LUKS block driver (Eric)
- Many misc typos/docs fixes (Fam, Eric)
Changes since v1:
- Unit testing coverage of the full disk encryption APIs
- Functional testing of LUKS driver via the qemu-iotests for the block layer
- Use GNUTLS random API for the random byte source
- Use Makefile.objs for conditional compilation of pbkdf files (Fam)
- Use 'key-secret' instead of 'key-id' as property for decryption key (Paolo)
- Rename 'fde' to 'crypto' in block code (Kevin)
- Fix accounting of encryption header clusters when checking refcounts (Kevin)
- Rename format 'qcowaes' to 'qcow' (Eric)
- Fix qapi syntax for marking optional parameters (Eric)
- Add assertions in I/O path for BDRV_O_NO_IO flag
- Cleanup return codes / error reporting (Kevin)
- Other misc fixes identified in testing
Changes since WIP:
- QAPI parameters defined for the encryption key ID
- The qcow2 integration of LUKS is working, using extra allocated clusters to store LUKS header instead of trying to expand the main qcow2 header region to > 1 cluster
- qemu-img info now works without prompting for decryption password
- Unit testing of pbkdf2, afsplit and ivgen APis.
Daniel P. Berrange (26):
crypto: add cryptographic random byte source
crypto: add support for PBKDF2 algorithm
crypto: add support for generating initialization vectors
crypto: add support for anti-forensic split algorithm
crypto: skip testing of unsupported cipher algorithms
crypto: add support for the cast5-128 cipher algorithm
crypto: add support for the serpent cipher algorithm
crypto: add support for the twofish cipher algorithm
crypto: import an implementation of the XTS cipher mode
crypto: refactor code for dealing with AES cipher
crypto: wire up XTS mode for cipher APIs
crypto: add block encryption framework
crypto: implement the LUKS block encryption format
block: add flag to indicate that no I/O will be performed qemu-img/qemu-io: don't prompt for passwords if not required
tests: redirect stderr to stdout for iotests
tests: refactor python I/O tests helper main method
tests: add output filter to python I/O tests helper
block: add generic full disk encryption driver
qcow2: make qcow2_encrypt_sectors encrypt in place
qcow2: convert QCow2 to use QCryptoBlock for encryption
qcow: make encrypt_sectors encrypt in place
qcow: convert QCow to use QCryptoBlock for encryption
block: rip out all traces of password prompting
block: remove all encryption handling APIs
block: remove support for legecy AES qcow/qcow2 encryption
Makefile.objs | 2 +- block.c | 99 +- block/Makefile.objs | 2 + block/crypto.c | 578 +++++++++++ block/io.c | 2 + block/qapi.c | 2 +- block/qcow.c | 204 ++-- block/qcow2-cluster.c | 53 +- block/qcow2-refcount.c | 10 + block/qcow2.c | 519 ++++++++-- block/qcow2.h | 24 +- blockdev.c | 41 +- crypto/Makefile.objs | 17 + crypto/afsplit.c | 158 +++ crypto/block-luks.c | 1313 ++++++++++++++++++++++++ crypto/block-luks.h | 28 + crypto/block-qcow.c | 173 ++++ crypto/block-qcow.h | 28 + crypto/block.c | 260 +++++ crypto/blockpriv.h | 92 ++ crypto/cipher-builtin.c | 207 +++- crypto/cipher-gcrypt.c | 171 +++- crypto/cipher-nettle.c | 197 +++- crypto/cipher.c | 41 +- crypto/ivgen-essiv.c | 118 +++ crypto/ivgen-essiv.h | 28 + crypto/ivgen-plain.c | 59 ++ crypto/ivgen-plain.h | 28 + crypto/ivgen-plain64.c | 59 ++ crypto/ivgen-plain64.h | 28 + crypto/ivgen.c | 99 ++ crypto/ivgenpriv.h | 49 + crypto/pbkdf-gcrypt.c | 68 ++ crypto/pbkdf-nettle.c | 65 ++ crypto/pbkdf-stub.c | 42 + crypto/pbkdf.c | 77 ++ crypto/random-gcrypt.c | 33 + crypto/random-gnutls.c | 43 + crypto/random-stub.c | 31 + crypto/xts.c | 256 +++++ docs/specs/qcow2.txt | 76 ++ hmp.c | 31 - hw/usb/dev-storage.c | 34 - include/block/block.h | 6 +- include/block/block_int.h | 1 - include/crypto/afsplit.h | 135 +++ include/crypto/block.h | 232 +++++ include/crypto/ivgen.h | 206 ++++ include/crypto/pbkdf.h | 152 +++ include/crypto/random.h | 44 + include/crypto/xts.h | 86 ++ include/monitor/monitor.h | 7 - include/qemu/osdep.h | 2 - monitor.c | 68 -- qapi/block-core.json | 46 +- qapi/crypto.json | 145 ++- qemu-img.c | 66 +- qemu-io.c | 21 - qmp.c | 10 +- tests/.gitignore | 5 + tests/Makefile | 10 + tests/qemu-iotests/049 | 2 +- tests/qemu-iotests/049.out | 10 +- tests/qemu-iotests/082.out | 189 ++++ tests/qemu-iotests/087 | 30 +- tests/qemu-iotests/087.out | 28 +- tests/qemu-iotests/134 | 18 +- tests/qemu-iotests/134.out | 33 +- tests/qemu-iotests/146 | 521 ++++++++++ tests/qemu-iotests/146.out | 2252 +++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/common | 8 + tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 48 +- tests/test-crypto-afsplit.c | 190 ++++ tests/test-crypto-block.c | 334 ++++++ tests/test-crypto-cipher.c | 215 +++- tests/test-crypto-ivgen.c | 169 ++++ tests/test-crypto-pbkdf.c | 377 +++++++ tests/test-crypto-xts.c | 423 ++++++++ util/oslib-posix.c | 66 -- util/oslib-win32.c | 24 - 81 files changed, 10745 insertions(+), 880 deletions(-) create mode 100644 block/crypto.c create mode 100644 crypto/afsplit.c create mode 100644 crypto/block-luks.c create mode 100644 crypto/block-luks.h create mode 100644 crypto/block-qcow.c create mode 100644 crypto/block-qcow.h create mode 100644 crypto/block.c create mode 100644 crypto/blockpriv.h create mode 100644 crypto/ivgen-essiv.c create mode 100644 crypto/ivgen-essiv.h create mode 100644 crypto/ivgen-plain.c create mode 100644 crypto/ivgen-plain.h create mode 100644 crypto/ivgen-plain64.c create mode 100644 crypto/ivgen-plain64.h create mode 100644 crypto/ivgen.c create mode 100644 crypto/ivgenpriv.h create mode 100644 crypto/pbkdf-gcrypt.c create mode 100644 crypto/pbkdf-nettle.c create mode 100644 crypto/pbkdf-stub.c create mode 100644 crypto/pbkdf.c create mode 100644 crypto/random-gcrypt.c create mode 100644 crypto/random-gnutls.c create mode 100644 crypto/random-stub.c create mode 100644 crypto/xts.c create mode 100644 include/crypto/afsplit.h create mode 100644 include/crypto/block.h create mode 100644 include/crypto/ivgen.h create mode 100644 include/crypto/pbkdf.h create mode 100644 include/crypto/random.h create mode 100644 include/crypto/xts.h create mode 100755 tests/qemu-iotests/146 create mode 100644 tests/qemu-iotests/146.out create mode 100644 tests/test-crypto-afsplit.c create mode 100644 tests/test-crypto-block.c create mode 100644 tests/test-crypto-ivgen.c create mode 100644 tests/test-crypto-pbkdf.c create mode 100644 tests/test-crypto-xts.c
Signature · 1 line
2.5.0