Skip to contentSkip to conversation

RFC crypto/luks: encryption key managment using amend interface

PATCH In review Maxim Levitsky 6 participants 52 messages Sep 12, 2019 – Nov 12, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover RFC crypto/luks: encryption key managment using amend interface

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Best regards,
Maxim Levitsky

Maxim Levitsky (11):
qcrypto: add suport for amend options
qcrypto-luks: extend the create options for upcoming encryption key
management
qcrypto-luks: implement the encryption key management
block: amend: add 'force' option
block/crypto: implement the encryption key management
qcow2: implement crypto amend options
block: add x-blockdev-amend qmp command
block/crypto: implement blockdev-amend
block/qcow2: implement blockdev-amend
iotests: filter few more luks specific create options iotests : add tests for encryption key management

 block.c                          |   4 +-
 block/Makefile.objs              |   2 +-
 block/amend.c                    | 116 ++++++++++
 block/crypto.c                   | 167 +++++++++++++-
 block/crypto.h                   |  16 ++
 block/qcow2.c                    | 151 ++++++++++--
 crypto/block-luks.c              | 382 ++++++++++++++++++++++++++++++-
 crypto/block.c                   |  31 +++
 crypto/blockpriv.h               |   8 +
 include/block/block.h            |   1 +
 include/block/block_int.h        |  22 +-
 include/crypto/block.h           |  22 ++
 qapi/block-core.json             |  39 +++-
 qapi/crypto.json                 |  19 ++
 qapi/job.json                    |   4 +-
 qemu-img-cmds.hx                 |   4 +-
 qemu-img.c                       |   8 +-
 qemu-img.texi                    |   6 +-
 tests/qemu-iotests/082.out       |  54 +++++
 tests/qemu-iotests/087.out       |   6 +-
 tests/qemu-iotests/134.out       |   2 +-
 tests/qemu-iotests/158.out       |   4 +-
 tests/qemu-iotests/188.out       |   2 +-
 tests/qemu-iotests/189.out       |   4 +-
 tests/qemu-iotests/198.out       |   4 +-
 tests/qemu-iotests/300           | 202 ++++++++++++++++
 tests/qemu-iotests/300.out       |  98 ++++++++
 tests/qemu-iotests/301           |  90 ++++++++
 tests/qemu-iotests/301.out       |  30 +++
 tests/qemu-iotests/302           | 252 ++++++++++++++++++++
 tests/qemu-iotests/302.out       |  18 ++
 tests/qemu-iotests/303           | 228 ++++++++++++++++++
 tests/qemu-iotests/303.out       |  28 +++
 tests/qemu-iotests/common.filter |   6 +-
 tests/qemu-iotests/group         |   9 +
 35 files changed, 1986 insertions(+), 53 deletions(-)
 create mode 100644 block/amend.c
 create mode 100755 tests/qemu-iotests/300
 create mode 100644 tests/qemu-iotests/300.out
 create mode 100755 tests/qemu-iotests/301
 create mode 100644 tests/qemu-iotests/301.out
 create mode 100644 tests/qemu-iotests/302
 create mode 100644 tests/qemu-iotests/302.out
 create mode 100644 tests/qemu-iotests/303
 create mode 100644 tests/qemu-iotests/303.out
Signature · 1 line
2.17.2
#1 lore raw

PATCH v2 1/11 qcrypto: add suport for amend options

This adds the qcrypto_amend_options and corresponding crypto driver callbacks for the for encrypted key managedment

---
 crypto/block.c         | 31 +++++++++++++++++++++++++++++++
 crypto/blockpriv.h     |  8 ++++++++
 include/crypto/block.h | 22 ++++++++++++++++++++++
 3 files changed, 61 insertions(+)
crypto/block.c +31 -0
index 325752871c..14b684de7f 100644
@@ -115,6 +115,37 @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
 }
 
 
+int qcrypto_block_amend_options(QCryptoBlock *block,
+                                QCryptoBlockReadFunc readfunc,
+                                QCryptoBlockWriteFunc writefunc,
+                                void *opaque,
+                                QCryptoBlockCreateOptions *options,
+                                bool force,
+                                Error **errp)
+{
+    if (options->format != block->format) {
+        error_setg(errp,
+                   "Its not possible to change encryption format with amend interface");
+        return -1;
+    }
+
+    if (!block->driver->amend) {
+        error_setg(errp,
+                   "Crypto format %s doesn't support format options amendment",
+                   QCryptoBlockFormat_str(block->format));
+        return -1;
+    }
+
+    return block->driver->amend(block,
+                                readfunc,
+                                writefunc,
+                                opaque,
+                                options,
+                                force,
+                                errp);
+}
+
+
 QCryptoBlockInfo *qcrypto_block_get_info(QCryptoBlock *block,
                                          Error **errp)
 {
crypto/blockpriv.h +8 -0
index 71c59cb542..c18a4e0b43 100644
@@ -62,6 +62,14 @@ struct QCryptoBlockDriver {
                   void *opaque,
                   Error **errp);
 
+    int (*amend)(QCryptoBlock *block,
+                 QCryptoBlockReadFunc readfunc,
+                 QCryptoBlockWriteFunc writefunc,
+                 void *opaque,
+                 QCryptoBlockCreateOptions *options,
+                 bool force,
+                 Error **errp);
+
     int (*get_info)(QCryptoBlock *block,
                     QCryptoBlockInfo *info,
                     Error **errp);
include/crypto/block.h +22 -0
index d49d2c2da9..777fd51ebe 100644
@@ -144,6 +144,28 @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
                                    void *opaque,
                                    Error **errp);
 
+/**
+ * qcrypto_block_amend_options:
+ * @block: the block encryption object
+ *
+ * @readfunc: callback for reading data from the volume header
+ * @writefunc: callback for writing data to the volume header
+ * @opaque: data to pass to @readfunc and @writefunc
+ * @options: the new/amended encryption options
+ * @force: hint for the driver to allow unsafe operation
+ * @errp: error pointer
+ *
+ * Changes the crypto options of the encryption format
+ *
+ */
+int qcrypto_block_amend_options(QCryptoBlock *block,
+                                QCryptoBlockReadFunc readfunc,
+                                QCryptoBlockWriteFunc writefunc,
+                                void *opaque,
+                                QCryptoBlockCreateOptions *options,
+                                bool force,
+                                Error **errp);
+
 
 /**
  * qcrypto_block_get_info:
Signature · 1 line
2.17.2
#2 lore raw

Re: [PATCH v2 01/11] qcrypto: add suport for amend options

Quoted 2 lines · Maxim Levitsky

This adds the qcrypto_amend_options and corresponding crypto driver callbacks for the for encrypted

grammar is off, did you miss a word where that double space is?

Quoted 1 line

key managedment

management

Quoted 19 lines
---
 crypto/block.c         | 31 +++++++++++++++++++++++++++++++
 crypto/blockpriv.h     |  8 ++++++++
 include/crypto/block.h | 22 ++++++++++++++++++++++
 3 files changed, 61 insertions(+)
crypto/block.c +2 -0
index 325752871c..14b684de7f 100644
@@ -115,6 +115,37 @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
 }
 
 
+int qcrypto_block_amend_options(QCryptoBlock *block,
+                                QCryptoBlockReadFunc readfunc,

crypto/block.c QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options, · line 129

Quoted 10 lines
crypto/block.c +10 -0
+                                QCryptoBlockWriteFunc writefunc,
+                                void *opaque,
+                                QCryptoBlockCreateOptions *options,
+                                bool force,
+                                Error **errp)
+{
+    if (options->format != block->format) {
+        error_setg(errp,
+                   "Its not possible to change encryption format with amend interface");
+        return -1;

"It's" (here, you want the form meaning "It is")

Or reword the entire error to something shorter:

error_setg(errp, "cannot amend encryption format")

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#3 lore raw

Re: [PATCH v2 01/11] qcrypto: add suport for amend options

Quoted 9 lines · Eric Blake
Quoted 2 lines · Maxim Levitsky

This adds the qcrypto_amend_options and corresponding crypto driver callbacks for the for encrypted

grammar is off, did you miss a word where that double space is?

Quoted 1 line

key managedment

management

Thank you!
I'll try my best in the future to have less spelling and grammar errors like that. I need to double check every message prior to sending the patches.

Quoted 33 lines
Quoted 20 lines
---
 crypto/block.c         | 31 +++++++++++++++++++++++++++++++
 crypto/blockpriv.h     |  8 ++++++++
 include/crypto/block.h | 22 ++++++++++++++++++++++
 3 files changed, 61 insertions(+)
crypto/block.c +2 -0
index 325752871c..14b684de7f 100644
@@ -115,6 +115,37 @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
 }
 
 
+int qcrypto_block_amend_options(QCryptoBlock *block,
+                                QCryptoBlockReadFunc readfunc,

crypto/block.c QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options, · line 129

Quoted 10 lines
crypto/block.c +10 -0
+                                QCryptoBlockWriteFunc writefunc,
+                                void *opaque,
+                                QCryptoBlockCreateOptions *options,
+                                bool force,
+                                Error **errp)
+{
+    if (options->format != block->format) {
+        error_setg(errp,
+                   "Its not possible to change encryption format with amend interface");
+        return -1;

"It's" (here, you want the form meaning "It is")

Quoted 3 lines

Or reword the entire error to something shorter:

error_setg(errp, "cannot amend encryption format")

Same here.

Quoted 1 line

Thanks for the review,
Best regards,
Maxim Levitsky

#4 lore raw

PATCH v2 2/11 qcrypto-luks: extend the create options for upcoming encryption key management

In reply to Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

---
 block/crypto.c             |  2 ++
 block/crypto.h             | 16 +++++++++++
 block/qcow2.c              |  2 ++
 crypto/block-luks.c        | 26 +++++++++++++++---
 qapi/crypto.json           | 19 ++++++++++++++
 tests/qemu-iotests/082.out | 54 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 115 insertions(+), 4 deletions(-)
block/crypto.c +2 -0
index 6e822c6e50..a6a3e1f1d8 100644
@@ -144,6 +144,8 @@ static QemuOptsList block_crypto_create_opts_luks = {
         BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG(""),
         BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG(""),
         BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME(""),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_SLOT(""),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_ACTIVE(""),
         { /* end of list */ }
     },
 };
block/crypto.h +16 -0
index b935695e79..05cc43d9bc 100644
@@ -35,12 +35,14 @@
         "ID of the secret that provides the AES encryption key")
 
 #define BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET "key-secret"
+#define BLOCK_CRYPTO_OPT_LUKS_SLOT "slot"
 #define BLOCK_CRYPTO_OPT_LUKS_CIPHER_ALG "cipher-alg"
 #define BLOCK_CRYPTO_OPT_LUKS_CIPHER_MODE "cipher-mode"
 #define BLOCK_CRYPTO_OPT_LUKS_IVGEN_ALG "ivgen-alg"
 #define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg"
 #define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg"
 #define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time"
+#define BLOCK_CRYPTO_OPT_LUKS_ACTIVE "active"
 
 #define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(prefix)                    \
     BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix,                             \
@@ -88,6 +90,20 @@
         .help = "Time to spend in PBKDF in milliseconds",     \
     }
 
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_SLOT(prefix)           \
+    {                                                         \
+        .name = prefix BLOCK_CRYPTO_OPT_LUKS_SLOT,       \
+        .type = QEMU_OPT_NUMBER,                              \
+        .help = "Controls the slot where the secret is added/erased",     \
+    }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_ACTIVE(prefix)           \
+    {                                                         \
+        .name = prefix BLOCK_CRYPTO_OPT_LUKS_ACTIVE,       \
+        .type = QEMU_OPT_BOOL,                              \
+        .help = "Controls if the added secret is added or erased",     \
+    }
+
 QCryptoBlockCreateOptions *
 block_crypto_create_opts_init(QDict *opts, Error **errp);
 
block/qcow2.c +2 -0
index 0882ff6e92..5bdb8b18f4 100644
@@ -5166,6 +5166,8 @@ static QemuOptsList qcow2_create_opts = {
         BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),
         BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),
         BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_SLOT("encrypt."),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_ACTIVE("encrypt."),
         {
             .name = BLOCK_OPT_CLUSTER_SIZE,
             .type = QEMU_OPT_SIZE,
crypto/block-luks.c +22 -4
index 6c53bdc428..fed80e6646 100644
@@ -1211,6 +1211,7 @@ qcrypto_block_luks_create(QCryptoBlock *block,
     const char *hash_alg;
     g_autofree char *cipher_mode_spec = NULL;
     uint64_t iters;
+    unsigned int slot_idx = 0;
 
     memcpy(&luks_opts, &options->u.luks, sizeof(luks_opts));
     if (!luks_opts.has_iter_time) {
@@ -1244,12 +1245,30 @@ qcrypto_block_luks_create(QCryptoBlock *block,
     luks->ivgen_hash_alg = luks_opts.ivgen_hash_alg;
     luks->hash_alg = luks_opts.hash_alg;
 
+    if (luks_opts.has_active && !luks_opts.active) {
+        error_setg(errp,
+                   "For image creation, the added secret must be active!");
+        goto error;
+
+    }
+
+    if (luks_opts.has_slot) {
+        if (luks_opts.slot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS ||
+            luks_opts.slot < 0) {
+                error_setg(errp,
+                           "Invalid slot %" PRId64 " is specified",
+                           luks_opts.slot);
+                goto error;
+        }
+        slot_idx = (unsigned int)luks_opts.slot;
+    }
+
 
     /* Note we're allowing ivgen_hash_alg to be set even for
      * non-essiv iv generators that don't need a hash. It will
      * be silently ignored, for compatibility with dm-crypt */
 
-    if (!options->u.luks.key_secret) {
+    if (!luks_opts.has_key_secret) {
         error_setg(errp, "Parameter '%skey-secret' is required for cipher",
                    optprefix ? optprefix : "");
         goto error;
@@ -1455,11 +1474,10 @@ qcrypto_block_luks_create(QCryptoBlock *block,
         goto error;
     }
 
-
-    /* populate the slot 0 with the password encrypted master key*/
+    /* populate one of the slots with the password encrypted master key*/
     /* This will also store the header */
     if (qcrypto_block_luks_store_key(block,
-                                     0,
+                                     slot_idx,
                                      password,
                                      masterkey,
                                      luks_opts.iter_time,
qapi/crypto.json +19 -0
index b2a4cff683..9b83a70634 100644
@@ -190,6 +190,20 @@
 #                  Currently defaults to 'sha256'
 # @hash-alg: the master key hash algorithm
 #            Currently defaults to 'sha256'
+#
+# @active: Should the new secret be added (true) or erased (false)
+#          (amend only, since 4.2)
+#
+# @slot: The slot in which to put/erase the secret
+#        if not given, will select first free slot for secret addtion
+#        and erase all matching keyslots for erase. except last one
+#        (optional, since 4.2)
+#
+# @unlock-secret: The secret to use to unlock the image
+#        If not given, will use the secret that was used
+#        when opening the image.
+#        (optional, for amend only, since 4.2)
+#
 # @iter-time: number of milliseconds to spend in
 #             PBKDF passphrase processing. Currently defaults
 #             to 2000. (since 2.8)
@@ -201,7 +215,12 @@
             '*cipher-mode': 'QCryptoCipherMode',
             '*ivgen-alg': 'QCryptoIVGenAlgorithm',
             '*ivgen-hash-alg': 'QCryptoHashAlgorithm',
+
             '*hash-alg': 'QCryptoHashAlgorithm',
+            '*active' : 'bool',
+            '*slot': 'int',
+            '*unlock-secret': 'str',
+
             '*iter-time': 'int'}}
 
 
tests/qemu-iotests/082.out +54 -0

378 lines not on this page · raw message

Signature · 1 line
2.17.2
#5 lore raw

Re: [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

Quoted 6 lines · Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

Quoted 8 lines
---
 block/crypto.c             |  2 ++
 block/crypto.h             | 16 +++++++++++
 block/qcow2.c              |  2 ++
 crypto/block-luks.c        | 26 +++++++++++++++---
 qapi/crypto.json           | 19 ++++++++++++++
 tests/qemu-iotests/082.out | 54 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 115 insertions(+), 4 deletions(-)

(Just doing a cursory RFC-style review)

I think we also want to reject unlock-secret if it’s given for creation; and I suppose it’d be more important to print which slots are OK than the slot the user has given. (It isn’t like we shouldn’t print that slot index, but it’s more likely the user knows that than what the limits are. I think.)

Max

#6 lore raw

Re: [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

Quoted 16 lines · Max Reitz
Quoted 6 lines · Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

Quoted 8 lines
---
 block/crypto.c             |  2 ++
 block/crypto.h             | 16 +++++++++++
 block/qcow2.c              |  2 ++
 crypto/block-luks.c        | 26 +++++++++++++++---
 qapi/crypto.json           | 19 ++++++++++++++
 tests/qemu-iotests/082.out | 54 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 115 insertions(+), 4 deletions(-)
Quoted 3 lines

(Just doing a cursory RFC-style review)

I think we also want to reject unlock-secret if it’s given for creation;

Agree, I'll do this in the next version.

Quoted 4 lines

and I suppose it’d be more important to print which slots are OK than the slot the user has given. (It isn’t like we shouldn’t print that slot index, but it’s more likely the user knows that than what the limits are. I think.)

I don't really understand what you mean here :-(

Since this is qmp interface,
I can't really print anything from it, other that error messages.

Quoted 3 lines

Max

Best regards,
Maxim Levitsky

#7 lore raw

Re: [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

Quoted 22 lines · Maxim Levitsky
Quoted 16 lines · Max Reitz
Quoted 6 lines · Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

Quoted 8 lines
---
 block/crypto.c             |  2 ++
 block/crypto.h             | 16 +++++++++++
 block/qcow2.c              |  2 ++
 crypto/block-luks.c        | 26 +++++++++++++++---
 qapi/crypto.json           | 19 ++++++++++++++
 tests/qemu-iotests/082.out | 54 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 115 insertions(+), 4 deletions(-)
Quoted 3 lines

(Just doing a cursory RFC-style review)

I think we also want to reject unlock-secret if it’s given for creation;

Agree, I'll do this in the next version.

Quoted 8 lines
Quoted 4 lines

and I suppose it’d be more important to print which slots are OK than the slot the user has given. (It isn’t like we shouldn’t print that slot index, but it’s more likely the user knows that than what the limits are. I think.)

I don't really understand what you mean here :-(

Since this is qmp interface,
I can't really print anything from it, other that error messages.

Exactly, I’m referring to the error message. Right now it’s:

"Invalid slot %" PRId64 " is specified", luks_opts.slot

I think it should be something like:

"Invalid slot %" PRId64 " specified, must be between 0 and %u", luks_opt.slot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1

Max

#8 lore raw

Re: [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

Quoted 34 lines · Max Reitz
Quoted 28 lines · Maxim Levitsky
Quoted 16 lines · Max Reitz
Quoted 6 lines · Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

Quoted 8 lines
---
 block/crypto.c             |  2 ++
 block/crypto.h             | 16 +++++++++++
 block/qcow2.c              |  2 ++
 crypto/block-luks.c        | 26 +++++++++++++++---
 qapi/crypto.json           | 19 ++++++++++++++
 tests/qemu-iotests/082.out | 54 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 115 insertions(+), 4 deletions(-)
Quoted 3 lines

(Just doing a cursory RFC-style review)

I think we also want to reject unlock-secret if it’s given for creation;

Agree, I'll do this in the next version.

Quoted 4 lines

and I suppose it’d be more important to print which slots are OK than the slot the user has given. (It isn’t like we shouldn’t print that slot index, but it’s more likely the user knows that than what the limits are. I think.)

Quoted 4 lines

I don't really understand what you mean here :-(

Since this is qmp interface,
I can't really print anything from it, other that error messages.

Quoted 8 lines

Exactly, I’m referring to the error message. Right now it’s:

"Invalid slot %" PRId64 " is specified", luks_opts.slot

I think it should be something like:

"Invalid slot %" PRId64 " specified, must be between 0 and %u", luks_opt.slot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1

This is a very good idea! implemented now and will post in the next version.

Best regards,
Maxim Levitsky

#9 lore raw

Re: [Qemu-devel] [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

In reply to Maxim Levitsky

Quick QAPI schema review only.

Quoted 6 lines · Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

[...]

qapi/crypto.json

Quoted 5 lines
qapi/crypto.json +0 -0
index b2a4cff683..9b83a70634 100644
@@ -190,6 +190,20 @@

##
# @QCryptoBlockCreateOptionsLUKS:
#
# The options that apply to LUKS encryption format initialization #
# @cipher-alg: the cipher algorithm for data encryption
# Currently defaults to 'aes-256'.
# @cipher-mode: the cipher mode for data encryption
# Currently defaults to 'xts'
# @ivgen-alg: the initialization vector generator
# Currently defaults to 'plain64'
# @ivgen-hash-alg: the initialization vector generator hash

qapi/crypto.json line 195

Quoted 6 lines
qapi/crypto.json +3 -0
 #                  Currently defaults to 'sha256'
 # @hash-alg: the master key hash algorithm
 #            Currently defaults to 'sha256'
+#
+# @active: Should the new secret be added (true) or erased (false)
+#          (amend only, since 4.2)

Is "active" established terminology? I wouldn't have guessed its meaning from its name...

As far as I can see, QCryptoBlockCreateOptionsLUKS is used just for blockdev-create with options.driver \in { luks, qcow, qcow2 }:

{ 'command': 'blockdev-create',

     'data': { ...
               'options': 'BlockdevCreateOptions' } }
   { 'union': 'BlockdevCreateOptions',
     ...
     'data': {
         ...
         'luks':           'BlockdevCreateOptionsLUKS',
         ...
         'qcow':           'BlockdevCreateOptionsQcow',
         'qcow2':          'BlockdevCreateOptionsQcow2',
         ... } }

With luks:

{ 'struct': 'BlockdevCreateOptionsLUKS',

     'base': 'QCryptoBlockCreateOptionsLUKS',
     ... }

With qcow and qcow2:

    { 'struct': 'BlockdevCreateOptionsQcow',
      'data': { ...
                '*encrypt':         'QCryptoBlockCreateOptions' } }
    { 'struct': 'BlockdevCreateOptionsQcow2',
      'data': { ...
                '*encrypt':         'QCryptoBlockCreateOptions',
                ... } }

    { 'union': 'QCryptoBlockCreateOptions',
      'base': 'QCryptoBlockOptionsBase',
      'discriminator': 'format',
      'data': { ...
                'luks': 'QCryptoBlockCreateOptionsLUKS' } }

I think I understand why we want blockdev-create to be able to specify a new secret.

Why do we want it to be able to delete an existing secret? How would that even work? Color me confused...

qapi/crypto.json line 200

Quoted 5 lines
qapi/crypto.json +5 -0
+#
+# @slot: The slot in which to put/erase the secret
+#        if not given, will select first free slot for secret addtion
+#        and erase all matching keyslots for erase. except last one
+#        (optional, since 4.2)

Excuse my possibly ignorant question: what exactly is a "matching keyslot"?

qapi/crypto.json line 205

Quoted 5 lines
qapi/crypto.json +5 -0
+#
+# @unlock-secret: The secret to use to unlock the image
+#        If not given, will use the secret that was used
+#        when opening the image.
+#        (optional, for amend only, since 4.2)

More ignorance: what is "amend"? No mention of it in qapi/*json...

Quoted 17 lines
qapi/crypto.json +6 -0
+#
 # @iter-time: number of milliseconds to spend in
 #             PBKDF passphrase processing. Currently defaults
 #             to 2000. (since 2.8)
@@ -201,7 +215,12 @@
             '*cipher-mode': 'QCryptoCipherMode',
             '*ivgen-alg': 'QCryptoIVGenAlgorithm',
             '*ivgen-hash-alg': 'QCryptoHashAlgorithm',
+
             '*hash-alg': 'QCryptoHashAlgorithm',
+            '*active' : 'bool',
+            '*slot': 'int',
+            '*unlock-secret': 'str',
+
             '*iter-time': 'int'}}
 
 

[...]

#10 lore raw

Re: [Qemu-devel] [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

Quoted 30 lines · Markus Armbruster

Quick QAPI schema review only.

Quoted 6 lines · Maxim Levitsky

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

[...]

qapi/crypto.json

Quoted 5 lines
qapi/crypto.json +0 -0
index b2a4cff683..9b83a70634 100644
@@ -190,6 +190,20 @@

##
# @QCryptoBlockCreateOptionsLUKS:
#
# The options that apply to LUKS encryption format initialization #
# @cipher-alg: the cipher algorithm for data encryption
# Currently defaults to 'aes-256'.
# @cipher-mode: the cipher mode for data encryption
# Currently defaults to 'xts'
# @ivgen-alg: the initialization vector generator
# Currently defaults to 'plain64'
# @ivgen-hash-alg: the initialization vector generator hash

Quoted 9 lines

qapi/crypto.json line 195

Quoted 6 lines
qapi/crypto.json +3 -0
 #                  Currently defaults to 'sha256'
 # @hash-alg: the master key hash algorithm
 #            Currently defaults to 'sha256'
+#
+# @active: Should the new secret be added (true) or erased (false)
+#          (amend only, since 4.2)

Is "active" established terminology? I wouldn't have guessed its meaning from its name...

Yea, this is one of the warts of the interface that I wanted to discuss with everyone basically the result of using same API for creation and amend.

blockdev-amend, similiar to qemu-img amend will allow the user to change the format driver settings, and will use the same BlockdevCreateOptions for that, similiar to how qemu-img amend works.

For creation of course the 'active' parameter is redundant, and it is forced to true. For amend it allows to add or erase a new key.
We couldn't really think about any better name, so I kind of decided just to document this and leave it like that.

Quoted 40 lines

As far as I can see, QCryptoBlockCreateOptionsLUKS is used just for blockdev-create with options.driver \in { luks, qcow, qcow2 }:

   { 'command': 'blockdev-create',
     'data': { ...
               'options': 'BlockdevCreateOptions' } }

   { 'union': 'BlockdevCreateOptions',
     ...
     'data': {
         ...
         'luks':           'BlockdevCreateOptionsLUKS',
         ...
         'qcow':           'BlockdevCreateOptionsQcow',
         'qcow2':          'BlockdevCreateOptionsQcow2',
         ... } }

With luks:

{ 'struct': 'BlockdevCreateOptionsLUKS',

     'base': 'QCryptoBlockCreateOptionsLUKS',
     ... }

With qcow and qcow2:

    { 'struct': 'BlockdevCreateOptionsQcow',
      'data': { ...
                '*encrypt':         'QCryptoBlockCreateOptions' } }
    { 'struct': 'BlockdevCreateOptionsQcow2',
      'data': { ...
                '*encrypt':         'QCryptoBlockCreateOptions',
                ... } }

    { 'union': 'QCryptoBlockCreateOptions',
      'base': 'QCryptoBlockOptionsBase',
      'discriminator': 'format',
      'data': { ...
                'luks': 'QCryptoBlockCreateOptionsLUKS' } }
Quoted 5 lines

I think I understand why we want blockdev-create to be able to specify a new secret.

Why do we want it to be able to delete an existing secret? How would that even work? Color me confused...

The BlockdevCreateOptions will now be used in
both creation and amend of a block device. Of course the deletion of an existing secret doesn't make sense on creation time, and a check is present to disallow the user to do that.

At the same time, the size and 'file' arguments are made optional, so that during amend you could change the block device without specifying those.

Quoted 9 lines

qapi/crypto.json line 200

Quoted 6 lines
qapi/crypto.json +5 -0
+#
+# @slot: The slot in which to put/erase the secret
+#        if not given, will select first free slot for secret addtion
+#        and erase all matching keyslots for erase. except last one
+#        (optional, since 4.2)

Excuse my possibly ignorant question: what exactly is a "matching keyslot"?

Not ignorant at all, I dropped a word here.
I meant to say that it will erase all the keyslots which match the given secret, except last one. The 'active' is what decides if to add to to remove a secret.

Quoted 8 lines

qapi/crypto.json line 205

Quoted 6 lines
qapi/crypto.json +5 -0
+#
+# @unlock-secret: The secret to use to unlock the image
+#        If not given, will use the secret that was used
+#        when opening the image.
+#        (optional, for amend only, since 4.2)

More ignorance: what is "amend"? No mention of it in qapi/*json...

Not ignorant at all again! This interface will be added in the next patch, and all the changes in this patch (other that specifying the keyslot, which can be in theory useful anyway) are for it.

Quoted 12 lines
Quoted 12 lines
qapi/crypto.json +3 -0
+#
 # @iter-time: number of milliseconds to spend in
 #             PBKDF passphrase processing. Currently defaults
 #             to 2000. (since 2.8)
@@ -201,7 +215,12 @@
             '*cipher-mode': 'QCryptoCipherMode',
             '*ivgen-alg': 'QCryptoIVGenAlgorithm',
             '*ivgen-hash-alg': 'QCryptoHashAlgorithm',
+
             '*hash-alg': 'QCryptoHashAlgorithm',
+            '*active' : 'bool',
Quoted 8 lines
Quoted 6 lines
qapi/crypto.json +3 -0
+            '*slot': 'int',
+            '*unlock-secret': 'str',
+
             '*iter-time': 'int'}}
 
 

[...]

Best regards,
Maxim Levitsky

#11 lore raw

Re: [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

In reply to Maxim Levitsky

Am 13.09.2019 um 00:30 hat Maxim Levitsky geschrieben:

Quoted 22 lines

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

qapi/crypto.json +7 -0
index b2a4cff683..9b83a70634 100644
@@ -190,6 +190,20 @@
 #                  Currently defaults to 'sha256'
 # @hash-alg: the master key hash algorithm
 #            Currently defaults to 'sha256'
+#
+# @active: Should the new secret be added (true) or erased (false)
+#          (amend only, since 4.2)
+#
+# @slot: The slot in which to put/erase the secret
+#        if not given, will select first free slot for secret addtion
+#        and erase all matching keyslots for erase. except last one

qapi/crypto.json line 209

Quoted 10 lines
qapi/crypto.json +7 -0
+#        (optional, since 4.2)
+#
+# @unlock-secret: The secret to use to unlock the image
+#        If not given, will use the secret that was used
+#        when opening the image.
+#        (optional, for amend only, since 4.2)
+#
 # @iter-time: number of milliseconds to spend in
 #             PBKDF passphrase processing. Currently defaults
 #             to 2000. (since 2.8)

This approach doesn't look right to me. BlockdevCreateOptions should describe the state of the image after the operation. You're describing an update instead (and in a way that doesn't allow you to change everything that you may want to change, so that you need to call the operation multiple times).

I imagined the syntax of a blockdev-amend QMP command similar to
x-blockdev-reopen: Describe the full set of options that you want to have in effect after the operation; if you don't want to change some option, you just specify it again with its old value.

Specifically for luks, this probably means that you have a @slots, which is a list that contains at least the secret for each slot, or JSON null for a slot that should be left empty.

With the same approach, you don't have to make 'size' optional in later patches, you can just require that the current size is re-specified. And later, blockdev-amend could actually allow changing the size of images if you provide a different value.

Kevin

#12 lore raw

Re: [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

Quoted 39 lines · Kevin Wolf

Am 13.09.2019 um 00:30 hat Maxim Levitsky geschrieben:

Quoted 21 lines

Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it.
Check that active=true it when creating.

qapi/crypto.json +7 -0
index b2a4cff683..9b83a70634 100644
@@ -190,6 +190,20 @@
 #                  Currently defaults to 'sha256'
 # @hash-alg: the master key hash algorithm
 #            Currently defaults to 'sha256'
+#
+# @active: Should the new secret be added (true) or erased (false)
+#          (amend only, since 4.2)
+#
+# @slot: The slot in which to put/erase the secret
+#        if not given, will select first free slot for secret addtion
+#        and erase all matching keyslots for erase. except last one

qapi/crypto.json line 209

Quoted 10 lines
qapi/crypto.json +7 -0
+#        (optional, since 4.2)
+#
+# @unlock-secret: The secret to use to unlock the image
+#        If not given, will use the secret that was used
+#        when opening the image.
+#        (optional, for amend only, since 4.2)
+#
 # @iter-time: number of milliseconds to spend in
 #             PBKDF passphrase processing. Currently defaults
 #             to 2000. (since 2.8)

This approach doesn't look right to me. BlockdevCreateOptions should describe the state of the image after the operation. You're describing an update instead (and in a way that doesn't allow you to change everything that you may want to change, so that you need to call the operation multiple times).

Quoted 4 lines

I imagined the syntax of a blockdev-amend QMP command similar to
x-blockdev-reopen: Describe the full set of options that you want to have in effect after the operation; if you don't want to change some option, you just specify it again with its old value.

This approach is a compromise trying to create more or less usable interface. In particular we (I and Daniel) wanted the following to work:

1. ability to add a new password to an empty keyslot and then remove the old password. This is probably the most common operation and it won't require the caller to know anything about the keyslots.

2. Allow the user to not know the passwords of some keyslots. For example if I want to add a new keyslot, I might not know some of the other keyslots. Specifying all the active keyslots, on each amend would force the user to know all the passwords (you can't 'extract' a password by reading a keyslot, since only hash of it is stored there for the security reasons)

Thus the amend interface either allows you to add a keyslot (either a specific one, or first free one), and to remove a keyslot (again, either a specific one, or one that matches given password).

Quoted 9 lines

Specifically for luks, this probably means that you have a @slots, which is a list that contains at least the secret for each slot, or JSON null for a slot that should be left empty.

With the same approach, you don't have to make 'size' optional in later patches, you can just require that the current size is re-specified. And later, blockdev-amend could actually allow changing the size of images if you provide a different value.

This can be done IMHO.

Quoted 2 lines

Kevin

Best regards,
Maxim Levitsky

#13 lore raw

PATCH v2 3/11 qcrypto-luks: implement the encryption key management

In reply to Maxim Levitsky
---
 crypto/block-luks.c | 356 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 354 insertions(+), 2 deletions(-)
crypto/block-luks.c +354 -2
index fed80e6646..26ce50b111 100644
@@ -70,6 +70,9 @@ typedef struct QCryptoBlockLUKSKeySlot QCryptoBlockLUKSKeySlot;
 
 #define QCRYPTO_BLOCK_LUKS_SECTOR_SIZE 512LL
 
+#define QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS 2000
+#define QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS 40
+
 static const char qcrypto_block_luks_magic[QCRYPTO_BLOCK_LUKS_MAGIC_LEN] = {
     'L', 'U', 'K', 'S', 0xBA, 0xBE
 };
@@ -219,6 +222,9 @@ struct QCryptoBlockLUKS {
 
     /* Hash algorithm used in pbkdf2 function */
     QCryptoHashAlgorithm hash_alg;
+
+    /* Name of the secret that was used to open the image */
+    char *secret;
 };
 
 
@@ -1070,6 +1076,170 @@ qcrypto_block_luks_find_key(QCryptoBlock *block,
 }
 
 
+
+/*
+ * Returns true if a slot i is marked as active
+ * (contains encrypted copy of the master key)
+ */
+static bool
+qcrypto_block_luks_slot_active(const QCryptoBlockLUKS *luks,
+                               unsigned int slot_idx)
+{
+    uint32_t val = luks->header.key_slots[slot_idx].active;
+    return val ==  QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED;
+}
+
+/*
+ * Returns the number of slots that are marked as active
+ * (contains encrypted copy of the master key)
+ */
+static unsigned int
+qcrypto_block_luks_count_active_slots(const QCryptoBlockLUKS *luks)
+{
+    size_t i = 0;
+    unsigned int ret = 0;
+
+    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+        if (qcrypto_block_luks_slot_active(luks, i)) {
+            ret++;
+        }
+    }
+    return ret;
+}
+
+
+/*
+ * Finds first key slot which is not active
+ * Returns the key slot index, or -1 if doesn't exist
+ */
+static int
+qcrypto_block_luks_find_free_keyslot(const QCryptoBlockLUKS *luks)
+{
+    size_t i;
+
+    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+        if (!qcrypto_block_luks_slot_active(luks, i)) {
+            return i;
+        }
+    }
+    return -1;
+
+}
+
+/*
+ * Erases an keyslot given its index
+ * Returns:
+ *    0 if the keyslot was erased successfully
+ *   -1 if a error occurred while erasing the keyslot
+ *
+ */
+static int
+qcrypto_block_luks_erase_key(QCryptoBlock *block,
+                             unsigned int slot_idx,
+                             QCryptoBlockWriteFunc writefunc,
+                             void *opaque,
+                             Error **errp)
+{
+    QCryptoBlockLUKS *luks = block->opaque;
+    QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
+    g_autofree uint8_t *garbagesplitkey = NULL;
+    size_t splitkeylen = luks->header.master_key_len * slot->stripes;
+    size_t i;
+
+    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+    assert(splitkeylen > 0);
+
+    garbagesplitkey = g_new0(uint8_t, splitkeylen);
+
+    /* Reset the key slot header */
+    memset(slot->salt, 0, QCRYPTO_BLOCK_LUKS_SALT_LEN);
+    slot->iterations = 0;
+    slot->active = QCRYPTO_BLOCK_LUKS_KEY_SLOT_DISABLED;
+
+    qcrypto_block_luks_store_header(block,  writefunc, opaque, errp);
+
+    /*
+     * Now try to erase the key material, even if the header
+     * update failed
+     */
+
+    for (i = 0 ; i < QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS ; i++) {
+        if (qcrypto_random_bytes(garbagesplitkey, splitkeylen, errp) < 0) {
+                /*
+                 * If we failed to get the random data, still write
+                 * at least zeros to the key slot at least once
+                 */
+
+                if (i > 0) {
+                    return -1;
+                }
+        }
+
+        if (writefunc(block,
+                      slot->key_offset_sector * QCRYPTO_BLOCK_LUKS_SECTOR_SIZE,
+                      garbagesplitkey,
+                      splitkeylen,
+                      opaque,
+                      errp) != splitkeylen) {
+            return -1;
+        }
+    }
+    return 0;
+}
+
+
+/*
+ * Erase all the keys that match the given password
+ * Will stop when only one keyslot is remaining
+ * Returns number of slots that were erased or -1 on failure
+ */
+static int
+qcrypto_block_luks_erase_matching_keys(QCryptoBlock *block,
+                                       const char *password,
+                                       QCryptoBlockReadFunc readfunc,
+                                       QCryptoBlockWriteFunc writefunc,
+                                       void *opaque,
+                                       bool force,
+                                       Error **errp)
+{
+    QCryptoBlockLUKS *luks = block->opaque;
+    size_t i;
+    int rv;
+    g_autofree uint8_t *masterkey = NULL;
+    unsigned int erased_cnt = 0;
+    unsigned int active_slot_cnt = qcrypto_block_luks_count_active_slots(luks);
+
+    masterkey = g_new0(uint8_t, luks->header.master_key_len);
+
+    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+
+        /* refuse to erase last key if not forced */
+        if (!force && active_slot_cnt == 1) {
+            break;
+        }
+
+        rv = qcrypto_block_luks_load_key(block, i, password, masterkey,
+                                         readfunc, opaque, errp);
+        if (rv < 0) {
+            return -1;
+        }
+        if (rv == 0) {
+            continue;
+        }
+
+        rv = qcrypto_block_luks_erase_key(block, i, writefunc, opaque, errp);
+        if (rv < 0) {
+            return -1;
+        }
+
+        erased_cnt++;
+        active_slot_cnt--;
+    }
+
+    return erased_cnt;
+}
+
+
 static int
 qcrypto_block_luks_open(QCryptoBlock *block,
                         QCryptoBlockOpenOptions *options,
@@ -1100,6 +1270,7 @@ qcrypto_block_luks_open(QCryptoBlock *block,
 
     luks = g_new0(QCryptoBlockLUKS, 1);
     block->opaque = luks;
+    luks->secret = g_strdup(options->u.luks.key_secret);
 
     ret = qcrypto_block_luks_load_header(block, readfunc, opaque, errp);
     if (ret < 0) {
@@ -1215,7 +1386,7 @@ qcrypto_block_luks_create(QCryptoBlock *block,
 
     memcpy(&luks_opts, &options->u.luks, sizeof(luks_opts));
     if (!luks_opts.has_iter_time) {
-        luks_opts.iter_time = 2000;
+        luks_opts.iter_time = QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
     }
     if (!luks_opts.has_cipher_alg) {
         luks_opts.cipher_alg = QCRYPTO_CIPHER_ALG_AES_256;
@@ -1273,6 +1444,8 @@ qcrypto_block_luks_create(QCryptoBlock *block,
                    optprefix ? optprefix : "");
         goto error;
     }
+    luks->secret = g_strdup(options->u.luks.key_secret);
+
     password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret, errp);
     if (!password) {
         goto error;
@@ -1504,6 +1677,182 @@ qcrypto_block_luks_create(QCryptoBlock *block,
 }
 
 
+#define CHECK_NON_AMEND_OPTION(luks, luks_opts, name) \
+    if (luks_opts.has_##name && luks_opts.name != luks->name) { \
+            error_setg(errp, "Option \"" #name "\" can't be amended"); \
+            goto cleanup; \
+    }
+
+static int
+qcrypto_block_luks_amend_options(QCryptoBlock *block,
+                                 QCryptoBlockReadFunc readfunc,
+                                 QCryptoBlockWriteFunc writefunc,
+                                 void *opaque,
+                                 QCryptoBlockCreateOptions *options,
+                                 bool force,
+                                 Error **errp)
+{
+    QCryptoBlockLUKS *luks = block->opaque;
+    QCryptoBlockCreateOptionsLUKS luks_opts;
+    g_autofree char *old_password = NULL;
+    g_autofree char *password = NULL;
+    const char *unlock_secret = luks->secret;
+    g_autofree uint8_t *masterkey = NULL;
+    int slot = -1;
+    int ret = -1;
+    bool active = true;
+    int64_t iter_time = QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
+
+    memcpy(&luks_opts, &options->u.luks, sizeof(luks_opts));
+
+    CHECK_NON_AMEND_OPTION(luks, luks_opts, cipher_alg);
+    CHECK_NON_AMEND_OPTION(luks, luks_opts, cipher_mode);
+    CHECK_NON_AMEND_OPTION(luks, luks_opts, ivgen_alg);
+    CHECK_NON_AMEND_OPTION(luks, luks_opts, ivgen_hash_alg);
+    CHECK_NON_AMEND_OPTION(luks, luks_opts, hash_alg);
+
+    /* Read given slot and check it */
+    if (luks_opts.has_slot) {
+        slot = luks_opts.slot;
+        if (slot < 0 || slot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
+            error_setg(errp,
+                       "Given key slot %i is not supported by LUKS", slot);
+            goto cleanup;
+        }
+    }
+
+    if (luks_opts.has_iter_time) {
+        iter_time = luks_opts.iter_time;
+    }
+
+    if (luks_opts.has_active && luks_opts.active == false) {
+        active = false;
+    }
+
+    if (active) {
+        /* Check that we are not overwriting an active slot */
+        if (!force && slot != -1 &&
+            qcrypto_block_luks_slot_active(luks, slot)) {
+
+            error_setg(errp, "Can't update an active key slot %i",
+                       slot);
+            goto cleanup;
+        }
+
+        /* check that we have the passwords*/
+        if (!luks_opts.has_key_secret) {
+            error_setg(errp, "Can't add a key slot without a  password");
+            goto cleanup;
+        }
+
+        if (luks_opts.has_unlock_secret) {
+            unlock_secret = luks_opts.unlock_secret;
+        }
+
+        /* Read the old password */
+        old_password = qcrypto_secret_lookup_as_utf8(unlock_secret, errp);
+        if (!old_password) {
+            goto cleanup;
+        }
+
+        masterkey = g_new0(uint8_t, luks->header.master_key_len);
+
+        /* Retrieve the master key*/
+        if (qcrypto_block_luks_find_key(block, old_password, masterkey,
+                                        readfunc, opaque,
+                                        errp) < 0) {
+            error_append_hint(errp,
+                              "'unlock-secret' failed to unlock the image");
+            goto cleanup;
+        }
+
+        /* Read the new password*/
+        password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret, errp);
+        if (!password) {
+            goto cleanup;
+        }
+
+        /* Find the new slot to write to */
+        if (slot == -1) {
+            slot = qcrypto_block_luks_find_free_keyslot(luks);
+
+            if (slot == -1) {
+                error_setg(errp,
+                           "Can't add a keyslot - all key slots are in use");
+                goto cleanup;
+            }
+        }
+
+        /* Store the master key to the new slot */
+        if (qcrypto_block_luks_store_key(block, slot, password, masterkey,
+                                         iter_time, writefunc, opaque,
+                                         errp)) {
+            error_append_hint(errp, "Failed to store the keyslot %i", slot);
+            goto cleanup;
+        }
+
+    } else {
+        /* Check that we are not erasing last key slot */
+        if (qcrypto_block_luks_count_active_slots(luks) <= 1) {
+            if (!force) {
+                error_setg(errp, "Only one slot active - can't erase");
+                goto cleanup;
+            }
+        }
+
+        if (slot != -1) {
+            /* Check that we are not erasing an inactive slot */
+            if (!qcrypto_block_luks_slot_active(luks, luks_opts.slot)) {
+                if (!force) {
+                    error_setg(errp, "Can't erase an inactive key slot %i",
+                               slot);
+                    goto cleanup;
+                }
+            }
+
+            /* Erase the given slot */
+            if (qcrypto_block_luks_erase_key(block, slot,
+                                             writefunc, opaque, errp)) {
+                goto cleanup;
+            }
+
+        } else {
+            if (!luks_opts.has_key_secret) {
+                error_setg(errp,
+                           "To erase a keyslot you have to specify either the"
+                           "slot index or a password "
+                           "(to erase all slots that match it)");
+                goto cleanup;
+            }
+
+            password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret,
+                                                     errp);
+            if (!password) {
+                goto cleanup;
+            }
+
+            ret = qcrypto_block_luks_erase_matching_keys(block, password,
+                                                       readfunc, writefunc,
+                                                       opaque, force, errp);
+            if (ret == 0) {
+                error_setg(errp,
+                           "Didn't erase a keyslot, because no keyslots match"
+                           " the given password");
+                ret = -EINVAL;
+                goto cleanup;
+            }
+
+            if (ret < 0) {
+                goto cleanup;
+            }
+        }
+    }
+    ret = 0;
+cleanup:
+    return ret;
+}
+
+
 static int qcrypto_block_luks_get_info(QCryptoBlock *block,
                                        QCryptoBlockInfo *info,
                                        Error **errp)
@@ -1551,7 +1900,9 @@ static int qcrypto_block_luks_get_info(QCryptoBlock *block,
 
 static void qcrypto_block_luks_cleanup(QCryptoBlock *block)
 {
-    g_free(block->opaque);
+    QCryptoBlockLUKS *luks = block->opaque;
+    g_free(luks->secret);
+    g_free(luks);
 }
 
 
@@ -1588,6 +1939,7 @@ qcrypto_block_luks_encrypt(QCryptoBlock *block,
 const QCryptoBlockDriver qcrypto_block_driver_luks = {
     .open = qcrypto_block_luks_open,
     .create = qcrypto_block_luks_create,
+    .amend = qcrypto_block_luks_amend_options,
     .get_info = qcrypto_block_luks_get_info,
     .cleanup = qcrypto_block_luks_cleanup,
     .decrypt = qcrypto_block_luks_decrypt,
Signature · 1 line
2.17.2
#14 lore raw

PATCH v2 4/11 block: amend: add 'force' option

In reply to Maxim Levitsky

'force' optinion will be used for some unsafe option amend operations.

This includes things like erasing last keyslot in luks (which pretty much guarantees destroying the data, unless the master key is backed up by extrnal means, but that _might_ be desired result)

---
 block.c                   | 4 +++-
 block/qcow2.c             | 1 +
 include/block/block.h     | 1 +
 include/block/block_int.h | 1 +
 qemu-img-cmds.hx          | 4 ++--
 qemu-img.c                | 8 +++++++-
 qemu-img.texi             | 6 +++++-
 7 files changed, 20 insertions(+), 5 deletions(-)
block.c +3 -1
index 5944124845..414303f76d 100644
@@ -6141,6 +6141,7 @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs,
 
 int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
                        BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
+                       bool force,
                        Error **errp)
 {
     if (!bs->drv) {
@@ -6152,7 +6153,8 @@ int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
                    bs->drv->format_name);
         return -ENOTSUP;
     }
-    return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque, errp);
+    return bs->drv->bdrv_amend_options(bs, opts, status_cb,
+                                       cb_opaque, force, errp);
 }
 
 /* This function will be called by the bdrv_recurse_is_first_non_filter method
block/qcow2.c +1 -0
index 5bdb8b18f4..0618a63793 100644
@@ -4822,6 +4822,7 @@ static void qcow2_amend_helper_cb(BlockDriverState *bs,
 static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
                                BlockDriverAmendStatusCB *status_cb,
                                void *cb_opaque,
+                               bool force,
                                Error **errp)
 {
     BDRVQcow2State *s = bs->opaque;
include/block/block.h +1 -0
index 124ad40809..6bc89c7667 100644
@@ -400,6 +400,7 @@ typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset,
                                       int64_t total_work_size, void *opaque);
 int bdrv_amend_options(BlockDriverState *bs_new, QemuOpts *opts,
                        BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
+                       bool force,
                        Error **errp);
 
 /* external snapshots */
include/block/block_int.h +1 -0
index 0422acdf1c..5ea30f9d58 100644
@@ -411,6 +411,7 @@ struct BlockDriver {
     int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
                               BlockDriverAmendStatusCB *status_cb,
                               void *cb_opaque,
+                              bool force,
                               Error **errp);
 
     void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
qemu-img-cmds.hx +2 -2
index 1c93e6d185..323ea10ad0 100644
@@ -14,9 +14,9 @@ STEXI
 ETEXI
 
 DEF("amend", img_amend,
-    "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename")
+    "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] [--force] -o options filename")
 STEXI
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] [--force] -o @var{options} @var{filename}
 ETEXI
 
 DEF("bench", img_bench,
qemu-img.c +7 -1
index 4ee436fc94..30300870ff 100644
@@ -70,6 +70,7 @@ enum {
     OPTION_PREALLOCATION = 265,
     OPTION_SHRINK = 266,
     OPTION_SALVAGE = 267,
+    OPTION_FORCE = 268,
 };
 
 typedef enum OutputFormat {
@@ -3915,6 +3916,7 @@ static int img_amend(int argc, char **argv)
     BlockBackend *blk = NULL;
     BlockDriverState *bs = NULL;
     bool image_opts = false;
+    bool force = false;
 
     cache = BDRV_DEFAULT_CACHE;
     for (;;) {
@@ -3922,6 +3924,7 @@ static int img_amend(int argc, char **argv)
             {"help", no_argument, 0, 'h'},
             {"object", required_argument, 0, OPTION_OBJECT},
             {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
+            {"force", no_argument, 0, OPTION_FORCE},
             {0, 0, 0, 0}
         };
         c = getopt_long(argc, argv, ":ho:f:t:pq",
@@ -3977,6 +3980,9 @@ static int img_amend(int argc, char **argv)
         case OPTION_IMAGE_OPTS:
             image_opts = true;
             break;
+        case OPTION_FORCE:
+            force = true;
+            break;
         }
     }
 
@@ -4054,7 +4060,7 @@ static int img_amend(int argc, char **argv)
 
     /* In case the driver does not call amend_status_cb() */
     qemu_progress_print(0.f, 0);
-    ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, &err);
+    ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, force, &err);
     qemu_progress_print(100.f, 0);
     if (ret < 0) {
         error_report_err(err);
qemu-img.texi +5 -1
index b5156d6316..b6ed4357e8 100644
@@ -201,11 +201,15 @@ Command description:
 
 @table @option
 
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] [--force] -o @var{options} @var{filename}
 
 Amends the image format specific @var{options} for the image file
 @var{filename}. Not all file formats support this operation.
 
+--force allows some unsafe operations. Currently for -f luks,
+it allows to erase last encryption key, and to overwrite an active
+encryption key.
+
 @item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
 
 Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
Signature · 1 line
2.17.2
#15 lore raw

PATCH v2 5/11 block/crypto: implement the encryption key management

In reply to Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +115 -3
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)
+{
+    BlockDriverState *bs = opaque;
+    ssize_t ret;
+
+    ret = bdrv_pwrite(bs->file, offset, buf, buflen);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not write encryption header");
+        return ret;
+    }
+    return ret;
+}
+
 
 struct BlockCryptoCreateData {
     BlockBackend *blk;
@@ -647,6 +666,100 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
     return spec_info;
 }
 
+
+static int
+block_crypto_amend_options(BlockDriverState *bs,
+                           QemuOpts *opts,
+                           BlockDriverAmendStatusCB *status_cb,
+                           void *cb_opaque,
+                           bool force,
+                           Error **errp)
+{
+    BlockCrypto *crypto = bs->opaque;
+    QDict *cryptoopts = NULL;
+    QCryptoBlockCreateOptions *amend_options = NULL;
+    int ret;
+
+    assert(crypto);
+    assert(crypto->block);
+
+    crypto->updating_keys = true;
+
+    ret = bdrv_child_refresh_perms(bs, bs->file, errp);
+    if (ret < 0) {
+        goto cleanup;
+    }
+
+    cryptoopts = qemu_opts_to_qdict_filtered(opts, NULL,
+                                             &block_crypto_create_opts_luks,
+                                             true);
+
+    qdict_put_str(cryptoopts, "format", "luks");
+    amend_options = block_crypto_create_opts_init(cryptoopts, errp);
+    if (!amend_options) {
+        ret = -EINVAL;
+        goto cleanup;
+    }
+
+    ret = qcrypto_block_amend_options(crypto->block,
+                                      block_crypto_read_func,
+                                      block_crypto_write_func,
+                                      bs,
+                                      amend_options,
+                                      force,
+                                      errp);
+cleanup:
+    crypto->updating_keys = false;
+    bdrv_child_refresh_perms(bs, bs->file, errp);
+    qapi_free_QCryptoBlockCreateOptions(amend_options);
+    qobject_unref(cryptoopts);
+    return ret;
+}
+
+
+static void
+block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
+                         const BdrvChildRole *role,
+                         BlockReopenQueue *reopen_queue,
+                         uint64_t perm, uint64_t shared,
+                         uint64_t *nperm, uint64_t *nshared)
+{
+
+    BlockCrypto *crypto = bs->opaque;
+
+    /*
+     * Ask for consistent read permission so that if
+     * someone else tries to open this image with this permission
+     * neither will be able to edit encryption keys
+     */
+    if (!(bs->open_flags & BDRV_O_NO_IO)) {
+        perm |= BLK_PERM_CONSISTENT_READ;
+    }
+
+    /*
+     * This driver doesn't modify LUKS metadata except
+     * when updating the encryption slots.
+     * Thus unlike a proper format driver we don't ask for
+     * shared write permission. However we need it
+     * when we area updating keys, to ensure that only we
+     * had opened the device r/w
+     *
+     * Encryption update will set the crypto->updating_keys
+     * during that period and refresh permissions
+     *
+     */
+
+    if (crypto->updating_keys) {
+        /*need exclusive write access for header update  */
+        perm |= BLK_PERM_WRITE;
+        shared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
+    }
+
+    bdrv_filter_default_perms(bs, c, role, reopen_queue,
+            perm, shared, nperm, nshared);
+}
+
+
 static const char *const block_crypto_strong_runtime_opts[] = {
     BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET,
 
@@ -659,9 +772,7 @@ static BlockDriver bdrv_crypto_luks = {
     .bdrv_probe         = block_crypto_probe_luks,
     .bdrv_open          = block_crypto_open_luks,
     .bdrv_close         = block_crypto_close,
-    /* This driver doesn't modify LUKS metadata except when creating image.
-     * Allow share-rw=on as a special case. */
-    .bdrv_child_perm    = bdrv_filter_default_perms,
+    .bdrv_child_perm    = block_crypto_child_perms,
     .bdrv_co_create     = block_crypto_co_create_luks,
     .bdrv_co_create_opts = block_crypto_co_create_opts_luks,
     .bdrv_co_truncate   = block_crypto_co_truncate,
@@ -674,6 +785,7 @@ static BlockDriver bdrv_crypto_luks = {
     .bdrv_getlength     = block_crypto_getlength,
     .bdrv_get_info      = block_crypto_get_info_luks,
     .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
+    .bdrv_amend_options = block_crypto_amend_options,
 
     .strong_runtime_opts = block_crypto_strong_runtime_opts,
 };
Signature · 1 line
2.17.2
#16 lore raw

Re: [PATCH v2 05/11] block/crypto: implement the encryption key management

Quoted 47 lines · Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +1 -0
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 

block/crypto.c static ssize_t block_crypto_read_func(QCryptoBlock *block, · line 79

Quoted 9 lines
block/crypto.c +6 -0
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)

There’s already a function of this name for creation.

Quoted 15 lines
block/crypto.c +12 -0
+{
+    BlockDriverState *bs = opaque;
+    ssize_t ret;
+
+    ret = bdrv_pwrite(bs->file, offset, buf, buflen);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not write encryption header");
+        return ret;
+    }
+    return ret;
+}
+
 
 struct BlockCryptoCreateData {
     BlockBackend *blk;

[...]

Quoted 31 lines
block/crypto.c +31 -0
+static void
+block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
+                         const BdrvChildRole *role,
+                         BlockReopenQueue *reopen_queue,
+                         uint64_t perm, uint64_t shared,
+                         uint64_t *nperm, uint64_t *nshared)
+{
+
+    BlockCrypto *crypto = bs->opaque;
+
+    /*
+     * Ask for consistent read permission so that if
+     * someone else tries to open this image with this permission
+     * neither will be able to edit encryption keys
+     */
+    if (!(bs->open_flags & BDRV_O_NO_IO)) {
+        perm |= BLK_PERM_CONSISTENT_READ;
+    }
+
+    /*
+     * This driver doesn't modify LUKS metadata except
+     * when updating the encryption slots.
+     * Thus unlike a proper format driver we don't ask for
+     * shared write permission. However we need it
+     * when we area updating keys, to ensure that only we
+     * had opened the device r/w
+     *
+     * Encryption update will set the crypto->updating_keys
+     * during that period and refresh permissions
+     *
+     */

block/crypto.c block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp) · line 760

Quoted 10 lines
block/crypto.c +10 -0
+
+    if (crypto->updating_keys) {
+        /*need exclusive write access for header update  */
+        perm |= BLK_PERM_WRITE;
+        shared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
+    }
+
+    bdrv_filter_default_perms(bs, c, role, reopen_queue,
+            perm, shared, nperm, nshared);
+}

This will probably work, but usually drivers do it the other way around:
First call any of the default_perms(), and then adjust *nperm and *nshared as required.

(perm/shared are what the parents need, *nperm/*nshared is what this driver needs, so it makes more sense that way; and this way nobody has to check whether the settings survived the default_perms() call.)

((But the permissions themselves do look correct.))

Max

#17 lore raw

Re: [PATCH v2 05/11] block/crypto: implement the encryption key management

Quoted 51 lines · Max Reitz
Quoted 50 lines · Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +1 -0
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
Quoted 8 lines

block/crypto.c static ssize_t block_crypto_read_func(QCryptoBlock *block, · line 79

Quoted 6 lines
block/crypto.c +6 -0
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)

There’s already a function of this name for creation.

There is a long story why two write functions are needed. i tried to use only one, but at the end I and Daniel both agreed that its just better to have two functions.

The reason is that during creation, the luks BlockDriverState doesn't exist yet, and so the creation routine basically just writes to the underlying protocol driver.

Thats is why the block_crypto_create_write_func receives a BlockBackend pointer, to which the BlockDriverState of the underlying protocol driver is inserted.

On the other hand, for amend, the luks block device is open, and it only knows about its own BlockDriverState, and thus the io should be done on bs->file

So instead of trying to coerce a single callback to do both of this, we decided to just have a little code duplication.

Quoted 65 lines
Quoted 16 lines
block/crypto.c +12 -0
+{
+    BlockDriverState *bs = opaque;
+    ssize_t ret;
+
+    ret = bdrv_pwrite(bs->file, offset, buf, buflen);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not write encryption header");
+        return ret;
+    }
+    return ret;
+}
+
 
 struct BlockCryptoCreateData {
     BlockBackend *blk;

[...]

Quoted 31 lines
block/crypto.c +31 -0
+static void
+block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
+                         const BdrvChildRole *role,
+                         BlockReopenQueue *reopen_queue,
+                         uint64_t perm, uint64_t shared,
+                         uint64_t *nperm, uint64_t *nshared)
+{
+
+    BlockCrypto *crypto = bs->opaque;
+
+    /*
+     * Ask for consistent read permission so that if
+     * someone else tries to open this image with this permission
+     * neither will be able to edit encryption keys
+     */
+    if (!(bs->open_flags & BDRV_O_NO_IO)) {
+        perm |= BLK_PERM_CONSISTENT_READ;
+    }
+
+    /*
+     * This driver doesn't modify LUKS metadata except
+     * when updating the encryption slots.
+     * Thus unlike a proper format driver we don't ask for
+     * shared write permission. However we need it
+     * when we area updating keys, to ensure that only we
+     * had opened the device r/w
+     *
+     * Encryption update will set the crypto->updating_keys
+     * during that period and refresh permissions
+     *
+     */

block/crypto.c block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp) · line 760

Quoted 10 lines
block/crypto.c +10 -0
+
+    if (crypto->updating_keys) {
+        /*need exclusive write access for header update  */
+        perm |= BLK_PERM_WRITE;
+        shared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
+    }
+
+    bdrv_filter_default_perms(bs, c, role, reopen_queue,
+            perm, shared, nperm, nshared);
+}

This will probably work, but usually drivers do it the other way around:
First call any of the default_perms(), and then adjust *nperm and *nshared as required.

Quoted 5 lines

(perm/shared are what the parents need, *nperm/*nshared is what this driver needs, so it makes more sense that way; and this way nobody has to check whether the settings survived the default_perms() call.)

((But the permissions themselves do look correct.))

You are right! I made this change now and include
it in the next iteration of the patches.

Quoted 3 lines

Max

Best regards,
Maxim Levitsky

#18 lore raw

Re: [PATCH v2 05/11] block/crypto: implement the encryption key management

Quoted 72 lines · Maxim Levitsky
Quoted 51 lines · Max Reitz
Quoted 50 lines · Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +1 -0
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
Quoted 8 lines

block/crypto.c static ssize_t block_crypto_read_func(QCryptoBlock *block, · line 79

Quoted 6 lines
block/crypto.c +6 -0
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)

There’s already a function of this name for creation.

There is a long story why two write functions are needed. i tried to use only one, but at the end I and Daniel both agreed that its just better to have two functions.

The reason is that during creation, the luks BlockDriverState doesn't exist yet, and so the creation routine basically just writes to the underlying protocol driver.

Thats is why the block_crypto_create_write_func receives a BlockBackend pointer, to which the BlockDriverState of the underlying protocol driver is inserted.

Quoted 5 lines

On the other hand, for amend, the luks block device is open, and it only knows about its own BlockDriverState, and thus the io should be done on bs->file

So instead of trying to coerce a single callback to do both of this, we decided to just have a little code duplication.

I meant: This doesn’t compile. There’s already another function of this name.

Max

#19 lore raw

Re: [PATCH v2 05/11] block/crypto: implement the encryption key management

Quoted 73 lines · Max Reitz
Quoted 72 lines · Maxim Levitsky
Quoted 51 lines · Max Reitz
Quoted 50 lines · Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +1 -0
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
Quoted 8 lines

block/crypto.c static ssize_t block_crypto_read_func(QCryptoBlock *block, · line 79

Quoted 6 lines
block/crypto.c +6 -0
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)

There’s already a function of this name for creation.

There is a long story why two write functions are needed. i tried to use only one, but at the end I and Daniel both agreed that its just better to have two functions.

The reason is that during creation, the luks BlockDriverState doesn't exist yet, and so the creation routine basically just writes to the underlying protocol driver.

Thats is why the block_crypto_create_write_func receives a BlockBackend pointer, to which the BlockDriverState of the underlying protocol driver is inserted.

Quoted 9 lines
Quoted 5 lines

On the other hand, for amend, the luks block device is open, and it only knows about its own BlockDriverState, and thus the io should be done on bs->file

So instead of trying to coerce a single callback to do both of this, we decided to just have a little code duplication.

I meant: This doesn’t compile. There’s already another function of this name.

You probably didn't apply the 'block-crypto: misc refactoring' patch,
or I forgot to send it.
All that patch does is to rename block_crypto_write_func to block_crypto_create_write_func and same (for consistency) for block_crypto_init_func -> block_crypto_create_init_func

And then in this patch I add the block_crypto_write_func, to be used for anything but creation code, together with existing block_crypto_read_func which is already not used for creation.

Best regards,
Maxim Levitsky

#20 lore raw

Re: [PATCH v2 05/11] block/crypto: implement the encryption key management

Quoted 80 lines · Maxim Levitsky
Quoted 79 lines · Max Reitz
Quoted 77 lines · Maxim Levitsky
Quoted 51 lines · Max Reitz
Quoted 50 lines · Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +1 -0
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
Quoted 8 lines

block/crypto.c static ssize_t block_crypto_read_func(QCryptoBlock *block, · line 79

Quoted 6 lines
block/crypto.c +6 -0
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)

There’s already a function of this name for creation.

There is a long story why two write functions are needed. i tried to use only one, but at the end I and Daniel both agreed that its just better to have two functions.

The reason is that during creation, the luks BlockDriverState doesn't exist yet, and so the creation routine basically just writes to the underlying protocol driver.

Thats is why the block_crypto_create_write_func receives a BlockBackend pointer, to which the BlockDriverState of the underlying protocol driver is inserted.

On the other hand, for amend, the luks block device is open, and it only knows about its own BlockDriverState, and thus the io should be done on bs->file

So instead of trying to coerce a single callback to do both of this, we decided to just have a little code duplication.

Quoted 6 lines
Quoted 3 lines

I meant: This doesn’t compile. There’s already another function of this name.

You probably didn't apply the 'block-crypto: misc refactoring' patch, or I forgot to send it.

Maybe you forgot to mention anywhere that I should.

Max

#21 lore raw

Re: [PATCH v2 05/11] block/crypto: implement the encryption key management

Quoted 81 lines · Max Reitz
Quoted 80 lines · Maxim Levitsky
Quoted 79 lines · Max Reitz
Quoted 77 lines · Maxim Levitsky
Quoted 51 lines · Max Reitz
Quoted 50 lines · Maxim Levitsky

This implements the encryption key management
using the generic code in qcrypto layer
(currently only for qemu-img amend)

This code adds another 'write_func' because the initialization write_func works directly on the underlying file, because during the creation, there is no open instance of the luks driver, but during regular use, we have it, and should use it instead.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver still support write sharing,
but be safe against concurrent metadata update (the keys) Eventually write sharing for luks driver will be deprecated and removed together with this hack.

The hack is that we ask (as a format driver) for
BLK_PERM_CONSISTENT_READ always
(technically always unless opened with BDRV_O_NO_IO)

and then when we want to update the keys, we
unshare that permission. So if someone else
has the image open, even readonly, this will fail.

Also thanks to Daniel Berrange for the variant of
that hack that involves asking for read,
rather that write permission

---
 block/crypto.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 115 insertions(+), 3 deletions(-)
block/crypto.c +1 -0
index a6a3e1f1d8..f42fa057e6 100644
@@ -36,6 +36,7 @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -70,6 +71,24 @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
Quoted 8 lines

block/crypto.c static ssize_t block_crypto_read_func(QCryptoBlock *block, · line 79

Quoted 6 lines
block/crypto.c +6 -0
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)

There’s already a function of this name for creation.

There is a long story why two write functions are needed. i tried to use only one, but at the end I and Daniel both agreed that its just better to have two functions.

The reason is that during creation, the luks BlockDriverState doesn't exist yet, and so the creation routine basically just writes to the underlying protocol driver.

Thats is why the block_crypto_create_write_func receives a BlockBackend pointer, to which the BlockDriverState of the underlying protocol driver is inserted.

On the other hand, for amend, the luks block device is open, and it only knows about its own BlockDriverState, and thus the io should be done on bs->file

So instead of trying to coerce a single callback to do both of this, we decided to just have a little code duplication.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

I meant: This doesn’t compile. There’s already another function of this name.

You probably didn't apply the 'block-crypto: misc refactoring' patch, or I forgot to send it.

Maybe you forgot to mention anywhere that I should.

Now I remember that this patch was in the original re-factoring patch series, and for some reason it was dropped.
Its now locally in my git tree, so I forgot that it wasn't in this patch series already.

Sorry for the mess, I'll soon resend the updated patch series.

Best regards,
Maxim Levitsky

#22 lore raw

PATCH v2 6/11 qcow2: implement crypto amend options

In reply to Maxim Levitsky
---
 block/qcow2.c | 77 +++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 62 insertions(+), 15 deletions(-)
block/qcow2.c +62 -15
index 0618a63793..26f83aeb44 100644
@@ -172,6 +172,25 @@ static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset,
     return ret;
 }
 
+static QCryptoBlockCreateOptions*
+qcow2_extract_crypto_create_opts(QemuOpts *opts, const char *fmt, Error **errp)
+{
+    QDict *cryptoopts_qdict;
+    QCryptoBlockCreateOptions *cryptoopts;
+    QDict *opts_qdict;
+
+    /* Extract "encrypt." options into a qdict */
+    opts_qdict = qemu_opts_to_qdict(opts, NULL);
+    qdict_extract_subqdict(opts_qdict, &cryptoopts_qdict, "encrypt.");
+    qobject_unref(opts_qdict);
+
+    /* Build QCryptoBlockCreateOptions object from qdict */
+    qdict_put_str(cryptoopts_qdict, "format", "luks");
+    cryptoopts = block_crypto_create_opts_init(cryptoopts_qdict, errp);
+    qobject_unref(cryptoopts_qdict);
+    return cryptoopts;
+}
+
 
 /* 
  * read qcow2 extension and fill bs
@@ -4365,20 +4384,10 @@ static ssize_t qcow2_measure_crypto_hdr_write_func(QCryptoBlock *block,
 static bool qcow2_measure_luks_headerlen(QemuOpts *opts, size_t *len,
                                          Error **errp)
 {
-    QDict *opts_qdict;
-    QDict *cryptoopts_qdict;
     QCryptoBlockCreateOptions *cryptoopts;
     QCryptoBlock *crypto;
 
-    /* Extract "encrypt." options into a qdict */
-    opts_qdict = qemu_opts_to_qdict(opts, NULL);
-    qdict_extract_subqdict(opts_qdict, &cryptoopts_qdict, "encrypt.");
-    qobject_unref(opts_qdict);
-
-    /* Build QCryptoBlockCreateOptions object from qdict */
-    qdict_put_str(cryptoopts_qdict, "format", "luks");
-    cryptoopts = block_crypto_create_opts_init(cryptoopts_qdict, errp);
-    qobject_unref(cryptoopts_qdict);
+    cryptoopts = qcow2_extract_crypto_create_opts(opts, "luks", errp);
     if (!cryptoopts) {
         return false;
     }
@@ -4755,6 +4764,7 @@ typedef enum Qcow2AmendOperation {
      * invocation from an operation change */
     QCOW2_NO_OPERATION = 0,
 
+    QCOW2_UPDATING_ENCRYPTION,
     QCOW2_CHANGING_REFCOUNT_ORDER,
     QCOW2_DOWNGRADING,
 } Qcow2AmendOperation;
@@ -4839,6 +4849,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     int ret;
     QemuOptDesc *desc = opts->list->desc;
     Qcow2AmendHelperCBInfo helper_cb_info;
+    bool encryption_update = false;
 
     while (desc && desc->name) {
         if (!qemu_opt_find(opts, desc->name)) {
@@ -4887,9 +4898,22 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
                 return -ENOTSUP;
             }
         } else if (g_str_has_prefix(desc->name, "encrypt.")) {
-            error_setg(errp,
-                       "Changing the encryption parameters is not supported");
-            return -ENOTSUP;
+
+            if (!s->crypto) {
+                error_setg(errp,
+                           "Can't amend encryption options - encryption not present");
+                return -EINVAL;
+
+            }
+
+            if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+                error_setg(errp,
+                           "Only LUKS encryption options can be amended");
+                return -ENOTSUP;
+            }
+
+            encryption_update = true;
+
         } else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) {
             cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE,
                                              cluster_size);
@@ -4939,7 +4963,8 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
         .original_status_cb = status_cb,
         .original_cb_opaque = cb_opaque,
         .total_operations = (new_version < old_version)
-                          + (s->refcount_bits != refcount_bits)
+                          + (s->refcount_bits != refcount_bits) +
+                          (encryption_update == true)
     };
 
     /* Upgrade first (some features may require compat=1.1) */
@@ -4953,6 +4978,28 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
         }
     }
 
+    if (encryption_update) {
+        QCryptoBlockCreateOptions *cryptoopts;
+
+        cryptoopts = qcow2_extract_crypto_create_opts(opts, "luks", errp);
+        if (!cryptoopts) {
+            return -EINVAL;
+        }
+
+        helper_cb_info.current_operation = QCOW2_UPDATING_ENCRYPTION;
+
+        ret = qcrypto_block_amend_options(s->crypto,
+                                          qcow2_crypto_hdr_read_func,
+                                          qcow2_crypto_hdr_write_func,
+                                          bs,
+                                          cryptoopts,
+                                          force,
+                                          errp);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
     if (s->refcount_bits != refcount_bits) {
         int refcount_order = ctz32(refcount_bits);
 
Signature · 1 line
2.17.2
#23 lore raw

PATCH v2 7/11 block: add x-blockdev-amend qmp command

In reply to Maxim Levitsky
---
 block/Makefile.objs       |   2 +-
 block/amend.c             | 116 ++++++++++++++++++++++++++++++++++++++
 include/block/block_int.h |  23 ++++++--
 qapi/block-core.json      |  26 +++++++++
 qapi/job.json             |   4 +-
 5 files changed, 163 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c
block/Makefile.objs +1 -1
index 35f3bca4d9..10d0308792 100644
@@ -18,7 +18,7 @@ block-obj-y += block-backend.o snapshot.o qapi.o
 block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
-block-obj-y += null.o mirror.o commit.o io.o create.o
+block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
 
block/amend.c new file +116 -0

116 lines not on this page · raw message

include/block/block_int.h +17 -6
index 5ea30f9d58..3001114361 100644
@@ -133,11 +133,28 @@ struct BlockDriver {
     int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
                           Error **errp);
     void (*bdrv_close)(BlockDriverState *bs);
+
+
     int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
                                        Error **errp);
     int coroutine_fn (*bdrv_co_create_opts)(const char *filename,
                                             QemuOpts *opts,
                                             Error **errp);
+
+
+    int coroutine_fn (*bdrv_co_amend)(BlockDriverState *bs,
+                                      BlockdevCreateOptions *opts,
+                                      bool force,
+                                      Error **errp);
+
+    int (*bdrv_amend_options)(BlockDriverState *bs,
+                              QemuOpts *opts,
+                              BlockDriverAmendStatusCB *status_cb,
+                              void *cb_opaque,
+                              bool force,
+                              Error **errp);
+
+
     int (*bdrv_make_empty)(BlockDriverState *bs);
 
     /*
@@ -408,12 +425,6 @@ struct BlockDriver {
                                       BdrvCheckResult *result,
                                       BdrvCheckMode fix);
 
-    int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
-                              BlockDriverAmendStatusCB *status_cb,
-                              void *cb_opaque,
-                              bool force,
-                              Error **errp);
-
     void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
 
     /* TODO Better pass a option string/QDict/QemuOpts to add any rule? */
qapi/block-core.json +26 -0
index e6edd641f1..7900914506 100644
@@ -4650,6 +4650,32 @@
   'data': { 'job-id': 'str',
             'options': 'BlockdevCreateOptions' } }
 
+##
+# @x-blockdev-amend:
+#
+# Starts a job to amend format specific options of an existing open block device.
+# The job is automatically finalized, but a manual job-dismiss is required.
+#
+# @job-id:          Identifier for the newly created job.
+#
+# @node-name:       Name of the block node to work on
+#
+# @options:         Options (same as for image creation)
+#
+# @force:           Allow unsafe operations, format specific
+#                   For luks that allows erase of the last active keyslot
+#                   (permanent loss of data),
+#                   and replacement of an active keyslot
+#                   (possible loss of data if IO error happens)
+#
+# Since: 4.2
+##
+{ 'command': 'x-blockdev-amend',
+  'data': { 'job-id': 'str',
+            'node-name': 'str',
+            'options': 'BlockdevCreateOptions',
+            '*force': 'bool' } }
+
 ##
 # @blockdev-open-tray:
 #
qapi/job.json +3 -1
index a121b615fb..342d29a7aa 100644
@@ -19,10 +19,12 @@
 #
 # @create: image creation job type, see "blockdev-create" (since 3.0)
 #
+# @amend: image options amend job type, see "x-blockdev-amend" (since 4.2)
+#
 # Since: 1.7
 ##
 { 'enum': 'JobType',
-  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] }
 
 ##
 # @JobStatus:
Signature · 1 line
2.17.2
#24 lore raw

Re: [PATCH v2 07/11] block: add x-blockdev-amend qmp command

Quoted 9 lines · Maxim Levitsky
---
 block/Makefile.objs       |   2 +-
 block/amend.c             | 116 ++++++++++++++++++++++++++++++++++++++
 include/block/block_int.h |  23 ++++++--
 qapi/block-core.json      |  26 +++++++++
 qapi/job.json             |   4 +-
 5 files changed, 163 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c

I think I’d move this (and everything to belongs to it) to a different series.

Quoted 60 lines
block/Makefile.objs +1 -1
index 35f3bca4d9..10d0308792 100644
@@ -18,7 +18,7 @@ block-obj-y += block-backend.o snapshot.o qapi.o
 block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
-block-obj-y += null.o mirror.o commit.o io.o create.o
+block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
 
block/amend.c new file +41 -0
new file mode 100644
index 0000000000..9bd28e08e7
@@ -0,0 +1,116 @@
+/*
+ * Block layer code related to image options amend
+ *
+ * Copyright (c) 2018 Kevin Wolf <[email protected]>
+ * Copyright (c) 2019 Maxim Levitsky <[email protected]>
+ *
+ * Heavily based on create.c
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qemu/job.h"
+#include "qemu/main-loop.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-visit-block-core.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevAmendJob {
+    Job common;
+    BlockdevCreateOptions *opts;
+    BlockDriverState *bs;
+    bool force;

block/amend.c line 51

Quoted 10 lines
block/amend.c new file +10 -0
+} BlockdevAmendJob;
+
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
+    int ret;
+
+    job_progress_set_remaining(&s->common, 1);
+    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+    job_progress_update(&s->common, 1);

It would be nice if the amend job could make use of the progress reporting that we have in place for amend.

Quoted 41 lines
block/amend.c new file +41 -0
+
+    qapi_free_BlockdevCreateOptions(s->opts);
+
+    return ret;
+}
+
+static const JobDriver blockdev_amend_job_driver = {
+    .instance_size = sizeof(BlockdevAmendJob),
+    .job_type      = JOB_TYPE_AMEND,
+    .run           = blockdev_amend_run,
+};
+
+void qmp_x_blockdev_amend(const char *job_id,
+                        const char *node_name,
+                        BlockdevCreateOptions *options,
+                        bool has_force,
+                        bool force,
+                        Error **errp)
+{
+    BlockdevAmendJob *s;
+    const char *fmt = BlockdevDriver_str(options->driver);
+    BlockDriver *drv = bdrv_find_format(fmt);
+    BlockDriverState *bs = bdrv_find_node(node_name);
+
+    /*
+     * If the driver is in the schema, we know that it exists. But it may not
+     * be whitelisted.
+     */
+    assert(drv);
+    if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+        error_setg(errp, "Driver is not whitelisted");
+        return;
+    }
+
+    if (bs->drv != drv) {
+        error_setg(errp,
+                   "x-blockdev-amend doesn't support changing the block driver");
+        return;
+
+    }
+

block/amend.c line 102

Quoted 10 lines
block/amend.c new file +10 -0
+    /* Error out if the driver doesn't support .bdrv_co_amend */
+    if (!drv->bdrv_co_amend) {
+        error_setg(errp, "Driver does not support x-blockdev-amend");
+        return;
+    }
+
+    /*
+     * Create the block job
+     * TODO Running in the main context. Block drivers need to error out or add
+     * locking when they use a BDS in a different AioContext.

Why shouldn’t the job just run in the node’s context?

Max

#25 lore raw

Re: [PATCH v2 07/11] block: add x-blockdev-amend qmp command

Quoted 11 lines · Max Reitz
Quoted 9 lines · Maxim Levitsky
---
 block/Makefile.objs       |   2 +-
 block/amend.c             | 116 ++++++++++++++++++++++++++++++++++++++
 include/block/block_int.h |  23 ++++++--
 qapi/block-core.json      |  26 +++++++++
 qapi/job.json             |   4 +-
 5 files changed, 163 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c
Quoted 2 lines

I think I’d move this (and everything to belongs to it) to a different series.

I already kind of do this, patches prior to this one fully implment the existing amend code path, while this and patches after this one implement the qmp x-blockdev-amend code path.

i don't mind sending this as two separate patch series, now that first refactoring patch series is committed upsteam.

Quoted 66 lines
Quoted 66 lines
block/Makefile.objs +1 -1
index 35f3bca4d9..10d0308792 100644
@@ -18,7 +18,7 @@ block-obj-y += block-backend.o snapshot.o qapi.o
 block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
-block-obj-y += null.o mirror.o commit.o io.o create.o
+block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
 
block/amend.c new file +46 -0
new file mode 100644
index 0000000000..9bd28e08e7
@@ -0,0 +1,116 @@
+/*
+ * Block layer code related to image options amend
+ *
+ * Copyright (c) 2018 Kevin Wolf <[email protected]>
+ * Copyright (c) 2019 Maxim Levitsky <[email protected]>
+ *
+ * Heavily based on create.c
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qemu/job.h"
+#include "qemu/main-loop.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-visit-block-core.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevAmendJob {
+    Job common;
+    BlockdevCreateOptions *opts;
+    BlockDriverState *bs;
+    bool force;
+} BlockdevAmendJob;
+
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
Quoted 8 lines

block/amend.c line 51

Quoted 5 lines
block/amend.c new file +5 -0
+    int ret;
+
+    job_progress_set_remaining(&s->common, 1);
+    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+    job_progress_update(&s->common, 1);

It would be nice if the amend job could make use of the progress reporting that we have in place for amend.

I also thought about it, but is it worth it?

I looked through the status reporting of the qcow2 amend code (which doesn't really allowed to be run through qmp blockdev-amend, due to complexity of changing the qcow2 format on the fly).

Quoted 46 lines
Quoted 46 lines
block/amend.c new file +45 -0
+
+    qapi_free_BlockdevCreateOptions(s->opts);
+
+    return ret;
+}
+
+static const JobDriver blockdev_amend_job_driver = {
+    .instance_size = sizeof(BlockdevAmendJob),
+    .job_type      = JOB_TYPE_AMEND,
+    .run           = blockdev_amend_run,
+};
+
+void qmp_x_blockdev_amend(const char *job_id,
+                        const char *node_name,
+                        BlockdevCreateOptions *options,
+                        bool has_force,
+                        bool force,
+                        Error **errp)
+{
+    BlockdevAmendJob *s;
+    const char *fmt = BlockdevDriver_str(options->driver);
+    BlockDriver *drv = bdrv_find_format(fmt);
+    BlockDriverState *bs = bdrv_find_node(node_name);
+
+    /*
+     * If the driver is in the schema, we know that it exists. But it may not
+     * be whitelisted.
+     */
+    assert(drv);
+    if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+        error_setg(errp, "Driver is not whitelisted");
+        return;
+    }
+
+    if (bs->drv != drv) {
+        error_setg(errp,
+                   "x-blockdev-amend doesn't support changing the block driver");
+        return;
+
+    }
+
+    /* Error out if the driver doesn't support .bdrv_co_amend */
+    if (!drv->bdrv_co_amend) {
+        error_setg(errp, "Driver does not support x-blockdev-amend");
+        return;
Quoted 8 lines

block/amend.c line 102

Quoted 6 lines
block/amend.c new file +6 -0
+    }
+
+    /*
+     * Create the block job
+     * TODO Running in the main context. Block drivers need to error out or add
+     * locking when they use a BDS in a different AioContext.

Why shouldn’t the job just run in the node’s context?

This is shameless copy&pasta from the blockdev-create code (which I did note in the copyright of the file)

Best regards,
Maxim Levitsky

Quoted 3 lines

Max

#26 lore raw

Re: [PATCH v2 07/11] block: add x-blockdev-amend qmp command

Quoted 11 lines · Maxim Levitsky
Quoted 10 lines · Max Reitz
Quoted 9 lines · Maxim Levitsky
---
 block/Makefile.objs       |   2 +-
 block/amend.c             | 116 ++++++++++++++++++++++++++++++++++++++
 include/block/block_int.h |  23 ++++++--
 qapi/block-core.json      |  26 +++++++++
 qapi/job.json             |   4 +-
 5 files changed, 163 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c

[...]

Quoted 12 lines
Quoted 11 lines

block/amend.c line 51

Quoted 8 lines
block/amend.c new file +8 -0
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
+    int ret;
+
+    job_progress_set_remaining(&s->common, 1);
+    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+    job_progress_update(&s->common, 1);

It would be nice if the amend job could make use of the progress reporting that we have in place for amend.

Quoted 6 lines

I also thought about it, but is it worth it?

I looked through the status reporting of the qcow2 amend code (which doesn't really allowed to be run through qmp blockdev-amend, due to complexity of changing the qcow2 format on the fly).

True, and we could always add it later.

I suppose I was mostly wondering because bdrv_amend_options already has all of that infrastructure and I was assuming that qcow2's bdrv_co_amend implementation would make some use of the existing function. Well, it doesn’t, so *shrug*

[...]

Quoted 9 lines
Quoted 6 lines

block/amend.c line 102

Quoted 4 lines
block/amend.c new file +4 -0
+    /*
+     * Create the block job
+     * TODO Running in the main context. Block drivers need to error out or add
+     * locking when they use a BDS in a different AioContext.

Why shouldn’t the job just run in the node’s context?

This is shameless copy&pasta from the blockdev-create code (which I did note in the copyright of the file)

Well, you noted that it’s heavily based on it, not that it’s just C&P.

So I suppose the comment is just wrong here?

Max

#27 lore raw

Re: [PATCH v2 07/11] block: add x-blockdev-amend qmp command

Quoted 48 lines · Max Reitz
Quoted 11 lines · Maxim Levitsky
Quoted 10 lines · Max Reitz
Quoted 9 lines · Maxim Levitsky
---
 block/Makefile.objs       |   2 +-
 block/amend.c             | 116 ++++++++++++++++++++++++++++++++++++++
 include/block/block_int.h |  23 ++++++--
 qapi/block-core.json      |  26 +++++++++
 qapi/job.json             |   4 +-
 5 files changed, 163 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c

[...]

Quoted 12 lines
Quoted 11 lines

block/amend.c line 51

Quoted 8 lines
block/amend.c new file +8 -0
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
+    int ret;
+
+    job_progress_set_remaining(&s->common, 1);
+    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+    job_progress_update(&s->common, 1);

It would be nice if the amend job could make use of the progress reporting that we have in place for amend.

Quoted 6 lines

I also thought about it, but is it worth it?

I looked through the status reporting of the qcow2 amend code (which doesn't really allowed to be run through qmp blockdev-amend, due to complexity of changing the qcow2 format on the fly).

True, and we could always add it later.

I suppose I was mostly wondering because bdrv_amend_options already has all of that infrastructure and I was assuming that qcow2's bdrv_co_amend implementation would make some use of the existing function. Well, it doesn’t, so *shrug*

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
block/amend.c new file +4 -0
+    /*
+     * Create the block job
+     * TODO Running in the main context. Block drivers need to error out or add
+     * locking when they use a BDS in a different AioContext.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Why shouldn’t the job just run in the node’s context?

This is shameless copy&pasta from the blockdev-create code (which I did note in the copyright of the file)

Well, you noted that it’s heavily based on it, not that it’s just C&P.

'heavily based' is a polite way to say that file is copied and then changed to fit new purpose, isn't it :-)

Quoted 2 lines

So I suppose the comment is just wrong here?

Yes, I absolutely missed this part since I don't know the block layer well enough. Thanks for explaining this on IRC, that blockdev_create job is special in a sense that block device state doesn't exit yet for it (I also was already bitten by this in luks block driver), thus the job runs in main AIO context. No need to do so in amend, and thus I'll remove that wrongly copy&pasta'ed comment.

Thanks a lot!!

Best regards,
Maxim Levitsky

Quoted 3 lines

Max

#28 lore raw

Re: [PATCH v2 07/11] block: add x-blockdev-amend qmp command

In reply to Max Reitz
Quoted 11 lines · Max Reitz
Quoted 9 lines · Maxim Levitsky
---
 block/Makefile.objs       |   2 +-
 block/amend.c             | 116 ++++++++++++++++++++++++++++++++++++++
 include/block/block_int.h |  23 ++++++--
 qapi/block-core.json      |  26 +++++++++
 qapi/job.json             |   4 +-
 5 files changed, 163 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c
Quoted 2 lines

I think I’d move this (and everything to belongs to it) to a different series.

I already kind of do this, patches prior to this one fully implement the existing amend code path, while this and patches after this one implement the qmp x-blockdev-amend code path.

i don't mind sending this as two separate patch series, now that first refactoring patch series is committed upsteam.

Quoted 66 lines
Quoted 66 lines
block/Makefile.objs +1 -1
index 35f3bca4d9..10d0308792 100644
@@ -18,7 +18,7 @@ block-obj-y += block-backend.o snapshot.o qapi.o
 block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
-block-obj-y += null.o mirror.o commit.o io.o create.o
+block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
 
block/amend.c new file +46 -0
new file mode 100644
index 0000000000..9bd28e08e7
@@ -0,0 +1,116 @@
+/*
+ * Block layer code related to image options amend
+ *
+ * Copyright (c) 2018 Kevin Wolf <[email protected]>
+ * Copyright (c) 2019 Maxim Levitsky <[email protected]>
+ *
+ * Heavily based on create.c
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qemu/job.h"
+#include "qemu/main-loop.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-visit-block-core.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevAmendJob {
+    Job common;
+    BlockdevCreateOptions *opts;
+    BlockDriverState *bs;
+    bool force;
+} BlockdevAmendJob;
+
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
Quoted 8 lines

block/amend.c line 51

Quoted 5 lines
block/amend.c new file +5 -0
+    int ret;
+
+    job_progress_set_remaining(&s->common, 1);
+    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+    job_progress_update(&s->common, 1);

It would be nice if the amend job could make use of the progress reporting that we have in place for amend.

I also thought about it, but is it worth it?

I looked through the status reporting of the qcow2 amend code (which doesn't really allowed to be run through qmp blockdev-amend, due to complexity of changing the qcow2 format on the fly).

Quoted 46 lines
Quoted 46 lines
block/amend.c new file +45 -0
+
+    qapi_free_BlockdevCreateOptions(s->opts);
+
+    return ret;
+}
+
+static const JobDriver blockdev_amend_job_driver = {
+    .instance_size = sizeof(BlockdevAmendJob),
+    .job_type      = JOB_TYPE_AMEND,
+    .run           = blockdev_amend_run,
+};
+
+void qmp_x_blockdev_amend(const char *job_id,
+                        const char *node_name,
+                        BlockdevCreateOptions *options,
+                        bool has_force,
+                        bool force,
+                        Error **errp)
+{
+    BlockdevAmendJob *s;
+    const char *fmt = BlockdevDriver_str(options->driver);
+    BlockDriver *drv = bdrv_find_format(fmt);
+    BlockDriverState *bs = bdrv_find_node(node_name);
+
+    /*
+     * If the driver is in the schema, we know that it exists. But it may not
+     * be whitelisted.
+     */
+    assert(drv);
+    if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+        error_setg(errp, "Driver is not whitelisted");
+        return;
+    }
+
+    if (bs->drv != drv) {
+        error_setg(errp,
+                   "x-blockdev-amend doesn't support changing the block driver");
+        return;
+
+    }
+
+    /* Error out if the driver doesn't support .bdrv_co_amend */
+    if (!drv->bdrv_co_amend) {
+        error_setg(errp, "Driver does not support x-blockdev-amend");
+        return;
Quoted 8 lines

block/amend.c line 102

Quoted 6 lines
block/amend.c new file +6 -0
+    }
+
+    /*
+     * Create the block job
+     * TODO Running in the main context. Block drivers need to error out or add
+     * locking when they use a BDS in a different AioContext.

Why shouldn’t the job just run in the node’s context?

This is shameless copy&pasta from the blockdev-create code (which I did note in the copyright of the file)

Best regards,
Maxim Levitsky

Quoted 3 lines

Max

#29 lore raw

Re: [Qemu-devel] [PATCH v2 07/11] block: add x-blockdev-amend qmp command

In reply to Maxim Levitsky
Quoted 2 lines · Maxim Levitsky

---

[...]

Quoted 45 lines
qapi/block-core.json +26 -0
index e6edd641f1..7900914506 100644
@@ -4650,6 +4650,32 @@
   'data': { 'job-id': 'str',
             'options': 'BlockdevCreateOptions' } }
 
+##
+# @x-blockdev-amend:
+#
+# Starts a job to amend format specific options of an existing open block device.
+# The job is automatically finalized, but a manual job-dismiss is required.
+#
+# @job-id:          Identifier for the newly created job.
+#
+# @node-name:       Name of the block node to work on
+#
+# @options:         Options (same as for image creation)
+#
+# @force:           Allow unsafe operations, format specific
+#                   For luks that allows erase of the last active keyslot
+#                   (permanent loss of data),
+#                   and replacement of an active keyslot
+#                   (possible loss of data if IO error happens)
+#
+# Since: 4.2
+##
+{ 'command': 'x-blockdev-amend',
+  'data': { 'job-id': 'str',
+            'node-name': 'str',
+            'options': 'BlockdevCreateOptions',
+            '*force': 'bool' } }
+
 ##
 # @blockdev-open-tray:
 #
qapi/job.json +0 -0
index a121b615fb..342d29a7aa 100644
@@ -19,10 +19,12 @@
 #
 # @create: image creation job type, see "blockdev-create" (since 3.0)
 #

qapi/job.json line 30

Quoted 10 lines
qapi/job.json +3 -1
+# @amend: image options amend job type, see "x-blockdev-amend" (since 4.2)
+#
 # Since: 1.7
 ##
 { 'enum': 'JobType',
-  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] }
 
 ##
 # @JobStatus:

Aha, you introduce "amend" after using the concept in PATCH 02. Rather confusing, I'm afraid.

I guess secret deletion makes sense with amend somehow, and only with amend. If yes, the QAPI documentaion should spell that out, and the code should diagnose invalid use (maybe it does already; I haven't looked at it).

#30 lore raw

Re: [Qemu-devel] [PATCH v2 07/11] block: add x-blockdev-amend qmp command

Quoted 65 lines · Markus Armbruster
Quoted 2 lines · Maxim Levitsky

---

[...]

Quoted 45 lines
qapi/block-core.json +26 -0
index e6edd641f1..7900914506 100644
@@ -4650,6 +4650,32 @@
   'data': { 'job-id': 'str',
             'options': 'BlockdevCreateOptions' } }
 
+##
+# @x-blockdev-amend:
+#
+# Starts a job to amend format specific options of an existing open block device.
+# The job is automatically finalized, but a manual job-dismiss is required.
+#
+# @job-id:          Identifier for the newly created job.
+#
+# @node-name:       Name of the block node to work on
+#
+# @options:         Options (same as for image creation)
+#
+# @force:           Allow unsafe operations, format specific
+#                   For luks that allows erase of the last active keyslot
+#                   (permanent loss of data),
+#                   and replacement of an active keyslot
+#                   (possible loss of data if IO error happens)
+#
+# Since: 4.2
+##
+{ 'command': 'x-blockdev-amend',
+  'data': { 'job-id': 'str',
+            'node-name': 'str',
+            'options': 'BlockdevCreateOptions',
+            '*force': 'bool' } }
+
 ##
 # @blockdev-open-tray:
 #
qapi/job.json +0 -0
index a121b615fb..342d29a7aa 100644
@@ -19,10 +19,12 @@
 #
 # @create: image creation job type, see "blockdev-create" (since 3.0)
 #

qapi/job.json line 30

Quoted 10 lines
qapi/job.json +3 -1
+# @amend: image options amend job type, see "x-blockdev-amend" (since 4.2)
+#
 # Since: 1.7
 ##
 { 'enum': 'JobType',
-  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] }
 
 ##
 # @JobStatus:

Aha, you introduce "amend" after using the concept in PATCH 02. Rather confusing, I'm afraid.

Quoted 4 lines

I guess secret deletion makes sense with amend somehow, and only with amend. If yes, the QAPI documentaion should spell that out, and the code should diagnose invalid use (maybe it does already; I haven't looked at it).

I think I did mentioned that in QAPI,
and I do check for optional arguments in the code.

I will probably stop using the blockdev-create options for amend after all.

Best regards,
Maxim Levitsky

#31 lore raw

PATCH v2 8/11 block/crypto: implement blockdev-amend

In reply to Maxim Levitsky
---
 block/crypto.c       | 85 ++++++++++++++++++++++++++++++++++----------
 qapi/block-core.json |  7 ++--
 2 files changed, 71 insertions(+), 21 deletions(-)
block/crypto.c +66 -19
index f42fa057e6..5905f7f520 100644
@@ -534,6 +534,17 @@ block_crypto_co_create_luks(BlockdevCreateOptions *create_options, Error **errp)
     assert(create_options->driver == BLOCKDEV_DRIVER_LUKS);
     luks_opts = &create_options->u.luks;
 
+    if (!luks_opts->has_size) {
+        error_setg(errp, "'size' is manadatory for image creation");
+        return -EINVAL;
+    }
+
+    if (!luks_opts->has_file) {
+        error_setg(errp, "'file' is manadatory for image creation");
+        return -EINVAL;
+    }
+
+
     bs = bdrv_open_blockdev_ref(luks_opts->file, errp);
     if (bs == NULL) {
         return -EIO;
@@ -667,6 +678,39 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
 }
 
 
+static int
+block_crypto_amend_options_generic(BlockDriverState *bs,
+                                   QCryptoBlockCreateOptions *amend_options,
+                                   bool force,
+                                   Error **errp)
+{
+    BlockCrypto *crypto = bs->opaque;
+    int ret = -1;
+
+    assert(crypto);
+    assert(crypto->block);
+
+    /* apply for exclusive write permissions to the underlying file*/
+    crypto->updating_keys = true;
+    ret = bdrv_child_refresh_perms(bs, bs->file, errp);
+    if (ret) {
+        goto cleanup;
+    }
+
+    ret = qcrypto_block_amend_options(crypto->block,
+                                      block_crypto_read_func,
+                                      block_crypto_write_func,
+                                      bs,
+                                      amend_options,
+                                      force,
+                                      errp);
+cleanup:
+    /* release exclusive write permissions to the underlying file*/
+    crypto->updating_keys = false;
+    bdrv_child_refresh_perms(bs, bs->file, errp);
+    return ret;
+}
+
 static int
 block_crypto_amend_options(BlockDriverState *bs,
                            QemuOpts *opts,
@@ -678,44 +722,45 @@ block_crypto_amend_options(BlockDriverState *bs,
     BlockCrypto *crypto = bs->opaque;
     QDict *cryptoopts = NULL;
     QCryptoBlockCreateOptions *amend_options = NULL;
-    int ret;
+    int ret = -EINVAL;
 
     assert(crypto);
     assert(crypto->block);
 
-    crypto->updating_keys = true;
-
-    ret = bdrv_child_refresh_perms(bs, bs->file, errp);
-    if (ret < 0) {
-        goto cleanup;
-    }
-
     cryptoopts = qemu_opts_to_qdict_filtered(opts, NULL,
                                              &block_crypto_create_opts_luks,
                                              true);
 
     qdict_put_str(cryptoopts, "format", "luks");
     amend_options = block_crypto_create_opts_init(cryptoopts, errp);
+
     if (!amend_options) {
-        ret = -EINVAL;
         goto cleanup;
     }
 
-    ret = qcrypto_block_amend_options(crypto->block,
-                                      block_crypto_read_func,
-                                      block_crypto_write_func,
-                                      bs,
-                                      amend_options,
-                                      force,
-                                      errp);
+    ret = block_crypto_amend_options_generic(bs, amend_options, force, errp);
 cleanup:
-    crypto->updating_keys = false;
-    bdrv_child_refresh_perms(bs, bs->file, errp);
     qapi_free_QCryptoBlockCreateOptions(amend_options);
     qobject_unref(cryptoopts);
     return ret;
 }
 
+static int
+coroutine_fn block_crypto_co_amend(BlockDriverState *bs,
+                                   BlockdevCreateOptions *opts,
+                                   bool force,
+                                   Error **errp)
+{
+    QCryptoBlockCreateOptions amend_opts;
+
+    amend_opts = (QCryptoBlockCreateOptions) {
+        .format = Q_CRYPTO_BLOCK_FORMAT_LUKS,
+        .u.luks = *qapi_BlockdevCreateOptionsLUKS_base(&opts->u.luks),
+    };
+
+    return block_crypto_amend_options_generic(bs, &amend_opts, force, errp);
+}
+
 
 static void
 block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
@@ -750,7 +795,8 @@ block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
      */
 
     if (crypto->updating_keys) {
-        /*need exclusive write access for header update  */
+        assert(!(bs->open_flags & BDRV_O_NO_IO));
+        /*need exclusive read and write access for header update  */
         perm |= BLK_PERM_WRITE;
         shared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
     }
@@ -786,6 +832,7 @@ static BlockDriver bdrv_crypto_luks = {
     .bdrv_get_info      = block_crypto_get_info_luks,
     .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
     .bdrv_amend_options = block_crypto_amend_options,
+    .bdrv_co_amend      = block_crypto_co_amend,
 
     .strong_runtime_opts = block_crypto_strong_runtime_opts,
 };
qapi/block-core.json +5 -2
index 7900914506..4a6db98938 100644
@@ -4211,8 +4211,11 @@
 # Driver specific image creation options for LUKS.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @preallocation    Preallocation mode for the new image
+#                   Only for create
 #                   (since: 4.2)
 #                   (default: off; allowed values: off, metadata, falloc, full)
 #
@@ -4220,8 +4223,8 @@
 ##
 { 'struct': 'BlockdevCreateOptionsLUKS',
   'base': 'QCryptoBlockCreateOptionsLUKS',
-  'data': { 'file':             'BlockdevRef',
-            'size':             'size',
+  'data': { '*file':             'BlockdevRef',
+            '*size':             'size',
             '*preallocation':   'PreallocMode' } }
 
 ##
Signature · 1 line
2.17.2
#32 lore raw

Re: [Qemu-devel] [PATCH v2 08/11] block/crypto: implement blockdev-amend

Quoted 3 lines · Maxim Levitsky

---

[...]

Quoted 17 lines
qapi/block-core.json +3 -0
index 7900914506..4a6db98938 100644
@@ -4211,8 +4211,11 @@
 # Driver specific image creation options for LUKS.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @preallocation    Preallocation mode for the new image
+#                   Only for create
 #                   (since: 4.2)
 #                   (default: off; allowed values: off, metadata, falloc, full)
 #

qapi/block-core.json line 4230

Quoted 10 lines
qapi/block-core.json +2 -2
@@ -4220,8 +4223,8 @@
 ##
 { 'struct': 'BlockdevCreateOptionsLUKS',
   'base': 'QCryptoBlockCreateOptionsLUKS',
-  'data': { 'file':             'BlockdevRef',
-            'size':             'size',
+  'data': { '*file':             'BlockdevRef',
+            '*size':             'size',
             '*preallocation':   'PreallocMode' } }
 
 ##

Why is this change needed?

When the commit message says "implement FOO" and nothing else, then I don't expect QAPI schema changes. Working the answer to my question into the commit message might avoid the surprise.

#33 lore raw

Re: [Qemu-devel] [PATCH v2 08/11] block/crypto: implement blockdev-amend

Quoted 33 lines · Markus Armbruster
Quoted 3 lines · Maxim Levitsky

---

[...]

Quoted 26 lines
qapi/block-core.json +5 -2
index 7900914506..4a6db98938 100644
@@ -4211,8 +4211,11 @@
 # Driver specific image creation options for LUKS.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @preallocation    Preallocation mode for the new image
+#                   Only for create
 #                   (since: 4.2)
 #                   (default: off; allowed values: off, metadata, falloc, full)
 #
@@ -4220,8 +4223,8 @@
 ##
 { 'struct': 'BlockdevCreateOptionsLUKS',
   'base': 'QCryptoBlockCreateOptionsLUKS',
-  'data': { 'file':             'BlockdevRef',
-            'size':             'size',
+  'data': { '*file':             'BlockdevRef',
+            '*size':             'size',
             '*preallocation':   'PreallocMode' } }
 
Quoted 7 lines

qapi/block-core.json line 4230

Quoted 1 line
qapi/block-core.json +0 -0
 ##

Why is this change needed?

When the commit message says "implement FOO" and nothing else, then I don't expect QAPI schema changes. Working the answer to my question into the commit message might avoid the surprise.

It is because for the amend interface, it doesn't make much sense to pass the size, and the underlying block device.

Thats why I made these optional but I check that these parameters are present on creation and not present on amend.

Still I am more and more inclined to think that I should not use the create options for amend, but rather split into a new structure. This is just not worth it IMHO.

Best regards,
Maxim Levitsky

#34 lore raw

PATCH v2 9/11 block/qcow2: implement blockdev-amend

In reply to Maxim Levitsky

Currently only for changing crypto parameters

---
 block/qcow2.c        | 71 ++++++++++++++++++++++++++++++++++++++++++++
 qapi/block-core.json |  6 ++--
 2 files changed, 75 insertions(+), 2 deletions(-)
block/qcow2.c +71 -0
index 26f83aeb44..c8847ec6e2 100644
@@ -3079,6 +3079,18 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
     assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
     qcow2_opts = &create_options->u.qcow2;
 
+    if (!qcow2_opts->has_size) {
+        error_setg(errp, "Size is manadatory for image creation");
+        return -EINVAL;
+
+    }
+
+    if (!qcow2_opts->has_file) {
+        error_setg(errp, "'file' is manadatory for image creation");
+        return -EINVAL;
+
+    }
+
     bs = bdrv_open_blockdev_ref(qcow2_opts->file, errp);
     if (bs == NULL) {
         return -EIO;
@@ -5111,6 +5123,64 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     return 0;
 }
 
+
+static int coroutine_fn qcow2_co_amend(BlockDriverState *bs,
+                                       BlockdevCreateOptions *opts,
+                                       bool force,
+                                       Error **errp)
+{
+    BlockdevCreateOptionsQcow2 *qopts = &opts->u.qcow2;
+    BDRVQcow2State *s = bs->opaque;
+    int ret;
+
+    /*
+     * This is ugly as hell, in later versions of this patch
+     * something has to be done about this
+     */
+    if (qopts->has_file || qopts->has_size || qopts->has_data_file ||
+        qopts->has_data_file_raw || qopts->has_version ||
+        qopts->has_backing_file || qopts->has_backing_fmt ||
+        qopts->has_cluster_size || qopts->has_preallocation ||
+        qopts->has_lazy_refcounts || qopts->has_refcount_bits) {
+
+        error_setg(errp,
+                "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
+        return -EOPNOTSUPP;
+
+    }
+
+    if (qopts->has_encrypt) {
+        if (!s->crypto) {
+            error_setg(errp, "QCOW2 image is not encrypted, can't amend");
+            return -EOPNOTSUPP;
+        }
+
+        if (qopts->encrypt->format != Q_CRYPTO_BLOCK_FORMAT_LUKS) {
+            error_setg(errp,
+                       "Amend can't be used to change the qcow2 encryption format");
+            return -EOPNOTSUPP;
+        }
+
+        if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+            error_setg(errp,
+                       "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
+            return -EOPNOTSUPP;
+        }
+
+        ret = qcrypto_block_amend_options(s->crypto,
+                                          qcow2_crypto_hdr_read_func,
+                                          qcow2_crypto_hdr_write_func,
+                                          bs,
+                                          qopts->encrypt,
+                                          force,
+                                          errp);
+        if (ret) {
+            return ret;
+        }
+    }
+    return 0;
+}
+
 /*
  * If offset or size are negative, respectively, they will not be included in
  * the BLOCK_IMAGE_CORRUPTED event emitted.
@@ -5303,6 +5373,7 @@ BlockDriver bdrv_qcow2 = {
     .mutable_opts        = mutable_opts,
     .bdrv_co_check       = qcow2_co_check,
     .bdrv_amend_options  = qcow2_amend_options,
+    .bdrv_co_amend       = qcow2_co_amend,
 
     .bdrv_detach_aio_context  = qcow2_detach_aio_context,
     .bdrv_attach_aio_context  = qcow2_attach_aio_context,
qapi/block-core.json +4 -2
index 4a6db98938..0eb4e45168 100644
@@ -4294,6 +4294,7 @@
 # Driver specific image creation options for qcow2.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @data-file        Node to use as an external data file in which all guest
 #                   data is stored so that only metadata remains in the qcow2
 #                   file (since: 4.0)
@@ -4301,6 +4302,7 @@
 #                   standalone (read-only) raw image without looking at qcow2
 #                   metadata (default: false; since: 4.0)
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @version          Compatibility level (default: v3)
 # @backing-file     File name of the backing file if a backing file
 #                   should be used
@@ -4315,10 +4317,10 @@
 # Since: 2.12
 ##
 { 'struct': 'BlockdevCreateOptionsQcow2',
-  'data': { 'file':             'BlockdevRef',
+  'data': { '*file':            'BlockdevRef',
             '*data-file':       'BlockdevRef',
             '*data-file-raw':   'bool',
-            'size':             'size',
+            '*size':            'size',
             '*version':         'BlockdevQcow2Version',
             '*backing-file':    'str',
             '*backing-fmt':     'BlockdevDriver',
Signature · 1 line
2.17.2
#35 lore raw

Re: [PATCH v2 09/11] block/qcow2: implement blockdev-amend

Quoted 1 line · Maxim Levitsky

Currently only for changing crypto parameters

Yep, that elegantly avoids most of the problems we’d have otherwise. :-)

Quoted 36 lines
---
 block/qcow2.c        | 71 ++++++++++++++++++++++++++++++++++++++++++++
 qapi/block-core.json |  6 ++--
 2 files changed, 75 insertions(+), 2 deletions(-)
block/qcow2.c +15 -0
index 26f83aeb44..c8847ec6e2 100644
@@ -3079,6 +3079,18 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
     assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
     qcow2_opts = &create_options->u.qcow2;
 
+    if (!qcow2_opts->has_size) {
+        error_setg(errp, "Size is manadatory for image creation");
+        return -EINVAL;
+
+    }
+
+    if (!qcow2_opts->has_file) {
+        error_setg(errp, "'file' is manadatory for image creation");
+        return -EINVAL;
+
+    }
+
     bs = bdrv_open_blockdev_ref(qcow2_opts->file, errp);
     if (bs == NULL) {
         return -EIO;
@@ -5111,6 +5123,64 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     return 0;
 }
 
+
+static int coroutine_fn qcow2_co_amend(BlockDriverState *bs,
+                                       BlockdevCreateOptions *opts,

block/qcow2.c static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, · line 5138

Quoted 10 lines
block/qcow2.c +10 -0
+                                       bool force,
+                                       Error **errp)
+{
+    BlockdevCreateOptionsQcow2 *qopts = &opts->u.qcow2;
+    BDRVQcow2State *s = bs->opaque;
+    int ret;
+
+    /*
+     * This is ugly as hell, in later versions of this patch
+     * something has to be done about this

Well, at least the language of the comment needs adjustment. :-)

Quoted 33 lines
block/qcow2.c +33 -0
+     */
+    if (qopts->has_file || qopts->has_size || qopts->has_data_file ||
+        qopts->has_data_file_raw || qopts->has_version ||
+        qopts->has_backing_file || qopts->has_backing_fmt ||
+        qopts->has_cluster_size || qopts->has_preallocation ||
+        qopts->has_lazy_refcounts || qopts->has_refcount_bits) {
+
+        error_setg(errp,
+                "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
+        return -EOPNOTSUPP;
+
+    }
+
+    if (qopts->has_encrypt) {
+        if (!s->crypto) {
+            error_setg(errp, "QCOW2 image is not encrypted, can't amend");
+            return -EOPNOTSUPP;
+        }
+
+        if (qopts->encrypt->format != Q_CRYPTO_BLOCK_FORMAT_LUKS) {
+            error_setg(errp,
+                       "Amend can't be used to change the qcow2 encryption format");
+            return -EOPNOTSUPP;
+        }
+
+        if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+            error_setg(errp,
+                       "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
+            return -EOPNOTSUPP;
+        }
+
+        ret = qcrypto_block_amend_options(s->crypto,
+                                          qcow2_crypto_hdr_read_func,

block/qcow2.c static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, · line 5181

Quoted 10 lines
block/qcow2.c +10 -0
+                                          qcow2_crypto_hdr_write_func,
+                                          bs,
+                                          qopts->encrypt,
+                                          force,
+                                          errp);
+        if (ret) {
+            return ret;
+        }
+    }
+    return 0;

I suppose we need to do the same permission stuff as for LUKS, though; i.e., unshare CONSISTENT_READ while this operation is ongoing.

Quoted 37 lines
block/qcow2.c +3 -0
+}
+
 /*
  * If offset or size are negative, respectively, they will not be included in
  * the BLOCK_IMAGE_CORRUPTED event emitted.
@@ -5303,6 +5373,7 @@ BlockDriver bdrv_qcow2 = {
     .mutable_opts        = mutable_opts,
     .bdrv_co_check       = qcow2_co_check,
     .bdrv_amend_options  = qcow2_amend_options,
+    .bdrv_co_amend       = qcow2_co_amend,
 
     .bdrv_detach_aio_context  = qcow2_detach_aio_context,
     .bdrv_attach_aio_context  = qcow2_attach_aio_context,
qapi/block-core.json +2 -0
index 4a6db98938..0eb4e45168 100644
@@ -4294,6 +4294,7 @@
 # Driver specific image creation options for qcow2.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @data-file        Node to use as an external data file in which all guest
 #                   data is stored so that only metadata remains in the qcow2
 #                   file (since: 4.0)
@@ -4301,6 +4302,7 @@
 #                   standalone (read-only) raw image without looking at qcow2
 #                   metadata (default: false; since: 4.0)
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @version          Compatibility level (default: v3)
 # @backing-file     File name of the backing file if a backing file
 #                   should be used
@@ -4315,10 +4317,10 @@
 # Since: 2.12
 ##

qapi/block-core.json line 4326

Quoted 10 lines
qapi/block-core.json +2 -2
 { 'struct': 'BlockdevCreateOptionsQcow2',
-  'data': { 'file':             'BlockdevRef',
+  'data': { '*file':            'BlockdevRef',
             '*data-file':       'BlockdevRef',
             '*data-file-raw':   'bool',
-            'size':             'size',
+            '*size':            'size',
             '*version':         'BlockdevQcow2Version',
             '*backing-file':    'str',
             '*backing-fmt':     'BlockdevDriver',

Making these fields optional makes me wonder whether it actually make sense to have both create and amend share a single QAPI structure. Wouldn’t it better to have a separate amend structure that then actually reflects what we support? (This would also solve the problem of how to express in the code what is and what isn’t supported through blockdev-amend.)

Max

#36 lore raw

Re: [PATCH v2 09/11] block/qcow2: implement blockdev-amend

Quoted 6 lines · Max Reitz
Quoted 1 line · Maxim Levitsky

Currently only for changing crypto parameters

Yep, that elegantly avoids most of the problems we’d have otherwise. :-)

Quoted 1 line

[...]

Quoted 26 lines
Quoted 26 lines
qapi/block-core.json +3 -1
index 4a6db98938..0eb4e45168 100644
@@ -4294,6 +4294,7 @@
 # Driver specific image creation options for qcow2.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @data-file        Node to use as an external data file in which all guest
 #                   data is stored so that only metadata remains in the qcow2
 #                   file (since: 4.0)
@@ -4301,6 +4302,7 @@
 #                   standalone (read-only) raw image without looking at qcow2
 #                   metadata (default: false; since: 4.0)
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @version          Compatibility level (default: v3)
 # @backing-file     File name of the backing file if a backing file
 #                   should be used
@@ -4315,10 +4317,10 @@
 # Since: 2.12
 ##
 { 'struct': 'BlockdevCreateOptionsQcow2',
-  'data': { 'file':             'BlockdevRef',
+  'data': { '*file':            'BlockdevRef',
Quoted 8 lines

qapi/block-core.json line 4326

Quoted 8 lines
qapi/block-core.json +1 -1
             '*data-file':       'BlockdevRef',
             '*data-file-raw':   'bool',
-            'size':             'size',
+            '*size':            'size',
             '*version':         'BlockdevQcow2Version',
             '*backing-file':    'str',
             '*backing-fmt':     'BlockdevDriver',

My comments to the previous patch apply.

Quoted 6 lines

Making these fields optional makes me wonder whether it actually make sense to have both create and amend share a single QAPI structure. Wouldn’t it better to have a separate amend structure that then actually reflects what we support? (This would also solve the problem of how to express in the code what is and what isn’t supported through blockdev-amend.)

Good point. As is, the schema is rather confusing, at least to me. We reduce or avoid the confusion in documentation or in code. I'd prefer code unless it leads to too much duplication. "Too much" is of course subjective. Maxim, would you mind exploring it for us?

#37 lore raw

Re: [PATCH v2 09/11] block/qcow2: implement blockdev-amend

Quoted 54 lines · Markus Armbruster
Quoted 6 lines · Max Reitz
Quoted 1 line · Maxim Levitsky

Currently only for changing crypto parameters

Yep, that elegantly avoids most of the problems we’d have otherwise. :-)

Quoted 1 line

[...]

Quoted 26 lines
Quoted 26 lines
qapi/block-core.json +3 -1
index 4a6db98938..0eb4e45168 100644
@@ -4294,6 +4294,7 @@
 # Driver specific image creation options for qcow2.
 #
 # @file             Node to create the image format on
+#                   Mandatory for create
 # @data-file        Node to use as an external data file in which all guest
 #                   data is stored so that only metadata remains in the qcow2
 #                   file (since: 4.0)
@@ -4301,6 +4302,7 @@
 #                   standalone (read-only) raw image without looking at qcow2
 #                   metadata (default: false; since: 4.0)
 # @size             Size of the virtual disk in bytes
+#                   Mandatory for create
 # @version          Compatibility level (default: v3)
 # @backing-file     File name of the backing file if a backing file
 #                   should be used
@@ -4315,10 +4317,10 @@
 # Since: 2.12
 ##
 { 'struct': 'BlockdevCreateOptionsQcow2',
-  'data': { 'file':             'BlockdevRef',
+  'data': { '*file':            'BlockdevRef',
Quoted 8 lines

qapi/block-core.json line 4326

Quoted 8 lines
qapi/block-core.json +1 -1
             '*data-file':       'BlockdevRef',
             '*data-file-raw':   'bool',
-            'size':             'size',
+            '*size':            'size',
             '*version':         'BlockdevQcow2Version',
             '*backing-file':    'str',
             '*backing-fmt':     'BlockdevDriver',

My comments to the previous patch apply.

Quoted 6 lines

Making these fields optional makes me wonder whether it actually make sense to have both create and amend share a single QAPI structure. Wouldn’t it better to have a separate amend structure that then actually reflects what we support? (This would also solve the problem of how to express in the code what is and what isn’t supported through blockdev-amend.)

Quoted 4 lines

Good point. As is, the schema is rather confusing, at least to me. We reduce or avoid the confusion in documentation or in code. I'd prefer code unless it leads to too much duplication. "Too much" is of course subjective. Maxim, would you mind exploring it for us?

Nothing against having a separate amend structure, I actually prefer this, and I don't think it will complicate the code.

Best regards,
Maxim Levitsky

#38 lore raw

Re: [PATCH v2 09/11] block/qcow2: implement blockdev-amend

In reply to Max Reitz
Quoted 38 lines · Max Reitz
Quoted 1 line · Maxim Levitsky

Currently only for changing crypto parameters

Yep, that elegantly avoids most of the problems we’d have otherwise. :-)

Quoted 33 lines
---
 block/qcow2.c        | 71 ++++++++++++++++++++++++++++++++++++++++++++
 qapi/block-core.json |  6 ++--
 2 files changed, 75 insertions(+), 2 deletions(-)
block/qcow2.c +12 -0
index 26f83aeb44..c8847ec6e2 100644
@@ -3079,6 +3079,18 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
     assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
     qcow2_opts = &create_options->u.qcow2;
 
+    if (!qcow2_opts->has_size) {
+        error_setg(errp, "Size is manadatory for image creation");
+        return -EINVAL;
+
+    }
+
+    if (!qcow2_opts->has_file) {
+        error_setg(errp, "'file' is manadatory for image creation");
+        return -EINVAL;
+
+    }
+
     bs = bdrv_open_blockdev_ref(qcow2_opts->file, errp);
     if (bs == NULL) {
         return -EIO;
@@ -5111,6 +5123,64 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     return 0;
 }
 
Quoted 15 lines

block/qcow2.c static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, · line 5138

Quoted 13 lines
block/qcow2.c +13 -0
+
+static int coroutine_fn qcow2_co_amend(BlockDriverState *bs,
+                                       BlockdevCreateOptions *opts,
+                                       bool force,
+                                       Error **errp)
+{
+    BlockdevCreateOptionsQcow2 *qopts = &opts->u.qcow2;
+    BDRVQcow2State *s = bs->opaque;
+    int ret;
+
+    /*
+     * This is ugly as hell, in later versions of this patch
+     * something has to be done about this

Well, at least the language of the comment needs adjustment. :-)

Thats for sure :-)

BTW, if I opt for having a separate amend parameter struct, this will fix this problem as well, as all unsupported fields will just not be there.

[..]

Best regards,
Maxim Levitsky

#39 lore raw

PATCH v2 10/11 iotests: filter few more luks specific create options

In reply to Maxim Levitsky

Those options are test input anyway, and this allows more tests to be able to have same output on both qcow2 luks encrypted images and raw luks images

---
 tests/qemu-iotests/087.out       | 6 +++---
 tests/qemu-iotests/134.out       | 2 +-
 tests/qemu-iotests/158.out       | 4 ++--
 tests/qemu-iotests/188.out       | 2 +-
 tests/qemu-iotests/189.out       | 4 ++--
 tests/qemu-iotests/198.out       | 4 ++--
 tests/qemu-iotests/common.filter | 6 ++++--
 7 files changed, 15 insertions(+), 13 deletions(-)
tests/qemu-iotests/087.out +3 -3
index 2d92ea847b..b61ba638af 100644
@@ -34,7 +34,7 @@ QMP_VERSION
 
 === Encrypted image QCow ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 Testing:
 QMP_VERSION
 {"return": {}}
@@ -46,7 +46,7 @@ QMP_VERSION
 
 === Encrypted image LUKS ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 Testing:
 QMP_VERSION
 {"return": {}}
@@ -58,7 +58,7 @@ QMP_VERSION
 
 === Missing driver ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 Testing: -S
 QMP_VERSION
 {"return": {}}
tests/qemu-iotests/134.out +1 -1
index 09d46f6b17..4abc5b5f7d 100644
@@ -1,5 +1,5 @@
 QA output created by 134
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 
 == reading whole image ==
 read 134217728/134217728 bytes at offset 0
tests/qemu-iotests/158.out +2 -2
index 6def216e55..f28a17626b 100644
@@ -1,6 +1,6 @@
 QA output created by 158
 == create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on
 
 == writing whole image ==
 wrote 134217728/134217728 bytes at offset 0
@@ -10,7 +10,7 @@ wrote 134217728/134217728 bytes at offset 0
 read 134217728/134217728 bytes at offset 0
 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on
 
 == writing part of a cluster ==
 wrote 1024/1024 bytes at offset 0
tests/qemu-iotests/188.out +1 -1
index c568ef3701..5426861b18 100644
@@ -1,5 +1,5 @@
 QA output created by 188
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216
 
 == reading whole image ==
 read 16777216/16777216 bytes at offset 0
tests/qemu-iotests/189.out +2 -2
index a0b7c9c24c..bc213cbe14 100644
@@ -1,6 +1,6 @@
 QA output created by 189
 == create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
 
 == writing whole image ==
 wrote 16777216/16777216 bytes at offset 0
@@ -10,7 +10,7 @@ wrote 16777216/16777216 bytes at offset 0
 read 16777216/16777216 bytes at offset 0
 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
 
 == writing part of a cluster ==
 wrote 1024/1024 bytes at offset 0
tests/qemu-iotests/198.out +2 -2
index e86b175e39..2eff420795 100644
@@ -1,12 +1,12 @@
 QA output created by 198
 == create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
 
 == writing whole image base ==
 wrote 16777216/16777216 bytes at offset 0
 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
 
 == writing whole image layer ==
 wrote 16777216/16777216 bytes at offset 0
tests/qemu-iotests/common.filter +4 -2
index 445a1c23e0..b295a3fec7 100644
@@ -138,8 +138,10 @@ _filter_img_create()
         -e "s# block_state_zero=\\(on\\|off\\)##g" \
         -e "s# log_size=[0-9]\\+##g" \
         -e "s# refcount_bits=[0-9]\\+##g" \
-        -e "s# key-secret=[a-zA-Z0-9]\\+##g" \
-        -e "s# iter-time=[0-9]\\+##g" \
+        -e "s# \\(encrypt\\.\\)\\?key-secret=[a-zA-Z0-9]\\+##g" \
+        -e "s# \\(encrypt\\.\\)\\?slot=[0-9]\\+##g" \
+        -e "s# \\(encrypt\\.\\)\\?iter-time=[0-9]\\+##g" \
+        -e "s# encrypt\\.format=[a-zA-Z0-9]\\+##g" \
         -e "s# force_size=\\(on\\|off\\)##g"
 }
 
Signature · 1 line
2.17.2
#40 lore raw

PATCH v2 11/11 iotests : add tests for encryption key management

In reply to Maxim Levitsky

Note that currently I add tests 300-302, which are placeholders to ease the rebase. In final version of these patches I will update these.

---
 tests/qemu-iotests/300     | 202 +++++++++++++++++++++++++++++
 tests/qemu-iotests/300.out |  98 +++++++++++++++
 tests/qemu-iotests/301     |  90 +++++++++++++
 tests/qemu-iotests/301.out |  30 +++++
 tests/qemu-iotests/302     | 252 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/302.out |  18 +++
 tests/qemu-iotests/303     | 228 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/303.out |  28 +++++
 tests/qemu-iotests/group   |   9 ++
 9 files changed, 955 insertions(+)
 create mode 100755 tests/qemu-iotests/300
 create mode 100644 tests/qemu-iotests/300.out
 create mode 100755 tests/qemu-iotests/301
 create mode 100644 tests/qemu-iotests/301.out
 create mode 100644 tests/qemu-iotests/302
 create mode 100644 tests/qemu-iotests/302.out
 create mode 100644 tests/qemu-iotests/303
 create mode 100644 tests/qemu-iotests/303.out
tests/qemu-iotests/300 new file +202 -0
new file mode 100755
index 0000000000..5b65ef95de
@@ -0,0 +1,202 @@
+#!/usr/bin/env bash
+#
+# Test encryption key management with luks
+# Based on 134
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2 luks
+_supported_proto file #TODO
+
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+if [ "$IMGFMT" = "qcow2" ] ; then
+	PR="encrypt."
+	EXTRA_IMG_ARGS="-o encrypt.format=luks"
+fi
+
+
+# secrets: you are supposed to see the password as *******, see :-)
+S0="--object secret,id=sec0,data=hunter0"
+S1="--object secret,id=sec1,data=hunter1"
+S2="--object secret,id=sec2,data=hunter2"
+S3="--object secret,id=sec3,data=hunter3"
+S4="--object secret,id=sec4,data=hunter4"
+SECRETS="$S0 $S1 $S2 $S3 $S4"
+
+# image with given secret
+IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec0"
+IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec1"
+IMGS2="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec2"
+IMGS3="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec3"
+IMGS4="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec4"
+
+
+echo "== creating a test image =="
+_make_test_img $S4 $EXTRA_IMG_ARGS -o ${PR}key-secret=sec4,${PR}iter-time=10,${PR}slot=4 32M
+
+echo
+echo "== test that key 4 opens the image =="
+$QEMU_IO $S4 -c "read 0 4096" $IMGS4 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== adding a password to slot 0 =="
+$QEMU_IMG amend $SECRETS $IMGS4 -o ${PR}key-secret=sec0,${PR}iter-time=10
+echo "== adding a password to slot 1 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}key-secret=sec1,${PR}iter-time=10
+echo "== adding a password to slot 3 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}key-secret=sec3,${PR}iter-time=10,${PR}slot=3
+echo "== adding a password to slot 2 =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}key-secret=sec2,${PR}iter-time=10
+
+
+echo "== erase slot 4 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}slot=4 | _filter_img_create
+
+
+echo
+echo "== all secrets should work =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== erase slot 0 and try it =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}key-secret=sec0 | _filter_img_create
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== erase slot 2 and try it =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}slot=2| _filter_img_create
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS2 | _filter_qemu_io | _filter_testdir
+
+
+# at this point slots 1 and 3 should be active
+
+echo
+echo "== filling  4 slots with secret 2 =="
+for i in $(seq 0 3) ; do
+	$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}key-secret=sec2,${PR}iter-time=10
+done
+
+echo
+echo "== adding secret 0 =="
+	$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}key-secret=sec0,${PR}iter-time=10
+
+echo
+echo "== adding secret 3 (last slot) =="
+	$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}key-secret=sec3,${PR}iter-time=10
+
+echo
+echo "== trying to add another slot (should fail) =="
+$QEMU_IMG amend $SECRETS $IMGS2 -o ${PR}key-secret=sec3,${PR}iter-time=10
+
+echo
+echo "== all secrets should work again =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== erase all keys of secret 2=="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}key-secret=sec2
+echo "== erase all keys of secret 1=="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}key-secret=sec1
+echo "== erase all keys of secret 0=="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}active=off,${PR}key-secret=sec0
+echo "== erase all keys of secret 3, except a remaining key =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}active=off,${PR}key-secret=sec3
+
+
+echo
+echo "== only secret3 should work now  =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== add secret0  =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}key-secret=sec0,${PR}iter-time=10
+
+echo "== erase secret3 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}active=off,${PR}key-secret=sec3
+
+echo
+echo "== only secret0 should work now  =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== replace secret0 with secret1 (should fail)  =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}key-secret=sec1,${PR}iter-time=10,${PR}slot=0
+
+echo
+echo "== replace secret0 with secret1 with force (should work)  =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}key-secret=sec1,${PR}iter-time=10,${PR}slot=0 --force
+
+echo
+echo "== only secret1 should work now  =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+
+echo
+echo "== erase last secret (should fail)  =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}slot=0
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}key-secret=sec1
+
+
+echo "== erase non existing secrets (should fail)  =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}key-secret=sec5 --force
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}key-secret=sec0 --force
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}slot=1 --force
+
+echo
+echo "== erase last secret with force by slot (should work)  =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}active=off,${PR}slot=0 --force
+
+echo
+echo "== we have no secrets now, data is lost forever =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
+
tests/qemu-iotests/300.out new file +98 -0

98 lines not on this page · raw message

tests/qemu-iotests/301 new file +90 -0

90 lines not on this page · raw message

tests/qemu-iotests/301.out new file +30 -0
new file mode 100644
index 0000000000..e653c30330
@@ -0,0 +1,30 @@
+QA output created by 301
+== creating a test image ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
+
+== test that key 0 opens the image ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== adding a password to slot 1 ==
+
+== 'backup' the image header ==
+1+0 records in
+1+0 records out
+
+== erase slot 0 ==
+
+== test that key 0 doesn't open the image ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== 'restore' the image header ==
+1+0 records in
+1+0 records out
+
+== test that key 0 still doesn't open the image (key material is erased) ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== test that key 1 still works ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+*** done
tests/qemu-iotests/302 new file +252 -0

252 lines not on this page · raw message

tests/qemu-iotests/302.out new file +18 -0
new file mode 100644
index 0000000000..78bfc0864c
@@ -0,0 +1,18 @@
+..
+----------------------------------------------------------------------
+Ran 2 tests
+
+OK
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job1"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job1"}}
+{"return": {}}
+Job failed: Invalid password, cannot unlock any keyslot
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
tests/qemu-iotests/303 new file +228 -0

228 lines not on this page · raw message

tests/qemu-iotests/303.out new file +28 -0
new file mode 100644
index 0000000000..1cf3917208
@@ -0,0 +1,28 @@
+qemu-img: Failed to get shared "consistent read" lock
+Is another process using the image [/home/mlevitsk/USERSPACE/qemu/build-luks/tests/qemu-iotests/scratch/test.img]?
+..
+----------------------------------------------------------------------
+Ran 2 tests
+
+OK
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Block node is read-only
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
tests/qemu-iotests/group +9 -0
index d95d556414..ea96215801 100644
@@ -274,3 +274,12 @@
 257 rw
 258 rw quick
 262 rw quick migration
+
+
+
+
+
+300 rw auto
+301 rw auto quick
+302 rw auto
+303 rw auto
Signature · 1 line
2.17.2
#41 lore raw

Re: [PATCH v2 11/11] iotests : add tests for encryption key management

Quoted 24 lines · Maxim Levitsky

Note that currently I add tests 300-302, which are placeholders to ease the rebase. In final version of these patches I will update these.

---
 tests/qemu-iotests/300     | 202 +++++++++++++++++++++++++++++
 tests/qemu-iotests/300.out |  98 +++++++++++++++
 tests/qemu-iotests/301     |  90 +++++++++++++
 tests/qemu-iotests/301.out |  30 +++++
 tests/qemu-iotests/302     | 252 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/302.out |  18 +++
 tests/qemu-iotests/303     | 228 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/303.out |  28 +++++
 tests/qemu-iotests/group   |   9 ++
 9 files changed, 955 insertions(+)
 create mode 100755 tests/qemu-iotests/300
 create mode 100644 tests/qemu-iotests/300.out
 create mode 100755 tests/qemu-iotests/301
 create mode 100644 tests/qemu-iotests/301.out
 create mode 100644 tests/qemu-iotests/302
 create mode 100644 tests/qemu-iotests/302.out
 create mode 100644 tests/qemu-iotests/303
 create mode 100644 tests/qemu-iotests/303.out

[...]

tests/qemu-iotests/303.out line 2

Quoted 8 lines
tests/qemu-iotests/303.out new file +2 -0
new file mode 100644
index 0000000000..1cf3917208
@@ -0,0 +1,28 @@
+qemu-img: Failed to get shared "consistent read" lock
+Is another process using the image [/home/mlevitsk/USERSPACE/qemu/build-luks/tests/qemu-iotests/scratch/test.img]?

Ah, this should be filtered.

Max

#42 lore raw

Re: [PATCH v2 11/11] iotests : add tests for encryption key management

Quoted 28 lines · Max Reitz
Quoted 24 lines · Maxim Levitsky

Note that currently I add tests 300-302, which are placeholders to ease the rebase. In final version of these patches I will update these.

---
 tests/qemu-iotests/300     | 202 +++++++++++++++++++++++++++++
 tests/qemu-iotests/300.out |  98 +++++++++++++++
 tests/qemu-iotests/301     |  90 +++++++++++++
 tests/qemu-iotests/301.out |  30 +++++
 tests/qemu-iotests/302     | 252 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/302.out |  18 +++
 tests/qemu-iotests/303     | 228 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/303.out |  28 +++++
 tests/qemu-iotests/group   |   9 ++
 9 files changed, 955 insertions(+)
 create mode 100755 tests/qemu-iotests/300
 create mode 100644 tests/qemu-iotests/300.out
 create mode 100755 tests/qemu-iotests/301
 create mode 100644 tests/qemu-iotests/301.out
 create mode 100644 tests/qemu-iotests/302
 create mode 100644 tests/qemu-iotests/302.out
 create mode 100644 tests/qemu-iotests/303
 create mode 100644 tests/qemu-iotests/303.out

[...]

Quoted 10 lines

tests/qemu-iotests/303.out line 2

Quoted 8 lines
tests/qemu-iotests/303.out new file +2 -0
new file mode 100644
index 0000000000..1cf3917208
@@ -0,0 +1,28 @@
+qemu-img: Failed to get shared "consistent read" lock
+Is another process using the image [/home/mlevitsk/USERSPACE/qemu/build-luks/tests/qemu-iotests/scratch/test.img]?

Ah, this should be filtered.

Ooops, missed this one, thanks!
Fixed now.

Quoted 3 lines

Max

Best regards,
Maxim Levitsky

#43 lore raw
In reply to Maxim Levitsky
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 3 lines

Best regards,
Maxim Levitsky

What branch is this based on?
It doesn't seem to apply to origin/master.

--js

#44 lore raw
Quoted 13 lines · John Snow
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 8 lines
Quoted 3 lines

Best regards,
Maxim Levitsky

What branch is this based on?
It doesn't seem to apply to origin/master.

--js

It is based on refactoring patch series I send before,
which is also under review:
"[PATCH v2 00/13] RFC crypto/luks: preparation for encryption key managment"

Best regards,
Maxim Levitsky

#45 lore raw
Quoted 18 lines · Maxim Levitsky
Quoted 17 lines · John Snow
Quoted 14 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Best regards,
Maxim Levitsky

Quoted 7 lines
Quoted 4 lines

What branch is this based on?
It doesn't seem to apply to origin/master.

--js

It is based on refactoring patch series I send before,
which is also under review:
"[PATCH v2 00/13] RFC crypto/luks: preparation for encryption key managment"

Recommend to note such dependencies in the cover letter as

Based-on: <message-id>

#46 lore raw
Quoted 25 lines · Markus Armbruster
Quoted 23 lines · Maxim Levitsky
Quoted 21 lines · John Snow
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 3 lines

Best regards,
Maxim Levitsky

What branch is this based on?
It doesn't seem to apply to origin/master.

--js

Quoted 7 lines
Quoted 3 lines

It is based on refactoring patch series I send before,
which is also under review:
"[PATCH v2 00/13] RFC crypto/luks: preparation for encryption key managment"

Recommend to note such dependencies in the cover letter as

Based-on: <message-id>

I'll take a note!
Note that now all these patches are merged thus, this patch series should more or less apply on top of master branch.
I'll probably resend a V3 after I finish going over the review of this series.

Best regards,
Maxim Levitsky

#47 lore raw
In reply to John Snow
Quoted 13 lines · John Snow
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 8 lines
Quoted 3 lines

Best regards,
Maxim Levitsky

What branch is this based on?
It doesn't seem to apply to origin/master.

--js

Hi!

Most of the refactoring patches are now on the master branch, (one patch was dropped due to me being blind :-(), so should I resend this patch series with the missing patch or wait for some review?

At that stage I would like to hear about agreement/disagreement on the new APIs, and stuff like that.

Best regards,
Maxim Levitsky

#48 lore raw
In reply to Maxim Levitsky
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 2 lines

Best regards,
Maxim Levitsky

At least for an RFC looks good from my perspective. I didn’t look at the crypto things very closely (assuming Dan would do so), and I didn’t check the iotests in detail. (But it definitely doesn’t look like they lack in breadth. Maybe I’d like to see a test that you cannot have other useful nodes attached to the LUKS or qcow2 node while the amendment process is ongoing (because CONSISTENT_READ is unshared). But that’s the only thing I can think of.)

Max

#49 lore raw
Quoted 15 lines · Max Reitz
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 2 lines

Best regards,
Maxim Levitsky

Quoted 7 lines

At least for an RFC looks good from my perspective. I didn’t look at the crypto things very closely (assuming Dan would do so), and I didn’t check the iotests in detail. (But it definitely doesn’t look like they lack in breadth. Maybe I’d like to see a test that you cannot have other useful nodes attached to the LUKS or qcow2 node while the amendment process is ongoing (because CONSISTENT_READ is unshared). But that’s the only thing I can think of.)

Could you elaborate on this?

Inside the same process several users can access that luks node at the same time while one of them changes encryption keys, since this doesn't affect IO of the data.

Two users in same process I was *I think* told that can't do the amend in the same time since qmp is protected with a lock. However since I use a block job (to be consistent with blockdev-create) I wonder if several qmp amend commands couldn't race one with another. These jobs is running on the block device AIO context (I changed this recently after a review), but stil I am not sure there can't be a race.

And when there is access to the same image from multiple processes, I do have a test that checks that as long as more that one process has the image open, noone can change the encryption keys (this is only relevant for raw luks format, since for qcow2 this is forbidden anyway).

I probably missed something though.

Best regards,
Maxim Levitsky

Quoted 3 lines

Max

#50 lore raw
Quoted 34 lines · Maxim Levitsky
Quoted 15 lines · Max Reitz
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 2 lines

Best regards,
Maxim Levitsky

Quoted 7 lines

At least for an RFC looks good from my perspective. I didn’t look at the crypto things very closely (assuming Dan would do so), and I didn’t check the iotests in detail. (But it definitely doesn’t look like they lack in breadth. Maybe I’d like to see a test that you cannot have other useful nodes attached to the LUKS or qcow2 node while the amendment process is ongoing (because CONSISTENT_READ is unshared). But that’s the only thing I can think of.)

Could you elaborate on this?

Inside the same process several users can access that luks node at the same time while one of them changes encryption keys, since this doesn't affect IO of the data.

Two users in same process I was *I think* told that can't do the amend in the same time since qmp is protected with a lock. However since I use a block job (to be consistent with blockdev-create) I wonder if several qmp amend commands couldn't race one with another. These jobs is running on the block device AIO context (I changed this recently after a review), but stil I am not sure there can't be a race.

Quoted 3 lines

And when there is access to the same image from multiple processes, I do have a test that checks that as long as more that one process has the image open, noone can change the encryption keys (this is only relevant for raw luks format, since for qcow2 this is forbidden anyway).

Yes, sorry, I don’t remember/know where I got the qcow2 part from. (I probably just forgot during after reviewing that only LUKS’s permissions are changed by this series.)

But for LUKS, those changed permissions do apply. If you can’t do something between two different qemu instances, you can’t do it in a single one: The file locks are equivalent to the internal permission mask.

So if you can’t change the encryption keys while another process has the image open, you can’t change the encryption keys while another node uses the file node in the same process. For example, you can’t attach two LUKS nodes to a single file node and then change the keys on one of the nodes.

Max

#51 lore raw
Quoted 48 lines · Max Reitz
Quoted 35 lines · Maxim Levitsky
Quoted 15 lines · Max Reitz
Quoted 11 lines · Maxim Levitsky

This patch series is continuation of my work to add encryption key managment to luks/qcow2 with luks.

This is second version of this patch set.
The changes are mostly addressing the review feedback, plus I tested (and fixed sadly) the somewhat ugly code that allows to still write share a raw luks device, while preveting the key managment from happening in this case, as it is unsafe.
I added a new iotest dedicated to that as well.

Quoted 2 lines

Best regards,
Maxim Levitsky

Quoted 7 lines

At least for an RFC looks good from my perspective. I didn’t look at the crypto things very closely (assuming Dan would do so), and I didn’t check the iotests in detail. (But it definitely doesn’t look like they lack in breadth. Maybe I’d like to see a test that you cannot have other useful nodes attached to the LUKS or qcow2 node while the amendment process is ongoing (because CONSISTENT_READ is unshared). But that’s the only thing I can think of.)

Could you elaborate on this?

Inside the same process several users can access that luks node at the same time while one of them changes encryption keys, since this doesn't affect IO of the data.

Two users in same process I was *I think* told that can't do the amend in the same time since qmp is protected with a lock. However since I use a block job (to be consistent with blockdev-create) I wonder if several qmp amend commands couldn't race one with another. These jobs is running on the block device AIO context (I changed this recently after a review), but stil I am not sure there can't be a race.

Quoted 3 lines

And when there is access to the same image from multiple processes, I do have a test that checks that as long as more that one process has the image open, noone can change the encryption keys (this is only relevant for raw luks format, since for qcow2 this is forbidden anyway).

Yes, sorry, I don’t remember/know where I got the qcow2 part from. (I probably just forgot during after reviewing that only LUKS’s permissions are changed by this series.)

But for LUKS, those changed permissions do apply. If you can’t do something between two different qemu instances, you can’t do it in a single one: The file locks are equivalent to the internal permission mask.

Quoted 8 lines

So if you can’t change the encryption keys while another process has the image open, you can’t change the encryption keys while another node uses the file node in the same process. For example, you can’t attach two LUKS nodes to a single file node and then change the keys on one of the nodes.

Max

Ah, I understand now. I'll add a test for that!

Best regards,
Maxim Levitsky