block: Try to create well-typed json:{} filenames
v3 cover block: Try to create well-typed json:{} filenames
This series depends on my "Fix some filename generation issues" series.
Based-on: <[email protected]>
When qemu reports json:{} filename, it just uses whatever type you gave an option in. With -drive, all options are strings and they do not have to pass the test of the typing firewall of the QAPI schema, so you just get strings thrown back at you even if that does not match the schema. (Also, if you use json:{} yourself, you’re free to give the options as strings as well.)
Example:
$ ./qemu-img info --image-opts driver=raw,size=512,file.driver=null-co
image: json:{"driver": "raw", "size": "512", "file": {"driver": "null-co"}}@size is supposed to be an integer, according to the schema, so the correct result would be (which is what you get after this series):
$ ./qemu-img info --image-opts driver=raw,size=512,file.driver=null-co
image: json:{"driver": "raw", "size": 512, "file": {"driver": "null-co"}}This is achieved by patch 6, which makes bdrv_refresh_filename() run the options through the flat-confused input visitor, and then through the output visitor to get all to the correct type. If anything fails, the result is as before (hence the “Try” in the title).
There are cases where this cannot work. Those are the ones where -drive accepts something that is not allowed by the QAPI schema. One of these cases is rbd’s =keyvalue-pairs, which is just broken altogether, so let’s simply ignore that. (I don’t think anybody’s going to complain that the json:{} filename they get is not correctly typed after they’ve used that option.)
The other case (I know of) is qcow(2)’s encryption. In the QAPI schema, encrypt.format is not optional because it is the discriminator for which kind of options to use. However, for -drive, it is optional because the qcow2 driver can infer the encryption format from the image header.
The solution that’s proposed by this series is to make flat union discriminators optional and provide a default. Both AES and LUKS encryption allow only a key-secret option, so we can add a new pseudo-format “auto” that accepts exactly that option and makes the qcow2 driver read the real format from the image header. This pseudo-format is made the default for encrypt.format, and thus you can then specify encrypt.key-secret without having to specify encrypt.format (while still adhering to the QAPI schema).
So, in this series:
- The QAPI code generator is modified to allow optional discriminators for flat unions. In such a case, a default-variant must be supplied to choose when the discriminator value is not present on the wire.
- Accordingly, documentation, tests, and introspection are adjusted.
- This is used to make qcow’s and qcow2’s encrypt.format parameter optional. It now defaults to “from-image” which is a new pseudo-format that allows a key-secret to be given, and otherwise leaves it to the format driver to determine the encryption format.
- json:{} filenames are attempted to be typed correctly when they are generated, by running bs->full_open_options through a healthy mix of qdict_flatten(), the flat-confused input visitor for BlockdevOptions, and the output visitor.
This may not always work but I hope it usually will. Fingers crossed. (At least it won’t make things worse.)
- Tests, tests, tests.
(Yes, I know that “In this series tests, tests, tests.” is not a
sentence.)
v3:
- Rebased
- Dropped the new superfluous empty struct ImageInfoSpecificQCow2EncryptionNoInfo, and moved some of its description to ImageInfoSpecificQCow2Encryption
git-backport-diff against v2:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/8:[0037] [FC] 'qapi: Add default-variant for flat unions' 002/8:[----] [-C] 'docs/qapi: Document optional discriminators' 003/8:[0005] [FC] 'tests: Add QAPI optional discriminator tests' 004/8:[0021] [FC] 'qapi: Formalize qcow2 encryption probing' 005/8:[----] [--] 'qapi: Formalize qcow encryption probing' 006/8:[0006] [FC] 'block: Try to create well typed json:{} filenames' 007/8:[----] [--] 'iotests: Test internal option typing' 008/8:[0006] [FC] 'iotests: qcow2's encrypt.format is now optional'
Max Reitz (8):
qapi: Add default-variant for flat unions docs/qapi: Document optional discriminators
tests: Add QAPI optional discriminator tests
qapi: Formalize qcow2 encryption probing
qapi: Formalize qcow encryption probing
block: Try to create well typed json:{} filenames
iotests: Test internal option typing
iotests: qcow2's encrypt.format is now optional
docs/devel/qapi-code-gen.txt | 21 +++++-
tests/Makefile.include | 5 +-
qapi/block-core.json | 34 ++++++++--
qapi/introspect.json | 8 +++
...ptional-discriminator-invalid-default.json | 12 ++++
...l-discriminator-invalid-specification.json | 12 ++++
...on-optional-discriminator-no-default.json} | 5 +-
...lat-union-superfluous-default-variant.json | 11 +++
tests/qapi-schema/qapi-schema-test.json | 11 +++
block.c | 68 ++++++++++++++++++-
block/qcow2.c | 3 +
scripts/qapi/common.py | 58 +++++++++++++---
scripts/qapi/doc.py | 10 ++-
scripts/qapi/introspect.py | 10 ++-
scripts/qapi/visit.py | 13 ++++
...optional-discriminator-invalid-default.err | 1 +
...tional-discriminator-invalid-default.exit} | 0
...ptional-discriminator-invalid-default.out} | 0
...al-discriminator-invalid-specification.err | 1 +
...l-discriminator-invalid-specification.exit | 1 +
...al-discriminator-invalid-specification.out | 0
...nion-optional-discriminator-no-default.err | 1 +
...ion-optional-discriminator-no-default.exit | 1 +
...nion-optional-discriminator-no-default.out | 0
.../flat-union-optional-discriminator.err | 1 -
...flat-union-superfluous-default-variant.err | 1 +
...lat-union-superfluous-default-variant.exit | 1 +
...flat-union-superfluous-default-variant.out | 0
tests/qapi-schema/qapi-schema-test.out | 10 +++
tests/qapi-schema/test-qapi.py | 2 +
tests/qemu-iotests/059.out | 2 +-
tests/qemu-iotests/087 | 65 +++++++++++-------
tests/qemu-iotests/087.out | 26 ++++++-
tests/qemu-iotests/089 | 25 +++++++
tests/qemu-iotests/089.out | 9 +++
tests/qemu-iotests/099.out | 4 +-
tests/qemu-iotests/110.out | 2 +-
tests/qemu-iotests/198.out | 4 +-
tests/qemu-iotests/207.out | 10 +--
39 files changed, 384 insertions(+), 64 deletions(-)
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.json
rename tests/qapi-schema/{flat-union-optional-discriminator.json => flat-union-optional-discriminator-no-default.json} (70%)
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
rename tests/qapi-schema/{flat-union-optional-discriminator.exit => flat-union-optional-discriminator-invalid-default.exit} (100%)
rename tests/qapi-schema/{flat-union-optional-discriminator.out => flat-union-optional-discriminator-invalid-default.out} (100%)
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.out
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.out
delete mode 100644 tests/qapi-schema/flat-union-optional-discriminator.err
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.err
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.exit
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.outSignature · 1 line
2.20.1
PATCH v3 1/8 qapi: Add default-variant for flat unions
This patch allows specifying a discriminator that is an optional member of the base struct. In such a case, a default value must be provided that is used when no value is given.
--- qapi/introspect.json | 8 +++++ scripts/qapi/common.py | 58 ++++++++++++++++++++++++++++------ scripts/qapi/doc.py | 10 ++++-- scripts/qapi/introspect.py | 10 ++++-- scripts/qapi/visit.py | 13 ++++++++ tests/qapi-schema/test-qapi.py | 2 ++ 6 files changed, 86 insertions(+), 15 deletions(-)
qapi/introspect.json +8 -0
@@ -168,6 +168,13 @@ # @tag: the name of the member serving as type tag. # An element of @members with this name must exist. # +# @default-variant: if the @tag element of @members is optional, this +# is the default value for choosing a variant. Its +# value will be a valid value for @tag. +# Present exactly when @tag is present and the +# associated element of @members is optional. +# (Since: 4.0) +# # @variants: variant members, i.e. additional members that # depend on the type tag's value. Present exactly when # @tag is present. The variants are in no particular order,
@@ -181,6 +188,7 @@ { 'struct': 'SchemaInfoObject', 'data': { 'members': [ 'SchemaInfoObjectMember' ], '*tag': 'str', + '*default-variant': 'str', '*variants': [ 'SchemaInfoObjectVariant' ] } } ##
scripts/qapi/common.py +48 -10
119 lines not on this page · raw message
scripts/qapi/doc.py +8 -2
@@ -168,8 +168,14 @@ def texi_members(doc, what, base, variants, member_func): items += '@item The members of @code{%s}\n' % base.doc_type() if variants: for v in variants.variants: - when = ' when @code{%s} is @t{"%s"}%s' % ( - variants.tag_member.name, v.name, texi_if(v.ifcond, " (", ")")) + if v.name == variants.default_tag_value: + when = ' when @code{%s} is @t{"%s"} or not given%s' % ( + variants.tag_member.name, v.name, + texi_if(v.ifcond, " (", ")")) + else: + when = ' when @code{%s} is @t{"%s"}%s' % ( + variants.tag_member.name, v.name, + texi_if(v.ifcond, " (", ")")) if v.type.is_implicit(): assert not v.type.base and not v.type.variants for m in v.type.local_members:
scripts/qapi/introspect.py +7 -3
@@ -166,9 +166,12 @@ const QLitObject %(c_name)s = %(c_string)s; ret = (ret, {'if': member.ifcond}) return ret - def _gen_variants(self, tag_name, variants): - return {'tag': tag_name, - 'variants': [self._gen_variant(v) for v in variants]} + def _gen_variants(self, tag_name, default_variant, variants): + ret = {'tag': tag_name, + 'variants': [self._gen_variant(v) for v in variants]} + if default_variant: + ret['default-variant'] = default_variant + return ret def _gen_variant(self, variant): return ({'case': variant.name, 'type': self._use_type(variant.type)},
@@ -192,6 +195,7 @@ const QLitObject %(c_name)s = %(c_string)s; obj = {'members': [self._gen_member(m) for m in members]} if variants: obj.update(self._gen_variants(variants.tag_member.name, + variants.default_tag_value, variants.variants)) self._gen_qlit(name, 'object', obj, ifcond)
scripts/qapi/visit.py +13 -0
@@ -77,6 +77,19 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp) ret += gen_endif(memb.ifcond) if variants: + if variants.default_tag_value is not None: + ret += mcgen(''' + if (!obj->has_%(c_name)s) { + obj->has_%(c_name)s = true; + obj->%(c_name)s = %(enum_const)s; + } +''', + c_name=c_name(variants.tag_member.name), + enum_const=c_enum_const( + variants.tag_member.type.name, + variants.default_tag_value, + variants.tag_member.type.prefix)) + ret += mcgen(''' switch (obj->%(c_name)s) { ''',
tests/qapi-schema/test-qapi.py +2 -0
@@ -66,6 +66,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor): def _print_variants(variants): if variants: print(' tag %s' % variants.tag_member.name) + if variants.default_tag_value: + print(' default variant: %s' % variants.default_tag_value) for v in variants.variants: print(' case %s: %s' % (v.name, v.type.name)) QAPISchemaTestVisitor._print_if(v.ifcond, indent=8)
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 1/8] qapi: Add default-variant for flat unions
Quoted 5 lines · Max Reitz
This patch allows specifying a discriminator that is an optional member of the base struct. In such a case, a default value must be provided that is used when no value is given.
Signed-off-by Max Reitz <[email protected]>
Quoted 8 lines
--- qapi/introspect.json | 8 +++++ scripts/qapi/common.py | 58 ++++++++++++++++++++++++++++------ scripts/qapi/doc.py | 10 ++++-- scripts/qapi/introspect.py | 10 ++++-- scripts/qapi/visit.py | 13 ++++++++ tests/qapi-schema/test-qapi.py | 2 ++ 6 files changed, 86 insertions(+), 15 deletions(-)
Missing a change to docs/devel/qapi-code-gen.txt? [re-reads diffstat in cover letter...] Oh, you got that later in the series. Maybe those two patches are worth squashing?
Quoted 8 lines
+++
b/scripts/qapi/visit.pyscripts/qapi/visit.py +3
-0@@ -77,6 +77,19 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp) ret += gen_endif(memb.ifcond) if variants: + if variants.default_tag_value is not None: + ret += mcgen(''' + if (!obj->has_%(c_name)s) {
scripts/qapi/visit.py void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp) · line 92
Quoted 10 lines
scripts/qapi/visit.py +10
-0+ obj->has_%(c_name)s = true; + obj->%(c_name)s = %(enum_const)s; + } +''', + c_name=c_name(variants.tag_member.name), + enum_const=c_enum_const( + variants.tag_member.type.name, + variants.default_tag_value, + variants.tag_member.type.prefix)) +
I like that you de-sugar it as early in the input parse as possible.
Markus may have comments, but it looks reasonable to me.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v3 1/8] qapi: Add default-variant for flat unions
In reply to Max ReitzQuoted 33 lines · Max Reitz
This patch allows specifying a discriminator that is an optional member of the base struct. In such a case, a default value must be provided that is used when no value is given.
Signed-off-by Max Reitz <[email protected]>--- qapi/introspect.json | 8 +++++ scripts/qapi/common.py | 58 ++++++++++++++++++++++++++++------ scripts/qapi/doc.py | 10 ++++-- scripts/qapi/introspect.py | 10 ++++-- scripts/qapi/visit.py | 13 ++++++++ tests/qapi-schema/test-qapi.py | 2 ++ 6 files changed, 86 insertions(+), 15 deletions(-)qapi/introspect.json +7
-0@@ -168,6 +168,13 @@ # @tag: the name of the member serving as type tag. # An element of @members with this name must exist. # +# @default-variant: if the @tag element of @members is optional, this +# is the default value for choosing a variant. Its +# value will be a valid value for @tag. +# Present exactly when @tag is present and the +# associated element of @members is optional. +# (Since: 4.0) +# # @variants: variant members, i.e. additional members that # depend on the type tag's value. Present exactly when # @tag is present. The variants are in no particular order,
qapi/introspect.json line 194
Quoted 7 lines
qapi/introspect.json +1
-0@@ -181,6 +188,7 @@ { 'struct': 'SchemaInfoObject', 'data': { 'members': [ 'SchemaInfoObjectMember' ], '*tag': 'str', + '*default-variant': 'str', '*variants': [ 'SchemaInfoObjectVariant' ] } } ##
I'm afraid this could become awkward later on. Let me explain.
In many programming languages, absent optional arguments / members default to a default value specified in the declaration. Simple.
In others, 'absent' is effectively an additional value. The code consuming the argument / member can interpret 'absent' however it wants. It may eliminate the additional value by mapping it to a default value, but it can also interpret 'absent' unlike any value. If there's more than one consumer, their interpretations need not be consistent. The declaration provides no clue on semantics of 'absent'.
QAPI is in the latter camp. I trust you can already sense how much I like that.
Now you want to permit optional variant discriminators. As per general rule, their interpretation is left to the code consuming it. One instance of such code is the generated union visitor, which needs to decide which variant to visit. Your default-variant tells it which variant to visit. Other code interpreting the discriminator better be consistent with that, but that's the other code's problem. Hmm.
If I could go back in time, I'd flip QAPI to "'absent' defaults to a default value". Lacking a time machine, we're stuck with cases of "'absent' means something you can't express with a value" and "'absent' means different things in different contexts" that have been enshrined in external interfaces. Is there anything we could do to improve matters for saner cases?
I think there is: we could provide for an *optional* default value. If the schema specifies it, that's what 'absent' means. If it doesn't, all bets are off, just like they are now.
Say we do that (for what it's worth, introspect.json is already prepared for it). How would it play with your default-variant?
If an optional discriminator specifies a default value, then that's the default variant. But wait, if there's also a default-variant, *that's* the default variant! Awkward clash. To resolve it, we could either forbid combining the two, or rule default-variant overrides the default. Feels needlessly complicated.
Could we get away with "if you want a default variant, the discriminator must specify a default"?
[...]
Re: [Qemu-devel] [PATCH v3 1/8] qapi: Add default-variant for flat unions
Quoted 38 lines · Markus Armbruster
Quoted 6 lines · Max Reitz
This patch allows specifying a discriminator that is an optional member of the base struct. In such a case, a default value must be provided that is used when no value is given.
Signed-off-by Max Reitz <[email protected]>---
I'm afraid this could become awkward later on. Let me explain.
In many programming languages, absent optional arguments / members default to a default value specified in the declaration. Simple.
In others, 'absent' is effectively an additional value. The code consuming the argument / member can interpret 'absent' however it wants. It may eliminate the additional value by mapping it to a default value, but it can also interpret 'absent' unlike any value. If there's more than one consumer, their interpretations need not be consistent. The declaration provides no clue on semantics of 'absent'.
QAPI is in the latter camp. I trust you can already sense how much I like that.
Now you want to permit optional variant discriminators. As per general rule, their interpretation is left to the code consuming it. One instance of such code is the generated union visitor, which needs to decide which variant to visit. Your default-variant tells it which variant to visit. Other code interpreting the discriminator better be consistent with that, but that's the other code's problem. Hmm.
If I could go back in time, I'd flip QAPI to "'absent' defaults to a default value". Lacking a time machine, we're stuck with cases of "'absent' means something you can't express with a value" and "'absent' means different things in different contexts" that have been enshrined in external interfaces. Is there anything we could do to improve matters for saner cases?
Quoted 3 lines
I think there is: we could provide for an *optional* default value. If the schema specifies it, that's what 'absent' means. If it doesn't, all bets are off, just like they are now.
And we already have the planned syntax, thanks to our recent work on adding conditionals - where we now have:
{ '*field': 'mytype' }
we can also do long-hand:
{ { 'name': '*field', 'type': 'mytype' } }
which also lends itself well to declaring a default:
{ { 'name': '*field', 'type': 'mytype', 'default': 'xyz' } }
Quoted 4 lines
Say we do that (for what it's worth, introspect.json is already prepared for it). How would it play with your default-variant?
Quoted 8 lines
If an optional discriminator specifies a default value, then that's the default variant. But wait, if there's also a default-variant, *that's* the default variant! Awkward clash. To resolve it, we could either forbid combining the two, or rule default-variant overrides the default. Feels needlessly complicated.
Could we get away with "if you want a default variant, the discriminator must specify a default"?
I like the idea. It means finally biting the bullet and implementing default values, but we've known we've wanted them for a while (as evidenced by the existing introspection output, and by the syntax that we have ready to put into use for it).
It means that representing the default variant in a union now depends on the base class declaring a default for the optional parameter (that is, an optional parameter can only be used as discriminator if it has a declared default), and gives us variable defaults in more uses than just unions.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v3 1/8] qapi: Add default-variant for flat unions
Quoted 4 lines · Eric Blake
Quoted 3 lines
I think there is: we could provide for an *optional* default value. If the schema specifies it, that's what 'absent' means. If it doesn't, all bets are off, just like they are now.
Quoted 8 lines
And we already have the planned syntax, thanks to our recent work on adding conditionals - where we now have:
{ '*field': 'mytype' }
we can also do long-hand:
{ { 'name': '*field', 'type': 'mytype' } }
I'd better use the actual syntax, instead of inventing non-JSON off the top of my head:
{ '*field': { 'type': 'mytype' } }
Quoted 4 lines
which also lends itself well to declaring a default:
{ { 'name': '*field', 'type': 'mytype', 'default': 'xyz' } }
{ '*field': { 'type': 'mytype', 'default': 'xyz' } }
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v3 1/8] qapi: Add default-variant for flat unions
In reply to Markus ArmbrusterQuoted 72 lines · Markus Armbruster
Quoted 33 lines · Max Reitz
This patch allows specifying a discriminator that is an optional member of the base struct. In such a case, a default value must be provided that is used when no value is given.
Signed-off-by Max Reitz <[email protected]>--- qapi/introspect.json | 8 +++++ scripts/qapi/common.py | 58 ++++++++++++++++++++++++++++------ scripts/qapi/doc.py | 10 ++++-- scripts/qapi/introspect.py | 10 ++++-- scripts/qapi/visit.py | 13 ++++++++ tests/qapi-schema/test-qapi.py | 2 ++ 6 files changed, 86 insertions(+), 15 deletions(-)qapi/introspect.json +7
-0@@ -168,6 +168,13 @@ # @tag: the name of the member serving as type tag. # An element of @members with this name must exist. # +# @default-variant: if the @tag element of @members is optional, this +# is the default value for choosing a variant. Its +# value will be a valid value for @tag. +# Present exactly when @tag is present and the +# associated element of @members is optional. +# (Since: 4.0) +# # @variants: variant members, i.e. additional members that # depend on the type tag's value. Present exactly when # @tag is present. The variants are in no particular order,qapi/introspect.json line 194
Quoted 7 lines
qapi/introspect.json +1
-0@@ -181,6 +188,7 @@ { 'struct': 'SchemaInfoObject', 'data': { 'members': [ 'SchemaInfoObjectMember' ], '*tag': 'str', + '*default-variant': 'str', '*variants': [ 'SchemaInfoObjectVariant' ] } } ##I'm afraid this could become awkward later on. Let me explain.
In many programming languages, absent optional arguments / members default to a default value specified in the declaration. Simple.
In others, 'absent' is effectively an additional value. The code consuming the argument / member can interpret 'absent' however it wants. It may eliminate the additional value by mapping it to a default value, but it can also interpret 'absent' unlike any value. If there's more than one consumer, their interpretations need not be consistent. The declaration provides no clue on semantics of 'absent'.
QAPI is in the latter camp. I trust you can already sense how much I like that.
Now you want to permit optional variant discriminators. As per general rule, their interpretation is left to the code consuming it. One instance of such code is the generated union visitor, which needs to decide which variant to visit. Your default-variant tells it which variant to visit. Other code interpreting the discriminator better be consistent with that, but that's the other code's problem. Hmm.
If I could go back in time, I'd flip QAPI to "'absent' defaults to a default value". Lacking a time machine, we're stuck with cases of "'absent' means something you can't express with a value" and "'absent' means different things in different contexts" that have been enshrined in external interfaces. Is there anything we could do to improve matters for saner cases?
Quoted 6 lines
I think there is: we could provide for an *optional* default value. If the schema specifies it, that's what 'absent' means. If it doesn't, all bets are off, just like they are now.
Say we do that (for what it's worth, introspect.json is already prepared for it).
If somebody(tm) does that, great. :-)
Quoted 7 lines
How would it play with your default-variant?
If an optional discriminator specifies a default value, then that's the default variant. But wait, if there's also a default-variant, *that's* the default variant! Awkward clash. To resolve it, we could either forbid combining the two, or rule default-variant overrides the default. Feels needlessly complicated.
I agree on the needless, not so sure about the complicated. (Other than it being double the work, but, well, the default-variant work is already right here.)
Quoted 2 lines
Could we get away with "if you want a default variant, the discriminator must specify a default"?
I think so, yes. I agree that it'd be the better solution. But to be honest I'd really rather not delve any deeper into the QAPI dungeon than I've done in this series so far...
Max
Re: [Qemu-devel] [PATCH v3 1/8] qapi: Add default-variant for flat unions
Quoted 84 lines · Max Reitz
Quoted 62 lines · Markus Armbruster
Quoted 23 lines · Max Reitz
This patch allows specifying a discriminator that is an optional member of the base struct. In such a case, a default value must be provided that is used when no value is given.
Signed-off-by Max Reitz <[email protected]>---
+++
b/qapi/introspect.jsonqapi/introspect.json +7
-0@@ -168,6 +168,13 @@ # @tag: the name of the member serving as type tag. # An element of @members with this name must exist. # +# @default-variant: if the @tag element of @members is optional, this +# is the default value for choosing a variant. Its +# value will be a valid value for @tag. +# Present exactly when @tag is present and the +# associated element of @members is optional. +# (Since: 4.0) +# # @variants: variant members, i.e. additional members that # depend on the type tag's value. Present exactly when # @tag is present. The variants are in no particular order,qapi/introspect.json line 194
Quoted 7 lines
qapi/introspect.json +1
-0@@ -181,6 +188,7 @@ { 'struct': 'SchemaInfoObject', 'data': { 'members': [ 'SchemaInfoObjectMember' ], '*tag': 'str', + '*default-variant': 'str', '*variants': [ 'SchemaInfoObjectVariant' ] } } ##I'm afraid this could become awkward later on. Let me explain.
In many programming languages, absent optional arguments / members default to a default value specified in the declaration. Simple.
In others, 'absent' is effectively an additional value. The code consuming the argument / member can interpret 'absent' however it wants. It may eliminate the additional value by mapping it to a default value, but it can also interpret 'absent' unlike any value. If there's more than one consumer, their interpretations need not be consistent. The declaration provides no clue on semantics of 'absent'.
QAPI is in the latter camp. I trust you can already sense how much I like that.
Now you want to permit optional variant discriminators. As per general rule, their interpretation is left to the code consuming it. One instance of such code is the generated union visitor, which needs to decide which variant to visit. Your default-variant tells it which variant to visit. Other code interpreting the discriminator better be consistent with that, but that's the other code's problem. Hmm.
If I could go back in time, I'd flip QAPI to "'absent' defaults to a default value". Lacking a time machine, we're stuck with cases of "'absent' means something you can't express with a value" and "'absent' means different things in different contexts" that have been enshrined in external interfaces. Is there anything we could do to improve matters for saner cases?
Quoted 6 lines
I think there is: we could provide for an *optional* default value. If the schema specifies it, that's what 'absent' means. If it doesn't, all bets are off, just like they are now.
Say we do that (for what it's worth, introspect.json is already prepared for it).
If somebody(tm) does that, great. :-)
Quoted 7 lines
How would it play with your default-variant?
If an optional discriminator specifies a default value, then that's the default variant. But wait, if there's also a default-variant, *that's* the default variant! Awkward clash. To resolve it, we could either forbid combining the two, or rule default-variant overrides the default. Feels needlessly complicated.
I agree on the needless, not so sure about the complicated. (Other than it being double the work, but, well, the default-variant work is already right here.)
Quoted 6 lines
Quoted 2 lines
Could we get away with "if you want a default variant, the discriminator must specify a default"?
I think so, yes. I agree that it'd be the better solution. But to be honest I'd really rather not delve any deeper into the QAPI dungeon than I've done in this series so far...
I've now hit a case where I'd like a default-variant (namely, improving nbd-server-add to avoid SocketAddressLegacy); maybe I'll find time to revive at least this part of the series.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 2/8 docs/qapi: Document optional discriminators
In reply to Max Reitz--- docs/devel/qapi-code-gen.txt | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)
docs/devel/qapi-code-gen.txt +19 -2
@@ -472,8 +472,8 @@ All branches of the union must be complex types, and the top-level members of the union dictionary on the wire will be combination of members from both the base type and the appropriate branch type (when merging two dictionaries, there must be no keys in common). The -'discriminator' member must be the name of a non-optional enum-typed -member of the base struct. +'discriminator' member must be the name of an enum-typed member of the +base struct. The following example enhances the above simple union example by adding an optional common member 'read-only', renaming the
@@ -504,6 +504,23 @@ In the resulting generated C data types, a flat union is represented as a struct with the base members included directly, and then a union of structures for each branch of the struct. +If the discriminator points to an optional member of the base struct, +its default value must be specified as a 'default-variant'. In the +following example, the above BlockDriver struct is changed so it +defaults to the 'file' driver if that field is omitted on the wire: + + { 'union': 'BlockdevOptions', + 'base': { '*driver': 'BlockdevDriver', '*read-only': 'bool' }, + 'discriminator': 'driver', + 'default-variant': 'file', + 'data': { 'file': 'BlockdevOptionsFile', + 'qcow2': 'BlockdevOptionsQcow2' } } + +Now the 'file' JSON object can be abbreviated to: + + { "read-only": "true", + "filename": "/some/place/my-image" } + A simple union can always be re-written as a flat union where the base class has a single member named 'type', and where each branch of the union has a struct with a single member named 'data'. That is,
Signature · 1 line
2.20.1
PATCH v3 3/8 tests: Add QAPI optional discriminator tests
In reply to Max ReitzThere already is an optional discriminator test, although it also noted the discriminator name itself as optional. This already gives us one error test case, to which this patch adds various others.
Furthermore, a passing test case is added to qapi-schema-test.
---
tests/Makefile.include | 5 ++++-
...union-optional-discriminator-invalid-default.json | 12 ++++++++++++
...optional-discriminator-invalid-specification.json | 12 ++++++++++++
...lat-union-optional-discriminator-no-default.json} | 5 +++--
.../flat-union-superfluous-default-variant.json | 11 +++++++++++
tests/qapi-schema/qapi-schema-test.json | 11 +++++++++++
...-union-optional-discriminator-invalid-default.err | 1 +
...nion-optional-discriminator-invalid-default.exit} | 0
...union-optional-discriminator-invalid-default.out} | 0
...-optional-discriminator-invalid-specification.err | 1 +
...optional-discriminator-invalid-specification.exit | 1 +
...-optional-discriminator-invalid-specification.out | 0
.../flat-union-optional-discriminator-no-default.err | 1 +
...flat-union-optional-discriminator-no-default.exit | 1 +
.../flat-union-optional-discriminator-no-default.out | 0
.../flat-union-optional-discriminator.err | 1 -
.../flat-union-superfluous-default-variant.err | 1 +
.../flat-union-superfluous-default-variant.exit | 1 +
.../flat-union-superfluous-default-variant.out | 0
tests/qapi-schema/qapi-schema-test.out | 10 ++++++++++
20 files changed, 70 insertions(+), 4 deletions(-)
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.json
rename tests/qapi-schema/{flat-union-optional-discriminator.json => flat-union-optional-discriminator-no-default.json} (70%)
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err
rename tests/qapi-schema/{flat-union-optional-discriminator.exit => flat-union-optional-discriminator-invalid-default.exit} (100%)
rename tests/qapi-schema/{flat-union-optional-discriminator.out => flat-union-optional-discriminator-invalid-default.out} (100%)
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.out
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator-no-default.out
delete mode 100644 tests/qapi-schema/flat-union-optional-discriminator.err
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.err
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.exit
create mode 100644 tests/qapi-schema/flat-union-superfluous-default-variant.outtests/Makefile.include +4 -1
@@ -422,8 +422,11 @@ qapi-schema += flat-union-invalid-branch-key.json qapi-schema += flat-union-invalid-discriminator.json qapi-schema += flat-union-invalid-if-discriminator.json qapi-schema += flat-union-no-base.json -qapi-schema += flat-union-optional-discriminator.json +qapi-schema += flat-union-optional-discriminator-invalid-specification.json +qapi-schema += flat-union-optional-discriminator-no-default.json +qapi-schema += flat-union-optional-discriminator-invalid-default.json qapi-schema += flat-union-string-discriminator.json +qapi-schema += flat-union-superfluous-default-variant.json qapi-schema += funny-char.json qapi-schema += ident-with-escape.json qapi-schema += include-before-err.json
tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json new file +12 -0
@@ -0,0 +1,12 @@ +# default-variant must refer to an actual value of the discriminator's +# enum +{ 'enum': 'Enum', 'data': [ 'one', 'two' ] } +{ 'struct': 'Base', + 'data': { '*switch': 'Enum' } } +{ 'struct': 'Branch', 'data': { 'name': 'str' } } +{ 'union': 'MyUnion', + 'base': 'Base', + 'discriminator': 'switch', + 'default-variant': 'three', + 'data': { 'one': 'Branch', + 'two': 'Branch' } }
tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.json new file +12 -0
@@ -0,0 +1,12 @@ +# For using optional discriminators, only the field in the base struct +# must be marked optional, not the discriminator name itself +{ 'enum': 'Enum', 'data': [ 'one', 'two' ] } +{ 'struct': 'Base', + 'data': { '*switch': 'Enum' } } +{ 'struct': 'Branch', 'data': { 'name': 'str' } } +{ 'union': 'MyUnion', + 'base': 'Base', + 'discriminator': '*switch', + 'default-variant': 'one', + 'data': { 'one': 'Branch', + 'two': 'Branch' } }
tests/qapi-schema/flat-union-optional-discriminator-no-default.json renamed from tests/qapi-schema/flat-union-optional-discriminator.json +3 -2
@@ -1,10 +1,11 @@ -# we require the discriminator to be non-optional +# Using an optional discriminator requires specifying a default +# variant { 'enum': 'Enum', 'data': [ 'one', 'two' ] } { 'struct': 'Base', 'data': { '*switch': 'Enum' } } { 'struct': 'Branch', 'data': { 'name': 'str' } } { 'union': 'MyUnion', 'base': 'Base', - 'discriminator': '*switch', + 'discriminator': 'switch', 'data': { 'one': 'Branch', 'two': 'Branch' } }
tests/qapi-schema/flat-union-superfluous-default-variant.json new file +11 -0
@@ -0,0 +1,11 @@ +# default-variant only makes sense with an optional discriminator +{ 'enum': 'Enum', 'data': [ 'one', 'two' ] } +{ 'struct': 'Base', + 'data': { 'switch': 'Enum' } } +{ 'struct': 'Branch', 'data': { 'name': 'str' } } +{ 'union': 'MyUnion', + 'base': 'Base', + 'discriminator': 'switch', + 'default-variant': 'one', + 'data': { 'one': 'Branch', + 'two': 'Branch' } }
tests/qapi-schema/qapi-schema-test.json +11 -0
@@ -104,6 +104,17 @@ { 'struct': 'UserDefC', 'data': { 'string1': 'str', 'string2': 'str' } } +# for testing unions with an optional discriminator +{ 'union': 'UserDefFlatUnion3', + 'base': { '*enum1': 'EnumOne' }, + 'discriminator': 'enum1', + 'default-variant': 'value1', + 'data': { 'value1' : 'UserDefA', + 'value2' : 'UserDefB', + 'value3' : 'UserDefB' + # 'value4' defaults to empty + } } + # for testing use of 'number' within alternates { 'alternate': 'AltEnumBool', 'data': { 'e': 'EnumOne', 'b': 'bool' } } { 'alternate': 'AltEnumNum', 'data': { 'e': 'EnumOne', 'n': 'number' } }
tests/qapi-schema/flat-union-optional-discriminator-invalid-default.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/flat-union-optional-discriminator-invalid-default.json:7: Default variant 'three' of flat union 'MyUnion' is not part of 'Enum'
tests/qapi-schema/flat-union-optional-discriminator-invalid-default.exit renamed from tests/qapi-schema/flat-union-optional-discriminator.exit +0 -0
tests/qapi-schema/flat-union-optional-discriminator-invalid-default.out renamed from tests/qapi-schema/flat-union-optional-discriminator.out +0 -0
tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.json:7: Discriminator of flat union 'MyUnion' does not allow optional name '*switch'
tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/flat-union-optional-discriminator-invalid-specification.out new file +0 -0
tests/qapi-schema/flat-union-optional-discriminator-no-default.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/flat-union-optional-discriminator-no-default.json:7: Default variant must be specified for optional discriminator 'switch'
tests/qapi-schema/flat-union-optional-discriminator-no-default.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/flat-union-optional-discriminator-no-default.out new file +0 -0
tests/qapi-schema/flat-union-optional-discriminator.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/flat-union-optional-discriminator.json:6: Discriminator of flat union 'MyUnion' does not allow optional name '*switch'
tests/qapi-schema/flat-union-superfluous-default-variant.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/flat-union-superfluous-default-variant.json:6: Must not specify a default variant for non-optional discriminator 'switch'
tests/qapi-schema/flat-union-superfluous-default-variant.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/flat-union-superfluous-default-variant.out new file +0 -0
tests/qapi-schema/qapi-schema-test.out +10 -0
@@ -92,6 +92,16 @@ alternate UserDefAlternate object UserDefC member string1: str optional=False member string2: str optional=False +object q_obj_UserDefFlatUnion3-base + member enum1: EnumOne optional=True +object UserDefFlatUnion3 + base q_obj_UserDefFlatUnion3-base + tag enum1 + default variant: value1 + case value1: UserDefA + case value2: UserDefB + case value3: UserDefB + case value4: q_empty alternate AltEnumBool tag type case e: EnumOne
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 3/8] tests: Add QAPI optional discriminator tests
Quoted 8 lines · Max Reitz
There already is an optional discriminator test, although it also noted the discriminator name itself as optional. This already gives us one error test case, to which this patch adds various others.
Furthermore, a passing test case is added to qapi-schema-test.
Signed-off-by Max Reitz <[email protected]>---
This could also be squashed with patch 1 to avoid a window of 'make check' breakage during a bisect; but as that makes the overall changes bigger to review, leaving them separate is not too terrible.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 4/8 qapi: Formalize qcow2 encryption probing
In reply to Max ReitzCurrently, you can give no encryption format for a qcow2 file while still passing a key-secret. That does not conform to the schema, so this patch changes the schema to allow it.
--- qapi/block-core.json | 31 ++++++++++++++++++++++++++++--- block/qcow2.c | 3 +++ 2 files changed, 31 insertions(+), 3 deletions(-)
qapi/block-core.json +28 -3
@@ -47,6 +47,9 @@ ## # @ImageInfoSpecificQCow2Encryption: # +# @format will never be "auto", as this pseudo-format just tells the +# qcow2 driver to read the actual format from the image header. +# # Since: 2.10 ## { 'union': 'ImageInfoSpecificQCow2Encryption',
@@ -2961,10 +2964,30 @@ # @BlockdevQcow2EncryptionFormat: # @aes: AES-CBC with plain64 initialization venctors # +# @auto: Determine the encryption format from the image +# header. This only allows the use of the +# key-secret option. (Since: 4.0) +# # Since: 2.10 ## { 'enum': 'BlockdevQcow2EncryptionFormat', - 'data': [ 'aes', 'luks' ] } + 'data': [ 'aes', 'luks', 'auto' ] } + +## +# @BlockdevQcow2EncryptionSecret: +# +# Allows specifying a key-secret without specifying the exact +# encryption format, which is determined automatically from the image +# header. +# +# @key-secret: The ID of a QCryptoSecret object providing the +# decryption key. Mandatory except when probing +# image for metadata only. +# +# Since: 4.0 +## +{ 'struct': 'BlockdevQcow2EncryptionSecret', + 'data': { '*key-secret': 'str' } } ## # @BlockdevQcow2Encryption:
@@ -2972,10 +2995,12 @@ # Since: 2.10 ## { 'union': 'BlockdevQcow2Encryption', - 'base': { 'format': 'BlockdevQcow2EncryptionFormat' }, + 'base': { '*format': 'BlockdevQcow2EncryptionFormat' }, 'discriminator': 'format', + 'default-variant': 'auto', 'data': { 'aes': 'QCryptoBlockOptionsQCow', - 'luks': 'QCryptoBlockOptionsLUKS'} } + 'luks': 'QCryptoBlockOptionsLUKS', + 'auto': 'BlockdevQcow2EncryptionSecret' } } ## # @BlockdevOptionsQcow2:
block/qcow2.c +3 -0
@@ -883,6 +883,9 @@ static int qcow2_update_options_prepare(BlockDriverState *bs,qdict_extract_subqdict (options, &encryptopts,"encrypt." ); encryptfmt =qdict_get_try_str (encryptopts,"format" ); +if (!g_strcmp0 (encryptfmt,"auto" )) { + encryptfmt =NULL ; + } opts =qemu_opts_create (&qcow2_runtime_opts,NULL ,0 , &error_abort);qemu_opts_absorb_qdict (opts, options, &local_err);
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 4/8] qapi: Formalize qcow2 encryption probing
Quoted 12 lines · Max Reitz
Currently, you can give no encryption format for a qcow2 file while still passing a key-secret. That does not conform to the schema, so this patch changes the schema to allow it.
Signed-off-by Max Reitz <[email protected]>--- qapi/block-core.json | 31 ++++++++++++++++++++++++++++--- block/qcow2.c | 3 +++ 2 files changed, 31 insertions(+), 3 deletions(-)
qapi/block-core.json line 2965
Quoted 2 lines
qapi/block-core.json +0
-0@@ -2961,10 +2964,30 @@ # @BlockdevQcow2EncryptionFormat: # @aes: AES-CBC with plain64 initialization venctors
We should fix that typo.
qapi/block-core.json line 2971
Quoted 6 lines
qapi/block-core.json +4
-0# +# @auto: Determine the encryption format from the image +# header. This only allows the use of the +# key-secret option. (Since: 4.0) +# # Since: 2.10
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 5/8 qapi: Formalize qcow encryption probing
In reply to Max Reitzqcow only supports a single encryption (and there is no reason why that would change in the future), so we can make it the default.
--- qapi/block-core.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
qapi/block-core.json +2 -1
@@ -2939,8 +2939,9 @@ # Since: 2.10 ## { 'union': 'BlockdevQcowEncryption', - 'base': { 'format': 'BlockdevQcowEncryptionFormat' }, + 'base': { '*format': 'BlockdevQcowEncryptionFormat' }, 'discriminator': 'format', + 'default-variant': 'aes', 'data': { 'aes': 'QCryptoBlockOptionsQCow' } } ##
Signature · 1 line
2.20.1
PATCH v3 6/8 block: Try to create well typed json:{} filenames
In reply to Max ReitzBy applying a health mix of qdict_flatten(), qdict_crumple(), qdict_stringify_for_keyval(), the keyval input visitor, and the QObject output visitor (not necessarily in that order), we can at least try to bring bs->full_open_options into accordance with the QAPI schema. This may not always work (there are some options left that have not been QAPI-fied yet), but in practice it usually will.
In any case, sometimes emitting wrongly typed json:{} filenames is better than doing it effectively half the time.
This affects some iotests because json:{} filenames are now usually crumpled. In 198, "format": "auto" now appears in the qcow2 encryption options because going through a visitor makes optional discriminators' default values explicit.
--- block.c | 68 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/059.out | 2 +- tests/qemu-iotests/099.out | 4 +-- tests/qemu-iotests/110.out | 2 +- tests/qemu-iotests/198.out | 4 +-- tests/qemu-iotests/207.out | 10 +++--- 6 files changed, 78 insertions(+), 12 deletions(-)
block.c +67 -1
@@ -36,6 +36,7 @@#include "qapi/qmp/qjson.h" #include "qapi/qmp/qnull.h" #include "qapi/qmp/qstring.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qobject-output-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "sysemu/block-backend.h"
@@ -5581,6 +5582,56 @@ static bool bdrv_backing_overridden(BlockDriverState *bs) } } +/** +* Take a blockdev @options QDict and convert its values to the +* correct type. +* +* Fail if @options does not match the QAPI schema of BlockdevOptions. +* +* In case of failure, return NULL and set @errp. +* +* In case of success, return a correctly typed new QDict. +*/ +static QDict *bdrv_type_blockdev_opts (const QDict *options,Error **errp) +{ +Visitor *v; +BlockdevOptions *blockdev_options; +QObject *typed_opts; +QDict *string_options; +Error *local_err =NULL ; + + string_options =qdict_clone_shallow (options); + +qdict_flatten (string_options); + v =qobject_input_visitor_new_flat_confused (string_options, errp); +if (!v) { +error_prepend (errp,"Failed to prepare options: " ); +return NULL ; + } + +visit_type_BlockdevOptions (v,NULL , &blockdev_options, &local_err); +visit_free (v); +if (local_err) { +error_propagate (errp, local_err); +error_prepend (errp,"Not a valid BlockdevOptions object: " ); +return NULL ; + } + + v =qobject_output_visitor_new (&typed_opts); +visit_type_BlockdevOptions (v,NULL , &blockdev_options, &local_err); +if (!local_err) { +visit_complete (v, &typed_opts); + } +visit_free (v); +qapi_free_BlockdevOptions (blockdev_options); +if (local_err) { +error_propagate (errp, local_err); +return NULL ; + } + +return qobject_to (QDict, typed_opts); +} +/* Updates the following BDS fields: * - exact_filename: A filename which may be used for opening a block device * which (mostly) equals the given BDS (even without any
@@ -5698,10 +5749,25 @@ void bdrv_refresh_filename(BlockDriverState *bs)if (bs->exact_filename[0 ]) {pstrcpy (bs->filename,sizeof (bs->filename), bs->exact_filename); }else { -QString *json =qobject_to_json (QOBJECT (bs->full_open_options)); +QString *json; +QDict *typed_opts, *json_opts; + + typed_opts =bdrv_type_blockdev_opts (bs->full_open_options,NULL ); + +/* +* We cannot be certain that bs->full_open_options matches +* BlockdevOptions, so bdrv_type_blockdev_opts() may fail. +* That is not fatal, we can just emit bs->full_open_options +* directly -- qemu will accept that, even if it does not +* match the schema. +*/ + json_opts = typed_opts ?: bs->full_open_options; + + json =qobject_to_json (QOBJECT (json_opts));snprintf (bs->filename,sizeof (bs->filename),"json:%s" ,qstring_get_str (json));qobject_unref (json); +qobject_unref (typed_opts); } }
tests/qemu-iotests/059.out +1 -1
@@ -2050,7 +2050,7 @@ wrote 512/512 bytes at offset 10240 === Testing monolithicFlat with internally generated JSON file name === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat -can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}' +can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"inject-error": [{"event": "read_aio"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}' === Testing version 3 === image: TEST_DIR/iotest-version3.IMGFMT
tests/qemu-iotests/099.out +2 -2
@@ -12,11 +12,11 @@ blkverify:TEST_DIR/t.IMGFMT.compare:TEST_DIR/t.IMGFMT === Testing JSON filename for blkdebug === -json:{"driver": "IMGFMT", "file": {"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "l1_update"}} +json:{"driver": "IMGFMT", "file": {"inject-error": [{"event": "l1_update"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}} === Testing indirectly enforced JSON filename === -json:{"driver": "raw", "file": {"test": {"driver": "IMGFMT", "file": {"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "l1_update"}}, "driver": "blkverify", "raw": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.compare"}}} +json:{"driver": "raw", "file": {"test": {"driver": "IMGFMT", "file": {"inject-error": [{"event": "l1_update"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}}, "driver": "blkverify", "raw": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.compare"}}} === Testing plain filename for blkdebug ===
tests/qemu-iotests/110.out +1 -1
@@ -11,7 +11,7 @@ backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base) === Non-reconstructable filename === -image: json:{"driver": "IMGFMT", "file": {"set-state.0.event": "read_aio", "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "set-state.0.new_state": 42}} +image: json:{"driver": "IMGFMT", "file": {"set-state": [{"new_state": 42, "event": "read_aio"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}} file format: IMGFMT virtual size: 64M (67108864 bytes) backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
tests/qemu-iotests/198.out +2 -2
@@ -32,7 +32,7 @@ read 16777216/16777216 bytes at offset 0 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == checking image base == -image: json:{"encrypt.key-secret": "sec0", "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.base"}} +image: json:{"driver": "IMGFMT", "encrypt": {"format": "auto", "key-secret": "sec0"}, "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT.base"}} file format: IMGFMT virtual size: 16M (16777216 bytes) Format specific information:
@@ -74,7 +74,7 @@ Format specific information: master key iters: 1024 == checking image layer == -image: json:{"encrypt.key-secret": "sec1", "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}} +image: json:{"driver": "IMGFMT", "encrypt": {"format": "auto", "key-secret": "sec1"}, "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}} file format: IMGFMT virtual size: 16M (16777216 bytes) backing file: TEST_DIR/t.IMGFMT.base
tests/qemu-iotests/207.out +5 -5
@@ -5,7 +5,7 @@ {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} -image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} +image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_IMG", "server": {"port": "22", "host": "127.0.0.1"}}} file format: IMGFMT virtual size: 4.0M (4194304 bytes)
@@ -21,7 +21,7 @@ virtual size: 4.0M (4194304 bytes) {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} -image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} +image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_IMG", "server": {"port": "22", "host": "127.0.0.1"}}} file format: IMGFMT virtual size: 8.0M (8388608 bytes)
@@ -30,7 +30,7 @@ virtual size: 8.0M (8388608 bytes) {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} -image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} +image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_IMG", "server": {"port": "22", "host": "127.0.0.1"}}} file format: IMGFMT virtual size: 4.0M (4194304 bytes)
@@ -45,7 +45,7 @@ Job failed: remote host key does not match host_key_check 'wrong' {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} -image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} +image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_IMG", "server": {"port": "22", "host": "127.0.0.1"}}} file format: IMGFMT virtual size: 8.0M (8388608 bytes)
@@ -60,7 +60,7 @@ Job failed: remote host key does not match host_key_check 'wrong' {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} -image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} +image: json:{"driver": "IMGFMT", "file": {"driver": "ssh", "path": "TEST_IMG", "server": {"port": "22", "host": "127.0.0.1"}}} file format: IMGFMT virtual size: 4.0M (4194304 bytes)
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 6/8] block: Try to create well typed json:{} filenames
On 2/6/19 1:55 PM, Max Reitz wrote:
In the subject, s/well typed/well-typed/
Quoted 1 line
By applying a health mix of
qdict_flatten(),qdict_crumple(),
s/health/healty/
Quoted 8 lines
qdict_stringify_for_keyval(), the keyval input visitor, and the QObject output visitor (not necessarily in that order), we can at least try to bringbs->full_open_optionsinto accordance with the QAPI schema. This may not always work (there are some options left that have not been QAPI-fied yet), but in practice it usually will.In any case, sometimes emitting wrongly typed json:{} filenames is better than doing it effectively half the time.
And someday, if we ever switch the block layer to use QAPI types all the way, we can drop some of these hacks that have built up over time. But not a show-stopper for this series.
Quoted 83 lines
This affects some iotests because json:{} filenames are now usually crumpled. In 198, "format": "auto" now appears in the qcow2 encryption options because going through a visitor makes optional discriminators' default values explicit.
Signed-off-by Max Reitz <[email protected]>--- block.c | 68 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/059.out | 2 +- tests/qemu-iotests/099.out | 4 +-- tests/qemu-iotests/110.out | 2 +- tests/qemu-iotests/198.out | 4 +-- tests/qemu-iotests/207.out | 10 +++--- 6 files changed, 78 insertions(+), 12 deletions(-)block.c +57
-1+/** +* Take a blockdev @options QDict and convert its values to the +* correct type. +* +* Fail if @options does not match the QAPI schema of BlockdevOptions. +* +* In case of failure, return NULL and set @errp. +* +* In case of success, return a correctly typed new QDict. +*/ +static QDict *bdrv_type_blockdev_opts (const QDict *options,Error **errp) +{ +Visitor *v; +BlockdevOptions *blockdev_options; +QObject *typed_opts; +QDict *string_options; +Error *local_err =NULL ; + + string_options =qdict_clone_shallow (options); + +qdict_flatten (string_options); + v =qobject_input_visitor_new_flat_confused (string_options, errp); +if (!v) { +error_prepend (errp,"Failed to prepare options: " ); +return NULL ; + } + +visit_type_BlockdevOptions (v,NULL , &blockdev_options, &local_err); +visit_free (v); +if (local_err) { +error_propagate (errp, local_err); +error_prepend (errp,"Not a valid BlockdevOptions object: " ); +return NULL ; + } + + v =qobject_output_visitor_new (&typed_opts); +visit_type_BlockdevOptions (v,NULL , &blockdev_options, &local_err); +if (!local_err) { +visit_complete (v, &typed_opts); + } +visit_free (v); +qapi_free_BlockdevOptions (blockdev_options); +if (local_err) { +error_propagate (errp, local_err); +return NULL ; + } + +return qobject_to (QDict, typed_opts); +} +/* Updates the following BDS fields: * - exact_filename: A filename which may be used for opening a block device * which (mostly) equals the given BDS (even without any @@ -5698,10 +5749,25 @@ void bdrv_refresh_filename(BlockDriverState *bs)if (bs->exact_filename[0 ]) {pstrcpy (bs->filename,sizeof (bs->filename), bs->exact_filename); }else { -QString *json =qobject_to_json (QOBJECT (bs->full_open_options)); +QString *json; +QDict *typed_opts, *json_opts; + + typed_opts =bdrv_type_blockdev_opts (bs->full_open_options,NULL ); + +/* +* We cannot be certain that bs->full_open_options matches
block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 5768
Quoted 10 lines
block.c +8
-0+* BlockdevOptions, so bdrv_type_blockdev_opts() may fail. +* That is not fatal, we can just emit bs->full_open_options +* directly -- qemu will accept that, even if it does not +* match the schema. +*/ + json_opts = typed_opts ?: bs->full_open_options; + + json =qobject_to_json (QOBJECT (json_opts));snprintf (bs->filename,sizeof (bs->filename),"json:%s" ,qstring_get_str (json));
I so need to revive my series that created a JSON output visitor (to skip the qobject_to_json() intermediate step). But that's a topic for another day.
tests/qemu-iotests/059.out wrote 512/512 bytes at offset 10240 · line 2053
Quoted 7 lines
tests/qemu-iotests/059.out +2
-1+++ b/tests/qemu-iotests/059.out@@ -2050,7 +2050,7 @@ wrote 512/512 bytes at offset 10240 === Testing monolithicFlat with internally generated JSON file name === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat -can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}' +can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"inject-error": [{"event": "read_aio"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}'
"inject-error" sorts after "image"; meanwhile, the QAPI definition of BlockdevOptionsBlkdebug sorts image before inject-error. It looks like this output is randomized; can that bite us (as we've recently found with other iotests where python 2 vs. 3 sorting mattered)? And can we do better? (My QAPI JSON output visitor would guarantee the output in QAPI definition order)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v3 6/8] block: Try to create well typed json:{} filenames
Quoted 7 lines · Eric Blake
On 2/6/19 1:55 PM, Max Reitz wrote:
In the subject, s/well typed/well-typed/
Quoted 1 line
By applying a health mix of
qdict_flatten(),qdict_crumple(),s/health/healty/
typo in my typo fix for healthy :)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v3 6/8] block: Try to create well typed json:{} filenames
In reply to Eric BlakeQuoted 10 lines · Eric Blake
On 2/6/19 1:55 PM, Max Reitz wrote:
In the subject, s/well typed/well-typed/
Quoted 1 line
By applying a health mix of
qdict_flatten(),qdict_crumple(),s/health/healty/
Quoted 2 lines
qdict_stringify_for_keyval(), the keyval input visitor, and the QObject output visitor (not necessarily in that order),
More importantly, this is just wrong (or rather, outdated). I fixed it in the cover letter but forgot to fix it here. This version just uses qdict_flatten(), the flat-confused input visitor and the output visitor.
There is no longer a qdict_stringify_for_keyval() because Markus's
flat-confused visitor made it unnecessary.
Quoted 113 lines
Quoted 7 lines
we can at least try to
bringbs->full_open_optionsinto accordance with the QAPI schema. This may not always work (there are some options left that have not been QAPI-fied yet), but in practice it usually will.In any case, sometimes emitting wrongly typed json:{} filenames is better than doing it effectively half the time.
And someday, if we ever switch the block layer to use QAPI types all the way, we can drop some of these hacks that have built up over time. But not a show-stopper for this series.
Quoted 83 lines
This affects some iotests because json:{} filenames are now usually crumpled. In 198, "format": "auto" now appears in the qcow2 encryption options because going through a visitor makes optional discriminators' default values explicit.
Signed-off-by Max Reitz <[email protected]>--- block.c | 68 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/059.out | 2 +- tests/qemu-iotests/099.out | 4 +-- tests/qemu-iotests/110.out | 2 +- tests/qemu-iotests/198.out | 4 +-- tests/qemu-iotests/207.out | 10 +++--- 6 files changed, 78 insertions(+), 12 deletions(-)block.c +57
-1+/** +* Take a blockdev @options QDict and convert its values to the +* correct type. +* +* Fail if @options does not match the QAPI schema of BlockdevOptions. +* +* In case of failure, return NULL and set @errp. +* +* In case of success, return a correctly typed new QDict. +*/ +static QDict *bdrv_type_blockdev_opts (const QDict *options,Error **errp) +{ +Visitor *v; +BlockdevOptions *blockdev_options; +QObject *typed_opts; +QDict *string_options; +Error *local_err =NULL ; + + string_options =qdict_clone_shallow (options); + +qdict_flatten (string_options); + v =qobject_input_visitor_new_flat_confused (string_options, errp); +if (!v) { +error_prepend (errp,"Failed to prepare options: " ); +return NULL ; + } + +visit_type_BlockdevOptions (v,NULL , &blockdev_options, &local_err); +visit_free (v); +if (local_err) { +error_propagate (errp, local_err); +error_prepend (errp,"Not a valid BlockdevOptions object: " ); +return NULL ; + } + + v =qobject_output_visitor_new (&typed_opts); +visit_type_BlockdevOptions (v,NULL , &blockdev_options, &local_err); +if (!local_err) { +visit_complete (v, &typed_opts); + } +visit_free (v); +qapi_free_BlockdevOptions (blockdev_options); +if (local_err) { +error_propagate (errp, local_err); +return NULL ; + } + +return qobject_to (QDict, typed_opts); +} +/* Updates the following BDS fields: * - exact_filename: A filename which may be used for opening a block device * which (mostly) equals the given BDS (even without any @@ -5698,10 +5749,25 @@ void bdrv_refresh_filename(BlockDriverState *bs)if (bs->exact_filename[0 ]) {pstrcpy (bs->filename,sizeof (bs->filename), bs->exact_filename); }else { -QString *json =qobject_to_json (QOBJECT (bs->full_open_options)); +QString *json; +QDict *typed_opts, *json_opts; + + typed_opts =bdrv_type_blockdev_opts (bs->full_open_options,NULL ); + +/* +* We cannot be certain that bs->full_open_options matches block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 5768
Quoted 10 lines
block.c +8
-0+* BlockdevOptions, so bdrv_type_blockdev_opts() may fail. +* That is not fatal, we can just emit bs->full_open_options +* directly -- qemu will accept that, even if it does not +* match the schema. +*/ + json_opts = typed_opts ?: bs->full_open_options; + + json =qobject_to_json (QOBJECT (json_opts));snprintf (bs->filename,sizeof (bs->filename),"json:%s" ,qstring_get_str (json));I so need to revive my series that created a JSON output visitor (to skip the
qobject_to_json()intermediate step). But that's a topic for another day.Quoted 3 lines
(quoted diff) +1
-0+++ b/tests/qemu-iotests/059.out@@ -2050,7 +2050,7 @@ wrote 512/512 bytes at offset 10240
Quoted 11 lines
tests/qemu-iotests/059.out wrote 512/512 bytes at offset 10240 · line 2053
Quoted 4 lines
tests/qemu-iotests/059.out +1
-1=== Testing monolithicFlat with internally generated JSON file name === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat -can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}' +can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"inject-error": [{"event": "read_aio"}], "image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug"}'"inject-error" sorts after "image"; meanwhile, the QAPI definition of BlockdevOptionsBlkdebug sorts image before inject-error. It looks like this output is randomized; can that bite us (as we've recently found with other iotests where python 2 vs. 3 sorting mattered)? And can we do better? (My QAPI JSON output visitor would guarantee the output in QAPI definition order)
It can't bite us in the Python 2 vs. 3 sense because the order is determined by qemu code alone (so it is stable for one version of qemu), but of course a real order would be nicer.
I don't think it makes sense to parse the filename in the iotest (for one thing, we'd need to convert this test to Python for that) so we'd be able to guarantee some order in the output, so I think it's best to just live with it for now.
Quoted 1 line
Reviewed-by Eric Blake <[email protected]>
Once again, thanks!
Max
PATCH v3 7/8 iotests: Test internal option typing
In reply to Max ReitzIt would be nice if qemu used the correct types for blockdev options internally, even if the user specified string values (either through -drive or by being not so nice and using json:{} with string values). This patch adds a test verifying that fact.
--- tests/qemu-iotests/089 | 25 +++++++++++++++++++++++++ tests/qemu-iotests/089.out | 9 +++++++++ 2 files changed, 34 insertions(+)
tests/qemu-iotests/089 +25 -0
@@ -35,6 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common.rc . ./common.filter +. ./common.qemu _supported_fmt qcow2 _supported_proto file
@@ -144,6 +145,30 @@ $QEMU_IO -c "open -o driver=qcow2 json:{\"file.filename\":\"$TEST_IMG\"}" \ $QEMU_IO -c "open -o driver=qcow2 json:{\"driver\":\"raw\",\"file.filename\":\"$TEST_IMG\"}" \ -c "info" 2>&1 | _filter_img_info +echo +echo "=== Testing option typing ===" +echo + +# json:{} accepts both strings and correctly typed values (even mixed, +# although we probably do not want to support that...), but when +# creating a json:{} filename, it should be correctly typed. +# Therefore, both of these should make the "size" value an integer. + +TEST_IMG="json:{'driver': 'null-co', 'size': 42 }" _img_info | grep '^image' +TEST_IMG="json:{'driver': 'null-co', 'size': '42'}" _img_info | grep '^image' + +echo + +# This should even work when some driver abuses bs->options to store +# non-QAPI options (and the given -drive options are not complete) +# (Watch for whether file.align appears as an int or a string) +qemu_comm_method=monitor _launch_qemu \ + -drive if=none,id=drv0,node-name=node0,format=raw,file=blkdebug::null-co://,file.align=512 + +_send_qemu_cmd $QEMU_HANDLE 'info block' 'json:' + +_cleanup_qemu + # success, all done echo "*** done"
tests/qemu-iotests/089.out +9 -0
@@ -49,4 +49,13 @@ vm state offset: 512 MiB format name: IMGFMT cluster size: 64 KiB vm state offset: 512 MiB + +=== Testing option typing === + +image: json:{"driver": "null-co", "size": 42} +image: json:{"driver": "null-co", "size": 42} + +QEMU X.Y.Z monitor - type 'help' for more information +(qemu) info block +drv0 (node0): json:{"driver": "raw", "file": {"image": {"driver": "null-co"}, "driver": "blkdebug", "align": 512}} (raw) *** done
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 7/8] iotests: Test internal option typing
Quoted 11 lines · Max Reitz
It would be nice if qemu used the correct types for blockdev options internally, even if the user specified string values (either through -drive or by being not so nice and using json:{} with string values). This patch adds a test verifying that fact.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/089 | 25 +++++++++++++++++++++++++ tests/qemu-iotests/089.out | 9 +++++++++ 2 files changed, 34 insertions(+)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 8/8 iotests: qcow2's encrypt.format is now optional
In reply to Max ReitzRemove the encrypt.format option from the two blockdev-add test cases for encrypted qcow2 images in 087 to explicitly test that this parameter is now optional.
Additionally, show that explicitly specifying encrypt.format=auto works just as well, the same for specifying the correct format (encrypt.format=luks here), and that specifying the wrong format (encrypt.format=aes) results in an error.
While touching this test case, reduce the LUKS iteration time to 10 so the test stays reasonably fast.
--- tests/qemu-iotests/087 | 65 +++++++++++++++++++++++--------------- tests/qemu-iotests/087.out | 26 ++++++++++++++- 2 files changed, 64 insertions(+), 27 deletions(-)
tests/qemu-iotests/087 +39 -26
@@ -149,7 +149,6 @@ run_qemu <<EOF "filename": "$TEST_IMG" }, "encrypt": { - "format": "aes", "key-secret": "sec0" } }
@@ -161,34 +160,48 @@ echo echo === Encrypted image LUKS === echo -_make_test_img --object secret,id=sec0,data=123456 -o encrypt.format=luks,encrypt.key-secret=sec0 $size -run_qemu <<EOF -{ "execute": "qmp_capabilities" } -{ "execute": "object-add", - "arguments": { - "qom-type": "secret", - "id": "sec0", - "props": { - "data": "123456" - } - } -} -{ "execute": "blockdev-add", - "arguments": { - "driver": "$IMGFMT", - "node-name": "disk", - "file": { - "driver": "file", - "filename": "$TEST_IMG" - }, - "encrypt": { - "format": "luks", - "key-secret": "sec0" +_make_test_img \ + --object secret,id=sec0,data=123456 \ + -o encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10 \ + $size + +# Adds the qcow2+LUKS image via blockdev-add. +# First parameter: Optional entry for the 'encrypt' option dict +function luks_test() +{ + run_qemu <<EOF + { "execute": "qmp_capabilities" } + { "execute": "object-add", + "arguments": { + "qom-type": "secret", + "id": "sec0", + "props": { + "data": "123456" + } } } - } -{ "execute": "quit" } + { "execute": "blockdev-add", + "arguments": { + "driver": "$IMGFMT", + "node-name": "disk", + "file": { + "driver": "file", + "filename": "$TEST_IMG" + }, + "encrypt": { + $1 + "key-secret": "sec0" + } + } + } + { "execute": "quit" } EOF +} + +luks_test '' # Implicit encrypt.format=auto +luks_test '"format": "auto",' # Explicit encrypt.format=auto +luks_test '"format": "luks",' # Explicit encrypt.format=luks +luks_test '"format": "aes",' # Explicit encrypt.format=aes (wrong) echo echo === Missing driver ===
tests/qemu-iotests/087.out +25 -1
@@ -46,7 +46,7 @@ QMP_VERSION === Encrypted image LUKS === -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 Testing: QMP_VERSION {"return": {}}
@@ -55,6 +55,30 @@ QMP_VERSION {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +Testing: +QMP_VERSION +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} + +Testing: +QMP_VERSION +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} + +Testing: +QMP_VERSION +{"return": {}} +{"return": {}} +{"error": {"class": "GenericError", "desc": "Header reported 'luks' encryption format but options specify 'aes'"}} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} + === Missing driver ===
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 8/8] iotests: qcow2's encrypt.format is now optional
Quoted 12 lines · Max Reitz
Remove the encrypt.format option from the two blockdev-add test cases for encrypted qcow2 images in 087 to explicitly test that this parameter is now optional.
Additionally, show that explicitly specifying encrypt.format=auto works just as well, the same for specifying the correct format (encrypt.format=luks here), and that specifying the wrong format (encrypt.format=aes) results in an error.
While touching this test case, reduce the LUKS iteration time to 10 so the test stays reasonably fast.
Quoted 7 lines
Signed-off-by Max Reitz <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>--- tests/qemu-iotests/087 | 65 +++++++++++++++++++++++--------------- tests/qemu-iotests/087.out | 26 ++++++++++++++- 2 files changed, 64 insertions(+), 27 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org