qapi: allow unions to contain further unions
PATCH v2 1/3 qapi: improve specificity of type/member descriptions
In reply to Daniel P. BerrangéWhen describing member types always include the context of the containing type. Although this is often redundant, in some cases it will help to reduce ambiguity.
--- scripts/qapi/schema.py | 5 ++--- tests/qapi-schema/alternate-any.err | 2 +- tests/qapi-schema/alternate-conflict-bool-string.err | 2 +- tests/qapi-schema/alternate-conflict-dict.err | 2 +- tests/qapi-schema/alternate-conflict-enum-bool.err | 2 +- tests/qapi-schema/alternate-conflict-enum-int.err | 2 +- tests/qapi-schema/alternate-conflict-lists.err | 2 +- tests/qapi-schema/alternate-conflict-num-string.err | 2 +- tests/qapi-schema/alternate-conflict-string.err | 2 +- tests/qapi-schema/alternate-nested.err | 2 +- tests/qapi-schema/alternate-unknown.err | 2 +- tests/qapi-schema/args-member-unknown.err | 2 +- tests/qapi-schema/enum-clash-member.err | 2 +- tests/qapi-schema/features-duplicate-name.err | 2 +- tests/qapi-schema/struct-base-clash-deep.err | 2 +- tests/qapi-schema/struct-base-clash.err | 2 +- tests/qapi-schema/struct-member-name-clash.err | 2 +- tests/qapi-schema/union-bad-base.err | 2 +- tests/qapi-schema/union-int-branch.err | 2 +- tests/qapi-schema/union-unknown.err | 2 +- 20 files changed, 21 insertions(+), 22 deletions(-)
scripts/qapi/schema.py +2 -3
@@ -713,9 +713,8 @@ def describe(self, info): role = 'base ' + role else: assert False - elif defined_in != info.defn_name: - return "%s '%s' of type '%s'" % (role, self.name, defined_in) - return "%s '%s'" % (role, self.name) + + return "%s '%s' of type '%s'" % (role, self.name, defined_in) class QAPISchemaEnumMember(QAPISchemaMember):
tests/qapi-schema/alternate-any.err +1 -1
@@ -1,2 +1,2 @@ alternate-any.json: In alternate 'Alt': -alternate-any.json:2: branch 'one' cannot use built-in type 'any' +alternate-any.json:2: branch 'one' of type 'Alt' cannot use built-in type 'any'
tests/qapi-schema/alternate-conflict-bool-string.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-bool-string.json: In alternate 'Alt': -alternate-conflict-bool-string.json:2: branch 'two' can't be distinguished from 'one' +alternate-conflict-bool-string.json:2: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-conflict-dict.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-dict.json: In alternate 'Alt': -alternate-conflict-dict.json:6: branch 'two' can't be distinguished from 'one' +alternate-conflict-dict.json:6: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-conflict-enum-bool.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-enum-bool.json: In alternate 'Alt': -alternate-conflict-enum-bool.json:4: branch 'two' can't be distinguished from 'one' +alternate-conflict-enum-bool.json:4: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-conflict-enum-int.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-enum-int.json: In alternate 'Alt': -alternate-conflict-enum-int.json:4: branch 'two' can't be distinguished from 'one' +alternate-conflict-enum-int.json:4: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-conflict-lists.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-lists.json: In alternate 'Alt': -alternate-conflict-lists.json:4: branch 'two' can't be distinguished from 'one' +alternate-conflict-lists.json:4: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-conflict-num-string.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-num-string.json: In alternate 'Alt': -alternate-conflict-num-string.json:2: branch 'two' can't be distinguished from 'one' +alternate-conflict-num-string.json:2: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-conflict-string.err +1 -1
@@ -1,2 +1,2 @@ alternate-conflict-string.json: In alternate 'Alt': -alternate-conflict-string.json:2: branch 'two' can't be distinguished from 'one' +alternate-conflict-string.json:2: branch 'two' of type 'Alt' can't be distinguished from 'one'
tests/qapi-schema/alternate-nested.err +1 -1
@@ -1,2 +1,2 @@ alternate-nested.json: In alternate 'Alt2': -alternate-nested.json:4: branch 'nested' cannot use alternate type 'Alt1' +alternate-nested.json:4: branch 'nested' of type 'Alt2' cannot use alternate type 'Alt1'
tests/qapi-schema/alternate-unknown.err +1 -1
@@ -1,2 +1,2 @@ alternate-unknown.json: In alternate 'Alt': -alternate-unknown.json:2: branch 'unknown' uses unknown type 'MissingType' +alternate-unknown.json:2: branch 'unknown' of type 'Alt' uses unknown type 'MissingType'
tests/qapi-schema/args-member-unknown.err +1 -1
@@ -1,2 +1,2 @@ args-member-unknown.json: In command 'oops': -args-member-unknown.json:2: parameter 'member' uses unknown type 'NoSuchType' +args-member-unknown.json:2: parameter 'member' of type 'oops-arg' uses unknown type 'NoSuchType'
tests/qapi-schema/enum-clash-member.err +1 -1
@@ -1,2 +1,2 @@ enum-clash-member.json: In enum 'MyEnum': -enum-clash-member.json:3: value 'one_two' collides with value 'one-two' +enum-clash-member.json:3: value 'one_two' of type 'MyEnum' collides with value 'one-two' of type 'MyEnum'
tests/qapi-schema/features-duplicate-name.err +1 -1
@@ -1,2 +1,2 @@ features-duplicate-name.json: In struct 'FeatureStruct0': -features-duplicate-name.json:1: feature 'foo' collides with feature 'foo' +features-duplicate-name.json:1: feature 'foo' of type 'FeatureStruct0' collides with feature 'foo' of type 'FeatureStruct0'
tests/qapi-schema/struct-base-clash-deep.err +1 -1
@@ -1,2 +1,2 @@ struct-base-clash-deep.json: In struct 'Sub': -struct-base-clash-deep.json:10: member 'name' collides with member 'name' of type 'Base' +struct-base-clash-deep.json:10: member 'name' of type 'Sub' collides with member 'name' of type 'Base'
tests/qapi-schema/struct-base-clash.err +1 -1
@@ -1,2 +1,2 @@ struct-base-clash.json: In struct 'Sub': -struct-base-clash.json:5: member 'name' collides with member 'name' of type 'Base' +struct-base-clash.json:5: member 'name' of type 'Sub' collides with member 'name' of type 'Base'
tests/qapi-schema/struct-member-name-clash.err +1 -1
@@ -1,2 +1,2 @@ struct-member-name-clash.json: In struct 'Oops': -struct-member-name-clash.json:5: member 'a_b' collides with member 'a-b' +struct-member-name-clash.json:5: member 'a_b' of type 'Oops' collides with member 'a-b' of type 'Oops'
tests/qapi-schema/union-bad-base.err +1 -1
@@ -1,2 +1,2 @@ union-bad-base.json: In union 'TestUnion': -union-bad-base.json:8: member 'string' of type 'TestTypeA' collides with base member 'string' +union-bad-base.json:8: member 'string' of type 'TestTypeA' collides with base member 'string' of type 'TestUnion-base'
tests/qapi-schema/union-int-branch.err +1 -1
@@ -1,2 +1,2 @@ union-int-branch.json: In union 'TestUnion': -union-int-branch.json:8: branch 'value1' cannot use built-in type 'int' +union-int-branch.json:8: branch 'value1' of type 'TestUnion' cannot use built-in type 'int'
tests/qapi-schema/union-unknown.err +1 -1
@@ -1,2 +1,2 @@ union-unknown.json: In union 'Union': -union-unknown.json:3: branch 'unknown' uses unknown type 'MissingType' +union-unknown.json:3: branch 'unknown' of type 'Union' uses unknown type 'MissingType'
Signature · 1 line
2.39.2
v2 cover qapi: allow unions to contain further unions
Currently it is not possible for a union type to contain a further union as one (or more) of its branches. This relaxes that restriction and adds the calls needed to validate field name uniqueness as unions are flattened.
In v2:
* Improve specificity of type/members descriptions for error reporting scenarios
* Make it easier to regenerate qapi test output
* Move expected "good data" into qapi-schema-test.json
* Add description to "bad data" test files
* Add unit tests to cover union-in-union serialization / deserialization to/from JSON
Daniel P. Berrangé (3):
qapi: improve specificity of type/member descriptions
qapi: use env var to trigger qapi test output updates
qapi: allow unions to contain further unions
scripts/qapi/schema.py | 11 ++-- tests/qapi-schema/alternate-any.err | 2 +- .../alternate-conflict-bool-string.err | 2 +- tests/qapi-schema/alternate-conflict-dict.err | 2 +- .../alternate-conflict-enum-bool.err | 2 +- .../alternate-conflict-enum-int.err | 2 +- .../qapi-schema/alternate-conflict-lists.err | 2 +- .../alternate-conflict-num-string.err | 2 +- .../qapi-schema/alternate-conflict-string.err | 2 +- tests/qapi-schema/alternate-nested.err | 2 +- tests/qapi-schema/alternate-unknown.err | 2 +- tests/qapi-schema/args-member-unknown.err | 2 +- tests/qapi-schema/enum-clash-member.err | 2 +- tests/qapi-schema/features-duplicate-name.err | 2 +- tests/qapi-schema/meson.build | 2 + tests/qapi-schema/qapi-schema-test.json | 32 ++++++++++ tests/qapi-schema/qapi-schema-test.out | 29 ++++++++++ tests/qapi-schema/struct-base-clash-deep.err | 2 +- tests/qapi-schema/struct-base-clash.err | 2 +- .../qapi-schema/struct-member-name-clash.err | 2 +- tests/qapi-schema/test-qapi.py | 3 +- tests/qapi-schema/union-bad-base.err | 2 +- tests/qapi-schema/union-int-branch.err | 2 +- .../union-invalid-union-subfield.err | 2 + .../union-invalid-union-subfield.json | 30 ++++++++++ .../union-invalid-union-subfield.out | 0 .../union-invalid-union-subtype.err | 2 + .../union-invalid-union-subtype.json | 29 ++++++++++ .../union-invalid-union-subtype.out | 0 tests/qapi-schema/union-unknown.err | 2 +- tests/unit/test-qobject-input-visitor.c | 47 +++++++++++++++ tests/unit/test-qobject-output-visitor.c | 58 +++++++++++++++++++ 32 files changed, 257 insertions(+), 26 deletions(-) create mode 100644 tests/qapi-schema/union-invalid-union-subfield.err create mode 100644 tests/qapi-schema/union-invalid-union-subfield.json create mode 100644 tests/qapi-schema/union-invalid-union-subfield.out create mode 100644 tests/qapi-schema/union-invalid-union-subtype.err create mode 100644 tests/qapi-schema/union-invalid-union-subtype.json create mode 100644 tests/qapi-schema/union-invalid-union-subtype.out
Signature · 1 line
2.39.2
PATCH v2 3/3 qapi: allow unions to contain further unions
This extends the QAPI schema validation to permit unions inside unions, provided the checks for clashing fields pass.
--- scripts/qapi/schema.py | 6 +- tests/qapi-schema/meson.build | 2 + tests/qapi-schema/qapi-schema-test.json | 32 ++++++++++ tests/qapi-schema/qapi-schema-test.out | 29 ++++++++++ .../union-invalid-union-subfield.err | 2 + .../union-invalid-union-subfield.json | 30 ++++++++++ .../union-invalid-union-subfield.out | 0 .../union-invalid-union-subtype.err | 2 + .../union-invalid-union-subtype.json | 29 ++++++++++ .../union-invalid-union-subtype.out | 0 tests/unit/test-qobject-input-visitor.c | 47 +++++++++++++++ tests/unit/test-qobject-output-visitor.c | 58 +++++++++++++++++++ 12 files changed, 234 insertions(+), 3 deletions(-) create mode 100644 tests/qapi-schema/union-invalid-union-subfield.err create mode 100644 tests/qapi-schema/union-invalid-union-subfield.json create mode 100644 tests/qapi-schema/union-invalid-union-subfield.out create mode 100644 tests/qapi-schema/union-invalid-union-subtype.err create mode 100644 tests/qapi-schema/union-invalid-union-subtype.json create mode 100644 tests/qapi-schema/union-invalid-union-subtype.out
scripts/qapi/schema.py +3 -3
@@ -465,9 +465,10 @@ def check(self, schema): # on behalf of info, which is not necessarily self.info def check_clash(self, info, seen): assert self._checked - assert not self.variants # not implemented for m in self.members: m.check_clash(info, seen) + if self.variants: + self.variants.check_clash(info, seen) def connect_doc(self, doc=None): super().connect_doc(doc)
@@ -652,8 +653,7 @@ def check(self, schema, seen): self.info, "branch '%s' is not a value of %s" % (v.name, self.tag_member.type.describe())) - if (not isinstance(v.type, QAPISchemaObjectType) - or v.type.variants): + if not isinstance(v.type, QAPISchemaObjectType): raise QAPISemError( self.info, "%s cannot use %s"
tests/qapi-schema/meson.build +2 -0
@@ -194,6 +194,8 @@ schemas = [ 'union-invalid-data.json', 'union-invalid-discriminator.json', 'union-invalid-if-discriminator.json', + 'union-invalid-union-subfield.json', + 'union-invalid-union-subtype.json', 'union-no-base.json', 'union-optional-discriminator.json', 'union-string-discriminator.json',
tests/qapi-schema/qapi-schema-test.json +32 -0
@@ -114,6 +114,38 @@ { 'struct': 'UserDefC', 'data': { 'string1': 'str', 'string2': 'str' } } +# this tests that unions can contain other unions in their branches +{ 'enum': 'TestUnionEnum', + 'data': [ 'value-a', 'value-b' ] } + +{ 'enum': 'TestUnionEnumA', + 'data': [ 'value-a1', 'value-a2' ] } + +{ 'struct': 'TestUnionTypeA1', + 'data': { 'integer': 'int', + 'name': 'str'} } + +{ 'struct': 'TestUnionTypeA2', + 'data': { 'integer': 'int', + 'size': 'int' } } + +{ 'union': 'TestUnionTypeA', + 'base': { 'type-a': 'TestUnionEnumA' }, + 'discriminator': 'type-a', + 'data': { 'value-a1': 'TestUnionTypeA1', + 'value-a2': 'TestUnionTypeA2' } } + +{ 'struct': 'TestUnionTypeB', + 'data': { 'integer': 'int', + 'onoff': 'bool' } } + +{ 'union': 'TestUnionInUnion', + 'base': { 'type': 'TestUnionEnum' }, + 'discriminator': 'type', + 'data': { 'value-a': 'TestUnionTypeA', + 'value-b': 'TestUnionTypeB' } } + + # for testing use of 'number' within alternates { 'alternate': 'AltEnumBool', 'data': { 'e': 'EnumOne', 'b': 'bool' } } { 'alternate': 'AltEnumNum', 'data': { 'e': 'EnumOne', 'n': 'number' } }
tests/qapi-schema/qapi-schema-test.out +29 -0
@@ -105,6 +105,35 @@ alternate UserDefAlternate object UserDefC member string1: str optional=False member string2: str optional=False +enum TestUnionEnum + member value-a + member value-b +enum TestUnionEnumA + member value-a1 + member value-a2 +object TestUnionTypeA1 + member integer: int optional=False + member name: str optional=False +object TestUnionTypeA2 + member integer: int optional=False + member size: int optional=False +object q_obj_TestUnionTypeA-base + member type-a: TestUnionEnumA optional=False +object TestUnionTypeA + base q_obj_TestUnionTypeA-base + tag type-a + case value-a1: TestUnionTypeA1 + case value-a2: TestUnionTypeA2 +object TestUnionTypeB + member integer: int optional=False + member onoff: bool optional=False +object q_obj_TestUnionInUnion-base + member type: TestUnionEnum optional=False +object TestUnionInUnion + base q_obj_TestUnionInUnion-base + tag type + case value-a: TestUnionTypeA + case value-b: TestUnionTypeB alternate AltEnumBool tag type case e: EnumOne
tests/qapi-schema/union-invalid-union-subfield.err new file +2 -0
@@ -0,0 +1,2 @@ +union-invalid-union-subfield.json: In union 'TestUnion': +union-invalid-union-subfield.json:25: member 'teeth' of type 'TestTypeFish' collides with base member 'teeth' of type 'TestUnion-base'
tests/qapi-schema/union-invalid-union-subfield.json new file +30 -0
@@ -0,0 +1,30 @@ +# Clash between common member and union variant's variant member +# Base's member 'teeth' clashes with TestTypeFish's + +{ 'enum': 'TestEnum', + 'data': [ 'animals', 'plants' ] } + +{ 'enum': 'TestAnimals', + 'data': [ 'fish', 'birds'] } + +{ 'struct': 'TestTypeFish', + 'data': { 'scales': 'int', 'teeth': 'int' } } + +{ 'struct': 'TestTypeBirds', + 'data': { 'feathers': 'int' } } + +{ 'union': 'TestTypeAnimals', + 'base': { 'atype': 'TestAnimals' }, + 'discriminator': 'atype', + 'data': { 'fish': 'TestTypeFish', + 'birds': 'TestTypeBirds' } } + +{ 'struct': 'TestTypePlants', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestUnion', + 'base': { 'type': 'TestEnum', + 'teeth': 'int' }, + 'discriminator': 'type', + 'data': { 'animals': 'TestTypeAnimals', + 'plants': 'TestTypePlants' } }
tests/qapi-schema/union-invalid-union-subfield.out new file +0 -0
tests/qapi-schema/union-invalid-union-subtype.err new file +2 -0
@@ -0,0 +1,2 @@ +union-invalid-union-subtype.json: In union 'TestUnion': +union-invalid-union-subtype.json:25: base member 'type' of type 'TestTypeA-base' collides with base member 'type' of type 'TestUnion-base'
tests/qapi-schema/union-invalid-union-subtype.json new file +29 -0
@@ -0,0 +1,29 @@ +# Clash between common member and union variant's common member +# Base's member 'type' clashes with TestTypeA's + +{ 'enum': 'TestEnum', + 'data': [ 'value-a', 'value-b' ] } + +{ 'enum': 'TestEnumA', + 'data': [ 'value-a1', 'value-a2' ] } + +{ 'struct': 'TestTypeA1', + 'data': { 'integer': 'int' } } + +{ 'struct': 'TestTypeA2', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestTypeA', + 'base': { 'type': 'TestEnumA' }, + 'discriminator': 'type', + 'data': { 'value-a1': 'TestTypeA1', + 'value-a2': 'TestTypeA2' } } + +{ 'struct': 'TestTypeB', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestUnion', + 'base': { 'type': 'TestEnum' }, + 'discriminator': 'type', + 'data': { 'value-a': 'TestTypeA', + 'value-b': 'TestTypeB' } }
tests/qapi-schema/union-invalid-union-subtype.out new file +0 -0
tests/unit/test-qobject-input-visitor.c +47 -0
59 lines not on this page · raw message
tests/unit/test-qobject-output-visitor.c +58 -0
70 lines not on this page · raw message
Signature · 1 line
2.39.2
PATCH v2 2/3 qapi: use env var to trigger qapi test output updates
In reply to Daniel P. BerrangéIt is possible to pass --update to tests/qapi-schema/test-qapi.py to make it update the output files on error. This is inconvient to achieve though when test-qapi.py is run indirectly by make/meson.
Instead simply allow for an env variable to be set:
$ QAPI_TEST_UPDATE=1 make check-qapi-schema
--- tests/qapi-schema/test-qapi.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/qapi-schema/test-qapi.py +2 -1
@@ -215,7 +215,8 @@ def main(argv): (dir_name, base_name) = os.path.split(t) dir_name = dir_name or args.dir test_name = os.path.splitext(base_name)[0] - status |= test_and_diff(test_name, dir_name, args.update) + update = args.update or "QAPI_TEST_UPDATE" in os.environ + status |= test_and_diff(test_name, dir_name, update) exit(status)
Signature · 1 line
2.39.2
Re: [PATCH v2 2/3] qapi: use env var to trigger qapi test output updates
Quoted 2 lines · Daniel P. Berrangé
It is possible to pass --update to
tests/qapi-schema/test-qapi.pyto make it update the output files on error. This is inconvient
inconvenient
Quoted 16 lines
to achieve though when
test-qapi.pyis run indirectly by make/meson.Instead simply allow for an env variable to be set:
$ QAPI_TEST_UPDATE=1 make check-qapi-schemaSigned-off-by Daniel P. Berrangé <[email protected]>--- tests/qapi-schema/test-qapi.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tests/qapi-schema/test-qapi.py +0
-0
tests/qapi-schema/test-qapi.py def main(argv): · line 219
Quoted 6 lines
tests/qapi-schema/test-qapi.py +2
-1@@ -215,7 +215,8 @@ def main(argv): (dir_name, base_name) = os.path.split(t) dir_name = dir_name or args.dir test_name = os.path.splitext(base_name)[0] - status |= test_and_diff(test_name, dir_name, args.update) + update = args.update or "QAPI_TEST_UPDATE" in os.environ + status |= test_and_diff(test_name, dir_name, update)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Hi,
Quoted 373 lines · Daniel P. Berrangé
This extends the QAPI schema validation to permit unions inside unions, provided the checks for clashing fields pass.
Signed-off-by Daniel P. Berrangé <[email protected]>--- scripts/qapi/schema.py | 6 +- tests/qapi-schema/meson.build | 2 + tests/qapi-schema/qapi-schema-test.json | 32 ++++++++++ tests/qapi-schema/qapi-schema-test.out | 29 ++++++++++ .../union-invalid-union-subfield.err | 2 + .../union-invalid-union-subfield.json | 30 ++++++++++ .../union-invalid-union-subfield.out | 0 .../union-invalid-union-subtype.err | 2 + .../union-invalid-union-subtype.json | 29 ++++++++++ .../union-invalid-union-subtype.out | 0 tests/unit/test-qobject-input-visitor.c | 47 +++++++++++++++ tests/unit/test-qobject-output-visitor.c | 58 +++++++++++++++++++ 12 files changed, 234 insertions(+), 3 deletions(-)create mode 100644 tests/qapi-schema/union-invalid-union-subfield.err create mode 100644
tests/qapi-schema/union-invalid-union-subfield.jsoncreate mode 100644 tests/qapi-schema/union-invalid-union-subfield.out create mode 100644 tests/qapi-schema/union-invalid-union-subtype.err create mode 100644tests/qapi-schema/union-invalid-union-subtype.jsoncreate mode 100644 tests/qapi-schema/union-invalid-union-subtype.outscripts/qapi/schema.py +3
-3@@ -465,9 +465,10 @@ def check(self, schema): # on behalf of info, which is not necessarily self.info def check_clash(self, info, seen): assert self._checked - assert not self.variants # not implemented for m in self.members: m.check_clash(info, seen) + if self.variants: + self.variants.check_clash(info, seen) def connect_doc(self, doc=None): super().connect_doc(doc)@@ -652,8 +653,7 @@ def check(self, schema, seen): self.info, "branch '%s' is not a value of %s" % (v.name, self.tag_member.type.describe())) - if (not isinstance(v.type, QAPISchemaObjectType) - or v.type.variants): + if not isinstance(v.type, QAPISchemaObjectType): raise QAPISemError( self.info, "%s cannot use %s"tests/qapi-schema/meson.build +2
-0@@ -194,6 +194,8 @@ schemas = [ 'union-invalid-data.json', 'union-invalid-discriminator.json', 'union-invalid-if-discriminator.json', + 'union-invalid-union-subfield.json', + 'union-invalid-union-subtype.json', 'union-no-base.json', 'union-optional-discriminator.json', 'union-string-discriminator.json',tests/qapi-schema/qapi-schema-test.json +32
-0@@ -114,6 +114,38 @@ { 'struct': 'UserDefC', 'data': { 'string1': 'str', 'string2': 'str' } } +# this tests that unions can contain other unions in their branches +{ 'enum': 'TestUnionEnum', + 'data': [ 'value-a', 'value-b' ] } + +{ 'enum': 'TestUnionEnumA', + 'data': [ 'value-a1', 'value-a2' ] } + +{ 'struct': 'TestUnionTypeA1', + 'data': { 'integer': 'int', + 'name': 'str'} } + +{ 'struct': 'TestUnionTypeA2', + 'data': { 'integer': 'int', + 'size': 'int' } } + +{ 'union': 'TestUnionTypeA', + 'base': { 'type-a': 'TestUnionEnumA' }, + 'discriminator': 'type-a', + 'data': { 'value-a1': 'TestUnionTypeA1', + 'value-a2': 'TestUnionTypeA2' } } + +{ 'struct': 'TestUnionTypeB', + 'data': { 'integer': 'int', + 'onoff': 'bool' } } + +{ 'union': 'TestUnionInUnion', + 'base': { 'type': 'TestUnionEnum' }, + 'discriminator': 'type', + 'data': { 'value-a': 'TestUnionTypeA', + 'value-b': 'TestUnionTypeB' } } + + # for testing use of 'number' within alternates { 'alternate': 'AltEnumBool', 'data': { 'e': 'EnumOne', 'b': 'bool' } } { 'alternate': 'AltEnumNum', 'data': { 'e': 'EnumOne', 'n': 'number' } }tests/qapi-schema/qapi-schema-test.out +29
-0@@ -105,6 +105,35 @@ alternate UserDefAlternate object UserDefC member string1: str optional=False member string2: str optional=False +enum TestUnionEnum + member value-a + member value-b +enum TestUnionEnumA + member value-a1 + member value-a2 +object TestUnionTypeA1 + member integer: int optional=False + member name: str optional=False +object TestUnionTypeA2 + member integer: int optional=False + member size: int optional=False +object q_obj_TestUnionTypeA-base + member type-a: TestUnionEnumA optional=False +object TestUnionTypeA + base q_obj_TestUnionTypeA-base + tag type-a + case value-a1: TestUnionTypeA1 + case value-a2: TestUnionTypeA2 +object TestUnionTypeB + member integer: int optional=False + member onoff: bool optional=False +object q_obj_TestUnionInUnion-base + member type: TestUnionEnum optional=False +object TestUnionInUnion + base q_obj_TestUnionInUnion-base + tag type + case value-a: TestUnionTypeA + case value-b: TestUnionTypeB alternate AltEnumBool tag type case e: EnumOnetests/qapi-schema/union-invalid-union-subfield.err new file +2
-0@@ -0,0 +1,2 @@ +union-invalid-union-subfield.json: In union 'TestUnion': +union-invalid-union-subfield.json:25: member 'teeth' of type 'TestTypeFish' collides with base member 'teeth' of type 'TestUnion-base'tests/qapi-schema/union-invalid-union-subfield.json new file +30
-0@@ -0,0 +1,30 @@ +# Clash between common member and union variant's variant member +# Base's member 'teeth' clashes with TestTypeFish's + +{ 'enum': 'TestEnum', + 'data': [ 'animals', 'plants' ] } + +{ 'enum': 'TestAnimals', + 'data': [ 'fish', 'birds'] } + +{ 'struct': 'TestTypeFish', + 'data': { 'scales': 'int', 'teeth': 'int' } } + +{ 'struct': 'TestTypeBirds', + 'data': { 'feathers': 'int' } } + +{ 'union': 'TestTypeAnimals', + 'base': { 'atype': 'TestAnimals' }, + 'discriminator': 'atype', + 'data': { 'fish': 'TestTypeFish', + 'birds': 'TestTypeBirds' } } + +{ 'struct': 'TestTypePlants', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestUnion', + 'base': { 'type': 'TestEnum', + 'teeth': 'int' }, + 'discriminator': 'type', + 'data': { 'animals': 'TestTypeAnimals', + 'plants': 'TestTypePlants' } }tests/qapi-schema/union-invalid-union-subfield.out new file +0
-0tests/qapi-schema/union-invalid-union-subtype.err new file +2
-0@@ -0,0 +1,2 @@ +union-invalid-union-subtype.json: In union 'TestUnion': +union-invalid-union-subtype.json:25: base member 'type' of type 'TestTypeA-base' collides with base member 'type' of type 'TestUnion-base'tests/qapi-schema/union-invalid-union-subtype.json new file +29
-0@@ -0,0 +1,29 @@ +# Clash between common member and union variant's common member +# Base's member 'type' clashes with TestTypeA's + +{ 'enum': 'TestEnum', + 'data': [ 'value-a', 'value-b' ] } + +{ 'enum': 'TestEnumA', + 'data': [ 'value-a1', 'value-a2' ] } + +{ 'struct': 'TestTypeA1', + 'data': { 'integer': 'int' } } + +{ 'struct': 'TestTypeA2', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestTypeA', + 'base': { 'type': 'TestEnumA' }, + 'discriminator': 'type', + 'data': { 'value-a1': 'TestTypeA1', + 'value-a2': 'TestTypeA2' } } + +{ 'struct': 'TestTypeB', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestUnion', + 'base': { 'type': 'TestEnum' }, + 'discriminator': 'type', + 'data': { 'value-a': 'TestTypeA', + 'value-b': 'TestTypeB' } }tests/qapi-schema/union-invalid-union-subtype.out new file +0
-0tests/unit/test-qobject-input-visitor.c +47
-0@@ -706,6 +706,51 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,g_assert (&base->enum1 == &tmp->enum1); } +static void test_visitor_in_union_in_union (TestInputVisitorData *data, +const void *unused) +{ +Visitor *v; +g_autoptr (TestUnionInUnion) tmp =NULL ; + + v =visitor_input_test_init (data, +"{ 'type': 'value-a', " +" 'type-a': 'value-a1', " +" 'integer': 2, " +" 'name': 'fish' }" ); + +visit_type_TestUnionInUnion (v,NULL , &tmp, &error_abort); + g_assert_cmpint(tmp->type, ==, TEST_UNION_ENUM_VALUE_A); + g_assert_cmpint(tmp->u.value_a.type_a, ==, TEST_UNION_ENUMA_VALUE_A1); + g_assert_cmpint(tmp->u.value_a.u.value_a1.integer, ==, 2); + g_assert_cmpint(strcmp(tmp->u.value_a.u.value_a1.name, "fish"), ==, 0); + +qapi_free_TestUnionInUnion (tmp); + + v =visitor_input_test_init (data, +"{ 'type': 'value-a', " +" 'type-a': 'value-a2', " +" 'integer': 1729, " +" 'size': 87539319 }" ); + +visit_type_TestUnionInUnion (v,NULL , &tmp, &error_abort); + g_assert_cmpint(tmp->type, ==, TEST_UNION_ENUM_VALUE_A); + g_assert_cmpint(tmp->u.value_a.type_a, ==, TEST_UNION_ENUMA_VALUE_A2); + g_assert_cmpint(tmp->u.value_a.u.value_a2.integer, ==, 1729); + g_assert_cmpint(tmp->u.value_a.u.value_a2.size, ==, 87539319); + +qapi_free_TestUnionInUnion (tmp); + + v =visitor_input_test_init (data, +"{ 'type': 'value-b', " +" 'integer': 1729, " +" 'onoff': true }" ); + +visit_type_TestUnionInUnion (v,NULL , &tmp, &error_abort); + g_assert_cmpint(tmp->type, ==, TEST_UNION_ENUM_VALUE_B); + g_assert_cmpint(tmp->u.value_b.integer, ==, 1729); + g_assert_cmpint(tmp->u.value_b.onoff, ==, true); +} +static void test_visitor_in_alternate (TestInputVisitorData *data,const void *unused) {@@ -1216,6 +1261,8 @@ int main(int argc, char **argv) NULL, test_visitor_in_null);input_visitor_test_add ("/visitor/input/union-flat" ,NULL , test_visitor_in_union_flat); +input_visitor_test_add ("/visitor/input/union-in-union" , +NULL , test_visitor_in_union_in_union);input_visitor_test_add ("/visitor/input/alternate" ,NULL , test_visitor_in_alternate); input_visitor_test_add("/visitor/input/errors",tests/unit/test-qobject-output-visitor.c +56
-0@@ -352,6 +352,62 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,qapi_free_UserDefFlatUnion (tmp); } +static void test_visitor_out_union_in_union (TestOutputVisitorData *data, +const void *unused) +{ +QDict *qdict; + +TestUnionInUnion *tmp =g_new0 (TestUnionInUnion,1 ); + tmp->type =TEST_UNION_ENUM_VALUE_A ; + tmp->u.value_a.type_a =TEST_UNION_ENUMA_VALUE_A1 ; + tmp->u.value_a.u.value_a1.integer =42 ; + tmp->u.value_a.u.value_a1.name =g_strdup ("fish" ); + +visit_type_TestUnionInUnion (data->ov,NULL , &tmp, &error_abort); + qdict =qobject_to (QDict,visitor_get (data)); +g_assert (qdict); + g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "value-a"); + g_assert_cmpstr(qdict_get_str(qdict, "type-a"), ==, "value-a1"); + g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 42); + g_assert_cmpstr(qdict_get_str(qdict, "name"), ==, "fish"); + +qapi_free_TestUnionInUnion (tmp); + + +visitor_reset (data); + tmp =g_new0 (TestUnionInUnion,1 ); + tmp->type =TEST_UNION_ENUM_VALUE_A ; + tmp->u.value_a.type_a =TEST_UNION_ENUMA_VALUE_A2 ; + tmp->u.value_a.u.value_a2.integer =1729 ; + tmp->u.value_a.u.value_a2.size =87539319 ; + +visit_type_TestUnionInUnion (data->ov,NULL , &tmp, &error_abort); + qdict =qobject_to (QDict,visitor_get (data)); +g_assert (qdict); + g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "value-a"); + g_assert_cmpstr(qdict_get_str(qdict, "type-a"), ==, "value-a2"); + g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 1729); + g_assert_cmpint(qdict_get_int(qdict, "size"), ==, 87539319); + +qapi_free_TestUnionInUnion (tmp); + + +visitor_reset (data); + tmp =g_new0 (TestUnionInUnion,1 ); + tmp->type =TEST_UNION_ENUM_VALUE_B ; + tmp->u.value_b.integer =1729 ; + tmp->u.value_b.onoff = true; + +visit_type_TestUnionInUnion (data->ov,NULL , &tmp, &error_abort); + qdict =qobject_to (QDict,visitor_get (data)); +g_assert (qdict); + g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "value-b"); + g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 1729); + g_assert_cmpint(qdict_get_bool(qdict, "onoff"), ==, true); + +qapi_free_TestUnionInUnion (tmp); +} +static void test_visitor_out_alternate (TestOutputVisitorData *data,const void *unused) {
tests/unit/test-qobject-output-visitor.c int main(int argc, char **argv) · line 649
Quoted 8 lines
tests/unit/test-qobject-output-visitor.c +2
-0@@ -586,6 +642,8 @@ int main(int argc, char **argv) &out_visitor_data, test_visitor_out_list_qapi_free);output_visitor_test_add ("/visitor/output/union-flat" , &out_visitor_data, test_visitor_out_union_flat); +output_visitor_test_add ("/visitor/output/union-in-union" , + &out_visitor_data, test_visitor_out_union_in_union);output_visitor_test_add ("/visitor/output/alternate" , &out_visitor_data, test_visitor_out_alternate); output_visitor_test_add("/visitor/output/null",
The changes look good to me. I have tried to built migrate QAPI changes on top of Daniel's v1 changes, and it was successful.
Once Markus and other maintainers approves this patchset, I can post v4 version of 'migrate' qapi changes on top of this change.
Looking forward for replies on this patchset :)
Regards,
Het Gala
Re: [PATCH v2 2/3] qapi: use env var to trigger qapi test output updates
In reply to Daniel P. BerrangéI have a mild dislike of abbreviations like "env var". Perhaps:
qapi: Support updating expected test output via make
Quoted 7 lines · Daniel P. Berrangé
It is possible to pass --update to
tests/qapi-schema/test-qapi.pyto make it update the output files on error. This is inconvient to achieve though whentest-qapi.pyis run indirectly by make/meson.Instead simply allow for an env variable to be set:
$ QAPI_TEST_UPDATE=1 make check-qapi-schema
Suggest to omit the value
$ QAPI_TEST_UPDATE= make check-qapi-schema
The value doesn't actually matter. A value of 1 looks as if it did, and as if a value of 0 would disable the thing.
I'm no fan of environment variables duplicating options, but I understand the desire for a make invocation, and I don't have a better idea.
Quoted 10 lines
Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/qapi-schema/test-qapi.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tests/qapi-schema/test-qapi.py +0
-0
tests/qapi-schema/test-qapi.py def main(argv): · line 221
Quoted 8 lines
tests/qapi-schema/test-qapi.py +2
-1@@ -215,7 +215,8 @@ def main(argv): (dir_name, base_name) = os.path.split(t) dir_name = dir_name or args.dir test_name = os.path.splitext(base_name)[0] - status |= test_and_diff(test_name, dir_name, args.update) + update = args.update or "QAPI_TEST_UPDATE" in os.environ + status |= test_and_diff(test_name, dir_name, update) exit(status)
Let's use argparse instead:
parser.add_argument('-u', '--update', action='store_true', + default='QAPI_TEST_UPDATE' in os.environ,
help="update expected test results")
Re: [PATCH v2 1/3] qapi: improve specificity of type/member descriptions
In reply to Daniel P. BerrangéQuoted 3 lines · Daniel P. Berrangé
When describing member types always include the context of the containing type. Although this is often redundant, in some cases it will help to reduce ambiguity.
Unfortunately, it can also be confusing, as we shall see below.
Quoted 45 lines
Signed-off-by Daniel P. Berrangé <[email protected]>--- scripts/qapi/schema.py | 5 ++--- tests/qapi-schema/alternate-any.err | 2 +- tests/qapi-schema/alternate-conflict-bool-string.err | 2 +- tests/qapi-schema/alternate-conflict-dict.err | 2 +- tests/qapi-schema/alternate-conflict-enum-bool.err | 2 +- tests/qapi-schema/alternate-conflict-enum-int.err | 2 +- tests/qapi-schema/alternate-conflict-lists.err | 2 +- tests/qapi-schema/alternate-conflict-num-string.err | 2 +- tests/qapi-schema/alternate-conflict-string.err | 2 +- tests/qapi-schema/alternate-nested.err | 2 +- tests/qapi-schema/alternate-unknown.err | 2 +- tests/qapi-schema/args-member-unknown.err | 2 +- tests/qapi-schema/enum-clash-member.err | 2 +- tests/qapi-schema/features-duplicate-name.err | 2 +- tests/qapi-schema/struct-base-clash-deep.err | 2 +- tests/qapi-schema/struct-base-clash.err | 2 +- tests/qapi-schema/struct-member-name-clash.err | 2 +- tests/qapi-schema/union-bad-base.err | 2 +- tests/qapi-schema/union-int-branch.err | 2 +- tests/qapi-schema/union-unknown.err | 2 +- 20 files changed, 21 insertions(+), 22 deletions(-)scripts/qapi/schema.py +2
-3@@ -713,9 +713,8 @@ def describe(self, info): role = 'base ' + role else: assert False - elif defined_in != info.defn_name: - return "%s '%s' of type '%s'" % (role, self.name, defined_in) - return "%s '%s'" % (role, self.name) + + return "%s '%s' of type '%s'" % (role, self.name, defined_in) class QAPISchemaEnumMember(QAPISchemaMember):tests/qapi-schema/alternate-any.err +0
-0
tests/qapi-schema/alternate-any.err line 2
Quoted 3 lines
tests/qapi-schema/alternate-any.err +1
-1@@ -1,2 +1,2 @@ alternate-any.json: In alternate 'Alt': -alternate-any.json:2: branch 'one' cannot use built-in type 'any' +alternate-any.json:2: branch 'one' of type 'Alt' cannot use built-in type 'any'
Redundant with the "In alternate" context line. I don't like it, but if that was all I don't like, I'd be willing to accept it to get PATCH 3 over the hump.
Quoted 69 lines
tests/qapi-schema/alternate-conflict-bool-string.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-bool-string.json: In alternate 'Alt': -alternate-conflict-bool-string.json:2: branch 'two' can't be distinguished from 'one' +alternate-conflict-bool-string.json:2: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-conflict-dict.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-dict.json: In alternate 'Alt': -alternate-conflict-dict.json:6: branch 'two' can't be distinguished from 'one' +alternate-conflict-dict.json:6: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-conflict-enum-bool.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-enum-bool.json: In alternate 'Alt': -alternate-conflict-enum-bool.json:4: branch 'two' can't be distinguished from 'one' +alternate-conflict-enum-bool.json:4: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-conflict-enum-int.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-enum-int.json: In alternate 'Alt': -alternate-conflict-enum-int.json:4: branch 'two' can't be distinguished from 'one' +alternate-conflict-enum-int.json:4: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-conflict-lists.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-lists.json: In alternate 'Alt': -alternate-conflict-lists.json:4: branch 'two' can't be distinguished from 'one' +alternate-conflict-lists.json:4: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-conflict-num-string.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-num-string.json: In alternate 'Alt': -alternate-conflict-num-string.json:2: branch 'two' can't be distinguished from 'one' +alternate-conflict-num-string.json:2: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-conflict-string.err +1
-1@@ -1,2 +1,2 @@ alternate-conflict-string.json: In alternate 'Alt': -alternate-conflict-string.json:2: branch 'two' can't be distinguished from 'one' +alternate-conflict-string.json:2: branch 'two' of type 'Alt' can't be distinguished from 'one'tests/qapi-schema/alternate-nested.err +1
-1@@ -1,2 +1,2 @@ alternate-nested.json: In alternate 'Alt2': -alternate-nested.json:4: branch 'nested' cannot use alternate type 'Alt1' +alternate-nested.json:4: branch 'nested' of type 'Alt2' cannot use alternate type 'Alt1'tests/qapi-schema/alternate-unknown.err +0
-0
tests/qapi-schema/alternate-unknown.err line 2
Quoted 3 lines
tests/qapi-schema/alternate-unknown.err +1
-1@@ -1,2 +1,2 @@ alternate-unknown.json: In alternate 'Alt': -alternate-unknown.json:2: branch 'unknown' uses unknown type 'MissingType' +alternate-unknown.json:2: branch 'unknown' of type 'Alt' uses unknown type 'MissingType'
Redundant the same way in all of the test cases so far.
tests/qapi-schema/args-member-unknown.err line 2
Quoted 8 lines
tests/qapi-schema/args-member-unknown.err +1
-1@@ -1,2 +1,2 @@ args-member-unknown.json: In command 'oops': -args-member-unknown.json:2: parameter 'member' uses unknown type 'NoSuchType' +args-member-unknown.json:2: parameter 'member' of type 'oops-arg' uses unknown type 'NoSuchType'
This is args-member-unknown.json:
# we reject data if it does not contain a known type
{ 'command': 'oops', 'data': { 'member': 'NoSuchType' } }There is no type 'oops-arg'. Confusing.
'oops-arg' is in fact the argument type implicitly defined by { 'member': 'NoSuchType' }.
Quoted 37 lines
tests/qapi-schema/enum-clash-member.err +1
-1@@ -1,2 +1,2 @@ enum-clash-member.json: In enum 'MyEnum': -enum-clash-member.json:3: value 'one_two' collides with value 'one-two' +enum-clash-member.json:3: value 'one_two' of type 'MyEnum' collides with value 'one-two' of type 'MyEnum'tests/qapi-schema/features-duplicate-name.err +1
-1@@ -1,2 +1,2 @@ features-duplicate-name.json: In struct 'FeatureStruct0': -features-duplicate-name.json:1: feature 'foo' collides with feature 'foo' +features-duplicate-name.json:1: feature 'foo' of type 'FeatureStruct0' collides with feature 'foo' of type 'FeatureStruct0'tests/qapi-schema/struct-base-clash-deep.err +1
-1@@ -1,2 +1,2 @@ struct-base-clash-deep.json: In struct 'Sub': -struct-base-clash-deep.json:10: member 'name' collides with member 'name' of type 'Base' +struct-base-clash-deep.json:10: member 'name' of type 'Sub' collides with member 'name' of type 'Base'tests/qapi-schema/struct-base-clash.err +1
-1@@ -1,2 +1,2 @@ struct-base-clash.json: In struct 'Sub': -struct-base-clash.json:5: member 'name' collides with member 'name' of type 'Base' +struct-base-clash.json:5: member 'name' of type 'Sub' collides with member 'name' of type 'Base'tests/qapi-schema/struct-member-name-clash.err +0
-0
tests/qapi-schema/struct-member-name-clash.err line 2
Quoted 3 lines
tests/qapi-schema/struct-member-name-clash.err +1
-1@@ -1,2 +1,2 @@ struct-member-name-clash.json: In struct 'Oops': -struct-member-name-clash.json:5: member 'a_b' collides with member 'a-b' +struct-member-name-clash.json:5: member 'a_b' of type 'Oops' collides with member 'a-b' of type 'Oops'
More test cases where it's redundant.
tests/qapi-schema/union-bad-base.err line 2
Quoted 8 lines
tests/qapi-schema/union-bad-base.err +1
-1@@ -1,2 +1,2 @@ union-bad-base.json: In union 'TestUnion': -union-bad-base.json:8: member 'string' of type 'TestTypeA' collides with base member 'string' +union-bad-base.json:8: member 'string' of type 'TestTypeA' collides with base member 'string' of type 'TestUnion-base'
This is union-bad-base.json:
# we allow anonymous base, but enforce no duplicate keys
{ 'enum': 'TestEnum',
'data': [ 'value1', 'value2' ] }
{ 'struct': 'TestTypeA',
'data': { 'string': 'str' } }
{ 'struct': 'TestTypeB',
'data': { 'integer': 'int' } }
{ 'union': 'TestUnion',
'base': { 'enum1': 'TestEnum', 'string': 'str' },
'discriminator': 'enum1',
'data': { 'value1': 'TestTypeA',
'value2': 'TestTypeB' } }There is no type 'TestUnion-base'. Confusing.
'TestUnion-base' is in fact the base type implicitly defined by { 'enum1': 'TestEnum', 'string': 'str' }.
Quoted 13 lines
tests/qapi-schema/union-int-branch.err +1
-1@@ -1,2 +1,2 @@ union-int-branch.json: In union 'TestUnion': -union-int-branch.json:8: branch 'value1' cannot use built-in type 'int' +union-int-branch.json:8: branch 'value1' of type 'TestUnion' cannot use built-in type 'int'tests/qapi-schema/union-unknown.err +0
-0
tests/qapi-schema/union-unknown.err line 2
Quoted 3 lines
tests/qapi-schema/union-unknown.err +1
-1@@ -1,2 +1,2 @@ union-unknown.json: In union 'Union': -union-unknown.json:3: branch 'unknown' uses unknown type 'MissingType' +union-unknown.json:3: branch 'unknown' of type 'Union' uses unknown type 'MissingType'
Redundant again in these two.
No test case demonstrates the commit message's claim "in some cases it will help to reduce ambiguity". PATCH 3 adds one:
union-invalid-union-subfield. It also adds another confusing one:
union-invalid-union-subtype.
Let's have a closer look at QAPISchemaMember.describe() before the patch:
def describe(self, info):
role = self.role
@role is 'member' when self is a QAPISchemaMember, 'value' when it's a QAPISchemaEnumMember, 'feature' when it's a QAPISchemaFeature, and 'branch' when it's a QAPISchemaVariant.
defined_in = self.defined_in
This is the name of the thing where this QAPISchemaMember is defined. When object type T inherits member M from B, it's B's name, not T's.
assert defined_in
if defined_in.startswith('q_obj_'):It's a member of an implicitly defined object type.
Special treatment to avoid exposing the made-up names of implicitly defined object types to the user:
# See QAPISchema._make_implicit_object_type() - reverse the # mapping there to create a nice human-readable description defined_in = defined_in[6:]
if defined_in.endswith('-arg'):
# Implicit type created for a command's dict 'data'
Defined in a command's implicit argument type (you get that when the value of 'data' is { ... }).
assert role == 'member'
role = 'parameter'
@defined_in already got adjusted to the command name. Adjust role.
elif defined_in.endswith('-base'):
# Implicit type created for a union's dict 'base'
Defined in a union's implicit base type (you get that when the value of 'base' is { ... }).
role = 'base ' + role
@defined_in already got adjusted to the union name. Adjust role.
else:
assert False
Explodes when we add another kind of implicitly defined object type and forget to cover it here.
elif defined_in != info.defn_name:
Not a member of an implicit object type, and @defined_in is not in the context line "FILE: In META 'NAME':" printed by QAPISourceInfo.__str__(), typically via. QAPISourceError.__str__() from main()'s exception handler for QAPIError.
Use format "ROLE 'MEMBER-NAME' of type 'DEFINED-IN'":
return "%s '%s' of type '%s'" % (role, self.name, defined_in)
Either a member of an implicit object type, or @defined_in is in the context line.
Description is "ROLE 'MEMBER-NAME':
return "%s '%s'" % (role, self.name)
The patch ditches the second format, exposing @defined_in to the user even when it's the name of an implicitly defined object type.
Before we discuss how to avoid that, let's take a step back and examine the problem we're trying to solve.
PATCH 3 permits union branches to be unions. It adds test case union-invalid-union-subtype.json:
# Clash between common member and union variant's common member
# Base's member 'type' clashes with TestTypeA's
{ 'enum': 'TestEnum',
'data': [ 'value-a', 'value-b' ] }
{ 'enum': 'TestEnumA',
'data': [ 'value-a1', 'value-a2' ] }
{ 'struct': 'TestTypeA1',
'data': { 'integer': 'int' } }
{ 'struct': 'TestTypeA2',
'data': { 'integer': 'int' } }
{ 'union': 'TestTypeA',
'base': { 'type': 'TestEnumA' },
'discriminator': 'type',
'data': { 'value-a1': 'TestTypeA1',
'value-a2': 'TestTypeA2' } }
{ 'struct': 'TestTypeB',
'data': { 'integer': 'int' } }
{ 'union': 'TestUnion',
'base': { 'type': 'TestEnum' },
'discriminator': 'type',
'data': { 'value-a': 'TestTypeA',
'value-b': 'TestTypeB' } }Without this patch, the clash is reported like this:
union-invalid-union-subtype.json: In union 'TestUnion':
union-invalid-union-subtype.json:25: base member 'type' collides with base member 'type'
Crap.
Comes from QAPISchemaMember.check_clash():
raise QAPISemError(
info,
"%s collides with %s"
% (self.describe(info), seen[cname].describe(info)))
@self is member 'type' of TestTypeA's base, and seen[cname] is member 'type' of TestUnion's base.
What we'd like to see instead is something like "base member 'type' of type 'TestTypeA' collides with base member 'type'".
Here's my attempt to get us this message:
def describe(self, info):
role = self.role
meta = 'type'
defined_in = self.defined_in
assert defined_in
if defined_in.startswith('q_obj_'):
# See QAPISchema._make_implicit_object_type() - reverse the # mapping there to create a nice human-readable description defined_in = defined_in[6:]
if defined_in.endswith('-arg'):
# Implicit type created for a command's dict 'data' assert role == 'member'
role = 'parameter'
meta = 'command'
defined_in = defined_in[:-4]
elif defined_in.endswith('-base'):
# Implicit type created for a union's dict 'base' role = 'base ' + role
defined_in = defined_in[:-5]
else:
assert False
if defined_in != info.defn_name:
return "%s '%s' of %s '%s'" % (role, self.name, meta, defined_in)
return "%s '%s'" % (role, self.name)
No change to output of existing tests.
Quoted 42 lines · Daniel P. Berrangé
This extends the QAPI schema validation to permit unions inside unions, provided the checks for clashing fields pass.
Signed-off-by Daniel P. Berrangé <[email protected]>--- scripts/qapi/schema.py | 6 +- tests/qapi-schema/meson.build | 2 + tests/qapi-schema/qapi-schema-test.json | 32 ++++++++++ tests/qapi-schema/qapi-schema-test.out | 29 ++++++++++ .../union-invalid-union-subfield.err | 2 + .../union-invalid-union-subfield.json | 30 ++++++++++ .../union-invalid-union-subfield.out | 0 .../union-invalid-union-subtype.err | 2 + .../union-invalid-union-subtype.json | 29 ++++++++++ .../union-invalid-union-subtype.out | 0 tests/unit/test-qobject-input-visitor.c | 47 +++++++++++++++ tests/unit/test-qobject-output-visitor.c | 58 +++++++++++++++++++ 12 files changed, 234 insertions(+), 3 deletions(-) create mode 100644 tests/qapi-schema/union-invalid-union-subfield.err create mode 100644 tests/qapi-schema/union-invalid-union-subfield.json create mode 100644 tests/qapi-schema/union-invalid-union-subfield.out create mode 100644 tests/qapi-schema/union-invalid-union-subtype.err create mode 100644 tests/qapi-schema/union-invalid-union-subtype.json create mode 100644 tests/qapi-schema/union-invalid-union-subtype.outscripts/qapi/schema.py +2
-1@@ -465,9 +465,10 @@ def check(self, schema): # on behalf of info, which is not necessarily self.info def check_clash(self, info, seen): assert self._checked - assert not self.variants # not implemented for m in self.members: m.check_clash(info, seen) + if self.variants: + self.variants.check_clash(info, seen) def connect_doc(self, doc=None): super().connect_doc(doc)
scripts/qapi/schema.py def check(self, schema, seen): · line 659
Quoted 9 lines
scripts/qapi/schema.py +1
-2@@ -652,8 +653,7 @@ def check(self, schema, seen): self.info, "branch '%s' is not a value of %s" % (v.name, self.tag_member.type.describe())) - if (not isinstance(v.type, QAPISchemaObjectType) - or v.type.variants): + if not isinstance(v.type, QAPISchemaObjectType): raise QAPISemError( self.info, "%s cannot use %s"
I stared at the code some to convince myself this is complete.
Quoted 86 lines
tests/qapi-schema/meson.build +2
-0@@ -194,6 +194,8 @@ schemas = [ 'union-invalid-data.json', 'union-invalid-discriminator.json', 'union-invalid-if-discriminator.json', + 'union-invalid-union-subfield.json', + 'union-invalid-union-subtype.json', 'union-no-base.json', 'union-optional-discriminator.json', 'union-string-discriminator.json',tests/qapi-schema/qapi-schema-test.json +32
-0@@ -114,6 +114,38 @@ { 'struct': 'UserDefC', 'data': { 'string1': 'str', 'string2': 'str' } } +# this tests that unions can contain other unions in their branches +{ 'enum': 'TestUnionEnum', + 'data': [ 'value-a', 'value-b' ] } + +{ 'enum': 'TestUnionEnumA', + 'data': [ 'value-a1', 'value-a2' ] } + +{ 'struct': 'TestUnionTypeA1', + 'data': { 'integer': 'int', + 'name': 'str'} } + +{ 'struct': 'TestUnionTypeA2', + 'data': { 'integer': 'int', + 'size': 'int' } } + +{ 'union': 'TestUnionTypeA', + 'base': { 'type-a': 'TestUnionEnumA' }, + 'discriminator': 'type-a', + 'data': { 'value-a1': 'TestUnionTypeA1', + 'value-a2': 'TestUnionTypeA2' } } + +{ 'struct': 'TestUnionTypeB', + 'data': { 'integer': 'int', + 'onoff': 'bool' } } + +{ 'union': 'TestUnionInUnion', + 'base': { 'type': 'TestUnionEnum' }, + 'discriminator': 'type', + 'data': { 'value-a': 'TestUnionTypeA', + 'value-b': 'TestUnionTypeB' } } + + # for testing use of 'number' within alternates { 'alternate': 'AltEnumBool', 'data': { 'e': 'EnumOne', 'b': 'bool' } } { 'alternate': 'AltEnumNum', 'data': { 'e': 'EnumOne', 'n': 'number' } }tests/qapi-schema/qapi-schema-test.out +22
-0@@ -105,6 +105,35 @@ alternate UserDefAlternate object UserDefC member string1: str optional=False member string2: str optional=False +enum TestUnionEnum + member value-a + member value-b +enum TestUnionEnumA + member value-a1 + member value-a2 +object TestUnionTypeA1 + member integer: int optional=False + member name: str optional=False +object TestUnionTypeA2 + member integer: int optional=False + member size: int optional=False +object q_obj_TestUnionTypeA-base + member type-a: TestUnionEnumA optional=False +object TestUnionTypeA + base q_obj_TestUnionTypeA-base + tag type-a + case value-a1: TestUnionTypeA1 + case value-a2: TestUnionTypeA2 +object TestUnionTypeB + member integer: int optional=False + member onoff: bool optional=False
tests/qapi-schema/qapi-schema-test.out alternate UserDefAlternate · line 139
Quoted 10 lines
tests/qapi-schema/qapi-schema-test.out +7
-0+object q_obj_TestUnionInUnion-base + member type: TestUnionEnum optional=False +object TestUnionInUnion + base q_obj_TestUnionInUnion-base + tag type + case value-a: TestUnionTypeA + case value-b: TestUnionTypeB alternate AltEnumBool tag type case e: EnumOne
Looks good to me. I also inspected the generated code; no complaints.
tests/qapi-schema/union-invalid-union-subfield.err line 2
Quoted 8 lines
tests/qapi-schema/union-invalid-union-subfield.err new file +2
-0@@ -0,0 +1,2 @@ +union-invalid-union-subfield.json: In union 'TestUnion': +union-invalid-union-subfield.json:25: member 'teeth' of type 'TestTypeFish' collides with base member 'teeth' of type 'TestUnion-base'
Bad error message, see my review of PATCH 1.
Quoted 45 lines
tests/qapi-schema/union-invalid-union-subfield.json new file +30
-0@@ -0,0 +1,30 @@ +# Clash between common member and union variant's variant member +# Base's member 'teeth' clashes with TestTypeFish's + +{ 'enum': 'TestEnum', + 'data': [ 'animals', 'plants' ] } + +{ 'enum': 'TestAnimals', + 'data': [ 'fish', 'birds'] } + +{ 'struct': 'TestTypeFish', + 'data': { 'scales': 'int', 'teeth': 'int' } } + +{ 'struct': 'TestTypeBirds', + 'data': { 'feathers': 'int' } } + +{ 'union': 'TestTypeAnimals', + 'base': { 'atype': 'TestAnimals' }, + 'discriminator': 'atype', + 'data': { 'fish': 'TestTypeFish', + 'birds': 'TestTypeBirds' } } + +{ 'struct': 'TestTypePlants', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestUnion', + 'base': { 'type': 'TestEnum', + 'teeth': 'int' }, + 'discriminator': 'type', + 'data': { 'animals': 'TestTypeAnimals', + 'plants': 'TestTypePlants' } }tests/qapi-schema/union-invalid-union-subfield.out new file +0
-0tests/qapi-schema/union-invalid-union-subtype.err new file +0
-0
tests/qapi-schema/union-invalid-union-subtype.err line 2
Quoted 2 lines
tests/qapi-schema/union-invalid-union-subtype.err new file +2
-0@@ -0,0 +1,2 @@ +union-invalid-union-subtype.json: In union 'TestUnion': +union-invalid-union-subtype.json:25: base member 'type' of type 'TestTypeA-base' collides with base member 'type' of type 'TestUnion-base'
Likewise.
Quoted 171 lines
tests/qapi-schema/union-invalid-union-subtype.json new file +29
-0@@ -0,0 +1,29 @@ +# Clash between common member and union variant's common member +# Base's member 'type' clashes with TestTypeA's + +{ 'enum': 'TestEnum', + 'data': [ 'value-a', 'value-b' ] } + +{ 'enum': 'TestEnumA', + 'data': [ 'value-a1', 'value-a2' ] } + +{ 'struct': 'TestTypeA1', + 'data': { 'integer': 'int' } } + +{ 'struct': 'TestTypeA2', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestTypeA', + 'base': { 'type': 'TestEnumA' }, + 'discriminator': 'type', + 'data': { 'value-a1': 'TestTypeA1', + 'value-a2': 'TestTypeA2' } } + +{ 'struct': 'TestTypeB', + 'data': { 'integer': 'int' } } + +{ 'union': 'TestUnion', + 'base': { 'type': 'TestEnum' }, + 'discriminator': 'type', + 'data': { 'value-a': 'TestTypeA', + 'value-b': 'TestTypeB' } }tests/qapi-schema/union-invalid-union-subtype.out new file +0
-0tests/unit/test-qobject-input-visitor.c +47
-0@@ -706,6 +706,51 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,g_assert (&base->enum1 == &tmp->enum1); } +static void test_visitor_in_union_in_union (TestInputVisitorData *data, +const void *unused) +{ +Visitor *v; +g_autoptr (TestUnionInUnion) tmp =NULL ; + + v =visitor_input_test_init (data, +"{ 'type': 'value-a', " +" 'type-a': 'value-a1', " +" 'integer': 2, " +" 'name': 'fish' }" ); + +visit_type_TestUnionInUnion (v,NULL , &tmp, &error_abort); + g_assert_cmpint(tmp->type, ==, TEST_UNION_ENUM_VALUE_A); + g_assert_cmpint(tmp->u.value_a.type_a, ==, TEST_UNION_ENUMA_VALUE_A1); + g_assert_cmpint(tmp->u.value_a.u.value_a1.integer, ==, 2); + g_assert_cmpint(strcmp(tmp->u.value_a.u.value_a1.name, "fish"), ==, 0); + +qapi_free_TestUnionInUnion (tmp); + + v =visitor_input_test_init (data, +"{ 'type': 'value-a', " +" 'type-a': 'value-a2', " +" 'integer': 1729, " +" 'size': 87539319 }" ); + +visit_type_TestUnionInUnion (v,NULL , &tmp, &error_abort); + g_assert_cmpint(tmp->type, ==, TEST_UNION_ENUM_VALUE_A); + g_assert_cmpint(tmp->u.value_a.type_a, ==, TEST_UNION_ENUMA_VALUE_A2); + g_assert_cmpint(tmp->u.value_a.u.value_a2.integer, ==, 1729); + g_assert_cmpint(tmp->u.value_a.u.value_a2.size, ==, 87539319); + +qapi_free_TestUnionInUnion (tmp); + + v =visitor_input_test_init (data, +"{ 'type': 'value-b', " +" 'integer': 1729, " +" 'onoff': true }" ); + +visit_type_TestUnionInUnion (v,NULL , &tmp, &error_abort); + g_assert_cmpint(tmp->type, ==, TEST_UNION_ENUM_VALUE_B); + g_assert_cmpint(tmp->u.value_b.integer, ==, 1729); + g_assert_cmpint(tmp->u.value_b.onoff, ==, true); +} +static void test_visitor_in_alternate (TestInputVisitorData *data,const void *unused) {@@ -1216,6 +1261,8 @@ int main(int argc, char **argv) NULL, test_visitor_in_null);input_visitor_test_add ("/visitor/input/union-flat" ,NULL , test_visitor_in_union_flat); +input_visitor_test_add ("/visitor/input/union-in-union" , +NULL , test_visitor_in_union_in_union);input_visitor_test_add ("/visitor/input/alternate" ,NULL , test_visitor_in_alternate); input_visitor_test_add("/visitor/input/errors",tests/unit/test-qobject-output-visitor.c +56
-0@@ -352,6 +352,62 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,qapi_free_UserDefFlatUnion (tmp); } +static void test_visitor_out_union_in_union (TestOutputVisitorData *data, +const void *unused) +{ +QDict *qdict; + +TestUnionInUnion *tmp =g_new0 (TestUnionInUnion,1 ); + tmp->type =TEST_UNION_ENUM_VALUE_A ; + tmp->u.value_a.type_a =TEST_UNION_ENUMA_VALUE_A1 ; + tmp->u.value_a.u.value_a1.integer =42 ; + tmp->u.value_a.u.value_a1.name =g_strdup ("fish" ); + +visit_type_TestUnionInUnion (data->ov,NULL , &tmp, &error_abort); + qdict =qobject_to (QDict,visitor_get (data)); +g_assert (qdict); + g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "value-a"); + g_assert_cmpstr(qdict_get_str(qdict, "type-a"), ==, "value-a1"); + g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 42); + g_assert_cmpstr(qdict_get_str(qdict, "name"), ==, "fish"); + +qapi_free_TestUnionInUnion (tmp); + + +visitor_reset (data); + tmp =g_new0 (TestUnionInUnion,1 ); + tmp->type =TEST_UNION_ENUM_VALUE_A ; + tmp->u.value_a.type_a =TEST_UNION_ENUMA_VALUE_A2 ; + tmp->u.value_a.u.value_a2.integer =1729 ; + tmp->u.value_a.u.value_a2.size =87539319 ; + +visit_type_TestUnionInUnion (data->ov,NULL , &tmp, &error_abort); + qdict =qobject_to (QDict,visitor_get (data)); +g_assert (qdict); + g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "value-a"); + g_assert_cmpstr(qdict_get_str(qdict, "type-a"), ==, "value-a2"); + g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 1729); + g_assert_cmpint(qdict_get_int(qdict, "size"), ==, 87539319); + +qapi_free_TestUnionInUnion (tmp); + + +visitor_reset (data); + tmp =g_new0 (TestUnionInUnion,1 ); + tmp->type =TEST_UNION_ENUM_VALUE_B ; + tmp->u.value_b.integer =1729 ; + tmp->u.value_b.onoff = true; + +visit_type_TestUnionInUnion (data->ov,NULL , &tmp, &error_abort); + qdict =qobject_to (QDict,visitor_get (data)); +g_assert (qdict); + g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "value-b"); + g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 1729); + g_assert_cmpint(qdict_get_bool(qdict, "onoff"), ==, true); + +qapi_free_TestUnionInUnion (tmp); +} +static void test_visitor_out_alternate (TestOutputVisitorData *data,const void *unused) {
tests/unit/test-qobject-output-visitor.c int main(int argc, char **argv) · line 649
Quoted 8 lines
tests/unit/test-qobject-output-visitor.c +2
-0@@ -586,6 +642,8 @@ int main(int argc, char **argv) &out_visitor_data, test_visitor_out_list_qapi_free);output_visitor_test_add ("/visitor/output/union-flat" , &out_visitor_data, test_visitor_out_union_flat); +output_visitor_test_add ("/visitor/output/union-in-union" , + &out_visitor_data, test_visitor_out_union_in_union);output_visitor_test_add ("/visitor/output/alternate" , &out_visitor_data, test_visitor_out_alternate); output_visitor_test_add("/visitor/output/null",
Quoted 4 lines · Daniel P. Berrangé
Currently it is not possible for a union type to contain a further union as one (or more) of its branches. This relaxes that restriction and adds the calls needed to validate field name uniqueness as unions are flattened.
I apologize for the long delay. Sick child, sick me, much snot, little sleep.
PATCH 1 is wrong, but I was able to figure out what's going on there, and suggested a patch that hopefully works.
PATCH 2 is okay. I suggested a few tweaks. I'd put it first, but that's up to you.
PATCH 3 looks good.
Looking forward to v3.
Hi all,
Quoted 12 lines · Markus Armbruster
Quoted 4 lines · Daniel P. Berrangé
Currently it is not possible for a union type to contain a further union as one (or more) of its branches. This relaxes that restriction and adds the calls needed to validate field name uniqueness as unions are flattened.
I apologize for the long delay. Sick child, sick me, much snot, little sleep.
PATCH 1 is wrong, but I was able to figure out what's going on there, and suggested a patch that hopefully works.
Quoted 6 lines
PATCH 2 is okay. I suggested a few tweaks. I'd put it first, but that's up to you.
PATCH 3 looks good.
Looking forward to v3.
Thankyou Markus for your suggestions and I hope everyone is in good health now. This is just a friendly reminder if Daniel is ready with v3 patches for the same :)
Regards,
Het Gala
Quoted 18 lines · Het Gala
Hi all,
Quoted 15 lines · Markus Armbruster
Quoted 4 lines · Daniel P. Berrangé
Currently it is not possible for a union type to contain a further union as one (or more) of its branches. This relaxes that restriction and adds the calls needed to validate field name uniqueness as unions are flattened.
I apologize for the long delay. Sick child, sick me, much snot, little sleep.
PATCH 1 is wrong, but I was able to figure out what's going on there, and suggested a patch that hopefully works.
PATCH 2 is okay. I suggested a few tweaks. I'd put it first, but that's up to you.
Quoted 8 lines
Quoted 3 lines
PATCH 3 looks good.
Looking forward to v3.
Thankyou Markus for your suggestions and I hope everyone is in good health now. This is just a friendly reminder if Daniel is ready with v3 patches for the same :)
Regards,
Het Gala
Hi, this is just a reminder mail to check if Daniel has plan to post v3 patches in the coming days. Would like these patches to get merged in qemu as soon as possible, so that we all can focus on restructuring of 'migrate' QAPI :)
Regards,
Het Gala
Quoted 1 line · Het Gala
Hi, this is just a reminder mail to check if Daniel has plan to post v3 patches in the coming days. Would like these patches to get merged in qemu as soon as possible, so that we all can focus on restructuring of 'migrate' QAPI :)
I just queued v3. Expect a pull request today or tomorrow.