qapi: Frontend fixes and cleanups
cover qapi: Frontend fixes and cleanups
Here's the next batch of qapi patches, based on my "[PATCH v3 00/16]
qapi: Schema language cleanups & doc improvements". There's more in the pipeline.
Based-on: <[email protected]>
Markus Armbruster (19):
tests/qapi-schema: Cover unknown pragma tests/qapi-schema: Delete two redundant tests tests/qapi-schema: Demonstrate misleading optional tag error tests/qapi-schema: Demonstrate broken discriminator errors tests/qapi-schema: Demonstrate insufficient 'if' checking tests/qapi-schema: Demonstrate suboptimal lexical errors
qapi: Use quotes more consistently in frontend error messages
qapi: Improve reporting of lexical errors
qapi: Remove null from schema language
qapi: Fix broken discriminator error messages
qapi: Reject blank 'if' conditions in addition to empty ones
qapi: Fix missing 'if' checks in struct, union, alternate 'data'
qapi: Normalize 'if' in check_exprs(), like other sugar
qapi: Simplify check_keys()
qapi: Clean up around check_known_keys()
qapi: Delete useless check_exprs() code for simple union kind
qapi: Fix to .check() empty structs just once
qapi: Fix excessive QAPISchemaEntity.check() recursion
qapi: Assert .visit() and .check_clash() run only after .check()
docs/devel/qapi-code-gen.txt | 4 +-
scripts/qapi/common.py | 233 +++++++++---------
tests/Makefile.include | 9 +-
.../alternate-branch-if-invalid.err | 1 +
....exit => alternate-branch-if-invalid.exit} | 0
.../alternate-branch-if-invalid.json | 3 +
...ta.out => alternate-branch-if-invalid.out} | 0
tests/qapi-schema/bad-if-list.err | 2 +-
tests/qapi-schema/bad-if-list.json | 2 +-
tests/qapi-schema/bad-type-int.err | 2 +-
tests/qapi-schema/bad-type-int.json | 2 +-
tests/qapi-schema/doc-missing-colon.err | 2 +-
tests/qapi-schema/double-data.err | 1 -
tests/qapi-schema/double-data.json | 2 -
tests/qapi-schema/duplicate-key.err | 2 +-
tests/qapi-schema/duplicate-key.json | 2 +-
tests/qapi-schema/enum-int-member.err | 2 +-
tests/qapi-schema/escape-outside-string.err | 1 +
tests/qapi-schema/features-if-invalid.err | 1 +
...rmat-err.exit => features-if-invalid.exit} | 0
tests/qapi-schema/features-if-invalid.json | 4 +
...format-err.out => features-if-invalid.out} | 0
.../flat-union-discriminator-bad-name.err | 1 +
.../flat-union-discriminator-bad-name.exit | 1 +
.../flat-union-discriminator-bad-name.json | 11 +
.../flat-union-discriminator-bad-name.out | 0
.../flat-union-invalid-discriminator.err | 2 +-
.../flat-union-invalid-discriminator.json | 5 +-
.../flat-union-invalid-if-discriminator.err | 2 +-
.../flat-union-invalid-if-discriminator.json | 5 +-
.../flat-union-optional-discriminator.err | 2 +-
.../flat-union-optional-discriminator.json | 3 +-
tests/qapi-schema/funny-char.err | 2 +-
tests/qapi-schema/funny-word.err | 1 +
tests/qapi-schema/funny-word.exit | 1 +
tests/qapi-schema/funny-word.json | 1 +
tests/qapi-schema/funny-word.out | 0
tests/qapi-schema/include-before-err.err | 2 +-
tests/qapi-schema/include-format-err.err | 1 -
tests/qapi-schema/include-format-err.json | 2 -
tests/qapi-schema/include-nested-err.err | 2 +-
tests/qapi-schema/leading-comma-list.err | 2 +-
tests/qapi-schema/leading-comma-object.err | 2 +-
tests/qapi-schema/missing-colon.err | 2 +-
tests/qapi-schema/missing-comma-list.err | 2 +-
tests/qapi-schema/missing-comma-object.err | 2 +-
tests/qapi-schema/non-objects.err | 2 +-
.../pragma-name-case-whitelist-crap.json | 2 +-
tests/qapi-schema/pragma-unknown.err | 1 +
tests/qapi-schema/pragma-unknown.exit | 1 +
tests/qapi-schema/pragma-unknown.json | 1 +
tests/qapi-schema/pragma-unknown.out | 0
tests/qapi-schema/quoted-structural-chars.err | 2 +-
.../qapi-schema/struct-member-if-invalid.err | 1 +
.../qapi-schema/struct-member-if-invalid.exit | 1 +
.../qapi-schema/struct-member-if-invalid.json | 3 +
.../qapi-schema/struct-member-if-invalid.out | 0
tests/qapi-schema/trailing-comma-list.err | 2 +-
tests/qapi-schema/unclosed-list.err | 2 +-
tests/qapi-schema/unclosed-object.err | 2 +-
tests/qapi-schema/union-base-empty.err | 2 +-
tests/qapi-schema/union-branch-if-invalid.err | 1 +
.../qapi-schema/union-branch-if-invalid.exit | 1 +
.../qapi-schema/union-branch-if-invalid.json | 6 +
tests/qapi-schema/union-branch-if-invalid.out | 0
65 files changed, 202 insertions(+), 157 deletions(-)
create mode 100644 tests/qapi-schema/alternate-branch-if-invalid.err
rename tests/qapi-schema/{double-data.exit => alternate-branch-if-invalid.exit} (100%)
create mode 100644 tests/qapi-schema/alternate-branch-if-invalid.json
rename tests/qapi-schema/{double-data.out => alternate-branch-if-invalid.out} (100%)
delete mode 100644 tests/qapi-schema/double-data.err
delete mode 100644 tests/qapi-schema/double-data.json
create mode 100644 tests/qapi-schema/escape-outside-string.err
create mode 100644 tests/qapi-schema/features-if-invalid.err
rename tests/qapi-schema/{include-format-err.exit => features-if-invalid.exit} (100%)
create mode 100644 tests/qapi-schema/features-if-invalid.json
rename tests/qapi-schema/{include-format-err.out => features-if-invalid.out} (100%)
create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.err
create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.exit
create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.json
create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.out
create mode 100644 tests/qapi-schema/funny-word.err
create mode 100644 tests/qapi-schema/funny-word.exit
create mode 100644 tests/qapi-schema/funny-word.json
create mode 100644 tests/qapi-schema/funny-word.out
delete mode 100644 tests/qapi-schema/include-format-err.err
delete mode 100644 tests/qapi-schema/include-format-err.json
create mode 100644 tests/qapi-schema/pragma-unknown.err
create mode 100644 tests/qapi-schema/pragma-unknown.exit
create mode 100644 tests/qapi-schema/pragma-unknown.json
create mode 100644 tests/qapi-schema/pragma-unknown.out
create mode 100644 tests/qapi-schema/struct-member-if-invalid.err
create mode 100644 tests/qapi-schema/struct-member-if-invalid.exit
create mode 100644 tests/qapi-schema/struct-member-if-invalid.json
create mode 100644 tests/qapi-schema/struct-member-if-invalid.out
create mode 100644 tests/qapi-schema/union-branch-if-invalid.err
create mode 100644 tests/qapi-schema/union-branch-if-invalid.exit
create mode 100644 tests/qapi-schema/union-branch-if-invalid.json
create mode 100644 tests/qapi-schema/union-branch-if-invalid.outSignature · 1 line
2.21.0
PATCH 1/19 tests/qapi-schema: Cover unknown pragma
--- tests/Makefile.include | 1 + tests/qapi-schema/pragma-unknown.err | 1 + tests/qapi-schema/pragma-unknown.exit | 1 + tests/qapi-schema/pragma-unknown.json | 1 + tests/qapi-schema/pragma-unknown.out | 0 5 files changed, 4 insertions(+) create mode 100644 tests/qapi-schema/pragma-unknown.err create mode 100644 tests/qapi-schema/pragma-unknown.exit create mode 100644 tests/qapi-schema/pragma-unknown.json create mode 100644 tests/qapi-schema/pragma-unknown.out
tests/Makefile.include +1 -0
@@ -428,6 +428,7 @@ qapi-schema += pragma-doc-required-crap.json qapi-schema += pragma-extra-junk.json qapi-schema += pragma-name-case-whitelist-crap.json qapi-schema += pragma-non-dict.json +qapi-schema += pragma-unknown.json qapi-schema += pragma-returns-whitelist-crap.json qapi-schema += qapi-schema-test.json qapi-schema += quoted-structural-chars.json
tests/qapi-schema/pragma-unknown.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/pragma-unknown.json:1: Unknown pragma 'no-such-pragma'
tests/qapi-schema/pragma-unknown.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/pragma-unknown.json new file +1 -0
@@ -0,0 +1 @@ +{ 'pragma': { 'no-such-pragma': false } }
tests/qapi-schema/pragma-unknown.out new file +0 -0
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 01/19] tests/qapi-schema: Cover unknown pragma
Quoted 2 lines · Markus Armbruster
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 2/19 tests/qapi-schema: Delete two redundant tests
In reply to Markus ArmbrusterTests duplicate-key and double-data test the same thing. The former predates the latter, and it has a better name. Delete the latter, and tweak the former's comment.
Tests include-format-err and include-extra-junk test the same thing. The former predates the latter, but the latter has a better name and a comment. Delete the former.
--- tests/Makefile.include | 2 -- tests/qapi-schema/double-data.err | 1 - tests/qapi-schema/double-data.exit | 1 - tests/qapi-schema/double-data.json | 2 -- tests/qapi-schema/double-data.out | 0 tests/qapi-schema/duplicate-key.json | 2 +- tests/qapi-schema/include-format-err.err | 1 - tests/qapi-schema/include-format-err.exit | 1 - tests/qapi-schema/include-format-err.json | 2 -- tests/qapi-schema/include-format-err.out | 0 10 files changed, 1 insertion(+), 11 deletions(-) delete mode 100644 tests/qapi-schema/double-data.err delete mode 100644 tests/qapi-schema/double-data.exit delete mode 100644 tests/qapi-schema/double-data.json delete mode 100644 tests/qapi-schema/double-data.out delete mode 100644 tests/qapi-schema/include-format-err.err delete mode 100644 tests/qapi-schema/include-format-err.exit delete mode 100644 tests/qapi-schema/include-format-err.json delete mode 100644 tests/qapi-schema/include-format-err.out
tests/Makefile.include +0 -2
@@ -360,7 +360,6 @@ qapi-schema += doc-missing-expr.json qapi-schema += doc-missing-space.json qapi-schema += doc-missing.json qapi-schema += doc-no-symbol.json -qapi-schema += double-data.json qapi-schema += double-type.json qapi-schema += duplicate-key.json qapi-schema += empty.json
@@ -405,7 +404,6 @@ qapi-schema += ident-with-escape.json qapi-schema += include-before-err.json qapi-schema += include-cycle.json qapi-schema += include-extra-junk.json -qapi-schema += include-format-err.json qapi-schema += include-nested-err.json qapi-schema += include-no-file.json qapi-schema += include-non-file.json
tests/qapi-schema/double-data.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/double-data.json:2:41: Duplicate key "data"
tests/qapi-schema/double-data.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/double-data.json deleted +0 -2
@@ -1,2 +0,0 @@ -# we reject an expression with duplicate top-level keys -{ 'struct': 'bar', 'data': { }, 'data': { 'string': 'str'} }
tests/qapi-schema/double-data.out deleted +0 -0
tests/qapi-schema/duplicate-key.json +1 -1
@@ -1,3 +1,3 @@ -# QAPI cannot include the same key more than once in any {} +# Cannot include the same key more than once in any {} { 'key': 'value', 'key': 'value' }
tests/qapi-schema/include-format-err.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/include-format-err.json:1: Invalid 'include' directive
tests/qapi-schema/include-format-err.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/include-format-err.json deleted +0 -2
@@ -1,2 +0,0 @@ -{ 'include': 'include-simple-sub.json', - 'foo': 'bar' }
tests/qapi-schema/include-format-err.out deleted +0 -0
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 02/19] tests/qapi-schema: Delete two redundant tests
Quoted 10 lines · Markus Armbruster
Tests duplicate-key and double-data test the same thing. The former predates the latter, and it has a better name. Delete the latter, and tweak the former's comment.
Tests include-format-err and include-extra-junk test the same thing. The former predates the latter, but the latter has a better name and a comment. Delete the former.
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 3/19 tests/qapi-schema: Demonstrate misleading optional tag error
In reply to Markus ArmbrusterTest flat-union-optional-discriminator declares its union tag as '*switch': 'Enum', and points to it with 'discriminator': '*switch'. This gets rejected as "discriminator of flat union 'MyUnion' uses invalid name '*switch'". Correct; member 'discriminator' doesn't accept a '*' prefix.
However, this merely tests name validity checking, which we already cover elsewhere. More interesting is testing the valid name 'switch'. This reports "discriminator 'switch' is not a member of base struct 'Base'", which is misleading.
Copy the existing 'discriminator': '*switch' test to flat-union-discriminator-bad-name, and rewrite its comment. Change flat-union-optional-discriminator to test 'discriminator': 'switch', and mark it FIXME.
--- tests/Makefile.include | 1 + .../qapi-schema/flat-union-discriminator-bad-name.err | 1 + .../flat-union-discriminator-bad-name.exit | 1 + .../flat-union-discriminator-bad-name.json | 11 +++++++++++ .../qapi-schema/flat-union-discriminator-bad-name.out | 0 .../qapi-schema/flat-union-optional-discriminator.err | 2 +- .../flat-union-optional-discriminator.json | 3 ++- 7 files changed, 17 insertions(+), 2 deletions(-) create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.err create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.exit create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.json create mode 100644 tests/qapi-schema/flat-union-discriminator-bad-name.out
tests/Makefile.include +1 -0
@@ -389,6 +389,7 @@ qapi-schema += flat-union-bad-discriminator.json qapi-schema += flat-union-base-any.json qapi-schema += flat-union-base-union.json qapi-schema += flat-union-clash-member.json +qapi-schema += flat-union-discriminator-bad-name.json qapi-schema += flat-union-empty.json qapi-schema += flat-union-inline.json qapi-schema += flat-union-inline-invalid-dict.json
tests/qapi-schema/flat-union-discriminator-bad-name.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/flat-union-discriminator-bad-name.json:7: Discriminator of flat union 'MyUnion' does not allow optional name '*switch'
tests/qapi-schema/flat-union-discriminator-bad-name.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/flat-union-discriminator-bad-name.json new file +11 -0
@@ -0,0 +1,11 @@ +# discriminator '*switch' isn't a member of base, 'switch' is +# reports "does not allow optional name", which is good enough +{ 'enum': 'Enum', 'data': [ 'one', 'two' ] } +{ 'struct': 'Base', + 'data': { '*switch': 'Enum' } } +{ 'struct': 'Branch', 'data': { 'name': 'str' } } +{ 'union': 'MyUnion', + 'base': 'Base', + 'discriminator': '*switch', + 'data': { 'one': 'Branch', + 'two': 'Branch' } }
tests/qapi-schema/flat-union-discriminator-bad-name.out new file +0 -0
tests/qapi-schema/flat-union-optional-discriminator.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/flat-union-optional-discriminator.json:6: Discriminator of flat union 'MyUnion' does not allow optional name '*switch' +tests/qapi-schema/flat-union-optional-discriminator.json:7: Discriminator 'switch' is not a member of base struct 'Base'
tests/qapi-schema/flat-union-optional-discriminator.json +2 -1
@@ -1,10 +1,11 @@ # we require the discriminator to be non-optional +# FIXME reports "discriminator 'switch' is not a member of base struct 'Base'" { 'enum': 'Enum', 'data': [ 'one', 'two' ] } { 'struct': 'Base', 'data': { '*switch': 'Enum' } } { 'struct': 'Branch', 'data': { 'name': 'str' } } { 'union': 'MyUnion', 'base': 'Base', - 'discriminator': '*switch', + 'discriminator': 'switch', 'data': { 'one': 'Branch', 'two': 'Branch' } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 03/19] tests/qapi-schema: Demonstrate misleading optional tag error
Quoted 11 lines · Markus Armbruster
Test flat-union-optional-discriminator declares its union tag as '*switch': 'Enum', and points to it with 'discriminator': '*switch'. This gets rejected as "discriminator of flat union 'MyUnion' uses invalid name '*switch'". Correct; member 'discriminator' doesn't accept a '*' prefix.
However, this merely tests name validity checking, which we already cover elsewhere. More interesting is testing the valid name 'switch'. This reports "discriminator 'switch' is not a member of base struct 'Base'", which is misleading.
Quoted 7 lines
Copy the existing 'discriminator': '*switch' test to flat-union-discriminator-bad-name, and rewrite its comment. Change flat-union-optional-discriminator to test 'discriminator': 'switch', and mark it FIXME.
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 4/19 tests/qapi-schema: Demonstrate broken discriminator errors
In reply to Markus ArmbrusterWhen the union definition's base is an object, some error messages show it as an OrderedDict. Oops. Mark FIXME.
--- tests/qapi-schema/flat-union-invalid-discriminator.err | 2 +- tests/qapi-schema/flat-union-invalid-discriminator.json | 6 ++---- tests/qapi-schema/flat-union-invalid-if-discriminator.err | 2 +- tests/qapi-schema/flat-union-invalid-if-discriminator.json | 6 ++---- 4 files changed, 6 insertions(+), 10 deletions(-)
tests/qapi-schema/flat-union-invalid-discriminator.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/flat-union-invalid-discriminator.json:13: Discriminator 'enum_wrong' is not a member of base struct 'TestBase' +tests/qapi-schema/flat-union-invalid-discriminator.json:11: Discriminator 'enum_wrong' is not a member of base struct 'OrderedDict([('enum1', {'type': 'TestEnum'})])'
tests/qapi-schema/flat-union-invalid-discriminator.json +2 -4
@@ -1,9 +1,7 @@ +# FIXME error message shows base as OrderedDict { 'enum': 'TestEnum', 'data': [ 'value1', 'value2' ] } -{ 'struct': 'TestBase', - 'data': { 'enum1': 'TestEnum' } } - { 'struct': 'TestTypeA', 'data': { 'string': 'str' } }
@@ -11,7 +9,7 @@ 'data': { 'integer': 'int' } } { 'union': 'TestUnion', - 'base': 'TestBase', + 'base': { 'enum1': 'TestEnum' }, 'discriminator': 'enum_wrong', 'data': { 'value1': 'TestTypeA', 'value2': 'TestTypeB' } }
tests/qapi-schema/flat-union-invalid-if-discriminator.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/flat-union-invalid-if-discriminator.json:13: The discriminator TestBase.enum1 for union TestUnion must not be conditional +tests/qapi-schema/flat-union-invalid-if-discriminator.json:11: The discriminator OrderedDict([('enum1', OrderedDict([('type', 'TestEnum'), ('if', 'FOO')]))]).enum1 for union TestUnion must not be conditional
tests/qapi-schema/flat-union-invalid-if-discriminator.json +2 -4
@@ -1,9 +1,7 @@ +# FIXME error message shows base as OrderedDict { 'enum': 'TestEnum', 'data': [ 'value1', 'value2' ] } -{ 'struct': 'TestBase', - 'data': { 'enum1': { 'type': 'TestEnum', 'if': 'FOO' } } } - { 'struct': 'TestTypeA', 'data': { 'string': 'str' } }
@@ -11,7 +9,7 @@ 'data': { 'integer': 'int' } } { 'union': 'TestUnion', - 'base': 'TestBase', + 'base': { 'enum1': { 'type': 'TestEnum', 'if': 'FOO' } }, 'discriminator': 'enum1', 'data': { 'value1': 'TestTypeA', 'value2': 'TestTypeB' } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 04/19] tests/qapi-schema: Demonstrate broken discriminator errors
Quoted 10 lines · Markus Armbruster
When the union definition's base is an object, some error messages show it as an OrderedDict. Oops. Mark FIXME.
Signed-off-by Markus Armbruster <[email protected]>--- tests/qapi-schema/flat-union-invalid-discriminator.err | 2 +- tests/qapi-schema/flat-union-invalid-discriminator.json | 6 ++---- tests/qapi-schema/flat-union-invalid-if-discriminator.err | 2 +- tests/qapi-schema/flat-union-invalid-if-discriminator.json | 6 ++---- 4 files changed, 6 insertions(+), 10 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 5/19 tests/qapi-schema: Demonstrate insufficient 'if' checking
In reply to Markus ArmbrusterCover invalid 'if' in struct members, features, union and alternate branches. Four out of four are broken. Mark FIXME.
--- tests/Makefile.include | 4 ++++ .../alternate-branch-if-invalid.err | 0 .../alternate-branch-if-invalid.exit | 1 + .../alternate-branch-if-invalid.json | 4 ++++ .../alternate-branch-if-invalid.out | 16 +++++++++++++ tests/qapi-schema/features-if-invalid.err | 0 tests/qapi-schema/features-if-invalid.exit | 1 + tests/qapi-schema/features-if-invalid.json | 5 ++++ tests/qapi-schema/features-if-invalid.out | 14 +++++++++++ .../qapi-schema/struct-member-if-invalid.err | 0 .../qapi-schema/struct-member-if-invalid.exit | 1 + .../qapi-schema/struct-member-if-invalid.json | 4 ++++ .../qapi-schema/struct-member-if-invalid.out | 15 ++++++++++++ tests/qapi-schema/union-branch-if-invalid.err | 0 .../qapi-schema/union-branch-if-invalid.exit | 1 + .../qapi-schema/union-branch-if-invalid.json | 7 ++++++ tests/qapi-schema/union-branch-if-invalid.out | 23 +++++++++++++++++++ 17 files changed, 96 insertions(+) create mode 100644 tests/qapi-schema/alternate-branch-if-invalid.err create mode 100644 tests/qapi-schema/alternate-branch-if-invalid.exit create mode 100644 tests/qapi-schema/alternate-branch-if-invalid.json create mode 100644 tests/qapi-schema/alternate-branch-if-invalid.out create mode 100644 tests/qapi-schema/features-if-invalid.err create mode 100644 tests/qapi-schema/features-if-invalid.exit create mode 100644 tests/qapi-schema/features-if-invalid.json create mode 100644 tests/qapi-schema/features-if-invalid.out create mode 100644 tests/qapi-schema/struct-member-if-invalid.err create mode 100644 tests/qapi-schema/struct-member-if-invalid.exit create mode 100644 tests/qapi-schema/struct-member-if-invalid.json create mode 100644 tests/qapi-schema/struct-member-if-invalid.out create mode 100644 tests/qapi-schema/union-branch-if-invalid.err create mode 100644 tests/qapi-schema/union-branch-if-invalid.exit create mode 100644 tests/qapi-schema/union-branch-if-invalid.json create mode 100644 tests/qapi-schema/union-branch-if-invalid.out
tests/Makefile.include +4 -0
@@ -295,6 +295,7 @@ check-qtest-generic-y += tests/test-hmp$(EXESUF) qapi-schema += alternate-any.json qapi-schema += alternate-array.json qapi-schema += alternate-base.json +qapi-schema += alternate-branch-if-invalid.json qapi-schema += alternate-clash.json qapi-schema += alternate-conflict-dict.json qapi-schema += alternate-conflict-enum-bool.json
@@ -379,6 +380,7 @@ qapi-schema += event-member-invalid-dict.json qapi-schema += event-nest-struct.json qapi-schema += features-bad-type.json qapi-schema += features-duplicate-name.json +qapi-schema += features-if-invalid.json qapi-schema += features-missing-name.json qapi-schema += features-name-bad-type.json qapi-schema += features-no-list.json
@@ -453,6 +455,7 @@ qapi-schema += string-code-point-127.json qapi-schema += struct-base-clash-deep.json qapi-schema += struct-base-clash.json qapi-schema += struct-data-invalid.json +qapi-schema += struct-member-if-invalid.json qapi-schema += struct-member-invalid-dict.json qapi-schema += struct-member-invalid.json qapi-schema += trailing-comma-list.json
@@ -464,6 +467,7 @@ qapi-schema += unclosed-string.json qapi-schema += union-base-empty.json qapi-schema += union-base-no-discriminator.json qapi-schema += union-branch-case.json +qapi-schema += union-branch-if-invalid.json qapi-schema += union-branch-invalid-dict.json qapi-schema += union-clash-branches.json qapi-schema += union-empty.json
tests/qapi-schema/alternate-branch-if-invalid.err new file +0 -0
tests/qapi-schema/alternate-branch-if-invalid.exit new file +1 -0
@@ -0,0 +1 @@ +0
tests/qapi-schema/alternate-branch-if-invalid.json new file +4 -0
@@ -0,0 +1,4 @@ +# Cover alternative with invalid 'if' +# FIXME not rejected, would generate '#if \n' +{ 'alternate': 'Alt', + 'data': { 'branch': { 'type': 'int', 'if': ' ' } } }
tests/qapi-schema/alternate-branch-if-invalid.out new file +16 -0
@@ -0,0 +1,16 @@ +module None +object q_empty +enum QType + prefix QTYPE + member none + member qnull + member qnum + member qstring + member qdict + member qlist + member qbool +module alternate-branch-if-invalid.json +alternate Alt + tag type + case branch: int + if [' ']
tests/qapi-schema/features-if-invalid.err new file +0 -0
tests/qapi-schema/features-if-invalid.exit new file +1 -0
@@ -0,0 +1 @@ +0
tests/qapi-schema/features-if-invalid.json new file +5 -0
@@ -0,0 +1,5 @@ +# Cover feature with invalid 'if' +# FIXME not rejected, misinterpreded as unconditional +{ 'struct': 'Stru', + 'data': {}, + 'features': [{'name': 'f', 'if': null }] }
tests/qapi-schema/features-if-invalid.out new file +14 -0
@@ -0,0 +1,14 @@ +module None +object q_empty +enum QType + prefix QTYPE + member none + member qnull + member qnum + member qstring + member qdict + member qlist + member qbool +module features-if-invalid.json +object Stru + feature f
tests/qapi-schema/struct-member-if-invalid.err new file +0 -0
tests/qapi-schema/struct-member-if-invalid.exit new file +1 -0
@@ -0,0 +1 @@ +0
tests/qapi-schema/struct-member-if-invalid.json new file +4 -0
@@ -0,0 +1,4 @@ +# Cover member with invalid 'if' +# FIXME not rejected, would generate '#if True\n' +{ 'struct': 'Stru', + 'data': { 'member': { 'type': 'int', 'if': true } } }
tests/qapi-schema/struct-member-if-invalid.out new file +15 -0
@@ -0,0 +1,15 @@ +module None +object q_empty +enum QType + prefix QTYPE + member none + member qnull + member qnum + member qstring + member qdict + member qlist + member qbool +module struct-member-if-invalid.json +object Stru + member member: int optional=False + if [True]
tests/qapi-schema/union-branch-if-invalid.err new file +0 -0
tests/qapi-schema/union-branch-if-invalid.exit new file +1 -0
@@ -0,0 +1 @@ +0
tests/qapi-schema/union-branch-if-invalid.json new file +7 -0
@@ -0,0 +1,7 @@ +# Cover branch with invalid 'if' +# FIXME not rejected, would generate '#if \n' +{ 'enum': 'Branches', 'data': ['branch1'] } +{ 'struct': 'Stru', 'data': { 'member': 'str' } } +{ 'union': 'Uni', + 'base': { 'tag': 'Branches' }, 'discriminator': 'tag', + 'data': { 'branch1': { 'type': 'Stru', 'if': [''] } } }
tests/qapi-schema/union-branch-if-invalid.out new file +23 -0
@@ -0,0 +1,23 @@ +module None +object q_empty +enum QType + prefix QTYPE + member none + member qnull + member qnum + member qstring + member qdict + member qlist + member qbool +module union-branch-if-invalid.json +enum Branches + member branch1 +object Stru + member member: str optional=False +object q_obj_Uni-base + member tag: Branches optional=False +object Uni + base q_obj_Uni-base + tag tag + case branch1: Stru + if ['']
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 05/19] tests/qapi-schema: Demonstrate insufficient 'if' checking
Quoted 5 lines · Markus Armbruster
Cover invalid 'if' in struct members, features, union and alternate branches. Four out of four are broken. Mark FIXME.
Signed-off-by Markus Armbruster <[email protected]>---
Embarrassing. But the fact that you're pointing them out presumably means that you fix it later in the series ;)
tests/qapi-schema/features-if-invalid.json line 2
Quoted 4 lines
+++
b/tests/qapi-schema/features-if-invalid.jsontests/qapi-schema/features-if-invalid.json new file +2
-0@@ -0,0 +1,5 @@ +# Cover feature with invalid 'if' +# FIXME not rejected, misinterpreded as unconditional
misinterpreted
With the typo fix,
tests/qapi-schema/struct-member-if-invalid.json line 2
Quoted 4 lines
tests/qapi-schema/struct-member-if-invalid.json new file +3
-0+++ b/tests/qapi-schema/struct-member-if-invalid.json@@ -0,0 +1,4 @@ +# Cover member with invalid 'if' +# FIXME not rejected, would generate '#if True\n'
Which might actually compile, depending on what else is present in various headers! But certainly not what was intended.
tests/qapi-schema/union-branch-if-invalid.json line 7
Quoted 9 lines
tests/qapi-schema/union-branch-if-invalid.json new file +8
-0+++ b/tests/qapi-schema/union-branch-if-invalid.json@@ -0,0 +1,7 @@ +# Cover branch with invalid 'if' +# FIXME not rejected, would generate '#if \n' +{ 'enum': 'Branches', 'data': ['branch1'] } +{ 'struct': 'Stru', 'data': { 'member': 'str' } } +{ 'union': 'Uni', + 'base': { 'tag': 'Branches' }, 'discriminator': 'tag', + 'data': { 'branch1': { 'type': 'Stru', 'if': [''] } } }
So you're pointing out a difference between an empty string and a string not containing a C macro name (possibly because later patches will give them different error messages).
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 05/19] tests/qapi-schema: Demonstrate insufficient 'if' checking
Quoted 9 lines · Eric Blake
Quoted 5 lines · Markus Armbruster
Cover invalid 'if' in struct members, features, union and alternate branches. Four out of four are broken. Mark FIXME.
Signed-off-by Markus Armbruster <[email protected]>---
Embarrassing. But the fact that you're pointing them out presumably means that you fix it later in the series ;)
Yes:
[PATCH 09/19] qapi: Remove null from schema language
[PATCH 12/19] qapi: Fix missing 'if' checks in struct, union, alternate 'data'
Quoted 10 lines
tests/qapi-schema/features-if-invalid.json line 2
Quoted 4 lines
+++
b/tests/qapi-schema/features-if-invalid.jsontests/qapi-schema/features-if-invalid.json new file +2
-0@@ -0,0 +1,5 @@ +# Cover feature with invalid 'if' +# FIXME not rejected, misinterpreded as unconditionalmisinterpreted
With the typo fix,
Reviewed-by Eric Blake <[email protected]>
Thanks!
Quoted 8 lines
tests/qapi-schema/struct-member-if-invalid.json line 2
Quoted 4 lines
tests/qapi-schema/struct-member-if-invalid.json new file +3
-0+++ b/tests/qapi-schema/struct-member-if-invalid.json@@ -0,0 +1,4 @@ +# Cover member with invalid 'if' +# FIXME not rejected, would generate '#if True\n'Which might actually compile, depending on what else is present in various headers! But certainly not what was intended.
Quoted 13 lines
tests/qapi-schema/union-branch-if-invalid.json line 7
Quoted 9 lines
tests/qapi-schema/union-branch-if-invalid.json new file +8
-0+++ b/tests/qapi-schema/union-branch-if-invalid.json@@ -0,0 +1,7 @@ +# Cover branch with invalid 'if' +# FIXME not rejected, would generate '#if \n' +{ 'enum': 'Branches', 'data': ['branch1'] } +{ 'struct': 'Stru', 'data': { 'member': 'str' } } +{ 'union': 'Uni', + 'base': { 'tag': 'Branches' }, 'discriminator': 'tag', + 'data': { 'branch1': { 'type': 'Stru', 'if': [''] } } }So you're pointing out a difference between an empty string and a string not containing a C macro name (possibly because later patches will give them different error messages).
Not sure I got this comment.
Re: [Qemu-devel] [PATCH 05/19] tests/qapi-schema: Demonstrate insufficient 'if' checking
Quoted 14 lines · Markus Armbruster
Quoted 5 lines
Quoted 5 lines
+++
b/tests/qapi-schema/union-branch-if-invalid.jsontests/qapi-schema/union-branch-if-invalid.json new file +3
-0@@ -0,0 +1,7 @@ +# Cover branch with invalid 'if' +# FIXME not rejected, would generate '#if \n' +{ 'enum': 'Branches', 'data': ['branch1'] }Quoted 8 lines
tests/qapi-schema/union-branch-if-invalid.json line 7
Quoted 4 lines
tests/qapi-schema/union-branch-if-invalid.json new file +4
-0+{ 'struct': 'Stru', 'data': { 'member': 'str' } } +{ 'union': 'Uni', + 'base': { 'tag': 'Branches' }, 'discriminator': 'tag', + 'data': { 'branch1': { 'type': 'Stru', 'if': [''] } } }So you're pointing out a difference between an empty string and a string not containing a C macro name (possibly because later patches will give them different error messages).
Quoted 1 line
Not sure I got this comment.
I was comparing:
Quoted 2 lines
+++
b/tests/qapi-schema/union-branch-if-invalid.json
+ 'data': { 'branch1': { 'type': 'Stru', 'if': [''] } } }
with:
Quoted 2 lines
+++
b/tests/qapi-schema/alternate-branch-if-invalid.json
+ 'data': { 'branch': { 'type': 'int', 'if': ' ' } } }
Both of which produce invalid expansions, but because of the difference between empty string vs. all-whitespace might be fixed differently in later patches. At any rate, nothing to change in those tests.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 6/19 tests/qapi-schema: Demonstrate suboptimal lexical errors
In reply to Markus ArmbrusterThe error message for forgotten quotes around a name shows just the name's first character, which isn't as nice as it could be. Same for attempting to use a number.
--- tests/Makefile.include | 1 + tests/qapi-schema/bad-type-int.json | 2 +- tests/qapi-schema/funny-word.err | 1 + tests/qapi-schema/funny-word.exit | 1 + tests/qapi-schema/funny-word.json | 1 + tests/qapi-schema/funny-word.out | 0 6 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 tests/qapi-schema/funny-word.err create mode 100644 tests/qapi-schema/funny-word.exit create mode 100644 tests/qapi-schema/funny-word.json create mode 100644 tests/qapi-schema/funny-word.out
tests/Makefile.include +1 -0
@@ -403,6 +403,7 @@ qapi-schema += flat-union-no-base.json qapi-schema += flat-union-optional-discriminator.json qapi-schema += flat-union-string-discriminator.json qapi-schema += funny-char.json +qapi-schema += funny-word.json qapi-schema += ident-with-escape.json qapi-schema += include-before-err.json qapi-schema += include-cycle.json
tests/qapi-schema/bad-type-int.json +1 -1
@@ -1,3 +1,3 @@ # we reject an expression with a metatype that is not a string # FIXME: once the parser understands integer inputs, improve the error message -{ 'struct': 1, 'data': { } } +{ 'struct': 123, 'data': { } }
tests/qapi-schema/funny-word.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/funny-word.json:1:3: Stray "c"
tests/qapi-schema/funny-word.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/funny-word.json new file +1 -0
@@ -0,0 +1 @@ +{ command: 'foo' }
tests/qapi-schema/funny-word.out new file +0 -0
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 06/19] tests/qapi-schema: Demonstrate suboptimal lexical errors
Quoted 3 lines · Markus Armbruster
The error message for forgotten quotes around a name shows just the name's first character, which isn't as nice as it could be. Same for attempting to use a number.
And we'll want to use numbers for defaults soon enough...
Quoted 3 lines
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 7/19 qapi: Use quotes more consistently in frontend error messages
In reply to Markus ArmbrusterConsistently enclose error messages in double quotes. Use single quotes within, except for one case of "'".
--- scripts/qapi/common.py | 37 ++++++++++--------- tests/qapi-schema/bad-type-int.err | 2 +- tests/qapi-schema/doc-missing-colon.err | 2 +- tests/qapi-schema/duplicate-key.err | 2 +- tests/qapi-schema/enum-int-member.err | 2 +- tests/qapi-schema/escape-outside-string.err | 1 + tests/qapi-schema/funny-char.err | 2 +- tests/qapi-schema/funny-word.err | 2 +- tests/qapi-schema/include-before-err.err | 2 +- tests/qapi-schema/include-nested-err.err | 2 +- tests/qapi-schema/leading-comma-list.err | 2 +- tests/qapi-schema/leading-comma-object.err | 2 +- tests/qapi-schema/missing-colon.err | 2 +- tests/qapi-schema/missing-comma-list.err | 2 +- tests/qapi-schema/missing-comma-object.err | 2 +- tests/qapi-schema/non-objects.err | 2 +- tests/qapi-schema/quoted-structural-chars.err | 2 +- tests/qapi-schema/trailing-comma-list.err | 2 +- tests/qapi-schema/unclosed-list.err | 2 +- tests/qapi-schema/unclosed-object.err | 2 +- 20 files changed, 38 insertions(+), 36 deletions(-) create mode 100644 tests/qapi-schema/escape-outside-string.err
scripts/qapi/common.py +19 -18
@@ -214,7 +214,7 @@ class QAPIDoc(object): # recognized, and get silently treated as ordinary text if not self.symbol and not self.body.text and line.startswith('@'): if not line.endswith(':'): - raise QAPIParseError(self._parser, "Line should end with :") + raise QAPIParseError(self._parser, "Line should end with ':'") self.symbol = line[1:-1] # FIXME invalid names other than the empty string aren't flagged if not self.symbol:
@@ -470,7 +470,7 @@ class QAPISchemaParser(object): else: fobj = open(incl_fname, 'r') except IOError as e: - raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname)) + raise QAPISemError(info, "%s: %s" % (e.strerror, incl_fname)) return QAPISchemaParser(fobj, previously_included, info) def _pragma(self, name, value, info):
@@ -522,7 +522,7 @@ class QAPISchemaParser(object): ch = self.src[self.cursor] self.cursor += 1 if ch == '\n': - raise QAPIParseError(self, 'Missing terminating "\'"') + raise QAPIParseError(self, "Missing terminating \"'\"") if esc: # Note: we recognize only \\ because we have # no use for funny characters in strings
@@ -559,7 +559,7 @@ class QAPISchemaParser(object): self.line += 1 self.line_pos = self.cursor elif not self.tok.isspace(): - raise QAPIParseError(self, 'Stray "%s"' % self.tok) + raise QAPIParseError(self, "Stray '%s'" % self.tok) def get_members(self): expr = OrderedDict()
@@ -567,24 +567,24 @@ class QAPISchemaParser(object): self.accept() return expr if self.tok != "'": - raise QAPIParseError(self, 'Expected string or "}"') + raise QAPIParseError(self, "Expected string or '}'") while True: key = self.val self.accept() if self.tok != ':': - raise QAPIParseError(self, 'Expected ":"') + raise QAPIParseError(self, "Expected ':'") self.accept() if key in expr: - raise QAPIParseError(self, 'Duplicate key "%s"' % key) + raise QAPIParseError(self, "Duplicate key '%s'" % key) expr[key] = self.get_expr(True) if self.tok == '}': self.accept() return expr if self.tok != ',': - raise QAPIParseError(self, 'Expected "," or "}"') + raise QAPIParseError(self, "Expected ',' or '}'") self.accept() if self.tok != "'": - raise QAPIParseError(self, 'Expected string') + raise QAPIParseError(self, "Expected string") def get_values(self): expr = []
@@ -592,20 +592,20 @@ class QAPISchemaParser(object): self.accept() return expr if self.tok not in "{['tfn": - raise QAPIParseError(self, 'Expected "{", "[", "]", string, ' - 'boolean or "null"') + raise QAPIParseError( + self, "Expected '{', '[', ']', string, boolean or 'null'") while True: expr.append(self.get_expr(True)) if self.tok == ']': self.accept() return expr if self.tok != ',': - raise QAPIParseError(self, 'Expected "," or "]"') + raise QAPIParseError(self, "Expected ',' or ']'") self.accept() def get_expr(self, nested): if self.tok != '{' and not nested: - raise QAPIParseError(self, 'Expected "{"') + raise QAPIParseError(self, "Expected '{'") if self.tok == '{': self.accept() expr = self.get_members()
@@ -616,8 +616,8 @@ class QAPISchemaParser(object): expr = self.val self.accept() else: - raise QAPIParseError(self, 'Expected "{", "[", string, ' - 'boolean or "null"') + raise QAPIParseError( + self, "Expected '{', '[', string, boolean or 'null'") return expr def get_doc(self, info):
@@ -881,9 +881,10 @@ def check_union(expr, info): "struct '%s'" % (discriminator, base)) if discriminator_value.get('if'): - raise QAPISemError(info, 'The discriminator %s.%s for union %s ' - 'must not be conditional' % - (base, discriminator, name)) + raise QAPISemError( + info, + "The discriminator %s.%s for union %s must not be conditional" + % (base, discriminator, name)) enum_define = enum_types.get(discriminator_value['type']) # Do not allow string discriminator if not enum_define:
tests/qapi-schema/bad-type-int.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/bad-type-int.json:3:13: Stray "1" +tests/qapi-schema/bad-type-int.json:3:13: Stray '1'
tests/qapi-schema/doc-missing-colon.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/doc-missing-colon.json:4:1: Line should end with : +tests/qapi-schema/doc-missing-colon.json:4:1: Line should end with ':'
tests/qapi-schema/duplicate-key.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/duplicate-key.json:3:10: Duplicate key "key" +tests/qapi-schema/duplicate-key.json:3:10: Duplicate key 'key'
tests/qapi-schema/enum-int-member.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/enum-int-member.json:3:31: Stray "1" +tests/qapi-schema/enum-int-member.json:3:31: Stray '1'
tests/qapi-schema/escape-outside-string.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/escape-outside-string.json:3:27: Stray '\'
tests/qapi-schema/funny-char.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/funny-char.json:2:36: Stray ";" +tests/qapi-schema/funny-char.json:2:36: Stray ';'
tests/qapi-schema/funny-word.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/funny-word.json:1:3: Stray "c" +tests/qapi-schema/funny-word.json:1:3: Stray 'c'
tests/qapi-schema/include-before-err.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/include-before-err.json:2:13: Expected ":" +tests/qapi-schema/include-before-err.json:2:13: Expected ':'
tests/qapi-schema/include-nested-err.err +1 -1
@@ -1,2 +1,2 @@ In file included from tests/qapi-schema/include-nested-err.json:1: -tests/qapi-schema/missing-colon.json:1:10: Expected ":" +tests/qapi-schema/missing-colon.json:1:10: Expected ':'
tests/qapi-schema/leading-comma-list.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/leading-comma-list.json:2:13: Expected "{", "[", "]", string, boolean or "null" +tests/qapi-schema/leading-comma-list.json:2:13: Expected '{', '[', ']', string, boolean or 'null'
tests/qapi-schema/leading-comma-object.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/leading-comma-object.json:1:3: Expected string or "}" +tests/qapi-schema/leading-comma-object.json:1:3: Expected string or '}'
tests/qapi-schema/missing-colon.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/missing-colon.json:1:10: Expected ":" +tests/qapi-schema/missing-colon.json:1:10: Expected ':'
tests/qapi-schema/missing-comma-list.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/missing-comma-list.json:2:20: Expected "," or "]" +tests/qapi-schema/missing-comma-list.json:2:20: Expected ',' or ']'
tests/qapi-schema/missing-comma-object.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/missing-comma-object.json:2:3: Expected "," or "}" +tests/qapi-schema/missing-comma-object.json:2:3: Expected ',' or '}'
tests/qapi-schema/non-objects.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/non-objects.json:1:1: Expected "{" +tests/qapi-schema/non-objects.json:1:1: Expected '{'
tests/qapi-schema/quoted-structural-chars.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/quoted-structural-chars.json:1:1: Expected "{" +tests/qapi-schema/quoted-structural-chars.json:1:1: Expected '{'
tests/qapi-schema/trailing-comma-list.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/trailing-comma-list.json:2:36: Expected "{", "[", string, boolean or "null" +tests/qapi-schema/trailing-comma-list.json:2:36: Expected '{', '[', string, boolean or 'null'
tests/qapi-schema/unclosed-list.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/unclosed-list.json:1:20: Expected "," or "]" +tests/qapi-schema/unclosed-list.json:1:20: Expected ',' or ']'
tests/qapi-schema/unclosed-object.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/unclosed-object.json:1:21: Expected "," or "}" +tests/qapi-schema/unclosed-object.json:1:21: Expected ',' or '}'
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 07/19] qapi: Use quotes more consistently in frontend error messages
scripts/qapi/common.py class QAPIDoc(object): · line 217
Quoted 13 lines · Markus Armbruster
Consistently enclose error messages in double quotes. Use single quotes within, except for one case of "'".
Signed-off-by Markus Armbruster <[email protected]>---
+++
b/scripts/qapi/common.pyscripts/qapi/common.py +1
-1@@ -214,7 +214,7 @@ class QAPIDoc(object): # recognized, and get silently treated as ordinary text if not self.symbol and not self.body.text and line.startswith('@'): if not line.endswith(':'): - raise QAPIParseError(self._parser, "Line should end with :") + raise QAPIParseError(self._parser, "Line should end with ':'")
Makes it more obvious what belongs in the .json file (where we DO want the use of ''), so this looks nicer.
scripts/qapi/common.py class QAPISchemaParser(object): · line 525
Quoted 6 lines
scripts/qapi/common.py +1
-1@@ -522,7 +522,7 @@ class QAPISchemaParser(object): ch = self.src[self.cursor] self.cursor += 1 if ch == '\n': - raise QAPIParseError(self, 'Missing terminating "\'"') + raise QAPIParseError(self, "Missing terminating \"'\"")
And this would be the one exception mentioned in the commit message.
scripts/qapi/common.py class QAPISchemaParser(object): · line 596
Quoted 8 lines
scripts/qapi/common.py +2
-2@@ -592,20 +592,20 @@ class QAPISchemaParser(object): self.accept() return expr if self.tok not in "{['tfn": - raise QAPIParseError(self, 'Expected "{", "[", "]", string, ' - 'boolean or "null"') + raise QAPIParseError( + self, "Expected '{', '[', ']', string, boolean or 'null'")
Definite improvement here.
And of course, you can correlate which tests changed output to a measure of how well we've covered the various error paths.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 8/19 qapi: Improve reporting of lexical errors
In reply to Markus ArmbrusterShow text up to next structural character, whitespace, or quote character instead of just the first character.
Forgotten quotes now get reported like "Stray 'command'" instead of "Stray 'c'".
--- scripts/qapi/common.py | 6 +++++- tests/qapi-schema/bad-type-int.err | 2 +- tests/qapi-schema/funny-word.err | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-)
scripts/qapi/common.py +5 -1
@@ -559,7 +559,11 @@ class QAPISchemaParser(object): self.line += 1 self.line_pos = self.cursor elif not self.tok.isspace(): - raise QAPIParseError(self, "Stray '%s'" % self.tok) + # Show up to next structural, whitespace or quote + # character + match = re.match('[^[\\]{}:,\\s\'"]+', + self.src[self.cursor-1:]) + raise QAPIParseError(self, "Stray '%s'" % match.group(0)) def get_members(self): expr = OrderedDict()
tests/qapi-schema/bad-type-int.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/bad-type-int.json:3:13: Stray '1' +tests/qapi-schema/bad-type-int.json:3:13: Stray '123'
tests/qapi-schema/funny-word.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/funny-word.json:1:3: Stray 'c' +tests/qapi-schema/funny-word.json:1:3: Stray 'command'
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 08/19] qapi: Improve reporting of lexical errors
Quoted 8 lines · Markus Armbruster
Show text up to next structural character, whitespace, or quote character instead of just the first character.
Forgotten quotes now get reported like "Stray 'command'" instead of "Stray 'c'".
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 9/19 qapi: Remove null from schema language
In reply to Markus ArmbrusterWe represent the parse tree as OrderedDict. We fetch optional dict members with .get(). So far, so good.
We represent null literals as None. .get() returns None both for "absent" and for "present, value is the null literal". Uh-oh.
Test features-if-invalid exposes this bug: "'if': null" is misinterpreted as absent "if".
We added null to the schema language to "allow [...] an explicit default value" (commit e53188ada5 "qapi: Allow true, false and null in schema json", v2.4.0). Hasn't happened; null is still unused except as generic invalid value in tests/.
To fix, we'd have to replace .get() by something more careful, or represent null differently. Feasible, but we got more and bigger fish to fry right now. Remove the null literal from the schema language. Replace null in tests by another invalid value.
Test features-if-invalid now behaves as it should.
--- docs/devel/qapi-code-gen.txt | 4 ++-- scripts/qapi/common.py | 4 ---- tests/qapi-schema/features-if-invalid.err | 1 + tests/qapi-schema/features-if-invalid.exit | 2 +- tests/qapi-schema/features-if-invalid.json | 3 +-- tests/qapi-schema/features-if-invalid.out | 14 -------------- .../pragma-name-case-whitelist-crap.json | 2 +- 7 files changed, 6 insertions(+), 24 deletions(-)
docs/devel/qapi-code-gen.txt +2 -2
@@ -52,7 +52,7 @@ Differences: * Strings are restricted to printable ASCII, and escape sequences to just '\\'. -* Numbers are not supported. +* Numbers and null are not supported. A second layer of syntax defines the sequences of JSON texts that are a correctly structured QAPI schema. We provide a grammar for this
@@ -67,7 +67,7 @@ syntax in an EBNF-like notation: expression A separated by , * Grouping: expression ( A ) matches expression A * JSON's structural characters are terminals: { } [ ] : , -* JSON's literal names are terminals: false true null +* JSON's literal names are terminals: false true * String literals enclosed in 'single quotes' are terminal, and match this JSON string, with a leading '*' stripped off * When JSON object member's name starts with '*', the member is
scripts/qapi/common.py +0 -4
@@ -548,10 +548,6 @@ class QAPISchemaParser(object): self.val = False self.cursor += 4 return - elif self.src.startswith('null', self.pos): - self.val = None - self.cursor += 3 - return elif self.tok == '\n': if self.cursor == len(self.src): self.tok = None
tests/qapi-schema/features-if-invalid.err +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/features-if-invalid.json:2: 'if' condition must be a string or a list of strings
tests/qapi-schema/features-if-invalid.exit +1 -1
@@ -1 +1 @@ -0 +1
tests/qapi-schema/features-if-invalid.json +1 -2
@@ -1,5 +1,4 @@ # Cover feature with invalid 'if' -# FIXME not rejected, misinterpreded as unconditional { 'struct': 'Stru', 'data': {}, - 'features': [{'name': 'f', 'if': null }] } + 'features': [{'name': 'f', 'if': false }] }
tests/qapi-schema/features-if-invalid.out +0 -14
@@ -1,14 +0,0 @@ -module None -object q_empty -enum QType - prefix QTYPE - member none - member qnull - member qnum - member qstring - member qdict - member qlist - member qbool -module features-if-invalid.json -object Stru - feature f
tests/qapi-schema/pragma-name-case-whitelist-crap.json +1 -1
@@ -1,3 +1,3 @@ # 'name-case-whitelist' must be list of strings -{ 'pragma': { 'name-case-whitelist': null } } +{ 'pragma': { 'name-case-whitelist': false } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 09/19] qapi: Remove null from schema language
Quoted 6 lines · Markus Armbruster
We represent the parse tree as OrderedDict. We fetch optional dict members with .get(). So far, so good.
We represent null literals as None. .get() returns None both for "absent" and for "present, value is the null literal". Uh-oh.
Quoted 7 lines
Test features-if-invalid exposes this bug: "'if': null" is misinterpreted as absent "if".
We added null to the schema language to "allow [...] an explicit default value" (commit
e53188ada5"qapi: Allow true, false and null in schema json", v2.4.0). Hasn't happened; null is still unused except as generic invalid value in tests/.
Might happen as a way to default a '*value':'str', but we can deal with that at the point where someone actually justifies a need for a way to express a default like that.
Quoted 4 lines
To fix, we'd have to replace .get() by something more careful, or represent null differently. Feasible, but we got more and bigger fish to fry right now. Remove the null literal from the schema language.
Fair enough.
tests/qapi-schema/features-if-invalid.json line 2
Quoted 11 lines
Replace null in tests by another invalid value.
Test features-if-invalid now behaves as it should.
Signed-off-by Markus Armbruster <[email protected]>---
+++
b/tests/qapi-schema/features-if-invalid.jsontests/qapi-schema/features-if-invalid.json +0
-1@@ -1,5 +1,4 @@ # Cover feature with invalid 'if' -# FIXME not rejected, misinterpreded as unconditional
Obvious conflict resolution with the typo fix earlier in the series.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 10/19 qapi: Fix broken discriminator error messages
In reply to Markus Armbrustercheck_union() checks the discriminator exists in base and makes sense. Two error messages mention the base. These are broken for anonymous bases, as demonstrated by tests flat-union-invalid-discriminator and flat-union-invalid-if-discriminator.err. The third one doesn't bother.
First broken when commit ac4338f8eb "qapi: Allow anonymous base for flat union" (v2.6.0) neglected to adjust the "not a member of base" error message. Commit ccadd6bcba "qapi: Add 'if' to implicit struct members" (v4.0.0) then cloned the flawed error message.
Dumb them down not to mention the base.
--- scripts/qapi/common.py | 9 ++++----- tests/qapi-schema/flat-union-invalid-discriminator.err | 2 +- tests/qapi-schema/flat-union-invalid-discriminator.json | 1 - .../qapi-schema/flat-union-invalid-if-discriminator.err | 2 +- .../qapi-schema/flat-union-invalid-if-discriminator.json | 1 - tests/qapi-schema/flat-union-optional-discriminator.err | 2 +- tests/qapi-schema/union-base-empty.err | 2 +- 7 files changed, 8 insertions(+), 11 deletions(-)
scripts/qapi/common.py +4 -5
@@ -877,14 +877,13 @@ def check_union(expr, info): discriminator_value = base_members.get(discriminator) if not discriminator_value: raise QAPISemError(info, - "Discriminator '%s' is not a member of base " - "struct '%s'" - % (discriminator, base)) + "Discriminator '%s' is not a member of 'base'" + % discriminator) if discriminator_value.get('if'): raise QAPISemError( info, - "The discriminator %s.%s for union %s must not be conditional" - % (base, discriminator, name)) + "The discriminator '%s' for union %s must not be conditional" + % (discriminator, name)) enum_define = enum_types.get(discriminator_value['type']) # Do not allow string discriminator if not enum_define:
tests/qapi-schema/flat-union-invalid-discriminator.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/flat-union-invalid-discriminator.json:11: Discriminator 'enum_wrong' is not a member of base struct 'OrderedDict([('enum1', {'type': 'TestEnum'})])' +tests/qapi-schema/flat-union-invalid-discriminator.json:10: Discriminator 'enum_wrong' is not a member of 'base'
tests/qapi-schema/flat-union-invalid-discriminator.json +0 -1
@@ -1,4 +1,3 @@ -# FIXME error message shows base as OrderedDict { 'enum': 'TestEnum', 'data': [ 'value1', 'value2' ] }
tests/qapi-schema/flat-union-invalid-if-discriminator.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/flat-union-invalid-if-discriminator.json:11: The discriminator OrderedDict([('enum1', OrderedDict([('type', 'TestEnum'), ('if', 'FOO')]))]).enum1 for union TestUnion must not be conditional +tests/qapi-schema/flat-union-invalid-if-discriminator.json:10: The discriminator 'enum1' for union TestUnion must not be conditional
tests/qapi-schema/flat-union-invalid-if-discriminator.json +0 -1
@@ -1,4 +1,3 @@ -# FIXME error message shows base as OrderedDict { 'enum': 'TestEnum', 'data': [ 'value1', 'value2' ] }
tests/qapi-schema/flat-union-optional-discriminator.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/flat-union-optional-discriminator.json:7: Discriminator 'switch' is not a member of base struct 'Base' +tests/qapi-schema/flat-union-optional-discriminator.json:7: Discriminator 'switch' is not a member of 'base'
tests/qapi-schema/union-base-empty.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/union-base-empty.json:5: Discriminator 'type' is not a member of base struct 'Empty' +tests/qapi-schema/union-base-empty.json:5: Discriminator 'type' is not a member of 'base'
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 10/19] qapi: Fix broken discriminator error messages
Quoted 6 lines · Markus Armbruster
check_union()checks the discriminator exists in base and makes sense. Two error messages mention the base. These are broken for anonymous bases, as demonstrated by tests flat-union-invalid-discriminator and flat-union-invalid-if-discriminator.err. The third one doesn't bother.
Quoted 6 lines
First broken when commit
ac4338f8eb"qapi: Allow anonymous base for flat union" (v2.6.0) neglected to adjust the "not a member of base" error message. Commitccadd6bcba"qapi: Add 'if' to implicit struct members" (v4.0.0) then cloned the flawed error message.Dumb them down not to mention the base.
A trick for good error reporting: deciding when "less is more" :)
Quoted 3 lines
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 11/19 qapi: Reject blank 'if' conditions in addition to empty ones
In reply to Markus Armbruster"'if': 'COND'" generates "#if COND". We reject empty COND because it won't compile. Blank COND won't compile any better, so reject that, too.
--- scripts/qapi/common.py | 5 +++-- tests/qapi-schema/bad-if-list.err | 2 +- tests/qapi-schema/bad-if-list.json | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-)
scripts/qapi/common.py +3 -2
@@ -742,8 +742,9 @@ def check_if(expr, info): if not isinstance(ifcond, str): raise QAPISemError( info, "'if' condition must be a string or a list of strings") - if ifcond == '': - raise QAPISemError(info, "'if' condition '' makes no sense") + if ifcond.strip() == '': + raise QAPISemError(info, "'if' condition '%s' makes no sense" + % ifcond) ifcond = expr.get('if') if ifcond is None:
tests/qapi-schema/bad-if-list.err +1 -1
@@ -1 +1 @@ -tests/qapi-schema/bad-if-list.json:2: 'if' condition '' makes no sense +tests/qapi-schema/bad-if-list.json:2: 'if' condition ' ' makes no sense
tests/qapi-schema/bad-if-list.json +1 -1
@@ -1,3 +1,3 @@ # check invalid 'if' content { 'struct': 'TestIfStruct', 'data': { 'foo': 'int' }, - 'if': ['foo', ''] } + 'if': ['foo', ' '] }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 11/19] qapi: Reject blank 'if' conditions in addition to empty ones
Quoted 6 lines · Markus Armbruster
"'if': 'COND'" generates "#if COND". We reject empty COND because it won't compile. Blank COND won't compile any better, so reject that, too.
Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 12/19 qapi: Fix missing 'if' checks in struct, union, alternate 'data'
In reply to Markus ArmbrusterCommit 87adbbffd4..3e270dcacc "qapi: Add 'if' to (implicit struct|union|alternate) members" (v4.0.0) neglected test coverage, and promptly failed to check the conditions. Review fail.
Recent commit "tests/qapi-schema: Demonstrate insufficient 'if' checking" added test coverage, demonstrating the bug. Fix it by add the missing check_if().
--- scripts/qapi/common.py | 3 +++ .../alternate-branch-if-invalid.err | 1 + .../alternate-branch-if-invalid.exit | 2 +- .../alternate-branch-if-invalid.json | 1 - .../alternate-branch-if-invalid.out | 16 ------------- .../qapi-schema/struct-member-if-invalid.err | 1 + .../qapi-schema/struct-member-if-invalid.exit | 2 +- .../qapi-schema/struct-member-if-invalid.json | 1 - .../qapi-schema/struct-member-if-invalid.out | 15 ------------ tests/qapi-schema/union-branch-if-invalid.err | 1 + .../qapi-schema/union-branch-if-invalid.exit | 2 +- .../qapi-schema/union-branch-if-invalid.json | 1 - tests/qapi-schema/union-branch-if-invalid.out | 23 ------------------- 13 files changed, 9 insertions(+), 60 deletions(-)
scripts/qapi/common.py +3 -0
@@ -803,6 +803,7 @@ def check_type(info, source, value, # an optional argument. check_known_keys(info, "member '%s' of %s" % (key, source), arg, ['type'], ['if']) + check_if(arg, info) check_type(info, "Member '%s' of %s" % (key, source), arg['type'], allow_array=True, allow_metas=['built-in', 'union', 'alternate', 'struct',
@@ -902,6 +903,7 @@ def check_union(expr, info): check_known_keys(info, "member '%s' of union '%s'" % (key, name), value, ['type'], ['if']) + check_if(value, info) # Each value must name a known type check_type(info, "Member '%s' of union '%s'" % (key, name), value['type'],
@@ -930,6 +932,7 @@ def check_alternate(expr, info): check_known_keys(info, "member '%s' of alternate '%s'" % (key, name), value, ['type'], ['if']) + check_if(value, info) typ = value['type'] # Ensure alternates have no type conflicts.
tests/qapi-schema/alternate-branch-if-invalid.err +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' makes no sense
tests/qapi-schema/alternate-branch-if-invalid.exit +1 -1
@@ -1 +1 @@ -0 +1
tests/qapi-schema/alternate-branch-if-invalid.json +0 -1
@@ -1,4 +1,3 @@ # Cover alternative with invalid 'if' -# FIXME not rejected, would generate '#if \n' { 'alternate': 'Alt', 'data': { 'branch': { 'type': 'int', 'if': ' ' } } }
tests/qapi-schema/alternate-branch-if-invalid.out +0 -16
@@ -1,16 +0,0 @@ -module None -object q_empty -enum QType - prefix QTYPE - member none - member qnull - member qnum - member qstring - member qdict - member qlist - member qbool -module alternate-branch-if-invalid.json -alternate Alt - tag type - case branch: int - if [' ']
tests/qapi-schema/struct-member-if-invalid.err +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/struct-member-if-invalid.json:2: 'if' condition must be a string or a list of strings
tests/qapi-schema/struct-member-if-invalid.exit +1 -1
@@ -1 +1 @@ -0 +1
tests/qapi-schema/struct-member-if-invalid.json +0 -1
@@ -1,4 +1,3 @@ # Cover member with invalid 'if' -# FIXME not rejected, would generate '#if True\n' { 'struct': 'Stru', 'data': { 'member': { 'type': 'int', 'if': true } } }
tests/qapi-schema/struct-member-if-invalid.out +0 -15
@@ -1,15 +0,0 @@ -module None -object q_empty -enum QType - prefix QTYPE - member none - member qnull - member qnum - member qstring - member qdict - member qlist - member qbool -module struct-member-if-invalid.json -object Stru - member member: int optional=False - if [True]
tests/qapi-schema/union-branch-if-invalid.err +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/union-branch-if-invalid.json:4: 'if' condition '' makes no sense
tests/qapi-schema/union-branch-if-invalid.exit +1 -1
@@ -1 +1 @@ -0 +1
tests/qapi-schema/union-branch-if-invalid.json +0 -1
@@ -1,5 +1,4 @@ # Cover branch with invalid 'if' -# FIXME not rejected, would generate '#if \n' { 'enum': 'Branches', 'data': ['branch1'] } { 'struct': 'Stru', 'data': { 'member': 'str' } } { 'union': 'Uni',
tests/qapi-schema/union-branch-if-invalid.out +0 -23
@@ -1,23 +0,0 @@ -module None -object q_empty -enum QType - prefix QTYPE - member none - member qnull - member qnum - member qstring - member qdict - member qlist - member qbool -module union-branch-if-invalid.json -enum Branches - member branch1 -object Stru - member member: str optional=False -object q_obj_Uni-base - member tag: Branches optional=False -object Uni - base q_obj_Uni-base - tag tag - case branch1: Stru - if ['']
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 12/19] qapi: Fix missing 'if' checks in struct, union, alternate 'data'
Quoted 10 lines · Markus Armbruster
Commit
87adbbffd4..3e270dcacc"qapi: Add 'if' to (implicit struct|union|alternate) members" (v4.0.0) neglected test coverage, and promptly failed to check the conditions. Review fail.Recent commit "tests/qapi-schema: Demonstrate insufficient 'if' checking" added test coverage, demonstrating the bug. Fix it by add the missing
check_if().Signed-off-by Markus Armbruster <[email protected]>---
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 13/19 qapi: Normalize 'if' in check_exprs(), like other sugar
In reply to Markus ArmbrusterWe normalize shorthand to longhand forms in check_expr(): enumeration values with normalize_enum(), feature values with normalize_features(), struct members, union branches and alternate branches with normalize_members(). If conditions are an exception: we normalize them in QAPISchemaEntity.check() and QAPISchemaMember.__init(), with listify_cond(). The idea goes back to commit 2cbc94376e "qapi: pass 'if' condition into QAPISchemaEntity objects", v3.0.0.
Normalize in check_expr() instead, with new helper normalize_if().
--- scripts/qapi/common.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)
scripts/qapi/common.py +15 -11
@@ -804,6 +804,7 @@ def check_type(info, source, value, check_known_keys(info, "member '%s' of %s" % (key, source), arg, ['type'], ['if']) check_if(arg, info) + normalize_if(arg) check_type(info, "Member '%s' of %s" % (key, source), arg['type'], allow_array=True, allow_metas=['built-in', 'union', 'alternate', 'struct',
@@ -904,6 +905,7 @@ def check_union(expr, info): check_known_keys(info, "member '%s' of union '%s'" % (key, name), value, ['type'], ['if']) check_if(value, info) + normalize_if(value) # Each value must name a known type check_type(info, "Member '%s' of union '%s'" % (key, name), value['type'],
@@ -933,6 +935,7 @@ def check_alternate(expr, info): "member '%s' of alternate '%s'" % (key, name), value, ['type'], ['if']) check_if(value, info) + normalize_if(value) typ = value['type'] # Ensure alternates have no type conflicts.
@@ -978,6 +981,7 @@ def check_enum(expr, info): check_known_keys(info, "member of enum '%s'" % name, member, ['name'], ['if']) check_if(member, info) + normalize_if(member) check_name(info, "Member of enum '%s'" % name, member['name'], enum_member=True)
@@ -1003,6 +1007,7 @@ def check_struct(expr, info): ['name'], ['if']) check_if(f, info) + normalize_if(f) check_name(info, "Feature of struct %s" % name, f['name'])
@@ -1067,6 +1072,12 @@ def normalize_features(features): for f in features] +def normalize_if(expr): + ifcond = expr.get('if') + if isinstance(ifcond, str): + expr['if'] = [ifcond] + + def check_exprs(exprs): global all_names
@@ -1123,6 +1134,7 @@ def check_exprs(exprs): else: raise QAPISemError(expr_elem['info'], "Expression is missing metatype") + normalize_if(expr) name = expr[meta] add_name(name, info, meta) if doc and doc.symbol != name:
@@ -1177,14 +1189,6 @@ def check_exprs(exprs): # Schema compiler frontend # -def listify_cond(ifcond): - if not ifcond: - return [] - if not isinstance(ifcond, list): - return [ifcond] - return ifcond - - class QAPISchemaEntity(object): def __init__(self, name, info, doc, ifcond=None): assert name is None or isinstance(name, str)
@@ -1197,7 +1201,7 @@ class QAPISchemaEntity(object): # such place). self.info = info self.doc = doc - self._ifcond = ifcond # self.ifcond is set only after .check() + self._ifcond = ifcond or [] def c_name(self): return c_name(self.name)
@@ -1209,7 +1213,7 @@ class QAPISchemaEntity(object): typ.check(schema) self.ifcond = typ.ifcond else: - self.ifcond = listify_cond(self._ifcond) + self.ifcond = self._ifcond if self.info: self.module = os.path.relpath(self.info['file'], os.path.dirname(schema.fname))
@@ -1515,7 +1519,7 @@ class QAPISchemaMember(object): def __init__(self, name, ifcond=None): assert isinstance(name, str) self.name = name - self.ifcond = listify_cond(ifcond) + self.ifcond = ifcond or [] self.owner = None def set_owner(self, name):
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 13/19] qapi: Normalize 'if' in check_exprs(), like other sugar
Quoted 9 lines · Markus Armbruster
We normalize shorthand to longhand forms in
check_expr(): enumeration values withnormalize_enum(), feature values withnormalize_features(), struct members, union branches and alternate branches withnormalize_members(). If conditions are an exception: we normalize them inQAPISchemaEntity.check()andQAPISchemaMember.__init(), withlistify_cond(). The idea goes back to commit2cbc94376e"qapi: pass 'if' condition into QAPISchemaEntity objects", v3.0.0.
Quoted 7 lines
Normalize in
check_expr()instead, with new helpernormalize_if().Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)
Definitely more consistent.
scripts/qapi/common.py def check_union(expr, info): · line 908
Quoted 5 lines
scripts/qapi/common.py +1
-0@@ -904,6 +905,7 @@ def check_union(expr, info): check_known_keys(info, "member '%s' of union '%s'" % (key, name), value, ['type'], ['if']) check_if(value, info) + normalize_if(value)
Is it worth combining check_if() and normalize_if() into a single helper function, since they always seem to be paired?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 13/19] qapi: Normalize 'if' in check_exprs(), like other sugar
Quoted 21 lines · Eric Blake
Quoted 9 lines · Markus Armbruster
We normalize shorthand to longhand forms in
check_expr(): enumeration values withnormalize_enum(), feature values withnormalize_features(), struct members, union branches and alternate branches withnormalize_members(). If conditions are an exception: we normalize them inQAPISchemaEntity.check()andQAPISchemaMember.__init(), withlistify_cond(). The idea goes back to commit2cbc94376e"qapi: pass 'if' condition into QAPISchemaEntity objects", v3.0.0.Quoted 7 lines
Normalize in
check_expr()instead, with new helpernormalize_if().Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)Definitely more consistent.
Quoted 8 lines
scripts/qapi/common.py def check_union(expr, info): · line 908
Quoted 5 lines
scripts/qapi/common.py +1
-0@@ -904,6 +905,7 @@ def check_union(expr, info): check_known_keys(info, "member '%s' of union '%s'" % (key, name), value, ['type'], ['if']) check_if(value, info) + normalize_if(value)Is it worth combining
check_if()andnormalize_if()into a single helper function, since they always seem to be paired?
The normalize_{enum,members,features}() are similarly paired with check_keys(). I sidestepped the question whether checking and normalizing should be fused. We can revisit it later.
Quoted 1 line
Reviewed-by Eric Blake <[email protected]>
Thanks!
PATCH 14/19 qapi: Simplify check_keys()
In reply to Markus Armbrustercheck_keys() parameter expr_elem expects a dictionary with keys 'expr' and 'info'. Passing the two values separately is simpler, so do that.
--- scripts/qapi/common.py | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-)
scripts/qapi/common.py +8 -11
@@ -1029,9 +1029,7 @@ def check_known_keys(info, source, keys, required, optional): source, pprint(allowed))) -def check_keys(expr_elem, meta, required, optional=[]): - expr = expr_elem['expr'] - info = expr_elem['info'] +def check_keys(expr, info, meta, required, optional=[]): name = expr[meta] if not isinstance(name, str): raise QAPISemError(info, "'%s' key must have a string value" % meta)
@@ -1100,40 +1098,39 @@ def check_exprs(exprs): if 'enum' in expr: meta = 'enum' - check_keys(expr_elem, 'enum', ['data'], ['if', 'prefix']) + check_keys(expr, info, 'enum', ['data'], ['if', 'prefix']) normalize_enum(expr) enum_types[expr[meta]] = expr elif 'union' in expr: meta = 'union' - check_keys(expr_elem, 'union', ['data'], + check_keys(expr, info, 'union', ['data'], ['base', 'discriminator', 'if']) normalize_members(expr.get('base')) normalize_members(expr['data']) union_types[expr[meta]] = expr elif 'alternate' in expr: meta = 'alternate' - check_keys(expr_elem, 'alternate', ['data'], ['if']) + check_keys(expr, info, 'alternate', ['data'], ['if']) normalize_members(expr['data']) elif 'struct' in expr: meta = 'struct' - check_keys(expr_elem, 'struct', ['data'], + check_keys(expr, info, 'struct', ['data'], ['base', 'if', 'features']) normalize_members(expr['data']) normalize_features(expr.get('features')) struct_types[expr[meta]] = expr elif 'command' in expr: meta = 'command' - check_keys(expr_elem, 'command', [], + check_keys(expr, info, 'command', [], ['data', 'returns', 'gen', 'success-response', 'boxed', 'allow-oob', 'allow-preconfig', 'if']) normalize_members(expr.get('data')) elif 'event' in expr: meta = 'event' - check_keys(expr_elem, 'event', [], ['data', 'boxed', 'if']) + check_keys(expr, info, 'event', [], ['data', 'boxed', 'if']) normalize_members(expr.get('data')) else: - raise QAPISemError(expr_elem['info'], - "Expression is missing metatype") + raise QAPISemError(info, "Expression is missing metatype") normalize_if(expr) name = expr[meta] add_name(name, info, meta)
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 14/19] qapi: Simplify check_keys()
Quoted 17 lines · Markus Armbruster
check_keys()parameterexpr_elemexpects a dictionary with keys 'expr' and 'info'. Passing the two values separately is simpler, so do that.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-)scripts/qapi/common.py +1
-3-def check_keys(expr_elem, meta, required, optional=[]): - expr = expr_elem['expr'] - info = expr_elem['info'] +def check_keys(expr, info, meta, required, optional=[]): name = expr[meta] if not isinstance(name, str): raise QAPISemError(info, "'%s' key must have a string value" % meta)
scripts/qapi/common.py def check_exprs(exprs): · line 1101
Quoted 5 lines
scripts/qapi/common.py +1
-1@@ -1100,40 +1098,39 @@ def check_exprs(exprs): if 'enum' in expr: meta = 'enum' - check_keys(expr_elem, 'enum', ['data'], ['if', 'prefix']) + check_keys(expr, info, 'enum', ['data'], ['if', 'prefix'])
Works since the caller already separated out the two values in question.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 15/19 qapi: Clean up around check_known_keys()
In reply to Markus ArmbrusterAll callers pass a dict argument to @keys, except check_keys() passes a dict's .keys(). Drop .keys() there, and rename parameter @keys to @value.
--- scripts/qapi/common.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
scripts/qapi/common.py +4 -4
@@ -1011,18 +1011,18 @@ def check_struct(expr, info): check_name(info, "Feature of struct %s" % name, f['name']) -def check_known_keys(info, source, keys, required, optional): +def check_known_keys(info, source, value, required, optional): def pprint(elems): return ', '.join("'" + e + "'" for e in sorted(elems)) - missing = set(required) - set(keys) + missing = set(required) - set(value) if missing: raise QAPISemError(info, "Key%s %s %s missing from %s" % ('s' if len(missing) > 1 else '', pprint(missing), 'are' if len(missing) > 1 else 'is', source)) allowed = set(required + optional) - unknown = set(keys) - allowed + unknown = set(value) - allowed if unknown: raise QAPISemError(info, "Unknown key%s %s in %s\nValid keys are %s." % ('s' if len(unknown) > 1 else '', pprint(unknown),
@@ -1035,7 +1035,7 @@ def check_keys(expr, info, meta, required, optional=[]): raise QAPISemError(info, "'%s' key must have a string value" % meta) required = required + [meta] source = "%s '%s'" % (meta, name) - check_known_keys(info, source, expr.keys(), required, optional) + check_known_keys(info, source, expr, required, optional) for (key, value) in expr.items(): if key in ['gen', 'success-response'] and value is not False: raise QAPISemError(info,
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 15/19] qapi: Clean up around check_known_keys()
Quoted 9 lines · Markus Armbruster
All callers pass a dict argument to @keys, except
check_keys()passes a dict's .keys(). Drop .keys() there, and rename parameter @keys to @value.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 16/19 qapi: Delete useless check_exprs() code for simple union kind
In reply to Markus ArmbrusterCommit bceae7697f "qapi script: support enum type as discriminator in union" made check_exprs() add the implicit enum types of simple unions to global @enum_types. I'm not sure it was needed even then. It's certainly not needed now. Delete it.
discriminator_find_enum_define() and add_name() parameter @implicit are now dead. Bury them.
--- scripts/qapi/common.py | 39 ++------------------------------------- 1 file changed, 2 insertions(+), 37 deletions(-)
scripts/qapi/common.py +2 -37
@@ -672,26 +672,6 @@ def find_alternate_member_qtype(qapi_type): return None -# Return the discriminator enum define if discriminator is specified as an -# enum type, otherwise return None. -def discriminator_find_enum_define(expr): - base = expr.get('base') - discriminator = expr.get('discriminator') - - if not (discriminator and base): - return None - - base_members = find_base_members(base) - if not base_members: - return None - - discriminator_value = base_members.get(discriminator) - if not discriminator_value: - return None - - return enum_types.get(discriminator_value['type']) - - # Names must be letters, numbers, -, and _. They must start with letter, # except for downstream extensions which must start with __RFQDN_. # Dots are only valid in the downstream extension prefix.
@@ -722,7 +702,7 @@ def check_name(info, source, name, allow_optional=False, raise QAPISemError(info, "%s uses invalid name '%s'" % (source, name)) -def add_name(name, info, meta, implicit=False): +def add_name(name, info, meta): global all_names check_name(info, "'%s'" % meta, name) # FIXME should reject names that differ only in '_' vs. '.'
@@ -730,7 +710,7 @@ def add_name(name, info, meta, implicit=False): if name in all_names: raise QAPISemError(info, "%s '%s' is already defined" % (all_names[name], name)) - if not implicit and (name.endswith('Kind') or name.endswith('List')): + if name.endswith('Kind') or name.endswith('List'): raise QAPISemError(info, "%s '%s' should not end in '%s'" % (meta, name, name[-4:])) all_names[name] = meta
@@ -1138,21 +1118,6 @@ def check_exprs(exprs): raise QAPISemError(info, "Definition of '%s' follows documentation" " for '%s'" % (name, doc.symbol)) - # Try again for hidden UnionKind enum - for expr_elem in exprs: - expr = expr_elem['expr'] - - if 'include' in expr: - continue - if 'union' in expr and not discriminator_find_enum_define(expr): - name = '%sKind' % expr['union'] - elif 'alternate' in expr: - name = '%sKind' % expr['alternate'] - else: - continue - enum_types[name] = {'enum': name} - add_name(name, info, 'enum', implicit=True) - # Validate that exprs make sense for expr_elem in exprs: expr = expr_elem['expr']
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 16/19] qapi: Delete useless check_exprs() code for simple union kind
Quoted 5 lines · Markus Armbruster
Commit
bceae7697f"qapi script: support enum type as discriminator in union" madecheck_exprs()add the implicit enum types of simple unions to global @enum_types. I'm not sure it was needed even then. It's certainly not needed now. Delete it.
Quoted 7 lines
discriminator_find_enum_define()andadd_name()parameter @implicit are now dead. Bury them.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 39 ++------------------------------------- 1 file changed, 2 insertions(+), 37 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 17/19 qapi: Fix to .check() empty structs just once
In reply to Markus ArmbrusterQAPISchemaObjectType.check() does nothing for types that have been checked already. Except the "has been checked" predicate is broken for empty types: self.members is [] then, which isn't true. The bug is harmless, but fix it anyway: use self.member is not None instead.
--- scripts/qapi/common.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/qapi/common.py +1 -1
@@ -1406,7 +1406,7 @@ class QAPISchemaObjectType(QAPISchemaType): if self.members is False: # check for cycles raise QAPISemError(self.info, "Object %s contains itself" % self.name) - if self.members: + if self.members is not None: # already checked return self.members = False # mark as being checked seen = OrderedDict()
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 17/19] qapi: Fix to .check() empty structs just once
Quoted 9 lines · Markus Armbruster
QAPISchemaObjectType.check()does nothing for types that have been checked already. Except the "has been checked" predicate is broken for empty types: self.members is [] then, which isn't true. The bug is harmless, but fix it anyway: use self.member is not None instead.Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines
scripts/qapi/common.py +1
-1@@ -1406,7 +1406,7 @@ class QAPISchemaObjectType(QAPISchemaType): if self.members is False: # check for cycles raise QAPISemError(self.info, "Object %s contains itself" % self.name) - if self.members: + if self.members is not None: # already checked return self.members = False # mark as being checked seen = OrderedDict()
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 18/19 qapi: Fix excessive QAPISchemaEntity.check() recursion
In reply to Markus ArmbrusterEntity checking goes back to commit ac88219a6c "qapi: New QAPISchema intermediate representation", v2.5.0. It's designed to work as
follows: QAPISchema.check() calls .check() for all the schema's entities. An entity's .check() recurses into another entity's .check() only if the C struct generated for the former contains the C struct generated for the latter (pointers don't count). This is used to detect "object contains itself".
There are two instances of this:
* An object's C struct contains its base's C struct
QAPISchemaObjectType.check() calls self.base.check()
* An object's C struct contains its variants' C structs
QAPISchemaObjectTypeVariants().check calls v.type.check(). Since commit b807a1e1e3 "qapi: Check for QAPI collisions involving variant members", v2.6.0.
Thus, only object types can participate in recursion.
QAPISchemaObjectType.check() is made for that: it checks @self when called the first time, recursing into base and variants, it reports an "contains itself" error when this recursion reaches an object being checked, and does nothing it reaches an object that has been checked already.
The other .check() may safely assume they get called exactly once.
Sadly, this design has since eroded:
* QAPISchemaCommand.check() and QAPISchemaEvent.check() call .args_type.check(). Since commit c818408e44 "qapi: Implement boxed types for commands/events", v2.7.0. Harmless, since args_type can only be an object type.
* QAPISchemaEntity.check() calls ._ifcond.check() when inheriting the condition from another type. Since commit 4fca21c1b0 qapi: leave the ifcond attribute undefined until check(), v3.0.0. This makes simple union wrapper types recurse into the wrapped type (nothing else uses this condition inheritance). The .check() of types used as simple union branch type get called multiple times.
* QAPISchemaObjectType.check() calls its super type's .check() *before* the conditional handling multiple calls. Also since commit 4fca21c1b0. QAPISchemaObjectType.check()'s guard against multiple checking doesn't protect QAPISchemaEntity.check().
* QAPISchemaArrayType.check() calls .element_type.check(). Also since commit 4fca21c1b0. The .check() of types used as array element types get called multiple times.
Commit 56a4689582 "qapi: Fix array first used in a different module" (v4.0.0) added more code relying on this .element_type.check().
The absence of explosions suggests the .check() involved happen to be effectively idempotent.
Fix the unwanted recursion anyway:
* QAPISchemaCommand.check() and QAPISchemaEvent.check() calling .args_type.check() is unnecessary. Delete the calls.
* Fix QAPISchemaObjectType.check() to call its super type's .check() after the conditional handling multiple calls.
* A QAPISchemaEntity's .ifcond becomes valid at .check(). This is due to arrays and simple unions.
Most types get ifcond and info passed to their constructor.
Array types don't: they get it from their element type, which becomes known only in .element_type.check().
The implicit wrapper object types for simple union branches don't:
they get it from the wrapped type, which might be an array.
Ditch the idea to set .ifcond in .check(). Instead, turn it into a property and compute it on demand. Safe because it's only used after the schema has been checked.
Most types simply return the ifcond passed to their constructor.
Array types forward to their .element_type instead, and the wrapper types forward to the wrapped type.
* A QAPISchemaEntity's .module becomes valid at .check(). This is because computing it needs info and schema.fname, and because array types get it from their element type instead.
Make it a property just like .ifcond.
Additionally, have QAPISchemaEntity.check() assert it gets called at most once, so the design invariant will stick this time. Neglecting that was entirely my fault.
--- scripts/qapi/common.py | 74 +++++++++++++++++++++++++++++------------- 1 file changed, 52 insertions(+), 22 deletions(-)
scripts/qapi/common.py +52 -22
@@ -1155,7 +1155,7 @@ class QAPISchemaEntity(object): def __init__(self, name, info, doc, ifcond=None): assert name is None or isinstance(name, str) self.name = name - self.module = None + self._module = None # For explicitly defined entities, info points to the (explicit) # definition. For builtins (and their arrays), info is None. # For implicitly defined entities, info points to a place that
@@ -1164,21 +1164,27 @@ class QAPISchemaEntity(object): self.info = info self.doc = doc self._ifcond = ifcond or [] + self._checked = False def c_name(self): return c_name(self.name) def check(self, schema): - if isinstance(self._ifcond, QAPISchemaType): - # inherit the condition from a type - typ = self._ifcond - typ.check(schema) - self.ifcond = typ.ifcond - else: - self.ifcond = self._ifcond + assert not self._checked if self.info: - self.module = os.path.relpath(self.info['file'], - os.path.dirname(schema.fname)) + self._module = os.path.relpath(self.info['file'], + os.path.dirname(schema.fname)) + self._checked = True + + @property + def ifcond(self): + assert self._checked + return self._ifcond + + @property + def module(self): + assert self._checked + return self._module def is_implicit(self): return not self.info
@@ -1353,9 +1359,16 @@ class QAPISchemaArrayType(QAPISchemaType): QAPISchemaType.check(self, schema) self.element_type = schema.lookup_type(self._element_type_name) assert self.element_type - self.element_type.check(schema) - self.module = self.element_type.module - self.ifcond = self.element_type.ifcond + + @property + def ifcond(self): + assert self._checked + return self.element_type.ifcond + + @property + def module(self): + assert self._checked + return self.element_type.module def is_implicit(self): return True
@@ -1402,13 +1415,20 @@ class QAPISchemaObjectType(QAPISchemaType): self.features = features def check(self, schema): - QAPISchemaType.check(self, schema) - if self.members is False: # check for cycles + # This calls another type T's .check() exactly when the C + # struct emitted by gen_object() contains that T's C struct + # (pointers don't count). + if self.members is not None: + # A previous .check() completed: nothing to do + return + if self._checked: + # Recursed: C struct contains itself raise QAPISemError(self.info, "Object %s contains itself" % self.name) - if self.members is not None: # already checked - return - self.members = False # mark as being checked + + QAPISchemaType.check(self, schema) + assert self._checked and self.members is None + seen = OrderedDict() if self._base_name: self.base = schema.lookup_type(self._base_name)
@@ -1420,10 +1440,11 @@ class QAPISchemaObjectType(QAPISchemaType): m.check_clash(self.info, seen) if self.doc: self.doc.connect_member(m) - self.members = seen.values() + members = seen.values() + if self.variants: self.variants.check(schema, seen) - assert self.variants.tag_member in self.members + assert self.variants.tag_member in members self.variants.check_clash(self.info, seen) # Features are in a name space separate from members
@@ -1434,6 +1455,8 @@ class QAPISchemaObjectType(QAPISchemaType): if self.doc: self.doc.check() + self.members = members # mark completed + # Check that the members of this type do not cause duplicate JSON members, # and update seen to track the members seen so far. Report any errors # on behalf of info, which is not necessarily self.info
@@ -1442,6 +1465,15 @@ class QAPISchemaObjectType(QAPISchemaType): for m in self.members: m.check_clash(info, seen) + @property + def ifcond(self): + assert self._checked + if isinstance(self._ifcond, QAPISchemaType): + # Simple union wrapper type inherits from wrapped type; + # see _make_implicit_object_type() + return self._ifcond.ifcond + return self._ifcond + def is_implicit(self): # See QAPISchema._make_implicit_object_type(), as well as # _def_predefineds()
@@ -1658,7 +1690,6 @@ class QAPISchemaCommand(QAPISchemaEntity): if self._arg_type_name: self.arg_type = schema.lookup_type(self._arg_type_name) assert isinstance(self.arg_type, QAPISchemaObjectType) - self.arg_type.check(schema) assert not self.arg_type.variants or self.boxed elif self.boxed: raise QAPISemError(self.info, "Use of 'boxed' requires 'data'")
@@ -1687,7 +1718,6 @@ class QAPISchemaEvent(QAPISchemaEntity): if self._arg_type_name: self.arg_type = schema.lookup_type(self._arg_type_name) assert isinstance(self.arg_type, QAPISchemaObjectType) - self.arg_type.check(schema) assert not self.arg_type.variants or self.boxed elif self.boxed: raise QAPISemError(self.info, "Use of 'boxed' requires 'data'")
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 18/19] qapi: Fix excessive QAPISchemaEntity.check() recursion
Quoted 21 lines · Markus Armbruster
Entity checking goes back to commit
ac88219a6c"qapi: New QAPISchema intermediate representation", v2.5.0. It's designed to work as
follows:QAPISchema.check()calls .check() for all the schema's entities. An entity's .check() recurses into another entity's .check() only if the C struct generated for the former contains the C struct generated for the latter (pointers don't count). This is used to detect "object contains itself".There are two instances of this:
* An object's C struct contains its base's C struct
QAPISchemaObjectType.check()callsself.base.check()* An object's C struct contains its variants' C structs
QAPISchemaObjectTypeVariants().check calls
v.type.check(). Since commitb807a1e1e3"qapi: Check for QAPI collisions involving variant members", v2.6.0.
Quoted 8 lines
Sadly, this design has since eroded:
* A QAPISchemaEntity's .module becomes valid at .check(). This is because computing it needs info and schema.fname, and because array types get it from their element type instead.
Make it a property just like .ifcond.
Extra space.
Quoted 4 lines
Additionally, have
QAPISchemaEntity.check()assert it gets called at most once, so the design invariant will stick this time. Neglecting that was entirely my fault.
Lengthy explanation, but it makes sense.
Quoted 6 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 74 +++++++++++++++++++++++++++++------------- 1 file changed, 52 insertions(+), 22 deletions(-)
Took me a bit longer to read this (and refresh myself on what '@property' actually does in Python, but you're not the first user of it in qemu), but it matches the commit message.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 18/19] qapi: Fix excessive QAPISchemaEntity.check() recursion
Quoted 25 lines · Eric Blake
Quoted 23 lines · Markus Armbruster
Entity checking goes back to commit
ac88219a6c"qapi: New QAPISchema intermediate representation", v2.5.0. It's designed to work as
follows:QAPISchema.check()calls .check() for all the schema's entities. An entity's .check() recurses into another entity's .check() only if the C struct generated for the former contains the C struct generated for the latter (pointers don't count). This is used to detect "object contains itself".There are two instances of this:
* An object's C struct contains its base's C struct
QAPISchemaObjectType.check()callsself.base.check()* An object's C struct contains its variants' C structs
QAPISchemaObjectTypeVariants().check calls
v.type.check(). Since commitb807a1e1e3"qapi: Check for QAPI collisions involving variant members", v2.6.0.Sadly, this design has since eroded:
Quoted 7 lines
Quoted 5 lines
* A QAPISchemaEntity's .module becomes valid at .check(). This is because computing it needs info and schema.fname, and because array types get it from their element type instead.
Make it a property just like .ifcond.
Extra space.
Will fix.
Quoted 5 lines
Quoted 3 lines
Additionally, have
QAPISchemaEntity.check()assert it gets called at most once, so the design invariant will stick this time. Neglecting that was entirely my fault.Lengthy explanation, but it makes sense.
Writing the commit message was more work than writing the patch...
Quoted 7 lines
Quoted 6 lines
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 74 +++++++++++++++++++++++++++++------------- 1 file changed, 52 insertions(+), 22 deletions(-)
Quoted 5 lines
Took me a bit longer to read this (and refresh myself on what '@property' actually does in Python, but you're not the first user of it in qemu), but it matches the commit message.
Reviewed-by Eric Blake <[email protected]>
Thanks!
PATCH 19/19 qapi: Assert .visit() and .check_clash() run only after .check()
In reply to Markus ArmbrusterEasy since the previous commit provides .checked.
--- scripts/qapi/common.py | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
scripts/qapi/common.py +10 -1
@@ -1190,7 +1190,7 @@ class QAPISchemaEntity(object): return not self.info def visit(self, visitor): - pass + assert self._checked class QAPISchemaVisitor(object):
@@ -1245,6 +1245,7 @@ class QAPISchemaInclude(QAPISchemaEntity): self.fname = fname def visit(self, visitor): + QAPISchemaEntity.visit(self, visitor) visitor.visit_include(self.fname, self.info)
@@ -1309,6 +1310,7 @@ class QAPISchemaBuiltinType(QAPISchemaType): return self.json_type() def visit(self, visitor): + QAPISchemaType.visit(self, visitor) visitor.visit_builtin_type(self.name, self.info, self.json_type())
@@ -1344,6 +1346,7 @@ class QAPISchemaEnumType(QAPISchemaType): return 'string' def visit(self, visitor): + QAPISchemaType.visit(self, visitor) visitor.visit_enum_type(self.name, self.info, self.ifcond, self.members, self.prefix)
@@ -1386,6 +1389,7 @@ class QAPISchemaArrayType(QAPISchemaType): return 'array of ' + elt_doc_type def visit(self, visitor): + QAPISchemaType.visit(self, visitor) visitor.visit_array_type(self.name, self.info, self.ifcond, self.element_type)
@@ -1461,6 +1465,7 @@ class QAPISchemaObjectType(QAPISchemaType): # and update seen to track the members seen so far. Report any errors # on behalf of info, which is not necessarily self.info def check_clash(self, info, seen): + assert self._checked assert not self.variants # not implemented for m in self.members: m.check_clash(info, seen)
@@ -1498,6 +1503,7 @@ class QAPISchemaObjectType(QAPISchemaType): return 'object' def visit(self, visitor): + QAPISchemaType.visit(self, visitor) visitor.visit_object_type(self.name, self.info, self.ifcond, self.base, self.local_members, self.variants, self.features)
@@ -1665,6 +1671,7 @@ class QAPISchemaAlternateType(QAPISchemaType): return 'value' def visit(self, visitor): + QAPISchemaType.visit(self, visitor) visitor.visit_alternate_type(self.name, self.info, self.ifcond, self.variants)
@@ -1698,6 +1705,7 @@ class QAPISchemaCommand(QAPISchemaEntity): assert isinstance(self.ret_type, QAPISchemaType) def visit(self, visitor): + QAPISchemaEntity.visit(self, visitor) visitor.visit_command(self.name, self.info, self.ifcond, self.arg_type, self.ret_type, self.gen, self.success_response,
@@ -1723,6 +1731,7 @@ class QAPISchemaEvent(QAPISchemaEntity): raise QAPISemError(self.info, "Use of 'boxed' requires 'data'") def visit(self, visitor): + QAPISchemaEntity.visit(self, visitor) visitor.visit_event(self.name, self.info, self.ifcond, self.arg_type, self.boxed)
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH 19/19] qapi: Assert .visit() and .check_clash() run only after .check()
Quoted 7 lines · Markus Armbruster
Easy since the previous commit provides .checked.
Signed-off-by Markus Armbruster <[email protected]>--- scripts/qapi/common.py | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org