qapi member collision, alternate layout (post-introspection cleanups, subset D)
Subsystems (2)
v12 cover qapi member collision, alternate layout (post-introspection cleanups, subset D)
No pending prerequisites; based on qemu.git master
Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv12d
and will soon be part of my branch with the rest of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi
v12 notes:
Delay 26-28 to a later subset, and instead add lots of new prereq patches that tackle some cleanups that make case-insensitive collision detection easier. Series is now slated for 2.6.
001/36:[----] [--] 'qapi: Track simple union tag in object.local_members' 002/36:[----] [--] 'qapi-types: Consolidate gen_struct() and gen_union()' 003/36:[----] [--] 'qapi-types: Simplify gen_struct_field[s]' 004/36:[----] [--] 'qapi: Drop obsolete tag value collision assertions' 005/36:[----] [--] 'qapi: Simplify QAPISchemaObjectTypeMember.check()' 006/36:[----] [--] 'qapi: Clean up after previous commit' 007/36:[----] [--] 'qapi: Fix up commit 7618b91's clash sanity checking change' 008/36:[----] [--] 'qapi: Eliminate QAPISchemaObjectType.check() variable members' 009/36:[----] [--] 'qapi: Factor out QAPISchemaObjectTypeMember.check_clash()' 010/36:[----] [--] 'qapi: Simplify QAPISchemaObjectTypeVariants.check()' 011/36:[down] 'qapi: Check for qapi collisions involving variant members' 012/36:[----] [--] 'qapi: Factor out QAPISchemaObjectType.check_clash()' 013/36:[----] [--] 'qapi: Hoist tag collision check to Variants.check()' 014/36:[----] [--] 'qapi: Remove outdated tests related to QMP/branch collisions' 015/36:[----] [--] 'qapi: Track owner of each object member' 016/36:[----] [--] 'qapi: Detect collisions in C member names' 017/36:[down] 'qapi: Fix c_name() munging'
018/36:[down] 'qapi: Remove dead visitor code'
019/36:[down] 'blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum' 020/36:[down] 'blkdebug: Avoid '.' in enum values'
021/36:[down] 'qapi: Tighten the regex on valid names'
022/36:[down] 'qapi: Don't let implicit enum MAX member collide' 023/36:[down] 'qapi: Remove dead tests for max collision' 024/36:[0018] [FC] 'cpu: Convert CpuInfo into flat union' 025/36:[down] 'qapi: Add alias for ErrorClass'
026/36:[0098] [FC] 'qapi: Change munging of CamelCase enum values' 027/36:[0049] [FC] 'qapi: Forbid case-insensitive clashes' 028/36:[down] 'qapi: Simplify QObject'
029/36:[down] 'qobject: Rename qtype_code to QType'
030/36:[down] 'qapi: Convert QType into qapi builtin enum type' 031/36:[0020] [FC] 'qapi: Simplify visiting of alternate types' 032/36:[down] 'qapi: Inline _make_implicit_tag()'
033/36:[0015] [FC] 'qapi: Fix alternates that accept 'number' but not 'int'' 034/36:[----] [--] 'qapi: Add positive tests to qapi-schema-test' 035/36:[0010] [FC] 'qapi: Simplify visits of optional fields' 036/36:[down] 'qapi: Shorter visits of optional fields'
v11 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg02563.html
First half of v10 subset C has been applied, so consolidate the last half of that along with all of subset D (which was at v9) into one group. Address list reviews, in particular, add a new patch 21 that makes alternate layouts a lot nicer by making qtype_code a builtin qapi type; and new patches 18-19 that try to reduce confusion on the use of camel_to_upper() in c_enum_const().
Probably too late to get these into 2.5, in which case 17/28 will need tweaks to call out 2.6.
v10 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg01249.html
Split several patches, redo the middle patches from Markus to be back in the order they were first posted, some fallout change to my patches due to the nicer pattern of minimizing conditionals inside .check(), by instead calling .check_clash() as needed. Change data->err magic in tests to instead use a new helper error_free_or_abort(). Add a patch that would prevent qapi case-insensitive clashes.
I am redoing my subset boundaries slightly: patches 23-27 of v9 (updating the alternate layout) will be delayed to subset D, and 2 other patches previously posted in subset D are now here (turning qapi clash checking into actual error messages), so the subject line of this cover letter is slightly different.
Hopefully, we are converging on something that will be ready for a pull request, especially for the earlier patches of this subset.
v9 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg00652.html
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg06999.html
More patches added, and several reorganized. Lots of new patches from Markus, although not in the order originally proposed.
The first 8 patches are fairly straightforward, and could probably be taken as-is. Patch 9 is a rewrite of v8 4/17, but in the opposite direction (document that no sorting is done, rather than attempting to sort), so it may need further fine-tuning. Patches 12-21 represents a fusion of Markus' and my attempts to rewrite v5 7/17 into a more-reviewable set of patches, and caused further churn later in the series.
Patch 23 still uses tag_member.type == None; I ran out of time to work on Markus' idea of providing an instance of QAPISchemaBuiltinType to fill the role for 'qtype_code' without being exposed through .json files and without breaking the invariant of a valid member.type after check(), and wanted to get the rest of the series started under revew. So I may need a followup patch or even a v10 of the later half of this series after exploring that idea more.
v8 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg06674.html
Minor changes when rebasing to latest, improved commit messages in a few places, plus the addition of a few new patches. Markus started reviewing v7, but hadn't gotten very far.
v7 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04112.html
Patches 1-3 of the previous round have moved to subset B; patch 7 is gone, and a couple of new patches are present in this round. The latest version of subset B made it much easier to reason about collision detection (namely, tag values can't collide with QMP values thanks to a named rather than anonymous union in the C type), and I like how things turned out. I suspect the hardest part of the review will be patches 5/14 and 7/14, although none of this has really had much review in any earlier versions.
v6 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg01980.html
Add some patches and rebase onto work on subset B. Rearrange some patches from v5 (this set includes 17-20, 23, 25-27). Backport diff gets a bit confused by one patch title changing.
Not much direct review comments, although some of the changes here are updated based on comments made on other patches in the v5 series.
Subset D (and more?) will come later.
In v5:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg05410.html
I _did_ rearrange patches to try and group related features:
1-2: Groundwork cleanups
3-5: Add more test cases
6-16: Front-end cleanups
17-18: Introspection output cleanups
19-20: 'alternate' type cleanups
21-29: qapi visitor cleanups
30-45: qapi-ify netdev_add
46: add qapi shorthand for flat unions
Lots of fixes based on additional testing, and rebased to track other changes that happened in the meantime. The series is huge; I can split off smaller portions as requested.
In v4:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02580.html
add some more clean up patches
rebase to Markus' recent work
pull in part of Zoltán's work to make netdev_add a flat union, further enhancing it to be introspectible
I might be able to rearrange some of these patches, or separate it into smaller independent series, if requested; but I'm posting now to get review started.
In v3:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02059.html
redo cleanup of dealloc of partial struct
add patches to make all visit_type_*() avoid leaks on failure add patches to allow boxed command arguments and events
In v2:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00900.html
rebase to Markus' v3 series
rework how comments are emitted for fields inherited from base additional patches added for deleting colliding 'void *data' documentation updates to match code changes
v1 was here:
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05266.html
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05325.html
Eric Blake (29):
qapi: Track simple union tag in object.local_members
qapi-types: Consolidate gen_struct() and gen_union()
qapi-types: Simplify gen_struct_field[s]
qapi: Check for qapi collisions involving variant members
qapi: Factor out QAPISchemaObjectType.check_clash()
qapi: Hoist tag collision check to Variants.check()
qapi: Remove outdated tests related to QMP/branch collisions
qapi: Track owner of each object member
qapi: Detect collisions in C member names
qapi: Fix c_name() munging
qapi: Remove dead visitor code
blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
blkdebug: Avoid '.' in enum values
qapi: Tighten the regex on valid names
qapi: Don't let implicit enum MAX member collide
qapi: Remove dead tests for max collision
cpu: Convert CpuInfo into flat union
qapi: Add alias for ErrorClass
qapi: Change munging of CamelCase enum values
qapi: Forbid case-insensitive clashes
qapi: Simplify QObject
qobject: Rename qtype_code to QType
qapi: Convert QType into qapi builtin enum type
qapi: Simplify visiting of alternate types
qapi: Inline _make_implicit_tag()
qapi: Fix alternates that accept 'number' but not 'int'
qapi: Add positive tests to qapi-schema-test
qapi: Simplify visits of optional fields
qapi: Shorter visits of optional fields
Markus Armbruster (7):
qapi: Drop obsolete tag value collision assertions
qapi: Simplify QAPISchemaObjectTypeMember.check()
qapi: Clean up after previous commit
qapi: Fix up commit 7618b91's clash sanity checking change
qapi: Eliminate QAPISchemaObjectType.check() variable members
qapi: Factor out QAPISchemaObjectTypeMember.check_clash()
qapi: Simplify QAPISchemaObjectTypeVariants.check()
block.c | 2 +-
block/blkdebug.c | 79 +------
block/parallels.c | 4 +-
block/qapi.c | 4 +-
block/qcow2.c | 2 +-
block/quorum.c | 2 +-
block/raw-posix.c | 2 +-
blockdev.c | 2 +-
cpus.c | 31 +--
docs/blkdebug.txt | 7 +-
docs/qapi-code-gen.txt | 34 +--
hmp.c | 44 ++--
hw/char/escc.c | 2 +-
hw/i386/pc_piix.c | 2 +-
hw/i386/pc_q35.c | 2 +-
hw/input/hid.c | 6 +-
hw/input/ps2.c | 6 +-
hw/input/virtio-input-hid.c | 12 +-
include/block/block.h | 62 +-----
include/block/block_int.h | 2 +-
include/hw/qdev-core.h | 2 +-
include/migration/migration.h | 4 +-
include/qapi/error.h | 14 ++
include/qapi/qmp/qobject.h | 53 ++---
include/qapi/visitor-impl.h | 8 +-
include/qapi/visitor.h | 22 +-
migration/migration.c | 4 +-
monitor.c | 18 +-
net/net.c | 4 +-
qapi-schema.json | 120 +++++++++--
qapi/block-core.json | 20 +-
qapi/common.json | 5 +-
qapi/opts-visitor.c | 2 +-
qapi/qapi-visit-core.c | 10 +-
qapi/qmp-dispatch.c | 2 +-
qapi/qmp-input-visitor.c | 10 +-
qapi/string-input-visitor.c | 3 +-
qemu-nbd.c | 2 +-
qmp-commands.hx | 4 +
qobject/qbool.c | 7 +-
qobject/qdict.c | 10 +-
qobject/qfloat.c | 7 +-
qobject/qint.c | 7 +-
qobject/qlist.c | 7 +-
qobject/qnull.c | 13 +-
qobject/qstring.c | 7 +-
replay/replay-input.c | 8 +-
scripts/qapi-types.py | 116 +++-------
scripts/qapi-visit.py | 35 +++-
scripts/qapi.py | 233 ++++++++++++++-------
tests/Makefile | 9 +-
tests/qapi-schema/alternate-empty.out | 3 +-
tests/qapi-schema/args-case-clash.err | 1 +
.../{union-max.exit => args-case-clash.exit} | 0
tests/qapi-schema/args-case-clash.json | 4 +
.../{union-max.out => args-case-clash.out} | 0
tests/qapi-schema/args-name-clash.err | 1 +
tests/qapi-schema/args-name-clash.exit | 2 +-
tests/qapi-schema/args-name-clash.json | 5 +-
tests/qapi-schema/args-name-clash.out | 6 -
tests/qapi-schema/command-type-case-clash.err | 1 +
...lash-type.exit => command-type-case-clash.exit} | 0
tests/qapi-schema/command-type-case-clash.json | 3 +
...-clash-type.out => command-type-case-clash.out} | 0
tests/qapi-schema/comments.out | 2 +
tests/qapi-schema/empty.out | 2 +
tests/qapi-schema/enum-max-member.err | 1 -
tests/qapi-schema/enum-max-member.exit | 1 -
tests/qapi-schema/enum-max-member.json | 3 -
tests/qapi-schema/enum-max-member.out | 0
tests/qapi-schema/event-case.out | 2 +
tests/qapi-schema/event-max.err | 1 -
tests/qapi-schema/event-max.exit | 1 -
tests/qapi-schema/event-max.json | 2 -
tests/qapi-schema/event-max.out | 0
tests/qapi-schema/flat-union-clash-branch.err | 0
tests/qapi-schema/flat-union-clash-branch.exit | 1 -
tests/qapi-schema/flat-union-clash-branch.json | 18 --
tests/qapi-schema/flat-union-clash-branch.out | 14 --
tests/qapi-schema/flat-union-clash-type.err | 1 -
tests/qapi-schema/flat-union-clash-type.json | 14 --
tests/qapi-schema/flat-union-empty.out | 2 +
tests/qapi-schema/ident-with-escape.out | 2 +
tests/qapi-schema/include-relpath.out | 2 +
tests/qapi-schema/include-repetition.out | 2 +
tests/qapi-schema/include-simple.out | 2 +
tests/qapi-schema/indented-expr.out | 2 +
tests/qapi-schema/qapi-schema-test.json | 21 +-
tests/qapi-schema/qapi-schema-test.out | 39 +++-
tests/qapi-schema/reserved-member-underscore.err | 1 +
...h-type.exit => reserved-member-underscore.exit} | 0
tests/qapi-schema/reserved-member-underscore.json | 4 +
...ash-type.out => reserved-member-underscore.out} | 0
tests/qapi-schema/union-clash-data.out | 3 +
tests/qapi-schema/union-clash-type.err | 1 -
tests/qapi-schema/union-clash-type.json | 9 -
tests/qapi-schema/union-empty.out | 3 +
tests/qapi-schema/union-max.err | 1 -
tests/qapi-schema/union-max.json | 3 -
tests/qemu-iotests/026 | 18 +-
tests/qemu-iotests/026.out | 80 +++----
tests/qemu-iotests/026.out.nocache | 80 +++----
tests/qemu-iotests/077 | 24 +--
tests/test-qmp-commands.c | 4 +
tests/test-qmp-input-visitor.c | 29 ++-
tests/test-qmp-output-visitor.c | 10 +-
tests/test-string-output-visitor.c | 4 +-
tpm.c | 10 +-
ui/cocoa.m | 6 +-
ui/console.c | 2 +-
ui/gtk.c | 4 +-
ui/input-keymap.c | 4 +-
ui/input-legacy.c | 10 +-
ui/input.c | 6 +-
ui/sdl.c | 6 +-
ui/sdl2.c | 6 +-
ui/spice-input.c | 6 +-
ui/vnc.c | 6 +-
vl.c | 18 +-
119 files changed, 806 insertions(+), 786 deletions(-)
create mode 100644 tests/qapi-schema/args-case-clash.err
rename tests/qapi-schema/{union-max.exit => args-case-clash.exit} (100%)
create mode 100644 tests/qapi-schema/args-case-clash.json
rename tests/qapi-schema/{union-max.out => args-case-clash.out} (100%)
create mode 100644 tests/qapi-schema/command-type-case-clash.err
rename tests/qapi-schema/{union-clash-type.exit => command-type-case-clash.exit} (100%)
create mode 100644 tests/qapi-schema/command-type-case-clash.json
rename tests/qapi-schema/{union-clash-type.out => command-type-case-clash.out} (100%)
delete mode 100644 tests/qapi-schema/enum-max-member.err
delete mode 100644 tests/qapi-schema/enum-max-member.exit
delete mode 100644 tests/qapi-schema/enum-max-member.json
delete mode 100644 tests/qapi-schema/enum-max-member.out
delete mode 100644 tests/qapi-schema/event-max.err
delete mode 100644 tests/qapi-schema/event-max.exit
delete mode 100644 tests/qapi-schema/event-max.json
delete mode 100644 tests/qapi-schema/event-max.out
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.err
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.json
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.out
delete mode 100644 tests/qapi-schema/flat-union-clash-type.err
delete mode 100644 tests/qapi-schema/flat-union-clash-type.json
create mode 100644 tests/qapi-schema/reserved-member-underscore.err
rename tests/qapi-schema/{flat-union-clash-type.exit => reserved-member-underscore.exit} (100%)
create mode 100644 tests/qapi-schema/reserved-member-underscore.json
rename tests/qapi-schema/{flat-union-clash-type.out => reserved-member-underscore.out} (100%)
delete mode 100644 tests/qapi-schema/union-clash-type.err
delete mode 100644 tests/qapi-schema/union-clash-type.json
delete mode 100644 tests/qapi-schema/union-max.err
delete mode 100644 tests/qapi-schema/union-max.jsonSignature · 1 line
2.4.3
PATCH v12 1/36 qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for local_members. However, it will make it easier to unify struct and union generation if we include the generated tag member in local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit the variants (if any). The local_members of a flat union remains empty (because the discriminator is already visited as part of the base). Then, by visiting tag_member.check() during AlternateType.check(), we no longer need to call it during Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to avoid assertions in the clients.
No change to generated code.
---
v12: no change
v11: drop comment churn
v10: no change
v9: improve commit message, add comments, tweak how alternates check tag_member
v8: new patch
--- scripts/qapi-types.py | 5 ++++- scripts/qapi-visit.py | 5 ++++- scripts/qapi.py | 15 ++++++++++----- tests/qapi-schema/qapi-schema-test.out | 2 ++ tests/qapi-schema/union-clash-data.out | 1 + tests/qapi-schema/union-empty.out | 1 + 6 files changed, 22 insertions(+), 7 deletions(-)
scripts/qapi-types.py +4 -1
@@ -269,7 +269,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def visit_object_type(self, name, info, base, members, variants): self._fwdecl += gen_fwd_object_or_array(name) if variants: - assert not members # not implemented + if members: + # Members other than variants.tag_member not implemented + assert len(members) == 1 + assert members[0] == variants.tag_member self.decl += gen_union(name, base, variants) else: self.decl += gen_struct(name, base, members)
scripts/qapi-visit.py +4 -1
@@ -364,7 +364,10 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): def visit_object_type(self, name, info, base, members, variants): self.decl += gen_visit_decl(name) if variants: - assert not members # not implemented + if members: + # Members other than variants.tag_member not implemented + assert len(members) == 1 + assert members[0] == variants.tag_member self.defn += gen_visit_union(name, base, variants) else: self.defn += gen_visit_struct(name, base, members)
scripts/qapi.py +10 -5
@@ -957,6 +957,9 @@ class QAPISchemaArrayType(QAPISchemaType): class QAPISchemaObjectType(QAPISchemaType): def __init__(self, name, info, base, local_members, variants): + # struct has local_members, optional base, and no variants + # flat union has base, variants, and no local_members + # simple union has local_members, variants, and no base QAPISchemaType.__init__(self, name, info) assert base is None or isinstance(base, str) for m in local_members:
@@ -1048,10 +1051,10 @@ class QAPISchemaObjectTypeVariants(object): self.variants = variants def check(self, schema, members, seen): - if self.tag_name: + if self.tag_name: # flat union self.tag_member = seen[self.tag_name] - else: - self.tag_member.check(schema, members, seen) + if seen: + assert self.tag_member in seen.itervalues() assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: vseen = dict(seen)
@@ -1085,6 +1088,7 @@ class QAPISchemaAlternateType(QAPISchemaType): self.variants = variants def check(self, schema): + self.variants.tag_member.check(schema, [], {}) self.variants.check(schema, [], {}) def json_type(self):
@@ -1270,13 +1274,14 @@ class QAPISchema(object): if tag_name: variants = [self._make_variant(key, value) for (key, value) in data.iteritems()] + members = [] else: variants = [self._make_simple_variant(key, value, info) for (key, value) in data.iteritems()] tag_member = self._make_implicit_tag(name, info, variants) + members = [tag_member] self._def_entity( - QAPISchemaObjectType(name, info, base, - self._make_members(OrderedDict(), info), + QAPISchemaObjectType(name, info, base, members, QAPISchemaObjectTypeVariants(tag_name, tag_member, variants)))
tests/qapi-schema/qapi-schema-test.out +2 -0
@@ -132,6 +132,7 @@ object UserDefFlatUnion2 case value2: UserDefB case value3: UserDefA object UserDefNativeListUnion + member type: UserDefNativeListUnionKind optional=False case integer: :obj-intList-wrapper case s8: :obj-int8List-wrapper case s16: :obj-int16List-wrapper
@@ -187,6 +188,7 @@ object __org.qemu_x-Struct object __org.qemu_x-Struct2 member array: __org.qemu_x-Union1List optional=False object __org.qemu_x-Union1 + member type: __org.qemu_x-Union1Kind optional=False case __org.qemu_x-branch: :obj-str-wrapper enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch'] object __org.qemu_x-Union2
tests/qapi-schema/union-clash-data.out +1 -0
@@ -2,5 +2,6 @@ object :empty object :obj-int-wrapper member data: int optional=False object TestUnion + member type: TestUnionKind optional=False case data: :obj-int-wrapper enum TestUnionKind ['data']
tests/qapi-schema/union-empty.out +1 -0
@@ -1,3 +1,4 @@ object :empty object Union + member type: UnionKind optional=False enum UnionKind []
Signature · 1 line
2.4.3
PATCH v12 2/36 qapi-types: Consolidate gen_struct() and gen_union()
In reply to Eric BlakeThese two methods are now close enough that we can finally merge them, relying on the fact that simple unions now provide a reasonable local_members. Change gen_struct() to gen_object() that handles all forms of QAPISchemaObjectType, and rename and shrink gen_union() to gen_variants() to handle the portion of gen_object() needed when variants are present.
gen_struct_fields() now has a single caller, so it no longer needs an optional parameter; however, I did not choose to inline it into the caller.
No difference to generated code.
---
v12: no change
v11: no change
v10: no change
v9: rebase to earlier changes
v8: new patch
--- scripts/qapi-types.py | 37 +++++++++++-------------------------- 1 file changed, 11 insertions(+), 26 deletions(-)
scripts/qapi-types.py +11 -26
@@ -51,7 +51,7 @@ def gen_struct_field(member): return ret -def gen_struct_fields(local_members, base=None): +def gen_struct_fields(local_members, base): ret = '' if base:
@@ -70,7 +70,7 @@ def gen_struct_fields(local_members, base=None): return ret -def gen_struct(name, base, members): +def gen_object(name, base, members, variants): ret = mcgen(''' struct %(c_name)s {
@@ -79,11 +79,14 @@ struct %(c_name)s { ret += gen_struct_fields(members, base) + if variants: + ret += gen_variants(variants) + # Make sure that all structs have at least one field; this avoids # potential issues with attempting to malloc space for zero-length # structs in C, and also incompatibility with C++ (where an empty # struct is size 1). - if not (base and base.members) and not members: + if not (base and base.members) and not members and not variants: ret += mcgen(''' char qapi_dummy_field_for_empty_struct; ''')
@@ -140,17 +143,7 @@ const int %(c_name)s_qtypes[QTYPE_MAX] = { return ret -def gen_union(name, base, variants): - ret = mcgen(''' - -struct %(c_name)s { -''', - c_name=c_name(name)) - if base: - ret += gen_struct_fields([], base) - else: - ret += gen_struct_field(variants.tag_member) - +def gen_variants(variants): # FIXME: What purpose does data serve, besides preventing a union that # has a branch named 'data'? We use it in qapi-visit.py to decide # whether to bypass the switch statement if visiting the discriminator
@@ -159,11 +152,11 @@ struct %(c_name)s { # should not be any data leaks even without a data pointer. Or, if # 'data' is merely added to guarantee we don't have an empty union, # shouldn't we enforce that at .json parse time? - ret += mcgen(''' + ret = mcgen(''' union { /* union tag is @%(c_name)s */ void *data; ''', - c_name=c_name(variants.tag_member.name)) + c_name=c_name(variants.tag_member.name)) for var in variants.variants: # Ugly special case for simple union TODO get rid of it
@@ -176,7 +169,6 @@ struct %(c_name)s { ret += mcgen(''' } u; -}; ''') return ret
@@ -268,14 +260,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def visit_object_type(self, name, info, base, members, variants): self._fwdecl += gen_fwd_object_or_array(name) - if variants: - if members: - # Members other than variants.tag_member not implemented - assert len(members) == 1 - assert members[0] == variants.tag_member - self.decl += gen_union(name, base, variants) - else: - self.decl += gen_struct(name, base, members) + self.decl += gen_object(name, base, members, variants) if base: self.decl += gen_upcast(name, base) self._gen_type_cleanup(name)
@@ -283,7 +268,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def visit_alternate_type(self, name, info, variants): self._fwdecl += gen_fwd_object_or_array(name) self._fwdefn += gen_alternate_qtypes(name, variants) - self.decl += gen_union(name, None, variants) + self.decl += gen_object(name, None, [variants.tag_member], variants) self.decl += gen_alternate_qtypes_decl(name) self._gen_type_cleanup(name)
Signature · 1 line
2.4.3
PATCH v12 3/36 qapi-types: Simplify gen_struct_field[s]
In reply to Eric BlakeSimplify gen_struct_fields() back to a single iteration over a list of fields (like it was prior to commit f87ab7f9), by moving the generated comments to gen_object(). Then, inline gen_struct_field() into its only caller.
---
v12: no change
v11: no change
v10: no change
v9: new patch
--- scripts/qapi-types.py | 40 +++++++++++++++------------------------- 1 file changed, 15 insertions(+), 25 deletions(-)
scripts/qapi-types.py +15 -25
@@ -36,48 +36,38 @@ struct %(c_name)s { c_name=c_name(name), c_type=element_type.c_type()) -def gen_struct_field(member): +def gen_struct_fields(members): ret = '' - - if member.optional: - ret += mcgen(''' + for memb in members: + if memb.optional: + ret += mcgen(''' bool has_%(c_name)s; ''', - c_name=c_name(member.name)) - ret += mcgen(''' + c_name=c_name(memb.name)) + ret += mcgen(''' %(c_type)s %(c_name)s; ''', - c_type=member.type.c_type(), c_name=c_name(member.name)) + c_type=memb.type.c_type(), c_name=c_name(memb.name)) return ret -def gen_struct_fields(local_members, base): - ret = '' +def gen_object(name, base, members, variants): + ret = mcgen(''' + +struct %(c_name)s { +''', + c_name=c_name(name)) if base: ret += mcgen(''' /* Members inherited from %(c_name)s: */ ''', c_name=base.c_name()) - for memb in base.members: - ret += gen_struct_field(memb) + ret += gen_struct_fields(base.members) ret += mcgen(''' /* Own members: */ ''') - - for memb in local_members: - ret += gen_struct_field(memb) - return ret - - -def gen_object(name, base, members, variants): - ret = mcgen(''' - -struct %(c_name)s { -''', - c_name=c_name(name)) - - ret += gen_struct_fields(members, base) + ret += gen_struct_fields(members) if variants: ret += gen_variants(variants)
Signature · 1 line
2.4.3
PATCH v12 4/36 qapi: Drop obsolete tag value collision assertions
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
Union tag values can't clash with member names in generated C anymore since commit e4ba22b, but QAPISchemaObjectTypeVariants.check() still asserts they don't. Drop it.
Message-Id: <[email protected]>
---
v12: no change
v11: no change
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
scripts/qapi.py +1 -2
@@ -1057,8 +1057,7 @@ class QAPISchemaObjectTypeVariants(object): assert self.tag_member in seen.itervalues() assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: - vseen = dict(seen) - v.check(schema, self.tag_member.type, vseen) + v.check(schema, self.tag_member.type, {}) class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
Signature · 1 line
2.4.3
PATCH v12 5/36 qapi: Simplify QAPISchemaObjectTypeMember.check()
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
QAPISchemaObjectTypeMember.check() currently does four things:
1. Compute self.type
2. Accumulate members in all_members
Only one caller cares: QAPISchemaObjectType.check() uses it to compute self.members. The other callers pass a throw-away accumulator.
3. Accumulate a map from names to members in seen
Only one caller cares: QAPISchemaObjectType.check() uses it to compute its local variable seen, for self.variants.check(), which uses it to compute self.variants.tag_member from self.variants.tag_name. The other callers pass a throw-away accumulator.
4. Check for collisions
This piggybacks on 3: before adding a new entry, we assert it's new.
Only one caller cares: QAPISchemaObjectType.check() uses it to assert non-variant members don't clash.
Simplify QAPISchemaObjectType.check(): move 2.-4. to QAPISchemaObjectType.check(), and drop parameters all_members and seen.
Message-Id: <[email protected]> [rebase to earlier changes that moved tag_member.check() of alternate types, commit message typo fix]
---
v12: no change
v11: typo fix
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
scripts/qapi.py +7 -7
@@ -990,7 +990,10 @@ class QAPISchemaObjectType(QAPISchemaType): assert c_name(m.name) not in seen seen[m.name] = m for m in self.local_members: - m.check(schema, members, seen) + m.check(schema) + assert m.name not in seen + seen[m.name] = m + members.append(m) if self.variants: self.variants.check(schema, members, seen) self.members = members
@@ -1027,12 +1030,9 @@ class QAPISchemaObjectTypeMember(object): self.type = None self.optional = optional - def check(self, schema, all_members, seen): - assert self.name not in seen + def check(self, schema): self.type = schema.lookup_type(self._type_name) assert self.type - all_members.append(self) - seen[self.name] = self class QAPISchemaObjectTypeVariants(object):
@@ -1065,7 +1065,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember): QAPISchemaObjectTypeMember.__init__(self, name, typ, False) def check(self, schema, tag_type, seen): - QAPISchemaObjectTypeMember.check(self, schema, [], seen) + QAPISchemaObjectTypeMember.check(self, schema) assert self.name in tag_type.values # This function exists to support ugly simple union special cases
@@ -1087,7 +1087,7 @@ class QAPISchemaAlternateType(QAPISchemaType): self.variants = variants def check(self, schema): - self.variants.tag_member.check(schema, [], {}) + self.variants.tag_member.check(schema) self.variants.check(schema, [], {}) def json_type(self):
Signature · 1 line
2.4.3
PATCH v12 6/36 qapi: Clean up after previous commit
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
QAPISchemaObjectTypeVariants.check() parameter members and QAPISchemaObjectTypeVariant.check() parameter seen are no longer used, drop them.
Message-Id: <[email protected]> [rebase to earlier changes that moved tag_member.check() of alternate types]
---
v12: fix git authorship
v11: rebase for less comment churn
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
scripts/qapi.py +5 -5
@@ -995,7 +995,7 @@ class QAPISchemaObjectType(QAPISchemaType): seen[m.name] = m members.append(m) if self.variants: - self.variants.check(schema, members, seen) + self.variants.check(schema, seen) self.members = members def is_implicit(self):
@@ -1050,21 +1050,21 @@ class QAPISchemaObjectTypeVariants(object): self.tag_member = tag_member self.variants = variants - def check(self, schema, members, seen): + def check(self, schema, seen): if self.tag_name: # flat union self.tag_member = seen[self.tag_name] if seen: assert self.tag_member in seen.itervalues() assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: - v.check(schema, self.tag_member.type, {}) + v.check(schema, self.tag_member.type) class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember): def __init__(self, name, typ): QAPISchemaObjectTypeMember.__init__(self, name, typ, False) - def check(self, schema, tag_type, seen): + def check(self, schema, tag_type): QAPISchemaObjectTypeMember.check(self, schema) assert self.name in tag_type.values
@@ -1088,7 +1088,7 @@ class QAPISchemaAlternateType(QAPISchemaType): def check(self, schema): self.variants.tag_member.check(schema) - self.variants.check(schema, [], {}) + self.variants.check(schema, {}) def json_type(self): return 'value'
Signature · 1 line
2.4.3
PATCH v12 7/36 qapi: Fix up commit 7618b91's clash sanity checking change
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
This hunk
@@ -964,6 +965,7 @@ class QAPISchemaObjectType(QAPISchemaType):
members = []
seen = {}
for m in members:
+ assert c_name(m.name) not in seen
seen[m.name] = m
for m in self.local_members:
m.check(schema, members, seen)
is plainly broken.
Asserting the members inherited from base don't clash is somewhat redundant, because self.base.check() just checked that. But it doesn't hurt.
The idea to use c_name(m.name) instead of m.name for collision checking is sound, because we need to catch clashes between the m.name and between the c_name(m.name), and when two m.name clash, then their c_name() also clash.
However, using c_name(m.name) instead of m.name in one of several places doesn't work. See the very next line.
Keep the assertion, but drop the c_name() for now. A future commit will bring it back.
Message-Id: <[email protected]>
---
v12: no change
v11: no change
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/qapi.py +1 -1
@@ -987,7 +987,7 @@ class QAPISchemaObjectType(QAPISchemaType): members = [] seen = {} for m in members: - assert c_name(m.name) not in seen + assert m.name not in seen seen[m.name] = m for m in self.local_members: m.check(schema)
Signature · 1 line
2.4.3
PATCH v12 8/36 qapi: Eliminate QAPISchemaObjectType.check() variable members
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
We can use seen.values() instead if we make it an OrderedDict.
Message-Id: <[email protected]>
---
v12: no change
v11: no change
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
scripts/qapi.py +5 -9
@@ -977,26 +977,22 @@ class QAPISchemaObjectType(QAPISchemaType): if self.members: return self.members = False # mark as being checked + seen = OrderedDict() if self._base_name: self.base = schema.lookup_type(self._base_name) assert isinstance(self.base, QAPISchemaObjectType) assert not self.base.variants # not implemented self.base.check(schema) - members = list(self.base.members) - else: - members = [] - seen = {} - for m in members: - assert m.name not in seen - seen[m.name] = m + for m in self.base.members: + assert m.name not in seen + seen[m.name] = m for m in self.local_members: m.check(schema) assert m.name not in seen seen[m.name] = m - members.append(m) if self.variants: self.variants.check(schema, seen) - self.members = members + self.members = seen.values() def is_implicit(self): # See QAPISchema._make_implicit_object_type()
Signature · 1 line
2.4.3
PATCH v12 9/36 qapi: Factor out QAPISchemaObjectTypeMember.check_clash()
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
While there, stick in a TODO change key of seen from QAPI name to C name. Can't do it right away, because it would fail the assertion for tests/qapi-schema/args-has-clash.json.
Message-Id: <[email protected]>
---
v12: no change
v11: no change
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
scripts/qapi.py +7 -4
@@ -984,12 +984,10 @@ class QAPISchemaObjectType(QAPISchemaType): assert not self.base.variants # not implemented self.base.check(schema) for m in self.base.members: - assert m.name not in seen - seen[m.name] = m + m.check_clash(seen) for m in self.local_members: m.check(schema) - assert m.name not in seen - seen[m.name] = m + m.check_clash(seen) if self.variants: self.variants.check(schema, seen) self.members = seen.values()
@@ -1030,6 +1028,11 @@ class QAPISchemaObjectTypeMember(object): self.type = schema.lookup_type(self._type_name) assert self.type + def check_clash(self, seen): + # TODO change key of seen from QAPI name to C name + assert self.name not in seen + seen[self.name] = self + class QAPISchemaObjectTypeVariants(object): def __init__(self, tag_name, tag_member, variants):
Signature · 1 line
2.4.3
PATCH v12 10/36 qapi: Simplify QAPISchemaObjectTypeVariants.check()
In reply to Eric BlakeFrom: Markus Armbruster <[email protected]>
Reduce the ugly flat union / simple union conditional by doing just the essential work here, namely setting self.tag_member. Move the rest to callers.
Message-Id: <[email protected]> [rebase to earlier changes that moved tag_member.check() of alternate types, and tweak commit title and wording]
---
v12: no change
v11: don't drop change of 'if not self.tag_member'
v10: redo closer to Markus' original proposal
v9: new patch
--- scripts/qapi.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
scripts/qapi.py +3 -4
@@ -988,9 +988,10 @@ class QAPISchemaObjectType(QAPISchemaType): for m in self.local_members: m.check(schema) m.check_clash(seen) + self.members = seen.values() if self.variants: self.variants.check(schema, seen) - self.members = seen.values() + assert self.variants.tag_member in self.members def is_implicit(self): # See QAPISchema._make_implicit_object_type()
@@ -1050,10 +1051,8 @@ class QAPISchemaObjectTypeVariants(object): self.variants = variants def check(self, schema, seen): - if self.tag_name: # flat union + if not self.tag_member: # flat union self.tag_member = seen[self.tag_name] - if seen: - assert self.tag_member in seen.itervalues() assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: v.check(schema, self.tag_member.type)
Signature · 1 line
2.4.3
PATCH v12 11/36 qapi: Check for qapi collisions involving variant members
In reply to Eric BlakeRight now, our ad hoc parser ensures that we cannot have a flat union that introduces any members that would clash with non-variant members inherited from the union's base type (see flat-union-clash-member.json). We want ObjectType.check() to make the same check, so we can later reduce some of the ad hoc checks.
We already have a map 'seen' of all non-variant members. We still need to check for collisions between each variant type's members and the non-variant ones.
To know the variant type's members, we need to call variant.type.check(). This also detects when a type contains itself in a variant, exactly like the existing base.check() detects when a type contains itself as a base. (Except that we currently forbid anything but a struct as the type of a variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary type, but only an object type's check() may be called outside QAPISchema.check(). We could either skip the call for variants of alternates, or skip it for non-object types. For now, do the latter, because it's easier.
Then we call each variant member's check_clash() with the appropriate 'seen' map. Since members of different variants can't clash, we have to clone a fresh seen for each variant. Wrap this in a new helper method Variants.check_clash().
Note that cloning 'seen' inside Variants.check_clash() resembles the one we just removed from Variants.check() in 'qapi: Drop obsolete tag value collision assertions'; the difference here is that we are now checking for clashes among the qapi members of the variant type, rather than for a single clash with the variant tag name itself.
Note that, by construction, collisions can't actually happen for simple unions: each variant's type is a wrapper with a single member 'data', which will never collide with the only non-variant member 'type'.
For alternates, there's nothing for a variant object type's members to clash with, and therefore no need to call the new variants.check_clash().
No change to generated code.
---
v12: retitle; use Markus' improved commit wording
v11: keep type.check() in check(), add comment to alternate
v10: create new Variants.check_clash() rather than piggybacking on .check()
v9: new patch, split off from v8 7/17
--- scripts/qapi.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
scripts/qapi.py +15 -0
@@ -992,6 +992,7 @@ class QAPISchemaObjectType(QAPISchemaType): if self.variants: self.variants.check(schema, seen) assert self.variants.tag_member in self.members + self.variants.check_clash(schema, seen) def is_implicit(self): # See QAPISchema._make_implicit_object_type()
@@ -1056,6 +1057,18 @@ class QAPISchemaObjectTypeVariants(object): assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: v.check(schema, self.tag_member.type) + if isinstance(v.type, QAPISchemaObjectType): + v.type.check(schema) + + def check_clash(self, schema, seen): + for v in self.variants: + # Reset seen map for each variant, since qapi names from one + # branch do not affect another branch + vseen = dict(seen) + assert isinstance(v.type, QAPISchemaObjectType) + assert not v.type.variants # not implemented + for m in v.type.members: + m.check_clash(vseen) class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
@@ -1086,6 +1099,8 @@ class QAPISchemaAlternateType(QAPISchemaType): def check(self, schema): self.variants.tag_member.check(schema) + # Not calling self.variants.check_clash(), because there's nothing + # to clash with self.variants.check(schema, {}) def json_type(self):
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 11/36] qapi: Check for qapi collisions involving variant members
Quoted 4 lines · Eric Blake
Right now, our ad hoc parser ensures that we cannot have a flat union that introduces any members that would clash with non-variant members inherited from the union's base type (see flat-union-clash-member.json). We want
ObjectType.check()to
Recommend not to abbreviate QAPISchemaObjectType.check().
Quoted 20 lines
make the same check, so we can later reduce some of the ad hoc checks.
We already have a map 'seen' of all non-variant members. We still need to check for collisions between each variant type's members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains itself in a variant, exactly like the existingbase.check()detects when a type contains itself as a base. (Except that we currently forbid anything but a struct as the type of a variant, so we can't actually trigger this type of loop yet.)Slight complication: an alternate's variant can have arbitrary type, but only an object type's
check()may be called outsideQAPISchema.check(). We could either skip the call for variants of alternates, or skip it for non-object types. For now, do the latter, because it's easier.
Quoted 4 lines
Then we call each variant member's
check_clash()with the appropriate 'seen' map. Since members of different variants can't clash, we have to clone a fresh seen for each variant. Wrap this in a new helper methodVariants.check_clash().
Likewise: QAPISchemaObjectTypeVariants.check_clash().
Quoted 2 lines
Note that cloning 'seen' inside
Variants.check_clash()resembles the one we just removed fromVariants.check()in 'qapi: Drop
Here, we can say .check_clash() and .check(), and leave the class to context.
Quoted 10 lines
obsolete tag value collision assertions'; the difference here is that we are now checking for clashes among the qapi members of the variant type, rather than for a single clash with the variant tag name itself.
Note that, by construction, collisions can't actually happen for simple unions: each variant's type is a wrapper with a single member 'data', which will never collide with the only non-variant member 'type'.
Quoted 7 lines
For alternates, there's nothing for a variant object type's members to clash with, and therefore no need to call the new
variants.check_clash().No change to generated code.
Signed-off-by Eric Blake <[email protected]>
Only commit message polishing, happy to do that in my tree.
PATCH v12 12/36 qapi: Factor out QAPISchemaObjectType.check_clash()
In reply to Eric BlakeConsolidate two common sequences of clash detection into a new QAPISchemaObjectType.check_clash() helper method.
No change to generated code.
---
v12: no change
v11: don't lose isinstance check, and don't hide type.check() inside check_clash()
v10: rebase on new Variants.check_clash()
v9: new patch, split off from v8 7/17
--- scripts/qapi.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
scripts/qapi.py +7 -7
@@ -981,10 +981,8 @@ class QAPISchemaObjectType(QAPISchemaType): if self._base_name: self.base = schema.lookup_type(self._base_name) assert isinstance(self.base, QAPISchemaObjectType) - assert not self.base.variants # not implemented self.base.check(schema) - for m in self.base.members: - m.check_clash(seen) + self.base.check_clash(schema, seen) for m in self.local_members: m.check(schema) m.check_clash(seen)
@@ -994,6 +992,11 @@ class QAPISchemaObjectType(QAPISchemaType): assert self.variants.tag_member in self.members self.variants.check_clash(schema, seen) + def check_clash(self, schema, seen): + assert not self.variants # not implemented + for m in self.members: + m.check_clash(seen) + def is_implicit(self): # See QAPISchema._make_implicit_object_type() return self.name[0] == ':'
@@ -1064,11 +1067,8 @@ class QAPISchemaObjectTypeVariants(object): for v in self.variants: # Reset seen map for each variant, since qapi names from one # branch do not affect another branch - vseen = dict(seen) assert isinstance(v.type, QAPISchemaObjectType) - assert not v.type.variants # not implemented - for m in v.type.members: - m.check_clash(vseen) + v.type.check_clash(schema, dict(seen)) class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
Signature · 1 line
2.4.3
PATCH v12 13/36 qapi: Hoist tag collision check to Variants.check()
In reply to Eric BlakeChecking that a given QAPISchemaObjectTypeVariant.name is a member of the corresponding QAPISchemaEnumType of the owning QAPISchemaObjectTypeVariants.tag_member ensures that there are no collisions in the generated C union for those tag values (since the enum itself should have no collisions).
However, ever since its introduction in f51d8c3d, this was the only additional action of of Variant.check(), beyond calling the superclass Member.check(). This forces a difference in .check() signatures, just to pass the enum type down.
Simplify things by instead doing the tag name check as part of Variants.check(), at which point we can rely on inheritance instead of overriding Variant.check().
---
v12: improve commit message
v11: don't use tag_type local variable, rebase to v.type.check()
v10: new patch
--- scripts/qapi.py | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
scripts/qapi.py +2 -5
@@ -1059,7 +1059,8 @@ class QAPISchemaObjectTypeVariants(object): self.tag_member = seen[self.tag_name] assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: - v.check(schema, self.tag_member.type) + v.check(schema) + assert v.name in self.tag_member.type.values if isinstance(v.type, QAPISchemaObjectType): v.type.check(schema)
@@ -1075,10 +1076,6 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember): def __init__(self, name, typ): QAPISchemaObjectTypeMember.__init__(self, name, typ, False) - def check(self, schema, tag_type): - QAPISchemaObjectTypeMember.check(self, schema) - assert self.name in tag_type.values - # This function exists to support ugly simple union special cases # TODO get rid of them, and drop the function def simple_union_type(self):
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 13/36] qapi: Hoist tag collision check to Variants.check()
Quoted 4 lines · Eric Blake
*** WARNING: THE ATTACHED DOCUMENT(S) CONTAIN MACROS ***
*** MACROS MAY CONTAIN MALICIOUS CODE ***
*** Open only if you can verify and trust the sender ***
*** Please contact [email protected] if you have questions or concerns **
Looks like infosec crapped over your commit message.
Quoted 11 lines
Checking that a given QAPISchemaObjectTypeVariant.name is a member of the corresponding QAPISchemaEnumType of the owning QAPISchemaObjectTypeVariants.tag_member ensures that there are no collisions in the generated C union for those tag values (since the enum itself should have no collisions).
However, ever since its introduction in
f51d8c3d, this was the only additional action of ofVariant.check(), beyond calling the superclassMember.check(). This forces a difference in .check() signatures, just to pass the enum type down.
Quoted 5 lines
Simplify things by instead doing the tag name check as part of
Variants.check(), at which point we can rely on inheritance instead of overridingVariant.check().Signed-off-by Eric Blake <[email protected]>
Can bury the infosec turd on merge.
Re: [Qemu-devel] [PATCH v12 13/36] qapi: Hoist tag collision check to Variants.check()
Quoted 8 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
*** WARNING: THE ATTACHED DOCUMENT(S) CONTAIN MACROS ***
*** MACROS MAY CONTAIN MALICIOUS CODE ***
*** Open only if you can verify and trust the sender ***
*** Please contact [email protected] if you have questions or concerns **Looks like infosec crapped over your commit message.
I don't see that text in my copy of Eric's mail, nor in the archives
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg04028.html
Wierd that it would get added to the mail you receive, but not to the mail I receive, given that we have the same incoming mail server. I guess the difference was you got CC'd directly but I got it via the mailing list, so something about that delivery path caused it not to trigger for me.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
PATCH v12 14/36 qapi: Remove outdated tests related to QMP/branch collisions
In reply to Eric BlakeNow that branches are in a separate C namespace, we can remove the restrictions in the parser that claim a branch name would collide with QMP, and delete the negative tests that are no longer problematic. A separate patch can then add positive tests to qapi-schema-test to test that any corner cases will compile correctly.
This reverts the scripts/qapi.py portion of commit 7b2a5c2, now that the assertions that it plugged are no longer possible.
---
v12: no change
v11: no change
v10: no change
v9: no change
v8: retitle and update commit message, finish reversion of 7b2a5c2
v7: new patch (also temporarily appeared in subset B v10)
--- scripts/qapi.py | 11 ++--------- tests/Makefile | 3 --- tests/qapi-schema/flat-union-clash-branch.err | 0 tests/qapi-schema/flat-union-clash-branch.exit | 1 - tests/qapi-schema/flat-union-clash-branch.json | 18 ------------------ tests/qapi-schema/flat-union-clash-branch.out | 14 -------------- tests/qapi-schema/flat-union-clash-type.err | 1 - tests/qapi-schema/flat-union-clash-type.exit | 1 - tests/qapi-schema/flat-union-clash-type.json | 14 -------------- tests/qapi-schema/flat-union-clash-type.out | 0 tests/qapi-schema/union-clash-type.err | 1 - tests/qapi-schema/union-clash-type.exit | 1 - tests/qapi-schema/union-clash-type.json | 9 --------- tests/qapi-schema/union-clash-type.out | 0 14 files changed, 2 insertions(+), 72 deletions(-) delete mode 100644 tests/qapi-schema/flat-union-clash-branch.err delete mode 100644 tests/qapi-schema/flat-union-clash-branch.exit delete mode 100644 tests/qapi-schema/flat-union-clash-branch.json delete mode 100644 tests/qapi-schema/flat-union-clash-branch.out delete mode 100644 tests/qapi-schema/flat-union-clash-type.err delete mode 100644 tests/qapi-schema/flat-union-clash-type.exit delete mode 100644 tests/qapi-schema/flat-union-clash-type.json delete mode 100644 tests/qapi-schema/flat-union-clash-type.out delete mode 100644 tests/qapi-schema/union-clash-type.err delete mode 100644 tests/qapi-schema/union-clash-type.exit delete mode 100644 tests/qapi-schema/union-clash-type.json delete mode 100644 tests/qapi-schema/union-clash-type.out
scripts/qapi.py +2 -9
@@ -548,8 +548,7 @@ def check_union(expr, expr_info): base = expr.get('base') discriminator = expr.get('discriminator') members = expr['data'] - values = {'MAX': '(automatic)', 'KIND': '(automatic)', - 'TYPE': '(automatic)'} + values = {'MAX': '(automatic)'} # Two types of unions, determined by discriminator.
@@ -607,19 +606,13 @@ def check_union(expr, expr_info): " of branch '%s'" % key) # If the discriminator names an enum type, then all members - # of 'data' must also be members of the enum type, which in turn - # must not collide with the discriminator name. + # of 'data' must also be members of the enum type. if enum_define: if key not in enum_define['enum_values']: raise QAPIExprError(expr_info, "Discriminator value '%s' is not found in " "enum '%s'" % (key, enum_define["enum_name"])) - if discriminator in enum_define['enum_values']: - raise QAPIExprError(expr_info, - "Discriminator name '%s' collides with " - "enum value in '%s'" % - (discriminator, enum_define["enum_name"])) # Otherwise, check for conflicts in the generated enum else:
tests/Makefile +0 -3
@@ -280,9 +280,7 @@ qapi-schema += flat-union-bad-base.json qapi-schema += flat-union-bad-discriminator.json qapi-schema += flat-union-base-any.json qapi-schema += flat-union-base-union.json -qapi-schema += flat-union-clash-branch.json qapi-schema += flat-union-clash-member.json -qapi-schema += flat-union-clash-type.json qapi-schema += flat-union-empty.json qapi-schema += flat-union-inline.json qapi-schema += flat-union-int-branch.json
@@ -344,7 +342,6 @@ qapi-schema += union-bad-branch.json qapi-schema += union-base-no-discriminator.json qapi-schema += union-clash-branches.json qapi-schema += union-clash-data.json -qapi-schema += union-clash-type.json qapi-schema += union-empty.json qapi-schema += union-invalid-base.json qapi-schema += union-max.json
tests/qapi-schema/flat-union-clash-branch.err deleted +0 -0
tests/qapi-schema/flat-union-clash-branch.exit deleted +0 -1
@@ -1 +0,0 @@ -0
tests/qapi-schema/flat-union-clash-branch.json deleted +0 -18
@@ -1,18 +0,0 @@ -# Flat union branch name collision -# FIXME: this parses, but then fails to compile due to a duplicate 'c_d' -# (one from the base member, the other from the branch name). We should -# either reject the collision at parse time, or munge the generated branch -# name to allow this to compile. -{ 'enum': 'TestEnum', - 'data': [ 'base', 'c-d' ] } -{ 'struct': 'Base', - 'data': { 'enum1': 'TestEnum', '*c_d': 'str' } } -{ 'struct': 'Branch1', - 'data': { 'string': 'str' } } -{ 'struct': 'Branch2', - 'data': { 'value': 'int' } } -{ 'union': 'TestUnion', - 'base': 'Base', - 'discriminator': 'enum1', - 'data': { 'base': 'Branch1', - 'c-d': 'Branch2' } }
tests/qapi-schema/flat-union-clash-branch.out deleted +0 -14
@@ -1,14 +0,0 @@ -object :empty -object Base - member enum1: TestEnum optional=False - member c_d: str optional=True -object Branch1 - member string: str optional=False -object Branch2 - member value: int optional=False -enum TestEnum ['base', 'c-d'] -object TestUnion - base Base - tag enum1 - case base: Branch1 - case c-d: Branch2
tests/qapi-schema/flat-union-clash-type.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/flat-union-clash-type.json:11: Discriminator name 'type' collides with enum value in 'TestEnum'
tests/qapi-schema/flat-union-clash-type.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/flat-union-clash-type.json deleted +0 -14
@@ -1,14 +0,0 @@ -# Flat union branch 'type' -# Reject this, because we would have a clash in generated C, between the -# outer tag 'type' and the branch name 'type' within the union. -# TODO: We could munge the generated C branch name to let it compile. -{ 'enum': 'TestEnum', - 'data': [ 'type' ] } -{ 'struct': 'Base', - 'data': { 'type': 'TestEnum' } } -{ 'struct': 'Branch1', - 'data': { 'string': 'str' } } -{ 'union': 'TestUnion', - 'base': 'Base', - 'discriminator': 'type', - 'data': { 'type': 'Branch1' } }
tests/qapi-schema/flat-union-clash-type.out deleted +0 -0
tests/qapi-schema/union-clash-type.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/union-clash-type.json:8: Union 'TestUnion' member 'kind' clashes with '(automatic)'
tests/qapi-schema/union-clash-type.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/union-clash-type.json deleted +0 -9
@@ -1,9 +0,0 @@ -# Union branch 'type' -# Reject this, because we would have a clash in generated C, between the -# simple union's implicit tag member 'kind' and the branch name 'kind' -# within the union. -# TODO: Even when the generated C is switched to use 'type' rather than -# 'kind', to match the QMP spelling, the collision should still be detected. -# Or, we could munge the branch name to allow compilation. -{ 'union': 'TestUnion', - 'data': { 'kind': 'int', 'type': 'str' } }
tests/qapi-schema/union-clash-type.out deleted +0 -0
Signature · 1 line
2.4.3
PATCH v12 15/36 qapi: Track owner of each object member
In reply to Eric BlakeFuture commits will migrate semantic checking away from parsing and over to the various QAPISchema*.check() methods. But to report an error message about an incorrect semantic use of a member of an object type, it helps to know which type, command, or event owns the member. In particular, when a member is inherited from a base type, it is desirable to associate the member name with the base type (and not the type calling member.check()).
Rather than packing additional information into the seen array passed to each member.check() (as in seen[m.name] = {'member':m, 'owner':type}), it is easier to have each member track the name of the owner type in the first place (keeping things simpler with the existing seen[m.name] = m). The new member.owner field is set via a new set_owner() method, called when registering the members and variants arrays with an object or variant type. Track only a name, and not the actual type object, to avoid creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner for the tag_member field; this field is set earlier either as part of an object's non-variant members, or explicitly by alternates.
The source information is intended for human consumption in error messages, and a new describe() method is added to access the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without duplicating efforts, the describe() method has to reverse the name of implicit types, via the helper _pretty_owner().
No change to generated code.
---
v12: no change
v11: set alternate tag_member owner during init, tweak comments, keep events with '-args' instead of '-data', prefer '(parameter of foo)'
v10 (now in subset C): rebase to latest, set alternate tag_member owner from alternate
v9 (now in subset D): rebase to earlier changes, hoist 'role' to top of class, split out _pretty_helper(), manage owner when tag_member appears as part of local_members for unions
v8: don't munge implicit type names [except for event data], and instead make describe() create nicer messages. Add set_owner(), and use field 'role' instead of method _describe()
v7: total rewrite: rework implicit object names, assign owner when initializing owner type rather than when creating member python object
v6: rebase on new lazy array creation and simple union 'type' motion; tweak commit message
--- scripts/qapi.py | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-)
scripts/qapi.py +37 -2
@@ -957,8 +957,10 @@ class QAPISchemaObjectType(QAPISchemaType): assert base is None or isinstance(base, str) for m in local_members: assert isinstance(m, QAPISchemaObjectTypeMember) - assert (variants is None or - isinstance(variants, QAPISchemaObjectTypeVariants)) + m.set_owner(name) + if variants is not None: + assert isinstance(variants, QAPISchemaObjectTypeVariants) + variants.set_owner(name) self._base_name = base self.base = None self.local_members = local_members
@@ -1013,6 +1015,8 @@ class QAPISchemaObjectType(QAPISchemaType): class QAPISchemaObjectTypeMember(object): + role = 'member' + def __init__(self, name, typ, optional): assert isinstance(name, str) assert isinstance(typ, str)
@@ -1021,8 +1025,14 @@ class QAPISchemaObjectTypeMember(object): self._type_name = typ self.type = None self.optional = optional + self.owner = None + + def set_owner(self, name): + assert not self.owner + self.owner = name def check(self, schema): + assert self.owner self.type = schema.lookup_type(self._type_name) assert self.type
@@ -1031,6 +1041,22 @@ class QAPISchemaObjectTypeMember(object): assert self.name not in seen seen[self.name] = self + def _pretty_owner(self): + owner = self.owner + if owner.startswith(':obj-'): + # See QAPISchema._make_implicit_object_type() - reverse the + # mapping there to create a nice human-readable description + owner = owner[5:] + if owner.endswith('-arg'): + return '(parameter of %s)' % owner[:-4] + else: + assert owner.endswith('-wrapper') + return '(branch of %s)' % owner[:-8] + return '(%s of %s)' % (self.role, owner) + + def describe(self): + return "'%s' %s" % (self.name, self._pretty_owner()) + class QAPISchemaObjectTypeVariants(object): def __init__(self, tag_name, tag_member, variants):
@@ -1047,6 +1073,10 @@ class QAPISchemaObjectTypeVariants(object): self.tag_member = tag_member self.variants = variants + def set_owner(self, name): + for v in self.variants: + v.set_owner(name) + def check(self, schema, seen): if not self.tag_member: # flat union self.tag_member = seen[self.tag_name]
@@ -1066,6 +1096,8 @@ class QAPISchemaObjectTypeVariants(object): class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember): + role = 'branch' + def __init__(self, name, typ): QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
@@ -1085,6 +1117,8 @@ class QAPISchemaAlternateType(QAPISchemaType): QAPISchemaType.__init__(self, name, info) assert isinstance(variants, QAPISchemaObjectTypeVariants) assert not variants.tag_name + variants.set_owner(name) + variants.tag_member.set_owner(self.name) self.variants = variants def check(self, schema):
@@ -1217,6 +1251,7 @@ class QAPISchema(object): def _make_implicit_object_type(self, name, info, role, members): if not members: return None + # See also QAPISchemaObjectTypeMember._pretty_owner() name = ':obj-%s-%s' % (name, role) if not self.lookup_entity(name, QAPISchemaObjectType): self._def_entity(QAPISchemaObjectType(name, info, None,
Signature · 1 line
2.4.3
PATCH v12 16/36 qapi: Detect collisions in C member names
In reply to Eric BlakeDetect attempts to declare two object members that would result in the same C member name, by keying the 'seen' dictionary off of the C name rather than the qapi name. It also requires passing info through the check_clash() methods.
This addresses a TODO and fixes the previously-broken args-name-clash test. The resulting error message demonstrates the utility of the .describe() method added previously. No change to generated code.
---
v12: no change
v11: rebase to earlier changes, use 'cname' local variable, shorter message in test file
v10 (now in subset C): rebase to latest; update commit message v9 (now in subset D): rebase to earlier changes, now only one test affected
v8: rebase to earlier changes
v7: split out error reporting prep and member.c_name() addition
v6: rebase to earlier testsuite and info improvements
--- scripts/qapi.py | 31 +++++++++++++++++++------------ tests/qapi-schema/args-name-clash.err | 1 + tests/qapi-schema/args-name-clash.exit | 2 +- tests/qapi-schema/args-name-clash.json | 5 ++--- tests/qapi-schema/args-name-clash.out | 6 ------ 5 files changed, 23 insertions(+), 22 deletions(-)
scripts/qapi.py +19 -12
@@ -977,20 +977,23 @@ class QAPISchemaObjectType(QAPISchemaType): self.base = schema.lookup_type(self._base_name) assert isinstance(self.base, QAPISchemaObjectType) self.base.check(schema) - self.base.check_clash(schema, seen) + self.base.check_clash(schema, self.info, seen) for m in self.local_members: m.check(schema) - m.check_clash(seen) + m.check_clash(self.info, seen) self.members = seen.values() if self.variants: self.variants.check(schema, seen) assert self.variants.tag_member in self.members - self.variants.check_clash(schema, seen) + self.variants.check_clash(schema, self.info, seen) - def check_clash(self, schema, seen): + # Check that the members of this type do not cause duplicate JSON fields, + # and update seen to track the members seen so far. Report any errors + # on behalf of info, which is not necessarily self.info + def check_clash(self, schema, info, seen): assert not self.variants # not implemented for m in self.members: - m.check_clash(seen) + m.check_clash(info, seen) def is_implicit(self): # See QAPISchema._make_implicit_object_type()
@@ -1036,10 +1039,13 @@ class QAPISchemaObjectTypeMember(object): self.type = schema.lookup_type(self._type_name) assert self.type - def check_clash(self, seen): - # TODO change key of seen from QAPI name to C name - assert self.name not in seen - seen[self.name] = self + def check_clash(self, info, seen): + cname = c_name(self.name) + if cname in seen: + raise QAPIExprError(info, + "%s collides with %s" + % (self.describe(), seen[cname].describe())) + seen[cname] = self def _pretty_owner(self): owner = self.owner
@@ -1079,7 +1085,8 @@ class QAPISchemaObjectTypeVariants(object): def check(self, schema, seen): if not self.tag_member: # flat union - self.tag_member = seen[self.tag_name] + self.tag_member = seen[c_name(self.tag_name)] + assert self.tag_name == self.tag_member.name assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: v.check(schema)
@@ -1087,12 +1094,12 @@ class QAPISchemaObjectTypeVariants(object): if isinstance(v.type, QAPISchemaObjectType): v.type.check(schema) - def check_clash(self, schema, seen): + def check_clash(self, schema, info, seen): for v in self.variants: # Reset seen map for each variant, since qapi names from one # branch do not affect another branch assert isinstance(v.type, QAPISchemaObjectType) - v.type.check_clash(schema, dict(seen)) + v.type.check_clash(schema, info, dict(seen)) class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
tests/qapi-schema/args-name-clash.err +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/args-name-clash.json:4: 'a_b' (parameter of oops) collides with 'a-b' (parameter of oops)
tests/qapi-schema/args-name-clash.exit +1 -1
@@ -1 +1 @@ -0 +1
tests/qapi-schema/args-name-clash.json +2 -3
@@ -1,5 +1,4 @@ # C member name collision -# FIXME - This parses, but fails to compile, because the C struct is given -# two 'a_b' members. Either reject this at parse time, or munge the C names -# to avoid the collision. +# Reject members that clash when mapped to C names (we would have two 'a_b' +# members). { 'command': 'oops', 'data': { 'a-b': 'str', 'a_b': 'str' } }
tests/qapi-schema/args-name-clash.out +0 -6
@@ -1,6 +0,0 @@ -object :empty -object :obj-oops-arg - member a-b: str optional=False - member a_b: str optional=False -command oops :obj-oops-arg -> None - gen=True success_response=True
Signature · 1 line
2.4.3
PATCH v12 17/36 qapi: Fix c_name() munging
In reply to Eric BlakeThe method c_name() is supposed to do two different actions: munge '-' into '_', and add a 'q_' prefix to ticklish names. But it did these steps out of order, making it possible to submit input that is not ticklish until after munging, where the output then lacked the desired prefix.
The failure is exposed easily if you have a compiler that recognizes C11 keywords, and try to name a member '_Thread-local', as it would result in trying to compile the declaration 'uint64_t _Thread_local;' which is not valid. However, this name violates our conventions (ultimately, want to enforce that no qapi names start with single underscore), so the test is slightly weaker by instead testing 'wchar-t'; the declaration 'uint64_t wchar_t;' is valid in C (where wchar_t is only a typedef) but fails with a C++ compiler (where it is a keyword).
Fix things by reversing the order of actions within c_name().
---
v12: new patch
--- scripts/qapi.py | 3 ++- tests/qapi-schema/qapi-schema-test.json | 5 +++-- tests/qapi-schema/qapi-schema-test.out | 1 + tests/test-qmp-commands.c | 4 ++++ 4 files changed, 10 insertions(+), 3 deletions(-)
scripts/qapi.py +2 -1
@@ -1481,10 +1481,11 @@ def c_name(name, protect=True): 'not_eq', 'or', 'or_eq', 'xor', 'xor_eq']) # namespace pollution: polluted_words = set(['unix', 'errno']) + name = name.translate(c_name_trans) if protect and (name in c89_words | c99_words | c11_words | gcc_words | cpp_words | polluted_words): return "q_" + name - return name.translate(c_name_trans) + return name eatspace = '\033EATSPACE.' pointer_suffix = ' *' + eatspace
tests/qapi-schema/qapi-schema-test.json +3 -2
@@ -152,12 +152,13 @@ { 'event': 'EVENT_D', 'data': { 'a' : 'EventStructOne', 'b' : 'str', '*c': 'str', '*enum3': 'EnumOne' } } -# test that we correctly compile downstream extensions +# test that we correctly compile downstream extensions, as well as munge +# ticklish names { 'enum': '__org.qemu_x-Enum', 'data': [ '__org.qemu_x-value' ] } { 'struct': '__org.qemu_x-Base', 'data': { '__org.qemu_x-member1': '__org.qemu_x-Enum' } } { 'struct': '__org.qemu_x-Struct', 'base': '__org.qemu_x-Base', - 'data': { '__org.qemu_x-member2': 'str' } } + 'data': { '__org.qemu_x-member2': 'str', '*wchar-t': 'int' } } { 'union': '__org.qemu_x-Union1', 'data': { '__org.qemu_x-branch': 'str' } } { 'struct': '__org.qemu_x-Struct2', 'data': { 'array': ['__org.qemu_x-Union1'] } }
tests/qapi-schema/qapi-schema-test.out +1 -0
@@ -185,6 +185,7 @@ enum __org.qemu_x-Enum ['__org.qemu_x-value'] object __org.qemu_x-Struct base __org.qemu_x-Base member __org.qemu_x-member2: str optional=False + member wchar-t: int optional=True object __org.qemu_x-Struct2 member array: __org.qemu_x-Union1List optional=False object __org.qemu_x-Union1
tests/test-qmp-commands.c +4 -0
@@ -65,6 +65,10 @@ __org_qemu_x_Union1 *qmp___org_qemu_x_command(__org_qemu_x_EnumList *a, ret->type =ORG_QEMU_X_UNION1_KIND___ORG_QEMU_X_BRANCH ; ret->u.__org_qemu_x_branch =strdup ("blah1" ); +/* Also test that 'wchar-t' was munged to 'q_wchar_t' */ +if (b && b->value && !b->value->has_q_wchar_t) { + b->value->q_wchar_t =1 ; + }return ret; }
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 17/36] qapi: Fix c_name() munging
Quoted 4 lines · Eric Blake
*** WARNING: THE ATTACHED DOCUMENT(S) CONTAIN MACROS ***
*** MACROS MAY CONTAIN MALICIOUS CODE ***
*** Open only if you can verify and trust the sender ***
*** Please contact [email protected] if you have questions or concerns **
Another one.
Quoted 6 lines
The method
c_name()is supposed to do two different actions: munge '-' into '_', and add a 'q_' prefix to ticklish names. But it did these steps out of order, making it possible to submit input that is not ticklish until after munging, where the output then lacked the desired prefix.
Quoted 9 lines
The failure is exposed easily if you have a compiler that recognizes C11 keywords, and try to name a member '_Thread-local', as it would result in trying to compile the declaration '
uint64_t_Thread_local;' which is not valid. However, this name violates our conventions (ultimately, want to enforce that no qapi names start with single underscore), so the test is slightly weaker by instead testing 'wchar-t'; the declaration 'uint64_twchar_t;' is valid in C (wherewchar_tis only a typedef) but fails with a C++ compiler (where it is a keyword).
Do we support compiling it with a C++ compiler? To sidestep the question, I'd say "but would fail".
In my private opinion, the one sane way to compile C code with a C++ compiler is wrapping it in extern "C" { ... }.
Quoted 3 lines
Fix things by reversing the order of actions within
c_name().Signed-off-by Eric Blake <[email protected]>
Again, just commit message polish, can do on merge.
Re: [Qemu-devel] [PATCH v12 17/36] qapi: Fix c_name() munging
Quoted 25 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
*** WARNING: THE ATTACHED DOCUMENT(S) CONTAIN MACROS ***
*** MACROS MAY CONTAIN MALICIOUS CODE ***
*** Open only if you can verify and trust the sender ***
*** Please contact [email protected] if you have questions or concerns **Another one.
Quoted 6 lines
The method
c_name()is supposed to do two different actions: munge '-' into '_', and add a 'q_' prefix to ticklish names. But it did these steps out of order, making it possible to submit input that is not ticklish until after munging, where the output then lacked the desired prefix.Quoted 9 lines
The failure is exposed easily if you have a compiler that recognizes C11 keywords, and try to name a member '_Thread-local', as it would result in trying to compile the declaration '
uint64_t_Thread_local;' which is not valid. However, this name violates our conventions (ultimately, want to enforce that no qapi names start with single underscore), so the test is slightly weaker by instead testing 'wchar-t'; the declaration 'uint64_twchar_t;' is valid in C (wherewchar_tis only a typedef) but fails with a C++ compiler (where it is a keyword).
Quoted 2 lines
Do we support compiling it with a C++ compiler? To sidestep the question, I'd say "but would fail".
I know we require a C++ compiler for qga on Windows, and qga uses qapi, so I think the problem is real; but as I do not have a working setup for compiling qga for windows, I can only speculate. Changing s/fails/but would fail/ is a nice hedge.
Quoted 3 lines
In my private opinion, the one sane way to compile C code with a C++ compiler is wrapping it in extern "C" { ... }.
True - but I don't think that changes the set of C++ keywords inside the extern block, does it? (The thought of context-sensitive keywords makes me cringe for how one would write a sane parser for that language).
Quoted 7 lines
Quoted 4 lines
Fix things by reversing the order of actions within
c_name().Signed-off-by Eric Blake <[email protected]>Again, just commit message polish, can do on merge.
Don't know why you got it on some messages and not others; I also got a round of those pollutions on other threads. It looks like the responsible party has cleaned up their false positives in the meantime, so hopefully we aren't hit by more of the annoyance.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 17/36] qapi: Fix c_name() munging
Quoted 35 lines · Eric Blake
Quoted 25 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
*** WARNING: THE ATTACHED DOCUMENT(S) CONTAIN MACROS ***
*** MACROS MAY CONTAIN MALICIOUS CODE ***
*** Open only if you can verify and trust the sender ***
*** Please contact [email protected] if you have questions or concerns **Another one.
Quoted 6 lines
The method
c_name()is supposed to do two different actions: munge '-' into '_', and add a 'q_' prefix to ticklish names. But it did these steps out of order, making it possible to submit input that is not ticklish until after munging, where the output then lacked the desired prefix.Quoted 9 lines
The failure is exposed easily if you have a compiler that recognizes C11 keywords, and try to name a member '_Thread-local', as it would result in trying to compile the declaration '
uint64_t_Thread_local;' which is not valid. However, this name violates our conventions (ultimately, want to enforce that no qapi names start with single underscore), so the test is slightly weaker by instead testing 'wchar-t'; the declaration 'uint64_twchar_t;' is valid in C (wherewchar_tis only a typedef) but fails with a C++ compiler (where it is a keyword).Quoted 2 lines
Do we support compiling it with a C++ compiler? To sidestep the question, I'd say "but would fail".
I know we require a C++ compiler for qga on Windows, and qga uses qapi, so I think the problem is real; but as I do not have a working setup for compiling qga for windows, I can only speculate. Changing s/fails/but would fail/ is a nice hedge.
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
In my private opinion, the one sane way to compile C code with a C++ compiler is wrapping it in extern "C" { ... }.
True - but I don't think that changes the set of C++ keywords inside the extern block, does it? (The thought of context-sensitive keywords makes me cringe for how one would write a sane parser for that language).
The truly sane way to compile C with a C++ compiler is of course "not":
compile it with C and link it.
The exception is headers. Stick to declaring extern symbols and the types they need.
Quoted 11 lines
Quoted 6 lines
Quoted 3 lines
Fix things by reversing the order of actions within
c_name().Signed-off-by Eric Blake <[email protected]>Again, just commit message polish, can do on merge.
Don't know why you got it on some messages and not others; I also got a round of those pollutions on other threads. It looks like the responsible party has cleaned up their false positives in the meantime, so hopefully we aren't hit by more of the annoyance.
I reported it, and was told it's been fixed.
PATCH v12 18/36 qapi: Remove dead visitor code
In reply to Eric BlakeCommit cbc95538 removed unused start_handle() and end_handle(), but forgot to remove their declarations.
---
v12 (no v10-11): Split off obvious piece from v9 26/27
v9: no change
v8: no change
v7: no change
v6: no change
--- include/qapi/visitor.h | 3 --- 1 file changed, 3 deletions(-)
include/qapi/visitor.h +0 -3
@@ -27,9 +27,6 @@ typedef struct GenericListstruct GenericList *next; } GenericList; -void visit_start_handle (Visitor *v,void **obj,const char *kind, -const char *name,Error **errp); -void visit_end_handle (Visitor *v,Error **errp);void visit_start_struct (Visitor *v,void **obj,const char *kind,const char *name,size_t size,Error **errp);void visit_end_struct (Visitor *v,Error **errp);
Signature · 1 line
2.4.3
PATCH v12 19/36 blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
In reply to Eric BlakeNo need to keep two separate enums, where editing one is likely to forget the other. Now that we can specify a qapi enum prefix, we don't even have to change the bulk of the uses.
get_event_by_name() could perhaps be replaced by qapi_enum_parse(), but I left that for another day.
---
v12: new patch
--- block.c | 2 +- block/blkdebug.c | 79 +++++++---------------------------------------- docs/blkdebug.txt | 7 +++-- include/block/block.h | 62 +------------------------------------ include/block/block_int.h | 2 +- qapi/block-core.json | 4 ++- 6 files changed, 21 insertions(+), 135 deletions(-)
block.c +1 -1
@@ -2851,7 +2851,7 @@ ImageInfoSpecific *bdrv_get_specific_info(BlockDriverState *bs)return NULL ; } -void bdrv_debug_event (BlockDriverState *bs,BlkDebugEvent event) +void bdrv_debug_event (BlockDriverState *bs,BlkdebugEvent event) {if (!bs || !bs->drv || !bs->drv->bdrv_debug_event) {return ;
block/blkdebug.c +11 -68
135 lines not on this page · raw message
docs/blkdebug.txt +4 -3
@@ -1,6 +1,6 @@ Block I/O error injection using blkdebug ---------------------------------------- -Copyright (C) 2014 Red Hat Inc +Copyright (C) 2014-2015 Red Hat Inc This work is licensed under the terms of the GNU GPL, version 2 or later. See the COPYING file in the top-level directory.
@@ -92,8 +92,9 @@ The core events are: flush_to_disk - flush the host block device's disk cache -See block/blkdebug.c:event_names[] for the full list of events. You may need -to grep block driver source code to understand the meaning of specific events. +See qapi/block-core.json:BlkdebugEvent for the full list of events. +You may need to grep block driver source code to understand the +meaning of specific events. State transitions -----------------
include/block/block.h +1 -61
74 lines not on this page · raw message
include/block/block_int.h +1 -1
@@ -244,7 +244,7 @@ struct BlockDriver {int (*bdrv_amend_options)(BlockDriverState *bs,QemuOpts *opts,BlockDriverAmendStatusCB *status_cb); -void (*bdrv_debug_event)(BlockDriverState *bs,BlkDebugEvent event); +void (*bdrv_debug_event)(BlockDriverState *bs,BlkdebugEvent event);/* TODO Better pass a option string/QDict/QemuOpts to add any rule? */ int (*bdrv_debug_breakpoint)(BlockDriverState *bs, const char *event,
qapi/block-core.json +3 -1
@@ -1776,8 +1776,10 @@ # @BlkdebugEvent # # Trigger events supported by blkdebug. +# +# Since: 2.0 ## -{ 'enum': 'BlkdebugEvent', +{ 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', 'l1_grow.activate_table', 'l2_load', 'l2_update', 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write',
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 19/36] blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
Quoted 304 lines · Eric Blake
No need to keep two separate enums, where editing one is likely to forget the other. Now that we can specify a qapi enum prefix, we don't even have to change the bulk of the uses.
get_event_by_name()could perhaps be replaced byqapi_enum_parse(), but I left that for another day.CC Kevin Wolf <[email protected]>Signed-off-by Eric Blake <[email protected]>---
v12: new patch--- block.c | 2 +- block/blkdebug.c | 79 +++++++---------------------------------------- docs/blkdebug.txt | 7 +++-- include/block/block.h | 62 +------------------------------------ include/block/block_int.h | 2 +- qapi/block-core.json | 4 ++- 6 files changed, 21 insertions(+), 135 deletions(-)block.c +1
-1@@ -2851,7 +2851,7 @@ ImageInfoSpecific *bdrv_get_specific_info(BlockDriverState *bs)return NULL ; } -void bdrv_debug_event (BlockDriverState *bs,BlkDebugEvent event) +void bdrv_debug_event (BlockDriverState *bs,BlkdebugEvent event) {if (!bs || !bs->drv || !bs->drv->bdrv_debug_event) {return ;block/blkdebug.c +11
-68@@ -35,7 +35,7 @@ typedef struct BDRVBlkdebugState {int state;int new_state; - QLIST_HEAD(, BlkdebugRule) rules[BLKDBG_EVENT_MAX]; + QLIST_HEAD(, BlkdebugRule) rules[BLKDBG_MAX]; QSIMPLEQ_HEAD(, BlkdebugRule) active_rules; QLIST_HEAD(, BlkdebugSuspendedReq) suspended_reqs; } BDRVBlkdebugState;@@ -63,7 +63,7 @@ enum { }; typedef struct BlkdebugRule { - BlkDebugEvent event; + BlkdebugEvent event; int action; int state; union {@@ -142,69 +142,12 @@ static QemuOptsList *config_groups[] = { NULL }; -static const char *event_names[BLKDBG_EVENT_MAX ] = { - [BLKDBG_L1_UPDATE ] ="l1_update" , - [BLKDBG_L1_GROW_ALLOC_TABLE ] ="l1_grow.alloc_table" , - [BLKDBG_L1_GROW_WRITE_TABLE ] ="l1_grow.write_table" , - [BLKDBG_L1_GROW_ACTIVATE_TABLE ] ="l1_grow.activate_table" , - - [BLKDBG_L2_LOAD ] ="l2_load" , - [BLKDBG_L2_UPDATE ] ="l2_update" , - [BLKDBG_L2_UPDATE_COMPRESSED ] ="l2_update_compressed" , - [BLKDBG_L2_ALLOC_COW_READ ] ="l2_alloc.cow_read" , - [BLKDBG_L2_ALLOC_WRITE ] ="l2_alloc.write" , - - [BLKDBG_READ_AIO ] ="read_aio" , - [BLKDBG_READ_BACKING_AIO ] ="read_backing_aio" , - [BLKDBG_READ_COMPRESSED ] ="read_compressed" , - - [BLKDBG_WRITE_AIO ] ="write_aio" , - [BLKDBG_WRITE_COMPRESSED ] ="write_compressed" , - - [BLKDBG_VMSTATE_LOAD ] ="vmstate_load" , - [BLKDBG_VMSTATE_SAVE ] ="vmstate_save" , - - [BLKDBG_COW_READ ] ="cow_read" , - [BLKDBG_COW_WRITE ] ="cow_write" , - - [BLKDBG_REFTABLE_LOAD ] ="reftable_load" , - [BLKDBG_REFTABLE_GROW ] ="reftable_grow" , - [BLKDBG_REFTABLE_UPDATE ] ="reftable_update" , - - [BLKDBG_REFBLOCK_LOAD ] ="refblock_load" , - [BLKDBG_REFBLOCK_UPDATE ] ="refblock_update" , - [BLKDBG_REFBLOCK_UPDATE_PART ] ="refblock_update_part" , - [BLKDBG_REFBLOCK_ALLOC ] ="refblock_alloc" , - [BLKDBG_REFBLOCK_ALLOC_HOOKUP ] ="refblock_alloc.hookup" , - [BLKDBG_REFBLOCK_ALLOC_WRITE ] ="refblock_alloc.write" , - [BLKDBG_REFBLOCK_ALLOC_WRITE_BLOCKS ] ="refblock_alloc.write_blocks" , - [BLKDBG_REFBLOCK_ALLOC_WRITE_TABLE ] ="refblock_alloc.write_table" , - [BLKDBG_REFBLOCK_ALLOC_SWITCH_TABLE ] ="refblock_alloc.switch_table" , - - [BLKDBG_CLUSTER_ALLOC ] ="cluster_alloc" , - [BLKDBG_CLUSTER_ALLOC_BYTES ] ="cluster_alloc_bytes" , - [BLKDBG_CLUSTER_FREE ] ="cluster_free" , - - [BLKDBG_FLUSH_TO_OS ] ="flush_to_os" , - [BLKDBG_FLUSH_TO_DISK ] ="flush_to_disk" , - - [BLKDBG_PWRITEV_RMW_HEAD ] ="pwritev_rmw.head" , - [BLKDBG_PWRITEV_RMW_AFTER_HEAD ] ="pwritev_rmw.after_head" , - [BLKDBG_PWRITEV_RMW_TAIL ] ="pwritev_rmw.tail" , - [BLKDBG_PWRITEV_RMW_AFTER_TAIL ] ="pwritev_rmw.after_tail" , - [BLKDBG_PWRITEV ] ="pwritev" , - [BLKDBG_PWRITEV_ZERO ] ="pwritev_zero" , - [BLKDBG_PWRITEV_DONE ] ="pwritev_done" , - - [BLKDBG_EMPTY_IMAGE_PREPARE ] ="empty_image_prepare" , -}; - -static int get_event_by_name (const char *name,BlkDebugEvent *event) +static int get_event_by_name (const char *name,BlkdebugEvent *event) {int i; -for (i =0 ; i <BLKDBG_EVENT_MAX ; i++) { -if (!strcmp (event_names[i], name)) { +for (i =0 ; i <BLKDBG_MAX ; i++) { +if (!strcmp (BlkdebugEvent_lookup[i], name)) { *event = i;return 0 ; }@@ -223,7 +166,7 @@ static int add_rule(void *opaque, QemuOpts *opts, Error **errp)struct add_rule_data *d = opaque;BDRVBlkdebugState *s = d->s;const char * event_name; -BlkDebugEvent event; +BlkdebugEvent event;struct BlkdebugRule *rule;/* Find the right event for the rule */ @@ -563,7 +506,7 @@ static void blkdebug_close(BlockDriverState *bs)BlkdebugRule *rule, *next;int i; -for (i =0 ; i <BLKDBG_EVENT_MAX ; i++) { +for (i =0 ; i <BLKDBG_MAX ; i++) { QLIST_FOREACH_SAFE(rule, &s->rules[i], next, next) {remove_rule (rule); }@@ -622,13 +565,13 @@ static bool process_rule(BlockDriverState *bs, struct BlkdebugRule *rule,return injected; } -static void blkdebug_debug_event (BlockDriverState *bs,BlkDebugEvent event) +static void blkdebug_debug_event (BlockDriverState *bs,BlkdebugEvent event) {BDRVBlkdebugState *s = bs->opaque;struct BlkdebugRule *rule, *next;bool injected; -assert ((int )event >=0 && event <BLKDBG_EVENT_MAX ); +assert ((int )event >=0 && event <BLKDBG_MAX ); injected = false; s->new_state = s->state;@@ -643,7 +586,7 @@ static int blkdebug_debug_breakpoint(BlockDriverState *bs, const char *event, {BDRVBlkdebugState *s = bs->opaque;struct BlkdebugRule *rule; -BlkDebugEvent blkdebug_event; +BlkdebugEvent blkdebug_event;if (get_event_by_name (event, &blkdebug_event) <0 ) {return -ENOENT ;@@ -685,7 +628,7 @@ static int blkdebug_debug_remove_breakpoint(BlockDriverState *bs,BlkdebugRule *rule, *next;int i, ret = -ENOENT ; -for (i =0 ; i <BLKDBG_EVENT_MAX ; i++) { +for (i =0 ; i <BLKDBG_MAX ; i++) { QLIST_FOREACH_SAFE(rule, &s->rules[i], next, next) {if (rule->action ==ACTION_SUSPEND && !strcmp (rule->options.suspend.tag, tag)) {docs/blkdebug.txt +4
-3@@ -1,6 +1,6 @@ Block I/O error injection using blkdebug ---------------------------------------- -Copyright (C) 2014 Red Hat Inc +Copyright (C) 2014-2015 Red Hat Inc This work is licensed under the terms of the GNU GPL, version 2 or later. See the COPYING file in the top-level directory.@@ -92,8 +92,9 @@ The core events are: flush_to_disk - flush the host block device's disk cache -See block/blkdebug.c:event_names[] for the full list of events. You may need -to grep block driver source code to understand the meaning of specific events. +See qapi/block-core.json:BlkdebugEvent for the full list of events. +You may need to grep block driver source code to understand the +meaning of specific events. State transitions -----------------include/block/block.h +1
-61@@ -520,66 +520,6 @@ void bdrv_op_block_all(BlockDriverState *bs, Error *reason);void bdrv_op_unblock_all (BlockDriverState *bs,Error *reason);bool bdrv_op_blocker_is_empty (BlockDriverState *bs); -typedef enum { -BLKDBG_L1_UPDATE , - -BLKDBG_L1_GROW_ALLOC_TABLE , -BLKDBG_L1_GROW_WRITE_TABLE , -BLKDBG_L1_GROW_ACTIVATE_TABLE , - -BLKDBG_L2_LOAD , -BLKDBG_L2_UPDATE , -BLKDBG_L2_UPDATE_COMPRESSED , -BLKDBG_L2_ALLOC_COW_READ , -BLKDBG_L2_ALLOC_WRITE , - -BLKDBG_READ_AIO , -BLKDBG_READ_BACKING_AIO , -BLKDBG_READ_COMPRESSED , - -BLKDBG_WRITE_AIO , -BLKDBG_WRITE_COMPRESSED , - -BLKDBG_VMSTATE_LOAD , -BLKDBG_VMSTATE_SAVE , - -BLKDBG_COW_READ , -BLKDBG_COW_WRITE , - -BLKDBG_REFTABLE_LOAD , -BLKDBG_REFTABLE_GROW , -BLKDBG_REFTABLE_UPDATE , - -BLKDBG_REFBLOCK_LOAD , -BLKDBG_REFBLOCK_UPDATE , -BLKDBG_REFBLOCK_UPDATE_PART , -BLKDBG_REFBLOCK_ALLOC , -BLKDBG_REFBLOCK_ALLOC_HOOKUP , -BLKDBG_REFBLOCK_ALLOC_WRITE , -BLKDBG_REFBLOCK_ALLOC_WRITE_BLOCKS , -BLKDBG_REFBLOCK_ALLOC_WRITE_TABLE , -BLKDBG_REFBLOCK_ALLOC_SWITCH_TABLE , - -BLKDBG_CLUSTER_ALLOC , -BLKDBG_CLUSTER_ALLOC_BYTES , -BLKDBG_CLUSTER_FREE , - -BLKDBG_FLUSH_TO_OS , -BLKDBG_FLUSH_TO_DISK , - -BLKDBG_PWRITEV_RMW_HEAD , -BLKDBG_PWRITEV_RMW_AFTER_HEAD , -BLKDBG_PWRITEV_RMW_TAIL , -BLKDBG_PWRITEV_RMW_AFTER_TAIL , -BLKDBG_PWRITEV , -BLKDBG_PWRITEV_ZERO , -BLKDBG_PWRITEV_DONE , - -BLKDBG_EMPTY_IMAGE_PREPARE , - -BLKDBG_EVENT_MAX , -}BlkDebugEvent ; -#define BLKDBG_EVENT (child, evt) \ do { \ if (child) { \@@ -587,7 +527,7 @@ typedef enum { } \ }while (0 ) -void bdrv_debug_event(BlockDriverState *bs, BlkDebugEvent event); +void bdrv_debug_event(BlockDriverState *bs, BlkdebugEvent event);int bdrv_debug_breakpoint (BlockDriverState *bs,const char *event,const char *tag);include/block/block_int.h +1
-1@@ -244,7 +244,7 @@ struct BlockDriver {int (*bdrv_amend_options)(BlockDriverState *bs,QemuOpts *opts,BlockDriverAmendStatusCB *status_cb); -void (*bdrv_debug_event)(BlockDriverState *bs,BlkDebugEvent event); +void (*bdrv_debug_event)(BlockDriverState *bs,BlkdebugEvent event);/* TODO Better pass a option string/QDict/QemuOpts to add any rule? */ int (*bdrv_debug_breakpoint)(BlockDriverState *bs, const char *event,qapi/block-core.json +0
-0@@ -1776,8 +1776,10 @@ # @BlkdebugEvent
qapi/block-core.json line 1785
Quoted 10 lines
qapi/block-core.json +3
-1# # Trigger events supported by blkdebug. +# +# Since: 2.0 ## -{ 'enum': 'BlkdebugEvent', +{ 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', 'l1_grow.activate_table', 'l2_load', 'l2_update', 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write',
I'm no friend of the 'prefix' feature. You could avoid it here by renaming BlkdebugEvent to Blkdbg. No additional churn, because you already replace hand-written BlkDebugEvent by generated BlkdebugEvent.
Alternatively, you could reduce churn by renaming it to BlkDebugEvent.
Matter of taste. Perhaps Kevin has a preference.
Re: [Qemu-devel] [PATCH v12 19/36] blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
Am 18.11.2015 um 11:30 hat Markus Armbruster geschrieben:
Quoted 33 lines
Quoted 16 lines · Eric Blake
No need to keep two separate enums, where editing one is likely to forget the other. Now that we can specify a qapi enum prefix, we don't even have to change the bulk of the uses.
get_event_by_name()could perhaps be replaced byqapi_enum_parse(), but I left that for another day.CC Kevin Wolf <[email protected]>Signed-off-by Eric Blake <[email protected]>qapi/block-core.json +0
-0@@ -1776,8 +1776,10 @@ # @BlkdebugEventqapi/block-core.json line 1785
Quoted 10 lines
qapi/block-core.json +3
-1# # Trigger events supported by blkdebug. +# +# Since: 2.0 ## -{ 'enum': 'BlkdebugEvent', +{ 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', 'l1_grow.activate_table', 'l2_load', 'l2_update', 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write',I'm no friend of the 'prefix' feature. You could avoid it here by renaming BlkdebugEvent to Blkdbg. No additional churn, because you already replace hand-written BlkDebugEvent by generated BlkdebugEvent.
Quoted 3 lines
Alternatively, you could reduce churn by renaming it to BlkDebugEvent.
Matter of taste. Perhaps Kevin has a preference.
Wouldn't that mean that we end up with a C type called Blkdbg? In my opinion that's a bit unspecific, so if you ask me, I would paint my bikeshed in a different colour.
Kevin
Re: [Qemu-devel] [PATCH v12 19/36] blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
Quoted 19 lines · Kevin Wolf
Am 18.11.2015 um 11:30 hat Markus Armbruster geschrieben:
Quoted 18 lines
qapi/block-core.json line 1785
Quoted 11 lines · Eric Blake
No need to keep two separate enums, where editing one is likely to forget the other. Now that we can specify a qapi enum prefix, we don't even have to change the bulk of the uses.
##
qapi/block-core.json +1
-1-{ 'enum': 'BlkdebugEvent', +{ 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', 'l1_grow.activate_table', 'l2_load', 'l2_update', 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write',I'm no friend of the 'prefix' feature. You could avoid it here by renaming BlkdebugEvent to Blkdbg. No additional churn, because you already replace hand-written BlkDebugEvent by generated BlkdebugEvent.
Quoted 7 lines
Quoted 3 lines
Alternatively, you could reduce churn by renaming it to BlkDebugEvent.
Matter of taste. Perhaps Kevin has a preference.
Wouldn't that mean that we end up with a C type called Blkdbg? In my opinion that's a bit unspecific, so if you ask me, I would paint my bikeshed in a different colour.
Most of the existing qapi names are Blkdebug, it was only the internal enum being replaced that used BlkDebug. Also, qapi would munge BlkDebug into BLK_DEBUG, so I think we want to stick with the lowercase 'd' so that the munging (without 'prefix') would stick to BLKDEBUG.
I'm also fine if you want me to do a followup patch that renames all enums from BLKDBG_L1_UPDATE to BLKDEBUG_EVENT_L1_UPDATE, at which point we could drop the 'prefix' (I don't know how many lines it would make long enough to need different wrapping, but modulo wrapping it would all be a mechanically scripted change).
I don't have a favorite color in this painting match, so let me know yours :)
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 19/36] blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
Am 18.11.2015 um 17:26 hat Eric Blake geschrieben:
Quoted 24 lines · Kevin Wolf
Quoted 23 lines
Am 18.11.2015 um 11:30 hat Markus Armbruster geschrieben:
Quoted 21 lines
qapi/block-core.json line 1785
Quoted 11 lines · Eric Blake
No need to keep two separate enums, where editing one is likely to forget the other. Now that we can specify a qapi enum prefix, we don't even have to change the bulk of the uses.
##
qapi/block-core.json +1
-1-{ 'enum': 'BlkdebugEvent', +{ 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', 'l1_grow.activate_table', 'l2_load', 'l2_update', 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write',I'm no friend of the 'prefix' feature. You could avoid it here by renaming BlkdebugEvent to Blkdbg. No additional churn, because you already replace hand-written BlkDebugEvent by generated BlkdebugEvent.
Alternatively, you could reduce churn by renaming it to BlkDebugEvent.
Matter of taste. Perhaps Kevin has a preference.
Quoted 8 lines
Quoted 3 lines
Wouldn't that mean that we end up with a C type called Blkdbg? In my opinion that's a bit unspecific, so if you ask me, I would paint my bikeshed in a different colour.
Most of the existing qapi names are Blkdebug, it was only the internal enum being replaced that used BlkDebug. Also, qapi would munge BlkDebug into
BLK_DEBUG, so I think we want to stick with the lowercase 'd' so that the munging (without 'prefix') would stick to BLKDEBUG.
Oh, I don't really have an opinion whether BlockDebug, BlkDebug or Blkdebug is the best spelling. I think Blkdebug makes sense.
But I do have a strong opinion on whether the enum should be called BlkdebugEvent or just Blkdbg (without any mention of "Event"). The latter doesn't describe any more what the type is about.
Quoted 5 lines
I'm also fine if you want me to do a followup patch that renames all enums from
BLKDBG_L1_UPDATEtoBLKDEBUG_EVENT_L1_UPDATE, at which point we could drop the 'prefix' (I don't know how many lines it would make long enough to need different wrapping, but modulo wrapping it would all be a mechanically scripted change).
I don't mind the 'prefix' feature. I think it's nice to have short, but still unique enough constant names, but if Markus is bothered enough to send a patch, I can live with BLKDEBUG_EVENT_*.
Kevin
Re: [Qemu-devel] [PATCH v12 19/36] blkdebug: Merge hand-rolled and qapi BlkdebugEvent enum
Quoted 34 lines · Kevin Wolf
Am 18.11.2015 um 17:26 hat Eric Blake geschrieben:
Quoted 24 lines · Kevin Wolf
Quoted 23 lines
Am 18.11.2015 um 11:30 hat Markus Armbruster geschrieben:
Quoted 21 lines
qapi/block-core.json line 1785
Quoted 11 lines · Eric Blake
No need to keep two separate enums, where editing one is likely to forget the other. Now that we can specify a qapi enum prefix, we don't even have to change the bulk of the uses.
##
qapi/block-core.json +1
-1-{ 'enum': 'BlkdebugEvent', +{ 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', 'l1_grow.activate_table', 'l2_load', 'l2_update', 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write',I'm no friend of the 'prefix' feature. You could avoid it here by renaming BlkdebugEvent to Blkdbg. No additional churn, because you already replace hand-written BlkDebugEvent by generated BlkdebugEvent.
Alternatively, you could reduce churn by renaming it to BlkDebugEvent.
Matter of taste. Perhaps Kevin has a preference.
Quoted 8 lines
Quoted 3 lines
Wouldn't that mean that we end up with a C type called Blkdbg? In my opinion that's a bit unspecific, so if you ask me, I would paint my bikeshed in a different colour.
Most of the existing qapi names are Blkdebug, it was only the internal enum being replaced that used BlkDebug. Also, qapi would munge BlkDebug into
BLK_DEBUG, so I think we want to stick with the lowercase 'd' so that the munging (without 'prefix') would stick to BLKDEBUG.
Quoted 6 lines
Oh, I don't really have an opinion whether BlockDebug, BlkDebug or Blkdebug is the best spelling. I think Blkdebug makes sense.
But I do have a strong opinion on whether the enum should be called BlkdebugEvent or just Blkdbg (without any mention of "Event"). The latter doesn't describe any more what the type is about.
Okay, we'll take the patch as is then.
Quoted 9 lines
Quoted 5 lines
I'm also fine if you want me to do a followup patch that renames all enums from
BLKDBG_L1_UPDATEtoBLKDEBUG_EVENT_L1_UPDATE, at which point we could drop the 'prefix' (I don't know how many lines it would make long enough to need different wrapping, but modulo wrapping it would all be a mechanically scripted change).I don't mind the 'prefix' feature. I think it's nice to have short, but still unique enough constant names, but if Markus is bothered enough to send a patch, I can live with BLKDEBUG_EVENT_*.
Fair enough :) For now, let's move the queue forward.
PATCH v12 20/36 blkdebug: Avoid '.' in enum values
In reply to Eric BlakeOur qapi conventions document that '.' should only be used in the prefix of downstream names. BlkdebugEvent was a lone exception to this. Changing this is not backwards compatible to the 'blockdev-add' QMP command; however, that command is not yet fully stable. It can also be argued that the testsuite is the biggest user of blkdebug, and that any other user can be taught to deal with the change by paying attention to introspection results.
Done with:
$ for str in \
l1_grow.{alloc,write,activate}_table \
l2_alloc.{cow_read,write} \
refblock_alloc.{hookup,write,write_blocks,write_table,switch_table} \
pwritev_rmw.{head,after_head,tail,after_tail}; do
str1=$(echo "$str" | sed 's/\./\\./')
str2=$(echo "$str" | sed 's/\./_/')
git grep -l "$str1" | xargs -r sed -i "s/$str1/$str2/g"
donefollowed by a manual touchup to test 77 to keep the test working.
---
v12: new commit. Arguably, test 77 should have failed immediately on an unrecognized event name instead of hanging forever, but that fix belongs in a separate patch, not part of this series.
--- qapi/block-core.json | 16 ++++---- tests/qemu-iotests/026 | 18 ++++----- tests/qemu-iotests/026.out | 80 +++++++++++++++++++------------------- tests/qemu-iotests/026.out.nocache | 80 +++++++++++++++++++------------------- tests/qemu-iotests/077 | 24 ++++++------ 5 files changed, 109 insertions(+), 109 deletions(-)
qapi/block-core.json +8 -8
@@ -1780,19 +1780,19 @@ # Since: 2.0 ## { 'enum': 'BlkdebugEvent', 'prefix': 'BLKDBG', - 'data': [ 'l1_update', 'l1_grow.alloc_table', 'l1_grow.write_table', - 'l1_grow.activate_table', 'l2_load', 'l2_update', - 'l2_update_compressed', 'l2_alloc.cow_read', 'l2_alloc.write', + 'data': [ 'l1_update', 'l1_grow_alloc_table', 'l1_grow_write_table', + 'l1_grow_activate_table', 'l2_load', 'l2_update', + 'l2_update_compressed', 'l2_alloc_cow_read', 'l2_alloc_write', 'read_aio', 'read_backing_aio', 'read_compressed', 'write_aio', 'write_compressed', 'vmstate_load', 'vmstate_save', 'cow_read', 'cow_write', 'reftable_load', 'reftable_grow', 'reftable_update', 'refblock_load', 'refblock_update', 'refblock_update_part', - 'refblock_alloc', 'refblock_alloc.hookup', 'refblock_alloc.write', - 'refblock_alloc.write_blocks', 'refblock_alloc.write_table', - 'refblock_alloc.switch_table', 'cluster_alloc', + 'refblock_alloc', 'refblock_alloc_hookup', 'refblock_alloc_write', + 'refblock_alloc_write_blocks', 'refblock_alloc_write_table', + 'refblock_alloc_switch_table', 'cluster_alloc', 'cluster_alloc_bytes', 'cluster_free', 'flush_to_os', - 'flush_to_disk', 'pwritev_rmw.head', 'pwritev_rmw.after_head', - 'pwritev_rmw.tail', 'pwritev_rmw.after_tail', 'pwritev', + 'flush_to_disk', 'pwritev_rmw_head', 'pwritev_rmw_after_head', + 'pwritev_rmw_tail', 'pwritev_rmw_after_tail', 'pwritev', 'pwritev_zero', 'pwritev_done', 'empty_image_prepare' ] } ##
tests/qemu-iotests/026 +9 -9
@@ -66,7 +66,7 @@ for event in \ \ l2_load \ l2_update \ - l2_alloc.write \ + l2_alloc_write \ \ write_aio \ \
@@ -126,11 +126,11 @@ CLUSTER_SIZE=512 for event in \ - refblock_alloc.hookup \ - refblock_alloc.write \ - refblock_alloc.write_blocks \ - refblock_alloc.write_table \ - refblock_alloc.switch_table \ + refblock_alloc_hookup \ + refblock_alloc_write \ + refblock_alloc_write_blocks \ + refblock_alloc_write_table \ + refblock_alloc_switch_table \ do
@@ -170,9 +170,9 @@ CLUSTER_SIZE=1024 for event in \ - l1_grow.alloc_table \ - l1_grow.write_table \ - l1_grow.activate_table \ + l1_grow_alloc_table \ + l1_grow_write_table \ + l1_grow_activate_table \ do
tests/qemu-iotests/026.out +40 -40
280 lines not on this page · raw message
tests/qemu-iotests/026.out.nocache +40 -40
264 lines not on this page · raw message
tests/qemu-iotests/077 +12 -12
@@ -63,7 +63,7 @@ EOF off=0x1000 for ev in "head" "after_head" "tail" "after_tail"; do cat <<EOF -break pwritev_rmw.$ev A +break pwritev_rmw_$ev A aio_write -P 10 $((off + 0x200)) 0x200 wait_break A aio_write -P 11 $((off + 0x400)) 0x200
@@ -76,7 +76,7 @@ done # Chained dependencies cat <<EOF -break pwritev_rmw.after_tail A +break pwritev_rmw_after_tail A aio_write -P 10 0x5000 0x200 wait_break A aio_write -P 11 0x5200 0x200
@@ -93,10 +93,10 @@ EOF # Overlapping multiple requests cat <<EOF -break pwritev_rmw.after_tail A +break pwritev_rmw_after_tail A aio_write -P 10 0x6000 0x200 wait_break A -break pwritev_rmw.after_head B +break pwritev_rmw_after_head B aio_write -P 10 0x7e00 0x200 wait_break B aio_write -P 11 0x6800 0x1000
@@ -107,10 +107,10 @@ aio_flush EOF cat <<EOF -break pwritev_rmw.after_tail A +break pwritev_rmw_after_tail A aio_write -P 10 0x8000 0x200 wait_break A -break pwritev_rmw.after_head B +break pwritev_rmw_after_head B aio_write -P 10 0x9e00 0x200 wait_break B aio_write -P 11 0x8800 0x1000
@@ -121,11 +121,11 @@ aio_flush EOF cat <<EOF -break pwritev_rmw.after_tail A +break pwritev_rmw_after_tail A aio_write -P 10 0xa000 0x200 wait_break A aio_write -P 11 0xa800 0x1000 -break pwritev_rmw.after_head B +break pwritev_rmw_after_head B aio_write -P 10 0xbe00 0x200 wait_break B resume A
@@ -135,11 +135,11 @@ aio_flush EOF cat <<EOF -break pwritev_rmw.after_tail A +break pwritev_rmw_after_tail A aio_write -P 10 0xc000 0x200 wait_break A aio_write -P 11 0xc800 0x1000 -break pwritev_rmw.after_head B +break pwritev_rmw_after_head B aio_write -P 10 0xde00 0x200 wait_break B resume B
@@ -150,7 +150,7 @@ EOF # Only RMW for the tail part cat <<EOF -break pwritev_rmw.after_tail A +break pwritev_rmw_after_tail A aio_write -P 10 0xe000 0x1800 wait_break A aio_write -P 11 0xf000 0xc00
@@ -163,7 +163,7 @@ cat <<EOF break pwritev A aio_write -P 10 0x10000 0x800 wait_break A -break pwritev_rmw.after_tail B +break pwritev_rmw_after_tail B aio_write -P 11 0x10000 0x400 break pwritev_done C resume A
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 20/36] blkdebug: Avoid '.' in enum values
git-am complains about the trailing whitespace in expected test output. Not this patch's fault. Would be nice to avoid, but not in this series.
PATCH v12 21/36 qapi: Tighten the regex on valid names
In reply to Eric BlakeWe already documented that qapi names should match specific patterns (such as starting with a letter unless it was an enum value or a downstream extension). Tighten that from a suggestion into a hard requirement, which frees up names beginning with a single underscore for qapi internal usage. Also restrict things to avoid 'a__b' or 'a--b' (that is, dash and underscore must not occur consecutively).
Add a new test, reserved-member-underscore, to demonstrate the tighter checking.
---
v12: new patch
--- docs/qapi-code-gen.txt | 19 ++++++++++--------- scripts/qapi.py | 12 +++++++----- tests/Makefile | 1 + tests/qapi-schema/reserved-member-underscore.err | 1 + tests/qapi-schema/reserved-member-underscore.exit | 1 + tests/qapi-schema/reserved-member-underscore.json | 4 ++++ tests/qapi-schema/reserved-member-underscore.out | 0 7 files changed, 24 insertions(+), 14 deletions(-) create mode 100644 tests/qapi-schema/reserved-member-underscore.err create mode 100644 tests/qapi-schema/reserved-member-underscore.exit create mode 100644 tests/qapi-schema/reserved-member-underscore.json create mode 100644 tests/qapi-schema/reserved-member-underscore.out
docs/qapi-code-gen.txt +10 -9
@@ -118,17 +118,18 @@ tracking optional fields. Any name (command, event, type, field, or enum value) beginning with "x-" is marked experimental, and may be withdrawn or changed -incompatibly in a future release. Downstream vendors may add -extensions; such extensions should begin with a prefix matching +incompatibly in a future release. All names must begin with a letter, +and contain only ASCII letters, digits, dash, and underscore, where +dash and underscore cannot occur consecutively. There are two +exceptions: enum values may start with a digit, and any extensions +added by downstream vendors should start with a prefix matching "__RFQDN_" (for the reverse-fully-qualified-domain-name of the vendor), even if the rest of the name uses dash (example: -__com.redhat_drive-mirror). Other than downstream extensions (with -leading underscore and the use of dots), all names should begin with a -letter, and contain only ASCII letters, digits, dash, and underscore. -Names beginning with 'q_' are reserved for the generator: QMP names -that resemble C keywords or other problematic strings will be munged -in C to use this prefix. For example, a field named "default" in -qapi becomes "q_default" in the generated C code. +__com.redhat_drive-mirror). Names beginning with 'q_' are reserved +for the generator: QMP names that resemble C keywords or other +problematic strings will be munged in C to use this prefix. For +example, a field named "default" in qapi becomes "q_default" in the +generated C code. In the rest of this document, usage lines are given for each expression type, with literal strings written in lower case and
scripts/qapi.py +7 -5
@@ -353,9 +353,11 @@ def discriminator_find_enum_define(expr): return find_enum(discriminator_type) -# FIXME should enforce "other than downstream extensions [...], all -# names should begin with a letter". -valid_name = re.compile('^[a-zA-Z_][a-zA-Z0-9_.-]*$') +# Names must be letters, numbers, -, and _. They must start with letter, +# except for downstream extensions which must start with __RFQDN_. +# Dots are only valid in the downstream extension prefix. +valid_name = re.compile('^(__[a-zA-Z][a-zA-Z0-9.]*_)?' + '[a-zA-Z][a-zA-Z0-9]*([_-][a-zA-Z0-9]+)*$') def check_name(expr_info, source, name, allow_optional=False,
@@ -374,8 +376,8 @@ def check_name(expr_info, source, name, allow_optional=False, % (source, name)) # Enum members can start with a digit, because the generated C # code always prefixes it with the enum name - if enum_member: - membername = '_' + membername + if enum_member and membername[0].isdigit(): + membername = 'D' + membername # Reserve the entire 'q_' namespace for c_name() if not valid_name.match(membername) or \ c_name(membername, False).startswith('q_'):
tests/Makefile +1 -0
@@ -320,6 +320,7 @@ qapi-schema += reserved-command-q.json qapi-schema += reserved-member-has.json qapi-schema += reserved-member-q.json qapi-schema += reserved-member-u.json +qapi-schema += reserved-member-underscore.json qapi-schema += reserved-type-kind.json qapi-schema += reserved-type-list.json qapi-schema += returns-alternate.json
tests/qapi-schema/reserved-member-underscore.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/reserved-member-underscore.json:4: Member of 'data' for struct 'Oops' uses invalid name '_oops'
tests/qapi-schema/reserved-member-underscore.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/reserved-member-underscore.json new file +4 -0
@@ -0,0 +1,4 @@ +# C member name collision +# We reject use of a single leading underscore in all names (names must +# begin with a letter or a downstream extension double-underscore prefix). +{ 'struct': 'Oops', 'data': { '_oops': 'str' } }
tests/qapi-schema/reserved-member-underscore.out new file +0 -0
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 21/36] qapi: Tighten the regex on valid names
Quoted 5 lines · Eric Blake
We already documented that qapi names should match specific patterns (such as starting with a letter unless it was an enum value or a downstream extension). Tighten that from a suggestion into a hard requirement, which frees up names beginning with a single underscore for qapi internal usage.
If we care enough about naming conventions to document them, enforcing them makes only sense.
Quoted 3 lines
Also restrict things
to avoid 'a__b' or 'a--b' (that is, dash and underscore must not occur consecutively).
I feel this is entering "foolish names" territory, where things like "IcAnFiNdNaMeSeVeNlEsSrEaDaBlEtHaNStudlyCaps" live. Catching fools automatically is generally futile, they're too creative :)
Let's drop this part.
Quoted 59 lines
Add a new test, reserved-member-underscore, to demonstrate the tighter checking.
Signed-off-by Eric Blake <[email protected]>---
v12: new patch--- docs/qapi-code-gen.txt | 19 ++++++++++--------- scripts/qapi.py | 12 +++++++----- tests/Makefile | 1 + tests/qapi-schema/reserved-member-underscore.err | 1 + tests/qapi-schema/reserved-member-underscore.exit | 1 + tests/qapi-schema/reserved-member-underscore.json | 4 ++++ tests/qapi-schema/reserved-member-underscore.out | 0 7 files changed, 24 insertions(+), 14 deletions(-) create mode 100644 tests/qapi-schema/reserved-member-underscore.err create mode 100644 tests/qapi-schema/reserved-member-underscore.exit create mode 100644 tests/qapi-schema/reserved-member-underscore.json create mode 100644 tests/qapi-schema/reserved-member-underscore.outdocs/qapi-code-gen.txt +10
-9@@ -118,17 +118,18 @@ tracking optional fields. Any name (command, event, type, field, or enum value) beginning with "x-" is marked experimental, and may be withdrawn or changed -incompatibly in a future release. Downstream vendors may add -extensions; such extensions should begin with a prefix matching +incompatibly in a future release. All names must begin with a letter, +and contain only ASCII letters, digits, dash, and underscore, where +dash and underscore cannot occur consecutively. There are two +exceptions: enum values may start with a digit, and any extensions +added by downstream vendors should start with a prefix matching "__RFQDN_" (for the reverse-fully-qualified-domain-name of the vendor), even if the rest of the name uses dash (example: -__com.redhat_drive-mirror). Other than downstream extensions (with -leading underscore and the use of dots), all names should begin with a -letter, and contain only ASCII letters, digits, dash, and underscore. -Names beginning with 'q_' are reserved for the generator: QMP names -that resemble C keywords or other problematic strings will be munged -in C to use this prefix. For example, a field named "default" in -qapi becomes "q_default" in the generated C code. +__com.redhat_drive-mirror). Names beginning with 'q_' are reserved +for the generator: QMP names that resemble C keywords or other +problematic strings will be munged in C to use this prefix. For +example, a field named "default" in qapi becomes "q_default" in the +generated C code. In the rest of this document, usage lines are given for each expression type, with literal strings written in lower case andscripts/qapi.py +0
-0@@ -353,9 +353,11 @@ def discriminator_find_enum_define(expr): return find_enum(discriminator_type)
scripts/qapi.py def discriminator_find_enum_define(expr): · line 360
Quoted 10 lines
scripts/qapi.py +5
-3-# FIXME should enforce "other than downstream extensions [...], all -# names should begin with a letter". -valid_name = re.compile('^[a-zA-Z_][a-zA-Z0-9_.-]*$') +# Names must be letters, numbers, -, and _. They must start with letter, +# except for downstream extensions which must start with __RFQDN_. +# Dots are only valid in the downstream extension prefix. +valid_name = re.compile('^(__[a-zA-Z][a-zA-Z0-9.]*_)?' + '[a-zA-Z][a-zA-Z0-9]*([_-][a-zA-Z0-9]+)*$')
This regexp consists of two parts: optional __RFQDN_ prefix and the name proper.
The latter stays simpler if we don't attempt to catch adjacent [-_].
The former isn't quite right. According to RFC 822 Appendix 1 - Domain Name Syntax Specification:
* We must accept '-' in addition to digits.
* We require RFQDN to start with a letter. This is still a loose match. The tight match is "labels separated by dots", where labels consist of letters, digits and '-', starting with a letter. I wouldn't bother checking first characters are letters at all.
Recommend
valid_name = re.compile('^(__[a-zA-Z0-9.-]+_)?'
'[a-zA-Z][a-zA-Z0-9_-]*$')
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 379
Quoted 10 lines
scripts/qapi.py +1
-2def check_name(expr_info, source, name, allow_optional=False,@@ -374,8 +376,8 @@ def check_name(expr_info, source, name, allow_optional=False, % (source, name)) # Enum members can start with a digit, because the generated C # code always prefixes it with the enum name - if enum_member: - membername = '_' + membername + if enum_member and membername[0].isdigit():
What's wrong with the old condition?
Quoted 43 lines
scripts/qapi.py +1
-0+ membername = 'D' + membername # Reserve the entire 'q_' namespace for c_name() if not valid_name.match(membername) or \ c_name(membername, False).startswith('q_'):tests/Makefile +1
-0@@ -320,6 +320,7 @@ qapi-schema += reserved-command-q.json qapi-schema += reserved-member-has.json qapi-schema += reserved-member-q.json qapi-schema += reserved-member-u.json +qapi-schema += reserved-member-underscore.json qapi-schema += reserved-type-kind.json qapi-schema += reserved-type-list.json qapi-schema += returns-alternate.jsontests/qapi-schema/reserved-member-underscore.err new file +1
-0@@ -0,0 +1 @@ +tests/qapi-schema/reserved-member-underscore.json:4: Member of 'data' for struct 'Oops' uses invalid name '_oops'tests/qapi-schema/reserved-member-underscore.exit new file +1
-0@@ -0,0 +1 @@ +1tests/qapi-schema/reserved-member-underscore.json new file +4
-0@@ -0,0 +1,4 @@ +# C member name collision +# We reject use of a single leading underscore in all names (names must +# begin with a letter or a downstream extension double-underscore prefix). +{ 'struct': 'Oops', 'data': { '_oops': 'str' } }tests/qapi-schema/reserved-member-underscore.out new file +0
-0
Re: [Qemu-devel] [PATCH v12 21/36] qapi: Tighten the regex on valid names
Quoted 11 lines · Markus Armbruster
Quoted 5 lines · Eric Blake
We already documented that qapi names should match specific patterns (such as starting with a letter unless it was an enum value or a downstream extension). Tighten that from a suggestion into a hard requirement, which frees up names beginning with a single underscore for qapi internal usage.
If we care enough about naming conventions to document them, enforcing them makes only sense.
Quoted 6 lines
Quoted 3 lines
Also restrict things
to avoid 'a__b' or 'a--b' (that is, dash and underscore must not occur consecutively).I feel this is entering "foolish names" territory, where things like "IcAnFiNdNaMeSeVeNlEsSrEaDaBlEtHaNStudlyCaps" live. Catching fools
IhAdToOmUcHfUnReAdInGtHaT
[It's amazing that scholars can ever manage to correctly interpret old written languages, thanks to omitted word breaks (scriptio continua) or omitted vowels (such as in Hebrew)]
Quoted 3 lines
automatically is generally futile, they're too creative :)
Let's drop this part.
Sure, I can do that. I'll post a fixup patch, as it will affect docs too.
Quoted 6 lines
Quoted 6 lines
Add a new test, reserved-member-underscore, to demonstrate the tighter checking.
Signed-off-by Eric Blake <[email protected]>
Quoted 7 lines
docs/qapi-code-gen.txt tracking optional fields. · line 123
Quoted 7 lines
---
v12: new patch
---docs/qapi-code-gen.txt +3
-0+incompatibly in a future release. All names must begin with a letter, +and contain only ASCII letters, digits, dash, and underscore, where +dash and underscore cannot occur consecutively. There are two
Namely, right here.
Quoted 21 lines
scripts/qapi.py def discriminator_find_enum_define(expr): · line 360
Quoted 5 lines
scripts/qapi.py +5
-0+# Names must be letters, numbers, -, and _. They must start with letter, +# except for downstream extensions which must start with __RFQDN_. +# Dots are only valid in the downstream extension prefix. +valid_name = re.compile('^(__[a-zA-Z][a-zA-Z0-9.]*_)?' + '[a-zA-Z][a-zA-Z0-9]*([_-][a-zA-Z0-9]+)*$')This regexp consists of two parts: optional
__RFQDN_prefix and the name proper.The latter stays simpler if we don't attempt to catch adjacent [-_].
The former isn't quite right. According to RFC 822 Appendix 1 - Domain Name Syntax Specification:
* We must accept '-' in addition to digits.
* We require RFQDN to start with a letter. This is still a loose match. The tight match is "labels separated by dots", where labels consist of letters, digits and '-', starting with a letter. I wouldn't bother checking first characters are letters at all.
Quoted 4 lines
Recommend
valid_name = re.compile('^(__[a-zA-Z0-9.-]+_)?' '[a-zA-Z][a-zA-Z0-9_-]*$')
Sure, that works for me. It's tighter than what we had before, but looser than what I proposed so that it allows more RFQDN. It potentially lets users confuse us by abusing 'foo__max' or similar, but I'm less worried about that abuse - it's okay to stop the blatantly obvious mistakes without worrying about the corner cases, if it makes the maintenance easier for the cases we do catch.
Quoted 15 lines
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 379
Quoted 11 lines
def check_name(expr_info, source, name, allow_optional=False,
scripts/qapi.py +1
-2@@ -374,8 +376,8 @@ def check_name(expr_info, source, name, allow_optional=False, % (source, name)) # Enum members can start with a digit, because the generated C # code always prefixes it with the enum name - if enum_member: - membername = '_' + membername + if enum_member and membername[0].isdigit():What's wrong with the old condition?
Quoted 1 line
scripts/qapi.py +1
-0+ membername = 'D' + membername
The old code prepended a lone '_', which doesn't work any more with the tighter valid_name regex, so we have to use some other prefix (I picked 'D').
Quoted 3 lines
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 383
Quoted 3 lines
scripts/qapi.py +0
-0# Reserve the entire 'q_' namespace for c_name() if not valid_name.match(membername) or \ c_name(membername, False).startswith('q_'):
The other thing is that I realized that an enum value of 'q-int' was getting munged to '_q-int' which no longer gets flagged by the c_name() filter. But neither would 'Dq-int' get flagged. So limiting the munging to just enum values that start with a digit (where we know it doesn't start with q) means we don't weaken the second condition.
I guess I need to call that out in the commit message; all the more reason for me to send a fixup.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 21/36] qapi: Tighten the regex on valid names
Quoted 16 lines · Eric Blake
Quoted 15 lines · Markus Armbruster
Quoted 5 lines · Eric Blake
We already documented that qapi names should match specific patterns (such as starting with a letter unless it was an enum value or a downstream extension). Tighten that from a suggestion into a hard requirement, which frees up names beginning with a single underscore for qapi internal usage.
If we care enough about naming conventions to document them, enforcing them makes only sense.
Quoted 3 lines
Also restrict things
to avoid 'a__b' or 'a--b' (that is, dash and underscore must not occur consecutively).
Quoted 8 lines
Quoted 2 lines
I feel this is entering "foolish names" territory, where things like "IcAnFiNdNaMeSeVeNlEsSrEaDaBlEtHaNStudlyCaps" live. Catching fools
IhAdToOmUcHfUnReAdInGtHaT
[It's amazing that scholars can ever manage to correctly interpret old written languages, thanks to omitted word breaks (scriptio continua) or omitted vowels (such as in Hebrew)]
Indeed.
Quoted 79 lines
Quoted 3 lines
automatically is generally futile, they're too creative :)
Let's drop this part.
Sure, I can do that. I'll post a fixup patch, as it will affect docs too.
Quoted 6 lines
Quoted 6 lines
Add a new test, reserved-member-underscore, to demonstrate the tighter checking.
Signed-off-by Eric Blake <[email protected]>Quoted 7 lines
docs/qapi-code-gen.txt tracking optional fields. · line 123
Quoted 7 lines
---
v12: new patch
---docs/qapi-code-gen.txt +3
-0+incompatibly in a future release. All names must begin with a letter, +and contain only ASCII letters, digits, dash, and underscore, where +dash and underscore cannot occur consecutively. There are twoNamely, right here.
Quoted 21 lines
scripts/qapi.py def discriminator_find_enum_define(expr): · line 360
Quoted 5 lines
scripts/qapi.py +5
-0+# Names must be letters, numbers, -, and _. They must start with letter, +# except for downstream extensions which must start with __RFQDN_. +# Dots are only valid in the downstream extension prefix. +valid_name = re.compile('^(__[a-zA-Z][a-zA-Z0-9.]*_)?' + '[a-zA-Z][a-zA-Z0-9]*([_-][a-zA-Z0-9]+)*$')This regexp consists of two parts: optional
__RFQDN_prefix and the name proper.The latter stays simpler if we don't attempt to catch adjacent [-_].
The former isn't quite right. According to RFC 822 Appendix 1 - Domain Name Syntax Specification:
* We must accept '-' in addition to digits.
* We require RFQDN to start with a letter. This is still a loose match. The tight match is "labels separated by dots", where labels consist of letters, digits and '-', starting with a letter. I wouldn't bother checking first characters are letters at all.
Quoted 4 lines
Recommend
valid_name = re.compile('^(__[a-zA-Z0-9.-]+_)?' '[a-zA-Z][a-zA-Z0-9_-]*$')Sure, that works for me. It's tighter than what we had before, but looser than what I proposed so that it allows more RFQDN. It potentially lets users confuse us by abusing '
foo__max' or similar, but I'm less worried about that abuse - it's okay to stop the blatantly obvious mistakes without worrying about the corner cases, if it makes the maintenance easier for the cases we do catch.Quoted 15 lines
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 379
Quoted 11 lines
def check_name(expr_info, source, name, allow_optional=False,
scripts/qapi.py +1
-2@@ -374,8 +376,8 @@ def check_name(expr_info, source, name, allow_optional=False, % (source, name)) # Enum members can start with a digit, because the generated C # code always prefixes it with the enum name - if enum_member: - membername = '_' + membername + if enum_member and membername[0].isdigit():What's wrong with the old condition?
Quoted 1 line
scripts/qapi.py +1
-0+ membername = 'D' + membernameThe old code prepended a lone '_', which doesn't work any more with the tighter
valid_nameregex, so we have to use some other prefix (I picked 'D').Quoted 3 lines
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 383
Quoted 3 lines
scripts/qapi.py +0
-0# Reserve the entire 'q_' namespace for c_name() if not valid_name.match(membername) or \ c_name(membername, False).startswith('q_'):
Quoted 5 lines
The other thing is that I realized that an enum value of 'q-int' was getting munged to '_q-int' which no longer gets flagged by the
c_name()filter. But neither would 'Dq-int' get flagged. So limiting the munging to just enum values that start with a digit (where we know it doesn't start with q) means we don't weaken the second condition.
Aha! Worth a comment, I think.
Quoted 2 lines
I guess I need to call that out in the commit message; all the more reason for me to send a fixup.
Yes, please!
PATCH 1/1 fixup! qapi: Tighten the regex on valid names
In reply to Eric Blake[Replace the old commit message with this:]
qapi: Tighten the regex on valid names
We already documented that qapi names should match specific patterns (such as starting with a letter unless it was an enum value or a downstream extension). Tighten that from a suggestion into a hard requirement, which frees up names beginning with a single underscore for qapi internal usage.
The tighter regex doesn't forbid everything insane that a user could provide (for example, a user could name a type 'Foo-lookup' to collide with the generated 'Foo_lookup[]' for an enum 'Foo'), but does a good job at protecting the most obvious uses, and also happens to reserve single leading underscore for later use.
The handling of enum values starting with a digit is tricky:
commit 9fb081e introduced a subtle bug by using c_name() on a munged value, which would allow an enum to include the member 'q-int' in spite of our reservation. Furthermore, munging with a leading '_' would fail our tighter regex. So fix it by only munging for leading digits (which are never ticklish in c_name()) and by using a different prefix (I picked 'D', although any letter should do).
Add new tests, reserved-member-underscore and reserved-enum-q, to demonstrate the tighter checking.
--- docs/qapi-code-gen.txt | 11 +++++------ scripts/qapi.py | 4 ++-- tests/Makefile | 1 + tests/qapi-schema/reserved-enum-q.err | 1 + tests/qapi-schema/reserved-enum-q.exit | 1 + tests/qapi-schema/reserved-enum-q.json | 4 ++++ tests/qapi-schema/reserved-enum-q.out | 0 7 files changed, 14 insertions(+), 8 deletions(-) create mode 100644 tests/qapi-schema/reserved-enum-q.err create mode 100644 tests/qapi-schema/reserved-enum-q.exit create mode 100644 tests/qapi-schema/reserved-enum-q.json create mode 100644 tests/qapi-schema/reserved-enum-q.out
docs/qapi-code-gen.txt +5 -6
@@ -119,12 +119,11 @@ tracking optional fields. Any name (command, event, type, field, or enum value) beginning with "x-" is marked experimental, and may be withdrawn or changed incompatibly in a future release. All names must begin with a letter, -and contain only ASCII letters, digits, dash, and underscore, where -dash and underscore cannot occur consecutively. There are two -exceptions: enum values may start with a digit, and any extensions -added by downstream vendors should start with a prefix matching -"__RFQDN_" (for the reverse-fully-qualified-domain-name of the -vendor), even if the rest of the name uses dash (example: +and contain only ASCII letters, digits, dash, and underscore. There +are two exceptions: enum values may start with a digit, and any +extensions added by downstream vendors should start with a prefix +matching "__RFQDN_" (for the reverse-fully-qualified-domain-name of +the vendor), even if the rest of the name uses dash (example: __com.redhat_drive-mirror). Names beginning with 'q_' are reserved for the generator: QMP names that resemble C keywords or other problematic strings will be munged in C to use this prefix. For
scripts/qapi.py +2 -2
@@ -356,8 +356,8 @@ def discriminator_find_enum_define(expr): # Names must be letters, numbers, -, and _. They must start with letter, # except for downstream extensions which must start with __RFQDN_. # Dots are only valid in the downstream extension prefix. -valid_name = re.compile('^(__[a-zA-Z][a-zA-Z0-9.]*_)?' - '[a-zA-Z][a-zA-Z0-9]*([_-][a-zA-Z0-9]+)*$') +valid_name = re.compile('^(__[a-zA-Z0-9.-]+_)?' + '[a-zA-Z][a-zA-Z0-9_-]*$') def check_name(expr_info, source, name, allow_optional=False,
tests/Makefile +1 -0
@@ -317,6 +317,7 @@ qapi-schema += redefined-command.json qapi-schema += redefined-event.json qapi-schema += redefined-type.json qapi-schema += reserved-command-q.json +qapi-schema += reserved-enum-q.json qapi-schema += reserved-member-has.json qapi-schema += reserved-member-q.json qapi-schema += reserved-member-u.json
tests/qapi-schema/reserved-enum-q.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/reserved-enum-q.json:4: Member of enum 'Foo' uses invalid name 'q-Unix'
tests/qapi-schema/reserved-enum-q.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/reserved-enum-q.json new file +4 -0
@@ -0,0 +1,4 @@ +# C entity name collision +# We reject names like 'q-unix', because they can collide with the mangled +# name for 'unix' in generated C. +{ 'enum': 'Foo', 'data': [ 'unix', 'q-Unix' ] }
tests/qapi-schema/reserved-enum-q.out new file +0 -0
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH] fixup! qapi: Tighten the regex on valid names
Quoted 1 line · Eric Blake
[Replace the old commit message with this:]
Fixup patch squashed, commit message replaced.
Re: [Qemu-devel] [PATCH] fixup! qapi: Tighten the regex on valid names
Quoted 5 lines · Markus Armbruster
Quoted 1 line · Eric Blake
[Replace the old commit message with this:]
Fixup patch squashed, commit message replaced.
The version of the patch currently on qapi-next (id 22c38fe) missed the addition of the new tests/qapi-schema/reserved-enum-q.* files but caught the Makefile change, which means it fails 'make check-qapi-schema'.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH] fixup! qapi: Tighten the regex on valid names
Quoted 10 lines · Eric Blake
Quoted 5 lines · Markus Armbruster
Quoted 1 line · Eric Blake
[Replace the old commit message with this:]
Fixup patch squashed, commit message replaced.
The version of the patch currently on qapi-next (id 22c38fe) missed the addition of the new tests/qapi-schema/reserved-enum-q.* files but caught the
Makefilechange, which means it fails 'make check-qapi-schema'.
I suck at squashing patches. Fixed & Thanks!
PATCH v12 22/36 qapi: Don't let implicit enum MAX member collide
In reply to Eric BlakeNow that we guarantee the user doesn't have any enum values beginning with a single underscore, we can use that for our own purposes. Renaming ENUM_MAX to ENUM__MAX makes it obvious that the sentinel is generated.
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
| max_index = c_enum_const(name, 'MAX', prefix)
| ret += mcgen('''
| [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
| max_index=max_index)then running:
$ cat qapi-{types,event}.c tests/test-qapi-types.c |
sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f listThe only things not generated are the changes in scripts/qapi.py.
---
v12: new patch
--- block/blkdebug.c | 10 +++++----- block/parallels.c | 4 ++-- block/qcow2.c | 2 +- block/quorum.c | 2 +- block/raw-posix.c | 2 +- blockdev.c | 2 +- docs/qapi-code-gen.txt | 4 ++-- hmp.c | 14 +++++++------- hw/char/escc.c | 2 +- hw/i386/pc_piix.c | 2 +- hw/i386/pc_q35.c | 2 +- hw/input/hid.c | 2 +- hw/input/ps2.c | 2 +- hw/input/virtio-input-hid.c | 8 ++++---- include/migration/migration.h | 4 ++-- migration/migration.c | 4 ++-- monitor.c | 14 +++++++------- net/net.c | 4 ++-- qemu-nbd.c | 2 +- replay/replay-input.c | 8 ++++---- scripts/qapi.py | 6 +++--- tests/test-qmp-output-visitor.c | 6 +++--- tests/test-string-output-visitor.c | 4 ++-- tpm.c | 10 +++++----- ui/cocoa.m | 2 +- ui/console.c | 2 +- ui/input-keymap.c | 4 ++-- ui/input-legacy.c | 6 +++--- ui/input.c | 6 +++--- ui/sdl.c | 2 +- ui/sdl2.c | 2 +- ui/spice-input.c | 2 +- ui/vnc.c | 2 +- vl.c | 18 +++++++++--------- 34 files changed, 83 insertions(+), 83 deletions(-)
block/blkdebug.c +5 -5
@@ -35,7 +35,7 @@ typedef struct BDRVBlkdebugState {int state;int new_state; - QLIST_HEAD(, BlkdebugRule) rules[BLKDBG_MAX]; + QLIST_HEAD(, BlkdebugRule) rules[BLKDBG__MAX]; QSIMPLEQ_HEAD(, BlkdebugRule) active_rules; QLIST_HEAD(, BlkdebugSuspendedReq) suspended_reqs; } BDRVBlkdebugState;
@@ -146,7 +146,7 @@ static int get_event_by_name(const char *name, BlkdebugEvent *event) {int i; -for (i =0 ; i <BLKDBG_MAX ; i++) { +for (i =0 ; i <BLKDBG__MAX ; i++) {if (!strcmp (BlkdebugEvent_lookup[i], name)) { *event = i;return 0 ;
@@ -506,7 +506,7 @@ static void blkdebug_close(BlockDriverState *bs)BlkdebugRule *rule, *next;int i; -for (i =0 ; i <BLKDBG_MAX ; i++) { +for (i =0 ; i <BLKDBG__MAX ; i++) { QLIST_FOREACH_SAFE(rule, &s->rules[i], next, next) {remove_rule (rule); }
@@ -571,7 +571,7 @@ static void blkdebug_debug_event(BlockDriverState *bs, BlkdebugEvent event)struct BlkdebugRule *rule, *next;bool injected; -assert ((int )event >=0 && event <BLKDBG_MAX ); +assert ((int )event >=0 && event <BLKDBG__MAX ); injected = false; s->new_state = s->state;
@@ -628,7 +628,7 @@ static int blkdebug_debug_remove_breakpoint(BlockDriverState *bs,BlkdebugRule *rule, *next;int i, ret = -ENOENT ; -for (i =0 ; i <BLKDBG_MAX ; i++) { +for (i =0 ; i <BLKDBG__MAX ; i++) { QLIST_FOREACH_SAFE(rule, &s->rules[i], next, next) {if (rule->action ==ACTION_SUSPEND && !strcmp (rule->options.suspend.tag, tag)) {
block/parallels.c +2 -2
@@ -61,7 +61,7 @@ typedef struct ParallelsHeader {typedef enum ParallelsPreallocMode {PRL_PREALLOC_MODE_FALLOCATE =0 ,PRL_PREALLOC_MODE_TRUNCATE =1 , -PRL_PREALLOC_MODE_MAX =2 , +PRL_PREALLOC_MODE__MAX =2 , }ParallelsPreallocMode ;static const char *prealloc_mode_lookup[] = {
@@ -660,7 +660,7 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, s->prealloc_size =MAX (s->tracks, s->prealloc_size >>BDRV_SECTOR_BITS ); buf =qemu_opt_get_del (opts,PARALLELS_OPT_PREALLOC_MODE ); s->prealloc_mode =qapi_enum_parse (prealloc_mode_lookup, buf, -PRL_PREALLOC_MODE_MAX ,PRL_PREALLOC_MODE_FALLOCATE , &local_err); +PRL_PREALLOC_MODE__MAX ,PRL_PREALLOC_MODE_FALLOCATE , &local_err);g_free (buf);if (local_err !=NULL ) { goto fail_options;
block/qcow2.c +1 -1
@@ -2269,7 +2269,7 @@ static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp) DEFAULT_CLUSTER_SIZE); buf =qemu_opt_get_del (opts,BLOCK_OPT_PREALLOC ); prealloc =qapi_enum_parse (PreallocMode_lookup, buf, -PREALLOC_MODE_MAX ,PREALLOC_MODE_OFF , +PREALLOC_MODE__MAX ,PREALLOC_MODE_OFF , &local_err);if (local_err) {error_propagate (errp, local_err);
block/quorum.c +1 -1
@@ -847,7 +847,7 @@ static int parse_read_pattern(const char *opt)return QUORUM_READ_PATTERN_QUORUM ; } -for (i =0 ; i <QUORUM_READ_PATTERN_MAX ; i++) { +for (i =0 ; i <QUORUM_READ_PATTERN__MAX ; i++) {if (!strcmp (opt, QuorumReadPattern_lookup[i])) {return i; }
block/raw-posix.c +1 -1
@@ -1636,7 +1636,7 @@ static int raw_create(const char *filename, QemuOpts *opts, Error **errp) nocow =qemu_opt_get_bool (opts,BLOCK_OPT_NOCOW , false); buf =qemu_opt_get_del (opts,BLOCK_OPT_PREALLOC ); prealloc =qapi_enum_parse (PreallocMode_lookup, buf, -PREALLOC_MODE_MAX ,PREALLOC_MODE_OFF , +PREALLOC_MODE__MAX ,PREALLOC_MODE_OFF , &local_err);g_free (buf);if (local_err) {
blockdev.c +1 -1
@@ -454,7 +454,7 @@ static void extract_common_blockdev_options(QemuOpts *opts, int *bdrv_flags, *detect_zeroes =qapi_enum_parse (BlockdevDetectZeroesOptions_lookup,qemu_opt_get (opts,"detect-zeroes" ), -BLOCKDEV_DETECT_ZEROES_OPTIONS_MAX , +BLOCKDEV_DETECT_ZEROES_OPTIONS__MAX ,BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF , &local_error);if (local_error) {
docs/qapi-code-gen.txt +2 -2
@@ -1054,7 +1054,7 @@ Example: const char *const example_QAPIEvent_lookup[] = { [EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT", - [EXAMPLE_QAPI_EVENT_MAX] = NULL, + [EXAMPLE_QAPI_EVENT__MAX] = NULL, }; $ cat qapi-generated/example-qapi-event.h [Uninteresting stuff omitted...]
@@ -1071,7 +1071,7 @@ Example: typedef enum example_QAPIEvent { EXAMPLE_QAPI_EVENT_MY_EVENT = 0, - EXAMPLE_QAPI_EVENT_MAX = 1, + EXAMPLE_QAPI_EVENT__MAX = 1, } example_QAPIEvent; extern const char *const example_QAPIEvent_lookup[];
hmp.c +7 -7
@@ -855,7 +855,7 @@ void hmp_info_tpm(Monitor *mon, const QDict *qdict) tpo->has_cancel_path ?",cancel-path=" :"" , tpo->has_cancel_path ? tpo->cancel_path : "");break ; -case TPM_TYPE_OPTIONS_KIND_MAX : +case TPM_TYPE_OPTIONS_KIND__MAX :break ; }monitor_printf (mon,"\n" );
@@ -1203,7 +1203,7 @@ void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict)MigrationCapabilityStatusList *caps =g_malloc0 (sizeof (*caps));int i; -for (i =0 ; i <MIGRATION_CAPABILITY_MAX ; i++) { +for (i =0 ; i <MIGRATION_CAPABILITY__MAX ; i++) {if (strcmp (cap, MigrationCapability_lookup[i]) ==0 ) { caps->value =g_malloc0 (sizeof (*caps->value)); caps->value->capability = i;
@@ -1214,7 +1214,7 @@ void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict) } } -if (i ==MIGRATION_CAPABILITY_MAX ) { +if (i ==MIGRATION_CAPABILITY__MAX ) {error_setg (&err,QERR_INVALID_PARAMETER , cap); }
@@ -1239,7 +1239,7 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)bool has_x_cpu_throttle_increment = false;int i; -for (i =0 ; i <MIGRATION_PARAMETER_MAX ; i++) { +for (i =0 ; i <MIGRATION_PARAMETER__MAX ; i++) {if (strcmp (param, MigrationParameter_lookup[i]) ==0 ) {switch (i) {case MIGRATION_PARAMETER_COMPRESS_LEVEL :
@@ -1268,7 +1268,7 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) } } -if (i ==MIGRATION_PARAMETER_MAX ) { +if (i ==MIGRATION_PARAMETER__MAX ) {error_setg (&err,QERR_INVALID_PARAMETER , param); }
@@ -1368,7 +1368,7 @@ void hmp_change(Monitor *mon, const QDict *qdict)if (read_only) { read_only_mode =qapi_enum_parse (BlockdevChangeReadOnlyMode_lookup, - read_only,BLOCKDEV_CHANGE_READ_ONLY_MODE_MAX , + read_only,BLOCKDEV_CHANGE_READ_ONLY_MODE__MAX ,BLOCKDEV_CHANGE_READ_ONLY_MODE_RETAIN , &err);if (err) {hmp_handle_error (mon, &err);
@@ -1771,7 +1771,7 @@ void hmp_sendkey(Monitor *mon, const QDict *qdict) keylist->value->u.number = value; }else {int idx =index_from_key (keyname_buf); -if (idx ==Q_KEY_CODE_MAX ) { +if (idx ==Q_KEY_CODE__MAX ) { goto err_out; } keylist->value->type =KEY_VALUE_KIND_QCODE ;
hw/char/escc.c +1 -1
@@ -714,7 +714,7 @@ MemoryRegion *escc_init(hwaddr base, qemu_irq irqA, qemu_irq irqB,return &d->mmio; } -static const uint8_t qcode_to_keycode[Q_KEY_CODE_MAX ] = { +static const uint8_t qcode_to_keycode[Q_KEY_CODE__MAX ] = { [Q_KEY_CODE_SHIFT ] =99 , [Q_KEY_CODE_SHIFT_R ] =110 , [Q_KEY_CODE_ALT ] =19 ,
hw/i386/pc_piix.c +1 -1
@@ -225,7 +225,7 @@ static void pc_init1(MachineState *machine,pc_vga_init (isa_bus, pci_enabled ? pci_bus :NULL ); -assert (pcms->vmport !=ON_OFF_AUTO_MAX ); +assert (pcms->vmport !=ON_OFF_AUTO__MAX );if (pcms->vmport ==ON_OFF_AUTO_AUTO ) { pcms->vmport =xen_enabled () ?ON_OFF_AUTO_OFF :ON_OFF_AUTO_ON ; }
hw/i386/pc_q35.c +1 -1
@@ -233,7 +233,7 @@ static void pc_q35_init(MachineState *machine)pc_register_ferr_irq (gsi[13 ]); -assert (pcms->vmport !=ON_OFF_AUTO_MAX ); +assert (pcms->vmport !=ON_OFF_AUTO__MAX );if (pcms->vmport ==ON_OFF_AUTO_AUTO ) { pcms->vmport =xen_enabled () ?ON_OFF_AUTO_OFF :ON_OFF_AUTO_ON ; }
hw/input/hid.c +1 -1
@@ -108,7 +108,7 @@ void hid_set_next_idle(HIDState *hs)static void hid_pointer_event (DeviceState *dev,QemuConsole *src,InputEvent *evt) { -static const int bmap[INPUT_BUTTON_MAX ] = { +static const int bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =0x01 , [INPUT_BUTTON_RIGHT ] =0x02 , [INPUT_BUTTON_MIDDLE ] =0x04 ,
hw/input/ps2.c +1 -1
@@ -382,7 +382,7 @@ static void ps2_mouse_send_packet(PS2MouseState *s)static void ps2_mouse_event (DeviceState *dev,QemuConsole *src,InputEvent *evt) { -static const int bmap[INPUT_BUTTON_MAX ] = { +static const int bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =MOUSE_EVENT_LBUTTON , [INPUT_BUTTON_MIDDLE ] =MOUSE_EVENT_MBUTTON , [INPUT_BUTTON_RIGHT ] =MOUSE_EVENT_RBUTTON ,
hw/input/virtio-input-hid.c +4 -4
@@ -21,7 +21,7 @@/* ----------------------------------------------------------------- */ -static const unsigned int keymap_qcode[Q_KEY_CODE_MAX ] = { +static const unsigned int keymap_qcode[Q_KEY_CODE__MAX ] = { [Q_KEY_CODE_ESC ] =KEY_ESC , [Q_KEY_CODE_1 ] =KEY_1 , [Q_KEY_CODE_2 ] =KEY_2 ,
@@ -138,7 +138,7 @@ static const unsigned int keymap_qcode[Q_KEY_CODE_MAX] = { [Q_KEY_CODE_MENU] = KEY_MENU, }; -static const unsigned int keymap_button[INPUT_BUTTON_MAX ] = { +static const unsigned int keymap_button[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =BTN_LEFT , [INPUT_BUTTON_RIGHT ] =BTN_RIGHT , [INPUT_BUTTON_MIDDLE ] =BTN_MIDDLE ,
@@ -146,12 +146,12 @@ static const unsigned int keymap_button[INPUT_BUTTON_MAX] = { [INPUT_BUTTON_WHEEL_DOWN] = BTN_GEAR_DOWN, }; -static const unsigned int axismap_rel[INPUT_AXIS_MAX ] = { +static const unsigned int axismap_rel[INPUT_AXIS__MAX ] = { [INPUT_AXIS_X ] =REL_X , [INPUT_AXIS_Y ] =REL_Y , }; -static const unsigned int axismap_abs[INPUT_AXIS_MAX ] = { +static const unsigned int axismap_abs[INPUT_AXIS__MAX ] = { [INPUT_AXIS_X ] =ABS_X , [INPUT_AXIS_Y ] =ABS_Y , };
include/migration/migration.h +2 -2
@@ -133,7 +133,7 @@ struct MigrationStateQemuThread thread;QEMUBH *cleanup_bh;QEMUFile *file; -int parameters[MIGRATION_PARAMETER_MAX ]; +int parameters[MIGRATION_PARAMETER__MAX ];int state;MigrationParams params;
@@ -151,7 +151,7 @@ struct MigrationStateint64_t expected_downtime;int64_t dirty_pages_rate;int64_t dirty_bytes_rate; -bool enabled_capabilities[MIGRATION_CAPABILITY_MAX ]; +bool enabled_capabilities[MIGRATION_CAPABILITY__MAX ];int64_t xbzrle_cache_size;int64_t setup_time;int64_t dirty_sync_count;
migration/migration.c +2 -2
@@ -202,7 +202,7 @@ static int global_state_post_load(void *opaque, int version_id) s->received = true;trace_migrate_global_state_post_load (runstate); - r =qapi_enum_parse (RunState_lookup, runstate,RUN_STATE_MAX , + r =qapi_enum_parse (RunState_lookup, runstate,RUN_STATE__MAX ,-1 , &local_err);if (r ==-1 ) {
@@ -479,7 +479,7 @@ MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp)int i; caps =NULL ;/* silence compiler warning */ -for (i =0 ; i <MIGRATION_CAPABILITY_MAX ; i++) { +for (i =0 ; i <MIGRATION_CAPABILITY__MAX ; i++) {if (head ==NULL ) { head =g_malloc0 (sizeof (*caps)); caps = head;
monitor.c +7 -7
@@ -441,7 +441,7 @@ static void monitor_protocol_emitter(Monitor *mon, QObject *data, } -static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT_MAX ] = { +static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT__MAX ] = {/* Limit guest-triggerable events to 1 per second */ [QAPI_EVENT_RTC_CHANGE ] = {1000 *SCALE_MS }, [QAPI_EVENT_WATCHDOG ] = {1000 *SCALE_MS },
@@ -481,7 +481,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, Error **errp)MonitorQAPIEventConf *evconf;MonitorQAPIEventState *evstate; -assert (event <QAPI_EVENT_MAX ); +assert (event <QAPI_EVENT__MAX ); evconf = &monitor_qapi_event_conf[event];trace_monitor_protocol_event_queue (event, qdict, evconf->rate);
@@ -946,7 +946,7 @@ EventInfoList *qmp_query_events(Error **errp)EventInfoList *info, *ev_list =NULL ;QAPIEvent e; -for (e =0 ; e <QAPI_EVENT_MAX ; e++) { +for (e =0 ; e <QAPI_EVENT__MAX ; e++) {const char *event_name = QAPIEvent_lookup[e];assert (event_name !=NULL ); info =g_malloc0 (sizeof (*info));
@@ -1386,7 +1386,7 @@ static void hmp_mouse_move(Monitor *mon, const QDict *qdict)static void hmp_mouse_button (Monitor *mon,const QDict *qdict) { -static uint32_t bmap[INPUT_BUTTON_MAX ] = { +static uint32_t bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =MOUSE_EVENT_LBUTTON , [INPUT_BUTTON_MIDDLE ] =MOUSE_EVENT_MBUTTON , [INPUT_BUTTON_RIGHT ] =MOUSE_EVENT_RBUTTON ,
@@ -3217,7 +3217,7 @@ void sendkey_completion(ReadLineState *rs, int nb_args, const char *str) } len =strlen (str);readline_set_completion_index (rs, len); -for (i =0 ; i <Q_KEY_CODE_MAX ; i++) { +for (i =0 ; i <Q_KEY_CODE__MAX ; i++) {if (!strncmp (str, QKeyCode_lookup[i], len)) {readline_add_completion (rs, QKeyCode_lookup[i]); }
@@ -3316,7 +3316,7 @@ void migrate_set_capability_completion(ReadLineState *rs, int nb_args,readline_set_completion_index (rs, len);if (nb_args ==2 ) {int i; -for (i =0 ; i <MIGRATION_CAPABILITY_MAX ; i++) { +for (i =0 ; i <MIGRATION_CAPABILITY__MAX ; i++) {const char *name = MigrationCapability_lookup[i];if (!strncmp (str, name, len)) {readline_add_completion (rs, name);
@@ -3337,7 +3337,7 @@ void migrate_set_parameter_completion(ReadLineState *rs, int nb_args,readline_set_completion_index (rs, len);if (nb_args ==2 ) {int i; -for (i =0 ; i <MIGRATION_PARAMETER_MAX ; i++) { +for (i =0 ; i <MIGRATION_PARAMETER__MAX ; i++) {const char *name = MigrationParameter_lookup[i];if (!strncmp (str, name, len)) {readline_add_completion (rs, name);
net/net.c +2 -2
@@ -943,7 +943,7 @@ static int net_init_nic(const NetClientOptions *opts, const char *name, } -static int (*const net_client_init_fun[NET_CLIENT_OPTIONS_KIND_MAX ])( +static int (*const net_client_init_fun[NET_CLIENT_OPTIONS_KIND__MAX ])(const NetClientOptions *opts,const char *name,NetClientState *peer,Error **errp) = {
@@ -1296,7 +1296,7 @@ void qmp_set_link(const char *name, bool up, Error **errp)int queues, i; queues =qemu_find_net_clients_except (name, ncs, -NET_CLIENT_OPTIONS_KIND_MAX , +NET_CLIENT_OPTIONS_KIND__MAX ,MAX_QUEUE_NUM );if (queues ==0 ) {
qemu-nbd.c +1 -1
@@ -487,7 +487,7 @@ int main(int argc, char **argv) detect_zeroes =qapi_enum_parse (BlockdevDetectZeroesOptions_lookup, optarg, -BLOCKDEV_DETECT_ZEROES_OPTIONS_MAX , +BLOCKDEV_DETECT_ZEROES_OPTIONS__MAX ,BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF , &local_err);if (local_err) {
replay/replay-input.c +4 -4
@@ -61,7 +61,7 @@ void replay_save_input_event(InputEvent *evt)replay_put_dword (evt->u.key->key->u.qcode);replay_put_byte (evt->u.key->down);break ; -case KEY_VALUE_KIND_MAX : +case KEY_VALUE_KIND__MAX :/* keep gcc happy */ break ; }
@@ -78,7 +78,7 @@ void replay_save_input_event(InputEvent *evt)replay_put_dword (evt->u.abs->axis);replay_put_qword (evt->u.abs->value);break ; -case INPUT_EVENT_KIND_MAX : +case INPUT_EVENT_KIND__MAX :/* keep gcc happy */ break ; }
@@ -109,7 +109,7 @@ InputEvent *replay_read_input_event(void) evt.u.key->key->u.qcode = (QKeyCode )replay_get_dword (); evt.u.key->down =replay_get_byte ();break ; -case KEY_VALUE_KIND_MAX : +case KEY_VALUE_KIND__MAX :/* keep gcc happy */ break ; }
@@ -129,7 +129,7 @@ InputEvent *replay_read_input_event(void) evt.u.abs->axis = (InputAxis )replay_get_dword (); evt.u.abs->value =replay_get_qword ();break ; -case INPUT_EVENT_KIND_MAX : +case INPUT_EVENT_KIND__MAX :/* keep gcc happy */ break ; }
scripts/qapi.py +3 -3
@@ -918,7 +918,7 @@ class QAPISchemaEnumType(QAPISchemaType): return c_name(self.name) def c_null(self): - return c_enum_const(self.name, (self.values + ['MAX'])[0], + return c_enum_const(self.name, (self.values + ['_MAX'])[0], self.prefix) def json_type(self):
@@ -1567,7 +1567,7 @@ const char *const %(c_name)s_lookup[] = { ''', index=index, value=value) - max_index = c_enum_const(name, 'MAX', prefix) + max_index = c_enum_const(name, '_MAX', prefix) ret += mcgen(''' [%(max_index)s] = NULL, };
@@ -1578,7 +1578,7 @@ const char *const %(c_name)s_lookup[] = { def gen_enum(name, values, prefix=None): # append automatically generated _MAX value - enum_values = values + ['MAX'] + enum_values = values + ['_MAX'] ret = mcgen('''
tests/test-qmp-output-visitor.c +3 -3
@@ -128,7 +128,7 @@ static void test_visitor_out_enum(TestOutputVisitorData *data,QObject *obj;EnumOne i; -for (i =0 ; i <ENUM_ONE_MAX ; i++) { +for (i =0 ; i <ENUM_ONE__MAX ; i++) {visit_type_EnumOne (data->ov, &i,"unused" , &error_abort); obj =qmp_output_get_qobject (data->qov);
@@ -143,7 +143,7 @@ static void test_visitor_out_enum(TestOutputVisitorData *data,static void test_visitor_out_enum_errors (TestOutputVisitorData *data,const void *unused) { -EnumOne i, bad_values[] = {ENUM_ONE_MAX ,-1 }; +EnumOne i, bad_values[] = {ENUM_ONE__MAX ,-1 };Error *err;for (i =0 ; i <ARRAY_SIZE (bad_values) ; i++) {
@@ -247,7 +247,7 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,static void test_visitor_out_struct_errors (TestOutputVisitorData *data,const void *unused) { -EnumOne bad_values[] = {ENUM_ONE_MAX ,-1 }; +EnumOne bad_values[] = {ENUM_ONE__MAX ,-1 };UserDefOne u = {0 };UserDefOne *pu = &u;Error *err;
tests/test-string-output-visitor.c +2 -2
@@ -194,7 +194,7 @@ static void test_visitor_out_enum(TestOutputVisitorData *data,char *str;EnumOne i; -for (i =0 ; i <ENUM_ONE_MAX ; i++) { +for (i =0 ; i <ENUM_ONE__MAX ; i++) {char *str_human;visit_type_EnumOne (data->ov, &i,"unused" , &err);
@@ -217,7 +217,7 @@ static void test_visitor_out_enum(TestOutputVisitorData *data,static void test_visitor_out_enum_errors (TestOutputVisitorData *data,const void *unused) { -EnumOne i, bad_values[] = {ENUM_ONE_MAX ,-1 }; +EnumOne i, bad_values[] = {ENUM_ONE__MAX ,-1 };Error *err;for (i =0 ; i <ARRAY_SIZE (bad_values) ; i++) {
tpm.c +5 -5
@@ -32,7 +32,7 @@ static TPMDriverOps const *be_drivers[TPM_MAX_DRIVERS] = { };static enum TpmModel tpm_models[TPM_MAX_MODELS ] = { -TPM_MODEL_MAX , +TPM_MODEL__MAX , };int tpm_register_model (enum TpmModel model)
@@ -40,7 +40,7 @@ int tpm_register_model(enum TpmModel model)int i;for (i =0 ; i <TPM_MAX_MODELS ; i++) { -if (tpm_models[i] ==TPM_MODEL_MAX ) { +if (tpm_models[i] ==TPM_MODEL__MAX ) { tpm_models[i] = model;return 0 ; }
@@ -272,7 +272,7 @@ static TPMInfo *qmp_query_tpm_inst(TPMBackend *drv) tpo->has_cancel_path = true; }break ; -case TPM_TYPE_MAX : +case TPM_TYPE__MAX :break ; }
@@ -311,7 +311,7 @@ TpmTypeList *qmp_query_tpm_types(Error **errp)unsigned int i =0 ;TpmTypeList *head =NULL , *prev =NULL , *cur_item; -for (i =0 ; i <TPM_TYPE_MAX ; i++) { +for (i =0 ; i <TPM_TYPE__MAX ; i++) {if (!tpm_driver_find_by_type (i)) {continue ; }
@@ -335,7 +335,7 @@ TpmModelList *qmp_query_tpm_models(Error **errp)unsigned int i =0 ;TpmModelList *head =NULL , *prev =NULL , *cur_item; -for (i =0 ; i <TPM_MODEL_MAX ; i++) { +for (i =0 ; i <TPM_MODEL__MAX ; i++) {if (!tpm_model_is_registered (i)) {continue ; }
ui/cocoa.m +1 -1
@@ -725,7 +725,7 @@ QemuCocoaView *cocoaView; if (mouse_event) { if (last_buttons != buttons) { - static uint32_t bmap[INPUT_BUTTON_MAX] = { + static uint32_t bmap[INPUT_BUTTON__MAX] = { [INPUT_BUTTON_LEFT] = MOUSE_EVENT_LBUTTON, [INPUT_BUTTON_MIDDLE] = MOUSE_EVENT_MBUTTON, [INPUT_BUTTON_RIGHT] = MOUSE_EVENT_RBUTTON,
ui/console.c +1 -1
@@ -1108,7 +1108,7 @@ void kbd_put_keysym_console(QemuConsole *s, int keysym) } } -static const int qcode_to_keysym[Q_KEY_CODE_MAX ] = { +static const int qcode_to_keysym[Q_KEY_CODE__MAX ] = { [Q_KEY_CODE_UP ] =QEMU_KEY_UP , [Q_KEY_CODE_DOWN ] =QEMU_KEY_DOWN , [Q_KEY_CODE_RIGHT ] =QEMU_KEY_RIGHT ,
ui/input-keymap.c +2 -2
@@ -132,7 +132,7 @@ static const int qcode_to_number[] = { [Q_KEY_CODE_RO] = 0x73, [Q_KEY_CODE_KP_COMMA] = 0x7e, - [Q_KEY_CODE_MAX] = 0, + [Q_KEY_CODE__MAX] = 0, };static int number_to_qcode[0x100 ];
@@ -154,7 +154,7 @@ int qemu_input_key_number_to_qcode(uint8_t nr)if (first) {int qcode, number; first = false; -for (qcode =0 ; qcode <Q_KEY_CODE_MAX ; qcode++) { +for (qcode =0 ; qcode <Q_KEY_CODE__MAX ; qcode++) { number = qcode_to_number[qcode];assert (number <ARRAY_SIZE (number_to_qcode)); number_to_qcode[number] = qcode;
ui/input-legacy.c +3 -3
@@ -38,7 +38,7 @@ struct QEMUPutMouseEntry {/* new input core */ QemuInputHandler h;QemuInputHandlerState *s; -int axis[INPUT_AXIS_MAX ]; +int axis[INPUT_AXIS__MAX ];int buttons; };
@@ -67,7 +67,7 @@ int index_from_key(const char *key) } } -/* Return Q_KEY_CODE_MAX if the key is invalid */ +/* Return Q_KEY_CODE__MAX if the key is invalid */ return i; }
@@ -143,7 +143,7 @@ QEMUPutKbdEntry *qemu_add_kbd_event_handler(QEMUPutKBDEvent *func, void *opaque)static void legacy_mouse_event (DeviceState *dev,QemuConsole *src,InputEvent *evt) { -static const int bmap[INPUT_BUTTON_MAX ] = { +static const int bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =MOUSE_EVENT_LBUTTON , [INPUT_BUTTON_MIDDLE ] =MOUSE_EVENT_MBUTTON , [INPUT_BUTTON_RIGHT ] =MOUSE_EVENT_RBUTTON ,
ui/input.c +3 -3
@@ -211,7 +211,7 @@ static void qemu_input_event_trace(QemuConsole *src, InputEvent *evt) name = QKeyCode_lookup[evt->u.key->key->u.qcode];trace_input_event_key_qcode (idx, name, evt->u.key->down);break ; -case KEY_VALUE_KIND_MAX : +case KEY_VALUE_KIND__MAX :/* keep gcc happy */ break ; }
@@ -228,7 +228,7 @@ static void qemu_input_event_trace(QemuConsole *src, InputEvent *evt) name = InputAxis_lookup[evt->u.abs->axis];trace_input_event_abs (idx, name, evt->u.abs->value);break ; -case INPUT_EVENT_KIND_MAX : +case INPUT_EVENT_KIND__MAX :/* keep gcc happy */ break ; }
@@ -430,7 +430,7 @@ void qemu_input_update_buttons(QemuConsole *src, uint32_t *button_map,InputButton btn;uint32_t mask; -for (btn =0 ; btn <INPUT_BUTTON_MAX ; btn++) { +for (btn =0 ; btn <INPUT_BUTTON__MAX ; btn++) { mask = button_map[btn];if ((button_old & mask) == (button_new & mask)) {continue ;
ui/sdl.c +1 -1
@@ -465,7 +465,7 @@ static void sdl_mouse_mode_change(Notifier *notify, void *data)static void sdl_send_mouse_event (int dx,int dy,int x,int y,int state) { -static uint32_t bmap[INPUT_BUTTON_MAX ] = { +static uint32_t bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =SDL_BUTTON (SDL_BUTTON_LEFT ), [INPUT_BUTTON_MIDDLE ] =SDL_BUTTON (SDL_BUTTON_MIDDLE ), [INPUT_BUTTON_RIGHT ] =SDL_BUTTON (SDL_BUTTON_RIGHT ),
ui/sdl2.c +1 -1
@@ -256,7 +256,7 @@ static void sdl_mouse_mode_change(Notifier *notify, void *data)static void sdl_send_mouse_event (struct sdl2_console *scon,int dx,int dy,int x,int y,int state) { -static uint32_t bmap[INPUT_BUTTON_MAX ] = { +static uint32_t bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =SDL_BUTTON (SDL_BUTTON_LEFT ), [INPUT_BUTTON_MIDDLE ] =SDL_BUTTON (SDL_BUTTON_MIDDLE ), [INPUT_BUTTON_RIGHT ] =SDL_BUTTON (SDL_BUTTON_RIGHT ),
ui/spice-input.c +1 -1
@@ -107,7 +107,7 @@ typedef struct QemuSpicePointer {static void spice_update_buttons (QemuSpicePointer *pointer,int wheel,uint32_t button_mask) { -static uint32_t bmap[INPUT_BUTTON_MAX ] = { +static uint32_t bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =0x01 , [INPUT_BUTTON_MIDDLE ] =0x04 , [INPUT_BUTTON_RIGHT ] =0x02 ,
ui/vnc.c +1 -1
@@ -1671,7 +1671,7 @@ static void check_pointer_type_change(Notifier *notifier, void *data)static void pointer_event (VncState *vs,int button_mask,int x,int y) { -static uint32_t bmap[INPUT_BUTTON_MAX ] = { +static uint32_t bmap[INPUT_BUTTON__MAX ] = { [INPUT_BUTTON_LEFT ] =0x01 , [INPUT_BUTTON_MIDDLE ] =0x02 , [INPUT_BUTTON_RIGHT ] =0x04 ,
vl.c +9 -9
@@ -570,8 +570,8 @@ static int default_driver_check(void *opaque, QemuOpts *opts, Error **errp)static RunState current_run_state =RUN_STATE_PRELAUNCH ; -/* We use RUN_STATE_MAX but any invalid value will do */ -static RunState vmstop_requested =RUN_STATE_MAX ; +/* We use RUN_STATE__MAX but any invalid value will do */ +static RunState vmstop_requested =RUN_STATE__MAX ;static QemuMutex vmstop_lock; typedef struct {
@@ -642,10 +642,10 @@ static const RunStateTransition runstate_transitions_def[] = { { RUN_STATE_GUEST_PANICKED, RUN_STATE_RUNNING }, { RUN_STATE_GUEST_PANICKED, RUN_STATE_FINISH_MIGRATE }, - { RUN_STATE_MAX, RUN_STATE_MAX }, + { RUN_STATE__MAX, RUN_STATE__MAX }, }; -static bool runstate_valid_transitions[RUN_STATE_MAX ][RUN_STATE_MAX ]; +static bool runstate_valid_transitions[RUN_STATE__MAX ][RUN_STATE__MAX ];bool runstate_check (RunState state) {
@@ -669,7 +669,7 @@ static void runstate_init(void)const RunStateTransition *p;memset (&runstate_valid_transitions,0 ,sizeof (runstate_valid_transitions)); -for (p = &runstate_transitions_def[0 ]; p->from !=RUN_STATE_MAX ; p++) { +for (p = &runstate_transitions_def[0 ]; p->from !=RUN_STATE__MAX ; p++) { runstate_valid_transitions[p->from][p->to] = true; }
@@ -679,7 +679,7 @@ static void runstate_init(void)/* This function will abort() on invalid state transitions */ void runstate_set (RunState new_state) { -assert (new_state <RUN_STATE_MAX ); +assert (new_state <RUN_STATE__MAX );if (!runstate_valid_transitions[current_run_state][new_state]) { error_report("invalid runstate transition: '%s' -> '%s'",
@@ -717,9 +717,9 @@ static bool qemu_vmstop_requested(RunState *r) {qemu_mutex_lock (&vmstop_lock); *r = vmstop_requested; - vmstop_requested =RUN_STATE_MAX ; + vmstop_requested =RUN_STATE__MAX ;qemu_mutex_unlock (&vmstop_lock); -return *r <RUN_STATE_MAX ; +return *r <RUN_STATE__MAX ; }void qemu_system_vmstop_request_prepare (void )
@@ -739,7 +739,7 @@ void vm_start(void)RunState requested;qemu_vmstop_requested (&requested); -if (runstate_is_running () && requested ==RUN_STATE_MAX ) { +if (runstate_is_running () && requested ==RUN_STATE__MAX ) {return ; }
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 22/36] qapi: Don't let implicit enum MAX member collide
Quoted 22 lines · Eric Blake
Now that we guarantee the user doesn't have any enum values beginning with a single underscore, we can use that for our own purposes. Renaming
ENUM_MAXtoENUM__MAXmakes it obvious that the sentinel is generated.This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py |index e6d014b..b862ec9 100644 |--- a/scripts/qapi.py |+++ b/scripts/qapi.py |@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = { | max_index = c_enum_const(name, 'MAX', prefix) | ret += mcgen(''' | [%(max_index)s] = NULL, |+// %(max_index)s | }; | ''', | max_index=max_index)then running:
Quoted 7 lines
$ cat qapi-{types,event}.c tests/test-qapi-types.c | sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list $ git grep -l _MAX | xargs sed -i -f listThe only things not generated are the changes in
scripts/qapi.py.Signed-off-by Eric Blake <[email protected]>
For migration bits, I have zero objections about the changes. I trust you that you have done all the required changes (i.e. I haven't compiled it). Rest of the patch is as trivial as the commit log explains, so
Thanks, Juan.
Re: [Qemu-devel] [PATCH v12 22/36] qapi: Don't let implicit enum MAX member collide
In reply to Eric BlakeQuoted 4 lines · Eric Blake
Now that we guarantee the user doesn't have any enum values beginning with a single underscore, we can use that for our own purposes. Renaming
ENUM_MAXtoENUM__MAXmakes it obvious that the sentinel is generated.
The ENUM__MAX are mildly ugly. If we cared, we could get rid of most or all uses. Right now, I don't.
Hmm, perhaps these ENUM__MAX are your motivation for rejecting adjacent [-_] in the previous patch, to catch clashes like (foolishly named) type 'FOO--MAX' with enumeration type 'Foo'. I acknowledge the clash. However, there are many more clashes we don't attempt to catch, e.g. type 'Foo-lookup' with enumeration type 'Foo'. Perhaps we want to build a real fence later, but right now I don't want us to ram in more scattered fence laths.
Might want to say something like "Rejecting enum members named 'MAX' is now useless, and will be dropped in the next patch".
Quoted 17 lines
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py |index e6d014b..b862ec9 100644 |--- a/scripts/qapi.py |+++ b/scripts/qapi.py |@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = { | max_index = c_enum_const(name, 'MAX', prefix) | ret += mcgen(''' | [%(max_index)s] = NULL, |+// %(max_index)s | }; | ''', | max_index=max_index)then running:
Quoted 7 lines
$ cat qapi-{types,event}.c tests/test-qapi-types.c | sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list $ git grep -l _MAX | xargs sed -i -f listThe only things not generated are the changes in
scripts/qapi.py.Signed-off-by Eric Blake <[email protected]>
Patch looks good.
Re: [Qemu-devel] [PATCH v12 22/36] qapi: Don't let implicit enum MAX member collide
Quoted 7 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
Now that we guarantee the user doesn't have any enum values beginning with a single underscore, we can use that for our own purposes. Renaming
ENUM_MAXtoENUM__MAXmakes it obvious that the sentinel is generated.
Quoted 6 lines
The
ENUM__MAXare mildly ugly. If we cared, we could get rid of most or all uses. Right now, I don't.Hmm, perhaps these
ENUM__MAXare your motivation for rejecting adjacent [-_] in the previous patch, to catch clashes like (foolishly named) type 'FOO--MAX' with enumeration type 'Foo'. I acknowledge the clash.
Yes, and that was part of my reasoning on 21/36 - but you've convinced me to relax that one, so it no longer applies as an argument here.
Quoted 7 lines
However, there are many more clashes we don't attempt to catch, e.g. type 'Foo-lookup' with enumeration type 'Foo'. Perhaps we want to build a real fence later, but right now I don't want us to ram in more scattered fence laths.
Might want to say something like "Rejecting enum members named 'MAX' is now useless, and will be dropped in the next patch".
Yes, that helps the commit message. Can you handle it, or do I need to send a fixup?
Quoted 22 lines
Quoted 22 lines
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py |index e6d014b..b862ec9 100644 |--- a/scripts/qapi.py |+++ b/scripts/qapi.py |@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = { | max_index = c_enum_const(name, 'MAX', prefix) | ret += mcgen(''' | [%(max_index)s] = NULL, |+// %(max_index)s | }; | ''', | max_index=max_index)then running:
$ cat qapi-{types,event}.c tests/test-qapi-types.c | sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list $ git grep -l _MAX | xargs sed -i -f list
Quoted 5 lines
Quoted 3 lines
The only things not generated are the changes in
scripts/qapi.py.Signed-off-by Eric Blake <[email protected]>Patch looks good.
The fun part was coming up with the automation, and then making sure it was reproducible. :)
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 22/36] qapi: Don't let implicit enum MAX member collide
Quoted 23 lines · Eric Blake
Quoted 7 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
Now that we guarantee the user doesn't have any enum values beginning with a single underscore, we can use that for our own purposes. Renaming
ENUM_MAXtoENUM__MAXmakes it obvious that the sentinel is generated.Quoted 6 lines
The
ENUM__MAXare mildly ugly. If we cared, we could get rid of most or all uses. Right now, I don't.Hmm, perhaps these
ENUM__MAXare your motivation for rejecting adjacent [-_] in the previous patch, to catch clashes like (foolishly named) type 'FOO--MAX' with enumeration type 'Foo'. I acknowledge the clash.Yes, and that was part of my reasoning on 21/36 - but you've convinced me to relax that one, so it no longer applies as an argument here.
Quoted 5 lines
However, there are many more clashes we don't attempt to catch, e.g. type 'Foo-lookup' with enumeration type 'Foo'. Perhaps we want to build a real fence later, but right now I don't want us to ram in more scattered fence laths.
Quoted 5 lines
Quoted 2 lines
Might want to say something like "Rejecting enum members named 'MAX' is now useless, and will be dropped in the next patch".
Yes, that helps the commit message. Can you handle it, or do I need to send a fixup?
Can do!
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py |index e6d014b..b862ec9 100644 |--- a/scripts/qapi.py |+++ b/scripts/qapi.py |@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = { | max_index = c_enum_const(name, 'MAX', prefix) | ret += mcgen(''' | [%(max_index)s] = NULL, |+// %(max_index)s | }; | ''', | max_index=max_index)then running:
$ cat qapi-{types,event}.c tests/test-qapi-types.c | sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list $ git grep -l _MAX | xargs sed -i -f list
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
The only things not generated are the changes in
scripts/qapi.py.Signed-off-by Eric Blake <[email protected]>Patch looks good.
The fun part was coming up with the automation, and then making sure it was reproducible. :)
May not always be faster than manual, but it sure is a lot more fun! More reliable, too, in my experience.
PATCH v12 23/36 qapi: Remove dead tests for max collision
In reply to Eric BlakeNow that we no longer collide with an implicit _MAX enum member, we no longer need to reject it in the ad hoc parser, and can remove several tests that are no longer needed.
---
v12: new patch
--- scripts/qapi.py | 8 +++----- tests/Makefile | 3 --- tests/qapi-schema/enum-max-member.err | 1 - tests/qapi-schema/enum-max-member.exit | 1 - tests/qapi-schema/enum-max-member.json | 3 --- tests/qapi-schema/enum-max-member.out | 0 tests/qapi-schema/event-max.err | 1 - tests/qapi-schema/event-max.exit | 1 - tests/qapi-schema/event-max.json | 2 -- tests/qapi-schema/event-max.out | 0 tests/qapi-schema/union-max.err | 1 - tests/qapi-schema/union-max.exit | 1 - tests/qapi-schema/union-max.json | 3 --- tests/qapi-schema/union-max.out | 0 14 files changed, 3 insertions(+), 22 deletions(-) delete mode 100644 tests/qapi-schema/enum-max-member.err delete mode 100644 tests/qapi-schema/enum-max-member.exit delete mode 100644 tests/qapi-schema/enum-max-member.json delete mode 100644 tests/qapi-schema/enum-max-member.out delete mode 100644 tests/qapi-schema/event-max.err delete mode 100644 tests/qapi-schema/event-max.exit delete mode 100644 tests/qapi-schema/event-max.json delete mode 100644 tests/qapi-schema/event-max.out delete mode 100644 tests/qapi-schema/union-max.err delete mode 100644 tests/qapi-schema/union-max.exit delete mode 100644 tests/qapi-schema/union-max.json delete mode 100644 tests/qapi-schema/union-max.out
scripts/qapi.py +3 -5
@@ -537,8 +537,6 @@ def check_event(expr, expr_info): global events name = expr['event'] - if name.upper() == 'MAX': - raise QAPIExprError(expr_info, "Event name 'MAX' cannot be created") events.append(name) check_type(expr_info, "'data' for event '%s'" % name, expr.get('data'), allow_dict=True, allow_optional=True,
@@ -550,7 +548,7 @@ def check_union(expr, expr_info): base = expr.get('base') discriminator = expr.get('discriminator') members = expr['data'] - values = {'MAX': '(automatic)'} + values = {} # Two types of unions, determined by discriminator.
@@ -629,7 +627,7 @@ def check_union(expr, expr_info): def check_alternate(expr, expr_info): name = expr['alternate'] members = expr['data'] - values = {'MAX': '(automatic)'} + values = {} types_seen = {} # Check every branch
@@ -662,7 +660,7 @@ def check_enum(expr, expr_info): name = expr['enum'] members = expr.get('data') prefix = expr.get('prefix') - values = {'MAX': '(automatic)'} + values = {} if not isinstance(members, list): raise QAPIExprError(expr_info,
tests/Makefile +0 -3
@@ -266,14 +266,12 @@ qapi-schema += enum-bad-prefix.json qapi-schema += enum-clash-member.json qapi-schema += enum-dict-member.json qapi-schema += enum-int-member.json -qapi-schema += enum-max-member.json qapi-schema += enum-missing-data.json qapi-schema += enum-wrong-data.json qapi-schema += escape-outside-string.json qapi-schema += escape-too-big.json qapi-schema += escape-too-short.json qapi-schema += event-case.json -qapi-schema += event-max.json qapi-schema += event-nest-struct.json qapi-schema += flat-union-array-branch.json qapi-schema += flat-union-bad-base.json
@@ -345,7 +343,6 @@ qapi-schema += union-clash-branches.json qapi-schema += union-clash-data.json qapi-schema += union-empty.json qapi-schema += union-invalid-base.json -qapi-schema += union-max.json qapi-schema += union-optional-branch.json qapi-schema += union-unknown.json qapi-schema += unknown-escape.json
tests/qapi-schema/enum-max-member.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/enum-max-member.json:3: Enum 'MyEnum' member 'max' clashes with '(automatic)'
tests/qapi-schema/enum-max-member.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/enum-max-member.json deleted +0 -3
@@ -1,3 +0,0 @@ -# we reject user-supplied 'max' for clashing with implicit enum end -# TODO: should we instead munge the implicit value to avoid the clash? -{ 'enum': 'MyEnum', 'data': [ 'max' ] }
tests/qapi-schema/enum-max-member.out deleted +0 -0
tests/qapi-schema/event-max.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/event-max.json:2: Event name 'MAX' cannot be created
tests/qapi-schema/event-max.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/event-max.json deleted +0 -2
@@ -1,2 +0,0 @@ -# an event named 'MAX' would conflict with implicit C enum -{ 'event': 'MAX' }
tests/qapi-schema/event-max.out deleted +0 -0
tests/qapi-schema/union-max.err deleted +0 -1
@@ -1 +0,0 @@ -tests/qapi-schema/union-max.json:2: Union 'Union' member 'max' clashes with '(automatic)'
tests/qapi-schema/union-max.exit deleted +0 -1
@@ -1 +0,0 @@ -1
tests/qapi-schema/union-max.json deleted +0 -3
@@ -1,3 +0,0 @@ -# we reject 'max' branch in a union, for collision with C enum -{ 'union': 'Union', - 'data': { 'max': 'int' } }
tests/qapi-schema/union-max.out deleted +0 -0
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 23/36] qapi: Remove dead tests for max collision
Tweaking subject to "qapi: Remove obsolete tests for MAX collision".
PATCH v12 24/36 cpu: Convert CpuInfo into flat union
In reply to Eric BlakeThe CpuInfo struct is used only by the 'query-cpus' output command, so we are free to modify it by adding fields (clients are already supposed to ignore unknown output fields), or by changing optional members to mandatory, while still keeping QMP wire compatibility with older versions of qemu.
When qapi type CpuInfo was originally created for 0.14, we had no notion of a flat union, and instead just listed a bunch of optional fields with documentation about the mutually-exclusive choice of which instruction pointer field(s) would be provided for a given architecture. But now that we have flat unions and introspection, it is better to segregate off which fields will be provided according to the actual architecture. With this in place, we no longer need the fields to be optional, because the choice of the new 'arch' discriminator serves that role.
This has an additional benefit: the old all-in-one struct was the only place in the code base that had a case-sensitive naming of members 'pc' vs. 'PC'. Separating these spellings into different branches of the flat union will allow us to add restrictions against future case-insensitive collisions, since that is generally a poor interface practice.
---
v12: s/2.5/2.6/, typo fix
v11: also fix qmp-commands.hx, improve commit message. If this misses 2.5 (likely), it will need updates to call out 2.6
v10: new patch
--- cpus.c | 31 ++++++++------ hmp.c | 30 +++++++++----- qapi-schema.json | 120 ++++++++++++++++++++++++++++++++++++++++++++++--------- qmp-commands.hx | 4 ++ 4 files changed, 143 insertions(+), 42 deletions(-)
cpus.c +19 -12
@@ -1556,22 +1556,29 @@ CpuInfoList *qmp_query_cpus(Error **errp) info->value->qom_path =object_get_canonical_path (OBJECT (cpu)); info->value->thread_id = cpu->thread_id;#if defined(TARGET_I386 ) - info->value->has_pc = true; - info->value->pc = env->eip + env->segs[R_CS ].base; + info->value->arch =CPU_INFO_ARCH_X86 ; + info->value->u.x86 =g_new0 (CpuInfoX86,1 ); + info->value->u.x86->pc = env->eip + env->segs[R_CS ].base;#elif defined(TARGET_PPC ) - info->value->has_nip = true; - info->value->nip = env->nip; + info->value->arch =CPU_INFO_ARCH_PPC ; + info->value->u.ppc =g_new0 (CpuInfoPpc,1 ); + info->value->u.ppc->nip = env->nip;#elif defined(TARGET_SPARC ) - info->value->has_pc = true; - info->value->pc = env->pc; - info->value->has_npc = true; - info->value->npc = env->npc; + info->value->arch =CPU_INFO_ARCH_SPARC ; + info->value->u.sparc =g_new0 (CpuInfoSPARC,1 ); + info->value->u.sparc->pc = env->pc; + info->value->u.sparc->npc = env->npc;#elif defined(TARGET_MIPS ) - info->value->has_PC = true; - info->value->PC = env->active_tc.PC; + info->value->arch =CPU_INFO_ARCH_MIPS ; + info->value->u.mips =g_new0 (CpuInfoMips,1 ); + info->value->u.mips->PC = env->active_tc.PC;#elif defined(TARGET_TRICORE ) - info->value->has_PC = true; - info->value->PC = env->PC; + info->value->arch =CPU_INFO_ARCH_TRICORE ; + info->value->u.tricore =g_new0 (CpuInfoTricore,1 ); + info->value->u.tricore->PC = env->PC; +#else + info->value->arch =CPU_INFO_ARCH_OTHER ; + info->value->u.other =g_new0 (CpuInfoOther,1 );#endif /* XXX: waiting for the qapi to support GSList */
hmp.c +19 -11
@@ -311,17 +311,25 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)monitor_printf (mon,"%c CPU #%" PRId64":" , active, cpu->value->CPU); -if (cpu->value->has_pc) { -monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->pc); - } -if (cpu->value->has_nip) { -monitor_printf (mon," nip=0x%016" PRIx64, cpu->value->nip); - } -if (cpu->value->has_npc) { -monitor_printf (mon," npc=0x%016" PRIx64, cpu->value->npc); - } -if (cpu->value->has_PC) { -monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->PC); +switch (cpu->value->arch) { +case CPU_INFO_ARCH_X86 : +monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->u.x86->pc); +break ; +case CPU_INFO_ARCH_PPC : +monitor_printf (mon," nip=0x%016" PRIx64, cpu->value->u.ppc->nip); +break ; +case CPU_INFO_ARCH_SPARC : +monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->u.sparc->pc); +monitor_printf (mon," npc=0x%016" PRIx64, cpu->value->u.sparc->npc); +break ; +case CPU_INFO_ARCH_MIPS : +monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->u.mips->PC); +break ; +case CPU_INFO_ARCH_TRICORE : +monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->u.tricore->PC); +break ; +default : +break ; }if (cpu->value->halted) {
qapi-schema.json +101 -19
144 lines not on this page · raw message
qmp-commands.hx +4 -0
@@ -2765,6 +2765,8 @@ Return a json-array. Each CPU is represented by a json-object, which contains: - "current": true if this is the current CPU, false otherwise (json-bool) - "halted": true if the cpu is halted, false otherwise (json-bool) - "qom_path": path to the CPU object in the QOM tree (json-str) +- "arch": architecture of the cpu, which determines what additional + keys will be present (json-str) - Current program counter. The key's name depends on the architecture: "pc": i386/x86_64 (json-int) "nip": PPC (json-int)
@@ -2782,6 +2784,7 @@ Example: "current":true, "halted":false, "qom_path":"/machine/unattached/device[0]", + "arch":"x86", "pc":3227107138, "thread_id":3134 },
@@ -2790,6 +2793,7 @@ Example: "current":false, "halted":true, "qom_path":"/machine/unattached/device[2]", + "arch":"x86", "pc":7108165, "thread_id":3135 }
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 24/36] cpu: Convert CpuInfo into flat union
Quoted 53 lines · Eric Blake
The CpuInfo struct is used only by the 'query-cpus' output command, so we are free to modify it by adding fields (clients are already supposed to ignore unknown output fields), or by changing optional members to mandatory, while still keeping QMP wire compatibility with older versions of qemu.
When qapi type CpuInfo was originally created for 0.14, we had no notion of a flat union, and instead just listed a bunch of optional fields with documentation about the mutually-exclusive choice of which instruction pointer field(s) would be provided for a given architecture. But now that we have flat unions and introspection, it is better to segregate off which fields will be provided according to the actual architecture. With this in place, we no longer need the fields to be optional, because the choice of the new 'arch' discriminator serves that role.
This has an additional benefit: the old all-in-one struct was the only place in the code base that had a case-sensitive naming of members 'pc' vs. 'PC'. Separating these spellings into different branches of the flat union will allow us to add restrictions against future case-insensitive collisions, since that is generally a poor interface practice.
Signed-off-by Eric Blake <[email protected]>---
v12: s/2.5/2.6/, typo fix
v11: also fix qmp-commands.hx, improve commit message. If this misses 2.5 (likely), it will need updates to call out 2.6
v10: new patch--- cpus.c | 31 ++++++++------ hmp.c | 30 +++++++++----- qapi-schema.json | 120 ++++++++++++++++++++++++++++++++++++++++++++++--------- qmp-commands.hx | 4 ++ 4 files changed, 143 insertions(+), 42 deletions(-)cpus.c +3
-4@@ -1556,22 +1556,29 @@ CpuInfoList *qmp_query_cpus(Error **errp) info->value->qom_path =object_get_canonical_path (OBJECT (cpu)); info->value->thread_id = cpu->thread_id;#if defined(TARGET_I386 ) - info->value->has_pc = true; - info->value->pc = env->eip + env->segs[R_CS ].base; + info->value->arch =CPU_INFO_ARCH_X86 ; + info->value->u.x86 =g_new0 (CpuInfoX86,1 ); + info->value->u.x86->pc = env->eip + env->segs[R_CS ].base;#elif defined(TARGET_PPC ) - info->value->has_nip = true; - info->value->nip = env->nip;
cpus.c CpuInfoList *qmp_query_cpus(Error **errp) · line 1568
Quoted 10 lines
cpus.c +5
-4+ info->value->arch =CPU_INFO_ARCH_PPC ; + info->value->u.ppc =g_new0 (CpuInfoPpc,1 ); + info->value->u.ppc->nip = env->nip;#elif defined(TARGET_SPARC ) - info->value->has_pc = true; - info->value->pc = env->pc; - info->value->has_npc = true; - info->value->npc = env->npc; + info->value->arch =CPU_INFO_ARCH_SPARC ; + info->value->u.sparc =g_new0 (CpuInfoSPARC,1 );
CpuInfoSparc.
Quoted 144 lines
cpus.c +11
-4+ info->value->u.sparc->pc = env->pc; + info->value->u.sparc->npc = env->npc;#elif defined(TARGET_MIPS ) - info->value->has_PC = true; - info->value->PC = env->active_tc.PC; + info->value->arch =CPU_INFO_ARCH_MIPS ; + info->value->u.mips =g_new0 (CpuInfoMips,1 ); + info->value->u.mips->PC = env->active_tc.PC;#elif defined(TARGET_TRICORE ) - info->value->has_PC = true; - info->value->PC = env->PC; + info->value->arch =CPU_INFO_ARCH_TRICORE ; + info->value->u.tricore =g_new0 (CpuInfoTricore,1 ); + info->value->u.tricore->PC = env->PC; +#else + info->value->arch =CPU_INFO_ARCH_OTHER ; + info->value->u.other =g_new0 (CpuInfoOther,1 );#endif /* XXX: waiting for the qapi to support GSList */ hmp.c +19
-11@@ -311,17 +311,25 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)monitor_printf (mon,"%c CPU #%" PRId64":" , active, cpu->value->CPU); -if (cpu->value->has_pc) { -monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->pc); - } -if (cpu->value->has_nip) { -monitor_printf (mon," nip=0x%016" PRIx64, cpu->value->nip); - } -if (cpu->value->has_npc) { -monitor_printf (mon," npc=0x%016" PRIx64, cpu->value->npc); - } -if (cpu->value->has_PC) { -monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->PC); +switch (cpu->value->arch) { +case CPU_INFO_ARCH_X86 : +monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->u.x86->pc); +break ; +case CPU_INFO_ARCH_PPC : +monitor_printf (mon," nip=0x%016" PRIx64, cpu->value->u.ppc->nip); +break ; +case CPU_INFO_ARCH_SPARC : +monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->u.sparc->pc); +monitor_printf (mon," npc=0x%016" PRIx64, cpu->value->u.sparc->npc); +break ; +case CPU_INFO_ARCH_MIPS : +monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->u.mips->PC); +break ; +case CPU_INFO_ARCH_TRICORE : +monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->u.tricore->PC); +break ; +default : +break ; }if (cpu->value->halted) {qapi-schema.json +38
-19@@ -744,43 +744,125 @@ { 'command': 'query-mice', 'returns': ['MouseInfo'] } ## -# @CpuInfo: +# @CpuInfoArch: # -# Information about a virtual CPU +# An enumeration of cpu types that enable additional information during +# @query-cpus. +# +# Since: 2.6 +## +{ 'enum': 'CpuInfoArch', + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] } + +## +# @CpuInfoBase: +# +# Common information about a virtual CPU # # @CPU: the index of the virtual CPU # -# @current: this only exists for backwards compatible and should be ignored +# @current: this only exists for backwards compatibility and should be ignored # # @halted: true if the virtual CPU is in the halt state. Halt usually refers # to a processor specific low power mode. # # @qom_path: path to the CPU object in the QOM tree (since 2.4) # -# @pc: #optional If the target is i386 or x86_64, this is the 64-bit instruction -# pointer. -# If the target is Sparc, this is the PC component of the -# instruction pointer. -# -# @nip: #optional If the target is PPC, the instruction pointer -# -# @npc: #optional If the target is Sparc, the NPC component of the instruction -# pointer -# -# @PC: #optional If the target is MIPS, the instruction pointer -# # @thread_id: ID of the underlying host thread # +# @arch: architecture of the cpu, which determines which additional fields +# will be listed (since 2.6) +# # Since: 0.14.0 # # Notes: @halted is a transient state that changes frequently. By the time the # data is sent to the client, the guest may no longer be halted. ## -{ 'struct': 'CpuInfo', +{ 'struct': 'CpuInfoBase', 'data': {'CPU': 'int', 'current': 'bool', 'halted': 'bool', - 'qom_path': 'str', - '*pc': 'int', '*nip': 'int', '*npc': 'int', '*PC': 'int', - 'thread_id': 'int'} } + 'qom_path': 'str', 'thread_id': 'int', 'arch': 'CpuInfoArch' } } + +## +# @CpuInfo: +# +# Information about a virtual CPU +# +# Since: 0.14.0 +## +{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch', + 'data': { 'x86': 'CpuInfoX86', + 'sparc': 'CpuInfoSparc', + 'ppc': 'CpuInfoPpc', + 'mips': 'CpuInfoMips', + 'tricore': 'CpuInfoTricore', + 'other': 'CpuInfoOther' } } + +## +# @CpuInfoX86: +#
qapi-schema.json line 812
Quoted 10 lines
qapi-schema.json +10
-0+# Additional information about a virtual i386 or x86_64 CPU +# +# @pc: the 64-bit instruction pointer +# +# Since 2.6 +## +{ 'struct': 'CpuInfoX86', 'data': { 'pc': 'int' } } + +## +# @CpuInfoSparc:
CpuInfoSPARC would be slightly prettier, actually.
qapi-schema.json line 825
Quoted 13 lines
qapi-schema.json +13
-0+# +# Additional information about a virtual Sparc CPU +# +# @pc: the PC component of the instruction pointer +# +# @npc: the NPC component of the instruction pointer +# +# Since 2.6 +## +{ 'struct': 'CpuInfoSparc', 'data': { 'pc': 'int', 'npc': 'int' } } + +## +# @CpuInfoPpc:
CpuInfoPPC would definitely be prettier.
qapi-schema.json line 836
Quoted 11 lines
qapi-schema.json +11
-0+# +# Additional information about a virtual PPC CPU +# +# @nip: the instruction pointer +# +# Since 2.6 +## +{ 'struct': 'CpuInfoPpc', 'data': { 'nip': 'int' } } + +## +# @CpuInfoMips:
CpuInfoMIPS?
Quoted 61 lines
qapi-schema.json +29
-0+# +# Additional information about a virtual MIPS CPU +# +# @PC: the instruction pointer +# +# Since 2.6 +## +{ 'struct': 'CpuInfoMips', 'data': { 'PC': 'int' } } + +## +# @CpuInfoTricore: +# +# Additional information about a virtual Tricore CPU +# +# @PC: the instruction pointer +# +# Since 2.6 +## +{ 'struct': 'CpuInfoTricore', 'data': { 'PC': 'int' } } + +## +# @CpuInfoOther: +# +# No additional information is available about the virtual CPU +# +# Since 2.6 +# +## +{ 'struct': 'CpuInfoOther', 'data': { } } ## # @query-cpus:qmp-commands.hx +4
-0@@ -2765,6 +2765,8 @@ Return a json-array. Each CPU is represented by a json-object, which contains: - "current": true if this is the current CPU, false otherwise (json-bool) - "halted": true if the cpu is halted, false otherwise (json-bool) - "qom_path": path to the CPU object in the QOM tree (json-str) +- "arch": architecture of the cpu, which determines what additional + keys will be present (json-str) - Current program counter. The key's name depends on the architecture: "pc": i386/x86_64 (json-int) "nip": PPC (json-int)@@ -2782,6 +2784,7 @@ Example: "current":true, "halted":false, "qom_path":"/machine/unattached/device[0]", + "arch":"x86", "pc":3227107138, "thread_id":3134 },@@ -2790,6 +2793,7 @@ Example: "current":false, "halted":true, "qom_path":"/machine/unattached/device[2]", + "arch":"x86", "pc":7108165, "thread_id":3135 }
Re: [Qemu-devel] [PATCH v12 24/36] cpu: Convert CpuInfo into flat union
Quoted 26 lines · Markus Armbruster
Quoted 24 lines · Eric Blake
The CpuInfo struct is used only by the 'query-cpus' output command, so we are free to modify it by adding fields (clients are already supposed to ignore unknown output fields), or by changing optional members to mandatory, while still keeping QMP wire compatibility with older versions of qemu.
+++
b/cpus.ccpus.c +6
-4@@ -1556,22 +1556,29 @@ CpuInfoList *qmp_query_cpus(Error **errp) info->value->qom_path =object_get_canonical_path (OBJECT (cpu)); info->value->thread_id = cpu->thread_id;#if defined(TARGET_I386 ) - info->value->has_pc = true; - info->value->pc = env->eip + env->segs[R_CS ].base; + info->value->arch =CPU_INFO_ARCH_X86 ; + info->value->u.x86 =g_new0 (CpuInfoX86,1 ); + info->value->u.x86->pc = env->eip + env->segs[R_CS ].base;#elif defined(TARGET_PPC ) - info->value->has_nip = true; - info->value->nip = env->nip; + info->value->arch =CPU_INFO_ARCH_PPC ; + info->value->u.ppc =g_new0 (CpuInfoPpc,1 ); + info->value->u.ppc->nip = env->nip;#elif defined(TARGET_SPARC )
Quoted 8 lines
cpus.c CpuInfoList *qmp_query_cpus(Error **errp) · line 1568
Quoted 6 lines
cpus.c +2
-4- info->value->has_pc = true; - info->value->pc = env->pc; - info->value->has_npc = true; - info->value->npc = env->npc; + info->value->arch =CPU_INFO_ARCH_SPARC ; + info->value->u.sparc =g_new0 (CpuInfoSPARC,1 );CpuInfoSparc.
#ifdef compilation problems. Yes, we'll have to double-check that whatever case we use here matches our qapi.
Quoted 18 lines
Quoted 17 lines
qapi-schema.json +10
-2+++ b/qapi-schema.json@@ -744,43 +744,125 @@ { 'command': 'query-mice', 'returns': ['MouseInfo'] } ## -# @CpuInfo: +# @CpuInfoArch: # -# Information about a virtual CPU +# An enumeration of cpu types that enable additional information during +# @query-cpus. +# +# Since: 2.6 +## +{ 'enum': 'CpuInfoArch', + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] } +
Quoted 14 lines
Quoted 4 lines
qapi-schema.json +4
-0+## +# @CpuInfo: +# +# Information about a virtual CPUqapi-schema.json line 798
Quoted 10 lines
qapi-schema.json +10
-0+# +# Since: 0.14.0 +## +{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch', + 'data': { 'x86': 'CpuInfoX86', + 'sparc': 'CpuInfoSparc', + 'ppc': 'CpuInfoPpc', + 'mips': 'CpuInfoMips', + 'tricore': 'CpuInfoTricore', + 'other': 'CpuInfoOther' } }
As to the qapi types, changing the case to CpuInfoSPARC, CpuInfoPPC, and CpuInfoMIPS all work for me. For MIPS, it definitely matches prevailing preference (here searching for any use of the arch name as part of a larger CamelCase name); for SPARC and PPC, we could sway the vote:
$ git grep '[a-z]Sparc' origin | wc
39 262 2705
$ git grep '[a-z]SPARC' origin | wc
38 316 3385$ git grep '[a-z]Ppc' origin | wc
1 4 48
$ git grep '[a-z]PPC' origin | wc
1 4 56$ git grep '[a-z]Mips' origin | wc
0 0 0
$ git grep '[a-z]MIPS' origin | wc
69 407 5475I'm assuming you plan on doing the touchup?
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 24/36] cpu: Convert CpuInfo into flat union
Quoted 88 lines · Eric Blake
Quoted 26 lines · Markus Armbruster
Quoted 24 lines · Eric Blake
The CpuInfo struct is used only by the 'query-cpus' output command, so we are free to modify it by adding fields (clients are already supposed to ignore unknown output fields), or by changing optional members to mandatory, while still keeping QMP wire compatibility with older versions of qemu.
+++
b/cpus.ccpus.c +6
-4@@ -1556,22 +1556,29 @@ CpuInfoList *qmp_query_cpus(Error **errp) info->value->qom_path =object_get_canonical_path (OBJECT (cpu)); info->value->thread_id = cpu->thread_id;#if defined(TARGET_I386 ) - info->value->has_pc = true; - info->value->pc = env->eip + env->segs[R_CS ].base; + info->value->arch =CPU_INFO_ARCH_X86 ; + info->value->u.x86 =g_new0 (CpuInfoX86,1 ); + info->value->u.x86->pc = env->eip + env->segs[R_CS ].base;#elif defined(TARGET_PPC ) - info->value->has_nip = true; - info->value->nip = env->nip; + info->value->arch =CPU_INFO_ARCH_PPC ; + info->value->u.ppc =g_new0 (CpuInfoPpc,1 ); + info->value->u.ppc->nip = env->nip;#elif defined(TARGET_SPARC )Quoted 8 lines
cpus.c CpuInfoList *qmp_query_cpus(Error **errp) · line 1568
Quoted 6 lines
cpus.c +2
-4- info->value->has_pc = true; - info->value->pc = env->pc; - info->value->has_npc = true; - info->value->npc = env->npc; + info->value->arch =CPU_INFO_ARCH_SPARC ; + info->value->u.sparc =g_new0 (CpuInfoSPARC,1 );CpuInfoSparc.
#ifdef compilation problems. Yes, we'll have to double-check that whatever case we use here matches our qapi.
Quoted 18 lines
Quoted 17 lines
qapi-schema.json +10
-2+++ b/qapi-schema.json@@ -744,43 +744,125 @@ { 'command': 'query-mice', 'returns': ['MouseInfo'] } ## -# @CpuInfo: +# @CpuInfoArch: # -# Information about a virtual CPU +# An enumeration of cpu types that enable additional information during +# @query-cpus. +# +# Since: 2.6 +## +{ 'enum': 'CpuInfoArch', + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] } +Quoted 14 lines
Quoted 4 lines
qapi-schema.json +4
-0+## +# @CpuInfo: +# +# Information about a virtual CPUqapi-schema.json line 798
Quoted 10 lines
qapi-schema.json +10
-0+# +# Since: 0.14.0 +## +{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch', + 'data': { 'x86': 'CpuInfoX86', + 'sparc': 'CpuInfoSparc', + 'ppc': 'CpuInfoPpc', + 'mips': 'CpuInfoMips', + 'tricore': 'CpuInfoTricore', + 'other': 'CpuInfoOther' } }As to the qapi types, changing the case to CpuInfoSPARC, CpuInfoPPC, and CpuInfoMIPS all work for me. For MIPS, it definitely matches prevailing preference (here searching for any use of the arch name as part of a larger CamelCase name); for SPARC and PPC, we could sway the vote:
$ git grep '[a-z]Sparc' origin | wc 39 262 2705 $ git grep '[a-z]SPARC' origin | wc 38 316 3385$ git grep '[a-z]Ppc' origin | wc 1 4 48 $ git grep '[a-z]PPC' origin | wc 1 4 56
Quoted 4 lines
$ git grep '[a-z]Mips' origin | wc
0 0 0
$ git grep '[a-z]MIPS' origin | wc
69 407 5475Wikipedia and USPTO spell it SPARC, MIPS and PowerPC. PPC is a common abbreviation of the latter. I find CpuInfoPpc even uglier than CpuInfoPPC.
Quoted 1 line
I'm assuming you plan on doing the touchup?
Yes, fixup is being compiled.
Re: [Qemu-devel] [PATCH v12 24/36] cpu: Convert CpuInfo into flat union
In reply to Eric BlakeHi Eric,
Quoted 8 lines · Eric Blake
(quoted diff) +8
-0+# An enumeration of cpu types that enable additional information during +# @query-cpus. +# +# Since: 2.6 +## +{ 'enum': 'CpuInfoArch', + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] } +
...
Review
Quoted 12 lines
(quoted diff) +12
-0+## +# @CpuInfo: +# +# Information about a virtual CPU +# +# Since: 0.14.0 +## +{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch', + 'data': { 'x86': 'CpuInfoX86', + 'sparc': 'CpuInfoSparc', + 'ppc': 'CpuInfoPpc', + 'mips': 'CpuInfoMips',
This is causing compilation issues on MIPS, because gcc bizarrely defines mips=1:
$ mips-img-linux-gnu-gcc -dM -E - </dev/null | grep '\bmips\b' #define mips 1
For example:
CC qga/commands.o
In file included from /work/mips/qemu/msa/include/qapi/qmp/qobject.h:37:0,
from /work/mips/qemu/msa/include/qapi/qmp/dispatch.h:17,
from ./qga/guest-agent-core.h:13,
from qga/commands.c:14:
./qapi-types.h:2568:22: error: expected identifier or ‘(’ before
numeric constant
CpuInfoMIPS *mips;
^/work/mips/qemu/msa/rules.mak:57: recipe for target 'qga/commands.o' failed
make: *** [qga/commands.o] Error 1
Any suggestions for a fix that is less ugly than this:
cpus.c +0 -0
@@ -1580,9 +1580,9 @@ CpuInfoList *qmp_query_cpus(Error **errp) info->value->u.sparc->pc = env->pc; info->value->u.sparc->npc = env->npc;
#elif defined(TARGET_MIPS)
- info->value->arch = CPU_INFO_ARCH_MIPS;
- info->value->u.mips = g_new0(CpuInfoMIPS, 1);
- info->value->u.mips->PC = env->active_tc.PC;
+ info->value->arch = CPU_INFO_ARCH_MIPSARCH;
+ info->value->u.mipsarch = g_new0(CpuInfoMIPS, 1);
+ info->value->u.mipsarch->PC = env->active_tc.PC;
#elif defined(TARGET_TRICORE)
info->value->arch = CPU_INFO_ARCH_TRICORE;
info->value->u.tricore = g_new0(CpuInfoTricore, 1);hmp.c +2 -2
@@ -321,8 +321,8 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)monitor_printf (mon," pc=0x%016" PRIx64, cpu->value->u.sparc->pc);monitor_printf (mon," npc=0x%016" PRIx64, cpu->value->u.sparc->npc);break ; -case CPU_INFO_ARCH_MIPS : -monitor_printf (mon," PC=0x%016" PRIx64, cpu->value->u.mips->PC); +case CPU_INFO_ARCH_MIPSARCH : + monitor_printf(mon, " PC=0x%016" PRIx64,
cpu->value->u.mipsarch->PC);
break;
case CPU_INFO_ARCH_TRICORE:
monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->u.tricore->PC);qapi-schema.json +0 -0
@@ -750,7 +750,7 @@
# Since: 2.6
##
{ 'enum': 'CpuInfoArch',
- 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] }
+ 'data': ['x86', 'sparc', 'ppc', 'mipsarch', 'tricore', 'other' ] }
##
# @CpuInfoBase:
@@ -791,7 +791,7 @@
'data': { 'x86': 'CpuInfoX86',
'sparc': 'CpuInfoSPARC',
'ppc': 'CpuInfoPPC',
- 'mips': 'CpuInfoMIPS',
+ 'mipsarch': 'CpuInfoMIPS',
'tricore': 'CpuInfoTricore',
'other': 'CpuInfoOther' } }Cheers
James
Re: [Qemu-devel] [PATCH v12 24/36] cpu: Convert CpuInfo into flat union
Quoted 30 lines · James Hogan
Hi Eric,
Quoted 8 lines · Eric Blake
(quoted diff) +8
-0+# An enumeration of cpu types that enable additional information during +# @query-cpus. +# +# Since: 2.6 +## +{ 'enum': 'CpuInfoArch', + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] } +...
Review
Quoted 12 lines
(quoted diff) +12
-0+## +# @CpuInfo: +# +# Information about a virtual CPU +# +# Since: 0.14.0 +## +{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch', + 'data': { 'x86': 'CpuInfoX86', + 'sparc': 'CpuInfoSparc', + 'ppc': 'CpuInfoPpc', + 'mips': 'CpuInfoMips',This is causing compilation issues on MIPS, because gcc bizarrely defines mips=1:
Quoted 2 lines
$ mips-img-linux-gnu-gcc -dM -E - </dev/null | grep '\bmips\b' #define mips 1
Bizarre indeed.
Quoted 10 lines
For example:
CCqga/commands.o
In file included from /work/mips/qemu/msa/include/qapi/qmp/qobject.h:37:0,from /work/mips/qemu/msa/include/qapi/qmp/dispatch.h:17, from ./qga/guest-agent-core.h:13, from qga/commands.c:14: ./qapi-types.h:2568:22: error: expected identifier or ‘(’ before numeric constant CpuInfoMIPS *mips; ^
Quoted 5 lines
/work/mips/qemu/msa/rules.mak:57: recipe for target '
qga/commands.o' failed
make: *** [qga/commands.o] Error 1Any suggestions for a fix that is less ugly than this:
#undef mips with a suitable comment?
[...]
Re: [Qemu-devel] [PATCH v12 24/36] cpu: Convert CpuInfo into flat union
In reply to James HoganQuoted 16 lines · James Hogan
Hi Eric,
Review
Quoted 8 lines · Eric Blake
(quoted diff) +8
-0+# An enumeration of cpu types that enable additional information during +# @query-cpus. +# +# Since: 2.6 +## +{ 'enum': 'CpuInfoArch', + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] } +This is causing compilation issues on MIPS, because gcc bizarrely defines mips=1:
Quoted 2 lines
$ mips-img-linux-gnu-gcc -dM -E - </dev/null | grep '\bmips\b' #define mips 1
Not the first instance of namespace pollution.
Quoted 1 line
Any suggestions for a fix that is less ugly than this:
Yes, I'm posting the proper patch to poison QAPI 'mips' into C 'q_mips' shortly. That way, QMP remains backwards compatible (your approach did not preserve that).
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
PATCH v12 25/36 qapi: Add alias for ErrorClass
In reply to Eric BlakeThe qapi enum ErrorClass is unusual that it uses 'CamelCase' names, contrary to our documented convention of preferring 'lower-case'. However, this enum is entrenched in the API; we cannot change what strings QMP outputs. Meanwhile, we want to simplify how c_enum_const() is used to generate enum constants, by moving away from the heuristics of camel_to_upper() to a more straightforward c_name(N).upper() - but doing so will rename all of the ErrorClass constants and cause churn to all client files, where the new names are aesthetically less pleasing (ERROR_CLASS_DEVICENOTFOUND looks like we can't make up our minds on whether to break between words).
So as always in computer science, solve the problem by some more
indirection: rename the qapi type to QapiErrorClass, and add a new enum ErrorClass in error.h whose members are aliases of the qapi type, but with the spelling expected elsewhere in the tree. Then, when c_enum_const() changes the munging, we only have to adjust the one alias spot.
Suggested by: Markus Armbruster <[email protected]>
---
v12: new patch
--- include/qapi/error.h | 14 ++++++++++++++ monitor.c | 2 +- qapi/common.json | 5 +++-- qapi/qmp-dispatch.c | 2 +- 4 files changed, 19 insertions(+), 4 deletions(-)
include/qapi/error.h +14 -0
@@ -108,6 +108,20 @@typedef struct Error Error ;/* +* Overall category of an error. +* Based on the qapi type QapiErrorClass, but reproduced here for nicer +* enum names. +*/ +typedef enum ErrorClass { +ERROR_CLASS_GENERIC_ERROR =QAPI_ERROR_CLASS_GENERIC_ERROR , +ERROR_CLASS_COMMAND_NOT_FOUND =QAPI_ERROR_CLASS_COMMAND_NOT_FOUND , +ERROR_CLASS_DEVICE_ENCRYPTED =QAPI_ERROR_CLASS_DEVICE_ENCRYPTED , +ERROR_CLASS_DEVICE_NOT_ACTIVE =QAPI_ERROR_CLASS_DEVICE_NOT_ACTIVE , +ERROR_CLASS_DEVICE_NOT_FOUND =QAPI_ERROR_CLASS_DEVICE_NOT_FOUND , +ERROR_CLASS_KVM_MISSING_CAP =QAPI_ERROR_CLASS_KVM_MISSING_CAP , +}ErrorClass ; + +/* * Get @err's human-readable error message. */ const char *error_get_pretty (Error *err);
monitor.c +1 -1
@@ -403,7 +403,7 @@ static QDict *build_qmp_error_dict(Error *err)QObject *obj; obj =qobject_from_jsonf ("{ 'error': { 'class': %s, 'desc': %s } }" , - ErrorClass_lookup[error_get_class (err)], + QapiErrorClass_lookup[error_get_class (err)],error_get_pretty (err));return qobject_to_qdict (obj);
qapi/common.json +3 -2
@@ -3,7 +3,7 @@ # QAPI common definitions ## -# @ErrorClass +# @QapiErrorClass # # QEMU error classes #
@@ -24,7 +24,8 @@ # # Since: 1.2 ## -{ 'enum': 'ErrorClass', +{ 'enum': 'QapiErrorClass', + # Keep this in sync with ErrorClass in error.h 'data': [ 'GenericError', 'CommandNotFound', 'DeviceEncrypted', 'DeviceNotActive', 'DeviceNotFound', 'KVMMissingCap' ] }
qapi/qmp-dispatch.c +1 -1
@@ -114,7 +114,7 @@ static QObject *do_qmp_dispatch(QObject *request, Error **errp)QObject *qmp_build_error_object (Error *err) {return qobject_from_jsonf ("{ 'class': %s, 'desc': %s }" , - ErrorClass_lookup[error_get_class (err)], + QapiErrorClass_lookup[error_get_class (err)],error_get_pretty (err)); }
Signature · 1 line
2.4.3
PATCH v12 26/36 qapi: Change munging of CamelCase enum values
In reply to Eric BlakeWhen munging enum values, the fact that we were passing the entire prefix + value through camel_to_upper() meant that enum values spelled with CamelCase could be turned into CAMEL_CASE. However, this provides a potential collision (both OneTwo and One-Two would munge into ONE_TWO) for enum types, when the same two names are valid side-by-side as qapi member names. By changing the generation of enum constants to always be prefix + '_' + c_name(value, False).upper(), and ensuring that there are no case collisions (in the next patches), we no longer have to worry about names that would be distinct as qapi members but collide as variant tag names, without having to think about what munging the heuristics in camel_to_upper() will actually perform on an enum value.
Making the change will affect enums that did not follow coding conventions, using 'CamelCase' rather than desired 'lower-case'.
Thankfully, there are only two culprits: InputButton and ErrorClass. We already tweaked ErrorClass to make it an alias of QapiErrorClass, where only the alias needs changingrather than the whole tree. So the bulk of this change is modifying INPUT_BUTTON_WHEEL_UP to the new INPUT_BUTTON_WHEELUP (and likewise for WHEELDOWN). That part of this commit may later need reverting if we rename the enum constants from 'WheelUp' to 'wheel-up' as part of moving x-input-send-event to a stable interface; but at least we have documentation bread crumbs in place to remind us (commit 513e7cd), and it matches the fact that SDL constants are also spelled SDL_BUTTON_WHEELUP.
Suggested by: Markus Armbruster <[email protected]>
---
v12: rebase to simpler ErrorClass changes, document decisions on InputButton
v11: new patch
--- hw/input/hid.c | 4 ++-- hw/input/ps2.c | 4 ++-- hw/input/virtio-input-hid.c | 4 ++-- include/qapi/error.h | 12 ++++++------ monitor.c | 2 +- scripts/qapi.py | 2 +- ui/cocoa.m | 4 ++-- ui/gtk.c | 4 ++-- ui/input-legacy.c | 4 ++-- ui/sdl.c | 4 ++-- ui/sdl2.c | 4 ++-- ui/spice-input.c | 4 ++-- ui/vnc.c | 4 ++-- 13 files changed, 28 insertions(+), 28 deletions(-)
hw/input/hid.c +2 -2
@@ -139,9 +139,9 @@ static void hid_pointer_event(DeviceState *dev, QemuConsole *src,case INPUT_EVENT_KIND_BTN :if (evt->u.btn->down) { e->buttons_state |= bmap[evt->u.btn->button]; -if (evt->u.btn->button ==INPUT_BUTTON_WHEEL_UP ) { +if (evt->u.btn->button ==INPUT_BUTTON_WHEELUP ) { e->dz--; - }else if (evt->u.btn->button ==INPUT_BUTTON_WHEEL_DOWN ) { + }else if (evt->u.btn->button ==INPUT_BUTTON_WHEELDOWN ) { e->dz++; } }else {
hw/input/ps2.c +2 -2
@@ -405,9 +405,9 @@ static void ps2_mouse_event(DeviceState *dev, QemuConsole *src,case INPUT_EVENT_KIND_BTN :if (evt->u.btn->down) { s->mouse_buttons |= bmap[evt->u.btn->button]; -if (evt->u.btn->button ==INPUT_BUTTON_WHEEL_UP ) { +if (evt->u.btn->button ==INPUT_BUTTON_WHEELUP ) { s->mouse_dz--; - }else if (evt->u.btn->button ==INPUT_BUTTON_WHEEL_DOWN ) { + }else if (evt->u.btn->button ==INPUT_BUTTON_WHEELDOWN ) { s->mouse_dz++; } }else {
hw/input/virtio-input-hid.c +2 -2
@@ -142,8 +142,8 @@ static const unsigned int keymap_button[INPUT_BUTTON__MAX] = { [INPUT_BUTTON_LEFT] = BTN_LEFT, [INPUT_BUTTON_RIGHT] = BTN_RIGHT, [INPUT_BUTTON_MIDDLE] = BTN_MIDDLE, - [INPUT_BUTTON_WHEEL_UP] = BTN_GEAR_UP, - [INPUT_BUTTON_WHEEL_DOWN] = BTN_GEAR_DOWN, + [INPUT_BUTTON_WHEELUP] = BTN_GEAR_UP, + [INPUT_BUTTON_WHEELDOWN] = BTN_GEAR_DOWN, };static const unsigned int axismap_rel[INPUT_AXIS__MAX ] = {
include/qapi/error.h +6 -6
@@ -113,12 +113,12 @@ typedef struct Error Error;* enum names. */ typedef enum ErrorClass { -ERROR_CLASS_GENERIC_ERROR =QAPI_ERROR_CLASS_GENERIC_ERROR , -ERROR_CLASS_COMMAND_NOT_FOUND =QAPI_ERROR_CLASS_COMMAND_NOT_FOUND , -ERROR_CLASS_DEVICE_ENCRYPTED =QAPI_ERROR_CLASS_DEVICE_ENCRYPTED , -ERROR_CLASS_DEVICE_NOT_ACTIVE =QAPI_ERROR_CLASS_DEVICE_NOT_ACTIVE , -ERROR_CLASS_DEVICE_NOT_FOUND =QAPI_ERROR_CLASS_DEVICE_NOT_FOUND , -ERROR_CLASS_KVM_MISSING_CAP =QAPI_ERROR_CLASS_KVM_MISSING_CAP , +ERROR_CLASS_GENERIC_ERROR =QAPI_ERROR_CLASS_GENERICERROR , +ERROR_CLASS_COMMAND_NOT_FOUND =QAPI_ERROR_CLASS_COMMANDNOTFOUND , +ERROR_CLASS_DEVICE_ENCRYPTED =QAPI_ERROR_CLASS_DEVICEENCRYPTED , +ERROR_CLASS_DEVICE_NOT_ACTIVE =QAPI_ERROR_CLASS_DEVICENOTACTIVE , +ERROR_CLASS_DEVICE_NOT_FOUND =QAPI_ERROR_CLASS_DEVICENOTFOUND , +ERROR_CLASS_KVM_MISSING_CAP =QAPI_ERROR_CLASS_KVMMISSINGCAP , }ErrorClass ;/*
monitor.c +1 -1
@@ -1375,7 +1375,7 @@ static void hmp_mouse_move(Monitor *mon, const QDict *qdict)if (dz_str) { dz =strtol (dz_str,NULL ,0 );if (dz !=0 ) { - button = (dz >0 ) ?INPUT_BUTTON_WHEEL_UP :INPUT_BUTTON_WHEEL_DOWN ; + button = (dz >0 ) ?INPUT_BUTTON_WHEELUP :INPUT_BUTTON_WHEELDOWN ;qemu_input_queue_btn (NULL , button, true);qemu_input_event_sync ();qemu_input_queue_btn (NULL , button, false);
scripts/qapi.py +1 -1
@@ -1439,7 +1439,7 @@ def camel_to_upper(value): def c_enum_const(type_name, const_name, prefix=None): if prefix is not None: type_name = prefix - return camel_to_upper(type_name + '_' + const_name) + return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper() c_name_trans = string.maketrans('.-', '__')
ui/cocoa.m +2 -2
@@ -729,8 +729,8 @@ QemuCocoaView *cocoaView; [INPUT_BUTTON_LEFT] = MOUSE_EVENT_LBUTTON, [INPUT_BUTTON_MIDDLE] = MOUSE_EVENT_MBUTTON, [INPUT_BUTTON_RIGHT] = MOUSE_EVENT_RBUTTON, - [INPUT_BUTTON_WHEEL_UP] = MOUSE_EVENT_WHEELUP, - [INPUT_BUTTON_WHEEL_DOWN] = MOUSE_EVENT_WHEELDN, + [INPUT_BUTTON_WHEELUP] = MOUSE_EVENT_WHEELUP, + [INPUT_BUTTON_WHEELDOWN] = MOUSE_EVENT_WHEELDN, }; qemu_input_update_buttons(dcl->con, bmap, last_buttons, buttons); last_buttons = buttons;
ui/gtk.c +2 -2
@@ -965,9 +965,9 @@ static gboolean gd_scroll_event(GtkWidget *widget, GdkEventScroll *scroll,InputButton btn;if (scroll->direction ==GDK_SCROLL_UP ) { - btn =INPUT_BUTTON_WHEEL_UP ; + btn =INPUT_BUTTON_WHEELUP ; }else if (scroll->direction ==GDK_SCROLL_DOWN ) { - btn =INPUT_BUTTON_WHEEL_DOWN ; + btn =INPUT_BUTTON_WHEELDOWN ; }else {return TRUE; }
ui/input-legacy.c +2 -2
@@ -157,7 +157,7 @@ static void legacy_mouse_event(DeviceState *dev, QemuConsole *src, }else { s->buttons &= ~bmap[evt->u.btn->button]; } -if (evt->u.btn->down && evt->u.btn->button ==INPUT_BUTTON_WHEEL_UP ) { +if (evt->u.btn->down && evt->u.btn->button ==INPUT_BUTTON_WHEELUP ) { s->qemu_put_mouse_event(s->qemu_put_mouse_event_opaque, s->axis[INPUT_AXIS_X], s->axis[INPUT_AXIS_Y],
@@ -165,7 +165,7 @@ static void legacy_mouse_event(DeviceState *dev, QemuConsole *src, s->buttons); }if (evt->u.btn->down && - evt->u.btn->button ==INPUT_BUTTON_WHEEL_DOWN ) { + evt->u.btn->button ==INPUT_BUTTON_WHEELDOWN ) { s->qemu_put_mouse_event(s->qemu_put_mouse_event_opaque, s->axis[INPUT_AXIS_X], s->axis[INPUT_AXIS_Y],
ui/sdl.c +2 -2
@@ -469,8 +469,8 @@ static void sdl_send_mouse_event(int dx, int dy, int x, int y, int state) [INPUT_BUTTON_LEFT] = SDL_BUTTON(SDL_BUTTON_LEFT), [INPUT_BUTTON_MIDDLE] = SDL_BUTTON(SDL_BUTTON_MIDDLE), [INPUT_BUTTON_RIGHT] = SDL_BUTTON(SDL_BUTTON_RIGHT), - [INPUT_BUTTON_WHEEL_UP] = SDL_BUTTON(SDL_BUTTON_WHEELUP), - [INPUT_BUTTON_WHEEL_DOWN] = SDL_BUTTON(SDL_BUTTON_WHEELDOWN), + [INPUT_BUTTON_WHEELUP] = SDL_BUTTON(SDL_BUTTON_WHEELUP), + [INPUT_BUTTON_WHEELDOWN] = SDL_BUTTON(SDL_BUTTON_WHEELDOWN), };static uint32_t prev_state;
ui/sdl2.c +2 -2
@@ -504,9 +504,9 @@ static void handle_mousewheel(SDL_Event *ev)InputButton btn;if (wev->y >0 ) { - btn =INPUT_BUTTON_WHEEL_UP ; + btn =INPUT_BUTTON_WHEELUP ; }else if (wev->y <0 ) { - btn =INPUT_BUTTON_WHEEL_DOWN ; + btn =INPUT_BUTTON_WHEELDOWN ; }else {return ; }
ui/spice-input.c +2 -2
@@ -111,8 +111,8 @@ static void spice_update_buttons(QemuSpicePointer *pointer, [INPUT_BUTTON_LEFT] = 0x01, [INPUT_BUTTON_MIDDLE] = 0x04, [INPUT_BUTTON_RIGHT] = 0x02, - [INPUT_BUTTON_WHEEL_UP] = 0x10, - [INPUT_BUTTON_WHEEL_DOWN] = 0x20, + [INPUT_BUTTON_WHEELUP] = 0x10, + [INPUT_BUTTON_WHEELDOWN] = 0x20, };if (wheel <0 ) {
ui/vnc.c +2 -2
@@ -1675,8 +1675,8 @@ static void pointer_event(VncState *vs, int button_mask, int x, int y) [INPUT_BUTTON_LEFT] = 0x01, [INPUT_BUTTON_MIDDLE] = 0x02, [INPUT_BUTTON_RIGHT] = 0x04, - [INPUT_BUTTON_WHEEL_UP] = 0x08, - [INPUT_BUTTON_WHEEL_DOWN ] =0x10 , + [INPUT_BUTTON_WHEELUP] = 0x08, + [INPUT_BUTTON_WHEELDOWN ] =0x10 , }; QemuConsole *con = vs->vd->dcl.con;int width =pixman_image_get_width (vs->vd->server);
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 26/36] qapi: Change munging of CamelCase enum values
Quoted 13 lines · Eric Blake
When munging enum values, the fact that we were passing the entire prefix + value through
camel_to_upper()meant that enum values spelled with CamelCase could be turned intoCAMEL_CASE. However, this provides a potential collision (both OneTwo and One-Two would munge intoONE_TWO) for enum types, when the same two names are valid side-by-side as qapi member names. By changing the generation of enum constants to always be prefix + '_' +c_name(value, False).upper(), and ensuring that there are no case collisions (in the next patches), we no longer have to worry about names that would be distinct as qapi members but collide as variant tag names, without having to think about what munging the heuristics incamel_to_upper()will actually perform on an enum value.
Quoted 6 lines
Making the change will affect enums that did not follow coding conventions, using 'CamelCase' rather than desired 'lower-case'.
Thankfully, there are only two culprits: InputButton and ErrorClass. We already tweaked ErrorClass to make it an alias of QapiErrorClass, where only the alias needs changingrather than the whole tree. So
s/changingrather/changing rather/
Quoted 9 lines
the bulk of this change is modifying
INPUT_BUTTON_WHEEL_UPto the newINPUT_BUTTON_WHEELUP(and likewise for WHEELDOWN). That part of this commit may later need reverting if we rename the enum constants from 'WheelUp' to 'wheel-up' as part of moving x-input-send-event to a stable interface; but at least we have documentation bread crumbs in place to remind us (commit 513e7cd), and it matches the fact that SDL constants are also spelledSDL_BUTTON_WHEELUP.
Quoted 6 lines
Suggested by: Markus Armbruster <[email protected]>
Signed-off-by Eric Blake <[email protected]>---
v12: rebase to simpler ErrorClass changes, document decisions on InputButton
The ugliness is now confined to just the definition of enum ErrorClass. Good.
PATCH v12 27/36 qapi: Forbid case-insensitive clashes
In reply to Eric BlakeIn general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by c_name(), but which cannot be distinguished when mapped to C as ALL_CAPS by c_enum_const() [via c_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.
We do not have to care about c_name(name, True) vs. c_name(name, False), as long as any pair of munged names being compared were munged using the same flag value to c_name(). This is because commit 9fb081e already reserved names munging to 'q_', and this patch extends the reservation to 'Q_'; and because recent patches fixed things to ensure the only thing done by the flag is adding the prefix 'q_', that the only use of '.' (which also munges to '_') is in downstream extension prefixes, and that enum munging does not add underscores to any CamelCase values.
However, we DO have to care about the fact that we have a command 'stop' and an event 'STOP' (currently the only case collision of any of our .json entities). To solve that, use some tricks in the lookup map for entities. With some careful choice of keys, we can bisect the map into two collision pools (name.upper() for events, name.lower() for all else). Since we already document that no two entities can have the exact same spelling, an entity can only occur under one of the two partitions of the map. We could go further and enforce that events are named with 'ALL_CAPS' and that nothing else is named in that manner; but that can be done as a followup if desired. We could also separate commands from type names, but then we no longer have a convenient upper vs. lower partitioning allowing us to share a single dictionary.
In order to keep things stable, the visit() method has to ensure that it visits entities sorted by their real name, and not by the new munged keys of the dictionary; Python's attrgetter is a lifesaver for this task.
There is also the possibility that we may want to add a future extension to QMP of teaching it to be case-insensitive (the user could request command 'Quit' instead of 'quit', or could spell a struct field as 'CPU' instead of 'cpu'). But for that to be a practical extension, we cannot break backwards compatibility with any existing struct that was already relying on case sensitivity. Fortunately, after a recent patch cleaned up CpuInfo, there are no such existing qapi structs. Of course, the idea of a future extension is not as strong of a reason to make the change.
At any rate, it is easier to be strict now, and relax things later if we find a reason to need case-sensitive QMP members, than it would be to wish we had the restriction in place.
Add new tests args-case-clash.json and command-type-case-clash.json to demonstrate that the collision detection works.
---
v12: add in entity case collisions (required sharing two maps), improve commit message
v11: rebase to latest, don't focus so hard on future case-insensitive extensions, adjust commit message
v10: new patch
--- docs/qapi-code-gen.txt | 7 +++++ scripts/qapi.py | 41 +++++++++++++++++++++----- tests/Makefile | 2 ++ tests/qapi-schema/args-case-clash.err | 1 + tests/qapi-schema/args-case-clash.exit | 1 + tests/qapi-schema/args-case-clash.json | 4 +++ tests/qapi-schema/args-case-clash.out | 0 tests/qapi-schema/command-type-case-clash.err | 1 + tests/qapi-schema/command-type-case-clash.exit | 1 + tests/qapi-schema/command-type-case-clash.json | 3 ++ tests/qapi-schema/command-type-case-clash.out | 0 11 files changed, 53 insertions(+), 8 deletions(-) create mode 100644 tests/qapi-schema/args-case-clash.err create mode 100644 tests/qapi-schema/args-case-clash.exit create mode 100644 tests/qapi-schema/args-case-clash.json create mode 100644 tests/qapi-schema/args-case-clash.out create mode 100644 tests/qapi-schema/command-type-case-clash.err create mode 100644 tests/qapi-schema/command-type-case-clash.exit create mode 100644 tests/qapi-schema/command-type-case-clash.json create mode 100644 tests/qapi-schema/command-type-case-clash.out
docs/qapi-code-gen.txt +7 -0
@@ -102,6 +102,13 @@ single-dimension array of that type; multi-dimension arrays are not directly supported (although an array of a complex struct that contains an array member is possible). +Client JSON Protocol is case-sensitive. However, the generator +rejects attempts to create object members that differ only in case +after normalization (thus 'a-b' and 'A_B' collide); and likewise +rejects attempts to create commands or types that differ only in case, +or events that differ only in case (it is possible to have a command +and event map to the same case-insensitive name, though). + Types, commands, and events share a common namespace. Therefore, generally speaking, type definitions should always use CamelCase for user-defined type names, while built-in types are lowercase. Type
scripts/qapi.py +33 -8
86 lines not on this page · raw message
tests/Makefile +2 -0
@@ -242,6 +242,7 @@ qapi-schema += args-alternate.json qapi-schema += args-any.json qapi-schema += args-array-empty.json qapi-schema += args-array-unknown.json +qapi-schema += args-case-clash.json qapi-schema += args-int.json qapi-schema += args-invalid.json qapi-schema += args-member-array-bad.json
@@ -256,6 +257,7 @@ qapi-schema += bad-type-bool.json qapi-schema += bad-type-dict.json qapi-schema += bad-type-int.json qapi-schema += command-int.json +qapi-schema += command-type-case-clash.json qapi-schema += comments.json qapi-schema += double-data.json qapi-schema += double-type.json
tests/qapi-schema/args-case-clash.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/args-case-clash.json:4: 'A' (parameter of oops) collides with 'a' (parameter of oops)
tests/qapi-schema/args-case-clash.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/args-case-clash.json new file +4 -0
@@ -0,0 +1,4 @@ +# C member name collision +# Reject members that clash case-insensitively, even though our mapping to +# C names preserves case. +{ 'command': 'oops', 'data': { 'a': 'str', 'A': 'int' } }
tests/qapi-schema/args-case-clash.out new file +0 -0
tests/qapi-schema/command-type-case-clash.err new file +1 -0
@@ -0,0 +1 @@ +tests/qapi-schema/command-type-case-clash.json:3: Entity 'foo' collides with 'Foo'
tests/qapi-schema/command-type-case-clash.exit new file +1 -0
@@ -0,0 +1 @@ +1
tests/qapi-schema/command-type-case-clash.json new file +3 -0
@@ -0,0 +1,3 @@ +# we reject commands that would differ only case from a type +{ 'struct': 'Foo', 'data': { 'i': 'int' } } +{ 'command': 'foo', 'data': { 'character': 'str' } }
tests/qapi-schema/command-type-case-clash.out new file +0 -0
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
Quoted 123 lines · Eric Blake
In general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by
c_name(), but which cannot be distinguished when mapped to C asALL_CAPSbyc_enum_const()[viac_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.We do not have to care about
c_name(name, True)vs.c_name(name, False), as long as any pair of munged names being compared were munged using the same flag value toc_name(). This is because commit 9fb081e already reserved names munging to 'q_', and this patch extends the reservation to 'Q_'; and because recent patches fixed things to ensure the only thing done by the flag is adding the prefix 'q_', that the only use of '.' (which also munges to '_') is in downstream extension prefixes, and that enum munging does not add underscores to any CamelCase values.However, we DO have to care about the fact that we have a command 'stop' and an event 'STOP' (currently the only case collision of any of our .json entities). To solve that, use some tricks in the lookup map for entities. With some careful choice of keys, we can bisect the map into two collision pools (
name.upper()for events,name.lower()for all else). Since we already document that no two entities can have the exact same spelling, an entity can only occur under one of the two partitions of the map. We could go further and enforce that events are named with 'ALL_CAPS' and that nothing else is named in that manner; but that can be done as a followup if desired. We could also separate commands from type names, but then we no longer have a convenient upper vs. lower partitioning allowing us to share a single dictionary.In order to keep things stable, the
visit()method has to ensure that it visits entities sorted by their real name, and not by the new munged keys of the dictionary; Python's attrgetter is a lifesaver for this task.There is also the possibility that we may want to add a future extension to QMP of teaching it to be case-insensitive (the user could request command 'Quit' instead of 'quit', or could spell a struct field as 'CPU' instead of 'cpu'). But for that to be a practical extension, we cannot break backwards compatibility with any existing struct that was already relying on case sensitivity. Fortunately, after a recent patch cleaned up CpuInfo, there are no such existing qapi structs. Of course, the idea of a future extension is not as strong of a reason to make the change.
At any rate, it is easier to be strict now, and relax things later if we find a reason to need case-sensitive QMP members, than it would be to wish we had the restriction in place.
Add new tests args-case-clash.json and command-type-case-clash.json to demonstrate that the collision detection works.
Signed-off-by Eric Blake <[email protected]>---
v12: add in entity case collisions (required sharing two maps), improve commit message
v11: rebase to latest, don't focus so hard on future case-insensitive extensions, adjust commit message
v10: new patch--- docs/qapi-code-gen.txt | 7 +++++ scripts/qapi.py | 41 +++++++++++++++++++++----- tests/Makefile | 2 ++ tests/qapi-schema/args-case-clash.err | 1 + tests/qapi-schema/args-case-clash.exit | 1 + tests/qapi-schema/args-case-clash.json | 4 +++ tests/qapi-schema/args-case-clash.out | 0 tests/qapi-schema/command-type-case-clash.err | 1 + tests/qapi-schema/command-type-case-clash.exit | 1 + tests/qapi-schema/command-type-case-clash.json | 3 ++ tests/qapi-schema/command-type-case-clash.out | 0 11 files changed, 53 insertions(+), 8 deletions(-) create mode 100644 tests/qapi-schema/args-case-clash.err create mode 100644 tests/qapi-schema/args-case-clash.exit create mode 100644 tests/qapi-schema/args-case-clash.json create mode 100644 tests/qapi-schema/args-case-clash.out create mode 100644 tests/qapi-schema/command-type-case-clash.err create mode 100644 tests/qapi-schema/command-type-case-clash.exit create mode 100644 tests/qapi-schema/command-type-case-clash.json create mode 100644 tests/qapi-schema/command-type-case-clash.outdocs/qapi-code-gen.txt +7
-0@@ -102,6 +102,13 @@ single-dimension array of that type; multi-dimension arrays are not directly supported (although an array of a complex struct that contains an array member is possible). +Client JSON Protocol is case-sensitive. However, the generator +rejects attempts to create object members that differ only in case +after normalization (thus 'a-b' and 'A_B' collide); and likewise +rejects attempts to create commands or types that differ only in case, +or events that differ only in case (it is possible to have a command +and event map to the same case-insensitive name, though). + Types, commands, and events share a common namespace. Therefore, generally speaking, type definitions should always use CamelCase for user-defined type names, while built-in types are lowercase. Typescripts/qapi.py +1
-0@@ -13,6 +13,7 @@ import re from ordereddict import OrderedDict +from operator import attrgetter import errno import getopt import os@@ -378,9 +379,9 @@ def check_name(expr_info, source, name, allow_optional=False, # code always prefixes it with the enum name
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 387
Quoted 10 lines
scripts/qapi.py +2
-2if enum_member and membername[0].isdigit(): membername = 'D' + membername - # Reserve the entire 'q_' namespace for c_name() + # Reserve the entire 'q_'/'Q_' namespace for c_name() if not valid_name.match(membername) or \ - c_name(membername, False).startswith('q_'): + c_name(membername, False).upper().startswith('Q_'): raise QAPIExprError(expr_info, "%s uses invalid name '%s'" % (source, name))
We use .upper() to get a case-insensitive (in the C locale!) string compare. That's fine. .lower() would be just as fine. But see [*] below.
Quoted 53 lines
scripts/qapi.py +21
-5@@ -1040,7 +1041,7 @@ class QAPISchemaObjectTypeMember(object): assert self.type def check_clash(self, info, seen): - cname = c_name(self.name) + cname = c_name(self.name).upper() if cname in seen: raise QAPIExprError(info, "%s collides with %s"@@ -1085,7 +1086,7 @@ class QAPISchemaObjectTypeVariants(object): def check(self, schema, seen): if not self.tag_member: # flat union - self.tag_member = seen[c_name(self.tag_name)] + self.tag_member = seen[c_name(self.tag_name).upper()] assert self.tag_name == self.tag_member.name assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants:@@ -1189,6 +1190,8 @@ class QAPISchema(object): def __init__(self, fname): try: self.exprs = check_exprs(QAPISchemaParser(open(fname, "r")).exprs) + # _entity_dict holds two namespaces: events are stored via + # name.upper(), commands and types are stored via name.lower(). self._entity_dict = {} self._predefining = True self._def_predefineds()@@ -1202,11 +1205,32 @@ class QAPISchema(object): def _def_entity(self, ent): # Only the predefined types are allowed to not have info assert ent.info or self._predefining - assert ent.name not in self._entity_dict - self._entity_dict[ent.name] = ent + # On insertion, we need to check for an exact match in either + # namespace, then for case collision in a single namespace + if self.lookup_entity(ent.name): + raise QAPIExprError(ent.info, + "Entity '%s' already defined" % end.name) + cname = c_name(ent.name) + if isinstance(ent, QAPISchemaEvent): + cname = cname.upper() + else: + cname = cname.lower() + if cname in self._entity_dict: + raise QAPIExprError(ent.info, + "Entity '%s' collides with '%s'" + % (ent.name, self._entity_dict[cname].name)) + self._entity_dict[cname] = ent def lookup_entity(self, name, typ=None): - ent = self._entity_dict.get(name) + # No thanks to 'stop'/'STOP', we have to check two namespaces during + # lookups, but only return a result on exact match.
scripts/qapi.py class QAPISchema(object): · line 1236
Quoted 10 lines
scripts/qapi.py +7
-0+ ent1 = self._entity_dict.get(c_name(name).lower()) # commands, types + ent2 = self._entity_dict.get(c_name(name).upper()) # events + ent = None + if ent1 and ent1.name == name: + ent = ent1 + elif ent2 and ent2.name == name: + ent = ent2 if typ and not isinstance(ent, typ): return None return ent
Let's see how this works.
Before the patch, _entity_dict's key serves one purpose: mapping names to entities the obvious way: use entity name as dictionary key.
lookup_entity():
ent = self._entity_dict.get(name)
# not ent or ent.name == name
_def_entity():
assert ent.name not in self._entity_dict
self._entity_dict[ent.name] = ent
Your patch presses _entity_dict into clash detection service. I'm going to split the change into pieces I can understand.
First step changes the key from name to c_name(name) to ignore [-._] differences (the ticklish identifier protection has no effect). The dictionary lookup changes from "the entity with this name" to "the entity whose name clashes with this name". To preserve semantics, the two functions need to compensate.
lookup_entity():
ent = self._entity_dict.get(c_name(name))
if ent and ent.name != name
ent = None
# not ent or ent.name == name
_def_entity():
cname = c_name(ent.name)
assert cname not in self._entity_dict
self._entity_dict[cname] = ent
A bit less obvious, but not too bad.
We can then turn _def_entity()'s assertion into error reporting. It splits into two separate errors, "already defined" and "collides".
_def_entity():
if self.lookup_entity(ent.name):
raise QAPIExprError(ent.info,
"Entity '%s' already defined" % end.name)
cname = c_name(ent.name)
if cname in self._entity_dict:
raise QAPIExprError(ent.info,
"Entity '%s' collides with '%s'" % (ent.name, self._entity_dict[cname].name))
self._entity_dict[cname] = ent
To catch case-insensitive clashes, you change the key further to .lower(). No additional compensation necessary.
However, this makes command 'stop' clash with event 'STOP'. You cleverly sidestep that by partitioning _entity_dict's key space into upper case for events and lower case for everything else. Works, because every entity name contains at least one letter.
_def_entity()'s cname = c_name(ent.name).lower() becomes
cname = c_name(ent.name)
if isinstance(ent, QAPISchemaEvent):
cname = cname.upper()
else:
cname = cname.lower()
Digression [*]: I think using .lower() in check_name() would be slightly cleaner, because it matches the case mapping we use here.
lookup_entity() is even worse off. There, we don't know whether we're looking for an event or not, so we try both. If the entity is defined, exactly one will succeed, else none. Resulting code:
ent1 = self._entity_dict.get(c_name(name).lower()) # commands, types
ent2 = self._entity_dict.get(c_name(name).upper()) # events
ent = None
if ent1 and ent1.name == name:
ent = ent1
elif ent2 and ent2.name == name:
ent = ent2
All right, this should work.
Now let me try claw back some clarity.
First try: refine your approach.
Observe that all but one caller of lookup_entity() actually look up types. The one caller that doesn't is _def_entity(). Change it to:
cname = c_name(ent.name)
if isinstance(ent, QAPISchemaEvent):
cname = cname.upper()
else:
cname = cname.lower()
if self._lookup_entity(cname)
raise QAPIExprError(ent.info,
"Entity '%s' already defined" % end.name)
if cname in self._entity_dict:
raise QAPIExprError(ent.info,
"Entity '%s' collides with '%s'" % (ent.name, self._entity_dict[cname].name))
where _lookup_entity() is back to the much simpler version:
ent = self._entity_dict.get(cname)
if ent and ent.name != cname
ent = None
lookup_type() becomes:
def lookup_type(self, name, typ=QAPISchemaType):
return self._lookup_entity(name, typ)
and the third caller _make_implicit_object_type() goes from
if not self.lookup_entity(name, QAPISchemaObjectType):
to
if not self.lookup_type(name, QAPISchemaObjectType):
Another possible improvement is hiding the case-folding logic in methods. Have a QAPISchemaEntity.c_namecase() that returns self.c_name().lower(). Overwrite it in QAPISchemaEvent to return .upper() instead. Use it to make _def_entity() less ugly.
Probably not worthwhile unless more uses of .c_namecase() pop up.
Second approach: don't use _entity_dict for clash detection! Have a second dictionary.
lookup_entity() remains untouched. _def_entity() becomes something like:
if ent.name in self._entity_dict:
raise QAPIExprError(ent.info,
"Entity '%s' already defined" % end.name)
if ent.c_namecase() in self._clash_dict:
raise QAPIExprError(ent.info,
"Entity '%s' collides with '%s'" % (ent.name, self._entity_dict[cname].name))
self._entity_dict[ent.name] = ent self._clash_dict[cname] = ent
or the equivalent code without c_namecase().
scripts/qapi.py class QAPISchema(object): · line 1421
Quoted 10 lines
scripts/qapi.py +2
-1@@ -1390,7 +1414,8 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) - for (name, entity) in sorted(self._entity_dict.items()): + for entity in sorted(self._entity_dict.values(), + key=attrgetter('name')): if visitor.visit_needed(entity): entity.visit(visitor) visitor.visit_end()
Cool trick.
The order of items in dict.items() may vary:
Keys and values are listed in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary’s history of insertions and deletions.
https://docs.python.org/2.7/library/stdtypes.html#dict.items
By sorting, we go from "arbitrary order that can vary" to "arbitrary fixed order". That's our only reason for sorting.
Your trick preserves the arbitrary order we use before the patch. Good idea, because it makes verifying the patch doesn't mess up output easier.
We could revert the trick in a followup patch, however.
Quoted 53 lines
tests/Makefile +2
-0@@ -242,6 +242,7 @@ qapi-schema += args-alternate.json qapi-schema += args-any.json qapi-schema += args-array-empty.json qapi-schema += args-array-unknown.json +qapi-schema += args-case-clash.json qapi-schema += args-int.json qapi-schema += args-invalid.json qapi-schema += args-member-array-bad.json@@ -256,6 +257,7 @@ qapi-schema += bad-type-bool.json qapi-schema += bad-type-dict.json qapi-schema += bad-type-int.json qapi-schema += command-int.json +qapi-schema += command-type-case-clash.json qapi-schema += comments.json qapi-schema += double-data.json qapi-schema += double-type.jsontests/qapi-schema/args-case-clash.err new file +1
-0@@ -0,0 +1 @@ +tests/qapi-schema/args-case-clash.json:4: 'A' (parameter of oops) collides with 'a' (parameter of oops)tests/qapi-schema/args-case-clash.exit new file +1
-0@@ -0,0 +1 @@ +1tests/qapi-schema/args-case-clash.json new file +4
-0@@ -0,0 +1,4 @@ +# C member name collision +# Reject members that clash case-insensitively, even though our mapping to +# C names preserves case. +{ 'command': 'oops', 'data': { 'a': 'str', 'A': 'int' } }tests/qapi-schema/args-case-clash.out new file +0
-0tests/qapi-schema/command-type-case-clash.err new file +0
-0
tests/qapi-schema/command-type-case-clash.err line 1
Quoted 1 line
tests/qapi-schema/command-type-case-clash.err new file +1
-0@@ -0,0 +1 @@ +tests/qapi-schema/command-type-case-clash.json:3: Entity 'foo' collides with 'Foo'
The message's location is foo's. An additional message "'Foo' defined here" would be nice. Just an idea, could be done later.
Quoted 19 lines
tests/qapi-schema/command-type-case-clash.exit new file +1
-0@@ -0,0 +1 @@ +1tests/qapi-schema/command-type-case-clash.json new file +3
-0@@ -0,0 +1,3 @@ +# we reject commands that would differ only case from a type +{ 'struct': 'Foo', 'data': { 'i': 'int' } } +{ 'command': 'foo', 'data': { 'character': 'str' } }tests/qapi-schema/command-type-case-clash.out new file +0
-0
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
Quoted 15 lines · Markus Armbruster
Quoted 12 lines · Eric Blake
In general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by
c_name(), but which cannot be distinguished when mapped to C asALL_CAPSbyc_enum_const()[viac_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.
Quoted 11 lines
scripts/qapi.py def check_name(expr_info, source, name, allow_optional=False, · line 386
Quoted 11 lines
scripts/qapi.py +2
-2@@ -378,9 +379,9 @@ def check_name(expr_info, source, name, allow_optional=False, # code always prefixes it with the enum name if enum_member and membername[0].isdigit(): membername = 'D' + membername - # Reserve the entire 'q_' namespace for c_name() + # Reserve the entire 'q_'/'Q_' namespace for c_name() if not valid_name.match(membername) or \ - c_name(membername, False).startswith('q_'): + c_name(membername, False).upper().startswith('Q_'): raise QAPIExprError(expr_info, "%s uses invalid name '%s'" % (source, name))
I'll switch to lower() here, for consistency.
Quoted 7 lines
Now let me try claw back some clarity.
First try: refine your approach.
Observe that all but one caller of
lookup_entity()actually look up types. The one caller that doesn't is_def_entity(). Change it to:
Quoted 6 lines
cname =
c_name(ent.name)
ifisinstance(ent, QAPISchemaEvent):
cname =cname.upper()
else:
cname =cname.lower()
ifself._lookup_entity(cname)
I think lookup_entity would need two parameters: the munged name, and the actual name...
Quoted 11 lines
raise QAPIExprError(ent.info,
"Entity '%s' already defined" % end.name)
if cname in self._entity_dict:
raise QAPIExprError(ent.info,
"Entity '%s' collides with '%s'" % (ent.name, self._entity_dict[cname].name))where
_lookup_entity()is back to the much simpler version:ent =
self._entity_dict.get(cname)
if ent and ent.name != cname
...because ent.name does not have to match cname.
Quoted 6 lines
ent = None
lookup_type()becomes:def
lookup_type(self, name, typ=QAPISchemaType):
returnself._lookup_entity(name, typ)
I don't know that lookup_type needs an optional parameter.
Quoted 8 lines
and the third caller
_make_implicit_object_type()goes fromif not
self.lookup_entity(name, QAPISchemaObjectType):to
if not
self.lookup_type(name, QAPISchemaObjectType):
And a plain lookup_type(name) is what we should have been using here all along.
Quoted 7 lines
Another possible improvement is hiding the case-folding logic in methods. Have a
QAPISchemaEntity.c_namecase()that returns self.c_name().lower(). Overwrite it in QAPISchemaEvent to return .upper() instead. Use it to make_def_entity()less ugly.Probably not worthwhile unless more uses of .c_namecase() pop up.
I kind of like it. I think I'll propose a 26.5 that implements that, then redo 27 on top of it (without reposting the entire series).
Quoted 3 lines
Second approach: don't use
_entity_dictfor clash detection! Have a second dictionary.
I debated about doing that the first time around. But it's probably a bit easier to follow, so I think I'll do it.
Quoted 12 lines
scripts/qapi.py class QAPISchema(object): · line 1421
Quoted 10 lines
scripts/qapi.py +2
-1@@ -1390,7 +1414,8 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) - for (name, entity) in sorted(self._entity_dict.items()): + for entity in sorted(self._entity_dict.values(), + key=attrgetter('name')): if visitor.visit_needed(entity): entity.visit(visitor) visitor.visit_end()Cool trick.
and with two dicts, I wouldn't need this trick.
Quoted 6 lines
tests/qapi-schema/command-type-case-clash.err line 1
Quoted 3 lines
+++ b/tests/qapi-schema/command-type-case-clash.err
tests/qapi-schema/command-type-case-clash.err new file +1
-0@@ -0,0 +1 @@ +tests/qapi-schema/command-type-case-clash.json:3: Entity 'foo' collides with 'Foo'The message's location is foo's. An additional message "'Foo' defined here" would be nice. Just an idea, could be done later.
The way we currently generate a single line message is by raising a python exception. I guess we could rework things to build up lines at a time, then finally raise the complete multi-line message on a pre-formed string, rather than the current building the message as a parameter to the exception constructor. But any changes on this front will have to wait for later.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
Quoted 22 lines · Eric Blake
Quoted 21 lines · Markus Armbruster
Quoted 12 lines · Eric Blake
In general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by
c_name(), but which cannot be distinguished when mapped to C asALL_CAPSbyc_enum_const()[viac_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.Another possible improvement is hiding the case-folding logic in methods. Have a
QAPISchemaEntity.c_namecase()that returns self.c_name().lower(). Overwrite it in QAPISchemaEvent to return .upper() instead. Use it to make_def_entity()less ugly.
Quoted 4 lines
Quoted 1 line
Probably not worthwhile unless more uses of .c_namecase() pop up.
I kind of like it. I think I'll propose a 26.5 that implements that, then redo 27 on top of it (without reposting the entire series).
Actually, it's not worth splitting that into a separate patch (it was just 6 lines, and I couldn't find any other uses).
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
In reply to Eric BlakeQuoted 14 lines · Eric Blake
In general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by
c_name(), but which cannot be distinguished when mapped to C asALL_CAPSbyc_enum_const()[viac_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.
scripts/qapi.py class QAPISchema(object): · line 1212
Quoted 10 lines
scripts/qapi.py +5
-2@@ -1202,11 +1205,32 @@ class QAPISchema(object): def _def_entity(self, ent): # Only the predefined types are allowed to not have info assert ent.info or self._predefining - assert ent.name not in self._entity_dict - self._entity_dict[ent.name] = ent + # On insertion, we need to check for an exact match in either + # namespace, then for case collision in a single namespace + if self.lookup_entity(ent.name): + raise QAPIExprError(ent.info, + "Entity '%s' already defined" % end.name)
Latent typo (s/end/ent/); we can't hit this line of code until we nuke ad hoc parser checks. So using an assert here is better in the meantime.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
PATCH 1/1 fixup! qapi: Forbid case-insensitive clashes
In reply to Eric Blake[Replace the old commit message with this:]
qapi: Forbid case-insensitive clashes
In general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by c_name(), but which cannot be distinguished when mapped to C as ALL_CAPS by c_enum_const() [via c_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.
We do not have to care about c_name(name, True) vs. c_name(name, False), as long as any pair of munged names being compared were munged using the same flag value to c_name(). This is because commit 9fb081e already reserved names munging to 'q_', and this patch extends the reservation to 'Q_'; and because recent patches fixed things to ensure the only thing done by the flag is adding the prefix 'q_', that the only use of '.' (which also munges to '_') is in downstream extension prefixes, and that enum munging does not add underscores to any CamelCase values.
However, we DO have to care about the fact that we have a command 'stop' and an event 'STOP' (currently the only case collision of any of our .json entities). To solve that, add a new QAPISchemaEntity.c_namecase() that returns a munged version of the name that can be used in checking for case clashes, and override it in QAPISchemaEvent to use a different case (thus, indexing by .c_namecase() will create two natural key partitions), then add a new _clash_dict to QAPISchema._def_entity() that checks for case collisions after ruling out identical spellings. This works because all entities have at least a leading letter in their name.
We could go further and enforce that events are named with 'ALL_CAPS' and that nothing else is named in that manner; but that can be done as a followup if desired. We could also separate commands from type names (the way we already separated events into a separate namespace), but then we'd have to make .c_namecase() pick something other than the convenient upper() vs. lower() in order to partition dictionary keys into three sets (perhaps a leading digit, since all entities start with a letter).
There is also the possibility that we may want to add a future extension to QMP of teaching it to be case-insensitive (the user could request command 'Quit' instead of 'quit', or could spell a struct field as 'CPU' instead of 'cpu'). But for that to be a practical extension, we cannot break backwards compatibility with any existing struct that was already relying on case sensitivity. Fortunately, after a recent patch cleaned up CpuInfo, there are no such existing qapi structs. Of course, the idea of a future extension is not as strong of a reason to make the change.
At any rate, it is easier to be strict now, and relax things later if we find a reason to need case-sensitive QMP members, than it would be to wish we had the restriction in place.
Add new tests args-case-clash.json and command-type-case-clash.json to demonstrate that the collision detection works.
---
Use a separate clash dictionary, fix a typo that we couldn't trigger, add .c_namecase(), no longer need to use attrgetter
--- scripts/qapi.py | 48 ++++++++++++++++++------------------------------ 1 file changed, 18 insertions(+), 30 deletions(-)
scripts/qapi.py +18 -30
@@ -13,7 +13,6 @@ import re from ordereddict import OrderedDict -from operator import attrgetter import errno import getopt import os
@@ -381,7 +380,7 @@ def check_name(expr_info, source, name, allow_optional=False, membername = 'D' + membername # Reserve the entire 'q_'/'Q_' namespace for c_name() if not valid_name.match(membername) or \ - c_name(membername, False).upper().startswith('Q_'): + c_name(membername, False).lower().startswith('q_'): raise QAPIExprError(expr_info, "%s uses invalid name '%s'" % (source, name))
@@ -801,6 +800,9 @@ class QAPISchemaEntity(object): def c_name(self): return c_name(self.name) + def c_namecase(self): + return c_name(self.name).lower() + def check(self, schema): pass
@@ -1041,7 +1043,7 @@ class QAPISchemaObjectTypeMember(object): assert self.type def check_clash(self, info, seen): - cname = c_name(self.name).upper() + cname = c_name(self.name).lower() if cname in seen: raise QAPIExprError(info, "%s collides with %s"
@@ -1086,7 +1088,7 @@ class QAPISchemaObjectTypeVariants(object): def check(self, schema, seen): if not self.tag_member: # flat union - self.tag_member = seen[c_name(self.tag_name).upper()] + self.tag_member = seen[c_name(self.tag_name).lower()] assert self.tag_name == self.tag_member.name assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants:
@@ -1185,14 +1187,16 @@ class QAPISchemaEvent(QAPISchemaEntity): def visit(self, visitor): visitor.visit_event(self.name, self.info, self.arg_type) + def c_namecase(self): + return c_name(self.name).upper() + class QAPISchema(object): def __init__(self, fname): try: self.exprs = check_exprs(QAPISchemaParser(open(fname, "r")).exprs) - # _entity_dict holds two namespaces: events are stored via - # name.upper(), commands and types are stored via name.lower(). self._entity_dict = {} + self._clash_dict = {} self._predefining = True self._def_predefineds() self._predefining = False
@@ -1205,32 +1209,17 @@ class QAPISchema(object): def _def_entity(self, ent): # Only the predefined types are allowed to not have info assert ent.info or self._predefining - # On insertion, we need to check for an exact match in either - # namespace, then for case collision in a single namespace - if self.lookup_entity(ent.name): - raise QAPIExprError(ent.info, - "Entity '%s' already defined" % end.name) - cname = c_name(ent.name) - if isinstance(ent, QAPISchemaEvent): - cname = cname.upper() - else: - cname = cname.lower() - if cname in self._entity_dict: + assert ent.name not in self._entity_dict + cname = ent.c_namecase() + if cname in self._clash_dict: raise QAPIExprError(ent.info, "Entity '%s' collides with '%s'" - % (ent.name, self._entity_dict[cname].name)) - self._entity_dict[cname] = ent + % (ent.name, self._clash_dict[cname])) + self._entity_dict[ent.name] = ent + self._clash_dict[cname] = ent.name def lookup_entity(self, name, typ=None): - # No thanks to 'stop'/'STOP', we have to check two namespaces during - # lookups, but only return a result on exact match. - ent1 = self._entity_dict.get(c_name(name).lower()) # commands, types - ent2 = self._entity_dict.get(c_name(name).upper()) # events - ent = None - if ent1 and ent1.name == name: - ent = ent1 - elif ent2 and ent2.name == name: - ent = ent2 + ent = self._entity_dict.get(name) if typ and not isinstance(ent, typ): return None return ent
@@ -1414,8 +1403,7 @@ class QAPISchema(object): def visit(self, visitor): visitor.visit_begin(self) - for entity in sorted(self._entity_dict.values(), - key=attrgetter('name')): + for (name, entity) in sorted(self._entity_dict.items()): if visitor.visit_needed(entity): entity.visit(visitor) visitor.visit_end()
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH] fixup! qapi: Forbid case-insensitive clashes
Quoted 27 lines · Eric Blake
[Replace the old commit message with this:]
qapi: Forbid case-insensitive clashes
In general, designing user interfaces that rely on case distinction is poor practice. Another benefit of enforcing a restriction against case-insensitive clashes is that we no longer have to worry about the situation of enum values that could be distinguished by case if mapped by
c_name(), but which cannot be distinguished when mapped to C asALL_CAPSbyc_enum_const()[viac_name(name, False).upper()]. Thus, having the generator look for case collisions up front will prevent developers from worrying whether different munging rules for member names compared to enum values as a discriminator will cause any problems in qapi unions.We do not have to care about
c_name(name, True)vs.c_name(name, False), as long as any pair of munged names being compared were munged using the same flag value toc_name(). This is because commit 9fb081e already reserved names munging to 'q_', and this patch extends the reservation to 'Q_'; and because recent patches fixed things to ensure the only thing done by the flag is adding the prefix 'q_', that the only use of '.' (which also munges to '_') is in downstream extension prefixes, and that enum munging does not add underscores to any CamelCase values.
Quoted 6 lines
However, we DO have to care about the fact that we have a command 'stop' and an event 'STOP' (currently the only case collision of any of our .json entities). To solve that, add a new
QAPISchemaEntity.c_namecase()that returns a munged version of the name that can be used in checking for case clashes,
For [-._] clashes, too. Easiest fix it to just say "checking for clashes".
Quoted 4 lines
and override it in QAPISchemaEvent to use a different
case (thus, indexing by .c_namecase() will create two natural key partitions), then add a new_clash_dicttoQAPISchema._def_entity()that checks for case collisions after
To QAPISchema, actually. In _def_entity(), we only add an entry.
Quoted 12 lines
ruling out identical spellings. This works because all entities have at least a leading letter in their name.
We could go further and enforce that events are named with '
ALL_CAPS' and that nothing else is named in that manner; but that can be done as a followup if desired. We could also separate commands from type names (the way we already separated events into a separate namespace), but then we'd have to make .c_namecase() pick something other than the convenientupper()vs.lower()in order to partition dictionary keys into three sets (perhaps a leading digit, since all entities start with a letter).
The upper / lower partition works for events / anything else, and is kind of cute, although two separate clash dictionaries might be clearer. I'd like to sketch that next, to help us decide.
Even if we decide to stick to the "partition the key space" idea now, I'm reluctant to push it farther. Prepending a digit would feel odd. Title case would be slightly better. But I think three separate clash dictionaries would be simpler and clearer then.
Depending on what we do, the paragraph above may need some more tweaking.
Quoted 18 lines
There is also the possibility that we may want to add a future extension to QMP of teaching it to be case-insensitive (the user could request command 'Quit' instead of 'quit', or could spell a struct field as 'CPU' instead of 'cpu'). But for that to be a practical extension, we cannot break backwards compatibility with any existing struct that was already relying on case sensitivity. Fortunately, after a recent patch cleaned up CpuInfo, there are no such existing qapi structs. Of course, the idea of a future extension is not as strong of a reason to make the change.
At any rate, it is easier to be strict now, and relax things later if we find a reason to need case-sensitive QMP members, than it would be to wish we had the restriction in place.
Add new tests args-case-clash.json and command-type-case-clash.json to demonstrate that the collision detection works.
Quoted 8 lines
Signed-off-by Eric Blake <[email protected]>---
Use a separate clash dictionary, fix a typo that we couldn't trigger, add .c_namecase(), no longer need to use attrgetter--- scripts/qapi.py | 48 ++++++++++++++++++------------------------------ 1 file changed, 18 insertions(+), 30 deletions(-)
I find the squashed patch easier to understand, so here it is:
| diff --git a/scripts/qapi.py b/scripts/qapi.py
| index 72925c3..114e07a 100644
| --- a/scripts/qapi.py
| +++ b/scripts/qapi.py
| @@ -378,9 +378,9 @@ def check_name(expr_info, source, name, allow_optional=False,
| # code always prefixes it with the enum name
| if enum_member and membername[0].isdigit():
| membername = 'D' + membername
| - # Reserve the entire 'q_' namespace for c_name()
| + # Reserve the entire 'q_'/'Q_' namespace for c_name()
| if not valid_name.match(membername) or \
| - c_name(membername, False).startswith('q_'):
| + c_name(membername, False).lower().startswith('q_'):
| raise QAPIExprError(expr_info,
| "%s uses invalid name '%s'" % (source, name))
|
| @@ -800,6 +800,9 @@ class QAPISchemaEntity(object):
| def c_name(self):
| return c_name(self.name)
|
| + def c_namecase(self):
| + return c_name(self.name).lower()
| +
| def check(self, schema):
| pass
| The entities are: commands, events, the various types. This implementation of c_namecase() covers commands and types. It gets overwritten for events.
Commands should not use upper case. If we enforced that, the .lower() would be superfluous there. But it isn't wrong.
| @@ -1040,7 +1043,7 @@ class QAPISchemaObjectTypeMember(object): | assert self.type | | def check_clash(self, info, seen): | - cname = c_name(self.name) | + cname = c_name(self.name).lower() | if cname in seen: | raise QAPIExprError(info, | "%s collides with %s"
Not an entity, therefore QAPISchemaEntity.c_namecase() isn't inherited.
Members should not use upper case. If we enforced that, the .lower() would be superfluous.
| @@ -1085,7 +1088,7 @@ class QAPISchemaObjectTypeVariants(object): | | def check(self, schema, seen): | if not self.tag_member: # flat union | - self.tag_member = seen[c_name(self.tag_name)] | + self.tag_member = seen[c_name(self.tag_name).lower()] | assert self.tag_name == self.tag_member.name | assert isinstance(self.tag_member.type, QAPISchemaEnumType) | for v in self.variants:
Key needs to match whatever we use in QAPISchemaObjectTypeMember.check_clash(). It does.
| @@ -1184,12 +1187,16 @@ class QAPISchemaEvent(QAPISchemaEntity): | def visit(self, visitor): | visitor.visit_event(self.name, self.info, self.arg_type) | | + def c_namecase(self): | + return c_name(self.name).upper() | +
Event names should not use lower case. If we enforced that, the .upper() would be superfluous.
If we enforced our naming conventions, namely types in CamelCase, events in ALL_CAPS, everything else lower-case, the c_namecase() would become slightly simpler: it's same as c_name() except for types, where it additionally folds to lower case. The name would be misleading then, however. Anyway, we can consider that once we enforce naming conventions.
|
| class QAPISchema(object):
| def __init__(self, fname):
| try:
| self.exprs = check_exprs(QAPISchemaParser(open(fname, "r")).exprs)
| self._entity_dict = {}
| + self._clash_dict = {}
| self._predefining = True
| self._def_predefineds()
| self._predefining = False
| @@ -1203,7 +1210,13 @@ class QAPISchema(object):
| # Only the predefined types are allowed to not have info
| assert ent.info or self._predefining
| assert ent.name not in self._entity_dict
| + cname = ent.c_namecase()
| + if cname in self._clash_dict:
| + raise QAPIExprError(ent.info,
| + "Entity '%s' collides with '%s'"
| + % (ent.name, self._clash_dict[cname]))
| self._entity_dict[ent.name] = ent
| + self._clash_dict[cname] = ent.name
|
| def lookup_entity(self, name, typ=None):
| ent = self._entity_dict.get(name)Re: [Qemu-devel] [PATCH] fixup! qapi: Forbid case-insensitive clashes
Quoted 17 lines · Markus Armbruster
Quoted 11 lines · Eric Blake
[Replace the old commit message with this:]
qapi: Forbid case-insensitive clashes
However, we DO have to care about the fact that we have a command 'stop' and an event 'STOP' (currently the only case collision of any of our .json entities). To solve that, add a new
QAPISchemaEntity.c_namecase()that returns a munged version of the name that can be used in checking for case clashes,For [-._] clashes, too. Easiest fix it to just say "checking for clashes".
Quoted 6 lines
Quoted 4 lines
and override it in QAPISchemaEvent to use a different
case (thus, indexing by .c_namecase() will create two natural key partitions), then add a new_clash_dicttoQAPISchema._def_entity()that checks for case collisions afterTo QAPISchema, actually. In
_def_entity(), we only add an entry.
Easy fixes.
Quoted 14 lines
Quoted 4 lines
ruling out identical spellings. This works because all entities have at least a leading letter in their name.
Quoted 9 lines
We could go further and enforce that events are named with '
ALL_CAPS' and that nothing else is named in that manner; but that can be done as a followup if desired. We could also separate commands from type names (the way we already separated events into a separate namespace), but then we'd have to make .c_namecase() pick something other than the convenientupper()vs.lower()in order to partition dictionary keys into three sets (perhaps a leading digit, since all entities start with a letter).
Quoted 3 lines
The upper / lower partition works for events / anything else, and is kind of cute, although two separate clash dictionaries might be clearer. I'd like to sketch that next, to help us decide.
Okay, sounds like I should post a v12.5 with the alternative of 2 separate clash dictionaries (3 dictionaries total), and as a complete patch rather than a fixup.
Quoted 11 lines
| @@ -800,6 +800,9 @@ class QAPISchemaEntity(object): | def c_name(self): | return c_name(self.name) | | + def c_namecase(self): | + return c_name(self.name).lower() | + | def check(self, schema): | pass |
Quoted 6 lines
The entities are: commands, events, the various types. This implementation of
c_namecase()covers commands and types. It gets overwritten for events.Commands should not use upper case. If we enforced that, the .lower() would be superfluous there. But it isn't wrong.
I won't need c_namecase() with separate dictionaries. Once we have the two approaches to compare, we can decide which one looks nicer.
Quoted 11 lines
| @@ -1040,7 +1043,7 @@ class QAPISchemaObjectTypeMember(object): | assert self.type | | def check_clash(self, info, seen): | - cname = c_name(self.name) | + cname = c_name(self.name).lower() | if cname in seen: | raise QAPIExprError(info, | "%s collides with %s"
Quoted 4 lines
Not an entity, therefore
QAPISchemaEntity.c_namecase()isn't inherited.Members should not use upper case. If we enforced that, the .lower() would be superfluous.
Even if we enforce it, we have to whitelist exceptions for backward compatibility; those exceptions would have to use .lower(), or we can just skip collision detection on those whitelisted types (where the existence of the whitelist reminds us to maintain no case collisions by hand).
Quoted 10 lines
| @@ -1184,12 +1187,16 @@ class QAPISchemaEvent(QAPISchemaEntity): | def visit(self, visitor): | visitor.visit_event(self.name, self.info, self.arg_type) | | + def c_namecase(self): | + return c_name(self.name).upper() | +Event names should not use lower case. If we enforced that, the .upper() would be superfluous.
Looks like we could enforce this one without a whitelist.
Quoted 7 lines
If we enforced our naming conventions, namely types in CamelCase, events in
ALL_CAPS, everything else lower-case, thec_namecase()would become slightly simpler: it's same asc_name()except for types, where it additionally folds to lower case. The name would be misleading then, however. Anyway, we can consider that once we enforce naming conventions.
And enforcing conventions certainly won't happen any sooner than the rest of the pending queue is flushed.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
In reply to Eric BlakeLet's think through this on a higher level.
I figure the motivation for this patch is twofold:
1. C identifier clash detection
We generate C identifiers derived from QAPI names. These can clash with (1) each other, (2) C keywords and selected other well-known identifiers, and (3) the user's identifiers.
We avoid (2), and we make an effort to catch (1). because that's friendlier than letting the compiler catch it. (3) is the user's problem, and there's nothing we can do about it.
Clashes are due to (a) folding [-._] to _, (b) case folding, and (c) name space intrusion, e.g. a type Foo_lookup clashes with an enum type Foo's lookup table.
We make an effort to prevent (c) by reserving names. We make an effort to catch (a) and (b).
2. Dislike for interfaces that differ only in case
I agree defining such interfaces is a bad idea. However, forbidding bad ideas is problematic, because where do you stop? There's an infinite supply.
Forbidding makes sense when the bad idea is a recurring problem, or when it causes technical difficulties.
There's been one instance of this bad idea so far: command 'stop' and event 'STOP'. We can't fix it, because it's ABI. The patches proposed so far can't even prevent its recurrence; events and non-event entities that differ only in case remain valid. Any other combination becomes invalid.
A rule "cannot differ only in case" is okay, but with "except for events and non-events" tacked on, it becomes ugly.
The case for rejecting names that differ only in case appears weak.
Our naming conventions actually make clashes due to folding relatively unlikely. They are:
* All names consist of letters, digits, '-' and '_', starting with a letter.
* Event names are ALL_CAPS, i.e. they use neither lower case nor '-'.
Without lower case, '-' and '.', clash due to folding is impossible.
* Command names, member names and built-in type names are dashed-lower, i.e. they use neither upper case nor '_'.
Without upper case, '_', and '.', clash due to folding is impossible.
* Type names are CamelCase. They do not use '_' or '-'.
Can theoretically clash in amusing ways: ONegus, OneGus. Ain'tCamelCaseFun!
We can get rid of the clashes by not case folding type names. See "[PATCH RFC 3/5] qapi: Use common name mangling for enumeration constants". The patch additionally drops folding of enumeration member names, which isn't necessary. Controversial.
* Additionally, any name may be prefixed by __RFQDN_. RFQDN consists of letters, digits '-' and '.'.
Because unprefixed names start with a letter, and the prefix starts with '__', prefixed names cannot clash with unprefixed names.
If we additionally stipulated that event prefixes are upper case, and all others lower case, prefixes couldn't contribute to clashes at all.
Strict adherence to our naming conventions would eliminate clashes due to folding except for type names. A single extra dictionary mapping c_name(typ.name).lower() to typ would suffice.
What happens to the rest of your queue if we shelve this patch for now?
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
Quoted 9 lines · Markus Armbruster
Let's think through this on a higher level.
I figure the motivation for this patch is twofold:
1. C identifier clash detection
2. Dislike for interfaces that differ only in case
And the related dislike for interfaces that differ only in '-' or '_'.
Quoted 9 lines
Our naming conventions actually make clashes due to folding relatively unlikely. They are:
* All names consist of letters, digits, '-' and '_', starting with a letter.
* Event names are
ALL_CAPS, i.e. they use neither lower case nor '-'.Without lower case, '-' and '.', clash due to folding is impossible.
So if we enforce that convention, we don't have to worry about case clash, and I don't think we have any outliers to whitelist.
Quoted 5 lines
* Command names, member names and built-in type names are dashed-lower, i.e. they use neither upper case nor '_'.
Without upper case, '_', and '.', clash due to folding is impossible.
If we enforce that convention, we have to whitelist outliers (for example, 'netdev_add' as a command name, or a number of types whose members still use '_'). The existence of a whitelist will discourage future clashes, without any of the hassle of coding up clash checks.
Quoted 5 lines
* Type names are CamelCase. They do not use '_' or '-'.
Can theoretically clash in amusing ways: ONegus, OneGus. Ain'tCamelCaseFun!
Do we have any outliers?
Quoted 5 lines
We can get rid of the clashes by not case folding type names. See "[PATCH RFC 3/5] qapi: Use common name mangling for enumeration constants". The patch additionally drops folding of enumeration member names, which isn't necessary. Controversial.
There is even the possibility of mixed treatment (enumeration name portion as-is, member name portion shouted, as in 'MyTypeVALUE1'). Not sure if we'll get a clear answer to the controversy, but also not sure it is worth holding up other patches while discussing that.
Quoted 9 lines
* Additionally, any name may be prefixed by
__RFQDN_. RFQDN consists of letters, digits '-' and '.'.Because unprefixed names start with a letter, and the prefix starts with '__', prefixed names cannot clash with unprefixed names.
If we additionally stipulated that event prefixes are upper case, and all others lower case, prefixes couldn't contribute to clashes at all.
It's a bit weird that we'd have __org.qemu_command and __ORG.QEMU_EVENT for the same vendor. On the other hand, FQDN is already case-insensitive (qemu.org and QEMU.ORG resolve to the same address). So there won't be clashes between vendors (as no two vendors can share a FQDN that differs in case); beyond that, if a vendor wants to play weird case games, that's their (downstream) problem, not ours.
Quoted 4 lines
Strict adherence to our naming conventions would eliminate clashes due to folding except for type names. A single extra dictionary mapping c_name(typ.name).lower() to typ would suffice.
Certainly may be simpler than carrying three dictionaries for command/event/type.
Quoted 2 lines
What happens to the rest of your queue if we shelve this patch for now?
Not much; in fact, according to the patch version log:
---
v12: add in entity case collisions (required sharing two maps), improve commit message
v11: rebase to latest, don't focus so hard on future case-insensitive extensions, adjust commit message
v10: new patch
---
I only even added it due to conversations on v9, and intentionally kept it separate from 'qapi: Detect collisions in C member names' (we absolutely want to report exact-name collisions, whether or not we also decide to report case collisions). It should not be a problem to defer this patch (or a better version of it that enforces conventions, adds whitelist handling, then only worries about type name case collisions) to much later, if at all.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 27/36] qapi: Forbid case-insensitive clashes
Quoted 5 lines · Eric Blake
Quoted 4 lines · Markus Armbruster
Let's think through this on a higher level.
I figure the motivation for this patch is twofold:
Quoted 7 lines
Quoted 5 lines
1. C identifier clash detection
2. Dislike for interfaces that differ only in case
And the related dislike for interfaces that differ only in '-' or '_'.
Yes.
Quoted 6 lines
Quoted 6 lines
Our naming conventions actually make clashes due to folding relatively unlikely. They are:
* All names consist of letters, digits, '-' and '_', starting with a letter.
Quoted 6 lines
Quoted 3 lines
* Event names are
ALL_CAPS, i.e. they use neither lower case nor '-'.Without lower case, '-' and '.', clash due to folding is impossible.
So if we enforce that convention, we don't have to worry about case clash, and I don't think we have any outliers to whitelist.
qmp-introspect.c shows none.
Quoted 8 lines
Quoted 4 lines
* Command names, member names and built-in type names are dashed-lower, i.e. they use neither upper case nor '_'.
Without upper case, '_', and '.', clash due to folding is impossible.
If we enforce that convention, we have to whitelist outliers (for example, '
netdev_add' as a command name, or a number of types whose members still use '_').
Yes.
Quoted 2 lines
The existence of a whitelist will discourage
future clashes, without any of the hassle of coding up clash checks.
I hope so.
Quoted 6 lines
Quoted 4 lines
* Type names are CamelCase. They do not use '_' or '-'.
Can theoretically clash in amusing ways: ONegus, OneGus. Ain'tCamelCaseFun!
Do we have any outliers?
Let's look. Stick
print '%s %s' % (entity.__class__.__name__, name)
into QAPISchema.visit()'s loop.
Pipe output through "sed -n 's/^.*Type \([^:].*\)$/\1/p'" to get the type names. Grep for '[-_]', -v '[a-z]' and -v '[A-Z]' shows there are no names with [-_], no names without lower case letters, and the only names without upper case letters are the 14 built-in types. Both for qapi-schema.json and qga/qapi-schema.json.
Quoted 9 lines
Quoted 4 lines
We can get rid of the clashes by not case folding type names. See "[PATCH RFC 3/5] qapi: Use common name mangling for enumeration constants". The patch additionally drops folding of enumeration member names, which isn't necessary. Controversial.
There is even the possibility of mixed treatment (enumeration name portion as-is, member name portion shouted, as in 'MyTypeVALUE1'). Not sure if we'll get a clear answer to the controversy, but also not sure it is worth holding up other patches while discussing that.
I'm content to shout the member name. I'd stick a '_' between the prefix and the member name, though.
Quoted 11 lines
Quoted 8 lines
* Additionally, any name may be prefixed by
__RFQDN_. RFQDN consists of letters, digits '-' and '.'.Because unprefixed names start with a letter, and the prefix starts with '__', prefixed names cannot clash with unprefixed names.
If we additionally stipulated that event prefixes are upper case, and all others lower case, prefixes couldn't contribute to clashes at all.
It's a bit weird that we'd have __org.qemu_command and __ORG.QEMU_EVENT for the same vendor.
Yes.
Quoted 2 lines
On the other hand, FQDN is already
case-insensitive (qemu.org and QEMU.ORG resolve to the same address).
Yes, and that makes stupilating case now somewhat problematic.
Quoted 3 lines
So there won't be clashes between vendors (as no two vendors can share a FQDN that differs in case); beyond that, if a vendor wants to play weird case games, that's their (downstream) problem, not ours.
Yes, not our worry.
Quoted 20 lines
Quoted 3 lines
Strict adherence to our naming conventions would eliminate clashes due to folding except for type names. A single extra dictionary mapping c_name(typ.name).lower() to typ would suffice.
Certainly may be simpler than carrying three dictionaries for command/event/type.
Quoted 2 lines
What happens to the rest of your queue if we shelve this patch for now?
Not much; in fact, according to the patch version log:
---
v12: add in entity case collisions (required sharing two maps), improve commit message
v11: rebase to latest, don't focus so hard on future case-insensitive extensions, adjust commit message
v10: new patch
---
Quoted 7 lines
I only even added it due to conversations on v9, and intentionally kept it separate from 'qapi: Detect collisions in C member names' (we absolutely want to report exact-name collisions, whether or not we also decide to report case collisions). It should not be a problem to defer this patch (or a better version of it that enforces conventions, adds whitelist handling, then only worries about type name case collisions) to much later, if at all.
Okay, let's shelve the patch for now, keep the queue moving, and revisit the patch at a more opportune time.
I hope the detour didn't eat too much of your time. We learned a few things, and perhaps we can still reuse some of the work.
PATCH v12 28/36 qapi: Simplify QObject
In reply to Eric BlakeThe QObject hierarchy is small enough, and unlikely to grow further (since we only use it to map to JSON and already cover all JSON types), that we can simplify things by not tracking a separate vtable, but just inline all two elements of the vtable QType directly into QObject. We can drop qnull_destroy_obj() in the process.
This also has the nice benefit of moving the typename 'QType' out of the way, so that the next patch can repurpose it for a nicer name for 'qtype_code'.
The various objects are now 8 bytes larger on 64-bit platforms, but hopefully this is in the noise. If size is absolutely critical due to cache line speed effects, we could do some bit packing to restore the previous size (qtype_code currently only requires three distinct bits, so we could assert an assumption that architecture ABIs will give function pointers at least an 8-byte alignment, and that the three low-order bits of destroy are always zero; or we could use a bitfield for refcnt, with a maximum of SIZE_MAX/8, and given that a QObject already occupies at least 8 bytes, that won't artificially limit us). Or we could expose the currently static destroy methods and instead create a table in qobject.h that maps QType to destructor. But let's save any size compression for a followup patch, if at all.
On the other hand, by avoiding a separate vtable, there is less indirection, so use of QObjects may be slightly more efficient.
However, as we don't have evidence pointing at either QObject size or access speed as being a hot spot, I didn't think it was worth benchmarking.
---
v12: new patch, split off from 21/28
--- include/qapi/qmp/qobject.h | 38 +++++++++++++++++++++----------------- qobject/qbool.c | 7 +------ qobject/qdict.c | 7 +------ qobject/qfloat.c | 7 +------ qobject/qint.c | 7 +------ qobject/qlist.c | 7 +------ qobject/qnull.c | 13 ++----------- qobject/qstring.c | 7 +------ 8 files changed, 29 insertions(+), 64 deletions(-)
include/qapi/qmp/qobject.h +21 -17
@@ -47,21 +47,21 @@ typedef enum {QTYPE_MAX , } qtype_code; -struct QObject ; +typedef struct QObject QObject ; -typedef struct QType { -qtype_code code; -void (*destroy)(struct QObject *); -}QType ; - -typedef struct QObject { -const QType *type; +struct QObject { +qtype_code type;size_t refcnt; -}QObject ; +void (*destroy)(QObject *); +};/* Get the 'base' part of an object */ #define QOBJECT (obj) (&(obj)->base) +/* High-level interface for qobject_init() */ +#define QOBJECT_INIT (obj, type, destroy) \ + qobject_init(QOBJECT(obj), type, destroy) +/* High-level interface for qobject_incref() */ #define QINCREF (obj) \ qobject_incref(QOBJECT(obj))
@@ -71,9 +71,14 @@ typedef struct QObject {qobject_decref (obj ?QOBJECT (obj) :NULL ) /* Initialize an object to default values */ -#define QOBJECT_INIT (obj, qtype_type) \ - obj->base.refcnt = 1; \ - obj->base.type = qtype_type +static inline void qobject_init (QObject *obj,qtype_code type, +void (*destroy)(struct QObject *)) +{ +assert (type); + obj->refcnt =1 ; + obj->type = type; + obj->destroy = destroy; +}/** * qobject_incref(): Increment QObject's reference count
@@ -92,9 +97,8 @@ static inline void qobject_decref(QObject *obj) {assert (!obj || obj->refcnt);if (obj && --obj->refcnt ==0 ) { -assert (obj->type !=NULL ); -assert (obj->type->destroy !=NULL ); - obj->type->destroy (obj); +assert (obj->destroy); + obj->destroy (obj); } }
@@ -103,8 +107,8 @@ static inline void qobject_decref(QObject *obj)*/ static inline qtype_code qobject_type (const QObject *obj) { -assert (obj->type !=NULL ); -return obj->type->code; +assert (obj->type); +return obj->type; }extern QObject qnull_;
qobject/qbool.c +1 -6
@@ -17,11 +17,6 @@static void qbool_destroy_obj (QObject *obj); -static const QType qbool_type = { - .code =QTYPE_QBOOL , - .destroy = qbool_destroy_obj, -}; -/** * qbool_from_bool(): Create a new QBool from a bool *
@@ -33,7 +28,7 @@ QBool *qbool_from_bool(bool value) qb =g_malloc (sizeof (*qb)); qb->value = value; -QOBJECT_INIT (qb, &qbool_type); +QOBJECT_INIT (qb,QTYPE_QBOOL , qbool_destroy_obj);return qb; }
qobject/qdict.c +1 -6
@@ -21,11 +21,6 @@static void qdict_destroy_obj (QObject *obj); -static const QType qdict_type = { - .code =QTYPE_QDICT , - .destroy = qdict_destroy_obj, -}; -/** * qdict_new(): Create a new QDict *
@@ -36,7 +31,7 @@ QDict *qdict_new(void)QDict *qdict; qdict =g_malloc0 (sizeof (*qdict)); -QOBJECT_INIT (qdict, &qdict_type); +QOBJECT_INIT (qdict,QTYPE_QDICT , qdict_destroy_obj);return qdict; }
qobject/qfloat.c +1 -6
@@ -17,11 +17,6 @@static void qfloat_destroy_obj (QObject *obj); -static const QType qfloat_type = { - .code =QTYPE_QFLOAT , - .destroy = qfloat_destroy_obj, -}; -/** * qfloat_from_int(): Create a new QFloat from a float *
@@ -33,7 +28,7 @@ QFloat *qfloat_from_double(double value) qf =g_malloc (sizeof (*qf)); qf->value = value; -QOBJECT_INIT (qf, &qfloat_type); +QOBJECT_INIT (qf,QTYPE_QFLOAT , qfloat_destroy_obj);return qf; }
qobject/qint.c +1 -6
@@ -16,11 +16,6 @@static void qint_destroy_obj (QObject *obj); -static const QType qint_type = { - .code =QTYPE_QINT , - .destroy = qint_destroy_obj, -}; -/** * qint_from_int(): Create a new QInt from an int64_t *
@@ -32,7 +27,7 @@ QInt *qint_from_int(int64_t value) qi =g_malloc (sizeof (*qi)); qi->value = value; -QOBJECT_INIT (qi, &qint_type); +QOBJECT_INIT (qi,QTYPE_QINT , qint_destroy_obj);return qi; }
qobject/qlist.c +1 -6
@@ -17,11 +17,6 @@static void qlist_destroy_obj (QObject *obj); -static const QType qlist_type = { - .code =QTYPE_QLIST , - .destroy = qlist_destroy_obj, -}; -/** * qlist_new(): Create a new QList *
@@ -33,7 +28,7 @@ QList *qlist_new(void) qlist =g_malloc (sizeof (*qlist));QTAILQ_INIT (&qlist->head); -QOBJECT_INIT (qlist, &qlist_type); +QOBJECT_INIT (qlist,QTYPE_QLIST , qlist_destroy_obj);return qlist; }
qobject/qnull.c +2 -11
@@ -13,17 +13,8 @@#include "qemu-common.h" #include "qapi/qmp/qobject.h" -static void qnull_destroy_obj (QObject *obj) -{ -assert (0 ); -} - -static const QType qnull_type = { - .code =QTYPE_QNULL , - .destroy = qnull_destroy_obj, -}; -QObject qnull_ = { - .type = &qnull_type, + .type =QTYPE_QNULL , .refcnt =1 , + .destroy =NULL , };
qobject/qstring.c +1 -6
@@ -16,11 +16,6 @@static void qstring_destroy_obj (QObject *obj); -static const QType qstring_type = { - .code =QTYPE_QSTRING , - .destroy = qstring_destroy_obj, -}; -/** * qstring_new(): Create a new empty QString *
@@ -57,7 +52,7 @@ QString *qstring_from_substr(const char *str, int start, int end)memcpy (qstring->string, str + start, qstring->length); qstring->string[qstring->length] =0 ; -QOBJECT_INIT (qstring, &qstring_type); +QOBJECT_INIT (qstring,QTYPE_QSTRING , qstring_destroy_obj);return qstring; }
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 28/36] qapi: Simplify QObject
Quoted 11 lines · Eric Blake
The QObject hierarchy is small enough, and unlikely to grow further (since we only use it to map to JSON and already cover all JSON types), that we can simplify things by not tracking a separate vtable, but just inline all two elements of the vtable QType directly into QObject. We can drop
qnull_destroy_obj()in the process.This also has the nice benefit of moving the typename 'QType' out of the way, so that the next patch can repurpose it for a nicer name for '
qtype_code'.
Quoted 12 lines
The various objects are now 8 bytes larger on 64-bit platforms, but hopefully this is in the noise. If size is absolutely critical due to cache line speed effects, we could do some bit packing to restore the previous size (
qtype_codecurrently only requires three distinct bits, so we could assert an assumption that architecture ABIs will give function pointers at least an 8-byte alignment, and that the three low-order bits of destroy are always zero; or we could use a bitfield for refcnt, with a maximum of SIZE_MAX/8, and given that a QObject already occupies at least 8 bytes, that won't artificially limit us). Or we could expose the currently static destroy methods and instead create a table inqobject.hthat maps QType to destructor.
That's what I'd do.
A type registration function would keep the destructors static. But why bother.
Quoted 2 lines
But let's save any size compression for a
followup patch, if at all.
Okay.
Quoted 9 lines
On the other hand, by avoiding a separate vtable, there is less indirection, so use of QObjects may be slightly more efficient.
However, as we don't have evidence pointing at either QObject size or access speed as being a hot spot, I didn't think it was worth benchmarking.
Suggested-by Markus Armbruster <[email protected]>Signed-off-by Eric Blake <[email protected]>
Works for me.
PATCH 1/1 fixup? qapi: Simplify QObject
In reply to Eric Blake[If we squash, replace the old commit message with this; you'll also have some merge conflicts in 29/36 and 30/36. Note that while this mail is net lines added, squashing would still result in a net reduction:
16 files changed, 67 insertions(+), 83 deletions(-)
]
qapi: Simplify QObject
The QObject hierarchy is small enough, and unlikely to grow further (since we only use it to map to JSON and already cover all JSON types), that we can simplify things by not tracking a separate vtable, but just inline the refcnt element of the vtable QType directly into QObject, and track a separate array of destroy functions. We can drop qnull_destroy_obj() in the process.
The remaining QObject subclasses must export their destructor.
This also has the nice benefit of moving the typename 'QType' out of the way, so that the next patch can repurpose it for a nicer name for 'qtype_code'.
The various objects are still the same size (so no change in cache line pressure), but now have less indirection (although I didn't bother benchmarking to see if there is a noticeable speedup, as we don't have hard evidence that this was in a performance hotspot in the first place).
---
If we don't want to squash, I can instead rebase this to appear after 30/36 as a standalone patch, with a message something like:
qapi: Don't duplicate destructor pointer per QObject
The previous patch "qapi: Simplify QObject" made each QObject instance larger; restore the original size for less cache line pressure by moving the destroy function pointer out of each object and instead maintaining a global table of destroy functions. This requires exporting the various functions, and is made easier to read through a typedef.
--- include/qapi/qmp/qbool.h | 1 + include/qapi/qmp/qdict.h | 1 + include/qapi/qmp/qfloat.h | 1 + include/qapi/qmp/qint.h | 1 + include/qapi/qmp/qlist.h | 1 + include/qapi/qmp/qobject.h | 16 ++++++++-------- include/qapi/qmp/qstring.h | 1 + qobject/Makefile.objs | 2 +- qobject/qbool.c | 6 ++---- qobject/qdict.c | 6 ++---- qobject/qfloat.c | 6 ++---- qobject/qint.c | 6 ++---- qobject/qlist.c | 6 ++---- qobject/qnull.c | 1 - qobject/qobject.c | 26 ++++++++++++++++++++++++++ qobject/qstring.c | 6 ++---- 16 files changed, 53 insertions(+), 34 deletions(-) create mode 100644 qobject/qobject.c
include/qapi/qmp/qbool.h +1 -0
@@ -25,5 +25,6 @@ typedef struct QBool {QBool *qbool_from_bool (bool value);bool qbool_get_bool (const QBool *qb);QBool *qobject_to_qbool (const QObject *obj); +void qbool_destroy_obj (QObject *obj);#endif /* QBOOL_H */
include/qapi/qmp/qdict.h +1 -0
@@ -48,6 +48,7 @@ void qdict_iter(const QDict *qdict, void *opaque);const QDictEntry *qdict_first (const QDict *qdict);const QDictEntry *qdict_next (const QDict *qdict,const QDictEntry *entry); +void qdict_destroy_obj (QObject *obj);/* Helper to qdict_put_obj(), accepts any object */ #define qdict_put (qdict, key, obj) \
include/qapi/qmp/qfloat.h +1 -0
@@ -25,5 +25,6 @@ typedef struct QFloat {QFloat *qfloat_from_double (double value);double qfloat_get_double (const QFloat *qi);QFloat *qobject_to_qfloat (const QObject *obj); +void qfloat_destroy_obj (QObject *obj);#endif /* QFLOAT_H */
include/qapi/qmp/qint.h +1 -0
@@ -24,5 +24,6 @@ typedef struct QInt {QInt *qint_from_int (int64_t value);int64_t qint_get_int (const QInt *qi);QInt *qobject_to_qint (const QObject *obj); +void qint_destroy_obj (QObject *obj);#endif /* QINT_H */
include/qapi/qmp/qlist.h +1 -0
@@ -49,6 +49,7 @@ QObject *qlist_peek(QList *qlist);int qlist_empty (const QList *qlist);size_t qlist_size (const QList *qlist);QList *qobject_to_qlist (const QObject *obj); +void qlist_destroy_obj (QObject *obj);static inline const QListEntry *qlist_first (const QList *qlist) {
include/qapi/qmp/qobject.h +8 -8
@@ -52,15 +52,17 @@ typedef struct QObject QObject;struct QObject {qtype_code type;size_t refcnt; -void (*destroy)(QObject *); }; +typedef void (*QDestroy )(QObject *); +extern QDestroy qdestroy[QTYPE_MAX ]; +/* Get the 'base' part of an object */ #define QOBJECT (obj) (&(obj)->base)/* High-level interface for qobject_init() */ -#define QOBJECT_INIT (obj, type, destroy) \ - qobject_init(QOBJECT(obj), type, destroy) +#define QOBJECT_INIT (obj, type) \ + qobject_init(QOBJECT(obj), type)/* High-level interface for qobject_incref() */ #define QINCREF (obj) \
@@ -71,13 +73,11 @@ struct QObject {qobject_decref (obj ?QOBJECT (obj) :NULL ) /* Initialize an object to default values */ -static inline void qobject_init (QObject *obj,qtype_code type, -void (*destroy)(struct QObject *)) +static inline void qobject_init (QObject *obj,qtype_code type) {assert (type); obj->refcnt =1 ; obj->type = type; - obj->destroy = destroy; }/**
@@ -97,8 +97,8 @@ static inline void qobject_decref(QObject *obj) {assert (!obj || obj->refcnt);if (obj && --obj->refcnt ==0 ) { -assert (obj->destroy); - obj->destroy (obj); +assert (qdestroy[obj->type]); + qdestroy[obj->type](obj); } }
include/qapi/qmp/qstring.h +1 -0
@@ -32,5 +32,6 @@ void qstring_append_int(QString *qstring, int64_t value);void qstring_append (QString *qstring,const char *str);void qstring_append_chr (QString *qstring,int c);QString *qobject_to_qstring (const QObject *obj); +void qstring_destroy_obj (QObject *obj);#endif /* QSTRING_H */
qobject/Makefile.objs +1 -1
@@ -1,2 +1,2 @@ util-obj-y = qnull.o qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o -util-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o +util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o
qobject/qbool.c +2 -4
@@ -15,8 +15,6 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -static void qbool_destroy_obj (QObject *obj); -/** * qbool_from_bool(): Create a new QBool from a bool *
@@ -28,7 +26,7 @@ QBool *qbool_from_bool(bool value) qb =g_malloc (sizeof (*qb)); qb->value = value; -QOBJECT_INIT (qb,QTYPE_QBOOL , qbool_destroy_obj); +QOBJECT_INIT (qb,QTYPE_QBOOL );return qb; }
@@ -56,7 +54,7 @@ QBool *qobject_to_qbool(const QObject *obj)* qbool_destroy_obj(): Free all memory allocated by a * QBool object */ -static void qbool_destroy_obj (QObject *obj) +void qbool_destroy_obj (QObject *obj) {assert (obj !=NULL );g_free (qobject_to_qbool (obj));
qobject/qdict.c +2 -4
@@ -19,8 +19,6 @@#include "qemu/queue.h" #include "qemu-common.h" -static void qdict_destroy_obj (QObject *obj); -/** * qdict_new(): Create a new QDict *
@@ -31,7 +29,7 @@ QDict *qdict_new(void)QDict *qdict; qdict =g_malloc0 (sizeof (*qdict)); -QOBJECT_INIT (qdict,QTYPE_QDICT , qdict_destroy_obj); +QOBJECT_INIT (qdict,QTYPE_QDICT );return qdict; }
@@ -436,7 +434,7 @@ void qdict_del(QDict *qdict, const char *key)/** * qdict_destroy_obj(): Free all the memory allocated by a QDict */ -static void qdict_destroy_obj (QObject *obj) +void qdict_destroy_obj (QObject *obj) {int i;QDict *qdict;
qobject/qfloat.c +2 -4
@@ -15,8 +15,6 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -static void qfloat_destroy_obj (QObject *obj); -/** * qfloat_from_int(): Create a new QFloat from a float *
@@ -28,7 +26,7 @@ QFloat *qfloat_from_double(double value) qf =g_malloc (sizeof (*qf)); qf->value = value; -QOBJECT_INIT (qf,QTYPE_QFLOAT , qfloat_destroy_obj); +QOBJECT_INIT (qf,QTYPE_QFLOAT );return qf; }
@@ -56,7 +54,7 @@ QFloat *qobject_to_qfloat(const QObject *obj)* qfloat_destroy_obj(): Free all memory allocated by a * QFloat object */ -static void qfloat_destroy_obj (QObject *obj) +void qfloat_destroy_obj (QObject *obj) {assert (obj !=NULL );g_free (qobject_to_qfloat (obj));
qobject/qint.c +2 -4
@@ -14,8 +14,6 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -static void qint_destroy_obj (QObject *obj); -/** * qint_from_int(): Create a new QInt from an int64_t *
@@ -27,7 +25,7 @@ QInt *qint_from_int(int64_t value) qi =g_malloc (sizeof (*qi)); qi->value = value; -QOBJECT_INIT (qi,QTYPE_QINT , qint_destroy_obj); +QOBJECT_INIT (qi,QTYPE_QINT );return qi; }
@@ -55,7 +53,7 @@ QInt *qobject_to_qint(const QObject *obj)* qint_destroy_obj(): Free all memory allocated by a * QInt object */ -static void qint_destroy_obj (QObject *obj) +void qint_destroy_obj (QObject *obj) {assert (obj !=NULL );g_free (qobject_to_qint (obj));
qobject/qlist.c +2 -4
@@ -15,8 +15,6 @@#include "qemu/queue.h" #include "qemu-common.h" -static void qlist_destroy_obj (QObject *obj); -/** * qlist_new(): Create a new QList *
@@ -28,7 +26,7 @@ QList *qlist_new(void) qlist =g_malloc (sizeof (*qlist));QTAILQ_INIT (&qlist->head); -QOBJECT_INIT (qlist,QTYPE_QLIST , qlist_destroy_obj); +QOBJECT_INIT (qlist,QTYPE_QLIST );return qlist; }
@@ -146,7 +144,7 @@ QList *qobject_to_qlist(const QObject *obj)/** * qlist_destroy_obj(): Free all the memory allocated by a QList */ -static void qlist_destroy_obj (QObject *obj) +void qlist_destroy_obj (QObject *obj) {QList *qlist;QListEntry *entry, *next_entry;
qobject/qnull.c +0 -1
@@ -16,5 +16,4 @@QObject qnull_ = { .type =QTYPE_QNULL , .refcnt =1 , - .destroy =NULL , };
qobject/qobject.c new file +26 -0
@@ -0,0 +1,26 @@ +/* +* QObject +* +* Copyright (C) 2015 Red Hat, Inc. +* +* This work is licensed under the terms of the GNU LGPL, version 2.1 +* or later. See the COPYING.LIB file in the top-level directory. +*/ + +#include "qemu-common.h" +#include "qapi/qmp/qbool.h" +#include "qapi/qmp/qdict.h" +#include "qapi/qmp/qfloat.h" +#include "qapi/qmp/qint.h" +#include "qapi/qmp/qlist.h" +#include "qapi/qmp/qstring.h" + +QDestroy qdestroy[QTYPE_MAX ] = { + [QTYPE_QBOOL ] = qbool_destroy_obj, + [QTYPE_QDICT ] = qdict_destroy_obj, + [QTYPE_QFLOAT ] = qfloat_destroy_obj, + [QTYPE_QINT ] = qint_destroy_obj, + [QTYPE_QLIST ] = qlist_destroy_obj, + [QTYPE_QSTRING ] = qstring_destroy_obj, +/* [QTYPE_QNULL] = NULL, */ +};
qobject/qstring.c +2 -4
@@ -14,8 +14,6 @@#include "qapi/qmp/qstring.h" #include "qemu-common.h" -static void qstring_destroy_obj (QObject *obj); -/** * qstring_new(): Create a new empty QString *
@@ -52,7 +50,7 @@ QString *qstring_from_substr(const char *str, int start, int end)memcpy (qstring->string, str + start, qstring->length); qstring->string[qstring->length] =0 ; -QOBJECT_INIT (qstring,QTYPE_QSTRING , qstring_destroy_obj); +QOBJECT_INIT (qstring,QTYPE_QSTRING );return qstring; }
@@ -133,7 +131,7 @@ const char *qstring_get_str(const QString *qstring)* qstring_destroy_obj(): Free all memory allocated by a QString * object */ -static void qstring_destroy_obj (QObject *obj) +void qstring_destroy_obj (QObject *obj) {QString *qs;
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH] fixup? qapi: Simplify QObject
Quoted 162 lines · Eric Blake
[If we squash, replace the old commit message with this; you'll also have some merge conflicts in 29/36 and 30/36. Note that while this mail is net lines added, squashing would still result in a net reduction:
16 files changed, 67 insertions(+), 83 deletions(-)]
qapi: Simplify QObject
The QObject hierarchy is small enough, and unlikely to grow further (since we only use it to map to JSON and already cover all JSON types), that we can simplify things by not tracking a separate vtable, but just inline the refcnt element of the vtable QType directly into QObject, and track a separate array of destroy functions. We can drop
qnull_destroy_obj()in the process.The remaining QObject subclasses must export their destructor.
This also has the nice benefit of moving the typename 'QType' out of the way, so that the next patch can repurpose it for a nicer name for '
qtype_code'.The various objects are still the same size (so no change in cache line pressure), but now have less indirection (although I didn't bother benchmarking to see if there is a noticeable speedup, as we don't have hard evidence that this was in a performance hotspot in the first place).
Suggested-by Markus Armbruster <[email protected]>Signed-off-by Eric Blake <[email protected]>---
If we don't want to squash, I can instead rebase this to appear after 30/36 as a standalone patch, with a message something like:
qapi: Don't duplicate destructor pointer per QObject
The previous patch "qapi: Simplify QObject" made each QObject instance larger; restore the original size for less cache line pressure by moving the destroy function pointer out of each object and instead maintaining a global table of destroy functions. This requires exporting the various functions, and is made easier to read through a typedef.
--- include/qapi/qmp/qbool.h | 1 + include/qapi/qmp/qdict.h | 1 + include/qapi/qmp/qfloat.h | 1 + include/qapi/qmp/qint.h | 1 + include/qapi/qmp/qlist.h | 1 + include/qapi/qmp/qobject.h | 16 ++++++++-------- include/qapi/qmp/qstring.h | 1 + qobject/Makefile.objs | 2 +- qobject/qbool.c | 6 ++---- qobject/qdict.c | 6 ++---- qobject/qfloat.c | 6 ++---- qobject/qint.c | 6 ++---- qobject/qlist.c | 6 ++---- qobject/qnull.c | 1 - qobject/qobject.c | 26 ++++++++++++++++++++++++++ qobject/qstring.c | 6 ++---- 16 files changed, 53 insertions(+), 34 deletions(-) create mode 100644 qobject/qobject.cinclude/qapi/qmp/qbool.h +1
-0@@ -25,5 +25,6 @@ typedef struct QBool {QBool *qbool_from_bool (bool value);bool qbool_get_bool (const QBool *qb);QBool *qobject_to_qbool (const QObject *obj); +void qbool_destroy_obj (QObject *obj);#endif /* QBOOL_H */ include/qapi/qmp/qdict.h +1
-0@@ -48,6 +48,7 @@ void qdict_iter(const QDict *qdict, void *opaque);const QDictEntry *qdict_first (const QDict *qdict);const QDictEntry *qdict_next (const QDict *qdict,const QDictEntry *entry); +void qdict_destroy_obj (QObject *obj);/* Helper to qdict_put_obj(), accepts any object */ #define qdict_put (qdict, key, obj) \include/qapi/qmp/qfloat.h +1
-0@@ -25,5 +25,6 @@ typedef struct QFloat {QFloat *qfloat_from_double (double value);double qfloat_get_double (const QFloat *qi);QFloat *qobject_to_qfloat (const QObject *obj); +void qfloat_destroy_obj (QObject *obj);#endif /* QFLOAT_H */ include/qapi/qmp/qint.h +1
-0@@ -24,5 +24,6 @@ typedef struct QInt {QInt *qint_from_int (int64_t value);int64_t qint_get_int (const QInt *qi);QInt *qobject_to_qint (const QObject *obj); +void qint_destroy_obj (QObject *obj);#endif /* QINT_H */ include/qapi/qmp/qlist.h +1
-0@@ -49,6 +49,7 @@ QObject *qlist_peek(QList *qlist);int qlist_empty (const QList *qlist);size_t qlist_size (const QList *qlist);QList *qobject_to_qlist (const QObject *obj); +void qlist_destroy_obj (QObject *obj);static inline const QListEntry *qlist_first (const QList *qlist) {include/qapi/qmp/qobject.h +6
-6@@ -52,15 +52,17 @@ typedef struct QObject QObject;struct QObject {qtype_code type;size_t refcnt; -void (*destroy)(QObject *); }; +typedef void (*QDestroy )(QObject *); +extern QDestroy qdestroy[QTYPE_MAX ]; +/* Get the 'base' part of an object */ #define QOBJECT (obj) (&(obj)->base)/* High-level interface for qobject_init() */ -#define QOBJECT_INIT (obj, type, destroy) \ - qobject_init(QOBJECT(obj), type, destroy) +#define QOBJECT_INIT (obj, type) \ + qobject_init(QOBJECT(obj), type)/* High-level interface for qobject_incref() */ #define QINCREF (obj) \@@ -71,13 +73,11 @@ struct QObject {qobject_decref (obj ?QOBJECT (obj) :NULL ) /* Initialize an object to default values */ -static inline void qobject_init (QObject *obj,qtype_code type, -void (*destroy)(struct QObject *)) +static inline void qobject_init (QObject *obj,qtype_code type) {assert (type); obj->refcnt =1 ; obj->type = type; - obj->destroy = destroy; }/**
include/qapi/qmp/qobject.h static inline void qobject_decref(QObject *obj) · line 104
Quoted 10 lines
include/qapi/qmp/qobject.h +2
-2@@ -97,8 +97,8 @@ static inline void qobject_decref(QObject *obj) {assert (!obj || obj->refcnt);if (obj && --obj->refcnt ==0 ) { -assert (obj->destroy); - obj->destroy (obj); +assert (qdestroy[obj->type]); + qdestroy[obj->type](obj); } }
Preexisting: the assertion is of little value, because all it does is convert one kind of crash into another.
Asserting obj->type is in range might be more useful, i.e.
- assert(obj->destroy); - obj->destroy(obj); + assert(QTYPE_QNULL < obj->type && obj->type < QTYPE_MAX); + qdestroy[obj->type](obj);
We could outline the conditional part, and keep qdestroy[] static. Could save a bit of code (unless NDEBUG, but we shouldn't optimize for that).
Quoted 206 lines
include/qapi/qmp/qstring.h +1
-0@@ -32,5 +32,6 @@ void qstring_append_int(QString *qstring, int64_t value);void qstring_append (QString *qstring,const char *str);void qstring_append_chr (QString *qstring,int c);QString *qobject_to_qstring (const QObject *obj); +void qstring_destroy_obj (QObject *obj);#endif /* QSTRING_H */ qobject/Makefile.objs +1
-1@@ -1,2 +1,2 @@ util-obj-y = qnull.o qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o -util-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o +util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.oqobject/qbool.c +2
-4@@ -15,8 +15,6 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -static void qbool_destroy_obj (QObject *obj); -/** * qbool_from_bool(): Create a new QBool from a bool * @@ -28,7 +26,7 @@ QBool *qbool_from_bool(bool value) qb =g_malloc (sizeof (*qb)); qb->value = value; -QOBJECT_INIT (qb,QTYPE_QBOOL , qbool_destroy_obj); +QOBJECT_INIT (qb,QTYPE_QBOOL );return qb; }@@ -56,7 +54,7 @@ QBool *qobject_to_qbool(const QObject *obj)* qbool_destroy_obj(): Free all memory allocated by a * QBool object */ -static void qbool_destroy_obj (QObject *obj) +void qbool_destroy_obj (QObject *obj) {assert (obj !=NULL );g_free (qobject_to_qbool (obj));qobject/qdict.c +2
-4@@ -19,8 +19,6 @@#include "qemu/queue.h" #include "qemu-common.h" -static void qdict_destroy_obj (QObject *obj); -/** * qdict_new(): Create a new QDict * @@ -31,7 +29,7 @@ QDict *qdict_new(void)QDict *qdict; qdict =g_malloc0 (sizeof (*qdict)); -QOBJECT_INIT (qdict,QTYPE_QDICT , qdict_destroy_obj); +QOBJECT_INIT (qdict,QTYPE_QDICT );return qdict; }@@ -436,7 +434,7 @@ void qdict_del(QDict *qdict, const char *key)/** * qdict_destroy_obj(): Free all the memory allocated by a QDict */ -static void qdict_destroy_obj (QObject *obj) +void qdict_destroy_obj (QObject *obj) {int i;QDict *qdict;qobject/qfloat.c +2
-4@@ -15,8 +15,6 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -static void qfloat_destroy_obj (QObject *obj); -/** * qfloat_from_int(): Create a new QFloat from a float * @@ -28,7 +26,7 @@ QFloat *qfloat_from_double(double value) qf =g_malloc (sizeof (*qf)); qf->value = value; -QOBJECT_INIT (qf,QTYPE_QFLOAT , qfloat_destroy_obj); +QOBJECT_INIT (qf,QTYPE_QFLOAT );return qf; }@@ -56,7 +54,7 @@ QFloat *qobject_to_qfloat(const QObject *obj)* qfloat_destroy_obj(): Free all memory allocated by a * QFloat object */ -static void qfloat_destroy_obj (QObject *obj) +void qfloat_destroy_obj (QObject *obj) {assert (obj !=NULL );g_free (qobject_to_qfloat (obj));qobject/qint.c +2
-4@@ -14,8 +14,6 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -static void qint_destroy_obj (QObject *obj); -/** * qint_from_int(): Create a new QInt from an int64_t * @@ -27,7 +25,7 @@ QInt *qint_from_int(int64_t value) qi =g_malloc (sizeof (*qi)); qi->value = value; -QOBJECT_INIT (qi,QTYPE_QINT , qint_destroy_obj); +QOBJECT_INIT (qi,QTYPE_QINT );return qi; }@@ -55,7 +53,7 @@ QInt *qobject_to_qint(const QObject *obj)* qint_destroy_obj(): Free all memory allocated by a * QInt object */ -static void qint_destroy_obj (QObject *obj) +void qint_destroy_obj (QObject *obj) {assert (obj !=NULL );g_free (qobject_to_qint (obj));qobject/qlist.c +2
-4@@ -15,8 +15,6 @@#include "qemu/queue.h" #include "qemu-common.h" -static void qlist_destroy_obj (QObject *obj); -/** * qlist_new(): Create a new QList * @@ -28,7 +26,7 @@ QList *qlist_new(void) qlist =g_malloc (sizeof (*qlist));QTAILQ_INIT (&qlist->head); -QOBJECT_INIT (qlist,QTYPE_QLIST , qlist_destroy_obj); +QOBJECT_INIT (qlist,QTYPE_QLIST );return qlist; }@@ -146,7 +144,7 @@ QList *qobject_to_qlist(const QObject *obj)/** * qlist_destroy_obj(): Free all the memory allocated by a QList */ -static void qlist_destroy_obj (QObject *obj) +void qlist_destroy_obj (QObject *obj) {QList *qlist;QListEntry *entry, *next_entry;qobject/qnull.c +0
-1@@ -16,5 +16,4 @@QObject qnull_ = { .type =QTYPE_QNULL , .refcnt =1 , - .destroy =NULL , };qobject/qobject.c new file +16
-0@@ -0,0 +1,26 @@ +/* +* QObject +* +* Copyright (C) 2015 Red Hat, Inc. +* +* This work is licensed under the terms of the GNU LGPL, version 2.1 +* or later. See the COPYING.LIB file in the top-level directory. +*/ + +#include "qemu-common.h" +#include "qapi/qmp/qbool.h" +#include "qapi/qmp/qdict.h" +#include "qapi/qmp/qfloat.h" +#include "qapi/qmp/qint.h" +#include "qapi/qmp/qlist.h" +#include "qapi/qmp/qstring.h"
qobject/qobject.c line 26
Quoted 10 lines
qobject/qobject.c new file +10
-0+ +QDestroy qdestroy[QTYPE_MAX ] = { + [QTYPE_QBOOL ] = qbool_destroy_obj, + [QTYPE_QDICT ] = qdict_destroy_obj, + [QTYPE_QFLOAT ] = qfloat_destroy_obj, + [QTYPE_QINT ] = qint_destroy_obj, + [QTYPE_QLIST ] = qlist_destroy_obj, + [QTYPE_QSTRING ] = qstring_destroy_obj, +/* [QTYPE_QNULL] = NULL, */ +};
Suggest
QDestroy qdestroy[QTYPE_MAX] = {
[QTYPE_QNULL] = NULL, /* no such object exists */
[QTYPE_QNULL] = NULL, /* qnull_ is indestructible */
...
};Quoted 23 lines
qobject/qstring.c +1
-3@@ -14,8 +14,6 @@#include "qapi/qmp/qstring.h" #include "qemu-common.h" -static void qstring_destroy_obj (QObject *obj); -/** * qstring_new(): Create a new empty QString * @@ -52,7 +50,7 @@ QString *qstring_from_substr(const char *str, int start, int end)memcpy (qstring->string, str + start, qstring->length); qstring->string[qstring->length] =0 ; -QOBJECT_INIT (qstring,QTYPE_QSTRING , qstring_destroy_obj); +QOBJECT_INIT (qstring,QTYPE_QSTRING );return qstring; }
qobject/qstring.c const char *qstring_get_str(const QString *qstring) · line 136
Quoted 7 lines
qobject/qstring.c +1
-1@@ -133,7 +131,7 @@ const char *qstring_get_str(const QString *qstring)* qstring_destroy_obj(): Free all memory allocated by a QString * object */ -static void qstring_destroy_obj (QObject *obj) +void qstring_destroy_obj (QObject *obj) {QString *qs;
If you'd like to respin, I'd prefer a non-incremental [PATCH v12.5 28/36] replacing v12.
If not, I'm happy to squash in trivial changes like the one I suggested for qdestroy[]'s initializer.
Re: [Qemu-devel] [PATCH] fixup? qapi: Simplify QObject
Quoted 1 line · Markus Armbruster
Eric Blake <[email protected]> writes:
[...]
Quoted 35 lines
Quoted 22 lines
qobject/qobject.c new file +16
-0@@ -0,0 +1,26 @@ +/* +* QObject +* +* Copyright (C) 2015 Red Hat, Inc. +* +* This work is licensed under the terms of the GNU LGPL, version 2.1 +* or later. See the COPYING.LIB file in the top-level directory. +*/ + +#include "qemu-common.h" +#include "qapi/qmp/qbool.h" +#include "qapi/qmp/qdict.h" +#include "qapi/qmp/qfloat.h" +#include "qapi/qmp/qint.h" +#include "qapi/qmp/qlist.h" +#include "qapi/qmp/qstring.h" qobject/qobject.c line 26
Quoted 10 lines
qobject/qobject.c new file +10
-0+ +QDestroy qdestroy[QTYPE_MAX ] = { + [QTYPE_QBOOL ] = qbool_destroy_obj, + [QTYPE_QDICT ] = qdict_destroy_obj, + [QTYPE_QFLOAT ] = qfloat_destroy_obj, + [QTYPE_QINT ] = qint_destroy_obj, + [QTYPE_QLIST ] = qlist_destroy_obj, + [QTYPE_QSTRING ] = qstring_destroy_obj, +/* [QTYPE_QNULL] = NULL, */ +};Suggest
Quoted 2 lines
QDestroy qdestroy[QTYPE_MAX] = {
[QTYPE_QNULL] = NULL, /* no such object exists */QTYPE_NONE, of course.
Quoted 3 lines
[
QTYPE_QNULL] = NULL, /*qnull_is indestructible */
...
};
[...]
Re: [Qemu-devel] [PATCH] fixup? qapi: Simplify QObject
In reply to Markus ArmbrusterQuoted 8 lines · Markus Armbruster
Quoted 6 lines · Eric Blake
[If we squash, replace the old commit message with this; you'll also have some merge conflicts in 29/36 and 30/36. Note that while this mail is net lines added, squashing would still result in a net reduction:
16 files changed, 67 insertions(+), 83 deletions(-)]
I'm leaning towards squashing, so I'll post a v12.5 of 28-30.
Quoted 11 lines
include/qapi/qmp/qobject.h typedef struct QObject QObject; · line 59
Quoted 11 lines
+++
b/include/qapi/qmp/qobject.hinclude/qapi/qmp/qobject.h +3
-1@@ -52,15 +52,17 @@ typedef struct QObject QObject;struct QObject {qtype_code type;size_t refcnt; -void (*destroy)(QObject *); }; +typedef void (*QDestroy )(QObject *); +extern QDestroy qdestroy[QTYPE_MAX ]; +
So if I'm understanding you, instead of declaring the table,...
Quoted 17 lines
include/qapi/qmp/qobject.h static inline void qobject_decref(QObject *obj) · line 104
Quoted 11 lines
include/qapi/qmp/qobject.h +2
-2@@ -97,8 +97,8 @@ static inline void qobject_decref(QObject *obj) {assert (!obj || obj->refcnt);if (obj && --obj->refcnt ==0 ) { -assert (obj->destroy); - obj->destroy (obj); +assert (qdestroy[obj->type]); + qdestroy[obj->type](obj); } }Preexisting: the assertion is of little value, because all it does is convert one kind of crash into another.
Asserting
obj->typeis in range might be more useful, i.e.
Quoted 8 lines
- assert(obj->destroy); - obj->destroy(obj); + assert(QTYPE_QNULL < obj->type && obj->type < QTYPE_MAX); + qdestroy[obj->type](obj);We could outline the conditional part, and keep qdestroy[] static. Could save a bit of code (unless NDEBUG, but we shouldn't optimize for that).
...I could do:
/* High-level interface for qobject_decref() */
#define QDECREF(obj) \
do { \
if (obj) { \
assert(QOBJECT(obj)->refcnt)); \
if (!--QOBJECT(obj)->refcnt) { \
qobject_decref(QOBJECT(obj)); \
} \
} \
} while (0)and move qobject_decref() into qobject.c where the destroy table is then static to that file.
Correct?
Oh wait, it won't work quite like that. We have direct callers of qobject_decref() (QDECREF is only usable on subclasses, but if you have a QObject* you can't use QDECREF), so I can't change semantics by hoisting things into the macro.
Quoted 12 lines
qobject/qobject.c line 26
Quoted 9 lines
qobject/qobject.c new file +9
-0+QDestroy qdestroy[QTYPE_MAX ] = { + [QTYPE_QBOOL ] = qbool_destroy_obj, + [QTYPE_QDICT ] = qdict_destroy_obj, + [QTYPE_QFLOAT ] = qfloat_destroy_obj, + [QTYPE_QINT ] = qint_destroy_obj, + [QTYPE_QLIST ] = qlist_destroy_obj, + [QTYPE_QSTRING ] = qstring_destroy_obj, +/* [QTYPE_QNULL] = NULL, */ +};Suggest
Quoted 3 lines
QDestroy qdestroy[QTYPE_MAX] = {
[QTYPE_QNULL] = NULL, /* no such object exists */
[QTYPE_QNULL] = NULL, /* qnull_ is indestructible */Makes sense (with the obvious fix for QTYPE_NONE).
Quoted 2 lines
If you'd like to respin, I'd prefer a non-incremental [PATCH v12.5 28/36] replacing v12.
Plus 29 and 30 for conflict resolutions.
Quoted 4 lines
If not, I'm happy to squash in trivial changes like the one I suggested for qdestroy[]'s initializer.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH] fixup? qapi: Simplify QObject
Quoted 11 lines · Eric Blake
Quoted 8 lines · Markus Armbruster
Quoted 6 lines · Eric Blake
[If we squash, replace the old commit message with this; you'll also have some merge conflicts in 29/36 and 30/36. Note that while this mail is net lines added, squashing would still result in a net reduction:
16 files changed, 67 insertions(+), 83 deletions(-)]
I'm leaning towards squashing, so I'll post a v12.5 of 28-30.
I'll push my current branch shortly. I'm happy with it up to PATCH 26. I'm still pondering PATCH 27.
Quoted 53 lines
Quoted 11 lines
include/qapi/qmp/qobject.h typedef struct QObject QObject; · line 59
Quoted 11 lines
+++
b/include/qapi/qmp/qobject.hinclude/qapi/qmp/qobject.h +3
-1@@ -52,15 +52,17 @@ typedef struct QObject QObject;struct QObject {qtype_code type;size_t refcnt; -void (*destroy)(QObject *); }; +typedef void (*QDestroy )(QObject *); +extern QDestroy qdestroy[QTYPE_MAX ]; +So if I'm understanding you, instead of declaring the table,...
Quoted 17 lines
include/qapi/qmp/qobject.h static inline void qobject_decref(QObject *obj) · line 104
Quoted 11 lines
include/qapi/qmp/qobject.h +2
-2@@ -97,8 +97,8 @@ static inline void qobject_decref(QObject *obj) {assert (!obj || obj->refcnt);if (obj && --obj->refcnt ==0 ) { -assert (obj->destroy); - obj->destroy (obj); +assert (qdestroy[obj->type]); + qdestroy[obj->type](obj); } }Preexisting: the assertion is of little value, because all it does is convert one kind of crash into another.
Asserting
obj->typeis in range might be more useful, i.e.Quoted 8 lines
- assert(obj->destroy); - obj->destroy(obj); + assert(QTYPE_QNULL < obj->type && obj->type < QTYPE_MAX); + qdestroy[obj->type](obj);We could outline the conditional part, and keep qdestroy[] static. Could save a bit of code (unless NDEBUG, but we shouldn't optimize for that).
...I could do:
/* High-level interface for
qobject_decref()*/#define QDECREF(obj) \ do { \ if (obj) { \ assert(QOBJECT(obj)->refcnt)); \ if (!--QOBJECT(obj)->refcnt) { \ qobject_decref(QOBJECT(obj)); \ } \ } \ } while (0)
Quoted 4 lines
and move
qobject_decref()intoqobject.cwhere the destroy table is then static to that file.Correct?
Almost.
Quoted 4 lines
Oh wait, it won't work quite like that. We have direct callers of
qobject_decref()(QDECREF is only usable on subclasses, but if you have a QObject* you can't use QDECREF), so I can't change semantics by hoisting things into the macro.
In qobject.h:
/**
* qobject_decref(): Decrement QObject's reference count, deallocate
* when it reaches zero
*/
static inline void qobject_decref(QObject *obj)
{
assert(!obj || obj->refcnt);
if (obj && --obj->refcnt == 0) {
qobject_destroy(obj);
}
}In qobject.c:
void qobject_destroy(QObject *obj)
{
assert(!obj->refcnt);
assert(QTYPE_QNULL < obj->type && obj->type < QTYPE_MAX);
qdestroy[obj->type](obj);
}Quoted 27 lines
Quoted 12 lines
qobject/qobject.c line 26
Quoted 9 lines
qobject/qobject.c new file +9
-0+QDestroy qdestroy[QTYPE_MAX ] = { + [QTYPE_QBOOL ] = qbool_destroy_obj, + [QTYPE_QDICT ] = qdict_destroy_obj, + [QTYPE_QFLOAT ] = qfloat_destroy_obj, + [QTYPE_QINT ] = qint_destroy_obj, + [QTYPE_QLIST ] = qlist_destroy_obj, + [QTYPE_QSTRING ] = qstring_destroy_obj, +/* [QTYPE_QNULL] = NULL, */ +};Suggest
Quoted 3 lines
QDestroy qdestroy[QTYPE_MAX] = { [QTYPE_QNULL] = NULL, /* no such object exists */ [QTYPE_QNULL] = NULL, /* qnull_ is indestructible */Makes sense (with the obvious fix for
QTYPE_NONE).Quoted 2 lines
If you'd like to respin, I'd prefer a non-incremental [PATCH v12.5 28/36] replacing v12.
Plus 29 and 30 for conflict resolutions.
Quoted 3 lines
If not, I'm happy to squash in trivial changes like the one I suggested for qdestroy[]'s initializer.
PATCH v12 29/36 qobject: Rename qtype_code to QType
In reply to Eric BlakeThe name QType is more in line with our conventions for qapi types, and matches the fact that each enum member has a prefix of QTYPE_.
---
v12: new patch split off of 21/28
--- block/qapi.c | 4 ++-- include/hw/qdev-core.h | 2 +- include/qapi/qmp/qobject.h | 8 ++++---- qobject/qdict.c | 3 +-- scripts/qapi.py | 2 +- 5 files changed, 9 insertions(+), 10 deletions(-)
block/qapi.c +2 -2
@@ -590,7 +590,7 @@ static void dump_qlist(fprintf_function func_fprintf, void *f, int indentation,int i =0 ;for (entry =qlist_first (list); entry; entry =qlist_next (entry), i++) { -qtype_code type =qobject_type (entry->value); +QType type =qobject_type (entry->value);bool composite = (type ==QTYPE_QDICT || type ==QTYPE_QLIST );const char *format = composite ?"%*s[%i]:\n" :"%*s[%i]: " ;
@@ -608,7 +608,7 @@ static void dump_qdict(fprintf_function func_fprintf, void *f, int indentation,const QDictEntry *entry;for (entry =qdict_first (dict); entry; entry =qdict_next (dict, entry)) { -qtype_code type =qobject_type (entry->value); +QType type =qobject_type (entry->value);bool composite = (type ==QTYPE_QDICT || type ==QTYPE_QLIST );const char *format = composite ?"%*s%s:\n" :"%*s%s: " ;char key[strlen (entry->key) +1 ];
include/hw/qdev-core.h +1 -1
@@ -239,7 +239,7 @@ struct Property {PropertyInfo *info;int offset;uint8_t bitnr; -qtype_code qtype; +QType qtype;int64_t defval;int arrayoffset;PropertyInfo *arrayinfo;
include/qapi/qmp/qobject.h +4 -4
@@ -45,12 +45,12 @@ typedef enum {QTYPE_QFLOAT ,QTYPE_QBOOL ,QTYPE_MAX , -} qtype_code; +} QType;typedef struct QObject QObject ;struct QObject { -qtype_code type; +QType type;size_t refcnt;void (*destroy)(QObject *); };
@@ -71,7 +71,7 @@ struct QObject {qobject_decref (obj ?QOBJECT (obj) :NULL ) /* Initialize an object to default values */ -static inline void qobject_init (QObject *obj,qtype_code type, +static inline void qobject_init (QObject *obj,QType type,void (*destroy)(struct QObject *)) {assert (type);
@@ -105,7 +105,7 @@ static inline void qobject_decref(QObject *obj)/** * qobject_type(): Return the QObject's type */ -static inline qtype_code qobject_type (const QObject *obj) +static inline QType qobject_type (const QObject *obj) {assert (obj->type);return obj->type;
qobject/qdict.c +1 -2
@@ -179,8 +179,7 @@ size_t qdict_size(const QDict *qdict)/** * qdict_get_obj(): Get a QObject of a specific type */ -static QObject *qdict_get_obj (const QDict *qdict,const char *key, -qtype_code type) +static QObject *qdict_get_obj (const QDict *qdict,const char *key,QType type) {QObject *obj;
scripts/qapi.py +1 -1
@@ -34,7 +34,7 @@ builtin_types = { 'uint32': 'QTYPE_QINT', 'uint64': 'QTYPE_QINT', 'size': 'QTYPE_QINT', - 'any': None, # any qtype_code possible, actually + 'any': None, # any QType possible, actually } # Whitelist of commands allowed to return a non-dictionary
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 29/36] qobject: Rename qtype_code to QType
Quoted 5 lines · Eric Blake
The name QType is more in line with our conventions for qapi types, and matches the fact that each enum member has a prefix of
QTYPE_.Signed-off-by Eric Blake <[email protected]>
At this point, the connection to QAPI is unobvious.
You can either point to CODING_STYLE, which ask for type names in CamelCase, or to the next patch.
Patch looks good.
Re: [Qemu-devel] [PATCH v12 29/36] qobject: Rename qtype_code to QType
Quoted 6 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
The name QType is more in line with our conventions for qapi types, and matches the fact that each enum member has a prefix of
QTYPE_.
Quoted 6 lines
Quoted 1 line
Signed-off-by Eric Blake <[email protected]>At this point, the connection to QAPI is unobvious.
You can either point to
CODING_STYLE, which ask for type names in CamelCase, or to the next patch.
How about:
The name QType matches our CODING_STYLE conventions for type names in CamelCase. It also matches the fact that we are already naming all the enum members with a prefix of QTYPE, not QTYPE_CODE. And doing the rename will also make it easier for the next patch to use QAPI for the enum, which also wants CamelCase type names.
Feel free to touch up the commit message as you see fit.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 29/36] qobject: Rename qtype_code to QType
Quoted 16 lines · Eric Blake
Quoted 6 lines · Markus Armbruster
Quoted 4 lines · Eric Blake
The name QType is more in line with our conventions for qapi types, and matches the fact that each enum member has a prefix of
QTYPE_.Quoted 6 lines
Quoted 1 line
Signed-off-by Eric Blake <[email protected]>At this point, the connection to QAPI is unobvious.
You can either point to
CODING_STYLE, which ask for type names in CamelCase, or to the next patch.How about:
Quoted 7 lines
The name QType matches our
CODING_STYLEconventions for type names in CamelCase. It also matches the fact that we are already naming all the enum members with a prefix of QTYPE, notQTYPE_CODE. And doing the rename will also make it easier for the next patch to use QAPI for the enum, which also wants CamelCase type names.Feel free to touch up the commit message as you see fit.
Sold!
PATCH v12 30/36 qapi: Convert QType into qapi builtin enum type
In reply to Eric BlakeWhat's more meta than using qapi to define qapi? :)
Convert QType into a full-fledged[*] builtin qapi enum type, so that a subsequent patch can then use it as the discriminator type of qapi alternate types. Fortunately, the judicious use of 'prefix' in the qapi definition avoids churn to the spelling of the enum constants.
To avoid circular definitions, we have to flip the order of inclusion between "qobject.h" vs. "qapi-types.h". Back in commit 28770e0, we had the latter include the former, so that we could use 'QObject *' for our implementation of 'any'. But that usage also works with only a forward declaration, whereas the definition of QObject requires QType to be a complete type.
[*] The type has to be builtin, rather than declared in qapi/common.json, because we want to use it for alternates even when common.json is not included. But since it is the first builtin enum type, we have to add special cases to qapi-types and qapi-visit to only emit definitions once, even when two qapi files are being compiled into the same binary (the way we already handled builtin list types like 'intList'). We may need to revisit how multiple qapi files share common types, but that's a project for another day.
---
v12: split out preparatory renames, retitle patch, use info rather than name comparison
v11: new patch
--- docs/qapi-code-gen.txt | 1 + include/qapi/qmp/qobject.h | 15 +-------------- scripts/qapi-types.py | 16 ++++++++++++---- scripts/qapi-visit.py | 11 +++++++++-- scripts/qapi.py | 6 ++++++ tests/qapi-schema/alternate-empty.out | 2 ++ tests/qapi-schema/comments.out | 2 ++ tests/qapi-schema/empty.out | 2 ++ tests/qapi-schema/event-case.out | 2 ++ tests/qapi-schema/flat-union-empty.out | 2 ++ tests/qapi-schema/ident-with-escape.out | 2 ++ tests/qapi-schema/include-relpath.out | 2 ++ tests/qapi-schema/include-repetition.out | 2 ++ tests/qapi-schema/include-simple.out | 2 ++ tests/qapi-schema/indented-expr.out | 2 ++ tests/qapi-schema/qapi-schema-test.out | 2 ++ tests/qapi-schema/union-clash-data.out | 2 ++ tests/qapi-schema/union-empty.out | 2 ++ 18 files changed, 55 insertions(+), 20 deletions(-)
docs/qapi-code-gen.txt +1 -0
@@ -168,6 +168,7 @@ The following types are predefined, and map to C as follows: accepts size suffixes bool bool JSON true or false any QObject * any JSON value + QType QType JSON string matching enum QType values === Includes ===
include/qapi/qmp/qobject.h +1 -14
@@ -34,20 +34,7 @@#include <stddef.h> #include <assert.h> - -typedef enum { -QTYPE_NONE ,/* sentinel value, no QObject has this type code */ -QTYPE_QNULL , -QTYPE_QINT , -QTYPE_QSTRING , -QTYPE_QDICT , -QTYPE_QLIST , -QTYPE_QFLOAT , -QTYPE_QBOOL , -QTYPE_MAX , -}QType ; - -typedef struct QObject QObject ; +#include "qapi-types.h" struct QObject {QType type;
scripts/qapi-types.py +12 -4
@@ -112,7 +112,7 @@ extern const int %(c_name)s_qtypes[]; def gen_alternate_qtypes(name, variants): ret = mcgen(''' -const int %(c_name)s_qtypes[QTYPE_MAX] = { +const int %(c_name)s_qtypes[QTYPE__MAX] = { ''', c_name=c_name(name))
@@ -233,8 +233,15 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): self.defn += gen_type_cleanup(name) def visit_enum_type(self, name, info, values, prefix): - self._fwdecl += gen_enum(name, values, prefix) - self._fwdefn += gen_enum_lookup(name, values, prefix) + # Special case for our lone builtin enum type + # TODO use something cleaner than existence of info + if not info: + self._btin += gen_enum(name, values, prefix) + if do_builtins: + self.defn += gen_enum_lookup(name, values, prefix) + else: + self._fwdecl += gen_enum(name, values, prefix) + self._fwdefn += gen_enum_lookup(name, values, prefix) def visit_array_type(self, name, info, element_type): if isinstance(element_type, QAPISchemaBuiltinType):
@@ -319,7 +326,8 @@ fdef.write(mcgen(''' fdecl.write(mcgen(''' #include <stdbool.h> #include <stdint.h> -#include "qapi/qmp/qobject.h" + +typedef struct QObject QObject; ''')) schema = QAPISchema(input_file)
scripts/qapi-visit.py +9 -2
@@ -347,8 +347,15 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor): isinstance(entity, QAPISchemaObjectType)) def visit_enum_type(self, name, info, values, prefix): - self.decl += gen_visit_decl(name, scalar=True) - self.defn += gen_visit_enum(name) + # Special case for our lone builtin enum type + # TODO use something cleaner than existence of info + if not info: + self._btin += gen_visit_decl(name, scalar=True) + if do_builtins: + self.defn += gen_visit_enum(name) + else: + self.decl += gen_visit_decl(name, scalar=True) + self.defn += gen_visit_enum(name) def visit_array_type(self, name, info, element_type): decl = gen_visit_decl(name)
scripts/qapi.py +6 -0
@@ -35,6 +35,7 @@ builtin_types = { 'uint64': 'QTYPE_QINT', 'size': 'QTYPE_QINT', 'any': None, # any QType possible, actually + 'QType': 'QTYPE_QSTRING', } # Whitelist of commands allowed to return a non-dictionary
@@ -1267,6 +1268,11 @@ class QAPISchema(object): self.the_empty_object_type = QAPISchemaObjectType(':empty', None, None, [], None) self._def_entity(self.the_empty_object_type) + self._def_entity(QAPISchemaEnumType('QType', None, + ['none', 'qnull', 'qint', + 'qstring', 'qdict', 'qlist', + 'qfloat', 'qbool'], + 'QTYPE')) def _make_implicit_enum_type(self, name, info, values): name = name + 'Kind' # Use namespace reserved by add_name()
tests/qapi-schema/alternate-empty.out +2 -0
@@ -2,3 +2,5 @@ object :empty alternate Alt case i: int enum AltKind ['i'] +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE
tests/qapi-schema/comments.out +2 -0
@@ -1,2 +1,4 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/empty.out +2 -0
@@ -1 +1,3 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE
tests/qapi-schema/event-case.out +2 -0
@@ -1,2 +1,4 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE event oops None
tests/qapi-schema/flat-union-empty.out +2 -0
@@ -2,6 +2,8 @@ object :empty object Base member type: Empty optional=False enum Empty [] +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE object Union base Base tag type
tests/qapi-schema/ident-with-escape.out +2 -0
@@ -1,5 +1,7 @@ object :empty object :obj-fooA-arg member bar1: str optional=False +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE command fooA :obj-fooA-arg -> None gen=True success_response=True
tests/qapi-schema/include-relpath.out +2 -0
@@ -1,2 +1,4 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/include-repetition.out +2 -0
@@ -1,2 +1,4 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/include-simple.out +2 -0
@@ -1,2 +1,4 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE enum Status ['good', 'bad', 'ugly']
tests/qapi-schema/indented-expr.out +2 -0
@@ -1,4 +1,6 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE command eins None -> None gen=True success_response=True command zwei None -> None
tests/qapi-schema/qapi-schema-test.out +2 -0
@@ -101,6 +101,8 @@ object NestedEnumsOne member enum4: EnumOne optional=True enum QEnumTwo ['value1', 'value2'] prefix QENUM_TWO +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE object TestStruct member integer: int optional=False member boolean: bool optional=False
tests/qapi-schema/union-clash-data.out +2 -0
@@ -1,6 +1,8 @@ object :empty object :obj-int-wrapper member data: int optional=False +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE object TestUnion member type: TestUnionKind optional=False case data: :obj-int-wrapper
tests/qapi-schema/union-empty.out +2 -0
@@ -1,4 +1,6 @@ object :empty +enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] + prefix QTYPE object Union member type: UnionKind optional=False enum UnionKind []
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 30/36] qapi: Convert QType into qapi builtin enum type
Quoted 8 lines · Eric Blake
What's more meta than using qapi to define qapi? :)
Convert QType into a full-fledged[*] builtin qapi enum type, so that a subsequent patch can then use it as the discriminator type of qapi alternate types. Fortunately, the judicious use of 'prefix' in the qapi definition avoids churn to the spelling of the enum constants.
Quoted 6 lines
To avoid circular definitions, we have to flip the order of inclusion between "
qobject.h" vs. "qapi-types.h". Back in commit 28770e0, we had the latter include the former, so that we could use 'QObject *' for our implementation of 'any'. But that usage also works with only a forward declaration, whereas the definition of QObject requires QType to be a complete type.
I figure putting the typedef in include/qemu/typedefs.h would be simpler. Didn't think of it when I reviewed v11.
Quoted 11 lines
[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even when common.json is not included. But since it is the first builtin enum type, we have to add special cases to qapi-types and qapi-visit to only emit definitions once, even when two qapi files are being compiled into the same binary (the way we already handled builtin list types like 'intList'). We may need to revisit how multiple qapi files share common types, but that's a project for another day.
Quoted 1 line
Signed-off-by Eric Blake <[email protected]>
Patch looks sane.
Re: [Qemu-devel] [PATCH v12 30/36] qapi: Convert QType into qapi builtin enum type
In reply to Eric BlakeSuggest to spell it "QAPI built-in enum type" in the subject.
In general, I prefer "QAPI" when referring to the concept, because it's actually an acronym.
PATCH 1/1 fixup! qapi: Convert QType into qapi builtin enum type
In reply to Eric Blake[keep the original commit body, but change the title:]
qapi: Convert QType into QAPI built-in enum type
--- include/qemu/typedefs.h | 1 + scripts/qapi-types.py | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-)
include/qemu/typedefs.h +1 -0
@@ -80,6 +80,7 @@ typedef struct QEMUSGList QEMUSGList;typedef struct QEMUSizedBuffer QEMUSizedBuffer ;typedef struct QEMUTimerListGroup QEMUTimerListGroup ;typedef struct QEMUTimer QEMUTimer ; +typedef struct QObject QObject ;typedef struct Range Range ;typedef struct RAMBlock RAMBlock ;typedef struct SerialState SerialState ;
scripts/qapi-types.py +1 -2
@@ -326,8 +326,7 @@ fdef.write(mcgen(''' fdecl.write(mcgen(''' #include <stdbool.h> #include <stdint.h> - -typedef struct QObject QObject; +#include "qemu/typedefs.h" ''')) schema = QAPISchema(input_file)
Signature · 1 line
2.4.3
PATCH v12 31/36 qapi: Simplify visiting of alternate types
In reply to Eric BlakePreviously, working with alternates required two lookup arrays and some indirection: for type Foo, we created Foo_qtypes[] which maps each qtype to a value of the generated FooKind enum, then look up that value in FooKind_lookup[] like we do for other union types.
This has a couple of subtle bugs. First, the generator was creating a call with a parameter '(int *) &(*obj)->type' where type is an enum type; this is unsafe if the compiler chooses to store the enum type in a different size than int, where assigning through the wrong size pointer can corrupt data or cause a SIGBUS. [We still have the casting bug for our enum visitors, but that's a topic for a different patch.]
Second, since the values of the FooKind enum start at zero, all entries of the Foo_qtypes[] array that were not explicitly initialized will map to the same branch of the union as the first member of the alternate, rather than triggering a desired failure in visit_get_next_type(). Fortunately, the bug seldom bites; the very next thing the input visitor does is try to parse the incoming JSON with the wrong parser, which normally fails; the output visitor is not used with a C struct in that state, and the dealloc visitor has nothing to clean up (so there is no leak).
However, the second bug IS observable in one case: parsing an integer causes unusual behavior in an alternate that contains at least a 'number' member but no 'int' member, because the 'number' parser accepts QTYPE_QINT in addition to the expected QTYPE_QFLOAT (that is, since 'int' is not a member, the type QTYPE_QINT accidentally maps to FooKind 0; if this enum value is the 'number' branch the integer parses successfully, but if the 'number' branch is not first, some other branch tries to parse the integer and rejects it). A later patch will worry about fixing alternates to always parse all inputs that a non-alternate 'number' would accept, for now this is still marked FIXME in the updated test-qmp-input-visitor.c, to merely point out that new undesired behavior of 'ans' matches the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the indirection, and modifying get_next_type() to directly assign a QTypeCode parameter. This in turn fixes the type-casting bug, as we are no longer casting a pointer to enum to a questionable size. There is no longer a need to generate an implicit FooKind enum associated with the alternate type (since the QMP wire format never uses the stringized counterparts of the C union member names). Since the updated visit_get_next_type() does not know which qtypes are expected, the generated visitor is modified to generate an error statement if an unexpected type is encountered.
Callers now have to know the QTYPE_* mapping when looking at the discriminator; but so far, only the testsuite was even using the C struct of an alternate types. I considered the possibility of keeping the internal enum FooKind, but initialized differently than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b when inspecting foo->type; but it turned out to add too much complexity, especially without a client.
There is a user-visible side effect to this change, but I consider it to be an improvement. Previously, the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}(visit_get_next_type() succeeded, and the error comes from the visit_type_BlockdevOptions() expecting {}; there is no mention of the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for the overall alternate).
---
v12: rebase to earlier 'max' collision avoidance, some variable renames v11 (no v10): rebase to new QTypeCode, with fewer special cases; tweak commit message to match
v9: rebase to earlier changes, rework commit message to mention second bug fix; move positive test in qapi-schema-test to later patch
v8: no change
v7: rebase onto earlier changes, rework how subtype makes things work
v6: rebase onto tag_member subclass, testsuite, gen_err_check(), and info improvements
--- docs/qapi-code-gen.txt | 3 --- include/qapi/visitor-impl.h | 3 ++- include/qapi/visitor.h | 8 +++++++- qapi/qapi-visit-core.c | 4 ++-- qapi/qmp-input-visitor.c | 4 ++-- scripts/qapi-types.py | 34 ---------------------------------- scripts/qapi-visit.py | 14 +++++++++----- scripts/qapi.py | 18 +++++++++++++----- tests/qapi-schema/alternate-empty.out | 1 - tests/qapi-schema/qapi-schema-test.out | 8 -------- tests/test-qmp-input-visitor.c | 31 ++++++++++++++++--------------- tests/test-qmp-output-visitor.c | 4 ++-- 12 files changed, 53 insertions(+), 79 deletions(-)
docs/qapi-code-gen.txt +0 -3
@@ -392,9 +392,6 @@ where each branch of the union names a QAPI type. For example: 'data': { 'definition': 'BlockdevOptions', 'reference': 'str' } } -Just like for a simple union, an implicit C enum 'NameKind' is created -to enumerate the branches for the alternate 'Name'. - Unlike a union, the discriminator string is never passed on the wire for the Client JSON Protocol. Instead, the value's JSON type serves as an implicit discriminator, which in turn means that an alternate
include/qapi/visitor-impl.h +2 -1
@@ -32,7 +32,8 @@ struct Visitorvoid (*type_enum)(Visitor *v,int *obj,const char *const strings[],const char *kind,const char *name,Error **errp); -void (*get_next_type)(Visitor *v,int *kind,const int *qobjects, +/* May be NULL; only needed for input visitors. */ +void (*get_next_type)(Visitor *v,QType *type,const char *name,Error **errp);void (*type_int)(Visitor *v,int64_t *obj,const char *name,Error **errp);
include/qapi/visitor.h +7 -1
@@ -38,7 +38,13 @@ GenericList *visit_next_list(Visitor *v, GenericList **list, Error **errp);void visit_end_list (Visitor *v,Error **errp);void visit_optional (Visitor *v,bool *present,const char *name,Error **errp); -void visit_get_next_type (Visitor *v,int *obj,const int *qtypes, + +/** +* Determine the qtype of the item @name in the current object visit. +* For input visitors, set *@type to the correct qtype of a qapi +* alternate type; for other visitors, leave *@type unchanged. +*/ +void visit_get_next_type (Visitor *v,QType *type,const char *name,Error **errp);void visit_type_enum (Visitor *v,int *obj,const char *const strings[],const char *kind,const char *name,Error **errp);
qapi/qapi-visit-core.c +2 -2
@@ -81,11 +81,11 @@ void visit_optional(Visitor *v, bool *present, const char *name, } } -void visit_get_next_type (Visitor *v,int *obj,const int *qtypes, +void visit_get_next_type (Visitor *v,QType *type,const char *name,Error **errp) {if (v->get_next_type) { - v->get_next_type (v, obj, qtypes, name, errp); + v->get_next_type (v, type, name, errp); } }
qapi/qmp-input-visitor.c +2 -2
@@ -208,7 +208,7 @@ static void qmp_input_end_list(Visitor *v, Error **errp)qmp_input_pop (qiv, errp); } -static void qmp_input_get_next_type (Visitor *v,int *kind,const int *qobjects, +static void qmp_input_get_next_type (Visitor *v,QType *type,const char *name,Error **errp) {QmpInputVisitor *qiv =to_qiv (v);
@@ -218,7 +218,7 @@ static void qmp_input_get_next_type(Visitor *v, int *kind, const int *qobjects,error_setg (errp,QERR_MISSING_PARAMETER , name ? name :"null" );return ; } - *kind = qobjects[qobject_type (qobj)]; + *type =qobject_type (qobj); } static void qmp_input_type_int(Visitor *v, int64_t *obj, const char *name,
scripts/qapi-types.py +0 -34
@@ -101,38 +101,6 @@ static inline %(base)s *qapi_%(c_name)s_base(const %(c_name)s *obj) c_name=c_name(name), base=base.c_name()) -def gen_alternate_qtypes_decl(name): - return mcgen(''' - -extern const int %(c_name)s_qtypes[]; -''', - c_name=c_name(name)) - - -def gen_alternate_qtypes(name, variants): - ret = mcgen(''' - -const int %(c_name)s_qtypes[QTYPE__MAX] = { -''', - c_name=c_name(name)) - - for var in variants.variants: - qtype = var.type.alternate_qtype() - assert qtype - - ret += mcgen(''' - [%(qtype)s] = %(enum_const)s, -''', - qtype=qtype, - enum_const=c_enum_const(variants.tag_member.type.name, - var.name)) - - ret += mcgen(''' -}; -''') - return ret - - def gen_variants(variants): # FIXME: What purpose does data serve, besides preventing a union that # has a branch named 'data'? We use it in qapi-visit.py to decide
@@ -264,9 +232,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): def visit_alternate_type(self, name, info, variants): self._fwdecl += gen_fwd_object_or_array(name) - self._fwdefn += gen_alternate_qtypes(name, variants) self.decl += gen_object(name, None, [variants.tag_member], variants) - self.decl += gen_alternate_qtypes_decl(name) self._gen_type_cleanup(name) # If you link code generated from multiple schemata, you want only one
scripts/qapi-visit.py +9 -5
@@ -193,7 +193,7 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error if (err) { goto out; } - visit_get_next_type(v, (int*) &(*obj)->type, %(c_name)s_qtypes, name, &err); + visit_get_next_type(v, &(*obj)->type, name, &err); if (err) { goto out_obj; }
@@ -201,20 +201,22 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error ''', c_name=c_name(name)) + # FIXME: When 'number' but not 'int' is present in the alternate, we + # should allow QTYPE_INT to promote to QTYPE_FLOAT. for var in variants.variants: ret += mcgen(''' case %(case)s: visit_type_%(c_type)s(v, &(*obj)->u.%(c_name)s, name, &err); break; ''', - case=c_enum_const(variants.tag_member.type.name, - var.name), + case=var.type.alternate_qtype(), c_type=var.type.c_name(), c_name=c_name(var.name)) ret += mcgen(''' default: - abort(); + error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null", + "%(name)s"); } out_obj: error_propagate(errp, err);
@@ -223,7 +225,8 @@ out_obj: out: error_propagate(errp, err); } -''') +''', + name=name) return ret
@@ -437,6 +440,7 @@ fdef.write(mcgen(''' fdecl.write(mcgen(''' #include "qapi/visitor.h" +#include "qapi/qmp/qerror.h" #include "%(prefix)sqapi-types.h" ''',
scripts/qapi.py +13 -5
@@ -636,8 +636,8 @@ def check_alternate(expr, expr_info): for (key, value) in members.items(): check_name(expr_info, "Member of alternate '%s'" % name, key) - # Check for conflicts in the generated enum - c_key = camel_to_upper(key) + # Check for conflicts in the branch names + c_key = c_name(key) if c_key in values: raise QAPIExprError(expr_info, "Alternate '%s' member '%s' clashes with '%s'"
@@ -1092,8 +1092,11 @@ class QAPISchemaObjectTypeVariants(object): assert isinstance(self.tag_member.type, QAPISchemaEnumType) for v in self.variants: v.check(schema) - assert v.name in self.tag_member.type.values - if isinstance(v.type, QAPISchemaObjectType): + # Union names must match enum values; alternate names are + # checked separately. Use 'seen' to tell the two apart. + if seen: + assert v.name in self.tag_member.type.values + assert isinstance(v.type, QAPISchemaObjectType) v.type.check(schema) def check_clash(self, schema, info, seen):
@@ -1135,6 +1138,11 @@ class QAPISchemaAlternateType(QAPISchemaType): # Not calling self.variants.check_clash(), because there's nothing # to clash with self.variants.check(schema, {}) + # Alternate branch names have no relation to the tag enum values; + # so we have to check for potential name collisions ourselves. + seen = {} + for v in self.variants.variants: + v.check_clash(self.info, seen) def json_type(self): return 'value'
@@ -1365,7 +1373,7 @@ class QAPISchema(object): data = expr['data'] variants = [self._make_variant(key, value) for (key, value) in data.iteritems()] - tag_member = self._make_implicit_tag(name, info, variants) + tag_member = QAPISchemaObjectTypeMember('type', 'QType', False) self._def_entity( QAPISchemaAlternateType(name, info, QAPISchemaObjectTypeVariants(None,
tests/qapi-schema/alternate-empty.out +0 -1
@@ -1,6 +1,5 @@ object :empty alternate Alt case i: int -enum AltKind ['i'] enum QType ['none', 'qnull', 'qint', 'qstring', 'qdict', 'qlist', 'qfloat', 'qbool'] prefix QTYPE
tests/qapi-schema/qapi-schema-test.out +0 -8
@@ -56,27 +56,21 @@ object :obj-user_def_cmd2-arg alternate AltIntNum case i: int case n: number -enum AltIntNumKind ['i', 'n'] alternate AltNumInt case n: number case i: int -enum AltNumIntKind ['n', 'i'] alternate AltNumStr case n: number case s: str -enum AltNumStrKind ['n', 's'] alternate AltStrBool case s: str case b: bool -enum AltStrBoolKind ['s', 'b'] alternate AltStrInt case s: str case i: int -enum AltStrIntKind ['s', 'i'] alternate AltStrNum case s: str case n: number -enum AltStrNumKind ['s', 'n'] event EVENT_A None event EVENT_B None event EVENT_C :obj-EVENT_C-arg
@@ -114,7 +108,6 @@ alternate UserDefAlternate case uda: UserDefA case s: str case i: int -enum UserDefAlternateKind ['uda', 's', 'i'] object UserDefB member intb: int optional=False member a-b: bool optional=True
@@ -180,7 +173,6 @@ event __ORG.QEMU_X-EVENT __org.qemu_x-Struct alternate __org.qemu_x-Alt case __org.qemu_x-branch: str case b: __org.qemu_x-Base -enum __org.qemu_x-AltKind ['__org.qemu_x-branch', 'b'] object __org.qemu_x-Base member __org.qemu_x-member1: __org.qemu_x-Enum optional=False enum __org.qemu_x-Enum ['__org.qemu_x-value']
tests/test-qmp-input-visitor.c +16 -15
@@ -312,13 +312,13 @@ static void test_visitor_in_alternate(TestInputVisitorData *data, v =visitor_input_test_init (data,"42" );visit_type_UserDefAlternate (v, &tmp,NULL , &error_abort); - g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_I); + g_assert_cmpint(tmp->type, ==, QTYPE_QINT); g_assert_cmpint(tmp->u.i, ==, 42);qapi_free_UserDefAlternate (tmp); v =visitor_input_test_init (data,"'string'" );visit_type_UserDefAlternate (v, &tmp,NULL , &error_abort); - g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_S); + g_assert_cmpint(tmp->type, ==, QTYPE_QSTRING); g_assert_cmpstr(tmp->u.s, ==, "string");qapi_free_UserDefAlternate (tmp);
@@ -347,36 +347,37 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,error_free_or_abort (&err);qapi_free_AltStrBool (asb); -/* FIXME: Order of alternate should not affect semantics; asn should -* parse the same as ans */ +/* FIXME: integer should parse as number */ v =visitor_input_test_init (data,"42" );visit_type_AltStrNum (v, &asn,NULL , &err); -/* FIXME g_assert_cmpint(asn->type, == ALT_STR_NUM_KIND_N); */ +/* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */ /* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */ error_free_or_abort (&err);qapi_free_AltStrNum (asn); +/* FIXME: integer should parse as number */ v =visitor_input_test_init (data,"42" ); -visit_type_AltNumStr (v, &ans,NULL , &error_abort); - g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N); - g_assert_cmpfloat(ans->u.n, ==, 42); +visit_type_AltNumStr (v, &ans,NULL , &err); +/* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */ +/* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */ +error_free_or_abort (&err);qapi_free_AltNumStr (ans); v =visitor_input_test_init (data,"42" );visit_type_AltStrInt (v, &asi,NULL , &error_abort); - g_assert_cmpint(asi->type, ==, ALT_STR_INT_KIND_I); + g_assert_cmpint(asi->type, ==, QTYPE_QINT); g_assert_cmpint(asi->u.i, ==, 42);qapi_free_AltStrInt (asi); v =visitor_input_test_init (data,"42" );visit_type_AltIntNum (v, &ain,NULL , &error_abort); - g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_I); + g_assert_cmpint(ain->type, ==, QTYPE_QINT); g_assert_cmpint(ain->u.i, ==, 42);qapi_free_AltIntNum (ain); v =visitor_input_test_init (data,"42" );visit_type_AltNumInt (v, &ani,NULL , &error_abort); - g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_I); + g_assert_cmpint(ani->type, ==, QTYPE_QINT); g_assert_cmpint(ani->u.i, ==, 42);qapi_free_AltNumInt (ani);
@@ -389,13 +390,13 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data, v =visitor_input_test_init (data,"42.5" );visit_type_AltStrNum (v, &asn,NULL , &error_abort); - g_assert_cmpint(asn->type, ==, ALT_STR_NUM_KIND_N); + g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); g_assert_cmpfloat(asn->u.n, ==, 42.5);qapi_free_AltStrNum (asn); v =visitor_input_test_init (data,"42.5" );visit_type_AltNumStr (v, &ans,NULL , &error_abort); - g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N); + g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); g_assert_cmpfloat(ans->u.n, ==, 42.5);qapi_free_AltNumStr (ans);
@@ -406,13 +407,13 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data, v =visitor_input_test_init (data,"42.5" );visit_type_AltIntNum (v, &ain,NULL , &error_abort); - g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_N); + g_assert_cmpint(ain->type, ==, QTYPE_QFLOAT); g_assert_cmpfloat(ain->u.n, ==, 42.5);qapi_free_AltIntNum (ain); v =visitor_input_test_init (data,"42.5" );visit_type_AltNumInt (v, &ani,NULL , &error_abort); - g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_N); + g_assert_cmpint(ani->type, ==, QTYPE_QFLOAT); g_assert_cmpfloat(ani->u.n, ==, 42.5);qapi_free_AltNumInt (ani); }
tests/test-qmp-output-visitor.c +2 -2
@@ -428,7 +428,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,UserDefAlternate *tmp; tmp =g_new0 (UserDefAlternate,1 ); - tmp->type =USER_DEF_ALTERNATE_KIND_I ; + tmp->type =QTYPE_QINT ; tmp->u.i =42 ;visit_type_UserDefAlternate (data->ov, &tmp,NULL , &error_abort);
@@ -441,7 +441,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,qobject_decref (arg); tmp =g_new0 (UserDefAlternate,1 ); - tmp->type =USER_DEF_ALTERNATE_KIND_S ; + tmp->type =QTYPE_QSTRING ; tmp->u.s =g_strdup ("hello" );visit_type_UserDefAlternate (data->ov, &tmp,NULL , &error_abort);
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 31/36] qapi: Simplify visiting of alternate types
Quoted 6 lines · Eric Blake
Previously, working with alternates required two lookup arrays and some indirection: for type Foo, we created Foo_qtypes[] which maps each qtype to a value of the generated FooKind enum, then look up that value in FooKind_lookup[] like we do for other union types.
Quoted 7 lines
This has a couple of subtle bugs. First, the generator was creating a call with a parameter '(int *) &(*obj)->type' where type is an enum type; this is unsafe if the compiler chooses to store the enum type in a different size than int, where assigning through the wrong size pointer can corrupt data or cause a SIGBUS. [We still have the casting bug for our enum visitors, but that's a topic for a different patch.]
I'm not sure I get the last sentence.
Quoted 63 lines
Second, since the values of the FooKind enum start at zero, all entries of the Foo_qtypes[] array that were not explicitly initialized will map to the same branch of the union as the first member of the alternate, rather than triggering a desired failure in
visit_get_next_type(). Fortunately, the bug seldom bites; the very next thing the input visitor does is try to parse the incoming JSON with the wrong parser, which normally fails; the output visitor is not used with a C struct in that state, and the dealloc visitor has nothing to clean up (so there is no leak).However, the second bug IS observable in one case: parsing an integer causes unusual behavior in an alternate that contains at least a 'number' member but no 'int' member, because the 'number' parser accepts
QTYPE_QINTin addition to the expectedQTYPE_QFLOAT(that is, since 'int' is not a member, the typeQTYPE_QINTaccidentally maps to FooKind 0; if this enum value is the 'number' branch the integer parses successfully, but if the 'number' branch is not first, some other branch tries to parse the integer and rejects it). A later patch will worry about fixing alternates to always parse all inputs that a non-alternate 'number' would accept, for now this is still marked FIXME in the updatedtest-qmp-input-visitor.c, to merely point out that new undesired behavior of 'ans' matches the existing undesired behavior of 'asn'.This patch fixes the default-initialization bug by deleting the indirection, and modifying
get_next_type()to directly assign a QTypeCode parameter. This in turn fixes the type-casting bug, as we are no longer casting a pointer to enum to a questionable size. There is no longer a need to generate an implicit FooKind enum associated with the alternate type (since the QMP wire format never uses the stringized counterparts of the C union member names). Since the updatedvisit_get_next_type()does not know which qtypes are expected, the generated visitor is modified to generate an error statement if an unexpected type is encountered.Callers now have to know the QTYPE_* mapping when looking at the discriminator; but so far, only the testsuite was even using the C struct of an alternate types. I considered the possibility of keeping the internal enum FooKind, but initialized differently than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A=QTYPE_QDICT,
FOO_KIND_B=QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to usefoo->aorfoo->bwhen inspectingfoo->type; but it turned out to add too much complexity, especially without a client.There is a user-visible side effect to this change, but I consider it to be an improvement. Previously, the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":{"driver":"raw", "id":"a", "file":true}}} failed with: {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'file', expected: QDict"}}(
visit_get_next_type()succeeded, and the error comes from thevisit_type_BlockdevOptions()expecting {}; there is no mention of the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
Quoted 5 lines
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}(the error when the next type doesn't match any expected types for the overall alternate).
Signed-off-by Eric Blake <[email protected]>
Patch looks good.
Re: [Qemu-devel] [PATCH v12 31/36] qapi: Simplify visiting of alternate types
Quoted 16 lines · Markus Armbruster
Quoted 6 lines · Eric Blake
Previously, working with alternates required two lookup arrays and some indirection: for type Foo, we created Foo_qtypes[] which maps each qtype to a value of the generated FooKind enum, then look up that value in FooKind_lookup[] like we do for other union types.
Quoted 7 lines
This has a couple of subtle bugs. First, the generator was creating a call with a parameter '(int *) &(*obj)->type' where type is an enum type; this is unsafe if the compiler chooses to store the enum type in a different size than int, where assigning through the wrong size pointer can corrupt data or cause a SIGBUS. [We still have the casting bug for our enum visitors, but that's a topic for a different patch.]
Quoted 1 line
I'm not sure I get the last sentence.
I was referring to our casts of enum types to int* inside visit_type_Enum():
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg03408.html
If you have a better wording for it, or want to drop the parenthetical altogether, I'm fine.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 31/36] qapi: Simplify visiting of alternate types
Quoted 17 lines · Eric Blake
Quoted 16 lines · Markus Armbruster
Quoted 13 lines · Eric Blake
Previously, working with alternates required two lookup arrays and some indirection: for type Foo, we created Foo_qtypes[] which maps each qtype to a value of the generated FooKind enum, then look up that value in FooKind_lookup[] like we do for other union types.
This has a couple of subtle bugs. First, the generator was creating a call with a parameter '(int *) &(*obj)->type' where type is an enum type; this is unsafe if the compiler chooses to store the enum type in a different size than int, where assigning through the wrong size pointer can corrupt data or cause a SIGBUS. [We still have the casting bug for our enum visitors, but that's a topic for a different patch.]
Quoted 7 lines
Quoted 1 line
I'm not sure I get the last sentence.
I was referring to our casts of enum types to int* inside
visit_type_Enum():
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg03408.htmlIf you have a better wording for it, or want to drop the parenthetical altogether, I'm fine.
What about this: replace the parenthetical with
Related bug, not fixed in this patch: qapi-visit.py's gen_visit_enum() generates a cast of its enum * argument to int *. Marked FIXME.
and squash in
scripts/qapi-visit.py +1 -0
@@ -172,6 +172,7 @@ out: def gen_visit_enum(name): + # FIXME cast from enum *obj to int * invalidly assumes enum is int return mcgen(''' void visit_type_%(c_name)s(Visitor *v, %(c_name)s *obj, const char *name, Error **errp)
Re: [Qemu-devel] [PATCH v12 31/36] qapi: Simplify visiting of alternate types
Quoted 29 lines · Markus Armbruster
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
This has a couple of subtle bugs. First, the generator was creating a call with a parameter '(int *) &(*obj)->type' where type is an enum type; this is unsafe if the compiler chooses to store the enum type in a different size than int, where assigning through the wrong size pointer can corrupt data or cause a SIGBUS. [We still have the casting bug for our enum visitors, but that's a topic for a different patch.]
Quoted 7 lines
Quoted 1 line
I'm not sure I get the last sentence.
I was referring to our casts of enum types to int* inside
visit_type_Enum():
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg03408.htmlIf you have a better wording for it, or want to drop the parenthetical altogether, I'm fine.
What about this: replace the parenthetical with
Related bug, not fixed in this patch: qapi-visit.py's
gen_visit_enum()generates a cast of its enum * argument to int *. Marked FIXME.and squash in
scripts/qapi-visit.py +0
-0
scripts/qapi-visit.py out: · line 176
Quoted 5 lines
scripts/qapi-visit.py +1
-0@@ -172,6 +172,7 @@ out: def gen_visit_enum(name): + # FIXME cast from enum *obj to int * invalidly assumes enum is int return mcgen('''
That works for me.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
PATCH v12 32/36 qapi: Inline _make_implicit_tag()
In reply to Eric BlakeNow that alternates no longer use an implicit tag, we can inline _make_implicit_tag() into its one caller of _def_union_type().
No change to generated code.
---
v12: new patch
--- scripts/qapi.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
scripts/qapi.py +3 -6
@@ -1342,11 +1342,6 @@ class QAPISchema(object): typ, info, 'wrapper', [self._make_member('data', typ, info)]) return QAPISchemaObjectTypeVariant(case, typ) - def _make_implicit_tag(self, type_name, info, variants): - typ = self._make_implicit_enum_type(type_name, info, - [v.name for v in variants]) - return QAPISchemaObjectTypeMember('type', typ, False) - def _def_union_type(self, expr, info): name = expr['union'] data = expr['data']
@@ -1360,7 +1355,9 @@ class QAPISchema(object): else: variants = [self._make_simple_variant(key, value, info) for (key, value) in data.iteritems()] - tag_member = self._make_implicit_tag(name, info, variants) + typ = self._make_implicit_enum_type(name, info, + [v.name for v in variants]) + tag_member = QAPISchemaObjectTypeMember('type', typ, False) members = [tag_member] self._def_entity( QAPISchemaObjectType(name, info, base, members,
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 32/36] qapi: Inline _make_implicit_tag()
Quoted 3 lines · Eric Blake
Now that alternates no longer use an implicit tag, we can inline
_make_implicit_tag()into its one caller of_def_union_type().
Scratch "of".
Quoted 4 lines
No change to generated code.
Suggested-by Markus Armbruster <[email protected]>Signed-off-by Eric Blake <[email protected]>
Patch looks good.
PATCH v12 33/36 qapi: Fix alternates that accept 'number' but not 'int'
In reply to Eric BlakeThe QMP input visitor allows integral values to be assigned by promotion to a QTYPE_QFLOAT. However, when parsing an alternate, we did not take this into account, such that an alternate that accepts 'number' and some other type, but not 'int', would reject integral values.
With this patch, we now have the following desirable table:
alternate has case selected for
'int' 'number' QTYPE_QINT QTYPE_QFLOAT
no no error error
no yes 'number' 'number'
yes no 'int' error
yes yes 'int' 'number'While it is unlikely that we will ever use 'number' in an alternate other than in the testsuite, it never hurts to be more precise in what we allow.
---
v12: rebase to QType cleanups
v11 (no v10): slight commit message tweak, rebase to earlier changes
v9: rebase to earlier changes
v8: no change
v7: rebase to named .u union
v6: rebase onto earlier testsuite and gen_err_check() improvements
--- include/qapi/visitor-impl.h | 2 +- include/qapi/visitor.h | 3 ++- qapi/qapi-visit-core.c | 4 ++-- qapi/qmp-input-visitor.c | 5 ++++- scripts/qapi-visit.py | 11 +++++++---- tests/test-qmp-input-visitor.c | 16 ++++++---------- 6 files changed, 22 insertions(+), 19 deletions(-)
include/qapi/visitor-impl.h +1 -1
@@ -33,7 +33,7 @@ struct Visitorvoid (*type_enum)(Visitor *v,int *obj,const char *const strings[],const char *kind,const char *name,Error **errp);/* May be NULL; only needed for input visitors. */ -void (*get_next_type)(Visitor *v,QType *type, +void (*get_next_type)(Visitor *v,QType *type,bool promote_int,const char *name,Error **errp);void (*type_int)(Visitor *v,int64_t *obj,const char *name,Error **errp);
include/qapi/visitor.h +2 -1
@@ -43,8 +43,9 @@ void visit_optional(Visitor *v, bool *present, const char *name,* Determine the qtype of the item @name in the current object visit. * For input visitors, set *@type to the correct qtype of a qapi * alternate type; for other visitors, leave *@type unchanged. +* If @promote_int, treat integers as QTYPE_FLOAT. */ -void visit_get_next_type (Visitor *v,QType *type, +void visit_get_next_type (Visitor *v,QType *type,bool promote_int,const char *name,Error **errp);void visit_type_enum (Visitor *v,int *obj,const char *const strings[],const char *kind,const char *name,Error **errp);
qapi/qapi-visit-core.c +2 -2
@@ -81,11 +81,11 @@ void visit_optional(Visitor *v, bool *present, const char *name, } } -void visit_get_next_type (Visitor *v,QType *type, +void visit_get_next_type (Visitor *v,QType *type,bool promote_int,const char *name,Error **errp) {if (v->get_next_type) { - v->get_next_type (v, type, name, errp); + v->get_next_type (v, type, promote_int, name, errp); } }
qapi/qmp-input-visitor.c +4 -1
@@ -208,7 +208,7 @@ static void qmp_input_end_list(Visitor *v, Error **errp)qmp_input_pop (qiv, errp); } -static void qmp_input_get_next_type (Visitor *v,QType *type, +static void qmp_input_get_next_type (Visitor *v,QType *type,bool promote_int,const char *name,Error **errp) {QmpInputVisitor *qiv =to_qiv (v);
@@ -219,6 +219,9 @@ static void qmp_input_get_next_type(Visitor *v, QType *type,return ; } *type =qobject_type (qobj); +if (promote_int && *type ==QTYPE_QINT ) { + *type =QTYPE_QFLOAT ; + } } static void qmp_input_type_int(Visitor *v, int64_t *obj, const char *name,
scripts/qapi-visit.py +7 -4
@@ -183,6 +183,11 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s *obj, const char *name, Error def gen_visit_alternate(name, variants): + promote_int = 'true' + for var in variants.variants: + if var.type.alternate_qtype() == 'QTYPE_QINT': + promote_int = 'false' + ret = mcgen(''' void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error **errp)
@@ -193,16 +198,14 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error if (err) { goto out; } - visit_get_next_type(v, &(*obj)->type, name, &err); + visit_get_next_type(v, &(*obj)->type, %(promote_int)s, name, &err); if (err) { goto out_obj; } switch ((*obj)->type) { ''', - c_name=c_name(name)) + c_name=c_name(name), promote_int=promote_int) - # FIXME: When 'number' but not 'int' is present in the alternate, we - # should allow QTYPE_INT to promote to QTYPE_FLOAT. for var in variants.variants: ret += mcgen(''' case %(case)s:
tests/test-qmp-input-visitor.c +6 -10
@@ -347,20 +347,16 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,error_free_or_abort (&err);qapi_free_AltStrBool (asb); -/* FIXME: integer should parse as number */ v =visitor_input_test_init (data,"42" ); -visit_type_AltStrNum (v, &asn,NULL , &err); -/* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */ -/* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */ -error_free_or_abort (&err); +visit_type_AltStrNum (v, &asn,NULL , &error_abort); + g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); + g_assert_cmpfloat(asn->u.n, ==, 42);qapi_free_AltStrNum (asn); -/* FIXME: integer should parse as number */ v =visitor_input_test_init (data,"42" ); -visit_type_AltNumStr (v, &ans,NULL , &err); -/* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */ -/* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */ -error_free_or_abort (&err); +visit_type_AltNumStr (v, &ans,NULL , &error_abort); + g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); + g_assert_cmpfloat(ans->u.n, ==, 42);qapi_free_AltNumStr (ans); v =visitor_input_test_init (data,"42" );
Signature · 1 line
2.4.3
PATCH v12 34/36 qapi: Add positive tests to qapi-schema-test
In reply to Eric BlakeAdd positive tests to qapi-schema-test for things that were made possible by recent patches but which caused compile errors due to collisions prior to that point. The focus is mainly on collisions due to names we have reserved for qapi, even though it is unlikely that anyone will want to abuse these names in actual .json files.
The added tests includes:
Use of a member name ending in 'Kind' or 'List' [1, 3] Use of a type name starting with 'has_' [1, 4] Use of a type named 'u' [1, 5]
Use of a union branch name of 'u' [2, 5]
Use of a union branch name starting with 'has_' [2, 4]
[1] Never broken, but could break if reservations are too strict
[2] Broken prior to commit e4ba22b
[3] See reservations in commit 4dc2e69 and 255960d
[4] See reservations in commit 9fb081e
[5] See reservation in commit 5e59baf
Not worth testing here: we no longer have a collision with a member named 'base' (commit ddf2190), with a branch named 'type' (commit e4ba22b), or with an alternate member named 'max' (previous commits); these names were more accidental namespace pollutions than intentional reservations.
---
v12: no change
v11 (no v10): drop test of 'max'
v9: reorder in series (was 9/17); fewer tests of 'base' and 'type' non-collision; fold in alternate 'max' test; update commit message
v8: new, but collects portions of subset B v10 patches 2, 3, and 16 and subset C v7 patch 6 that were deferred to later.
It might be worth dropping or simplifying this patch, depending on how many corner cases we actually want to test.
--- tests/qapi-schema/qapi-schema-test.json | 16 ++++++++++++++++ tests/qapi-schema/qapi-schema-test.out | 26 ++++++++++++++++++++++++++ 2 files changed, 42 insertions(+)
tests/qapi-schema/qapi-schema-test.json +16 -0
@@ -113,6 +113,22 @@ 'sizes': ['size'], 'any': ['any'] } } +# Even though 'u' and 'has_*' are forbidden as struct member names, they +# should still be valid as a type or union branch name. And although +# '*Kind' and '*List' are forbidden as type names, they should not be +# forbidden as a member or branch name. Flat union branches do not +# collide with base members. +{ 'enum': 'EnumName', 'data': [ 'value1', 'has_a', 'u' ] } +{ 'struct': 'has_a', 'data': { 'MyKind': 'int', 'MyList': ['int'], + 'value1': 'EnumName' } } +{ 'union': 'u', 'data': { 'u': 'uint8', 'myKind': 'has_a', + 'myList': 'has_a', 'has_a': 'has_a' } } +{ 'union': 'UnionName', 'base': 'has_a', 'discriminator': 'value1', + 'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero', + 'u': 'UserDefZero' } } +{ 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool', + 'myKind': 'has_a' } } + # testing commands { 'command': 'user_def_cmd', 'data': {} } { 'command': 'user_def_cmd1', 'data': {'ud1a': 'UserDefOne'} }
tests/qapi-schema/qapi-schema-test.out +26 -0
@@ -22,6 +22,8 @@ object :obj-guest-get-time-arg member b: int optional=True object :obj-guest-sync-arg member arg: any optional=False +object :obj-has_a-wrapper + member data: has_a optional=False object :obj-int16List-wrapper member data: int16List optional=False object :obj-int32List-wrapper
@@ -46,6 +48,8 @@ object :obj-uint32List-wrapper member data: uint32List optional=False object :obj-uint64List-wrapper member data: uint64List optional=False +object :obj-uint8-wrapper + member data: uint8 optional=False object :obj-uint8List-wrapper member data: uint8List optional=False object :obj-user_def_cmd1-arg
@@ -56,6 +60,10 @@ object :obj-user_def_cmd2-arg alternate AltIntNum case i: int case n: number +alternate AltName + case type: int + case u: bool + case myKind: has_a alternate AltNumInt case n: number case i: int
@@ -78,6 +86,7 @@ event EVENT_D :obj-EVENT_D-arg object Empty1 object Empty2 base Empty1 +enum EnumName ['value1', 'has_a', 'u'] enum EnumOne ['value1', 'value2', 'value3'] object EventStructOne member struct1: UserDefOne optional=False
@@ -101,6 +110,12 @@ object TestStruct member integer: int optional=False member boolean: bool optional=False member string: str optional=False +object UnionName + base has_a + tag value1 + case value1: UserDefZero + case has_a: UserDefZero + case u: UserDefZero object UserDefA member boolean: bool optional=False member a_b: int optional=True
@@ -196,6 +211,17 @@ command guest-get-time :obj-guest-get-time-arg -> int gen=True success_response=True command guest-sync :obj-guest-sync-arg -> any gen=True success_response=True +object has_a + member MyKind: int optional=False + member MyList: intList optional=False + member value1: EnumName optional=False +object u + member type: uKind optional=False + case u: :obj-uint8-wrapper + case myKind: :obj-has_a-wrapper + case myList: :obj-has_a-wrapper + case has_a: :obj-has_a-wrapper +enum uKind ['u', 'myKind', 'myList', 'has_a'] command user_def_cmd None -> None gen=True success_response=True command user_def_cmd1 :obj-user_def_cmd1-arg -> None
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 34/36] qapi: Add positive tests to qapi-schema-test
Quoted 9 lines · Eric Blake
Add positive tests to qapi-schema-test for things that were made possible by recent patches but which caused compile errors due to collisions prior to that point. The focus is mainly on collisions due to names we have reserved for qapi, even though it is unlikely that anyone will want to abuse these names in actual .json files.
The added tests includes:
Use of a member name ending in 'Kind' or 'List' [1, 3]
Violates our coding convention of all lower case, so not worth testing.
Quoted 1 line
Use of a type name starting with '
has_' [1, 4]
Ditto.
Quoted 1 line
Use of a type named 'u' [1, 5]
Ditto.
Quoted 2 lines
Use of a union branch name of 'u' [2, 5]
Use of a union branch name starting with 'has_' [2, 4]
Leaving just these two feels pretty weak. I think I'll just drop this patch from my queue entirely (it was great at proving that recent layout improvements worked, but it's not the end of the world to lose it entirely, especially now that v13 is starting to enforce naming conventions except for a few whitelisted names, and I don't want to make the whitelist longer just for the testsuite).
Quoted 15 lines
[1] Never broken, but could break if reservations are too strict
[2] Broken prior to commit e4ba22b
[3] See reservations in commit 4dc2e69 and 255960d
[4] See reservations in commit 9fb081e
[5] See reservation in commit 5e59bafNot worth testing here: we no longer have a collision with a member named 'base' (commit ddf2190), with a branch named 'type' (commit e4ba22b), or with an alternate member named 'max' (previous commits); these names were more accidental namespace pollutions than intentional reservations.
Signed-off-by Eric Blake <[email protected]>
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v12 34/36] qapi: Add positive tests to qapi-schema-test
Quoted 24 lines · Eric Blake
Quoted 9 lines · Eric Blake
Add positive tests to qapi-schema-test for things that were made possible by recent patches but which caused compile errors due to collisions prior to that point. The focus is mainly on collisions due to names we have reserved for qapi, even though it is unlikely that anyone will want to abuse these names in actual .json files.
The added tests includes:
Use of a member name ending in 'Kind' or 'List' [1, 3]Violates our coding convention of all lower case, so not worth testing.
Quoted 1 line
Use of a type name starting with '
has_' [1, 4]Ditto.
Quoted 1 line
Use of a type named 'u' [1, 5]
Ditto.
Quoted 2 lines
Use of a union branch name of 'u' [2, 5]
Use of a union branch name starting with 'has_' [2, 4]
Quoted 6 lines
Leaving just these two feels pretty weak. I think I'll just drop this patch from my queue entirely (it was great at proving that recent layout improvements worked, but it's not the end of the world to lose it entirely, especially now that v13 is starting to enforce naming conventions except for a few whitelisted names, and I don't want to make the whitelist longer just for the testsuite).
Makes sense.
[...]
PATCH v12 35/36 qapi: Simplify visits of optional fields
In reply to Eric BlakeNone of the visitor callbacks would set an error when testing if an optional field was present; make this part of the interface contract by eliminating the errp argument.
The resulting generated code has a nice diff:
|- visit_optional(v, &has_fdset_id, "fdset-id", &err);
|- if (err) {
|- goto out;
|- }
|- if (has_fdset_id) {
|+ visit_optional(v, &has_fdset_id, "fdset-id");
|+ if (has_fdset_id) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }---
v12: split error removal from 'if' compression
v11 (no v10): no change
v9: no change
v8: no change
v7: rebase to no member.c_name()
v6: rebase onto earlier testsuite and gen_err_check() improvements
--- include/qapi/visitor-impl.h | 5 ++--- include/qapi/visitor.h | 10 ++++++++-- qapi/opts-visitor.c | 2 +- qapi/qapi-visit-core.c | 5 ++--- qapi/qmp-input-visitor.c | 3 +-- qapi/string-input-visitor.c | 3 +-- scripts/qapi.py | 8 ++------ 7 files changed, 17 insertions(+), 19 deletions(-)
include/qapi/visitor-impl.h +2 -3
@@ -44,9 +44,8 @@ struct Visitorvoid (*type_any)(Visitor *v,QObject **obj,const char *name,Error **errp); -/* May be NULL */ -void (*optional)(Visitor *v,bool *present,const char *name, -Error **errp); +/* May be NULL; most useful for input visitors. */ +void (*optional)(Visitor *v,bool *present,const char *name);void (*type_uint8)(Visitor *v,uint8_t *obj,const char *name,Error **errp);void (*type_uint16)(Visitor *v,uint16_t *obj,const char *name,Error **errp);
include/qapi/visitor.h +8 -2
@@ -36,8 +36,14 @@ void visit_end_implicit_struct(Visitor *v, Error **errp);void visit_start_list (Visitor *v,const char *name,Error **errp);GenericList *visit_next_list (Visitor *v,GenericList **list,Error **errp);void visit_end_list (Visitor *v,Error **errp); -void visit_optional (Visitor *v,bool *present,const char *name, -Error **errp); + +/** +* Check if an optional member @name of an object needs visiting. +* For input visitors, set *@present according to whether the +* corresponding visit_type_*() needs calling; for other visitors, +* leave *@present unchanged. +*/ +void visit_optional (Visitor *v,bool *present,const char *name);/** * Determine the qtype of the item @name in the current object visit.
qapi/opts-visitor.c +1 -1
@@ -488,7 +488,7 @@ opts_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp)static void -opts_optional (Visitor *v,bool *present,const char *name,Error **errp) +opts_optional (Visitor *v,bool *present,const char *name) {OptsVisitor *ov =DO_UPCAST (OptsVisitor, visitor, v);
qapi/qapi-visit-core.c +2 -3
@@ -73,11 +73,10 @@ void visit_end_union(Visitor *v, bool data_present, Error **errp) } } -void visit_optional (Visitor *v,bool *present,const char *name, -Error **errp) +void visit_optional (Visitor *v,bool *present,const char *name) {if (v->optional) { - v->optional (v, present, name, errp); + v->optional (v, present, name); } }
qapi/qmp-input-visitor.c +1 -2
@@ -303,8 +303,7 @@ static void qmp_input_type_any(Visitor *v, QObject **obj, const char *name, *obj = qobj; } -static void qmp_input_optional (Visitor *v,bool *present,const char *name, -Error **errp) +static void qmp_input_optional (Visitor *v,bool *present,const char *name) {QmpInputVisitor *qiv =to_qiv (v);QObject *qobj =qmp_input_get_object (qiv, name, true);
qapi/string-input-visitor.c +1 -2
@@ -299,8 +299,7 @@ static void parse_type_number(Visitor *v, double *obj, const char *name, *obj = val; } -static void parse_optional (Visitor *v,bool *present,const char *name, -Error **errp) +static void parse_optional (Visitor *v,bool *present,const char *name) {StringInputVisitor *siv =DO_UPCAST (StringInputVisitor, visitor, v);
scripts/qapi.py +2 -6
@@ -1680,15 +1680,11 @@ def gen_visit_fields(members, prefix='', need_cast=False, skiperr=False): for memb in members: if memb.optional: ret += mcgen(''' - visit_optional(v, &%(prefix)shas_%(c_name)s, "%(name)s", %(errp)s); + visit_optional(v, &%(prefix)shas_%(c_name)s, "%(name)s"); + if (%(prefix)shas_%(c_name)s) { ''', prefix=prefix, c_name=c_name(memb.name), name=memb.name, errp=errparg) - ret += gen_err_check(skiperr=skiperr) - ret += mcgen(''' - if (%(prefix)shas_%(c_name)s) { -''', - prefix=prefix, c_name=c_name(memb.name)) push_indent() # Ugly: sometimes we need to cast away const
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 35/36] qapi: Simplify visits of optional fields
Quoted 6 lines · Eric Blake
None of the visitor callbacks would set an error when testing if an optional field was present; make this part of the interface contract by eliminating the errp argument.
The resulting generated code has a nice diff:
Quoted 12 lines
|- visit_optional(v, &has_fdset_id, "fdset-id", &err);
|- if (err) {
|- goto out;
|- }
|- if (has_fdset_id) {
|+ visit_optional(v, &has_fdset_id, "fdset-id");
|+ if (has_fdset_id) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }I think this should be
|- visit_optional(v, &has_fdset_id, "fdset-id", &err);
|- if (err) {
|- goto out;
|- }
|+ visit_optional(v, &has_fdset_id, "fdset-id");
| if (has_fdset_id) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }Quoted 2 lines
Signed-off-by Eric Blake <[email protected]>
Patch looks good.
Re: [Qemu-devel] [PATCH v12 35/36] qapi: Simplify visits of optional fields
Quoted 21 lines · Markus Armbruster
Quoted 6 lines · Eric Blake
None of the visitor callbacks would set an error when testing if an optional field was present; make this part of the interface contract by eliminating the errp argument.
The resulting generated code has a nice diff:
Quoted 12 lines
|- visit_optional(v, &has_fdset_id, "fdset-id", &err); |- if (err) { |- goto out; |- } |- if (has_fdset_id) { |+ visit_optional(v, &has_fdset_id, "fdset-id"); |+ if (has_fdset_id) { | visit_type_int(v, &fdset_id, "fdset-id", &err); | if (err) { | goto out; | } | }
Quoted 8 lines
I think this should be
|- visit_optional(v, &has_fdset_id, "fdset-id", &err); |- if (err) { |- goto out; |- } |+ visit_optional(v, &has_fdset_id, "fdset-id"); | if (has_fdset_id) {
Oh, yeah. Serves me right for hand-editing a diff in my commit message.
(Patch applies either diff just fine, it's just that mine was longer
than necessary).
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
PATCH v12 36/36 qapi: Shorter visits of optional fields
In reply to Eric BlakeFor less code, reflect the determined boolean value of an optional visit back to the caller instead of making the caller read the boolean after the fact.
The resulting generated code has the following diff:
|- visit_optional(v, &has_fdset_id, "fdset-id");
|- if (has_fdset_id) {
|+ if (visit_optional(v, &has_fdset_id, "fdset-id")) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }---
v12: split error removal from 'if' compression
v11 (no v10): no change
v9: no change
v8: no change
v7: rebase to no member.c_name()
v6: rebase onto earlier testsuite and gen_err_check() improvements
--- include/qapi/visitor.h | 4 ++-- qapi/qapi-visit-core.c | 3 ++- scripts/qapi.py | 3 +-- 3 files changed, 5 insertions(+), 5 deletions(-)
include/qapi/visitor.h +2 -2
@@ -41,9 +41,9 @@ void visit_end_list(Visitor *v, Error **errp);* Check if an optional member @name of an object needs visiting. * For input visitors, set *@present according to whether the * corresponding visit_type_*() needs calling; for other visitors, -* leave *@present unchanged. +* leave *@present unchanged. Return *@present for convenience. */ -void visit_optional (Visitor *v,bool *present,const char *name); +bool visit_optional (Visitor *v,bool *present,const char *name);/** * Determine the qtype of the item @name in the current object visit.
qapi/qapi-visit-core.c +2 -1
@@ -73,11 +73,12 @@ void visit_end_union(Visitor *v, bool data_present, Error **errp) } } -void visit_optional (Visitor *v,bool *present,const char *name) +bool visit_optional (Visitor *v,bool *present,const char *name) {if (v->optional) { v->optional (v, present, name); } +return *present; } void visit_get_next_type(Visitor *v, QType *type, bool promote_int,
scripts/qapi.py +1 -2
@@ -1680,8 +1680,7 @@ def gen_visit_fields(members, prefix='', need_cast=False, skiperr=False): for memb in members: if memb.optional: ret += mcgen(''' - visit_optional(v, &%(prefix)shas_%(c_name)s, "%(name)s"); - if (%(prefix)shas_%(c_name)s) { + if (visit_optional(v, &%(prefix)shas_%(c_name)s, "%(name)s")) { ''', prefix=prefix, c_name=c_name(memb.name), name=memb.name, errp=errparg)
Signature · 1 line
2.4.3
Re: [Qemu-devel] [PATCH v12 36/36] qapi: Shorter visits of optional fields
Quoted 15 lines · Eric Blake
For less code, reflect the determined boolean value of an optional visit back to the caller instead of making the caller read the boolean after the fact.
The resulting generated code has the following diff:
|- visit_optional(v, &has_fdset_id, "fdset-id"); |- if (has_fdset_id) { |+ if (visit_optional(v, &has_fdset_id, "fdset-id")) { | visit_type_int(v, &fdset_id, "fdset-id", &err); | if (err) { | goto out; | } | }
Quoted 1 line
Signed-off-by Eric Blake <[email protected]>
Feels like a wash to me, but I'm willing to take it anyway, in recognition of the massive amount of work you've been doing in this area :)
Feels close. The only issues in need of discussion are in PATCH 27.
I expect the fixup to PATCH 21 to be straightforward. I'd do PATCH 28 a bit different, but it works as is. PATCH 30 could perhaps be simplified slightly. The rest are just commit message polish.
Quoted 5 lines · Markus Armbruster
Feels close. The only issues in need of discussion are in PATCH 27.
I expect the fixup to PATCH 21 to be straightforward. I'd do PATCH 28 a bit different, but it works as is. PATCH 30 could perhaps be simplified slightly. The rest are just commit message polish.
Fixups proposed to 21, 27, 28, and 30; at this point, I'll be happy to review whatever you post on your staging tree to make sure you caught all the commit message polish you suggested, and we can probably avoid the need for a (lucky?) v13.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org