Skip to contentSkip to conversation

qapi: Doc generation fixes

PATCH New Markus Armbruster 1 participant 20 messages Oct 24, 2019

lore
Share & tools
lore
#0 lore raw

cover qapi: Doc generation fixes

The recent patches to introduce feature flags neglected doc generation. While fixing that, I found similar neglect in non-recent patches. Fix all that, and clean up some.

Markus Armbruster (19):
tests/qapi-schema: Demonstrate feature and enum doc comment bugs tests/qapi-schema: Demonstrate command and event doc comment bugs tests/qapi-schema: Cover alternate documentation comments tests/qapi-schema: Fix feature documentation testing
qemu-doc: Belatedly document QMP command deprecation
qapi: Implement boxed event argument documentation
qapi: De-duplicate entity documentation generation code
qapi: Split .connect_doc(), .check_doc() off .check()
qapi: Fix enum doc comment checking
qapi: Clean up doc comment checking for implicit union base
qapi: Fix doc comment checking for commands and events
qapi: Simplify ._make_implicit_object_type()
qapi: Eliminate .check_doc() overrides
qapi: Fold normalize_if() into check_if()
qapi: Fold normalize_features() into check_features()
qapi: Fold normalize_enum() into check_enum()
qapi: Lift features into QAPISchemaEntity
qapi: Polish reporting of bogus member documentation
qapi: Check feature documentation against the schema

 qemu-deprecated.texi                          | 12 +++
 tests/qapi-schema/doc-good.texi               | 27 ++++++
 qapi/net.json                                 |  2 -
 scripts/qapi/doc.py                           | 75 +++++++-------
 scripts/qapi/expr.py                          | 32 +-----
 scripts/qapi/parser.py                        | 29 ++++--
 scripts/qapi/schema.py                        | 97 +++++++++++--------
 tests/Makefile.include                        |  5 +
 .../qapi-schema/doc-bad-alternate-member.err  |  2 +-
 .../qapi-schema/doc-bad-boxed-command-arg.err |  1 +
 .../doc-bad-boxed-command-arg.json            | 14 +++
 .../qapi-schema/doc-bad-boxed-command-arg.out |  0
 tests/qapi-schema/doc-bad-command-arg.err     |  2 +-
 tests/qapi-schema/doc-bad-enum-member.err     |  1 +
 tests/qapi-schema/doc-bad-enum-member.json    |  8 ++
 tests/qapi-schema/doc-bad-enum-member.out     |  0
 tests/qapi-schema/doc-bad-event-arg.err       |  1 +
 tests/qapi-schema/doc-bad-event-arg.json      |  7 ++
 tests/qapi-schema/doc-bad-event-arg.out       |  0
 tests/qapi-schema/doc-bad-feature.err         |  1 +
 tests/qapi-schema/doc-bad-feature.json        |  9 ++
 tests/qapi-schema/doc-bad-feature.out         |  0
 tests/qapi-schema/doc-bad-union-member.err    |  2 +-
 tests/qapi-schema/doc-good.json               | 14 +++
 tests/qapi-schema/doc-good.out                | 27 ++++++
 tests/qapi-schema/doc-undoc-feature.err       |  2 +
 tests/qapi-schema/doc-undoc-feature.json      |  9 ++
 tests/qapi-schema/doc-undoc-feature.out       |  0
 tests/qapi-schema/test-qapi.py                |  2 +
 29 files changed, 270 insertions(+), 111 deletions(-)
 create mode 100644 tests/qapi-schema/doc-bad-boxed-command-arg.err
 create mode 100644 tests/qapi-schema/doc-bad-boxed-command-arg.json
 create mode 100644 tests/qapi-schema/doc-bad-boxed-command-arg.out
 create mode 100644 tests/qapi-schema/doc-bad-enum-member.err
 create mode 100644 tests/qapi-schema/doc-bad-enum-member.json
 create mode 100644 tests/qapi-schema/doc-bad-enum-member.out
 create mode 100644 tests/qapi-schema/doc-bad-event-arg.err
 create mode 100644 tests/qapi-schema/doc-bad-event-arg.json
 create mode 100644 tests/qapi-schema/doc-bad-event-arg.out
 create mode 100644 tests/qapi-schema/doc-bad-feature.err
 create mode 100644 tests/qapi-schema/doc-bad-feature.json
 create mode 100644 tests/qapi-schema/doc-bad-feature.out
 create mode 100644 tests/qapi-schema/doc-undoc-feature.err
 create mode 100644 tests/qapi-schema/doc-undoc-feature.json
 create mode 100644 tests/qapi-schema/doc-undoc-feature.out
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/19 tests/qapi-schema: Demonstrate feature and enum doc comment bugs

Add negative tests doc-bad-enum-member and doc-bad-feature to cover documentation for nonexistent enum members and features, and test doc-undoc-feature to cover features lacking documentation. None of them works. To be fixed later in this series.

---
 tests/Makefile.include                     |  3 +++
 tests/qapi-schema/doc-bad-enum-member.err  |  0
 tests/qapi-schema/doc-bad-enum-member.json |  9 +++++++++
 tests/qapi-schema/doc-bad-enum-member.out  | 21 +++++++++++++++++++++
 tests/qapi-schema/doc-bad-feature.err      |  0
 tests/qapi-schema/doc-bad-feature.json     | 10 ++++++++++
 tests/qapi-schema/doc-bad-feature.out      | 17 +++++++++++++++++
 tests/qapi-schema/doc-undoc-feature.err    |  0
 tests/qapi-schema/doc-undoc-feature.json   | 10 ++++++++++
 tests/qapi-schema/doc-undoc-feature.out    | 19 +++++++++++++++++++
 10 files changed, 89 insertions(+)
 create mode 100644 tests/qapi-schema/doc-bad-enum-member.err
 create mode 100644 tests/qapi-schema/doc-bad-enum-member.json
 create mode 100644 tests/qapi-schema/doc-bad-enum-member.out
 create mode 100644 tests/qapi-schema/doc-bad-feature.err
 create mode 100644 tests/qapi-schema/doc-bad-feature.json
 create mode 100644 tests/qapi-schema/doc-bad-feature.out
 create mode 100644 tests/qapi-schema/doc-undoc-feature.err
 create mode 100644 tests/qapi-schema/doc-undoc-feature.json
 create mode 100644 tests/qapi-schema/doc-undoc-feature.out
tests/Makefile.include +3 -0
index 09e5b410dc..b9deb131fb 100644
@@ -342,6 +342,8 @@ qapi-schema += command-int.json
 qapi-schema += comments.json
 qapi-schema += doc-bad-alternate-member.json
 qapi-schema += doc-bad-command-arg.json
+qapi-schema += doc-bad-enum-member.json
+qapi-schema += doc-bad-feature.json
 qapi-schema += doc-bad-section.json
 qapi-schema += doc-bad-symbol.json
 qapi-schema += doc-bad-union-member.json
@@ -365,6 +367,7 @@ qapi-schema += doc-missing-expr.json
 qapi-schema += doc-missing-space.json
 qapi-schema += doc-missing.json
 qapi-schema += doc-no-symbol.json
+qapi-schema += doc-undoc-feature.json
 qapi-schema += double-type.json
 qapi-schema += duplicate-key.json
 qapi-schema += empty.json
tests/qapi-schema/doc-bad-enum-member.err new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
tests/qapi-schema/doc-bad-enum-member.json new file +9 -0
new file mode 100644
index 0000000000..9f32fe64b4
@@ -0,0 +1,9 @@
+# Members listed in the doc comment must exist in the actual schema
+# BUG: nonexistent @a is not rejected
+
+##
+# @Foo:
+# @a: a
+# @b: b
+##
+{ 'enum': 'Foo', 'data': [ 'b' ] }
tests/qapi-schema/doc-bad-enum-member.out new file +21 -0
new file mode 100644
index 0000000000..6ca31c1e9b
@@ -0,0 +1,21 @@
+module None
+object q_empty
+enum QType
+    prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
+module doc-bad-enum-member.json
+enum Foo
+    member b
+doc symbol=Foo
+    body=
+
+    arg=a
+a
+    arg=b
+b
tests/qapi-schema/doc-bad-feature.err new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
tests/qapi-schema/doc-bad-feature.json new file +10 -0
new file mode 100644
index 0000000000..2a78e3b1db
@@ -0,0 +1,10 @@
+# Features listed in the doc comment must exist in the actual schema
+# BUG: nonexistent @a is not rejected
+
+##
+# @foo:
+#
+# Features:
+# @a: a
+##
+{ 'command': 'foo' }
tests/qapi-schema/doc-bad-feature.out new file +17 -0
new file mode 100644
index 0000000000..4670be3a72
@@ -0,0 +1,17 @@
+module None
+object q_empty
+enum QType
+    prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
+module doc-bad-feature.json
+command foo None -> None
+    gen=True success_response=True boxed=False oob=False preconfig=False
+doc symbol=foo
+    body=
+
tests/qapi-schema/doc-undoc-feature.err new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
tests/qapi-schema/doc-undoc-feature.json new file +10 -0
new file mode 100644
index 0000000000..c7650d9974
@@ -0,0 +1,10 @@
+# Doc comment must cover all features
+# BUG: missing documentation for @undoc not caught
+
+##
+# @foo:
+#
+# Features:
+# @doc: documented feature
+##
+{ 'command': 'foo', 'features': ['undoc', 'doc'] }
tests/qapi-schema/doc-undoc-feature.out new file +19 -0
new file mode 100644
index 0000000000..faae777f0d
@@ -0,0 +1,19 @@
+module None
+object q_empty
+enum QType
+    prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
+module doc-undoc-feature.json
+command foo None -> None
+    gen=True success_response=True boxed=False oob=False preconfig=False
+    feature undoc
+    feature doc
+doc symbol=foo
+    body=
+
Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/19 tests/qapi-schema: Demonstrate command and event doc comment bugs

In reply to Markus Armbruster

Add negative tests doc-bad-boxed-command-arg and doc-bad-event-arg to cover boxed and no arguments. They demonstrate insufficient doc comment checking.

Update positive test doc-good to cover boxed event arguments. It demonstrates the generated doc comment misses arguments.

These bugs will be fixed later in this series.

---
 tests/qapi-schema/doc-good.texi               |  8 ++++++
 tests/Makefile.include                        |  2 ++
 .../qapi-schema/doc-bad-boxed-command-arg.err |  0
 .../doc-bad-boxed-command-arg.json            | 15 +++++++++++
 .../qapi-schema/doc-bad-boxed-command-arg.out | 26 +++++++++++++++++++
 tests/qapi-schema/doc-bad-event-arg.err       |  0
 tests/qapi-schema/doc-bad-event-arg.json      |  8 ++++++
 tests/qapi-schema/doc-bad-event-arg.out       | 19 ++++++++++++++
 tests/qapi-schema/doc-good.json               |  7 +++++
 tests/qapi-schema/doc-good.out                |  5 ++++
 10 files changed, 90 insertions(+)
 create mode 100644 tests/qapi-schema/doc-bad-boxed-command-arg.err
 create mode 100644 tests/qapi-schema/doc-bad-boxed-command-arg.json
 create mode 100644 tests/qapi-schema/doc-bad-boxed-command-arg.out
 create mode 100644 tests/qapi-schema/doc-bad-event-arg.err
 create mode 100644 tests/qapi-schema/doc-bad-event-arg.json
 create mode 100644 tests/qapi-schema/doc-bad-event-arg.out
tests/qapi-schema/doc-good.texi +8 -0
index 2ce8b883c9..d6ecdac94f 100644
@@ -258,3 +258,11 @@ another feature
 
 @end deftypefn
 
+
+
+@deftypefn Event {} EVT-BOXED
+
+BUG: generated doc misses arguments
+
+@end deftypefn
+
tests/Makefile.include +2 -0
index b9deb131fb..ea35cd54cc 100644
@@ -341,8 +341,10 @@ qapi-schema += base-cycle-indirect.json
 qapi-schema += command-int.json
 qapi-schema += comments.json
 qapi-schema += doc-bad-alternate-member.json
+qapi-schema += doc-bad-boxed-command-arg.json
 qapi-schema += doc-bad-command-arg.json
 qapi-schema += doc-bad-enum-member.json
+qapi-schema += doc-bad-event-arg.json
 qapi-schema += doc-bad-feature.json
 qapi-schema += doc-bad-section.json
 qapi-schema += doc-bad-symbol.json
tests/qapi-schema/doc-bad-boxed-command-arg.err new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
tests/qapi-schema/doc-bad-boxed-command-arg.json new file +15 -0
new file mode 100644
index 0000000000..2c265d2ca3
@@ -0,0 +1,15 @@
+# Boxed arguments are not to be documented with the command
+# BUG: not rejected
+
+##
+# @Args:
+# @a: an argument
+##
+{ 'struct': 'Args', 'data': { 'a': 'int' } }
+
+##
+# @cmd-boxed:
+# @a: bogus
+##
+{ 'command': 'cmd-boxed', 'boxed': true,
+  'data': 'Args' }
tests/qapi-schema/doc-bad-boxed-command-arg.out new file +26 -0
new file mode 100644
index 0000000000..4ccd788253
@@ -0,0 +1,26 @@
+module None
+object q_empty
+enum QType
+    prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
+module doc-bad-boxed-command-arg.json
+object Args
+    member a: int optional=False
+command cmd-boxed Args -> None
+    gen=True success_response=True boxed=True oob=False preconfig=False
+doc symbol=Args
+    body=
+
+    arg=a
+an argument
+doc symbol=cmd-boxed
+    body=
+
+    arg=a
+bogus
tests/qapi-schema/doc-bad-event-arg.err new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
tests/qapi-schema/doc-bad-event-arg.json new file +8 -0
new file mode 100644
index 0000000000..80d4e1240b
@@ -0,0 +1,8 @@
+# Arguments listed in the doc comment must exist in the actual schema
+# BUG: nonexistent @a is not rejected
+
+##
+# @FOO:
+# @a: a
+##
+{ 'event': 'FOO' }
tests/qapi-schema/doc-bad-event-arg.out new file +19 -0
new file mode 100644
index 0000000000..ad0367cd45
@@ -0,0 +1,19 @@
+module None
+object q_empty
+enum QType
+    prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
+module doc-bad-event-arg.json
+event FOO None
+    boxed=False
+doc symbol=FOO
+    body=
+
+    arg=a
+a
tests/qapi-schema/doc-good.json +7 -0
index 7dc21e58a3..1ae6c3a9da 100644
@@ -149,3 +149,10 @@
 { 'command': 'cmd-boxed', 'boxed': true,
   'data': 'Object',
   'features': [ 'cmd-feat1', 'cmd-feat2' ] }
+
+##
+# @EVT-BOXED:
+# BUG: generated doc misses arguments
+##
+{ 'event': 'EVT-BOXED',  'boxed': true,
+  'data': 'Object' }
tests/qapi-schema/doc-good.out +5 -0
index f78fdef6a9..a8fc39d288 100644
@@ -54,6 +54,8 @@ command cmd-boxed Object -> None
     gen=True success_response=True boxed=True oob=False preconfig=False
     feature cmd-feat1
     feature cmd-feat2
+event EVT-BOXED Object
+    boxed=True
 doc freeform
     body=
 = Section
@@ -170,3 +172,6 @@ If you're bored enough to read this, go see a video of boxed cats
 -> in
 
 <- out
+doc symbol=EVT-BOXED
+    body=
+BUG: generated doc misses arguments
Signature · 1 line
2.21.0
#3 lore raw

PATCH 3/19 tests/qapi-schema: Cover alternate documentation comments

In reply to Markus Armbruster
---
 tests/qapi-schema/doc-good.texi | 17 +++++++++++++++++
 tests/qapi-schema/doc-good.json |  8 ++++++++
 tests/qapi-schema/doc-good.out  | 12 ++++++++++++
 3 files changed, 37 insertions(+)
tests/qapi-schema/doc-good.texi +17 -0
index d6ecdac94f..98aa78e1fb 100644
@@ -170,6 +170,23 @@ One of @t{"one"}, @t{"two"}
 @end deftp
 
 
+
+@deftp {Alternate} Alternate
+
+
+
+@b{Members:}
+@table @asis
+@item @code{i: int}
+an integer
+@code{b} is undocumented
+@item @code{b: boolean}
+Not documented
+@end table
+
+@end deftp
+
+
 @subsection Another subsection
 
 
tests/qapi-schema/doc-good.json +8 -0
index 1ae6c3a9da..df50a877e3 100644
@@ -98,6 +98,14 @@
 { 'union': 'SugaredUnion',
   'data': { 'one': 'Variant1', 'two': { 'type': 'Variant2', 'if': 'IFTWO' } } }
 
+##
+# @Alternate:
+# @i: an integer
+# @b is undocumented
+##
+{ 'alternate': 'Alternate',
+  'data': { 'i': 'int', 'b': 'bool' } }
+
 ##
 # == Another subsection
 ##
tests/qapi-schema/doc-good.out +12 -0
index a8fc39d288..b0e99f2110 100644
@@ -42,6 +42,10 @@ object SugaredUnion
     case one: q_obj_Variant1-wrapper
     case two: q_obj_Variant2-wrapper
         if ['IFTWO']
+alternate Alternate
+    tag type
+    case i: int
+    case b: bool
 object q_obj_cmd-arg
     member arg1: int optional=False
     member arg2: str optional=True
@@ -133,6 +137,14 @@ doc symbol=SugaredUnion
 
     arg=type
 
+doc symbol=Alternate
+    body=
+
+    arg=i
+an integer
+@b is undocumented
+    arg=b
+
 doc freeform
     body=
 == Another subsection
Signature · 1 line
2.21.0
#4 lore raw

PATCH 4/19 tests/qapi-schema: Fix feature documentation testing

In reply to Markus Armbruster

Commit 8aa3a33e44 "tests/qapi-schema: Test for good feature lists in structs" made test-qapi.py show features, but neglected to show their documentation. Fix that.

---
 tests/qapi-schema/doc-bad-feature.out   |  2 ++
 tests/qapi-schema/doc-good.out          | 10 ++++++++++
 tests/qapi-schema/doc-undoc-feature.out |  2 ++
 tests/qapi-schema/test-qapi.py          |  2 ++
 4 files changed, 16 insertions(+)
tests/qapi-schema/doc-bad-feature.out +2 -0
index 4670be3a72..fef4a3e400 100644
@@ -15,3 +15,5 @@ command foo None -> None
 doc symbol=foo
     body=
 
+    feature=a
+a
tests/qapi-schema/doc-good.out +10 -0
index b0e99f2110..8cc29fce50 100644
@@ -126,6 +126,8 @@ A paragraph
 Another paragraph (but no @var: line)
     arg=var1
 
+    feature=variant1-feat
+a feature
 doc symbol=Variant2
     body=
 
@@ -158,6 +160,10 @@ the second
 argument
     arg=arg3
 
+    feature=cmd-feat1
+a feature
+    feature=cmd-feat2
+another feature
     section=Note
 @arg3 is undocumented
     section=Returns
@@ -180,6 +186,10 @@ Duis aute irure dolor
 doc symbol=cmd-boxed
     body=
 If you're bored enough to read this, go see a video of boxed cats
+    feature=cmd-feat1
+a feature
+    feature=cmd-feat2
+another feature
     section=Example
 -> in
 
tests/qapi-schema/doc-undoc-feature.out +2 -0
index faae777f0d..cdb097361f 100644
@@ -17,3 +17,5 @@ command foo None -> None
 doc symbol=foo
     body=
 
+    feature=doc
+documented feature
tests/qapi-schema/test-qapi.py +2 -0
index 2bd9fd8742..bad14edb47 100755
@@ -117,6 +117,8 @@ def test_frontend(fname):
         print('    body=\n%s' % doc.body.text)
         for arg, section in doc.args.items():
             print('    arg=%s\n%s' % (arg, section.text))
+        for feat, section in doc.features.items():
+            print('    feature=%s\n%s' % (feat, section.text))
         for section in doc.sections:
             print('    section=%s\n%s' % (section.name, section.text))
 
Signature · 1 line
2.21.0
#5 lore raw

PATCH 5/19 qemu-doc: Belatedly document QMP command deprecation

In reply to Markus Armbruster

Commit 24fb413300 "qmp: Introduce blockdev-change-medium" (v2.5.0) deprecated change.

Commit 2ff3025797 "migrate: move max-bandwidth and downtime-limit to migrate_set_parameter" (v2.8.0) deprecated migrate_set_downtime and migrate_set_speed.

These wre missed in commit eb22aeca65 "docs: document deprecation policy & deprecated features in appendix" (v2.10.0).

Commit 73af8dd8d7 "migration: Make xbzrle_cache_size a migration parameter" (v2.11.0) deprecated migrate-set-cache-size and query-migrate-cache-size, but neglected to update qemu-doc.

Make up for that.

---
 qemu-deprecated.texi | 12 ++++++++++++
 1 file changed, 12 insertions(+)
qemu-deprecated.texi +12 -0
index 7239e0959d..f727bd3932 100644
@@ -149,6 +149,18 @@ QEMU 4.1 has three options, please migrate to one of these three:
 
 @section QEMU Machine Protocol (QMP) commands
 
+@subsection change (since 2.5.0)
+
+Use ``blockdev-change-medium'' or ``change-vnc-password'' instead.
+
+@subsection migrate_set_downtime and migrate_set_speed (since 2.8.0)
+
+Use ``migrate-set-parameters'' instead.
+
+@subsection migrate-set-cache-size and query-migrate-cache-size (since 2.11.0)
+
+Use ``migrate-set-parameters'' and ``query-migrate-parameters'' instead.
+
 @subsection query-block result field dirty-bitmaps[i].status (since 4.0)
 
 The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
Signature · 1 line
2.21.0
#6 lore raw

PATCH 6/19 qapi: Implement boxed event argument documentation

In reply to Markus Armbruster

Generate a reference "Arguments: the members of ...", just like we do for commands since commit c2dd311cb7 "qapi2texi: Implement boxed argument documentation".

No change to generated QMP documentation; we don't yet use boxed events outside tests/.

---
 tests/qapi-schema/doc-good.texi |  4 +++-
 scripts/qapi/doc.py             | 10 +++++++++-
 tests/qapi-schema/doc-good.json |  1 -
 tests/qapi-schema/doc-good.out  |  2 +-
 4 files changed, 13 insertions(+), 4 deletions(-)
tests/qapi-schema/doc-good.texi +3 -1
index 98aa78e1fb..d4b15dabf0 100644
@@ -279,7 +279,9 @@ another feature
 
 @deftypefn Event {} EVT-BOXED
 
-BUG: generated doc misses arguments
+
+
+@b{Arguments:} the members of @code{Object}
 
 @end deftypefn
 
scripts/qapi/doc.py +9 -1
index 6d5726cf6e..f2462c9877 100644
@@ -266,9 +266,17 @@ class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
 
     def visit_event(self, name, info, ifcond, arg_type, boxed):
         doc = self.cur_doc
+        if boxed:
+            body = texi_body(doc)
+            body += ('\n@b{Arguments:} the members of @code{%s}\n'
+                     % arg_type.name)
+            body += texi_features(doc)
+            body += texi_sections(doc, ifcond)
+        else:
+            body = texi_entity(doc, 'Arguments', ifcond)
         self._gen.add(MSG_FMT(type='Event',
                               name=doc.symbol,
-                              body=texi_entity(doc, 'Arguments', ifcond)))
+                              body=body))
 
     def symbol(self, doc, entity):
         if self._gen._body:
tests/qapi-schema/doc-good.json +0 -1
index df50a877e3..d992e713d9 100644
@@ -160,7 +160,6 @@
 
 ##
 # @EVT-BOXED:
-# BUG: generated doc misses arguments
 ##
 { 'event': 'EVT-BOXED',  'boxed': true,
   'data': 'Object' }
tests/qapi-schema/doc-good.out +1 -1
index 8cc29fce50..4c9406a464 100644
@@ -196,4 +196,4 @@ another feature
 <- out
 doc symbol=EVT-BOXED
     body=
-BUG: generated doc misses arguments
+
Signature · 1 line
2.21.0
#7 lore raw

PATCH 7/19 qapi: De-duplicate entity documentation generation code

In reply to Markus Armbruster

QAPISchemaGenDocVisitor.visit_command() duplicates texi_entity() for its boxed arguments case. The previous commit added another copy in .visit_event().

Replace texi_entity() by texi_type() and texi_msg(). Use texi_msg() for the boxed arguments case as well.

---
 scripts/qapi/doc.py | 82 ++++++++++++++++++++++-----------------------
 1 file changed, 40 insertions(+), 42 deletions(-)
scripts/qapi/doc.py +40 -42
index f2462c9877..c8c4bda153 100644
@@ -12,7 +12,7 @@ from qapi.gen import QAPIGenDoc, QAPISchemaVisitor
 MSG_FMT = """
 @deftypefn {type} {{}} {name}
 
-{body}
+{body}{members}{features}{sections}
 @end deftypefn
 
 """.format
@@ -20,7 +20,7 @@ MSG_FMT = """
 TYPE_FMT = """
 @deftp {{{type}}} {name}
 
-{body}
+{body}{members}{features}{sections}
 @end deftp
 
 """.format
@@ -149,7 +149,8 @@ def texi_member(member, desc, suffix):
         suffix, desc, texi_if(member.ifcond, prefix='@*'))
 
 
-def texi_members(doc, what, base, variants, member_func):
+def texi_members(doc, what, base=None, variants=None,
+                 member_func=texi_member):
     """Format the table of members"""
     items = ''
     for section in doc.args.values():
@@ -182,6 +183,13 @@ def texi_members(doc, what, base, variants, member_func):
     return '\n@b{%s:}\n@table @asis\n%s@end table\n' % (what, items)
 
 
+def texi_arguments(doc, boxed_arg_type):
+    if boxed_arg_type:
+        return ('\n@b{Arguments:} the members of @code{%s}\n'
+                % boxed_arg_type.name)
+    return texi_members(doc, 'Arguments')
+
+
 def texi_features(doc):
     """Format the table of features"""
     items = ''
@@ -208,12 +216,22 @@ def texi_sections(doc, ifcond):
     return body
 
 
-def texi_entity(doc, what, ifcond, base=None, variants=None,
-                member_func=texi_member):
-    return (texi_body(doc)
-            + texi_members(doc, what, base, variants, member_func)
-            + texi_features(doc)
-            + texi_sections(doc, ifcond))
+def texi_type(typ, doc, ifcond, members):
+    return TYPE_FMT(type=typ,
+                    name=doc.symbol,
+                    body=texi_body(doc),
+                    members=members,
+                    features=texi_features(doc),
+                    sections=texi_sections(doc, ifcond))
+
+
+def texi_msg(typ, doc, ifcond, members):
+    return MSG_FMT(type=typ,
+                   name=doc.symbol,
+                   body=texi_body(doc),
+                   members=members,
+                   features=texi_features(doc),
+                   sections=texi_sections(doc, ifcond))
 
 
 class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
@@ -227,56 +245,36 @@ class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
 
     def visit_enum_type(self, name, info, ifcond, members, prefix):
         doc = self.cur_doc
-        self._gen.add(TYPE_FMT(type='Enum',
-                               name=doc.symbol,
-                               body=texi_entity(doc, 'Values', ifcond,
-                                                member_func=texi_enum_value)))
+        self._gen.add(texi_type('Enum', doc, ifcond,
+                                texi_members(doc, 'Values',
+                                             member_func=texi_enum_value)))
 
     def visit_object_type(self, name, info, ifcond, base, members, variants,
                           features):
         doc = self.cur_doc
         if base and base.is_implicit():
             base = None
-        self._gen.add(TYPE_FMT(type='Object',
-                               name=doc.symbol,
-                               body=texi_entity(doc, 'Members', ifcond,
-                                                base, variants)))
+        self._gen.add(texi_type('Object', doc, ifcond,
+                                texi_members(doc, 'Members', base, variants)))
 
     def visit_alternate_type(self, name, info, ifcond, variants):
         doc = self.cur_doc
-        self._gen.add(TYPE_FMT(type='Alternate',
-                               name=doc.symbol,
-                               body=texi_entity(doc, 'Members', ifcond)))
+        self._gen.add(texi_type('Alternate', doc, ifcond,
+                                texi_members(doc, 'Members')))
 
     def visit_command(self, name, info, ifcond, arg_type, ret_type, gen,
                       success_response, boxed, allow_oob, allow_preconfig,
                       features):
         doc = self.cur_doc
-        if boxed:
-            body = texi_body(doc)
-            body += ('\n@b{Arguments:} the members of @code{%s}\n'
-                     % arg_type.name)
-            body += texi_features(doc)
-            body += texi_sections(doc, ifcond)
-        else:
-            body = texi_entity(doc, 'Arguments', ifcond)
-        self._gen.add(MSG_FMT(type='Command',
-                              name=doc.symbol,
-                              body=body))
+        self._gen.add(texi_msg('Command', doc, ifcond,
+                               texi_arguments(doc,
+                                              arg_type if boxed else None)))
 
     def visit_event(self, name, info, ifcond, arg_type, boxed):
         doc = self.cur_doc
-        if boxed:
-            body = texi_body(doc)
-            body += ('\n@b{Arguments:} the members of @code{%s}\n'
-                     % arg_type.name)
-            body += texi_features(doc)
-            body += texi_sections(doc, ifcond)
-        else:
-            body = texi_entity(doc, 'Arguments', ifcond)
-        self._gen.add(MSG_FMT(type='Event',
-                              name=doc.symbol,
-                              body=body))
+        self._gen.add(texi_msg('Event', doc, ifcond,
+                               texi_arguments(doc,
+                                              arg_type if boxed else None)))
 
     def symbol(self, doc, entity):
         if self._gen._body:
Signature · 1 line
2.21.0
#8 lore raw

PATCH 8/19 qapi: Split .connect_doc(), .check_doc() off .check()

In reply to Markus Armbruster

Splitting documentation checking off the .check() methods makes them a bit more focused, which is welcome, as some of them are pretty big. It also prepares the ground for the following commits.

---
 scripts/qapi/schema.py | 34 +++++++++++++++++++++++++++-------
 1 file changed, 27 insertions(+), 7 deletions(-)
scripts/qapi/schema.py +27 -7
index f7d68a35f4..9b62c8d74d 100644
@@ -51,6 +51,12 @@ class QAPISchemaEntity(object):
                                            os.path.dirname(schema.fname))
         self._checked = True
 
+    def connect_doc(self):
+        pass
+
+    def check_doc(self):
+        pass
+
     @property
     def ifcond(self):
         assert self._checked
@@ -217,7 +223,10 @@ class QAPISchemaEnumType(QAPISchemaType):
         seen = {}
         for m in self.members:
             m.check_clash(self.info, seen)
-            if self.doc:
+
+    def connect_doc(self):
+        if self.doc:
+            for m in self.members:
                 self.doc.connect_member(m)
 
     def is_implicit(self):
@@ -345,8 +354,6 @@ class QAPISchemaObjectType(QAPISchemaType):
         for m in self.local_members:
             m.check(schema)
             m.check_clash(self.info, seen)
-            if self.doc:
-                self.doc.connect_member(m)
         members = seen.values()
 
         if self.variants:
@@ -358,9 +365,6 @@ class QAPISchemaObjectType(QAPISchemaType):
         for f in self.features:
             f.check_clash(self.info, seen)
 
-        if self.doc:
-            self.doc.check()
-
         self.members = members  # mark completed
 
     # Check that the members of this type do not cause duplicate JSON members,
@@ -372,6 +376,15 @@ class QAPISchemaObjectType(QAPISchemaType):
         for m in self.members:
             m.check_clash(info, seen)
 
+    def connect_doc(self):
+        if self.doc:
+            for m in self.local_members:
+                self.doc.connect_member(m)
+
+    def check_doc(self):
+        if self.doc:
+            self.doc.check()
+
     @property
     def ifcond(self):
         assert self._checked
@@ -639,8 +652,13 @@ class QAPISchemaAlternateType(QAPISchemaType):
                         "%s can't be distinguished from '%s'"
                         % (v.describe(self.info), types_seen[qt]))
                 types_seen[qt] = v.name
-            if self.doc:
+
+    def connect_doc(self):
+        if self.doc:
+            for v in self.variants.variants:
                 self.doc.connect_member(v)
+
+    def check_doc(self):
         if self.doc:
             self.doc.check()
 
@@ -1043,6 +1061,8 @@ class QAPISchema(object):
     def check(self):
         for ent in self._entity_list:
             ent.check(self)
+            ent.connect_doc()
+            ent.check_doc()
 
     def visit(self, visitor):
         visitor.visit_begin(self)
Signature · 1 line
2.21.0
#9 lore raw

PATCH 9/19 qapi: Fix enum doc comment checking

In reply to Markus Armbruster

Enumeration type documentation comments are not checked, as demonstrated by test doc-bad-enum-member. This is because we neglect to call self.doc.check() for enumeration types. Messed up in 816a57cd6e "qapi: Fix detection of bogus member documentation". Fix it.

---
 scripts/qapi/schema.py                     |  4 ++++
 tests/qapi-schema/doc-bad-enum-member.err  |  1 +
 tests/qapi-schema/doc-bad-enum-member.json |  1 -
 tests/qapi-schema/doc-bad-enum-member.out  | 21 ---------------------
 4 files changed, 5 insertions(+), 22 deletions(-)
scripts/qapi/schema.py +4 -0
index 9b62c8d74d..0381e3cb40 100644
@@ -229,6 +229,10 @@ class QAPISchemaEnumType(QAPISchemaType):
             for m in self.members:
                 self.doc.connect_member(m)
 
+    def check_doc(self):
+        if self.doc:
+            self.doc.check()
+
     def is_implicit(self):
         # See QAPISchema._make_implicit_enum_type() and ._def_predefineds()
         return self.name.endswith('Kind') or self.name == 'QType'
tests/qapi-schema/doc-bad-enum-member.err +1 -0
index e69de29bb2..dfa1e786d7 100644
@@ -0,0 +1 @@
+doc-bad-enum-member.json:3: the following documented members are not in the declaration: a
tests/qapi-schema/doc-bad-enum-member.json +0 -1
index 9f32fe64b4..9cab35c6e8 100644
@@ -1,5 +1,4 @@
 # Members listed in the doc comment must exist in the actual schema
-# BUG: nonexistent @a is not rejected
 
 ##
 # @Foo:
tests/qapi-schema/doc-bad-enum-member.out +0 -21
index 6ca31c1e9b..e69de29bb2 100644
@@ -1,21 +0,0 @@
-module None
-object q_empty
-enum QType
-    prefix QTYPE
-    member none
-    member qnull
-    member qnum
-    member qstring
-    member qdict
-    member qlist
-    member qbool
-module doc-bad-enum-member.json
-enum Foo
-    member b
-doc symbol=Foo
-    body=
-
-    arg=a
-a
-    arg=b
-b
Signature · 1 line
2.21.0
#10 lore raw

PATCH 10/19 qapi: Clean up doc comment checking for implicit union base

In reply to Markus Armbruster

An object type's doc comment describes the type's members, less the ones defined in a named base type. Cases:

* Struct: the members are defined in 'data' and inherited from 'base'. Since the base type cannot be implicit, the doc comment describes just 'data'.

* Simple union: the only member is the implicit tag member @type, and the doc comment describes it.

* Flat union with implicit base type: the members are defined in 'base', and the doc comment describes it.

* Flat union with named base type: the members are inherited from 'base'. The doc comment describes no members.

Before we can check a doc comment with .check_doc(), we need .connect_doc() connect each of its "argument sections" to the member it documents.

For structs and simple unions, this is straightforward: the members in question are in .local_members, and .connect_doc() connects them.

For flat unions with a named base type, it's trivial: .local_members is empty, and .connect_doc() does nothing.

For flat unions with an implicit base type, it's tricky. We have QAPISchema._make_implicit_object_type() forward the union's doc comment to the implicit base type, so that the base type's .connect_doc() connects the members. The union's .connect_doc() does nothing, as .local_members is empty.

Dirt effect: we check the doc comment twice, once for the union type, and once for the implicit base type.

This is needlessly brittle and hard to understand. Clean up as follows. Make the union's .connect_doc() connect an implicit base's members itself. Do not forward the union's doc comment to its implicit base type.

Requires extending .connect_doc() so it can work with a doc comment other than self.doc. Add an optional argument for that.

---
 scripts/qapi/schema.py | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
scripts/qapi/schema.py +16 -11
index 0381e3cb40..c16dce1fe0 100644
@@ -51,7 +51,7 @@ class QAPISchemaEntity(object):
                                            os.path.dirname(schema.fname))
         self._checked = True
 
-    def connect_doc(self):
+    def connect_doc(self, doc=None):
         pass
 
     def check_doc(self):
@@ -224,10 +224,11 @@ class QAPISchemaEnumType(QAPISchemaType):
         for m in self.members:
             m.check_clash(self.info, seen)
 
-    def connect_doc(self):
-        if self.doc:
+    def connect_doc(self, doc=None):
+        doc = doc or self.doc
+        if doc:
             for m in self.members:
-                self.doc.connect_member(m)
+                doc.connect_member(m)
 
     def check_doc(self):
         if self.doc:
@@ -380,10 +381,13 @@ class QAPISchemaObjectType(QAPISchemaType):
         for m in self.members:
             m.check_clash(info, seen)
 
-    def connect_doc(self):
-        if self.doc:
+    def connect_doc(self, doc=None):
+        doc = doc or self.doc
+        if doc:
+            if self.base and self.base.is_implicit():
+                self.base.connect_doc(doc)
             for m in self.local_members:
-                self.doc.connect_member(m)
+                doc.connect_member(m)
 
     def check_doc(self):
         if self.doc:
@@ -657,10 +661,11 @@ class QAPISchemaAlternateType(QAPISchemaType):
                         % (v.describe(self.info), types_seen[qt]))
                 types_seen[qt] = v.name
 
-    def connect_doc(self):
-        if self.doc:
+    def connect_doc(self, doc=None):
+        doc = doc or self.doc
+        if doc:
             for v in self.variants.variants:
-                self.doc.connect_member(v)
+                doc.connect_member(v)
 
     def check_doc(self):
         if self.doc:
@@ -974,7 +979,7 @@ class QAPISchema(object):
         tag_member = None
         if isinstance(base, dict):
             base = self._make_implicit_object_type(
-                name, info, doc, ifcond,
+                name, info, None, ifcond,
                 'base', self._make_members(base, info))
         if tag_name:
             variants = [self._make_variant(key, value['type'],
Signature · 1 line
2.21.0
#11 lore raw

PATCH 11/19 qapi: Fix doc comment checking for commands and events

In reply to Markus Armbruster

When a command's 'data' is an object, its doc comment describes the arguments defined there. When 'data' names a type, the doc comment does not describe arguments. Instead, the doc generator inserts a pointer to the named type.

An event's doc comment works the same.

We don't actually check doc comments for commands and events. Instead, QAPISchema._def_command() forwards the doc comment to the implicit argument type, where it gets checked. Works because the check only cares for the implicit argument type's members.

Not only is this needlessly hard to understand, it actually falls apart in two cases:

* When 'data' is empty, there is nothing to forward to, and the doc comment remains unchecked. Demonstrated by test doc-bad-event-arg.

* When 'data' names a type, we can't forward, as the type has its own doc comment. The command or event's doc comment remains unchecked. Demonstrated by test doc-bad-boxed-command-arg.

The forwarding goes back to commit 069fb5b250 "qapi: Prepare for requiring more complete documentation", put to use in commit 816a57cd6e "qapi: Fix detection of bogus member documentation". That fix was incomplete.

To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc comments, and drop the forwarding of doc comments to implicit argument types.

---
 qapi/net.json                                 |  2 --
 scripts/qapi/doc.py                           |  1 +
 scripts/qapi/schema.py                        | 24 +++++++++++++++--
 .../qapi-schema/doc-bad-boxed-command-arg.err |  1 +
 .../doc-bad-boxed-command-arg.json            |  1 -
 .../qapi-schema/doc-bad-boxed-command-arg.out | 26 -------------------
 tests/qapi-schema/doc-bad-event-arg.err       |  1 +
 tests/qapi-schema/doc-bad-event-arg.json      |  1 -
 tests/qapi-schema/doc-bad-event-arg.out       | 19 --------------
 9 files changed, 25 insertions(+), 51 deletions(-)
qapi/net.json +0 -2
index 728990f4fb..4c96137811 100644
@@ -723,8 +723,6 @@
 # Trigger generation of broadcast RARP frames to update network switches.
 # This can be useful when network bonds fail-over the active slave.
 #
-# @params: AnnounceParameters giving timing and repetition count of announce
-#
 # Example:
 #
 # -> { "execute": "announce-self",
scripts/qapi/doc.py +1 -0
index c8c4bda153..6f1c17f71f 100644
@@ -185,6 +185,7 @@ def texi_members(doc, what, base=None, variants=None,
 
 def texi_arguments(doc, boxed_arg_type):
     if boxed_arg_type:
+        assert not doc.args
         return ('\n@b{Arguments:} the members of @code{%s}\n'
                 % boxed_arg_type.name)
     return texi_members(doc, 'Arguments')
scripts/qapi/schema.py +22 -2
index c16dce1fe0..06e37c9c49 100644
@@ -739,6 +739,16 @@ class QAPISchemaCommand(QAPISchemaEntity):
         for f in self.features:
             f.check_clash(self.info, seen)
 
+    def connect_doc(self, doc=None):
+        doc = doc or self.doc
+        if doc:
+            if self.arg_type and self.arg_type.is_implicit():
+                self.arg_type.connect_doc(doc)
+
+    def check_doc(self):
+        if self.doc:
+            self.doc.check()
+
     def visit(self, visitor):
         QAPISchemaEntity.visit(self, visitor)
         visitor.visit_command(self.name, self.info, self.ifcond,
@@ -775,6 +785,16 @@ class QAPISchemaEvent(QAPISchemaEntity):
                     "event's 'data' can take %s only with 'boxed': true"
                     % self.arg_type.describe())
 
+    def connect_doc(self, doc=None):
+        doc = doc or self.doc
+        if doc:
+            if self.arg_type and self.arg_type.is_implicit():
+                self.arg_type.connect_doc(doc)
+
+    def check_doc(self):
+        if self.doc:
+            self.doc.check()
+
     def visit(self, visitor):
         QAPISchemaEntity.visit(self, visitor)
         visitor.visit_event(self.name, self.info, self.ifcond,
@@ -1026,7 +1046,7 @@ class QAPISchema(object):
         features = expr.get('features', [])
         if isinstance(data, OrderedDict):
             data = self._make_implicit_object_type(
-                name, info, doc, ifcond, 'arg', self._make_members(data, info))
+                name, info, None, ifcond, 'arg', self._make_members(data, info))
         if isinstance(rets, list):
             assert len(rets) == 1
             rets = self._make_array_type(rets[0], info)
@@ -1042,7 +1062,7 @@ class QAPISchema(object):
         ifcond = expr.get('if')
         if isinstance(data, OrderedDict):
             data = self._make_implicit_object_type(
-                name, info, doc, ifcond, 'arg', self._make_members(data, info))
+                name, info, None, ifcond, 'arg', self._make_members(data, info))
         self._def_entity(QAPISchemaEvent(name, info, doc, ifcond, data, boxed))
 
     def _def_exprs(self, exprs):
tests/qapi-schema/doc-bad-boxed-command-arg.err +1 -0
index e69de29bb2..e1101b1667 100644
@@ -0,0 +1 @@
+doc-bad-boxed-command-arg.json:9: the following documented members are not in the declaration: a
tests/qapi-schema/doc-bad-boxed-command-arg.json +0 -1
index 2c265d2ca3..bd143241ec 100644
@@ -1,5 +1,4 @@
 # Boxed arguments are not to be documented with the command
-# BUG: not rejected
 
 ##
 # @Args:
tests/qapi-schema/doc-bad-boxed-command-arg.out +0 -26
index 4ccd788253..e69de29bb2 100644
@@ -1,26 +0,0 @@
-module None
-object q_empty
-enum QType
-    prefix QTYPE
-    member none
-    member qnull
-    member qnum
-    member qstring
-    member qdict
-    member qlist
-    member qbool
-module doc-bad-boxed-command-arg.json
-object Args
-    member a: int optional=False
-command cmd-boxed Args -> None
-    gen=True success_response=True boxed=True oob=False preconfig=False
-doc symbol=Args
-    body=
-
-    arg=a
-an argument
-doc symbol=cmd-boxed
-    body=
-
-    arg=a
-bogus
tests/qapi-schema/doc-bad-event-arg.err +1 -0
index e69de29bb2..114ff4a3c7 100644
@@ -0,0 +1 @@
+doc-bad-event-arg.json:3: the following documented members are not in the declaration: a
tests/qapi-schema/doc-bad-event-arg.json +0 -1
index 80d4e1240b..23c83cc81f 100644
@@ -1,5 +1,4 @@
 # Arguments listed in the doc comment must exist in the actual schema
-# BUG: nonexistent @a is not rejected
 
 ##
 # @FOO:
tests/qapi-schema/doc-bad-event-arg.out +0 -19
index ad0367cd45..e69de29bb2 100644
@@ -1,19 +0,0 @@
-module None
-object q_empty
-enum QType
-    prefix QTYPE
-    member none
-    member qnull
-    member qnum
-    member qstring
-    member qdict
-    member qlist
-    member qbool
-module doc-bad-event-arg.json
-event FOO None
-    boxed=False
-doc symbol=FOO
-    body=
-
-    arg=a
-a
Signature · 1 line
2.21.0
#12 lore raw

PATCH 12/19 qapi: Simplify ._make_implicit_object_type()

In reply to Markus Armbruster

All callers now pass doc=None. Drop the argument.

---
 scripts/qapi/schema.py | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
scripts/qapi/schema.py +6 -7
index 06e37c9c49..27da4e0f7d 100644
@@ -920,8 +920,7 @@ class QAPISchema(object):
             self._def_entity(QAPISchemaArrayType(name, info, element_type))
         return name
 
-    def _make_implicit_object_type(self, name, info, doc, ifcond,
-                                   role, members):
+    def _make_implicit_object_type(self, name, info, ifcond, role, members):
         if not members:
             return None
         # See also QAPISchemaObjectTypeMember.describe()
@@ -939,7 +938,7 @@ class QAPISchema(object):
             # TODO kill simple unions or implement the disjunction
             assert (ifcond or []) == typ._ifcond # pylint: disable=protected-access
         else:
-            self._def_entity(QAPISchemaObjectType(name, info, doc, ifcond,
+            self._def_entity(QAPISchemaObjectType(name, info, None, ifcond,
                                                   None, members, None, []))
         return name
 
@@ -986,7 +985,7 @@ class QAPISchema(object):
             assert len(typ) == 1
             typ = self._make_array_type(typ[0], info)
         typ = self._make_implicit_object_type(
-            typ, info, None, self.lookup_type(typ),
+            typ, info, self.lookup_type(typ),
             'wrapper', [self._make_member('data', typ, None, info)])
         return QAPISchemaObjectTypeVariant(case, info, typ, ifcond)
 
@@ -999,7 +998,7 @@ class QAPISchema(object):
         tag_member = None
         if isinstance(base, dict):
             base = self._make_implicit_object_type(
-                name, info, None, ifcond,
+                name, info, ifcond,
                 'base', self._make_members(base, info))
         if tag_name:
             variants = [self._make_variant(key, value['type'],
@@ -1046,7 +1045,7 @@ class QAPISchema(object):
         features = expr.get('features', [])
         if isinstance(data, OrderedDict):
             data = self._make_implicit_object_type(
-                name, info, None, ifcond, 'arg', self._make_members(data, info))
+                name, info, ifcond, 'arg', self._make_members(data, info))
         if isinstance(rets, list):
             assert len(rets) == 1
             rets = self._make_array_type(rets[0], info)
@@ -1062,7 +1061,7 @@ class QAPISchema(object):
         ifcond = expr.get('if')
         if isinstance(data, OrderedDict):
             data = self._make_implicit_object_type(
-                name, info, None, ifcond, 'arg', self._make_members(data, info))
+                name, info, ifcond, 'arg', self._make_members(data, info))
         self._def_entity(QAPISchemaEvent(name, info, doc, ifcond, data, boxed))
 
     def _def_exprs(self, exprs):
Signature · 1 line
2.21.0
#13 lore raw

PATCH 13/19 qapi: Eliminate .check_doc() overrides

In reply to Markus Armbruster

All sub-classes of QAPISchemaEntity now override .check_doc() the same way, except for QAPISchemaType and and QAPISchemaArrayType.

Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the overrides. QAPISchemaType doesn't care because it's abstract. QAPISchemaArrayType doesn't care because its .doc is always None.

---
 scripts/qapi/schema.py | 23 ++---------------------
 1 file changed, 2 insertions(+), 21 deletions(-)
scripts/qapi/schema.py +2 -21
index 27da4e0f7d..ee510f129b 100644
@@ -55,7 +55,8 @@ class QAPISchemaEntity(object):
         pass
 
     def check_doc(self):
-        pass
+        if self.doc:
+            self.doc.check()
 
     @property
     def ifcond(self):
@@ -230,10 +231,6 @@ class QAPISchemaEnumType(QAPISchemaType):
             for m in self.members:
                 doc.connect_member(m)
 
-    def check_doc(self):
-        if self.doc:
-            self.doc.check()
-
     def is_implicit(self):
         # See QAPISchema._make_implicit_enum_type() and ._def_predefineds()
         return self.name.endswith('Kind') or self.name == 'QType'
@@ -389,10 +386,6 @@ class QAPISchemaObjectType(QAPISchemaType):
             for m in self.local_members:
                 doc.connect_member(m)
 
-    def check_doc(self):
-        if self.doc:
-            self.doc.check()
-
     @property
     def ifcond(self):
         assert self._checked
@@ -667,10 +660,6 @@ class QAPISchemaAlternateType(QAPISchemaType):
             for v in self.variants.variants:
                 doc.connect_member(v)
 
-    def check_doc(self):
-        if self.doc:
-            self.doc.check()
-
     def c_type(self):
         return c_name(self.name) + pointer_suffix
 
@@ -745,10 +734,6 @@ class QAPISchemaCommand(QAPISchemaEntity):
             if self.arg_type and self.arg_type.is_implicit():
                 self.arg_type.connect_doc(doc)
 
-    def check_doc(self):
-        if self.doc:
-            self.doc.check()
-
     def visit(self, visitor):
         QAPISchemaEntity.visit(self, visitor)
         visitor.visit_command(self.name, self.info, self.ifcond,
@@ -791,10 +776,6 @@ class QAPISchemaEvent(QAPISchemaEntity):
             if self.arg_type and self.arg_type.is_implicit():
                 self.arg_type.connect_doc(doc)
 
-    def check_doc(self):
-        if self.doc:
-            self.doc.check()
-
     def visit(self, visitor):
         QAPISchemaEntity.visit(self, visitor)
         visitor.visit_event(self.name, self.info, self.ifcond,
Signature · 1 line
2.21.0
#14 lore raw

PATCH 14/19 qapi: Fold normalize_if() into check_if()

In reply to Markus Armbruster

check_if() is always called together with normalize_if(). Fold the latter into the former.

---
 scripts/qapi/expr.py | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)
scripts/qapi/expr.py +1 -12
index 7c7394f835..aa0fe69f99 100644
@@ -95,12 +95,6 @@ def check_flags(expr, info):
                 info, "flag '%s' may only use true value" % key)
 
 
-def normalize_if(expr):
-    ifcond = expr.get('if')
-    if isinstance(ifcond, str):
-        expr['if'] = [ifcond]
-
-
 def check_if(expr, info, source):
 
     def check_if_str(ifcond, info):
@@ -126,6 +120,7 @@ def check_if(expr, info, source):
             check_if_str(elt, info)
     else:
         check_if_str(ifcond, info)
+        expr['if'] = [ifcond]
 
 
 def normalize_members(members):
@@ -175,7 +170,6 @@ def check_type(value, info, source,
             raise QAPISemError(info, "%s uses reserved name" % key_source)
         check_keys(arg, info, key_source, ['type'], ['if'])
         check_if(arg, info, key_source)
-        normalize_if(arg)
         check_type(arg['type'], info, key_source, allow_array=True)
 
 
@@ -198,7 +192,6 @@ def check_features(features, info):
         source = "%s '%s'" % (source, f['name'])
         check_name_str(f['name'], info, source)
         check_if(f, info, source)
-        normalize_if(f)
 
 
 def normalize_enum(expr):
@@ -227,7 +220,6 @@ def check_enum(expr, info):
         check_name_str(member['name'], info, source,
                        enum_member=True, permit_upper=permit_upper)
         check_if(member, info, source)
-        normalize_if(member)
 
 
 def check_struct(expr, info):
@@ -259,7 +251,6 @@ def check_union(expr, info):
         check_name_str(key, info, source)
         check_keys(value, info, source, ['type'], ['if'])
         check_if(value, info, source)
-        normalize_if(value)
         check_type(value['type'], info, source, allow_array=not base)
 
 
@@ -273,7 +264,6 @@ def check_alternate(expr, info):
         check_name_str(key, info, source)
         check_keys(value, info, source, ['type'], ['if'])
         check_if(value, info, source)
-        normalize_if(value)
         check_type(value['type'], info, source)
 
 
@@ -376,7 +366,6 @@ def check_exprs(exprs):
         else:
             assert False, 'unexpected meta type'
 
-        normalize_if(expr)
         check_if(expr, info, meta)
         check_flags(expr, info)
 
Signature · 1 line
2.21.0
#15 lore raw

PATCH 15/19 qapi: Fold normalize_features() into check_features()

In reply to Markus Armbruster

check_features() is always called together with normalize_features():
the former in check_struct() and check_command(), the latter in their caller check_exprs(). Fold the latter into the former.

---
 scripts/qapi/expr.py | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)
scripts/qapi/expr.py +2 -8
index aa0fe69f99..66014e2bb2 100644
@@ -173,17 +173,13 @@ def check_type(value, info, source,
         check_type(arg['type'], info, key_source, allow_array=True)
 
 
-def normalize_features(features):
-    if isinstance(features, list):
-        features[:] = [f if isinstance(f, dict) else {'name': f}
-                       for f in features]
-
-
 def check_features(features, info):
     if features is None:
         return
     if not isinstance(features, list):
         raise QAPISemError(info, "'features' must be an array")
+    features[:] = [f if isinstance(f, dict) else {'name': f}
+                   for f in features]
     for f in features:
         source = "'features' member"
         assert isinstance(f, dict)
@@ -347,7 +343,6 @@ def check_exprs(exprs):
             check_keys(expr, info, meta,
                        ['struct', 'data'], ['base', 'if', 'features'])
             normalize_members(expr['data'])
-            normalize_features(expr.get('features'))
             check_struct(expr, info)
         elif meta == 'command':
             check_keys(expr, info, meta,
@@ -356,7 +351,6 @@ def check_exprs(exprs):
                         'gen', 'success-response', 'allow-oob',
                         'allow-preconfig'])
             normalize_members(expr.get('data'))
-            normalize_features(expr.get('features'))
             check_command(expr, info)
         elif meta == 'event':
             check_keys(expr, info, meta,
Signature · 1 line
2.21.0
#16 lore raw

PATCH 16/19 qapi: Fold normalize_enum() into check_enum()

In reply to Markus Armbruster

check_features() is always called together with normalize_features(). Fold the latter into the former.

---
 scripts/qapi/expr.py | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
scripts/qapi/expr.py +2 -7
index 66014e2bb2..d7a289eded 100644
@@ -190,12 +190,6 @@ def check_features(features, info):
         check_if(f, info, source)
 
 
-def normalize_enum(expr):
-    if isinstance(expr['data'], list):
-        expr['data'] = [m if isinstance(m, dict) else {'name': m}
-                        for m in expr['data']]
-
-
 def check_enum(expr, info):
     name = expr['enum']
     members = expr['data']
@@ -208,6 +202,8 @@ def check_enum(expr, info):
 
     permit_upper = name in info.pragma.name_case_whitelist
 
+    members[:] = [m if isinstance(m, dict) else {'name': m}
+                  for m in members]
     for member in members:
         source = "'data' member"
         check_keys(member, info, source, ['name'], ['if'])
@@ -325,7 +321,6 @@ def check_exprs(exprs):
         if meta == 'enum':
             check_keys(expr, info, meta,
                        ['enum', 'data'], ['if', 'prefix'])
-            normalize_enum(expr)
             check_enum(expr, info)
         elif meta == 'union':
             check_keys(expr, info, meta,
Signature · 1 line
2.21.0
#17 lore raw

PATCH 17/19 qapi: Lift features into QAPISchemaEntity

In reply to Markus Armbruster

Commit 6a8c0b5102 "qapi: Add feature flags to struct types" added features to QAPISchemaObjectType. Commit a95daa5093 "qapi: Add feature flags to commands in qapi" added them to QAPISchemaCommand, duplicating the code. Tolerable, but the duplication will only get worse as we add features to more definitions.

To de-duplicate, lift features from QAPISchemaObjectType and QAPISchemaCommand into QAPISchemaEntity.

---
 scripts/qapi/schema.py | 32 +++++++++++---------------------
 1 file changed, 11 insertions(+), 21 deletions(-)
scripts/qapi/schema.py +11 -21
index ee510f129b..bdea9482fc 100644
@@ -27,8 +27,11 @@ from qapi.parser import QAPISchemaParser
 class QAPISchemaEntity(object):
     meta = None
 
-    def __init__(self, name, info, doc, ifcond=None):
+    def __init__(self, name, info, doc, ifcond=None, features=None):
         assert name is None or isinstance(name, str)
+        for f in features or []:
+            assert isinstance(f, QAPISchemaFeature)
+            f.set_defined_in(name)
         self.name = name
         self._module = None
         # For explicitly defined entities, info points to the (explicit)
@@ -39,6 +42,7 @@ class QAPISchemaEntity(object):
         self.info = info
         self.doc = doc
         self._ifcond = ifcond or []
+        self.features = features or []
         self._checked = False
 
     def c_name(self):
@@ -49,6 +53,10 @@ class QAPISchemaEntity(object):
         if self.info:
             self._module = os.path.relpath(self.info.fname,
                                            os.path.dirname(schema.fname))
+        seen = {}
+        for f in self.features:
+            f.check_clash(self.info, seen)
+
         self._checked = True
 
     def connect_doc(self, doc=None):
@@ -307,7 +315,7 @@ class QAPISchemaObjectType(QAPISchemaType):
         # struct has local_members, optional base, and no variants
         # flat union has base, variants, and no local_members
         # simple union has local_members, variants, and no base
-        QAPISchemaType.__init__(self, name, info, doc, ifcond)
+        QAPISchemaType.__init__(self, name, info, doc, ifcond, features)
         self.meta = 'union' if variants else 'struct'
         assert base is None or isinstance(base, str)
         for m in local_members:
@@ -316,15 +324,11 @@ class QAPISchemaObjectType(QAPISchemaType):
         if variants is not None:
             assert isinstance(variants, QAPISchemaObjectTypeVariants)
             variants.set_defined_in(name)
-        for f in features:
-            assert isinstance(f, QAPISchemaFeature)
-            f.set_defined_in(name)
         self._base_name = base
         self.base = None
         self.local_members = local_members
         self.variants = variants
         self.members = None
-        self.features = features
 
     def check(self, schema):
         # This calls another type T's .check() exactly when the C
@@ -362,11 +366,6 @@ class QAPISchemaObjectType(QAPISchemaType):
             self.variants.check(schema, seen)
             self.variants.check_clash(self.info, seen)
 
-        # Features are in a name space separate from members
-        seen = {}
-        for f in self.features:
-            f.check_clash(self.info, seen)
-
         self.members = members  # mark completed
 
     # Check that the members of this type do not cause duplicate JSON members,
@@ -678,12 +677,9 @@ class QAPISchemaCommand(QAPISchemaEntity):
     def __init__(self, name, info, doc, ifcond, arg_type, ret_type,
                  gen, success_response, boxed, allow_oob, allow_preconfig,
                  features):
-        QAPISchemaEntity.__init__(self, name, info, doc, ifcond)
+        QAPISchemaEntity.__init__(self, name, info, doc, ifcond, features)
         assert not arg_type or isinstance(arg_type, str)
         assert not ret_type or isinstance(ret_type, str)
-        for f in features:
-            assert isinstance(f, QAPISchemaFeature)
-            f.set_defined_in(name)
         self._arg_type_name = arg_type
         self.arg_type = None
         self._ret_type_name = ret_type
@@ -693,7 +689,6 @@ class QAPISchemaCommand(QAPISchemaEntity):
         self.boxed = boxed
         self.allow_oob = allow_oob
         self.allow_preconfig = allow_preconfig
-        self.features = features
 
     def check(self, schema):
         QAPISchemaEntity.check(self, schema)
@@ -723,11 +718,6 @@ class QAPISchemaCommand(QAPISchemaEntity):
                         "command's 'returns' cannot take %s"
                         % self.ret_type.describe())
 
-        # Features are in a name space separate from members
-        seen = {}
-        for f in self.features:
-            f.check_clash(self.info, seen)
-
     def connect_doc(self, doc=None):
         doc = doc or self.doc
         if doc:
Signature · 1 line
2.21.0
#18 lore raw

PATCH 18/19 qapi: Polish reporting of bogus member documentation

In reply to Markus Armbruster

Improve error messages from

the following documented members are not in the declaration: a the following documented members are not in the declaration: aa, bb

to the more concise

documented member 'a' does not exist
documented members 'aa', 'bb' do not exist

---
 scripts/qapi/parser.py                          | 6 ++++--
 tests/qapi-schema/doc-bad-alternate-member.err  | 2 +-
 tests/qapi-schema/doc-bad-boxed-command-arg.err | 2 +-
 tests/qapi-schema/doc-bad-command-arg.err       | 2 +-
 tests/qapi-schema/doc-bad-enum-member.err       | 2 +-
 tests/qapi-schema/doc-bad-event-arg.err         | 2 +-
 tests/qapi-schema/doc-bad-union-member.err      | 2 +-
 7 files changed, 10 insertions(+), 8 deletions(-)
scripts/qapi/parser.py +4 -2
index e800876ad1..6c45a00cf4 100644
@@ -566,5 +566,7 @@ class QAPIDoc(object):
         if bogus:
             raise QAPISemError(
                 self.info,
-                "the following documented members are not in "
-                "the declaration: %s" % ", ".join(bogus))
+                "documented member%s '%s' %s not exist"
+                % ("s" if len(bogus) > 1 else "",
+                   "', '".join(bogus),
+                   "do" if len(bogus) > 1 else "does"))
tests/qapi-schema/doc-bad-alternate-member.err +1 -1
index a1c282f935..d7286bb57c 100644
@@ -1 +1 @@
-doc-bad-alternate-member.json:3: the following documented members are not in the declaration: aa, bb
+doc-bad-alternate-member.json:3: documented members 'aa', 'bb' do not exist
tests/qapi-schema/doc-bad-boxed-command-arg.err +1 -1
index e1101b1667..7137af3ec9 100644
@@ -1 +1 @@
-doc-bad-boxed-command-arg.json:9: the following documented members are not in the declaration: a
+doc-bad-boxed-command-arg.json:9: documented member 'a' does not exist
tests/qapi-schema/doc-bad-command-arg.err +1 -1
index 153ea0330a..18ed076cef 100644
@@ -1 +1 @@
-doc-bad-command-arg.json:3: the following documented members are not in the declaration: b
+doc-bad-command-arg.json:3: documented member 'b' does not exist
tests/qapi-schema/doc-bad-enum-member.err +1 -1
index dfa1e786d7..7efeb47363 100644
@@ -1 +1 @@
-doc-bad-enum-member.json:3: the following documented members are not in the declaration: a
+doc-bad-enum-member.json:3: documented member 'a' does not exist
tests/qapi-schema/doc-bad-event-arg.err +1 -1
index 114ff4a3c7..d13cacf21f 100644
@@ -1 +1 @@
-doc-bad-event-arg.json:3: the following documented members are not in the declaration: a
+doc-bad-event-arg.json:3: documented member 'a' does not exist
tests/qapi-schema/doc-bad-union-member.err +1 -1
index 8b9d36eab1..6dd2726a65 100644
@@ -1 +1 @@
-doc-bad-union-member.json:3: the following documented members are not in the declaration: a, b
+doc-bad-union-member.json:3: documented members 'a', 'b' do not exist
Signature · 1 line
2.21.0
#19 lore raw

PATCH 19/19 qapi: Check feature documentation against the schema

In reply to Markus Armbruster

Commit f3ed93d545 "qapi: Allow documentation for features" neglected to check documentation against the schema. Fix that: check them the same way we check arguments.

---
 scripts/qapi/parser.py                   | 31 +++++++++++++++++-------
 scripts/qapi/schema.py                   |  2 ++
 tests/qapi-schema/doc-bad-feature.err    |  1 +
 tests/qapi-schema/doc-bad-feature.json   |  1 -
 tests/qapi-schema/doc-bad-feature.out    | 19 ---------------
 tests/qapi-schema/doc-undoc-feature.err  |  2 ++
 tests/qapi-schema/doc-undoc-feature.json |  1 -
 tests/qapi-schema/doc-undoc-feature.out  | 21 ----------------
 8 files changed, 27 insertions(+), 51 deletions(-)
scripts/qapi/parser.py +22 -9
index 6c45a00cf4..342792e410 100644
@@ -555,18 +555,31 @@ class QAPIDoc(object):
             self.args[member.name] = QAPIDoc.ArgSection(member.name)
         self.args[member.name].connect(member)
 
+    def connect_feature(self, feature):
+        if feature.name not in self.features:
+            raise QAPISemError(feature.info,
+                               "feature '%s' lacks documentation"
+                               % feature.name)
+            self.features[feature.name] = QAPIDoc.ArgSection(feature.name)
+        self.features[feature.name].connect(feature)
+
     def check_expr(self, expr):
         if self.has_section('Returns') and 'command' not in expr:
             raise QAPISemError(self.info,
                                "'Returns:' is only valid for commands")
 
     def check(self):
-        bogus = [name for name, section in self.args.items()
-                 if not section.member]
-        if bogus:
-            raise QAPISemError(
-                self.info,
-                "documented member%s '%s' %s not exist"
-                % ("s" if len(bogus) > 1 else "",
-                   "', '".join(bogus),
-                   "do" if len(bogus) > 1 else "does"))
+
+        def check_args_section(args, info, what):
+            bogus = [name for name, section in args.items()
+                     if not section.member]
+            if bogus:
+                raise QAPISemError(
+                    self.info,
+                    "documented member%s '%s' %s not exist"
+                    % ("s" if len(bogus) > 1 else "",
+                       "', '".join(bogus),
+                       "do" if len(bogus) > 1 else "does"))
+
+        check_args_section(self.args, self.info, 'members')
+        check_args_section(self.features, self.info, 'features')
scripts/qapi/schema.py +2 -0
index bdea9482fc..cf0045f34e 100644
@@ -56,6 +56,8 @@ class QAPISchemaEntity(object):
         seen = {}
         for f in self.features:
             f.check_clash(self.info, seen)
+            if self.doc:
+                self.doc.connect_feature(f)
 
         self._checked = True
 
tests/qapi-schema/doc-bad-feature.err +1 -0
index e69de29bb2..e4c62adfa3 100644
@@ -0,0 +1 @@
+doc-bad-feature.json:3: documented member 'a' does not exist
tests/qapi-schema/doc-bad-feature.json +0 -1
index 2a78e3b1db..3d49b8e607 100644
@@ -1,5 +1,4 @@
 # Features listed in the doc comment must exist in the actual schema
-# BUG: nonexistent @a is not rejected
 
 ##
 # @foo:
tests/qapi-schema/doc-bad-feature.out +0 -19
index fef4a3e400..e69de29bb2 100644
@@ -1,19 +0,0 @@
-module None
-object q_empty
-enum QType
-    prefix QTYPE
-    member none
-    member qnull
-    member qnum
-    member qstring
-    member qdict
-    member qlist
-    member qbool
-module doc-bad-feature.json
-command foo None -> None
-    gen=True success_response=True boxed=False oob=False preconfig=False
-doc symbol=foo
-    body=
-
-    feature=a
-a
tests/qapi-schema/doc-undoc-feature.err +2 -0
index e69de29bb2..62fc82d2b9 100644
@@ -0,0 +1,2 @@
+doc-undoc-feature.json: In command 'foo':
+doc-undoc-feature.json:9: feature 'undoc' lacks documentation
tests/qapi-schema/doc-undoc-feature.json +0 -1
index c7650d9974..c52f88e2cd 100644
@@ -1,5 +1,4 @@
 # Doc comment must cover all features
-# BUG: missing documentation for @undoc not caught
 
 ##
 # @foo:
tests/qapi-schema/doc-undoc-feature.out +0 -21
index cdb097361f..e69de29bb2 100644
@@ -1,21 +0,0 @@
-module None
-object q_empty
-enum QType
-    prefix QTYPE
-    member none
-    member qnull
-    member qnum
-    member qstring
-    member qdict
-    member qlist
-    member qbool
-module doc-undoc-feature.json
-command foo None -> None
-    gen=True success_response=True boxed=False oob=False preconfig=False
-    feature undoc
-    feature doc
-doc symbol=foo
-    body=
-
-    feature=doc
-documented feature
Signature · 1 line
2.21.0