Skip to contentSkip to conversation

qapi: add formal "intro" section

PATCH Superseded John Snow 2 participants 18 messages Apr 23 – Apr 28

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

John Snow

author
#0 lore raw

cover qapi: add formal "intro" section

Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.

It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.

This series demonstrates conversion of just four modules; if I'm given a thumbs up, I will convert the rest of QAPI, one module (or maintainer stanza) per patch like how I handled adding cross-references.

John Snow (12):
tests/qapi: generate output in source order qapi/docs: remove unused QAPIDoc subsection members qapi/docs: make remaining subsection members "private" qapi/docs: add "Intro" section
qapi/docs: adjust stub member insertion algorithm qapi/docs: remove implicit Plain section qapi/docs: add "Intro" section parsing qapi/docs: Add rendering for INTRO sections
qapi: convert intro sections for accelerator.json
qapi: convert intro sections for acpi-hest.json
qapi: convert intro sections for acpi.json
qapi: convert intro sections for audio.json

 docs/devel/qapi-code-gen.rst            |  17 +-
 docs/sphinx/qapidoc.py                  |  52 +++---
 qapi/accelerator.json                   |  24 +--
 qapi/acpi-hest.json                     |   5 +-
 qapi/acpi.json                          |  23 +--
 qapi/audio.json                         | 112 ++++---------
 .editorconfig                           |   1 +
 scripts/qapi/parser.py                  | 204 ++++++++++++++----------
 tests/qapi-schema/doc-good.out          |  30 ++--
 tests/qapi-schema/doc-missing-colon.err |   2 +-
 tests/qapi-schema/test-qapi.py          |  15 +-
 11 files changed, 239 insertions(+), 246 deletions(-)
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH 1/12 tests/qapi: generate output in source order

Rewrite the test doc generator to produce output in source order instead of arbitrarily by section name.

This patch removes our last use of the "body" field, which has an effect on how the first plaintext section of each test documented is printed:
we now print "section=Plain" followed by the section text instead of "body=[...]".

This patch is motivated by a desire to move the QAPIDoc API away from named fields for specific sections in a bid to force all users to simply iterate through all_sections in order, instead - and to remove the named subsections.

---
 tests/qapi-schema/doc-good.out | 26 +++++++++++++-------------
 tests/qapi-schema/test-qapi.py | 15 ++++++++-------
 2 files changed, 21 insertions(+), 20 deletions(-)
tests/qapi-schema/doc-good.out +13 -13
index 04a55072646..e2be6f96bbf 100644
@@ -54,15 +54,15 @@ event EVT_BOXED Object
     boxed=True
     feature feat3
 doc freeform
-    body=
+    section=Plain
 *******
 Section
 *******
 doc freeform
-    body=
+    section=Plain
 Just text, no heading.
 doc freeform
-    body=
+    section=Plain
 Subsection
 ==========
 
@@ -106,7 +106,7 @@ Examples:
 - *verbatim*
 - {braces}
 doc symbol=Enum
-    body=
+    section=Plain
 
     arg=one
 The _one_ {and only}, description on the same line
@@ -119,13 +119,13 @@ a member feature
     section=Plain
 @two is undocumented
 doc symbol=Base
-    body=
+    section=Plain
 
     arg=base1
  description starts on a new line,
  minimally indented
 doc symbol=Variant1
-    body=
+    section=Plain
 A paragraph
 
 Another paragraph
@@ -138,15 +138,15 @@ a feature
     feature=member-feat
 a member feature
 doc symbol=Variant2
-    body=
+    section=Plain
 
 doc symbol=Object
-    body=
+    section=Plain
 
     feature=union-feat1
 a feature
 doc symbol=Alternate
-    body=
+    section=Plain
 
     arg=i
 description starts on the same line
@@ -157,11 +157,11 @@ description starts on the same line
     feature=alt-feat
 a feature
 doc freeform
-    body=
+    section=Plain
 Another subsection
 ==================
 doc symbol=cmd
-    body=
+    section=Plain
 
     arg=arg1
     description starts on a new line,
@@ -210,7 +210,7 @@ Note::
     section=Since
 2.10
 doc symbol=cmd-boxed
-    body=
+    section=Plain
 If you're bored enough to read this, go see a video of boxed cats
     feature=cmd-feat1
 a feature
@@ -223,7 +223,7 @@ another feature
 
    <- ... has no title ...
 doc symbol=EVT_BOXED
-    body=
+    section=Plain
 
     feature=feat3
 a feature
tests/qapi-schema/test-qapi.py +8 -7
index cf7fb8a6df5..27885147b4d 100755
@@ -19,6 +19,7 @@
 from io import StringIO
 
 from qapi.error import QAPIError
+from qapi.parser import QAPIDoc
 from qapi.schema import QAPISchema, QAPISchemaVisitor
 
 
@@ -116,13 +117,13 @@ def test_frontend(fname):
             print('doc symbol=%s' % doc.symbol)
         else:
             print('doc freeform')
-        print('    body=\n%s' % doc.body.text)
-        for arg, section in doc.args.items():
-            print('    arg=%s\n%s' % (arg, section.text))
-        for feat, section in doc.features.items():
-            print('    feature=%s\n%s' % (feat, section.text))
-        for section in doc.sections:
-            print('    section=%s\n%s' % (section.kind, section.text))
+        for section in doc.all_sections:
+            if section.kind == QAPIDoc.Kind.MEMBER:
+                print('    arg=%s\n%s' % (section.name, section.text))
+            elif section.kind == QAPIDoc.Kind.FEATURE:
+                print('    feature=%s\n%s' % (section.name, section.text))
+            else:
+                print('    section=%s\n%s' % (section.kind, section.text))
 
 
 def open_test_result(dir_name, file_name, update):
Signature · 1 line
2.53.0
author
#2 lore raw

PATCH 2/12 qapi/docs: remove unused QAPIDoc subsection members

In reply to John Snow

All users of the QAPIDoc object should be iterating over all_sections and not grabbing arbitrary subsections, if possible. Remove the 'body' and 'sections' members, as they are no longer used.

---
 scripts/qapi/parser.py | 6 ------
 1 file changed, 6 deletions(-)
scripts/qapi/parser.py +0 -6
index c3cf33904ef..b33edbba74f 100644
@@ -732,8 +732,6 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
         self.all_sections: List[QAPIDoc.Section] = [
             QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
         ]
-        # the body section
-        self.body: Optional[QAPIDoc.Section] = self.all_sections[0]
         # dicts mapping parameter/feature names to their description
         self.args: Dict[str, QAPIDoc.ArgSection] = {}
         self.features: Dict[str, QAPIDoc.ArgSection] = {}
@@ -742,8 +740,6 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
         self.errors: Optional[QAPIDoc.Section] = None
         # "Since" section
         self.since: Optional[QAPIDoc.Section] = None
-        # sections other than .body, .args, .features
-        self.sections: List[QAPIDoc.Section] = []
 
     def end(self) -> None:
         for section in self.all_sections:
@@ -766,7 +762,6 @@ def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
 
         # start new section
         section = self.Section(info, kind)
-        self.sections.append(section)
         self.all_sections.append(section)
 
     def new_tagged_section(
@@ -790,7 +785,6 @@ def new_tagged_section(
                 raise QAPISemError(
                     info, "duplicated '%s' section" % kind)
             self.since = section
-        self.sections.append(section)
         self.all_sections.append(section)
 
     def _new_description(
Signature · 1 line
2.53.0
author
#3 lore raw

PATCH 3/12 qapi/docs: make remaining subsection members "private"

In reply to John Snow

These fields are used to provide error checking and internal logistics and should not be used by a user of the library to directly access documentation sections, so make them private.

The "since" field alone is left public, as the qapidoc generator does use this field to pull that section out of the regular flow of the document.

---
 scripts/qapi/parser.py | 54 +++++++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 25 deletions(-)
scripts/qapi/parser.py +29 -25
index b33edbba74f..da4756a7424 100644
@@ -555,7 +555,7 @@ def get_doc(self) -> 'QAPIDoc':
                     break
                 # Non-blank line, first of a section
                 if line == 'Features:':
-                    if doc.features:
+                    if doc.has_features:
                         raise QAPIParseError(
                             self, "duplicated 'Features:' line")
                     self.accept(False)
@@ -570,7 +570,7 @@ def get_doc(self) -> 'QAPIDoc':
                         if text:
                             doc.append_line(text)
                         line = self.get_doc_indented(doc)
-                    if not doc.features:
+                    if not doc.has_features:
                         raise QAPIParseError(
                             self, 'feature descriptions expected')
                     no_more_args = True
@@ -733,14 +733,18 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
             QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
         ]
         # dicts mapping parameter/feature names to their description
-        self.args: Dict[str, QAPIDoc.ArgSection] = {}
-        self.features: Dict[str, QAPIDoc.ArgSection] = {}
+        self._args: Dict[str, QAPIDoc.ArgSection] = {}
+        self._features: Dict[str, QAPIDoc.ArgSection] = {}
         # a command's "Returns" and "Errors" section
-        self.returns: Optional[QAPIDoc.Section] = None
-        self.errors: Optional[QAPIDoc.Section] = None
+        self._returns: Optional[QAPIDoc.Section] = None
+        self._errors: Optional[QAPIDoc.Section] = None
         # "Since" section
         self.since: Optional[QAPIDoc.Section] = None
 
+    @property
+    def has_features(self) -> bool:
+        return bool(self._features)
+
     def end(self) -> None:
         for section in self.all_sections:
             section.text = section.text.strip('\n')
@@ -771,15 +775,15 @@ def new_tagged_section(
     ) -> None:
         section = self.Section(info, kind)
         if kind == QAPIDoc.Kind.RETURNS:
-            if self.returns:
+            if self._returns:
                 raise QAPISemError(
                     info, "duplicated '%s' section" % kind)
-            self.returns = section
+            self._returns = section
         elif kind == QAPIDoc.Kind.ERRORS:
-            if self.errors:
+            if self._errors:
                 raise QAPISemError(
                     info, "duplicated '%s' section" % kind)
-            self.errors = section
+            self._errors = section
         elif kind == QAPIDoc.Kind.SINCE:
             if self.since:
                 raise QAPISemError(
@@ -803,16 +807,16 @@ def _new_description(
         desc[name] = section
 
     def new_argument(self, info: QAPISourceInfo, name: str) -> None:
-        self._new_description(info, name, QAPIDoc.Kind.MEMBER, self.args)
+        self._new_description(info, name, QAPIDoc.Kind.MEMBER, self._args)
 
     def new_feature(self, info: QAPISourceInfo, name: str) -> None:
-        self._new_description(info, name, QAPIDoc.Kind.FEATURE, self.features)
+        self._new_description(info, name, QAPIDoc.Kind.FEATURE, self._features)
 
     def append_line(self, line: str) -> None:
         self.all_sections[-1].append_line(line)
 
     def connect_member(self, member: 'QAPISchemaMember') -> None:
-        if member.name not in self.args:
+        if member.name not in self._args:
             assert member.info
             if self.symbol not in member.info.pragma.documentation_exceptions:
                 raise QAPISemError(member.info,
@@ -823,7 +827,7 @@ def connect_member(self, member: 'QAPISchemaMember') -> None:
 
             section = QAPIDoc.ArgSection(
                 self.info, QAPIDoc.Kind.MEMBER, member.name)
-            self.args[member.name] = section
+            self._args[member.name] = section
 
             # Determine where to insert stub doc - it should go at the
             # end of the members section(s), if any. Note that index 0
@@ -835,14 +839,14 @@ def connect_member(self, member: 'QAPISchemaMember') -> None:
                     index += 1
             self.all_sections.insert(index, section)
 
-        self.args[member.name].connect(member)
+        self._args[member.name].connect(member)
 
     def connect_feature(self, feature: 'QAPISchemaFeature') -> None:
-        if feature.name not in self.features:
+        if feature.name not in self._features:
             raise QAPISemError(feature.info,
                                "feature '%s' lacks documentation"
                                % feature.name)
-        self.features[feature.name].connect(feature)
+        self._features[feature.name].connect(feature)
 
     def ensure_returns(self, info: QAPISourceInfo) -> None:
 
@@ -883,18 +887,18 @@ def _insert_near_kind(
 
     def check_expr(self, expr: QAPIExpression) -> None:
         if 'command' in expr:
-            if self.returns and 'returns' not in expr:
+            if self._returns and 'returns' not in expr:
                 raise QAPISemError(
-                    self.returns.info,
+                    self._returns.info,
                     "'Returns' section, but command doesn't return anything")
         else:
-            if self.returns:
+            if self._returns:
                 raise QAPISemError(
-                    self.returns.info,
+                    self._returns.info,
                     "'Returns' section is only valid for commands")
-            if self.errors:
+            if self._errors:
                 raise QAPISemError(
-                    self.errors.info,
+                    self._errors.info,
                     "'Errors' section is only valid for commands")
 
     def check(self) -> None:
@@ -914,5 +918,5 @@ def check_args_section(
                         "do" if len(bogus) > 1 else "does"
                     ))
 
-        check_args_section(self.args, 'member')
-        check_args_section(self.features, 'feature')
+        check_args_section(self._args, 'member')
+        check_args_section(self._features, 'feature')
Signature · 1 line
2.53.0
author
#4 lore raw

PATCH 4/12 qapi/docs: add "Intro" section

In reply to John Snow

This patch adds an "Intro" doc section, intended to eventually replace the "Plain" doc section alongside a forthcoming "Details" section.

For now, this section is not actually instantiated or used, but subsequent patches will slowly convert the leading introductory sections of QAPIDoc documentation to use the new Intro section.

A main motivation of this series of changes is to more explicitly delineate the "Introductory" documentation for each QAPI definition for the sake of the inliner. When inlining members, examples, and details from multiple QAPIDoc sections, we will want to omit the "Introductory" text from inlined definitions while keeping notes, caution boxes, examples, and so on.

---
 docs/sphinx/qapidoc.py | 2 +-
 scripts/qapi/parser.py | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)
docs/sphinx/qapidoc.py +1 -1
index c2f09bac16c..1f7c15b7075 100644
@@ -368,7 +368,7 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
         for i, section in enumerate(sections):
             section.text = self.reformat_arobase(section.text)
 
-            if section.kind == QAPIDoc.Kind.PLAIN:
+            if section.kind.name in ("PLAIN", "INTRO"):
                 self.visit_paragraph(section)
             elif section.kind == QAPIDoc.Kind.MEMBER:
                 assert isinstance(section, QAPIDoc.ArgSection)
scripts/qapi/parser.py +3 -2
index da4756a7424..97e7dacb0fd 100644
@@ -631,7 +631,7 @@ def get_doc(self) -> 'QAPIDoc':
                     line = self.get_doc_indented(doc)
                     no_more_args = True
                 else:
-                    # plain paragraph
+                    # plain paragraph(s)
                     doc.ensure_untagged_section(self.info)
                     doc.append_line(line)
                     line = self.get_doc_paragraph(doc)
@@ -681,6 +681,7 @@ class Kind(enum.Enum):
         ERRORS = 4
         SINCE = 5
         TODO = 6
+        INTRO = 7
 
         @staticmethod
         def from_string(kind: str) -> 'QAPIDoc.Kind':
@@ -748,7 +749,7 @@ def has_features(self) -> bool:
     def end(self) -> None:
         for section in self.all_sections:
             section.text = section.text.strip('\n')
-            if section.kind != QAPIDoc.Kind.PLAIN and section.text == '':
+            if not (section.kind.name in ("INTRO", "PLAIN") or section.text):
                 raise QAPISemError(
                     section.info, "text required after '%s:'" % section.kind)
 
Signature · 1 line
2.53.0
author
#5 lore raw

PATCH 5/12 qapi/docs: adjust stub member insertion algorithm

In reply to John Snow

A forthcoming patch removes the implicit PLAIN section that always starts a QAPIDoc section list. Further future changes begin converting "PLAIN" sections to "INTRO" sections. To accommodate this, the insertion algorithm that places stub and dummy members must be adjusted to cope.

This algorithm can handle zero-or-more PLAIN *or* INTRO sections at the beginning of a QAPIDoc object.

Since we have three places that need to insert stub members, take the opportunity to unify and deduplicate this code.

---
 docs/sphinx/qapidoc.py | 36 ++++++++---------
 scripts/qapi/parser.py | 90 +++++++++++++++++++++++++++---------------
 2 files changed, 75 insertions(+), 51 deletions(-)
docs/sphinx/qapidoc.py +17 -19
index 1f7c15b7075..70ab9cdc214 100644
@@ -349,30 +349,32 @@ def _get_target(
                 )
 
     def visit_sections(self, ent: QAPISchemaDefinition) -> None:
+        # Generate a placeholder right after the member section(s) which
+        # will be used to generate documentation for "The members of..."
+        # pointers in the rendered document.
+        # This is a temporary hack until the inliner is merged.
+        if ent.doc:
+            ent.doc.append_member_stub(
+                QAPIDoc.ArgSection(
+                    ent.doc.info, QAPIDoc.Kind.MEMBER, "q_dummy"
+                )
+            )
+
         sections = ent.doc.all_sections if ent.doc else []
 
-        # Determine the index location at which we should generate
-        # documentation for "The members of ..." pointers. This should
-        # go at the end of the members section(s) if any. Note that
-        # index 0 is assumed to be a plain intro section, even if it is
-        # empty; and that a members section if present will always
-        # immediately follow the opening PLAIN section.
-        gen_index = 1
-        if len(sections) > 1:
-            while sections[gen_index].kind == QAPIDoc.Kind.MEMBER:
-                gen_index += 1
-                if gen_index >= len(sections):
-                    break
-
         # Add sections in source order:
-        for i, section in enumerate(sections):
+        for section in sections:
             section.text = self.reformat_arobase(section.text)
 
             if section.kind.name in ("PLAIN", "INTRO"):
                 self.visit_paragraph(section)
             elif section.kind == QAPIDoc.Kind.MEMBER:
                 assert isinstance(section, QAPIDoc.ArgSection)
-                self.visit_member(section)
+                if section.name == "q_dummy":
+                    # Generate "The members of ..." entries if necessary
+                    self._insert_member_pointer(ent)
+                else:
+                    self.visit_member(section)
             elif section.kind == QAPIDoc.Kind.FEATURE:
                 assert isinstance(section, QAPIDoc.ArgSection)
                 self.visit_feature(section)
@@ -386,10 +388,6 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
             else:
                 assert False
 
-            # Generate "The members of ..." entries if necessary:
-            if i == gen_index - 1:
-                self._insert_member_pointer(ent)
-
         self.ensure_blank_line()
 
     # Transmogrification core methods
scripts/qapi/parser.py +58 -32

121 lines not on this page · raw message

Signature · 1 line
2.53.0
author
#6 lore raw

PATCH 6/12 qapi/docs: remove implicit Plain section

In reply to John Snow

Prior to this patch, we always instantiate an empty Plain section. Removing this allows us to gradually phase out the "Plain" section in favor of "Intro" and "Details" sections while keeping "Plain" around for the interim churn during the series - meaning that we don't actually know at __init__ time which type of section we'll have first.

---
 scripts/qapi/parser.py         |  4 +---
 tests/qapi-schema/doc-good.out | 14 --------------
 2 files changed, 1 insertion(+), 17 deletions(-)
scripts/qapi/parser.py +1 -3
index b21796b3e80..6612f471bb8 100644
@@ -730,9 +730,7 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
         # definition doc's symbol, None for free-form doc
         self.symbol: Optional[str] = symbol
         # the sections in textual order
-        self.all_sections: List[QAPIDoc.Section] = [
-            QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
-        ]
+        self.all_sections: List[QAPIDoc.Section] = []
         # dicts mapping parameter/feature names to their description
         self._args: Dict[str, QAPIDoc.ArgSection] = {}
         self._features: Dict[str, QAPIDoc.ArgSection] = {}
tests/qapi-schema/doc-good.out +0 -14
index e2be6f96bbf..6fcc8175cfe 100644
@@ -106,8 +106,6 @@ Examples:
 - *verbatim*
 - {braces}
 doc symbol=Enum
-    section=Plain
-
     arg=one
 The _one_ {and only}, description on the same line
     arg=two
@@ -119,8 +117,6 @@ a member feature
     section=Plain
 @two is undocumented
 doc symbol=Base
-    section=Plain
-
     arg=base1
  description starts on a new line,
  minimally indented
@@ -138,16 +134,10 @@ a feature
     feature=member-feat
 a member feature
 doc symbol=Variant2
-    section=Plain
-
 doc symbol=Object
-    section=Plain
-
     feature=union-feat1
 a feature
 doc symbol=Alternate
-    section=Plain
-
     arg=i
 description starts on the same line
     remainder indented the same
@@ -161,8 +151,6 @@ doc freeform
 Another subsection
 ==================
 doc symbol=cmd
-    section=Plain
-
     arg=arg1
     description starts on a new line,
     indented
@@ -223,7 +211,5 @@ another feature
 
    <- ... has no title ...
 doc symbol=EVT_BOXED
-    section=Plain
-
     feature=feat3
 a feature
Signature · 1 line
2.53.0
author
#7 lore raw

PATCH 7/12 qapi/docs: add "Intro" section parsing

In reply to John Snow

Add parsing for explicit Intro section syntax.

A side effect of this patch is that we will always create an empty Intro section, similar to how we used to have an empty Plain section. The tests are adjusted accordingly, rendered document output does not change at all.

---
 docs/devel/qapi-code-gen.rst            | 17 +++++----
 scripts/qapi/parser.py                  | 47 ++++++++++++++++++-------
 tests/qapi-schema/doc-good.out          | 18 ++++++++++
 tests/qapi-schema/doc-missing-colon.err |  2 +-
 4 files changed, 64 insertions(+), 20 deletions(-)
docs/devel/qapi-code-gen.rst +11 -6
index 3a632b4a648..a8175934d52 100644
@@ -862,7 +862,7 @@ documentation comment.
 If the documentation comment starts like ::
 
     ##
-    # @SYMBOL:
+    # @SYMBOL: [...]
 
 it documents the definition of SYMBOL, else it's free-form
 documentation.
@@ -984,11 +984,11 @@ definition it documents.
 When documentation is required (see pragma_ 'doc-required'), every
 definition must have documentation.
 
-Definition documentation starts with a line naming the definition,
-followed by an optional overview, a description of each argument (for
-commands and events), member (for structs and unions), branch (for
-alternates), or value (for enums), a description of each feature (if
-any), and finally optional tagged sections.
+Definition documentation starts with a description naming the definition
+with an optional overview, a description of each argument (for commands
+and events), member (for structs and unions), branch (for alternates),
+or value (for enums), a description of each feature (if any), and
+finally optional tagged sections.
 
 Descriptions start with '\@name:'.  The description text must be
 indented like this::
@@ -996,6 +996,11 @@ indented like this::
  # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
  #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.
 
+Definition descriptions are special: the optional introductory overview
+describing the definition will not be inlined when referenced by other
+definitions (such as when using 'base' to include members from another
+definition), while other descriptions and tagged sections will be.
+
 .. FIXME The parser accepts these things in almost any order.
 
 .. FIXME union branches should be described, too.
scripts/qapi/parser.py +34 -13

72 lines not on this page · raw message

tests/qapi-schema/doc-good.out +18 -0
index 6fcc8175cfe..bc89853765f 100644
@@ -106,6 +106,8 @@ Examples:
 - *verbatim*
 - {braces}
 doc symbol=Enum
+    section=Intro
+
     arg=one
 The _one_ {and only}, description on the same line
     arg=two
@@ -117,10 +119,14 @@ a member feature
     section=Plain
 @two is undocumented
 doc symbol=Base
+    section=Intro
+
     arg=base1
  description starts on a new line,
  minimally indented
 doc symbol=Variant1
+    section=Intro
+
     section=Plain
 A paragraph
 
@@ -134,10 +140,16 @@ a feature
     feature=member-feat
 a member feature
 doc symbol=Variant2
+    section=Intro
+
 doc symbol=Object
+    section=Intro
+
     feature=union-feat1
 a feature
 doc symbol=Alternate
+    section=Intro
+
     arg=i
 description starts on the same line
     remainder indented the same
@@ -151,6 +163,8 @@ doc freeform
 Another subsection
 ==================
 doc symbol=cmd
+    section=Intro
+
     arg=arg1
     description starts on a new line,
     indented
@@ -198,6 +212,8 @@ Note::
     section=Since
 2.10
 doc symbol=cmd-boxed
+    section=Intro
+
     section=Plain
 If you're bored enough to read this, go see a video of boxed cats
     feature=cmd-feat1
@@ -211,5 +227,7 @@ another feature
 
    <- ... has no title ...
 doc symbol=EVT_BOXED
+    section=Intro
+
     feature=feat3
 a feature
tests/qapi-schema/doc-missing-colon.err +1 -1
index cbcea007153..bd5862b30f3 100644
@@ -1 +1 @@
-doc-missing-colon.json:4:1: line should end with ':'
+doc-missing-colon.json:4:1: @name must end with ':'
Signature · 1 line
2.53.0
author
#8 lore raw

PATCH 8/12 qapi/docs: Add rendering for INTRO sections

In reply to John Snow

The only real difference here is that we need to dedent all but the first line so that it renders correctly. We don't need to do this for members and features because they are always rendered as part of a field list directive which expects indented lines - Undecorated plaintext doesn't, so we chop the indent off.

This does not reflow the text or mess with the source info in any way, so "blame" for error messages should be unchanged.

---
 docs/sphinx/qapidoc.py | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
docs/sphinx/qapidoc.py +11 -3
index 70ab9cdc214..6b8e4ecd76a 100644
@@ -35,6 +35,7 @@
 from pathlib import Path
 import re
 import sys
+import textwrap
 from typing import TYPE_CHECKING
 
 from docutils import nodes
@@ -150,8 +151,14 @@ def add_lines(
         self,
         content: str,
         info: QAPISourceInfo,
+        dedent: bool = False,
     ) -> None:
         lines = content.splitlines(True)
+
+        if dedent:
+            txt = "".join(lines[1:])
+            lines[1:] = textwrap.dedent(txt).splitlines(True)
+
         for i, line in enumerate(lines):
             self.add_line_raw(line, info.fname, info.line + i)
 
@@ -223,13 +230,14 @@ def reformat_arobase(text: str) -> str:
 
     # Transmogrification helpers
 
-    def visit_paragraph(self, section: QAPIDoc.Section) -> None:
+    def visit_text(self, section: QAPIDoc.Section) -> None:
         # Squelch empty paragraphs.
         if not section.text:
             return
 
+        dedent = bool(section.kind == QAPIDoc.Kind.INTRO)
         self.ensure_blank_line()
-        self.add_lines(section.text, section.info)
+        self.add_lines(section.text, section.info, dedent)
         self.ensure_blank_line()
 
     def visit_member(self, section: QAPIDoc.ArgSection) -> None:
@@ -367,7 +375,7 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
             section.text = self.reformat_arobase(section.text)
 
             if section.kind.name in ("PLAIN", "INTRO"):
-                self.visit_paragraph(section)
+                self.visit_text(section)
             elif section.kind == QAPIDoc.Kind.MEMBER:
                 assert isinstance(section, QAPIDoc.ArgSection)
                 if section.name == "q_dummy":
Signature · 1 line
2.53.0
author
#9 lore raw

PATCH 9/12 qapi: convert intro sections for accelerator.json

In reply to John Snow
---
 qapi/accelerator.json | 24 ++++++------------------
 1 file changed, 6 insertions(+), 18 deletions(-)
qapi/accelerator.json +6 -18
index 0cf5e0f9d94..71047538ee1 100644
@@ -12,9 +12,7 @@
 { 'include': 'common.json' }
 
 ##
-# @KvmInfo:
-#
-# Information about support for KVM acceleration
+# @KvmInfo: Information about support for KVM acceleration
 #
 # @enabled: true if KVM acceleration is active
 #
@@ -25,9 +23,7 @@
 { 'struct': 'KvmInfo', 'data': {'enabled': 'bool', 'present': 'bool'} }
 
 ##
-# @query-kvm:
-#
-# Return information about KVM acceleration
+# @query-kvm: Return information about KVM acceleration
 #
 # Since: 0.14
 #
@@ -39,9 +35,7 @@
 { 'command': 'query-kvm', 'returns': 'KvmInfo' }
 
 ##
-# @x-accel-stats:
-#
-# Query accelerator statistics
+# @x-accel-stats: Query accelerator statistics
 #
 # Features:
 #
@@ -56,9 +50,7 @@
   'features': [ 'unstable' ] }
 
 ##
-# @Accelerator:
-#
-# Information about support for MSHV acceleration
+# @Accelerator: Information about support for MSHV acceleration
 #
 # @hvf: Apple Hypervisor.framework
 #
@@ -81,9 +73,7 @@
 { 'enum': 'Accelerator', 'data': ['hvf', 'kvm', 'mshv', 'nvmm', 'qtest', 'tcg', 'whpx', 'xen'] }
 
 ##
-# @AcceleratorInfo:
-#
-# Information about support for various accelerators
+# @AcceleratorInfo: Information about support for various accelerators
 #
 # @enabled: the accelerator that is in use
 #
@@ -95,9 +85,7 @@
 { 'struct': 'AcceleratorInfo', 'data': {'enabled': 'Accelerator', 'present': ['Accelerator']} }
 
 ##
-# @query-accelerators:
-#
-# Return information about accelerators
+# @query-accelerators: Return information about accelerators
 #
 # Returns: @AcceleratorInfo
 #
Signature · 1 line
2.53.0
author
#10 lore raw

PATCH 10/12 qapi: convert intro sections for acpi-hest.json

In reply to John Snow
---
 qapi/acpi-hest.json | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
qapi/acpi-hest.json +2 -3
index a01f1dee095..6a48e80082e 100644
@@ -12,9 +12,8 @@
 ##
 
 ##
-# @inject-ghes-v2-error:
-#
-# Inject an error with additional ACPI 6.1 GHESv2 error information
+# @inject-ghes-v2-error: Inject an error with additional ACPI 6.1
+#     GHESv2 error information
 #
 # @cper: contains a base64 encoded string with raw data for a single
 #     CPER record with Generic Error Status Block, Generic Error Data
Signature · 1 line
2.53.0
author
#11 lore raw

PATCH 11/12 qapi: convert intro sections for acpi.json

In reply to John Snow
---
 qapi/acpi.json | 23 ++++++++---------------
 1 file changed, 8 insertions(+), 15 deletions(-)
qapi/acpi.json +8 -15
index 906b3687a55..ea6ad2cd45f 100644
@@ -12,9 +12,8 @@
 ##
 
 ##
-# @AcpiTableOptions:
-#
-# Specify an ACPI table on the command line to load.
+# @AcpiTableOptions: Specify an ACPI table on the
+#     command line to load.
 #
 # At most one of @file and @data can be specified.  The list of files
 # specified by any one of them is loaded and concatenated in order.
@@ -80,11 +79,9 @@
 { 'enum': 'ACPISlotType', 'data': [ 'DIMM', 'CPU' ] }
 
 ##
-# @ACPIOSTInfo:
-#
-# OSPM Status Indication for a device.  For description of possible
-# values of @source and @status fields see "_OST (OSPM Status
-# Indication)" chapter of ACPI5.0 spec.
+# @ACPIOSTInfo: OSPM Status Indication for a device.  For description
+#     of possible values of @source and @status fields see "_OST (OSPM
+#     Status Indication)" chapter of ACPI5.0 spec.
 #
 # @device: device ID associated with slot
 #
@@ -106,10 +103,8 @@
               'status': 'int' } }
 
 ##
-# @query-acpi-ospm-status:
-#
-# Return a list of `ACPIOSTInfo` for devices that support status
-# reporting via ACPI _OST method.
+# @query-acpi-ospm-status: Return a list of `ACPIOSTInfo` for devices
+#     that support status reporting via ACPI _OST method.
 #
 # Since: 2.1
 #
@@ -125,9 +120,7 @@
 { 'command': 'query-acpi-ospm-status', 'returns': ['ACPIOSTInfo'] }
 
 ##
-# @ACPI_DEVICE_OST:
-#
-# Emitted when guest executes ACPI _OST method.
+# @ACPI_DEVICE_OST: Emitted when guest executes ACPI _OST method.
 #
 # @info: OSPM Status Indication
 #
Signature · 1 line
2.53.0
author
#12 lore raw

PATCH 12/12 qapi: convert intro sections for audio.json

In reply to John Snow
---
 qapi/audio.json | 112 ++++++++++++++----------------------------------
 .editorconfig   |   1 +
 2 files changed, 33 insertions(+), 80 deletions(-)
qapi/audio.json +32 -80
index 2df87b97101..62d64eabf1c 100644
@@ -13,10 +13,8 @@
 ##
 
 ##
-# @AudiodevPerDirectionOptions:
-#
-# General audio backend options that are used for both playback and
-# recording.
+# @AudiodevPerDirectionOptions: General audio backend options that are
+#     used for both playback and recording.
 #
 # @mixing-engine: use QEMU's mixing engine to mix all streams inside
 #     QEMU and convert audio formats when not supported by the
@@ -52,9 +50,7 @@
     '*buffer-length':  'uint32' } }
 
 ##
-# @AudiodevGenericOptions:
-#
-# Generic driver-specific options.
+# @AudiodevGenericOptions: Generic driver-specific options.
 #
 # @in: options of the capture stream
 #
@@ -68,9 +64,7 @@
     '*out': 'AudiodevPerDirectionOptions' } }
 
 ##
-# @AudiodevDBusOptions:
-#
-# Options of the D-Bus audio backend.
+# @AudiodevDBusOptions: Options of the D-Bus audio backend.
 #
 # @in: options of the capture stream
 #
@@ -88,10 +82,8 @@
     '*nsamples': 'uint32'} }
 
 ##
-# @AudiodevAlsaPerDirectionOptions:
-#
-# Options of the ALSA backend that are used for both playback and
-# recording.
+# @AudiodevAlsaPerDirectionOptions: Options of the ALSA backend that
+#     are used for both playback and recording.
 #
 # @dev: the name of the ALSA device to use (default 'default')
 #
@@ -110,9 +102,7 @@
     '*try-poll':      'bool' } }
 
 ##
-# @AudiodevAlsaOptions:
-#
-# Options of the ALSA audio backend.
+# @AudiodevAlsaOptions: Options of the ALSA audio backend.
 #
 # @in: options of the capture stream
 #
@@ -129,9 +119,7 @@
     '*threshold': 'uint32' } }
 
 ##
-# @AudiodevSndioOptions:
-#
-# Options of the sndio audio backend.
+# @AudiodevSndioOptions: Options of the sndio audio backend.
 #
 # @in: options of the capture stream
 #
@@ -151,10 +139,8 @@
     '*latency':   'uint32'} }
 
 ##
-# @AudiodevCoreaudioPerDirectionOptions:
-#
-# Options of the Core Audio backend that are used for both playback
-# and recording.
+# @AudiodevCoreaudioPerDirectionOptions: Options of the Core Audio
+#     backend that are used for both playback and recording.
 #
 # @buffer-count: number of buffers
 #
@@ -166,9 +152,7 @@
     '*buffer-count': 'uint32' } }
 
 ##
-# @AudiodevCoreaudioOptions:
-#
-# Options of the coreaudio audio backend.
+# @AudiodevCoreaudioOptions: Options of the coreaudio audio backend.
 #
 # @in: options of the capture stream
 #
@@ -182,9 +166,7 @@
     '*out': 'AudiodevCoreaudioPerDirectionOptions' } }
 
 ##
-# @AudiodevDsoundOptions:
-#
-# Options of the DirectSound audio backend.
+# @AudiodevDsoundOptions: Options of the DirectSound audio backend.
 #
 # @in: options of the capture stream
 #
@@ -202,10 +184,8 @@
     '*latency': 'uint32' } }
 
 ##
-# @AudiodevJackPerDirectionOptions:
-#
-# Options of the JACK backend that are used for both playback and
-# recording.
+# @AudiodevJackPerDirectionOptions: Options of the JACK backend that
+#     are used for both playback and recording.
 #
 # @server-name: select from among several possible concurrent server
 #     instances (default: environment variable $JACK_DEFAULT_SERVER if
@@ -236,9 +216,7 @@
     '*exact-name':    'bool' } }
 
 ##
-# @AudiodevJackOptions:
-#
-# Options of the JACK audio backend.
+# @AudiodevJackOptions: Options of the JACK audio backend.
 #
 # @in: options of the capture stream
 #
@@ -252,10 +230,8 @@
     '*out': 'AudiodevJackPerDirectionOptions' } }
 
 ##
-# @AudiodevOssPerDirectionOptions:
-#
-# Options of the OSS backend that are used for both playback and
-# recording.
+# @AudiodevOssPerDirectionOptions: Options of the OSS backend that are
+#     used for both playback and recording.
 #
 # @dev: file name of the OSS device (default '/dev/dsp')
 #
@@ -274,9 +250,7 @@
     '*try-poll':     'bool' } }
 
 ##
-# @AudiodevOssOptions:
-#
-# Options of the OSS audio backend.
+# @AudiodevOssOptions: Options of the OSS audio backend.
 #
 # @in: options of the capture stream
 #
@@ -304,10 +278,8 @@
     '*dsp-policy': 'uint32' } }
 
 ##
-# @AudiodevPaPerDirectionOptions:
-#
-# Options of the Pulseaudio backend that are used for both playback
-# and recording.
+# @AudiodevPaPerDirectionOptions: Options of the Pulseaudio backend
+#     that are used for both playback and recording.
 #
 # @name: name of the sink/source to use
 #
@@ -329,9 +301,7 @@
     '*latency': 'uint32' } }
 
 ##
-# @AudiodevPaOptions:
-#
-# Options of the PulseAudio audio backend.
+# @AudiodevPaOptions: Options of the PulseAudio audio backend.
 #
 # @in: options of the capture stream
 #
@@ -348,10 +318,8 @@
     '*server': 'str' } }
 
 ##
-# @AudiodevPipewirePerDirectionOptions:
-#
-# Options of the PipeWire backend that are used for both playback and
-# recording.
+# @AudiodevPipewirePerDirectionOptions: Options of the PipeWire
+#     backend that are used for both playback and recording.
 #
 # @name: name of the sink/source to use
 #
@@ -373,9 +341,7 @@
     '*latency': 'uint32' } }
 
 ##
-# @AudiodevPipewireOptions:
-#
-# Options of the PipeWire audio backend.
+# @AudiodevPipewireOptions: Options of the PipeWire audio backend.
 #
 # @in: options of the capture stream
 #
@@ -389,10 +355,8 @@
     '*out':    'AudiodevPipewirePerDirectionOptions' } }
 
 ##
-# @AudiodevSdlPerDirectionOptions:
-#
-# Options of the SDL audio backend that are used for both playback and
-# recording.
+# @AudiodevSdlPerDirectionOptions: Options of the SDL audio backend
+#     that are used for both playback and recording.
 #
 # @buffer-count: number of buffers (default 4)
 #
@@ -404,9 +368,7 @@
     '*buffer-count': 'uint32' } }
 
 ##
-# @AudiodevSdlOptions:
-#
-# Options of the SDL audio backend.
+# @AudiodevSdlOptions: Options of the SDL audio backend.
 #
 # @in: options of the recording stream
 #
@@ -420,9 +382,7 @@
     '*out': 'AudiodevSdlPerDirectionOptions' } }
 
 ##
-# @AudiodevWavOptions:
-#
-# Options of the wav audio backend.
+# @AudiodevWavOptions: Options of the wav audio backend.
 #
 # @in: options of the capture stream
 #
@@ -439,9 +399,7 @@
     '*path': 'str' } }
 
 ##
-# @AudioFormat:
-#
-# An enumeration of possible audio formats.
+# @AudioFormat: An enumeration of possible audio formats.
 #
 # @u8: unsigned 8 bit integer
 #
@@ -463,9 +421,7 @@
   'data': [ 'u8', 's8', 'u16', 's16', 'u32', 's32', 'f32' ] }
 
 ##
-# @AudiodevDriver:
-#
-# An enumeration of possible audio backend drivers.
+# @AudiodevDriver: An enumeration of possible audio backend drivers.
 #
 # @jack: JACK audio backend (since 5.1)
 #
@@ -487,9 +443,7 @@
             'wav' ] }
 
 ##
-# @Audiodev:
-#
-# Options of an audio backend.
+# @Audiodev: Options of an audio backend.
 #
 # @id: identifier of the backend
 #
@@ -533,9 +487,7 @@
     'wav':       'AudiodevWavOptions' } }
 
 ##
-# @query-audiodevs:
-#
-# Return information about audiodev configuration
+# @query-audiodevs: Return information about audiodev configuration
 #
 # Since: 8.0
 ##
.editorconfig +1 -0
index 258d41ab485..dcf8f6ded6d 100644
@@ -47,6 +47,7 @@ emacs_mode = glsl
 [*.json]
 indent_style = space
 emacs_mode = python
+max_line_length = 70
 
 # by default follow QEMU's style
 [*.pl]
Signature · 1 line
2.53.0
#13 lore raw
In reply to John Snow
Quoted 6 lines · John Snow

Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.

Let me elaborate a bit.

1. Why "intro"?

Generated documentation often refers to types like this:

    Command announce-self (Since: 4.0)

       Trigger generation of broadcast RARP frames to update network
       switches.  This can be useful when network bonds fail-over the
       active slave.

       Arguments:
          * The members of "AnnounceParameters".

       Example::

          -> { "execute": "announce-self",
               "arguments": {
                   "initial": 50, "max": 550, "rounds": 10, "step": 50,
                   "interfaces": ["vn2", "vn3"], "id": "bob" } }
          <- { "return": {} }

The arguments are hidden behind link "AnnounceParameters". Following leads to:

Object AnnounceParameters (Since: 4.0)

Parameters for self-announce timers

Members:
* initial ("int") -- Initial delay (in ms) before sending the first GARP/RARP announcement

* max ("int") -- Maximum delay (in ms) between GARP/RARP announcement packets

* rounds ("int") -- Number of self-announcement attempts

* step ("int") -- Delay increase (in ms) after each self- announcement attempt

* interfaces ("[string]", *optional*) -- An optional list of interface names, which restricts the announcement to the listed interfaces. (Since 4.1)

* id ("string", *optional*) -- A name to be used to identify an instance of announce-timers and to allow it to modified later. Not for use as part of the migration parameters. (Since 4.1)

This is bad UX. Especially when you have to chase multiple links.

John's doc inliner inlines AnnounceParameters documentation into announce-self documentation.

However, we don't want to inline "Parameters for self-announce timers".

Definitions typically start with a short description of the thing being defined. John calls this "intro". We don't want to inline these.

Syntactically, a doc comment begins with optional plain paragraphs. These end when something else begins: argument or member description, tagged section such as "Returns:", ...

Most of the time, these initial paragraphs are exactly the "intro". But not always.

Even without the inliner, we often need to know where "intro" ends. In the example above, "Arguments:" is auto-generated right after the "intro", and for that, the generator needs to know where "intro" ends.

In his "qapi: enforce section ordering" series, John proposed syntax to explicitly end "intro": a line "Details:". He had to add it to roughly one in 25 definition docs.

I fear adding "Details:" when needed is easily forgotten when it's so rarely needed. And I don't even trust myself to catch it patch review. So we looked for clearer syntax.

This series implements one: instead of letting intro end when something else starts, use indentation like we do for descriptions and tagged sections.

Let's have a look at the current doc source for announce-self:

    ##
    # @announce-self:
    #
    # Trigger generation of broadcast RARP frames to update network
    # switches.  This can be useful when network bonds fail-over the
    # active slave.
    #
    # TODO: This line is a hack to separate the example from the body
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

"Intro" ends when something else starts, namely the TODO: section. We need this hack so the doc generator inserts the "Arguments:" part where we want it.

John's "qapi: enforce section ordering" series replaces the TODO hack with a "Details:" line.

In new syntax, this instead becomes:

    ##
    # @announce-self: Trigger generation of broadcast RARP frames to
    #     update network switches.  This can be useful when network bonds
    #     fail-over the active slave.
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

Now "intro" is indented, and its end is obvious. Mistakes seem unlikely.

We could instead do:

    ##
    # @announce-self:
    #     Trigger generation of broadcast RARP frames to update network
    #     switches.  This can be useful when network bonds fail-over the
    #     active slave.
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

I think I like this a bit better.

Quoted 4 lines

It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.

Why not wholesale conversion? As long as generated output stays the same. It does in this series, except for harmless line breaks.

Quoted 4 lines

This series demonstrates conversion of just four modules; if I'm given a thumbs up, I will convert the rest of QAPI, one module (or maintainer stanza) per patch like how I handled adding cross-references.

author
#14 lore raw
Quoted 164 lines · Markus Armbruster
Quoted 6 lines · John Snow

Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.

Let me elaborate a bit.

1. Why "intro"?

Generated documentation often refers to types like this:

    Command announce-self (Since: 4.0)

       Trigger generation of broadcast RARP frames to update network
       switches.  This can be useful when network bonds fail-over the
       active slave.

       Arguments:
          * The members of "AnnounceParameters".

       Example::

          -> { "execute": "announce-self",
               "arguments": {
                   "initial": 50, "max": 550, "rounds": 10, "step": 50,
                   "interfaces": ["vn2", "vn3"], "id": "bob" } }
          <- { "return": {} }

The arguments are hidden behind link "AnnounceParameters". Following leads to:

Object AnnounceParameters (Since: 4.0)

Parameters for self-announce timers

Members:
* initial ("int") -- Initial delay (in ms) before sending the first GARP/RARP announcement

* max ("int") -- Maximum delay (in ms) between GARP/RARP announcement packets

* rounds ("int") -- Number of self-announcement attempts

* step ("int") -- Delay increase (in ms) after each self- announcement attempt

* interfaces ("[string]", *optional*) -- An optional list of interface names, which restricts the announcement to the listed interfaces. (Since 4.1)

* id ("string", *optional*) -- A name to be used to identify an instance of announce-timers and to allow it to modified later. Not for use as part of the migration parameters. (Since 4.1)

This is bad UX. Especially when you have to chase multiple links.

John's doc inliner inlines AnnounceParameters documentation into announce-self documentation.

However, we don't want to inline "Parameters for self-announce timers".

Definitions typically start with a short description of the thing being defined. John calls this "intro". We don't want to inline these.

Syntactically, a doc comment begins with optional plain paragraphs. These end when something else begins: argument or member description, tagged section such as "Returns:", ...

Most of the time, these initial paragraphs are exactly the "intro". But not always.

Even without the inliner, we often need to know where "intro" ends. In the example above, "Arguments:" is auto-generated right after the "intro", and for that, the generator needs to know where "intro" ends.

In his "qapi: enforce section ordering" series, John proposed syntax to explicitly end "intro": a line "Details:". He had to add it to roughly one in 25 definition docs.

I fear adding "Details:" when needed is easily forgotten when it's so rarely needed. And I don't even trust myself to catch it patch review. So we looked for clearer syntax.

This series implements one: instead of letting intro end when something else starts, use indentation like we do for descriptions and tagged sections.

Let's have a look at the current doc source for announce-self:

    ##
    # @announce-self:
    #
    # Trigger generation of broadcast RARP frames to update network
    # switches.  This can be useful when network bonds fail-over the
    # active slave.
    #
    # TODO: This line is a hack to separate the example from the body
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

"Intro" ends when something else starts, namely the TODO: section. We need this hack so the doc generator inserts the "Arguments:" part where we want it.

John's "qapi: enforce section ordering" series replaces the TODO hack with a "Details:" line.

In new syntax, this instead becomes:

    ##
    # @announce-self: Trigger generation of broadcast RARP frames to
    #     update network switches.  This can be useful when network bonds
    #     fail-over the active slave.
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

Now "intro" is indented, and its end is obvious. Mistakes seem unlikely.

We could instead do:

    ##
    # @announce-self:
    #     Trigger generation of broadcast RARP frames to update network
    #     switches.  This can be useful when network bonds fail-over the
    #     active slave.
    #
    # .. qmp-example::
    #
    #     -> { "execute": "announce-self",
    #          "arguments": {
    #              "initial": 50, "max": 550, "rounds": 10, "step": 50,
    #              "interfaces": ["vn2", "vn3"], "id": "bob" } }
    #     <- { "return": {} }
    #
    # Since: 4.0
    ##

I think I like this a bit better.

Quoted 8 lines
Quoted 4 lines

It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.

Why not wholesale conversion? As long as generated output stays the same. It does in this series, except for harmless line breaks.

When we last spoke, we weren't convinced this was the right approach and we both wanted to see it. So I did enough to get a taste.

Quoted 6 lines
Quoted 4 lines

This series demonstrates conversion of just four modules; if I'm given a thumbs up, I will convert the rest of QAPI, one module (or maintainer stanza) per patch like how I handled adding cross-references.

#15 lore raw
Quoted 10 lines · John Snow
Quoted 8 lines · Markus Armbruster
Quoted 6 lines · John Snow

Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.

[...]

Quoted 6 lines
Quoted 3 lines

Why not wholesale conversion? As long as generated output stays the same. It does in this series, except for harmless line breaks.

When we last spoke, we weren't convinced this was the right approach and we both wanted to see it. So I did enough to get a taste.

Yes, and it makes sense.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This series demonstrates conversion of just four modules; if I'm given a thumbs up, I will convert the rest of QAPI, one module (or maintainer stanza) per patch like how I handled adding cross-references.

author
#16 lore raw
In reply to John Snow
Quoted 31 lines · John Snow

Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.

It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.

This series demonstrates conversion of just four modules; if I'm given a thumbs up, I will convert the rest of QAPI, one module (or maintainer stanza) per patch like how I handled adding cross-references.

John Snow (12):
tests/qapi: generate output in source order qapi/docs: remove unused QAPIDoc subsection members qapi/docs: make remaining subsection members "private" qapi/docs: add "Intro" section
qapi/docs: adjust stub member insertion algorithm qapi/docs: remove implicit Plain section qapi/docs: add "Intro" section parsing qapi/docs: Add rendering for INTRO sections
qapi: convert intro sections for accelerator.json
qapi: convert intro sections for acpi-hest.json
qapi: convert intro sections for acpi.json
qapi: convert intro sections for audio.json

Quoted 8 lines
 docs/devel/qapi-code-gen.rst            |  17 +-
 docs/sphinx/qapidoc.py                  |  52 +++---
 qapi/accelerator.json                   |  24 +--
 qapi/acpi-hest.json                     |   5 +-
 qapi/acpi.json                          |  23 +--
 qapi/audio.json                         | 112 ++++---------
 .editorconfig                           |   1 +

Whoops, that snuck in by accident. Ignore! I needed some help doing the line wraps in qapi :)

 scripts/qapi/parser.py                  | 204 ++++++++++++++----------
Quoted 10 lines
 tests/qapi-schema/doc-good.out          |  30 ++--
 tests/qapi-schema/doc-missing-colon.err |   2 +-
 tests/qapi-schema/test-qapi.py          |  15 +-
 11 files changed, 239 insertions(+), 246 deletions(-)

--
2.53.0

#17 lore raw
Quoted 42 lines · John Snow
Quoted 31 lines · John Snow

Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.

It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.

This series demonstrates conversion of just four modules; if I'm given a thumbs up, I will convert the rest of QAPI, one module (or maintainer stanza) per patch like how I handled adding cross-references.

John Snow (12):
tests/qapi: generate output in source order qapi/docs: remove unused QAPIDoc subsection members qapi/docs: make remaining subsection members "private" qapi/docs: add "Intro" section
qapi/docs: adjust stub member insertion algorithm qapi/docs: remove implicit Plain section qapi/docs: add "Intro" section parsing qapi/docs: Add rendering for INTRO sections
qapi: convert intro sections for accelerator.json
qapi: convert intro sections for acpi-hest.json
qapi: convert intro sections for acpi.json
qapi: convert intro sections for audio.json

Quoted 8 lines
 docs/devel/qapi-code-gen.rst            |  17 +-
 docs/sphinx/qapidoc.py                  |  52 +++---
 qapi/accelerator.json                   |  24 +--
 qapi/acpi-hest.json                     |   5 +-
 qapi/acpi.json                          |  23 +--
 qapi/audio.json                         | 112 ++++---------
 .editorconfig                           |   1 +
Quoted 2 lines

Whoops, that snuck in by accident. Ignore! I needed some help doing the line wraps in qapi :)

Would it make sense as a separate patch?

Quoted 11 lines
Quoted 11 lines
 scripts/qapi/parser.py                  | 204 ++++++++++++++----------
 tests/qapi-schema/doc-good.out          |  30 ++--
 tests/qapi-schema/doc-missing-colon.err |   2 +-
 tests/qapi-schema/test-qapi.py          |  15 +-
 11 files changed, 239 insertions(+), 246 deletions(-)

--
2.53.0