qapi: add formal "intro" section
v4 cover qapi: add formal "intro" section
Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.
It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.
V4: A few minor comment touch-ups; omitted module conversions.
V3: changed intro section syntax, made various fixes and changes; I'm going to be honest Markus I did not keep a strict account this time ;_;
V2: punt intro sections to the subsequent line and adjust the parser accordingly. Left as an RFC patch that undoes some things this series does instead of properly factoring into the series.
John Snow (13):
python: temporarily restrict max mypy version tests/qapi: generate output in source order qapi/docs: remove unused QAPIDoc subsection members qapi/docs: add has_features property qapi/docs: make remaining subsection members "private" qapi/docs: fix comment phrasing
qapi/docs: add "Intro" section
qapi/parser: move _insert_near_kind() method qapi/parser: add mea culpa comment for ensure_returns qapi/docs: adjust stub member insertion algorithm qapi/docs: remove implicit Plain section qapi/docs: add rendering for INTRO sections qapi/docs: add "Intro" section parsing
docs/devel/qapi-code-gen.rst | 16 ++-- docs/sphinx/qapidoc.py | 61 +++++++----- python/setup.cfg | 3 +- scripts/qapi/parser.py | 167 +++++++++++++++++++-------------- tests/qapi-schema/doc-good.out | 86 +++++++++-------- tests/qapi-schema/test-qapi.py | 14 +-- 6 files changed, 197 insertions(+), 150 deletions(-)
Signature · 1 line
2.54.0
PATCH v4 1/13 python: temporarily restrict max mypy version
The newest versions of mypy do not support targeting Python 3.9, which we still support. I want to address that soon, but in the meantime it's nice if the tests pass.
--- python/setup.cfg | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
python/setup.cfg +2 -1
@@ -40,7 +40,8 @@ devel = flake8 >= 5.0.4 fusepy >= 2.0.4 isort >= 5.6.0 - mypy >= 1.4.0 + # Max ver is temporary until we drop 3.9 + mypy >= 1.4.0, <2.0.0 pylint >= 2.17.3 pylint != 3.2.4; python_version<"3.9" pytest >= 6.0.2
Signature · 1 line
2.54.0
PATCH v4 2/13 tests/qapi: generate output in source order
In reply to John SnowRewrite the test doc generator to produce output in source order instead of arbitrarily by section name.
This patch removes our last use of the "body" field, which has an effect on how the sections of each test documention block are printed. We now print the name of the section followed by the section text for all sections except Members and Features, which are printed as "Member=%s" or "Feature=%s" followed by the section text, respectively.
This patch is motivated by a desire to move the QAPIDoc API away from named fields for specific sections in a bid to force all users to simply iterate through all_sections in order, instead - and to remove the named subsections.
--- tests/qapi-schema/doc-good.out | 82 +++++++++++++++++----------------- tests/qapi-schema/test-qapi.py | 14 +++--- 2 files changed, 48 insertions(+), 48 deletions(-)
tests/qapi-schema/doc-good.out +41 -41
@@ -54,15 +54,15 @@ event EVT_BOXED Object boxed=True feature feat3 doc freeform - body= + Plain ******* Section ******* doc freeform - body= + Plain Just text, no heading. doc freeform - body= + Plain Subsection ==========
@@ -106,84 +106,84 @@ Examples: - *verbatim* - {braces} doc symbol=Enum - body= + Plain - arg=one + Member=one The _one_ {and only}, description on the same line - arg=two + Member=two - feature=enum-feat + Feature=enum-feat Also _one_ {and only} - feature=enum-member-feat + Feature=enum-member-feat a member feature - section=Plain + Plain @two is undocumented doc symbol=Base - body= + Plain - arg=base1 + Member=base1 description starts on a new line, minimally indented doc symbol=Variant1 - body= + Plain A paragraph Another paragraph @var1 is undocumented - arg=var1 + Member=var1 - feature=variant1-feat + Feature=variant1-feat a feature - feature=member-feat + Feature=member-feat a member feature doc symbol=Variant2 - body= + Plain doc symbol=Object - body= + Plain - feature=union-feat1 + Feature=union-feat1 a feature doc symbol=Alternate - body= + Plain - arg=i + Member=i description starts on the same line remainder indented the same @b is undocumented - arg=b + Member=b - feature=alt-feat + Feature=alt-feat a feature doc freeform - body= + Plain Another subsection ================== doc symbol=cmd - body= + Plain - arg=arg1 + Member=arg1 description starts on a new line, indented - arg=arg2 + Member=arg2 description starts on the same line remainder indented differently - arg=arg3 + Member=arg3 - feature=cmd-feat1 + Feature=cmd-feat1 a feature - feature=cmd-feat2 + Feature=cmd-feat2 another feature - section=Plain + Plain .. note:: @arg3 is undocumented - section=Returns + Returns @Object - section=Errors + Errors some - section=Todo + Todo frobnicate - section=Plain + Plain .. admonition:: Notes - Lorem ipsum dolor sit amet
@@ -207,23 +207,23 @@ Examples:: Note:: Ceci n'est pas une note - section=Since + Since 2.10 doc symbol=cmd-boxed - body= + Plain If you're bored enough to read this, go see a video of boxed cats - feature=cmd-feat1 + Feature=cmd-feat1 a feature - feature=cmd-feat2 + Feature=cmd-feat2 another feature - section=Plain + Plain .. qmp-example:: -> "this example" <- ... has no title ... doc symbol=EVT_BOXED - body= + Plain - feature=feat3 + Feature=feat3 a feature
tests/qapi-schema/test-qapi.py +7 -7
@@ -19,6 +19,7 @@ from io import StringIO from qapi.error import QAPIError +from qapi.parser import QAPIDoc from qapi.schema import QAPISchema, QAPISchemaVisitor
@@ -116,13 +117,12 @@ def test_frontend(fname): print('doc symbol=%s' % doc.symbol) else: print('doc freeform') - print(' body=\n%s' % doc.body.text) - for arg, section in doc.args.items(): - print(' arg=%s\n%s' % (arg, section.text)) - for feat, section in doc.features.items(): - print(' feature=%s\n%s' % (feat, section.text)) - for section in doc.sections: - print(' section=%s\n%s' % (section.kind, section.text)) + for section in doc.all_sections: + if isinstance(section, QAPIDoc.ArgSection): + print(' %s=%s' % (section.kind, section.name)) + else: + print(' %s' % section.kind) + print(section.text) def open_test_result(dir_name, file_name, update):
Signature · 1 line
2.54.0
PATCH v4 3/13 qapi/docs: remove unused QAPIDoc subsection members
In reply to John SnowAll users of the QAPIDoc object should be iterating over all_sections and not grabbing arbitrary subsections, if possible. Remove the 'body' and 'sections' members, as they are no longer used.
--- scripts/qapi/parser.py | 6 ------ 1 file changed, 6 deletions(-)
scripts/qapi/parser.py +0 -6
@@ -732,8 +732,6 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None): self.all_sections: List[QAPIDoc.Section] = [ QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN) ] - # the body section - self.body: Optional[QAPIDoc.Section] = self.all_sections[0] # dicts mapping parameter/feature names to their description self.args: Dict[str, QAPIDoc.ArgSection] = {} self.features: Dict[str, QAPIDoc.ArgSection] = {}
@@ -742,8 +740,6 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None): self.errors: Optional[QAPIDoc.Section] = None # "Since" section self.since: Optional[QAPIDoc.Section] = None - # sections other than .body, .args, .features - self.sections: List[QAPIDoc.Section] = [] def end(self) -> None: for section in self.all_sections:
@@ -766,7 +762,6 @@ def ensure_untagged_section(self, info: QAPISourceInfo) -> None: # start new section section = self.Section(info, kind) - self.sections.append(section) self.all_sections.append(section) def new_tagged_section(
@@ -790,7 +785,6 @@ def new_tagged_section( raise QAPISemError( info, "duplicated '%s' section" % kind) self.since = section - self.sections.append(section) self.all_sections.append(section) def _new_description(
Signature · 1 line
2.54.0
PATCH v4 4/13 qapi/docs: add has_features property
In reply to John SnowBegin preparing to remove public access to the .features member by introducing a semantic "has features" property instead.
--- scripts/qapi/parser.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
scripts/qapi/parser.py +6 -2
@@ -555,7 +555,7 @@ def get_doc(self) -> 'QAPIDoc': break # Non-blank line, first of a section if line == 'Features:': - if doc.features: + if doc.has_features: raise QAPIParseError( self, "duplicated 'Features:' line") self.accept(False)
@@ -570,7 +570,7 @@ def get_doc(self) -> 'QAPIDoc': if text: doc.append_line(text) line = self.get_doc_indented(doc) - if not doc.features: + if not doc.has_features: raise QAPIParseError( self, 'feature descriptions expected') no_more_args = True
@@ -741,6 +741,10 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None): # "Since" section self.since: Optional[QAPIDoc.Section] = None + @property + def has_features(self) -> bool: + return bool(self.features) + def end(self) -> None: for section in self.all_sections: section.text = section.text.strip('\n')
Signature · 1 line
2.54.0
PATCH v4 5/13 qapi/docs: make remaining subsection members "private"
In reply to John SnowThese fields are used to provide error checking and internal logistics and should not be used by a user of the library to directly access documentation sections, so make them private.
The "since" field alone is left public, as the qapidoc generator does use this field to pull that section out of the regular flow of the document.
--- scripts/qapi/parser.py | 48 +++++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 24 deletions(-)
scripts/qapi/parser.py +24 -24
@@ -733,17 +733,17 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None): QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN) ] # dicts mapping parameter/feature names to their description - self.args: Dict[str, QAPIDoc.ArgSection] = {} - self.features: Dict[str, QAPIDoc.ArgSection] = {} + self._args: Dict[str, QAPIDoc.ArgSection] = {} + self._features: Dict[str, QAPIDoc.ArgSection] = {} # a command's "Returns" and "Errors" section - self.returns: Optional[QAPIDoc.Section] = None - self.errors: Optional[QAPIDoc.Section] = None + self._returns: Optional[QAPIDoc.Section] = None + self._errors: Optional[QAPIDoc.Section] = None # "Since" section self.since: Optional[QAPIDoc.Section] = None @property def has_features(self) -> bool: - return bool(self.features) + return bool(self._features) def end(self) -> None: for section in self.all_sections:
@@ -775,15 +775,15 @@ def new_tagged_section( ) -> None: section = self.Section(info, kind) if kind == QAPIDoc.Kind.RETURNS: - if self.returns: + if self._returns: raise QAPISemError( info, "duplicated '%s' section" % kind) - self.returns = section + self._returns = section elif kind == QAPIDoc.Kind.ERRORS: - if self.errors: + if self._errors: raise QAPISemError( info, "duplicated '%s' section" % kind) - self.errors = section + self._errors = section elif kind == QAPIDoc.Kind.SINCE: if self.since: raise QAPISemError(
@@ -807,16 +807,16 @@ def _new_description( desc[name] = section def new_argument(self, info: QAPISourceInfo, name: str) -> None: - self._new_description(info, name, QAPIDoc.Kind.MEMBER, self.args) + self._new_description(info, name, QAPIDoc.Kind.MEMBER, self._args) def new_feature(self, info: QAPISourceInfo, name: str) -> None: - self._new_description(info, name, QAPIDoc.Kind.FEATURE, self.features) + self._new_description(info, name, QAPIDoc.Kind.FEATURE, self._features) def append_line(self, line: str) -> None: self.all_sections[-1].append_line(line) def connect_member(self, member: 'QAPISchemaMember') -> None: - if member.name not in self.args: + if member.name not in self._args: assert member.info if self.symbol not in member.info.pragma.documentation_exceptions: raise QAPISemError(member.info,
@@ -827,7 +827,7 @@ def connect_member(self, member: 'QAPISchemaMember') -> None: section = QAPIDoc.ArgSection( self.info, QAPIDoc.Kind.MEMBER, member.name) - self.args[member.name] = section + self._args[member.name] = section # Determine where to insert stub doc - it should go at the # end of the members section(s), if any. Note that index 0
@@ -839,14 +839,14 @@ def connect_member(self, member: 'QAPISchemaMember') -> None: index += 1 self.all_sections.insert(index, section) - self.args[member.name].connect(member) + self._args[member.name].connect(member) def connect_feature(self, feature: 'QAPISchemaFeature') -> None: - if feature.name not in self.features: + if feature.name not in self._features: raise QAPISemError(feature.info, "feature '%s' lacks documentation" % feature.name) - self.features[feature.name].connect(feature) + self._features[feature.name].connect(feature) def ensure_returns(self, info: QAPISourceInfo) -> None:
@@ -887,18 +887,18 @@ def _insert_near_kind( def check_expr(self, expr: QAPIExpression) -> None: if 'command' in expr: - if self.returns and 'returns' not in expr: + if self._returns and 'returns' not in expr: raise QAPISemError( - self.returns.info, + self._returns.info, "'Returns' section, but command doesn't return anything") else: - if self.returns: + if self._returns: raise QAPISemError( - self.returns.info, + self._returns.info, "'Returns' section is only valid for commands") - if self.errors: + if self._errors: raise QAPISemError( - self.errors.info, + self._errors.info, "'Errors' section is only valid for commands") def check(self) -> None:
@@ -918,5 +918,5 @@ def check_args_section( "do" if len(bogus) > 1 else "does" )) - check_args_section(self.args, 'member') - check_args_section(self.features, 'feature') + check_args_section(self._args, 'member') + check_args_section(self._features, 'feature')
Signature · 1 line
2.54.0
PATCH v4 6/13 qapi/docs: fix comment phrasing
In reply to John SnowPlaintext sections can be one or more paragraphs, the original comment was a mistake.
--- scripts/qapi/parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/qapi/parser.py +1 -1
@@ -631,7 +631,7 @@ def get_doc(self) -> 'QAPIDoc': line = self.get_doc_indented(doc) no_more_args = True else: - # plain paragraph + # plain paragraph(s) doc.ensure_untagged_section(self.info) doc.append_line(line) line = self.get_doc_paragraph(doc)
Signature · 1 line
2.54.0
PATCH v4 7/13 qapi/docs: add "Intro" section
In reply to John SnowThis patch adds an "Intro" doc section, intended to eventually replace the "Plain" doc section alongside a forthcoming "Details" section.
For now, this section is not actually instantiated or used, but subsequent patches will slowly convert the leading introductory sections of QAPIDoc documentation to use this new section.
A main motivation of this series of changes is to more explicitly delineate the "Introductory" documentation for each QAPI definition for the sake of the inliner. When inlining members, examples, and details from multiple QAPIDoc sections, we will want to omit the "Introductory" text from inlined definitions while keeping notes, caution boxes, examples, and so on. This new section facilitates that.
--- docs/sphinx/qapidoc.py | 2 +- scripts/qapi/parser.py | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-)
docs/sphinx/qapidoc.py +1 -1
@@ -368,7 +368,7 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None: for i, section in enumerate(sections): section.text = self.reformat_arobase(section.text) - if section.kind == QAPIDoc.Kind.PLAIN: + if section.kind.name in ("PLAIN", "INTRO"): self.visit_paragraph(section) elif section.kind == QAPIDoc.Kind.MEMBER: assert isinstance(section, QAPIDoc.ArgSection)
scripts/qapi/parser.py +2 -1
@@ -681,6 +681,7 @@ class Kind(enum.Enum): ERRORS = 4 SINCE = 5 TODO = 6 + INTRO = 7 @staticmethod def from_string(kind: str) -> 'QAPIDoc.Kind':
@@ -748,7 +749,7 @@ def has_features(self) -> bool: def end(self) -> None: for section in self.all_sections: section.text = section.text.strip('\n') - if section.kind != QAPIDoc.Kind.PLAIN and section.text == '': + if not (section.kind.name in ("INTRO", "PLAIN") or section.text): raise QAPISemError( section.info, "text required after '%s:'" % section.kind)
Signature · 1 line
2.54.0
PATCH v4 8/13 qapi/parser: move _insert_near_kind() method
In reply to John SnowMove this function out from underneath ensure_returns and make it available for general purpose use as an object method instead. This is purely a scope-level patch with no functional changes.
--- scripts/qapi/parser.py | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-)
scripts/qapi/parser.py +17 -16
@@ -816,6 +816,22 @@ def new_feature(self, info: QAPISourceInfo, name: str) -> None: def append_line(self, line: str) -> None: self.all_sections[-1].append_line(line) + def _insert_near_kind( + self, + kind: 'QAPIDoc.Kind', + new_sect: 'QAPIDoc.Section', + after: bool = False, + ) -> bool: + """Insert or append a new doc section at a specific point.""" + for idx, sect in enumerate(reversed(self.all_sections)): + if sect.kind == kind: + pos = len(self.all_sections) - idx - 1 + if after: + pos += 1 + self.all_sections.insert(pos, new_sect) + return True + return False + def connect_member(self, member: 'QAPISchemaMember') -> None: if member.name not in self._args: assert member.info
@@ -850,28 +866,13 @@ def connect_feature(self, feature: 'QAPISchemaFeature') -> None: self._features[feature.name].connect(feature) def ensure_returns(self, info: QAPISourceInfo) -> None: - - def _insert_near_kind( - kind: QAPIDoc.Kind, - new_sect: QAPIDoc.Section, - after: bool = False, - ) -> bool: - for idx, sect in enumerate(reversed(self.all_sections)): - if sect.kind == kind: - pos = len(self.all_sections) - idx - 1 - if after: - pos += 1 - self.all_sections.insert(pos, new_sect) - return True - return False - if any(s.kind == QAPIDoc.Kind.RETURNS for s in self.all_sections): return # Stub "Returns" section for undocumented returns value stub = QAPIDoc.Section(info, QAPIDoc.Kind.RETURNS) - if any(_insert_near_kind(kind, stub, after) for kind, after in ( + if any(self._insert_near_kind(kind, stub, after) for kind, after in ( # 1. If arguments, right after those. (QAPIDoc.Kind.MEMBER, True), # 2. Elif errors, right *before* those.
Signature · 1 line
2.54.0
PATCH v4 9/13 qapi/parser: add mea culpa comment for ensure_returns
In reply to John SnowThese algorithms are quite a mess currently, but they are temporary until we add the inliner which will address these issues more holistically. For now, add the "mea culpa".
--- scripts/qapi/parser.py | 7 +++++++ 1 file changed, 7 insertions(+)
scripts/qapi/parser.py +7 -0
@@ -866,6 +866,13 @@ def connect_feature(self, feature: 'QAPISchemaFeature') -> None: self._features[feature.name].connect(feature) def ensure_returns(self, info: QAPISourceInfo) -> None: + # This is more complicated than it ought to be. The doc + # parser should already know where a generated RETURNS section + # should go. It currently doesn't, mostly because it accepts + # tagged sections in any order. + # + # TODO: Tighten doc syntax and simplify. + if any(s.kind == QAPIDoc.Kind.RETURNS for s in self.all_sections): return
Signature · 1 line
2.54.0
PATCH v4 10/13 qapi/docs: adjust stub member insertion algorithm
In reply to John SnowA forthcoming patch removes the implicit PLAIN section that always starts a QAPIDoc section list. Further future changes begin converting "PLAIN" sections to "INTRO" sections.
This will affect the code that inserts "Not documented" descriptions for undocumented members ("stub sections") and the dummy section that marks the spot for "The members of ..." references.
Adjust the algorithm to cope with not only the finished state, but temporary intermediate states while the series is merged.
This algorithm can handle zero-or-more PLAIN *or* INTRO sections at the beginning of a QAPIDoc object, in contrast to the previous algorithm which assumed and relied upon there being always one PLAIN section at the beginning of every QAPIDoc section list.
In other words: (PLAIN | INTRO)* <EverythingElse>
This does not impact what the parser itself will actually produce. As of this patch, the parser will still always generate QAPIDoc section lists that start with precisely one PLAIN section (whether or not it is empty), followed by the remaining sections. Those remaining sections may or may not include additional PLAIN sections, but never two such sections contiguously as the parser will always treat that layout as one PLAIN section consisting of multiple paragraph(s).
In other other words: This insertion algorithm is more lenient than the parser, but this is on purpose for flexibility mid-stream as we convert QAPI to using explicit introductory sections. The allowed order of sections will eventually become strictly enforced in the parser, which will in turn allow dramatic simplifications to the insertion algorithm. This only exists as transitory code until we are able to enforce that order.
Fear not: the intermediate ReST output before and after this patch are byte identical, so failing all else, we at least know it doesn't make anything worse.
Lastly, because we have three places in the code that need to insert stub/dummy sections, we take the opportunity to consolidate this code to handle all three cases with one function. This winds up necessitating the qapidoc.py generator actually modify the section list to insert a "dummy" member that acts as a placeholder for "The members of ..." text. While it looks like a code smell to modify the caller's argument, it is ultimately safe because the QAPI Schema object is re-parsed and re-constructed in memory for each individual process that needs to operate on it. In other words, the Sphinx document generator already does have "its own copy" of the section lists, so it is "safe" to modify here without regards to other consumers of the QAPIDoc objects. It only *looks* like it smells bad. Ultimately, this code will also be removed once the inliner is merged, so it is only a temporary aesthetic issue regardless.
That's my story and I'm sticking to it.
--- docs/sphinx/qapidoc.py | 42 ++++++++++++++++-------------- scripts/qapi/parser.py | 58 +++++++++++++++++++++++++++++------------- 2 files changed, 64 insertions(+), 36 deletions(-)
docs/sphinx/qapidoc.py +23 -19
@@ -349,30 +349,38 @@ def _get_target( ) def visit_sections(self, ent: QAPISchemaDefinition) -> None: + # Generate a placeholder right after the member section(s) which + # may be used to generate documentation for "The members of..." + # pointers in the rendered document. + # + # This is a temporary hack until the inliner is merged. Note + # that although we modify the caller's section list, the + # Sphinx document generator has its own copy of the parsed + # schema in memory, so this action does not interfere with + # other users of the QAPISchema or QAPIDoc objects outside of + # the document generator. Fishy, but not harmful. + if ent.doc: + ent.doc.append_member_stub( + QAPIDoc.ArgSection( + ent.doc.info, QAPIDoc.Kind.MEMBER, "q_dummy" + ) + ) + sections = ent.doc.all_sections if ent.doc else [] - # Determine the index location at which we should generate - # documentation for "The members of ..." pointers. This should - # go at the end of the members section(s) if any. Note that - # index 0 is assumed to be a plain intro section, even if it is - # empty; and that a members section if present will always - # immediately follow the opening PLAIN section. - gen_index = 1 - if len(sections) > 1: - while sections[gen_index].kind == QAPIDoc.Kind.MEMBER: - gen_index += 1 - if gen_index >= len(sections): - break - # Add sections in source order: - for i, section in enumerate(sections): + for section in sections: section.text = self.reformat_arobase(section.text) if section.kind.name in ("PLAIN", "INTRO"): self.visit_paragraph(section) elif section.kind == QAPIDoc.Kind.MEMBER: assert isinstance(section, QAPIDoc.ArgSection) - self.visit_member(section) + if section.name == "q_dummy": + # Generate "The members of ..." entries if necessary + self._insert_member_pointer(ent) + else: + self.visit_member(section) elif section.kind == QAPIDoc.Kind.FEATURE: assert isinstance(section, QAPIDoc.ArgSection) self.visit_feature(section)
@@ -386,10 +394,6 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None: else: assert False - # Generate "The members of ..." entries if necessary: - if i == gen_index - 1: - self._insert_member_pointer(ent) - self.ensure_blank_line() # Transmogrification core methods
scripts/qapi/parser.py +41 -17
77 lines not on this page · raw message
Signature · 1 line
2.54.0
PATCH v4 11/13 qapi/docs: remove implicit Plain section
In reply to John SnowPrior to this patch, we always create an empty Plain section. Removing this allows us to gradually phase out the "Plain" section in favor of "Intro" and "Details" sections while keeping "Plain" around for the interim churn during the series - meaning that we don't actually know at __init__ time which type of section we'll have first.
--- scripts/qapi/parser.py | 4 +--- tests/qapi-schema/doc-good.out | 14 -------------- 2 files changed, 1 insertion(+), 17 deletions(-)
scripts/qapi/parser.py +1 -3
@@ -730,9 +730,7 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None): # definition doc's symbol, None for free-form doc self.symbol: Optional[str] = symbol # the sections in textual order - self.all_sections: List[QAPIDoc.Section] = [ - QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN) - ] + self.all_sections: List[QAPIDoc.Section] = [] # dicts mapping parameter/feature names to their description self._args: Dict[str, QAPIDoc.ArgSection] = {} self._features: Dict[str, QAPIDoc.ArgSection] = {}
tests/qapi-schema/doc-good.out +0 -14
@@ -106,8 +106,6 @@ Examples: - *verbatim* - {braces} doc symbol=Enum - Plain - Member=one The _one_ {and only}, description on the same line Member=two
@@ -119,8 +117,6 @@ a member feature Plain @two is undocumented doc symbol=Base - Plain - Member=base1 description starts on a new line, minimally indented
@@ -138,16 +134,10 @@ a feature Feature=member-feat a member feature doc symbol=Variant2 - Plain - doc symbol=Object - Plain - Feature=union-feat1 a feature doc symbol=Alternate - Plain - Member=i description starts on the same line remainder indented the same
@@ -161,8 +151,6 @@ doc freeform Another subsection ================== doc symbol=cmd - Plain - Member=arg1 description starts on a new line, indented
@@ -223,7 +211,5 @@ another feature <- ... has no title ... doc symbol=EVT_BOXED - Plain - Feature=feat3 a feature
Signature · 1 line
2.54.0
PATCH v4 12/13 qapi/docs: add rendering for INTRO sections
In reply to John SnowAmend the qapidoc generator to handle and render INTRO sections.
The only real difference here from other sections is that we need to dedent the text so it renders correctly. Members and Features are also indented, but do not require a dedent() because they are always used in tandem with an rST construct that forms the start of a new indented block; there is coincidental harmony.
Plaintext sections, however, do not start their own block and thus need to be dedented to prevent accidentally rendering them as a blockquote or a syntax error.
This dedent transformation on the text does not reflow the text, so source line information remains accurate, and the "blame" chain of custody for sphinx rST parsing error messages continues to be correct even through this transformation.
--- docs/sphinx/qapidoc.py | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
docs/sphinx/qapidoc.py +14 -3
@@ -35,6 +35,7 @@ from pathlib import Path import re import sys +import textwrap from typing import TYPE_CHECKING from docutils import nodes
@@ -150,8 +151,15 @@ def add_lines( self, content: str, info: QAPISourceInfo, + dedent: bool = False, ) -> None: lines = content.splitlines(True) + + if dedent: + lines = textwrap.dedent(content).splitlines(True) + else: + lines = content.splitlines(True) + for i, line in enumerate(lines): self.add_line_raw(line, info.fname, info.line + i)
@@ -223,13 +231,16 @@ def reformat_arobase(text: str) -> str: # Transmogrification helpers - def visit_paragraph(self, section: QAPIDoc.Section) -> None: + def visit_plaintext(self, section: QAPIDoc.Section) -> None: # Squelch empty paragraphs. if not section.text: return + # Intro sections, which are indented in QAPI source, need to + # be dedented to avoid accidental block quotes in ReST syntax. + dedent = bool(section.kind == QAPIDoc.Kind.INTRO) self.ensure_blank_line() - self.add_lines(section.text, section.info) + self.add_lines(section.text, section.info, dedent) self.ensure_blank_line() def visit_member(self, section: QAPIDoc.ArgSection) -> None:
@@ -373,7 +384,7 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None: section.text = self.reformat_arobase(section.text) if section.kind.name in ("PLAIN", "INTRO"): - self.visit_paragraph(section) + self.visit_plaintext(section) elif section.kind == QAPIDoc.Kind.MEMBER: assert isinstance(section, QAPIDoc.ArgSection) if section.name == "q_dummy":
Signature · 1 line
2.54.0
PATCH v4 13/13 qapi/docs: add "Intro" section parsing
In reply to John SnowAdd parsing for explicit Intro section syntax.
A side effect of this patch is that we will (currently) always create an empty Intro section, similar to how we used to have an empty Plain section. The tests are adjusted accordingly, rendered document output does not change at all.
--- docs/devel/qapi-code-gen.rst | 16 +++++++--------- scripts/qapi/parser.py | 4 ++-- tests/qapi-schema/doc-good.out | 18 ++++++++++++++++++ 3 files changed, 27 insertions(+), 11 deletions(-)
docs/devel/qapi-code-gen.rst +7 -9
@@ -984,11 +984,11 @@ definition it documents. When documentation is required (see pragma_ 'doc-required'), every definition must have documentation. -Definition documentation starts with a line naming the definition, -followed by an optional overview, a description of each argument (for -commands and events), member (for structs and unions), branch (for -alternates), or value (for enums), a description of each feature (if -any), and finally optional tagged sections. +Definition documentation starts with a description naming the +definition with an optional indented overview, a description of each +argument (for commands and events), member (for structs and unions), +branch (for alternates), or value (for enums), a description of each +feature (if any), and finally optional tagged sections. Descriptions start with '\@name:'. The description text must be indented like this::
@@ -1093,8 +1093,7 @@ Examples of complete definition documentation:: ## # @BlockStats: - # - # Statistics of a virtual block device or a block backing device. + # Statistics of a virtual block device or a block backing device. # # @device: If the stats are for a virtual block device, the name # corresponding to the virtual block device.
@@ -1111,8 +1110,7 @@ Examples of complete definition documentation:: ## # @query-blockstats: - # - # Query the @BlockStats for all virtual block devices. + # Query the @BlockStats for all virtual block devices. # # @query-nodes: If true, the command will query all the block nodes # ... explain, explain ...
scripts/qapi/parser.py +2 -2
@@ -542,8 +542,8 @@ def get_doc(self) -> 'QAPIDoc': if not symbol: raise QAPIParseError(self, "name required after '@'") doc = QAPIDoc(info, symbol) - self.accept(False) - line = self.get_doc_line() + doc.all_sections.append(QAPIDoc.Section(info, QAPIDoc.Kind.INTRO)) + line = self.get_doc_indented(doc) no_more_args = False while line is not None:
tests/qapi-schema/doc-good.out +18 -0
@@ -106,6 +106,8 @@ Examples: - *verbatim* - {braces} doc symbol=Enum + Intro + Member=one The _one_ {and only}, description on the same line Member=two
@@ -117,10 +119,14 @@ a member feature Plain @two is undocumented doc symbol=Base + Intro + Member=base1 description starts on a new line, minimally indented doc symbol=Variant1 + Intro + Plain A paragraph
@@ -134,10 +140,16 @@ a feature Feature=member-feat a member feature doc symbol=Variant2 + Intro + doc symbol=Object + Intro + Feature=union-feat1 a feature doc symbol=Alternate + Intro + Member=i description starts on the same line remainder indented the same
@@ -151,6 +163,8 @@ doc freeform Another subsection ================== doc symbol=cmd + Intro + Member=arg1 description starts on a new line, indented
@@ -198,6 +212,8 @@ Note:: Since 2.10 doc symbol=cmd-boxed + Intro + Plain If you're bored enough to read this, go see a video of boxed cats Feature=cmd-feat1
@@ -211,5 +227,7 @@ another feature <- ... has no title ... doc symbol=EVT_BOXED + Intro + Feature=feat3 a feature
Signature · 1 line
2.54.0
[Cc: trimmed]
Quoted 11 lines · John Snow
Hiya, this is a series that explores a potential syntax for a designated "Intro" section. Markus knows why I want this, but for everyone else: a designated "Introduction" section is useful for the desired "inliner" feature for the new QAPI doc system. Commits explain a bit more. This is prep work and doesn't really change anything tangibly except source code syntax for the QAPI docs.
It is designed so that this conversion can happen incrementally with no actual difference to the rendered manuals, so each QAPI module can be converted one at a time for easier review and merging in an arbitrary order.
Series
Markus Armbruster
AppliedQueued. Thanks!