Skip to contentSkip to conversation

qapi: static typing conversion, pt1

PATCH Superseded John Snow 3 participants 75 messages Sep 25, 2020 – Sep 29, 2020

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

John Snow

author
#0 lore raw

v3 cover qapi: static typing conversion, pt1

Hi, this series adds static type hints to the QAPI module. This is part one!

Part 1: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt1
Everything: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt6

- Requires Python 3.6+
- Requires mypy 0.770 or newer (for type analysis only)
- Requires pylint 2.6.0 or newer (for lint checking only)

In general, this series tackles the cleanup of one individual QAPI module at a time. Once it passes pylint or mypy checks, those checks are enabled for that file.

Type hints are added in patches that add *only* type hints and change no other behavior. Any necessary changes to behavior to accommodate typing are split out into their own tiny patches.

Notes:

- make sphinxdocs should work on every commit. It begins to include docstring content after the DO-NOT-MERGE patch 6.

- After patch 12, isort -c should pass 100% on this and every future commit.

- After patch 13, flake8 qapi/ should pass 100% on this and every future commit.

- After patch 14, pylint --rcfile=3Dqapi/pylintrc qapi/ should pass 100% on this and every future commit.

- After patch 23, mypy --config-file=3Dqapi/mypy.ini qapi/ should pass 100% on this and every future commit.

V3:
- Use isort to enforce import consistency
- Use sphinx apidoc to check docstring format
- 16: Changed docstring formatting, changed import ordering.
- 17: Changed import ordering.
- 21: Changed formatting and added some comments for Sphinx.
- 22: Changed import ordering.
- 31: Fixed, thanks Eduardo.
- 32: Shortened type annotation, thanks Cleber.
- 39: Spiritually the same, but contextually extremely different.
- 42: Very different, it now inlines the 'Extra' fields directly.
- 43: Import ordering changes.
- 45: Import ordering changes.
- 47: Import ordering changes.

Status:

(This is my stgit summary with reviewer tags visible.)

+ [01] docs-replace-single-backtick   #
+ [02] docs-repair-broken-references  #
+ [03] docs-sphinx-change-default     #
+ [04] qapi-modify-docstrings-to-be   #
+ [05] qapi-doc-py-change-code        #
+ [06] docs-enable-sphinx-autodoc-for #
+ [07] qapi-gen-separate-arg-parsing  # [SOB] JS [RB] EH
+ [08] qapi-move-generator-entrypoint # [SOB] JS [RB] CR,EH [TB] CR
+ [09] do-not-merge-more-apidoc       #
+ [10] qapi-prefer-explicit-relative  # [SOB] JS [RB] CR,EH
+ [11] qapi-remove-wildcard-includes  # [SOB] JS [RB] CR,EH
+ [12] qapi-enforce-import-order      # [SOB] JS
+ [13] qapi-delint-using-flake8       # [SOB] JS [RB] CR,EH
+ [14] qapi-add-pylintrc              # [SOB] JS [TB] CR,EH [RB] CR
+ [15] qapi-common-py-remove-python   # [SOB] JS [RB] CR,EH
+ [16] qapi-common-add-indent-manager # [SOB] JS [RB] CR,EH
+ [17] qapi-common-py-delint-with     # [SOB] JS [RB] CR,EH
+ [18] replace-c-by-char              # [SOB] JS [RB] CR,EH
+ [19] qapi-common-py-check-with      # [SOB] JS [RB] CR [TB] CR,EH
+ [20] qapi-common-py-add-notational  # [SOB] JS [RB] CR,EH
+ [21] qapi-common-move-comments-into # [SOB] JS [RB] CR,EH
+ [22] qapi-split-build_params-into   # [SOB] JS [RB] CR,EH
+ [23] qapi-establish-mypy-type       # [SOB] JS [TB] EH
+ [24] qapi-events-py-add-notational  # [SOB] JS [RB] CR,EH
+ [25] qapi-events-move-comments-into # [SOB] JS [RB] CR,EH
+ [26] qapi-commands-py-don-t-re-bind # [SOB] JS [RB] CR,EH
+ [27] qapi-commands-py-add           # [SOB] JS [RB] CR,EH
+ [28] qapi-commands-py-enable        # [SOB] JS [RB] CR,EH
+ [29] qapi-source-py-add-notational  # [SOB] JS [RB] EH
+ [30] qapi-source-py-delint-with     # [SOB] JS [RB] CR,EH [TB] CR
+ [31] qapi-gen-py-fix-edge-case-of   #
+ [32] qapi-gen-py-add-notational     # [SOB] JS [RB] CR,EH
+ [33] qapi-gen-py-enable-checking    # [SOB] JS [RB] CR,EH [TB] CR
+ [34] qapi-gen-py-remove-unused      # [SOB] JS [RB] CR,EH
+ [35] qapi-gen-py-update-write-to-be # [SOB] JS [RB] CR,EH
+ [36] qapi-gen-py-delint-with-pylint # [SOB] JS [RB] CR,EH
+ [37] qapi-introspect-py-assert-obj  #
+ [38] qapi-introspect-py-create-a    # [SOB] EH,JS
+ [39] qapi-introspect-py-add         #
+ [40] qapi-introspect-py-unify       #
+ [41] qapi-introspect-py-replace     #
+ [42] qapi-introspect-py-create-a-0  #
+ [43] qapi-types-py-add-type-hint    # [SOB] JS [RB] CR,EH
+ [44] qapi-types-py-remove-one       # [SOB] JS [RB] CR,EH
+ [45] qapi-visit-py-assert           # [SOB] JS [RB] CR,EH
+ [46] qapi-visit-py-remove-unused    # [SOB] JS [RB] CR,EH [TB] CR
> [47] qapi-visit-py-add-notational   # [SOB] JS [RB] CR,EH [TB] CR

Changelog:

001/47:[down] '[DO-NOT-MERGE] docs: replace single backtick (`) with double-b= acktick (``)'
002/47:[down] '[DO-NOT-MERGE] docs: repair broken references' 003/47:[down] '[DO-NOT-MERGE] docs/sphinx: change default role to "any"' 004/47:[down] 'qapi: modify docstrings to be sphinx-compatible' 005/47:[down] 'qapi/doc.py: Change code templates from function to string' 006/47:[down] '[DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qapi' 009/47:[down] '[DO-NOT-MERGE] docs: add scripts/qapi/main to python manual' 012/47:[down] 'qapi: enforce import order/styling with isort' 016/47:[0012] [FC] 'qapi/common.py: Add indent manager' 017/47:[0003] [FC] 'qapi/common.py: delint with pylint' 021/47:[0006] [FC] 'qapi/common.py: Convert comments into docstrings, and ela= borate'
022/47:[0020] [FC] 'qapi/common.py: move build_params into gen.py' 027/47:[down] 'qapi/commands.py: add type hint annotations' 031/47:[0002] [FC] 'qapi/gen.py: Fix edge-case of _is_user_module' 032/47:[0011] [FC] 'qapi/gen.py: add type hint annotations' 037/47:[down] 'qapi/introspect.py: assert obj is a dict when features are giv= en'
038/47:[down] 'qapi/instrospect.py: add preliminary type hint annotations' 039/47:[0013] [FC] 'qapi/introspect.py: add _gen_features helper' 040/47:[down] 'qapi/introspect.py: Unify return type of _make_tree()' 041/47:[down] 'qapi/introspect.py: replace 'extra' dict with 'comment' argume= nt'
042/47:[0062] [FC] 'qapi/introspect.py: create a typed 'Node' data structure' 043/47:[0004] [FC] 'qapi/types.py: add type hint annotations' 045/47:[0005] [FC] 'qapi/visit.py: assert tag_member contains a QAPISchemaEnu= mType'
047/47:[0005] [FC] 'qapi/visit.py: add type hint annotations'

V2:
- Removed Python3.6 patch in favor of Thomas Huth's
- Addressed (most) feedback from Markus
- Renamed type hint annotation commits

Eduardo Habkost (1):
qapi/instrospect.py: add preliminary type hint annotations

John Snow (46):
[DO-NOT-MERGE] docs: replace single backtick (`) with double-backtick

    (``)

[DO-NOT-MERGE] docs: repair broken references [DO-NOT-MERGE] docs/sphinx: change default role to "any"
qapi: modify docstrings to be sphinx-compatible qapi/doc.py: Change code templates from function to string [DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qapi
qapi-gen: Separate arg-parsing from generation
qapi: move generator entrypoint into module [DO-NOT-MERGE] docs: add scripts/qapi/main to python manual
qapi: Prefer explicit relative imports
qapi: Remove wildcard includes
qapi: enforce import order/styling with isort
qapi: delint using flake8
qapi: add pylintrc
qapi/common.py: Remove python compatibility workaround qapi/common.py: Add indent manager
qapi/common.py: delint with pylint
qapi/common.py: Replace one-letter 'c' variable qapi/common.py: check with pylint
qapi/common.py: add type hint annotations qapi/common.py: Convert comments into docstrings, and elaborate qapi/common.py: move build_params into gen.py
qapi: establish mypy type-checking baseline qapi/events.py: add type hint annotations qapi/events.py: Move comments into docstrings qapi/commands.py: Don't re-bind to variable of different type qapi/commands.py: add type hint annotations qapi/commands.py: enable checking with mypy qapi/source.py: add type hint annotations qapi/source.py: delint with pylint
qapi/gen.py: Fix edge-case of _is_user_module qapi/gen.py: add type hint annotations
qapi/gen.py: Enable checking with mypy
qapi/gen.py: Remove unused parameter
qapi/gen.py: update write() to be more idiomatic qapi/gen.py: delint with pylint
qapi/introspect.py: assert obj is a dict when features are given qapi/introspect.py: add _gen_features helper qapi/introspect.py: Unify return type of _make_tree() qapi/introspect.py: replace 'extra' dict with 'comment' argument qapi/introspect.py: create a typed 'Node' data structure qapi/types.py: add type hint annotations qapi/types.py: remove one-letter variables qapi/visit.py: assert tag_member contains a QAPISchemaEnumType qapi/visit.py: remove unused parameters from gen_visit_object qapi/visit.py: add type hint annotations

 docs/conf.py                           |   9 +-
 docs/devel/build-system.rst            | 118 +++++++-------
 docs/devel/index.rst                   |   1 +
 docs/devel/migration.rst               |  59 +++----
 docs/devel/multi-thread-tcg.rst        |   2 +-
 docs/devel/python/index.rst            |   7 +
 docs/devel/python/qapi.commands.rst    |   7 +
 docs/devel/python/qapi.common.rst      |   7 +
 docs/devel/python/qapi.doc.rst         |   7 +
 docs/devel/python/qapi.error.rst       |   7 +
 docs/devel/python/qapi.events.rst      |   7 +
 docs/devel/python/qapi.expr.rst        |   7 +
 docs/devel/python/qapi.gen.rst         |   7 +
 docs/devel/python/qapi.introspect.rst  |   7 +
 docs/devel/python/qapi.main.rst        |   7 +
 docs/devel/python/qapi.parser.rst      |   8 +
 docs/devel/python/qapi.rst             |  27 ++++
 docs/devel/python/qapi.schema.rst      |   7 +
 docs/devel/python/qapi.source.rst      |   7 +
 docs/devel/python/qapi.types.rst       |   7 +
 docs/devel/python/qapi.visit.rst       |   7 +
 docs/devel/tcg-plugins.rst             |  14 +-
 docs/devel/testing.rst                 |   4 +-
 docs/interop/live-block-operations.rst |   4 +-
 docs/system/arm/cpu-features.rst       | 110 ++++++-------
 docs/system/arm/nuvoton.rst            |   2 +-
 docs/system/s390x/protvirt.rst         |  10 +-
 scripts/qapi-gen.py                    |  59 ++-----
 scripts/qapi/.flake8                   |   2 +
 scripts/qapi/.isort.cfg                |   5 +
 scripts/qapi/commands.py               |  87 +++++++---
 scripts/qapi/common.py                 | 162 ++++++++++---------
 scripts/qapi/doc.py                    |  53 ++++---
 scripts/qapi/events.py                 |  56 +++++--
 scripts/qapi/expr.py                   |   6 +-
 scripts/qapi/gen.py                    | 184 +++++++++++++---------
 scripts/qapi/introspect.py             | 209 +++++++++++++++++--------
 scripts/qapi/main.py                   |  90 +++++++++++
 scripts/qapi/mypy.ini                  |  30 ++++
 scripts/qapi/parser.py                 |  15 +-
 scripts/qapi/pylintrc                  |  71 +++++++++
 scripts/qapi/schema.py                 |  35 +++--
 scripts/qapi/source.py                 |  34 ++--
 scripts/qapi/types.py                  | 124 ++++++++++-----
 scripts/qapi/visit.py                  | 116 ++++++++++----
 45 files changed, 1210 insertions(+), 594 deletions(-)
 create mode 100644 docs/devel/python/index.rst
 create mode 100644 docs/devel/python/qapi.commands.rst
 create mode 100644 docs/devel/python/qapi.common.rst
 create mode 100644 docs/devel/python/qapi.doc.rst
 create mode 100644 docs/devel/python/qapi.error.rst
 create mode 100644 docs/devel/python/qapi.events.rst
 create mode 100644 docs/devel/python/qapi.expr.rst
 create mode 100644 docs/devel/python/qapi.gen.rst
 create mode 100644 docs/devel/python/qapi.introspect.rst
 create mode 100644 docs/devel/python/qapi.main.rst
 create mode 100644 docs/devel/python/qapi.parser.rst
 create mode 100644 docs/devel/python/qapi.rst
 create mode 100644 docs/devel/python/qapi.schema.rst
 create mode 100644 docs/devel/python/qapi.source.rst
 create mode 100644 docs/devel/python/qapi.types.rst
 create mode 100644 docs/devel/python/qapi.visit.rst
 create mode 100644 scripts/qapi/.flake8
 create mode 100644 scripts/qapi/.isort.cfg
 create mode 100644 scripts/qapi/main.py
 create mode 100644 scripts/qapi/mypy.ini
 create mode 100644 scripts/qapi/pylintrc

--=20
2.26.2

author
#1 lore raw

PATCH v3 1/47 docs: replace single backtick (`) with double-backtick (``)

The single backtick in ReST is the "default role". Currently, Sphinx's default role is called "content". Sphinx suggests you can use the "Any" role instead to turn any single-backtick enclosed item into a cross-reference.

Before we do that, though, we'll need to turn all existing usages of the "content" role to inline verbatim markup by using double backticks instead.

---
 docs/devel/build-system.rst            | 118 ++++++++++++-------------
 docs/devel/migration.rst               |  59 +++++++------
 docs/devel/tcg-plugins.rst             |  14 +--
 docs/devel/testing.rst                 |   2 +-
 docs/interop/live-block-operations.rst |   4 +-
 docs/system/arm/cpu-features.rst       | 110 +++++++++++------------
 docs/system/arm/nuvoton.rst            |   2 +-
 docs/system/s390x/protvirt.rst         |  10 +--
 8 files changed, 161 insertions(+), 158 deletions(-)
docs/devel/build-system.rst +59 -59
index 08e85c69e1..a4711cd4e1 100644
@@ -53,14 +53,14 @@ following tasks:
  - Add a Meson build option to meson_options.txt.
 
  - Add support to the command line arg parser to handle any new
-   `--enable-XXX`/`--disable-XXX` flags required by the feature.
+   ``--enable-XXX``/``--disable-XXX`` flags required by the feature.
 
  - Add information to the help output message to report on the new
    feature flag.
 
  - Add code to perform the actual feature check.
 
- - Add code to include the feature status in `config-host.h`
+ - Add code to include the feature status in ``config-host.h``
 
  - Add code to print out the feature status in the configure summary
    upon completion.
@@ -116,51 +116,51 @@ Helper functions
 The configure script provides a variety of helper functions to assist
 developers in checking for system features:
 
-`do_cc $ARGS...`
+``do_cc $ARGS...``
    Attempt to run the system C compiler passing it $ARGS...
 
-`do_cxx $ARGS...`
+``do_cxx $ARGS...``
    Attempt to run the system C++ compiler passing it $ARGS...
 
-`compile_object $CFLAGS`
+``compile_object $CFLAGS``
    Attempt to compile a test program with the system C compiler using
    $CFLAGS. The test program must have been previously written to a file
-   called $TMPC.  The replacement in Meson is the compiler object `cc`,
-   which has methods such as `cc.compiles()`,
-   `cc.check_header()`, `cc.has_function()`.
+   called $TMPC.  The replacement in Meson is the compiler object ``cc``,
+   which has methods such as ``cc.compiles()``,
+   ``cc.check_header()``, ``cc.has_function()``.
 
-`compile_prog $CFLAGS $LDFLAGS`
+``compile_prog $CFLAGS $LDFLAGS``
    Attempt to compile a test program with the system C compiler using
    $CFLAGS and link it with the system linker using $LDFLAGS. The test
    program must have been previously written to a file called $TMPC.
-   The replacement in Meson is `cc.find_library()` and `cc.links()`.
+   The replacement in Meson is ``cc.find_library()`` and ``cc.links()``.
 
-`has $COMMAND`
+``has $COMMAND``
    Determine if $COMMAND exists in the current environment, either as a
    shell builtin, or executable binary, returning 0 on success.  The
-   replacement in Meson is `find_program()`.
+   replacement in Meson is ``find_program()``.
 
-`check_define $NAME`
+``check_define $NAME``
    Determine if the macro $NAME is defined by the system C compiler
 
-`check_include $NAME`
+``check_include $NAME``
    Determine if the include $NAME file is available to the system C
-   compiler.  The replacement in Meson is `cc.has_header()`.
+   compiler.  The replacement in Meson is ``cc.has_header()``.
 
-`write_c_skeleton`
+``write_c_skeleton``
    Write a minimal C program main() function to the temporary file
    indicated by $TMPC
 
-`feature_not_found $NAME $REMEDY`
+``feature_not_found $NAME $REMEDY``
    Print a message to stderr that the feature $NAME was not available
    on the system, suggesting the user try $REMEDY to address the
    problem.
 
-`error_exit $MESSAGE $MORE...`
+``error_exit $MESSAGE $MORE...``
    Print $MESSAGE to stderr, followed by $MORE... and then exit from the
    configure script with non-zero status
 
-`query_pkg_config $ARGS...`
+``query_pkg_config $ARGS...``
    Run pkg-config passing it $ARGS. If QEMU is doing a static build,
    then --static will be automatically added to $ARGS
 
@@ -194,8 +194,8 @@ to list the files and their dependency on various configuration
 symbols.
 
 Various subsystems that are common to both tools and emulators have
-their own sourceset, for example `block_ss` for the block device subsystem,
-`chardev_ss` for the character device subsystem, etc.  These sourcesets
+their own sourceset, for example ``block_ss`` for the block device subsystem,
+``chardev_ss`` for the character device subsystem, etc.  These sourcesets
 are then turned into static libraries as follows::
 
     libchardev = static_library('chardev', chardev_ss.sources(),
@@ -204,8 +204,8 @@ are then turned into static libraries as follows::
 
     chardev = declare_dependency(link_whole: libchardev)
 
-As of Meson 0.55.1, the special `.fa` suffix should be used for everything
-that is used with `link_whole`, to ensure that the link flags are placed
+As of Meson 0.55.1, the special ``.fa`` suffix should be used for everything
+that is used with ``link_whole``, to ensure that the link flags are placed
 correctly in the command line.
 
 Files linked into emulator targets there can be split into two distinct groups
@@ -216,25 +216,25 @@ In the target-independent set lives various general purpose helper code,
 such as error handling infrastructure, standard data structures,
 platform portability wrapper functions, etc. This code can be compiled
 once only and the .o files linked into all output binaries.
-Target-independent code lives in the `common_ss`, `softmmu_ss` and
-`user_ss` sourcesets.  `common_ss` is linked into all emulators, `softmmu_ss`
-only in system emulators, `user_ss` only in user-mode emulators.
+Target-independent code lives in the ``common_ss``, ``softmmu_ss`` and
+``user_ss`` sourcesets.  ``common_ss`` is linked into all emulators, ``softmmu_ss``
+only in system emulators, ``user_ss`` only in user-mode emulators.
 
 In the target-dependent set lives CPU emulation, device emulation and
 much glue code. This sometimes also has to be compiled multiple times,
 once for each target being built.  Target-dependent files are included
-in the `specific_ss` sourceset.
+in the ``specific_ss`` sourceset.
 
-All binaries link with a static library `libqemuutil.a`, which is then
-linked to all the binaries.  `libqemuutil.a` is built from several
+All binaries link with a static library ``libqemuutil.a``, which is then
+linked to all the binaries.  ``libqemuutil.a`` is built from several
 sourcesets; most of them however host generated code, and the only two
-of general interest are `util_ss` and `stub_ss`.
+of general interest are ``util_ss`` and ``stub_ss``.
 
 The separation between these two is purely for documentation purposes.
-`util_ss` contains generic utility files.  Even though this code is only
+``util_ss`` contains generic utility files.  Even though this code is only
 linked in some binaries, sometimes it requires hooks only in some of
 these and depend on other functions that are not fully implemented by
-all QEMU binaries.  `stub_ss` links dummy stubs that will only be linked
+all QEMU binaries.  ``stub_ss`` links dummy stubs that will only be linked
 into the binary if the real implementation is not present.  In a way,
 the stubs can be thought of as a portable implementation of the weak
 symbols concept.
@@ -242,7 +242,7 @@ symbols concept.
 The following files concur in the definition of which files are linked
 into each emulator:
 
-`default-configs/*.mak`
+``default-configs/*.mak``
   The files under default-configs/ control what emulated hardware is built
   into each QEMU system and userspace emulator targets. They merely contain
   a list of config variable definitions like the machines that should be
@@ -252,8 +252,8 @@ into each emulator:
     CONFIG_XLNX_ZYNQMP_ARM=y
     CONFIG_XLNX_VERSAL=y
 
-`*/Kconfig`
-  These files are processed together with `default-configs/*.mak` and
+``*/Kconfig``
+  These files are processed together with ``default-configs/*.mak`` and
   describe the dependencies between various features, subsystems and
   device models.  They are described in kconfig.rst.
 
@@ -265,19 +265,19 @@ Support scripts
 ---------------
 
 Meson has a special convention for invoking Python scripts: if their
-first line is `#! /usr/bin/env python3` and the file is *not* executable,
+first line is ``#! /usr/bin/env python3`` and the file is *not* executable,
 find_program() arranges to invoke the script under the same Python
 interpreter that was used to invoke Meson.  This is the most common
 and preferred way to invoke support scripts from Meson build files,
 because it automatically uses the value of configure's --python= option.
 
-In case the script is not written in Python, use a `#! /usr/bin/env ...`
+In case the script is not written in Python, use a ``#! /usr/bin/env ...``
 line and make the script executable.
 
 Scripts written in Python, where it is desirable to make the script
 executable (for example for test scripts that developers may want to
 invoke from the command line, such as tests/qapi-schema/test-qapi.py),
-should be invoked through the `python` variable in meson.build. For
+should be invoked through the ``python`` variable in meson.build. For
 example::
 
   test('QAPI schema regression tests', python,
@@ -301,10 +301,10 @@ rules and wraps them so that e.g. submodules are built before QEMU.
 The resulting build system is largely non-recursive in nature, in
 contrast to common practices seen with automake.
 
-Tests are also ran by the Makefile with the traditional `make check`
-phony target, while benchmarks are run with `make bench`.  Meson test
-suites such as `unit` can be ran with `make check-unit` too.  It is also
-possible to run tests defined in meson.build with `meson test`.
+Tests are also ran by the Makefile with the traditional ``make check``
+phony target, while benchmarks are run with ``make bench``.  Meson test
+suites such as ``unit`` can be ran with ``make check-unit`` too.  It is also
+possible to run tests defined in meson.build with ``meson test``.
 
 Important files for the build system
 ====================================
@@ -316,28 +316,28 @@ The following key files are statically defined in the source tree, with
 the rules needed to build QEMU. Their behaviour is influenced by a
 number of dynamically created files listed later.
 
-`Makefile`
+``Makefile``
   The main entry point used when invoking make to build all the components
   of QEMU. The default 'all' target will naturally result in the build of
   every component. Makefile takes care of recursively building submodules
   directly via a non-recursive set of rules.
 
-`*/meson.build`
+``*/meson.build``
   The meson.build file in the root directory is the main entry point for the
   Meson build system, and it coordinates the configuration and build of all
   executables.  Build rules for various subdirectories are included in
   other meson.build files spread throughout the QEMU source tree.
 
-`tests/Makefile.include`
+``tests/Makefile.include``
   Rules for external test harnesses. These include the TCG tests,
-  `qemu-iotests` and the Avocado-based acceptance tests.
+  ``qemu-iotests`` and the Avocado-based acceptance tests.
 
-`tests/docker/Makefile.include`
+``tests/docker/Makefile.include``
   Rules for Docker tests. Like tests/Makefile, this file is included
   directly by the top level Makefile, anything defined in this file will
   influence the entire build system.
 
-`tests/vm/Makefile.include`
+``tests/vm/Makefile.include``
   Rules for VM-based tests. Like tests/Makefile, this file is included
   directly by the top level Makefile, anything defined in this file will
   influence the entire build system.
@@ -353,11 +353,11 @@ Makefile.
 
 Built by configure:
 
-`config-host.mak`
+``config-host.mak``
   When configure has determined the characteristics of the build host it
   will write a long list of variables to config-host.mak file. This
   provides the various install directories, compiler / linker flags and a
-  variety of `CONFIG_*` variables related to optionally enabled features.
+  variety of ``CONFIG_*`` variables related to optionally enabled features.
   This is imported by the top level Makefile and meson.build in order to
   tailor the build output.
 
@@ -369,7 +369,7 @@ Built by configure:
   build outputs. Variables which are potentially different for each
   emulator target are defined by the next file...
 
-`$TARGET-NAME/config-target.mak`
+``$TARGET-NAME/config-target.mak``
   TARGET-NAME is the name of a system or userspace emulator, for example,
   x86_64-softmmu denotes the system emulator for the x86_64 architecture.
   This file contains the variables which need to vary on a per-target
@@ -380,31 +380,31 @@ Built by configure:
 
 Built by Meson:
 
-`${TARGET-NAME}-config-devices.mak`
+``${TARGET-NAME}-config-devices.mak``
   TARGET-NAME is again the name of a system or userspace emulator. The
   config-devices.mak file is automatically generated by make using the
   scripts/make_device_config.sh program, feeding it the
   default-configs/$TARGET-NAME file as input.
 
-`config-host.h`, `$TARGET-NAME/config-target.h`, `$TARGET-NAME/config-devices.h`
+``config-host.h``, ``$TARGET-NAME/config-target.h``, ``$TARGET-NAME/config-devices.h``
   These files are used by source code to determine what features
   are enabled.  They are generated from the contents of the corresponding
-  `*.h` files using the scripts/create_config program. This extracts
+  ``*.h`` files using the scripts/create_config program. This extracts
   relevant variables and formats them as C preprocessor macros.
 
-`build.ninja`
+``build.ninja``
   The build rules.
 
 
 Built by Makefile:
 
-`Makefile.ninja`
+``Makefile.ninja``
   A Makefile conversion of the build rules in build.ninja.  The conversion
   is straightforward and, were it necessary to debug the rules produced
   by Meson, it should be enough to look at build.ninja.  The conversion
   is performed by scripts/ninjatool.py.
 
-`Makefile.mtest`
+``Makefile.mtest``
   The Makefile definitions that let "make check" run tests defined in
   meson.build.  The rules are produced from Meson's JSON description of
   tests (obtained with "meson introspect --tests") through the script
@@ -414,9 +414,9 @@ Built by Makefile:
 Useful make targets
 -------------------
 
-`help`
+``help``
   Print a help message for the most common build targets.
 
-`print-VAR`
+``print-VAR``
   Print the value of the variable VAR. Useful for debugging the build
   system.
docs/devel/migration.rst +31 -28

119 lines not on this page · raw message

docs/devel/tcg-plugins.rst +7 -7
index 0568dfa6a4..5057b6e1b2 100644
@@ -34,11 +34,11 @@ version they were built against. This can be done simply by::
   QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
 
 The core code will refuse to load a plugin that doesn't export a
-`qemu_plugin_version` symbol or if plugin version is outside of QEMU's
+``qemu_plugin_version`` symbol or if plugin version is outside of QEMU's
 supported range of API versions.
 
-Additionally the `qemu_info_t` structure which is passed to the
-`qemu_plugin_install` method of a plugin will detail the minimum and
+Additionally the ``qemu_info_t`` structure which is passed to the
+``qemu_plugin_install`` method of a plugin will detail the minimum and
 current API versions supported by QEMU. The API version will be
 incremented if new APIs are added. The minimum API version will be
 incremented if existing APIs are changed or removed.
@@ -140,12 +140,12 @@ Example Plugins
 
 There are a number of plugins included with QEMU and you are
 encouraged to contribute your own plugins plugins upstream. There is a
-`contrib/plugins` directory where they can go.
+``contrib/plugins`` directory where they can go.
 
 - tests/plugins
 
 These are some basic plugins that are used to test and exercise the
-API during the `make check-tcg` target.
+API during the ``make check-tcg`` target.
 
 - contrib/plugins/hotblocks.c
 
@@ -157,7 +157,7 @@ with linux-user execution as system emulation tends to generate
 re-translations as blocks from different programs get swapped in and
 out of system memory.
 
-If your program is single-threaded you can use the `inline` option for
+If your program is single-threaded you can use the ``inline`` option for
 slightly faster (but not thread safe) counters.
 
 Example::
@@ -245,7 +245,7 @@ which will lead to a sorted list after the class breakdown::
   ...
 
 To find the argument shorthand for the class you need to examine the
-source code of the plugin at the moment, specifically the `*opt`
+source code of the plugin at the moment, specifically the ``*opt``
 argument in the InsnClassExecCount tables.
 
 - contrib/plugins/lockstep.c
docs/devel/testing.rst +1 -1
index bd64c1bdcd..666c4d7240 100644
@@ -749,7 +749,7 @@ The base test class has also support for tests with more than one
 QEMUMachine. The way to get machines is through the ``self.get_vm()``
 method which will return a QEMUMachine instance. The ``self.get_vm()``
 method accepts arguments that will be passed to the QEMUMachine creation
-and also an optional `name` attribute so you can identify a specific
+and also an optional ``name`` attribute so you can identify a specific
 machine and get it more than once through the tests methods. A simple
 and hypothetical example follows:
 
docs/interop/live-block-operations.rst +2 -2
index e13f5a21f8..5a3f045827 100644
@@ -638,7 +638,7 @@ at this point:
         (QEMU) block-job-complete device=job0
 
 In either of the above cases, if you once again run the
-`query-block-jobs` command, there should not be any active block
+``query-block-jobs`` command, there should not be any active block
 operation.
 
 Comparing 'commit' and 'mirror': In both then cases, the overlay images
@@ -777,7 +777,7 @@ the content of image [D].
         }
 
 (6) [On *destination* QEMU] Finally, resume the guest vCPUs by issuing the
-    QMP command `cont`::
+    QMP command ``cont``::
 
         (QEMU) cont
         {
docs/system/arm/cpu-features.rst +55 -55

260 lines not on this page · raw message

docs/system/arm/nuvoton.rst +1 -1
index e3e1a3a3a7..b0a35b54bd 100644
@@ -77,7 +77,7 @@ Boot options
 ------------
 
 The Nuvoton machines can boot from an OpenBMC firmware image, or directly into
-a kernel using the ``-kernel`` option. OpenBMC images for `quanta-gsj` and
+a kernel using the ``-kernel`` option. OpenBMC images for ``quanta-gsj`` and
 possibly others can be downloaded from the OpenPOWER jenkins :
 
    https://openpower.xyz/
docs/system/s390x/protvirt.rst +5 -5
index 712974ad87..d208c12a96 100644
@@ -14,11 +14,11 @@ Prerequisites
 To run PVMs, a machine with the Protected Virtualization feature, as
 indicated by the Ultravisor Call facility (stfle bit 158), is
 required. The Ultravisor needs to be initialized at boot by setting
-`prot_virt=1` on the host's kernel command line.
+``prot_virt=1`` on the host's kernel command line.
 
 Running PVMs requires using the KVM hypervisor.
 
-If those requirements are met, the capability `KVM_CAP_S390_PROTECTED`
+If those requirements are met, the capability ``KVM_CAP_S390_PROTECTED``
 will indicate that KVM can support PVMs on that LPAR.
 
 
@@ -26,8 +26,8 @@ QEMU Settings
 -------------
 
 To indicate to the VM that it can transition into protected mode, the
-`Unpack facility` (stfle bit 161 represented by the feature
-`unpack`/`S390_FEAT_UNPACK`) needs to be part of the cpu model of
+``Unpack facility`` (stfle bit 161 represented by the feature
+``unpack``/``S390_FEAT_UNPACK``) needs to be part of the cpu model of
 the VM.
 
 All I/O devices need to use the IOMMU.
@@ -56,5 +56,5 @@ from the disk boot. This memory layout includes the encrypted
 components (kernel, initrd, cmdline), the stage3a loader and
 metadata. In case this boot method is used, the command line
 options -initrd and -cmdline are ineffective. The preparation of a PVM
-image is done via the `genprotimg` tool from the s390-tools
+image is done via the ``genprotimg`` tool from the s390-tools
 collection.
Signature · 1 line
2.26.2
Reviewed-by on v3 1/47
#2 lore raw

Re: [PATCH v3 01/47] [DO-NOT-MERGE] docs: replace single backtick (`) with double-backtick (``)

Quoted 10 lines · John Snow

The single backtick in ReST is the "default role". Currently, Sphinx's default role is called "content". Sphinx suggests you can use the "Any" role instead to turn any single-backtick enclosed item into a cross-reference.

Before we do that, though, we'll need to turn all existing usages of the "content" role to inline verbatim markup by using double backticks instead.

While reviewing this, looking for broken usage of single backticks, I also found what you found on the next patch, and nothing else.

author
#3 lore raw

PATCH v3 2/47 docs: repair broken references

In reply to John Snow
---
 docs/devel/multi-thread-tcg.rst | 2 +-
 docs/devel/testing.rst          | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
docs/devel/multi-thread-tcg.rst +1 -1
index 21483870db..92a9eba13c 100644
@@ -267,7 +267,7 @@ of view of external observers (e.g. another processor core). They can
 apply to any memory operations as well as just loads or stores.
 
 The Linux kernel has an excellent `write-up
-<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`
+<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`_
 on the various forms of memory barrier and the guarantees they can
 provide.
 
docs/devel/testing.rst +1 -1
index 666c4d7240..f21f3f58eb 100644
@@ -953,7 +953,7 @@ compiler flags are needed to build for a given target.
 If you have the ability to run containers as the user you can also
 take advantage of the build systems "Docker" support. It will then use
 containers to build any test case for an enabled guest where there is
-no system compiler available. See :ref: `_docker-ref` for details.
+no system compiler available. See `docker-ref` for details.
 
 Running subset of tests
 -----------------------
Signature · 1 line
2.26.2
#5 lore raw

Re: [PATCH v3 02/47] [DO-NOT-MERGE] docs: repair broken references

In reply to John Snow
Quoted 24 lines · John Snow
---
 docs/devel/multi-thread-tcg.rst | 2 +-
 docs/devel/testing.rst          | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
docs/devel/multi-thread-tcg.rst +1 -1
index 21483870db..92a9eba13c 100644
@@ -267,7 +267,7 @@ of view of external observers (e.g. another processor core). They can
 apply to any memory operations as well as just loads or stores.
 
 The Linux kernel has an excellent `write-up
-<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`
+<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`_
 on the various forms of memory barrier and the guarantees they can
 provide.
 
docs/devel/testing.rst +0 -0
index 666c4d7240..f21f3f58eb 100644

docs/devel/testing.rst compiler flags are needed to build for a given target. · line 957

Quoted 6 lines
docs/devel/testing.rst +1 -1
@@ -953,7 +953,7 @@ compiler flags are needed to build for a given target.
 If you have the ability to run containers as the user you can also
 take advantage of the build systems "Docker" support. It will then use
 containers to build any test case for an enabled guest where there is
-no system compiler available. See :ref: `_docker-ref` for details.
+no system compiler available. See `docker-ref` for details.
 

Actually, I take the "r-b" back because I missed this line... it should be:

no system compiler available. See docker-ref_ for details.

- Cleber.

author
#6 lore raw

Re: [PATCH v3 02/47] [DO-NOT-MERGE] docs: repair broken references

Quoted 35 lines · Cleber Rosa
Quoted 24 lines · John Snow
---
  docs/devel/multi-thread-tcg.rst | 2 +-
  docs/devel/testing.rst          | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)
docs/devel/multi-thread-tcg.rst +1 -1
index 21483870db..92a9eba13c 100644
@@ -267,7 +267,7 @@ of view of external observers (e.g. another processor core). They can
  apply to any memory operations as well as just loads or stores.
 
  The Linux kernel has an excellent `write-up
-<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`
+<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`_
  on the various forms of memory barrier and the guarantees they can
  provide.
 
docs/devel/testing.rst +0 -0
index 666c4d7240..f21f3f58eb 100644

docs/devel/testing.rst compiler flags are needed to build for a given target. · line 957

Quoted 6 lines
docs/devel/testing.rst +1 -1
@@ -953,7 +953,7 @@ compiler flags are needed to build for a given target.
  If you have the ability to run containers as the user you can also
  take advantage of the build systems "Docker" support. It will then use
  containers to build any test case for an enabled guest where there is
-no system compiler available. See :ref: `_docker-ref` for details.
+no system compiler available. See `docker-ref` for details.
 

Actually, I take the "r-b" back because I missed this line... it should be:

Quoted 4 lines

no system compiler available. See docker-ref_ for details.

- Cleber.

I can send that as a *real* patch to fix it *right now*, but it actually works just fine using the "any" role.

(Or, it did for me.)

--js

author
#7 lore raw

Re: [PATCH v3 02/47] [DO-NOT-MERGE] docs: repair broken references

In reply to Cleber Rosa
Quoted 35 lines · Cleber Rosa
Quoted 24 lines · John Snow
---
  docs/devel/multi-thread-tcg.rst | 2 +-
  docs/devel/testing.rst          | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)
docs/devel/multi-thread-tcg.rst +1 -1
index 21483870db..92a9eba13c 100644
@@ -267,7 +267,7 @@ of view of external observers (e.g. another processor core). They can
  apply to any memory operations as well as just loads or stores.
 
  The Linux kernel has an excellent `write-up
-<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`
+<https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt>`_
  on the various forms of memory barrier and the guarantees they can
  provide.
 
docs/devel/testing.rst +0 -0
index 666c4d7240..f21f3f58eb 100644

docs/devel/testing.rst compiler flags are needed to build for a given target. · line 957

Quoted 6 lines
docs/devel/testing.rst +1 -1
@@ -953,7 +953,7 @@ compiler flags are needed to build for a given target.
  If you have the ability to run containers as the user you can also
  take advantage of the build systems "Docker" support. It will then use
  containers to build any test case for an enabled guest where there is
-no system compiler available. See :ref: `_docker-ref` for details.
+no system compiler available. See `docker-ref` for details.
 

Actually, I take the "r-b" back because I missed this line... it should be:

Quoted 4 lines

no system compiler available. See docker-ref_ for details.

- Cleber.

I'm reworking this patch and dropping the DO-NOT-MERGE and fixing this reference to read:

:ref:docker-ref

This works *without* my other sphinx patches, so it should now stand alone.

I've sent this as a standalone patch to qemu-trivial.

(I'll fix it in this branch too, but it's DO-NOT-MERGE here, so ... whatever!)

--js

author
#8 lore raw

PATCH v3 3/47 docs/sphinx: change default role to "any"

In reply to John Snow
---
 docs/conf.py | 3 +++
 1 file changed, 3 insertions(+)
docs/conf.py +3 -0
index 0dbd90dc11..a68f616d5a 100644
@@ -81,6 +81,9 @@
 # The master toctree document.
 master_doc = 'index'
 
+# Interpret `this` to be a cross-reference to "anything".
+default_role = 'any'
+
 # General information about the project.
 project = u'QEMU'
 copyright = u'2020, The QEMU Project Developers'
Signature · 1 line
2.26.2
#9 lore raw

Re: [PATCH v3 03/47] [DO-NOT-MERGE] docs/sphinx: change default role to "any"

Quoted 22 lines · John Snow
---
 docs/conf.py | 3 +++
 1 file changed, 3 insertions(+)
docs/conf.py +3 -0
index 0dbd90dc11..a68f616d5a 100644
@@ -81,6 +81,9 @@
 # The master toctree document.
 master_doc = 'index'
 
+# Interpret `this` to be a cross-reference to "anything".
+default_role = 'any'
+
 # General information about the project.
 project = u'QEMU'
 copyright = u'2020, The QEMU Project Developers'
Signature · 1 line
2.26.2

After this I get:

qemu/docs/cpu-hotplug.rst:81: WARNING: 'any' reference target not found: query-cpus-fast

So I missed it during the review of the first patch ("docs: replace single backtick (`) with double-backtick (``)").

- Cleber.

author
#10 lore raw

Re: [PATCH v3 03/47] [DO-NOT-MERGE] docs/sphinx: change default role to "any"

Quoted 26 lines · Cleber Rosa
Quoted 22 lines · John Snow
---
  docs/conf.py | 3 +++
  1 file changed, 3 insertions(+)
docs/conf.py +3 -0
index 0dbd90dc11..a68f616d5a 100644
@@ -81,6 +81,9 @@
  # The master toctree document.
  master_doc = 'index'
 
+# Interpret `this` to be a cross-reference to "anything".
+default_role = 'any'
+
  # General information about the project.
  project = u'QEMU'
  copyright = u'2020, The QEMU Project Developers'
Signature · 1 line
2.26.2

After this I get:

Quoted 7 lines

qemu/docs/cpu-hotplug.rst:81: WARNING: 'any' reference target not found: query-cpus-fast

So I missed it during the review of the first patch ("docs: replace single backtick (`) with double-backtick (``)").

- Cleber.

Not sure why I didn't encounter that. Maybe version troubles again? Configuration differences?

--js

author
#11 lore raw

Re: [PATCH v3 03/47] [DO-NOT-MERGE] docs/sphinx: change default role to "any"

In reply to Cleber Rosa
Quoted 26 lines · Cleber Rosa
Quoted 22 lines · John Snow
---
  docs/conf.py | 3 +++
  1 file changed, 3 insertions(+)
docs/conf.py +3 -0
index 0dbd90dc11..a68f616d5a 100644
@@ -81,6 +81,9 @@
  # The master toctree document.
  master_doc = 'index'
 
+# Interpret `this` to be a cross-reference to "anything".
+default_role = 'any'
+
  # General information about the project.
  project = u'QEMU'
  copyright = u'2020, The QEMU Project Developers'
Signature · 1 line
2.26.2

After this I get:

Quoted 7 lines

qemu/docs/cpu-hotplug.rst:81: WARNING: 'any' reference target not found: query-cpus-fast

So I missed it during the review of the first patch ("docs: replace single backtick (`) with double-backtick (``)").

- Cleber.

What are you invoking to get this to fail?

I am typing "make sphinxdocs".

It doesn't look like this document is actually built as part of the manual for me.

author
#12 lore raw

PATCH v3 4/47 qapi: modify docstrings to be sphinx-compatible

In reply to John Snow

I did not say "sphinx beautiful", just "sphinx compatible". They will not throw errors when parsed and interpreted as ReST.

---
 scripts/qapi/doc.py    | 10 +++++-----
 scripts/qapi/gen.py    |  6 ++++--
 scripts/qapi/parser.py |  9 +++++----
 3 files changed, 14 insertions(+), 11 deletions(-)
scripts/qapi/doc.py +5 -5
index 92f584edcf..c41e9d29f5 100644
@@ -65,11 +65,11 @@ def texi_format(doc):
     Format documentation
 
     Lines starting with:
-    - |: generates an @example
-    - =: generates @section
-    - ==: generates @subsection
-    - 1. or 1): generates an @enumerate @item
-    - */-: generates an @itemize list
+    - ``|:`` generates an @example
+    - ``=:`` generates @section
+    - ``==:`` generates @subsection
+    - ``1.`` or ``1):`` generates an @enumerate @item
+    - ``*/-:`` generates an @itemize list
     """
     ret = ''
     doc = subst_braces(doc)
scripts/qapi/gen.py +4 -2
index bf5552a4e7..3d25a8cff4 100644
@@ -154,9 +154,11 @@ def _bottom(self):
 
 @contextmanager
 def ifcontext(ifcond, *args):
-    """A 'with' statement context manager to wrap with start_if()/end_if()
+    """
+    A 'with' statement context manager to wrap with start_if()/end_if()
 
-    *args: any number of QAPIGenCCode
+    :param ifcond: List of conditionals
+    :param args: any number of QAPIGenCCode
 
     Example::
 
scripts/qapi/parser.py +5 -4
index 165925ca72..00738cea8c 100644
@@ -366,10 +366,11 @@ def append(self, line):
 
         The way that the line is dealt with depends on which part of
         the documentation we're parsing right now:
-        * The body section: ._append_line is ._append_body_line
-        * An argument section: ._append_line is ._append_args_line
-        * A features section: ._append_line is ._append_features_line
-        * An additional section: ._append_line is ._append_various_line
+
+         * The body section: ._append_line is ._append_body_line
+         * An argument section: ._append_line is ._append_args_line
+         * A features section: ._append_line is ._append_features_line
+         * An additional section: ._append_line is ._append_various_line
         """
         line = line[1:]
         if not line:
Signature · 1 line
2.26.2
#13 lore raw

Re: [PATCH v3 04/47] qapi: modify docstrings to be sphinx-compatible

Quoted 36 lines · John Snow

I did not say "sphinx beautiful", just "sphinx compatible". They will not throw errors when parsed and interpreted as ReST.

---
 scripts/qapi/doc.py    | 10 +++++-----
 scripts/qapi/gen.py    |  6 ++++--
 scripts/qapi/parser.py |  9 +++++----
 3 files changed, 14 insertions(+), 11 deletions(-)
scripts/qapi/doc.py +5 -5
index 92f584edcf..c41e9d29f5 100644
@@ -65,11 +65,11 @@ def texi_format(doc):
     Format documentation
 
     Lines starting with:
-    - |: generates an @example
-    - =: generates @section
-    - ==: generates @subsection
-    - 1. or 1): generates an @enumerate @item
-    - */-: generates an @itemize list
+    - ``|:`` generates an @example
+    - ``=:`` generates @section
+    - ``==:`` generates @subsection
+    - ``1.`` or ``1):`` generates an @enumerate @item
+    - ``*/-:`` generates an @itemize list
     """
     ret = ''
     doc = subst_braces(doc)
scripts/qapi/gen.py +0 -0
index bf5552a4e7..3d25a8cff4 100644

scripts/qapi/gen.py def _bottom(self): · line 159

Quoted 7 lines
scripts/qapi/gen.py +2 -1
@@ -154,9 +154,11 @@ def _bottom(self):
 
 @contextmanager
 def ifcontext(ifcond, *args):
-    """A 'with' statement context manager to wrap with start_if()/end_if()
+    """
+    A 'with' statement context manager to wrap with start_if()/end_if()
 

If you're making it compatible, why not take the time to give backticks to start_if and end_if?

Bonus points for setting the "meth" role, but not lost points for not doing it, as I understand this is beyond what you're attempting at this time.

scripts/qapi/gen.py def _bottom(self): · line 162

Quoted 4 lines
scripts/qapi/gen.py +2 -1
-    *args: any number of QAPIGenCCode
+    :param ifcond: List of conditionals
+    :param args: any number of QAPIGenCCode
 

I would argue that this is not a strict sphinx compatibility change, but a fix to a broken docstring.

- Cleber.

author
#14 lore raw

Re: [PATCH v3 04/47] qapi: modify docstrings to be sphinx-compatible

Quoted 38 lines · Cleber Rosa
Quoted 37 lines · John Snow

I did not say "sphinx beautiful", just "sphinx compatible". They will not throw errors when parsed and interpreted as ReST.

---
  scripts/qapi/doc.py    | 10 +++++-----
  scripts/qapi/gen.py    |  6 ++++--
  scripts/qapi/parser.py |  9 +++++----
  3 files changed, 14 insertions(+), 11 deletions(-)
scripts/qapi/doc.py +5 -5
index 92f584edcf..c41e9d29f5 100644
@@ -65,11 +65,11 @@ def texi_format(doc):
      Format documentation
 
      Lines starting with:
-    - |: generates an @example
-    - =: generates @section
-    - ==: generates @subsection
-    - 1. or 1): generates an @enumerate @item
-    - */-: generates an @itemize list
+    - ``|:`` generates an @example
+    - ``=:`` generates @section
+    - ``==:`` generates @subsection
+    - ``1.`` or ``1):`` generates an @enumerate @item
+    - ``*/-:`` generates an @itemize list
      """
      ret = ''
      doc = subst_braces(doc)
scripts/qapi/gen.py +0 -0
index bf5552a4e7..3d25a8cff4 100644
@@ -154,9 +154,11 @@ def _bottom(self):
 
Quoted 10 lines

scripts/qapi/gen.py def _bottom(self): · line 159

Quoted 6 lines
scripts/qapi/gen.py +2 -1
  @contextmanager
  def ifcontext(ifcond, *args):
-    """A 'with' statement context manager to wrap with start_if()/end_if()
+    """
+    A 'with' statement context manager to wrap with start_if()/end_if()
 

If you're making it compatible, why not take the time to give backticks to start_if and end_if?

I forget. Must not have been a good reason, then...?

Quoted 4 lines

Bonus points for setting the "meth" role, but not lost points for not doing it, as I understand this is beyond what you're attempting at this time.

I remain unsold on using explicit roles for references in docstrings, because I'd like to keep the amount of Sphinx-ese syntax down to a minimum if I can. The double backticks are bad enough ...

Quoted 8 lines

scripts/qapi/gen.py def _bottom(self): · line 162

Quoted 4 lines
scripts/qapi/gen.py +2 -1
-    *args: any number of QAPIGenCCode
+    :param ifcond: List of conditionals
+    :param args: any number of QAPIGenCCode
 

I would argue that this is not a strict sphinx compatibility change, but a fix to a broken docstring.

Not entirely correct: the syntax of *args breaks document generation.

(OK, I could make QAPIGenCCode a reference ...)

Quoted 2 lines

- Cleber.

author
#15 lore raw

PATCH v3 5/47 qapi/doc.py: Change code templates from function to string

In reply to John Snow

For whatever reason, when these are stored as functions instead of strings, it confuses sphinx-autodoc into believing them to be docstrings, and it chokes on the syntax.

Keeping them as dumb strings instead avoids the problem.

---
 scripts/qapi/doc.py | 40 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)
scripts/qapi/doc.py +20 -20
index c41e9d29f5..d12eda9e1e 100644
@@ -8,26 +8,26 @@
 from qapi.gen import QAPIGenDoc, QAPISchemaVisitor
 
 
-MSG_FMT = """
+_MSG = '''
 @deftypefn {type} {{}} {name}
 
 {body}{members}{features}{sections}
 @end deftypefn
 
-""".format
+'''
 
-TYPE_FMT = """
+_TYPE = '''
 @deftp {{{type}}} {name}
 
 {body}{members}{features}{sections}
 @end deftp
 
-""".format
+'''
 
-EXAMPLE_FMT = """@example
+_EXAMPLE = '''@example
 {code}
 @end example
-""".format
+'''
 
 
 def subst_strong(doc):
@@ -57,7 +57,7 @@ def texi_example(doc):
     # function to subst_special() or subs_texi_special().  If we do that, we
     # need to delay it until after subst_vars() in texi_format().
     doc = subst_braces(doc).strip('\n')
-    return EXAMPLE_FMT(code=doc)
+    return _EXAMPLE.format(code=doc)
 
 
 def texi_format(doc):
@@ -90,7 +90,7 @@ def texi_format(doc):
         # Make sure to update section "Documentation markup" in
         # docs/devel/qapi-code-gen.txt when fixing this.
         if line.startswith('| '):
-            line = EXAMPLE_FMT(code=line[2:])
+            line = _EXAMPLE.format(code=line[2:])
         elif line.startswith('= '):
             line = '@section ' + line[2:]
         elif line.startswith('== '):
@@ -217,21 +217,21 @@ def texi_sections(doc, ifcond):
 
 
 def texi_type(typ, doc, ifcond, members):
-    return TYPE_FMT(type=typ,
-                    name=doc.symbol,
-                    body=texi_body(doc),
-                    members=members,
-                    features=texi_features(doc),
-                    sections=texi_sections(doc, ifcond))
+    return _TYPE.format(type=typ,
+                        name=doc.symbol,
+                        body=texi_body(doc),
+                        members=members,
+                        features=texi_features(doc),
+                        sections=texi_sections(doc, ifcond))
 
 
 def texi_msg(typ, doc, ifcond, members):
-    return MSG_FMT(type=typ,
-                   name=doc.symbol,
-                   body=texi_body(doc),
-                   members=members,
-                   features=texi_features(doc),
-                   sections=texi_sections(doc, ifcond))
+    return _MSG.format(type=typ,
+                       name=doc.symbol,
+                       body=texi_body(doc),
+                       members=members,
+                       features=texi_features(doc),
+                       sections=texi_sections(doc, ifcond))
 
 
 class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
Signature · 1 line
2.26.2
Reviewed-by on v3 5/47
#16 lore raw

Re: [PATCH v3 05/47] qapi/doc.py: Change code templates from function to string

Quoted 7 lines · John Snow

For whatever reason, when these are stored as functions instead of strings, it confuses sphinx-autodoc into believing them to be docstrings, and it chokes on the syntax.

Keeping them as dumb strings instead avoids the problem.

Quoting my review of the patch that added it: "This .format business is perhaps a bit too clever. But let's move on."

doc.py is about to be replaced, obsoleting this patch. Regardless:

author
#17 lore raw

Re: [PATCH v3 05/47] qapi/doc.py: Change code templates from function to string

Quoted 6 lines · Markus Armbruster
Quoted 4 lines · John Snow

For whatever reason, when these are stored as functions instead of strings, it confuses sphinx-autodoc into believing them to be docstrings, and it chokes on the syntax.

Quoted 7 lines
Quoted 3 lines

Keeping them as dumb strings instead avoids the problem.

Quoting my review of the patch that added it: "This .format business is perhaps a bit too clever. But let's move on."

It seems perfectly fine, and I have no real explanation for why it confuses autodoc.

I think autodoc is using regex that it maybe shouldn't be to identify some constructs instead of the proper Python AST. I looked extremely briefly but made the assessment it wasn't going to be quick.

Quoted 3 lines

doc.py is about to be replaced, obsoleting this patch. Regardless:

Yes, understood -- until such time as it is actually deleted, I am keeping it updated with the style I am trying to enforce just to make my own life simpler.

It's no problem to just trash it when the time comes, but in this case I wanted to explore what stopped us from using Sphinx autodoc earnestly -- and it just so happened that this was the sticking point.

Thanks for looking!

--js

#18 lore raw

Re: [PATCH v3 05/47] qapi/doc.py: Change code templates from function to string

In reply to John Snow
Quoted 4 lines · John Snow

For whatever reason, when these are stored as functions instead of strings, it confuses sphinx-autodoc into believing them to be docstrings, and it chokes on the syntax.

Interesting...

Quoted 2 lines

Keeping them as dumb strings instead avoids the problem.

I actually think it's a more honest approach, and easier to read.

Quoted 13 lines
---
 scripts/qapi/doc.py | 40 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)
scripts/qapi/doc.py +0 -0
index c41e9d29f5..d12eda9e1e 100644
@@ -8,26 +8,26 @@
 from qapi.gen import QAPIGenDoc, QAPISchemaVisitor
 
 

scripts/qapi/doc.py line 18

Quoted 10 lines
scripts/qapi/doc.py +2 -2
-MSG_FMT = """
+_MSG = '''
 @deftypefn {type} {{}} {name}
 
 {body}{members}{features}{sections}
 @end deftypefn
 
-""".format
+'''
 

I know it doesn't make syntactic difference, but is there a reson for also changing the quote style?

- Cleber.

author
#19 lore raw

Re: [PATCH v3 05/47] qapi/doc.py: Change code templates from function to string

Quoted 33 lines · Cleber Rosa
Quoted 4 lines · John Snow

For whatever reason, when these are stored as functions instead of strings, it confuses sphinx-autodoc into believing them to be docstrings, and it chokes on the syntax.

Interesting...

Quoted 2 lines

Keeping them as dumb strings instead avoids the problem.

I actually think it's a more honest approach, and easier to read.

Quoted 20 lines
---
  scripts/qapi/doc.py | 40 ++++++++++++++++++++--------------------
  1 file changed, 20 insertions(+), 20 deletions(-)
scripts/qapi/doc.py +1 -1
index c41e9d29f5..d12eda9e1e 100644
@@ -8,26 +8,26 @@
  from qapi.gen import QAPIGenDoc, QAPISchemaVisitor
 
 
-MSG_FMT = """
+_MSG = '''
  @deftypefn {type} {{}} {name}
 
  {body}{members}{features}{sections}
  @end deftypefn
 
Quoted 9 lines

scripts/qapi/doc.py line 18

Quoted 3 lines
scripts/qapi/doc.py +1 -1
-""".format
+'''
 

I know it doesn't make syntactic difference, but is there a reson for also changing the quote style?

- Cleber.

Was just trying to differentiate it from docstrings to see if it helped. It didn't, but it also didn't hurt, so I left it.

Luckily, this patch won't matter after Peter Maydell's series goes in, so whatever!

author
#20 lore raw

PATCH v3 6/47 docs: enable sphinx-autodoc for scripts/qapi

In reply to John Snow
---
 docs/conf.py                          |  6 +++++-
 docs/devel/index.rst                  |  1 +
 docs/devel/python/index.rst           |  7 +++++++
 docs/devel/python/qapi.commands.rst   |  7 +++++++
 docs/devel/python/qapi.common.rst     |  7 +++++++
 docs/devel/python/qapi.doc.rst        |  7 +++++++
 docs/devel/python/qapi.error.rst      |  7 +++++++
 docs/devel/python/qapi.events.rst     |  7 +++++++
 docs/devel/python/qapi.expr.rst       |  7 +++++++
 docs/devel/python/qapi.gen.rst        |  7 +++++++
 docs/devel/python/qapi.introspect.rst |  7 +++++++
 docs/devel/python/qapi.parser.rst     |  8 ++++++++
 docs/devel/python/qapi.rst            | 26 ++++++++++++++++++++++++++
 docs/devel/python/qapi.schema.rst     |  7 +++++++
 docs/devel/python/qapi.source.rst     |  7 +++++++
 docs/devel/python/qapi.types.rst      |  7 +++++++
 docs/devel/python/qapi.visit.rst      |  7 +++++++
 17 files changed, 131 insertions(+), 1 deletion(-)
 create mode 100644 docs/devel/python/index.rst
 create mode 100644 docs/devel/python/qapi.commands.rst
 create mode 100644 docs/devel/python/qapi.common.rst
 create mode 100644 docs/devel/python/qapi.doc.rst
 create mode 100644 docs/devel/python/qapi.error.rst
 create mode 100644 docs/devel/python/qapi.events.rst
 create mode 100644 docs/devel/python/qapi.expr.rst
 create mode 100644 docs/devel/python/qapi.gen.rst
 create mode 100644 docs/devel/python/qapi.introspect.rst
 create mode 100644 docs/devel/python/qapi.parser.rst
 create mode 100644 docs/devel/python/qapi.rst
 create mode 100644 docs/devel/python/qapi.schema.rst
 create mode 100644 docs/devel/python/qapi.source.rst
 create mode 100644 docs/devel/python/qapi.types.rst
 create mode 100644 docs/devel/python/qapi.visit.rst
docs/conf.py +5 -1
index a68f616d5a..d03ef465a6 100644
@@ -54,6 +54,9 @@
 #
 sys.path.insert(0, os.path.join(qemu_docdir, "sphinx"))
 
+# Make scripts/qapi module available for autodoc
+sys.path.insert(0, os.path.join(qemu_docdir, "../scripts"))
+
 
 # -- General configuration ------------------------------------------------
 
@@ -67,7 +70,8 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile']
+extensions = ['kerneldoc', 'qmp_lexer', 'hxtool',
+              'depfile', 'sphinx.ext.autodoc']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
docs/devel/index.rst +1 -0
index 04773ce076..04726ca787 100644
@@ -31,3 +31,4 @@ Contents:
    reset
    s390-dasd-ipl
    clocks
+   python/index
docs/devel/python/index.rst new file +7 -0
new file mode 100644
index 0000000000..31c470154b
@@ -0,0 +1,7 @@
+qapi
+====
+
+.. toctree::
+   :maxdepth: 4
+
+   qapi
docs/devel/python/qapi.commands.rst new file +7 -0
new file mode 100644
index 0000000000..018f7b08a9
@@ -0,0 +1,7 @@
+qapi.commands module
+====================
+
+.. automodule:: qapi.commands
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.common.rst new file +7 -0
new file mode 100644
index 0000000000..128a90d74b
@@ -0,0 +1,7 @@
+qapi.common module
+==================
+
+.. automodule:: qapi.common
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.doc.rst new file +7 -0
new file mode 100644
index 0000000000..556c2bca1f
@@ -0,0 +1,7 @@
+qapi.doc module
+===============
+
+.. automodule:: qapi.doc
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.error.rst new file +7 -0
new file mode 100644
index 0000000000..980e32b63d
@@ -0,0 +1,7 @@
+qapi.error module
+=================
+
+.. automodule:: qapi.error
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.events.rst new file +7 -0
new file mode 100644
index 0000000000..1fce85b044
@@ -0,0 +1,7 @@
+qapi.events module
+==================
+
+.. automodule:: qapi.events
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.expr.rst new file +7 -0
new file mode 100644
index 0000000000..0660270629
@@ -0,0 +1,7 @@
+qapi.expr module
+================
+
+.. automodule:: qapi.expr
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.gen.rst new file +7 -0
new file mode 100644
index 0000000000..7b495fd4bf
@@ -0,0 +1,7 @@
+qapi.gen module
+===============
+
+.. automodule:: qapi.gen
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.introspect.rst new file +7 -0
new file mode 100644
index 0000000000..f65ebfccd1
@@ -0,0 +1,7 @@
+qapi.introspect module
+======================
+
+.. automodule:: qapi.introspect
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.parser.rst new file +8 -0
new file mode 100644
index 0000000000..1a8f7b347e
@@ -0,0 +1,8 @@
+qapi.parser module
+==================
+
+.. automodule:: qapi.parser
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :private-members:
docs/devel/python/qapi.rst new file +26 -0
new file mode 100644
index 0000000000..24fbf34981
@@ -0,0 +1,26 @@
+qapi package
+============
+
+.. automodule:: qapi
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Submodules
+----------
+
+.. toctree::
+
+   qapi.commands
+   qapi.common
+   qapi.doc
+   qapi.error
+   qapi.events
+   qapi.expr
+   qapi.gen
+   qapi.introspect
+   qapi.parser
+   qapi.schema
+   qapi.source
+   qapi.types
+   qapi.visit
docs/devel/python/qapi.schema.rst new file +7 -0
new file mode 100644
index 0000000000..a08f75ed72
@@ -0,0 +1,7 @@
+qapi.schema module
+==================
+
+.. automodule:: qapi.schema
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.source.rst new file +7 -0
new file mode 100644
index 0000000000..e61e9f6021
@@ -0,0 +1,7 @@
+qapi.source module
+==================
+
+.. automodule:: qapi.source
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.types.rst new file +7 -0
new file mode 100644
index 0000000000..6eea827557
@@ -0,0 +1,7 @@
+qapi.types module
+=================
+
+.. automodule:: qapi.types
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.visit.rst new file +7 -0
new file mode 100644
index 0000000000..84307cbc23
@@ -0,0 +1,7 @@
+qapi.visit module
+=================
+
+.. automodule:: qapi.visit
+   :members:
+   :undoc-members:
+   :show-inheritance:
Signature · 1 line
2.26.2
#21 lore raw

Re: [PATCH v3 06/47] [DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qapi

Quoted 41 lines · John Snow
---
 docs/conf.py                          |  6 +++++-
 docs/devel/index.rst                  |  1 +
 docs/devel/python/index.rst           |  7 +++++++
 docs/devel/python/qapi.commands.rst   |  7 +++++++
 docs/devel/python/qapi.common.rst     |  7 +++++++
 docs/devel/python/qapi.doc.rst        |  7 +++++++
 docs/devel/python/qapi.error.rst      |  7 +++++++
 docs/devel/python/qapi.events.rst     |  7 +++++++
 docs/devel/python/qapi.expr.rst       |  7 +++++++
 docs/devel/python/qapi.gen.rst        |  7 +++++++
 docs/devel/python/qapi.introspect.rst |  7 +++++++
 docs/devel/python/qapi.parser.rst     |  8 ++++++++
 docs/devel/python/qapi.rst            | 26 ++++++++++++++++++++++++++
 docs/devel/python/qapi.schema.rst     |  7 +++++++
 docs/devel/python/qapi.source.rst     |  7 +++++++
 docs/devel/python/qapi.types.rst      |  7 +++++++
 docs/devel/python/qapi.visit.rst      |  7 +++++++
 17 files changed, 131 insertions(+), 1 deletion(-)
 create mode 100644 docs/devel/python/index.rst
 create mode 100644 docs/devel/python/qapi.commands.rst
 create mode 100644 docs/devel/python/qapi.common.rst
 create mode 100644 docs/devel/python/qapi.doc.rst
 create mode 100644 docs/devel/python/qapi.error.rst
 create mode 100644 docs/devel/python/qapi.events.rst
 create mode 100644 docs/devel/python/qapi.expr.rst
 create mode 100644 docs/devel/python/qapi.gen.rst
 create mode 100644 docs/devel/python/qapi.introspect.rst
 create mode 100644 docs/devel/python/qapi.parser.rst
 create mode 100644 docs/devel/python/qapi.rst
 create mode 100644 docs/devel/python/qapi.schema.rst
 create mode 100644 docs/devel/python/qapi.source.rst
 create mode 100644 docs/devel/python/qapi.types.rst
 create mode 100644 docs/devel/python/qapi.visit.rst
docs/conf.py +0 -0
index a68f616d5a..d03ef465a6 100644

docs/conf.py line 59

Quoted 6 lines
docs/conf.py +3 -0
@@ -54,6 +54,9 @@
 #
 sys.path.insert(0, os.path.join(qemu_docdir, "sphinx"))
 
+# Make scripts/qapi module available for autodoc
+sys.path.insert(0, os.path.join(qemu_docdir, "../scripts"))
+

Do I hear qapi asking to moved out of "scripts" and becoming more like "python/qemu"?

Quoted 159 lines
docs/conf.py +2 -1
 # -- General configuration ------------------------------------------------
 
@@ -67,7 +70,8 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile']
+extensions = ['kerneldoc', 'qmp_lexer', 'hxtool',
+              'depfile', 'sphinx.ext.autodoc']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
docs/devel/index.rst +1 -0
index 04773ce076..04726ca787 100644
@@ -31,3 +31,4 @@ Contents:
    reset
    s390-dasd-ipl
    clocks
+   python/index
docs/devel/python/index.rst new file +7 -0
new file mode 100644
index 0000000000..31c470154b
@@ -0,0 +1,7 @@
+qapi
+====
+
+.. toctree::
+   :maxdepth: 4
+
+   qapi
docs/devel/python/qapi.commands.rst new file +7 -0
new file mode 100644
index 0000000000..018f7b08a9
@@ -0,0 +1,7 @@
+qapi.commands module
+====================
+
+.. automodule:: qapi.commands
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.common.rst new file +7 -0
new file mode 100644
index 0000000000..128a90d74b
@@ -0,0 +1,7 @@
+qapi.common module
+==================
+
+.. automodule:: qapi.common
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.doc.rst new file +7 -0
new file mode 100644
index 0000000000..556c2bca1f
@@ -0,0 +1,7 @@
+qapi.doc module
+===============
+
+.. automodule:: qapi.doc
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.error.rst new file +7 -0
new file mode 100644
index 0000000000..980e32b63d
@@ -0,0 +1,7 @@
+qapi.error module
+=================
+
+.. automodule:: qapi.error
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.events.rst new file +7 -0
new file mode 100644
index 0000000000..1fce85b044
@@ -0,0 +1,7 @@
+qapi.events module
+==================
+
+.. automodule:: qapi.events
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.expr.rst new file +7 -0
new file mode 100644
index 0000000000..0660270629
@@ -0,0 +1,7 @@
+qapi.expr module
+================
+
+.. automodule:: qapi.expr
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.gen.rst new file +7 -0
new file mode 100644
index 0000000000..7b495fd4bf
@@ -0,0 +1,7 @@
+qapi.gen module
+===============
+
+.. automodule:: qapi.gen
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.introspect.rst new file +7 -0
new file mode 100644
index 0000000000..f65ebfccd1
@@ -0,0 +1,7 @@
+qapi.introspect module
+======================
+
+.. automodule:: qapi.introspect
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.parser.rst new file +8 -0
new file mode 100644
index 0000000000..1a8f7b347e
@@ -0,0 +1,8 @@
+qapi.parser module
+==================
+
+.. automodule:: qapi.parser
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :private-members:
docs/devel/python/qapi.rst new file +0 -0
new file mode 100644
index 0000000000..24fbf34981

docs/devel/python/qapi.rst line 8

Quoted 8 lines
docs/devel/python/qapi.rst new file +8 -0
@@ -0,0 +1,26 @@
+qapi package
+============
+
+.. automodule:: qapi
+   :members:
+   :undoc-members:
+   :show-inheritance:
+

This renders *very* timidly, ending up almost invisible. Also, there's no useful introduction to what this "qapi" when you enter this section.

Maybe create a "API reference" section, explaining it first?

- Cleber.

author
#22 lore raw

Re: [PATCH v3 06/47] [DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qapi

Quoted 45 lines · Cleber Rosa
Quoted 44 lines · John Snow
---
  docs/conf.py                          |  6 +++++-
  docs/devel/index.rst                  |  1 +
  docs/devel/python/index.rst           |  7 +++++++
  docs/devel/python/qapi.commands.rst   |  7 +++++++
  docs/devel/python/qapi.common.rst     |  7 +++++++
  docs/devel/python/qapi.doc.rst        |  7 +++++++
  docs/devel/python/qapi.error.rst      |  7 +++++++
  docs/devel/python/qapi.events.rst     |  7 +++++++
  docs/devel/python/qapi.expr.rst       |  7 +++++++
  docs/devel/python/qapi.gen.rst        |  7 +++++++
  docs/devel/python/qapi.introspect.rst |  7 +++++++
  docs/devel/python/qapi.parser.rst     |  8 ++++++++
  docs/devel/python/qapi.rst            | 26 ++++++++++++++++++++++++++
  docs/devel/python/qapi.schema.rst     |  7 +++++++
  docs/devel/python/qapi.source.rst     |  7 +++++++
  docs/devel/python/qapi.types.rst      |  7 +++++++
  docs/devel/python/qapi.visit.rst      |  7 +++++++
  17 files changed, 131 insertions(+), 1 deletion(-)

create mode 100644 docs/devel/python/index.rst create mode 100644 docs/devel/python/qapi.commands.rst create mode 100644 docs/devel/python/qapi.common.rst create mode 100644 docs/devel/python/qapi.doc.rst create mode 100644 docs/devel/python/qapi.error.rst create mode 100644 docs/devel/python/qapi.events.rst create mode 100644 docs/devel/python/qapi.expr.rst create mode 100644 docs/devel/python/qapi.gen.rst create mode 100644 docs/devel/python/qapi.introspect.rst create mode 100644 docs/devel/python/qapi.parser.rst create mode 100644 docs/devel/python/qapi.rst create mode 100644 docs/devel/python/qapi.schema.rst create mode 100644 docs/devel/python/qapi.source.rst create mode 100644 docs/devel/python/qapi.types.rst create mode 100644 docs/devel/python/qapi.visit.rst

docs/conf.py +0 -0
index a68f616d5a..d03ef465a6 100644
@@ -54,6 +54,9 @@
  #
  sys.path.insert(0, os.path.join(qemu_docdir, "sphinx"))
 
Quoted 7 lines

docs/conf.py line 59

Quoted 3 lines
docs/conf.py +3 -0
+# Make scripts/qapi module available for autodoc
+sys.path.insert(0, os.path.join(qemu_docdir, "../scripts"))
+

Do I hear qapi asking to moved out of "scripts" and becoming more like "python/qemu"?

Once all six parts of this series goes in I will move it to the python folder, but you're about 150 patches early.

Quoted 172 lines
Quoted 159 lines
docs/conf.py +2 -1
  # -- General configuration ------------------------------------------------
 
@@ -67,7 +70,8 @@
  # Add any Sphinx extension module names here, as strings. They can be
  # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
  # ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile']
+extensions = ['kerneldoc', 'qmp_lexer', 'hxtool',
+              'depfile', 'sphinx.ext.autodoc']
 
  # Add any paths that contain templates here, relative to this directory.
  templates_path = ['_templates']
docs/devel/index.rst +1 -0
index 04773ce076..04726ca787 100644
@@ -31,3 +31,4 @@ Contents:
     reset
     s390-dasd-ipl
     clocks
+   python/index
docs/devel/python/index.rst new file +7 -0
new file mode 100644
index 0000000000..31c470154b
@@ -0,0 +1,7 @@
+qapi
+====
+
+.. toctree::
+   :maxdepth: 4
+
+   qapi
docs/devel/python/qapi.commands.rst new file +7 -0
new file mode 100644
index 0000000000..018f7b08a9
@@ -0,0 +1,7 @@
+qapi.commands module
+====================
+
+.. automodule:: qapi.commands
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.common.rst new file +7 -0
new file mode 100644
index 0000000000..128a90d74b
@@ -0,0 +1,7 @@
+qapi.common module
+==================
+
+.. automodule:: qapi.common
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.doc.rst new file +7 -0
new file mode 100644
index 0000000000..556c2bca1f
@@ -0,0 +1,7 @@
+qapi.doc module
+===============
+
+.. automodule:: qapi.doc
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.error.rst new file +7 -0
new file mode 100644
index 0000000000..980e32b63d
@@ -0,0 +1,7 @@
+qapi.error module
+=================
+
+.. automodule:: qapi.error
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.events.rst new file +7 -0
new file mode 100644
index 0000000000..1fce85b044
@@ -0,0 +1,7 @@
+qapi.events module
+==================
+
+.. automodule:: qapi.events
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.expr.rst new file +7 -0
new file mode 100644
index 0000000000..0660270629
@@ -0,0 +1,7 @@
+qapi.expr module
+================
+
+.. automodule:: qapi.expr
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.gen.rst new file +7 -0
new file mode 100644
index 0000000000..7b495fd4bf
@@ -0,0 +1,7 @@
+qapi.gen module
+===============
+
+.. automodule:: qapi.gen
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.introspect.rst new file +7 -0
new file mode 100644
index 0000000000..f65ebfccd1
@@ -0,0 +1,7 @@
+qapi.introspect module
+======================
+
+.. automodule:: qapi.introspect
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.parser.rst new file +8 -0
new file mode 100644
index 0000000000..1a8f7b347e
@@ -0,0 +1,8 @@
+qapi.parser module
+==================
+
+.. automodule:: qapi.parser
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :private-members:
docs/devel/python/qapi.rst new file +0 -0
new file mode 100644
index 0000000000..24fbf34981

docs/devel/python/qapi.rst line 8

Quoted 8 lines
docs/devel/python/qapi.rst new file +8 -0
@@ -0,0 +1,26 @@
+qapi package
+============
+
+.. automodule:: qapi
+   :members:
+   :undoc-members:
+   :show-inheritance:
+

This renders *very* timidly, ending up almost invisible. Also, there's no useful introduction to what this "qapi" when you enter this section.

Quoted 2 lines

Maybe create a "API reference" section, explaining it first?

Respectfully, "DO NOT MERGE" is written in large capital letters in the subject of this patch.

You are free to try to format the manual as you please at a later date, but it's not something that I will be doing right now. The purpose of this patch is solely to prove that my docstring formatting adheres to some kind of standard, and the method to prove it is nontrivial, hence these patches.

--js

author
#23 lore raw

PATCH v3 7/47 qapi-gen: Separate arg-parsing from generation

In reply to John Snow

This is a minor re-work of the entrypoint script. It isolates a generate() method from the actual command-line mechanism.

The regex match error checking was modified slightly to check that we actually got a regex match.

---
 scripts/qapi-gen.py | 87 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 63 insertions(+), 24 deletions(-)
scripts/qapi-gen.py +63 -24
index 4b03f7d53b..59becba3e1 100644
@@ -1,9 +1,13 @@
 #!/usr/bin/env python3
-# QAPI generator
-#
+
 # This work is licensed under the terms of the GNU GPL, version 2 or later.
 # See the COPYING file in the top-level directory.
 
+"""
+QAPI Generator
+
+This script is the main entry point for generating C code from the QAPI schema.
+"""
 
 import argparse
 import re
@@ -11,21 +15,65 @@
 
 from qapi.commands import gen_commands
 from qapi.doc import gen_doc
+from qapi.error import QAPIError
 from qapi.events import gen_events
 from qapi.introspect import gen_introspect
-from qapi.schema import QAPIError, QAPISchema
+from qapi.schema import QAPISchema
 from qapi.types import gen_types
 from qapi.visit import gen_visit
 
 
-def main(argv):
+DEFAULT_OUTPUT_DIR = ''
+DEFAULT_PREFIX = ''
+
+
+def generate(schema_file: str,
+             output_dir: str,
+             prefix: str,
+             unmask: bool = False,
+             builtins: bool = False) -> None:
+    """
+    generate uses a given schema to produce C code in the target directory.
+
+    :param schema_file: The primary QAPI schema file.
+    :param output_dir: The output directory to store generated code.
+    :param prefix: Optional C-code prefix for symbol names.
+    :param unmask: Expose non-ABI names through introspection?
+    :param builtins: Generate code for built-in types?
+
+    :raise QAPIError: On failures.
+    """
+    match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix)
+    if match and match.end() != len(prefix):
+        msg = "funny character '{:s}' in prefix '{:s}'".format(
+            prefix[match.end()], prefix)
+        raise QAPIError('', None, msg)
+
+    schema = QAPISchema(schema_file)
+    gen_types(schema, output_dir, prefix, builtins)
+    gen_visit(schema, output_dir, prefix, builtins)
+    gen_commands(schema, output_dir, prefix)
+    gen_events(schema, output_dir, prefix)
+    gen_introspect(schema, output_dir, prefix, unmask)
+    gen_doc(schema, output_dir, prefix)
+
+
+def main() -> int:
+    """
+    gapi-gen shell script entrypoint.
+    Expects arguments via sys.argv, see --help for details.
+
+    :return: int, 0 on success, 1 on failure.
+    """
     parser = argparse.ArgumentParser(
         description='Generate code from a QAPI schema')
     parser.add_argument('-b', '--builtins', action='store_true',
                         help="generate code for built-in types")
-    parser.add_argument('-o', '--output-dir', action='store', default='',
+    parser.add_argument('-o', '--output-dir', action='store',
+                        default=DEFAULT_OUTPUT_DIR,
                         help="write output to directory OUTPUT_DIR")
-    parser.add_argument('-p', '--prefix', action='store', default='',
+    parser.add_argument('-p', '--prefix', action='store',
+                        default=DEFAULT_PREFIX,
                         help="prefix for symbols")
     parser.add_argument('-u', '--unmask-non-abi-names', action='store_true',
                         dest='unmask',
@@ -33,26 +81,17 @@ def main(argv):
     parser.add_argument('schema', action='store')
     args = parser.parse_args()
 
-    match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', args.prefix)
-    if match.end() != len(args.prefix):
-        print("%s: 'funny character '%s' in argument of --prefix"
-              % (sys.argv[0], args.prefix[match.end()]),
-              file=sys.stderr)
-        sys.exit(1)
-
     try:
-        schema = QAPISchema(args.schema)
+        generate(args.schema,
+                 output_dir=args.output_dir,
+                 prefix=args.prefix,
+                 unmask=args.unmask,
+                 builtins=args.builtins)
     except QAPIError as err:
-        print(err, file=sys.stderr)
-        exit(1)
-
-    gen_types(schema, args.output_dir, args.prefix, args.builtins)
-    gen_visit(schema, args.output_dir, args.prefix, args.builtins)
-    gen_commands(schema, args.output_dir, args.prefix)
-    gen_events(schema, args.output_dir, args.prefix)
-    gen_introspect(schema, args.output_dir, args.prefix, args.unmask)
-    gen_doc(schema, args.output_dir, args.prefix)
+        print(f"{sys.argv[0]}: {str(err)}", file=sys.stderr)
+        return 1
+    return 0
 
 
 if __name__ == '__main__':
-    main(sys.argv)
+    sys.exit(main())
Signature · 1 line
2.26.2
Reviewed-by Tested-by on v3 7/47
#24 lore raw

Re: [PATCH v3 07/47] qapi-gen: Separate arg-parsing from generation

Quoted 8 lines · John Snow

This is a minor re-work of the entrypoint script. It isolates a generate() method from the actual command-line mechanism.

The regex match error checking was modified slightly to check that we actually got a regex match.

author
#25 lore raw

PATCH v3 8/47 qapi: move generator entrypoint into module

In reply to John Snow

As part of delinting and adding type hints to the QAPI generator, it's helpful for the entrypoint to be part of the package, only leaving a very tiny entrypoint shim outside of the module.

---
 scripts/qapi-gen.py  | 90 +++----------------------------------------
 scripts/qapi/main.py | 91 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+), 84 deletions(-)
 create mode 100644 scripts/qapi/main.py
scripts/qapi-gen.py +6 -84
index 59becba3e1..f3518d29a5 100644
@@ -4,94 +4,16 @@
 # See the COPYING file in the top-level directory.
 
 """
-QAPI Generator
+QAPI code generation execution shim.
 
-This script is the main entry point for generating C code from the QAPI schema.
+This standalone script exists primarily to facilitate the running of the QAPI
+code generator without needing to install the python module to the current
+execution environment.
 """
 
-import argparse
-import re
 import sys
 
-from qapi.commands import gen_commands
-from qapi.doc import gen_doc
-from qapi.error import QAPIError
-from qapi.events import gen_events
-from qapi.introspect import gen_introspect
-from qapi.schema import QAPISchema
-from qapi.types import gen_types
-from qapi.visit import gen_visit
-
-
-DEFAULT_OUTPUT_DIR = ''
-DEFAULT_PREFIX = ''
-
-
-def generate(schema_file: str,
-             output_dir: str,
-             prefix: str,
-             unmask: bool = False,
-             builtins: bool = False) -> None:
-    """
-    generate uses a given schema to produce C code in the target directory.
-
-    :param schema_file: The primary QAPI schema file.
-    :param output_dir: The output directory to store generated code.
-    :param prefix: Optional C-code prefix for symbol names.
-    :param unmask: Expose non-ABI names through introspection?
-    :param builtins: Generate code for built-in types?
-
-    :raise QAPIError: On failures.
-    """
-    match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix)
-    if match and match.end() != len(prefix):
-        msg = "funny character '{:s}' in prefix '{:s}'".format(
-            prefix[match.end()], prefix)
-        raise QAPIError('', None, msg)
-
-    schema = QAPISchema(schema_file)
-    gen_types(schema, output_dir, prefix, builtins)
-    gen_visit(schema, output_dir, prefix, builtins)
-    gen_commands(schema, output_dir, prefix)
-    gen_events(schema, output_dir, prefix)
-    gen_introspect(schema, output_dir, prefix, unmask)
-    gen_doc(schema, output_dir, prefix)
-
-
-def main() -> int:
-    """
-    gapi-gen shell script entrypoint.
-    Expects arguments via sys.argv, see --help for details.
-
-    :return: int, 0 on success, 1 on failure.
-    """
-    parser = argparse.ArgumentParser(
-        description='Generate code from a QAPI schema')
-    parser.add_argument('-b', '--builtins', action='store_true',
-                        help="generate code for built-in types")
-    parser.add_argument('-o', '--output-dir', action='store',
-                        default=DEFAULT_OUTPUT_DIR,
-                        help="write output to directory OUTPUT_DIR")
-    parser.add_argument('-p', '--prefix', action='store',
-                        default=DEFAULT_PREFIX,
-                        help="prefix for symbols")
-    parser.add_argument('-u', '--unmask-non-abi-names', action='store_true',
-                        dest='unmask',
-                        help="expose non-ABI names in introspection")
-    parser.add_argument('schema', action='store')
-    args = parser.parse_args()
-
-    try:
-        generate(args.schema,
-                 output_dir=args.output_dir,
-                 prefix=args.prefix,
-                 unmask=args.unmask,
-                 builtins=args.builtins)
-    except QAPIError as err:
-        print(f"{sys.argv[0]}: {str(err)}", file=sys.stderr)
-        return 1
-    return 0
-
+from qapi import main
 
 if __name__ == '__main__':
-    sys.exit(main())
+    sys.exit(main.main())
scripts/qapi/main.py new file +91 -0

91 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#26 lore raw

PATCH v3 9/47 docs: add scripts/qapi/main to python manual

In reply to John Snow
---
 docs/devel/python/qapi.main.rst | 7 +++++++
 docs/devel/python/qapi.rst      | 1 +
 2 files changed, 8 insertions(+)
 create mode 100644 docs/devel/python/qapi.main.rst
docs/devel/python/qapi.main.rst new file +7 -0
new file mode 100644
index 0000000000..1255fcda63
@@ -0,0 +1,7 @@
+qapi.main module
+================
+
+.. automodule:: qapi.main
+   :members:
+   :undoc-members:
+   :show-inheritance:
docs/devel/python/qapi.rst +1 -0
index 24fbf34981..1d73509e98 100644
@@ -19,6 +19,7 @@ Submodules
    qapi.expr
    qapi.gen
    qapi.introspect
+   qapi.main
    qapi.parser
    qapi.schema
    qapi.source
Signature · 1 line
2.26.2
author
#27 lore raw

PATCH v3 10/47 qapi: Prefer explicit relative imports

In reply to John Snow

All of the QAPI include statements are changed to be package-aware, as explicit relative imports.

A quirk of Python packages is that the name of the package exists only *outside* of the package. This means that to a module inside of the qapi folder, there is inherently no such thing as the "qapi" package. The reason these imports work is because the "qapi" package exists in the context of the caller -- the execution shim, where sys.path includes a directory that has a 'qapi' folder in it.

When we write "from qapi import sibling", we are NOT referencing the folder 'qapi', but rather "any package named qapi in sys.path". If you should so happen to have a 'qapi' package in your path, it will use *that* package.

When we write "from .sibling import foo", we always reference explicitly our sibling module; guaranteeing consistency in *where* we are importing these modules from.

This can be useful when working with virtual environments and packages in development mode. In development mode, a package is installed as a series of symlinks that forwards to your same source files. The problem arises because code quality checkers will follow "import qapi.x" to the "installed" version instead of the sibling file and -- even though they are the same file -- they have different module paths, and this causes cyclic import problems, false positive type mismatch errors, and more.

It can also be useful when dealing with hierarchical packages, e.g. if we allow qemu.core.qmp, qemu.qapi.parser, etc.

---
 scripts/qapi/commands.py   |  4 ++--
 scripts/qapi/doc.py        |  2 +-
 scripts/qapi/events.py     |  8 ++++----
 scripts/qapi/expr.py       |  4 ++--
 scripts/qapi/gen.py        |  4 ++--
 scripts/qapi/introspect.py |  8 ++++----
 scripts/qapi/main.py       | 16 ++++++++--------
 scripts/qapi/parser.py     |  4 ++--
 scripts/qapi/schema.py     |  8 ++++----
 scripts/qapi/types.py      |  6 +++---
 scripts/qapi/visit.py      |  6 +++---
 11 files changed, 35 insertions(+), 35 deletions(-)
scripts/qapi/commands.py +2 -2
index 3cf9e1110b..ce5926146a 100644
@@ -13,8 +13,8 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
+from .common import *
+from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
 
 
 def gen_command_decl(name, arg_type, boxed, ret_type):
scripts/qapi/doc.py +1 -1
index d12eda9e1e..b764a8ccc0 100644
@@ -5,7 +5,7 @@
 """This script produces the documentation of a qapi schema in texinfo format"""
 
 import re
-from qapi.gen import QAPIGenDoc, QAPISchemaVisitor
+from .gen import QAPIGenDoc, QAPISchemaVisitor
 
 
 _MSG = '''
scripts/qapi/events.py +4 -4
index b544af5a1c..0467272438 100644
@@ -12,10 +12,10 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPISchemaModularCVisitor, ifcontext
-from qapi.schema import QAPISchemaEnumMember
-from qapi.types import gen_enum, gen_enum_lookup
+from .common import *
+from .gen import QAPISchemaModularCVisitor, ifcontext
+from .schema import QAPISchemaEnumMember
+from .types import gen_enum, gen_enum_lookup
 
 
 def build_event_send_proto(name, arg_type, boxed):
scripts/qapi/expr.py +2 -2
index 2942520399..03b31ecfc1 100644
@@ -16,8 +16,8 @@
 
 import re
 from collections import OrderedDict
-from qapi.common import c_name
-from qapi.error import QAPISemError
+from .common import c_name
+from .error import QAPISemError
 
 
 # Names must be letters, numbers, -, and _.  They must start with letter,
scripts/qapi/gen.py +2 -2
index 3d25a8cff4..f82ce9b661 100644
@@ -17,8 +17,8 @@
 import re
 from contextlib import contextmanager
 
-from qapi.common import *
-from qapi.schema import QAPISchemaVisitor
+from .common import *
+from .schema import QAPISchemaVisitor
 
 
 class QAPIGen:
scripts/qapi/introspect.py +4 -4
index 23652be810..2a34cd1e8e 100644
@@ -10,10 +10,10 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPISchemaMonolithicCVisitor
-from qapi.schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
-                         QAPISchemaType)
+from .common import *
+from .gen import QAPISchemaMonolithicCVisitor
+from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
+                     QAPISchemaType)
 
 
 def _make_tree(obj, ifcond, features, extra=None):
scripts/qapi/main.py +8 -8
index 18c246bbb4..3f8338ade8 100644
@@ -11,14 +11,14 @@
 import re
 import sys
 
-from qapi.commands import gen_commands
-from qapi.doc import gen_doc
-from qapi.error import QAPIError
-from qapi.events import gen_events
-from qapi.introspect import gen_introspect
-from qapi.schema import QAPISchema
-from qapi.types import gen_types
-from qapi.visit import gen_visit
+from .commands import gen_commands
+from .doc import gen_doc
+from .error import QAPIError
+from .events import gen_events
+from .introspect import gen_introspect
+from .schema import QAPISchema
+from .types import gen_types
+from .visit import gen_visit
 
 
 DEFAULT_OUTPUT_DIR = ''
scripts/qapi/parser.py +2 -2
index 00738cea8c..76d28c1ce9 100644
@@ -18,8 +18,8 @@
 import re
 from collections import OrderedDict
 
-from qapi.error import QAPIParseError, QAPISemError
-from qapi.source import QAPISourceInfo
+from .error import QAPIParseError, QAPISemError
+from .source import QAPISourceInfo
 
 
 class QAPISchemaParser:
scripts/qapi/schema.py +4 -4
index 78309a00f0..a835ee6fde 100644
@@ -18,10 +18,10 @@
 import re
 from collections import OrderedDict
 
-from qapi.common import c_name, pointer_suffix
-from qapi.error import QAPIError, QAPISemError
-from qapi.expr import check_exprs
-from qapi.parser import QAPISchemaParser
+from .common import c_name, pointer_suffix
+from .error import QAPIError, QAPISemError
+from .expr import check_exprs
+from .parser import QAPISchemaParser
 
 
 class QAPISchemaEntity:
scripts/qapi/types.py +3 -3
index 3640f17cd6..ca9a5aacb3 100644
@@ -13,9 +13,9 @@
 # See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPISchemaModularCVisitor, ifcontext
-from qapi.schema import QAPISchemaEnumMember, QAPISchemaObjectType
+from .common import *
+from .gen import QAPISchemaModularCVisitor, ifcontext
+from .schema import QAPISchemaEnumMember, QAPISchemaObjectType
 
 
 # variants must be emitted before their container; track what has already
scripts/qapi/visit.py +3 -3
index cdabc5fa28..7850f6e848 100644
@@ -13,9 +13,9 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPISchemaModularCVisitor, ifcontext
-from qapi.schema import QAPISchemaObjectType
+from .common import *
+from .gen import QAPISchemaModularCVisitor, ifcontext
+from .schema import QAPISchemaObjectType
 
 
 def gen_visit_decl(name, scalar=False):
Signature · 1 line
2.26.2
author
#28 lore raw

PATCH v3 11/47 qapi: Remove wildcard includes

In reply to John Snow

Wildcard includes become hard to manage when refactoring and dealing with circular dependencies with strictly typed mypy.

flake8 also flags each one as a warning, as it is not smart enough to know which names exist in the imported file.

Remove them and include things explicitly by name instead.

---
 scripts/qapi/commands.py   |  6 +++++-
 scripts/qapi/events.py     |  7 ++++++-
 scripts/qapi/gen.py        | 12 +++++++++---
 scripts/qapi/introspect.py |  7 ++++++-
 scripts/qapi/types.py      |  8 +++++++-
 scripts/qapi/visit.py      | 10 +++++++++-
 6 files changed, 42 insertions(+), 8 deletions(-)
scripts/qapi/commands.py +5 -1
index ce5926146a..e1df0e341f 100644
@@ -13,7 +13,11 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+    build_params,
+    c_name,
+    mcgen,
+)
 from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
 
 
scripts/qapi/events.py +6 -1
index 0467272438..6b3afa14d7 100644
@@ -12,7 +12,12 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+    build_params,
+    c_enum_const,
+    c_name,
+    mcgen,
+)
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaEnumMember
 from .types import gen_enum, gen_enum_lookup
scripts/qapi/gen.py +9 -3
index f82ce9b661..d37339fe8f 100644
@@ -11,13 +11,19 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-
+from contextlib import contextmanager
 import errno
 import os
 import re
-from contextlib import contextmanager
 
-from .common import *
+from .common import (
+    c_fname,
+    gen_endif,
+    gen_if,
+    guardend,
+    guardstart,
+    mcgen,
+)
 from .schema import QAPISchemaVisitor
 
 
scripts/qapi/introspect.py +6 -1
index 2a34cd1e8e..b036fcf9ce 100644
@@ -10,7 +10,12 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+    c_name,
+    gen_endif,
+    gen_if,
+    mcgen,
+)
 from .gen import QAPISchemaMonolithicCVisitor
 from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
                      QAPISchemaType)
scripts/qapi/types.py +7 -1
index ca9a5aacb3..53b47f9e58 100644
@@ -13,7 +13,13 @@
 # See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+    c_enum_const,
+    c_name,
+    gen_endif,
+    gen_if,
+    mcgen,
+)
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaEnumMember, QAPISchemaObjectType
 
scripts/qapi/visit.py +9 -1
index 7850f6e848..ea277e7704 100644
@@ -13,7 +13,15 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+    c_enum_const,
+    c_name,
+    gen_endif,
+    gen_if,
+    mcgen,
+    pop_indent,
+    push_indent,
+)
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaObjectType
 
Signature · 1 line
2.26.2
author
#29 lore raw

PATCH v3 12/47 qapi: enforce import order/styling with isort

In reply to John Snow

While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us.

force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself.

force_grid_wrap=4: Four or more imports from a single module will force the one-per-line style that's more git-friendly. This will generally happen for 'typing' imports.

multi_line_output=3: Uses the one-per-line indented style for long imports.

include_trailing_comma: Adds a comma to the last import in a group, which makes git conflicts nicer to deal with, generally.

---
 scripts/qapi/.isort.cfg    | 5 +++++
 scripts/qapi/commands.py   | 6 +-----
 scripts/qapi/doc.py        | 2 +-
 scripts/qapi/expr.py       | 4 ++--
 scripts/qapi/introspect.py | 3 +--
 scripts/qapi/main.py       | 1 -
 scripts/qapi/parser.py     | 2 +-
 scripts/qapi/schema.py     | 2 +-
 scripts/qapi/types.py      | 1 -
 9 files changed, 12 insertions(+), 14 deletions(-)
 create mode 100644 scripts/qapi/.isort.cfg
scripts/qapi/.isort.cfg new file +5 -0
new file mode 100644
index 0000000000..b0aeffec26
@@ -0,0 +1,5 @@
+[settings]
+force_sort_within_sections=True
+force_grid_wrap=4
+multi_line_output=3
+include_trailing_comma=True
scripts/qapi/commands.py +1 -5
index e1df0e341f..64ed5278f9 100644
@@ -13,11 +13,7 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import (
-    build_params,
-    c_name,
-    mcgen,
-)
+from .common import build_params, c_name, mcgen
 from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
 
 
scripts/qapi/doc.py +1 -1
index b764a8ccc0..1acb773e0a 100644
@@ -5,9 +5,9 @@
 """This script produces the documentation of a qapi schema in texinfo format"""
 
 import re
+
 from .gen import QAPIGenDoc, QAPISchemaVisitor
 
-
 _MSG = '''
 @deftypefn {type} {{}} {name}
 
scripts/qapi/expr.py +2 -2
index 03b31ecfc1..3e952a1462 100644
@@ -14,12 +14,12 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-import re
 from collections import OrderedDict
+import re
+
 from .common import c_name
 from .error import QAPISemError
 
-
 # Names must be letters, numbers, -, and _.  They must start with letter,
 # except for downstream extensions which must start with __RFQDN_.
 # Dots are only valid in the downstream extension prefix.
scripts/qapi/introspect.py +1 -2
index b036fcf9ce..2850121cbd 100644
@@ -17,8 +17,7 @@
     mcgen,
 )
 from .gen import QAPISchemaMonolithicCVisitor
-from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
-                     QAPISchemaType)
+from .schema import QAPISchemaArrayType, QAPISchemaBuiltinType, QAPISchemaType
 
 
 def _make_tree(obj, ifcond, features, extra=None):
scripts/qapi/main.py +0 -1
index 3f8338ade8..b2f20581fd 100644
@@ -20,7 +20,6 @@
 from .types import gen_types
 from .visit import gen_visit
 
-
 DEFAULT_OUTPUT_DIR = ''
 DEFAULT_PREFIX = ''
 
scripts/qapi/parser.py +1 -1
index 76d28c1ce9..fd89e2188b 100644
@@ -14,9 +14,9 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
+from collections import OrderedDict
 import os
 import re
-from collections import OrderedDict
 
 from .error import QAPIParseError, QAPISemError
 from .source import QAPISourceInfo
scripts/qapi/schema.py +1 -1
index a835ee6fde..093f7a38d8 100644
@@ -14,9 +14,9 @@
 
 # TODO catching name collisions in generated code would be nice
 
+from collections import OrderedDict
 import os
 import re
-from collections import OrderedDict
 
 from .common import c_name, pointer_suffix
 from .error import QAPIError, QAPISemError
scripts/qapi/types.py +0 -1
index 53b47f9e58..cc6dad4c89 100644
@@ -23,7 +23,6 @@
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaEnumMember, QAPISchemaObjectType
 
-
 # variants must be emitted before their container; track what has already
 # been output
 objects_seen = set()
Signature · 1 line
2.26.2
#30 lore raw

Re: [PATCH v3 12/47] qapi: enforce import order/styling with isort

Quoted 48 lines · John Snow

While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us.

force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself.

force_grid_wrap=4: Four or more imports from a single module will force the one-per-line style that's more git-friendly. This will generally happen for 'typing' imports.

multi_line_output=3: Uses the one-per-line indented style for long imports.

include_trailing_comma: Adds a comma to the last import in a group, which makes git conflicts nicer to deal with, generally.

---
 scripts/qapi/.isort.cfg    | 5 +++++
 scripts/qapi/commands.py   | 6 +-----
 scripts/qapi/doc.py        | 2 +-
 scripts/qapi/expr.py       | 4 ++--
 scripts/qapi/introspect.py | 3 +--
 scripts/qapi/main.py       | 1 -
 scripts/qapi/parser.py     | 2 +-
 scripts/qapi/schema.py     | 2 +-
 scripts/qapi/types.py      | 1 -
 9 files changed, 12 insertions(+), 14 deletions(-)
 create mode 100644 scripts/qapi/.isort.cfg
scripts/qapi/.isort.cfg new file +5 -0
new file mode 100644
index 0000000000..b0aeffec26
@@ -0,0 +1,5 @@
+[settings]
+force_sort_within_sections=True
+force_grid_wrap=4
+multi_line_output=3
+include_trailing_comma=True
scripts/qapi/commands.py +0 -0
index e1df0e341f..64ed5278f9 100644

scripts/qapi/commands.py line 17

Quoted 10 lines
scripts/qapi/commands.py +1 -5
@@ -13,11 +13,7 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import (
-    build_params,
-    c_name,
-    mcgen,
-)
+from .common import build_params, c_name, mcgen
 from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext

Squash this hunk into the previous commit, please.

Quoted 7 lines
scripts/qapi/doc.py +0 -0
index b764a8ccc0..1acb773e0a 100644

scripts/qapi/doc.py line 10

Quoted 7 lines
scripts/qapi/doc.py +1 -1
@@ -5,9 +5,9 @@
 """This script produces the documentation of a qapi schema in texinfo format"""
 
 import re
+
 from .gen import QAPIGenDoc, QAPISchemaVisitor
 
-

Do you delete the blank line to keep isort happy?

More of the same below.

Quoted 27 lines
scripts/qapi/doc.py +0 -0
 _MSG = '''
 @deftypefn {type} {{}} {name}
 
scripts/qapi/expr.py +2 -2
index 03b31ecfc1..3e952a1462 100644
@@ -14,12 +14,12 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-import re
 from collections import OrderedDict
+import re
+
 from .common import c_name
 from .error import QAPISemError
 
-
 # Names must be letters, numbers, -, and _.  They must start with letter,
 # except for downstream extensions which must start with __RFQDN_.
 # Dots are only valid in the downstream extension prefix.
scripts/qapi/introspect.py +0 -0
index b036fcf9ce..2850121cbd 100644

scripts/qapi/introspect.py line 20

Quoted 6 lines
scripts/qapi/introspect.py +1 -2
@@ -17,8 +17,7 @@
     mcgen,
 )
 from .gen import QAPISchemaMonolithicCVisitor
-from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
-                     QAPISchemaType)
+from .schema import QAPISchemaArrayType, QAPISchemaBuiltinType, QAPISchemaType

Line is rather long for my taste.

So far, we didn't even try to have a consistent Python style in QEMU, so I've enforced a consistent style I like within QAPI.

Having a consistent Python style in QEMU would be worth letting go of personal preferences. Do you intend to pursue it?

[...]

author
#31 lore raw

Re: [PATCH v3 12/47] qapi: enforce import order/styling with isort

Quoted 53 lines · Markus Armbruster
Quoted 51 lines · John Snow

While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us.

force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself.

force_grid_wrap=4: Four or more imports from a single module will force the one-per-line style that's more git-friendly. This will generally happen for 'typing' imports.

multi_line_output=3: Uses the one-per-line indented style for long imports.

include_trailing_comma: Adds a comma to the last import in a group, which makes git conflicts nicer to deal with, generally.

---
  scripts/qapi/.isort.cfg    | 5 +++++
  scripts/qapi/commands.py   | 6 +-----
  scripts/qapi/doc.py        | 2 +-
  scripts/qapi/expr.py       | 4 ++--
  scripts/qapi/introspect.py | 3 +--
  scripts/qapi/main.py       | 1 -
  scripts/qapi/parser.py     | 2 +-
  scripts/qapi/schema.py     | 2 +-
  scripts/qapi/types.py      | 1 -
  9 files changed, 12 insertions(+), 14 deletions(-)

create mode 100644 scripts/qapi/.isort.cfg

scripts/qapi/.isort.cfg new file +5 -0
new file mode 100644
index 0000000000..b0aeffec26
@@ -0,0 +1,5 @@
+[settings]
+force_sort_within_sections=True
+force_grid_wrap=4
+multi_line_output=3
+include_trailing_comma=True
scripts/qapi/commands.py +0 -0
index e1df0e341f..64ed5278f9 100644
@@ -13,11 +13,7 @@
  See the COPYING file in the top-level directory.
  """
 
Quoted 10 lines

scripts/qapi/commands.py line 17

Quoted 7 lines
scripts/qapi/commands.py +1 -5
-from .common import (
-    build_params,
-    c_name,
-    mcgen,
-)
+from .common import build_params, c_name, mcgen
  from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext

Squash this hunk into the previous commit, please.

Yes, OK.

Quoted 9 lines
Quoted 9 lines
scripts/qapi/doc.py +0 -0
index b764a8ccc0..1acb773e0a 100644
@@ -5,9 +5,9 @@
  """This script produces the documentation of a qapi schema in texinfo format"""
 
Quoted 8 lines

scripts/qapi/doc.py line 10

Quoted 5 lines
scripts/qapi/doc.py +1 -1
  import re
+
  from .gen import QAPIGenDoc, QAPISchemaVisitor
 
-

Do you delete the blank line to keep isort happy?

isort offers to actually edit your files for you, it isn't just a checker, it actually rewrites.

By default, it leaves one space below imports. You can configure it to add two. I didn't really have a strong preference, so I left it at the default.

I'm going to leave it at the default for now, but I can change it if you have a preference.

Quoted 29 lines

More of the same below.

Quoted 27 lines
scripts/qapi/doc.py +0 -0
  _MSG = '''
  @deftypefn {type} {{}} {name}
 
scripts/qapi/expr.py +2 -2
index 03b31ecfc1..3e952a1462 100644
@@ -14,12 +14,12 @@
  # This work is licensed under the terms of the GNU GPL, version 2.
  # See the COPYING file in the top-level directory.
 
-import re
  from collections import OrderedDict
+import re
+
  from .common import c_name
  from .error import QAPISemError
 
-
  # Names must be letters, numbers, -, and _.  They must start with letter,
  # except for downstream extensions which must start with __RFQDN_.
  # Dots are only valid in the downstream extension prefix.
scripts/qapi/introspect.py +0 -0
index b036fcf9ce..2850121cbd 100644
@@ -17,8 +17,7 @@
Quoted 9 lines

scripts/qapi/introspect.py line 20

Quoted 6 lines
scripts/qapi/introspect.py +1 -2
      mcgen,
  )
  from .gen import QAPISchemaMonolithicCVisitor
-from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
-                     QAPISchemaType)
+from .schema import QAPISchemaArrayType, QAPISchemaBuiltinType, QAPISchemaType

Line is rather long for my taste.

78 chars and three names, so it didn't trigger the grid-wrap condition of isort.

If you have an arbitrarily smaller number you'd like to use, I can feed that to isort's config file. 72?

Quoted 3 lines

So far, we didn't even try to have a consistent Python style in QEMU, so I've enforced a consistent style I like within QAPI.

I am now trying!

Quoted 3 lines

Having a consistent Python style in QEMU would be worth letting go of personal preferences. Do you intend to pursue it?

100%.

--js

#32 lore raw

Re: [PATCH v3 12/47] qapi: enforce import order/styling with isort

Quoted 82 lines · John Snow
Quoted 52 lines · Markus Armbruster
Quoted 50 lines · John Snow

While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us.

force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself.

force_grid_wrap=4: Four or more imports from a single module will force the one-per-line style that's more git-friendly. This will generally happen for 'typing' imports.

multi_line_output=3: Uses the one-per-line indented style for long imports.

include_trailing_comma: Adds a comma to the last import in a group, which makes git conflicts nicer to deal with, generally.

---
  scripts/qapi/.isort.cfg    | 5 +++++
  scripts/qapi/commands.py   | 6 +-----
  scripts/qapi/doc.py        | 2 +-
  scripts/qapi/expr.py       | 4 ++--
  scripts/qapi/introspect.py | 3 +--
  scripts/qapi/main.py       | 1 -
  scripts/qapi/parser.py     | 2 +-
  scripts/qapi/schema.py     | 2 +-
  scripts/qapi/types.py      | 1 -
  9 files changed, 12 insertions(+), 14 deletions(-)

create mode 100644 scripts/qapi/.isort.cfg

scripts/qapi/.isort.cfg new file +5 -0
new file mode 100644
index 0000000000..b0aeffec26
@@ -0,0 +1,5 @@
+[settings]
+force_sort_within_sections=True
+force_grid_wrap=4
+multi_line_output=3
+include_trailing_comma=True
scripts/qapi/commands.py +0 -0
index e1df0e341f..64ed5278f9 100644
@@ -13,11 +13,7 @@
  See the COPYING file in the top-level directory.
  """
Quoted 9 lines

scripts/qapi/commands.py line 17

Quoted 7 lines
scripts/qapi/commands.py +1 -4
  -from .common import (
-    build_params,
-    c_name,
-    mcgen,
-)
+from .common import build_params, c_name, mcgen
  from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext

Squash this hunk into the previous commit, please.

Yes, OK.

Quoted 13 lines

scripts/qapi/doc.py line 9

Quoted 11 lines
scripts/qapi/doc.py +1 -0
index b764a8ccc0..1acb773e0a 100644
@@ -5,9 +5,9 @@
  """This script produces the documentation of a qapi schema in texinfo format"""
    import re
+
  from .gen import QAPIGenDoc, QAPISchemaVisitor
  -

Do you delete the blank line to keep isort happy?

isort offers to actually edit your files for you, it isn't just a checker, it actually rewrites.

Quoted 6 lines

By default, it leaves one space below imports. You can configure it to add two. I didn't really have a strong preference, so I left it at the default.

I'm going to leave it at the default for now, but I can change it if you have a preference.

PEP 8: "Surround top-level function and class definitions with two blank lines."

[...]

author
#33 lore raw

Re: [PATCH v3 12/47] qapi: enforce import order/styling with isort

Quoted 6 lines · Markus Armbruster

PEP 8: "Surround top-level function and class definitions with two blank lines."

[...]

Yep, but flake8 does not complain about the first definitions that occur below imports. Why not? I don't know.

Regardless, I can change it and fold the changes in; they won't affect much here.

--js

#34 lore raw

Re: [PATCH v3 12/47] qapi: enforce import order/styling with isort

Quoted 8 lines · John Snow
Quoted 6 lines · Markus Armbruster

PEP 8: "Surround top-level function and class definitions with two blank lines."

[...]

Quoted 6 lines

Yep, but flake8 does not complain about the first definitions that occur below imports. Why not? I don't know.

Regardless, I can change it and fold the changes in; they won't affect much here.

Well, on all of the cases here, it's not function or class definition that follows, so that aspect of PEP 8 is not being violated.

- Cleber.

Reviewed-by Tested-by on v3 12/47
#35 lore raw

Re: [PATCH v3 12/47] qapi: enforce import order/styling with isort

In reply to John Snow
Quoted 11 lines · John Snow

While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us.

force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself.

force_grid_wrap=4: Four or more imports from a single module will force the one-per-line style that's more git-friendly. This will generally happen for 'typing' imports.

Quoted 8 lines

multi_line_output=3: Uses the one-per-line indented style for long imports.

include_trailing_comma: Adds a comma to the last import in a group, which makes git conflicts nicer to deal with, generally.

author
#36 lore raw

PATCH v3 13/47 qapi: delint using flake8

In reply to John Snow

Petty style guide fixes and line length enforcement. Not a big win, not a big loss, but flake8 passes 100% on the qapi module, which gives us an easy baseline to enforce hereafter.

A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception.

---
 scripts/qapi/.flake8     |  2 ++
 scripts/qapi/commands.py |  3 ++-
 scripts/qapi/schema.py   |  8 +++++---
 scripts/qapi/visit.py    | 16 +++++++++++-----
 4 files changed, 20 insertions(+), 9 deletions(-)
 create mode 100644 scripts/qapi/.flake8
scripts/qapi/.flake8 new file +2 -0
new file mode 100644
index 0000000000..6b158c68b8
@@ -0,0 +1,2 @@
+[flake8]
+extend-ignore = E722  # Prefer pylint's bare-except checks to flake8's
scripts/qapi/commands.py +2 -1
index 64ed5278f9..5dc2f5a9fa 100644
@@ -65,7 +65,8 @@ def gen_call(name, arg_type, boxed, ret_type):
 def gen_marshal_output(ret_type):
     return mcgen('''
 
-static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject **ret_out, Error **errp)
+static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in,
+                                QObject **ret_out, Error **errp)
 {
     Visitor *v;
 
scripts/qapi/schema.py +5 -3
index 093f7a38d8..cfc52e1ae4 100644
@@ -536,7 +536,7 @@ def set_defined_in(self, name):
             v.set_defined_in(name)
 
     def check(self, schema, seen):
-        if not self.tag_member: # flat union
+        if not self.tag_member:  # flat union
             self.tag_member = seen.get(c_name(self._tag_name))
             base = "'base'"
             # Pointing to the base type when not implicit would be
@@ -821,7 +821,7 @@ def __init__(self, fname):
         self._entity_dict = {}
         self._module_dict = OrderedDict()
         self._schema_dir = os.path.dirname(fname)
-        self._make_module(None) # built-ins
+        self._make_module(None)  # built-ins
         self._make_module(fname)
         self._predefining = True
         self._def_predefineds()
@@ -965,7 +965,9 @@ def _make_implicit_object_type(self, name, info, ifcond, role, members):
             # But it's not tight: the disjunction need not imply it.  We
             # may end up compiling useless wrapper types.
             # TODO kill simple unions or implement the disjunction
-            assert (ifcond or []) == typ._ifcond # pylint: disable=protected-access
+
+            # pylint: disable=protected-access
+            assert (ifcond or []) == typ._ifcond
         else:
             self._def_entity(QAPISchemaObjectType(
                 name, info, None, ifcond, None, None, members, None))
scripts/qapi/visit.py +11 -5
index ea277e7704..808410d6f1 100644
@@ -31,7 +31,9 @@ def gen_visit_decl(name, scalar=False):
     if not scalar:
         c_type += '*'
     return mcgen('''
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_type)sobj, Error **errp);
+
+bool visit_type_%(c_name)s(Visitor *v, const char *name,
+                 %(c_type)sobj, Error **errp);
 ''',
                  c_name=c_name(name), c_type=c_type)
 
@@ -125,7 +127,8 @@ def gen_visit_object_members(name, base, members, variants):
 def gen_visit_list(name, element_type):
     return mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
+bool visit_type_%(c_name)s(Visitor *v, const char *name,
+                 %(c_name)s **obj, Error **errp)
 {
     bool ok = false;
     %(c_name)s *tail;
@@ -158,7 +161,8 @@ def gen_visit_list(name, element_type):
 def gen_visit_enum(name):
     return mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s *obj, Error **errp)
+bool visit_type_%(c_name)s(Visitor *v, const char *name,
+                 %(c_name)s *obj, Error **errp)
 {
     int value = *obj;
     bool ok = visit_type_enum(v, name, &value, &%(c_name)s_lookup, errp);
@@ -172,7 +176,8 @@ def gen_visit_enum(name):
 def gen_visit_alternate(name, variants):
     ret = mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
+bool visit_type_%(c_name)s
+    (Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
 {
     bool ok = false;
 
@@ -247,7 +252,8 @@ def gen_visit_alternate(name, variants):
 def gen_visit_object(name, base, members, variants):
     return mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
+bool visit_type_%(c_name)s
+    (Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
 {
     bool ok = false;
 
Signature · 1 line
2.26.2
author
#37 lore raw

PATCH v3 14/47 qapi: add pylintrc

In reply to John Snow

Using pylint --generate-rcfile > pylintrc, generate a skeleton pylintrc file. Sections that are not presently relevant (by the end of this series) are removed leaving just the empty section as a search engine / documentation hint to future authors.

I am targeting pylint 2.6.0. In the future (and hopefully before 5.2 is released), I aim to have gitlab CI running the specific targeted versions of pylint, mypy, flake8, etc in a job.

2.5.x will work if you additionally pass --disable=bad-whitespace. This warning was removed from 2.6.x, for lack of consistent support.

Right now, quite a few modules are ignored as they are known to fail as of this commit. modules will be removed from the known-bad list throughout this and following series as they are repaired.

Note: Normally, pylintrc would go in the folder above the module, but as that folder is shared by many things, it is going inside the module folder (for now). Due to a bug in pylint 2.5+, pylint does not correctly recognize when it is being run from "inside" a package, and must be run *outside* of the package.

Therefore, to run it, you must:

Quoted 1 line

pylint scripts/qapi/ --rcfile=scripts/qapi/pylintrc

---
 scripts/qapi/pylintrc | 74 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)
 create mode 100644 scripts/qapi/pylintrc
scripts/qapi/pylintrc new file +74 -0
new file mode 100644
index 0000000000..02dd22c863
@@ -0,0 +1,74 @@
+[MASTER]
+
+# Add files or directories matching the regex patterns to the ignore list.
+# The regex matches against base names, not paths.
+ignore-patterns=common.py,
+                doc.py,
+                error.py,
+                expr.py,
+                gen.py,
+                parser.py,
+                schema.py,
+                source.py,
+                types.py,
+                visit.py,
+
+
+[MESSAGES CONTROL]
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once). You can also use "--disable=all" to
+# disable everything first and then reenable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use "--disable=all --enable=classes
+# --disable=W".
+disable=fixme,
+        missing-docstring,
+        too-many-arguments,
+        too-many-branches,
+        too-many-statements,
+        too-many-instance-attributes,
+
+[REPORTS]
+
+[REFACTORING]
+
+[MISCELLANEOUS]
+
+[LOGGING]
+
+[BASIC]
+
+# Good variable names which should always be accepted, separated by a comma.
+good-names=i,
+           j,
+           k,
+           ex,
+           Run,
+           _
+
+[VARIABLES]
+
+[STRING]
+
+[SPELLING]
+
+[FORMAT]
+
+[SIMILARITIES]
+
+# Ignore import statements themselves when computing similarities.
+ignore-imports=yes
+
+[TYPECHECK]
+
+[CLASSES]
+
+[IMPORTS]
+
+[DESIGN]
+
+[EXCEPTIONS]
Signature · 1 line
2.26.2
author
#38 lore raw

PATCH v3 15/47 qapi/common.py: Remove python compatibility workaround

In reply to John Snow
---
 scripts/qapi/common.py | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
scripts/qapi/common.py +1 -4
index ba35abea47..cee63eb95c 100644
@@ -119,10 +119,7 @@ def cgen(code, **kwds):
     raw = code % kwds
     if indent_level:
         indent = genindent(indent_level)
-        # re.subn() lacks flags support before Python 2.7, use re.compile()
-        raw = re.subn(re.compile(r'^(?!(#|$))', re.MULTILINE),
-                      indent, raw)
-        raw = raw[0]
+        raw = re.sub(r'^(?!(#|$))', indent, raw, flags=re.MULTILINE)
     return re.sub(re.escape(eatspace) + r' *', '', raw)
 
 
Signature · 1 line
2.26.2
author
#39 lore raw

PATCH v3 16/47 qapi/common.py: Add indent manager

In reply to John Snow

Code style tools really dislike the use of global keywords, because it generally involves re-binding the name at runtime which can have strange effects depending on when and how that global name is referenced in other modules.

Make a little indent level manager instead.

---
 scripts/qapi/common.py | 49 ++++++++++++++++++++++++++++--------------
 scripts/qapi/visit.py  |  7 +++---
 2 files changed, 36 insertions(+), 20 deletions(-)
scripts/qapi/common.py +33 -16
index cee63eb95c..b35318b72c 100644
@@ -93,33 +93,50 @@ def c_name(name, protect=True):
 pointer_suffix = ' *' + eatspace
 
 
-def genindent(count):
-    ret = ''
-    for _ in range(count):
-        ret += ' '
-    return ret
+class Indentation:
+    """
+    Indentation level management.
 
+    :param initial: Initial number of spaces, default 0.
+    """
+    def __init__(self, initial: int = 0) -> None:
+        self._level = initial
 
-indent_level = 0
+    def __int__(self) -> int:
+        return self._level
 
+    def __repr__(self) -> str:
+        return "{}({:d})".format(type(self).__name__, self._level)
 
-def push_indent(indent_amount=4):
-    global indent_level
-    indent_level += indent_amount
+    def __str__(self) -> str:
+        """Return the current indentation as a string of spaces."""
+        return ' ' * self._level
 
+    def __bool__(self) -> bool:
+        """True when there is a non-zero indentation."""
+        return bool(self._level)
 
-def pop_indent(indent_amount=4):
-    global indent_level
-    indent_level -= indent_amount
+    def increase(self, amount: int = 4) -> None:
+        """Increase the indentation level by ``amount``, default 4."""
+        self._level += amount
+
+    def decrease(self, amount: int = 4) -> None:
+        """Decrease the indentation level by ``amount``, default 4."""
+        if self._level < amount:
+            raise ArithmeticError(
+                f"Can't remove {amount:d} spaces from {self!r}")
+        self._level -= amount
+
+
+indent = Indentation()
 
 
 # Generate @code with @kwds interpolated.
-# Obey indent_level, and strip eatspace.
+# Obey indent, and strip eatspace.
 def cgen(code, **kwds):
     raw = code % kwds
-    if indent_level:
-        indent = genindent(indent_level)
-        raw = re.sub(r'^(?!(#|$))', indent, raw, flags=re.MULTILINE)
+    if indent:
+        raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
     return re.sub(re.escape(eatspace) + r' *', '', raw)
 
 
scripts/qapi/visit.py +3 -4
index 808410d6f1..14f30c228b 100644
@@ -18,9 +18,8 @@
     c_name,
     gen_endif,
     gen_if,
+    indent,
     mcgen,
-    pop_indent,
-    push_indent,
 )
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaObjectType
@@ -69,7 +68,7 @@ def gen_visit_object_members(name, base, members, variants):
     if (visit_optional(v, "%(name)s", &obj->has_%(c_name)s)) {
 ''',
                          name=memb.name, c_name=c_name(memb.name))
-            push_indent()
+            indent.increase()
         ret += mcgen('''
     if (!visit_type_%(c_type)s(v, "%(name)s", &obj->%(c_name)s, errp)) {
         return false;
@@ -78,7 +77,7 @@ def gen_visit_object_members(name, base, members, variants):
                      c_type=memb.type.c_name(), name=memb.name,
                      c_name=c_name(memb.name))
         if memb.optional:
-            pop_indent()
+            indent.decrease()
             ret += mcgen('''
     }
 ''')
Signature · 1 line
2.26.2
author
#40 lore raw

PATCH v3 17/47 qapi/common.py: delint with pylint

In reply to John Snow

At this point, that just means using a consistent strategy for constant names. constants get UPPER_CASE and names not used externally get a leading underscore.

As a preference, while renaming constants to be UPPERCASE, move them to the head of the file. Generally, it's nice to be able to audit the code that runs on import in one central place.

---
 scripts/qapi/common.py | 17 +++++++----------
 scripts/qapi/schema.py | 14 +++++++-------
 2 files changed, 14 insertions(+), 17 deletions(-)
scripts/qapi/common.py +7 -10
index b35318b72c..c096df0d2d 100644
@@ -13,6 +13,10 @@
 
 import re
 
+EATSPACE = '\033EATSPACE.'
+POINTER_SUFFIX = ' *' + EATSPACE
+_C_NAME_TRANS = str.maketrans('.-', '__')
+
 
 # ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
 # ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
@@ -42,9 +46,6 @@ def c_enum_const(type_name, const_name, prefix=None):
     return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
 
 
-c_name_trans = str.maketrans('.-', '__')
-
-
 # Map @name to a valid C identifier.
 # If @protect, avoid returning certain ticklish identifiers (like
 # C keywords) by prepending 'q_'.
@@ -82,17 +83,13 @@ def c_name(name, protect=True):
                      'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'])
     # namespace pollution:
     polluted_words = set(['unix', 'errno', 'mips', 'sparc', 'i386'])
-    name = name.translate(c_name_trans)
+    name = name.translate(_C_NAME_TRANS)
     if protect and (name in c89_words | c99_words | c11_words | gcc_words
                     | cpp_words | polluted_words):
         return 'q_' + name
     return name
 
 
-eatspace = '\033EATSPACE.'
-pointer_suffix = ' *' + eatspace
-
-
 class Indentation:
     """
     Indentation level management.
@@ -132,12 +129,12 @@ def decrease(self, amount: int = 4) -> None:
 
 
 # Generate @code with @kwds interpolated.
-# Obey indent, and strip eatspace.
+# Obey indent, and strip EATSPACE.
 def cgen(code, **kwds):
     raw = code % kwds
     if indent:
         raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
-    return re.sub(re.escape(eatspace) + r' *', '', raw)
+    return re.sub(re.escape(EATSPACE) + r' *', '', raw)
 
 
 def mcgen(code, **kwds):
scripts/qapi/schema.py +7 -7
index cfc52e1ae4..74c6b96d39 100644
@@ -18,7 +18,7 @@
 import os
 import re
 
-from .common import c_name, pointer_suffix
+from .common import POINTER_SUFFIX, c_name
 from .error import QAPIError, QAPISemError
 from .expr import check_exprs
 from .parser import QAPISchemaParser
@@ -309,7 +309,7 @@ def is_implicit(self):
         return True
 
     def c_type(self):
-        return c_name(self.name) + pointer_suffix
+        return c_name(self.name) + POINTER_SUFFIX
 
     def json_type(self):
         return 'array'
@@ -430,7 +430,7 @@ def c_name(self):
 
     def c_type(self):
         assert not self.is_implicit()
-        return c_name(self.name) + pointer_suffix
+        return c_name(self.name) + POINTER_SUFFIX
 
     def c_unboxed_type(self):
         return c_name(self.name)
@@ -504,7 +504,7 @@ def connect_doc(self, doc=None):
             v.connect_doc(doc)
 
     def c_type(self):
-        return c_name(self.name) + pointer_suffix
+        return c_name(self.name) + POINTER_SUFFIX
 
     def json_type(self):
         return 'value'
@@ -896,7 +896,7 @@ def _def_builtin_type(self, name, json_type, c_type):
         self._make_array_type(name, None)
 
     def _def_predefineds(self):
-        for t in [('str',    'string',  'char' + pointer_suffix),
+        for t in [('str',    'string',  'char' + POINTER_SUFFIX),
                   ('number', 'number',  'double'),
                   ('int',    'int',     'int64_t'),
                   ('int8',   'int',     'int8_t'),
@@ -909,8 +909,8 @@ def _def_predefineds(self):
                   ('uint64', 'int',     'uint64_t'),
                   ('size',   'int',     'uint64_t'),
                   ('bool',   'boolean', 'bool'),
-                  ('any',    'value',   'QObject' + pointer_suffix),
-                  ('null',   'null',    'QNull' + pointer_suffix)]:
+                  ('any',    'value',   'QObject' + POINTER_SUFFIX),
+                  ('null',   'null',    'QNull' + POINTER_SUFFIX)]:
             self._def_builtin_type(*t)
         self.the_empty_object_type = QAPISchemaObjectType(
             'q_empty', None, None, None, None, None, [], None)
Signature · 1 line
2.26.2
author
#41 lore raw

PATCH v3 18/47 qapi/common.py: Replace one-letter 'c' variable

In reply to John Snow
---
 scripts/qapi/common.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
scripts/qapi/common.py +4 -4
index c096df0d2d..9e50070f62 100644
@@ -29,14 +29,14 @@ def camel_to_upper(value):
     new_name = ''
     length = len(c_fun_str)
     for i in range(length):
-        c = c_fun_str[i]
-        # When c is upper and no '_' appears before, do more checks
-        if c.isupper() and (i > 0) and c_fun_str[i - 1] != '_':
+        char = c_fun_str[i]
+        # When char is upper case and no '_' appears before, do more checks
+        if char.isupper() and (i > 0) and c_fun_str[i - 1] != '_':
             if i < length - 1 and c_fun_str[i + 1].islower():
                 new_name += '_'
             elif c_fun_str[i - 1].isdigit():
                 new_name += '_'
-        new_name += c
+        new_name += char
     return new_name.lstrip('_').upper()
 
 
Signature · 1 line
2.26.2
author
#42 lore raw

PATCH v3 19/47 qapi/common.py: check with pylint

In reply to John Snow

Remove qapi/common.py from the pylintrc ignore list.

---
 scripts/qapi/pylintrc | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
scripts/qapi/pylintrc +1 -2
index 02dd22c863..6151427a51 100644
@@ -2,8 +2,7 @@
 
 # Add files or directories matching the regex patterns to the ignore list.
 # The regex matches against base names, not paths.
-ignore-patterns=common.py,
-                doc.py,
+ignore-patterns=doc.py,
                 error.py,
                 expr.py,
                 gen.py,
Signature · 1 line
2.26.2
author
#43 lore raw

PATCH v3 20/47 qapi/common.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior. This commit *only* adds annotations.

---
 scripts/qapi/common.py | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
scripts/qapi/common.py +16 -11
index 9e50070f62..d1d39fdb4c 100644
@@ -12,6 +12,7 @@
 # See the COPYING file in the top-level directory.
 
 import re
+from typing import Optional, Sequence
 
 EATSPACE = '\033EATSPACE.'
 POINTER_SUFFIX = ' *' + EATSPACE
@@ -21,7 +22,7 @@
 # ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
 # ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
 # ENUM24_Name -> ENUM24_NAME
-def camel_to_upper(value):
+def camel_to_upper(value: str) -> str:
     c_fun_str = c_name(value, False)
     if value.isupper():
         return c_fun_str
@@ -40,7 +41,9 @@ def camel_to_upper(value):
     return new_name.lstrip('_').upper()
 
 
-def c_enum_const(type_name, const_name, prefix=None):
+def c_enum_const(type_name: str,
+                 const_name: str,
+                 prefix: Optional[str] = None) -> str:
     if prefix is not None:
         type_name = prefix
     return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
@@ -55,7 +58,7 @@ def c_enum_const(type_name, const_name, prefix=None):
 # into substrings of a generated C function name.
 # '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
 # protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
-def c_name(name, protect=True):
+def c_name(name: str, protect: bool = True) -> str:
     # ANSI X3J11/88-090, 3.1.1
     c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
                      'default', 'do', 'double', 'else', 'enum', 'extern',
@@ -130,24 +133,24 @@ def decrease(self, amount: int = 4) -> None:
 
 # Generate @code with @kwds interpolated.
 # Obey indent, and strip EATSPACE.
-def cgen(code, **kwds):
+def cgen(code: str, **kwds: object) -> str:
     raw = code % kwds
     if indent:
         raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
     return re.sub(re.escape(EATSPACE) + r' *', '', raw)
 
 
-def mcgen(code, **kwds):
+def mcgen(code: str, **kwds: object) -> str:
     if code[0] == '\n':
         code = code[1:]
     return cgen(code, **kwds)
 
 
-def c_fname(filename):
+def c_fname(filename: str) -> str:
     return re.sub(r'[^A-Za-z0-9_]', '_', filename)
 
 
-def guardstart(name):
+def guardstart(name: str) -> str:
     return mcgen('''
 #ifndef %(name)s
 #define %(name)s
@@ -156,7 +159,7 @@ def guardstart(name):
                  name=c_fname(name).upper())
 
 
-def guardend(name):
+def guardend(name: str) -> str:
     return mcgen('''
 
 #endif /* %(name)s */
@@ -164,7 +167,7 @@ def guardend(name):
                  name=c_fname(name).upper())
 
 
-def gen_if(ifcond):
+def gen_if(ifcond: Sequence[str]) -> str:
     ret = ''
     for ifc in ifcond:
         ret += mcgen('''
@@ -173,7 +176,7 @@ def gen_if(ifcond):
     return ret
 
 
-def gen_endif(ifcond):
+def gen_endif(ifcond: Sequence[str]) -> str:
     ret = ''
     for ifc in reversed(ifcond):
         ret += mcgen('''
@@ -182,7 +185,9 @@ def gen_endif(ifcond):
     return ret
 
 
-def build_params(arg_type, boxed, extra=None):
+def build_params(arg_type,
+                 boxed: bool,
+                 extra: Optional[str] = None) -> str:
     ret = ''
     sep = ''
     if boxed:
Signature · 1 line
2.26.2
author
#44 lore raw

PATCH v3 21/47 qapi/common.py: Convert comments into docstrings, and elaborate

In reply to John Snow

As docstrings, they'll show up in documentation and IDE help.

The docstring style being targeted is the Sphinx documentation style. Sphinx uses an extension of ReST with "domains". We use the (implicit) Python domain, which supports a number of custom "info fields". Those info fields are documented here:
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#info-field-lists

Primarily, we use :param X: descr, :return[s]: descr, and `:raise[s]
Z: when`. Everything else is the Sphinx dialect of ReST.

(No, nothing checks or enforces this style that I am aware of. Sphinx either chokes or succeeds, but does not enforce a standard of what is otherwise inside the docstring. Pycharm does highlight when your param fields are not aligned with the actual fields present. It does not highlight missing return or exception statements. There is no existing style guide I am aware of that covers a standard for a minimally acceptable docstring. I am debating writing one.)

---
 scripts/qapi/common.py | 53 +++++++++++++++++++++++++++++++-----------
 1 file changed, 39 insertions(+), 14 deletions(-)
scripts/qapi/common.py +39 -14
index d1d39fdb4c..82266fe56a 100644
@@ -14,15 +14,24 @@
 import re
 from typing import Optional, Sequence
 
+#: Sentinel value that causes all space to its right to be removed.
 EATSPACE = '\033EATSPACE.'
 POINTER_SUFFIX = ' *' + EATSPACE
 _C_NAME_TRANS = str.maketrans('.-', '__')
 
 
-# ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
-# ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
-# ENUM24_Name -> ENUM24_NAME
 def camel_to_upper(value: str) -> str:
+    """
+    Converts CamelCase to CAMEL_CASE.
+
+    Examples:
+      ENUMName -> ENUM_NAME
+      EnumName1 -> ENUM_NAME1
+      ENUM_NAME -> ENUM_NAME
+      ENUM_NAME1 -> ENUM_NAME1
+      ENUM_Name2 -> ENUM_NAME2
+      ENUM24_Name -> ENUM24_NAME
+    """
     c_fun_str = c_name(value, False)
     if value.isupper():
         return c_fun_str
@@ -44,21 +53,33 @@ def camel_to_upper(value: str) -> str:
 def c_enum_const(type_name: str,
                  const_name: str,
                  prefix: Optional[str] = None) -> str:
+    """
+    Generate a C enumeration constant name.
+
+    :param type_name: The name of the enumeration.
+    :param const_name: The name of this constant.
+    :param prefix: Optional, prefix that overrides the type_name.
+    """
     if prefix is not None:
         type_name = prefix
     return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
 
 
-# Map @name to a valid C identifier.
-# If @protect, avoid returning certain ticklish identifiers (like
-# C keywords) by prepending 'q_'.
-#
-# Used for converting 'name' from a 'name':'type' qapi definition
-# into a generated struct member, as well as converting type names
-# into substrings of a generated C function name.
-# '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
-# protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
 def c_name(name: str, protect: bool = True) -> str:
+    """
+    Map ``name`` to a valid C identifier.
+
+    Used for converting 'name' from a 'name':'type' qapi definition
+    into a generated struct member, as well as converting type names
+    into substrings of a generated C function name.
+
+    '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
+    protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
+
+    :param name: The name to map.
+    :param protect: If true, avoid returning certain ticklish identifiers
+                    (like C keywords) by prepending ``q_``.
+    """
     # ANSI X3J11/88-090, 3.1.1
     c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
                      'default', 'do', 'double', 'else', 'enum', 'extern',
@@ -128,12 +149,16 @@ def decrease(self, amount: int = 4) -> None:
         self._level -= amount
 
 
+#: Global, current indent level for code generation.
 indent = Indentation()
 
 
-# Generate @code with @kwds interpolated.
-# Obey indent, and strip EATSPACE.
 def cgen(code: str, **kwds: object) -> str:
+    """
+    Generate ``code`` with ``kwds`` interpolated.
+
+    Obey `indent`, and strip `EATSPACE`.
+    """
     raw = code % kwds
     if indent:
         raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
Signature · 1 line
2.26.2
author
#45 lore raw

PATCH v3 22/47 qapi/common.py: move build_params into gen.py

In reply to John Snow

Including it in common.py creates a circular import dependency; schema relies on common, but common.build_params requires a type annotation from schema. To type this properly, it needs to be moved outside the cycle.

---
 scripts/qapi/commands.py |  9 +++++++--
 scripts/qapi/common.py   | 23 -----------------------
 scripts/qapi/events.py   |  9 ++-------
 scripts/qapi/gen.py      | 31 +++++++++++++++++++++++++++++--
 4 files changed, 38 insertions(+), 34 deletions(-)
scripts/qapi/commands.py +7 -2
index 5dc2f5a9fa..f67393f871 100644
@@ -13,8 +13,13 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import build_params, c_name, mcgen
-from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
+from .common import c_name, mcgen
+from .gen import (
+    QAPIGenCCode,
+    QAPISchemaModularCVisitor,
+    build_params,
+    ifcontext,
+)
 
 
 def gen_command_decl(name, arg_type, boxed, ret_type):
scripts/qapi/common.py +0 -23
index 82266fe56a..430466be49 100644
@@ -208,26 +208,3 @@ def gen_endif(ifcond: Sequence[str]) -> str:
 #endif /* %(cond)s */
 ''', cond=ifc)
     return ret
-
-
-def build_params(arg_type,
-                 boxed: bool,
-                 extra: Optional[str] = None) -> str:
-    ret = ''
-    sep = ''
-    if boxed:
-        assert arg_type
-        ret += '%s arg' % arg_type.c_param_type()
-        sep = ', '
-    elif arg_type:
-        assert not arg_type.variants
-        for memb in arg_type.members:
-            ret += sep
-            sep = ', '
-            if memb.optional:
-                ret += 'bool has_%s, ' % c_name(memb.name)
-            ret += '%s %s' % (memb.type.c_param_type(),
-                              c_name(memb.name))
-    if extra:
-        ret += sep + extra
-    return ret if ret else 'void'
scripts/qapi/events.py +2 -7
index 6b3afa14d7..f840a62ed9 100644
@@ -12,13 +12,8 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import (
-    build_params,
-    c_enum_const,
-    c_name,
-    mcgen,
-)
-from .gen import QAPISchemaModularCVisitor, ifcontext
+from .common import c_enum_const, c_name, mcgen
+from .gen import QAPISchemaModularCVisitor, build_params, ifcontext
 from .schema import QAPISchemaEnumMember
 from .types import gen_enum, gen_enum_lookup
 
scripts/qapi/gen.py +29 -2
index d37339fe8f..c1e65f2e52 100644
@@ -2,9 +2,11 @@
 #
 # QAPI code generation
 #
-# Copyright (c) 2018-2019 Red Hat Inc.
+# Copyright IBM, Corp. 2011
+# Copyright (c) 2013-2019 Red Hat Inc.
 #
 # Authors:
+#  Anthony Liguori <[email protected]>
 #  Markus Armbruster <[email protected]>
 #  Marc-André Lureau <[email protected]>
 #
@@ -15,16 +17,18 @@
 import errno
 import os
 import re
+from typing import Optional
 
 from .common import (
     c_fname,
+    c_name,
     gen_endif,
     gen_if,
     guardend,
     guardstart,
     mcgen,
 )
-from .schema import QAPISchemaVisitor
+from .schema import QAPISchemaObjectType, QAPISchemaVisitor
 
 
 class QAPIGen:
@@ -90,6 +94,29 @@ def _wrap_ifcond(ifcond, before, after):
     return out
 
 
+def build_params(arg_type: Optional[QAPISchemaObjectType],
+                 boxed: bool,
+                 extra: Optional[str] = None) -> str:
+    ret = ''
+    sep = ''
+    if boxed:
+        assert arg_type
+        ret += '%s arg' % arg_type.c_param_type()
+        sep = ', '
+    elif arg_type:
+        assert not arg_type.variants
+        for memb in arg_type.members:
+            ret += sep
+            sep = ', '
+            if memb.optional:
+                ret += 'bool has_%s, ' % c_name(memb.name)
+            ret += '%s %s' % (memb.type.c_param_type(),
+                              c_name(memb.name))
+    if extra:
+        ret += sep + extra
+    return ret if ret else 'void'
+
+
 class QAPIGenCCode(QAPIGen):
 
     def __init__(self, fname):
Signature · 1 line
2.26.2
author
#46 lore raw

PATCH v3 23/47 qapi: establish mypy type-checking baseline

In reply to John Snow

Fix two very minor issues, and then establish a mypy type-checking baseline.

Like pylint, this should be run from the folder above:

Quoted 1 line

mypy --config-file=qapi/mypy.ini qapi/

This is designed and tested for mypy 0.770 or greater.

---
 scripts/qapi/doc.py    |  3 +-
 scripts/qapi/mypy.ini  | 65 ++++++++++++++++++++++++++++++++++++++++++
 scripts/qapi/schema.py |  3 +-
 3 files changed, 69 insertions(+), 2 deletions(-)
 create mode 100644 scripts/qapi/mypy.ini
scripts/qapi/doc.py +2 -1
index 1acb773e0a..42ef19ddf6 100644
@@ -6,7 +6,8 @@
 
 import re
 
-from .gen import QAPIGenDoc, QAPISchemaVisitor
+from .gen import QAPIGenDoc
+from .schema import QAPISchemaVisitor
 
 _MSG = '''
 @deftypefn {type} {{}} {name}
scripts/qapi/mypy.ini new file +65 -0

65 lines not on this page · raw message

scripts/qapi/schema.py +2 -1
index 74c6b96d39..483b4b68df 100644
@@ -17,6 +17,7 @@
 from collections import OrderedDict
 import os
 import re
+from typing import Optional
 
 from .common import POINTER_SUFFIX, c_name
 from .error import QAPIError, QAPISemError
@@ -25,7 +26,7 @@
 
 
 class QAPISchemaEntity:
-    meta = None
+    meta: Optional[str] = None
 
     def __init__(self, name, info, doc, ifcond=None, features=None):
         assert name is None or isinstance(name, str)
Signature · 1 line
2.26.2
Reviewed-by Tested-by on v3 23/47
#47 lore raw

Re: [PATCH v3 23/47] qapi: establish mypy type-checking baseline

Quoted 11 lines · John Snow

Fix two very minor issues, and then establish a mypy type-checking baseline.

Like pylint, this should be run from the folder above:

Quoted 1 line

mypy --config-file=qapi/mypy.ini qapi/

This is designed and tested for mypy 0.770 or greater.

author
#48 lore raw

PATCH v3 24/47 qapi/events.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior. This commit *only* adds annotations.

---
 scripts/qapi/events.py | 46 ++++++++++++++++++++++++++++++++----------
 scripts/qapi/mypy.ini  |  5 -----
 2 files changed, 35 insertions(+), 16 deletions(-)
scripts/qapi/events.py +35 -11
index f840a62ed9..57e0939e96 100644
@@ -12,19 +12,31 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import List
+
 from .common import c_enum_const, c_name, mcgen
 from .gen import QAPISchemaModularCVisitor, build_params, ifcontext
-from .schema import QAPISchemaEnumMember
+from .schema import (
+    QAPISchema,
+    QAPISchemaEnumMember,
+    QAPISchemaFeature,
+    QAPISchemaObjectType,
+)
+from .source import QAPISourceInfo
 from .types import gen_enum, gen_enum_lookup
 
 
-def build_event_send_proto(name, arg_type, boxed):
+def build_event_send_proto(name: str,
+                           arg_type: QAPISchemaObjectType,
+                           boxed: bool) -> str:
     return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
         'c_name': c_name(name.lower()),
         'param': build_params(arg_type, boxed)}
 
 
-def gen_event_send_decl(name, arg_type, boxed):
+def gen_event_send_decl(name: str,
+                        arg_type: QAPISchemaObjectType,
+                        boxed: bool) -> str:
     return mcgen('''
 
 %(proto)s;
@@ -33,7 +45,7 @@ def gen_event_send_decl(name, arg_type, boxed):
 
 
 # Declare and initialize an object 'qapi' using parameters from build_params()
-def gen_param_var(typ):
+def gen_param_var(typ: QAPISchemaObjectType) -> str:
     assert not typ.variants
     ret = mcgen('''
     %(c_name)s param = {
@@ -61,7 +73,11 @@ def gen_param_var(typ):
     return ret
 
 
-def gen_event_send(name, arg_type, boxed, event_enum_name, event_emit):
+def gen_event_send(name: str,
+                   arg_type: QAPISchemaObjectType,
+                   boxed: bool,
+                   event_enum_name: str,
+                   event_emit: str) -> str:
     # FIXME: Our declaration of local variables (and of 'errp' in the
     # parameter list) can collide with exploded members of the event's
     # data type passed in as parameters.  If this collision ever hits in
@@ -137,15 +153,15 @@ def gen_event_send(name, arg_type, boxed, event_enum_name, event_emit):
 
 class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
 
-    def __init__(self, prefix):
+    def __init__(self, prefix: str):
         super().__init__(
             prefix, 'qapi-events',
             ' * Schema-defined QAPI/QMP events', None, __doc__)
         self._event_enum_name = c_name(prefix + 'QAPIEvent', protect=False)
-        self._event_enum_members = []
+        self._event_enum_members: List[QAPISchemaEnumMember] = []
         self._event_emit_name = c_name(prefix + 'qapi_event_emit')
 
-    def _begin_user_module(self, name):
+    def _begin_user_module(self, name: str) -> None:
         events = self._module_basename('qapi-events', name)
         types = self._module_basename('qapi-types', name)
         visit = self._module_basename('qapi-visit', name)
@@ -168,7 +184,7 @@ def _begin_user_module(self, name):
 ''',
                              types=types))
 
-    def visit_end(self):
+    def visit_end(self) -> None:
         self._add_system_module('emit', ' * QAPI Events emission')
         self._genc.preamble_add(mcgen('''
 #include "qemu/osdep.h"
@@ -189,7 +205,13 @@ def visit_end(self):
                              event_emit=self._event_emit_name,
                              event_enum=self._event_enum_name))
 
-    def visit_event(self, name, info, ifcond, features, arg_type, boxed):
+    def visit_event(self,
+                    name: str,
+                    info: QAPISourceInfo,
+                    ifcond: List[str],
+                    features: List[QAPISchemaFeature],
+                    arg_type: QAPISchemaObjectType,
+                    boxed: bool) -> None:
         with ifcontext(ifcond, self._genh, self._genc):
             self._genh.add(gen_event_send_decl(name, arg_type, boxed))
             self._genc.add(gen_event_send(name, arg_type, boxed,
@@ -200,7 +222,9 @@ def visit_event(self, name, info, ifcond, features, arg_type, boxed):
         self._event_enum_members.append(QAPISchemaEnumMember(name, None))
 
 
-def gen_events(schema, output_dir, prefix):
+def gen_events(schema: QAPISchema,
+               output_dir: str,
+               prefix: str) -> None:
     vis = QAPISchemaGenEventVisitor(prefix)
     schema.visit(vis)
     vis.write(output_dir)
scripts/qapi/mypy.ini +0 -5
index a0f2365a53..b668776d94 100644
@@ -19,11 +19,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.events]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.expr]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
Signature · 1 line
2.26.2
author
#49 lore raw

PATCH v3 25/47 qapi/events.py: Move comments into docstrings

In reply to John Snow
---
 scripts/qapi/events.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
scripts/qapi/events.py +3 -1
index 57e0939e96..9f868589bb 100644
@@ -44,8 +44,10 @@ def gen_event_send_decl(name: str,
                  proto=build_event_send_proto(name, arg_type, boxed))
 
 
-# Declare and initialize an object 'qapi' using parameters from build_params()
 def gen_param_var(typ: QAPISchemaObjectType) -> str:
+    """
+    Declare and initialize a qapi object, using parameters from `build_params`.
+    """
     assert not typ.variants
     ret = mcgen('''
     %(c_name)s param = {
Signature · 1 line
2.26.2
author
#50 lore raw

PATCH v3 26/47 qapi/commands.py: Don't re-bind to variable of different type

In reply to John Snow

Mypy isn't a fan of rebinding a variable with a new data type. It's easy enough to avoid.

---
 scripts/qapi/commands.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
scripts/qapi/commands.py +1 -3
index f67393f871..586774a23c 100644
@@ -195,14 +195,12 @@ def gen_register_command(name, success_response, allow_oob, allow_preconfig):
     if not options:
         options = ['QCO_NO_OPTIONS']
 
-    options = " | ".join(options)
-
     ret = mcgen('''
     qmp_register_command(cmds, "%(name)s",
                          qmp_marshal_%(c_name)s, %(opts)s);
 ''',
                 name=name, c_name=c_name(name),
-                opts=options)
+                opts=" | ".join(options))
     return ret
 
 
Signature · 1 line
2.26.2
author
#51 lore raw

PATCH v3 27/47 qapi/commands.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior. This commit *only* adds annotations.

---
 scripts/qapi/commands.py | 71 ++++++++++++++++++++++++++++++----------
 1 file changed, 54 insertions(+), 17 deletions(-)
scripts/qapi/commands.py +54 -17
index 586774a23c..24f347a3f1 100644
@@ -13,16 +13,34 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import (
+    Dict,
+    List,
+    Optional,
+    Set,
+)
+
 from .common import c_name, mcgen
 from .gen import (
+    QAPIGenC,
     QAPIGenCCode,
     QAPISchemaModularCVisitor,
     build_params,
     ifcontext,
 )
+from .schema import (
+    QAPISchema,
+    QAPISchemaFeature,
+    QAPISchemaObjectType,
+    QAPISchemaType,
+)
+from .source import QAPISourceInfo
 
 
-def gen_command_decl(name, arg_type, boxed, ret_type):
+def gen_command_decl(name: str,
+                     arg_type: Optional[QAPISchemaObjectType],
+                     boxed: bool,
+                     ret_type: Optional[QAPISchemaType]) -> str:
     return mcgen('''
 %(c_type)s qmp_%(c_name)s(%(params)s);
 ''',
@@ -31,7 +49,10 @@ def gen_command_decl(name, arg_type, boxed, ret_type):
                  params=build_params(arg_type, boxed, 'Error **errp'))
 
 
-def gen_call(name, arg_type, boxed, ret_type):
+def gen_call(name: str,
+             arg_type: Optional[QAPISchemaObjectType],
+             boxed: bool,
+             ret_type: Optional[QAPISchemaType]) -> str:
     ret = ''
 
     argstr = ''
@@ -67,7 +88,7 @@ def gen_call(name, arg_type, boxed, ret_type):
     return ret
 
 
-def gen_marshal_output(ret_type):
+def gen_marshal_output(ret_type: QAPISchemaType) -> str:
     return mcgen('''
 
 static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in,
@@ -88,19 +109,22 @@ def gen_marshal_output(ret_type):
                  c_type=ret_type.c_type(), c_name=ret_type.c_name())
 
 
-def build_marshal_proto(name):
+def build_marshal_proto(name: str) -> str:
     return ('void qmp_marshal_%s(QDict *args, QObject **ret, Error **errp)'
             % c_name(name))
 
 
-def gen_marshal_decl(name):
+def gen_marshal_decl(name: str) -> str:
     return mcgen('''
 %(proto)s;
 ''',
                  proto=build_marshal_proto(name))
 
 
-def gen_marshal(name, arg_type, boxed, ret_type):
+def gen_marshal(name: str,
+                arg_type: Optional[QAPISchemaObjectType],
+                boxed: bool,
+                ret_type: Optional[QAPISchemaType]) -> str:
     have_args = boxed or (arg_type and not arg_type.is_empty())
 
     ret = mcgen('''
@@ -182,7 +206,10 @@ def gen_marshal(name, arg_type, boxed, ret_type):
     return ret
 
 
-def gen_register_command(name, success_response, allow_oob, allow_preconfig):
+def gen_register_command(name: str,
+                         success_response: bool,
+                         allow_oob: bool,
+                         allow_preconfig: bool) -> str:
     options = []
 
     if not success_response:
@@ -204,7 +231,7 @@ def gen_register_command(name, success_response, allow_oob, allow_preconfig):
     return ret
 
 
-def gen_registry(registry, prefix):
+def gen_registry(registry: str, prefix: str) -> str:
     ret = mcgen('''
 
 void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds)
@@ -221,15 +248,14 @@ def gen_registry(registry, prefix):
 
 
 class QAPISchemaGenCommandVisitor(QAPISchemaModularCVisitor):
-
-    def __init__(self, prefix):
+    def __init__(self, prefix: str):
         super().__init__(
             prefix, 'qapi-commands',
             ' * Schema-defined QAPI/QMP commands', None, __doc__)
         self._regy = QAPIGenCCode(None)
-        self._visited_ret_types = {}
+        self._visited_ret_types: Dict[QAPIGenC, Set[QAPISchemaType]] = {}
 
-    def _begin_user_module(self, name):
+    def _begin_user_module(self, name: str) -> None:
         self._visited_ret_types[self._genc] = set()
         commands = self._module_basename('qapi-commands', name)
         types = self._module_basename('qapi-types', name)
@@ -253,7 +279,7 @@ def _begin_user_module(self, name):
 ''',
                              types=types))
 
-    def visit_end(self):
+    def visit_end(self) -> None:
         self._add_system_module('init', ' * QAPI Commands initialization')
         self._genh.add(mcgen('''
 #include "qapi/qmp/dispatch.h"
@@ -269,9 +295,18 @@ def visit_end(self):
                                       prefix=self._prefix))
         self._genc.add(gen_registry(self._regy.get_content(), self._prefix))
 
-    def visit_command(self, name, info, ifcond, features,
-                      arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+    def visit_command(self,
+                      name: str,
+                      info: QAPISourceInfo,
+                      ifcond: List[str],
+                      features: List[QAPISchemaFeature],
+                      arg_type: Optional[QAPISchemaObjectType],
+                      ret_type: Optional[QAPISchemaType],
+                      gen: bool,
+                      success_response: bool,
+                      boxed: bool,
+                      allow_oob: bool,
+                      allow_preconfig: bool) -> None:
         if not gen:
             return
         # FIXME: If T is a user-defined type, the user is responsible
@@ -292,7 +327,9 @@ def visit_command(self, name, info, ifcond, features,
                                                 allow_oob, allow_preconfig))
 
 
-def gen_commands(schema, output_dir, prefix):
+def gen_commands(schema: QAPISchema,
+                 output_dir: str,
+                 prefix: str) -> None:
     vis = QAPISchemaGenCommandVisitor(prefix)
     schema.visit(vis)
     vis.write(output_dir)
Signature · 1 line
2.26.2
author
#52 lore raw

PATCH v3 28/47 qapi/commands.py: enable checking with mypy

In reply to John Snow
---
 scripts/qapi/mypy.ini | 5 -----
 1 file changed, 5 deletions(-)
scripts/qapi/mypy.ini +0 -5
index b668776d94..9da1dccef4 100644
@@ -4,11 +4,6 @@ strict_optional = False
 disallow_untyped_calls = False
 python_version = 3.6
 
-[mypy-qapi.commands]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.doc]
 disallow_subclassing_any = False
 disallow_untyped_defs = False
Signature · 1 line
2.26.2
author
#53 lore raw

PATCH v3 29/47 qapi/source.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior.
This commit *only* adds annotations.

---
 scripts/qapi/mypy.ini  |  5 -----
 scripts/qapi/source.py | 31 ++++++++++++++++++-------------
 2 files changed, 18 insertions(+), 18 deletions(-)
scripts/qapi/mypy.ini +0 -5
index 9da1dccef4..43c8bd1973 100644
@@ -39,11 +39,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.source]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.types]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
scripts/qapi/source.py +18 -13
index e97b9a8e15..1cc6a5b82d 100644
@@ -11,37 +11,42 @@
 
 import copy
 import sys
+from typing import List, Optional, TypeVar
 
 
 class QAPISchemaPragma:
-    def __init__(self):
+    def __init__(self) -> None:
         # Are documentation comments required?
         self.doc_required = False
         # Whitelist of commands allowed to return a non-dictionary
-        self.returns_whitelist = []
+        self.returns_whitelist: List[str] = []
         # Whitelist of entities allowed to violate case conventions
-        self.name_case_whitelist = []
+        self.name_case_whitelist: List[str] = []
 
 
 class QAPISourceInfo:
-    def __init__(self, fname, line, parent):
+    T = TypeVar('T', bound='QAPISourceInfo')
+
+    def __init__(self: T, fname: str, line: int, parent: Optional[T]):
         self.fname = fname
         self.line = line
         self.parent = parent
-        self.pragma = parent.pragma if parent else QAPISchemaPragma()
-        self.defn_meta = None
-        self.defn_name = None
+        self.pragma: QAPISchemaPragma = (
+            parent.pragma if parent else QAPISchemaPragma()
+        )
+        self.defn_meta: Optional[str] = None
+        self.defn_name: Optional[str] = None
 
-    def set_defn(self, meta, name):
+    def set_defn(self, meta: str, name: str) -> None:
         self.defn_meta = meta
         self.defn_name = name
 
-    def next_line(self):
+    def next_line(self: T) -> T:
         info = copy.copy(self)
         info.line += 1
         return info
 
-    def loc(self):
+    def loc(self) -> str:
         if self.fname is None:
             return sys.argv[0]
         ret = self.fname
@@ -49,13 +54,13 @@ def loc(self):
             ret += ':%d' % self.line
         return ret
 
-    def in_defn(self):
+    def in_defn(self) -> str:
         if self.defn_name:
             return "%s: In %s '%s':\n" % (self.fname,
                                           self.defn_meta, self.defn_name)
         return ''
 
-    def include_path(self):
+    def include_path(self) -> str:
         ret = ''
         parent = self.parent
         while parent:
@@ -63,5 +68,5 @@ def include_path(self):
             parent = parent.parent
         return ret
 
-    def __str__(self):
+    def __str__(self) -> str:
         return self.include_path() + self.in_defn() + self.loc()
Signature · 1 line
2.26.2
author
#55 lore raw

PATCH v3 30/47 qapi/source.py: delint with pylint

In reply to John Snow

Shush an error and leave a hint for future cleanups when we're allowed to use Python 3.7+.

---
 scripts/qapi/pylintrc  | 1 -
 scripts/qapi/source.py | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)
scripts/qapi/pylintrc +0 -1
index 6151427a51..7438806096 100644
@@ -8,7 +8,6 @@ ignore-patterns=doc.py,
                 gen.py,
                 parser.py,
                 schema.py,
-                source.py,
                 types.py,
                 visit.py,
 
scripts/qapi/source.py +3 -0
index 1cc6a5b82d..ba991d798f 100644
@@ -15,6 +15,9 @@
 
 
 class QAPISchemaPragma:
+    # Replace with @dataclass in Python 3.7+
+    # pylint: disable=too-few-public-methods
+
     def __init__(self) -> None:
         # Are documentation comments required?
         self.doc_required = False
Signature · 1 line
2.26.2
author
#56 lore raw

PATCH v3 31/47 qapi/gen.py: Fix edge-case of _is_user_module

In reply to John Snow

The edge case is that if the name is '', this expression returns a string instead of a bool, which violates our declared type.

---
 scripts/qapi/gen.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/qapi/gen.py +1 -1
index c1e65f2e52..75471c4cc2 100644
@@ -250,7 +250,7 @@ def __init__(self, prefix, what, user_blurb, builtin_blurb, pydoc):
 
     @staticmethod
     def _is_user_module(name):
-        return name and not name.startswith('./')
+        return bool(name and not name.startswith('./'))
 
     @staticmethod
     def _is_builtin_module(name):
Signature · 1 line
2.26.2
author
#58 lore raw

PATCH v3 32/47 qapi/gen.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior. This commit *only* adds annotations.

---
 scripts/qapi/gen.py | 107 ++++++++++++++++++++++++--------------------
 1 file changed, 58 insertions(+), 49 deletions(-)
scripts/qapi/gen.py +58 -49
index 75471c4cc2..eec941074b 100644
@@ -17,7 +17,13 @@
 import errno
 import os
 import re
-from typing import Optional
+from typing import (
+    Dict,
+    Iterator,
+    List,
+    Optional,
+    Tuple,
+)
 
 from .common import (
     c_fname,
@@ -29,31 +35,31 @@
     mcgen,
 )
 from .schema import QAPISchemaObjectType, QAPISchemaVisitor
+from .source import QAPISourceInfo
 
 
 class QAPIGen:
-
-    def __init__(self, fname):
+    def __init__(self, fname: Optional[str]):
         self.fname = fname
         self._preamble = ''
         self._body = ''
 
-    def preamble_add(self, text):
+    def preamble_add(self, text: str) -> None:
         self._preamble += text
 
-    def add(self, text):
+    def add(self, text: str) -> None:
         self._body += text
 
-    def get_content(self):
+    def get_content(self) -> str:
         return self._top() + self._preamble + self._body + self._bottom()
 
-    def _top(self):
+    def _top(self) -> str:
         return ''
 
-    def _bottom(self):
+    def _bottom(self) -> str:
         return ''
 
-    def write(self, output_dir):
+    def write(self, output_dir: str) -> None:
         # Include paths starting with ../ are used to reuse modules of the main
         # schema in specialised schemas. Don't overwrite the files that are
         # already generated for the main schema.
@@ -78,7 +84,7 @@ def write(self, output_dir):
         f.close()
 
 
-def _wrap_ifcond(ifcond, before, after):
+def _wrap_ifcond(ifcond: List[str], before: str, after: str) -> str:
     if before == after:
         return after   # suppress empty #if ... #endif
 
@@ -118,40 +124,38 @@ def build_params(arg_type: Optional[QAPISchemaObjectType],
 
 
 class QAPIGenCCode(QAPIGen):
-
-    def __init__(self, fname):
+    def __init__(self, fname: Optional[str]):
         super().__init__(fname)
-        self._start_if = None
+        self._start_if: Optional[Tuple[List[str], str, str]] = None
 
-    def start_if(self, ifcond):
+    def start_if(self, ifcond: List[str]) -> None:
         assert self._start_if is None
         self._start_if = (ifcond, self._body, self._preamble)
 
-    def end_if(self):
+    def end_if(self) -> None:
         assert self._start_if
         self._wrap_ifcond()
         self._start_if = None
 
-    def _wrap_ifcond(self):
+    def _wrap_ifcond(self) -> None:
         self._body = _wrap_ifcond(self._start_if[0],
                                   self._start_if[1], self._body)
         self._preamble = _wrap_ifcond(self._start_if[0],
                                       self._start_if[2], self._preamble)
 
-    def get_content(self):
+    def get_content(self) -> str:
         assert self._start_if is None
         return super().get_content()
 
 
 class QAPIGenC(QAPIGenCCode):
-
-    def __init__(self, fname, blurb, pydoc):
+    def __init__(self, fname: str, blurb: str, pydoc: str):
         super().__init__(fname)
         self._blurb = blurb
         self._copyright = '\n * '.join(re.findall(r'^Copyright .*', pydoc,
                                                   re.MULTILINE))
 
-    def _top(self):
+    def _top(self) -> str:
         return mcgen('''
 /* AUTOMATICALLY GENERATED, DO NOT MODIFY */
 
@@ -167,7 +171,7 @@ def _top(self):
 ''',
                      blurb=self._blurb, copyright=self._copyright)
 
-    def _bottom(self):
+    def _bottom(self) -> str:
         return mcgen('''
 
 /* Dummy declaration to prevent empty .o file */
@@ -177,16 +181,15 @@ def _bottom(self):
 
 
 class QAPIGenH(QAPIGenC):
-
-    def _top(self):
+    def _top(self) -> str:
         return super()._top() + guardstart(self.fname)
 
-    def _bottom(self):
+    def _bottom(self) -> str:
         return guardend(self.fname)
 
 
 @contextmanager
-def ifcontext(ifcond, *args):
+def ifcontext(ifcond: List[str], *args: QAPIGenCCode) -> Iterator[None]:
     """
     A 'with' statement context manager to wrap with start_if()/end_if()
 
@@ -214,15 +217,17 @@ def ifcontext(ifcond, *args):
 
 
 class QAPIGenDoc(QAPIGen):
-
-    def _top(self):
+    def _top(self) -> str:
         return (super()._top()
                 + '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n\n')
 
 
 class QAPISchemaMonolithicCVisitor(QAPISchemaVisitor):
-
-    def __init__(self, prefix, what, blurb, pydoc):
+    def __init__(self,
+                 prefix: str,
+                 what: str,
+                 blurb: str,
+                 pydoc: str):
         self._prefix = prefix
         self._what = what
         self._genc = QAPIGenC(self._prefix + self._what + '.c',
@@ -230,38 +235,42 @@ def __init__(self, prefix, what, blurb, pydoc):
         self._genh = QAPIGenH(self._prefix + self._what + '.h',
                               blurb, pydoc)
 
-    def write(self, output_dir):
+    def write(self, output_dir: str) -> None:
         self._genc.write(output_dir)
         self._genh.write(output_dir)
 
 
 class QAPISchemaModularCVisitor(QAPISchemaVisitor):
-
-    def __init__(self, prefix, what, user_blurb, builtin_blurb, pydoc):
+    def __init__(self,
+                 prefix: str,
+                 what: str,
+                 user_blurb: str,
+                 builtin_blurb: Optional[str],
+                 pydoc: str):
         self._prefix = prefix
         self._what = what
         self._user_blurb = user_blurb
         self._builtin_blurb = builtin_blurb
         self._pydoc = pydoc
-        self._genc = None
-        self._genh = None
-        self._module = {}
-        self._main_module = None
+        self._genc: Optional[QAPIGenC] = None
+        self._genh: Optional[QAPIGenH] = None
+        self._module: Dict[Optional[str], Tuple[QAPIGenC, QAPIGenH]] = {}
+        self._main_module: Optional[str] = None
 
     @staticmethod
-    def _is_user_module(name):
+    def _is_user_module(name: Optional[str]) -> bool:
         return bool(name and not name.startswith('./'))
 
     @staticmethod
-    def _is_builtin_module(name):
+    def _is_builtin_module(name: Optional[str]) -> bool:
         return not name
 
-    def _module_dirname(self, what, name):
+    def _module_dirname(self, what: str, name: Optional[str]) -> str:
         if self._is_user_module(name):
             return os.path.dirname(name)
         return ''
 
-    def _module_basename(self, what, name):
+    def _module_basename(self, what: str, name: Optional[str]) -> str:
         ret = '' if self._is_builtin_module(name) else self._prefix
         if self._is_user_module(name):
             basename = os.path.basename(name)
@@ -273,27 +282,27 @@ def _module_basename(self, what, name):
             ret += re.sub(r'-', '-' + name + '-', what)
         return ret
 
-    def _module_filename(self, what, name):
+    def _module_filename(self, what: str, name: Optional[str]) -> str:
         return os.path.join(self._module_dirname(what, name),
                             self._module_basename(what, name))
 
-    def _add_module(self, name, blurb):
+    def _add_module(self, name: Optional[str], blurb: str) -> None:
         basename = self._module_filename(self._what, name)
         genc = QAPIGenC(basename + '.c', blurb, self._pydoc)
         genh = QAPIGenH(basename + '.h', blurb, self._pydoc)
         self._module[name] = (genc, genh)
         self._genc, self._genh = self._module[name]
 
-    def _add_user_module(self, name, blurb):
+    def _add_user_module(self, name: str, blurb: str) -> None:
         assert self._is_user_module(name)
         if self._main_module is None:
             self._main_module = name
         self._add_module(name, blurb)
 
-    def _add_system_module(self, name, blurb):
+    def _add_system_module(self, name: Optional[str], blurb: str) -> None:
         self._add_module(name and './' + name, blurb)
 
-    def write(self, output_dir, opt_builtins=False):
+    def write(self, output_dir: str, opt_builtins: bool = False) -> None:
         for name in self._module:
             if self._is_builtin_module(name) and not opt_builtins:
                 continue
@@ -301,13 +310,13 @@ def write(self, output_dir, opt_builtins=False):
             genc.write(output_dir)
             genh.write(output_dir)
 
-    def _begin_system_module(self, name):
+    def _begin_system_module(self, name: None) -> None:
         pass
 
-    def _begin_user_module(self, name):
+    def _begin_user_module(self, name: str) -> None:
         pass
 
-    def visit_module(self, name):
+    def visit_module(self, name: Optional[str]) -> None:
         if name is None:
             if self._builtin_blurb:
                 self._add_system_module(None, self._builtin_blurb)
@@ -321,7 +330,7 @@ def visit_module(self, name):
             self._add_user_module(name, self._user_blurb)
             self._begin_user_module(name)
 
-    def visit_include(self, name, info):
+    def visit_include(self, name: str, info: QAPISourceInfo) -> None:
         relname = os.path.relpath(self._module_filename(self._what, name),
                                   os.path.dirname(self._genh.fname))
         self._genh.preamble_add(mcgen('''
Signature · 1 line
2.26.2
author
#59 lore raw

PATCH v3 33/47 qapi/gen.py: Enable checking with mypy

In reply to John Snow
---
 scripts/qapi/mypy.ini | 5 -----
 1 file changed, 5 deletions(-)
scripts/qapi/mypy.ini +0 -5
index 43c8bd1973..dbfeda748c 100644
@@ -19,11 +19,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.gen]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.introspect]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
Signature · 1 line
2.26.2
author
#60 lore raw

PATCH v3 34/47 qapi/gen.py: Remove unused parameter

In reply to John Snow

module_basename doesn't use the 'what' argument, so remove it.

---
 scripts/qapi/gen.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
scripts/qapi/gen.py +2 -2
index eec941074b..ece908a058 100644
@@ -265,7 +265,7 @@ def _is_user_module(name: Optional[str]) -> bool:
     def _is_builtin_module(name: Optional[str]) -> bool:
         return not name
 
-    def _module_dirname(self, what: str, name: Optional[str]) -> str:
+    def _module_dirname(self, name: Optional[str]) -> str:
         if self._is_user_module(name):
             return os.path.dirname(name)
         return ''
@@ -283,7 +283,7 @@ def _module_basename(self, what: str, name: Optional[str]) -> str:
         return ret
 
     def _module_filename(self, what: str, name: Optional[str]) -> str:
-        return os.path.join(self._module_dirname(what, name),
+        return os.path.join(self._module_dirname(name),
                             self._module_basename(what, name))
 
     def _add_module(self, name: Optional[str], blurb: str) -> None:
Signature · 1 line
2.26.2
author
#61 lore raw

PATCH v3 35/47 qapi/gen.py: update write() to be more idiomatic

In reply to John Snow

Make the file handling here just a tiny bit more idiomatic. (I realize this is heavily subjective.)

Use exist_ok=True for os.makedirs and remove the exception, use fdopen() to wrap the file descriptor in a File-like object, and use a context manager for managing the file pointer.

---
 scripts/qapi/gen.py | 24 ++++++++++--------------
 1 file changed, 10 insertions(+), 14 deletions(-)
scripts/qapi/gen.py +10 -14
index ece908a058..201f5af5a5 100644
@@ -14,7 +14,6 @@
 # See the COPYING file in the top-level directory.
 
 from contextlib import contextmanager
-import errno
 import os
 import re
 from typing import (
@@ -67,21 +66,18 @@ def write(self, output_dir: str) -> None:
             return
         pathname = os.path.join(output_dir, self.fname)
         odir = os.path.dirname(pathname)
+
         if odir:
-            try:
-                os.makedirs(odir)
-            except os.error as e:
-                if e.errno != errno.EEXIST:
-                    raise
+            os.makedirs(odir, exist_ok=True)
+
         fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
-        f = open(fd, 'r+', encoding='utf-8')
-        text = self.get_content()
-        oldtext = f.read(len(text) + 1)
-        if text != oldtext:
-            f.seek(0)
-            f.truncate(0)
-            f.write(text)
-        f.close()
+        with os.fdopen(fd, 'r+', encoding='utf-8') as fp:
+            text = self.get_content()
+            oldtext = fp.read(len(text) + 1)
+            if text != oldtext:
+                fp.seek(0)
+                fp.truncate(0)
+                fp.write(text)
 
 
 def _wrap_ifcond(ifcond: List[str], before: str, after: str) -> str:
Signature · 1 line
2.26.2
author
#62 lore raw

PATCH v3 36/47 qapi/gen.py: delint with pylint

In reply to John Snow

'fp' and 'fd' are self-evident in context, add them to the list of OK names.

_top and _bottom also need to stay standard methods because some users override the method and need to use self. Tell pylint to shush.

---
 scripts/qapi/gen.py   | 2 ++
 scripts/qapi/pylintrc | 5 +++--
 2 files changed, 5 insertions(+), 2 deletions(-)
scripts/qapi/gen.py +2 -0
index 201f5af5a5..bce3737d3b 100644
@@ -53,9 +53,11 @@ def get_content(self) -> str:
         return self._top() + self._preamble + self._body + self._bottom()
 
     def _top(self) -> str:
+        # pylint: disable=no-self-use
         return ''
 
     def _bottom(self) -> str:
+        # pylint: disable=no-self-use
         return ''
 
     def write(self, output_dir: str) -> None:
scripts/qapi/pylintrc +3 -2
index 7438806096..de132d03cf 100644
@@ -5,7 +5,6 @@
 ignore-patterns=doc.py,
                 error.py,
                 expr.py,
-                gen.py,
                 parser.py,
                 schema.py,
                 types.py,
@@ -46,7 +45,9 @@ good-names=i,
            k,
            ex,
            Run,
-           _
+           _,
+           fp,  # fp = open(...)
+           fd,  # fd = os.open(...)
 
 [VARIABLES]
 
Signature · 1 line
2.26.2
author
#63 lore raw

PATCH v3 37/47 qapi/introspect.py: assert obj is a dict when features are given

In reply to John Snow

This is necessary to keep mypy passing in the next patch when we add preliminary type hints. It will be removed shortly.

---
 scripts/qapi/introspect.py | 1 +
 1 file changed, 1 insertion(+)
scripts/qapi/introspect.py +1 -0
index 2850121cbd..c0cdb6d93a 100644
@@ -26,6 +26,7 @@ def _make_tree(obj, ifcond, features, extra=None):
     if ifcond:
         extra['if'] = ifcond
     if features:
+        assert isinstance(obj, dict)
         obj['features'] = [(f.name, {'if': f.ifcond}) for f in features]
     if extra:
         return (obj, extra)
Signature · 1 line
2.26.2
author
#64 lore raw

PATCH v3 38/47 qapi/instrospect.py: add preliminary type hint annotations

In reply to John Snow

From: Eduardo Habkost <[email protected]>

The typing of _make_tree and friends is a bit involved, but it can be done with some stubbed out types and a bit of elbow grease. The forthcoming patches attempt to make some simplifications, but having the type hints in advance can aid in review.

---
 scripts/qapi/introspect.py | 138 +++++++++++++++++++++++++++----------
 scripts/qapi/mypy.ini      |   5 --
 scripts/qapi/schema.py     |   2 +-
 3 files changed, 102 insertions(+), 43 deletions(-)
scripts/qapi/introspect.py +101 -37
index c0cdb6d93a..7d4dc3f8ee 100644
@@ -10,6 +10,15 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import (
+    Dict,
+    List,
+    Optional,
+    Sequence,
+    Tuple,
+    Union,
+)
+
 from .common import (
     c_name,
     gen_endif,
@@ -17,10 +26,34 @@
     mcgen,
 )
 from .gen import QAPISchemaMonolithicCVisitor
-from .schema import QAPISchemaArrayType, QAPISchemaBuiltinType, QAPISchemaType
+from .schema import (
+    QAPISchema,
+    QAPISchemaArrayType,
+    QAPISchemaBuiltinType,
+    QAPISchemaEntity,
+    QAPISchemaEnumMember,
+    QAPISchemaFeature,
+    QAPISchemaObjectType,
+    QAPISchemaObjectTypeMember,
+    QAPISchemaType,
+    QAPISchemaVariant,
+    QAPISchemaVariants,
+)
+from .source import QAPISourceInfo
 
+# The correct type for TreeNode is actually:
+# Union[AnnotatedNode, List[TreeNode], Dict[str, TreeNode], str, bool]
+# but mypy does not support recursive types yet.
+TreeNode = object
+_DObject = Dict[str, object]
+Extra = Dict[str, object]
+AnnotatedNode = Tuple[TreeNode, Extra]
 
-def _make_tree(obj, ifcond, features, extra=None):
+
+def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
+               features: List[QAPISchemaFeature],
+               extra: Optional[Extra] = None
+               ) -> Union[TreeNode, AnnotatedNode]:
     if extra is None:
         extra = {}
     if ifcond:
@@ -33,9 +66,11 @@ def _make_tree(obj, ifcond, features, extra=None):
     return obj
 
 
-def _tree_to_qlit(obj, level=0, suppress_first_indent=False):
+def _tree_to_qlit(obj: TreeNode,
+                  level: int = 0,
+                  suppress_first_indent: bool = False) -> str:
 
-    def indent(level):
+    def indent(level: int) -> str:
         return level * 4 * ' '
 
     if isinstance(obj, tuple):
@@ -85,21 +120,20 @@ def indent(level):
     return ret
 
 
-def to_c_string(string):
+def to_c_string(string: str) -> str:
     return '"' + string.replace('\\', r'\\').replace('"', r'\"') + '"'
 
 
 class QAPISchemaGenIntrospectVisitor(QAPISchemaMonolithicCVisitor):
-
-    def __init__(self, prefix, unmask):
+    def __init__(self, prefix: str, unmask: bool):
         super().__init__(
             prefix, 'qapi-introspect',
             ' * QAPI/QMP schema introspection', __doc__)
         self._unmask = unmask
-        self._schema = None
-        self._trees = []
-        self._used_types = []
-        self._name_map = {}
+        self._schema: Optional[QAPISchema] = None
+        self._trees: List[TreeNode] = []
+        self._used_types: List[QAPISchemaType] = []
+        self._name_map: Dict[str, str] = {}
         self._genc.add(mcgen('''
 #include "qemu/osdep.h"
 #include "%(prefix)sqapi-introspect.h"
@@ -107,10 +141,10 @@ def __init__(self, prefix, unmask):
 ''',
                              prefix=prefix))
 
-    def visit_begin(self, schema):
+    def visit_begin(self, schema: QAPISchema) -> None:
         self._schema = schema
 
-    def visit_end(self):
+    def visit_end(self) -> None:
         # visit the types that are actually used
         for typ in self._used_types:
             typ.visit(self)
@@ -132,18 +166,18 @@ def visit_end(self):
         self._used_types = []
         self._name_map = {}
 
-    def visit_needed(self, entity):
+    def visit_needed(self, entity: QAPISchemaEntity) -> bool:
         # Ignore types on first pass; visit_end() will pick up used types
         return not isinstance(entity, QAPISchemaType)
 
-    def _name(self, name):
+    def _name(self, name: str) -> str:
         if self._unmask:
             return name
         if name not in self._name_map:
             self._name_map[name] = '%d' % len(self._name_map)
         return self._name_map[name]
 
-    def _use_type(self, typ):
+    def _use_type(self, typ: QAPISchemaType) -> str:
         # Map the various integer types to plain int
         if typ.json_type() == 'int':
             typ = self._schema.lookup_type('int')
@@ -162,8 +196,10 @@ def _use_type(self, typ):
             return '[' + self._use_type(typ.element_type) + ']'
         return self._name(typ.name)
 
-    def _gen_tree(self, name, mtype, obj, ifcond, features):
-        extra = None
+    def _gen_tree(self, name: str, mtype: str, obj: _DObject,
+                  ifcond: List[str],
+                  features: Optional[List[QAPISchemaFeature]]) -> None:
+        extra: Extra = None
         if mtype not in ('command', 'event', 'builtin', 'array'):
             if not self._unmask:
                 # Output a comment to make it easy to map masked names
@@ -174,44 +210,64 @@ def _gen_tree(self, name, mtype, obj, ifcond, features):
         obj['meta-type'] = mtype
         self._trees.append(_make_tree(obj, ifcond, features, extra))
 
-    def _gen_member(self, member):
-        obj = {'name': member.name, 'type': self._use_type(member.type)}
+    def _gen_member(self,
+                    member: QAPISchemaObjectTypeMember) -> TreeNode:
+        obj: _DObject = {
+            'name': member.name,
+            'type': self._use_type(member.type)
+        }
         if member.optional:
             obj['default'] = None
         return _make_tree(obj, member.ifcond, member.features)
 
-    def _gen_variants(self, tag_name, variants):
+    def _gen_variants(self, tag_name: str,
+                      variants: List[QAPISchemaVariant]) -> _DObject:
         return {'tag': tag_name,
                 'variants': [self._gen_variant(v) for v in variants]}
 
-    def _gen_variant(self, variant):
-        obj = {'case': variant.name, 'type': self._use_type(variant.type)}
+    def _gen_variant(self, variant: QAPISchemaVariant) -> TreeNode:
+        obj: _DObject = {
+            'case': variant.name,
+            'type': self._use_type(variant.type)
+        }
         return _make_tree(obj, variant.ifcond, None)
 
-    def visit_builtin_type(self, name, info, json_type):
+    def visit_builtin_type(self, name: str, info: Optional[QAPISourceInfo],
+                           json_type: str) -> None:
         self._gen_tree(name, 'builtin', {'json-type': json_type}, [], None)
 
-    def visit_enum_type(self, name, info, ifcond, features, members, prefix):
+    def visit_enum_type(self, name: str, info: QAPISourceInfo,
+                        ifcond: List[str], features: List[QAPISchemaFeature],
+                        members: List[QAPISchemaEnumMember],
+                        prefix: Optional[str]) -> None:
         self._gen_tree(name, 'enum',
                        {'values': [_make_tree(m.name, m.ifcond, None)
                                    for m in members]},
                        ifcond, features)
 
-    def visit_array_type(self, name, info, ifcond, element_type):
+    def visit_array_type(self, name: str, info: Optional[QAPISourceInfo],
+                         ifcond: List[str],
+                         element_type: QAPISchemaType) -> None:
         element = self._use_type(element_type)
         self._gen_tree('[' + element + ']', 'array', {'element-type': element},
                        ifcond, None)
 
-    def visit_object_type_flat(self, name, info, ifcond, features,
-                               members, variants):
-        obj = {'members': [self._gen_member(m) for m in members]}
+    def visit_object_type_flat(self, name: str, info: Optional[QAPISourceInfo],
+                               ifcond: List[str],
+                               features: List[QAPISchemaFeature],
+                               members: Sequence[QAPISchemaObjectTypeMember],
+                               variants: Optional[QAPISchemaVariants]) -> None:
+        obj: _DObject = {'members': [self._gen_member(m) for m in members]}
         if variants:
             obj.update(self._gen_variants(variants.tag_member.name,
                                           variants.variants))
 
         self._gen_tree(name, 'object', obj, ifcond, features)
 
-    def visit_alternate_type(self, name, info, ifcond, features, variants):
+    def visit_alternate_type(self, name: str, info: QAPISourceInfo,
+                             ifcond: List[str],
+                             features: List[QAPISchemaFeature],
+                             variants: QAPISchemaVariants) -> None:
         self._gen_tree(name, 'alternate',
                        {'members': [
                            _make_tree({'type': self._use_type(m.type)},
@@ -219,24 +275,32 @@ def visit_alternate_type(self, name, info, ifcond, features, variants):
                            for m in variants.variants]},
                        ifcond, features)
 
-    def visit_command(self, name, info, ifcond, features,
-                      arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+    def visit_command(self, name: str, info: QAPISourceInfo, ifcond: List[str],
+                      features: List[QAPISchemaFeature],
+                      arg_type: QAPISchemaObjectType,
+                      ret_type: Optional[QAPISchemaType], gen: bool,
+                      success_response: bool, boxed: bool, allow_oob: bool,
+                      allow_preconfig: bool) -> None:
         arg_type = arg_type or self._schema.the_empty_object_type
         ret_type = ret_type or self._schema.the_empty_object_type
-        obj = {'arg-type': self._use_type(arg_type),
-               'ret-type': self._use_type(ret_type)}
+        obj: _DObject = {
+            'arg-type': self._use_type(arg_type),
+            'ret-type': self._use_type(ret_type)
+        }
         if allow_oob:
             obj['allow-oob'] = allow_oob
         self._gen_tree(name, 'command', obj, ifcond, features)
 
-    def visit_event(self, name, info, ifcond, features, arg_type, boxed):
+    def visit_event(self, name: str, info: QAPISourceInfo,
+                    ifcond: List[str], features: List[QAPISchemaFeature],
+                    arg_type: QAPISchemaObjectType, boxed: bool) -> None:
         arg_type = arg_type or self._schema.the_empty_object_type
         self._gen_tree(name, 'event', {'arg-type': self._use_type(arg_type)},
                        ifcond, features)
 
 
-def gen_introspect(schema, output_dir, prefix, opt_unmask):
+def gen_introspect(schema: QAPISchema, output_dir: str, prefix: str,
+                   opt_unmask: bool) -> None:
     vis = QAPISchemaGenIntrospectVisitor(prefix, opt_unmask)
     schema.visit(vis)
     vis.write(output_dir)
scripts/qapi/mypy.ini +0 -5
index dbfeda748c..9ce8b56f22 100644
@@ -19,11 +19,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.introspect]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.parser]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
scripts/qapi/schema.py +1 -1
index 483b4b68df..a26b25c5e9 100644
@@ -28,7 +28,7 @@
 class QAPISchemaEntity:
     meta: Optional[str] = None
 
-    def __init__(self, name, info, doc, ifcond=None, features=None):
+    def __init__(self, name: str, info, doc, ifcond=None, features=None):
         assert name is None or isinstance(name, str)
         for f in features or []:
             assert isinstance(f, QAPISchemaFeature)
Signature · 1 line
2.26.2
author
#65 lore raw

PATCH v3 39/47 qapi/introspect.py: add _gen_features helper

In reply to John Snow

_make_tree might receive a dict or some other type. Adding features information should arguably be performed by the caller at such a time when we know the type of the object and don't have to re-interrogate it.

---
 scripts/qapi/introspect.py | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)
scripts/qapi/introspect.py +12 -7
index 7d4dc3f8ee..08b3783ba7 100644
@@ -51,16 +51,12 @@
 
 
 def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
-               features: List[QAPISchemaFeature],
                extra: Optional[Extra] = None
                ) -> Union[TreeNode, AnnotatedNode]:
     if extra is None:
         extra = {}
     if ifcond:
         extra['if'] = ifcond
-    if features:
-        assert isinstance(obj, dict)
-        obj['features'] = [(f.name, {'if': f.ifcond}) for f in features]
     if extra:
         return (obj, extra)
     return obj
@@ -196,6 +192,11 @@ def _use_type(self, typ: QAPISchemaType) -> str:
             return '[' + self._use_type(typ.element_type) + ']'
         return self._name(typ.name)
 
+    @classmethod
+    def _gen_features(cls,
+                      features: List[QAPISchemaFeature]) -> List[TreeNode]:
+        return [_make_tree(f.name, f.ifcond) for f in features]
+
     def _gen_tree(self, name: str, mtype: str, obj: _DObject,
                   ifcond: List[str],
                   features: Optional[List[QAPISchemaFeature]]) -> None:
@@ -208,7 +209,9 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
             name = self._name(name)
         obj['name'] = name
         obj['meta-type'] = mtype
-        self._trees.append(_make_tree(obj, ifcond, features, extra))
+        if features:
+            obj['features'] = self._gen_features(features)
+        self._trees.append(_make_tree(obj, ifcond, extra))
 
     def _gen_member(self,
                     member: QAPISchemaObjectTypeMember) -> TreeNode:
@@ -218,7 +221,9 @@ def _gen_member(self,
         }
         if member.optional:
             obj['default'] = None
-        return _make_tree(obj, member.ifcond, member.features)
+        if member.features:
+            obj['features'] = self._gen_features(member.features)
+        return _make_tree(obj, member.ifcond)
 
     def _gen_variants(self, tag_name: str,
                       variants: List[QAPISchemaVariant]) -> _DObject:
@@ -230,7 +235,7 @@ def _gen_variant(self, variant: QAPISchemaVariant) -> TreeNode:
             'case': variant.name,
             'type': self._use_type(variant.type)
         }
-        return _make_tree(obj, variant.ifcond, None)
+        return _make_tree(obj, variant.ifcond)
 
     def visit_builtin_type(self, name: str, info: Optional[QAPISourceInfo],
                            json_type: str) -> None:
Signature · 1 line
2.26.2
author
#66 lore raw

PATCH v3 40/47 qapi/introspect.py: Unify return type of _make_tree()

In reply to John Snow

Returning a *something* or a Tuple of *something* is hard to accurately type. Let's just always return a tuple for structural consistency.

Instances of the 'TreeNode' type can be replaced with the slightly more specific 'AnnotatedNode' type.

---
 scripts/qapi/introspect.py | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
scripts/qapi/introspect.py +7 -8
index 08b3783ba7..54bc75393c 100644
@@ -52,14 +52,12 @@
 
 def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
                extra: Optional[Extra] = None
-               ) -> Union[TreeNode, AnnotatedNode]:
+               ) -> AnnotatedNode:
     if extra is None:
         extra = {}
     if ifcond:
         extra['if'] = ifcond
-    if extra:
-        return (obj, extra)
-    return obj
+    return (obj, extra)
 
 
 def _tree_to_qlit(obj: TreeNode,
@@ -127,7 +125,7 @@ def __init__(self, prefix: str, unmask: bool):
             ' * QAPI/QMP schema introspection', __doc__)
         self._unmask = unmask
         self._schema: Optional[QAPISchema] = None
-        self._trees: List[TreeNode] = []
+        self._trees: List[AnnotatedNode] = []
         self._used_types: List[QAPISchemaType] = []
         self._name_map: Dict[str, str] = {}
         self._genc.add(mcgen('''
@@ -194,7 +192,8 @@ def _use_type(self, typ: QAPISchemaType) -> str:
 
     @classmethod
     def _gen_features(cls,
-                      features: List[QAPISchemaFeature]) -> List[TreeNode]:
+                      features: List[QAPISchemaFeature]
+                      ) -> List[AnnotatedNode]:
         return [_make_tree(f.name, f.ifcond) for f in features]
 
     def _gen_tree(self, name: str, mtype: str, obj: _DObject,
@@ -214,7 +213,7 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
         self._trees.append(_make_tree(obj, ifcond, extra))
 
     def _gen_member(self,
-                    member: QAPISchemaObjectTypeMember) -> TreeNode:
+                    member: QAPISchemaObjectTypeMember) -> AnnotatedNode:
         obj: _DObject = {
             'name': member.name,
             'type': self._use_type(member.type)
@@ -230,7 +229,7 @@ def _gen_variants(self, tag_name: str,
         return {'tag': tag_name,
                 'variants': [self._gen_variant(v) for v in variants]}
 
-    def _gen_variant(self, variant: QAPISchemaVariant) -> TreeNode:
+    def _gen_variant(self, variant: QAPISchemaVariant) -> AnnotatedNode:
         obj: _DObject = {
             'case': variant.name,
             'type': self._use_type(variant.type)
Signature · 1 line
2.26.2
author
#67 lore raw

PATCH v3 41/47 qapi/introspect.py: replace 'extra' dict with 'comment' argument

In reply to John Snow

This is only used to pass in a dictionary with a comment already set, so skip the runaround and just accept the comment.

---
 scripts/qapi/introspect.py | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
scripts/qapi/introspect.py +8 -9
index 54bc75393c..c444214090 100644
@@ -51,12 +51,11 @@
 
 
 def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
-               extra: Optional[Extra] = None
-               ) -> AnnotatedNode:
-    if extra is None:
-        extra = {}
-    if ifcond:
-        extra['if'] = ifcond
+               comment: Optional[str] = None) -> AnnotatedNode:
+    extra: Extra = {
+        'if': ifcond,
+        'comment': comment,
+    }
     return (obj, extra)
 
 
@@ -199,18 +198,18 @@ def _gen_features(cls,
     def _gen_tree(self, name: str, mtype: str, obj: _DObject,
                   ifcond: List[str],
                   features: Optional[List[QAPISchemaFeature]]) -> None:
-        extra: Extra = None
+        comment: Optional[str] = None
         if mtype not in ('command', 'event', 'builtin', 'array'):
             if not self._unmask:
                 # Output a comment to make it easy to map masked names
                 # back to the source when reading the generated output.
-                extra = {'comment': '"%s" = %s' % (self._name(name), name)}
+                comment = f'"{self._name(name)}" = {name}'
             name = self._name(name)
         obj['name'] = name
         obj['meta-type'] = mtype
         if features:
             obj['features'] = self._gen_features(features)
-        self._trees.append(_make_tree(obj, ifcond, extra))
+        self._trees.append(_make_tree(obj, ifcond, comment))
 
     def _gen_member(self,
                     member: QAPISchemaObjectTypeMember) -> AnnotatedNode:
Signature · 1 line
2.26.2
author
#68 lore raw

PATCH v3 42/47 qapi/introspect.py: create a typed 'Node' data structure

In reply to John Snow

This replaces _make_tree with Node.__init__(), effectively. By creating it as a generic container, we can more accurately describe the exact nature of this particular Node.

---
 scripts/qapi/introspect.py | 77 +++++++++++++++++++-------------------
 1 file changed, 38 insertions(+), 39 deletions(-)
scripts/qapi/introspect.py +38 -39
index c444214090..3832edaf9d 100644
@@ -12,11 +12,12 @@
 
 from typing import (
     Dict,
+    Generic,
+    Iterable,
     List,
     Optional,
     Sequence,
-    Tuple,
-    Union,
+    TypeVar,
 )
 
 from .common import (
@@ -42,42 +43,42 @@
 from .source import QAPISourceInfo
 
 # The correct type for TreeNode is actually:
-# Union[AnnotatedNode, List[TreeNode], Dict[str, TreeNode], str, bool]
+# Union[Node[TreeNode], List[TreeNode], Dict[str, TreeNode], str, bool]
 # but mypy does not support recursive types yet.
 TreeNode = object
+_NodeType = TypeVar('_NodeType', bound=TreeNode)
 _DObject = Dict[str, object]
-Extra = Dict[str, object]
-AnnotatedNode = Tuple[TreeNode, Extra]
 
 
-def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
-               comment: Optional[str] = None) -> AnnotatedNode:
-    extra: Extra = {
-        'if': ifcond,
-        'comment': comment,
-    }
-    return (obj, extra)
+class Node(Generic[_NodeType]):
+    """
+    Node generally contains a SchemaInfo-like type (as a dict),
+    But it also used to wrap comments/ifconds around leaf value types.
+    """
+    # Remove after 3.7 adds @dataclass:
+    # pylint: disable=too-few-public-methods
+    def __init__(self, data: _NodeType, ifcond: Iterable[str],
+                 comment: Optional[str] = None):
+        self.data = data
+        self.comment: Optional[str] = comment
+        self.ifcond: Sequence[str] = tuple(ifcond)
 
 
-def _tree_to_qlit(obj: TreeNode,
-                  level: int = 0,
+def _tree_to_qlit(obj: TreeNode, level: int = 0,
                   suppress_first_indent: bool = False) -> str:
 
     def indent(level: int) -> str:
         return level * 4 * ' '
 
-    if isinstance(obj, tuple):
-        ifobj, extra = obj
-        ifcond = extra.get('if')
-        comment = extra.get('comment')
+    if isinstance(obj, Node):
         ret = ''
-        if comment:
-            ret += indent(level) + '/* %s */\n' % comment
-        if ifcond:
-            ret += gen_if(ifcond)
-        ret += _tree_to_qlit(ifobj, level)
-        if ifcond:
-            ret += '\n' + gen_endif(ifcond)
+        if obj.comment:
+            ret += indent(level) + '/* %s */\n' % obj.comment
+        if obj.ifcond:
+            ret += gen_if(obj.ifcond)
+        ret += _tree_to_qlit(obj.data, level)
+        if obj.ifcond:
+            ret += '\n' + gen_endif(obj.ifcond)
         return ret
 
     ret = ''
@@ -124,7 +125,7 @@ def __init__(self, prefix: str, unmask: bool):
             ' * QAPI/QMP schema introspection', __doc__)
         self._unmask = unmask
         self._schema: Optional[QAPISchema] = None
-        self._trees: List[AnnotatedNode] = []
+        self._trees: List[Node[_DObject]] = []
         self._used_types: List[QAPISchemaType] = []
         self._name_map: Dict[str, str] = {}
         self._genc.add(mcgen('''
@@ -191,9 +192,8 @@ def _use_type(self, typ: QAPISchemaType) -> str:
 
     @classmethod
     def _gen_features(cls,
-                      features: List[QAPISchemaFeature]
-                      ) -> List[AnnotatedNode]:
-        return [_make_tree(f.name, f.ifcond) for f in features]
+                      features: List[QAPISchemaFeature]) -> List[Node[str]]:
+        return [Node(f.name, f.ifcond) for f in features]
 
     def _gen_tree(self, name: str, mtype: str, obj: _DObject,
                   ifcond: List[str],
@@ -209,10 +209,10 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
         obj['meta-type'] = mtype
         if features:
             obj['features'] = self._gen_features(features)
-        self._trees.append(_make_tree(obj, ifcond, comment))
+        self._trees.append(Node(obj, ifcond, comment))
 
     def _gen_member(self,
-                    member: QAPISchemaObjectTypeMember) -> AnnotatedNode:
+                    member: QAPISchemaObjectTypeMember) -> Node[_DObject]:
         obj: _DObject = {
             'name': member.name,
             'type': self._use_type(member.type)
@@ -221,19 +221,19 @@ def _gen_member(self,
             obj['default'] = None
         if member.features:
             obj['features'] = self._gen_features(member.features)
-        return _make_tree(obj, member.ifcond)
+        return Node(obj, member.ifcond)
 
     def _gen_variants(self, tag_name: str,
                       variants: List[QAPISchemaVariant]) -> _DObject:
         return {'tag': tag_name,
                 'variants': [self._gen_variant(v) for v in variants]}
 
-    def _gen_variant(self, variant: QAPISchemaVariant) -> AnnotatedNode:
+    def _gen_variant(self, variant: QAPISchemaVariant) -> Node[_DObject]:
         obj: _DObject = {
             'case': variant.name,
             'type': self._use_type(variant.type)
         }
-        return _make_tree(obj, variant.ifcond)
+        return Node(obj, variant.ifcond)
 
     def visit_builtin_type(self, name: str, info: Optional[QAPISourceInfo],
                            json_type: str) -> None:
@@ -244,8 +244,7 @@ def visit_enum_type(self, name: str, info: QAPISourceInfo,
                         members: List[QAPISchemaEnumMember],
                         prefix: Optional[str]) -> None:
         self._gen_tree(name, 'enum',
-                       {'values': [_make_tree(m.name, m.ifcond, None)
-                                   for m in members]},
+                       {'values': [Node(m.name, m.ifcond) for m in members]},
                        ifcond, features)
 
     def visit_array_type(self, name: str, info: Optional[QAPISourceInfo],
@@ -273,9 +272,9 @@ def visit_alternate_type(self, name: str, info: QAPISourceInfo,
                              variants: QAPISchemaVariants) -> None:
         self._gen_tree(name, 'alternate',
                        {'members': [
-                           _make_tree({'type': self._use_type(m.type)},
-                                      m.ifcond, None)
-                           for m in variants.variants]},
+                           Node({'type': self._use_type(m.type)}, m.ifcond)
+                           for m in variants.variants
+                       ]},
                        ifcond, features)
 
     def visit_command(self, name: str, info: QAPISourceInfo, ifcond: List[str],
Signature · 1 line
2.26.2
author
#69 lore raw

PATCH v3 43/47 qapi/types.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior. This commit *only* adds annotations.

---
 scripts/qapi/mypy.ini |  5 ---
 scripts/qapi/types.py | 86 ++++++++++++++++++++++++++++++++-----------
 2 files changed, 64 insertions(+), 27 deletions(-)
scripts/qapi/mypy.ini +0 -5
index 9ce8b56f22..3babc380a8 100644
@@ -29,11 +29,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.types]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.visit]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
scripts/qapi/types.py +64 -22

207 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#70 lore raw

PATCH v3 44/47 qapi/types.py: remove one-letter variables

In reply to John Snow

"John, if pylint told you to jump off a bridge, would you?" Hey, if it looked like fun, I might.

Now that this file is clean, enable pylint checks on this file.

---
 scripts/qapi/pylintrc |  1 -
 scripts/qapi/types.py | 29 +++++++++++++++--------------
 2 files changed, 15 insertions(+), 15 deletions(-)
scripts/qapi/pylintrc +0 -1
index de132d03cf..cebaf600f9 100644
@@ -7,7 +7,6 @@ ignore-patterns=doc.py,
                 expr.py,
                 parser.py,
                 schema.py,
-                types.py,
                 visit.py,
 
 
scripts/qapi/types.py +15 -14
index b65188bac3..20b75c3505 100644
@@ -48,14 +48,14 @@ def gen_enum_lookup(name: str,
     .array = (const char *const[]) {
 ''',
                 c_name=c_name(name))
-    for m in members:
-        ret += gen_if(m.ifcond)
-        index = c_enum_const(name, m.name, prefix)
+    for member in members:
+        ret += gen_if(member.ifcond)
+        index = c_enum_const(name, member.name, prefix)
         ret += mcgen('''
         [%(index)s] = "%(name)s",
 ''',
-                     index=index, name=m.name)
-        ret += gen_endif(m.ifcond)
+                     index=index, name=member.name)
+        ret += gen_endif(member.ifcond)
 
     ret += mcgen('''
     },
@@ -78,13 +78,13 @@ def gen_enum(name: str,
 ''',
                 c_name=c_name(name))
 
-    for m in enum_members:
-        ret += gen_if(m.ifcond)
+    for member in enum_members:
+        ret += gen_if(member.ifcond)
         ret += mcgen('''
     %(c_enum)s,
 ''',
-                     c_enum=c_enum_const(name, m.name, prefix))
-        ret += gen_endif(m.ifcond)
+                     c_enum=c_enum_const(name, member.name, prefix))
+        ret += gen_endif(member.ifcond)
 
     ret += mcgen('''
 } %(c_name)s;
@@ -147,11 +147,12 @@ def gen_object(name: str, ifcond: List[str],
     objects_seen.add(name)
 
     ret = ''
-    if variants:
-        for v in variants.variants:
-            if isinstance(v.type, QAPISchemaObjectType):
-                ret += gen_object(v.type.name, v.type.ifcond, v.type.base,
-                                  v.type.local_members, v.type.variants)
+    for variant in variants.variants if variants else ():
+        obj = variant.type
+        if not isinstance(obj, QAPISchemaObjectType):
+            continue
+        ret += gen_object(obj.name, obj.ifcond, obj.base,
+                          obj.local_members, obj.variants)
 
     ret += mcgen('''
 
Signature · 1 line
2.26.2
author
#71 lore raw

PATCH v3 45/47 qapi/visit.py: assert tag_member contains a QAPISchemaEnumType

In reply to John Snow

This is true by design, but not presently able to be expressed in the type system. An assertion helps mypy understand our constraints.

---
 scripts/qapi/visit.py | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
scripts/qapi/visit.py +7 -5
index 14f30c228b..4f11fd325b 100644
@@ -22,7 +22,7 @@
     mcgen,
 )
 from .gen import QAPISchemaModularCVisitor, ifcontext
-from .schema import QAPISchemaObjectType
+from .schema import QAPISchemaEnumType, QAPISchemaObjectType
 
 
 def gen_visit_decl(name, scalar=False):
@@ -84,15 +84,17 @@ def gen_visit_object_members(name, base, members, variants):
         ret += gen_endif(memb.ifcond)
 
     if variants:
+        tag_member = variants.tag_member
+        assert isinstance(tag_member.type, QAPISchemaEnumType)
+
         ret += mcgen('''
     switch (obj->%(c_name)s) {
 ''',
-                     c_name=c_name(variants.tag_member.name))
+                     c_name=c_name(tag_member.name))
 
         for var in variants.variants:
-            case_str = c_enum_const(variants.tag_member.type.name,
-                                    var.name,
-                                    variants.tag_member.type.prefix)
+            case_str = c_enum_const(tag_member.type.name, var.name,
+                                    tag_member.type.prefix)
             ret += gen_if(var.ifcond)
             if var.type.name == 'q_empty':
                 # valid variant and nothing to do
Signature · 1 line
2.26.2
author
#72 lore raw

PATCH v3 46/47 qapi/visit.py: remove unused parameters from gen_visit_object

In reply to John Snow

And this fixes the pylint report for this file, so make sure we check this in the future, too.

---
 scripts/qapi/pylintrc | 1 -
 scripts/qapi/visit.py | 4 ++--
 2 files changed, 2 insertions(+), 3 deletions(-)
scripts/qapi/pylintrc +0 -1
index cebaf600f9..581755351b 100644
@@ -7,7 +7,6 @@ ignore-patterns=doc.py,
                 expr.py,
                 parser.py,
                 schema.py,
-                visit.py,
 
 
 [MESSAGES CONTROL]
scripts/qapi/visit.py +2 -2
index 4f11fd325b..e54694e23d 100644
@@ -250,7 +250,7 @@ def gen_visit_alternate(name, variants):
     return ret
 
 
-def gen_visit_object(name, base, members, variants):
+def gen_visit_object(name):
     return mcgen('''
 
 bool visit_type_%(c_name)s
@@ -343,7 +343,7 @@ def visit_object_type(self, name, info, ifcond, features,
             if not name.startswith('q_'):
                 # only explicit types need an allocating visit
                 self._genh.add(gen_visit_decl(name))
-                self._genc.add(gen_visit_object(name, base, members, variants))
+                self._genc.add(gen_visit_object(name))
 
     def visit_alternate_type(self, name, info, ifcond, features, variants):
         with ifcontext(ifcond, self._genh, self._genc):
Signature · 1 line
2.26.2
author
#73 lore raw

PATCH v3 47/47 qapi/visit.py: add type hint annotations

In reply to John Snow

Annotations do not change runtime behavior. This commit *only* adds annotations.

---
 scripts/qapi/mypy.ini |  5 ---
 scripts/qapi/visit.py | 73 +++++++++++++++++++++++++++++++++----------
 2 files changed, 56 insertions(+), 22 deletions(-)
scripts/qapi/mypy.ini +0 -5
index 3babc380a8..5ab3433c5f 100644
@@ -28,8 +28,3 @@ check_untyped_defs = False
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
-
-[mypy-qapi.visit]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
scripts/qapi/visit.py +56 -17

164 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#74 lore raw
In reply to John Snow
Quoted 6 lines · John Snow

Hi, this series adds static type hints to the QAPI module. This is part one!

Part 1: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt1
Everything: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt6

At this point, I have made only extremely minor edits to what has been reviewed so far, so I won't re-spin -- but I will announce that the most up-to-date version is rebased and pushed here:

https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt1

Thanks,
--js