Skip to contentSkip to conversation

Use Doxygen and sphinx for html documentation

PATCH Superseded Luca Fancellu 2 participants 12 messages Apr 19, 2021 – Apr 26, 2021

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover Use Doxygen and sphinx for html documentation

This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that should be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).

To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.

Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):

./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-html

now in docs/sphinx/html/ we have the generated docs starting from the index.html page.

Luca Fancellu (3):
docs: add doxygen support for html documentation
docs: hypercalls sphinx skeleton for generated html docs/doxygen: doxygen documentation for grant_table.h

 .gitignore                                    |    7 +
 config/Docs.mk.in                             |    2 +
 docs/Makefile                                 |   51 +-
 docs/conf.py                                  |   48 +-
 docs/configure                                |  258 ++
 docs/configure.ac                             |   15 +
 docs/hypercall-interfaces/arm32.rst           |    4 +
 docs/hypercall-interfaces/arm64.rst           |   33 +
 .../arm64/grant_tables.rst                    |    8 +
 docs/hypercall-interfaces/index.rst.in        |    7 +
 docs/hypercall-interfaces/x86_64.rst          |    4 +
 docs/index.rst                                |    8 +
 docs/xen-doxygen/customdoxygen.css            |   36 +
 docs/xen-doxygen/doxy_input.list              |    1 +
 docs/xen-doxygen/doxy_input_template.in       |   21 +
 docs/xen-doxygen/footer.html                  |   21 +
 docs/xen-doxygen/header.html                  |   56 +
 docs/xen-doxygen/mainpage.md                  |    5 +
 docs/xen-doxygen/xen_project_logo_165x67.png  |  Bin 0 -> 18223 bytes
 docs/xen.doxyfile.in                          | 2314 +++++++++++++++++
 m4/ax_python_module.m4                        |   56 +
 m4/docs_tool.m4                               |    9 +
 xen/include/public/grant_table.h              |   97 +-
 23 files changed, 3021 insertions(+), 40 deletions(-)
 create mode 100644 docs/hypercall-interfaces/arm32.rst
 create mode 100644 docs/hypercall-interfaces/arm64.rst
 create mode 100644 docs/hypercall-interfaces/arm64/grant_tables.rst
 create mode 100644 docs/hypercall-interfaces/index.rst.in
 create mode 100644 docs/hypercall-interfaces/x86_64.rst
 create mode 100644 docs/xen-doxygen/customdoxygen.css
 create mode 100644 docs/xen-doxygen/doxy_input.list
 create mode 100644 docs/xen-doxygen/doxy_input_template.in
 create mode 100644 docs/xen-doxygen/footer.html
 create mode 100644 docs/xen-doxygen/header.html
 create mode 100644 docs/xen-doxygen/mainpage.md
 create mode 100644 docs/xen-doxygen/xen_project_logo_165x67.png
 create mode 100644 docs/xen.doxyfile.in
 create mode 100644 m4/ax_python_module.m4
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/3 docs: add doxygen support for html documentation

Add doxygen support to build html documentation with sphinx, to do that the following modification are applied:

1) Modify docs/configure.ac and consequently configure script to check, through the ./configure script, the presence in the system of the sphinx-build binary, if it is found then it checks also the presence of the doxygen binary, breathe and sphinx-rtd-theme python packages.
Doxygen and the packages are required only if sphinx-build is found, otherwise the Makefile will simply skip the sphinx html generation.
The ax_python_module.m4 support is needed to check for python packages.
2) Add doxygen templates and configuration file
3) Modify docs/Makefile to call in sequence doxygen and sphinx-build, the doxygen configuration file will be modified to include the xen absolute path and
a list of folder to check.
A doxygen_input.h file is generated to include every header listed in the doxy_input.list file.

---
v2 changes:
- Fix bug in Makefile when sphinx is not found in the system

---
 .gitignore                                   |    6 +
 config/Docs.mk.in                            |    2 +
 docs/Makefile                                |   47 +-
 docs/conf.py                                 |   48 +-
 docs/configure                               |  258 ++
 docs/configure.ac                            |   15 +
 docs/xen-doxygen/customdoxygen.css           |   36 +
 docs/xen-doxygen/doxy_input.list             |    0
 docs/xen-doxygen/doxy_input_template.in      |   21 +
 docs/xen-doxygen/footer.html                 |   21 +
 docs/xen-doxygen/header.html                 |   56 +
 docs/xen-doxygen/mainpage.md                 |    5 +
 docs/xen-doxygen/xen_project_logo_165x67.png |  Bin 0 -> 18223 bytes
 docs/xen.doxyfile.in                         | 2314 ++++++++++++++++++
 m4/ax_python_module.m4                       |   56 +
 m4/docs_tool.m4                              |    9 +
 16 files changed, 2888 insertions(+), 6 deletions(-)
 create mode 100644 docs/xen-doxygen/customdoxygen.css
 create mode 100644 docs/xen-doxygen/doxy_input.list
 create mode 100644 docs/xen-doxygen/doxy_input_template.in
 create mode 100644 docs/xen-doxygen/footer.html
 create mode 100644 docs/xen-doxygen/header.html
 create mode 100644 docs/xen-doxygen/mainpage.md
 create mode 100644 docs/xen-doxygen/xen_project_logo_165x67.png
 create mode 100644 docs/xen.doxyfile.in
 create mode 100644 m4/ax_python_module.m4
.gitignore +6 -0
index 1c2fa1530b..a5eebf1213 100644
@@ -58,6 +58,12 @@ docs/man7/
 docs/man8/
 docs/pdf/
 docs/txt/
+docs/doxygen-output
+docs/sphinx
+docs/xen.doxyfile
+docs/xen.doxyfile.tmp
+docs/doxygen_input.h
+docs/doxygen_input.h.tmp
 extras/mini-os*
 install/*
 stubdom/*-minios-config.mk
config/Docs.mk.in +2 -0
index e76e5cd5ff..dfd4a02838 100644
@@ -7,3 +7,5 @@ POD2HTML            := @POD2HTML@
 POD2TEXT            := @POD2TEXT@
 PANDOC              := @PANDOC@
 PERL                := @PERL@
+SPHINXBUILD         := @SPHINXBUILD@
+DOXYGEN             := @DOXYGEN@
docs/Makefile +44 -3

65 lines not on this page · raw message

docs/conf.py +45 -3

83 lines not on this page · raw message

docs/configure +258 -0

294 lines not on this page · raw message

docs/configure.ac +15 -0
index c2e5edd3b3..a2ff55f30a 100644
@@ -20,6 +20,7 @@ m4_include([../m4/docs_tool.m4])
 m4_include([../m4/path_or_fail.m4])
 m4_include([../m4/features.m4])
 m4_include([../m4/paths.m4])
+m4_include([../m4/ax_python_module.m4])
 
 AX_XEN_EXPAND_CONFIG()
 
@@ -29,6 +30,20 @@ AX_DOCS_TOOL_PROG([POD2HTML], [pod2html])
 AX_DOCS_TOOL_PROG([POD2TEXT], [pod2text])
 AX_DOCS_TOOL_PROG([PANDOC], [pandoc])
 
+# If sphinx is installed, make sure to have also the dependencies to build
+# Sphinx documentation.
+AC_CHECK_PROGS([SPHINXBUILD], [sphinx-build], [no])
+    AS_IF([test "x$SPHINXBUILD" = xno],
+        [AC_MSG_WARN(sphinx-build is not available so sphinx documentation \
+won't be built)],
+        [
+            AC_PATH_PROG([SPHINXBUILD], [sphinx-build])
+            AX_DOCS_TOOL_REQ_PROG([DOXYGEN], [doxygen])
+            AX_PYTHON_MODULE([breathe],[yes])
+            AX_PYTHON_MODULE([sphinx_rtd_theme], [yes])
+        ]
+    )
+
 AC_ARG_VAR([PERL], [Path to Perl parser])
 AX_PATH_PROG_OR_FAIL([PERL], [perl])
 
docs/xen-doxygen/customdoxygen.css new file +36 -0
new file mode 100644
index 0000000000..4735e41cf5
@@ -0,0 +1,36 @@
+/* Custom CSS for Doxygen-generated HTML
+ * Copyright (c) 2015 Intel Corporation
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+code {
+  font-family: Monaco,Menlo,Consolas,"Courier New",monospace;
+  background-color: #D8D8D8;
+  padding: 0 0.25em 0 0.25em;
+}
+
+pre.fragment {
+  display: block;
+  font-family: Monaco,Menlo,Consolas,"Courier New",monospace;
+  padding: 1rem;
+  word-break: break-all;
+  word-wrap: break-word;
+  white-space: pre;
+  background-color: #D8D8D8;
+}
+
+#projectlogo
+{
+  vertical-align: middle;
+}
+
+#projectname
+{
+  font: 200% Tahoma, Arial,sans-serif;
+  color: #3D578C;
+}
+
+#projectbrief
+{
+  color: #3D578C;
+}
docs/xen-doxygen/doxy_input.list new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
docs/xen-doxygen/doxy_input_template.in new file +21 -0
new file mode 100644
index 0000000000..fc7565ab5b
@@ -0,0 +1,21 @@
+#define __XEN__
+
+#include "@XEN_BASE@/xen/include/xen/config.h"
+
+#if defined(CONFIG_X86_64)
+
+#define __x86_64__ 1
+
+#elif defined(CONFIG_ARM_64)
+
+#define __aarch64__ 1
+
+#elif defined(CONFIG_ARM_32)
+
+#define __arm__ 1
+
+#else
+
+#error Architecture not supported/recognized.
+
+#endif
docs/xen-doxygen/footer.html new file +21 -0
new file mode 100644
index 0000000000..a24bf2b9b4
@@ -0,0 +1,21 @@
+<!-- HTML footer for doxygen 1.8.13-->
+<!-- start footer part -->
+<!--BEGIN GENERATE_TREEVIEW-->
+<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
+  <ul>
+    $navpath
+    <li class="footer">$generatedby
+    <a href="http://www.doxygen.org/index.html">
+    <img class="footer" src="$relpath^doxygen.png" alt="doxygen"/></a> $doxygenversion </li>
+  </ul>
+</div>
+<!--END GENERATE_TREEVIEW-->
+<!--BEGIN !GENERATE_TREEVIEW-->
+<hr class="footer"/><address class="footer"><small>
+$generatedby &#160;<a href="http://www.doxygen.org/index.html">
+<img class="footer" src="$relpath^doxygen.png" alt="doxygen"/>
+</a> $doxygenversion
+</small></address>
+<!--END !GENERATE_TREEVIEW-->
+</body>
+</html>
docs/xen-doxygen/header.html new file +56 -0

56 lines not on this page · raw message

docs/xen-doxygen/mainpage.md new file +5 -0
new file mode 100644
index 0000000000..ff548b87fc
@@ -0,0 +1,5 @@
+# API Documentation   {#index}
+
+## Introduction
+
+## Licensing
docs/xen-doxygen/xen_project_logo_165x67.png new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..7244959d59cdeb9f23c5202160ea45508dfc7265
GIT binary patch
docs/xen.doxyfile.in new file +2314 -0

2,314 lines not on this page · raw message

m4/ax_python_module.m4 new file +56 -0

57 lines not on this page · raw message

m4/docs_tool.m4 +9 -0
index 3e8814ac8d..39aa348026 100644
@@ -15,3 +15,12 @@ dnl
         AC_MSG_WARN([$2 is not available so some documentation won't be built])
     ])
 ])
+
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#2 lore raw

PATCH v2 2/3 docs: hypercalls sphinx skeleton for generated html

In reply to Luca Fancellu

Create a skeleton for the documentation about hypercalls

---
 .gitignore                             |  1 +
 docs/Makefile                          |  4 ++++
 docs/hypercall-interfaces/arm32.rst    |  4 ++++
 docs/hypercall-interfaces/arm64.rst    | 32 ++++++++++++++++++++++++++
 docs/hypercall-interfaces/index.rst.in |  7 ++++++
 docs/hypercall-interfaces/x86_64.rst   |  4 ++++
 docs/index.rst                         |  8 +++++++
 7 files changed, 60 insertions(+)
 create mode 100644 docs/hypercall-interfaces/arm32.rst
 create mode 100644 docs/hypercall-interfaces/arm64.rst
 create mode 100644 docs/hypercall-interfaces/index.rst.in
 create mode 100644 docs/hypercall-interfaces/x86_64.rst
.gitignore +1 -0
index a5eebf1213..dd3ca96b6f 100644
@@ -64,6 +64,7 @@ docs/xen.doxyfile
 docs/xen.doxyfile.tmp
 docs/doxygen_input.h
 docs/doxygen_input.h.tmp
+docs/hypercall-interfaces/index.rst
 extras/mini-os*
 install/*
 stubdom/*-minios-config.mk
docs/Makefile +4 -0
index a736a1714f..7ec209e502 100644
@@ -65,6 +65,9 @@ build: html txt pdf man-pages figs
 sphinx-html: $(DOXY_DEPS) $(DOXY_LIST_SOURCES)
 ifneq ($(SPHINXBUILD),no)
 	$(DOXYGEN) xen.doxyfile
+	@echo "Generating hypercall-interfaces/index.rst"
+	@sed -e "s,@XEN_TARGET_ARCH@,$(XEN_TARGET_ARCH),g" \
+		hypercall-interfaces/index.rst.in > hypercall-interfaces/index.rst
 	XEN_ROOT=$(realpath $(XEN_ROOT)) $(SPHINXBUILD) -b html . sphinx/html
 else
 	@echo "Sphinx is not installed; skipping sphinx-html documentation."
@@ -113,6 +116,7 @@ clean: clean-man-pages
 	rm -f xen.doxyfile.tmp
 	rm -f doxygen_input.h
 	rm -f doxygen_input.h.tmp
+	rm -f hypercall-interfaces/index.rst
 
 .PHONY: distclean
 distclean: clean
docs/hypercall-interfaces/arm32.rst new file +4 -0
new file mode 100644
index 0000000000..4e973fbbaf
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces - arm32
+============================
docs/hypercall-interfaces/arm64.rst new file +32 -0
new file mode 100644
index 0000000000..5e701a2adc
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces - arm64
+============================
+
+Starting points
+---------------
+.. toctree::
+   :maxdepth: 2
+
+
+
+Functions
+---------
+
+
+Structs
+-------
+
+
+Enums and sets of #defines
+--------------------------
+
+
+Typedefs
+--------
+
+
+Enum values and individual #defines
+-----------------------------------
+
+
docs/hypercall-interfaces/index.rst.in new file +7 -0
new file mode 100644
index 0000000000..e4dcc5db8d
@@ -0,0 +1,7 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces
+====================
+
+.. toctree::
+   @XEN_TARGET_ARCH@
docs/hypercall-interfaces/x86_64.rst new file +4 -0
new file mode 100644
index 0000000000..3ed70dff95
@@ -0,0 +1,4 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Hypercall Interfaces - x86_64
+=============================
docs/index.rst +8 -0
index b75487a05d..52226a42d8 100644
@@ -53,6 +53,14 @@ kind of development environment.
    hypervisor-guide/index
 
 
+Hypercall Interfaces documentation
+----------------------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   hypercall-interfaces/index
+
 Miscellanea
 -----------
 
Signature · 1 line
2.17.1
#3 lore raw

PATCH v2 3/3 docs/doxygen: doxygen documentation for grant_table.h

In reply to Luca Fancellu

Modification to include/public/grant_table.h:

1) Add doxygen tags to:
- Create Grant tables section
- include variables in the generated documentation
2) Add .rst file for grant table for Arm64

---
v2 changes:
- Revert back to anonymous union/struct
- add doxygen tags to skip anonymous union/struct

---
 docs/hypercall-interfaces/arm64.rst           |  1 +
 .../arm64/grant_tables.rst                    |  8 ++
 docs/xen-doxygen/doxy_input.list              |  1 +
 xen/include/public/grant_table.h              | 97 ++++++++++++-------
 4 files changed, 73 insertions(+), 34 deletions(-)
 create mode 100644 docs/hypercall-interfaces/arm64/grant_tables.rst
docs/hypercall-interfaces/arm64.rst +1 -0
index 5e701a2adc..c30a7142b1 100644
@@ -8,6 +8,7 @@ Starting points
 .. toctree::
    :maxdepth: 2
 
+   arm64/grant_tables
 
 
 Functions
docs/hypercall-interfaces/arm64/grant_tables.rst new file +8 -0
new file mode 100644
index 0000000000..8955ec5812
@@ -0,0 +1,8 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Grant Tables
+============
+
+.. doxygengroup:: grant_table
+   :project: Xen
+   :members:
docs/xen-doxygen/doxy_input.list +1 -0
index e69de29bb2..233d692fa7 100644
@@ -0,0 +1 @@
+xen/include/public/grant_table.h
xen/include/public/grant_table.h +63 -34

247 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 6 lines · Luca Fancellu

Modification to include/public/grant_table.h:

1) Add doxygen tags to:
- Create Grant tables section
- include variables in the generated documentation
2) Add .rst file for grant table for Arm64

I'm missing some reasoning about at least some of the changes done to grant_table.h. Looking at this and the earlier patches I also couldn't spot any general outline of what is acceptable or even necessary in such a header to be understood by doxygen. Without this written down somewhere (or, if documented elsewhere, a pointer provided to that doc) I'm afraid things might get screwed up again later on.

Quoted 16 lines
docs/hypercall-interfaces/arm64.rst +1 -0
@@ -8,6 +8,7 @@ Starting points
 .. toctree::
    :maxdepth: 2
 
+   arm64/grant_tables
 
 
 Functions
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -0
new file mode 100644
index 0000000000..8955ec5812

docs/hypercall-interfaces/arm64/grant_tables.rst line 6

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -0
@@ -0,0 +1,8 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Grant Tables
+============
+
+.. doxygengroup:: grant_table

Why is this Arm64-specific?

Quoted 32 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -0
@@ -73,20 +75,25 @@
  *                           frame, or zero if none.
  *  3. Write memory barrier (WMB).
  *  4. Write ent->flags, inc. valid type.
+ * @endcode
  *
  * Invalidating an unused GTF_permit_access entry:
+ * @code
  *  1. flags = ent->flags.
  *  2. Observe that !(flags & (GTF_reading|GTF_writing)).
  *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
  *  NB. No need for WMB as reuse of entry is control-dependent on success of
  *      step 3, and all architectures guarantee ordering of ctrl-dep writes.
+ * @endcode
  *
  * Invalidating an in-use GTF_permit_access entry:
+ *
  *  This cannot be done directly. Request assistance from the domain controller
  *  which can set a timeout on the use of a grant entry and take necessary
  *  action. (NB. This is not yet implemented!).
  *
  * Invalidating an unused GTF_accept_transfer entry:
+ * @code
  *  1. flags = ent->flags.
  *  2. Observe that !(flags & GTF_transfer_committed). [*]
  *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
@@ -97,47 +104,55 @@
  *      transferred frame is written. It is safe for the guest to spin waiting
  *      for this to occur (detect by observing GTF_transfer_completed in
  *      ent->flags).
+ * @endcode
  *

docs/hypercall-interfaces/arm64/grant_tables.rst line 118

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +2 -0
  * Invalidating a committed GTF_accept_transfer entry:
  *  1. Wait for (ent->flags & GTF_transfer_completed).
  *
  * Changing a GTF_permit_access from writable to read-only:
+ *
  *  Use SMP-safe CMPXCHG to set GTF_readonly, while checking !GTF_writing.
  *
  * Changing a GTF_permit_access from read-only to writable:
+ *
  *  Use SMP-safe bit-setting instruction.

For example - are the blank lines you add necessary or merely nice to have in your personal opinion?

docs/hypercall-interfaces/arm64/grant_tables.rst line 114

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -6
- */
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 11 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +5 -2
-
-/*
+ *
  * A grant table comprises a packed array of grant entries in one or more
  * page frames shared between Xen and a guest.
+ *
  * [XEN]: This field is written by Xen and read by the sharing guest.
+ *
  * [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables

docs/hypercall-interfaces/arm64/grant_tables.rst line 134

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -3
+ * @{
  */
 
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.
  */
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

Quoted 30 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
      * In that case, the frame field has the same semantics as the
      * field of the same name in the V1 entry structure.
      */
+    /** @cond skip anonymous struct/union for doxygen */
     struct {
         grant_entry_header_t hdr;
         uint32_t pad0;
         uint64_t frame;
     } full_page;
+    /** @endcond */
 
     /*
      * If the grant type is GTF_grant_access and GTF_sub_page is set,
      * @domid is allowed to access bytes [@page_off,@page_off+@length)
      * in frame @frame.
      */
+    /** @cond skip anonymous struct/union for doxygen */
     struct {
         grant_entry_header_t hdr;
         uint16_t page_off;
         uint16_t length;
         uint64_t frame;
     } sub_page;
+    /** @endcond */
 
     /*
      * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
      * The current version of Xen does not allow transitive grants

docs/hypercall-interfaces/arm64/grant_tables.rst union grant_entry_v2 { · line 299

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +2 -0
      * to be mapped.
      */
+    /** @cond skip anonymous struct/union for doxygen */
     struct {
         grant_entry_header_t hdr;
         domid_t trans_domid;
         uint16_t pad0;
         grant_ref_t gref;
     } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Quoted 15 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -1
@@ -433,7 +454,12 @@ typedef struct gnttab_transfer gnttab_transfer_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 
 
-/*
+#define _GNTCOPY_source_gref      (0)
+#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
+#define _GNTCOPY_dest_gref        (1)
+#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
+
+/**
  * GNTTABOP_copy: Hypervisor based copy
  * source and destinations can be eithers MFNs or, for foreign domains,
  * grant references. the foreign domain has to grant read/write access

docs/hypercall-interfaces/arm64/grant_tables.rst DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t); · line 480

Quoted 9 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -5
@@ -451,18 +477,15 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
  * bytes to be copied.
  */
 
-#define _GNTCOPY_source_gref      (0)
-#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
-#define _GNTCOPY_dest_gref        (1)
-#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
-
 struct gnttab_copy {

Again the question - why the movement?

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -1
@@ -579,17 +602,19 @@ struct gnttab_swap_grant_ref {
 typedef struct gnttab_swap_grant_ref gnttab_swap_grant_ref_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);
 
-/*
+/**
  * Issue one or more cache maintenance operations on a portion of a
  * page granted to the calling domain by a foreign domain.
  */
 struct gnttab_cache_flush {

docs/hypercall-interfaces/arm64/grant_tables.rst struct gnttab_swap_grant_ref { · line 617

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -2
+    /** @cond skip anonymous struct/union for doxygen */
     union {
         uint64_t dev_bus_addr;
         grant_ref_t ref;
     } a;
-    uint16_t offset; /* offset from start of grant */
-    uint16_t length; /* size within the grant */
+    /** @endcond */
+    uint16_t offset; /**< offset from start of grant */
+    uint16_t length; /**< size within the grant */

Skipping just part of a struct is perhaps even more confusing than omitting it altogether.

Also, what's the significance of "/**<" ?

Jan

#5 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 12 lines · Jan Beulich
Quoted 6 lines · Luca Fancellu

Modification to include/public/grant_table.h:

1) Add doxygen tags to:
- Create Grant tables section
- include variables in the generated documentation
2) Add .rst file for grant table for Arm64

Quoted 7 lines

I'm missing some reasoning about at least some of the changes done to grant_table.h. Looking at this and the earlier patches I also couldn't spot any general outline of what is acceptable or even necessary in such a header to be understood by doxygen. Without this written down somewhere (or, if documented elsewhere, a pointer provided to that doc) I'm afraid things might get screwed up again later on.

Hi Jan,

Doxygen is a tool that generates documentation based on parsing the source code comments, it recognises some commands in the comments and builds the documentation sticking to what you wrote.

Here the doxygen docs: https://www.doxygen.nl/manual/docblocks.html

Basically it doesn’t react to all comments, it parses only some well crafted comments as explained in its documentation.

Quoted 10 lines
Quoted 10 lines
docs/hypercall-interfaces/arm64.rst +0 -0
@@ -8,6 +8,7 @@ Starting points

.. toctree::
:maxdepth: 2

+ arm64/grant_tables

Quoted 15 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 6

Quoted 13 lines

Functions

docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -0
new file mode 100644
index 0000000000..8955ec5812
@@ -0,0 +1,8 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Grant Tables
+============
+
+.. doxygengroup:: grant_table

Why is this Arm64-specific?

This particular one is Arm64 specific, but it doesn’t mean that grant tables are arm specific, it is only that for now I’m Introducing a partial documentation in the arm side. Any other user can in the future add more documentation for each platform.

Quoted 38 lines
Quoted 38 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +7 -0
@@ -73,20 +75,25 @@
 *                           frame, or zero if none.
 *  3. Write memory barrier (WMB).
 *  4. Write ent->flags, inc. valid type.
+ * @endcode
 *
 * Invalidating an unused GTF_permit_access entry:
+ * @code
 *  1. flags = ent->flags.
 *  2. Observe that !(flags & (GTF_reading|GTF_writing)).
 *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
 *  NB. No need for WMB as reuse of entry is control-dependent on success of
 *      step 3, and all architectures guarantee ordering of ctrl-dep writes.
+ * @endcode
 *
 * Invalidating an in-use GTF_permit_access entry:
+ *
 *  This cannot be done directly. Request assistance from the domain controller
 *  which can set a timeout on the use of a grant entry and take necessary
 *  action. (NB. This is not yet implemented!).
 *
 * Invalidating an unused GTF_accept_transfer entry:
+ * @code
 *  1. flags = ent->flags.
 *  2. Observe that !(flags & GTF_transfer_committed). [*]
 *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
@@ -97,47 +104,55 @@
 *      transferred frame is written. It is safe for the guest to spin waiting
 *      for this to occur (detect by observing GTF_transfer_completed in
 *      ent->flags).
+ * @endcode
 *
 * Invalidating a committed GTF_accept_transfer entry:
 *  1. Wait for (ent->flags & GTF_transfer_completed).
 *
 * Changing a GTF_permit_access from writable to read-only:
+ *
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 118

Quoted 5 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -0
 *  Use SMP-safe CMPXCHG to set GTF_readonly, while checking !GTF_writing.
 *
 * Changing a GTF_permit_access from read-only to writable:
+ *
 *  Use SMP-safe bit-setting instruction.

For example - are the blank lines you add necessary or merely nice to have in your personal opinion?

The blank lines makes the docs output more good looking

Quoted 24 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 114

Quoted 7 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -6
- */
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 14 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -2
-
-/*
+ *
 * A grant table comprises a packed array of grant entries in one or more
 * page frames shared between Xen and a guest.
+ *
 * [XEN]: This field is written by Xen and read by the sharing guest.
+ *
 * [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables
+ * @{
 */
 
Quoted 9 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 134

Quoted 7 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +3 -3
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.
 */
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

Quoted 38 lines
Quoted 38 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +5 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
     * In that case, the frame field has the same semantics as the
     * field of the same name in the V1 entry structure.
     */
+    /** @cond skip anonymous struct/union for doxygen */
    struct {
        grant_entry_header_t hdr;
        uint32_t pad0;
        uint64_t frame;
    } full_page;
+    /** @endcond */
 
    /*
     * If the grant type is GTF_grant_access and GTF_sub_page is set,
     * @domid is allowed to access bytes [@page_off,@page_off+@length)
     * in frame @frame.
     */
+    /** @cond skip anonymous struct/union for doxygen */
    struct {
        grant_entry_header_t hdr;
        uint16_t page_off;
        uint16_t length;
        uint64_t frame;
    } sub_page;
+    /** @endcond */
 
    /*
     * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
     * The current version of Xen does not allow transitive grants
     * to be mapped.
     */
+    /** @cond skip anonymous struct/union for doxygen */
    struct {
        grant_entry_header_t hdr;
        domid_t trans_domid;
        uint16_t pad0;
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst union grant_entry_v2 { · line 299

Quoted 3 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -0
        grant_ref_t gref;
    } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Are you suggesting to remove the structure from docs?

Quoted 19 lines
Quoted 19 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -1
@@ -433,7 +454,12 @@ typedef struct gnttab_transfer gnttab_transfer_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 
 
-/*
+#define _GNTCOPY_source_gref      (0)
+#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
+#define _GNTCOPY_dest_gref        (1)
+#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
+
+/**
 * GNTTABOP_copy: Hypervisor based copy
 * source and destinations can be eithers MFNs or, for foreign domains,
 * grant references. the foreign domain has to grant read/write access
@@ -451,18 +477,15 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 * bytes to be copied.
 */
 
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t); · line 480

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -5
-#define _GNTCOPY_source_gref      (0)
-#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
-#define _GNTCOPY_dest_gref        (1)
-#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
-
 struct gnttab_copy {

Again the question - why the movement?

Doxygen takes the comment just above the data structure to build the docs, so here we are moving the Comment just on top of the described structure.

Quoted 25 lines
Quoted 11 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -1
@@ -579,17 +602,19 @@ struct gnttab_swap_grant_ref {
 typedef struct gnttab_swap_grant_ref gnttab_swap_grant_ref_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);
 
-/*
+/**
 * Issue one or more cache maintenance operations on a portion of a
 * page granted to the calling domain by a foreign domain.
 */
 struct gnttab_cache_flush {

docs/hypercall-interfaces/arm64/grant_tables.rst struct gnttab_swap_grant_ref { · line 617

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -2
+    /** @cond skip anonymous struct/union for doxygen */
    union {
        uint64_t dev_bus_addr;
        grant_ref_t ref;
    } a;
-    uint16_t offset; /* offset from start of grant */
-    uint16_t length; /* size within the grant */
+    /** @endcond */
+    uint16_t offset; /**< offset from start of grant */
+    uint16_t length; /**< size within the grant */

Skipping just part of a struct is perhaps even more confusing than omitting it altogether.

Quoted 1 line

Also, what's the significance of "/**<" ?

It is a doxygen pattern that tells it to use the comment as a field related documentation. If you build the documentation you will find the result, I encourage you to see it to realize the power of the tool and the benefits that Xen can get with it.

Cheers,
Luca

Quoted 2 lines

Jan

#6 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 18 lines · Luca Fancellu
Quoted 10 lines · Jan Beulich
Quoted 6 lines · Luca Fancellu

Modification to include/public/grant_table.h:

1) Add doxygen tags to:
- Create Grant tables section
- include variables in the generated documentation
2) Add .rst file for grant table for Arm64

Quoted 7 lines

I'm missing some reasoning about at least some of the changes done to grant_table.h. Looking at this and the earlier patches I also couldn't spot any general outline of what is acceptable or even necessary in such a header to be understood by doxygen. Without this written down somewhere (or, if documented elsewhere, a pointer provided to that doc) I'm afraid things might get screwed up again later on.

Quoted 6 lines

Doxygen is a tool that generates documentation based on parsing the source code comments, it recognises some commands in the comments and builds the documentation sticking to what you wrote.

Here the doxygen docs: https://www.doxygen.nl/manual/docblocks.html

Basically it doesn’t react to all comments, it parses only some well crafted comments as explained in its documentation.

Providing this link somewhere might be helpful. However, also seeing some of your further comments, it feels like to edit a public header enabled for doxygen one has to know fair parts of this documentation. While I'm certainly in favor of having a way to generate docs from the headers, I'm afraid I don't think such knowledge should become a prereq to e.g. adding a new sub-function of a hypercall. So far I was assuming that the formatting requirements would be quite limited, and that it would hence be possible to just glean them from existing code. But e.g. the "/**<" notation isn't going to be obvious to spot.

Quoted 25 lines
Quoted 9 lines
Quoted 9 lines
docs/hypercall-interfaces/arm64.rst +0 -0
@@ -8,6 +8,7 @@ Starting points

.. toctree::
:maxdepth: 2

+ arm64/grant_tables

Quoted 15 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 6

Quoted 13 lines

Functions

docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -0
new file mode 100644
index 0000000000..8955ec5812
@@ -0,0 +1,8 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Grant Tables
+============
+
+.. doxygengroup:: grant_table

Why is this Arm64-specific?

Quoted 3 lines

This particular one is Arm64 specific, but it doesn’t mean that grant tables are arm specific, it is only that for now I’m Introducing a partial documentation in the arm side. Any other user can in the future add more documentation for each platform.

I'm of the pretty strong opinion that common hypercalls should be documented as common, and hence not live in an arch-specific section.

Quoted 46 lines
Quoted 37 lines
Quoted 37 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +7 -0
@@ -73,20 +75,25 @@
 *                           frame, or zero if none.
 *  3. Write memory barrier (WMB).
 *  4. Write ent->flags, inc. valid type.
+ * @endcode
 *
 * Invalidating an unused GTF_permit_access entry:
+ * @code
 *  1. flags = ent->flags.
 *  2. Observe that !(flags & (GTF_reading|GTF_writing)).
 *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
 *  NB. No need for WMB as reuse of entry is control-dependent on success of
 *      step 3, and all architectures guarantee ordering of ctrl-dep writes.
+ * @endcode
 *
 * Invalidating an in-use GTF_permit_access entry:
+ *
 *  This cannot be done directly. Request assistance from the domain controller
 *  which can set a timeout on the use of a grant entry and take necessary
 *  action. (NB. This is not yet implemented!).
 *
 * Invalidating an unused GTF_accept_transfer entry:
+ * @code
 *  1. flags = ent->flags.
 *  2. Observe that !(flags & GTF_transfer_committed). [*]
 *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
@@ -97,47 +104,55 @@
 *      transferred frame is written. It is safe for the guest to spin waiting
 *      for this to occur (detect by observing GTF_transfer_completed in
 *      ent->flags).
+ * @endcode
 *
 * Invalidating a committed GTF_accept_transfer entry:
 *  1. Wait for (ent->flags & GTF_transfer_completed).
 *
 * Changing a GTF_permit_access from writable to read-only:
+ *
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 118

Quoted 5 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -0
 *  Use SMP-safe CMPXCHG to set GTF_readonly, while checking !GTF_writing.
 *
 * Changing a GTF_permit_access from read-only to writable:
+ *
 *  Use SMP-safe bit-setting instruction.

For example - are the blank lines you add necessary or merely nice to have in your personal opinion?

Quoted 1 line

The blank lines makes the docs output more good looking

I'm inclined to suggest to split beautification from basic enabling.

Quoted 33 lines
Quoted 23 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 114

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -6
- */
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 14 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -2
-
-/*
+ *
 * A grant table comprises a packed array of grant entries in one or more
 * page frames shared between Xen and a guest.
+ *
 * [XEN]: This field is written by Xen and read by the sharing guest.
+ *
 * [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables
+ * @{
 */
 
Quoted 9 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 134

Quoted 7 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +3 -3
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.
 */
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

Quoted 5 lines

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

But it's the other comment ("A grant table comprises ...") that I was asking about.

Quoted 46 lines
Quoted 37 lines
Quoted 37 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +5 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
     * In that case, the frame field has the same semantics as the
     * field of the same name in the V1 entry structure.
     */
+    /** @cond skip anonymous struct/union for doxygen */
    struct {
        grant_entry_header_t hdr;
        uint32_t pad0;
        uint64_t frame;
    } full_page;
+    /** @endcond */
 
    /*
     * If the grant type is GTF_grant_access and GTF_sub_page is set,
     * @domid is allowed to access bytes [@page_off,@page_off+@length)
     * in frame @frame.
     */
+    /** @cond skip anonymous struct/union for doxygen */
    struct {
        grant_entry_header_t hdr;
        uint16_t page_off;
        uint16_t length;
        uint64_t frame;
    } sub_page;
+    /** @endcond */
 
    /*
     * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
     * The current version of Xen does not allow transitive grants
     * to be mapped.
     */
+    /** @cond skip anonymous struct/union for doxygen */
    struct {
        grant_entry_header_t hdr;
        domid_t trans_domid;
        uint16_t pad0;
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst union grant_entry_v2 { · line 299

Quoted 3 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -0
        grant_ref_t gref;
    } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Quoted 1 line

Are you suggesting to remove the structure from docs?

Just yet I'm not suggesting anything here - I was merely guessing at the effect of "@cond" and the associated "skip ..." to mean that this construct makes doxygen skip the enclosed section. If that's not the effect, then maybe the comment wants rewording? (And then - what does @cond mean?)

Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -1
@@ -433,7 +454,12 @@ typedef struct gnttab_transfer gnttab_transfer_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 
 
-/*
+#define _GNTCOPY_source_gref      (0)
+#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
+#define _GNTCOPY_dest_gref        (1)
+#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
+
+/**
 * GNTTABOP_copy: Hypervisor based copy
 * source and destinations can be eithers MFNs or, for foreign domains,
 * grant references. the foreign domain has to grant read/write access
@@ -451,18 +477,15 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 * bytes to be copied.
 */
 
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t); · line 480

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -5
-#define _GNTCOPY_source_gref      (0)
-#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
-#define _GNTCOPY_dest_gref        (1)
-#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
-
 struct gnttab_copy {

Again the question - why the movement?

Quoted 2 lines

Doxygen takes the comment just above the data structure to build the docs, so here we are moving the Comment just on top of the described structure.

Well, okay, this then is an explanation _that_ the #define-s want moving, but not an explanation for where they got moved (father away from what they actually relate to). Personally I consider it good practice to have such #define-s next to the field they relate to (and we have such placement elsewhere). Perhaps worth considering to move them down rather than up?

Jan

Quoted 32 lines
Quoted 24 lines
Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -1
@@ -579,17 +602,19 @@ struct gnttab_swap_grant_ref {
 typedef struct gnttab_swap_grant_ref gnttab_swap_grant_ref_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);
 
-/*
+/**
 * Issue one or more cache maintenance operations on a portion of a
 * page granted to the calling domain by a foreign domain.
 */
 struct gnttab_cache_flush {

docs/hypercall-interfaces/arm64/grant_tables.rst struct gnttab_swap_grant_ref { · line 617

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -2
+    /** @cond skip anonymous struct/union for doxygen */
    union {
        uint64_t dev_bus_addr;
        grant_ref_t ref;
    } a;
-    uint16_t offset; /* offset from start of grant */
-    uint16_t length; /* size within the grant */
+    /** @endcond */
+    uint16_t offset; /**< offset from start of grant */
+    uint16_t length; /**< size within the grant */

Skipping just part of a struct is perhaps even more confusing than omitting it altogether.

Quoted 1 line

Also, what's the significance of "/**<" ?

It is a doxygen pattern that tells it to use the comment as a field related documentation. If you build the documentation you will find the result, I encourage you to see it to realize the power of the tool and the benefits that Xen can get with it.

Cheers,
Luca

#7 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 30 lines · Jan Beulich
Quoted 18 lines · Luca Fancellu
Quoted 10 lines · Jan Beulich
Quoted 6 lines · Luca Fancellu

Modification to include/public/grant_table.h:

1) Add doxygen tags to:
- Create Grant tables section
- include variables in the generated documentation
2) Add .rst file for grant table for Arm64

Quoted 7 lines

I'm missing some reasoning about at least some of the changes done to grant_table.h. Looking at this and the earlier patches I also couldn't spot any general outline of what is acceptable or even necessary in such a header to be understood by doxygen. Without this written down somewhere (or, if documented elsewhere, a pointer provided to that doc) I'm afraid things might get screwed up again later on.

Quoted 6 lines

Doxygen is a tool that generates documentation based on parsing the source code comments, it recognises some commands in the comments and builds the documentation sticking to what you wrote.

Here the doxygen docs: https://www.doxygen.nl/manual/docblocks.html

Basically it doesn’t react to all comments, it parses only some well crafted comments as explained in its documentation.

Quoted 9 lines

Providing this link somewhere might be helpful. However, also seeing some of your further comments, it feels like to edit a public header enabled for doxygen one has to know fair parts of this documentation. While I'm certainly in favor of having a way to generate docs from the headers, I'm afraid I don't think such knowledge should become a prereq to e.g. adding a new sub-function of a hypercall. So far I was assuming that the formatting requirements would be quite limited, and that it would hence be possible to just glean them from existing code. But e.g. the "/**<" notation isn't going to be obvious to spot.

Yes I see, some knowledge of the documentation tool is required but I’m sure that if anyone is able to put his/her hands on the xen code, he/she will be able to use some of the doxygen tag too, as you can see from this patch the usage is easy and some less obvious thing can be understood looking the source code and the corresponding generated doc page.

Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 17 lines
docs/hypercall-interfaces/arm64.rst +0 -0
@@ -8,6 +8,7 @@ Starting points

.. toctree::
:maxdepth: 2

+ arm64/grant_tables

Functions

docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -0
new file mode 100644
index 0000000000..8955ec5812

docs/hypercall-interfaces/arm64/grant_tables.rst line 6

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -0
@@ -0,0 +1,8 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Grant Tables
+============
+
+.. doxygengroup:: grant_table

Why is this Arm64-specific?

Quoted 7 lines
Quoted 3 lines

This particular one is Arm64 specific, but it doesn’t mean that grant tables are arm specific, it is only that for now I’m Introducing a partial documentation in the arm side. Any other user can in the future add more documentation for each platform.

I'm of the pretty strong opinion that common hypercalls should be documented as common, and hence not live in an arch-specific section.

Yes sure, I agree, but this is the first step to enable a kind of documentation, it’s a work in progress. On a future serie this can be modify. For now I’ve tried to reproduce this page: https://xenbits.xen.org/docs/unstable/
In the section “Hypercall Interfaces"

Quoted 47 lines
Quoted 47 lines
Quoted 46 lines
Quoted 33 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -0
@@ -73,20 +75,25 @@
 *                           frame, or zero if none.
 *  3. Write memory barrier (WMB).
 *  4. Write ent->flags, inc. valid type.
+ * @endcode
 *
 * Invalidating an unused GTF_permit_access entry:
+ * @code
 *  1. flags = ent->flags.
 *  2. Observe that !(flags & (GTF_reading|GTF_writing)).
 *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
 *  NB. No need for WMB as reuse of entry is control-dependent on success of
 *      step 3, and all architectures guarantee ordering of ctrl-dep writes.
+ * @endcode
 *
 * Invalidating an in-use GTF_permit_access entry:
+ *
 *  This cannot be done directly. Request assistance from the domain controller
 *  which can set a timeout on the use of a grant entry and take necessary
 *  action. (NB. This is not yet implemented!).
 *
 * Invalidating an unused GTF_accept_transfer entry:
+ * @code
 *  1. flags = ent->flags.
 *  2. Observe that !(flags & GTF_transfer_committed). [*]
 *  3. Check result of SMP-safe CMPXCHG(&ent->flags, flags, 0).
@@ -97,47 +104,55 @@
 *      transferred frame is written. It is safe for the guest to spin waiting
 *      for this to occur (detect by observing GTF_transfer_completed in
 *      ent->flags).
+ * @endcode
 *

docs/hypercall-interfaces/arm64/grant_tables.rst line 118

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +2 -0
 * Invalidating a committed GTF_accept_transfer entry:
 *  1. Wait for (ent->flags & GTF_transfer_completed).
 *
 * Changing a GTF_permit_access from writable to read-only:
+ *
 *  Use SMP-safe CMPXCHG to set GTF_readonly, while checking !GTF_writing.
 *
 * Changing a GTF_permit_access from read-only to writable:
+ *
 *  Use SMP-safe bit-setting instruction.

For example - are the blank lines you add necessary or merely nice to have in your personal opinion?

Quoted 3 lines
Quoted 1 line

The blank lines makes the docs output more good looking

I'm inclined to suggest to split beautification from basic enabling.

I will take this into consideration

Quoted 34 lines
Quoted 34 lines
Quoted 33 lines

docs/hypercall-interfaces/arm64/grant_tables.rst line 114

Quoted 7 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -6
- */
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 11 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +5 -2
-
-/*
+ *
 * A grant table comprises a packed array of grant entries in one or more
 * page frames shared between Xen and a guest.
+ *
 * [XEN]: This field is written by Xen and read by the sharing guest.
+ *
 * [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables

docs/hypercall-interfaces/arm64/grant_tables.rst line 134

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -3
+ * @{
 */
 
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.
 */
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

Quoted 8 lines
Quoted 5 lines

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

But it's the other comment ("A grant table comprises ...") that I was asking about.

I thought it was part of the brief about grant tables, am I wrong? For that reason I moved it.

Quoted 47 lines
Quoted 47 lines
Quoted 46 lines
Quoted 31 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
    * In that case, the frame field has the same semantics as the
    * field of the same name in the V1 entry structure.
    */
+    /** @cond skip anonymous struct/union for doxygen */
   struct {
       grant_entry_header_t hdr;
       uint32_t pad0;
       uint64_t frame;
   } full_page;
+    /** @endcond */
 
   /*
    * If the grant type is GTF_grant_access and GTF_sub_page is set,
    * @domid is allowed to access bytes [@page_off,@page_off+@length)
    * in frame @frame.
    */
+    /** @cond skip anonymous struct/union for doxygen */
   struct {
       grant_entry_header_t hdr;
       uint16_t page_off;
       uint16_t length;
       uint64_t frame;
   } sub_page;
+    /** @endcond */
 
   /*
    * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
    * The current version of Xen does not allow transitive grants

docs/hypercall-interfaces/arm64/grant_tables.rst union grant_entry_v2 { · line 299

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +2 -0
    * to be mapped.
    */
+    /** @cond skip anonymous struct/union for doxygen */
   struct {
       grant_entry_header_t hdr;
       domid_t trans_domid;
       uint16_t pad0;
       grant_ref_t gref;
   } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Quoted 7 lines
Quoted 1 line

Are you suggesting to remove the structure from docs?

Just yet I'm not suggesting anything here - I was merely guessing at the effect of "@cond" and the associated "skip ..." to mean that this construct makes doxygen skip the enclosed section. If that's not the effect, then maybe the comment wants rewording? (And then - what does @cond mean?)

Yes you were right, in the documentation the structure grant_entry_v2 won’t display the anonymous union.

Quoted 31 lines
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +6 -1
@@ -433,7 +454,12 @@ typedef struct gnttab_transfer gnttab_transfer_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 
 
-/*
+#define _GNTCOPY_source_gref      (0)
+#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
+#define _GNTCOPY_dest_gref        (1)
+#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
+
+/**
 * GNTTABOP_copy: Hypervisor based copy
 * source and destinations can be eithers MFNs or, for foreign domains,
 * grant references. the foreign domain has to grant read/write access
@@ -451,18 +477,15 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t);
 * bytes to be copied.
 */
 
Quoted 8 lines

docs/hypercall-interfaces/arm64/grant_tables.rst DEFINE_XEN_GUEST_HANDLE(gnttab_transfer_t); · line 480

Quoted 6 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +0 -5
-#define _GNTCOPY_source_gref      (0)
-#define GNTCOPY_source_gref       (1<<_GNTCOPY_source_gref)
-#define _GNTCOPY_dest_gref        (1)
-#define GNTCOPY_dest_gref         (1<<_GNTCOPY_dest_gref)
-
 struct gnttab_copy {

Again the question - why the movement?

Quoted 2 lines

Doxygen takes the comment just above the data structure to build the docs, so here we are moving the Comment just on top of the described structure.

Quoted 6 lines

Well, okay, this then is an explanation _that_ the #define-s want moving, but not an explanation for where they got moved (father away from what they actually relate to). Personally I consider it good practice to have such #define-s next to the field they relate to (and we have such placement elsewhere). Perhaps worth considering to move them down rather than up?

Sure they can be moved down without problem.

Quoted 35 lines

Jan

Quoted 32 lines
Quoted 24 lines
Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +1 -1
@@ -579,17 +602,19 @@ struct gnttab_swap_grant_ref {
 typedef struct gnttab_swap_grant_ref gnttab_swap_grant_ref_t;
 DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);
 
-/*
+/**
 * Issue one or more cache maintenance operations on a portion of a
 * page granted to the calling domain by a foreign domain.
 */
 struct gnttab_cache_flush {

docs/hypercall-interfaces/arm64/grant_tables.rst struct gnttab_swap_grant_ref { · line 617

Quoted 10 lines
docs/hypercall-interfaces/arm64/grant_tables.rst new file +4 -2
+    /** @cond skip anonymous struct/union for doxygen */
   union {
       uint64_t dev_bus_addr;
       grant_ref_t ref;
   } a;
-    uint16_t offset; /* offset from start of grant */
-    uint16_t length; /* size within the grant */
+    /** @endcond */
+    uint16_t offset; /**< offset from start of grant */
+    uint16_t length; /**< size within the grant */

Skipping just part of a struct is perhaps even more confusing than omitting it altogether.

Quoted 1 line

Also, what's the significance of "/**<" ?

It is a doxygen pattern that tells it to use the comment as a field related documentation. If you build the documentation you will find the result, I encourage you to see it to realize the power of the tool and the benefits that Xen can get with it.

Cheers,
Luca

#8 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 40 lines · Luca Fancellu
Quoted 40 lines · Jan Beulich
Quoted 38 lines · Luca Fancellu
Quoted 25 lines · Jan Beulich

xen/include/public/grant_table.h line 114

Quoted 6 lines · Luca Fancellu

- */

xen/include/public/grant_table.h +0 -5
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 14 lines
xen/include/public/grant_table.h +1 -2
-
-/*
+ *

* A grant table comprises a packed array of grant entries in one or more
* page frames shared between Xen and a guest.
+ *
* [XEN]: This field is written by Xen and read by the sharing guest.
+ *
* [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables
+ * @{
*/

Quoted 9 lines
Quoted 7 lines
xen/include/public/grant_table.h +2 -3
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.

*/
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

But it's the other comment ("A grant table comprises ...") that I was asking about.

I thought it was part of the brief about grant tables, am I wrong? For that reason I moved it.

Well, the comment talks about grant table entries (the layout of which gets defined immediately below, as visible in the original patch), not grant references.

Quoted 48 lines
Quoted 48 lines
Quoted 47 lines
Quoted 37 lines
Quoted 37 lines
xen/include/public/grant_table.h +5 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
    * In that case, the frame field has the same semantics as the
    * field of the same name in the V1 entry structure.
    */
+    /** @cond skip anonymous struct/union for doxygen */
   struct {
       grant_entry_header_t hdr;
       uint32_t pad0;
       uint64_t frame;
   } full_page;
+    /** @endcond */
 
   /*
    * If the grant type is GTF_grant_access and GTF_sub_page is set,
    * @domid is allowed to access bytes [@page_off,@page_off+@length)
    * in frame @frame.
    */
+    /** @cond skip anonymous struct/union for doxygen */
   struct {
       grant_entry_header_t hdr;
       uint16_t page_off;
       uint16_t length;
       uint64_t frame;
   } sub_page;
+    /** @endcond */
 
   /*
    * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
    * The current version of Xen does not allow transitive grants
    * to be mapped.
    */
+    /** @cond skip anonymous struct/union for doxygen */
   struct {
       grant_entry_header_t hdr;
       domid_t trans_domid;
       uint16_t pad0;
Quoted 8 lines

xen/include/public/grant_table.h union grant_entry_v2 { · line 299

Quoted 3 lines
xen/include/public/grant_table.h +1 -0
       grant_ref_t gref;
   } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Are you suggesting to remove the structure from docs?

Quoted 8 lines
Quoted 5 lines

Just yet I'm not suggesting anything here - I was merely guessing at the effect of "@cond" and the associated "skip ..." to mean that this construct makes doxygen skip the enclosed section. If that's not the effect, then maybe the comment wants rewording? (And then - what does @cond mean?)

Yes you were right, in the documentation the structure grant_entry_v2 won’t display the anonymous union.

In which case I'm now completely unclear about your prior question.

Jan

#9 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 48 lines · Jan Beulich
Quoted 43 lines · Luca Fancellu
Quoted 43 lines · Jan Beulich
Quoted 40 lines · Luca Fancellu
Quoted 25 lines · Jan Beulich

xen/include/public/grant_table.h line 114

Quoted 6 lines · Luca Fancellu

- */

xen/include/public/grant_table.h +0 -5
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 14 lines
xen/include/public/grant_table.h +1 -2
-
-/*
+ *

* A grant table comprises a packed array of grant entries in one or more
* page frames shared between Xen and a guest.
+ *
* [XEN]: This field is written by Xen and read by the sharing guest.
+ *
* [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables
+ * @{
*/

Quoted 9 lines
Quoted 7 lines
xen/include/public/grant_table.h +2 -3
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.

*/
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

But it's the other comment ("A grant table comprises ...") that I was asking about.

I thought it was part of the brief about grant tables, am I wrong? For that reason I moved it.

Well, the comment talks about grant table entries (the layout of which gets defined immediately below, as visible in the original patch), not grant references.

Hi Jan,

Of course this can be reverted back if it is wrong.

I’ve prepared a page with the output of all my commits (some of them are not yet in the ML), so anyone can have a look on what is the output and how can sphinx+doxygen improve the quality of the documentation.

Here:

https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/arm64.html

the document Is trying to reproduce this other one from the current docs:

https://xenbits.xen.org/docs/unstable/hypercall/arm/index.html

You will receive a warning from the browser because the gitlab certificate has problems when the username has a dot in it.

Quoted 55 lines
Quoted 55 lines
Quoted 54 lines
Quoted 47 lines
Quoted 38 lines
Quoted 38 lines
xen/include/public/grant_table.h +5 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
   * In that case, the frame field has the same semantics as the
   * field of the same name in the V1 entry structure.
   */
+    /** @cond skip anonymous struct/union for doxygen */
  struct {
      grant_entry_header_t hdr;
      uint32_t pad0;
      uint64_t frame;
  } full_page;
+    /** @endcond */
 
  /*
   * If the grant type is GTF_grant_access and GTF_sub_page is set,
   * @domid is allowed to access bytes [@page_off,@page_off+@length)
   * in frame @frame.
   */
+    /** @cond skip anonymous struct/union for doxygen */
  struct {
      grant_entry_header_t hdr;
      uint16_t page_off;
      uint16_t length;
      uint64_t frame;
  } sub_page;
+    /** @endcond */
 
  /*
   * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
   * The current version of Xen does not allow transitive grants
   * to be mapped.
   */
+    /** @cond skip anonymous struct/union for doxygen */
  struct {
      grant_entry_header_t hdr;
      domid_t trans_domid;
      uint16_t pad0;
Quoted 8 lines

xen/include/public/grant_table.h union grant_entry_v2 { · line 299

Quoted 3 lines
xen/include/public/grant_table.h +1 -0
      grant_ref_t gref;
  } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Quoted 1 line

Are you suggesting to remove the structure from docs?

Just yet I'm not suggesting anything here - I was merely guessing at the effect of "@cond" and the associated "skip ..." to mean that this construct makes doxygen skip the enclosed section. If that's not the effect, then maybe the comment wants rewording? (And then - what does @cond mean?)

Quoted 4 lines
Quoted 2 lines

Yes you were right, in the documentation the structure grant_entry_v2 won’t display the anonymous union.

In which case I'm now completely unclear about your prior question.

We have to choose just if the struct is useful even if it’s partially documented or if it’s better to hide it completely from the docs since it can’t be fully documented.

Cheers,
Luca

Quoted 2 lines

Jan

#10 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 58 lines · Luca Fancellu
Quoted 45 lines · Jan Beulich
Quoted 43 lines · Luca Fancellu
Quoted 43 lines · Jan Beulich
Quoted 40 lines · Luca Fancellu
Quoted 25 lines · Jan Beulich

Review

Quoted 6 lines · Luca Fancellu

- */

(quoted diff) +0 -5
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 14 lines
(quoted diff) +1 -2
-
-/*
+ *

* A grant table comprises a packed array of grant entries in one or more
* page frames shared between Xen and a guest.
+ *
* [XEN]: This field is written by Xen and read by the sharing guest.
+ *
* [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables
+ * @{
*/

Quoted 9 lines
Quoted 7 lines
(quoted diff) +2 -3
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.

*/
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

But it's the other comment ("A grant table comprises ...") that I was asking about.

I thought it was part of the brief about grant tables, am I wrong? For that reason I moved it.

Well, the comment talks about grant table entries (the layout of which gets defined immediately below, as visible in the original patch), not grant references.

Hi Jan,

Of course this can be reverted back if it is wrong.

Quoted 7 lines

I’ve prepared a page with the output of all my commits (some of them are not yet in the ML), so anyone can have a look on what is the output and how can sphinx+doxygen improve the quality of the documentation.

Here:

https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/arm64.html

The doc looks fine in this regard, but the C header should remain properly ordered as well.

Quoted 54 lines
Quoted 54 lines
Quoted 54 lines
Quoted 53 lines
Quoted 46 lines
Quoted 37 lines
Quoted 37 lines
(quoted diff) +5 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
   * In that case, the frame field has the same semantics as the
   * field of the same name in the V1 entry structure.
   */
+    /** @cond skip anonymous struct/union for doxygen */
  struct {
      grant_entry_header_t hdr;
      uint32_t pad0;
      uint64_t frame;
  } full_page;
+    /** @endcond */
 
  /*
   * If the grant type is GTF_grant_access and GTF_sub_page is set,
   * @domid is allowed to access bytes [@page_off,@page_off+@length)
   * in frame @frame.
   */
+    /** @cond skip anonymous struct/union for doxygen */
  struct {
      grant_entry_header_t hdr;
      uint16_t page_off;
      uint16_t length;
      uint64_t frame;
  } sub_page;
+    /** @endcond */
 
  /*
   * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
   * The current version of Xen does not allow transitive grants
   * to be mapped.
   */
+    /** @cond skip anonymous struct/union for doxygen */
  struct {
      grant_entry_header_t hdr;
      domid_t trans_domid;
      uint16_t pad0;
Quoted 8 lines

Review union grant_entry_v2 { · line 299

Quoted 3 lines
(quoted diff) +1 -0
      grant_ref_t gref;
  } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Quoted 1 line

Are you suggesting to remove the structure from docs?

Just yet I'm not suggesting anything here - I was merely guessing at the effect of "@cond" and the associated "skip ..." to mean that this construct makes doxygen skip the enclosed section. If that's not the effect, then maybe the comment wants rewording? (And then - what does @cond mean?)

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Yes you were right, in the documentation the structure grant_entry_v2 won’t display the anonymous union.

In which case I'm now completely unclear about your prior question.

We have to choose just if the struct is useful even if it’s partially documented or if it’s better to hide it completely from the docs since it can’t be fully documented.

I've never suggested to hide it completely (aiui doing so would mean widening the scope of the @cond, i.e. there would still be extra clutter). Instead I was trying to suggest to make sure the struct gets fully documented despite the absence of struct tags.

Jan

#11 lore raw

Re: [PATCH v2 3/3] docs/doxygen: doxygen documentation for grant_table.h

Quoted 67 lines · Jan Beulich
Quoted 62 lines · Luca Fancellu
Quoted 45 lines · Jan Beulich
Quoted 43 lines · Luca Fancellu
Quoted 43 lines · Jan Beulich
Quoted 40 lines · Luca Fancellu
Quoted 25 lines · Jan Beulich

Review

Quoted 6 lines · Luca Fancellu

- */

(quoted diff) +0 -5
-
-/*
- * Reference to a grant entry in a specified domain's grant table.
- */
-typedef uint32_t grant_ref_t;

Why does this get moved ...

Quoted 14 lines
(quoted diff) +1 -2
-
-/*
+ *

* A grant table comprises a packed array of grant entries in one or more
* page frames shared between Xen and a guest.
+ *
* [XEN]: This field is written by Xen and read by the sharing guest.
+ *
* [GST]: This field is written by the guest and read by Xen.
+ *
+ * @addtogroup grant_table Grant Tables
+ * @{
*/

Quoted 9 lines
Quoted 7 lines
(quoted diff) +2 -3
-/*
- * Version 1 of the grant table entry structure is maintained purely
- * for backwards compatibility.  New guests should use version 2.
+/**
+ * Reference to a grant entry in a specified domain's grant table.

*/
+typedef uint32_t grant_ref_t;

... here, past a comment unrelated to it?

The comment “Version 1 of the grant table entry […]” was moved on top of the struct grant_entry_v1, in this way Doxygen associate the comment to that structure.

The comment “Reference to a grant entry in a specified domain's grant table.” Is moved on top of typedef uint32_t grant_ref_t for the same reason above

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

But it's the other comment ("A grant table comprises ...") that I was asking about.

I thought it was part of the brief about grant tables, am I wrong? For that reason I moved it.

Well, the comment talks about grant table entries (the layout of which gets defined immediately below, as visible in the original patch), not grant references.

Hi Jan,

Of course this can be reverted back if it is wrong.

I’ve prepared a page with the output of all my commits (some of them are not yet in the ML), so anyone can have a look on what is the output and how can sphinx+doxygen improve the quality of the documentation.

Quoted 6 lines
Quoted 3 lines

Here:

https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/arm64.html

The doc looks fine in this regard, but the C header should remain properly ordered as well.

Hi Jan,

I’ve pushed outside the v3 addressing your comment

Quoted 60 lines
Quoted 60 lines
Quoted 59 lines
Quoted 49 lines
Quoted 49 lines
Quoted 48 lines
Quoted 38 lines
Quoted 38 lines
(quoted diff) +5 -0
@@ -243,23 +258,27 @@ union grant_entry_v2 {
  * In that case, the frame field has the same semantics as the
  * field of the same name in the V1 entry structure.
  */
+    /** @cond skip anonymous struct/union for doxygen */
 struct {
     grant_entry_header_t hdr;
     uint32_t pad0;
     uint64_t frame;
 } full_page;
+    /** @endcond */
 
 /*
  * If the grant type is GTF_grant_access and GTF_sub_page is set,
  * @domid is allowed to access bytes [@page_off,@page_off+@length)
  * in frame @frame.
  */
+    /** @cond skip anonymous struct/union for doxygen */
 struct {
     grant_entry_header_t hdr;
     uint16_t page_off;
     uint16_t length;
     uint64_t frame;
 } sub_page;
+    /** @endcond */
 
 /*
  * If the grant is GTF_transitive, @domid is allowed to use the
@@ -270,12 +289,14 @@ union grant_entry_v2 {
  * The current version of Xen does not allow transitive grants
  * to be mapped.
  */
+    /** @cond skip anonymous struct/union for doxygen */
 struct {
     grant_entry_header_t hdr;
     domid_t trans_domid;
     uint16_t pad0;
Quoted 8 lines

Review union grant_entry_v2 { · line 299

Quoted 3 lines
(quoted diff) +1 -0
     grant_ref_t gref;
 } transitive;
+    /** @endcond */

While already better than the introduction of strange struct tags, I'm still not convinced we want this extra clutter (sorry). Plus - don't these additions mean the sub-structures then won't be represented in the generated doc, rendering it (partly) useless?

Are you suggesting to remove the structure from docs?

Quoted 8 lines
Quoted 5 lines

Just yet I'm not suggesting anything here - I was merely guessing at the effect of "@cond" and the associated "skip ..." to mean that this construct makes doxygen skip the enclosed section. If that's not the effect, then maybe the comment wants rewording? (And then - what does @cond mean?)

Yes you were right, in the documentation the structure grant_entry_v2 won’t display the anonymous union.

In which case I'm now completely unclear about your prior question.

Quoted 7 lines
Quoted 2 lines

We have to choose just if the struct is useful even if it’s partially documented or if it’s better to hide it completely from the docs since it can’t be fully documented.

I've never suggested to hide it completely (aiui doing so would mean widening the scope of the @cond, i.e. there would still be extra clutter). Instead I was trying to suggest to make sure the struct gets fully documented despite the absence of struct tags.

In the v3 I preprocess the header to give a name to the anonymous struct so that it appears in the docs without touching the header.

Cheers,
Luca

Quoted 2 lines

Jan