Use Doxygen and sphinx for html documentation
cover Use Doxygen and sphinx for html documentation
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):
./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-html
now in docs/sphinx/html/ we have the generated docs starting from the index.html page.
Luca Fancellu (3):
docs: add doxygen support for html documentation
docs: hypercalls sphinx skeleton for generated html docs/doxygen: doxygen documentation for grant_table.h
.gitignore | 7 + config/Docs.mk.in | 2 + docs/Makefile | 51 +- docs/conf.py | 48 +- docs/configure | 258 ++ docs/configure.ac | 15 + docs/hypercall-interfaces/arm32.rst | 4 + docs/hypercall-interfaces/arm64.rst | 33 + .../arm64/grant_tables.rst | 8 + docs/hypercall-interfaces/index.rst.in | 7 + docs/hypercall-interfaces/x86_64.rst | 4 + docs/index.rst | 8 + docs/xen-doxygen/customdoxygen.css | 36 + docs/xen-doxygen/doxy_input.list | 1 + docs/xen-doxygen/doxy_input_template.in | 21 + docs/xen-doxygen/footer.html | 21 + docs/xen-doxygen/header.html | 56 + docs/xen-doxygen/mainpage.md | 5 + docs/xen-doxygen/xen_project_logo_165x67.png | Bin 0 -> 18223 bytes docs/xen.doxyfile.in | 2314 +++++++++++++++++ m4/ax_python_module.m4 | 56 + m4/docs_tool.m4 | 9 + xen/include/public/grant_table.h | 79 +- 23 files changed, 3007 insertions(+), 36 deletions(-) create mode 100644 docs/hypercall-interfaces/arm32.rst create mode 100644 docs/hypercall-interfaces/arm64.rst create mode 100644 docs/hypercall-interfaces/arm64/grant_tables.rst create mode 100644 docs/hypercall-interfaces/index.rst.in create mode 100644 docs/hypercall-interfaces/x86_64.rst create mode 100644 docs/xen-doxygen/customdoxygen.css create mode 100644 docs/xen-doxygen/doxy_input.list create mode 100644 docs/xen-doxygen/doxy_input_template.in create mode 100644 docs/xen-doxygen/footer.html create mode 100644 docs/xen-doxygen/header.html create mode 100644 docs/xen-doxygen/mainpage.md create mode 100644 docs/xen-doxygen/xen_project_logo_165x67.png create mode 100644 docs/xen.doxyfile.in create mode 100644 m4/ax_python_module.m4
Signature · 1 line
2.17.1
PATCH 1/3 docs: add doxygen support for html documentation
Add doxygen support to build html documentation with sphinx, to do that the following modification are applied:
1) Modify docs/configure.ac and consequently configure script to check, through the ./configure script, the presence in the system of the sphinx-build binary, if it is found then it checks also the presence of the doxygen binary, breathe and sphinx-rtd-theme python packages.
Doxygen and the packages are required only if sphinx-build is found, otherwise the Makefile will simply skip the sphinx html generation.
The ax_python_module.m4 support is needed to check for python packages.
2) Add doxygen templates and configuration file
3) Modify docs/Makefile to call in sequence doxygen and sphinx-build, the doxygen configuration file will be modified to include the xen absolute path and
a list of folder to check.
A doxygen_input.h file is generated to include every header listed in the doxy_input.list file.
--- .gitignore | 6 + config/Docs.mk.in | 2 + docs/Makefile | 47 +- docs/conf.py | 48 +- docs/configure | 258 ++ docs/configure.ac | 15 + docs/xen-doxygen/customdoxygen.css | 36 + docs/xen-doxygen/doxy_input.list | 0 docs/xen-doxygen/doxy_input_template.in | 21 + docs/xen-doxygen/footer.html | 21 + docs/xen-doxygen/header.html | 56 + docs/xen-doxygen/mainpage.md | 5 + docs/xen-doxygen/xen_project_logo_165x67.png | Bin 0 -> 18223 bytes docs/xen.doxyfile.in | 2314 ++++++++++++++++++ m4/ax_python_module.m4 | 56 + m4/docs_tool.m4 | 9 + 16 files changed, 2888 insertions(+), 6 deletions(-) create mode 100644 docs/xen-doxygen/customdoxygen.css create mode 100644 docs/xen-doxygen/doxy_input.list create mode 100644 docs/xen-doxygen/doxy_input_template.in create mode 100644 docs/xen-doxygen/footer.html create mode 100644 docs/xen-doxygen/header.html create mode 100644 docs/xen-doxygen/mainpage.md create mode 100644 docs/xen-doxygen/xen_project_logo_165x67.png create mode 100644 docs/xen.doxyfile.in create mode 100644 m4/ax_python_module.m4
.gitignore +6 -0
@@ -58,6 +58,12 @@ docs/man7/ docs/man8/ docs/pdf/ docs/txt/ +docs/doxygen-output +docs/sphinx +docs/xen.doxyfile +docs/xen.doxyfile.tmp +docs/doxygen_input.h +docs/doxygen_input.h.tmp extras/mini-os* install/* stubdom/*-minios-config.mk
config/Docs.mk.in +2 -0
@@ -7,3 +7,5 @@ POD2HTML := @POD2HTML@ POD2TEXT := @POD2TEXT@ PANDOC := @PANDOC@ PERL := @PERL@ +SPHINXBUILD := @SPHINXBUILD@ +DOXYGEN := @DOXYGEN@
docs/Makefile +44 -3
65 lines not on this page · raw message
docs/conf.py +45 -3
83 lines not on this page · raw message
docs/configure +258 -0
294 lines not on this page · raw message
docs/configure.ac +15 -0
@@ -20,6 +20,7 @@ m4_include([../m4/docs_tool.m4]) m4_include([../m4/path_or_fail.m4]) m4_include([../m4/features.m4]) m4_include([../m4/paths.m4]) +m4_include([../m4/ax_python_module.m4]) AX_XEN_EXPAND_CONFIG()
@@ -29,6 +30,20 @@ AX_DOCS_TOOL_PROG([POD2HTML], [pod2html]) AX_DOCS_TOOL_PROG([POD2TEXT], [pod2text]) AX_DOCS_TOOL_PROG([PANDOC], [pandoc]) +# If sphinx is installed, make sure to have also the dependencies to build +# Sphinx documentation. +AC_CHECK_PROGS([SPHINXBUILD], [sphinx-build], [no]) + AS_IF([test "x$SPHINXBUILD" = xno], + [AC_MSG_WARN(sphinx-build is not available so sphinx documentation \ +won't be built)], + [ + AC_PATH_PROG([SPHINXBUILD], [sphinx-build]) + AX_DOCS_TOOL_REQ_PROG([DOXYGEN], [doxygen]) + AX_PYTHON_MODULE([breathe],[yes]) + AX_PYTHON_MODULE([sphinx_rtd_theme], [yes]) + ] + ) + AC_ARG_VAR([PERL], [Path to Perl parser]) AX_PATH_PROG_OR_FAIL([PERL], [perl])
docs/xen-doxygen/customdoxygen.css new file +36 -0
@@ -0,0 +1,36 @@ +/* Custom CSS for Doxygen-generated HTML + * Copyright (c) 2015 Intel Corporation + * SPDX-License-Identifier: Apache-2.0 + */ + +code { + font-family: Monaco,Menlo,Consolas,"Courier New",monospace; + background-color: #D8D8D8; + padding: 0 0.25em 0 0.25em; +} + +pre.fragment { + display: block; + font-family: Monaco,Menlo,Consolas,"Courier New",monospace; + padding: 1rem; + word-break: break-all; + word-wrap: break-word; + white-space: pre; + background-color: #D8D8D8; +} + +#projectlogo +{ + vertical-align: middle; +} + +#projectname +{ + font: 200% Tahoma, Arial,sans-serif; + color: #3D578C; +} + +#projectbrief +{ + color: #3D578C; +}
docs/xen-doxygen/doxy_input.list new file +0 -0
docs/xen-doxygen/doxy_input_template.in new file +21 -0
@@ -0,0 +1,21 @@ +#define __XEN__ + +#include "@XEN_BASE@/xen/include/xen/config.h" + +#if defined(CONFIG_X86_64) + +#define __x86_64__ 1 + +#elif defined(CONFIG_ARM_64) + +#define __aarch64__ 1 + +#elif defined(CONFIG_ARM_32) + +#define __arm__ 1 + +#else + +#error Architecture not supported/recognized. + +#endif
docs/xen-doxygen/footer.html new file +21 -0
@@ -0,0 +1,21 @@ +<!-- HTML footer for doxygen 1.8.13--> +<!-- start footer part --> +<!--BEGIN GENERATE_TREEVIEW--> +<div id="nav-path" class="navpath"><!-- id is needed for treeview function! --> + <ul> + $navpath + <li class="footer">$generatedby + <a href="http://www.doxygen.org/index.html"> + <img class="footer" src="$relpath^doxygen.png" alt="doxygen"/></a> $doxygenversion </li> + </ul> +</div> +<!--END GENERATE_TREEVIEW--> +<!--BEGIN !GENERATE_TREEVIEW--> +<hr class="footer"/><address class="footer"><small> +$generatedby  <a href="http://www.doxygen.org/index.html"> +<img class="footer" src="$relpath^doxygen.png" alt="doxygen"/> +</a> $doxygenversion +</small></address> +<!--END !GENERATE_TREEVIEW--> +</body> +</html>
docs/xen-doxygen/header.html new file +56 -0
56 lines not on this page · raw message
docs/xen-doxygen/mainpage.md new file +5 -0
@@ -0,0 +1,5 @@ +# API Documentation {#index} + +## Introduction + +## Licensing
docs/xen-doxygen/xen_project_logo_165x67.png new file binary +0 -0
docs/xen.doxyfile.in new file +2314 -0
2,314 lines not on this page · raw message
m4/ax_python_module.m4 new file +56 -0
57 lines not on this page · raw message
m4/docs_tool.m4 +9 -0
@@ -15,3 +15,12 @@ dnl AC_MSG_WARN([$2 is not available so some documentation won't be built]) ]) ]) +
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
PATCH 2/3 docs: hypercalls sphinx skeleton for generated html
In reply to Luca FancelluCreate a skeleton for the documentation about hypercalls
--- .gitignore | 1 + docs/Makefile | 4 ++++ docs/hypercall-interfaces/arm32.rst | 4 ++++ docs/hypercall-interfaces/arm64.rst | 32 ++++++++++++++++++++++++++ docs/hypercall-interfaces/index.rst.in | 7 ++++++ docs/hypercall-interfaces/x86_64.rst | 4 ++++ docs/index.rst | 8 +++++++ 7 files changed, 60 insertions(+) create mode 100644 docs/hypercall-interfaces/arm32.rst create mode 100644 docs/hypercall-interfaces/arm64.rst create mode 100644 docs/hypercall-interfaces/index.rst.in create mode 100644 docs/hypercall-interfaces/x86_64.rst
.gitignore +1 -0
@@ -64,6 +64,7 @@ docs/xen.doxyfile docs/xen.doxyfile.tmp docs/doxygen_input.h docs/doxygen_input.h.tmp +docs/hypercall-interfaces/index.rst extras/mini-os* install/* stubdom/*-minios-config.mk
docs/Makefile +4 -0
@@ -65,6 +65,9 @@ build: html txt pdf man-pages figs sphinx-html: $(DOXY_DEPS) $(DOXY_LIST_SOURCES) ifneq ($(SPHINXBUILD),) $(DOXYGEN) xen.doxyfile + @echo "Generating hypercall-interfaces/index.rst" + @sed -e "s,@XEN_TARGET_ARCH@,$(XEN_TARGET_ARCH),g" \ + hypercall-interfaces/index.rst.in > hypercall-interfaces/index.rst XEN_ROOT=$(realpath $(XEN_ROOT)) $(SPHINXBUILD) -b html . sphinx/html else @echo "Sphinx is not installed; skipping sphinx-html documentation."
@@ -113,6 +116,7 @@ clean: clean-man-pages rm -f xen.doxyfile.tmp rm -f doxygen_input.h rm -f doxygen_input.h.tmp + rm -f hypercall-interfaces/index.rst .PHONY: distclean distclean: clean
docs/hypercall-interfaces/arm32.rst new file +4 -0
@@ -0,0 +1,4 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Hypercall Interfaces - arm32 +============================
docs/hypercall-interfaces/arm64.rst new file +32 -0
@@ -0,0 +1,32 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Hypercall Interfaces - arm64 +============================ + +Starting points +--------------- +.. toctree:: + :maxdepth: 2 + + + +Functions +--------- + + +Structs +------- + + +Enums and sets of #defines +-------------------------- + + +Typedefs +-------- + + +Enum values and individual #defines +----------------------------------- + +
docs/hypercall-interfaces/index.rst.in new file +7 -0
@@ -0,0 +1,7 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Hypercall Interfaces +==================== + +.. toctree:: + @XEN_TARGET_ARCH@
docs/hypercall-interfaces/x86_64.rst new file +4 -0
@@ -0,0 +1,4 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Hypercall Interfaces - x86_64 +=============================
docs/index.rst +8 -0
@@ -53,6 +53,14 @@ kind of development environment. hypervisor-guide/index +Hypercall Interfaces documentation +---------------------------------- + +.. toctree:: + :maxdepth: 2 + + hypercall-interfaces/index + Miscellanea -----------
Signature · 1 line
2.17.1
PATCH 3/3 docs/doxygen: doxygen documentation for grant_table.h
In reply to Luca FancelluModification to include/public/grant_table.h:
1) Change anonymous structure to be named structure, because doxygen can't deal with them.
2) Add doxygen tags to:
- Create Grant tables section
- include variables in the generated documentation
3) Add .rst file for grant table for Arm64
--- docs/hypercall-interfaces/arm64.rst | 1 + .../arm64/grant_tables.rst | 8 ++ docs/xen-doxygen/doxy_input.list | 1 + xen/include/public/grant_table.h | 79 ++++++++++++------- 4 files changed, 59 insertions(+), 30 deletions(-) create mode 100644 docs/hypercall-interfaces/arm64/grant_tables.rst
docs/hypercall-interfaces/arm64.rst +1 -0
@@ -8,6 +8,7 @@ Starting points .. toctree:: :maxdepth: 2 + arm64/grant_tables Functions
docs/hypercall-interfaces/arm64/grant_tables.rst new file +8 -0
@@ -0,0 +1,8 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Grant Tables +============ + +.. doxygengroup:: grant_table + :project: Xen + :members:
docs/xen-doxygen/doxy_input.list +1 -0
@@ -0,0 +1 @@ +xen/include/public/grant_table.h
xen/include/public/grant_table.h +49 -30
191 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 4 lines · Luca Fancellu
Modification to
include/public/grant_table.h:1) Change anonymous structure to be named structure, because doxygen can't deal with them.
Especially in the form presented (adding further name space clutter for consumers to fall over) I object to this, most notably ...
xen/include/public/grant_table.h DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t); · line 602
Quoted 6 lines
xen/include/public/grant_table.h +1
-1@@ -584,7 +599,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);* page granted to the calling domain by a foreign domain. */ struct gnttab_cache_flush { - union { + union a {
... this one.
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 9 lines · Jan Beulich
Quoted 4 lines · Luca Fancellu
Modification to
include/public/grant_table.h:1) Change anonymous structure to be named structure, because doxygen can't deal with them.
Especially in the form presented (adding further name space clutter for consumers to fall over) I object to this, most notably ...
Quoted 8 lines
xen/include/public/grant_table.h DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t); · line 602
Quoted 6 lines
xen/include/public/grant_table.h +1
-1@@ -584,7 +599,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);* page granted to the calling domain by a foreign domain. */ struct gnttab_cache_flush { - union { + union a {... this one.
Hi Jan,
It is unfortunate that none of these tools support anonymous structs or unions well. (You might recall we also had issues with the older kernel-doc series too, although a bit different.)
It is difficult to provide a good name here, a suggestion would be more than welcome. I agree with you that calling it "a" is a bad idea: "a" becomes a globally-visible union name.
Maybe we could call it: StructName_MemberName, so in this case:
union gnttab_cache_flush_a
It makes sure it is unique and doesn't risk clashing with anything else. We can follow this pattern elsewhere as well.
Any better suggestions?
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 19 lines · Stefano Stabellini
Quoted 9 lines · Jan Beulich
Quoted 4 lines · Luca Fancellu
Modification to
include/public/grant_table.h:1) Change anonymous structure to be named structure, because doxygen can't deal with them.
Especially in the form presented (adding further name space clutter for consumers to fall over) I object to this, most notably ...
Quoted 8 lines
xen/include/public/grant_table.h DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t); · line 602
Quoted 6 lines
xen/include/public/grant_table.h +1
-1@@ -584,7 +599,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);* page granted to the calling domain by a foreign domain. */ struct gnttab_cache_flush { - union { + union a {... this one.
Quoted 3 lines
It is unfortunate that none of these tools support anonymous structs or unions well. (You might recall we also had issues with the older kernel-doc series too, although a bit different.)
While I wouldn't veto such changes, I think it is a very bad approach to make adjustments like this to cover for a docs tool shortcoming. Is it entirely unreasonable to have the tool fixed? In fact, if the issue was run into before, isn't it a bad sign that the tool hasn't been fixed already, and we merely need to require a certain minimum version?
Quoted 4 lines
It is difficult to provide a good name here, a suggestion would be more than welcome. I agree with you that calling it "a" is a bad idea: "a" becomes a globally-visible union name.
Quoted 8 lines
Maybe we could call it:
StructName_MemberName, so in this case:union
gnttab_cache_flush_aIt makes sure it is unique and doesn't risk clashing with anything else. We can follow this pattern elsewhere as well.
Any better suggestions?
First and foremost any new additions ought to use a xen_, Xen_, or XEN_ prefix. For the specific case here, since "a" is already a rather bad choice for a member name (What does it stand for? In lieu of any better name we typically use "u" in such cases.), the badness shouldn't be extended. Sadly the ref as being one way of expressing the target MFN is also not accompanied by a GFN (as it ought to be), but an address. Otherwise I would have suggested to abstract the similar union also used by struct gnttab_copy. In the end I can't see many alternatives to something like xen_gnttab_cache_flush_target or xen_gnttab_cache_flush_ref_or_addr.
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 48 lines · Jan Beulich
Quoted 19 lines · Stefano Stabellini
Quoted 9 lines · Jan Beulich
Quoted 4 lines · Luca Fancellu
Modification to
include/public/grant_table.h:1) Change anonymous structure to be named structure, because doxygen can't deal with them.
Especially in the form presented (adding further name space clutter for consumers to fall over) I object to this, most notably ...
Quoted 8 lines
xen/include/public/grant_table.h DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t); · line 602
Quoted 6 lines
xen/include/public/grant_table.h +1
-1@@ -584,7 +599,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_swap_grant_ref_t);* page granted to the calling domain by a foreign domain. */ struct gnttab_cache_flush { - union { + union a {... this one.
Quoted 3 lines
It is unfortunate that none of these tools support anonymous structs or unions well. (You might recall we also had issues with the older kernel-doc series too, although a bit different.)
While I wouldn't veto such changes, I think it is a very bad approach to make adjustments like this to cover for a docs tool shortcoming. Is it entirely unreasonable to have the tool fixed? In fact, if the issue was run into before, isn't it a bad sign that the tool hasn't been fixed already, and we merely need to require a certain minimum version?
Quoted 4 lines
It is difficult to provide a good name here, a suggestion would be more than welcome. I agree with you that calling it "a" is a bad idea: "a" becomes a globally-visible union name.
Quoted 8 lines
Maybe we could call it:
StructName_MemberName, so in this case:union
gnttab_cache_flush_aIt makes sure it is unique and doesn't risk clashing with anything else. We can follow this pattern elsewhere as well.
Any better suggestions?
Quoted 10 lines
First and foremost any new additions ought to use a
xen_,Xen_, orXEN_prefix. For the specific case here, since "a" is already a rather bad choice for a member name (What does it stand for? In lieu of any better name we typically use "u" in such cases.), the badness shouldn't be extended. Sadly the ref as being one way of expressing the target MFN is also not accompanied by a GFN (as it ought to be), but an address. Otherwise I would have suggested to abstract the similar union also used by structgnttab_copy. In the end I can't see many alternatives to something likexen_gnttab_cache_flush_targetorxen_gnttab_cache_flush_ref_or_addr.
Hi Jan,
Thank you for your feedback, I agree with you all that “a” is not really a good name, I will be happy to change it if we define a pattern.
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct gnttab_cache_flush {
- union {
+ union xen_gnttab_cache_flush_a {
uint64_t dev_bus_addr;
grant_ref_t ref;
} a;Following this kind of pattern: xen_<upper struct name>_<member name> ?
Cheers,
Luca
Quoted 2 lines
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 11 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{xen/include/public/grant_table.h +1
-1-union { +union xen_gnttab_cache_flush_a {uint64_t dev_bus_addr;grant_ref_t ref; } a;Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo. I'm also unconvinced this would be scalable to the case where there's further struct/union nesting.
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{xen/include/public/grant_table.h +1
-1-union { +union xen_gnttab_cache_flush_a {uint64_t dev_bus_addr;grant_ref_t ref; } a;
Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.
How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 12 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{(quoted diff) +1
-1- union { + union xen_gnttab_cache_flush_a { uint64_t dev_bus_addr; grant_ref_t ref; } a;
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
We need to be careful with changing _anything_ in the public interface. Consumers importing our headers directly (as was e.g. done for the old linux-2.6.18-xen.hg tree) could break with a field name change as much as with any other changes to what had been made available to them.
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
In reply to Stefano StabelliniQuoted 26 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{(quoted diff) +1
-1- union { + union xen_gnttab_cache_flush_a { uint64_t dev_bus_addr; grant_ref_t ref; } a;Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 5 lines
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
There are not many multilevel nesting instances of anonymous struct/union and the maximum level of nesting I found in the public headers is 2:
union {
union/struct {
…
} <name>
} <name>I also see that in the majority of cases the unions have not meaningful names like “a” or “u” as member name, instead struct names are fine, It could be fine to keep the meaningful name the same for the struct type name and use the pattern for the non-meaningful ones as long as the names doesn’t create compilation errors?
Example:
struct upper_level {
union {
struct {
} meaningful_name1;
struct {
} meaningful_name2;
} u;
};becomes:
struct upper_level {
union upper_level_u {
struct meaningful_name1 {
} meaningful_name1;
struct meaningful_name2 {
} meaningful_name2;
} u;
};Doing this will help a lot the documentation side because the html page will show the "struct upper_level" with inside the “union upper_level_u" and inside again the two struct “meaningful_name1" and “meaningful_name2", and from the point of view of the developer it can tell her/him exactly the name of the member to access when writing code (apart from the upper_level_u that can be accessed by u, but we can’t clearly change it).
If this sounds difficult to understand when reading, please generate the documentation and have a look on the page in one side and the source code in another.
Moreover the change should be back-compatible since we are not changing member names.
Cheers,
Luca
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 59 lines · Luca Fancellu
Quoted 26 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{(quoted diff) +1
-1- union { + union xen_gnttab_cache_flush_a { uint64_t dev_bus_addr; grant_ref_t ref; } a;Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 5 lines
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
There are not many multilevel nesting instances of anonymous struct/union and the maximum level of nesting I found in the public headers is 2:
union { union/struct { … } <name> } <name>I also see that in the majority of cases the unions have not meaningful names like “a” or “u” as member name, instead struct names are fine, It could be fine to keep the meaningful name the same for the struct type name and use the pattern for the non-meaningful ones as long as the names doesn’t create compilation errors?
Example:
struct upper_level { union { struct { } meaningful_name1; struct { } meaningful_name2; } u; };becomes:
Quoted 10 lines
struct upper_level {
union upper_level_u {
struct meaningful_name1 {
} meaningful_name1;
struct meaningful_name2 {
} meaningful_name2;
} u;
};As you say - as long as there aren't any compilation issues. Two top level struct-s could have identically named struct/union fields without tag, in which case your approach outlined above will fail. And even if there was no such case right now, the case would need to be covered in whatever naming model was to be used (except, of course, the one without any names).
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
In reply to Luca FancelluHi Luca,
Quoted 59 lines · Luca Fancellu
Quoted 26 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{(quoted diff) +1
-1- union { + union xen_gnttab_cache_flush_a { uint64_t dev_bus_addr; grant_ref_t ref; } a;Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 5 lines
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
There are not many multilevel nesting instances of anonymous struct/union and the maximum level of nesting I found in the public headers is 2:
union { union/struct { … } <name> } <name>I also see that in the majority of cases the unions have not meaningful names like “a” or “u” as member name, instead struct names are fine, It could be fine to keep the meaningful name the same for the struct type name and use the pattern for the non-meaningful ones as long as the names doesn’t create compilation errors?
Example:
struct upper_level { union { struct { } meaningful_name1; struct { } meaningful_name2; } u; };becomes:
Quoted 10 lines
struct upper_level {
union upper_level_u {
struct meaningful_name1 {
} meaningful_name1;
struct meaningful_name2 {
} meaningful_name2;
} u;
};If I understand correctly your proposal, the name of the structure would be the name of the field. The name of the fields are usually pretty generic so you will likely end up to redefine the structure name.
Unless we want to provide random name, the only safe naming would be to define the structure as upper_level_u_meaningful_name{1, 2}. But, this is going to be pretty awful to read.
But I am still a bit puzzled by the fact doxygen is not capable to deal with anynomous/unamed union. How do other projects deal with them?
Quoted 4 lines
Doing this will help a lot the documentation side because the html page will show the "struct
upper_level" with inside the “unionupper_level_u" and inside again the two struct “meaningful_name1" and “meaningful_name2", and from the point of view of the developer it can tell her/him exactly the name of the member to access when writing code (apart from theupper_level_uthat can be accessed by u, but we can’t clearly change it).
I don't quite understand the last point. Wouldn't the developper see the field name? So how is it going to be different from seeing the structure name?
Quoted 1 line
If this sounds difficult to understand when reading, please generate the documentation and have a look on the page in one side and the source code in another.
Just to be clear, do you mean understanding what you wrote or a developper trying to understand the code?
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 73 lines · Julien Grall
Hi Luca,
Quoted 63 lines · Luca Fancellu
Quoted 26 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{
- union {
+ unionxen_gnttab_cache_flush_a{
uint64_tdev_bus_addr;
grant_ref_tref;
} a;Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 5 lines
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
There are not many multilevel nesting instances of anonymous struct/union and the maximum level of nesting I found in the public headers is 2:
union {
union/struct {
…
} <name>
} <name>I also see that in the majority of cases the unions have not meaningful names like “a” or “u” as member name, instead struct names are fine, It could be fine to keep the meaningful name the same for the struct type name and use the pattern for the non-meaningful ones as long as the names doesn’t create compilation errors?
Example:
struct
upper_level{
union {
struct {}
meaningful_name1;
struct {}
meaningful_name2;
} u;
};becomes:
struct
upper_level{
unionupper_level_u{
structmeaningful_name1{Quoted 6 lines
}
meaningful_name1;
structmeaningful_name2{}
meaningful_name2;
} u;
};
Quoted 5 lines
If I understand correctly your proposal, the name of the structure would be the name of the field. The name of the fields are usually pretty generic so you will likely end up to redefine the structure name.
Unless we want to provide random name, the only safe naming would be to define the structure as upper_level_u_meaningful_name{1, 2}. But, this is going to be pretty awful to read.
But I am still a bit puzzled by the fact doxygen is not capable to deal with anynomous/unamed union. How do other projects deal with them?
While going through the list of anynomous union in Xen, I noticed we also have something like:
struct test {
union {
int a;
int b;
};
};We can't name them because of syntactic reasons. What's your plan for them?
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 84 lines · Julien Grall
Quoted 68 lines · Julien Grall
Hi Luca,
Quoted 66 lines · Luca Fancellu
Quoted 26 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{(quoted diff) +1
-1- union { + union xen_gnttab_cache_flush_a { uint64_t dev_bus_addr; grant_ref_t ref; } a;Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 5 lines
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
There are not many multilevel nesting instances of anonymous struct/union and the maximum level of nesting I found in the public headers is 2:
union { union/struct { … } <name> } <name>I also see that in the majority of cases the unions have not meaningful names like “a” or “u” as member name, instead struct names are fine, It could be fine to keep the meaningful name the same for the struct type name and use the pattern for the non-meaningful ones as long as the names doesn’t create compilation errors?
Example:
struct upper_level { union { struct { } meaningful_name1; struct { } meaningful_name2; } u; };becomes:
struct upper_level { union upper_level_u { struct meaningful_name1 { } meaningful_name1; struct meaningful_name2 {Quoted 6 lines
Quoted 3 lines
} meaningful_name2; } u; };If I understand correctly your proposal, the name of the structure would be the name of the field. The name of the fields are usually pretty generic so you will likely end up to redefine the structure name. Unless we want to provide random name, the only safe naming would be to define the structure as upper_level_u_meaningful_name{1, 2}. But, this is going to be pretty awful to read. But I am still a bit puzzled by the fact doxygen is not capable to deal with anynomous/unamed union. How do other projects deal with them?
While going through the list of anynomous union in Xen, I noticed we also have something like:
Quoted 8 lines
struct test { union { int a; int b; }; };We can't name them because of syntactic reasons. What's your plan for them?
I would say that if the fields a and b are not meant to be described in the documentation, they can be hidden, so there is no need to change the structure itself but we might just add some comment containing Doxygen tags for skipping them.
In Zephyr they have some kind of structures like that.
Quoted 5 lines
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
In reply to Julien GrallQuoted 72 lines · Julien Grall
Hi Luca,
Quoted 52 lines · Luca Fancellu
Quoted 24 lines · Stefano Stabellini
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Luca Fancellu
Just to be sure that we are in the same page, are you suggesting to modify the name In this way?
struct
gnttab_cache_flush{(quoted diff) +1
-1- union { + union xen_gnttab_cache_flush_a { uint64_t dev_bus_addr; grant_ref_t ref; } a;Quoted 4 lines
Quoted 1 line
Following this kind of pattern: xen_<upper struct name>_<member name> ?
While in general I would be fine with this scheme, for field names like "a" or "u" it doesn't fit well imo.
"a" is a bad name anyway, even for the member. We can take the opportunity to find a better name. Almost anything would be better than "a". Maybe "refaddr"?
Quoted 5 lines
Quoted 2 lines
I'm also unconvinced this would be
scalable to the case where there's further struct/union nesting.How many of these instances of multilevel nesting do we have? Luca might know. Probably not many? They could be special-cased.
There are not many multilevel nesting instances of anonymous struct/union and the maximum level of nesting I found in the public headers is 2:
union { union/struct { … } <name> } <name>I also see that in the majority of cases the unions have not meaningful names like “a” or “u” as member name, instead struct names are fine, It could be fine to keep the meaningful name the same for the struct type name and use the pattern for the non-meaningful ones as long as the names doesn’t create compilation errors?
Example:struct upper_level { union { struct { } meaningful_name1; struct { } meaningful_name2; } u; }; becomes: struct upper_level { union upper_level_u {Quoted 8 lines
struct meaningful_name1 { } meaningful_name1; struct meaningful_name2 { } meaningful_name2; } u; };If I understand correctly your proposal, the name of the structure would be the name of the field. The name of the fields are usually pretty generic so you will likely end up to redefine the structure name.
Unless we want to provide random name, the only safe naming would be to define the structure as upper_level_u_meaningful_name{1, 2}. But, this is going to be pretty awful to read.
Quoted 7 lines
But I am still a bit puzzled by the fact doxygen is not capable to deal with anynomous/unamed union. How do other projects deal with them?
Quoted 3 lines
Doing this will help a lot the documentation side because the html page will show the "struct
upper_level" with inside the “unionupper_level_u" and inside again the two struct “meaningful_name1" and “meaningful_name2", and from the point of view of the developer it can tell her/him exactly the name of the member to access when writing code (apart from theupper_level_uthat can be accessed by u, but we can’t clearly change it).I don't quite understand the last point. Wouldn't the developper see the field name? So how is it going to be different from seeing the structure name?
The developer, that is using the documentation generated with sphinx+doxygen, will see the structure name and not the field name because this is the way sphinx+doxygen is rendering the code structures. You can see it in the generated documentation using this serie.
Quoted 4 lines
Quoted 2 lines
If this sounds difficult to understand when reading, please generate the documentation and have a look on the page in one side and the source code in another.
Just to be clear, do you mean understanding what you wrote or a developper trying to understand the code?
I meant understanding what I wrote, because I know it’s difficult to describe the concept without visualising the html page, it would have been much easier to attach an image to clarify.
Quoted 5 lines
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
In reply to Luca FancelluHi,
Quoted 4 lines · Luca Fancellu
Modification to
include/public/grant_table.h:1) Change anonymous structure to be named structure, because doxygen can't deal with them.
What do you mean by can't deal with them? I had a quick try with doxygen build and couldn't find any failure (although I haven't looked at the output).
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 13 lines · Julien Grall
Hi,
Quoted 3 lines · Luca Fancellu
Modification to
include/public/grant_table.h:
1) Change anonymous structure to be named structure, because doxygen can't deal with them.What do you mean by can't deal with them? I had a quick try with doxygen build and couldn't find any failure (although I haven't looked at the output).
Hi Julien,
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
Cheers,
Luca
Quoted 4 lines
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Hi Luca,
Quoted 11 lines · Luca Fancellu
Quoted 11 lines · Julien Grall
Hi,
Quoted 3 lines · Luca Fancellu
Modification to
include/public/grant_table.h:
1) Change anonymous structure to be named structure, because doxygen can't deal with them.
Quoted 7 lines
Quoted 2 lines
What do you mean by can't deal with them? I had a quick try with doxygen build and couldn't find any failure (although I haven't looked at the output).
Hi Julien,
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
I might do something wrong because I cannot spot any significant difference in the doxygen ouput if I switch back to anonymous union. Would you mind to post more details (such as a diff) on how doxygen doesn't generate properly documentation?
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 16 lines · Julien Grall
Hi Luca,
Quoted 9 lines · Luca Fancellu
Quoted 9 lines · Julien Grall
Hi,
Quoted 3 lines · Luca Fancellu
Modification to
include/public/grant_table.h:
1) Change anonymous structure to be named structure, because doxygen can't deal with them.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
What do you mean by can't deal with them? I had a quick try with doxygen build and couldn't find any failure (although I haven't looked at the output).
Hi Julien,
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.I might do something wrong because I cannot spot any significant difference in the doxygen ouput if I switch back to anonymous union. Would you mind to post more details (such as a diff) on how doxygen doesn't generate properly documentation?
Hi Julien,
Here the explanation of the problem: https://vovkos.github.io/doxyrest/manual/unnamed-structs.html
Clearly the proposed solution is not suitable because they are just hiding the anonymous union/struct field from the documentation.
I did two test:
1) with the anonymous structure:
struct gnttab_cache_flush {
union {
uint64_t dev_bus_addr;
grant_ref_t ref;} a;
I get a warning from sphinx (because the XML output of Doxygen is not in a good shape) when I generate the documentation, here follow the output:
$ make -C docs XEN_TARGET_ARCH="arm64" sphinx-html make: Entering directory '/home/user/prj_xen/xen/docs' Generating xen.doxyfile mv xen.doxyfile.tmp xen.doxyfile Generating doxygen_input.h /usr/bin/doxygen xen.doxyfile Generating hypercall-interfaces/index.rst XEN_ROOT=/home/user/prj_xen/xen /usr/bin/sphinx-build -b html . sphinx/html Running Sphinx v1.6.7 making output directory... loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [html]: targets for 21 source files that are out of date updating environment: 21 added, 0 changed, 0 removed reading sources... [100%] misc/xen-makefiles/makefiles /home/user/prj_xen/xen/docs/hypercall-interfaces/arm64/grant_tables.rst:6: WARNING: Invalid definition: Expected end of definition. [error at 18] gnttab_cache_flush.a ------------------^ looking for now-outdated files... none found pickling environment... done checking consistency... /home/user/prj_xen/xen/docs/hypercall-interfaces/arm32.rst: WARNING: document isn't included in any toctree /home/user/prj_xen/xen/docs/hypercall-interfaces/x86_64.rst: WARNING: document isn't included in any toctree /home/user/prj_xen/xen/docs/misc/kconfig.rst: WARNING: document isn't included in any toctree /home/user/prj_xen/xen/docs/misc/kconfig-language.rst: WARNING: document isn't included in any toctree /home/user/prj_xen/xen/docs/misc/kconfig-macro-language.rst: WARNING: document isn't included in any toctree /home/user/prj_xen/xen/docs/misc/xen-makefiles/makefiles.rst: WARNING: document isn't included in any toctree done preparing documents... done writing output... [100%] misc/xen-makefiles/makefiles generating indices... genindex writing additional pages... search copying images... [100%] admin-guide/xen-overview.drawio.svg copying static files... done copying extra files... done dumping search index in English (code: en) ... done dumping object inventory... done build succeeded, 7 warnings. make: Leaving directory '/home/user/prj_xen/xen/docs’
And checking the generated html page html/hypercall-interfaces/arm64/grant_tables.html you can see that there is a union without name or fields right above "union __guest_handle_gnttab_cache_flush_t".
2) without the anonymous structure:
struct gnttab_cache_flush {
- union {
+ union a {
uint64_t dev_bus_addr;
grant_ref_t ref;} a;
I don’t get the warning from sphinx and looking in the html/hypercall-interfaces/arm64/grant_tables.html page I can see the proper documentation for the union a right above "union __guest_handle_gnttab_cache_flush_t”.
Let me know if you get different results.
Cheers,
Luca
Quoted 4 lines
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
In reply to Luca FancelluLuca Fancellu writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h"):
Quoted 2 lines
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
This does not seem like it would be an impossibly hard feature to add to doxygen.
Ian.
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Hi Ian,
Quoted 8 lines · Ian Jackson
Luca Fancellu writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for
grant_table.h"):Quoted 2 lines
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
This does not seem like it would be an impossibly hard feature to add to doxygen.
Modifying doxygen is not really in our planned efforts and if someone does it that would put an hard constraint on the version of doxygen possible to use.
But is adding names to anonymous elements really an issue here ?
If we agree on names or on a convention for name the result will not impact the code or backward compatibility.
Regards
Bertrand
Quoted 2 lines
Ian.
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Quoted 6 lines · Bertrand Marquis
Quoted 6 lines · Ian Jackson
Luca Fancellu writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for
grant_table.h"):Quoted 2 lines
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
Quoted 6 lines
Quoted 2 lines
This does not seem like it would be an impossibly hard feature to add to doxygen.
Modifying doxygen is not really in our planned efforts and if someone does it that would put an hard constraint on the version of doxygen possible to use.
But is adding names to anonymous elements really an issue here ?
It's clutter in the code base, making things harder to read (even if just slightly). It's certainly odd to make such source changes just for a doc tool. If changing doxygen is not an option for you, how about pre-processing the header and inserting the names the tool wants, before handing the result as input to it?
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Hi,
Quoted 14 lines · Jan Beulich
Quoted 11 lines · Bertrand Marquis
Quoted 8 lines · Ian Jackson
Luca Fancellu writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for
grant_table.h"):Quoted 2 lines
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
This does not seem like it would be an impossibly hard feature to add to doxygen.
Modifying doxygen is not really in our planned efforts and if someone does it that would put an hard constraint on the version of doxygen possible to use.
Quoted 7 lines
Quoted 1 line
But is adding names to anonymous elements really an issue here ?
It's clutter in the code base, making things harder to read (even if just slightly). It's certainly odd to make such source changes just for a doc tool. If changing doxygen is not an option for you, how about pre-processing the header and inserting the names the tool wants, before handing the result as input to it?
Introducing a new tool or find a pre-processing solution is for sure possible but will be more complex and less error prone.
Also as said in my mail to Julien, passing this issue of anonymous entries now might just push the problem as we might have to solve it later if we want to become MISRA compliant (which is also something we are looking in FuSa).
Regards
Bertrand
Quoted 3 lines
Jan
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
In reply to Bertrand MarquisQuoted 4 lines · Bertrand Marquis
Hi Ian,
Quoted 2 lines · Ian Jackson
Quoted 8 lines
Quoted 6 lines
Luca Fancellu writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for
grant_table.h"):Quoted 2 lines
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
This does not seem like it would be an impossibly hard feature to add to doxygen.
Modifying doxygen is not really in our planned efforts and if someone does it that would put an hard constraint on the version of doxygen possible to use.
Are you saying that anyone who want to use doxygen has to waive off the use of anonymous union/struct? Is it the only thing doxygen can't deal with?
Quoted 2 lines
But is adding names to anonymous elements really an issue here ?
If we agree on names or on a convention for name the result will not impact the code or backward compatibility.
I think the naming is only the tip of the problem. One advantage of anymous union/struct is you make clear they are not meant to be used outside of the context. So they should mostly be seen as an easy way to access some part of the "parent" structure directly. Therefore, IMHO, they don't deserve to be documented separately.
In fact, this is the first thing I noticed when building the documentation because 'union a' was in global index.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Hi Julien,
Quoted 9 lines · Julien Grall
Quoted 4 lines · Bertrand Marquis
Hi Ian,
Quoted 3 lines · Ian Jackson
Luca Fancellu writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for
grant_table.h"):
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
The problem is that Doxygen can’t generate proper documentation for anonymous union/structure, it ends up with warning and/or producing wrong documentation like changing names or giving field description to the wrong field.
This does not seem like it would be an impossibly hard feature to add to doxygen.
Modifying doxygen is not really in our planned efforts and if someone does it that would put an hard constraint on the version of doxygen possible to use.
Are you saying that anyone who want to use doxygen has to waive off the use of anonymous union/struct? Is it the only thing doxygen can't deal with?
That is the main one we came into while doing this but there might be other going forward, hard to be sure at this stage.
Quoted 5 lines
Quoted 3 lines
But is adding names to anonymous elements really an issue here ?
If we agree on names or on a convention for name the result will not impact the code or backward compatibility.I think the naming is only the tip of the problem. One advantage of anymous union/struct is you make clear they are not meant to be used outside of the context. So they should mostly be seen as an easy way to access some part of the "parent" structure directly. Therefore, IMHO, they don't deserve to be documented separately.
Somehow in the documentation when you have a union you will need to document that it is a union and the possible entries. Having a name to refer to it sounds to me a lot easier than making it anonymous.
One way or an other most standards like MISRA are forbidding anonymous entries as they cannot be referred to.
Quoted 2 lines
In fact, this is the first thing I noticed when building the documentation because 'union a' was in global index.
Definitely I agree the “a” is not a good solution and we need to find meaningful names.
But this is in fact true for the sub-element in the structure (from which the name was taken), using “a” as an identifier is not really explanatory of what that is. “u” for union can be see as a standard though.
This is why i think we should put names which a meaning but this is not always easy to find.
Cheers
Bertrand
Quoted 5 lines
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h
Bertrand Marquis writes ("Re: [PATCH 3/3] docs/doxygen: doxygen documentation for grant_table.h"):
Quoted 1 line
Somehow in the documentation when you have a union you will need to document that it is a union and the possible entries.
I would expect to find the documentation for an anonymous struct or union folded into the documentation of the containing structure, just as it is in the source.
Quoted 1 line
One way or an other most standards like MISRA are forbidding anonymous entries as they cannot be referred to.
An anonymous union or struct like this is always the type of a single field in a containing aggegate type. So if one needs to speak of it, one can specify the container's type and the field. So it *can* be named.
I am assuming we don't have *unused* anonymous structs and unions.
Ian.
Quoted 17 lines · Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):
Quoted 7 lines
./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-htmlnow in docs/sphinx/html/ we have the generated docs starting from the index.html page.
Do you have a sample rendered output?
The plan was to try and use Linux's kernel-doc plugin for Sphinx, which is very doxygen-like. Did you experiment with this option?
~Andrew
Quoted 23 lines · Andrew Cooper
Quoted 22 lines · Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-html
Quoted 7 lines
Quoted 2 lines
now in docs/sphinx/html/ we have the generated docs starting from the index.html page.
Do you have a sample rendered output?
The plan was to try and use Linux's kernel-doc plugin for Sphinx, which is very doxygen-like. Did you experiment with this option?
As you probably know the end goal for Luca (and the Xen FuSa SIG as a whole) is to generate all FuSa documents, including requirements docs, interface docs, etc.
FuSa requires us to follow the famous V model, where the high level requirements are linked to the lower level requirements, which are linked to the interface docs, which are linked all the way down to the code.
Maintaining the linking is difficult and typically done with expensive proprietary FuSa tools.
Fortunately, an engineer working with the Zephyr project developed a set of scripts for Doxygen that are able to generate the required FuSa docs and also links from in-code comments and markdown/rst docs in the tree.
This is great work, and in the FuSa SIG we thought it would be best to align ourselves with Zephyr to be able to pull our efforts together on the tooling side instead of doing the same thing again on our own.
This is the reason why we ended up with Doxygen.
Quoted 46 lines · Stefano Stabellini
Quoted 23 lines · Andrew Cooper
Quoted 22 lines · Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-htmlQuoted 6 lines
Quoted 2 lines
now in docs/sphinx/html/ we have the generated docs starting from the index.html page.
Do you have a sample rendered output?
The plan was to try and use Linux's kernel-doc plugin for Sphinx, which is very doxygen-like. Did you experiment with this option?
As you probably know the end goal for Luca (and the Xen FuSa SIG as a whole) is to generate all FuSa documents, including requirements docs, interface docs, etc.
FuSa requires us to follow the famous V model, where the high level requirements are linked to the lower level requirements, which are linked to the interface docs, which are linked all the way down to the code.
Maintaining the linking is difficult and typically done with expensive proprietary FuSa tools.
Fortunately, an engineer working with the Zephyr project developed a set of scripts for Doxygen that are able to generate the required FuSa docs and also links from in-code comments and markdown/rst docs in the tree.
Quoted 5 lines
This is great work, and in the FuSa SIG we thought it would be best to align ourselves with Zephyr to be able to pull our efforts together on the tooling side instead of doing the same thing again on our own.
This is the reason why we ended up with Doxygen.
So are we're saying that Doxygen is a hard dependency because there is an extension for Doxygen to generate other safety docs?
~Andrew
Hi Andrew,
Quoted 49 lines · Andrew Cooper
Quoted 46 lines · Stefano Stabellini
Quoted 23 lines · Andrew Cooper
Quoted 22 lines · Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-htmlQuoted 6 lines
Quoted 2 lines
now in docs/sphinx/html/ we have the generated docs starting from the index.html page.
Do you have a sample rendered output?
The plan was to try and use Linux's kernel-doc plugin for Sphinx, which is very doxygen-like. Did you experiment with this option?
As you probably know the end goal for Luca (and the Xen FuSa SIG as a whole) is to generate all FuSa documents, including requirements docs, interface docs, etc.
FuSa requires us to follow the famous V model, where the high level requirements are linked to the lower level requirements, which are linked to the interface docs, which are linked all the way down to the code.
Maintaining the linking is difficult and typically done with expensive proprietary FuSa tools.
Fortunately, an engineer working with the Zephyr project developed a set of scripts for Doxygen that are able to generate the required FuSa docs and also links from in-code comments and markdown/rst docs in the tree.
Quoted 8 lines
Quoted 5 lines
This is great work, and in the FuSa SIG we thought it would be best to align ourselves with Zephyr to be able to pull our efforts together on the tooling side instead of doing the same thing again on our own.
This is the reason why we ended up with Doxygen.
So are we're saying that Doxygen is a hard dependency because there is an extension for Doxygen to generate other safety docs?
hard no as we could find other solution but the current strategy we take together with Zephyr project is based on using Doxygen for requirement linking.
Also an other argument is that the documentation generated is actually nice and that could also be useful for developers and users (see [1] for Zephyr doc).
Our global idea is also that using doxygen we can extract a big part of the documentation (which will partly be used as certification) directly from the code which will make it a lot easier for developers to maintain.
Regards
Bertrand
[1] https://docs.zephyrproject.org/latest/
Quoted 3 lines
~Andrew
Quoted 17 lines · Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, some package may be required on your machine, everything is suggested by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):
Quoted 4 lines
./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-html
I have tried this instruction and got:
make: Entering directory '/home/ANT.AMAZON.COM/jgrall/works/oss/xen/docs' xen.doxyfile
make: xen.doxyfile: Command not found
Makefile:67: recipe for target 'sphinx-html' failed
make: *** [sphinx-html] Error 127
make: Leaving directory '/home/ANT.AMAZON.COM/jgrall/works/oss/xen/docs'
AFAICT, $DOXYGEN will only get defined when sphinx-build is installed.
When sphinx-build is not installed SPHINXBUILD will be equal to 'no', but the Makefile check for emptiness.
Cheers,
Signature · 1 line
Julien Grall
Quoted 35 lines · Julien Grall
Quoted 10 lines · Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that must be addressed modifying the current codebase (for example doxygen can't parse anonymous structure/union).
To reproduce the documentation xen must be compiled because most of the headers are generated on compilation time from the makefiles.
Here follows the steps to generate the sphinx html docs, someQuoted 8 lines
package may be required on your machine, everything is suggested
by the autoconf script.
Here I'm building the arm64 docs (the only introduced for now by this serie):
./configure
make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" menuconfig make -C xen XEN_TARGET_ARCH="arm64" CROSS_COMPILE="aarch64-linux-gnu-" make -C docs XEN_TARGET_ARCH="arm64" sphinx-htmlI have tried this instruction and got:
make: Entering directory '
/home/ANT.AMAZON.COM/jgrall/works/oss/xen/docs' xen.doxyfile
make: xen.doxyfile: Command not found
Makefile:67: recipe for target 'sphinx-html' failed
make: *** [sphinx-html] Error 127
make: Leaving directory '/home/ANT.AMAZON.COM/jgrall/works/oss/xen/docs'
Quoted 3 lines
AFAICT, $DOXYGEN will only get defined when sphinx-build is installed.
When sphinx-build is not installed SPHINXBUILD will be equal to 'no', but theMakefilecheck for emptiness.
Hi Julien,
Thank you for spotting it, I’ll fix it in the v2 patch
Cheers,
Luca
Quoted 4 lines
Cheers,
Signature · 1 line
Julien Grall