Skip to contentSkip to conversation

doc: build manpages as well as html output

PATCH Superseded Bruce Richardson 5 participants 23 messages Jun 1, 2023 – Sep 27, 2023 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc: build manpages as well as html output

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

---
 doc/api/doxy-api.conf.in    |  7 ++---
 doc/api/generate_doxygen.py |  2 +-
 doc/api/meson.build         | 52 +++++++++++++++++++++++++++++++------
 3 files changed, 49 insertions(+), 12 deletions(-)
doc/api/doxy-api.conf.in +4 -3
index 967aa11701..6e5be9b469 100644
@@ -125,10 +125,11 @@ EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
doc/api/generate_doxygen.py +1 -1
index d3a22869f6..c704f13018 100755
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
doc/api/meson.build +44 -8

74 lines not on this page · raw message

Signature · 1 line
2.39.2
#1 lore raw
Quoted 21 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

doc/api/meson.build doxy_build = custom_target('doxygen', · line 95

Quoted 4 lines
doc/api/meson.build +4 -0
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man... Checking for stray cats under /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man/cat1... Purging old database entries in /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man... Checking for stray cats under /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man/cat1... 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

[main][dpdk.org] $ man rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create

Quoted 4 lines

+endif
--
2.39.2

#2 lore raw
Quoted 46 lines · Jerin Jacob
Quoted 21 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

doc/api/meson.build doxy_build = custom_target('doxygen', · line 95

Quoted 4 lines
doc/api/meson.build +4 -0
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man... Checking for stray cats under /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man/cat1... Purging old database entries in /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man... Checking for stray cats under /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man/cat1... 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

Quoted 7 lines

[main][dpdk.org] $ man rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create

Yes, that is correct.

If you install to a non-standard location, then yes you need to update manpath yourself. However, in case you install to a "standard" location, then running mandb will update the database for you. I believe this is the behaviour we should have. I view it as the same as installing binaries in a standard vs non-standard path - if the binaries are placed in a standard location then they are found automatically, but if installed in a custom location, then the user is responsible for ensuring all paths are correct.

/Bruce

#3 lore raw
Quoted 57 lines · Bruce Richardson
Quoted 46 lines · Jerin Jacob
Quoted 21 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

doc/api/meson.build doxy_build = custom_target('doxygen', · line 95

Quoted 4 lines
doc/api/meson.build +4 -0
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man... Checking for stray cats under /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man/cat1... Purging old database entries in /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man... Checking for stray cats under /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man/cat1... 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

Quoted 7 lines

[main][dpdk.org] $ man rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create

Yes, that is correct.

Quoted 7 lines

If you install to a non-standard location, then yes you need to update manpath yourself. However, in case you install to a "standard" location, then running mandb will update the database for you. I believe this is the behaviour we should have. I view it as the same as installing binaries in a standard vs non-standard path - if the binaries are placed in a standard location then they are found automatically, but if installed in a custom location, then the user is responsible for ensuring all paths are correct.

OK. Then I think, we can move "meson.add_install_script(mandb)" invocation under !DESTDIR not provided.
As if DESTDIR is provided then there is no use for mandb.

Quoted 2 lines

/Bruce

#4 lore raw
Quoted 67 lines · Jerin Jacob
Quoted 57 lines · Bruce Richardson
Quoted 46 lines · Jerin Jacob
Quoted 21 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

doc/api/meson.build doxy_build = custom_target('doxygen', · line 95

Quoted 4 lines
doc/api/meson.build +4 -0
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man... Checking for stray cats under /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man/cat1... Purging old database entries in /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man... Checking for stray cats under /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man/cat1... 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

Quoted 7 lines

[main][dpdk.org] $ man rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create

Yes, that is correct.

Quoted 7 lines

If you install to a non-standard location, then yes you need to update manpath yourself. However, in case you install to a "standard" location, then running mandb will update the database for you. I believe this is the behaviour we should have. I view it as the same as installing binaries in a standard vs non-standard path - if the binaries are placed in a standard location then they are found automatically, but if installed in a custom location, then the user is responsible for ensuring all paths are correct.

Quoted 4 lines

OK. Then I think, we can move "meson.add_install_script(mandb)" invocation under !DESTDIR not provided.
As if DESTDIR is provided then there is no use for mandb.

That would mean adding a new build script to run mandb, since:
* meson doesn't directly support getting environment variables
* DESTDIR may only be set at install time rather than at configuration time, so we'll always need to set up the build script

Furthermore, I'd point out that DESTDIR is not going to avoid unnecessary runs of mandb, since using a special "prefix" setting would also cause the manpages to not be found.

Overall, I'd view the complexity of avoiding the mandb call not to be worth it, since calling mandb is largely harmless - bar the time taken to run it. However, if you feel strongly it should be skipped when installing with DESTDIR, I'll spin a v2 with the wrapper script added. Let me know what you think?

/Bruce

#5 lore raw
Quoted 78 lines · Bruce Richardson
Quoted 67 lines · Jerin Jacob
Quoted 57 lines · Bruce Richardson
Quoted 46 lines · Jerin Jacob
Quoted 21 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

doc/api/meson.build doxy_build = custom_target('doxygen', · line 95

Quoted 4 lines
doc/api/meson.build +4 -0
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man... Checking for stray cats under /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man/cat1... Purging old database entries in /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man... Checking for stray cats under /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man/cat1... 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

Quoted 7 lines

[main][dpdk.org] $ man rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create

Yes, that is correct.

Quoted 7 lines

If you install to a non-standard location, then yes you need to update manpath yourself. However, in case you install to a "standard" location, then running mandb will update the database for you. I believe this is the behaviour we should have. I view it as the same as installing binaries in a standard vs non-standard path - if the binaries are placed in a standard location then they are found automatically, but if installed in a custom location, then the user is responsible for ensuring all paths are correct.

Quoted 4 lines

OK. Then I think, we can move "meson.add_install_script(mandb)" invocation under !DESTDIR not provided.
As if DESTDIR is provided then there is no use for mandb.

That would mean adding a new build script to run mandb, since:
* meson doesn't directly support getting environment variables
* DESTDIR may only be set at install time rather than at configuration time, so we'll always need to set up the build script

Quoted 8 lines

Furthermore, I'd point out that DESTDIR is not going to avoid unnecessary runs of mandb, since using a special "prefix" setting would also cause the manpages to not be found.

Overall, I'd view the complexity of avoiding the mandb call not to be worth it, since calling mandb is largely harmless - bar the time taken to run it. However, if you feel strongly it should be skipped when installing with DESTDIR

No strong option. OK to skip DESTDIR check.

Quoted 2 lines

, I'll spin a v2 with the wrapper script added. Let me know what you think?

I think, it is good to update doc/guides/contributing/documentation.rst on how to use the man page(i.e use with --manpage)

Quoted 2 lines

/Bruce

#6 lore raw
Quoted 89 lines · Jerin Jacob
Quoted 78 lines · Bruce Richardson
Quoted 67 lines · Jerin Jacob
Quoted 57 lines · Bruce Richardson
Quoted 46 lines · Jerin Jacob
Quoted 21 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

doc/api/meson.build doxy_build = custom_target('doxygen', · line 95

Quoted 4 lines
doc/api/meson.build +4 -0
+
+mandb = find_program('mandb', required: false)
+if mandb.found() and get_option('enable_docs') and meson.version().version_compare('>=0.55.0')
+    meson.add_install_script(mandb)

It does not look like just executing mandb it is adding these man pages to database

log:
Running custom install script '/usr/bin/mandb'
Purging old database entries in /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man... Checking for stray cats under /home/jerin/.local/man... Processing manual pages under /home/jerin/.local/man/cat1... Purging old database entries in /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man... Checking for stray cats under /home/jerin/.local/share/man... Processing manual pages under /home/jerin/.local/share/man/cat1... 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

Quoted 7 lines

[main][dpdk.org] $ man rte_flow_create
No manual entry for rte_flow_create

# Following works by providing the path i.e man pages created properly only db update is missing
man --manpath=/tmp/i/usr/local/share/man/ rte_flow_create

Yes, that is correct.

Quoted 7 lines

If you install to a non-standard location, then yes you need to update manpath yourself. However, in case you install to a "standard" location, then running mandb will update the database for you. I believe this is the behaviour we should have. I view it as the same as installing binaries in a standard vs non-standard path - if the binaries are placed in a standard location then they are found automatically, but if installed in a custom location, then the user is responsible for ensuring all paths are correct.

Quoted 4 lines

OK. Then I think, we can move "meson.add_install_script(mandb)" invocation under !DESTDIR not provided.
As if DESTDIR is provided then there is no use for mandb.

That would mean adding a new build script to run mandb, since:
* meson doesn't directly support getting environment variables
* DESTDIR may only be set at install time rather than at configuration time, so we'll always need to set up the build script

Quoted 8 lines

Furthermore, I'd point out that DESTDIR is not going to avoid unnecessary runs of mandb, since using a special "prefix" setting would also cause the manpages to not be found.

Overall, I'd view the complexity of avoiding the mandb call not to be worth it, since calling mandb is largely harmless - bar the time taken to run it. However, if you feel strongly it should be skipped when installing with DESTDIR

Quoted 8 lines

No strong option. OK to skip DESTDIR check.

Quoted 2 lines

, I'll spin a v2 with the wrapper script added. Let me know what you think?

I think, it is good to update doc/guides/contributing/documentation.rst on how to use the man page(i.e use with --manpage)

Ok, will spin a v2.

#7 lore raw

PATCH v2 1/1 doc: build manpages as well as html output

In reply to Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

---
V2: add doc update about building or using the manpages

---
 doc/api/doxy-api.conf.in                  |  7 +--
 doc/api/generate_doxygen.py               |  2 +-
 doc/api/meson.build                       | 52 +++++++++++++++++++----
 doc/guides/contributing/documentation.rst | 12 +++++-
 4 files changed, 60 insertions(+), 13 deletions(-)
doc/api/doxy-api.conf.in +4 -3
index 967aa11701..6e5be9b469 100644
@@ -125,10 +125,11 @@ EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
doc/api/generate_doxygen.py +1 -1
index d3a22869f6..c704f13018 100755
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
doc/api/meson.build +44 -8

74 lines not on this page · raw message

doc/guides/contributing/documentation.rst +11 -1
index 7fcbb7fc43..e0b672ffbb 100644
@@ -182,7 +182,17 @@ To build the documentation::
 
 See :doc:`../linux_gsg/build_dpdk` for more detail on compiling DPDK with meson.
 
-The output is generated in the directories ``build/doc/html/{api,guides}``.
+The output is generated in the directory ``build/doc/``, with:
+
+* HTML versions of the guide docs, e.g. Getting Started Guides, Programmers Guide, in ``build/doc/guides/html``
+* HTML version of the API documentation in ``build/doc/api/html``
+* Man-page version of the API documentation in ``build/doc/api/man``.
+  If not installing DPDK system-wise, these pages can be accessed by adding this directory to the ``MANPATH`` environment variable.
+  For example:
+
+.. code-block:: console
+
+    export MANPATH=:/path/to/build/doc/api/man
 
 .. Note::
 
Signature · 1 line
2.39.2
Reviewed-by on v2 1/1
#8 lore raw
Quoted 19 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

Quoted 1 line

Nice.. we should have done this sooner :-).

There may be some polishing to do later.
Looking at the result for rte_eal_init, I see that the generated manual starts with a reference to the rte_eal.h header with a path relative to its location in the DPDK tree.

$ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
lib/eal/include/rte_eal.h(3)
                                           DPDK

lib/eal/include/rte_eal.h(3)

NAME
lib/eal/include/rte_eal.h

At least, it is possible to ask for this header man with "man rte_eal.h", but it is a bit confusing.
Is there something we can do on this side?

Signature · 1 line
David Marchand
#9 lore raw
Quoted 36 lines · David Marchand
Quoted 19 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

Quoted 1 line

Nice.. we should have done this sooner :-).

There may be some polishing to do later.
Looking at the result for rte_eal_init, I see that the generated manual starts with a reference to the rte_eal.h header with a path relative to its location in the DPDK tree.

$ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
lib/eal/include/rte_eal.h(3)
                                           DPDK
Quoted 9 lines

lib/eal/include/rte_eal.h(3)

NAME
lib/eal/include/rte_eal.h

At least, it is possible to ask for this header man with "man rte_eal.h", but it is a bit confusing.
Is there something we can do on this side?

Not sure, not really familiar with how doxygen works generating manpages and the options supported, etc. etc. Mainly I just looked at the build-system side to support this, since I really missed having manpages for DPDK functions to quickly check parameter order.

/Bruce

#10 lore raw
Quoted 18 lines · Bruce Richardson
Quoted 8 lines

There may be some polishing to do later.
Looking at the result for rte_eal_init, I see that the generated manual starts with a reference to the rte_eal.h header with a path relative to its location in the DPDK tree.

$ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
lib/eal/include/rte_eal.h(3)
                                           DPDK
Quoted 9 lines

lib/eal/include/rte_eal.h(3)

NAME
lib/eal/include/rte_eal.h

At least, it is possible to ask for this header man with "man rte_eal.h", but it is a bit confusing.
Is there something we can do on this side?

Quoted 4 lines

Not sure, not really familiar with how doxygen works generating manpages and the options supported, etc. etc. Mainly I just looked at the build-system side to support this, since I really missed having manpages for DPDK functions to quickly check parameter order.

doxygen is invoked on the sources tree, so this is probably the reason why such path is in the generated manual.
Maybe Thomas has an idea how we could handle this.

In any case, I am fine with this patch as it is now.

Signature · 1 line
David Marchand
#11 lore raw
Quoted 25 lines · David Marchand
Quoted 18 lines · Bruce Richardson
Quoted 8 lines

There may be some polishing to do later.
Looking at the result for rte_eal_init, I see that the generated manual starts with a reference to the rte_eal.h header with a path relative to its location in the DPDK tree.

$ MANPATH=build-gcc/install/share/man man rte_eal_init | head -5
lib/eal/include/rte_eal.h(3)
                                           DPDK
Quoted 9 lines

lib/eal/include/rte_eal.h(3)

NAME
lib/eal/include/rte_eal.h

At least, it is possible to ask for this header man with "man rte_eal.h", but it is a bit confusing.
Is there something we can do on this side?

Quoted 4 lines

Not sure, not really familiar with how doxygen works generating manpages and the options supported, etc. etc. Mainly I just looked at the build-system side to support this, since I really missed having manpages for DPDK functions to quickly check parameter order.

Quoted 6 lines

doxygen is invoked on the sources tree, so this is probably the reason why such path is in the generated manual.
Maybe Thomas has an idea how we could handle this.

In any case, I am fine with this patch as it is now.

I think I've found a fix for this.
doxygen setting "FULL_PATH_NAMES" defaults to "YES", which is what we want for the HTML pages. However, setting it to "NO", I believe is correct for generating the manpages.

Patch V3 on its way.

/Bruce

#12 lore raw

PATCH v3 1/1 doc: build manpages as well as html output

In reply to Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

---

V3: don't use full file paths when generating manpages

V2: add doc update about building or using the manpages

---
 doc/api/doxy-api.conf.in                  |  8 ++--
 doc/api/generate_doxygen.py               |  2 +-
 doc/api/meson.build                       | 54 +++++++++++++++++++----
 doc/guides/contributing/documentation.rst | 12 ++++-
 4 files changed, 63 insertions(+), 13 deletions(-)
doc/api/doxy-api.conf.in +5 -3
index 1a4210b948..c77bdb328b 100644
@@ -124,11 +124,13 @@ EXAMPLE_PATTERNS        = *.c
 EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
+FULL_PATH_NAMES         = @FULL_PATH_NAMES@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
doc/api/generate_doxygen.py +1 -1
index d3a22869f6..c704f13018 100755
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
doc/api/meson.build +46 -8

76 lines not on this page · raw message

doc/guides/contributing/documentation.rst +11 -1
index 7fcbb7fc43..e0b672ffbb 100644
@@ -182,7 +182,17 @@ To build the documentation::
 
 See :doc:`../linux_gsg/build_dpdk` for more detail on compiling DPDK with meson.
 
-The output is generated in the directories ``build/doc/html/{api,guides}``.
+The output is generated in the directory ``build/doc/``, with:
+
+* HTML versions of the guide docs, e.g. Getting Started Guides, Programmers Guide, in ``build/doc/guides/html``
+* HTML version of the API documentation in ``build/doc/api/html``
+* Man-page version of the API documentation in ``build/doc/api/man``.
+  If not installing DPDK system-wise, these pages can be accessed by adding this directory to the ``MANPATH`` environment variable.
+  For example:
+
+.. code-block:: console
+
+    export MANPATH=:/path/to/build/doc/api/man
 
 .. Note::
 
Signature · 1 line
2.39.2
#13 lore raw
Quoted 19 lines · Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found.

Quoted 6 lines

---

V3: don't use full file paths when generating manpages

Nice!
Thank you Bruce.

Signature · 1 line
David Marchand
#14 lore raw
In reply to Bruce Richardson

3/08/2023, Bruce Richardson:

Quoted 1 line

+#set up common doxygen configuration

A space is missing here

Quoted 1 line

+man_cdata.set('FULL_PATH_NAMES', 'NO')

Why it has to be disabled? Maybe add a comment?

Quoted 1 line

+ meson.add_install_script(mandb)

When is it executed exactly?
Will it update the database in case we install in a staging directory, when preparing a package for later deploying on another machine?

Review

Quoted 3 lines

+.. code-block:: console

(quoted diff) +2 -0
+
+    export MANPATH=:/path/to/build/doc/api/man

Styling consideration: I think we should indent with 3 spaces so it is aligned with "code-block".

#15 lore raw
Quoted 5 lines · Thomas Monjalon

3/08/2023, Bruce Richardson:

Quoted 1 line

+#set up common doxygen configuration

A space is missing here

Ack

Quoted 4 lines
Quoted 1 line

+man_cdata.set('FULL_PATH_NAMES', 'NO')

Why it has to be disabled? Maybe add a comment?

Sure, will add comment. It's so that the man pages don't include full paths to the headers, but are just e.g. "rte_eal.h".

Quoted 5 lines
Quoted 1 line

+ meson.add_install_script(mandb)

When is it executed exactly?
Will it update the database in case we install in a staging directory, when preparing a package for later deploying on another machine?

Yes, it will. Unfortunately, I can't find any way to just call mandb if we are installing in a system manpage location on the local machine. Therefore, I had two options:
1. don't update the manpage database. In this case, the user won't be able to actually get the newly install manpages
2. always update the local manpage database. In this case, the user installing the docs will find them, but anyone installing to staging will experience a slight delay while their local mandb is updated.

I went for #2 on the basis that the delay in the staging case is pretty harmless, while not actually finding the manpages is more serious.

However, I'm open to other suggestions on how to work this?

Quoted 8 lines

Review

Quoted 4 lines

+.. code-block:: console

(quoted diff) +2 -0
+
+    export MANPATH=:/path/to/build/doc/api/man

Styling consideration: I think we should indent with 3 spaces so it is aligned with "code-block".

Ack.

#16 lore raw

29/08/2023 12:10, Bruce Richardson:

Quoted 8 lines · Thomas Monjalon
Quoted 6 lines

3/08/2023, Bruce Richardson:

Quoted 1 line

+ meson.add_install_script(mandb)

When is it executed exactly?
Will it update the database in case we install in a staging directory, when preparing a package for later deploying on another machine?

Quoted 5 lines

Yes, it will. Unfortunately, I can't find any way to just call mandb if we are installing in a system manpage location on the local machine. Therefore, I had two options:
1. don't update the manpage database. In this case, the user won't be able to actually get the newly install manpages

The user can update the manpage database himself.
And if installing from a package, it should have been done automatically.

Quoted 8 lines

2. always update the local manpage database. In this case, the user installing the docs will find them, but anyone installing to staging will experience a slight delay while their local mandb is updated.

I went for #2 on the basis that the delay in the staging case is pretty harmless, while not actually finding the manpages is more serious.

However, I'm open to other suggestions on how to work this?

My concern is polluting the machine of the packager. What happens when the staging directory is removed? Is it an error when opening a manpage later?

#17 lore raw
Quoted 22 lines · Thomas Monjalon

29/08/2023 12:10, Bruce Richardson:

Quoted 8 lines · Thomas Monjalon
Quoted 6 lines

3/08/2023, Bruce Richardson:

Quoted 1 line

+ meson.add_install_script(mandb)

When is it executed exactly? Will it update the database in case we install in a staging directory, when preparing a package for later deploying on another machine?

Quoted 5 lines

Yes, it will. Unfortunately, I can't find any way to just call mandb if we are installing in a system manpage location on the local machine. Therefore, I had two options: 1. don't update the manpage database. In this case, the user won't be able to actually get the newly install manpages

The user can update the manpage database himself. And if installing from a package, it should have been done automatically.

Quoted 4 lines

2. always update the local manpage database. In this case, the user installing the docs will find them, but anyone installing to staging will experience a slight delay while their local mandb is updated.

Quoted 9 lines
Quoted 4 lines

I went for #2 on the basis that the delay in the staging case is pretty harmless, while not actually finding the manpages is more serious.

However, I'm open to other suggestions on how to work this?

My concern is polluting the machine of the packager. What happens when the staging directory is removed? Is it an error when opening a manpage later?

Running mandb doesn't add the manpages from the staging directory to the database, it simply triggers a scan of the registered manpage directories on the system, adding new pages found there. It's the man equivalent of doing an "ldconfig".

/Bruce

#18 lore raw

30/08/2023 12:20, Bruce Richardson:

Quoted 28 lines · Thomas Monjalon
Quoted 27 lines

29/08/2023 12:10, Bruce Richardson:

Quoted 8 lines · Thomas Monjalon
Quoted 6 lines

3/08/2023, Bruce Richardson:

Quoted 1 line

+ meson.add_install_script(mandb)

When is it executed exactly? Will it update the database in case we install in a staging directory, when preparing a package for later deploying on another machine?

Quoted 5 lines

Yes, it will. Unfortunately, I can't find any way to just call mandb if we are installing in a system manpage location on the local machine. Therefore, I had two options: 1. don't update the manpage database. In this case, the user won't be able to actually get the newly install manpages

The user can update the manpage database himself. And if installing from a package, it should have been done automatically.

Quoted 8 lines

2. always update the local manpage database. In this case, the user installing the docs will find them, but anyone installing to staging will experience a slight delay while their local mandb is updated.

I went for #2 on the basis that the delay in the staging case is pretty harmless, while not actually finding the manpages is more serious.

However, I'm open to other suggestions on how to work this?

Quoted 8 lines
Quoted 4 lines

My concern is polluting the machine of the packager. What happens when the staging directory is removed? Is it an error when opening a manpage later?

Running mandb doesn't add the manpages from the staging directory to the database, it simply triggers a scan of the registered manpage directories on the system, adding new pages found there. It's the man equivalent of doing an "ldconfig".

Ah OK
So there is no more concern, thanks.

#19 lore raw

PATCH v4 1/1 doc: build manpages as well as html output

In reply to Bruce Richardson

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found. If the manpages are not installed in system location i.e. one not in MANPATH, then this update will have no effect, as only system locations are scanned.

---

V4: fix whitespace issues.
add clarifying notes about mandb updating the database by scanning system locations only.

V3: don't use full file paths when generating manpages

V2: add doc update about building or using the manpages

---
 doc/api/doxy-api.conf.in                  |  8 ++--
 doc/api/generate_doxygen.py               |  2 +-
 doc/api/meson.build                       | 58 +++++++++++++++++++----
 doc/guides/contributing/documentation.rst | 12 ++++-
 4 files changed, 67 insertions(+), 13 deletions(-)
doc/api/doxy-api.conf.in +5 -3
index a88accd907..1ba72e6221 100644
@@ -123,11 +123,13 @@ EXAMPLE_PATTERNS        = *.c
 EXAMPLE_RECURSIVE       = YES
 
 OUTPUT_DIRECTORY        = @OUTPUT@
+FULL_PATH_NAMES         = @FULL_PATH_NAMES@
 STRIP_FROM_PATH         = @STRIP_FROM_PATH@
-GENERATE_HTML           = YES
-HTML_OUTPUT             = @HTML_OUTPUT@
+GENERATE_HTML           = @GENERATE_HTML@
+HTML_OUTPUT             = html
 GENERATE_LATEX          = NO
-GENERATE_MAN            = NO
+GENERATE_MAN            = @GENERATE_MAN@
+MAN_LINKS               = YES
 
 HAVE_DOT                = NO
 
doc/api/generate_doxygen.py +1 -1
index d3a22869f6..c704f13018 100755
@@ -7,7 +7,7 @@
 
 pattern = re.compile('^Preprocessing (.*)...$')
 out_dir, *doxygen_command = sys.argv[1:]
-out_file = os.path.join(os.path.dirname(out_dir), 'doxygen.out')
+out_file = os.path.join(out_dir + '.out')
 dep_file = f'{out_dir}.d'
 with open(out_file, 'w') as out:
     subprocess.run(doxygen_command, check=True, stdout=out)
doc/api/meson.build +50 -8

80 lines not on this page · raw message

doc/guides/contributing/documentation.rst +11 -1
index 79616e5610..4eb62fc36a 100644
@@ -182,7 +182,17 @@ To build the documentation::
 
 See :doc:`../linux_gsg/build_dpdk` for more detail on compiling DPDK with meson.
 
-The output is generated in the directories ``build/doc/html/{api,guides}``.
+The output is generated in the directory ``build/doc/``, with:
+
+* HTML versions of the guide docs, e.g. Getting Started Guides, Programmers Guide, in ``build/doc/guides/html``
+* HTML version of the API documentation in ``build/doc/api/html``
+* Man-page version of the API documentation in ``build/doc/api/man``.
+  If not installing DPDK system-wise, these pages can be accessed by adding this directory to the ``MANPATH`` environment variable.
+  For example:
+
+.. code-block:: console
+
+   export MANPATH=:/path/to/build/doc/api/man
 
 .. Note::
 
Signature · 1 line
2.39.2
Acked-by on v4 1/1
#20 lore raw

31/08/2023 11:49, Bruce Richardson:

Quoted 21 lines

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found. If the manpages are not installed in system location i.e. one not in MANPATH, then this update will have no effect, as only system locations are scanned.

Quoted 2 lines
#21 lore raw
Quoted 25 lines

From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, 31 August 2023 12.12

31/08/2023 11:49, Bruce Richardson:

Quoted 21 lines

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found. If the manpages are not installed in system location i.e. one not in MANPATH, then this update will have no effect, as only system locations are scanned.

Quoted 5 lines
Quoted 2 lines

Having followed the discussion where the "cross build" details were clarified...

#22 lore raw

31/08/2023 17:48, Morten Brørup:

Quoted 28 lines
Quoted 28 lines

From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, 31 August 2023 12.12

31/08/2023 11:49, Bruce Richardson:

Quoted 23 lines

Doxygen can produce manpage output as well as html output for the DPDK APIs. However, we need to do this as a separate task as the manpage output needs to be placed in a different location post-install to the html output (/usr/local/share/man vs /usr/local/share/doc/).

Changes required are:
* Add configurable options for manpage output and html output to the doxygen config template. (Remove option for html output path as it's always "html")
* Modify API meson.build file to configure two separate doxygen config files, for HTML and manpages respectively.
* Change doxygen wrapper script to have separate output log files for the manpage and HTML jobs, to avoid conflicts
* Add "custom_targets" to meson.build file to build the HTML pages and the manpages, with individual install locations for each.
* Where supported by meson version, call "mandb" post-install to update the man database to ensure the new manpages can be found. If the manpages are not installed in system location i.e. one not in MANPATH, then this update will have no effect, as only system locations are scanned.

Quoted 6 lines
Quoted 2 lines

Having followed the discussion where the "cross build" details were clarified...

Applied, thanks.