meson build improvements
cover meson build improvements
These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.
Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file
buildtools/call-sphinx-build.py | 29 ++++++++++ buildtools/meson.build | 6 +- doc/api/meson.build | 99 +++++++++++++++++---------------- doc/guides/meson.build | 38 ++++++------- doc/meson.build | 4 ++ kernel/linux/kni/meson.build | 1 - 6 files changed, 104 insertions(+), 73 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
PATCH 1/6 kernel/linux/kni: fix meson warning about console keyword
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
--- kernel/linux/kni/meson.build | 1 - 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
@@ -23,7 +23,6 @@ custom_target('rte_kni', ' -I' + meson.current_source_dir(), 'modules'], depends: kni_mkfile, - console: true, install: true, install_dir: kernel_dir + '/extra/dpdk', build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
PATCH 2/6 build: skip processing docs folder if docs disabled
In reply to Bruce RichardsonWhile each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.
--- doc/meson.build | 4 ++++ 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
@@ -1,6 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <[email protected]> +if not get_option('enable_docs') + subdir_done() +endif + doc_targets = [] doc_target_names = [] subdir('api')
Signature · 1 line
2.24.1
PATCH 3/6 doc/api: fix warning with meson build
In reply to Bruce RichardsonThe install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:
"When omitted it defaults to true when install_dir is set and not empty, false otherwise."
Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.
--- doc/api/meson.build | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
@@ -38,8 +38,7 @@ if doxygen.found() doxy_conf = configure_file(input: 'doxy-api.conf.in', output: 'doxy-api.conf', - configuration: cdata, - install: false) + configuration: cdata) doxy_build = custom_target('doxygen', depends: example,
Signature · 1 line
2.24.1
PATCH 4/6 doc/guides: reduce whitespace in meson build file
In reply to Bruce RichardsonFor building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.
--- doc/guides/meson.build | 44 ++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
@@ -3,26 +3,28 @@ sphinx = find_program('sphinx-build', required: get_option('enable_docs')) -if sphinx.found() - htmldir = join_paths('share', 'doc', 'dpdk') - html_guides_build = custom_target('html_guides_build', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], - build_by_default: get_option('enable_docs'), - install: get_option('enable_docs'), - install_dir: htmldir) +if not sphinx.found() + subdir_done() +endif - doc_targets += html_guides_build - doc_target_names += 'HTML_Guides' +htmldir = join_paths('share', 'doc', 'dpdk') +html_guides = custom_target('html_guides', + input: meson.current_source_dir(), + output: 'guides', + command: [sphinx, '-b', 'html', + '-d', meson.current_build_dir() + '/.doctrees', + '@INPUT@', meson.current_build_dir() + '/guides'], + build_by_default: get_option('enable_docs'), + install: get_option('enable_docs'), + install_dir: htmldir) - # sphinx leaves a .buildinfo in the target directory, which we don't - # want to install. Note that sh -c has to be used, otherwise the - # env var does not get expanded if calling rm/install directly. - meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') - meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css') -endif +doc_targets += html_guides +doc_target_names += 'HTML_Guides' + +# sphinx leaves a .buildinfo in the target directory, which we don't +# want to install. Note that sh -c has to be used, otherwise the +# env var does not get expanded if calling rm/install directly. +meson.add_install_script('sh', '-c', + 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') +meson.add_install_script('sh', '-c', + 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
PATCH 5/6 doc/guides: rebuild with meson whenever a file changes
In reply to Bruce RichardsonAdd proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
--- buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 41 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +29 -0
@@ -0,0 +1,29 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +# + +import sys +import os +import os.path +import subprocess + +sphinx = sys.argv[1] +src = sys.argv[2] +dst = sys.argv[3] +depfile = os.path.join(dst,'.html.d') + +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + for f in files: + srcfiles.append(os.path.join(root, f)) + +# run sphinx, putting the html output in a "html" directory +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src, + os.path.join(dst, 'html')], check = True) + +# create a gcc format .d file giving all the dependencies of this doc build +with open(depfile, 'w') as d: + d.write('html: ' + ' '.join(srcfiles) + '\n') +subprocess.run(['cp', '-f', depfile, '/tmp'])
buildtools/meson.build +4 -2
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external python3 = import('python').find_installation(required: false) if python3.found() - map_to_def_cmd = [python3, files('map_to_def.py')] + py3 = [python3] else - map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')] + py3 = ['meson', 'runpython'] endif +map_to_def_cmd = py3 + files('map_to_def.py') +sphinx_wrapper = py3 + files('call-sphinx-build.py') # stable ABI always starts with "DPDK_" is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
@@ -7,24 +7,18 @@ if not sphinx.found() subdir_done() endif -htmldir = join_paths('share', 'doc', 'dpdk') +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk') html_guides = custom_target('html_guides', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], + input: files('index.rst'), + output: 'html', + command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()], + depfile: '.html.d', build_by_default: get_option('enable_docs'), install: get_option('enable_docs'), install_dir: htmldir) +install_data(files('custom.css'), + install_dir: join_paths(htmldir,'_static', 'css')) + doc_targets += html_guides doc_target_names += 'HTML_Guides' - -# sphinx leaves a .buildinfo in the target directory, which we don't -# want to install. Note that sh -c has to be used, otherwise the -# env var does not get expanded if calling rm/install directly. -meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') -meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
PATCH 6/6 doc/api: reduce indentation in meson build file
In reply to Bruce RichardsonWhen building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.
--- doc/api/meson.build | 98 +++++++++++++++++++++++---------------------- 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
@@ -3,52 +3,54 @@ doxygen = find_program('doxygen', required: get_option('enable_docs')) -if doxygen.found() - # due to the CSS customisation script, which needs to run on a file that - # is in a subdirectory that is created at build time and thus it cannot - # be an individual custom_target, we need to wrap the doxygen call in a - # script to run the CSS modification afterwards - generate_doxygen = find_program('generate_doxygen.sh') - generate_examples = find_program('generate_examples.sh') - generate_css = find_program('doxy-html-custom.sh') - - inputdir = join_paths(meson.source_root(), 'examples') - htmldir = join_paths('share', 'doc', 'dpdk') - - # due to the following bug: https://github.com/mesonbuild/meson/issues/4107 - # if install is set to true it will override build_by_default and it will - # cause the target to always be built. If install were to be always set to - # false it would be impossible to install the docs. - # So use a configure option for now. - example = custom_target('examples.dox', - input: inputdir, - output: 'examples.dox', - command: [generate_examples, '@INPUT@', '@OUTPUT@'], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - cdata = configuration_data() - cdata.set('VERSION', meson.project_version()) - cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) - cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) - cdata.set('HTML_OUTPUT', 'api') - cdata.set('TOPDIR', meson.source_root()) - cdata.set('STRIP_FROM_PATH', meson.source_root()) - - doxy_conf = configure_file(input: 'doxy-api.conf.in', - output: 'doxy-api.conf', - configuration: cdata) - - doxy_build = custom_target('doxygen', - depends: example, - input: doxy_conf, - output: 'api', - command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - doc_targets += doxy_build - doc_target_names += 'Doxygen_API' +if not doxygen.found() + subdir_done() endif + +# due to the CSS customisation script, which needs to run on a file that +# is in a subdirectory that is created at build time and thus it cannot +# be an individual custom_target, we need to wrap the doxygen call in a +# script to run the CSS modification afterwards +generate_doxygen = find_program('generate_doxygen.sh') +generate_examples = find_program('generate_examples.sh') +generate_css = find_program('doxy-html-custom.sh') + +inputdir = join_paths(meson.source_root(), 'examples') +htmldir = join_paths('share', 'doc', 'dpdk') + +# due to the following bug: https://github.com/mesonbuild/meson/issues/4107 +# if install is set to true it will override build_by_default and it will +# cause the target to always be built. If install were to be always set to +# false it would be impossible to install the docs. +# So use a configure option for now. +example = custom_target('examples.dox', + input: inputdir, + output: 'examples.dox', + command: [generate_examples, '@INPUT@', '@OUTPUT@'], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +cdata = configuration_data() +cdata.set('VERSION', meson.project_version()) +cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) +cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) +cdata.set('HTML_OUTPUT', 'api') +cdata.set('TOPDIR', meson.source_root()) +cdata.set('STRIP_FROM_PATH', meson.source_root()) + +doxy_conf = configure_file(input: 'doxy-api.conf.in', + output: 'doxy-api.conf', + configuration: cdata) + +doxy_build = custom_target('doxygen', + depends: example, + input: doxy_conf, + output: 'api', + command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +doc_targets += doxy_build +doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
v2 cover meson build improvements
In reply to Bruce RichardsonThese patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.
V2: resend to correct email addresses
Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file
buildtools/call-sphinx-build.py | 29 ++++++++++ buildtools/meson.build | 6 +- doc/api/meson.build | 99 +++++++++++++++++---------------- doc/guides/meson.build | 38 ++++++------- doc/meson.build | 4 ++ kernel/linux/kni/meson.build | 1 - 6 files changed, 104 insertions(+), 73 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
v2 cover meson build improvements
In reply to Bruce RichardsonThese patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.
V2: resend to correct email addresses
Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file
buildtools/call-sphinx-build.py | 29 ++++++++++ buildtools/meson.build | 6 +- doc/api/meson.build | 99 +++++++++++++++++---------------- doc/guides/meson.build | 38 ++++++------- doc/meson.build | 4 ++ kernel/linux/kni/meson.build | 1 - 6 files changed, 104 insertions(+), 73 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
PATCH v2 1/6 kernel/linux/kni: fix meson warning about console keyword
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
--- kernel/linux/kni/meson.build | 1 - 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
@@ -23,7 +23,6 @@ custom_target('rte_kni', ' -I' + meson.current_source_dir(), 'modules'], depends: kni_mkfile, - console: true, install: true, install_dir: kernel_dir + '/extra/dpdk', build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v2 1/6] kernel/linux/kni: fix meson warning about console keyword
Quoted 9 lines · Bruce Richardson
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
Fixes b78f32cff94d ("kni: support meson build")Signed-off-by Bruce Richardson <[email protected]>---
PATCH v2 2/6 build: skip processing docs folder if docs disabled
In reply to Bruce RichardsonWhile each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.
--- doc/meson.build | 4 ++++ 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
@@ -1,6 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <[email protected]> +if not get_option('enable_docs') + subdir_done() +endif + doc_targets = [] doc_target_names = [] subdir('api')
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v2 2/6] build: skip processing docs folder if docs disabled
Quoted 5 lines · Bruce Richardson
While each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.
Signed-off-by Bruce Richardson <[email protected]>---
Makes sense.
PATCH v2 3/6 doc/api: fix warning with meson build
In reply to Bruce RichardsonThe install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:
"When omitted it defaults to true when install_dir is set and not empty, false otherwise."
Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.
--- doc/api/meson.build | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
@@ -38,8 +38,7 @@ if doxygen.found() doxy_conf = configure_file(input: 'doxy-api.conf.in', output: 'doxy-api.conf', - configuration: cdata, - install: false) + configuration: cdata) doxy_build = custom_target('doxygen', depends: example,
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v2 3/6] doc/api: fix warning with meson build
Quoted 7 lines · Bruce Richardson
The install parameter to
configure_fileis new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:"When omitted it defaults to true when
install_diris set and not empty, false otherwise."
Quoted 8 lines
Given that
install_diris not set for this file, install defaults to false so no need to explicitly specify it.Fixes 720b14db3ae2 ("build: generate API documentation with meson")Signed-off-by Bruce Richardson <[email protected]>---
PATCH v2 4/6 doc/guides: reduce whitespace in meson build file
In reply to Bruce RichardsonFor building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.
--- doc/guides/meson.build | 44 ++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
@@ -3,26 +3,28 @@ sphinx = find_program('sphinx-build', required: get_option('enable_docs')) -if sphinx.found() - htmldir = join_paths('share', 'doc', 'dpdk') - html_guides_build = custom_target('html_guides_build', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], - build_by_default: get_option('enable_docs'), - install: get_option('enable_docs'), - install_dir: htmldir) +if not sphinx.found() + subdir_done() +endif - doc_targets += html_guides_build - doc_target_names += 'HTML_Guides' +htmldir = join_paths('share', 'doc', 'dpdk') +html_guides = custom_target('html_guides', + input: meson.current_source_dir(), + output: 'guides', + command: [sphinx, '-b', 'html', + '-d', meson.current_build_dir() + '/.doctrees', + '@INPUT@', meson.current_build_dir() + '/guides'], + build_by_default: get_option('enable_docs'), + install: get_option('enable_docs'), + install_dir: htmldir) - # sphinx leaves a .buildinfo in the target directory, which we don't - # want to install. Note that sh -c has to be used, otherwise the - # env var does not get expanded if calling rm/install directly. - meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') - meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css') -endif +doc_targets += html_guides +doc_target_names += 'HTML_Guides' + +# sphinx leaves a .buildinfo in the target directory, which we don't +# want to install. Note that sh -c has to be used, otherwise the +# env var does not get expanded if calling rm/install directly. +meson.add_install_script('sh', '-c', + 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') +meson.add_install_script('sh', '-c', + 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v2 4/6] doc/guides: reduce whitespace in meson build file
Quoted 5 lines · Bruce Richardson
For building the guides, we can make the meson.build easier to read by using the
subdir_donefunction to quit early.Signed-off-by Bruce Richardson <[email protected]>---
Nice.
PATCH v2 5/6 doc/guides: rebuild with meson whenever a file changes
In reply to Bruce RichardsonAdd proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
--- buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 41 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +29 -0
@@ -0,0 +1,29 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +# + +import sys +import os +import os.path +import subprocess + +sphinx = sys.argv[1] +src = sys.argv[2] +dst = sys.argv[3] +depfile = os.path.join(dst,'.html.d') + +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + for f in files: + srcfiles.append(os.path.join(root, f)) + +# run sphinx, putting the html output in a "html" directory +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src, + os.path.join(dst, 'html')], check = True) + +# create a gcc format .d file giving all the dependencies of this doc build +with open(depfile, 'w') as d: + d.write('html: ' + ' '.join(srcfiles) + '\n') +subprocess.run(['cp', '-f', depfile, '/tmp'])
buildtools/meson.build +4 -2
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external python3 = import('python').find_installation(required: false) if python3.found() - map_to_def_cmd = [python3, files('map_to_def.py')] + py3 = [python3] else - map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')] + py3 = ['meson', 'runpython'] endif +map_to_def_cmd = py3 + files('map_to_def.py') +sphinx_wrapper = py3 + files('call-sphinx-build.py') # stable ABI always starts with "DPDK_" is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
@@ -7,24 +7,18 @@ if not sphinx.found() subdir_done() endif -htmldir = join_paths('share', 'doc', 'dpdk') +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk') html_guides = custom_target('html_guides', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], + input: files('index.rst'), + output: 'html', + command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()], + depfile: '.html.d', build_by_default: get_option('enable_docs'), install: get_option('enable_docs'), install_dir: htmldir) +install_data(files('custom.css'), + install_dir: join_paths(htmldir,'_static', 'css')) + doc_targets += html_guides doc_target_names += 'HTML_Guides' - -# sphinx leaves a .buildinfo in the target directory, which we don't -# want to install. Note that sh -c has to be used, otherwise the -# env var does not get expanded if calling rm/install directly. -meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') -meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 6 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Quoted 8 lines
Signed-off-by Bruce Richardson <[email protected]>--- buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 41 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Some nits follow - it would be good to clean them up before applying but they are fairly inconsequential.
Quoted 10 lines
buildtools/call-sphinx-build.py new file +4
-0@@ -0,0 +1,29 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +#
buildtools/call-sphinx-build.py line 14
Quoted 10 lines
buildtools/call-sphinx-build.py new file +10
-0+ +import sys +import os +import os.path +import subprocess + +sphinx = sys.argv[1] +src = sys.argv[2] +dst = sys.argv[3] +depfile = os.path.join(dst,'.html.d')
^ whitespace here
buildtools/call-sphinx-build.py line 24
Quoted 10 lines
buildtools/call-sphinx-build.py new file +10
-0+ +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + for f in files: + srcfiles.append(os.path.join(root, f)) + +# run sphinx, putting the html output in a "html" directory +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src, + os.path.join(dst, 'html')], check = True)
^ no whitespace
around =
Quoted 61 lines
buildtools/call-sphinx-build.py new file +5
-0+ +# create a gcc format .d file giving all the dependencies of this doc build +with open(depfile, 'w') as d: + d.write('html: ' + ' '.join(srcfiles) + '\n') +subprocess.run(['cp', '-f', depfile, '/tmp'])buildtools/meson.build +4
-2@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external python3 = import('python').find_installation(required: false) if python3.found() - map_to_def_cmd = [python3, files('map_to_def.py')] + py3 = [python3] else - map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')] + py3 = ['meson', 'runpython'] endif +map_to_def_cmd = py3 + files('map_to_def.py') +sphinx_wrapper = py3 + files('call-sphinx-build.py') # stable ABI always starts with "DPDK_" is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']doc/guides/meson.build +8
-14@@ -7,24 +7,18 @@ if not sphinx.found() subdir_done() endif -htmldir = join_paths('share', 'doc', 'dpdk') +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk') html_guides = custom_target('html_guides', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], + input: files('index.rst'), + output: 'html', + command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()], + depfile: '.html.d', build_by_default: get_option('enable_docs'), install: get_option('enable_docs'), install_dir: htmldir) +install_data(files('custom.css'), + install_dir: join_paths(htmldir,'_static', 'css')) + doc_targets += html_guides doc_target_names += 'HTML_Guides' - -# sphinx leaves a .buildinfo in the target directory, which we don't -# want to install. Note that sh -c has to be used, otherwise the -# env var does not get expanded if calling rm/install directly. -meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') -meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Re: [dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 55 lines · Aaron Conole
Quoted 6 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Quoted 8 lines
Signed-off-by Bruce Richardson <[email protected]>--- buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 41 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.pyAcked-by Aaron Conole <[email protected]>Some nits follow - it would be good to clean them up before applying but they are fairly inconsequential.
Quoted 10 lines
buildtools/call-sphinx-build.py new file +4
-0@@ -0,0 +1,29 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +#buildtools/call-sphinx-build.py line 14
Quoted 10 lines
buildtools/call-sphinx-build.py new file +10
-0+ +import sys +import os +import os.path +import subprocess + +sphinx = sys.argv[1] +src = sys.argv[2] +dst = sys.argv[3] +depfile = os.path.join(dst,'.html.d')^ whitespace here
buildtools/call-sphinx-build.py line 24
Quoted 10 lines
buildtools/call-sphinx-build.py new file +10
-0+ +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + for f in files: + srcfiles.append(os.path.join(root, f)) + +# run sphinx, putting the html output in a "html" directory +subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src, + os.path.join(dst, 'html')], check = True)^ no whitespace
around =
Quoted 5 lines
buildtools/call-sphinx-build.py line 29
Quoted 5 lines
buildtools/call-sphinx-build.py new file +5
-0+ +# create a gcc format .d file giving all the dependencies of this doc build +with open(depfile, 'w') as d: + d.write('html: ' + ' '.join(srcfiles) + '\n') +subprocess.run(['cp', '-f', depfile, '/tmp'])
Ok, will fix. I also see I left in an unnecessary subprocess.run at the end to copy the file to /tmp. This was for debugging only since ninja deletes the .d files after processing them.
[https://ninja-build.org/manual.html#_deps, see behaviour for deps=gcc]
/Bruce
PATCH v2 6/6 doc/api: reduce indentation in meson build file
In reply to Bruce RichardsonWhen building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.
--- doc/api/meson.build | 98 +++++++++++++++++++++++---------------------- 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
@@ -3,52 +3,54 @@ doxygen = find_program('doxygen', required: get_option('enable_docs')) -if doxygen.found() - # due to the CSS customisation script, which needs to run on a file that - # is in a subdirectory that is created at build time and thus it cannot - # be an individual custom_target, we need to wrap the doxygen call in a - # script to run the CSS modification afterwards - generate_doxygen = find_program('generate_doxygen.sh') - generate_examples = find_program('generate_examples.sh') - generate_css = find_program('doxy-html-custom.sh') - - inputdir = join_paths(meson.source_root(), 'examples') - htmldir = join_paths('share', 'doc', 'dpdk') - - # due to the following bug: https://github.com/mesonbuild/meson/issues/4107 - # if install is set to true it will override build_by_default and it will - # cause the target to always be built. If install were to be always set to - # false it would be impossible to install the docs. - # So use a configure option for now. - example = custom_target('examples.dox', - input: inputdir, - output: 'examples.dox', - command: [generate_examples, '@INPUT@', '@OUTPUT@'], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - cdata = configuration_data() - cdata.set('VERSION', meson.project_version()) - cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) - cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) - cdata.set('HTML_OUTPUT', 'api') - cdata.set('TOPDIR', meson.source_root()) - cdata.set('STRIP_FROM_PATH', meson.source_root()) - - doxy_conf = configure_file(input: 'doxy-api.conf.in', - output: 'doxy-api.conf', - configuration: cdata) - - doxy_build = custom_target('doxygen', - depends: example, - input: doxy_conf, - output: 'api', - command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - doc_targets += doxy_build - doc_target_names += 'Doxygen_API' +if not doxygen.found() + subdir_done() endif + +# due to the CSS customisation script, which needs to run on a file that +# is in a subdirectory that is created at build time and thus it cannot +# be an individual custom_target, we need to wrap the doxygen call in a +# script to run the CSS modification afterwards +generate_doxygen = find_program('generate_doxygen.sh') +generate_examples = find_program('generate_examples.sh') +generate_css = find_program('doxy-html-custom.sh') + +inputdir = join_paths(meson.source_root(), 'examples') +htmldir = join_paths('share', 'doc', 'dpdk') + +# due to the following bug: https://github.com/mesonbuild/meson/issues/4107 +# if install is set to true it will override build_by_default and it will +# cause the target to always be built. If install were to be always set to +# false it would be impossible to install the docs. +# So use a configure option for now. +example = custom_target('examples.dox', + input: inputdir, + output: 'examples.dox', + command: [generate_examples, '@INPUT@', '@OUTPUT@'], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +cdata = configuration_data() +cdata.set('VERSION', meson.project_version()) +cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) +cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) +cdata.set('HTML_OUTPUT', 'api') +cdata.set('TOPDIR', meson.source_root()) +cdata.set('STRIP_FROM_PATH', meson.source_root()) + +doxy_conf = configure_file(input: 'doxy-api.conf.in', + output: 'doxy-api.conf', + configuration: cdata) + +doxy_build = custom_target('doxygen', + depends: example, + input: doxy_conf, + output: 'api', + command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +doc_targets += doxy_build +doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v2 6/6] doc/api: reduce indentation in meson build file
Quoted 5 lines · Bruce Richardson
When building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using
subdir_done()to quit early.Signed-off-by Bruce Richardson <[email protected]>---
Quoted 34 lines · Bruce Richardson
These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.V2: resend to correct email addresses
Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file
Quoted 9 lines
buildtools/call-sphinx-build.py | 29 ++++++++++ buildtools/meson.build | 6 +- doc/api/meson.build | 99 +++++++++++++++++------------ ---- doc/guides/meson.build | 38 ++++++------- doc/meson.build | 4 ++ kernel/linux/kni/meson.build | 1 - 6 files changed, 104 insertions(+), 73 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Signature · 2 lines
Kind regards, Luca Boccassi
v3 cover meson build improvements
In reply to Bruce RichardsonThese patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.
V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.
V2: resend to correct email addresses
Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file
buildtools/call-sphinx-build.py | 23 ++++++++ buildtools/meson.build | 6 +- doc/api/meson.build | 99 +++++++++++++++++---------------- doc/guides/meson.build | 38 ++++++------- doc/meson.build | 4 ++ kernel/linux/kni/meson.build | 1 - 6 files changed, 98 insertions(+), 73 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
PATCH v3 1/6 kernel/linux/kni: fix meson warning about console keyword
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
--- kernel/linux/kni/meson.build | 1 - 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
@@ -23,7 +23,6 @@ custom_target('rte_kni', ' -I' + meson.current_source_dir(), 'modules'], depends: kni_mkfile, - console: true, install: true, install_dir: kernel_dir + '/extra/dpdk', build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword
Quoted 7 lines · Bruce Richardson
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
Fixes b78f32cff94d ("kni: support meson build")
Since this is linked to kni ethtool stuff removed from 19.11, I would go with a Cc: [email protected].
Then stable maintainers can ignore for everything but 19.11 ?
--
David Marchand
Re: [dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword
Quoted 7 lines · David Marchand
Quoted 5 lines · Bruce Richardson
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
Quoted 7 lines
Quoted 2 lines
Fixes b78f32cff94d ("kni: support meson build")Since this is linked to kni ethtool stuff removed from 19.11, I would go with a Cc: [email protected].
Then stable maintainers can ignore for everything but 19.11 ?
+1
Quoted 4 lines
--
David Marchand
PATCH v3 2/6 build: skip processing docs folder if docs disabled
In reply to Bruce RichardsonWhile each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.
--- doc/meson.build | 4 ++++ 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
@@ -1,6 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <[email protected]> +if not get_option('enable_docs') + subdir_done() +endif + doc_targets = [] doc_target_names = [] subdir('api')
Signature · 1 line
2.24.1
PATCH v3 3/6 doc/api: fix warning with meson build
In reply to Bruce RichardsonThe install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:
"When omitted it defaults to true when install_dir is set and not empty, false otherwise."
Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.
--- doc/api/meson.build | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
@@ -38,8 +38,7 @@ if doxygen.found() doxy_conf = configure_file(input: 'doxy-api.conf.in', output: 'doxy-api.conf', - configuration: cdata, - install: false) + configuration: cdata) doxy_build = custom_target('doxygen', depends: example,
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build
Quoted 5 lines · Bruce Richardson
The install parameter to
configure_fileis new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:
Quoted 7 lines
"When omitted it defaults to true when
install_diris set and not empty, false otherwise."Given that
install_diris not set for this file, install defaults to false so no need to explicitly specify it.Fixes 720b14db3ae2 ("build: generate API documentation with meson")
--
David Marchand
Re: [dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build
Quoted 10 lines · David Marchand
Quoted 8 lines · Bruce Richardson
The install parameter to
configure_fileis new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:"When omitted it defaults to true when
install_diris set and not empty, false otherwise."
Quoted 7 lines
Quoted 4 lines
Given that
install_diris not set for this file, install defaults to false so no need to explicitly specify it.Fixes 720b14db3ae2 ("build: generate API documentation with meson")Cc [email protected] ? or does it depend on other changes?
I don't think it requires any other changes. Should probably go to 19.11 anyway - the 18.11 tree can skip it, I think.
PATCH v3 4/6 doc/guides: reduce whitespace in meson build file
In reply to Bruce RichardsonFor building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.
--- doc/guides/meson.build | 44 ++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
@@ -3,26 +3,28 @@ sphinx = find_program('sphinx-build', required: get_option('enable_docs')) -if sphinx.found() - htmldir = join_paths('share', 'doc', 'dpdk') - html_guides_build = custom_target('html_guides_build', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], - build_by_default: get_option('enable_docs'), - install: get_option('enable_docs'), - install_dir: htmldir) +if not sphinx.found() + subdir_done() +endif - doc_targets += html_guides_build - doc_target_names += 'HTML_Guides' +htmldir = join_paths('share', 'doc', 'dpdk') +html_guides = custom_target('html_guides', + input: meson.current_source_dir(), + output: 'guides', + command: [sphinx, '-b', 'html', + '-d', meson.current_build_dir() + '/.doctrees', + '@INPUT@', meson.current_build_dir() + '/guides'], + build_by_default: get_option('enable_docs'), + install: get_option('enable_docs'), + install_dir: htmldir) - # sphinx leaves a .buildinfo in the target directory, which we don't - # want to install. Note that sh -c has to be used, otherwise the - # env var does not get expanded if calling rm/install directly. - meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') - meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css') -endif +doc_targets += html_guides +doc_target_names += 'HTML_Guides' + +# sphinx leaves a .buildinfo in the target directory, which we don't +# want to install. Note that sh -c has to be used, otherwise the +# env var does not get expanded if calling rm/install directly. +meson.add_install_script('sh', '-c', + 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') +meson.add_install_script('sh', '-c', + 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
PATCH v3 5/6 doc/guides: rebuild with meson whenever a file changes
In reply to Bruce RichardsonAdd proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
--- buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 35 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +23 -0
@@ -0,0 +1,23 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +# + +import sys +import os +from os.path import join +from subprocess import run + +(sphinx, src, dst) = sys.argv[1:] # assign parameters to variables + +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + srcfiles.extend([join(root, f) for f in files]) + +# run sphinx, putting the html output in a "html" directory +run([sphinx, '-j', 'auto', '-b', 'html', src, join(dst, 'html')], check=True) + +# create a gcc format .d file giving all the dependencies of this doc build +with open(join(dst, '.html.d'), 'w') as d: + d.write('html: ' + ' '.join(srcfiles) + '\n')
buildtools/meson.build +4 -2
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external python3 = import('python').find_installation(required: false) if python3.found() - map_to_def_cmd = [python3, files('map_to_def.py')] + py3 = [python3] else - map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')] + py3 = ['meson', 'runpython'] endif +map_to_def_cmd = py3 + files('map_to_def.py') +sphinx_wrapper = py3 + files('call-sphinx-build.py') # stable ABI always starts with "DPDK_" is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
@@ -7,24 +7,18 @@ if not sphinx.found() subdir_done() endif -htmldir = join_paths('share', 'doc', 'dpdk') +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk') html_guides = custom_target('html_guides', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], + input: files('index.rst'), + output: 'html', + command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()], + depfile: '.html.d', build_by_default: get_option('enable_docs'), install: get_option('enable_docs'), install_dir: htmldir) +install_data(files('custom.css'), + install_dir: join_paths(htmldir,'_static', 'css')) + doc_targets += html_guides doc_target_names += 'HTML_Guides' - -# sphinx leaves a .buildinfo in the target directory, which we don't -# want to install. Note that sh -c has to be used, otherwise the -# env var does not get expanded if calling rm/install directly. -meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') -meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 7 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Quoted 8 lines
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>--- buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 35 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
This new file is not added to MAINTAINERS.
--
David Marchand
Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 19 lines · David Marchand
Quoted 7 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Quoted 8 lines
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>--- buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 35 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Quoted 1 line
This new file is not added to
MAINTAINERS.
I can fix this when applying.
I would add this to the section:
Meson build
M: Bruce Richardson <[email protected]>
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 20 lines · David Marchand
Quoted 19 lines · David Marchand
Quoted 15 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>--- buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 35 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Quoted 8 lines
Quoted 1 line
This new file is not added to
MAINTAINERS.I can fix this when applying.
I would add this to the section:Meson build
M: Bruce Richardson <[email protected]>
Agreed.
Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes
In reply to Bruce RichardsonQuoted 31 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>--- buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 3 files changed, 35 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.pybuildtools/call-sphinx-build.py new file +10
-0@@ -0,0 +1,23 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +# + +import sys +import os +from os.path import join +from subprocess import run +
buildtools/call-sphinx-build.py line 20
Quoted 10 lines
buildtools/call-sphinx-build.py new file +10
-0+(sphinx, src, dst) = sys.argv[1:] # assign parameters to variables + +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + srcfiles.extend([join(root, f) for f in files]) + +# run sphinx, putting the html output in a "html" directory +run([sphinx, '-j', 'auto', '-b', 'html', src, join(dst, 'html')], check=True) +
Looking in travis, it appears that sphinx there is too old to support the "-j auto" flag, so I'll do a new revision to fix that.
/Bruce
PATCH v3 6/6 doc/api: reduce indentation in meson build file
In reply to Bruce RichardsonWhen building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.
--- doc/api/meson.build | 98 +++++++++++++++++++++++---------------------- 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
@@ -3,52 +3,54 @@ doxygen = find_program('doxygen', required: get_option('enable_docs')) -if doxygen.found() - # due to the CSS customisation script, which needs to run on a file that - # is in a subdirectory that is created at build time and thus it cannot - # be an individual custom_target, we need to wrap the doxygen call in a - # script to run the CSS modification afterwards - generate_doxygen = find_program('generate_doxygen.sh') - generate_examples = find_program('generate_examples.sh') - generate_css = find_program('doxy-html-custom.sh') - - inputdir = join_paths(meson.source_root(), 'examples') - htmldir = join_paths('share', 'doc', 'dpdk') - - # due to the following bug: https://github.com/mesonbuild/meson/issues/4107 - # if install is set to true it will override build_by_default and it will - # cause the target to always be built. If install were to be always set to - # false it would be impossible to install the docs. - # So use a configure option for now. - example = custom_target('examples.dox', - input: inputdir, - output: 'examples.dox', - command: [generate_examples, '@INPUT@', '@OUTPUT@'], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - cdata = configuration_data() - cdata.set('VERSION', meson.project_version()) - cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) - cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) - cdata.set('HTML_OUTPUT', 'api') - cdata.set('TOPDIR', meson.source_root()) - cdata.set('STRIP_FROM_PATH', meson.source_root()) - - doxy_conf = configure_file(input: 'doxy-api.conf.in', - output: 'doxy-api.conf', - configuration: cdata) - - doxy_build = custom_target('doxygen', - depends: example, - input: doxy_conf, - output: 'api', - command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - doc_targets += doxy_build - doc_target_names += 'Doxygen_API' +if not doxygen.found() + subdir_done() endif + +# due to the CSS customisation script, which needs to run on a file that +# is in a subdirectory that is created at build time and thus it cannot +# be an individual custom_target, we need to wrap the doxygen call in a +# script to run the CSS modification afterwards +generate_doxygen = find_program('generate_doxygen.sh') +generate_examples = find_program('generate_examples.sh') +generate_css = find_program('doxy-html-custom.sh') + +inputdir = join_paths(meson.source_root(), 'examples') +htmldir = join_paths('share', 'doc', 'dpdk') + +# due to the following bug: https://github.com/mesonbuild/meson/issues/4107 +# if install is set to true it will override build_by_default and it will +# cause the target to always be built. If install were to be always set to +# false it would be impossible to install the docs. +# So use a configure option for now. +example = custom_target('examples.dox', + input: inputdir, + output: 'examples.dox', + command: [generate_examples, '@INPUT@', '@OUTPUT@'], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +cdata = configuration_data() +cdata.set('VERSION', meson.project_version()) +cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) +cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) +cdata.set('HTML_OUTPUT', 'api') +cdata.set('TOPDIR', meson.source_root()) +cdata.set('STRIP_FROM_PATH', meson.source_root()) + +doxy_conf = configure_file(input: 'doxy-api.conf.in', + output: 'doxy-api.conf', + configuration: cdata) + +doxy_build = custom_target('doxygen', + depends: example, + input: doxy_conf, + output: 'api', + command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +doc_targets += doxy_build +doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
v4 cover meson build improvements
In reply to Bruce RichardsonThese patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.
V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.
V2: resend to correct email addresses
Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file
MAINTAINERS | 1 + buildtools/call-sphinx-build.py | 31 +++++++++++ buildtools/meson.build | 6 +- doc/api/meson.build | 99 +++++++++++++++++---------------- doc/guides/meson.build | 38 ++++++------- doc/meson.build | 4 ++ kernel/linux/kni/meson.build | 1 - 7 files changed, 107 insertions(+), 73 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
PATCH v4 1/6 kernel/linux/kni: fix meson warning about console keyword
Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.
--- kernel/linux/kni/meson.build | 1 - 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
@@ -23,7 +23,6 @@ custom_target('rte_kni', ' -I' + meson.current_source_dir(), 'modules'], depends: kni_mkfile, - console: true, install: true, install_dir: kernel_dir + '/extra/dpdk', build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
PATCH v4 2/6 build: skip processing docs folder if docs disabled
In reply to Bruce RichardsonWhile each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.
--- doc/meson.build | 4 ++++ 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
@@ -1,6 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <[email protected]> +if not get_option('enable_docs') + subdir_done() +endif + doc_targets = [] doc_target_names = [] subdir('api')
Signature · 1 line
2.24.1
PATCH v4 3/6 doc/api: fix warning with meson build
In reply to Bruce RichardsonThe install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:
"When omitted it defaults to true when install_dir is set and not empty, false otherwise."
Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.
--- doc/api/meson.build | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
@@ -38,8 +38,7 @@ if doxygen.found() doxy_conf = configure_file(input: 'doxy-api.conf.in', output: 'doxy-api.conf', - configuration: cdata, - install: false) + configuration: cdata) doxy_build = custom_target('doxygen', depends: example,
Signature · 1 line
2.24.1
PATCH v4 4/6 doc/guides: reduce whitespace in meson build file
In reply to Bruce RichardsonFor building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.
---
@stable, this may be considered for 19.11 backport to make other backporting to this file easier.
--- doc/guides/meson.build | 44 ++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
@@ -3,26 +3,28 @@ sphinx = find_program('sphinx-build', required: get_option('enable_docs')) -if sphinx.found() - htmldir = join_paths('share', 'doc', 'dpdk') - html_guides_build = custom_target('html_guides_build', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], - build_by_default: get_option('enable_docs'), - install: get_option('enable_docs'), - install_dir: htmldir) +if not sphinx.found() + subdir_done() +endif - doc_targets += html_guides_build - doc_target_names += 'HTML_Guides' +htmldir = join_paths('share', 'doc', 'dpdk') +html_guides = custom_target('html_guides', + input: meson.current_source_dir(), + output: 'guides', + command: [sphinx, '-b', 'html', + '-d', meson.current_build_dir() + '/.doctrees', + '@INPUT@', meson.current_build_dir() + '/guides'], + build_by_default: get_option('enable_docs'), + install: get_option('enable_docs'), + install_dir: htmldir) - # sphinx leaves a .buildinfo in the target directory, which we don't - # want to install. Note that sh -c has to be used, otherwise the - # env var does not get expanded if calling rm/install directly. - meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') - meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css') -endif +doc_targets += html_guides +doc_target_names += 'HTML_Guides' + +# sphinx leaves a .buildinfo in the target directory, which we don't +# want to install. Note that sh -c has to be used, otherwise the +# env var does not get expanded if calling rm/install directly. +meson.add_install_script('sh', '-c', + 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') +meson.add_install_script('sh', '-c', + 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
PATCH v4 5/6 doc/guides: rebuild with meson whenever a file changes
In reply to Bruce RichardsonAdd proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
--- MAINTAINERS | 1 + buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 4 files changed, 44 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
MAINTAINERS +1 -0
@@ -128,6 +128,7 @@ F: meson.build F: lib/librte_eal/freebsd/BSDmakefile.meson F: meson_options.txt F: config/rte_config.h +F: buildtools/call-sphinx-build.py F: buildtools/gen-pmdinfo-cfile.sh F: buildtools/map_to_def.py F: buildtools/symlink-drivers-solibs.sh
buildtools/call-sphinx-build.py new file +31 -0
@@ -0,0 +1,31 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019 Intel Corporation +# + +import sys +import os +from os.path import join +from subprocess import run, PIPE +from distutils.version import StrictVersion + +(sphinx, src, dst) = sys.argv[1:] # assign parameters to variables + +# for sphinx version >= 1.7 add parallelism using "-j auto" +ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1] +sphinx_cmd = [sphinx] +if StrictVersion(ver) >= StrictVersion('1.7'): + sphinx_cmd += ['-j', 'auto'] + +# find all the files sphinx will process so we can write them as dependencies +srcfiles = [] +for root, dirs, files in os.walk(src): + srcfiles.extend([join(root, f) for f in files]) + +# run sphinx, putting the html output in a "html" directory +process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')], check=True) +print(str(process.args) + ' Done OK') + +# create a gcc format .d file giving all the dependencies of this doc build +with open(join(dst, '.html.d'), 'w') as d: + d.write('html: ' + ' '.join(srcfiles) + '\n')
buildtools/meson.build +4 -2
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external python3 = import('python').find_installation(required: false) if python3.found() - map_to_def_cmd = [python3, files('map_to_def.py')] + py3 = [python3] else - map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')] + py3 = ['meson', 'runpython'] endif +map_to_def_cmd = py3 + files('map_to_def.py') +sphinx_wrapper = py3 + files('call-sphinx-build.py') # stable ABI always starts with "DPDK_" is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
@@ -7,24 +7,18 @@ if not sphinx.found() subdir_done() endif -htmldir = join_paths('share', 'doc', 'dpdk') +htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk') html_guides = custom_target('html_guides', - input: meson.current_source_dir(), - output: 'guides', - command: [sphinx, '-b', 'html', - '-d', meson.current_build_dir() + '/.doctrees', - '@INPUT@', meson.current_build_dir() + '/guides'], + input: files('index.rst'), + output: 'html', + command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()], + depfile: '.html.d', build_by_default: get_option('enable_docs'), install: get_option('enable_docs'), install_dir: htmldir) +install_data(files('custom.css'), + install_dir: join_paths(htmldir,'_static', 'css')) + doc_targets += html_guides doc_target_names += 'HTML_Guides' - -# sphinx leaves a .buildinfo in the target directory, which we don't -# want to install. Note that sh -c has to be used, otherwise the -# env var does not get expanded if calling rm/install directly. -meson.add_install_script('sh', '-c', - 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo') -meson.add_install_script('sh', '-c', - 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 10 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>Acked-by Luca Boccassi <[email protected]>
Quoted 7 lines
--- MAINTAINERS | 1 + buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 4 files changed, 44 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.py
Caught an issue on fc30:
[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py /usr/libexec/python2-sphinx/sphinx-build /home/dmarchan/dpdk/doc/guides /home/dmarchan/builds/build-x86-default/doc/guides sphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
IndexError: list index out of range
Double checked, the version went to stderr:
$ sphinx-build --version >/dev/null sphinx-build 1.8.4 $ sphinx-build --version 2>/dev/null
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes
Quoted 31 lines · David Marchand
Quoted 10 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>Acked-by Luca Boccassi <[email protected]>Quoted 7 lines
--- MAINTAINERS | 1 + buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 4 files changed, 44 insertions(+), 16 deletions(-) create mode 100755 buildtools/call-sphinx-build.pyCaught an issue on fc30:
[2094/2338] Generating
html_guideswith a custom command.
FAILED: doc/guides/html/usr/bin/python3../../dpdk/buildtools/call-sphinx-build.py/usr/libexec/python2-sphinx/sphinx-build/home/dmarchan/dpdk/doc/guides/home/dmarchan/builds/build-x86-default/doc/guidessphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
Quoted 9 lines
ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]IndexError: list index out of range
Double checked, the version went to stderr:
$ sphinx-build --version >/dev/null sphinx-build 1.8.4 $ sphinx-build --version 2>/dev/null
That is strange. Didn't hit any issues on fedora 31 which I am using here. I'll see if I can make it work with both stderr and stdout reporting.
/Bruce
Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes
In reply to David MarchandQuoted 44 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, January 14, 2020 3:59 PM
To: Richardson, Bruce <[email protected]>
Cc: Mcnamara, John <[email protected]>; Luca Boccassi <[email protected]>; dev <[email protected]>; Aaron Conole <[email protected]>; Thomas Monjalon <[email protected]>
Subject: Re: [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changesQuoted 10 lines · Bruce Richardson
Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Aaron Conole <[email protected]>Acked-by Luca Boccassi <[email protected]>Quoted 7 lines
--- MAINTAINERS | 1 + buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++ buildtools/meson.build | 6 ++++-- doc/guides/meson.build | 22 ++++++++-------------- 4 files changed, 44 insertions(+), 16 deletions(-) create mode100755
buildtools/call-sphinx-build.pyCaught an issue on fc30:
[2094/2338] Generating
html_guideswith a custom command.
FAILED: doc/guides/html/usr/bin/python3../../dpdk/buildtools/call-sphinx-build.py/usr/libexec/python2-sphinx/sphinx-build/home/dmarchan/dpdk/doc/guides/home/dmarchan/builds/build-x86-default/doc/guidessphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>ver = run([sphinx, '--version'],
Quoted 9 lines
stdout=PIPE).stdout.decode().split()[-1]
IndexError: list index out of rangeDouble checked, the version went to stderr:
$ sphinx-build --version >/dev/null sphinx-build 1.8.4 $ sphinx-build --version 2>/dev/null
The series http://patches.dpdk.org/project/dpdk/list/?series=8190 should fix this issue and also adds support for the "werror" option for building the guides.
/Bruce
PATCH v4 6/6 doc/api: reduce indentation in meson build file
In reply to Bruce RichardsonWhen building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.
---
@stable, this patch may be considered for backporting to 19.11 to allow easier merging of other patches to the affected file.
--- doc/api/meson.build | 98 +++++++++++++++++++++++---------------------- 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
@@ -3,52 +3,54 @@ doxygen = find_program('doxygen', required: get_option('enable_docs')) -if doxygen.found() - # due to the CSS customisation script, which needs to run on a file that - # is in a subdirectory that is created at build time and thus it cannot - # be an individual custom_target, we need to wrap the doxygen call in a - # script to run the CSS modification afterwards - generate_doxygen = find_program('generate_doxygen.sh') - generate_examples = find_program('generate_examples.sh') - generate_css = find_program('doxy-html-custom.sh') - - inputdir = join_paths(meson.source_root(), 'examples') - htmldir = join_paths('share', 'doc', 'dpdk') - - # due to the following bug: https://github.com/mesonbuild/meson/issues/4107 - # if install is set to true it will override build_by_default and it will - # cause the target to always be built. If install were to be always set to - # false it would be impossible to install the docs. - # So use a configure option for now. - example = custom_target('examples.dox', - input: inputdir, - output: 'examples.dox', - command: [generate_examples, '@INPUT@', '@OUTPUT@'], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - cdata = configuration_data() - cdata.set('VERSION', meson.project_version()) - cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) - cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) - cdata.set('HTML_OUTPUT', 'api') - cdata.set('TOPDIR', meson.source_root()) - cdata.set('STRIP_FROM_PATH', meson.source_root()) - - doxy_conf = configure_file(input: 'doxy-api.conf.in', - output: 'doxy-api.conf', - configuration: cdata) - - doxy_build = custom_target('doxygen', - depends: example, - input: doxy_conf, - output: 'api', - command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], - install: get_option('enable_docs'), - install_dir: htmldir, - build_by_default: get_option('enable_docs')) - - doc_targets += doxy_build - doc_target_names += 'Doxygen_API' +if not doxygen.found() + subdir_done() endif + +# due to the CSS customisation script, which needs to run on a file that +# is in a subdirectory that is created at build time and thus it cannot +# be an individual custom_target, we need to wrap the doxygen call in a +# script to run the CSS modification afterwards +generate_doxygen = find_program('generate_doxygen.sh') +generate_examples = find_program('generate_examples.sh') +generate_css = find_program('doxy-html-custom.sh') + +inputdir = join_paths(meson.source_root(), 'examples') +htmldir = join_paths('share', 'doc', 'dpdk') + +# due to the following bug: https://github.com/mesonbuild/meson/issues/4107 +# if install is set to true it will override build_by_default and it will +# cause the target to always be built. If install were to be always set to +# false it would be impossible to install the docs. +# So use a configure option for now. +example = custom_target('examples.dox', + input: inputdir, + output: 'examples.dox', + command: [generate_examples, '@INPUT@', '@OUTPUT@'], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +cdata = configuration_data() +cdata.set('VERSION', meson.project_version()) +cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox')) +cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api')) +cdata.set('HTML_OUTPUT', 'api') +cdata.set('TOPDIR', meson.source_root()) +cdata.set('STRIP_FROM_PATH', meson.source_root()) + +doxy_conf = configure_file(input: 'doxy-api.conf.in', + output: 'doxy-api.conf', + configuration: cdata) + +doxy_build = custom_target('doxygen', + depends: example, + input: doxy_conf, + output: 'api', + command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css], + install: get_option('enable_docs'), + install_dir: htmldir, + build_by_default: get_option('enable_docs')) + +doc_targets += doxy_build +doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
Quoted 21 lines · Bruce Richardson
These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.
Quoted 6 lines
V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.V2: resend to correct email addresses
Tried this series in Travis.
It went fine through my own checks too.
But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04. The packaging fails when ninja can't find the 'doc' target.
--
David Marchand
Quoted 28 lines · David Marchand
Quoted 26 lines · Bruce Richardson
These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as flagged by Aaron.
Quoted 7 lines
Quoted 1 line
V2: resend to correct email addresses
Tried this series in Travis. It went fine through my own checks too.
But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04. The packaging fails when ninja can't find the 'doc' target.
Can you send on a link to the failure logs?
I suspect this may be due to returning early from the doc folder if the document building is disabled (but need to see logs to confirm). However, I'm not sure it makes sense to have a doc target defined if doc building is disabled in build config.
Quoted 37 lines · Bruce Richardson
Quoted 28 lines · David Marchand
Quoted 26 lines · Bruce Richardson
These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.
The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.
For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.
Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as flagged by Aaron.
Quoted 7 lines
Quoted 1 line
V2: resend to correct email addresses
Tried this series in Travis. It went fine through my own checks too.
But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04. The packaging fails when ninja can't find the 'doc' target.
Quoted 7 lines
Can you send on a link to the failure logs?
I suspect this may be due to returning early from the doc folder if the document building is disabled (but need to see logs to confirm). However, I'm not sure it makes sense to have a doc target defined if doc building is disabled in build config.
Currently restarting my jobs and I might lose the logs, so I saved it here:
https://paste.centos.org/view/7d6e0545
Signature · 1 line
David Marchand
Quoted 53 lines · Bruce Richardson
Quoted 40 lines · David Marchand
On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<
[email protected]Quoted 37 lines
wrote:
These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued
by meson e.g. warnings about newer features unsupported in baseline.The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.Quoted 9 lines
Quoted 1 line
V2: resend to correct email addresses
Tried this series in Travis. It went fine through my own checks too.
But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.Can you send on a link to the failure logs?
Quoted 7 lines
I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.
Sorry, should have thought about this when reviewing: the usefulness of having the target work without the option set is that the docs will only be built when called explicitly, without having to change the configuration.
Given the doc build is very slow and it's arch-independent, in Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).
Signature · 2 lines
Kind regards, Luca Boccassi
Quoted 62 lines · Luca Boccassi
Quoted 53 lines · Bruce Richardson
Quoted 40 lines · David Marchand
On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<
[email protected]Quoted 37 lines
wrote:
These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued
by meson e.g. warnings about newer features unsupported in baseline.The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.Quoted 9 lines
Quoted 1 line
V2: resend to correct email addresses
Tried this series in Travis. It went fine through my own checks too.
But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.Can you send on a link to the failure logs?
Quoted 7 lines
I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.
Quoted 9 lines
Sorry, should have thought about this when reviewing: the usefulness of having the target work without the option set is that the docs will only be built when called explicitly, without having to change the configuration.
Given the doc build is very slow and it's arch-independent, in Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).
Right. I think the offending patch #2, can just be dropped from the set.
/Bruce
Quoted 69 lines · Bruce Richardson
Quoted 67 lines · Luca Boccassi
Quoted 53 lines · Bruce Richardson
Quoted 40 lines · David Marchand
On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<
[email protected]Quoted 37 lines
wrote:
These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued
by meson e.g. warnings about newer features unsupported in baseline.The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.V4: fix build with older sphinx
V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.Quoted 9 lines
Quoted 1 line
V2: resend to correct email addresses
Tried this series in Travis. It went fine through my own checks too.
But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.Can you send on a link to the failure logs?
Quoted 7 lines
I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.Sorry, should have thought about this when reviewing: the usefulness of having the target work without the option set is that the docs will only be built when called explicitly, without having to change the configuration.
Quoted 5 lines
Quoted 4 lines
Given the doc build is very slow and it's arch-independent, in Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).Right. I think the offending patch #2, can just be dropped from the set.
Ok, I dropped this patch.
Packages for debian distributions in OBS build fine now.
I am ok with a followup patch for fc30, I disabled the doc check in my env for now.
Series & ~patch2 applied.
Thanks.
--
David Marchand