Skip to contentSkip to conversation

meson build improvements

PATCH Superseded Bruce Richardson 5 participants 54 messages Jan 9, 2020 – Jan 17, 2020 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

cover meson build improvements

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
#1 lore raw

PATCH 1/6 kernel/linux/kni: fix meson warning about console keyword

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Fixes b78f32cff94d ("kni: support meson build")
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
index 955eec949..f93e97fa0 100644
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
#2 lore raw

PATCH 2/6 build: skip processing docs folder if docs disabled

In reply to Bruce Richardson

While each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.

---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
index c5410d85d..c49ec8476 100644
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
Signature · 1 line
2.24.1
#3 lore raw

PATCH 3/6 doc/api: fix warning with meson build

In reply to Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
index 1c48b7672..23a7dfc75 100644
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
Signature · 1 line
2.24.1
#4 lore raw

PATCH 4/6 doc/guides: reduce whitespace in meson build file

In reply to Bruce Richardson

For building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.

---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
index 7931ef3bb..80c21d168 100644
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#5 lore raw

PATCH 5/6 doc/guides: rebuild with meson whenever a file changes

In reply to Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +29 -0
new file mode 100755
index 000000000..027317b9b
@@ -0,0 +1,29 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+import os.path
+import subprocess
+
+sphinx = sys.argv[1]
+src = sys.argv[2]
+dst = sys.argv[3]
+depfile = os.path.join(dst,'.html.d')
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    for f in files:
+        srcfiles.append(os.path.join(root, f))
+
+# run sphinx, putting the html output in a "html" directory
+subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
+                os.path.join(dst, 'html')], check = True)
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(depfile, 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
+subprocess.run(['cp', '-f', depfile, '/tmp'])
buildtools/meson.build +4 -2
index 6ef2c5721..cd1d05403 100644
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
index 80c21d168..732e7ad3a 100644
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#6 lore raw

PATCH 6/6 doc/api: reduce indentation in meson build file

In reply to Bruce Richardson

When building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.

---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
index 23a7dfc75..c72b880e1 100644
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
#7 lore raw

v2 cover meson build improvements

In reply to Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

V2: resend to correct email addresses

Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
#8 lore raw

v2 cover meson build improvements

In reply to Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

V2: resend to correct email addresses

Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
#9 lore raw

PATCH v2 1/6 kernel/linux/kni: fix meson warning about console keyword

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Fixes b78f32cff94d ("kni: support meson build")
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
index 955eec949..f93e97fa0 100644
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
Acked-by on v2 1/6
#10 lore raw

Re: [dpdk-dev] [PATCH v2 1/6] kernel/linux/kni: fix meson warning about console keyword

Quoted 9 lines · Bruce Richardson

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Fixes b78f32cff94d ("kni: support meson build")

---

#11 lore raw

PATCH v2 2/6 build: skip processing docs folder if docs disabled

In reply to Bruce Richardson

While each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.

---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
index c5410d85d..c49ec8476 100644
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
Signature · 1 line
2.24.1
#13 lore raw

PATCH v2 3/6 doc/api: fix warning with meson build

In reply to Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
index 1c48b7672..23a7dfc75 100644
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
Signature · 1 line
2.24.1
Acked-by on v2 3/6
#14 lore raw

Re: [dpdk-dev] [PATCH v2 3/6] doc/api: fix warning with meson build

Quoted 7 lines · Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Quoted 8 lines

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")

---

#15 lore raw

PATCH v2 4/6 doc/guides: reduce whitespace in meson build file

In reply to Bruce Richardson

For building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.

---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
index 7931ef3bb..80c21d168 100644
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#17 lore raw

PATCH v2 5/6 doc/guides: rebuild with meson whenever a file changes

In reply to Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +29 -0
new file mode 100755
index 000000000..027317b9b
@@ -0,0 +1,29 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+import os.path
+import subprocess
+
+sphinx = sys.argv[1]
+src = sys.argv[2]
+dst = sys.argv[3]
+depfile = os.path.join(dst,'.html.d')
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    for f in files:
+        srcfiles.append(os.path.join(root, f))
+
+# run sphinx, putting the html output in a "html" directory
+subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
+                os.path.join(dst, 'html')], check = True)
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(depfile, 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
+subprocess.run(['cp', '-f', depfile, '/tmp'])
buildtools/meson.build +4 -2
index 6ef2c5721..cd1d05403 100644
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
index 80c21d168..732e7ad3a 100644
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
Acked-by on v2 5/6
#18 lore raw

Re: [dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 6 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 8 lines
---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

Some nits follow - it would be good to clean them up before applying but they are fairly inconsequential.

Quoted 10 lines
buildtools/call-sphinx-build.py new file +4 -0
new file mode 100755
index 000000000..027317b9b
@@ -0,0 +1,29 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#

buildtools/call-sphinx-build.py line 14

Quoted 10 lines
buildtools/call-sphinx-build.py new file +10 -0
+
+import sys
+import os
+import os.path
+import subprocess
+
+sphinx = sys.argv[1]
+src = sys.argv[2]
+dst = sys.argv[3]
+depfile = os.path.join(dst,'.html.d')

^ whitespace here

buildtools/call-sphinx-build.py line 24

Quoted 10 lines
buildtools/call-sphinx-build.py new file +10 -0
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    for f in files:
+        srcfiles.append(os.path.join(root, f))
+
+# run sphinx, putting the html output in a "html" directory
+subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
+                os.path.join(dst, 'html')], check = True)

^ no whitespace
around =

Quoted 61 lines
buildtools/call-sphinx-build.py new file +5 -0
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(depfile, 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
+subprocess.run(['cp', '-f', depfile, '/tmp'])
buildtools/meson.build +4 -2
index 6ef2c5721..cd1d05403 100644
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
index 80c21d168..732e7ad3a 100644
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
#19 lore raw

Re: [dpdk-dev] [PATCH v2 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 55 lines · Aaron Conole
Quoted 6 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 8 lines
---
 buildtools/call-sphinx-build.py | 29 +++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 41 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

Some nits follow - it would be good to clean them up before applying but they are fairly inconsequential.

Quoted 10 lines
buildtools/call-sphinx-build.py new file +4 -0
new file mode 100755
index 000000000..027317b9b
@@ -0,0 +1,29 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#

buildtools/call-sphinx-build.py line 14

Quoted 10 lines
buildtools/call-sphinx-build.py new file +10 -0
+
+import sys
+import os
+import os.path
+import subprocess
+
+sphinx = sys.argv[1]
+src = sys.argv[2]
+dst = sys.argv[3]
+depfile = os.path.join(dst,'.html.d')

^ whitespace here

buildtools/call-sphinx-build.py line 24

Quoted 10 lines
buildtools/call-sphinx-build.py new file +10 -0
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    for f in files:
+        srcfiles.append(os.path.join(root, f))
+
+# run sphinx, putting the html output in a "html" directory
+subprocess.run([sphinx, '-j', 'auto', '-b', 'html', src,
+                os.path.join(dst, 'html')], check = True)

^ no whitespace
around =

Quoted 5 lines

buildtools/call-sphinx-build.py line 29

Quoted 5 lines
buildtools/call-sphinx-build.py new file +5 -0
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(depfile, 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
+subprocess.run(['cp', '-f', depfile, '/tmp'])

Ok, will fix. I also see I left in an unnecessary subprocess.run at the end to copy the file to /tmp. This was for debugging only since ninja deletes the .d files after processing them.
[https://ninja-build.org/manual.html#_deps, see behaviour for deps=gcc]

/Bruce

#20 lore raw

PATCH v2 6/6 doc/api: reduce indentation in meson build file

In reply to Bruce Richardson

When building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.

---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
index 23a7dfc75..c72b880e1 100644
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
#22 lore raw
In reply to Bruce Richardson
Quoted 34 lines · Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by
meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.

V2: resend to correct email addresses

Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file

Quoted 9 lines
 buildtools/call-sphinx-build.py | 29 ++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++------------
----
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Signature · 2 lines
Kind regards,
Luca Boccassi
#23 lore raw

v3 cover meson build improvements

In reply to Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.

V2: resend to correct email addresses

Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file

 buildtools/call-sphinx-build.py | 23 ++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 6 files changed, 98 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
#24 lore raw

PATCH v3 1/6 kernel/linux/kni: fix meson warning about console keyword

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Fixes b78f32cff94d ("kni: support meson build")
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
index 955eec949..f93e97fa0 100644
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
#25 lore raw

Re: [dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword

Quoted 7 lines · Bruce Richardson

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Fixes b78f32cff94d ("kni: support meson build")

Since this is linked to kni ethtool stuff removed from 19.11, I would go with a Cc: [email protected].
Then stable maintainers can ignore for everything but 19.11 ?

--
David Marchand

#26 lore raw

Re: [dpdk-dev] [PATCH v3 1/6] kernel/linux/kni: fix meson warning about console keyword

Quoted 7 lines · David Marchand
Quoted 5 lines · Bruce Richardson

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Quoted 7 lines
Quoted 2 lines
Fixes b78f32cff94d ("kni: support meson build")

Since this is linked to kni ethtool stuff removed from 19.11, I would go with a Cc: [email protected].
Then stable maintainers can ignore for everything but 19.11 ?

+1

Quoted 4 lines

--
David Marchand

#27 lore raw

PATCH v3 2/6 build: skip processing docs folder if docs disabled

In reply to Bruce Richardson

While each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.

---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
index c5410d85d..c49ec8476 100644
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
Signature · 1 line
2.24.1
#28 lore raw

PATCH v3 3/6 doc/api: fix warning with meson build

In reply to Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
index 1c48b7672..23a7dfc75 100644
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
Signature · 1 line
2.24.1
#29 lore raw

Re: [dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build

Quoted 5 lines · Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

Quoted 7 lines

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")
Cc [email protected] ? or does it depend on other changes?

--
David Marchand

#30 lore raw

Re: [dpdk-dev] [PATCH v3 3/6] doc/api: fix warning with meson build

Quoted 10 lines · David Marchand
Quoted 8 lines · Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Quoted 7 lines
Quoted 4 lines

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")
Cc [email protected] ? or does it depend on other changes?

I don't think it requires any other changes. Should probably go to 19.11 anyway - the 18.11 tree can skip it, I think.

#31 lore raw

PATCH v3 4/6 doc/guides: reduce whitespace in meson build file

In reply to Bruce Richardson

For building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.

---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
index 7931ef3bb..80c21d168 100644
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#32 lore raw

PATCH v3 5/6 doc/guides: rebuild with meson whenever a file changes

In reply to Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

---
 buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 35 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +23 -0
new file mode 100755
index 000000000..80bda2661
@@ -0,0 +1,23 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run
+
+(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+run([sphinx, '-j', 'auto', '-b', 'html', src, join(dst, 'html')], check=True)
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(join(dst, '.html.d'), 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
buildtools/meson.build +4 -2
index 6ef2c5721..cd1d05403 100644
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
index 80c21d168..732e7ad3a 100644
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#33 lore raw

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 7 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 8 lines
---
 buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 35 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

This new file is not added to MAINTAINERS.

--
David Marchand

#34 lore raw

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 19 lines · David Marchand
Quoted 7 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 8 lines
---
 buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 35 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Quoted 1 line

This new file is not added to MAINTAINERS.

I can fix this when applying.
I would add this to the section:

Meson build
M: Bruce Richardson <[email protected]>

Signature · 1 line
David Marchand
#35 lore raw

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 20 lines · David Marchand
Quoted 19 lines · David Marchand
Quoted 15 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

---
 buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 35 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Quoted 8 lines
Quoted 1 line

This new file is not added to MAINTAINERS.

I can fix this when applying.
I would add this to the section:

Meson build
M: Bruce Richardson <[email protected]>

Agreed.

#36 lore raw

Re: [dpdk-dev] [PATCH v3 5/6] doc/guides: rebuild with meson whenever a file changes

In reply to Bruce Richardson
Quoted 31 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

---
 buildtools/call-sphinx-build.py | 23 +++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 3 files changed, 35 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
buildtools/call-sphinx-build.py new file +10 -0
new file mode 100755
index 000000000..80bda2661
@@ -0,0 +1,23 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run
+

buildtools/call-sphinx-build.py line 20

Quoted 10 lines
buildtools/call-sphinx-build.py new file +10 -0
+(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+run([sphinx, '-j', 'auto', '-b', 'html', src, join(dst, 'html')], check=True)
+

Looking in travis, it appears that sphinx there is too old to support the "-j auto" flag, so I'll do a new revision to fix that.

/Bruce

#37 lore raw

PATCH v3 6/6 doc/api: reduce indentation in meson build file

In reply to Bruce Richardson

When building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.

---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
index 23a7dfc75..c72b880e1 100644
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
#38 lore raw

v4 cover meson build improvements

In reply to Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.

V2: resend to correct email addresses

Bruce Richardson (6):
kernel/linux/kni: fix meson warning about console keyword
build: skip processing docs folder if docs disabled doc/api: fix warning with meson build doc/guides: reduce whitespace in meson build file doc/guides: rebuild with meson whenever a file changes doc/api: reduce indentation in meson build file

 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++
 buildtools/meson.build          |  6 +-
 doc/api/meson.build             | 99 +++++++++++++++++----------------
 doc/guides/meson.build          | 38 ++++++-------
 doc/meson.build                 |  4 ++
 kernel/linux/kni/meson.build    |  1 -
 7 files changed, 107 insertions(+), 73 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
Signature · 1 line
2.24.1
#39 lore raw

PATCH v4 1/6 kernel/linux/kni: fix meson warning about console keyword

Since kni no longer includes the ethtool code and so is faster to build, we no longer need the console parameter to have incremental screen updates as it builds. Therefore, we drop the keyword which removes the warning.

Fixes b78f32cff94d ("kni: support meson build")
---
 kernel/linux/kni/meson.build | 1 -
 1 file changed, 1 deletion(-)
kernel/linux/kni/meson.build +0 -1
index 955eec949..f93e97fa0 100644
@@ -23,7 +23,6 @@ custom_target('rte_kni',
 		' -I' + meson.current_source_dir(),
 		'modules'],
 	depends: kni_mkfile,
-	console: true,
 	install: true,
 	install_dir: kernel_dir + '/extra/dpdk',
 	build_by_default: get_option('enable_kmods'))
Signature · 1 line
2.24.1
#40 lore raw

PATCH v4 2/6 build: skip processing docs folder if docs disabled

In reply to Bruce Richardson

While each target is set to be ignored if the docs are disabled in the meson build, there is little reason to process the docs folder at all.

---
 doc/meson.build | 4 ++++
 1 file changed, 4 insertions(+)
doc/meson.build +4 -0
index c5410d85d..c49ec8476 100644
@@ -1,6 +1,10 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+if not get_option('enable_docs')
+	subdir_done()
+endif
+
 doc_targets = []
 doc_target_names = []
 subdir('api')
Signature · 1 line
2.24.1
#41 lore raw

PATCH v4 3/6 doc/api: fix warning with meson build

In reply to Bruce Richardson

The install parameter to configure_file is new in 0.50 and generates a warning since it is newer than our minimum version of 0.47.1. The parameter, however, is unneeded as the documentation states:

"When omitted it defaults to true when install_dir is set and not empty, false otherwise."

Given that install_dir is not set for this file, install defaults to false so no need to explicitly specify it.

Fixes 720b14db3ae2 ("build: generate API documentation with meson")
---
 doc/api/meson.build | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
doc/api/meson.build +1 -2
index 1c48b7672..23a7dfc75 100644
@@ -38,8 +38,7 @@ if doxygen.found()
 
 	doxy_conf = configure_file(input: 'doxy-api.conf.in',
 		output: 'doxy-api.conf',
-		configuration: cdata,
-		install: false)
+		configuration: cdata)
 
 	doxy_build = custom_target('doxygen',
 		depends: example,
Signature · 1 line
2.24.1
#42 lore raw

PATCH v4 4/6 doc/guides: reduce whitespace in meson build file

In reply to Bruce Richardson

For building the guides, we can make the meson.build easier to read by using the subdir_done function to quit early.

---
@stable, this may be considered for 19.11 backport to make other backporting to this file easier.

---
 doc/guides/meson.build | 44 ++++++++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
doc/guides/meson.build +23 -21
index 7931ef3bb..80c21d168 100644
@@ -3,26 +3,28 @@
 
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
-if sphinx.found()
-	htmldir = join_paths('share', 'doc', 'dpdk')
-	html_guides_build = custom_target('html_guides_build',
-		input: meson.current_source_dir(),
-		output: 'guides',
-		command: [sphinx, '-b', 'html',
-			'-d', meson.current_build_dir() + '/.doctrees',
-			'@INPUT@', meson.current_build_dir() + '/guides'],
-		build_by_default: get_option('enable_docs'),
-		install: get_option('enable_docs'),
-		install_dir: htmldir)
+if not sphinx.found()
+	subdir_done()
+endif
 
-	doc_targets += html_guides_build
-	doc_target_names += 'HTML_Guides'
+htmldir = join_paths('share', 'doc', 'dpdk')
+html_guides = custom_target('html_guides',
+	input: meson.current_source_dir(),
+	output: 'guides',
+	command: [sphinx, '-b', 'html',
+		'-d', meson.current_build_dir() + '/.doctrees',
+		'@INPUT@', meson.current_build_dir() + '/guides'],
+	build_by_default: get_option('enable_docs'),
+	install: get_option('enable_docs'),
+	install_dir: htmldir)
 
-	# sphinx leaves a .buildinfo in the target directory, which we don't
-	# want to install. Note that sh -c has to be used, otherwise the
-	# env var does not get expanded if calling rm/install directly.
-	meson.add_install_script('sh', '-c',
-		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-	meson.add_install_script('sh', '-c',
-		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
-endif
+doc_targets += html_guides
+doc_target_names += 'HTML_Guides'
+
+# sphinx leaves a .buildinfo in the target directory, which we don't
+# want to install. Note that sh -c has to be used, otherwise the
+# env var does not get expanded if calling rm/install directly.
+meson.add_install_script('sh', '-c',
+	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+meson.add_install_script('sh', '-c',
+	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#43 lore raw

PATCH v4 5/6 doc/guides: rebuild with meson whenever a file changes

In reply to Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

---
 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 4 files changed, 44 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py
MAINTAINERS +1 -0
index 4395d8df1..aa32bfc59 100644
@@ -128,6 +128,7 @@ F: meson.build
 F: lib/librte_eal/freebsd/BSDmakefile.meson
 F: meson_options.txt
 F: config/rte_config.h
+F: buildtools/call-sphinx-build.py
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/map_to_def.py
 F: buildtools/symlink-drivers-solibs.sh
buildtools/call-sphinx-build.py new file +31 -0
new file mode 100755
index 000000000..b9a3994e1
@@ -0,0 +1,31 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run, PIPE
+from distutils.version import StrictVersion
+
+(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+
+# for sphinx version >= 1.7 add parallelism using "-j auto"
+ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+sphinx_cmd = [sphinx]
+if StrictVersion(ver) >= StrictVersion('1.7'):
+    sphinx_cmd += ['-j', 'auto']
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+    srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')], check=True)
+print(str(process.args) + ' Done OK')
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(join(dst, '.html.d'), 'w') as d:
+    d.write('html: ' + ' '.join(srcfiles) + '\n')
buildtools/meson.build +4 -2
index 6ef2c5721..cd1d05403 100644
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
 # set up map-to-def script using python, either built-in or external
 python3 = import('python').find_installation(required: false)
 if python3.found()
-	map_to_def_cmd = [python3, files('map_to_def.py')]
+	py3 = [python3]
 else
-	map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+	py3 = ['meson', 'runpython']
 endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
 # stable ABI always starts with "DPDK_"
 is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
doc/guides/meson.build +8 -14
index 80c21d168..732e7ad3a 100644
@@ -7,24 +7,18 @@ if not sphinx.found()
 	subdir_done()
 endif
 
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
-	input: meson.current_source_dir(),
-	output: 'guides',
-	command: [sphinx, '-b', 'html',
-		'-d', meson.current_build_dir() + '/.doctrees',
-		'@INPUT@', meson.current_build_dir() + '/guides'],
+	input: files('index.rst'),
+	output: 'html',
+	command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+	depfile: '.html.d',
 	build_by_default: get_option('enable_docs'),
 	install: get_option('enable_docs'),
 	install_dir: htmldir)
 
+install_data(files('custom.css'),
+			install_dir: join_paths(htmldir,'_static', 'css'))
+
 doc_targets += html_guides
 doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
-	'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
-	'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
Signature · 1 line
2.24.1
#44 lore raw

Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 10 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 7 lines
---
 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 4 files changed, 44 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

Caught an issue on fc30:

[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html
/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py /usr/libexec/python2-sphinx/sphinx-build /home/dmarchan/dpdk/doc/guides /home/dmarchan/builds/build-x86-default/doc/guides sphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>

    ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]

IndexError: list index out of range

Double checked, the version went to stderr:

$ sphinx-build --version >/dev/null
sphinx-build 1.8.4
$ sphinx-build --version 2>/dev/null
Signature · 1 line
David Marchand
#45 lore raw

Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 31 lines · David Marchand
Quoted 10 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 7 lines
---
 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 4 files changed, 44 insertions(+), 16 deletions(-)
 create mode 100755 buildtools/call-sphinx-build.py

Caught an issue on fc30:

[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html
/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py /usr/libexec/python2-sphinx/sphinx-build /home/dmarchan/dpdk/doc/guides /home/dmarchan/builds/build-x86-default/doc/guides sphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>

Quoted 9 lines
    ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]

IndexError: list index out of range

Double checked, the version went to stderr:

$ sphinx-build --version >/dev/null
sphinx-build 1.8.4
$ sphinx-build --version 2>/dev/null

That is strange. Didn't hit any issues on fedora 31 which I am using here. I'll see if I can make it work with both stderr and stdout reporting.

/Bruce

#46 lore raw

Re: [dpdk-dev] [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

In reply to David Marchand
Quoted 44 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Tuesday, January 14, 2020 3:59 PM
To: Richardson, Bruce <[email protected]>
Cc: Mcnamara, John <[email protected]>; Luca Boccassi <[email protected]>; dev <[email protected]>; Aaron Conole <[email protected]>; Thomas Monjalon <[email protected]>
Subject: Re: [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file changes

Quoted 10 lines · Bruce Richardson

Add proper support for calling sphinx whenever a file in the doc directory changes. This is accomplished by using a wrapper script for sphinx, which runs sphinx but also emits a gcc-format dependency file listing all the doc files. This is used by ninja so that any change to the doc files triggers a rebuild of the docs.

Quoted 7 lines
---
 MAINTAINERS                     |  1 +
 buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
 buildtools/meson.build          |  6 ++++--
 doc/guides/meson.build          | 22 ++++++++--------------
 4 files changed, 44 insertions(+), 16 deletions(-)  create mode

100755 buildtools/call-sphinx-build.py

Caught an issue on fc30:

[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html
/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py /usr/libexec/python2-sphinx/sphinx-build /home/dmarchan/dpdk/doc/guides /home/dmarchan/builds/build-x86-default/doc/guides sphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>

    ver = run([sphinx, '--version'],
Quoted 9 lines

stdout=PIPE).stdout.decode().split()[-1]
IndexError: list index out of range

Double checked, the version went to stderr:

$ sphinx-build --version >/dev/null
sphinx-build 1.8.4
$ sphinx-build --version 2>/dev/null

The series http://patches.dpdk.org/project/dpdk/list/?series=8190 should fix this issue and also adds support for the "werror" option for building the guides.

/Bruce

#47 lore raw

PATCH v4 6/6 doc/api: reduce indentation in meson build file

In reply to Bruce Richardson

When building the API docs, we can make the meson.build file easier to read, and allow more code per line, by using subdir_done() to quit early.

---
@stable, this patch may be considered for backporting to 19.11 to allow easier merging of other patches to the affected file.

---
 doc/api/meson.build | 98 +++++++++++++++++++++++----------------------
 1 file changed, 50 insertions(+), 48 deletions(-)
doc/api/meson.build +50 -48
index 23a7dfc75..c72b880e1 100644
@@ -3,52 +3,54 @@
 
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
-if doxygen.found()
-	# due to the CSS customisation script, which needs to run on a file that
-	# is in a subdirectory that is created at build time and thus it cannot
-	# be an individual custom_target, we need to wrap the doxygen call in a
-	# script to run the CSS modification afterwards
-	generate_doxygen = find_program('generate_doxygen.sh')
-	generate_examples = find_program('generate_examples.sh')
-	generate_css = find_program('doxy-html-custom.sh')
-
-	inputdir = join_paths(meson.source_root(), 'examples')
-	htmldir = join_paths('share', 'doc', 'dpdk')
-
-	# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
-	# if install is set to true it will override build_by_default and it will
-	# cause the target to always be built. If install were to be always set to
-	# false it would be impossible to install the docs.
-	# So use a configure option for now.
-	example = custom_target('examples.dox',
-		input: inputdir,
-		output: 'examples.dox',
-		command: [generate_examples, '@INPUT@', '@OUTPUT@'],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	cdata = configuration_data()
-	cdata.set('VERSION', meson.project_version())
-	cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
-	cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
-	cdata.set('HTML_OUTPUT', 'api')
-	cdata.set('TOPDIR', meson.source_root())
-	cdata.set('STRIP_FROM_PATH', meson.source_root())
-
-	doxy_conf = configure_file(input: 'doxy-api.conf.in',
-		output: 'doxy-api.conf',
-		configuration: cdata)
-
-	doxy_build = custom_target('doxygen',
-		depends: example,
-		input: doxy_conf,
-		output: 'api',
-		command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
-		install: get_option('enable_docs'),
-		install_dir: htmldir,
-		build_by_default: get_option('enable_docs'))
-
-	doc_targets += doxy_build
-	doc_target_names += 'Doxygen_API'
+if not doxygen.found()
+  subdir_done()
 endif
+
+# due to the CSS customisation script, which needs to run on a file that
+# is in a subdirectory that is created at build time and thus it cannot
+# be an individual custom_target, we need to wrap the doxygen call in a
+# script to run the CSS modification afterwards
+generate_doxygen = find_program('generate_doxygen.sh')
+generate_examples = find_program('generate_examples.sh')
+generate_css = find_program('doxy-html-custom.sh')
+
+inputdir = join_paths(meson.source_root(), 'examples')
+htmldir = join_paths('share', 'doc', 'dpdk')
+
+# due to the following bug: https://github.com/mesonbuild/meson/issues/4107
+# if install is set to true it will override build_by_default and it will
+# cause the target to always be built. If install were to be always set to
+# false it would be impossible to install the docs.
+# So use a configure option for now.
+example = custom_target('examples.dox',
+	input: inputdir,
+	output: 'examples.dox',
+	command: [generate_examples, '@INPUT@', '@OUTPUT@'],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+cdata = configuration_data()
+cdata.set('VERSION', meson.project_version())
+cdata.set('API_EXAMPLES', join_paths(meson.build_root(), 'doc', 'api', 'examples.dox'))
+cdata.set('OUTPUT', join_paths(meson.build_root(), 'doc', 'api'))
+cdata.set('HTML_OUTPUT', 'api')
+cdata.set('TOPDIR', meson.source_root())
+cdata.set('STRIP_FROM_PATH', meson.source_root())
+
+doxy_conf = configure_file(input: 'doxy-api.conf.in',
+	output: 'doxy-api.conf',
+	configuration: cdata)
+
+doxy_build = custom_target('doxygen',
+	depends: example,
+	input: doxy_conf,
+	output: 'api',
+	command: [generate_doxygen, '@INPUT@', '@OUTPUT@', generate_css],
+	install: get_option('enable_docs'),
+	install_dir: htmldir,
+	build_by_default: get_option('enable_docs'))
+
+doc_targets += doxy_build
+doc_target_names += 'Doxygen_API'
Signature · 1 line
2.24.1
#48 lore raw
In reply to Bruce Richardson
Quoted 21 lines · Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

Quoted 6 lines

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.

V2: resend to correct email addresses

Tried this series in Travis.
It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04. The packaging fails when ninja can't find the 'doc' target.

--
David Marchand

#49 lore raw
Quoted 28 lines · David Marchand
Quoted 26 lines · Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as flagged by Aaron.

Quoted 7 lines
Quoted 1 line

V2: resend to correct email addresses

Tried this series in Travis. It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04. The packaging fails when ninja can't find the 'doc' target.

Can you send on a link to the failure logs?

I suspect this may be due to returning early from the doc folder if the document building is disabled (but need to see logs to confirm). However, I'm not sure it makes sense to have a doc target defined if doc building is disabled in build config.

#50 lore raw
Quoted 37 lines · Bruce Richardson
Quoted 28 lines · David Marchand
Quoted 26 lines · Bruce Richardson

These patches make some improvements to the meson build, particularly for documentation. They also remove many, but not all warnings issued by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs. The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output correct dependency information for the sphinx build in a .d file. This .d file is processed by ninja (not meson) on build, so that any changes to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/ is included in this set, and will be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as flagged by Aaron.

Quoted 7 lines
Quoted 1 line

V2: resend to correct email addresses

Tried this series in Travis. It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04. The packaging fails when ninja can't find the 'doc' target.

Quoted 7 lines

Can you send on a link to the failure logs?

I suspect this may be due to returning early from the doc folder if the document building is disabled (but need to see logs to confirm). However, I'm not sure it makes sense to have a doc target defined if doc building is disabled in build config.

Currently restarting my jobs and I might lose the logs, so I saved it here:
https://paste.centos.org/view/7d6e0545

Signature · 1 line
David Marchand
#51 lore raw
In reply to Bruce Richardson
Quoted 53 lines · Bruce Richardson
Quoted 40 lines · David Marchand

On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<
[email protected]

Quoted 37 lines

wrote:
These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued
by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.

Quoted 9 lines
Quoted 1 line

V2: resend to correct email addresses

Tried this series in Travis. It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.

Can you send on a link to the failure logs?

Quoted 7 lines

I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.

Sorry, should have thought about this when reviewing: the usefulness of having the target work without the option set is that the docs will only be built when called explicitly, without having to change the configuration.

Given the doc build is very slow and it's arch-independent, in Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).

Signature · 2 lines
Kind regards,
Luca Boccassi
#52 lore raw
Quoted 62 lines · Luca Boccassi
Quoted 53 lines · Bruce Richardson
Quoted 40 lines · David Marchand

On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<
[email protected]

Quoted 37 lines

wrote:
These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued
by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.

Quoted 9 lines
Quoted 1 line

V2: resend to correct email addresses

Tried this series in Travis. It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.

Can you send on a link to the failure logs?

Quoted 7 lines

I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.

Quoted 9 lines

Sorry, should have thought about this when reviewing: the usefulness of having the target work without the option set is that the docs will only be built when called explicitly, without having to change the configuration.

Given the doc build is very slow and it's arch-independent, in Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).

Right. I think the offending patch #2, can just be dropped from the set.

/Bruce

#53 lore raw
Quoted 69 lines · Bruce Richardson
Quoted 67 lines · Luca Boccassi
Quoted 53 lines · Bruce Richardson
Quoted 40 lines · David Marchand

On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<
[email protected]

Quoted 37 lines

wrote:
These patches make some improvements to the meson build, particularly
for documentation. They also remove many, but not all warnings issued
by meson e.g. warnings about newer features unsupported in baseline.

The biggest change is to improve the handling of the guide html docs.
The change here is more significant, and the doc build now uses a wrapper script around sphinx. This wrapper script allows us to output
correct dependency information for the sphinx build in a .d file. This
.d file is processed by ninja (not meson) on build, so that any changes
to doc files trigger a rebuild to the guides using sphinx.

For now, the two patches which remove the meson version warnings are
CC'ed to stable for backport, theoretically this who set could be backported if so desired, as all changes could be considered fixes to
some degree or other, and nothing introduces a whole new feature.

Note: for completeness and simplicity, previously submitted patch
http://patches.dpdk.org/patch/64189/
is included in this set, and will
be marked superceded in patchwork.

V4: fix build with older sphinx

V3: Shorten the python code in patch 5, and ensure style compliance as
flagged by Aaron.

Quoted 9 lines
Quoted 1 line

V2: resend to correct email addresses

Tried this series in Travis. It went fine through my own checks too.

But OBS is not happy with Debian 10, Debian Next and Ubuntu >= 18.04.
The packaging fails when ninja can't find the 'doc' target.

Can you send on a link to the failure logs?

Quoted 7 lines

I suspect this may be due to returning early from the doc folder if the
document building is disabled (but need to see logs to confirm). However,
I'm not sure it makes sense to have a doc target defined if doc building is
disabled in build config.

Sorry, should have thought about this when reviewing: the usefulness of having the target work without the option set is that the docs will only be built when called explicitly, without having to change the configuration.

Quoted 5 lines
Quoted 4 lines

Given the doc build is very slow and it's arch-independent, in Debian/Ubuntu we only do it once, in the arch-independent target (arch:
all).

Right. I think the offending patch #2, can just be dropped from the set.

Ok, I dropped this patch.
Packages for debian distributions in OBS build fine now.

I am ok with a followup patch for fc30, I disabled the doc check in my env for now.

Series & ~patch2 applied.
Thanks.

--
David Marchand