Skip to contentSkip to conversation

doc: support building HTML guides with meson

PATCH In review Bruce Richardson 4 participants 13 messages Sep 11, 2018 – Oct 29, 2018 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc: support building HTML guides with meson

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++
 3 files changed, 29 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/meson.build
doc/api/meson.build +2 -1
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
 		install_dir: htmldir,
 		build_by_default: false)
 
-	run_target('doc', command: 'true', depends: doxy_build)
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'
 endif
doc/guides/meson.build new file +16 -0
new file mode 100644
index 000000000..6d1e2990d
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+
+if sphinx.found()
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'index.html',
+		command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +11 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+doc_targets = []
+doc_target_names = []
 subdir('api')
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)
Signature · 1 line
2.11.0
#1 lore raw
Quoted 4 lines · Bruce Richardson

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

Just a reminder that's v2, and the patch won't apply cleanly on the latest revision

Quoted 24 lines

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++

 3 files changed, 29 insertions(+), 1 deletion(-)

 create mode 100644 doc/guides/meson.build

doc/api/meson.build +0 -0
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
  		install_dir: htmldir,
  		build_by_default: false)

doc/api/meson.build +2 -1
-	run_target('doc', command: 'true', depends: doxy_build)
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'

 endif

doc/guides/meson.build new file +0 -0
new file mode 100644
index 000000000..6d1e2990d

doc/guides/meson.build line 2

Quoted 2 lines
doc/guides/meson.build new file +2 -0
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation

s/2017/2018

doc/guides/meson.build

Quoted 7 lines
doc/guides/meson.build new file +1 -0
+sphinx = find_program('sphinx-build', required:

get_option('enable_docs'))

doc/guides/meson.build new file +5 -0
+
+if sphinx.found()
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'index.html',

As we discussed I don't have a good solution, but right now running ninja will rebuild these sphinx docs again and again, which will be a bit annoying when using enable_docs=true as it will always happen. Changing output to 'html' fixes the issue and it makes it build only once, but then they won't rebuild if the docs change as you noted.

Quoted 19 lines
doc/guides/meson.build new file +1 -0
+		command: [sphinx, '-b', 'html', '@INPUT@',

meson.current_build_dir() + '/html'],

doc/guides/meson.build new file +6 -0
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +0 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2018 Luca Boccassi <[email protected]>

doc/meson.build +2 -0
+doc_targets = []
+doc_target_names = []

 subdir('api')

doc/meson.build

Quoted 9 lines
doc/meson.build +9 -0
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)

One thing that's missing is the install_dir, without which ninja install doesn't work (actually errors out for the whole tree).

To keep the install the same as the legacy makefiles this diff is needed (I need to change the outdir in the doxygen stuff too, v5 coming) (in the build dir it will be slightly awkward as it's build/doc/guides/guides and build/doc/api/api, but I can't find another way to get it to work and install in the expected directories):

doc/guides/meson.build +5 -3
@@ -4,12 +4,14 @@
 sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
 
 if sphinx.found()
+       htmldir = join_paths('share', 'doc', 'dpdk')
        html_guides_build = custom_target('html_guides_build',
                input: meson.current_source_dir(),
-               output: 'index.html',
-               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+               output: 'guides',
+               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/guides'],
                build_by_default: false,
-               install: get_option('enable_docs'))
+               install: get_option('enable_docs'),
+               install_dir: htmldir)
 
        doc_targets += html_guides_build
        doc_target_names += 'HTML_Guides'
Signature · 2 lines
Kind regards,
Luca Boccassi
#2 lore raw
Quoted 106 lines · Luca Boccassi
Quoted 4 lines · Bruce Richardson

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

Just a reminder that's v2, and the patch won't apply cleanly on the latest revision

Quoted 24 lines

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++

 3 files changed, 29 insertions(+), 1 deletion(-)

 create mode 100644 doc/guides/meson.build

doc/api/meson.build +0 -0
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
  		install_dir: htmldir,
  		build_by_default: false)

doc/api/meson.build +2 -1
-	run_target('doc', command: 'true', depends: doxy_build)
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'

 endif

doc/guides/meson.build new file +0 -0
new file mode 100644
index 000000000..6d1e2990d

doc/guides/meson.build line 2

Quoted 2 lines
doc/guides/meson.build new file +2 -0
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation

s/2017/2018

doc/guides/meson.build

Quoted 7 lines
doc/guides/meson.build new file +1 -0
+sphinx = find_program('sphinx-build', required:

get_option('enable_docs'))

doc/guides/meson.build new file +5 -0
+
+if sphinx.found()
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'index.html',

As we discussed I don't have a good solution, but right now running ninja will rebuild these sphinx docs again and again, which will be a bit annoying when using enable_docs=true as it will always happen. Changing output to 'html' fixes the issue and it makes it build only once, but then they won't rebuild if the docs change as you noted.

Quoted 19 lines
doc/guides/meson.build new file +1 -0
+		command: [sphinx, '-b', 'html', '@INPUT@',

meson.current_build_dir() + '/html'],

doc/guides/meson.build new file +6 -0
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +0 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2018 Luca Boccassi <[email protected]>

doc/meson.build +2 -0
+doc_targets = []
+doc_target_names = []

 subdir('api')

doc/meson.build

Quoted 9 lines
doc/meson.build +9 -0
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)

One thing that's missing is the install_dir, without which ninja install doesn't work (actually errors out for the whole tree).

To keep the install the same as the legacy makefiles this diff is needed (I need to change the outdir in the doxygen stuff too, v5 coming) (in the build dir it will be slightly awkward as it's build/doc/guides/guides and build/doc/api/api, but I can't find another
way to get it to work and install in the expected directories):

doc/guides/meson.build +0 -0
@@ -4,12 +4,14 @@

 sphinx = find_program('sphinx-build', required:
get_option('enable_docs'))

 if sphinx.found()
+       htmldir = join_paths('share', 'doc', 'dpdk') html_guides_build = custom_target('html_guides_build',
                input: meson.current_source_dir(),
-               output: 'index.html',
-               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+               output: 'guides',
+               command: [sphinx, '-b', 'html', '@INPUT@',

Quoted 8 lines

meson.current_build_dir() + '/guides'],
                build_by_default: false,
-               install: get_option('enable_docs')) +               install: get_option('enable_docs'), +               install_dir: htmldir)

        doc_targets += html_guides_build doc_target_names += 'HTML_Guides'

Couple more issues: I diffed the installed doc with ye olde make and this one, and:

1) This one will install sphinx temporary cache files, which among other things will screw up reproducible builds. It's easy to get rid of the .doctrees by adding the following to the sphinx command:

'-d', meson.current_build_dir() + '/.doctrees'

so that the doctrees are stored in the build directory, rather than in the output directory, and so they will not be installed. But the .buildinfo file, with some hashes (and so more reproducibility problems) is still installed and I can't see how to get rid of it.

2) The custom.css file is installed manually by ye olde makefiles and thus is missing from guides/_static/css/custom.css

Signature · 2 lines
Kind regards,
Luca Boccassi
#3 lore raw
Quoted 125 lines · Luca Boccassi
Quoted 106 lines · Luca Boccassi
Quoted 4 lines · Bruce Richardson

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

Just a reminder that's v2, and the patch won't apply cleanly on the latest revision

Quoted 24 lines

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++

 3 files changed, 29 insertions(+), 1 deletion(-)

 create mode 100644 doc/guides/meson.build

doc/api/meson.build +0 -0
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
  		install_dir: htmldir,
  		build_by_default: false)

doc/api/meson.build +2 -1
-	run_target('doc', command: 'true', depends: doxy_build)
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'

 endif

doc/guides/meson.build new file +0 -0
new file mode 100644
index 000000000..6d1e2990d

doc/guides/meson.build line 2

Quoted 2 lines
doc/guides/meson.build new file +2 -0
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation

s/2017/2018

doc/guides/meson.build

Quoted 7 lines
doc/guides/meson.build new file +1 -0
+sphinx = find_program('sphinx-build', required:

get_option('enable_docs'))

doc/guides/meson.build new file +5 -0
+
+if sphinx.found()
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'index.html',

As we discussed I don't have a good solution, but right now running ninja will rebuild these sphinx docs again and again, which will be a bit annoying when using enable_docs=true as it will always happen. Changing output to 'html' fixes the issue and it makes it build only once, but then they won't rebuild if the docs change as you noted.

Quoted 19 lines
doc/guides/meson.build new file +1 -0
+		command: [sphinx, '-b', 'html', '@INPUT@',

meson.current_build_dir() + '/html'],

doc/guides/meson.build new file +6 -0
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +0 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2018 Luca Boccassi <[email protected]>

doc/meson.build +2 -0
+doc_targets = []
+doc_target_names = []

 subdir('api')

doc/meson.build

Quoted 9 lines
doc/meson.build +9 -0
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)

One thing that's missing is the install_dir, without which ninja install doesn't work (actually errors out for the whole tree).

To keep the install the same as the legacy makefiles this diff is needed (I need to change the outdir in the doxygen stuff too, v5 coming) (in the build dir it will be slightly awkward as it's build/doc/guides/guides and build/doc/api/api, but I can't find another
way to get it to work and install in the expected directories):

doc/guides/meson.build +0 -0
@@ -4,12 +4,14 @@

 sphinx = find_program('sphinx-build', required:
get_option('enable_docs'))

 if sphinx.found()
+       htmldir = join_paths('share', 'doc', 'dpdk') html_guides_build = custom_target('html_guides_build',
                input: meson.current_source_dir(),
-               output: 'index.html',
-               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+               output: 'guides',
+               command: [sphinx, '-b', 'html', '@INPUT@',

Quoted 8 lines

meson.current_build_dir() + '/guides'],
                build_by_default: false,
-               install: get_option('enable_docs')) +               install: get_option('enable_docs'), +               install_dir: htmldir)

        doc_targets += html_guides_build doc_target_names += 'HTML_Guides'

Couple more issues: I diffed the installed doc with ye olde make and this one, and:

1) This one will install sphinx temporary cache files, which among other things will screw up reproducible builds. It's easy to get rid of the .doctrees by adding the following to the sphinx command:

'-d', meson.current_build_dir() + '/.doctrees'

Quoted 8 lines

so that the doctrees are stored in the build directory, rather than in the output directory, and so they will not be installed. But the .buildinfo file, with some hashes (and so more reproducibility problems) is still installed and I can't see how to get rid of it.

2) The custom.css file is installed manually by ye olde makefiles and thus is missing from guides/_static/css/custom.css

Great, thanks for all the reviews. I'll attempt to fix as much as possible and do a V2.

/Bruce

#4 lore raw
Quoted 148 lines · Bruce Richardson
Quoted 142 lines · Luca Boccassi
Quoted 114 lines · Luca Boccassi
Quoted 4 lines · Bruce Richardson

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

Just a reminder that's v2, and the patch won't apply cleanly on the
latest revision

Quoted 25 lines

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++

 3 files changed, 29 insertions(+), 1 deletion(-)

 create mode 100644 doc/guides/meson.build

doc/api/meson.build +0 -0
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
  		install_dir: htmldir,
  		build_by_default: false)

doc/api/meson.build +0 -1
-	run_target('doc', command: 'true', depends:

doxy_build)

doc/api/meson.build +2 -0
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'

 endif

doc/guides/meson.build new file +0 -0
new file mode 100644
index 000000000..6d1e2990d

doc/guides/meson.build line 2

Quoted 2 lines
doc/guides/meson.build new file +2 -0
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation

s/2017/2018

doc/guides/meson.build

Quoted 7 lines
doc/guides/meson.build new file +1 -0
+sphinx = find_program('sphinx-build', required:

get_option('enable_docs'))

doc/guides/meson.build new file +5 -0
+
+if sphinx.found()
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'index.html',

As we discussed I don't have a good solution, but right now running
ninja will rebuild these sphinx docs again and again, which will be a
bit annoying when using enable_docs=true as it will always happen.
Changing output to 'html' fixes the issue and it makes it build only
once, but then they won't rebuild if the docs change as you noted.

Quoted 28 lines
doc/guides/meson.build new file +1 -0
+		command: [sphinx, '-b', 'html', '@INPUT@',

meson.current_build_dir() + '/html'],

doc/guides/meson.build new file +6 -0
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +0 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2018 Luca Boccassi <[email protected]>

doc/meson.build +2 -0
+doc_targets = []
+doc_target_names = []

 subdir('api')

doc/meson.build +8 -0
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message,

doc_target_names],

doc/meson.build

Quoted 1 line
doc/meson.build +1 -0
+	depends: doc_targets)

One thing that's missing is the install_dir, without which ninja install doesn't work (actually errors out for the whole tree).

To keep the install the same as the legacy makefiles this diff is needed (I need to change the outdir in the doxygen stuff too, v5 coming) (in the build dir it will be slightly awkward as it's build/doc/guides/guides and build/doc/api/api, but I can't find another
way to get it to work and install in the expected directories):

doc/guides/meson.build +0 -0
@@ -4,12 +4,14 @@

 sphinx = find_program('sphinx-build', required:
get_option('enable_docs'))

 if sphinx.found()
+       htmldir = join_paths('share', 'doc', 'dpdk') html_guides_build = custom_target('html_guides_build',
                input: meson.current_source_dir(),
-               output: 'index.html',
-               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+               output: 'guides',
+               command: [sphinx, '-b', 'html', '@INPUT@',

Quoted 8 lines

meson.current_build_dir() + '/guides'],
                build_by_default: false,
-               install: get_option('enable_docs')) +               install: get_option('enable_docs'), +               install_dir: htmldir)

        doc_targets += html_guides_build doc_target_names += 'HTML_Guides'

Couple more issues: I diffed the installed doc with ye olde make and
this one, and:

1) This one will install sphinx temporary cache files, which among other things will screw up reproducible builds. It's easy to get rid of
the .doctrees by adding the following to the sphinx command:

 '-d', meson.current_build_dir() + '/.doctrees'

so that the doctrees are stored in the build directory, rather than in
the output directory, and so they will not be installed. But the .buildinfo file, with some hashes (and so more reproducibility
problems) is still installed and I can't see how to get rid of it.

Quoted 4 lines

2) The custom.css file is installed manually by ye olde makefiles and
thus is missing from guides/_static/css/custom.css

Quoted 5 lines

Great, thanks for all the reviews. I'll attempt to fix as much as possible
and do a V2.

/Bruce

For the last 2 things, the only thing I can think of is having a little bash script like for doxygen, that calls sphinx, copies the css file and then deletes the .buildinfo. Not the cleanest, but I can't think of other options - I looked into sphinx-build and I can't see an option to determine where .buildinfo is saved to unfortunately

Signature · 2 lines
Kind regards,
Luca Boccassi
#5 lore raw
Quoted 153 lines · Luca Boccassi
Quoted 152 lines · Bruce Richardson
Quoted 142 lines · Luca Boccassi
Quoted 114 lines · Luca Boccassi
Quoted 4 lines · Bruce Richardson

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

Just a reminder that's v2, and the patch won't apply cleanly on the
latest revision

Quoted 25 lines

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++

 3 files changed, 29 insertions(+), 1 deletion(-)

 create mode 100644 doc/guides/meson.build

doc/api/meson.build +0 -0
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
  		install_dir: htmldir,
  		build_by_default: false)

doc/api/meson.build +0 -1
-	run_target('doc', command: 'true', depends:

doxy_build)

doc/api/meson.build +2 -0
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'

 endif

doc/guides/meson.build new file +0 -0
new file mode 100644
index 000000000..6d1e2990d

doc/guides/meson.build line 2

Quoted 2 lines
doc/guides/meson.build new file +2 -0
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation

s/2017/2018

doc/guides/meson.build

Quoted 7 lines
doc/guides/meson.build new file +1 -0
+sphinx = find_program('sphinx-build', required:

get_option('enable_docs'))

doc/guides/meson.build new file +5 -0
+
+if sphinx.found()
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'index.html',

As we discussed I don't have a good solution, but right now running
ninja will rebuild these sphinx docs again and again, which will be a
bit annoying when using enable_docs=true as it will always happen.
Changing output to 'html' fixes the issue and it makes it build only
once, but then they won't rebuild if the docs change as you noted.

Quoted 28 lines
doc/guides/meson.build new file +1 -0
+		command: [sphinx, '-b', 'html', '@INPUT@',

meson.current_build_dir() + '/html'],

doc/guides/meson.build new file +6 -0
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +0 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2018 Luca Boccassi <[email protected]>

doc/meson.build +2 -0
+doc_targets = []
+doc_target_names = []

 subdir('api')

doc/meson.build +8 -0
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message,

doc_target_names],

doc/meson.build

Quoted 1 line
doc/meson.build +1 -0
+	depends: doc_targets)

One thing that's missing is the install_dir, without which ninja install doesn't work (actually errors out for the whole tree).

To keep the install the same as the legacy makefiles this diff is needed (I need to change the outdir in the doxygen stuff too, v5 coming) (in the build dir it will be slightly awkward as it's build/doc/guides/guides and build/doc/api/api, but I can't find another
way to get it to work and install in the expected directories):

doc/guides/meson.build +0 -0
@@ -4,12 +4,14 @@

 sphinx = find_program('sphinx-build', required:
get_option('enable_docs'))

 if sphinx.found()
+       htmldir = join_paths('share', 'doc', 'dpdk') html_guides_build = custom_target('html_guides_build',
                input: meson.current_source_dir(),
-               output: 'index.html',
-               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+               output: 'guides',
+               command: [sphinx, '-b', 'html', '@INPUT@',

Quoted 8 lines

meson.current_build_dir() + '/guides'],
                build_by_default: false,
-               install: get_option('enable_docs')) +               install: get_option('enable_docs'), +               install_dir: htmldir)

        doc_targets += html_guides_build doc_target_names += 'HTML_Guides'

Couple more issues: I diffed the installed doc with ye olde make and
this one, and:

1) This one will install sphinx temporary cache files, which among other things will screw up reproducible builds. It's easy to get rid of
the .doctrees by adding the following to the sphinx command:

 '-d', meson.current_build_dir() + '/.doctrees'

so that the doctrees are stored in the build directory, rather than in
the output directory, and so they will not be installed. But the .buildinfo file, with some hashes (and so more reproducibility
problems) is still installed and I can't see how to get rid of it.

Quoted 4 lines

2) The custom.css file is installed manually by ye olde makefiles and
thus is missing from guides/_static/css/custom.css

Great, thanks for all the reviews. I'll attempt to fix as much as possible
and do a V2.

Quoted 8 lines
Quoted 1 line

/Bruce

For the last 2 things, the only thing I can think of is having a little bash script like for doxygen, that calls sphinx, copies the css file and then deletes the .buildinfo. Not the cleanest, but I can't think of other options - I looked into sphinx-build and I can't see an option to determine where .buildinfo is saved to unfortunately

Just to check that I see what you see, what are the commands you are using to compare the old vs new docs?

/Bruce

#6 lore raw
Quoted 186 lines · Bruce Richardson
Quoted 174 lines · Luca Boccassi
Quoted 167 lines · Bruce Richardson
Quoted 160 lines · Luca Boccassi
Quoted 127 lines · Luca Boccassi
Quoted 5 lines · Bruce Richardson
Quoted 1 line

---
NOTE: this patch depends upon:
http://patches.dpdk.org/project/dpdk/list/?series=1232

Just a reminder that's v2, and the patch won't apply cleanly on
the
latest revision

Quoted 26 lines

 doc/api/meson.build    |  3 ++-
 doc/guides/meson.build | 16 ++++++++++++++++
 doc/meson.build        | 11 +++++++++++

 3 files changed, 29 insertions(+), 1 deletion(-)

 create mode 100644 doc/guides/meson.build

doc/api/meson.build +0 -0
index 5dfa0fe04..f9bee4dac 100644
@@ -50,5 +50,6 @@ if doxygen.found()
  		install_dir: htmldir,
  		build_by_default: false)

doc/api/meson.build +0 -1
-	run_target('doc', command: 'true', depends:

doxy_build)

doc/api/meson.build +2 -0
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'

 endif

doc/guides/meson.build new file +0 -0
new file mode 100644
index 000000000..6d1e2990d

doc/guides/meson.build line 2

Quoted 2 lines
doc/guides/meson.build new file +2 -0
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation

s/2017/2018

doc/guides/meson.build

Quoted 8 lines
doc/guides/meson.build new file +1 -0
+sphinx = find_program('sphinx-build', required:

get_option('enable_docs'))

doc/guides/meson.build new file +3 -0
+
+if sphinx.found()
+	html_guides_build =

custom_target('html_guides_build',

doc/guides/meson.build new file +2 -0
+		input: meson.current_source_dir(),
+		output: 'index.html',

As we discussed I don't have a good solution, but right now running
ninja will rebuild these sphinx docs again and again, which will
be a
bit annoying when using enable_docs=true as it will always happen.
Changing output to 'html' fixes the issue and it makes it build
only
once, but then they won't rebuild if the docs change as you noted.

Quoted 28 lines
doc/guides/meson.build new file +1 -0
+		command: [sphinx, '-b', 'html', '@INPUT@',

meson.current_build_dir() + '/html'],

doc/guides/meson.build new file +6 -0
+		build_by_default: false,
+		install: get_option('enable_docs'))
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+endif
doc/meson.build +0 -0
index afca2e713..c5410d85d 100644
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: BSD-3-Clause
  # Copyright(c) 2018 Luca Boccassi <[email protected]>

doc/meson.build +2 -0
+doc_targets = []
+doc_target_names = []

 subdir('api')

doc/meson.build +8 -0
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message,

doc_target_names],

doc/meson.build

Quoted 1 line
doc/meson.build +1 -0
+	depends: doc_targets)

One thing that's missing is the install_dir, without which ninja
install doesn't work (actually errors out for the whole tree).

To keep the install the same as the legacy makefiles this diff is
needed (I need to change the outdir in the doxygen stuff too, v5
coming) (in the build dir it will be slightly awkward as it's build/doc/guides/guides and build/doc/api/api, but I can't find
another
way to get it to work and install in the expected directories):

doc/guides/meson.build +0 -0
@@ -4,12 +4,14 @@

 sphinx = find_program('sphinx-build', required:
get_option('enable_docs'))

 if sphinx.found()
+       htmldir = join_paths('share', 'doc', 'dpdk') html_guides_build = custom_target('html_guides_build',
                input: meson.current_source_dir(), -               output: 'index.html',
-               command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/html'],
+               output: 'guides',
+               command: [sphinx, '-b', 'html', '@INPUT@',

Quoted 8 lines

meson.current_build_dir() + '/guides'],
                build_by_default: false,
-               install: get_option('enable_docs')) +               install: get_option('enable_docs'), +               install_dir: htmldir)

        doc_targets += html_guides_build doc_target_names += 'HTML_Guides'

Couple more issues: I diffed the installed doc with ye olde make
and
this one, and:

1) This one will install sphinx temporary cache files, which among
other things will screw up reproducible builds. It's easy to get
rid of
the .doctrees by adding the following to the sphinx command:

 '-d', meson.current_build_dir() + '/.doctrees'

so that the doctrees are stored in the build directory, rather than
in
the output directory, and so they will not be installed. But the .buildinfo file, with some hashes (and so more reproducibility
problems) is still installed and I can't see how to get rid of it.

Quoted 5 lines

2) The custom.css file is installed manually by ye olde makefiles
and
thus is missing from guides/_static/css/custom.css

Quoted 5 lines

Great, thanks for all the reviews. I'll attempt to fix as much as possible
and do a V2.

/Bruce

Quoted 10 lines

For the last 2 things, the only thing I can think of is having a little
bash script like for doxygen, that calls sphinx, copies the css file
and then deletes the .buildinfo. Not the cleanest, but I can't think of
other options - I looked into sphinx-build and I can't see an option to
determine where .buildinfo is saved to unfortunately

Quoted 5 lines

Just to check that I see what you see, what are the commands you are using
to compare the old vs new docs?

/Bruce

I'm tarring up the installed share/dpdk/doc directory (with: GZIP="- 9fn" tar --sort=name --mtime="@1534269491" --clamp-mtime --owner=0 -- group=0 --numeric-owner -czf ) and then using diffoscope on them, which will compare everything down to the bytes :-)

Signature · 2 lines
Kind regards,
Luca Boccassi
#7 lore raw

PATCH v2 1/1 doc: support building HTML guides with meson

In reply to Bruce Richardson

From: Bruce Richardson <[email protected]>

---
v2: send on behalf of Bruce on request.
- tell sphinx to create .doctrees working files in the parent of the target directory so that they don't get installed
- change the output directory so that it matches the legacy makefiles directory structure (on install)
- add install_dir to fix ninja install
- add post-install calls to delete .buildinfo sphinx temporary file and to install custom.css as the makefiles do
The installed directory has been verified to be bit-by-bit identical to the legacy makefiles with diffoscope.

 doc/api/meson.build    |  5 ++---
 doc/guides/meson.build | 27 +++++++++++++++++++++++++++
 doc/meson.build        | 11 +++++++++++
 3 files changed, 40 insertions(+), 3 deletions(-)
 create mode 100644 doc/guides/meson.build
doc/api/meson.build +2 -3
index 13fcbb8cd7..30bdc573b5 100644
@@ -50,7 +50,6 @@ if doxygen.found()
 		install_dir: htmldir,
 		build_by_default: false)
 
-	run_target('doc', command: 'true', depends: doxy_build)
-else
-	run_target('doc', command: ['echo', 'doxygen', 'not', 'found'])
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'
 endif
doc/guides/meson.build new file +27 -0
new file mode 100644
index 0000000000..24f5316c5c
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+
+if sphinx.found()
+	htmldir = join_paths('share', 'doc', 'dpdk')
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'guides',
+		command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/guides',
+			'-d', meson.current_build_dir() + '/.doctrees'],
+		build_by_default: false,
+		install: get_option('enable_docs'),
+		install_dir: htmldir)
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+
+	# sphinx leaves a .buildinfo in the target directory, which we don't
+	# want to install. Note that sh -c has to be used, otherwise the
+	# env var does not get expanded if calling rm/install directly.
+	meson.add_install_script('sh', '-c',
+		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+	meson.add_install_script('sh', '-c',
+		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/')
+endif
doc/meson.build +11 -0
index afca2e7133..c5410d85d6 100644
@@ -1,4 +1,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+doc_targets = []
+doc_target_names = []
 subdir('api')
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)
Signature · 1 line
2.18.0
#8 lore raw
Quoted 46 lines · On Wed, 19 Sep 2018 14:48:27 +0100 Luca Boccassi <[email protected]> wrote:

From: Bruce Richardson <[email protected]>

---
v2: send on behalf of Bruce on request.
- tell sphinx to create .doctrees working files in the parent of the target directory so that they don't get installed
- change the output directory so that it matches the legacy makefiles directory structure (on install)
- add install_dir to fix ninja install
- add post-install calls to delete .buildinfo sphinx temporary file and to install custom.css as the makefiles do
The installed directory has been verified to be bit-by-bit identical to the legacy makefiles with diffoscope.

 doc/api/meson.build    |  5 ++---
 doc/guides/meson.build | 27 +++++++++++++++++++++++++++
 doc/meson.build        | 11 +++++++++++
 3 files changed, 40 insertions(+), 3 deletions(-)
 create mode 100644 doc/guides/meson.build
doc/api/meson.build +2 -3
index 13fcbb8cd7..30bdc573b5 100644
@@ -50,7 +50,6 @@ if doxygen.found()
 		install_dir: htmldir,
 		build_by_default: false)
 
-	run_target('doc', command: 'true', depends: doxy_build)
-else
-	run_target('doc', command: ['echo', 'doxygen', 'not', 'found'])
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'
 endif
doc/guides/meson.build new file +2 -0
new file mode 100644
index 0000000000..24f5316c5c
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation

doc/guides/meson.build line 12

Quoted 10 lines
doc/guides/meson.build new file +10 -0
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+
+if sphinx.found()
+	htmldir = join_paths('share', 'doc', 'dpdk')
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'guides',
+		command: [sphinx, '-b', 'html', '@INPUT@', meson.current_build_dir() + '/guides',
+			'-d', meson.current_build_dir() + '/.doctrees'],

This on RHEL 7.5 doesn't work, since old versions of sphinx (I have v1.1.3) need all options before the paths. I tried to move "'-d', meson.current_build_dir() + '/.doctrees'" before "'@INPUT@'" and it worked.

Quoted 4 lines
doc/guides/meson.build new file +4 -0
+		build_by_default: false,
+		install: get_option('enable_docs'),
+		install_dir: htmldir)
+

doc/guides/meson.build line 26

Quoted 10 lines
doc/guides/meson.build new file +10 -0
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+
+	# sphinx leaves a .buildinfo in the target directory, which we don't
+	# want to install. Note that sh -c has to be used, otherwise the
+	# env var does not get expanded if calling rm/install directly.
+	meson.add_install_script('sh', '-c',
+		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+	meson.add_install_script('sh', '-c',
+		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/')

This on RHEL 7.5 doesn't work. I had to append custom.css to DEST ($MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css) to make it work.

Quoted 21 lines
doc/guides/meson.build new file +1 -0
+endif
doc/meson.build +11 -0
index afca2e7133..c5410d85d6 100644
@@ -1,4 +1,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+doc_targets = []
+doc_target_names = []
 subdir('api')
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)
#9 lore raw

PATCH v3 1/1 doc: support building HTML guides with meson

In reply to Bruce Richardson

From: Bruce Richardson <[email protected]>

---
v2: send on behalf of Bruce on request.
- tell sphinx to create .doctrees working files in the parent of the target directory so that they don't get installed
- change the output directory so that it matches the legacy makefiles directory structure (on install)
- add install_dir to fix ninja install
- add post-install calls to delete .buildinfo sphinx temporary file and to install custom.css as the makefiles do
The installed directory has been verified to be bit-by-bit identical to the legacy makefiles with diffoscope.

v3: re-arranged sphinx parameters to make it work on older
versions (RHEL 7), and specify DEST filename on the install call, to make it work on older version (RHEL 7)

 doc/api/meson.build    |  5 ++---
 doc/guides/meson.build | 28 ++++++++++++++++++++++++++++
 doc/meson.build        | 11 +++++++++++
 3 files changed, 41 insertions(+), 3 deletions(-)
 create mode 100644 doc/guides/meson.build
doc/api/meson.build +2 -3
index 13fcbb8cd7..30bdc573b5 100644
@@ -50,7 +50,6 @@ if doxygen.found()
 		install_dir: htmldir,
 		build_by_default: false)
 
-	run_target('doc', command: 'true', depends: doxy_build)
-else
-	run_target('doc', command: ['echo', 'doxygen', 'not', 'found'])
+	doc_targets += doxy_build
+	doc_target_names += 'Doxygen_API'
 endif
doc/guides/meson.build new file +28 -0
new file mode 100644
index 0000000000..06f14882bb
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+
+if sphinx.found()
+	htmldir = join_paths('share', 'doc', 'dpdk')
+	html_guides_build = custom_target('html_guides_build',
+		input: meson.current_source_dir(),
+		output: 'guides',
+		command: [sphinx, '-b', 'html',
+			'-d', meson.current_build_dir() + '/.doctrees',
+			'@INPUT@', meson.current_build_dir() + '/guides'],
+		build_by_default: false,
+		install: get_option('enable_docs'),
+		install_dir: htmldir)
+
+	doc_targets += html_guides_build
+	doc_target_names += 'HTML_Guides'
+
+	# sphinx leaves a .buildinfo in the target directory, which we don't
+	# want to install. Note that sh -c has to be used, otherwise the
+	# env var does not get expanded if calling rm/install directly.
+	meson.add_install_script('sh', '-c',
+		'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
+	meson.add_install_script('sh', '-c',
+		'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')
+endif
doc/meson.build +11 -0
index afca2e7133..c5410d85d6 100644
@@ -1,4 +1,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi <[email protected]>
 
+doc_targets = []
+doc_target_names = []
 subdir('api')
+subdir('guides')
+
+if doc_targets.length() == 0
+	message = 'No docs targets found'
+else
+	message = 'Building docs:'
+endif
+run_target('doc', command: ['echo', message, doc_target_names],
+	depends: doc_targets)
Signature · 1 line
2.18.0
#11 lore raw

20/09/2018 15:51, Timothy Redaelli:

Quoted 11 lines
Quoted 4 lines · On Thu, 20 Sep 2018 14:22:08 +0100 Luca Boccassi <[email protected]> wrote:

From: Bruce Richardson <[email protected]>

Tested on Fedora 28 (sphinx 1.7.5) and RHEL 7.5 (sphinx 1.1.3)

Applied, thanks

In the build directory, docs are in
- doc/api/api/
- doc/guides/guides/
Why having the last level of (redundant) directory?

#12 lore raw
Quoted 11 lines · Thomas Monjalon

20/09/2018 15:51, Timothy Redaelli:

Quoted 10 lines
Quoted 4 lines · On Thu, 20 Sep 2018 14:22:08 +0100 Luca Boccassi <[email protected]> wrote:

From: Bruce Richardson <[email protected]>

Tested on Fedora 28 (sphinx 1.7.5) and RHEL 7.5 (sphinx 1.1.3)

Quoted 8 lines
Quoted 1 line

Applied, thanks

In the build directory, docs are in
- doc/api/api/
- doc/guides/guides/
Why having the last level of (redundant) directory?

Quirks due to the intersection of Meson and Sphinx/Doxygen and theirs respective quirks - it needs to be a new directory, and it needs to be called "guides/api" as that's what gets installed via ninja install. Yes it's not the prettiest in the build tree - suggestions to improve it are most welcome of course

Signature · 2 lines
Kind regards,
Luca Boccassi