Skip to contentSkip to conversation

doc: fix doc build when sphinx reports version to stderr

PATCH Superseded Bruce Richardson 5 participants 11 messages Jan 17, 2020 – Jul 30, 2020 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 doc: fix doc build when sphinx reports version to stderr

When sphinx-build reports its version information to stderr rather than stdout, the wrapper script misses it, and then fails to run. We can fix this by redirecting stderr to stdout for the version query call.

Fixes f5ab2074cfba ("doc: rebuild with meson whenever a file changes")
---
 buildtools/call-sphinx-build.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
buildtools/call-sphinx-build.py +3 -2
index b9a3994e1..85c9e0156 100755
@@ -6,13 +6,14 @@
 import sys
 import os
 from os.path import join
-from subprocess import run, PIPE
+from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
 (sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+ver = run([sphinx, '--version'], stdout=PIPE,
+          stderr=STDOUT).stdout.decode().split()[-1]
 sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
Signature · 1 line
2.20.1
#1 lore raw

PATCH 2/2 doc: pass "werror" setting through to doc build

When werror is set for the build, we should pass that flag through to sphinx so that it can flag warnings as errors too.

---
 buildtools/call-sphinx-build.py | 5 ++---
 doc/guides/meson.build          | 4 ++++
 2 files changed, 6 insertions(+), 3 deletions(-)
buildtools/call-sphinx-build.py +2 -3
index 85c9e0156..fc4e9d040 100755
@@ -9,12 +9,11 @@
 from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
-(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+(*sphinx_cmd, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE,
+ver = run(sphinx_cmd + ['--version'], stdout=PIPE,
           stderr=STDOUT).stdout.decode().split()[-1]
-sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
 
doc/guides/meson.build +4 -0
index 732e7ad3a..5a2b854e8 100644
@@ -7,6 +7,10 @@ if not sphinx.found()
 	subdir_done()
 endif
 
+if get_option('werror')
+	sphinx = [sphinx, '-W']
+endif
+
 htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
 	input: files('index.rst'),
Signature · 1 line
2.20.1
#2 lore raw

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

Quoted 5 lines · Bruce Richardson

When werror is set for the build, we should pass that flag through to sphinx so that it can flag warnings as errors too.

---

I see that this actually works to generate the errors... BUT

if we merge this it will break the build. Can you also insert a patch to address the warning so that the series could be merged?

Quoted 38 lines
 buildtools/call-sphinx-build.py | 5 ++---
 doc/guides/meson.build          | 4 ++++
 2 files changed, 6 insertions(+), 3 deletions(-)
buildtools/call-sphinx-build.py +2 -3
index 85c9e0156..fc4e9d040 100755
@@ -9,12 +9,11 @@
 from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
-(sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
+(*sphinx_cmd, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE,
+ver = run(sphinx_cmd + ['--version'], stdout=PIPE,
           stderr=STDOUT).stdout.decode().split()[-1]
-sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
 
doc/guides/meson.build +4 -0
index 732e7ad3a..5a2b854e8 100644
@@ -7,6 +7,10 @@ if not sphinx.found()
 	subdir_done()
 endif
 
+if get_option('werror')
+	sphinx = [sphinx, '-W']
+endif
+
 htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
 html_guides = custom_target('html_guides',
 	input: files('index.rst'),
#3 lore raw

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

Quoted 5 lines · Aaron Conole
Quoted 3 lines · Bruce Richardson

When werror is set for the build, we should pass that flag through to sphinx so that it can flag warnings as errors too.

Quoted 8 lines
Quoted 2 lines

---

I see that this actually works to generate the errors... BUT

if we merge this it will break the build. Can you also insert a patch to address the warning so that the series could be merged?

Ok, I didn't have any warnings in my setup, which is why I didn't see any problems. I assume that the warnings are showing up in travis? Anywhere else?

/Bruce

#4 lore raw

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

Quoted 11 lines · Bruce Richardson
Quoted 10 lines · Aaron Conole
Quoted 5 lines · Bruce Richardson

When werror is set for the build, we should pass that flag through to sphinx so that it can flag warnings as errors too.

---

I see that this actually works to generate the errors... BUT

Quoted 7 lines
Quoted 3 lines

if we merge this it will break the build. Can you also insert a patch to address the warning so that the series could be merged?

Ok, I didn't have any warnings in my setup, which is why I didn't see any problems. I assume that the warnings are showing up in travis? Anywhere else?

I only saw them on Travis.

Quoted 1 line

/Bruce

#5 lore raw

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

Quoted 17 lines · Aaron Conole
Quoted 15 lines · Bruce Richardson
Quoted 13 lines · Aaron Conole
Quoted 5 lines · Bruce Richardson

When werror is set for the build, we should pass that flag through to sphinx so that it can flag warnings as errors too.

---

I see that this actually works to generate the errors... BUT

if we merge this it will break the build. Can you also insert a patch to address the warning so that the series could be merged?

Quoted 6 lines
Quoted 3 lines

Ok, I didn't have any warnings in my setup, which is why I didn't see any problems. I assume that the warnings are showing up in travis? Anywhere else?

I only saw them on Travis.

The error from sphinx in travis looks like a false positive that is fixed in later versions of sphinx. The error I see is:

/home/travis/build/bruce-richardson/dpdk/doc/guides/linux_gsg/eal_args.include.rst:: WARNING: document isn't included in any toctree

However, that file is an include one that is included in both the linux and freebsd eal parameters docs, and so is not missing though not included in the index. What is the best approach to deal with this, do you think?

* rework so it has a toctree entry e.g. by creating a new section for common parameters
* other workaround in the code, e.g. rename the file to not end in .rst
* can we update sphinx in the travis build to avoid the warning altogether?

/Bruce

#6 lore raw

Re: [dpdk-dev] [PATCH 2/2] doc: pass "werror" setting through to doc build

Quoted 30 lines · Bruce Richardson
Quoted 17 lines · Aaron Conole
Quoted 15 lines · Bruce Richardson
Quoted 13 lines · Aaron Conole
Quoted 5 lines · Bruce Richardson

When werror is set for the build, we should pass that flag through to sphinx so that it can flag warnings as errors too.

---

I see that this actually works to generate the errors... BUT

if we merge this it will break the build. Can you also insert a patch to address the warning so that the series could be merged?

Quoted 6 lines
Quoted 3 lines

Ok, I didn't have any warnings in my setup, which is why I didn't see any problems. I assume that the warnings are showing up in travis? Anywhere else?

I only saw them on Travis.

The error from sphinx in travis looks like a false positive that is fixed in later versions of sphinx. The error I see is:

/home/travis/build/bruce-richardson/dpdk/doc/guides/linux_gsg/eal_args.include.rst::
WARNING: document isn't included in any toctree

Quoted 8 lines

However, that file is an include one that is included in both the linux and freebsd eal parameters docs, and so is not missing though not included in the index. What is the best approach to deal with this, do you think?

* rework so it has a toctree entry e.g. by creating a new section for common parameters
* other workaround in the code, e.g. rename the file to not end in .rst
* can we update sphinx in the travis build to avoid the warning altogether?

I think the third option is best. Distributions can always disable werror on doc builds or upgrade their own sphinx packages.

If that isn't acceptable to anyone else, then the 1st option is my next choice. I dislike the approach of renaming the file.

Quoted 1 line

/Bruce

#7 lore raw

PATCH v2 1/1 doc: fix doc build when sphinx reports version to stderr

In reply to Bruce Richardson

When sphinx-build reports its version information to stderr rather than stdout, the wrapper script misses it, and then fails to run. We can fix this by redirecting stderr to stdout for the version query call.

Fixes f5ab2074cfba ("doc: rebuild with meson whenever a file changes")

---
V2: no changes to this patch, dropped patch 2 from series.

---
 buildtools/call-sphinx-build.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
buildtools/call-sphinx-build.py +3 -2
index b9a3994e1..85c9e0156 100755
@@ -6,13 +6,14 @@
 import sys
 import os
 from os.path import join
-from subprocess import run, PIPE
+from subprocess import run, PIPE, STDOUT
 from distutils.version import StrictVersion
 
 (sphinx, src, dst) = sys.argv[1:]  # assign parameters to variables
 
 # for sphinx version >= 1.7 add parallelism using "-j auto"
-ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+ver = run([sphinx, '--version'], stdout=PIPE,
+          stderr=STDOUT).stdout.decode().split()[-1]
 sphinx_cmd = [sphinx]
 if StrictVersion(ver) >= StrictVersion('1.7'):
     sphinx_cmd += ['-j', 'auto']
Signature · 1 line
2.25.1
Tested-by on v2 1/1
#8 lore raw
Quoted 42 lines

-----Original Message-----
From: dev <[email protected]> On Behalf Of Bruce Richardson
Sent: Monday 22 June 2020 15:00
To: [email protected]
Cc: [email protected]; Richardson, Bruce <[email protected]>
Subject: [dpdk-dev] [PATCH v2] doc: fix doc build when sphinx reports version to stderr

When sphinx-build reports its version information to stderr rather than stdout, the wrapper script misses it, and then fails to run.
We can fix this by redirecting stderr to stdout for the version query call.

Fixes f5ab2074cfba ("doc: rebuild with meson whenever a file changes")

---
V2: no changes to this patch, dropped patch 2 from series.
---
buildtools/call-sphinx-build.py | 5 +++--

1 file changed, 3 insertions(+), 2 deletions(-)
buildtools/call-sphinx-build.py +0 -0
index b9a3994e1..85c9e0156 100755
@@ -6,13 +6,14 @@

import sys
import os
from os.path import join

buildtools/call-sphinx-build.py +1 -1
-from subprocess import run, PIPE
+from subprocess import run, PIPE, STDOUT

from distutils.version import StrictVersion

(sphinx, src, dst) = sys.argv[1:] # assign parameters to variables

# for sphinx version >= 1.7 add parallelism using "-j auto"
buildtools/call-sphinx-build.py +2 -1
-ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+ver = run([sphinx, '--version'], stdout=PIPE,
+          stderr=STDOUT).stdout.decode().split()[-1]

sphinx_cmd = [sphinx]
if StrictVersion(ver) >= StrictVersion('1.7'):

Quoted 3 lines
    sphinx_cmd += ['-j', 'auto']

--
2.25.1

Hi Bruce,

I hit the issue described here, and this patch fixed it for me. Thanks!

Tested-by on v2 1/1
#9 lore raw
In reply to Bruce Richardson
Quoted 11 lines · Bruce Richardson

When sphinx-build reports its version information to stderr rather than stdout, the wrapper script misses it, and then fails to run. We can fix this by redirecting stderr to stdout for the version query call.

Fixes f5ab2074cfba ("doc: rebuild with meson whenever a file changes")

---
V2: no changes to this patch, dropped patch 2 from series.

I initially encountered the issue on a fc30 when checking doc generation. fc30 is now EOL and the problem does not happen anymore on fc31.

Still worth fixing from my pov.

Thanks Bruce.

Signature · 1 line
David Marchand
#10 lore raw

22/06/2020 16:20, David Marchand:

Quoted 14 lines
Quoted 11 lines · Bruce Richardson

When sphinx-build reports its version information to stderr rather than stdout, the wrapper script misses it, and then fails to run. We can fix this by redirecting stderr to stdout for the version query call.

Fixes f5ab2074cfba ("doc: rebuild with meson whenever a file changes")

---
V2: no changes to this patch, dropped patch 2 from series.

Quoted 8 lines

I initially encountered the issue on a fc30 when checking doc generation. fc30 is now EOL and the problem does not happen anymore on fc31.

Still worth fixing from my pov.

Thanks Bruce.

Applied, thanks