Documentation/media/uapi/cec/ sporadically unnecessarily rebuilding

I think I saw some of this in the past [1], but then couldn't reproduce it after all. Now I'm seeing it again. Sporadically Documentation/media/uapi/cec/ gets rebuilt on successive runs of make htmldocs, even when nothing has changed.

Output of 'make SPHINXOPTS="-v -v" htmldocs' attached for both cases.

Using Sphinx (sphinx-build) 1.4.6

BR,
Jani.

[1] id:[email protected]

Jani Nikula, Intel Open Source Technology Center

SPHINX htmldocs --> file:///home/jani/src/linux/Documentation/output;
make[2]: Nothing to be done for 'all'.
Running Sphinx v1.4.6
[app] setting up extension: 'kernel-doc'
[app] adding config value: ('kerneldoc_bin', None, 'env')
[app] adding config value: ('kerneldoc_srctree', None, 'env')
[app] adding config value: ('kerneldoc_verbosity', 1, 'env')
[app] adding directive: ('kernel-doc', <class 'kernel-doc.KernelDocDirective'>, None, None, {})
[app] setting up extension: 'rstFlatTable'
[app] adding directive: ('flat-table', <class 'rstFlatTable.FlatTable'>, None, None, {})
[app] setting up extension: 'kernel_include'
[app] adding directive: ('kernel-include', <class 'kernel_include.KernelInclude'>, None, None, {})
[app] setting up extension: 'cdomain'
[app] overriding domain: <class 'cdomain.CDomain'>
[app] setting up extension: 'sphinx.ext.imgmath'
[app] adding config value: ('math_number_all', False, 'html')
[app] adding node: (<class 'sphinx.ext.mathbase.math'>, {'latex': (<function latex_visit_math at 0x7f27758aea60>, None), 'man': (<function man_visit_math at 0x7f27758aee18>, None), 'text': (<function text_visit_math at 0x7f27758aec80>, None), 'override': True, 'html': (<function html_visit_math at 0x7f27758ba6a8>, None), 'texinfo': (<function texinfo_visit_math at 0x7f27758ba0d0>, None)})
[app] adding node: (<class 'sphinx.ext.mathbase.displaymath'>, {'latex': (<function latex_visit_displaymath at 0x7f27758aeb70>, None), 'man': (<function man_visit_displaymath at 0x7f27758aeea0>, <function man_depart_displaymath at 0x7f27758aef28>), 'text': (<function text_visit_displaymath at 0x7f27758aed08>, None), 'html': (<function html_visit_displaymath at 0x7f27758ba730>, None), 'texinfo': (<function texinfo_visit_displaymath at 0x7f27758ba158>, <function texinfo_depart_displaymath at 0x7f27758ba1e0>)}) [app] adding node: (<class 'sphinx.ext.mathbase.eqref'>, {'latex': (<function latex_visit_eqref at 0x7f27758aebf8>, None), 'man': (<function man_visit_eqref at 0x7f27758ba048>, None), 'text': (<function text_visit_eqref at 0x7f27758aed90>, None), 'html': (<function html_visit_eqref at 0x7f27758ba2f0>, <function html_depart_eqref at 0x7f27758ba378>), 'texinfo': (<function texinfo_visit_eqref at 0x7f27758ba268>, None)})
[app] adding role: ('math', <function math_role at 0x7f27758ae8c8>)
[app] adding role: ('eq', <function eq_role at 0x7f27758ae950>)
[app] adding directive: ('math', <class 'sphinx.ext.mathbase.MathDirective'>, None, None, {})
[app] connecting event 'doctree-resolved': <function number_equations at 0x7f27758ba400> [id=0]
[app] adding config value: ('imgmath_image_format', 'png', 'html')
[app] adding config value: ('imgmath_dvipng', 'dvipng', 'html')
[app] adding config value: ('imgmath_dvisvgm', 'dvisvgm', 'html')
[app] adding config value: ('imgmath_latex', 'latex', 'html')
[app] adding config value: ('imgmath_use_preview', False, 'html')
[app] adding config value: ('imgmath_dvipng_args', ['-gamma', '1.5', '-D', '110', '-bg', 'Transparent'], 'html')
[app] adding config value: ('imgmath_dvisvgm_args', ['--no-fonts'], 'html')
[app] adding config value: ('imgmath_latex_args', [], 'html')
[app] adding config value: ('imgmath_latex_preamble', '', 'html')
[app] adding config value: ('imgmath_add_tooltips', True, 'html')
[app] adding config value: ('imgmath_font_size', 12, 'html')
[app] connecting event 'build-finished': <function cleanup_tempdir at 0x7f27758ba598> [id=1]
[app] setting up extension: 'alabaster'
[app] connecting event 'html-page-context': <function update_context at 0x7f27751f1ea0> [id=2]
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... 12 found
pickling environment... done
checking consistency... done
docnames to write: media/uapi/cec/cec-func-close, media/uapi/cec/cec-func-ioctl, media/uapi/cec/cec-func-open, media/uapi/cec/cec-func-poll, media/uapi/cec/cec-funcs, media/uapi/cec/cec-header, media/uapi/cec/cec-ioc-adap-g-caps, media/uapi/cec/cec-ioc-adap-g-log-addrs, media/uapi/cec/cec-ioc-adap-g-phys-addr, media/uapi/cec/cec-ioc-dqevent, media/uapi/cec/cec-ioc-g-mode, media/uapi/cec/cec-ioc-receive
preparing documents... WARNING: search index couldn't be loaded, but not all documents will be built: the index will be incomplete.
done
writing output... [ 7%] index
writing output... [ 14%] media/uapi/cec/cec-api
writing output... [ 21%] media/uapi/cec/cec-func-close
writing output... [ 28%] media/uapi/cec/cec-func-ioctl
writing output... [ 35%] media/uapi/cec/cec-func-open
writing output... [ 42%] media/uapi/cec/cec-func-poll
writing output... [ 50%] media/uapi/cec/cec-funcs
writing output... [ 57%] media/uapi/cec/cec-header
writing output... [ 64%] media/uapi/cec/cec-ioc-adap-g-caps
writing output... [ 71%] media/uapi/cec/cec-ioc-adap-g-log-addrs
writing output... [ 78%] media/uapi/cec/cec-ioc-adap-g-phys-addr
writing output... [ 85%] media/uapi/cec/cec-ioc-dqevent
writing output... [ 92%] media/uapi/cec/cec-ioc-g-mode
writing output... [100%] media/uapi/cec/cec-ioc-receive

generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done dumping object inventory... done
build succeeded, 1 warning.
HTML Documentation/DocBook/index.html

SPHINX htmldocs --> file:///home/jani/src/linux/Documentation/output;
make[2]: Nothing to be done for 'all'.
Running Sphinx v1.4.6
[app] setting up extension: 'kernel-doc'
[app] adding config value: ('kerneldoc_bin', None, 'env')
[app] adding config value: ('kerneldoc_srctree', None, 'env')
[app] adding config value: ('kerneldoc_verbosity', 1, 'env')
[app] adding directive: ('kernel-doc', <class 'kernel-doc.KernelDocDirective'>, None, None, {})
[app] setting up extension: 'rstFlatTable'
[app] adding directive: ('flat-table', <class 'rstFlatTable.FlatTable'>, None, None, {})
[app] setting up extension: 'kernel_include'
[app] adding directive: ('kernel-include', <class 'kernel_include.KernelInclude'>, None, None, {})
[app] setting up extension: 'cdomain'
[app] overriding domain: <class 'cdomain.CDomain'>
[app] setting up extension: 'sphinx.ext.imgmath'
[app] adding config value: ('math_number_all', False, 'html')
[app] adding node: (<class 'sphinx.ext.mathbase.math'>, {'override': True, 'html': (<function html_visit_math at 0x7f365c4fe6a8>, None), 'man': (<function man_visit_math at 0x7f365c4f2e18>, None), 'text': (<function text_visit_math at 0x7f365c4f2c80>, None), 'texinfo': (<function texinfo_visit_math at 0x7f365c4fe0d0>, None), 'latex': (<function latex_visit_math at 0x7f365c4f2a60>, None)})
[app] adding node: (<class 'sphinx.ext.mathbase.displaymath'>, {'text': (<function text_visit_displaymath at 0x7f365c4f2d08>, None), 'texinfo': (<function texinfo_visit_displaymath at 0x7f365c4fe158>, <function texinfo_depart_displaymath at 0x7f365c4fe1e0>), 'latex': (<function latex_visit_displaymath at 0x7f365c4f2b70>, None), 'html': (<function html_visit_displaymath at 0x7f365c4fe730>, None), 'man': (<function man_visit_displaymath at 0x7f365c4f2ea0>, <function man_depart_displaymath at 0x7f365c4f2f28>)}) [app] adding node: (<class 'sphinx.ext.mathbase.eqref'>, {'text': (<function text_visit_eqref at 0x7f365c4f2d90>, None), 'texinfo': (<function texinfo_visit_eqref at 0x7f365c4fe268>, None), 'latex': (<function latex_visit_eqref at 0x7f365c4f2bf8>, None), 'html': (<function html_visit_eqref at 0x7f365c4fe2f0>, <function html_depart_eqref at 0x7f365c4fe378>), 'man': (<function man_visit_eqref at 0x7f365c4fe048>, None)})
[app] adding role: ('math', <function math_role at 0x7f365c4f28c8>)
[app] adding role: ('eq', <function eq_role at 0x7f365c4f2950>)
[app] adding directive: ('math', <class 'sphinx.ext.mathbase.MathDirective'>, None, None, {})
[app] connecting event 'doctree-resolved': <function number_equations at 0x7f365c4fe400> [id=0]
[app] adding config value: ('imgmath_image_format', 'png', 'html')
[app] adding config value: ('imgmath_dvipng', 'dvipng', 'html')
[app] adding config value: ('imgmath_dvisvgm', 'dvisvgm', 'html')
[app] adding config value: ('imgmath_latex', 'latex', 'html')
[app] adding config value: ('imgmath_use_preview', False, 'html')
[app] adding config value: ('imgmath_dvipng_args', ['-gamma', '1.5', '-D', '110', '-bg', 'Transparent'], 'html')
[app] adding config value: ('imgmath_dvisvgm_args', ['--no-fonts'], 'html')
[app] adding config value: ('imgmath_latex_args', [], 'html')
[app] adding config value: ('imgmath_latex_preamble', '', 'html')
[app] adding config value: ('imgmath_add_tooltips', True, 'html')
[app] adding config value: ('imgmath_font_size', 12, 'html')
[app] connecting event 'build-finished': <function cleanup_tempdir at 0x7f365c4fe598> [id=1]
[app] setting up extension: 'alabaster'
[app] connecting event 'html-page-context': <function update_context at 0x7f365bddd8c8> [id=2]
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.
build succeeded.
HTML Documentation/DocBook/index.html

Em Mon, 24 Oct 2016 12:04:31 +0300
Jani Nikula <[email protected]> escreveu:

I notice some erratic behavior like that too, when I was writing the admin-guide and process books... Sometimes, Sphinx decide to rebuild everything (media, gpu, etc) without a good reason. Also, sometimes, I was forced to run make cleandocs, as just touching at an rst file or at a file parsed by kernel-doc is not enough.

The only safe way to be sure that the documentation will be OK and won't take too long to build is to use:

make cleandocs; make DOCBOOKS="" SPHINXDIRS="admin-guide process" SPHINXOPTS="-j11" htmldocs

It will generate warnings, though, due to the cross-references between admin-guide and process books. Due to that, after testing for individual changes, I run:

make cleandocs; make DOCBOOKS="" SPHINXOPTS="-j11" htmldocs

To check those references.

I didn't see any improvement for that on Sphinx 1.4.7 or 1.4.8.

Thanks,
Mauro

Hi Jani,

Am 24.10.2016 um 11:04 schrieb Jani Nikula <[email protected]>:

I can't see what's wrong with your "rebuild" file ...

<build-cec-rebuilding.txt --------->
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date building [html]: targets for 0 source files that are out of date updating environment: 0 added, 0 changed, 0 removed looking for now-outdated files... none found
no targets are out of date.
build succeeded.
HTML Documentation/DocBook/index.html
<build-cec-rebuilding.txt --------->

Sphinx loads the cached (pickled) environment and says, that no target was outdated. The build succeeded without any rebuild. IMO it is sane build ... or do I misunderstood you?

-- Markus --

Awesome, I screwed up the file names, please check again with build-cec-rebuilding.txt <-> build-ok.txt...

BR,
Jani.

Jani Nikula, Intel Open Source Technology Center

Ah, ok .. I can reproduce the error.

It seems that sphinx's ".. toctree::" don't like it, if you build a structure where severals ".. toctrees" in the same folder include files of this folder.

E.g. if you have "myfolder" with "index.rst", "f1.rst" and "f2.rst" in (content see below) and you rebuild it 5 or more times, the files f1.rst and f2.rst will sporadically rebuild.

May I have time to find the bug and send a fix to sphinx.

-- Markus --

------------------------------------------------------
index
=====

.. toctree::

    :maxdepth: 1
    :numbered:

    f1

------------------------------------------------------
f1
==

.. toctree::

    :maxdepth: 1
    :numbered:

    f2

------------------------------------------------------
f2
==

lorem
------------------------------------------------------