sphinx-build-wrapper: add support for skipping sphinx-build
Subsystems (2)
cover sphinx-build-wrapper: add support for skipping sphinx-build
Hi Jon,
This patch adds support for not running sphinx-build at the wrapper tool. It was requested by Akira, who wanted to be able to ignore Sphinx errors during latex build and still try to build PDF.
This patch is against docs/build-script and applies after the 3 patch series I sent yesterday:
https://lore.kernel.org/linux-doc/[email protected]/
While Akira's original intention is to have pdfdocs target depend on latexdocs, IMO, this is overkill, as probably only Akira and a couple of other developers might want to have such behavior.
See, after all changes, the makefile rule for *all* doc build targets is simple:
$(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check
+$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
--sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \
--builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
--theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)After applying patch 1 from this series, it is really easy to replicate "make -i" by writing a small script that does:
tools/docs/sphinx-pre-install --version-check tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway" tools/docs/sphinx-build-wrapper -s pdfdocs
So, I'm not proposing doing any changes to Makefile, as IMHO, for most people, the intermediate steps don't matter.
Mauro Carvalho Chehab (1):
tools/docs/sphinx-build-wrapper: allow skipping sphinx-build step
tools/docs/sphinx-build-wrapper | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
Signature · 1 line
2.51.0
PATCH 1/1 tools/docs/sphinx-build-wrapper: allow skipping sphinx-build step
Most targets have two steps:
- step 1: run sphinx-build;
- step 2: run a post-build logic.
The second step can be as simple as copying static files like CSS, but may may also envolve running make. allowing to skip the first step helps debugging what's broken, and also allows using make command line arguments like --ignore-errors.
Add an option to skip step 1.
--- tools/docs/sphinx-build-wrapper | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
tools/docs/sphinx-build-wrapper +12 -6
@@ -603,7 +603,8 @@ class SphinxBuilder: shutil.rmtree(self.builddir, ignore_errors=True) def build(self, target, sphinxdirs=None, - theme=None, css=None, paper=None, deny_vf=None, rustdoc=False): + theme=None, css=None, paper=None, deny_vf=None, rustdoc=False, + skip_sphinx=False): """ Build documentation using Sphinx. This is the core function of this module. It prepares all arguments required by sphinx-build.
@@ -625,9 +626,10 @@ class SphinxBuilder: # # Other targets require sphinx-build, so check if it exists # - sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"]) - if not sphinxbuild and target != "mandocs": - sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") + if not skip_sphinx: + sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"]) + if not sphinxbuild and target != "mandocs": + sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") if builder == "latex": if not self.pdflatex_cmd and not self.latexmk_cmd:
@@ -713,7 +715,7 @@ class SphinxBuilder: if target == "mandocs": self.handle_man(kerneldoc, docs_dir, src_dir, output_dir) - else: + elif not skip_sphinx: try: result = self.run_sphinx(sphinxbuild, build_args, env=self.env)
@@ -795,6 +797,9 @@ def main(): parser.add_argument('-i', '--interactive', action='store_true', help="Change latex default to run in interactive mode") + parser.add_argument('-s', '--skip-sphinx-build', action='store_true', + help="Skip sphinx-build step") + parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}', default=None, help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})')
@@ -810,7 +815,8 @@ def main(): builder.build(args.target, sphinxdirs=args.sphinxdirs, theme=args.theme, css=args.css, paper=args.paper, - rustdoc=args.rustdoc, deny_vf=args.deny_vf) + rustdoc=args.rustdoc, deny_vf=args.deny_vf, + skip_sphinx=args.skip_sphinx_build) if __name__ == "__main__": main()
Signature · 1 line
2.51.0
Quoted 5 lines · Mauro Carvalho Chehab
Hi Jon,
This patch adds support for not running sphinx-build at the wrapper tool. It was requested by Akira, who wanted to be able to ignore Sphinx errors during latex build and still try to build PDF.
Thank you for trying to figure out my intention.
However, you failed to see the point.
Quoted 10 lines
This patch is against docs/build-script and applies after the 3 patch series I sent yesterday:
https://lore.kernel.org/linux-doc/[email protected]/While Akira's original intention is to have pdfdocs target depend on latexdocs, IMO, this is overkill, as probably only Akira and a couple of other developers might want to have such behavior.
I think it is only you who don't want such behavior.
Quoted 10 lines
See, after all changes, the makefile rule for *all* doc build targets is simple:
$(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \ --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \ --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)
Quoted 7 lines
After applying patch 1 from this series, it is really easy to replicate "make -i" by writing a small script that does:
tools/docs/sphinx-pre-install --version-check tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway" tools/docs/sphinx-build-wrapper -s pdfdocs
Hello?
You are the one who is changing the way "make pdfdocs" behaves. All I want is to restore the current behavior, without any need to use such an ad-hoc script.
Sorry, but I think I have to NAK this.
Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs" needs a successful "latexdocs" stage.
I believe Documentation/Makefile is the right place to describe it.
Good luck,
Akira
Em Mon, 22 Sep 2025 20:30:40 +0900
Akira Yokosawa <[email protected]> escreveu:
Quoted 14 lines · Mauro Carvalho Chehab
Quoted 5 lines
Hi Jon,
This patch adds support for not running sphinx-build at the wrapper tool. It was requested by Akira, who wanted to be able to ignore Sphinx errors during latex build and still try to build PDF.
Thank you for trying to figure out my intention.
However, you failed to see the point.Quoted 4 lines
This patch is against docs/build-script and applies after the 3 patch series I sent yesterday:
Quoted 8 lines
Quoted 6 lines
https://lore.kernel.org/linux-doc/[email protected]/While Akira's original intention is to have pdfdocs target depend on latexdocs, IMO, this is overkill, as probably only Akira and a couple of other developers might want to have such behavior.
I think it is only you who don't want such behavior.
Whom else wants to use "make -i" to skip failed latexdocs build and still do pdf builds? For what reason?
Quoted 12 lines
Quoted 12 lines
See, after all changes, the makefile rule for *all* doc build targets is simple:
$(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \ --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \ --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)After applying patch 1 from this series, it is really easy to replicate "make -i" by writing a small script that does:
Quoted 8 lines
Quoted 4 lines
tools/docs/sphinx-pre-install --version-check tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway" tools/docs/sphinx-build-wrapper -s pdfdocsHello?
You are the one who is changing the way "make pdfdocs" behaves.
It doesn't change. if latex is not built, it won't try to build PDF files, just like before.
See, for almost all targets, docs build is a two-step procedure. That includes htmldocs as well, as it requires a post-build step to copy css and static files. Right now, the logic is hidden inside complex make macros.
The new logic place the target-dependent extra steps on separate functions which are now properly documented, executing them only if the first step works.
With the "-s" parameter, one can skip the first step, running only the second one.
Quoted 2 lines
All I want is to restore the current behavior, without any need to use such an ad-hoc script.
Huh?
You said you want to be able to do just the reverse: to ignore failures at latex build ("make -i"). If you want to have exactly the same behavior, you don't need anything.
Quoted 6 lines
Sorry, but I think I have to NAK this.
Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs" needs a successful "latexdocs" stage.
I believe
Documentation/Makefileis the right place to describe it.
If you want to propose such change, be my guest. As I said, *I* won't propose it, as IMO it is a bad idea, but if you want, feel free to submit a patch after this one similar to:
-htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
+htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs linkcheckdocs:
+
+pdfdocs: latexdocs
+ +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ -s \
+ --sphinxdirs="$(SPHINXDIRS)" \
+ --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
+ --paper=$(PAPER)
and let's discuss its pros/cons in separate.
Thanks,
Mauro
Quoted 20 lines · Mauro Carvalho Chehab
Em Mon, 22 Sep 2025 20:30:40 +0900
Akira Yokosawa <[email protected]> escreveu:Quoted 17 lines · Mauro Carvalho Chehab
Quoted 5 lines
Hi Jon,
This patch adds support for not running sphinx-build at the wrapper tool. It was requested by Akira, who wanted to be able to ignore Sphinx errors during latex build and still try to build PDF.
Thank you for trying to figure out my intention.
However, you failed to see the point.Quoted 7 lines
This patch is against docs/build-script and applies after the 3 patch series I sent yesterday:
https://lore.kernel.org/linux-doc/[email protected]/
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
While Akira's original intention is to have pdfdocs target depend on latexdocs, IMO, this is overkill, as probably only Akira and a couple of other developers might want to have such behavior.
I think it is only you who don't want such behavior.
Whom else wants to use "make -i" to skip failed latexdocs build and still do pdf builds? For what reason?
Hmm, looks like you were suffered from context mismatch.
My message you tried to understand (archived at:
https://lore.kernel.org/[email protected]/
) started this way:
Hi, (just got v8, but sending anyway ...),
, and the message assumed the "make pdfdocs" behavior of v7.
At the time, I thought it was what you had intended and it looked as though you wanted latexmk/xelatex to continue running as much as possible ignoring any intermediate errors.
You have since fixed the missed exception handling at the latexdocs phase, but your rapid respins of this series simply overwhelmed me and I have failed to see "make pdfdocs" now gives up at latexdocs error.
My intention of mentioning GNU Make's options of -i, -k, etc. in the message was to show you that the make--sub-make scheme is so flexible and it can even emulate the buggy behavior of then sphinx-build-wrapper.
In normal cases, I won't use any of those options.
But as a human developer, I sometimes use them when I observe build errors somewhere in "make pdfdocs".
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
See, after all changes, the makefile rule for *all* doc build targets is simple:
$(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \ --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \ --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER)After applying patch 1 from this series, it is really easy to replicate "make -i" by writing a small script that does:
tools/docs/sphinx-pre-install --version-check tools/docs/sphinx-build-wrapper latexdocs || echo "LaTeX build failed, but we'll try build PDF anyway" tools/docs/sphinx-build-wrapper -s pdfdocs
Quoted 6 lines
Quoted 3 lines
Hello?
You are the one who is changing the way "make pdfdocs" behaves.
It doesn't change. if latex is not built, it won't try to build PDF files, just like before.
Didn't you mean "if latexdocs is not build"?
Yes, I now see this is the current behavior.
Quoted 13 lines
See, for almost all targets, docs build is a two-step procedure. That includes htmldocs as well, as it requires a post-build step to copy css and static files. Right now, the logic is hidden inside complex make macros.
The new logic place the target-dependent extra steps on separate functions which are now properly documented, executing them only if the first step works.
With the "-s" parameter, one can skip the first step, running only the second one.
Quoted 7 lines
Quoted 2 lines
All I want is to restore the current behavior, without any need to use such an ad-hoc script.
Huh?
You said you want to be able to do just the reverse: to ignore failures at latex build ("make -i").
No, as I said above, I don't want it in most cases.
But in rare occasions, I might want it. That depends on the situation at hand.
Quoted 23 lines
If you want to have exactly
the same behavior, you don't need anything.Quoted 6 lines
Sorry, but I think I have to NAK this.
Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs" needs a successful "latexdocs" stage.
I believe
Documentation/Makefileis the right place to describe it.If you want to propose such change, be my guest. As I said, *I* won't propose it, as IMO it is a bad idea, but if you want, feel free to submit a patch after this one similar to:
-htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
+htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs linkcheckdocs:
+
+pdfdocs: latexdocs
+ +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ -s \
+ --sphinxdirs="$(SPHINXDIRS)" \
+ --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
+ --paper=$(PAPER)
Quoted 1 line
and let's discuss its pros/cons in separate.
OK. This looks like the way forward to me.
I'll prepare a patch (set) and submit it after v6.18-rc1.
In the meantime, I think we might be better off if we could root-cause the "latexmk bug" you repeatedly claimed you had observed, without providing any steps to reproduce.
Let me continue in another thread.
It would be far more interesting for me to (try to) figure out what you had observed.
Thanks,
Akira
Em Wed, 24 Sep 2025 16:03:59 +0900
Akira Yokosawa <[email protected]> escreveu:
Quoted 38 lines · Mauro Carvalho Chehab
Quoted 20 lines
Em Mon, 22 Sep 2025 20:30:40 +0900
Akira Yokosawa <[email protected]> escreveu:Quoted 17 lines · Mauro Carvalho Chehab
Quoted 5 lines
Hi Jon,
This patch adds support for not running sphinx-build at the wrapper tool. It was requested by Akira, who wanted to be able to ignore Sphinx errors during latex build and still try to build PDF.
Thank you for trying to figure out my intention.
However, you failed to see the point.Quoted 7 lines
This patch is against docs/build-script and applies after the 3 patch series I sent yesterday:
https://lore.kernel.org/linux-doc/[email protected]/Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
While Akira's original intention is to have pdfdocs target depend on latexdocs, IMO, this is overkill, as probably only Akira and a couple of other developers might want to have such behavior.
I think it is only you who don't want such behavior.
Whom else wants to use "make -i" to skip failed latexdocs build and still do pdf builds? For what reason?
Hmm, looks like you were suffered from context mismatch.
My message you tried to understand (archived at:
https://lore.kernel.org/[email protected]/) started this way:
Quoted 7 lines
Hi, (just got v8, but sending anyway ...),
, and the message assumed the "make pdfdocs" behavior of v7.
At the time, I thought it was what you had intended and it looked as though you wanted latexmk/xelatex to continue running as much as possible ignoring any intermediate errors.
No, this was a bug, as I answered you already and sent the fix:
if sphinx-build fails, the script shall bail out.
Quoted 11 lines
You have since fixed the missed exception handling at the latexdocs phase, but your rapid respins of this series simply overwhelmed me and I have failed to see "make pdfdocs" now gives up at latexdocs error.
My intention of mentioning GNU Make's options of -i, -k, etc. in the message was to show you that the make--sub-make scheme is so flexible and it can even emulate the buggy behavior of then sphinx-build-wrapper.
In normal cases, I won't use any of those options.
But as a human developer, I sometimes use them when I observe build errors somewhere in "make pdfdocs".
...
Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
All I want is to restore the current behavior, without any need to use such an ad-hoc script.
Huh?
You said you want to be able to do just the reverse: to ignore failures at latex build ("make -i").
No, as I said above, I don't want it in most cases.
But in rare occasions, I might want it. That depends on the situation at hand.
There is exactly the usecase where running directly via makefile may not be enough, as you may want to play with different scenarios.
On such cases, you'll be better served by calling the script directly, as it provides more flexibility without requiring to pass parameters via env vars nor nor enable/disable different venvs to test with different versions of docutils/spinx packages.
On an hypothetical scenario where pdfdocs is not working fine, calling the script directly allows to easily test different scenarios, and it is faster, as you won't do anything else but running directly the doc build logic:
# equivalent to "make pdfdocs": # build tex. If sphinx-build succeeds, build pdf for each tex ./tools/docs/sphinx-build-wrapper pdfdocs
# equivalent to "make latexdocs" ./tools/docs/sphinx-build-wrapper latexdocs
# generate latex with Sphinx 3.4.3 in verbose mode # equivalent to: # . Sphinx_3.4.3/bin/activate # make V=1 latexdocs # deactivate ./tools/docs/sphinx-build-wrapper latexdocs -V Sphinx_3.4.3 -v
# build only translations.tex with Sphinx 5.2.0 in verbose mode ./tools/docs/sphinx-build-wrapper latexdocs -V Sphinx_5.2.0 -v --sphinxdirs translations
Once you're happy with the above, you could proceed with the pdf build step, forcing the script to skip sphinx-build step with:
# Run just PDF build step, producing a summary at the end $ ./tools/docs/sphinx-build-wrapper pdfdocs -s -v ... Summary ======= ... translations/pdf/translations.pdf: FAILED ... # After building latexdocs for translations.tex and core-api.tex # run the run test PDF build, producing a single line # with all failures for each generated .tex file: $ ./tools/docs/sphinx-build-wrapper pdfdocs -s --sphinxdirs translations core-api ... Error: Can't build 1 PDF file(s): translations/pdf/translations.pdf # Use a different variable font deny configuration directory $ ./tools/docs/sphinx-build-wrapper pdfdocs -s --sphinxdirs translations --deny-vf ~/new-deny-vf/
Quoted 18 lines
Quoted 18 lines
Quoted 4 lines
Furthermore, your "cleanup" is obfuscating the very fact that "pdfdocs" needs a successful "latexdocs" stage.
I believe
Documentation/Makefileis the right place to describe it.If you want to propose such change, be my guest. As I said, *I* won't propose it, as IMO it is a bad idea, but if you want, feel free to submit a patch after this one similar to:
-htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs:
+htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs linkcheckdocs:
+
+pdfdocs: latexdocs
+ +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ -s \
+ --sphinxdirs="$(SPHINXDIRS)" \
+ --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \
+ --paper=$(PAPER)
Quoted 5 lines
Quoted 1 line
and let's discuss its pros/cons in separate.
OK. This looks like the way forward to me.
I'll prepare a patch (set) and submit it after v6.18-rc1.
Ok, but as I said, IMO we're better served running the script directly while debugging issues.
Quoted 3 lines
In the meantime, I think we might be better off if we could root-cause the "latexmk bug" you repeatedly claimed you had observed, without providing any steps to reproduce.
This is on my todo list.
With regards to the steps to reproduce you need to remove this from make:
|| sh $(srctree)/scripts/check-variable-fonts.sh || exit
to ensure that errors will be handled by make.
As we depend on error handling logic, it means that you may need to build docs with different configurations, different types of latex errors produced by sphinx-build, and test with different versions of docutils, sphinx, xelatex and latexmk, and hope to have a .tex file that has errors but still generate .pdf.
Getting such issues is time consuming and it needs a certain amount of luck to catch it, as you depend on changes at the *.rst files that will produce a .tex file with the exact amount of spicy to produce such behavior.
With me, it only happens when I don't want it to happen ;-)
-
Another option would be to try to manually generate a defective *.tex file that doesn't produce a critical error but still generates a.pdf output when xelatex is called by hand.
Anyway, this requires a calm week where you don't have anything else to do, and you're prepared to waste several hours trying to pick the hot spot on away that it would be reproducible by others.
So, this has low priority.
Quoted 7 lines
Let me continue in another thread.
It would be far more interesting for me to (try to) figure out what you had observed.Thanks,
Akira
Thanks,
Mauro