docs: Detect variable fonts and suggest removing them
Subsystems (2)
PATCH 1/1 docs: Detect variable fonts and suggest removing them
xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.
To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.
---
This is not in line with Jon's suggestion of tweaking sphinx-pre-install and/or kerneldoc-preamble.sty [1].
[1]: https://lore.kernel.org/r/[email protected]/
It would be ideal if kerneldoc-preamble.sty can be taught about the way to detect the existence of variable fonts and to skip CJK contents. However, it seems that it would add overly complex code for choosing fonts by file names with meticulous options to select CJK variant (SC, TC, KR, or JP) to use. Furthermore, as location and names of font files can be different among distro packages, keeping portability of the preamble is impossible in practice.
As for sphinx-pre-install, even if variable "Noto CJK" fonts are found, they don't matter for non-CJK PDFs. Emitting warning every time would be noisy.
Therefore, as a minimal approach, to help developers find a way forward, giving suggestions after the fact sounds like a reasonable approach to me, at least for the time being.
Jon, you said in [1]:
Quoted 8 lines
Quoted 5 lines
What does
fc-list | grep NotoSansCJK-VF.ttc
say?
No output at all, even though I had google-noto-sans-cjk-vf-fonts installed.
So I'm wondering if scripts/check-variable-font.sh in this patch emits useful suggestions. Could you give it a try?
- Install google-noto-sans-cjk-vf-fonts
- Install google-noto-serif-cjk-vf-fonts
- What does "sh scripts/check-variable-font.sh" say?
- Run "make cleandocs; make SPHINXDIRS=translations pdfdocs". Do you see the suggestion after the build error?
- Uninstall above 2 font packages.
- What does "sh scripts/check-variable-font.sh" say now?
- Run "make cleandocs; make SPHINXDIRS=translations pdfdocs" again. Does it complete successfully?
Thanks, Akira
---
Cc: [email protected]
--- Documentation/Makefile | 2 +- MAINTAINERS | 1 + scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+), 1 deletion(-) create mode 100755 scripts/check-variable-fonts.sh
Documentation/Makefile +1 -1
@@ -154,7 +154,7 @@ else # HAVE_PDFLATEX pdfdocs: latexdocs @$(srctree)/scripts/sphinx-pre-install --version-check $(foreach var,$(SPHINXDIRS), \ - $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \ + $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \ mkdir -p $(BUILDDIR)/$(var)/pdf; \ mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \ )
MAINTAINERS +1 -0
@@ -6346,6 +6346,7 @@ S: Maintained P: Documentation/doc-guide/maintainer-profile.rst T: git git://git.lwn.net/linux.git docs-next F: Documentation/ +F: scripts/check-variable-font.sh F: scripts/documentation-file-ref-check F: scripts/kernel-doc F: scripts/sphinx-pre-install
scripts/check-variable-fonts.sh new file +30 -0
@@ -0,0 +1,30 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# For "make pdfdocs", recent trend of deploying variable type of +# "Noto Sans CJK" and "Noto Serif CJK" fonts breaks xelatex, which does +# not understand variable fonts. +# +# It is hard to distinguish variable fonts from static ones in the preamble +# of LaTeX source code. Instead, this script is invoked in the error path +# of "make pdfdocs" and emit suggestions if such font files are found. +# +# Assumption: +# File names are not changed from those of upstream Noto CJK fonts: +# https://github.com/notofonts/noto-cjk/ + +vffonts=`fc-list -b | grep -i noto | grep -i cjk | grep -F -i -e "-vf" | \ + sort | uniq | sed -e 's/\tfile:/ file:/' | sed -e 's/(s)$//'` + +if [ "x$vffonts" != "x" ] ; then + echo "=====================================================================" + echo "Detected variable form of Noto CJK fonts incompatible with xelatex:" + echo "$vffonts" + echo "If you need CJK contents in PDF, remove them and install static ones." + echo "Otherwise, get rid of texlive-xecjk." + echo "=====================================================================" +fi + +# As this script is invoked from Makefile's error path, always error exit +# even if no variable font is detected. +exit 1
base-commit: b8cfda5c9065cd619a97c17da081cbfab3b1e756
Signature · 1 line
2.34.1
Randy Dunlap
reviewerHi Akira,
Quoted 15 lines · Akira Yokosawa
xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.
To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.
Signed-off-by Akira Yokosawa <[email protected]>Reported-by Jonathan Corbet <[email protected]>Reported-by "Иван Иванович" <[email protected]>---
[snip]
Quoted 9 lines
---
Cc: [email protected]--- Documentation/Makefile | 2 +- MAINTAINERS | 1 + scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+), 1 deletion(-) create mode 100755 scripts/check-variable-fonts.sh
I don't have much to say about the patch except that it seems like a good idea...
However, some of the terminology could be improved IMO. E.g., "variable type" or "variable font" should be something like variable-width font or proportionally-spaced font.
"static ones" should be something like fixed-space fonts or monospaced fonts.
Unless I just completely don't understand the uses of "variable" and "static" here.
Quoted 37 lines
scripts/check-variable-fonts.sh new file +30
-0@@ -0,0 +1,30 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# For "make pdfdocs", recent trend of deploying variable type of +# "Noto Sans CJK" and "Noto Serif CJK" fonts breaks xelatex, which does +# not understand variable fonts. +# +# It is hard to distinguish variable fonts from static ones in the preamble +# of LaTeX source code. Instead, this script is invoked in the error path +# of "make pdfdocs" and emit suggestions if such font files are found. +# +# Assumption: +# File names are not changed from those of upstream Noto CJK fonts: +# https://github.com/notofonts/noto-cjk/ + +vffonts=`fc-list -b | grep -i noto | grep -i cjk | grep -F -i -e "-vf" | \ + sort | uniq | sed -e 's/\tfile:/ file:/' | sed -e 's/(s)$//'` + +if [ "x$vffonts" != "x" ] ; then + echo "=====================================================================" + echo "Detected variable form of Noto CJK fonts incompatible with xelatex:" + echo "$vffonts" + echo "If you need CJK contents in PDF, remove them and install static ones." + echo "Otherwise, get rid of texlive-xecjk." + echo "=====================================================================" +fi + +# As this script is invoked from Makefile's error path, always error exit +# even if no variable font is detected. +exit 1
Quoted 1 line
base-commit:
b8cfda5c9065cd619a97c17da081cbfab3b1e756
Thanks.
Signature · 1 line
#Randy
Quoted 1 line · Randy Dunlap
Hi Akira,
Thank you for your comments, Randy.
Quoted 30 lines · Akira Yokosawa
Quoted 15 lines
xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.
To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.
Signed-off-by Akira Yokosawa <[email protected]>Reported-by Jonathan Corbet <[email protected]>Reported-by "Иван Иванович" <[email protected]>---
[snip]
Quoted 9 lines
---
Cc: [email protected]--- Documentation/Makefile | 2 +- MAINTAINERS | 1 + scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+), 1 deletion(-) create mode 100755 scripts/check-variable-fonts.sh
Quoted 5 lines
I don't have much to say about the patch except that it seems like a good idea...
However, some of the terminology could be improved IMO. E.g., "variable type" or "variable font" should be something like variable-width font or proportionally-spaced font.
Good point.
I need to be more careful in using terms rarely found in kernel development.
In this case, "variable font" means "font file which employs the OpenType font variations technology".
There is a Wikipedia page at: https://en.wikipedia.org/wiki/Variable_font
Quoted 4 lines
"static ones" should be something like fixed-space fonts or monospaced fonts.
Unless I just completely don't understand the uses of "variable" and "static" here.
"static" here means "non-variable".
I'll expand the changelog a bit in v2 with your comments in mind.
Thanks, Akira
Randy Dunlap
reviewerQuoted 17 lines
Quoted 11 lines
Quoted 10 lines
---
Cc: [email protected]--- Documentation/Makefile | 2 +- MAINTAINERS | 1 + scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+), 1 deletion(-) create mode 100755 scripts/check-variable-fonts.shQuoted 5 lines
I don't have much to say about the patch except that it seems like a good idea...
However, some of the terminology could be improved IMO. E.g., "variable type" or "variable font" should be something like variable-width font or proportionally-spaced font.
Quoted 9 lines
Good point.
I need to be more careful in using terms rarely found in kernel development.
In this case, "variable font" means "font file which employs the OpenType font variations technology".
There is a Wikipedia page at: https://en.wikipedia.org/wiki/Variable_font
Ah, I see. Thank you.
Quoted 10 lines
Quoted 4 lines
"static ones" should be something like fixed-space fonts or monospaced fonts.
Unless I just completely don't understand the uses of "variable" and "static" here.
"static" here means "non-variable".
I'll expand the changelog a bit in v2 with your comments in mind.
Thanks, Akira
Signature · 1 line
#Randy
Jonathan Corbet
maintainerQuoted 8 lines · Akira Yokosawa
xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.
To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.
[Back after three weeks far from home...]
The problem with this is: removing those fonts breaks other things. I ended up putting them back onto my system after, as I recall, Emacs stopped displaying non-ASCII text correctly (or at all). So we may be giving advice that some users come to regret having followed.
As a result, I'm really not sure what the best thing to do here is.
jon
On Fri, 29 Mar 2024 08:35:54 -0600, Jonathan Corbet wrote:
...
Quoted 4 lines
The problem with this is: removing those fonts breaks other things. I ended up putting them back onto my system after, as I recall, Emacs stopped displaying non-ASCII text correctly (or at all).
That's something I was worrying about.
Quoted 4 lines
So we may be
giving advice that some users come to regret having followed.As a result, I'm really not sure what the best thing to do here is.
I've been experimenting a per-user fontconfig trick for denylisting those "variable fonts" for "make pdfdocs" only. Looks like there is hoping.
Let me submit a v2 with that trick listed as the safest option in a week or so.
Thanks, Akira
Quoted 2 lines
jon
PATCH v2 1/1 docs: Detect variable fonts and suggest denylisting them
In reply to Akira YokosawaFedora and openSUSE has started deploying "variable font" [1] format Noto CJK fonts [2, 3]. "CJK" here stands for "Chinese, Japanese, and Korean".
Unfortunately, XeTeX/XeLaTeX doesn't understand those fonts for historical reasons and builds of translations.pdf end up in errors if such fonts are present on the build host.
To help developers work around the issue, add a script to check the presence of "variable font" Noto CJK fonts and to emit suggestions. The script is invoked in the error path of "make pdfdocs" so that the suggestions are made only when a PDF build actually fails.
The first suggestion is to denylist those "variable font" files by activating a per-user and command-local fontconfig setting.
For further info and backgrounds, please refer to the header comment of scripts/check-variable-font.sh newly added in this commit.
---
Changes in v2:
- Stop suggesting removal of variable-font packages. (Jon)
- Rewrite changelog and add a couple of links for reference. (Randy)
- Suggest denylisting "variable font" files for XeLaTeX in "make pdfdocs" as a less invasive option.
- Simplify message from check-variable-font.sh and expand the header comments of the script.
- Add template of fonts.conf for denylisting in the header comments.
- Add rules for activating the XeLaTeX only fonts.conf in Documentation/Makefile.
v1: https://lore.kernel.org/r/[email protected]/
Jon, how does the suggestion of denylisting sound to you?
For the denylisting to work, $HOME/deny-vf/fontconfig/fonts.conf needs to be created manually.
It would be possible to automate this step, but fonts.conf might need adjustments in case "variable font" files to be denied reside somewhere else.
Thanks, Akira
--- Documentation/Makefile | 7 +- Documentation/sphinx/kerneldoc-preamble.sty | 9 +- MAINTAINERS | 1 + scripts/check-variable-fonts.sh | 117 ++++++++++++++++++++ 4 files changed, 129 insertions(+), 5 deletions(-) create mode 100755 scripts/check-variable-fonts.sh
Documentation/Makefile +6 -1
@@ -28,6 +28,10 @@ BUILDDIR = $(obj)/output PDFLATEX = xelatex LATEXOPTS = -interaction=batchmode -no-shell-escape +# For denylisting "variable font" files +# Can be overridden by setting as an env variable +FONTS_CONF_DENY_VF ?= $(HOME)/deny-vf + ifeq ($(findstring 1, $(KBUILD_VERBOSE)),) SPHINXOPTS += "-q" endif
@@ -151,10 +155,11 @@ pdfdocs: else # HAVE_PDFLATEX +pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF) pdfdocs: latexdocs @$(srctree)/scripts/sphinx-pre-install --version-check $(foreach var,$(SPHINXDIRS), \ - $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \ + $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \ mkdir -p $(BUILDDIR)/$(var)/pdf; \ mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \ )
Documentation/sphinx/kerneldoc-preamble.sty +5 -4
@@ -215,11 +215,12 @@ due to the lack of suitable font families and/or the texlive-xecjk package. - If you want them, please install ``Noto Sans CJK'' font families - along with the texlive-xecjk package by following instructions from + If you want them, please install non-variable ``Noto Sans CJK'' + font families along with the texlive-xecjk package by following + instructions from \sphinxcode{./scripts/sphinx-pre-install}. - Having optional ``Noto Serif CJK'' font families will improve - the looks of those translations. + Having optional non-variable ``Noto Serif CJK'' font families will + improve the looks of those translations. \end{sphinxadmonition}} \newcommand{\kerneldocEndSC}{} \newcommand{\kerneldocBeginTC}[1]{}
MAINTAINERS +1 -0
@@ -6406,6 +6406,7 @@ S: Maintained P: Documentation/doc-guide/maintainer-profile.rst T: git git://git.lwn.net/linux.git docs-next F: Documentation/ +F: scripts/check-variable-font.sh F: scripts/documentation-file-ref-check F: scripts/kernel-doc F: scripts/sphinx-pre-install
scripts/check-variable-fonts.sh new file +117 -0
117 lines not on this page · raw message
base-commit: 9e192b39a5992d8b730383d57416964b44ea1041
Signature · 1 line
2.34.1
Jonathan Corbet
maintainerRe: [PATCH v2] docs: Detect variable fonts and suggest denylisting them
Quoted 31 lines · Akira Yokosawa
Fedora and openSUSE has started deploying "variable font" [1] format Noto CJK fonts [2, 3]. "CJK" here stands for "Chinese, Japanese, and Korean".
Unfortunately, XeTeX/XeLaTeX doesn't understand those fonts for historical reasons and builds of translations.pdf end up in errors if such fonts are present on the build host.
To help developers work around the issue, add a script to check the presence of "variable font" Noto CJK fonts and to emit suggestions. The script is invoked in the error path of "make pdfdocs" so that the suggestions are made only when a PDF build actually fails.
The first suggestion is to denylist those "variable font" files by activating a per-user and command-local fontconfig setting.
For further info and backgrounds, please refer to the header comment of
scripts/check-variable-font.shnewly added in this commit.Reported-by Jonathan Corbet <[email protected]>Reported-by "Иван Иванович" <[email protected]>Cc Randy Dunlap <[email protected]>Signed-off-by Akira Yokosawa <[email protected]>---
Changes in v2:
Quoted 9 lines
- Stop suggesting removal of variable-font packages. (Jon)
- Rewrite changelog and add a couple of links for reference. (Randy)
- Suggest denylisting "variable font" files for XeLaTeX in "make pdfdocs" as a less invasive option.
- Simplify message fromcheck-variable-font.shand expand the header comments of the script.
- Add template of fonts.conf for denylisting in the header comments.
- Add rules for activating the XeLaTeX only fonts.conf inDocumentation/Makefile.
Thanks for working on this.
I've verified that the message comes up at the right time (though before waiting a half-hour for things to fail would be better :) and that following the advice in the script makes the build work. I *did* have to do a "make cleandocs" after adding the fonts.conf file, though, before the build would succeed.
So I've applied this, for now at least, but I do wonder: might it be better to stash this fonts.conf file in-tree somewhere and just pull it in automatically? Would that create problems for anybody if we were to do so? That might be a bit nicer than failing and making people set up the workaround on their own.
Meanwhile, it occurred to me that it would be good to let the Fedora folks know that this breaks, so I've filed a bug there; we'll see if they have any thoughts on the matter as well.
Thanks,
jon
Re: [PATCH v2] docs: Detect variable fonts and suggest denylisting them
On Wed, 10 Apr 2024 14:52:37 -0600, Jonathan Corbet wrote:
[...]
Quoted 3 lines
Meanwhile, it occurred to me that it would be good to let the Fedora folks know that this breaks, so I've filed a bug there; we'll see if they have any thoughts on the matter as well.
Actually, I opened:
https://bugzilla.redhat.com/show_bug.cgi?id=2271559
"google-noto-sans-cjk-vf-fonts is not compatible with XeTeX"
the other day as a bug in google-noto-sans-cjk-vf-fonts.
In response, Peng Wu (one of font package maintainers) opened:
https://bugzilla.redhat.com/show_bug.cgi?id=2272153
"xelatex doesn't support font face from named instance of variable fonts"
as a bug in texlive-base.
In #2271559, I was asked to help report this issue to upstream XeTeX, which is in my to-do list.
It sounds like there should be a way for XeTeX to identify variable fonts and ignore them.
Thanks, Akira
Jonathan Corbet
maintainerRe: [PATCH v2] docs: Detect variable fonts and suggest denylisting them
Quoted 6 lines · Akira Yokosawa
On Wed, 10 Apr 2024 14:52:37 -0600, Jonathan Corbet wrote:
[...]Quoted 3 lines
Meanwhile, it occurred to me that it would be good to let the Fedora folks know that this breaks, so I've filed a bug there; we'll see if they have any thoughts on the matter as well.
Quoted 6 lines
Actually, I opened:
https://bugzilla.redhat.com/show_bug.cgi?id=2271559
"google-noto-sans-cjk-vf-fonts is not compatible with XeTeX"the other day as a bug in google-noto-sans-cjk-vf-fonts.
Ah...I filed mine under texlive, no wonder I didn't find yours first :)
Quoted 6 lines
In response, Peng Wu (one of font package maintainers) opened:
https://bugzilla.redhat.com/show_bug.cgi?id=2272153
"xelatex doesn't support font face from named instance of variable fonts"as a bug in texlive-base.
I'm not sure why I didn't find that one, though...I did look.
Quoted 5 lines
In #2271559, I was asked to help report this issue to upstream XeTeX, which is in my to-do list.
It sounds like there should be a way for XeTeX to identify variable fonts and ignore them.
It seems like it should certainly be possible. We may end up carrying the workaround for a long time, though, before any fix filters through to users.
Thanks,
jon