Skip to contentSkip to conversation

docs: Detect variable fonts and suggest removing them

PATCH Superseded Akira Yokosawa 3 participants 10 messages Mar 23, 2024 – Apr 10, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 docs: Detect variable fonts and suggest removing them

xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.

To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.

---
This is not in line with Jon's suggestion of tweaking sphinx-pre-install and/or kerneldoc-preamble.sty [1].

[1]: https://lore.kernel.org/r/[email protected]/

It would be ideal if kerneldoc-preamble.sty can be taught about the way to detect the existence of variable fonts and to skip CJK contents. However, it seems that it would add overly complex code for choosing fonts by file names with meticulous options to select CJK variant (SC, TC, KR, or JP) to use. Furthermore, as location and names of font files can be different among distro packages, keeping portability of the preamble is impossible in practice.

As for sphinx-pre-install, even if variable "Noto CJK" fonts are found, they don't matter for non-CJK PDFs. Emitting warning every time would be noisy.

Therefore, as a minimal approach, to help developers find a way forward, giving suggestions after the fact sounds like a reasonable approach to me, at least for the time being.

Jon, you said in [1]:

Quoted 8 lines
Quoted 5 lines

What does

fc-list | grep NotoSansCJK-VF.ttc

say?

No output at all, even though I had google-noto-sans-cjk-vf-fonts installed.

So I'm wondering if scripts/check-variable-font.sh in this patch emits useful suggestions. Could you give it a try?

- Install google-noto-sans-cjk-vf-fonts
- Install google-noto-serif-cjk-vf-fonts
- What does "sh scripts/check-variable-font.sh" say?
- Run "make cleandocs; make SPHINXDIRS=translations pdfdocs". Do you see the suggestion after the build error?
- Uninstall above 2 font packages.
- What does "sh scripts/check-variable-font.sh" say now?
- Run "make cleandocs; make SPHINXDIRS=translations pdfdocs" again. Does it complete successfully?

Thanks, Akira
---
Cc: [email protected]

---
 Documentation/Makefile          |  2 +-
 MAINTAINERS                     |  1 +
 scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100755 scripts/check-variable-fonts.sh
Documentation/Makefile +1 -1
index b68f8c816897..e7ff288bfd15 100644
@@ -154,7 +154,7 @@ else # HAVE_PDFLATEX
 pdfdocs: latexdocs
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	$(foreach var,$(SPHINXDIRS), \
-	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \
+	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \
 	   mkdir -p $(BUILDDIR)/$(var)/pdf; \
 	   mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \
 	)
MAINTAINERS +1 -0
index 741d9142b343..3858416a2d67 100644
@@ -6346,6 +6346,7 @@ S:	Maintained
 P:	Documentation/doc-guide/maintainer-profile.rst
 T:	git git://git.lwn.net/linux.git docs-next
 F:	Documentation/
+F:	scripts/check-variable-font.sh
 F:	scripts/documentation-file-ref-check
 F:	scripts/kernel-doc
 F:	scripts/sphinx-pre-install
scripts/check-variable-fonts.sh new file +30 -0
new file mode 100755
index 000000000000..775800edb9fc
@@ -0,0 +1,30 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# For "make pdfdocs", recent trend of deploying variable type of
+# "Noto Sans CJK" and "Noto Serif CJK" fonts breaks xelatex, which does
+# not understand variable fonts.
+#
+# It is hard to distinguish variable fonts from static ones in the preamble
+# of LaTeX source code.  Instead, this script is invoked in the error path
+# of "make pdfdocs" and emit suggestions if such font files are found.
+#
+# Assumption:
+# File names are not changed from those of upstream Noto CJK fonts:
+#     https://github.com/notofonts/noto-cjk/
+
+vffonts=`fc-list -b | grep -i noto | grep -i cjk | grep -F -i -e "-vf" | \
+	 sort | uniq | sed -e 's/\tfile:/  file:/' | sed -e 's/(s)$//'`
+
+if [ "x$vffonts" != "x" ] ; then
+	echo "====================================================================="
+	echo "Detected variable form of Noto CJK fonts incompatible with xelatex:"
+	echo "$vffonts"
+	echo "If you need CJK contents in PDF, remove them and install static ones."
+	echo "Otherwise, get rid of texlive-xecjk."
+	echo "====================================================================="
+fi
+
+# As this script is invoked from Makefile's error path, always error exit
+# even if no variable font is detected.
+exit 1

base-commit: b8cfda5c9065cd619a97c17da081cbfab3b1e756

Signature · 1 line
2.34.1
reviewer
#1 lore raw

Hi Akira,

Quoted 15 lines · Akira Yokosawa

xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.

To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.

---

[snip]

Quoted 9 lines

---
Cc: [email protected]

---
 Documentation/Makefile          |  2 +-
 MAINTAINERS                     |  1 +
 scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100755 scripts/check-variable-fonts.sh

I don't have much to say about the patch except that it seems like a good idea...

However, some of the terminology could be improved IMO. E.g., "variable type" or "variable font" should be something like variable-width font or proportionally-spaced font.

"static ones" should be something like fixed-space fonts or monospaced fonts.

Unless I just completely don't understand the uses of "variable" and "static" here.

Quoted 37 lines
scripts/check-variable-fonts.sh new file +30 -0
new file mode 100755
index 000000000000..775800edb9fc
@@ -0,0 +1,30 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# For "make pdfdocs", recent trend of deploying variable type of
+# "Noto Sans CJK" and "Noto Serif CJK" fonts breaks xelatex, which does
+# not understand variable fonts.
+#
+# It is hard to distinguish variable fonts from static ones in the preamble
+# of LaTeX source code.  Instead, this script is invoked in the error path
+# of "make pdfdocs" and emit suggestions if such font files are found.
+#
+# Assumption:
+# File names are not changed from those of upstream Noto CJK fonts:
+#     https://github.com/notofonts/noto-cjk/
+
+vffonts=`fc-list -b | grep -i noto | grep -i cjk | grep -F -i -e "-vf" | \
+	 sort | uniq | sed -e 's/\tfile:/  file:/' | sed -e 's/(s)$//'`
+
+if [ "x$vffonts" != "x" ] ; then
+	echo "====================================================================="
+	echo "Detected variable form of Noto CJK fonts incompatible with xelatex:"
+	echo "$vffonts"
+	echo "If you need CJK contents in PDF, remove them and install static ones."
+	echo "Otherwise, get rid of texlive-xecjk."
+	echo "====================================================================="
+fi
+
+# As this script is invoked from Makefile's error path, always error exit
+# even if no variable font is detected.
+exit 1
Quoted 1 line

base-commit: b8cfda5c9065cd619a97c17da081cbfab3b1e756

Thanks.

Signature · 1 line
#Randy
#2 lore raw
Quoted 1 line · Randy Dunlap

Hi Akira,

Thank you for your comments, Randy.

Quoted 30 lines · Akira Yokosawa
Quoted 15 lines

xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.

To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.

---

[snip]

Quoted 9 lines

---
Cc: [email protected]

---
 Documentation/Makefile          |  2 +-
 MAINTAINERS                     |  1 +
 scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100755 scripts/check-variable-fonts.sh
Quoted 5 lines

I don't have much to say about the patch except that it seems like a good idea...

However, some of the terminology could be improved IMO. E.g., "variable type" or "variable font" should be something like variable-width font or proportionally-spaced font.

Good point.

I need to be more careful in using terms rarely found in kernel development.

In this case, "variable font" means "font file which employs the OpenType font variations technology".

There is a Wikipedia page at: https://en.wikipedia.org/wiki/Variable_font

Quoted 4 lines

"static ones" should be something like fixed-space fonts or monospaced fonts.

Unless I just completely don't understand the uses of "variable" and "static" here.

"static" here means "non-variable".

I'll expand the changelog a bit in v2 with your comments in mind.

Thanks, Akira

reviewer
#3 lore raw
Quoted 17 lines
Quoted 11 lines
Quoted 10 lines

---
Cc: [email protected]

---
 Documentation/Makefile          |  2 +-
 MAINTAINERS                     |  1 +
 scripts/check-variable-fonts.sh | 30 ++++++++++++++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100755 scripts/check-variable-fonts.sh
Quoted 5 lines

I don't have much to say about the patch except that it seems like a good idea...

However, some of the terminology could be improved IMO. E.g., "variable type" or "variable font" should be something like variable-width font or proportionally-spaced font.

Quoted 9 lines

Good point.

I need to be more careful in using terms rarely found in kernel development.

In this case, "variable font" means "font file which employs the OpenType font variations technology".

There is a Wikipedia page at: https://en.wikipedia.org/wiki/Variable_font

Ah, I see. Thank you.

Quoted 10 lines
Quoted 4 lines

"static ones" should be something like fixed-space fonts or monospaced fonts.

Unless I just completely don't understand the uses of "variable" and "static" here.

"static" here means "non-variable".

I'll expand the changelog a bit in v2 with your comments in mind.

Thanks, Akira

Signature · 1 line
#Randy
maintainer
#4 lore raw
In reply to Akira Yokosawa
Quoted 8 lines · Akira Yokosawa

xelatex doesn't understand variable font format. Recent deployment of variable Noto CJK fonts in Fedora and openSUSE tumbleweed breaks builds of translations.pdf.

To help developers work around the build error, add a script for checking existence of variable form of those fonts and emitting suggestions. Invoke it in the error path of "make pdfdocs" so that it is activated only when PDF build actually fails.

[Back after three weeks far from home...]

The problem with this is: removing those fonts breaks other things. I ended up putting them back onto my system after, as I recall, Emacs stopped displaying non-ASCII text correctly (or at all). So we may be giving advice that some users come to regret having followed.

As a result, I'm really not sure what the best thing to do here is.

jon

#5 lore raw

On Fri, 29 Mar 2024 08:35:54 -0600, Jonathan Corbet wrote:
...

Quoted 4 lines

The problem with this is: removing those fonts breaks other things. I ended up putting them back onto my system after, as I recall, Emacs stopped displaying non-ASCII text correctly (or at all).

That's something I was worrying about.

Quoted 4 lines

So we may be
giving advice that some users come to regret having followed.

As a result, I'm really not sure what the best thing to do here is.

I've been experimenting a per-user fontconfig trick for denylisting those "variable fonts" for "make pdfdocs" only. Looks like there is hoping.

Let me submit a v2 with that trick listed as the safest option in a week or so.

Thanks, Akira

Quoted 2 lines

jon

#6 lore raw

PATCH v2 1/1 docs: Detect variable fonts and suggest denylisting them

In reply to Akira Yokosawa

Fedora and openSUSE has started deploying "variable font" [1] format Noto CJK fonts [2, 3]. "CJK" here stands for "Chinese, Japanese, and Korean".

Unfortunately, XeTeX/XeLaTeX doesn't understand those fonts for historical reasons and builds of translations.pdf end up in errors if such fonts are present on the build host.

To help developers work around the issue, add a script to check the presence of "variable font" Noto CJK fonts and to emit suggestions. The script is invoked in the error path of "make pdfdocs" so that the suggestions are made only when a PDF build actually fails.

The first suggestion is to denylist those "variable font" files by activating a per-user and command-local fontconfig setting.

For further info and backgrounds, please refer to the header comment of scripts/check-variable-font.sh newly added in this commit.

---
Changes in v2:

- Stop suggesting removal of variable-font packages. (Jon)
- Rewrite changelog and add a couple of links for reference. (Randy)
- Suggest denylisting "variable font" files for XeLaTeX in "make pdfdocs" as a less invasive option.
- Simplify message from check-variable-font.sh and expand the header comments of the script.
- Add template of fonts.conf for denylisting in the header comments.
- Add rules for activating the XeLaTeX only fonts.conf in Documentation/Makefile.

v1: https://lore.kernel.org/r/[email protected]/

Jon, how does the suggestion of denylisting sound to you?

For the denylisting to work, $HOME/deny-vf/fontconfig/fonts.conf needs to be created manually.

It would be possible to automate this step, but fonts.conf might need adjustments in case "variable font" files to be denied reside somewhere else.

Thanks, Akira

---
 Documentation/Makefile                      |   7 +-
 Documentation/sphinx/kerneldoc-preamble.sty |   9 +-
 MAINTAINERS                                 |   1 +
 scripts/check-variable-fonts.sh             | 117 ++++++++++++++++++++
 4 files changed, 129 insertions(+), 5 deletions(-)
 create mode 100755 scripts/check-variable-fonts.sh
Documentation/Makefile +6 -1
index b68f8c816897..a961692baa12 100644
@@ -28,6 +28,10 @@ BUILDDIR      = $(obj)/output
 PDFLATEX      = xelatex
 LATEXOPTS     = -interaction=batchmode -no-shell-escape
 
+# For denylisting "variable font" files
+# Can be overridden by setting as an env variable
+FONTS_CONF_DENY_VF ?= $(HOME)/deny-vf
+
 ifeq ($(findstring 1, $(KBUILD_VERBOSE)),)
 SPHINXOPTS    += "-q"
 endif
@@ -151,10 +155,11 @@ pdfdocs:
 
 else # HAVE_PDFLATEX
 
+pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
 pdfdocs: latexdocs
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	$(foreach var,$(SPHINXDIRS), \
-	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \
+	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \
 	   mkdir -p $(BUILDDIR)/$(var)/pdf; \
 	   mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \
 	)
Documentation/sphinx/kerneldoc-preamble.sty +5 -4
index 3092df051c95..d479cfa73658 100644
@@ -215,11 +215,12 @@
 	    due to the lack of suitable font families and/or the texlive-xecjk
 	    package.
 
-	    If you want them, please install ``Noto Sans CJK'' font families
-	    along with the texlive-xecjk package by following instructions from
+	    If you want them, please install non-variable ``Noto Sans CJK''
+	    font families along with the texlive-xecjk package by following
+	    instructions from
 	    \sphinxcode{./scripts/sphinx-pre-install}.
-	    Having optional ``Noto Serif CJK'' font families will improve
-	    the looks of those translations.
+	    Having optional non-variable ``Noto Serif CJK'' font families will
+	    improve the looks of those translations.
 	\end{sphinxadmonition}}
     \newcommand{\kerneldocEndSC}{}
     \newcommand{\kerneldocBeginTC}[1]{}
MAINTAINERS +1 -0
index aa3b947fb080..3a4768c2f712 100644
@@ -6406,6 +6406,7 @@ S:	Maintained
 P:	Documentation/doc-guide/maintainer-profile.rst
 T:	git git://git.lwn.net/linux.git docs-next
 F:	Documentation/
+F:	scripts/check-variable-font.sh
 F:	scripts/documentation-file-ref-check
 F:	scripts/kernel-doc
 F:	scripts/sphinx-pre-install
scripts/check-variable-fonts.sh new file +117 -0

117 lines not on this page · raw message

base-commit: 9e192b39a5992d8b730383d57416964b44ea1041

Signature · 1 line
2.34.1
maintainer
#7 lore raw

Re: [PATCH v2] docs: Detect variable fonts and suggest denylisting them

Quoted 31 lines · Akira Yokosawa

Fedora and openSUSE has started deploying "variable font" [1] format Noto CJK fonts [2, 3]. "CJK" here stands for "Chinese, Japanese, and Korean".

Unfortunately, XeTeX/XeLaTeX doesn't understand those fonts for historical reasons and builds of translations.pdf end up in errors if such fonts are present on the build host.

To help developers work around the issue, add a script to check the presence of "variable font" Noto CJK fonts and to emit suggestions. The script is invoked in the error path of "make pdfdocs" so that the suggestions are made only when a PDF build actually fails.

The first suggestion is to denylist those "variable font" files by activating a per-user and command-local fontconfig setting.

For further info and backgrounds, please refer to the header comment of scripts/check-variable-font.sh newly added in this commit.

---
Changes in v2:

Quoted 9 lines

- Stop suggesting removal of variable-font packages. (Jon)
- Rewrite changelog and add a couple of links for reference. (Randy)
- Suggest denylisting "variable font" files for XeLaTeX in "make pdfdocs" as a less invasive option.
- Simplify message from check-variable-font.sh and expand the header comments of the script.
- Add template of fonts.conf for denylisting in the header comments.
- Add rules for activating the XeLaTeX only fonts.conf in Documentation/Makefile.

Thanks for working on this.

I've verified that the message comes up at the right time (though before waiting a half-hour for things to fail would be better :) and that following the advice in the script makes the build work. I *did* have to do a "make cleandocs" after adding the fonts.conf file, though, before the build would succeed.

So I've applied this, for now at least, but I do wonder: might it be better to stash this fonts.conf file in-tree somewhere and just pull it in automatically? Would that create problems for anybody if we were to do so? That might be a bit nicer than failing and making people set up the workaround on their own.

Meanwhile, it occurred to me that it would be good to let the Fedora folks know that this breaks, so I've filed a bug there; we'll see if they have any thoughts on the matter as well.

Thanks,

jon

#8 lore raw

Re: [PATCH v2] docs: Detect variable fonts and suggest denylisting them

On Wed, 10 Apr 2024 14:52:37 -0600, Jonathan Corbet wrote:
[...]

Quoted 3 lines

Meanwhile, it occurred to me that it would be good to let the Fedora folks know that this breaks, so I've filed a bug there; we'll see if they have any thoughts on the matter as well.

Actually, I opened:

https://bugzilla.redhat.com/show_bug.cgi?id=2271559
"google-noto-sans-cjk-vf-fonts is not compatible with XeTeX"

the other day as a bug in google-noto-sans-cjk-vf-fonts.

In response, Peng Wu (one of font package maintainers) opened:

https://bugzilla.redhat.com/show_bug.cgi?id=2272153
"xelatex doesn't support font face from named instance of variable fonts"

as a bug in texlive-base.

In #2271559, I was asked to help report this issue to upstream XeTeX, which is in my to-do list.

It sounds like there should be a way for XeTeX to identify variable fonts and ignore them.

Thanks, Akira

maintainer
#9 lore raw

Re: [PATCH v2] docs: Detect variable fonts and suggest denylisting them

Quoted 6 lines · Akira Yokosawa

On Wed, 10 Apr 2024 14:52:37 -0600, Jonathan Corbet wrote:
[...]

Quoted 3 lines

Meanwhile, it occurred to me that it would be good to let the Fedora folks know that this breaks, so I've filed a bug there; we'll see if they have any thoughts on the matter as well.

Quoted 6 lines

Actually, I opened:

https://bugzilla.redhat.com/show_bug.cgi?id=2271559
"google-noto-sans-cjk-vf-fonts is not compatible with XeTeX"

the other day as a bug in google-noto-sans-cjk-vf-fonts.

Ah...I filed mine under texlive, no wonder I didn't find yours first :)

Quoted 6 lines

In response, Peng Wu (one of font package maintainers) opened:

https://bugzilla.redhat.com/show_bug.cgi?id=2272153
"xelatex doesn't support font face from named instance of variable fonts"

as a bug in texlive-base.

I'm not sure why I didn't find that one, though...I did look.

Quoted 5 lines

In #2271559, I was asked to help report this issue to upstream XeTeX, which is in my to-do list.

It sounds like there should be a way for XeTeX to identify variable fonts and ignore them.

It seems like it should certainly be possible. We may end up carrying the workaround for a long time, though, before any fix filters through to users.

Thanks,

jon