Skip to contentSkip to conversation

Convert *.log files into UTF-8 encoding

PATCH Applied Akira Yokosawa 2 participants 2 messages Jun 13, 2025

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Convert *.log files into UTF-8 encoding

pdflatex outputs .log files in iso-8859-1 encoding. In our build scripts, grep assumes UTF-8 encoding for its input by default.

Practically speaking, the mismatch of encoding is unlikely to cause any harm. That said, in the build log, there is a suspicious message from grep:

    ...
    pdflatex 1 for perfbook.pdf
    grep: perfbook.log: binary file matches     <===
    ...

Converting .log files into UTF-8 on-the-fly can avoid the message.

Let's do it.

---
Note 1:
Unicode ready modern engines such as XeTeX and LuaTex emit .log files in UTF-8 encoding.

Note 2:
Currently, perfbook's preamble is not compatible with lualatex or xelatex.

Note 3:
iconv should be available by default.
encguess should also be available where perl is installed.
--

 utilities/runfirstlatex.sh | 10 ++++++++++
 utilities/runlatex.sh      | 10 ++++++++++
 2 files changed, 20 insertions(+)
utilities/runfirstlatex.sh +10 -0
index a89e8a70..676502b2 100644
@@ -72,6 +72,16 @@ basename=`echo $1 | sed -e 's/\.tex$//'`
 echo "$LATEX 1 for $basename.pdf"
 $LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null
 exitcode=$?
+ENCGUESS_CMD=`command -v encguess`
+if [ "x$ENCGUESS_CMD" != "x" ]
+then
+	if encguess -s iso-8859-1 $basename.log | grep -q ISO-8859-1
+	then
+		mv $basename.log $basename-tmp.log
+		iconv -f ISO-8859-1 -t UTF-8 $basename-tmp.log > $basename.log
+		rm $basename-tmp.log
+	fi
+fi
 if grep -q 'LaTeX Warning: You have requested' $basename.log
 then
 	grep -A 4 'LaTeX Warning: You have requested' $basename.log
utilities/runlatex.sh +10 -0
index 358b6baf..17d833ad 100644
@@ -79,6 +79,16 @@ iterate_latex () {
 	makeglossaries $basename > /dev/null 2>&1
 	$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null
 	exitcode=$?
+	ENCGUESS_CMD=`command -v encguess`
+	if [ "x$ENCGUESS_CMD" != "x" ]
+	then
+		if encguess -s iso-8859-1 $basename.log | grep -q ISO-8859-1
+		then
+			mv $basename.log $basename-tmp.log
+			iconv -f ISO-8859-1 -t UTF-8 $basename-tmp.log > $basename.log
+			rm $basename-tmp.log
+		fi
+	fi
 	if grep -q '! Emergency stop.' $basename.log
 	then
 		grep -B 10 -A 5 '! Emergency stop.' $basename.log

base-commit: a1bf2e61230301bfc12eba19abfc1c5a880f53be

Signature · 1 line
2.43.0
#1 lore raw
Quoted 12 lines · Akira Yokosawa

pdflatex outputs .log files in iso-8859-1 encoding. In our build scripts, grep assumes UTF-8 encoding for its input by default.

Practically speaking, the mismatch of encoding is unlikely to cause any harm. That said, in the build log, there is a suspicious message from grep:

    ...
    pdflatex 1 for perfbook.pdf
    grep: perfbook.log: binary file matches     <===
    ...
Quoted 5 lines

Converting .log files into UTF-8 on-the-fly can avoid the message.

Let's do it.

Queued and pushed, thank you!

Thanx, Paul

Quoted 64 lines

---
Note 1:
Unicode ready modern engines such as XeTeX and LuaTex emit .log files in UTF-8 encoding.

Note 2:
Currently, perfbook's preamble is not compatible with lualatex or xelatex.

Note 3:
iconv should be available by default.
encguess should also be available where perl is installed.
--

 utilities/runfirstlatex.sh | 10 ++++++++++
 utilities/runlatex.sh      | 10 ++++++++++
 2 files changed, 20 insertions(+)
utilities/runfirstlatex.sh +10 -0
index a89e8a70..676502b2 100644
@@ -72,6 +72,16 @@ basename=`echo $1 | sed -e 's/\.tex$//'`
 echo "$LATEX 1 for $basename.pdf"
 $LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null
 exitcode=$?
+ENCGUESS_CMD=`command -v encguess`
+if [ "x$ENCGUESS_CMD" != "x" ]
+then
+	if encguess -s iso-8859-1 $basename.log | grep -q ISO-8859-1
+	then
+		mv $basename.log $basename-tmp.log
+		iconv -f ISO-8859-1 -t UTF-8 $basename-tmp.log > $basename.log
+		rm $basename-tmp.log
+	fi
+fi
 if grep -q 'LaTeX Warning: You have requested' $basename.log
 then
 	grep -A 4 'LaTeX Warning: You have requested' $basename.log
utilities/runlatex.sh +10 -0
index 358b6baf..17d833ad 100644
@@ -79,6 +79,16 @@ iterate_latex () {
 	makeglossaries $basename > /dev/null 2>&1
 	$LATEX $LATEX_OPT $basename > /dev/null 2>&1 < /dev/null
 	exitcode=$?
+	ENCGUESS_CMD=`command -v encguess`
+	if [ "x$ENCGUESS_CMD" != "x" ]
+	then
+		if encguess -s iso-8859-1 $basename.log | grep -q ISO-8859-1
+		then
+			mv $basename.log $basename-tmp.log
+			iconv -f ISO-8859-1 -t UTF-8 $basename-tmp.log > $basename.log
+			rm $basename-tmp.log
+		fi
+	fi
 	if grep -q '! Emergency stop.' $basename.log
 	then
 		grep -B 10 -A 5 '! Emergency stop.' $basename.log

base-commit: a1bf2e61230301bfc12eba19abfc1c5a880f53be

Signature · 1 line
2.43.0