qqz and runlatex improvements

Latex Warning: There were undefined references Latex Warning: There were multiply-defined labels.

Quoted 1 line

From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001

From: Akira Yokosawa <[email protected]>
Date: Thu, 21 Apr 2016 22:11:06 +0900
Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition

After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"), 'make' becomes to terminate prematurely with warning messages such as

Quoted 2 lines

Latex Warning: There were undefined references Latex Warning: There were multiply-defined labels.

This can be dealt with loosening loop condition in runlatex.sh which is given to 'grep' command.
Also, This commit increases the iteration max count to accommodate the required number of 'pdflatex' invocations caused by qqz cross-link.

Signed-off-by Akira Yokosawa <[email protected]>

---
 utilities/runlatex.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

utilities/runlatex.sh +0 -0

index 8d2d77a..15d96c1 100644

utilities/runlatex.sh +1 -1

 	fi
-	if test "$iter" -eq 4
+	if test "$iter" -eq 5
 	then
 		echo "Iteration limit: $iter passes through pdflatex"
 		exit 1

Signature · 1 line

1.9.1

Quoted 11 lines · Akira Yokosawa

Quoted 1 line

From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001

From: Akira Yokosawa <[email protected]>
Date: Thu, 21 Apr 2016 22:11:06 +0900
Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition

After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"), 'make' becomes to terminate prematurely with warning messages such as

Quoted 2 lines

Latex Warning: There were undefined references Latex Warning: There were multiply-defined labels.

Quoted 6 lines

This can be dealt with loosening loop condition in runlatex.sh which is given to 'grep' command.
Also, This commit increases the iteration max count to accommodate the required number of 'pdflatex' invocations caused by qqz cross-link.

Signed-off-by Akira Yokosawa <[email protected]>

I pulled this into a test branch, and it seemed to work for me, thank you! I am running on Ubuntu, any experience on other platforms?

Please see below for a couple of questions.

Thanx, Paul

Quoted 13 lines

---
 utilities/runlatex.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

utilities/runlatex.sh +0 -1

index 8d2d77a..15d96c1 100644
@@ -47,7 +47,7 @@ then
 		echo "No bibliography directory, skipping bibtex."
 	fi
 fi
-while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log

utilities/runlatex.sh then · line 53

Quoted 4 lines

utilities/runlatex.sh +1 -0

+while grep -q 'LaTeX Warning:' $basename.log
 do
 	iter=`expr $iter + 1`
 	echo "pdflatex $iter"

Given the loosening above, should we put a "grep LaTeX Warning:" here, so that users could interrupt the build if they felt that rerunning latex wasn't going to help?

utilities/runlatex.sh do · line 57

Quoted 3 lines

utilities/runlatex.sh +0 -0

@@ -56,7 +56,7 @@ do
 	then
 		echo "----- Fatal latex error, see $basename.log for details. -----"

Also given the loosening above, should there be an "exit 1" here?

Quoted 10 lines

utilities/runlatex.sh +1 -1

 	fi
-	if test "$iter" -eq 4
+	if test "$iter" -eq 5
 	then
 		echo "Iteration limit: $iter passes through pdflatex"
 		exit 1

Signature · 1 line

1.9.1

Hi Paul,

Quoted 39 lines · Paul E. McKenney

Quoted 11 lines · Akira Yokosawa

Quoted 1 line

From a5cb77713f4720592f75db71394fcad784ca480e Mon Sep 17 00:00:00 2001

From: Akira Yokosawa <[email protected]>
Date: Thu, 21 Apr 2016 22:11:06 +0900
Subject: [PATCH 2/2] runlatex.sh: Fix while loop condition

After commit 33b93f8258f5 ("qqz: Cross-link questions and answers"), 'make' becomes to terminate prematurely with warning messages such as

Quoted 2 lines

Latex Warning: There were undefined references Latex Warning: There were multiply-defined labels.

Quoted 6 lines

This can be dealt with loosening loop condition in runlatex.sh which is given to 'grep' command.
Also, This commit increases the iteration max count to accommodate the required number of 'pdflatex' invocations caused by qqz cross-link.

Signed-off-by Akira Yokosawa <[email protected]>

I pulled this into a test branch, and it seemed to work for me, thank you! I am running on Ubuntu, any experience on other platforms?

Please see below for a couple of questions.

Thanx, Paul

Quoted 13 lines

---
 utilities/runlatex.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

utilities/runlatex.sh +0 -1

index 8d2d77a..15d96c1 100644
@@ -47,7 +47,7 @@ then
 		echo "No bibliography directory, skipping bibtex."
 	fi
 fi
-while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log

Quoted 8 lines

utilities/runlatex.sh then · line 54

Quoted 4 lines

utilities/runlatex.sh +1 -0

+while grep -q 'LaTeX Warning:' $basename.log
 do
 	iter=`expr $iter + 1`
 	echo "pdflatex $iter"

Given the loosening above, should we put a "grep LaTeX Warning:" here, so that users could interrupt the build if they felt that rerunning latex wasn't going to help?

So, I loosened the condition too much. I tried another idea and it also seems to work. See below for the patch. (Note that it is the output of
git diff 514f21cd4156c68e17 utilities/runlatex.sh )
I have not enough time to do git commit & git format patch right now...

Quoted 5 lines

utilities/runlatex.sh do · line 57

Quoted 3 lines

utilities/runlatex.sh +0 -0

@@ -56,7 +56,7 @@ do
 	then
 		echo "----- Fatal latex error, see $basename.log for details. -----"

Also given the loosening above, should there be an "exit 1" here?

Yes, I think so. (This is added in the following patch)

Quoted 12 lines

Quoted 10 lines

utilities/runlatex.sh +1 -1

 	fi
-	if test "$iter" -eq 4
+	if test "$iter" -eq 5
 	then
 		echo "Iteration limit: $iter passes through pdflatex"
 		exit 1

Signature · 1 line

1.9.1

Can you try this one?
Thanks, Akira

---

utilities/runlatex.sh +4 -3

index 8d2d77a..eba7e16 100644
@@ -47,7 +47,8 @@ then
                echo "No bibliography directory, skipping bibtex."
        fi
 fi
-while grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
+while grep -q 'LaTeX Warning: There were undefined references' $basename.log || \
+    grep -q 'LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.' $basename.log
 do
        iter=`expr $iter + 1`
        echo "pdflatex $iter"

@@ -55,8 +56,9 @@ do
        if grep -q '! Emergency stop.' $basename.log
        then
                echo "----- Fatal latex error, see $basename.log for details. -----"
+               exit 1
        fi
-       if test "$iter" -eq 4
+       if test "$iter" -eq 5
        then
                echo "Iteration limit: $iter passes through pdflatex"
                exit 1
--

To unsubscribe from this list: send the line "unsubscribe perfbook" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 2 lines

Can you try this one?
Thanks, Akira

I did, though for some reason I had to hand-apply it on my test branch. Not sure what "git apply" didn't like.

It does repeat upon undefined, which is good. Nicer behavior after bibliography changes!

There is another "Fatal latex error" earlier that I missed, and it would be good to have an "exit 1" on that one as well.

The one thing I am still concerned about is a real undefined reference, which would consume a full five iterations. Not really sure what would be improved behavior, though. Thoughts?

Thanx, Paul

On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:
[snip]

Quoted 3 lines

Quoted 2 lines

Can you try this one?
Thanks, Akira

Quoted 2 lines

I did, though for some reason I had to hand-apply it on my test branch. Not sure what "git apply" didn't like.

Well, I'm afraid you tried to apply it after [PATCH 2/2] applied. I should have made it clear it was a diff from master at that moment. Sorry for your trouble.

Quoted 10 lines

It does repeat upon undefined, which is good. Nicer behavior after bibliography changes!

There is another "Fatal latex error" earlier that I missed, and it would be good to have an "exit 1" on that one as well.

The one thing I am still concerned about is a real undefined reference, which would consume a full five iterations. Not really sure what would be improved behavior, though. Thoughts?

Quoted 1 line

Thanx, Paul

So your question is why you need to change the maximum value from 4 to 5, isn't it? That's because the count is checked at the end of the loop, and even when the 4th round of pdflatex is successful, the check will fire and 'make' will regard it as an error. At such a later stage, remaining warning should only be 'LaTeX Warning: Label(s) may have changed', if any.

So I made a fairly invasive modification to the script so that the count is checked at the beginning of the loop. Also the loop is separated for each of the warning message.

Appended is the whole source code of the script (without the comment part at the head). I'm not sure which point should I make a diff from. Note that I truncated the warning messages given to grep. They should be long enough.

What do you think?

If it is OK, I'll resubmit the V2 of the patch series relative to the current master.

Thanks, Akira.

---
#!/bin/sh
# [snip]
#

if test -z "$1"
then
echo No latex file, aborting.
exit 1
fi

basename=echo $1 | sed -e 's/\.tex$//'

iter=1
echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----"

fi
if grep -q 'LaTeX Warning: There were undefined references' $basename.log then

	if test -d "$2"
	then
		bibtex $basename || :
	else
		echo "No bibliography directory, skipping bibtex."
	fi

fi
while grep -q 'LaTeX Warning: There were undefined references' $basename.log do
if test "$iter" -eq 4
then
echo "Iteration limit: $iter passes through pdflatex" exit 1
fi
iter=expr $iter + 1

echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----" exit 1
fi
done

while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log do
if test "$iter" -ge 4
then
echo "Iteration limit: $iter passes through pdflatex" exit 1
fi
iter=expr $iter + 1

echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----" exit 1
fi
done
grep "LaTeX Warning:" $basename.log
exit 0

Quoted 1 line · Akira Yokosawa

On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:

Quoted 11 lines

[snip]

Quoted 5 lines

Quoted 2 lines

Can you try this one?
Thanks, Akira

I did, though for some reason I had to hand-apply it on my test branch. Not sure what "git apply" didn't like.

Well, I'm afraid you tried to apply it after [PATCH 2/2] applied. I should have made it clear it was a diff from master at that moment. Sorry for your trouble.

I am pretty sure that I was applying it to the current master, but I cannot prove anything at this point. ;-)

Quoted 10 lines

Quoted 10 lines

It does repeat upon undefined, which is good. Nicer behavior after bibliography changes!

There is another "Fatal latex error" earlier that I missed, and it would be good to have an "exit 1" on that one as well.

The one thing I am still concerned about is a real undefined reference, which would consume a full five iterations. Not really sure what would be improved behavior, though. Thoughts?

Quoted 8 lines

Quoted 1 line

Thanx, Paul

So your question is why you need to change the maximum value from 4 to 5, isn't it? That's because the count is checked at the end of the loop, and even when the 4th round of pdflatex is successful, the check will fire and 'make' will regard it as an error. At such a later stage, remaining warning should only be 'LaTeX Warning: Label(s) may have changed', if any.

Yes, I do understand that. Your (very welcome!) changes to the quick-quiz links can require additional passes through latex. Therefore, the script should change to provide this.

To see what I am getting at, please add a "\ref{this is a typo}" or a reference to any other non-\label string, then type "make". This is a pure error that no number of reruns will fix, but the build script will nevertheless crank away multiple times.

Before, it would run bibtex and stop. (Which, I freely admit, wasn't particularly good behavior either!)

My question is whether it is possible to make the script let the user know why it is re-running latex, so that the user can choose to kill it early, if appropriate. Or, alternatively (and for extra credit!), recognize which labels are undefined, and do more passes if they are quick-quiz links.

Quoted 7 lines

So I made a fairly invasive modification to the script so that the count is checked at the beginning of the loop. Also the loop is separated for each of the warning message.

Appended is the whole source code of the script (without the comment part at the head). I'm not sure which point should I make a diff from. Note that I truncated the warning messages given to grep. They should be long enough.

Good idea!

Quoted 1 line

What do you think?

Please see below.

Thanx, Paul

Quoted 16 lines

If it is OK, I'll resubmit the V2 of the patch series relative to the current master.

Thanks, Akira.

---
#!/bin/sh
# [snip]
#

if test -z "$1"
then
echo No latex file, aborting.
exit 1
fi

Quoted 8 lines

basename=echo $1 | sed -e 's/\.tex$//'

iter=1
echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----"

Please add "exit 1" here. (Yes, this was a bug in my orignal. But as long as we are here...)

Quoted 9 lines

fi
if grep -q 'LaTeX Warning: There were undefined references' $basename.log then

	if test -d "$2"
	then
		bibtex $basename || :
	else
		echo "No bibliography directory, skipping bibtex."
	fi

Quoted 9 lines

fi
while grep -q 'LaTeX Warning: There were undefined references' $basename.log do
if test "$iter" -eq 4
then
echo "Iteration limit: $iter passes through pdflatex" exit 1
fi
iter=expr $iter + 1

This would be one point to add some indication of why we are doing another pass through the loop. Might be as simple as adding "Undefined references" to the "echo" below. Or maybe even show the last symbol listed as undefined.

Quoted 8 lines

echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----" exit 1
fi
done

Quoted 8 lines

while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log do
if test "$iter" -ge 4
then
echo "Iteration limit: $iter passes through pdflatex" exit 1
fi
iter=expr $iter + 1

This would be one point to add some indication of why we are doing another pass through the loop. Might be as simple as adding "Label(s) may have changed" to the "echo" below.

Quoted 11 lines

echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----" exit 1
fi
done
grep "LaTeX Warning:" $basename.log
exit 0

Hi,

Quoted 2 lines · Paul E. McKenney

Quoted 1 line · Akira Yokosawa

On 2016/04/22 16:54 -0700, Paul E. McKenney wrote:

Oh, I miss-converted the date. This should have been "On 2016/04/21 16:54 -0700."

Quoted 43 lines

Quoted 11 lines

[snip]

Quoted 5 lines

Quoted 2 lines

Can you try this one?
Thanks, Akira

I did, though for some reason I had to hand-apply it on my test branch. Not sure what "git apply" didn't like.

Well, I'm afraid you tried to apply it after [PATCH 2/2] applied. I should have made it clear it was a diff from master at that moment. Sorry for your trouble.

I am pretty sure that I was applying it to the current master, but I cannot prove anything at this point. ;-)

Quoted 10 lines

Quoted 10 lines

It does repeat upon undefined, which is good. Nicer behavior after bibliography changes!

There is another "Fatal latex error" earlier that I missed, and it would be good to have an "exit 1" on that one as well.

The one thing I am still concerned about is a real undefined reference, which would consume a full five iterations. Not really sure what would be improved behavior, though. Thoughts?

Quoted 8 lines

Quoted 1 line

Thanx, Paul

So your question is why you need to change the maximum value from 4 to 5, isn't it? That's because the count is checked at the end of the loop, and even when the 4th round of pdflatex is successful, the check will fire and 'make' will regard it as an error. At such a later stage, remaining warning should only be 'LaTeX Warning: Label(s) may have changed', if any.

Yes, I do understand that. Your (very welcome!) changes to the quick-quiz links can require additional passes through latex. Therefore, the script should change to provide this.

To see what I am getting at, please add a "\ref{this is a typo}" or a reference to any other non-\label string, then type "make". This is a pure error that no number of reruns will fix, but the build script will nevertheless crank away multiple times.

Quoted 8 lines

Before, it would run bibtex and stop. (Which, I freely admit, wasn't particularly good behavior either!)

My question is whether it is possible to make the script let the user know why it is re-running latex, so that the user can choose to kill it early, if appropriate. Or, alternatively (and for extra credit!), recognize which labels are undefined, and do more passes if they are quick-quiz links.

I see. I'll try to add those features and improve the behavior. I'm not sure how far I can do, but please wait for a while. And thank you so much for your valuable suggestions.

Regards, Akira

Quoted 102 lines

Quoted 7 lines

So I made a fairly invasive modification to the script so that the count is checked at the beginning of the loop. Also the loop is separated for each of the warning message.

Appended is the whole source code of the script (without the comment part at the head). I'm not sure which point should I make a diff from. Note that I truncated the warning messages given to grep. They should be long enough.

Good idea!

Quoted 1 line

What do you think?

Please see below.

Thanx, Paul

Quoted 16 lines

If it is OK, I'll resubmit the V2 of the patch series relative to the current master.

Thanks, Akira.

---
#!/bin/sh
# [snip]
#

if test -z "$1"
then
echo No latex file, aborting.
exit 1
fi

Quoted 8 lines

basename=echo $1 | sed -e 's/\.tex$//'

iter=1
echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----"

Please add "exit 1" here. (Yes, this was a bug in my orignal. But as long as we are here...)

Quoted 9 lines

fi
if grep -q 'LaTeX Warning: There were undefined references' $basename.log then

	if test -d "$2"
	then
		bibtex $basename || :
	else
		echo "No bibliography directory, skipping bibtex."
	fi

Quoted 9 lines

fi
while grep -q 'LaTeX Warning: There were undefined references' $basename.log do
if test "$iter" -eq 4
then
echo "Iteration limit: $iter passes through pdflatex" exit 1
fi
iter=expr $iter + 1

This would be one point to add some indication of why we are doing another pass through the loop. Might be as simple as adding "Undefined references" to the "echo" below. Or maybe even show the last symbol listed as undefined.

Quoted 8 lines

echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----" exit 1
fi
done

Quoted 8 lines

while grep -q 'LaTeX Warning: Label(s) may have changed' $basename.log do
if test "$iter" -ge 4
then
echo "Iteration limit: $iter passes through pdflatex" exit 1
fi
iter=expr $iter + 1

This would be one point to add some indication of why we are doing another pass through the loop. Might be as simple as adding "Label(s) may have changed" to the "echo" below.

Quoted 11 lines

echo "pdflatex $iter"
pdflatex $basename > /dev/null 2>&1 < /dev/null || :
if grep -q '! Emergency stop.' $basename.log
then
echo "----- Fatal latex error, see $basename.log for details. -----" exit 1
fi
done
grep "LaTeX Warning:" $basename.log
exit 0

Hi, Paul.

I'm tempted to move the bibtex part out from runlatex.sh into Makefile. If I make targets for .bbl files in Makefile, then the .bbl files won't be necessary in the repository any more.

I'm wondering why those .bbl files are in the repository in the first place. Was there any reason I can't think of?

Yes, I know controlling latex with Makefile is sometimes tricky because of its side effects, but I think I can manage.

Just want to confirm the circumstances.