Skip to contentSkip to conversation

tools: checkman.sh portability

PATCH Applied JWP 2 participants 6 messages Jan 13, 2015 – Jan 14, 2015 Util-Linux package

lore
Share & tools
lore

JWP

author
#0 lore raw

cover tools: checkman.sh portability

So, Sami Kerola's checkmans.sh tool caught me stuttering in hwclock.8 and I had to find out why checkmans did not work on my system:

1) Any test file with 'git' or 'lib' in its path fails. Including my entire repo under $PATH/github/util-linux and the ./lib/terminal-colors.d.5 man-page.

2) It depends on the man-db package.

The changes seem to be an improvement as it detected 4 new man-page problems (see separate patch).

I did alter the behavior slightly, previously failed pages required checking them again manually. Now it shows the troff parsing errors in progress. When grog fails there is nothing more to check, and when lexgrog fails I left the old behavior telling the caller to rerun the command, because I did not bother to install man-db to see what lexgrog actually does.

The following changes since commit 2e5bf10e32b00c92a3f7fb8b38d793ea6fc1dff2:

build-sys: add sfdisk.static (2015-01-13 16:59:03 +0100)

are available in the git repository at:

[email protected]:jwpi/util-linux.git checkman

J William Piggott (4):
tools: checkman.sh portability
tools: checkman.sh portability lexgrog
tools: checkmans.sh clean up 'let'
tools: checkmans.sh add stats

 tools/checkmans.sh | 85 +++++++++++++++++++++++++++++++++---------------------
 1 file changed, 52 insertions(+), 33 deletions(-)
author
#1 lore raw

PATCH 1/4 tools: checkman.sh portability

checkman.sh fails if 'git' or 'lib' are
anywhere in the tested file's path:
Change to using relative paths.
Improve the 'find' command call.

checkman.sh depends upon the man-db package:
Remove all instants of the man command and
use troff directly to improve portability.

---
 tools/checkmans.sh | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)
tools/checkmans.sh +12 -14
index bc3ba8f..6b80853 100755
@@ -39,10 +39,6 @@ while getopts vVh OPTIONS; do
 	esac
 done
 
-ERROR_FILE=$(mktemp ${SCRIPT_INVOCATION_SHORT_NAME}.XXXXXXXXXX)
-# remove tmp file at exit
-trap "rm -f ${ERROR_FILE}" 0
-
 # Try to find missing manuals matching build targets with manual files.
 declare -A MAN_LIST BIN_LIST
 
@@ -68,9 +64,10 @@ remove_repeats()
 	set -u
 }
 
+cd $(git rev-parse --show-toplevel)
+
 for I in $(
-	find $(git rev-parse --show-toplevel) -name '*.[1-8]' |
-	egrep -v '(Documentation|.git|/.libs/|autom4te.cache)'
+	find -path './autom4te.cache' -prune -o -name '*[[:alpha:]].[1-8]' -print
 ); do
 	MAN_FILE=${I##*/}
 	MAN_LIST[${MAN_FILE%%.[0-9]}]=1
@@ -78,7 +75,7 @@ for I in $(
 		# Some manuals, such as x86_64, call inclusion and they
 		# should not be tested any further.
 		if ${VERBOSE}; then
-			printf "skipping: ${I##*util-linux/}: includes "
+			printf "skipping: ${I}: includes "
 			awk '{print $2}' ${I}
 		fi
 		continue
@@ -87,16 +84,17 @@ for I in $(
 	if ${VERBOSE}; then
 		echo "testing: ${I}"
 	fi
-	MANWIDTH=80 man --warnings=all ${I} >/dev/null 2>| ${ERROR_FILE}
-	if [ -s ${ERROR_FILE} ]; then
-		echo "error: run: man --warnings=all ${I##*util-linux/} >/dev/null" >&2
-		I_ERR=1
+	RET=1
+	cat ${I} | troff -mandoc -ww -z |& grep "<" && RET=$?
+	if [ $RET = 0 ]; then
+	echo "From: cat ${I} | troff -mandoc -ww -z"
+	echo "=================================================="
 	fi
 	if ! lexgrog ${I} >/dev/null; then
-		echo "error: run: lexgrog ${I##*util-linux/}" >&2
+		echo "error: run: lexgrog ${I}" >&2
 		I_ERR=1
 	fi
-	REPEATS=( $(MANWIDTH=2000 man -l ${I} |
+	REPEATS=( $(cat ${I} | troff -mandoc -ww -Tascii | grotty |
 		col -b |
 		sed  -e 's/\s\+/\n/g; /^$/d' |
 		awk 'BEGIN { p="" } { if (0 < length($0)) { if (p == $0) { print } } p = $0 }') )
@@ -109,7 +107,7 @@ for I in $(
 			let ITER=${ITER}-1 || true
 		done
 		if [ 0 -lt "${#REPEATS[@]}" ]; then
-			echo "warning: ${I##*util-linux/} has repeating words: ${REPEATS[@]}"
+			echo "warning: ${I} has repeating words: ${REPEATS[@]}"
 		fi
 	fi
 	# The 'let' may cause exit on error.
author
#2 lore raw

PATCH 2/4 tools: checkman.sh portability lexgrog

In reply to JWP

checkman.sh depends upon the man-db package:
Add using grog when lexgrog not installed.

/dev/null some troff noise.

---
 tools/checkmans.sh | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
tools/checkmans.sh +21 -5
index 6b80853..513a750 100755
@@ -90,11 +90,23 @@ for I in $(
 	echo "From: cat ${I} | troff -mandoc -ww -z"
 	echo "=================================================="
 	fi
-	if ! lexgrog ${I} >/dev/null; then
-		echo "error: run: lexgrog ${I}" >&2
-		I_ERR=1
+	GROG=1
+	if command -v lexgrog &> /dev/null; then
+		if ! lexgrog ${I} >/dev/null; then
+			echo "error: run: lexgrog ${I}"
+			echo "=================================================="
+			((++COUNT_ERRORS))
+		fi
+	elif command -v grog &> /dev/null; then
+		if ! grog ${I} | grep man >/dev/null; then
+			echo "error: grog ${I} is not a man file"
+			echo "=================================================="
+			((++COUNT_ERRORS))
+		fi
+	else
+	GROG=0
 	fi
-	REPEATS=( $(cat ${I} | troff -mandoc -ww -Tascii | grotty |
+	REPEATS=( $(cat ${I} | troff -mandoc -Tascii 2>/dev/null | grotty |
 		col -b |
 		sed  -e 's/\s\+/\n/g; /^$/d' |
 		awk 'BEGIN { p="" } { if (0 < length($0)) { if (p == $0) { print } } p = $0 }') )
@@ -147,8 +159,12 @@ for I in ${!BIN_LIST[@]}; do
 done
 set -u
 
+if [ ${GROG} = 0 ]; then
+echo "warning: neither grog nor lexgrog commands were found"
+fi
+
 if [ ${COUNT_ERRORS} -ne 0 ]; then
-	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_ERRORS} manuals failed" >&2
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_ERRORS} manuals failed"
 	exit 1
 fi
 
author
#3 lore raw

PATCH 3/4 tools: checkmans.sh clean up 'let'

In reply to JWP

Clean up 'let' issues and loop iteration.

---
 tools/checkmans.sh | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
tools/checkmans.sh +4 -10
index 513a750..bd2bc43 100755
@@ -42,7 +42,8 @@ done
 # Try to find missing manuals matching build targets with manual files.
 declare -A MAN_LIST BIN_LIST
 
-COUNT_ERRORS=0
+declare -i ITER
+declare -i COUNT_ERRORS=0
 declare -a REPEATS
 declare -A KNOWN_REPEATS
 KNOWN_REPEATS[mount.8]='foo'
@@ -111,21 +112,14 @@ for I in $(
 		sed  -e 's/\s\+/\n/g; /^$/d' |
 		awk 'BEGIN { p="" } { if (0 < length($0)) { if (p == $0) { print } } p = $0 }') )
 	if [ 0 -lt "${#REPEATS[@]}" ]; then
-		ITER=${#REPEATS[@]}
-		while [ -1 -lt ${ITER} ]; do
+		ITER=${#REPEATS[@]}+1
+		while ((ITER--)); do
 			remove_repeats ${ITER}
-			# The 'let' may cause exit on error.
-			# When ITER == 0 -> let returns 1, bash bug?
-			let ITER=${ITER}-1 || true
 		done
 		if [ 0 -lt "${#REPEATS[@]}" ]; then
 			echo "warning: ${I} has repeating words: ${REPEATS[@]}"
 		fi
 	fi
-	# The 'let' may cause exit on error.
-	# COUNT_ERRORS=0; let COUNT_ERRORS=$COUNT_ERRORS+0; echo $?
-	# Is this a bash bug?
-	let COUNT_ERRORS=$COUNT_ERRORS+$I_ERR || true
 done
 
 # Create a list of build targets.
author
#4 lore raw

PATCH 4/4 tools: checkmans.sh add stats

In reply to JWP

Add test results statistics.
Fix-up 'KNOWN_REPEATS'.
Version bump.

---
 tools/checkmans.sh | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)
tools/checkmans.sh +22 -11
index bd2bc43..618f8ba 100755
@@ -26,7 +26,7 @@ while getopts vVh OPTIONS; do
 			VERBOSE='true'
 			;;
 		V)
-			echo "util-linux: ${SCRIPT_INVOCATION_SHORT_NAME}: 2"
+			echo "util-linux: ${SCRIPT_INVOCATION_SHORT_NAME}: v2.1"
 			exit 0
 			;;
 		h)
@@ -43,11 +43,13 @@ done
 declare -A MAN_LIST BIN_LIST
 
 declare -i ITER
-declare -i COUNT_ERRORS=0
+declare -i COUNT_GROG=0
+declare -i COUNT_TROFF=0
+declare -i COUNT_REPEATS=0
 declare -a REPEATS
 declare -A KNOWN_REPEATS
-KNOWN_REPEATS[mount.8]='foo'
-KNOWN_REPEATS[sfdisk.8]="0 <c,h,s>"
+KNOWN_REPEATS[mount.8]='foo l2 l c'
+KNOWN_REPEATS[hexdump.1]='l'
 KNOWN_REPEATS[flock.1]='"$0"'
 KNOWN_REPEATS[switch_root.8]='$DIR'
 
@@ -81,7 +83,6 @@ for I in $(
 		fi
 		continue
 	fi
-	I_ERR=0
 	if ${VERBOSE}; then
 		echo "testing: ${I}"
 	fi
@@ -90,22 +91,23 @@ for I in $(
 	if [ $RET = 0 ]; then
 	echo "From: cat ${I} | troff -mandoc -ww -z"
 	echo "=================================================="
+	((++COUNT_TROFF))
 	fi
 	GROG=1
 	if command -v lexgrog &> /dev/null; then
 		if ! lexgrog ${I} >/dev/null; then
 			echo "error: run: lexgrog ${I}"
 			echo "=================================================="
-			((++COUNT_ERRORS))
+			((++COUNT_GROG))
 		fi
 	elif command -v grog &> /dev/null; then
 		if ! grog ${I} | grep man >/dev/null; then
 			echo "error: grog ${I} is not a man file"
 			echo "=================================================="
-			((++COUNT_ERRORS))
+			((++COUNT_GROG))
 		fi
 	else
-	GROG=0
+		GROG=0
 	fi
 	REPEATS=( $(cat ${I} | troff -mandoc -Tascii 2>/dev/null | grotty |
 		col -b |
@@ -118,6 +120,8 @@ for I in $(
 		done
 		if [ 0 -lt "${#REPEATS[@]}" ]; then
 			echo "warning: ${I} has repeating words: ${REPEATS[@]}"
+			echo "=================================================="
+			((++COUNT_REPEATS))
 		fi
 	fi
 done
@@ -148,17 +152,24 @@ done
 set +u
 for I in ${!BIN_LIST[@]}; do
 	if [ -v ${MAN_LIST[$I]} ]; then
-		echo "warning: ${I} does not have man page"
+		echo "warning: ${SCRIPT_INVOCATION_SHORT_NAME}: ${I} does not have man page"
 	fi
 done
 set -u
 
+echo "${SCRIPT_INVOCATION_SHORT_NAME}: ${#BIN_LIST[*]} build targets were found"
+echo "${SCRIPT_INVOCATION_SHORT_NAME}: ${#MAN_LIST[*]} files were tested"
+
 if [ ${GROG} = 0 ]; then
 echo "warning: neither grog nor lexgrog commands were found"
 fi
 
-if [ ${COUNT_ERRORS} -ne 0 ]; then
-	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_ERRORS} manuals failed"
+if [ ${COUNT_GROG} -ne 0 -o ${COUNT_TROFF} -ne 0 -o ${COUNT_REPEATS} -ne 0 ]; then
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_GROG} files failed (lex)grog man-page test"
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_TROFF} files failed troff parsing test"
+	echo "error: ${SCRIPT_INVOCATION_SHORT_NAME}: ${COUNT_REPEATS} files have repeating words"
+	ITER=${#MAN_LIST[*]}-${COUNT_GROG}-${COUNT_TROFF}-${COUNT_REPEATS}
+	echo "${SCRIPT_INVOCATION_SHORT_NAME}: ${ITER} man-pages approved"
 	exit 1
 fi
 
maintainer Applied
#5 lore raw
In reply to JWP
Quoted 5 lines · JWP

J William Piggott (4):
tools: checkman.sh portability
tools: checkman.sh portability lexgrog
tools: checkmans.sh clean up 'let'
tools: checkmans.sh add stats

Merged, thanks.

Quoted 1 line
Signature · 2 lines
 Karel Zak  <[email protected]>
 http://karelzak.blogspot.com