Skip to contentSkip to conversation

DocBook: Avoid stdout junk with no man pages to compress

PATCH In review Ulf Magnusson 2 participants 5 messages Jul 12, 2015 – Jul 14, 2015 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 DocBook: Avoid stdout junk with no man pages to compress

gzip would run as 'gzip -f' when no uncompressed man pages were found, making it compress the (empty) stdin to stdout.

A few alternative solutions:

- 'find' with {} + might be speedier, but maybe that's not portable enough (though it's in POSIX 2001 at least AFAICS)

- xargs --no-run-if-empty, but that's a GNU extension

- Always discarding stdout, if it's unlikely to ever be helpful

- More fancy stuff like the following, though maybe some of them could run into shell limits too, re. d56fcf299fb4 (DocBook: Do not exceed argument list limit)

* A plain shell 'for' loop

* mandocs: $(MAN)
if [ find $(obj)/man -name '*.9' | wc -l -gt 0 ]; then \
find $(obj)/man -name '*.9' | xargs gzip -f; \
fi

* mandocs: $(MAN)
man_pages=find $(obj)/man -name '*.9'; \
if [ -n "$$man_pages" ]; then \
echo "$$man_pages" | xargs gzip -f; \
fi

---
 Documentation/DocBook/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/DocBook/Makefile +1 -1
index b6a6a2e..73bddf7 100644
@@ -56,7 +56,7 @@ htmldocs: $(HTML)
 
 MAN := $(patsubst %.xml, %.9, $(BOOKS))
 mandocs: $(MAN)
-	find $(obj)/man -name '*.9' | xargs gzip -f
+	find $(obj)/man -name '*.9' -exec gzip -f {} \;
 
 installmandocs: mandocs
 	mkdir -p /usr/local/man/man9/
Signature · 1 line
2.1.4
Tested-by on v1 1/1
#1 lore raw
Quoted 6 lines · Ulf Magnusson

gzip would run as 'gzip -f' when no uncompressed man pages were found, making it compress the (empty) stdin to stdout.

Documentation/DocBook/Makefile htmldocs: $(HTML) · line 62

Quoted 8 lines
Documentation/DocBook/Makefile +1 -1
@@ -56,7 +56,7 @@ htmldocs: $(HTML)
 
 MAN := $(patsubst %.xml, %.9, $(BOOKS))
 mandocs: $(MAN)
-       find $(obj)/man -name '*.9' | xargs gzip -f
+       find $(obj)/man -name '*.9' -exec gzip -f {} \;
 
 installmandocs: mandocs
        mkdir -p /usr/local/man/man9/

That does get rid of the binary burp, but 'xargs gzip -f' has been in the Makefile since January, and gzipping '\n' just started recently. So what's changed?

It looks like, for whatever reason, make installmandocs always ends up rerunning mandocs -- there's now a 'GEN Documentation Docbook//v4l2.xml' printed, and that extra mandocs invocation is where the problematic second invocation of find is coming from. I won't pretend to understand the Makefile flow to guess at why that's happening, but obviously 'make mandocs; make installmandocs' shouldn't need to regenerate things already generated.

In any event,

Jim

#2 lore raw
Quoted 13 lines · Jim Davis
Quoted 12 lines · Ulf Magnusson

gzip would run as 'gzip -f' when no uncompressed man pages were found, making it compress the (empty) stdin to stdout.

Documentation/DocBook/Makefile +1 -1
@@ -56,7 +56,7 @@ htmldocs: $(HTML)
 
 MAN := $(patsubst %.xml, %.9, $(BOOKS))
 mandocs: $(MAN)
-       find $(obj)/man -name '*.9' | xargs gzip -f
+       find $(obj)/man -name '*.9' -exec gzip -f {} \;
 
Quoted 7 lines

Documentation/DocBook/Makefile htmldocs: $(HTML) · line 62

Quoted 2 lines
Documentation/DocBook/Makefile +0 -0
 installmandocs: mandocs
        mkdir -p /usr/local/man/man9/

That does get rid of the binary burp, but 'xargs gzip -f' has been in the Makefile since January, and gzipping '\n' just started recently. So what's changed?

No idea. I just assumed it had been broken since then, since the version before d56fcf299fb4 (DocBook: Do not exceed argument list limit) looked for *.9 files before running gzip:

mandocs: $(MAN)

	$(if $(wildcard $(obj)/man/*.9),gzip -f $(obj)/man/*.9)
Quoted 7 lines

It looks like, for whatever reason, make installmandocs always ends up rerunning mandocs -- there's now a 'GEN Documentation Docbook//v4l2.xml' printed, and that extra mandocs invocation is where the problematic second invocation of find is coming from. I won't pretend to understand the Makefile flow to guess at why that's happening, but obviously 'make mandocs; make installmandocs' shouldn't need to regenerate things already generated.

I won't pretend to understand the Makefile flow either. Guess it might be worth looking into v4l2.xml as well then. Could be some directory shenanigans going on judging from the '//'.

Quoted 6 lines

In any event,

Jim

I just noticed the commit message only mentions the alternative solutions and not the implemented solution. Could send a v2 that fixes that, but I'll wait for more comments first.

Cheers,
Ulf

#3 lore raw
Quoted 37 lines · Ulf Magnusson
Quoted 13 lines · Jim Davis
Quoted 12 lines · Ulf Magnusson

gzip would run as 'gzip -f' when no uncompressed man pages were found, making it compress the (empty) stdin to stdout.

Documentation/DocBook/Makefile +1 -1
@@ -56,7 +56,7 @@ htmldocs: $(HTML)
 
 MAN := $(patsubst %.xml, %.9, $(BOOKS))
 mandocs: $(MAN)
-       find $(obj)/man -name '*.9' | xargs gzip -f
+       find $(obj)/man -name '*.9' -exec gzip -f {} \;
 
Quoted 7 lines

Documentation/DocBook/Makefile htmldocs: $(HTML) · line 62

Quoted 2 lines
Documentation/DocBook/Makefile +0 -0
 installmandocs: mandocs
        mkdir -p /usr/local/man/man9/

That does get rid of the binary burp, but 'xargs gzip -f' has been in the Makefile since January, and gzipping '\n' just started recently. So what's changed?

No idea. I just assumed it had been broken since then, since the version before d56fcf299fb4 (DocBook: Do not exceed argument list limit) looked for *.9 files before running gzip:

mandocs: $(MAN)

        $(if $(wildcard $(obj)/man/*.9),gzip -f $(obj)/man/*.9)
Quoted 7 lines

It looks like, for whatever reason, make installmandocs always ends up rerunning mandocs -- there's now a 'GEN Documentation Docbook//v4l2.xml' printed, and that extra mandocs invocation is where the problematic second invocation of find is coming from. I won't pretend to understand the Makefile flow to guess at why that's happening, but obviously 'make mandocs; make installmandocs' shouldn't need to regenerate things already generated.

Quoted 4 lines

I won't pretend to understand the Makefile flow either. Guess it might be worth looking into v4l2.xml as well then. Could be some directory shenanigans going on judging from the '//'.

I looked into it some more, and I'm now fairly certain that 'mandocs' always runs its recipe regardless of the status of the prerequisites. Changing the top-level Makefile to do

        $(Q)$(MAKE) --debug=v $(build)=Documentation/DocBook $@

and running 'make mandocs', you see some FORCEs in the output, which -- unless my make fu is weak -- causes all the targets above that to always be considered out of date:

File 'mandocs' does not exist.
Considering target file 'Documentation/DocBook/z8530book.9'.
Considering target file 'Documentation/DocBook/z8530book.xml'.
...
Considering target file 'FORCE'.
File 'FORCE' does not exist.
Finished prerequisites of target file 'FORCE'.
Must remake target 'FORCE'.
Successfully remade target file 'FORCE'.
...
Must remake target 'Documentation/DocBook/z8530book.xml'.
...
Must remake target 'mandocs'.

Re. the 'GEN Documentation Docbook//v4l2.xml', I think the problem is the following rule in Documentation/DocBook/media/Makefile:

$(MEDIA_OBJ_DIR)/v4l2.xml: $(OBJIMGFILES)

        @$($(quiet)gen_xml)
        @(ln -sf `cd $(MEDIA_SRC_DIR) && /bin/pwd`/v4l/*xml $(MEDIA_OBJ_DIR)/)
        @(ln -sf `cd $(MEDIA_SRC_DIR) && /bin/pwd`/dvb/*xml $(MEDIA_OBJ_DIR)/)

./Documentation/DocBook/v4l2.xml is a symlink, so make compares the modification time of the symlink *target* (which is never updated) against the image (*.png, *.gif, etc.) files in $(OBJIMGFILES). Updating the symlink itself won't change that modification time, so that's why it always runs.

Passing --check-symlink-times to make so that it also looks at the modification time of the symlink seems to change a bunch of other stuff in the output too. Maybe there's other problems lurking here as well.

Quoted 5 lines
Quoted 5 lines

In any event,

Quoted 8 lines
Quoted 1 line

Jim

I just noticed the commit message only mentions the alternative solutions and not the implemented solution. Could send a v2 that fixes that, but I'll wait for more comments first.

Cheers,
Ulf

Cheers,
Ulf

#4 lore raw
Quoted 53 lines · Ulf Magnusson
Quoted 37 lines · Ulf Magnusson
Quoted 13 lines · Jim Davis
Quoted 12 lines · Ulf Magnusson

gzip would run as 'gzip -f' when no uncompressed man pages were found, making it compress the (empty) stdin to stdout.

Documentation/DocBook/Makefile +1 -1
@@ -56,7 +56,7 @@ htmldocs: $(HTML)
 
 MAN := $(patsubst %.xml, %.9, $(BOOKS))
 mandocs: $(MAN)
-       find $(obj)/man -name '*.9' | xargs gzip -f
+       find $(obj)/man -name '*.9' -exec gzip -f {} \;
 
Quoted 7 lines

Documentation/DocBook/Makefile htmldocs: $(HTML) · line 62

Quoted 2 lines
Documentation/DocBook/Makefile +0 -0
 installmandocs: mandocs
        mkdir -p /usr/local/man/man9/

That does get rid of the binary burp, but 'xargs gzip -f' has been in the Makefile since January, and gzipping '\n' just started recently. So what's changed?

No idea. I just assumed it had been broken since then, since the version before d56fcf299fb4 (DocBook: Do not exceed argument list limit) looked for *.9 files before running gzip:

mandocs: $(MAN)

        $(if $(wildcard $(obj)/man/*.9),gzip -f $(obj)/man/*.9)
Quoted 7 lines

It looks like, for whatever reason, make installmandocs always ends up rerunning mandocs -- there's now a 'GEN Documentation Docbook//v4l2.xml' printed, and that extra mandocs invocation is where the problematic second invocation of find is coming from. I won't pretend to understand the Makefile flow to guess at why that's happening, but obviously 'make mandocs; make installmandocs' shouldn't need to regenerate things already generated.

Quoted 4 lines

I won't pretend to understand the Makefile flow either. Guess it might be worth looking into v4l2.xml as well then. Could be some directory shenanigans going on judging from the '//'.

I looked into it some more, and I'm now fairly certain that 'mandocs' always runs its recipe regardless of the status of the prerequisites. Changing the top-level Makefile to do

        $(Q)$(MAKE) --debug=v $(build)=Documentation/DocBook $@

and running 'make mandocs', you see some FORCEs in the output, which -- unless my make fu is weak -- causes all the targets above that to always be considered out of date:

Quoted 13 lines

File 'mandocs' does not exist.
Considering target file 'Documentation/DocBook/z8530book.9'.
Considering target file 'Documentation/DocBook/z8530book.xml'.
...
Considering target file 'FORCE'.
File 'FORCE' does not exist.
Finished prerequisites of target file 'FORCE'.
Must remake target 'FORCE'.
Successfully remade target file 'FORCE'.
...
Must remake target 'Documentation/DocBook/z8530book.xml'.
...
Must remake target 'mandocs'.

Had forgotten that 'mandocs' is a phony target too. That alone makes it always run its recipe when it's a prerequisite of some target that's run.

Quoted 30 lines

Re. the 'GEN Documentation Docbook//v4l2.xml', I think the problem is the following rule in Documentation/DocBook/media/Makefile:

$(MEDIA_OBJ_DIR)/v4l2.xml: $(OBJIMGFILES)

        @$($(quiet)gen_xml)
        @(ln -sf `cd $(MEDIA_SRC_DIR) && /bin/pwd`/v4l/*xml $(MEDIA_OBJ_DIR)/)
        @(ln -sf `cd $(MEDIA_SRC_DIR) && /bin/pwd`/dvb/*xml $(MEDIA_OBJ_DIR)/)

./Documentation/DocBook/v4l2.xml is a symlink, so make compares the modification time of the symlink *target* (which is never updated) against the image (*.png, *.gif, etc.) files in $(OBJIMGFILES). Updating the symlink itself won't change that modification time, so that's why it always runs.

Passing --check-symlink-times to make so that it also looks at the modification time of the symlink seems to change a bunch of other stuff in the output too. Maybe there's other problems lurking here as well.

Quoted 11 lines
Quoted 6 lines

In any event,

Jim

I just noticed the commit message only mentions the alternative solutions and not the implemented solution. Could send a v2 that fixes that, but I'll wait for more comments first.

Quoted 5 lines
Quoted 2 lines

Cheers,
Ulf

Cheers,
Ulf

Cheers,
Ulf