Skip to contentSkip to conversation

Update to the mandocs build target

Kernel-janitors Michael Still 3 participants 4 messages Apr 19, 2004 – Apr 20, 2004

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Heya,

a while ago I whipped up a mandocs build target for 2.6 which is now included with the default kernel. The attached patch updates this build target to more exhaustively identify potential man pages. This makes the build of this target a little slower than before, but gives you a _lot_ more manpages.

I'm interested in some people applying the patch to the latest kernel, and letting me know if it works for them before I unleash it upon the masses.

Thanks,
Mikal

Signature · 3 lines
Michael Still ([email protected]) | "All my life I've had one dream,
http://www.stillhq.com            |  to achieve my many goals"
UTC + 11                          |    -- Homer Simpson
Documentation/DocBook/Makefile +15 -11
diff -Nur linux-2.6.5-cset-20040404_0407/Documentation/DocBook/Makefile linux-2.6.5-cset-20040404_0407-mandocsimp/Documentation/DocBook/Makefile
@@ -24,7 +24,7 @@
 
 ###
 # The targets that may be used.
-.PHONY:	sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs
+.PHONY:	sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs cleanmandocs
 
 BOOKS := $(addprefix $(obj)/,$(DOCBOOKS))
 sgmldocs: $(BOOKS)
@@ -38,12 +38,6 @@
 HTML := $(patsubst %.sgml, %.html, $(BOOKS))
 htmldocs: $(HTML)
 
-MAN := $(patsubst %.sgml, %.9, $(BOOKS))
-mandocs: $(MAN)
-
-installmandocs: mandocs
-	$(MAKEMAN) install Documentation/DocBook/man
-
 ###
 #External programs used
 KERNELDOC = scripts/kernel-doc
@@ -139,9 +133,18 @@
 ###
 # Rule to generate man files - output is placed in the man subdirectory
 
-%.9:	%.sgml
-	$(SPLITMAN) $< $(objtree)/Documentation/DocBook/man "$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)"
-	$(MAKEMAN) convert $(objtree)/Documentation/DocBook/man $<
+mandocs:
+	echo "" > $(obj)/allcomments.sgml
+	find $(objtree) -type f -name "*.[ch]" -exec $(KERNELDOC) -docbook -silentfail {} >> $(obj)/allcomments.sgml \;
+	$(SPLITMAN) $(obj)/allcomments.sgml $(objtree)/Documentation/DocBook/man "$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)"
+	$(MAKEMAN) convert $(objtree)/Documentation/DocBook/man $(obj)/allcomments.sgml
+
+installmandocs: $(objtree)/Documentation/DocBook/allcomments.sgml
+	$(MAKEMAN) install Documentation/DocBook/man
+
+cleanmandocs:
+	find $(objtree)/Documentation/DocBook/man -type f -name "*.sgml" -exec rm -f {} \;
+	find $(objtree)/Documentation/DocBook/man -type f -name "*.9.gz" -exec rm -f {} \;
 
 ###
 # Rules to generate postscripts and PNG imgages from .fig format files
@@ -196,7 +199,8 @@
 	$(patsubst %.sgml, %.9,    $(DOCBOOKS)) \
 	$(patsubst %.fig,%.eps,    $(IMG-parportbook)) \
 	$(patsubst %.fig,%.png,    $(IMG-parportbook)) \
-	$(C-procfs-example)
+	$(C-procfs-example) \
+	allcomments.sgml
 
 ifneq ($(wildcard $(patsubst %.html,%,$(HTML))),)
 clean-rule := rm -rf $(wildcard $(patsubst %.html,%,$(HTML)))
Documentation/DocBook/man/Makefile +1 -1
diff -Nur linux-2.6.5-cset-20040404_0407/Documentation/DocBook/man/Makefile linux-2.6.5-cset-20040404_0407-mandocsimp/Documentation/DocBook/man/Makefile
@@ -1,3 +1,3 @@
 # Rules are put in Documentation/DocBook
 
-clean-files := *.9.gz *.sgml manpage.links manpage.refs
+clean-files := manpage.links manpage.refs
Makefile +1 -1
diff -Nur linux-2.6.5-cset-20040404_0407/Makefile linux-2.6.5-cset-20040404_0407-mandocsimp/Makefile
@@ -799,7 +799,7 @@
 quiet_cmd_rmfiles = $(if $(rm-files),CLEAN   $(rm-files))
       cmd_rmfiles = rm -rf $(rm-files)
 
-clean: archclean $(clean-dirs)
+clean: archclean $(clean-dirs) cleanmandocs
 	$(call cmd,rmdirs)
 	$(call cmd,rmfiles)
 	@find . $(RCS_FIND_IGNORE) \
scripts/kernel-doc +16 -8
diff -Nur linux-2.6.5-cset-20040404_0407/scripts/kernel-doc linux-2.6.5-cset-20040404_0407-mandocsimp/scripts/kernel-doc
@@ -2,12 +2,13 @@
 
 use strict;
 
-## Copyright (c) 1998 Michael Zucchi, All Rights Reserved        ##
+## Copyright (C) 1998     Michael Zucchi, All Rights Reserved    ##
 ## Copyright (C) 2000, 1  Tim Waugh <[email protected]>          ##
-## Copyright (C) 2001  Simon Huggins                             ##
+## Copyright (C) 2001     Simon Huggins                          ##
+## Copyright (C) 2004     Michael Still <[email protected]>      ##
 ## 								 ##
 ## #define enhancements by Armin Kuster <[email protected]>	 ##
-## Copyright (c) 2000 MontaVista Software, Inc.			 ##
+## Copyright (C) 2000 MontaVista Software, Inc.			 ##
 ## 								 ##
 ## This software falls under the GNU General Public License.     ##
 ## Please read the COPYING file for more information             ##
@@ -36,6 +37,7 @@
 #              Small fixes (like spaces vs. \s in regex)
 # -- Tim Jansen <[email protected]>
 
+# 15/03/2004 - Tweaked script to better support mandocs target
 
 #
 # This will read a 'c' file and scan for embedded comments in the
@@ -45,7 +47,7 @@
 # Note: This only supports 'c'.
 
 # usage:
-# kerneldoc [ -docbook | -html | -text | -man ]
+# kerneldoc [ -docbook [-silentfail] | -html | -text | -man ]
 #           [ -function funcname [ -function funcname ...] ] c file(s)s > outputfile
 # or
 #           [ -nofunction funcname [ -function funcname ...] ] c file(s)s > outputfile
@@ -206,7 +208,7 @@
 
 
 sub usage {
-    print "Usage: $0 [ -v ] [ -docbook | -html | -text | -man ]\n";
+    print "Usage: $0 [ -v ] [ -docbook [-silentfail] | -html | -text | -man ]\n";
     print "         [ -function funcname [ -function funcname ...] ]\n";
     print "         [ -nofunction funcname [ -nofunction funcname ...] ]\n";
     print "         c source file(s) > outputfile\n";
@@ -283,6 +285,8 @@
 
 my $undescribed = "-- undescribed --";
 
+my $silentfail = 0;
+
 reset_state();
 
 while ($ARGV[0] =~ m/^-(.*)/) {
@@ -322,7 +326,9 @@
     } elsif (($cmd eq "-h") || ($cmd eq "--help")) {
 	usage();
     } elsif ($cmd eq '-filelist') {
-	    $filelist = shift @ARGV;
+	$filelist = shift @ARGV;
+    } elsif ($cmd eq '-silentfail') {
+	$silentfail = 1;
     }
 }
 
@@ -1701,8 +1707,10 @@
           }
     }
     if ($initial_section_counter == $section_counter) {
-	print STDERR "Warning(${file}): no structured comments found\n";
-	if ($output_mode eq "sgml") {
+        if($silentfail == 0){
+	    print STDERR "Warning(${file}): no structured comments found\n";
+	}
+	if (($output_mode eq "sgml") && ($silentfail == 0)) {
 	    # The template wants at least one RefEntry here; make one.
 	    print "<refentry>\n";
 	    print " <refnamediv>\n";
scripts/makeman +2 -2
diff -Nur linux-2.6.5-cset-20040404_0407/scripts/makeman linux-2.6.5-cset-20040404_0407-mandocsimp/scripts/makeman
@@ -2,7 +2,7 @@
 
 use strict;
 
-## Copyright (C) Michael Still ([email protected])
+## Copyright (C) 2003, 4     Michael Still ([email protected])
 ## Released under the terms of the GNU GPL
 ##
 ## A script to make or install the manpages extracted by split-man
@@ -176,7 +176,7 @@
   }
 }
 elsif($ARGV[0] eq "install"){
-  system("mkdir -p /usr/local/man/man9/; install $ARGV[1]/*.9.gz /usr/local/man/man9/");
+  system("mkdir -p /usr/local/man/man9/; find $ARGV[1] -type f -name '*.9.gz' -exec install {} /usr/local/man/man9/ \\;");
 }
 else{
   die "Usage: makeman [convert | install] <dir> <file>\n";

_______________________________________________
Kernel-janitors mailing list
[email protected]
http://lists.osdl.org/mailman/listinfo/kernel-janitors

#1 lore raw

On Mon, 19 Apr 2004 10:19:53 +1000 Michael Still wrote:

| 
| Heya,
| 
| a while ago I whipped up a mandocs build target for 2.6 which is now 
| included with the default kernel. The attached patch updates this build 
| target to more exhaustively identify potential man pages. This makes the 
| build of this target a little slower than before, but gives you a _lot_ 
| more manpages.
| 
| I'm interested in some people applying the patch to the latest kernel, 
| and letting me know if it works for them before I unleash it upon the 
| masses.

Hi,
I'll share my experiences and comments on this patch.

1. Before applying this patch, if I did 'make mandocs', I had 21 empty (0-length) man files generated.[1]
They don't appear when using this patch (at least not yet, but it hasn't finished yet either -- that's #2.).

2. 'make mandocs' has now been running for about 1/2 hour. I hope it finishes soon. :)
... well, about 1 hour and still going (dual-proc P4, 1.7 GHz, 1 GB RAM).

3. I have a man page/file named 'man/if.9.gz'. Some 'if' statement got picked up from somewhere. Sometimes it is nice to include a line about the file that an API was found in (even without having found this odd one).

--
~Randy
"We have met the enemy and he is us." -- Pogo (by Walt Kelly) (Again. Sometimes I think ln -s /usr/src/linux/.config .signature)

[1]:

-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 deviceiobook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:07 gadget.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:05 journal-api.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:59 kernel-api.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 kernel-hacking.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 kernel-locking.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:07 libata.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:06 lsm.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 mcabook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 mousedrivers.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 parportbook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 procfs-guide.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 scsidrivers.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 sis900.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 tulip-user.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:06 usb.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 via-audio.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 videobook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 wanbook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 writing_usb_driver.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 z8530book.9

_______________________________________________
Kernel-janitors mailing list
[email protected]
http://lists.osdl.org/mailman/listinfo/kernel-janitors

#2 lore raw
In reply to Michael Still
Quoted 1 line · Randy.Dunlap

Hi,

Hello

Quoted 1 line

I'll share my experiences and comments on this patch.

Thanks...

Quoted 4 lines

1. Before applying this patch, if I did 'make mandocs', I had 21 empty (0-length) man files generated.[1]
They don't appear when using this patch (at least not yet, but it hasn't finished yet either -- that's #2.).

See my comments below (after your list of files).

Quoted 4 lines

2. 'make mandocs' has now been running for about 1/2 hour. I hope it finishes soon. :)
... well, about 1 hour and still going (dual-proc P4, 1.7 GHz, 1 GB RAM).

# make clean
# time make mandocs
...output...
real    40m28.109s
user    30m24.941s
sys     3m48.073s

(That's a Pentium=M 1.3 GHz, with a 4200 RPM disc).

I would expect the target to be IO bound. Was the disc system on the box busy at the same time? In a nutshell, the old build target built all of the sgmldocs in the Documentation/DocBook directory, and then extracted the manual pages from there. I went away from this approach as many (ie most) kernel-doc comments in the source code never make it into the sgml files in Documentation/DocBook. Instead, the new patch does something along the lines of:

find . -type f -name "*.[cChHsS]" -exec ...extract comments... \;

Which takes a while. It then converts all the stuff it found into manual pages, which itself takes longer than before because there are a _lot_ of manual pages now -- 4033 to be precise.

I don't like the find approach, but I couldn't think of a better way to find all of the comments. My attitude is that the build target will be infrequently used, and if someone went to the trouble to put the comment into the source code, then it deserves to be a manual page as well. If the build time is a big problem, then hopefully distributions will simply ship the output of the generation step with their kernel packages.

Quoted 4 lines

3. I have a man page/file named 'man/if.9.gz'. Some 'if' statement got picked up from somewhere. Sometimes it is nice to include a line about the file that an API was found in (even without having found this odd one).

Yes. I hadn't thought of that. The previous version of mandocs copied the copyright statement from the sgml file into the manpage as a comment, but the new version can't do this as it doesn't know what (if any) sgml file the comment is used in. I'll work on a tweaked version ASAP.

The comment extraction is done with a patched version of the original kernel-doc, so you've probably found a bug with that. I will look into it.

Another interesting problem is that of repeated comments. There are places where more than one comment documents a function of a given name. At the moment the last found comment wins. Ultimately it would be nice to audit those comments, and work out why there are repeats (multiple architectures repeating code?).

Quoted 12 lines

[1]:

(quoted diff) +0 -11
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 deviceiobook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:07 gadget.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:05 journal-api.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:59 kernel-api.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 kernel-hacking.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 kernel-locking.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:07 libata.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:06 lsm.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 mcabook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 mousedrivers.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 parportbook.9

Review

Quoted 10 lines
(quoted diff) +0 -10
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 procfs-guide.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 scsidrivers.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 sis900.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 tulip-user.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 14:06 usb.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 via-audio.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 videobook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 wanbook.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:58 writing_usb_driver.9
-rw-r--r--    1 rddunlap staff           0 Apr 19 13:57 z8530book.9

I have no manual pages of zero length with the new patch.

Mikal

Signature · 3 lines
Michael Still ([email protected]) | "All my life I've had one dream,
http://www.stillhq.com            |  to achieve my many goals"
UTC + 11                          |    -- Homer Simpson

_______________________________________________
Kernel-janitors mailing list
[email protected]
http://lists.osdl.org/mailman/listinfo/kernel-janitors

#3 lore raw
In reply to Michael Still
Quoted 5 lines · Michael Still

Another interesting problem is that of repeated comments. There are places where more than one comment documents a function of a given name. At the moment the last found comment wins. Ultimately it would be nice to audit those comments, and work out why there are repeats (multiple architectures repeating code?).

Architectures should not be including kerneldoc comments. i386 is and always has been the refereence platform for Linux and thus is the only architecture which should have kerneldoc comments. If there's one thing worse than having one set of buggy comments, it's having eight sets of buggy comments, all subtly incompatible.

Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

_______________________________________________
Kernel-janitors mailing list
[email protected]
http://lists.osdl.org/mailman/listinfo/kernel-janitors