Skip to contentSkip to conversation

kernel-doc: add DOC: sections support

PATCH In review Randy Dunlap 4 participants 14 messages Oct 24, 2007 – Oct 26, 2007 Kbuild

lore
Share & tools
lore
#1 lore raw

PATCH 1/6 kernel-doc: fix xml output mode

In reply to Randy Dunlap

From: Johannes Berg <[email protected]>

After Randy's patch fixing the HTML output in DOC: sections (6b5b55f6c404fa730a09a8254eb19f5a038afcc2) the same bug remained in XML mode, this fixes it.

---
 scripts/kernel-doc |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
scripts/kernel-doc +3 -3
@@ -182,10 +182,10 @@ my $blankline_html = $local_lt . "p" . $
 my %highlights_xml = ( "([^=])\\\"([^\\\"<]+)\\\"", "\$1<quote>\$2</quote>",
 			$type_constant, "<constant>\$1</constant>",
 			$type_func, "<function>\$1</function>",
-			$type_struct, "<structname>\$1</structname>",
+			$type_struct_xml, "<structname>\$1</structname>",
 			$type_env, "<envar>\$1</envar>",
 			$type_param, "<parameter>\$1</parameter>" );
-my $blankline_xml = "</para><para>\n";
+my $blankline_xml = $local_lt . "/para" . $local_gt . $local_lt . "para" . $local_gt . "\n";
 
 # gnome, docbook format
 my %highlights_gnome = ( $type_constant, "<replaceable class=\"option\">\$1</replaceable>",
@@ -394,7 +394,7 @@ sub output_highlight {
 #	confess "output_highlight got called with no args?\n";
 #   }
 
-    if ($output_mode eq "html") {
+    if ($output_mode eq "html" || $output_mode eq "xml") {
 	$contents = local_unescape($contents);
 	# convert data read & converted thru xml_escape() into &xyz; format:
 	$contents =~ s/\\\\\\/&/g;
#0 lore raw

cover kernel-doc: add DOC: sections support

This patch series from Johannes Berg adds support for DOC: sections that are embedded in source files.

#5 lore raw

PATCH 5/6 kernel-doc: use no-doc option

From: Johannes Berg <[email protected]>

When asked by a template to include all functions from a file, it will also include DOC: sections wreaking havoc in the generated docbook file. This patch makes it use the new -no-doc-sections flag for kernel-doc to avoid this.

---
 scripts/basic/docproc.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
scripts/basic/docproc.c +3 -1
@@ -65,6 +65,7 @@ FILELINE * entity_system;
 #define DOCBOOK       "-docbook"
 #define FUNCTION      "-function"
 #define NOFUNCTION    "-nofunction"
+#define NODOCSECTIONS "-no-doc-sections"
 
 char *srctree;
 
@@ -231,13 +232,14 @@ void docfunctions(char * filename, char 
 
 	for (i=0; i <= symfilecnt; i++)
 		symcnt += symfilelist[i].symbolcnt;
-	vec = malloc((2 + 2 * symcnt + 2) * sizeof(char*));
+	vec = malloc((2 + 2 * symcnt + 3) * sizeof(char*));
 	if (vec == NULL) {
 		perror("docproc: ");
 		exit(1);
 	}
 	vec[idx++] = KERNELDOC;
 	vec[idx++] = DOCBOOK;
+	vec[idx++] = NODOCSECTIONS;
 	for (i=0; i < symfilecnt; i++) {
 		struct symfile * sym = &symfilelist[i];
 		for (j=0; j < sym->symbolcnt; j++) {
#4 lore raw

PATCH 4/6 kernel-doc: process functions, not DOC:

In reply to Randy Dunlap

From: Johannes Berg <[email protected]>

This flag is necessary for the next patch for docproc to output only the functions and not DOC: sections when a function list is requested.

---
 scripts/kernel-doc |   14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
scripts/kernel-doc +12 -2
@@ -46,13 +46,16 @@ use strict;
 # Note: This only supports 'c'.
 
 # usage:
-# kernel-doc [ -docbook | -html | -text | -man ]
+# kernel-doc [ -docbook | -html | -text | -man ] [ -no-doc-sections ]
 #           [ -function funcname [ -function funcname ...] ] c file(s)s > outputfile
 # or
 #           [ -nofunction funcname [ -function funcname ...] ] c file(s)s > outputfile
 #
 #  Set output format using one of -docbook -html -text or -man.  Default is man.
 #
+#  -no-doc-sections
+#	Do not output DOC: sections
+#
 #  -function funcname
 #	If set, then only generate documentation for the given function(s) or
 #	DOC: section titles.  All other functions and DOC: sections are ignored.
@@ -211,7 +214,7 @@ my $blankline_text = "";
 
 
 sub usage {
-    print "Usage: $0 [ -v ] [ -docbook | -html | -text | -man ]\n";
+    print "Usage: $0 [ -v ] [ -docbook | -html | -text | -man ] [ -no-doc-sections ]\n";
     print "         [ -function funcname [ -function funcname ...] ]\n";
     print "         [ -nofunction funcname [ -nofunction funcname ...] ]\n";
     print "         c source file(s) > outputfile\n";
@@ -225,6 +228,7 @@ if ($#ARGV==-1) {
 
 my $verbose = 0;
 my $output_mode = "man";
+my $no_doc_sections = 0;
 my %highlights = %highlights_man;
 my $blankline = $blankline_man;
 my $modulename = "Kernel API";
@@ -329,6 +333,8 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 	usage();
     } elsif ($cmd eq '-filelist') {
 	    $filelist = shift @ARGV;
+    } elsif ($cmd eq '-no-doc-sections') {
+	    $no_doc_sections = 1;
     }
 }
 
@@ -380,6 +386,10 @@ sub dump_doc_section {
     my $name = shift;
     my $contents = join "\n", @_;
 
+    if ($no_doc_sections) {
+	return;
+    }
+
     if (($function_only==0) ||
 	( $function_only == 1 && defined($function_table{$name})) ||
 	( $function_only == 2 && !defined($function_table{$name})))
#3 lore raw

PATCH 3/6 kernel-doc: single DOC: selection

In reply to Randy Dunlap

From: Johannes Berg <[email protected]>

Currently, DOC: sections are always output even if only a single function is requested, fix this and also make it possible to just output a single DOC: section by giving its title as the function name to output.

Also fixes docbook XML well-formedness for sections with examples.

---
 scripts/kernel-doc |   63 ++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 20 deletions(-)
scripts/kernel-doc +43 -20
@@ -54,13 +54,13 @@ use strict;
 #  Set output format using one of -docbook -html -text or -man.  Default is man.
 #
 #  -function funcname
-#	If set, then only generate documentation for the given function(s).  All
-#	other functions are ignored.
+#	If set, then only generate documentation for the given function(s) or
+#	DOC: section titles.  All other functions and DOC: sections are ignored.
 #
 #  -nofunction funcname
-#	If set, then only generate documentation for the other function(s).
-#	Cannot be used together with -function
-#	(yes, that's a bug -- perl hackers can fix it 8))
+#	If set, then only generate documentation for the other function(s)/DOC:
+#	sections. Cannot be used together with -function (yes, that's a bug --
+#	perl hackers can fix it 8))
 #
 #  c files - list of 'c' files to process
 #
@@ -374,6 +374,25 @@ sub dump_section {
 }
 
 ##
+# dump DOC: section after checking that it should go out
+#
+sub dump_doc_section {
+    my $name = shift;
+    my $contents = join "\n", @_;
+
+    if (($function_only == 0) ||
+	( $function_only == 1 && defined($function_table{$name})) ||
+	( $function_only == 2 && !defined($function_table{$name})))
+    {
+	dump_section $name, $contents;
+	output_blockhead({'sectionlist' => \@sectionlist,
+			  'sections' => \%sections,
+			  'module' => $modulename,
+			  'content-only' => ($function_only != 0), });
+    }
+}
+
+##
 # output function
 #
 # parameterdescs, a hash.
@@ -564,8 +583,8 @@ sub output_function_html(%) {
     print "<hr>\n";
 }
 
-# output intro in html
-sub output_intro_html(%) {
+# output DOC: block header in html
+sub output_blockhead_html(%) {
     my %args = %{$_[0]};
     my ($parameter, $section);
     my $count;
@@ -871,7 +890,7 @@ sub output_typedef_xml(%) {
 }
 
 # output in XML DocBook
-sub output_intro_xml(%) {
+sub output_blockhead_xml(%) {
     my %args = %{$_[0]};
     my ($parameter, $section);
     my $count;
@@ -882,15 +901,23 @@ sub output_intro_xml(%) {
     # print out each section
     $lineprefix="   ";
     foreach $section (@{$args{'sectionlist'}}) {
-	print "<refsect1>\n <title>$section</title>\n <para>\n";
+    	if (!$args{'content-only'}) {
+		print "<refsect1>\n <title>$section</title>\n";
+	}
 	if ($section =~ m/EXAMPLE/i) {
 	    print "<example><para>\n";
+	} else {
+	    print "<para>\n";
 	}
 	output_highlight($args{'sections'}{$section});
 	if ($section =~ m/EXAMPLE/i) {
 	    print "</para></example>\n";
+	} else {
+	    print "</para>";
+	}
+	if (!$args{'content-only'}) {
+		print "\n</refsect1>\n";
 	}
-	print " </para>\n</refsect1>\n";
     }
 
     print "\n\n";
@@ -1137,7 +1164,7 @@ sub output_typedef_man(%) {
     }
 }
 
-sub output_intro_man(%) {
+sub output_blockhead_man(%) {
     my %args = %{$_[0]};
     my ($parameter, $section);
     my $count;
@@ -1294,7 +1321,7 @@ sub output_struct_text(%) {
     output_section_text(@_);
 }
 
-sub output_intro_text(%) {
+sub output_blockhead_text(%) {
     my %args = %{$_[0]};
     my ($parameter, $section);
 
@@ -1325,9 +1352,9 @@ sub output_declaration {
 
 ##
 # generic output function - calls the right one based on current output mode.
-sub output_intro {
+sub output_blockhead {
     no strict 'refs';
-    my $func = "output_intro_".$output_mode;
+    my $func = "output_blockhead_".$output_mode;
     &$func(@_);
     $section_counter++;
 }
@@ -1926,9 +1953,7 @@ sub process_file($) {
 	} elsif ($state == 4) {
 		# Documentation block
 		if (/$doc_block/) {
-			dump_section($section, xml_escape($contents));
-			output_intro({'sectionlist' => \@sectionlist,
-				      'sections' => \%sections });
+			dump_doc_section($section, xml_escape($contents));
 			$contents = "";
 			$function = "";
 			%constants = ();
@@ -1946,9 +1971,7 @@ sub process_file($) {
 		}
 		elsif (/$doc_end/)
 		{
-			dump_section($section, xml_escape($contents));
-			output_intro({'sectionlist' => \@sectionlist,
-				      'sections' => \%sections });
+			dump_doc_section($section, xml_escape($contents));
 			$contents = "";
 			$function = "";
 			%constants = ();
#2 lore raw

PATCH 2/6 kernel-doc: init kernel version

In reply to Randy Dunlap

From: Johannes Berg <[email protected]>

The kernel-doc script triggers a perl warning when invoked without KERNELVERSION in the environment, rather make it use the string "unknown kernel version" instead.

---
 scripts/kernel-doc |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kernel-doc +1 -1
@@ -334,7 +334,7 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 
 # get kernel version from env
 sub get_kernel_version() {
-    my $version;
+    my $version = 'unknown kernel version';
 
     if (defined($ENV{'KERNELVERSION'})) {
 	$version = $ENV{'KERNELVERSION'};
#6 lore raw

PATCH 6/6 kernel-doc: new P directive for DOC: sections

In reply to Randy Dunlap

From: Johannes Berg <[email protected]>

The !P directive includes the contents of a DOC: section given by title, e.g.
!Pfilename Title of the section

---
 scripts/basic/docproc.c |   40 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)
scripts/basic/docproc.c +39 -1
@@ -30,6 +30,7 @@
  *		!Ifilename
  *		!Dfilename
  *		!Ffilename
+ *		!Pfilename
  *
  */
 
@@ -57,6 +58,7 @@ FILEONLY *symbolsonly;
 typedef void FILELINE(char * file, char * line);
 FILELINE * singlefunctions;
 FILELINE * entity_system;
+FILELINE * docsection;
 
 #define MAXLINESZ     2048
 #define MAXFILES      250
@@ -289,12 +291,36 @@ void singfunc(char * filename, char * li
 }
 
 /*
+ * Insert specific documentation section from a file.
+ * Call kernel-doc with the following parameters:
+ * kernel-doc -docbook -function "doc section" filename
+ */
+void docsect(char *filename, char *line)
+{
+	char *vec[6]; /* kerneldoc -docbook -function "section" file NULL */
+	char *s;
+
+	for (s = line; *s; s++)
+		if (*s == '\n')
+			*s = '\0';
+
+	vec[0] = KERNELDOC;
+	vec[1] = DOCBOOK;
+	vec[2] = FUNCTION;
+	vec[3] = line;
+	vec[4] = filename;
+	vec[5] = NULL;
+	exec_kernel_doc(vec);
+}
+
+/*
  * Parse file, calling action specific functions for:
  * 1) Lines containing !E
  * 2) Lines containing !I
  * 3) Lines containing !D
  * 4) Lines containing !F
- * 5) Default lines - lines not matching the above
+ * 5) Lines containing !P
+ * 6) Default lines - lines not matching the above
  */
 void parse_file(FILE *infile)
 {
@@ -328,6 +354,15 @@ void parse_file(FILE *infile)
 						s++;
 					singlefunctions(line +2, s);
 					break;
+				case 'P':
+					/* filename */
+					while (*s && !isspace(*s)) s++;
+					*s++ = '\0';
+					/* DOC: section name */
+					while (isspace(*s))
+						s++;
+					docsection(line +2, s);
+					break;
 				default:
 					defaultline(line);
 			}
@@ -374,6 +409,7 @@ int main(int argc, char *argv[])
 		externalfunctions = find_export_symbols;
 		symbolsonly       = find_export_symbols;
 		singlefunctions   = noaction2;
+		docsection        = noaction2;
 		parse_file(infile);
 
 		/* Rewind to start from beginning of file again */
@@ -383,6 +419,7 @@ int main(int argc, char *argv[])
 		externalfunctions = extfunc;
 		symbolsonly       = printline;
 		singlefunctions   = singfunc;
+		docsection        = docsect;
 
 		parse_file(infile);
 	}
@@ -396,6 +433,7 @@ int main(int argc, char *argv[])
 		externalfunctions = adddep;
 		symbolsonly       = adddep;
 		singlefunctions   = adddep2;
+		docsection        = adddep2;
 		parse_file(infile);
 		printf("\n");
 	}
#7 lore raw
In reply to Randy Dunlap
Quoted 3 lines · Randy Dunlap

This patch series from Johannes Berg adds support for DOC: sections that are embedded in source files.

What do these look like?

Kerneldoc's documentation is sadly lacking..

Signature · 1 line
Mathematics is the supreme nostalgia of our time.
#8 lore raw
Quoted 6 lines · On Thu, 25 Oct 2007 12:29:32 -0500 Matt Mackall wrote:
Quoted 3 lines · Randy Dunlap

This patch series from Johannes Berg adds support for DOC: sections that are embedded in source files.

What do these look like?

There are currently 2 files in the kernel tree that contain DOC:
sections. These are not used in the docbooks, but you can see what they look like here:

./drivers/net/3c501.c: * DOC: 3c501 Card Notes ./drivers/net/3c501.c: * DOC: Problems ./drivers/net/3c527.c: * DOC: Traps for the unwary ./drivers/net/3c527.c: * DOC: Theory Of Operation ./drivers/net/3c527.c: * DOC: Notes

Also, Johannes has patches to use DOC: in mac80211 files.

Quoted 1 line

Kerneldoc's documentation is sadly lacking..

point taken.

---
~Randy

#9 lore raw
Quoted 13 lines · Randy Dunlap
Quoted 6 lines · On Thu, 25 Oct 2007 12:29:32 -0500 Matt Mackall wrote:
Quoted 3 lines · Randy Dunlap

This patch series from Johannes Berg adds support for DOC: sections that are embedded in source files.

What do these look like?

There are currently 2 files in the kernel tree that contain DOC:
sections. These are not used in the docbooks, but you can see what they look like here:

Quoted 5 lines

./drivers/net/3c501.c: * DOC: 3c501 Card Notes ./drivers/net/3c501.c: * DOC: Problems ./drivers/net/3c527.c: * DOC: Traps for the unwary ./drivers/net/3c527.c: * DOC: Theory Of Operation ./drivers/net/3c527.c: * DOC: Notes

Aside from those being about the least interesting drivers in the tree, that's pretty neat.

The docbook infrastructure probably needs to figure out a way to pick up this stuff automatically if it's not otherwise tied in.

With Python, I can do pydoc foo.py or pydoc foo/ and immediately get browsable extracted docs in $PAGER for the associated file or directory. Would be nice to get a little closer to that here.

Signature · 1 line
Mathematics is the supreme nostalgia of our time.
#10 lore raw
Quoted 8 lines
Quoted 2 lines

./drivers/net/3c527.c: * DOC: Notes

Aside from those being about the least interesting drivers in the tree, that's pretty neat.

The docbook infrastructure probably needs to figure out a way to pick up this stuff automatically if it's not otherwise tied in.

The 5th and 6th patch in this series do that by allowing you to write

!Pfilename section title

in a docbook template file to insert the contents of such a section. It's what I'm doing for mac80211's docs so that browsing the files gives you the same documentation as reading the mac80211 docbook book just organised differently.

johannes

#11 lore raw
Quoted 9 lines · Johannes Berg
Quoted 8 lines
Quoted 2 lines

./drivers/net/3c527.c: * DOC: Notes

Aside from those being about the least interesting drivers in the tree, that's pretty neat.

The docbook infrastructure probably needs to figure out a way to pick up this stuff automatically if it's not otherwise tied in.

Quoted 8 lines

The 5th and 6th patch in this series do that by allowing you to write

!Pfilename section title

in a docbook template file to insert the contents of such a section. It's what I'm doing for mac80211's docs so that browsing the files gives you the same documentation as reading the mac80211 docbook book just organised differently.

There are lots and lots of files that currently contain kerneldoc comments that are unreferenced by docbook (and have been for years). We should somehow sweep them all up into some sort of bucket automatically.

Signature · 1 line
Mathematics is the supreme nostalgia of our time.
#13 lore raw
In reply to Randy Dunlap
Quoted 3 lines · Randy Dunlap

This patch series from Johannes Berg adds support for DOC: sections that are embedded in source files.

Thanks - added to kbuild.git after a few small fix-ups. [Whitespace + the last bit did not apply].

Sam

#12 lore raw
In reply to Matt Mackall
Quoted 5 lines

There are lots and lots of files that currently contain kerneldoc comments that are unreferenced by docbook (and have been for years). We should somehow sweep them all up into some sort of bucket automatically.

That'd be another thing, but yes, it'd be nice as well. Not something I'm aiming for right now though.

johannes