Skip to contentSkip to conversation

kernel-doc improvements

PATCH Applied Mauro Carvalho Chehab 2 participants 15 messages Mar 6 – Mar 9

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover kernel-doc improvements

Hi Jon,

This patch kernel-doc improvements. On v2, I'm keeping just the patches related to man pages generation.

The first patches on this series are focused mostly on .TH (troff header) line, but, as a side effect, it also change the name of man pages generated from DOC kernel-doc annotations. At the previous state, those were overriden due to lots of duplicated names.

The rationale for most of such changes is that modern troff/man page specs say that .TH has up to 5 arguments,, as defined at [1]:

.TH topic section [footer-middle] [footer-inside] [header-middle]

[1] https://man7.org/linux/man-pages/man7/groff_man_style.7.html

Right now, Kernel uses 6 arguments, probably due to some legacy man page definitions.

After double checking, modern man pages use this format:

	.TH "{name}" {section} "{date}" "{modulename}" "{manual}"

Right now, man pages generation are messing up on how it encodes each position at .TH, depending on the type of object it emits.

After this series, the definition is more consistent and file output is better named.

It also fixes two issues at sphinx-build-wrapper related to how it generate files names from the .TH header.

The last 4 patches on this series are new: they fix lots of issues related to groff format: there, new lines continue the test from previous pagragraph. This cause issues mainly on:

- tables;
- code blocks;
- lists

With the changes, the output now looks a lot better.

Please notice that the code there is not meant to fully implement rst -> troff/groff conversion. Instead, it is meant to make the output reasonable.

A more complete approach would be to use docutils or Sphinx libraries, but that would likely require to also write a troff output plugin, as the "man" builder is very limited. Also, this could be problematic, as kernel-doc classes can be called from Sphinx. I don't think we need that much complexity, as what we mainly need is to avoid bad line grouping when generating man pages.

This series should not affect HTML documentation. It only affect man page generation and ManFormat output class.

Mauro Carvalho Chehab (13):
docs: sphinx-build-wrapper: better handle troff .TH markups
docs: sphinx-build-wrapper: don't allow "/" on file names
docs: kdoc_output: use a method to emit the .TH header
docs: kdoc_output: remove extra attribute on man .TH headers
docs: kdoc_output: use a single manual for everything
docs: kdoc_output: don't use a different modulename for functions
docs: kdoc_output: fix naming for DOC markups
docs: kdoc_output: describe the class init parameters
docs: kdoc_output: pick a better default for modulename
docs: kdoc_output: Change the logic to handle man highlight
docs: kdoc_output: add a logic to handle tables inside kernel-doc
markups
docs: kdoc_output: add support to handle code blocks
docs: kdoc_output: better handle lists

 tools/docs/kernel-doc                |   1 -
 tools/docs/sphinx-build-wrapper      |  10 +-
 tools/lib/python/kdoc/kdoc_output.py | 297 +++++++++++++++++++++++++--
 3 files changed, 283 insertions(+), 25 deletions(-)
Signature · 1 line
2.52.0
#1 lore raw

PATCH v2 1/13 docs: sphinx-build-wrapper: better handle troff .TH markups

Using a regular expression to match .TH is problematic, as it doesn't handle well quotation marks.

Use shlex instead.

---
 tools/docs/sphinx-build-wrapper | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tools/docs/sphinx-build-wrapper +4 -4
index b7c149dff06b..e6418e22e2ff 100755
@@ -576,7 +576,6 @@ class SphinxBuilder:
         """
 
         re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)")
-        re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"')
 
         if docs_dir == src_dir:
             #
@@ -616,8 +615,7 @@ class SphinxBuilder:
         fp = None
         try:
             for line in result.stdout.split("\n"):
-                match = re_man.match(line)
-                if not match:
+                if not line.startswith(".TH"):
                     if fp:
                         fp.write(line + '\n')
                     continue
@@ -625,7 +623,9 @@ class SphinxBuilder:
                 if fp:
                     fp.close()
 
-                fname = f"{output_dir}/{match.group(2)}.{match.group(1)}"
+                # Use shlex here, as it handles well parameters with commas
+                args = shlex.split(line)
+                fname = f"{output_dir}/{args[3]}.{args[2]}"
 
                 if self.verbose:
                     print(f"Creating {fname}")
Signature · 1 line
2.52.0
#2 lore raw

PATCH v2 2/13 docs: sphinx-build-wrapper: don't allow "/" on file names

In reply to Mauro Carvalho Chehab

When handling "DOC:" sections, slash characters may be there. Prevent using it at the file names, as this is used for directory separator.

---
 tools/docs/sphinx-build-wrapper | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/docs/sphinx-build-wrapper +3 -1
index e6418e22e2ff..d3f0dba13da1 100755
@@ -625,7 +625,9 @@ class SphinxBuilder:
 
                 # Use shlex here, as it handles well parameters with commas
                 args = shlex.split(line)
-                fname = f"{output_dir}/{args[3]}.{args[2]}"
+                fname = f"{args[3]}.{args[2]}"
+                fname = fname.replace("/", " ")
+                fname = f"{output_dir}/{fname}"
 
                 if self.verbose:
                     print(f"Creating {fname}")
Signature · 1 line
2.52.0
#3 lore raw

PATCH v2 3/13 docs: kdoc_output: use a method to emit the .TH header

In reply to Mauro Carvalho Chehab

All man emit functions need to add a .TH header. Move the code to a common function, as we'll be addressing some issues at the common code.

---
 tools/lib/python/kdoc/kdoc_output.py | 34 +++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 8 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +26 -8
index 4210b91dde5f..fb6b90c54c8a 100644
@@ -607,7 +607,20 @@ class ManFormat(OutputFormat):
         "%m %d %Y",
     ]
 
-    def __init__(self, modulename):
+    def emit_th(self, name, modulename = None, manual=None):
+        """Emit a title header line."""
+        name = name.strip()
+
+        if not manual:
+            manual = self.manual
+
+        if not modulename:
+            modulename = self.modulename
+
+        self.data += f'.TH "{modulename}" {self.section} "{name}" '
+        self.data += f'"{self.date}" "{manual}" LINUX\n'
+
+    def __init__(self, modulename, section="9", manual="API Manual"):
         """
         Creates class variables.
 
@@ -616,7 +629,11 @@ class ManFormat(OutputFormat):
         """
 
         super().__init__()
+
         self.modulename = modulename
+        self.section = section
+        self.manual = manual
+
         self.symbols = []
 
         dt = None
@@ -632,7 +649,7 @@ class ManFormat(OutputFormat):
         if not dt:
             dt = datetime.now()
 
-        self.man_date = dt.strftime("%B %Y")
+        self.date = dt.strftime("%B %Y")
 
     def arg_name(self, args, name):
         """
@@ -724,7 +741,7 @@ class ManFormat(OutputFormat):
 
         out_name = self.arg_name(args, name)
 
-        self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.emit_th(out_name)
 
         for section, text in args.sections.items():
             self.data += f'.SH "{section}"' + "\n"
@@ -734,7 +751,8 @@ class ManFormat(OutputFormat):
 
         out_name = self.arg_name(args, name)
 
-        self.data += f'.TH "{name}" 9 "{out_name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
+        self.emit_th(out_name, modulename = name,
+                     manual="Kernel Hacker\'s Manual")
 
         self.data += ".SH NAME\n"
         self.data += f"{name} \\- {args['purpose']}\n"
@@ -780,7 +798,7 @@ class ManFormat(OutputFormat):
     def out_enum(self, fname, name, args):
         out_name = self.arg_name(args, name)
 
-        self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.emit_th(out_name)
 
         self.data += ".SH NAME\n"
         self.data += f"enum {name} \\- {args['purpose']}\n"
@@ -813,7 +831,7 @@ class ManFormat(OutputFormat):
         out_name = self.arg_name(args, name)
         full_proto = args.other_stuff["full_proto"]
 
-        self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.emit_th(out_name)
 
         self.data += ".SH NAME\n"
         self.data += f"{name} \\- {args['purpose']}\n"
@@ -834,7 +852,7 @@ class ManFormat(OutputFormat):
         purpose = args.get('purpose')
         out_name = self.arg_name(args, name)
 
-        self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.emit_th(out_name)
 
         self.data += ".SH NAME\n"
         self.data += f"typedef {name} \\- {purpose}\n"
@@ -849,7 +867,7 @@ class ManFormat(OutputFormat):
         definition = args.get('definition')
         out_name = self.arg_name(args, name)
 
-        self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.emit_th(out_name)
 
         self.data += ".SH NAME\n"
         self.data += f"{args.type} {name} \\- {purpose}\n"
Signature · 1 line
2.52.0
#4 lore raw

PATCH v2 4/13 docs: kdoc_output: remove extra attribute on man .TH headers

In reply to Mauro Carvalho Chehab

According with modern documents, groff .TH supports up to 5 arguments, but the logic passes 6. Drop the lastest one ("LINUX").

---
 tools/lib/python/kdoc/kdoc_output.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_output.py +1 -1
index fb6b90c54c8a..d0b237c09391 100644
@@ -618,7 +618,7 @@ class ManFormat(OutputFormat):
             modulename = self.modulename
 
         self.data += f'.TH "{modulename}" {self.section} "{name}" '
-        self.data += f'"{self.date}" "{manual}" LINUX\n'
+        self.data += f'"{self.date}" "{manual}"\n'
 
     def __init__(self, modulename, section="9", manual="API Manual"):
         """
Signature · 1 line
2.52.0
#5 lore raw

PATCH v2 5/13 docs: kdoc_output: use a single manual for everything

In reply to Mauro Carvalho Chehab

There's no reason why functions will be on a different manual. Unify its name, calling it as "Kernel API Manual".

---
 tools/lib/python/kdoc/kdoc_output.py | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +4 -8
index d0b237c09391..24ee1fad681e 100644
@@ -607,20 +607,17 @@ class ManFormat(OutputFormat):
         "%m %d %Y",
     ]
 
-    def emit_th(self, name, modulename = None, manual=None):
+    def emit_th(self, name, modulename = None):
         """Emit a title header line."""
         name = name.strip()
 
-        if not manual:
-            manual = self.manual
-
         if not modulename:
             modulename = self.modulename
 
         self.data += f'.TH "{modulename}" {self.section} "{name}" '
-        self.data += f'"{self.date}" "{manual}"\n'
+        self.data += f'"{self.date}" "{self.manual}"\n'
 
-    def __init__(self, modulename, section="9", manual="API Manual"):
+    def __init__(self, modulename, section="9", manual="Kernel API Manual"):
         """
         Creates class variables.
 
@@ -751,8 +748,7 @@ class ManFormat(OutputFormat):
 
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name, modulename = name,
-                     manual="Kernel Hacker\'s Manual")
+        self.emit_th(out_name, modulename = name)
 
         self.data += ".SH NAME\n"
         self.data += f"{name} \\- {args['purpose']}\n"
Signature · 1 line
2.52.0
#6 lore raw

PATCH v2 6/13 docs: kdoc_output: don't use a different modulename for functions

In reply to Mauro Carvalho Chehab

It doesn't make much sense to have a different modulename just for functions, but not for structs/enums/...

Use the same header everywere.

---
 tools/lib/python/kdoc/kdoc_output.py | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +3 -6
index 24ee1fad681e..62e300e65405 100644
@@ -607,14 +607,11 @@ class ManFormat(OutputFormat):
         "%m %d %Y",
     ]
 
-    def emit_th(self, name, modulename = None):
+    def emit_th(self, name):
         """Emit a title header line."""
         name = name.strip()
 
-        if not modulename:
-            modulename = self.modulename
-
-        self.data += f'.TH "{modulename}" {self.section} "{name}" '
+        self.data += f'.TH "{self.modulename}" {self.section} "{name}" '
         self.data += f'"{self.date}" "{self.manual}"\n'
 
     def __init__(self, modulename, section="9", manual="Kernel API Manual"):
@@ -748,7 +745,7 @@ class ManFormat(OutputFormat):
 
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name, modulename = name)
+        self.emit_th(out_name)
 
         self.data += ".SH NAME\n"
         self.data += f"{name} \\- {args['purpose']}\n"
Signature · 1 line
2.52.0
#7 lore raw

PATCH v2 7/13 docs: kdoc_output: fix naming for DOC markups

In reply to Mauro Carvalho Chehab

Right now, DOC markups aren't being handled properly, as it was using the same name for all output.

Fix it by filling the title argument on a different way.

---
 tools/docs/sphinx-build-wrapper      |  2 +-
 tools/lib/python/kdoc/kdoc_output.py | 38 +++++++++++++++++-----------
 2 files changed, 24 insertions(+), 16 deletions(-)
tools/docs/sphinx-build-wrapper +1 -1
index d3f0dba13da1..2c63d28f639d 100755
@@ -625,7 +625,7 @@ class SphinxBuilder:
 
                 # Use shlex here, as it handles well parameters with commas
                 args = shlex.split(line)
-                fname = f"{args[3]}.{args[2]}"
+                fname = f"{args[1]}.{args[2]}"
                 fname = fname.replace("/", " ")
                 fname = f"{output_dir}/{fname}"
 
tools/lib/python/kdoc/kdoc_output.py +23 -15
index 62e300e65405..cf834dbf2725 100644
@@ -607,14 +607,21 @@ class ManFormat(OutputFormat):
         "%m %d %Y",
     ]
 
-    def emit_th(self, name):
+    def modulename(self, args):
+        if self._modulename:
+            return self._modulename
+
+        return os.path.dirname(args.fname)
+
+    def emit_th(self, name, args):
         """Emit a title header line."""
-        name = name.strip()
+        title = name.strip()
+        module = self.modulename(args)
 
-        self.data += f'.TH "{self.modulename}" {self.section} "{name}" '
-        self.data += f'"{self.date}" "{self.manual}"\n'
+        self.data += f'.TH "{name}" {self.section} "{self.date}" '
+        self.data += f'"{self.modulename}" "{self.manual}"\n'
 
-    def __init__(self, modulename, section="9", manual="Kernel API Manual"):
+    def __init__(self, modulename=None, section="9", manual="Kernel API Manual"):
         """
         Creates class variables.
 
@@ -624,7 +631,7 @@ class ManFormat(OutputFormat):
 
         super().__init__()
 
-        self.modulename = modulename
+        self._modulename = modulename
         self.section = section
         self.manual = manual
 
@@ -658,7 +665,8 @@ class ManFormat(OutputFormat):
         dtype = args.type
 
         if dtype == "doc":
-            return self.modulename
+            return name
+#            return os.path.basename(self.modulename(args))
 
         if dtype in ["function", "typedef"]:
             return name
@@ -735,7 +743,7 @@ class ManFormat(OutputFormat):
 
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name)
+        self.emit_th(out_name, args)
 
         for section, text in args.sections.items():
             self.data += f'.SH "{section}"' + "\n"
@@ -745,7 +753,7 @@ class ManFormat(OutputFormat):
 
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name)
+        self.emit_th(out_name, args)
 
         self.data += ".SH NAME\n"
         self.data += f"{name} \\- {args['purpose']}\n"
@@ -791,7 +799,7 @@ class ManFormat(OutputFormat):
     def out_enum(self, fname, name, args):
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name)
+        self.emit_th(out_name, args)
 
         self.data += ".SH NAME\n"
         self.data += f"enum {name} \\- {args['purpose']}\n"
@@ -824,7 +832,7 @@ class ManFormat(OutputFormat):
         out_name = self.arg_name(args, name)
         full_proto = args.other_stuff["full_proto"]
 
-        self.emit_th(out_name)
+        self.emit_th(out_name, args)
 
         self.data += ".SH NAME\n"
         self.data += f"{name} \\- {args['purpose']}\n"
@@ -841,11 +849,11 @@ class ManFormat(OutputFormat):
             self.output_highlight(text)
 
     def out_typedef(self, fname, name, args):
-        module = self.modulename
+        module = self.modulename(args)
         purpose = args.get('purpose')
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name)
+        self.emit_th(out_name, args)
 
         self.data += ".SH NAME\n"
         self.data += f"typedef {name} \\- {purpose}\n"
@@ -855,12 +863,12 @@ class ManFormat(OutputFormat):
             self.output_highlight(text)
 
     def out_struct(self, fname, name, args):
-        module = self.modulename
+        module = self.modulename(args)
         purpose = args.get('purpose')
         definition = args.get('definition')
         out_name = self.arg_name(args, name)
 
-        self.emit_th(out_name)
+        self.emit_th(out_name, args)
 
         self.data += ".SH NAME\n"
         self.data += f"{args.type} {name} \\- {purpose}\n"
Signature · 1 line
2.52.0
#8 lore raw

PATCH v2 8/13 docs: kdoc_output: describe the class init parameters

In reply to Mauro Carvalho Chehab

As this class is part of the ABI used by both Sphinx kerneldoc extension and docs/tools/kernel-doc, better describe what parmeters are used to initialize ManOutput class.

---
 tools/lib/python/kdoc/kdoc_output.py | 29 +++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_output.py +28 -1
index cf834dbf2725..7a181b40810d 100644
@@ -580,7 +580,34 @@ class RestFormat(OutputFormat):
 
 
 class ManFormat(OutputFormat):
-    """Consts and functions used by man pages output."""
+    """
+    Consts and functions used by man pages output.
+
+    This class has one mandatory parameter and some optional ones, which
+    are needed to define the title header contents:
+
+    ``modulename``
+        Defines the module name to be used at the troff ``.TH`` output.
+
+        This argument is mandatory.
+
+    ``section``
+        Usually a numeric value from 0 to 9, but man pages also accept
+        some strings like "p".
+
+        Defauls to ``9``
+
+    ``manual``
+        Defaults to ``Kernel API Manual``.
+
+    The above controls the output of teh corresponding fields on troff
+    title headers, which will be filled like this::
+
+        .TH "{name}" {section} "{date}" "{modulename}" "{manual}"
+
+    where ``name``` will match the API symbol name, and ``date`` will be
+    either the date where the Kernel was compiled or the current date
+    """
 
     highlights = (
         (type_constant, r"\1"),
Signature · 1 line
2.52.0
#9 lore raw

PATCH v2 9/13 docs: kdoc_output: pick a better default for modulename

In reply to Mauro Carvalho Chehab

Instead of placing the same data for modulename for all generated man pages, use the directory from the filename used to produce kernel docs as basis.

---
 tools/docs/kernel-doc                | 1 -
 tools/lib/python/kdoc/kdoc_output.py | 7 ++++---
 2 files changed, 4 insertions(+), 4 deletions(-)
tools/docs/kernel-doc +0 -1
index aed09f9a54dd..3a932f95bdf5 100755
@@ -210,7 +210,6 @@ def main():
                         help="Enable debug messages")
 
     parser.add_argument("-M", "-modulename", "--modulename",
-                        default="Kernel API",
                         help="Allow setting a module name at the output.")
 
     parser.add_argument("-l", "-enable-lineno", "--enable_lineno",
tools/lib/python/kdoc/kdoc_output.py +4 -3
index 7a181b40810d..c25f80a39cdc 100644
@@ -589,7 +589,8 @@ class ManFormat(OutputFormat):
     ``modulename``
         Defines the module name to be used at the troff ``.TH`` output.
 
-        This argument is mandatory.
+        This argument is optional. If not specified, it will be filled
+        with the directory which contains the documented file.
 
     ``section``
         Usually a numeric value from 0 to 9, but man pages also accept
@@ -645,8 +646,8 @@ class ManFormat(OutputFormat):
         title = name.strip()
         module = self.modulename(args)
 
-        self.data += f'.TH "{name}" {self.section} "{self.date}" '
-        self.data += f'"{self.modulename}" "{self.manual}"\n'
+        self.data += f'.TH "{title}" {self.section} "{self.date}" '
+        self.data += f'"{module}" "{self.manual}"\n'
 
     def __init__(self, modulename=None, section="9", manual="Kernel API Manual"):
         """
Signature · 1 line
2.52.0
#10 lore raw

PATCH v2 10/13 docs: kdoc_output: Change the logic to handle man highlight

In reply to Mauro Carvalho Chehab

The code inside ManFormat code to output man pages is too simple:
it produces very bad results when the content has tables or code blocks.

Change the way lines are parsed there to allow adding extra logic to handle some special cases.

---
 tools/lib/python/kdoc/kdoc_output.py | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +17 -9
index c25f80a39cdc..9caffe0d9753 100644
@@ -755,15 +755,23 @@ class ManFormat(OutputFormat):
         if isinstance(contents, list):
             contents = "\n".join(contents)
 
-        for line in contents.strip("\n").split("\n"):
-            line = KernRe(r"^\s*").sub("", line)
-            if not line:
-                continue
-
-            if line[0] == ".":
-                self.data += "\\&" + line + "\n"
-            else:
-                self.data += line + "\n"
+        lines = contents.strip("\n").split("\n")
+        i = 0
+
+        while i < len(lines):
+            org_line = lines[i]
+
+            line = KernRe(r"^\s*").sub("", org_line)
+
+            if line:
+                if line[0] == ".":
+                    self.data += "\\&" + line + "\n"
+                    i += 1
+                    continue
+
+            i += 1
+
+            self.data += line + "\n"
 
     def out_doc(self, fname, name, args):
         if not self.check_doc(name, args):
Signature · 1 line
2.52.0
#11 lore raw

PATCH v2 11/13 docs: kdoc_output: add a logic to handle tables inside kernel-doc markups

In reply to Mauro Carvalho Chehab

specially when DOC is used, it is not uncommon to have tables inside a kernel-doc markup.

Add support for simple tables and complex grid tables when output in groff format.

---
 tools/lib/python/kdoc/kdoc_output.py | 130 +++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)
tools/lib/python/kdoc/kdoc_output.py +130 -0
index 9caffe0d9753..7848514a4d22 100644
@@ -744,6 +744,126 @@ class ManFormat(OutputFormat):
 
         return self.data
 
+    def emit_table(self, colspec_row, rows):
+
+        if not rows:
+            return ""
+
+        out = ""
+        colspec = "\t".join(["l"] * len(rows[0]))
+
+        out += "\n.TS\n"
+        out += "box;\n"
+        out += f"{colspec}.\n"
+
+        if colspec_row:
+            out_row = []
+
+            for text in colspec_row:
+                out_row.append(f"\\fB{text}\\fP")
+
+            out += "\t".join(out_row) + "\n_\n"
+
+        for r in rows:
+            out += "\t".join(r) + "\n"
+
+        out += ".TE\n"
+
+        return out
+
+    def grid_table(self, lines, start):
+        """
+        Ancillary function to help handling a grid table inside the text.
+        """
+
+        i = start + 1
+        rows = []
+        colspec_row = None
+
+        while i < len(lines):
+            line = lines[i]
+
+            if KernRe(r"^\s*\|.*\|\s*$").match(line):
+                parts = []
+
+                for p in line.strip('|').split('|'):
+                    parts.append(p.strip())
+
+                rows.append(parts)
+
+            elif KernRe(r'^\+\=[\+\=]+\+\s*$').match(line):
+                if rows and rows[0]:
+                    if not colspec_row:
+                        colspec_row = [""] * len(rows[0])
+
+                    for j in range(0, len(rows[0])):
+                        content = []
+                        for row in rows:
+                            content.append(row[j])
+
+                        colspec_row[j] = " ".join(content)
+
+                    rows = []
+
+            elif KernRe(r"^\s*\+[-+]+\+.*$").match(line):
+                pass
+
+            else:
+                break
+
+            i += 1
+
+        return i, self.emit_table(colspec_row, rows)
+
+    def simple_table(self, lines, start):
+        """
+        Ancillary function to help handling a simple table inside the text.
+        """
+
+        i = start
+        rows = []
+        colspec_row = None
+
+        pos = []
+        for m in KernRe(r'\-+').finditer(lines[i]):
+            pos.append((m.start(), m.end() - 1))
+
+        i += 1
+        while i < len(lines):
+            line = lines[i]
+
+            if KernRe(r"^\s*[\-]+[ \t\-]+$").match(line):
+                i += 1
+                break
+
+            elif KernRe(r'^[\s=]+$').match(line):
+                if rows and rows[0]:
+                    if not colspec_row:
+                        colspec_row = [""] * len(rows[0])
+
+                    for j in range(0, len(rows[0])):
+                        content = []
+                        for row in rows:
+                            content.append(row[j])
+
+                        colspec_row[j] = " ".join(content)
+
+                    rows = []
+
+            else:
+                row = [""] * len(pos)
+
+                for j in range(0, len(pos)):
+                    start, end = pos[j]
+
+                    row[j] = line[start:end].strip()
+
+                rows.append(row)
+
+            i += 1
+
+        return i, self.emit_table(colspec_row, rows)
+
     def output_highlight(self, block):
         """
         Outputs a C symbol that may require being highlighted with
@@ -764,6 +884,16 @@ class ManFormat(OutputFormat):
             line = KernRe(r"^\s*").sub("", org_line)
 
             if line:
+                if KernRe(r"^\+\-[-+]+\+.*$").match(line):
+                    i, text = self.grid_table(lines, i)
+                    self.data += text
+                    continue
+
+                if KernRe(r"^\-+[ \t]\-[ \t\-]+$").match(line):
+                    i, text = self.simple_table(lines, i)
+                    self.data += text
+                    continue
+
                 if line[0] == ".":
                     self.data += "\\&" + line + "\n"
                     i += 1
Signature · 1 line
2.52.0
#12 lore raw

PATCH v2 12/13 docs: kdoc_output: add support to handle code blocks

In reply to Mauro Carvalho Chehab

It is common to have code blocks inside kernel-doc markups. By default, troff will group all lines altogether, producing a very weird output. If a code block is detected by disabling filling inside code blocks, re-enabling it afterwards.

---
 tools/lib/python/kdoc/kdoc_output.py | 64 ++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
tools/lib/python/kdoc/kdoc_output.py +64 -0
index 7848514a4d22..df9af444da57 100644
@@ -864,6 +864,65 @@ class ManFormat(OutputFormat):
 
         return i, self.emit_table(colspec_row, rows)
 
+    def code_block(self, lines, start):
+        """
+        Ensure that code blocks won't be messed up at the output.
+
+        By default, troff join lines at the same paragraph. Disable it,
+        on code blocks.
+        """
+
+        line = lines[start]
+
+        if "code-block" in line:
+            out = "\n.nf\n"
+        elif line.startswith("..") and line.endswith("::"):
+            #
+            # Handle note, warning, error, ... markups
+            #
+            line = line[2:-1].strip().upper()
+            out = f"\n.nf\n\\fB{line}\\fP\n"
+        elif line.endswith("::"):
+            out = line[:-1]
+            out += "\n.nf\n"
+        else:
+            # Just in case. Should never happen in practice
+            out = "\n.nf\n"
+
+        i = start + 1
+        ident = None
+
+        while i < len(lines):
+            line = lines[i]
+
+            m = KernRe(r"\S").match(line)
+            if not m:
+                out += line + "\n"
+                i += 1
+                continue
+
+            pos = m.start()
+            if not ident:
+                if pos > 0:
+                    ident = pos
+                else:
+                    out += "\n.fi\n"
+                    if i > start + 1:
+                        return i - 1, out
+                    else:
+                        # Just in case. Should never happen in practice
+                        return i, out
+
+            if pos >= ident:
+                out += line + "\n"
+                i += 1
+                continue
+
+            break
+
+        out += "\n.fi\n"
+        return i, out
+
     def output_highlight(self, block):
         """
         Outputs a C symbol that may require being highlighted with
@@ -894,6 +953,11 @@ class ManFormat(OutputFormat):
                     self.data += text
                     continue
 
+                if line.endswith("::") or KernRe(r"\.\.\s+code-block.*::").match(line):
+                    i, text = self.code_block(lines, i)
+                    self.data += text
+                    continue
+
                 if line[0] == ".":
                     self.data += "\\&" + line + "\n"
                     i += 1
Signature · 1 line
2.52.0
#13 lore raw

PATCH v2 13/13 docs: kdoc_output: better handle lists

In reply to Mauro Carvalho Chehab

On several functions, the return values are inside a bullet list. Also, on some places, there are numbered lists as well.

Use a troff markup to format them, to avoid placing everything on a single line.

---
 tools/lib/python/kdoc/kdoc_output.py | 8 ++++++++
 1 file changed, 8 insertions(+)
tools/lib/python/kdoc/kdoc_output.py +8 -0
index df9af444da57..08539dd92cbb 100644
@@ -963,6 +963,14 @@ class ManFormat(OutputFormat):
                     i += 1
                     continue
 
+                #
+                # Handle lists
+                #
+                line = KernRe(r'^[-*]\s+').sub(r'.IP \[bu]\n', line)
+                line = KernRe(r'^(\d+|a-z)[\.\)]\s+').sub(r'.IP \1\n', line)
+            else:
+                line = ".PP\n"
+
             i += 1
 
             self.data += line + "\n"
Signature · 1 line
2.52.0