Skip to contentSkip to conversation

additional fixes to kernel-doc.py

PATCH In review Mauro Carvalho Chehab 2 participants 8 messages Feb 19, 2025 – Feb 21, 2025 Documentation

lore
Share & tools
lore

Mauro Carvalho Chehab

author
#0 lore raw

cover additional fixes to kernel-doc.py

Hi Jon,

This series comes after:

https://lore.kernel.org/linux-doc/[email protected]/T/#t

it basically contains:

- a regression fix when -none/--none parameter is used;
- some backward-compatibility fixes;
- a fix to KBUILD_BUILD_TIMESTAMP, which also drops importing dateutil (*);
- a cleanup related to man modulename.

(*) despite almost everybody has it installed, dateutil is actually not
part of the official Python libraries, so using it would require an extra dependency check.

While our minimal Python version is 3.9, I opted to make it compatible with Python 3.6, as the only issue preventing 3.6 was a single string function (removesuffix) that can be also served by rstrip().

After those changes, support for 3.6 on kernel-doc.py comes almost for
free: both man and ReST output works with just Python 3.6 installed without any additional modules.

So, no need to change any dependencies at scripts/sphinx-pre-install.

Mauro Carvalho Chehab (6):
scripts/kernel-doc.py: Set an output format for --none scripts/kernel-doc.py: adjust some coding style issues scripts/lib/kdoc/kdoc_parser.py: fix Python compat with < v3.13 scripts/kernel-doc.py: move modulename to man class scripts/kernel-doc.py: properly handle KBUILD_BUILD_TIMESTAMP scripts/lib/kdoc/kdoc_parser.py: remove a python 3.9 dependency

 scripts/kernel-doc.py           |  8 ++-
 scripts/lib/kdoc/kdoc_files.py  | 19 ++-----
 scripts/lib/kdoc/kdoc_output.py | 97 ++++++++++++++++++++++++---------
 scripts/lib/kdoc/kdoc_parser.py | 15 ++---
 4 files changed, 83 insertions(+), 56 deletions(-)
Signature · 1 line
2.48.1
author
#1 lore raw

PATCH 1/6 scripts/kernel-doc.py: Set an output format for --none

Now that warnings output is deferred to the output plugin, we need to have an output style for none as well.

So, use the OutputFormat base class on such cases.

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 scripts/lib/kdoc/kdoc_files.py | 4 ++++
 1 file changed, 4 insertions(+)
scripts/lib/kdoc/kdoc_files.py +4 -0
index c215ae3047b8..957aaeaed7e6 100755
@@ -19,6 +19,7 @@ from datetime import datetime
 from dateutil import tz
 
 from kdoc_parser import KernelDoc
+from kdoc_output import OutputFormat
 
 
 class GlobSourceFiles:

@@ -137,6 +138,9 @@ class KernelFiles():
         if not modulename:
             modulename = "Kernel API"
 
+        if out_style is None:
+            out_style = OutputFormat()
+
         dt = datetime.now()
         if os.environ.get("KBUILD_BUILD_TIMESTAMP", None):
             # use UTC TZ
Signature · 1 line
2.48.1
author
#2 lore raw

PATCH 2/6 scripts/kernel-doc.py: adjust some coding style issues

In reply to Mauro Carvalho Chehab

Make pylint happier by adding some missing documentation and addressing a couple of pylint warnings.

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 scripts/kernel-doc.py           |  2 ++
 scripts/lib/kdoc/kdoc_files.py  |  4 +--
 scripts/lib/kdoc/kdoc_output.py | 51 ++++++++++++++++++++++++++-------
 3 files changed, 43 insertions(+), 14 deletions(-)
scripts/kernel-doc.py +2 -0
index d700451e9541..daae2b4f3307 100755
@@ -90,6 +90,8 @@ neither here nor at the original Perl script.
 
 
 class MsgFormatter(logging.Formatter):
+    """Helper class to format warnings on a similar way to kernel-doc.pl"""
+
     def format(self, record):
         record.levelname = record.levelname.capitalize()
         return logging.Formatter.format(self, record)
scripts/lib/kdoc/kdoc_files.py +1 -3
index 957aaeaed7e6..e1ed2f6dae94 100755
@@ -3,8 +3,6 @@
 # Copyright(c) 2025: Mauro Carvalho Chehab <[email protected]>.
 # SPDX-License-Identifier: GPL-2.0
 
-# TODO: implement warning filtering
-
 """
 Parse lernel-doc tags on multiple kernel source files.
 """

@@ -127,7 +125,7 @@ class KernelFiles():
     def __init__(self, verbose=False, out_style=None,
                  werror=False, wreturn=False, wshort_desc=False,
                  wcontents_before_sections=False,
-                 logger=None, modulename=None, export_file=None):
+                 logger=None, modulename=None):
         """
         Initialize startup variables and parse all files
         """
scripts/lib/kdoc/kdoc_output.py +40 -11

123 lines not on this page · raw message

Signature · 1 line
2.48.1
author
#3 lore raw

PATCH 3/6 scripts/lib/kdoc/kdoc_parser.py: fix Python compat with < v3.13

In reply to Mauro Carvalho Chehab

- str.replace count was introduced only in Python 3.13;
- before Python 3.13, f-string dict arguments can't use the same delimiter of the main string.

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 scripts/lib/kdoc/kdoc_output.py | 8 ++++----
 scripts/lib/kdoc/kdoc_parser.py | 4 ++--
 2 files changed, 6 insertions(+), 6 deletions(-)
scripts/lib/kdoc/kdoc_output.py +4 -4
index 97ec671e0b6e..df3c15bb1c10 100755
@@ -645,16 +645,16 @@ class ManFormat(OutputFormat):
         sectionlist = args.get('sectionlist', [])
         sections = args.get('sections', {})
 
-        self.data += f'.TH "{args['function']}" 9 "{args['function']}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
+        self.data += f'.TH "{args["function"]}" 9 "{args["function"]}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
 
         self.data += ".SH NAME\n"
         self.data += f"{args['function']} \\- {args['purpose']}\n"
 
         self.data += ".SH SYNOPSIS\n"
         if args.get('functiontype', ''):
-            self.data += f'.B "{args['functiontype']}" {args['function']}' + "\n"
+            self.data += f'.B "{args["functiontype"]}" {args["function"]}' + "\n"
         else:
-            self.data += f'.B "{args['function']}' + "\n"
+            self.data += f'.B "{args["function"]}' + "\n"
 
         count = 0
         parenth = "("

@@ -695,7 +695,7 @@ class ManFormat(OutputFormat):
         sectionlist = args.get('sectionlist', [])
         sections = args.get('sections', {})
 
-        self.data += f'.TH "{args['module']}" 9 "enum {args['enum']}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.data += f'.TH "{args["module"]}" 9 "enum {args["enum"]}" "{self.man_date}" "API Manual" LINUX' + "\n"
 
         self.data += ".SH NAME\n"
         self.data += f"enum {args['enum']} \\- {args['purpose']}\n"
scripts/lib/kdoc/kdoc_parser.py +2 -2
index a71145d531f2..51ac2d69a587 100755
@@ -1455,9 +1455,9 @@ class KernelDoc:
 
         r = Re(r'long\s+(sys_.*?),')
         if r.search(proto):
-            proto = proto.replace(',', '(', count=1)
+            proto = Re(',').sub('(', proto, count=1)
         elif is_void:
-            proto = proto.replace(')', '(void)', count=1)
+            proto = Re(r'\)').sub('(void)', proto, count=1)
 
         # Now delete all of the odd-numbered commas in the proto
         # so that argument types & names don't have a comma between them
Signature · 1 line
2.48.1
author
#4 lore raw

PATCH 4/6 scripts/kernel-doc.py: move modulename to man class

In reply to Mauro Carvalho Chehab

Only man output requires a modulename. Move its definition to the man class.

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 scripts/kernel-doc.py           |  6 +++---
 scripts/lib/kdoc/kdoc_files.py  |  6 +-----
 scripts/lib/kdoc/kdoc_output.py | 12 ++++++------
 scripts/lib/kdoc/kdoc_parser.py |  9 +--------
 4 files changed, 11 insertions(+), 22 deletions(-)
scripts/kernel-doc.py +3 -3
index daae2b4f3307..064106c18d8b 100755
@@ -111,6 +111,7 @@ def main():
                         help="Enable debug messages")
 
     parser.add_argument("-M", "-modulename", "--modulename",
+                        default="Kernel API",
                         help="Allow setting a module name at the output.")
 
     parser.add_argument("-l", "-enable-lineno", "--enable_lineno",

@@ -198,7 +199,7 @@ def main():
     logger.addHandler(handler)
 
     if args.man:
-        out_style = ManFormat()
+        out_style = ManFormat(modulename=args.modulename)
     elif args.none:
         out_style = None
     else:

@@ -207,8 +208,7 @@ def main():
     kfiles = KernelFiles(verbose=args.verbose,
                          out_style=out_style, werror=args.werror,
                          wreturn=args.wreturn, wshort_desc=args.wshort_desc,
-                         wcontents_before_sections=args.wcontents_before_sections,
-                         modulename=args.modulename)
+                         wcontents_before_sections=args.wcontents_before_sections)
 
     kfiles.parse(args.files, export_file=args.export_file)
scripts/lib/kdoc/kdoc_files.py +1 -5
index e1ed2f6dae94..a2417cafb1c8 100755
@@ -125,7 +125,7 @@ class KernelFiles():
     def __init__(self, verbose=False, out_style=None,
                  werror=False, wreturn=False, wshort_desc=False,
                  wcontents_before_sections=False,
-                 logger=None, modulename=None):
+                 logger=None):
         """
         Initialize startup variables and parse all files
         """

@@ -133,9 +133,6 @@ class KernelFiles():
         if not verbose:
             verbose = bool(os.environ.get("KBUILD_VERBOSE", 0))
 
-        if not modulename:
-            modulename = "Kernel API"
-
         if out_style is None:
             out_style = OutputFormat()
 

@@ -167,7 +164,6 @@ class KernelFiles():
         self.config.wreturn = wreturn
         self.config.wshort_desc = wshort_desc
         self.config.wcontents_before_sections = wcontents_before_sections
-        self.config.modulename = modulename
 
         self.config.function_table = set()
         self.config.source_map = {}
scripts/lib/kdoc/kdoc_output.py +6 -6
index df3c15bb1c10..13a74a687f89 100755
@@ -584,7 +584,7 @@ class ManFormat(OutputFormat):
     )
     blankline = ""
 
-    def __init__(self):
+    def __init__(self, modulename):
         """
         Creates class variables.
 

@@ -593,6 +593,7 @@ class ManFormat(OutputFormat):
         """
 
         super().__init__()
+        self.modulename = modulename
 
         dt = datetime.now()
         if os.environ.get("KBUILD_BUILD_TIMESTAMP", None):

@@ -624,14 +625,13 @@ class ManFormat(OutputFormat):
                 self.data += line + "\n"
 
     def out_doc(self, fname, name, args):
-        module = args.get('module')
         sectionlist = args.get('sectionlist', [])
         sections = args.get('sections', {})
 
         if not self.check_doc(name, args):
             return
 
-        self.data += f'.TH "{module}" 9 "{module}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.data += f'.TH "{self.modulename}" 9 "{self.modulename}" "{self.man_date}" "API Manual" LINUX' + "\n"
 
         for section in sectionlist:
             self.data += f'.SH "{section}"' + "\n"

@@ -695,7 +695,7 @@ class ManFormat(OutputFormat):
         sectionlist = args.get('sectionlist', [])
         sections = args.get('sections', {})
 
-        self.data += f'.TH "{args["module"]}" 9 "enum {args["enum"]}" "{self.man_date}" "API Manual" LINUX' + "\n"
+        self.data += f'.TH "{self.modulename}" 9 "enum {args["enum"]}" "{self.man_date}" "API Manual" LINUX' + "\n"
 
         self.data += ".SH NAME\n"
         self.data += f"enum {args['enum']} \\- {args['purpose']}\n"

@@ -725,7 +725,7 @@ class ManFormat(OutputFormat):
             self.output_highlight(sections[section])
 
     def out_typedef(self, fname, name, args):
-        module = args.get('module')
+        module = self.modulename
         typedef = args.get('typedef')
         purpose = args.get('purpose')
         sectionlist = args.get('sectionlist', [])

@@ -741,7 +741,7 @@ class ManFormat(OutputFormat):
             self.output_highlight(sections.get(section))
 
     def out_struct(self, fname, name, args):
-        module = args.get('module')
+        module = self.modulename
         struct_type = args.get('type')
         struct_name = args.get('struct')
         purpose = args.get('purpose')
scripts/lib/kdoc/kdoc_parser.py +1 -8
index 51ac2d69a587..0c0fa10b942b 100755
@@ -801,7 +801,6 @@ class KernelDoc:
 
         self.output_declaration(decl_type, declaration_name,
                                 struct=declaration_name,
-                                module=self.entry.modulename,
                                 definition=declaration,
                                 parameterlist=self.entry.parameterlist,
                                 parameterdescs=self.entry.parameterdescs,

@@ -879,7 +878,6 @@ class KernelDoc:
 
         self.output_declaration('enum', declaration_name,
                                 enum=declaration_name,
-                                module=self.config.modulename,
                                 parameterlist=self.entry.parameterlist,
                                 parameterdescs=self.entry.parameterdescs,
                                 parameterdesc_start_lines=self.entry.parameterdesc_start_lines,

@@ -1051,7 +1049,6 @@ class KernelDoc:
             self.output_declaration(decl_type, declaration_name,
                                     function=declaration_name,
                                     typedef=True,
-                                    module=self.config.modulename,
                                     functiontype=return_type,
                                     parameterlist=self.entry.parameterlist,
                                     parameterdescs=self.entry.parameterdescs,

@@ -1066,7 +1063,6 @@ class KernelDoc:
             self.output_declaration(decl_type, declaration_name,
                                     function=declaration_name,
                                     typedef=False,
-                                    module=self.config.modulename,
                                     functiontype=return_type,
                                     parameterlist=self.entry.parameterlist,
                                     parameterdescs=self.entry.parameterdescs,

@@ -1113,7 +1109,6 @@ class KernelDoc:
             self.output_declaration(decl_type, declaration_name,
                                     function=declaration_name,
                                     typedef=True,
-                                    module=self.entry.modulename,
                                     functiontype=return_type,
                                     parameterlist=self.entry.parameterlist,
                                     parameterdescs=self.entry.parameterdescs,

@@ -1141,7 +1136,6 @@ class KernelDoc:
 
             self.output_declaration('typedef', declaration_name,
                                     typedef=declaration_name,
-                                    module=self.entry.modulename,
                                     sectionlist=self.entry.sectionlist,
                                     sections=self.entry.sections,
                                     section_start_lines=self.entry.section_start_lines,

@@ -1630,8 +1624,7 @@ class KernelDoc:
             self.output_declaration("doc", self.entry.identifier,
                                     sectionlist=self.entry.sectionlist,
                                     sections=self.entry.sections,
-                                    section_start_lines=self.entry.section_start_lines,
-                                    module=self.config.modulename)
+                                    section_start_lines=self.entry.section_start_lines)
             self.reset_state(ln)
 
         elif doc_content.search(line):
Signature · 1 line
2.48.1
author
#5 lore raw

PATCH 5/6 scripts/kernel-doc.py: properly handle KBUILD_BUILD_TIMESTAMP

In reply to Mauro Carvalho Chehab

The logic that handles KBUILD_BUILD_TIMESTAMP is wrong, and adds a dependency of a third party module (dateutil).

Fix it.

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 scripts/lib/kdoc/kdoc_files.py  |  9 ---------
 scripts/lib/kdoc/kdoc_output.py | 28 +++++++++++++++++++++-------
 2 files changed, 21 insertions(+), 16 deletions(-)
scripts/lib/kdoc/kdoc_files.py +0 -9
index a2417cafb1c8..19fe9c6f5352 100755
@@ -12,9 +12,6 @@ import logging
 import os
 import re
 import sys
-from datetime import datetime
-
-from dateutil import tz
 
 from kdoc_parser import KernelDoc
 from kdoc_output import OutputFormat

@@ -136,12 +133,6 @@ class KernelFiles():
         if out_style is None:
             out_style = OutputFormat()
 
-        dt = datetime.now()
-        if os.environ.get("KBUILD_BUILD_TIMESTAMP", None):
-            # use UTC TZ
-            to_zone = tz.gettz('UTC')
-            dt = dt.astimezone(to_zone)
-
         if not werror:
             kcflags = os.environ.get("KCFLAGS", None)
             if kcflags:
scripts/lib/kdoc/kdoc_output.py +21 -7
index 13a74a687f89..166fcabbb4cf 100755
@@ -18,8 +18,6 @@ import os
 import re
 from datetime import datetime
 
-from dateutil import tz
-
 from kdoc_parser import KernelDoc, type_param
 from kdoc_re import Re
 

@@ -584,6 +582,15 @@ class ManFormat(OutputFormat):
     )
     blankline = ""
 
+    date_formats = [
+        "%a %b %d %H:%M:%S %Z %Y",
+        "%a %b %d %H:%M:%S %Y",
+        "%Y-%m-%d",
+        "%b %d %Y",
+        "%B %d %Y",
+        "%m %d %Y",
+    ]
+
     def __init__(self, modulename):
         """
         Creates class variables.

@@ -595,11 +602,18 @@ class ManFormat(OutputFormat):
         super().__init__()
         self.modulename = modulename
 
-        dt = datetime.now()
-        if os.environ.get("KBUILD_BUILD_TIMESTAMP", None):
-            # use UTC TZ
-            to_zone = tz.gettz('UTC')
-            dt = dt.astimezone(to_zone)
+        dt = None
+        tstamp = os.environ.get("KBUILD_BUILD_TIMESTAMP")
+        if tstamp:
+            for fmt in self.date_formats:
+                try:
+                    dt = datetime.strptime(tstamp, fmt)
+                    break
+                except ValueError:
+                    pass
+
+        if not dt:
+            dt = datetime.now()
 
         self.man_date = dt.strftime("%B %Y")
Signature · 1 line
2.48.1
author
#6 lore raw

PATCH 6/6 scripts/lib/kdoc/kdoc_parser.py: remove a python 3.9 dependency

In reply to Mauro Carvalho Chehab

str.removesuffix() was added on Python 3.9, but rstrip() actually does the same thing, as we just want to remove a single character. It is also shorter.

So, use it.

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 scripts/lib/kdoc/kdoc_parser.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/lib/kdoc/kdoc_parser.py +1 -1
index 0c0fa10b942b..feac699a3893 100755
@@ -1652,7 +1652,7 @@ class KernelDoc:
                     # Group continuation lines on prototypes
                     if self.state == self.STATE_PROTO:
                         if line.endswith("\\"):
-                            prev += line.removesuffix("\\")
+                            prev += line.rstrip("\\")
                             cont = True
 
                             if not prev_ln:
Signature · 1 line
2.48.1