Some kernel-doc fixes

PATCH Superseded Mauro Carvalho Chehab 2 participants 6 messages May 21, 2025

Subsystems (3)

Documentation Kbuild Documentation scripts

lore

Share & tools

lore

Mauro Carvalho Chehab

author

2025-05-21 06:59 UTC #0 lore raw

v2 cover Some kernel-doc fixes

Hi Jon,

That's the second version of the kernel-doc fixup patch series.

I discovered the root cause why Sphinx logger was not working: there was a call there for logger.verbose(). According with:

https://www.sphinx-doc.org/en/master/extdev/logging.html

This is a valid call, but it doesn't verbose messages. Instead, it is crashing with modern Sphinx versions, causing the log to not work.

I got rid of it, replacing by logger.info(). I took the time to also address an issue pointed by Andy: not having the same log message placed everywhere. With such change, we can keep using Sphinx logger (which produces colored messages) inside kernel-doc classes.

With that, we have:

Patch 1: makes Lore and kernel-doc ML receive patches related
to kernel-doc.py and get_abi.py.
Patch 2: cleanup try/except logic and get rid of logger.verbose();
Patch 3: fix a KeyError when trying to acess data from non-existing files;

If you test just patch 1 on the top of next-20250516, you'll see the keyerror message (in red):

ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -export ./drivers/gpio/gpiolib-acpi.c' processing failed with: KeyError('./drivers/gpio/gpiolib-acpi.c')

And the script doesn't crash anymore. After patch 2, the try/except warning gets replaced by a proper message:

ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
WARNING: No kernel-doc for file ./drivers/gpio/gpiolib-acpi.c

Please apply it to solve the issues that are reported by Stephen and Akira.

Regards,
Mauro
Mauro Carvalho Chehab (3):
MAINTAINERS: update linux-doc entry to cover new Python scripts
docs: kerneldoc.py: simplify exception handling logic
scripts: kernel-doc: prevent a KeyError when checking output

 Documentation/sphinx/kerneldoc.py | 108 +++++++++++++-----------------
 MAINTAINERS                       |   5 +-
 scripts/lib/kdoc/kdoc_files.py    |   4 ++
 3 files changed, 53 insertions(+), 64 deletions(-)

Signature · 1 line

2.49.0

Mauro Carvalho Chehab

author

2025-05-21 06:59 UTC #1 lore raw

PATCH v2 1/3 MAINTAINERS: update linux-doc entry to cover new Python scripts

Changes to ABI and kernel-doc need to be c/c linux-doc. Update the maintainer's entry to cover those files.

Signed-off-by Mauro Carvalho Chehab <[email protected]>

---
 MAINTAINERS | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

MAINTAINERS +4 -1

index 20e07e61a148..a668808769b6 100644
@@ -7173,7 +7173,10 @@ T:	git git://git.lwn.net/linux.git docs-next
 F:	Documentation/
 F:	scripts/check-variable-fonts.sh
 F:	scripts/documentation-file-ref-check
-F:	scripts/kernel-doc
+F:	scripts/get_abi.py
+F:	scripts/kernel-doc*
+F	scripts/lib/abi/*
+F	scripts/lib/kdoc/*
 F:	scripts/sphinx-pre-install
 X:	Documentation/ABI/
 X:	Documentation/admin-guide/media/

Signature · 1 line

2.49.0

Akira Yokosawa

2025-05-21 08:21 UTC #2 lore raw

Re: [PATCH v2 1/3] MAINTAINERS: update linux-doc entry to cover new Python scripts

Hi Mauro,

Quoted 2 lines · Mauro Carvalho Chehab

Changes to ABI and kernel-doc need to be c/c linux-doc. Update the maintainer's entry to cover those files.

See below for a couple of trivial typo.

Quoted 11 lines

Signed-off-by Mauro Carvalho Chehab <[email protected]>
---
 MAINTAINERS | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
MAINTAINERS +0 -0
index 20e07e61a148..a668808769b6 100644

MAINTAINERS T: git git://git.lwn.net/linux.git docs-next · line 7178

Quoted 7 lines

MAINTAINERS +3 -1

@@ -7173,7 +7173,10 @@ T:	git git://git.lwn.net/linux.git docs-next
 F:	Documentation/
 F:	scripts/check-variable-fonts.sh
 F:	scripts/documentation-file-ref-check
-F:	scripts/kernel-doc
+F:	scripts/get_abi.py
+F:	scripts/kernel-doc*
+F	scripts/lib/abi/*

F: scripts/lib/abi/*

MAINTAINERS T: git git://git.lwn.net/linux.git docs-next · line 7179

Quoted 1 line

MAINTAINERS +1 -0
+F	scripts/lib/kdoc/*

F: scripts/lib/kdoc/*

MAINTAINERS T: git git://git.lwn.net/linux.git docs-next · line 7182

Quoted 3 lines

MAINTAINERS +0 -0

 F:	scripts/sphinx-pre-install
 X:	Documentation/ABI/
 X:	Documentation/admin-guide/media/

Regards,
Akira

Mauro Carvalho Chehab

author

2025-05-21 06:59 UTC #3 lore raw

PATCH v2 2/3 docs: kerneldoc.py: simplify exception handling logic

In reply to Mauro Carvalho Chehab

Get rid of logger.verbose() which is causing the logger to not work.

Also, instead of having try/except everywhere, place them on a common place.

While here, get rid of some bogus logs.

Signed-off-by Mauro Carvalho Chehab <[email protected]>

---
 Documentation/sphinx/kerneldoc.py | 108 +++++++++++++-----------------
 1 file changed, 45 insertions(+), 63 deletions(-)

Documentation/sphinx/kerneldoc.py +45 ~~-63~~

index b713a2c4a615..b818d4c77924 100644
@@ -40,6 +40,7 @@ from docutils.parsers.rst import directives, Directive
 import sphinx
 from sphinx.util.docutils import switch_source_input
 from sphinx.util import logging
+from pprint import pformat
 
 srctree = os.path.abspath(os.environ["srctree"])
 sys.path.insert(0, os.path.join(srctree, "scripts/lib/kdoc"))

@@ -49,7 +50,7 @@ from kdoc_output import RestFormat
 
 __version__  = '1.0'
 kfiles = None
-logger = logging.getLogger('kerneldoc')
+logger = logging.getLogger(__name__)
 
 def cmd_str(cmd):
     """

@@ -190,46 +191,31 @@ class KernelDocDirective(Directive):
 
         return cmd
 
-    def run_cmd(self):
+    def run_cmd(self, cmd):
         """
         Execute an external kernel-doc command.
         """
 
         env = self.state.document.settings.env
-        cmd = self.handle_args()
-
-        if self.verbose >= 1:
-            print(cmd_str(cmd))
-
         node = nodes.section()
 
-        try:
-            logger.verbose("calling kernel-doc '%s'" % (" ".join(cmd)))
+        p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        out, err = p.communicate()
 
-            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-            out, err = p.communicate()
+        out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
 
-            out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+        if p.returncode != 0:
+            sys.stderr.write(err)
 
-            if p.returncode != 0:
-                sys.stderr.write(err)
-
-                logger.warning("kernel-doc '%s' failed with return code %d"
-                                    % (" ".join(cmd), p.returncode))
-                return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
-            elif env.config.kerneldoc_verbosity > 0:
-                sys.stderr.write(err)
-
-        except Exception as e:  # pylint: disable=W0703
-            logger.warning("kernel-doc '%s' processing failed with: %s" %
-                                (" ".join(cmd), str(e)))
+            logger.warning("kernel-doc '%s' failed with return code %d"
+                                % (" ".join(cmd), p.returncode))
             return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
+        elif env.config.kerneldoc_verbosity > 0:
+            sys.stderr.write(err)
 
         filenames = self.parse_args["file_list"]
         for filename in filenames:
-            ret = self.parse_msg(filename, node, out, cmd)
-            if ret:
-                return ret
+            self.parse_msg(filename, node, out, cmd)
 
         return node.children

@@ -240,40 +226,31 @@ class KernelDocDirective(Directive):
 
         env = self.state.document.settings.env
 
-        try:
-            lines = statemachine.string2lines(out, self.tab_width,
-                                              convert_whitespace=True)
-            result = ViewList()
+        lines = statemachine.string2lines(out, self.tab_width,
+                                            convert_whitespace=True)
+        result = ViewList()
 
-            lineoffset = 0;
-            line_regex = re.compile(r"^\.\. LINENO ([0-9]+)$")
-            for line in lines:
-                match = line_regex.search(line)
-                if match:
-                    # sphinx counts lines from 0
-                    lineoffset = int(match.group(1)) - 1
-                    # we must eat our comments since the upset the markup
-                else:
-                    doc = str(env.srcdir) + "/" + env.docname + ":" + str(self.lineno)
-                    result.append(line, doc + ": " + filename, lineoffset)
-                    lineoffset += 1
+        lineoffset = 0;
+        line_regex = re.compile(r"^\.\. LINENO ([0-9]+)$")
+        for line in lines:
+            match = line_regex.search(line)
+            if match:
+                # sphinx counts lines from 0
+                lineoffset = int(match.group(1)) - 1
+                # we must eat our comments since the upset the markup
+            else:
+                doc = str(env.srcdir) + "/" + env.docname + ":" + str(self.lineno)
+                result.append(line, doc + ": " + filename, lineoffset)
+                lineoffset += 1
 
-            self.do_parse(result, node)
+        self.do_parse(result, node)
 
-        except Exception as e:  # pylint: disable=W0703
-            logger.warning("kernel-doc '%s' processing failed with: %s" %
-                                (cmd_str(cmd), str(e)))
-            return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
-
-        return None
-
-    def run_kdoc(self, kfiles):
+    def run_kdoc(self, cmd, kfiles):
         """
         Execute kernel-doc classes directly instead of running as a separate
         command.
         """
 
-        cmd = self.handle_args()
         env = self.state.document.settings.env
 
         node = nodes.section()

@@ -282,22 +259,27 @@ class KernelDocDirective(Directive):
         filenames = self.parse_args["file_list"]
 
         for filename, out in kfiles.msg(**self.msg_args, filenames=filenames):
-            if self.verbose >= 1:
-                print(cmd_str(cmd))
-
-            ret = self.parse_msg(filename, node, out, cmd)
-            if ret:
-                return ret
+            self.parse_msg(filename, node, out, cmd)
 
         return node.children
 
     def run(self):
         global kfiles
 
-        if kfiles:
-            return self.run_kdoc(kfiles)
-        else:
-            return self.run_cmd()
+        cmd = self.handle_args()
+        if self.verbose >= 1:
+            logger.info(cmd_str(cmd))
+
+        try:
+            if kfiles:
+                return self.run_kdoc(cmd, kfiles)
+            else:
+                return self.run_cmd(cmd)
+
+        except Exception as e:  # pylint: disable=W0703
+            logger.warning("kernel-doc '%s' processing failed with: %s" %
+                           (cmd_str(cmd), pformat(e)))
+            return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
 
     def do_parse(self, result, node):
         with switch_source_input(self.state, result):

Signature · 1 line

2.49.0

Mauro Carvalho Chehab

author

2025-05-21 06:59 UTC #4 lore raw

PATCH v2 3/3 scripts: kernel-doc: prevent a KeyError when checking output

In reply to Mauro Carvalho Chehab

If a file sent to KernelFiles.msg() method doesn't exist, instead of producing a KeyError, output an error message.

Reported-by Randy Dunlap <[email protected]>

Closes https://lore.kernel.org/linux-doc/[email protected]/T/#ma43ae9d8d0995b535cf5099e5381dace0410de04 open here

Signed-off-by Mauro Carvalho Chehab <[email protected]>

---
 scripts/lib/kdoc/kdoc_files.py | 4 ++++
 1 file changed, 4 insertions(+)

scripts/lib/kdoc/kdoc_files.py +4 -0

index 630aa5ca6460..9be4a64df71d 100644
@@ -271,6 +271,10 @@ class KernelFiles():
                                       no_doc_sections)
 
             msg = ""
+            if fname not in self.results:
+                self.config.log.warning("No kernel-doc for file %s", fname)
+                continue
+
             for name, arg in self.results[fname]:
                 m = self.out_msg(fname, name, arg)

Signature · 1 line

2.49.0

Akira Yokosawa

Acked-by on the v2 series

2025-05-21 08:30 UTC #5 lore raw

In reply to Mauro Carvalho Chehab

Hi,

Quoted 32 lines · Mauro Carvalho Chehab

Hi Jon,
That's the second version of the kernel-doc fixup patch series.
I discovered the root cause why Sphinx logger was not working: there was a call there for logger.verbose(). According with:
https://www.sphinx-doc.org/en/master/extdev/logging.html
This is a valid call, but it doesn't verbose messages. Instead, it is crashing with modern Sphinx versions, causing the log to not work.
I got rid of it, replacing by logger.info(). I took the time to also address an issue pointed by Andy: not having the same log message placed everywhere. With such change, we can keep using Sphinx logger (which produces colored messages) inside kernel-doc classes.
With that, we have:
Patch 1: makes Lore and kernel-doc ML receive patches related
to kernel-doc.py and get_abi.py.
Patch 2: cleanup try/except logic and get rid of logger.verbose();
Patch 3: fix a KeyError when trying to acess data from non-existing files;
If you test just patch 1 on the top of next-20250516, you'll see the keyerror message (in red):
ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -export ./drivers/gpio/gpiolib-acpi.c' processing failed with: KeyError('./drivers/gpio/gpiolib-acpi.c')

Quoted 8 lines

And the script doesn't crash anymore. After patch 2, the try/except warning gets replaced by a proper message:
ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
WARNING: No kernel-doc for file ./drivers/gpio/gpiolib-acpi.c
Please apply it to solve the issues that are reported by Stephen and Akira.

Other than the small typo in 1/3, this looks promising.

For the series,

Acked-by Akira Yokosawa <[email protected]>

Regards,
Akira

Quoted 13 lines

Regards,
Mauro
Mauro Carvalho Chehab (3):
MAINTAINERS: update linux-doc entry to cover new Python scripts
docs: kerneldoc.py: simplify exception handling logic
scripts: kernel-doc: prevent a KeyError when checking output
 Documentation/sphinx/kerneldoc.py | 108 +++++++++++++-----------------
 MAINTAINERS                       |   5 +-
 scripts/lib/kdoc/kdoc_files.py    |   4 ++
 3 files changed, 53 insertions(+), 64 deletions(-)