Restore kernel-doc support for prehistoric Python

Hi Jon,

Sendind a v2 that was actually tested with versions 2.7 and 3.4.

While discussing patch v2 12/12 from your kdoc series, I realized that kernel-doc command line should not crash with Python 3.6, nor cause compilation breakages with older versions.

The problem mainly affect builds with CONFIG_DRM enabled, due to:

drivers/gpu/drm/Makefile:                PYTHONDONTWRITEBYTECODE=1 $(KERNELDOC) -none $(if $(CONFIG_WERROR)$(CONFIG_DRM_WERROR),-Werror) $<; \
drivers/gpu/drm/i915/Makefile:    cmd_checkdoc = PYTHONDONTWRITEBYTECODE=1 $(KERNELDOC) -none -Werror $<
include/drm/Makefile:           PYTHONDONTWRITEBYTECODE=1 $(KERNELDOC) -none $(if $(CONFIG_WERROR)$(CONFIG_DRM_WERROR),-Werror) $<; \

This small series prevent Kernel build breakages on such case, and it is meant to be merged after your kdoc patches.

Patch 1 emits a warning wih Python 3.7 siilar to kdoc class, but on an early stage. More importantly, it emits a warning and exits the script for versions <= 3.6.

We still need the kdoc warning, as it detect troubles when calling kernel-doc as a class.

Patch 2 fix a backward-compatibility issue that otherwise would require Python 3.9.

With that, building the Kernel with older versions won't break.

Regards,
Mauro

---

v2:
- added a coding line to kernel-doc exec;
- only include kdoc lib after checking version;
- don't use f-strings at main().

Mauro Carvalho Chehab (2):
docs: kernel-doc: emit warnings for ancient versions of Python
scripts: kdoc: make it backward-compatible with Python 3.7

 scripts/kernel-doc.py           | 35 ++++++++++++++++++++++++++-------
 scripts/lib/kdoc/kdoc_parser.py |  4 +++-
 2 files changed, 31 insertions(+), 8 deletions(-)

2.50.0

Kernel-doc requires at least version 3.6 to run, as it uses f-string. Yet, Kernel build currently calls kernel-doc with -none on some places. Better not to bail out when older versions are found.

Versions of Python prior to 3.7 do not guarantee to remember the insertion order of dicts; since kernel-doc depends on that guarantee, running with such older versions could result in output with reordered sections.

Check Python version when called via command line and ensure that it will run with a much older version, failing after checking arguments, but return 0 to avoid breaking Kernel compilation.

With that, potentially this will run with python 2.7 and 3.2+, according with vermin:

	$ vermin --no-tips -v ./scripts/kernel-doc
	Detecting python files..
	Analyzing using 24 processes..
	2.7, 3.2     /new_devel/v4l/docs/scripts/kernel-doc
	Minimum required versions: 2.7, 3.2

I tested myself with versions 2.7 and 3.4 using anaconda (version 3 < 3.4 is not available there)

	$ conda create -n py27 python=2.7
	$ conda activate py27
	$ python ./scripts/kernel-doc --none include/media
	Error: Python 3.6 or later is required by kernel-doc
	$ conda deactivate

	$ conda create -n py34 python=3.4
	$ conda activate py34
	python --version
        Python 3.4.5
        $ python ./scripts/kernel-doc --none include/media
	Error: Python 3.6 or later is required by kernel-doc
	$ conda deactivate

	$ python --version
	Python 3.13.5
        $ python ./scripts/kernel-doc --none include/media

---
 scripts/kernel-doc.py | 35 ++++++++++++++++++++++++++++-------
 1 file changed, 28 insertions(+), 7 deletions(-)

index 12ae66f40bd7..c59eea3e0fe9 100755
@@ -1,9 +1,19 @@
 #!/usr/bin/env python3
+# -*- coding: utf-8 -*-
 # SPDX-License-Identifier: GPL-2.0
 # Copyright(c) 2025: Mauro Carvalho Chehab <[email protected]>.
 #
-# pylint: disable=C0103,R0915
-#
+# pylint: disable=C0103,R0912,R0914,R0915
+
+# NOTE: While kernel-doc requires at least version 3.6 to run, the
+#       command line explicitly supports Python 2,7 and 3.4+.
+#       The rationale is that it shall fail gracefully during Kernel
+#       compilation with older Kernel versions. Due to that:
+#       - encoding line is needed here;
+#       - no f-strings can be used on this file.
+#       - the libraries that require newer versions can only be included
+#         after Python version is checked.
+
 # Converted from the kernel-doc script originally written in Perl
 # under GPLv2, copyrighted since 1998 by the following authors:
 #

@@ -107,9 +117,6 @@ SRC_DIR = os.path.dirname(os.path.realpath(__file__))
 
 sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
 
-from kdoc_files import KernelFiles                      # pylint: disable=C0413
-from kdoc_output import RestFormat, ManFormat           # pylint: disable=C0413
-
 DESC = """
 Read C language source or header FILEs, extract embedded documentation comments,
 and print formatted documentation to standard output.

@@ -271,6 +278,20 @@ def main():
 
     logger.addHandler(handler)
 
+    python_ver = sys.version_info[:2]
+    if python_ver < (3,6):
+        logger.error("Python 3.6 or later is required by kernel-doc")
+
+        # Return 0 here to avoid breaking compilation
+        sys.exit(0)
+
+    if python_ver < (3,7):
+        logger.warning("Python 3.7 or later is required for correct results")
+
+    # Import kernel-doc libraries only after checking Python version
+    from kdoc_files import KernelFiles                  # pylint: disable=C0415
+    from kdoc_output import RestFormat, ManFormat       # pylint: disable=C0415
+
     if args.man:
         out_style = ManFormat(modulename=args.modulename)
     elif args.none:

@@ -298,11 +319,11 @@ def main():
         sys.exit(0)
 
     if args.werror:
-        print(f"{error_count} warnings as errors")
+        print("%s warnings as errors" % error_count)    # pylint: disable=C0209
         sys.exit(error_count)
 
     if args.verbose:
-        print(f"{error_count} errors")
+        print("%s errors" % error_count)                # pylint: disable=C0209
 
     if args.none:
         sys.exit(0)

2.50.0

There was a change at kdoc that ended breaking compatibility with Python 3.7: str.removesuffix() was introduced on version 3.9.

Restore backward compatibility.

---
 scripts/lib/kdoc/kdoc_parser.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

index 831f061f61b8..6273141033a8 100644
@@ -1214,7 +1214,9 @@ class KernelDoc:
         # Found an export, trim out any special suffixes
         #
         for suffix in suffixes:
-            symbol = symbol.removesuffix(suffix)
+            # Be backward compatible with Python < 3.9
+            if symbol.endswith(suffix):
+                symbol = symbol[:-len(suffix)]
         function_set.add(symbol)
         return True
 

2.50.0

Python is listed as an optional dependency, but this is not true, as kernel-doc is called during compilation when DRM is enabled. Better document that.

---
 Documentation/process/changes.rst | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

index bccfa19b45df..6a7d7c1ee274 100644
@@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
 GNU tar                1.28             tar --version
 gtags (optional)       6.6.5            gtags --version
 mkimage (optional)     2017.01          mkimage --version
-Python (optional)      3.9.x            python3 --version
+Python                 3.9.x            python3 --version
 GNU AWK (optional)     5.1.0            gawk --version
 ====================== ===============  ========================================
 

@@ -154,6 +154,13 @@ Perl
 You will need perl 5 and the following modules: ``Getopt::Long``,
 ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
 
+Python
+------
+
+At least Python 2.7 or 3.4 is required if CONFIG_DRM is selected to avoid
+breaking compilation. Documentation build and kernel-doc won't produce
+valid results if version is below 3.7.
+
 BC
 --
 

2.50.0

Hi Mauro,

Thank you for the patch.

Isn't it only when CONFIG_DRM_HEADER_TEST is enabled ? That option depends on EXPERT && BROKEN, so I wouldn't expect it to be widely enabled. A quick grep shows that CONFIG_DRM_I915_WERROR does the same (with a dependency on EXPERT but not BROKEN though).

Is there something else in DRM that invokes kernel-doc ?

Regards,

Laurent Pinchart

Em Sat, 12 Jul 2025 19:31:55 +0300
Laurent Pinchart <[email protected]> escreveu:

Well, EXPERT is currently enabled on several distros. The three ones I have it handy all have it:

Fedora 42:

	$ grep CONFIG_EXPERT /boot/config*
	/boot/config-6.14.9-300.fc42.x86_64:CONFIG_EXPERT=y
	/boot/config-6.15.3-200.fc42.x86_64:CONFIG_EXPERT=y
	/boot/config-6.15.4-200.fc42.x86_64:CONFIG_EXPERT=y

Ubuntu 24.10:

	$ grep CONFIG_EXPERT /boot/config*
	/boot/config-6.11.0-26-generic:CONFIG_EXPERT=y
	/boot/config-6.8.0-60-generic:CONFIG_EXPERT=y

Debian 12:

	$ grep CONFIG_EXPERT /boot/config*
	/boot/config-6.1.0-34-amd64:CONFIG_EXPERT=y
	/boot/config-6.1.0-37-amd64:CONFIG_EXPERT=y
	/boot/config-6.1.0-37-rt-amd64:CONFIG_EXPERT=y

So, expert on distros seem quite common those days.

Fedora has it enabled for a long time. On Fedora 42:

	$ grep CONFIG_DRM_WERROR /boot/config*
	/boot/config-6.14.9-300.fc42.x86_64:CONFIG_DRM_WERROR=y
	/boot/config-6.15.3-200.fc42.x86_64:CONFIG_DRM_WERROR=y
	/boot/config-6.15.4-200.fc42.x86_64:CONFIG_DRM_WERROR=y

I would expect to have it enabled on other distros as well.

Maybe I can place instead CONFIG_DRM_I915_WERROR.

Thanks,
Mauro

But not CONFIG_BROKEN, right ? That would leave only CONFIG_DRM_I915_WERROR.

CONFIG_DRM_WERROR doesn't seem related to running kernel-doc though, is it ?

I think that's better. I also wouldn't consider CONFIG_DRM_I915_WERROR to make Python "required".

Regards,

Laurent Pinchart

I've held off on this series on the expectation that a new version would come. I guess, at this point, it will be a post-merge-window thing?

Thanks,

jon

Em Thu, 24 Jul 2025 08:42:59 -0600
Jonathan Corbet <[email protected]> escreveu:

Feel free to postpone. I have already a new version of it here somewhere on my branches, but I had to take some days off. So, I ended not sending you the (probably) final version.

I intend to send what I have here during the merge window for you to review and apply post-merge-window.

Thanks,
Mauro

I think the main questions here are 1) how to handle optional build tool dependencies, and 2) whether Python is an optional or required dependency.

It might be nice to be able to have an actual Kconfig and dependency for optional tools. "depends on TOOL_PYTHON" or something. Enable the option, and you should have Python.

This in turn raises the question for allyesconfig. It's cumbersome (though not impossible) to add config options that you actually have to enable manually.

The header test stuff really isn't required to actually build the kernel or drm, however DRM_MSM does depend on Python for building the driver.

BR,
Jani.

Jani Nikula, Intel

Em Mon, 28 Jul 2025 12:28:45 +0300
Jani Nikula <[email protected]> escreveu:

with regards to (2), besides doc build and kernel-doc --none, there is at least another place at the building system requiring Python:
scripts/Makefile.vmlinux_o: There, it has:

	quiet_cmd_gen_initcalls_lds = GEN     $@
	      cmd_gen_initcalls_lds = \
	        $(PYTHON3) $(srctree)/scripts/jobserver-exec \
	        $(PERL) $(real-prereqs) > $@

	.tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
	                vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
	        $(call if_changed,gen_initcalls_lds)

	targets := .tmp_initcalls.lds

	ifdef CONFIG_LTO_CLANG
	initcalls-lds := .tmp_initcalls.lds
	endif

Now, I didn't check exactly what conditions trigger .tmp_initcalls.lds, but there are some places that use select:

arch/Kconfig: select LTO_CLANG

On a quick look, it sounds that some archs will select this automatically when built with clang.

I didn't check if other parts of the building system requires it.

In any case, on its current state, I'd say that currently this is not optional.

That would be an option. The question is: is it worth spending time on it?

IMO it doesn't make sense to manually enable something with *config. If they depend on Python, be it: for such targets, Python is mandatory.

Good to know. It means that, for those *config targets:

arch/arm/configs/imx_v6_v7_defconfig:CONFIG_DRM_MSM=y arch/arm/configs/multi_v7_defconfig:CONFIG_DRM_MSM=m arch/arm/configs/qcom_defconfig:CONFIG_DRM_MSM=m

(plus all{mod|yes}config)

Python is mandatory. As multi_v7_defconfig is one of them, we may assume, that, o practical cases, Python 2.7 or 3.2+ is mandatory for arm support. The current requirement is:

	$ vermin -v --no-tips ./drivers/gpu/drm/msm/registers/gen_header.py
	Detecting python files..
	Analyzing using 8 processes..
	2.7, 3.2     /new_devel/v4l/docs/drivers/gpu/drm/msm/registers/gen_header.py
	Minimum required versions: 2.7, 3.2

Heh, looking for files that end with .py at Makefile (not all Python scripts at the Kernel end with such extension), it seems that there are more:

	grep \\\.py $(git ls-files|grep Makefile|grep -v tools)


	Makefile:KERNELDOC       = $(srctree)/scripts/kernel-doc.py
	Makefile:		  vmlinux-gdb.py \
	Makefile:	$(Q)ln -fsn $(abspath $(srctree)/scripts/gdb/vmlinux-gdb.py)
	Makefile:compile_commands.json: $(srctree)/scripts/clang-tools/gen_compile_commands.py \
	Makefile:      cmd_clang_tools = $(PYTHON3) $(srctree)/scripts/clang-tools/run-clang-tools.py $@ $<
	drivers/gpu/drm/msm/Makefile:      cmd_headergen = mkdir -p $(obj)/generated && $(PYTHON3) $(src)/registers/gen_header.py \
	drivers/gpu/drm/msm/Makefile:		$(src)/registers/gen_header.py \
	drivers/gpu/drm/msm/Makefile:		$(src)/registers/gen_header.py \
	drivers/tty/vt/Makefile:#GENERATE_UCS_TABLES := 2  # invokes gen_ucs_recompose_table.py with --full
	drivers/tty/vt/Makefile:$(obj)/ucs_width_table.h: $(src)/gen_ucs_width_table.py
	drivers/tty/vt/Makefile:$(obj)/ucs_recompose_table.h: $(src)/gen_ucs_recompose_table.py
	drivers/tty/vt/Makefile:$(obj)/ucs_fallback_table.h: $(src)/gen_ucs_fallback_table.py
	rust/Makefile:	$(Q)MAKEFLAGS= $(srctree)/scripts/generate_rust_analyzer.py \
	scripts/Makefile.lib:MAKE_FIT := $(srctree)/scripts/make_fit.py
	scripts/gdb/linux/Makefile:symlinks := $(patsubst $(src)/%,%,$(wildcard $(src)/*.py))
	scripts/gdb/linux/Makefile:always-y += constants.py
	scripts/gdb/linux/Makefile:$(obj)/constants.py: $(src)/constants.py.in FORCE
	scripts/gdb/linux/Makefile:clean-files := *.pyc *.pyo

(that not including tools/*)

Some seem false positives, but on the other hand, looking for tools, several scripts seem to be executed by non-tools Makefiles. I didn't check if any of them are written in python, though.

Considering the above, for me it seems that the bus already
departed: there are several cases where Python is required during build time. So, adding a "depends on TOOL_PYTHON" doesn't seem to be trivial.

From my side, I don't mind much about that, as this is something that affects only the Kernel build. I would very much prefer to have things like config BPF optional.

Regards,
Mauro

FWIW, if it was up to me, I'd make Python 3+ a non-optional build dependency. I'd also forget about any Python 2 backward compat stuff.

I would find it very useful for code/header generation during build time, instead of having to resort to C hostprogs. Similar to what MSM is doing.

That said, I know there's going to be people vehemently opposed.

Agreed. Forget about that idea.

BR,
Jani.

Jani Nikula, Intel

Em Tue, 29 Jul 2025 13:45:26 +0300
Jani Nikula <[email protected]> escreveu:

I don't think we should do much effort to support Python 2, but it comes almost for free: only shebang needs to be different, and, if the comments inside the doc contains non-utf8 chars, an encoding line.

The current tools during Kernel build currently supports it (again, except for shebang).

Anyway, from my side I'm happy either way.

Fine from my side ;-) There are some precedents here, so I guess it should be up to each subsystem to decide using it or not.

As said in the other thread, changing the shebang is not a trivial thing. I don't have 'python' installed, haven't had for years, just 'python3'.

BR,
Jani.

Jani Nikula, Intel