Better handle and document Python needs for Kernel build
Subsystems (2)
cover Better handle and document Python needs for Kernel build
Hi Jon,
As promised, I'm sending you the missing patches from the python backward-compatibility series. They're not urgent, IMO it should be OK to have them merged for 6.17.
The first patch fix process/changes.rst. Currently, it makes one think that Python is optional. While not having python may work on some environments, there are some python scripts called during Kernel build time that require python.
Document them.
Also, in order to avoid breaking compilation, make kernel-doc's main script executable with elder Python versions.
Backward-compatibility tested with Python 2.7 and 3.4 (although it may work since 3.2). On elder versions, it would emit a warning and do nothing, as the actual code depends on features at 3.6 (f-strings) and 3.7 (ordered dict).
Mauro Carvalho Chehab (2):
docs: kernel-doc: avoid script crash on ancient Python
docs: changes: better document Python needs
Documentation/process/changes.rst | 13 ++++++++++- scripts/kernel-doc.py | 37 ++++++++++++++++++++++--------- 2 files changed, 38 insertions(+), 12 deletions(-)
Signature · 1 line
2.49.0
PATCH 1/2 docs: kernel-doc: avoid script crash on ancient Python
While we do need at least 3.6 for kernel-doc to work, and at least 3.7 for it to output functions and structs with parameters at the right order, let the python binary be compatible with legacy versions.
The rationale is that the Kernel build nowadays calls kernel-doc with -none on some places. Better not to bail out when older versions are found.
With that, potentially this will run with python 2.7 and 3.2+, according with vermin:
$ vermin --no-tips -v ./scripts/kernel-doc Detecting python files.. Analyzing using 24 processes.. 2.7, 3.2 /new_devel/v4l/docs/scripts/kernel-doc Minimum required versions: 2.7, 3.2
I actually tested it with versions 2.7 and 3.4 using anaconda (version 3 < 3.4 was not available there). For both, the script didn't crash and emitted the right warning:
$ conda create -n py27 python=2.7 $ conda activate py27 $ python ./scripts/kernel-doc --none include/media Error: Python 3.6 or later is required by kernel-doc $ conda deactivate
$ conda create -n py34 python=3.4
$ conda activate py34
python --version
Python 3.4.5
$ python ./scripts/kernel-doc --none include/media
Error: Python 3.6 or later is required by kernel-doc
$ conda deactivate $ python --version
Python 3.13.5
$ python ./scripts/kernel-doc --none include/media
(no warnings and script ran properly)--- scripts/kernel-doc.py | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-)
scripts/kernel-doc.py +26 -11
@@ -1,9 +1,19 @@ -#!/usr/bin/env python3 +#!/usr/bin/env python +# -*- coding: utf-8 -*- # SPDX-License-Identifier: GPL-2.0 # Copyright(c) 2025: Mauro Carvalho Chehab <[email protected]>. # -# pylint: disable=C0103,R0915 -# +# pylint: disable=C0103,R0912,R0914,R0915 + +# NOTE: While kernel-doc requires at least version 3.6 to run, the +# command line explicitly supports Python 2,7 and 3.4+. +# The rationale is that it shall fail gracefully during Kernel +# compilation with older Kernel versions. Due to that: +# - encoding line is needed here; +# - no f-strings can be used on this file. +# - the libraries that require newer versions can only be included +# after Python version is checked. + # Converted from the kernel-doc script originally written in Perl # under GPLv2, copyrighted since 1998 by the following authors: #
@@ -107,9 +117,6 @@ SRC_DIR = os.path.dirname(os.path.realpath(__file__)) sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) -from kdoc_files import KernelFiles # pylint: disable=C0413 -from kdoc_output import RestFormat, ManFormat # pylint: disable=C0413 - DESC = """ Read C language source or header FILEs, extract embedded documentation comments, and print formatted documentation to standard output.
@@ -273,14 +280,22 @@ def main(): python_ver = sys.version_info[:2] if python_ver < (3,6): - logger.warning("Python 3.6 or later is required by kernel-doc") + # Depending on Kernel configuration, kernel-doc --none is called at + # build time. As we don't want to break compilation due to the + # usage of an old Python version, return 0 here. + if args.none: + logger.error("Python 3.6 or later is required by kernel-doc. skipping checks") + sys.exit(0) - # Return 0 here to avoid breaking compilation - sys.exit(0) + sys.exit("Python 3.6 or later is required by kernel-doc. Aborting.") if python_ver < (3,7): logger.warning("Python 3.7 or later is required for correct results") + # Import kernel-doc libraries only after checking Python version + from kdoc_files import KernelFiles # pylint: disable=C0415 + from kdoc_output import RestFormat, ManFormat # pylint: disable=C0415 + if args.man: out_style = ManFormat(modulename=args.modulename) elif args.none:
@@ -308,11 +323,11 @@ def main(): sys.exit(0) if args.werror: - print(f"{error_count} warnings as errors") + print("%s warnings as errors" % error_count) # pylint: disable=C0209 sys.exit(error_count) if args.verbose: - print(f"{error_count} errors") + print("%s errors" % error_count) # pylint: disable=C0209 if args.none: sys.exit(0)
Signature · 1 line
2.49.0
Re: [PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python
Hi,
Quoted 3 lines · Mauro Carvalho Chehab
While we do need at least 3.6 for kernel-doc to work, and at least 3.7 for it to output functions and structs with parameters at the right order, let the python binary be compatible with legacy
code?
Quoted 2 lines
versions.
[...]
scripts/kernel-doc.py line 1
Quoted 13 lines
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- scripts/kernel-doc.py | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-)scripts/kernel-doc.py +1
-1@@ -1,9 +1,19 @@ -#!/usr/bin/env python3 +#!/usr/bin/env python
This would conflict with my existing setup which has /usr/bin/python3 only.
Please don't do this!
Thanks, Akira
Re: [PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python
Quoted 28 lines · Akira Yokosawa
Hi,
Quoted 3 lines · Mauro Carvalho Chehab
While we do need at least 3.6 for kernel-doc to work, and at least 3.7 for it to output functions and structs with parameters at the right order, let the python binary be compatible with legacy
code?
Quoted 2 lines
versions.
[...]
scripts/kernel-doc.py line 1
Quoted 13 lines
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- scripts/kernel-doc.py | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-)scripts/kernel-doc.py +1
-1@@ -1,9 +1,19 @@ -#!/usr/bin/env python3 +#!/usr/bin/env pythonThis would conflict with my existing setup which has
/usr/bin/python3only.
Quoted 1 line
Please don't do this!
Agreed, this breaks more than it fixes.
Python 2.7 reached end-of-life over five years ago. Do we really have to cater for ancient stuff? Which actual real world cases do not have Python 3+ available? Please just let it go, and see if anyone ever notices?
BR,
Jani.
Signature · 1 line
Jani Nikula, Intel
Jonathan Corbet
maintainerRe: [PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python
Quoted 6 lines · Jani Nikula
Agreed, this breaks more than it fixes.
Python 2.7 reached end-of-life over five years ago. Do we really have to cater for ancient stuff? Which actual real world cases do not have Python 3+ available? Please just let it go, and see if anyone ever notices?
I kind of have to agree. The only real Python 2 user I know about is OpenOffice.org, which promises to get off any year now. Meanwhile, as I recall, the advice from the Python project is to say "python3" and not count on bare "python" being available.
jon
PATCH 2/2 docs: changes: better document Python needs
In reply to Mauro Carvalho ChehabPython is listed as an optional dependency, but this is not true, as:
1) CONFIG_LTO_CLANG runs a python script at scripts/Makefile.vmlinux_o;
2) kernel-doc is called during compilation when some DRM options like CONFIG_DRM_I915_WERROR are enabled;
3) allyesconfig/allmodconfig will enable both.
So, better document that.
--- Documentation/process/changes.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
Documentation/process/changes.rst +12 -1
@@ -61,7 +61,7 @@ Sphinx\ [#f1]_ 3.4.3 sphinx-build --version GNU tar 1.28 tar --version gtags (optional) 6.6.5 gtags --version mkimage (optional) 2017.01 mkimage --version -Python (optional) 3.9.x python3 --version +Python 3.9.x python3 --version GNU AWK (optional) 5.1.0 gawk --version ====================== =============== ========================================
@@ -154,6 +154,17 @@ Perl You will need perl 5 and the following modules: ``Getopt::Long``, ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel. +Python +------ + +CONFIG_LTO_CLANG requires python 2.7 or 3.0+; some DRM config options like +CONFIG_DRM_I915_WERROR require at least Python 2.7 or 3.4+. + +The kernel-doc tool and docs build require at least 3.6, but they depend on +dict changes that happened on 3.7 to produce valid results. + +Other tools within the Kernel tree require newer versions. + BC --
Signature · 1 line
2.49.0
Re: [PATCH 2/2] docs: changes: better document Python needs
[+CC Laurent and Jani]
Hi,
Quoted 9 lines · Mauro Carvalho Chehab
Python is listed as an optional dependency, but this is not true, as:
1)
CONFIG_LTO_CLANGruns a python script atscripts/Makefile.vmlinux_o;2) kernel-doc is called during compilation when some DRM options like
CONFIG_DRM_I915_WERRORare enabled;3) allyesconfig/allmodconfig will enable both.
Well, these conditions still sound to me optional.
Quoted 26 lines
So, better document that.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- Documentation/process/changes.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)Documentation/process/changes.rst +2
-1@@ -61,7 +61,7 @@ Sphinx\ [#f1]_ 3.4.3 sphinx-build --version GNU tar 1.28 tar --version gtags (optional) 6.6.5 gtags --version mkimage (optional) 2017.01 mkimage --version -Python (optional) 3.9.x python3 --version +Python 3.9.x python3 --version GNU AWK (optional) 5.1.0 gawk --version ====================== =============== ========================================@@ -154,6 +154,17 @@ Perl You will need perl 5 and the following modules: ``Getopt::Long``, ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel. +Python
Documentation/process/changes.rst Perl · line 167
Quoted 10 lines
Documentation/process/changes.rst +10
-0+------ + +CONFIG_LTO_CLANG requires python 2.7 or 3.0+; some DRM config options like +CONFIG_DRM_I915_WERROR require at least Python 2.7 or 3.4+. + +The kernel-doc tool and docs build require at least 3.6, but they depend on +dict changes that happened on 3.7 to produce valid results. + +Other tools within the Kernel tree require newer versions. +
These details look confusing in changes.rst. The table above says python >=3.9.x. All you need to say here would just something lile:
+Python
+------
+
+Required with CONFIG_LTO_CLANG, some DRM config options like +CONFIG_DRM_I915_WERROR, the kernel-doc tool, and docs build (Sphinx), +among others.
Other details can go to comments in each script or other docs if necessary.
Thanks, Akira
Re: [PATCH 2/2] docs: changes: better document Python needs
Em Tue, 29 Jul 2025 19:35:57 +0900
Akira Yokosawa <[email protected]> escreveu:
Quoted 8 lines
[+CC Laurent and Jani]
Hi,
Quoted 3 lines · Mauro Carvalho Chehab
Python is listed as an optional dependency, but this is not true, as:
Quoted 8 lines
Quoted 6 lines
1)
CONFIG_LTO_CLANGruns a python script atscripts/Makefile.vmlinux_o;2) kernel-doc is called during compilation when some DRM options like
CONFIG_DRM_I915_WERRORare enabled;3) allyesconfig/allmodconfig will enable both.
Well, these conditions still sound to me optional.
Then we agree to disagree: if a missing package prevents building the kernel with allyesconfig/allmodconfig, IMO it is mandatory.
In any case, as Jani pointed out, DRM_MSM, which is part of arm default configs also require Python. Building with clang also requires it on several archs. There are other parts that seem to require it, but I didn't dig into it.
So, IMHO, even if you consider allmodconfig/allyesconfig as "optional", and you also consider clang as "optional" (it is marked as optional at changes), it is still mandatory for arm 32-bit most common config (multi_v7_defconfig) and for arm64 defconfig.
It sounds pretty much mandatory to me: if a distro doesn't have it, it can't support arm architectures.
IMO, it should not be tagged as optional there. At max, a note under Python description could mention what scenarios one still may not have it.
Now, adding a note like that means that someone would need to be periodically updating it when people add more auto-generated stuff that may require it.
Quoted 46 lines
Quoted 25 lines
So, better document that.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- Documentation/process/changes.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)Documentation/process/changes.rst +2
-1@@ -61,7 +61,7 @@ Sphinx\ [#f1]_ 3.4.3 sphinx-build --version GNU tar 1.28 tar --version gtags (optional) 6.6.5 gtags --version mkimage (optional) 2017.01 mkimage --version -Python (optional) 3.9.x python3 --version +Python 3.9.x python3 --version GNU AWK (optional) 5.1.0 gawk --version ====================== =============== ========================================@@ -154,6 +154,17 @@ Perl You will need perl 5 and the following modules: ``Getopt::Long``, ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel. +PythonDocumentation/process/changes.rst Perl · line 167
Quoted 10 lines
Documentation/process/changes.rst +10
-0+------ + +CONFIG_LTO_CLANG requires python 2.7 or 3.0+; some DRM config options like +CONFIG_DRM_I915_WERROR require at least Python 2.7 or 3.4+. + +The kernel-doc tool and docs build require at least 3.6, but they depend on +dict changes that happened on 3.7 to produce valid results. + +Other tools within the Kernel tree require newer versions. +These details look confusing in
changes.rst. The table above says python >=3.9.x. All you need to say here would just something lile:Documentation/process/changes.rst +6
-0+Python +------ + +Required with CONFIG_LTO_CLANG, some DRM config options like +CONFIG_DRM_I915_WERROR, the kernel-doc tool, and docs build (Sphinx), +among others.
Quoted 2 lines
Other details can go to comments in each script or other docs if necessary.
Maybe something like:
Python
------
Several config options require it: it is required for arm/arm64 default configs, CONFIG_LTO_CLANG, some DRM optional configs, the kernel-doc tool, and docs build (Sphinx), among others.
Regards,
Mauro