kernel-doc improvements
Subsystems (2)
cover kernel-doc improvements
Hi Jon,
This series contains some patches that were submitted originally on this /41 patch series:
https://patchew.org/linux/[email protected]/fc6723d13b96db014eaf0f14354d8821ea2085b8.1769867954.git.mchehab+huawei@kernel.org/
It consists on two parts:
- the first two patches are meant to better allow using kernel-doc inside QEMU and to improve KernelFiles ABI, which will be used there. I'm placing it here just to avoid on extra PR with just two patches on it.
- the second part do some improvements on how man pages are produced.
The second part is related to man pages. It is focused mostly on TH, but, as a side effect, it also change the name of man pages generated from DOC kernel-doc annotations.
The rationale is that modern troff/man page specs say that .TH has up to 5 arguments,, as defined at [1]:
.TH topic section [footer-middle] [footer-inside] [header-middle]
[1] https://man7.org/linux/man-pages/man7/groff_man_style.7.html
Right now, Kernel uses 6 arguments, probably due to some legacy man page definitions.
After double checking, this is typically used like this:
.TH "{name}" {section} "{date}" "{modulename}" "{manual}"Right now, man pages generation are messing up on how it encodes each position at .TH, depending on the type of object it emits.
After this series, the definition is more consistent and file output is better named.
It also fixes two issues at sphinx-build-wrapper related to how it generate files names from the .TH header.
This series hould not affect HTML documentation. It only changes man pages.
As the names are now better defined, it also prevents some file name duplication.
Mauro Carvalho Chehab (11):
docs: kdoc_files: allows the caller to use a different xforms class
docs: kdoc_files: document KernelFiles() ABI
docs: sphinx-build-wrapper: better handle troff .TH markups
docs: sphinx-build-wrapper: don't allow "/" on file names
docs: kdoc_output: use a method to emit the .TH header
docs: kdoc_output: remove extra attribute on man .TH headers
docs: kdoc_output: use a single manual for everything
docs: kdoc_output: don't use a different modulename for functions
docs: kdoc_output: use a more standard order for .TH on man pages
docs: kdoc_output: describe the class init parameters
docs: kdoc_output: pick a better default for modulename
tools/docs/kernel-doc | 1 - tools/docs/sphinx-build-wrapper | 9 ++-- tools/lib/python/kdoc/kdoc_files.py | 53 ++++++++++++++++++-- tools/lib/python/kdoc/kdoc_output.py | 73 +++++++++++++++++++++++----- 4 files changed, 115 insertions(+), 21 deletions(-)
Signature · 1 line
2.52.0
PATCH 1/11 docs: kdoc_files: allows the caller to use a different xforms class
While the main goal for kernel-doc is to be used inside the Linux Kernel, other open source projects could benefit for it. That's currently the case of QEMU, which has a fork, mainly due to two reasons:
- they need an extra C function transform rule;
- they handle the html output a little bit different.
Add an extra optional argument to make easier for the code to be shared, as, with that, QEMU can just create a new derivated class that will contain its specific rulesets, and just copy the remaining kernel-doc files as-is.
--- tools/lib/python/kdoc/kdoc_files.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
tools/lib/python/kdoc/kdoc_files.py +7 -2
@@ -118,7 +118,7 @@ class KernelFiles(): if fname in self.files: return - doc = KernelDoc(self.config, fname, CTransforms()) + doc = KernelDoc(self.config, fname, self.xforms) export_table, entries = doc.parse_kdoc() self.export_table[fname] = export_table
@@ -154,7 +154,7 @@ class KernelFiles(): self.error(f"Cannot find file {fname}") - def __init__(self, verbose=False, out_style=None, + def __init__(self, verbose=False, out_style=None, xforms=None, werror=False, wreturn=False, wshort_desc=False, wcontents_before_sections=False, logger=None):
@@ -193,6 +193,11 @@ class KernelFiles(): self.config.wshort_desc = wshort_desc self.config.wcontents_before_sections = wcontents_before_sections + if xforms: + self.xforms = xforms + else: + self.xforms = CTransforms() + if not logger: self.config.log = logging.getLogger("kernel-doc") else:
Signature · 1 line
2.52.0
PATCH 2/11 docs: kdoc_files: document KernelFiles() ABI
In reply to Mauro Carvalho ChehabThe KernelFiles is the main entry point to run kernel-doc, being used by both tools/docs/kernel-doc and Documentation/sphinx/kerneldoc.py.
It is also used on QEMU, which also uses the kernel-doc libraries from tools/lib/python/kdoc.
Properly describe its ABI contract.
--- tools/lib/python/kdoc/kdoc_files.py | 44 ++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_files.py +43 -1
@@ -91,7 +91,49 @@ class KernelFiles(): """ Parse kernel-doc tags on multiple kernel source files. - There are two type of parsers defined here: + This is the main entry point to run kernel-doc. This class is initialized + using a series of optional arguments: + + ``verbose`` + If True, enables kernel-doc verbosity. Default: False. + + ``out_style`` + Class to be used to format output. If None (default), + only report errors. + + ``xforms`` + Transforms to be applied to C prototypes and data structs. + If not specified, defaults to xforms = CFunction() + + ``werror`` + If True, treat warnings as errors, retuning an error code on warnings. + + Default: False. + + ``wreturn`` + If True, warns about the lack of a return markup on functions. + + Default: False. + ``wshort_desc`` + If True, warns if initial short description is missing. + + Default: False. + + ``wcontents_before_sections`` + If True, warn if there are contents before sections (deprecated). + This option is kept just for backward-compatibility, but it does + nothing, neither here nor at the original Perl script. + + Default: False. + + ``logger`` + Optional logger class instance. + + If not specified, defaults to use: ``logging.getLogger("kernel-doc")`` + + Note: + There are two type of parsers defined here: + - self.parse_file(): parses both kernel-doc markups and ``EXPORT_SYMBOL*`` macros; - self.process_export_file(): parses only ``EXPORT_SYMBOL*`` macros.
Signature · 1 line
2.52.0
PATCH 3/11 docs: sphinx-build-wrapper: better handle troff .TH markups
In reply to Mauro Carvalho ChehabUsing a regular expression to match .TH is problematic, as it doesn't handle well quotation marks.
Use shlex instead.
--- tools/docs/sphinx-build-wrapper | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tools/docs/sphinx-build-wrapper +4 -4
@@ -576,7 +576,6 @@ class SphinxBuilder: """ re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)") - re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"') if docs_dir == src_dir: #
@@ -616,8 +615,7 @@ class SphinxBuilder: fp = None try: for line in result.stdout.split("\n"): - match = re_man.match(line) - if not match: + if not line.startswith(".TH"): if fp: fp.write(line + '\n') continue
@@ -625,7 +623,9 @@ class SphinxBuilder: if fp: fp.close() - fname = f"{output_dir}/{match.group(2)}.{match.group(1)}" + # Use shlex here, as it handles well parameters with commas + args = shlex.split(line) + fname = f"{output_dir}/{args[3]}.{args[2]}" if self.verbose: print(f"Creating {fname}")
Signature · 1 line
2.52.0
PATCH 4/11 docs: sphinx-build-wrapper: don't allow "/" on file names
In reply to Mauro Carvalho ChehabWhen handling "DOC:" sections, slash characters may be there. Prevent using it at the file names, as this is used for directory separator.
--- tools/docs/sphinx-build-wrapper | 1 + 1 file changed, 1 insertion(+)
tools/docs/sphinx-build-wrapper +1 -0
@@ -625,6 +625,7 @@ class SphinxBuilder: # Use shlex here, as it handles well parameters with commas args = shlex.split(line) + name = args[1].replace("/", " ") fname = f"{output_dir}/{args[3]}.{args[2]}" if self.verbose:
Signature · 1 line
2.52.0
PATCH 5/11 docs: kdoc_output: use a method to emit the .TH header
In reply to Mauro Carvalho ChehabAll man emit functions need to add a .TH header. Move the code to a common function, as we'll be addressing some issues at the common code.
--- tools/lib/python/kdoc/kdoc_output.py | 34 +++++++++++++++++++++------- 1 file changed, 26 insertions(+), 8 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +26 -8
@@ -607,7 +607,20 @@ class ManFormat(OutputFormat): "%m %d %Y", ] - def __init__(self, modulename): + def emit_th(self, name, modulename = None, manual=None): + """Emit a title header line.""" + name = name.strip() + + if not manual: + manual = self.manual + + if not modulename: + modulename = self.modulename + + self.data += f'.TH "{modulename}" {self.section} "{name}" ' + self.data += f'"{self.date}" "{manual}" LINUX\n' + + def __init__(self, modulename, section="9", manual="API Manual"): """ Creates class variables.
@@ -616,7 +629,11 @@ class ManFormat(OutputFormat): """ super().__init__() + self.modulename = modulename + self.section = section + self.manual = manual + self.symbols = [] dt = None
@@ -632,7 +649,7 @@ class ManFormat(OutputFormat): if not dt: dt = datetime.now() - self.man_date = dt.strftime("%B %Y") + self.date = dt.strftime("%B %Y") def arg_name(self, args, name): """
@@ -724,7 +741,7 @@ class ManFormat(OutputFormat): out_name = self.arg_name(args, name) - self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + self.emit_th(out_name) for section, text in args.sections.items(): self.data += f'.SH "{section}"' + "\n"
@@ -734,7 +751,8 @@ class ManFormat(OutputFormat): out_name = self.arg_name(args, name) - self.data += f'.TH "{name}" 9 "{out_name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n" + self.emit_th(out_name, modulename = name, + manual="Kernel Hacker\'s Manual") self.data += ".SH NAME\n" self.data += f"{name} \\- {args['purpose']}\n"
@@ -780,7 +798,7 @@ class ManFormat(OutputFormat): def out_enum(self, fname, name, args): out_name = self.arg_name(args, name) - self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + self.emit_th(out_name) self.data += ".SH NAME\n" self.data += f"enum {name} \\- {args['purpose']}\n"
@@ -813,7 +831,7 @@ class ManFormat(OutputFormat): out_name = self.arg_name(args, name) full_proto = args.other_stuff["full_proto"] - self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + self.emit_th(out_name) self.data += ".SH NAME\n" self.data += f"{name} \\- {args['purpose']}\n"
@@ -834,7 +852,7 @@ class ManFormat(OutputFormat): purpose = args.get('purpose') out_name = self.arg_name(args, name) - self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + self.emit_th(out_name) self.data += ".SH NAME\n" self.data += f"typedef {name} \\- {purpose}\n"
@@ -849,7 +867,7 @@ class ManFormat(OutputFormat): definition = args.get('definition') out_name = self.arg_name(args, name) - self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + self.emit_th(out_name) self.data += ".SH NAME\n" self.data += f"{args.type} {name} \\- {purpose}\n"
Signature · 1 line
2.52.0
PATCH 6/11 docs: kdoc_output: remove extra attribute on man .TH headers
In reply to Mauro Carvalho ChehabAccording with modern documents, groff .TH supports up to 5 arguments, but the logic passes 6. Drop the lastest one ("LINUX").
--- tools/lib/python/kdoc/kdoc_output.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_output.py +1 -1
@@ -618,7 +618,7 @@ class ManFormat(OutputFormat): modulename = self.modulename self.data += f'.TH "{modulename}" {self.section} "{name}" ' - self.data += f'"{self.date}" "{manual}" LINUX\n' + self.data += f'"{self.date}" "{manual}"\n' def __init__(self, modulename, section="9", manual="API Manual"): """
Signature · 1 line
2.52.0
PATCH 7/11 docs: kdoc_output: use a single manual for everything
In reply to Mauro Carvalho ChehabThere's no reason why functions will be on a different manual. Unify its name, calling it as "Kernel API Manual".
--- tools/lib/python/kdoc/kdoc_output.py | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +4 -8
@@ -607,20 +607,17 @@ class ManFormat(OutputFormat): "%m %d %Y", ] - def emit_th(self, name, modulename = None, manual=None): + def emit_th(self, name, modulename = None): """Emit a title header line.""" name = name.strip() - if not manual: - manual = self.manual - if not modulename: modulename = self.modulename self.data += f'.TH "{modulename}" {self.section} "{name}" ' - self.data += f'"{self.date}" "{manual}"\n' + self.data += f'"{self.date}" "{self.manual}"\n' - def __init__(self, modulename, section="9", manual="API Manual"): + def __init__(self, modulename, section="9", manual="Kernel API Manual"): """ Creates class variables.
@@ -751,8 +748,7 @@ class ManFormat(OutputFormat): out_name = self.arg_name(args, name) - self.emit_th(out_name, modulename = name, - manual="Kernel Hacker\'s Manual") + self.emit_th(out_name, modulename = name) self.data += ".SH NAME\n" self.data += f"{name} \\- {args['purpose']}\n"
Signature · 1 line
2.52.0
PATCH 8/11 docs: kdoc_output: don't use a different modulename for functions
In reply to Mauro Carvalho ChehabIt doesn't make much sense to have a different modulename just for functions, but not for structs/enums/...
Use the same header everywere.
--- tools/lib/python/kdoc/kdoc_output.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +3 -6
@@ -607,14 +607,11 @@ class ManFormat(OutputFormat): "%m %d %Y", ] - def emit_th(self, name, modulename = None): + def emit_th(self, name): """Emit a title header line.""" name = name.strip() - if not modulename: - modulename = self.modulename - - self.data += f'.TH "{modulename}" {self.section} "{name}" ' + self.data += f'.TH "{self.modulename}" {self.section} "{name}" ' self.data += f'"{self.date}" "{self.manual}"\n' def __init__(self, modulename, section="9", manual="Kernel API Manual"):
@@ -748,7 +745,7 @@ class ManFormat(OutputFormat): out_name = self.arg_name(args, name) - self.emit_th(out_name, modulename = name) + self.emit_th(out_name) self.data += ".SH NAME\n" self.data += f"{name} \\- {args['purpose']}\n"
Signature · 1 line
2.52.0
PATCH 9/11 docs: kdoc_output: use a more standard order for .TH on man pages
In reply to Mauro Carvalho ChehabThe generated man pages are not following the current standards for Linux documentation. Reorder .TH fields for them to look like other Linux man pages.
--- tools/docs/sphinx-build-wrapper | 2 +- tools/lib/python/kdoc/kdoc_output.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
tools/docs/sphinx-build-wrapper +1 -1
@@ -626,7 +626,7 @@ class SphinxBuilder: # Use shlex here, as it handles well parameters with commas args = shlex.split(line) name = args[1].replace("/", " ") - fname = f"{output_dir}/{args[3]}.{args[2]}" + fname = f"{output_dir}/{args[1]}.{args[2]}" if self.verbose: print(f"Creating {fname}")
tools/lib/python/kdoc/kdoc_output.py +2 -2
@@ -611,8 +611,8 @@ class ManFormat(OutputFormat): """Emit a title header line.""" name = name.strip() - self.data += f'.TH "{self.modulename}" {self.section} "{name}" ' - self.data += f'"{self.date}" "{self.manual}"\n' + self.data += f'.TH "{name}" {self.section} "{self.date}" ' + self.data += f'"{self.modulename}" "{self.manual}"\n' def __init__(self, modulename, section="9", manual="Kernel API Manual"): """
Signature · 1 line
2.52.0
PATCH 10/11 docs: kdoc_output: describe the class init parameters
In reply to Mauro Carvalho ChehabAs this class is part of the ABI used by both Sphinx kerneldoc extension and docs/tools/kernel-doc, better describe what parmeters are used to initialize ManOutput class.
--- tools/lib/python/kdoc/kdoc_output.py | 29 +++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_output.py +28 -1
@@ -580,7 +580,34 @@ class RestFormat(OutputFormat): class ManFormat(OutputFormat): - """Consts and functions used by man pages output.""" + """ + Consts and functions used by man pages output. + + This class has one mandatory parameter and some optional ones, which + are needed to define the title header contents: + + ``modulename`` + Defines the module name to be used at the troff ``.TH`` output. + + This argument is mandatory. + + ``section`` + Usually a numeric value from 0 to 9, but man pages also accept + some strings like "p". + + Defauls to ``9`` + + ``manual`` + Defaults to ``Kernel API Manual``. + + The above controls the output of teh corresponding fields on troff + title headers, which will be filled like this:: + + .TH "{name}" {section} "{date}" "{modulename}" "{manual}" + + where ``name``` will match the API symbol name, and ``date`` will be + either the date where the Kernel was compiled or the current date + """ highlights = ( (type_constant, r"\1"),
Signature · 1 line
2.52.0
PATCH 11/11 docs: kdoc_output: pick a better default for modulename
In reply to Mauro Carvalho ChehabInstead of placing the same data for modulename for all generated man pages, use the directory from the filename used to produce kernel docs as basis.
--- tools/docs/kernel-doc | 1 - tools/lib/python/kdoc/kdoc_output.py | 41 +++++++++++++++++----------- 2 files changed, 25 insertions(+), 17 deletions(-)
tools/docs/kernel-doc +0 -1
@@ -210,7 +210,6 @@ def main(): help="Enable debug messages") parser.add_argument("-M", "-modulename", "--modulename", - default="Kernel API", help="Allow setting a module name at the output.") parser.add_argument("-l", "-enable-lineno", "--enable_lineno",
tools/lib/python/kdoc/kdoc_output.py +25 -16
111 lines not on this page · raw message
Signature · 1 line
2.52.0
Quoted 6 lines · Mauro Carvalho Chehab
Hi Jon,
This series contains some patches that were submitted originally on this /41 patch series:
https://patchew.org/linux/[email protected]/fc6723d13b96db014eaf0f14354d8821ea2085b8.1769867954.git.mchehab+huawei@kernel.org/
Hi Jon,
Don't apply this one. It ends I lost something among all rebases this series passed thoug on this patch:
fcf10d80e7a5 ("docs: kdoc_output: use a more standard order for .TH on man pages")
I'll be sending a version 2 later after sorting things out.
Signature · 2 lines
Thanks, Mauro