kernel-doc: add support for documenting vars
Subsystems (2)
v5 cover kernel-doc: add support for documenting vars
Hi Jon,
As suggested and discussed with Randy, this small series add support for documenting variables using kernel-doc.
- patch 1: add support for the new feature;
- patch 2: extends to support DEFINE_*;
- patch 3: document two media vars;
- patch 4: fix an issue on kernel-doc.rst markups and automarkup;
- patch 5: document it.
On this version, I'm using "c:macro" to describe variables, as it avoids Sphinx C domain to try parse the variable. This makes it more flexible and easier to maintain in long term.
---
v5:
- use ``foo`` for prototype and on patch 3.
v4:
- document the new markup;
- fix an issue on kernel-doc.rst due to automarkup;
- add support for DEFINE_* macros
Mauro Carvalho Chehab (5):
kernel-doc: add support for handling global variables
kernel-doc: add support to handle DEFINE_ variables
docs: media: v4l2-ioctl.h: document two global variables
docs: kernel-doc.rst: don't let automarkup mangle with consts
docs: kernel-doc.rst: document the new "var" kernel-doc markup
Documentation/doc-guide/kernel-doc.rst | 48 +++++++++++------ include/media/v4l2-ioctl.h | 15 ++++++ tools/lib/python/kdoc/kdoc_output.py | 46 ++++++++++++++++ tools/lib/python/kdoc/kdoc_parser.py | 73 +++++++++++++++++++++++++- 4 files changed, 166 insertions(+), 16 deletions(-)
Signature · 1 line
2.51.1
PATCH v5 1/5 kernel-doc: add support for handling global variables
Specially on kAPI, sometimes it is desirable to be able to describe global variables that are part of kAPI.
Documenting vars with Sphinx is simple, as we don't need to parse a data struct. All we need is the variable declaration and use natice C domain ::c:var: to format it for us.
Add support for it.
--- tools/lib/python/kdoc/kdoc_output.py | 47 +++++++++++++++++++++++ tools/lib/python/kdoc/kdoc_parser.py | 56 +++++++++++++++++++++++++++- 2 files changed, 102 insertions(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_output.py +47 -0
@@ -199,6 +199,10 @@ class OutputFormat: self.out_enum(fname, name, args) return self.data + if dtype == "var": + self.out_var(fname, name, args) + return self.data + if dtype == "typedef": self.out_typedef(fname, name, args) return self.data
@@ -227,6 +231,9 @@ class OutputFormat: def out_enum(self, fname, name, args): """Outputs an enum""" + def out_var(self, fname, name, args): + """Outputs a variable""" + def out_typedef(self, fname, name, args): """Outputs a typedef"""
@@ -472,6 +479,25 @@ class RestFormat(OutputFormat): self.lineprefix = oldprefix self.out_section(args) + def out_var(self, fname, name, args): + oldprefix = self.lineprefix + ln = args.declaration_start_line + full_proto = args.other_stuff["full_proto"] + + self.lineprefix = " " + + self.data += f"\n\n.. c:macro:: {name}\n\n{self.lineprefix}{full_proto}\n\n" + + self.print_lineno(ln) + self.output_highlight(args.get('purpose', '')) + self.data += "\n" + + if args.other_stuff["default_val"]: + self.data += f'{self.lineprefix}**Initialization**\n\n' + self.output_highlight(f'default: ``{args.other_stuff["default_val"]}``') + + self.out_section(args) + def out_typedef(self, fname, name, args): oldprefix = self.lineprefix
@@ -773,6 +799,27 @@ class ManFormat(OutputFormat): self.data += f'.SH "{section}"' + "\n" self.output_highlight(text) + def out_var(self, fname, name, args): + out_name = self.arg_name(args, name) + prototype = args.other_stuff["var_type"] + full_proto = args.other_stuff["full_proto"] + + self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + + self.data += ".SH NAME\n" + self.data += f"{prototype} \\- {args['purpose']}\n" + + self.data += ".SH SYNOPSIS\n" + self.data += f"{full_proto}\n" + + if args.other_stuff["default_val"]: + self.data += f'.SH "Initialization"' + "\n" + self.output_highlight(f'default: {args.other_stuff["default_val"]}') + + for section, text in args.sections.items(): + self.data += f'.SH "{section}"' + "\n" + self.output_highlight(text) + def out_typedef(self, fname, name, args): module = self.modulename purpose = args.get('purpose')
tools/lib/python/kdoc/kdoc_parser.py +55 -1
74 lines not on this page · raw message
Signature · 1 line
2.51.1
Re: [PATCH v5 1/5] kernel-doc: add support for handling global variables
Quoted 6 lines · Mauro Carvalho Chehab
Specially on kAPI, sometimes it is desirable to be able to describe global variables that are part of kAPI.
Documenting vars with Sphinx is simple, as we don't need to parse a data struct. All we need is the variable declaration and use natice C domain ::c:var: to format it
native
Quoted 7 lines
for us.
Add support for it.
Thanks.
Quoted 165 lines
--- tools/lib/python/kdoc/kdoc_output.py | 47 +++++++++++++++++++++++ tools/lib/python/kdoc/kdoc_parser.py | 56 +++++++++++++++++++++++++++- 2 files changed, 102 insertions(+), 1 deletion(-)tools/lib/python/kdoc/kdoc_output.py +47
-0@@ -199,6 +199,10 @@ class OutputFormat: self.out_enum(fname, name, args) return self.data + if dtype == "var": + self.out_var(fname, name, args) + return self.data + if dtype == "typedef": self.out_typedef(fname, name, args) return self.data@@ -227,6 +231,9 @@ class OutputFormat: def out_enum(self, fname, name, args): """Outputs an enum""" + def out_var(self, fname, name, args): + """Outputs a variable""" + def out_typedef(self, fname, name, args): """Outputs a typedef"""@@ -472,6 +479,25 @@ class RestFormat(OutputFormat): self.lineprefix = oldprefix self.out_section(args) + def out_var(self, fname, name, args): + oldprefix = self.lineprefix + ln = args.declaration_start_line + full_proto = args.other_stuff["full_proto"] + + self.lineprefix = " " + + self.data += f"\n\n.. c:macro:: {name}\n\n{self.lineprefix}{full_proto}\n\n" + + self.print_lineno(ln) + self.output_highlight(args.get('purpose', '')) + self.data += "\n" + + if args.other_stuff["default_val"]: + self.data += f'{self.lineprefix}**Initialization**\n\n' + self.output_highlight(f'default: ``{args.other_stuff["default_val"]}``') + + self.out_section(args) + def out_typedef(self, fname, name, args): oldprefix = self.lineprefix@@ -773,6 +799,27 @@ class ManFormat(OutputFormat): self.data += f'.SH "{section}"' + "\n" self.output_highlight(text) + def out_var(self, fname, name, args): + out_name = self.arg_name(args, name) + prototype = args.other_stuff["var_type"] + full_proto = args.other_stuff["full_proto"] + + self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" + + self.data += ".SH NAME\n" + self.data += f"{prototype} \\- {args['purpose']}\n" + + self.data += ".SH SYNOPSIS\n" + self.data += f"{full_proto}\n" + + if args.other_stuff["default_val"]: + self.data += f'.SH "Initialization"' + "\n" + self.output_highlight(f'default: {args.other_stuff["default_val"]}') + + for section, text in args.sections.items(): + self.data += f'.SH "{section}"' + "\n" + self.output_highlight(text) + def out_typedef(self, fname, name, args): module = self.modulename purpose = args.get('purpose')tools/lib/python/kdoc/kdoc_parser.py +55
-1@@ -64,7 +64,7 @@ type_param = KernRe(r"@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=False) # Tests for the beginning of a kerneldoc block in its various forms. # doc_block = doc_com + KernRe(r'DOC:\s*(.*)?', cache=False) -doc_begin_data = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)", cache = False) +doc_begin_data = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef|var)\b\s*(\w*)", cache = False) doc_begin_func = KernRe(str(doc_com) + # initial " * ' r"(?:\w+\s*\*\s*)?" + # type (not captured) r'(?:define\s+)?' + # possible "define" (not captured)@@ -924,6 +924,58 @@ class KernelDoc: self.output_declaration('enum', declaration_name, purpose=self.entry.declaration_purpose) + def dump_var(self, ln, proto): + """ + Store variables that are part of kAPI. + """ + VAR_ATTRIBS = [ + "extern", + ] + OPTIONAL_VAR_ATTR = "^(?:" + "|".join(VAR_ATTRIBS) + ")?" + + sub_prefixes = [ + (KernRe(r"__read_mostly"), ""), + (KernRe(r"__ro_after_init"), ""), + (KernRe(r"(?://.*)$"), ""), + (KernRe(r"(?:/\*.*\*/)"), ""), + (KernRe(r";$"), ""), + (KernRe(r"=.*"), ""), + ] + + # + # Store the full prototype before modifying it + # + full_proto = proto + + # + # Drop comments and macros to have a pure C prototype + # + for search, sub in sub_prefixes: + proto = search.sub(sub, proto) + + proto = proto.rstrip() + + # + # Variable name is at the end of the declaration + # + + r= KernRe(OPTIONAL_VAR_ATTR + r"\w.*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") + if not r.match(proto): + self.emit_msg(ln,f"{proto}: can't parse variable") + return + + var_type = r.group(0) + declaration_name = r.group(1) + default_val = r.group(2) + if default_val: + default_val = default_val.lstrip("=").strip() + + self.output_declaration("var", declaration_name, + full_proto=full_proto, + var_type=var_type, + default_val=default_val, + purpose=self.entry.declaration_purpose) + def dump_declaration(self, ln, prototype): """ Stores a data declaration inside self.entries array.@@ -935,6 +987,8 @@ class KernelDoc: self.dump_typedef(ln, prototype) elif self.entry.decl_type in ["union", "struct"]: self.dump_struct(ln, prototype) + elif self.entry.decl_type == "var": + self.dump_var(ln, prototype) else: # This would be a bug self.emit_message(ln, f'Unknown declaration type: {self.entry.decl_type}')
Signature · 1 line
~Randy
PATCH v5 2/5 kernel-doc: add support to handle DEFINE_ variables
In reply to Mauro Carvalho ChehabImprove the parser and output plugin to work with macros, adding support for the common pattern of using DEFINE_* to create variables.
--- tools/lib/python/kdoc/kdoc_output.py | 5 ++--- tools/lib/python/kdoc/kdoc_parser.py | 25 +++++++++++++++++++++---- 2 files changed, 23 insertions(+), 7 deletions(-)
tools/lib/python/kdoc/kdoc_output.py +2 -3
@@ -486,7 +486,7 @@ class RestFormat(OutputFormat): self.lineprefix = " " - self.data += f"\n\n.. c:macro:: {name}\n\n{self.lineprefix}{full_proto}\n\n" + self.data += f"\n\n.. c:macro:: {name}\n\n{self.lineprefix}``{full_proto}``\n\n" self.print_lineno(ln) self.output_highlight(args.get('purpose', ''))
@@ -801,13 +801,12 @@ class ManFormat(OutputFormat): def out_var(self, fname, name, args): out_name = self.arg_name(args, name) - prototype = args.other_stuff["var_type"] full_proto = args.other_stuff["full_proto"] self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" self.data += ".SH NAME\n" - self.data += f"{prototype} \\- {args['purpose']}\n" + self.data += f"{name} \\- {args['purpose']}\n" self.data += ".SH SYNOPSIS\n" self.data += f"{full_proto}\n"
tools/lib/python/kdoc/kdoc_parser.py +21 -4
@@ -946,12 +946,27 @@ class KernelDoc: # Store the full prototype before modifying it # full_proto = proto + declaration_name = None + + # + # Handle macro definitions + # + macro_prefixes = [ + KernRe(r"DEFINE_[\w_]+\s*\(([\w_]+)\)"), + ] + + for r in macro_prefixes: + match = r.search(proto) + if match: + declaration_name = match.group(1) + break # # Drop comments and macros to have a pure C prototype # - for search, sub in sub_prefixes: - proto = search.sub(sub, proto) + if not declaration_name: + for r, sub in sub_prefixes: + proto = r.sub(sub, proto) proto = proto.rstrip()
@@ -965,14 +980,16 @@ class KernelDoc: return var_type = r.group(0) - declaration_name = r.group(1) + + if not declaration_name: + declaration_name = r.group(1) + default_val = r.group(2) if default_val: default_val = default_val.lstrip("=").strip() self.output_declaration("var", declaration_name, full_proto=full_proto, - var_type=var_type, default_val=default_val, purpose=self.entry.declaration_purpose)
Signature · 1 line
2.51.1
Randy Dunlap
reviewerRe: [PATCH v5 2/5] kernel-doc: add support to handle DEFINE_ variables
Hi Mauro,
Quoted 4 lines · Mauro Carvalho Chehab
Improve the parser and output plugin to work with macros, adding support for the common pattern of using DEFINE_* to create variables.
I can't get this one to work: it is completely ignored AFAICT:
from the init/kdoc-globals-test.c file that I provided earlier:
/**
* var rtnl_mutex - historical global lock for networking control operations. *
* @rtnl_mutex is used to serialize rtnetlink requests
* and protect all kernel internal data structures related to networking. *
* See Documentation/networking/netdevices.rst for details.
* Often known as the rtnl_lock, although rtnl_lock is a kernel function. */
static DEFINE_MUTEX(rtnl_mutex);
If I remove the beginning "static " and change its name to test_mutex, kdoc reports:
WARNING: ../init/kdoc-globals-test.c:100 DEFINE_MUTEX(test_mutex);: can't parse variable
I have both of these mutexes in my test file now. Here is the second one:
/**
* var test_mutex - historical global lock for networking control operations. *
* @test_mutex is used to serialize rtnetlink requests
* and protect all kernel internal data structures related to networking. *
* See Documentation/networking/netdevices.rst for details.
* Often known as the test_lock, although test_lock is a kernel function. */DEFINE_MUTEX(test_mutex);
Signature · 1 line
~Randy
PATCH v5 3/5 docs: media: v4l2-ioctl.h: document two global variables
In reply to Mauro Carvalho ChehabThe media kAPI has two global variables at v4l2-ioctl.h. Document them.
--- include/media/v4l2-ioctl.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
include/media/v4l2-ioctl.h +15 -0
@@ -663,7 +663,22 @@ void v4l_printk_ioctl(const char *prefix, unsigned int cmd);struct video_device ;/* names for fancy debug output */ + +/** +* var v4l2_field_names - Helper array mapping ``V4L2_FIELD_*`` to strings. +* +* Specially when printing debug messages, it is interesting to output +* the field order at the V4L2 buffers. This array associates all possible +* values of field pix format from V4L2 API into a string. +*/ extern const char *v4l2_field_names[]; + +/** +* var v4l2_type_names - Helper array mapping ``V4L2_BUF_TYPE_*`` to strings. +* +* When printing debug messages, it is interesting to output the V4L2 buffer +* type number with a name that represents its content. +*/ extern const char *v4l2_type_names[];#ifdef CONFIG_COMPAT
Signature · 1 line
2.51.1
Re: [PATCH v5 3/5] docs: media: v4l2-ioctl.h: document two global variables
Quoted 4 lines · Mauro Carvalho Chehab
The media kAPI has two global variables at
v4l2-ioctl.h. Document them.Signed-off-by Mauro Carvalho Chehab <[email protected]>
Quoted 31 lines
--- include/media/v4l2-ioctl.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)include/media/v4l2-ioctl.h +15
-0@@ -663,7 +663,22 @@ void v4l_printk_ioctl(const char *prefix, unsigned int cmd);struct video_device ;/* names for fancy debug output */ + +/** +* var v4l2_field_names - Helper array mapping ``V4L2_FIELD_*`` to strings. +* +* Specially when printing debug messages, it is interesting to output +* the field order at the V4L2 buffers. This array associates all possible +* values of field pix format from V4L2 API into a string. +*/ extern const char *v4l2_field_names[]; + +/** +* var v4l2_type_names - Helper array mapping ``V4L2_BUF_TYPE_*`` to strings. +* +* When printing debug messages, it is interesting to output the V4L2 buffer +* type number with a name that represents its content. +*/ extern const char *v4l2_type_names[];#ifdef CONFIG_COMPAT
Signature · 1 line
~Randy
PATCH v5 4/5 docs: kernel-doc.rst: don't let automarkup mangle with consts
In reply to Mauro Carvalho ChehabThis document contains several words that tricks automarkup.
Ensure that all of them will be inside a ``const`` markup, avoiding automarkup to touch them.
--- Documentation/doc-guide/kernel-doc.rst | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-)
Documentation/doc-guide/kernel-doc.rst +13 -12
@@ -174,7 +174,8 @@ named ``Return`` (or ``Returns``). Structure, union, and enumeration documentation ----------------------------------------------- -The general format of a struct, union, and enum kernel-doc comment is:: +The general format of a ``struct``, ``union``, and ``enum`` kernel-doc +comment is:: /** * struct struct_name - Brief description.
@@ -187,8 +188,8 @@ The general format of a struct, union, and enum kernel-doc comment is:: */ You can replace the ``struct`` in the above example with ``union`` or -``enum`` to describe unions or enums. ``member`` is used to mean struct -and union member names as well as enumerations in an enum. +``enum`` to describe unions or enums. ``member`` is used to mean ``struct`` +and ``union`` member names as well as enumerations in an ``enum``. The brief description following the structure name may span multiple lines, and ends with a member description, a blank comment line, or the
@@ -201,7 +202,7 @@ Members of structs, unions and enums should be documented the same way as function parameters; they immediately succeed the short description and may be multi-line. -Inside a struct or union description, you can use the ``private:`` and +Inside a ``struct`` or ``union`` description, you can use the ``private:`` and ``public:`` comment tags. Structure fields that are inside a ``private:`` area are not listed in the generated output documentation.
@@ -273,11 +274,11 @@ It is possible to document nested structs and unions, like:: .. note:: - #) When documenting nested structs or unions, if the struct/union ``foo`` - is named, the member ``bar`` inside it should be documented as + #) When documenting nested structs or unions, if the ``struct``/``union`` + ``foo`` is named, the member ``bar`` inside it should be documented as ``@foo.bar:`` - #) When the nested struct/union is anonymous, the member ``bar`` in it - should be documented as ``@bar:`` + #) When the nested ``struct``/``union`` is anonymous, the member ``bar`` in + it should be documented as ``@bar:`` In-line member documentation comments ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -319,7 +320,7 @@ on a line of their own, like all other kernel-doc comments:: Typedef documentation --------------------- -The general format of a typedef kernel-doc comment is:: +The general format of a ``typedef`` kernel-doc comment is:: /** * typedef type_name - Brief description.
@@ -432,8 +433,8 @@ Domain`_ references. Typedef reference. ``&struct_name->member`` or ``&struct_name.member`` - Structure or union member reference. The cross-reference will be to the struct - or union definition, not the member directly. + ``struct`` or ``union`` member reference. The cross-reference will be to the + ``struct`` or ``union`` definition, not the member directly. ``&name`` A generic type reference. Prefer using the full reference described above
@@ -537,7 +538,7 @@ identifiers: *[ function/type ...]* Include documentation for each *function* and *type* in *source*. If no *function* is specified, the documentation for all functions and types in the *source* will be included. - *type* can be a struct, union, enum, or typedef identifier. + *type* can be a ``struct``, ``union``, ``enum``, or ``typedef`` identifier. Examples::
Signature · 1 line
2.51.1
PATCH v5 5/5 docs: kernel-doc.rst: document the new "var" kernel-doc markup
In reply to Mauro Carvalho ChehabAdd a description containing the new syntax to document variables within kernel-doc markups.
--- Documentation/doc-guide/kernel-doc.rst | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
Documentation/doc-guide/kernel-doc.rst +21 -4
@@ -342,6 +342,18 @@ Typedefs with function prototypes can also be documented:: */ typedef void (*type_name)(struct v4l2_ctrl *arg1, void *arg2); +Variables documentation +----------------------- + +The general format of a kernel-doc variable comment is:: + + /** + * var var_name - Brief description. + * + * Description of the var_name variable. + */ + extern int var_name; + Object-like macro documentation -------------------------------
@@ -463,14 +475,18 @@ through the following syntax:: For further details, please refer to the `Sphinx C Domain`_ documentation. +.. note:: + Variables aren't automatically cross referenced. For those, you need to + explicitly add a C domain cross-reference. + Overview documentation comments ------------------------------- To facilitate having source code and comments close together, you can include kernel-doc documentation blocks that are free-form comments instead of being -kernel-doc for functions, structures, unions, enums, or typedefs. This could be -used for something like a theory of operation for a driver or library code, for -example. +kernel-doc for functions, structures, unions, enums, typedefs or variables. +This could be used for something like a theory of operation for a driver or +library code, for example. This is done by using a ``DOC:`` section keyword with a section title.
@@ -538,7 +554,8 @@ identifiers: *[ function/type ...]* Include documentation for each *function* and *type* in *source*. If no *function* is specified, the documentation for all functions and types in the *source* will be included. - *type* can be a ``struct``, ``union``, ``enum``, or ``typedef`` identifier. + *type* can be a ``struct``, ``union``, ``enum``, ``typedef`` or ``var`` + identifier. Examples::
Signature · 1 line
2.51.1
Re: [PATCH v5 5/5] docs: kernel-doc.rst: document the new "var" kernel-doc markup
Quoted 8 lines · Mauro Carvalho Chehab
Add a description containing the new syntax to document variables within kernel-doc markups.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- Documentation/doc-guide/kernel-doc.rst | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
Thanks.
Signature · 1 line
~Randy