kernel-doc: make it parse new functions and structs
Subsystems (5)
v2 cover kernel-doc: make it parse new functions and structs
Hi Jon,
It is impressive how a single patch became a series with 25 ones ;-)
This version was rebased on next-20260127, which is after docs-mw got merged with my doc patches, solving some merge conflicts.
It should now apply cleanly on the top of upstream and it will address the kernel-doc issues with context analysis patches that are already at linux-next.
On this version, I opted to do something I always wanted when porting from the venerable kernel-doc perl logic: properly handle macros without writing really complex regular expressions.
After those changes, the replacement logic becomes a lot more easier to write, as we can have a cleaner logic to handle function calls:
function_xforms = [
(KernRe(r"^static +"), ""),
(KernRe(r"^extern +"), ""),
...
(NestedMatch(r"__cond_acquires\s*\("), ""),
(NestedMatch(r"__cond_releases\s*\("), ""),
...
}It can even be used to replace comma-separated arguments like here:
struct_xforms = [
...
(NestedMatch(r'\bstruct_group\s*\('), r'\2'),
(NestedMatch(r'\bstruct_group_attr\s*\('), r'\3'),
(NestedMatch(r'\bstruct_group_tagged\s*\('), r'struct \1 { \3 } \2;'),
(NestedMatch(r'\b__struct_group\s*\('), r'\4'),
...
]As both KernRe and NestedMatch have sub methods, they can be placed altogether at the same transforms tables.
The regex syntax for them are quite simple: they should end with a delimiter (usually open parenthesis, but brackets and square brackets are also supported).
The first 15 patches on this series were co-developed with Randy, with came up with the original patch. I ended helping identifying kernel-doc issues and doing some changes to make the parser more reliable.
Then, I added 10 other patches to improve NestedMatch.
The final two patches on it fixes an issue when parsing include/net/page_pool/types.h. Right now, it doesn't parse well struct page_pool_params, as the second struct_group_tagged() has a /* private */ on it, causing the end parenthesis to be commented out. After the series, the documentation for it should be ok.
-
Even NestedMatch being more complex than KernRe, on my machine, parsing all files is 5% faster than before, because we're not parsing anymore macro definitions.
Ah, due to the complexity of NestedMatch, I opted to write some unit tests to verify that the logic there is correct. We can use it to add other border cases.
Using it is as easy as running:
$ tools/unittests/nested_match.py
(I opted to create a separate directory for it, as this is not really documentation)
Mauro Carvalho Chehab (23):
docs: kdoc_re: add support for groups()
docs: kdoc_re: don't go past the end of a line
docs: kdoc_parser: move var transformers to the beginning
docs: kdoc_parser: don't mangle with function defines
docs: kdoc_parser: add functions support for NestedMatch
docs: kdoc_parser: use NestedMatch to handle __attribute__ on
functions
docs: kdoc_parser: fix variable regexes to work with size_t
docs: kdoc_parser: fix the default_value logic for variables
docs: kdoc_parser: add some debug for variable parsing
docs: kdoc_parser: don't exclude defaults from prototype
docs: kdoc_parser: fix parser to support multi-word types
docs: kdoc_parser: add support for LIST_HEAD
docs: kdoc_re: properly handle strings and escape chars on it
docs: kdoc_re: better show KernRe() at documentation
docs: kdoc_re: don't recompile NextMatch regex every time
docs: kdoc_re: Change NestedMath args replacement to \0
docs: kdoc_re: make NextedMatch use KernRe
tools: kdoc_re: add support on NestedMatch for argument replacement
tools: python: add helpers to run unit tests
unittests: add tests for NestedMatch class tools/lib/python/unittest_helper.py
docs: kdoc_parser: better handle struct_group macros
docs: kdoc_re: fix a parse bug on struct page_pool_params
Randy Dunlap (2):
docs: kdoc_parser: ignore context analysis and lock attributes
kdoc_parser: handle struct member macro VIRTIO_DECLARE_FEATURES(name)
tools/lib/python/kdoc/kdoc_parser.py | 109 +++-- tools/lib/python/kdoc/kdoc_re.py | 154 +++++-- tools/lib/python/unittest_helper.py | 279 +++++++++++++ tools/unittests/nested_match.py | 589 +++++++++++++++++++++++++++ 4 files changed, 1063 insertions(+), 68 deletions(-) create mode 100755 tools/lib/python/unittest_helper.py create mode 100755 tools/unittests/nested_match.py
Signature · 1 line
2.52.0
PATCH v2 1/25 docs: kdoc_re: add support for groups()
Add an equivalent to re groups() method.
This is useful on debug messages.
--- tools/lib/python/kdoc/kdoc_re.py | 7 +++++++ 1 file changed, 7 insertions(+)
tools/lib/python/kdoc/kdoc_re.py +7 -0
@@ -106,6 +106,13 @@ class KernRe: return self.last_match.group(num) + def groups(self): + """ + Returns the group results of the last match + """ + + return self.last_match.groups() + class NestedMatch: """
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 01/25] docs: kdoc_re: add support for groups()
Quoted 46 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 01/25] docs:kdoc_re: add support forgroups()Add an equivalent to re
groups()method.
This is useful on debug messages.Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 7 +++++++ 1 file changed, 7 insertions(+)tools/lib/python/kdoc/kdoc_re.py +7
-1@@ -106,6 +106,13 @@ class KernRe: return self.last_match.group(num) + def groups(self): + """ + Returns the group results of the last match + """ + + return self.last_match.groups() + class NestedMatch: """ --
Quoted 1 line
2.52.0
PATCH v2 2/25 docs: kdoc_re: don't go past the end of a line
In reply to Mauro Carvalho ChehabThe logic which checks if the line ends with ";" is currently
broken: it may try to read past the buffer.
Fix it by checking before trying to access line[pos].
--- tools/lib/python/kdoc/kdoc_re.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_re.py +1 -1
@@ -265,7 +265,7 @@ class NestedMatch: out += new_sub # Drop end ';' if any - if line[pos] == ';': + if pos < len(line) and line[pos] == ';': pos += 1 cur_pos = pos
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 02/25] docs: kdoc_re: don't go past the end of a line
Quoted 43 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 02/25] docs:kdoc_re: don't go past the end of a lineThe logic which checks if the line ends with ";" is currently
broken: it may try to read past the buffer.Fix it by checking before trying to access line[pos].
Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/lib/python/kdoc/kdoc_re.py +1
-2@@ -265,7 +265,7 @@ class NestedMatch: out += new_sub # Drop end ';' if any - if line[pos] == ';': + if pos < len(line) and line[pos] == ';': pos += 1 cur_pos = pos --
Quoted 1 line
2.52.0
PATCH v2 3/25 docs: kdoc_parser: move var transformers to the beginning
In reply to Mauro Carvalho ChehabJust like functions and structs had their transform variables placed at the beginning, move variable transforms to there as well.
No functional changes.
--- tools/lib/python/kdoc/kdoc_parser.py | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +13 -10
@@ -191,6 +191,18 @@ function_xforms = [ (KernRe(r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"), ""), ] +# +# Transforms for variable prototypes +# +var_xforms = [ + (KernRe(r"__read_mostly"), ""), + (KernRe(r"__ro_after_init"), ""), + (KernRe(r"(?://.*)$"), ""), + (KernRe(r"(?:/\*.*\*/)"), ""), + (KernRe(r";$"), ""), + (KernRe(r"=.*"), ""), +] + # # Ancillary functions #
@@ -971,15 +983,6 @@ class KernelDoc: ] OPTIONAL_VAR_ATTR = "^(?:" + "|".join(VAR_ATTRIBS) + ")?" - sub_prefixes = [ - (KernRe(r"__read_mostly"), ""), - (KernRe(r"__ro_after_init"), ""), - (KernRe(r"(?://.*)$"), ""), - (KernRe(r"(?:/\*.*\*/)"), ""), - (KernRe(r";$"), ""), - (KernRe(r"=.*"), ""), - ] - # # Store the full prototype before modifying it #
@@ -1003,7 +1006,7 @@ class KernelDoc: # Drop comments and macros to have a pure C prototype # if not declaration_name: - for r, sub in sub_prefixes: + for r, sub in var_xforms: proto = r.sub(sub, proto) proto = proto.rstrip()
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 03/25] docs: kdoc_parser: move var transformers to the beginning
Quoted 79 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 03/25] docs:kdoc_parser: move var transformers to the beginningJust like functions and structs had their transform variables placed at the beginning, move variable transforms to there as well.
No functional changes.
Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +0
-0@@ -191,6 +191,18 @@ function_xforms = [ (KernRe(r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"), ""), ]
tools/lib/python/kdoc/kdoc_parser.py +13
-11+# +# Transforms for variable prototypes +# +var_xforms = [ + (KernRe(r"__read_mostly"), ""), + (KernRe(r"__ro_after_init"), ""), + (KernRe(r"(?://.*)$"), ""), + (KernRe(r"(?:/\*.*\*/)"), ""), + (KernRe(r";$"), ""), + (KernRe(r"=.*"), ""), +] + # # Ancillary functions #@@ -971,15 +983,6 @@ class KernelDoc: ] OPTIONAL_VAR_ATTR = "^(?:" + "|".join(VAR_ATTRIBS) + ")?" - sub_prefixes = [ - (KernRe(r"__read_mostly"), ""), - (KernRe(r"__ro_after_init"), ""), - (KernRe(r"(?://.*)$"), ""), - (KernRe(r"(?:/\*.*\*/)"), ""), - (KernRe(r";$"), ""), - (KernRe(r"=.*"), ""), - ] - # # Store the full prototype before modifying it #@@ -1003,7 +1006,7 @@ class KernelDoc: # Drop comments and macros to have a pure C prototype # if not declaration_name: - for r, sub in sub_prefixes: + for r, sub in var_xforms: proto = r.sub(sub, proto) proto = proto.rstrip() --
Quoted 1 line
2.52.0
PATCH v2 4/25 docs: kdoc_parser: don't mangle with function defines
In reply to Mauro Carvalho ChehabMangling with #defines is not nice, as we may end removing the macro names, preventing several macros from being properly documented.
Also, on defines, we have something like:
#define foo(a1, a2, a3, ...) \ /* some real implementation */
The prototype part (first line on this example) won't contain any macros, so no need to apply any regexes on it.
With that, move the apply_transforms() logic to ensure that it will be called only on functions.
--- tools/lib/python/kdoc/kdoc_parser.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +7 -5
@@ -163,7 +163,7 @@ struct_nested_prefixes = [ # # Transforms for function prototypes # -function_xforms = [ +function_xforms = [ (KernRe(r"^static +"), ""), (KernRe(r"^extern +"), ""), (KernRe(r"^asmlinkage +"), ""),
@@ -1065,10 +1065,7 @@ class KernelDoc: found = func_macro = False return_type = '' decl_type = 'function' - # - # Apply the initial transformations. - # - prototype = apply_transforms(function_xforms, prototype) + # # If we have a macro, remove the "#define" at the front. #
@@ -1087,6 +1084,11 @@ class KernelDoc: declaration_name = r.group(1) func_macro = True found = True + else: + # + # Apply the initial transformations. + # + prototype = apply_transforms(function_xforms, prototype) # Yes, this truly is vile. We are looking for: # 1. Return type (may be nothing if we're looking at a macro)
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 04/25] docs: kdoc_parser: don't mangle with function defines
Quoted 73 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 04/25] docs:kdoc_parser: don't mangle with function definesMangling with #defines is not nice, as we may end removing the macro names, preventing several macros from being properly documented.
Also, on defines, we have something like:
#define foo(a1, a2, a3, ...) \ /* some real implementation */The prototype part (first line on this example) won't contain any macros, so no need to apply any regexes on it.
With that, move the
apply_transforms()logic to ensure that it will be called only on functions.Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +0
-0@@ -163,7 +163,7 @@ struct_nested_prefixes = [ # # Transforms forfunction prototypes # -function_xforms = [
tools/lib/python/kdoc/kdoc_parser.py +7
-5+function_xforms = [ (KernRe(r"^static +"), ""), (KernRe(r"^extern +"), ""), (KernRe(r"^asmlinkage +"), ""),@@ -1065,10 +1065,7 @@ class KernelDoc: found = func_macro = False return_type = '' decl_type = 'function' - # - # Apply the initial transformations. - # - prototype = apply_transforms(function_xforms, prototype) + # # If we have a macro, remove the "#define" at the front. #@@ -1087,6 +1084,11 @@ class KernelDoc: declaration_name = r.group(1) func_macro = True found = True + else: + # + # Apply the initial transformations. + # + prototype = apply_transforms(function_xforms, prototype) # Yes, this truly is vile. We are looking for: # 1. Return type (may be nothing if we're looking at a macro) --
Quoted 1 line
2.52.0
PATCH v2 5/25 docs: kdoc_parser: add functions support for NestedMatch
In reply to Mauro Carvalho ChehabSome annotations macros may have nested parenthesis, causing normal regex parsing to fail.
Extend apply_transforms to also use NestedMatch and add support for nested functions.
--- tools/lib/python/kdoc/kdoc_parser.py | 38 ++++++++++++++++++---------- 1 file changed, 25 insertions(+), 13 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +25 -13
@@ -152,7 +152,7 @@ struct_xforms = [ (KernRe(r'DEFINE_DMA_UNMAP_LEN\s*\(' + struct_args_pattern + r'\)', re.S), r'__u32 \1'), ] # -# Regexes here are guaranteed to have the end delimiter matching +# Struct regexes here are guaranteed to have the end delimiter matching # the start delimiter. Yet, right now, only one replace group # is allowed. #
@@ -160,6 +160,13 @@ struct_nested_prefixes = [ (re.compile(r'\bSTRUCT_GROUP\('), r'\1'), ] +# +# Function Regexes here are guaranteed to have the end delimiter matching +# the start delimiter. +# +function_nested_prefixes = [ +] + # # Transforms for function prototypes #
@@ -207,13 +214,6 @@ var_xforms = [ # Ancillary functions # -def apply_transforms(xforms, text): - """ - Apply a set of transforms to a block of text. - """ - for search, subst in xforms: - text = search.sub(subst, text) - return text multi_space = KernRe(r'\s\s+') def trim_whitespace(s):
@@ -408,6 +408,8 @@ class KernelDoc: # Place all potential outputs into an array self.entries = [] + self.nested = NestedMatch() + # # We need Python 3.7 for its "dicts remember the insertion # order" guarantee
@@ -505,6 +507,16 @@ class KernelDoc: # State flags self.state = state.NORMAL + def apply_transforms(self, regex_xforms, nested_xforms, text): + """Apply a set of transforms to a block of text.""" + for search, subst in regex_xforms: + text = search.sub(subst, text) + + for search, sub in nested_xforms: + text = self.nested.sub(search, sub, text) + + return text.strip() + def push_parameter(self, ln, decl_type, param, dtype, org_arg, declaration_name): """
@@ -881,11 +893,9 @@ class KernelDoc: # Go through the list of members applying all of our transformations. # members = trim_private_members(members) - members = apply_transforms(struct_xforms, members) + members = self.apply_transforms(struct_xforms, struct_nested_prefixes, + members) - nested = NestedMatch() - for search, sub in struct_nested_prefixes: - members = nested.sub(search, sub, members) # # Deal with embedded struct and union members, and drop enums entirely. #
@@ -1088,7 +1098,9 @@ class KernelDoc: # # Apply the initial transformations. # - prototype = apply_transforms(function_xforms, prototype) + prototype = self.apply_transforms(function_xforms, + function_nested_prefixes, + prototype) # Yes, this truly is vile. We are looking for: # 1. Return type (may be nothing if we're looking at a macro)
Signature · 1 line
2.52.0
PATCH v2 6/25 docs: kdoc_parser: use NestedMatch to handle __attribute__ on functions
In reply to Mauro Carvalho ChehabSome annotations macros may have nested parenthesis, causing normal regex parsing to fail. The __attribute__ regex is currently very complex to try to avoid that, but it doesn't catch all cases.
Ensure that the parenthesis will be properly handled by using the NestedMatch() logic.
--- tools/lib/python/kdoc/kdoc_parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_parser.py +1 -1
@@ -165,6 +165,7 @@ struct_nested_prefixes = [ # the start delimiter. # function_nested_prefixes = [ + (re.compile(r"__attribute__\s*\("), ""), ] #
@@ -195,7 +196,6 @@ function_xforms = [ (KernRe(r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +"), ""), (KernRe(r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)"), r"\1, \2"), (KernRe(r"__attribute_const__ +"), ""), - (KernRe(r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"), ""), ] #
Signature · 1 line
2.52.0
PATCH v2 7/25 docs: kdoc_parser: fix variable regexes to work with size_t
In reply to Mauro Carvalho ChehabThe regular expressions meant to pick variable types are too
naive: they forgot that the type word may contain underlines.
--- tools/lib/python/kdoc/kdoc_parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +2 -2
@@ -1027,14 +1027,14 @@ class KernelDoc: default_val = None - r= KernRe(OPTIONAL_VAR_ATTR + r"\w.*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR + r"[\w_]*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): if not declaration_name: declaration_name = r.group(1) default_val = r.group(2) else: - r= KernRe(OPTIONAL_VAR_ATTR + r"(?:\w.*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): default_val = r.group(1)
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 07/25] docs: kdoc_parser: fix variable regexes to work with size_t
Quoted 53 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 07/25] docs:kdoc_parser: fix variable regexes to work withsize_tThe regular expressions meant to pick variable types are too
naive: they forgot that the type word may contain underlines.Co-developed-by Randy Dunlap <[email protected]>Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +2
-2@@ -1027,14 +1027,14 @@ class KernelDoc: default_val = None - r= KernRe(OPTIONAL_VAR_ATTR + r"\w.*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR + r"[\w_]*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): if not declaration_name: declaration_name = r.group(1) default_val = r.group(2) else: - r= KernRe(OPTIONAL_VAR_ATTR + r"(?:\w.*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR +r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?")
ifr.match(proto):
default_val=r.group(1)
Quoted 2 lines
--
2.52.0
PATCH v2 8/25 docs: kdoc_parser: fix the default_value logic for variables
In reply to Mauro Carvalho ChehabThe indentation is wrong for the second regex, which causes problems on variables with defaults.
--- tools/lib/python/kdoc/kdoc_parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +2 -2
@@ -1035,9 +1035,9 @@ class KernelDoc: default_val = r.group(2) else: r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") - if r.match(proto): - default_val = r.group(1) + if r.match(proto): + default_val = r.group(1) if not declaration_name: self.emit_msg(ln,f"{proto}: can't parse variable") return
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 08/25] docs: kdoc_parser: fix the default_value logic for variables
Quoted 43 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 08/25] docs:kdoc_parser: fix thedefault_valuelogic for variablesThe indentation is wrong for the second regex, which causes problems on variables with defaults.
Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +2
-2@@ -1035,9 +1035,9 @@ class KernelDoc: default_val = r.group(2) else: r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") - if r.match(proto): - default_val = r.group(1) + if r.match(proto): + default_val = r.group(1) if not declaration_name: self.emit_msg(ln,f"{proto}: can't parse variable")
Quoted 3 lines
return
--
2.52.0
PATCH v2 9/25 docs: kdoc_parser: add some debug for variable parsing
In reply to Mauro Carvalho ChehabThis is a new parser that we're still fine-tuning. Add some extra debug messages to help addressing issues over there.
--- tools/lib/python/kdoc/kdoc_parser.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/lib/python/kdoc/kdoc_parser.py +11 -0
@@ -1033,11 +1033,19 @@ class KernelDoc: declaration_name = r.group(1) default_val = r.group(2) + + self.config.log.debug("Variable proto parser: %s from '%s'", + r.groups(), proto) + else: r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): default_val = r.group(1) + + if default_val: + self.config.log.debug("default: '%s'", default_val) + if not declaration_name: self.emit_msg(ln,f"{proto}: can't parse variable") return
@@ -1045,6 +1053,9 @@ class KernelDoc: if default_val: default_val = default_val.lstrip("=").strip() + self.config.log.debug("'%s' variable prototype: '%s', default: %s", + declaration_name, proto, default_val) + self.output_declaration("var", declaration_name, full_proto=full_proto, default_val=default_val,
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 09/25] docs: kdoc_parser: add some debug for variable parsing
Quoted 63 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 09/25] docs:kdoc_parser: add some debug for variable parsingThis is a new parser that we're still fine-tuning. Add some extra debug messages to help addressing issues over there.
Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 11 +++++++++++ 1 file changed, 11 insertions(+)tools/lib/python/kdoc/kdoc_parser.py +8
-0@@ -1033,11 +1033,19 @@ class KernelDoc: declaration_name = r.group(1) default_val = r.group(2) + + self.config.log.debug("Variable proto parser: %s from '%s'", + r.groups(), proto) + else: r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): default_val = r.group(1) + + if default_val: + self.config.log.debug("default: '%s'", default_val) + if not declaration_name:
self.emit_msg(ln,f"{proto}: can't parse variable")returntools/lib/python/kdoc/kdoc_parser.py +3
-0@@ -1045,6 +1053,9 @@ class KernelDoc: if default_val: default_val = default_val.lstrip("=").strip() + self.config.log.debug("'%s' variable prototype: '%s', default: %s", + declaration_name, proto, default_val) + self.output_declaration("var", declaration_name, full_proto=full_proto,
Quoted 3 lines
default_val=default_val,
--
2.52.0
PATCH v2 10/25 docs: kdoc_parser: don't exclude defaults from prototype
In reply to Mauro Carvalho ChehabIf we do that, the defaults won't be parsed.
--- tools/lib/python/kdoc/kdoc_parser.py | 1 - 1 file changed, 1 deletion(-)
tools/lib/python/kdoc/kdoc_parser.py +0 -1
@@ -207,7 +207,6 @@ var_xforms = [ (KernRe(r"(?://.*)$"), ""), (KernRe(r"(?:/\*.*\*/)"), ""), (KernRe(r";$"), ""), - (KernRe(r"=.*"), ""), ] #
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 10/25] docs: kdoc_parser: don't exclude defaults from prototype
Quoted 39 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 10/25] docs:kdoc_parser: don't exclude defaults from prototypeIf we do that, the defaults won't be parsed.
Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 1 - 1 file changed, 1 deletion(-)tools/lib/python/kdoc/kdoc_parser.py +0
-2@@ -207,7 +207,6 @@ var_xforms = [ (KernRe(r"(?://.*)$"), ""), (KernRe(r"(?:/\*.*\*/)"), ""), (KernRe(r";$"), ""), - (KernRe(r"=.*"), ""), ] # --
Quoted 1 line
2.52.0
PATCH v2 11/25 docs: kdoc_parser: fix parser to support multi-word types
In reply to Mauro Carvalho ChehabThe regular expression currently expects a single word for the type, but it may be something like "struct foo".
Add support for it.
--- tools/lib/python/kdoc/kdoc_parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +2 -2
@@ -1026,7 +1026,7 @@ class KernelDoc: default_val = None - r= KernRe(OPTIONAL_VAR_ATTR + r"[\w_]*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR + r"\s*[\w_\s]*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): if not declaration_name: declaration_name = r.group(1)
@@ -1037,7 +1037,7 @@ class KernelDoc: r.groups(), proto) else: - r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR + r"(?:[\w_\s]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): default_val = r.group(1)
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 11/25] docs: kdoc_parser: fix parser to support multi-word types
Quoted 53 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 11/25] docs:kdoc_parser: fix parser to support multi-word typesThe regular expression currently expects a single word for the type, but it may be something like "struct foo".
Add support for it.
Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +2
-1@@ -1026,7 +1026,7 @@ class KernelDoc: default_val = None - r= KernRe(OPTIONAL_VAR_ATTR + r"[\w_]*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") + r= KernRe(OPTIONAL_VAR_ATTR + + r"\s*[\w_\s]*\s+(?:\*+)?([\w_]+)\s*[\d\]\[]*\s*(=.*)?") if r.match(proto): if not declaration_name: declaration_name = r.group(1) @@ -1037,7 +1037,7 @@class KernelDoc:
r.groups(), proto) else:- r= KernRe(OPTIONAL_VAR_ATTR +
r"(?:[\w_]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?")
+ r= KernRe(OPTIONAL_VAR_ATTR +
+ r"(?:[\w_\s]*)?\s+(?:\*+)?(?:[\w_]+)\s*[\d\]\[]*\s*(=.*)?")
Quoted 4 lines
if
r.match(proto):
default_val=r.group(1)
--
2.52.0
PATCH v2 12/25 docs: kdoc_parser: ignore context analysis and lock attributes
In reply to Mauro Carvalho ChehabFrom: Randy Dunlap <[email protected]>
Drop all context analysis and lock (tracking) attributes to avoid kernel-doc warnings.
Documentation/core-api/kref:328: ../include/linux/kref.h:72: WARNING: Invalid C declaration: Expected end of definition. [error at 96]
int kref_put_mutex (struct kref *kref, void (*release)(struct kref *kref), struct mutex *mutex) __cond_acquires(true# mutex) ------------------------------------------------------------------------------------------------^Documentation/core-api/kref:328: ../include/linux/kref.h:94: WARNING: Invalid C declaration: Expected end of definition. [error at 92]
int kref_put_lock (struct kref *kref, void (*release)(struct kref *kref), spinlock_t *lock) __cond_acquires(true# lock) --------------------------------------------------------------------------------------------^
The regex is suggested by Mauro; mine was too greedy. Thanks. Updated context analysis and lock macros list provided by PeterZ. Thanks.
--- tools/lib/python/kdoc/kdoc_parser.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
tools/lib/python/kdoc/kdoc_parser.py +15 -0
@@ -81,6 +81,8 @@ struct_xforms = [ (KernRe(r'\s*__aligned\s*\([^;]*\)', re.S), ' '), (KernRe(r'\s*__counted_by\s*\([^;]*\)', re.S), ' '), (KernRe(r'\s*__counted_by_(le|be)\s*\([^;]*\)', re.S), ' '), + (KernRe(r'\s*__guarded_by\s*\([^\)]*\)', re.S), ' '), + (KernRe(r'\s*__pt_guarded_by\s*\([^\)]*\)', re.S), ' '), (KernRe(r'\s*__packed\s*', re.S), ' '), (KernRe(r'\s*CRYPTO_MINALIGN_ATTR', re.S), ' '), (KernRe(r'\s*__private', re.S), ' '),
@@ -165,6 +167,16 @@ struct_nested_prefixes = [ # the start delimiter. # function_nested_prefixes = [ + (re.compile(r"__cond_acquires\s*\("), ""), + (re.compile(r"__cond_releases\s*\("), ""), + (re.compile(r"__acquires\s*\("), ""), + (re.compile(r"__releases\s*\("), ""), + (re.compile(r"__must_hold\s*\("), ""), + (re.compile(r"__must_not_hold\s*\("), ""), + (re.compile(r"__must_hold_shared\s*\("), ""), + (re.compile(r"__cond_acquires_shared\s*\("), ""), + (re.compile(r"__acquires_shared\s*\("), ""), + (re.compile(r"__releases_shared\s*\("), ""), (re.compile(r"__attribute__\s*\("), ""), ]
@@ -195,6 +207,7 @@ function_xforms = [ (KernRe(r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +"), ""), (KernRe(r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +"), ""), (KernRe(r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)"), r"\1, \2"), + (KernRe(r"__no_context_analysis\s*"), ""), (KernRe(r"__attribute_const__ +"), ""), ]
@@ -204,6 +217,8 @@ function_xforms = [ var_xforms = [ (KernRe(r"__read_mostly"), ""), (KernRe(r"__ro_after_init"), ""), + (KernRe(r'\s*__guarded_by\s*\([^\)]*\)', re.S), ""), + (KernRe(r'\s*__pt_guarded_by\s*\([^\)]*\)', re.S), ""), (KernRe(r"(?://.*)$"), ""), (KernRe(r"(?:/\*.*\*/)"), ""), (KernRe(r";$"), ""),
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 12/25] docs: kdoc_parser: ignore context analysis and lock attributes
Quoted 96 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 12/25] docs:kdoc_parser: ignore context analysis and lock attributesFrom: Randy Dunlap <[email protected]>
Drop all context analysis and lock (tracking) attributes to avoid kernel-doc warnings.
Documentation/core-api/kref:328:../include/linux/kref.h:72: WARNING:
Invalid C declaration: Expected end of definition. [error at 96]
intkref_put_mutex(struct kref *kref, void (*release)(struct kref
*kref), struct mutex *mutex)__cond_acquires(true# mutex)
--------------------------------------------------------------------
----------------------------^Documentation/core-api/kref:328:../include/linux/kref.h:94: WARNING:
Invalid C declaration: Expected end of definition. [error at 92]
intkref_put_lock(struct kref *kref, void (*release)(struct kref
*kref),spinlock_t*lock)__cond_acquires(true# lock)
--------------------------------------------------------------------
------------------------^The regex is suggested by Mauro; mine was too greedy. Thanks. Updated context analysis and lock macros list provided by PeterZ. Thanks.
Reported-by Stephen Rothwell <[email protected]>Closes:
https://lore.kernel.org/all/[email protected]/Signed-off-by Randy Dunlap <[email protected]>Reviewed-by Mauro Carvalho Chehab <[email protected]>Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+)tools/lib/python/kdoc/kdoc_parser.py +2
-0@@ -81,6 +81,8 @@ struct_xforms = [ (KernRe(r'\s*__aligned\s*\([^;]*\)', re.S), ' '), (KernRe(r'\s*__counted_by\s*\([^;]*\)', re.S), ' '), (KernRe(r'\s*__counted_by_(le|be)\s*\([^;]*\)', re.S), ' '), + (KernRe(r'\s*__guarded_by\s*\([^\)]*\)', re.S), ' '), + (KernRe(r'\s*__pt_guarded_by\s*\([^\)]*\)', re.S), ' '), (KernRe(r'\s*__packed\s*', re.S), ' '), (KernRe(r'\s*CRYPTO_MINALIGN_ATTR', re.S), ' '), (KernRe(r'\s*__private', re.S), ' '), @@ -165,6 +167,16 @@
struct_nested_prefixes= [ # the start delimiter.
#
function_nested_prefixes= [tools/lib/python/kdoc/kdoc_parser.py +13
-1+ (re.compile(r"__cond_acquires\s*\("), ""), + (re.compile(r"__cond_releases\s*\("), ""), + (re.compile(r"__acquires\s*\("), ""), + (re.compile(r"__releases\s*\("), ""), + (re.compile(r"__must_hold\s*\("), ""), + (re.compile(r"__must_not_hold\s*\("), ""), + (re.compile(r"__must_hold_shared\s*\("), ""), + (re.compile(r"__cond_acquires_shared\s*\("), ""), + (re.compile(r"__acquires_shared\s*\("), ""), + (re.compile(r"__releases_shared\s*\("), ""), (re.compile(r"__attribute__\s*\("), ""), ]@@ -195,6 +207,7 @@ function_xforms = [ (KernRe(r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +"), ""), (KernRe(r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +"), ""), (KernRe(r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)"), r"\1, \2"), + (KernRe(r"__no_context_analysis\s*"), ""), (KernRe(r"__attribute_const__ +"), ""), ]@@ -204,6 +217,8 @@ function_xforms = [ var_xforms = [ (KernRe(r"__read_mostly"), ""), (KernRe(r"__ro_after_init"), ""), + (KernRe(r'\s*__guarded_by\s*\([^\)]*\)', re.S), ""), + (KernRe(r'\s*__pt_guarded_by\s*\([^\)]*\)', re.S), ""), (KernRe(r"(?://.*)$"), ""), (KernRe(r"(?:/\*.*\*/)"), ""), (KernRe(r";$"), ""), --
Quoted 1 line
2.52.0
PATCH v2 13/25 docs: kdoc_parser: add support for LIST_HEAD
In reply to Mauro Carvalho ChehabConvert LIST_HEAD into struct list_head when handling its prototype.
--- tools/lib/python/kdoc/kdoc_parser.py | 1 + 1 file changed, 1 insertion(+)
tools/lib/python/kdoc/kdoc_parser.py +1 -0
@@ -219,6 +219,7 @@ var_xforms = [ (KernRe(r"__ro_after_init"), ""), (KernRe(r'\s*__guarded_by\s*\([^\)]*\)', re.S), ""), (KernRe(r'\s*__pt_guarded_by\s*\([^\)]*\)', re.S), ""), + (KernRe(r"LIST_HEAD\(([\w_]+)\)"), r"struct list_head \1"), (KernRe(r"(?://.*)$"), ""), (KernRe(r"(?:/\*.*\*/)"), ""), (KernRe(r";$"), ""),
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 13/25] docs: kdoc_parser: add support for LIST_HEAD
Quoted 39 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 13/25] docs:kdoc_parser: add support forLIST_HEADConvert
LIST_HEADinto structlist_headwhen handling its prototype.Signed-off-by Mauro Carvalho Chehab <[email protected]>Acked-by Randy Dunlap <[email protected]>Tested-by Randy Dunlap <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 1 + 1 file changed, 1 insertion(+)tools/lib/python/kdoc/kdoc_parser.py +1
-1@@ -219,6 +219,7 @@ var_xforms = [ (KernRe(r"__ro_after_init"), ""), (KernRe(r'\s*__guarded_by\s*\([^\)]*\)', re.S), ""), (KernRe(r'\s*__pt_guarded_by\s*\([^\)]*\)', re.S), ""), + (KernRe(r"LIST_HEAD\(([\w_]+)\)"), r"struct list_head \1"), (KernRe(r"(?://.*)$"), ""), (KernRe(r"(?:/\*.*\*/)"), ""), (KernRe(r";$"), ""), --
Quoted 1 line
2.52.0
PATCH v2 14/25 kdoc_parser: handle struct member macro VIRTIO_DECLARE_FEATURES(name)
In reply to Mauro Carvalho ChehabFrom: Randy Dunlap <[email protected]>
Parse the macro VIRTIO_DECLARE_FEATURES(name) and expand it to its definition. These prevents one build warning:
WARNING: include/linux/virtio.h:188 struct member 'VIRTIO_DECLARE_FEATURES(features' not described in 'virtio_device'
--- tools/lib/python/kdoc/kdoc_parser.py | 1 + 1 file changed, 1 insertion(+)
tools/lib/python/kdoc/kdoc_parser.py +1 -0
@@ -152,6 +152,7 @@ struct_xforms = [ struct_args_pattern + r'\)', re.S), r'\1 \2[]'), (KernRe(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + struct_args_pattern + r'\)', re.S), r'dma_addr_t \1'), (KernRe(r'DEFINE_DMA_UNMAP_LEN\s*\(' + struct_args_pattern + r'\)', re.S), r'__u32 \1'), + (KernRe(r'VIRTIO_DECLARE_FEATURES\(([\w_]+)\)'), r'union { u64 \1; u64 \1_array[VIRTIO_FEATURES_U64S]; }'), ] # # Struct regexes here are guaranteed to have the end delimiter matching
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 14/25] kdoc_parser: handle struct member macro VIRTIO_DECLARE_FEATURES(name)
Quoted 47 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 14/25]kdoc_parser: handle struct member macroVIRTIO_DECLARE_FEATURES(name)From: Randy Dunlap <[email protected]>
Parse the macro
VIRTIO_DECLARE_FEATURES(name)and expand it to its definition. These prevents one build warning:WARNING: include/linux/virtio.h:188 struct member'VIRTIO_DECLARE_FEATURES(features' not described in '
virtio_device'Signed-off-by Randy Dunlap <[email protected]>Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 1 + 1 file changed, 1 insertion(+)tools/lib/python/kdoc/kdoc_parser.py +2
-0@@ -152,6 +152,7 @@ struct_xforms = [ struct_args_pattern + r'\)', re.S), r'\1 \2[]'), (KernRe(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + struct_args_pattern + r'\)', re.S), r'dma_addr_t \1'), (KernRe(r'DEFINE_DMA_UNMAP_LEN\s*\(' + struct_args_pattern + r'\)', re.S), r'__u32 \1'), + (KernRe(r'VIRTIO_DECLARE_FEATURES\(([\w_]+)\)'), r'union { u64 \1; + u64 \1_array[VIRTIO_FEATURES_U64S]; }'), ] # # Struct regexes here are guaranteed to have the end delimiter
Quoted 3 lines
matching
--
2.52.0
PATCH v2 15/25 docs: kdoc_re: properly handle strings and escape chars on it
In reply to Mauro Carvalho ChehabThe logic inside NestedMatch currently doesn't consider that function arguments may have chars and strings, which may eventually contain delimiters.
Add logic to handle strings and escape characters on them.
--- tools/lib/python/kdoc/kdoc_re.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
tools/lib/python/kdoc/kdoc_re.py +18 -0
@@ -195,6 +195,8 @@ class NestedMatch: for match_re in regex.finditer(line): start = match_re.start() offset = match_re.end() + string_char = None + escape = False d = line[offset - 1] if d not in self.DELIMITER_PAIRS:
@@ -208,6 +210,22 @@ class NestedMatch: d = line[pos] + if escape: + escape = False + continue + + if string_char: + if d == '\\': + escape = True + elif d == string_char: + string_char = None + + continue + + if d in ('"', "'"): + string_char = d + continue + if d in self.DELIMITER_PAIRS: end = self.DELIMITER_PAIRS[d]
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 15/25] docs: kdoc_re: properly handle strings and escape chars on it
Quoted 65 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 15/25] docs:kdoc_re: properly handle strings and escape chars on itThe logic inside NestedMatch currently doesn't consider that function arguments may have chars and strings, which may eventually contain delimiters.
Add logic to handle strings and escape characters on them.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)tools/lib/python/kdoc/kdoc_re.py +18
-1@@ -195,6 +195,8 @@ class NestedMatch: for match_re in regex.finditer(line): start = match_re.start() offset = match_re.end() + string_char = None + escape = False d = line[offset - 1] if d not in self.DELIMITER_PAIRS:@@ -208,6 +210,22 @@ class NestedMatch: d = line[pos] + if escape: + escape = False + continue + + if string_char: + if d == '\\': + escape = True + elif d == string_char: + string_char = None + + continue + + if d in ('"', "'"): + string_char = d + continue + if d in self.DELIMITER_PAIRS: end = self.DELIMITER_PAIRS[d] --
Quoted 1 line
2.52.0
PATCH v2 16/25 docs: kdoc_re: better show KernRe() at documentation
In reply to Mauro Carvalho Chehabthe __repr__() function is used by autodoc to document macro initialization.
Add a better representation for them.
--- tools/lib/python/kdoc/kdoc_re.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_re.py +4 -1
@@ -52,7 +52,10 @@ class KernRe: return self.regex.pattern def __repr__(self): - return f're.compile("{self.regex.pattern}")' + if self.regex.flags: + return f'KernRe("{self.regex.pattern}", {self.regex.flags})' + else: + return f'KernRe("{self.regex.pattern}")' def __add__(self, other): """
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 16/25] docs: kdoc_re: better show KernRe() at documentation
Quoted 45 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 16/25] docs:kdoc_re: better showKernRe()at documentationthe
__repr__()function is used by autodoc to document macro initialization.Add a better representation for them.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tools/lib/python/kdoc/kdoc_re.py +2
-1@@ -52,7 +52,10 @@ class KernRe: return self.regex.pattern def __repr__(self): - return f're.compile("{self.regex.pattern}")' + if self.regex.flags: + return f'KernRe("{self.regex.pattern}",{self.regex.flags})'
tools/lib/python/kdoc/kdoc_re.py +2
-1+ else: + return f'KernRe("{self.regex.pattern}")' def __add__(self, other): """ --
Quoted 1 line
2.52.0
PATCH v2 17/25 docs: kdoc_re: don't recompile NextMatch regex every time
In reply to Mauro Carvalho ChehabStore delimiters and its regex-compiled version as const vars.
--- tools/lib/python/kdoc/kdoc_re.py | 35 ++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 13 deletions(-)
tools/lib/python/kdoc/kdoc_re.py +22 -13
@@ -81,6 +81,13 @@ class KernRe: self.last_match = self.regex.search(string) return self.last_match + def finditer(self, string): + """ + Alias to re.finditer. + """ + + return self.regex.finditer(string) + def findall(self, string): """ Alias to re.findall.
@@ -116,6 +123,16 @@ class KernRe: return self.last_match.groups() +#: Nested delimited pairs (brackets and parenthesis) +DELIMITER_PAIRS = { + '{': '}', + '(': ')', + '[': ']', +} + +#: compiled delimiters +RE_DELIM = KernRe(r'[\{\}\[\]\(\)]') + class NestedMatch: """
@@ -165,14 +182,6 @@ class NestedMatch: # # FOO(arg1, arg2, arg3) - DELIMITER_PAIRS = { - '{': '}', - '(': ')', - '[': ']', - } - - RE_DELIM = re.compile(r'[\{\}\[\]\(\)]') - def _search(self, regex, line): """ Finds paired blocks for a regex that ends with a delimiter.
@@ -202,13 +211,13 @@ class NestedMatch: escape = False d = line[offset - 1] - if d not in self.DELIMITER_PAIRS: + if d not in DELIMITER_PAIRS: continue - end = self.DELIMITER_PAIRS[d] + end = DELIMITER_PAIRS[d] stack.append(end) - for match in self.RE_DELIM.finditer(line[offset:]): + for match in RE_DELIM.finditer(line[offset:]): pos = match.start() + offset d = line[pos]
@@ -229,8 +238,8 @@ class NestedMatch: string_char = d continue - if d in self.DELIMITER_PAIRS: - end = self.DELIMITER_PAIRS[d] + if d in DELIMITER_PAIRS: + end = DELIMITER_PAIRS[d] stack.append(end) continue
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 17/25] docs: kdoc_re: don't recompile NextMatch regex every time
Quoted 105 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 17/25] docs:kdoc_re: don't recompile NextMatch regex every timeStore delimiters and its regex-compiled version as const vars.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 35 ++++++++++++++++++++-----------(quoted diff) +0
-1- 1 file changed, 22 insertions(+), 13 deletions(-)tools/lib/python/kdoc/kdoc_re.py +8
-0@@ -81,6 +81,13 @@ class KernRe: self.last_match = self.regex.search(string) return self.last_match + def finditer(self, string): + """ + Alias to re.finditer. + """ + + return self.regex.finditer(string) + def findall(self, string): """ Alias to re.findall.@@ -116,6 +123,16 @@ class KernRe: return self.last_match.groups() +#: Nested delimited pairs (brackets and parenthesis) DELIMITER_PAIRS=
tools/lib/python/kdoc/kdoc_re.py +14
-14+{ + '{': '}', + '(': ')', + '[': ']', +} + +#: compiled delimiters +RE_DELIM = KernRe(r'[\{\}\[\]\(\)]') + class NestedMatch: """@@ -165,14 +182,6 @@ class NestedMatch: # # FOO(arg1, arg2, arg3) - DELIMITER_PAIRS = { - '{': '}', - '(': ')', - '[': ']', - } - - RE_DELIM = re.compile(r'[\{\}\[\]\(\)]') - def _search(self, regex, line): """ Finds paired blocks for a regex that ends with a delimiter.@@ -202,13 +211,13 @@ class NestedMatch: escape = False d = line[offset - 1] - if d not in self.DELIMITER_PAIRS: + if d not in DELIMITER_PAIRS: continue - end = self.DELIMITER_PAIRS[d] + end = DELIMITER_PAIRS[d] stack.append(end) - for match in self.RE_DELIM.finditer(line[offset:]): + for match in RE_DELIM.finditer(line[offset:]): pos = match.start() + offset d = line[pos]@@ -229,8 +238,8 @@ class NestedMatch: string_char = d continue - if d in self.DELIMITER_PAIRS: - end = self.DELIMITER_PAIRS[d] + if d in DELIMITER_PAIRS: + end = DELIMITER_PAIRS[d] stack.append(end) continue --
Quoted 1 line
2.52.0
PATCH v2 18/25 docs: kdoc_re: Change NestedMath args replacement to \0
In reply to Mauro Carvalho ChehabFuture patches will allow parsing each argument instead of the hole set. Prepare for it by changing the replace all args from \1 to \0.
No functional changes.
--- tools/lib/python/kdoc/kdoc_parser.py | 2 +- tools/lib/python/kdoc/kdoc_re.py | 9 +++++---- 2 files changed, 6 insertions(+), 5 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +1 -1
@@ -160,7 +160,7 @@ struct_xforms = [ # is allowed. # struct_nested_prefixes = [ - (re.compile(r'\bSTRUCT_GROUP\('), r'\1'), + (re.compile(r'\bSTRUCT_GROUP\('), r'\0'), ] #
tools/lib/python/kdoc/kdoc_re.py +5 -4
@@ -271,8 +271,9 @@ class NestedMatch: It matches a regex that it is followed by a delimiter, replacing occurrences only if all delimiters are paired. - if r'\1' is used, it works just like re: it places there the - matched paired data with the delimiter stripped. + if r'\0' is used, it works on a similar way of using re.group(0): + it places the entire args of the matched paired data, with the + delimiter stripped. If count is different than zero, it will replace at most count items.
@@ -288,9 +289,9 @@ class NestedMatch: # Value, ignoring start/end delimiters value = line[end:pos - 1] - # replaces \1 at the sub string, if \1 is used there + # replaces \0 at the sub string, if \0 is used there new_sub = sub - new_sub = new_sub.replace(r'\1', value) + new_sub = new_sub.replace(r'\0', value) out += new_sub
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 18/25] docs: kdoc_re: Change NestedMath args replacement to \0
Quoted 75 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 18/25] docs:kdoc_re: Change NestedMath args replacement to \0Future patches will allow parsing each argument instead of the hole set. Prepare for it by changing the replace all args from \1 to \0.
No functional changes.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 2 +- tools/lib/python/kdoc/kdoc_re.py | 9 +++++---- 2 files changed, 6 insertions(+), 5 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +1
-1@@ -160,7 +160,7 @@ struct_xforms = [ # is allowed. # struct_nested_prefixes = [ - (re.compile(r'\bSTRUCT_GROUP\('), r'\1'), + (re.compile(r'\bSTRUCT_GROUP\('), r'\0'), ] #tools/lib/python/kdoc/kdoc_re.py +1
-2@@ -271,8 +271,9 @@ class NestedMatch: It matches a regex that it is followed by a delimiter, replacing occurrences only if all delimiters are paired. - if r'\1' is used, it works just like re: it places there the - matched paired data with the delimiter stripped. + if r'\0' is used, it works on a similar way of using
re.group(0):
+ it places the entire args of the matched paired data, with
the
+ delimiter stripped.If count is different than zero, it will replace at most
count
items.tools/lib/python/kdoc/kdoc_re.py +2
-3@@ -288,9 +289,9 @@ class NestedMatch: # Value, ignoring start/end delimiters value = line[end:pos - 1] - # replaces \1 at the sub string, if \1 is used there + # replaces \0 at the sub string, if \0 is used there new_sub = sub - new_sub = new_sub.replace(r'\1', value) + new_sub = new_sub.replace(r'\0', value) out += new_sub --
Quoted 1 line
2.52.0
PATCH v2 19/25 docs: kdoc_re: make NextedMatch use KernRe
In reply to Mauro Carvalho ChehabInstead of using re_compile, let's create the class with the rejex and use KernRe to keep it cached.
--- tools/lib/python/kdoc/kdoc_parser.py | 55 ++++++++-------------------- tools/lib/python/kdoc/kdoc_re.py | 15 +++++--- 2 files changed, 25 insertions(+), 45 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +16 -39
@@ -153,32 +153,7 @@ struct_xforms = [ (KernRe(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + struct_args_pattern + r'\)', re.S), r'dma_addr_t \1'), (KernRe(r'DEFINE_DMA_UNMAP_LEN\s*\(' + struct_args_pattern + r'\)', re.S), r'__u32 \1'), (KernRe(r'VIRTIO_DECLARE_FEATURES\(([\w_]+)\)'), r'union { u64 \1; u64 \1_array[VIRTIO_FEATURES_U64S]; }'), -] -# -# Struct regexes here are guaranteed to have the end delimiter matching -# the start delimiter. Yet, right now, only one replace group -# is allowed. -# -struct_nested_prefixes = [ - (re.compile(r'\bSTRUCT_GROUP\('), r'\0'), -] - -# -# Function Regexes here are guaranteed to have the end delimiter matching -# the start delimiter. -# -function_nested_prefixes = [ - (re.compile(r"__cond_acquires\s*\("), ""), - (re.compile(r"__cond_releases\s*\("), ""), - (re.compile(r"__acquires\s*\("), ""), - (re.compile(r"__releases\s*\("), ""), - (re.compile(r"__must_hold\s*\("), ""), - (re.compile(r"__must_not_hold\s*\("), ""), - (re.compile(r"__must_hold_shared\s*\("), ""), - (re.compile(r"__cond_acquires_shared\s*\("), ""), - (re.compile(r"__acquires_shared\s*\("), ""), - (re.compile(r"__releases_shared\s*\("), ""), - (re.compile(r"__attribute__\s*\("), ""), + (NestedMatch(r'\bSTRUCT_GROUP\('), r'\0'), ] #
@@ -210,6 +185,17 @@ function_xforms = [ (KernRe(r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)"), r"\1, \2"), (KernRe(r"__no_context_analysis\s*"), ""), (KernRe(r"__attribute_const__ +"), ""), + (NestedMatch(r"__cond_acquires\s*\("), ""), + (NestedMatch(r"__cond_releases\s*\("), ""), + (NestedMatch(r"__acquires\s*\("), ""), + (NestedMatch(r"__releases\s*\("), ""), + (NestedMatch(r"__must_hold\s*\("), ""), + (NestedMatch(r"__must_not_hold\s*\("), ""), + (NestedMatch(r"__must_hold_shared\s*\("), ""), + (NestedMatch(r"__cond_acquires_shared\s*\("), ""), + (NestedMatch(r"__acquires_shared\s*\("), ""), + (NestedMatch(r"__releases_shared\s*\("), ""), + (NestedMatch(r"__attribute__\s*\("), ""), ] #
@@ -230,7 +216,6 @@ var_xforms = [ # Ancillary functions # - multi_space = KernRe(r'\s\s+') def trim_whitespace(s): """
@@ -424,8 +409,6 @@ class KernelDoc: # Place all potential outputs into an array self.entries = [] - self.nested = NestedMatch() - # # We need Python 3.7 for its "dicts remember the insertion # order" guarantee
@@ -523,14 +506,11 @@ class KernelDoc: # State flags self.state = state.NORMAL - def apply_transforms(self, regex_xforms, nested_xforms, text): + def apply_transforms(self, xforms, text): """Apply a set of transforms to a block of text.""" - for search, subst in regex_xforms: + for search, subst in xforms: text = search.sub(subst, text) - for search, sub in nested_xforms: - text = self.nested.sub(search, sub, text) - return text.strip() def push_parameter(self, ln, decl_type, param, dtype,
@@ -909,8 +889,7 @@ class KernelDoc: # Go through the list of members applying all of our transformations. # members = trim_private_members(members) - members = self.apply_transforms(struct_xforms, struct_nested_prefixes, - members) + members = self.apply_transforms(struct_xforms, members) # # Deal with embedded struct and union members, and drop enums entirely.
@@ -1125,9 +1104,7 @@ class KernelDoc: # # Apply the initial transformations. # - prototype = self.apply_transforms(function_xforms, - function_nested_prefixes, - prototype) + prototype = self.apply_transforms(function_xforms, prototype) # Yes, this truly is vile. We are looking for: # 1. Return type (may be nothing if we're looking at a macro)
tools/lib/python/kdoc/kdoc_re.py +9 -6
@@ -182,7 +182,10 @@ class NestedMatch: # # FOO(arg1, arg2, arg3) - def _search(self, regex, line): + def __init__(self, regex): + self.regex = KernRe(regex) + + def _search(self, line): """ Finds paired blocks for a regex that ends with a delimiter.
@@ -204,7 +207,7 @@ class NestedMatch: stack = [] - for match_re in regex.finditer(line): + for match_re in self.regex.finditer(line): start = match_re.start() offset = match_re.end() string_char = None
@@ -252,7 +255,7 @@ class NestedMatch: yield start, offset, pos + 1 break - def search(self, regex, line): + def search(self, line): """ This is similar to re.search:
@@ -260,11 +263,11 @@ class NestedMatch: returning occurrences only if all delimiters are paired. """ - for t in self._search(regex, line): + for t in self._search(line): yield line[t[0]:t[2]] - def sub(self, regex, sub, line, count=0): + def sub(self, sub, line, count=0): """ This is similar to re.sub:
@@ -283,7 +286,7 @@ class NestedMatch: cur_pos = 0 n = 0 - for start, end, pos in self._search(regex, line): + for start, end, pos in self._search(line): out += line[cur_pos:start] # Value, ignoring start/end delimiters
Signature · 1 line
2.52.0
PATCH v2 20/25 tools: kdoc_re: add support on NestedMatch for argument replacement
In reply to Mauro Carvalho ChehabCurrently, NestedMatch has very limited support for aguments
replacement: it is all or nothing.
Add support to allow replacing individual arguments as well.
--- tools/lib/python/kdoc/kdoc_re.py | 61 ++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 2 deletions(-)
tools/lib/python/kdoc/kdoc_re.py +59 -2
@@ -267,6 +267,59 @@ class NestedMatch: yield line[t[0]:t[2]] + @staticmethod + def _split_args(all_args, delim=","): + """ + Helper method to split comma-separated function arguments + or struct elements, if delim is set to ";". + + It returns a list of arguments that can be used later on by + the sub() method. + """ + args = [all_args] + stack = [] + arg_start = 0 + string_char = None + escape = False + + for idx, d in enumerate(all_args): + if escape: + escape = False + continue + + if string_char: + if d == '\\': + escape = True + elif d == string_char: + string_char = None + + continue + + if d in ('"', "'"): + string_char = d + continue + + if d in DELIMITER_PAIRS: + end = DELIMITER_PAIRS[d] + + stack.append(end) + continue + + if stack and d == stack[-1]: + stack.pop() + continue + + if d == delim and not stack: + args.append(all_args[arg_start:idx].strip()) + arg_start = idx + 1 + + # Add the last argument (if any) + last = all_args[arg_start:].strip() + if last: + args.append(last) + + return args + def sub(self, sub, line, count=0): """ This is similar to re.sub:
@@ -292,9 +345,13 @@ class NestedMatch: # Value, ignoring start/end delimiters value = line[end:pos - 1] - # replaces \0 at the sub string, if \0 is used there + # replace arguments new_sub = sub - new_sub = new_sub.replace(r'\0', value) + if "\\" in sub: + args = self._split_args(value) + + new_sub = re.sub(r'\\(\d+)', + lambda m: args[int(m.group(1))], new_sub) out += new_sub
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 20/25] tools: kdoc_re: add support on NestedMatch for argument replacement
Quoted 110 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 20/25] tools:kdoc_re: add support on NestedMatch for argument replacementCurrently, NestedMatch has very limited support for aguments
replacement: it is all or nothing.Add support to allow replacing individual arguments as well.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 61 ++++++++++++++++++++++++++++++-(quoted diff) +0
-1- 1 file changed, 59 insertions(+), 2 deletions(-)tools/lib/python/kdoc/kdoc_re.py +59
-3@@ -267,6 +267,59 @@ class NestedMatch: yield line[t[0]:t[2]] + @staticmethod + def _split_args(all_args, delim=","): + """ + Helper method to split comma-separated function arguments + or struct elements, if delim is set to ";". + + It returns a list of arguments that can be used later on by + the sub() method. + """ + args = [all_args] + stack = [] + arg_start = 0 + string_char = None + escape = False + + for idx, d in enumerate(all_args): + if escape: + escape = False + continue + + if string_char: + if d == '\\': + escape = True + elif d == string_char: + string_char = None + + continue + + if d in ('"', "'"): + string_char = d + continue + + if d in DELIMITER_PAIRS: + end = DELIMITER_PAIRS[d] + + stack.append(end) + continue + + if stack and d == stack[-1]: + stack.pop() + continue + + if d == delim and not stack: + args.append(all_args[arg_start:idx].strip()) + arg_start = idx + 1 + + # Add the last argument (if any) + last = all_args[arg_start:].strip() + if last: + args.append(last) + + return args + def sub(self, sub, line, count=0): """ This is similar to re.sub:@@ -292,9 +345,13 @@ class NestedMatch: # Value, ignoring start/end delimiters value = line[end:pos - 1] - # replaces \0 at the sub string, if \0 is used there + # replace arguments new_sub = sub - new_sub = new_sub.replace(r'\0', value) + if "\\" in sub: + args = self._split_args(value) + + new_sub = re.sub(r'\\(\d+)', + lambda m: args[int(m.group(1))], new_sub) out += new_sub --
Quoted 1 line
2.52.0
PATCH v2 21/25 tools: python: add helpers to run unit tests
In reply to Mauro Carvalho ChehabWhile python internal libraries have support for unit tests, its output is not nice. Add a helper module to improve its output.
I wrote this module last year while testing some scripts I used internally. The initial skeleton was generated with the help of LLM tools, but it was higly modified to ensure that it will work as I would expect.
--- tools/lib/python/unittest_helper.py | 293 ++++++++++++++++++++++++++++ 1 file changed, 293 insertions(+) create mode 100755 tools/lib/python/unittest_helper.py
tools/lib/python/unittest_helper.py new file +293 -0
@@ -0,0 +1,293 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2025-2026: Mauro Carvalho Chehab <[email protected]>. +# +# pylint: disable=C0103,R0912,R0914,E1101 + +""" +Helper class to better display unittest results. + +Those help functions provide a nice colored output summary of each +executed test and, when a test fails, it shows the different in diff +format when running in verbose mode, like:: + + $ tools/unittests/nested_match.py -v + ... + Traceback (most recent call last): + File "/new_devel/docs/tools/unittests/nested_match.py", line 69, in test_count_limit + self.assertEqual(replaced, "bar(a); bar(b); foo(c)") + ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + AssertionError: 'bar(a) foo(b); foo(c)' != 'bar(a); bar(b); foo(c)' + - bar(a) foo(b); foo(c) + ? ^^^^ + + bar(a); bar(b); foo(c) + ? ^^^^^ + ... + +It also allows filtering what tests will be executed via ``-k`` parameter. + +Typical usage is to do:: + + from unittest_helper import run_unittest + ... + + if __name__ == "__main__": + run_unittest(__file__) + +If passing arguments is needed, on a more complex scenario, it can be +used like on this example:: + + from unittest_helper import TestUnits, run_unittest + ... + env = {'sudo': ""} + ... + if __name__ == "__main__": + runner = TestUnits() + base_parser = runner.parse_args() + base_parser.add_argument('--sudo', action='store_true', + help='Enable tests requiring sudo privileges') + + args = base_parser.parse_args() + + # Update module-level flag + if args.sudo: + env['sudo'] = "1" + + # Run tests with customized arguments + runner.run(__file__, parser=base_parser, args=args, env=env) +""" + +import argparse +import atexit +import os +import re +import unittest +import sys + +from unittest.mock import patch + + +class Summary(unittest.TestResult): + """ + Overrides unittest.TestResult class to provide a nice colored + summary. When in verbose mode, displays actual/expected difference in + unified diff format. + """ + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + #: Dictionary to store organized test results + self.test_results = {} + + #: max length of the test names + self.max_name_length = 0 + + def startTest(self, test): + super().startTest(test) + test_id = test.id() + parts = test_id.split(".") + # Extract module, class, and method names + if len(parts) >= 3: + module_name = parts[-3] + else: + module_name = "" + if len(parts) >= 2: + class_name = parts[-2] + else: + class_name = "" + method_name = parts[-1] + # Build the hierarchical structure + if module_name not in self.test_results: + self.test_results[module_name] = {} + if class_name not in self.test_results[module_name]: + self.test_results[module_name][class_name] = [] + # Track maximum test name length for alignment + display_name = f"{method_name}:" + + self.max_name_length = max(len(display_name), self.max_name_length) + + def _record_test(self, test, status): + test_id = test.id() + parts = test_id.split(".") + if len(parts) >= 3: + module_name = parts[-3] + else: + module_name = "" + if len(parts) >= 2: + class_name = parts[-2] + else: + class_name = "" + method_name = parts[-1] + self.test_results[module_name][class_name].append((method_name, status)) + + def addSuccess(self, test): + super().addSuccess(test) + self._record_test(test, "OK") + + def addFailure(self, test, err): + super().addFailure(test, err) + self._record_test(test, "FAIL") + + def addError(self, test, err): + super().addError(test, err) + self._record_test(test, "ERROR") + + def addSkip(self, test, reason): + super().addSkip(test, reason) + self._record_test(test, f"SKIP ({reason})") + + def printResults(self): + """ + Print results using colors if tty. + """ + # Check for ANSI color support + use_color = sys.stdout.isatty() + COLORS = { + "OK": "\033[32m", # Green + "FAIL": "\033[31m", # Red + "SKIP": "\033[1;33m", # Yellow + "PARTIAL": "\033[33m", # Orange + "EXPECTED_FAIL": "\033[36m", # Cyan + "reset": "\033[0m", # Reset to default terminal color + } + if not use_color: + for c in COLORS: + COLORS[c] = "" + + # Calculate maximum test name length + if not self.test_results: + return + try: + lengths = [] + for module in self.test_results.values(): + for tests in module.values(): + for test_name, _ in tests: + lengths.append(len(test_name) + 1) # +1 for colon + max_length = max(lengths) + 2 # Additional padding + except ValueError: + sys.exit("Test list is empty") + + # Print results + for module_name, classes in self.test_results.items(): + print(f"{module_name}:") + for class_name, tests in classes.items(): + print(f" {class_name}:") + for test_name, status in tests: + # Get base status without reason for SKIP + if status.startswith("SKIP"): + status_code = status.split()[0] + else: + status_code = status + color = COLORS.get(status_code, "") + print( + f" {test_name + ':':<{max_length}}{color}{status}{COLORS['reset']}" + ) + print() + + # Print summary + print(f"\nRan {self.testsRun} tests", end="") + if hasattr(self, "timeTaken"): + print(f" in {self.timeTaken:.3f}s", end="") + print() + + if not self.wasSuccessful(): + print(f"\n{COLORS['FAIL']}FAILED (", end="") + failures = getattr(self, "failures", []) + errors = getattr(self, "errors", []) + if failures: + print(f"failures={len(failures)}", end="") + if errors: + if failures: + print(", ", end="") + print(f"errors={len(errors)}", end="") + print(f"){COLORS['reset']}") + + +def flatten_suite(suite): + """Flatten test suite hierarchy""" + tests = [] + for item in suite: + if isinstance(item, unittest.TestSuite): + tests.extend(flatten_suite(item)) + else: + tests.append(item) + return tests + + +class TestUnits: + """ + Helper class to set verbosity level + """ + def parse_args(self): + """Returns a parser for command line arguments.""" + parser = argparse.ArgumentParser(description="Test runner with regex filtering") + parser.add_argument("-v", "--verbose", action="count", default=1) + parser.add_argument("-f", "--failfast", action="store_true") + parser.add_argument("-k", "--keyword", + help="Regex pattern to filter test methods") + return parser + + def run(self, caller_file, parser=None, args=None, env=None): + """Execute all tests from the unity test file""" + if not args: + if not parser: + parser = self.parse_args() + args = parser.parse_args() + + if env: + patcher = patch.dict(os.environ, env) + patcher.start() + # ensure it gets stopped after + atexit.register(patcher.stop) + + verbose = args.verbose + + if verbose >= 2: + unittest.TextTestRunner(verbosity=verbose).run = lambda suite: suite + + # Load ONLY tests from the calling file + loader = unittest.TestLoader() + suite = loader.discover(start_dir=os.path.dirname(caller_file), + pattern=os.path.basename(caller_file)) + + # Flatten the suite for environment injection + tests_to_inject = flatten_suite(suite) + + # Filter tests by method name if -k specified + if args.keyword: + try: + pattern = re.compile(args.keyword) + filtered_suite = unittest.TestSuite() + for test in tests_to_inject: # Use the pre-flattened list + method_name = test.id().split(".")[-1] + if pattern.search(method_name): + filtered_suite.addTest(test) + suite = filtered_suite + except re.error as e: + sys.stderr.write(f"Invalid regex pattern: {e}\n") + sys.exit(1) + else: + # Maintain original suite structure if no keyword filtering + suite = unittest.TestSuite(tests_to_inject) + + if verbose >= 2: + resultclass = None + else: + resultclass = Summary + + runner = unittest.TextTestRunner(verbosity=args.verbose, + resultclass=resultclass, + failfast=args.failfast) + result = runner.run(suite) + if resultclass: + result.printResults() + + sys.exit(not result.wasSuccessful()) + + +def run_unittest(fname): + """ + Basic usage of TestUnits class. + Use it when there's no need to pass any extra argument to the tests. + """ + TestUnits().run(fname)
Signature · 1 line
2.52.0
PATCH v2 22/25 unittests: add tests for NestedMatch class
In reply to Mauro Carvalho ChehabThe NestedMatch logic is complex enough to justify tests to ensure that it is doing its job.
Add unittests to check the functionality provided by NestedMatch by replicating expected patterns.
The NestedMatch class handles with complex macros. Add an unittest to check if its doing the right thing and detect eventual regressions as we improve its code.
LLMs are pretty good writing unit tests, as those are just repetitive patterns that are built from the existing code.
To speedup writing this one, I used gpt-oss:latest running on my local GPU, feeding it with my class and with replacement patterns from the kernel.
I highly modified the generated code, though to ensure that the testset is representative and that all tests pass.
$ tools/unittests/nested_match.py
Ran 35 tests in 0.001s OK
nested_match:
TestStructGroup:
test_struct_group_01: OK
test_struct_group_02: OK
test_struct_group_03: OK
test_struct_group_04: OK
test_struct_group_05: OK
test_struct_group_06: OK
test_struct_group_07: OK
test_struct_group_08: OK
test_struct_group_09: OK
test_struct_group_10: OK
test_struct_group_11: OK
test_struct_group_12: OK
test_struct_group_13: OK
test_struct_group_14: OK
test_struct_group_15: OK
test_struct_group_16: OK
test_struct_group_17: OK
test_struct_group_18: OK
test_struct_group_19: OK
test_struct_group_sub: OK
TestSubMacros:
test_acquires_multiple: OK
test_acquires_nested_paren: OK
test_acquires_simple: OK
test_mixed_macros: OK
test_must_hold: OK
test_must_hold_shared: OK
test_no_false_positive: OK
test_no_macro_remains: OK
TestSubReplacement:
test_sub_count_parameter: OK
test_sub_mixed_placeholders: OK
test_sub_multiple_placeholders: OK
test_sub_no_placeholder: OK
test_sub_single_placeholder: OK
test_sub_with_capture: OK
test_sub_zero_placeholder: OK
Ran 35 tests
--- tools/unittests/nested_match.py | 589 ++++++++++++++++++++++++++++++++ 1 file changed, 589 insertions(+) create mode 100755 tools/unittests/nested_match.py
tools/unittests/nested_match.py new file +589 -0
@@ -0,0 +1,589 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2026: Mauro Carvalho Chehab <[email protected]>. +# +# pylint: disable=C0413,R0904 + + +""" +Unit tests for kernel-doc NestedMatch. +""" + +import os +import re +import sys +import unittest + +# Import Python modules + +SRC_DIR = os.path.dirname(os.path.realpath(__file__)) +sys.path.insert(0, os.path.join(SRC_DIR, "../lib/python")) + +from kdoc.kdoc_re import NestedMatch +from unittest_helper import run_unittest + +# -------------------------------------------------------------------------- +# Verify if struct_group matches are properly handled +# -------------------------------------------------------------------------- + + +class TestStructGroup(unittest.TestCase): + """ + Test diferent struct_group patterns. + + Please notice that in this class, there are multiple whitespaces on + some places. That's because it tries to mimic how kernel-doc parser + internally works. + """ + + @classmethod + def setUpClass(cls): + """ + Define a NestedMatch to be used for all tests picking all + struct_group macros. + """ + cls.matcher = NestedMatch(r"\bstruct_group[\w\_]*\(") + + def _check_matches(self, line: str, expected_count: int): + """count and validate each match""" + + matches = list(self.matcher.search(line)) + self.assertEqual(len(matches), expected_count, + msg=f"Expected {expected_count} matches, got {len(matches)}") + + for m in matches: + self.assertTrue(m.startswith("struct_group") and "(" in m, + msg=f"Match does not start correctly: {m!r}") + self.assertTrue(m.endswith(")"), + msg=f"Match does not end correctly: {m!r}") + + def test_struct_group_01(self): + """one struct_group with nested delimiters.""" + line = ( + "__be16 id; struct_group(body, __be16 epl_len; u8 lpl_len; u8" + " chk_code; u8 resv1; u8 resv2; u8" + " payload[ETHTOOL_CMIS_CDB_LPL_MAX_PL_LENGTH]; ); u8 *epl;" + ) + self._check_matches(line, 1) + + def test_struct_group_02(self): + """two struct_group_tagged, one per page_pool_params.""" + line = ( + "struct_group_tagged(page_pool_params_fast, fast, unsigned int " + " order; unsigned int pool_size; int nid; struct" + " device *dev; struct napi_struct *napi; enum dma_data_direction" + " dma_dir; unsigned int max_len; unsigned int offset; );" + " struct_group_tagged(page_pool_params_slow, slow, struct" + " net_device *netdev; unsigned int queue_idx; unsigned int " + " flags;)" + ) + self._check_matches(line, 2) + + def test_struct_group_03(self): + """two struct_group_tagged, one with nested structs.""" + line = ( + "struct_group_tagged(libeth_xskfq_fp, fp, struct xsk_buff_pool " + " *pool; struct libeth_xdp_buff **fqes; void " + " *descs; u32 ntu; u32 " + " count; );u32 pending; u32 " + " thresh; u32 buf_len; int " + " nid;" + ) + self._check_matches(line, 1) + + def test_struct_group_04(self): + """one struct_group_tagged with many fields.""" + line = ( + "struct_group_tagged(libeth_fq_fp, fp, struct page_pool " + " *pp; struct libeth_fqe *fqes; u32 " + " truesize; u32 count; );enum libeth_fqe_type " + " type:2; bool hsplit:1; bool " + " xdp:1; u32 buf_len; int " + " nid;" + ) + self._check_matches(line, 1) + + def test_struct_group_05(self): + """long line with a struct_group(priv_flags_fast).""" + line = ( + " struct_group(priv_flags_fast, unsigned long " + " priv_flags:32; unsigned long lltx:1; unsigned long " + " netmem_tx:1; ); const struct net_device_ops *netdev_ops;" + " const struct header_ops *header_ops; struct netdev_queue " + " *_tx; netdev_features_t gso_partial_features; unsigned int" + " real_num_tx_queues; unsigned int " + " gso_max_size; unsigned int gso_ipv4_max_size; u16 " + " gso_max_segs; s16 " + " num_tc;unsigned int mtu; unsigned short " + " needed_headroom; struct netdev_tc_txq " + " tc_to_txq[TC_MAX_QUEUE]; #ifdef CONFIG_XPS; struct xps_dev_maps " + " *xps_maps[XPS_MAPS_MAX]; #endif; #ifdef CONFIG_NETFILTER_EGRESS;" + " struct nf_hook_entries *nf_hooks_egress; #endif; #ifdef" + " CONFIG_NET_XGRESS; struct bpf_mprog_entry *tcx_egress; #endif;" + " union { struct pcpu_lstats __percpu *lstats; struct" + " pcpu_sw_netstats __percpu *tstats; struct pcpu_dstats" + " __percpu *dstats; }; unsigned long state;" + " unsigned int flags; unsigned short " + " hard_header_len; netdev_features_t features; struct" + " inet6_dev *ip6_ptr; struct bpf_prog *xdp_prog; struct" + " list_head ptype_specific; int " + " ifindex; unsigned int real_num_rx_queues; struct" + " netdev_rx_queue *_rx; unsigned int gro_max_size;" + " unsigned int gro_ipv4_max_size; rx_handler_func_t " + " *rx_handler; void *rx_handler_data; possible_net_t" + " nd_net; #ifdef CONFIG_NETPOLL; struct" + " netpoll_info *npinfo; #endif; #ifdef CONFIG_NET_XGRESS;" + " struct bpf_mprog_entry *tcx_ingress; #endif; char " + " name[IFNAMSIZ]; struct netdev_name_node *name_node; struct" + " dev_ifalias *ifalias;unsigned long mem_end; unsigned" + " long mem_start; unsigned long " + " base_addr;struct list_head dev_list; struct list_head " + " napi_list; struct list_head unreg_list; struct" + " list_head close_list; struct list_head ptype_all;" + " struct { struct list_head upper; struct list_head lower; }" + " adj_list;xdp_features_t xdp_features; const struct" + " xdp_metadata_ops *xdp_metadata_ops; const struct" + " xsk_tx_metadata_ops *xsk_tx_metadata_ops; unsigned short " + " gflags; unsigned short needed_tailroom;" + " netdev_features_t hw_features; netdev_features_t " + " wanted_features; netdev_features_t vlan_features;" + " netdev_features_t hw_enc_features; netdev_features_t " + " mpls_features; unsigned int min_mtu; unsigned int " + " max_mtu; unsigned short type; unsigned char " + " min_header_len; unsigned char name_assign_type;" + " int group; struct net_device_stats" + " stats;struct net_device_core_stats __percpu *core_stats;atomic_t" + " tx_request;" + ) + self._check_matches(line, 1) + + def test_struct_group_06(self): + """struct_group(addrs).""" + line = ( + "struct_group(addrs, unsigned char h_dest[ETH_ALEN]; unsigned" + " char h_source[ETH_ALEN]; ); __be16 h_vlan_proto;" + " __be16 h_vlan_TCI; __be16 " + " h_vlan_encapsulated_proto;" + ) + self._check_matches(line, 1) + + def test_struct_group_07(self): + """one struct_group(headers).""" + line = ( + "union { struct {struct sk_buff *next; struct sk_buff " + " *prev; union { struct net_device *dev;unsigned long " + " dev_scratch; }; }; struct rb_node rbnode;struct" + " list_head list; struct llist_node ll_node; };" + " struct sock *sk; union { ktime_t tstamp; u64" + " skb_mstamp_ns;};char cb[48] ;" + " union { struct { unsigned long _skb_refdst; void " + " (*destructor)(struct sk_buff *skb); }; struct list_head " + " tcp_tsorted_anchor; #ifdef CONFIG_NET_SOCK_MSG; unsigned long " + " _sk_redir; #endif; }; #if defined(CONFIG_NF_CONNTRACK) ||" + " defined(CONFIG_NF_CONNTRACK_MODULE); unsigned long " + " _nfct; #endif; unsigned int len, data_len; __u16 " + " mac_len, hdr_len;__u16 " + " queue_mapping;#ifdef __BIG_ENDIAN_BITFIELD; #define CLONED_MASK " + " (1 << 7); #else; #define CLONED_MASK 1; #endif; #define" + " CLONED_OFFSET offsetof(struct sk_buff," + " __cloned_offset); __u8 cloned:1, nohdr:1," + " fclone:2, peeked:1, head_frag:1, pfmemalloc:1," + " pp_recycle:1;#ifdef CONFIG_SKB_EXTENSIONS; __u8 " + " active_extensions; #endif;struct_group(headers, __u8 " + " pkt_type:3;__u8 ignore_df:1; __u8 " + " dst_pending_confirm:1; __u8 " + " ip_summed:2; __u8 ooo_okay:1; __u8 " + " tstamp_type:2;#ifdef CONFIG_NET_XGRESS; __u8 " + " tc_at_ingress:1;__u8 " + " tc_skip_classify:1; #endif; __u8 " + " remcsum_offload:1; __u8 csum_complete_sw:1;" + " __u8 csum_level:2; __u8 " + " inner_protocol_type:1; __u8 l4_hash:1; __u8 " + " sw_hash:1; #ifdef CONFIG_WIRELESS; __u8 " + " wifi_acked_valid:1; __u8 " + " wifi_acked:1; #endif; __u8 no_fcs:1;__u8 " + " encapsulation:1; __u8 " + " encap_hdr_csum:1; __u8 csum_valid:1; #ifdef" + " CONFIG_IPV6_NDISC_NODETYPE; __u8 " + " ndisc_nodetype:2; #endif; #if IS_ENABLED(CONFIG_IP_VS); __u8 " + " ipvs_property:1; #endif; #if" + " IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TRACE) ||" + " IS_ENABLED(CONFIG_NF_TABLES); __u8 " + " nf_trace:1; #endif; #ifdef CONFIG_NET_SWITCHDEV; __u8 " + " offload_fwd_mark:1; __u8 " + " offload_l3_fwd_mark:1; #endif; __u8 " + " redirected:1; #ifdef CONFIG_NET_REDIRECT; __u8 " + " from_ingress:1; #endif; #ifdef CONFIG_NETFILTER_SKIP_EGRESS;" + " __u8 nf_skip_egress:1; #endif; #ifdef" + " CONFIG_SKB_DECRYPTED; __u8 decrypted:1;" + " #endif; __u8 slow_gro:1; #if" + " IS_ENABLED(CONFIG_IP_SCTP); __u8 " + " csum_not_inet:1; #endif; __u8 unreadable:1;" + " #if defined(CONFIG_NET_SCHED) || defined(CONFIG_NET_XGRESS);" + " __u16 tc_index;#endif; u16 " + " alloc_cpu; union { __wsum csum; struct { __u16 " + " csum_start; __u16 csum_offset; }; }; __u32 " + " priority; int skb_iif; __u32 " + " hash; union { u32 vlan_all; struct { __be16 " + " vlan_proto; __u16 vlan_tci; }; }; #if" + " defined(CONFIG_NET_RX_BUSY_POLL) || defined(CONFIG_XPS); union {" + " unsigned int napi_id; unsigned int sender_cpu; }; };" + " #ifdef CONFIG_NETWORK_SECMARK; __u32 secmark; #endif;" + " union { __u32 mark; __u32 reserved_tailroom;" + " }; union { __be16 inner_protocol; __u8 " + " inner_ipproto; }; __u16 " + " inner_transport_header; __u16 " + " inner_network_header; __u16 inner_mac_header;" + " __be16 protocol; __u16 " + " transport_header; __u16 network_header; __u16 " + " mac_header; #ifdef CONFIG_KCOV; u64 " + " kcov_handle; #endif; );sk_buff_data_t tail;" + " sk_buff_data_t end; unsigned char *head," + " *data; unsigned int truesize; refcount_t " + " users; #ifdef CONFIG_SKB_EXTENSIONS;struct skb_ext " + " *extensions; #endif;" + ) + self._check_matches(line, 1) + + def test_struct_group_08(self): + """two struct_group(stats).""" + line = ( + "enum ethtool_mac_stats_src src; struct_group(stats, u64" + " tx_pause_frames; u64 rx_pause_frames; ); enum" + " ethtool_mac_stats_src src; struct_group(stats, u64" + " undersize_pkts; u64 oversize_pkts; u64 fragments; u64 jabbers;" + " u64 hist[ETHTOOL_RMON_HIST_MAX]; u64" + " hist_tx[ETHTOOL_RMON_HIST_MAX]; );" + ) + self._check_matches(line, 2) + + def test_struct_group_09(self): + """struct_group(tx_stats).""" + line = ( + "struct_group(tx_stats, u64 pkts; u64 onestep_pkts_unconfirmed;" + " u64 lost; u64 err; );" + ) + self._check_matches(line, 1) + + def test_struct_group_10(self): + """struct_group(zeroed_on_hw_restart) with a nested struct.""" + line = ( + "struct_group(zeroed_on_hw_restart, u16 fw_id; struct { u8" + " allocated:1; u8 stop_full:1; } status; ); struct list_head list;" + " atomic_t tx_request;" + ) + self._check_matches(line, 1) + + def test_struct_group_11(self): + """struct_group(zeroed_on_hw_restart) with many fields.""" + line = ( + "struct_group(zeroed_on_hw_restart, unsigned int status; u32" + " uid_status[IWL_MAX_UMAC_SCANS]; u64 start_tsf; bool" + " last_ebs_failed; enum iwl_mld_pass_all_sched_results_states" + " pass_all_sched_res; u8 fw_link_id; struct { u32" + " last_stats_ts_usec; enum iwl_mld_traffic_load status; }" + " traffic_load; );size_t cmd_size; void *cmd; unsigned long" + " last_6ghz_passive_jiffies; unsigned long" + " last_start_time_jiffies; u64 last_mlo_scan_time;" + ) + self._check_matches(line, 1) + + def test_struct_group_12(self): + """struct_group(zeroed_on_hw_restart) with a huge struct.""" + line = ( + "struct_group(zeroed_on_hw_restart, struct ieee80211_bss_conf " + " *fw_id_to_bss_conf[IWL_FW_MAX_LINK_ID + 1]; struct ieee80211_vif" + " *fw_id_to_vif[NUM_MAC_INDEX_DRIVER]; struct ieee80211_txq " + " *fw_id_to_txq[IWL_MAX_TVQM_QUEUES]; u8 used_phy_ids:" + " NUM_PHY_CTX; u8 num_igtks; struct { bool on; u32 ampdu_ref; bool" + " ampdu_toggle; u8 p80; struct { struct" + " iwl_rx_phy_air_sniffer_ntfy data; u8 valid:1, used:1; } phy;" + " #ifdef CONFIG_IWLWIFI_DEBUGFS; __le16 cur_aid; u8" + " cur_bssid[ETH_ALEN]; bool ptp_time; #endif; } monitor; #ifdef" + " CONFIG_PM_SLEEP; bool netdetect; #endif; struct ieee80211_vif" + " *p2p_device_vif; bool bt_is_active; struct ieee80211_vif" + " *nan_device_vif; ); struct ieee80211_link_sta " + " *fw_id_to_link_sta[IWL_STATION_COUNT_MAX];struct device *dev;" + " struct iwl_trans *trans; const struct iwl_rf_cfg *cfg; const" + " struct iwl_fw *fw; struct ieee80211_hw *hw; struct wiphy *wiphy;" + " struct wiphy_iftype_ext_capab" + " ext_capab[IWL_MLD_EXT_CAPA_NUM_IFTYPES]; u8" + " sta_ext_capab[IWL_MLD_STA_EXT_CAPA_SIZE]; struct iwl_nvm_data" + " *nvm_data; struct iwl_fw_runtime fwrt; struct dentry" + " *debugfs_dir; struct iwl_notif_wait_data notif_wait; struct" + " list_head async_handlers_list; spinlock_t async_handlers_lock;" + " struct wiphy_work async_handlers_wk; struct wiphy_delayed_work" + " ct_kill_exit_wk; struct { u32 running:1, do_not_dump_once:1," + " #ifdef CONFIG_PM_SLEEP; in_d3:1, resuming:1, #endif;" + " in_hw_restart:1; } fw_status; struct { u32 hw:1, ct:1; }" + " radio_kill; u32 power_budget_mw; struct mac_address" + " addresses[IWL_MLD_MAX_ADDRESSES]; struct iwl_mld_scan scan;" + " struct iwl_mld_survey *channel_survey; #ifdef CONFIG_PM_SLEEP;" + " struct wiphy_wowlan_support wowlan; u32 debug_max_sleep; #endif;" + " #ifdef CONFIG_IWLWIFI_LEDS; struct led_classdev led; #endif;" + " enum iwl_mcc_source mcc_src; bool bios_enable_puncturing; struct" + " iwl_mld_baid_data *fw_id_to_ba[IWL_MAX_BAID]; u8" + " num_rx_ba_sessions; struct iwl_mld_rx_queues_sync rxq_sync;" + " struct list_head txqs_to_add; struct wiphy_work add_txqs_wk;" + " spinlock_t add_txqs_lock; u8 *error_recovery_buf; struct" + " iwl_mcast_filter_cmd *mcast_filter_cmd; u8 mgmt_tx_ant; u8" + " set_tx_ant; u8 set_rx_ant; bool fw_rates_ver_3; struct" + " iwl_mld_low_latency low_latency; bool ibss_manager; #ifdef" + " CONFIG_THERMAL; struct thermal_zone_device *tzone; struct" + " iwl_mld_cooling_device cooling_dev; #endif; struct ptp_data" + " ptp_data; struct iwl_mld_time_sync_data *time_sync; struct" + " ftm_initiator_data ftm_initiator;" + ) + self._check_matches(line, 1) + + def test_struct_group_13(self): + """struct_group(zeroed_on_not_authorized).""" + line = ( + "struct_group(zeroed_on_not_authorized, u8 primary; u8" + " selected_primary; u16 selected_links; enum iwl_mld_emlsr_blocked" + " blocked_reasons; enum iwl_mld_emlsr_exit last_exit_reason;" + " unsigned long last_exit_ts; u8 exit_repeat_count; unsigned long" + " last_entry_ts; ); struct wiphy_work unblock_tpt_wk; struct" + " wiphy_delayed_work check_tpt_wk; struct wiphy_delayed_work" + " prevent_done_wk; struct wiphy_delayed_work tmp_non_bss_done_wk;" + ) + self._check_matches(line, 1) + + def test_struct_group_14(self): + """struct_group(zeroed_on_hw_restart) with nested struct.""" + line = ( + "struct_group(zeroed_on_hw_restart, u8 fw_id; struct" + " iwl_mld_session_protect session_protect; struct ieee80211_sta" + " *ap_sta; bool authorized; u8 num_associated_stas; bool" + " ap_ibss_active; enum iwl_mld_cca_40mhz_wa_status" + " cca_40mhz_workaround; #ifdef CONFIG_IWLWIFI_DEBUGFS; bool" + " beacon_inject_active; #endif; u8 low_latency_causes; bool" + " ps_disabled; time64_t last_link_activation_time; );struct" + " iwl_mld *mld; struct iwl_mld_link deflink; struct iwl_mld_link " + " *link[IEEE80211_MLD_MAX_NUM_LINKS]; struct iwl_mld_emlsr emlsr;" + " #ifdef CONFIG_PM_SLEEP; struct iwl_mld_wowlan_data wowlan_data;" + " #endif; #ifdef CONFIG_IWLWIFI_DEBUGFS; bool use_ps_poll; bool" + " disable_bf; struct dentry *dbgfs_slink; #endif; enum" + " iwl_roc_activity roc_activity; struct iwl_mld_int_sta aux_sta;" + " struct wiphy_delayed_work mlo_scan_start_wk;" + ) + self._check_matches(line, 1) + + def test_struct_group_15(self): + """struct_group(zeroed_on_hw_restart) with small struct.""" + line = ( + "struct_group(zeroed_on_hw_restart, u32 last_rate_n_flags; bool" + " in_fw; s8 signal_avg; );struct rcu_head rcu_head; u32 fw_id;" + ) + self._check_matches(line, 1) + + def test_struct_group_16(self): + """struct_group(zeroed_on_hw_restart) with many enums.""" + line = ( + "struct_group(zeroed_on_hw_restart, enum ieee80211_sta_state" + " sta_state; enum iwl_fw_sta_type sta_type; );struct iwl_mld *mld;" + " struct ieee80211_vif *vif; struct iwl_mld_rxq_dup_data" + " *dup_data; u8 tid_to_baid[IWL_MAX_TID_COUNT]; u8 data_tx_ant;" + " struct iwl_mld_link_sta deflink; struct iwl_mld_link_sta " + " *link[IEEE80211_MLD_MAX_NUM_LINKS]; struct iwl_mld_ptk_pn " + " *ptk_pn[IWL_NUM_DEFAULT_KEYS]; struct iwl_mld_per_q_mpdu_counter" + " *mpdu_counters;" + ) + self._check_matches(line, 1) + + def test_struct_group_17(self): + """struct_group(zeroed_on_hw_restart) with channel data.""" + line = ( + "struct_group(zeroed_on_hw_restart, u8 fw_id; struct" + " cfg80211_chan_def chandef; );u32 channel_load_by_us; u32" + " avg_channel_load_not_by_us; struct iwl_mld *mld;" + ) + self._check_matches(line, 1) + + def test_struct_group_18(self): + """mixture of struct_group and struct rcu_head.""" + line = ( + "struct rcu_head rcu_head;struct_group(zeroed_on_hw_restart, u8" + " fw_id; bool active; struct ieee80211_tx_queue_params" + " queue_params[IEEE80211_NUM_ACS]; struct ieee80211_chanctx_conf " + " *chan_ctx; bool he_ru_2mhz_block; struct ieee80211_key_conf" + " *igtk; struct ieee80211_key_conf *bigtks[2]; );struct" + " iwl_mld_int_sta bcast_sta; struct iwl_mld_int_sta mcast_sta;" + " struct iwl_mld_int_sta mon_sta;struct ieee80211_key_conf" + " *ap_early_keys[6]; u32 average_beacon_energy; bool" + " silent_deactivation; struct iwl_probe_resp_data " + " *probe_resp_data;" + ) + self._check_matches(line, 1) + + def test_struct_group_19(self): + """x(ice_health_tx_hang_buf).""" + line = ( + "struct devlink_health_reporter *fw; struct" + " devlink_health_reporter *mdd; struct devlink_health_reporter" + " *port; struct devlink_health_reporter *tx_hang;" + " struct_group_tagged(ice_health_tx_hang_buf, tx_hang_buf, struct" + " ice_tx_ring *tx_ring; u32 head; u32 intr; u16 vsi_num; ); struct" + " ice_aqc_health_status_elem fw_status; struct" + " ice_aqc_health_status_elem port_status;" + ) + self._check_matches(line, 1) + + def test_struct_group_sub(self): + """Replace struct_group body with a placeholder.""" + line = "foo bar struct_group(my, a(b{c}), d); qux;" + + result = NestedMatch(r"\bstruct_group\(").sub("REPLACED", line) + expected = "foo bar REPLACED qux;" + + self.assertEqual(result, expected) + + +class TestSubMacros(unittest.TestCase): + """ + Test macros that will be dropped. + """ + + def test_acquires_simple(self): + """Simple replacement test with __acquires""" + line = "__acquires(ctx) foo();" + result = NestedMatch(r"__acquires\s*\(").sub("REPLACED", line) + + self.assertNotIn("__acquires(", result) + self.assertIn("foo();", result) + + def test_acquires_multiple(self): + """Multiple __acquires""" + line = "__acquires(ctx) __acquires(other) bar();" + result = NestedMatch(r"__acquires\s*\(").sub("REPLACED", line) + + self.assertNotIn("__acquires(", result) + self.assertEqual(result.count("REPLACED"), 2) + + def test_acquires_nested_paren(self): + """__acquires with nested pattern""" + line = "__acquires((ctx1, ctx2)) baz();" + result = NestedMatch(r"__acquires\s*\(").sub("REPLACED", line) + + self.assertNotIn("__acquires(", result) + self.assertIn("baz();", result) + + def test_must_hold(self): + """__must_hold with a pointer""" + line = "__must_hold(&lock) do_something();" + result = NestedMatch(r"__must_hold\s*\(").sub("REPLACED", line) + + self.assertNotIn("__must_hold(", result) + self.assertIn("do_something();", result) + + def test_must_hold_shared(self): + """__must_hold with an upercase defined value""" + line = "__must_hold_shared(RCU) other();" + result = NestedMatch(r"__must_hold_shared\s*\(").sub("REPLACED", line) + + self.assertNotIn("__must_hold_shared(", result) + self.assertIn("other();", result) + + def test_no_false_positive(self): + """ + Ensure that unrelated text containing similar patterns is preserved + """ + line = "call__acquires(foo); // should stay intact" + result = NestedMatch(r"\b__acquires\s*\(").sub("REPLACED", line) + + self.assertEqual(result, line) + + def test_mixed_macros(self): + """Add a mix of macros""" + line = "__acquires(ctx) __releases(ctx) __must_hold(&lock) foo();" + + result = NestedMatch(r"__acquires\s*\(").sub("REPLACED", line) + result = NestedMatch(r"__releases\s*\(").sub("REPLACED", result) + result = NestedMatch(r"__must_hold\s*\(").sub("REPLACED", result) + + self.assertNotIn("__acquires(", result) + self.assertNotIn("__releases(", result) + self.assertNotIn("__must_hold(", result) + + self.assertIn("foo();", result) + + def test_no_macro_remains(self): + """Ensures that unmatched macros are untouched""" + line = "do_something_else();" + result = NestedMatch(r"__acquires\s*\(").sub("REPLACED", line) + + self.assertEqual(result, line) + + +class TestSubReplacement(unittest.TestCase): + """Test argument replacements""" + + @classmethod + def setUpClass(cls): + """Define a NestedMatch to be used for all tests""" + cls.matcher = NestedMatch(re.compile(r"__acquires\s*\(")) + + def test_sub_with_capture(self): + """Test all arguments replacement with a single arg""" + line = "__acquires(&ctx) foo();" + + result = self.matcher.sub(r"ACQUIRED(\0)", line) + + self.assertIn("ACQUIRED(&ctx)", result) + self.assertIn("foo();", result) + + def test_sub_zero_placeholder(self): + """Test all arguments replacement with a multiple args""" + line = "__acquires(arg1, arg2) bar();" + + result = self.matcher.sub(r"REPLACED(\0)", line) + + self.assertIn("bar();", result) + self.assertIn("REPLACED(arg1, arg2)", result) + + def test_sub_single_placeholder(self): + """Single replacement rule for \1""" + line = "__acquires(ctx) foo();" + result = self.matcher.sub(r"ACQUIRED(\1)", line) + + self.assertIn("foo();", result) + self.assertIn("ACQUIRED(ctx)", result) + + def test_sub_multiple_placeholders(self): + """Replacement rule for both \1 and \2""" + line = "__acquires(arg1, arg2) bar();" + result = self.matcher.sub(r"REPLACE(\1, \2)", line) + + self.assertIn("bar();", result) + self.assertIn("REPLACE(arg1, arg2)", result) + + def test_sub_mixed_placeholders(self): + """Replacement rule for \0, \1 and additional text""" + line = "__acquires(foo, bar) baz();" + result = self.matcher.sub(r"START(\0) END(\1)", line) + + self.assertIn("baz();", result) + self.assertIn("START(foo, bar) END(foo)", result) + + def test_sub_no_placeholder(self): + """Replacement without placeholders""" + line = "__acquires(arg) foo();" + result = self.matcher.sub(r"NO_BACKREFS()", line) + + self.assertIn("foo();", result) + self.assertIn("NO_BACKREFS()", result) + + def test_sub_count_parameter(self): + """Verify that the algorithm stops after the requested count""" + line = "__acquires(a1) x(); __acquires(a2) y();" + result = self.matcher.sub(r"ONLY_FIRST(\1)", line, count=1) + + self.assertIn("ONLY_FIRST(a1) x();", result) + self.assertIn("__acquires(a2) y();", result) + + +# ---------------------------------------------------------------------- +# Run all tests +# ---------------------------------------------------------------------- +if __name__ == "__main__": + run_unittest(__file__)
Signature · 1 line
2.52.0
PATCH v2 23/25 tools/lib/python/unittest_helper.py
In reply to Mauro Carvalho Chehab--- tools/lib/python/unittest_helper.py | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)
tools/lib/python/unittest_helper.py +1 -15
@@ -8,21 +8,7 @@ Helper class to better display unittest results. Those help functions provide a nice colored output summary of each -executed test and, when a test fails, it shows the different in diff -format when running in verbose mode, like:: - - $ tools/unittests/nested_match.py -v - ... - Traceback (most recent call last): - File "/new_devel/docs/tools/unittests/nested_match.py", line 69, in test_count_limit - self.assertEqual(replaced, "bar(a); bar(b); foo(c)") - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - AssertionError: 'bar(a) foo(b); foo(c)' != 'bar(a); bar(b); foo(c)' - - bar(a) foo(b); foo(c) - ? ^^^^ - + bar(a); bar(b); foo(c) - ? ^^^^^ - ... +executed test. It also allows filtering what tests will be executed via ``-k`` parameter.
Signature · 1 line
2.52.0
Re: [PATCH v2 23/25] tools/lib/python/unittest_helper.py
Quoted 1 line · On Wed, 28 Jan 2026 17:50:21 +0100 Mauro Carvalho Chehab…
Signed-off-by Mauro Carvalho Chehab <[email protected]>
This is obviously wrong... it should have been merged with patch 21, after a cleanup I did on it to remove some dead code.
Quoted 21 lines
--- tools/lib/python/unittest_helper.py | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)tools/lib/python/unittest_helper.py +0
-9@@ -8,21 +8,7 @@ Helper class to better display unittest results. Those help functions provide a nice colored output summary of each -executed test and, when a test fails, it shows the different in diff -format when running in verbose mode, like:: - - $ tools/unittests/nested_match.py -v - ... - Traceback (most recent call last): - File "/new_devel/docs/tools/unittests/nested_match.py", line 69, in test_count_limit - self.assertEqual(replaced, "bar(a); bar(b); foo(c)") - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tools/lib/python/unittest_helper.py line 14
Quoted 10 lines
tools/lib/python/unittest_helper.py +1
-6- AssertionError: 'bar(a) foo(b); foo(c)' != 'bar(a); bar(b); foo(c)' - - bar(a) foo(b); foo(c) - ? ^^^^ - + bar(a); bar(b); foo(c) - ? ^^^^^ - ... +executed test. It also allows filtering what tests will be executed via ``-k`` parameter.
Thanks,
Mauro
RE: [Intel-wired-lan] [PATCH v2 23/25] tools/lib/python/unittest_helper.py
In reply to Mauro Carvalho ChehabQuoted 17 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>; Mauro Carvalho Chehab <[email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Shuah Khan <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 23/25]tools/lib/python/unittest_helper.py
Could be nice if you add proper commit title following subsystem: brief description format
Quoted 38 lines
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/unittest_helper.py | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)tools/lib/python/unittest_helper.py +0
-0@@ -8,21 +8,7 @@ Helper class to better display unittest results. Those help functions provide a nice colored output summary of each -executed test and, when a test fails, it shows the different in diff - format when running in verbose mode, like::
tools/lib/python/unittest_helper.py +0
-5- - $ tools/unittests/nested_match.py -v - ... - Traceback (most recent call last): - File "/new_devel/docs/tools/unittests/nested_match.py", line 69,in
test_count_limittools/lib/python/unittest_helper.py +0
-3- self.assertEqual(replaced, "bar(a); bar(b); foo(c)") - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - AssertionError: 'bar(a) foo(b); foo(c)' != 'bar(a); bar(b);foo(c)'
tools/lib/python/unittest_helper.py +1
-5- - bar(a) foo(b); foo(c) - ? ^^^^ - + bar(a); bar(b); foo(c) - ? ^^^^^ - ... +executed test. It also allows filtering what tests will be executed via ``-k``parameter.
--
2.52.0
Re: [Intel-wired-lan] [PATCH v2 23/25] tools/lib/python/unittest_helper.py
In reply to Mauro Carvalho ChehabQuoted 2 lines · Mauro Carvalho Chehab
Signed-off-by Mauro Carvalho Chehab <[email protected]>---
What is this patch trying to do? the title is not descriptive, and its contents don't make sense.
Quoted 18 lines
tools/lib/python/unittest_helper.py | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)tools/lib/python/unittest_helper.py +0
-7@@ -8,21 +8,7 @@ Helper class to better display unittest results. Those help functions provide a nice colored output summary of each -executed test and, when a test fails, it shows the different in diff -format when running in verbose mode, like:: - - $ tools/unittests/nested_match.py -v - ... - Traceback (most recent call last): - File "/new_devel/docs/tools/unittests/nested_match.py", line 69, in test_count_limit
tools/lib/python/unittest_helper.py line 12
Quoted 10 lines
tools/lib/python/unittest_helper.py +1
-8- self.assertEqual(replaced, "bar(a); bar(b); foo(c)") - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - AssertionError: 'bar(a) foo(b); foo(c)' != 'bar(a); bar(b); foo(c)' - - bar(a) foo(b); foo(c) - ? ^^^^ - + bar(a); bar(b); foo(c) - ? ^^^^^ - ... +executed test.
You delete a bunch of the output here, but don't explain why. If this part of the doc is no longer valid this should be squashed into whatever patch made it invalid. I suspect this is the result of the new wrapper you added?
Thanks,
Jake
Quoted 2 lines
tools/lib/python/unittest_helper.py +0
-0It also allows filtering what tests will be executed via ``-k`` parameter.
Re: [Intel-wired-lan] [PATCH v2 23/25] tools/lib/python/unittest_helper.py
Hi Jacob,
Quoted 32 lines · On Wed, 28 Jan 2026 10:09:18 -0800 Jacob Keller <[email protected]>…
Quoted 2 lines · Mauro Carvalho Chehab
Signed-off-by Mauro Carvalho Chehab <[email protected]>---
What is this patch trying to do? the title is not descriptive, and its contents don't make sense.
Quoted 25 lines
tools/lib/python/unittest_helper.py | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)tools/lib/python/unittest_helper.py +0
-14@@ -8,21 +8,7 @@ Helper class to better display unittest results. Those help functions provide a nice colored output summary of each -executed test and, when a test fails, it shows the different in diff -format when running in verbose mode, like:: - - $ tools/unittests/nested_match.py -v - ... - Traceback (most recent call last): - File "/new_devel/docs/tools/unittests/nested_match.py", line 69, in test_count_limit - self.assertEqual(replaced, "bar(a); bar(b); foo(c)") - ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - AssertionError: 'bar(a) foo(b); foo(c)' != 'bar(a); bar(b); foo(c)' - - bar(a) foo(b); foo(c) - ? ^^^^ - + bar(a); bar(b); foo(c) - ? ^^^^^
Quoted 8 lines
tools/lib/python/unittest_helper.py line 12
Quoted 3 lines
tools/lib/python/unittest_helper.py +1
-1- ... +executed test.You delete a bunch of the output here, but don't explain why. If this part of the doc is no longer valid this should be squashed into whatever patch made it invalid. I suspect this is the result of the new wrapper you added?
This patch came from a tool I wrote about one year ago for my own personal usage.
It was meant to be merged with patch 21/25, as it cleans up the module description. I ended removing a function that was requiring:
from difflib import unified_diff
as it was unused, but, a second look at it, I guess we can just drop this patch, as this is plain unittest output.
Anyway, I'll handle it either way at the next version.
Thanks,
Mauro
Re: [Intel-wired-lan] [PATCH v2 23/25] tools/lib/python/unittest_helper.py
Quoted 2 lines · Mauro Carvalho Chehab
Anyway, I'll handle it either way at the next version.
I also saw you had replied already.. but I hadn't gotten that far in the thread yet when reviewing :) Thanks!
Quoted 2 lines
Thanks,
Mauro
PATCH v2 24/25 docs: kdoc_parser: better handle struct_group macros
In reply to Mauro Carvalho ChehabInstead of converting them on two steps, implement a single logic to parse them using the new sub functionality of NestedMatch.sub().
--- tools/lib/python/kdoc/kdoc_parser.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/lib/python/kdoc/kdoc_parser.py +5 -5
@@ -124,10 +124,11 @@ struct_xforms = [ # matched. So, the implementation to drop STRUCT_GROUP() will be # handled in separate. # - (KernRe(r'\bstruct_group\s*\(([^,]*,)', re.S), r'STRUCT_GROUP('), - (KernRe(r'\bstruct_group_attr\s*\(([^,]*,){2}', re.S), r'STRUCT_GROUP('), - (KernRe(r'\bstruct_group_tagged\s*\(([^,]*),([^,]*),', re.S), r'struct \1 \2; STRUCT_GROUP('), - (KernRe(r'\b__struct_group\s*\(([^,]*,){3}', re.S), r'STRUCT_GROUP('), + (NestedMatch(r'\bstruct_group\s*\('), r'\2'), + (NestedMatch(r'\bstruct_group_attr\s*\('), r'\3'), + (NestedMatch(r'\bstruct_group_tagged\s*\('), r'struct \1 { \3 } \2;'), + (NestedMatch(r'\b__struct_group\s*\('), r'\4'), + # # Replace macros #
@@ -153,7 +154,6 @@ struct_xforms = [ (KernRe(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + struct_args_pattern + r'\)', re.S), r'dma_addr_t \1'), (KernRe(r'DEFINE_DMA_UNMAP_LEN\s*\(' + struct_args_pattern + r'\)', re.S), r'__u32 \1'), (KernRe(r'VIRTIO_DECLARE_FEATURES\(([\w_]+)\)'), r'union { u64 \1; u64 \1_array[VIRTIO_FEATURES_U64S]; }'), - (NestedMatch(r'\bSTRUCT_GROUP\('), r'\0'), ] #
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 24/25] docs: kdoc_parser: better handle struct_group macros
Quoted 59 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 24/25] docs:kdoc_parser: better handlestruct_groupmacrosInstead of converting them on two steps, implement a single logic to parse them using the new sub functionality of
NestedMatch.sub().Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_parser.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)tools/lib/python/kdoc/kdoc_parser.py +3
-4@@ -124,10 +124,11 @@ struct_xforms = [ # matched. So, the implementation to drop STRUCT_GROUP() will be # handled in separate. # - (KernRe(r'\bstruct_group\s*\(([^,]*,)', re.S), r'STRUCT_GROUP('), - (KernRe(r'\bstruct_group_attr\s*\(([^,]*,){2}', re.S), r'STRUCT_GROUP('), - (KernRe(r'\bstruct_group_tagged\s*\(([^,]*),([^,]*),', re.S), r'struct \1 \2; STRUCT_GROUP('), - (KernRe(r'\b__struct_group\s*\(([^,]*,){3}', re.S), r'STRUCT_GROUP('), + (NestedMatch(r'\bstruct_group\s*\('), r'\2'), + (NestedMatch(r'\bstruct_group_attr\s*\('), r'\3'), + (NestedMatch(r'\bstruct_group_tagged\s*\('), r'struct \1 { \3 }\2;'),
tools/lib/python/kdoc/kdoc_parser.py +2
-1+ (NestedMatch(r'\b__struct_group\s*\('), r'\4'), + # # Replace macros #@@ -153,7 +154,6 @@ struct_xforms = [ (KernRe(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + struct_args_pattern + r'\)', re.S), r'dma_addr_t \1'), (KernRe(r'DEFINE_DMA_UNMAP_LEN\s*\(' + struct_args_pattern + r'\)', re.S), r'__u32 \1'), (KernRe(r'VIRTIO_DECLARE_FEATURES\(([\w_]+)\)'), r'union { u64 \1; u64 \1_array[VIRTIO_FEATURES_U64S]; }'), - (NestedMatch(r'\bSTRUCT_GROUP\('), r'\0'),]
Quoted 3 lines
#
--
2.52.0
PATCH v2 25/25 docs: kdoc_re: fix a parse bug on struct page_pool_params
In reply to Mauro Carvalho ChehabThe struct page_pool_params definition has a private definition on it:
struct page_pool_params {
struct_group_tagged(page_pool_params_fast, fast,
unsigned int order;
unsigned int pool_size;
int nid;
struct device *dev;
struct napi_struct *napi;
enum dma_data_direction dma_dir;
unsigned int max_len;
unsigned int offset;
);
struct_group_tagged(page_pool_params_slow, slow,
struct net_device *netdev;
unsigned int queue_idx;
unsigned int flags;
/* private: used by test code only */
void (*init_callback)(netmem_ref netmem, void *arg);
void *init_arg;
);};
This makes kernel-doc parser to miss the end parenthesis of the second struct_group_tagged, causing documentation issues.
Address it by ensuring that, if are there anything at the stack, it will be placed as the last part of the argument.
--- tools/lib/python/kdoc/kdoc_re.py | 8 ++++++++ 1 file changed, 8 insertions(+)
tools/lib/python/kdoc/kdoc_re.py +8 -0
@@ -206,6 +206,9 @@ class NestedMatch: """ stack = [] + start = 0 + offset = 0 + pos = 0 for match_re in self.regex.finditer(line): start = match_re.start()
@@ -255,6 +258,11 @@ class NestedMatch: yield start, offset, pos + 1 break + # When /* private */ is used, it may end the end delimiterq + if stack: + stack.pop() + yield start, offset, len(line) + 1 + def search(self, line): """ This is similar to re.search:
Signature · 1 line
2.52.0
RE: [Intel-wired-lan] [PATCH v2 25/25] docs: kdoc_re: fix a parse bug on struct page_pool_params
Quoted 78 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Mauro Carvalho Chehab
Sent: Wednesday, January 28, 2026 5:50 PM
To: Jonathan Corbet <[email protected]>; Linux Doc Mailing List <linux- [email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; Peter Zijlstra <[email protected]>; Randy Dunlap <[email protected]>; Stephen Rothwell <[email protected]>
Subject: [Intel-wired-lan] [PATCH v2 25/25] docs:kdoc_re: fix a parse bug on structpage_pool_paramsThe struct
page_pool_paramsdefinition has a private definition on it:struct page_pool_params { struct_group_tagged(page_pool_params_fast, fast, unsigned int order; unsigned int pool_size; int nid; struct device *dev; struct napi_struct *napi; enum dma_data_direction dma_dir; unsigned int max_len; unsigned int offset; ); struct_group_tagged(page_pool_params_slow, slow, struct net_device *netdev; unsigned int queue_idx; unsigned int flags; /* private: used by test code only */ void (*init_callback)(netmem_ref netmem, void *arg); void *init_arg; );};
This makes kernel-doc parser to miss the end parenthesis of the second
struct_group_tagged, causing documentation issues.Address it by ensuring that, if are there anything at the stack, it will be placed as the last part of the argument.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- tools/lib/python/kdoc/kdoc_re.py | 8 ++++++++ 1 file changed, 8 insertions(+)tools/lib/python/kdoc/kdoc_re.py +8
-1@@ -206,6 +206,9 @@ class NestedMatch: """ stack = [] + start = 0 + offset = 0 + pos = 0 for match_re in self.regex.finditer(line): start = match_re.start()@@ -255,6 +258,11 @@ class NestedMatch: yield start, offset, pos + 1 break + # When /* private */ is used, it may end the end delimiterq + if stack: + stack.pop() + yield start, offset, len(line) + 1 + def search(self, line): """ This is similar to re.search: --
Quoted 1 line
2.52.0
Jonathan Corbet
maintainerQuoted 3 lines · Mauro Carvalho Chehab
Hi Jon,
It is impressive how a single patch became a series with 25 ones ;-)
*sigh*
I will try to have a good look at these shortly. It seems pretty clear that this isn't 7.0 material at this point, though.
One thing that jumped at me:
Quoted 10 lines
Ah, due to the complexity of NestedMatch, I opted to write some unit tests to verify that the logic there is correct. We can use it to add other border cases.
Using it is as easy as running:
$ tools/unittests/nested_match.py(I opted to create a separate directory for it, as this is not really documentation)
Do we really need another unit-testing setup in the kernel? I can't say I'm familiar enough with kunit to say whether it would work for non-kernel code; have you looked and verified that it isn't suitable?
Thanks,
jon
Quoted 8 lines · Jonathan Corbet
Quoted 3 lines · Mauro Carvalho Chehab
Hi Jon,
It is impressive how a single patch became a series with 25 ones ;-)
*sigh*
Splitting things up helped me understand all the changes at least :)
Quoted 11 lines
I will try to have a good look at these shortly. It seems pretty clear that this isn't 7.0 material at this point, though.
One thing that jumped at me:
Quoted 6 lines
Ah, due to the complexity of NestedMatch, I opted to write some unit tests to verify that the logic there is correct. We can use it to add other border cases.
Using it is as easy as running:
Quoted 9 lines
Quoted 4 lines
$ tools/unittests/nested_match.py(I opted to create a separate directory for it, as this is not really documentation)
Do we really need another unit-testing setup in the kernel? I can't say I'm familiar enough with kunit to say whether it would work for non-kernel code; have you looked and verified that it isn't suitable?
I'm not sure kunit would be suitable here, since its meant for running kernel code and does a lot of stuff to make that possible. It might be able to be extended, but.. this is python code. Why *shouldn't* we use one of the python unit test frameworks for it?
We have other python code in tree. Does any of that code have unit tests?
I agree that it doesn't make sense to build new bespoke unit tests different or unique per each python module, so if we want to adopt python unit tests we should try to pick something that works for the python tools in the kernel.
Perhaps finding a way to integrate this with kunit so that you can use "kunit run" and get python tests executed as well would make sense? But.. then again this isn't kernel code so I'm not sure it makes sense to conflate the tests with kernel unit tests.
Quoted 4 lines
Thanks,
jon
Quoted 33 lines · On Wed, 28 Jan 2026 10:15:51 -0800 Jacob Keller <[email protected]>…
Quoted 8 lines · Jonathan Corbet
Quoted 3 lines · Mauro Carvalho Chehab
Hi Jon,
It is impressive how a single patch became a series with 25 ones ;-)
*sigh*
Splitting things up helped me understand all the changes at least :)
Quoted 11 lines
I will try to have a good look at these shortly. It seems pretty clear that this isn't 7.0 material at this point, though.
One thing that jumped at me:
Quoted 6 lines
Ah, due to the complexity of NestedMatch, I opted to write some unit tests to verify that the logic there is correct. We can use it to add other border cases.
Using it is as easy as running:
Quoted 9 lines
Quoted 4 lines
$ tools/unittests/nested_match.py(I opted to create a separate directory for it, as this is not really documentation)
Do we really need another unit-testing setup in the kernel? I can't say I'm familiar enough with kunit to say whether it would work for non-kernel code; have you looked and verified that it isn't suitable?
Quoted 4 lines
I'm not sure kunit would be suitable here, since its meant for running kernel code and does a lot of stuff to make that possible. It might be able to be extended, but.. this is python code. Why *shouldn't* we use one of the python unit test frameworks for it?
This is not using kunit. It is using standard "import unittest" from Python internal lib.
Quoted 1 line
We have other python code in tree. Does any of that code have unit tests?
Good question. On a quick grep, it sounds so:
$ git grep "import unittest" tools scripts scripts/rust_is_available_test.py:import unittest tools/crypto/ccp/test_dbc.py:import unittest tools/perf/pmu-events/metric_test.py:import unittest tools/testing/kunit/kunit_tool_test.py:import unittest tools/testing/selftests/bpf/test_bpftool.py:import unittest tools/testing/selftests/tpm2/tpm2.py:import unittest tools/testing/selftests/tpm2/tpm2_tests.py:import unittest
Quoted 9 lines
I agree that it doesn't make sense to build new bespoke unit tests different or unique per each python module, so if we want to adopt python unit tests we should try to pick something that works for the python tools in the kernel.
Perhaps finding a way to integrate this with kunit so that you can use "kunit run" and get python tests executed as well would make sense? But.. then again this isn't kernel code so I'm not sure it makes sense to conflate the tests with kernel unit tests.
It shouldn't be hard to add it there - or to have a separate script to run python unittests.
Assuming that we place all unittests at the same directory (tools/unittests), the enclosed path will run all of them altogether:
$ tools/unittests/run.py Ran 35 tests in 0.001s
OK
nested_match:
TestStructGroup:
test_struct_group_01: OK
test_struct_group_02: OK
test_struct_group_03: OK
test_struct_group_04: OK
test_struct_group_05: OK
test_struct_group_06: OK
test_struct_group_07: OK
test_struct_group_08: OK
test_struct_group_09: OK
test_struct_group_10: OK
test_struct_group_11: OK
test_struct_group_12: OK
test_struct_group_13: OK
test_struct_group_14: OK
test_struct_group_15: OK
test_struct_group_16: OK
test_struct_group_17: OK
test_struct_group_18: OK
test_struct_group_19: OK
test_struct_group_sub: OK
TestSubMacros:
test_acquires_multiple: OK
test_acquires_nested_paren: OK
test_acquires_simple: OK
test_mixed_macros: OK
test_must_hold: OK
test_must_hold_shared: OK
test_no_false_positive: OK
test_no_macro_remains: OK
TestSubReplacement:
test_sub_count_parameter: OK
test_sub_mixed_placeholders: OK
test_sub_multiple_placeholders: OK
test_sub_no_placeholder: OK
test_sub_single_placeholder: OK
test_sub_with_capture: OK
test_sub_zero_placeholder: OK
Ran 35 tests
And the helper will also provide an argparse to allow filtering tests, change verbosity and filtering them with a regex:
$ tools/unittests/run.py --help usage: run.py [-h] [-v] [-f] [-k KEYWORD]
Test runner with regex filtering options: -h, --help show this help message and exit -v, --verbose -f, --failfast -k, --keyword KEYWORD Regex pattern to filter test methods
That's said, some integration with kunit can be interesting to have it producing a KTAP output if needed by some CI.
---
[PATCH] [RFC] Run all tests from tools/unittests
This small example runs all unittests from tools/unittests with:
$ tools/unittests/run.py
tools/lib/python/unittest_helper.py +5 -4
@@ -213,7 +213,7 @@ class TestUnits: help="Regex pattern to filter test methods") return parser - def run(self, caller_file, parser=None, args=None, env=None): + def run(self, caller_file, suite=None, parser=None, args=None, env=None): """Execute all tests from the unity test file""" if not args: if not parser:
@@ -232,9 +232,10 @@ class TestUnits: unittest.TextTestRunner(verbosity=verbose).run = lambda suite: suite # Load ONLY tests from the calling file - loader = unittest.TestLoader() - suite = loader.discover(start_dir=os.path.dirname(caller_file), - pattern=os.path.basename(caller_file)) + if not suite: + loader = unittest.TestLoader() + suite = loader.discover(start_dir=os.path.dirname(caller_file), + pattern=os.path.basename(caller_file)) # Flatten the suite for environment injection tests_to_inject = flatten_suite(suite)
tools/unittests/run.py new file +17 -0
@@ -0,0 +1,17 @@ +#!/bin/env python3 +import os +import unittest +import sys + +TOOLS_DIR=os.path.join(os.path.dirname(os.path.realpath(__file__)), "..") +sys.path.insert(0, TOOLS_DIR) + +from lib.python.unittest_helper import TestUnits + +if __name__ == "__main__": + loader = unittest.TestLoader() + + suite = loader.discover(start_dir=os.path.join(TOOLS_DIR, "unittests"), + pattern="*.py") + + TestUnits().run("", suite=suite)
Thanks,
Mauro
Quoted 8 lines · Mauro Carvalho Chehab
Quoted 6 lines · On Wed, 28 Jan 2026 10:15:51 -0800 Jacob Keller <[email protected]>…
Quoted 4 lines · Jonathan Corbet
Do we really need another unit-testing setup in the kernel? I can't say I'm familiar enough with kunit to say whether it would work for non-kernel code; have you looked and verified that it isn't suitable?
Quoted 8 lines
Quoted 4 lines
I'm not sure kunit would be suitable here, since its meant for running kernel code and does a lot of stuff to make that possible. It might be able to be extended, but.. this is python code. Why *shouldn't* we use one of the python unit test frameworks for it?
This is not using kunit. It is using standard "import unittest" from Python internal lib.
Right. I think it makes perfect sense to use unittest for python files. That was the point of my reply above :D
Quoted 18 lines
Quoted 1 line
We have other python code in tree. Does any of that code have unit tests?
Good question. On a quick grep, it sounds so:
$ git grep "import unittest" tools scripts scripts/rust_is_available_test.py:import unittest tools/crypto/ccp/test_dbc.py:import unittest tools/perf/pmu-events/metric_test.py:import unittest tools/testing/kunit/kunit_tool_test.py:import unittest tools/testing/selftests/bpf/test_bpftool.py:import unittest tools/testing/selftests/tpm2/tpm2.py:import unittest tools/testing/selftests/tpm2/tpm2_tests.py:import unittestQuoted 5 lines
I agree that it doesn't make sense to build new bespoke unit tests different or unique per each python module, so if we want to adopt python unit tests we should try to pick something that works for the python tools in the kernel.
Quoted 8 lines
Quoted 4 lines
Perhaps finding a way to integrate this with kunit so that you can use "kunit run" and get python tests executed as well would make sense? But.. then again this isn't kernel code so I'm not sure it makes sense to conflate the tests with kernel unit tests.
It shouldn't be hard to add it there - or to have a separate script to run python unittests.
Right. Some way to have all unit tests run is nice so that its easy to hook up into various CI processes. Sounds like you have a solid idea on that.
Quoted 3 lines
That's said, some integration with kunit can be interesting to have it producing a KTAP output if needed by some CI.
That could also be interesting, as it would make it easier for other tooling to work with all the tests.
Personally I am not sure how useful that would be vs just making use of the unittest stuff provided as-is with python.
Quoted 52 lines · On Wed, 28 Jan 2026 14:08:50 -0800 Jacob Keller <[email protected]>…
Quoted 8 lines · Mauro Carvalho Chehab
Quoted 6 lines · On Wed, 28 Jan 2026 10:15:51 -0800 Jacob Keller <[email protected]>…
Quoted 4 lines · Jonathan Corbet
Do we really need another unit-testing setup in the kernel? I can't say I'm familiar enough with kunit to say whether it would work for non-kernel code; have you looked and verified that it isn't suitable?
Quoted 8 lines
Quoted 4 lines
I'm not sure kunit would be suitable here, since its meant for running kernel code and does a lot of stuff to make that possible. It might be able to be extended, but.. this is python code. Why *shouldn't* we use one of the python unit test frameworks for it?
This is not using kunit. It is using standard "import unittest" from Python internal lib.
Right. I think it makes perfect sense to use unittest for python files. That was the point of my reply above :D
Quoted 18 lines
Quoted 1 line
We have other python code in tree. Does any of that code have unit tests?
Good question. On a quick grep, it sounds so:
$ git grep "import unittest" tools scripts scripts/rust_is_available_test.py:import unittest tools/crypto/ccp/test_dbc.py:import unittest tools/perf/pmu-events/metric_test.py:import unittest tools/testing/kunit/kunit_tool_test.py:import unittest tools/testing/selftests/bpf/test_bpftool.py:import unittest tools/testing/selftests/tpm2/tpm2.py:import unittest tools/testing/selftests/tpm2/tpm2_tests.py:import unittestQuoted 5 lines
I agree that it doesn't make sense to build new bespoke unit tests different or unique per each python module, so if we want to adopt python unit tests we should try to pick something that works for the python tools in the kernel.
Quoted 8 lines
Quoted 4 lines
Perhaps finding a way to integrate this with kunit so that you can use "kunit run" and get python tests executed as well would make sense? But.. then again this isn't kernel code so I'm not sure it makes sense to conflate the tests with kernel unit tests.
It shouldn't be hard to add it there - or to have a separate script to run python unittests.
Right. Some way to have all unit tests run is nice so that its easy to hook up into various CI processes. Sounds like you have a solid idea on that.
Quoted 8 lines
Quoted 3 lines
That's said, some integration with kunit can be interesting to have it producing a KTAP output if needed by some CI.
That could also be interesting, as it would make it easier for other tooling to work with all the tests.
Personally I am not sure how useful that would be vs just making use of the unittest stuff provided as-is with python.
I'd say that, for now, we don't need a KTAP output, but as things go more complex and more parts of the tools get unittests added, it could make sense to add it.
Adding proper support for it shouldn't be hard with the definitions inside unittest_helper. All we need to do would be to write a new inherited class from unittest.TestResult, placing there a printResults() method that would generate the KTAP format. We may add a new "--ktap" argparse argument that, if enabled, it would use the newer class instead of the Summary class.
Thanks,
Mauro
Quoted 10 lines · On Wed, 28 Jan 2026 10:27:58 -0700 Jonathan Corbet <[email protected]> wrote:
Quoted 3 lines · Mauro Carvalho Chehab
Hi Jon,
It is impressive how a single patch became a series with 25 ones ;-)
*sigh*
I will try to have a good look at these shortly. It seems pretty clear that this isn't 7.0 material at this point, though.
I ended missing your e-mail...
Yeah, it is too late for 7.0.
Quoted 9 lines
One thing that jumped at me:
Quoted 6 lines
Ah, due to the complexity of NestedMatch, I opted to write some unit tests to verify that the logic there is correct. We can use it to add other border cases.
Using it is as easy as running:
Quoted 8 lines
Quoted 4 lines
$ tools/unittests/nested_match.py(I opted to create a separate directory for it, as this is not really documentation)
Do we really need another unit-testing setup in the kernel? I can't say I'm familiar enough with kunit to say whether it would work for non-kernel code; have you looked and verified that it isn't suitable?
I'm familiar with kunit: I wrote myself a bunch of tests using it for some DRM stuff.
Kunit is focused on testing kernel content. It basically adds a way for a python script to run self-test logic inside a kernel module. It is not meant to test things outside the Kernel.
Also, I'm using the python internal library for it. The only "fancy" thing I added is a helper module to better work with unit tests:
- it formats unittest output;
- it passes some parameters to the unittest discover to make it more useable;
- it adds argparse parameters to filter tests and control their verbosity levels;
- it adds a way to pass argparse values to the tests, as some tests may require parameters (like passing a different yaml file to a dynamically-generated unit test).
-
Anyway, I intend to send a new version of this series either later this week or at the next one.
I'll probably split the unittest part from the part meant to avoid new warnings after merging from linux-next.
Signature · 2 lines
Thanks, Mauro