kernel-doc: fix -Werror issues
cover kernel-doc: fix -Werror issues
Hi Jon,
As pointed by Jani, and previously by Randy, several warnings are currently not considered as errors. The issue is related to changeset 469c1c9eb6c9 ("kernel-doc: Issue warnings that were silently discarded"), which was using directly a low-level interface.
Patch 1 fix that.
Patch 2 fixes a related issue: when there are more than 255 errors, the return code with -Werror can be 0 even on errors, because of glibc behavior: programs can only use 8 bits for error codes.
Patches 3 and 4 are not directly related, and touches only on comments inside python-doc.py script itself:
- patch 3 is just a coding style change to match the comment style we agreed;
- patch 4 fixes some English issues on comments. The issues were pointed by LLM, which was used as any other spelling
tool: I picked just the relevant changes, ignoring the ones that were false positives. I also didn't agree with one of the changes, replacing it by something else.
I wrote it more as an experiment about how to properly use LLM. IMO, it could be useful to use LLM to review spelling on docs, provided that the one using the tool will filter out LLM-generated trash - just like one does when using any other spelling tool.
Mauro Carvalho Chehab (4):
scripts/kernel-doc: fix logic to handle unissued warnings scripts/kernel-doc: avoid error_count overflows scripts/kernel-doc: ensure that comments are using our coding style scripts/kernel-doc: some fixes to kernel-doc comments
scripts/kernel-doc.py | 72 +++++++++++++++++++--------- tools/lib/python/kdoc/kdoc_parser.py | 6 ++- 2 files changed, 55 insertions(+), 23 deletions(-)
Signature · 1 line
2.52.0
PATCH 1/4 scripts/kernel-doc: fix logic to handle unissued warnings
Changeset 469c1c9eb6c9 ("kernel-doc: Issue warnings that were silently discarded") didn't properly addressed the missing messages behavior, as
it was calling directly python logger low-level function,
instead of using the expected method to emit warnings.
Basically, there are two methods to log messages:
- self.config.log.warning() - This is the raw level to emit a warning. It just writes the a message at stderr, via python logging, as it is initialized as:
self.config.log = logging.getLogger("kernel-doc")
- self.config.warning() - This is where we actually consider a message as a warning, properly incrementing error count.
Due to that, several parsing error messages are internally considered as success, causing -Werror to not work on such messages.
While here, ensure that the last ignored entry will also be handled by adding an extra check at the end of the parse handler.
--- tools/lib/python/kdoc/kdoc_parser.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tools/lib/python/kdoc/kdoc_parser.py +5 -1
@@ -459,7 +459,7 @@ class KernelDoc: # if self.entry and self.entry not in self.entries: for log_msg in self.entry.warnings: - self.config.log.warning(log_msg) + self.config.warning(log_msg) self.entry = KernelEntry(self.config, self.fname, ln)
@@ -1741,6 +1741,10 @@ class KernelDoc: # Hand this line to the appropriate state handler self.state_actions[self.state](self, ln, line) + if self.entry and self.entry not in self.entries: + for log_msg in self.entry.warnings: + self.config.warning(log_msg) + except OSError: self.config.log.error(f"Error: Cannot open file {self.fname}")
Signature · 1 line
2.52.0
Re: [PATCH 1/4] scripts/kernel-doc: fix logic to handle unissued warnings
Quoted 13 lines · Mauro Carvalho Chehab
Changeset
469c1c9eb6c9("kernel-doc: Issue warnings that were silently discarded") didn't properly addressed the missing messages behavior, as
it was calling directly python logger low-level function,
instead of using the expected method to emit warnings.Basically, there are two methods to log messages:
-
self.config.log.warning()- This is the raw level to emit a warning. It just writes the a message at stderr, via python logging, as it is initialized as:self.config.log =
logging.getLogger("kernel-doc")
Quoted 8 lines
-
self.config.warning()- This is where we actually consider a message as a warning, properly incrementing error count.Due to that, several parsing error messages are internally considered as success, causing -Werror to not work on such messages.
While here, ensure that the last ignored entry will also be handled by adding an extra check at the end of the parse handler.
Thanks for addressing these issues, Mauro!
I am not promising I will test this soon, but from code perspective it looks good to me,
Signature · 2 lines
With Best Regards, Andy Shevchenko
PATCH 2/4 scripts/kernel-doc: avoid error_count overflows
In reply to Mauro Carvalho ChehabThe glibc library limits the return code to 8 bits. We need to stick to this limit when using sys.exit(error_count).
--- scripts/kernel-doc.py | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)
scripts/kernel-doc.py +22 -10
@@ -176,7 +176,14 @@ class MsgFormatter(logging.Formatter): return logging.Formatter.format(self, record) def main(): - """Main program""" + """ + Main program + By default, the return value is zero on parsing errors or when the + Python version is not compatible with kernel-doc. The rationale is + to not break Linux compilation on such cases. + If -Werror is used, it will return the number of parse errors, up to + 255 errors, as this is the maximum value allowed by glibc. + """ parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, description=DESC)
@@ -321,18 +328,23 @@ def main(): if not error_count: sys.exit(0) + if args.verbose: + print("%s errors" % error_count) # pylint: disable=C0209 + + if args.werror: print("%s warnings as errors" % error_count) # pylint: disable=C0209 + + # + # Return code is 8-bits, as seen at: + # https://www.gnu.org/software/libc/manual/html_node/Exit-Status.html + # Truncate to avoid overflow + # + if error_count > 255: + error_count = 255 + sys.exit(error_count) - - if args.verbose: - print("%s errors" % error_count) # pylint: disable=C0209 - - if args.none: - sys.exit(0) - - sys.exit(error_count) - + sys.exit(0) # Call main method if __name__ == "__main__":
Signature · 1 line
2.52.0
Re: [PATCH 2/4] scripts/kernel-doc: avoid error_count overflows
Quoted 36 lines · Mauro Carvalho Chehab
The glibc library limits the return code to 8 bits. We need to stick to this limit when using
sys.exit(error_count).Signed-off-by Mauro Carvalho Chehab <[email protected]>--- scripts/kernel-doc.py | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)scripts/kernel-doc.py +12
-1@@ -176,7 +176,14 @@ class MsgFormatter(logging.Formatter): return logging.Formatter.format(self, record) def main(): - """Main program""" + """ + Main program + By default, the return value is zero on parsing errors or when the + Python version is not compatible with kernel-doc. The rationale is + to not break Linux compilation on such cases. + If -Werror is used, it will return the number of parse errors, up to + 255 errors, as this is the maximum value allowed by glibc. + """ parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, description=DESC)@@ -321,18 +328,23 @@ def main(): if not error_count: sys.exit(0) + if args.verbose: + print("%s errors" % error_count) # pylint: disable=C0209 + +
scripts/kernel-doc.py def main(): · line 344
Quoted 10 lines
scripts/kernel-doc.py +8
-0if args.werror: print("%s warnings as errors" % error_count) # pylint: disable=C0209 + + # + # Return code is 8-bits, as seen at: + # https://www.gnu.org/software/libc/manual/html_node/Exit-Status.html + # Truncate to avoid overflow + # + if error_count > 255: + error_count = 255
What's the point in returning the error count anyway?
I'd rather see some error/warning classification in the exit code than a count. Like, the argparser uses exit code 2 by default, so you can't even trust the exit code to return the count anyway.
BR,
Jani.
Quoted 15 lines
scripts/kernel-doc.py +2
-9+ sys.exit(error_count) - - if args.verbose: - print("%s errors" % error_count) # pylint: disable=C0209 - - if args.none: - sys.exit(0) - - sys.exit(error_count) - + sys.exit(0) # Call main method if __name__ == "__main__":
Signature · 1 line
Jani Nikula, Intel
Re: [PATCH 2/4] scripts/kernel-doc: avoid error_count overflows
Quoted 33 lines · Jani Nikula
Quoted 32 lines · Mauro Carvalho Chehab
The glibc library limits the return code to 8 bits. We need to stick to this limit when using
sys.exit(error_count).Signed-off-by Mauro Carvalho Chehab <[email protected]>--- scripts/kernel-doc.py | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-)scripts/kernel-doc.py +8
-1@@ -176,7 +176,14 @@ class MsgFormatter(logging.Formatter): return logging.Formatter.format(self, record) def main(): - """Main program""" + """ + Main program + By default, the return value is zero on parsing errors or when the + Python version is not compatible with kernel-doc. The rationale is + to not break Linux compilation on such cases. + If -Werror is used, it will return the number of parse errors, up to + 255 errors, as this is the maximum value allowed by glibc. + """ parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, description=DESC)@@ -321,18 +328,23 @@ def main(): if not error_count: sys.exit(0)
Quoted 16 lines
Quoted 4 lines
scripts/kernel-doc.py +4
-0+ if args.verbose: + print("%s errors" % error_count) # pylint: disable=C0209 + +scripts/kernel-doc.py def main(): · line 344
Quoted 10 lines
scripts/kernel-doc.py +8
-0if args.werror: print("%s warnings as errors" % error_count) # pylint: disable=C0209 + + # + # Return code is 8-bits, as seen at: + # https://www.gnu.org/software/libc/manual/html_node/Exit-Status.html + # Truncate to avoid overflow + # + if error_count > 255: + error_count = 255What's the point in returning the error count anyway?
Mostly historical reasons. I used the return value during the conversion to help verifying if the Python version was doing the right thing.
Quoted 3 lines
I'd rather see some error/warning classification in the exit code than a count. Like, the argparser uses exit code 2 by default, so you can't even trust the exit code to return the count anyway.
Right now, python can exit with:
1. argparse errors;
2. sys.exit() with messages like:
sys.exit(f"Error running fc-list: {repr(exc)}")
3. eventually unhandled errors that would rise
4. sys.exit(0) if:
- No warnings;
- the -Werror is not used and warnings were issued;
- Python version < 3.6 (we want kernel to keep building)
5. error count > 0
(2) and (3) returns 1; argparse returns 2.
So, maybe we can use sys.exit(3) for warnings.
Signature · 2 lines
Thanks, Mauro
PATCH 3/4 scripts/kernel-doc: ensure that comments are using our coding style
In reply to Mauro Carvalho ChehabAlong kernel-doc libs, we opted to have all comments starting/ending with a blank comment line. Use the same style here.
--- scripts/kernel-doc.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
scripts/kernel-doc.py +16 -0
@@ -188,7 +188,9 @@ def main(): parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, description=DESC) + # # Normal arguments + # parser.add_argument("-v", "-verbose", "--verbose", action="store_true", help="Verbose output, more warnings and other information.")
@@ -204,7 +206,9 @@ def main(): action="store_true", help="Enable line number output (only in ReST mode)") + # # Arguments to control the warning behavior + # parser.add_argument("-Wreturn", "--wreturn", action="store_true", help="Warns about the lack of a return markup on functions.")
@@ -226,7 +230,9 @@ def main(): parser.add_argument("-export-file", "--export-file", action='append', help=EXPORT_FILE_DESC) + # # Output format mutually-exclusive group + # out_group = parser.add_argument_group("Output format selection (mutually exclusive)")
@@ -239,7 +245,9 @@ def main(): out_fmt.add_argument("-N", "-none", "--none", action="store_true", help="Do not output documentation, only warnings.") + # # Output selection mutually-exclusive group + # sel_group = parser.add_argument_group("Output selection (mutually exclusive)") sel_mut = sel_group.add_mutually_exclusive_group()
@@ -253,7 +261,9 @@ def main(): sel_mut.add_argument("-s", "-function", "--symbol", action='append', help=FUNCTION_DESC) + # # Those are valid for all 3 types of filter + # parser.add_argument("-n", "-nosymbol", "--nosymbol", action='append', help=NOSYMBOL_DESC)
@@ -286,9 +296,11 @@ def main(): python_ver = sys.version_info[:2] if python_ver < (3,6): + # # Depending on Kernel configuration, kernel-doc --none is called at # build time. As we don't want to break compilation due to the # usage of an old Python version, return 0 here. + # if args.none: logger.error("Python 3.6 or later is required by kernel-doc. skipping checks") sys.exit(0)
@@ -298,7 +310,9 @@ def main(): if python_ver < (3,7): logger.warning("Python 3.7 or later is required for correct results") + # # Import kernel-doc libraries only after checking Python version + # from kdoc.kdoc_files import KernelFiles # pylint: disable=C0415 from kdoc.kdoc_output import RestFormat, ManFormat # pylint: disable=C0415
@@ -346,6 +360,8 @@ def main(): sys.exit(error_count) sys.exit(0) +# # Call main method +# if __name__ == "__main__": main()
Signature · 1 line
2.52.0
PATCH 4/4 scripts/kernel-doc: some fixes to kernel-doc comments
In reply to Mauro Carvalho ChehabThere are some typos and English errors at the kernel-doc.py comments.
Locate them with the help of LLM (gpt-oss 14B), locally excecuted with this prompt:
review English grammar andsyntax at the comments on the code below:
<cat scripts/kernel-doc.py>
Not all results are flowers, although it caught several minor issues there. Add the pertinent fixes, discarding the bad ones.
--- scripts/kernel-doc.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
scripts/kernel-doc.py +12 -12
@@ -9,10 +9,10 @@ # The rationale is that it shall fail gracefully during Kernel # compilation with older Kernel versions. Due to that: # - encoding line is needed here; -# - no f-strings can be used on this file. -# - the libraries that require newer versions can only be included -# after Python version is checked. - +# - f‑strings cannot be used in this file. +# - libraries that require newer versions can only be included +# after the Python version has been checked. +# # Converted from the kernel-doc script originally written in Perl # under GPLv2, copyrighted since 1998 by the following authors: #
@@ -132,13 +132,13 @@ May be used multiple times. """ EXPORT_DESC = """ -Only output documentation for the symbols that have been +Only output documentation for symbols that have been exported using EXPORT_SYMBOL() and related macros in any input FILE or -export-file FILE. """ INTERNAL_DESC = """ -Only output documentation for the symbols that have NOT been +Only output documentation for symbols that have NOT been exported using EXPORT_SYMBOL() and related macros in any input FILE or -export-file FILE. """
@@ -161,7 +161,7 @@ Header and C source files to be parsed. """ WARN_CONTENTS_BEFORE_SECTIONS_DESC = """ -Warns if there are contents before sections (deprecated). +Warn if there are contents before sections (deprecated). This option is kept just for backward-compatibility, but it does nothing, neither here nor at the original Perl script.
@@ -169,7 +169,7 @@ neither here nor at the original Perl script. class MsgFormatter(logging.Formatter): - """Helper class to format warnings on a similar way to kernel-doc.pl""" + """Helper class to format warnings in a similar way to kernel-doc.pl.""" def format(self, record): record.levelname = record.levelname.capitalize()
@@ -268,7 +268,7 @@ def main(): help=NOSYMBOL_DESC) parser.add_argument("-D", "-no-doc-sections", "--no-doc-sections", - action='store_true', help="Don't outputt DOC sections") + action='store_true', help="Don't output DOC sections") parser.add_argument("files", metavar="FILE", nargs="+", help=FILES_DESC)
@@ -297,12 +297,12 @@ def main(): python_ver = sys.version_info[:2] if python_ver < (3,6): # - # Depending on Kernel configuration, kernel-doc --none is called at + # Depending on the Kernel configuration, kernel-doc --none is called at # build time. As we don't want to break compilation due to the # usage of an old Python version, return 0 here. # if args.none: - logger.error("Python 3.6 or later is required by kernel-doc. skipping checks") + logger.error("Python 3.6 or later is required by kernel‑doc. Skipping checks") sys.exit(0) sys.exit("Python 3.6 or later is required by kernel-doc. Aborting.")
@@ -311,7 +311,7 @@ def main(): logger.warning("Python 3.7 or later is required for correct results") # - # Import kernel-doc libraries only after checking Python version + # Import kernel-doc libraries only after checking the Python version # from kdoc.kdoc_files import KernelFiles # pylint: disable=C0415 from kdoc.kdoc_output import RestFormat, ManFormat # pylint: disable=C0415
Signature · 1 line
2.52.0
Re: [PATCH 4/4] scripts/kernel-doc: some fixes to kernel-doc comments
Quoted 81 lines · Mauro Carvalho Chehab
There are some typos and English errors at the
kernel-doc.pycomments.Locate them with the help of LLM (gpt-oss 14B), locally excecuted with this prompt:
review English grammar andsyntax at the comments on the code below:
<catscripts/kernel-doc.py>Not all results are flowers, although it caught several minor issues there. Add the pertinent fixes, discarding the bad ones.
Signed-off-by Mauro Carvalho Chehab <[email protected]>--- scripts/kernel-doc.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)scripts/kernel-doc.py +9
-9@@ -9,10 +9,10 @@ # The rationale is that it shall fail gracefully during Kernel # compilation with older Kernel versions. Due to that: # - encoding line is needed here; -# - no f-strings can be used on this file. -# - the libraries that require newer versions can only be included -# after Python version is checked. - +# - f‑strings cannot be used in this file. +# - libraries that require newer versions can only be included +# after the Python version has been checked. +# # Converted from the kernel-doc script originally written in Perl # under GPLv2, copyrighted since 1998 by the following authors: #@@ -132,13 +132,13 @@ May be used multiple times. """ EXPORT_DESC = """ -Only output documentation for the symbols that have been +Only output documentation for symbols that have been exported using EXPORT_SYMBOL() and related macros in any input FILE or -export-file FILE. """ INTERNAL_DESC = """ -Only output documentation for the symbols that have NOT been +Only output documentation for symbols that have NOT been exported using EXPORT_SYMBOL() and related macros in any input FILE or -export-file FILE. """@@ -161,7 +161,7 @@ Header and C source files to be parsed. """ WARN_CONTENTS_BEFORE_SECTIONS_DESC = """ -Warns if there are contents before sections (deprecated). +Warn if there are contents before sections (deprecated). This option is kept just for backward-compatibility, but it does nothing, neither here nor at the original Perl script.@@ -169,7 +169,7 @@ neither here nor at the original Perl script. class MsgFormatter(logging.Formatter): - """Helper class to format warnings on a similar way to kernel-doc.pl""" + """Helper class to format warnings in a similar way to kernel-doc.pl.""" def format(self, record): record.levelname = record.levelname.capitalize()@@ -268,7 +268,7 @@ def main(): help=NOSYMBOL_DESC) parser.add_argument("-D", "-no-doc-sections", "--no-doc-sections", - action='store_true', help="Don't outputt DOC sections") + action='store_true', help="Don't output DOC sections") parser.add_argument("files", metavar="FILE", nargs="+", help=FILES_DESC)@@ -297,12 +297,12 @@ def main(): python_ver = sys.version_info[:2]
scripts/kernel-doc.py def main(): · line 305
Quoted 10 lines
scripts/kernel-doc.py +2
-2if python_ver < (3,6): # - # Depending on Kernel configuration, kernel-doc --none is called at + # Depending on the Kernel configuration, kernel-doc --none is called at # build time. As we don't want to break compilation due to the # usage of an old Python version, return 0 here. # if args.none: - logger.error("Python 3.6 or later is required by kernel-doc. skipping checks") + logger.error("Python 3.6 or later is required by kernel‑doc. Skipping checks")
Please don't add silly non-breaking space in there.
Quoted 12 lines
scripts/kernel-doc.py +1
-1sys.exit(0) sys.exit("Python 3.6 or later is required by kernel-doc. Aborting.")@@ -311,7 +311,7 @@ def main(): logger.warning("Python 3.7 or later is required for correct results") # - # Import kernel-doc libraries only after checking Python version + # Import kernel-doc libraries only after checking the Python version # from kdoc.kdoc_files import KernelFiles # pylint: disable=C0415 from kdoc.kdoc_output import RestFormat, ManFormat # pylint: disable=C0415
Signature · 1 line
Jani Nikula, Intel