Warning when compiling with python3.12
Subsystems (3)
Hi,
When trying to build on v6.5-rc4 with python 3.12 aka Python 3.12.0rc1 I am hitting the below Warning messages.
I didn't see something similar reported upstream, hence thought of reporting.
/home/srikar/linux.git/scripts/bpf_doc.py:62: SyntaxWarning: invalid escape sequence '\w'
arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')/home/srikar/linux.git/scripts/bpf_doc.py:64: SyntaxWarning: invalid escape sequence '\*'
proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')/home/srikar/linux.git/scripts/bpf_doc.py:117: SyntaxWarning: invalid escape sequence '\*'
p = re.compile(' \* ?(BPF\w+)$')/home/srikar/linux.git/scripts/bpf_doc.py:121: SyntaxWarning: invalid escape sequence '\*'
end_re = re.compile(' \* ?NOTES$')/home/srikar/linux.git/scripts/bpf_doc.py:136: SyntaxWarning: invalid escape sequence '\*'
p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')/home/srikar/linux.git/scripts/bpf_doc.py:144: SyntaxWarning: invalid escape sequence '\*'
p = re.compile(' \* ?(?:\t| {5,8})Description$')/home/srikar/linux.git/scripts/bpf_doc.py:157: SyntaxWarning: invalid escape sequence '\*'
p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')/home/srikar/linux.git/scripts/bpf_doc.py:170: SyntaxWarning: invalid escape sequence '\*'
p = re.compile(' \* ?(?:\t| {5,8})Return$')/home/srikar/linux.git/scripts/bpf_doc.py:183: SyntaxWarning: invalid escape sequence '\*'
p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')/home/srikar/linux.git/scripts/bpf_doc.py:222: SyntaxWarning: invalid escape sequence '\s'
bpf_p = re.compile('\s*(BPF\w+)+')/home/srikar/linux.git/scripts/bpf_doc.py:227: SyntaxWarning: invalid escape sequence '\s'
assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')/home/srikar/linux.git/scripts/bpf_doc.py:242: SyntaxWarning: invalid escape sequence '\w'
self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)/home/srikar/linux.git/scripts/bpf_doc.py:266: SyntaxWarning: invalid escape sequence '\s'
p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')/home/srikar/linux.git/scripts/bpf_doc.py:281: SyntaxWarning: invalid escape sequence '\('
self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)/home/srikar/linux.git/scripts/bpf_doc.py:428: SyntaxWarning: invalid escape sequence '\*'
'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]/home/srikar/linux.git/scripts/bpf_doc.py:499: SyntaxWarning: invalid escape sequence '\ '
footer = '''/home/srikar/linux.git/scripts/bpf_doc.py:601: SyntaxWarning: invalid escape sequence '\ '
one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))
However I am not seeing this when using python 3.10 (Python 3.10.12) and python 3.11 (Python 3.11.4). Note this is just a warning and Kernel build does complete.
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
PATCH 1/1 Fix invalid escape sequence warnings
The Python script bpf_doc.py uses regular expressions with backslashes in string literals, which results in SyntaxWarnings during its execution.
This patch addresses these warnings by converting relevant string literals to raw strings, which interpret backslashes as literal characters. This ensures that the regular expressions are parsed correctly without causing any warnings.
--- scripts/bpf_doc.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-)
scripts/bpf_doc.py +17 -17
@@ -59,9 +59,9 @@ class Helper(APIElement): Break down helper function protocol into smaller chunks: return type, name, distincts arguments. """ - arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$') + arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$') res = {} - proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') + proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') capture = proto_re.match(self.proto) res['ret_type'] = capture.group(1)
@@ -114,11 +114,11 @@ class HeaderParser(object): return Helper(proto=proto, desc=desc, ret=ret) def parse_symbol(self): - p = re.compile(' \* ?(BPF\w+)$') + p = re.compile(r' \* ?(BPF\w+)$') capture = p.match(self.line) if not capture: raise NoSyscallCommandFound - end_re = re.compile(' \* ?NOTES$') + end_re = re.compile(r' \* ?NOTES$') end = end_re.match(self.line) if end: raise NoSyscallCommandFound
@@ -133,7 +133,7 @@ class HeaderParser(object): # - Same as above, with "const" and/or "struct" in front of type # - "..." (undefined number of arguments, for bpf_trace_printk()) # There is at least one term ("void"), and at most five arguments. - p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') + p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') capture = p.match(self.line) if not capture: raise NoHelperFound
@@ -141,7 +141,7 @@ class HeaderParser(object): return capture.group(1) def parse_desc(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Description$') + p = re.compile(r' \* ?(?:\t| {5,8})Description$') capture = p.match(self.line) if not capture: raise Exception("No description section found for " + proto)
@@ -154,7 +154,7 @@ class HeaderParser(object): if self.line == ' *\n': desc += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: desc_present = True
@@ -167,7 +167,7 @@ class HeaderParser(object): return desc def parse_ret(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Return$') + p = re.compile(r' \* ?(?:\t| {5,8})Return$') capture = p.match(self.line) if not capture: raise Exception("No return section found for " + proto)
@@ -180,7 +180,7 @@ class HeaderParser(object): if self.line == ' *\n': ret += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: ret_present = True
@@ -219,12 +219,12 @@ class HeaderParser(object): self.seek_to('enum bpf_cmd {', 'Could not find start of bpf_cmd enum', 0) # Searches for either one or more BPF\w+ enums - bpf_p = re.compile('\s*(BPF\w+)+') + bpf_p = re.compile(r'\s*(BPF\w+)+') # Searches for an enum entry assigned to another entry, # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is # not documented hence should be skipped in check to # determine if the right number of syscalls are documented - assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)') + assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)') bpf_cmd_str = '' while True: capture = assign_p.match(self.line)
@@ -239,7 +239,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of BPF\w+ - self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str) + self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str) def parse_desc_helpers(self): self.seek_to(helpersDocStart,
@@ -263,7 +263,7 @@ class HeaderParser(object): self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)', 'Could not find start of eBPF helper definition list') # Searches for one FN(\w+) define or a backslash for newline - p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\') + p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\') fn_defines_str = '' i = 0 while True:
@@ -278,7 +278,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of FN(\w+) - self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str) + self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str) def validate_helpers(self): last_helper = ''
@@ -425,7 +425,7 @@ class PrinterRST(Printer): try: cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch', '-L', - '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)] + r'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)] date = subprocess.run(cmd, cwd=linuxRoot, capture_output=True, check=True) return date.stdout.decode().rstrip()
@@ -496,7 +496,7 @@ HELPERS date=lastUpdate)) def print_footer(self): - footer = ''' + footer = r''' EXAMPLES ========
@@ -598,7 +598,7 @@ SEE ALSO one_arg = '{}{}'.format(comma, a['type']) if a['name']: if a['star']: - one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*')) + one_arg += r' {}**\ '.format(a['star'].replace('*', '\\*')) else: one_arg += '** ' one_arg += '*{}*\\ **'.format(a['name'])
Signature · 1 line
2.41.0
Re: [PATCH] Fix invalid escape sequence warnings
Quoted 135 lines · Vishal Chourasia
The Python script
bpf_doc.pyuses regular expressions with backslashes in string literals, which results in SyntaxWarnings during its execution.This patch addresses these warnings by converting relevant string literals to raw strings, which interpret backslashes as literal characters. This ensures that the regular expressions are parsed correctly without causing any warnings.
Signed-off-by Vishal Chourasia <[email protected]>Reported-by Srikar Dronamraju <[email protected]>--- scripts/bpf_doc.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-)scripts/bpf_doc.py +14
-14@@ -59,9 +59,9 @@ class Helper(APIElement): Break down helper function protocol into smaller chunks: return type, name, distincts arguments. """ - arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$') + arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$') res = {} - proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') + proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') capture = proto_re.match(self.proto) res['ret_type'] = capture.group(1)@@ -114,11 +114,11 @@ class HeaderParser(object): return Helper(proto=proto, desc=desc, ret=ret) def parse_symbol(self): - p = re.compile(' \* ?(BPF\w+)$') + p = re.compile(r' \* ?(BPF\w+)$') capture = p.match(self.line) if not capture: raise NoSyscallCommandFound - end_re = re.compile(' \* ?NOTES$') + end_re = re.compile(r' \* ?NOTES$') end = end_re.match(self.line) if end: raise NoSyscallCommandFound@@ -133,7 +133,7 @@ class HeaderParser(object): # - Same as above, with "const" and/or "struct" in front of type # - "..." (undefined number of arguments, for bpf_trace_printk()) # There is at least one term ("void"), and at most five arguments. - p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') + p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') capture = p.match(self.line) if not capture: raise NoHelperFound@@ -141,7 +141,7 @@ class HeaderParser(object): return capture.group(1) def parse_desc(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Description$') + p = re.compile(r' \* ?(?:\t| {5,8})Description$') capture = p.match(self.line) if not capture: raise Exception("No description section found for " + proto)@@ -154,7 +154,7 @@ class HeaderParser(object): if self.line == ' *\n': desc += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: desc_present = True@@ -167,7 +167,7 @@ class HeaderParser(object): return desc def parse_ret(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Return$') + p = re.compile(r' \* ?(?:\t| {5,8})Return$') capture = p.match(self.line) if not capture: raise Exception("No return section found for " + proto)@@ -180,7 +180,7 @@ class HeaderParser(object): if self.line == ' *\n': ret += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: ret_present = True@@ -219,12 +219,12 @@ class HeaderParser(object): self.seek_to('enum bpf_cmd {', 'Could not find start of bpf_cmd enum', 0) # Searches for either one or more BPF\w+ enums - bpf_p = re.compile('\s*(BPF\w+)+') + bpf_p = re.compile(r'\s*(BPF\w+)+') # Searches for an enum entry assigned to another entry, # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is # not documented hence should be skipped in check to # determine if the right number of syscalls are documented - assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)') + assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)') bpf_cmd_str = '' while True: capture = assign_p.match(self.line)@@ -239,7 +239,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of BPF\w+ - self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str) + self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str) def parse_desc_helpers(self): self.seek_to(helpersDocStart,@@ -263,7 +263,7 @@ class HeaderParser(object): self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)', 'Could not find start of eBPF helper definition list') # Searches for one FN(\w+) define or a backslash for newline - p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\') + p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\') fn_defines_str = '' i = 0 while True:@@ -278,7 +278,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of FN(\w+) - self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str) + self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str) def validate_helpers(self): last_helper = ''
scripts/bpf_doc.py class PrinterRST(Printer): · line 428
Quoted 5 lines
scripts/bpf_doc.py +1
-1@@ -425,7 +425,7 @@ class PrinterRST(Printer): try: cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch', '-L', - '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)] + r'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
this one is not a regex, do we still need to change it?
scripts/bpf_doc.py HELPERS · line 499
Quoted 9 lines
scripts/bpf_doc.py +1
-1date = subprocess.run(cmd, cwd=linuxRoot, capture_output=True, check=True) return date.stdout.decode().rstrip()@@ -496,7 +496,7 @@ HELPERS date=lastUpdate)) def print_footer(self): - footer = ''' + footer = r'''
same here, not a regex string
scripts/bpf_doc.py SEE ALSO · line 601
Quoted 9 lines
scripts/bpf_doc.py +1
-1EXAMPLES ========@@ -598,7 +598,7 @@ SEE ALSO one_arg = '{}{}'.format(comma, a['type']) if a['name']: if a['star']: - one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*')) + one_arg += r' {}**\ '.format(a['star'].replace('*', '\\*'))
and this one as well?
Quoted 6 lines
scripts/bpf_doc.py +0
-1else: one_arg += '** ' one_arg += '*{}*\\ **'.format(a['name']) --2.41.0
Re: [PATCH] Fix invalid escape sequence warnings
Quoted 133 lines · Andrii Nakryiko
Quoted 132 lines · Vishal Chourasia
The Python script
bpf_doc.pyuses regular expressions with backslashes in string literals, which results in SyntaxWarnings during its execution.This patch addresses these warnings by converting relevant string literals to raw strings, which interpret backslashes as literal characters. This ensures that the regular expressions are parsed correctly without causing any warnings.
Signed-off-by Vishal Chourasia <[email protected]>Reported-by Srikar Dronamraju <[email protected]>--- scripts/bpf_doc.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-)scripts/bpf_doc.py +14
-14@@ -59,9 +59,9 @@ class Helper(APIElement): Break down helper function protocol into smaller chunks: return type, name, distincts arguments. """ - arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$') + arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$') res = {} - proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') + proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') capture = proto_re.match(self.proto) res['ret_type'] = capture.group(1)@@ -114,11 +114,11 @@ class HeaderParser(object): return Helper(proto=proto, desc=desc, ret=ret) def parse_symbol(self): - p = re.compile(' \* ?(BPF\w+)$') + p = re.compile(r' \* ?(BPF\w+)$') capture = p.match(self.line) if not capture: raise NoSyscallCommandFound - end_re = re.compile(' \* ?NOTES$') + end_re = re.compile(r' \* ?NOTES$') end = end_re.match(self.line) if end: raise NoSyscallCommandFound@@ -133,7 +133,7 @@ class HeaderParser(object): # - Same as above, with "const" and/or "struct" in front of type # - "..." (undefined number of arguments, for bpf_trace_printk()) # There is at least one term ("void"), and at most five arguments. - p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') + p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') capture = p.match(self.line) if not capture: raise NoHelperFound@@ -141,7 +141,7 @@ class HeaderParser(object): return capture.group(1) def parse_desc(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Description$') + p = re.compile(r' \* ?(?:\t| {5,8})Description$') capture = p.match(self.line) if not capture: raise Exception("No description section found for " + proto)@@ -154,7 +154,7 @@ class HeaderParser(object): if self.line == ' *\n': desc += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: desc_present = True@@ -167,7 +167,7 @@ class HeaderParser(object): return desc def parse_ret(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Return$') + p = re.compile(r' \* ?(?:\t| {5,8})Return$') capture = p.match(self.line) if not capture: raise Exception("No return section found for " + proto)@@ -180,7 +180,7 @@ class HeaderParser(object): if self.line == ' *\n': ret += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: ret_present = True@@ -219,12 +219,12 @@ class HeaderParser(object): self.seek_to('enum bpf_cmd {', 'Could not find start of bpf_cmd enum', 0) # Searches for either one or more BPF\w+ enums - bpf_p = re.compile('\s*(BPF\w+)+') + bpf_p = re.compile(r'\s*(BPF\w+)+') # Searches for an enum entry assigned to another entry, # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is # not documented hence should be skipped in check to # determine if the right number of syscalls are documented - assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)') + assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)') bpf_cmd_str = '' while True: capture = assign_p.match(self.line)@@ -239,7 +239,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of BPF\w+ - self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str) + self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str) def parse_desc_helpers(self): self.seek_to(helpersDocStart,@@ -263,7 +263,7 @@ class HeaderParser(object): self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)', 'Could not find start of eBPF helper definition list') # Searches for one FN(\w+) define or a backslash for newline - p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\') + p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\') fn_defines_str = '' i = 0 while True:@@ -278,7 +278,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of FN(\w+) - self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str) + self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str)
Quoted 10 lines
scripts/bpf_doc.py class PrinterRST(Printer): · line 428
Quoted 8 lines
scripts/bpf_doc.py +1
-1def validate_helpers(self): last_helper = ''@@ -425,7 +425,7 @@ class PrinterRST(Printer): try: cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch', '-L', - '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)] + r'/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]this one is not a regex, do we still need to change it?
Indeed, it's essential to modify this aspect, especially since we're encountering warnings when compiling the kernel with Python 3.12.
The choice between using an 'r'-prefixed string or manually escaping each backslash is largely a matter of preference. It's worth noting that the 'r' prefix is not only commonly employed for regular expressions but also in contexts where backslashes should be treated as literal characters rather than escape sequences.
Should I send another patch escaping backslash for non regex string literals?
Quoted 30 lines
scripts/bpf_doc.py HELPERS · line 499
Quoted 9 lines
scripts/bpf_doc.py +1
-1date = subprocess.run(cmd, cwd=linuxRoot, capture_output=True, check=True) return date.stdout.decode().rstrip()@@ -496,7 +496,7 @@ HELPERS date=lastUpdate)) def print_footer(self): - footer = ''' + footer = r'''same here, not a regex string
scripts/bpf_doc.py SEE ALSO · line 601
Quoted 9 lines
scripts/bpf_doc.py +1
-1EXAMPLES ========@@ -598,7 +598,7 @@ SEE ALSO one_arg = '{}{}'.format(comma, a['type']) if a['name']: if a['star']: - one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*')) + one_arg += r' {}**\ '.format(a['star'].replace('*', '\\*'))and this one as well?
Quoted 6 lines
scripts/bpf_doc.py +0
-1else: one_arg += '** ' one_arg += '*{}*\\ **'.format(a['name']) --2.41.0
PATCH v2 1/1 Fix invalid escape sequence warnings
The script bpf_doc.py generates multiple SyntaxWarnings related to invalid escape sequences when executed with Python 3.12. These warnings do not appear in Python 3.10 and 3.11 and do not affect the kernel build, which completes successfully.
This patch resolves these SyntaxWarnings by converting the relevant string literals to raw strings or by escaping backslashes. This ensures that backslashes are interpreted as literal characters, eliminating the warnings.
--- scripts/bpf_doc.py | 56 +++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 28 deletions(-)
scripts/bpf_doc.py +28 -28
@@ -59,9 +59,9 @@ class Helper(APIElement): Break down helper function protocol into smaller chunks: return type, name, distincts arguments. """ - arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$') + arg_re = re.compile(r'((\w+ )*?(\w+|...))( (\**)(\w+))?$') res = {} - proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') + proto_re = re.compile(r'(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$') capture = proto_re.match(self.proto) res['ret_type'] = capture.group(1)
@@ -114,11 +114,11 @@ class HeaderParser(object): return Helper(proto=proto, desc=desc, ret=ret) def parse_symbol(self): - p = re.compile(' \* ?(BPF\w+)$') + p = re.compile(r' \* ?(BPF\w+)$') capture = p.match(self.line) if not capture: raise NoSyscallCommandFound - end_re = re.compile(' \* ?NOTES$') + end_re = re.compile(r' \* ?NOTES$') end = end_re.match(self.line) if end: raise NoSyscallCommandFound
@@ -133,7 +133,7 @@ class HeaderParser(object): # - Same as above, with "const" and/or "struct" in front of type # - "..." (undefined number of arguments, for bpf_trace_printk()) # There is at least one term ("void"), and at most five arguments. - p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') + p = re.compile(r' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$') capture = p.match(self.line) if not capture: raise NoHelperFound
@@ -141,7 +141,7 @@ class HeaderParser(object): return capture.group(1) def parse_desc(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Description$') + p = re.compile(r' \* ?(?:\t| {5,8})Description$') capture = p.match(self.line) if not capture: raise Exception("No description section found for " + proto)
@@ -154,7 +154,7 @@ class HeaderParser(object): if self.line == ' *\n': desc += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: desc_present = True
@@ -167,7 +167,7 @@ class HeaderParser(object): return desc def parse_ret(self, proto): - p = re.compile(' \* ?(?:\t| {5,8})Return$') + p = re.compile(r' \* ?(?:\t| {5,8})Return$') capture = p.match(self.line) if not capture: raise Exception("No return section found for " + proto)
@@ -180,7 +180,7 @@ class HeaderParser(object): if self.line == ' *\n': ret += '\n' else: - p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') + p = re.compile(r' \* ?(?:\t| {5,8})(?:\t| {8})(.*)') capture = p.match(self.line) if capture: ret_present = True
@@ -219,12 +219,12 @@ class HeaderParser(object): self.seek_to('enum bpf_cmd {', 'Could not find start of bpf_cmd enum', 0) # Searches for either one or more BPF\w+ enums - bpf_p = re.compile('\s*(BPF\w+)+') + bpf_p = re.compile(r'\s*(BPF\w+)+') # Searches for an enum entry assigned to another entry, # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is # not documented hence should be skipped in check to # determine if the right number of syscalls are documented - assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)') + assign_p = re.compile(r'\s*(BPF\w+)\s*=\s*(BPF\w+)') bpf_cmd_str = '' while True: capture = assign_p.match(self.line)
@@ -239,7 +239,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of BPF\w+ - self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str) + self.enum_syscalls = re.findall(r'(BPF\w+)+', bpf_cmd_str) def parse_desc_helpers(self): self.seek_to(helpersDocStart,
@@ -263,7 +263,7 @@ class HeaderParser(object): self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)', 'Could not find start of eBPF helper definition list') # Searches for one FN(\w+) define or a backslash for newline - p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\') + p = re.compile(r'\s*FN\((\w+), (\d+), ##ctx\)|\\\\') fn_defines_str = '' i = 0 while True:
@@ -278,7 +278,7 @@ class HeaderParser(object): break self.line = self.reader.readline() # Find the number of occurences of FN(\w+) - self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str) + self.define_unique_helpers = re.findall(r'FN\(\w+, \d+, ##ctx\)', fn_defines_str) def validate_helpers(self): last_helper = ''
@@ -425,7 +425,7 @@ class PrinterRST(Printer): try: cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch', '-L', - '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)] + '/{}/,/\\*\\//:include/uapi/linux/bpf.h'.format(delimiter)] date = subprocess.run(cmd, cwd=linuxRoot, capture_output=True, check=True) return date.stdout.decode().rstrip()
@@ -516,7 +516,7 @@ as "Dual BSD/GPL", may be used). Some helper functions are only accessible to programs that are compatible with the GNU Privacy License (GPL). In order to use such helpers, the eBPF program must be loaded with the correct -license string passed (via **attr**) to the **bpf**\ () system call, and this +license string passed (via **attr**) to the **bpf**\\ () system call, and this generally translates into the C source code of the program containing a line similar to the following:
@@ -550,7 +550,7 @@ may be interested in: * The bpftool utility can be used to probe the availability of helper functions on the system (as well as supported program and map types, and a number of other parameters). To do so, run **bpftool feature probe** (see - **bpftool-feature**\ (8) for details). Add the **unprivileged** keyword to + **bpftool-feature**\\ (8) for details). Add the **unprivileged** keyword to list features available to unprivileged users. Compatibility between helper functions and program types can generally be found
@@ -562,23 +562,23 @@ other functions, themselves allowing access to additional helpers. The requirement for GPL license is also in those **struct bpf_func_proto**. Compatibility between helper functions and map types can be found in the -**check_map_func_compatibility**\ () function in file *kernel/bpf/verifier.c*. +**check_map_func_compatibility**\\ () function in file *kernel/bpf/verifier.c*. Helper functions that invalidate the checks on **data** and **data_end** pointers for network processing are listed in function -**bpf_helper_changes_pkt_data**\ () in file *net/core/filter.c*. +**bpf_helper_changes_pkt_data**\\ () in file *net/core/filter.c*. SEE ALSO ======== -**bpf**\ (2), -**bpftool**\ (8), -**cgroups**\ (7), -**ip**\ (8), -**perf_event_open**\ (2), -**sendmsg**\ (2), -**socket**\ (7), -**tc-bpf**\ (8)''' +**bpf**\\ (2), +**bpftool**\\ (8), +**cgroups**\\ (7), +**ip**\\ (8), +**perf_event_open**\\ (2), +**sendmsg**\\ (2), +**socket**\\ (7), +**tc-bpf**\\ (8)''' print(footer) def print_proto(self, helper):
@@ -598,7 +598,7 @@ SEE ALSO one_arg = '{}{}'.format(comma, a['type']) if a['name']: if a['star']: - one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*')) + one_arg += ' {}**\\ '.format(a['star'].replace('*', '\\*')) else: one_arg += '** ' one_arg += '*{}*\\ **'.format(a['name'])
Signature · 1 line
2.41.0
Re: [PATCH v2] Fix invalid escape sequence warnings
Quoted 11 lines · Vishal Chourasia
The script
bpf_doc.pygenerates multiple SyntaxWarnings related to invalid escape sequences when executed with Python 3.12. These warnings do not appear in Python 3.10 and 3.11 and do not affect the kernel build, which completes successfully.This patch resolves these SyntaxWarnings by converting the relevant string literals to raw strings or by escaping backslashes. This ensures that backslashes are interpreted as literal characters, eliminating the warnings.
Signed-off-by Vishal Chourasia <[email protected]>Reported-by Srikar Dronamraju <[email protected]>
Thanks! I observed that this patch fixes warnings reported by pyright in my editor. I've also validated that the generated files (helpers man page, syscall man page, bpf_helper_defs.h) remain unchanged.
Re: [PATCH v2] Fix invalid escape sequence warnings
In reply to Vishal ChourasiaThis patch was applied to bpf/bpf.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This patch was applied to bpf/bpf.git (master)
by Daniel Borkmann <[email protected]>:
Quoted 10 lines · On Tue, 29 Aug 2023 13:19:31 +0530 you wrote:
The script
bpf_doc.pygenerates multiple SyntaxWarnings related to invalid escape sequences when executed with Python 3.12. These warnings do not appear in Python 3.10 and 3.11 and do not affect the kernel build, which completes successfully.This patch resolves these SyntaxWarnings by converting the relevant string literals to raw strings or by escaping backslashes. This ensures that backslashes are interpreted as literal characters, eliminating the warnings.
[...]
Here is the summary with links:
- [v2] Fix invalid escape sequence warnings
https://git.kernel.org/bpf/bpf/c/121fd33bf2d9
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Re: [PATCH] Fix invalid escape sequence warnings
In reply to Vishal ChourasiaQuoted 11 lines · Vishal Chourasia
The Python script
bpf_doc.pyuses regular expressions with backslashes in string literals, which results in SyntaxWarnings during its execution.This patch addresses these warnings by converting relevant string literals to raw strings, which interpret backslashes as literal characters. This ensures that the regular expressions are parsed correctly without causing any warnings.
Signed-off-by Vishal Chourasia <[email protected]>Reported-by Srikar Dronamraju <[email protected]>
Thanks, Works for me
Signature · 2 lines
Thanks and Regards Srikar Dronamraju