scripts/spdxcheck.py: always open files in binary mode
Subsystems (2)
PATCH 11/11 scripts/spdxcheck.py: always open files in binary mode
From: Thierry Reding <[email protected]>
Subject: scripts/spdxcheck.py: always open files in binary mode
The spdxcheck script currently falls over when confronted with a binary file (such as Documentation/logo.gif). To avoid that, always open files in binary mode and decode line-by-line, ignoring encoding errors.
One tricky case is when piping data into the script and reading it from standard input. By default, standard input will be opened in text mode, so we need to reopen it in binary mode.
The breakage only happens with python3 and results in a UnicodeDecodeError (according to Uwe).
Fixes 6f4d29df66ac ("scripts/spdxcheck.py: make python3 compliant")
Signed-off-by Thierry Reding <[email protected]>
Reviewed-by Jeremy Cline <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Joe Perches <[email protected]>
Cc Uwe Kleine-König <[email protected]>
Cc <[email protected]>
Signed-off-by Andrew Morton <[email protected]>
---
scripts/spdxcheck.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
scripts/spdxcheck.py renamed from scripts/spdxcheck.py~scripts-spdxcheckpy-always-open-files-in-binary-mode +4 -2
@@ -168,6 +168,7 @@ class id_parser(object): self.curline = 0 try: for line in fd: + line = line.decode(locale.getpreferredencoding(False), errors='ignore') self.curline += 1 if self.curline > maxlines: break
@@ -249,12 +250,13 @@ if __name__ == '__main__': try: if len(args.path) and args.path[0] == '-': - parser.parse_lines(sys.stdin, args.maxlines, '-') + stdin = os.fdopen(sys.stdin.fileno(), 'rb') + parser.parse_lines(stdin, args.maxlines, '-') else: if args.path: for p in args.path: if os.path.isfile(p): - parser.parse_lines(open(p), args.maxlines, p) + parser.parse_lines(open(p, 'rb'), args.maxlines, p) elif os.path.isdir(p): scan_git_subtree(repo.head.reference.commit.tree, p) else:
_