Skip to contentSkip to conversation

utils/check-package: handle missing files

PATCH Applied James Knight 2 participants 4 messages Jul 1, 2025 – Jul 11, 2025

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 utils/check-package: handle missing files

When running check-package before completing commits for a change, if any files are setup for removal, check-package will throw FileNotFoundError exceptions instead of generating a warning state. For example:

 $ utils/docker-run make check-package
 Traceback (most recent call last):
   ...
 FileNotFoundError: [Errno 2] No such file or directory: 'package/.../0001-some-removed-patch.patch'
 make: *** [Makefile:1264: check-package] Error 1

This commit will now catch FileNotFoundError and populate a warning message:

 $ utils/docker-run make check-package
 package/.../0001-some-removed-patch.patch: missing; unstaged file removal?
 package/.../0002-another-removed-patch.patch: missing; unstaged file removal?
 427843 lines processed
 3 warnings generated
 make: *** [Makefile:1264: check-package] Error 1
---
 utils/check-package | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
utils/check-package +17 -12
index a5ecc2eed8..8781d21d8f 100755
@@ -257,18 +257,23 @@ def check_file_using_lib(fname):
         nwarnings += warn
 
     lastline = ""
-    with open(fname, "r", errors="surrogateescape") as f:
-        for lineno, text in enumerate(f):
-            nlines += 1
-            for name, cf in objects:
-                if cf.disable.search(lastline):
-                    continue
-                line_sts = cf.check_line(lineno + 1, text)
-                warn, fail = print_warnings(line_sts, name in xfail)
-                if fail > 0:
-                    failed.add(name)
-                nwarnings += warn
-            lastline = text
+    try:
+        with open(fname, "r", errors="surrogateescape") as f:
+            for lineno, text in enumerate(f):
+                nlines += 1
+                for name, cf in objects:
+                    if cf.disable.search(lastline):
+                        continue
+                    line_sts = cf.check_line(lineno + 1, text)
+                    warn, fail = print_warnings(line_sts, name in xfail)
+                    if fail > 0:
+                        failed.add(name)
+                    nwarnings += warn
+                lastline = text
+    except FileNotFoundError:
+        print(f"{fname}: missing; unstaged file removal?")
+        nwarnings += 1
+        return nwarnings, nlines
 
     for name, cf in objects:
         warn, fail = print_warnings(cf.after(), name in xfail)
Signature · 6 lines
2.49.0.windows.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Reviewed-by Romain Naour on v1 1/1
#1 lore raw

Hello James, All,

Le 01/07/2025 à 20:56, James Knight a écrit :

Quoted 10 lines

When running check-package before completing commits for a change, if any files are setup for removal, check-package will throw FileNotFoundError exceptions instead of generating a warning state. For example:

 $ utils/docker-run make check-package
 Traceback (most recent call last):
   ...
 FileNotFoundError: [Errno 2] No such file or directory: 'package/.../0001-some-removed-patch.patch'
 make: *** [Makefile:1264: check-package] Error 1

Indeed, it's due to git ls-tree -r --name-only HEAD being used by check-package Make target:

./utils/check-package `git ls-tree -r --name-only HEAD` \
	--ignore-list=$(TOPDIR)/.checkpackageignore

When working on a specific package, we could use check-package directly without error:

git rm boot/syslinux/0017-Replace-builtin-strlen-that-appears-to-get-optimized.patch

./utils/check-package boot/syslinux/*

While running check-package before completing commits for a change is probably not a good practice, throwing FileNotFoundError exception is not nice :)

Best regards,
Romain

Quoted 46 lines

This commit will now catch FileNotFoundError and populate a warning message:

 $ utils/docker-run make check-package
 package/.../0001-some-removed-patch.patch: missing; unstaged file removal?
 package/.../0002-another-removed-patch.patch: missing; unstaged file removal?
 427843 lines processed
 3 warnings generated
 make: *** [Makefile:1264: check-package] Error 1
---
 utils/check-package | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
utils/check-package +10 -12
index a5ecc2eed8..8781d21d8f 100755
@@ -257,18 +257,23 @@ def check_file_using_lib(fname):
         nwarnings += warn
 
     lastline = ""
-    with open(fname, "r", errors="surrogateescape") as f:
-        for lineno, text in enumerate(f):
-            nlines += 1
-            for name, cf in objects:
-                if cf.disable.search(lastline):
-                    continue
-                line_sts = cf.check_line(lineno + 1, text)
-                warn, fail = print_warnings(line_sts, name in xfail)
-                if fail > 0:
-                    failed.add(name)
-                nwarnings += warn
-            lastline = text
+    try:
+        with open(fname, "r", errors="surrogateescape") as f:
+            for lineno, text in enumerate(f):
+                nlines += 1
+                for name, cf in objects:
+                    if cf.disable.search(lastline):
+                        continue
+                    line_sts = cf.check_line(lineno + 1, text)
+                    warn, fail = print_warnings(line_sts, name in xfail)
+                    if fail > 0:

utils/check-package def check_file_using_lib(fname): · line 279

Quoted 10 lines
utils/check-package +7 -0
+                        failed.add(name)
+                    nwarnings += warn
+                lastline = text
+    except FileNotFoundError:
+        print(f"{fname}: missing; unstaged file removal?")
+        nwarnings += 1
+        return nwarnings, nlines
 
     for name, cf in objects:
         warn, fail = print_warnings(cf.after(), name in xfail)

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#2 lore raw
In reply to James Knight
Quoted 24 lines · James Knight

When running check-package before completing commits for a change, if any files are setup for removal, check-package will throw FileNotFoundError exceptions instead of generating a warning state. For example:

 $ utils/docker-run make check-package
 Traceback (most recent call last):
   ...
 FileNotFoundError: [Errno 2] No such file or directory: 
'package/.../0001-some-removed-patch.patch'
 make: *** [Makefile:1264: check-package] Error 1

This commit will now catch FileNotFoundError and populate a warning message:

 $ utils/docker-run make check-package
 package/.../0001-some-removed-patch.patch: missing; unstaged file 
removal?
 package/.../0002-another-removed-patch.patch: missing; unstaged file 
removal?
 427843 lines processed
 3 warnings generated
 make: *** [Makefile:1264: check-package] Error 1
Quoted 1 line

Applied to master, thanks.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw
In reply to James Knight

In reply of:

Quoted 14 lines

When running check-package before completing commits for a change, if any files are setup for removal, check-package will throw FileNotFoundError exceptions instead of generating a warning state. For example:

 $ utils/docker-run make check-package
 Traceback (most recent call last):
   ...
 FileNotFoundError: [Errno 2] No such file or directory: 'package/.../0001-some-removed-patch.patch'
 make: *** [Makefile:1264: check-package] Error 1

This commit will now catch FileNotFoundError and populate a warning message:

Quoted 8 lines
 $ utils/docker-run make check-package
 package/.../0001-some-removed-patch.patch: missing; unstaged file removal?
 package/.../0002-another-removed-patch.patch: missing; unstaged file removal?
 427843 lines processed
 3 warnings generated
 make: *** [Makefile:1264: check-package] Error 1

Applied to 2025.02.x & 2025.05.x. Thanks

Quoted 51 lines
---
 utils/check-package | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
utils/check-package +17 -12
index a5ecc2eed8..8781d21d8f 100755
@@ -257,18 +257,23 @@ def check_file_using_lib(fname):
         nwarnings += warn
 
     lastline = ""
-    with open(fname, "r", errors="surrogateescape") as f:
-        for lineno, text in enumerate(f):
-            nlines += 1
-            for name, cf in objects:
-                if cf.disable.search(lastline):
-                    continue
-                line_sts = cf.check_line(lineno + 1, text)
-                warn, fail = print_warnings(line_sts, name in xfail)
-                if fail > 0:
-                    failed.add(name)
-                nwarnings += warn
-            lastline = text
+    try:
+        with open(fname, "r", errors="surrogateescape") as f:
+            for lineno, text in enumerate(f):
+                nlines += 1
+                for name, cf in objects:
+                    if cf.disable.search(lastline):
+                        continue
+                    line_sts = cf.check_line(lineno + 1, text)
+                    warn, fail = print_warnings(line_sts, name in xfail)
+                    if fail > 0:
+                        failed.add(name)
+                    nwarnings += warn
+                lastline = text
+    except FileNotFoundError:
+        print(f"{fname}: missing; unstaged file removal?")
+        nwarnings += 1
+        return nwarnings, nlines
 
     for name, cf in objects:
         warn, fail = print_warnings(cf.after(), name in xfail)
Signature · 6 lines
2.49.0.windows.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot