Skip to contentSkip to conversation

autobuild-run: prepare_build(): ignore directories in dldir

PATCH New Peter Korsgaard 2 participants 2 messages Apr 4, 2018

lore
Share & tools
lore

Peter Korsgaard

author
#0 lore raw

PATCH v2 1/1 autobuild-run: prepare_build(): ignore directories in dldir

With the change to use subdirs in dldir, prepare_build may end up trying to remove a non-empty subdir, causing an exception:

Process Process-1:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap

    self.run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)

File "/home/peko/buildroot-test/scripts/autobuild-run", line 627, in run_instance
ret = prepare_build(**kwargs)
File "/home/peko/buildroot-test/scripts/autobuild-run", line 301, in prepare_build
os.remove(os.path.join(dldir, f))
OSError: [Errno 21] Is a directory: 'instance-0/dl/mtools'

We only want to delete individual download files, so recursively search for files under dldir.

With this change, flist contains the full path to the files, so also adjust the printing/removal logic.

Signed-off-by Peter Korsgaard <[email protected]>
---
 scripts/autobuild-run | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
scripts/autobuild-run +10 -3
index 0f1c7d7..33d0ae9 100755
@@ -288,17 +288,24 @@ def prepare_build(**kwargs):
     if not os.path.exists(dldir):
         os.mkdir(dldir)
 
+    # recursively find files under root
+    def find_files(root):
+        for r, d, f in os.walk(root):
+            for i in f:
+                yield os.path.join(r, i)
+
     # Remove 5 random files from the download directory. Removing
     # random files from the download directory allows to ensure we
     # regularly re-download files to check that their upstream
     # location is still correct.
     for i in range(0, 5):
-        flist = os.listdir(dldir)
+        flist = list(find_files(dldir))
         if not flist:
             break
         f = flist[randint(0, len(flist) - 1)]
-        log_write(log, "INFO: removing %s from downloads" % f)
-        os.remove(os.path.join(dldir, f))
+        log_write(log, "INFO: removing %s from downloads" %
+                  os.path.relpath(f, dldir))
+        os.remove(f)
 
     branch = get_branch()
     log_write(log, "INFO: testing branch '%s'" % branch)
Signature · 1 line
2.11.0
#1 lore raw

PATCH v2 1/1 autobuild-run: prepare_build(): ignore directories in dldir

Hello,

Quoted 18 lines · Peter Korsgaard

With the change to use subdirs in dldir, prepare_build may end up trying to remove a non-empty subdir, causing an exception:

Process Process-1:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap

    self.run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)

File "/home/peko/buildroot-test/scripts/autobuild-run", line 627, in run_instance
ret = prepare_build(**kwargs)
File "/home/peko/buildroot-test/scripts/autobuild-run", line 301, in prepare_build
os.remove(os.path.join(dldir, f))
OSError: [Errno 21] Is a directory: 'instance-0/dl/mtools'

We only want to delete individual download files, so recursively search for files under dldir.

Quoted 7 lines

With this change, flist contains the full path to the files, so also adjust the printing/removal logic.

Signed-off-by Peter Korsgaard <[email protected]>
---
 scripts/autobuild-run | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

Applied to buildroot-test repo. Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com