fetch2: Fix bug in file checksum generation

For a while its been puzzling me why connman-gnome rebuilds as often as it does. It turns out you can trigger this with a new checkout of the metadata.

The SRC_URI that is causing the problems is:

SRC_URI = "file://images/*"

and rather oddly the results in checksums for a file "." being added to the tree, e.g.:

('.', 'ab48a68186f0e0f277c21ef4cb390b4b')

The problem is that when iterating files lists, the checksum variable can become set yet we don't break the out from the for loop, which leads to odd (and non-deterministic) entries being added into the file checksum list. The exact item added probably depends on the order of items on the disk.

Before this change, bitbake-diffsigs on connman-gnome:do_fetch would report:

This task depends on the checksums of files: [
('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'), ('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'), ('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'), ('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('.', 'ab48a68186f0e0f277c21ef4cb390b4b')]

Afterwards:

This task depends on the checksums of files: [
('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'), ('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'), ('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'), ('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85')]

which is correct and deterministic without the "." entry.

---

index 4335e7a..8e5342f 100644
@@ -969,6 +969,7 @@ def get_file_checksums(filelist, pn):
                 checksum = checksum_file(f)
                 if checksum:
                     checksums.append((f, checksum))
+            continue
         elif os.path.isdir(pth):
             # Handle directories
             for root, dirs, files in os.walk(pth):

@@ -977,6 +978,7 @@ def get_file_checksums(filelist, pn):
                     checksum = checksum_file(fullpth)
                     if checksum:
                         checksums.append((fullpth, checksum))
+            continue
         else:
             checksum = checksum_file(pth)
 

The other patch indicates more specifically what the problem was, this is probably the cleaner code. They amount to the same thing...

Cheers,

Richard

Hi Richard,

b/bitbake/lib/bb/fetch2/__init__.py

index 4335e7a..03ab234 100644
@@ -979,9 +979,8 @@ def get_file_checksums(filelist, pn):
                         checksums.append((fullpth, checksum))
         else:
             checksum = checksum_file(pth)
-
-        if checksum:
-            checksums.append((pth, checksum))
+            if checksum:
+                checksums.append((pth, checksum))
 
     checksums.sort(key=operator.itemgetter(1))
     return checksums