Avoid build failures due to setscene errors

Occasionally, we see errors on our autobuilders where a setscene task fails to retrieve a file from our global sstate cache. It typically looks something like this:

WARNING: zip-3.0-r2 do_populate_sysroot_setscene: Failed to fetch URL file://66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz;\
downloadfilename=66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz, attempting MIRRORS if available
ERROR: zip-3.0-r2 do_populate_sysroot_setscene: Fetcher failure:
Unable to find file file://66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz;\
downloadfilename=66/sstate:zip:core2-64-poky-linux:3.0:r2:core2-64:3:\
66832b8c4e7babe0eac9d9579d1e2b6a_populate_sysroot.tgz anywhere. The paths that were searched were:
/home/pkj/.openembedded/sstate-cache
ERROR: zip-3.0-r2 do_populate_sysroot_setscene: No suitable staging package found
WARNING: Setscene task (meta/recipes-extended/zip/zip_3.0.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead

As the last warning indicates, the build will proceed and the real task will run and the build will eventually complete. However, due to the two errors above, bitbake will return with an error code which causes the autobuilder to treat the build as failed and it proceeds to throw everything it built away.

Since this is quite pointless and causes unnecessary build resources to be spent and grief from the developers, the two patches in this change set turn the errors from setscene tasks into warnings.

Well, there is actually only one patch in this change set, as the other patch is for OE-Core...

//Peter

The following changes since commit bc2e0b2e9b95707d96c840dade12b00e1450ecc3:

libsdl: Move PACKAGECONFIG options from meta-mingw (2017-08-29 12:23:10 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib pkj/setscene-errors-bitbake
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/setscene-errors-bitbake

Peter Kjellerstedt (1):
fetch2: Allow Fetch.download() to warn instead of error

 bitbake/lib/bb/fetch2/__init__.py | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

2.12.0

Under some situations it can be allowed for Fetch.download() to fail to fetch a file without causing bitbake to fail. By adding only_warn=True as argument to Fetch.download(), it will call logger.warning() instead of logger.error() and thus not cause build failures.

---
 bitbake/lib/bb/fetch2/__init__.py | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

index 3eb0e4d211..58f65ada84 100644
@@ -1608,9 +1608,10 @@ class Fetch(object):
 
         return local
 
-    def download(self, urls=None):
+    def download(self, urls=None, only_warn=False):
         """
-        Fetch all urls
+        Fetch all urls. In case only_warn is True, a failure to fetch a url
+        will only result in a warning message, rather than an error message.
         """
         if not urls:
             urls = self.urls

@@ -1688,19 +1689,28 @@ class Fetch(object):
 
                 if not localpath or ((not os.path.exists(localpath)) and localpath.find("*") == -1):
                     if firsterr:
-                        logger.error(str(firsterr))
+                        if only_warn:
+                            logger.warning(str(firsterr))
+                        else:
+                            logger.error(str(firsterr))
                     raise FetchError("Unable to fetch URL from any source.", u)
 
                 update_stamp(ud, self.d)
 
             except IOError as e:
                 if e.errno in [os.errno.ESTALE]:
-                    logger.error("Stale Error Observed %s." % u)
+                    if only_warn:
+                        logger.warning("Stale Error Observed %s." % u)
+                    else:
+                        logger.error("Stale Error Observed %s." % u)
                     raise ChecksumError("Stale Error Detected")
 
             except BBFetchException as e:
                 if isinstance(e, ChecksumError):
-                    logger.error("Checksum failure fetching %s" % u)
+                    if only_warn:
+                        logger.warning("Checksum failure fetching %s" % u)
+                    else:
+                        logger.error("Checksum failure fetching %s" % u)
                 raise
 
             finally:

2.12.0