A couple more memres fixes

PATCH New Paul Eggleton 1 participant 3 messages Jun 14, 2017

lore

Share & tools

lore

Paul Eggleton

author

2017-06-14 10:10 UTC #0 lore raw

cover A couple more memres fixes

The following changes since commit 8d0a76f5a595dddf16b7268bae2c00ef5f568316:

ConfHandler.py: allow require or include with multiple parameters (2017-06-12 15:28:25 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib paule/bb-memres-fixes2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bb-memres-fixes2

Paul Eggleton (2):
cooker: fix always loading cache on every UI start with memres
cooker: ensure graceful exit after exception during BuildCompleted handler

 lib/bb/cooker.py | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

Signature · 1 line

2.9.4

Paul Eggleton

author

2017-06-14 10:10 UTC #1 lore raw

PATCH 1/2 cooker: fix always loading cache on every UI start with memres

The main point of memory resident bitbake is to avoid loading data unnecessarily on every bitbake invocation. Unfortunately the code that updated options from the UI was simply treating the fact that either of the "prefile" or "postfile" options were in the list of options passed in as an indication that the configuration was invalid, which was bad because these are always passed in. We only need to mark the configuration as invalid and thus reload it (and thus reload the cache) if the option value has actually changed.

At the same time, the recently handled "tracking" option needs to be treated in a similar manner since the configuration needs to be reparsed if that has changed. Also, add a few extra debug messages to aid debugging this code in future.

Signed-off-by Paul Eggleton <[email protected]>

---
 lib/bb/cooker.py | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

lib/bb/cooker.py +12 -5

index 661cd0e..6a1b649 100644
@@ -415,17 +415,24 @@ class BBCooker:
         self.ui_cmdline = cmdline
         clean = True
         for o in options:
-            if o in ['prefile', 'postfile']:
-                clean = False
-                server_val = getattr(self.configuration, "%s_server" % o)
+            if o in ['prefile', 'postfile', 'tracking']:
+                server_val = getattr(self.configuration, "%s_server" % o, None)
                 if not options[o] and server_val:
                     # restore value provided on server start
+                    logger.debug(1, "Restoring server value for option '%s'" % o)
                     setattr(self.configuration, o, server_val)
+                    clean = False
+                    continue
+                if getattr(self.configuration, o) == options[o]:
+                    # Value is the same, no need to mark dirty
                     continue
+                else:
+                    logger.debug(1, "Marking as dirty due to '%s' option change to '%s'" % (o, options[o]))
+                    clean = False
             setattr(self.configuration, o, options[o])
         for k in bb.utils.approved_variables():
             if k in environment and k not in self.configuration.env:
-                logger.debug(1, "Updating environment variable %s to %s" % (k, environment[k]))
+                logger.debug(1, "Updating new environment variable %s to %s" % (k, environment[k]))
                 self.configuration.env[k] = environment[k]
                 clean = False
             if k in self.configuration.env and k not in environment:

@@ -435,7 +442,7 @@ class BBCooker:
             if k not in self.configuration.env and k not in environment:
                  continue
             if environment[k] != self.configuration.env[k]:
-                logger.debug(1, "Updating environment variable %s to %s" % (k, environment[k]))
+                logger.debug(1, "Updating environment variable %s from %s to %s" % (k, self.configuration.env[k], environment[k]))
                 self.configuration.env[k] = environment[k]
                 clean = False
         if not clean:

Signature · 1 line

2.9.4

Paul Eggleton

author

2017-06-14 10:10 UTC #2 lore raw

PATCH 2/2 cooker: ensure graceful exit after exception during BuildCompleted handler

In reply to Paul Eggleton

If an event handler for bb.event.BuildCompleted fails, we still need to call finishAsyncCommand() or else BitBake will just exit immediately without showing any error summary, or worse in the case of memory resident mode BitBake will hang and if you Ctrl+C to break out, the command won't be marked as finished which means that no further commands will be able to be executed until the server is manually restarted.

Signed-off-by Paul Eggleton <[email protected]>

---
 lib/bb/cooker.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

lib/bb/cooker.py +4 -2

index 6a1b649..479dc5a 100644
@@ -1361,8 +1361,10 @@ class BBCooker:
                 return False
 
             if not retval:
-                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, targets, failures, interrupted), self.data)
-                self.command.finishAsyncCommand(msg)
+                try:
+                    bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runtaskentries), buildname, targets, failures, interrupted), self.data)
+                finally:
+                    self.command.finishAsyncCommand(msg)
                 return False
             if retval is True:
                 return True

Signature · 1 line

2.9.4