Skip to contentSkip to conversation

Various hob related patches

PATCH Applied Joshua Lock 2 participants 10 messages Aug 11, 2011 – Aug 12, 2011 Arm STM32

lore
Share & tools
lore
#0 lore raw

cover Various hob related patches

This pull request includes the series I submitted yesterday as well as some additional bug fixes.

Regards,
Joshua

The following changes since commit 705d14d1e1108e0544c7eab827f1242f0839add9:

bb/cooker: only emit ConfigFilePathFound for files which were parsed (2011-08-10 13:38:11 +0100)

are available in the git repository at:
git://github.com/incandescant/bitbake hob
https://github.com/incandescant/bitbake/tree/hob

Joshua Lock (8):
bb/ui/crumbs/tasklistmodel: fix some typos and add comments to mark() bb/ui/crumbs/tasklistmodel: correctly uniquify dependency list bb/ui/crumbs/tasklistmodel: don't include an item in its own depends bb/cache: rename confusing variable
bb/command|cooker: refactor the reparseFiles logic bb/ui/crumbs/hobeventhandler: adapt to reset -> reparse change bb/ui/crumbs/configurator: introduce writeConfFile method for all
writes
lib/bb/ui/crumbs/hobprefs: fix erroneous save/reparse

 lib/bb/cache.py                     |    6 ++--
 lib/bb/command.py                   |    7 +++++
 lib/bb/cooker.py                    |    9 ++++--
 lib/bb/ui/crumbs/configurator.py    |   46 ++++++++++++++--------------------
 lib/bb/ui/crumbs/hobeventhandler.py |    1 +
 lib/bb/ui/crumbs/hobprefs.py        |    1 +
 lib/bb/ui/crumbs/tasklistmodel.py   |   28 +++++++++++++++------
 7 files changed, 57 insertions(+), 41 deletions(-)
Signature · 1 line
1.7.6
#1 lore raw

PATCH 1/8 bb/ui/crumbs/tasklistmodel: fix some typos and add comments to mark()

Two similarly named variables in the mark() method resulted in the wrong variable being used in a couple of places. This patch adresses this in several ways:
1) Renames the variables to be less similar
2) Uses the correct variables
3) Adds some coments to document the methods intent

Partially addresses [YOCTO #1355]

---
 lib/bb/ui/crumbs/tasklistmodel.py |   22 +++++++++++++++-------
 1 files changed, 15 insertions(+), 7 deletions(-)
lib/bb/ui/crumbs/tasklistmodel.py +15 -7
index 3e09757..96814c2 100644
@@ -316,9 +316,13 @@ class TaskListModel(gtk.ListStore):
     def mark(self, opath):
         usersel = {}
         removed = []
+
         it = self.get_iter_first()
-        name = self[opath][self.COL_NAME]
+        # The name of the item we're removing, so that we can use it to find
+        # other items which either depend on it, or were brought in by it
+        marked_name = self[opath][self.COL_NAME]
 
+        # Remove the passed item
         self.remove_item_path(opath)
 
         # Remove all dependent packages, update binb
@@ -330,7 +334,7 @@ class TaskListModel(gtk.ListStore):
             deps = self[path][self.COL_DEPS]
             binb = self[path][self.COL_BINB]
             itype = self[path][self.COL_TYPE]
-            iname = self[path][self.COL_NAME]
+            itname = self[path][self.COL_NAME]
 
             # We ignore anything that isn't a package
             if not itype == "package":
@@ -341,16 +345,20 @@ class TaskListModel(gtk.ListStore):
             # is to save its name and re-mark it for inclusion once dependency
             # processing is complete
             if binb == "User Selected":
-                usersel[iname] = self[path][self.COL_IMG]
+                usersel[itname] = self[path][self.COL_IMG]
 
+            # If the iterated item is included and depends on the removed
+            # item it should also be removed.
             # FIXME: need to ensure partial name matching doesn't happen
-            if inc and deps.count(name) and name not in removed:
+            if inc and deps.count(marked_name) and itname not in removed:
                 # found a dependency, remove it
-                removed.append(name)
+                removed.append(itname)
                 self.mark(path)
 
-            if inc and binb.count(name):
-                bib = self.find_alt_dependency(name)
+            # If the iterated item was brought in by the removed (passed) item
+            # try and find an alternative dependee and update the binb column
+            if inc and binb.count(marked_name):
+                bib = self.find_alt_dependency(itname)
                 self[path][self.COL_BINB] = bib
 
         # Re-add any removed user selected items
Signature · 1 line
1.7.6
#2 lore raw

PATCH 2/8 bb/ui/crumbs/tasklistmodel: correctly uniquify dependency list

In reply to Joshua Lock

Fix thinko - the squish method returns a uniquified list, it doesn't modify the list in place. Therefore the call to squish() was useless as its return value was never assigned.

---
 lib/bb/ui/crumbs/tasklistmodel.py |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
lib/bb/ui/crumbs/tasklistmodel.py +2 -1
index 96814c2..aec80e2 100644
@@ -235,7 +235,8 @@ class TaskListModel(gtk.ListStore):
                         if pn:
                             depends.append(pn)
 
-            self.squish(depends)
+            # uniquify the list of depends
+            depends = self.squish(depends)
             deps = " ".join(depends)
 
             if name.count('task-') > 0:
Signature · 1 line
1.7.6
#3 lore raw

PATCH 3/8 bb/ui/crumbs/tasklistmodel: don't include an item in its own depends

In reply to Joshua Lock

This causes the simple removal algorithm to perform needless circular logic

---
 lib/bb/ui/crumbs/tasklistmodel.py |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
lib/bb/ui/crumbs/tasklistmodel.py +3 -0
index aec80e2..7a463a6 100644
@@ -237,6 +237,9 @@ class TaskListModel(gtk.ListStore):
 
             # uniquify the list of depends
             depends = self.squish(depends)
+            # remove circular dependencies
+            if name in depends:
+                depends.remove(name)
             deps = " ".join(depends)
 
             if name.count('task-') > 0:
Signature · 1 line
1.7.6
#4 lore raw

PATCH 4/8 bb/cache: rename confusing variable

In reply to Joshua Lock

The bNeedUpdate variable doesn't reflect its purpose, and doesn't match coding style (type encoded in variable name, camel case) - rename to cache_ok.

---
 lib/bb/cache.py |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)
lib/bb/cache.py +3 -3
index d805d46..d495f9e 100644
@@ -286,14 +286,14 @@ class Cache(object):
         old_mtimes.append(newest_mtime)
         newest_mtime = max(old_mtimes)
 
-        bNeedUpdate = True
+        cache_ok = True
         if self.caches_array:
             for cache_class in self.caches_array:
                 if type(cache_class) is type and issubclass(cache_class, RecipeInfoCommon):
                     cachefile = getCacheFile(self.cachedir, cache_class.cachefile)
-                    bNeedUpdate = bNeedUpdate and (bb.parse.cached_mtime_noerror(cachefile) >= newest_mtime)
+                    cache_ok = cache_ok and (bb.parse.cached_mtime_noerror(cachefile) >= newest_mtime)
                     cache_class.init_cacheData(self)
-        if bNeedUpdate:
+        if cache_ok:
             self.load_cachefile()
         elif os.path.isfile(self.cachefile):
             logger.info("Out of date cache found, rebuilding...")
Signature · 1 line
1.7.6
#5 lore raw

PATCH 5/8 bb/command|cooker: refactor the reparseFiles logic

In reply to Joshua Lock

Turn the reparseFiles logic into a command to reset the cooker's state machine and a noop which triggers a cache rebuild.

The resetCooker command resets the cookers state machine such that a cache update will be triggered by any async command which requires the cache. The reparseFiles command remains as a noop async command that has the needcache property set to True so that when called it ensures the cache is built.

Patch from Richard with the addition of removing the force parameter from the updateCache method.

CC Richard Purdie <[email protected]>
---
 lib/bb/command.py |    7 +++++++
 lib/bb/cooker.py  |    9 ++++++---
 2 files changed, 13 insertions(+), 3 deletions(-)
lib/bb/command.py +7 -0
index 893a6d9..41796f6 100644
@@ -172,6 +172,13 @@ class CommandsSync:
         value = params[1]
         bb.data.setVar(varname, value, command.cooker.configuration.data)
 
+    def resetCooker(self, command, params):
+        """
+        Reset the cooker to its initial state, thus forcing a reparse for
+        any async command that has the needcache property set to True
+        """
+        command.cooker.reset()
+
 
 class CommandsAsync:
     """
lib/bb/cooker.py +6 -3
index a8edfe7..7641d60 100644
@@ -1091,8 +1091,8 @@ class BBCooker:
 
         self.server_registration_cb(buildTargetsIdle, rq)
 
-    def updateCache(self, force=False):
-        if self.state == state.running and not force:
+    def updateCache(self):
+        if self.state == state.running:
             return
 
         if self.state in (state.shutdown, state.stop):
@@ -1277,8 +1277,11 @@ class BBCooker:
         self.state = state.stop
 
     def reparseFiles(self):
+        return
+
+    def reset(self):
+        self.state = state.initial
         self.loadConfigurationData()
-        self.updateCache(force=True)
 
 def server_main(cooker, func, *args):
     cooker.pre_serve()
Signature · 1 line
1.7.6
#6 lore raw

PATCH 6/8 bb/ui/crumbs/hobeventhandler: adapt to reset -> reparse change

In reply to Joshua Lock

The API has changed from just a reparse call to a reset call followed by a reparse call.

---
 lib/bb/ui/crumbs/hobeventhandler.py |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
lib/bb/ui/crumbs/hobeventhandler.py +1 -0
index 2e0a999..1a4c7b5 100644
@@ -120,6 +120,7 @@ class HobHandler(gobject.GObject):
                 self.current_command = self.BUILD_IMAGE
             else:
                 self.current_command = self.CFG_PATH_LAYERS
+            self.server.runCommand(["resetCooker"])
             self.server.runCommand(["reparseFiles"])
         elif self.current_command == self.BUILD_IMAGE:
             self.building = "image"
Signature · 1 line
1.7.6
#7 lore raw

PATCH 7/8 bb/ui/crumbs/configurator: introduce writeConfFile method for all writes

In reply to Joshua Lock

Configuration files are written in several places, this refactors the code to use a common method.

---
 lib/bb/ui/crumbs/configurator.py |   46 +++++++++++++++----------------------
 1 files changed, 19 insertions(+), 27 deletions(-)
lib/bb/ui/crumbs/configurator.py +19 -27
index e17d555..c37e917 100644
@@ -207,6 +207,19 @@ class Configurator(gobject.GObject):
 
         return "".join(layer_entry)
 
+    def writeConfFile(self, conffile, contents):
+        """
+        Make a backup copy of conffile and write a new file in its stead with
+        the lines in the contents list.
+        """
+        # Create a backup of the conf file
+        bkup = "%s~" % conffile
+        os.rename(conffile, bkup)
+
+        # Write the contents list object to the conf file
+        with open(conffile, "w") as new:
+            new.write("".join(contents))
+
     def writeLocalConf(self):
         # Dictionary containing only new or modified variables
         changed_values = {}
@@ -218,12 +231,8 @@ class Configurator(gobject.GObject):
         if not len(changed_values):
             return
 
-        # Create a backup of the local.conf
-        bkup = "%s~" % self.local
-        os.rename(self.local, bkup)
-
         # read the original conf into a list
-        with open(bkup, 'r') as config:
+        with open(self.local, 'r') as config:
             config_lines = config.readlines()
 
         new_config_lines = ["\n"]
@@ -259,20 +268,14 @@ class Configurator(gobject.GObject):
         # Add the modified variables
         config_lines.extend(new_config_lines)
 
-        # Write the updated lines list object to the local.conf
-        with open(self.local, "w") as n:
-            n.write("".join(config_lines))
+        self.writeConfFile(self.local, config_lines)
 
         del self.orig_config
         self.orig_config = copy.deepcopy(self.config)
 
     def insertTempBBPath(self, bbpath, bbfiles):
-        # Create a backup of the local.conf
-        bkup = "%s~" % self.local
-        os.rename(self.local, bkup)
-
         # read the original conf into a list
-        with open(bkup, 'r') as config:
+        with open(self.local, 'r') as config:
             config_lines = config.readlines()
 
         if bbpath:
@@ -280,9 +283,7 @@ class Configurator(gobject.GObject):
         if bbfiles:
             config_lines.append("BBFILES := \"${BBFILES} %s\"\n" % bbfiles)
 
-        # Write the updated lines list object to the local.conf
-        with open(self.local, "w") as n:
-            n.write("".join(config_lines))
+        self.writeConfFile(self.local, config_lines)
 
     def writeLayerConf(self):
         # If we've not added/removed new layers don't write
@@ -292,23 +293,14 @@ class Configurator(gobject.GObject):
         # This pattern should find the existing BBLAYERS
         pattern = 'BBLAYERS\s=\s\".*\"'
 
-        # Backup the users bblayers.conf
-        bkup = "%s~" % self.bblayers
-        os.rename(self.bblayers, bkup)
-
         replacement = self._constructLayerEntry()
 
-        with open(bkup, "r") as f:
+        with open(self.bblayers, "r") as f:
             contents = f.read()
             p = re.compile(pattern, re.DOTALL)
             new = p.sub(replacement, contents)
 
-        with open(self.bblayers, "w") as n:
-            n.write(new)
-
-        # At some stage we should remove the backup we've created
-        # though we should probably verify it first
-        #os.remove(bkup)
+        self.writeConfFile(self.bblayers, new)
 
         # set loaded_layers for dirtiness tracking
         self.loaded_layers = copy.deepcopy(self.enabled_layers)
Signature · 1 line
1.7.6
#8 lore raw

PATCH 8/8 lib/bb/ui/crumbs/hobprefs: fix erroneous save/reparse

In reply to Joshua Lock

We need to unset the reload_required variable once we've triggered a reload so that we don't cause a reload each time the preferences dialog is shown and dismissed, regardless of whether anything has changed.

---
 lib/bb/ui/crumbs/hobprefs.py |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
lib/bb/ui/crumbs/hobprefs.py +1 -0
index 8ebfba2..014087b 100644
@@ -163,6 +163,7 @@ class HobPrefs(gtk.Dialog):
     def prefs_response_cb(self, dialog, response):
         if self.reload_required:
             glib.idle_add(self.handler.reload_data)
+            self.reload_required = False
 
     def __init__(self, configurator, handler, curr_sdk_mach, curr_distro, pclass,
                  cpu_cnt, pmake, bbthread, selected_image_types, all_image_types,
Signature · 1 line
1.7.6
#9 lore raw
In reply to Joshua Lock
Quoted 14 lines · Joshua Lock

This pull request includes the series I submitted yesterday as well as some additional bug fixes.

Regards,
Joshua

The following changes since commit 705d14d1e1108e0544c7eab827f1242f0839add9:

bb/cooker: only emit ConfigFilePathFound for files which were parsed (2011-08-10 13:38:11 +0100)

are available in the git repository at:
git://github.com/incandescant/bitbake hob
https://github.com/incandescant/bitbake/tree/hob

Quoted 10 lines

Joshua Lock (8):
bb/ui/crumbs/tasklistmodel: fix some typos and add comments to mark() bb/ui/crumbs/tasklistmodel: correctly uniquify dependency list bb/ui/crumbs/tasklistmodel: don't include an item in its own depends bb/cache: rename confusing variable
bb/command|cooker: refactor the reparseFiles logic bb/ui/crumbs/hobeventhandler: adapt to reset -> reparse change bb/ui/crumbs/configurator: introduce writeConfFile method for all
writes
lib/bb/ui/crumbs/hobprefs: fix erroneous save/reparse

Merged to master thanks.

Richard