Skip to contentSkip to conversation

data: fix performance regression

PATCH In review Paul Eggleton 2 participants 2 messages Apr 15, 2013

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 data: fix performance regression

BitBake commit 7c568132c54a21161de28907159f902462f1e2bb resulted in a fairly serious performance regression during parsing, almost doubling the time taken to do a full parse and almost certainly impacting performance during building. The expandKeys function is called frequently, and if we avoid using keys() and instead just use the normal variable lookup mechanism, performance is restored.

---
 lib/bb/data.py |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/bb/data.py +2 -2
index 110666c..abf210a 100644
@@ -158,9 +158,9 @@ def expandKeys(alterdata, readdata = None):
 
     for key in todolist:
         ekey = todolist[key]
-        if ekey in keys(alterdata):
+        newval = alterdata.getVar(ekey, 0)
+        if newval:
             val = alterdata.getVar(key, 0)
-            newval = alterdata.getVar(ekey, 0)
             if val is not None and newval is not None:
                 bb.warn("Variable key %s (%s) replaces original key %s (%s)." % (key, val, ekey, newval))
         alterdata.renameVar(key, ekey)
Signature · 1 line
1.7.10.4
#1 lore raw
Quoted 9 lines · Paul Eggleton

BitBake commit 7c568132c54a21161de28907159f902462f1e2bb resulted in a fairly serious performance regression during parsing, almost doubling the time taken to do a full parse and almost certainly impacting performance during building. The expandKeys function is called frequently, and if we avoid using keys() and instead just use the normal variable lookup mechanism, performance is restored.

I would suggest using the existing __contains__ mechanism provided by DataSmart.

if newval in alterdata:

Should be sufficient, as MutableMapping from collections provides __contains__ from Mapping, which in turn uses __getitem__.

Signature · 1 line
Christopher Larson