Skip to contentSkip to conversation

Developer workflow related enhancements / fixes

PATCH In review Paul Eggleton 2 participants 8 messages Dec 8, 2014

lore
Share & tools
lore

Paul Eggleton

author
#0 lore raw

cover Developer workflow related enhancements / fixes

The following series is the set of BitBake changes that are part of the developer workflow improvements currently being worked on [1].

Of particular note is the final patch for resetting the class handlers object - this was discovered incidentally and fixes a rather nasty bug. It should probably be considered separately from the rest of the series.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=6662

The following changes since commit 94d9590a4310f96396e8e782bcf65918f4dcdb36:

bitbake: Update to 1.25.0 as development version after release (2014-12-03 12:14:43 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib paule/devtool-bb
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/devtool-bb

Paul Eggleton (4):
utils: add exec_flat_python_func()
fetch2: add means of disabling SRC_URI checksums
event: add a means of filtering events internally
event: fix resetting class handlers object

 lib/bb/event.py           |  9 +++++++++
 lib/bb/fetch2/__init__.py |  3 ++-
 lib/bb/utils.py           | 22 ++++++++++++++++++++++
 3 files changed, 33 insertions(+), 1 deletion(-)
Signature · 1 line
1.9.3
author
#1 lore raw

PATCH 1/4 utils: add exec_flat_python_func()

Add a function that allows executing a flat python function (defined with def funcname(args): ...).

Signed-off-by Paul Eggleton <[email protected]>
---
 lib/bb/utils.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
lib/bb/utils.py +22 -0
index c6f4071..d7c5067 100644
@@ -893,3 +893,25 @@ def multiprocessingpool(*args, **kwargs):
 
     return multiprocessing.Pool(*args, **kwargs)
 
+def exec_flat_python_func(func, *args, **kwargs):
+    """Execute a flat python function (defined with def funcname(args):...)"""
+    # Prepare a small piece of python code which calls the requested function
+    # To do this we need to prepare two things - a set of variables we can use to pass
+    # the values of arguments into the calling function, and the list of arguments for
+    # the function being called
+    context = {}
+    funcargs = []
+    # Handle unnamed arguments
+    aidx = 1
+    for arg in args:
+        argname = 'arg_%s' % aidx
+        context[argname] = arg
+        funcargs.append(argname)
+        aidx += 1
+    # Handle keyword arguments
+    context.update(kwargs)
+    funcargs.extend(['%s=%s' % (arg, arg) for arg in kwargs.iterkeys()])
+    code = 'retval = %s(%s)' % (func, ', '.join(funcargs))
+    comp = bb.utils.better_compile(code, '<string>', '<string>')
+    bb.utils.better_exec(comp, context, code, '<string>')
+    return context['retval']
Signature · 1 line
1.9.3
author
#2 lore raw

PATCH 2/4 fetch2: add means of disabling SRC_URI checksums

In reply to Paul Eggleton

If we're fetching outside of the context of a recipe, it's handy to be able to disable checksum functionality so you don't get a meaningless warning about the signatures being missing.

Signed-off-by Paul Eggleton <[email protected]>
---
 lib/bb/fetch2/__init__.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
lib/bb/fetch2/__init__.py +2 -1
index 3696e24..5b26524 100644
@@ -535,7 +535,7 @@ def verify_checksum(ud, d):
 
     """
 
-    if not ud.method.supports_checksum(ud):
+    if ud.ignore_checksums or not ud.method.supports_checksum(ud):
         return
 
     md5data = bb.utils.md5_file(ud.localpath)

@@ -1041,6 +1041,7 @@ class FetchData(object):
             self.sha256_expected = None
         else:
             self.sha256_expected = d.getVarFlag("SRC_URI", self.sha256_name)
+        self.ignore_checksums = False
 
         self.names = self.parm.get("name",'default').split(',')
Signature · 1 line
1.9.3
author
#3 lore raw

PATCH 3/4 event: add a means of filtering events internally

In reply to Paul Eggleton

When using external tinfoil-based utilities, it is useful to be able to turn off most of the event handlers; for example sstate_eventhandler doesn't like being sent events for any recipe which has been skipped.

Signed-off-by Paul Eggleton <[email protected]>
---
 lib/bb/event.py | 8 ++++++++
 1 file changed, 8 insertions(+)
lib/bb/event.py +8 -0
index 32df779..9645476 100644
@@ -67,6 +67,7 @@ _ui_logfilters = {}
 _ui_handler_seq = 0
 _event_handler_map = {}
 _catchall_handlers = {}
+_eventfilter = None
 
 def execute_handler(name, handler, event, d):
     event.data = d

@@ -94,6 +95,9 @@ def fire_class_handlers(event, d):
     evt_hmap = _event_handler_map.get(eid, {})
     for name, handler in _handlers.iteritems():
         if name in _catchall_handlers or name in evt_hmap:
+            if _eventfilter:
+                if not _eventfilter(name, handler, event, d):
+                    continue
             execute_handler(name, handler, event, d)
 
 ui_queue = []

@@ -204,6 +208,10 @@ def remove(name, handler):
     """Remove an Event handler"""
     _handlers.pop(name)
 
+def set_eventfilter(func):
+    global _eventfilter
+    _eventfilter = func
+
 def register_UIHhandler(handler):
     bb.event._ui_handler_seq = bb.event._ui_handler_seq + 1
     _ui_handlers[_ui_handler_seq] = handler
Signature · 1 line
1.9.3
author
#4 lore raw

PATCH 4/4 event: fix resetting class handlers object

In reply to Paul Eggleton

If you don't explicitly specify to use a global variable when doing an assignment, you will be setting a local variable instead, which means this function wasn't working at all. It explains some odd behaviour we have seen in the layer index where event handlers were sometimes bleeding into other contexts where they should not have been.

Signed-off-by Paul Eggleton <[email protected]>
---
 lib/bb/event.py | 1 +
 1 file changed, 1 insertion(+)
lib/bb/event.py +1 -0
index 9645476..8bc6b80 100644
@@ -55,6 +55,7 @@ def get_class_handlers():
     return _handlers
 
 def set_class_handlers(h):
+    global handlers
     _handlers = h
 
 def clean_class_handlers():
Signature · 1 line
1.9.3
#5 lore raw

Re: [PATCH 4/4] event: fix resetting class handlers object

Quoted 16 lines · Paul Eggleton

If you don't explicitly specify to use a global variable when doing an assignment, you will be setting a local variable instead, which means this function wasn't working at all. It explains some odd behaviour we have seen in the layer index where event handlers were sometimes bleeding into other contexts where they should not have been.

Signed-off-by Paul Eggleton <[email protected]>
---
 lib/bb/event.py | 1 +
 1 file changed, 1 insertion(+)
lib/bb/event.py +0 -0
index 9645476..8bc6b80 100644

lib/bb/event.py def get_class_handlers(): · line 59

Quoted 5 lines
lib/bb/event.py +1 -0
@@ -55,6 +55,7 @@ def get_class_handlers():
     return _handlers
 
 def set_class_handlers(h):
+    global handlers
     _handlers = h

Er, shouldn't this be _handlers?

Cheers,

Richard

Quoted 4 lines
lib/bb/event.py +0 -0
 def clean_class_handlers():
Signature · 1 line
1.9.3
author
#6 lore raw

Re: [PATCH 4/4] event: fix resetting class handlers object

Quoted 21 lines · On Monday 08 December 2014 17:42:19 Richard Purdie wrote:
Quoted 20 lines · Paul Eggleton

If you don't explicitly specify to use a global variable when doing an assignment, you will be setting a local variable instead, which means this function wasn't working at all. It explains some odd behaviour we have seen in the layer index where event handlers were sometimes bleeding into other contexts where they should not have been.

Signed-off-by Paul Eggleton <[email protected]>

---

 lib/bb/event.py | 1 +
 1 file changed, 1 insertion(+)
lib/bb/event.py +0 -0
index 9645476..8bc6b80 100644
@@ -55,6 +55,7 @@ def get_class_handlers():
     return _handlers
Quoted 6 lines

lib/bb/event.py def get_class_handlers():

Quoted 4 lines
lib/bb/event.py +1 -0
 def set_class_handlers(h):
+    global handlers
 
     _handlers = h

Er, shouldn't this be _handlers?

Oh dear. Yes, it should. I've fixed that on the branch, but I'll do a bit more testing just to make sure it isn't breaking anything since the previous change wouldn't have made any impact at all.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#7 lore raw

Re: [PATCH 4/4] event: fix resetting class handlers object

In reply to Richard Purdie
Quoted 19 lines · Richard Purdie
Quoted 18 lines · Paul Eggleton

If you don't explicitly specify to use a global variable when doing an assignment, you will be setting a local variable instead, which means this function wasn't working at all. It explains some odd behaviour we have seen in the layer index where event handlers were sometimes bleeding into other contexts where they should not have been.

Signed-off-by Paul Eggleton <[email protected]>
---
 lib/bb/event.py | 1 +
 1 file changed, 1 insertion(+)
lib/bb/event.py +0 -0
index 9645476..8bc6b80 100644
@@ -55,6 +55,7 @@ def get_class_handlers():
     return _handlers
Quoted 5 lines

lib/bb/event.py def get_class_handlers(): · line 59

Quoted 3 lines
lib/bb/event.py +1 -0
 def set_class_handlers(h):
+    global handlers
     _handlers = h

Er, shouldn't this be _handlers?

I put the patch in master-next and tweaked this.

Cheers,

Richard