Skip to contentSkip to conversation

(morty) Python exception during rootfs

PATCH In review Mike Looijmans 4 participants 6 messages Feb 7, 2017 – Feb 8, 2017

lore
Share & tools
lore
#0 lore raw

I get this stacktrace from bitbake. No idea what's wrong. Apparently, d.getVarFlags(func) returns None and the code cannot handle that. Is this a bug in bitbake, or should 'flags' never evaluate to None?

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>

     0001:
 *** 0002:do_rootfs(d)
     0003:
File: '/home/mike/projects/zynq-platform/oe-core/meta/classes/image.bbclass', lineno: 249, function: do_rootfs
     0245:
     0246:    progress_reporter.next_stage()
     0247:
     0248:    # generate rootfs
 *** 0249:    create_rootfs(d, progress_reporter=progress_reporter)
     0250:
     0251:    progress_reporter.finish()
     0252:}
     0253:do_rootfs[dirs] = "${TOPDIR}"
File: '/home/mike/projects/zynq-platform/oe-core/meta/lib/oe/rootfs.py', lineno: 1006, function: create_rootfs
     1002:    img_type = d.getVar('IMAGE_PKGTYPE', True)
     1003:    if img_type == "rpm":
     1004:        RpmRootfs(d, manifest_dir, progress_reporter).create()
     1005:    elif img_type == "ipk":
 *** 1006:        OpkgRootfs(d, manifest_dir, progress_reporter).create()
     1007:    elif img_type == "deb":
     1008:        DpkgRootfs(d, manifest_dir, progress_reporter).create()
     1009:
     1010:    os.environ.clear()
File: '/home/mike/projects/zynq-platform/oe-core/meta/lib/oe/rootfs.py', lineno: 210, function: create
     0206:        execute_pre_post_process(self.d, rootfs_post_install_cmds)
     0207:
     0208:        self._run_intercepts()
     0209:
 *** 0210:        execute_pre_post_process(self.d, post_process_cmds)
     0211:
     0212:        if self.progress_reporter:
     0213:            self.progress_reporter.next_stage()
     0214:
File: '/home/mike/projects/zynq-platform/oe-core/meta/lib/oe/utils.py', lineno: 176, function: execute_pre_post_process
     0172:    for cmd in cmds.strip().split(';'):
     0173:        cmd = cmd.strip()
     0174:        if cmd != '':
     0175:            bb.note("Executing %s ..." % cmd)

*** 0176: bb.build.exec_func(cmd, d)
0177:
0178:def multiprocess_exec(commands, function):
0179: import signal
0180: import multiprocessing
File: '/home/mike/projects/zynq-platform/bitbake/lib/bb/build.py', lineno: 197, function: exec_func
0193: except:
0194: oldcwd = None
0195:
0196: flags = d.getVarFlags(func)
*** 0197: cleandirs = flags.get('cleandirs')
0198: if cleandirs:
0199: for cdir in d.expand(cleandirs).split():
0200: bb.utils.remove(cdir, True)
0201: bb.utils.mkdirhier(cdir)
Exception: AttributeError: 'NoneType' object has no attribute 'get'

ERROR: pr-demo-image-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in: /home/mike/projects/zynq-platform/build/tmp-glibc/work/topic_miami_florida_gen-oe-linux-gnueabi/pr-demo-image/1.0-r0/temp/log.do_rootfs.28716
ERROR: Task (/home/mike/projects/zynq-platform/meta-topic-internal/recipes-topic/embedded-world/pr-demo-image.bb:do_rootfs) failed with exit code '1'

Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: [email protected]
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

#1 lore raw

Found it. A rootfs post-process function was misspelled, and that caused it.

I'll make a patch to improve the error message here, to make it just say that it cannot find "func".

Quoted 85 lines · Mike Looijmans

I get this stacktrace from bitbake. No idea what's wrong. Apparently, d.getVarFlags(func) returns None and the code cannot handle that. Is this a bug in bitbake, or should 'flags' never evaluate to None?

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>

     0001:
 *** 0002:do_rootfs(d)
     0003:
File: '/home/mike/projects/zynq-platform/oe-core/meta/classes/image.bbclass',
lineno: 249, function: do_rootfs
     0245:
     0246:    progress_reporter.next_stage()
     0247:
     0248:    # generate rootfs
 *** 0249:    create_rootfs(d, progress_reporter=progress_reporter)
     0250:
     0251:    progress_reporter.finish()
     0252:}
     0253:do_rootfs[dirs] = "${TOPDIR}"
File: '/home/mike/projects/zynq-platform/oe-core/meta/lib/oe/rootfs.py',
lineno: 1006, function: create_rootfs
     1002:    img_type = d.getVar('IMAGE_PKGTYPE', True)
     1003:    if img_type == "rpm":
     1004:        RpmRootfs(d, manifest_dir, progress_reporter).create()
     1005:    elif img_type == "ipk":
 *** 1006:        OpkgRootfs(d, manifest_dir, progress_reporter).create()
     1007:    elif img_type == "deb":
     1008:        DpkgRootfs(d, manifest_dir, progress_reporter).create()
     1009:
     1010:    os.environ.clear()
File: '/home/mike/projects/zynq-platform/oe-core/meta/lib/oe/rootfs.py',
lineno: 210, function: create
     0206:        execute_pre_post_process(self.d, rootfs_post_install_cmds)
     0207:
     0208:        self._run_intercepts()
     0209:
 *** 0210:        execute_pre_post_process(self.d, post_process_cmds)
     0211:
     0212:        if self.progress_reporter:
     0213:            self.progress_reporter.next_stage()
     0214:
File: '/home/mike/projects/zynq-platform/oe-core/meta/lib/oe/utils.py',
lineno: 176, function: execute_pre_post_process
     0172:    for cmd in cmds.strip().split(';'):
     0173:        cmd = cmd.strip()
     0174:        if cmd != '':
     0175:            bb.note("Executing %s ..." % cmd)

*** 0176: bb.build.exec_func(cmd, d)
0177:
0178:def multiprocess_exec(commands, function):
0179: import signal
0180: import multiprocessing
File: '/home/mike/projects/zynq-platform/bitbake/lib/bb/build.py', lineno:
197, function: exec_func
0193: except:
0194: oldcwd = None
0195:
0196: flags = d.getVarFlags(func)
*** 0197: cleandirs = flags.get('cleandirs')
0198: if cleandirs:
0199: for cdir in d.expand(cleandirs).split():
0200: bb.utils.remove(cdir, True)
0201: bb.utils.mkdirhier(cdir)
Exception: AttributeError: 'NoneType' object has no attribute 'get'

ERROR: pr-demo-image-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in:
/home/mike/projects/zynq-platform/build/tmp-glibc/work/topic_miami_florida_gen-oe-linux-gnueabi/pr-demo-image/1.0-r0/temp/log.do_rootfs.28716

ERROR: Task (/home/mike/projects/zynq-platform/meta-topic-internal/recipes-topic/embedded-world/pr-demo-image.bb:do_rootfs) failed with exit code '1'

Kind regards,

Mike Looijmans
System Expert

Quoted 13 lines

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: [email protected]
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: [email protected]
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

#2 lore raw
Quoted 4 lines · Mike Looijmans

Found it. A rootfs post-process function was misspelled, and that caused it.

I'll make a patch to improve the error message here, to make it just say that it cannot find "func".

I already fixed that in bitbake master, see commit 25df3db5 "build.py:
avoid exception when function is not defined". Sorry, only now saw your original email.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#3 lore raw

PATCH 1/1 lib/bb/build.py: Raise proper KeyError if function does not exist

In reply to Mike Looijmans

If one adds the following line to an image recipe without defining the function:
ROOTFS_POSTPROCESS_COMMAND += "rootfs_funky_stuff;" bitbake will terminate with a cryptic error message:
AttributeError: 'NoneType' object has no attribute 'get'

To inform the user of what bitbake wasn't able to find, raise a KeyError exception denoting the function name before that happens. This changes the message to the much more useful:
KeyError: 'rootfs_funky_stuff'

---
 lib/bb/build.py | 2 ++
 1 file changed, 2 insertions(+)
lib/bb/build.py +2 -0
index c4c8aeb..5bc18c1 100644
@@ -194,6 +194,8 @@ def exec_func(func, d, dirs = None, pythonexception=False):
         oldcwd = None
 
     flags = d.getVarFlags(func)
+    if flags is None:
+        raise KeyError(func)
     cleandirs = flags.get('cleandirs')
     if cleandirs:
         for cdir in d.expand(cleandirs).split():
Signature · 1 line
1.9.1
#4 lore raw

Re: [bitbake][PATCH] lib/bb/build.py: Raise proper KeyError if function does not exist

Hi Mike,

Quoted 24 lines · NZDT Mike Looijmans

If one adds the following line to an image recipe without defining the function:
ROOTFS_POSTPROCESS_COMMAND += "rootfs_funky_stuff;" bitbake will terminate with a cryptic error message:
AttributeError: 'NoneType' object has no attribute 'get'

To inform the user of what bitbake wasn't able to find, raise a KeyError exception denoting the function name before that happens. This changes the message to the much more useful:
KeyError: 'rootfs_funky_stuff'

---
 lib/bb/build.py | 2 ++
 1 file changed, 2 insertions(+)
lib/bb/build.py +0 -0
index c4c8aeb..5bc18c1 100644
@@ -194,6 +194,8 @@ def exec_func(func, d, dirs = None,

pythonexception=False): oldcwd = None

flags = d.getVarFlags(func)

lib/bb/build.py def exec_func(func, d, dirs = None, pythonexception=False): · line 201

Quoted 5 lines
lib/bb/build.py +2 -0
+    if flags is None:
+        raise KeyError(func)
     cleandirs = flags.get('cleandirs')
     if cleandirs:
         for cdir in d.expand(cleandirs).split():

This should got to the bitbake-devel list.

Whilst I agree this is an improvement, since we're trying to print a more reasonable error message, wouldn't we be better off explicitly saying "Function %s not defined" or similar?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#5 lore raw

✗ patchtest: failure for lib/bb/build.py: Raise proper KeyError if function does not exist

In reply to Mike Looijmans

== Series Details ==

Series: lib/bb/build.py: Raise proper KeyError if function does not exist
Revision: 1
URL : https://patchwork.openembedded.org/series/5202/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at 68da6c9884)

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe