package.py: log rpmdeps call
PATCH 1/1 package.py: log rpmdeps call
* I've noticed errors like this in log.do_package:
DEBUG: Executing python function package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python function package_do_filedeps finished
which are actually caused by some filenames included in package containing '()' characters
Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))
--- meta/lib/oe/package.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/lib/oe/package.py +3 -1
@@ -89,7 +89,9 @@ def filedeprunner(arg): return provides, requires - dep_pipe = os.popen(rpmdeps + " " + " ".join(pkgfiles)) + rpmdepscmd = rpmdeps + " " + " ".join(pkgfiles) + bb.debug(1, "runrpmdeps: %s" % rpmdepscmd) + dep_pipe = os.popen(rpmdepscmd) provides, requires = process_deps(dep_pipe, pkg, pkgdest, provides, requires)
Signature · 1 line
1.8.5.3
Quoted 6 lines · Martin Jansa
Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed =list(pool.imap(oe.package.filedeprunner, pkglist))
We'd likely be better off just avoiding os.popen entirely in favor of subprocess based operations, which generally support specifying the command arguments as a list, and can bypass the shell entirely, avoiding the need to escape anything. Alternatively, use os.popen2(), which can accept a list to bypass the shell as well.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
RFC PATCH 1/1 package.py: use subprocess.Popen for rpmdeps call
* I've noticed errors like this in log.do_package:
DEBUG: Executing python function package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python function package_do_filedeps finished
which are actually caused by some filenames included in package containing '()' characters
Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))
* don't use shell=True
* show the command when it fails and let do_package task to fail
--- meta/lib/oe/package.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
meta/lib/oe/package.py +7 -4
@@ -54,7 +54,7 @@ def file_translate(file): return ft def filedeprunner(arg): - import re + import re, subprocess, shlex (pkg, pkgfiles, rpmdeps, pkgdest) = arg provides = {}
@@ -89,8 +89,11 @@ def filedeprunner(arg): return provides, requires - dep_pipe = os.popen(rpmdeps + " " + " ".join(pkgfiles)) - - provides, requires = process_deps(dep_pipe, pkg, pkgdest, provides, requires) + try: + dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE) + provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, provides, requires) + except OSError as e: + bb.error("rpmdeps: '%s' command failed, '%s'" % (shlex.split(rpmdeps) + pkgfiles, e)) + raise e return (pkg, provides, requires)
Signature · 1 line
1.9.0
Quoted 7 lines · Martin Jansa
* I've noticed errors like this in log.do_package:
DEBUG: Executing python function
package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python functionpackage_do_filedepsfinished
Quoted 8 lines
which are actually caused by some filenames included in package containing '()' characters
Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed =list(pool.imap(oe.package.filedeprunner, pkglist))
Please don't merge this one yet, It's possible that it's causing
ValueError: insecure string pickle:
http://lists.openembedded.org/pipermail/openembedded-core/2014-February/089805.html
Quoted 24 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/lib/oe/package.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)meta/lib/oe/package.py +3
-1@@ -89,7 +89,9 @@ def filedeprunner(arg): return provides, requires - dep_pipe = os.popen(rpmdeps + " " + " ".join(pkgfiles)) + rpmdepscmd = rpmdeps + " " + " ".join(pkgfiles) + bb.debug(1, "runrpmdeps: %s" % rpmdepscmd) + dep_pipe = os.popen(rpmdepscmd) provides, requires = process_deps(dep_pipe, pkg, pkgdest, provides, requires)Signature · 1 line
1.8.5.3
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]