Skip to contentSkip to conversation

package.py: log rpmdeps call

PATCH In review Martin Jansa 2 participants 4 messages Feb 18, 2014 – Feb 23, 2014

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package.py: log rpmdeps call

* I've noticed errors like this in log.do_package:

DEBUG: Executing python function package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python function package_do_filedeps finished

which are actually caused by some filenames included in package containing '()' characters

Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))

---
 meta/lib/oe/package.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/lib/oe/package.py +3 -1
index 9a0ddb8..1f78a8d 100644
@@ -89,7 +89,9 @@ def filedeprunner(arg):
 
         return provides, requires
 
-    dep_pipe = os.popen(rpmdeps + " " + " ".join(pkgfiles))
+    rpmdepscmd = rpmdeps + " " + " ".join(pkgfiles)
+    bb.debug(1, "runrpmdeps: %s" % rpmdepscmd)
+    dep_pipe = os.popen(rpmdepscmd)
 
     provides, requires = process_deps(dep_pipe, pkg, pkgdest, provides, requires)
 
Signature · 1 line
1.8.5.3
#1 lore raw
Quoted 6 lines · Martin Jansa

Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))

We'd likely be better off just avoiding os.popen entirely in favor of subprocess based operations, which generally support specifying the command arguments as a list, and can bypass the shell entirely, avoiding the need to escape anything. Alternatively, use os.popen2(), which can accept a list to bypass the shell as well.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#2 lore raw

RFC PATCH 1/1 package.py: use subprocess.Popen for rpmdeps call

* I've noticed errors like this in log.do_package:

DEBUG: Executing python function package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python function package_do_filedeps finished

which are actually caused by some filenames included in package containing '()' characters

Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))

* don't use shell=True
* show the command when it fails and let do_package task to fail

---
 meta/lib/oe/package.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
meta/lib/oe/package.py +7 -4
index 9a0ddb8..f8b5322 100644
@@ -54,7 +54,7 @@ def file_translate(file):
     return ft
 
 def filedeprunner(arg):
-    import re
+    import re, subprocess, shlex
 
     (pkg, pkgfiles, rpmdeps, pkgdest) = arg
     provides = {}
@@ -89,8 +89,11 @@ def filedeprunner(arg):
 
         return provides, requires
 
-    dep_pipe = os.popen(rpmdeps + " " + " ".join(pkgfiles))
-
-    provides, requires = process_deps(dep_pipe, pkg, pkgdest, provides, requires)
+    try:
+        dep_popen = subprocess.Popen(shlex.split(rpmdeps) + pkgfiles, stdout=subprocess.PIPE)
+        provides, requires = process_deps(dep_popen.stdout, pkg, pkgdest, provides, requires)
+    except OSError as e:
+        bb.error("rpmdeps: '%s' command failed, '%s'" % (shlex.split(rpmdeps) + pkgfiles, e))
+        raise e
 
     return (pkg, provides, requires)
Signature · 1 line
1.9.0
#3 lore raw
In reply to Martin Jansa
Quoted 7 lines · Martin Jansa

* I've noticed errors like this in log.do_package:

DEBUG: Executing python function package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python function package_do_filedeps finished

Quoted 8 lines

which are actually caused by some filenames included in package containing '()' characters

Maybe we should change meta/classes/package.bbclass to fail when some filedeprunner call fails like this and fix filedeprunner to escape '()' and other possibly dangerous chars it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))

Please don't merge this one yet, It's possible that it's causing
ValueError: insecure string pickle:
http://lists.openembedded.org/pipermail/openembedded-core/2014-February/089805.html

Quoted 24 lines
---
 meta/lib/oe/package.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/lib/oe/package.py +3 -1
index 9a0ddb8..1f78a8d 100644
@@ -89,7 +89,9 @@ def filedeprunner(arg):
 
         return provides, requires
 
-    dep_pipe = os.popen(rpmdeps + " " + " ".join(pkgfiles))
+    rpmdepscmd = rpmdeps + " " + " ".join(pkgfiles)
+    bb.debug(1, "runrpmdeps: %s" % rpmdepscmd)
+    dep_pipe = os.popen(rpmdepscmd)
 
     provides, requires = process_deps(dep_pipe, pkg, pkgdest, provides, requires)
 
Signature · 1 line
1.8.5.3
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]