Skip to contentSkip to conversation

sanity: Make the required utilities more platform specific.

PATCH In review Martin Ertsaas 3 participants 10 messages Jan 10, 2013 – Jan 15, 2013

lore
Share & tools
lore
#0 lore raw

PATCH 1/5 sanity: Make the required utilities more platform specific.

This might make us able to build on mac os X.

---
 meta/classes/sanity.bbclass |   10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
meta/classes/sanity.bbclass +8 -2
index 0ffa52d..03651be 100644
@@ -2,7 +2,9 @@
 # Sanity check the users setup for common misconfigurations
 #
 
-SANITY_REQUIRED_UTILITIES ?= "patch diffstat makeinfo git bzip2 tar gzip gawk chrpath wget cpio"
+SANITY_REQUIRED_UTILITIES ?= "patch diffstat makeinfo git bzip2 tar gzip gawk wget cpio"
+SANITY_REQUIRED_UTILITIES_Linux ?= "${SANITY_REQUIRED_UTILITIES} chrpath"
+SANITY_REQUIRED_UTILITIES_Darwin ?= "${SANITY_REQUIRED_UTILITIES} install_name_tool"
 
 python check_bblayers_conf() {
     bblayers_fn = os.path.join(d.getVar('TOPDIR', True), 'conf/bblayers.conf')
@@ -340,6 +342,10 @@ def check_sanity_validmachine(sanity_data):
 
     return messages
 
+def get_required_utilities(sanity_data):
+    import platform
+    sanity_var = 'SANITY_REQUIRED_UTILITIES_%s' %platform.system()
+    return sanity_data.getVar(sanity_var, True)
 
 def check_sanity(sanity_data):
     import subprocess
@@ -444,7 +450,7 @@ def check_sanity(sanity_data):
     if not check_app_exists('${BUILD_PREFIX}g++', sanity_data):
         missing = missing + "C++ Compiler (%sg++)," % sanity_data.getVar("BUILD_PREFIX", True)
 
-    required_utilities = sanity_data.getVar('SANITY_REQUIRED_UTILITIES', True)
+    required_utilities = get_required_utilities(sanity_data)
 
     if "qemu-native" in assume_provided:
         if not check_app_exists("qemu-arm", sanity_data):
Signature · 1 line
1.7.10.2 (Apple Git-33)
#1 lore raw

PATCH 2/5 quilt: Remove non-gnu.patch, and added configure flags for darwin.

---
 meta/recipes-devtools/quilt/quilt-0.60.inc      |    4 +
 meta/recipes-devtools/quilt/quilt-native.inc    |    1 -
 meta/recipes-devtools/quilt/quilt/non-gnu.patch |  225 -----------------------
 3 files changed, 4 insertions(+), 226 deletions(-)
 delete mode 100644 meta/recipes-devtools/quilt/quilt/non-gnu.patch
meta/recipes-devtools/quilt/quilt-0.60.inc +4 -0
index 1e240a0..e0a2e51 100644
@@ -12,6 +12,10 @@ SRC_URI[sha256sum] = "3d72a292e432beb9a73f9d0acfe3a77c9b4d7e42209919bb244e9958c7
 
 inherit autotools
 
+EXTRA_OECONF_darwin += "--without-date \
+                        --without-getopt \
+                        "
+
 PACKAGES += "guards guards-doc"
 FILES_${PN} = "${sysconfdir} ${datadir}/quilt \
                ${bindir}/quilt ${libdir}/quilt"
meta/recipes-devtools/quilt/quilt-native.inc +0 -1
index 9345d88..5c4b0a2 100644
@@ -1,4 +1,3 @@
-SRC_URI_append_build-darwin = "?   file://non-gnu.patch "
 RDEPENDS_${PN} = "diffstat-native patch-native bzip2-native util-linux-native"
 
 INHIBIT_AUTOTOOLS_DEPS = "1"
meta/recipes-devtools/quilt/quilt/non-gnu.patch deleted +0 -225

225 lines not on this page · raw message

Signature · 1 line
1.7.10.2 (Apple Git-33)
#2 lore raw

PATCH 3/5 quilt: Don't use BUILD_ROOT on darwin.

In reply to Martin Ertsaas
---
 meta/recipes-devtools/quilt/quilt-0.60.inc |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/quilt/quilt-0.60.inc +4 -1
index e0a2e51..5fe201f 100644
@@ -25,9 +25,12 @@ FILES_guards-doc = "${mandir}/man1/guards.1"
 
 RDEPENDS_${PN} = "bash"
 
+EXTRA_OE_MAKE_ARGS_darwin ?= ""
+EXTRA_OE_MAKE_ARGS ?= "BUILD_ROOT=${D}"
+
 # quilt ignores DESTDIR
 do_install () {
-	oe_runmake 'BUILD_ROOT=${D}' install
+	oe_runmake ${EXTRA_OE_MAKE_ARGS} install
 	# cleanup unpackaged files
 	rm -rf ${D}/${datadir}/emacs
 }
Signature · 1 line
1.7.10.2 (Apple Git-33)
#3 lore raw

PATCH 4/5 lib/oe/path.py: Use shutil.rmtree if the path we wish to remove is a directory.

In reply to Martin Ertsaas

On mac, os.unlink can not be done to remove directories, and so we have to explicitly use shutil.rmtree instead to support mac.

---
 meta/lib/oe/path.py |    9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
meta/lib/oe/path.py +1 -8
index 7197b23..1da39b7 100644
@@ -86,14 +86,7 @@ def copytree(src, dst):
 
 def remove(path, recurse=True):
     """Equivalent to rm -f or rm -rf"""
-    for name in glob.glob(path):
-        try:
-            os.unlink(name)
-        except OSError, exc:
-            if recurse and exc.errno == errno.EISDIR:
-                shutil.rmtree(name)
-            elif exc.errno != errno.ENOENT:
-                raise
+    bb.utils.remove(path, recurse)
 
 def symlink(source, destination, force=False):
     """Create a symbolic link"""
Signature · 1 line
1.7.10.2 (Apple Git-33)
#4 lore raw

Re: [PATCH 4/5] lib/oe/path.py: Use shutil.rmtree if the path we wish to remove is a directory.

Quoted 3 lines · Martin Ertsaas

On mac, os.unlink can not be done to remove directories, and so we have to explicitly use shutil.rmtree instead to support mac.

As pointed out for the bitbake patch, the functions can already handle this case if you pass recurse=True, and mac isn't at all special in this context.

Signature · 1 line
Christopher Larson
#5 lore raw

Re: [PATCH 4/5] lib/oe/path.py: Use shutil.rmtree if the path we wish to remove is a directory.

Quoted 4 lines · Chris Larson

On Thu, Jan 10, 2013 at 1:50 AM, Martin Ertsaas <[email protected] <mailto:[email protected]>> wrote:

Quoted 10 lines

On mac, os.unlink can not be done to remove directories, and so we have to
explicitly use shutil.rmtree instead to support mac.

As pointed out for the bitbake patch, the functions can already handle this case if you pass recurse=True, and mac isn't at all special in this context.

Signature · 1 line
Christopher Larson

True. In this case though, I feel the patch is still valid though, and that I should instead change the commit message. What I want from this patch, as I see it again now, is to not duplicate the same code in both bitbake and oe-core, when the oe-core function 'remove' can just be calling bb.utils.remove. So personally I still want this patch, I just wish to change the commit message to something sane. Agreed?

- Martin

#6 lore raw

Re: [PATCH 4/5] lib/oe/path.py: Use shutil.rmtree if the path we wish to remove is a directory.

Quoted 15 lines · Martin Ertsaas

On 01/10/13 15:15, Chris Larson wrote:

Quoted 3 lines · Martin Ertsaas

On mac, os.unlink can not be done to remove directories, and so we have to explicitly use shutil.rmtree instead to support mac.

As pointed out for the bitbake patch, the functions can already handle
this case if you pass recurse=True, and mac isn't at all special in this context.
--
Christopher Larson

Quoted 7 lines

True. In this case though, I feel the patch is still valid though, and that I should instead change the commit message. What I want from this patch, as I see it again now, is to not duplicate the same code in both bitbake and oe-core, when the oe-core function 'remove' can just be calling bb.utils.remove. So personally I still want this patch, I just wish to change the commit message to something sane. Agreed?

Agreed.

Signature · 1 line
Christopher Larson
#7 lore raw

PATCH 5/5 sstate: Do not add the --no-run-if-empty arguement to xargs when on Darwin, as it is not supported.

In reply to Martin Ertsaas
---
 meta/classes/sstate.bbclass |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
meta/classes/sstate.bbclass +7 -3
index 68fd996..e92fbae 100644
@@ -368,7 +368,7 @@ python sstate_cleanall() {
 }
 
 def sstate_hardcode_path(d):
-    import subprocess
+    import subprocess, platform
 
     # Need to remove hardcoded paths and fix these when we install the
     # staging packages.
@@ -390,7 +390,7 @@ def sstate_hardcode_path(d):
     else:
         sstate_grep_cmd = "grep -l -e '%s'" % (staging_host)
         sstate_sed_cmd = "sed -i -e 's:%s:FIXMESTAGINGDIRHOST:g'" % (staging_host)
-    
+
     fixmefn =  sstate_builddir + "fixmepath"
 
     sstate_scan_cmd = d.getVar('SSTATE_SCAN_CMD', True)
@@ -399,9 +399,13 @@ def sstate_hardcode_path(d):
     # fixmepath file needs relative paths, drop sstate_builddir prefix
     sstate_filelist_relative_cmd = "sed -i -e 's:^%s::g' %s" % (sstate_builddir, fixmefn)
 
+    xargs_no_empty_run_cmd = '--no-run-if-empty'
+    if platform.system() == 'Darwin':
+        xargs_no_empty_run_cmd = ''
+
     # Limit the fixpaths and sed operations based on the initial grep search
     # This has the side effect of making sure the vfs cache is hot
-    sstate_hardcode_cmd = "%s | xargs %s | %s | xargs --no-run-if-empty %s" % (sstate_scan_cmd, sstate_grep_cmd, sstate_filelist_cmd, sstate_sed_cmd)
+    sstate_hardcode_cmd = "%s | xargs %s | %s | xargs %s %s" % (sstate_scan_cmd, sstate_grep_cmd, sstate_filelist_cmd, xargs_no_empty_run_cmd, sstate_sed_cmd)
 
     print "Removing hardcoded paths from sstate package: '%s'" % (sstate_hardcode_cmd)
     subprocess.call(sstate_hardcode_cmd, shell=True)
Signature · 1 line
1.7.10.2 (Apple Git-33)
#8 lore raw
In reply to Martin Ertsaas
Quoted 12 lines · Martin Ertsaas

This might make us able to build on mac os X.

---
 meta/classes/sanity.bbclass |   10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
meta/classes/sanity.bbclass +0 -0
index 0ffa52d..03651be 100644

meta/classes/sanity.bbclass line 7

Quoted 7 lines
meta/classes/sanity.bbclass +3 -1
@@ -2,7 +2,9 @@
 # Sanity check the users setup for common misconfigurations
 #
 
-SANITY_REQUIRED_UTILITIES ?= "patch diffstat makeinfo git bzip2 tar gzip gawk chrpath wget cpio"
+SANITY_REQUIRED_UTILITIES ?= "patch diffstat makeinfo git bzip2 tar gzip gawk wget cpio"
+SANITY_REQUIRED_UTILITIES_Linux ?= "${SANITY_REQUIRED_UTILITIES} chrpath"
+SANITY_REQUIRED_UTILITIES_Darwin ?= "${SANITY_REQUIRED_UTILITIES} install_name_tool"

You should be able to make this:

SANITY_REQUIRED_UTILITIES_COMMON ?= "patch diffstat makeinfo git bzip2 tar gzip gawk wget cpio" SANITY_REQUIRED_UTILITIES ?= "${SANITY_REQUIRED_UTILITIES_COMMON} chrpath" SANITY_REQUIRED_UTILITIES_build-darwin ?= "${SANITY_REQUIRED_UTILITIES_COMMON} install_name_tool"

which will then simplify the other part of the patch (make it unnecessary).

Before we go and merge a ton of these type of changes I'd also like to understand how close we are to making this work?

Cheers,

Richard

#9 lore raw
Quoted 2 lines · Richard Purdie

which will then simplify the other part of the patch (make it unnecessary).

That is true, will make it a lot simpler.

What I know is still missing:

1) There are some errors with the paths, seems like stuff is not correctly transfered to the sysroot, this is next on the agenda.
2) Make a class that does what chrpath does, but for install_name_tool, going to do this as soon as I get an error on it.
3) Probably some errors after I get past the error I have now.

I agree that we should not merge a ton of these patches.

For my part, I can keep these in my repository until I have stuff working. When we can compile everything I can do a pull request. If someone wants to help, they can just pull from me and work against that repo.

- Martin