Skip to contentSkip to conversation

Fix support for Icecream

PATCH Superseded Joshua Watt 4 participants 40 messages Feb 6, 2018 – Feb 17, 2018

lore
Share & tools
lore
#0 lore raw

cover Fix support for Icecream

Fix up support for using Icecream to do distributed builds, which appears to have been broken for some time.

In addition, Icecream support can now be enabled in the SDK. When enabled, the SDK install process will check if the host supports icecc and if so will construct a proper environment tarball and use it when compiling.

Most of the changes should be straightforward. icecc-create-env has been almost completely re-written but the changes were broken up into hopefully logical chunks.

The following changes since commit 385944254d0ef88bd6450a221a54cbcb40020b42:

tcmode-default.inc: drop preferred version of gzip-native (2018-01-31 17:01:20 +0000)

are available in the Git repository at:

git://push.yoctoproject.org/poky-contrib jpew/icecream

Joshua Watt (14):
icecc.bbclass: Move to shared work directory icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage icecc.bbclass: Skip canadian-cross compiles
icecc-create-env: Reformat
icecc-create-env: Add /bin/true to environment
icecc-create-env: Use program interpreter for deps
icecc-create-env: Symlink alternate names
icecc-create-env: Archive directory
icecc-create-env: Fix executable rpaths
icecc-create-env: Add support for nativesdk
icecc-create-env: Allow multiple tool aliases
toolchain-shar-extract: Add post-relocate scripts
icecc-toolchain: Add SDK icecream setup icecc.bbclass: Add IceCream support to SDK

 meta/classes/icecc.bbclass                         |  37 ++-
 meta/classes/toolchain-scripts.bbclass             |  15 +
 meta/files/toolchain-shar-extract.sh               |   8 +
 meta/recipes-core/meta/meta-environment.bb         |   2 +
 .../icecc-create-env/icecc-create-env              | 328 ++++++++++++++-------
 ...e-env-native_0.1.bb => icecc-create-env_0.1.bb} |   4 +-
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   |  37 +++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh |  43 +++
 .../nativesdk-icecc-toolchain_0.1.bb               |  38 +++
 9 files changed, 390 insertions(+), 122 deletions(-)
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb} (98%)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb
Signature · 1 line
2.14.3
#1 lore raw

PATCH 1/14 icecc.bbclass: Move to shared work directory

Generate the icecc toolchains in a shared work directory. This class was already setup to correctly synchronize creating the toolchains in a shared location before the RSS changes, so return to that behavior instead of generated the toolchains in each recipe's sysroot. Additionally, it makes no sense for each recipe to generate a toolchain, only to find it has already generated an uploaded to the compile server by another recipe.

---
 meta/classes/icecc.bbclass | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
meta/classes/icecc.bbclass +5 -2
index 1cc1c4ddb0f..e8725b582b3 100644
@@ -151,6 +151,9 @@ def icecc_is_native(bb, d):
         bb.data.inherits_class("cross", d) or \
         bb.data.inherits_class("native", d);
 
+def icecc_dir(bb, d):
+    return d.expand('${TMPDIR}/work-shared/ice')
+
 # Don't pollute allarch signatures with TARGET_FPU
 icecc_version[vardepsexclude] += "TARGET_FPU"
 def icecc_version(bb, d):
@@ -175,8 +178,8 @@ def icecc_version(bb, d):
             archive_name += "-kernel"
 
     import socket
-    ice_dir = d.expand('${STAGING_DIR_NATIVE}${prefix_native}')
-    tar_file = os.path.join(ice_dir, 'ice', archive_name + "-@VERSION@-" + socket.gethostname() + '.tar.gz')
+    ice_dir = icecc_dir(bb, d)
+    tar_file = os.path.join(ice_dir, archive_name + "-@VERSION@-" + socket.gethostname() + '.tar.gz')
 
     return tar_file
 
Signature · 1 line
2.14.3
#2 lore raw

PATCH 2/14 icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage

In reply to Joshua Watt

STAGING_BINDIR_TOOLCHAIN is actually a path list, not a single path. Fix icecc.bbclass to try all the paths in the variable instead of treating it as a single path.

---
 meta/classes/icecc.bbclass | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
meta/classes/icecc.bbclass +8 -9
index e8725b582b3..35a1aaef86f 100644
@@ -210,15 +210,14 @@ def icecc_get_tool(bb, d, tool):
     else:
         ice_dir = d.expand('${STAGING_BINDIR_TOOLCHAIN}')
         target_sys = d.expand('${TARGET_SYS}')
-        tool_bin = os.path.join(ice_dir, "%s-%s" % (target_sys, tool))
-        if os.path.isfile(tool_bin):
-            return tool_bin
-        else:
-            external_tool_bin = icecc_get_external_tool(bb, d, tool)
-            if os.path.isfile(external_tool_bin):
-                return external_tool_bin
-            else:
-                return ""
+        for p in ice_dir.split(':'):
+            tool_bin = os.path.join(p, "%s-%s" % (target_sys, tool))
+            if os.path.isfile(tool_bin):
+                return tool_bin
+        external_tool_bin = icecc_get_external_tool(bb, d, tool)
+        if os.path.isfile(external_tool_bin):
+            return external_tool_bin
+        return ""
 
 def icecc_get_and_check_tool(bb, d, tool):
     # Check that g++ or gcc is not a symbolic link to icecc binary in
Signature · 1 line
2.14.3
#3 lore raw

PATCH 3/14 icecc.bbclass: Skip canadian-cross compiles

In reply to Joshua Watt

icecc.bbclass will no longer attempt to distribute cross-canadian compiles. While technically possible to generate a toolchain that runs on the build system and generates executables for the host system, this is not the normal way that icecc operates and there are so few of these recipes that it is probably not worth maintaining.

---
 meta/classes/icecc.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)
meta/classes/icecc.bbclass +6 -0
index 35a1aaef86f..9d5eaed57ce 100644
@@ -101,6 +101,9 @@ def use_icecc(bb,d):
     if icecc_is_allarch(bb, d):
         return "no"
 
+    if icecc_is_cross_canadian(bb, d):
+        return "no"
+
     pn = d.getVar('PN')
 
     system_class_blacklist = []
@@ -151,6 +154,9 @@ def icecc_is_native(bb, d):
         bb.data.inherits_class("cross", d) or \
         bb.data.inherits_class("native", d);
 
+def icecc_is_cross_canadian(bb, d):
+    return bb.data.inherits_class("cross-canadian", d)
+
 def icecc_dir(bb, d):
     return d.expand('${TMPDIR}/work-shared/ice')
 
Signature · 1 line
2.14.3
#4 lore raw

PATCH 4/14 icecc-create-env: Reformat

In reply to Joshua Watt

The environment script used an annoying mix of tabs and spaces, and no mapping of tabs to spaces would produce pleasant indentation. Reformat to eliminate tab characters and settle on 4 spaces for indentation (which matches the upstream icecream script from which this is derived)

---
 .../icecc-create-env/icecc-create-env              | 208 ++++++++++-----------
 1 file changed, 104 insertions(+), 104 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +104 -104
index 7e4dbc414e7..723f1f9bd90 100755
@@ -8,63 +8,63 @@ target_files=
 
 is_contained ()
 {
-  case " $target_files " in
-    *" $1 "* ) return 0 ;;
-    *"=$1 "* ) return 0;;
-    * ) return 1 ;;
-  esac
+    case " $target_files " in
+        *" $1 "* ) return 0 ;;
+        *"=$1 "* ) return 0;;
+        * ) return 1 ;;
+    esac
 }
 
 add_file ()
 {
-  local name="$1"
-  local path="$1";
-  if test -n "$2"; then
-    name="$2"
-  fi
-  test -z "$name" && return
-  # ls -H isn't really the same as readlink, but
-  # readlink is not portable enough.
-  path=`ls -H $path`
-  toadd="$name=$path"
-  is_contained "$toadd" && return
-  if test -z "$silent"; then
-  echo "adding file $toadd"
-  fi
-  target_files="$target_files $toadd"
-  if test -x "$path"; then
-    # Only call ldd when it makes sense
-    if file -L "$path" | grep 'ELF' > /dev/null 2>&1; then
-	if ! file -L "$path" | grep 'static' > /dev/null 2>&1; then
-	   # ldd now outputs ld as /lib/ld-linux.so.xx on current nptl based glibc
-		# this regexp parse the outputs like:
-		# ldd /usr/bin/gcc
-		#         linux-gate.so.1 =>  (0xffffe000)
-		#         libc.so.6 => /lib/tls/libc.so.6 (0xb7e81000)
-		#         /lib/ld-linux.so.2 (0xb7fe8000)
-		# covering both situations ( with => and without )
-          for lib in `ldd "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
-	    test -f "$lib" || continue
-	    # Check wether the same library also exists in the parent directory,
-	    # and prefer that on the assumption that it is a more generic one.
-	    local baselib=`echo "$lib" | sed 's,\(/[^/]*\)/.*\(/[^/]*\)$,\1\2,'`
-	    test -f "$baselib" && lib=$baselib
-	  add_file "$lib"
-        done
-      fi
+    local name="$1"
+    local path="$1";
+    if test -n "$2"; then
+        name="$2"
+    fi
+    test -z "$name" && return
+    # ls -H isn't really the same as readlink, but
+    # readlink is not portable enough.
+    path=`ls -H $path`
+    toadd="$name=$path"
+    is_contained "$toadd" && return
+    if test -z "$silent"; then
+        echo "adding file $toadd"
+    fi
+    target_files="$target_files $toadd"
+    if test -x "$path"; then
+        # Only call ldd when it makes sense
+        if file -L "$path" | grep 'ELF' > /dev/null 2>&1; then
+            if ! file -L "$path" | grep 'static' > /dev/null 2>&1; then
+                # ldd now outputs ld as /lib/ld-linux.so.xx on current nptl based glibc
+                # this regexp parse the outputs like:
+                # ldd /usr/bin/gcc
+                #         linux-gate.so.1 =>  (0xffffe000)
+                #         libc.so.6 => /lib/tls/libc.so.6 (0xb7e81000)
+                #         /lib/ld-linux.so.2 (0xb7fe8000)
+                # covering both situations ( with => and without )
+                for lib in `ldd "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
+                    test -f "$lib" || continue
+                    # Check wether the same library also exists in the parent directory,
+                    # and prefer that on the assumption that it is a more generic one.
+                    local baselib=`echo "$lib" | sed 's,\(/[^/]*\)/.*\(/[^/]*\)$,\1\2,'`
+                    test -f "$baselib" && lib=$baselib
+                    add_file "$lib"
+                done
+            fi
+        fi
     fi
-  fi
 }
 
 # backward compat
 if test "$1" = "--respect-path"; then
-  shift
+    shift
 fi
 
 #add a --silent switch to avoid "broken pipe" errors when calling this scipt from within OE
 if test "$1" = "--silent"; then
- silent=1
- shift
+    silent=1
+    shift
 fi
 
 
@@ -77,18 +77,18 @@ shift
 archive_name=$1
 
 if test -z "$added_gcc" || test -z "$added_gxx" ; then
-	echo "usage: $0 <gcc_path> <g++_path>"
-	exit 1
+    echo "usage: $0 <gcc_path> <g++_path>"
+    exit 1
 fi
 
 if ! test -x "$added_gcc" ; then
-  echo "'$added_gcc' is no executable."
-  exit 1
+    echo "'$added_gcc' is no executable."
+    exit 1
 fi
 
 if ! test -x "$added_gxx" ; then
-  echo "'$added_gcc' is no executable."
-  exit 1
+    echo "'$added_gcc' is no executable."
+    exit 1
 fi
 
 
@@ -97,51 +97,51 @@ add_file $added_gcc /usr/bin/gcc
 add_file $added_gxx /usr/bin/g++
 
 if test -z "$added_as" ; then
- add_file /usr/bin/as /usr/bin/as
+    add_file /usr/bin/as /usr/bin/as
 else
- if ! test -x "$added_as" ; then
-  echo "'$added_as' is no executable."
-  exit 1
- fi
+    if ! test -x "$added_as" ; then
+        echo "'$added_as' is no executable."
+        exit 1
+    fi
 
- add_file $added_as  /usr/bin/as
+    add_file $added_as  /usr/bin/as
 fi
 
 add_file `$added_gcc -print-prog-name=cc1` /usr/bin/cc1
 add_file `$added_gxx -print-prog-name=cc1plus` /usr/bin/cc1plus
 specfile=`$added_gcc -print-file-name=specs`
 if test -n "$specfile" && test -e "$specfile"; then
-  add_file "$specfile"
+    add_file "$specfile"
 fi
 
 ltofile=`$added_gcc -print-prog-name=lto1`
 pluginfile="${ltofile%lto1}liblto_plugin.so"
 if test -r "$pluginfile"
 then
-  add_file $pluginfile  ${pluginfile#*usr}
-  add_file $pluginfile  /usr${pluginfile#*usr}
+    add_file $pluginfile  ${pluginfile#*usr}
+    add_file $pluginfile  /usr${pluginfile#*usr}
 fi
 
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
 new_target_files=
 for i in $target_files; do
- case $i in
-   *=/*)
-    target=`echo $i | cut -d= -f1`
-    path=`echo $i | cut -d= -f2`
-    ;;
-   *)
-    path=$i
-    target=$i
-    ;;
-  esac
-  mkdir -p $tempdir/`dirname $target`
-  cp -p $path $tempdir/$target
-  if test -f $tempdir/$target -a -x $tempdir/$target; then
-    strip -s $tempdir/$target 2>/dev/null
-  fi
-  target=`echo $target | cut -b2-`
-  new_target_files="$new_target_files $target"
+    case $i in
+        *=/*)
+            target=`echo $i | cut -d= -f1`
+            path=`echo $i | cut -d= -f2`
+            ;;
+        *)
+            path=$i
+            target=$i
+            ;;
+    esac
+    mkdir -p $tempdir/`dirname $target`
+    cp -p $path $tempdir/$target
+    if test -f $tempdir/$target -a -x $tempdir/$target; then
+        strip -s $tempdir/$target 2>/dev/null
+    fi
+    target=`echo $target | cut -b2-`
+    new_target_files="$new_target_files $target"
 done
 
 #sort the files
@@ -150,43 +150,43 @@ target_files=`for i in $new_target_files; do echo $i; done | sort`
 #test if an archive name was supplied
 #if not use the md5 of all files as the archive name
 if test -z "$archive_name"; then
-  md5sum=NONE
-  for file in /usr/bin/md5sum /bin/md5 /usr/bin/md5; do
-    if test -x $file; then
-      md5sum=$file
-      break
-    fi
-  done
-
-  #calculate md5 and use it as the archive name
-  archive_name=`for i in $target_files; do test -f $tempdir/$i && $md5sum $tempdir/$i; done | sed -e 's/ .*$//' | $md5sum | sed -e 's/ .*$//'`.tar.gz || {
-    if test -z "$silent"; then
-     echo "Couldn't compute MD5 sum."
-    fi
-    exit 2
-  }
-  mydir=`pwd`
+    md5sum=NONE
+    for file in /usr/bin/md5sum /bin/md5 /usr/bin/md5; do
+        if test -x $file; then
+            md5sum=$file
+            break
+        fi
+    done
+
+    #calculate md5 and use it as the archive name
+    archive_name=`for i in $target_files; do test -f $tempdir/$i && $md5sum $tempdir/$i; done | sed -e 's/ .*$//' | $md5sum | sed -e 's/ .*$//'`.tar.gz || {
+        if test -z "$silent"; then
+            echo "Couldn't compute MD5 sum."
+        fi
+        exit 2
+    }
+    mydir=`pwd`
 else
-  mydir="`dirname "$archive_name"`"
+    mydir="`dirname "$archive_name"`"
 
-  #check if we have a full path or only a filename
-  if test "$mydir" = "." ; then
-    mydir=`pwd`
-  else
-    mydir=""
-  fi
+    #check if we have a full path or only a filename
+    if test "$mydir" = "." ; then
+        mydir=`pwd`
+    else
+        mydir=""
+    fi
 fi
 
 if test -z "$silent"; then
-echo "creating $archive_name"
+    echo "creating $archive_name"
 fi
 
 cd $tempdir
 tar -czhf "$mydir/$archive_name" $target_files || {
- if test -z "$silent"; then
-  echo "Couldn't create archive"
- fi
-  exit 3
+    if test -z "$silent"; then
+        echo "Couldn't create archive"
+    fi
+    exit 3
 }
 cd ..
 rm -rf $tempdir
Signature · 1 line
2.14.3
#5 lore raw

PATCH 5/14 icecc-create-env: Add /bin/true to environment

In reply to Joshua Watt

icecream daemons execute /bin/true from the environment as a check to determine if the environment is valid at all, so it needs to be included.

---
 .../icecc-create-env/icecc-create-env/icecc-create-env         | 10 ++++++++++
 1 file changed, 10 insertions(+)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +10 -0
index 723f1f9bd90..c838256f748 100755
@@ -122,6 +122,16 @@ then
     add_file $pluginfile  /usr${pluginfile#*usr}
 fi
 
+# for testing the environment is usable at all
+if test -x /bin/true; then
+    add_file /bin/true
+elif test -x /usr/bin/true; then
+    add_file /usr/bin/true /bin/true
+else
+    echo "'true' not found"
+    exit 1
+fi
+
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
 new_target_files=
 for i in $target_files; do
Signature · 1 line
2.14.3
#6 lore raw

PATCH 6/14 icecc-create-env: Use program interpreter for deps

In reply to Joshua Watt

ldd cannot always be used to determine a programs dependencies correctly, particularly when the program specifies an alternate program interpreter (dynamic loader). This commonly happens when using a uninative tarball. Instead, determine the program's requested interpreter, and ask it to list the dependencies.

---
 .../icecc-create-env/icecc-create-env                | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +17 -3
index c838256f748..7636d090a48 100755
@@ -6,6 +6,12 @@
 
 target_files=
 
+is_dynamic_elf ()
+{
+    # Is the file an dynamically linked ELF executable?
+    (file -L "$1" | grep 'ELF' > /dev/null 2>&1) && (! file -L "$1" | grep 'static' > /dev/null 2>&1)
+}
+
 is_contained ()
 {
     case " $target_files " in
@@ -34,8 +40,16 @@ add_file ()
     target_files="$target_files $toadd"
     if test -x "$path"; then
         # Only call ldd when it makes sense
-        if file -L "$path" | grep 'ELF' > /dev/null 2>&1; then
-            if ! file -L "$path" | grep 'static' > /dev/null 2>&1; then
+        if is_dynamic_elf "$path"; then
+            # Request the program interpeter (dynamic loader)
+            interp=`readelf -w -l "$path" | grep "Requesting program interpreter:" | sed "s/\s*\[Requesting program interpreter:\s*\(.*\)\]/\1/g"`
+
+            if test -n "$interp" && test -x "$interp"; then
+                # Use the dynamic loaders --list argument to list the
+                # depenencies. The program may have a a different program
+                # interpeter (typical when using uninative tarballs), which is
+                # why we can't just call ldd.
+                #
                 # ldd now outputs ld as /lib/ld-linux.so.xx on current nptl based glibc
                 # this regexp parse the outputs like:
                 # ldd /usr/bin/gcc
@@ -43,7 +57,7 @@ add_file ()
                 #         libc.so.6 => /lib/tls/libc.so.6 (0xb7e81000)
                 #         /lib/ld-linux.so.2 (0xb7fe8000)
                 # covering both situations ( with => and without )
-                for lib in `ldd "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
+                for lib in `$interp --list "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
                     test -f "$lib" || continue
                     # Check wether the same library also exists in the parent directory,
                     # and prefer that on the assumption that it is a more generic one.
Signature · 1 line
2.14.3
#7 lore raw

PATCH 7/14 icecc-create-env: Symlink alternate names

In reply to Joshua Watt

Instead of renaming files to a new path in the toolchain archive, keep the files with their original paths and create a relative symbolic link from the new path to the original file.

---
 .../icecc-create-env/icecc-create-env              | 64 +++++++++++++++++++---
 1 file changed, 57 insertions(+), 7 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +57 -7
index 7636d090a48..0791bd54b27 100755
@@ -21,10 +21,28 @@ is_contained ()
     esac
 }
 
+normalize_path ()
+{
+    # Normalizes the path to a file or directory, removing all "." and ".."
+    # entries. Use pwd -L to explicitly prevent symlink expansion
+    local path=$1
+    if test -f "$path"; then
+        pushd $(dirname $path) > /dev/null 2>&1
+        dir_path=$(pwd -L)
+        path=$dir_path/$(basename $path)
+        popd > /dev/null 2>&1
+    elif test -d "$path"; then
+        pushd $path > /dev/null 2>&1
+        path=$(pwd -L)
+        popd > /dev/null 2>&1
+    fi
+    echo $path
+}
+
 add_file ()
 {
-    local name="$1"
-    local path="$1";
+    local path=`normalize_path $1`
+    local name="$path"
     if test -n "$2"; then
         name="$2"
     fi
@@ -129,7 +147,7 @@ if test -n "$specfile" && test -e "$specfile"; then
 fi
 
 ltofile=`$added_gcc -print-prog-name=lto1`
-pluginfile="${ltofile%lto1}liblto_plugin.so"
+pluginfile=`normalize_path "${ltofile%lto1}liblto_plugin.so"`
 if test -r "$pluginfile"
 then
     add_file $pluginfile  ${pluginfile#*usr}
@@ -146,6 +164,19 @@ else
     exit 1
 fi
 
+link_rel ()
+{
+    local target="$1"
+    local name="$2"
+    local base="$3"
+
+    local prefix=`dirname $name`
+
+    prefix=`echo $prefix | sed 's,[^/]\+,..,g' | sed 's,^/*,,g'`
+
+    ln -s $prefix/$target $base/$name
+}
+
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
 new_target_files=
 for i in $target_files; do
@@ -159,11 +190,30 @@ for i in $target_files; do
             target=$i
             ;;
     esac
-    mkdir -p $tempdir/`dirname $target`
-    cp -p $path $tempdir/$target
-    if test -f $tempdir/$target -a -x $tempdir/$target; then
-        strip -s $tempdir/$target 2>/dev/null
+    if test "$target" == "$path"; then
+        mkdir -p $tempdir/`dirname $target`
+        cp -pH $target $tempdir/$target
+
+        if test -f $tempdir/$target -a -x $tempdir/$target; then
+            strip -s $tempdir/$target 2>/dev/null
+        fi
+    else
+        mkdir -p $tempdir/`dirname $path`
+        cp -pH $path $tempdir/$path
+
+        mkdir -p $tempdir/`dirname $target`
+        # Relative links are used because the files are checked for being
+        # executable outside the chroot
+        link_rel $path $target $tempdir
+
+        if test -f $tempdir/$path -a -x $tempdir/$path; then
+            strip -s $tempdir/$path 2>/dev/null
+        fi
+
+        path=`echo $path | cut -b2-`
+        new_target_files="$new_target_files $path"
     fi
+
     target=`echo $target | cut -b2-`
     new_target_files="$new_target_files $target"
 done
Signature · 1 line
2.14.3
#8 lore raw

PATCH 8/14 icecc-create-env: Archive directory

In reply to Joshua Watt

Taring up the toolchain is now done by adding the entire working directory, instead of listing all the files individually. This is done because the list of files may contain ".." entries, which tar does not like and strips out, resulting in bad archives. This should result in an identical archive to what was previously generated.

In addition, symbolic links are no longer dereferenced when creating the archive, as they are purposely included to provide alternate names for files

---
 .../icecc-create-env/icecc-create-env/icecc-create-env              | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +5 -1
index 0791bd54b27..426b093d91c 100755
@@ -256,7 +256,11 @@ if test -z "$silent"; then
 fi
 
 cd $tempdir
-tar -czhf "$mydir/$archive_name" $target_files || {
+# Add everything in the temp directory. Tar doesn't like to be given files with
+# ".." in them, which frequently happens in $target_files, and will strip off
+# the path prefix past the offending "..". This makes the archive generate
+# incorrectly
+tar -czf "$mydir/$archive_name" . || {
     if test -z "$silent"; then
         echo "Couldn't create archive"
     fi
Signature · 1 line
2.14.3
#9 lore raw

PATCH 9/14 icecc-create-env: Fix executable rpaths

In reply to Joshua Watt

Executables in the toolchain archive occasionally contain runtime library search paths (RPATH) that use the $ORIGIN placeholder. However, in order for that placeholder to work, /proc must be mounted. When iceccd executes the toolchain in the chroot environment, it doesn't mount /proc, so it is unable to resolve $ORIGIN resulting in a failure to find dynamic libraries.

The fix is to replace $ORIGIN in executable RPATH entries with the known chroot executable path. In order for this to work, the actual real path to the executable must be resolved to remove any symlinks, otherwise the calculate $ORIGIN replacement will be wrong. This is done by using "readlink -f", which is an acceptable dependency because Yocto already requires it.

---
 .../icecc-create-env/icecc-create-env              | 40 ++++++++++++++++++++--
 1 file changed, 37 insertions(+), 3 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +37 -3
index 426b093d91c..90d249df9f3 100755
@@ -12,6 +12,26 @@ is_dynamic_elf ()
     (file -L "$1" | grep 'ELF' > /dev/null 2>&1) && (! file -L "$1" | grep 'static' > /dev/null 2>&1)
 }
 
+fix_rpath ()
+{
+    # Patches the RPATH for a file. When the program is executed in the chroot
+    # be iceccd, /proc is not mounted. As such, $ORIGIN can't be resolved. To
+    # work around this, replace all instances of $ORIGIN in RPATH with the
+    # known chroot path to the executables directory
+    local path="$1"
+    local origin="$2"
+    if ! is_dynamic_elf "$path"; then
+        return
+    fi
+    local new_rpath="`readelf -w -d "$path" | grep RPATH | \
+        sed 's/.*\[\(.*\)\]/\1/g' | \
+        sed "s,\\\$ORIGIN,/$origin,g"`"
+
+    if test -n "$new_rpath"; then
+        $PATCHELF --set-rpath "$new_rpath" "$path"
+    fi
+}
+
 is_contained ()
 {
     case " $target_files " in
@@ -47,9 +67,8 @@ add_file ()
         name="$2"
     fi
     test -z "$name" && return
-    # ls -H isn't really the same as readlink, but
-    # readlink is not portable enough.
-    path=`ls -H $path`
+    # readlink is required for Yocto, so we can use it
+    path=`readlink -f "$path"`
     toadd="$name=$path"
     is_contained "$toadd" && return
     if test -z "$silent"; then
@@ -108,6 +127,17 @@ added_as=$1
 shift
 archive_name=$1
 
+if test -z "$PATCHELF"; then
+    PATCHELF=`which patchelf 2> /dev/null`
+fi
+if test -z "$PATCHELF"; then
+    PATCHELF=`which patchelf-uninative 2> /dev/null`
+fi
+if test -z "$PATCHELF"; then
+    echo "patchelf is required"
+    exit 1
+fi
+
 if test -z "$added_gcc" || test -z "$added_gxx" ; then
     echo "usage: $0 <gcc_path> <g++_path>"
     exit 1
@@ -197,6 +227,8 @@ for i in $target_files; do
         if test -f $tempdir/$target -a -x $tempdir/$target; then
             strip -s $tempdir/$target 2>/dev/null
         fi
+
+        fix_rpath $tempdir/$target `dirname $target`
     else
         mkdir -p $tempdir/`dirname $path`
         cp -pH $path $tempdir/$path
@@ -210,6 +242,8 @@ for i in $target_files; do
             strip -s $tempdir/$path 2>/dev/null
         fi
 
+        fix_rpath $tempdir/$path `dirname $path`
+
         path=`echo $path | cut -b2-`
         new_target_files="$new_target_files $path"
     fi
Signature · 1 line
2.14.3
#10 lore raw

PATCH 10/14 icecc-create-env: Add support for nativesdk

In reply to Joshua Watt

icecc-create-env can now be built as a nativesdk recipe, allowing the script to be included as part of an SDK

---
 .../{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb}       | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb} (98%)
meta/recipes-devtools/icecc-create-env/icecc-create-env_0.1.bb renamed from meta/recipes-devtools/icecc-create-env/icecc-create-env-native_0.1.bb +2 -2
similarity index 98%
rename from meta/recipes-devtools/icecc-create-env/icecc-create-env-native_0.1.bb
rename to meta/recipes-devtools/icecc-create-env/icecc-create-env_0.1.bb
index f8f4aef3923..053945ebbc6 100644
@@ -12,8 +12,6 @@ PR = "r2"
 DEPENDS = ""
 INHIBIT_DEFAULT_DEPS = "1"
 
-inherit native
-
 # This is needed, because otherwise there is dependency loop from quilt-native
 # Dependency loop #1 found:
 #  Task 10907 (meta/recipes-devtools/quilt/quilt-native_0.60.bb, do_install) (dependent Tasks ['quilt-native, do_compile'])
@@ -34,3 +32,5 @@ do_install() {
     install -d ${D}/${bindir}
     install -m 0755 ${WORKDIR}/icecc-create-env ${D}/${bindir}
 }
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.14.3
#11 lore raw

PATCH 11/14 icecc-create-env: Allow multiple tool aliases

In reply to Joshua Watt

When files are added to the environment, multiple aliases can be given for the file (by calling add_path multiple times with a second argument). All of these names will end up with a symlink to the original file.

---
 .../icecc-create-env/icecc-create-env              | 110 ++++++++++-----------
 1 file changed, 54 insertions(+), 56 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +54 -56
index 90d249df9f3..537e38a9ba0 100755
@@ -4,7 +4,8 @@
 # Copyright (C) 2004 by the Icecream Authors
 # GPL
 
-target_files=
+target_paths=
+target_aliases=
 
 is_dynamic_elf ()
 {
@@ -32,15 +33,33 @@ fix_rpath ()
     fi
 }
 
-is_contained ()
+add_path ()
 {
-    case " $target_files " in
-        *" $1 "* ) return 0 ;;
-        *"=$1 "* ) return 0;;
-        * ) return 1 ;;
+    case " $target_paths " in
+        *" $1 "*)
+            return 1
+            ;;
+        *)
+            target_paths="$target_paths $1"
+            return 0
+            ;;
     esac
 }
 
+add_alias ()
+{
+    if test "$1" != "$2"; then
+        local alias="$1=$2"
+        case " $target_aliases " in
+            *" $alias "*)
+                ;;
+            *)
+                target_aliases="$target_aliases $alias"
+                ;;
+        esac
+    fi
+}
+
 normalize_path ()
 {
     # Normalizes the path to a file or directory, removing all "." and ".."
@@ -61,20 +80,17 @@ normalize_path ()
 
 add_file ()
 {
-    local path=`normalize_path $1`
-    local name="$path"
-    if test -n "$2"; then
-        name="$2"
-    fi
-    test -z "$name" && return
+    local p=`normalize_path $1`
     # readlink is required for Yocto, so we can use it
-    path=`readlink -f "$path"`
-    toadd="$name=$path"
-    is_contained "$toadd" && return
-    if test -z "$silent"; then
-        echo "adding file $toadd"
+    local path=`readlink -f "$p"`
+
+    add_alias "$path" "$p"
+    if test -n "$2"; then
+        add_alias "$path" "$2"
     fi
-    target_files="$target_files $toadd"
+
+    add_path "$path" || return
+
     if test -x "$path"; then
         # Only call ldd when it makes sense
         if is_dynamic_elf "$path"; then
@@ -208,52 +224,34 @@ link_rel ()
 }
 
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
-new_target_files=
-for i in $target_files; do
-    case $i in
-        *=/*)
-            target=`echo $i | cut -d= -f1`
-            path=`echo $i | cut -d= -f2`
-            ;;
-        *)
-            path=$i
-            target=$i
-            ;;
-    esac
-    if test "$target" == "$path"; then
-        mkdir -p $tempdir/`dirname $target`
-        cp -pH $target $tempdir/$target
-
-        if test -f $tempdir/$target -a -x $tempdir/$target; then
-            strip -s $tempdir/$target 2>/dev/null
-        fi
-
-        fix_rpath $tempdir/$target `dirname $target`
-    else
-        mkdir -p $tempdir/`dirname $path`
-        cp -pH $path $tempdir/$path
+target_files=
+for path in $target_paths; do
+    mkdir -p $tempdir/`dirname $path`
+    cp -pH $path $tempdir/$path
 
-        mkdir -p $tempdir/`dirname $target`
-        # Relative links are used because the files are checked for being
-        # executable outside the chroot
-        link_rel $path $target $tempdir
+    if test -f $tempdir/$path -a -x $tempdir/$path; then
+        strip -s $tempdir/$path 2>/dev/null
+    fi
 
-        if test -f $tempdir/$path -a -x $tempdir/$path; then
-            strip -s $tempdir/$path 2>/dev/null
-        fi
+    fix_rpath $tempdir/$path `dirname $path`
+    target_files="$target_files $path"
+done
 
-        fix_rpath $tempdir/$path `dirname $path`
+for i in $target_aliases; do
+    target=`echo $i | cut -d= -f1`
+    link_name=`echo $i | cut -d= -f2`
 
-        path=`echo $path | cut -b2-`
-        new_target_files="$new_target_files $path"
-    fi
+    mkdir -p $tempdir/`dirname $link_name`
+    # Relative links are used because the files are checked for being
+    # executable outside the chroot
+    link_rel $target $link_name $tempdir
 
-    target=`echo $target | cut -b2-`
-    new_target_files="$new_target_files $target"
+    link_name=`echo $link_name | cut -b2-`
+    target_files="$target_files $link_name"
 done
 
 #sort the files
-target_files=`for i in $new_target_files; do echo $i; done | sort`
+target_files=`for i in $target_files; do echo $i; done | sort`
 
 #test if an archive name was supplied
 #if not use the md5 of all files as the archive name
Signature · 1 line
2.14.3
#12 lore raw

PATCH 12/14 toolchain-shar-extract: Add post-relocate scripts

In reply to Joshua Watt

Recipes can now install post-relocation scripts which will be run when the SDK is installed.

---
 meta/classes/toolchain-scripts.bbclass     | 15 +++++++++++++++
 meta/files/toolchain-shar-extract.sh       |  8 ++++++++
 meta/recipes-core/meta/meta-environment.bb |  2 ++
 3 files changed, 25 insertions(+)
meta/classes/toolchain-scripts.bbclass +15 -0
index 9bcfe708c7c..eeb320d15a8 100644
@@ -116,6 +116,21 @@ fi
 EOF
 }
 
+toolchain_create_post_relocate_script() {
+	script=$1
+	rm -f $script
+	touch $script
+
+    cat >> $script <<EOF
+if [ -d "${SDKPATHNATIVE}/post-relocate-setup.d/" ]; then
+    for s in ${SDKPATHNATIVE}/post-relocate-setup.d/*.sh; do
+        \$s "\$1"
+    done
+    rm -rf "${SDKPATHNATIVE}/post-relocate-setup.d"
+fi
+EOF
+}
+
 #we get the cached site config in the runtime
 TOOLCHAIN_CONFIGSITE_NOCACHE = "${@siteinfo_get_files(d)}"
 TOOLCHAIN_CONFIGSITE_SYSROOTCACHE = "${STAGING_DIR}/${MLPREFIX}${MACHINE}/${target_datadir}/${TARGET_SYS}_config_site.d"
meta/files/toolchain-shar-extract.sh +8 -0
index 91804ec2818..f7ab43066fb 100644
@@ -259,6 +259,14 @@ if [ $savescripts = 0 ] ; then
 	$SUDO_EXEC rm -f ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh
 fi
 
+# Execute post-relocation script
+post_relocate="$target_sdk_dir/post-relocate-setup.sh"
+if [ -e "$post_relocate" ]; then
+	$SUDO_EXEC sed -e "s:@SDKPATH@:$target_sdk_dir:g" -i $post_relocate
+	$SUDO_EXEC /bin/sh $post_relocate "$target_sdk_dir" "@SDKPATH@"
+	$SUDO_EXEC rm -f $post_relocate
+fi
+
 echo "SDK has been successfully set up and is ready to be used."
 echo "Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g."
 for env_setup_script in `ls $target_sdk_dir/environment-setup-*`; do
meta/recipes-core/meta/meta-environment.bb +2 -0
index 29da121a3a9..09f757a087c 100644
@@ -55,6 +55,8 @@ create_sdk_files() {
 
 	# Add version information
 	toolchain_create_sdk_version ${SDK_OUTPUT}/${SDKPATH}/version-${REAL_MULTIMACH_TARGET_SYS}
+
+	toolchain_create_post_relocate_script ${SDK_OUTPUT}/${SDKPATH}/post-relocate-setup.sh
 }
 
 do_install() {
Signature · 1 line
2.14.3
#13 lore raw

PATCH 13/14 icecc-toolchain: Add SDK icecream setup

In reply to Joshua Watt

IceCream can now be optionally included in the generated SDK by including nativesdk-icecc-toolchain to TOOLCHAIN_HOST_TASK. When the SDK is installed, it will check if icecc exists, and if so will generate the toolchain environment.

---
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   | 37 +++++++++++++++++++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh | 43 ++++++++++++++++++++++
 .../nativesdk-icecc-toolchain_0.1.bb               | 38 +++++++++++++++++++
 3 files changed, 118 insertions(+)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +37 -0
new file mode 100644
index 00000000000..4c34edbbc5d
@@ -0,0 +1,37 @@
+#! /bin/sh
+# Copyright 2018 Garmin Ltd. or its subsidiaries
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+if [ -z "$ICECC_PATH" ]; then
+    ICECC_PATH=$(which icecc 2> /dev/null)
+fi
+
+if [ -n "$ICECC_PATH" ]; then
+    export ICECC_PATH
+    export ICECC_VERSION="$OECORE_NATIVE_SYSROOT/usr/share/icecream/@TOOLCHAIN_ENV@"
+    export ICECC="$(which ${CROSS_COMPILE}gcc)"
+    export ICECXX="$(which ${CROSS_COMPILE}g++)"
+    export ICEAS="$(which ${CROSS_COMPILE}as)"
+    export PATH="$OECORE_NATIVE_SYSROOT/usr/share/icecream/bin:$PATH"
+else
+    echo "Icecc not found. Disabling distributed compiling"
+fi
+
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +43 -0

43 lines not on this page · raw message

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb new file +38 -0
new file mode 100644
index 00000000000..28b2924ebfb
@@ -0,0 +1,38 @@
+# Copyright (C) 2018 Joshua Watt <[email protected]>
+# Released under the MIT license (see COPYING.MIT for the terms)
+DESCRIPTION = "Generates IceCream toolchains for SDKs"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${WORKDIR}/icecc-env.sh;beginline=2;endline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c"
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI = "\
+    file://icecc-env.sh \
+    file://icecc-setup.sh \
+    "
+
+inherit nativesdk
+
+ENV_NAME="${DISTRO}-${TCLIBC}-${SDK_ARCH}-${TUNE_PKGARCH}-${DISTRO_VERSION}.tar.gz"
+
+do_compile() {
+}
+
+do_install() {
+    install -d ${D}${SDKPATHNATIVE}${datadir}/icecream/bin
+
+    install -d ${D}${SDKPATHNATIVE}/environment-setup.d/
+    install -m 0644 ${WORKDIR}/icecc-env.sh ${D}${SDKPATHNATIVE}/environment-setup.d/
+    sed -i ${D}${SDKPATHNATIVE}/environment-setup.d/icecc-env.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+
+    install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
+    install -m 0755 ${WORKDIR}/icecc-setup.sh ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
+    sed -i ${D}${SDKPATHNATIVE}/post-relocate-setup.d/icecc-setup.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+}
+
+PACKAGES = "${PN}"
+FILES_${PN} = "${SDKPATHNATIVE}"
+RDEPENDS_${PN} += "nativesdk-icecc-create-env"
+
Signature · 1 line
2.14.3
#17 lore raw

PATCH 14/14 icecc.bbclass: Add IceCream support to SDK

In reply to Joshua Watt

If icecc is inherited, generated SDKs will automatically have optional support for compiling using the IceCream distributed compiler

---
 meta/classes/icecc.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)
meta/classes/icecc.bbclass +7 -0
index 9d5eaed57ce..8492ece60c3 100644
@@ -346,3 +346,10 @@ do_compile_kernelmodules_prepend() {
 do_install_prepend() {
     set_icecc_env
 }
+
+# IceCream is not (currently) supported in the extensible SDK
+ICECC_SDK_HOST_TASK = "nativesdk-icecc-toolchain"
+ICECC_SDK_HOST_TASK_task-populate-sdk-ext = ""
+
+# Add the toolchain scripts to the SDK
+TOOLCHAIN_HOST_TASK_append = " ${ICECC_SDK_HOST_TASK}"
Signature · 1 line
2.14.3
#18 lore raw

✗ patchtest: failure for Fix support for Icecream

In reply to Joshua Watt

== Series Details ==

Series: Fix support for Icecream
Revision: 1
URL : https://patchwork.openembedded.org/series/10867/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue SUMMARY is missing in newly added recipe [test_summary_presence]
Suggested fix Specify the variable SUMMARY in nativesdk-icecc-toolchain

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
#19 lore raw
In reply to Joshua Watt

Hi Joshua,

Quoted 2 lines · On Tue, 6 Feb 2018 16:01:19 -0600 Joshua Watt <[email protected]> wrote:

Fix up support for using Icecream to do distributed builds, which appears to have been broken for some time.

we were using icecream a year ago and we set ICECC_CARET_WORKAROUND env variable to 0 (in set_icecc_env() in icecc.bbclass). The default is 1 and it causes a local rebuild when any warning is emitted.

I don't have exact numbers, but the extreme case was opencv. There was so many warnings that distributed build on ~10 threads was slower than local build on 4 threads.

I know that it is some kind of configuration, but I would set the default to
0. Or do it at least configurable.

Quoted 4 lines

In addition, Icecream support can now be enabled in the SDK. When enabled, the SDK install process will check if the host supports icecc and if so will construct a proper environment tarball and use it when compiling.

Thanks for work on that, I'm keen to use it in our environment.

Tomas

Quoted 43 lines

Most of the changes should be straightforward. icecc-create-env has been almost completely re-written but the changes were broken up into hopefully logical chunks.

The following changes since commit 385944254d0ef88bd6450a221a54cbcb40020b42:

tcmode-default.inc: drop preferred version of gzip-native (2018-01-31 17:01:20 +0000)

are available in the Git repository at:

git://push.yoctoproject.org/poky-contrib jpew/icecream

Joshua Watt (14):
icecc.bbclass: Move to shared work directory icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage icecc.bbclass: Skip canadian-cross compiles
icecc-create-env: Reformat
icecc-create-env: Add /bin/true to environment
icecc-create-env: Use program interpreter for deps
icecc-create-env: Symlink alternate names
icecc-create-env: Archive directory
icecc-create-env: Fix executable rpaths
icecc-create-env: Add support for nativesdk
icecc-create-env: Allow multiple tool aliases
toolchain-shar-extract: Add post-relocate scripts
icecc-toolchain: Add SDK icecream setup icecc.bbclass: Add IceCream support to SDK

 meta/classes/icecc.bbclass                         |  37 ++-
 meta/classes/toolchain-scripts.bbclass             |  15 +
 meta/files/toolchain-shar-extract.sh               |   8 +
 meta/recipes-core/meta/meta-environment.bb         |   2 +
 .../icecc-create-env/icecc-create-env              | 328 ++++++++++++++-------
 ...e-env-native_0.1.bb => icecc-create-env_0.1.bb} |   4 +-
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   |  37 +++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh |  43 +++
 .../nativesdk-icecc-toolchain_0.1.bb               |  38 +++
 9 files changed, 390 insertions(+), 122 deletions(-)
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb} (98%)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb
#20 lore raw
Quoted 14 lines · Tomas Novotny

Hi Joshua,

Quoted 2 lines · On Tue, 6 Feb 2018 16:01:19 -0600 Joshua Watt <[email protected]> wrote:

Fix up support for using Icecream to do distributed builds, which appears to have been broken for some time.

we were using icecream a year ago and we set ICECC_CARET_WORKAROUND env
variable to 0 (in set_icecc_env() in icecc.bbclass). The default is 1 and it
causes a local rebuild when any warning is emitted.

Quoted 5 lines

I don't have exact numbers, but the extreme case was opencv. There was so
many warnings that distributed build on ~10 threads was slower than local
build on 4 threads.

Ok, that make sense. I'll add a patch to default to ICECC_CARET_WORKAROUND to my patch series. Thanks for the heads up!

Quoted 67 lines

I know that it is some kind of configuration, but I would set the default to
0. Or do it at least configurable.

Quoted 6 lines

In addition, Icecream support can now be enabled in the SDK. When enabled, the SDK install process will check if the host supports icecc
and if so will construct a proper environment tarball and use it when
compiling.

Thanks for work on that, I'm keen to use it in our environment.

Tomas

Quoted 51 lines

Most of the changes should be straightforward. icecc-create-env has been almost completely re-written but the changes were broken up into
hopefully logical chunks.

The following changes since commit
385944254d0ef88bd6450a221a54cbcb40020b42:

tcmode-default.inc: drop preferred version of gzip-native (2018-
01-31 17:01:20 +0000)

are available in the Git repository at:

git://push.yoctoproject.org/poky-contrib jpew/icecream

Joshua Watt (14):
icecc.bbclass: Move to shared work directory icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage icecc.bbclass: Skip canadian-cross compiles
icecc-create-env: Reformat
icecc-create-env: Add /bin/true to environment
icecc-create-env: Use program interpreter for deps
icecc-create-env: Symlink alternate names
icecc-create-env: Archive directory
icecc-create-env: Fix executable rpaths
icecc-create-env: Add support for nativesdk
icecc-create-env: Allow multiple tool aliases
toolchain-shar-extract: Add post-relocate scripts
icecc-toolchain: Add SDK icecream setup icecc.bbclass: Add IceCream support to SDK

 meta/classes/icecc.bbclass                         |  37 ++-
 meta/classes/toolchain-scripts.bbclass             |  15 +
 meta/files/toolchain-shar-extract.sh               |   8 +
 meta/recipes-core/meta/meta-environment.bb         |   2 +
 .../icecc-create-env/icecc-create-env              | 328
++++++++++++++-------
 ...e-env-native_0.1.bb => icecc-create-env_0.1.bb} |   4 +-
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   |  37 +++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh |  43 +++
 .../nativesdk-icecc-toolchain_0.1.bb               |  38 +++
 9 files changed, 390 insertions(+), 122 deletions(-)
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-
native_0.1.bb => icecc-create-env_0.1.bb} (98%)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-
toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-
toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-

toolchain/nativesdk-icecc-toolchain_0.1.bb

#21 lore raw

Hi Joshua,

Quoted 15 lines · On Wed, 07 Feb 2018 15:19:12 -0600 Joshua Watt <[email protected]> wrote:
Quoted 14 lines · Tomas Novotny

Hi Joshua,

Quoted 2 lines · On Tue, 6 Feb 2018 16:01:19 -0600 Joshua Watt <[email protected]> wrote:

Fix up support for using Icecream to do distributed builds, which appears to have been broken for some time.

we were using icecream a year ago and we set ICECC_CARET_WORKAROUND env
variable to 0 (in set_icecc_env() in icecc.bbclass). The default is 1 and it
causes a local rebuild when any warning is emitted.

Quoted 8 lines
Quoted 5 lines

I don't have exact numbers, but the extreme case was opencv. There was so
many warnings that distributed build on ~10 threads was slower than local
build on 4 threads.

Ok, that make sense. I'll add a patch to default to ICECC_CARET_WORKAROUND to my patch series. Thanks for the heads up!

thanks a lot. Please note one smaller consequence. The warning message is retained, but the second line of warning containing "^" pointing to the exact position is not printed. I guess that this is ok for OE.

Tomas

Quoted 67 lines
Quoted 66 lines

I know that it is some kind of configuration, but I would set the default to
0. Or do it at least configurable.

Quoted 6 lines

In addition, Icecream support can now be enabled in the SDK. When enabled, the SDK install process will check if the host supports icecc
and if so will construct a proper environment tarball and use it when
compiling.

Thanks for work on that, I'm keen to use it in our environment.

Tomas

Quoted 51 lines

Most of the changes should be straightforward. icecc-create-env has been almost completely re-written but the changes were broken up into
hopefully logical chunks.

The following changes since commit
385944254d0ef88bd6450a221a54cbcb40020b42:

tcmode-default.inc: drop preferred version of gzip-native (2018-
01-31 17:01:20 +0000)

are available in the Git repository at:

git://push.yoctoproject.org/poky-contrib jpew/icecream

Joshua Watt (14):
icecc.bbclass: Move to shared work directory icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage icecc.bbclass: Skip canadian-cross compiles
icecc-create-env: Reformat
icecc-create-env: Add /bin/true to environment
icecc-create-env: Use program interpreter for deps
icecc-create-env: Symlink alternate names
icecc-create-env: Archive directory
icecc-create-env: Fix executable rpaths
icecc-create-env: Add support for nativesdk
icecc-create-env: Allow multiple tool aliases
toolchain-shar-extract: Add post-relocate scripts
icecc-toolchain: Add SDK icecream setup icecc.bbclass: Add IceCream support to SDK

 meta/classes/icecc.bbclass                         |  37 ++-
 meta/classes/toolchain-scripts.bbclass             |  15 +
 meta/files/toolchain-shar-extract.sh               |   8 +
 meta/recipes-core/meta/meta-environment.bb         |   2 +
 .../icecc-create-env/icecc-create-env              | 328
++++++++++++++-------
 ...e-env-native_0.1.bb => icecc-create-env_0.1.bb} |   4 +-
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   |  37 +++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh |  43 +++
 .../nativesdk-icecc-toolchain_0.1.bb               |  38 +++
 9 files changed, 390 insertions(+), 122 deletions(-)
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-
native_0.1.bb => icecc-create-env_0.1.bb} (98%)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-
toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-
toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-

toolchain/nativesdk-icecc-toolchain_0.1.bb

#22 lore raw

v2 cover Fix support for Icecream

In reply to Joshua Watt

Fix up support for using Icecream to do distributed builds, which appears to have been broken for some time.

In addition, Icecream support can now be enabled in the SDK. When enabled, the SDK install process will check if the host supports icecc and if so will construct a proper environment tarball and use it when compiling.

Most of the changes should be straightforward. icecc-create-env has been almost completely re-written but the changes were broken up into hopefully logical chunks.

V2:
* Default ICECC_CARET_WORKAROUND = "0" which greatly speeds up build times in recipes with a lot of warnings. (Thanks to Tomas Novotny <[email protected]>)
* Fix elfutils build when ICECC_CARET_WORKAROUND = "0"
* Minor fixes to commit messages and patchtest failures

The following changes since commit 44ea6ded0e8e73f5258ce55bd1b851e24243b76a:

bitbake: bitbake-user-manual: Commented out Placeholder Paragraph (2018-02-06 18:10:35 +0000)

are available in the Git repository at:

git://push.yoctoproject.org/poky-contrib jpew/icecream

Joshua Watt (17):
icecc.bbclass: Move to shared work directory icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage icecc.bbclass: Skip canadian-cross compiles
icecc-create-env: Reformat
icecc-create-env: Add /bin/true to environment
icecc-create-env: Use program interpreter for deps
icecc-create-env: Symlink alternate names
icecc-create-env: Archive directory
icecc-create-env: Fix executable rpaths
icecc-create-env: Add support for nativesdk
icecc-create-env: Allow multiple tool aliases
toolchain-shar-extract: Add post-relocate scripts
icecc-toolchain: Add SDK icecream setup icecc.bbclass: Add Icecream support to SDK icecc.bbclass: Disable caret workaround by default
icecc-toolchain: Disable caret workaround
elfutils: Use fallthrough attribute

 meta/classes/icecc.bbclass                         |  57 +-
 meta/classes/toolchain-scripts.bbclass             |  15 +
 meta/files/toolchain-shar-extract.sh               |   8 +
 meta/recipes-core/meta/meta-environment.bb         |   2 +
 meta/recipes-devtools/elfutils/elfutils_0.170.bb   |   4 +-
 .../files/0001-Use-fallthrough-attribute.patch     | 791 +++++++++++++++++++++
 .../elfutils/files/debian/fallthrough.patch        |  36 -
 .../icecc-create-env/icecc-create-env              | 328 ++++++---
 ...e-env-native_0.1.bb => icecc-create-env_0.1.bb} |   4 +-
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   |  49 ++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh |  43 ++
 .../nativesdk-icecc-toolchain_0.1.bb               |  38 +
 12 files changed, 1214 insertions(+), 161 deletions(-)
 create mode 100644 meta/recipes-devtools/elfutils/files/0001-Use-fallthrough-attribute.patch
 delete mode 100644 meta/recipes-devtools/elfutils/files/debian/fallthrough.patch
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb} (98%)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb
Signature · 1 line
2.14.3
#23 lore raw

PATCH v2 1/17 icecc.bbclass: Move to shared work directory

Generate the icecc toolchains in a shared work directory. This class was already setup to correctly synchronize creating the toolchains in a shared location before the RSS changes, so return to that behavior instead of generated the toolchains in each recipe's sysroot. Additionally, it makes no sense for each recipe to generate a toolchain, only to find it was already generated and uploaded to the compile server by another recipe.

---
 meta/classes/icecc.bbclass | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
meta/classes/icecc.bbclass +5 -2
index 1cc1c4ddb0f..e8725b582b3 100644
@@ -151,6 +151,9 @@ def icecc_is_native(bb, d):
         bb.data.inherits_class("cross", d) or \
         bb.data.inherits_class("native", d);
 
+def icecc_dir(bb, d):
+    return d.expand('${TMPDIR}/work-shared/ice')
+
 # Don't pollute allarch signatures with TARGET_FPU
 icecc_version[vardepsexclude] += "TARGET_FPU"
 def icecc_version(bb, d):
@@ -175,8 +178,8 @@ def icecc_version(bb, d):
             archive_name += "-kernel"
 
     import socket
-    ice_dir = d.expand('${STAGING_DIR_NATIVE}${prefix_native}')
-    tar_file = os.path.join(ice_dir, 'ice', archive_name + "-@VERSION@-" + socket.gethostname() + '.tar.gz')
+    ice_dir = icecc_dir(bb, d)
+    tar_file = os.path.join(ice_dir, archive_name + "-@VERSION@-" + socket.gethostname() + '.tar.gz')
 
     return tar_file
 
Signature · 1 line
2.14.3
#24 lore raw

PATCH v2 2/17 icecc.bbclass: Fix STAGING_BINDIR_TOOLCHAIN usage

In reply to Joshua Watt

STAGING_BINDIR_TOOLCHAIN is actually a path list, not a single path. Fix icecc.bbclass to try all the paths in the variable instead of treating it as a single path.

---
 meta/classes/icecc.bbclass | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
meta/classes/icecc.bbclass +8 -9
index e8725b582b3..35a1aaef86f 100644
@@ -210,15 +210,14 @@ def icecc_get_tool(bb, d, tool):
     else:
         ice_dir = d.expand('${STAGING_BINDIR_TOOLCHAIN}')
         target_sys = d.expand('${TARGET_SYS}')
-        tool_bin = os.path.join(ice_dir, "%s-%s" % (target_sys, tool))
-        if os.path.isfile(tool_bin):
-            return tool_bin
-        else:
-            external_tool_bin = icecc_get_external_tool(bb, d, tool)
-            if os.path.isfile(external_tool_bin):
-                return external_tool_bin
-            else:
-                return ""
+        for p in ice_dir.split(':'):
+            tool_bin = os.path.join(p, "%s-%s" % (target_sys, tool))
+            if os.path.isfile(tool_bin):
+                return tool_bin
+        external_tool_bin = icecc_get_external_tool(bb, d, tool)
+        if os.path.isfile(external_tool_bin):
+            return external_tool_bin
+        return ""
 
 def icecc_get_and_check_tool(bb, d, tool):
     # Check that g++ or gcc is not a symbolic link to icecc binary in
Signature · 1 line
2.14.3
#25 lore raw

PATCH v2 3/17 icecc.bbclass: Skip canadian-cross compiles

In reply to Joshua Watt

icecc.bbclass will no longer attempt to distribute cross-canadian compiles. While it is technically possible to generate a toolchain that runs on the build system and generates executables for the host system, this is not the normal way that icecc operates. There are so few of these recipes that it is probably not worth maintaining a distinct code path for them.

---
 meta/classes/icecc.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)
meta/classes/icecc.bbclass +6 -0
index 35a1aaef86f..9d5eaed57ce 100644
@@ -101,6 +101,9 @@ def use_icecc(bb,d):
     if icecc_is_allarch(bb, d):
         return "no"
 
+    if icecc_is_cross_canadian(bb, d):
+        return "no"
+
     pn = d.getVar('PN')
 
     system_class_blacklist = []
@@ -151,6 +154,9 @@ def icecc_is_native(bb, d):
         bb.data.inherits_class("cross", d) or \
         bb.data.inherits_class("native", d);
 
+def icecc_is_cross_canadian(bb, d):
+    return bb.data.inherits_class("cross-canadian", d)
+
 def icecc_dir(bb, d):
     return d.expand('${TMPDIR}/work-shared/ice')
 
Signature · 1 line
2.14.3
#26 lore raw

PATCH v2 4/17 icecc-create-env: Reformat

In reply to Joshua Watt

The environment script used an annoying mix of tabs and spaces and no mapping of tabs to spaces would produce pleasant indentation. Reformat to eliminate tab characters and settle on 4 spaces for indentation (which matches the upstream icecream script from which this is derived)

---
 .../icecc-create-env/icecc-create-env              | 208 ++++++++++-----------
 1 file changed, 104 insertions(+), 104 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +104 -104
index 7e4dbc414e7..723f1f9bd90 100755
@@ -8,63 +8,63 @@ target_files=
 
 is_contained ()
 {
-  case " $target_files " in
-    *" $1 "* ) return 0 ;;
-    *"=$1 "* ) return 0;;
-    * ) return 1 ;;
-  esac
+    case " $target_files " in
+        *" $1 "* ) return 0 ;;
+        *"=$1 "* ) return 0;;
+        * ) return 1 ;;
+    esac
 }
 
 add_file ()
 {
-  local name="$1"
-  local path="$1";
-  if test -n "$2"; then
-    name="$2"
-  fi
-  test -z "$name" && return
-  # ls -H isn't really the same as readlink, but
-  # readlink is not portable enough.
-  path=`ls -H $path`
-  toadd="$name=$path"
-  is_contained "$toadd" && return
-  if test -z "$silent"; then
-  echo "adding file $toadd"
-  fi
-  target_files="$target_files $toadd"
-  if test -x "$path"; then
-    # Only call ldd when it makes sense
-    if file -L "$path" | grep 'ELF' > /dev/null 2>&1; then
-	if ! file -L "$path" | grep 'static' > /dev/null 2>&1; then
-	   # ldd now outputs ld as /lib/ld-linux.so.xx on current nptl based glibc
-		# this regexp parse the outputs like:
-		# ldd /usr/bin/gcc
-		#         linux-gate.so.1 =>  (0xffffe000)
-		#         libc.so.6 => /lib/tls/libc.so.6 (0xb7e81000)
-		#         /lib/ld-linux.so.2 (0xb7fe8000)
-		# covering both situations ( with => and without )
-          for lib in `ldd "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
-	    test -f "$lib" || continue
-	    # Check wether the same library also exists in the parent directory,
-	    # and prefer that on the assumption that it is a more generic one.
-	    local baselib=`echo "$lib" | sed 's,\(/[^/]*\)/.*\(/[^/]*\)$,\1\2,'`
-	    test -f "$baselib" && lib=$baselib
-	  add_file "$lib"
-        done
-      fi
+    local name="$1"
+    local path="$1";
+    if test -n "$2"; then
+        name="$2"
+    fi
+    test -z "$name" && return
+    # ls -H isn't really the same as readlink, but
+    # readlink is not portable enough.
+    path=`ls -H $path`
+    toadd="$name=$path"
+    is_contained "$toadd" && return
+    if test -z "$silent"; then
+        echo "adding file $toadd"
+    fi
+    target_files="$target_files $toadd"
+    if test -x "$path"; then
+        # Only call ldd when it makes sense
+        if file -L "$path" | grep 'ELF' > /dev/null 2>&1; then
+            if ! file -L "$path" | grep 'static' > /dev/null 2>&1; then
+                # ldd now outputs ld as /lib/ld-linux.so.xx on current nptl based glibc
+                # this regexp parse the outputs like:
+                # ldd /usr/bin/gcc
+                #         linux-gate.so.1 =>  (0xffffe000)
+                #         libc.so.6 => /lib/tls/libc.so.6 (0xb7e81000)
+                #         /lib/ld-linux.so.2 (0xb7fe8000)
+                # covering both situations ( with => and without )
+                for lib in `ldd "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
+                    test -f "$lib" || continue
+                    # Check wether the same library also exists in the parent directory,
+                    # and prefer that on the assumption that it is a more generic one.
+                    local baselib=`echo "$lib" | sed 's,\(/[^/]*\)/.*\(/[^/]*\)$,\1\2,'`
+                    test -f "$baselib" && lib=$baselib
+                    add_file "$lib"
+                done
+            fi
+        fi
     fi
-  fi
 }
 
 # backward compat
 if test "$1" = "--respect-path"; then
-  shift
+    shift
 fi
 
 #add a --silent switch to avoid "broken pipe" errors when calling this scipt from within OE
 if test "$1" = "--silent"; then
- silent=1
- shift
+    silent=1
+    shift
 fi
 
 
@@ -77,18 +77,18 @@ shift
 archive_name=$1
 
 if test -z "$added_gcc" || test -z "$added_gxx" ; then
-	echo "usage: $0 <gcc_path> <g++_path>"
-	exit 1
+    echo "usage: $0 <gcc_path> <g++_path>"
+    exit 1
 fi
 
 if ! test -x "$added_gcc" ; then
-  echo "'$added_gcc' is no executable."
-  exit 1
+    echo "'$added_gcc' is no executable."
+    exit 1
 fi
 
 if ! test -x "$added_gxx" ; then
-  echo "'$added_gcc' is no executable."
-  exit 1
+    echo "'$added_gcc' is no executable."
+    exit 1
 fi
 
 
@@ -97,51 +97,51 @@ add_file $added_gcc /usr/bin/gcc
 add_file $added_gxx /usr/bin/g++
 
 if test -z "$added_as" ; then
- add_file /usr/bin/as /usr/bin/as
+    add_file /usr/bin/as /usr/bin/as
 else
- if ! test -x "$added_as" ; then
-  echo "'$added_as' is no executable."
-  exit 1
- fi
+    if ! test -x "$added_as" ; then
+        echo "'$added_as' is no executable."
+        exit 1
+    fi
 
- add_file $added_as  /usr/bin/as
+    add_file $added_as  /usr/bin/as
 fi
 
 add_file `$added_gcc -print-prog-name=cc1` /usr/bin/cc1
 add_file `$added_gxx -print-prog-name=cc1plus` /usr/bin/cc1plus
 specfile=`$added_gcc -print-file-name=specs`
 if test -n "$specfile" && test -e "$specfile"; then
-  add_file "$specfile"
+    add_file "$specfile"
 fi
 
 ltofile=`$added_gcc -print-prog-name=lto1`
 pluginfile="${ltofile%lto1}liblto_plugin.so"
 if test -r "$pluginfile"
 then
-  add_file $pluginfile  ${pluginfile#*usr}
-  add_file $pluginfile  /usr${pluginfile#*usr}
+    add_file $pluginfile  ${pluginfile#*usr}
+    add_file $pluginfile  /usr${pluginfile#*usr}
 fi
 
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
 new_target_files=
 for i in $target_files; do
- case $i in
-   *=/*)
-    target=`echo $i | cut -d= -f1`
-    path=`echo $i | cut -d= -f2`
-    ;;
-   *)
-    path=$i
-    target=$i
-    ;;
-  esac
-  mkdir -p $tempdir/`dirname $target`
-  cp -p $path $tempdir/$target
-  if test -f $tempdir/$target -a -x $tempdir/$target; then
-    strip -s $tempdir/$target 2>/dev/null
-  fi
-  target=`echo $target | cut -b2-`
-  new_target_files="$new_target_files $target"
+    case $i in
+        *=/*)
+            target=`echo $i | cut -d= -f1`
+            path=`echo $i | cut -d= -f2`
+            ;;
+        *)
+            path=$i
+            target=$i
+            ;;
+    esac
+    mkdir -p $tempdir/`dirname $target`
+    cp -p $path $tempdir/$target
+    if test -f $tempdir/$target -a -x $tempdir/$target; then
+        strip -s $tempdir/$target 2>/dev/null
+    fi
+    target=`echo $target | cut -b2-`
+    new_target_files="$new_target_files $target"
 done
 
 #sort the files
@@ -150,43 +150,43 @@ target_files=`for i in $new_target_files; do echo $i; done | sort`
 #test if an archive name was supplied
 #if not use the md5 of all files as the archive name
 if test -z "$archive_name"; then
-  md5sum=NONE
-  for file in /usr/bin/md5sum /bin/md5 /usr/bin/md5; do
-    if test -x $file; then
-      md5sum=$file
-      break
-    fi
-  done
-
-  #calculate md5 and use it as the archive name
-  archive_name=`for i in $target_files; do test -f $tempdir/$i && $md5sum $tempdir/$i; done | sed -e 's/ .*$//' | $md5sum | sed -e 's/ .*$//'`.tar.gz || {
-    if test -z "$silent"; then
-     echo "Couldn't compute MD5 sum."
-    fi
-    exit 2
-  }
-  mydir=`pwd`
+    md5sum=NONE
+    for file in /usr/bin/md5sum /bin/md5 /usr/bin/md5; do
+        if test -x $file; then
+            md5sum=$file
+            break
+        fi
+    done
+
+    #calculate md5 and use it as the archive name
+    archive_name=`for i in $target_files; do test -f $tempdir/$i && $md5sum $tempdir/$i; done | sed -e 's/ .*$//' | $md5sum | sed -e 's/ .*$//'`.tar.gz || {
+        if test -z "$silent"; then
+            echo "Couldn't compute MD5 sum."
+        fi
+        exit 2
+    }
+    mydir=`pwd`
 else
-  mydir="`dirname "$archive_name"`"
+    mydir="`dirname "$archive_name"`"
 
-  #check if we have a full path or only a filename
-  if test "$mydir" = "." ; then
-    mydir=`pwd`
-  else
-    mydir=""
-  fi
+    #check if we have a full path or only a filename
+    if test "$mydir" = "." ; then
+        mydir=`pwd`
+    else
+        mydir=""
+    fi
 fi
 
 if test -z "$silent"; then
-echo "creating $archive_name"
+    echo "creating $archive_name"
 fi
 
 cd $tempdir
 tar -czhf "$mydir/$archive_name" $target_files || {
- if test -z "$silent"; then
-  echo "Couldn't create archive"
- fi
-  exit 3
+    if test -z "$silent"; then
+        echo "Couldn't create archive"
+    fi
+    exit 3
 }
 cd ..
 rm -rf $tempdir
Signature · 1 line
2.14.3
#27 lore raw

PATCH v2 5/17 icecc-create-env: Add /bin/true to environment

In reply to Joshua Watt

icecream daemons execute /bin/true from the environment as a check to determine if the environment is valid at all, so it needs to be included.

---
 .../icecc-create-env/icecc-create-env/icecc-create-env         | 10 ++++++++++
 1 file changed, 10 insertions(+)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +10 -0
index 723f1f9bd90..c838256f748 100755
@@ -122,6 +122,16 @@ then
     add_file $pluginfile  /usr${pluginfile#*usr}
 fi
 
+# for testing the environment is usable at all
+if test -x /bin/true; then
+    add_file /bin/true
+elif test -x /usr/bin/true; then
+    add_file /usr/bin/true /bin/true
+else
+    echo "'true' not found"
+    exit 1
+fi
+
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
 new_target_files=
 for i in $target_files; do
Signature · 1 line
2.14.3
#28 lore raw

PATCH v2 6/17 icecc-create-env: Use program interpreter for deps

In reply to Joshua Watt

ldd cannot always be used to determine a program's dependencies correctly, particularly when the program specifies an alternate program interpreter (dynamic loader). This commonly happens when using a uninative tarball. Instead, determine the program's requested interpreter, and ask it to list the dependencies.

---
 .../icecc-create-env/icecc-create-env                | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +17 -3
index c838256f748..7636d090a48 100755
@@ -6,6 +6,12 @@
 
 target_files=
 
+is_dynamic_elf ()
+{
+    # Is the file an dynamically linked ELF executable?
+    (file -L "$1" | grep 'ELF' > /dev/null 2>&1) && (! file -L "$1" | grep 'static' > /dev/null 2>&1)
+}
+
 is_contained ()
 {
     case " $target_files " in
@@ -34,8 +40,16 @@ add_file ()
     target_files="$target_files $toadd"
     if test -x "$path"; then
         # Only call ldd when it makes sense
-        if file -L "$path" | grep 'ELF' > /dev/null 2>&1; then
-            if ! file -L "$path" | grep 'static' > /dev/null 2>&1; then
+        if is_dynamic_elf "$path"; then
+            # Request the program interpeter (dynamic loader)
+            interp=`readelf -w -l "$path" | grep "Requesting program interpreter:" | sed "s/\s*\[Requesting program interpreter:\s*\(.*\)\]/\1/g"`
+
+            if test -n "$interp" && test -x "$interp"; then
+                # Use the dynamic loaders --list argument to list the
+                # depenencies. The program may have a a different program
+                # interpeter (typical when using uninative tarballs), which is
+                # why we can't just call ldd.
+                #
                 # ldd now outputs ld as /lib/ld-linux.so.xx on current nptl based glibc
                 # this regexp parse the outputs like:
                 # ldd /usr/bin/gcc
@@ -43,7 +57,7 @@ add_file ()
                 #         libc.so.6 => /lib/tls/libc.so.6 (0xb7e81000)
                 #         /lib/ld-linux.so.2 (0xb7fe8000)
                 # covering both situations ( with => and without )
-                for lib in `ldd "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
+                for lib in `$interp --list "$path" | sed -n 's,^[^/]*\(/[^ ]*\).*,\1,p'`; do
                     test -f "$lib" || continue
                     # Check wether the same library also exists in the parent directory,
                     # and prefer that on the assumption that it is a more generic one.
Signature · 1 line
2.14.3
#29 lore raw

PATCH v2 7/17 icecc-create-env: Symlink alternate names

In reply to Joshua Watt

Instead of renaming files to a new path in the toolchain archive, keep the files with their original paths and create a relative symbolic link from the new path to the original file.

---
 .../icecc-create-env/icecc-create-env              | 64 +++++++++++++++++++---
 1 file changed, 57 insertions(+), 7 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +57 -7
index 7636d090a48..0791bd54b27 100755
@@ -21,10 +21,28 @@ is_contained ()
     esac
 }
 
+normalize_path ()
+{
+    # Normalizes the path to a file or directory, removing all "." and ".."
+    # entries. Use pwd -L to explicitly prevent symlink expansion
+    local path=$1
+    if test -f "$path"; then
+        pushd $(dirname $path) > /dev/null 2>&1
+        dir_path=$(pwd -L)
+        path=$dir_path/$(basename $path)
+        popd > /dev/null 2>&1
+    elif test -d "$path"; then
+        pushd $path > /dev/null 2>&1
+        path=$(pwd -L)
+        popd > /dev/null 2>&1
+    fi
+    echo $path
+}
+
 add_file ()
 {
-    local name="$1"
-    local path="$1";
+    local path=`normalize_path $1`
+    local name="$path"
     if test -n "$2"; then
         name="$2"
     fi
@@ -129,7 +147,7 @@ if test -n "$specfile" && test -e "$specfile"; then
 fi
 
 ltofile=`$added_gcc -print-prog-name=lto1`
-pluginfile="${ltofile%lto1}liblto_plugin.so"
+pluginfile=`normalize_path "${ltofile%lto1}liblto_plugin.so"`
 if test -r "$pluginfile"
 then
     add_file $pluginfile  ${pluginfile#*usr}
@@ -146,6 +164,19 @@ else
     exit 1
 fi
 
+link_rel ()
+{
+    local target="$1"
+    local name="$2"
+    local base="$3"
+
+    local prefix=`dirname $name`
+
+    prefix=`echo $prefix | sed 's,[^/]\+,..,g' | sed 's,^/*,,g'`
+
+    ln -s $prefix/$target $base/$name
+}
+
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
 new_target_files=
 for i in $target_files; do
@@ -159,11 +190,30 @@ for i in $target_files; do
             target=$i
             ;;
     esac
-    mkdir -p $tempdir/`dirname $target`
-    cp -p $path $tempdir/$target
-    if test -f $tempdir/$target -a -x $tempdir/$target; then
-        strip -s $tempdir/$target 2>/dev/null
+    if test "$target" == "$path"; then
+        mkdir -p $tempdir/`dirname $target`
+        cp -pH $target $tempdir/$target
+
+        if test -f $tempdir/$target -a -x $tempdir/$target; then
+            strip -s $tempdir/$target 2>/dev/null
+        fi
+    else
+        mkdir -p $tempdir/`dirname $path`
+        cp -pH $path $tempdir/$path
+
+        mkdir -p $tempdir/`dirname $target`
+        # Relative links are used because the files are checked for being
+        # executable outside the chroot
+        link_rel $path $target $tempdir
+
+        if test -f $tempdir/$path -a -x $tempdir/$path; then
+            strip -s $tempdir/$path 2>/dev/null
+        fi
+
+        path=`echo $path | cut -b2-`
+        new_target_files="$new_target_files $path"
     fi
+
     target=`echo $target | cut -b2-`
     new_target_files="$new_target_files $target"
 done
Signature · 1 line
2.14.3
#30 lore raw

PATCH v2 8/17 icecc-create-env: Archive directory

In reply to Joshua Watt

Taring up the toolchain is now done by adding the entire working directory, instead of listing all the files individually. This is done because the list of files may contain ".." entries, which tar does not like and strips out, resulting in bad archives. This should result in an identical archive to what was previously generated.

In addition, symbolic links are no longer dereferenced when creating the archive, as they are purposely included to provide alternate names for files

---
 .../icecc-create-env/icecc-create-env/icecc-create-env              | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +5 -1
index 0791bd54b27..426b093d91c 100755
@@ -256,7 +256,11 @@ if test -z "$silent"; then
 fi
 
 cd $tempdir
-tar -czhf "$mydir/$archive_name" $target_files || {
+# Add everything in the temp directory. Tar doesn't like to be given files with
+# ".." in them, which frequently happens in $target_files, and will strip off
+# the path prefix past the offending "..". This makes the archive generate
+# incorrectly
+tar -czf "$mydir/$archive_name" . || {
     if test -z "$silent"; then
         echo "Couldn't create archive"
     fi
Signature · 1 line
2.14.3
#31 lore raw

PATCH v2 9/17 icecc-create-env: Fix executable rpaths

In reply to Joshua Watt

Executables in the toolchain archive occasionally contain runtime library search paths (RPATH) that use the $ORIGIN placeholder. However, in order for that placeholder to work, /proc must be mounted. When iceccd executes the toolchain in the chroot environment, it doesn't mount /proc, so it is unable to resolve $ORIGIN resulting in a failure to find dynamic libraries.

The fix is to replace $ORIGIN in executable RPATH entries with the known chroot executable path. In order for this to work, the actual real path to the executable must be resolved to remove any symlinks, otherwise the calculate $ORIGIN replacement will be wrong. This is done by using "readlink -f", which is an acceptable dependency because Yocto already requires it.

---
 .../icecc-create-env/icecc-create-env              | 40 ++++++++++++++++++++--
 1 file changed, 37 insertions(+), 3 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +37 -3
index 426b093d91c..90d249df9f3 100755
@@ -12,6 +12,26 @@ is_dynamic_elf ()
     (file -L "$1" | grep 'ELF' > /dev/null 2>&1) && (! file -L "$1" | grep 'static' > /dev/null 2>&1)
 }
 
+fix_rpath ()
+{
+    # Patches the RPATH for a file. When the program is executed in the chroot
+    # be iceccd, /proc is not mounted. As such, $ORIGIN can't be resolved. To
+    # work around this, replace all instances of $ORIGIN in RPATH with the
+    # known chroot path to the executables directory
+    local path="$1"
+    local origin="$2"
+    if ! is_dynamic_elf "$path"; then
+        return
+    fi
+    local new_rpath="`readelf -w -d "$path" | grep RPATH | \
+        sed 's/.*\[\(.*\)\]/\1/g' | \
+        sed "s,\\\$ORIGIN,/$origin,g"`"
+
+    if test -n "$new_rpath"; then
+        $PATCHELF --set-rpath "$new_rpath" "$path"
+    fi
+}
+
 is_contained ()
 {
     case " $target_files " in
@@ -47,9 +67,8 @@ add_file ()
         name="$2"
     fi
     test -z "$name" && return
-    # ls -H isn't really the same as readlink, but
-    # readlink is not portable enough.
-    path=`ls -H $path`
+    # readlink is required for Yocto, so we can use it
+    path=`readlink -f "$path"`
     toadd="$name=$path"
     is_contained "$toadd" && return
     if test -z "$silent"; then
@@ -108,6 +127,17 @@ added_as=$1
 shift
 archive_name=$1
 
+if test -z "$PATCHELF"; then
+    PATCHELF=`which patchelf 2> /dev/null`
+fi
+if test -z "$PATCHELF"; then
+    PATCHELF=`which patchelf-uninative 2> /dev/null`
+fi
+if test -z "$PATCHELF"; then
+    echo "patchelf is required"
+    exit 1
+fi
+
 if test -z "$added_gcc" || test -z "$added_gxx" ; then
     echo "usage: $0 <gcc_path> <g++_path>"
     exit 1
@@ -197,6 +227,8 @@ for i in $target_files; do
         if test -f $tempdir/$target -a -x $tempdir/$target; then
             strip -s $tempdir/$target 2>/dev/null
         fi
+
+        fix_rpath $tempdir/$target `dirname $target`
     else
         mkdir -p $tempdir/`dirname $path`
         cp -pH $path $tempdir/$path
@@ -210,6 +242,8 @@ for i in $target_files; do
             strip -s $tempdir/$path 2>/dev/null
         fi
 
+        fix_rpath $tempdir/$path `dirname $path`
+
         path=`echo $path | cut -b2-`
         new_target_files="$new_target_files $path"
     fi
Signature · 1 line
2.14.3
#32 lore raw

PATCH v2 10/17 icecc-create-env: Add support for nativesdk

In reply to Joshua Watt

icecc-create-env can now be built as a nativesdk recipe, allowing the script to be included as part of an SDK

---
 .../{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb}       | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-devtools/icecc-create-env/{icecc-create-env-native_0.1.bb => icecc-create-env_0.1.bb} (98%)
meta/recipes-devtools/icecc-create-env/icecc-create-env_0.1.bb renamed from meta/recipes-devtools/icecc-create-env/icecc-create-env-native_0.1.bb +2 -2
similarity index 98%
rename from meta/recipes-devtools/icecc-create-env/icecc-create-env-native_0.1.bb
rename to meta/recipes-devtools/icecc-create-env/icecc-create-env_0.1.bb
index f8f4aef3923..053945ebbc6 100644
@@ -12,8 +12,6 @@ PR = "r2"
 DEPENDS = ""
 INHIBIT_DEFAULT_DEPS = "1"
 
-inherit native
-
 # This is needed, because otherwise there is dependency loop from quilt-native
 # Dependency loop #1 found:
 #  Task 10907 (meta/recipes-devtools/quilt/quilt-native_0.60.bb, do_install) (dependent Tasks ['quilt-native, do_compile'])
@@ -34,3 +32,5 @@ do_install() {
     install -d ${D}/${bindir}
     install -m 0755 ${WORKDIR}/icecc-create-env ${D}/${bindir}
 }
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.14.3
#33 lore raw

PATCH v2 11/17 icecc-create-env: Allow multiple tool aliases

In reply to Joshua Watt

When files are added to the environment, multiple aliases can be given for the file (by calling add_path multiple times with a second argument). All of these names will end up with a symlink to the original file.

---
 .../icecc-create-env/icecc-create-env              | 110 ++++++++++-----------
 1 file changed, 54 insertions(+), 56 deletions(-)
meta/recipes-devtools/icecc-create-env/icecc-create-env/icecc-create-env +54 -56
index 90d249df9f3..537e38a9ba0 100755
@@ -4,7 +4,8 @@
 # Copyright (C) 2004 by the Icecream Authors
 # GPL
 
-target_files=
+target_paths=
+target_aliases=
 
 is_dynamic_elf ()
 {
@@ -32,15 +33,33 @@ fix_rpath ()
     fi
 }
 
-is_contained ()
+add_path ()
 {
-    case " $target_files " in
-        *" $1 "* ) return 0 ;;
-        *"=$1 "* ) return 0;;
-        * ) return 1 ;;
+    case " $target_paths " in
+        *" $1 "*)
+            return 1
+            ;;
+        *)
+            target_paths="$target_paths $1"
+            return 0
+            ;;
     esac
 }
 
+add_alias ()
+{
+    if test "$1" != "$2"; then
+        local alias="$1=$2"
+        case " $target_aliases " in
+            *" $alias "*)
+                ;;
+            *)
+                target_aliases="$target_aliases $alias"
+                ;;
+        esac
+    fi
+}
+
 normalize_path ()
 {
     # Normalizes the path to a file or directory, removing all "." and ".."
@@ -61,20 +80,17 @@ normalize_path ()
 
 add_file ()
 {
-    local path=`normalize_path $1`
-    local name="$path"
-    if test -n "$2"; then
-        name="$2"
-    fi
-    test -z "$name" && return
+    local p=`normalize_path $1`
     # readlink is required for Yocto, so we can use it
-    path=`readlink -f "$path"`
-    toadd="$name=$path"
-    is_contained "$toadd" && return
-    if test -z "$silent"; then
-        echo "adding file $toadd"
+    local path=`readlink -f "$p"`
+
+    add_alias "$path" "$p"
+    if test -n "$2"; then
+        add_alias "$path" "$2"
     fi
-    target_files="$target_files $toadd"
+
+    add_path "$path" || return
+
     if test -x "$path"; then
         # Only call ldd when it makes sense
         if is_dynamic_elf "$path"; then
@@ -208,52 +224,34 @@ link_rel ()
 }
 
 tempdir=`mktemp -d /tmp/iceccenvXXXXXX`
-new_target_files=
-for i in $target_files; do
-    case $i in
-        *=/*)
-            target=`echo $i | cut -d= -f1`
-            path=`echo $i | cut -d= -f2`
-            ;;
-        *)
-            path=$i
-            target=$i
-            ;;
-    esac
-    if test "$target" == "$path"; then
-        mkdir -p $tempdir/`dirname $target`
-        cp -pH $target $tempdir/$target
-
-        if test -f $tempdir/$target -a -x $tempdir/$target; then
-            strip -s $tempdir/$target 2>/dev/null
-        fi
-
-        fix_rpath $tempdir/$target `dirname $target`
-    else
-        mkdir -p $tempdir/`dirname $path`
-        cp -pH $path $tempdir/$path
+target_files=
+for path in $target_paths; do
+    mkdir -p $tempdir/`dirname $path`
+    cp -pH $path $tempdir/$path
 
-        mkdir -p $tempdir/`dirname $target`
-        # Relative links are used because the files are checked for being
-        # executable outside the chroot
-        link_rel $path $target $tempdir
+    if test -f $tempdir/$path -a -x $tempdir/$path; then
+        strip -s $tempdir/$path 2>/dev/null
+    fi
 
-        if test -f $tempdir/$path -a -x $tempdir/$path; then
-            strip -s $tempdir/$path 2>/dev/null
-        fi
+    fix_rpath $tempdir/$path `dirname $path`
+    target_files="$target_files $path"
+done
 
-        fix_rpath $tempdir/$path `dirname $path`
+for i in $target_aliases; do
+    target=`echo $i | cut -d= -f1`
+    link_name=`echo $i | cut -d= -f2`
 
-        path=`echo $path | cut -b2-`
-        new_target_files="$new_target_files $path"
-    fi
+    mkdir -p $tempdir/`dirname $link_name`
+    # Relative links are used because the files are checked for being
+    # executable outside the chroot
+    link_rel $target $link_name $tempdir
 
-    target=`echo $target | cut -b2-`
-    new_target_files="$new_target_files $target"
+    link_name=`echo $link_name | cut -b2-`
+    target_files="$target_files $link_name"
 done
 
 #sort the files
-target_files=`for i in $new_target_files; do echo $i; done | sort`
+target_files=`for i in $target_files; do echo $i; done | sort`
 
 #test if an archive name was supplied
 #if not use the md5 of all files as the archive name
Signature · 1 line
2.14.3
#34 lore raw

PATCH v2 12/17 toolchain-shar-extract: Add post-relocate scripts

In reply to Joshua Watt

Recipes can now install post-relocation scripts which will be run when the SDK is installed.

---
 meta/classes/toolchain-scripts.bbclass     | 15 +++++++++++++++
 meta/files/toolchain-shar-extract.sh       |  8 ++++++++
 meta/recipes-core/meta/meta-environment.bb |  2 ++
 3 files changed, 25 insertions(+)
meta/classes/toolchain-scripts.bbclass +15 -0
index 9bcfe708c7c..eeb320d15a8 100644
@@ -116,6 +116,21 @@ fi
 EOF
 }
 
+toolchain_create_post_relocate_script() {
+	script=$1
+	rm -f $script
+	touch $script
+
+    cat >> $script <<EOF
+if [ -d "${SDKPATHNATIVE}/post-relocate-setup.d/" ]; then
+    for s in ${SDKPATHNATIVE}/post-relocate-setup.d/*.sh; do
+        \$s "\$1"
+    done
+    rm -rf "${SDKPATHNATIVE}/post-relocate-setup.d"
+fi
+EOF
+}
+
 #we get the cached site config in the runtime
 TOOLCHAIN_CONFIGSITE_NOCACHE = "${@siteinfo_get_files(d)}"
 TOOLCHAIN_CONFIGSITE_SYSROOTCACHE = "${STAGING_DIR}/${MLPREFIX}${MACHINE}/${target_datadir}/${TARGET_SYS}_config_site.d"
meta/files/toolchain-shar-extract.sh +8 -0
index 91804ec2818..f7ab43066fb 100644
@@ -259,6 +259,14 @@ if [ $savescripts = 0 ] ; then
 	$SUDO_EXEC rm -f ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh
 fi
 
+# Execute post-relocation script
+post_relocate="$target_sdk_dir/post-relocate-setup.sh"
+if [ -e "$post_relocate" ]; then
+	$SUDO_EXEC sed -e "s:@SDKPATH@:$target_sdk_dir:g" -i $post_relocate
+	$SUDO_EXEC /bin/sh $post_relocate "$target_sdk_dir" "@SDKPATH@"
+	$SUDO_EXEC rm -f $post_relocate
+fi
+
 echo "SDK has been successfully set up and is ready to be used."
 echo "Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g."
 for env_setup_script in `ls $target_sdk_dir/environment-setup-*`; do
meta/recipes-core/meta/meta-environment.bb +2 -0
index 29da121a3a9..09f757a087c 100644
@@ -55,6 +55,8 @@ create_sdk_files() {
 
 	# Add version information
 	toolchain_create_sdk_version ${SDK_OUTPUT}/${SDKPATH}/version-${REAL_MULTIMACH_TARGET_SYS}
+
+	toolchain_create_post_relocate_script ${SDK_OUTPUT}/${SDKPATH}/post-relocate-setup.sh
 }
 
 do_install() {
Signature · 1 line
2.14.3
#35 lore raw

PATCH v2 13/17 icecc-toolchain: Add SDK icecream setup

In reply to Joshua Watt

Icecream can now be optionally included in the generated SDK by including nativesdk-icecc-toolchain to TOOLCHAIN_HOST_TASK. When the SDK is installed a post-relocation script will check if icecc exists and if so will generate the toolchain environment.

---
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   | 37 +++++++++++++++++++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh | 43 ++++++++++++++++++++++
 .../nativesdk-icecc-toolchain_0.1.bb               | 38 +++++++++++++++++++
 3 files changed, 118 insertions(+)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +37 -0
new file mode 100644
index 00000000000..9c9bb4dc4ea
@@ -0,0 +1,37 @@
+#! /bin/sh
+# Copyright (c) 2018 Joshua Watt, Garmin International,Inc.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+if [ -z "$ICECC_PATH" ]; then
+    ICECC_PATH=$(which icecc 2> /dev/null)
+fi
+
+if [ -n "$ICECC_PATH" ]; then
+    export ICECC_PATH
+    export ICECC_VERSION="$OECORE_NATIVE_SYSROOT/usr/share/icecream/@TOOLCHAIN_ENV@"
+    export ICECC="$(which ${CROSS_COMPILE}gcc)"
+    export ICECXX="$(which ${CROSS_COMPILE}g++)"
+    export ICEAS="$(which ${CROSS_COMPILE}as)"
+    export PATH="$OECORE_NATIVE_SYSROOT/usr/share/icecream/bin:$PATH"
+else
+    echo "Icecc not found. Disabling distributed compiling"
+fi
+
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +43 -0

43 lines not on this page · raw message

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb new file +38 -0
new file mode 100644
index 00000000000..91958d7fbcc
@@ -0,0 +1,38 @@
+# Copyright (c) 2018 Joshua Watt, Garmin International,Inc.
+# Released under the MIT license (see COPYING.MIT for the terms)
+SUMMARY = "Generates Icecream toolchain for SDK"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${WORKDIR}/icecc-env.sh;beginline=2;endline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c"
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI = "\
+    file://icecc-env.sh \
+    file://icecc-setup.sh \
+    "
+
+inherit nativesdk
+
+ENV_NAME="${DISTRO}-${TCLIBC}-${SDK_ARCH}-${TUNE_PKGARCH}-${DISTRO_VERSION}.tar.gz"
+
+do_compile() {
+}
+
+do_install() {
+    install -d ${D}${SDKPATHNATIVE}${datadir}/icecream/bin
+
+    install -d ${D}${SDKPATHNATIVE}/environment-setup.d/
+    install -m 0644 ${WORKDIR}/icecc-env.sh ${D}${SDKPATHNATIVE}/environment-setup.d/
+    sed -i ${D}${SDKPATHNATIVE}/environment-setup.d/icecc-env.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+
+    install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
+    install -m 0755 ${WORKDIR}/icecc-setup.sh ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
+    sed -i ${D}${SDKPATHNATIVE}/post-relocate-setup.d/icecc-setup.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+}
+
+PACKAGES = "${PN}"
+FILES_${PN} = "${SDKPATHNATIVE}"
+RDEPENDS_${PN} += "nativesdk-icecc-create-env"
+
Signature · 1 line
2.14.3
#36 lore raw

PATCH v2 14/17 icecc.bbclass: Add Icecream support to SDK

In reply to Joshua Watt

If icecc is inherited, generated SDKs will automatically have optional support for compiling using the Icecream distributed compiler

---
 meta/classes/icecc.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)
meta/classes/icecc.bbclass +7 -0
index 9d5eaed57ce..8492ece60c3 100644
@@ -346,3 +346,10 @@ do_compile_kernelmodules_prepend() {
 do_install_prepend() {
     set_icecc_env
 }
+
+# IceCream is not (currently) supported in the extensible SDK
+ICECC_SDK_HOST_TASK = "nativesdk-icecc-toolchain"
+ICECC_SDK_HOST_TASK_task-populate-sdk-ext = ""
+
+# Add the toolchain scripts to the SDK
+TOOLCHAIN_HOST_TASK_append = " ${ICECC_SDK_HOST_TASK}"
Signature · 1 line
2.14.3
#37 lore raw

PATCH v2 15/17 icecc.bbclass: Disable caret workaround by default

In reply to Joshua Watt

Icecream has a behavior that causes it to recompile files locally if gcc generates any warnings or errors. The reason for this is that GCC tries to re-read the input file in order to display the offending line with a caret below it, which doesn't work in the remote chroot.

Default to disabling this this workaround and add -fno-diagnostics-show-caret to the GCC flags so that errors and warnings generated by GCC do not show erroneous results.

---
 meta/classes/icecc.bbclass | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
meta/classes/icecc.bbclass +19 -1
index 8492ece60c3..e8f7eab1bc1 100644
@@ -28,10 +28,23 @@
 #Error checking is kept to minimum so double check any parameters you pass to the class
 ###########################################################################################
 
-BB_HASHBASE_WHITELIST += "ICECC_PARALLEL_MAKE ICECC_DISABLED ICECC_USER_PACKAGE_BL ICECC_USER_CLASS_BL ICECC_USER_PACKAGE_WL ICECC_PATH ICECC_ENV_EXEC"
+BB_HASHBASE_WHITELIST += "ICECC_PARALLEL_MAKE ICECC_DISABLED ICECC_USER_PACKAGE_BL \
+    ICECC_USER_CLASS_BL ICECC_USER_PACKAGE_WL ICECC_PATH ICECC_ENV_EXEC \
+    ICECC_CARET_WORKAROUND ICECC_CFLAGS"
 
 ICECC_ENV_EXEC ?= "${STAGING_BINDIR_NATIVE}/icecc-create-env"
 
+# Default to disabling the caret workaround, If set to "1" in local.conf, icecc
+# will locally recompile any files that have warnings, which can adversely
+# affect performance.
+#
+# See: https://github.com/icecc/icecream/issues/190
+export ICECC_CARET_WORKAROUND ??= "0"
+
+ICECC_CFLAGS = ""
+CFLAGS += "${ICECC_CFLAGS}"
+CXXFLAGS += "${ICECC_CFLAGS}"
+
 def icecc_dep_prepend(d):
     # INHIBIT_DEFAULT_DEPS doesn't apply to the patch command.  Whether or  not
     # we need that built is the responsibility of the patch function / class, not
@@ -170,6 +183,11 @@ def icecc_version(bb, d):
     if not d.getVar('PARALLEL_MAKE', False) == "" and parallel:
         d.setVar("PARALLEL_MAKE", parallel)
 
+    # Disable showing the caret in the GCC compiler output if the workaround is
+    # disabled
+    if d.getVar('ICECC_CARET_WORKAROUND', True) == '0':
+        d.setVar('ICECC_CFLAGS', '-fno-diagnostics-show-caret')
+
     if icecc_is_native(bb, d):
         archive_name = "local-host-env"
     elif d.expand('${HOST_PREFIX}') == "":
Signature · 1 line
2.14.3
#38 lore raw

PATCH v2 16/17 icecc-toolchain: Disable caret workaround

In reply to Joshua Watt

Icecream has a behavior that causes it to recompile files locally if gcc generates any warnings or errors. The reason for this is that GCC tries to re-read the input file in order to display the offending line with a caret below it, which doesn't work in the remote chroot.

Default to disabling this this workaround and add -fno-diagnostics-show-caret to the GCC flags so that errors and warnings generated by GCC do not show erroneous results. Users can override this default in the SDK by defining ICECC_CARET_WORKAROUND="1" either before or after sourcing the SDK environment.

---
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh           | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh +13 -1
index 9c9bb4dc4ea..94760076bfa 100644
@@ -25,7 +25,19 @@ if [ -z "$ICECC_PATH" ]; then
 fi
 
 if [ -n "$ICECC_PATH" ]; then
-    export ICECC_PATH
+    # Default to disabling the caret workaround. If set to "1", icecc will
+    # locally recompile any files that have warnings, which can adversely
+    # affect performance.
+    #
+    # See: https://github.com/icecc/icecream/issues/190
+    if [ -z "$ICECC_CARET_WORKAROUND" ]; then
+        ICECC_CARET_WORKAROUND="0"
+    fi
+    if [ "$ICECC_CARET_WORKAROUND" != "1" ]; then
+        CFLAGS="$CFLAGS -fno-diagnostics-show-caret"
+        CXXFLAGS="$CXXFLAGS -fno-diagnostics-show-caret"
+    fi
+    export ICECC_PATH ICECC_CARET_WORKAROUND
     export ICECC_VERSION="$OECORE_NATIVE_SYSROOT/usr/share/icecream/@TOOLCHAIN_ENV@"
     export ICECC="$(which ${CROSS_COMPILE}gcc)"
     export ICECXX="$(which ${CROSS_COMPILE}g++)"
Signature · 1 line
2.14.3
#39 lore raw

PATCH v2 17/17 elfutils: Use fallthrough attribute

In reply to Joshua Watt

Patches elfutils to use the fallthrough attribute instead of comments to satisfy the -Wimplicit-fallthrough warning. Using comments is insufficient when compiling remotely with Icecream because the file gets pre-processed locally, removing the comments

---
 meta/recipes-devtools/elfutils/elfutils_0.170.bb   |   4 +-
 .../files/0001-Use-fallthrough-attribute.patch     | 791 +++++++++++++++++++++
 .../elfutils/files/debian/fallthrough.patch        |  36 -
 3 files changed, 793 insertions(+), 38 deletions(-)
 create mode 100644 meta/recipes-devtools/elfutils/files/0001-Use-fallthrough-attribute.patch
 delete mode 100644 meta/recipes-devtools/elfutils/files/debian/fallthrough.patch
meta/recipes-devtools/elfutils/elfutils_0.170.bb +2 -2
index 3b81e287b01..9242aea26b5 100644
@@ -32,8 +32,8 @@ SRC_URI += "\
         file://debian/0002-Add-support-for-mips64-abis-in-mips_retval.c.patch \
         file://debian/0003-Add-mips-n64-relocation-format-hack.patch \
 "
-# Fix the patches from Debian with GCC7
-SRC_URI += "file://debian/fallthrough.patch"
+# Fix fallthrough with GCC7
+SRC_URI += "file://0001-Use-fallthrough-attribute.patch"
 
 # The buildsystem wants to generate 2 .h files from source using a binary it just built,
 # which can not pass the cross compiling, so let's work around it by adding 2 .h files
meta/recipes-devtools/elfutils/files/0001-Use-fallthrough-attribute.patch new file +791 -0

791 lines not on this page · raw message

meta/recipes-devtools/elfutils/files/debian/fallthrough.patch deleted +0 -36
deleted file mode 100644
index b2623f9d2e5..00000000000
@@ -1,36 +0,0 @@
-GCC7 adds -Wimplicit-fallthrough to warn when a switch case falls through,
-however this causes warnings (which are promoted to errors) with the elfutils
-patches from Debian for mips and parisc, which use fallthrough's by design.
-
-Explicitly mark the intentional fallthrough switch cases with a comment to
-disable the warnings where the fallthrough behaviour is desired.
-
-Upstream-Status: Pending [debian]
-Signed-off-by: Joshua Lock <[email protected]>
-
-Index: elfutils-0.168/backends/parisc_retval.c
-===================================================================
---- elfutils-0.168.orig/backends/parisc_retval.c
-+++ elfutils-0.168/backends/parisc_retval.c
-@@ -166,7 +166,7 @@ parisc_return_value_location_ (Dwarf_Die
- 	  return nloc_intregpair;
- 
- 	/* Else fall through.  */
--      }
-+      } // fallthrough
- 
-     case DW_TAG_structure_type:
-     case DW_TAG_class_type:
-Index: elfutils-0.168/backends/mips_retval.c
-===================================================================
---- elfutils-0.168.orig/backends/mips_retval.c
-+++ elfutils-0.168/backends/mips_retval.c
-@@ -387,7 +387,7 @@ mips_return_value_location (Dwarf_Die *f
-               else
-                 return nloc_intregpair;
-             }
--        }
-+        } // fallthrough
- 
-       /* Fallthrough to handle large types */
- 
Signature · 1 line
2.14.3
#14 lore raw

Re: [PATCH 13/14] icecc-toolchain: Add SDK icecream setup

In reply to Joshua Watt

Is the LIC_FILES_CHKSUM correct? Doesn't match here:

nativesdk-icecc-toolchain/0.1-r0/icecc-env.sh;beginline=2;en dline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c
nativesdk-icecc-toolchain: The new md5 checksum is dd6b68c1efed8a9fb04e409b3b287d47
nativesdk-icecc-toolchain: Here is the selected license text:
vvvvvvvvvvvvvvvvvvvvvvvvvvvv beginline=2 vvvvvvvvvvvvvvvvvvvvvvvvvvvvv # Copyright (c) 2018 Joshua Watt, Garmin International,Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy

# of this software and associated documentation files (the "Software"), to
deal
# in the Software without restriction, including without limitation the
rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE
# SOFTWARE.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ endline=20 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 194 lines · Joshua Watt

IceCream can now be optionally included in the generated SDK by including nativesdk-icecc-toolchain to TOOLCHAIN_HOST_TASK. When the SDK is installed, it will check if icecc exists, and if so will generate the toolchain environment.

---
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   | 37
+++++++++++++++++++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh | 43
++++++++++++++++++++++
 .../nativesdk-icecc-toolchain_0.1.bb               | 38
+++++++++++++++++++
 3 files changed, 118 insertions(+)
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/
icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/icecc-toolchain/
icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-

toolchain_0.1.bb

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +4 -0
new file mode 100644
index 00000000000..4c34edbbc5d
@@ -0,0 +1,37 @@
+#! /bin/sh
+# Copyright 2018 Garmin Ltd. or its subsidiaries
+#
+# Permission is hereby granted, free of charge, to any person obtaining a

copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +4 -0
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included

in

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +3 -0
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS

OR

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +2 -0
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +2 -0
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +10 -0
+# SOFTWARE.
+#
+
+if [ -z "$ICECC_PATH" ]; then
+    ICECC_PATH=$(which icecc 2> /dev/null)
+fi
+
+if [ -n "$ICECC_PATH" ]; then
+    export ICECC_PATH
+    export ICECC_VERSION="$OECORE_NATIVE_SYSROOT/usr/share/icecream/@

TOOLCHAIN_ENV@"

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +8 -0
+    export ICECC="$(which ${CROSS_COMPILE}gcc)"
+    export ICECXX="$(which ${CROSS_COMPILE}g++)"
+    export ICEAS="$(which ${CROSS_COMPILE}as)"
+    export PATH="$OECORE_NATIVE_SYSROOT/usr/share/icecream/bin:$PATH"
+else
+    echo "Icecc not found. Disabling distributed compiling"
+fi
+
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +4 -0
new file mode 100644
index 00000000000..a433e55b029
@@ -0,0 +1,43 @@
+#! /bin/sh
+# Copyright 2018 Garmin Ltd. or its subsidiaries
+#
+# Permission is hereby granted, free of charge, to any person obtaining a

copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +4 -0
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included

in

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +3 -0
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS

OR

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +2 -0
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +2 -0
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +24 -0
+# SOFTWARE.
+#
+
+# Setup environment
+for env_setup_script in `ls $1/environment-setup-*`; do
+       . $env_setup_script
+done
+
+# ICECC_PATH will have been found icecc-env.sh
+if [ -z "$ICECC_PATH" ]; then
+    exit 0
+fi
+
+echo "Setting up IceCream distributed compiling..."
+
+# Create the environment
+mkdir -p "`dirname $ICECC_VERSION`"
+icecc-create-env $ICECC $ICECXX $ICEAS $ICECC_VERSION || exit $?
+
+# Create symbolic links
+d="$OECORE_NATIVE_SYSROOT/usr/share/icecream/bin"
+mkdir -p "$d"
+ln -s "$ICECC_PATH" "$d/${CROSS_COMPILE}gcc"
+ln -s "$ICECC_PATH" "$d/${CROSS_COMPILE}g++"
(quoted diff) +0 -0

toolchain_0.1.bb b/meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- toolchain_0.1.bb
new file mode 100644
index 00000000000..28b2924ebfb

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- new file +0 -0

toolchain_0.1.bb

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- new file +5 -0
@@ -0,0 +1,38 @@
+# Copyright (C) 2018 Joshua Watt <[email protected]>
+# Released under the MIT license (see COPYING.MIT for the terms)
+DESCRIPTION = "Generates IceCream toolchains for SDKs"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${WORKDIR}/icecc-env.

sh;beginline=2;endline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c"

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- new file +11 -0
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI = "\
+    file://icecc-env.sh \
+    file://icecc-setup.sh \
+    "
+
+inherit nativesdk
+
+ENV_NAME="${DISTRO}-${TCLIBC}-${SDK_ARCH}-${TUNE_PKGARCH}-$

{DISTRO_VERSION}.tar.gz"

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- new file +9 -0
+
+do_compile() {
+}
+
+do_install() {
+    install -d ${D}${SDKPATHNATIVE}${datadir}/icecream/bin
+
+    install -d ${D}${SDKPATHNATIVE}/environment-setup.d/
+    install -m 0644 ${WORKDIR}/icecc-env.sh ${D}${SDKPATHNATIVE}/

environment-setup.d/

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- new file +5 -0
+    sed -i ${D}${SDKPATHNATIVE}/environment-setup.d/icecc-env.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+
+    install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
+    install -m 0755 ${WORKDIR}/icecc-setup.sh ${D}${SDKPATHNATIVE}/post-

relocate-setup.d/

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc- new file +8 -1
+    sed -i ${D}${SDKPATHNATIVE}/post-relocate-setup.d/icecc-setup.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+}
+
+PACKAGES = "${PN}"
+FILES_${PN} = "${SDKPATHNATIVE}"
+RDEPENDS_${PN} += "nativesdk-icecc-create-env"
+
--

2.14.3

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#15 lore raw

Re: [PATCH 13/14] icecc-toolchain: Add SDK icecream setup

On Feb 16, 2018 17:44, "Martin Jansa" <[email protected]> wrote:

Is the LIC_FILES_CHKSUM correct? Doesn't match here:

No I must have updated the copyright and missed the checksum

nativesdk-icecc-toolchain/0.1-r0/icecc-env.sh;beginline=2;en dline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c
nativesdk-icecc-toolchain: The new md5 checksum is dd6b68c1efed8a9fb04e409b3b287d47
nativesdk-icecc-toolchain: Here is the selected license text:
vvvvvvvvvvvvvvvvvvvvvvvvvvvv beginline=2 vvvvvvvvvvvvvvvvvvvvvvvvvvvvv # Copyright (c) 2018 Joshua Watt, Garmin International,Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy

# of this software and associated documentation files (the "Software"), to
deal
# in the Software without restriction, including without limitation the
rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE
# SOFTWARE.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ endline=20 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 194 lines · Joshua Watt

IceCream can now be optionally included in the generated SDK by including nativesdk-icecc-toolchain to TOOLCHAIN_HOST_TASK. When the SDK is installed, it will check if icecc exists, and if so will generate the toolchain environment.

---
 .../icecc-toolchain/icecc-toolchain/icecc-env.sh   | 37
+++++++++++++++++++
 .../icecc-toolchain/icecc-toolchain/icecc-setup.sh | 43
++++++++++++++++++++++
 .../nativesdk-icecc-toolchain_0.1.bb               | 38
+++++++++++++++++++
 3 files changed, 118 insertions(+)
 create mode 100644 meta/recipes-devtools/icecc-to
olchain/icecc-toolchain/icecc-env.sh
 create mode 100644 meta/recipes-devtools/icecc-to
olchain/icecc-toolchain/icecc-setup.sh
 create mode 100644 meta/recipes-devtools/icecc-toolchain/

nativesdk-icecc-toolchain_0.1.bb

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +4 -0
new file mode 100644
index 00000000000..4c34edbbc5d
@@ -0,0 +1,37 @@
+#! /bin/sh
+# Copyright 2018 Garmin Ltd. or its subsidiaries
+#
+# Permission is hereby granted, free of charge, to any person obtaining a

copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +4 -0
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included

in

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +3 -0
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS

OR

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +2 -0
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +2 -0
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +10 -0
+# SOFTWARE.
+#
+
+if [ -z "$ICECC_PATH" ]; then
+    ICECC_PATH=$(which icecc 2> /dev/null)
+fi
+
+if [ -n "$ICECC_PATH" ]; then
+    export ICECC_PATH
+    export ICECC_VERSION="$OECORE_NATIVE_SYSROOT/usr/share/icecream/@TO

OLCHAIN_ENV@"

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-env.sh new file +8 -0
+    export ICECC="$(which ${CROSS_COMPILE}gcc)"
+    export ICECXX="$(which ${CROSS_COMPILE}g++)"
+    export ICEAS="$(which ${CROSS_COMPILE}as)"
+    export PATH="$OECORE_NATIVE_SYSROOT/usr/share/icecream/bin:$PATH"
+else
+    echo "Icecc not found. Disabling distributed compiling"
+fi
+
meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +4 -0
new file mode 100644
index 00000000000..a433e55b029
@@ -0,0 +1,43 @@
+#! /bin/sh
+# Copyright 2018 Garmin Ltd. or its subsidiaries
+#
+# Permission is hereby granted, free of charge, to any person obtaining a

copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +4 -0
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included

in

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +3 -0
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS

OR

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +2 -0
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +2 -0
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

meta/recipes-devtools/icecc-toolchain/icecc-toolchain/icecc-setup.sh new file +24 -0
+# SOFTWARE.
+#
+
+# Setup environment
+for env_setup_script in `ls $1/environment-setup-*`; do
+       . $env_setup_script
+done
+
+# ICECC_PATH will have been found icecc-env.sh
+if [ -z "$ICECC_PATH" ]; then
+    exit 0
+fi
+
+echo "Setting up IceCream distributed compiling..."
+
+# Create the environment
+mkdir -p "`dirname $ICECC_VERSION`"
+icecc-create-env $ICECC $ICECXX $ICEAS $ICECC_VERSION || exit $?
+
+# Create symbolic links
+d="$OECORE_NATIVE_SYSROOT/usr/share/icecream/bin"
+mkdir -p "$d"
+ln -s "$ICECC_PATH" "$d/${CROSS_COMPILE}gcc"
+ln -s "$ICECC_PATH" "$d/${CROSS_COMPILE}g++"
(quoted diff) +0 -0

chain_0.1.bb b/meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool chain_0.1.bb
new file mode 100644
index 00000000000..28b2924ebfb

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool new file +0 -0

chain_0.1.bb

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool new file +5 -0
@@ -0,0 +1,38 @@
+# Copyright (C) 2018 Joshua Watt <[email protected]>
+# Released under the MIT license (see COPYING.MIT for the terms)
+DESCRIPTION = "Generates IceCream toolchains for SDKs"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${WORKDIR}/icecc-env.s

h;beginline=2;endline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c"

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool new file +11 -0
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI = "\
+    file://icecc-env.sh \
+    file://icecc-setup.sh \
+    "
+
+inherit nativesdk
+
+ENV_NAME="${DISTRO}-${TCLIBC}-${SDK_ARCH}-${TUNE_PKGARCH}-$

{DISTRO_VERSION}.tar.gz"

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool new file +9 -0
+
+do_compile() {
+}
+
+do_install() {
+    install -d ${D}${SDKPATHNATIVE}${datadir}/icecream/bin
+
+    install -d ${D}${SDKPATHNATIVE}/environment-setup.d/
+    install -m 0644 ${WORKDIR}/icecc-env.sh ${D}${SDKPATHNATIVE}/environme

nt-setup.d/

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool new file +5 -0
+    sed -i ${D}${SDKPATHNATIVE}/environment-setup.d/icecc-env.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+
+    install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
+    install -m 0755 ${WORKDIR}/icecc-setup.sh

${D}${SDKPATHNATIVE}/post-relocate-setup.d/

meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-tool new file +8 -1
+    sed -i ${D}${SDKPATHNATIVE}/post-relocate-setup.d/icecc-setup.sh \
+        -e "s,@TOOLCHAIN_ENV@,${ENV_NAME},g"
+}
+
+PACKAGES = "${PN}"
+FILES_${PN} = "${SDKPATHNATIVE}"
+RDEPENDS_${PN} += "nativesdk-icecc-create-env"
+
--

2.14.3

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#16 lore raw

PATCH 1/1 nativesdk-icecc-toolchain: Fix LIC_FILES_CHKSUM md5

---
 meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb +1 -1
index 91958d7fbc..9d2750e479 100644
@@ -2,7 +2,7 @@
 # Released under the MIT license (see COPYING.MIT for the terms)
 SUMMARY = "Generates Icecream toolchain for SDK"
 LICENSE = "MIT"
-LIC_FILES_CHKSUM = "file://${WORKDIR}/icecc-env.sh;beginline=2;endline=20;md5=aafdb7bc2aa7ac5d039fda0c8733983c"
+LIC_FILES_CHKSUM = "file://${WORKDIR}/icecc-env.sh;beginline=2;endline=20;md5=dd6b68c1efed8a9fb04e409b3b287d47"
 
 INHIBIT_DEFAULT_DEPS = "1"
 
Signature · 1 line
2.15.1