Implement multilib SDK support

PATCH Applied Mark Hatle 2 participants 6 messages Oct 22, 2012 – Oct 28, 2012

Share & tools

Mark Hatle

author

2012-10-22 15:25 UTC #0 lore raw

cover Implement multilib SDK support

The attached patch set implements multilib SDK support. This means that when generating an SDK (either by a traditional SDK recipe or the do_populate_sdk task), that support infrastructure for each multilib will be included in the SDK.

Note, this code does -not- change the toolchain. If the generated toolchain does not work properly with the multilib configuration, that will need to be a seperate fix!

This was tested with both the oe-core and our custom Wind River toolchain. Generation passed on the oe-core SDK, but the functionality of the toolchain was not tested. The Wind River toolchain did get properly generated and functioned as expected.

Note: The 0003 patch that affects processing the environment file was agreed to with the other author of the function.

The following changes since commit 06d27cf0fbcc4004e6f456880eca49893c9290bf:

xorg: remove XF86 BigFont extension (2012-10-22 14:57:26 +0100)

are available in the git repository at:
git://git.yoctoproject.org/poky-contrib mhatle/sdk
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/sdk

Mark Hatle (4):
multilib: Add support for cross-canadian multilib packages multilib - crosssdk: Stop building multilib for crosssdk packages
populate_sdk_base: Update extraction script for multilibs
populate_sdk_base: Ensure that the multilib cross-canadian tools are
used

 meta/classes/base.bbclass              |   43 +++++++++++++++++++++++++++++---
 meta/classes/multilib.bbclass          |   20 +++++++++++---
 meta/classes/populate_sdk_base.bbclass |    7 +++--
 meta/lib/oe/classextend.py             |    4 +-
 4 files changed, 60 insertions(+), 14 deletions(-)

Signature · 1 line

1.7.3.4

Mark Hatle

author

2012-10-22 15:25 UTC #1 lore raw

PATCH 1/4 multilib: Add support for cross-canadian multilib packages

Add support for the generation of cross-canadian packages.

Each cross-canadian package has:
PN = "pkg-cross-canadian-${TRANSLATED_TARGET_ARCH}"

in order for that to be evaluated properly with multilibs enabled, it was necessary to detect both the presence of the cross-canadian packages and then update the vars using the OVERRIDE for the multilib. Additional checks were made to ensure that any dependency that sais "cross-canadian" did not get prefixed with the MLPREFIX.

Also, make sure that even when building multilib cross-canadian packages, we only use the single SDK PACKAGE_ARCH, we don't want or need variants.

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/base.bbclass     |   39 +++++++++++++++++++++++++++++++++++++--
 meta/classes/multilib.bbclass |   18 ++++++++++++++----
 meta/lib/oe/classextend.py    |    4 ++--
 3 files changed, 53 insertions(+), 8 deletions(-)

meta/classes/base.bbclass +37 -2

index f254306..12a30c9 100644
@@ -183,6 +183,8 @@ def preferred_ml_updates(d):
             providers.append(v)
 
     for pkg, reason in blacklists.items():
+        if pkg.endswith("-native") or pkg.startswith("nativesdk-") or 'cross-canadian' in pkg:
+            continue
         for p in prefixes:
             newpkg = p + "-" + pkg
             if not d.getVarFlag('PNBLACKLIST', newpkg, True):

@@ -193,6 +195,17 @@ def preferred_ml_updates(d):
         pkg = v.replace("PREFERRED_VERSION_", "")
         if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
             continue
+        if 'cross-canadian' in pkg:
+            for p in prefixes:
+                localdata = bb.data.createCopy(d)
+                override = ":virtclass-multilib-" + p
+                localdata.setVar("OVERRIDES", localdata.getVar("OVERRIDES", False) + override)
+                bb.data.update_data(localdata)
+                newname = localdata.expand(v)
+                if newname != v:
+                    newval = localdata.getVar(v, True)
+                    d.setVar(newname, newval)
+            continue
         for p in prefixes:
             newname = "PREFERRED_VERSION_" + p + "-" + pkg
             if not d.getVar(newname, False):

@@ -203,14 +216,36 @@ def preferred_ml_updates(d):
         pkg = prov.replace("PREFERRED_PROVIDER_", "")
         if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
             continue
+        if 'cross-canadian' in pkg:
+            for p in prefixes:
+                localdata = bb.data.createCopy(d)
+                override = ":virtclass-multilib-" + p
+                localdata.setVar("OVERRIDES", localdata.getVar("OVERRIDES", False) + override)
+                bb.data.update_data(localdata)
+                newname = localdata.expand(prov)
+                if newname != prov:
+                    newval = localdata.expand(val)
+                    d.setVar(newname, newval)
+            continue
         virt = ""
         if pkg.startswith("virtual/"):
             pkg = pkg.replace("virtual/", "")
             virt = "virtual/"
         for p in prefixes:
-            newname = "PREFERRED_PROVIDER_" + virt + p + "-" + pkg
             if pkg != "kernel":
                 val = p + "-" + val
+
+            # implement variable keys
+            localdata = bb.data.createCopy(d)
+            override = ":virtclass-multilib-" + p
+            localdata.setVar("OVERRIDES", localdata.getVar("OVERRIDES", False) + override)
+            bb.data.update_data(localdata)
+            newname = localdata.expand(prov)
+            if newname != prov:
+                d.setVar(newname, localdata.expand(val))
+
+            # implement alternative multilib name
+            newname = localdata.expand("PREFERRED_PROVIDER_" + virt + p + "-" + pkg)
             if not d.getVar(newname, False):
                 d.setVar(newname, val)

@@ -218,7 +253,7 @@ def preferred_ml_updates(d):
     mp = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split()
     extramp = []
     for p in mp:
-        if p.endswith("-native") or p.startswith("nativesdk-"):
+        if p.endswith("-native") or p.startswith("nativesdk-") or 'cross-canadian' in p:
             continue
         virt = ""
         if p.startswith("virtual/"):

meta/classes/multilib.bbclass +14 -4

index 3824f3b..de6e59e 100644
@@ -18,6 +18,13 @@ python multilib_virtclass_handler () {
         e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False))
         return
 
+    if bb.data.inherits_class('cross-canadian', e.data):
+        e.data.setVar("MLPREFIX", variant + "-")
+        override = ":virtclass-multilib-" + variant
+        e.data.setVar("OVERRIDES", e.data.getVar("OVERRIDES", False) + override)
+        bb.data.update_data(e.data)
+        return
+
     if bb.data.inherits_class('native', e.data):
         raise bb.parse.SkipPackage("We can't extend native recipes")

@@ -69,12 +76,16 @@ python __anonymous () {
     if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
         return
 
+    clsextend.map_depends_variable("DEPENDS")
+    clsextend.map_variable("PROVIDES")
+
+    if bb.data.inherits_class('cross-canadian', d):
+        return
+
     clsextend.rename_packages()
     clsextend.rename_package_variables((d.getVar("PACKAGEVARS", True) or "").split())
 
-    clsextend.map_depends_variable("DEPENDS")
     clsextend.map_packagevars()
-    clsextend.map_variable("PROVIDES")
     clsextend.map_regexp_variable("PACKAGES_DYNAMIC")
     clsextend.map_variable("PACKAGE_INSTALL")
     clsextend.map_variable("INITSCRIPT_PACKAGES")

@@ -90,7 +101,7 @@ python do_package_qa_multilib() {
         for i in values:
             if i.startswith('virtual/'):
                 i = i[len('virtual/'):]
-            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)):
+            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)) and (not 'cross-canadian' in i):
                 candidates.append(i)
         if len(candidates) > 0:
             bb.warn("Multilib QA Issue: %s package %s - suspicious values '%s' in %s"

@@ -109,4 +120,3 @@ python do_package_qa_multilib() {
         check_mlprefix(pkg, 'RREPLACES', ml)
         check_mlprefix(pkg, 'RCONFLICTS', ml)
 }
-

meta/lib/oe/classextend.py +2 -2

index 857a6c9..c982fb7 100644
@@ -50,7 +50,7 @@ class ClassExtender(object):
         return newdata
 
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime")):
+        if dep.endswith(("-native", "-native-runtime")) or ('nativesdk-' in dep) or ('cross-canadian' in dep):
             return dep
         else:
             return self.extend_name(dep)

@@ -93,7 +93,7 @@ class ClassExtender(object):
 
 class NativesdkClassExtender(ClassExtender):
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime", "-cross")):
+        if dep.endswith(("-native", "-native-runtime", "-cross")) or ('nativesdk-' in dep):
             return dep
         elif dep.endswith(("-gcc-intermediate", "-gcc-initial", "-gcc", "-g++")):
             return dep + "-crosssdk"

Signature · 1 line

1.7.3.4

Mark Hatle

author

2012-10-22 15:25 UTC #2 lore raw

PATCH 2/4 multilib - crosssdk: Stop building multilib for crosssdk packages

In reply to Mark Hatle

Crosssdk packages are not actually multilib packages, so treat them the same as other nativesdk packages in the multilib, base, and classextend components.

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/base.bbclass     |    8 ++++----
 meta/classes/multilib.bbclass |    4 ++--
 meta/lib/oe/classextend.py    |    4 ++--
 3 files changed, 8 insertions(+), 8 deletions(-)

meta/classes/base.bbclass +4 -4

index 12a30c9..783b64d 100644
@@ -183,7 +183,7 @@ def preferred_ml_updates(d):
             providers.append(v)
 
     for pkg, reason in blacklists.items():
-        if pkg.endswith("-native") or pkg.startswith("nativesdk-") or 'cross-canadian' in pkg:
+        if pkg.endswith(("-native", "-crosssdk")) or pkg.startswith("nativesdk-") or 'cross-canadian' in pkg:
             continue
         for p in prefixes:
             newpkg = p + "-" + pkg

@@ -193,7 +193,7 @@ def preferred_ml_updates(d):
     for v in versions:
         val = d.getVar(v, False)
         pkg = v.replace("PREFERRED_VERSION_", "")
-        if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
+        if pkg.endswith(("-native", "-crosssdk")) or pkg.startswith("nativesdk-"):
             continue
         if 'cross-canadian' in pkg:
             for p in prefixes:

@@ -214,7 +214,7 @@ def preferred_ml_updates(d):
     for prov in providers:
         val = d.getVar(prov, False)
         pkg = prov.replace("PREFERRED_PROVIDER_", "")
-        if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
+        if pkg.endswith(("-native", "-crosssdk")) or pkg.startswith("nativesdk-"):
             continue
         if 'cross-canadian' in pkg:
             for p in prefixes:

@@ -253,7 +253,7 @@ def preferred_ml_updates(d):
     mp = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split()
     extramp = []
     for p in mp:
-        if p.endswith("-native") or p.startswith("nativesdk-") or 'cross-canadian' in p:
+        if p.endswith(("-native", "-crosssdk")) or p.startswith("nativesdk-") or 'cross-canadian' in p:
             continue
         virt = ""
         if p.startswith("virtual/"):

meta/classes/multilib.bbclass +2 -2

index de6e59e..c249925 100644
@@ -28,7 +28,7 @@ python multilib_virtclass_handler () {
     if bb.data.inherits_class('native', e.data):
         raise bb.parse.SkipPackage("We can't extend native recipes")
 
-    if bb.data.inherits_class('nativesdk', e.data):
+    if bb.data.inherits_class('nativesdk', e.data) or bb.data.inherits_class('crosssdk', e.data):
         raise bb.parse.SkipPackage("We can't extend nativesdk recipes")
 
     save_var_name=e.data.getVar("MULTILIB_SAVE_VARNAME", True) or ""

@@ -101,7 +101,7 @@ python do_package_qa_multilib() {
         for i in values:
             if i.startswith('virtual/'):
                 i = i[len('virtual/'):]
-            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)) and (not 'cross-canadian' in i):
+            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)) and (not 'cross-canadian' in i) and (not i.startswith("nativesdk-")):
                 candidates.append(i)
         if len(candidates) > 0:
             bb.warn("Multilib QA Issue: %s package %s - suspicious values '%s' in %s"

meta/lib/oe/classextend.py +2 -2

index c982fb7..c46332c 100644
@@ -50,7 +50,7 @@ class ClassExtender(object):
         return newdata
 
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime")) or ('nativesdk-' in dep) or ('cross-canadian' in dep):
+        if dep.endswith(("-native", "-native-runtime", "-crosssdk")) or ('nativesdk-' in dep) or ('cross-canadian' in dep):
             return dep
         else:
             return self.extend_name(dep)

@@ -93,7 +93,7 @@ class ClassExtender(object):
 
 class NativesdkClassExtender(ClassExtender):
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime", "-cross")) or ('nativesdk-' in dep):
+        if dep.endswith(("-native", "-native-runtime", "-cross", "-crosssdk")) or ('nativesdk-' in dep):
             return dep
         elif dep.endswith(("-gcc-intermediate", "-gcc-initial", "-gcc", "-g++")):
             return dep + "-crosssdk"

Signature · 1 line

1.7.3.4

Mark Hatle

author

2012-10-22 15:25 UTC #3 lore raw

PATCH 3/4 populate_sdk_base: Update extraction script for multilibs

In reply to Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/populate_sdk_base.bbclass |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

meta/classes/populate_sdk_base.bbclass +3 -2

index a914a2c..8258224 100644
@@ -160,8 +160,9 @@ echo "done"
 
 printf "Setting it up..."
 # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `ls $target_sdk_dir/environment-setup-*`; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done
 
 # fix dynamic loader paths in all ELF SDK binaries
 native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')

Signature · 1 line

1.7.3.4

Mark Hatle

author

2012-10-22 15:25 UTC #4 lore raw

PATCH 4/4 populate_sdk_base: Ensure that the multilib cross-canadian tools are used

In reply to Mark Hatle

Update the host toolchain list, for cross-canadian toolchains, to ensure that all of the supported multilibs are built and installed. This dynamically generates the dependnecy set based on the current multilib configuration.

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/populate_sdk_base.bbclass |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

meta/classes/populate_sdk_base.bbclass +1 -1

index 8258224..2fe7f15 100644
@@ -7,7 +7,7 @@ SDK_DEPLOY = "${TMPDIR}/deploy/sdk"
 
 SDKTARGETSYSROOT = "${SDKPATH}/sysroots/${MULTIMACH_TARGET_SYS}"
 
-TOOLCHAIN_HOST_TASK ?= "nativesdk-packagegroup-sdk-host packagegroup-cross-canadian-${TRANSLATED_TARGET_ARCH}"
+TOOLCHAIN_HOST_TASK ?= "nativesdk-packagegroup-sdk-host packagegroup-cross-canadian-${@' packagegroup-cross-canadian-'.join(all_multilib_tune_values(d, 'TRANSLATED_TARGET_ARCH').split())}"
 TOOLCHAIN_HOST_TASK_ATTEMPTONLY ?= ""
 TOOLCHAIN_TARGET_TASK ?= "packagegroup-core-standalone-sdk-target packagegroup-core-standalone-sdk-target-dbg"
 TOOLCHAIN_TARGET_TASK_ATTEMPTONLY ?= ""

Signature · 1 line

1.7.3.4

Saul Wold

Applied

2012-10-28 01:28 UTC #5 lore raw

In reply to Mark Hatle

Quoted 32 lines · Mark Hatle

The attached patch set implements multilib SDK support. This means that when generating an SDK (either by a traditional SDK recipe or the do_populate_sdk task), that support infrastructure for each multilib will be included in the SDK.
Note, this code does -not- change the toolchain. If the generated toolchain does not work properly with the multilib configuration, that will need to be a seperate fix!
This was tested with both the oe-core and our custom Wind River toolchain. Generation passed on the oe-core SDK, but the functionality of the toolchain was not tested. The Wind River toolchain did get properly generated and functioned as expected.
Note: The 0003 patch that affects processing the environment file was agreed to with the other author of the function.
The following changes since commit 06d27cf0fbcc4004e6f456880eca49893c9290bf:
   xorg: remove XF86 BigFont extension (2012-10-22 14:57:26 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib mhatle/sdk
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/sdk
Mark Hatle (4):
multilib: Add support for cross-canadian multilib packages multilib - crosssdk: Stop building multilib for crosssdk packages
populate_sdk_base: Update extraction script for multilibs
populate_sdk_base: Ensure that the multilib cross-canadian tools are
used

Quoted 6 lines

  meta/classes/base.bbclass              |   43 +++++++++++++++++++++++++++++---
  meta/classes/multilib.bbclass          |   20 +++++++++++---
  meta/classes/populate_sdk_base.bbclass |    7 +++--
  meta/lib/oe/classextend.py             |    4 +-
  4 files changed, 60 insertions(+), 14 deletions(-)

Merged into OE-Core

Thanks
Sau!