Image/SDK multilib fixes

PATCH Applied Mark Hatle 3 participants 8 messages Jul 6, 2012 – Jul 9, 2012

Share & tools

Mark Hatle

author

2012-07-06 19:16 UTC #0 lore raw

cover Image/SDK multilib fixes

The image generation for RPM needs to be synced with the python fragement in the populate_sdk case. Otherwise the multilib configuations get out of sync.

Similar changes are NOT needed for deb or ipkg.

We also needed to enable multilib remapping when the populate_sdk_base is loaded, otherwise we don't get matching package and sdk images.

Finally to avoid conflicts in the SDK generation, we compute the install solutions by adding --replacepkgs to the temporary database. This does not change the installed image behavior in any way.

The following changes since commit 8f16811a8d51982a8b3d70e6087aef4a41926840:

bootimg: Use STAGING_KERNEL_DIR (2012-07-06 09:38:17 +0100)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/sdk
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/sdk

Mark Hatle (3):
populate_sdk_rpm: Sync multilib configuration with rootfs_rpm
multilib: Enable multilib remapping for SDK generation
package_rpm: Avoid duplicate package generation failures

 meta/classes/multilib.bbclass         |    6 ++++++
 meta/classes/package_rpm.bbclass      |   10 +++++-----
 meta/classes/populate_sdk_rpm.bbclass |   15 ++++++++++-----
 meta/classes/rootfs_rpm.bbclass       |    1 +
 4 files changed, 22 insertions(+), 10 deletions(-)

Signature · 1 line

1.7.3.4

Mark Hatle

author

2012-07-06 19:15 UTC #1 lore raw

PATCH 1/3 populate_sdk_rpm: Sync multilib configuration with rootfs_rpm

An anonymous python chunk configures some multilib settings for both populate_sdk_rpm and rootfs_rpm. The two classes should contain identical versions to ensure that the generated multilib configuration is the same for the SDK and the rootfs.

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/populate_sdk_rpm.bbclass |   15 ++++++++++-----
 meta/classes/rootfs_rpm.bbclass       |    1 +
 2 files changed, 11 insertions(+), 5 deletions(-)

meta/classes/populate_sdk_rpm.bbclass +10 -5

index b115c27..365a337 100644
@@ -115,18 +115,23 @@ EOF
 }
 
 python () {
+    # The following code should be kept in sync w/ the rootfs_rpm version.
     ml_package_archs = ""
+    ml_prefix_list = ""
     multilibs = d.getVar('MULTILIBS', True) or ""
     for ext in multilibs.split():
         eext = ext.split(':')
         if len(eext) > 1 and eext[0] == 'multilib':
             localdata = bb.data.createCopy(d)
-            overrides = localdata.getVar("OVERRIDES", False) + ":virtclass-multilib-" + eext[1]
-            localdata.setVar("OVERRIDES", overrides)
-            # TEMP: OVERRIDES isn't working right
-            localdata.setVar("DEFAULTTUNE", localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False) or "")
-            ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
+            default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False)
+            if default_tune:
+                localdata.setVar("DEFAULTTUNE", default_tune)
+            package_archs = localdata.getVar("PACKAGE_ARCHS", True) or ""
+            package_archs = " ".join([i in "all noarch any".split() and i or eext[1]+"_"+i for i in package_archs.split()])
+            ml_package_archs += " " + package_archs
+            ml_prefix_list += " " + eext[1]
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     d.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs)
+    d.setVar('MULTILIB_PREFIX_LIST', ml_prefix_list)
 }

meta/classes/rootfs_rpm.bbclass +1 -0

index 202f9ce..4551f7a 100644
@@ -198,6 +198,7 @@ python () {
         d.setVar('RPM_PREPROCESS_COMMANDS', '')
         d.setVar('RPM_POSTPROCESS_COMMANDS', '')
 
+    # The following code should be kept in sync w/ the populate_sdk_rpm version.
     ml_package_archs = ""
     ml_prefix_list = ""
     multilibs = d.getVar('MULTILIBS', True) or ""

Signature · 1 line

1.7.3.4

Mark Hatle

author

2012-07-06 19:15 UTC #2 lore raw

PATCH 2/3 multilib: Enable multilib remapping for SDK generation

In reply to Mark Hatle

Enable the remapping for SDK generation, this is required to be able to create an SDK that targets an alternative multilib. Note, this work does not finish SDK/multilib support, but it is one more step toward making it work properly.

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/multilib.bbclass |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

meta/classes/multilib.bbclass +6 -0

index f3e7618..b92a3ff 100644
@@ -54,6 +54,12 @@ python __anonymous () {
         d.setVar("LINGUAS_INSTALL", "")
         # FIXME, we need to map this to something, not delete it!
         d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
+
+    if bb.data.inherits_class('populate_sdk_base', d):
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
+
+    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
         return
 
     clsextend.rename_packages()

Signature · 1 line

1.7.3.4

Richard Purdie

2012-07-09 10:51 UTC #3 lore raw

Re: [PATCH 2/3] multilib: Enable multilib remapping for SDK generation

Quoted 16 lines · Mark Hatle

Enable the remapping for SDK generation, this is required to be able to create an SDK that targets an alternative multilib. Note, this work does not finish SDK/multilib support, but it is one more step toward making it work properly.
Signed-off-by Mark Hatle <[email protected]>
---
 meta/classes/multilib.bbclass |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
meta/classes/multilib.bbclass +0 -0
index f3e7618..b92a3ff 100644
@@ -54,6 +54,12 @@ python __anonymous () {
         d.setVar("LINGUAS_INSTALL", "")

meta/classes/multilib.bbclass python __anonymous () { · line 64

Quoted 10 lines

meta/classes/multilib.bbclass +6 -0

         # FIXME, we need to map this to something, not delete it!
         d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
+
+    if bb.data.inherits_class('populate_sdk_base', d):
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
+
+    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
         return

There is something odd going on with this patch. You've removed the earlier return but this patch doesn't show that...

Cheers,

Richard

Mark Hatle

author

2012-07-09 13:19 UTC #4 lore raw

Re: [PATCH 2/3] multilib: Enable multilib remapping for SDK generation

Quoted 22 lines · Richard Purdie

Quoted 21 lines · Mark Hatle
Enable the remapping for SDK generation, this is required to be able to create an SDK that targets an alternative multilib. Note, this work does not finish SDK/multilib support, but it is one more step toward making it work properly.
Signed-off-by Mark Hatle <[email protected]>
---
  meta/classes/multilib.bbclass |    6 ++++++
  1 files changed, 6 insertions(+), 0 deletions(-)
meta/classes/multilib.bbclass +3 -0
index f3e7618..b92a3ff 100644
@@ -54,6 +54,12 @@ python __anonymous () {
          d.setVar("LINGUAS_INSTALL", "")
          # FIXME, we need to map this to something, not delete it!
          d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
+
+    if bb.data.inherits_class('populate_sdk_base', d):
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")

Quoted 8 lines

meta/classes/multilib.bbclass python __anonymous () { · line 64
Quoted 5 lines
meta/classes/multilib.bbclass +3 -0
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
+
+    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
          return
 
There is something odd going on with this patch. You've removed the earlier return but this patch doesn't show that...

I didn't remove it, I 'moved' it later. Otherwise the populate_sdk_base case is never run, when the image case has already been loaded.

So the flow goes from:

if image class:
do work
return

do other stuff

if image class:
do work

if sdk class:
do other work

if either image or sdk class:
return

do other stuff

The behavior is the same for the image only class, the only new behavior is adding in the remap for the SDK only (or combined w/ the image) case.

--Mark

Quoted 10 lines

Cheers,
Richard
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Richard Purdie

2012-07-09 15:03 UTC #5 lore raw

Re: [PATCH 2/3] multilib: Enable multilib remapping for SDK generation

Quoted 32 lines · Mark Hatle

Quoted 22 lines · Richard Purdie
Quoted 21 lines · Mark Hatle
Enable the remapping for SDK generation, this is required to be able to create an SDK that targets an alternative multilib. Note, this work does not finish SDK/multilib support, but it is one more step toward making it work properly.
Signed-off-by Mark Hatle <[email protected]>
---
  meta/classes/multilib.bbclass |    6 ++++++
  1 files changed, 6 insertions(+), 0 deletions(-)
meta/classes/multilib.bbclass +3 -0
index f3e7618..b92a3ff 100644
@@ -54,6 +54,12 @@ python __anonymous () {
          d.setVar("LINGUAS_INSTALL", "")
          # FIXME, we need to map this to something, not delete it!
          d.setVar("PACKAGE_INSTALL_ATTEMPTONLY", "")
+
+    if bb.data.inherits_class('populate_sdk_base', d):
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK")
Quoted 8 lines
meta/classes/multilib.bbclass python __anonymous () { · line 64
Quoted 5 lines
meta/classes/multilib.bbclass +3 -0
+        clsextend.map_depends_variable("TOOLCHAIN_TARGET_TASK_ATTEMPTONLY")
+
+    if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
          return
 
There is something odd going on with this patch. You've removed the earlier return but this patch doesn't show that...

Quoted 4 lines

I didn't remove it, I 'moved' it later. Otherwise the populate_sdk_base case is never run, when the image case has already been loaded.
So the flow goes from:

I understand this. Let me put this in a simpler way. Could you send a patch that cleanly applies against master please? There is something wrong with it and it doesn't apply.

Cheers,

Richard

Mark Hatle

author

2012-07-06 19:16 UTC #6 lore raw

PATCH 3/3 package_rpm: Avoid duplicate package generation failures

In reply to Mark Hatle

When constructing a multilib based image, it's possible to end up with the same package listed in multiple steps of the install. During resolution we use the --replacepkgs option to avoid errors, as the resulting install solution will be as expected. (We do not enable the replacepkgs option for the final install step, only the generation of the install solution.)

Signed-off-by Mark Hatle <[email protected]>

---
 meta/classes/package_rpm.bbclass |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

meta/classes/package_rpm.bbclass +5 -5

index 78279f5..2a29917 100644
@@ -318,7 +318,7 @@ package_install_internal_rpm () {
 			--root "${target_rootfs}/install" \
 			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}-base_archs.macro`" \
 			-D "__dbi_txn create nofsync" \
-			-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+			-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 			${target_rootfs}/install/install.manifest
 	fi

@@ -348,7 +348,7 @@ package_install_internal_rpm () {
 				--root "${target_rootfs}/install" \
 				-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
 				-D "__dbi_txn create nofsync private" \
-				-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 			$pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" || true
 		done
 	fi

@@ -399,7 +399,7 @@ package_install_internal_rpm () {
 				--root "${target_rootfs}/install" \
 				-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
 				-D "__dbi_txn create nofsync private" \
-				-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 				$pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_recommend.${PID}" 2>&1 || true
 		done
 		cat ${target_rootfs}/install/recommend.list ${target_rootfs}/install/recommend.new | sort -u > ${target_rootfs}/install/recommend.new.list

@@ -428,7 +428,7 @@ package_install_internal_rpm () {
 			--root "${target_rootfs}/install" \
 			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}-ml_archs.macro`" \
 			-D "__dbi_txn create nofsync" \
-			-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+			-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 			${target_rootfs}/install/install_multilib.manifest
 
 		# Now that we have a solution, pull out a list of what to install...

@@ -524,7 +524,7 @@ EOF
 				--root "${target_rootfs}/install" \
 				-D "_dbpath ${target_rootfs}/initial" -D "`cat ${confbase}.macro`" \
 				-D "__dbi_txn create nofsync" \
-				-U --justdb --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
+				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
 				${target_rootfs}/install/initial_install.manifest
 
 			${RPM} -D "_dbpath ${target_rootfs}/initial" -qa --qf "%{packageorigin}\n" \

Signature · 1 line

1.7.3.4

Saul Wold

Applied

2012-07-09 17:41 UTC #7 lore raw

In reply to Mark Hatle

Quoted 25 lines · Mark Hatle

The image generation for RPM needs to be synced with the python fragement in the populate_sdk case. Otherwise the multilib configuations get out of sync.
Similar changes are NOT needed for deb or ipkg.
We also needed to enable multilib remapping when the populate_sdk_base is loaded, otherwise we don't get matching package and sdk images.
Finally to avoid conflicts in the SDK generation, we compute the install solutions by adding --replacepkgs to the temporary database. This does not change the installed image behavior in any way.
The following changes since commit 8f16811a8d51982a8b3d70e6087aef4a41926840:
   bootimg: Use STAGING_KERNEL_DIR (2012-07-06 09:38:17 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/sdk
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/sdk
Mark Hatle (3):
populate_sdk_rpm: Sync multilib configuration with rootfs_rpm
multilib: Enable multilib remapping for SDK generation
package_rpm: Avoid duplicate package generation failures

Quoted 6 lines

  meta/classes/multilib.bbclass         |    6 ++++++
  meta/classes/package_rpm.bbclass      |   10 +++++-----
  meta/classes/populate_sdk_rpm.bbclass |   15 ++++++++++-----
  meta/classes/rootfs_rpm.bbclass       |    1 +
  4 files changed, 22 insertions(+), 10 deletions(-)

Merged into OE-Core with the updated request

Thanks
Sau!