Skip to contentSkip to conversation

Misc changes relating to toolchain and image gen

PATCH In review Mark Hatle 7 participants 27 messages Nov 13, 2013 – Nov 19, 2013

lore
Share & tools
lore

Mark Hatle

author
#0 lore raw

cover Misc changes relating to toolchain and image gen

A few misc fixes relating to toolchain and image generation.

The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.

The next fixes an issue w/ the previous binutils patch and upgrades.

Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.

The final patch fixes an issue w/ the image generatation. If the IMAGE_INSTALL contains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.

Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in the IMAGE_INSTALL line. The system will attempt to determine which to use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)

The following changes since commit 616354f13732d13c17434d5b60b166f691c25761:

binutils: Add gnu-config-native to DEPENDS (2013-11-12 16:00:20 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib mhatle/misc1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/misc1

Mark Hatle (6):
gcc: Use alternatives for the *-symlinks packages.
gcc: Drop *-symlinks
binutils: Ensure old -symlinks packages get removed
packagegroups: Remove toolchain *-symlinks packages
packagegroup-self-hosted: Use packagegroup-core-buildessentials
package_rpm: Allow translation of requirement to package name

 meta/classes/package_rpm.bbclass                   |  79 +++++++++++++++-
 .../packagegroup-core-buildessential.bb            |   4 -
 .../packagegroups/packagegroup-self-hosted.bb      |  16 +---
 meta/recipes-devtools/binutils/binutils.inc        |   1 +
 meta/recipes-devtools/gcc/gcc-target.inc           | 103 +++++++++++++--------
 5 files changed, 141 insertions(+), 62 deletions(-)
Signature · 1 line
1.8.1.2.545.g2f19ada
author
#1 lore raw

PATCH 1/6 gcc: Use alternatives for the *-symlinks packages.

The various gcc related symlinks should be provided as alternatives instead of hard coded symlinks. This will permit multiple toolchains on a system.

Multiple toolchains could come from multilib configurations or alternative open source or commerical sources.

Note, gccbug was skipped since it doesn't seem to be generated anymore.

---
 meta/recipes-devtools/gcc/gcc-target.inc | 91 ++++++++++++++++++++------------
 1 file changed, 58 insertions(+), 33 deletions(-)
meta/recipes-devtools/gcc/gcc-target.inc +58 -33
index e1179ac..ee41d3f 100644
@@ -50,11 +50,8 @@ FILES_${PN}-plugin-dev = "\
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \
   ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \
 "
-FILES_${PN}-symlinks = "\
-  ${bindir}/cc \
-  ${bindir}/gcc \
-  ${bindir}/gccbug \
-"
+FILES_${PN}-symlinks = ""
+ALLOW_EMPTY_${PN}-symlinks = "1"
 
 FILES_${PN}-plugins = "\
   ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -65,36 +62,33 @@ FILES_g77 = "\
   ${bindir}/${TARGET_PREFIX}g77 \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \
 "
-FILES_g77-symlinks = "\
-  ${bindir}/g77 \
-  ${bindir}/f77 \
-"
+FILES_g77-symlinks = ""
+ALLOW_EMPTY_g77-symlinks = "1"
+
 FILES_gfortran = "\
   ${bindir}/${TARGET_PREFIX}gfortran \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \
 "
-FILES_gfortran-symlinks = "\
-  ${bindir}/gfortran \
-  ${bindir}/f95"
+FILES_gfortran-symlinks = ""
+ALLOW_EMPTY_gfortran-symlinks = "1"
 
 FILES_cpp = "\
   ${bindir}/${TARGET_PREFIX}cpp \
   ${base_libdir}/cpp \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1"
-FILES_cpp-symlinks = "${bindir}/cpp"
+FILES_cpp-symlinks = ""
+ALLOW_EMPTY_cpp-symlinks = "1"
 
 FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov"
-FILES_gcov-symlinks = "${bindir}/gcov"
+FILES_gcov-symlinks = ""
+ALLOW_EMPTY_gconv-symlinks = "1"
 
 FILES_g++ = "\
   ${bindir}/${TARGET_PREFIX}g++ \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
 "
-FILES_g++-symlinks = "\
-  ${bindir}/c++ \
-  ${bindir}/g++ \
-"
-
+FILES_g++-symlinks = ""
+ALLOW_EMPTY_g++-symlinks = "1"
 
 FILES_${PN}-doc = "\
   ${infodir} \
@@ -133,22 +127,53 @@ do_install () {
 	# Not sure why we end up with these but we don't want them...
 	rm -f ${TARGET_PREFIX}${TARGET_PREFIX}*
 
-	# Symlinks so we can use these trivially on the target
-	if [ -e ${TARGET_PREFIX}g77 ]; then
-		ln -sf ${TARGET_PREFIX}g77 g77 || true
-		ln -sf g77 f77 || true
-	fi
-	if [ -e ${TARGET_PREFIX}gfortran ]; then
-		ln -sf ${TARGET_PREFIX}gfortran gfortran || true
-		ln -sf gfortran f95 || true
-	fi
-	ln -sf ${TARGET_PREFIX}g++ g++
-	ln -sf ${TARGET_PREFIX}gcc gcc
-	ln -sf ${TARGET_PREFIX}cpp cpp
 	install -d ${D}${base_libdir}
 	ln -sf ${bindir}/${TARGET_PREFIX}cpp ${D}${base_libdir}/cpp
-	ln -sf g++ c++
-	ln -sf gcc cc
 
 	chown -R root:root ${D}
 }
+
+inherit update-alternatives
+
+ALTERNATIVE_PRIORITY = "100"
+
+ALTERNATIVE_${PN}-symlinks = "cc gcc"
+ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_cpp-symlinks = "cpp"
+ALTERNATIVE_gcov-symlinks = "gcov"
+ALTERNATIVE_g++-symlinks = "c++ g++"
+
+ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc"
+ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
+
+ALTERNATIVE_LINK_NAME[gcc] = "${bindir}/gcc"
+ALTERNATIVE_TARGET[gcc] = "${bindir}/${TARGET_PREFIX}gcc"
+
+# Not really generated any longer, avoid warnings...
+#ALTERNATIVE_LINK_NAME[gccbug] = "${bindir}/gccbug"
+#ALTERNATIVE_TARGET[gccbug] = "${bindir}/${TARGET_PREFIX}gccbug"
+
+ALTERNATIVE_LINK_NAME[g77] = "${bindir}/g77"
+ALTERNATIVE_TARGET[g77] = "${bindir}/${TARGET_PREFIX}g77"
+
+ALTERNATIVE_LINK_NAME[f77] = "${bindir}/f77"
+ALTERNATIVE_TARGET[f77] = "${bindir}/${TARGET_PREFIX}g77"
+
+ALTERNATIVE_LINK_NAME[gfortran] = "${bindir}/gfortran"
+ALTERNATIVE_TARGET[gfortran] = "${bindir}/${TARGET_PREFIX}gfortran"
+
+ALTERNATIVE_LINK_NAME[f95] = "${bindir}/f95"
+ALTERNATIVE_TARGET[f95] = "${bindir}/${TARGET_PREFIX}gfortran"
+
+ALTERNATIVE_LINK_NAME[cpp] = "${bindir}/cpp"
+ALTERNATIVE_TARGET[cpp] = "${bindir}/${TARGET_PREFIX}cpp"
+
+ALTERNATIVE_LINK_NAME[gcov] = "${bindir}/gcov"
+ALTERNATIVE_TARGET[gcov] = "${bindir}/${TARGET_PREFIX}gcov"
+
+ALTERNATIVE_LINK_NAME[c++] = "${bindir}/c++"
+ALTERNATIVE_TARGET[c++] = "${bindir}/${TARGET_PREFIX}g++"
+
+ALTERNATIVE_LINK_NAME[g++] = "${bindir}/g++"
+ALTERNATIVE_TARGET[g++] = "${bindir}/${TARGET_PREFIX}g++"
Signature · 1 line
1.8.1.2.545.g2f19ada
#2 lore raw

Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.

Quoted 9 lines · Mark Hatle

The various gcc related symlinks should be provided as alternatives instead of hard coded symlinks. This will permit multiple toolchains on a system.

Multiple toolchains could come from multilib configurations or alternative open source or commerical sources.

Note, gccbug was skipped since it doesn't seem to be generated anymore.

Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic change'? You add code to remove in patch 2, I think the end patch would be easier to review, no?

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
author
#3 lore raw

Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.

Quoted 7 lines · Otavio Salvador
Quoted 6 lines · Mark Hatle

The various gcc related symlinks should be provided as alternatives instead of hard coded symlinks. This will permit multiple toolchains on a system.

Multiple toolchains could come from multilib configurations or alternative open source or commerical sources.

Quoted 8 lines
Quoted 3 lines

Note, gccbug was skipped since it doesn't seem to be generated anymore.

Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic change'? You add code to remove in patch 2, I think the end patch would be easier to review, no?

It was done this way to match how binutils was implemented. These are actually two logically separate patches. The first switches from hard coded symlinks to using update-alternatives. The second patch says that a separate -symlinks package is no longer needed.

See binutils:

1395aefcaeac94dd0e6ed3a718b7e58dd43b355e
24093e26f246f222c385dc37a2f8cf8b0f183175

(The second of the patches can be reverted -- if ever needed -- and the update alternatives functionality will still work properly.)

--Mark

#4 lore raw

Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.

Quoted 29 lines · Mark Hatle
Quoted 13 lines · Otavio Salvador
Quoted 10 lines · Mark Hatle

The various gcc related symlinks should be provided as alternatives instead
of hard coded symlinks. This will permit multiple toolchains on a system.

Multiple toolchains could come from multilib configurations or alternative
open source or commerical sources.

Quoted 9 lines
Quoted 3 lines

Note, gccbug was skipped since it doesn't seem to be generated anymore.

Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic change'? You add code to remove in patch 2, I think the end patch would be easier to review, no?

It was done this way to match how binutils was implemented. These are actually two logically separate patches. The first switches from hard coded symlinks to using update-alternatives. The second patch says that a separate -symlinks package is no longer needed.

Quoted 7 lines

See binutils:

1395aefcaeac94dd0e6ed3a718b7e58dd43b355e
24093e26f246f222c385dc37a2f8cf8b0f183175

(The second of the patches can be reverted -- if ever needed -- and the update alternatives functionality will still work properly.)

Ok; you got a point :-)

Agreed.

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
author
#5 lore raw

PATCH 2/6 gcc: Drop *-symlinks

In reply to Mark Hatle

The -symlinks packages don't really add any value if we're using update-alternatives. Drop it, leaving an RPROVIDES in case anyone thought they needed it still.

---
 meta/recipes-devtools/gcc/gcc-target.inc | 48 ++++++++++++++++----------------
 1 file changed, 24 insertions(+), 24 deletions(-)
meta/recipes-devtools/gcc/gcc-target.inc +24 -24
index ee41d3f..426be15 100644
@@ -10,12 +10,12 @@ EXTRA_OECONF_PATHS = " \
 ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR} -I${B}/gcc/include/ "
 
 PACKAGES = "\
-  ${PN} ${PN}-plugins ${PN}-symlinks \
-  g++ g++-symlinks \
-  cpp cpp-symlinks \
-  g77 g77-symlinks \
-  gfortran gfortran-symlinks \
-  gcov gcov-symlinks \
+  ${PN} ${PN}-plugins \
+  g++ \
+  cpp \
+  g77 \
+  gfortran \
+  gcov \
   ${PN}-plugin-dev \
   ${PN}-doc \
   ${PN}-dev \
@@ -36,6 +36,8 @@ FILES_${PN} = "\
   ${gcclibdir}/${TARGET_SYS}/${BINV}/include-fixed \
 "
 INSANE_SKIP_${PN} += "dev-so"
+RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
 
 FILES_${PN}-dbg += "\
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/.debug/ \
@@ -50,8 +52,6 @@ FILES_${PN}-plugin-dev = "\
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \
   ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \
 "
-FILES_${PN}-symlinks = ""
-ALLOW_EMPTY_${PN}-symlinks = "1"
 
 FILES_${PN}-plugins = "\
   ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -62,33 +62,33 @@ FILES_g77 = "\
   ${bindir}/${TARGET_PREFIX}g77 \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \
 "
-FILES_g77-symlinks = ""
-ALLOW_EMPTY_g77-symlinks = "1"
+RPROVIDES_g77 += "g77-symlinks"
+RREPLACES_g77 += "g77-symlinks"
 
 FILES_gfortran = "\
   ${bindir}/${TARGET_PREFIX}gfortran \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \
 "
-FILES_gfortran-symlinks = ""
-ALLOW_EMPTY_gfortran-symlinks = "1"
+RPROVIDES_gfortran += "gfortran-symlinks"
+RREPLACES_gfortran += "gfortran-symlinks"
 
 FILES_cpp = "\
   ${bindir}/${TARGET_PREFIX}cpp \
   ${base_libdir}/cpp \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1"
-FILES_cpp-symlinks = ""
-ALLOW_EMPTY_cpp-symlinks = "1"
+RPROVIDES_cpp += "cpp-symlinks"
+RREPLACES_cpp += "cpp-symlinks"
 
 FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov"
-FILES_gcov-symlinks = ""
-ALLOW_EMPTY_gconv-symlinks = "1"
+RPROVIDES_gcov += "gcov-symlinks"
+RREPLACES_gcov += "gcov-symlinks"
 
 FILES_g++ = "\
   ${bindir}/${TARGET_PREFIX}g++ \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
 "
-FILES_g++-symlinks = ""
-ALLOW_EMPTY_g++-symlinks = "1"
+RPROVIDES_g++ += "g++-symlinks"
+RREPLACES_g++ += "g++-symlinks"
 
 FILES_${PN}-doc = "\
   ${infodir} \
@@ -137,12 +137,12 @@ inherit update-alternatives
 
 ALTERNATIVE_PRIORITY = "100"
 
-ALTERNATIVE_${PN}-symlinks = "cc gcc"
-ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_cpp-symlinks = "cpp"
-ALTERNATIVE_gcov-symlinks = "gcov"
-ALTERNATIVE_g++-symlinks = "c++ g++"
+ALTERNATIVE_${PN} = "cc gcc"
+ALTERNATIVE_g77 = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_gfortran = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_cpp = "cpp"
+ALTERNATIVE_gcov = "gcov"
+ALTERNATIVE_g++ = "c++ g++"
 
 ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc"
 ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
Signature · 1 line
1.8.1.2.545.g2f19ada
author
#6 lore raw

PATCH 3/6 binutils: Ensure old -symlinks packages get removed

In reply to Mark Hatle

When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.

---
 meta/recipes-devtools/binutils/binutils.inc | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/binutils/binutils.inc +1 -0
index 17c66bc..ea2383e 100644
@@ -21,6 +21,7 @@ FILES_${PN} = " \
 	${prefix}/${TARGET_SYS}/bin/*"
 
 RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
 
 FILES_${PN}-dev = " \
 	${includedir} \
Signature · 1 line
1.8.1.2.545.g2f19ada
#7 lore raw

Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed

Quoted 13 lines · Mark Hatle

When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.

---
 meta/recipes-devtools/binutils/binutils.inc | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/binutils/binutils.inc +0 -0
index 17c66bc..ea2383e 100644

meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24

Quoted 4 lines
meta/recipes-devtools/binutils/binutils.inc +1 -0
@@ -21,6 +21,7 @@ FILES_${PN} = " \
 	${prefix}/${TARGET_SYS}/bin/*"
 
 RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"

IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.

Quoted 10 lines
meta/recipes-devtools/binutils/binutils.inc +0 -0
 FILES_${PN}-dev = " \
 	${includedir} \
Signature · 6 lines
1.8.1.2.545.g2f19ada

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#8 lore raw

Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed

Quoted 16 lines · Martin Jansa
Quoted 15 lines · Mark Hatle

When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.

---
 meta/recipes-devtools/binutils/binutils.inc | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/binutils/binutils.inc +0 -0
index 17c66bc..ea2383e 100644
@@ -21,6 +21,7 @@ FILES_${PN} = " \
 	${prefix}/${TARGET_SYS}/bin/*"
 
Quoted 5 lines

meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24

Quoted 2 lines
meta/recipes-devtools/binutils/binutils.inc +1 -0
 RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"

IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.

Also for dpkg. If you have Replaces: without Conflicts: then this just means that the replacing package is allowed to overwrite files in the replaced one, not that the replaced one will actually be uninstalled.

p.

author
#9 lore raw

Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed

Quoted 23 lines · Phil Blundell
Quoted 16 lines · Martin Jansa
Quoted 15 lines · Mark Hatle

When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.

---
  meta/recipes-devtools/binutils/binutils.inc | 1 +
  1 file changed, 1 insertion(+)
meta/recipes-devtools/binutils/binutils.inc +0 -0
index 17c66bc..ea2383e 100644
@@ -21,6 +21,7 @@ FILES_${PN} = " \
  	${prefix}/${TARGET_SYS}/bin/*"
 
Quoted 5 lines

meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24

Quoted 2 lines
meta/recipes-devtools/binutils/binutils.inc +1 -0
  RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"

IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.

Quoted 3 lines

Also for dpkg. If you have Replaces: without Conflicts: then this just means that the replacing package is allowed to overwrite files in the replaced one, not that the replaced one will actually be uninstalled.

This seems incredibly strange to me. But I can adjust the 2/6 and 3/6 patches by adding the appropriate RCONFLICTS.

--Mark

Quoted 3 lines

p.

#10 lore raw

Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed

Quoted 24 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 21 lines · Martin Jansa
Quoted 13 lines · Mark Hatle

When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.

---
  meta/recipes-devtools/binutils/binutils.inc | 1 +
  1 file changed, 1 insertion(+)
meta/recipes-devtools/binutils/binutils.inc +0 -0
index 17c66bc..ea2383e 100644

meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24

Quoted 4 lines
meta/recipes-devtools/binutils/binutils.inc +1 -0
@@ -21,6 +21,7 @@ FILES_${PN} = " \
  	${prefix}/${TARGET_SYS}/bin/*"
 
  RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"

IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.

Quoted 5 lines
Quoted 3 lines

Also for dpkg. If you have Replaces: without Conflicts: then this just means that the replacing package is allowed to overwrite files in the replaced one, not that the replaced one will actually be uninstalled.

This seems incredibly strange to me.

Well, perhaps, but it's always been that way. See the documentation for Replaces:

http://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces

p.

author
#11 lore raw

PATCH 4/6 packagegroups: Remove toolchain *-symlinks packages

In reply to Mark Hatle

We are no longer producing the *-symlinks packages, so remove the dependencies to those items.

---
 meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb | 4 ----
 meta/recipes-core/packagegroups/packagegroup-self-hosted.bb         | 4 ----
 2 files changed, 8 deletions(-)
meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb +0 -4
index 74ed247..e21f8c1 100644
@@ -12,13 +12,9 @@ RDEPENDS_packagegroup-core-buildessential = "\
     autoconf \
     automake \
     binutils \
-    binutils-symlinks \
     cpp \
-    cpp-symlinks \
     gcc \
-    gcc-symlinks \
     g++ \
-    g++-symlinks \
     gettext \
     make \
     libstdc++ \
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb +0 -4
index 108e678..8e43201 100644
@@ -70,20 +70,16 @@ RDEPENDS_packagegroup-self-hosted-sdk = "\
     autoconf \
     automake \
     binutils \
-    binutils-symlinks \
     ccache \
     coreutils \
     cpp \
-    cpp-symlinks \
     distcc \
     eglibc-utils \
     eglibc-gconv-ibm850 \
     file \
     findutils \
     g++ \
-    g++-symlinks \
     gcc \
-    gcc-symlinks \
     intltool \
     ldd \
     less \
Signature · 1 line
1.8.1.2.545.g2f19ada
author
#12 lore raw

PATCH 5/6 packagegroup-self-hosted: Use packagegroup-core-buildessentials

In reply to Mark Hatle

Instead of manually specifying build dependencies, use the packagegroup-core-buildessentials. This ensures there is only one place that needs to be modified when toolchain items and dependencies change.

---
 meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb +1 -11
index 8e43201..e91a8d9 100644
@@ -67,33 +67,23 @@ RRECOMMENDS_packagegroup-self-hosted-host-tools = "\
 
 # eglibc-utils: for rpcgen
 RDEPENDS_packagegroup-self-hosted-sdk = "\
-    autoconf \
-    automake \
-    binutils \
     ccache \
     coreutils \
-    cpp \
     distcc \
     eglibc-utils \
     eglibc-gconv-ibm850 \
     file \
     findutils \
-    g++ \
-    gcc \
     intltool \
     ldd \
     less \
     libssp \
     libssp-dev \
     libssp-staticdev \
-    libstdc++ \
-    libstdc++-dev \
-    libtool \
-    make \
     mktemp \
+    packagegroup-core-buildessential \
     perl-module-re \
     perl-module-text-wrap \
-    pkgconfig \
     quilt \
     sed \
     "
Signature · 1 line
1.8.1.2.545.g2f19ada
author
#13 lore raw

PATCH 6/6 package_rpm: Allow translation of requirement to package name

In reply to Mark Hatle

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
 meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 75 insertions(+), 4 deletions(-)
meta/classes/package_rpm.bbclass +75 -4
index 31265d9..64da1f5 100644
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
 	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
 
 	pkgs_to_install=""
+	not_found=""
 	for pkg in "$@" ; do
 		new_pkg="$pkg"
 		if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
 				fi
 				subst=${pkg#${mlib}-}
 				if [ "$subst" != "$pkg" ]; then
-					feeds=$@
 					while [ -n "$1" ]; do
 						arch="$1"
 						arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
 					done
 					if [ "$pkg" = "$new_pkg" ]; then
 						# Failed to translate, package not found!
+						not_found="$not_found $pkg"
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				not_found="$not_found $pkg"
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+
+	# Parse the not_found items and see if they were dependencies (RPROVIDES)
+	# Follow the parsing example above...
+	for pkg in $not_found ; do
+		new_pkg="$pkg"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
+		grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+		prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
+		if [ -n "$prov" ]; then
+			# Nothing to do, already in the list
+			#echo "Skipping $pkg -> $prov, already in install set" >&2
+			continue
+		fi
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+					grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+					while [ -n "$1" ]; do
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						# Select first found, we don't know if one is better then another...
+						prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+						if [ -n "$prov" ]; then
+							new_pkg=$prov
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
 						echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
 						if [ "$attemptonly" = "Error" ]; then
 							exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
 		if [ "$pkg" = "$new_pkg" ]; then
 			default_archs_fixed=`echo "$default_archs" | tr - _`
 			for arch in $default_archs_fixed ; do
-				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
-					new_pkg="$pkg@$arch"
-					# First found is best match
+				# Select first found, we don't know if one is better then another...
+				prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+				if [ -n "$prov" ]; then
+					new_pkg=$prov
 					break
 				fi
 			done
Signature · 1 line
1.8.1.2.545.g2f19ada
#14 lore raw
In reply to Mark Hatle
Quoted 14 lines · Mark Hatle

A few misc fixes relating to toolchain and image generation.

The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.

The next fixes an issue w/ the previous binutils patch and upgrades.

Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.

The final patch fixes an issue w/ the image generatation. If the IMAGE_INSTALL contains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.

Quoted 8 lines

Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in the IMAGE_INSTALL line. The system will attempt to determine which to use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)

Which package backends has been tested with those changes?

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
author
#15 lore raw
Quoted 24 lines · Otavio Salvador
Quoted 14 lines · Mark Hatle

A few misc fixes relating to toolchain and image generation.

The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.

The next fixes an issue w/ the previous binutils patch and upgrades.

Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.

The final patch fixes an issue w/ the image generatation. If the IMAGE_INSTALL contains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.

Quoted 8 lines

Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in the IMAGE_INSTALL line. The system will attempt to determine which to use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)

Quoted 2 lines

Which package backends has been tested with those changes?

6/6 only affects RPM. So only RPM back end.

The other changes are generic and follow the existing models. However, I only tested the RPM backend.

(Note v2 of 5/6 and 6/6 has been sent. I accidentally sent a version prior to two minor revisions.)

--Mark

author
#16 lore raw
Quoted 25 lines · Mark Hatle
Quoted 24 lines · Otavio Salvador
Quoted 22 lines · Mark Hatle

A few misc fixes relating to toolchain and image generation.

The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.

The next fixes an issue w/ the previous binutils patch and upgrades.

Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.

The final patch fixes an issue w/ the image generatation. If the IMAGE_INSTALL contains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.

Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in the IMAGE_INSTALL line. The system will attempt to determine which to use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)

Quoted 7 lines
Quoted 2 lines

Which package backends has been tested with those changes?

6/6 only affects RPM. So only RPM back end.

The other changes are generic and follow the existing models. However, I only tested the RPM backend.

BTW I just verified that the 1-5/6 work properly with an opkg backend as well.

--Mark

Quoted 9 lines

(Note v2 of 5/6 and 6/6 has been sent. I accidentally sent a version prior to two minor revisions.)

--Mark
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

author
#17 lore raw

PATCH v2 5/6 packagegroup-self-hosted: Use packagegroup-core-buildessential

In reply to Mark Hatle

Instead of manually specifying build dependencies, use the packagegroup-core-buildessential. This ensures there is only one place that needs to be modified when toolchain items and dependencies change.

---
 meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb +1 -11
index 8e43201..e91a8d9 100644
@@ -67,33 +67,23 @@ RRECOMMENDS_packagegroup-self-hosted-host-tools = "\
 
 # eglibc-utils: for rpcgen
 RDEPENDS_packagegroup-self-hosted-sdk = "\
-    autoconf \
-    automake \
-    binutils \
     ccache \
     coreutils \
-    cpp \
     distcc \
     eglibc-utils \
     eglibc-gconv-ibm850 \
     file \
     findutils \
-    g++ \
-    gcc \
     intltool \
     ldd \
     less \
     libssp \
     libssp-dev \
     libssp-staticdev \
-    libstdc++ \
-    libstdc++-dev \
-    libtool \
-    make \
     mktemp \
+    packagegroup-core-buildessential \
     perl-module-re \
     perl-module-text-wrap \
-    pkgconfig \
     quilt \
     sed \
     "
Signature · 1 line
1.8.1.2.545.g2f19ada
author
#18 lore raw

PATCH v2 6/6 package_rpm: Allow translation of requirement to package name

In reply to Mark Hatle

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
 meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 75 insertions(+), 4 deletions(-)
meta/classes/package_rpm.bbclass +75 -4
index 31265d9..9fe0e6c 100644
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
 	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
 
 	pkgs_to_install=""
+	not_found=""
 	for pkg in "$@" ; do
 		new_pkg="$pkg"
 		if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
 				fi
 				subst=${pkg#${mlib}-}
 				if [ "$subst" != "$pkg" ]; then
-					feeds=$@
 					while [ -n "$1" ]; do
 						arch="$1"
 						arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
 					done
 					if [ "$pkg" = "$new_pkg" ]; then
 						# Failed to translate, package not found!
+						not_found="$not_found $pkg"
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				not_found="$not_found $pkg"
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+
+	# Parse the not_found items and see if they were dependencies (RPROVIDES)
+	# Follow the parsing example above...
+	for pkg in $not_found ; do
+		new_pkg="$pkg"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
+		grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+		prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
+		if [ -n "$prov" ]; then
+			# Nothing to do, already in the list
+			#echo "Skipping $pkg -> $prov, already in install set" >&2
+			continue
+		fi
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+					grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+					while [ -n "$1" ]; do
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						# Select first found, we don't know if one is better then another...
+						prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+						if [ -n "$prov" ]; then
+							new_pkg=$prov
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
 						echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
 						if [ "$attemptonly" = "Error" ]; then
 							exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
 		if [ "$pkg" = "$new_pkg" ]; then
 			default_archs_fixed=`echo "$default_archs" | tr - _`
 			for arch in $default_archs_fixed ; do
-				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
-					new_pkg="$pkg@$arch"
-					# First found is best match
+				# Select first found, we don't know if one is better then another...
+				prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+				if [ -n "$prov" ]; then
+					new_pkg=$prov
 					break
 				fi
 			done
Signature · 1 line
1.8.1.2.545.g2f19ada
#19 lore raw

Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name

Hi Mark,

Quoted 78 lines · On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
 meta/classes/package_rpm.bbclass | 79

++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)

(quoted diff) +0 -0

b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644

meta/classes/package_rpm.bbclass +0 -1
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
 	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart
--data-dir=${target_rootfs}/var/lib/smart query --output

${target_rootfs}/install/tmp/fullpkglist.query

meta/classes/package_rpm.bbclass +12 -1
 	pkgs_to_install=""
+	not_found=""
 	for pkg in "$@" ; do
 		new_pkg="$pkg"
 		if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
 				fi
 				subst=${pkg#${mlib}-}
 				if [ "$subst" != "$pkg" ]; then
-					feeds=$@
 					while [ -n "$1" ]; do
 						arch="$1"
 						arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
 					done
 					if [ "$pkg" = "$new_pkg" ]; then
 						# Failed to translate, package not found!
+						not_found="$not_found $pkg"
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'

${target_rootfs}/install/tmp/fullpkglist.query ; then

meta/classes/package_rpm.bbclass +20 -1
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				not_found="$not_found $pkg"
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+
+	# Parse the not_found items and see if they were dependencies (RPROVIDES)
+	# Follow the parsing example above...
+	for pkg in $not_found ; do
+		new_pkg="$pkg"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg
--output ${target_rootfs}/install/tmp/provide.query +		grep '^[^@
Quoted 5 lines

]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
$pkgs_to_install | xargs -n 1 echo | grep -f
${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"

];

Quoted 15 lines

then

meta/classes/package_rpm.bbclass +14 -0
+			# Nothing to do, already in the list
+			#echo "Skipping $pkg -> $prov, already in install set" >&2
+			continue
+		fi
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 10 lines
meta/classes/package_rpm.bbclass +9 -1
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					smart --data-dir=${target_rootfs}/var/lib/smart query
--provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+					grep '^[^@ ]*@[^@]*$' 

${target_rootfs}/install/tmp/provide.query |

Quoted 2 lines

sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + while [

-n "$1"

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 5 lines

]; do

meta/classes/package_rpm.bbclass +4 -0
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						# Select first found, we don't know if one is better then 

another...

Quoted 2 lines
meta/classes/package_rpm.bbclass +1 -0
+						prov=`grep '^[^@ ]*@'$arch'$'

${target_rootfs}/install/tmp/provide.query.list | head -n 1` +

if [ -n

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 8 lines

"$prov" ]; then

meta/classes/package_rpm.bbclass +6 -0
+							new_pkg=$prov
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
 						echo "$attemptonly: $pkg not found in the $mlib feeds 

($feeds)." >&2

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 11 lines
meta/classes/package_rpm.bbclass +0 -1
 						if [ "$attemptonly" = "Error" ]; then
 							exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
 		if [ "$pkg" = "$new_pkg" ]; then
 			default_archs_fixed=`echo "$default_archs" | tr - _`
 			for arch in $default_archs_fixed ; do
-				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'

${target_rootfs}/install/tmp/fullpkglist.query ; then

meta/classes/package_rpm.bbclass +1 -2
-					new_pkg="$pkg@$arch"
-					# First found is best match
+				# Select first found, we don't know if one is better then 

another...

Quoted 2 lines
meta/classes/package_rpm.bbclass +1 -0
+				prov=`grep '^[^@ ]*@'$arch'$'

${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if

[ -n

meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289

Quoted 5 lines

"$prov" ]; then

meta/classes/package_rpm.bbclass +1 -0
+					new_pkg=$prov
 					break
 				fi
 			done

Sigh... this code is getting ridiculously complicated. Shouldn't Smart be doing more of the heavy lifting here?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#20 lore raw

Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name

Quoted 147 lines · Paul Eggleton
Quoted 78 lines · On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
 meta/classes/package_rpm.bbclass | 79

++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)

(quoted diff) +0 -0

b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644

meta/classes/package_rpm.bbclass +0 -1
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
      [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart
--data-dir=${target_rootfs}/var/lib/smart query --output

${target_rootfs}/install/tmp/fullpkglist.query

      pkgs_to_install=""
meta/classes/package_rpm.bbclass +12 -1
+     not_found=""
      for pkg in "$@" ; do
              new_pkg="$pkg"
              if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
                              fi
                              subst=${pkg#${mlib}-}
                              if [ "$subst" != "$pkg" ]; then
-                                     feeds=$@
                                      while [ -n "$1" ]; do
                                              arch="$1"
                                              arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
                                      done
                                      if [ "$pkg" = "$new_pkg" ]; then
                                              # Failed to translate, package not found!
+                                             not_found="$not_found $pkg"
+                                             continue
+                                     fi
+                             fi
+                     done
+             fi
+             # Apparently not a multilib package...
+             if [ "$pkg" = "$new_pkg" ]; then
+                     default_archs_fixed=`echo "$default_archs" | tr - _`
+                     for arch in $default_archs_fixed ; do
+                             if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'

${target_rootfs}/install/tmp/fullpkglist.query ; then

meta/classes/package_rpm.bbclass +20 -1
+                                     new_pkg="$pkg@$arch"
+                                     # First found is best match
+                                     break
+                             fi
+                     done
+                     if [ "$pkg" = "$new_pkg" ]; then
+                             # Failed to translate, package not found!
+                             not_found="$not_found $pkg"
+                             continue
+                     fi
+             fi
+             #echo "$pkg -> $new_pkg" >&2
+             pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+     done
+
+     # Parse the not_found items and see if they were dependencies (RPROVIDES)
+     # Follow the parsing example above...
+     for pkg in $not_found ; do
+             new_pkg="$pkg"
+             smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg
--output ${target_rootfs}/install/tmp/provide.query +         grep '^[^@
Quoted 5 lines

]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
$pkgs_to_install | xargs -n 1 echo | grep -f
${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"

];

Quoted 15 lines

then

meta/classes/package_rpm.bbclass +14 -0
+                     # Nothing to do, already in the list
+                     #echo "Skipping $pkg -> $prov, already in install set" >&2
+                     continue
+             fi
+             if [ -z "$sdk_mode" ]; then
+                     for i in ${MULTILIB_PREFIX_LIST} ; do
+                             old_IFS="$IFS"
+                             IFS=":"
+                             set $i
+                             IFS="$old_IFS"
+                             mlib="$1"
+                             shift
+                             if [ "$mlib" = "default" ]; then
+                                     if [ -z "$default_archs" ]; then

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 10 lines
meta/classes/package_rpm.bbclass +9 -1
+                                             default_archs=$@
+                                     fi
+                                     continue
+                             fi
+                             subst=${pkg#${mlib}-}
+                             if [ "$subst" != "$pkg" ]; then
+                                     feeds=$@
+                                     smart --data-dir=${target_rootfs}/var/lib/smart query
--provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+                                     grep '^[^@ ]*@[^@]*$'

${target_rootfs}/install/tmp/provide.query |

Quoted 2 lines

sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + while [

-n "$1"

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 5 lines

]; do

meta/classes/package_rpm.bbclass +4 -0
+                                             arch="$1"
+                                             arch=`echo "$arch" | tr - _`
+                                             shift
+                                             # Select first found, we don't know if one is better then

another...

Quoted 2 lines

+ prov=`grep '^[^@ ]*@'$arch'$'
${target_rootfs}/install/tmp/provide.query.list | head -n 1` +

if [ -n

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 8 lines

"$prov" ]; then

meta/classes/package_rpm.bbclass +6 -0
+                                                     new_pkg=$prov
+                                                     break
+                                             fi
+                                     done
+                                     if [ "$pkg" = "$new_pkg" ]; then
+                                             # Failed to translate, package not found!
                                              echo "$attemptonly: $pkg not found in the $mlib feeds

($feeds)." >&2

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 11 lines

if [ "$attemptonly" = "Error" ]; then
exit 1

meta/classes/package_rpm.bbclass +0 -1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
              if [ "$pkg" = "$new_pkg" ]; then
                      default_archs_fixed=`echo "$default_archs" | tr - _`
                      for arch in $default_archs_fixed ; do
-                             if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'

${target_rootfs}/install/tmp/fullpkglist.query ; then

meta/classes/package_rpm.bbclass +1 -2
-                                     new_pkg="$pkg@$arch"
-                                     # First found is best match
+                             # Select first found, we don't know if one is better then

another...

Quoted 2 lines

+ prov=`grep '^[^@ ]*@'$arch'$'
${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if

[ -n

Quoted 8 lines
Quoted 5 lines

"$prov" ]; then
+ new_pkg=$prov
break
fi
done

Sigh... this code is getting ridiculously complicated. Shouldn't Smart be doing more of the heavy lifting here?

Agreed; this seems OE is doing too much.

Another think which bothers me is it being RPM specific ... :-(

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
author
#21 lore raw

Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name

In reply to Paul Eggleton
Quoted 40 lines · Paul Eggleton

Hi Mark,

Quoted 37 lines · On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
  meta/classes/package_rpm.bbclass | 79

++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)

(quoted diff) +0 -0

b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644

meta/classes/package_rpm.bbclass +0 -1
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
  	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart
--data-dir=${target_rootfs}/var/lib/smart query --output

${target_rootfs}/install/tmp/fullpkglist.query

pkgs_to_install=""

meta/classes/package_rpm.bbclass +1 -0
+	not_found=""
  	for pkg in "$@" ; do
  		new_pkg="$pkg"
  		if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
  				fi
  				subst=${pkg#${mlib}-}
  				if [ "$subst" != "$pkg" ]; then
Quoted 8 lines

meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 199

Quoted 8 lines
meta/classes/package_rpm.bbclass +0 -1
-					feeds=$@
  					while [ -n "$1" ]; do
  						arch="$1"
  						arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
  					done
  					if [ "$pkg" = "$new_pkg" ]; then
  						# Failed to translate, package not found!

Just to be clear the following diff is rather confusing.. there are two one line changes (the error to the no_found line below) in the existing code, and then the new chunk of code after both to process the not_found entries, which are presumed to be dependencies.)

If you apply this, or look at it in context it's easier to see..

Quoted 101 lines
Quoted 33 lines
meta/classes/package_rpm.bbclass +11 -0
+						not_found="$not_found $pkg"
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'

${target_rootfs}/install/tmp/fullpkglist.query ; then

meta/classes/package_rpm.bbclass +20 -1
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				not_found="$not_found $pkg"
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+
+	# Parse the not_found items and see if they were dependencies (RPROVIDES)
+	# Follow the parsing example above...
+	for pkg in $not_found ; do
+		new_pkg="$pkg"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg
--output ${target_rootfs}/install/tmp/provide.query +		grep '^[^@
Quoted 5 lines

]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
$pkgs_to_install | xargs -n 1 echo | grep -f
${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"

];

Quoted 15 lines

then

meta/classes/package_rpm.bbclass +14 -0
+			# Nothing to do, already in the list
+			#echo "Skipping $pkg -> $prov, already in install set" >&2
+			continue
+		fi
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 10 lines
meta/classes/package_rpm.bbclass +9 -1
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					smart --data-dir=${target_rootfs}/var/lib/smart query
--provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+					grep '^[^@ ]*@[^@]*$'

${target_rootfs}/install/tmp/provide.query |

Quoted 2 lines

sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + while [

-n "$1"

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 5 lines

]; do

meta/classes/package_rpm.bbclass +4 -0
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						# Select first found, we don't know if one is better then

another...

Quoted 2 lines
meta/classes/package_rpm.bbclass +1 -0
+						prov=`grep '^[^@ ]*@'$arch'$'

${target_rootfs}/install/tmp/provide.query.list | head -n 1` +

if [ -n

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 8 lines

"$prov" ]; then

meta/classes/package_rpm.bbclass +6 -0
+							new_pkg=$prov
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
  						echo "$attemptonly: $pkg not found in the $mlib feeds

($feeds)." >&2

meta/classes/package_rpm.bbclass translate_oe_to_smart() {

Quoted 11 lines

if [ "$attemptonly" = "Error" ]; then
exit 1

meta/classes/package_rpm.bbclass +0 -1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
  		if [ "$pkg" = "$new_pkg" ]; then
  			default_archs_fixed=`echo "$default_archs" | tr - _`
  			for arch in $default_archs_fixed ; do
-				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'

${target_rootfs}/install/tmp/fullpkglist.query ; then

meta/classes/package_rpm.bbclass +1 -2
-					new_pkg="$pkg@$arch"
-					# First found is best match
+				# Select first found, we don't know if one is better then

another...

Quoted 2 lines
meta/classes/package_rpm.bbclass +1 -0
+				prov=`grep '^[^@ ]*@'$arch'$'

${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if

[ -n

Quoted 8 lines

meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289

Quoted 5 lines

"$prov" ]; then

meta/classes/package_rpm.bbclass +1 -0
+					new_pkg=$prov
  					break
  				fi
  			done

Sigh... this code is getting ridiculously complicated. Shouldn't Smart be doing more of the heavy lifting here?

This function translate_oe_to_smart (and the corresponding translate_smart_to_oe) are used to switch from one name space to another. Remember in the RPM world, we don't use prefixes like "lib32-" on our package names, instead we use specific arch's that refer to specific ABI and processor tunings.

This is what enables the seamless multilib install on RPM.

What this code does is allows for dependencies to be added to the mix. If we didn't have multilibs then none of this code would be needed, but since we do -- the name space of both the package and dependencies need to be translate, not just packages as we'd done before.

Generally what this function does:

for each multilib prefx:
if PN starts w/ a known multilib prefix:
we translate it to: BPN@arch
check if the package exists
if it does add to pkg_list
else add to "not_found"
anything remaining is translated to the default arch
check if the package exists
if it does add to pkg_list
else add to "not_found"

if there are any "not_found", we assume they must be provides:
for each multilib prefix:
if the DEPENDENCY starts w/ a known multilib prefix:
we translate it without the prefix
ask smart what provides that dependency
check if something already in the pkg_list is there, if so skip it else use priority ordering to select the best provider -- add to pkg_list
for anything w/o a multilib prefix, follow the same steps

The reverse smart_to_oe function doesn't ever need to pay attention to dependencies so that is not an issue, and the translation is significantly more straight forward.

If we had unique namespaces in oe, and didn't use the lib* markers on multilibs.. then none of this would be necessary.. (but then of course dpkg and opkg wouldn't work properly.) So the translation is a necessary evil.

Could it be done differently, most likely. Perhaps identifying a way to do it in python would be quicker.. but it's actually pretty fast sine the majority of translations happen via a static list and a single grep. It's only the (hopefully) small number of dependencies that happen via individual calls to smart which impose a time penalty.

--Mark

Quoted 3 lines

Cheers,
Paul

#22 lore raw

Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name

In reply to Mark Hatle
Quoted 3 lines · Mark Hatle

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL.

For what its worth, I find this description hard to understand, I think as you leave the keyword "provide" right until the end. A mention of RPROVIDES earlier here would help a lot.

Quoted 117 lines

So on a failure to translate a package name, we fall back
and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
 meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 75 insertions(+), 4 deletions(-)
meta/classes/package_rpm.bbclass +71 -1
index 31265d9..9fe0e6c 100644
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
 	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
 
 	pkgs_to_install=""
+	not_found=""
 	for pkg in "$@" ; do
 		new_pkg="$pkg"
 		if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
 				fi
 				subst=${pkg#${mlib}-}
 				if [ "$subst" != "$pkg" ]; then
-					feeds=$@
 					while [ -n "$1" ]; do
 						arch="$1"
 						arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
 					done
 					if [ "$pkg" = "$new_pkg" ]; then
 						# Failed to translate, package not found!
+						not_found="$not_found $pkg"
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				not_found="$not_found $pkg"
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+
+	# Parse the not_found items and see if they were dependencies (RPROVIDES)
+	# Follow the parsing example above...
+	for pkg in $not_found ; do
+		new_pkg="$pkg"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
+		grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+		prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
+		if [ -n "$prov" ]; then
+			# Nothing to do, already in the list
+			#echo "Skipping $pkg -> $prov, already in install set" >&2
+			continue
+		fi
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+					grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+					while [ -n "$1" ]; do
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						# Select first found, we don't know if one is better then another...
+						prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+						if [ -n "$prov" ]; then
+							new_pkg=$prov
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
 						echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
 						if [ "$attemptonly" = "Error" ]; then
 							exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
 		if [ "$pkg" = "$new_pkg" ]; then
 			default_archs_fixed=`echo "$default_archs" | tr - _`
 			for arch in $default_archs_fixed ; do

meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289

Quoted 10 lines
meta/classes/package_rpm.bbclass +4 -3
-				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
-					new_pkg="$pkg@$arch"
-					# First found is best match
+				# Select first found, we don't know if one is better then another...
+				prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+				if [ -n "$prov" ]; then
+					new_pkg=$prov
 					break
 				fi
 			done

This 'shell calling rpm commands' code with all the mangling is near impossible to read/understand. This patch just makes the mess more complicated and harder to follow.

What is the plan for turning this into something better?

My instinct says more functions and a language better at data manipulation (python instead of shell) might start to resolve this.

One of the reasons for using smart was that we could probably enhance it to better suit our needs. Can we consider that too?

I really don't want to see more patches like this one piling on top of already crumbling foundations.

Cheers,

Richard

author
#23 lore raw

Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name

Quoted 8 lines · Richard Purdie
Quoted 3 lines · Mark Hatle

In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL.

For what its worth, I find this description hard to understand, I think as you leave the keyword "provide" right until the end. A mention of RPROVIDES earlier here would help a lot.

Ok.

Quoted 123 lines
Quoted 123 lines

So on a failure to translate a package name, we fall back
and attempt to resolve based on a package's provide.

Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.

If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.

---
  meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
  1 file changed, 75 insertions(+), 4 deletions(-)
meta/classes/package_rpm.bbclass +74 -4
index 31265d9..9fe0e6c 100644
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
  	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
 
  	pkgs_to_install=""
+	not_found=""
  	for pkg in "$@" ; do
  		new_pkg="$pkg"
  		if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
  				fi
  				subst=${pkg#${mlib}-}
  				if [ "$subst" != "$pkg" ]; then
-					feeds=$@
  					while [ -n "$1" ]; do
  						arch="$1"
  						arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
  					done
  					if [ "$pkg" = "$new_pkg" ]; then
  						# Failed to translate, package not found!
+						not_found="$not_found $pkg"
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				not_found="$not_found $pkg"
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+
+	# Parse the not_found items and see if they were dependencies (RPROVIDES)
+	# Follow the parsing example above...
+	for pkg in $not_found ; do
+		new_pkg="$pkg"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
+		grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+		prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
+		if [ -n "$prov" ]; then
+			# Nothing to do, already in the list
+			#echo "Skipping $pkg -> $prov, already in install set" >&2
+			continue
+		fi
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+					grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+					while [ -n "$1" ]; do
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						# Select first found, we don't know if one is better then another...
+						prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+						if [ -n "$prov" ]; then
+							new_pkg=$prov
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
  						echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
  						if [ "$attemptonly" = "Error" ]; then
  							exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
  		if [ "$pkg" = "$new_pkg" ]; then
  			default_archs_fixed=`echo "$default_archs" | tr - _`
  			for arch in $default_archs_fixed ; do
-				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
-					new_pkg="$pkg@$arch"
-					# First found is best match
+				# Select first found, we don't know if one is better then another...
+				prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+				if [ -n "$prov" ]; then
Quoted 8 lines

meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289

Quoted 4 lines
meta/classes/package_rpm.bbclass +1 -0
+					new_pkg=$prov
  					break
  				fi
  			done

This 'shell calling rpm commands' code with all the mangling is near impossible to read/understand. This patch just makes the mess more complicated and harder to follow.

The root of this is the namespace issue. In RPM, the architecture (and arch compatibility), as well as file contents all provide the namespace and conflict resolution. While in opkg/deb the name of the package itself is where the namespace is defined. (i.e. lib32-...)

The translation is necessary due to this multilib support, otherwise we wouldn't need to do any of it. Somehow we have to translate one namespace to another -- and since the translation can't occur until the packages are built, it'd be difficult to do it 'earlier'.

Quoted 7 lines

What is the plan for turning this into something better?

My instinct says more functions and a language better at data manipulation (python instead of shell) might start to resolve this.

One of the reasons for using smart was that we could probably enhance it to better suit our needs. Can we consider that too?

The only thing we can do is build a python (or similar) app that takes the inputs of the translation 'namespace' IDs and the corresponding architectures, and RPM feed. Then translate from one to the other. My biggest concern with this translation is to cover all of the corner cases we could start getting more and more complex.

Quoted 2 lines

I really don't want to see more patches like this one piling on top of already crumbling foundations.

Easiest way to avoid this is to make dependencies in the IMAGE/PACKAGE_INSTALL forbidden and do the translation at a higher level generically. But frankly I don't see that happening -- so the solution becomes RPM specific and we end up with code like this. I'm not sure if we can move this code into a python function -- or if we need to simply create an external translation program...

As far as plans, frankly I don't have any plans to focus on this work for the time being. There are too many other things I need to get working properly first. If someone else wants to step in and work on a solution, I'm more then happy to help them out.

--Mark

Quoted 4 lines

Cheers,

Richard

author
#24 lore raw

PATCH v3 2/6 gcc: Drop *-symlinks

In reply to Mark Hatle

The -symlinks packages don't really add any value if we're using update-alternatives. Drop it, leaving an RPROVIDES in case anyone thought they needed it still.

---
 meta/recipes-devtools/gcc/gcc-target.inc | 54 ++++++++++++++++++--------------
 1 file changed, 30 insertions(+), 24 deletions(-)
meta/recipes-devtools/gcc/gcc-target.inc +30 -24
index ee41d3f..019dbc0 100644
@@ -10,12 +10,12 @@ EXTRA_OECONF_PATHS = " \
 ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR} -I${B}/gcc/include/ "
 
 PACKAGES = "\
-  ${PN} ${PN}-plugins ${PN}-symlinks \
-  g++ g++-symlinks \
-  cpp cpp-symlinks \
-  g77 g77-symlinks \
-  gfortran gfortran-symlinks \
-  gcov gcov-symlinks \
+  ${PN} ${PN}-plugins \
+  g++ \
+  cpp \
+  g77 \
+  gfortran \
+  gcov \
   ${PN}-plugin-dev \
   ${PN}-doc \
   ${PN}-dev \
@@ -36,6 +36,9 @@ FILES_${PN} = "\
   ${gcclibdir}/${TARGET_SYS}/${BINV}/include-fixed \
 "
 INSANE_SKIP_${PN} += "dev-so"
+RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
+RCONFLICTS_${PN} += "${PN}-symlinks"
 
 FILES_${PN}-dbg += "\
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/.debug/ \
@@ -50,8 +53,6 @@ FILES_${PN}-plugin-dev = "\
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \
   ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \
 "
-FILES_${PN}-symlinks = ""
-ALLOW_EMPTY_${PN}-symlinks = "1"
 
 FILES_${PN}-plugins = "\
   ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -62,33 +63,38 @@ FILES_g77 = "\
   ${bindir}/${TARGET_PREFIX}g77 \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \
 "
-FILES_g77-symlinks = ""
-ALLOW_EMPTY_g77-symlinks = "1"
+RPROVIDES_g77 += "g77-symlinks"
+RREPLACES_g77 += "g77-symlinks"
+RCONFLICTS_g77 += "${PN}-symlinks"
 
 FILES_gfortran = "\
   ${bindir}/${TARGET_PREFIX}gfortran \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \
 "
-FILES_gfortran-symlinks = ""
-ALLOW_EMPTY_gfortran-symlinks = "1"
+RPROVIDES_gfortran += "gfortran-symlinks"
+RREPLACES_gfortran += "gfortran-symlinks"
+RCONFLICTS_gfortran += "${PN}-symlinks"
 
 FILES_cpp = "\
   ${bindir}/${TARGET_PREFIX}cpp \
   ${base_libdir}/cpp \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1"
-FILES_cpp-symlinks = ""
-ALLOW_EMPTY_cpp-symlinks = "1"
+RPROVIDES_cpp += "cpp-symlinks"
+RREPLACES_cpp += "cpp-symlinks"
+RCONFLICTS_cpp += "${PN}-symlinks"
 
 FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov"
-FILES_gcov-symlinks = ""
-ALLOW_EMPTY_gconv-symlinks = "1"
+RPROVIDES_gcov += "gcov-symlinks"
+RREPLACES_gcov += "gcov-symlinks"
+RCONFLICTS_gcov += "${PN}-symlinks"
 
 FILES_g++ = "\
   ${bindir}/${TARGET_PREFIX}g++ \
   ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
 "
-FILES_g++-symlinks = ""
-ALLOW_EMPTY_g++-symlinks = "1"
+RPROVIDES_g++ += "g++-symlinks"
+RREPLACES_g++ += "g++-symlinks"
+RCONFLICTS_g++ += "${PN}-symlinks"
 
 FILES_${PN}-doc = "\
   ${infodir} \
@@ -137,12 +143,12 @@ inherit update-alternatives
 
 ALTERNATIVE_PRIORITY = "100"
 
-ALTERNATIVE_${PN}-symlinks = "cc gcc"
-ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_cpp-symlinks = "cpp"
-ALTERNATIVE_gcov-symlinks = "gcov"
-ALTERNATIVE_g++-symlinks = "c++ g++"
+ALTERNATIVE_${PN} = "cc gcc"
+ALTERNATIVE_g77 = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_gfortran = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_cpp = "cpp"
+ALTERNATIVE_gcov = "gcov"
+ALTERNATIVE_g++ = "c++ g++"
 
 ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc"
 ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
Signature · 1 line
1.8.3.4
author
#25 lore raw

PATCH v3 3/6 binutils: Ensure old -symlinks packages get removed

In reply to Mark Hatle

When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.

---
 meta/recipes-devtools/binutils/binutils.inc | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/binutils/binutils.inc +2 -0
index 17c66bc..77b3718 100644
@@ -21,6 +21,8 @@ FILES_${PN} = " \
 	${prefix}/${TARGET_SYS}/bin/*"
 
 RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
+RCONFLICTS_${PN} += "${PN}-symlinks"
 
 FILES_${PN}-dev = " \
 	${includedir} \
Signature · 1 line
1.8.3.4
#26 lore raw
In reply to Mark Hatle
Quoted 27 lines · Mark Hatle

A few misc fixes relating to toolchain and image generation.

The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.

The next fixes an issue w/ the previous binutils patch and upgrades.

Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.

The final patch fixes an issue w/ the image generatation. If the IMAGE_INSTALL contains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.

Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in the IMAGE_INSTALL line. The system will attempt to determine which to use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)

The following changes since commit 616354f13732d13c17434d5b60b166f691c25761:

   binutils: Add gnu-config-native to DEPENDS (2013-11-12 16:00:20 +0000)
Quoted 8 lines

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib mhatle/misc1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/misc1

Mark Hatle (6):
gcc: Use alternatives for the *-symlinks packages.
gcc: Drop *-symlinks

Mark

This change set seems to be giving the Autobuilder some issues, please take a look at the AB (for example: http://autobuilder.yoctoproject.org:8011/builders/nightly-mips/builds/1/steps/Building%20Images/logs/stdio

Thanks
Sau!

Quoted 12 lines

binutils: Ensure old -symlinks packages get removed
packagegroups: Remove toolchain *-symlinks packages
packagegroup-self-hosted: Use packagegroup-core-buildessentials
package_rpm: Allow translation of requirement to package name

  meta/classes/package_rpm.bbclass                   |  79 +++++++++++++++-
  .../packagegroup-core-buildessential.bb            |   4 -
  .../packagegroups/packagegroup-self-hosted.bb      |  16 +---
  meta/recipes-devtools/binutils/binutils.inc        |   1 +
  meta/recipes-devtools/gcc/gcc-target.inc           | 103 +++++++++++++--------
  5 files changed, 141 insertions(+), 62 deletions(-)