Misc changes relating to toolchain and image gen
cover Misc changes relating to toolchain and image gen
A few misc fixes relating to toolchain and image generation.
The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.
The next fixes an issue w/ the previous binutils patch and upgrades.
Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.
The final patch fixes an issue w/ the image generatation. If the IMAGE_INSTALL contains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.
Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in the IMAGE_INSTALL line. The system will attempt to determine which to use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)
The following changes since commit 616354f13732d13c17434d5b60b166f691c25761:
binutils: Add gnu-config-native to DEPENDS (2013-11-12 16:00:20 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib mhatle/misc1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/misc1
Mark Hatle (6):
gcc: Use alternatives for the *-symlinks packages.
gcc: Drop *-symlinks
binutils: Ensure old -symlinks packages get removed
packagegroups: Remove toolchain *-symlinks packages
packagegroup-self-hosted: Use packagegroup-core-buildessentials
package_rpm: Allow translation of requirement to package name
meta/classes/package_rpm.bbclass | 79 +++++++++++++++- .../packagegroup-core-buildessential.bb | 4 - .../packagegroups/packagegroup-self-hosted.bb | 16 +--- meta/recipes-devtools/binutils/binutils.inc | 1 + meta/recipes-devtools/gcc/gcc-target.inc | 103 +++++++++++++-------- 5 files changed, 141 insertions(+), 62 deletions(-)
Signature · 1 line
1.8.1.2.545.g2f19ada
PATCH 1/6 gcc: Use alternatives for the *-symlinks packages.
The various gcc related symlinks should be provided as alternatives instead of hard coded symlinks. This will permit multiple toolchains on a system.
Multiple toolchains could come from multilib configurations or alternative open source or commerical sources.
Note, gccbug was skipped since it doesn't seem to be generated anymore.
--- meta/recipes-devtools/gcc/gcc-target.inc | 91 ++++++++++++++++++++------------ 1 file changed, 58 insertions(+), 33 deletions(-)
meta/recipes-devtools/gcc/gcc-target.inc +58 -33
@@ -50,11 +50,8 @@ FILES_${PN}-plugin-dev = "\ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \ ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \ " -FILES_${PN}-symlinks = "\ - ${bindir}/cc \ - ${bindir}/gcc \ - ${bindir}/gccbug \ -" +FILES_${PN}-symlinks = "" +ALLOW_EMPTY_${PN}-symlinks = "1" FILES_${PN}-plugins = "\ ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -65,36 +62,33 @@ FILES_g77 = "\ ${bindir}/${TARGET_PREFIX}g77 \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \ " -FILES_g77-symlinks = "\ - ${bindir}/g77 \ - ${bindir}/f77 \ -" +FILES_g77-symlinks = "" +ALLOW_EMPTY_g77-symlinks = "1" + FILES_gfortran = "\ ${bindir}/${TARGET_PREFIX}gfortran \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \ " -FILES_gfortran-symlinks = "\ - ${bindir}/gfortran \ - ${bindir}/f95" +FILES_gfortran-symlinks = "" +ALLOW_EMPTY_gfortran-symlinks = "1" FILES_cpp = "\ ${bindir}/${TARGET_PREFIX}cpp \ ${base_libdir}/cpp \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1" -FILES_cpp-symlinks = "${bindir}/cpp" +FILES_cpp-symlinks = "" +ALLOW_EMPTY_cpp-symlinks = "1" FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov" -FILES_gcov-symlinks = "${bindir}/gcov" +FILES_gcov-symlinks = "" +ALLOW_EMPTY_gconv-symlinks = "1" FILES_g++ = "\ ${bindir}/${TARGET_PREFIX}g++ \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \ " -FILES_g++-symlinks = "\ - ${bindir}/c++ \ - ${bindir}/g++ \ -" - +FILES_g++-symlinks = "" +ALLOW_EMPTY_g++-symlinks = "1" FILES_${PN}-doc = "\ ${infodir} \
@@ -133,22 +127,53 @@ do_install () { # Not sure why we end up with these but we don't want them... rm -f ${TARGET_PREFIX}${TARGET_PREFIX}* - # Symlinks so we can use these trivially on the target - if [ -e ${TARGET_PREFIX}g77 ]; then - ln -sf ${TARGET_PREFIX}g77 g77 || true - ln -sf g77 f77 || true - fi - if [ -e ${TARGET_PREFIX}gfortran ]; then - ln -sf ${TARGET_PREFIX}gfortran gfortran || true - ln -sf gfortran f95 || true - fi - ln -sf ${TARGET_PREFIX}g++ g++ - ln -sf ${TARGET_PREFIX}gcc gcc - ln -sf ${TARGET_PREFIX}cpp cpp install -d ${D}${base_libdir} ln -sf ${bindir}/${TARGET_PREFIX}cpp ${D}${base_libdir}/cpp - ln -sf g++ c++ - ln -sf gcc cc chown -R root:root ${D} } + +inherit update-alternatives + +ALTERNATIVE_PRIORITY = "100" + +ALTERNATIVE_${PN}-symlinks = "cc gcc" +ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}" +ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}" +ALTERNATIVE_cpp-symlinks = "cpp" +ALTERNATIVE_gcov-symlinks = "gcov" +ALTERNATIVE_g++-symlinks = "c++ g++" + +ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc" +ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc" + +ALTERNATIVE_LINK_NAME[gcc] = "${bindir}/gcc" +ALTERNATIVE_TARGET[gcc] = "${bindir}/${TARGET_PREFIX}gcc" + +# Not really generated any longer, avoid warnings... +#ALTERNATIVE_LINK_NAME[gccbug] = "${bindir}/gccbug" +#ALTERNATIVE_TARGET[gccbug] = "${bindir}/${TARGET_PREFIX}gccbug" + +ALTERNATIVE_LINK_NAME[g77] = "${bindir}/g77" +ALTERNATIVE_TARGET[g77] = "${bindir}/${TARGET_PREFIX}g77" + +ALTERNATIVE_LINK_NAME[f77] = "${bindir}/f77" +ALTERNATIVE_TARGET[f77] = "${bindir}/${TARGET_PREFIX}g77" + +ALTERNATIVE_LINK_NAME[gfortran] = "${bindir}/gfortran" +ALTERNATIVE_TARGET[gfortran] = "${bindir}/${TARGET_PREFIX}gfortran" + +ALTERNATIVE_LINK_NAME[f95] = "${bindir}/f95" +ALTERNATIVE_TARGET[f95] = "${bindir}/${TARGET_PREFIX}gfortran" + +ALTERNATIVE_LINK_NAME[cpp] = "${bindir}/cpp" +ALTERNATIVE_TARGET[cpp] = "${bindir}/${TARGET_PREFIX}cpp" + +ALTERNATIVE_LINK_NAME[gcov] = "${bindir}/gcov" +ALTERNATIVE_TARGET[gcov] = "${bindir}/${TARGET_PREFIX}gcov" + +ALTERNATIVE_LINK_NAME[c++] = "${bindir}/c++" +ALTERNATIVE_TARGET[c++] = "${bindir}/${TARGET_PREFIX}g++" + +ALTERNATIVE_LINK_NAME[g++] = "${bindir}/g++" +ALTERNATIVE_TARGET[g++] = "${bindir}/${TARGET_PREFIX}g++"
Signature · 1 line
1.8.1.2.545.g2f19ada
Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
Quoted 9 lines · Mark Hatle
The various gcc related symlinks should be provided as alternatives instead of hard coded symlinks. This will permit multiple toolchains on a system.
Multiple toolchains could come from multilib configurations or alternative open source or commerical sources.
Note, gccbug was skipped since it doesn't seem to be generated anymore.
Signed-off-by Mark Hatle <[email protected]>
Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic change'? You add code to remove in patch 2, I think the end patch would be easier to review, no?
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
Quoted 7 lines · Otavio Salvador
Quoted 6 lines · Mark Hatle
The various gcc related symlinks should be provided as alternatives instead of hard coded symlinks. This will permit multiple toolchains on a system.
Multiple toolchains could come from multilib configurations or alternative open source or commerical sources.
Quoted 8 lines
Quoted 3 lines
Note, gccbug was skipped since it doesn't seem to be generated anymore.
Signed-off-by Mark Hatle <[email protected]>Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic change'? You add code to remove in patch 2, I think the end patch would be easier to review, no?
It was done this way to match how binutils was implemented. These are actually two logically separate patches. The first switches from hard coded symlinks to using update-alternatives. The second patch says that a separate -symlinks package is no longer needed.
See binutils:
1395aefcaeac94dd0e6ed3a718b7e58dd43b355e24093e26f246f222c385dc37a2f8cf8b0f183175
(The second of the patches can be reverted -- if ever needed -- and the update alternatives functionality will still work properly.)
--Mark
Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
Quoted 29 lines · Mark Hatle
Quoted 13 lines · Otavio Salvador
Quoted 10 lines · Mark Hatle
The various gcc related symlinks should be provided as alternatives instead
of hard coded symlinks. This will permit multiple toolchains on a system.Multiple toolchains could come from multilib configurations or alternative
open source or commerical sources.Quoted 9 lines
Quoted 3 lines
Note, gccbug was skipped since it doesn't seem to be generated anymore.
Signed-off-by Mark Hatle <[email protected]>Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic change'? You add code to remove in patch 2, I think the end patch would be easier to review, no?
It was done this way to match how binutils was implemented. These are actually two logically separate patches. The first switches from hard coded symlinks to using update-alternatives. The second patch says that a separate -symlinks package is no longer needed.
Quoted 7 lines
See binutils:
1395aefcaeac94dd0e6ed3a718b7e58dd43b355e24093e26f246f222c385dc37a2f8cf8b0f183175(The second of the patches can be reverted -- if ever needed -- and the update alternatives functionality will still work properly.)
Ok; you got a point :-)
Agreed.
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
PATCH 2/6 gcc: Drop *-symlinks
In reply to Mark HatleThe -symlinks packages don't really add any value if we're using update-alternatives. Drop it, leaving an RPROVIDES in case anyone thought they needed it still.
--- meta/recipes-devtools/gcc/gcc-target.inc | 48 ++++++++++++++++---------------- 1 file changed, 24 insertions(+), 24 deletions(-)
meta/recipes-devtools/gcc/gcc-target.inc +24 -24
@@ -10,12 +10,12 @@ EXTRA_OECONF_PATHS = " \ ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR} -I${B}/gcc/include/ " PACKAGES = "\ - ${PN} ${PN}-plugins ${PN}-symlinks \ - g++ g++-symlinks \ - cpp cpp-symlinks \ - g77 g77-symlinks \ - gfortran gfortran-symlinks \ - gcov gcov-symlinks \ + ${PN} ${PN}-plugins \ + g++ \ + cpp \ + g77 \ + gfortran \ + gcov \ ${PN}-plugin-dev \ ${PN}-doc \ ${PN}-dev \
@@ -36,6 +36,8 @@ FILES_${PN} = "\ ${gcclibdir}/${TARGET_SYS}/${BINV}/include-fixed \ " INSANE_SKIP_${PN} += "dev-so" +RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks" FILES_${PN}-dbg += "\ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/.debug/ \
@@ -50,8 +52,6 @@ FILES_${PN}-plugin-dev = "\ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \ ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \ " -FILES_${PN}-symlinks = "" -ALLOW_EMPTY_${PN}-symlinks = "1" FILES_${PN}-plugins = "\ ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -62,33 +62,33 @@ FILES_g77 = "\ ${bindir}/${TARGET_PREFIX}g77 \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \ " -FILES_g77-symlinks = "" -ALLOW_EMPTY_g77-symlinks = "1" +RPROVIDES_g77 += "g77-symlinks" +RREPLACES_g77 += "g77-symlinks" FILES_gfortran = "\ ${bindir}/${TARGET_PREFIX}gfortran \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \ " -FILES_gfortran-symlinks = "" -ALLOW_EMPTY_gfortran-symlinks = "1" +RPROVIDES_gfortran += "gfortran-symlinks" +RREPLACES_gfortran += "gfortran-symlinks" FILES_cpp = "\ ${bindir}/${TARGET_PREFIX}cpp \ ${base_libdir}/cpp \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1" -FILES_cpp-symlinks = "" -ALLOW_EMPTY_cpp-symlinks = "1" +RPROVIDES_cpp += "cpp-symlinks" +RREPLACES_cpp += "cpp-symlinks" FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov" -FILES_gcov-symlinks = "" -ALLOW_EMPTY_gconv-symlinks = "1" +RPROVIDES_gcov += "gcov-symlinks" +RREPLACES_gcov += "gcov-symlinks" FILES_g++ = "\ ${bindir}/${TARGET_PREFIX}g++ \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \ " -FILES_g++-symlinks = "" -ALLOW_EMPTY_g++-symlinks = "1" +RPROVIDES_g++ += "g++-symlinks" +RREPLACES_g++ += "g++-symlinks" FILES_${PN}-doc = "\ ${infodir} \
@@ -137,12 +137,12 @@ inherit update-alternatives ALTERNATIVE_PRIORITY = "100" -ALTERNATIVE_${PN}-symlinks = "cc gcc" -ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}" -ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}" -ALTERNATIVE_cpp-symlinks = "cpp" -ALTERNATIVE_gcov-symlinks = "gcov" -ALTERNATIVE_g++-symlinks = "c++ g++" +ALTERNATIVE_${PN} = "cc gcc" +ALTERNATIVE_g77 = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}" +ALTERNATIVE_gfortran = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}" +ALTERNATIVE_cpp = "cpp" +ALTERNATIVE_gcov = "gcov" +ALTERNATIVE_g++ = "c++ g++" ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc" ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
Signature · 1 line
1.8.1.2.545.g2f19ada
PATCH 3/6 binutils: Ensure old -symlinks packages get removed
In reply to Mark HatleWhen the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.
--- meta/recipes-devtools/binutils/binutils.inc | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/binutils/binutils.inc +1 -0
@@ -21,6 +21,7 @@ FILES_${PN} = " \ ${prefix}/${TARGET_SYS}/bin/*" RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks" FILES_${PN}-dev = " \ ${includedir} \
Signature · 1 line
1.8.1.2.545.g2f19ada
Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
Quoted 13 lines · Mark Hatle
When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.
Signed-off-by Mark Hatle <[email protected]>--- meta/recipes-devtools/binutils/binutils.inc | 1 + 1 file changed, 1 insertion(+)meta/recipes-devtools/binutils/binutils.inc +0
-0
meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24
Quoted 4 lines
meta/recipes-devtools/binutils/binutils.inc +1
-0@@ -21,6 +21,7 @@ FILES_${PN} = " \ ${prefix}/${TARGET_SYS}/bin/*" RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks"
IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.
Quoted 10 lines
meta/recipes-devtools/binutils/binutils.inc +0
-0FILES_${PN}-dev = " \ ${includedir} \Signature · 6 lines
1.8.1.2.545.g2f19ada _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
Quoted 16 lines · Martin Jansa
Quoted 15 lines · Mark Hatle
When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.
Signed-off-by Mark Hatle <[email protected]>--- meta/recipes-devtools/binutils/binutils.inc | 1 + 1 file changed, 1 insertion(+)meta/recipes-devtools/binutils/binutils.inc +0
-0@@ -21,6 +21,7 @@ FILES_${PN} = " \ ${prefix}/${TARGET_SYS}/bin/*"
Quoted 5 lines
meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24
Quoted 2 lines
meta/recipes-devtools/binutils/binutils.inc +1
-0RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks"IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.
Also for dpkg. If you have Replaces: without Conflicts: then this just means that the replacing package is allowed to overwrite files in the replaced one, not that the replaced one will actually be uninstalled.
p.
Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
Quoted 23 lines · Phil Blundell
Quoted 16 lines · Martin Jansa
Quoted 15 lines · Mark Hatle
When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.
Signed-off-by Mark Hatle <[email protected]>--- meta/recipes-devtools/binutils/binutils.inc | 1 + 1 file changed, 1 insertion(+)meta/recipes-devtools/binutils/binutils.inc +0
-0@@ -21,6 +21,7 @@ FILES_${PN} = " \ ${prefix}/${TARGET_SYS}/bin/*"Quoted 5 lines
meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24
Quoted 2 lines
meta/recipes-devtools/binutils/binutils.inc +1
-0RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks"IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.
Quoted 3 lines
Also for dpkg. If you have Replaces: without Conflicts: then this just means that the replacing package is allowed to overwrite files in the replaced one, not that the replaced one will actually be uninstalled.
This seems incredibly strange to me. But I can adjust the 2/6 and 3/6 patches by adding the appropriate RCONFLICTS.
--Mark
Quoted 3 lines
p.
Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
Quoted 24 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 21 lines · Martin Jansa
Quoted 13 lines · Mark Hatle
When the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.
Signed-off-by Mark Hatle <[email protected]>--- meta/recipes-devtools/binutils/binutils.inc | 1 + 1 file changed, 1 insertion(+)meta/recipes-devtools/binutils/binutils.inc +0
-0meta/recipes-devtools/binutils/binutils.inc FILES_${PN} = " \ · line 24
Quoted 4 lines
meta/recipes-devtools/binutils/binutils.inc +1
-0@@ -21,6 +21,7 @@ FILES_${PN} = " \ ${prefix}/${TARGET_SYS}/bin/*" RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks"IIRC for opkg backend you need also RCONFLICTS for opkg to really remove old ${PN}-symlinks.
Quoted 5 lines
Quoted 3 lines
Also for dpkg. If you have Replaces: without Conflicts: then this just means that the replacing package is allowed to overwrite files in the replaced one, not that the replaced one will actually be uninstalled.
This seems incredibly strange to me.
Well, perhaps, but it's always been that way. See the documentation for Replaces:
http://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces
p.
PATCH 4/6 packagegroups: Remove toolchain *-symlinks packages
In reply to Mark HatleWe are no longer producing the *-symlinks packages, so remove the dependencies to those items.
--- meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb | 4 ---- meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 4 ---- 2 files changed, 8 deletions(-)
meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb +0 -4
@@ -12,13 +12,9 @@ RDEPENDS_packagegroup-core-buildessential = "\ autoconf \ automake \ binutils \ - binutils-symlinks \ cpp \ - cpp-symlinks \ gcc \ - gcc-symlinks \ g++ \ - g++-symlinks \ gettext \ make \ libstdc++ \
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb +0 -4
@@ -70,20 +70,16 @@ RDEPENDS_packagegroup-self-hosted-sdk = "\ autoconf \ automake \ binutils \ - binutils-symlinks \ ccache \ coreutils \ cpp \ - cpp-symlinks \ distcc \ eglibc-utils \ eglibc-gconv-ibm850 \ file \ findutils \ g++ \ - g++-symlinks \ gcc \ - gcc-symlinks \ intltool \ ldd \ less \
Signature · 1 line
1.8.1.2.545.g2f19ada
PATCH 5/6 packagegroup-self-hosted: Use packagegroup-core-buildessentials
In reply to Mark HatleInstead of manually specifying build dependencies, use the packagegroup-core-buildessentials. This ensures there is only one place that needs to be modified when toolchain items and dependencies change.
--- meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-)
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb +1 -11
@@ -67,33 +67,23 @@ RRECOMMENDS_packagegroup-self-hosted-host-tools = "\ # eglibc-utils: for rpcgen RDEPENDS_packagegroup-self-hosted-sdk = "\ - autoconf \ - automake \ - binutils \ ccache \ coreutils \ - cpp \ distcc \ eglibc-utils \ eglibc-gconv-ibm850 \ file \ findutils \ - g++ \ - gcc \ intltool \ ldd \ less \ libssp \ libssp-dev \ libssp-staticdev \ - libstdc++ \ - libstdc++-dev \ - libtool \ - make \ mktemp \ + packagegroup-core-buildessential \ perl-module-re \ perl-module-text-wrap \ - pkgconfig \ quilt \ sed \ "
Signature · 1 line
1.8.1.2.545.g2f19ada
PATCH 6/6 package_rpm: Allow translation of requirement to package name
In reply to Mark HatleIn the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.
Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
--- meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)
meta/classes/package_rpm.bbclass +75 -4
@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query pkgs_to_install="" + not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then - feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then + new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov" ]; then + # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; then + default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + while [ -n "$1" ]; do + arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2 if [ "$attemptonly" = "Error" ]; then exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then - new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov break fi done
Signature · 1 line
1.8.1.2.545.g2f19ada
Quoted 14 lines · Mark Hatle
A few misc fixes relating to toolchain and image generation.
The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.
The next fixes an issue w/ the previous binutils patch and upgrades.
Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.
The final patch fixes an issue w/ the image generatation. If the
IMAGE_INSTALLcontains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.
Quoted 8 lines
Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the
IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in theIMAGE_INSTALLline. The system will attempt to determine which to use, by checking if any of them are already in theIMAGE_INSTALL-- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)
Which package backends has been tested with those changes?
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Quoted 24 lines · Otavio Salvador
Quoted 14 lines · Mark Hatle
A few misc fixes relating to toolchain and image generation.
The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.
The next fixes an issue w/ the previous binutils patch and upgrades.
Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.
The final patch fixes an issue w/ the image generatation. If the
IMAGE_INSTALLcontains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.Quoted 8 lines
Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the
IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in theIMAGE_INSTALLline. The system will attempt to determine which to use, by checking if any of them are already in theIMAGE_INSTALL-- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)
Quoted 2 lines
Which package backends has been tested with those changes?
6/6 only affects RPM. So only RPM back end.
The other changes are generic and follow the existing models. However, I only tested the RPM backend.
(Note v2 of 5/6 and 6/6 has been sent. I accidentally sent a version prior to two minor revisions.)
--Mark
Quoted 25 lines · Mark Hatle
Quoted 24 lines · Otavio Salvador
Quoted 22 lines · Mark Hatle
A few misc fixes relating to toolchain and image generation.
The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.
The next fixes an issue w/ the previous binutils patch and upgrades.
Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.
The final patch fixes an issue w/ the image generatation. If the
IMAGE_INSTALLcontains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the
IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in theIMAGE_INSTALLline. The system will attempt to determine which to use, by checking if any of them are already in theIMAGE_INSTALL-- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)
Quoted 7 lines
Quoted 2 lines
Which package backends has been tested with those changes?
6/6 only affects RPM. So only RPM back end.
The other changes are generic and follow the existing models. However, I only tested the RPM backend.
BTW I just verified that the 1-5/6 work properly with an opkg backend as well.
--Mark
Quoted 9 lines
(Note v2 of 5/6 and 6/6 has been sent. I accidentally sent a version prior to two minor revisions.)
--Mark
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
PATCH v2 5/6 packagegroup-self-hosted: Use packagegroup-core-buildessential
In reply to Mark HatleInstead of manually specifying build dependencies, use the packagegroup-core-buildessential. This ensures there is only one place that needs to be modified when toolchain items and dependencies change.
--- meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-)
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb +1 -11
@@ -67,33 +67,23 @@ RRECOMMENDS_packagegroup-self-hosted-host-tools = "\ # eglibc-utils: for rpcgen RDEPENDS_packagegroup-self-hosted-sdk = "\ - autoconf \ - automake \ - binutils \ ccache \ coreutils \ - cpp \ distcc \ eglibc-utils \ eglibc-gconv-ibm850 \ file \ findutils \ - g++ \ - gcc \ intltool \ ldd \ less \ libssp \ libssp-dev \ libssp-staticdev \ - libstdc++ \ - libstdc++-dev \ - libtool \ - make \ mktemp \ + packagegroup-core-buildessential \ perl-module-re \ perl-module-text-wrap \ - pkgconfig \ quilt \ sed \ "
Signature · 1 line
1.8.1.2.545.g2f19ada
PATCH v2 6/6 package_rpm: Allow translation of requirement to package name
In reply to Mark HatleIn the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.
Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
--- meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)
meta/classes/package_rpm.bbclass +75 -4
@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query pkgs_to_install="" + not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then - feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then + new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov" ]; then + # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; then + default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + while [ -n "$1" ]; do + arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2 if [ "$attemptonly" = "Error" ]; then exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then - new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov break fi done
Signature · 1 line
1.8.1.2.545.g2f19ada
Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
Hi Mark,
Quoted 78 lines · On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:
In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
Signed-off-by Mark Hatle <[email protected]>--- meta/classes/package_rpm.bbclass | 79++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)
(quoted diff) +0
-0b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644
meta/classes/package_rpm.bbclass +0
-1@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output${target_rootfs}/install/tmp/fullpkglist.query
meta/classes/package_rpm.bbclass +12
-1pkgs_to_install="" + not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then - feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'${target_rootfs}/install/tmp/fullpkglist.query ; then
meta/classes/package_rpm.bbclass +20
-1+ new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@
Quoted 5 lines
]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
$pkgs_to_install | xargs -n 1 echo | grep -f
${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"
];
Quoted 15 lines
then
meta/classes/package_rpm.bbclass +14
-0+ # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; then
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 10 lines
meta/classes/package_rpm.bbclass +9
-1+ default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$'
${target_rootfs}/install/tmp/provide.query |
Quoted 2 lines
sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + while [
-n "$1"
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 5 lines
]; do
meta/classes/package_rpm.bbclass +4
-0+ arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better then
another...
Quoted 2 lines
meta/classes/package_rpm.bbclass +1
-0+ prov=`grep '^[^@ ]*@'$arch'$'${target_rootfs}/install/tmp/provide.query.list | head -n 1` +
if [ -n
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 8 lines
"$prov" ]; then
meta/classes/package_rpm.bbclass +6
-0+ new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds
($feeds)." >&2
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 11 lines
meta/classes/package_rpm.bbclass +0
-1if [ "$attemptonly" = "Error" ]; then exit 1@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'${target_rootfs}/install/tmp/fullpkglist.query ; then
meta/classes/package_rpm.bbclass +1
-2- new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better then
another...
Quoted 2 lines
meta/classes/package_rpm.bbclass +1
-0+ prov=`grep '^[^@ ]*@'$arch'$'${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if
[ -n
meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289
Quoted 5 lines
"$prov" ]; then
meta/classes/package_rpm.bbclass +1
-0+ new_pkg=$prov break fi done
Sigh... this code is getting ridiculously complicated. Shouldn't Smart be doing more of the heavy lifting here?
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
Quoted 147 lines · Paul Eggleton
Quoted 78 lines · On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:
In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
Signed-off-by Mark Hatle <[email protected]>--- meta/classes/package_rpm.bbclass | 79++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)
(quoted diff) +0
-0b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644
meta/classes/package_rpm.bbclass +0
-1@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output${target_rootfs}/install/tmp/fullpkglist.query
pkgs_to_install=""meta/classes/package_rpm.bbclass +12
-1+ not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then - feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'${target_rootfs}/install/tmp/fullpkglist.query ; then
meta/classes/package_rpm.bbclass +20
-1+ new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@Quoted 5 lines
]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
$pkgs_to_install | xargs -n 1 echo | grep -f
${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"];
Quoted 15 lines
then
meta/classes/package_rpm.bbclass +14
-0+ # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; thenmeta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 10 lines
meta/classes/package_rpm.bbclass +9
-1+ default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$'${target_rootfs}/install/tmp/provide.query |
Quoted 2 lines
sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + while [-n "$1"
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 5 lines
]; do
meta/classes/package_rpm.bbclass +4
-0+ arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better thenanother...
Quoted 2 lines
+ prov=`grep '^[^@ ]*@'$arch'$'
${target_rootfs}/install/tmp/provide.query.list | head -n 1` +if [ -n
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 8 lines
"$prov" ]; then
meta/classes/package_rpm.bbclass +6
-0+ new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds($feeds)." >&2
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 11 lines
if [ "$attemptonly" = "Error" ]; then
exit 1meta/classes/package_rpm.bbclass +0
-1@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'${target_rootfs}/install/tmp/fullpkglist.query ; then
meta/classes/package_rpm.bbclass +1
-2- new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better thenanother...
Quoted 2 lines
+ prov=`grep '^[^@ ]*@'$arch'$'
${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if[ -n
Quoted 8 lines
Quoted 5 lines
"$prov" ]; then
+ new_pkg=$prov
break
fi
doneSigh... this code is getting ridiculously complicated. Shouldn't Smart be doing more of the heavy lifting here?
Agreed; this seems OE is doing too much.
Another think which bothers me is it being RPM specific ... :-(
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
In reply to Paul EggletonQuoted 40 lines · Paul Eggleton
Hi Mark,
Quoted 37 lines · On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:
In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL. So on a failure to translate a package name, we fall back and attempt to resolve based on a package's provide.Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
Signed-off-by Mark Hatle <[email protected]>--- meta/classes/package_rpm.bbclass | 79++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)
(quoted diff) +0
-0b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644
meta/classes/package_rpm.bbclass +0
-1@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output${target_rootfs}/install/tmp/fullpkglist.query
pkgs_to_install=""
meta/classes/package_rpm.bbclass +1
-0+ not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then
Quoted 8 lines
meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 199
Quoted 8 lines
meta/classes/package_rpm.bbclass +0
-1- feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found!
Just to be clear the following diff is rather confusing.. there are two one line changes (the error to the no_found line below) in the existing code, and then the new chunk of code after both to process the not_found entries, which are presumed to be dependencies.)
If you apply this, or look at it in context it's easier to see..
Quoted 101 lines
Quoted 33 lines
meta/classes/package_rpm.bbclass +11
-0+ not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'${target_rootfs}/install/tmp/fullpkglist.query ; then
meta/classes/package_rpm.bbclass +20
-1+ new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@Quoted 5 lines
]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
$pkgs_to_install | xargs -n 1 echo | grep -f
${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"];
Quoted 15 lines
then
meta/classes/package_rpm.bbclass +14
-0+ # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; thenmeta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 10 lines
meta/classes/package_rpm.bbclass +9
-1+ default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$'${target_rootfs}/install/tmp/provide.query |
Quoted 2 lines
sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
${target_rootfs}/install/tmp/provide.query.list || : + while [-n "$1"
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 5 lines
]; do
meta/classes/package_rpm.bbclass +4
-0+ arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better thenanother...
Quoted 2 lines
meta/classes/package_rpm.bbclass +1
-0+ prov=`grep '^[^@ ]*@'$arch'$'${target_rootfs}/install/tmp/provide.query.list | head -n 1` +
if [ -n
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 8 lines
"$prov" ]; then
meta/classes/package_rpm.bbclass +6
-0+ new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds($feeds)." >&2
meta/classes/package_rpm.bbclass translate_oe_to_smart() {
Quoted 11 lines
if [ "$attemptonly" = "Error" ]; then
exit 1meta/classes/package_rpm.bbclass +0
-1@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'${target_rootfs}/install/tmp/fullpkglist.query ; then
meta/classes/package_rpm.bbclass +1
-2- new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better thenanother...
Quoted 2 lines
meta/classes/package_rpm.bbclass +1
-0+ prov=`grep '^[^@ ]*@'$arch'$'${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if
[ -n
Quoted 8 lines
meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289
Quoted 5 lines
"$prov" ]; then
meta/classes/package_rpm.bbclass +1
-0+ new_pkg=$prov break fi doneSigh... this code is getting ridiculously complicated. Shouldn't Smart be doing more of the heavy lifting here?
This function translate_oe_to_smart (and the corresponding translate_smart_to_oe) are used to switch from one name space to another. Remember in the RPM world, we don't use prefixes like "lib32-" on our package names, instead we use specific arch's that refer to specific ABI and processor tunings.
This is what enables the seamless multilib install on RPM.
What this code does is allows for dependencies to be added to the mix. If we didn't have multilibs then none of this code would be needed, but since we do -- the name space of both the package and dependencies need to be translate, not just packages as we'd done before.
Generally what this function does:
for each multilib prefx:
if PN starts w/ a known multilib prefix:
we translate it to: BPN@arch
check if the package exists
if it does add to pkg_list
else add to "not_found"
anything remaining is translated to the default arch
check if the package exists
if it does add to pkg_list
else add to "not_found"
if there are any "not_found", we assume they must be provides:
for each multilib prefix:
if the DEPENDENCY starts w/ a known multilib prefix:
we translate it without the prefix
ask smart what provides that dependency
check if something already in the pkg_list is there, if so skip it else use priority ordering to select the best provider -- add to pkg_list
for anything w/o a multilib prefix, follow the same steps
The reverse smart_to_oe function doesn't ever need to pay attention to dependencies so that is not an issue, and the translation is significantly more straight forward.
If we had unique namespaces in oe, and didn't use the lib* markers on multilibs.. then none of this would be necessary.. (but then of course dpkg and opkg wouldn't work properly.) So the translation is a necessary evil.
Could it be done differently, most likely. Perhaps identifying a way to do it in python would be quicker.. but it's actually pretty fast sine the majority of translations happen via a static list and a single grep. It's only the (hopefully) small number of dependencies that happen via individual calls to smart which impose a time penalty.
--Mark
Quoted 3 lines
Cheers,
Paul
Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
In reply to Mark HatleQuoted 3 lines · Mark Hatle
In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL.
For what its worth, I find this description hard to understand, I think as you leave the keyword "provide" right until the end. A mention of RPROVIDES earlier here would help a lot.
Quoted 117 lines
So on a failure to translate a package name, we fall back
and attempt to resolve based on a package's provide.Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
Signed-off-by Mark Hatle <[email protected]>--- meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)meta/classes/package_rpm.bbclass +71
-1@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query pkgs_to_install="" + not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then - feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then + new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov" ]; then + # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; then + default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + while [ -n "$1" ]; do + arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2 if [ "$attemptonly" = "Error" ]; then exit 1@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do
meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289
Quoted 10 lines
meta/classes/package_rpm.bbclass +4
-3- if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then - new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov break fi done
This 'shell calling rpm commands' code with all the mangling is near impossible to read/understand. This patch just makes the mess more complicated and harder to follow.
What is the plan for turning this into something better?
My instinct says more functions and a language better at data manipulation (python instead of shell) might start to resolve this.
One of the reasons for using smart was that we could probably enhance it to better suit our needs. Can we consider that too?
I really don't want to see more patches like this one piling on top of already crumbling foundations.
Cheers,
Richard
Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
Quoted 8 lines · Richard Purdie
Quoted 3 lines · Mark Hatle
In the translate oe to smart function, we only translated package names. However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL.For what its worth, I find this description hard to understand, I think as you leave the keyword "provide" right until the end. A mention of RPROVIDES earlier here would help a lot.
Ok.
Quoted 123 lines
Quoted 123 lines
So on a failure to translate a package name, we fall back
and attempt to resolve based on a package's provide.Note: it may be possible to generate an unsolvable install solution. If the dependency is provided by one or more things that conflict with something else set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with actual package names.
Signed-off-by Mark Hatle <[email protected]>--- meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-)meta/classes/package_rpm.bbclass +74
-4@@ -166,6 +166,7 @@ translate_oe_to_smart() { [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query pkgs_to_install="" + not_found="" for pkg in "$@" ; do new_pkg="$pkg" if [ -z "$sdk_mode" ]; then@@ -184,7 +185,6 @@ translate_oe_to_smart() { fi subst=${pkg#${mlib}-} if [ "$subst" != "$pkg" ]; then - feeds=$@ while [ -n "$1" ]; do arch="$1" arch=`echo "$arch" | tr - _`@@ -197,6 +197,76 @@ translate_oe_to_smart() { done if [ "$pkg" = "$new_pkg" ]; then # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + done + fi + # Apparently not a multilib package... + if [ "$pkg" = "$new_pkg" ]; then + default_archs_fixed=`echo "$default_archs" | tr - _` + for arch in $default_archs_fixed ; do + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then + new_pkg="$pkg@$arch" + # First found is best match + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! + not_found="$not_found $pkg" + continue + fi + fi + #echo "$pkg -> $new_pkg" >&2 + pkgs_to_install="${pkgs_to_install} ${new_pkg}" + done + + # Parse the not_found items and see if they were dependencies (RPROVIDES) + # Follow the parsing example above... + for pkg in $not_found ; do + new_pkg="$pkg" + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov" ]; then + # Nothing to do, already in the list + #echo "Skipping $pkg -> $prov, already in install set" >&2 + continue + fi + if [ -z "$sdk_mode" ]; then + for i in ${MULTILIB_PREFIX_LIST} ; do + old_IFS="$IFS" + IFS=":" + set $i + IFS="$old_IFS" + mlib="$1" + shift + if [ "$mlib" = "default" ]; then + if [ -z "$default_archs" ]; then + default_archs=$@ + fi + continue + fi + subst=${pkg#${mlib}-} + if [ "$subst" != "$pkg" ]; then + feeds=$@ + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || : + while [ -n "$1" ]; do + arch="$1" + arch=`echo "$arch" | tr - _` + shift + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then + new_pkg=$prov + break + fi + done + if [ "$pkg" = "$new_pkg" ]; then + # Failed to translate, package not found! echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2 if [ "$attemptonly" = "Error" ]; then exit 1@@ -210,9 +280,10 @@ translate_oe_to_smart() { if [ "$pkg" = "$new_pkg" ]; then default_archs_fixed=`echo "$default_archs" | tr - _` for arch in $default_archs_fixed ; do - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then - new_pkg="$pkg@$arch" - # First found is best match + # Select first found, we don't know if one is better then another... + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if [ -n "$prov" ]; then
Quoted 8 lines
meta/classes/package_rpm.bbclass translate_oe_to_smart() { · line 289
Quoted 4 lines
meta/classes/package_rpm.bbclass +1
-0+ new_pkg=$prov break fi doneThis 'shell calling rpm commands' code with all the mangling is near impossible to read/understand. This patch just makes the mess more complicated and harder to follow.
The root of this is the namespace issue. In RPM, the architecture (and arch compatibility), as well as file contents all provide the namespace and conflict resolution. While in opkg/deb the name of the package itself is where the namespace is defined. (i.e. lib32-...)
The translation is necessary due to this multilib support, otherwise we wouldn't need to do any of it. Somehow we have to translate one namespace to another -- and since the translation can't occur until the packages are built, it'd be difficult to do it 'earlier'.
Quoted 7 lines
What is the plan for turning this into something better?
My instinct says more functions and a language better at data manipulation (python instead of shell) might start to resolve this.
One of the reasons for using smart was that we could probably enhance it to better suit our needs. Can we consider that too?
The only thing we can do is build a python (or similar) app that takes the inputs of the translation 'namespace' IDs and the corresponding architectures, and RPM feed. Then translate from one to the other. My biggest concern with this translation is to cover all of the corner cases we could start getting more and more complex.
Quoted 2 lines
I really don't want to see more patches like this one piling on top of already crumbling foundations.
Easiest way to avoid this is to make dependencies in the IMAGE/PACKAGE_INSTALL forbidden and do the translation at a higher level generically. But frankly I don't see that happening -- so the solution becomes RPM specific and we end up with code like this. I'm not sure if we can move this code into a python function -- or if we need to simply create an external translation program...
As far as plans, frankly I don't have any plans to focus on this work for the time being. There are too many other things I need to get working properly first. If someone else wants to step in and work on a solution, I'm more then happy to help them out.
--Mark
Quoted 4 lines
Cheers,
Richard
PATCH v3 2/6 gcc: Drop *-symlinks
In reply to Mark HatleThe -symlinks packages don't really add any value if we're using update-alternatives. Drop it, leaving an RPROVIDES in case anyone thought they needed it still.
--- meta/recipes-devtools/gcc/gcc-target.inc | 54 ++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 24 deletions(-)
meta/recipes-devtools/gcc/gcc-target.inc +30 -24
@@ -10,12 +10,12 @@ EXTRA_OECONF_PATHS = " \ ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR} -I${B}/gcc/include/ " PACKAGES = "\ - ${PN} ${PN}-plugins ${PN}-symlinks \ - g++ g++-symlinks \ - cpp cpp-symlinks \ - g77 g77-symlinks \ - gfortran gfortran-symlinks \ - gcov gcov-symlinks \ + ${PN} ${PN}-plugins \ + g++ \ + cpp \ + g77 \ + gfortran \ + gcov \ ${PN}-plugin-dev \ ${PN}-doc \ ${PN}-dev \
@@ -36,6 +36,9 @@ FILES_${PN} = "\ ${gcclibdir}/${TARGET_SYS}/${BINV}/include-fixed \ " INSANE_SKIP_${PN} += "dev-so" +RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks" +RCONFLICTS_${PN} += "${PN}-symlinks" FILES_${PN}-dbg += "\ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/.debug/ \
@@ -50,8 +53,6 @@ FILES_${PN}-plugin-dev = "\ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \ ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \ " -FILES_${PN}-symlinks = "" -ALLOW_EMPTY_${PN}-symlinks = "1" FILES_${PN}-plugins = "\ ${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -62,33 +63,38 @@ FILES_g77 = "\ ${bindir}/${TARGET_PREFIX}g77 \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \ " -FILES_g77-symlinks = "" -ALLOW_EMPTY_g77-symlinks = "1" +RPROVIDES_g77 += "g77-symlinks" +RREPLACES_g77 += "g77-symlinks" +RCONFLICTS_g77 += "${PN}-symlinks" FILES_gfortran = "\ ${bindir}/${TARGET_PREFIX}gfortran \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \ " -FILES_gfortran-symlinks = "" -ALLOW_EMPTY_gfortran-symlinks = "1" +RPROVIDES_gfortran += "gfortran-symlinks" +RREPLACES_gfortran += "gfortran-symlinks" +RCONFLICTS_gfortran += "${PN}-symlinks" FILES_cpp = "\ ${bindir}/${TARGET_PREFIX}cpp \ ${base_libdir}/cpp \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1" -FILES_cpp-symlinks = "" -ALLOW_EMPTY_cpp-symlinks = "1" +RPROVIDES_cpp += "cpp-symlinks" +RREPLACES_cpp += "cpp-symlinks" +RCONFLICTS_cpp += "${PN}-symlinks" FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov" -FILES_gcov-symlinks = "" -ALLOW_EMPTY_gconv-symlinks = "1" +RPROVIDES_gcov += "gcov-symlinks" +RREPLACES_gcov += "gcov-symlinks" +RCONFLICTS_gcov += "${PN}-symlinks" FILES_g++ = "\ ${bindir}/${TARGET_PREFIX}g++ \ ${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \ " -FILES_g++-symlinks = "" -ALLOW_EMPTY_g++-symlinks = "1" +RPROVIDES_g++ += "g++-symlinks" +RREPLACES_g++ += "g++-symlinks" +RCONFLICTS_g++ += "${PN}-symlinks" FILES_${PN}-doc = "\ ${infodir} \
@@ -137,12 +143,12 @@ inherit update-alternatives ALTERNATIVE_PRIORITY = "100" -ALTERNATIVE_${PN}-symlinks = "cc gcc" -ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}" -ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}" -ALTERNATIVE_cpp-symlinks = "cpp" -ALTERNATIVE_gcov-symlinks = "gcov" -ALTERNATIVE_g++-symlinks = "c++ g++" +ALTERNATIVE_${PN} = "cc gcc" +ALTERNATIVE_g77 = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}" +ALTERNATIVE_gfortran = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}" +ALTERNATIVE_cpp = "cpp" +ALTERNATIVE_gcov = "gcov" +ALTERNATIVE_g++ = "c++ g++" ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc" ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
Signature · 1 line
1.8.3.4
PATCH v3 3/6 binutils: Ensure old -symlinks packages get removed
In reply to Mark HatleWhen the -symlinks package was removed and an RPROVIDES was added, nothing was put in to cause the removal of the older package in an upgrade.
--- meta/recipes-devtools/binutils/binutils.inc | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-devtools/binutils/binutils.inc +2 -0
@@ -21,6 +21,8 @@ FILES_${PN} = " \ ${prefix}/${TARGET_SYS}/bin/*" RPROVIDES_${PN} += "${PN}-symlinks" +RREPLACES_${PN} += "${PN}-symlinks" +RCONFLICTS_${PN} += "${PN}-symlinks" FILES_${PN}-dev = " \ ${includedir} \
Signature · 1 line
1.8.3.4
Quoted 27 lines · Mark Hatle
A few misc fixes relating to toolchain and image generation.
The first two in the set simply updated gcc to remove the -symlinks package in favor of update-alternatives.
The next fixes an issue w/ the previous binutils patch and upgrades.
Next 2 update packagegroups to make revise and use the packagegroup-core-buildessentials.
The final patch fixes an issue w/ the image generatation. If the
IMAGE_INSTALLcontains a dependency instead of a package name, the system was not able to translate it into a package. This now works properly.Note, there is a case where an unresolvable set could be determined (rare, but it is possible), in this case the user would need to add whatever is required to fix it to the
IMAGE_INSTALL. (This could happen when 2 or more conflicting packages all have the same provide, but none of them are in theIMAGE_INSTALLline. The system will attempt to determine which to use, by checking if any of them are already in theIMAGE_INSTALL-- otherwise it chooses the first entry. If something else implied by a packagegroup chooses a different version, a conflict would be generated.)The following changes since commit
616354f13732d13c17434d5b60b166f691c25761:binutils: Add gnu-config-native to DEPENDS (2013-11-12 16:00:20 +0000)
Quoted 8 lines
are available in the git repository at:
git://git.yoctoproject.org/poky-contribmhatle/misc1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/misc1Mark Hatle (6):
gcc: Use alternatives for the *-symlinks packages.
gcc: Drop *-symlinks
Mark
This change set seems to be giving the Autobuilder some issues, please take a look at the AB (for example: http://autobuilder.yoctoproject.org:8011/builders/nightly-mips/builds/1/steps/Building%20Images/logs/stdio
Thanks
Sau!
Quoted 12 lines
binutils: Ensure old -symlinks packages get removed
packagegroups: Remove toolchain *-symlinks packages
packagegroup-self-hosted: Use packagegroup-core-buildessentials
package_rpm: Allow translation of requirement to package namemeta/classes/package_rpm.bbclass | 79 +++++++++++++++- .../packagegroup-core-buildessential.bb | 4 - .../packagegroups/packagegroup-self-hosted.bb | 16 +--- meta/recipes-devtools/binutils/binutils.inc | 1 + meta/recipes-devtools/gcc/gcc-target.inc | 103 +++++++++++++-------- 5 files changed, 141 insertions(+), 62 deletions(-)