python-smartpm-native: prefer same arch when install
cover python-smartpm-native: prefer same arch when install
The following changes since commit f787b688f2884ce3fa888b4041030538c7d2bf55:
oeqa/utils/decorators: fix missing keyword arguments on decorators (2015-10-27 07:22:22 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/smart
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/smart
Robert Yang (1):
python-smartpm-native: prefer same arch when install
.../python-smartpm/smart-prefer-same-arch.patch | 119 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 2 + 2 files changed, 121 insertions(+) create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch
Signature · 1 line
1.7.9.5
PATCH 1/1 python-smartpm-native: prefer same arch when install
We had made smart install multilib RDEPENDS correctly from package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix the issue from python-smartpm-native, and make it work well.
The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use pkg_A's arch when possible.
This patch fixed:
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"RPM_PREFER_ELF_ARCH = "1"IMAGE_INSTALL_append = " lib32-connman-gnome"
MACHINE = "qemux86-64"
$ bitbake core-image-sato
Only 64bit loaders like libgdk-pixbuf-2.0-loader-jpeg and libgdk-pixbuf-2.0-loader-png were installed before this patch since they are in RRECOMMANDS, now both 32bit and 64bit are installed.
This patch is for native only to minimize the impact for this release, the main problem is the "if attempt:", otherwise, more packages like lib32-libc6-dev will be installed when populate_sdk, but only libc6-dev were installed before. We need reconstruct the way that we install the complentary packages in future release, the "if attempt:" make they work together atm.
[YOCTO #8570]
--- .../python-smartpm/smart-prefer-same-arch.patch | 119 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 2 + 2 files changed, 121 insertions(+) create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch
meta/recipes-devtools/python/python-smartpm/smart-prefer-same-arch.patch new file +119 -0
@@ -0,0 +1,119 @@ +From 1749592a76425449c1f3119fb14db3f5e14226ed Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Tue, 27 Oct 2015 06:20:45 -0700 +Subject: [PATCH] smart/transaction.py: prefer same arch when install + +We had made smart install multilib RDEPENDS correctly from +package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix +the issue from python-smartpm-native, and make it work well. + +The logic is: when pkg_A rdepends/rrecommands pkg_B, then let pkg_B use +pkg_A's arch when possible. + +This patch fixed: +require conf/multilib.conf +MULTILIBS = "multilib:lib32" +DEFAULTTUNE_virtclass-multilib-lib32 = "x86" +RPM_PREFER_ELF_ARCH = "1" +IMAGE_INSTALL_append = " lib32-connman-gnome" +MACHINE = "qemux86-64" + +$ bitbake core-image-sato + +Only 64bit loaders like libgdk-pixbuf-2.0-loader-jpeg and +libgdk-pixbuf-2.0-loader-png were installed before this patch since they +are in RRECOMMANDS, now both 32bit and 64bit are installed. + +This patch is for native only to minimize the impact for this release, +the main problem is the "if attempt:", otherwise, more packages like +lib32-libc6-dev will be installed when populate_sdk, but only libc6-dev +were installed before. We need reconstruct the way that we install the +complentary packages in future release, the "if attempt:" make they work +together atm. + +Upstream-Status: Inappropriate [oe specific] + +Signed-off-by: Robert Yang <[email protected]> +--- + smart/transaction.py | 28 +++++++++++++++++++++++++--- + 1 file changed, 25 insertions(+), 3 deletions(-) + +diff --git a/smart/transaction.py b/smart/transaction.py +index f022aae..63a837c 100644 +--- a/smart/transaction.py ++++ b/smart/transaction.py +@@ -22,6 +22,7 @@ + from smart.const import INSTALL, REMOVE, UPGRADE, FIX, REINSTALL, KEEP, LOCKED_EXCLUDE, LOCKED_INSTALL, LOCKED_CONFLICT, LOCKED_CONFLICT_BY, LOCKED_NO_COEXIST, LOCKED_SYSCONF, LOCKED_REMOVE + from smart.cache import PreRequires, Package + from smart import * ++from smart.backends.rpm.rpmver import splitarch + + def lock_reason(pkg, lockvalue): + try: +@@ -618,6 +619,7 @@ class Transaction(object): + self._remove(namepkg, changeset, locked, pending, depth) + + # Install packages required by this one. ++ pkgarch = splitarch(pkg.version)[1] + for req in pkg.requires + pkg.recommends: + + reqrequired = req in pkg.requires +@@ -626,6 +628,8 @@ class Transaction(object): + prvpkgs = {} + lockedpkgs = {} + found = False ++ found_same_arch = False ++ same_arch_in_provides = False + for prv in req.providedby: + for prvpkg in prv.packages: + if not reqrequired: +@@ -633,9 +637,19 @@ class Transaction(object): + continue + elif pkgconf.testFlag("ignore-recommends", prvpkg): + continue ++ prvpkgarch = splitarch(prvpkg.version)[1] ++ if pkgarch == prvpkgarch: ++ # Found a same arch in provides ++ same_arch_in_provides = True + if isinst(prvpkg): + found = True +- break ++ if attempt: ++ break ++ if pkgarch == prvpkgarch: ++ # Found the best match ++ found_same_arch = True ++ break ++ continue + if prvpkg not in locked: + prvpkgs[prvpkg] = True + else: +@@ -643,7 +657,7 @@ class Transaction(object): + else: + continue + break +- if found: ++ if found_same_arch or (found and attempt) or (found and not same_arch_in_provides): + # Someone is already providing it. Good. + continue + +@@ -672,7 +686,15 @@ class Transaction(object): + + # It's only a recommend, skip + continue +- ++ if len(prvpkgs) > 1 and not attempt: ++ prvarchs = set() ++ for prv in prvpkgs: ++ prvarchs.add(splitarch(prv.version)[1]) ++ if pkgarch in prvarchs: ++ # Prefer the one which has the same arch ++ for prv in prvpkgs.copy(): ++ if splitarch(prv.version)[1] != pkgarch: ++ del prvpkgs[prv] + if len(prvpkgs) == 1: + # Don't check locked here. prvpkgs was + # already filtered above. +-- +1.7.9.5 +
meta/recipes-devtools/python/python-smartpm_git.bb +2 -0
@@ -26,6 +26,8 @@ SRC_URI = "\ file://smart-cache.py-getPackages-matches-name-version.patch \ " +SRC_URI_append_class-native = "file://smart-prefer-same-arch.patch" + SRCREV = "407a7eca766431257dcd1da15175cc36a1bb22d0" PV = "1.5+git${SRCPV}"
Signature · 1 line
1.7.9.5
Quoted 7 lines · Robert Yang
We had made smart install multilib RDEPENDS correctly from
package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix the issue from python-smartpm-native, and make it work well.The logic is: when
pkg_Ardepends/rrecommandspkg_B, then letpkg_Buse pkg_A's arch when possible.
I added this patch and rebuilt core-image-sato, then ran buildhistory-diff:
Changes to images/intel_corei7_64/glibc/core-image-sato (installed-package-names.txt):
lib32-ncurses-terminfo-base was added lib32-libjpeg9 was added lib32-glibc-gconv-iso8859-1 was added lib32-shared-mime-info was added lib32-libxml2 was added lib32-wpa-supplicant-cli was added lib32-libgdk-pixbuf-2.0-loader-png was added lib32-wpa-supplicant-passphrase was added lib32-pango-module-basic-fc was added lib32-bash was added lib32-libgdk-pixbuf-2.0-loader-jpeg was added lib32-libgdk-pixbuf-2.0-loader-gif was added lib32-libgdk-pixbuf-2.0-loader-xpm was added lib32-glibc-gconv was added
Some of those are good (gdk-pixbuf and pango), some bad (bash, wpa-supplicant-cli), some probably bugs in packages that should be allarch (termino, shared-mime-info).
I'm undecided what we do about this. Fixing the dependency chains is good, but we're definitely pulling in more than expected.
Ross
Quoted 31 lines · Ross
On 27 October 2015 at 14:05, Robert Yang <[email protected] <mailto:[email protected]>> wrote:
We had made smart install multilib RDEPENDS correctly from
package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix the issue from python-smartpm-native, and make it work well.The logic is: when
pkg_Ardepends/rrecommandspkg_B, then letpkg_Buse pkg_A's arch when possible.I added this patch and rebuilt core-image-sato, then ran buildhistory-diff:
Changes to images/intel_corei7_64/glibc/core-image-sato (installed-package-names.txt):
lib32-ncurses-terminfo-base was added lib32-libjpeg9 was added lib32-glibc-gconv-iso8859-1 was added lib32-shared-mime-info was added lib32-libxml2 was added lib32-wpa-supplicant-cli was added lib32-libgdk-pixbuf-2.0-loader-png was added lib32-wpa-supplicant-passphrase was added lib32-pango-module-basic-fc was added lib32-bash was added lib32-libgdk-pixbuf-2.0-loader-jpeg was added lib32-libgdk-pixbuf-2.0-loader-gif was added lib32-libgdk-pixbuf-2.0-loader-xpm was added lib32-glibc-gconv was added
Quoted 3 lines
Some of those are good (gdk-pixbuf and pango), some bad (bash, wpa-supplicant-cli), some probably bugs in packages that should be allarch (termino, shared-mime-info).
bash, wpa-supplicant-cli is a good example of why this is an issue. RPM luckily says, well we've got two versions of bash, lets resolve this by following the ELF resolution rule. So while two packages may exist in the manifest, you won't get any conflicts and space is likely not duplicated.
But you can't automatically say "give me any bash, but give me arch specific versions of everything else". The default in RPM is if I ask for 'bash', give me any version of bash.. if I ask for 'bash.i586' I was -the- i586 version.
This is why the fix for this is either adjust the behavior (as Robert has done) to suck in more dependencies and consider the ELF resolver to be adequate -- or to add some kind of a hint to the recipe mechanisms that select dependencies must be of the same arch type. One we handle in the resolver, and one we could handle when we generate the packages themselves.
I'm not sure there is an obvious solution, I prefer the later with a variant of the former myself. (Variant being if all things are equal we weight dependencies to match the arch.. but if something is already in with that provide we use it..) Then the hint would cause the produced packages to be able to say they need explicit versions of the plugins.
Quoted 2 lines
I'm undecided what we do about this. Fixing the dependency chains is good, but we're definitely pulling in more than expected.
I agree, I have a feeling what I wrote about is the right answer... but I don't have any code to prove it.. and I'm not sure that we should be making that big of a change right now.
Robert's patch SHOULD resolve the problem at the expense of a slightly larger resolution manifest, which may be acceptable.
--Mark
Quoted 3 lines
Ross
Quoted 31 lines · Ross
On 27 October 2015 at 14:05, Robert Yang <[email protected] <mailto:[email protected]>> wrote:
We had made smart install multilib RDEPENDS correctly from
package_manager.py, but it couldn't handle RRECOMMANDS, this patch fix the issue from python-smartpm-native, and make it work well.The logic is: when
pkg_Ardepends/rrecommandspkg_B, then letpkg_Buse pkg_A's arch when possible.I added this patch and rebuilt core-image-sato, then ran buildhistory-diff:
Changes to images/intel_corei7_64/glibc/core-image-sato (installed-package-names.txt):
lib32-ncurses-terminfo-base was added lib32-libjpeg9 was added lib32-glibc-gconv-iso8859-1 was added lib32-shared-mime-info was added lib32-libxml2 was added lib32-wpa-supplicant-cli was added lib32-libgdk-pixbuf-2.0-loader-png was added lib32-wpa-supplicant-passphrase was added lib32-pango-module-basic-fc was added lib32-bash was added lib32-libgdk-pixbuf-2.0-loader-jpeg was added lib32-libgdk-pixbuf-2.0-loader-gif was added lib32-libgdk-pixbuf-2.0-loader-xpm was added lib32-glibc-gconv was added
Quoted 3 lines
Some of those are good (gdk-pixbuf and pango), some bad (bash, wpa-supplicant-cli), some probably bugs in packages that should be allarch (termino, shared-mime-info).
I think that lacking something so that it doesn't work is worse than have more things but works ?
// Robert
Quoted 5 lines
I'm undecided what we do about this. Fixing the dependency chains is good, but we're definitely pulling in more than expected.
Ross