smart: speed up installation (reduce 28% ~ 57%)
cover smart: speed up installation (reduce 28% ~ 57%)
Here is the test result:
MACHINE = "qemux86-64"
- When multilib enabled:
$ bitbake core-image-sato -cpopulate_sdk
time: 6m5s -> 2m34s (Reduce 57% ) $ bitbake core-image-minimal -cpopulate_sdk
time: 2m1s -> 1m26s (Reduce 28% ) $ bitbake core-image-sato-sdk
time: 10m15s -> 7m12s (Reduce 29% )- When multilib NOT enabled:
$ bitbake core-image-sato-sdk -cpopulate_sdk
time: 4m25s -> 2m28s (Reduce 44% )// Robert
The following changes since commit 16957f5f1de0f1fb4052d5aef93ee10c893f4a38:
oeqa/selftest/wic: Use SetupLocal instead of Setup (2015-10-01 07:40:37 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/smart
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/smart
Robert Yang (1):
smart:cache.py: getPackages() matches name + arch
...cache.py-getPackages-matches-name-version.patch | 43 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 1 + 2 files changed, 44 insertions(+) create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch
Signature · 1 line
1.7.9.5
PATCH 1/1 smart:cache.py: getPackages() matches name + arch
It only matched name ony in the past, for example:
smart install busybox (matched)
but:
smart install busybox@core2_64 (didn't match)
The installation is very slow when no match since it would seach all the packages in the repo, and what we use mostly in oe-core is the second case, so the installation is very slow when install COMPLEMENTARY packages such as the task do_populate_sdk.
This patch makes it match both.
* Speed up
MACHINE = "qemux86-64"
- When multilib enabled:
$ bitbake core-image-sato -cpopulate_sdk
time: 6m5s -> 2m34s (Reduce 57% ) $ bitbake core-image-minimal -cpopulate_sdk
time: 2m1s -> 1m26s (Reduce 28% ) $ bitbake core-image-sato-sdk
time: 10m15s -> 7m12s (Reduce 29% )- When multilib NOT enabled:
$ bitbake core-image-sato-sdk -cpopulate_sdk
time: 4m25s -> 2m28s (Reduce 44% )[YOCTO #8389]
--- ...cache.py-getPackages-matches-name-version.patch | 43 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 1 + 2 files changed, 44 insertions(+) create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch
meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch new file +43 -0
@@ -0,0 +1,43 @@ +From ee05e55e84b53f4bb0d0baba13ca47a8f84b7cb4 Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 30 Sep 2015 01:12:52 -0700 +Subject: [PATCH] smart:cache.py: getPackages() matches name + arch + +It only matched name ony in the past, for example: +smart install busybox (matched) +but: +smart install busybox@core2_64 (didn't match) + +The installation is very slow when no match since it would seach all the +packages in the repo +This patch makes it match both. + +Upstream-Status: Pending + +Signed-off-by: Robert Yang <[email protected]> +--- + smart/cache.py | 3 ++- + smart/ccache.c | 9 ++++++++- + 2 files changed, 10 insertions(+), 2 deletions(-) + +diff --git a/smart/control.py b/smart/control.py +index d44abe7..f23a604 100644 +--- a/smart/control.py ++++ b/smart/control.py +@@ -876,9 +876,13 @@ class Control(object): + objects = [] + + # If we find packages with exactly the given +- # name or name-version, use them. +- for pkg in self._cache.getPackages(s): +- if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s: ++ # name, name-version, or name@arch, use them. ++ s_name = s ++ if "@" in s: ++ s_name = s.split("@")[0] ++ for pkg in self._cache.getPackages(s_name): ++ if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s \ ++ or "%s@%s" % (pkg.name, pkg.version.split('@')[1]) == s: + objects.append((1.0, pkg)) + + if not objects:
meta/recipes-devtools/python/python-smartpm_git.bb +1 -0
@@ -23,6 +23,7 @@ SRC_URI = "\ file://smart-add-for-rpm-ignoresize-check.patch \ file://smart-already-installed-message.patch \ file://smart-set-noprogress-for-pycurl.patch \ + file://smart-cache.py-getPackages-matches-name-version.patch \ " SRCREV = "407a7eca766431257dcd1da15175cc36a1bb22d0"
Signature · 1 line
1.7.9.5
Re: [PATCH 1/1] smart:cache.py: getPackages() matches name + arch
Quoted 18 lines · Robert Yang
It only matched name ony in the past, for example:
smart install busybox (matched)
but:
smart install busybox@core2_64 (didn't match)The installation is very slow when no match since it would seach all the packages in the repo, and what we use mostly in oe-core is the second case, so the installation is very slow when install COMPLEMENTARY packages such as the task
do_populate_sdk.This patch makes it match both.
* Speed up
MACHINE = "qemux86-64"
- When multilib enabled:$ bitbake core-image-sato -cpopulate_sdk time: 6m5s -> 2m34s (Reduce 57% )
Quoted 8 lines
$ bitbake core-image-minimal -cpopulate_sdk time: 2m1s -> 1m26s (Reduce 28% )$ bitbake core-image-sato-sdk time: 10m15s -> 7m12s (Reduce 29% )- When multilib NOT enabled:
$ bitbake core-image-sato-sdk -cpopulate_sdk
Sorry , a typo, this should be :
$ bitbake core-image-sato -cpopulate_sdk.
Updated in in the repo.
// Robert
Quoted 73 lines
time: 4m25s -> 2m28s (Reduce 44% )[YOCTO #8389]
Signed-off-by Robert Yang <[email protected]>--- ...cache.py-getPackages-matches-name-version.patch | 43 ++++++++++++++++++++ meta/recipes-devtools/python/python-smartpm_git.bb | 1 + 2 files changed, 44 insertions(+)create mode 100644
meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patchmeta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch new file +43
-0@@ -0,0 +1,43 @@ +From ee05e55e84b53f4bb0d0baba13ca47a8f84b7cb4 Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 30 Sep 2015 01:12:52 -0700 +Subject: [PATCH] smart:cache.py: getPackages() matches name + arch + +It only matched name ony in the past, for example: +smart install busybox (matched) +but: +smart install busybox@core2_64 (didn't match) + +The installation is very slow when no match since it would seach all the +packages in the repo +This patch makes it match both. + +Upstream-Status: Pending + +Signed-off-by: Robert Yang <[email protected]> +--- + smart/cache.py | 3 ++- + smart/ccache.c | 9 ++++++++- + 2 files changed, 10 insertions(+), 2 deletions(-) + +diff --git a/smart/control.py b/smart/control.py +index d44abe7..f23a604 100644 +--- a/smart/control.py ++++ b/smart/control.py +@@ -876,9 +876,13 @@ class Control(object): + objects = [] + + # If we find packages with exactly the given +- # name or name-version, use them. +- for pkg in self._cache.getPackages(s): +- if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s: ++ # name, name-version, or name@arch, use them. ++ s_name = s ++ if "@" in s: ++ s_name = s.split("@")[0] ++ for pkg in self._cache.getPackages(s_name): ++ if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s \ ++ or "%s@%s" % (pkg.name, pkg.version.split('@')[1]) == s: + objects.append((1.0, pkg)) + + if not objects:meta/recipes-devtools/python/python-smartpm_git.bb +1
-0@@ -23,6 +23,7 @@ SRC_URI = "\ file://smart-add-for-rpm-ignoresize-check.patch \ file://smart-already-installed-message.patch \ file://smart-set-noprogress-for-pycurl.patch \ + file://smart-cache.py-getPackages-matches-name-version.patch \ " SRCREV = "407a7eca766431257dcd1da15175cc36a1bb22d0"