Skip to contentSkip to conversation

smart: speed up installation (reduce 28% ~ 57%)

PATCH In review Robert Yang 1 participant 3 messages Oct 1, 2015

lore
Share & tools
lore
#0 lore raw

cover smart: speed up installation (reduce 28% ~ 57%)

Here is the test result:
MACHINE = "qemux86-64"
- When multilib enabled:

    $ bitbake core-image-sato -cpopulate_sdk
    time: 6m5s -> 2m34s (Reduce 57% )
    $ bitbake core-image-minimal -cpopulate_sdk
    time: 2m1s -> 1m26s (Reduce 28% )
    $ bitbake core-image-sato-sdk
    time: 10m15s -> 7m12s (Reduce 29% )

- When multilib NOT enabled:

    $ bitbake core-image-sato-sdk -cpopulate_sdk
    time: 4m25s -> 2m28s (Reduce 44% )

// Robert

The following changes since commit 16957f5f1de0f1fb4052d5aef93ee10c893f4a38:

oeqa/selftest/wic: Use SetupLocal instead of Setup (2015-10-01 07:40:37 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/smart
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/smart

Robert Yang (1):
smart:cache.py: getPackages() matches name + arch

 ...cache.py-getPackages-matches-name-version.patch |   43 ++++++++++++++++++++
 meta/recipes-devtools/python/python-smartpm_git.bb |    1 +
 2 files changed, 44 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/1 smart:cache.py: getPackages() matches name + arch

It only matched name ony in the past, for example:
smart install busybox (matched)
but:
smart install busybox@core2_64 (didn't match)

The installation is very slow when no match since it would seach all the packages in the repo, and what we use mostly in oe-core is the second case, so the installation is very slow when install COMPLEMENTARY packages such as the task do_populate_sdk.

This patch makes it match both.

* Speed up
MACHINE = "qemux86-64"
- When multilib enabled:

    $ bitbake core-image-sato -cpopulate_sdk
    time: 6m5s -> 2m34s (Reduce 57% )
    $ bitbake core-image-minimal -cpopulate_sdk
    time: 2m1s -> 1m26s (Reduce 28% )
    $ bitbake core-image-sato-sdk
    time: 10m15s -> 7m12s (Reduce 29% )

- When multilib NOT enabled:

    $ bitbake core-image-sato-sdk -cpopulate_sdk
    time: 4m25s -> 2m28s (Reduce 44% )

[YOCTO #8389]

---
 ...cache.py-getPackages-matches-name-version.patch |   43 ++++++++++++++++++++
 meta/recipes-devtools/python/python-smartpm_git.bb |    1 +
 2 files changed, 44 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch
meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch new file +43 -0
new file mode 100644
index 0000000..225b02f
@@ -0,0 +1,43 @@
+From ee05e55e84b53f4bb0d0baba13ca47a8f84b7cb4 Mon Sep 17 00:00:00 2001
+From: Robert Yang <[email protected]>
+Date: Wed, 30 Sep 2015 01:12:52 -0700
+Subject: [PATCH] smart:cache.py: getPackages() matches name + arch
+
+It only matched name ony in the past, for example:
+smart install busybox (matched)
+but:
+smart install busybox@core2_64 (didn't match)
+
+The installation is very slow when no match since it would seach all the
+packages in the repo
+This patch makes it match both.
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <[email protected]>
+---
+ smart/cache.py |    3 ++-
+ smart/ccache.c |    9 ++++++++-
+ 2 files changed, 10 insertions(+), 2 deletions(-)
+
+diff --git a/smart/control.py b/smart/control.py
+index d44abe7..f23a604 100644
+--- a/smart/control.py
++++ b/smart/control.py
+@@ -876,9 +876,13 @@ class Control(object):
+         objects = []
+ 
+         # If we find packages with exactly the given
+-        # name or name-version, use them.
+-        for pkg in self._cache.getPackages(s):
+-            if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s:
++        # name, name-version, or name@arch, use them.
++        s_name = s
++        if "@" in s:
++            s_name = s.split("@")[0]
++        for pkg in self._cache.getPackages(s_name):
++            if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s \
++                    or "%s@%s" % (pkg.name, pkg.version.split('@')[1]) == s:
+                 objects.append((1.0, pkg))
+          
+         if not objects:
meta/recipes-devtools/python/python-smartpm_git.bb +1 -0
index 8b974b0..d6c378b 100644
@@ -23,6 +23,7 @@ SRC_URI = "\
           file://smart-add-for-rpm-ignoresize-check.patch \
           file://smart-already-installed-message.patch \
           file://smart-set-noprogress-for-pycurl.patch \
+          file://smart-cache.py-getPackages-matches-name-version.patch \
          "
 
 SRCREV = "407a7eca766431257dcd1da15175cc36a1bb22d0"
Signature · 1 line
1.7.9.5
#2 lore raw

Re: [PATCH 1/1] smart:cache.py: getPackages() matches name + arch

Quoted 18 lines · Robert Yang

It only matched name ony in the past, for example:
smart install busybox (matched)
but:
smart install busybox@core2_64 (didn't match)

The installation is very slow when no match since it would seach all the packages in the repo, and what we use mostly in oe-core is the second case, so the installation is very slow when install COMPLEMENTARY packages such as the task do_populate_sdk.

This patch makes it match both.

* Speed up
MACHINE = "qemux86-64"
- When multilib enabled:

     $ bitbake core-image-sato -cpopulate_sdk
     time: 6m5s -> 2m34s (Reduce 57% )
Quoted 8 lines
     $ bitbake core-image-minimal -cpopulate_sdk
     time: 2m1s -> 1m26s (Reduce 28% )
     $ bitbake core-image-sato-sdk
     time: 10m15s -> 7m12s (Reduce 29% )

- When multilib NOT enabled:

     $ bitbake core-image-sato-sdk -cpopulate_sdk

Sorry , a typo, this should be :

$ bitbake core-image-sato -cpopulate_sdk.

Updated in in the repo.

// Robert

Quoted 73 lines
     time: 4m25s -> 2m28s (Reduce 44% )

[YOCTO #8389]

---
  ...cache.py-getPackages-matches-name-version.patch |   43 ++++++++++++++++++++
  meta/recipes-devtools/python/python-smartpm_git.bb |    1 +
  2 files changed, 44 insertions(+)

create mode 100644 meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch

meta/recipes-devtools/python/python-smartpm/smart-cache.py-getPackages-matches-name-version.patch new file +43 -0
new file mode 100644
index 0000000..225b02f
@@ -0,0 +1,43 @@
+From ee05e55e84b53f4bb0d0baba13ca47a8f84b7cb4 Mon Sep 17 00:00:00 2001
+From: Robert Yang <[email protected]>
+Date: Wed, 30 Sep 2015 01:12:52 -0700
+Subject: [PATCH] smart:cache.py: getPackages() matches name + arch
+
+It only matched name ony in the past, for example:
+smart install busybox (matched)
+but:
+smart install busybox@core2_64 (didn't match)
+
+The installation is very slow when no match since it would seach all the
+packages in the repo
+This patch makes it match both.
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <[email protected]>
+---
+ smart/cache.py |    3 ++-
+ smart/ccache.c |    9 ++++++++-
+ 2 files changed, 10 insertions(+), 2 deletions(-)
+
+diff --git a/smart/control.py b/smart/control.py
+index d44abe7..f23a604 100644
+--- a/smart/control.py
++++ b/smart/control.py
+@@ -876,9 +876,13 @@ class Control(object):
+         objects = []
+
+         # If we find packages with exactly the given
+-        # name or name-version, use them.
+-        for pkg in self._cache.getPackages(s):
+-            if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s:
++        # name, name-version, or name@arch, use them.
++        s_name = s
++        if "@" in s:
++            s_name = s.split("@")[0]
++        for pkg in self._cache.getPackages(s_name):
++            if pkg.name == s or "%s-%s" % (pkg.name, pkg.version) == s \
++                    or "%s@%s" % (pkg.name, pkg.version.split('@')[1]) == s:
+                 objects.append((1.0, pkg))
+
+         if not objects:
meta/recipes-devtools/python/python-smartpm_git.bb +1 -0
index 8b974b0..d6c378b 100644
@@ -23,6 +23,7 @@ SRC_URI = "\
            file://smart-add-for-rpm-ignoresize-check.patch \
            file://smart-already-installed-message.patch \
            file://smart-set-noprogress-for-pycurl.patch \
+          file://smart-cache.py-getPackages-matches-name-version.patch \
           "
 
  SRCREV = "407a7eca766431257dcd1da15175cc36a1bb22d0"