update.py: performance improve and 2 fixes

Hi Paul,

This email contains two parts:
- The first 2 are bug fixes
- The other 3 are performance improvements, I have 124 layers,
the "update.py -b <branch>" needs about 9m20s to finish the running when everything is update, and we have several branches, we need them to run in periodically (e.g., per 30 mins, the short the better), so one branch 9m20s is really a problem. Only about 1m43s are needed with these improvements when set PARALLEL_JOBS to 10:
update.py: fetch repos parallelly # Saved 2 mins
update_layer.py: use it as a module # Saved 5 mins
update_layer.py: only call init_parser when needed # Saved 1 min

// Robert

The following changes since commit fa0b01082fb0596959841e95af8c516ae3f267a7:

admin: fix dependency display and search (2017-12-21 16:14:17 +1300)

are available in the git repository at:

   git://git.pokylinux.org/poky-contrib rbt/LI
   http://git.pokylinux.org/cgit.cgi//log/?h=rbt/LI

Robert Yang (5):
utils.py: fix remove obsolete dependencies update.py: make sure oe-core is fetched update.py: fetch repos parallelly update_layer.py: use it as a module update_layer.py: only call init_parser when needed

  layerindex/update.py       | 176 +++++++++++++++++++++++----------------------
  layerindex/update_layer.py | 146 +++++++++++--------------------------
  layerindex/utils.py        |   6 +-
  settings.py                |   2 +
  4 files changed, 137 insertions(+), 193 deletions(-)

Hi Paul,

This email contains two parts:
- The first 2 are bug fixes
- The other 3 are performance improvements, I have 124 layers,
the "update.py -b <branch>" needs about 9m20s to finish the running when everything is update, and we have several branches, we need them to run in periodically (e.g., per 30 mins, the short the better), so one branch 9m20s is really a problem. Only about 1m43s are needed with these improvements when set PARALLEL_JOBS to 10:
update.py: fetch repos parallelly # Saved 2 mins
update_layer.py: use it as a module # Saved 5 mins
update_layer.py: only call init_parser when needed # Saved 1 min

// Robert

The following changes since commit fa0b01082fb0596959841e95af8c516ae3f267a7:

admin: fix dependency display and search (2017-12-21 16:14:17 +1300)

are available in the git repository at:

   git://git.pokylinux.org/poky-contrib rbt/LI
   http://git.pokylinux.org/cgit.cgi//log/?h=rbt/LI

Robert Yang (5):
utils.py: fix remove obsolete dependencies update.py: make sure oe-core is fetched update.py: fetch repos parallelly update_layer.py: use it as a module update_layer.py: only call init_parser when needed

  layerindex/update.py       | 176 +++++++++++++++++++++++----------------------
  layerindex/update_layer.py | 146 +++++++++++--------------------------
  layerindex/utils.py        |   6 +-
  settings.py                |   2 +
  4 files changed, 137 insertions(+), 193 deletions(-)

Quoted 6 lines · Robert Yang

This email contains two parts:
- The first 2 are bug fixes
- The other 3 are performance improvements, I have 124 layers, the "update.py -b <branch>" needs about 9m20s to finish the running when everything is
update, and we have several branches, we need them to run in

Quoted 8 lines

periodically
(e.g., per 30 mins, the short the better), so one branch 9m20s is really a problem.
Only about 1m43s are needed with these improvements when set PARALLEL_JOBS to 10:
update.py: fetch repos parallelly # Saved 2 mins update_layer.py: use it as a module # Saved 5 mins update_layer.py: only call init_parser when needed # Saved 1 min

Any comments, please ?