Skip to content

RFC fetch overhaul - API reorg and git optimization

Yu Ke poky 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest11 patches 6 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11bitbake/fetch2/git: Add backwards compatibility code for branch name handlingbitbake/lib/bb/fetch2/git.py+80 -20 replies
2/11bb.fetch2: add unpack method in fetcherbitbake/lib/bb/fetch2/__init__.py+84 -00 replies
3/11bb.fetch2: revise the Fetch.unpack APIbitbake/lib/bb/fetch2/__init__.py+13 -160 replies
4/11bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2bitbake/lib/bb/fetch/__init__.py bitbake/lib/bb/fetch2/__init__.py+4 -00 replies
5/11base.bbclass: use bb.fetch2 unpack APImeta/classes/base.bbclass+7 -10 replies
6/11bb.fetch2: rename "go" with "download" to better reflect its functionalitybitbake/lib/bb/fetch2/__init__.py bitbake/lib/bb/fetch2/bzr.py bitbake/lib/bb/fetch2/cvs.py +10 more+19 -190 replies
7/11bbclasee: rename "go" with "download"meta/classes/base.bbclass meta/classes/sstate.bbclass+8 -20 replies
8/11bb.fetch2.git: split download to download() + build_mirror_data()bitbake/lib/bb/fetch2/__init__.py bitbake/lib/bb/fetch2/git.py+9 -30 replies
9/11bb.fetch2: remove the obsolate Fetch.try_mirrors referrencebitbake/lib/bb/fetch2/git.py+0 -80 replies
10/11bb.fetch2: add git unpackbitbake/lib/bb/fetch2/git.py+16 -06 replies
11/11git.py: remove the source tree tar ballbitbake/lib/bb/fetch2/git.py+4 -580 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/11] [RFC] fetch overhaul - API reorg and git optimization

Hi Richard,

This patch series is for the fetcher overhaul phase 2 - new API reorg, and part of phase 3 - git optimization. the details are:

- Split go() methods into the following steps:
i) download() - do whatever transfer is necessary to get the sources locally
ii) build_mirror_data() - generate any data that would be needed to construct a source mirror
iii) unpack() - takes a directory as an argument of where to place extracted sources
- Change Poky's fetch task to call a "download" method in the fetcher instead of go().
- Change Poky's unpack to call an unpack method in the fetchers.
- Optimise git fetcher for the new layout so for a git checkout, we clone the repository in download(), then unpack just does a clone with references to the original repo.

The remaining item for phase 3 are:
- Add a new "BB_NO_NETWORK" option to only perform local fetch
- Add multiple branch/srcrev support for git fetcher

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: kyu3/fetcher-api
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/fetcher-api

Thanks,
Yu Ke <[email protected]>
---

Yu Ke (11):
bitbake/fetch2/git: Add backwards compatibility code for branch name
handling
bb.fetch2: add unpack method in fetcher
bb.fetch2: revise the Fetch.unpack API
bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2 base.bbclass: use bb.fetch2 unpack API
bb.fetch2: rename "go" with "download" to better reflect its
functionality
bbclasee: rename "go" with "download"
bb.fetch2.git: split download to download() + build_mirror_data() bb.fetch2: remove the obsolate Fetch.try_mirrors referrence bb.fetch2: add git unpack
git.py: remove the source tree tar ball

 bitbake/lib/bb/fetch/__init__.py  |    2 +
 bitbake/lib/bb/fetch2/__init__.py |   95 +++++++++++++++++++++-
 bitbake/lib/bb/fetch2/bzr.py      |    2 +-
 bitbake/lib/bb/fetch2/cvs.py      |    2 +-
 bitbake/lib/bb/fetch2/git.py      |  158 ++++++++++++++++++++++--------------
 bitbake/lib/bb/fetch2/hg.py       |    2 +-
 bitbake/lib/bb/fetch2/local.py    |    2 +-
 bitbake/lib/bb/fetch2/osc.py      |    2 +-
 bitbake/lib/bb/fetch2/perforce.py |    2 +-
 bitbake/lib/bb/fetch2/repo.py     |    2 +-
 bitbake/lib/bb/fetch2/ssh.py      |    2 +-
 bitbake/lib/bb/fetch2/svk.py      |    2 +-
 bitbake/lib/bb/fetch2/svn.py      |    2 +-
 bitbake/lib/bb/fetch2/wget.py     |    4 +-
 meta/classes/base.bbclass         |   13 +++-
 meta/classes/sstate.bbclass       |    5 +-
 16 files changed, 216 insertions(+), 81 deletions(-)