Skip to content

RFC fetcher overhaul - api reorg and git optimization V2

Yu Ke poky 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest21 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/21bitbake/fetch2/git: Add backwards compatibility code for branch name handlingbitbake/lib/bb/fetch2/git.py+80 -20 replies
2/21bb.fetch2: add unpack method in fetcherbitbake/lib/bb/fetch2/__init__.py+84 -00 replies
3/21bb.fetch2: revise the Fetch.unpack APIbitbake/lib/bb/fetch2/__init__.py+15 -180 replies
4/21bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2bitbake/lib/bb/fetch/__init__.py bitbake/lib/bb/fetch2/__init__.py+4 -00 replies
5/21base.bbclass: use bb.fetch2 unpack APImeta/classes/base.bbclass+7 -10 replies
6/21bb.fetch2: rename "go" with "download" to better reflect its functionalitybitbake/lib/bb/fetch2/__init__.py bitbake/lib/bb/fetch2/bzr.py bitbake/lib/bb/fetch2/cvs.py +10 more+19 -190 replies
7/21bbclasee: rename "go" with "download"meta/classes/base.bbclass meta/classes/sstate.bbclass+8 -20 replies
8/21git.py: split download to download() and build_mirror_data()bitbake/lib/bb/fetch2/__init__.py bitbake/lib/bb/fetch2/git.py+9 -30 replies
9/21bb.fetch2: remove the obsolate Fetch.try_mirrors referrencebitbake/lib/bb/fetch2/git.py+0 -80 replies
10/21bb.fetch2: add git unpackbitbake/lib/bb/fetch2/git.py+16 -00 replies
11/21git.py: remove the source tree tar ballbitbake/lib/bb/fetch2/git.py+4 -580 replies
12/21bb.fetch2: add "BB_NO_NETWORK" optionbitbake/lib/bb/fetch2/__init__.py+10 -00 replies
13/21git.py: add no network access checkbitbake/lib/bb/fetch2/git.py+3 -00 replies
14/21wget.py: add no network checkbitbake/lib/bb/fetch2/wget.py+1 -00 replies
15/21svn.py: add no network access checkbitbake/lib/bb/fetch2/svn.py+3 -10 replies
16/21cvs.py: add no network access checkbitbake/lib/bb/fetch2/cvs.py+2 -00 replies
17/21hg.py: add no network access checkbitbake/lib/bb/fetch2/hg.py+3 -00 replies
18/21bzr.py: add no network access checkbitbake/lib/bb/fetch2/bzr.py+4 -00 replies
19/21repo.py: add no network access checkbitbake/lib/bb/fetch2/repo.py+2 -00 replies
20/21osc.py: add no network access checkbitbake/lib/bb/fetch2/osc.py+2 -00 replies
21/21ssh.py: add no network access checkbitbake/lib/bb/fetch2/ssh.py+2 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 00/21] fetcher overhaul - api reorg and git optimization V2

Hi Richard,

This is the V2 patch series for the fetcher overhaul phase 2 - new API reorg, and phase 3 - git optimization.

the changes compared with V1 are:
- fix some typo in api reorg patches, now api reorg patch has pass the test of x86 & arm poky-image-sato with empty DL_DIR
- add "BB_NO_NETWORK" option support

The overall changes are:
- Split go() methods into the following steps:
i) download() - do whatever transfer is necessary to get the sources locally
ii) build_mirror_data() - generate any data that would be needed to construct a source mirror
iii) unpack() - takes a directory as an argument of where to place extracted sources
- Change Poky's fetch task to call a "download" method in the fetcher instead of go().
- Change Poky's unpack to call an unpack method in the fetchers.
- Optimise git fetcher for the new layout so for a git checkout, we clone the repository in download(), then unpack just does a clone with references to the original repo.
- Add a new "BB_NO_NETWORK" option to only perform local fetch

The remaining item for phase 3 are:
- Add multiple branch/srcrev support for git fetcher

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: kyu3/fetcher-api-v2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/fetcher-api-v2

Thanks,
Yu Ke <[email protected]>
---

Yu Ke (21):
bitbake/fetch2/git: Add backwards compatibility code for branch name
handling
bb.fetch2: add unpack method in fetcher
bb.fetch2: revise the Fetch.unpack API
bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2 base.bbclass: use bb.fetch2 unpack API
bb.fetch2: rename "go" with "download" to better reflect its
functionality
bbclasee: rename "go" with "download"
git.py: split download to download() and build_mirror_data() bb.fetch2: remove the obsolate Fetch.try_mirrors referrence bb.fetch2: add git unpack
git.py: remove the source tree tar ball
bb.fetch2: add "BB_NO_NETWORK" option
git.py: add no network access check
wget.py: add no network check
svn.py: add no network access check
cvs.py: add no network access check
hg.py: add no network access check
bzr.py: add no network access check
repo.py: add no network access check
osc.py: add no network access check
ssh.py: add no network access check

 bitbake/lib/bb/fetch/__init__.py  |    2 +
 bitbake/lib/bb/fetch2/__init__.py |  105 ++++++++++++++++++++++-
 bitbake/lib/bb/fetch2/bzr.py      |    6 +-
 bitbake/lib/bb/fetch2/cvs.py      |    4 +-
 bitbake/lib/bb/fetch2/git.py      |  165 ++++++++++++++++++++++--------------
 bitbake/lib/bb/fetch2/hg.py       |    5 +-
 bitbake/lib/bb/fetch2/local.py    |    2 +-
 bitbake/lib/bb/fetch2/osc.py      |    4 +-
 bitbake/lib/bb/fetch2/perforce.py |    2 +-
 bitbake/lib/bb/fetch2/repo.py     |    4 +-
 bitbake/lib/bb/fetch2/ssh.py      |    4 +-
 bitbake/lib/bb/fetch2/svk.py      |    2 +-
 bitbake/lib/bb/fetch2/svn.py      |    6 +-
 bitbake/lib/bb/fetch2/wget.py     |    5 +-
 meta/classes/base.bbclass         |   13 +++-
 meta/classes/sstate.bbclass       |    5 +-
 16 files changed, 250 insertions(+), 84 deletions(-)