bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
PATCH 1/1 bitbake: fetch2/gitsm: avoid live submodule fetching during unpack()
From: Matt Hoosier <[email protected]>
Although the submodules' histories have been fetched during the do_fetch() phase, the mechanics used to clone the workdir copy of the repo haven't been transferring the actual .git/modules directory from the repo fetched into downloads/ during the fetch task.
Fix that, and for good measure also explicitly tell Git to avoid hitting the network during do_unpack() of the submodules.
Ref:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12739
--- bitbake/lib/bb/fetch2/gitsm.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
bitbake/lib/bb/fetch2/gitsm.py +11 -1
@@ -132,4 +132,14 @@ class GitSM(Git): if self.uses_submodules(ud, d, ud.destdir): runfetchcmd(ud.basecmd + " checkout " + ud.revisions[ud.names[0]], d, workdir=ud.destdir) - runfetchcmd(ud.basecmd + " submodule update --init --recursive", d, workdir=ud.destdir) + + # Copy over the submodules' fetched histories too. + if ud.bareclone: + repo_conf = ud.destdir + else: + repo_conf = os.path.join(ud.destdir, '.git') + runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir, 'modules'), repo_conf), d) + + # Careful not to hit the network during unpacking; all history should already + # be fetched. + runfetchcmd(ud.basecmd + " submodule update --init --recursive --no-fetch", d, workdir=ud.destdir)
Signature · 1 line
2.13.6
Quoted 8 lines · Matt Hoosier
From: Matt Hoosier <[email protected]>
Although the submodules' histories have been fetched during the
do_fetch()phase, the mechanics used to clone the workdir copy of the repo haven't been transferring the actual .git/modules directory from the repo fetched into downloads/ during the fetch task.
Quoted 5 lines
Fix that, and for good measure also explicitly tell Git to avoid hitting the network during
do_unpack()of the submodules.
Patches for bitbake should be sent to the bitbake-devel mailing list:
http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Quoted 32 lines
--- bitbake/lib/bb/fetch2/gitsm.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)bitbake/lib/bb/fetch2/gitsm.py +11
-2@@ -132,4 +132,14 @@ class GitSM(Git): if self.uses_submodules(ud, d, ud.destdir): runfetchcmd(ud.basecmd + " checkout " + ud.revisions[ud.names[0]], d, workdir=ud.destdir) - runfetchcmd(ud.basecmd + " submodule update --init --recursive", d, workdir=ud.destdir) + + # Copy over the submodules' fetched histories too. + if ud.bareclone: + repo_conf = ud.destdir + else: + repo_conf = os.path.join(ud.destdir, '.git') + runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir, 'modules'), repo_conf), d) + + # Careful not to hit the network during unpacking; all history should already + # be fetched. + runfetchcmd(ud.basecmd + " submodule update --init --recursive --no-fetch", d, workdir=ud.destdir) --2.13.6
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
Quoted 13 lines · Andre McCurdy
Quoted 11 lines · Matt Hoosier
From: Matt Hoosier <[email protected]>
Although the submodules' histories have been fetched during the
do_fetch()phase, the mechanics used to clone the workdir copy of the repo haven't been transferring the actual .git/modules directory from the repo fetched into downloads/ during the fetch task.Fix that, and for good measure also explicitly tell Git to avoid hitting the network during
do_unpack()of the submodules.
Quoted 6 lines
Quoted 2 lines
Patches for bitbake should be sent to the bitbake-devel mailing list:
http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Right, okay. How's that meant to work for stuff developed in-tree with poky? Do you manually want the 'bitbake/' leading directory stripped off before emailing the patch?
Quoted 41 lines
Review
Quoted 7 lines
--- bitbake/lib/bb/fetch2/gitsm.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/bitbake/lib/bb/fetch2/gitsm.pybitbake/lib/bb/fetch2/gitsm.py class GitSM(Git): · line 134
Quoted 7 lines
index
0aff1008e5..1f3fc44352100644bitbake/lib/bb/fetch2/gitsm.py +0
-0@@ -132,4 +132,14 @@ class GitSM(Git): if self.uses_submodules(ud, d, ud.destdir): runfetchcmd(ud.basecmd + " checkout " +ud.revisions[ud.names[0]], d, workdir=ud.destdir)
bitbake/lib/bb/fetch2/gitsm.py class GitSM(Git): · line 135
Quoted 1 line
bitbake/lib/bb/fetch2/gitsm.py +0
-1- runfetchcmd(ud.basecmd + " submodule update --init--recursive", d, workdir=ud.destdir)
bitbake/lib/bb/fetch2/gitsm.py class GitSM(Git):
Quoted 7 lines
bitbake/lib/bb/fetch2/gitsm.py +7
-0+ + # Copy over the submodules' fetched histories too. + if ud.bareclone: + repo_conf = ud.destdir + else: + repo_conf = os.path.join(ud.destdir, '.git') + runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,'modules'),
repo_conf), d)bitbake/lib/bb/fetch2/gitsm.py class GitSM(Git): · line 143
Quoted 2 lines
bitbake/lib/bb/fetch2/gitsm.py +2
-0+ + # Careful not to hit the network during unpacking; allhistory should already
bitbake/lib/bb/fetch2/gitsm.py class GitSM(Git): · line 145
Quoted 2 lines
bitbake/lib/bb/fetch2/gitsm.py +2
-0+ # be fetched. + runfetchcmd(ud.basecmd + " submodule update --init--recursive --no-fetch", d, workdir=ud.destdir)
Quoted 8 lines
--
2.13.6--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
Quoted 18 lines · Matt Hoosier
Quoted 17 lines · Andre McCurdy
Quoted 13 lines · Matt Hoosier
From: Matt Hoosier <[email protected]>
Although the submodules' histories have been fetched during the
do_fetch()phase, the mechanics used to clone the workdir copy of the repo haven't been transferring the actual .git/modules directory from the repo fetched into downloads/ during the fetch task.Fix that, and for good measure also explicitly tell Git to avoid hitting the network during
do_unpack()of the submodules.
Quoted 7 lines
Quoted 3 lines
Patches for bitbake should be sent to the bitbake-devel mailing list:
http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Right, okay. How's that meant to work for stuff developed in-tree with poky? Do you manually want the 'bitbake/' leading directory stripped off before emailing the patch?
Yes, patches for bitbake should apply directly to a clone of the bitbake repo.
Manually editing a patch created from poky is certainly do-able, but applying the patch to a genuine bikebake repo (with "git am -p2 ...") and re-generating the patch from there may be safer. Whichever you prefer.
Quoted 40 lines
Quoted 38 lines
Quoted 38 lines
--- bitbake/lib/bb/fetch2/gitsm.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)bitbake/lib/bb/fetch2/gitsm.py +7
-2@@ -132,4 +132,14 @@ class GitSM(Git): if self.uses_submodules(ud, d, ud.destdir): runfetchcmd(ud.basecmd + " checkout " + ud.revisions[ud.names[0]], d, workdir=ud.destdir) - runfetchcmd(ud.basecmd + " submodule update --init --recursive", d, workdir=ud.destdir) + + # Copy over the submodules' fetched histories too. + if ud.bareclone: + repo_conf = ud.destdir + else: + repo_conf = os.path.join(ud.destdir, '.git') + runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,'modules'),
repo_conf), d)bitbake/lib/bb/fetch2/gitsm.py +2
-0+ + # Careful not to hit the network during unpacking; allhistory should already
bitbake/lib/bb/fetch2/gitsm.py +2
-2+ # be fetched. + runfetchcmd(ud.basecmd + " submodule update --init --recursive --no-fetch", d, workdir=ud.destdir) --2.13.6
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
Quoted 20 lines · Matt Hoosier
Quoted 19 lines · Andre McCurdy
Quoted 8 lines · Matt Hoosier
From: Matt Hoosier <[email protected]>
Although the submodules' histories have been fetched during the
do_fetch()phase, the mechanics used to clone the workdir copy of the repo haven't been transferring the actual .git/modules directory from the repo fetched into downloads/ during the fetch task.Quoted 5 lines
Fix that, and for good measure also explicitly tell Git to avoid hitting the network during
do_unpack()of the submodules.Patches for bitbake should be sent to the bitbake-devel mailing list:
Quoted 7 lines
Right, okay. How's that meant to work for stuff developed in-tree with poky? Do you manually want the 'bitbake/' leading directory stripped off before emailing the patch?
If you're feeling lazy you can just send the patch to bitbake@ from a poky repo and git will figure it out. The same works the other way: a bitbake patch applies fine to a poky repository.
Ross