git: Use merge-base instead of log for testing if a commit is present
PATCH 1/1 git: Use merge-base instead of log for testing if a commit is present
The current use of git log to check if a given revision is present can be a little fragile.
For example if revision X was on branch A, and then later added to branch B, the update checks would not notice this since they just check for X being in the repository.
We also had some autobuilder corruption where an older packed-refs file was copied over a new repository containing newer pack files. There was no update to the refs file since the revision was present but not accessible in any branch.
The correct fix is to check that the required revisions are present on the specific branches. This patch does this using merge-base.
---
bitbake/lib/bb/fetch2/git.py +9 -8
@@ -150,7 +150,7 @@ class Git(FetchMethod): return True os.chdir(ud.clonedir) for name in ud.names: - if not self._contains_ref(ud.revisions[name], d): + if not self._contains_ref(ud.revisions[name], ud.branches[name], d): return True if ud.write_tarballs and not os.path.exists(ud.fullmirror): return True
@@ -197,7 +197,7 @@ class Git(FetchMethod): # Update the checkout if needed needupdate = False for name in ud.names: - if not self._contains_ref(ud.revisions[name], d): + if not self._contains_ref(ud.revisions[name], ud.branches[name], d): needupdate = True if needupdate: try:
@@ -281,13 +281,14 @@ class Git(FetchMethod): def supports_srcrev(self): return True - def _contains_ref(self, tag, d): + def _contains_ref(self, tag, branch, d): basecmd = data.getVar("FETCHCMD_git", d, True) or "git" - cmd = "%s log --pretty=oneline -n 1 %s -- 2> /dev/null | wc -l" % (basecmd, tag) - output = runfetchcmd(cmd, d, quiet=True) - if len(output.split()) > 1: - raise bb.fetch2.FetchError("The command '%s' gave output with more then 1 line unexpectedly, output: '%s'" % (cmd, output)) - return output.split()[0] != "0" + cmd = "%s merge-base --is-ancestorlog %s %s" % (basecmd, tag, branch) + try: + output = runfetchcmd(cmd, d, quiet=True) + except bb.fetch2.FetchError: + return False + return True def _revision_key(self, url, ud, d, name): """
Quoted 7 lines · Richard Purdie
The current use of git log to check if a given revision is present can be a little fragile.
For example if revision X was on branch A, and then later added to branch B, the update checks would not notice this since they just check for X being in the repository.
Quoted 7 lines
We also had some autobuilder corruption where an older packed-refs file was copied over a new repository containing newer pack files. There was no update to the refs file since the revision was present but not accessible in any branch.
The correct fix is to check that the required revisions are present on the specific branches. This patch does this using merge-base.
I guess that merge-base is probably faster or easier to use, but did you consider using git branch --contains?
e.g. checking if selected branch is in
git branch --contains ud.revisions[name]
I'm asking only because I'm using "git branch --contains" in some scripts and maybe there is good reason I should rewrite them to use git merge-base instead.
Thanks
Quoted 51 lines
Signed-off-by Richard Purdie <[email protected]>---
bitbake/lib/bb/fetch2/git.py +9
-8@@ -150,7 +150,7 @@ class Git(FetchMethod): return True os.chdir(ud.clonedir) for name in ud.names: - if not self._contains_ref(ud.revisions[name], d): + if not self._contains_ref(ud.revisions[name], ud.branches[name], d): return True if ud.write_tarballs and not os.path.exists(ud.fullmirror): return True@@ -197,7 +197,7 @@ class Git(FetchMethod): # Update the checkout if needed needupdate = False for name in ud.names: - if not self._contains_ref(ud.revisions[name], d): + if not self._contains_ref(ud.revisions[name], ud.branches[name], d): needupdate = True if needupdate: try:@@ -281,13 +281,14 @@ class Git(FetchMethod): def supports_srcrev(self): return True - def _contains_ref(self, tag, d): + def _contains_ref(self, tag, branch, d): basecmd = data.getVar("FETCHCMD_git", d, True) or "git" - cmd = "%s log --pretty=oneline -n 1 %s -- 2> /dev/null | wc -l" % (basecmd, tag) - output = runfetchcmd(cmd, d, quiet=True) - if len(output.split()) > 1: - raise bb.fetch2.FetchError("The command '%s' gave output with more then 1 line unexpectedly, output: '%s'" % (cmd, output)) - return output.split()[0] != "0" + cmd = "%s merge-base --is-ancestorlog %s %s" % (basecmd, tag, branch) + try: + output = runfetchcmd(cmd, d, quiet=True) + except bb.fetch2.FetchError: + return False + return True def _revision_key(self, url, ud, d, name): """_______________________________________________
bitbake-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Quoted 19 lines · Martin Jansa
Quoted 7 lines · Richard Purdie
The current use of git log to check if a given revision is present can be a little fragile.
For example if revision X was on branch A, and then later added to branch B, the update checks would not notice this since they just check for X being in the repository.
Quoted 7 lines
We also had some autobuilder corruption where an older packed-refs file was copied over a new repository containing newer pack files. There was no update to the refs file since the revision was present but not accessible in any branch.
The correct fix is to check that the required revisions are present on the specific branches. This patch does this using merge-base.
I guess that merge-base is probably faster or easier to use, but did you consider using git branch --contains?
Quoted 6 lines
e.g. checking if selected branch is in
git branch --contains ud.revisions[name]I'm asking only because I'm using "git branch --contains" in some scripts and maybe there is good reason I should rewrite them to use git merge-base instead.
git branch --contains gives you a branch list back which you'd have to then parse to see if branch X was listed. git merge-base is faster and we can directly ask what we need with an exit code. There is no more deeply technical reason than that, was just neater.
Cheers,
Richard