git-p4: speed up search for branch parent

In this iteration, I have added more context and measurements to the commit message.

I have also made small improvements to the code suggested by reviewers.

I enhanced t9801-git-p4-branch.sh to test for the functionality, namely that branches are branched off at the correct point in their parents' history.

Joachim Kuebart (2):
git-p4: ensure complex branches are cloned correctly
git-p4: speed up search for branch parent

 git-p4.py                | 21 ++++++++++-----------
 t/t9801-git-p4-branch.sh |  2 ++
 2 files changed, 12 insertions(+), 11 deletions(-)

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1013%2Fjkuebart%2Fp4-faster-parent-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1013/jkuebart/p4-faster-parent-v2
Pull-Request: https://github.com/git/git/pull/1013

Range-diff vs v1:

 -:  ------------ > 1:  0ee0b7b55691 git-p4: ensure complex branches are cloned correctly
 1:  a171f7e6c023 ! 2:  41b3a23f682c git-p4: speed up search for branch parent

@@ Metadata
## Commit message ##
git-p4: speed up search for branch parent

- Previously, the code iterated through the parent branch commits and
- compared each one to the target tree using diff-tree.
+ For every new branch that git-p4 imports, it needs to find the commit
+ where it branched off its parent branch. While p4 doesn't record this
+ information explicitly, the first changelist on a branch is usually an
+ identical copy of the parent branch.

- This patch outputs the revision's tree hash along with the commit hash,
- thereby saving the diff-tree invocation. This results in a considerable
- speed-up, at least on Windows.
+ The method searchParent() tries to find a commit in the history of the
+ given "parent" branch whose tree exactly matches the initial changelist
+ of the new branch, "target". The code iterates through the parent
+ commits and compares each of them to this initial changelist using
+ diff-tree.
+
+ Since we already know the tree object name we are looking for, spawning
+ diff-tree for each commit is wasteful.
+
+ Use the "--format" option of "rev-list" to find out the tree object name
+ of each commit in the history, and find the tree whose name is exactly
+ the same as the tree of the target commit to optimize this.
+
+ This results in a considerable speed-up, at least on Windows. On one
+ Windows machine with a fairly large repository of about 16000 commits in
+ the parent branch, the current code takes over 7 minutes, while the new
+ code only takes just over 10 seconds for the same changelist:
+
+ Before:
+
+ $ time git p4 sync
+ Importing from/into multiple branches
+ Depot paths: //depot
+ Importing revision 31274 (100.0%)
+ Updated branches: b1
+
+ real 7m41.458s
+ user 0m0.000s
+ sys 0m0.077s
+
+ After:
+
+ $ time git p4 sync
+ Importing from/into multiple branches
+ Depot paths: //depot
+ Importing revision 31274 (100.0%)
+ Updated branches: b1
+
+ real 0m10.235s
+ user 0m0.000s
+ sys 0m0.062s

+ Helped-by: Junio C Hamano <[email protected]>
+ Helped-by: Luke Diamand <[email protected]>

## git-p4.py ##
@@ git-p4.py: def importNewBranch(self, branch, maxChange):
@@ git-p4.py: def importNewBranch(self, branch, maxChange):
def searchParent(self, parent, branch, target):
- parentFound = False
- for blob in read_pipe_lines(["git", "rev-list", "--reverse",
-+ for tree in read_pipe_lines(["git", "rev-parse",
-+ "{}^{{tree}}".format(target)]):
-+ targetTree = tree.strip()
-+ for blob in read_pipe_lines(["git", "rev-list", "--format=%H %T",
++ targetTree = read_pipe(["git", "rev-parse",
++ "{}^{{tree}}".format(target)]).strip()
++ for line in read_pipe_lines(["git", "rev-list", "--format=%H %T",
"--no-merges", parent]):
- blob = blob.strip()
- if len(read_pipe(["git", "diff-tree", blob, target])) == 0:
- parentFound = True
-+ if blob[:7] == "commit ":
++ if line.startswith("commit "):
+ continue
-+ blob = blob.strip().split(" ")
-+ if blob[1] == targetTree:
++ commit, tree = line.strip().split(" ")
++ if tree == targetTree:
if self.verbose:
- print("Found parent of %s in commit %s" % (branch, blob))
- break
@@ git-p4.py: def importNewBranch(self, branch, maxChange):
- return blob
- else:
- return None
-+ print("Found parent of %s in commit %s" % (branch, blob[0]))
-+ return blob[0]
++ print("Found parent of %s in commit %s" % (branch, commit))
++ return commit
+ return None

def importChanges(self, changes, origin_revision=0):

gitgitgadget