Skip to contentSkip to conversation

combine git repo historically

DISCUSSION bill lam 6 participants 12 messages Jun 15, 2016

lore
Share & tools
lore

bill lam

author
#0 lore raw

I have two git repos, no branches.

repo 1.
emptyrootcommit -- A ... M

repo 2.
emptyrootcommit -- N ... Z

N was evolved from M but the time gap is large, how can I combine them into one repo

emptyrootcommit -- A ... M -- N ... Z

so that snapshots N .. Z will not be changed.

Signature · 4 lines
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
#1 lore raw
Quoted 8 lines · bill lam

I have two git repos, no branches.

repo 1.
emptyrootcommit -- A ... M

repo 2.
emptyrootcommit -- N ... Z

Quoted 6 lines

N was evolved from M but the time gap is large, how can I combine them into one repo

emptyrootcommit -- A ... M -- N ... Z

so that snapshots N .. Z will not be changed.

$ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
$ git filter-branch --tag-name-filter cat -- --all --not M

i.e. you graft the older history right before the younger history, then you use git filter-branch to rewrite the parentship of the younger commits.

-- Hannes

#2 lore raw
Quoted 14 lines · Johannes Sixt
Quoted 13 lines · bill lam

I have two git repos, no branches.

repo 1.
  emptyrootcommit -- A ... M

repo 2.
  emptyrootcommit -- N ... Z

N was evolved from M but the time gap is large, how can I combine them into one repo

emptyrootcommit -- A ... M -- N ... Z

Quoted 7 lines
Quoted 1 line

so that snapshots N .. Z will not be changed.

$ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
$ git filter-branch --tag-name-filter cat -- --all --not M

i.e. you graft the older history right before the younger history, then you use git filter-branch to rewrite the parentship of the younger commits.

If you cannot create a new history, using "git replace" could be better than using grafts.
("git replace" is in the "master" branch in the git repository. It will be in git 1.6.5 that should be released soon.)

Regards,
Christian

author
#3 lore raw
In reply to Johannes Sixt
Quoted 14 lines · Johannes Sixt
Quoted 13 lines · bill lam

I have two git repos, no branches.

repo 1.
emptyrootcommit -- A ... M

repo 2.
emptyrootcommit -- N ... Z

N was evolved from M but the time gap is large, how can I combine them into one repo

emptyrootcommit -- A ... M -- N ... Z

Quoted 7 lines
Quoted 1 line

so that snapshots N .. Z will not be changed.

$ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
$ git filter-branch --tag-name-filter cat -- --all --not M

i.e. you graft the older history right before the younger history, then you use git filter-branch to rewrite the parentship of the younger commits.

Thanks, graft is new to me. I tested it works but it needs first issue
git fetch /path/to/repo1
within repo2 to fetch tip M from repo1 into repo2, is it correct?

If I also want to fetch also all objects from repo1, what will the command to do it.

Christian Couder also mentioned the git-replace command, how to stitch with it?

Signature · 4 lines
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
#4 lore raw
Quoted 24 lines · bill lam
Quoted 14 lines · Johannes Sixt
Quoted 13 lines · bill lam

I have two git repos, no branches.

repo 1.
emptyrootcommit -- A ... M

repo 2.
emptyrootcommit -- N ... Z

N was evolved from M but the time gap is large, how can I combine them into one repo

emptyrootcommit -- A ... M -- N ... Z

Quoted 8 lines
Quoted 1 line

so that snapshots N .. Z will not be changed.

$ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
$ git filter-branch --tag-name-filter cat -- --all --not M

i.e. you graft the older history right before the younger history, then you use git filter-branch to rewrite the parentship of the younger commits.

Quoted 6 lines

Thanks, graft is new to me. I tested it works but it needs first issue
git fetch /path/to/repo1
within repo2 to fetch tip M from repo1 into repo2, is it correct?

If I also want to fetch also all objects from repo1, what will the command to do it.

Maybe you should do something like:

$ git remote add repo1 /path/to/repo1
$ git fetch repo1

The fetch command should output something like:

remote: Counting objects: 423, done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 363 (delta 276), reused 298 (delta 213) Receiving objects: 100% (363/363), 136.65 KiB, done. Resolving deltas: 100% (276/276), completed with 29 local objects. From /path/to/repo1

 * [new branch]      branch1       -> repo1/branch1
 * [new branch]      branch2       -> repo1/branch2

So the branches in repo1 will be available in repo2 as remote/repo1/branch1, remote/repo1/branch2, ...

Quoted 2 lines

Christian Couder also mentioned the git-replace command, how to stitch with it?

First you need to create a commit that will replace commit N, let's call it N' and after that you can use "git replace N N'". So the complex part is to create N'.

You want N' to have the same content as N but to have M as parent. So you could do something like the following:

(We suppose that commits A to M are in branch1 and that you are in the root directory of your repo2 working directory.)

$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit -a

And then use the commit message from commit N, and maybe also say in the commit message that it is replacement commit made to link repo1 with repo2 or something like that.

At this step you have created N' and you should make sure that it is exactly what you want. It should point to the same tree as N, it should have M as parent, ...

If everything is ok then you can use:

$ git replace N HEAD

And you should be done.

Regards,
Christian.

author
#5 lore raw
Quoted 6 lines · Christian Couder
$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
...
If everything is ok then you can use:
$ git replace N HEAD

Thanks for detail instruction, I tested it ok except that when trying to checkout it reported an error

  $ git checkout -b repo1-branch1 remote/oldjproject/master
  fatal: git checkout: updating paths is incompatible with switching branches. 
  Did you intend to checkout 'remote/oldjproject/master' which can not
  be resolved as commit?

but it ran ok by omitting that 'remote/'

$ git checkout -b repo1-branch1 oldjproject/master

Does it need to purge the file system tree before
git checkout N -- .

so that there will be no artifact leaved by M?

I found that it is necessary to do a
git reset --hard (original HEAD)

to complete the story. Is it correct?

git version 1.6.5.rc3.35.g3340

Signature · 4 lines
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
#6 lore raw
Quoted 16 lines · bill lam
Quoted 6 lines · Christian Couder
$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
...
If everything is ok then you can use:
$ git replace N HEAD

Thanks for detail instruction, I tested it ok except that when trying to checkout it reported an error

  $ git checkout -b repo1-branch1 remote/oldjproject/master
  fatal: git checkout: updating paths is incompatible with switching
branches. Did you intend to checkout 'remote/oldjproject/master' which
can not be resolved as commit?
Quoted 7 lines

but it ran ok by omitting that 'remote/'

$ git checkout -b repo1-branch1 oldjproject/master

Does it need to purge the file system tree before
git checkout N -- .

so that there will be no artifact leaved by M?

Yeah, perhaps, I forgot that there could be some files added and other files removed between M and N.

If you remove everything (except the .git directory of course), then doing:

$ git checkout N -- .
$ git add .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit

should do the trick if I am not missing something which may very well be the case.

So perhaps it is easier to use Jakub's suggestion instead.

Quoted 4 lines

I found that it is necessary to do a
git reset --hard (original HEAD)

to complete the story. Is it correct?

Yes, you should go back to your orginal branch at the end.

Regards,
Christian.

#7 lore raw
In reply to Christian Couder
Quoted 6 lines · Christian Couder

You want N' to have the same content as N but to have M as parent. So you could do something like the following:

(We suppose that commits A to M are in branch1 and that you are in the root directory of your repo2 working directory.)

Quoted 6 lines
$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit -a

Isn't that what git cherry-pick does?

Andreas.

Signature · 3 lines
Andreas Schwab, [email protected]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#8 lore raw
In reply to Christian Couder
Quoted 22 lines · Christian Couder
Quoted 2 lines · bill lam

Christian Couder also mentioned the git-replace command, how to stitch with it?

First you need to create a commit that will replace commit N, let's call it N' and after that you can use "git replace N N'". So the complex part is to create N'.

You want N' to have the same content as N but to have M as parent. So you could do something like the following:

(We suppose that commits A to M are in branch1 and that you are in the root directory of your repo2 working directory.)

$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit -a
Quoted 3 lines

And then use the commit message from commit N, and maybe also say in the commit message that it is replacement commit made to link repo1 with repo2 or something like that.

I think simpler solution would be to use plumbing for that. First save commit to be replaced in a (text) file:

 $ git cat-file commit N > COMMIT_N

If you want to edit only commit message and perhaps parentage (like I think in this case), you need to simply edit COMMIT_N file, and modify (or add) 'parent' header(s), which is between 'tree' and 'author' header.

Then put modified commit in repository object database

 $ git hash-object -t commit -w COMMIT_N
Quoted 7 lines

At this step you have created N' and you should make sure that it is exactly what you want. It should point to the same tree as N, it should have M as parent, ...

If everything is ok then you can use:

$ git replace N HEAD

And then do

 $ git replace N NEW_N

where N is _SHA-1_ of original comit ("git rev-parse --verify N^0"), and NEW_N is SHA-1 of replacement commit, as written by git-hash-object.

Quoted 1 line

And you should be done.

But I haven't tested this. YMMV.

See also my answer here:
http://stackoverflow.com/questions/1220557/how-do-i-prepend-history-to-a-git-repo/1547490#1547490

Signature · 3 lines
Jakub Narebski
Poland
ShadeHawk on #git
#9 lore raw
Quoted 7 lines · Jakub Narebski
Quoted 6 lines · Christian Couder
$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit -a

This would not preserver the committer, but in particular the committer date is rather important for history traversal. You really don't want to change it. I used a much more elaborate pipleine involving git-commit-tree because I did not think about this simple procedure:

Quoted 3 lines
 $ git cat-file commit N > COMMIT_N
[ $ edit COMMIT_N  ]
 $ git hash-object -t commit -w COMMIT_N

Thanks for this tip, Jakub.

Christian, one thing that worries me is that 'git cat-file commit foo' returns the replacement (bar) instead of the original (foo). Is it intended? There is no way to retrieve the original commit using plumbing unless the replacement is removed. Am I right?

-- Hannes

#10 lore raw
Quoted 19 lines · Johannes Sixt
Quoted 7 lines · Jakub Narebski
Quoted 6 lines · Christian Couder
$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit -a

This would not preserver the committer, but in particular the committer date is rather important for history traversal. You really don't want to change it. I used a much more elaborate pipleine involving git-commit-tree because I

did not think about this simple procedure:

Quoted 3 lines
 $ git cat-file commit N > COMMIT_N
[ $ edit COMMIT_N  ]
 $ git hash-object -t commit -w COMMIT_N
Quoted 5 lines

Thanks for this tip, Jakub.

Christian, one thing that worries me is that 'git cat-file commit foo' returns the replacement (bar) instead of the original (foo). Is it intended?

Yes it is intended. "git replace" was designed so that the original objects are not replaced only when reachability traversal (prune, pack transfer and fsck) wants to read the commit. See:

http://article.gmane.org/gmane.comp.version-control.git/104776/

Quoted 2 lines

There is no way to retrieve the original commit using plumbing unless the replacement is removed. Am I right?

I think you are right, but a flag could be easily added to some commands to retreive original objects. Or you can remove the replacement, retrieve the object and then put back the replacement.

Best regards,
Christian.

#11 lore raw
Quoted 8 lines · Christian Couder
Quoted 2 lines · Johannes Sixt

There is no way to retrieve the original commit using plumbing unless the replacement is removed. Am I right?

I think you are right, but a flag could be easily added to some commands to retreive original objects. Or you can remove the replacement, retrieve the object and then put back the replacement.

Wouldn't it be better for this option to be option for git wrapper, i.e.

 $ git --no-replace cat-file -p N

and not

 $ git cat-file --no-replace -p N
Signature · 2 lines
Jakub Narebski
Poland