Skip to contentSkip to conversation

Switch from svn to git and modify repo completely

DISCUSSION Matthias Fechner 3 participants 6 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

Dear Git List,

I just started to migrate a svn repo to a git repo and was successfully with the "git svn" command and waiting a long time :)

I have now some confidential documents in the repository I must remove completely (including the complete history). These are single files spread over the complete repository. Is this possible with git?

The next step I would like to do is separate the repo and smaller ones. The problem here is it is not only take this directory and move it out, it is a complete mess and the files are spread over the complete repo and over all the revisions (files were moved around in the old subversion repo from folder to folder). Is there a possibility to say take file1, file2, fileN and directory1, directoryN and move it to another repo or remove all not matching files/directories completely?

Thanks a lot for some hints which commands I could use here or if it is possible.

Bye
Matthias

#1 lore raw
Quoted 8 lines

Dear Git List,

I just started to migrate a svn repo to a git repo and was successfully with the "git svn" command and waiting a long time :)

I have now some confidential documents in the repository I must remove completely (including the complete history). These are single files spread over the complete repository. Is this possible with git?

take a look at "git filter-branch"

#2 lore raw

Am 19.11.2011 19:50, schrieb Alexey Shumkin:

Quoted 1 line

take a look at "git filter-branch"

thanks a lot for your answer. I checked the manual page, but it does not work like I expected it or it is described. What I did:

1. I searched the path name with the command:
git show --pretty="format:" --name-only firstrev..endrev | sort | uniq |grep sdcc

2. I tried to remove the file with the command:
git filter-branch -f --tree-filter 'rm -f software/tools/net.sourceforge.eclipsesdcc-1.0.0-win32.x86.zip' --prune-empty -- --all

git told me:
Ref 'refs/heads/master' was rewritten
Ref 'refs/remotes/origin/master' was rewritten
WARNING: Ref 'refs/remotes/origin/master' is unchanged

3. I cloned the repository with:
git clone --no-hardlinks repo-orig.git repo-filtered.git

4. After this I verified in the repo-filtered.git that the file was really completely removed, but it was not the case. I was able to checkout the revision which holds the file.

Regarding the manual page it should work... but reality is different. Has here anyone an idea what I did wrong?

Bye
Matthias

Signature · 4 lines
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook
#3 lore raw

(re-populating cc list)
Hi Matthias,

Quoted 10 lines · Matthias Fechner

git told me:
Ref 'refs/heads/master' was rewritten
Ref 'refs/remotes/origin/master' was rewritten
WARNING: Ref 'refs/remotes/origin/master' is unchanged

3. I cloned the repository with:
git clone --no-hardlinks repo-orig.git repo-filtered.git

4. After this I verified in the repo-filtered.git that the file was really completely removed, but it was not the case.

The section "CHECKLIST FOR SHRINKING A REPOSITORY" from the git-filter-branch(1) manual page has some hints. In particular, "git clone --no-hardlinks" still _copies_ all objects --- you probably would want "git clone file://$(pwd)/repo-orig" to make sure the ordinary transfer negotiation kicks in.

It's very important that the documentation not be misleading, so if you can point to places where the wording can be less confusing, that would be very welcome.

Thanks and hope that helps,
Jonathan

#4 lore raw

Am 21.11.2011 08:02, schrieb Jonathan Nieder:

Quoted 9 lines

The section "CHECKLIST FOR SHRINKING A REPOSITORY" from the git-filter-branch(1) manual page has some hints. In particular, "git clone --no-hardlinks" still _copies_ all objects --- you probably would want "git clone file://$(pwd)/repo-orig" to make sure the ordinary transfer negotiation kicks in.

It's very important that the documentation not be misleading, so if you can point to places where the wording can be less confusing, that would be very welcome.

it would be fantastic if exactly this information is noted in the manual. For me it is not really clear what is the difference between a git clone with and without --no-hardlinks.

I was now able to convert my svn repo to a git repo and split the new git repo in several different ones, including the complete history.

Here a short summary for all how need to do the same. At first convert the svn into a git reposity:
git svn clone file:///path/to/svn -A authors -s gitrepo.git

To create the authors file, check the man page.

To get a list of all files:
git show --pretty="format:" --name-only startrev..endrev | sort | uniq

Maybe you want to remove some branches from the repo because svn users used it in a completely wrong way:
git branch -rd badbranch

Now clone the repo and remove files from it you do not want anymore:
git clone gitrepo.git tofilter.git
cd tofilter.git
git filter-branch -f --tree-filter 'rm -Rf file1 file2.bla dir1 \
projekt1/dir2' --prune-empty -- --all
git gc
cd ..

Repeat this step for all combination you need.

Now create a bare repository and enable some options to use it as a central repo:
git clone --bare tofilter.git tofilter_bare.git
cd tofilter_bare.git
git config core.sharedRepository 1
git config receive.denyNonFastForwards true
git gc
cd ..

Now you can copy the repo to the target server/directory and clone it from there.

Bye
Matthias

Signature · 4 lines
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook
#5 lore raw

Am 27.11.11 11:56, schrieb Matthias Fechner:

Quoted 1 line

git clone gitrepo.git tofilter.git

one additional note.
To get it really working you should execute:
git clone file://gitrepo.git tofilter.git

And later if you copy it to a bare repo:
git clone --bare file://tofiler.git barerepo.git

This will ensure that removed files are not in the cloned repository.

Bye,
Matthias

Signature · 1 line
"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook