PATCH RESEND: git-svnimport memory leak plug
I was unable with todays git-svnimport to convert a modest 140mb svn repository to git using svn-git-import. the process would bomb out after 2000 revisions and I have 20000. It consumed over 1.5gb vm space on my 1gb machine, and died with a 'cannot fork error'. this also killed my project server today after I discivered it the wrong way.
The original patch send earlier by Santi Bejar fixes the problem. I include the patch again here so it can be merged. my mailer probably will nuke it so here's the link to the archive post:
http://marc.theaimsgroup.com/?l=git&m=114345884526971&w=2
Please include this patch.
Cheers,
Auke
Quoted 56 lines · Santi Béjar
Quoted 16 lines · Jan-Benedict Glaw
On Wed, 2006-03-22 14:33:37 +0100, Jan-Benedict Glaw <[email protected]> wrote:
Since it seems nobody looked at the GCC import run (which means to use the svnimport), I ran it again, under strace control:
Quoted 10 lines
GCC
~~~$ /home/jbglaw/bin/git svnimport -C gcc -v svn://gcc.gnu.org/svn/gccCommitted change 3936:/ 1993-03-31 05:44:03) Commit ID
ceff85145f8671fb2a9d826a761cedc2a507bd1eWriting to refs/heads/origin
DONE: 3936 originceff85145f8671fb2a9d826a761cedc2a507bd1e... 3937trunk/gcc/final.c...
Can't fork at/home/jbglaw/bin/git-svnimportline 379.I have the same (?) problem with one of my svn repository. It worked before (I've redone the import with the -r flag), so I bisected it. The problematic commit seems to be:
diff-tree 4802426... (from 525c0d7...)
Author: Karl Hasselström <[email protected]> Date: Sun Feb 26 06:11:27 2006 +0100svnimport: Convert executable flag
Convert the svn:executable property to file mode 755 when converting an SVN repository to GIT.
Signed-off-by Karl Hasselström <[email protected]>Signed-off-by Junio C Hamano <[email protected]>:100755 100755 ee2940f... 6603b96... M git-svnimport.perl
I think it has a memory leak, it used up to 140m of memory.
$ git reset --hard 4802426^ $ time ../git-svnimport.perl file:///path/ Use of uninitialized value in string eq at ../git-svnimport.perl line 463. Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466. real 0m55.801s user 0m30.578s sys 0m23.084s$ git reset --hard 4802426 $ time ../git-svnimport.perl file:///path/ Use of uninitialized value in string eq at ../git-svnimport.perl line 463. Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466. Can't fork at /home/santi/usr/src/scm/git/git-svnimport.perl line 331. real 6m2.163s user 0m20.332s sys 0m50.180s
Quoted 1 line
and it didn't finished. Hope it helps.
And this patch fixes my problems.
---
Introduced in 4802426.
--- git-svnimport.perl | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
git-svnimport.perl +3 -1
@@ -135,8 +135,10 @@ print "... $rev $path ...\n" if $opt_v; my (undef, $properties); + my $pool = SVN::Pool->new(); eval { (undef, $properties) - = $self->{'svn'}->get_file($path,$rev,$fh); }; + = $self->{'svn'}->get_file($path,$rev,$fh,$pool); }; + $pool->clear; if($@) { return undef if $@ =~ /Attempted to get checksum/; die $@;