Skip to contentSkip to conversation

PATCH RESEND: git-svnimport memory leak plug

PATCH Auke Kok 1 participant 1 message Jun 15, 2016

lore
Share & tools
lore

Auke Kok

author
#0 lore raw

I was unable with todays git-svnimport to convert a modest 140mb svn repository to git using svn-git-import. the process would bomb out after 2000 revisions and I have 20000. It consumed over 1.5gb vm space on my 1gb machine, and died with a 'cannot fork error'. this also killed my project server today after I discivered it the wrong way.

The original patch send earlier by Santi Bejar fixes the problem. I include the patch again here so it can be merged. my mailer probably will nuke it so here's the link to the archive post:

http://marc.theaimsgroup.com/?l=git&m=114345884526971&w=2

Please include this patch.

Cheers,

Auke

Quoted 56 lines · Santi Béjar
Quoted 16 lines · Jan-Benedict Glaw

On Wed, 2006-03-22 14:33:37 +0100, Jan-Benedict Glaw <[email protected]> wrote:

Since it seems nobody looked at the GCC import run (which means to use the svnimport), I ran it again, under strace control:

Quoted 10 lines

GCC
~~~

$ /home/jbglaw/bin/git svnimport -C gcc -v svn://gcc.gnu.org/svn/gcc

Committed change 3936:/ 1993-03-31 05:44:03) Commit ID ceff85145f8671fb2a9d826a761cedc2a507bd1e Writing to refs/heads/origin
DONE: 3936 origin ceff85145f8671fb2a9d826a761cedc2a507bd1e ... 3937 trunk/gcc/final.c ...
Can't fork at /home/jbglaw/bin/git-svnimport line 379.

I have the same (?) problem with one of my svn repository. It worked before (I've redone the import with the -r flag), so I bisected it. The problematic commit seems to be:

diff-tree 4802426... (from 525c0d7...)

Author: Karl  Hasselström <[email protected]>
Date:   Sun Feb 26 06:11:27 2006 +0100

svnimport: Convert executable flag

Convert the svn:executable property to file mode 755 when converting an SVN repository to GIT.

:100755 100755 ee2940f... 6603b96... M git-svnimport.perl

I think it has a memory leak, it used up to 140m of memory.

$ git reset --hard 4802426^
$ time ../git-svnimport.perl file:///path/
Use of uninitialized value in string eq at ../git-svnimport.perl line 463.
Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466.
real    0m55.801s
user    0m30.578s
sys     0m23.084s
$ git reset --hard 4802426
$ time ../git-svnimport.perl file:///path/
Use of uninitialized value in string eq at ../git-svnimport.perl line 463.
Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466.
Can't fork at /home/santi/usr/src/scm/git/git-svnimport.perl line 331.
real    6m2.163s
user    0m20.332s
sys     0m50.180s
Quoted 1 line

and it didn't finished. Hope it helps.

And this patch fixes my problems.

---

Introduced in 4802426.

---
 git-svnimport.perl |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
git-svnimport.perl +3 -1
index 639aa41..f2cf062 100755
@@ -135,8 +135,10 @@
 
        print "... $rev $path ...\n" if $opt_v;
        my (undef, $properties);
+       my $pool = SVN::Pool->new();
        eval { (undef, $properties)
-                  = $self->{'svn'}->get_file($path,$rev,$fh); };
+                  = $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
+       $pool->clear;
        if($@) {
                return undef if $@ =~ /Attempted to get checksum/;
                die $@;