git-remote-svn: Week 9
Hi,
I figured that I haven't posted a recent update about the RA svnclient, so here it is. The good news is that using a combination of trial-and-error and twisted logic, I seem to have to fix the memory leaks and pool lifetime issues. I also managed to ouput the text-content-length header by writing the dump to a temporary file and stat'ing it. Unfortunately, the headers are still malformed, and this requires some more work to get right. The client runs successfully and manages to dump 5000 revisions of the ASF repository:
11.34s user 3.50s system 21% cpu 1:07.69 total
That's about 75 revisions per second, which isn't too bad I hope. This is over the network, and my internet connection isn't all that fast.
The bad news is that the client seems to signal SIGABRT when it finishes its work for some strange reason. Fortunately, I suspect that this is a trivial issue where I'm leaking some error or forgetting to clean up something. Hopefully, I'll be able to resolve this and fix the dump headers within the next few days. After that, I'll scrub the history clean and roll a series.
-- Ram
Hi,
Quoted 8 lines · Ramkumar Ramachandra
[snip]
The client runs successfully and
manages to dump 5000 revisions of the ASF repository:11.34s user 3.50s system 21% cpu 1:07.69 total
That's about 75 revisions per second, which isn't too bad I hope. This is over the network, and my internet connection isn't all that fast.
I wonder if you could mirror the test SVN repository locally with svnsync so that we can a) spare their bandwidth b) minimize the effect of network bottlenecks on stats - I'm sure you could get more impressive stats that way.
Even better, having the SVN repo and the "exported" git repo on separate hard disks to minimize I/O as a bottleneck.
(Great work, by the way.)
Signature · 2 lines
Cheers, Ray Chuan
Hi Tay,
Quoted 9 lines · Tay Ray Chuan
Hi,
Quoted 6 lines · Ramkumar Ramachandra
[snip]
The client runs successfully and
manages to dump 5000 revisions of the ASF repository:11.34s user 3.50s system 21% cpu 1:07.69 total
Quoted 7 lines
Quoted 2 lines
That's about 75 revisions per second, which isn't too bad I hope. This is over the network, and my internet connection isn't all that fast.
I wonder if you could mirror the test SVN repository locally with svnsync so that we can a) spare their bandwidth b) minimize the effect of network bottlenecks on stats - I'm sure you could get more impressive stats that way.
Excellent idea. Unfortunately, I don't have enough disk space to mirror the whole ASF repository on my laptop- I'll try it when I get back to my desktop (in 15~20 days).
In my prelimiary benchmark, I also found that I beat svnsync over the same network connection :)
Quoted 2 lines
Even better, having the SVN repo and the "exported" git repo on separate hard disks to minimize I/O as a bottleneck.
Unfortunately, a lot of middleware components are missing and we cannot export it to a Git repository just yet- see my previous email about this [1]. Great idea though- will try it in future.
Quoted 1 line
(Great work, by the way.)
Thanks, and thanks for the feedback! :)
[1]: http://article.gmane.org/gmane.comp.version-control.git/150035
-- Ram