Skip to contentSkip to conversation

How to clone faster via ssh ?

DISCUSSION Alexander Litvinov 3 participants 6 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

Hello,

I have cvs project imported into git. When unpacked it contains about 205000 objects. I have made one pack and daily rerun cvsimport to get new changes.

When I do cloning via http it takes less than 3 minutes:
time git-clone -n http://127.0.0.1:8080/git/r/.git r2

real    2m31.062s
user    1m33.562s
sys     0m5.462s

When I do cloning via ssh it takes more than 10 minutes and horde of unpacked objects:
Packing 204607 objects
Unpacking 204607 objects
100% (204607/204607) done

real    13m41.510s
user    1m52.655s
sys     1m5.057s

There are no packs in cloned repo:
ls -la r3/.git/objects/pack/
total 6
drwxr-xr-x 2 lan users 48 Oct 25 13:51 .
drwxr-xr-x 260 lan users 6240 Oct 25 14:05 ..
lan@lan:~/tmp/git/billing

We can't share our project via http - we need authorization for it. Is is possible to clone this repo via ssh as http clone does ? Its will be much faster and will simplify work with such directory - it is hard to delete it after cloning - takes from 5 up to 10 minutes ! Filesystem is reiserfs.

Thanks for help,
Alexander Litvinov.

#1 lore raw

Forgot to tell:
clone via ssh was made using this command:
time git-clone -n ssh://lan@lan/home/lan/tmp/git/billing/repo r3

Quoted 9 lines

When I do cloning via ssh it takes more than 10 minutes and horde of unpacked objects:
Packing 204607 objects
Unpacking 204607 objects
100% (204607/204607) done

real    13m41.510s
user    1m52.655s
sys     1m5.057s
#2 lore raw

Hi,

Quoted 3 lines · Alexander Litvinov

Forgot to tell:
clone via ssh was made using this command:
time git-clone -n ssh://lan@lan/home/lan/tmp/git/billing/repo r3

If you have a working git on the other side, you could do

time git-clone lan@lan:/home/lan/tmp/git/billing/repo r3

which would utilize git-clone-pack. Way faster.

Ciao,
Dscho

#3 lore raw
Quoted 5 lines

If you have a working git on the other side, you could do

time git-clone lan@lan:/home/lan/tmp/git/billing/repo r3

which would utilize git-clone-pack. Way faster.

I have tried this command:
lan@lan> time git-clone lan@lan:/home/lan/tmp/git/billing/repo r3 defaulting to local storage area
lan@lan's password:
Packing 204607 objects
Unpacking 204607 objects
Killed by signal 2.) done

real    4m0.874s
user    0m0.013s
sys     0m0.120s

I have killed it at ~ 20%. As I understand it does exactly what ssh git thru ssh transport does: git-rev-list | git-pack | ssh | git-unpack or something like this. So it is not faster.

#4 lore raw
In reply to Johannes Schindelin
Quoted 11 lines · Johannes Schindelin
Quoted 3 lines · Alexander Litvinov

Forgot to tell:
clone via ssh was made using this command:
time git-clone -n ssh://lan@lan/home/lan/tmp/git/billing/repo r3

If you have a working git on the other side, you could do

time git-clone lan@lan:/home/lan/tmp/git/billing/repo r3

which would utilize git-clone-pack. Way faster.

I think both of these notations do the same.

The time to unpack the resulting pack on this end is eliminated if you use git from last week, namely this commit:

commit e1c7ada6dd1fdf249d0bb84f3293d3be768b9239

Author: Junio C Hamano <[email protected]>
Date:   Wed Oct 19 14:43:43 2005 -0700

git-clone: always keep pack sent from remote.

This deprecates --keep and -q flags and always keeps the pack
sent from the remote site. Corresponding configuration variables are also removed.

But you would still pay for creating a pack on the remote side.

#5 lore raw
Quoted 9 lines

The time to unpack the resulting pack on this end is eliminated if you use git from last week, namely this commit:

commit e1c7ada6dd1fdf249d0bb84f3293d3be768b9239

Author: Junio C Hamano <[email protected]>
Date:   Wed Oct 19 14:43:43 2005 -0700

git-clone: always keep pack sent from remote.

Quoted 8 lines

This deprecates --keep and -q flags and always keeps the pack
sent from the remote site. Corresponding configuration variables are also removed.

But you would still pay for creating a pack on the remote side.

Yeah ! Thanks.

Version for todays master branch works much faster:
lan@lan:~/tmp/git/billing> time git-clone -n ssh://lan@lan/home/lan/tmp/git/billing/repo r1 defaulting to local storage area
lan@lan's password:
Packing 204607 objects

real    4m0.107s
user    0m11.879s
sys     0m1.717s

And it makes only one pack. Great.