Moving "git remote add --mirror blah" functionality to "git clone --bare --origin=blah"

In the thread "git remote update -> rejected" Junio and Johannes came to the conclusion that "--mirror means that you do not have local branches", because "you give control away to the other end on the ref namespace". Furthermore, it was agreed that --mirror currently makes sense mostly (or only?) on a bare repository.

From here I gather that if you have "git remote add --mirror", most likely the mirrored repository will be the only remote you have. There is in general no point in having other remotes in a bare repository. And so there is no loss of generality if this remote is the "origin" remote.

Hence, my proposal is:

1) Add an option to "git clone", to be used with --bare, to create a mirror. --bare already leaves the original refs in place, without moving them under refs/remotes/origin, so it makes sense to optionally create the remote.

Actually you don't need to add a new option. Right now, specifying -o/--origin together with --bare would give an error; in my proposal, if you had "--bare --origin=source" a remote would be created in the config file like this:

[remote "source"]

	url = ...
	fetch = +refs/*:refs/*

2) Modify the --mirror option to "git remote add", so that it creates a remote *acting as a mirror of this one*. The config would look like this:

[remote "mirror"]

        url = ...
        fetch = +refs/heads/*:refs/remotes/mirror/*
        push = +refs/heads/*:refs/heads/*
        push = +refs/tags/*:refs/tags/*
        mirror

Now, I know that this is not what was proposed in the original --mirror thread, and it is backwards-incompatible. On the other hand, I think it has several advantages:

1) it does not add special cases such as "--mirror is only to be used in a --bare repository". It moves the functionality to a combination of options that used to give an error.

2) it provides an easy-to-use UI to create a mirror of this repository somewhere. With a patch I already posted, "git push" would automatically update this mirror.

3) overall, I think it makes a lot of sense: "git remote add" is about adding one remote of many, and "git clone" is about setting up a single special remote. Now, you can have many remotes that are mirrored-from-here, while you can only have one remote that is mirrored-to-here. So, I believe that "git remote add" should add mirrored-from-here remotes, while "git clone" should be used to create a mirrored-to-here remote.

Opinions?

Paolo

I actually had a slightly different vision. A mid-term goal should be to reimplement "clone" that has a lot of code duplication with "fetch" and "remote" in terms of "init + remote + fetch + checkout" [*1*]. For that to happen, I suspect that "remote" needs to learn a few more tricks than it currently knows (e.g. "figuring out the HEAD").

I would agree that it is useful for "clone" to create a bare repository in a mode that can be used for further cloning by other repositories (perhaps the former sits at the firewall boundary that it is cumbersome to cross by the latter). As you described above, we already have that (iow, "--bare").

And "remote add --mirror" would be an implementation detail to produce such a clone. It is mostly about fetching.

An option to add a back-up repository that you can maintain an exact mirror of your working repository would be useful, but that is different from what "remote add --mirror" is about.

[Footnote]

*1* In that sense, a more sensible order than rewriting "clone" in C in its current form would be to make necessary enhancements to the components in this sequence that need to implement clone, figure out how they should fit together and first make "clone" a four-liner shell script. Then rewriting the result in C may become more trivial.

On the other hand, a C version may have the necessary APIs available to support this kind of enhancement, but the APIs may not be there for a shell script.

It seems useless to have an option in "git remote add" just because it might be necessary in a future refactoring of "git clone", but without a good use case beside that one -- because right now "git remote add --mirror" is close to useless: anyone who needs it 99% of the time knows how to hack the config, unlike people who just want a quick way to remotely backup of their repository.

Paolo

Ooo. But I think that's the other way around. It was prepared for that purpose but people never followed through. Check the archive around commit 3894439 (Teach "git remote" a mirror mode, 2007-09-02).

Eh, in that sense, "git remote" itself is useless, isn't it? I have to say that this particular point of your argument does not give us any new or interesting insight to work from.

As I already said, I think "push to backup" option would be useful. It just cannot take over --mirror option, which means something different.

Well, the thread is not very explicative. Neither is the commit log message. There is no info as to *why* it is useful.

After finding no particularly good info in the commit, and seeing that the functionality is unused within git's shell script porcelain, I wrote my RFC on the intuition that it is a failed experiment. (The rest of this message is also assuming this possibly wrong intuition).

Hm, bad sentence on my side. My point was that my proposed improvements to clone:

  git clone --bare -o origin <url> foo.git

satisfy 99% of the usecase; you usually know beforehand if you're cloning for local development or for mirrors. Anyone who *still* needs --mirror after that (i.e. needs two mirror-mode remotes, or needs it on non-bare repositories), 99% of the time knows how to hack the config. Actually in the latter case (non-bare repos) he might have to hack the config soon, if Dscho's patch to forbid this is polished and goes in.

In the end, this piece of functionality is almost unused (surely unused within git's shell porcelain, if not in the wild) and is dubiously placed for two reasons:

1) If the point of git remote is to shield from the config, what the user will learn is that "git remote" is about "git branch -r" (maybe he'll also learn "refs/remotes/something"). Now instead you have an option that goes under "git remote" just because it creates remote.* configuration options. Indeed, "git remote add --mirror -f" will create *local* branches, so that "git branch -r" does not show them.

2) Similarly, "--mirror" suggests that the remote will go hand in hand with "git push" (that has the "--mirror" option, and this started my confusion) rather than "git fetch"/"git pull". Indeed, even after my patch[1] to add some DWIM functionality for mirrors to "git push", the user will need to hack the config file manually to activate the DWIM.

Instead, my proposed meaning of "git remote add --mirror" would fix both discrepancies...

Paolo

[1] http://marc.info/?l=git&m=120877130932254&w=2

Hi,

I think it is very much the other way round. "git remote add" is meant for adding _additional_ remotes. "git clone" just _happens_ to install an "origin" remote.

I even have a case where I used "--mirror" _twice_ in the same, _non_-bare repository. It has merit, you know?

Ciao,
Dscho

clone can't be a four-liner shell script, because it has to come up with the name of the working directory for a non-bare clone, which doesn't make sense for any other program. Currently, a local clone is special in that it duplicates all of the objects without checking whether the destination already has them (because it can't have anything) or whether they're actually needed; fetch could do some similar stuff, but couldn't do quite that much optimization, because it's not likely the same special case.

Something needs to get the code to figure out the remote HEAD, and it shouldn't be "remote" for clone's use, because then clone would take an extra connection, since "fetch" obviously has to connect and "remote" would run before it and in a separate process.

It also needs to support --reference (list another repository's refs as your refs, because you have that repository as an alternate but don't have your own refs that use its objects), which is sort of plausible for fetch but much less useful.

Aside from that, it's obviously got quite a few lines of option parsing to get the right arguments to the things it calls, as well as determining whether to check out a branch (depending on what we did with the origin) and such.

-Daniel
*This .sig left intentionally blank*

Actually, having "fetch" figure out the HEAD would be a nice feature, since then you could track the remote's idea of HEAD as time passes.

AIUI, we could get by with a single connection over git:// with a protocol extension.

-Peff

Hi,

Yes. But the opposite may not be true: if you plan to work on many workdirs, but the same repository, you might still want to have separate remotes layout.

In any case, I think that this discussion is too early: clone should be builtin first, making things much simpler because of being a very thin layer on top of init && remote && fetch.

But yes, there is a lot of work left on the builtin-clone to make it so.

Ciao,
Dscho