Use the alternates of the source repository for dissociating clone

The "--dissociate" option required reference repositories, which sometimes demanded a look into the objects/info/alternates by the user. As this is something which can be figured out automatically, do it in the clone unless there is no other reference repositories.

---

I often (enough) did something like this (preparing for manual backup):

# Run mc (the Midnight Commander, it is a console (ncurses) file manager with # two panels).
# Select a repository.
$ git clone --mirror --dissociate \
$(sed -e 's|\(.*\)/objects|--reference \1| --
%f/.git/objects/info/alternates' 2>/dev/null) \
%f %D/%f
# (except that this is one long line in mc)

The '%f' expands to a selected directory on the current panel, %D - to the current directory on the other panel. But as I see, the combination of the options "--dissociate" and no given references is not used for anything but giving the warning yet. So maybe it can be used as a shortcut for the above?

 builtin/clone.c | 42 ++++++++++++++++++++++++++++++++++++++----
 1 file changed, 38 insertions(+), 4 deletions(-)

index 578da85..344bf21 100644
@@ -791,6 +791,38 @@ static void write_refspec_config(const char *src_ref_prefix,
     strbuf_release(&value);
 }
 
+static int copy_alternates_to_references(const char *src_repo)
+{
+    int refcnt = -1;
+    char *src_alternates = mkpathdup("%s/objects/info/alternates", src_repo);
+    FILE *in = fopen(src_alternates, "r");
+    if (in) {
+        struct strbuf line = STRBUF_INIT;
+        refcnt = 0;
+        while (strbuf_getline(&line, in, '\n') != EOF) {
+            struct string_list_item item;
+            if (line.len < 8 || line.buf[0] == '#')
+                continue;
+            ++refcnt;
+            if (!strcmp(line.buf + line.len - 8, "/objects"))
+                line.buf[line.len - 8] = '\0';
+            if (is_absolute_path(line.buf)) {
+                item.string = line.buf;
+                add_one_reference(&item, NULL);
+                continue;
+            }
+            item.string = mkpathdup("%s/objects/%s", src_repo, line.buf);
+            normalize_path_copy(item.string, item.string);
+            add_one_reference(&item, NULL);
+            free(item.string);
+        }
+        strbuf_release(&line);
+        fclose(in);
+    }
+    free(src_alternates);
+    return refcnt;
+}
+
 static void dissociate_from_references(void)
 {
     static const char* argv[] = { "repack", "-a", "-d", NULL };

@@ -947,10 +979,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
     if (option_reference.nr)
         setup_reference();
-    else if (option_dissociate) {
-        warning(_("--dissociate given, but there is no --reference"));
-        option_dissociate = 0;
-    }
 
     fetch_pattern = value.buf;
     refspec = parse_fetch_refspec(1, &fetch_pattern);

@@ -976,6 +1004,12 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
         warning(_("--local is ignored"));
     transport->cloning = 1;
 
+    if (!option_reference.nr && option_dissociate &&
+        copy_alternates_to_references(path) <= 0) {
+        warning(_("--dissociate given, but there is no --reference"));
+        option_dissociate = 0;
+    }
+
     if (!transport->get_refs_list || (!is_local && !transport->fetch))
         die(_("Don't know how to clone %s"), transport->url);
 

2.6.1.150.gb633014

Hi Alex,

Would it not make sense to reuse the copy_alternates() function to simply copy the alternates and let --dissociate run its course with the copied .objects/info/alternate file? That would make for less new code...

Ciao,
Dscho

The option requiring the explicit reference repositories is a bit of overkill:
the alternates in the original repository *are* reference repositories and would be dissociated from should one pass any reference repository (even an unrelated one).

---

IIUC, I should validate the alternates in the source repository... But, the only thing the user looses if it is not validated, is the nice warning regarding no reference repositories to dissociate from, right?

So maybe we can just remove the reset of option_dissociate and be done with it? I would actually suggest removing the warning as well: the alternates are something to dissociate from. And I see no harm otherwise.

How about this instead?

 builtin/clone.c | 4 ----
 1 file changed, 4 deletions(-)

index 578da85..b33d6f9 100644
@@ -947,10 +947,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
     if (option_reference.nr)
         setup_reference();
-    else if (option_dissociate) {
-        warning(_("--dissociate given, but there is no --reference"));
-        option_dissociate = 0;
-    }
 
     fetch_pattern = value.buf;
     refspec = parse_fetch_refspec(1, &fetch_pattern);

2.6.1.151.g74e8091

Reminder. Is this (or rather the one I'm replying to) patch a better option?

Regards,
Alex

I suspect that the reason why you didn't get any quick response was because it was unclear from either one of these proposed log messages why any change is needed in the first place. At least that is what prevented me from commenting on either.

The "clone --dissociate" was designed to be used with "--reference". The mindset of those who saw the need for the feature being that "clone --reference" is the only way to make the resulting repository's objects incomplete, needing to borrow objects from some other place, which necessitates the "--dissociate" option.

The readers of this change need to be enlightened with a log message to remind them that "--reference" is not the only way. Namely, if you start from a repository with $GIT_DIR/objects/info/alternates, i.e. the original already borrows from somewhere, and bypass the normal "Git aware" transport mechanism, i.e. "git clone --local", then the resulting repository would also become dependent of the object store that the original depended on before the clone. In order to make it free-standing, you would need "--dissociate", but there is no "--reference" involved in that use case.

And once that is clarified, it becomes very clear why it is wrong to blindly require "--reference" to be there on the command line when "--dissociate" is given.

As to the patch, I think this one is much simpler and preferrable. It would hurt those who make a clone without bypassing the normal "Git aware" transport mechanism and pass "--dissociate" without "--reference". They will end up making a clone that does not need repacking to dissociate, but with this patch they would spend extra cycles to run an unnecessary repack. To avoid that, I think you can throw in an check at the beginning of dissociate_from_references() to see if git_path("objects/info/alternates") is there and make the function a no-op if there isn't.

Thanks.

From: Alex Riesen <[email protected]>

The "--reference" option is not the only way to provide a repository to borrow objects from.
For instance, the objects/info/alternates of the origin repository lists object stores which the origin repository borrowed objects from. During clone operations which bypass a git aware transport (i.e. simply copy the things over, like git clone --local) the file is copied into the cloned repository.
In such a case, even if there were no reference repositories given in the command-line, there might be still something to "dissociate" the cloned repository from, before it is really independent.

---

Indeed. Log message improved.

I think I understand. How about this?

 builtin/clone.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

index 9eaecd9..a7d0c07 100644
@@ -801,11 +801,15 @@ static void write_refspec_config(const char *src_ref_prefix,
 static void dissociate_from_references(void)
 {
     static const char* argv[] = { "repack", "-a", "-d", NULL };
+    char *alts = git_pathdup("objects/info/alternates");
 
+    if (access(alts, F_OK) < 0)
+        return;
     if (run_command_v_opt(argv, RUN_GIT_CMD|RUN_COMMAND_NO_STDIN))
         die(_("cannot repack to clean up"));
-    if (unlink(git_path("objects/info/alternates")) && errno != ENOENT)
+    if (unlink(alts) && errno != ENOENT)
         die_errno(_("cannot unlink temporary alternates file"));
+    free(alts);
 }
 
 int cmd_clone(int argc, const char **argv, const char *prefix)

@@ -954,10 +958,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
     if (option_reference.nr)
         setup_reference();
-    else if (option_dissociate) {
-        warning(_("--dissociate given, but there is no --reference"));
-        option_dissociate = 0;
-    }
 
     fetch_pattern = value.buf;
     refspec = parse_fetch_refspec(1, &fetch_pattern);

2.6.1.151.ge74ab91

From: Alex Riesen <[email protected]>

The "--reference" option is not the only way to provide a repository to borrow objects from.
For instance, the objects/info/alternates of the origin repository lists object stores which the origin repository borrowed objects from. During clone operations which bypass a git aware transport (i.e. simply copy the things over, like git clone --local) the file is copied into the cloned repository.
In such a case, even if there were no reference repositories given in the command-line, there might be still something to "dissociate" the cloned repository from, before it is really independent.

---

Fixed.

Added. t5700-clone-reference seemed like a logical place for it.

Regards,
Alex

---
 builtin/clone.c            | 11 ++++++-----
 t/t5700-clone-reference.sh | 11 +++++++++++
 2 files changed, 17 insertions(+), 5 deletions(-)

index 9eaecd9..1e14810 100644
@@ -801,11 +801,16 @@ static void write_refspec_config(const char *src_ref_prefix,
 static void dissociate_from_references(void)
 {
     static const char* argv[] = { "repack", "-a", "-d", NULL };
+    char *alts = git_pathdup("objects/info/alternates");
 
+    if (access(alts, F_OK) < 0)
+        goto done;
     if (run_command_v_opt(argv, RUN_GIT_CMD|RUN_COMMAND_NO_STDIN))
         die(_("cannot repack to clean up"));
-    if (unlink(git_path("objects/info/alternates")) && errno != ENOENT)
+    if (unlink(alts) && errno != ENOENT)
         die_errno(_("cannot unlink temporary alternates file"));
+done:
+    free(alts);
 }
 
 int cmd_clone(int argc, const char **argv, const char *prefix)

@@ -954,10 +959,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
     if (option_reference.nr)
         setup_reference();
-    else if (option_dissociate) {
-        warning(_("--dissociate given, but there is no --reference"));
-        option_dissociate = 0;
-    }
 
     fetch_pattern = value.buf;
     refspec = parse_fetch_refspec(1, &fetch_pattern);

index 2250ef4..dfa1bf7 100755
@@ -210,4 +210,15 @@ test_expect_success 'clone, dissociate from partial reference and repack' '
     test_line_count = 1 packs.txt
 '
 
+test_expect_success 'clone, dissociate from alternates' '
+    rm -fr A B C &&
+    test_create_repo A &&
+    commit_in A file1 &&
+    git clone --reference=A A B &&
+    test_line_count = 1 B/.git/objects/info/alternates &&
+    git clone --local --dissociate B C &&
+    ! test -f C/.git/objects/info/alternates &&
+    ( cd C && git fsck )
+'
+
 test_done

2.6.2.313.gdf7a1dc

I had to hand-munge it as the above lost all tabs and made the patch unusable for machines X-<.

Re-reading the documentation, I realized that the use case this new mode of operation allows is totally outside of the original design space that was described, so I added a note to teach users how the option can be used in a new way as well.

So here is what I tentatively queued.

Thanks.

-- >8 --
From: Alex Riesen <[email protected]>
Date: Thu, 22 Oct 2015 18:41:17 +0200
Subject: [PATCH] clone: allow "--dissociate" without reference

The "--reference" option is not the only way to provide a repository to borrow objects from. A repository that borrows from another repository can be cloned with "clone --local" and the resulting repository will borrow from the same repository, which the user may want to "--dissociate" from.

---
 Documentation/git-clone.txt |  9 +++++++--
 builtin/clone.c             | 16 ++++++++--------
 t/t5700-clone-reference.sh  | 11 +++++++++++
 3 files changed, 26 insertions(+), 10 deletions(-)

index f1f2a3f..a8c11e3 100644
@@ -104,8 +104,13 @@ objects from the source repository into a pack in the cloned repository.
 --dissociate::
 	Borrow the objects from reference repositories specified
 	with the `--reference` options only to reduce network
-	transfer and stop borrowing from them after a clone is made
-	by making necessary local copies of borrowed objects.
+	transfer, and stop borrowing from them after a clone is made
+	by making necessary local copies of borrowed objects.  This
+	option can also be used when cloning locally from a
+	repository that already borrows objects from another
+	repository---the new repository will borrow objects from the
+	same repository, and this option can be used to stop the
+	borrowing.
 
 --quiet::
 -q::

index 9eaecd9..caae43e 100644
@@ -801,11 +801,15 @@ static void write_refspec_config(const char *src_ref_prefix,
 static void dissociate_from_references(void)
 {
 	static const char* argv[] = { "repack", "-a", "-d", NULL };
+	char *alternates = git_pathdup("objects/info/alternates");
 
-	if (run_command_v_opt(argv, RUN_GIT_CMD|RUN_COMMAND_NO_STDIN))
-		die(_("cannot repack to clean up"));
-	if (unlink(git_path("objects/info/alternates")) && errno != ENOENT)
-		die_errno(_("cannot unlink temporary alternates file"));
+	if (!access(alternates, F_OK)) {
+		if (run_command_v_opt(argv, RUN_GIT_CMD|RUN_COMMAND_NO_STDIN))
+			die(_("cannot repack to clean up"));
+		if (unlink(alternates) && errno != ENOENT)
+			die_errno(_("cannot unlink temporary alternates file"));
+	}
+	free(alternates);
 }
 
 int cmd_clone(int argc, const char **argv, const char *prefix)

@@ -954,10 +958,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
 	if (option_reference.nr)
 		setup_reference();
-	else if (option_dissociate) {
-		warning(_("--dissociate given, but there is no --reference"));
-		option_dissociate = 0;
-	}
 
 	fetch_pattern = value.buf;
 	refspec = parse_fetch_refspec(1, &fetch_pattern);

index 2250ef4..dfa1bf7 100755
@@ -210,4 +210,15 @@ test_expect_success 'clone, dissociate from partial reference and repack' '
 	test_line_count = 1 packs.txt
 '
 
+test_expect_success 'clone, dissociate from alternates' '
+	rm -fr A B C &&
+	test_create_repo A &&
+	commit_in A file1 &&
+	git clone --reference=A A B &&
+	test_line_count = 1 B/.git/objects/info/alternates &&
+	git clone --local --dissociate B C &&
+	! test -f C/.git/objects/info/alternates &&
+	( cd C && git fsck )
+'
+
 test_done

2.6.2-383-g4ea3cbc

I'm very sorry. I don't know why Icedove does that, nor do I know how to stop it mangling the text. Right now I just hate the damn thing.

Thank you very much for reformatting the patch, as it would take quite some time until I configure a sane mail program to work here.

Incidentally, what does "---" mean in the documentation hunk?

Regards,
Alex

Tip: Just use git-send-email instead. There's even an example configuration for gmail in the manpage.

Sincerely,
 Johannes Löthberg
 PGP Key ID: 0x50FB9B273A9D0BB5
 https://theos.kyriasis.com/~kyrias/

Heh, good eyes. I was hoping that it would turn into an em-dash, but it seems I just get three dashes instead.

Outside relnotes, there seem to be a handful instances of those that expresses em-dash with "---".

$ git grep -E '[A-Za-z]{2,}---[A-Za-z]{2,}' Documentation/
git-bisect.txt:command not found, 126 is for command found but not executable---these
git-clone.txt:	repository---the new repository will borrow objects from the
git-fetch.txt:  the refspecs---they specify which refs to fetch and which local refs
git-push.txt:be omitted---such a push will update a ref that `<src>` normally updates
technical/index-format.txt:  first subtree---let's call this A---of the root level...

I'll leave it for a separate topic to clean these up later.

Again, we do not usually process release notes with AsciiDoc, but it is better to be consistent.

This incidentally reveals breakages left by an ancient 5e00439f (Documentation: build html for all files in technical and howto, 2012-10-23). The index-format documentation was originally written to be read as straight text without formatting and when the commit forced everything in Documentation/ to go through AsciiDoc, it did not do any adjustment--hence the double-dashes will be seen in the resulting text that is rendered as preformatted fixed-width without converted into em-dashes.

---

* An follow-up to recent AsciiDoc markup fixes.

 Documentation/RelNotes/1.7.7.txt         | 2 +-
 Documentation/RelNotes/1.9.0.txt         | 2 +-
 Documentation/git-bisect.txt             | 2 +-
 Documentation/git-fetch.txt              | 2 +-
 Documentation/git-push.txt               | 2 +-
 Documentation/technical/index-format.txt | 2 +-
 6 files changed, 6 insertions(+), 6 deletions(-)

index 7655ccc..6eff128 100644
@@ -84,7 +84,7 @@ Updates since v1.7.6
    logic used by "git diff" to determine the hunk header.
 
  * Invoking the low-level "git http-fetch" without "-a" option (which
-   git itself never did---normal users should not have to worry about
+   git itself never did--normal users should not have to worry about
    this) is now deprecated.
 
  * The "--decorate" option to "git log" and its family learned to

index 752d791..4e4b88a 100644
@@ -177,7 +177,7 @@ Performance, Internal Implementation, etc.
  * The naming convention of the packfiles has been updated; it used to
    be based on the enumeration of names of the objects that are
    contained in the pack, but now it also depends on how the packed
-   result is represented---packing the same set of objects using
+   result is represented--packing the same set of objects using
    different settings (or delta order) would produce a pack with
    different name.
 

index 4cb52a7..617efa0 100644
@@ -245,7 +245,7 @@ cannot be tested. If the script exits with this code, the current
 revision will be skipped (see `git bisect skip` above). 125 was chosen
 as the highest sensible value to use for this purpose, because 126 and 127
 are used by POSIX shells to signal specific error status (127 is for
-command not found, 126 is for command found but not executable---these
+command not found, 126 is for command found but not executable--these
 details do not matter, as they are normal errors in the script, as far as
 "bisect run" is concerned).
 

index 8deb614..ee51c1a 100644
@@ -71,7 +71,7 @@ This configuration is used in two ways:
 * When `git fetch` is run without specifying what branches
   and/or tags to fetch on the command line, e.g. `git fetch origin`
   or `git fetch`, `remote.<repository>.fetch` values are used as
-  the refspecs---they specify which refs to fetch and which local refs
+  the refspecs--they specify which refs to fetch and which local refs
   to update.  The example above will fetch
   all branches that exist in the `origin` (i.e. any ref that matches
   the left-hand side of the value, `refs/heads/*`) and update the

index b17283a..3267e21 100644
@@ -61,7 +61,7 @@ be named.
 If `git push [<repository>]` without any `<refspec>` argument is set to
 update some ref at the destination with `<src>` with
 `remote.<repository>.push` configuration variable, `:<dst>` part can
-be omitted---such a push will update a ref that `<src>` normally updates
+be omitted--such a push will update a ref that `<src>` normally updates
 without any `<refspec>` on the command line.  Otherwise, missing
 `:<dst>` means to update the same ref as the `<src>`.
 +

index 1250b5c..61cb55d 100644
@@ -170,7 +170,7 @@ Git index format
 
   The entries are written out in the top-down, depth-first order.  The
   first entry represents the root level of the repository, followed by the
-  first subtree---let's call this A---of the root level (with its name
+  first subtree--let's call this A--of the root level (with its name
   relative to the root level), followed by the first subtree of A (with
   its name relative to A), ...
 

2.6.2-390-g2f019b7

I do not quite get this.

Before "clone" with or without "--dissociate" there is no objects/info/alternates (before "clone", there is no ".git" to find that file in the first place).

Are you talking about making a clone of a repository that was created with "clone --reference", to borrow from the same third repository the original is borrowing from?

Resend. The previous replies didn't make it to git-ml because of html part inserted by gmail.

Yes. Or the alternates file was created manually in the source repository.