clone: support 'clone --shared' from a worktree
PATCH 1/1 clone: support 'clone --shared' from a worktree
When worktree functionality was originally implemented, the possibility of 'clone --local' from within a worktree was overlooked, with the result that the location of the "objects" directory of the source repository was computed incorrectly, thus the objects could not be copied or hard-linked by the clone. This shortcoming was addressed by 744e469755 (clone: allow --local from a linked checkout, 2015-09-28).
However, the related case of 'clone --shared' (despite being handled only a few lines away from the 'clone --local' case) was not fixed by 744e469755, with a similar result of the "objects" directory location being incorrectly computed for insertion into the 'alternates' file. Fix this.
--- builtin/clone.c | 3 ++- t/t2025-worktree-add.sh | 6 ++++++ 2 files changed, 8 insertions(+), 1 deletion(-)
builtin/clone.c +2 -1
@@ -452,7 +452,8 @@ static void clone_local(const char *src_repo, const char *dest_repo) {if (option_shared) {struct strbuf alt =STRBUF_INIT ; -strbuf_addf (&alt,"%s/objects" , src_repo); +get_common_dir (&alt, src_repo); +strbuf_addstr (&alt,"/objects" );add_to_alternates_file (alt.buf);strbuf_release (&alt); }else {
t/t2025-worktree-add.sh +6 -0
@@ -245,6 +245,12 @@ test_expect_success 'local clone from linked checkout' ' ( cd here-clone && git fsck ) ' +test_expect_success 'local clone --shared from linked checkout' ' + git -C bare worktree add --detach ../baretree && + git clone --local --shared baretree bare-clone && + grep /bare/ bare-clone/.git/objects/info/alternates +' + test_expect_success '"add" worktree with --no-checkout' ' git worktree add --no-checkout -b swamp swamp && ! test -e swamp/init.t &&
Signature · 1 line
2.15.1.502.gccaef8de57
Quoted 25 lines · Eric Sunshine
When worktree functionality was originally implemented, the possibility of 'clone --local' from within a worktree was overlooked, with the result that the location of the "objects" directory of the source repository was computed incorrectly, thus the objects could not be copied or hard-linked by the clone. This shortcoming was addressed by
744e469755(clone: allow --local from a linked checkout, 2015-09-28).However, the related case of 'clone --shared' (despite being handled only a few lines away from the 'clone --local' case) was not fixed by
744e469755, with a similar result of the "objects" directory location being incorrectly computed for insertion into the 'alternates' file. Fix this.Reported-by Marc-André Lureau <[email protected]>Signed-off-by Eric Sunshine <[email protected]>--- builtin/clone.c | 3 ++- t/t2025-worktree-add.sh | 6 ++++++ 2 files changed, 8 insertions(+), 1 deletion(-)builtin/clone.c +0
-0
builtin/clone.c static void clone_local(const char *src_repo, const char *dest_repo) · line 459
Quoted 9 lines
builtin/clone.c +2
-1@@ -452,7 +452,8 @@ static void clone_local(const char *src_repo, const char *dest_repo) {if (option_shared) {struct strbuf alt =STRBUF_INIT ; -strbuf_addf (&alt,"%s/objects" , src_repo); +get_common_dir (&alt, src_repo); +strbuf_addstr (&alt,"/objects" );add_to_alternates_file (alt.buf);strbuf_release (&alt); }else {
Makes sense. Will queue.
Thanks.
Quoted 25 lines · Eric Sunshine
When worktree functionality was originally implemented, the possibility of 'clone --local' from within a worktree was overlooked, with the result that the location of the "objects" directory of the source repository was computed incorrectly, thus the objects could not be copied or hard-linked by the clone. This shortcoming was addressed by
744e469755(clone: allow --local from a linked checkout, 2015-09-28).However, the related case of 'clone --shared' (despite being handled only a few lines away from the 'clone --local' case) was not fixed by
744e469755, with a similar result of the "objects" directory location being incorrectly computed for insertion into the 'alternates' file. Fix this.Reported-by Marc-André Lureau <[email protected]>Signed-off-by Eric Sunshine <[email protected]>--- builtin/clone.c | 3 ++- t/t2025-worktree-add.sh | 6 ++++++ 2 files changed, 8 insertions(+), 1 deletion(-)builtin/clone.c +0
-0
builtin/clone.c static void clone_local(const char *src_repo, const char *dest_repo) · line 456
Quoted 6 lines
builtin/clone.c +2
-1@@ -452,7 +452,8 @@ static void clone_local(const char *src_repo, const char *dest_repo) {if (option_shared) {struct strbuf alt =STRBUF_INIT ; -strbuf_addf (&alt,"%s/objects" , src_repo); +get_common_dir (&alt, src_repo); +strbuf_addstr (&alt,"/objects" );
If you wanted to do this in one function call you could either use 'strbuf_git_common_path()' or either 'strbuf_git_path()' or 'strbuf_repo_git_path()' which will do the proper path adjustments when working on a path which should be shared between worktrees (i.e. part of the common git dir).
Quoted 23 lines
builtin/clone.c +0
-0add_to_alternates_file (alt.buf);strbuf_release (&alt); }else {t/t2025-worktree-add.sh +6
-0@@ -245,6 +245,12 @@ test_expect_success 'local clone from linked checkout' ' ( cd here-clone && git fsck ) ' +test_expect_success 'local clone --shared from linked checkout' ' + git -C bare worktree add --detach ../baretree && + git clone --local --shared baretree bare-clone && + grep /bare/ bare-clone/.git/objects/info/alternates +' + test_expect_success '"add" worktree with --no-checkout' ' git worktree add --no-checkout -b swamp swamp && ! test -e swamp/init.t &&Signature · 1 line
2.15.1.502.gccaef8de57
Signature · 1 line
Brandon Williams
Quoted 11 lines · Brandon Williams
builtin/clone.c static void clone_local(const char *src_repo, const char *dest_repo) · line 456
Quoted 4 lines · Eric Sunshine
struct strbuf alt = STRBUF_INIT;builtin/clone.c +2
-1-strbuf_addf (&alt,"%s/objects" , src_repo); +get_common_dir (&alt, src_repo); +strbuf_addstr (&alt,"/objects" );If you wanted to do this in one function call you could either use '
strbuf_git_common_path()' or either 'strbuf_git_path()' or 'strbuf_repo_git_path()' which will do the proper path adjustments when working on a path which should be shared between worktrees (i.e. part of the common git dir).
Thanks for the pointers, however, the above fix mirrors the fix made by 744e469755 (clone: allow --local from a linked checkout, 2015-09-28) to code immediately below it in the 'else' arm:
get_common_dir(&src, src_repo);
get_common_dir(&dest, dest_repo);
strbuf_addstr(&src, "/objects");
strbuf_addstr(&dest, "/objects");It would be poor form and confusing to use one of the mechanisms you suggest while leaving the 'else' arm untouched.
Re-working both arms of the 'if' to use one of the suggested functions would make a fine follow-on or preparatory patch, however, I'd rather not hold up this fix merely to re-roll for such a minor cleanup. (I also considered a follow-on patch to reduce the duplication between the two cases but decided against it, for the present, since such a patch would almost be noise without much gain.)
By the way, is there any documentation explaining the differences between all these similar functions and when one should be used over the others?
Quoted 25 lines · Eric Sunshine
Quoted 11 lines · Brandon Williams
builtin/clone.c static void clone_local(const char *src_repo, const char *dest_repo) · line 456
Quoted 4 lines · Eric Sunshine
struct strbuf alt = STRBUF_INIT;builtin/clone.c +2
-1-strbuf_addf (&alt,"%s/objects" , src_repo); +get_common_dir (&alt, src_repo); +strbuf_addstr (&alt,"/objects" );If you wanted to do this in one function call you could either use '
strbuf_git_common_path()' or either 'strbuf_git_path()' or 'strbuf_repo_git_path()' which will do the proper path adjustments when working on a path which should be shared between worktrees (i.e. part of the common git dir).Thanks for the pointers, however, the above fix mirrors the fix made by
744e469755(clone: allow --local from a linked checkout, 2015-09-28) to code immediately below it in the 'else' arm:get_common_dir(&src, src_repo); get_common_dir(&dest, dest_repo); strbuf_addstr(&src, "/objects"); strbuf_addstr(&dest, "/objects");It would be poor form and confusing to use one of the mechanisms you suggest while leaving the 'else' arm untouched.
Quoted 6 lines
Re-working both arms of the 'if' to use one of the suggested functions would make a fine follow-on or preparatory patch, however, I'd rather not hold up this fix merely to re-roll for such a minor cleanup. (I also considered a follow-on patch to reduce the duplication between the two cases but decided against it, for the present, since such a patch would almost be noise without much gain.)
I didn't look close enough at what you were trying to fix, you're right I think what you have here is good as the alternative would require a lot more reworking I think (at least to change the above part too).
Either way though, I'm a little worried about what happens if you have GIT_COMMON_DIR set because then both the src and dest repo would share a common dir, I don't know if that is expected or not. Maybe something else to consider later.
Quoted 4 lines
By the way, is there any documentation explaining the differences between all these similar functions and when one should be used over the others?
I wish, I probably should have done a better job documenting it all in path.h when I added the repo_* flavor of functions. I'll add that to my list of things to do though :)
Signature · 1 line
Brandon Williams
PATCH 1/1 path: document path functions
---
As promised here is an update to the documentation for the path generating functions.
path.h | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 112 insertions(+), 21 deletions(-)
path.h +112 -21
@@ -4,53 +4,144 @@struct repository ;/* -* Return a statically allocated filename, either generically (mkpath), in -* the repository directory (git_path), or in a submodule's repository -* directory (git_path_submodule). In all cases, note that the result -* may be overwritten by another call to _any_ of the functions. Consider -* using the safer "dup" or "strbuf" formats below (in some cases, the -* unsafe versions have already been removed). +* The result to all functions which return statically allocated memory may be +* overwritten by another call to _any_ one of these functions. Consider using +* the safer variants which operate on strbufs or return allocated memory. */ -extern const char *mkpath (const char *fmt, ...) __attribute__((format (printf,1 ,2 ))); -extern const char *git_path (const char *fmt, ...) __attribute__((format (printf,1 ,2 ))); -extern const char *git_common_path (const char *fmt, ...) __attribute__((format (printf,1 ,2 ))); +/* +* Return a statically allocated path. +*/ +extern const char *mkpath (const char *fmt, ...) + __attribute__((format (printf,1 ,2 ))); + +/* +* Return a path. +*/ +extern char *mkpathdup (const char *fmt, ...) + __attribute__((format (printf,1 ,2 ))); + +/* +* Construct a path and place the result in the provided buffer `buf`. +*/ extern char *mksnpath (char *buf,size_t n,const char *fmt, ...) __attribute__((format (printf,3 ,4 ))); -extern void strbuf_git_path (struct strbuf *sb,const char *fmt, ...) - __attribute__((format (printf,2 ,3 ))); + +/* +* The `git_common_path` family of functions will construct a path into a +* repository's common git directory, which is shared by all worktrees. +*/ + +/* +* Constructs a path into the common git directory of repository `repo` and +* append it in the provided buffer `sb`. +*/ extern void strbuf_git_common_path (struct strbuf *sb,const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,3 ,4 ))); -extern char *git_path_buf (struct strbuf *buf,const char *fmt, ...) - __attribute__((format (printf,2 ,3 ))); -extern int strbuf_git_path_submodule (struct strbuf *sb,const char *path, -const char *fmt, ...) - __attribute__((format (printf,3 ,4 ))); -extern char *git_pathdup (const char *fmt, ...) - __attribute__((format (printf,1 ,2 ))); -extern char *mkpathdup (const char *fmt, ...) + +/* +* Return a statically allocated path into the main repository's +* (the_repository) common git directory. +*/ +extern const char *git_common_path (const char *fmt, ...) __attribute__((format (printf,1 ,2 ))); -extern char *git_pathdup_submodule (const char *path,const char *fmt, ...) - __attribute__((format (printf,2 ,3 ))); + +/* +* The `git_path` family of functions will construct a path into a repository's +* git directory. +* +* These functions will perform adjustments to the resultant path to account +* for special paths which are either considered common among worktrees (e.g. +* paths into the object directory) or have been explicitly set via an +* environment variable or config (e.g. path to the index file). +* +* For an exhaustive list of the adjustments made look at `common_list` and +* `adjust_git_path` in path.c. +*/ + +/* +* Return a path into the git directory of repository `repo`. +*/ extern char *repo_git_path (const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,2 ,3 ))); + +/* +* Construct a path into the git directory of repository `repo` and append it +* to the provided buffer `sb`. +*/ extern void strbuf_repo_git_path (struct strbuf *sb,const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,3 ,4 ))); +/* +* Return a statically allocated path into the main repository's +* (the_repository) git directory. +*/ +extern const char *git_path (const char *fmt, ...) + __attribute__((format (printf,1 ,2 ))); + +/* +* Return a path into the main repository's (the_repository) git directory. +*/ +extern char *git_pathdup (const char *fmt, ...) + __attribute__((format (printf,1 ,2 ))); + +/* +* Construct a path into the main repository's (the_repository) git directory +* and place it in the provided buffer `buf`, the contents of the buffer will +* be overridden. +*/ +extern char *git_path_buf (struct strbuf *buf,const char *fmt, ...) + __attribute__((format (printf,2 ,3 ))); + +/* +* Construct a path into the main repository's (the_repository) git directory +* and append it to the provided buffer `sb`. +*/ +extern void strbuf_git_path (struct strbuf *sb,const char *fmt, ...) + __attribute__((format (printf,2 ,3 ))); + +/* +* Return a path into the worktree of repository `repo`. +* +* If the repository doesn't have a worktree NULL is returned. +*/ extern char *repo_worktree_path (const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,2 ,3 ))); + +/* +* Construct a path into the worktree of repository `repo` and append it +* to the provided buffer `sb`. +* +* If the repository doesn't have a worktree nothing will be appended to `sb`. +*/ extern void strbuf_repo_worktree_path (struct strbuf *sb,const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,3 ,4 ))); +/* +* Return a path into a submodule's git directory located at `path`. `path` +* must only reference a submodule of the main repository (the_repository). +*/ +extern char *git_pathdup_submodule (const char *path,const char *fmt, ...) + __attribute__((format (printf,2 ,3 ))); + +/* +* Construct a path into a submodule's git directory located at `path` and +* append it to the provided buffer `sb`. `path` must only reference a +* submodule of the main repository (the_repository). +*/ +extern int strbuf_git_path_submodule (struct strbuf *sb,const char *path, +const char *fmt, ...) + __attribute__((format (printf,3 ,4 ))); +extern void report_linked_checkout_garbage (void );/*
Signature · 1 line
2.15.1.504.g5279b80103-goog
Re: [PATCH] path: document path functions
Quoted 17 lines · Brandon Williams
Signed-off-by Brandon Williams <[email protected]>---
As promised here is an update to the documentation for the path generating functions.
path.h | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 112 insertions(+), 21 deletions(-)path.h +0
-0@@ -4,53 +4,144 @@struct repository ;/*
path.h line 10
Quoted 10 lines
path.h +3
-6-* Return a statically allocated filename, either generically (mkpath), in -* the repository directory (git_path), or in a submodule's repository -* directory (git_path_submodule). In all cases, note that the result -* may be overwritten by another call to _any_ of the functions. Consider -* using the safer "dup" or "strbuf" formats below (in some cases, the -* unsafe versions have already been removed). +* The result to all functions which return statically allocated memory may be +* overwritten by another call to _any_ one of these functions. Consider using +* the safer variants which operate on strbufs or return allocated memory. */
The result of the update definitely is better, but stepping back a bit, it still does not answer some questions the original did not:
- is the resulting path absolute? if relative, relative to what (cwd)?
- what should a caller write in fmt? can we have a couple of typical calling example?
- does a caller need to know which hierarchies under $GIT_DIR are common and call git_common_path() for them, or does git_path() do the 'right' thing for the caller? If latter, what's the use of git_common_path() for callers (outside the implementation of path.c API)?
Will queue. Thanks.
Re: [PATCH] path: document path functions
In reply to Brandon WilliamsQuoted 2 lines · Brandon Williams
As promised here is an update to the documentation for the path generating functions.
Thanks for working on this. Aside from the review comments by Junio[1], see a few more below. Although I had some familiarity with the non-repo versions of these functions a couple year ago when working on git-worktree, I've since forgotten much of it, so take my comments as if from someone trying to learn this API from scratch (which is effectively what I'm doing again). In particular, as I read this patch, I find that I still have to consult the source code (path.c) to figure out what some of these functions are for or what they do.
[1]: https://public-inbox.org/git/[email protected]/
path.h line 14
Quoted 13 lines
path.h +8
-0@@ -4,53 +4,144 @@/* +* The result to all functions which return statically allocated memory may be +* overwritten by another call to _any_ one of these functions. Consider using +* the safer variants which operate on strbufs or return allocated memory. */ +/* +* Return a statically allocated path. +*/ +extern const char *mkpath (const char *fmt, ...) + __attribute__((format (printf,1 ,2 )));
It's not, at first glance, clear what benefit this function provides over simply constructing a path manually with, say, sprintf() or strbuf. Should this mention that a certain amount of normalization is performed on the path?
Furthermore, perhaps the comment block above which talks about "statically allocated memory" could do a better job of conveying that these functions rid the caller of the responsibility of managing string storage itself. I wonder if it also might make sense to mention that the returned path won't necessarily be overwritten by the very next call (or is that an implementation detail we'd rather people not rely upon?).
path.h line 22
Quoted 5 lines
path.h +5
-0+/* +* Return a path. +*/ +extern char *mkpathdup (const char *fmt, ...) + __attribute__((format (printf,1 ,2 )));
I realize that it's implied by "dup", but perhaps the documentation should state explicitly that the caller is responsible for freeing the result?
path.h line 33
Quoted 4 lines
path.h +4
-0+/* +* The `git_common_path` family of functions will construct a path into a +* repository's common git directory, which is shared by all worktrees. +*/
My first question when reading this was "What is a common git directory? (Is that where a bunch of repositories live or what?)". I suppose if it had said "common .git/ directory", it would have been clearer. Perhaps mentioning GIT_DIR might help clarify it, as well.
...repository's common .git/ directory (or $GIT_DIR), which is shared by all worktrees.
path.h line 42
Quoted 8 lines
path.h +4
-0+/* +* Constructs a path into the common git directory of repository `repo` and +* append it in the provided buffer `sb`. +*/ extern void strbuf_git_common_path (struct strbuf *sb,const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,3 ,4 )));
It's nice that this states explicitly that it _appends_ rather than overwrites. So often, one has to consult the implementation to determine the actual behavior.
Perhaps I'm simple-minded, but at this point in the read, the concept of "common git directory" still feels nebulous. If the documentation presented even a simple example of how this is used (such as, for instance, calling this function to obtain ".git/HEAD"), it would help make the concept more concrete. Or, perhaps the example could be presented in the comment block just above which talks about this family of functions.
path.h line 49
Quoted 6 lines
path.h +5
-0+/* +* Return a statically allocated path into the main repository's +* (the_repository) common git directory. +*/ +extern const char *git_common_path (const char *fmt, ...) __attribute__((format (printf,1 ,2 )));
I suppose that other repositories would be submodules? Would it make sense to mention something about that to clue in readers?
Return a statically allocated path into the common .git/ directory of the main repository (not a submodule repository).
Or something.
Aside: After reading "main repository" here and seeing the 'repo' variants below, one wonders why the corresponding repo_git_common_path() doesn't exist. (Just an idle question; not actionable, and outside the scope of this patch.)
path.h line 63
Quoted 12 lines
path.h +12
-0+/* +* The `git_path` family of functions will construct a path into a repository's +* git directory. +* +* These functions will perform adjustments to the resultant path to account +* for special paths which are either considered common among worktrees (e.g. +* paths into the object directory) or have been explicitly set via an +* environment variable or config (e.g. path to the index file). +* +* For an exhaustive list of the adjustments made look at `common_list` and +* `adjust_git_path` in path.c. +*/
I feel somewhat clueless after reading this. It doesn't necessarily give enough information for the reader to understand when or why these functions should be used as opposed to one of the functions described earlier. This may be due to it not stating early enough or forcefully enough that the location of certain files is not fixed, that they reside elsewhere when worktrees are involved. Perhaps something along these lines would help make it more concrete:
Administrative files within .git/ may reside at different locations depending upon whether worktrees are involved. Some files and directories are common to all worktrees (for instance, ".git/hooks", ".git/objects") but others are localized to the main directory or to each worktree (for instance, ".git/HEAD" vs ".git/worktrees/<id>/HEAD). Other factors, such as environment variables or configuration settings may also impact locations of administrative files.
The 'git_path' family of functions employs specialized knowledge of these issues when constructing a path to an administrative file or directory, thus relieving the caller of the burden of manually figuring out where resources reside.
path.h line 70
Quoted 6 lines
path.h +3
-0+/* +* Return a path into the git directory of repository `repo`. +*/ extern char *repo_git_path (const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,2 ,3 )));
I presume by the return type that the caller is responsible for freeing the result. Perhaps the documentation can state that explicitly.
path.h line 92
Quoted 5 lines
path.h +5
-0+/* +* Return a path into the main repository's (the_repository) git directory. +*/ +extern char *git_pathdup (const char *fmt, ...) + __attribute__((format (printf,1 ,2 )));
Perhaps mention that the caller is responsible for freeing the result.
path.h line 97
Quoted 4 lines
path.h +4
-0+/* +* Construct a path into the main repository's (the_repository) git directory +* and place it in the provided buffer `buf`, the contents of the buffer will +* be overridden.
s/overridden/overwritten/
path.h line 100
Quoted 3 lines
path.h +3
-0+*/ +extern char *git_path_buf (struct strbuf *buf,const char *fmt, ...) + __attribute__((format (printf,2 ,3 )));
Nice to see that this explains that the buffer will be overwritten (as opposed to appended). This function is a less flexible special case of strbuf_git_path() below. Does this mean it is deprecated? Should the documentation say something about preferring strbuf_git_path() instead?
Quoted 5 lines
path.h +5
-0+/* +* Construct a path into the main repository's (the_repository) git directory +* and append it to the provided buffer `sb`. +*/ +extern void strbuf_git_path (struct strbuf *sb,const char *fmt, ...)
path.h line 116
Quoted 10 lines
path.h +7
-0+ __attribute__((format (printf, 2, 3))); + +/* +* Return a path into the worktree of repository `repo`. +* +* If the repository doesn't have a worktree NULL is returned. +*/ extern char *repo_worktree_path (const struct repository *repo,const char *fmt, ...) __attribute__((format (printf,2 ,3 )));
Now this is getting confusing. The documentation for the 'git_path' family of functions said that it takes worktrees into consideration. So what precisely do the 'worktree_path' family of functions do? When does one use them over the 'git_path' family?
Also, caller is responsible for freeing result.
path.h line 132
Quoted 4 lines
path.h +4
-0+/* +* Return a path into a submodule's git directory located at `path`. `path` +* must only reference a submodule of the main repository (the_repository). +*/
Ouch, ambiguity hurts my brain. Does this mean that the submodule is located at 'path' or that the .git/ directory is located at 'path'? Perhaps:
Return a path into the .git/ directory of a submodule located at 'path'.