fetch: allow adding a filter after initial clone.
PATCH 1/1 fetch: allow adding a filter after initial clone.
--- builtin/fetch.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
builtin/fetch.c +10 -2
@@ -1790,8 +1790,16 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); +if (filter_options.choice && !has_promisor_remote ()) { +char repo_version_string[10 ]; + +xsnprintf (repo_version_string,sizeof (repo_version_string), +"%d" , (int )GIT_REPO_VERSION ); +git_config_set ("core.repositoryformatversion" , + repo_version_string); +git_config_set ("extensions.partialclone" ,"origin" ); +promisor_remote_reinit (); + }if (all) {if (argc ==1 )
Signature · 1 line
2.26.2.645.ge9eca65c58-goog
Xin Li <[email protected]> writes:
(nothnig).
Can you help readers by describing what this change is about?
This space is reserved for the patch author to describe why this change is a good idea (if this patch is adding a new feature), what is already broken without this patch (if this patch is a bugfix), and why this change is a safe thing to do (if this patch lifts a limitation we had before that has been protecting us from getting into a bad state).
Quoted 28 lines
Signed-off-by Xin Li <[email protected]>--- builtin/fetch.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)builtin/fetch.c +10
-2@@ -1790,8 +1790,16 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); +if (filter_options.choice && !has_promisor_remote ()) { +char repo_version_string[10 ]; + +xsnprintf (repo_version_string,sizeof (repo_version_string), +"%d" , (int )GIT_REPO_VERSION ); +git_config_set ("core.repositoryformatversion" , + repo_version_string); +git_config_set ("extensions.partialclone" ,"origin" ); +promisor_remote_reinit (); + }if (all) {if (argc ==1 )
The proposed change would allow converting an existing git clone to use partial clones. For example:
$ git clone --depth=1 https://android.googlesource.com/platform/bionic . $ git fetch --unshallow --filter=blob:none origin
Previously, to allow this one would have to do the following manually first; the existing code would handle the rest gracefully:
$ git config core.repositoryFormatVersion 1 $ git config extensions.partialClone origin
And the proposed change would have git do it for the user automatically, the existing workflow remains otherwise unchanged.
Quoted 6 lines · Xin Li
The proposed change would allow converting an existing git clone to use partial clones. For example:
$ git clone --depth=1 https://android.googlesource.com/platform/bionic . $ git fetch --unshallow --filter=blob:none origin
Quoted 8 lines
Previously, to allow this one would have to do the following manually first; the existing code would handle the rest gracefully:
$ git config core.repositoryFormatVersion 1 $ git config extensions.partialClone originAnd the proposed change would have git do it for the user automatically, the existing workflow remains otherwise unchanged.
Please do not explain these things to the list readers. Instead, remember to write them in the proposed log message when you are ready to send an updated patch, which you would probably want to wait before giving other people to comment on the patch.
Thanks.
Quoted 6 lines · Xin Li
The proposed change would allow converting an existing git clone to use partial clones. For example:
$ git clone --depth=1 https://android.googlesource.com/platform/bionic . $ git fetch --unshallow --filter=blob:none origin
Quoted 8 lines
Previously, to allow this one would have to do the following manually first; the existing code would handle the rest gracefully:
$ git config core.repositoryFormatVersion 1 $ git config extensions.partialClone originAnd the proposed change would have git do it for the user automatically, the existing workflow remains otherwise unchanged.
Actually, do we even know, when we "upgrade" the repository with the patch, that the 'origin' supports the required protocol extensions like on-demand fetching of objects?
At "git clone" time, "git clone --filter=..." against a server that is not prepared to serve as a lazy clone's backup remote would fail, so we will never write extensions.partialClone=origin when origin cannot serve as such.
But writing these config you suggest to "do manually" is already not a safe thing to do, without making sure it is supported, no? It does not sound like a good idea to do an unsafe thing for the user automatically.
Added a few folks on Cc: list, who know and care more about partial clones than I do for input.
Thanks.
Quoted 13 lines · Xin Li
Signed-off-by Xin Li <[email protected]>--- builtin/fetch.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)builtin/fetch.c +0
-0@@ -1790,8 +1790,16 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ;
builtin/fetch.c int cmd_fetch(int argc, const char **argv, const char *prefix) · line 1800
Quoted 10 lines
builtin/fetch.c +8
-2-if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); +if (filter_options.choice && !has_promisor_remote ()) { +char repo_version_string[10 ]; + +xsnprintf (repo_version_string,sizeof (repo_version_string), +"%d" , (int )GIT_REPO_VERSION ); +git_config_set ("core.repositoryformatversion" , + repo_version_string); +git_config_set ("extensions.partialclone" ,"origin" );
Some things stood out to me here. One, is this setting up the repository if it's not already created? If so, we'd probably want to use one of the appropriate functions in setup.c. Even if we're just changing it, we should probably use a helper function.
Two, it isn't necessarily safe to automatically change the repository version. Keys that start with "extensions." are not special in format version 0, but they are in format version 1. I can technically have an "extensions.tomatoSalad" in version 0 without any special meaning or negative consequences, but as soon as we change to version 1, Git will refuse to read it, since it doesn't know about the tomatoSalad extension and in v1 unknown extensions are fatal.
My example may sound silly, but since extensions can be set in the global config, users could well rely on v0 repositories ignoring them and having them automatically turned on for their v1 repositories. (I'm thinking of the future extensions.objectFormat as something somebody might try to do here, as dangerous as it may be.)
These aren't insurmountable problems, but they are things we'd need to check for before just changing the repository version, so we'd want to stuff some code in setup.c to handle this case.
Third, I'm not sure that "origin" is always the value we want to use here. At a previous employer, the upstream remote was called "upstream", and your personal fork was called "origin", so I'd have wanted upstream here.. We'd probably want to use whatever remote the user is already using in this case, and gracefully handle the URL case if that isn't allowed here (which it may be; I'm not that familiar with partial clone).
I also agree with Junio's assessment that you'd probably want to explain more about this feature in the commit message. For example, I'd want to know what this patch does and have some idea of how I might invoke this feature, why it's safe to change the repository version, how one sets the remote for fetch, and pretty much answers to all the other things I asked here. Even if I understand these things now, that doesn't mean a future developer will in six months' time, and mentioning these things in the commit message helps leave a note to them that you considered (or did not consider, as the case may be) certain issues and helps them understand the code as you saw and wrote it.
Signature · 2 lines
brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204
v2 cover fetch: allow adding a filter after initial clone
Previously, to retroactively add filter to an existing (shallow) clone one would have to manually change the repository configuration to make git to believe that there was an existing promisor, like:
git config core.repositoryFormatVersion 1 git config extensions.partialClone origin git fetch --unshallow --filter=blob:none origin
Because the code can already set up promisor, it would be safer and more convenient to just do that in git itself.
This version of change will also prevent the code from making damaging repository upgrades (when non-standard extensions exists) as pointed out by earlier reviewers.
Xin Li (1):
fetch: allow adding a filter after initial clone.
builtin/fetch.c | 3 --- list-objects-filter-options.c | 3 ++- repository.h | 6 ++++++ setup.c | 30 ++++++++++++++++++++++++++++++ t/t0410-partial-clone.sh | 21 +++++++++++++++++++++ 5 files changed, 59 insertions(+), 4 deletions(-)
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
PATCH v2 1/1 fetch: allow adding a filter after initial clone.
In reply to brian m. carlsonRetroactively adding filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Previously this is possible by manually amending the repository configuration to make git think there is an existing promisor. Because the code already does most of the hard work, it's safer for git to just perform the configuration change automatically instead.
Instead of bailing out immediately when no promisor is available, make the code check more specific issue (extension became special in repository version 1, while it can have any value in version 0, so upgrade should not happen if the repository have an unsupported configuration that would render it invalid if we upgraded).
--- builtin/fetch.c | 3 --- list-objects-filter-options.c | 3 ++- repository.h | 6 ++++++ setup.c | 30 ++++++++++++++++++++++++++++++ t/t0410-partial-clone.sh | 21 +++++++++++++++++++++ 5 files changed, 59 insertions(+), 4 deletions(-)
builtin/fetch.c +0 -3
@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));
list-objects-filter-options.c +2 -1
@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (the_repository,1 ) <0 ) +die (_ ("Unable to upgrade repository format to support partial clone" ));/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
repository.h +6 -0
@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary. +*/ +int upgrade_repository_format (struct repository *r,int target_version); +#endif /* REPOSITORY_H */
setup.c +30 -0
@@ -538,6 +538,36 @@ static int check_repository_format_gently(const char *gitdir, struct repository_return 0 ; } +int upgrade_repository_format (struct repository *r,int target_version) +{ +const char *gitdir =get_git_dir (); +struct strbuf sb =STRBUF_INIT ; +struct strbuf err =STRBUF_INIT ; +struct strbuf repo_version =STRBUF_INIT ; +struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ; + +strbuf_git_common_path (&sb, r,"/config" ); +read_repository_format (&repo_fmt, sb.buf); +strbuf_release (&sb); + +if (repo_fmt.version >= target_version) +return 0 ; + + repo_fmt.version = target_version; + +if (verify_repository_format (&repo_fmt, &err) <0 ) { +warning ("Unable to upgrade repository format to %d: %s" , + target_version, err.buf); +strbuf_release (&err); +return -1 ; + } + +strbuf_addf (&repo_version,"%d" , target_version); +git_config_set ("core.repositoryformatversion" , repo_version.buf); +strbuf_release (&repo_version); +return 1 ; +} +static void init_repository_format (struct repository_format *format) {const struct repository_format fresh =REPOSITORY_FORMAT_INIT ;
t/t0410-partial-clone.sh +21 -0
@@ -30,6 +30,27 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +' +test_expect_success 'convert shallow clone to partial clone must fail with invalid extension' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client config extensions.sandwidth true && + test_must_fail git -C client fetch --unshallow --filter="blob:none" +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Hi,
Quoted 9 lines · Xin Li
Retroactively adding filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Previously this is possible by manually amending the repository configuration to make git think there is an existing promisor. Because the code already does most of the hard work, it's safer for git to just perform the configuration change automatically instead.
Quoted 8 lines
Instead of bailing out immediately when no promisor is available, make the code check more specific issue (extension became special in repository version 1, while it can have any value in version 0, so upgrade should not happen if the repository have an unsupported configuration that would render it invalid if we upgraded).
Signed-off-by Xin Li <[email protected]>---
nit: the cover letter contains
Quoted 8 lines
Previously, to retroactively add filter to an existing (shallow) clone one would have to manually change the repository configuration to make git to believe that there was an existing promisor, like:
git config core.repositoryFormatVersion 1 git config extensions.partialClone origin git fetch --unshallow --filter=blob:none origin
Quoted 6 lines
Because the code can already set up promisor, it would be safer and more convenient to just do that in git itself.
This version of change will also prevent the code from making damaging repository upgrades (when non-standard extensions exists) as pointed out by earlier reviewers.
I think that would make a good commit message itself.
[...]
builtin/fetch.c int cmd_fetch(int argc, const char **argv, const char *prefix) · line 1795
Quoted 9 lines
builtin/fetch.c +0
-3@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -
Makes sense.
[...]
list-objects-filter-options.c void partial_clone_register( · line 330
Quoted 9 lines
list-objects-filter-options.c +2
-1@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (the_repository,1 ) <0 ) +die (_ ("Unable to upgrade repository format to support partial clone" ));
nit: Git's error messages tend to use lowercase (e.g., "fatal: cannot [etc]" instead of "fatal: Unable [etc]").
list-objects-filter-options.c void partial_clone_register( · line 332
Quoted 2 lines
list-objects-filter-options.c +0
-0/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
not about this patch: By the way, the repository format version bump is not sufficient to achieve its intended aim: we also need to set an extensions.* setting to ensure Git is new enough to know about partial clone. More discussion about this is in [1] (apologies for not having finished solving that). This isn't a regression introduced in this patch, and this patch does the right thing in the context of the current code.
[...]
repository.h void repo_update_index_if_able(struct repository *, struct lock_file *); · line 202
Quoted 10 lines
repository.h +4
-0@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary. +*/
Probably also worth mentioning that this returns -1 on error.
Quoted 3 lines
repository.h +2
-0+int upgrade_repository_format (struct repository *r,int target_version); +#endif /* REPOSITORY_H */
[...]
repository.h static int check_repository_format_gently(const char *gitdir, struct repository_ · line 543
Quoted 8 lines
+++
b/setup.crepository.h +3
-0@@ -538,6 +538,36 @@ static int check_repository_format_gently(const char *gitdir, struct repository_return 0 ; } +int upgrade_repository_format (struct repository *r,int target_version) +{ +const char *gitdir =get_git_dir ();
Unused variable.
repository.h static int check_repository_format_gently(const char *gitdir, struct repository_ · line 549
Quoted 6 lines
repository.h +6
-0+struct strbuf sb =STRBUF_INIT ; +struct strbuf err =STRBUF_INIT ; +struct strbuf repo_version =STRBUF_INIT ; +struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ; + +strbuf_git_common_path (&sb, r,"/config" );
nit: can leave out the '/' to avoid a double-/.
repository.h static int check_repository_format_gently(const char *gitdir, struct repository_ · line 559
Quoted 10 lines
repository.h +10
-0+read_repository_format (&repo_fmt, sb.buf); +strbuf_release (&sb); + +if (repo_fmt.version >= target_version) +return 0 ; + + repo_fmt.version = target_version; + +if (verify_repository_format (&repo_fmt, &err) <0 ) { + warning("Unable to upgrade repository format to %d: %s",
Same nit about capitalization.
repository.h static int check_repository_format_gently(const char *gitdir, struct repository_ · line 560
Quoted 1 line
repository.h +1
-0+ target_version, err.buf);
whitespace nit: this would typically use a tab, to line up with the paren on the previous line.
repository.h static int check_repository_format_gently(const char *gitdir, struct repository_ · line 566
Quoted 6 lines
repository.h +6
-0+strbuf_release (&err); +return -1 ; + } + +strbuf_addf (&repo_version,"%d" , target_version); +git_config_set ("core.repositoryformatversion" , repo_version.buf);
Ah, I think I misled you: the config_set API hasn't learned to take a struct repository yet, so we should hardcode the_repository in this function instead of taking a "struct repository" parameter.
Quoted 16 lines
repository.h +4
-0+strbuf_release (&repo_version); +return 1 ; +} +static void init_repository_format (struct repository_format *format) {const struct repository_format fresh =REPOSITORY_FORMAT_INIT ;t/t0410-partial-clone.sh +1
-0@@ -30,6 +30,27 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' '
t/t0410-partial-clone.sh test_expect_success 'extensions.partialclone without filter' ' · line 43
Quoted 10 lines
t/t0410-partial-clone.sh +10
-0+ rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +'
nit: Missing blank line.
Is there a different check this test could perform to check the user-facing behavior instead of how the configuration is encoded?
t/t0410-partial-clone.sh test_expect_success 'extensions.partialclone without filter' ' · line 56
Quoted 13 lines
t/t0410-partial-clone.sh +10
-0+test_expect_success 'convert shallow clone to partial clone must fail with invalid extension' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client config extensions.sandwidth true && + test_must_fail git -C client fetch --unshallow --filter="blob:none" +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
With whatever subset of the mentioned changes makes sense,
Thanks.
[1] https://lore.kernel.org/git/[email protected]/.
list-objects-filter-options.c +2 -2
@@ -326,8 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -if (upgrade_repository_format (the_repository,1 ) <0 ) -die (_ ("Unable to upgrade repository format to support partial clone" )); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("cannot enable partial clone support" ));/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
repository.h +4 -3
@@ -197,9 +197,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r);/* -* Return 1 if upgrade repository format to target_version succeeded, -* 0 if no upgrade is necessary. +* Upgrade the repository format to target_version. +* Returns 1 on success, 0 if no upgrade was necessary, and -1 after +* printing a diagnostic on error. */ -int upgrade_repository_format (struct repository *r,int target_version); +int upgrade_repository_format (int target_version);#endif /* REPOSITORY_H */
setup.c +4 -5
@@ -538,15 +538,14 @@ static int check_repository_format_gently(const char *gitdir, struct repository_return 0 ; } -int upgrade_repository_format (struct repository *r,int target_version) +int upgrade_repository_format (int target_version) { -const char *gitdir =get_git_dir ();struct strbuf sb =STRBUF_INIT ;struct strbuf err =STRBUF_INIT ;struct strbuf repo_version =STRBUF_INIT ;struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ; -strbuf_git_common_path (&sb, r,"/config" ); +strbuf_git_common_path (&sb, the_repository,"config" );read_repository_format (&repo_fmt, sb.buf);strbuf_release (&sb);
@@ -556,8 +555,8 @@ int upgrade_repository_format(struct repository *r, int target_version) repo_fmt.version = target_version;if (verify_repository_format (&repo_fmt, &err) <0 ) { -warning ("Unable to upgrade repository format to %d: %s" , - target_version, err.buf); +warning ("cannot upgrade repository format to %d: %s" , + target_version, err.buf);strbuf_release (&err);return -1 ; }
t/t0410-partial-clone.sh +1 -0
@@ -41,6 +41,7 @@ test_expect_success 'convert shallow clone to partial clone' ' test_cmp_config -C client blob:none remote.origin.partialclonefilter && test_cmp_config -C client 1 core.repositoryformatversion ' + test_expect_success 'convert shallow clone to partial clone must fail with invalid extension' ' rm -fr server client && test_create_repo server &&
PATCH v3 1/1 fetch: allow adding a filter after initial clone.
Retroactively adding filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:
git config core.repositoryFormatVersion 1 git config extensions.partialClone origin git fetch --unshallow --filter=blob:none origin
Since the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead.
Instead of bailing out immediately when no promisor is available, make the code perform a more precise check for any potential problems (extensions became special in repository version 1, while it can have any value in version 0, so upgrade should not happen if the repository have an unsupported configuration that would render it invalid if we upgraded).
--- builtin/fetch.c | 3 --- list-objects-filter-options.c | 3 ++- repository.h | 6 ++++++ setup.c | 29 +++++++++++++++++++++++++++++ t/t0410-partial-clone.sh | 22 ++++++++++++++++++++++ 5 files changed, 59 insertions(+), 4 deletions(-)
builtin/fetch.c +0 -3
@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));
list-objects-filter-options.c +2 -1
@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to support partial clone" ));/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
repository.h +6 -0
@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary; returns -1 when upgrade is not possible. +*/ +int upgrade_repository_format (int target_version); +#endif /* REPOSITORY_H */
setup.c +29 -0
@@ -538,6 +538,35 @@ static int check_repository_format_gently(const char *gitdir, struct repository_return 0 ; } +int upgrade_repository_format (int target_version) +{ +struct strbuf sb =STRBUF_INIT ; +struct strbuf err =STRBUF_INIT ; +struct strbuf repo_version =STRBUF_INIT ; +struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ; + +strbuf_git_common_path (&sb, the_repository,"config" ); +read_repository_format (&repo_fmt, sb.buf); +strbuf_release (&sb); + +if (repo_fmt.version >= target_version) +return 0 ; + + repo_fmt.version = target_version; + +if (verify_repository_format (&repo_fmt, &err) <0 ) { +warning ("unable to upgrade repository format to %d: %s" , + target_version, err.buf); +strbuf_release (&err); +return -1 ; + } + +strbuf_addf (&repo_version,"%d" , target_version); +git_config_set ("core.repositoryformatversion" , repo_version.buf); +strbuf_release (&repo_version); +return 1 ; +} +static void init_repository_format (struct repository_format *format) {const struct repository_format fresh =REPOSITORY_FORMAT_INIT ;
t/t0410-partial-clone.sh +22 -0
@@ -30,6 +30,28 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +' + +test_expect_success 'convert shallow clone to partial clone must fail with invalid extension' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client config extensions.sandwidth true && + test_must_fail git -C client fetch --unshallow --filter="blob:none" +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Quoted 5 lines · Xin Li
Instead of bailing out immediately when no promisor is available, make the code check more specific issue (extension became special in repository version 1, while it can have any value in version 0, so upgrade should not happen if the repository have an unsupported configuration that would render it invalid if we upgraded).
It probably has to be a lot stronger than that. In version 0, extension did not have any meaning, so an existing repository can safely have "[extension] worktreeConfig=~/hello" as long as it is version 0 and nobody would bitch about extension.worktreeConfig not being a boolean; worse yet, "[extension] worktreeConfig=true" in version 0 repository did not make its secondary worktrees to have separate configuration, but if we upgrade to version 1 merely because the version of Git knows what extension.worktreeConfig means, we broke the repository and its worktrees.
It would be safe to upgrade version 0 repository when there is *no* existing configuration variable in the "extension" section, but "the repository have an unsupported configuration" is not a useful or safe criteria to decide if we should refrain from upgrading, I would think.
Quoted 2 lines · Junio C Hamano
It would be safe to upgrade version 0 repository when there is *no* existing configuration variable in the "extension" section,
Yes, that makes sense (also for the case Brian mentioned where someone may try setting extension.objectFormat, see no effect, and then forget about it).
Thanks,
Jonathan
Quoted 9 lines · Jonathan Nieder
Quoted 2 lines · Junio C Hamano
It would be safe to upgrade version 0 repository when there is *no* existing configuration variable in the "extension" section,
Yes, that makes sense (also for the case Brian mentioned where someone may try setting extension.objectFormat, see no effect, and then forget about it).
I see that's a good point
I found that the current code doesn't really check repositoryformatversion for e.g. partialclone, so without setting repositoryformatversion to 1, just setting extensions.partialclone = origin will still make fetch --filter work and some test cases are depending on this, do we want to keep this feature/bug? I have added some additional checks to my local git version and found that we would also need to fix a lot of test cases for worktree, etc. if we would enforce the check.
Quoted 5 lines · Xin Li
I found that the current code doesn't really check repositoryformatversion for e.g. partialclone, so without setting repositoryformatversion to 1, just setting extensions.partialclone = origin will still make fetch --filter work and some test cases are depending on this, do we want to keep this feature/bug?
It's orthogonal to this patch, but no, we don't.
Thanks,
Jonathan
PATCH v4 1/1 fetch: allow adding a filter after initial clone.
Retroactively adding filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:
git config core.repositoryFormatVersion 1 git config extensions.partialClone origin git fetch --unshallow --filter=blob:none origin
Since the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead.
Instead of bailing out immediately when no promisor is available, make the code perform a more precise check for any potential problems (extensions became special in repository version 1, while it can have any value in version 0, so upgrade should not happen if the repository have an unsupported configuration that would render it invalid if we upgraded).
--- builtin/fetch.c | 3 -- builtin/sparse-checkout.c | 2 + cache.h | 2 + list-objects-filter-options.c | 3 +- repository.h | 6 +++ setup.c | 77 ++++++++++++++++++++++++++++++-- t/t0410-partial-clone.sh | 23 ++++++++++ t/t1090-sparse-checkout-scope.sh | 1 - t/t2404-worktree-config.sh | 4 +- t/t5500-fetch-pack.sh | 1 - t/t5702-protocol-v2.sh | 1 - 11 files changed, 111 insertions(+), 12 deletions(-)
builtin/fetch.c +0 -3
@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));
builtin/sparse-checkout.c +2 -0
@@ -249,6 +249,8 @@ static int set_config(enum sparse_checkout_mode mode) {const char *config_path; +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to enable worktreeConfig" ));if (git_config_set_gently ("extensions.worktreeConfig" ,"true" )) {error (_ ("failed to set extensions.worktreeConfig setting" ));return 1 ;
cache.h +2 -0
@@ -1042,6 +1042,7 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; +int has_extensions;char *work_tree;struct string_list unknown_extensions; };
@@ -1056,6 +1057,7 @@ struct repository_format { .version = -1, \ .is_bare = -1, \ .hash_algo = GIT_HASH_SHA1, \ + .has_extensions = 0, \ .unknown_extensions = STRING_LIST_INIT_DUP, \ }
list-objects-filter-options.c +2 -1
@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to support partial clone" ));/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
repository.h +6 -0
@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary; returns -1 when upgrade is not possible. +*/ +int upgrade_repository_format (int target_version); +#endif /* REPOSITORY_H */
setup.c +73 -4
119 lines not on this page · raw message
t/t0410-partial-clone.sh +23 -0
@@ -30,6 +30,29 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +' + +test_expect_success 'convert shallow clone to partial clone must fail with any extension' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + test_cmp_config -C client 0 core.repositoryformatversion && + git -C client config extensions.partialclone origin && + test_must_fail git -C client fetch --unshallow --filter="blob:none" +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
t/t1090-sparse-checkout-scope.sh +0 -1
@@ -63,7 +63,6 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs git -C server commit -m message && test_config -C client core.sparsecheckout 1 && - test_config -C client extensions.partialclone origin && echo "!/*" >client/.git/info/sparse-checkout && echo "/a" >>client/.git/info/sparse-checkout && git -C client fetch --filter=blob:none origin &&
t/t2404-worktree-config.sh +3 -1
@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' ' test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true && - test_cmp_config true extensions.worktreeConfig + test_cmp_config true extensions.worktreeConfig && + test_cmp_config 1 core.repositoryformatversion ' test_expect_success 'config is shared as before' '
t/t5500-fetch-pack.sh +0 -1
@@ -999,7 +999,6 @@ fetch_filter_blob_limit_zero () { test_config -C "$SERVER" uploadpack.allowfilter 1 && git clone "$URL" client && - test_config -C client extensions.partialclone origin && test_commit -C "$SERVER" two &&
t/t5702-protocol-v2.sh +0 -1
@@ -348,7 +348,6 @@ test_expect_success 'partial fetch' ' rm -rf client "$(pwd)/trace" && git init client && SERVER="file://$(pwd)/server" && - test_config -C client extensions.partialClone "$SERVER" && GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \ fetch --filter=blob:none "$SERVER" master:refs/heads/other &&
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Quoted 15 lines · Xin Li
Retroactively adding filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:
git config core.repositoryFormatVersion 1 git config extensions.partialClone origin git fetch --unshallow --filter=blob:none originSince the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead.
Quoted 6 lines
Instead of bailing out immediately when no promisor is available, make the code perform a more precise check for any potential problems (extensions became special in repository version 1, while it can have any value in version 0, so upgrade should not happen if the repository have an unsupported configuration that would render it invalid if we upgraded).
Upgrade from v0 to v1 must follow the more strict "no extension" rule, not "no unknown ones" rule, so the above description must be corrected. Perhaps like this?
... so upgrade from version 0 should not happen if the repository has ANY extension. A repository version 1 and later make Git fail if there is any unknown extension, so we need to fail an upgrade only if there is any extension that is unknown to us).
You can drop the second paragraph about upgrading from version 1 to a later version if you want, as the only interesting use cases in practice at this point are upgrading from v0 to v1 and staying at v1.
Quoted 2 lines
Signed-off-by Xin Li <[email protected]>Reviewed-by Jonathan Nieder <[email protected]>
I think the updated design looks good. Let's nitpick some styles ;-)
Quoted 19 lines
builtin/fetch.c +0
-3@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));builtin/sparse-checkout.c +0
-0
builtin/sparse-checkout.c static int set_config(enum sparse_checkout_mode mode) · line 256
Quoted 8 lines
builtin/sparse-checkout.c +2
-0@@ -249,6 +249,8 @@ static int set_config(enum sparse_checkout_mode mode) {const char *config_path; +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to enable worktreeConfig" ));if (git_config_set_gently ("extensions.worktreeConfig" ,"true" )) {error (_ ("failed to set extensions.worktreeConfig setting" ));return 1 ;
OK.
Quoted 13 lines
cache.h +1
-0@@ -1042,6 +1042,7 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; +int has_extensions;char *work_tree;struct string_list unknown_extensions; };
cache.h struct repository_format { · line 1060
Quoted 4 lines
cache.h +1
-0@@ -1056,6 +1057,7 @@ struct repository_format { .version = -1, \ .is_bare = -1, \ .hash_algo = GIT_HASH_SHA1, \ + .has_extensions = 0, \
I am on the fence between "explicitly initializing to zero value is pointless, especially when we use .designated_initializer" and "especially with .designated_initializer, it adds a documentation value to explicitly initialize a field to its zero value". Unless other reviewers weigh in, I am OK to let this stand as-is.
Quoted 8 lines
cache.h +0
-0.unknown_extensions = STRING_LIST_INIT_DUP, \ }list-objects-filter-options.c +0
-0
list-objects-filter-options.c void partial_clone_register( · line 330
Quoted 6 lines
list-objects-filter-options.c +2
-1@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to support partial clone" ));
OK.
repository.h void repo_update_index_if_able(struct repository *, struct lock_file *); · line 202
Quoted 12 lines
repository.h +4
-0@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary; returns -1 when upgrade is not possible. +*/
Do we want to start with "Return" but say "returns" later?
Return 1 if ..., 0 if ..., and -1 when upgrade is not possible.
Quoted 7 lines
repository.h +5
-0+int upgrade_repository_format (int target_version); +#endif /* REPOSITORY_H */ +int upgrade_repository_format (int target_version) +{ +struct strbuf sb =STRBUF_INIT ;
repository.h void repo_update_index_if_able(struct repository *, struct lock_file *);
Quoted 10 lines
repository.h +10
-0+struct strbuf err =STRBUF_INIT ; +struct strbuf repo_version =STRBUF_INIT ; +struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ; + +strbuf_git_common_path (&sb, the_repository,"config" ); +read_repository_format (&repo_fmt, sb.buf); +strbuf_release (&sb); + +if (repo_fmt.version >= target_version) +return 0 ;
OK. That's "already up-to-date" case.
repository.h
Quoted 2 lines
repository.h +2
-0+if (verify_repository_format_eligibility (&repo_fmt, &err, + target_version) <0 ) {
I do not think this _eligibility thing should be a separate helper function. One reason is that its name sounds nonsensical ("eligible for what? it deserives to be verified for its repo format?"), another is it makes it unclear what "upgrade" requires by hiding the logic inside that decides the eligibility for upgrading. Besides, there is only one callsite.
Open-coding the gist of the helper like this:
if (verify_repository_format(&repo_fmt, &err) < 0 ||
(!repo_fmt.version && repo_fmt.has_extensions)) {should make it a lot clearer to see. If the repository is unusable by the version of Git we are running already, or the repository is v0 and has configuration variable(s) in "extensions.*" section, we refuse to upgrade.
Which is slightly different from what you did with the three-way split of verify_repository_format(), which made the "eligibility" thing not to care about unknown extensions in a repository v1 and higher. I actually think we should refuse to update v1 or v2 repository to v3 with a running Git that knows only about v1 (i.e. the repository before upgrading may or may not be something we understand, and if we do not understand it, we shouldn't touch it).
repository.h
Quoted 6 lines
repository.h +6
-0+warning ("unable to upgrade repository format from %d to %d: %s" , + repo_fmt.version, target_version, err.buf); +strbuf_release (&err); +return -1 ; + } + ...
And with the suggested change to eliminate "eligibility" helper, none of the changes below would become necessary, I would think, so I won't say things like "we do not say 'if (result != 0)'; instead we just say 'if (result)'" ;-)
Thanks.
Quoted 7 lines · Junio C Hamano
Which is slightly different from what you did with the three-way split of
verify_repository_format(), which made the "eligibility" thing not to care about unknown extensions in a repository v1 and higher. I actually think we should refuse to update v1 or v2 repository to v3 with a running Git that knows only about v1 (i.e. the repository before upgrading may or may not be something we understand, and if we do not understand it, we shouldn't touch it).
It does not change the conclusion, but I think the above sample situation would not make much sense---a caller that asks this function to upgrade the repository to v3 when the version of Git it is linked in does not understand v3 is simply buggy.
But we should still refuse to update v1 to v2 with a version of Git that understands v2 if the repository has some extension that we do not know about, so
(1) if upgrading from v0, there must be no "extensions.*"; and
(2) if upgrading from other versions, there must be no "extensions.*" we do not recognise.
I suggested would still be the reasonably defensive rule.
Thanks.
(+cc: Jonathan Tan, partial clone expert)
Quoted 1 line · Xin Li
Subject: fetch: allow adding a filter after initial clone.
nit: should be no period at the end of the subject line
Quoted 2 lines
Signed-off-by Xin Li <[email protected]>Reviewed-by Jonathan Nieder <[email protected]>
Let's see whether I still stand by that. ;-)
[...]
setup.c static int work_tree_config_is_bogus; · line 17
Quoted 8 lines
setup.c +2
-0@@ -13,6 +13,9 @@ static int work_tree_config_is_bogus;static struct startup_info the_startup_info;struct startup_info *startup_info = &the_startup_info; +static int verify_repository_format_eligibility (const struct repository_format *, +struct strbuf *,int );
Odd wrapping, but after Junio's suggestion this goes away.
[...]
setup.c static int check_repository_format_gently(const char *gitdir, struct repository_ · line 568
Quoted 2 lines
setup.c +2
-0+warning ("unable to upgrade repository format from %d to %d: %s" , + repo_fmt.version, target_version, err.buf);
Same wrapping nit: this should use a tab to line up right after the parenthesis (Git uses 8-space tabs).
[...]
t/t1090-sparse-checkout-scope.sh test_expect_success 'in partial clone, sparse checkout only fetches needed blobs · line 66
Quoted 7 lines
t/t1090-sparse-checkout-scope.sh +0
-1@@ -63,7 +63,6 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs git -C server commit -m message && test_config -C client core.sparsecheckout 1 && - test_config -C client extensions.partialclone origin &&
Nice.
t/t2404-worktree-config.sh test_expect_success 'config --worktree without extension' ' · line 27
Quoted 13 lines
t/t1090-sparse-checkout-scope.sh +0
-0echo "!/*" >client/.git/info/sparse-checkout && echo "/a" >>client/.git/info/sparse-checkout && git -C client fetch --filter=blob:none origin &&t/t2404-worktree-config.sh +1
-0@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' ' test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true &&
Yes, makes sense. Does this patch need it, or could this go in a separate patch?
t/t2404-worktree-config.sh test_expect_success 'config --worktree without extension' ' · line 29
Quoted 3 lines
t/t2404-worktree-config.sh +2
-1- test_cmp_config true extensions.worktreeConfig + test_cmp_config true extensions.worktreeConfig && + test_cmp_config 1 core.repositoryformatversion
This (both the existing code and the modified version) is strange: we just set the config, so why are we checking it?
[...]
t/t5500-fetch-pack.sh fetch_filter_blob_limit_zero () { · line 1002
Quoted 8 lines
t/t5500-fetch-pack.sh +0
-1@@ -999,7 +999,6 @@ fetch_filter_blob_limit_zero () { test_config -C "$SERVER" uploadpack.allowfilter 1 && git clone "$URL" client && - test_config -C client extensions.partialclone origin &&
Nice.
[...]
t/t5702-protocol-v2.sh test_expect_success 'partial fetch' ' · line 353
Quoted 10 lines
t/t5702-protocol-v2.sh +0
-1@@ -348,7 +348,6 @@ test_expect_success 'partial fetch' ' rm -rf client "$(pwd)/trace" && git init client && SERVER="file://$(pwd)/server" && - test_config -C client extensions.partialClone "$SERVER" && GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \ fetch --filter=blob:none "$SERVER" master:refs/heads/other &&
This test is a bit unusual, since it's fetching by URL instead of from a remote. Looks like it comes from
commit ba95710a3bdcb2a80495b1d93a0e482dd69905e1 Author: Jonathan Tan <[email protected]> Date: Thu May 3 16:46:56 2018 -0700 {fetch,upload}-pack: support filter in protocol v2
Jonathan, thoughts about this one? Is making it set extensions.partialClone implicitly like the change above a good change?
With the nits above and the bits Junio described addressed, this would indeed be
Thanks.
PATCH v5 1/1 fetch: allow adding a filter after initial clone
Retroactively adding filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:
git config core.repositoryFormatVersion 1 git config extensions.partialClone origin git fetch --unshallow --filter=blob:none origin
Since the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead.
Instead of bailing out immediately when no promisor is available, make the code perform a more precise check for any potential problems (extensions became special in repository version 1, while it can have any value in version 0, so upgrade from version 0 should not happen if the repository has ANY extension).
--- builtin/fetch.c | 3 --- builtin/sparse-checkout.c | 2 ++ cache.h | 1 + list-objects-filter-options.c | 3 ++- repository.h | 6 +++++ setup.c | 41 +++++++++++++++++++++++++++++--- t/t0410-partial-clone.sh | 23 ++++++++++++++++++ t/t1090-sparse-checkout-scope.sh | 1 - t/t2404-worktree-config.sh | 4 +++- t/t5500-fetch-pack.sh | 1 - t/t5702-protocol-v2.sh | 1 - 11 files changed, 75 insertions(+), 11 deletions(-)
builtin/fetch.c +0 -3
@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));
builtin/sparse-checkout.c +2 -0
@@ -249,6 +249,8 @@ static int set_config(enum sparse_checkout_mode mode) {const char *config_path; +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to enable worktreeConfig" ));if (git_config_set_gently ("extensions.worktreeConfig" ,"true" )) {error (_ ("failed to set extensions.worktreeConfig setting" ));return 1 ;
cache.h +1 -0
@@ -1042,6 +1042,7 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; +int has_extensions;char *work_tree;struct string_list unknown_extensions; };
list-objects-filter-options.c +2 -1
@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to support partial clone" ));/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
repository.h +6 -0
@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary, and -1 when upgrade is not possible. +*/ +int upgrade_repository_format (int target_version); +#endif /* REPOSITORY_H */
setup.c +38 -3
59 lines not on this page · raw message
t/t0410-partial-clone.sh +23 -0
@@ -30,6 +30,29 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +' + +test_expect_success 'convert shallow clone to partial clone must fail with any extension' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + test_cmp_config -C client 0 core.repositoryformatversion && + git -C client config extensions.partialclone origin && + test_must_fail git -C client fetch --unshallow --filter="blob:none" +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
t/t1090-sparse-checkout-scope.sh +0 -1
@@ -63,7 +63,6 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs git -C server commit -m message && test_config -C client core.sparsecheckout 1 && - test_config -C client extensions.partialclone origin && echo "!/*" >client/.git/info/sparse-checkout && echo "/a" >>client/.git/info/sparse-checkout && git -C client fetch --filter=blob:none origin &&
t/t2404-worktree-config.sh +3 -1
@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' ' test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true && - test_cmp_config true extensions.worktreeConfig + test_cmp_config true extensions.worktreeConfig && + test_cmp_config 1 core.repositoryformatversion ' test_expect_success 'config is shared as before' '
t/t5500-fetch-pack.sh +0 -1
@@ -999,7 +999,6 @@ fetch_filter_blob_limit_zero () { test_config -C "$SERVER" uploadpack.allowfilter 1 && git clone "$URL" client && - test_config -C client extensions.partialclone origin && test_commit -C "$SERVER" two &&
t/t5702-protocol-v2.sh +0 -1
@@ -348,7 +348,6 @@ test_expect_success 'partial fetch' ' rm -rf client "$(pwd)/trace" && git init client && SERVER="file://$(pwd)/server" && - test_config -C client extensions.partialClone "$SERVER" && GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \ fetch --filter=blob:none "$SERVER" master:refs/heads/other &&
Signature · 1 line
2.27.0.rc0.183.gde8f92d652-goog
Regarding t/t2404-worktree-config.sh:
Quoted 9 lines · Jonathan Nieder
Quoted 7 lines · Xin Li
t/t2404-worktree-config.sh +0
-0@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' '
Quoted 6 lines
t/t2404-worktree-config.sh test_expect_success 'config --worktree without extension' ' · line 27
Quoted 3 lines
t/t2404-worktree-config.sh +1
-0test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true &&Yes, makes sense. Does this patch need it, or could this go in a separate patch?
Yes, this patch needs setting repositoryformatversion to 1 as we would no longer recognize extensions.worktreeConfig=true on version 0 repositories.
Quoted 6 lines
t/t2404-worktree-config.sh test_expect_success 'config --worktree without extension' ' · line 29
Quoted 3 lines
t/t2404-worktree-config.sh +2
-1- test_cmp_config true extensions.worktreeConfig + test_cmp_config true extensions.worktreeConfig && + test_cmp_config 1 core.repositoryformatversionThis (both the existing code and the modified version) is strange: we just set the config, so why are we checking it?
The check was mainly to match the existing pattern (which sets extensions.worktreeConfig and immediately asserts that they were set). These assertions are not strictly necessary but are harmless, so I don't feel strongly about keeping or removing them.
Quoted 7 lines · Jonathan Nieder
Quoted 7 lines
t/t2404-worktree-config.sh +0
-0@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' '
Quoted 6 lines
t/t2404-worktree-config.sh test_expect_success 'config --worktree without extension' ' · line 27
Quoted 3 lines
t/t2404-worktree-config.sh +1
-0test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true &&Yes, makes sense. Does this patch need it, or could this go in a separate patch?
I think it is the consequence of unrelated change in this hunk:
@@ -506,9 +510,15 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
die("%s", err.buf);
}
- repository_format_precious_objects = candidate->precious_objects;
- set_repository_format_partial_clone(candidate->partial_clone);
- repository_format_worktree_config = candidate->worktree_config;
+ if (candidate->version >= 1) {
+ repository_format_precious_objects = candidate->precious_objects;
+ set_repository_format_partial_clone(candidate->partial_clone);
+ repository_format_worktree_config = candidate->worktree_config;
+ } else {
+ repository_format_precious_objects = 0;
+ set_repository_format_partial_clone(NULL);
+ repository_format_worktree_config = 0;
+ }
string_list_clear(&candidate->unknown_extensions, 0);
if (repository_format_worktree_config) {
We used to honor extensions.*, as long as the version of Git understand it, even in a repository whose version is still v0.
I am not sure if this backward incompatible change is necessary for the purpose of "allow safe upgrade of repository format version" topic, but as you hinted, it does smell like it belongs to a separate (and much larger and potentially controversial) patch.
Thanks for carefully reading.
v6 cover fetch: allow adding a filter after initial clone
This series of patch makes fetch to allow adding a filter after initial clone, useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
This version of patchset is mostly identical to the v5 draft but splitted into smaller individual patches.
Please note that the last patch is an incompatible change that would make extensions stop working for version 0 repositories. Currently, unknown extensions are silently ignored for version 0, which means the user may see undesirable result when upgraded to a new Git version.
Xin Li (4):
repository: add a helper function to perform repository format upgrade
fetch: allow adding a filter after initial clone
sparse-checkout: upgrade repository to version 1 when enabling
extension
check_repository_format_gently(): refuse extensions for old
repositories
builtin/fetch.c | 3 --- builtin/sparse-checkout.c | 2 ++ cache.h | 1 + list-objects-filter-options.c | 3 ++- repository.h | 6 +++++ setup.c | 41 +++++++++++++++++++++++++++++--- t/t0410-partial-clone.sh | 23 ++++++++++++++++++ t/t1090-sparse-checkout-scope.sh | 1 - t/t2404-worktree-config.sh | 4 +++- t/t5500-fetch-pack.sh | 1 - t/t5702-protocol-v2.sh | 1 - 11 files changed, 75 insertions(+), 11 deletions(-)
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
PATCH v6 1/4 repository: add a helper function to perform repository format upgrade
In version 1 of repository format, "extensions" gained special meaning and it is safer to avoid upgrading when there are pre-existing extensions.
Make list-objects-filter to use the helper function instead of setting repository version directly as a prerequisite of exposing the upgrade capability.
--- cache.h | 1 + list-objects-filter-options.c | 3 ++- repository.h | 6 ++++++ setup.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 38 insertions(+), 1 deletion(-)
cache.h +1 -0
@@ -1042,6 +1042,7 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; +int has_extensions;char *work_tree;struct string_list unknown_extensions; };
list-objects-filter-options.c +2 -1
@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to support partial clone" ));/* Add promisor config for the remote */ cfg_name =xstrfmt ("remote.%s.promisor" , remote);
repository.h +6 -0
@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);void prepare_repo_settings (struct repository *r); +/* +* Return 1 if upgrade repository format to target_version succeeded, +* 0 if no upgrade is necessary, and -1 when upgrade is not possible. +*/ +int upgrade_repository_format (int target_version); +#endif /* REPOSITORY_H */
setup.c +29 -0
@@ -455,6 +455,7 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (strcmp (var,"core.repositoryformatversion" ) ==0 ) data->version =git_config_int (var, value);else if (skip_prefix (var,"extensions." , &ext)) { + data->has_extensions =1 ;/* * record any known extensions here; otherwise, * we fall through to recording it as unknown, and
@@ -538,6 +539,34 @@ static int check_repository_format_gently(const char *gitdir, struct repository_return 0 ; } +int upgrade_repository_format (int target_version) +{ +struct strbuf sb =STRBUF_INIT ; +struct strbuf err =STRBUF_INIT ; +struct strbuf repo_version =STRBUF_INIT ; +struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ; + +strbuf_git_common_path (&sb, the_repository,"config" ); +read_repository_format (&repo_fmt, sb.buf); +strbuf_release (&sb); + +if (repo_fmt.version >= target_version) +return 0 ; + +if (verify_repository_format (&repo_fmt, &err) <0 || + (!repo_fmt.version && repo_fmt.has_extensions)) { +warning ("unable to upgrade repository format from %d to %d: %s" , + repo_fmt.version, target_version, err.buf); +strbuf_release (&err); +return -1 ; + } + +strbuf_addf (&repo_version,"%d" , target_version); +git_config_set ("core.repositoryformatversion" , repo_version.buf); +strbuf_release (&repo_version); +return 1 ; +} +static void init_repository_format (struct repository_format *format) {const struct repository_format fresh =REPOSITORY_FORMAT_INIT ;
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
Re: [PATCH v6 1/4] repository: add a helper function to perform repository format upgrade
Quoted 3 lines · Xin Li
In version 1 of repository format, "extensions" gained special meaning and it is safer to avoid upgrading when there are pre-existing extensions.
I am tempted to suggest s/upgrading/& from version 0/ but if we assume everybody knows there are only v0 and v1, that may be unnecessary. I dunno.
Quoted 29 lines
Make list-objects-filter to use the helper function instead of setting repository version directly as a prerequisite of exposing the upgrade capability.
Signed-off-by Xin Li <[email protected]>--- cache.h | 1 + list-objects-filter-options.c | 3 ++- repository.h | 6 ++++++ setup.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 38 insertions(+), 1 deletion(-)cache.h +1
-0@@ -1042,6 +1042,7 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; +int has_extensions;char *work_tree;struct string_list unknown_extensions; };list-objects-filter-options.c +0
-0
list-objects-filter-options.c void partial_clone_register( · line 330
Quoted 6 lines
list-objects-filter-options.c +2
-1@@ -326,7 +326,8 @@ void partial_clone_register(/* Check if it is already registered */ if (!promisor_remote_find (remote)) { -git_config_set ("core.repositoryformatversion" ,"1" ); +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to support partial clone" ));
The idea is that builtin/fetch.c calls this before the actual fetch operation happens, which sounds sensible.
The other caller of this function is in builtin/clone.c; at that point, we have set up our $GIT_DIR/config file enough to be able to write the refspec into it, so we should be able to tell what version number we wrote to the repository. Could we have written version 0 there in today's code, though (just being curious, as we should be able to upgrade it to version 1 with this code)?
Quoted 6 lines
list-objects-filter-options.c +6
-0+int upgrade_repository_format (int target_version) +{ +struct strbuf sb =STRBUF_INIT ; +struct strbuf err =STRBUF_INIT ; +struct strbuf repo_version =STRBUF_INIT ; +struct repository_format repo_fmt =REPOSITORY_FORMAT_INIT ;
list-objects-filter-options.c void partial_clone_register(
Quoted 10 lines
list-objects-filter-options.c +10
-0+ +strbuf_git_common_path (&sb, the_repository,"config" ); +read_repository_format (&repo_fmt, sb.buf); +strbuf_release (&sb); + +if (repo_fmt.version >= target_version) +return 0 ; + +if (verify_repository_format (&repo_fmt, &err) <0 || + (!repo_fmt.version && repo_fmt.has_extensions)) {
"If we do not understand the extensions, or if the original is v0 and has any extensions, we shouldn't be upgrading"---makes sense.
Quoted 15 lines
list-objects-filter-options.c +12
-0+warning ("unable to upgrade repository format from %d to %d: %s" , + repo_fmt.version, target_version, err.buf); +strbuf_release (&err); +return -1 ; + } + +strbuf_addf (&repo_version,"%d" , target_version); +git_config_set ("core.repositoryformatversion" , repo_version.buf); +strbuf_release (&repo_version); +return 1 ; +} +static void init_repository_format (struct repository_format *format) {const struct repository_format fresh =REPOSITORY_FORMAT_INIT ;
PATCH v6 2/4 fetch: allow adding a filter after initial clone
In reply to Xin LiRetroactively adding a filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:
git config core.repositoryFormatVersion 1
git config extensions.partialClone origin
git fetch --unshallow --filter=blob:none origin
Since the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead.
Note that this change does not modify the existing git behavior which recognizes setting extensions.partialClone without changing repositoryFormatVersion.
--- builtin/fetch.c | 3 --- t/t0410-partial-clone.sh | 12 ++++++++++++ t/t1090-sparse-checkout-scope.sh | 1 - t/t5500-fetch-pack.sh | 1 - t/t5702-protocol-v2.sh | 1 - 5 files changed, 12 insertions(+), 6 deletions(-)
builtin/fetch.c +0 -3
@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));
t/t0410-partial-clone.sh +12 -0
@@ -30,6 +30,18 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
t/t1090-sparse-checkout-scope.sh +0 -1
@@ -63,7 +63,6 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs git -C server commit -m message && test_config -C client core.sparsecheckout 1 && - test_config -C client extensions.partialclone origin && echo "!/*" >client/.git/info/sparse-checkout && echo "/a" >>client/.git/info/sparse-checkout && git -C client fetch --filter=blob:none origin &&
t/t5500-fetch-pack.sh +0 -1
@@ -999,7 +999,6 @@ fetch_filter_blob_limit_zero () { test_config -C "$SERVER" uploadpack.allowfilter 1 && git clone "$URL" client && - test_config -C client extensions.partialclone origin && test_commit -C "$SERVER" two &&
t/t5702-protocol-v2.sh +0 -1
@@ -348,7 +348,6 @@ test_expect_success 'partial fetch' ' rm -rf client "$(pwd)/trace" && git init client && SERVER="file://$(pwd)/server" && - test_config -C client extensions.partialClone "$SERVER" && GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \ fetch --filter=blob:none "$SERVER" master:refs/heads/other &&
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
Re: [PATCH v6 2/4] fetch: allow adding a filter after initial clone
Quoted 11 lines · Xin Li
Retroactively adding a filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch.
Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:
git config core.repositoryFormatVersion 1
git config extensions.partialClone origin
git fetch --unshallow --filter=blob:none origin
Quoted 7 lines
Since the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead.
Note that this change does not modify the existing git behavior which recognizes setting extensions.partialClone without changing repositoryFormatVersion.
Hmph, I am not sure which part of the above corresponds to the removal of "in a repository without extensions.partialClone you cannot use the --filter option" logic. With or without this patch, users can butcher $GIT_DIR/config and pretend that the repository was created with the pertial clone extension from the beginning, but this step is not about automating those tasks, it seems.
Quoted 83 lines
Signed-off-by Xin Li <[email protected]>--- builtin/fetch.c | 3 --- t/t0410-partial-clone.sh | 12 ++++++++++++ t/t1090-sparse-checkout-scope.sh | 1 - t/t5500-fetch-pack.sh | 1 - t/t5702-protocol-v2.sh | 1 - 5 files changed, 12 insertions(+), 6 deletions(-)builtin/fetch.c +0
-3@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)if (depth || deepen_since || deepen_not.nr) deepen =1 ; -if (filter_options.choice && !has_promisor_remote ()) -die ("--filter can only be used when extensions.partialClone is set" ); -if (all) {if (argc ==1 )die (_ ("fetch --all does not take a repository argument" ));t/t0410-partial-clone.sh +12
-0@@ -30,6 +30,18 @@ test_expect_success 'extensions.partialclone without filter' ' git -C client fetch origin ' +test_expect_success 'convert shallow clone to partial clone' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + git -C client fetch --unshallow --filter="blob:none" && + test_cmp_config -C client true remote.origin.promisor && + test_cmp_config -C client blob:none remote.origin.partialclonefilter && + test_cmp_config -C client 1 core.repositoryformatversion +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&t/t1090-sparse-checkout-scope.sh +0
-1@@ -63,7 +63,6 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs git -C server commit -m message && test_config -C client core.sparsecheckout 1 && - test_config -C client extensions.partialclone origin && echo "!/*" >client/.git/info/sparse-checkout && echo "/a" >>client/.git/info/sparse-checkout && git -C client fetch --filter=blob:none origin &&t/t5500-fetch-pack.sh +0
-1@@ -999,7 +999,6 @@ fetch_filter_blob_limit_zero () { test_config -C "$SERVER" uploadpack.allowfilter 1 && git clone "$URL" client && - test_config -C client extensions.partialclone origin && test_commit -C "$SERVER" two &&t/t5702-protocol-v2.sh +0
-1@@ -348,7 +348,6 @@ test_expect_success 'partial fetch' ' rm -rf client "$(pwd)/trace" && git init client && SERVER="file://$(pwd)/server" && - test_config -C client extensions.partialClone "$SERVER" && GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \ fetch --filter=blob:none "$SERVER" master:refs/heads/other &&
PATCH v6 3/4 sparse-checkout: upgrade repository to version 1 when enabling extension
In reply to Xin LiThe 'extensions' configuration variable gets special meaning in the new repository version, so when enabling the extension we should upgrade the repository to version 1.
--- builtin/sparse-checkout.c | 2 ++ t/t2404-worktree-config.sh | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-)
builtin/sparse-checkout.c +2 -0
@@ -249,6 +249,8 @@ static int set_config(enum sparse_checkout_mode mode) {const char *config_path; +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to enable worktreeConfig" ));if (git_config_set_gently ("extensions.worktreeConfig" ,"true" )) {error (_ ("failed to set extensions.worktreeConfig setting" ));return 1 ;
t/t2404-worktree-config.sh +3 -1
@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' ' test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true && - test_cmp_config true extensions.worktreeConfig + test_cmp_config true extensions.worktreeConfig && + test_cmp_config 1 core.repositoryformatversion ' test_expect_success 'config is shared as before' '
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
Re: [PATCH v6 3/4] sparse-checkout: upgrade repository to version 1 when enabling extension
Quoted 9 lines · Xin Li
The 'extensions' configuration variable gets special meaning in the new repository version, so when enabling the extension we should upgrade the repository to version 1.
Signed-off-by Xin Li <[email protected]>--- builtin/sparse-checkout.c | 2 ++ t/t2404-worktree-config.sh | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-)
The other place that "extensions.*" is referred to is in builtin/init-db.c::initialize_repository_version() and it already makes sure that extensions.objectformat is set only in a repository whose verseion is GIT_REPO_VERSION_READ (which is 1---by the way I suspect we want a better name than _READ for the symbol---it wants to use the highest version supported by this binary), so with this patch we covered everything, hopefully.
Quoted 29 lines
builtin/sparse-checkout.c +2
-0@@ -249,6 +249,8 @@ static int set_config(enum sparse_checkout_mode mode) {const char *config_path; +if (upgrade_repository_format (1 ) <0 ) +die (_ ("unable to upgrade repository format to enable worktreeConfig" ));if (git_config_set_gently ("extensions.worktreeConfig" ,"true" )) {error (_ ("failed to set extensions.worktreeConfig setting" ));return 1 ;t/t2404-worktree-config.sh +3
-1@@ -23,8 +23,10 @@ test_expect_success 'config --worktree without extension' ' ' test_expect_success 'enable worktreeConfig extension' ' + git config core.repositoryformatversion 1 && git config extensions.worktreeConfig true && - test_cmp_config true extensions.worktreeConfig + test_cmp_config true extensions.worktreeConfig && + test_cmp_config 1 core.repositoryformatversion ' test_expect_success 'config is shared as before' '
PATCH v6 4/4 check_repository_format_gently(): refuse extensions for old repositories
In reply to Xin LiPreviously, extensions were recognized regardless of repository format version. If the user sets an undefined "extensions" value on a repository of version 0 and that value is used by a future git version, they might get an undesired result.
Because all extensions now also upgrade repository versions, tightening the check would help avoid this for future extensions.
--- setup.c | 12 +++++++++--- t/t0410-partial-clone.sh | 11 +++++++++++ 2 files changed, 20 insertions(+), 3 deletions(-)
setup.c +9 -3
@@ -507,9 +507,15 @@ static int check_repository_format_gently(const char *gitdir, struct repository_die ("%s" , err.buf); } - repository_format_precious_objects = candidate->precious_objects; -set_repository_format_partial_clone (candidate->partial_clone); - repository_format_worktree_config = candidate->worktree_config; +if (candidate->version >=1 ) { + repository_format_precious_objects = candidate->precious_objects; +set_repository_format_partial_clone (candidate->partial_clone); + repository_format_worktree_config = candidate->worktree_config; + }else { + repository_format_precious_objects =0 ; +set_repository_format_partial_clone (NULL ); + repository_format_worktree_config =0 ; + }string_list_clear (&candidate->unknown_extensions,0 );if (repository_format_worktree_config) {
t/t0410-partial-clone.sh +11 -0
@@ -42,6 +42,17 @@ test_expect_success 'convert shallow clone to partial clone' ' test_cmp_config -C client 1 core.repositoryformatversion ' +test_expect_success 'convert shallow clone to partial clone must fail with any extension' ' + rm -fr server client && + test_create_repo server && + test_commit -C server my_commit 1 && + test_commit -C server my_commit2 1 && + git clone --depth=1 "file://$(pwd)/server" client && + test_cmp_config -C client 0 core.repositoryformatversion && + git -C client config extensions.partialclone origin && + test_must_fail git -C client fetch --unshallow --filter="blob:none" +' + test_expect_success 'missing reflog object, but promised by a commit, passes fsck' ' rm -rf repo && test_create_repo repo &&
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
Re: [PATCH v6 4/4] check_repository_format_gently(): refuse extensions for old repositories
Quoted 5 lines · Xin Li
Previously, extensions were recognized regardless of repository format version. If the user sets an undefined "extensions" value on a repository of version 0 and that value is used by a future git version, they might get an undesired result.
Quoted 8 lines
Because all extensions now also upgrade repository versions, tightening the check would help avoid this for future extensions.
Signed-off-by Xin Li <[email protected]>--- setup.c | 12 +++++++++--- t/t0410-partial-clone.sh | 11 +++++++++++ 2 files changed, 20 insertions(+), 3 deletions(-)
Thanks.
People, I think this is the only bit that is potentially controversial with the risk of breaking existing set-up. Its potentially negative effect can be seen in the patch to test in the previous step.
test_expect_success 'enable worktreeConfig extension' '
+ git config core.repositoryformatversion 1 &&
git config extensions.worktreeConfig true &&It used to be that, by mistake, you could just say
git config extensions.worktreeConfig true
in an existing repository (with extra worktrees) and expect that alone is enough to let you use per-worktree configuration files.
With this patch, that is no longer true, because the extensions.* will not take effect unless you upgrade core.repositoryformatversion to 1.
It is the right thing to do in the longer term (we could even consider it a bugfix), and the workaround is simple enough.
Unless the user is somehow using a configuration variable whose name begins with "extensions." for whatever random purpose, that is. If the user's setup uses extensions.bar, there is no workaround possible other than renaming the variable to put it outside extensions.* hierarchy, and update whatever tool that is using that variable to use the renamed variable.
Comments?