setup: warn if extensions exist on old format
PATCH 1/1 setup: warn if extensions exist on old format
From: Derrick Stolee <[email protected]>
Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05), Git was honoring configured extensions, even if core.repositoryFormatVersion was 0 (or unset). This was incorrect, and is now fixed.
The issue now is that users who relied on that previously bad behavior will upgrade to the next version of Git and suddently be in a bad situation. In particular, users of the 'git sparse-checkout' builting will rely on the extensions.worktreeConfig for the core.sparseCheckout and core.sparseCheckoutCone config options. Without that extension, these users will suddenly have repositories stop acting like a sparse repo.
What is worse is that a user will be confronted with the following error if they try to run 'git sparse-checkout init' again:
warning: unable to upgrade repository format from 0 to 1
This is because the logic in 14c7fa269e4 refuses to upgrae repos when the version is unset but extensions exist.
While it is important to correct this improper behavior, create a warning so users in this situation can correct themselves without too much guesswork. By creating a warning in check_repository_format_gently(), we can alert the user if they have a ocnfigured extension but not a configured repository version.
---
setup: warn if extensions exist on old format
This is based on xl/upgrade-repo-format.
Can this be considered for v2.28.0-rc1? I think this is an important shift in behavior that will disrupt many users if they upgrade without it!
If not this warning or change, then how else can we help users who are in this situation? How can we save those who adopted the sparse-checkout builtin in recent versions from terrible post-upgrade headaches?
Thanks, -Stolee
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-674%2Fderrickstolee%2Fsparse-checkout-warning-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-674/derrickstolee/sparse-checkout-warning-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/674
setup.c | 5 +++++ t/t1091-sparse-checkout-builtin.sh | 9 +++++++++ 2 files changed, 14 insertions(+)
setup.c +5 -0
@@ -542,6 +542,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } } +if (candidate->version ==0 && candidate->has_extensions) { +warning (_ ("some extensions are enabled, but core.repositoryFormatVersion=0" )); +warning (_ ("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'" )); + } +return 0 ; }
t/t1091-sparse-checkout-builtin.sh +9 -0
@@ -68,6 +68,15 @@ test_expect_success 'git sparse-checkout init' ' check_files repo a ' +test_expect_success 'warning about core.repositoryFormatVersion' ' + test_when_finished git -C repo config core.repositoryFormatVersion 1 && + git -C repo status 2>err && + test_must_be_empty err && + git -C repo config --local core.repositoryFormatVersion 0 && + git -C repo status 2>err && + test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err +' + test_expect_success 'git sparse-checkout list after init' ' git -C repo sparse-checkout list >actual && cat >expect <<-\EOF &&
base-commit: 14c7fa269e42df4133edd9ae7763b678ed6594cd
Signature · 1 line
gitgitgadget
Quoted 15 lines · Derrick Stolee via GitGitGadget
From: Derrick Stolee <[email protected]>
Prior to
14c7fa269e4(check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05), Git was honoring configured extensions, even if core.repositoryFormatVersion was 0 (or unset). This was incorrect, and is now fixed.The issue now is that users who relied on that previously bad behavior will upgrade to the next version of Git and suddently be in a bad situation. In particular, users of the 'git sparse-checkout' builting will rely on the extensions.worktreeConfig for the core.sparseCheckout and core.sparseCheckoutCone config options. Without that extension, these users will suddenly have repositories stop acting like a sparse repo.
Quoted 6 lines
What is worse is that a user will be confronted with the following error if they try to run 'git sparse-checkout init' again:
warning: unable to upgrade repository format from 0 to 1
This is because the logic in
14c7fa269e4refuses to upgrae repos when
s/upgrae/upgrade
Quoted 1 line
the version is unset but extensions exist.
I'm not sure that I want to get into a discussion about whether or not this logic is right while -rc0 is already out, but it seems like 14c7fa269e4 could be tweaked to be a little more tolerant of this case.
On the other hand, I think that the approach here is perfectly sensible, absent of a more invasive change to the logic in 14c7fa269e4.
Quoted 5 lines
While it is important to correct this improper behavior, create a warning so users in this situation can correct themselves without too much guesswork. By creating a warning in
check_repository_format_gently(), we can alert the user if they have a ocnfigured extension but not a configured repository version.
s/ocnfigured/configured
Quoted 9 lines
Signed-off-by Derrick Stolee <[email protected]>---
setup: warn if extensions exist on old formatThis is based on xl/upgrade-repo-format.
Can this be considered for v2.28.0-rc1? I think this is an important shift in behavior that will disrupt many users if they upgrade without it!
I would be happy to see something like this in -rc1, unless Junio has reservations about making this change at this point in the release cycle (I do not have any such concerns).
Quoted 35 lines
If not this warning or change, then how else can we help users who are in this situation? How can we save those who adopted the sparse-checkout builtin in recent versions from terrible post-upgrade headaches?
Thanks, -Stolee
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-674%2Fderrickstolee%2Fsparse-checkout-warning-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-674/derrickstolee/sparse-checkout-warning-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/674setup.c | 5 +++++ t/t1091-sparse-checkout-builtin.sh | 9 +++++++++ 2 files changed, 14 insertions(+)setup.c +5
-0@@ -542,6 +542,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } } +if (candidate->version ==0 && candidate->has_extensions) { +warning (_ ("some extensions are enabled, but core.repositoryFormatVersion=0" )); +warning (_ ("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'" )); + } +return 0 ; }t/t1091-sparse-checkout-builtin.sh +0
-0
t/t1091-sparse-checkout-builtin.sh test_expect_success 'git sparse-checkout init' ' · line 75
Quoted 8 lines
t/t1091-sparse-checkout-builtin.sh +5
-0@@ -68,6 +68,15 @@ test_expect_success 'git sparse-checkout init' ' check_files repo a ' +test_expect_success 'warning about core.repositoryFormatVersion' ' + test_when_finished git -C repo config core.repositoryFormatVersion 1 && + git -C repo status 2>err && + test_must_be_empty err && + git -C repo config --local core.repositoryFormatVersion 0 &&
I don't think it's that difficult to see that this patch is correct, but it might be worth testing this for the case that 'core.repositoryFormatVersion' is unset, too.
Quoted 11 lines
t/t1091-sparse-checkout-builtin.sh +4
-0+ git -C repo status 2>err && + test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err +' + test_expect_success 'git sparse-checkout list after init' ' git -C repo sparse-checkout list >actual && cat >expect <<-\EOF &&base-commit:
14c7fa269e42df4133edd9ae7763b678ed6594cd--
gitgitgadget
Thanks,
Taylor
Quoted 2 lines · Taylor Blau
s/upgrae/upgrade
s/ocnfigured/configured
Oh man, what was going on with me when I was typing this message.
Thanks for the thorough inspection.
Quoted 24 lines
Quoted 24 lines
setup.c +5
-0@@ -542,6 +542,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } } +if (candidate->version ==0 && candidate->has_extensions) { +warning (_ ("some extensions are enabled, but core.repositoryFormatVersion=0" )); +warning (_ ("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'" )); + } +return 0 ; }t/t1091-sparse-checkout-builtin.sh +0
-0@@ -68,6 +68,15 @@ test_expect_success 'git sparse-checkout init' ' check_files repo a '
Quoted 9 lines
t/t1091-sparse-checkout-builtin.sh test_expect_success 'git sparse-checkout init' ' · line 75
Quoted 5 lines
t/t1091-sparse-checkout-builtin.sh +5
-0+test_expect_success 'warning about core.repositoryFormatVersion' ' + test_when_finished git -C repo config core.repositoryFormatVersion 1 && + git -C repo status 2>err && + test_must_be_empty err && + git -C repo config --local core.repositoryFormatVersion 0 &&I don't think it's that difficult to see that this patch is correct, but it might be worth testing this for the case that 'core.repositoryFormatVersion' is unset, too.
You were absolutely right to check this, since this diff causes the test to fail:
t/t1091-sparse-checkout-builtin.sh +3 -0
@@ -74,6 +74,9 @@ test_expect_success 'warning about core.repositoryFormatVersion' ' test_must_be_empty err && git -C repo config --local core.repositoryFormatVersion 0 && git -C repo status 2>err && + test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err && + git -C repo config --local --unset core.repositoryFormatVersion 0 && + git -C repo status 2>err && test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err '
I'll investigate why the "unset" case is different than the "0" case.
Hopefully the "should we do this?" question can continue being discussed while I work on a v2.
Thanks,
-Stolee
Quoted 7 lines · Derrick Stolee via GitGitGadget
From: Derrick Stolee <[email protected]>
Prior to
14c7fa269e4(check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05), Git was honoring configured extensions, even if core.repositoryFormatVersion was 0 (or unset). This was incorrect, and is now fixed.
Quoted 7 lines
The issue now is that users who relied on that previously bad behavior will upgrade to the next version of Git and suddently be in a bad situation. In particular, users of the 'git sparse-checkout' builting will rely on the extensions.worktreeConfig for the core.sparseCheckout and core.sparseCheckoutCone config options. Without that extension, these users will suddenly have repositories stop acting like a sparse repo.
s/builting/command/ perhaps???
Quoted 6 lines
What is worse is that a user will be confronted with the following error if they try to run 'git sparse-checkout init' again:
warning: unable to upgrade repository format from 0 to 1
Quoted 8 lines
This is because the logic in
14c7fa269e4refuses to upgrae repos when the version is unset but extensions exist.While it is important to correct this improper behavior, create a warning so users in this situation can correct themselves without too much guesswork. By creating a warning in
check_repository_format_gently(), we can alert the user if they have a ocnfigured extension but not a configured repository version.
s/ocn/con/
Quoted 2 lines
Signed-off-by Derrick Stolee <[email protected]>---
Thanks for a thoughtful analysis of the situation and coming up with a plan to remedy.
setup.c static int check_repository_format_gently(const char *gitdir, struct repository_ · line 548
Quoted 4 lines
setup.c +4
-0+if (candidate->version ==0 && candidate->has_extensions) { +warning (_ ("some extensions are enabled, but core.repositoryFormatVersion=0" )); +warning (_ ("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'" )); + }
That reads well.
An alternative may be to grandfather some extensions that were enabled by git by mistake without updating the format version, and we update the repository even if the repository has extensions that should not exist, but those offending extensions are limited only to those that we decide to special case. That would make the end-user experience even smoother.
Is extenions.worktreeCOnfig the only one that needs this escape hatch?
Quoted 9 lines · Junio C Hamano
An alternative may be to grandfather some extensions that were enabled by git by mistake without updating the format version, and we update the repository even if the repository has extensions that should not exist, but those offending extensions are limited only to those that we decide to special case. That would make the end-user experience even smoother.
Is extenions.worktreeCOnfig the only one that needs this escape hatch?
Assuming that worktreeconfig is the only thing, the change may look like this. With this change, we might want to drop the new warning in hunk ll.542- to avoid encouraging people to muck with their repository with random configuration variables that happen to share extensions.* prefix with us.
cache.h | 2 +- setup.c | 17 +++++++++++++---- 2 files changed, 14 insertions(+), 5 deletions(-)
cache.h +1 -1
@@ -1042,8 +1042,8 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; -int has_extensions;char *work_tree; +int has_unallowed_extensions;struct string_list unknown_extensions; };
setup.c +13 -4
@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (strcmp (var,"core.repositoryformatversion" ) ==0 ) data->version =git_config_int (var, value);else if (skip_prefix (var,"extensions." , &ext)) { - data->has_extensions =1 ;/* * record any known extensions here; otherwise, * we fall through to recording it as unknown, and * check_repository_format will complain */ +int is_unallowed_extension =1 ; +if (!strcmp (ext,"noop" )) ;else if (!strcmp (ext,"preciousobjects" ))
@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (!value)return config_error_nonbool (var); data->partial_clone =xstrdup (value); - }else if (!strcmp (ext,"worktreeconfig" )) + }else if (!strcmp (ext,"worktreeconfig" )) { data->worktree_config =git_config_bool (var, value); -else + is_unallowed_extension =0 ; + }else string_list_append (&data->unknown_extensions, ext); + + data->has_unallowed_extensions |= is_unallowed_extension; }return read_worktree_config (var, value, vdata);
@@ -542,6 +546,11 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } } +if (candidate->version ==0 && candidate->has_unallowed_extensions) { +warning (_ ("some extensions are enabled, but core.repositoryFormatVersion=0" )); +warning (_ ("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'" )); + } +return 0 ; }
@@ -560,7 +569,7 @@ int upgrade_repository_format(int target_version)return 0 ;if (verify_repository_format (&repo_fmt, &err) <0 || - (!repo_fmt.version && repo_fmt.has_extensions)) { + (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {warning ("unable to upgrade repository format from %d to %d: %s" , repo_fmt.version, target_version, err.buf);strbuf_release (&err);
PATCH 1/1 setup: tweak upgrade policy to grandfather worktreeconfig
The "fail and warn" approach introduced in the previous step may be with smaller impact to the codebase, but
- it requires the end-user to read, understand and execute the manual upgrade
- it encourages to follow the same procedure blindly, making the protection even less useful
Let's instead keep failing hard without teaching how to bypass the repository protection, but allow upgrading even when only the worktreeconfig extension exists in an old repository, which is likely to be set by a broke version of Git that did not update the repository version when setting the extension.
---
* This time with a log message to explain and justify the change, with a matching tweak to the test script, designed to be applied on top, but feel free to squash it in if you agree with me that we do not need two separate commits for this.
Thanks.
cache.h | 2 +- setup.c | 17 ++++++++--------- t/t1091-sparse-checkout-builtin.sh | 9 --------- 3 files changed, 9 insertions(+), 19 deletions(-)
cache.h +1 -1
@@ -1042,8 +1042,8 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; -int has_extensions;char *work_tree; +int has_unallowed_extensions;struct string_list unknown_extensions; };
setup.c +8 -9
@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (strcmp (var,"core.repositoryformatversion" ) ==0 ) data->version =git_config_int (var, value);else if (skip_prefix (var,"extensions." , &ext)) { - data->has_extensions =1 ;/* * record any known extensions here; otherwise, * we fall through to recording it as unknown, and * check_repository_format will complain */ +int is_unallowed_extension =1 ; +if (!strcmp (ext,"noop" )) ;else if (!strcmp (ext,"preciousobjects" ))
@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (!value)return config_error_nonbool (var); data->partial_clone =xstrdup (value); - }else if (!strcmp (ext,"worktreeconfig" )) + }else if (!strcmp (ext,"worktreeconfig" )) { data->worktree_config =git_config_bool (var, value); -else + is_unallowed_extension =0 ; + }else string_list_append (&data->unknown_extensions, ext); + + data->has_unallowed_extensions |= is_unallowed_extension; }return read_worktree_config (var, value, vdata);
@@ -542,11 +546,6 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } } -if (candidate->version ==0 && candidate->has_extensions) { -warning (_ ("some extensions are enabled, but core.repositoryFormatVersion=0" )); -warning (_ ("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'" )); - } -return 0 ; }
@@ -565,7 +564,7 @@ int upgrade_repository_format(int target_version)return 0 ;if (verify_repository_format (&repo_fmt, &err) <0 || - (!repo_fmt.version && repo_fmt.has_extensions)) { + (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {warning ("unable to upgrade repository format from %d to %d: %s" , repo_fmt.version, target_version, err.buf);strbuf_release (&err);
t/t1091-sparse-checkout-builtin.sh +0 -9
@@ -68,15 +68,6 @@ test_expect_success 'git sparse-checkout init' ' check_files repo a ' -test_expect_success 'warning about core.repositoryFormatVersion' ' - test_when_finished git -C repo config core.repositoryFormatVersion 1 && - git -C repo status 2>err && - test_must_be_empty err && - git -C repo config --local core.repositoryFormatVersion 0 && - git -C repo status 2>err && - test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err -' - test_expect_success 'git sparse-checkout list after init' ' git -C repo sparse-checkout list >actual && cat >expect <<-\EOF &&
Signature · 1 line
2.28.0-rc0
Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
Quoted 6 lines · Junio C Hamano
The "fail and warn" approach introduced in the previous step may be with smaller impact to the codebase, but
- it requires the end-user to read, understand and execute the manual upgrade
Quoted 8 lines
- it encourages to follow the same procedure blindly, making the protection even less useful
Let's instead keep failing hard without teaching how to bypass the repository protection, but allow upgrading even when only the worktreeconfig extension exists in an old repository, which is likely to be set by a broke version of Git that did not update the repository version when setting the extension.
This is a more subtle way to handle the case. In fact, it silently makes extensions.worktreeConfig work as it did in 2.27, which means users will not have any troubles after upgrading.
I also like that you are actually fixing the case where a user in the bad state _can_ get out using "git sparse-checkout init".
This can be verified by adding this test:
test_expect_success 'git sparse-checkout works if repository format is wrong' '
test_when_finished git -C repo config core.repositoryFormatVersion 1 && git -C repo sparse-checkout init && git -C repo config core.repositoryFormatVersion >actual && echo 1 >expect && test_cmp expect actual && git -C repo config core.repositoryFormatVersion 0 && git -C repo sparse-checkout init && git -C repo config core.repositoryFormatVersion >actual && test_cmp expect actual
'
Quoted 7 lines
Signed-off-by Junio C Hamano <[email protected]>---
* This time with a log message to explain and justify the change, with a matching tweak to the test script, designed to be applied on top, but feel free to squash it in if you agree with me that we do not need two separate commits for this.
Since this commit removes all evidence of the previous one, I would recommend just squashing them together.
Thanks for your fast feedback!
-Stolee
Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
Quoted 20 lines · Derrick Stolee
I also like that you are actually fixing the case where a user in the bad state _can_ get out using "git sparse-checkout init".
This can be verified by adding this test:
test_expect_success'git sparse-checkout works if repository format is wrong' 'test_when_finished git -C repo config core.repositoryFormatVersion 1 && git -C repo sparse-checkout init && git -C repo config core.repositoryFormatVersion >actual && echo 1 >expect && test_cmp expect actual && git -C repo config core.repositoryFormatVersion 0 && git -C repo sparse-checkout init && git -C repo config core.repositoryFormatVersion >actual && test_cmp expect actual'
Quoted 3 lines
Signed-off-by Junio C Hamano <[email protected]>---
Quoted 7 lines
Quoted 4 lines
* This time with a log message to explain and justify the change, with a matching tweak to the test script, designed to be applied on top, but feel free to squash it in if you agree with me that we do not need two separate commits for this.
Since this commit removes all evidence of the previous one, I would recommend just squashing them together.
Alright, then care to do the honors ;-)? Let's make sure we have it in -rc1 to avoid nasty "regression" in the upcoming release.
Thanks for raising the issue and exploring the solution space.
Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
In reply to Derrick StoleeHi,
Quoted 17 lines · Derrick Stolee
Quoted 6 lines · Junio C Hamano
The "fail and warn" approach introduced in the previous step may be with smaller impact to the codebase, but
- it requires the end-user to read, understand and execute the manual upgrade
Quoted 8 lines
- it encourages to follow the same procedure blindly, making the protection even less useful
Let's instead keep failing hard without teaching how to bypass the repository protection, but allow upgrading even when only the worktreeconfig extension exists in an old repository, which is likely to be set by a broke version of Git that did not update the repository version when setting the extension.
Quoted 3 lines
This is a more subtle way to handle the case. In fact, it silently makes extensions.worktreeConfig work as it did in 2.27, which means users will not have any troubles after upgrading.
I'd like to propose a different endgame:
Instead of looking at extensions.* settings one by one to see how various implementations handled them with repositoryFormatVersion=0, what if we treat repositoryFormatVersion=0 as a synonym of version=1?
That is:
1. in new repositories, set repositoryFormatVersion = 1, since (1) Git versions in the wild should all already know how to handle it and
(2) as we've learned, other Git implementations need to understand some of extensions.* anyway
2. whenever setting any extensions.*, automatically upgrade to repositoryFormatVersion = 1
3. when in an existing repository with extensions.* set and repositoryFormatVersion = 0, act as though repositoryFormatVersion = 1
4. document this new behavior with repositoryFormatVersion = 0 in Documentation/technical/repository-version.txt
This way, the result is simpler than where we started.
Unfortunately, we are not even that consistent about what to do with extensions.* settings when repositoryFormatVersion = 0 today. So we'll have to exercise some care and analyze them one by one to make sure this is safe (and if it isn't, at *that* point we'd come up with a more complex variant on (2) and (3) above).
It's too late to go that far for 2.28. It would be tempting to try a simple revert of 14c7fa269e4 (check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05) to get back to tried and true behavior but that does not do enough --- it still produces an error when trying to upgrade repository format when any extensions are set. So how about such a revert plus Junio's patch plus the analogous change to Junio's patch for
extensions.preciousObjects
extensions.partialClone
?
Thanks,
Jonathan
Re: [PATCH] setup: tweak upgrade policy to grandfather worktreeconfig
Quoted 10 lines · Jonathan Nieder
It's too late to go that far for 2.28. It would be tempting to try a simple revert of
14c7fa269e4(check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05) to get back to tried and true behavior but that does not do enough --- it still produces an error when trying to upgrade repository format when any extensions are set. So how about such a revert plus Junio's patch plus the analogous change to Junio's patch forextensions.preciousObjects
extensions.partialClone
My illustration patch was done "assuming that worktreeconfig is the only thing we wrote by mistake without updating the format version", and if these two also share the same problem, I obviously is 100% fine with covering these other ones with the same approach.
I like your "v0 and v1 are the same, but the repository is declared to be corrupt if the extentions that are not known by today's code is found in v0 repository", by the way. Assuming that the two you listed above plus worktreeconfig are the only ones known by today's code, that is. We seem to also know about "noop", so shouldn't it also be grandfathered in?
v2 cover setup: warn if extensions exist on old format
In reply to Derrick Stolee via GitGitGadgetThis is based on xl/upgrade-repo-format.
Thanks, Taylor and Junio for jumping in with helpful review.
Updates in v2:
1. My initial patch is essentially dropped in its entirety, with Junio's patch here instead. I added an extra test and kept some of my commit message.
2. A second patch has joined the fray, hopefully answering the concerned raise by Johannes [1].
[1]
https://lore.kernel.org/git/[email protected]/
Thanks, -Stolee
Derrick Stolee (2):
setup: tweak upgrade policy to grandfather worktreeconfig
config: provide extra detail about worktree config
builtin/config.c | 5 +++-- cache.h | 2 +- setup.c | 12 ++++++++---- t/t1091-sparse-checkout-builtin.sh | 12 ++++++++++++ t/t2404-worktree-config.sh | 12 ++++++++++++ 5 files changed, 36 insertions(+), 7 deletions(-)
base-commit: 14c7fa269e42df4133edd9ae7763b678ed6594cd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-674%2Fderrickstolee%2Fsparse-checkout-warning-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-674/derrickstolee/sparse-checkout-warning-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/674
Range-diff vs v1:
1: 113cb7cda2 ! 1: 1b26d9710a setup: warn if extensions exist on old format
@@ Metadata
Author: Derrick Stolee <[email protected]>
## Commit message ##
- setup: warn if extensions exist on old format
+ setup: tweak upgrade policy to grandfather worktreeconfig
Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05), Git was honoring configured
@@ Commit message
The issue now is that users who relied on that previously bad behavior will upgrade to the next version of Git and suddently be in a bad
- situation. In particular, users of the 'git sparse-checkout' builting
+ situation. In particular, users of the 'git sparse-checkout' command
will rely on the extensions.worktreeConfig for the core.sparseCheckout and core.sparseCheckoutCone config options. Without that extension, these users will suddenly have repositories stop acting like a sparse
@@ Commit message
This is because the logic in 14c7fa269e4 refuses to upgrae repos when the version is unset but extensions exist.
- While it is important to correct this improper behavior, create a
- warning so users in this situation can correct themselves without too
- much guesswork. By creating a warning in
- check_repository_format_gently(), we can alert the user if they have a
- ocnfigured extension but not a configured repository version.
+ One possible way to alert a user of this issue is to warn them when Git
+ notices an extension exists, but core.repositoryFormatVersion is not a
+ correct value. However,
+ - it requires the end-user to read, understand and execute the
+ manual upgrade
+
+ - it encourages to follow the same procedure blindly, making the
+ protection even less useful
+
+ Let's instead keep failing hard without teaching how to bypass the
+ repository protection, but allow upgrading even when only the
+ worktreeconfig extension exists in an old repository, which is
+ likely to be set by a broke version of Git that did not update the
+ repository version when setting the extension.
+
+ This change of behavior is made visible by testing how 'git
+ sparse-checkout init' behaves to upgrade the repository format version
+ even if the extension.worktreeConfig is already set. This would
+ previously fail without a clear way forward.
+
+ Signed-off-by: Junio C Hamano <[email protected]>
+ ## cache.h ##
+@@ cache.h: struct repository_format {
+ int worktree_config;
+ int is_bare;
+ int hash_algo;
+- int has_extensions;
+ char *work_tree;
++ int has_unallowed_extensions;
+ struct string_list unknown_extensions;
+ };
+
+
## setup.c ##
-@@ setup.c: static int check_repository_format_gently(const char *gitdir, struct repository_
- }
+@@ setup.c: static int check_repo_format(const char *var, const char *value, void *vdata)
+ if (strcmp(var, "core.repositoryformatversion") == 0)
+ data->version = git_config_int(var, value);
+ else if (skip_prefix(var, "extensions.", &ext)) {
+- data->has_extensions = 1;
+ /*
+ * record any known extensions here; otherwise,
+ * we fall through to recording it as unknown, and
+ * check_repository_format will complain
+ */
++ int is_unallowed_extension = 1;
++
+ if (!strcmp(ext, "noop"))
+ ;
+ else if (!strcmp(ext, "preciousobjects"))
+@@ setup.c: static int check_repo_format(const char *var, const char *value, void *vdata)
+ if (!value)
+ return config_error_nonbool(var);
+ data->partial_clone = xstrdup(value);
+- } else if (!strcmp(ext, "worktreeconfig"))
++ } else if (!strcmp(ext, "worktreeconfig")) {
+ data->worktree_config = git_config_bool(var, value);
+- else
++ is_unallowed_extension = 0;
++ } else
+ string_list_append(&data->unknown_extensions, ext);
++
++ data->has_unallowed_extensions |= is_unallowed_extension;
}
-+ if (candidate->version == 0 && candidate->has_extensions) {
-+ warning(_("some extensions are enabled, but core.repositoryFormatVersion=0"));
-+ warning(_("if you intended to use extensions, run 'git config core.repositoryFormatVersion 1'"));
-+ }
-+
- return 0;
- }
+ return read_worktree_config(var, value, vdata);
+@@ setup.c: int upgrade_repository_format(int target_version)
+ return 0;
+ if (verify_repository_format(&repo_fmt, &err) < 0 ||
+- (!repo_fmt.version && repo_fmt.has_extensions)) {
++ (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {
+ warning("unable to upgrade repository format from %d to %d: %s",
+ repo_fmt.version, target_version, err.buf);
+ strbuf_release(&err);
## t/t1091-sparse-checkout-builtin.sh ##
@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'git sparse-checkout init' '
check_files repo a
'
-+test_expect_success 'warning about core.repositoryFormatVersion' ' ++test_expect_success 'git sparse-checkout works if repository format is wrong' '
+ test_when_finished git -C repo config core.repositoryFormatVersion 1 &&
-+ git -C repo status 2>err &&
-+ test_must_be_empty err &&
-+ git -C repo config --local core.repositoryFormatVersion 0 &&
-+ git -C repo status 2>err &&
-+ test_i18ngrep "some extensions are enabled, but core.repositoryFormatVersion=0" err
++ git -C repo config --unset core.repositoryFormatVersion &&
++ git -C repo sparse-checkout init &&
++ git -C repo config core.repositoryFormatVersion >actual &&
++ echo 1 >expect &&
++ git -C repo config core.repositoryFormatVersion 0 &&
++ git -C repo sparse-checkout init &&
++ git -C repo config core.repositoryFormatVersion >actual &&
++ test_cmp expect actual
+'
+
test_expect_success 'git sparse-checkout list after init' '
-: ---------- > 2: e11e973c6f config: provide extra detail about worktree config
Signature · 1 line
gitgitgadget
PATCH v2 1/2 setup: tweak upgrade policy to grandfather worktreeconfig
From: Derrick Stolee <[email protected]>
Prior to 14c7fa269e4 (check_repository_format_gently(): refuse extensions for old repositories, 2020-06-05), Git was honoring configured extensions, even if core.repositoryFormatVersion was 0 (or unset). This was incorrect, and is now fixed.
The issue now is that users who relied on that previously bad behavior will upgrade to the next version of Git and suddently be in a bad situation. In particular, users of the 'git sparse-checkout' command will rely on the extensions.worktreeConfig for the core.sparseCheckout and core.sparseCheckoutCone config options. Without that extension, these users will suddenly have repositories stop acting like a sparse repo.
What is worse is that a user will be confronted with the following error if they try to run 'git sparse-checkout init' again:
warning: unable to upgrade repository format from 0 to 1
This is because the logic in 14c7fa269e4 refuses to upgrae repos when the version is unset but extensions exist.
One possible way to alert a user of this issue is to warn them when Git notices an extension exists, but core.repositoryFormatVersion is not a correct value. However,
- it requires the end-user to read, understand and execute the manual upgrade
- it encourages to follow the same procedure blindly, making the protection even less useful
Let's instead keep failing hard without teaching how to bypass the repository protection, but allow upgrading even when only the worktreeconfig extension exists in an old repository, which is likely to be set by a broke version of Git that did not update the repository version when setting the extension.
This change of behavior is made visible by testing how 'git sparse-checkout init' behaves to upgrade the repository format version even if the extension.worktreeConfig is already set. This would previously fail without a clear way forward.
--- cache.h | 2 +- setup.c | 12 ++++++++---- t/t1091-sparse-checkout-builtin.sh | 12 ++++++++++++ 3 files changed, 21 insertions(+), 5 deletions(-)
cache.h +1 -1
@@ -1042,8 +1042,8 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; -int has_extensions;char *work_tree; +int has_unallowed_extensions;struct string_list unknown_extensions; };
setup.c +8 -4
@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (strcmp (var,"core.repositoryformatversion" ) ==0 ) data->version =git_config_int (var, value);else if (skip_prefix (var,"extensions." , &ext)) { - data->has_extensions =1 ;/* * record any known extensions here; otherwise, * we fall through to recording it as unknown, and * check_repository_format will complain */ +int is_unallowed_extension =1 ; +if (!strcmp (ext,"noop" )) ;else if (!strcmp (ext,"preciousobjects" ))
@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (!value)return config_error_nonbool (var); data->partial_clone =xstrdup (value); - }else if (!strcmp (ext,"worktreeconfig" )) + }else if (!strcmp (ext,"worktreeconfig" )) { data->worktree_config =git_config_bool (var, value); -else + is_unallowed_extension =0 ; + }else string_list_append (&data->unknown_extensions, ext); + + data->has_unallowed_extensions |= is_unallowed_extension; }return read_worktree_config (var, value, vdata);
@@ -560,7 +564,7 @@ int upgrade_repository_format(int target_version)return 0 ;if (verify_repository_format (&repo_fmt, &err) <0 || - (!repo_fmt.version && repo_fmt.has_extensions)) { + (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {warning ("unable to upgrade repository format from %d to %d: %s" , repo_fmt.version, target_version, err.buf);strbuf_release (&err);
t/t1091-sparse-checkout-builtin.sh +12 -0
@@ -68,6 +68,18 @@ test_expect_success 'git sparse-checkout init' ' check_files repo a ' +test_expect_success 'git sparse-checkout works if repository format is wrong' ' + test_when_finished git -C repo config core.repositoryFormatVersion 1 && + git -C repo config --unset core.repositoryFormatVersion && + git -C repo sparse-checkout init && + git -C repo config core.repositoryFormatVersion >actual && + echo 1 >expect && + git -C repo config core.repositoryFormatVersion 0 && + git -C repo sparse-checkout init && + git -C repo config core.repositoryFormatVersion >actual && + test_cmp expect actual +' + test_expect_success 'git sparse-checkout list after init' ' git -C repo sparse-checkout list >actual && cat >expect <<-\EOF &&
Signature · 1 line
gitgitgadget
Re: [PATCH v2 1/2] setup: tweak upgrade policy to grandfather worktreeconfig
Hi Stolee,
Quoted 2 lines · Derrick Stolee via GitGitGadget
This is because the logic in
14c7fa269e4refuses to upgrae repos when the version is unset but extensions exist.
s/upgrae/upgrade/
The rest looks good to me.
Thank you for working on this,
Dscho
Quoted 114 lines
One possible way to alert a user of this issue is to warn them when Git notices an extension exists, but core.repositoryFormatVersion is not a correct value. However,
- it requires the end-user to read, understand and execute the manual upgrade
- it encourages to follow the same procedure blindly, making the protection even less useful
Let's instead keep failing hard without teaching how to bypass the repository protection, but allow upgrading even when only the worktreeconfig extension exists in an old repository, which is likely to be set by a broke version of Git that did not update the repository version when setting the extension.
This change of behavior is made visible by testing how 'git sparse-checkout init' behaves to upgrade the repository format version even if the extension.worktreeConfig is already set. This would previously fail without a clear way forward.
Signed-off-by Junio C Hamano <[email protected]>Signed-off-by Derrick Stolee <[email protected]>--- cache.h | 2 +- setup.c | 12 ++++++++---- t/t1091-sparse-checkout-builtin.sh | 12 ++++++++++++ 3 files changed, 21 insertions(+), 5 deletions(-)cache.h +1
-1@@ -1042,8 +1042,8 @@ struct repository_format {int worktree_config;int is_bare;int hash_algo; -int has_extensions;char *work_tree; +int has_unallowed_extensions;struct string_list unknown_extensions; };setup.c +8
-4@@ -455,12 +455,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (strcmp (var,"core.repositoryformatversion" ) ==0 ) data->version =git_config_int (var, value);else if (skip_prefix (var,"extensions." , &ext)) { - data->has_extensions =1 ;/* * record any known extensions here; otherwise, * we fall through to recording it as unknown, and * check_repository_format will complain */ +int is_unallowed_extension =1 ; +if (!strcmp (ext,"noop" )) ;else if (!strcmp (ext,"preciousobjects" ))@@ -469,10 +470,13 @@ static int check_repo_format(const char *var, const char *value, void *vdata)if (!value)return config_error_nonbool (var); data->partial_clone =xstrdup (value); - }else if (!strcmp (ext,"worktreeconfig" )) + }else if (!strcmp (ext,"worktreeconfig" )) { data->worktree_config =git_config_bool (var, value); -else + is_unallowed_extension =0 ; + }else string_list_append (&data->unknown_extensions, ext); + + data->has_unallowed_extensions |= is_unallowed_extension; }return read_worktree_config (var, value, vdata);@@ -560,7 +564,7 @@ int upgrade_repository_format(int target_version)return 0 ;if (verify_repository_format (&repo_fmt, &err) <0 || - (!repo_fmt.version && repo_fmt.has_extensions)) { + (!repo_fmt.version && repo_fmt.has_unallowed_extensions)) {warning ("unable to upgrade repository format from %d to %d: %s" , repo_fmt.version, target_version, err.buf);strbuf_release (&err);t/t1091-sparse-checkout-builtin.sh +12
-1@@ -68,6 +68,18 @@ test_expect_success 'git sparse-checkout init' ' check_files repo a ' +test_expect_success 'git sparse-checkout works if repository format is wrong' ' + test_when_finished git -C repo config core.repositoryFormatVersion 1 && + git -C repo config --unset core.repositoryFormatVersion && + git -C repo sparse-checkout init && + git -C repo config core.repositoryFormatVersion >actual && + echo 1 >expect && + git -C repo config core.repositoryFormatVersion 0 && + git -C repo sparse-checkout init && + git -C repo config core.repositoryFormatVersion >actual && + test_cmp expect actual +' + test_expect_success 'git sparse-checkout list after init' ' git -C repo sparse-checkout list >actual && cat >expect <<-\EOF && --gitgitgadget
PATCH v2 2/2 config: provide extra detail about worktree config
In reply to Derrick Stolee via GitGitGadgetFrom: Derrick Stolee <[email protected]>
When any extensions.* setting is configured, we newly ignore it unless core.repositoryFormatVersion is set to a positive value.
This might be quite surprising, e.g. when calling git config --worktree [...] elicits a warning that it requires extensions.worktreeConfig = true when that setting _is_ configured (but ignored because core.repositoryFormatVersion is unset).
Let's warn about this situation specifically, especially because there might be already setups out there that configured a sparse worktree using Git v2.27.0 (which does set extensions.worktreeConfig but not core.repositoryFormatVersion) and users might want to work in those setups with Git v2.28.0, too.
This warning is specifically placed inside an existing error message for 'git config --worktree' that already fails if the repository format version is not 1.
--- builtin/config.c | 5 +++-- t/t2404-worktree-config.sh | 12 ++++++++++++ 2 files changed, 15 insertions(+), 2 deletions(-)
builtin/config.c +3 -2
@@ -678,8 +678,9 @@ int cmd_config(int argc, const char **argv, const char *prefix)else if (worktrees[0 ] && worktrees[1 ])die (_ ("--worktree cannot be used with multiple " "working trees unless the config\n" -"extension worktreeConfig is enabled. " -"Please read \"CONFIGURATION FILE\"\n" +"extension worktreeConfig is enabled " +"and core.repositoryFormatVersion is at least\n" +"1. Please read \"CONFIGURATION FILE\"" "section in \"git help worktree\" for details" ));else given_config_source.file =git_pathdup ("config" );
t/t2404-worktree-config.sh +12 -0
@@ -77,5 +77,17 @@ test_expect_success 'config.worktree no longer read without extension' ' test_cmp_config -C wt1 shared this.is && test_cmp_config -C wt2 shared this.is ' +test_expect_success 'show advice when extensions.* are not enabled' ' + test_config core.repositoryformatversion 1 && + test_config extensions.worktreeConfig true && + git config --worktree test.one true && + test_cmp_config true test.one && + test_config core.repositoryformatversion 0 && + test_must_fail git config --worktree test.two true 2>err && + test_i18ngrep "extension worktreeConfig is enabled" err && + git config --unset core.repositoryformatversion && + test_must_fail git config --worktree test.three true 2>err && + test_i18ngrep "core.repositoryFormatVersion is at least" err +' test_done
Signature · 1 line
gitgitgadget
Hi Stolee,
Quoted 2 lines · Derrick Stolee via GitGitGadget
2. A second patch has joined the fray, hopefully answering the concerned raise by Johannes [1].
It answers my concern!
Thank you for providing these patches,
Dscho