Skip to contentSkip to conversation

prevent 'checkout -m' from losing staged changes

PATCH In review Nguyễn Thái Ngọc Duy 3 participants 7 messages Mar 22, 2019 – Mar 25, 2019 Documentation

lore
Share & tools
lore
#0 lore raw

cover prevent 'checkout -m' from losing staged changes

During the 'git switch' discusion, Phillip found out that staged changes could be silently lost when doing 'git checkout -m'. The first attempt
[2] tries to fix it by warning and moving on, because t7201.10 would fail if we simply die() when there are staged changes.

This round, I update read-tree to be used in that test instead of 'checkout -m'. Then we are able to die().

I start a new thread to keep spam level down for other people, and to break free from 'git switch' thread because it's unrelated now.

Jonathan is included, just in case he still remembers something from 6a143aa2b2 (checkout -m: attempt merge when deletion of path was staged, 2014-08-12)

[1] https://public-inbox.org/git/[email protected]/
[2] https://public-inbox.org/git/[email protected]/

Nguyễn Thái Ngọc Duy (4):
unpack-trees: keep gently check inside add_rejected_path
unpack-trees: rename "gently" flag to "quiet"
read-tree: add --quiet
checkout: prevent losing staged changes with --merge

 Documentation/git-read-tree.txt |  4 ++++
 builtin/checkout.c              | 13 +++++++++++--
 builtin/read-tree.c             |  1 +
 t/t7201-co.sh                   |  9 ++-------
 unpack-trees.c                  | 25 +++++++++++--------------
 unpack-trees.h                  |  2 +-
 6 files changed, 30 insertions(+), 24 deletions(-)
Signature · 1 line
2.21.0.548.gd3c7d92dc2
#1 lore raw

PATCH 1/4 unpack-trees: keep gently check inside add_rejected_path

This basically follows the footsteps of 6a143aa2b2 (checkout -m:
attempt merge when deletion of path was staged - 2014-08-12) where there gently check is moved inside reject_merge() so that callers do not accidentally forget it.

add_rejected_path() has the same usage pattern. All call sites check gently first, then decide to call add_rejected_path() if needed. Move the check inside.

---
 unpack-trees.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)
unpack-trees.c +9 -12
index 22c41a3ba8..e6c1cc8302 100644
@@ -219,6 +219,9 @@ static int add_rejected_path(struct unpack_trees_options *o,
 			     enum unpack_trees_error_types e,
 			     const char *path)
 {
+	if (o->gently)
+		return -1;
+
 	if (!o->show_all_errors)
 		return error(ERRORMSG(o, e), super_prefixed(path));
 
@@ -268,8 +271,7 @@ static int check_submodule_move_head(const struct cache_entry *ce,
 		flags |= SUBMODULE_MOVE_HEAD_FORCE;
 
 	if (submodule_move_head(ce->name, old_id, new_id, flags))
-		return o->gently ? -1 :
-				   add_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);
+		return add_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);
 	return 0;
 }
 
@@ -1645,8 +1647,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
 static int reject_merge(const struct cache_entry *ce,
 			struct unpack_trees_options *o)
 {
-	return o->gently ? -1 :
-		add_rejected_path(o, ERROR_WOULD_OVERWRITE, ce->name);
+	return add_rejected_path(o, ERROR_WOULD_OVERWRITE, ce->name);
 }
 
 static int same(const struct cache_entry *a, const struct cache_entry *b)
@@ -1693,8 +1694,7 @@ static int verify_uptodate_1(const struct cache_entry *ce,
 			int r = check_submodule_move_head(ce,
 				"HEAD", oid_to_hex(&ce->oid), o);
 			if (r)
-				return o->gently ? -1 :
-					add_rejected_path(o, error_type, ce->name);
+				return add_rejected_path(o, error_type, ce->name);
 			return 0;
 		}
 
@@ -1712,8 +1712,7 @@ static int verify_uptodate_1(const struct cache_entry *ce,
 	}
 	if (errno == ENOENT)
 		return 0;
-	return o->gently ? -1 :
-		add_rejected_path(o, error_type, ce->name);
+	return add_rejected_path(o, error_type, ce->name);
 }
 
 int verify_uptodate(const struct cache_entry *ce,
@@ -1835,8 +1834,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 		d.exclude_per_dir = o->dir->exclude_per_dir;
 	i = read_directory(&d, o->src_index, pathbuf, namelen+1, NULL);
 	if (i)
-		return o->gently ? -1 :
-			add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
+		return add_rejected_path(o, ERROR_NOT_UPTODATE_DIR, ce->name);
 	free(pathbuf);
 	return cnt;
 }
@@ -1905,8 +1903,7 @@ static int check_ok_to_remove(const char *name, int len, int dtype,
 			return 0;
 	}
 
-	return o->gently ? -1 :
-		add_rejected_path(o, error_type, name);
+	return add_rejected_path(o, error_type, name);
 }
 
 /*
Signature · 1 line
2.21.0.548.gd3c7d92dc2
#2 lore raw

PATCH 2/4 unpack-trees: rename "gently" flag to "quiet"

In reply to Nguyễn Thái Ngọc Duy

The gently flag was added in 17e4642667 (Add flag to make unpack_trees() not print errors. - 2008-02-07) to suppress error messages. The name "gently" does not quite express that. Granted, being quiet is gentle but it could mean not performing some other actions. Rename the flag to "quiet" to be more on point.

---
 builtin/checkout.c | 2 +-
 unpack-trees.c     | 6 +++---
 unpack-trees.h     | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)
builtin/checkout.c +1 -1
index 0e6037b296..22fb6c0cae 100644
@@ -700,7 +700,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 		topts.initial_checkout = is_cache_unborn();
 		topts.update = 1;
 		topts.merge = 1;
-		topts.gently = opts->merge && old_branch_info->commit;
+		topts.quiet = opts->merge && old_branch_info->commit;
 		topts.verbose_update = opts->show_progress;
 		topts.fn = twoway_merge;
 		if (opts->overwrite_ignore) {
unpack-trees.c +3 -3
index e6c1cc8302..2e5d7b202e 100644
@@ -219,7 +219,7 @@ static int add_rejected_path(struct unpack_trees_options *o,
 			     enum unpack_trees_error_types e,
 			     const char *path)
 {
-	if (o->gently)
+	if (o->quiet)
 		return -1;
 
 	if (!o->show_all_errors)
@@ -1042,7 +1042,7 @@ static int unpack_nondirectories(int n, unsigned long mask,
 static int unpack_failed(struct unpack_trees_options *o, const char *message)
 {
 	discard_index(&o->result);
-	if (!o->gently && !o->exiting_early) {
+	if (!o->quiet && !o->exiting_early) {
 		if (message)
 			return error("%s", message);
 		return -1;
@@ -2343,7 +2343,7 @@ int bind_merge(const struct cache_entry * const *src,
 		return error("Cannot do a bind merge of %d trees",
 			     o->merge_size);
 	if (a && old)
-		return o->gently ? -1 :
+		return o->quiet ? -1 :
 			error(ERRORMSG(o, ERROR_BIND_OVERLAP),
 			      super_prefixed(a->name),
 			      super_prefixed(old->name));
unpack-trees.h +1 -1
index 0135080a7b..d344d7d296 100644
@@ -56,7 +56,7 @@ struct unpack_trees_options {
 		     diff_index_cached,
 		     debug_unpack,
 		     skip_sparse_checkout,
-		     gently,
+		     quiet,
 		     exiting_early,
 		     show_all_errors,
 		     dry_run;
Signature · 1 line
2.21.0.548.gd3c7d92dc2
#3 lore raw

PATCH 3/4 read-tree: add --quiet

In reply to Nguyễn Thái Ngọc Duy

read-tree is basically the front end of unpack-trees code and shoud expose all of its functionality (unless it's designed for internal use). This "opts.quiet" (formerly "opts.gently") was added for builtin/checkout.c but there is no reason why other read-tree users won't find this useful.

The test that is updated to run 'read-tree --quiet' was added because unpack-trees was accidentally not being quiet [1] in 6a143aa2b2 (checkout -m: attempt merge when deletion of path was staged - 2014-08-12). Because checkout is the only "opts.quiet" user, there was no other way to test quiet behavior. But we can now test it directly.

6a143aa2b2 was manually reverted to verify that read-tree --quiet works correctly (i.e. test_must_be_empty fails).

[1] the commit message there say "errors out instead of performing a merge" but I'm pretty sure the "performing a merge" happens anyway even before that commit. That line should say "errors out _in addition to_ performing a merge"

---
 Documentation/git-read-tree.txt | 4 ++++
 builtin/read-tree.c             | 1 +
 t/t7201-co.sh                   | 3 +++
 3 files changed, 8 insertions(+)
Documentation/git-read-tree.txt +4 -0
index 5c70bc2878..1e81f9c4e6 100644
@@ -128,6 +128,10 @@ OPTIONS
 	Instead of reading tree object(s) into the index, just empty
 	it.
 
+-q::
+--quiet::
+	Quiet, suppress feedback messages.
+
 <tree-ish#>::
 	The id of the tree object(s) to be read/merged.
 
builtin/read-tree.c +1 -0
index 9083dcfa28..5c9c082595 100644
@@ -154,6 +154,7 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 		{ OPTION_CALLBACK, 0, "recurse-submodules", NULL,
 			    "checkout", "control recursive updating of submodules",
 			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
+		OPT__QUIET(&opts.quiet, N_("suppress feedback messages")),
 		OPT_END()
 	};
 
t/t7201-co.sh +3 -0
index 72b9b375ba..f165582019 100755
@@ -223,6 +223,9 @@ test_expect_success 'switch to another branch while carrying a deletion' '
 	test_must_fail git checkout simple 2>errs &&
 	test_i18ngrep overwritten errs &&
 
+	test_must_fail git read-tree --quiet -m -u HEAD simple 2>errs &&
+	test_must_be_empty errs &&
+
 	git checkout --merge simple 2>errs &&
 	test_i18ngrep ! overwritten errs &&
 	git ls-files -u &&
Signature · 1 line
2.21.0.548.gd3c7d92dc2
#4 lore raw

Re: [PATCH 3/4] read-tree: add --quiet

Quoted 1 line · Nguyễn Thái Ngọc Duy

read-tree is basically the front end of unpack-trees code and shoud

s/shoud/should/

Philip

Quoted 67 lines

expose all of its functionality (unless it's designed for internal use). This "opts.quiet" (formerly "opts.gently") was added for builtin/checkout.c but there is no reason why other read-tree users won't find this useful.

The test that is updated to run 'read-tree --quiet' was added because unpack-trees was accidentally not being quiet [1] in 6a143aa2b2 (checkout -m: attempt merge when deletion of path was staged - 2014-08-12). Because checkout is the only "opts.quiet" user, there was no other way to test quiet behavior. But we can now test it directly.

6a143aa2b2 was manually reverted to verify that read-tree --quiet works correctly (i.e. test_must_be_empty fails).

[1] the commit message there say "errors out instead of performing a merge" but I'm pretty sure the "performing a merge" happens anyway even before that commit. That line should say "errors out _in addition to_ performing a merge"

---
  Documentation/git-read-tree.txt | 4 ++++
  builtin/read-tree.c             | 1 +
  t/t7201-co.sh                   | 3 +++
  3 files changed, 8 insertions(+)
Documentation/git-read-tree.txt +4 -0
index 5c70bc2878..1e81f9c4e6 100644
@@ -128,6 +128,10 @@ OPTIONS
  	Instead of reading tree object(s) into the index, just empty
  	it.
 
+-q::
+--quiet::
+	Quiet, suppress feedback messages.
+
  <tree-ish#>::
  	The id of the tree object(s) to be read/merged.
 
builtin/read-tree.c +1 -0
index 9083dcfa28..5c9c082595 100644
@@ -154,6 +154,7 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
  		{ OPTION_CALLBACK, 0, "recurse-submodules", NULL,
  			    "checkout", "control recursive updating of submodules",
  			    PARSE_OPT_OPTARG, option_parse_recurse_submodules_worktree_updater },
+		OPT__QUIET(&opts.quiet, N_("suppress feedback messages")),
  		OPT_END()
  	};
 
t/t7201-co.sh +3 -0
index 72b9b375ba..f165582019 100755
@@ -223,6 +223,9 @@ test_expect_success 'switch to another branch while carrying a deletion' '
  	test_must_fail git checkout simple 2>errs &&
  	test_i18ngrep overwritten errs &&
 
+	test_must_fail git read-tree --quiet -m -u HEAD simple 2>errs &&
+	test_must_be_empty errs &&
+
  	git checkout --merge simple 2>errs &&
  	test_i18ngrep ! overwritten errs &&
  	git ls-files -u &&
#5 lore raw

PATCH 4/4 checkout: prevent losing staged changes with --merge

In reply to Nguyễn Thái Ngọc Duy

When --merge is specified, we may need to do a real merge (instead of three-way tree unpacking), the steps are best seen in git-checkout.sh version before it's removed:

    # Match the index to the working tree, and do a three-way.
    git diff-files --name-only | git update-index --remove --stdin &&
    work=`git write-tree` &&
    git read-tree $v --reset -u $new || exit

    git merge-recursive $old -- $new $work

    # Do not register the cleanly merged paths in the index yet.
    # this is not a real merge before committing, but just carrying
    # the working tree changes along.
    unmerged=`git ls-files -u`
    git read-tree $v --reset $new
    case "$unmerged" in
    '')     ;;
    *)
            (
                    z40=0000000000000000000000000000000000000000
                    echo "$unmerged" |
                    sed -e 's/^[0-7]* [0-9a-f]* /'"0 $z40 /"
                    echo "$unmerged"
            ) | git update-index --index-info
            ;;
    esac

Notice the last 'read-tree --reset' step. We restore worktree back to 'new' tree after worktree's messed up by merge-recursive. If there are staged changes before this whole command sequence is executed, they are lost because they are unlikely part of the 'new' tree to be restored.

There is no easy way to fix this. Elijah may have something up his sleeves [1], but until then, check if there are staged changes and refuse to run and lose them. The user would need to do "git reset" to continue in this case.

A note about the test update. 'checkout -m' in that test will fail because a deletion is staged. This 'checkout -m' was previously needed to verify quietness behavior of unpack-trees. But a different check has been put in place in the last patch. We can safely drop 'checkout -m' now.

[1] CABPp-BFoL_U=bzON4SEMaQSKU2TKwnOgNqjt5MUaOejTKGUJxw@mail.gmail.com

---
 builtin/checkout.c | 11 ++++++++++-
 t/t7201-co.sh      | 10 +---------
 2 files changed, 11 insertions(+), 10 deletions(-)
builtin/checkout.c +10 -1
index 22fb6c0cae..7cd01f62be 100644
@@ -725,7 +725,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 */
 			struct tree *result;
 			struct tree *work;
+			struct tree *old_tree;
 			struct merge_options o;
+			struct strbuf sb = STRBUF_INIT;
+
 			if (!opts->merge)
 				return 1;
 
@@ -735,6 +738,12 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 */
 			if (!old_branch_info->commit)
 				return 1;
+			old_tree = get_commit_tree(old_branch_info->commit);
+
+			if (repo_index_has_changes(the_repository, old_tree, &sb))
+				die(_("cannot continue with staged changes in "
+				      "the following files:\n%s"), sb.buf);
+			strbuf_release(&sb);
 
 			/* Do more real merge */
 
@@ -772,7 +781,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			ret = merge_trees(&o,
 					  get_commit_tree(new_branch_info->commit),
 					  work,
-					  get_commit_tree(old_branch_info->commit),
+					  old_tree,
 					  &result);
 			if (ret < 0)
 				exit(128);
t/t7201-co.sh +1 -9
index f165582019..5990299fc9 100755
@@ -224,15 +224,7 @@ test_expect_success 'switch to another branch while carrying a deletion' '
 	test_i18ngrep overwritten errs &&
 
 	test_must_fail git read-tree --quiet -m -u HEAD simple 2>errs &&
-	test_must_be_empty errs &&
-
-	git checkout --merge simple 2>errs &&
-	test_i18ngrep ! overwritten errs &&
-	git ls-files -u &&
-	test_must_fail git cat-file -t :0:two &&
-	test "$(git cat-file -t :1:two)" = blob &&
-	test "$(git cat-file -t :2:two)" = blob &&
-	test_must_fail git cat-file -t :3:two
+	test_must_be_empty errs
 '
 
 test_expect_success 'checkout to detach HEAD (with advice declined)' '
Signature · 1 line
2.21.0.548.gd3c7d92dc2
#6 lore raw

Re: [PATCH 4/4] checkout: prevent losing staged changes with --merge

Quoted 101 lines · Nguyễn Thái Ngọc Duy

When --merge is specified, we may need to do a real merge (instead of three-way tree unpacking), the steps are best seen in git-checkout.sh version before it's removed:

    # Match the index to the working tree, and do a three-way.
    git diff-files --name-only | git update-index --remove --stdin &&
    work=`git write-tree` &&
    git read-tree $v --reset -u $new || exit

    git merge-recursive $old -- $new $work

    # Do not register the cleanly merged paths in the index yet.
    # this is not a real merge before committing, but just carrying
    # the working tree changes along.
    unmerged=`git ls-files -u`
    git read-tree $v --reset $new
    case "$unmerged" in
    '')     ;;
    *)
            (
                    z40=0000000000000000000000000000000000000000
                    echo "$unmerged" |
                    sed -e 's/^[0-7]* [0-9a-f]* /'"0 $z40 /"
                    echo "$unmerged"
            ) | git update-index --index-info
            ;;
    esac

Notice the last 'read-tree --reset' step. We restore worktree back to 'new' tree after worktree's messed up by merge-recursive. If there are staged changes before this whole command sequence is executed, they are lost because they are unlikely part of the 'new' tree to be restored.

There is no easy way to fix this. Elijah may have something up his sleeves [1], but until then, check if there are staged changes and refuse to run and lose them. The user would need to do "git reset" to continue in this case.

A note about the test update. 'checkout -m' in that test will fail because a deletion is staged. This 'checkout -m' was previously needed to verify quietness behavior of unpack-trees. But a different check has been put in place in the last patch. We can safely drop 'checkout -m' now.

[1] CABPp-BFoL_U=bzON4SEMaQSKU2TKwnOgNqjt5MUaOejTKGUJxw@mail.gmail.com

---
 builtin/checkout.c | 11 ++++++++++-
 t/t7201-co.sh      | 10 +---------
 2 files changed, 11 insertions(+), 10 deletions(-)
builtin/checkout.c +10 -1
index 22fb6c0cae..7cd01f62be 100644
@@ -725,7 +725,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
                         */
                        struct tree *result;
                        struct tree *work;
+                       struct tree *old_tree;
                        struct merge_options o;
+                       struct strbuf sb = STRBUF_INIT;
+
                        if (!opts->merge)
                                return 1;
 
@@ -735,6 +738,12 @@ static int merge_working_tree(const struct checkout_opts *opts,
                         */
                        if (!old_branch_info->commit)
                                return 1;
+                       old_tree = get_commit_tree(old_branch_info->commit);
+
+                       if (repo_index_has_changes(the_repository, old_tree, &sb))
+                               die(_("cannot continue with staged changes in "
+                                     "the following files:\n%s"), sb.buf);
+                       strbuf_release(&sb);
 
                        /* Do more real merge */
 
@@ -772,7 +781,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
                        ret = merge_trees(&o,
                                          get_commit_tree(new_branch_info->commit),
                                          work,
-                                         get_commit_tree(old_branch_info->commit),
+                                         old_tree,
                                          &result);
                        if (ret < 0)
                                exit(128);
t/t7201-co.sh +0 -1
index f165582019..5990299fc9 100755
@@ -224,15 +224,7 @@ test_expect_success 'switch to another branch while carrying a deletion' '
        test_i18ngrep overwritten errs &&
 
        test_must_fail git read-tree --quiet -m -u HEAD simple 2>errs &&
-       test_must_be_empty errs &&

t/t7201-co.sh test_expect_success 'switch to another branch while carrying a deletion' ' · line 228

Quoted 10 lines
t/t7201-co.sh +1 -8
-
-       git checkout --merge simple 2>errs &&
-       test_i18ngrep ! overwritten errs &&
-       git ls-files -u &&
-       test_must_fail git cat-file -t :0:two &&
-       test "$(git cat-file -t :1:two)" = blob &&
-       test "$(git cat-file -t :2:two)" = blob &&
-       test_must_fail git cat-file -t :3:two
+       test_must_be_empty errs
 '

Ah, I see you avoid the other bug in checkout by just removing its usage. Seems fair enough; I can add a separate test to demonstrate that bug when I get some time to work on it. Hopefully I'll find a quick fix too.