Skip to contentSkip to conversation

rebase -i --update-refs can lead to deletion of branches

PATCH Superseded herr.kaste 6 participants 17 messages Oct 20, 2022 – Nov 8, 2022

lore
Share & tools
lore

herr.kaste

author
#0 lore raw

Hi,

I have the following:

While doing a

$ git rebase --interactive --update-refs X

I *removed* the "update-ref" lines from the todo list. The rebase runs as expected and prints e.g.

```
Successfully rebased and updated refs/heads/test.
Updated the following refs with --update-refs:
refs/heads/master
refs/heads/permissive-interactive-rebase
refs/heads/variable-annotations-meta-block
```

After that all refs have been removed/deleted.

```
$ git branch  --list
* test
```

Now, I should just have not used --update-refs in the first place but anyway I decide late that I rather don't want to update "master" etc. and it should probably not delete the local refs.

Actually, I so love the new feature that I switched it *on* by default, and just wanted to overwrite the behavior in the todo editor.

Regards
Caspar Duregger

#1 lore raw
Quoted 7 lines · herr.kaste

After that all refs have been removed/deleted.

```
$ git branch  --list
* test
```

:(

Quoted 2 lines

I decide late that I rather don't want to update "master" etc. and it should probably not delete the local refs.

Deleting refs if you remove from the rebase-todo seems undesirable in my opinion. It's too easy to use that footgun. It may be a good feature to be _able_ to delete branches during a rebase, in a similar manner as updating them, but with a different explicit flag, like d regs/heads/ or something

Quoted 2 lines

Actually, I so love the new feature that I switched it *on* by default, and just wanted to overwrite the behavior in the todo editor.

I didn't know this feature had been added but I'm very very pleased as I've wanted it. It doesn't seem to update tags though. That would've been nice.

#2 lore raw
In reply to herr.kaste

Hi Caspar

Quoted 26 lines · herr.kaste

Hi,

I have the following:

While doing a

$ git rebase --interactive --update-refs X

I *removed* the "update-ref" lines from the todo list. The rebase runs as expected and prints e.g.

```
Successfully rebased and updated refs/heads/test.
Updated the following refs with --update-refs:
refs/heads/master
refs/heads/permissive-interactive-rebase
refs/heads/variable-annotations-meta-block
```

After that all refs have been removed/deleted.

```
$ git branch  --list
* test
```
Quoted 6 lines

Now, I should just have not used --update-refs in the first place but anyway I decide late that I rather don't want to update "master" etc. and it should probably not delete the local refs.

Actually, I so love the new feature that I switched it *on* by default, and just wanted to overwrite the behavior in the todo editor.

Sorry for the slow reply, I'm afraid I still haven't found time to look at this. As far as I can remember deleting the "update-ref" lines should leave the ref unchanged. I've cc'd the author to see if they have any insight into what is going on

Best Wishes

Phillip

Quoted 2 lines

Regards
Caspar Duregger

author
#3 lore raw

I have the following reproduction

```
git init &&
git commit --allow-empty -m "Init" &&
git commit --allow-empty -m "A" &&
git checkout -b feature &&
git commit --allow-empty -m "B" &&
git commit --allow-empty -m "C" &&
GIT_SEQUENCE_EDITOR="sed -i -e '/^update-ref/d'" git rebase
--update-refs master^ --interactive
```

After that

```
$ git branch -l
* feature
```

and master is gone. Is that reproduction/test-case sane, even? I *think* that's what I originally described.

Regards
Caspar Duregger

Am Do., 3. Nov. 2022 um 10:32 Uhr schrieb Phillip Wood <[email protected]>:

Quoted 49 lines

Hi Caspar

Quoted 26 lines · herr.kaste

Hi,

I have the following:

While doing a

$ git rebase --interactive --update-refs X

I *removed* the "update-ref" lines from the todo list. The rebase runs as expected and prints e.g.

```
Successfully rebased and updated refs/heads/test.
Updated the following refs with --update-refs:
refs/heads/master
refs/heads/permissive-interactive-rebase
refs/heads/variable-annotations-meta-block
```

After that all refs have been removed/deleted.

```
$ git branch  --list
* test
```
Quoted 6 lines

Now, I should just have not used --update-refs in the first place but anyway I decide late that I rather don't want to update "master" etc. and it should probably not delete the local refs.

Actually, I so love the new feature that I switched it *on* by default, and just wanted to overwrite the behavior in the todo editor.

Sorry for the slow reply, I'm afraid I still haven't found time to look at this. As far as I can remember deleting the "update-ref" lines should leave the ref unchanged. I've cc'd the author to see if they have any insight into what is going on

Best Wishes

Phillip

Quoted 2 lines

Regards
Caspar Duregger

#4 lore raw
Quoted 13 lines · herr.kaste

I have the following reproduction

```
git init &&
git commit --allow-empty -m "Init" &&
git commit --allow-empty -m "A" &&
git checkout -b feature &&
git commit --allow-empty -m "B" &&
git commit --allow-empty -m "C" &&
GIT_SEQUENCE_EDITOR="sed -i -e '/^update-ref/d'" git rebase
--update-refs master^ --interactive
```

Some minor changes
Better explicitly name the branch master
git init -b master &&

also
GIT_SEQUENCE_EDITOR="sed -i -e '/^u/d'" git rebase --update-refs
master^ --interactive
in case of short-form interactive rebase

But yes, it deletes the master branch

#5 lore raw
In reply to herr.kaste
Quoted 4 lines · herr.kaste

Now, I should just have not used --update-refs in the first place but anyway I decide late that I rather don't want to update "master" etc. and it should probably not delete the local refs.

Agreed, this doesn't seem like desired behavior - the opposite of "update the ref" isn't "delete the ref". ;)

The reason it's happening is because, when '--update-refs' is used, the rebase starts by constructing a list of 'update_ref_record's for each of the refs that *could* be updated. Each item in that list contains the corresponding ref's "before" commit OID (i.e., what it currently points to) and initializes the "after" OID to null. When an 'update-ref' line is encountered in the 'rebase-todo', the "after" OID is updated with the newly-rebased value. However, if an 'update-ref' line is removed from the 'rebase-todo', the "after" value is never updated. Then, when the rebase finishes and the ref state data is applied, all of the entries with null "after" OIDs are deleted.

The three options for a fix I can think of are:

1. initialize the "after" OID to the value of "before".
2. don't update refs with a null "after" OID.
3. initialize the "after" OID to the value of "before", don't update the

     ref if "before" == "after".

I think #3 is the best option, since it avoids the unnecessary updates of #1 and leaves a cleaner path to a 'delete-ref' option (like the one proposed elsewhere in the thread [1]) than #2. I'll send a patch shortly.

[1] https://lore.kernel.org/git/CA+JQ7M-GbBTHZZ9xOLR=FitWFpUnkfuep9kSfNPxuSbJbKteGw@mail.gmail.com/

Thanks for reporting!
- Victoria

#6 lore raw

Hi Victoria

Quoted 22 lines · Victoria Dye
Quoted 4 lines · herr.kaste

Now, I should just have not used --update-refs in the first place but anyway I decide late that I rather don't want to update "master" etc. and it should probably not delete the local refs.

Agreed, this doesn't seem like desired behavior - the opposite of "update the ref" isn't "delete the ref". ;)

The reason it's happening is because, when '--update-refs' is used, the rebase starts by constructing a list of 'update_ref_record's for each of the refs that *could* be updated. Each item in that list contains the corresponding ref's "before" commit OID (i.e., what it currently points to) and initializes the "after" OID to null. When an 'update-ref' line is encountered in the 'rebase-todo', the "after" OID is updated with the newly-rebased value. However, if an 'update-ref' line is removed from the 'rebase-todo', the "after" value is never updated. Then, when the rebase finishes and the ref state data is applied, all of the entries with null "after" OIDs are deleted.

The three options for a fix I can think of are:

Quoted 8 lines

1. initialize the "after" OID to the value of "before".
2. don't update refs with a null "after" OID.
3. initialize the "after" OID to the value of "before", don't update the

      ref if "before" == "after".

I think #3 is the best option, since it avoids the unnecessary updates of #1 and leaves a cleaner path to a 'delete-ref' option (like the one proposed elsewhere in the thread [1]) than #2. I'll send a patch shortly.

We should be removing the entry entirely if the user removes it from the todo-list see b3b1a21d1a (sequencer: rewrite update-refs as user edits todo list, 2022-07-19) where the commit message says

1. If a '<ref>/<before>/<after>' triple in the update-refs file does not have a matching 'update-ref <ref>' command in the todo-list _and_ the <after> value is the null OID, then remove that triple. Here, the user removed the 'update-ref <ref>' command before it was executed, since if it was executed then the <after> value would store the commit at that position.

I think that is the best approach but it seems the implementation isn't actually doing that.

Best Wishes

Phillip

Quoted 5 lines

[1] https://lore.kernel.org/git/CA+JQ7M-GbBTHZZ9xOLR=FitWFpUnkfuep9kSfNPxuSbJbKteGw@mail.gmail.com/

Thanks for reporting!
- Victoria

#7 lore raw
Quoted 11 lines · Phillip Wood

We should be removing the entry entirely if the user removes it from the todo-list see b3b1a21d1a (sequencer: rewrite update-refs as user edits todo list, 2022-07-19) where the commit message says

1. If a '<ref>/<before>/<after>' triple in the update-refs file does not have a matching 'update-ref <ref>' command in the todo-list _and_ the <after> value is the null OID, then remove that triple. Here, the user removed the 'update-ref <ref>' command before it was executed, since if it was executed then the <after> value would store the commit at that position.

Quoted 2 lines

I think that is the best approach but it seems the implementation isn't actually doing that.

Thanks for pointing this out. This approach seems to have only been applied to 'git rebase --edit-todo', so ideally the fix will just be "do the same thing in the initial rebase."

I got sidetracked yesterday and didn't get as much time to work on this as I'd liked, but I should be able to send a patch today.

#8 lore raw

PATCH 1/1 rebase --update-refs: avoid unintended ref deletion

In b3b1a21d1a5 (sequencer: rewrite update-refs as user edits todo list, 2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it removes potential ref updates from the "update refs state" if a ref does not have a corresponding 'update-ref' line.

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

To fix this, add a 'force_if_empty' flag to allow writing the update refs state even if 'refs_to_oids' is empty. The three usages of 'write_update_refs_state()' are updated as follows:

- in 'todo_list_filter_update_refs()': force_if_empty is 1 because update ref entries are removed here. This setting fixes the ref deletion issue.
- in 'do_update_ref()': force_if_empty is 0, since this method only modifies (does not add or delete) ref update entries.
- in 'todo_list_add_update_ref_commands()': force_if_empty is 0, since this method strictly adds ref update entries.

Additionally, add a test covering the "all update-ref lines removed" case.

---
This fixes the issue reported in [1]. I initially misinterpreted the root cause (thought that 'todo_list_filter_update_refs()' was only applied in the case of '--edit-todo'). After looking into it a bit more, it appears that the actual failure case is much narrower, occurring only when *all* 'update-ref' lines were deleted from the 'rebase-todo'.

Thanks!
- Victoria

[1] https://lore.kernel.org/git/CAFzd1+5F4zqQ1CNeY2xaaf0r__JmE4ECiBt5h5OdiJHbaE78VA@mail.gmail.com/

 sequencer.c                   | 15 ++++++++++-----
 t/t3404-rebase-interactive.sh | 24 ++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 5 deletions(-)
sequencer.c +10 -5
index e658df7e8ff..4d99a4fd6ca 100644
@@ -4122,7 +4122,7 @@ static int do_merge(struct repository *r,
 	return ret;
 }
 
-static int write_update_refs_state(struct string_list *refs_to_oids)
+static int write_update_refs_state(struct string_list *refs_to_oids, int force_if_empty)
 {
 	int result = 0;
 	struct lock_file lock = LOCK_INIT;
@@ -4130,7 +4130,12 @@ static int write_update_refs_state(struct string_list *refs_to_oids)
 	struct string_list_item *item;
 	char *path;
 
-	if (!refs_to_oids->nr)
+	/*
+	 * If 'force' is specified, we want to write the updated refs even if
+	 * the list is empty. This is only needed for callers that may have
+	 * deleted items from 'refs_to_oids'.
+	 */
+	if (!refs_to_oids->nr && !force_if_empty)
 		return 0;
 
 	path = rebase_path_update_refs(the_repository->gitdir);
@@ -4260,7 +4265,7 @@ void todo_list_filter_update_refs(struct repository *r,
 	}
 
 	if (updated)
-		write_update_refs_state(&update_refs);
+		write_update_refs_state(&update_refs, 1);
 	string_list_clear(&update_refs, 1);
 }
 
@@ -4281,7 +4286,7 @@ static int do_update_ref(struct repository *r, const char *refname)
 		}
 	}
 
-	write_update_refs_state(&list);
+	write_update_refs_state(&list, 0);
 	string_list_clear(&list, 1);
 	return 0;
 }
@@ -6015,7 +6020,7 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list)
 		}
 	}
 
-	res = write_update_refs_state(&ctx.refs_to_oids);
+	res = write_update_refs_state(&ctx.refs_to_oids, 0);
 
 	string_list_clear(&ctx.refs_to_oids, 1);
 
t/t3404-rebase-interactive.sh +24 -0
index 4f5abb5ad25..e7d3721ece8 100755
@@ -1964,6 +1964,30 @@ test_expect_success 'respect user edits to update-ref steps' '
 	test_cmp_rev HEAD refs/heads/no-conflict-branch
 '
 
+test_expect_success '--update-refs: do not delete refs if all update-ref are removed' '
+	git checkout -b test-refs-not-removed no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f tip &&
+	(
+		set_cat_todo_editor &&
+		test_must_fail git rebase -i --update-refs base >todo.raw &&
+		sed -e "/^update-ref/d" <todo.raw >todo
+	) &&
+	(
+		set_replace_editor todo &&
+		git rebase -i --update-refs base
+	) &&
+
+	test_cmp_rev HEAD~3 refs/heads/first &&
+	test_cmp_rev HEAD~3 refs/heads/second &&
+	test_cmp_rev HEAD~1 refs/heads/third &&
+	test_cmp_rev HEAD refs/heads/tip &&
+	test_cmp_rev HEAD refs/heads/no-conflict-branch
+'
+
 test_expect_success '--update-refs: check failed ref update' '
 	git checkout -B update-refs-error no-conflict-branch &&
 	git branch -f base HEAD~4 &&

--
2.38.0

#9 lore raw

Re: [PATCH] rebase --update-refs: avoid unintended ref deletion

Quoted 5 lines · Victoria Dye

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

Good catch.

I wonder, though: should we only add pending ref updates to the update-refs state after we reach that point in the sequence?

IOW: there is no world where deleting an update-refs command means to drop the affected branch, right? So the initial state would be an empty list, which would cause us to not update any references.

Then as we proceed through the rebase, we accumulate update-refs commands, and know their after_oid immediately. Then when we're done, we can process the update-refs commands for the branches that we do have.

The more I think about this, the more that I am convinced that the bug is in how we initialize the pending list, not our treatment of it later on.

The bug fix works as-is, but I can't help wonder if the above approach might be more direct.

Thanks,
Taylor

#10 lore raw

Re: [PATCH] rebase --update-refs: avoid unintended ref deletion

Hi Taylor

Quoted 11 lines · Taylor Blau
Quoted 5 lines · Victoria Dye

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

Good catch.

I wonder, though: should we only add pending ref updates to the update-refs state after we reach that point in the sequence?

If I remember correctly the aim of the current behavior is to detect if another process also updates the ref while we're rebasing. To do that we need to record all the branch heads that have update-ref commands at the start of the rebase.

Best Wishes

Phillip

Quoted 17 lines

IOW: there is no world where deleting an update-refs command means to drop the affected branch, right? So the initial state would be an empty list, which would cause us to not update any references.

Then as we proceed through the rebase, we accumulate update-refs commands, and know their after_oid immediately. Then when we're done, we can process the update-refs commands for the branches that we do have.

The more I think about this, the more that I am convinced that the bug is in how we initialize the pending list, not our treatment of it later on.

The bug fix works as-is, but I can't help wonder if the above approach might be more direct.

Thanks,
Taylor

#11 lore raw

Re: [PATCH] rebase --update-refs: avoid unintended ref deletion

In reply to Victoria Dye

Hi Victoria

Quoted 11 lines · Victoria Dye

In b3b1a21d1a5 (sequencer: rewrite update-refs as user edits todo list, 2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it removes potential ref updates from the "update refs state" if a ref does not have a corresponding 'update-ref' line.

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

Thanks for taking the time to track down the cause of this bug and fix it.

Quoted 10 lines

To fix this, add a 'force_if_empty' flag to allow writing the update refs state even if 'refs_to_oids' is empty. The three usages of 'write_update_refs_state()' are updated as follows:

- in 'todo_list_filter_update_refs()': force_if_empty is 1 because update ref entries are removed here. This setting fixes the ref deletion issue.
- in 'do_update_ref()': force_if_empty is 0, since this method only modifies (does not add or delete) ref update entries.
- in 'todo_list_add_update_ref_commands()': force_if_empty is 0, since this method strictly adds ref update entries.

I think not writing the list if it is empty is just an optimization to avoid creating an empty file. I wonder if it would be simpler to unlink() any existing file if write_update_refs_state() is called with an empty list rather than adding the force flag.

Quoted 1 line

Additionally, add a test covering the "all update-ref lines removed" case.

That's great

Best Wishes

Phillip

Quoted 110 lines

---
This fixes the issue reported in [1]. I initially misinterpreted the root cause (thought that 'todo_list_filter_update_refs()' was only applied in the case of '--edit-todo'). After looking into it a bit more, it appears that the actual failure case is much narrower, occurring only when *all* 'update-ref' lines were deleted from the 'rebase-todo'.

Thanks!
- Victoria

[1] https://lore.kernel.org/git/CAFzd1+5F4zqQ1CNeY2xaaf0r__JmE4ECiBt5h5OdiJHbaE78VA@mail.gmail.com/

  sequencer.c                   | 15 ++++++++++-----
  t/t3404-rebase-interactive.sh | 24 ++++++++++++++++++++++++
  2 files changed, 34 insertions(+), 5 deletions(-)
sequencer.c +10 -5
index e658df7e8ff..4d99a4fd6ca 100644
@@ -4122,7 +4122,7 @@ static int do_merge(struct repository *r,
  	return ret;
  }
 
-static int write_update_refs_state(struct string_list *refs_to_oids)
+static int write_update_refs_state(struct string_list *refs_to_oids, int force_if_empty)
  {
  	int result = 0;
  	struct lock_file lock = LOCK_INIT;
@@ -4130,7 +4130,12 @@ static int write_update_refs_state(struct string_list *refs_to_oids)
  	struct string_list_item *item;
  	char *path;
 
-	if (!refs_to_oids->nr)
+	/*
+	 * If 'force' is specified, we want to write the updated refs even if
+	 * the list is empty. This is only needed for callers that may have
+	 * deleted items from 'refs_to_oids'.
+	 */
+	if (!refs_to_oids->nr && !force_if_empty)
  		return 0;
 
  	path = rebase_path_update_refs(the_repository->gitdir);
@@ -4260,7 +4265,7 @@ void todo_list_filter_update_refs(struct repository *r,
  	}
 
  	if (updated)
-		write_update_refs_state(&update_refs);
+		write_update_refs_state(&update_refs, 1);
  	string_list_clear(&update_refs, 1);
  }
 
@@ -4281,7 +4286,7 @@ static int do_update_ref(struct repository *r, const char *refname)
  		}
  	}
 
-	write_update_refs_state(&list);
+	write_update_refs_state(&list, 0);
  	string_list_clear(&list, 1);
  	return 0;
  }
@@ -6015,7 +6020,7 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list)
  		}
  	}
 
-	res = write_update_refs_state(&ctx.refs_to_oids);
+	res = write_update_refs_state(&ctx.refs_to_oids, 0);
 
  	string_list_clear(&ctx.refs_to_oids, 1);
 
t/t3404-rebase-interactive.sh +24 -1
index 4f5abb5ad25..e7d3721ece8 100755
@@ -1964,6 +1964,30 @@ test_expect_success 'respect user edits to update-ref steps' '
  	test_cmp_rev HEAD refs/heads/no-conflict-branch
  '
 
+test_expect_success '--update-refs: do not delete refs if all update-ref are removed' '
+	git checkout -b test-refs-not-removed no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f tip &&
+	(
+		set_cat_todo_editor &&
+		test_must_fail git rebase -i --update-refs base >todo.raw &&
+		sed -e "/^update-ref/d" <todo.raw >todo
+	) &&
+	(
+		set_replace_editor todo &&
+		git rebase -i --update-refs base
+	) &&
+
+	test_cmp_rev HEAD~3 refs/heads/first &&
+	test_cmp_rev HEAD~3 refs/heads/second &&
+	test_cmp_rev HEAD~1 refs/heads/third &&
+	test_cmp_rev HEAD refs/heads/tip &&
+	test_cmp_rev HEAD refs/heads/no-conflict-branch
+'
+
  test_expect_success '--update-refs: check failed ref update' '
  	git checkout -B update-refs-error no-conflict-branch &&
  	git branch -f base HEAD~4 &&
--

2.38.0

#12 lore raw

Re: [PATCH] rebase --update-refs: avoid unintended ref deletion

Quoted 9 lines · Phillip Wood

Hi Victoria

Quoted 6 lines · Victoria Dye

In b3b1a21d1a5 (sequencer: rewrite update-refs as user edits todo list, 2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it removes potential ref updates from the "update refs state" if a ref does not have a corresponding 'update-ref' line.

Quoted 7 lines
Quoted 5 lines

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

Thanks for taking the time to track down the cause of this bug and fix it.

I will add my thanks, too. Thanks for jumping in when I could not!

Quoted 4 lines
Quoted 4 lines

To fix this, add a 'force_if_empty' flag to allow writing the update refs state even if 'refs_to_oids' is empty. The three usages of 'write_update_refs_state()' are updated as follows:

Quoted 8 lines
Quoted 6 lines

- in 'todo_list_filter_update_refs()': force_if_empty is 1 because update ref entries are removed here. This setting fixes the ref deletion issue.
- in 'do_update_ref()': force_if_empty is 0, since this method only modifies (does not add or delete) ref update entries.
- in 'todo_list_add_update_ref_commands()': force_if_empty is 0, since this method strictly adds ref update entries.

I think not writing the list if it is empty is just an optimization to avoid creating an empty file. I wonder if it would be simpler to unlink() any existing file if write_update_refs_state() is called with an empty list rather than adding the force flag.

I agree that an unlink() is the best option, barring one point (that I will mention below).

Quoted 16 lines
Quoted 16 lines

+test_expect_success '--update-refs: do not delete refs if all update-ref are removed' ' +    git checkout -b test-refs-not-removed no-conflict-branch &&
+    git branch -f base HEAD~4 &&
+    git branch -f first HEAD~3 &&
+    git branch -f second HEAD~3 &&
+    git branch -f third HEAD~1 &&
+    git branch -f tip &&
+    (
+        set_cat_todo_editor &&
+        test_must_fail git rebase -i --update-refs base >todo.raw &&
+        sed -e "/^update-ref/d" <todo.raw >todo
+    ) &&
+    (
+        set_replace_editor todo &&
+        git rebase -i --update-refs base
+    ) &&

Quoted 8 lines

Review

Quoted 8 lines
(quoted diff) +8 -0
+
+    test_cmp_rev HEAD~3 refs/heads/first &&
+    test_cmp_rev HEAD~3 refs/heads/second &&
+    test_cmp_rev HEAD~1 refs/heads/third &&
+    test_cmp_rev HEAD refs/heads/tip &&
+    test_cmp_rev HEAD refs/heads/no-conflict-branch
+'
+

This is a great test! I'm glad that it handles the existing case. I think the only case that might be interesting is to make the rebase actually create new commits and show that the removed refs are no longer in the history of the new branch, but are instead reachable from the older tip.

For this test, we could create a 'fixup!' to 'first' and use --autosquash to generate new commits. At the end, we can compare first, second, and third to different ancestors of refs/heads/no-conflict-branch _and_ guarantee that no-conflict-branch did not move. Or:

git rev-parse first second third no-conflict-branch >expect-oids && ...do the rebase...
git rev-parse first second third no-conflict-branch >actual-oids && test_cmp expect-oids actual-oids

should work to guarantee these refs were not updated.

I wonder if anything interesting happens if after we remove the update-ref commands we have a 'break' command and then re-insert some of the commands. Will things like the unlink() direction cause a problem then?

Anyway, here is a potential diff on top of your patch that adds these modifications to the test. They work with your implementation changes, but I also didn't try the unlink() modification.

--- >8 ---

t/t3404-rebase-interactive.sh +46 -7
index e7d3721ece8..4b09b73525a 100755
@@ -1971,21 +1971,60 @@ test_expect_success '--update-refs: do not delete refs if all update-ref are rem
 	git branch -f second HEAD~3 &&
 	git branch -f third HEAD~1 &&
 	git branch -f tip &&
+
+	test_commit test-refs-not-removed &&
+	git commit --amend --fixup first &&
+
+	git rev-parse first second third tip no-conflict-branch >expect-oids &&
+
 	(
 		set_cat_todo_editor &&
-		test_must_fail git rebase -i --update-refs base >todo.raw &&
+		test_must_fail git rebase -i \
+			--autosquash --update-refs \
+			base >todo.raw &&
 		sed -e "/^update-ref/d" <todo.raw >todo
 	) &&
 	(
 		set_replace_editor todo &&
-		git rebase -i --update-refs base
+		git rebase -i --autosquash --update-refs base
 	) &&
 
-	test_cmp_rev HEAD~3 refs/heads/first &&
-	test_cmp_rev HEAD~3 refs/heads/second &&
-	test_cmp_rev HEAD~1 refs/heads/third &&
-	test_cmp_rev HEAD refs/heads/tip &&
-	test_cmp_rev HEAD refs/heads/no-conflict-branch
+	git rev-parse first second third tip no-conflict-branch >actual-oids &&
+	test_cmp expect-oids actual-oids
+'
+
+test_expect_success '--update-refs: do not delete refs if all update-ref are removed and some re-added' '
+	git checkout -b test-refs-not-removed2 no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f tip &&
+
+	test_commit test-refs-not-removed2 &&
+	git commit --amend --fixup first &&
+
+	git rev-parse first second third >expect-oids &&
+
+	(
+		set_cat_todo_editor &&
+		test_must_fail git rebase -i \
+			--autosquash --update-refs \
+			base >todo.raw &&
+		sed -e "/^update-ref/d" <todo.raw >todo
+	) &&
+	echo "break" >>todo &&
+	(
+		set_replace_editor todo &&
+		git rebase -i --autosquash --update-refs base &&
+		echo "update-ref refs/heads/tip" >todo &&
+		git rebase --edit-todo &&
+		git rebase --continue
+	) &&
+
+	git rev-parse first second third >actual-oids &&
+	test_cmp expect-oids actual-oids &&
+	test_cmp_rev HEAD tip
 '
 
 test_expect_success '--update-refs: check failed ref update' '

--- >8 ---

Thanks,
-Stolee

#13 lore raw

PATCH v2 1/1 rebase --update-refs: avoid unintended ref deletion

In reply to Victoria Dye

In b3b1a21d1a5 (sequencer: rewrite update-refs as user edits todo list, 2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it removes potential ref updates from the "update refs state" if a ref does not have a corresponding 'update-ref' line.

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

To fix this, delete the 'update-refs' state file when 'refs_to_oids' is empty. Additionally, add a tests covering "all update-ref lines removed" cases.

---
Changes since v1:
- Modified approach to handling empty 'refs_to_oids' from "optional force write empty file" to "always unlink"
- Added/updated tests

 sequencer.c                   |   9 ++-
 t/t3404-rebase-interactive.sh | 107 ++++++++++++++++++++++++++++++++++
 2 files changed, 113 insertions(+), 3 deletions(-)
sequencer.c +6 -3
index e658df7e8ff..798a9702961 100644
@@ -4130,11 +4130,14 @@ static int write_update_refs_state(struct string_list *refs_to_oids)
 	struct string_list_item *item;
 	char *path;
 
-	if (!refs_to_oids->nr)
-		return 0;
-
 	path = rebase_path_update_refs(the_repository->gitdir);
 
+	if (!refs_to_oids->nr) {
+		if (unlink(path) && errno != ENOENT)
+			result = error_errno(_("could not unlink: %s"), path);
+		goto cleanup;
+	}
+
 	if (safe_create_leading_directories(path)) {
 		result = error(_("unable to create leading directories of %s"),
 			       path);
t/t3404-rebase-interactive.sh +107 -0

113 lines not on this page · raw message

--
2.38.0

#14 lore raw

Re: [PATCH v2] rebase --update-refs: avoid unintended ref deletion

Quoted 3 lines · Victoria Dye
 sequencer.c                   |   9 ++-
 t/t3404-rebase-interactive.sh | 107 ++++++++++++++++++++++++++++++++++
 2 files changed, 113 insertions(+), 3 deletions(-)

Looks great, thanks. Will queue.

Thanks,
Taylor

#15 lore raw

Re: [PATCH v2] rebase --update-refs: avoid unintended ref deletion

In reply to Victoria Dye
Quoted 20 lines · Victoria Dye

In b3b1a21d1a5 (sequencer: rewrite update-refs as user edits todo list, 2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it removes potential ref updates from the "update refs state" if a ref does not have a corresponding 'update-ref' line.

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

To fix this, delete the 'update-refs' state file when 'refs_to_oids' is empty. Additionally, add a tests covering "all update-ref lines removed" cases.

Quoted 5 lines

---
Changes since v1:
- Modified approach to handling empty 'refs_to_oids' from "optional force write empty file" to "always unlink"
- Added/updated tests

This "always unlink" version is much cleaner. Thanks!

The new tests look great and I'm confident that they are exercising the unlink() followed by a retry of parsing the update-refs steps.

This version LGTM.

Thanks,
-Stolee

#16 lore raw

Re: [PATCH v2] rebase --update-refs: avoid unintended ref deletion

In reply to Victoria Dye

Hi Victoria

Quoted 12 lines · Victoria Dye

In b3b1a21d1a5 (sequencer: rewrite update-refs as user edits todo list, 2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it removes potential ref updates from the "update refs state" if a ref does not have a corresponding 'update-ref' line.

However, because 'write_update_refs_state()' will not update the state if the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will result in the state remaining unchanged from how it was initialized (with all refs' "after" OID being null). Then, when the ref update is applied, all refs will be updated to null and consequently deleted.

Quoted 3 lines

To fix this, delete the 'update-refs' state file when 'refs_to_oids' is empty. Additionally, add a tests covering "all update-ref lines removed" cases.

Thanks for re-rolling, unsurprisingly I prefer the unlink() approach to the previous version. As Stolee said the test coverage looks good too.

Best Wishes

Phillip

Quoted 157 lines

---
Changes since v1:
- Modified approach to handling empty 'refs_to_oids' from "optional force write empty file" to "always unlink"
- Added/updated tests

  sequencer.c                   |   9 ++-
  t/t3404-rebase-interactive.sh | 107 ++++++++++++++++++++++++++++++++++
  2 files changed, 113 insertions(+), 3 deletions(-)
sequencer.c +6 -3
index e658df7e8ff..798a9702961 100644
@@ -4130,11 +4130,14 @@ static int write_update_refs_state(struct string_list *refs_to_oids)
  	struct string_list_item *item;
  	char *path;
 
-	if (!refs_to_oids->nr)
-		return 0;
-
  	path = rebase_path_update_refs(the_repository->gitdir);
 
+	if (!refs_to_oids->nr) {
+		if (unlink(path) && errno != ENOENT)
+			result = error_errno(_("could not unlink: %s"), path);
+		goto cleanup;
+	}
+
  	if (safe_create_leading_directories(path)) {
  		result = error(_("unable to create leading directories of %s"),
  			       path);
t/t3404-rebase-interactive.sh +107 -1
index 4f5abb5ad25..462cefd25df 100755
@@ -1964,6 +1964,113 @@ test_expect_success 'respect user edits to update-ref steps' '
  	test_cmp_rev HEAD refs/heads/no-conflict-branch
  '
 
+test_expect_success '--update-refs: all update-ref lines removed' '
+	git checkout -b test-refs-not-removed no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f tip &&
+
+	test_commit test-refs-not-removed &&
+	git commit --amend --fixup first &&
+
+	git rev-parse first second third tip no-conflict-branch >expect-oids &&
+
+	(
+		set_cat_todo_editor &&
+		test_must_fail git rebase -i --update-refs base >todo.raw &&
+		sed -e "/^update-ref/d" <todo.raw >todo
+	) &&
+	(
+		set_replace_editor todo &&
+		git rebase -i --update-refs base
+	) &&
+
+	# Ensure refs are not deleted and their OIDs have not changed
+	git rev-parse first second third tip no-conflict-branch >actual-oids &&
+	test_cmp expect-oids actual-oids
+'
+
+test_expect_success '--update-refs: all update-ref lines removed, then some re-added' '
+	git checkout -b test-refs-not-removed2 no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f tip &&
+
+	test_commit test-refs-not-removed2 &&
+	git commit --amend --fixup first &&
+
+	git rev-parse first second third >expect-oids &&
+
+	(
+		set_cat_todo_editor &&
+		test_must_fail git rebase -i \
+			--autosquash --update-refs \
+			base >todo.raw &&
+		sed -e "/^update-ref/d" <todo.raw >todo
+	) &&
+
+	# Add a break to the end of the todo so we can edit later
+	echo "break" >>todo &&
+
+	(
+		set_replace_editor todo &&
+		git rebase -i --autosquash --update-refs base &&
+		echo "update-ref refs/heads/tip" >todo &&
+		git rebase --edit-todo &&
+		git rebase --continue
+	) &&
+
+	# Ensure first/second/third are unchanged, but tip is updated
+	git rev-parse first second third >actual-oids &&
+	test_cmp expect-oids actual-oids &&
+	test_cmp_rev HEAD tip
+'
+
+test_expect_success '--update-refs: --edit-todo with no update-ref lines' '
+	git checkout -b test-refs-not-removed3 no-conflict-branch &&
+	git branch -f base HEAD~4 &&
+	git branch -f first HEAD~3 &&
+	git branch -f second HEAD~3 &&
+	git branch -f third HEAD~1 &&
+	git branch -f tip &&
+
+	test_commit test-refs-not-removed3 &&
+	git commit --amend --fixup first &&
+
+	git rev-parse first second third tip no-conflict-branch >expect-oids &&
+
+	(
+		set_cat_todo_editor &&
+		test_must_fail git rebase -i \
+			--autosquash --update-refs \
+			base >todo.raw &&
+		sed -e "/^update-ref/d" <todo.raw >todo
+	) &&
+
+	# Add a break to the beginning of the todo so we can resume with no
+	# update-ref lines
+	echo "break" >todo.new &&
+	cat todo >>todo.new &&
+
+	(
+		set_replace_editor todo.new &&
+		git rebase -i --autosquash --update-refs base &&
+
+		# Make no changes when editing so update-refs is still empty
+		cat todo >todo.new &&
+		git rebase --edit-todo &&
+		git rebase --continue
+	) &&
+
+	# Ensure refs are not deleted and their OIDs have not changed
+	git rev-parse first second third tip no-conflict-branch >actual-oids &&
+	test_cmp expect-oids actual-oids
+'
+
  test_expect_success '--update-refs: check failed ref update' '
  	git checkout -B update-refs-error no-conflict-branch &&
  	git branch -f base HEAD~4 &&
--

2.38.0