Skip to contentSkip to conversation

reftable: wrong HEAD reflog when its target is updated in the same transaction

PATCH New 井上司 2 participants 5 messages Nov 8 – Nov 10

lore
Share & tools
lore

井上司

author
#0 lore raw

Hi,

With the reftable backend, if one transaction updates a branch and points HEAD at it, the new HEAD reflog entry records the branch's old value. If the branch is created in the same transaction, no HEAD reflog entry is written at all. The files backend records the new value in both cases.

To reproduce on current master (fb255012ac):

  git init -q --ref-format=reftable repo && cd repo
  git commit -q --allow-empty -m one
  git commit -q --allow-empty -m two
  git branch y HEAD~
  printf '%s\n' start "update refs/heads/y $(git rev-parse HEAD)" \
          'option no-deref' 'symref-update HEAD refs/heads/y' commit |
  git update-ref -m test --stdin >/dev/null
  git log -1 --format='%h %s' HEAD
  git reflog -1 --format='%h %gs' HEAD

HEAD is at "two", but the "test" reflog entry points at "one". With --ref-format=files it points at "two".

Using "create refs/heads/z <oid>" instead of updating y leaves HEAD's reflog without a new entry.

I think the cause is write_transaction_table() in refs/reftable-backend.c. For a symref update it gets the new oid for the log record with refs_resolve_ref_unsafe() on new_target, which reads the refs as they were before the transaction. So it sees the old value of y, or no y at all, in which case the update is treated as a dangling symref and the reflog entry is skipped.

I ran into this while looking at updating a new branch and HEAD in one transaction for "git switch -c". Would taking the new oid from the target's update in the same transaction, when there is one, be the right fix? I can try to write a patch.

Thanks,
Tsukasa

#1 lore raw
Quoted 19 lines · 井上司

Hi,

With the reftable backend, if one transaction updates a branch and points HEAD at it, the new HEAD reflog entry records the branch's old value. If the branch is created in the same transaction, no HEAD reflog entry is written at all. The files backend records the new value in both cases.

To reproduce on current master (fb255012ac):

  git init -q --ref-format=reftable repo && cd repo
  git commit -q --allow-empty -m one
  git commit -q --allow-empty -m two
  git branch y HEAD~
  printf '%s\n' start "update refs/heads/y $(git rev-parse HEAD)" \
          'option no-deref' 'symref-update HEAD refs/heads/y' commit |
  git update-ref -m test --stdin >/dev/null
  git log -1 --format='%h %s' HEAD
  git reflog -1 --format='%h %gs' HEAD
Quoted 6 lines

HEAD is at "two", but the "test" reflog entry points at "one". With --ref-format=files it points at "two".

Using "create refs/heads/z <oid>" instead of updating y leaves HEAD's reflog without a new entry.

With the files backend, this isn't an issue only because the files backend doesn't support atomicity. If it did, we'd have the same bug there too ;)

Quoted 7 lines

I think the cause is write_transaction_table() in refs/reftable-backend.c. For a symref update it gets the new oid for the log record with refs_resolve_ref_unsafe() on new_target, which reads the refs as they were before the transaction. So it sees the old value of y, or no y at all, in which case the update is treated as a dangling symref and the reflog entry is skipped.

I haven't looked at the code, but your inference seems to make sense to me.

Quoted 5 lines

I ran into this while looking at updating a new branch and HEAD in one transaction for "git switch -c". Would taking the new oid from the target's update in the same transaction, when there is one, be the right fix? I can try to write a patch.

That would definitely work, but this same bug would exist with any new backend that is introduced and supports atomic transactions.

I wonder if we can move move reflog logic to the generic 'refs.c' layer. So for a given transaction, 'refs.c' can already parse all the updates and also set the reflog values. Then the individual backends would simply receive the reflog entry to create and wouldn't have to worry about solving inter-transaction conflict resoultion.

But that might be a complex undertaking. I prefer the generic approach, but your approach would also definitely fix the issue and would be a very welcome fix.

Thanks,
Karthik

author
#2 lore raw
Quoted 3 lines · Karthik Nayak

With the files backend, this isn't an issue only because the files backend doesn't support atomicity. If it did, we'd have the same bug there too ;)

Thanks for looking at this so quickly. You're right, and it shows up with files already. files commits the updates one by one in the order they were queued, so the reflog is right only when the branch update comes before HEAD. With "symref-update HEAD" first, files also records the old oid, or no entry when the branch is created. I was wrong to say files gets it right in both cases.

Quoted 1 line

I wonder if we can move move reflog logic to the generic 'refs.c' layer.

I like that direction. As a first step toward it, I'd like to add a helper in refs.c that returns the new oid of a ref updated in the same transaction, and have both backends use it before falling back to refs_resolve_ref_unsafe(). That fixes both backends whatever the order, and one test can cover both. Moving the rest of the reflog logic to refs.c could then come on top of that.

Does that sound reasonable?

Thanks,
Tsukasa

2026年11月9日(月) 7:18 Karthik Nayak <[email protected]>:

Quoted 65 lines
Quoted 19 lines · 井上司

Hi,

With the reftable backend, if one transaction updates a branch and points HEAD at it, the new HEAD reflog entry records the branch's old value. If the branch is created in the same transaction, no HEAD reflog entry is written at all. The files backend records the new value in both cases.

To reproduce on current master (fb255012ac):

  git init -q --ref-format=reftable repo && cd repo
  git commit -q --allow-empty -m one
  git commit -q --allow-empty -m two
  git branch y HEAD~
  printf '%s\n' start "update refs/heads/y $(git rev-parse HEAD)" \
          'option no-deref' 'symref-update HEAD refs/heads/y' commit |
  git update-ref -m test --stdin >/dev/null
  git log -1 --format='%h %s' HEAD
  git reflog -1 --format='%h %gs' HEAD
Quoted 6 lines

HEAD is at "two", but the "test" reflog entry points at "one". With --ref-format=files it points at "two".

Using "create refs/heads/z <oid>" instead of updating y leaves HEAD's reflog without a new entry.

With the files backend, this isn't an issue only because the files backend doesn't support atomicity. If it did, we'd have the same bug there too ;)

Quoted 7 lines

I think the cause is write_transaction_table() in refs/reftable-backend.c. For a symref update it gets the new oid for the log record with refs_resolve_ref_unsafe() on new_target, which reads the refs as they were before the transaction. So it sees the old value of y, or no y at all, in which case the update is treated as a dangling symref and the reflog entry is skipped.

I haven't looked at the code, but your inference seems to make sense to me.

Quoted 5 lines

I ran into this while looking at updating a new branch and HEAD in one transaction for "git switch -c". Would taking the new oid from the target's update in the same transaction, when there is one, be the right fix? I can try to write a patch.

That would definitely work, but this same bug would exist with any new backend that is introduced and supports atomic transactions.

I wonder if we can move move reflog logic to the generic 'refs.c' layer. So for a given transaction, 'refs.c' can already parse all the updates and also set the reflog values. Then the individual backends would simply receive the reflog entry to create and wouldn't have to worry about solving inter-transaction conflict resoultion.

But that might be a complex undertaking. I prefer the generic approach, but your approach would also definitely fix the issue and would be a very welcome fix.

Thanks,
Karthik

#3 lore raw
Quoted 6 lines · 井上司
Quoted 3 lines · Karthik Nayak

With the files backend, this isn't an issue only because the files backend doesn't support atomicity. If it did, we'd have the same bug there too ;)

Quoted 7 lines

Thanks for looking at this so quickly. You're right, and it shows up with files already. files commits the updates one by one in the order they were queued, so the reflog is right only when the branch update comes before HEAD. With "symref-update HEAD" first, files also records the old oid, or no entry when the branch is created. I was wrong to say files gets it right in both cases.

Exactly. So it's a bigger bug than just reftables.

Quoted 11 lines
Quoted 1 line

I wonder if we can move move reflog logic to the generic 'refs.c' layer.

I like that direction. As a first step toward it, I'd like to add a helper in refs.c that returns the new oid of a ref updated in the same transaction, and have both backends use it before falling back to refs_resolve_ref_unsafe(). That fixes both backends whatever the order, and one test can cover both. Moving the rest of the reflog logic to refs.c could then come on top of that.

Does that sound reasonable?

Sounds like a good start to me! Looking forward to the patch. To also add more context, we've also been thinking about doing conflict resolution for updates within a given transaction for a while now. This seems to play into that.

So a transaction like:

delete refs/heads/foo
create refs/heads/foo HEAD

would currently fail, which doesn't have to, but I haven't been able to put some time into this yet.

Quoted 4 lines

Thanks,
Tsukasa

2026年11月9日(月) 7:18 Karthik Nayak <[email protected]>:

[snip]

#4 lore raw

PATCH 1/1 refs: fix reflog of a symref whose target is in the same transaction

When a transaction updates a ref and also points a symref at it, for example

update refs/heads/y <new>
symref-update HEAD refs/heads/y

the reflog entry for HEAD should record <new>. Both backends get the value for that entry by resolving the target with refs_resolve_ref_unsafe(), which reads the refs as they are before the transaction is applied.

The reftable backend writes all updates into one table at the end, so it always sees the old value of y. When y is created by the same transaction it sees no y at all, takes the symref as dangling, and writes no reflog entry. The files backend commits the updates one at a time in the order they were queued, so it gets it right only when the update of y comes before the one of HEAD.

Add ref_transaction_resolve_new_oid(), which takes the new value from the transaction when it updates the target, following symref updates within it, and reads the ref store otherwise. Use it in both backends.

---

Notes:
This follows up on the report at
https://lore.kernel.org/git/CAP8tz2L3Dfhd+ec4+SFkZgXmzQnBg+3Y0REAV+1Hf5X415qnfw@mail.gmail.com/

As discussed with Karthik there, this only moves the lookup of the target's new value to refs.c. Moving the rest of the reflog logic out of the backends, and resolving conflicting updates within a single transaction, are left for later.

 refs.c                  | 47 +++++++++++++++++++++++++++++++++++++++++
 refs/files-backend.c    | 10 +++++----
 refs/refs-internal.h    | 12 +++++++++++
 refs/reftable-backend.c |  9 ++++++--
 t/t1400-update-ref.sh   | 36 +++++++++++++++++++++++++++++++
 5 files changed, 108 insertions(+), 6 deletions(-)
refs.c +47 -0
index 951db56113..3a5c225fc9 100644
@@ -1388,6 +1388,53 @@ struct ref_update *ref_transaction_add_update(
 	return update;
 }
 
+static struct ref_update *find_ref_update(struct ref_transaction *transaction,
+					  const char *refname)
+{
+	size_t i;
+
+	for (i = 0; i < transaction->nr; i++) {
+		struct ref_update *update = transaction->updates[i];
+
+		if (!(update->flags & REF_LOG_ONLY) &&
+		    !strcmp(update->refname, refname))
+			return update;
+	}
+
+	return NULL;
+}
+
+int ref_transaction_resolve_new_oid(struct ref_transaction *transaction,
+				    const char *refname,
+				    struct object_id *oid)
+{
+	int depth;
+
+	for (depth = 0; depth < SYMREF_MAXDEPTH; depth++) {
+		struct ref_update *update;
+
+		update = find_ref_update(transaction, refname);
+		if (!update)
+			break;
+
+		if (!(update->flags & REF_HAVE_NEW) || update->rejection_err)
+			break;
+
+		if (update->new_target) {
+			refname = update->new_target;
+			continue;
+		}
+
+		if (is_null_oid(&update->new_oid))
+			return 0;
+		oidcpy(oid, &update->new_oid);
+		return 1;
+	}
+
+	return !!refs_resolve_ref_unsafe(transaction->ref_store, refname,
+					 RESOLVE_REF_READING, oid, NULL);
+}
+
 static int transaction_refname_valid(const char *refname,
 				     const struct object_id *new_oid,
 				     unsigned int flags, struct strbuf *err)
refs/files-backend.c +6 -4
index 87d04557a5..d8878f4df3 100644
@@ -3156,6 +3156,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 }
 
 static int parse_and_write_reflog(struct files_ref_store *refs,
+				  struct ref_transaction *transaction,
 				  struct ref_update *update,
 				  struct ref_lock *lock,
 				  struct strbuf *err)
@@ -3179,9 +3180,9 @@ static int parse_and_write_reflog(struct files_ref_store *refs,
 		 * We want to get the resolved OID for the target, to ensure
 		 * that the correct value is added to the reflog.
 		 */
-		if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
-					     RESOLVE_REF_READING,
-					     &update->new_oid, NULL)) {
+		if (!ref_transaction_resolve_new_oid(transaction,
+						     update->new_target,
+						     &update->new_oid)) {
 			/*
 			 * TODO: currently we skip creating reflogs for dangling
 			 * symref updates. It would be nice to capture this as
@@ -3377,7 +3378,8 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
-			if (parse_and_write_reflog(refs, update, lock, err)) {
+			if (parse_and_write_reflog(refs, transaction, update,
+						   lock, err)) {
 				ret = REF_TRANSACTION_ERROR_GENERIC;
 				goto cleanup;
 			}
refs/refs-internal.h +12 -0
index c3ac7b556f..33ba285dc4 100644
@@ -187,6 +187,18 @@ struct ref_update *ref_transaction_add_update(
 		const char *committer_info,
 		const char *msg);
 
+/*
+ * Resolve `refname` to the object ID it will have once `transaction` is
+ * committed: if the transaction updates `refname`, use the new value of
+ * that update (following `new_target` of symref updates in the
+ * transaction), otherwise read it from the ref store. Return 1 and fill
+ * `oid` if it resolves to an object, 0 if it would be dangling (missing,
+ * or deleted by the transaction). Meant to be used while committing.
+ */
+int ref_transaction_resolve_new_oid(struct ref_transaction *transaction,
+				    const char *refname,
+				    struct object_id *oid);
+
 /*
  * Transaction states.
  *
refs/reftable-backend.c +7 -2
index d0de066355..46142a4f73 100644
@@ -940,6 +940,7 @@ struct reftable_transaction_update {
 
 struct write_transaction_table_arg {
 	struct reftable_ref_store *refs;
+	struct ref_transaction *transaction;
 	struct reftable_backend *be;
 	struct reftable_addition *addition;
 	struct reftable_transaction_update *updates;
@@ -950,6 +951,7 @@ struct write_transaction_table_arg {
 };
 
 struct reftable_transaction_data {
+	struct ref_transaction *transaction;
 	struct write_transaction_table_arg *args;
 	size_t args_nr, args_alloc;
 };
@@ -1014,6 +1016,7 @@ static int prepare_transaction_update(struct write_transaction_table_arg **out,
 			   tx_data->args_alloc);
 		arg = &tx_data->args[tx_data->args_nr++];
 		arg->refs = refs;
+		arg->transaction = tx_data->transaction;
 		arg->be = be;
 		arg->addition = addition;
 		arg->updates = NULL;
@@ -1332,6 +1335,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		goto done;
 
 	tx_data = xcalloc(1, sizeof(*tx_data));
+	tx_data->transaction = transaction;
 
 	/*
 	 * Preprocess all updates. For one we check that there are no duplicate
@@ -1557,8 +1561,9 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			int create_reflog = 1;
 
 			if (u->new_target) {
-				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
-							     RESOLVE_REF_READING, &u->new_oid, NULL)) {
+				if (!ref_transaction_resolve_new_oid(arg->transaction,
+								     u->new_target,
+								     &u->new_oid)) {
 					/*
 					 * TODO: currently we skip creating reflogs for dangling
 					 * symref updates. It would be nice to capture this as
t/t1400-update-ref.sh +36 -0
index 1a164c96f9..19f0987d2f 100755
@@ -853,6 +853,42 @@ test_expect_success 'stdin delete symref works option no-deref' '
 	test_cmp expect actual
 '
 
+for action in update create
+do
+	for order in branch-first head-first
+	do
+		test_expect_success "stdin symref-update HEAD logs target's new value ($action, $order)" '
+			test_when_finished "rm -rf repo branch-cmd head-cmd" &&
+			git init repo &&
+			test_commit -C repo one &&
+			test_commit -C repo two &&
+			new=$(git -C repo rev-parse two) &&
+			if test "$action" = update
+			then
+				git -C repo branch topic one
+			fi &&
+			echo "$action refs/heads/topic $new" >branch-cmd &&
+			{
+				echo "option no-deref" &&
+				echo "symref-update HEAD refs/heads/topic"
+			} >head-cmd &&
+			if test "$order" = branch-first
+			then
+				cat branch-cmd head-cmd >stdin
+			else
+				cat head-cmd branch-cmd >stdin
+			fi &&
+			git -C repo update-ref -m "move HEAD" --stdin <stdin &&
+			echo "move HEAD" >expect &&
+			git -C repo reflog -1 --format=%gs HEAD >actual &&
+			test_cmp expect actual &&
+			echo $new >expect &&
+			git -C repo reflog -1 --format=%H HEAD >actual &&
+			test_cmp expect actual
+		'
+	done
+done
+
 test_expect_success 'stdin update symref works flag --no-deref' '
 	git symbolic-ref TESTSYMREFONE $b &&
 	git symbolic-ref TESTSYMREFTWO $b &&
Signature · 1 line
2.56.0