Skip to contentSkip to conversation

update-ref: add support for update-symref option

PATCH In review Karthik Nayak 7 participants 194 messages Mar 30, 2024 – May 13, 2024 Documentation Page 2 of 2

lore
Share & tools
lore
#111 lore raw

PATCH v4 3/7 update-ref: add support for 'symref-verify' command

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we added the required base for adding symref commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using them, add a new 'symref-verify' command to verify symrefs.

The 'symref-verify' command allows users to verify if a provided <ref> contains the provided <old-target> without changing the <ref>. If <old-target> is not provided, the command will verify that the <ref> doesn't exist. Since we're checking for symbolic refs, this command will only work with the 'no-deref' mode. This is because any dereferenced symbolic ref will point to an object and not a ref and the regular 'verify' command can be used in such situations.

Add and use ref_update_is_null_new_value, a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid. We also add tests to test the command in both the regular stdin mode and also with the '-z' flag.

We also disable the reference-transaction hook for symref-updates which will be tackled in its own commit.

Add required tests for symref support in 'verify' while also adding reflog checks for the pre-existing 'verify' tests.

---
 Documentation/git-update-ref.txt |  7 +++
 builtin/update-ref.c             | 80 +++++++++++++++++++++++----
 refs.c                           | 30 +++++++++--
 refs.h                           |  1 +
 refs/files-backend.c             | 43 +++++++++++++++
 refs/refs-internal.h             |  7 +++
 refs/reftable-backend.c          | 21 +++++++-
 t/t1400-update-ref.sh            | 93 +++++++++++++++++++++++++++++++-
 8 files changed, 264 insertions(+), 18 deletions(-)
Documentation/git-update-ref.txt +7 -0
index 374a2ebd2b..9fe78b3501 100644
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-verify SP <ref> [SP <old-target>] LF
 	option SP <opt> LF
 	start LF
 	prepare LF
@@ -86,6 +87,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-verify SP <ref> [NUL <old-target>] NUL
 	option SP <opt> NUL
 	start NUL
 	prepare NUL
@@ -117,6 +119,11 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-verify::
+	Verify symbolic <ref> against <old-target> but do not change it.
+	If <old-target> is missing, the ref must not exist.  Can only be
+	used in `no-deref` mode.
+
 option::
 	Modify the behavior of the next command naming a <ref>.
 	The only valid option is `no-deref` to avoid dereferencing
builtin/update-ref.c +70 -10

102 lines not on this page · raw message

refs.c +25 -5
index 060a31616d..0e1013b5ab 100644
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((void *)transaction->updates[i]->old_target);
+		free((void *)transaction->updates[i]->new_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);
 	update->msg = normalize_reflog_message(msg);
 	return update;
@@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err)
 {
-	if (!old_oid)
-		BUG("verify called with old_oid set to NULL");
+	if (!old_target && !old_oid)
+		BUG("verify called with old_oid and old_target set to NULL");
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("verify cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
-				      NULL, NULL,
+				      NULL, old_target,
 				      flags, NULL, err);
 }
 
@@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
+		/*
+		 * Skip reference transaction for symbolic refs.
+		 */
+		if (update->new_target || update->old_target)
+			continue;
+
 		strbuf_reset(&buf);
 		strbuf_addf(&buf, "%s %s %s\n",
 			    oid_to_hex(&update->old_oid),
@@ -2802,3 +2818,7 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_is_null_new_value(struct ref_update *update) {
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
refs.h +1 -0
index c792e13a64..27b9aeaf54 100644
@@ -780,6 +780,7 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err);
 
refs/files-backend.c +43 -0

55 lines not on this page · raw message

refs/refs-internal.h +7 -0
index 3040d4797c..23e65f65e8 100644
@@ -748,4 +748,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_is_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +20 -1
index 6104471199..a2474245aa 100644
@@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(u->old_target, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "provided target is empty",
+						    original_update_refname(u));
+				else if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}
+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
 					    "reference already exists"),
t/t1400-update-ref.sh +91 -2

112 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#112 lore raw

Re: [PATCH v4 3/7] update-ref: add support for 'symref-verify' command

Quoted 6 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we added the required base for adding symref commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using them, add a new 'symref-verify' command to verify symrefs.

Quoted 7 lines

The 'symref-verify' command allows users to verify if a provided <ref> contains the provided <old-target> without changing the <ref>. If <old-target> is not provided, the command will verify that the <ref> doesn't exist. Since we're checking for symbolic refs, this command will only work with the 'no-deref' mode. This is because any dereferenced symbolic ref will point to an object and not a ref and the regular 'verify' command can be used in such situations.

All makes sense, but a naïve reader may find it helpful if you explained why having "verify" command is a good idea in the first place ("I can just do 'git symoblic-ref' to read the current value, and see if it is what I expect"). Presumably the value of "verify" is that you can have it in a transaction and fail other operations in the same transaction if the symref moved from what you expected it to point at?

Quoted 5 lines

Add and use ref_update_is_null_new_value, a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid. We also add tests to test the command in both the regular stdin mode and also with the '-z' flag.

This looks out of place, primarily because the helper function is *NOT* used in this step. Without any actual user, and with the name that says only what it checks without hinting why a caller may want to check the condition it checks, it is hard to guess if it is a good idea to have such a helper.

"If a ref_update object specifies no new-oid and no new-target, it is not about updating but just validating" is how the callers are expected to use it, then instead of is_null_new_value that says what it checks, something like is_verify_only that says what the caller may want to use it for would be a more friendly name for readers and future developers.

builtin/update-ref.c static void parse_cmd_verify(struct ref_transaction *transaction, · line 328

Quoted 11 lines
builtin/update-ref.c +1 -1
@@ -297,11 +320,47 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 		die("verify %s: extra input: %s", refname, next);
 
 	if (ref_transaction_verify(transaction, refname, &old_oid,
-				   update_flags, &err))
+				   NULL, update_flags, &err))
 
 	update_flags = default_flags;
 	free(refname);
 	strbuf_release(&err);
 }

The only damage by this patch to parse_cmd_verify() is that ref_transaction_verify() gained another parameter NULL, but with the default "--diff-algorithm=myers" algorithm, it is very hard to see.

The "--patience" algorithm does a much beter job on this hunk.

And the following function is entirely new.

Quoted 11 lines
builtin/update-ref.c +11 -0
+static void parse_cmd_symref_verify(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	struct object_id old_oid;
+	char *refname, *old_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-verify: cannot operate with deref mode");
+
+	refname = parse_refname(&next);

builtin/update-ref.c static void parse_cmd_verify(struct ref_transaction *transaction, · line 351

Quoted 10 lines
builtin/update-ref.c +10 -0
+	if (!refname)
+		die("symref-verify: missing <ref>");
+
+	/*
+	 * old_ref is optional, but we want to differentiate between
+	 * a NULL and zero value.
+	 */
+	old_target = parse_next_refname(&next);
+	if (!old_target)
+		old_oid = *null_oid();

In many existing code paths, we do not do structure assignment like this. Instead we do

		oidcpy(&old_oid, null_oid());

We can see an existing example in a common context in a hunk for refs.c in this patch.

builtin/update-ref.c static void parse_cmd_verify(struct ref_transaction *transaction,

Quoted 7 lines
builtin/update-ref.c +7 -0
+	if (*next != line_termination)
+		die("symref-verify %s: extra input: %s", refname, next);
+
+	if (ref_transaction_verify(transaction, refname,
+				   old_target ? NULL : &old_oid,
+				   old_target, update_flags, &err))
+		die("%s", err.buf);

Are static analyzers smart enough to notice that we will not be using old_oid uninitialized here? Just wondering.

Anyway. This ensures ref_transaction_verify() gets either old_target or old_oid, but never both at the same time. The caller to ref_transaction_verify() in the previous function passed NULL for old_target but it always had a non-NULL old_oid so that is perfectly fine.

Quoted 20 lines
builtin/update-ref.c +5 -0
+	update_flags = default_flags;
+	free(refname);
+	free(old_target);
+	strbuf_release(&err);
+}
 
refs.c +2 -0
index 060a31616d..0e1013b5ab 100644
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((void *)transaction->updates[i]->old_target);
+		free((void *)transaction->updates[i]->new_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);

refs.c struct ref_update *ref_transaction_add_update( · line 1255

Quoted 8 lines
refs.c +4 -1
@@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);

Presumably "update" structure, when freshly initialized, has NULL in both of these _target members? Otherwise ref_transaction_free() would get in trouble, so double checking.

refs.c struct ref_update *ref_transaction_add_update( · line 1259

Quoted 5 lines
refs.c +2 -1
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);

Since we can ask to work on a symbolic ref, new_oid / old_oid can be NULL when REF_HAVE_NEW / REF_HAVE_OLD bit is on for _target members.

Makes me wonder if the code becomes easier to follow if the flag bits are split into four (_NEW -> _NEW_OID + _NEW_TARGET), but let's not worry about that for now.

Quoted 8 lines
refs.c +1 -0
@@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
@@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,

refs.c int ref_transaction_delete(struct ref_transaction *transaction, · line 1340

Quoted 10 lines
refs.c +3 -2
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err)
 {
-	if (!old_oid)
-		BUG("verify called with old_oid set to NULL");
+	if (!old_target && !old_oid)
+		BUG("verify called with old_oid and old_target set to NULL");

Is it normal if you get _both_ set, or is it equally a BUG()? The parse_*_verify() codepaths we saw earlier both made sure only one of the two is non-NULL, and it is unclear what should happen if both are non-NULL.

refs.c int ref_transaction_delete(struct ref_transaction *transaction, · line 1347

Quoted 8 lines
refs.c +3 -1
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("verify cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
-				      NULL, NULL,
+				      NULL, old_target,
 				      flags, NULL, err);
 }

So this queues an ref_update object whose .new_oid and .new_target are NULL, and .old_oid and .old_target are what the caller gave us to check. The NULLs in .new* members hopefully do not mean "delete this thing" ;-)

refs.c static int run_transaction_hook(struct ref_transaction *transaction, · line 2366

Quoted 9 lines
refs.c +5 -0
@@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
+		/*
+		 * Skip reference transaction for symbolic refs.
+		 */
+		if (update->new_target || update->old_target)
+			continue;

Is that a final design, or will the hooks have a chance to interfere?

Quoted 10 lines
refs/files-backend.c +2 -0
index 2420dac2aa..53197fa3af 100644
@@ -2425,6 +2425,37 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the REF_HAVE_OLD and old_target values stored in

refs/files-backend.c static const char *original_update_refname(struct ref_update *update) · line 2439

Quoted 10 lines
refs/files-backend.c +10 -0
+ * update are consistent with ref, which is the symbolic reference's
+ * current value. If everything is OK, return 0; otherwise, write an
+ * error message to err and return -1.
+ */
+static int check_old_target(struct ref_update *update, char *ref,
+			    struct strbuf *err)
+{
+	if (!(update->flags & REF_HAVE_OLD) ||
+	    !strcmp(update->old_target, ref))
+		return 0;

Earlier on the assignment side for "update" structure we saw above, the guard was (old_target && flags & REF_HAVE_OLD), but here we assume old_target is valid, which feels a bit asymmetric.

Yes, I can see that the caller does not call us when !old_target, but still... Perhaps

	if ((update->flags & REF_HAVE_OLD) && !update->old_target)
		BUG(...);

or something? Or alternatively, perhaps !!update->old_target should be the only thing we should check and ignore REF_HAVE_OLD bit? I am not sure, but it smells like that the non-NULL-ness of old_target is the only thing that matters (if it is not NULL, very early in the control flow somebody would have set REF_HAVE_OLD bit to flags, no?).

It brings me back to my earlier question. Does REF_HAVE_OLD bit serve a useful purpose in this code?

refs/files-backend.c static const char *original_update_refname(struct ref_update *update) · line 2449

Quoted 9 lines
refs/files-backend.c +9 -0
+	if (!strcmp(update->old_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference already exists",
+			    original_update_refname(update));
+	else if (!strcmp(ref, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);

So... for old_target and ref, an empty string is a special value? How? Shouldn't that be documented in the comment before the function?

Quoted 18 lines
refs/files-backend.c +11 -0
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    ref, update->old_target);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2528,6 +2559,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
+			}
+

refs/files-backend.c static int lock_ref_for_update(struct files_ref_store *refs, · line 2573

Quoted 10 lines
refs/files-backend.c +10 -0
+			/*
+			 * For symref verification, we need to check the reference value
+			 * rather than the oid. If we're dealing with regular refs or we're
+			 * verifying a dereferenced symref, we then check the oid.
+			 */
+			if (update->old_target) {
+				if (check_old_target(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}

We come here only when update->type has REF_ISSYMREF bit on (we learned that value by calling lock_raw_ref()), and know referent.buf has the current "target" value. That is consumed as "ref" parameter to check_old_target() we just saw. OK.

Quoted 18 lines
refs/reftable-backend.c +9 -1
index 6104471199..a2474245aa 100644
@@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(u->old_target, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "provided target is empty",
+						    original_update_refname(u));
+				else if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",

refs/reftable-backend.c static int reftable_be_transaction_prepare(struct ref_store *ref_store, · line 959

Quoted 10 lines
refs/reftable-backend.c +10 -0
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}

Again, the puzzling "empty string"s are handled here.

#113 lore raw

Re: [PATCH v4 3/7] update-ref: add support for 'symref-verify' command

Quoted 16 lines · Junio C Hamano
Quoted 6 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we added the required base for adding symref commands to the '--stdin' mode provided by 'git-update-ref(1)'. Using them, add a new 'symref-verify' command to verify symrefs.

Quoted 7 lines

The 'symref-verify' command allows users to verify if a provided <ref> contains the provided <old-target> without changing the <ref>. If <old-target> is not provided, the command will verify that the <ref> doesn't exist. Since we're checking for symbolic refs, this command will only work with the 'no-deref' mode. This is because any dereferenced symbolic ref will point to an object and not a ref and the regular 'verify' command can be used in such situations.

Quoted 8 lines

All makes sense, but a naïve reader may find it helpful if you explained why having "verify" command is a good idea in the first place ("I can just do 'git symoblic-ref' to read the current value, and see if it is what I expect"). Presumably the value of "verify" is that you can have it in a transaction and fail other operations in the same transaction if the symref moved from what you expected it to point at?

I would say none of the commits drive this point, and I would go ahead and add something on these lines to each of them. I think it would add good value to readers.

Quoted 6 lines
Quoted 5 lines

Add and use ref_update_is_null_new_value, a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid. We also add tests to test the command in both the regular stdin mode and also with the '-z' flag.

Quoted 6 lines

This looks out of place, primarily because the helper function is *NOT* used in this step. Without any actual user, and with the name that says only what it checks without hinting why a caller may want to check the condition it checks, it is hard to guess if it is a good idea to have such a helper.

I think over the revision, its usage from this commit was removed. It makes sense to move it to a commit where its used, I'll do that.

Quoted 6 lines

"If a ref_update object specifies no new-oid and no new-target, it is not about updating but just validating" is how the callers are expected to use it, then instead of is_null_new_value that says what it checks, something like is_verify_only that says what the caller may want to use it for would be a more friendly name for readers and future developers.

This is true for the old-oid and old-target. That is, when they are set to null, we're validating.

With the new-oid and new-target, if they're null, it usually signifies deletion. We could rename it to 'is_delete_only', but that would also need checking the 'REF_HAVE_NEW' flag. So we could ideally change it to

```
int ref_update_is_delete_only(struct ref_update *update) {
	return (update->flags & REF_HAVE_NEW) && !update->new_target &&
is_null_oid(&update->new_oid);
}
```

I'm okay with making this change.

Quoted 45 lines

builtin/update-ref.c static void parse_cmd_verify(struct ref_transaction *transaction, · line 328

Quoted 11 lines
builtin/update-ref.c +1 -1
@@ -297,11 +320,47 @@ static void parse_cmd_verify(struct ref_transaction *transaction,
 		die("verify %s: extra input: %s", refname, next);
 
 	if (ref_transaction_verify(transaction, refname, &old_oid,
-				   update_flags, &err))
+				   NULL, update_flags, &err))
 
 	update_flags = default_flags;
 	free(refname);
 	strbuf_release(&err);
 }

The only damage by this patch to parse_cmd_verify() is that ref_transaction_verify() gained another parameter NULL, but with the default "--diff-algorithm=myers" algorithm, it is very hard to see.

The "--patience" algorithm does a much beter job on this hunk.

And the following function is entirely new.

Quoted 11 lines
builtin/update-ref.c +11 -0
+static void parse_cmd_symref_verify(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	struct object_id old_oid;
+	char *refname, *old_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-verify: cannot operate with deref mode");
+
+	refname = parse_refname(&next);

builtin/update-ref.c static void parse_cmd_verify(struct ref_transaction *transaction, · line 351

Quoted 10 lines
builtin/update-ref.c +10 -0
+	if (!refname)
+		die("symref-verify: missing <ref>");
+
+	/*
+	 * old_ref is optional, but we want to differentiate between
+	 * a NULL and zero value.
+	 */
+	old_target = parse_next_refname(&next);
+	if (!old_target)
+		old_oid = *null_oid();

In many existing code paths, we do not do structure assignment like this. Instead we do

Quoted 5 lines
		oidcpy(&old_oid, null_oid());

We can see an existing example in a common context in a hunk for refs.c in this patch.

Yeah, makes sense to switch this. Will do.

Quoted 10 lines

builtin/update-ref.c static void parse_cmd_verify(struct ref_transaction *transaction,

Quoted 7 lines
builtin/update-ref.c +7 -0
+	if (*next != line_termination)
+		die("symref-verify %s: extra input: %s", refname, next);
+
+	if (ref_transaction_verify(transaction, refname,
+				   old_target ? NULL : &old_oid,
+				   old_target, update_flags, &err))
+		die("%s", err.buf);

Are static analyzers smart enough to notice that we will not be using old_oid uninitialized here? Just wondering.

Yup, at least the clang LSP server seems to detect and not bug me about it.

Quoted 29 lines

Anyway. This ensures ref_transaction_verify() gets either old_target or old_oid, but never both at the same time. The caller to ref_transaction_verify() in the previous function passed NULL for old_target but it always had a non-NULL old_oid so that is perfectly fine.

Quoted 23 lines
builtin/update-ref.c +5 -0
+	update_flags = default_flags;
+	free(refname);
+	free(old_target);
+	strbuf_release(&err);
+}
 
refs.c +2 -0
index 060a31616d..0e1013b5ab 100644
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((void *)transaction->updates[i]->old_target);
+		free((void *)transaction->updates[i]->new_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,9 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
Quoted 10 lines

refs.c struct ref_update *ref_transaction_add_update( · line 1255

Quoted 5 lines
refs.c +4 -1
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);

Presumably "update" structure, when freshly initialized, has NULL in both of these _target members? Otherwise ref_transaction_free() would get in trouble, so double checking.

This is a good point. My understanding was that FLEX_ALLOC_MEM should set everything to 0.

Quoted 9 lines

refs.c struct ref_update *ref_transaction_add_update( · line 1259

Quoted 5 lines
refs.c +2 -1
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);

Since we can ask to work on a symbolic ref, new_oid / old_oid can be NULL when REF_HAVE_NEW / REF_HAVE_OLD bit is on for _target members.

Quoted 4 lines

Makes me wonder if the code becomes easier to follow if the flag bits are split into four (_NEW -> _NEW_OID + _NEW_TARGET), but let's not worry about that for now.

The intersection of this is quite low currently, so I'm not really sure if there's added benefit. I did start that way before, but perhaps with the iterations in the last few version, maybe it makes the code simpler.

Quoted 15 lines
Quoted 15 lines
refs.c +2 -1
@@ -1286,6 +1292,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
@@ -1325,14 +1332,17 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 int ref_transaction_verify(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
+			   const char *old_target,
 			   unsigned int flags,
 			   struct strbuf *err)
 {
-	if (!old_oid)
Quoted 9 lines

refs.c int ref_transaction_delete(struct ref_transaction *transaction, · line 1340

Quoted 3 lines
refs.c +2 -1
-		BUG("verify called with old_oid set to NULL");
+	if (!old_target && !old_oid)
+		BUG("verify called with old_oid and old_target set to NULL");

Is it normal if you get _both_ set, or is it equally a BUG()? The parse_*_verify() codepaths we saw earlier both made sure only one of the two is non-NULL, and it is unclear what should happen if both are non-NULL.

It is a bug and this is caught in ref_transaction_add_update. Introduced in the first patch of the series.

Quoted 5 lines
Quoted 5 lines
refs.c +2 -1
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("verify cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
-				      NULL, NULL,
Quoted 9 lines

refs.c int ref_transaction_delete(struct ref_transaction *transaction, · line 1347

Quoted 3 lines
refs.c +1 -0
+				      NULL, old_target,
 				      flags, NULL, err);
 }

So this queues an ref_update object whose .new_oid and .new_target are NULL, and .old_oid and .old_target are what the caller gave us to check. The NULLs in .new* members hopefully do not mean "delete this thing" ;-)

So the 'new_oid' being set to zero should be the delete this thing queue.

Quoted 4 lines
Quoted 4 lines
refs.c +0 -0
@@ -2349,6 +2359,12 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
 
Quoted 8 lines

refs.c static int run_transaction_hook(struct ref_transaction *transaction, · line 2366

Quoted 5 lines
refs.c +5 -0
+		/*
+		 * Skip reference transaction for symbolic refs.
+		 */
+		if (update->new_target || update->old_target)
+			continue;

Is that a final design, or will the hooks have a chance to interfere?

The last patch adds hook support.

Quoted 25 lines
Quoted 10 lines
refs/files-backend.c +2 -0
index 2420dac2aa..53197fa3af 100644
@@ -2425,6 +2425,37 @@ static const char *original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+/*
+ * Check whether the REF_HAVE_OLD and old_target values stored in

refs/files-backend.c static const char *original_update_refname(struct ref_update *update) · line 2439

Quoted 10 lines
refs/files-backend.c +10 -0
+ * update are consistent with ref, which is the symbolic reference's
+ * current value. If everything is OK, return 0; otherwise, write an
+ * error message to err and return -1.
+ */
+static int check_old_target(struct ref_update *update, char *ref,
+			    struct strbuf *err)
+{
+	if (!(update->flags & REF_HAVE_OLD) ||
+	    !strcmp(update->old_target, ref))
+		return 0;

Earlier on the assignment side for "update" structure we saw above, the guard was (old_target && flags & REF_HAVE_OLD), but here we assume old_target is valid, which feels a bit asymmetric.

Quoted 6 lines

Yes, I can see that the caller does not call us when !old_target, but still... Perhaps

	if ((update->flags & REF_HAVE_OLD) && !update->old_target)
		BUG(...);

I will add something like this.

Quoted 9 lines

or something? Or alternatively, perhaps !!update->old_target should be the only thing we should check and ignore REF_HAVE_OLD bit? I am not sure, but it smells like that the non-NULL-ness of old_target is the only thing that matters (if it is not NULL, very early in the control flow somebody would have set REF_HAVE_OLD bit to flags, no?).

It brings me back to my earlier question. Does REF_HAVE_OLD bit serve a useful purpose in this code?

I checked and it doesn't, it can be removed from usage in this code. Will cleanup this part.

Quoted 70 lines

refs/files-backend.c static const char *original_update_refname(struct ref_update *update) · line 2449

Quoted 9 lines
refs/files-backend.c +9 -0
+	if (!strcmp(update->old_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference already exists",
+			    original_update_refname(update));
+	else if (!strcmp(ref, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);

So... for old_target and ref, an empty string is a special value? How? Shouldn't that be documented in the comment before the function?

Quoted 18 lines
refs/files-backend.c +11 -0
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    ref, update->old_target);
+
+	return -1;
+}
+
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2528,6 +2559,18 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
+			}
+

refs/files-backend.c static int lock_ref_for_update(struct files_ref_store *refs, · line 2573

Quoted 10 lines
refs/files-backend.c +10 -0
+			/*
+			 * For symref verification, we need to check the reference value
+			 * rather than the oid. If we're dealing with regular refs or we're
+			 * verifying a dereferenced symref, we then check the oid.
+			 */
+			if (update->old_target) {
+				if (check_old_target(update, referent.buf, err)) {
+					ret = TRANSACTION_GENERIC_ERROR;
+					goto out;
+				}

We come here only when update->type has REF_ISSYMREF bit on (we learned that value by calling lock_raw_ref()), and know referent.buf has the current "target" value. That is consumed as "ref" parameter to check_old_target() we just saw. OK.

Quoted 22 lines
refs/reftable-backend.c +13 -1
index 6104471199..a2474245aa 100644
@@ -938,7 +938,26 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		 * individual refs. But the error messages match what the files
 		 * backend returns, which keeps our tests happy.
 		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if ((u->flags & REF_HAVE_OLD) && u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {
+				if (!strcmp(u->old_target, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "provided target is empty",
+						    original_update_refname(u));
+				else if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
Quoted 8 lines

refs/reftable-backend.c static int reftable_be_transaction_prepare(struct ref_store *ref_store, · line 959

Quoted 6 lines
refs/reftable-backend.c +6 -0
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
+				ret = -1;
+				goto done;
+			}

Again, the puzzling "empty string"s are handled here.

For here and above, this too is dead code and no longer needed, old_target being empty string is left over code from before we decided to use zero_oid for deleting. I'll remove it. Thanks.

#114 lore raw

PATCH v4 4/7 update-ref: add support for 'symref-delete' command

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

Add a new command 'symref-delete' to allow deletions of symbolic refs in a transaction via the '--stdin' mode of the 'git-update-ref' command. The 'symref-delete' command can, when given an <old-target>, delete the provided <ref> only when it points to <old-target>. This will only work when used with the 'no-deref' mode as it doesn't make sense to deref a symref during deletion.

---
 Documentation/git-update-ref.txt | 11 ++++--
 builtin/fetch.c                  |  2 +-
 builtin/receive-pack.c           |  3 +-
 builtin/update-ref.c             | 33 ++++++++++++++++-
 refs.c                           | 12 ++++---
 refs.h                           |  4 ++-
 refs/files-backend.c             |  2 +-
 refs/reftable-backend.c          |  2 +-
 t/t1400-update-ref.sh            | 61 +++++++++++++++++++++++++++++++-
 9 files changed, 117 insertions(+), 13 deletions(-)
Documentation/git-update-ref.txt +9 -2
index 9fe78b3501..2924b9437e 100644
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-delete SP <ref> [SP <old-target>] LF
 	symref-verify SP <ref> [SP <old-target>] LF
 	option SP <opt> LF
 	start LF
@@ -87,6 +88,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-delete SP <ref> [NUL <old-target>] NUL
 	symref-verify SP <ref> [NUL <old-target>] NUL
 	option SP <opt> NUL
 	start NUL
@@ -112,13 +114,18 @@ create::
 	exist.  The given <new-oid> may not be zero.
 
 delete::
-	Delete <ref> after verifying it exists with <old-oid>, if
-	given.  If given, <old-oid> may not be zero.
+	Delete <ref> after verifying it exists with <old-oid>, if given.
+	If given, <old-oid> may not be zero.  If instead, ref:<old-target>
+	is provided, verify that the symbolic ref <ref> targets
+	<old-target> before deleting it.
 
 verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-delete::
+	Delete <ref> after verifying it exists with <old-target>, if given.
+
 symref-verify::
 	Verify symbolic <ref> against <old-target> but do not change it.
 	If <old-target> is missing, the ref must not exist.  Can only be
builtin/fetch.c +1 -1
index 66840b7c5b..d02592efca 100644
@@ -1383,7 +1383,7 @@ static int prune_refs(struct display_state *display_state,
 		if (transaction) {
 			for (ref = stale_refs; ref; ref = ref->next) {
 				result = ref_transaction_delete(transaction, ref->name, NULL, 0,
-								"fetch: prune", &err);
+								NULL, "fetch: prune", &err);
 				if (result)
 					goto cleanup;
 			}
builtin/receive-pack.c +2 -1
index b150ef39a8..9a4667d57d 100644
@@ -1576,7 +1576,8 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_delete(transaction,
 					   namespaced_name,
 					   old_oid,
-					   0, "push", &err)) {
+					   0, NULL,
+					   "push", &err)) {
 			rp_error("%s", err.buf);
 			ret = "failed to delete";
 		} else {
builtin/update-ref.c +32 -1
index 419b28169b..8fef3aed0a 100644
@@ -293,7 +293,7 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 
 	if (ref_transaction_delete(transaction, refname,
 				   have_old ? &old_oid : NULL,
-				   update_flags, msg, &err))
+				   update_flags, NULL, msg, &err))
 		die("%s", err.buf);
 
 	update_flags = default_flags;
@@ -301,6 +301,36 @@ static void parse_cmd_delete(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+
+static void parse_cmd_symref_delete(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *old_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-delete: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-delete: missing <ref>");
+
+	old_target = parse_next_refname(&next);
+
+	if (*next != line_termination)
+		die("symref-delete %s: extra input: %s", refname, next);
+
+	if (ref_transaction_delete(transaction, refname, NULL,
+				   update_flags, old_target, msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(old_target);
+	strbuf_release(&err);
+}
+
+
 static void parse_cmd_verify(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -443,6 +473,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
 	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
 	{ "start",         parse_cmd_start,         0, UPDATE_REFS_STARTED },
refs.c +8 -4
index 0e1013b5ab..6b7c46bfd8 100644
@@ -979,7 +979,7 @@ int refs_delete_ref(struct ref_store *refs, const char *msg,
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
 	    ref_transaction_delete(transaction, refname, old_oid,
-				   flags, msg, &err) ||
+				   flags, NULL, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ref_transaction_free(transaction);
@@ -1318,14 +1318,18 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_target,
+			   const char *msg,
 			   struct strbuf *err)
 {
 	if (old_oid && is_null_oid(old_oid))
 		BUG("delete called with old_oid set to zeros");
+	if (old_target && !(flags & REF_NO_DEREF))
+		BUG("delete cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      NULL, NULL, flags,
+				      NULL, old_target, flags,
 				      msg, err);
 }
 
@@ -2752,7 +2756,7 @@ int refs_delete_refs(struct ref_store *refs, const char *logmsg,
 
 	for_each_string_list_item(item, refnames) {
 		ret = ref_transaction_delete(transaction, item->string,
-					     NULL, flags, msg, &err);
+					     NULL, flags, NULL, msg, &err);
 		if (ret) {
 			warning(_("could not delete reference %s: %s"),
 				item->string, err.buf);
refs.h +3 -1
index 27b9aeaf54..4be4930f04 100644
@@ -766,7 +766,9 @@ int ref_transaction_create(struct ref_transaction *transaction,
 int ref_transaction_delete(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *old_oid,
-			   unsigned int flags, const char *msg,
+			   unsigned int flags,
+			   const char *old_target,
+			   const char *msg,
 			   struct strbuf *err);
 
 /*
refs/files-backend.c +1 -1
index 53197fa3af..fc5037fe5a 100644
@@ -2516,7 +2516,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 	files_assert_main_repository(refs, "lock_ref_for_update");
 
-	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
+	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
 		update->flags |= REF_DELETING;
 
 	if (head_ref) {
refs/reftable-backend.c +1 -1
index a2474245aa..2b2cbca8c0 100644
@@ -1120,7 +1120,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
t/t1400-update-ref.sh +60 -1

73 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#115 lore raw

PATCH v4 5/7 update-ref: add support for 'symref-create' command

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

Add 'symref-create' command to the '--stdin' mode 'git-update-ref' to allow creation of symbolic refs in a transaction. The 'symref-create' command takes in a <new-target>, which the created <ref> will point to.

Also, support the 'core.prefersymlinkrefs' config, wherein if the config is set and the filesystem supports symlinks, we create the symbolic ref as a symlink. We fallback to creating a regular symref if creating the symlink is unsuccessful.

---
 Documentation/git-update-ref.txt | 10 ++++-
 builtin/clone.c                  |  2 +-
 builtin/update-ref.c             | 35 ++++++++++++++++-
 refs.c                           |  9 +++--
 refs.h                           |  1 +
 refs/files-backend.c             | 42 +++++++++++++++++++++
 refs/reftable-backend.c          | 23 +++++++++--
 t/t0600-reffiles-backend.sh      | 32 ++++++++++++++++
 t/t1400-update-ref.sh            | 65 ++++++++++++++++++++++++++++++++
 9 files changed, 210 insertions(+), 9 deletions(-)
Documentation/git-update-ref.txt +9 -1
index 2924b9437e..7a33f70767 100644
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-create SP <ref> SP <new-target> LF
 	symref-delete SP <ref> [SP <old-target>] LF
 	symref-verify SP <ref> [SP <old-target>] LF
 	option SP <opt> LF
@@ -88,6 +89,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-create SP <ref> NUL <new-target> NUL
 	symref-delete SP <ref> [NUL <old-target>] NUL
 	symref-verify SP <ref> [NUL <old-target>] NUL
 	option SP <opt> NUL
@@ -111,7 +113,9 @@ update::
 
 create::
 	Create <ref> with <new-oid> after verifying it does not
-	exist.  The given <new-oid> may not be zero.
+	exist.  The given <new-oid> may not be zero.  If instead
+	ref:<new-target> is provided, a symbolic ref is created
+	which targets <new-target>.
 
 delete::
 	Delete <ref> after verifying it exists with <old-oid>, if given.
@@ -123,6 +127,10 @@ verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
 
+symref-create::
+	Create symbolic ref <ref> with <new-target> after verifying
+	it does not exist.  Can only be used in `no-deref` mode.
+
 symref-delete::
 	Delete <ref> after verifying it exists with <old-target>, if given.
 
builtin/clone.c +1 -1
index 74ec14542e..c0eed8e795 100644
@@ -547,7 +547,7 @@ static void write_remote_refs(const struct ref *local_refs)
 		if (!r->peer_ref)
 			continue;
 		if (ref_transaction_create(t, r->peer_ref->name, &r->old_oid,
-					   0, NULL, &err))
+					   NULL, 0, NULL, &err))
 			die("%s", err.buf);
 	}
 
builtin/update-ref.c +34 -1
index 8fef3aed0a..ae68ffde5e 100644
@@ -257,7 +257,7 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	if (*next != line_termination)
 		die("create %s: extra input: %s", refname, next);
 
-	if (ref_transaction_create(transaction, refname, &new_oid,
+	if (ref_transaction_create(transaction, refname, &new_oid, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
@@ -267,6 +267,38 @@ static void parse_cmd_create(struct ref_transaction *transaction,
 	strbuf_release(&err);
 }
 
+
+static void parse_cmd_symref_create(struct ref_transaction *transaction,
+				    const char *next, const char *end)
+{
+	struct strbuf err = STRBUF_INIT;
+	char *refname, *new_target;
+
+	if (!(update_flags & REF_NO_DEREF))
+		die("symref-create: cannot operate with deref mode");
+
+	refname = parse_refname(&next);
+	if (!refname)
+		die("symref-create: missing <ref>");
+
+	new_target = parse_next_refname(&next);
+	if (!new_target)
+		die("symref-create %s: missing <new-target>", refname);
+
+	if (*next != line_termination)
+		die("symref-create %s: extra input: %s", refname, next);
+
+	if (ref_transaction_create(transaction, refname, NULL, new_target,
+				   update_flags | create_reflog_flag,
+				   msg, &err))
+		die("%s", err.buf);
+
+	update_flags = default_flags;
+	free(refname);
+	free(new_target);
+	strbuf_release(&err);
+}
+
 static void parse_cmd_delete(struct ref_transaction *transaction,
 			     const char *next, const char *end)
 {
@@ -473,6 +505,7 @@ static const struct parse_cmd {
 	{ "create",        parse_cmd_create,        2, UPDATE_REFS_OPEN },
 	{ "delete",        parse_cmd_delete,        2, UPDATE_REFS_OPEN },
 	{ "verify",        parse_cmd_verify,        2, UPDATE_REFS_OPEN },
+	{ "symref-create", parse_cmd_symref_create, 2, UPDATE_REFS_OPEN },
 	{ "symref-delete", parse_cmd_symref_delete, 2, UPDATE_REFS_OPEN },
 	{ "symref-verify", parse_cmd_symref_verify, 2, UPDATE_REFS_OPEN },
 	{ "option",        parse_cmd_option,        1, UPDATE_REFS_OPEN },
refs.c +6 -3
index 6b7c46bfd8..42cb4126a7 100644
@@ -1303,15 +1303,18 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
-	if (!new_oid || is_null_oid(new_oid)) {
-		strbuf_addf(err, "'%s' has a null OID", refname);
+	if ((!new_oid || is_null_oid(new_oid)) && !new_target) {
+		strbuf_addf(err, "'%s' has a null OID or no new target", refname);
 		return 1;
 	}
+	if (new_target && !(flags & REF_NO_DEREF))
+		BUG("create cannot operate on symrefs with deref mode");
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), NULL, NULL, flags,
+				      null_oid(), new_target, NULL, flags,
 				      msg, err);
 }
 
refs.h +1 -0
index 4be4930f04..bde8606213 100644
@@ -752,6 +752,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 int ref_transaction_create(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
+			   const char *new_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
refs/files-backend.c +42 -0

60 lines not on this page · raw message

refs/reftable-backend.c +20 -3
index 2b2cbca8c0..e203c697f2 100644
@@ -856,7 +856,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			 * There is no need to write the reference deletion
 			 * when the reference in question doesn't exist.
 			 */
-			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
+			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
 				 ret = queue_transaction_update(refs, tx_data, u,
 								&current_oid, err);
 				 if (ret)
@@ -1062,7 +1062,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		 * - `core.logAllRefUpdates` tells us to create the reflog for
 		 *   the given ref.
 		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
 			struct reftable_log_record log = {0};
 			struct reftable_iterator it = {0};
 
@@ -1104,6 +1104,12 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 			    should_write_log(&arg->refs->base, u->refname))) {
 			struct reftable_log_record *log;
 
+			if (u->new_target)
+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
+							     RESOLVE_REF_READING, &u->new_oid, NULL))
+					/* for dangling symrefs we gracefully set the oid to zero */
+					u->new_oid = *null_oid();
+
 			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
 			log = &logs[logs_nr++];
 			memset(log, 0, sizeof(*log));
@@ -1120,7 +1126,18 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
 		if (u->flags & REF_LOG_ONLY)
 			continue;
 
-		if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
+		if (u->flags & REF_HAVE_NEW && u->new_target) {
+			struct reftable_ref_record ref = {
+				.refname = (char *)u->refname,
+				.value_type = REFTABLE_REF_SYMREF,
+				.value.symref = (char *)u->new_target,
+				.update_index = ts,
+			};
+
+			ret = reftable_writer_add_ref(writer, &ref);
+			if (ret < 0)
+				goto done;
+		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
 			struct reftable_ref_record ref = {
 				.refname = (char *)u->refname,
 				.update_index = ts,
t/t0600-reffiles-backend.sh +32 -0
index 64214340e7..c5061c26cf 100755
@@ -472,4 +472,36 @@ test_expect_success POSIXPERM 'git reflog expire honors core.sharedRepository' '
 	esac
 '
 
+test_expect_success SYMLINKS 'symref transaction supports symlinks' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs true &&
+	cat >stdin <<-EOF &&
+	start
+	symref-create TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_symlink .git/TESTSYMREFONE &&
+	test "$(test_readlink .git/TESTSYMREFONE)" = refs/heads/new
+'
+
+test_expect_success 'symref transaction supports false symlink config' '
+	test_when_finished "git symbolic-ref -d TESTSYMREFONE" &&
+	git update-ref refs/heads/new @ &&
+	test_config core.prefersymlinkrefs false &&
+	cat >stdin <<-EOF &&
+	start
+	symref-create TESTSYMREFONE refs/heads/new
+	prepare
+	commit
+	EOF
+	git update-ref --no-deref --stdin <stdin &&
+	test_path_is_file .git/TESTSYMREFONE &&
+	git symbolic-ref TESTSYMREFONE >actual &&
+	echo refs/heads/new >expect &&
+	test_cmp expect actual
+'
+
 test_done
t/t1400-update-ref.sh +65 -0

71 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#116 lore raw

PATCH v4 6/7 update-ref: add support for 'symref-update' command

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

Add 'symref-update' command to the '--stdin' mode of 'git-update-ref' to allow updates of symbolic refs. The 'symref-update' command takes in a <new-target>, which the <ref> will be updated to. If the <ref> doesn't exist it will be created.

It also optionally takes either an ref <old-target> or oid <old-oid>. If the <old-target> is provided, it checks to see if the <ref> targets the <old-target> before the update. If <old-oid> is provided it checks <ref> to ensure that it is a regular ref and <old-oid> is the OID before the update. This by extension also means that this when a zero <old-oid> is provided, it ensures that the ref didn't exist before.

This command will also support deref mode, to ensure that we can update dereferenced regular refs to symrefs.

---
 Documentation/git-update-ref.txt |   6 +
 builtin/update-ref.c             |  91 +++++++++++++-
 refs/files-backend.c             |  14 +--
 refs/reftable-backend.c          |   3 +-
 t/t1400-update-ref.sh            | 196 +++++++++++++++++++++++++++++++
 5 files changed, 299 insertions(+), 11 deletions(-)
Documentation/git-update-ref.txt +6 -0
index 7a33f70767..0cd4c37820 100644
@@ -65,6 +65,7 @@ performs all modifications together.  Specify commands of the form:
 	create SP <ref> SP <new-oid> LF
 	delete SP <ref> [SP <old-oid>] LF
 	verify SP <ref> [SP <old-oid>] LF
+	symref-update SP <ref> SP <new-target> [SP (ref SP <old-target> | oid SP <old-oid>)] LF
 	symref-create SP <ref> SP <new-target> LF
 	symref-delete SP <ref> [SP <old-target>] LF
 	symref-verify SP <ref> [SP <old-target>] LF
@@ -89,6 +90,7 @@ quoting:
 	create SP <ref> NUL <new-oid> NUL
 	delete SP <ref> NUL [<old-oid>] NUL
 	verify SP <ref> NUL [<old-oid>] NUL
+	symref-update SP <ref> NUL <new-target> [NUL (ref NUL <old-target> | oid NUL <old-oid>)] NUL
 	symref-create SP <ref> NUL <new-target> NUL
 	symref-delete SP <ref> [NUL <old-target>] NUL
 	symref-verify SP <ref> [NUL <old-target>] NUL
@@ -123,6 +125,10 @@ delete::
 	is provided, verify that the symbolic ref <ref> targets
 	<old-target> before deleting it.
 
+symref-update::
+	Set <ref> to <new-target> after verifying <old-target> or <old-oid>,
+	if given.  Can be used to delete or create symrefs too.
+
 verify::
 	Verify <ref> against <old-oid> but do not change it.  If
 	<old-oid> is zero or missing, the ref must not exist.
builtin/update-ref.c +89 -2

115 lines not on this page · raw message

refs/files-backend.c +6 -8
index f5e271a442..59d1ab3eeb 100644
@@ -2386,7 +2386,8 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target, update->old_target,
+			update->msg);
 
 	new_update->parent_update = update;
 
@@ -2610,7 +2611,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		}
 	}
 
-	if (update->new_target) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
 		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
 			ret = TRANSACTION_GENERIC_ERROR;
 			goto out;
@@ -2628,12 +2629,9 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 		 * phase of the transaction only needs to commit the lock.
 		 */
 		update->flags |= REF_NEEDS_COMMIT;
-	}
-
-
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	} else if ((update->flags & REF_HAVE_NEW) &&
+		   !(update->flags & REF_DELETING) &&
+		   !(update->flags & REF_LOG_ONLY)) {
 		if (!(update->type & REF_ISSYMREF) &&
 		    oideq(&lock->old_oid, &update->new_oid)) {
 			/*
refs/reftable-backend.c +2 -1
index e203c697f2..a00f55802a 100644
@@ -908,7 +908,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
+						&u->new_oid, &u->old_oid, u->new_target,
+						u->old_target, u->msg);
 				new_update->parent_update = u;
 
 				/*
t/t1400-update-ref.sh +196 -0

208 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#117 lore raw

PATCH v4 7/7 ref: support symrefs in 'reference-transaction' hook

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The 'reference-transaction' hook runs whenever a reference update is made to the system. In the previous commits, we added symref support for various commands in git-update-ref. While it allowed us to now manipulate symbolic refs via git-update-ref, it didn't activate the 'reference-transaction' hook.

Let's activate the hook for symbolic reference updates too. There is no new format described for this and we stick to the existing format of:
<old-value> SP <new-value> SP <ref-name> LF
but now, <old-value> and <new-value> could also denote references instead of objects, where the format is similar to that in 'git-update-ref', i.e. 'ref:<ref-target>'.

While this seems to be backward incompatible, it is okay, since the only way the reference-transaction hook has refs in its output is when git-update-ref is used to manipulate symrefs. Also the documentation for reference-transaction hook always stated that support for symbolic references may be added in the future.

---
 Documentation/githooks.txt       | 14 +++++++----
 refs.c                           | 21 ++++++++--------
 t/t1416-ref-transaction-hooks.sh | 41 ++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 15 deletions(-)
Documentation/githooks.txt +9 -5
index ee9b92c90d..0bf8ca87a6 100644
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also cover symbolic references.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects, denoted via the
+`ref:<ref-target>` format.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
refs.c +11 -10
index 42cb4126a7..9a510744a7 100644
@@ -2365,18 +2365,19 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		strbuf_reset(&buf);
 
-		/*
-		 * Skip reference transaction for symbolic refs.
-		 */
-		if (update->new_target || update->old_target)
-			continue;
+		if (update->flags & REF_HAVE_OLD && update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
t/t1416-ref-transaction-hooks.sh +41 -0

45 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#118 lore raw

Re: [PATCH v4 0/7] add symref-* commands to 'git-update-ref --stdin'

In reply to Karthik Nayak
Quoted 35 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The 'git-update-ref(1)' command allows transactional reference updates. But currently only supports regular reference updates. Meaning, if one wants to update HEAD (symbolic ref) in a transaction, there is no tool to do so.

One option to obtain transactional updates for the HEAD ref is to manually create the HEAD.lock file and commit. This is intrusive, where the user needs to mimic internal git behavior. Also, this only works when using the files backend.

At GitLab, we've been using the manual process till date, to allow users to set and change their default branch. But with the introduction of reftables as a reference backend, this becomes a necessity to be solved within git.

This patch series goes about introducing a set of commands symref-{create,verify,delete,update} to work with symrefs complimenting the existing commands for the regular refs in the '--stdin' mode of 'git-update-ref'.

The 'symref-verify' command can be used to verify if a symref exists and its existing value.

The 'symref-create' command can be used to create a new symref.

The 'symref-delete' command can be used to delete an existing symref while optionally checking its existing value.

The 'symref-update' command can be used to update a symref, create a symref, delete a symref or even convert an existing regular ref to a symref. Wherein like the regular 'update' command, the zero OID can be used to create/delete a symref.

Quoted 5 lines

While this series adds the commands and the required ground work, it only is accessile within the '--stdin' mode of 'git-update-ref'. However, it makes it easy to extend it further to the command line too, which will be present in a follow up series.

Also, I'll be pushing a new version with reduced scope soon. The idea being that we can review/merge the part which adds the symref support to the refs library first and then we can have a followup for exposing the same to the users via git-update-ref. This should make it also easier for reviewers.

[snip]

#119 lore raw

v5 cover refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation. Also by using this new functionality in a pre-existing command, we can leverage the existing tests to catch any inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/[email protected]/
V2: https://lore.kernel.org/git/[email protected]/
V3: https://lore.kernel.org/git/[email protected]/
V4: https://lore.kernel.org/r/[email protected]

Changes over v4 are:
- Dropped the patches for adding support in git-update-ref.
- Added changes to use transaction in 'refs_create_symref()' and deprecate 'create_symref'.
- Cleaned up the commit messages, documentation and comments.
- Added better handling, like if 'old_target' is set, the value of the reference is checked, irrelevant of its type.

Range diff:

1: 4a56e3ede4 ! 1: a354190905 refs: accept symref values in ref_transaction[_add]_update
@@ Metadata
Author: Karthik Nayak <[email protected]>

## Commit message ##
- refs: accept symref values in ref_transaction[_add]_update
+ refs: accept symref values in ref_transaction_update()

- The ref_transaction[_add]_update functions obtain ref information and
- flags to create a ref_update and add it to the transaction at hand.
+ The function ref_transaction_update() obtains ref information and
+ flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the
- old and new ref targets and process it. In this commit, let's add the
- required parameters to the function and modify all call sites.
+ old and new ref targets and process it. This commit adds the required
+ parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when
- the transaction is applied.
+ the transaction is applied. Some functions allow this parameter to be
+ NULL, meaning that the reference is not changed.

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

- The handling logic of these parameters will be added in consequent
- commits as we add symref commands to the '--stdin' mode of
- 'git-update-ref'.
+ We also update the internal function ref_transaction_add_update()
+ similarly to take the two new parameters.

@@ refs.c: struct ref_update *ref_transaction_add_update(
BUG("update called for transaction that is not open");

+ if (old_oid && !is_null_oid(old_oid) && old_target)
-+ BUG("Only one of old_oid and old_target should be non NULL");
++ BUG("only one of old_oid and old_target should be non NULL");
+ if (new_oid && !is_null_oid(new_oid) && new_target)
-+ BUG("Only one of new_oid and new_target should be non NULL");
++ BUG("only one of new_oid and new_target should be non NULL");
+
FLEX_ALLOC_STR(update, refname, refname); ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
@@ refs.h: struct ref_transaction *ref_transaction_begin(struct strbuf *err);
* transaction.
*
+ * new_target -- the target reference that the reference will be
-+ * update to point to. This takes precedence over new_oid when
-+ * set. If the reference is a regular reference, it will be
-+ * converted to a symbolic reference.
++ * updated to point to. If the reference is a regular reference,
++ * it will be converted to a symbolic reference. Cannot be set
++ * together with new_oid. A copy of this value is made in the
++ * transaction.
+ *
+ * old_target -- the reference that the reference must be pointing to.
-+ * Will only be taken into account when the reference is a symbolic
-+ * reference.
++ * Canont be set together with old_oid. A copy of this value is
++ * made in the transaction.
+ *
* flags -- flags affecting the update, passed to
* update_ref_lock(). Possible flags: REF_NO_DEREF,
@@ refs/refs-internal.h: struct ref_update {
+ /*
+ * If set, point the reference to this value. This can also be
+ * used to convert regular references to become symbolic refs.
++ * Cannot be set together with new_oid.
+ */
+ const char *new_target;
+
+ /*
-+ * If set and the reference is a symbolic ref, check that the
-+ * reference previously pointed to this value.
++ * If set, check that the reference previously pointed to this
++ * value. Cannot be set together with old_oid.
+ */
+ const char *old_target;
+
2: 496bf14f28 ! 2: 7dff21dbef files-backend: extract out create_symref_lock
@@ Metadata
Author: Karthik Nayak <[email protected]>

## Commit message ##
- files-backend: extract out create_symref_lock
+ files-backend: extract out create_symref_lock()

- The function create_symref_locked creates a symref by creating a
+ The function create_symref_locked() creates a symref by creating a
'<symref>.lock' file and then committing the symref lock, which creates the final symref.

- Split this into two individual functions create_and_commit_symref and
- create_symref_locked. This way we can create the symref lock and
- commit it at different times. This will be used to provide symref
- support in git-update-ref(1).
+ Extract the early half of create_symref_locked() into a new helper
+ function create_symref_lock(). Because the name of the new function is
+ too similar to the original, rename the original to
+ create_and_commit_symref() to avoid confusion.
+
+ The new function create_symref_locked() can be used to create the
+ symref lock in a separate step from that of committing it. This allows
+ to add transactional support for symrefs, where the lock would be
+ created in the preparation step and the lock would be committed in the
+ finish step.

@@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
+ return error("unable to fdopen %s: %s",
+ get_lock_file_path(&lock->lk), strerror(errno));
+
-+ /* no error check; commit_ref will check ferror */
-+ fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
++ if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
++ return error("unable to fprintf %s: %s",
++ get_lock_file_path(&lock->lk), strerror(errno));
+ return 0;
+}
+
@@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
- if (commit_ref(lock) < 0)
- return error("unable to write symref for %s: %s", refname,
- strerror(errno));
- return 0;
+- return 0;
++ return ret;
}

+ static int files_create_symref(struct ref_store *ref_store,
@@ refs/files-backend.c: static int files_create_symref(struct ref_store *ref_store,
return -1;
}

3:  6337859cbb < -:  ---------- update-ref: add support for 'symref-verify' command
4:  e611cb5a8c < -:  ---------- update-ref: add support for 'symref-delete' command
5:  37f8be2f3f < -:  ---------- update-ref: add support for 'symref-create' command
6:  b385f4d0d7 < -:  ---------- update-ref: add support for 'symref-update' command
7:  ef335e47d1 < -:  ---------- ref: support symrefs in 'reference-transaction' hook
-:  ---------- > 3:  901a586683 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  6c97f6a660 refs: add support for transactional symref updates
-:  ---------- > 5:  5b55406430 refs: use transaction in `refs_create_symref()`
-:  ---------- > 6:  9e25816e68 refs: rename `refs_create_symref()` to `refs_update_symref()`
-:  ---------- > 7:  3836e25932 refs: remove `create_symref` and associated dead code

Karthik Nayak (7):
refs: accept symref values in ref_transaction_update()
files-backend: extract out create_symref_lock()
refs: support symrefs in 'reference-transaction' hook
refs: add support for transactional symref updates
refs: use transaction in refs_create_symref()
refs: rename refs_create_symref() to refs_update_symref()
refs: remove create_symref and associated dead code

 Documentation/githooks.txt       |  14 ++-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           |  87 +++++++++++----
 refs.h                           |  20 +++-
 refs/debug.c                     |  13 ---
 refs/files-backend.c             | 185 +++++++++++++++++++------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  26 ++++-
 refs/reftable-backend.c          | 159 +++++++++-----------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t0610-reftable-basics.sh       |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 22 files changed, 321 insertions(+), 240 deletions(-)
Signature · 1 line
2.43.GIT
#120 lore raw

PATCH v5 1/7 refs: accept symref values in `ref_transaction_update()`

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)
branch.c +1 -1
index e4a738fc7b..48af4c3ceb 100644
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
builtin/fast-import.c +3 -2
index dc5a9d32dd..297dfb91a1 100644
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
builtin/fetch.c +1 -1
index 5857d860db..66840b7c5b 100644
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
builtin/receive-pack.c +1 -0
index e8d7df14b6..b150ef39a8 100644
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
builtin/replace.c +1 -1
index da59600ad2..7690687b0e 100644
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
builtin/tag.c +1 -0
index 9a33cb50b4..40a65fdebc 100644
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
builtin/update-ref.c +1 -0
index e46afbc46d..21fdbf6ac8 100644
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
refs.c +17 -5
index 55d2e0b2cb..47bc9dd103 100644
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
refs.h +17 -1
index d278775e08..c7851bf587 100644
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
refs/files-backend.c +6 -6
index a098d14ea0..e4d0aa3d41 100644
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
refs/refs-internal.h +14 -0
index 56641aa57a..108f4ec419 100644
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
refs/reftable-backend.c +2 -2
index 1cda48c504..6104471199 100644
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
sequencer.c +5 -4
index 88de4dc20f..61e007d85f 100644
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
walker.c +1 -1
index c0fd632d92..1b3df43906 100644
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
Signature · 1 line
2.43.GIT
#121 lore raw

PATCH v5 2/7 files-backend: extract out `create_symref_lock()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The function create_symref_locked() creates a symref by creating a '<symref>.lock' file and then committing the symref lock, which creates the final symref.

Extract the early half of create_symref_locked() into a new helper function create_symref_lock(). Because the name of the new function is too similar to the original, rename the original to create_and_commit_symref() to avoid confusion.

The new function create_symref_locked() can be used to create the symref lock in a separate step from that of committing it. This allows to add transactional support for symrefs, where the lock would be created in the preparation step and the lock would be committed in the finish step.

---
 refs/files-backend.c | 43 +++++++++++++++++++++++++++++--------------
 1 file changed, 29 insertions(+), 14 deletions(-)
refs/files-backend.c +29 -14
index e4d0aa3d41..878601ced0 100644
@@ -1920,27 +1920,41 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target)
 {
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
+		return error("unable to fprintf %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1960,7 +1974,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
Signature · 1 line
2.43.GIT
#122 lore raw

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

refs/files-backend.c static void update_symref_reflog(struct files_ref_store *refs, · line 1933

Quoted 7 lines · Karthik Nayak
refs/files-backend.c +7 -0
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
+		return error("unable to fprintf %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));

error() is end-user facing, so "fprintf" is probably a bit too precise? "fprintf" -> "write to"

Also we may want to make them (not just this new message but other error() messages in related code paths) localizable but that is probably beyond the scope of this topic.

#123 lore raw

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

Quoted 13 lines · Junio C Hamano

refs/files-backend.c static void update_symref_reflog(struct files_ref_store *refs, · line 1933

Quoted 7 lines · Karthik Nayak
refs/files-backend.c +7 -0
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
+		return error("unable to fprintf %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));

error() is end-user facing, so "fprintf" is probably a bit too precise? "fprintf" -> "write to"

Quoted 3 lines

Also we may want to make them (not just this new message but other error() messages in related code paths) localizable but that is probably beyond the scope of this topic.

It only occurred to me now, but shouldn't we also support passing in a struct strbuf *err here? The transactional code doesn't want us to print error messages to stderr, but always supplies a buffer.

Patrick

#124 lore raw

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

Quoted 18 lines · Patrick Steinhardt
Quoted 13 lines · Junio C Hamano

refs/files-backend.c static void update_symref_reflog(struct files_ref_store *refs, · line 1933

Quoted 7 lines · Karthik Nayak
refs/files-backend.c +7 -0
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
+		return error("unable to fprintf %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));

error() is end-user facing, so "fprintf" is probably a bit too precise? "fprintf" -> "write to"

Quoted 3 lines

Also we may want to make them (not just this new message but other error() messages in related code paths) localizable but that is probably beyond the scope of this topic.

Quoted 5 lines

It only occurred to me now, but shouldn't we also support passing in a struct strbuf *err here? The transactional code doesn't want us to print error messages to stderr, but always supplies a buffer.

Patrick

Yes I think that would fit better with the existing transaction code.

#125 lore raw

Re: [PATCH v5 2/7] files-backend: extract out `create_symref_lock()`

In reply to Patrick Steinhardt
Quoted 14 lines · Patrick Steinhardt
Quoted 13 lines · Junio C Hamano

refs/files-backend.c static void update_symref_reflog(struct files_ref_store *refs, · line 1933

Quoted 7 lines · Karthik Nayak
refs/files-backend.c +7 -0
+	if (!fdopen_lock_file(&lock->lk, "w"))
+		return error("unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
+		return error("unable to fprintf %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));

error() is end-user facing, so "fprintf" is probably a bit too precise? "fprintf" -> "write to"

Quoted 7 lines
Quoted 3 lines

Also we may want to make them (not just this new message but other error() messages in related code paths) localizable but that is probably beyond the scope of this topic.

It only occurred to me now, but shouldn't we also support passing in a struct strbuf *err here? The transactional code doesn't want us to print error messages to stderr, but always supplies a buffer.

Sounds sensible. Thanks.

#126 lore raw

PATCH v5 3/7 refs: support symrefs in 'reference-transaction' hook

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The 'reference-transaction' hook runs whenever a reference update is made to the system. In a previous commit, we added the old_target and new_target fields to the reference_transaction_update(). In following commits we'll also add the code to handle symref's in the reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying the current format:
<old-oid> SP <new-oid> SP <ref-name> LF
to be be:
<old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same. But when either 'old-value' or 'new-value' is a symref, we print the ref as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction' hook's documentation always stated that support for symbolic references may be added in the future.

We do not add any tests in this commit since there is no git command which activates this flow, in an upcoming commit, we'll start using transaction based symref updates as the default, we'll add tests there for the hook too.

---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 16 ++++++++++++----
 2 files changed, 21 insertions(+), 9 deletions(-)
Documentation/githooks.txt +9 -5
index ee9b92c90d..06e997131b 100644
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
refs.c +12 -4
index 47bc9dd103..5dfe93060a 100644
@@ -2350,10 +2350,18 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (update->flags & REF_HAVE_OLD && update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
Signature · 1 line
2.43.GIT
#127 lore raw

Re: [PATCH v5 3/7] refs: support symrefs in 'reference-transaction' hook

refs.c static int run_transaction_hook(struct ref_transaction *transaction, · line 2354

Quoted 1 line · Karthik Nayak
refs.c +1 -0
+		if (update->flags & REF_HAVE_OLD && update->old_target)

Although the precedence rule does not require it,

		if ((update->flags & REF_HAVE_OLD) && update_old_target)

is probably easier to read.

refs.c static int run_transaction_hook(struct ref_transaction *transaction, · line 2357

Quoted 3 lines
refs.c +3 -0
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));

So the promise this code assumes is that .old_target member is non-NULL if and only if the ref originally is a symbolic ref?

And if the "we do not care what the original value is, whether it is a normal ref or a symbolic one" case, .old_oid would be all '\0' and REF_HAVE_OLD bit is not set?

If we can write it like so:

	if (!(update->flags & REF_HAVE_OLD))
		strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
	else if (update->old_target)
		strbuf_addf(&buf, "ref:%s ", update->old_target);
	else
		strbuf_addf(&buf, "ref:%s ", oid_to_hex(update->old_oid));

it may make the intent of the code a lot more clear. If we are operating in "!HAVE_OLD" mode, we show 0{40}. Otherwise, old_target is non-NULL when the thing is symbolic, and if old_target is NULL, it is not symbolic and has its own value.

The same comment applies to the other side.

Quoted 10 lines
refs.c +5 -0
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
 
 
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
#128 lore raw

Re: [PATCH v5 3/7] refs: support symrefs in 'reference-transaction' hook

Quoted 10 lines · Junio C Hamano

refs.c static int run_transaction_hook(struct ref_transaction *transaction, · line 2354

Quoted 1 line · Karthik Nayak
refs.c +1 -0
+		if (update->flags & REF_HAVE_OLD && update->old_target)

Although the precedence rule does not require it,

		if ((update->flags & REF_HAVE_OLD) && update_old_target)

is probably easier to read.

Will add.

Quoted 7 lines

refs.c static int run_transaction_hook(struct ref_transaction *transaction, · line 2357

Quoted 3 lines
refs.c +3 -0
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));

So the promise this code assumes is that .old_target member is non-NULL if and only if the ref originally is a symbolic ref?

Yes, for old_target this is correct. new_target could be set for a ref to convert it to a symbolic ref.

Quoted 4 lines

And if the "we do not care what the original value is, whether it is a normal ref or a symbolic one" case, .old_oid would be all '\0' and REF_HAVE_OLD bit is not set?

Yup that's accurate.

Quoted 9 lines

If we can write it like so:

	if (!(update->flags & REF_HAVE_OLD))
		strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
	else if (update->old_target)
		strbuf_addf(&buf, "ref:%s ", update->old_target);
	else
		strbuf_addf(&buf, "ref:%s ", oid_to_hex(update->old_oid));
Quoted 7 lines

it may make the intent of the code a lot more clear. If we are operating in "!HAVE_OLD" mode, we show 0{40}. Otherwise, old_target is non-NULL when the thing is symbolic, and if old_target is NULL, it is not symbolic and has its own value.

The same comment applies to the other side.

I see how it makes it clearer, but I think the intent with the existing code was clear too. I'll add this change to my local for the next version.

Quoted 10 lines
Quoted 10 lines
refs.c +5 -0
+		if (update->flags & REF_HAVE_NEW && update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
 
 
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
#129 lore raw

PATCH v5 4/7 refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's add support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use ref_update_is_null_new_value(), a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid.

With this, now transactional updates (verify, create, delete, update) can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in 'git-update-ref' in the future.

We do not add reflog for dangling symref updates, because currently 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it would be best to keep this behavior consistent as we would move it to start using transaction based updates in the following commit.

---
 refs.c                  |  17 +++++-
 refs/files-backend.c    | 112 +++++++++++++++++++++++++++++++++++-----
 refs/refs-internal.h    |   7 +++
 refs/reftable-backend.c |  71 +++++++++++++++++++------
 4 files changed, 176 insertions(+), 31 deletions(-)
refs.c +15 -2
index 5dfe93060a..a4dca08244 100644
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,10 +1249,15 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);
+	if (new_oid && flags & REF_HAVE_NEW)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if (old_oid && flags & REF_HAVE_OLD)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1286,6 +1293,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2810,3 +2818,8 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_is_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
refs/files-backend.c +100 -12

154 lines not on this page · raw message

refs/refs-internal.h +7 -0
index 108f4ec419..9578665243 100644
@@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_is_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +54 -17

107 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#130 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

Quoted 11 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's add support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use ref_update_is_null_new_value(), a helper function which is used to check if there is a new_value in a reference update.

I know you want to express a condition where you answer yes to "Is the new value specified in this ref update NULL?", but "is" at that position in the name somehow sounds awkward. Any of

ref_update_has_null_new_value
ref_update_with_no_new_value
ref_update_without_new_value

might be nicer to ears.

Quoted 4 lines

We do not add reflog for dangling symref updates, because currently 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it would be best to keep this behavior consistent as we would move it to start using transaction based updates in the following commit.

If we are not changing the behaviour, does it deserve a four-line paragraph? It is not like we describe every no changes (i.e. "we could break the behaviour by introducing this and that bugs, but we did not" is not something we usually say in proposed log messages).

At most, if you want to highlight that behaviour, I would expect a brief mention like:

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

As a reflog entry records name of the object that is pointed by the ref (either directly or indirectly) before and after an operation, an operation that involve a dangling reflog that does not point at any object cannot be expressed in a reflog, no? It is way too late to change this, but it would have been interesting if the design of reflog had a room to log the change of symbolic ref target as well as object names. It would have allowed us to say "HEAD at time T pointed at refs/heads/main (which did not exist)", "HEAD at time T+1 directly pointed at commit X (detached)", "HEAD at time T+2 pointed at refs/heads/next", etc. and allowed us to much more cleanly support "previous branch".

refs.c struct ref_update *ref_transaction_add_update( · line 1255

Quoted 9 lines
refs.c +4 -1
@@ -1247,10 +1249,15 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);

"Is the assumption that *update is 0-initialized?" was the first question that came to my mind.

Doing an unconditional

	update->new_target = xstrdup_or_null(new_target);
	update->old_target = xstrdup_or_null(old_target);

would convey the intention much more clearly without having readers guess the answer to the above question.

refs.c struct ref_update *ref_transaction_add_update( · line 1256

Quoted 1 line
refs.c +1 -0
+	if (new_oid && flags & REF_HAVE_NEW)

Even though syntactically not required,

	if (new_oid && (flags & REF_HAVE_NEW))

or better yet

	if ((flags & REF_HAVE_NEW) && new_oid)

would be easier to see.

refs.c struct ref_update *ref_transaction_add_update( · line 1257

Quoted 1 line
refs.c +0 -0
 		oidcpy(&update->new_oid, new_oid);

Again is the expectation that update->new_oid is initialized to all-0? I am wondering if we want an else clause, i.e.

	if (!(flags & REF_HAVE_NEW))
		oidcpy(&update->new_oid, null_oid());
	else
		oidcpy(&update->new_oid, new_oid ? new_oid : null_oid());

to clarify the intention of the code, since the way you wrote the consumer of thes two members and REF_HAVE_NEW bit in the previous step implied that the new_oid member gets used even when REF_HAVE_* bit is off, only for its null_oid() value.

I'll stop here for now.

Thanks.

#131 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

Quoted 18 lines · Junio C Hamano
Quoted 11 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's add support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use ref_update_is_null_new_value(), a helper function which is used to check if there is a new_value in a reference update.

I know you want to express a condition where you answer yes to "Is the new value specified in this ref update NULL?", but "is" at that position in the name somehow sounds awkward. Any of

Quoted 6 lines

ref_update_has_null_new_value
ref_update_with_no_new_value
ref_update_without_new_value

might be nicer to ears.

Yes, thanks with this, I do agree that ref_update_has_null_new_value sounds better.

Quoted 16 lines
Quoted 4 lines

We do not add reflog for dangling symref updates, because currently 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it would be best to keep this behavior consistent as we would move it to start using transaction based updates in the following commit.

If we are not changing the behaviour, does it deserve a four-line paragraph? It is not like we describe every no changes (i.e. "we could break the behaviour by introducing this and that bugs, but we did not" is not something we usually say in proposed log messages).

At most, if you want to highlight that behaviour, I would expect a brief mention like:

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

Quoted 12 lines

As a reflog entry records name of the object that is pointed by the ref (either directly or indirectly) before and after an operation, an operation that involve a dangling reflog that does not point at any object cannot be expressed in a reflog, no? It is way too late to change this, but it would have been interesting if the design of reflog had a room to log the change of symbolic ref target as well as object names. It would have allowed us to say "HEAD at time T pointed at refs/heads/main (which did not exist)", "HEAD at time T+1 directly pointed at commit X (detached)", "HEAD at time T+2 pointed at refs/heads/next", etc. and allowed us to much more cleanly support "previous branch".

While I agree that four lines may seem excessive, I think it is indeed an important point to note. Mostly because this shows that when doing dangling symref updates, there is no record of this update. The best situation would be like you mentioned, to record the symref target changes. But even with the current design, it would have been nice to at least acknowledge that there was some update done to the symref. By having zero-oid for the new and old value in the reflog. But seems like we can't do that either.

Quoted 13 lines

refs.c struct ref_update *ref_transaction_add_update( · line 1255

Quoted 9 lines
refs.c +4 -1
@@ -1247,10 +1249,15 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	if (new_target)
+		update->new_target = xstrdup(new_target);
+	if (old_target)
+		update->old_target = xstrdup(old_target);

"Is the assumption that *update is 0-initialized?" was the first question that came to my mind.

Quoted 8 lines

Doing an unconditional

	update->new_target = xstrdup_or_null(new_target);
	update->old_target = xstrdup_or_null(old_target);

would convey the intention much more clearly without having readers guess the answer to the above question.

Right, I didn't catch the nuance last time, thanks for the explanation.

Quoted 22 lines

refs.c struct ref_update *ref_transaction_add_update( · line 1256

Quoted 1 line
refs.c +1 -0
+	if (new_oid && flags & REF_HAVE_NEW)

Even though syntactically not required,

	if (new_oid && (flags & REF_HAVE_NEW))

or better yet

	if ((flags & REF_HAVE_NEW) && new_oid)

would be easier to see.

refs.c struct ref_update *ref_transaction_add_update( · line 1257

Quoted 1 line
refs.c +0 -0
 		oidcpy(&update->new_oid, new_oid);

Again is the expectation that update->new_oid is initialized to all-0? I am wondering if we want an else clause, i.e.

	if (!(flags & REF_HAVE_NEW))
		oidcpy(&update->new_oid, null_oid());
	else
		oidcpy(&update->new_oid, new_oid ? new_oid : null_oid());
Quoted 5 lines

to clarify the intention of the code, since the way you wrote the consumer of thes two members and REF_HAVE_NEW bit in the previous step implied that the new_oid member gets used even when REF_HAVE_* bit is off, only for its null_oid() value.

Yes I understand what you're saying, but since we're doing a FLEX_ALLOC_MEM right above this code, I thought the fact that the 'update' struct is 0 initialized is known. With this, and the fact that here update->new_oid is a static variable, while new_oid is a pointer. I think being too verbose is not required.

Quoted 4 lines

I'll stop here for now.

Thanks.

Thanks Junio for taking the time to review.

#132 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

Quoted 32 lines · Karthik Nayak
Quoted 18 lines · Junio C Hamano
Quoted 5 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>
We do not add reflog for dangling symref updates, because currently 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it would be best to keep this behavior consistent as we would move it to start using transaction based updates in the following commit.

If we are not changing the behaviour, does it deserve a four-line paragraph? It is not like we describe every no changes (i.e. "we could break the behaviour by introducing this and that bugs, but we did not" is not something we usually say in proposed log messages).

At most, if you want to highlight that behaviour, I would expect a brief mention like:

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

Quoted 12 lines

As a reflog entry records name of the object that is pointed by the ref (either directly or indirectly) before and after an operation, an operation that involve a dangling reflog that does not point at any object cannot be expressed in a reflog, no? It is way too late to change this, but it would have been interesting if the design of reflog had a room to log the change of symbolic ref target as well as object names. It would have allowed us to say "HEAD at time T pointed at refs/heads/main (which did not exist)", "HEAD at time T+1 directly pointed at commit X (detached)", "HEAD at time T+2 pointed at refs/heads/next", etc. and allowed us to much more cleanly support "previous branch".

Quoted 8 lines

While I agree that four lines may seem excessive, I think it is indeed an important point to note. Mostly because this shows that when doing dangling symref updates, there is no record of this update. The best situation would be like you mentioned, to record the symref target changes. But even with the current design, it would have been nice to at least acknowledge that there was some update done to the symref. By having zero-oid for the new and old value in the reflog. But seems like we can't do that either.

I wouldn't say we can't do that. We already do log when symrefs become dangling when updating references via HEAD by logging a zero OID as new OID. That is, if we have "HEAD -> refs/heads/foo" and you delete the latter, then we create a new reflog message for "HEAD" with zero OID as new OID.

I would claim that the current behaviour where we don't create a reflog entry when updating a ref to become dangling is a mere bug. I think it's fair to declare this a #leftoverbit and handle it in a follow-up patch series. But it would be nice to say so in an in-code comment.

Patrick

#133 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

Quoted 48 lines · Patrick Steinhardt
Quoted 32 lines · Karthik Nayak
Quoted 18 lines · Junio C Hamano
Quoted 5 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>
We do not add reflog for dangling symref updates, because currently 'git-symbolic-ref' doesn't add reflog for dangling symref updates and it would be best to keep this behavior consistent as we would move it to start using transaction based updates in the following commit.

If we are not changing the behaviour, does it deserve a four-line paragraph? It is not like we describe every no changes (i.e. "we could break the behaviour by introducing this and that bugs, but we did not" is not something we usually say in proposed log messages).

At most, if you want to highlight that behaviour, I would expect a brief mention like:

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

Quoted 12 lines

As a reflog entry records name of the object that is pointed by the ref (either directly or indirectly) before and after an operation, an operation that involve a dangling reflog that does not point at any object cannot be expressed in a reflog, no? It is way too late to change this, but it would have been interesting if the design of reflog had a room to log the change of symbolic ref target as well as object names. It would have allowed us to say "HEAD at time T pointed at refs/heads/main (which did not exist)", "HEAD at time T+1 directly pointed at commit X (detached)", "HEAD at time T+2 pointed at refs/heads/next", etc. and allowed us to much more cleanly support "previous branch".

Quoted 8 lines

While I agree that four lines may seem excessive, I think it is indeed an important point to note. Mostly because this shows that when doing dangling symref updates, there is no record of this update. The best situation would be like you mentioned, to record the symref target changes. But even with the current design, it would have been nice to at least acknowledge that there was some update done to the symref. By having zero-oid for the new and old value in the reflog. But seems like we can't do that either.

I wouldn't say we can't do that. We already do log when symrefs become dangling when updating references via HEAD by logging a zero OID as new OID. That is, if we have "HEAD -> refs/heads/foo" and you delete the latter, then we create a new reflog message for "HEAD" with zero OID as new OID.

Quoted 5 lines

I would claim that the current behaviour where we don't create a reflog entry when updating a ref to become dangling is a mere bug. I think it's fair to declare this a #leftoverbit and handle it in a follow-up patch series. But it would be nice to say so in an in-code comment.

I think _can't_ wasn't the best terminology. My previous series actually added a reflog, but I noticed a bunch of tests were failing and I think it made sense to keep the existing behaviour.

But addressing it as a bug would definitely be a good way to go and fix this, I'll add a comment in the code for now.

#134 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

In reply to Patrick Steinhardt
Quoted 10 lines · Patrick Steinhardt

I wouldn't say we can't do that. We already do log when symrefs become dangling when updating references via HEAD by logging a zero OID as new OID. That is, if we have "HEAD -> refs/heads/foo" and you delete the latter, then we create a new reflog message for "HEAD" with zero OID as new OID.

I would claim that the current behaviour where we don't create a reflog entry when updating a ref to become dangling is a mere bug. I think it's fair to declare this a #leftoverbit and handle it in a follow-up patch series. But it would be nice to say so in an in-code comment.

I like that. Thanks.

#135 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 3 lines · Karthik Nayak

... By
having zero-oid for the new and old value in the reflog. But seems like we can't do that either.

Even if you could, the transition from refs/heads/main to refs/heads/next would not be captured.

In any case, that is not something this step changes. My assumption is that for most of developers it would be unexpected for the theme to "allow symbolic updates to be transactional" to change it, hence "we do not explain what we did not change".

Thanks.

#136 lore raw

Re: [PATCH v5 4/7] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 23 lines · Karthik Nayak
refs/files-backend.c +13 -6
@@ -2863,12 +2928,26 @@ static int files_transaction_finish(struct ref_store *ref_store,
 
 		if (update->flags & REF_NEEDS_COMMIT ||
 		    update->flags & REF_LOG_ONLY) {
-			if (files_log_ref_write(refs,
-						lock->ref_name,
-						&lock->old_oid,
-						&update->new_oid,
-						update->msg, update->flags,
-						err)) {
+			int create_reflog = 1;
+
+			if (update->new_target) {
+				/*
+				 * We want to get the resolved OID for the target, to ensure
+				 * that the correct value is added to the reflog.
+				 */
+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
+					/* for dangling symrefs we skip creating a reflog entry. */
+					create_reflog = 0;
+				}
+			}

refs/files-backend.c static int files_transaction_finish(struct ref_store *ref_store, · line 2953

Quoted 10 lines
refs/files-backend.c +7 -0
+
+			if (create_reflog && files_log_ref_write(refs,
+								 lock->ref_name,
+								 &lock->old_oid,
+								 &update->new_oid,
+								 update->msg, update->flags,
+								 err)) {
 				char *old_msg = strbuf_detach(err, NULL);
 
 				strbuf_addf(err, "cannot update the ref '%s': %s",

This hunk is overly wide.

You could of course fix it mechanically (e.g., by rewrapping overly wide comment block, wrapping an expression after &&-), but a code path that is too deeply indented may be a sign that the function may want to be split into calls to a smaller helper function for readability.

#137 lore raw

PATCH v5 5/7 refs: use transaction in `refs_create_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function updates a symref to a given new target. To do this, it uses a ref-backend specific function create_symref().

In this previous commit, we introduce symref support in transactions. This means we can now use transactions to perform symref updates and not have to resort to create_symref(). Doing this allows us to remove and cleanup create_symref(), which we will do in the following commit.

Modify the expected error message for a test in 't/t0610-reftable-basics.sh', since the error is now thrown from the 'refs.c'. This is because in transactional updates, F/D conflicts are caught before we're in the reference backend.

---
 refs.c                           | 24 +++++++++++++++++-------
 t/t0610-reftable-basics.sh       |  2 +-
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 3 files changed, 41 insertions(+), 8 deletions(-)
refs.c +17 -7
index a4dca08244..1b67c87f47 100644
@@ -2291,14 +2291,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
t/t0610-reftable-basics.sh +1 -1
index 178791e086..9e8d22bcbd 100755
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err
t/t1416-ref-transaction-hooks.sh +23 -0
index 2092488090..4433ac2177 100755
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+		prepared
+		$ZERO_OID ref:refs/heads/main refs/heads/symref
+		committed
+		$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
Signature · 1 line
2.43.GIT
#138 lore raw

Re: [PATCH v5 5/7] refs: use transaction in `refs_create_symref()`

Quoted 9 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function updates a symref to a given new target. To do this, it uses a ref-backend specific function create_symref().

In this previous commit, we introduce symref support in transactions. This means we can now use transactions to perform symref updates and not have to resort to create_symref(). Doing this allows us to remove and

Nit: "not have to" -> "don't have to"

[snip]

t/t0610-reftable-basics.sh test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl · line 292

Quoted 13 lines
t/t0610-reftable-basics.sh +1 -1
index 178791e086..9e8d22bcbd 100755
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err

Nice. Not only do we have less code to worry about, but the error message is better, too.

Quoted 17 lines
t/t1416-ref-transaction-hooks.sh +9 -0
index 2092488090..4433ac2177 100755
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual

t/t1416-ref-transaction-hooks.sh test_expect_success 'interleaving hook calls succeed' ' · line 155

Quoted 10 lines
t/t1416-ref-transaction-hooks.sh +10 -0
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+		prepared
+		$ZERO_OID ref:refs/heads/main refs/heads/symref
+		committed
+		$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF

Nit: the contents of the heredoc should be indented one level less.

Patrick

#139 lore raw

PATCH v5 6/7 refs: rename `refs_create_symref()` to `refs_update_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function is used to update/create a symref. But it doesn't check the old target of the symref, if existing. It force updates the symref. In this regard, the name refs_create_symref() is a bit misleading. So let's rename it to refs_update_symref(). This is akin to how 'git-update-ref(1)' also allows us to create apart from update.

While we're here, rename the arguments in the function to clarify what they actually signify and reduce confusion.

---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)
builtin/branch.c +1 -1
index dd3e3a7dc0..4491f7a20c 100644
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
builtin/worktree.c +1 -1
index 7c6c72536b..480202c517 100644
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
refs.c +5 -7
index 1b67c87f47..1215b0fae2 100644
@@ -2286,10 +2286,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2297,8 +2295,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2314,7 +2312,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
refs.h +1 -1
index c7851bf587..71cc1c58e0 100644
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
t/helper/test-ref-store.c +1 -1
index 82bbf6e2e6..4651e4ced7 100644
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
Signature · 1 line
2.43.GIT
#140 lore raw

Re: [PATCH v5 6/7] refs: rename `refs_create_symref()` to `refs_update_symref()`

Quoted 8 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function is used to update/create a symref. But it doesn't check the old target of the symref, if existing. It force updates the symref. In this regard, the name refs_create_symref() is a bit misleading. So let's rename it to refs_update_symref(). This is akin to how 'git-update-ref(1)' also allows us to create apart from update.

Arguably, as we are already updating all callsites anyway, I don't see a reason why we shouldn't also update the function to accept the old OID or old target so that callers can make raceless updates.

The only problem is that we don't have any users yet, and consequently we have no way to verify that it works as intended. So maybe this is better left for a future patch series, unless we have places where we can reasonably update the callers to pass in the old value, as well.

Patrick

#141 lore raw

Re: [PATCH v5 6/7] refs: rename `refs_create_symref()` to `refs_update_symref()`

Quoted 14 lines · Patrick Steinhardt
Quoted 8 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function is used to update/create a symref. But it doesn't check the old target of the symref, if existing. It force updates the symref. In this regard, the name refs_create_symref() is a bit misleading. So let's rename it to refs_update_symref(). This is akin to how 'git-update-ref(1)' also allows us to create apart from update.

Arguably, as we are already updating all callsites anyway, I don't see a reason why we shouldn't also update the function to accept the old OID or old target so that callers can make raceless updates.

Quoted 6 lines

The only problem is that we don't have any users yet, and consequently we have no way to verify that it works as intended. So maybe this is better left for a future patch series, unless we have places where we can reasonably update the callers to pass in the old value, as well.

Patrick

Yeah that did run through my mind too, but without a usecase, its hard to justify a change and also like you mentioned harder to write tests for.

#142 lore raw

PATCH v5 7/7 refs: remove `create_symref` and associated dead code

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

This means, we can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

---
 refs/debug.c            | 13 -------
 refs/files-backend.c    | 64 ------------------------------
 refs/packed-backend.c   |  1 -
 refs/refs-internal.h    |  5 ---
 refs/reftable-backend.c | 86 -----------------------------------------
 5 files changed, 169 deletions(-)
refs/debug.c +0 -13
index c7531b17f0..8be316bb67 100644
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
refs/files-backend.c +0 -64

82 lines not on this page · raw message

refs/packed-backend.c +0 -1
index 4e826c05ff..a937e7dbfc 100644
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
refs/refs-internal.h +0 -5
index 9578665243..fa46548a2f 100644
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
refs/reftable-backend.c +0 -86

98 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#143 lore raw

Re: [PATCH v5 7/7] refs: remove `create_symref` and associated dead code

Quoted 10 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

This means, we can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

Very nice.

Patrick

#144 lore raw

Re: [PATCH v5 7/7] refs: remove `create_symref` and associated dead code

Quoted 7 lines · Patrick Steinhardt
Quoted 6 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

Quoted 6 lines
Quoted 4 lines

This means, we can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

Very nice.

Indeed.

The last paragraph reads well without "This means, ", by the way.

#145 lore raw

Re: [PATCH v5 0/7] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 12 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

Quoted 4 lines

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation.

This split probably makes sense in the context of the evolution of this series. If this were without any prior discussion, a change to the internal mechanism, without showing how the end-user facing half would use it fully, would have been hard to evaluate, but now that we know where the new mechanism wants to take us, we can fairly evaluate it alone without the end-user facing part.

I've read only the earlier half of the series but so far the pieces make sense to me.

#146 lore raw

Re: [PATCH v5 0/7] refs: add support for transactional symref updates

Quoted 26 lines · Junio C Hamano
Quoted 12 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

Quoted 4 lines

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation.

This split probably makes sense in the context of the evolution of this series. If this were without any prior discussion, a change to the internal mechanism, without showing how the end-user facing half would use it fully, would have been hard to evaluate, but now that we know where the new mechanism wants to take us, we can fairly evaluate it alone without the end-user facing part.

Quoted 2 lines

I've read only the earlier half of the series but so far the pieces make sense to me.

Yes I agree, I think it's also nice to see how a bunch of code can be removed to use this generic functionality.

I'll wait for a day/two for other reviews. Thanks for your review.

#147 lore raw

v6 cover refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation. Also by using this new functionality in a pre-existing command, we can leverage the existing tests to catch any inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/[email protected]/
V2: https://lore.kernel.org/git/[email protected]/
V3: https://lore.kernel.org/git/[email protected]/
V4: https://lore.kernel.org/r/[email protected]
V5: https://lore.kernel.org/r/[email protected]

Changes over v5 are:
- More user friendly error messages.
- create_symref_lock now writes to an err buf, instead of directly to stderr.
- Refactor code to make it easier to read around logical operations.
- Cleanup commit message and fix typos.

Thanks to all reviewers!

Range diff:

1:  a354190905 = 1:  a354190905 refs: accept symref values in `ref_transaction_update()`
2:  7dff21dbef ! 2:  0d9c5b9804 files-backend: extract out `create_symref_lock()`
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     -				const char *target, const char *logmsg)
     +static int create_symref_lock(struct files_ref_store *refs,
     +			      struct ref_lock *lock, const char *refname,
    -+			      const char *target)
    ++			      const char *target, struct strbuf *err)
      {
    -+	if (!fdopen_lock_file(&lock->lk, "w"))
    -+		return error("unable to fdopen %s: %s",
    ++	if (!fdopen_lock_file(&lock->lk, "w")) {
    ++		strbuf_addf(err, "unable to fdopen %s: %s",
     +			     get_lock_file_path(&lock->lk), strerror(errno));
    ++		return -1;
    ++	}
     +
    -+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0)
    -+		return error("unable to fprintf %s: %s",
    ++	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
    ++		strbuf_addf(err, "unable to write to %s: %s",
     +			     get_lock_file_path(&lock->lk), strerror(errno));
    ++		return -1;
    ++	}
    ++
     +	return 0;
     +}
     +
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     +				    struct ref_lock *lock, const char *refname,
     +				    const char *target, const char *logmsg)
     +{
    ++	struct strbuf err = STRBUF_INIT;
     +	int ret;
     +
      	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     -	if (!fdopen_lock_file(&lock->lk, "w"))
     -		return error("unable to fdopen %s: %s",
     -			     get_lock_file_path(&lock->lk), strerror(errno));
    -+	ret = create_symref_lock(refs, lock, refname, target);
    ++	ret = create_symref_lock(refs, lock, refname, target, &err);
     +	if (!ret) {
     +		update_symref_reflog(refs, lock, refname, target, logmsg);
      
    @@ refs/files-backend.c: static void update_symref_reflog(struct files_ref_store *r
     +		if (commit_ref(lock) < 0)
     +			return error("unable to write symref for %s: %s", refname,
     +				     strerror(errno));
    ++	} else {
    ++		return error("%s", err.buf);
     +	}
      
     -	/* no error check; commit_ref will check ferror */
3:  901a586683 ! 3:  e0219ffd31 refs: support symrefs in 'reference-transaction' hook
    @@ refs.c: static int run_transaction_hook(struct ref_transaction *transaction,
     -			    oid_to_hex(&update->new_oid),
     -			    update->refname);
     +
    -+		if (update->flags & REF_HAVE_OLD && update->old_target)
    ++		if (!(update->flags & REF_HAVE_OLD))
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
    ++		else if (update->old_target)
     +			strbuf_addf(&buf, "ref:%s ", update->old_target);
     +		else
     +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
     +
    -+		if (update->flags & REF_HAVE_NEW && update->new_target)
    ++		if (!(update->flags & REF_HAVE_NEW))
    ++			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
    ++		else if (update->new_target)
     +			strbuf_addf(&buf, "ref:%s ", update->new_target);
     +		else
     +			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
4:  6c97f6a660 ! 4:  b22c59c722 refs: add support for transactional symref updates
    @@ Commit message
         However, we never supported transactional updates of symrefs. Let's add
         support for symrefs in both the 'files' and the 'reftable' backend.
     
    -    Here, we add and use `ref_update_is_null_new_value()`, a helper function
    -    which is used to check if there is a new_value in a reference update.
    -    The new value could either be a symref target `new_target` or a OID
    -    `new_oid`.
    +    Here, we add and use `ref_update_has_null_new_value()`, a helper
    +    function which is used to check if there is a new_value in a reference
    +    update. The new value could either be a symref target `new_target` or a
    +    OID `new_oid`.
     
         With this, now transactional updates (verify, create, delete, update)
         can be used for:
    @@ Commit message
         This also allows us to expose this to users via new commands in
         'git-update-ref' in the future.
     
    -    We do not add reflog for dangling symref updates, because currently
    -    'git-symbolic-ref' doesn't add reflog for dangling symref updates and it
    -    would be best to keep this behavior consistent as we would move it to
    -    start using transaction based updates in the following commit.
    +    Note that a dangling symref update does not record a new reflog entry,
    +    which is unchanged before and after this commit.
     
         Signed-off-by: Karthik Nayak <[email protected]>
     
    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	update->flags = flags;
      
     -	if (flags & REF_HAVE_NEW)
    -+	if (new_target)
    -+		update->new_target = xstrdup(new_target);
    -+	if (old_target)
    -+		update->old_target = xstrdup(old_target);
    -+	if (new_oid && flags & REF_HAVE_NEW)
    ++	update->new_target = xstrdup_or_null(new_target);
    ++	update->old_target = xstrdup_or_null(old_target);
    ++	if ((flags & REF_HAVE_NEW) && new_oid)
      		oidcpy(&update->new_oid, new_oid);
     -	if (flags & REF_HAVE_OLD)
    -+	if (old_oid && flags & REF_HAVE_OLD)
    ++	if ((flags & REF_HAVE_OLD) && old_oid)
      		oidcpy(&update->old_oid, old_oid);
     +
      	update->msg = normalize_reflog_message(msg);
    @@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char
      	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
      }
     +
    -+int ref_update_is_null_new_value(struct ref_update *update)
    ++int ref_update_has_null_new_value(struct ref_update *update)
     +{
     +	return !update->new_target && is_null_oid(&update->new_oid);
     +}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      	files_assert_main_repository(refs, "lock_ref_for_update");
      
     -	if ((update->flags & REF_HAVE_NEW) && is_null_oid(&update->new_oid))
    -+	if ((update->flags & REF_HAVE_NEW) && ref_update_is_null_new_value(update))
    ++	if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update))
      		update->flags |= REF_DELETING;
      
      	if (head_ref) {
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     -	    !(update->flags & REF_DELETING) &&
     -	    !(update->flags & REF_LOG_ONLY)) {
     +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_target)) {
    ++		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      		if (!(update->type & REF_ISSYMREF) &&
      		    oideq(&lock->old_oid, &update->new_oid)) {
      			/*
    +@@ refs/files-backend.c: static int files_transaction_prepare(struct ref_store *ref_store,
    + 	return ret;
    + }
    + 
    ++static int parse_and_write_reflog(struct files_ref_store *refs,
    ++				  struct ref_update *update,
    ++				  struct ref_lock *lock,
    ++				  struct strbuf *err)
    ++{
    ++	if (update->new_target) {
    ++		/*
    ++		 * We want to get the resolved OID for the target, to ensure
    ++		 * that the correct value is added to the reflog.
    ++		 */
    ++		if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
    ++					     RESOLVE_REF_READING,
    ++					     &update->new_oid, NULL)) {
    ++			/*
    ++			 * TODO: currently we skip creating reflogs for dangling
    ++			 * symref updates. It would be nice to capture this as
    ++			 * zero oid updates however.
    ++			 */
    ++			return 0;
    ++		}
    ++	}
    ++
    ++	if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid,
    ++				&update->new_oid, update->msg, update->flags, err)) {
    ++		char *old_msg = strbuf_detach(err, NULL);
    ++
    ++		strbuf_addf(err, "cannot update the ref '%s': %s",
    ++			    lock->ref_name, old_msg);
    ++		free(old_msg);
    ++		unlock_ref(lock);
    ++		update->backend_data = NULL;
    ++		return -1;
    ++	}
    ++
    ++	return 0;
    ++}
    ++
    + static int files_transaction_finish(struct ref_store *ref_store,
    + 				    struct ref_transaction *transaction,
    + 				    struct strbuf *err)
     @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
      
      		if (update->flags & REF_NEEDS_COMMIT ||
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
     -						&update->new_oid,
     -						update->msg, update->flags,
     -						err)) {
    -+			int create_reflog = 1;
    -+
    -+			if (update->new_target) {
    -+				/*
    -+				 * We want to get the resolved OID for the target, to ensure
    -+				 * that the correct value is added to the reflog.
    -+				 */
    -+				if (!refs_resolve_ref_unsafe(&refs->base, update->new_target,
    -+							     RESOLVE_REF_READING, &update->new_oid, NULL)) {
    -+					/* for dangling symrefs we skip creating a reflog entry. */
    -+					create_reflog = 0;
    -+				}
    -+			}
    -+
    -+			if (create_reflog && files_log_ref_write(refs,
    -+								 lock->ref_name,
    -+								 &lock->old_oid,
    -+								 &update->new_oid,
    -+								 update->msg, update->flags,
    -+								 err)) {
    - 				char *old_msg = strbuf_detach(err, NULL);
    - 
    - 				strbuf_addf(err, "cannot update the ref '%s': %s",
    -@@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_store,
    +-				char *old_msg = strbuf_detach(err, NULL);
    +-
    +-				strbuf_addf(err, "cannot update the ref '%s': %s",
    +-					    lock->ref_name, old_msg);
    +-				free(old_msg);
    +-				unlock_ref(lock);
    +-				update->backend_data = NULL;
    ++			if (parse_and_write_reflog(refs, update, lock, err)) {
    + 				ret = TRANSACTION_GENERIC_ERROR;
      				goto cleanup;
      			}
      		}
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * takes into consideration that the update could be a regular
     + * ref or a symbolic ref.
     + */
    -+int ref_update_is_null_new_value(struct ref_update *update);
    ++int ref_update_has_null_new_value(struct ref_update *update);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      			 * when the reference in question doesn't exist.
      			 */
     -			 if (u->flags & REF_HAVE_NEW && !is_null_oid(&u->new_oid)) {
    -+			 if (u->flags & REF_HAVE_NEW && !ref_update_is_null_new_value(u)) {
    ++			 if ((u->flags & REF_HAVE_NEW) && !ref_update_has_null_new_value(u)) {
      				 ret = queue_transaction_update(refs, tx_data, u,
      								&current_oid, err);
      				 if (ret)
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
     +				ret = -1;
     +				goto done;
     +			}
    -+		} else if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
    ++		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
      			if (is_null_oid(&u->old_oid))
      				strbuf_addf(err, _("cannot lock ref '%s': "
      					    "reference already exists"),
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && ref_update_is_null_new_value(u)) {
    ++		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      			    should_write_log(&arg->refs->base, u->refname))) {
      			struct reftable_log_record *log;
     +			int create_reflog = 1;
    ++
    ++			if (u->new_target) {
    ++				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
    ++							     RESOLVE_REF_READING, &u->new_oid, NULL)) {
    ++					/*
    ++					 * TODO: currently we skip creating reflogs for dangling
    ++					 * symref updates. It would be nice to capture this as
    ++					 * zero oid updates however.
    ++					 */
    ++					create_reflog = 0;
    ++				}
    ++			}
      
     -			ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
     -			log = &logs[logs_nr++];
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     -			memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     -			log->value.update.message =
     -				xstrndup(u->msg, arg->refs->write_options.block_size / 2);
    -+			if (u->new_target)
    -+				if (!refs_resolve_ref_unsafe(&arg->refs->base, u->new_target,
    -+							     RESOLVE_REF_READING, &u->new_oid, NULL))
    -+					/* for dangling symrefs we skip creating reflog */
    -+					create_reflog = 0;
    -+
     +			if (create_reflog) {
     +				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
     +				log = &logs[logs_nr++];
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      			continue;
      
     -		if (u->flags & REF_HAVE_NEW && is_null_oid(&u->new_oid)) {
    -+		if (u->flags & REF_HAVE_NEW && u->new_target) {
    ++		if (u->new_target) {
     +			struct reftable_ref_record ref = {
     +				.refname = (char *)u->refname,
     +				.value_type = REFTABLE_REF_SYMREF,
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     +			ret = reftable_writer_add_ref(writer, &ref);
     +			if (ret < 0)
     +				goto done;
    -+		} else if (u->flags & REF_HAVE_NEW && ref_update_is_null_new_value(u)) {
    ++		} else if ((u->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(u)) {
      			struct reftable_ref_record ref = {
      				.refname = (char *)u->refname,
      				.update_index = ts,

5: 5b55406430 ! 5: 636bf5ce98 refs: use transaction in refs_create_symref()
@@ Commit message
target. To do this, it uses a ref-backend specific function create_symref().

- In this previous commit, we introduce symref support in transactions.
- This means we can now use transactions to perform symref updates and not
- have to resort to create_symref(). Doing this allows us to remove and
- cleanup create_symref(), which we will do in the following commit.
+ In the previous commits, we introduced symref support in transactions.
+ This means we can now use transactions to perform symref updates and
+ don't have to resort to create_symref(). Doing this allows us to
+ remove and cleanup create_symref(), which we will do in the following
+ commit.

Modify the expected error message for a test in
- 't/t0610-reftable-basics.sh', since the error is now thrown from the
+ 't/t0610-reftable-basics.sh', since the error is now thrown from
'refs.c'. This is because in transactional updates, F/D conflicts are caught before we're in the reference backend.

@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls s
+ git symbolic-ref refs/heads/symref refs/heads/main &&
+
+ cat >expect <<-EOF &&
-+ prepared
-+ $ZERO_OID ref:refs/heads/main refs/heads/symref
-+ committed
-+ $ZERO_OID ref:refs/heads/main refs/heads/symref
++ prepared
++ $ZERO_OID ref:refs/heads/main refs/heads/symref
++ committed
++ $ZERO_OID ref:refs/heads/main refs/heads/symref
+ EOF
+
+ test_cmp expect actual

6:  9e25816e68 = 6:  07fb23374f refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  3836e25932 ! 7:  5c05813bcc refs: remove `create_symref` and associated dead code

@@ Commit message
transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

- This means, we can now remove create_symref() and any code associated
- with it which is no longer used. We remove create_symref() code from
- all the reference backends and also remove it entirely from the
- ref_storage_be struct.
+ We can now remove create_symref() and any code associated with it
+ which is no longer used. We remove create_symref() code from all the
+ reference backends and also remove it entirely from the ref_storage_be
+ struct.

@@ refs/files-backend.c: static int create_ref_symlink(struct ref_lock *lock, const
-
static int create_symref_lock(struct files_ref_store *refs,
struct ref_lock *lock, const char *refname,
- const char *target)
+ const char *target, struct strbuf *err)
@@ refs/files-backend.c: static int create_symref_lock(struct files_ref_store *refs,
return 0;
}
@@ refs/files-backend.c: static int create_symref_lock(struct files_ref_store *refs
- struct ref_lock *lock, const char *refname,
- const char *target, const char *logmsg)
-{
+- struct strbuf err = STRBUF_INIT;
- int ret;
-
- if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
@@ refs/files-backend.c: static int create_symref_lock(struct files_ref_store *refs
- return 0;
- }
-
-- ret = create_symref_lock(refs, lock, refname, target);
+- ret = create_symref_lock(refs, lock, refname, target, &err);
- if (!ret) {
- update_symref_reflog(refs, lock, refname, target, logmsg);
-
- if (commit_ref(lock) < 0)
- return error("unable to write symref for %s: %s", refname,
- strerror(errno));
+- } else {
+- return error("%s", err.buf);
- }
-
- return ret;

Karthik Nayak (7):
refs: accept symref values in ref_transaction_update()
files-backend: extract out create_symref_lock()
refs: support symrefs in 'reference-transaction' hook
refs: add support for transactional symref updates
refs: use transaction in refs_create_symref()
refs: rename refs_create_symref() to refs_update_symref()
refs: remove create_symref and associated dead code

 Documentation/githooks.txt       |  14 +-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           |  89 +++++++++----
 refs.h                           |  20 ++-
 refs/debug.c                     |  13 --
 refs/files-backend.c             | 213 +++++++++++++++++++------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  26 +++-
 refs/reftable-backend.c          | 165 +++++++++---------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t0610-reftable-basics.sh       |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 22 files changed, 351 insertions(+), 246 deletions(-)
Signature · 1 line
2.43.GIT
#148 lore raw

PATCH v6 1/7 refs: accept symref values in `ref_transaction_update()`

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)
branch.c +1 -1
index e4a738fc7b..48af4c3ceb 100644
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
builtin/fast-import.c +3 -2
index dc5a9d32dd..297dfb91a1 100644
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
builtin/fetch.c +1 -1
index 5857d860db..66840b7c5b 100644
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
builtin/receive-pack.c +1 -0
index e8d7df14b6..b150ef39a8 100644
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
builtin/replace.c +1 -1
index da59600ad2..7690687b0e 100644
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
builtin/tag.c +1 -0
index 9a33cb50b4..40a65fdebc 100644
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
builtin/update-ref.c +1 -0
index e46afbc46d..21fdbf6ac8 100644
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
refs.c +17 -5
index 55d2e0b2cb..47bc9dd103 100644
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
refs.h +17 -1
index d278775e08..c7851bf587 100644
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
refs/files-backend.c +6 -6
index a098d14ea0..e4d0aa3d41 100644
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
refs/refs-internal.h +14 -0
index 56641aa57a..108f4ec419 100644
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
refs/reftable-backend.c +2 -2
index 1cda48c504..6104471199 100644
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
sequencer.c +5 -4
index 88de4dc20f..61e007d85f 100644
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
walker.c +1 -1
index c0fd632d92..1b3df43906 100644
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
Signature · 1 line
2.43.GIT
#149 lore raw

Re: [PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

Hi Karthik

Quoted 14 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

Quoted 6 lines

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

The documentation for the new parameters looks good to me now - thanks for updating it. I'm confused about the assertions though as I mentioned in my other message [1].

Best Wishes

Phillip

[1] https://www.lore.kernel.org/git/[email protected]

Quoted 400 lines
---
  branch.c                |  2 +-
  builtin/fast-import.c   |  5 +++--
  builtin/fetch.c         |  2 +-
  builtin/receive-pack.c  |  1 +
  builtin/replace.c       |  2 +-
  builtin/tag.c           |  1 +
  builtin/update-ref.c    |  1 +
  refs.c                  | 22 +++++++++++++++++-----
  refs.h                  | 18 +++++++++++++++++-
  refs/files-backend.c    | 12 ++++++------
  refs/refs-internal.h    | 14 ++++++++++++++
  refs/reftable-backend.c |  4 ++--
  sequencer.c             |  9 +++++----
  walker.c                |  2 +-
  14 files changed, 71 insertions(+), 24 deletions(-)
branch.c +1 -1
index e4a738fc7b..48af4c3ceb 100644
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
  	if (!transaction ||
  		ref_transaction_update(transaction, ref.buf,
  					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
  		ref_transaction_commit(transaction, &err))
  		die("%s", err.buf);
  	ref_transaction_free(transaction);
builtin/fast-import.c +3 -2
index dc5a9d32dd..297dfb91a1 100644
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
  	transaction = ref_transaction_begin(&err);
  	if (!transaction ||
  	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
  	    ref_transaction_commit(transaction, &err)) {
  		ref_transaction_free(transaction);
  		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
  		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
  		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
  			failure |= error("%s", err.buf);
  			goto cleanup;
  		}
builtin/fetch.c +1 -1
index 5857d860db..66840b7c5b 100644
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
  	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
  				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
  	if (ret) {
  		ret = STORE_REF_ERROR_OTHER;
  		goto out;
builtin/receive-pack.c +1 -0
index e8d7df14b6..b150ef39a8 100644
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
  		if (ref_transaction_update(transaction,
  					   namespaced_name,
  					   new_oid, old_oid,
+					   NULL, NULL,
  					   0, "push",
  					   &err)) {
  			rp_error("%s", err.buf);
builtin/replace.c +1 -1
index da59600ad2..7690687b0e 100644
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
  	transaction = ref_transaction_begin(&err);
  	if (!transaction ||
  	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
  	    ref_transaction_commit(transaction, &err))
  		res = error("%s", err.buf);
 
builtin/tag.c +1 -0
index 9a33cb50b4..40a65fdebc 100644
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
  	transaction = ref_transaction_begin(&err);
  	if (!transaction ||
  	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
  				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
  				   reflog_msg.buf, &err) ||
  	    ref_transaction_commit(transaction, &err)) {
builtin/update-ref.c +1 -0
index e46afbc46d..21fdbf6ac8 100644
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
  	if (ref_transaction_update(transaction, refname,
  				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
  				   update_flags | create_reflog_flag,
  				   msg, &err))
  		die("%s", err.buf);
refs.c +17 -5
index 55d2e0b2cb..47bc9dd103 100644
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
  		const char *refname, unsigned int flags,
  		const struct object_id *new_oid,
  		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
  		const char *msg)
  {
  	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
  	if (transaction->state != REF_TRANSACTION_OPEN)
  		BUG("update called for transaction that is not open");
 
+	if (old_oid && !is_null_oid(old_oid) && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && !is_null_oid(new_oid) && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
  	FLEX_ALLOC_STR(update, refname, refname);
  	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
  	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
  			   const char *refname,
  			   const struct object_id *new_oid,
  			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
  			   unsigned int flags, const char *msg,
  			   struct strbuf *err)
  {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
  	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
  	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
  	return 0;
  }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
  		return 1;
  	}
  	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
  }
 
  int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
  		BUG("delete called with old_oid set to zeros");
  	return ref_transaction_update(transaction, refname,
  				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
  }
 
  int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
  		BUG("verify called with old_oid set to NULL");
  	return ref_transaction_update(transaction, refname,
  				      NULL, old_oid,
+				      NULL, NULL,
  				      flags, NULL, err);
  }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
  	t = ref_store_transaction_begin(refs, &err);
  	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
  	    ref_transaction_commit(t, &err)) {
  		ret = 1;
  		ref_transaction_free(t);
refs.h +17 -1
index d278775e08..c7851bf587 100644
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
   *         before the update. A copy of this value is made in the
   *         transaction.
   *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
   *     flags -- flags affecting the update, passed to
   *         update_ref_lock(). Possible flags: REF_NO_DEREF,
   *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
   * beforehand. The old value is checked after the lock is taken to
   * prevent races. If the old value doesn't agree with old_oid, the
   * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
   *
   * See the above comment "Reference transaction updates" for more
   * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
  			   const char *refname,
  			   const struct object_id *new_oid,
  			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
  			   unsigned int flags, const char *msg,
  			   struct strbuf *err);
 
refs/files-backend.c +6 -6
index a098d14ea0..e4d0aa3d41 100644
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
  	ref_transaction_add_update(
  			transaction, r->name,
  			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
  	if (ref_transaction_commit(transaction, &err))
  		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
  		 * packed-refs transaction:
  		 */
  		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
  					   REF_NO_DEREF, NULL, &err))
  			die("failure preparing to create packed reference %s: %s",
  			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
  			transaction, "HEAD",
  			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
  			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
  	/*
  	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
  	new_update = ref_transaction_add_update(
  			transaction, referent, new_flags,
  			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
  	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
  					packed_transaction, update->refname,
  					REF_HAVE_NEW | REF_NO_DEREF,
  					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
  		}
  	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
  		ref_transaction_add_update(packed_transaction, update->refname,
  					   update->flags & ~REF_HAVE_OLD,
  					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
  	}
 
  	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
refs/refs-internal.h +14 -0
index 56641aa57a..108f4ec419 100644
@@ -124,6 +124,19 @@ struct ref_update {
  	 */
  	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
  	/*
  	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
  	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
  		const char *refname, unsigned int flags,
  		const struct object_id *new_oid,
  		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
  		const char *msg);
 
  /*
refs/reftable-backend.c +2 -2
index 1cda48c504..6104471199 100644
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
  			new_update = ref_transaction_add_update(
  					transaction, "HEAD",
  					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
  			string_list_insert(&affected_refnames, new_update->refname);
  		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
  				 */
  				new_update = ref_transaction_add_update(
  						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
  				new_update->parent_update = u;
 
  				/*
sequencer.c +5 -4
index 88de4dc20f..61e007d85f 100644
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
  	if (!transaction ||
  	    ref_transaction_update(transaction, "HEAD",
  				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
  				   0, sb.buf, &err) ||
  	    ref_transaction_commit(transaction, &err)) {
  		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
  	if (!transaction ||
  	    ref_transaction_update(transaction, "HEAD", new_head,
  				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
  	    ref_transaction_commit(transaction, err)) {
  		ret = -1;
  	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
  	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
  		error(_("could not read HEAD"));
  		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
  		   ref_transaction_commit(transaction, &err)) {
  		error("%s", err.buf);
  		ret = -1;
walker.c +1 -1
index c0fd632d92..1b3df43906 100644
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
  		strbuf_reset(&refname);
  		strbuf_addf(&refname, "refs/%s", write_ref[i]);
  		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
  					   msg ? msg : "fetch (unknown)",
  					   &err)) {
  			error("%s", err.buf);
#150 lore raw

Re: [PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

Hey Phillip,

Quoted 28 lines · Phillip Wood

Hi Karthik

Quoted 14 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

Quoted 6 lines

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

The documentation for the new parameters looks good to me now - thanks for updating it. I'm confused about the assertions though as I mentioned in my other message [1].

Quoted 7 lines

Best Wishes

Phillip

[1]
https://www.lore.kernel.org/git/[email protected]

Responding here since this is a newer thread.

This is done because in files-backend we split symref updates (see split_symref_update) and add a new one for the target reference. Here we pass along the update struct. This update struct is memset to 0 and this is after the checks we do. So the 'new_oid' here would be set to 0 (null oid) even if the 'new_target' value is set. This made more sense in the earlier set of patches, but probably a diff like this should work for this series and can be amended later as needed (in the series which adds the symref-* commands).

refs.c +2 -2
index 3645b805c1..20d26da372 100644
@@ -1238,9 +1238,9 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
-	if (old_oid && !is_null_oid(old_oid) && old_target)
+	if (old_oid && old_target)
 		BUG("only one of old_oid and old_target should be non NULL");
-	if (new_oid && !is_null_oid(new_oid) && new_target)
+	if (new_oid && new_target)
 		BUG("only one of new_oid and new_target should be non NULL");
 
 	FLEX_ALLOC_STR(update, refname, refname);
refs/files-backend.c +3 -2
index 3ce260d07d..a718164798 100644
@@ -2328,8 +2328,9 @@ static int split_symref_update(struct ref_update *update,
 
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
-			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target ? NULL : &update->new_oid,
+			update->old_target ? NULL : &update->old_oid,
+			update->new_target, update->old_target, update->msg);
 
 	new_update->parent_update = update;
 

I think it makes sense to make it fool proof and add this, I'll wait for more reviews and re-roll in a day or so.

Thanks for following through.

[snip]

#151 lore raw

Re: [PATCH v6 1/7] refs: accept symref values in `ref_transaction_update()`

Hi Karthik

Quoted 42 lines · Karthik Nayak

Hey Phillip,

Quoted 28 lines · Phillip Wood

Hi Karthik

Quoted 14 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

Quoted 6 lines

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

The documentation for the new parameters looks good to me now - thanks for updating it. I'm confused about the assertions though as I mentioned in my other message [1].

Quoted 7 lines

Best Wishes

Phillip

[1]
https://www.lore.kernel.org/git/[email protected]

Responding here since this is a newer thread.

Quoted 8 lines

This is done because in files-backend we split symref updates (see split_symref_update) and add a new one for the target reference. Here we pass along the update struct. This update struct is memset to 0 and this is after the checks we do. So the 'new_oid' here would be set to 0 (null oid) even if the 'new_target' value is set. This made more sense in the earlier set of patches, but probably a diff like this should work for this series and can be amended later as needed (in the series which adds the symref-* commands).

Thanks for the explanation - it would be good to fix this because the assertions don't catch misuses of ref_transaction_update() at the moment when old_oid points to the null oid and old_target is also set.

Best Wishes

Phillip

Quoted 38 lines
refs.c +2 -2
index 3645b805c1..20d26da372 100644
@@ -1238,9 +1238,9 @@ struct ref_update *ref_transaction_add_update(
  	if (transaction->state != REF_TRANSACTION_OPEN)
  		BUG("update called for transaction that is not open");
 
-	if (old_oid && !is_null_oid(old_oid) && old_target)
+	if (old_oid && old_target)
  		BUG("only one of old_oid and old_target should be non NULL");
-	if (new_oid && !is_null_oid(new_oid) && new_target)
+	if (new_oid && new_target)
  		BUG("only one of new_oid and new_target should be non NULL");
 
  	FLEX_ALLOC_STR(update, refname, refname);
refs/files-backend.c +3 -2
index 3ce260d07d..a718164798 100644
@@ -2328,8 +2328,9 @@ static int split_symref_update(struct ref_update *update,
 
  	new_update = ref_transaction_add_update(
  			transaction, referent, new_flags,
-			&update->new_oid, &update->old_oid,
-			NULL, NULL, update->msg);
+			update->new_target ? NULL : &update->new_oid,
+			update->old_target ? NULL : &update->old_oid,
+			update->new_target, update->old_target, update->msg);
 
  	new_update->parent_update = update;
 

I think it makes sense to make it fool proof and add this, I'll wait for more reviews and re-roll in a day or so.

Thanks for following through.

[snip]

#152 lore raw

PATCH v6 2/7 files-backend: extract out `create_symref_lock()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The function create_symref_locked() creates a symref by creating a '<symref>.lock' file and then committing the symref lock, which creates the final symref.

Extract the early half of create_symref_locked() into a new helper function create_symref_lock(). Because the name of the new function is too similar to the original, rename the original to create_and_commit_symref() to avoid confusion.

The new function create_symref_locked() can be used to create the symref lock in a separate step from that of committing it. This allows to add transactional support for symrefs, where the lock would be created in the preparation step and the lock would be committed in the finish step.

---
 refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 14 deletions(-)
refs/files-backend.c +37 -14
index e4d0aa3d41..74a713090c 100644
@@ -1920,27 +1920,49 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target, struct strbuf *err)
 {
+	if (!fdopen_lock_file(&lock->lk, "w")) {
+		strbuf_addf(err, "unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
+		strbuf_addf(err, "unable to write to %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	struct strbuf err = STRBUF_INIT;
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target, &err);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	} else {
+		return error("%s", err.buf);
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1960,7 +1982,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
Signature · 1 line
2.43.GIT
#153 lore raw

PATCH v6 3/7 refs: support symrefs in 'reference-transaction' hook

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The 'reference-transaction' hook runs whenever a reference update is made to the system. In a previous commit, we added the old_target and new_target fields to the reference_transaction_update(). In following commits we'll also add the code to handle symref's in the reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying the current format:
<old-oid> SP <new-oid> SP <ref-name> LF
to be be:
<old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same. But when either 'old-value' or 'new-value' is a symref, we print the ref as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction' hook's documentation always stated that support for symbolic references may be added in the future.

We do not add any tests in this commit since there is no git command which activates this flow, in an upcoming commit, we'll start using transaction based symref updates as the default, we'll add tests there for the hook too.

---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 20 ++++++++++++++++----
 2 files changed, 25 insertions(+), 9 deletions(-)
Documentation/githooks.txt +9 -5
index ee9b92c90d..06e997131b 100644
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
refs.c +16 -4
index 47bc9dd103..203a101988 100644
@@ -2350,10 +2350,22 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (!(update->flags & REF_HAVE_OLD))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (!(update->flags & REF_HAVE_NEW))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
Signature · 1 line
2.43.GIT
#154 lore raw

PATCH v6 4/7 refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's add support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use ref_update_has_null_new_value(), a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid.

With this, now transactional updates (verify, create, delete, update) can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in 'git-update-ref' in the future.

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

---
 refs.c                  |  15 ++++-
 refs/files-backend.c    | 137 ++++++++++++++++++++++++++++++++++------
 refs/refs-internal.h    |   7 ++
 refs/reftable-backend.c |  77 +++++++++++++++++-----
 4 files changed, 198 insertions(+), 38 deletions(-)
refs.c +13 -2
index 203a101988..acb153b6f8 100644
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,10 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	update->new_target = xstrdup_or_null(new_target);
+	update->old_target = xstrdup_or_null(old_target);
+	if ((flags & REF_HAVE_NEW) && new_oid)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1286,6 +1291,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2814,3 +2820,8 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+int ref_update_has_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
refs/files-backend.c +118 -19

183 lines not on this page · raw message

refs/refs-internal.h +7 -0
index 108f4ec419..03eda70ea4 100644
@@ -749,4 +749,11 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_has_null_new_value(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +60 -17

113 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#155 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Hi Karthik

I've left a few comments below - the most important one is about the error messages in the reftable backend, non of the others are worth re-rolling for on their own.

Quoted 7 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's adds

s/we never supported/we do not support/

s/Let's/This commit/

Quoted 1 line

support for symrefs in both the 'files' and the 'reftable' backend.

s/backend/backends/

Quoted 6 lines

Here, we add and use ref_update_has_null_new_value(), a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid.

With this, now transactional updates (verify, create, delete, update)

s/With this, //

Quoted 4 lines

can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

Excellent

Quoted 2 lines

This also allows us to expose this to users via new commands in 'git-update-ref' in the future.

I'm slightly concerned that splitting out the update-ref changes means we don't have any test coverage of the new code beyond the part that is used by refs_create_symref()

Quoted 33 lines

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

refs/files-backend.c +29 -0
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with current_target, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+static int check_old_target(struct ref_update *update,
+			    const char *current_target,
+			    struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(update->old_target, current_target))
+		return 0;
+
+	if (!strcmp(current_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    current_target, update->old_target);
+
+	return -1;
+}

refs/files-backend.c static int lock_ref_for_update(struct files_ref_store *refs, · line 2627

Quoted 8 lines
refs/files-backend.c +2 -3
@@ -2576,9 +2623,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
  		}
  	}
 
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {

This line looks quite long

Quoted 9 lines
refs/reftable-backend.c +2 -1
@@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
  		 * individual refs. But the error messages match what the files
  		 * backend returns, which keeps our tests happy.
  		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {

refs/reftable-backend.c static int reftable_be_transaction_prepare(struct ref_store *ref_store, · line 954

Quoted 10 lines
refs/reftable-backend.c +10 -0
+				if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);

The messages in this function differ from the equivalent messages in check_old_target() from the files backend above. This is potentially confusing to users, creates more work for translators and makes it hard to write tests that are independent of the backend. Can we export check_old_target() so it can be reused here. If not we should reword these messages to match the other messages all of which talk about not being able to lock the ref.

refs/reftable-backend.c static int write_transaction_table(struct reftable_writer *writer, void *cb_data · line 1063

Quoted 13 lines
refs/reftable-backend.c +5 -1
+				ret = -1;
+				goto done;
+			}
+		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
  			if (is_null_oid(&u->old_oid))
  				strbuf_addf(err, _("cannot lock ref '%s': "
  					    "reference already exists"),
@@ -1043,7 +1060,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
  		 * - `core.logAllRefUpdates` tells us to create the reflog for
  		 *   the given ref.
  		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {

The old line was already quite long and the new one is even longer - perhaps we could break it after the second "&&"

refs/reftable-backend.c static int write_transaction_table(struct reftable_writer *writer, void *cb_data · line 1127

Quoted 10 lines
refs/reftable-backend.c +10 -0
+			if (create_reflog) {
+				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
+				log = &logs[logs_nr++];
+				memset(log, 0, sizeof(*log));
+
+				fill_reftable_log_record(log);
+				log->update_index = ts;
+				log->refname = xstrdup(u->refname);
+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);

Both these lines would benefit from being folded

Best Wishes

Phillip

#156 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Hello Phillip,

Quoted 23 lines · Phillip Wood

Hi Karthik

I've left a few comments below - the most important one is about the error messages in the reftable backend, non of the others are worth re-rolling for on their own.

Quoted 7 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we never supported transactional updates of symrefs. Let's adds

s/we never supported/we do not support/

s/Let's/This commit/

Quoted 1 line

support for symrefs in both the 'files' and the 'reftable' backend.

s/backend/backends/

Quoted 9 lines
Quoted 6 lines

Here, we add and use ref_update_has_null_new_value(), a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid.

With this, now transactional updates (verify, create, delete, update)

s/With this, //

Thanks, will make the above changes.

Quoted 5 lines
Quoted 4 lines

can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

Quoted 9 lines

Excellent

Quoted 2 lines

This also allows us to expose this to users via new commands in 'git-update-ref' in the future.

I'm slightly concerned that splitting out the update-ref changes means we don't have any test coverage of the new code beyond the part that is used by refs_create_symref()

This is definitely true. But I also caught a bunch of edge cases this way because the tests which indirectly use 'refs_create_symref()' are quite intensive.

Quoted 36 lines
Quoted 36 lines

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

refs/files-backend.c +29 -0
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with current_target, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+static int check_old_target(struct ref_update *update,
+			    const char *current_target,
+			    struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(update->old_target, current_target))
+		return 0;
+
+	if (!strcmp(current_target, ""))
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "reference is missing but expected %s",
+			    original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "cannot lock ref '%s': "
+			    "is at %s but expected %s",
+			    original_update_refname(update),
+			    current_target, update->old_target);
+
+	return -1;
+}
@@ -2576,9 +2623,27 @@ static int lock_ref_for_update(struct files_ref_store *refs,
  		}
  	}
 
Quoted 8 lines

refs/files-backend.c static int lock_ref_for_update(struct files_ref_store *refs, · line 2627

Quoted 5 lines
refs/files-backend.c +2 -3
-	if ((update->flags & REF_HAVE_NEW) &&
-	    !(update->flags & REF_DELETING) &&
-	    !(update->flags & REF_LOG_ONLY)) {
+	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {

This line looks quite long

Somehow I find it much easier to read these longer lines, unless there is a logical operator, but I'll split the line mid way with the arguments.

Quoted 20 lines
Quoted 9 lines
refs/reftable-backend.c +2 -1
@@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
  		 * individual refs. But the error messages match what the files
  		 * backend returns, which keeps our tests happy.
  		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {

refs/reftable-backend.c static int reftable_be_transaction_prepare(struct ref_store *ref_store, · line 954

Quoted 10 lines
refs/reftable-backend.c +10 -0
+				if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
Quoted 8 lines

The messages in this function differ from the equivalent messages in check_old_target() from the files backend above. This is potentially confusing to users, creates more work for translators and makes it hard to write tests that are independent of the backend. Can we export check_old_target() so it can be reused here. If not we should reword these messages to match the other messages all of which talk about not being able to lock the ref.

This is very intentional, the way the backends work at this point are quite different and while in the files backend, we talk about locking a particular ref. In the reftable backend we do not lock single refs. We lock tables. So keeping it consistent doesn't make sense here.

However, we could make the files backend similar to this one, I would be okay doing that.

Quoted 8 lines
Quoted 8 lines
refs/reftable-backend.c +4 -0
+				ret = -1;
+				goto done;
+			}
+		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
  			if (is_null_oid(&u->old_oid))
  				strbuf_addf(err, _("cannot lock ref '%s': "
  					    "reference already exists"),
@@ -1043,7 +1060,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data
Quoted 9 lines

refs/reftable-backend.c static int write_transaction_table(struct reftable_writer *writer, void *cb_data · line 1063

Quoted 5 lines
refs/reftable-backend.c +1 -1
  		 * - `core.logAllRefUpdates` tells us to create the reflog for
  		 *   the given ref.
  		 */
-		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {

The old line was already quite long and the new one is even longer - perhaps we could break it after the second "&&"

I will break it both the && I think that is easier on the eyes.

Quoted 13 lines

refs/reftable-backend.c static int write_transaction_table(struct reftable_writer *writer, void *cb_data · line 1127

Quoted 10 lines
refs/reftable-backend.c +10 -0
+			if (create_reflog) {
+				ALLOC_GROW(logs, logs_nr + 1, logs_alloc);
+				log = &logs[logs_nr++];
+				memset(log, 0, sizeof(*log));
+
+				fill_reftable_log_record(log);
+				log->update_index = ts;
+				log->refname = xstrdup(u->refname);
+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);

Both these lines would benefit from being folded

Quoted 3 lines

Best Wishes

Phillip

Thanks for the review.

Karthik

#157 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Hi Karthik

Quoted 31 lines · Karthik Nayak
Quoted 21 lines · Phillip Wood
Quoted 9 lines · Karthik Nayak
refs/reftable-backend.c +2 -1
@@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
   		 * individual refs. But the error messages match what the files
   		 * backend returns, which keeps our tests happy.
   		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {

refs/reftable-backend.c static int reftable_be_transaction_prepare(struct ref_store *ref_store, · line 954

Quoted 10 lines
refs/reftable-backend.c +10 -0
+				if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
Quoted 8 lines

The messages in this function differ from the equivalent messages in check_old_target() from the files backend above. This is potentially confusing to users, creates more work for translators and makes it hard to write tests that are independent of the backend. Can we export check_old_target() so it can be reused here. If not we should reword these messages to match the other messages all of which talk about not being able to lock the ref.

Quoted 3 lines

This is very intentional, the way the backends work at this point are quite different and while in the files backend, we talk about locking a particular ref.

I agree that the existing messages could be improved - these messages are returned when checking the old value of the ref so talking about being unable to lock the ref is not helpful as the important information is that the old value does not match the expected value. However that is not dependent on the backend or on whether the expected value is a symref or an oid so it feels a bit random to make these two messages different.

Quoted 2 lines

In the reftable backend we do not lock single refs. We lock tables. So keeping it consistent doesn't make sense here.

Where an update is prevented by another process holding a lock I think that the granularity of the lock that prevents the ref from being updated is not particularly relevant as far as the user is concerned. As far as I can see the existing error messages in the reftable backend try to be consistent with the messages in the files backend.

Quoted 2 lines

However, we could make the files backend similar to this one, I would be okay doing that.

I would be very happy to see the messages improved for both backends when the old value does not match the expected (oid or symref) value. I do think we should have consistent error messages in this case that are essentially independent of the backend and type of the expected value.

Best Wishes

Phillip

#158 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Quoted 46 lines · Phillip Wood

Hi Karthik

Quoted 31 lines · Karthik Nayak
Quoted 21 lines · Phillip Wood
Quoted 9 lines · Karthik Nayak
refs/reftable-backend.c +2 -1
@@ -938,7 +940,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
   		 * individual refs. But the error messages match what the files
   		 * backend returns, which keeps our tests happy.
   		 */
-		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
+		if (u->old_target) {
+			if (strcmp(referent.buf, u->old_target)) {

refs/reftable-backend.c static int reftable_be_transaction_prepare(struct ref_store *ref_store, · line 954

Quoted 10 lines
refs/reftable-backend.c +10 -0
+				if (!strcmp(referent.buf, ""))
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "reference is missing but expected %s",
+						    original_update_refname(u),
+						    u->old_target);
+				else
+					strbuf_addf(err, "verifying symref target: '%s': "
+						    "is at %s but expected %s",
+						    original_update_refname(u),
+						    referent.buf, u->old_target);
Quoted 8 lines

The messages in this function differ from the equivalent messages in check_old_target() from the files backend above. This is potentially confusing to users, creates more work for translators and makes it hard to write tests that are independent of the backend. Can we export check_old_target() so it can be reused here. If not we should reword these messages to match the other messages all of which talk about not being able to lock the ref.

Quoted 3 lines

This is very intentional, the way the backends work at this point are quite different and while in the files backend, we talk about locking a particular ref.

I agree that the existing messages could be improved - these messages are returned when checking the old value of the ref so talking about being unable to lock the ref is not helpful as the important information is that the old value does not match the expected value. However that is not dependent on the backend or on whether the expected value is a symref or an oid so it feels a bit random to make these two messages different.

Quoted 9 lines
Quoted 2 lines

In the reftable backend we do not lock single refs. We lock tables. So keeping it consistent doesn't make sense here.

Where an update is prevented by another process holding a lock I think that the granularity of the lock that prevents the ref from being updated is not particularly relevant as far as the user is concerned. As far as I can see the existing error messages in the reftable backend try to be consistent with the messages in the files backend.

Well, I agree. But we do have to note, that the files backend always had check_old_oid which has messages along the lines of 'cannot lock ref ...' and now the new check_old_target will not be consistent with those messages. Which is OK, since, like you mentioned, these messages could be improved.

Quoted 11 lines
Quoted 2 lines

However, we could make the files backend similar to this one, I would be okay doing that.

I would be very happy to see the messages improved for both backends when the old value does not match the expected (oid or symref) value. I do think we should have consistent error messages in this case that are essentially independent of the backend and type of the expected value.

Best Wishes

Phillip

Overall, I think we're reaching the same consensus, i.e. to export this functionality to a generic function and shared between the both backends. I will make this change. I think the message used in the reftable backend along the lines of 'verifying symref target ...' is verbose yet generic enough, so I'll keep those messages.

Thanks
Karthik

#159 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Hi Karthik

Quoted 6 lines · Karthik Nayak

Overall, I think we're reaching the same consensus, i.e. to export this functionality to a generic function and shared between the both backends. I will make this change. I think the message used in the reftable backend along the lines of 'verifying symref target ...' is verbose yet generic enough, so I'll keep those messages.

That sounds good to me

Thanks

Phillip

#160 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

In reply to Karthik Nayak

Hi Karthik

Quoted 9 lines · Karthik Nayak
Quoted 4 lines · Phillip Wood

I'm slightly concerned that splitting out the update-ref changes means we don't have any test coverage of the new code beyond the part that is used by refs_create_symref()

This is definitely true. But I also caught a bunch of edge cases this way because the tests which indirectly use 'refs_create_symref()' are quite intensive.

I forgot to say in my last mail that this is good to know. So it sounds like the only new code that isn't being exercised by the tests is the check for the old value?

Best Wishes

Phillip

#161 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Hello,

Quoted 13 lines · Phillip Wood

Hi Karthik

Quoted 9 lines · Karthik Nayak
Quoted 4 lines · Phillip Wood

I'm slightly concerned that splitting out the update-ref changes means we don't have any test coverage of the new code beyond the part that is used by refs_create_symref()

This is definitely true. But I also caught a bunch of edge cases this way because the tests which indirectly use 'refs_create_symref()' are quite intensive.

Quoted 7 lines

I forgot to say in my last mail that this is good to know. So it sounds like the only new code that isn't being exercised by the tests is the check for the old value?

Best Wishes

Phillip

That's correct. I think testing that _currently_ would require us to probably expose and test via the unit testing library. I plan to follow this patch series soon with the symref-* ones. While that's not the best argument for not having full test coverage, I hope it is an OK state to be since that path has no users as of this point.

Karthik

#162 lore raw

Re: [PATCH v6 4/7] refs: add support for transactional symref updates

Hi Karthik

Quoted 6 lines · Karthik Nayak

That's correct. I think testing that _currently_ would require us to probably expose and test via the unit testing library. I plan to follow this patch series soon with the symref-* ones. While that's not the best argument for not having full test coverage, I hope it is an OK state to be since that path has no users as of this point.

That seems reasonable - we have good test coverage for the code we're currently using and you're clearly committed to adding tests for the rest when you extend "git update-ref" to accommodate symrefs.

Thanks

Phillip

#163 lore raw

PATCH v6 5/7 refs: use transaction in `refs_create_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function updates a symref to a given new target. To do this, it uses a ref-backend specific function create_symref().

In the previous commits, we introduced symref support in transactions. This means we can now use transactions to perform symref updates and don't have to resort to create_symref(). Doing this allows us to remove and cleanup create_symref(), which we will do in the following commit.

Modify the expected error message for a test in 't/t0610-reftable-basics.sh', since the error is now thrown from 'refs.c'. This is because in transactional updates, F/D conflicts are caught before we're in the reference backend.

---
 refs.c                           | 24 +++++++++++++++++-------
 t/t0610-reftable-basics.sh       |  2 +-
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 3 files changed, 41 insertions(+), 8 deletions(-)
refs.c +17 -7
index acb153b6f8..195861b09b 100644
@@ -2289,14 +2289,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
t/t0610-reftable-basics.sh +1 -1
index 178791e086..9e8d22bcbd 100755
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err
t/t1416-ref-transaction-hooks.sh +23 -0
index 2092488090..067fd57290 100755
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+	prepared
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	committed
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
Signature · 1 line
2.43.GIT
#164 lore raw

PATCH v6 6/7 refs: rename `refs_create_symref()` to `refs_update_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function is used to update/create a symref. But it doesn't check the old target of the symref, if existing. It force updates the symref. In this regard, the name refs_create_symref() is a bit misleading. So let's rename it to refs_update_symref(). This is akin to how 'git-update-ref(1)' also allows us to create apart from update.

While we're here, rename the arguments in the function to clarify what they actually signify and reduce confusion.

---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)
builtin/branch.c +1 -1
index dd3e3a7dc0..4491f7a20c 100644
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
builtin/worktree.c +1 -1
index 7c6c72536b..480202c517 100644
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
refs.c +5 -7
index 195861b09b..3645b805c1 100644
@@ -2284,10 +2284,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2295,8 +2293,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2312,7 +2310,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
refs.h +1 -1
index c7851bf587..71cc1c58e0 100644
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
t/helper/test-ref-store.c +1 -1
index 82bbf6e2e6..4651e4ced7 100644
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
Signature · 1 line
2.43.GIT
#165 lore raw

PATCH v6 7/7 refs: remove `create_symref` and associated dead code

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

We can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

---
 refs/debug.c            | 13 -------
 refs/files-backend.c    | 67 --------------------------------
 refs/packed-backend.c   |  1 -
 refs/refs-internal.h    |  5 ---
 refs/reftable-backend.c | 86 -----------------------------------------
 5 files changed, 172 deletions(-)
refs/debug.c +0 -13
index c7531b17f0..8be316bb67 100644
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
refs/files-backend.c +0 -67

85 lines not on this page · raw message

refs/packed-backend.c +0 -1
index 4e826c05ff..a937e7dbfc 100644
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
refs/refs-internal.h +0 -5
index 03eda70ea4..0c620d4bce 100644
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
refs/reftable-backend.c +0 -86

98 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#166 lore raw

Re: [PATCH v6 7/7] refs: remove `create_symref` and associated dead code

Quoted 6 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

Quoted 7 lines

We can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

---

This has serious semantic conflicts with in-flight topics. I think I resolved them all correctly while merging it in 'seen', but please double check the result after I push it out.

This comment equally applies to the "force all callers to use get_main_ref_store() on the_repository and remove functions that implicitly used the main ref store of the_repository" topic by Patrick, but we really should devise a bit more smoother way to cope with out of tree and in-flight topics. For example, as the new refs_update_symref() function works exactly like the existing refs_create_symref() function, after renaming all the in-base (i.e., in-tree at the point this topic forks from) users to call the new function, instead of just removing the original one, it would have been nice to guide authors of other in-flight topics by (1) causing build failure and at the same time (2) telling them what they need to do to adjust to the new world order. This patch does only (1) but does a poor job for (2). We may want to establish a better convention than just outright removing and breaking others' topics.

Thanks.

#167 lore raw

Re: [PATCH v6 7/7] refs: remove `create_symref` and associated dead code

Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

We can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

Quoted 7 lines
Quoted 2 lines

---

This has serious semantic conflicts with in-flight topics. I think I resolved them all correctly while merging it in 'seen', but please double check the result after I push it out.

I had a look at the commit pushed to 'seen', looked good to me. Thanks.

Quoted 15 lines

This comment equally applies to the "force all callers to use get_main_ref_store() on the_repository and remove functions that implicitly used the main ref store of the_repository" topic by Patrick, but we really should devise a bit more smoother way to cope with out of tree and in-flight topics. For example, as the new refs_update_symref() function works exactly like the existing refs_create_symref() function, after renaming all the in-base (i.e., in-tree at the point this topic forks from) users to call the new function, instead of just removing the original one, it would have been nice to guide authors of other in-flight topics by (1) causing build failure and at the same time (2) telling them what they need to do to adjust to the new world order. This patch does only (1) but does a poor job for (2). We may want to establish a better convention than just outright removing and breaking others' topics.

Quoted 1 line

Thanks.

Just so I can do better next time, what is your suggestion for doing (2) better?

#168 lore raw

Re: [PATCH v6 0/7] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 11 lines · Karthik Nayak

Previous versions:
V1: https://lore.kernel.org/git/[email protected]/
V2: https://lore.kernel.org/git/[email protected]/
V3: https://lore.kernel.org/git/[email protected]/
V4: https://lore.kernel.org/r/[email protected]
V5: https://lore.kernel.org/r/[email protected]

Changes over v5 are:
- More user friendly error messages.
- create_symref_lock now writes to an err buf, instead of directly to stderr.
- Refactor code to make it easier to read around logical operations.

All changes I saw in the range-diff and diff output with v5 looked sensible improvements.

Will wait for comments and then hopefully we can mark it for 'next' sometime next week, if nothing happens.

Thanks.

#169 lore raw

Re: [PATCH v6 0/7] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 30 lines · Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation. Also by using this new functionality in a pre-existing command, we can leverage the existing tests to catch any inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/[email protected]/
V2: https://lore.kernel.org/git/[email protected]/
V3: https://lore.kernel.org/git/[email protected]/
V4: https://lore.kernel.org/r/[email protected]
V5: https://lore.kernel.org/r/[email protected]

Quoted 5 lines

Changes over v5 are:
- More user friendly error messages.
- create_symref_lock now writes to an err buf, instead of directly to stderr.
- Refactor code to make it easier to read around logical operations.
- Cleanup commit message and fix typos.

Just for the record, I don't really have much to add to this series besides what has already been said.

Thanks!

Patrick

#170 lore raw

v7 cover refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation. Also by using this new functionality in a pre-existing command, we can leverage the existing tests to catch any inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/[email protected]
V2: https://lore.kernel.org/git/[email protected]
V3: https://lore.kernel.org/git/[email protected]
V4: https://lore.kernel.org/r/[email protected]
V5: https://lore.kernel.org/r/[email protected]
V6: https://lore.kernel.org/r/[email protected]

Thanks for all the reviews!

Changes since v6:
* Made the check for old/new oid & target more stricter by removing the exception for null oid's.
* Extracted ref_update_check_old_target and original_update_refname to refs.c
* ^This allowed us to generalize the error messages for non-matching old_target values between files and reftable backend.
* Better line wrapping in code.
* Fixed some grammar in commit messages.

Junio, I've rebased my patches on top of, not sure if there is something else I missed here:
- ps/ci-python-2-deprecation (2a686e00d2)
- ps/reftable-write-options (f5545b173b)
- ps/pseudo-ref-terminology (ad302a68fd)

Range diff against v6:

1: a354190905 ! 1: 2a09350661 refs: accept symref values in ref_transaction_update()

    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	if (transaction->state != REF_TRANSACTION_OPEN)
      		BUG("update called for transaction that is not open");
      
    -+	if (old_oid && !is_null_oid(old_oid) && old_target)
    ++	if (old_oid && old_target)
     +		BUG("only one of old_oid and old_target should be non NULL");
    -+	if (new_oid && !is_null_oid(new_oid) && new_target)
    ++	if (new_oid && new_target)
     +		BUG("only one of new_oid and new_target should be non NULL");
     +
      	FLEX_ALLOC_STR(update, refname, refname);
2:  0d9c5b9804 = 2:  9e76b768b9 files-backend: extract out `create_symref_lock()`
3:  e0219ffd31 = 3:  c1d1c5a651 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  d7d455c9b6 refs: move `original_update_refname` to 'refs.c'
4:  b22c59c722 ! 5:  1c00cd69c6 refs: add support for transactional symref updates
    @@ Commit message
         updates. While this is exposed to users via 'git-update-ref' and its
         '--stdin' mode, it is also used internally within various commands.
     
    -    However, we never supported transactional updates of symrefs. Let's add
    -    support for symrefs in both the 'files' and the 'reftable' backend.
    +    However, we do not support transactional updates of symrefs. This commit
    +    adds support for symrefs in both the 'files' and the 'reftable' backend.
     
         Here, we add and use `ref_update_has_null_new_value()`, a helper
         function which is used to check if there is a new_value in a reference
         update. The new value could either be a symref target `new_target` or a
         OID `new_oid`.
     
    -    With this, now transactional updates (verify, create, delete, update)
    -    can be used for:
    +    We also add another common function `ref_update_check_old_target` which
    +    will be used to check if the update's old_target corresponds to a
    +    reference's current target.
    +
    +    Now transactional updates (verify, create, delete, update) can be used
    +    for:
         - regular refs
         - symbolic refs
         - conversion of regular to symbolic refs and vice versa
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      
      	ref_transaction_add_update(transaction, refname, flags,
      				   new_oid, old_oid, new_target,
    -@@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
    - {
    - 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
    +@@ refs.c: const char *ref_update_original_update_refname(struct ref_update *update)
    + 	return update->refname;
      }
    -+
    + 
     +int ref_update_has_null_new_value(struct ref_update *update)
     +{
     +	return !update->new_target && is_null_oid(&update->new_oid);
     +}
    -
    - ## refs/files-backend.c ##
    -@@ refs/files-backend.c: static const char *original_update_refname(struct ref_update *update)
    - 	return update->refname;
    - }
    - 
    -+/*
    -+ * Check whether the old_target values stored in update are consistent
    -+ * with current_target, which is the symbolic reference's current value.
    -+ * If everything is OK, return 0; otherwise, write an error message to
    -+ * err and return -1.
    -+ */
    -+static int check_old_target(struct ref_update *update,
    -+			    const char *current_target,
    -+			    struct strbuf *err)
    ++
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err)
     +{
     +	if (!update->old_target)
     +		BUG("called without old_target set");
     +
    -+	if (!strcmp(update->old_target, current_target))
    ++	if (!strcmp(referent, update->old_target))
     +		return 0;
     +
    -+	if (!strcmp(current_target, ""))
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++	if (!strcmp(referent, ""))
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "reference is missing but expected %s",
    -+			    original_update_refname(update),
    ++			    ref_update_original_update_refname(update),
     +			    update->old_target);
     +	else
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "is at %s but expected %s",
    -+			    original_update_refname(update),
    -+			    current_target, update->old_target);
    -+
    ++			    ref_update_original_update_refname(update),
    ++			    referent, update->old_target);
     +	return -1;
     +}
    -+
    - /*
    -  * Check whether the REF_HAVE_OLD and old_oid values stored in update
    -  * are consistent with oid, which is the reference's current value. If
    +
    + ## refs/files-backend.c ##
    +@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
    + 
    + 	new_update = ref_transaction_add_update(
    + 			transaction, referent, new_flags,
    +-			&update->new_oid, &update->old_oid,
    +-			NULL, NULL, update->msg);
    ++			update->new_target ? NULL : &update->new_oid,
    ++			update->old_target ? NULL : &update->old_oid,
    ++			update->new_target, update->old_target, update->msg);
    + 
    + 	new_update->parent_update = update;
    + 
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
      
      	files_assert_main_repository(refs, "lock_ref_for_update");
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +			}
     +
     +			if (update->old_target) {
    -+				if (check_old_target(update, referent.buf, err)) {
    ++				if (ref_update_check_old_target(referent.buf, update, err)) {
     +					ret = TRANSACTION_GENERIC_ERROR;
     +					goto out;
     +				}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +		 * be set for symrefs, but we're strict about its usage.
     +		 */
     +		if (update->old_target) {
    -+			if (check_old_target(update, referent.buf, err)) {
    ++			if (ref_update_check_old_target(referent.buf, update, err)) {
     +				ret = TRANSACTION_GENERIC_ERROR;
     +				goto out;
     +			}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     -	    !(update->flags & REF_DELETING) &&
     -	    !(update->flags & REF_LOG_ONLY)) {
     +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
    ++		if (create_symref_lock(refs, lock, update->refname,
    ++				       update->new_target, err)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
      			if (commit_ref(lock)) {
     
      ## refs/refs-internal.h ##
    -@@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct repository *repo,
    +@@ refs/refs-internal.h: struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
       */
    - struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
    + const char *ref_update_original_update_refname(struct ref_update *update);
      
     +/*
     + * Helper function to check if the new value is null, this
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * ref or a symbolic ref.
     + */
     +int ref_update_has_null_new_value(struct ref_update *update);
    ++
    ++/*
    ++ * Check whether the old_target values stored in update are consistent
    ++ * with the referent, which is the symbolic reference's current value.
    ++ * If everything is OK, return 0; otherwise, write an error message to
    ++ * err and return -1.
    ++ */
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      		 */
     -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
     +		if (u->old_target) {
    -+			if (strcmp(referent.buf, u->old_target)) {
    -+				if (!strcmp(referent.buf, ""))
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "reference is missing but expected %s",
    -+						    original_update_refname(u),
    -+						    u->old_target);
    -+				else
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "is at %s but expected %s",
    -+						    original_update_refname(u),
    -+						    referent.buf, u->old_target);
    ++			if (ref_update_check_old_target(referent.buf, u, err)) {
     +				ret = -1;
     +				goto done;
     +			}
     +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
      			if (is_null_oid(&u->old_oid))
      				strbuf_addf(err, _("cannot lock ref '%s': "
    - 					    "reference already exists"),
    + 						   "reference already exists"),
     @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
      		 * - `core.logAllRefUpdates` tells us to create the reflog for
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
    ++		if ((u->flags & REF_HAVE_NEW) &&
    ++		    !(u->type & REF_ISSYMREF) &&
    ++		    ref_update_has_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     -			log = &logs[logs_nr++];
     -			memset(log, 0, sizeof(*log));
     -
    --			fill_reftable_log_record(log);
    +-			fill_reftable_log_record(log, &committer_ident);
     -			log->update_index = ts;
     -			log->refname = xstrdup(u->refname);
     -			memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     +				log = &logs[logs_nr++];
     +				memset(log, 0, sizeof(*log));
     +
    -+				fill_reftable_log_record(log);
    ++				fill_reftable_log_record(log, &committer_ident);
     +				log->update_index = ts;
     +				log->refname = xstrdup(u->refname);
    -+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
    -+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.new_hash,
    ++				       u->new_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.old_hash,
    ++				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     +				log->value.update.message =
     +					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
     +			}

5: 636bf5ce98 ! 6: 027106cfe6 refs: use transaction in refs_create_symref()
@@ refs.c: int refs_create_symref(struct ref_store *refs,

int create_symref(const char *ref_target, const char *refs_heads_master,

- ## t/t0610-reftable-basics.sh ##
-@@ t/t0610-reftable-basics.sh: test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
- git init repo &&
- test_commit -C repo A &&
- cat >expect <<-EOF &&
-- error: unable to write symref for refs/heads: file/directory conflict
-+ error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
- EOF
- test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
- test_cmp expect err
-
## t/t1416-ref-transaction-hooks.sh ##
@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
test_cmp expect target-repo.git/actual

6:  07fb23374f = 7:  6939406143 refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  5c05813bcc ! 8:  bc787b903b refs: remove `create_symref` and associated dead code
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -		.value.symref = (char *)create->target,
     -		.update_index = ts,
     -	};
    +-	struct ident_split committer_ident = {0};
     -	struct reftable_log_record log = {0};
     -	struct object_id new_oid;
     -	struct object_id old_oid;
    +-	const char *committer_info;
     -	int ret;
     -
     -	reftable_writer_set_limits(writer, ts, ts);
     -
    +-	ret = refs_verify_refname_available(&create->refs->base, create->refname,
    +-					    NULL, NULL, create->err);
    +-	if (ret < 0)
    +-		return ret;
    +-
     -	ret = reftable_writer_add_ref(writer, &ref);
     -	if (ret)
     -		return ret;
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -	    !should_write_log(&create->refs->base, create->refname))
     -		return 0;
     -
    --	fill_reftable_log_record(&log);
    +-	committer_info = git_committer_info(0);
    +-	if (split_ident_line(&committer_ident, committer_info, strlen(committer_info)))
    +-		BUG("failed splitting committer info");
    +-
    +-	fill_reftable_log_record(&log, &committer_ident);
     -	log.refname = xstrdup(create->refname);
     -	log.update_index = ts;
     -	log.value.update.message = xstrndup(create->logmsg,
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -	struct reftable_ref_store *refs =
     -		reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_symref");
     -	struct reftable_stack *stack = stack_for(refs, refname, &refname);
    +-	struct strbuf err = STRBUF_INIT;
     -	struct write_create_symref_arg arg = {
     -		.refs = refs,
     -		.stack = stack,
     -		.refname = refname,
     -		.target = target,
     -		.logmsg = logmsg,
    +-		.err = &err,
     -	};
     -	int ret;
     -
    @@ refs/reftable-backend.c: struct write_create_symref_arg {
     -
     -done:
     -	assert(ret != REFTABLE_API_ERROR);
    --	if (ret)
    --		error("unable to write symref for %s: %s", refname,
    --		      reftable_error_str(ret));
    +-	if (ret) {
    +-		if (err.len)
    +-			error("%s", err.buf);
    +-		else
    +-			error("unable to write symref for %s: %s", refname,
    +-			      reftable_error_str(ret));
    +-	}
    +-
    +-	strbuf_release(&err);
     -	return ret;
     -}
     -

Karthik Nayak (8):
refs: accept symref values in ref_transaction_update()
files-backend: extract out create_symref_lock()
refs: support symrefs in 'reference-transaction' hook
refs: move original_update_refname to 'refs.c'
refs: add support for transactional symref updates
refs: use transaction in refs_create_symref()
refs: rename refs_create_symref() to refs_update_symref()
refs: remove create_symref and associated dead code

 Documentation/githooks.txt       |  14 ++-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           | 119 ++++++++++++++----
 refs.h                           |  20 ++-
 refs/debug.c                     |  13 --
 refs/files-backend.c             | 208 +++++++++++++++++--------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  40 +++++-
 refs/reftable-backend.c          | 202 +++++++++---------------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 21 files changed, 373 insertions(+), 298 deletions(-)
Signature · 1 line
2.43.GIT
#171 lore raw

PATCH v7 1/8 refs: accept symref values in `ref_transaction_update()`

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)
branch.c +1 -1
index e4a738fc7b..48af4c3ceb 100644
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
builtin/fast-import.c +3 -2
index dc5a9d32dd..297dfb91a1 100644
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
builtin/fetch.c +1 -1
index 5857d860db..66840b7c5b 100644
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
builtin/receive-pack.c +1 -0
index e8d7df14b6..b150ef39a8 100644
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
builtin/replace.c +1 -1
index da59600ad2..7690687b0e 100644
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
builtin/tag.c +1 -0
index 9a33cb50b4..40a65fdebc 100644
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
builtin/update-ref.c +1 -0
index e46afbc46d..21fdbf6ac8 100644
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
refs.c +17 -5
index 7c3c7465a4..d7df137904 100644
@@ -1268,6 +1268,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1275,6 +1276,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1293,6 +1299,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1327,7 +1335,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1342,7 +1351,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1355,7 +1365,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1368,6 +1379,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1382,8 +1394,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
refs.h +17 -1
index 8255989e7e..9abbfd35a2 100644
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
refs/files-backend.c +6 -6
index ea927c516d..7e432cf026 100644
@@ -1197,7 +1197,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1291,7 +1291,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2308,7 +2308,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2371,7 +2371,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2762,7 +2762,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3047,7 +3047,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
refs/refs-internal.h +14 -0
index 56641aa57a..108f4ec419 100644
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
refs/reftable-backend.c +2 -2
index 10ee92e1f7..f48ab5ad22 100644
@@ -870,7 +870,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -949,7 +949,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
sequencer.c +5 -4
index 88de4dc20f..61e007d85f 100644
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
walker.c +1 -1
index c0fd632d92..1b3df43906 100644
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
Signature · 1 line
2.43.GIT
#172 lore raw

PATCH v7 2/8 files-backend: extract out `create_symref_lock()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The function create_symref_locked() creates a symref by creating a '<symref>.lock' file and then committing the symref lock, which creates the final symref.

Extract the early half of create_symref_locked() into a new helper function create_symref_lock(). Because the name of the new function is too similar to the original, rename the original to create_and_commit_symref() to avoid confusion.

The new function create_symref_locked() can be used to create the symref lock in a separate step from that of committing it. This allows to add transactional support for symrefs, where the lock would be created in the preparation step and the lock would be committed in the finish step.

---
 refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 14 deletions(-)
refs/files-backend.c +37 -14
index 7e432cf026..40cc715ea7 100644
@@ -1919,27 +1919,49 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target, struct strbuf *err)
 {
+	if (!fdopen_lock_file(&lock->lk, "w")) {
+		strbuf_addf(err, "unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
+		strbuf_addf(err, "unable to write to %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	struct strbuf err = STRBUF_INIT;
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target, &err);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	} else {
+		return error("%s", err.buf);
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1959,7 +1981,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
Signature · 1 line
2.43.GIT
#173 lore raw

PATCH v7 3/8 refs: support symrefs in 'reference-transaction' hook

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The 'reference-transaction' hook runs whenever a reference update is made to the system. In a previous commit, we added the old_target and new_target fields to the reference_transaction_update(). In following commits we'll also add the code to handle symref's in the reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying the current format:
<old-oid> SP <new-oid> SP <ref-name> LF
to be be:
<old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same. But when either 'old-value' or 'new-value' is a symref, we print the ref as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction' hook's documentation always stated that support for symbolic references may be added in the future.

We do not add any tests in this commit since there is no git command which activates this flow, in an upcoming commit, we'll start using transaction based symref updates as the default, we'll add tests there for the hook too.

---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 20 ++++++++++++++++----
 2 files changed, 25 insertions(+), 9 deletions(-)
Documentation/githooks.txt +9 -5
index ee9b92c90d..06e997131b 100644
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
refs.c +16 -4
index d7df137904..2bb3e09425 100644
@@ -2366,10 +2366,22 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (!(update->flags & REF_HAVE_OLD))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (!(update->flags & REF_HAVE_NEW))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
Signature · 1 line
2.43.GIT
#174 lore raw

PATCH v7 4/8 refs: move `original_update_refname` to 'refs.c'

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The files backend and the reftable backend implement original_update_refname to obtain the original refname of the update. Move it out to 'refs.c' and only expose it internally to the refs library. This will be used in an upcoming commit to also introduce another common functionality for the two backends.

We also rename the function to ref_update_original_update_refname to keep it consistent with the upcoming other 'ref_update_*' functions that'll be introduced.

---
 refs.c                  |  9 +++++++++
 refs/files-backend.c    | 21 +++++----------------
 refs/refs-internal.h    |  5 +++++
 refs/reftable-backend.c | 24 +++++++-----------------
 4 files changed, 26 insertions(+), 33 deletions(-)
refs.c +9 -0
index 2bb3e09425..d65e4cc41e 100644
@@ -2830,3 +2830,12 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+const char *ref_update_original_update_refname(struct ref_update *update)
+{
+	while (update->parent_update)
+		update = update->parent_update;
+
+	return update->refname;
+}
+
refs/files-backend.c +5 -16
index 40cc715ea7..64d2a50e97 100644
@@ -2422,17 +2422,6 @@ static int split_symref_update(struct ref_update *update,
 	return 0;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-
-	return update->refname;
-}
-
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2449,16 +2438,16 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
 	if (is_null_oid(&update->old_oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference already exists",
-			    original_update_refname(update));
+			    ref_update_original_update_refname(update));
 	else if (is_null_oid(oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference is missing but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(&update->old_oid));
 	else
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "is at %s but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(oid),
 			    oid_to_hex(&update->old_oid));
 
@@ -2512,7 +2501,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 		reason = strbuf_detach(err, NULL);
 		strbuf_addf(err, "cannot lock ref '%s': %s",
-			    original_update_refname(update), reason);
+			    ref_update_original_update_refname(update), reason);
 		free(reason);
 		goto out;
 	}
@@ -2532,7 +2521,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 				if (update->flags & REF_HAVE_OLD) {
 					strbuf_addf(err, "cannot lock ref '%s': "
 						    "error reading reference",
-						    original_update_refname(update));
+						    ref_update_original_update_refname(update));
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
refs/refs-internal.h +5 -0
index 108f4ec419..617b93a6c8 100644
@@ -749,4 +749,9 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Return the refname under which update was originally requested.
+ */
+const char *ref_update_original_update_refname(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +7 -17
index f48ab5ad22..0988f59ed4 100644
@@ -622,16 +622,6 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store,
 	return ret;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-	return update->refname;
-}
-
 struct reftable_transaction_update {
 	struct ref_update *update;
 	struct object_id current_oid;
@@ -910,7 +900,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			/* The reference does not exist, but we expected it to. */
 			strbuf_addf(err, _("cannot lock ref '%s': "
 				    "unable to resolve reference '%s'"),
-				    original_update_refname(u), u->refname);
+				    ref_update_original_update_refname(u), u->refname);
 			ret = -1;
 			goto done;
 		}
@@ -982,17 +972,17 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference already exists"),
-					    original_update_refname(u));
+						   "reference already exists"),
+					    ref_update_original_update_refname(u));
 			else if (is_null_oid(&current_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference is missing but expected %s"),
-					    original_update_refname(u),
+						   "reference is missing but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&u->old_oid));
 			else
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "is at %s but expected %s"),
-					    original_update_refname(u),
+						   "is at %s but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&current_oid),
 					    oid_to_hex(&u->old_oid));
 			ret = -1;
Signature · 1 line
2.43.GIT
#175 lore raw

PATCH v7 5/8 refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we do not support transactional updates of symrefs. This commit adds support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use ref_update_has_null_new_value(), a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid.

We also add another common function ref_update_check_old_target which will be used to check if the update's old_target corresponds to a reference's current target.

Now transactional updates (verify, create, delete, update) can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in 'git-update-ref' in the future.

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

---
 refs.c                  |  36 ++++++++++++-
 refs/files-backend.c    | 113 ++++++++++++++++++++++++++++++++--------
 refs/refs-internal.h    |  16 ++++++
 refs/reftable-backend.c |  71 +++++++++++++++++++------
 4 files changed, 196 insertions(+), 40 deletions(-)
refs.c +34 -2
index d65e4cc41e..f3d6b3b8e3 100644
@@ -1257,6 +1257,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1287,10 +1289,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	update->new_target = xstrdup_or_null(new_target);
+	update->old_target = xstrdup_or_null(old_target);
+	if ((flags & REF_HAVE_NEW) && new_oid)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1333,6 +1338,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2839,3 +2845,29 @@ const char *ref_update_original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+int ref_update_has_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
+
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(referent, update->old_target))
+		return 0;
+
+	if (!strcmp(referent, ""))
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "reference is missing but expected %s",
+			    ref_update_original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "is at %s but expected %s",
+			    ref_update_original_update_refname(update),
+			    referent, update->old_target);
+	return -1;
+}
refs/files-backend.c +92 -21

159 lines not on this page · raw message

refs/refs-internal.h +16 -0
index 617b93a6c8..819157256e 100644
@@ -754,4 +754,20 @@ struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
  */
 const char *ref_update_original_update_refname(struct ref_update *update);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_has_null_new_value(struct ref_update *update);
+
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with the referent, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +54 -17

107 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#176 lore raw

PATCH v7 6/8 refs: use transaction in `refs_create_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function updates a symref to a given new target. To do this, it uses a ref-backend specific function create_symref().

In the previous commits, we introduced symref support in transactions. This means we can now use transactions to perform symref updates and don't have to resort to create_symref(). Doing this allows us to remove and cleanup create_symref(), which we will do in the following commit.

Modify the expected error message for a test in 't/t0610-reftable-basics.sh', since the error is now thrown from 'refs.c'. This is because in transactional updates, F/D conflicts are caught before we're in the reference backend.

---
 refs.c                           | 24 +++++++++++++++++-------
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 2 files changed, 40 insertions(+), 7 deletions(-)
refs.c +17 -7
index f3d6b3b8e3..7a693dbcfb 100644
@@ -2305,14 +2305,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
t/t1416-ref-transaction-hooks.sh +23 -0
index 2092488090..067fd57290 100755
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+	prepared
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	committed
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
Signature · 1 line
2.43.GIT
#177 lore raw

PATCH v7 7/8 refs: rename `refs_create_symref()` to `refs_update_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function is used to update/create a symref. But it doesn't check the old target of the symref, if existing. It force updates the symref. In this regard, the name refs_create_symref() is a bit misleading. So let's rename it to refs_update_symref(). This is akin to how 'git-update-ref(1)' also allows us to create apart from update.

While we're here, rename the arguments in the function to clarify what they actually signify and reduce confusion.

---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)
builtin/branch.c +1 -1
index dd3e3a7dc0..4491f7a20c 100644
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
builtin/worktree.c +1 -1
index 7c6c72536b..480202c517 100644
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
refs.c +5 -7
index 7a693dbcfb..85f53a2cbe 100644
@@ -2300,10 +2300,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2311,8 +2309,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2328,7 +2326,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
refs.h +1 -1
index 9abbfd35a2..7c69e8cc0d 100644
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
t/helper/test-ref-store.c +1 -1
index 82bbf6e2e6..4651e4ced7 100644
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
Signature · 1 line
2.43.GIT
#178 lore raw

PATCH v7 8/8 refs: remove `create_symref` and associated dead code

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

We can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

---
 refs/debug.c            |  13 -----
 refs/files-backend.c    |  67 -------------------------
 refs/packed-backend.c   |   1 -
 refs/refs-internal.h    |   5 --
 refs/reftable-backend.c | 105 ----------------------------------------
 5 files changed, 191 deletions(-)
refs/debug.c +0 -13
index c7531b17f0..8be316bb67 100644
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
refs/files-backend.c +0 -67

85 lines not on this page · raw message

refs/packed-backend.c +0 -1
index 4e826c05ff..a937e7dbfc 100644
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
refs/refs-internal.h +0 -5
index 819157256e..53a6c5d842 100644
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
refs/reftable-backend.c +0 -105

117 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#179 lore raw

Re: [PATCH v7 0/8] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 5 lines · Karthik Nayak

Junio, I've rebased my patches on top of, not sure if there is something else I missed here:
- ps/ci-python-2-deprecation (2a686e00d2)
- ps/reftable-write-options (f5545b173b)
- ps/pseudo-ref-terminology (ad302a68fd)

This is a somewhat puzzling set of topics. You are not touching ci/lib.sh at all so I do not see the point of including the ci-python one, for example.

#180 lore raw

Re: [PATCH v7 0/8] refs: add support for transactional symref updates

Quoted 11 lines · Junio C Hamano
Quoted 5 lines · Karthik Nayak

Junio, I've rebased my patches on top of, not sure if there is something else I missed here:
- ps/ci-python-2-deprecation (2a686e00d2)
- ps/reftable-write-options (f5545b173b)
- ps/pseudo-ref-terminology (ad302a68fd)

This is a somewhat puzzling set of topics. You are not touching ci/lib.sh at all so I do not see the point of including the ci-python one, for example.

Oh, it is even worse. 2a686e00d2 is not the ci-python thing at all. It is a random point between 'master' and 'seen' that happens to be a merge of that topic, so you basically based your series on top of random collection of many topics that are not even in 'next'. More than 30 of them.

2a686e00d2 Merge branch 'ps/ci-python-2-deprecation' into seen cfae85760b Merge branch 'ow/refspec-glossary-update' into seen 0ac6476973 Merge branch 'ps/ci-enable-minimal-fuzzers-at-gitlab' into seen 2e6fe8d39b Merge branch 'jp/tag-trailer' into seen
2954339080 Merge branch 'rs/external-diff-with-exit-code' into seen b16fe0b6f5 Merge branch 'it/refs-name-conflict' into seen bdec3a9a17 Merge branch 'ps/refs-without-the-repository' into seen b30d6b985f Merge branch 'kn/ref-transaction-symref' into seen 33c1562652 Merge branch 'ds/doc-config-reflow' into seen 053736edd8 Merge branch 'ie/config-includeif-hostname' into seen db453e610c Merge branch 'cw/git-std-lib' into seen
6858402556 Merge branch 'bk/complete-dirname-for-am-and-format-patch' into seen 2ec214595a Merge branch 'bk/complete-send-email' into seen 0810be3417 Merge branch 'jc/rerere-cleanup' into seen
a3d2847041 Merge branch 'ds/send-email-per-message-block' into seen 919b280123 Merge branch 'tb/attr-limits' into jch
27f675fb6d Merge branch 'ps/config-subcommands' into jch f5545b173b Merge branch 'ps/reftable-write-options' into jch c86de5db47 Merge branch 'ew/khash-to-khashl' into jch
ef2bdd9348 Merge branch 'jc/test-workaround-broken-mv' into jch fb794ba19c Merge branch 'jc/rev-parse-fatal-doc' into jch 96aa517d85 Merge branch 'jt/port-ci-whitespace-check-to-gitlab' into jch 941a03e7c9 Merge branch 'jl/git-no-advice' into jch
56c4ee21a1 Merge branch 'ma/win32-unix-domain-socket' into jch 9d2ea80011 Merge branch 'tb/pseudo-merge-reachability-bitmap' into jch ad302a68fd Merge branch 'ps/pseudo-ref-terminology' into jch 2754234e89 Merge branch 'ds/scalar-reconfigure-all-fix' into jch c0340717db Merge branch 'pw/rebase-i-error-message' into jch 89b67e5799 Merge branch 'tb/path-filter-fix' into jch
f40942e3c0 Merge branch 'la/hide-trailer-info' into jch
7b725fb75c Merge branch 'js/cmake-with-test-tool' into jch 1ddb41fd10 Merge branch 'js/unit-test-suite-runner' into jch e95b4f5420 Merge branch 'jc/no-default-attr-tree-in-bare' into jch ed26589653 ### match next

I'll not look at this round, as I know you know better than building a series deliberately on top of these random topics and this was an accidental mistake you'd rather want to correct before asking folks to take a look at the resulting series.

Thanks.

#181 lore raw

v8 cover refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The patch series takes over from the existing patch series, wherein we introduced symref-* commands to git-update-ref. Since there was a ton of discussions on the UX of the patch series and its application, I thought it would be best to shorten the series and split it into multiple smaller series.

This series adds transactional support for symrefs in the reference db. Then we switch refs_create_symref() to start using transactions for symref updates. This allows us to deprecate the create_symref code in the ref_storage_be interface and remove all associated code which is no longer used.

The split was primarily done so we can merge the non-user facing parts of the previous series. While pertaining the user facing components into another set of patches wherein deeper discussion on the UX can be held without worrying about the internal implementation. Also by using this new functionality in a pre-existing command, we can leverage the existing tests to catch any inconsistencies. One of which was how 'git-symbolic-ref' doesn't add reflog for dangling symrefs, which I've modified my patch to do the same.

We also modify the reference transaction hook to support symrefs. For any symref update the reference transaction hook will output the value with a 'ref:' prefix.

Previous versions:
V1: https://lore.kernel.org/git/[email protected]
V2: https://lore.kernel.org/git/[email protected]
V3: https://lore.kernel.org/git/[email protected]
V4: https://lore.kernel.org/r/[email protected]
V5: https://lore.kernel.org/r/[email protected]
V6: https://lore.kernel.org/r/[email protected]
V7: https://lore.kernel.org/r/[email protected]

Thanks for all the reviews!

Changes since v7:
* I had rebased v7 on next. I've rebased v8 on master. That's the only difference between the two versions.

Junio, this might cause conflicts when merging, I think you resolved them for v6 and hope its the same now. Let me know if I can help otherwise somehow.

Changes since v6:
* Made the check for old/new oid & target more stricter by removing the exception for null oid's.
* Extracted ref_update_check_old_target and original_update_refname to refs.c
* ^This allowed us to generalize the error messages for non-matching old_target values between files and reftable backend.
* Better line wrapping in code.
* Fixed some grammar in commit messages.

Range diff against v6:

1: a354190905 ! 1: defc1b3521 refs: accept symref values in ref_transaction_update()

    @@ refs.c: struct ref_update *ref_transaction_add_update(
      	if (transaction->state != REF_TRANSACTION_OPEN)
      		BUG("update called for transaction that is not open");
      
    -+	if (old_oid && !is_null_oid(old_oid) && old_target)
    ++	if (old_oid && old_target)
     +		BUG("only one of old_oid and old_target should be non NULL");
    -+	if (new_oid && !is_null_oid(new_oid) && new_target)
    ++	if (new_oid && new_target)
     +		BUG("only one of new_oid and new_target should be non NULL");
     +
      	FLEX_ALLOC_STR(update, refname, refname);
2:  0d9c5b9804 = 2:  54bb78a27c files-backend: extract out `create_symref_lock()`
3:  e0219ffd31 = 3:  c16b7c5da0 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  2d268f12cc refs: move `original_update_refname` to 'refs.c'
4:  b22c59c722 ! 5:  7db3a2245f refs: add support for transactional symref updates
    @@ Commit message
         updates. While this is exposed to users via 'git-update-ref' and its
         '--stdin' mode, it is also used internally within various commands.
     
    -    However, we never supported transactional updates of symrefs. Let's add
    -    support for symrefs in both the 'files' and the 'reftable' backend.
    +    However, we do not support transactional updates of symrefs. This commit
    +    adds support for symrefs in both the 'files' and the 'reftable' backend.
     
         Here, we add and use `ref_update_has_null_new_value()`, a helper
         function which is used to check if there is a new_value in a reference
         update. The new value could either be a symref target `new_target` or a
         OID `new_oid`.
     
    -    With this, now transactional updates (verify, create, delete, update)
    -    can be used for:
    +    We also add another common function `ref_update_check_old_target` which
    +    will be used to check if the update's old_target corresponds to a
    +    reference's current target.
    +
    +    Now transactional updates (verify, create, delete, update) can be used
    +    for:
         - regular refs
         - symbolic refs
         - conversion of regular to symbolic refs and vice versa
    @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
      
      	ref_transaction_add_update(transaction, refname, flags,
      				   new_oid, old_oid, new_target,
    -@@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
    - {
    - 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
    +@@ refs.c: const char *ref_update_original_update_refname(struct ref_update *update)
    + 	return update->refname;
      }
    -+
    + 
     +int ref_update_has_null_new_value(struct ref_update *update)
     +{
     +	return !update->new_target && is_null_oid(&update->new_oid);
     +}
    -
    - ## refs/files-backend.c ##
    -@@ refs/files-backend.c: static const char *original_update_refname(struct ref_update *update)
    - 	return update->refname;
    - }
    - 
    -+/*
    -+ * Check whether the old_target values stored in update are consistent
    -+ * with current_target, which is the symbolic reference's current value.
    -+ * If everything is OK, return 0; otherwise, write an error message to
    -+ * err and return -1.
    -+ */
    -+static int check_old_target(struct ref_update *update,
    -+			    const char *current_target,
    -+			    struct strbuf *err)
    ++
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err)
     +{
     +	if (!update->old_target)
     +		BUG("called without old_target set");
     +
    -+	if (!strcmp(update->old_target, current_target))
    ++	if (!strcmp(referent, update->old_target))
     +		return 0;
     +
    -+	if (!strcmp(current_target, ""))
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++	if (!strcmp(referent, ""))
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "reference is missing but expected %s",
    -+			    original_update_refname(update),
    ++			    ref_update_original_update_refname(update),
     +			    update->old_target);
     +	else
    -+		strbuf_addf(err, "cannot lock ref '%s': "
    ++		strbuf_addf(err, "verifying symref target: '%s': "
     +			    "is at %s but expected %s",
    -+			    original_update_refname(update),
    -+			    current_target, update->old_target);
    -+
    ++			    ref_update_original_update_refname(update),
    ++			    referent, update->old_target);
     +	return -1;
     +}
    -+
    - /*
    -  * Check whether the REF_HAVE_OLD and old_oid values stored in update
    -  * are consistent with oid, which is the reference's current value. If
    +
    + ## refs/files-backend.c ##
    +@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
    + 
    + 	new_update = ref_transaction_add_update(
    + 			transaction, referent, new_flags,
    +-			&update->new_oid, &update->old_oid,
    +-			NULL, NULL, update->msg);
    ++			update->new_target ? NULL : &update->new_oid,
    ++			update->old_target ? NULL : &update->old_oid,
    ++			update->new_target, update->old_target, update->msg);
    + 
    + 	new_update->parent_update = update;
    + 
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
      
      	files_assert_main_repository(refs, "lock_ref_for_update");
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +			}
     +
     +			if (update->old_target) {
    -+				if (check_old_target(update, referent.buf, err)) {
    ++				if (ref_update_check_old_target(referent.buf, update, err)) {
     +					ret = TRANSACTION_GENERIC_ERROR;
     +					goto out;
     +				}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     +		 * be set for symrefs, but we're strict about its usage.
     +		 */
     +		if (update->old_target) {
    -+			if (check_old_target(update, referent.buf, err)) {
    ++			if (ref_update_check_old_target(referent.buf, update, err)) {
     +				ret = TRANSACTION_GENERIC_ERROR;
     +				goto out;
     +			}
    @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
     -	    !(update->flags & REF_DELETING) &&
     -	    !(update->flags & REF_LOG_ONLY)) {
     +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
    -+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
    ++		if (create_symref_lock(refs, lock, update->refname,
    ++				       update->new_target, err)) {
     +			ret = TRANSACTION_GENERIC_ERROR;
     +			goto out;
     +		}
    @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
      			if (commit_ref(lock)) {
     
      ## refs/refs-internal.h ##
    -@@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct repository *repo,
    +@@ refs/refs-internal.h: struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
       */
    - struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
    + const char *ref_update_original_update_refname(struct ref_update *update);
      
     +/*
     + * Helper function to check if the new value is null, this
    @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
     + * ref or a symbolic ref.
     + */
     +int ref_update_has_null_new_value(struct ref_update *update);
    ++
    ++/*
    ++ * Check whether the old_target values stored in update are consistent
    ++ * with the referent, which is the symbolic reference's current value.
    ++ * If everything is OK, return 0; otherwise, write an error message to
    ++ * err and return -1.
    ++ */
    ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
    ++				struct strbuf *err);
     +
      #endif /* REFS_REFS_INTERNAL_H */
     
    @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
      		 */
     -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
     +		if (u->old_target) {
    -+			if (strcmp(referent.buf, u->old_target)) {
    -+				if (!strcmp(referent.buf, ""))
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "reference is missing but expected %s",
    -+						    original_update_refname(u),
    -+						    u->old_target);
    -+				else
    -+					strbuf_addf(err, "verifying symref target: '%s': "
    -+						    "is at %s but expected %s",
    -+						    original_update_refname(u),
    -+						    referent.buf, u->old_target);
    ++			if (ref_update_check_old_target(referent.buf, u, err)) {
     +				ret = -1;
     +				goto done;
     +			}
     +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
      			if (is_null_oid(&u->old_oid))
      				strbuf_addf(err, _("cannot lock ref '%s': "
    - 					    "reference already exists"),
    + 						   "reference already exists"),
     @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
      		 * - `core.logAllRefUpdates` tells us to create the reflog for
      		 *   the given ref.
      		 */
     -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
    -+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
    ++		if ((u->flags & REF_HAVE_NEW) &&
    ++		    !(u->type & REF_ISSYMREF) &&
    ++		    ref_update_has_null_new_value(u)) {
      			struct reftable_log_record log = {0};
      			struct reftable_iterator it = {0};
      
    @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
     +				fill_reftable_log_record(log);
     +				log->update_index = ts;
     +				log->refname = xstrdup(u->refname);
    -+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
    -+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.new_hash,
    ++				       u->new_oid.hash, GIT_MAX_RAWSZ);
    ++				memcpy(log->value.update.old_hash,
    ++				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     +				log->value.update.message =
     +					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
     +			}
5:  636bf5ce98 = 6:  354ebbe17f refs: use transaction in `refs_create_symref()`
6:  07fb23374f = 7:  c8a23b3454 refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  5c05813bcc = 8:  fa1b8f445b refs: remove `create_symref` and associated dead code

Karthik Nayak (8):
refs: accept symref values in ref_transaction_update()
files-backend: extract out create_symref_lock()
refs: support symrefs in 'reference-transaction' hook
refs: move original_update_refname to 'refs.c'
refs: add support for transactional symref updates
refs: use transaction in refs_create_symref()
refs: rename refs_create_symref() to refs_update_symref()
refs: remove create_symref and associated dead code

 Documentation/githooks.txt       |  14 ++-
 branch.c                         |   2 +-
 builtin/branch.c                 |   2 +-
 builtin/fast-import.c            |   5 +-
 builtin/fetch.c                  |   2 +-
 builtin/receive-pack.c           |   1 +
 builtin/replace.c                |   2 +-
 builtin/tag.c                    |   1 +
 builtin/update-ref.c             |   1 +
 builtin/worktree.c               |   2 +-
 refs.c                           | 119 ++++++++++++++----
 refs.h                           |  20 ++-
 refs/debug.c                     |  13 --
 refs/files-backend.c             | 208 +++++++++++++++++--------------
 refs/packed-backend.c            |   1 -
 refs/refs-internal.h             |  40 +++++-
 refs/reftable-backend.c          | 183 +++++++++------------------
 sequencer.c                      |   9 +-
 t/helper/test-ref-store.c        |   2 +-
 t/t0610-reftable-basics.sh       |   2 +-
 t/t1416-ref-transaction-hooks.sh |  23 ++++
 walker.c                         |   2 +-
 22 files changed, 374 insertions(+), 280 deletions(-)
Signature · 1 line
2.43.GIT
#182 lore raw

PATCH v8 1/8 refs: accept symref values in `ref_transaction_update()`

From: Karthik Nayak <[email protected]>

The function ref_transaction_update() obtains ref information and flags to create a ref_update and add them to the transaction at hand.

To extend symref support in transactions, we need to also accept the old and new ref targets and process it. This commit adds the required parameters to the function and modifies all call sites.

The two parameters added are new_target and old_target. The new_target is used to denote what the reference should point to when the transaction is applied. Some functions allow this parameter to be NULL, meaning that the reference is not changed.

The old_target denotes the value the reference must have before the update. Some functions allow this parameter to be NULL, meaning that the old value of the reference is not checked.

We also update the internal function ref_transaction_add_update() similarly to take the two new parameters.

---
 branch.c                |  2 +-
 builtin/fast-import.c   |  5 +++--
 builtin/fetch.c         |  2 +-
 builtin/receive-pack.c  |  1 +
 builtin/replace.c       |  2 +-
 builtin/tag.c           |  1 +
 builtin/update-ref.c    |  1 +
 refs.c                  | 22 +++++++++++++++++-----
 refs.h                  | 18 +++++++++++++++++-
 refs/files-backend.c    | 12 ++++++------
 refs/refs-internal.h    | 14 ++++++++++++++
 refs/reftable-backend.c |  4 ++--
 sequencer.c             |  9 +++++----
 walker.c                |  2 +-
 14 files changed, 71 insertions(+), 24 deletions(-)
branch.c +1 -1
index e4a738fc7b..48af4c3ceb 100644
@@ -627,7 +627,7 @@ void create_branch(struct repository *r,
 	if (!transaction ||
 		ref_transaction_update(transaction, ref.buf,
 					&oid, forcing ? NULL : null_oid(),
-					0, msg, &err) ||
+					NULL, NULL, 0, msg, &err) ||
 		ref_transaction_commit(transaction, &err))
 		die("%s", err.buf);
 	ref_transaction_free(transaction);
builtin/fast-import.c +3 -2
index dc5a9d32dd..297dfb91a1 100644
@@ -1634,7 +1634,7 @@ static int update_branch(struct branch *b)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, b->name, &b->oid, &old_oid,
-				   0, msg, &err) ||
+				   NULL, NULL, 0, msg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
 		error("%s", err.buf);
@@ -1675,7 +1675,8 @@ static void dump_tags(void)
 		strbuf_addf(&ref_name, "refs/tags/%s", t->name);
 
 		if (ref_transaction_update(transaction, ref_name.buf,
-					   &t->oid, NULL, 0, msg, &err)) {
+					   &t->oid, NULL, NULL, NULL,
+					   0, msg, &err)) {
 			failure |= error("%s", err.buf);
 			goto cleanup;
 		}
builtin/fetch.c +1 -1
index 5857d860db..66840b7c5b 100644
@@ -668,7 +668,7 @@ static int s_update_ref(const char *action,
 
 	ret = ref_transaction_update(transaction, ref->name, &ref->new_oid,
 				     check_old ? &ref->old_oid : NULL,
-				     0, msg, &err);
+				     NULL, NULL, 0, msg, &err);
 	if (ret) {
 		ret = STORE_REF_ERROR_OTHER;
 		goto out;
builtin/receive-pack.c +1 -0
index e8d7df14b6..b150ef39a8 100644
@@ -1595,6 +1595,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		if (ref_transaction_update(transaction,
 					   namespaced_name,
 					   new_oid, old_oid,
+					   NULL, NULL,
 					   0, "push",
 					   &err)) {
 			rp_error("%s", err.buf);
builtin/replace.c +1 -1
index da59600ad2..7690687b0e 100644
@@ -201,7 +201,7 @@ static int replace_object_oid(const char *object_ref,
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, repl, &prev,
-				   0, NULL, &err) ||
+				   NULL, NULL, 0, NULL, &err) ||
 	    ref_transaction_commit(transaction, &err))
 		res = error("%s", err.buf);
 
builtin/tag.c +1 -0
index 9a33cb50b4..40a65fdebc 100644
@@ -660,6 +660,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
 	    ref_transaction_update(transaction, ref.buf, &object, &prev,
+				   NULL, NULL,
 				   create_reflog ? REF_FORCE_CREATE_REFLOG : 0,
 				   reflog_msg.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
builtin/update-ref.c +1 -0
index e46afbc46d..21fdbf6ac8 100644
@@ -204,6 +204,7 @@ static void parse_cmd_update(struct ref_transaction *transaction,
 
 	if (ref_transaction_update(transaction, refname,
 				   &new_oid, have_old ? &old_oid : NULL,
+				   NULL, NULL,
 				   update_flags | create_reflog_flag,
 				   msg, &err))
 		die("%s", err.buf);
refs.c +17 -5
index 55d2e0b2cb..e7b7c48d92 100644
@@ -1228,6 +1228,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg)
 {
 	struct ref_update *update;
@@ -1235,6 +1236,11 @@ struct ref_update *ref_transaction_add_update(
 	if (transaction->state != REF_TRANSACTION_OPEN)
 		BUG("update called for transaction that is not open");
 
+	if (old_oid && old_target)
+		BUG("only one of old_oid and old_target should be non NULL");
+	if (new_oid && new_target)
+		BUG("only one of new_oid and new_target should be non NULL");
+
 	FLEX_ALLOC_STR(update, refname, refname);
 	ALLOC_GROW(transaction->updates, transaction->nr + 1, transaction->alloc);
 	transaction->updates[transaction->nr++] = update;
@@ -1253,6 +1259,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err)
 {
@@ -1280,7 +1288,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
-				   new_oid, old_oid, msg);
+				   new_oid, old_oid, new_target,
+				   old_target, msg);
 	return 0;
 }
 
@@ -1295,7 +1304,8 @@ int ref_transaction_create(struct ref_transaction *transaction,
 		return 1;
 	}
 	return ref_transaction_update(transaction, refname, new_oid,
-				      null_oid(), flags, msg, err);
+				      null_oid(), NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_delete(struct ref_transaction *transaction,
@@ -1308,7 +1318,8 @@ int ref_transaction_delete(struct ref_transaction *transaction,
 		BUG("delete called with old_oid set to zeros");
 	return ref_transaction_update(transaction, refname,
 				      null_oid(), old_oid,
-				      flags, msg, err);
+				      NULL, NULL, flags,
+				      msg, err);
 }
 
 int ref_transaction_verify(struct ref_transaction *transaction,
@@ -1321,6 +1332,7 @@ int ref_transaction_verify(struct ref_transaction *transaction,
 		BUG("verify called with old_oid set to NULL");
 	return ref_transaction_update(transaction, refname,
 				      NULL, old_oid,
+				      NULL, NULL,
 				      flags, NULL, err);
 }
 
@@ -1335,8 +1347,8 @@ int refs_update_ref(struct ref_store *refs, const char *msg,
 
 	t = ref_store_transaction_begin(refs, &err);
 	if (!t ||
-	    ref_transaction_update(t, refname, new_oid, old_oid, flags, msg,
-				   &err) ||
+	    ref_transaction_update(t, refname, new_oid, old_oid, NULL, NULL,
+				   flags, msg, &err) ||
 	    ref_transaction_commit(t, &err)) {
 		ret = 1;
 		ref_transaction_free(t);
refs.h +17 -1
index d278775e08..c7851bf587 100644
@@ -648,6 +648,16 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  *         before the update. A copy of this value is made in the
  *         transaction.
  *
+ *     new_target -- the target reference that the reference will be
+ *         updated to point to. If the reference is a regular reference,
+ *         it will be converted to a symbolic reference. Cannot be set
+ *         together with `new_oid`. A copy of this value is made in the
+ *         transaction.
+ *
+ *     old_target -- the reference that the reference must be pointing to.
+ *         Canont be set together with `old_oid`. A copy of this value is
+ *         made in the transaction.
+ *
  *     flags -- flags affecting the update, passed to
  *         update_ref_lock(). Possible flags: REF_NO_DEREF,
  *         REF_FORCE_CREATE_REFLOG. See those constants for more
@@ -713,7 +723,11 @@ struct ref_transaction *ref_transaction_begin(struct strbuf *err);
  * beforehand. The old value is checked after the lock is taken to
  * prevent races. If the old value doesn't agree with old_oid, the
  * whole transaction fails. If old_oid is NULL, then the previous
- * value is not checked.
+ * value is not checked. If `old_target` is not NULL, treat the reference
+ * as a symbolic ref and validate that its target before the update is
+ * `old_target`. If the `new_target` is not NULL, then the reference
+ * will be updated to a symbolic ref which targets `new_target`.
+ * Together, these allow us to update between regular refs and symrefs.
  *
  * See the above comment "Reference transaction updates" for more
  * information.
@@ -722,6 +736,8 @@ int ref_transaction_update(struct ref_transaction *transaction,
 			   const char *refname,
 			   const struct object_id *new_oid,
 			   const struct object_id *old_oid,
+			   const char *new_target,
+			   const char *old_target,
 			   unsigned int flags, const char *msg,
 			   struct strbuf *err);
 
refs/files-backend.c +6 -6
index a098d14ea0..e4d0aa3d41 100644
@@ -1198,7 +1198,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r)
 	ref_transaction_add_update(
 			transaction, r->name,
 			REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING,
-			null_oid(), &r->oid, NULL);
+			null_oid(), &r->oid, NULL, NULL, NULL);
 	if (ref_transaction_commit(transaction, &err))
 		goto cleanup;
 
@@ -1292,7 +1292,7 @@ static int files_pack_refs(struct ref_store *ref_store,
 		 * packed-refs transaction:
 		 */
 		if (ref_transaction_update(transaction, iter->refname,
-					   iter->oid, NULL,
+					   iter->oid, NULL, NULL, NULL,
 					   REF_NO_DEREF, NULL, &err))
 			die("failure preparing to create packed reference %s: %s",
 			    iter->refname, err.buf);
@@ -2309,7 +2309,7 @@ static int split_head_update(struct ref_update *update,
 			transaction, "HEAD",
 			update->flags | REF_LOG_ONLY | REF_NO_DEREF,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	/*
 	 * Add "HEAD". This insertion is O(N) in the transaction
@@ -2372,7 +2372,7 @@ static int split_symref_update(struct ref_update *update,
 	new_update = ref_transaction_add_update(
 			transaction, referent, new_flags,
 			&update->new_oid, &update->old_oid,
-			update->msg);
+			NULL, NULL, update->msg);
 
 	new_update->parent_update = update;
 
@@ -2763,7 +2763,7 @@ static int files_transaction_prepare(struct ref_store *ref_store,
 					packed_transaction, update->refname,
 					REF_HAVE_NEW | REF_NO_DEREF,
 					&update->new_oid, NULL,
-					NULL);
+					NULL, NULL, NULL);
 		}
 	}
 
@@ -3048,7 +3048,7 @@ static int files_initial_transaction_commit(struct ref_store *ref_store,
 		ref_transaction_add_update(packed_transaction, update->refname,
 					   update->flags & ~REF_HAVE_OLD,
 					   &update->new_oid, &update->old_oid,
-					   NULL);
+					   NULL, NULL, NULL);
 	}
 
 	if (packed_refs_lock(refs->packed_ref_store, 0, err)) {
refs/refs-internal.h +14 -0
index 56641aa57a..108f4ec419 100644
@@ -124,6 +124,19 @@ struct ref_update {
 	 */
 	struct object_id old_oid;
 
+	/*
+	 * If set, point the reference to this value. This can also be
+	 * used to convert regular references to become symbolic refs.
+	 * Cannot be set together with `new_oid`.
+	 */
+	const char *new_target;
+
+	/*
+	 * If set, check that the reference previously pointed to this
+	 * value. Cannot be set together with `old_oid`.
+	 */
+	const char *old_target;
+
 	/*
 	 * One or more of REF_NO_DEREF, REF_FORCE_CREATE_REFLOG,
 	 * REF_HAVE_NEW, REF_HAVE_OLD, or backend-specific flags.
@@ -173,6 +186,7 @@ struct ref_update *ref_transaction_add_update(
 		const char *refname, unsigned int flags,
 		const struct object_id *new_oid,
 		const struct object_id *old_oid,
+		const char *new_target, const char *old_target,
 		const char *msg);
 
 /*
refs/reftable-backend.c +2 -2
index 1cda48c504..6104471199 100644
@@ -829,7 +829,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			new_update = ref_transaction_add_update(
 					transaction, "HEAD",
 					u->flags | REF_LOG_ONLY | REF_NO_DEREF,
-					&u->new_oid, &u->old_oid, u->msg);
+					&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 			string_list_insert(&affected_refnames, new_update->refname);
 		}
 
@@ -908,7 +908,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 				 */
 				new_update = ref_transaction_add_update(
 						transaction, referent.buf, new_flags,
-						&u->new_oid, &u->old_oid, u->msg);
+						&u->new_oid, &u->old_oid, NULL, NULL, u->msg);
 				new_update->parent_update = u;
 
 				/*
sequencer.c +5 -4
index 88de4dc20f..61e007d85f 100644
@@ -665,7 +665,7 @@ static int fast_forward_to(struct repository *r,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD",
 				   to, unborn && !is_rebase_i(opts) ?
-				   null_oid() : from,
+				   null_oid() : from, NULL, NULL,
 				   0, sb.buf, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ref_transaction_free(transaction);
@@ -1298,7 +1298,7 @@ int update_head_with_reflog(const struct commit *old_head,
 	if (!transaction ||
 	    ref_transaction_update(transaction, "HEAD", new_head,
 				   old_head ? &old_head->object.oid : null_oid(),
-				   0, sb.buf, err) ||
+				   NULL, NULL, 0, sb.buf, err) ||
 	    ref_transaction_commit(transaction, err)) {
 		ret = -1;
 	}
@@ -3832,8 +3832,9 @@ static int do_label(struct repository *r, const char *name, int len)
 	} else if (repo_get_oid(r, "HEAD", &head_oid)) {
 		error(_("could not read HEAD"));
 		ret = -1;
-	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
-					  NULL, 0, msg.buf, &err) < 0 ||
+	} else if (ref_transaction_update(transaction, ref_name.buf,
+					  &head_oid, NULL, NULL, NULL,
+					  0, msg.buf, &err) < 0 ||
 		   ref_transaction_commit(transaction, &err)) {
 		error("%s", err.buf);
 		ret = -1;
walker.c +1 -1
index c0fd632d92..1b3df43906 100644
@@ -324,7 +324,7 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 		strbuf_reset(&refname);
 		strbuf_addf(&refname, "refs/%s", write_ref[i]);
 		if (ref_transaction_update(transaction, refname.buf,
-					   oids + i, NULL, 0,
+					   oids + i, NULL, NULL, NULL, 0,
 					   msg ? msg : "fetch (unknown)",
 					   &err)) {
 			error("%s", err.buf);
Signature · 1 line
2.43.GIT
#183 lore raw

PATCH v8 2/8 files-backend: extract out `create_symref_lock()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The function create_symref_locked() creates a symref by creating a '<symref>.lock' file and then committing the symref lock, which creates the final symref.

Extract the early half of create_symref_locked() into a new helper function create_symref_lock(). Because the name of the new function is too similar to the original, rename the original to create_and_commit_symref() to avoid confusion.

The new function create_symref_locked() can be used to create the symref lock in a separate step from that of committing it. This allows to add transactional support for symrefs, where the lock would be created in the preparation step and the lock would be committed in the finish step.

---
 refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 14 deletions(-)
refs/files-backend.c +37 -14
index e4d0aa3d41..74a713090c 100644
@@ -1920,27 +1920,49 @@ static void update_symref_reflog(struct files_ref_store *refs,
 	}
 }
 
-static int create_symref_locked(struct files_ref_store *refs,
-				struct ref_lock *lock, const char *refname,
-				const char *target, const char *logmsg)
+static int create_symref_lock(struct files_ref_store *refs,
+			      struct ref_lock *lock, const char *refname,
+			      const char *target, struct strbuf *err)
 {
+	if (!fdopen_lock_file(&lock->lk, "w")) {
+		strbuf_addf(err, "unable to fdopen %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	if (fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target) < 0) {
+		strbuf_addf(err, "unable to write to %s: %s",
+			     get_lock_file_path(&lock->lk), strerror(errno));
+		return -1;
+	}
+
+	return 0;
+}
+
+static int create_and_commit_symref(struct files_ref_store *refs,
+				    struct ref_lock *lock, const char *refname,
+				    const char *target, const char *logmsg)
+{
+	struct strbuf err = STRBUF_INIT;
+	int ret;
+
 	if (prefer_symlink_refs && !create_ref_symlink(lock, target)) {
 		update_symref_reflog(refs, lock, refname, target, logmsg);
 		return 0;
 	}
 
-	if (!fdopen_lock_file(&lock->lk, "w"))
-		return error("unable to fdopen %s: %s",
-			     get_lock_file_path(&lock->lk), strerror(errno));
+	ret = create_symref_lock(refs, lock, refname, target, &err);
+	if (!ret) {
+		update_symref_reflog(refs, lock, refname, target, logmsg);
 
-	update_symref_reflog(refs, lock, refname, target, logmsg);
+		if (commit_ref(lock) < 0)
+			return error("unable to write symref for %s: %s", refname,
+				     strerror(errno));
+	} else {
+		return error("%s", err.buf);
+	}
 
-	/* no error check; commit_ref will check ferror */
-	fprintf(get_lock_file_fp(&lock->lk), "ref: %s\n", target);
-	if (commit_ref(lock) < 0)
-		return error("unable to write symref for %s: %s", refname,
-			     strerror(errno));
-	return 0;
+	return ret;
 }
 
 static int files_create_symref(struct ref_store *ref_store,
@@ -1960,7 +1982,8 @@ static int files_create_symref(struct ref_store *ref_store,
 		return -1;
 	}
 
-	ret = create_symref_locked(refs, lock, refname, target, logmsg);
+	ret = create_and_commit_symref(refs, lock, refname, target, logmsg);
+
 	unlock_ref(lock);
 	return ret;
 }
Signature · 1 line
2.43.GIT
#184 lore raw

PATCH v8 3/8 refs: support symrefs in 'reference-transaction' hook

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The 'reference-transaction' hook runs whenever a reference update is made to the system. In a previous commit, we added the old_target and new_target fields to the reference_transaction_update(). In following commits we'll also add the code to handle symref's in the reference backends.

Support symrefs also in the 'reference-transaction' hook, by modifying the current format:
<old-oid> SP <new-oid> SP <ref-name> LF
to be be:
<old-value> SP <new-value> SP <ref-name> LF
where for regular refs the output would not change and remain the same. But when either 'old-value' or 'new-value' is a symref, we print the ref as 'ref:<ref-target>'.

This does break backward compatibility, but the 'reference-transaction' hook's documentation always stated that support for symbolic references may be added in the future.

We do not add any tests in this commit since there is no git command which activates this flow, in an upcoming commit, we'll start using transaction based symref updates as the default, we'll add tests there for the hook too.

---
 Documentation/githooks.txt | 14 +++++++++-----
 refs.c                     | 20 ++++++++++++++++----
 2 files changed, 25 insertions(+), 9 deletions(-)
Documentation/githooks.txt +9 -5
index ee9b92c90d..06e997131b 100644
@@ -486,7 +486,7 @@ reference-transaction
 This hook is invoked by any Git command that performs reference
 updates. It executes whenever a reference transaction is prepared,
 committed or aborted and may thus get called multiple times. The hook
-does not cover symbolic references (but that may change in the future).
+also supports symbolic reference updates.
 
 The hook takes exactly one argument, which is the current state the
 given reference transaction is in:
@@ -503,16 +503,20 @@ given reference transaction is in:
 For each reference update that was added to the transaction, the hook
 receives on standard input a line of the format:
 
-  <old-oid> SP <new-oid> SP <ref-name> LF
+  <old-value> SP <new-value> SP <ref-name> LF
 
-where `<old-oid>` is the old object name passed into the reference
-transaction, `<new-oid>` is the new object name to be stored in the
+where `<old-value>` is the old object name passed into the reference
+transaction, `<new-value>` is the new object name to be stored in the
 ref and `<ref-name>` is the full name of the ref. When force updating
 the reference regardless of its current value or when the reference is
-to be created anew, `<old-oid>` is the all-zeroes object name. To
+to be created anew, `<old-value>` is the all-zeroes object name. To
 distinguish these cases, you can inspect the current value of
 `<ref-name>` via `git rev-parse`.
 
+For symbolic reference updates the `<old_value>` and `<new-value>`
+fields could denote references instead of objects. A reference will be
+denoted with a 'ref:' prefix, like `ref:<ref-target>`.
+
 The exit status of the hook is ignored for any state except for the
 "prepared" state. In the "prepared" state, a non-zero exit status will
 cause the transaction to be aborted. The hook will not be called with
refs.c +16 -4
index e7b7c48d92..9d722d798a 100644
@@ -2350,10 +2350,22 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 		struct ref_update *update = transaction->updates[i];
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
+
+		if (!(update->flags & REF_HAVE_OLD))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->old_target)
+			strbuf_addf(&buf, "ref:%s ", update->old_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->old_oid));
+
+		if (!(update->flags & REF_HAVE_NEW))
+			strbuf_addf(&buf, "%s ", oid_to_hex(null_oid()));
+		else if (update->new_target)
+			strbuf_addf(&buf, "ref:%s ", update->new_target);
+		else
+			strbuf_addf(&buf, "%s ", oid_to_hex(&update->new_oid));
+
+		strbuf_addf(&buf, "%s\n", update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
 			if (errno != EPIPE) {
Signature · 1 line
2.43.GIT
#185 lore raw

PATCH v8 4/8 refs: move `original_update_refname` to 'refs.c'

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The files backend and the reftable backend implement original_update_refname to obtain the original refname of the update. Move it out to 'refs.c' and only expose it internally to the refs library. This will be used in an upcoming commit to also introduce another common functionality for the two backends.

We also rename the function to ref_update_original_update_refname to keep it consistent with the upcoming other 'ref_update_*' functions that'll be introduced.

---
 refs.c                  |  9 +++++++++
 refs/files-backend.c    | 21 +++++----------------
 refs/refs-internal.h    |  5 +++++
 refs/reftable-backend.c | 24 +++++++-----------------
 4 files changed, 26 insertions(+), 33 deletions(-)
refs.c +9 -0
index 9d722d798a..8de43dc0fc 100644
@@ -2814,3 +2814,12 @@ int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
 {
 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
 }
+
+const char *ref_update_original_update_refname(struct ref_update *update)
+{
+	while (update->parent_update)
+		update = update->parent_update;
+
+	return update->refname;
+}
+
refs/files-backend.c +5 -16
index 74a713090c..25e5d03496 100644
@@ -2423,17 +2423,6 @@ static int split_symref_update(struct ref_update *update,
 	return 0;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-
-	return update->refname;
-}
-
 /*
  * Check whether the REF_HAVE_OLD and old_oid values stored in update
  * are consistent with oid, which is the reference's current value. If
@@ -2450,16 +2439,16 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,
 	if (is_null_oid(&update->old_oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference already exists",
-			    original_update_refname(update));
+			    ref_update_original_update_refname(update));
 	else if (is_null_oid(oid))
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "reference is missing but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(&update->old_oid));
 	else
 		strbuf_addf(err, "cannot lock ref '%s': "
 			    "is at %s but expected %s",
-			    original_update_refname(update),
+			    ref_update_original_update_refname(update),
 			    oid_to_hex(oid),
 			    oid_to_hex(&update->old_oid));
 
@@ -2513,7 +2502,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 
 		reason = strbuf_detach(err, NULL);
 		strbuf_addf(err, "cannot lock ref '%s': %s",
-			    original_update_refname(update), reason);
+			    ref_update_original_update_refname(update), reason);
 		free(reason);
 		goto out;
 	}
@@ -2533,7 +2522,7 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 				if (update->flags & REF_HAVE_OLD) {
 					strbuf_addf(err, "cannot lock ref '%s': "
 						    "error reading reference",
-						    original_update_refname(update));
+						    ref_update_original_update_refname(update));
 					ret = TRANSACTION_GENERIC_ERROR;
 					goto out;
 				}
refs/refs-internal.h +5 -0
index 108f4ec419..617b93a6c8 100644
@@ -749,4 +749,9 @@ void base_ref_store_init(struct ref_store *refs, struct repository *repo,
  */
 struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
 
+/*
+ * Return the refname under which update was originally requested.
+ */
+const char *ref_update_original_update_refname(struct ref_update *update);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +7 -17
index 6104471199..8bba5d5096 100644
@@ -581,16 +581,6 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store,
 	return ret;
 }
 
-/*
- * Return the refname under which update was originally requested.
- */
-static const char *original_update_refname(struct ref_update *update)
-{
-	while (update->parent_update)
-		update = update->parent_update;
-	return update->refname;
-}
-
 struct reftable_transaction_update {
 	struct ref_update *update;
 	struct object_id current_oid;
@@ -869,7 +859,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 			/* The reference does not exist, but we expected it to. */
 			strbuf_addf(err, _("cannot lock ref '%s': "
 				    "unable to resolve reference '%s'"),
-				    original_update_refname(u), u->refname);
+				    ref_update_original_update_refname(u), u->refname);
 			ret = -1;
 			goto done;
 		}
@@ -941,17 +931,17 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,
 		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
 			if (is_null_oid(&u->old_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference already exists"),
-					    original_update_refname(u));
+						   "reference already exists"),
+					    ref_update_original_update_refname(u));
 			else if (is_null_oid(&current_oid))
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "reference is missing but expected %s"),
-					    original_update_refname(u),
+						   "reference is missing but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&u->old_oid));
 			else
 				strbuf_addf(err, _("cannot lock ref '%s': "
-					    "is at %s but expected %s"),
-					    original_update_refname(u),
+						   "is at %s but expected %s"),
+					    ref_update_original_update_refname(u),
 					    oid_to_hex(&current_oid),
 					    oid_to_hex(&u->old_oid));
 			ret = -1;
Signature · 1 line
2.43.GIT
#186 lore raw

PATCH v8 5/8 refs: add support for transactional symref updates

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The reference backends currently support transactional reference updates. While this is exposed to users via 'git-update-ref' and its '--stdin' mode, it is also used internally within various commands.

However, we do not support transactional updates of symrefs. This commit adds support for symrefs in both the 'files' and the 'reftable' backend.

Here, we add and use ref_update_has_null_new_value(), a helper function which is used to check if there is a new_value in a reference update. The new value could either be a symref target new_target or a OID new_oid.

We also add another common function ref_update_check_old_target which will be used to check if the update's old_target corresponds to a reference's current target.

Now transactional updates (verify, create, delete, update) can be used for:
- regular refs
- symbolic refs
- conversion of regular to symbolic refs and vice versa

This also allows us to expose this to users via new commands in 'git-update-ref' in the future.

Note that a dangling symref update does not record a new reflog entry, which is unchanged before and after this commit.

---
 refs.c                  |  36 ++++++++++++-
 refs/files-backend.c    | 113 ++++++++++++++++++++++++++++++++--------
 refs/refs-internal.h    |  16 ++++++
 refs/reftable-backend.c |  71 +++++++++++++++++++------
 4 files changed, 196 insertions(+), 40 deletions(-)
refs.c +34 -2
index 8de43dc0fc..d0ea7573d8 100644
@@ -1217,6 +1217,8 @@ void ref_transaction_free(struct ref_transaction *transaction)
 
 	for (i = 0; i < transaction->nr; i++) {
 		free(transaction->updates[i]->msg);
+		free((char *)transaction->updates[i]->new_target);
+		free((char *)transaction->updates[i]->old_target);
 		free(transaction->updates[i]);
 	}
 	free(transaction->updates);
@@ -1247,10 +1249,13 @@ struct ref_update *ref_transaction_add_update(
 
 	update->flags = flags;
 
-	if (flags & REF_HAVE_NEW)
+	update->new_target = xstrdup_or_null(new_target);
+	update->old_target = xstrdup_or_null(old_target);
+	if ((flags & REF_HAVE_NEW) && new_oid)
 		oidcpy(&update->new_oid, new_oid);
-	if (flags & REF_HAVE_OLD)
+	if ((flags & REF_HAVE_OLD) && old_oid)
 		oidcpy(&update->old_oid, old_oid);
+
 	update->msg = normalize_reflog_message(msg);
 	return update;
 }
@@ -1286,6 +1291,7 @@ int ref_transaction_update(struct ref_transaction *transaction,
 	flags &= REF_TRANSACTION_UPDATE_ALLOWED_FLAGS;
 
 	flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0);
+	flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0);
 
 	ref_transaction_add_update(transaction, refname, flags,
 				   new_oid, old_oid, new_target,
@@ -2823,3 +2829,29 @@ const char *ref_update_original_update_refname(struct ref_update *update)
 	return update->refname;
 }
 
+int ref_update_has_null_new_value(struct ref_update *update)
+{
+	return !update->new_target && is_null_oid(&update->new_oid);
+}
+
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err)
+{
+	if (!update->old_target)
+		BUG("called without old_target set");
+
+	if (!strcmp(referent, update->old_target))
+		return 0;
+
+	if (!strcmp(referent, ""))
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "reference is missing but expected %s",
+			    ref_update_original_update_refname(update),
+			    update->old_target);
+	else
+		strbuf_addf(err, "verifying symref target: '%s': "
+			    "is at %s but expected %s",
+			    ref_update_original_update_refname(update),
+			    referent, update->old_target);
+	return -1;
+}
refs/files-backend.c +92 -21

159 lines not on this page · raw message

refs/refs-internal.h +16 -0
index 617b93a6c8..819157256e 100644
@@ -754,4 +754,20 @@ struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
  */
 const char *ref_update_original_update_refname(struct ref_update *update);
 
+/*
+ * Helper function to check if the new value is null, this
+ * takes into consideration that the update could be a regular
+ * ref or a symbolic ref.
+ */
+int ref_update_has_null_new_value(struct ref_update *update);
+
+/*
+ * Check whether the old_target values stored in update are consistent
+ * with the referent, which is the symbolic reference's current value.
+ * If everything is OK, return 0; otherwise, write an error message to
+ * err and return -1.
+ */
+int ref_update_check_old_target(const char *referent, struct ref_update *update,
+				struct strbuf *err);
+
 #endif /* REFS_REFS_INTERNAL_H */
refs/reftable-backend.c +54 -17

107 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#187 lore raw

PATCH v8 6/8 refs: use transaction in `refs_create_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function updates a symref to a given new target. To do this, it uses a ref-backend specific function create_symref().

In the previous commits, we introduced symref support in transactions. This means we can now use transactions to perform symref updates and don't have to resort to create_symref(). Doing this allows us to remove and cleanup create_symref(), which we will do in the following commit.

Modify the expected error message for a test in 't/t0610-reftable-basics.sh', since the error is now thrown from 'refs.c'. This is because in transactional updates, F/D conflicts are caught before we're in the reference backend.

---
 refs.c                           | 24 +++++++++++++++++-------
 t/t0610-reftable-basics.sh       |  2 +-
 t/t1416-ref-transaction-hooks.sh | 23 +++++++++++++++++++++++
 3 files changed, 41 insertions(+), 8 deletions(-)
refs.c +17 -7
index d0ea7573d8..507f5f0525 100644
@@ -2289,14 +2289,24 @@ int refs_create_symref(struct ref_store *refs,
 		       const char *refs_heads_master,
 		       const char *logmsg)
 {
-	char *msg;
-	int retval;
+	struct ref_transaction *transaction;
+	struct strbuf err = STRBUF_INIT;
+	int ret = 0;
 
-	msg = normalize_reflog_message(logmsg);
-	retval = refs->be->create_symref(refs, ref_target, refs_heads_master,
-					 msg);
-	free(msg);
-	return retval;
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    ref_transaction_update(transaction, ref_target, NULL, NULL,
+				   refs_heads_master, NULL, REF_NO_DEREF,
+				   logmsg, &err) ||
+	    ref_transaction_commit(transaction, &err)) {
+		ret = error("%s", err.buf);
+	}
+
+	strbuf_release(&err);
+	if (transaction)
+		ref_transaction_free(transaction);
+
+	return ret;
 }
 
 int create_symref(const char *ref_target, const char *refs_heads_master,
t/t0610-reftable-basics.sh +1 -1
index 178791e086..9e8d22bcbd 100755
@@ -286,7 +286,7 @@ test_expect_success 'ref transaction: creating symbolic ref fails with F/D confl
 	git init repo &&
 	test_commit -C repo A &&
 	cat >expect <<-EOF &&
-	error: unable to write symref for refs/heads: file/directory conflict
+	error: ${SQ}refs/heads/main${SQ} exists; cannot create ${SQ}refs/heads${SQ}
 	EOF
 	test_must_fail git -C repo symbolic-ref refs/heads refs/heads/foo 2>err &&
 	test_cmp expect err
t/t1416-ref-transaction-hooks.sh +23 -0
index 2092488090..067fd57290 100755
@@ -134,4 +134,27 @@ test_expect_success 'interleaving hook calls succeed' '
 	test_cmp expect target-repo.git/actual
 '
 
+test_expect_success 'hook captures git-symbolic-ref updates' '
+	test_when_finished "rm actual" &&
+
+	test_hook reference-transaction <<-\EOF &&
+		echo "$*" >>actual
+		while read -r line
+		do
+			printf "%s\n" "$line"
+		done >>actual
+	EOF
+
+	git symbolic-ref refs/heads/symref refs/heads/main &&
+
+	cat >expect <<-EOF &&
+	prepared
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	committed
+	$ZERO_OID ref:refs/heads/main refs/heads/symref
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
Signature · 1 line
2.43.GIT
#188 lore raw

PATCH v8 7/8 refs: rename `refs_create_symref()` to `refs_update_symref()`

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

The refs_create_symref() function is used to update/create a symref. But it doesn't check the old target of the symref, if existing. It force updates the symref. In this regard, the name refs_create_symref() is a bit misleading. So let's rename it to refs_update_symref(). This is akin to how 'git-update-ref(1)' also allows us to create apart from update.

While we're here, rename the arguments in the function to clarify what they actually signify and reduce confusion.

---
 builtin/branch.c          |  2 +-
 builtin/worktree.c        |  2 +-
 refs.c                    | 12 +++++-------
 refs.h                    |  2 +-
 t/helper/test-ref-store.c |  2 +-
 5 files changed, 9 insertions(+), 11 deletions(-)
builtin/branch.c +1 -1
index dd3e3a7dc0..4491f7a20c 100644
@@ -555,7 +555,7 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees,
 			continue;
 
 		refs = get_worktree_ref_store(worktrees[i]);
-		if (refs_create_symref(refs, "HEAD", newref, logmsg))
+		if (refs_update_symref(refs, "HEAD", newref, logmsg))
 			ret = error(_("HEAD of working tree %s is not updated"),
 				    worktrees[i]->path);
 	}
builtin/worktree.c +1 -1
index 7c6c72536b..480202c517 100644
@@ -517,7 +517,7 @@ static int add_worktree(const char *path, const char *refname,
 		ret = refs_update_ref(wt_refs, NULL, "HEAD", &commit->object.oid,
 				      NULL, 0, UPDATE_REFS_MSG_ON_ERR);
 	else
-		ret = refs_create_symref(wt_refs, "HEAD", symref.buf, NULL);
+		ret = refs_update_symref(wt_refs, "HEAD", symref.buf, NULL);
 	if (ret)
 		goto done;
 
refs.c +5 -7
index 507f5f0525..fa5471d219 100644
@@ -2284,10 +2284,8 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled)
 	return peel_object(base, peeled) ? -1 : 0;
 }
 
-int refs_create_symref(struct ref_store *refs,
-		       const char *ref_target,
-		       const char *refs_heads_master,
-		       const char *logmsg)
+int refs_update_symref(struct ref_store *refs, const char *ref,
+		       const char *target, const char *logmsg)
 {
 	struct ref_transaction *transaction;
 	struct strbuf err = STRBUF_INIT;
@@ -2295,8 +2293,8 @@ int refs_create_symref(struct ref_store *refs,
 
 	transaction = ref_store_transaction_begin(refs, &err);
 	if (!transaction ||
-	    ref_transaction_update(transaction, ref_target, NULL, NULL,
-				   refs_heads_master, NULL, REF_NO_DEREF,
+	    ref_transaction_update(transaction, ref, NULL, NULL,
+				   target, NULL, REF_NO_DEREF,
 				   logmsg, &err) ||
 	    ref_transaction_commit(transaction, &err)) {
 		ret = error("%s", err.buf);
@@ -2312,7 +2310,7 @@ int refs_create_symref(struct ref_store *refs,
 int create_symref(const char *ref_target, const char *refs_heads_master,
 		  const char *logmsg)
 {
-	return refs_create_symref(get_main_ref_store(the_repository), ref_target,
+	return refs_update_symref(get_main_ref_store(the_repository), ref_target,
 				  refs_heads_master, logmsg);
 }
 
refs.h +1 -1
index c7851bf587..71cc1c58e0 100644
@@ -606,7 +606,7 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,
 int copy_existing_ref(const char *oldref, const char *newref,
 			const char *logmsg);
 
-int refs_create_symref(struct ref_store *refs, const char *refname,
+int refs_update_symref(struct ref_store *refs, const char *refname,
 		       const char *target, const char *logmsg);
 int create_symref(const char *refname, const char *target, const char *logmsg);
 
t/helper/test-ref-store.c +1 -1
index 82bbf6e2e6..4651e4ced7 100644
@@ -118,7 +118,7 @@ static int cmd_create_symref(struct ref_store *refs, const char **argv)
 	const char *target = notnull(*argv++, "target");
 	const char *logmsg = *argv++;
 
-	return refs_create_symref(refs, refname, target, logmsg);
+	return refs_update_symref(refs, refname, target, logmsg);
 }
 
 static struct flag_definition transaction_flags[] = {
Signature · 1 line
2.43.GIT
#189 lore raw

PATCH v8 8/8 refs: remove `create_symref` and associated dead code

In reply to Karthik Nayak

From: Karthik Nayak <[email protected]>

In the previous commits, we converted refs_create_symref() to utilize transactions to perform symref updates. Earlier refs_create_symref() used create_symref() to do the same.

We can now remove create_symref() and any code associated with it which is no longer used. We remove create_symref() code from all the reference backends and also remove it entirely from the ref_storage_be struct.

---
 refs/debug.c            | 13 -------
 refs/files-backend.c    | 67 --------------------------------
 refs/packed-backend.c   |  1 -
 refs/refs-internal.h    |  5 ---
 refs/reftable-backend.c | 86 -----------------------------------------
 5 files changed, 172 deletions(-)
refs/debug.c +0 -13
index c7531b17f0..8be316bb67 100644
@@ -131,18 +131,6 @@ static int debug_pack_refs(struct ref_store *ref_store, struct pack_refs_opts *o
 	return res;
 }
 
-static int debug_create_symref(struct ref_store *ref_store,
-			       const char *ref_name, const char *target,
-			       const char *logmsg)
-{
-	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
-	int res = drefs->refs->be->create_symref(drefs->refs, ref_name, target,
-						 logmsg);
-	trace_printf_key(&trace_refs, "create_symref: %s -> %s \"%s\": %d\n", ref_name,
-		target, logmsg, res);
-	return res;
-}
-
 static int debug_rename_ref(struct ref_store *ref_store, const char *oldref,
 			    const char *newref, const char *logmsg)
 {
@@ -441,7 +429,6 @@ struct ref_storage_be refs_be_debug = {
 	.initial_transaction_commit = debug_initial_transaction_commit,
 
 	.pack_refs = debug_pack_refs,
-	.create_symref = debug_create_symref,
 	.rename_ref = debug_rename_ref,
 	.copy_ref = debug_copy_ref,
 
refs/files-backend.c +0 -67

85 lines not on this page · raw message

refs/packed-backend.c +0 -1
index 4e826c05ff..a937e7dbfc 100644
@@ -1714,7 +1714,6 @@ struct ref_storage_be refs_be_packed = {
 	.initial_transaction_commit = packed_initial_transaction_commit,
 
 	.pack_refs = packed_pack_refs,
-	.create_symref = NULL,
 	.rename_ref = NULL,
 	.copy_ref = NULL,
 
refs/refs-internal.h +0 -5
index 819157256e..53a6c5d842 100644
@@ -566,10 +566,6 @@ typedef int ref_transaction_commit_fn(struct ref_store *refs,
 
 typedef int pack_refs_fn(struct ref_store *ref_store,
 			 struct pack_refs_opts *opts);
-typedef int create_symref_fn(struct ref_store *ref_store,
-			     const char *ref_target,
-			     const char *refs_heads_master,
-			     const char *logmsg);
 typedef int rename_ref_fn(struct ref_store *ref_store,
 			  const char *oldref, const char *newref,
 			  const char *logmsg);
@@ -690,7 +686,6 @@ struct ref_storage_be {
 	ref_transaction_commit_fn *initial_transaction_commit;
 
 	pack_refs_fn *pack_refs;
-	create_symref_fn *create_symref;
 	rename_ref_fn *rename_ref;
 	copy_ref_fn *copy_ref;
 
refs/reftable-backend.c +0 -86

98 lines not on this page · raw message

Signature · 1 line
2.43.GIT
#190 lore raw

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

In reply to Karthik Nayak

Hi Karthik

Quoted 9 lines · Karthik Nayak

Changes since v6:
* Made the check for old/new oid & target more stricter by removing the exception for null oid's.
* Extracted ref_update_check_old_target and original_update_refname to refs.c
* ^This allowed us to generalize the error messages for non-matching old_target values between files and reftable backend.
* Better line wrapping in code.
* Fixed some grammar in commit messages.

Thanks for re-rolling - these changes address all of my comments on v6 and the range-diff below looks good.

Best Wishes

Phillip

Quoted 270 lines

Range diff against v6:

1: a354190905 ! 1: defc1b3521 refs: accept symref values in ref_transaction_update()

     @@ refs.c: struct ref_update *ref_transaction_add_update(
       	if (transaction->state != REF_TRANSACTION_OPEN)
       		BUG("update called for transaction that is not open");
       
     -+	if (old_oid && !is_null_oid(old_oid) && old_target)
     ++	if (old_oid && old_target)
      +		BUG("only one of old_oid and old_target should be non NULL");
     -+	if (new_oid && !is_null_oid(new_oid) && new_target)
     ++	if (new_oid && new_target)
      +		BUG("only one of new_oid and new_target should be non NULL");
      +
       	FLEX_ALLOC_STR(update, refname, refname);
2:  0d9c5b9804 = 2:  54bb78a27c files-backend: extract out `create_symref_lock()`
3:  e0219ffd31 = 3:  c16b7c5da0 refs: support symrefs in 'reference-transaction' hook
-:  ---------- > 4:  2d268f12cc refs: move `original_update_refname` to 'refs.c'
4:  b22c59c722 ! 5:  7db3a2245f refs: add support for transactional symref updates
     @@ Commit message
          updates. While this is exposed to users via 'git-update-ref' and its
          '--stdin' mode, it is also used internally within various commands.
      
     -    However, we never supported transactional updates of symrefs. Let's add
     -    support for symrefs in both the 'files' and the 'reftable' backend.
     +    However, we do not support transactional updates of symrefs. This commit
     +    adds support for symrefs in both the 'files' and the 'reftable' backend.
      
          Here, we add and use `ref_update_has_null_new_value()`, a helper
          function which is used to check if there is a new_value in a reference
          update. The new value could either be a symref target `new_target` or a
          OID `new_oid`.
      
     -    With this, now transactional updates (verify, create, delete, update)
     -    can be used for:
     +    We also add another common function `ref_update_check_old_target` which
     +    will be used to check if the update's old_target corresponds to a
     +    reference's current target.
     +
     +    Now transactional updates (verify, create, delete, update) can be used
     +    for:
          - regular refs
          - symbolic refs
          - conversion of regular to symbolic refs and vice versa
     @@ refs.c: int ref_transaction_update(struct ref_transaction *transaction,
       
       	ref_transaction_add_update(transaction, refname, flags,
       				   new_oid, old_oid, new_target,
     -@@ refs.c: int copy_existing_ref(const char *oldref, const char *newref, const char *logmsg
     - {
     - 	return refs_copy_existing_ref(get_main_ref_store(the_repository), oldref, newref, logmsg);
     +@@ refs.c: const char *ref_update_original_update_refname(struct ref_update *update)
     + 	return update->refname;
       }
     -+
     +
      +int ref_update_has_null_new_value(struct ref_update *update)
      +{
      +	return !update->new_target && is_null_oid(&update->new_oid);
      +}
     -
     - ## refs/files-backend.c ##
     -@@ refs/files-backend.c: static const char *original_update_refname(struct ref_update *update)
     - 	return update->refname;
     - }
     -
     -+/*
     -+ * Check whether the old_target values stored in update are consistent
     -+ * with current_target, which is the symbolic reference's current value.
     -+ * If everything is OK, return 0; otherwise, write an error message to
     -+ * err and return -1.
     -+ */
     -+static int check_old_target(struct ref_update *update,
     -+			    const char *current_target,
     -+			    struct strbuf *err)
     ++
     ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
     ++				struct strbuf *err)
      +{
      +	if (!update->old_target)
      +		BUG("called without old_target set");
      +
     -+	if (!strcmp(update->old_target, current_target))
     ++	if (!strcmp(referent, update->old_target))
      +		return 0;
      +
     -+	if (!strcmp(current_target, ""))
     -+		strbuf_addf(err, "cannot lock ref '%s': "
     ++	if (!strcmp(referent, ""))
     ++		strbuf_addf(err, "verifying symref target: '%s': "
      +			    "reference is missing but expected %s",
     -+			    original_update_refname(update),
     ++			    ref_update_original_update_refname(update),
      +			    update->old_target);
      +	else
     -+		strbuf_addf(err, "cannot lock ref '%s': "
     ++		strbuf_addf(err, "verifying symref target: '%s': "
      +			    "is at %s but expected %s",
     -+			    original_update_refname(update),
     -+			    current_target, update->old_target);
     -+
     ++			    ref_update_original_update_refname(update),
     ++			    referent, update->old_target);
      +	return -1;
      +}
     -+
     - /*
     -  * Check whether the REF_HAVE_OLD and old_oid values stored in update
     -  * are consistent with oid, which is the reference's current value. If
     +
     + ## refs/files-backend.c ##
     +@@ refs/files-backend.c: static int split_symref_update(struct ref_update *update,
     +
     + 	new_update = ref_transaction_add_update(
     + 			transaction, referent, new_flags,
     +-			&update->new_oid, &update->old_oid,
     +-			NULL, NULL, update->msg);
     ++			update->new_target ? NULL : &update->new_oid,
     ++			update->old_target ? NULL : &update->old_oid,
     ++			update->new_target, update->old_target, update->msg);
     +
     + 	new_update->parent_update = update;
     +
      @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *refs,
       
       	files_assert_main_repository(refs, "lock_ref_for_update");
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      +			}
      +
      +			if (update->old_target) {
     -+				if (check_old_target(update, referent.buf, err)) {
     ++				if (ref_update_check_old_target(referent.buf, update, err)) {
      +					ret = TRANSACTION_GENERIC_ERROR;
      +					goto out;
      +				}
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      +		 * be set for symrefs, but we're strict about its usage.
      +		 */
      +		if (update->old_target) {
     -+			if (check_old_target(update, referent.buf, err)) {
     ++			if (ref_update_check_old_target(referent.buf, update, err)) {
      +				ret = TRANSACTION_GENERIC_ERROR;
      +				goto out;
      +			}
     @@ refs/files-backend.c: static int lock_ref_for_update(struct files_ref_store *ref
      -	    !(update->flags & REF_DELETING) &&
      -	    !(update->flags & REF_LOG_ONLY)) {
      +	if (update->new_target && !(update->flags & REF_LOG_ONLY)) {
     -+		if (create_symref_lock(refs, lock, update->refname, update->new_target, err)) {
     ++		if (create_symref_lock(refs, lock, update->refname,
     ++				       update->new_target, err)) {
      +			ret = TRANSACTION_GENERIC_ERROR;
      +			goto out;
      +		}
     @@ refs/files-backend.c: static int files_transaction_finish(struct ref_store *ref_
       			if (commit_ref(lock)) {
      
       ## refs/refs-internal.h ##
     -@@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct repository *repo,
     +@@ refs/refs-internal.h: struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_stor
        */
     - struct ref_store *maybe_debug_wrap_ref_store(const char *gitdir, struct ref_store *store);
     + const char *ref_update_original_update_refname(struct ref_update *update);
       
      +/*
      + * Helper function to check if the new value is null, this
     @@ refs/refs-internal.h: void base_ref_store_init(struct ref_store *refs, struct re
      + * ref or a symbolic ref.
      + */
      +int ref_update_has_null_new_value(struct ref_update *update);
     ++
     ++/*
     ++ * Check whether the old_target values stored in update are consistent
     ++ * with the referent, which is the symbolic reference's current value.
     ++ * If everything is OK, return 0; otherwise, write an error message to
     ++ * err and return -1.
     ++ */
     ++int ref_update_check_old_target(const char *referent, struct ref_update *update,
     ++				struct strbuf *err);
      +
       #endif /* REFS_REFS_INTERNAL_H */
      
     @@ refs/reftable-backend.c: static int reftable_be_transaction_prepare(struct ref_s
       		 */
      -		if (u->flags & REF_HAVE_OLD && !oideq(&current_oid, &u->old_oid)) {
      +		if (u->old_target) {
     -+			if (strcmp(referent.buf, u->old_target)) {
     -+				if (!strcmp(referent.buf, ""))
     -+					strbuf_addf(err, "verifying symref target: '%s': "
     -+						    "reference is missing but expected %s",
     -+						    original_update_refname(u),
     -+						    u->old_target);
     -+				else
     -+					strbuf_addf(err, "verifying symref target: '%s': "
     -+						    "is at %s but expected %s",
     -+						    original_update_refname(u),
     -+						    referent.buf, u->old_target);
     ++			if (ref_update_check_old_target(referent.buf, u, err)) {
      +				ret = -1;
      +				goto done;
      +			}
      +		} else if ((u->flags & REF_HAVE_OLD) && !oideq(&current_oid, &u->old_oid)) {
       			if (is_null_oid(&u->old_oid))
       				strbuf_addf(err, _("cannot lock ref '%s': "
     - 					    "reference already exists"),
     + 						   "reference already exists"),
      @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writer *writer, void *cb_data
       		 * - `core.logAllRefUpdates` tells us to create the reflog for
       		 *   the given ref.
       		 */
      -		if (u->flags & REF_HAVE_NEW && !(u->type & REF_ISSYMREF) && is_null_oid(&u->new_oid)) {
     -+		if ((u->flags & REF_HAVE_NEW) && !(u->type & REF_ISSYMREF) && ref_update_has_null_new_value(u)) {
     ++		if ((u->flags & REF_HAVE_NEW) &&
     ++		    !(u->type & REF_ISSYMREF) &&
     ++		    ref_update_has_null_new_value(u)) {
       			struct reftable_log_record log = {0};
       			struct reftable_iterator it = {0};
       
     @@ refs/reftable-backend.c: static int write_transaction_table(struct reftable_writ
      +				fill_reftable_log_record(log);
      +				log->update_index = ts;
      +				log->refname = xstrdup(u->refname);
     -+				memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ);
     -+				memcpy(log->value.update.old_hash, tx_update->current_oid.hash, GIT_MAX_RAWSZ);
     ++				memcpy(log->value.update.new_hash,
     ++				       u->new_oid.hash, GIT_MAX_RAWSZ);
     ++				memcpy(log->value.update.old_hash,
     ++				       tx_update->current_oid.hash, GIT_MAX_RAWSZ);
      +				log->value.update.message =
      +					xstrndup(u->msg, arg->refs->write_options.block_size / 2);
      +			}
5:  636bf5ce98 = 6:  354ebbe17f refs: use transaction in `refs_create_symref()`
6:  07fb23374f = 7:  c8a23b3454 refs: rename `refs_create_symref()` to `refs_update_symref()`
7:  5c05813bcc = 8:  fa1b8f445b refs: remove `create_symref` and associated dead code

Karthik Nayak (8):
refs: accept symref values in ref_transaction_update()
files-backend: extract out create_symref_lock()
refs: support symrefs in 'reference-transaction' hook
refs: move original_update_refname to 'refs.c'
refs: add support for transactional symref updates
refs: use transaction in refs_create_symref()
refs: rename refs_create_symref() to refs_update_symref()
refs: remove create_symref and associated dead code

  Documentation/githooks.txt       |  14 ++-
  branch.c                         |   2 +-
  builtin/branch.c                 |   2 +-
  builtin/fast-import.c            |   5 +-
  builtin/fetch.c                  |   2 +-
  builtin/receive-pack.c           |   1 +
  builtin/replace.c                |   2 +-
  builtin/tag.c                    |   1 +
  builtin/update-ref.c             |   1 +
  builtin/worktree.c               |   2 +-
  refs.c                           | 119 ++++++++++++++----
  refs.h                           |  20 ++-
  refs/debug.c                     |  13 --
  refs/files-backend.c             | 208 +++++++++++++++++--------------
  refs/packed-backend.c            |   1 -
  refs/refs-internal.h             |  40 +++++-
  refs/reftable-backend.c          | 183 +++++++++------------------
  sequencer.c                      |   9 +-
  t/helper/test-ref-store.c        |   2 +-
  t/t0610-reftable-basics.sh       |   2 +-
  t/t1416-ref-transaction-hooks.sh |  23 ++++
  walker.c                         |   2 +-
  22 files changed, 374 insertions(+), 280 deletions(-)
#191 lore raw

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

In reply to Karthik Nayak
Quoted 3 lines · Karthik Nayak

Changes since v7:
* I had rebased v7 on next. I've rebased v8 on master. That's the only difference between the two versions.

I've applied them to the same base as used to queue the previous round, which I think is "436d4e5b14 The seventeenth batch". It went without conflicts, and tests fine in isolation. I'll see if it plays well with other topics in 'seen' later in the day but not now.

Thanks.

Quoted 2 lines

Junio, this might cause conflicts when merging, I think you resolved them for v6 and hope its the same now. Let me know if I can help otherwise somehow.

The easiest for both of us would be to do this:

(1) Build on whatever base you want, and format-patch the series. If you are doing "rebase -i" in-place to update from the previous round, this will reuse the previous base so (2) and
(3) may become trivial.

(2) Find the base of where the last round was queued, something like

     $ mine='kn/ref-transaction-symref'
     $ git checkout "origin/seen^{/^Merge branch '$mine'}...master"

(3) Apply your format-patch result. There are three cases

(3)-1. Things apply cleanly and tests fine. Go to (4).

(3)-2. Things apply cleanly but does not build or test fails,
or things do not apply cleanly.

In the latter case, you have textual or semantic conflicts coming from the difference between the old base and the base you used to build in (1). Identify what caused the breakages (e.g., a topic or two may have merged since the base used by
(2) until the base used by (1)).

Check out the latest 'origin/master' (which may be newer than the base used by (2)), "merge --no-ff" the topics you newly depend on in there, and use the result of the merge(s) as the base, rebuild the series and test again. Run format-patch from the last such merges to the tip of your topic. If you did

$ git checkout origin/master
$ git merge --no-ff --into kn/ref-transaction-symref fo/obar $ git merge --no-ff --into kn/ref-transaction-symref ba/zqux ... rebuild the topic ...

Then you'd just format your topic above these "preparing the ground" merges, e.g.

$ git format-patch "HEAD^{/^Merge branch 'ba/zqux'}"..HEAD

Do not forget to write in the cover letter you did this, including the topics you have in your base on top of 'master'. Then go to (4).

(4) Make a trial merge of your topic into 'next' and 'seen', e.g.

     $ git checkout --detach 'origin/seen' &&
       git revert -m 1 <the merge of the previous iteration into seen> &&
       git merge kn/ref-transaction-symref

The "revert" is needed if the previous iteration of your topic is already in 'seen' (like in this case). You could choose to rebuild master..origin/seen from scratch while excluding your previous iteration, which may emulate what happens on my end more closely.

This trial merge may conflict. It is primarily to see what conflicts _other_ topics may have with your topic. In other words, you do not have to depend on to make your topic work on 'master'. It may become the job of the other topic owners to resolve conflicts if your topic goes to 'next' before theirs.

Make a note on what conflict you saw in the cover letter. You do not necessarily have to resolve them, but it would be a good opportunity to learn what others are doing in an related area.

$ git checkout --detach 'origin/next' &&
git merge kn/ref-transaction-symref

This is to see what conflicts your topic has with other topics that are already cooking. This should not conflict if (3)-2 prepared a base on top of updated master plus dependent topics taken from 'next'. Unless the context is severe (one way to tell is try the same trial merge with your old iteration, which may conflict in a similar way), expect that it will be handled on my end (if it gets unmanageable, I'll ask to rebase when I receive your patches).

Something like the above should be added to the SubmittingPatches document (or its successor to cover more advanced topics, perhaps).

Thanks.

#192 lore raw

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

Quoted 23 lines · Junio C Hamano
Quoted 3 lines · Karthik Nayak

Changes since v7:
* I had rebased v7 on next. I've rebased v8 on master. That's the only difference between the two versions.

I've applied them to the same base as used to queue the previous round, which I think is "436d4e5b14 The seventeenth batch". It went without conflicts, and tests fine in isolation. I'll see if it plays well with other topics in 'seen' later in the day but not now.

Thanks.

Quoted 2 lines

Junio, this might cause conflicts when merging, I think you resolved them for v6 and hope its the same now. Let me know if I can help otherwise somehow.

The easiest for both of us would be to do this:

(1) Build on whatever base you want, and format-patch the series. If you are doing "rebase -i" in-place to update from the previous round, this will reuse the previous base so (2) and
(3) may become trivial.

Quoted 5 lines

(2) Find the base of where the last round was queued, something like

     $ mine='kn/ref-transaction-symref'
     $ git checkout "origin/seen^{/^Merge branch '$mine'}...master"

I find the '...' always so confusing, I would say suggesting to use 'git-merge-base' would be much nicer here.

Quoted 7 lines

(3) Apply your format-patch result. There are three cases

(3)-1. Things apply cleanly and tests fine. Go to (4).

(3)-2. Things apply cleanly but does not build or test fails,
or things do not apply cleanly.

Quoted 8 lines

In the latter case, you have textual or semantic conflicts coming from the difference between the old base and the base you used to build in (1). Identify what caused the breakages (e.g., a topic or two may have merged since the base used by
(2) until the base used by (1)).

Check out the latest 'origin/master' (which may be newer than the base used by (2)), "merge --no-ff" the topics you newly

For my own understanding, even if we use '--ff' the end result should be the same, but using '--no-ff' would ensure that the changes and conflicts are isolated to the merge commit, right?

Quoted 9 lines

depend on in there, and use the result of the merge(s) as the base, rebuild the series and test again. Run format-patch from the last such merges to the tip of your topic. If you did

$ git checkout origin/master
$ git merge --no-ff --into kn/ref-transaction-symref fo/obar $ git merge --no-ff --into kn/ref-transaction-symref ba/zqux ... rebuild the topic ...

I guess you mean '--into-name' here? I would skip mentioning this since it doesn't have any real effect and is perhaps confusing.

Quoted 43 lines

Then you'd just format your topic above these "preparing the ground" merges, e.g.

$ git format-patch "HEAD^{/^Merge branch 'ba/zqux'}"..HEAD

Do not forget to write in the cover letter you did this, including the topics you have in your base on top of 'master'. Then go to (4).

(4) Make a trial merge of your topic into 'next' and 'seen', e.g.

     $ git checkout --detach 'origin/seen' &&
       git revert -m 1 <the merge of the previous iteration into seen> &&
       git merge kn/ref-transaction-symref

The "revert" is needed if the previous iteration of your topic is already in 'seen' (like in this case). You could choose to rebuild master..origin/seen from scratch while excluding your previous iteration, which may emulate what happens on my end more closely.

This trial merge may conflict. It is primarily to see what conflicts _other_ topics may have with your topic. In other words, you do not have to depend on to make your topic work on 'master'. It may become the job of the other topic owners to resolve conflicts if your topic goes to 'next' before theirs.

Make a note on what conflict you saw in the cover letter. You do not necessarily have to resolve them, but it would be a good opportunity to learn what others are doing in an related area.

$ git checkout --detach 'origin/next' &&
git merge kn/ref-transaction-symref

This is to see what conflicts your topic has with other topics that are already cooking. This should not conflict if (3)-2 prepared a base on top of updated master plus dependent topics taken from 'next'. Unless the context is severe (one way to tell is try the same trial merge with your old iteration, which may conflict in a similar way), expect that it will be handled on my end (if it gets unmanageable, I'll ask to rebase when I receive your patches).

Quoted 4 lines

Something like the above should be added to the SubmittingPatches document (or its successor to cover more advanced topics, perhaps).

Thanks.

The rest of this looks good, I'll cleanup add the appropriate syntax, merge in your patches [1] and send something soon!

[1]: https://lore.kernel.org/all/[email protected]/#t

#193 lore raw

Re: [PATCH v8 0/8] refs: add support for transactional symref updates

Quoted 2 lines · Karthik Nayak

I find the '...' always so confusing, I would say suggesting to use 'git-merge-base' would be much nicer here.

They are equivalent, except that "..." in

	git checkout --detach A...B

is internal as opposed to

	git checkout --detach $(merge-base A B)

that uses one extra process. I thought people loath spawning processes?

Quoted 6 lines
Quoted 2 lines

Check out the latest 'origin/master' (which may be newer than the base used by (2)), "merge --no-ff" the topics you newly

For my own understanding, even if we use '--ff' the end result should be the same, but using '--no-ff' would ensure that the changes and conflicts are isolated to the merge commit, right?

It will make it easy for you to look at

git log --first-parent --oneline master..

by hiding the base commits you depend on behind a single merge commit.

Quoted 5 lines
Quoted 2 lines
     $ git merge --no-ff --into kn/ref-transaction-symref ba/zqux
     ... rebuild the topic ...

I guess you mean '--into-name' here? I would skip mentioning this since it doesn't have any real effect and is perhaps confusing.

Again this is to help that "one liner first-parent chain" output.