Skip to contentSkip to conversation

git gc fails with "unable to resolve reference" for worktree

PATCH Superseded [email protected] 7 participants 41 messages Feb 18, 2019 – Mar 12, 2019

lore
Share & tools
lore
#0 lore raw

# Steps to reproduce (in terms of terminal commands)

   $ mkdir foo
   $ cd foo
   $ git init
   Initialized empty Git repository in /tmp/foo/.git/
   $ echo hello > testfile
   $ git add testfile && git commit -m "my commit1"
   [master (root-commit) d5f0b47] my commit1
   1 file changed, 1 insertion(+)
   create mode 100644 testfile
   $ git checkout -b bar
   Switched to a new branch 'bar'
   $ git worktree add ../bar\ \(worktree\) master
   Preparing worktree (checking out 'master')
   HEAD is now at d5f0b47 my commit1
   $ git gc
   error: cannot lock ref 'worktrees/bar (worktree)/HEAD': unable to 
resolve reference 'worktrees/bar (worktree)/HEAD': Invalid argument
   fatal: failed to run reflog

# Expected

No errors

# Actual

error: cannot lock ref 'worktrees/bar (worktree)/HEAD': unable to resolve reference 'worktrees/bar (worktree)/HEAD': Invalid argument

#1 lore raw
Quoted 12 lines · On Mon, Feb 18, 2019 at 9:44 PM <[email protected]> wrote:

# Steps to reproduce (in terms of terminal commands)

    $ mkdir foo
    $ cd foo
    $ git init
    Initialized empty Git repository in /tmp/foo/.git/
    $ echo hello > testfile
    $ git add testfile && git commit -m "my commit1"
    [master (root-commit) d5f0b47] my commit1
    1 file changed, 1 insertion(+)
    create mode 100644 testfile
Quoted 8 lines
    $ git checkout -b bar
    Switched to a new branch 'bar'
    $ git worktree add ../bar\ \(worktree\) master
    Preparing worktree (checking out 'master')
    HEAD is now at d5f0b47 my commit1
    $ git gc
    error: cannot lock ref 'worktrees/bar (worktree)/HEAD': unable to
resolve reference 'worktrees/bar (worktree)/HEAD': Invalid argument

Thanks for reporting. This is not a valid reference and causes the problem. The worktree's name has to sanitized. I'll fix it tomorrow.

Quoted 12 lines

fatal: failed to run reflog

# Expected

No errors

# Actual

error: cannot lock ref 'worktrees/bar (worktree)/HEAD': unable to resolve reference 'worktrees/bar (worktree)/HEAD': Invalid argument

Signature · 1 line
Duy
#2 lore raw
Quoted 23 lines · Duy Nguyen
Quoted 13 lines · On Mon, Feb 18, 2019 at 9:44 PM <[email protected]> wrote:

# Steps to reproduce (in terms of terminal commands)

     $ mkdir foo
     $ cd foo
     $ git init
     Initialized empty Git repository in /tmp/foo/.git/
     $ echo hello > testfile
     $ git add testfile && git commit -m "my commit1"
     [master (root-commit) d5f0b47] my commit1
     1 file changed, 1 insertion(+)
     create mode 100644 testfile
     $ git checkout -b bar
Quoted 8 lines
     Switched to a new branch 'bar'
     $ git worktree add ../bar\ \(worktree\) master
     Preparing worktree (checking out 'master')
     HEAD is now at d5f0b47 my commit1
     $ git gc
     error: cannot lock ref 'worktrees/bar (worktree)/HEAD': unable 
to
 resolve reference 'worktrees/bar (worktree)/HEAD': Invalid argument
Quoted 3 lines

Thanks for reporting. This is not a valid reference and causes the problem. The worktree's name has to sanitized. I'll fix it tomorrow.

Quoted 1 line

You mean, you want to prohibit such directory names as a worktree? But it's a proper directory naming, can perhaps git do the sanitizing transparently for end-user?

#3 lore raw
Quoted 27 lines · On Mon, Feb 18, 2019 at 10:09 PM <[email protected]> wrote:
Quoted 23 lines · Duy Nguyen
Quoted 21 lines · On Mon, Feb 18, 2019 at 9:44 PM <[email protected]> wrote:

# Steps to reproduce (in terms of terminal commands)

     $ mkdir foo
     $ cd foo
     $ git init
     Initialized empty Git repository in /tmp/foo/.git/
     $ echo hello > testfile
     $ git add testfile && git commit -m "my commit1"
     [master (root-commit) d5f0b47] my commit1
     1 file changed, 1 insertion(+)
     create mode 100644 testfile
     $ git checkout -b bar
     Switched to a new branch 'bar'
     $ git worktree add ../bar\ \(worktree\) master
     Preparing worktree (checking out 'master')
     HEAD is now at d5f0b47 my commit1
     $ git gc
     error: cannot lock ref 'worktrees/bar (worktree)/HEAD': unable
to
 resolve reference 'worktrees/bar (worktree)/HEAD': Invalid argument
Quoted 7 lines
Quoted 3 lines

Thanks for reporting. This is not a valid reference and causes the problem. The worktree's name has to sanitized. I'll fix it tomorrow.

Quoted 1 line

You mean, you want to prohibit such directory names as a worktree? But it's a proper directory naming, can perhaps git do the sanitizing transparently for end-user?

No, not inhibiting. When you do "git add ../(abc)" then the internal name could be simply "abc" or something like that instead of "(abc)" which is invalid to reflog and other commands. The worktree's location is still '(abc)'. There's also a work-in-progress option to let the user control this worktree name directly.

Signature · 1 line
Duy
#4 lore raw

I see, thanks!

Quoted 48 lines · Duy Nguyen
Quoted 32 lines · On Mon, Feb 18, 2019 at 10:09 PM <[email protected]> wrote:
Quoted 4 lines · Duy Nguyen
Quoted 3 lines · On Mon, Feb 18, 2019 at 9:44 PM <[email protected]> wrote:

# Steps to reproduce (in terms of terminal commands)

Quoted 16 lines
Quoted 8 lines
     $ mkdir foo
     $ cd foo
     $ git init
     Initialized empty Git repository in /tmp/foo/.git/
     $ echo hello > testfile
     $ git add testfile && git commit -m "my commit1"
     [master (root-commit) d5f0b47] my commit1
     1 file changed, 1 insertion(+)
Quoted 8 lines
     create mode 100644 testfile
     $ git checkout -b bar
     Switched to a new branch 'bar'
     $ git worktree add ../bar\ \(worktree\) master
     Preparing worktree (checking out 'master')
     HEAD is now at d5f0b47 my commit1
     $ git gc
     error: cannot lock ref 'worktrees/bar (worktree)/HEAD': 

unable

Quoted 2 lines
Quoted 2 lines

to
resolve reference 'worktrees/bar (worktree)/HEAD': Invalid

argument

Quoted 3 lines

Thanks for reporting. This is not a valid reference and causes the problem. The worktree's name has to sanitized. I'll fix it

Quoted 7 lines

tomorrow.

Quoted 1 line
Quoted 1 line

You mean, you want to prohibit such directory names as a worktree?
But
it's a proper directory naming, can perhaps git do the sanitizing transparently for end-user?

No, not inhibiting. When you do "git add ../(abc)" then the internal name could be simply "abc" or something like that instead of "(abc)" which is invalid to reflog and other commands. The worktree's location is still '(abc)'. There's also a work-in-progress option to let the user control this worktree name directly.
--
Duy

#5 lore raw

PATCH 1/1 worktree add: sanitize worktree names

In reply to [email protected]

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 47 ++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  5 +++++
 2 files changed, 51 insertions(+), 1 deletion(-)
builtin/worktree.c +46 -1
index 3f9907fcc9..ff36838a33 100644
@@ -262,6 +262,46 @@ static void validate_worktree_add(const char *path, const struct add_opts *opts)
 	free_worktrees(worktrees);
 }
 
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to make it
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+	int i;
+
+	/* no ending with .lock */
+	if (ends_with(name->buf, ".lock"))
+		strbuf_remove(name, name->len - strlen(".lock"),
+			      strlen(".lock"));
+
+	/*
+	 * All special chars replaced with dashes. See
+	 * check_refname_component() for reference.
+	 */
+	for (i = 0; i < name->len; i++) {
+		if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+			name->buf[i] = '-';
+	}
+
+	/* remove consecutive dashes, leading or trailing dashes */
+	for (i = 0; i < name->len; i++) {
+		while (name->buf[i] == '-' &&
+		       (i == 0 ||
+			i == name->len - 1 ||
+			(i < name->len - 1 && name->buf[i + 1] == '-')))
+			strbuf_remove(name, i, 1);
+	}
+
+	/* last resort, should never ever happen in practice */
+	if (name->len == 0)
+		strbuf_addstr(name, "worktree");
+
+	if (check_refname_format(name->buf, REFNAME_ALLOW_ONELEVEL))
+		BUG("worktree name '%s' is not a valid refname", name->buf);
+}
+
 static int add_worktree(const char *path, const char *refname,
 			const struct add_opts *opts)
 {
@@ -275,6 +315,7 @@ static int add_worktree(const char *path, const char *refname,
 	struct strbuf symref = STRBUF_INIT;
 	struct commit *commit = NULL;
 	int is_branch = 0;
+	struct strbuf sb_name = STRBUF_INIT;
 
 	validate_worktree_add(path, opts);
 
@@ -290,7 +331,10 @@ static int add_worktree(const char *path, const char *refname,
 		die(_("invalid reference: %s"), refname);
 
 	name = worktree_basename(path, &len);
-	git_path_buf(&sb_repo, "worktrees/%.*s", (int)(path + len - name), name);
+	strbuf_add(&sb_name, name, path + len - name);
+	sanitize_worktree_name(&sb_name);
+	name = sb_name.buf;
+	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
 	if (safe_create_leading_directories_const(sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
@@ -415,6 +459,7 @@ static int add_worktree(const char *path, const char *refname,
 	strbuf_release(&symref);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
+	strbuf_release(&sb_name);
 	return ret;
 }
 
t/t2025-worktree-add.sh +5 -0
index 286bba35d8..0d465adb54 100755
@@ -570,4 +570,9 @@ test_expect_success '"add" an existing locked but missing worktree' '
 	git worktree add --force --force --detach gnoo
 '
 
+test_expect_success 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock" &&
+	test -d .git/worktrees/weird
+'
+
 test_done
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#6 lore raw

Re: [PATCH] worktree add: sanitize worktree names

Quoted 53 lines · On Чт, Feb 21, 2019 at 2:00 PM, =?UTF-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41j?= Duy…

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 47 
++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  5 +++++
 2 files changed, 51 insertions(+), 1 deletion(-)
builtin/worktree.c +25 -0
index 3f9907fcc9..ff36838a33 100644
@@ -262,6 +262,46 @@ static void validate_worktree_add(const char 
 *path, const struct add_opts *opts)
 	free_worktrees(worktrees);
 }
 
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to make it
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+	int i;
+
+	/* no ending with .lock */
+	if (ends_with(name->buf, ".lock"))
+		strbuf_remove(name, name->len - strlen(".lock"),
+			      strlen(".lock"));
+
+	/*
+	 * All special chars replaced with dashes. See
+	 * check_refname_component() for reference.
+	 */
+	for (i = 0; i < name->len; i++) {
+		if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+			name->buf[i] = '-';
+	}
+
+	/* remove consecutive dashes, leading or trailing dashes */
+	for (i = 0; i < name->len; i++) {

builtin/worktree.c static void validate_worktree_add(const char · line 300

Quoted 10 lines
builtin/worktree.c +10 -0
+		while (name->buf[i] == '-' &&
+		       (i == 0 ||
+			i == name->len - 1 ||
+			(i < name->len - 1 && name->buf[i + 1] == '-')))
+			strbuf_remove(name, i, 1);
+	}
+
+	/* last resort, should never ever happen in practice */
+	if (name->len == 0)
+		strbuf_addstr(name, "worktree");

I assume this means a user have passed a zero-sized worktree name? But zero-sized file/directory names are not possible anyway, would it make sense to just return an error in this case?

Quoted 58 lines
builtin/worktree.c +5 -0
+
+	if (check_refname_format(name->buf, REFNAME_ALLOW_ONELEVEL))
+		BUG("worktree name '%s' is not a valid refname", name->buf);
+}
+
 static int add_worktree(const char *path, const char *refname,
 			const struct add_opts *opts)

{

builtin/worktree.c +5 -1
@@ -275,6 +315,7 @@ static int add_worktree(const char *path, const 
 char *refname,
 	struct strbuf symref = STRBUF_INIT;
 	struct commit *commit = NULL;
 	int is_branch = 0;
+	struct strbuf sb_name = STRBUF_INIT;
 
 	validate_worktree_add(path, opts);
@@ -290,7 +331,10 @@ static int add_worktree(const char *path, const 
 char *refname,
 		die(_("invalid reference: %s"), refname);
 
 	name = worktree_basename(path, &len);
-	git_path_buf(&sb_repo, "worktrees/%.*s", (int)(path + len - name), 
 name);
+	strbuf_add(&sb_name, name, path + len - name);
+	sanitize_worktree_name(&sb_name);
+	name = sb_name.buf;
+	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
builtin/worktree.c +1 -0
 	if (safe_create_leading_directories_const(sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
@@ -415,6 +459,7 @@ static int add_worktree(const char *path, const 
 char *refname,
 	strbuf_release(&symref);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
+	strbuf_release(&sb_name);
 	return ret;
 }
t/t2025-worktree-add.sh +0 -0
index 286bba35d8..0d465adb54 100755
@@ -570,4 +570,9 @@ test_expect_success '"add" an existing locked but 

missing worktree' '

t/t2025-worktree-add.sh +5 -1
 	git worktree add --force --force --detach gnoo
 '
 
+test_expect_success 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock" &&
+	test -d .git/worktrees/weird
+'
+
 test_done
--

2.21.0.rc1.337.gdf7f8d0522

#7 lore raw

Re: [PATCH] worktree add: sanitize worktree names

Quoted 64 lines · Konstantin Kharlamov
Quoted 59 lines · On Чт, Feb 21, 2019 at 2:00 PM, =?UTF-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41j?= Duy…

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 47
++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  5 +++++
 2 files changed, 51 insertions(+), 1 deletion(-)
builtin/worktree.c +31 -0
index 3f9907fcc9..ff36838a33 100644
@@ -262,6 +262,46 @@ static void validate_worktree_add(const char
 *path, const struct add_opts *opts)
      free_worktrees(worktrees);
 }
 
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to make it
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+     int i;
+
+     /* no ending with .lock */
+     if (ends_with(name->buf, ".lock"))
+             strbuf_remove(name, name->len - strlen(".lock"),
+                           strlen(".lock"));
+
+     /*
+      * All special chars replaced with dashes. See
+      * check_refname_component() for reference.
+      */
+     for (i = 0; i < name->len; i++) {
+             if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+                     name->buf[i] = '-';
+     }
+
+     /* remove consecutive dashes, leading or trailing dashes */
+     for (i = 0; i < name->len; i++) {
+             while (name->buf[i] == '-' &&
+                    (i == 0 ||
+                     i == name->len - 1 ||
+                     (i < name->len - 1 && name->buf[i + 1] == '-')))
+                     strbuf_remove(name, i, 1);
+     }
Quoted 8 lines

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 299

Quoted 4 lines
builtin/worktree.c +4 -0
+
+     /* last resort, should never ever happen in practice */
+     if (name->len == 0)
+             strbuf_addstr(name, "worktree");

I assume this means a user have passed a zero-sized worktree name? But zero-sized file/directory names are not possible anyway, would it make sense to just return an error in this case?

It could happen if you do "git worktree add .lock". The ".lock" part will be stripped out, leaving us with an empty string.

Signature · 1 line
Duy
#8 lore raw

Re: [PATCH] worktree add: sanitize worktree names

Quoted 77 lines · Duy Nguyen
Quoted 76 lines · Konstantin Kharlamov
Quoted 1 line · On Чт, Feb 21, 2019 at 2:00 PM, =?UTF-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41j?= Duy…

Worktree names are based on $(basename $GIT_WORK_TREE). They

aren't

Quoted 1 line

significant until 3a3b9d8cde (refs: new ref types to make

per-worktree

Quoted 1 line

refs visible to all worktrees - 2018-10-21), where worktree name

could

Quoted 5 lines

be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user

will

Quoted 16 lines

be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 47
++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  5 +++++
 2 files changed, 51 insertions(+), 1 deletion(-)
builtin/worktree.c +0 -0
index 3f9907fcc9..ff36838a33 100644

builtin/worktree.c static void validate_worktree_add(const char · line 268

Quoted 7 lines
builtin/worktree.c +3 -0
@@ -262,6 +262,46 @@ static void validate_worktree_add(const char
 *path, const struct add_opts *opts)
      free_worktrees(worktrees);
 }
 
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to 

make it

Quoted 16 lines
builtin/worktree.c +16 -0
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+     int i;
+
+     /* no ending with .lock */
+     if (ends_with(name->buf, ".lock"))
+             strbuf_remove(name, name->len - strlen(".lock"),
+                           strlen(".lock"));
+
+     /*
+      * All special chars replaced with dashes. See
+      * check_refname_component() for reference.
+      */
+     for (i = 0; i < name->len; i++) {

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts)

Quoted 10 lines
builtin/worktree.c +10 -0
+             if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+                     name->buf[i] = '-';
+     }
+
+     /* remove consecutive dashes, leading or trailing dashes */
+     for (i = 0; i < name->len; i++) {
+             while (name->buf[i] == '-' &&
+                    (i == 0 ||
+                     i == name->len - 1 ||
+                     (i < name->len - 1 && name->buf[i + 1] == 

'-')))

Quoted 6 lines
builtin/worktree.c +6 -0
+                     strbuf_remove(name, i, 1);
+     }
+
+     /* last resort, should never ever happen in practice */
+     if (name->len == 0)
+             strbuf_addstr(name, "worktree");
Quoted 8 lines
Quoted 5 lines

I assume this means a user have passed a zero-sized worktree name?
But
zero-sized file/directory names are not possible anyway, would it
make
sense to just return an error in this case?

It could happen if you do "git worktree add .lock". The ".lock" part will be stripped out, leaving us with an empty string.

Ah, I see. Then, would it maybe make sense to just sanitize the ".lock" out the same way as you did with special symbols, i.e. with dashes?

(I am not a git developer, so not sure if that's a good question, but I would also question why ".lock" needs to be deleted. I guess git uses the postfix internally, but why can't it be okay with "name.lock.lock")

#9 lore raw

Re: [PATCH] worktree add: sanitize worktree names

Quoted 11 lines · Konstantin Kharlamov
Quoted 11 lines
Quoted 10 lines

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 299

Quoted 4 lines
builtin/worktree.c +4 -0
+
+     /* last resort, should never ever happen in practice */
+     if (name->len == 0)
+             strbuf_addstr(name, "worktree");

I assume this means a user have passed a zero-sized worktree name?
But
zero-sized file/directory names are not possible anyway, would it
make
sense to just return an error in this case?

Quoted 5 lines
Quoted 2 lines

It could happen if you do "git worktree add .lock". The ".lock" part will be stripped out, leaving us with an empty string.

Ah, I see. Then, would it maybe make sense to just sanitize the ".lock" out the same way as you did with special symbols, i.e. with dashes?

Hmm.. I actually did not think of that. Then we could return the error if "name" is empty.

Quoted 2 lines

(I am not a git developer, so not sure if that's a good question, but I would also question why ".lock" needs to be deleted. I guess git uses

It's because "foo.lock" is usually a temporary file to prepare things before we do an atomic update to "foo". And the "refs guys" were just being careful when they designed reference names so they reject any reference names that end with .lock. You can try to create a branch named something.lock, it will not go through. This is actually documented in "git help check-ref-format".

Quoted 1 line

the postfix internally, but why can't it be okay with "name.lock.lock")

Uh oh I miss this case. I only delete ".lock" once, "name.lock" would still be rejected. Thanks for noticing.

Signature · 1 line
Duy
#10 lore raw

Re: [PATCH] worktree add: sanitize worktree names

Quoted 4 lines · Duy Nguyen
Quoted 1 line

the postfix internally, but why can't it be okay with "name.lock.lock")

Uh oh I miss this case. I only delete ".lock" once, "name.lock" would still be rejected. Thanks for noticing.

Another tricky case is "refs/heads/foo.lock/bar.lock", which would need both ".lock"s removed. I think your v2 handles this correctly, though (because it disallows "." entirely).

-Peff

#11 lore raw

v2 cover worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

v2 fixes bad ".lock" handling in v1. I keep the "name->len == 0" part though because I found another valid case that could end up there.

Nguyễn Thái Ngọc Duy (1):
worktree add: sanitize worktree names

 builtin/worktree.c      | 51 ++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  7 ++++++
 2 files changed, 57 insertions(+), 1 deletion(-)

Range-diff dựa trên v1:
1: 42a3144874 ! 1: d1b6e1c55b worktree add: sanitize worktree names

    @@ -13,7 +13,7 @@
         be able to specify the worktree name by themselves if they're not
         happy with this dumb character substitution.
     
    -    Reported-by: [email protected]
    +    Reported-by: Konstantin Kharlamov <[email protected]>
         Signed-off-by: Nguyễn Thái Ngọc Duy <[email protected]>
     
      diff --git a/builtin/worktree.c b/builtin/worktree.c
    @@ -30,16 +30,14 @@
     + */
     +static void sanitize_worktree_name(struct strbuf *name)
     +{
    ++	char *orig_name = xstrdup(name->buf);
     +	int i;
     +
    -+	/* no ending with .lock */
    -+	if (ends_with(name->buf, ".lock"))
    -+		strbuf_remove(name, name->len - strlen(".lock"),
    -+			      strlen(".lock"));
    -+
     +	/*
     +	 * All special chars replaced with dashes. See
     +	 * check_refname_component() for reference.
    ++	 * Note that .lock is also turned to -lock, removing its
    ++	 * special status.
     +	 */
     +	for (i = 0; i < name->len; i++) {
     +		if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
    @@ -55,12 +53,18 @@
     +			strbuf_remove(name, i, 1);
     +	}
     +
    -+	/* last resort, should never ever happen in practice */
    ++	/*
    ++	 * a worktree name of only special chars would be reduced to
    ++	 * an empty string
    ++	 */
     +	if (name->len == 0)
     +		strbuf_addstr(name, "worktree");
     +
     +	if (check_refname_format(name->buf, REFNAME_ALLOW_ONELEVEL))
    -+		BUG("worktree name '%s' is not a valid refname", name->buf);
    ++		BUG("worktree name '%s' (from '%s') is not a valid refname",
    ++		    name->buf, orig_name);
    ++
    ++	free(orig_name);
     +}
     +
      static int add_worktree(const char *path, const char *refname,
    @@ -103,8 +107,10 @@
      '
      
     +test_expect_success 'sanitize generated worktree name' '
    -+	git worktree add --detach ".  weird*..?.lock" &&
    -+	test -d .git/worktrees/weird
    ++	git worktree add --detach ".  weird*..?.lock.lock" &&
    ++	test -d .git/worktrees/weird-lock-lock &&
    ++	git worktree add --detach .... &&
    ++	test -d .git/worktrees/worktree
     +'
     +
      test_done
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#12 lore raw

PATCH v2 1/1 worktree add: sanitize worktree names

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 51 ++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  7 ++++++
 2 files changed, 57 insertions(+), 1 deletion(-)
builtin/worktree.c +50 -1
index 3f9907fcc9..53e41db229 100644
@@ -262,6 +262,50 @@ static void validate_worktree_add(const char *path, const struct add_opts *opts)
 	free_worktrees(worktrees);
 }
 
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to make it
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+	char *orig_name = xstrdup(name->buf);
+	int i;
+
+	/*
+	 * All special chars replaced with dashes. See
+	 * check_refname_component() for reference.
+	 * Note that .lock is also turned to -lock, removing its
+	 * special status.
+	 */
+	for (i = 0; i < name->len; i++) {
+		if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+			name->buf[i] = '-';
+	}
+
+	/* remove consecutive dashes, leading or trailing dashes */
+	for (i = 0; i < name->len; i++) {
+		while (name->buf[i] == '-' &&
+		       (i == 0 ||
+			i == name->len - 1 ||
+			(i < name->len - 1 && name->buf[i + 1] == '-')))
+			strbuf_remove(name, i, 1);
+	}
+
+	/*
+	 * a worktree name of only special chars would be reduced to
+	 * an empty string
+	 */
+	if (name->len == 0)
+		strbuf_addstr(name, "worktree");
+
+	if (check_refname_format(name->buf, REFNAME_ALLOW_ONELEVEL))
+		BUG("worktree name '%s' (from '%s') is not a valid refname",
+		    name->buf, orig_name);
+
+	free(orig_name);
+}
+
 static int add_worktree(const char *path, const char *refname,
 			const struct add_opts *opts)
 {
@@ -275,6 +319,7 @@ static int add_worktree(const char *path, const char *refname,
 	struct strbuf symref = STRBUF_INIT;
 	struct commit *commit = NULL;
 	int is_branch = 0;
+	struct strbuf sb_name = STRBUF_INIT;
 
 	validate_worktree_add(path, opts);
 
@@ -290,7 +335,10 @@ static int add_worktree(const char *path, const char *refname,
 		die(_("invalid reference: %s"), refname);
 
 	name = worktree_basename(path, &len);
-	git_path_buf(&sb_repo, "worktrees/%.*s", (int)(path + len - name), name);
+	strbuf_add(&sb_name, name, path + len - name);
+	sanitize_worktree_name(&sb_name);
+	name = sb_name.buf;
+	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
 	if (safe_create_leading_directories_const(sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
@@ -415,6 +463,7 @@ static int add_worktree(const char *path, const char *refname,
 	strbuf_release(&symref);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
+	strbuf_release(&sb_name);
 	return ret;
 }
 
t/t2025-worktree-add.sh +7 -0
index 286bba35d8..71aa6ab9c1 100755
@@ -570,4 +570,11 @@ test_expect_success '"add" an existing locked but missing worktree' '
 	git worktree add --force --force --detach gnoo
 '
 
+test_expect_success 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock.lock" &&
+	test -d .git/worktrees/weird-lock-lock &&
+	git worktree add --detach .... &&
+	test -d .git/worktrees/worktree
+'
+
 test_done
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#13 lore raw

Re: [PATCH v2 1/1] worktree add: sanitize worktree names

Quoted 10 lines · Nguyễn Thái Ngọc Duy
builtin/worktree.c +10 -0
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to make it
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+	char *orig_name = xstrdup(name->buf);
+	int i;
+

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 284

Quoted 10 lines
builtin/worktree.c +10 -0
+	/*
+	 * All special chars replaced with dashes. See
+	 * check_refname_component() for reference.
+	 * Note that .lock is also turned to -lock, removing its
+	 * special status.
+	 */
+	for (i = 0; i < name->len; i++) {
+		if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+			name->buf[i] = '-';
+	}

This is reject-known-bad, but I think there are still some other characters that are not allowed in refnames (e.g., ASCII control characters). Which would lead to us hitting the BUG() below.

It might make sense to provide access to refname_disposition() and use it here. Alternatively, I think if we did an allow-known-good, it might be OK to have a slightly more restrictive scheme (say, alnum plus dashes, plus high-bit chars).

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 293

Quoted 8 lines
builtin/worktree.c +8 -0
+	/* remove consecutive dashes, leading or trailing dashes */
+	for (i = 0; i < name->len; i++) {
+		while (name->buf[i] == '-' &&
+		       (i == 0 ||
+			i == name->len - 1 ||
+			(i < name->len - 1 && name->buf[i + 1] == '-')))
+			strbuf_remove(name, i, 1);
+	}

I think this is correct, though it is possibly to be quadratic in the string length due to the O(n) remove. I think this kind of sanitizing is more readable if done between two strings rather than in-place, like:

  for (i = 0; i < name->len; i++) {
	if (is_allowed(name->buf[i])) {
		strbuf_addch(&dest, name->buf[i]);
		last_was_dash = 0;
	} else if (!last_was_dash && dest->len)
		strbuf_addch(&dest, '-');
		last_was_dash = 1;
	}

}
/* still must handle removal from end of stray "-" and ".lock" */ strbuf_swap(name, &dest);
strbuf_release(&dest);

but that may just be personal preference. I'm OK with it if you prefer the in-place way.

-Peff

#14 lore raw

Re: [PATCH v2 1/1] worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy
Quoted 38 lines · Nguyễn Thái Ngọc Duy

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 51 ++++++++++++++++++++++++++++++++++++++++-
 t/t2025-worktree-add.sh |  7 ++++++
 2 files changed, 57 insertions(+), 1 deletion(-)
builtin/worktree.c +12 -0
index 3f9907fcc9..53e41db229 100644
@@ -262,6 +262,50 @@ static void validate_worktree_add(const char *path, const struct add_opts *opts)
 	free_worktrees(worktrees);
 }
 
+/*
+ * worktree name is part of refname and has to pass
+ * check_refname_component(). Remove unallowed characters to make it
+ * valid.
+ */
+static void sanitize_worktree_name(struct strbuf *name)
+{
+	char *orig_name = xstrdup(name->buf);
+	int i;
+
+	/*
+	 * All special chars replaced with dashes. See

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 286

Quoted 10 lines
builtin/worktree.c +10 -0
+	 * check_refname_component() for reference.
+	 * Note that .lock is also turned to -lock, removing its
+	 * special status.
+	 */
+	for (i = 0; i < name->len; i++) {
+		if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+			name->buf[i] = '-';
+	}
+
+	/* remove consecutive dashes, leading or trailing dashes */

Why? So, '[fred]' will be 'sanitized' to 'fred' (rather than '-fred-'), which would increase the chance of a 'collision' with the 'fred' worktree (not very likely, but still). Is that useful? How about 'x86_64-*-gnu' which now becomes 'x86_64-gnu'?

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 300

Quoted 13 lines
builtin/worktree.c +13 -0
+	for (i = 0; i < name->len; i++) {
+		while (name->buf[i] == '-' &&
+		       (i == 0 ||
+			i == name->len - 1 ||
+			(i < name->len - 1 && name->buf[i + 1] == '-')))
+			strbuf_remove(name, i, 1);
+	}
+
+	/*
+	 * a worktree name of only special chars would be reduced to
+	 * an empty string
+	 */> +	if (name->len == 0)
+		strbuf_addstr(name, "worktree");

If you didn't 'collapse' the name above, you could check for an empty name at the top and wouldn't need this (presumably an empty name would not be valid).

ATB,
Ramsay Jones

#15 lore raw

Re: [PATCH v2 1/1] worktree add: sanitize worktree names

Quoted 14 lines · Ramsay Jones
Quoted 14 lines
builtin/worktree.c +14 -0
+static void sanitize_worktree_name(struct strbuf *name)
+{
+     char *orig_name = xstrdup(name->buf);
+     int i;
+
+     /*
+      * All special chars replaced with dashes. See
+      * check_refname_component() for reference.
+      * Note that .lock is also turned to -lock, removing its
+      * special status.
+      */
+     for (i = 0; i < name->len; i++) {
+             if (strchr(":?[]\\~ \t@{}*/.", name->buf[i]))
+                     name->buf[i] = '-';
Quoted 8 lines

builtin/worktree.c static void validate_worktree_add(const char *path, const struct add_opts *opts) · line 286

Quoted 3 lines
builtin/worktree.c +3 -0
+     }
+
+     /* remove consecutive dashes, leading or trailing dashes */

Why? So, '[fred]' will be 'sanitized' to 'fred' (rather than '-fred-'), which would increase the chance of a 'collision' with the 'fred' worktree (not very likely, but still). Is that useful? How about 'x86_64-*-gnu' which now becomes 'x86_64-gnu'?

It is useful when you want to specify HEAD of [fred] for example. Writing worktrees/fred/HEAD is a bit better than worktrees/-fred-/HEAD. I haven't done it yet, but these names will be shown in "git worktree list" too and lots of dashes does not improve readability. Collision is not a problem because if fred is taken, the final name would be fred1 or fred<some other number>.

If you're really bothered with this, you will be able to specify the name you want (you can't, yet). You still have to pass the valid refname check, but you have a lot more flexibility.

So this code only needs to work mostly ok for the common case and I could go either way, clean up consecutive dashes or not. I suppose simpler code would be the tie breaker.

Signature · 1 line
Duy
#16 lore raw

v3 cover worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

v3 is rewritten to use refname_disposition[] to cover all invalid chars.

Nguyễn Thái Ngọc Duy (1):
worktree add: sanitize worktree names

 builtin/worktree.c      | 37 ++++++++++++++++++++++++++++++++++++-
 refs.c                  |  6 ++++++
 refs.h                  |  1 +
 t/t2025-worktree-add.sh |  7 +++++++
 4 files changed, 50 insertions(+), 1 deletion(-)
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#17 lore raw

PATCH v3 1/1 worktree add: sanitize worktree names

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      | 37 ++++++++++++++++++++++++++++++++++++-
 refs.c                  |  6 ++++++
 refs.h                  |  1 +
 t/t2025-worktree-add.sh |  7 +++++++
 4 files changed, 50 insertions(+), 1 deletion(-)
builtin/worktree.c +36 -1
index 3f9907fcc9..21469eb52c 100644
@@ -262,6 +262,36 @@ static void validate_worktree_add(const char *path, const struct add_opts *opts)
 	free_worktrees(worktrees);
 }
 
+static void sanitize_worktree_name(struct strbuf *name)
+{
+	struct strbuf sb = STRBUF_INIT;
+	int i;
+
+	for (i = 0; i < name->len; i++) {
+		int ch = name->buf[i];
+
+		if (char_allowed_in_refname(ch))
+			strbuf_addch(&sb, ch);
+		else if (sb.len > 0 && sb.buf[sb.len - 1] != '-')
+			strbuf_addch(&sb, '-');
+	}
+	if (sb.len > 0 && sb.buf[sb.len - 1] == '-')
+		strbuf_setlen(&sb, sb.len - 1);
+	/*
+	 * a worktree name of only special chars would be reduced to
+	 * an empty string
+	 */
+	if (sb.len == 0)
+		strbuf_addstr(&sb, "worktree");
+
+	if (check_refname_format(sb.buf, REFNAME_ALLOW_ONELEVEL))
+		BUG("worktree name '%s' (from '%s') is not a valid refname",
+		    sb.buf, name->buf);
+
+	strbuf_swap(&sb, name);
+	strbuf_release(&sb);
+}
+
 static int add_worktree(const char *path, const char *refname,
 			const struct add_opts *opts)
 {
@@ -275,6 +305,7 @@ static int add_worktree(const char *path, const char *refname,
 	struct strbuf symref = STRBUF_INIT;
 	struct commit *commit = NULL;
 	int is_branch = 0;
+	struct strbuf sb_name = STRBUF_INIT;
 
 	validate_worktree_add(path, opts);
 
@@ -290,7 +321,10 @@ static int add_worktree(const char *path, const char *refname,
 		die(_("invalid reference: %s"), refname);
 
 	name = worktree_basename(path, &len);
-	git_path_buf(&sb_repo, "worktrees/%.*s", (int)(path + len - name), name);
+	strbuf_add(&sb_name, name, path + len - name);
+	sanitize_worktree_name(&sb_name);
+	name = sb_name.buf;
+	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
 	if (safe_create_leading_directories_const(sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
@@ -415,6 +449,7 @@ static int add_worktree(const char *path, const char *refname,
 	strbuf_release(&symref);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
+	strbuf_release(&sb_name);
 	return ret;
 }
 
refs.c +6 -0
index 142888a40a..f23f583db1 100644
@@ -57,6 +57,12 @@ static unsigned char refname_disposition[256] = {
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4
 };
 
+int char_allowed_in_refname(int ch)
+{
+	return 0 <= ch && ch < ARRAY_SIZE(refname_disposition) &&
+		refname_disposition[ch] == 0;
+}
+
 /*
  * Try to read one refname component from the front of refname.
  * Return the length of the component found, or -1 if the component is
refs.h +1 -0
index 308fa1f03b..61b4073f76 100644
@@ -459,6 +459,7 @@ int for_each_reflog(each_ref_fn fn, void *cb_data);
  * repeated slashes are accepted.
  */
 int check_refname_format(const char *refname, int flags);
+int char_allowed_in_refname(int ch);
 
 const char *prettify_refname(const char *refname);
 
t/t2025-worktree-add.sh +7 -0
index 286bba35d8..ea22207361 100755
@@ -570,4 +570,11 @@ test_expect_success '"add" an existing locked but missing worktree' '
 	git worktree add --force --force --detach gnoo
 '
 
+test_expect_success 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock.lock." &&
+	test -d .git/worktrees/weird-lock-lock &&
+	git worktree add --detach .... &&
+	test -d .git/worktrees/worktree
+'
+
 test_done
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#18 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

Quoted 10 lines · Nguyễn Thái Ngọc Duy

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

So notably this gets around ".." and ".lock" by just disallowing "." entirely. I think I'm OK with that for worktrees. It does make me a little nervous to see this new public function, though:

refs.c static unsigned char refname_disposition[256] = { · line 64

Quoted 5 lines
refs.c +5 -0
+int char_allowed_in_refname(int ch)
+{
+	return 0 <= ch && ch < ARRAY_SIZE(refname_disposition) &&
+		refname_disposition[ch] == 0;
+}

because it's not entirely accurate, as you noted above. I wonder if we could name this differently to warn people that the refname rules are not so simple.

If we just cared about saying "is this worktree name valid", I'd suggest actually constructing a sample refname with the worktree name embedded in it and feeding that to check_refname_format(). But because you want to actually sanitize, I don't think there's an easy way to reuse it.

So this approach is probably the best we can do, though I do still think it's worth renaming that function (and/or putting a big warning comment in front of it).

Other than that, I didn't see anything objectionable in the patch.

-Peff

#19 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

Quoted 17 lines · Jeff King
Quoted 5 lines · Nguyễn Thái Ngọc Duy

Update 'worktree add' code to remove special characters to follow these rules. The code could replace chars with '-' more than necessary, but it keeps the code simple. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

So notably this gets around ".." and ".lock" by just disallowing "." entirely. I think I'm OK with that for worktrees. It does make me a little nervous to see this new public function, though:

Quoted 1 line

+int char_allowed_in_refname(int ch) [...]

because it's not entirely accurate, as you noted above. I wonder if we could name this differently to warn people that the refname rules are not so simple.

Quoted 8 lines

If we just cared about saying "is this worktree name valid", I'd suggest actually constructing a sample refname with the worktree name embedded in it and feeding that to check_refname_format(). But because you want to actually sanitize, I don't think there's an easy way to reuse it.

So this approach is probably the best we can do, though I do still think it's worth renaming that function (and/or putting a big warning comment in front of it).

The above arguments seem to suggest the introduction of a companion to check_refname_format() for sanitizing, perhaps named sanitize_refname_format(), in ref.[hc]. The potential difficulty with that is defining exactly what "sanitize" means. Will it be contextual? (That is, will git-worktree have differently sanitation needs than some other facility?) If so, perhaps a 'flags' argument could control how sanitization is done.

#20 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

Quoted 9 lines · Eric Sunshine
Quoted 8 lines

If we just cared about saying "is this worktree name valid", I'd suggest actually constructing a sample refname with the worktree name embedded in it and feeding that to check_refname_format(). But because you want to actually sanitize, I don't think there's an easy way to reuse it.

So this approach is probably the best we can do, though I do still think it's worth renaming that function (and/or putting a big warning comment in front of it).

Quoted 7 lines

The above arguments seem to suggest the introduction of a companion to check_refname_format() for sanitizing, perhaps named sanitize_refname_format(), in ref.[hc]. The potential difficulty with that is defining exactly what "sanitize" means. Will it be contextual? (That is, will git-worktree have differently sanitation needs than some other facility?) If so, perhaps a 'flags' argument could control how sanitization is done.

I agree that sanitize_refname_format() would be nice, but I'm pretty sure it's going to end up having to duplicate many of the rules from check_refname_format(). Which is ugly if the two ever get out of sync.

But if we could write it in a way that keeps the actual policy logic in one factored-out portion, I think it would be worth doing.

-Peff

#21 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

Quoted 6 lines · Jeff King

I agree that sanitize_refname_format() would be nice, but I'm pretty sure it's going to end up having to duplicate many of the rules from check_refname_format(). Which is ugly if the two ever get out of sync.

But if we could write it in a way that keeps the actual policy logic in one factored-out portion, I think it would be worth doing.

Yeah, I do too.

In the meantime, let's call v3 sufficient improvement from the current state for now and queue it.

#22 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

In reply to Jeff King
Quoted 20 lines · Jeff King
Quoted 9 lines · Eric Sunshine
Quoted 8 lines

If we just cared about saying "is this worktree name valid", I'd suggest actually constructing a sample refname with the worktree name embedded in it and feeding that to check_refname_format(). But because you want to actually sanitize, I don't think there's an easy way to reuse it.

So this approach is probably the best we can do, though I do still think it's worth renaming that function (and/or putting a big warning comment in front of it).

Quoted 7 lines

The above arguments seem to suggest the introduction of a companion to check_refname_format() for sanitizing, perhaps named sanitize_refname_format(), in ref.[hc]. The potential difficulty with that is defining exactly what "sanitize" means. Will it be contextual? (That is, will git-worktree have differently sanitation needs than some other facility?) If so, perhaps a 'flags' argument could control how sanitization is done.

Quoted 6 lines

I agree that sanitize_refname_format() would be nice, but I'm pretty sure it's going to end up having to duplicate many of the rules from check_refname_format(). Which is ugly if the two ever get out of sync.

But if we could write it in a way that keeps the actual policy logic in one factored-out portion, I think it would be worth doing.

I think we could make check_refname_format() returns the bad position and several different error codes depending on context. Then sanitize_.. can just repeatedly call check_refname_format and fix up whatever error it reports. Performance goes straight to hell but I don't think that's a big deal for git-worktree, and it keeps check_refname_format() simple (relatively speaking).

The second option is make check_refname_format() call some callback instead of returning error. This allows sanitize_ to fix up in one go (inside the callback), but check_refname_format could be a lot uglier, and verifying all refs (I think pack-refs does this?) could also be slowed down.

Signature · 1 line
Duy
#23 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

Quoted 28 lines · Duy Nguyen
Quoted 20 lines · Jeff King
Quoted 9 lines · Eric Sunshine
Quoted 8 lines

If we just cared about saying "is this worktree name valid", I'd suggest actually constructing a sample refname with the worktree name embedded in it and feeding that to check_refname_format(). But because you want to actually sanitize, I don't think there's an easy way to reuse it.

So this approach is probably the best we can do, though I do still think it's worth renaming that function (and/or putting a big warning comment in front of it).

Quoted 7 lines

The above arguments seem to suggest the introduction of a companion to check_refname_format() for sanitizing, perhaps named sanitize_refname_format(), in ref.[hc]. The potential difficulty with that is defining exactly what "sanitize" means. Will it be contextual? (That is, will git-worktree have differently sanitation needs than some other facility?) If so, perhaps a 'flags' argument could control how sanitization is done.

Quoted 6 lines

I agree that sanitize_refname_format() would be nice, but I'm pretty sure it's going to end up having to duplicate many of the rules from check_refname_format(). Which is ugly if the two ever get out of sync.

But if we could write it in a way that keeps the actual policy logic in one factored-out portion, I think it would be worth doing.

Quoted 6 lines

I think we could make check_refname_format() returns the bad position and several different error codes depending on context. Then sanitize_.. can just repeatedly call check_refname_format and fix up whatever error it reports. Performance goes straight to hell but I don't think that's a big deal for git-worktree, and it keeps check_refname_format() simple (relatively speaking).

The new refs.c code would look something like this. do_check_refname_component() does not look so bad.

-- 8< --

builtin/worktree.c +1 -31
index 21469eb52c..ca63dd3df6 100644
@@ -262,36 +262,6 @@ static void validate_worktree_add(const char *path, const struct add_opts *opts)
 	free_worktrees(worktrees);
 }
 
-static void sanitize_worktree_name(struct strbuf *name)
-{
-	struct strbuf sb = STRBUF_INIT;
-	int i;
-
-	for (i = 0; i < name->len; i++) {
-		int ch = name->buf[i];
-
-		if (char_allowed_in_refname(ch))
-			strbuf_addch(&sb, ch);
-		else if (sb.len > 0 && sb.buf[sb.len - 1] != '-')
-			strbuf_addch(&sb, '-');
-	}
-	if (sb.len > 0 && sb.buf[sb.len - 1] == '-')
-		strbuf_setlen(&sb, sb.len - 1);
-	/*
-	 * a worktree name of only special chars would be reduced to
-	 * an empty string
-	 */
-	if (sb.len == 0)
-		strbuf_addstr(&sb, "worktree");
-
-	if (check_refname_format(sb.buf, REFNAME_ALLOW_ONELEVEL))
-		BUG("worktree name '%s' (from '%s') is not a valid refname",
-		    sb.buf, name->buf);
-
-	strbuf_swap(&sb, name);
-	strbuf_release(&sb);
-}
-
 static int add_worktree(const char *path, const char *refname,
 			const struct add_opts *opts)
 {
@@ -322,7 +292,7 @@ static int add_worktree(const char *path, const char *refname,
 
 	name = worktree_basename(path, &len);
 	strbuf_add(&sb_name, name, path + len - name);
-	sanitize_worktree_name(&sb_name);
+	sanitize_worktree_refname(&sb_name);
 	name = sb_name.buf;
 	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
refs.c +85 -14

135 lines not on this page · raw message

refs.h +1 -1
index 61b4073f76..3b65b8d27a 100644
@@ -459,7 +459,7 @@ int for_each_reflog(each_ref_fn fn, void *cb_data);
  * repeated slashes are accepted.
  */
 int check_refname_format(const char *refname, int flags);
-int char_allowed_in_refname(int ch);
+void sanitize_worktree_refname(struct strbuf *name);
 
 const char *prettify_refname(const char *refname);
 
t/t2025-worktree-add.sh +3 -3
index ea22207361..24c574f365 100755
@@ -571,10 +571,10 @@ test_expect_success '"add" an existing locked but missing worktree' '
 '
 
 test_expect_success 'sanitize generated worktree name' '
-	git worktree add --detach ".  weird*..?.lock.lock." &&
-	test -d .git/worktrees/weird-lock-lock &&
+	git worktree add --detach ".  weird*..?.lock.lock" &&
+	test -d .git/worktrees/---weird-.--.lock-lock &&
 	git worktree add --detach .... &&
-	test -d .git/worktrees/worktree
+	test -d .git/worktrees/--.-
 '
 
 test_done

-- 8< --

--
Duy

#24 lore raw

Re: [PATCH v3 1/1] worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

Hi Duy,

Quoted 6 lines · Nguyễn Thái Ngọc Duy
t/t2025-worktree-add.sh +0 -0
index 286bba35d8..ea22207361 100755
@@ -570,4 +570,11 @@ test_expect_success '"add" an existing locked but missing worktree' '
 	git worktree add --force --force --detach gnoo

t/t2025-worktree-add.sh test_expect_success '"add" an existing locked but missing worktree' ' · line 580

Quoted 10 lines
t/t2025-worktree-add.sh +7 -0
 '
 
+test_expect_success 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock.lock." &&
+	test -d .git/worktrees/weird-lock-lock &&
+	git worktree add --detach .... &&
+	test -d .git/worktrees/worktree
+'
+
 test_done

You probably missed that this added test fails on Windows:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=3782&view=logs

The reason is that you use a "funny name" which cannot be represented on every filesystem.

Please use the FUNNYNAMES prerequisite (or introduce another one, if you are uncomfortable with using a test whether tabs are valid in filenames to indiciate whether wildcard characters are valid in filenames).

Ciao,
Johannes

#25 lore raw

v4 cover worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

v4 refactors check_refname_component() so that we could do more accurate substitution (and leave fewer traps).

Performance of sanitize_worktree_refname() goes back to horrible again. But since it's not really a big deal (no body is going to add 200 worktrees per second), I don't feel like we should optimize it. That may involve removing the for loop in do_check_refname_component() and making things uglier.

The test is also updated to have FUNNYNAMES prerequisite, which is always unset on Windows. This should fix the breakage there.

Nguyễn Thái Ngọc Duy (2):
refs.c: refactor check_refname_component()
worktree add: sanitize worktree names

 builtin/worktree.c      |   7 ++-
 refs.c                  | 114 ++++++++++++++++++++++++++++++++++------
 refs.h                  |   1 +
 t/t2025-worktree-add.sh |   5 ++
 4 files changed, 110 insertions(+), 17 deletions(-)
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#26 lore raw

PATCH v4 1/2 refs.c: refactor check_refname_component()

The core logic of this function is factored out to provide more information when the refname is invalid: what part that is and exact what is wrong with it. This will be useful for a coming function that has to turn a string into a valid refname component.

---
 refs.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 62 insertions(+), 16 deletions(-)
refs.c +62 -16
index 142888a40a..70c55ea1b6 100644
@@ -57,10 +57,21 @@ static unsigned char refname_disposition[256] = {
 	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 4, 4
 };
 
+enum refname_check_code {
+	 refname_ok = 0,
+	 refname_contains_dotdot,
+	 refname_contains_atopen,
+	 refname_has_badchar,
+	 refname_contains_wildcard,
+	 refname_starts_with_dot,
+	 refname_ends_with_dotlock,
+	 refname_component_has_zero_length
+};
+
 /*
  * Try to read one refname component from the front of refname.
- * Return the length of the component found, or -1 if the component is
- * not legal.  It is legal if it is something reasonable to have under
+ * If the component is legal, return the end of the component in cp_out.
+ * It is legal if it is something reasonable to have under
  * ".git/refs/"; We do not like it if:
  *
  * - any path component of it begins with ".", or
@@ -71,11 +82,15 @@ static unsigned char refname_disposition[256] = {
  * - it ends with a "/", or
  * - it ends with ".lock", or
  * - it contains a "@{" portion
+ *
+ * in which case cp_out points to the beginning of the illegal part.
  */
-static int check_refname_component(const char *refname, int *flags)
+static enum refname_check_code do_check_refname_component(
+	const char *refname, int *flags, const char **cp_out)
 {
 	const char *cp;
 	char last = '\0';
+	enum refname_check_code ret = refname_ok;
 
 	for (cp = refname; ; cp++) {
 		int ch = *cp & 255;
@@ -84,18 +99,28 @@ static int check_refname_component(const char *refname, int *flags)
 		case 1:
 			goto out;
 		case 2:
-			if (last == '.')
-				return -1; /* Refname contains "..". */
+			if (last == '.') {
+				cp--;
+				ret = refname_contains_dotdot;
+				goto done;
+			}
 			break;
 		case 3:
-			if (last == '@')
-				return -1; /* Refname contains "@{". */
+			if (last == '@') {
+				cp--;
+				ret = refname_contains_atopen; /* @{ */
+				goto done;
+			}
 			break;
 		case 4:
-			return -1;
+			ret = refname_has_badchar;
+			goto done;
 		case 5:
-			if (!(*flags & REFNAME_REFSPEC_PATTERN))
-				return -1; /* refspec can't be a pattern */
+			if (!(*flags & REFNAME_REFSPEC_PATTERN)) {
+				/* refspec can't be a pattern */
+				ret = refname_contains_wildcard;
+				goto done;
+			}
 
 			/*
 			 * Unset the pattern flag so that we only accept
@@ -107,13 +132,34 @@ static int check_refname_component(const char *refname, int *flags)
 		last = ch;
 	}
 out:
-	if (cp == refname)
-		return 0; /* Component has zero length. */
-	if (refname[0] == '.')
-		return -1; /* Component starts with '.'. */
+	if (cp == refname) {
+		ret = refname_component_has_zero_length;
+		goto done;
+	}
+	if (refname[0] == '.') {
+		cp = refname;
+		ret = refname_starts_with_dot;
+		goto done;
+	}
 	if (cp - refname >= LOCK_SUFFIX_LEN &&
-	    !memcmp(cp - LOCK_SUFFIX_LEN, LOCK_SUFFIX, LOCK_SUFFIX_LEN))
-		return -1; /* Refname ends with ".lock". */
+	    !memcmp(cp - LOCK_SUFFIX_LEN, LOCK_SUFFIX, LOCK_SUFFIX_LEN)) {
+		cp -= LOCK_SUFFIX_LEN;
+		ret = refname_ends_with_dotlock;
+	}
+done:
+	*cp_out = cp;
+	return ret;
+}
+
+/* Return the length of the component if it's legal otherwise -1 */
+static int check_refname_component(const char *refname, int *flags)
+{
+	const char *cp;
+	enum refname_check_code ret;
+
+	ret = do_check_refname_component(refname, flags, &cp);
+	if (ret)
+		return -1;
 	return cp - refname;
 }
 
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#27 lore raw

Re: [PATCH v4 1/2] refs.c: refactor check_refname_component()

Review static unsigned char refname_disposition[256] = { · line 89

Quoted 10 lines · Nguyễn Thái Ngọc Duy
(quoted diff) +4 -1
@@ -71,11 +82,15 @@ static unsigned char refname_disposition[256] = {
  * - it ends with a "/", or
  * - it ends with ".lock", or
  * - it contains a "@{" portion
+ *
+ * in which case cp_out points to the beginning of the illegal part.
  */
-static int check_refname_component(const char *refname, int *flags)
+static enum refname_check_code do_check_refname_component(
+	const char *refname, int *flags, const char **cp_out)

Hmm, OK, so we get to know what type of problem, but also the exact character where we found it. And then we just keep mutating that char until we have something that passes.

I can't think of any reason that wouldn't work. As you note, it's possibly quadratic, though that might be OK for our purposes.

I had envisioned just sanitizing each character into an output buffer as we did the checks. It does introduce some complexities, though, because now the checking function is doing the replacement (so it has to know the right sanitizing rule for each case).

The patch below is a rough cut at that, just for discussion. You can ignore the check-ref-format bits; they were just to make poking at it easier, though perhaps we'd want something like that in the long run.

I suspect check_refname_component() could be made a bit more readable by reordering a few bits. E.g., why do we check for a leading "." at the _end_, after having parsed the entire rest of the component for errors?

I dunno. I think I can live with what you've got in your series, but I figured I'd share this for the sake of completeness. If you really love it, feel free to adapt it.

builtin/check-ref-format.c +10 -0
index bc67d3f0a8..41b5434be2 100644
@@ -56,6 +56,7 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 	int i;
 	int normalize = 0;
 	int flags = 0;
+	int sanitize = 0;
 	const char *refname;
 
 	if (argc == 2 && !strcmp(argv[1], "-h"))
@@ -73,13 +74,22 @@ int cmd_check_ref_format(int argc, const char **argv, const char *prefix)
 			flags &= ~REFNAME_ALLOW_ONELEVEL;
 		else if (!strcmp(argv[i], "--refspec-pattern"))
 			flags |= REFNAME_REFSPEC_PATTERN;
+		else if (!strcmp(argv[i], "--sanitize"))
+			sanitize = 1;
 		else
 			usage(builtin_check_ref_format_usage);
 	}
 	if (! (i == argc - 1))
 		usage(builtin_check_ref_format_usage);
 
 	refname = argv[i];
+	if (sanitize) {
+		struct strbuf out = STRBUF_INIT;
+		sanitize_refname(refname, &out);
+		printf("%s\n", out.buf);
+		strbuf_release(&out);
+		return 0;
+	}
 	if (normalize)
 		refname = collapse_slashes(refname);
 	if (check_refname_format(refname, flags))
refs.c +85 -19

154 lines not on this page · raw message

refs.h +6 -0
index 308fa1f03b..b99c309dd9 100644
@@ -460,6 +460,12 @@ int for_each_reflog(each_ref_fn fn, void *cb_data);
  */
 int check_refname_format(const char *refname, int flags);
 
+/*
+ * Apply the rules from check_refname_format, but mutate the result until it
+ * is acceptable, and place the result in "out".
+ */
+void sanitize_refname(const char *refname, struct strbuf *out);
+
 const char *prettify_refname(const char *refname);
 
 char *shorten_unambiguous_ref(const char *refname, int strict);
#28 lore raw

Re: [PATCH v4 1/2] refs.c: refactor check_refname_component()

Quoted 9 lines · Jeff King

On Tue, Mar 05, 2019 at 07:08:33PM +0700, Nguyễn Thái Ngọc Duy wrote:
I had envisioned just sanitizing each character into an output buffer as we did the checks. It does introduce some complexities, though, because now the checking function is doing the replacement (so it has to know the right sanitizing rule for each case).

The patch below is a rough cut at that, just for discussion. You can ignore the check-ref-format bits; they were just to make poking at it easier, though perhaps we'd want something like that in the long run.

This is more along the lines of what I had envisioned, as well, after looking over the implementation of check_refname_component(). It's a bit noisy and loud but easy to follow, and doesn't give rise to concerns about quadratic behavior, etc.

#29 lore raw

PATCH v4 2/2 worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      |  7 ++++++-
 refs.c                  | 36 ++++++++++++++++++++++++++++++++++++
 refs.h                  |  1 +
 t/t2025-worktree-add.sh |  5 +++++
 4 files changed, 48 insertions(+), 1 deletion(-)
builtin/worktree.c +6 -1
index 3f9907fcc9..ca63dd3df6 100644
@@ -275,6 +275,7 @@ static int add_worktree(const char *path, const char *refname,
 	struct strbuf symref = STRBUF_INIT;
 	struct commit *commit = NULL;
 	int is_branch = 0;
+	struct strbuf sb_name = STRBUF_INIT;
 
 	validate_worktree_add(path, opts);
 
@@ -290,7 +291,10 @@ static int add_worktree(const char *path, const char *refname,
 		die(_("invalid reference: %s"), refname);
 
 	name = worktree_basename(path, &len);
-	git_path_buf(&sb_repo, "worktrees/%.*s", (int)(path + len - name), name);
+	strbuf_add(&sb_name, name, path + len - name);
+	sanitize_worktree_refname(&sb_name);
+	name = sb_name.buf;
+	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
 	if (safe_create_leading_directories_const(sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
@@ -415,6 +419,7 @@ static int add_worktree(const char *path, const char *refname,
 	strbuf_release(&symref);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
+	strbuf_release(&sb_name);
 	return ret;
 }
 
refs.c +36 -0
index 70c55ea1b6..a23a84e431 100644
@@ -163,6 +163,42 @@ static int check_refname_component(const char *refname, int *flags)
 	return cp - refname;
 }
 
+void sanitize_worktree_refname(struct strbuf *name)
+{
+	int flags = 0, i, max_tries;
+	const char *cp;
+	enum refname_check_code ret;
+
+	/*
+	 * name->len should be enough because we should never need to
+	 * substitute any position more than once, but let's just add
+	 * a couple more to be on the safe side.
+	 */
+	max_tries = name->len + 10;
+	for (i = 0; i < max_tries; i++) {
+		ret = do_check_refname_component(name->buf, &flags, &cp);
+		switch (ret) {
+		case refname_ok:
+			strbuf_setlen(name, cp - name->buf);
+			return;
+
+		case refname_component_has_zero_length:
+			strbuf_addstr(name, "worktree");
+			return;
+
+		case refname_contains_dotdot:
+		case refname_contains_atopen:
+		case refname_has_badchar:
+		case refname_contains_wildcard:
+		case refname_ends_with_dotlock:
+		case refname_starts_with_dot:
+			*(char *)cp = '-';
+			break;
+		}
+	}
+	BUG("stuck in infinite loop! buf = %s", name->buf);
+}
+
 int check_refname_format(const char *refname, int flags)
 {
 	int component_len, component_count = 0;
refs.h +1 -0
index 308fa1f03b..3b65b8d27a 100644
@@ -459,6 +459,7 @@ int for_each_reflog(each_ref_fn fn, void *cb_data);
  * repeated slashes are accepted.
  */
 int check_refname_format(const char *refname, int flags);
+void sanitize_worktree_refname(struct strbuf *name);
 
 const char *prettify_refname(const char *refname);
 
t/t2025-worktree-add.sh +5 -0
index 286bba35d8..6e2b90c84f 100755
@@ -570,4 +570,9 @@ test_expect_success '"add" an existing locked but missing worktree' '
 	git worktree add --force --force --detach gnoo
 '
 
+test_expect_success FUNNYNAMES 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock.lock" &&
+	test -d .git/worktrees/---weird--.-.lock-lock
+'
+
 test_done
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#30 lore raw

v5 cover worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

v5 is basically Jeff's version from one of the replies in v4, where check_refname_component is enhanced to optionally sanitize.

I was reluctant to go this way because it makes check_refname_component more complex (turns out still manageable) and burns worktree rules in it. But there may never be the second sanitization user, we deal with it when it comes.

As said, refs.c is pretty much Jeff's except two major changes:

- handle foo.lock.lock correctly by stripping .lock repeatedly

- sanitize refname _components_ instead of full refs. I could construct worktrees/<name> and pass to Jeff's sanitize_refname. But then I need to strip worktrees/ after that.

I took credits so that bugs come to me first (then I'll blame him anyway while doing some evil laughs)

Nguyễn Thái Ngọc Duy (1):
worktree add: sanitize worktree names

 builtin/worktree.c      |  10 +++-
 refs.c                  | 103 ++++++++++++++++++++++++++++++++--------
 refs.h                  |   6 +++
 t/t2400-worktree-add.sh |   5 ++
 4 files changed, 104 insertions(+), 20 deletions(-)
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#31 lore raw

PATCH v5 1/1 worktree add: sanitize worktree names

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---
 builtin/worktree.c      |  10 +++-
 refs.c                  | 103 ++++++++++++++++++++++++++++++++--------
 refs.h                  |   6 +++
 t/t2400-worktree-add.sh |   5 ++
 4 files changed, 104 insertions(+), 20 deletions(-)
builtin/worktree.c +9 -1
index 6cc094a453..756cf3a417 100644
@@ -275,6 +275,7 @@ static int add_worktree(const char *path, const char *refname,
 	struct strbuf symref = STRBUF_INIT;
 	struct commit *commit = NULL;
 	int is_branch = 0;
+	struct strbuf sb_name = STRBUF_INIT;
 
 	validate_worktree_add(path, opts);
 
@@ -290,7 +291,13 @@ static int add_worktree(const char *path, const char *refname,
 		die(_("invalid reference: %s"), refname);
 
 	name = worktree_basename(path, &len);
-	git_path_buf(&sb_repo, "worktrees/%.*s", (int)(path + len - name), name);
+	strbuf_add(&sb, name, path + len - name);
+	sanitize_refname_component(sb.buf, &sb_name);
+	if (!sb_name.len)
+		BUG("How come '%s' becomes empty after sanitization?", sb.buf);
+	strbuf_reset(&sb);
+	name = sb_name.buf;
+	git_path_buf(&sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
 	if (safe_create_leading_directories_const(sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
@@ -416,6 +423,7 @@ static int add_worktree(const char *path, const char *refname,
 	strbuf_release(&symref);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
+	strbuf_release(&sb_name);
 	return ret;
 }
 
refs.c +84 -19

153 lines not on this page · raw message

refs.h +6 -0
index 308fa1f03b..4d8c5465c3 100644
@@ -460,6 +460,12 @@ int for_each_reflog(each_ref_fn fn, void *cb_data);
  */
 int check_refname_format(const char *refname, int flags);
 
+/*
+ * Apply the rules from check_refname_format, but mutate the result until it
+ * is acceptable, and place the result in "out".
+ */
+void sanitize_refname_component(const char *refname, struct strbuf *out);
+
 const char *prettify_refname(const char *refname);
 
 char *shorten_unambiguous_ref(const char *refname, int strict);
t/t2400-worktree-add.sh +5 -0
index 286bba35d8..c989dbe321 100755
@@ -570,4 +570,9 @@ test_expect_success '"add" an existing locked but missing worktree' '
 	git worktree add --force --force --detach gnoo
 '
 
+test_expect_success FUNNYNAMES 'sanitize generated worktree name' '
+	git worktree add --detach ".  weird*..?.lock.lock" &&
+	test -d .git/worktrees/---weird-.-
+'
+
 test_done
Signature · 1 line
2.21.0.rc1.337.gdf7f8d0522
#32 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 19 lines · Nguyễn Thái Ngọc Duy

Worktree names are based on $(basename $GIT_WORK_TREE). They aren't significant until 3a3b9d8cde (refs: new ref types to make per-worktree refs visible to all worktrees - 2018-10-21), where worktree name could be part of a refname and must follow refname rules.

Update 'worktree add' code to remove special characters to follow these rules. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

---

refs.c +2 -0
@@ -72,30 +72,57 @@ static unsigned char refname_disposition[256] = {
+static int check_refname_component(const char *refname, int *flags,
+                                  struct strbuf *sanitized)
 {
        for (cp = refname; ; cp++) {
                unsigned char disp = refname_disposition[ch];

refs.c static unsigned char refname_disposition[256] = { · line 86

Quoted 10 lines
refs.c +6 -0
+               if (sanitized && disp != 1)
+                       strbuf_addch(sanitized, ch);
+
                switch (disp) {
                case 1:
                        goto out;
                case 2:
+                       if (last == '.') { /* Refname contains "..". */
+                               if (sanitized)
+                                       sanitized->len--; /* collapse ".." to single "." */

I think this needs to be:

    strbuf_setlen(sanitized, sanitized->len - 1);

to ensure that NUL-terminator ends up in the correct place if this "." is the very last character in 'refname'. (Otherwise, the NUL will remain after the second ".", thus ".." won't be collapsed to "." at all.)

Quoted 4 lines
refs.c +3 -0
+                               else
+                                       return -1;
+                       }
                        break;
#33 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 5 lines · Eric Sunshine

refs.c static unsigned char refname_disposition[256] = { · line 98

Quoted 4 lines
                case 2:
refs.c +3 -0
+                       if (last == '.') { /* Refname contains "..". */
+                               if (sanitized)
+                                       sanitized->len--; /* collapse ".." to single "." */
Quoted 8 lines

I think this needs to be:

    strbuf_setlen(sanitized, sanitized->len - 1);

to ensure that NUL-terminator ends up in the correct place if this "." is the very last character in 'refname'. (Otherwise, the NUL will remain after the second ".", thus ".." won't be collapsed to "." at all.)

True. Why doesn't it do the similar "replace with -" it does for other unfortunate characters, though?

#34 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Eric Sunshine

refs.c static unsigned char refname_disposition[256] = { · line 98

Quoted 4 lines
                case 2:
refs.c +3 -0
+                       if (last == '.') { /* Refname contains "..". */
+                               if (sanitized)
+                                       sanitized->len--; /* collapse ".." to single "." */

I think this needs to be:

    strbuf_setlen(sanitized, sanitized->len - 1);
Quoted 8 lines
Quoted 4 lines

to ensure that NUL-terminator ends up in the correct place if this "." is the very last character in 'refname'. (Otherwise, the NUL will remain after the second ".", thus ".." won't be collapsed to "." at all.)

True. Why doesn't it do the similar "replace with -" it does for other unfortunate characters, though?

I think Jeff saw an opportunity to keep it cleaner ("." looks better than ".-") and took it.

Signature · 1 line
Duy
#35 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 9 lines · Duy Nguyen
Quoted 9 lines
Quoted 8 lines

I think this needs to be:

    strbuf_setlen(sanitized, sanitized->len - 1);

to ensure that NUL-terminator ends up in the correct place if this "." is the very last character in 'refname'. (Otherwise, the NUL will remain after the second ".", thus ".." won't be collapsed to "." at all.)

Quoted 5 lines
Quoted 2 lines

True. Why doesn't it do the similar "replace with -" it does for other unfortunate characters, though?

I think Jeff saw an opportunity to keep it cleaner ("." looks better than ".-") and took it.

Yeah, that was one thing I was going to comment on your patch. The "rules" I made up were pretty ad-hoc as I was walking through the function (note it also drops ".lock" instead of sanitizing it into "-lock").

But it may make sense to make things more consistent (even if the result isn't entirely reversible).

Another option _is_ to actually make it reversible. I.e., use "%2e" instead of ".", which would also necessitate replacing "%". I don't know if that has a huge value for this use-case, but it's a nice property that two sanitized names can't collide (unless they originally identical).

-Peff

#36 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 5 lines · Jeff King

Another option _is_ to actually make it reversible. I.e., use "%2e" instead of ".", which would also necessitate replacing "%". I don't know if that has a huge value for this use-case, but it's a nice property that two sanitized names can't collide (unless they originally identical).

Yeah. That is a good property to have.

#37 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy
Quoted 4 lines · Nguyễn Thái Ngọc Duy

Update 'worktree add' code to remove special characters to follow these rules. In the future the user will be able to specify the worktree name by themselves if they're not happy with this dumb character substitution.

This replaces both of the two patches in v4, and applies to a more recent tip of 'master' (post 7d0c1f4556).

refs.c static unsigned char refname_disposition[256] = { · line 74

Quoted 8 lines
refs.c +0 -0
index 142888a40a..e9f83018f0 100644
@@ -72,30 +72,57 @@ static unsigned char refname_disposition[256] = {
  * - it ends with ".lock", or
  * - it contains a "@{" portion
  */

The comment above needs modernizing (see attached at the end).

refs.c static unsigned char refname_disposition[256] = { · line 78

Quoted 6 lines
refs.c +3 -2
 		case 2:
-			if (last == '.')
-				return -1; /* Refname contains "..". */
+			if (last == '.') { /* Refname contains "..". */
+				if (sanitized)
+					sanitized->len--; /* collapse ".." to single "." */

As Eric points out, this needs to be fixed.

I'll use the strbuf_setlen() version suggested by Eric in the meantime, but "sanitized->buf[sanitized->len-1] = '-'" as done to everything else in the function may be a better idea, especially since they'll be able to name the worktree themselves in the future anyway.

refs.c static unsigned char refname_disposition[256] = { · line 82

Quoted 4 lines
refs.c +4 -0
+
+	if (refname[0] == '.') { /* Component starts with '.'. */
+		if (sanitized)
+			sanitized->buf[component_start] = '-';

... and a dot turns into a dash in some cases anyway.

refs.c static unsigned char refname_disposition[256] = { · line 93

Quoted 13 lines
refs.c +10 -2
+		else
+			return -1;
+	}
 	if (cp - refname >= LOCK_SUFFIX_LEN &&
-	    !memcmp(cp - LOCK_SUFFIX_LEN, LOCK_SUFFIX, LOCK_SUFFIX_LEN))
-		return -1; /* Refname ends with ".lock". */
+	    !memcmp(cp - LOCK_SUFFIX_LEN, LOCK_SUFFIX, LOCK_SUFFIX_LEN)) {
+		if (!sanitized)
+			return -1;
+		/* Refname ends with ".lock". */
+		while (strbuf_strip_suffix(sanitized, LOCK_SUFFIX)) {
+			/* try again in case we have .lock.lock */
+		}

No need for {}; just have an empty statment

		while (...) 
			; /* try again ... */

This "strip all .lock repeatedly" made me stop and think a bit; this will never make the component empty, as the only way for this loop to make it empty is if we have a string that match "^\(.lock)\*$" in it, but the first dot would have already been turned into a dash, so we'll end up with "-lock", which is not empty.

refs.c static unsigned char refname_disposition[256] = { · line 96

Quoted 3 lines
refs.c +1 -0
+	}
 	return cp - refname;
 }

See below for a possible further polishment.

* The first hunk is not about this patch but a long-standing issue after the comment was given to this function for a single level (I do not know or care how it happened--perhaps we had a single function that verifies multiple levels which later was split into a caller that loops and this function that checks a single level, and the comment for the multi-level function was left stale).

* check_refname_component() can now try to sanitize; document it.

* The last hunk is from Eric's comment.

 refs.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
refs.c +7 -2
index e9f83018f0..3a1b2a8c3c 100644
@@ -63,7 +63,7 @@ static unsigned char refname_disposition[256] = {
  * not legal.  It is legal if it is something reasonable to have under
  * ".git/refs/"; We do not like it if:
  *
- * - any path component of it begins with ".", or
+ * - it begins with ".", or
  * - it has double dots "..", or
  * - it has ASCII control characters, or
  * - it has ":", "?", "[", "\", "^", "~", SP, or TAB anywhere, or
@@ -71,6 +71,10 @@ static unsigned char refname_disposition[256] = {
  * - it ends with a "/", or
  * - it ends with ".lock", or
  * - it contains a "@{" portion
+ *
+ * When sanitized is not NULL, instead of rejecting the input refname
+ * as an error, try to come up with a usable replacement for the input
+ * refname in it.
  */
 static int check_refname_component(const char *refname, int *flags,
 				   struct strbuf *sanitized)
@@ -95,7 +99,8 @@ static int check_refname_component(const char *refname, int *flags,
 		case 2:
 			if (last == '.') { /* Refname contains "..". */
 				if (sanitized)
-					sanitized->len--; /* collapse ".." to single "." */
+					/* collapse ".." to single "." */
+					strbuf_setlen(sanitized, sanitized->len - 1);
 				else
 					return -1;
 			}
#38 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 8 lines · Junio C Hamano
                while (...)
                        ; /* try again ... */

This "strip all .lock repeatedly" made me stop and think a bit; this will never make the component empty, as the only way for this loop to make it empty is if we have a string that match "^\(.lock)\*$" in it, but the first dot would have already been turned into a dash, so we'll end up with "-lock", which is not empty.

Yep. I added a BUG() check anyway in worktree.c just in case something slips through in the future.

Quoted 13 lines
Quoted 3 lines

+ }

      return cp - refname;

}

See below for a possible further polishment.

* The first hunk is not about this patch but a long-standing issue after the comment was given to this function for a single level (I do not know or care how it happened--perhaps we had a single function that verifies multiple levels which later was split into a caller that loops and this function that checks a single level, and the comment for the multi-level function was left stale).

Quoted 3 lines

* check_refname_component() can now try to sanitize; document it.

* The last hunk is from Eric's comment.

Thanks.

Signature · 1 line
Duy
#39 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

In reply to Nguyễn Thái Ngọc Duy

Hi Duy,

Quoted 5 lines · Nguyễn Thái Ngọc Duy
refs.c +0 -0
index 142888a40a..e9f83018f0 100644

refs.c static unsigned char refname_disposition[256] = { · line 80

Quoted 10 lines
refs.c +3 -1
@@ -72,30 +72,57 @@ static unsigned char refname_disposition[256] = {
  * - it ends with ".lock", or
  * - it contains a "@{" portion
  */
-static int check_refname_component(const char *refname, int *flags)
+static int check_refname_component(const char *refname, int *flags,
+				   struct strbuf *sanitized)
 {
 	const char *cp;
 	char last = '\0';
+	size_t component_start;

This variable is uninitialized. It is then...

refs.c static unsigned char refname_disposition[256] = { · line 83

Quoted 3 lines
refs.c +3 -0
+
+	if (sanitized)
+		component_start = sanitized->len;

... initialized only when sanitized is not NULL, and subsequently...

Quoted 53 lines
refs.c +29 -7
 	for (cp = refname; ; cp++) {
 		int ch = *cp & 255;
 		unsigned char disp = refname_disposition[ch];
+
+		if (sanitized && disp != 1)
+			strbuf_addch(sanitized, ch);
+
 		switch (disp) {
 		case 1:
 			goto out;
 		case 2:
-			if (last == '.')
-				return -1; /* Refname contains "..". */
+			if (last == '.') { /* Refname contains "..". */
+				if (sanitized)
+					sanitized->len--; /* collapse ".." to single "." */
+				else
+					return -1;
+			}
 			break;
 		case 3:
-			if (last == '@')
-				return -1; /* Refname contains "@{". */
+			if (last == '@') { /* Refname contains "@{". */
+				if (sanitized)
+					sanitized->buf[sanitized->len-1] = '-';
+				else
+					return -1;
+			}
 			break;
 		case 4:
-			return -1;
+			/* forbidden char */
+			if (sanitized)
+				sanitized->buf[sanitized->len-1] = '-';
+			else
+				return -1;
+			break;
 		case 5:
-			if (!(*flags & REFNAME_REFSPEC_PATTERN))
-				return -1; /* refspec can't be a pattern */
+			if (!(*flags & REFNAME_REFSPEC_PATTERN)) {
+				/* refspec can't be a pattern */
+				if (sanitized)
+					sanitized->buf[sanitized->len-1] = '-';
+				else
+					return -1;
+			}
 
 			/*
 			 * Unset the pattern flag so that we only accept

refs.c static int check_refname_component(const char *refname, int *flags) · line 142

Quoted 9 lines
refs.c +4 -2
@@ -109,26 +136,48 @@ static int check_refname_component(const char *refname, int *flags)
 out:
 	if (cp == refname)
 		return 0; /* Component has zero length. */
-	if (refname[0] == '.')
-		return -1; /* Component starts with '.'. */
+
+	if (refname[0] == '.') { /* Component starts with '.'. */
+		if (sanitized)
+			sanitized->buf[component_start] = '-';

... used a loooooooong time after that, also only if sanitized is not NULL.

Apparently for some GCC versions, this is too cute, and it complains that this variable might be used uninitialized:
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=4352&view=logs

And quite honestly, even for mere humans it is not all *that* clear that sanitized cannot be changed from NULL to non-NULL in the code in between, *in particular* because the changes extend over two hunks, the code between is not shown.

I would strongly advise against trying to be so cute, and just initialize the variable already. Over-optimization in such instances makes the code a lot harder to reason about.

Ciao,
Johannes

#40 lore raw

Re: [PATCH v5 1/1] worktree add: sanitize worktree names

Quoted 15 lines · Johannes Schindelin

refs.c static unsigned char refname_disposition[256] = { · line 80

Quoted 6 lines
refs.c +3 -0
+static int check_refname_component(const char *refname, int *flags,
+				   struct strbuf *sanitized)
 {
 	const char *cp;
 	char last = '\0';
+	size_t component_start;

This variable is uninitialized. It is then...

refs.c static unsigned char refname_disposition[256] = { · line 83

Quoted 3 lines
refs.c +3 -0
+
+	if (sanitized)
+		component_start = sanitized->len;

... initialized only when sanitized is not NULL, and subsequently...
...

Quoted 8 lines

refs.c static int check_refname_component(const char *refname, int *flags) · line 142

Quoted 3 lines
refs.c +3 -0
+	if (refname[0] == '.') { /* Component starts with '.'. */
+		if (sanitized)
+			sanitized->buf[component_start] = '-';

...
... used a loooooooong time after that, also only if sanitized is not NULL.

Apparently for some GCC versions, this is too cute, and it complains that

It does require humans (well, at least it did to this one) to be careful when reading the code to know that component_start is valid when it is used.

There unfortunately is no good "default" value to initialize the variable to. When checking a later component in a series of components, it would be looking at non-zero position, so even initializing it to 0 in this function is *not* a more sensible fallback default value than any other random garbage value (which would squelch the compiler, but it would mislead the humans nevertheless).

And that (i.e. the lack of any sensible default value when sanitized is NULL) is the reason why the variable is left uninitialized by the patch, I think. I do not think the code is trying to be cute at all.

I wonder if we make the caller pass a pointer to

	struct {
		struct strbuf result;
		size_t component_start;
	} sanitized;

	sanitized.component_start = sanitized.result.len
	check_refname_component(refname, flags, &sanitized);

and get rid of the assignment to component_start done by the callee, it would appease compilers and makes the code easier to vet. It does introduce one more ad-hoc type, which is a certain downside.

I dunno.