Make fsck check other worktree HEADs
cover Make fsck check other worktree HEADs
This patchset adds checking of other worktree HEADs to fsck.
The reason I've marked this RFC is that I'm worried my incidental reliance on "worktrees/$WORKTREE/HEAD" resolving as a ref (in patch 3) might raise some flags for others. In particular, in [1] Peff said that this refname resolves right now mostly by accident and will probably stop working in the future. However, I feel that since fsck checks the storage format as well as contents, it seems natural that a change of storage model would result in the fsck code changing and thus that I'm not locking in any particular ref format long term with these changes. But I want to flag this issue for discussion.
[1] https://public-inbox.org/git/[email protected]/
Elijah Newren (3):
fsck: Move fsck_head_link() to get_default_heads() to avoid some
globals
t1450-fsck: Add tests for HEAD of other worktrees
fsck: Check HEAD of other worktrees as well
builtin/fsck.c | 73 ++++++++++++++++++++++++++++++++++++++++++++------------- t/t1450-fsck.sh | 27 +++++++++++++++++++++ 2 files changed, 84 insertions(+), 16 deletions(-)
Signature · 1 line
2.16.1.75.gc01c8fdd7d
RFC PATCH 1/3 fsck: Move fsck_head_link() to get_default_heads() to avoid some globals
This will make it easier to check the HEAD of other worktrees from fsck.
--- builtin/fsck.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-)
builtin/fsck.c +16 -11
@@ -33,8 +33,6 @@ static int check_strict;static int keep_cache_objects;static struct fsck_options fsck_walk_options =FSCK_OPTIONS_DEFAULT ;static struct fsck_options fsck_obj_options =FSCK_OPTIONS_DEFAULT ; -static struct object_id head_oid; -static const char *head_points_at;static int errors_found;static int write_lost_and_found;static int verbose;
@@ -453,8 +451,15 @@ static int fsck_handle_ref(const char *refname, const struct object_id *oid,return 0 ; } +static int fsck_head_link (const char **head_points_at, +struct object_id *head_oid); +static void get_default_heads (void ) { +const char *head_points_at; +struct object_id head_oid; + +fsck_head_link (&head_points_at, &head_oid);if (head_points_at && !is_null_oid (&head_oid))fsck_handle_ref ("HEAD" , &head_oid,0 ,NULL );for_each_rawref (fsck_handle_ref,NULL );
@@ -548,33 +553,34 @@ static void fsck_object_dir(const char *path)stop_progress (&progress); } -static int fsck_head_link (void ) +static int fsck_head_link (const char **head_points_at, +struct object_id *head_oid) {int null_is_error =0 ;if (verbose)fprintf (stderr,"Checking HEAD link\n" ); - head_points_at =resolve_ref_unsafe ("HEAD" ,0 , &head_oid,NULL ); -if (!head_points_at) { + *head_points_at =resolve_ref_unsafe ("HEAD" ,0 , head_oid,NULL ); +if (!*head_points_at) { errors_found |=ERROR_REFS ;return error ("Invalid HEAD" ); } -if (!strcmp (head_points_at,"HEAD" )) +if (!strcmp (*head_points_at,"HEAD" ))/* detached HEAD */ null_is_error =1 ; -else if (!starts_with (head_points_at,"refs/heads/" )) { +else if (!starts_with (*head_points_at,"refs/heads/" )) { errors_found |=ERROR_REFS ;return error ("HEAD points to something strange (%s)" , - head_points_at); + *head_points_at); } -if (is_null_oid (&head_oid)) { +if (is_null_oid (head_oid)) {if (null_is_error) { errors_found |=ERROR_REFS ;return error ("HEAD: detached HEAD points at nothing" ); }fprintf (stderr,"notice: HEAD points to an unborn branch (%s)\n" , - head_points_at +11 ); + *head_points_at +11 ); }return 0 ; }
@@ -686,7 +692,6 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)git_config (fsck_config,NULL ); -fsck_head_link ();if (connectivity_only) {for_each_loose_object (mark_loose_for_connectivity,NULL ,0 );for_each_packed_object (mark_packed_for_connectivity,NULL ,0 );
Signature · 1 line
2.16.1.75.gc01c8fdd7d
RFC PATCH 2/3 t1450-fsck: Add tests for HEAD of other worktrees
In reply to Elijah Newren--- t/t1450-fsck.sh | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
t/t1450-fsck.sh +27 -0
@@ -102,6 +102,33 @@ test_expect_success 'HEAD link pointing at a funny place' ' grep "HEAD points to something strange" out ' +test_expect_failure 'other worktree HEAD link pointing at a funny object' ' + test_when_finished "rm -rf .git/worktrees" && + mkdir -p .git/worktrees/other && + echo 0000000000000000000000000000000000000000 >.git/worktrees/other/HEAD && + test_must_fail git fsck 2>out && + cat out && + grep "worktrees/other/HEAD: detached HEAD points" out +' + +test_expect_failure 'other worktree HEAD link pointing at missing object' ' + test_when_finished "rm -rf .git/worktrees" && + mkdir -p .git/worktrees/other && + echo "Contents missing from repo" | git hash-object --stdin >.git/worktrees/other/HEAD && + test_must_fail git fsck 2>out && + cat out && + grep "worktrees/other/HEAD: invalid sha1 pointer" out +' + +test_expect_failure 'other worktree HEAD link pointing at a funny place' ' + test_when_finished "rm -rf .git/worktrees" && + mkdir -p .git/worktrees/other && + echo "ref: refs/funny/place" >.git/worktrees/other/HEAD && + test_must_fail git fsck 2>out && + cat out && + grep "worktrees/other/HEAD points to something strange" out +' + test_expect_success 'email without @ is okay' ' git cat-file commit HEAD >basis && sed "s/@/AT/" basis >okay &&
Signature · 1 line
2.16.1.75.gc01c8fdd7d
RFC PATCH 3/3 fsck: Check HEAD of other worktrees as well
In reply to Elijah NewrenThis takes advantage of the fact that "worktrees/$WORKTREE/HEAD" will currently resolve as a ref, which may not be true in the future with different ref backends. I don't think it locks us in to supporting resolving other worktree HEADs with this syntax, as I view the user-visible error message as more of a pointer to a pathname that the user will need to fix. If the underlying ref storage changes, naturally both this code and the hint may need to change to match.
--- builtin/fsck.c | 60 +++++++++++++++++++++++++++++++++++++++++++++------------ t/t1450-fsck.sh | 6 +++--- 2 files changed, 51 insertions(+), 15 deletions(-)
builtin/fsck.c +48 -12
@@ -451,17 +451,51 @@ static int fsck_handle_ref(const char *refname, const struct object_id *oid,return 0 ; } -static int fsck_head_link (const char **head_points_at, +static void get_worktree_names (struct string_list *names) +{ +struct strbuf path =STRBUF_INIT ; +DIR *dir; +struct dirent *d; + +strbuf_addf (&path,"%s/worktrees" ,get_git_common_dir ()); + dir =opendir (path.buf); +strbuf_release (&path); +if (dir) { +while ((d =readdir (dir)) !=NULL ) { +if (!is_dot_or_dotdot (d->d_name)) +string_list_append (names, d->d_name); + } +closedir (dir); + } +} + +static int fsck_head_link (const char *head_ref_name, +const char **head_points_at,struct object_id *head_oid);static void get_default_heads (void ) {const char *head_points_at;struct object_id head_oid; +struct string_list worktree_names =STRING_LIST_INIT_DUP ; +struct string_list_item *worktree_item; +struct strbuf head_name =STRBUF_INIT ; -fsck_head_link (&head_points_at, &head_oid); +fsck_head_link ("HEAD" , &head_points_at, &head_oid);if (head_points_at && !is_null_oid (&head_oid))fsck_handle_ref ("HEAD" , &head_oid,0 ,NULL ); + +get_worktree_names (&worktree_names); +for_each_string_list_item (worktree_item, &worktree_names) { +strbuf_reset (&head_name); +strbuf_addf (&head_name,"worktrees/%s/HEAD" , + worktree_item->string); +fsck_head_link (head_name.buf, &head_points_at, &head_oid); +if (head_points_at && !is_null_oid (&head_oid)) +fsck_handle_ref (head_name.buf, &head_oid,0 ,NULL ); + } +strbuf_release (&head_name); +for_each_rawref (fsck_handle_ref,NULL );if (include_reflogs)for_each_reflog (fsck_handle_reflog,NULL );
@@ -553,34 +587,36 @@ static void fsck_object_dir(const char *path)stop_progress (&progress); } -static int fsck_head_link (const char **head_points_at, +static int fsck_head_link (const char *head_ref_name, +const char **head_points_at,struct object_id *head_oid) {int null_is_error =0 ;if (verbose) -fprintf (stderr,"Checking HEAD link\n" ); +fprintf (stderr,"Checking %s link\n" , head_ref_name); - *head_points_at =resolve_ref_unsafe ("HEAD" ,0 , head_oid,NULL ); + *head_points_at =resolve_ref_unsafe (head_ref_name,0 , head_oid,NULL );if (!*head_points_at) { errors_found |=ERROR_REFS ; -return error ("Invalid HEAD" ); +return error ("Invalid %s" , head_ref_name); } -if (!strcmp (*head_points_at,"HEAD" )) +if (!strcmp (*head_points_at, head_ref_name))/* detached HEAD */ null_is_error =1 ;else if (!starts_with (*head_points_at,"refs/heads/" )) { errors_found |=ERROR_REFS ; -return error ("HEAD points to something strange (%s)" , - *head_points_at); +return error ("%s points to something strange (%s)" , + head_ref_name, *head_points_at); }if (is_null_oid (head_oid)) {if (null_is_error) { errors_found |=ERROR_REFS ; -return error ("HEAD: detached HEAD points at nothing" ); +return error ("%s: detached HEAD points at nothing" , + head_ref_name); } -fprintf (stderr,"notice: HEAD points to an unborn branch (%s)\n" , - *head_points_at +11 ); +fprintf (stderr,"notice: %s points to an unborn branch (%s)\n" , + head_ref_name, *head_points_at +11 ); }return 0 ; }
t/t1450-fsck.sh +3 -3
@@ -102,7 +102,7 @@ test_expect_success 'HEAD link pointing at a funny place' ' grep "HEAD points to something strange" out ' -test_expect_failure 'other worktree HEAD link pointing at a funny object' ' +test_expect_success 'other worktree HEAD link pointing at a funny object' ' test_when_finished "rm -rf .git/worktrees" && mkdir -p .git/worktrees/other && echo 0000000000000000000000000000000000000000 >.git/worktrees/other/HEAD &&
@@ -111,7 +111,7 @@ test_expect_failure 'other worktree HEAD link pointing at a funny object' ' grep "worktrees/other/HEAD: detached HEAD points" out ' -test_expect_failure 'other worktree HEAD link pointing at missing object' ' +test_expect_success 'other worktree HEAD link pointing at missing object' ' test_when_finished "rm -rf .git/worktrees" && mkdir -p .git/worktrees/other && echo "Contents missing from repo" | git hash-object --stdin >.git/worktrees/other/HEAD &&
@@ -120,7 +120,7 @@ test_expect_failure 'other worktree HEAD link pointing at missing object' ' grep "worktrees/other/HEAD: invalid sha1 pointer" out ' -test_expect_failure 'other worktree HEAD link pointing at a funny place' ' +test_expect_success 'other worktree HEAD link pointing at a funny place' ' test_when_finished "rm -rf .git/worktrees" && mkdir -p .git/worktrees/other && echo "ref: refs/funny/place" >.git/worktrees/other/HEAD &&
Signature · 1 line
2.16.1.75.gc01c8fdd7d
Re: [RFC PATCH 3/3] fsck: Check HEAD of other worktrees as well
Quoted 7 lines · Elijah Newren
This takes advantage of the fact that "worktrees/$WORKTREE/HEAD" will currently resolve as a ref, which may not be true in the future with different ref backends. I don't think it locks us in to supporting resolving other worktree HEADs with this syntax, as I view the user-visible error message as more of a pointer to a pathname that the user will need to fix. If the underlying ref storage changes, naturally both this code and the hint may need to change to match.
I'm leaning more about something like this patch below (which does not even build, just to demonstrate). A couple points:
- Instead of doing the hacky refs worktrees/foo/HEAD, we get the correct ref store for each worktree
- We have an API for getting all (non-broken) worktrees. I realize for fsck, we may even want to examine semi-broken worktrees as well, but get_worktrees() can take a flag to accomplish that if needed.
- As you can see, I print %p from the ref store instead of something human friendly. This is something I was stuck at last time. I need better ref store description (or even the worktree name)
- This ref_name() function makes me think if we should have an extended sha1 syntax for accessing per-worktree refs from a different worktree, something like HEAD@{worktree:foo} to resolve to worktrees/foo/HEAD. Then we have a better description here that can actually be used from command line, as a regular ref, if needed.
-- 8< --
builtin/fsck.c +36 -11
@@ -16,6 +16,7 @@#include "streaming.h" #include "decorate.h" #include "packfile.h" +#include "worktree.h" #define REACHABLE 0x0001#define SEEN 0x0002
@@ -451,17 +452,39 @@ static int fsck_handle_ref(const char *refname, const struct object_id *oid,return 0 ; } -static int fsck_head_link (const char **head_points_at, +static const char *ref_name (struct ref_store *refs,const char *name) +{ +static struct strbuf sb =STRBUF_INIT ; + +if (!refs) +return name; +strbuf_reset (&sb); +strbuf_addf (&sb,"%s (from %p)" , name, refs); +return sb.buf; +} + +static int fsck_head_link (struct ref_store *refs, +const char **head_points_at,struct object_id *head_oid);static void get_default_heads (void ) {const char *head_points_at;struct object_id head_oid; +struct worktree **worktrees, **wt; -fsck_head_link (&head_points_at, &head_oid); +fsck_head_link (NULL , &head_points_at, &head_oid);if (head_points_at && !is_null_oid (&head_oid))fsck_handle_ref ("HEAD" , &head_oid,0 ,NULL ); + + worktrees =get_worktrees (0 ); +for (wt = worktrees; *wt; wt++) { +fsck_head_link (get_worktree_ref_store (*wt), &head_points_at, &head_oid); +if (head_points_at && !is_null_oid (&head_oid)) +fsck_handle_ref (*wt,"HEAD" , &head_oid,0 ,NULL ); + } +free_worktrees (wt); +for_each_rawref (fsck_handle_ref,NULL );if (include_reflogs)for_each_reflog (fsck_handle_reflog,NULL );
@@ -553,34 +576,36 @@ static void fsck_object_dir(const char *path)stop_progress (&progress); } -static int fsck_head_link (const char **head_points_at, +static int fsck_head_link (struct ref_store *refs, +const char **head_points_at,struct object_id *head_oid) {int null_is_error =0 ;if (verbose) -fprintf (stderr,"Checking HEAD link\n" ); +fprintf (stderr,"Checking %s link\n" ,ref_name (refs,"HEAD" )); - *head_points_at =resolve_ref_unsafe ("HEAD" ,0 , head_oid,NULL ); + *head_points_at =refs_resolve_ref_unsafe (refs,"HEAD" ,0 , head_oid,NULL );if (!*head_points_at) { errors_found |=ERROR_REFS ; -return error ("Invalid HEAD" ); +return error ("Invalid HEAD from ref-store %p" , refs); }if (!strcmp (*head_points_at,"HEAD" ))/* detached HEAD */ null_is_error =1 ;else if (!starts_with (*head_points_at,"refs/heads/" )) { errors_found |=ERROR_REFS ; -return error ("HEAD points to something strange (%s)" , - *head_points_at); +return error ("%s points to something strange (%s)" , +ref_name (refs,"HEAD" ), *head_points_at); }if (is_null_oid (head_oid)) {if (null_is_error) { errors_found |=ERROR_REFS ; -return error ("HEAD: detached HEAD points at nothing" ); +return error ("%s: detached HEAD points at nothing" , +ref_name (refs,"HEAD" )); } -fprintf (stderr,"notice: HEAD points to an unborn branch (%s)\n" , - *head_points_at +11 ); +fprintf (stderr,"notice: %s points to an unborn branch (%s)\n" , +ref_name (refs,"HEAD" ), *head_points_at +11 ); }return 0 ; }
t/t1450-fsck.sh +3 -3
@@ -102,7 +102,7 @@ test_expect_success 'HEAD link pointing at a funny place' ' grep "HEAD points to something strange" out ' -test_expect_failure 'other worktree HEAD link pointing at a funny object' ' +test_expect_success 'other worktree HEAD link pointing at a funny object' ' test_when_finished "rm -rf .git/worktrees" && mkdir -p .git/worktrees/other && echo 0000000000000000000000000000000000000000 >.git/worktrees/other/HEAD &&
@@ -111,7 +111,7 @@ test_expect_failure 'other worktree HEAD link pointing at a funny object' ' grep "worktrees/other/HEAD: detached HEAD points" out ' -test_expect_failure 'other worktree HEAD link pointing at missing object' ' +test_expect_success 'other worktree HEAD link pointing at missing object' ' test_when_finished "rm -rf .git/worktrees" && mkdir -p .git/worktrees/other && echo "Contents missing from repo" | git hash-object --stdin >.git/worktrees/other/HEAD &&
@@ -120,7 +120,7 @@ test_expect_failure 'other worktree HEAD link pointing at missing object' ' grep "worktrees/other/HEAD: invalid sha1 pointer" out ' -test_expect_failure 'other worktree HEAD link pointing at a funny place' ' +test_expect_success 'other worktree HEAD link pointing at a funny place' ' test_when_finished "rm -rf .git/worktrees" && mkdir -p .git/worktrees/other && echo "ref: refs/funny/place" >.git/worktrees/other/HEAD &&
-- 8< --
Re: [RFC PATCH 3/3] fsck: Check HEAD of other worktrees as well
Quoted 12 lines · Duy Nguyen
Quoted 7 lines · Elijah Newren
This takes advantage of the fact that "worktrees/$WORKTREE/HEAD" will currently resolve as a ref, which may not be true in the future with different ref backends. I don't think it locks us in to supporting resolving other worktree HEADs with this syntax, as I view the user-visible error message as more of a pointer to a pathname that the user will need to fix. If the underlying ref storage changes, naturally both this code and the hint may need to change to match.
I'm leaning more about something like this patch below (which does not even build, just to demonstrate). A couple points:
Quoted 8 lines
- Instead of doing the hacky refs worktrees/foo/HEAD, we get the correct ref store for each worktree
- We have an API for getting all (non-broken) worktrees. I realize for fsck, we may even want to examine semi-broken worktrees as well, butget_worktrees()can take a flag to accomplish that if needed.
- As you can see, I print %p from the ref store instead of something human friendly. This is something I was stuck at last time. I need better ref store description (or even the worktree name)
Yeah, I think this is the right approach. We know about worktrees internally, and we should be able to iterate over their ref stores, even if we have no way to succinctly name the resulting ref.
Quoted 5 lines
- This
ref_name()function makes me think if we should have an extended sha1 syntax for accessing per-worktree refs from a different worktree, something like HEAD@{worktree:foo} to resolve to worktrees/foo/HEAD. Then we have a better description here that can actually be used from command line, as a regular ref, if needed.
Yeah, I agree this would be very useful. I peeked at how bad it would be to hanlde this. The parsing is pretty easy in get_oid_basic(), but you'd have to plumb through the ref_store in quite a few places:
- interpret_nth_prior_checkout() would probably want to use the worktree's HEAD (for "@{worktree:foo}@{-1}")
- dwim_ref() would need to know about the ref store
- that implies that substitute_branch_name() knows about it, too
So it may turn into a big job. But I think it's moving in the right direction. And it may not be the end of the world if all features do not work from day one (e.g., if HEAD@{worktree:foo} works, but HEAD@{worktree:foo}@{upstream} does not yet, with plans to eventually make that work).
-Peff
Re: [RFC PATCH 3/3] fsck: Check HEAD of other worktrees as well
In reply to Duy NguyenQuoted 9 lines · Duy Nguyen
Quoted 7 lines · Elijah Newren
This takes advantage of the fact that "worktrees/$WORKTREE/HEAD" will currently resolve as a ref, which may not be true in the future with different ref backends. I don't think it locks us in to supporting resolving other worktree HEADs with this syntax, as I view the user-visible error message as more of a pointer to a pathname that the user will need to fix. If the underlying ref storage changes, naturally both this code and the hint may need to change to match.
Quoted 8 lines
I'm leaning more about something like this patch below (which does not even build, just to demonstrate). A couple points:
- Instead of doing the hacky refs worktrees/foo/HEAD, we get the correct ref store for each worktree
- We have an API for getting all (non-broken) worktrees. I realize for fsck, we may even want to examine semi-broken worktrees as well, butget_worktrees()can take a flag to accomplish that if needed.
Very sensible.
When I saw that "fsck" thing, the first thing I wondered was "wait, how are we doing prune and repack while making sure objects reachable only from HEAD in other worktrees are not lost? we must already have an API that gives us where the refs of them are stored in".
Thanks for a quick response for course correction.